2024-03-04T07:15:15.052 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-04T07:15:15.052 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} --verbose --owner scheduled_leonidus@teuthology --name leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671 -- /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671/orig.config.yaml 2024-03-04T07:15:15.103 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T07:15:15.189 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671 branch: wip-lusov-qdb description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} email: null first_in_suite: false job_id: '7580671' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-lusov-qdb ceph: cephfs: max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: false mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - Replacing daemon mds sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd owner: scheduled_leonidus@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 5072 sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd sleep_before_teardown: 0 subset: 246/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBmgs9EG1bFVgO8kH7WX4asSSzQG1qyixcVqkKDDu/xE7ar5Bhw6/tXfttArGJLoicuMCFuzGZILnHVQy85oZMw= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpg8Cq3BIkk2zVUoyQhy1vZvBoVz0qwOK3jmJBEFlxWyAijHvNwPk1r1ggibX/1BPQ1/s/AHDBSaPaBhWHGLIk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - mds_thrash: null - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-03-04_07:05:03 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-04T07:15:15.189 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa; will attempt to use it 2024-03-04T07:15:15.190 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks 2024-03-04T07:15:15.190 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-04T07:15:15.191 INFO:teuthology.task.internal:Checking packages... 2024-03-04T07:15:15.213 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd' 2024-03-04T07:15:15.213 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T07:15:15.213 INFO:teuthology.packaging:ref: None 2024-03-04T07:15:15.214 INFO:teuthology.packaging:tag: None 2024-03-04T07:15:15.214 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-03-04T07:15:15.214 INFO:teuthology.packaging:sha1: 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T07:15:15.214 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-lusov-qdb 2024-03-04T07:15:15.436 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1739-g1bc250c7-1jammy 2024-03-04T07:15:15.438 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-04T07:15:15.444 INFO:teuthology.task.internal:no buildpackages task found 2024-03-04T07:15:15.444 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-04T07:15:15.478 INFO:teuthology.task.internal:Saving configuration 2024-03-04T07:15:15.496 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-04T07:15:15.502 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T07:15:15.527 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 07:06:12.917406', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN/EWj6S5RMIDi6dNOUgVQiVTIaoYyefqkTdsOcAahUH'} 2024-03-04T07:15:15.548 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 07:06:12.916013', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPhiz6UhwSwTPRlVx3RXF2QBxG5UkccTxNTe6y4LFTWu'} 2024-03-04T07:15:15.548 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-04T07:15:15.554 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-04T07:15:15.554 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-03-04T07:15:15.554 INFO:teuthology.run_tasks:Running task console_log... 2024-03-04T07:15:15.633 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f772e95cf70>, signals=[15]) 2024-03-04T07:15:15.634 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-04T07:15:15.640 INFO:teuthology.task.internal:Opening connections... 2024-03-04T07:15:15.640 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T07:15:15.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T07:15:15.719 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T07:15:15.720 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T07:15:15.800 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-04T07:15:15.807 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-03-04T07:15:15.812 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-03-04T07:15:15.813 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-03-04T07:15:15.862 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-04T07:15:15.862 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="Ubuntu" 2024-03-04T07:15:15.862 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="22.04" 2024-03-04T07:15:15.862 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-04T07:15:15.862 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_CODENAME=jammy 2024-03-04T07:15:15.862 INFO:teuthology.orchestra.run.smithi113.stdout:ID=ubuntu 2024-03-04T07:15:15.862 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE=debian 2024-03-04T07:15:15.862 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-04T07:15:15.862 INFO:teuthology.orchestra.run.smithi113.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-04T07:15:15.862 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-04T07:15:15.863 INFO:teuthology.orchestra.run.smithi113.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-04T07:15:15.863 INFO:teuthology.orchestra.run.smithi113.stdout:UBUNTU_CODENAME=jammy 2024-03-04T07:15:15.863 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-03-04T07:15:15.884 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-03-04T07:15:15.890 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-03-04T07:15:15.890 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-03-04T07:15:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-04T07:15:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu" 2024-03-04T07:15:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="22.04" 2024-03-04T07:15:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-04T07:15:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=jammy 2024-03-04T07:15:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu 2024-03-04T07:15:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian 2024-03-04T07:15:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-04T07:15:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-04T07:15:15.939 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-04T07:15:15.939 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-04T07:15:15.939 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=jammy 2024-03-04T07:15:15.939 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-03-04T07:15:15.959 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-04T07:15:15.969 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-04T07:15:15.987 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-04T07:15:15.987 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-03-04T07:15:15.989 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-03-04T07:15:15.993 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-04T07:15:16.003 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-04T07:15:16.003 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-03-04T07:15:16.036 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-03-04T07:15:16.058 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-04T07:15:16.115 INFO:teuthology.run_tasks:Running task kernel... 2024-03-04T07:15:16.129 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-04T07:15:16.129 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-03-04T07:15:16.129 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-03-04T07:15:16.130 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T07:15:16.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:15:16.135 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-03-04T07:15:16.493 DEBUG:teuthology.task.kernel:sha1 for client.0 is 786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T07:15:16.493 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "786939134d7eee4dc7a805245d3f595929af37d7"... 2024-03-04T07:15:16.494 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-03-04T07:15:16.499 INFO:teuthology.orchestra.run.smithi113.stdout:5.15.0-56-generic 2024-03-04T07:15:16.500 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T07:15:16.500 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-04T07:15:16.500 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-03-04T07:15:16.501 DEBUG:teuthology.orchestra.run.smithi113:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-03-04T07:15:16.560 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-03-04T07:15:16.703 INFO:teuthology.orchestra.run.smithi113.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-03-04T07:15:16.703 INFO:teuthology.orchestra.run.smithi113.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-03-04T07:15:16.703 INFO:teuthology.orchestra.run.smithi113.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-03-04T07:15:16.703 INFO:teuthology.orchestra.run.smithi113.stderr:hint: 2024-03-04T07:15:16.703 INFO:teuthology.orchestra.run.smithi113.stderr:hint: git config --global init.defaultBranch 2024-03-04T07:15:16.704 INFO:teuthology.orchestra.run.smithi113.stderr:hint: 2024-03-04T07:15:16.704 INFO:teuthology.orchestra.run.smithi113.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-03-04T07:15:16.704 INFO:teuthology.orchestra.run.smithi113.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-03-04T07:15:16.704 INFO:teuthology.orchestra.run.smithi113.stderr:hint: 2024-03-04T07:15:16.704 INFO:teuthology.orchestra.run.smithi113.stderr:hint: git branch -m 2024-03-04T07:15:16.705 INFO:teuthology.orchestra.run.smithi113.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-03-04T07:15:16.707 DEBUG:teuthology.orchestra.run.smithi113:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-03-04T07:15:16.779 DEBUG:teuthology.orchestra.run.smithi113:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-03-04T07:15:16.836 DEBUG:teuthology.orchestra.run.smithi113:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-03-04T07:19:09.367 INFO:teuthology.orchestra.run.smithi113.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-03-04T07:19:09.367 INFO:teuthology.orchestra.run.smithi113.stderr: * [new branch] main -> origin/main 2024-03-04T07:19:09.399 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20190312 -> 20190312 2024-03-04T07:19:09.399 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20190416 -> 20190416 2024-03-04T07:19:09.399 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20190514 -> 20190514 2024-03-04T07:19:09.400 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20190618 -> 20190618 2024-03-04T07:19:09.400 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20190717 -> 20190717 2024-03-04T07:19:09.400 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20190815 -> 20190815 2024-03-04T07:19:09.400 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20190923 -> 20190923 2024-03-04T07:19:09.402 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20191022 -> 20191022 2024-03-04T07:19:09.402 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20191215 -> 20191215 2024-03-04T07:19:09.402 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20200122 -> 20200122 2024-03-04T07:19:09.402 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20200316 -> 20200316 2024-03-04T07:19:09.402 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20200421 -> 20200421 2024-03-04T07:19:09.402 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20200519 -> 20200519 2024-03-04T07:19:09.402 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20200619 -> 20200619 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20200721 -> 20200721 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20200817 -> 20200817 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20200918 -> 20200918 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20201022 -> 20201022 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20201118 -> 20201118 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20201218 -> 20201218 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20210208 -> 20210208 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20210315 -> 20210315 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20210511 -> 20210511 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20210716 -> 20210716 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20210818 -> 20210818 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20210919 -> 20210919 2024-03-04T07:19:09.403 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20211027 -> 20211027 2024-03-04T07:19:09.406 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20211216 -> 20211216 2024-03-04T07:19:09.406 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20220209 -> 20220209 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20220310 -> 20220310 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20220411 -> 20220411 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20220509 -> 20220509 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20220610 -> 20220610 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20220708 -> 20220708 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20220815 -> 20220815 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20220913 -> 20220913 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20221012 -> 20221012 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20221109 -> 20221109 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20221214 -> 20221214 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20230117 -> 20230117 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20230210 -> 20230210 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20230310 -> 20230310 2024-03-04T07:19:09.407 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20230404 -> 20230404 2024-03-04T07:19:09.408 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20230515 -> 20230515 2024-03-04T07:19:09.408 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20230625 -> 20230625 2024-03-04T07:19:09.408 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20230804 -> 20230804 2024-03-04T07:19:09.408 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20230919 -> 20230919 2024-03-04T07:19:09.408 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20231030 -> 20231030 2024-03-04T07:19:09.408 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20231111 -> 20231111 2024-03-04T07:19:09.408 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20231211 -> 20231211 2024-03-04T07:19:09.408 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20240115 -> 20240115 2024-03-04T07:19:09.408 INFO:teuthology.orchestra.run.smithi113.stderr: * [new tag] 20240220 -> 20240220 2024-03-04T07:19:13.576 INFO:teuthology.orchestra.run.smithi113.stderr:Updating files: 51% (1829/3565) Updating files: 52% (1854/3565) Updating files: 53% (1890/3565) Updating files: 54% (1926/3565) Updating files: 55% (1961/3565) Updating files: 56% (1997/3565) Updating files: 57% (2033/3565) Updating files: 58% (2068/3565) Updating files: 59% (2104/3565) Updating files: 60% (2139/3565) Updating files: 61% (2175/3565) Updating files: 62% (2211/3565) Updating files: 63% (2246/3565) Updating files: 64% (2282/3565) Updating files: 64% (2308/3565) Updating files: 65% (2318/3565) Updating files: 66% (2353/3565) Updating files: 67% (2389/3565) Updating files: 68% (2425/3565) Updating files: 69% (2460/3565) Updating files: 70% (2496/3565) Updating files: 71% (2532/3565) Updating files: 72% (2567/3565) Updating files: 73% (2603/3565) Updating files: 74% (2639/3565) Updating files: 75% (2674/3565) Updating files: 76% (2710/3565) Updating files: 77% (2746/3565) Updating files: 78% (2781/3565) Updating files: 79% (2817/3565) Updating files: 79% (2825/3565) Updating files: 80% (2852/3565) Updating files: 81% (2888/3565) Updating files: 82% (2924/3565) Updating files: 83% (2959/3565) Updating files: 84% (2995/3565) Updating files: 85% (3031/3565) Updating files: 86% (3066/3565) Updating files: 87% (3102/3565) Updating files: 88% (3138/3565) Updating files: 89% (3173/3565) Updating files: 90% (3209/3565) Updating files: 91% (3245/3565) Updating files: 92% (3280/3565) Updating files: 93% (3316/3565) Updating files: 94% (3352/3565) Updating files: 95% (3387/3565) Updating files: 96% (3423/3565) Updating files: 97% (3459/3565) Updating files: 98% (3494/3565) Updating files: 99% (3530/3565) Updating files: 100% (3565/3565) Updating files: 100% (3565/3565), done. 2024-03-04T07:19:13.581 INFO:teuthology.orchestra.run.smithi113.stdout:HEAD is now at 5cd471e3 Merge branch 'amd-staging' into 'main' 2024-03-04T07:19:13.602 INFO:teuthology.task.kernel:Downloading kernel 786939134d7eee4dc7a805245d3f595929af37d7 on client.0... 2024-03-04T07:19:13.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=786939134d7eee4dc7a805245d3f595929af37d7 2024-03-04T07:19:13.830 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/786939134d7eee4dc7a805245d3f595929af37d7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-rc6-g786939134d7e/ 2024-03-04T07:19:13.831 DEBUG:teuthology.orchestra.run.smithi113:> rm -f /tmp/linux-image.deb && echo linux-image-6.8.0-rc6-g786939134d7e_6.8.0-rc6-g786939134d7e-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/786939134d7eee4dc7a805245d3f595929af37d7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-rc6-g786939134d7e/ --input-file=- 2024-03-04T07:19:13.833 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi113.front.sepia.ceph.com to complete... 2024-03-04T07:19:16.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04 07:19:16 URL:https://3.chacra.ceph.com/r/kernel/testing/786939134d7eee4dc7a805245d3f595929af37d7/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-rc6-g786939134d7e/linux-image-6.8.0-rc6-g786939134d7e_6.8.0-rc6-g786939134d7e-1_amd64.deb [39268740/39268740] -> "/tmp/linux-image.deb" [1] 2024-03-04T07:19:16.086 INFO:teuthology.orchestra.run.smithi113.stderr:FINISHED --2024-03-04 07:19:16-- 2024-03-04T07:19:16.086 INFO:teuthology.orchestra.run.smithi113.stderr:Total wall clock time: 2.2s 2024-03-04T07:19:16.086 INFO:teuthology.orchestra.run.smithi113.stderr:Downloaded: 1 files, 37M in 2.0s (18.9 MB/s) 2024-03-04T07:19:16.089 INFO:teuthology.task.kernel:Installing kernel 786939134d7eee4dc7a805245d3f595929af37d7 on client.0... 2024-03-04T07:19:16.089 DEBUG:teuthology.orchestra.run.smithi113:> sudo dpkg -i /tmp/linux-image.deb 2024-03-04T07:19:16.333 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-image-6.8.0-rc6-g786939134d7e. 2024-03-04T07:19:18.377 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-03-04T07:19:18.381 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-03-04T07:19:18.542 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-image-6.8.0-rc6-g786939134d7e (6.8.0-rc6-g786939134d7e-1) ... 2024-03-04T07:19:22.260 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-image-6.8.0-rc6-g786939134d7e (6.8.0-rc6-g786939134d7e-1) ... 2024-03-04T07:19:22.393 INFO:teuthology.orchestra.run.smithi113.stderr:update-initramfs: Generating /boot/initrd.img-6.8.0-rc6-g786939134d7e 2024-03-04T07:19:44.124 INFO:teuthology.orchestra.run.smithi113.stderr:Sourcing file `/etc/default/grub' 2024-03-04T07:19:44.143 INFO:teuthology.orchestra.run.smithi113.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T07:19:44.174 INFO:teuthology.orchestra.run.smithi113.stderr:Generating grub configuration file ... 2024-03-04T07:19:44.490 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-6.8.0-rc6-g786939134d7e 2024-03-04T07:19:44.504 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-6.8.0-rc6-g786939134d7e 2024-03-04T07:19:44.833 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T07:19:44.842 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T07:19:44.913 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T07:19:44.914 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T07:19:45.082 INFO:teuthology.orchestra.run.smithi113.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T07:19:45.082 INFO:teuthology.orchestra.run.smithi113.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T07:19:45.082 INFO:teuthology.orchestra.run.smithi113.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T07:19:45.133 INFO:teuthology.orchestra.run.smithi113.stderr:done 2024-03-04T07:19:45.350 DEBUG:teuthology.orchestra.run.smithi113:> dpkg-deb -c /tmp/linux-image.deb 2024-03-04T07:19:45.401 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./ 2024-03-04T07:19:45.401 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./boot/ 2024-03-04T07:19:45.401 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 4474507 2024-02-29 05:44 ./boot/System.map-6.8.0-rc6-g786939134d7e 2024-03-04T07:19:45.413 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 170908 2024-02-29 05:44 ./boot/config-6.8.0-rc6-g786939134d7e 2024-03-04T07:19:45.413 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11551744 2024-02-29 05:44 ./boot/vmlinuz-6.8.0-rc6-g786939134d7e 2024-03-04T07:19:45.431 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./etc/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./etc/kernel/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./etc/kernel/postinst.d/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./etc/kernel/postrm.d/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./etc/kernel/preinst.d/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./etc/kernel/prerm.d/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/ 2024-03-04T07:19:45.432 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/ 2024-03-04T07:19:45.433 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 490449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/aesni-intel.ko 2024-03-04T07:19:45.435 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-04T07:19:45.436 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-04T07:19:45.436 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-04T07:19:45.436 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 93081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-04T07:19:45.436 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 106697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-04T07:19:45.436 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 152145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-04T07:19:45.436 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-04T07:19:45.436 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-04T07:19:45.438 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 83857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-04T07:19:45.438 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-04T07:19:45.438 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-04T07:19:45.438 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-04T07:19:45.438 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-04T07:19:45.439 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-04T07:19:45.439 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-04T07:19:45.439 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 58489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-04T07:19:45.439 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-04T07:19:45.439 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-04T07:19:45.439 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-04T07:19:45.439 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/ 2024-03-04T07:19:45.439 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/cpu/ 2024-03-04T07:19:45.439 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/cpu/mce/ 2024-03-04T07:19:45.440 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-04T07:19:45.440 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13593 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/cpuid.ko 2024-03-04T07:19:45.440 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kernel/msr.ko 2024-03-04T07:19:45.440 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm/ 2024-03-04T07:19:45.440 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 354001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-04T07:19:45.440 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 886177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-04T07:19:45.440 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 2294497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/arch/x86/kvm/kvm.ko 2024-03-04T07:19:45.543 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/ 2024-03-04T07:19:45.544 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/af_alg.ko 2024-03-04T07:19:45.544 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/algif_hash.ko 2024-03-04T07:19:45.544 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/algif_skcipher.ko 2024-03-04T07:19:45.544 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17113 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/ansi_cprng.ko 2024-03-04T07:19:45.544 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19553 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/anubis.ko 2024-03-04T07:19:45.544 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9585 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/arc4.ko 2024-03-04T07:19:45.555 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/asymmetric_keys/ 2024-03-04T07:19:45.555 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-03-04T07:19:45.555 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/ 2024-03-04T07:19:45.555 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_memcpy.ko 2024-03-04T07:19:45.555 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_pq.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_tx.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/async_xor.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/async_tx/raid6test.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/authenc.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/authencesn.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/blake2b_generic.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/blowfish_common.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/blowfish_generic.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/camellia_generic.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cast5_generic.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cast6_generic.ko 2024-03-04T07:19:45.556 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cast_common.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/ccm.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cmac.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/crc32_generic.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cryptd.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/crypto_engine.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26921 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/crypto_simd.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/crypto_user.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/cts.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/des_generic.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15593 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/dh_generic.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10913 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/echainiv.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/essiv.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17905 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/fcrypt.ko 2024-03-04T07:19:45.557 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/khazad.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/lrw.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/lz4.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/lz4hc.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/md4.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/michael_mic.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10585 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/pcbc.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/pcrypt.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/rmd160.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/seed.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/serpent_generic.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 143033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/tcrypt.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/tea.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/twofish_common.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/twofish_generic.ko 2024-03-04T07:19:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/vmac.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/wp512.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/xcbc.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/xor.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/xts.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/crypto/xxhash_generic.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/ 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/acpi_extlog.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/acpi_pad.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/apei/ 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/apei/einj.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/ec_sys.ko 2024-03-04T07:19:45.559 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/platform_profile.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/sbs.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/sbshc.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 104433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/acpi/video.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/ 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/acard-ahci.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 89937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/ahci.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/ahci_platform.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 88313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/libahci.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/libahci_platform.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_acpi.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ali.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_amd.ko 2024-03-04T07:19:45.560 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_artop.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_atiixp.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_atp867x.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_cmd640.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_cmd64x.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_cypress.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_efar.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt366.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt37x.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_it8213.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_it821x.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_jmicron.ko 2024-03-04T07:19:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 52177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_legacy.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_marvell.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_mpiix.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_netcell.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ninja32.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ns87410.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_ns87415.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_oldpiix.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_opti.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20585 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_optidma.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_pcmcia.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14129 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_piccolo.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_radisys.ko 2024-03-04T07:19:45.562 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_rdc.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_rz1000.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sch.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_serverworks.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sil680.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_sl82c105.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15913 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_triflex.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pata_via.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/pdc_adma.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_inic162x.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 83257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_mv.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 70305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_nv.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_promise.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_qstor.ko 2024-03-04T07:19:45.563 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sil.ko 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sil24.ko 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sis.ko 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_svw.ko 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_sx4.ko 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_uli.ko 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_via.ko 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ata/sata_vsc.ko 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/auxdisplay/ 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/auxdisplay/ks0108.ko 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/bcma/ 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 124457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/bcma/bcma.ko 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/ 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/aoe/ 2024-03-04T07:19:45.564 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 113833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/aoe/aoe.ko 2024-03-04T07:19:45.565 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/drbd/ 2024-03-04T07:19:45.565 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 964817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/drbd/drbd.ko 2024-03-04T07:19:45.565 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 169705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/floppy.ko 2024-03-04T07:19:45.565 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/mtip32xx/ 2024-03-04T07:19:45.565 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 94217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-04T07:19:45.565 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 116401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/nbd.ko 2024-03-04T07:19:45.565 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/null_blk/ 2024-03-04T07:19:45.576 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 110329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/null_blk/null_blk.ko 2024-03-04T07:19:45.576 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 98193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/pktcdvd.ko 2024-03-04T07:19:45.576 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 263641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/rbd.ko 2024-03-04T07:19:45.576 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/xen-blkback/ 2024-03-04T07:19:45.576 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 98121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-04T07:19:45.576 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/zram/ 2024-03-04T07:19:45.576 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 58777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/block/zram/zram.ko 2024-03-04T07:19:45.576 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/applicom.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11913 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hangcheck-timer.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/ 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/via-rng.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 148521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 139361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-04T07:19:45.577 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 70873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/lp.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/mwave/ 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 84025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/mwave/mwave.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/nvram.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/ppdev.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tlclk.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/ 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22609 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-04T07:19:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23465 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/xillybus/ 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16905 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/xillybus/xillybus_class.ko 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/xillybus/xillybus_core.ko 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/clk/ 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/clk/clk-palmas.ko 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/clk/clk-si5351.ko 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/clk/clk-twl6040.ko 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/clk/clk-wm831x.ko 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/ 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-04T07:19:45.579 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/ 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/ccp/ 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 70473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 246841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/ccp/ccp.ko 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/ 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/ 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_common/ 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 570497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19905 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/padlock-aes.ko 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/padlock-sha.ko 2024-03-04T07:19:45.580 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/virtio/ 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 70433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dca/ 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dca/dca.ko 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/ 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/dw/ 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/ioat/ 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 122817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/ 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 80305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/amd64_edac.ko 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/e752x_edac.ko 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/edac_mce_amd.ko 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i3000_edac.ko 2024-03-04T07:19:45.581 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i3200_edac.ko 2024-03-04T07:19:45.582 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i5100_edac.ko 2024-03-04T07:19:45.582 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i5400_edac.ko 2024-03-04T07:19:45.582 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i7300_edac.ko 2024-03-04T07:19:45.582 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i7core_edac.ko 2024-03-04T07:19:45.582 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/i82975x_edac.ko 2024-03-04T07:19:45.582 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/ie31200_edac.ko 2024-03-04T07:19:45.582 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 56737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/sb_edac.ko 2024-03-04T07:19:45.582 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/edac/x38_edac.ko 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/extcon/ 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/extcon/extcon-gpio.ko 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/extcon/extcon-max14577.ko 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/extcon/extcon-max77693.ko 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/extcon/extcon-max8997.ko 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/extcon/extcon-palmas.ko 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/extcon/extcon-rt8973a.ko 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27689 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/extcon/extcon-sm5502.ko 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware/ 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware/dmi-sysfs.ko 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware/efi/ 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware/efi/efi-pstore.ko 2024-03-04T07:19:45.583 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/ 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-adp5520.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-amd8111.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-arizona.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-da9052.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-da9055.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-dln2.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 77753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-f7188x.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-generic.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-ich.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-03-04T07:19:45.584 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-kempld.ko 2024-03-04T07:19:45.585 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-lp3943.ko 2024-03-04T07:19:45.585 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-max7300.ko 2024-03-04T07:19:45.585 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-max730x.ko 2024-03-04T07:19:45.585 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-max732x.ko 2024-03-04T07:19:45.589 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-03-04T07:19:45.589 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-pca953x.ko 2024-03-04T07:19:45.589 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-pcf857x.ko 2024-03-04T07:19:45.589 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-rdc321x.ko 2024-03-04T07:19:45.589 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-sch.ko 2024-03-04T07:19:45.589 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-sch311x.ko 2024-03-04T07:19:45.589 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-tps65912.ko 2024-03-04T07:19:45.589 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-twl4030.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-twl6040.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-vx855.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-wm831x.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-wm8350.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/gpio/gpio-wm8994.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/ 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-a4tech.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-apple.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-appleir.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-aureal.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-axff.ko 2024-03-04T07:19:45.590 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-belkin.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-cherry.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-chicony.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 60665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-cp2112.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-cypress.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-dr.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-elecom.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-elo.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-emsff.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-ezkey.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-gaff.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-generic.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-gt683r.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12585 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-gyration.ko 2024-03-04T07:19:45.591 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-04T07:19:45.592 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-04T07:19:45.592 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-holtekff.ko 2024-03-04T07:19:45.592 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-hyperv.ko 2024-03-04T07:19:45.592 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-icade.ko 2024-03-04T07:19:45.592 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-kensington.ko 2024-03-04T07:19:45.593 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-keytouch.ko 2024-03-04T07:19:45.593 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17113 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-kye.ko 2024-03-04T07:19:45.593 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-lcpower.ko 2024-03-04T07:19:45.593 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-led.ko 2024-03-04T07:19:45.593 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-lenovo.ko 2024-03-04T07:19:45.593 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-lg-g15.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 57977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 101545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 80921 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-logitech.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-magicmouse.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-microsoft.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-monterey.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-multitouch.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-ntrig.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-ortek.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-penmount.ko 2024-03-04T07:19:45.594 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-petalynx.ko 2024-03-04T07:19:45.595 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 72865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-picolcd.ko 2024-03-04T07:19:45.595 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-pl.ko 2024-03-04T07:19:45.595 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-plantronics.ko 2024-03-04T07:19:45.595 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-primax.ko 2024-03-04T07:19:45.595 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-rmi.ko 2024-03-04T07:19:45.595 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-04T07:19:45.595 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-common.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-roccat.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13585 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-saitek.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-samsung.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-04T07:19:45.596 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sjoy.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sony.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-speedlink.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-steelseries.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-sunplus.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-tivo.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-tmff.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-topseed.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-twinhan.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 70993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-uclogic.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-waltop.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 143393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-wiimote.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8065 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-xinmo.ko 2024-03-04T07:19:45.597 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-zpff.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid-zydacron.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 269233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/hid.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/uhid.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/usbhid/ 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 125641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/usbhid/usbhid.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/usbhid/usbkbd.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/usbhid/usbmouse.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 218881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hid/wacom.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv/ 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 64305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv/hv_balloon.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 89553 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv/hv_utils.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 305425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hv/hv_vmbus.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 66801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/abituguru.ko 2024-03-04T07:19:45.598 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 61817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/abituguru3.ko 2024-03-04T07:19:45.599 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-04T07:19:45.599 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ad7414.ko 2024-03-04T07:19:45.599 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ad7418.ko 2024-03-04T07:19:45.599 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adc128d818.ko 2024-03-04T07:19:45.599 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1025.ko 2024-03-04T07:19:45.600 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 97889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1026.ko 2024-03-04T07:19:45.600 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1029.ko 2024-03-04T07:19:45.600 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 45465 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm1031.ko 2024-03-04T07:19:45.600 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adm9240.ko 2024-03-04T07:19:45.600 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ads7828.ko 2024-03-04T07:19:45.600 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7410.ko 2024-03-04T07:19:45.600 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7411.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 74321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7462.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7470.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 69313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7475.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/adt7x10.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/amc6821.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 55177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/applesmc.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/asb100.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 58529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/asc7621.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 40793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/atxp1.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/coretemp.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/da9052-hwmon.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/da9055-hwmon.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-04T07:19:45.601 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 99513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/dme1737.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ds1621.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ds620.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/emc1403.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/emc2103.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/emc6w201.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 66897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/f71805f.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 94601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/f71882fg.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/f75375s.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/fam15h_power.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 55361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/fschmd.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/g760a.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/g762.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 46713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/gl518sm.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 45441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/gl520sm.ko 2024-03-04T07:19:45.602 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/hih6130.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/i5500_temp.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/i5k_amb.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ibmaem.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ibmpex.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ina209.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ina2xx.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 140137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/it87.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/jc42.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/k10temp.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/k8temp.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lineage-pem.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 48513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm63.ko 2024-03-04T07:19:45.603 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm73.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm75.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm77.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm78.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm80.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm83.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 72745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm85.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 46065 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm87.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm90.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm92.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 108753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm93.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm95234.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm95241.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/lm95245.ko 2024-03-04T07:19:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc2945.ko 2024-03-04T07:19:45.605 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4151.ko 2024-03-04T07:19:45.605 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15105 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4215.ko 2024-03-04T07:19:45.605 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4222.ko 2024-03-04T07:19:45.605 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4245.ko 2024-03-04T07:19:45.605 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4260.ko 2024-03-04T07:19:45.605 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14905 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/ltc4261.ko 2024-03-04T07:19:45.605 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max16065.ko 2024-03-04T07:19:45.605 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max1619.ko 2024-03-04T07:19:45.605 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max1668.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max197.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max6639.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max6650.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35689 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/max6697.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/mc13783-adc.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/mcp3021.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct6683.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 119849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct6775-core.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 61441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct6775.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 59649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/nct7802.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 82721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pc87360.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 54345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pc87427.ko 2024-03-04T07:19:45.606 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pcf8591.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/ 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 82481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20593 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-04T07:19:45.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/powr1220.ko 2024-03-04T07:19:45.608 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sch5627.ko 2024-03-04T07:19:45.608 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sch5636.ko 2024-03-04T07:19:45.608 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-04T07:19:45.608 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sht15.ko 2024-03-04T07:19:45.608 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sht21.ko 2024-03-04T07:19:45.608 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/shtc1.ko 2024-03-04T07:19:45.608 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 40489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/sis5595.ko 2024-03-04T07:19:45.608 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/smsc47b397.ko 2024-03-04T07:19:45.608 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 40361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/smsc47m1.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/smsc47m192.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26465 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/thmc50.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp102.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp103.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp401.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/tmp421.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/via-cputemp.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 43449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/via686a.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 52369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/vt1211.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/vt8231.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 58265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83627ehf.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 79841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83627hf.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 71257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83781d.ko 2024-03-04T07:19:45.609 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 63201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83791d.ko 2024-03-04T07:19:45.610 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 71233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83792d.ko 2024-03-04T07:19:45.610 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 101385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83793.ko 2024-03-04T07:19:45.610 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 119417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83795.ko 2024-03-04T07:19:45.610 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83l785ts.ko 2024-03-04T07:19:45.610 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/w83l786ng.ko 2024-03-04T07:19:45.610 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-03-04T07:19:45.610 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-03-04T07:19:45.610 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/ 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/algos/ 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/ 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-04T07:19:45.611 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 67577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-04T07:19:45.612 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-mux.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-smbus.ko 2024-03-04T07:19:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/i2c-stub.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/muxes/ 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 273529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 883945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_core.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 69185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 324337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 97841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 282345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 66057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-04T07:19:45.614 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/ 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/cxgb4/ 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 493185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx4/ 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 420593 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx5/ 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 741737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mthca/ 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 296881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/ocrdma/ 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 153729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/usnic/ 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 169145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-03-04T07:19:45.615 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/ 2024-03-04T07:19:45.616 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/ipoib/ 2024-03-04T07:19:45.616 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 266577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-04T07:19:45.616 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/iser/ 2024-03-04T07:19:45.616 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 100281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-04T07:19:45.616 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/isert/ 2024-03-04T07:19:45.616 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 111337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-04T07:19:45.616 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srp/ 2024-03-04T07:19:45.616 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 128665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-04T07:19:45.616 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srpt/ 2024-03-04T07:19:45.616 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 129801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/ 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/evbug.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/ff-memless.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport/ 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/gameport/gameport.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/joydev.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/ 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12609 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-03-04T07:19:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/lkkbd.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/lm8323.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/lm8333.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/newtonkbd.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/qt1070.ko 2024-03-04T07:19:45.618 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/qt2160.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/stowaway.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/sunkbd.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/keyboard/xtkbd.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/matrix-keymap.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/ 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/appletouch.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/bcm5974.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 123201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/cyapatp.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 89393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-04T07:19:45.619 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/gpio_mouse.ko 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 335921 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/psmouse.ko 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14593 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/sermouse.ko 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/rmi4/ 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 109601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/ 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/altera_ps2.ko 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/arc_ps2.ko 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17689 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/ct82c710.ko 2024-03-04T07:19:45.620 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-04T07:19:45.621 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/parkbd.ko 2024-03-04T07:19:45.621 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/pcips2.ko 2024-03-04T07:19:45.621 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/ps2mult.ko 2024-03-04T07:19:45.621 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/serio_raw.ko 2024-03-04T07:19:45.621 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/serio/serport.ko 2024-03-04T07:19:45.621 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/input/sparse-keymap.ko 2024-03-04T07:19:45.621 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/ 2024-03-04T07:19:45.621 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-88pm860x.ko 2024-03-04T07:19:45.621 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-adp5520.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 59657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-bd2802.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24689 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-blinkm.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-da903x.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-da9052.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16689 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-gpio.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-lm3530.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-lm3533.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-lm355x.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-lm3642.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-lp3944.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-lp8788.ko 2024-03-04T07:19:45.622 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-max8997.ko 2024-03-04T07:19:45.623 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-mc13783.ko 2024-03-04T07:19:45.623 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-menf21bmc.ko 2024-03-04T07:19:45.623 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-pca9532.ko 2024-03-04T07:19:45.623 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20465 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-pca955x.ko 2024-03-04T07:19:45.623 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-pca963x.ko 2024-03-04T07:19:45.623 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-ss4200.ko 2024-03-04T07:19:45.623 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23905 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-tca6507.ko 2024-03-04T07:19:45.623 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-wm831x-status.ko 2024-03-04T07:19:45.623 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/leds-wm8350.ko 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-04T07:19:45.624 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/ 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/bcache/ 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 647121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/bcache/bcache.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 40489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-bio-prison.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 93321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-bufio.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-cache-smq.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 155825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-cache.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 107937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-crypt.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-delay.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-era.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-flakey.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-log-userspace.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-log.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-mirror.ko 2024-03-04T07:19:45.625 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 77881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-multipath.ko 2024-03-04T07:19:45.626 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-queue-length.ko 2024-03-04T07:19:45.626 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 70985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-raid.ko 2024-03-04T07:19:45.626 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-region-hash.ko 2024-03-04T07:19:45.626 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-round-robin.ko 2024-03-04T07:19:45.626 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-service-time.ko 2024-03-04T07:19:45.626 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 109521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-snapshot.ko 2024-03-04T07:19:45.626 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-switch.ko 2024-03-04T07:19:45.626 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 181777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-thin-pool.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-verity.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/dm-zero.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/persistent-data/ 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 199881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid0.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 100841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid1.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 126369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid10.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 368497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/md/raid456.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/ 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/ 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 178049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptbase.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 63241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptctl.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 46249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptfc.ko 2024-03-04T07:19:45.627 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptlan.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 123233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptsas.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 80985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptscsih.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 52929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/message/fusion/mptspi.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/ 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/88pm800.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/88pm805.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11105 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/88pm80x.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/arizona-i2c.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 123649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/arizona.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/bcm590xx.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/cros_ec_dev.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/dln2.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16593 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/janz-cmodio.ko 2024-03-04T07:19:45.628 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/kempld-core.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/lm3533-core.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12921 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/lp3943.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 56969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/lpc_ich.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/lpc_sch.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/max8907.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/mc13xxx-core.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/menf21bmc.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/pcf50633-adc.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/pcf50633-gpio.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/pcf50633.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-03-04T07:19:45.629 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/retu-mfd.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 80345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/si476x-core.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 48369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/sm501.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/tps65010.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/tps6507x.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/viperboard.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/vx855.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/wl1273-core.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/mfd/wm8994.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/ 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 48393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/ad525x_dpot.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/altera-stapl/ 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 46513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-04T07:19:45.630 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/apds9802als.ko 2024-03-04T07:19:45.631 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/apds990x.ko 2024-03-04T07:19:45.631 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 45497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/bh1770glc.ko 2024-03-04T07:19:45.631 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/c2port/ 2024-03-04T07:19:45.631 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-03-04T07:19:45.631 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/c2port/core.ko 2024-03-04T07:19:45.631 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/cb710/ 2024-03-04T07:19:45.631 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/cb710/cb710.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/ds1682.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/dummy-irq.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/echo/ 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/echo/echo.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom/ 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom/at24.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/eeprom/max6875.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32913 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/enclosure.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/genwqe/ 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 152073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/hmc6352.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/hpilo.ko 2024-03-04T07:19:45.632 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/ibmasm/ 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 72561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19689 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/ics932s401.ko 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21593 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/isl29003.ko 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/isl29020.ko 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/lis3lv02d/ 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei/ 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 125865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei/mei-me.ko 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 56377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei/mei-txe.ko 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 313601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/mei/mei.ko 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/phantom.ko 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/ti-st/ 2024-03-04T07:19:45.633 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 74305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/ti-st/st_drv.ko 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/tifm_7xx1.ko 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28689 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/tifm_core.ko 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/tsl2550.ko 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 48345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/vmw_balloon.ko 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/vmw_vmci/ 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 183569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/bonding/ 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 491609 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/bonding/bonding.ko 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/ 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/b53/ 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 105937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 40225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-04T07:19:45.634 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/mv88e6xxx/ 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 459065 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/dummy.ko 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/eql.ko 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/ 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 45961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 130665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 68785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/ 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 64617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-04T07:19:45.635 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 101681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-04T07:19:45.636 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/adaptec/ 2024-03-04T07:19:45.636 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 54625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-04T07:19:45.636 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/agere/ 2024-03-04T07:19:45.636 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 69169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-04T07:19:45.636 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/alteon/ 2024-03-04T07:19:45.636 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 59993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-04T07:19:45.636 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/altera/ 2024-03-04T07:19:45.636 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-04T07:19:45.636 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd/ 2024-03-04T07:19:45.636 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 54617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 87457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/ 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/alx/ 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 97201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 102497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 90209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atlx/ 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 81321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 69545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-04T07:19:45.639 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/ 2024-03-04T07:19:45.641 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 78385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-04T07:19:45.641 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 176697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-04T07:19:45.641 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-03-04T07:19:45.641 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1434505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-04T07:19:45.641 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 128433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-04T07:19:45.641 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/genet/ 2024-03-04T07:19:45.641 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 131281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-04T07:19:45.641 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 314329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-04T07:19:45.641 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/brocade/ 2024-03-04T07:19:45.641 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/brocade/bna/ 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 323737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/ 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 193593 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 347217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1045449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 166145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-04T07:19:45.642 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/cisco/ 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/cisco/enic/ 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 204249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/ 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/ 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 69433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 61425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 141201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 62833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dlink/ 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 60673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-04T07:19:45.643 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 62833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 45489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/dnet.ko 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ec_bhf.ko 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/emulex/ 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/emulex/benet/ 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 281217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ethoc.ko 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/fealnx.ko 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/fujitsu/ 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 45553 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 90305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000/ 2024-03-04T07:19:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 295697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000e/ 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 584353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/fm10k/ 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 246041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/i40e/ 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 944545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/iavf/ 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 288529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igb/ 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 513761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igbvf/ 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 93489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 734729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-04T07:19:45.645 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 151745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 86993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/jme.ko 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell/ 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 102441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 122609 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/ 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 727481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 278881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1121769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/micrel/ 2024-03-04T07:19:45.646 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 52985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 40881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 95577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/myricom/ 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 108409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/natsemi/ 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 73617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/neterion/ 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 151097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-04T07:19:45.647 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/nvidia/ 2024-03-04T07:19:45.648 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 136801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-04T07:19:45.648 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/packetengines/ 2024-03-04T07:19:45.648 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-04T07:19:45.648 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-04T07:19:45.648 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/ 2024-03-04T07:19:45.648 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-03-04T07:19:45.648 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 227865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-04T07:19:45.806 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 87017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 672057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/rdc/ 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/ 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 67281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 83809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 194073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/samsung/ 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 104049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-03-04T07:19:45.807 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sfc/ 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1067001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/silan/ 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sis/ 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 54729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 63201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/ 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 61425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 67297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 73257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 68049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/ 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-03-04T07:19:45.808 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-03-04T07:19:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-04T07:19:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-03-04T07:19:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 495689 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-04T07:19:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/ 2024-03-04T07:19:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 103185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-04T07:19:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 166049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-04T07:19:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 66273 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-04T07:19:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 78721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-04T07:19:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/tehuti/ 2024-03-04T07:19:45.809 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ti/ 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 92345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/via/ 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 80001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 83721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/wiznet/ 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 46313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/xircom/ 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 59921 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/fddi/ 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/fddi/defxx.ko 2024-03-04T07:19:45.810 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/fddi/skfp/ 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 217113 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/fddi/skfp/skfp.ko 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 74729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/geneve.ko 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/hyperv/ 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 237265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ieee802154/ 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ifb.ko 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ipvlan/ 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 85857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 62897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/macvlan.ko 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/macvtap.ko 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/ 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-04T07:19:45.811 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/mdio-gpio.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mdio.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/mii.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 45057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/netconsole.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/nlmon.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/pcs/ 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/ 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/amd.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-04T07:19:45.812 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/bcm87xx.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/broadcom.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/cicada.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/davicom.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/et1011c.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/icplus.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/lxt.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 81961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/marvell.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 76569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/micrel.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/national.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 99009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/phylink.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/qsemi.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/realtek.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/smsc.ko 2024-03-04T07:19:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/ste10Xp.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/phy/vitesse.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/plip/ 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 43769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/plip/plip.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_async.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/pppoe.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/pppox.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/ppp/pptp.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/sb1000.ko 2024-03-04T07:19:45.814 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/slip/ 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/slip/slip.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/sungem_phy.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 64625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/tap.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/ 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 107233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_random.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 87105 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/veth.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vmxnet3/ 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 144361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vxlan/ 2024-03-04T07:19:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 307257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/vxlan/vxlan.ko 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/xen-netback/ 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 144017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ntb/ 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ntb/ntb.ko 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme/ 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme/host/ 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 267745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme/host/nvme-core.ko 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 103361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/nvme/host/nvme.ko 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/ 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 131329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport.ko 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14465 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport_cs.ko 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 85217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport_pc.ko 2024-03-04T07:19:45.816 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/parport/parport_serial.ko 2024-03-04T07:19:45.817 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/ 2024-03-04T07:19:45.817 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/hotplug/ 2024-03-04T07:19:45.817 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24913 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-04T07:19:45.817 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-03-04T07:19:45.817 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-03-04T07:19:45.817 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/pci-stub.ko 2024-03-04T07:19:45.817 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 46433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pci/xen-pcifront.ko 2024-03-04T07:19:45.817 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/ 2024-03-04T07:19:45.817 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/i82092.ko 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 146153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/pcmcia.ko 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 62081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/pcmcia_core.ko 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/pd6729.ko 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 112793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/phy/ 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/phy/broadcom/ 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/phy/samsung/ 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pinctrl/ 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pinctrl/intel/ 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 69033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/ 2024-03-04T07:19:45.818 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/chrome/ 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24585 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/chrome/cros_ec.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26105 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/ 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 73601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/acerhdf.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 59569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 131945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-04T07:19:45.819 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 40169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/hdaps.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/ibm_rtl.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/intel/ 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/intel_ips.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-04T07:19:45.820 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 48481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/platform/x86/wmi.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/ 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/ 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/88pm860x_battery.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/88pm860x_charger.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 54961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/bq2415x_charger.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 59505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/bq24190_charger.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/bq24735-charger.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-03-04T07:19:45.821 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/da9030_battery.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/da9052-battery.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/ds2760_battery.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/ds2780_battery.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/ds2781_battery.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23553 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/ds2782_battery.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/gpio-charger.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/isp1704_charger.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/lp8727_charger.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/max14577_charger.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/max17042_battery.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/max8903_charger.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/max8925_power.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/pcf50633-charger.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/sbs-battery.ko 2024-03-04T07:19:45.822 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/test_power.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/tps65090-charger.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/wm831x_backup.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/wm831x_power.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/power/supply/wm8350_power.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/powercap/ 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 58105 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/ 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients/ 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/clients/pps_parport.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/pps/pps_core.ko 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ptp/ 2024-03-04T07:19:45.823 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 70873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ptp/ptp.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ptp/ptp_kvm.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/ 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-88pm80x.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-88pm860x.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-da9052.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-da9055.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-da9063.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 68481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-04T07:19:45.824 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13273 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23609 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-em3027.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-lp8788.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-m48t86.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-max6900.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-max8907.ko 2024-03-04T07:19:45.825 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-max8925.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-max8997.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16585 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-max8998.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-palmas.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 46537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf50633.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rc5t583.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-04T07:19:45.826 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-04T07:19:45.827 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-04T07:19:45.827 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-04T07:19:45.827 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-04T07:19:45.827 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-s35390a.ko 2024-03-04T07:19:45.827 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-04T07:19:45.827 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-tps6586x.ko 2024-03-04T07:19:45.827 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-tps65910.ko 2024-03-04T07:19:45.827 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-wm831x.ko 2024-03-04T07:19:45.827 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-wm8350.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/rtc/rtc-x1205.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 77529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/3w-9xxx.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 57721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/3w-sas.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 55081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/3w-xxxx.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 135345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/BusLogic.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/a100u2w.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aacraid/ 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 271441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 100473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/advansys.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic7xxx/ 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 356865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 295993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-04T07:19:45.828 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic94xx/ 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 189145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/am53c974.ko 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/arcmsr/ 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 108273 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/atp870u.ko 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/be2iscsi/ 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 246937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bfa/ 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1171953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bfa/bfa.ko 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2fc/ 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 238401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2i/ 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 115329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 40785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ch.ko 2024-03-04T07:19:45.829 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/csiostor/ 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 308961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/ 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 76041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 101425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 146817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 75977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/dc395x.ko 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/dmx3191d.ko 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/esas2r/ 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 212385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 60505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/esp_scsi.ko 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fcoe/ 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 78225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-04T07:19:45.830 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 157729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fdomain.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fnic/ 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 294953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/fnic/fnic.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 201729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/hpsa.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/hptiop.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 48945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/hv_storvsc.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36129 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/imm.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 43977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/initio.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 265945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ipr.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 101881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ips.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/isci/ 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 317193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/isci/isci.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-04T07:19:45.831 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 52865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libfc/ 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 315385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libfc/libfc.ko 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 145497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libiscsi.ko 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libsas/ 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 216401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/libsas/libsas.ko 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/lpfc/ 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 2194089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid/ 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 73473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 310689 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 85521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/megaraid.ko 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mpt3sas/ 2024-03-04T07:19:45.832 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 659569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mvsas/ 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 119489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 78041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/mvumi.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pcmcia/ 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 81561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pm8001/ 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 366337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 124601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/pmcraid.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ppa.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 55337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla1280.ko 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla2xxx/ 2024-03-04T07:19:45.833 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1876753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 85193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla4xxx/ 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 558953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/qlogicfas408.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/raid_class.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 261081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_debug.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 173929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 280113 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 97625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 73865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/ses.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 115713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/st.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 46737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/stex.ko 2024-03-04T07:19:45.834 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/sym53c8xx_2/ 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 164889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/virtio_scsi.ko 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/wd719x.ko 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/scsi/xen-scsifront.ko 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ssb/ 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 129393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ssb/ssb.ko 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/ 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/iscsi/ 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 736761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/loopback/ 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_file.ko 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_iblock.ko 2024-03-04T07:19:45.835 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 931521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_mod.ko 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_pscsi.ko 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 122489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/target_core_user.ko 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/tcm_fc/ 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 75521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/ 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/ 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/ 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-04T07:19:45.836 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14905 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11465 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-04T07:19:45.837 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/ 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/ipwireless/ 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 78353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 57777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/moxa.ko 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/mxser.ko 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 108489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/n_gsm.ko 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/n_hdlc.ko 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 67841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/nozomi.ko 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/ 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/8250/ 2024-03-04T07:19:45.838 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 59401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/altera_uart.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/arc_uart.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 72489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/jsm/ 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 101993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/rp2.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/tty/serial/sc16is7xx.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/ 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/core/ 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 545401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/host/ 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-04T07:19:45.839 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/ 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_aec.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_cif.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_dmem_genirq.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_mf624.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_netx.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_pci_generic.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/uio/uio_sercos3.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/ 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/c67x00/ 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 73225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/c67x00/c67x00.ko 2024-03-04T07:19:45.840 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/chipidea/ 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 145793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/ 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 87009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/cdc-acm.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/usblp.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 89769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/class/usbtmc.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/common/ 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/common/ulpi.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/dwc2/ 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/dwc3/ 2024-03-04T07:19:45.841 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 452825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/dwc3/dwc3.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/ 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/ 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 60329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/u_ether.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/u_serial.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 92409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 50945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-03-04T07:19:45.842 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 122257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 59865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 57633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 85521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 58025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 67353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/ 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-03-04T07:19:45.843 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18921 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27113 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 62369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15105 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 161233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/libcomposite.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/ 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17609 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/bdc/ 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 157617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-03-04T07:19:45.844 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 52929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 69617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 55953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 75969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/net2272.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 74985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/net2280.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 73017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 65881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 66001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 81409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 156337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/ 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/bcma-hcd.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 80017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/isp116x-hcd.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 87193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-03-04T07:19:45.845 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 78777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/sl811_cs.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/ssb-hcd.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/image/ 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/image/mdc800.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/image/microtek.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/ 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/adutux.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/appledisplay.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/cytherm.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/ehset.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/emi26.ko 2024-03-04T07:19:45.846 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/emi62.ko 2024-03-04T07:19:45.847 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/ezusb.ko 2024-03-04T07:19:45.847 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/idmouse.ko 2024-03-04T07:19:45.847 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/iowarrior.ko 2024-03-04T07:19:45.847 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-04T07:19:45.847 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/ldusb.ko 2024-03-04T07:19:45.847 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/legousbtower.ko 2024-03-04T07:19:45.847 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/lvstest.ko 2024-03-04T07:19:45.847 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/sisusbvga/ 2024-03-04T07:19:45.847 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 64745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-04T07:19:45.847 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/usb3503.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/usblcd.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 70721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/usbtest.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/uss720.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/misc/yurex.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/mon/ 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 72825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/mon/usbmon.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/musb/ 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 301633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/musb/musb_hdrc.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/phy/ 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/phy/phy-generic.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-03-04T07:19:45.849 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/phy/phy-isp1301.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/phy/phy-tahvo.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/aircable.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ark3116.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ch341.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 70345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/cp210x.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/cyberjack.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 58121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/empeg.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/f81232.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 175105 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-04T07:19:45.850 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 136985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 124745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/io_ti.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 67681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ipaq.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ipw.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ir-usb.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 58137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 107433 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/keyspan.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33593 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/mct_u232.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/metro-usb.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 66361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/mos7720.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 78129 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/mos7840.ko 2024-03-04T07:19:45.851 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 56249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/mxuport.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/navman.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11921 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/omninet.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/opticon.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 191825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/option.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/oti6858.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 54265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/pl2303.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/qcaux.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/qcserial.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/quatech2.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/safe_serial.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/sierra.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21585 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29905 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ssu100.ko 2024-03-04T07:19:45.852 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/symbolserial.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 61393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/usb_debug.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33113 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 125529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/usbserial.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/visor.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/whiteheat.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/wishbone-serial.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 52761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/uas.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-04T07:19:45.853 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-karma.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 180329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/storage/usb-storage.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip/ 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 67401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 71025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-04T07:19:45.854 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 104065 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/ 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/pci/ 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 162865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 103673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/vfio.ko 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 76441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/ 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 113665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost.ko 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16913 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 58001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost_net.ko 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 89057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/vhost/vhost_scsi.ko 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/ 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/ 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-03-04T07:19:45.855 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/aat2870_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/adp5520_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/adp8860_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/adp8870_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/apple_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/as3711_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/bd6107.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/da903x_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/da9052_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/gpio_backlight.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/kb3886_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/lcd.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/lm3533_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/lm3639_bl.ko 2024-03-04T07:19:45.856 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/lv5207lp.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/max8925_bl.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/pandora_bl.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/video/backlight/wm831x_bl.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/ 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters/ 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters/ds2482.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters/ds2490.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters/matrox_w1.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11273 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/masters/w1-gpio.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/ 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-04T07:19:45.857 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 56337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 103049 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/w1/wire.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/ 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/acquirewdt.ko 2024-03-04T07:19:45.858 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/advantechwdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18465 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/cpu5wdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/da9052_wdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/da9055_wdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/da9063_wdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/dw_wdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/eurotechwdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/hpwdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/i6300esb.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/ib700wdt.ko 2024-03-04T07:19:45.859 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/ibmasr.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21129 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/it87_wdt.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21393 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/kempld_wdt.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/machzwd.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/mena21_wdt.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24585 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/nv_tco.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/pc87413_wdt.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39921 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/retu_wdt.ko 2024-03-04T07:19:45.860 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sc1200wdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/softdog.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/twl4030_wdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/via_wdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/wafer5823wdt.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/wdt_pci.ko 2024-03-04T07:19:45.861 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/wm831x_wdt.ko 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13689 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/wm8350_wdt.ko 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15609 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/watchdog/xen_wdt.ko 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/ 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-evtchn.ko 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-gntalloc.ko 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-gntdev.ko 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-pciback/ 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 176385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 40097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-privcmd.ko 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 71153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xen-scsiback.ko 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xenfs/ 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ 2024-03-04T07:19:45.862 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/9p/ 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 158281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/9p/9p.ko 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/adfs/ 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 61721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/adfs/adfs.ko 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/affs/ 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 163481 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/affs/affs.ko 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/afs/ 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1019225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/afs/kafs.ko 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/autofs/ 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 103225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/autofs/autofs4.ko 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/befs/ 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 64625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/befs/befs.ko 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/bfs/ 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 41017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/bfs/bfs.ko 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/binfmt_misc.ko 2024-03-04T07:19:45.863 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/btrfs/ 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 3485265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/btrfs/btrfs.ko 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cachefiles/ 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 305897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cachefiles/cachefiles.ko 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ceph/ 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1336001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ceph/ceph.ko 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/coda/ 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 97769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/coda/coda.ko 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/configfs/ 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 108577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/configfs/configfs.ko 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cramfs/ 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29593 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/cramfs/cramfs.ko 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/dlm/ 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 644249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/dlm/dlm.ko 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/efs/ 2024-03-04T07:19:45.864 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/efs/efs.ko 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/f2fs/ 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1527729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/f2fs/f2fs.ko 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fat/ 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fat/msdos.ko 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/freevxfs/ 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/freevxfs/freevxfs.ko 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fuse/ 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/fuse/cuse.ko 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/gfs2/ 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1231361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/gfs2/gfs2.ko 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfs/ 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 141713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfs/hfs.ko 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfsplus/ 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 223281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hfsplus/hfsplus.ko 2024-03-04T07:19:45.865 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hpfs/ 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 193545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/hpfs/hpfs.ko 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/isofs/ 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 74737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/isofs/isofs.ko 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/jfs/ 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 420449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/jfs/jfs.ko 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/lockd/ 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 281705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/lockd/lockd.ko 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/minix/ 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 88937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/minix/minix.ko 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/netfs/ 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 490505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/netfs/netfs.ko 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/ 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/blocklayout/ 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 81457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-04T07:19:45.866 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/filelayout/ 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 59353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/flexfilelayout/ 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 117377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1058377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/nfs.ko 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 66361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/nfsv2.ko 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 120193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/nfsv3.ko 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 2041825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs/nfsv4.ko 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs_common/ 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs_common/grace.ko 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfs_common/nfs_acl.ko 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfsd/ 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1511665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nfsd/nfsd.ko 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nilfs2/ 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 582529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nilfs2/nilfs2.ko 2024-03-04T07:19:45.867 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/ 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-celtic.ko 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-centeuro.ko 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-croatian.ko 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-cyrillic.ko 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-gaelic.ko 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-greek.ko 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-iceland.ko 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-inuit.ko 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-roman.ko 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-romanian.ko 2024-03-04T07:19:45.868 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/mac-turkish.ko 2024-03-04T07:19:45.871 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_ascii.ko 2024-03-04T07:19:45.871 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp1250.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp1251.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp1255.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp737.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp775.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12129 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp850.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12129 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp852.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12129 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp855.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp857.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp860.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13065 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp861.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp862.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13065 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp863.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp864.ko 2024-03-04T07:19:45.872 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13065 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp865.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp866.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12129 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp869.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp874.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 94041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp932.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 142577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp936.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 165649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp949.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 118025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_cp950.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_euc-jp.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-1.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-13.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-14.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-15.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-2.ko 2024-03-04T07:19:45.873 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-3.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-4.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-5.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-6.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-7.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_iso8859-9.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_koi8-r.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_koi8-ru.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_koi8-u.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/nls/nls_utf8.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ntfs/ 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 220545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ntfs/ntfs.ko 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ 2024-03-04T07:19:45.874 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/cluster/ 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 244673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlm/ 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 440681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlmfs/ 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 54177 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 2590849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2.ko 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/omfs/ 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/omfs/omfs.ko 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/overlayfs/ 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 360953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/overlayfs/overlay.ko 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/pstore/ 2024-03-04T07:19:45.875 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 56945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/pstore/ramoops.ko 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/qnx4/ 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26913 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/qnx4/qnx4.ko 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/qnx6/ 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 46625 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/qnx6/qnx6.ko 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/quota/ 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/quota/quota_tree.ko 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14273 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/quota/quota_v1.ko 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/quota/quota_v2.ko 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/reiserfs/ 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 532289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/reiserfs/reiserfs.ko 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/romfs/ 2024-03-04T07:19:45.876 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28905 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/romfs/romfs.ko 2024-03-04T07:19:45.877 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/ 2024-03-04T07:19:45.877 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/client/ 2024-03-04T07:19:45.877 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 2712601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/client/cifs.ko 2024-03-04T07:19:45.877 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/common/ 2024-03-04T07:19:45.878 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7065 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/common/cifs_arc4.ko 2024-03-04T07:19:45.878 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/smb/common/cifs_md4.ko 2024-03-04T07:19:45.878 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/squashfs/ 2024-03-04T07:19:45.878 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 134809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/squashfs/squashfs.ko 2024-03-04T07:19:45.878 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/sysv/ 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 91329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/sysv/sysv.ko 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/udf/ 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 248409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/udf/udf.ko 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ufs/ 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 162785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/ufs/ufs.ko 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/xfs/ 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 4036329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/fs/xfs/xfs.ko 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/ 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/locking/ 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 80489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/locking/locktorture.ko 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/time/ 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/time/test_udelay.ko 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 60265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/kernel/torture.ko 2024-03-04T07:19:45.879 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/ 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crc-itu-t.ko 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crc7.ko 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crc8.ko 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crypto/ 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crypto/libarc4.ko 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/crypto/libdes.ko 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/interval_tree_test.ko 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/libcrc32c.ko 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lru_cache.ko 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lz4/ 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lz4/lz4_compress.ko 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/lz4/lz4hc_compress.ko 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/math/ 2024-03-04T07:19:45.880 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/math/cordic.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/memory-notifier-error-inject.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/notifier-error-inject.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/percpu_test.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/pm-notifier-error-inject.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/raid6/ 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 143337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/raid6/raid6_pq.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/rbtree_test.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/reed_solomon/ 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/test-kstrtox.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/test-string_helpers.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 4863225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/test_bpf.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 48993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/test_firmware.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/test_module.ko 2024-03-04T07:19:45.881 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/test_user_copy.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/ts_bm.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/ts_fsm.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/ts_kmp.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/xz/ 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/lib/xz/xz_dec_test.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/mm/ 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18129 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/mm/hwpoison-inject.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/ 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 55777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/6lowpan.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_dest.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_fragment.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6961 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_hop.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-04T07:19:45.882 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_mobility.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 6969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_routing.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/6lowpan/nhc_udp.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/ 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/garp.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/mrp.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/p8022.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/psnap.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/802/stp.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/8021q/ 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 82897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/8021q/8021q.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/ 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 198833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet_fd.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet_rdma.ko 2024-03-04T07:19:45.883 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/9p/9pnet_virtio.ko 2024-03-04T07:19:45.884 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/batman-adv/ 2024-03-04T07:19:45.884 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 667545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/batman-adv/batman-adv.ko 2024-03-04T07:19:45.884 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/ 2024-03-04T07:19:45.884 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 65953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/br_netfilter.ko 2024-03-04T07:19:45.884 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 790673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/bridge.ko 2024-03-04T07:19:45.884 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ 2024-03-04T07:19:45.884 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-04T07:19:45.884 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-04T07:19:45.884 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-04T07:19:45.884 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-04T07:19:45.885 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 55945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/ebtables.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ceph/ 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1046889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ceph/libceph.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/core/ 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 125769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/core/pktgen.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/ 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 194505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp_diag.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 58745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp_ipv4.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 61401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dccp/dccp_ipv6.ko 2024-03-04T07:19:45.886 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/ 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 390089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/dsa_core.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13361 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_brcm.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12601 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_dsa.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_none.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/dsa/tag_trailer.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/hsr/ 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 102073 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/hsr/hsr.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/ 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/6lowpan/ 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 45505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 284721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/ieee802154.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 47881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ah4.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/esp4.ko 2024-03-04T07:19:45.887 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 48249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/fou.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/gre.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 51217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/inet_diag.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 61137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ip_gre.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 61385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ip_tunnel.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ip_vti.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ipcomp.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 29081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/ipip.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ip_tables.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-04T07:19:45.888 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9585 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14785 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13241 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-04T07:19:45.889 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8913 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_bic.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_dctcp.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15969 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_diag.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_highspeed.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_htcp.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_hybla.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_illinois.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_lp.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_scalable.ko 2024-03-04T07:19:45.890 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_vegas.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_veno.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10857 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_westwood.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tcp_yeah.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/tunnel4.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/udp_diag.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 45057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/udp_tunnel.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ah6.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/esp6.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/fou6.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 79929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ip6_gre.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 94169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ip6_tunnel.ko 2024-03-04T07:19:45.891 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 54273 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ip6_vti.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/ipcomp6.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/mip6.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42553 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14105 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16113 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-04T07:19:45.892 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-04T07:19:45.893 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-04T07:19:45.893 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-04T07:19:45.893 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-04T07:19:45.893 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-04T07:19:45.893 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-04T07:19:45.893 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-04T07:19:45.893 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-04T07:19:45.893 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8913 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 74881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/sit.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/tunnel6.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/key/ 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 84465 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/key/af_key.ko 2024-03-04T07:19:45.894 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/ 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 125209 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_core.ko 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 25817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_eth.ko 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36497 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_ip.ko 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_ip6.ko 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_netlink.ko 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 53233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/l2tp/l2tp_ppp.ko 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/lapb/ 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 48121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/lapb/lapb.ko 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/llc/ 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/llc/llc.ko 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 268521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/llc/llc2.ko 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mac802154/ 2024-03-04T07:19:45.895 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 309417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mac802154/mac802154.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mpls/ 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/mpls/mpls_gso.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 119705 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28617 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 99993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 100281 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 102553 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 102345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 111673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-04T07:19:45.896 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 59193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 108345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 112145 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 112409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 110337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 113289 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 412513 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11993 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-04T07:19:45.897 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10641 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9265 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13545 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7721 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conncount.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 411745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15881 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 118097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-04T07:19:45.898 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 108313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 66537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 92233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_flow_table.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9553 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_log_syslog.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 96521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12529 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_irc.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_sip.ko 2024-03-04T07:19:45.899 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 44137 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 623473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nf_tables.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 42577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_log.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 22937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49649 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_chain_nat.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_compat.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 40761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_ct.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9505 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_fib.ko 2024-03-04T07:19:45.900 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_hash.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20033 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_limit.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_log.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_masq.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14329 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_nat.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_numgen.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11753 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_osf.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_queue.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_redir.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_reject.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9985 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_reject_inet.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12841 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_socket.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_synproxy.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/nft_tproxy.ko 2024-03-04T07:19:45.901 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 89233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/x_tables.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_AUDIT.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7633 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 18081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_CT.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_DSCP.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_HL.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13345 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_HMARK.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 38777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13801 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_LED.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_LOG.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11561 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_NETMAP.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_NFLOG.ko 2024-03-04T07:19:45.902 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_RATEEST.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_SECMARK.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TEE.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TPROXY.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8233 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_TRACE.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13097 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_addrtype.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_bpf.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_cgroup.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11121 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_cluster.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_comment.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connbytes.ko 2024-03-04T07:19:45.903 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11257 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connlabel.ko 2024-03-04T07:19:45.904 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connlimit.ko 2024-03-04T07:19:45.904 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13353 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_connmark.ko 2024-03-04T07:19:45.904 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_conntrack.ko 2024-03-04T07:19:45.904 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_cpu.ko 2024-03-04T07:19:45.904 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10889 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_dccp.ko 2024-03-04T07:19:45.904 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7769 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_devgroup.ko 2024-03-04T07:19:45.904 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_dscp.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10569 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_ecn.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_esp.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 43201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_hashlimit.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_helper.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_hl.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11825 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_ipcomp.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_iprange.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11697 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_ipvs.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_l2tp.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7905 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_length.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_limit.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_mac.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8553 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_mark.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11729 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_multiport.ko 2024-03-04T07:19:45.905 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13737 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_nat.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_nfacct.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_osf.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_owner.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10017 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_physdev.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_pkttype.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11897 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_policy.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9385 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_quota.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_rateest.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8169 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_realm.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_recent.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13305 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_sctp.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30001 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_set.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 16817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_socket.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_state.ko 2024-03-04T07:19:45.906 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_statistic.ko 2024-03-04T07:19:45.907 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8713 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_string.ko 2024-03-04T07:19:45.907 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 8457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_tcpmss.ko 2024-03-04T07:19:45.907 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15681 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_tcpudp.ko 2024-03-04T07:19:45.907 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_time.ko 2024-03-04T07:19:45.907 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9313 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netfilter/xt_u32.ko 2024-03-04T07:19:45.907 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netlink/ 2024-03-04T07:19:45.907 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 14441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/netlink/netlink_diag.ko 2024-03-04T07:19:45.907 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/nsh/ 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 12337 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/nsh/nsh.ko 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/ 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 384201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/openvswitch.ko 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10369 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/vport-geneve.ko 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/vport-gre.ko 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11833 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/openvswitch/vport-vxlan.ko 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/packet/ 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13473 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/packet/af_packet_diag.ko 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/phonet/ 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 112553 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/phonet/phonet.ko 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 55057 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/phonet/pn_pep.ko 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds/ 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 254025 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds/rds.ko 2024-03-04T07:19:45.908 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 165105 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds/rds_rdma.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 68161 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rds/rds_tcp.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rxrpc/ 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 828297 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/rxrpc/rxrpc.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/ 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_csum.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23489 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_gact.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 37089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_mirred.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23377 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_nat.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 43417 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_pedit.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_police.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20193 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_simple.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35201 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_skbedit.ko 2024-03-04T07:19:45.909 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36577 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/act_vlan.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_basic.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33873 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_bpf.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 19185 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_cgroup.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_flow.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_fw.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 39521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_route.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 62657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/cls_u32.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7465 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_cmp.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 13321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_ipset.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 33425 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_meta.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7921 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_nbyte.ko 2024-03-04T07:19:45.910 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10929 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_text.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7273 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/em_u32.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21673 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_choke.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20865 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_codel.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 34657 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_drr.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_fq.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31521 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_fq_codel.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 35457 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_gred.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 49849 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_hfsc.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 21977 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_hhf.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 66449 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_htb.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_ingress.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 30225 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_mqprio.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10809 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_mqprio_lib.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26089 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_multiq.ko 2024-03-04T07:19:45.911 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 36153 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_netem.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 27409 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_pie.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10665 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_plug.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 24777 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_prio.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 48537 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_qfq.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 32817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_red.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 26793 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_sfb.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 31321 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_sfq.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_tbf.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 28273 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sched/sch_teql.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sctp/ 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 932945 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sctp/sctp.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sctp/sctp_diag.ko 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/ 2024-03-04T07:19:45.912 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/auth_gss/ 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 337009 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 55041 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 1513905 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/sunrpc.ko 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/xprtrdma/ 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 704761 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/tipc/ 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 10401 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/tipc/diag.ko 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 867249 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/tipc/tipc.ko 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/unix/ 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15937 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/unix/unix_diag.ko 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/ 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 87745 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 67953 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 98113 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vsock.ko 2024-03-04T07:19:45.913 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 11081 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 17817 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/x25/ 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 131273 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/x25/x25.ko 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/xfrm/ 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23065 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/xfrm/xfrm_algo.ko 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 20553 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 105217 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/net/xfrm/xfrm_user.ko 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/virt/ 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/virt/lib/ 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 15441 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/kernel/virt/lib/irqbypass.ko 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 642283 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.alias 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 625640 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 7677 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 23498 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T07:19:45.914 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 9916 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 70732 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.modinfo 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 147214 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.dep 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 233213 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 210 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.devname 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 72614 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.order 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 879 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.softdep 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 257955 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 312782 2024-02-29 05:44 ./lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./usr/ 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./usr/share/ 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./usr/share/doc/ 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:drwxr-xr-x root/root 0 2024-02-29 05:44 ./usr/share/doc/linux-image-6.8.0-rc6-g786939134d7e/ 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 175 2024-02-29 05:44 ./usr/share/doc/linux-image-6.8.0-rc6-g786939134d7e/changelog.Debian.gz 2024-03-04T07:19:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r-- root/root 691 2024-02-29 05:44 ./usr/share/doc/linux-image-6.8.0-rc6-g786939134d7e/copyright 2024-03-04T07:19:45.921 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc6-g786939134d7e 2024-03-04T07:19:45.921 INFO:teuthology.task.kernel:searching for kernel 6.8.0-rc6-g786939134d7e 2024-03-04T07:19:45.922 DEBUG:teuthology.orchestra.run.smithi113:> egrep '(submenu|menuentry.*6.8.0-rc6-g786939134d7e).*{' /boot/grub/grub.cfg 2024-03-04T07:19:45.930 INFO:teuthology.orchestra.run.smithi113.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-04T07:19:45.930 INFO:teuthology.orchestra.run.smithi113.stdout: menuentry 'Ubuntu, with Linux 6.8.0-rc6-g786939134d7e' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-rc6-g786939134d7e-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-04T07:19:45.930 INFO:teuthology.orchestra.run.smithi113.stdout: menuentry 'Ubuntu, with Linux 6.8.0-rc6-g786939134d7e (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-rc6-g786939134d7e-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-04T07:19:45.931 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-03-04T07:19:45.931 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.8.0-rc6-g786939134d7e 2024-03-04T07:19:45.931 DEBUG:teuthology.orchestra.run.smithi113:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-03-04T07:19:45.973 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi113.front.sepia.ceph.com to complete... 2024-03-04T07:19:46.367 INFO:teuthology.orchestra.run.smithi113.stderr:Sourcing file `/etc/default/grub' 2024-03-04T07:19:46.368 INFO:teuthology.orchestra.run.smithi113.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T07:19:46.389 INFO:teuthology.orchestra.run.smithi113.stderr:Generating grub configuration file ... 2024-03-04T07:19:46.612 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-6.8.0-rc6-g786939134d7e 2024-03-04T07:19:46.618 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-6.8.0-rc6-g786939134d7e 2024-03-04T07:19:46.879 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T07:19:46.884 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T07:19:46.924 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T07:19:46.929 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T07:19:47.076 INFO:teuthology.orchestra.run.smithi113.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T07:19:47.076 INFO:teuthology.orchestra.run.smithi113.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T07:19:47.076 INFO:teuthology.orchestra.run.smithi113.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T07:19:47.097 INFO:teuthology.orchestra.run.smithi113.stderr:done 2024-03-04T07:20:18.246 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-04T07:20:18.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-04T07:20:18.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T07:20:36.671 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-03-04T07:20:45.672 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-04T07:20:45.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T07:20:48.732 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-03-04T07:21:00.733 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-04T07:21:00.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T07:21:03.804 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-03-04T07:21:18.808 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-04T07:21:18.809 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T07:21:37.341 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-03-04T07:21:55.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-04T07:21:55.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T07:21:55.644 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-03-04T07:21:56.362 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-04T07:21:56.362 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc6-g786939134d7e"... 2024-03-04T07:21:56.362 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-03-04T07:21:56.413 INFO:teuthology.orchestra.run.smithi113.stdout:6.8.0-rc6-g786939134d7e 2024-03-04T07:21:56.413 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc6-g786939134d7e vs 6.8.0-rc6-g786939134d7e 2024-03-04T07:21:56.413 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T07:21:56.413 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-04T07:21:57.416 DEBUG:teuthology.parallel:result is None 2024-03-04T07:21:57.416 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-04T07:21:57.424 INFO:teuthology.task.internal:Creating test directory... 2024-03-04T07:21:57.425 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T07:21:57.439 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T07:21:57.447 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-04T07:21:57.455 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-04T07:21:57.481 INFO:teuthology.task.internal:Creating archive directory... 2024-03-04T07:21:57.481 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T07:21:57.492 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T07:21:57.831 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-04T07:21:57.839 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-04T07:21:57.839 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T07:21:57.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:21:57.845 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T07:21:57.851 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T07:21:57.937 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T07:21:57.944 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T07:21:57.953 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T07:21:57.968 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T07:21:57.973 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-04T07:21:57.981 INFO:teuthology.task.internal:Configuring sudo... 2024-03-04T07:21:57.981 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T07:21:57.987 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T07:21:58.004 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-04T07:21:58.013 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-04T07:21:58.020 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T07:21:58.055 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T07:21:58.062 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T07:21:58.110 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T07:21:58.162 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:21:58.162 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T07:21:58.236 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T07:21:58.242 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T07:21:58.289 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:21:58.289 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T07:21:58.354 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-03-04T07:21:58.357 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-03-04T07:21:58.460 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-04T07:21:58.471 INFO:teuthology.task.internal:Starting timer... 2024-03-04T07:21:58.471 INFO:teuthology.run_tasks:Running task pcp... 2024-03-04T07:21:58.519 INFO:teuthology.run_tasks:Running task selinux... 2024-03-04T07:21:58.545 DEBUG:teuthology.task.selinux:Excluding smithi113: OS 'ubuntu' does not support SELinux 2024-03-04T07:21:58.545 DEBUG:teuthology.task.selinux:Excluding smithi136: OS 'ubuntu' does not support SELinux 2024-03-04T07:21:58.546 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-04T07:21:58.546 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-04T07:21:58.546 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-04T07:21:58.546 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-04T07:21:58.585 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-04T07:21:58.604 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-04T07:21:58.876 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-04T07:21:58.921 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-04T07:21:58.923 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi113.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-04T07:29:46.309 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi113.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2024-03-04T07:29:46.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-04T07:29:46.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T07:29:46.395 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-03-04T07:29:46.518 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-04T07:29:46.519 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-04T07:29:46.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T07:29:46.603 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-03-04T07:29:46.700 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-03-04T07:29:46.701 INFO:teuthology.run_tasks:Running task clock... 2024-03-04T07:29:46.715 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-04T07:29:46.715 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T07:29:46.715 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T07:29:46.719 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T07:29:46.719 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T07:29:46.754 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-04T07:29:46.754 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: Command line: ntpd -gq 2024-03-04T07:29:46.754 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: ---------------------------------------------------- 2024-03-04T07:29:46.754 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: ntp-4 is maintained by Network Time Foundation, 2024-03-04T07:29:46.754 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-04T07:29:46.754 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: corporation. Support and training for ntp-4 are 2024-03-04T07:29:46.754 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: available at https://www.nwtime.org/support 2024-03-04T07:29:46.755 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: ---------------------------------------------------- 2024-03-04T07:29:46.755 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: proto: precision = 0.042 usec (-24) 2024-03-04T07:29:46.755 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: basedate set to 2022-02-04 2024-03-04T07:29:46.755 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: gps base set to 2022-02-06 (week 2196) 2024-03-04T07:29:46.756 INFO:teuthology.orchestra.run.smithi113.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T07:29:46.756 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T07:29:46.756 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: restrict ::: KOD does nothing without LIMITED. 2024-03-04T07:29:46.756 INFO:teuthology.orchestra.run.smithi113.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-04T07:29:46.757 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: Listen and drop on 0 v6wildcard [::]:123 2024-03-04T07:29:46.757 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-04T07:29:46.758 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: Listen normally on 2 lo 127.0.0.1:123 2024-03-04T07:29:46.758 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: Listen normally on 3 enp3s0f1 172.21.15.113:123 2024-03-04T07:29:46.758 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: Listen normally on 4 lo [::1]:123 2024-03-04T07:29:46.758 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d7c1%5]:123 2024-03-04T07:29:46.758 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:46 ntpd[17191]: Listening on routing socket on fd #22 for interface updates 2024-03-04T07:29:46.770 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-04T07:29:46.770 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: Command line: ntpd -gq 2024-03-04T07:29:46.770 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: ---------------------------------------------------- 2024-03-04T07:29:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: ntp-4 is maintained by Network Time Foundation, 2024-03-04T07:29:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-04T07:29:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: corporation. Support and training for ntp-4 are 2024-03-04T07:29:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: available at https://www.nwtime.org/support 2024-03-04T07:29:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: ---------------------------------------------------- 2024-03-04T07:29:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: proto: precision = 0.044 usec (-24) 2024-03-04T07:29:46.771 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: basedate set to 2022-02-04 2024-03-04T07:29:46.772 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: gps base set to 2022-02-06 (week 2196) 2024-03-04T07:29:46.772 INFO:teuthology.orchestra.run.smithi136.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T07:29:46.772 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T07:29:46.772 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: restrict ::: KOD does nothing without LIMITED. 2024-03-04T07:29:46.773 INFO:teuthology.orchestra.run.smithi136.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-04T07:29:46.773 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: Listen and drop on 0 v6wildcard [::]:123 2024-03-04T07:29:46.774 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-04T07:29:46.774 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: Listen normally on 2 lo 127.0.0.1:123 2024-03-04T07:29:46.774 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: Listen normally on 3 enp3s0f1 172.21.15.136:123 2024-03-04T07:29:46.774 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: Listen normally on 4 lo [::1]:123 2024-03-04T07:29:46.774 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e43%5]:123 2024-03-04T07:29:46.774 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:46 ntpd[17601]: Listening on routing socket on fd #22 for interface updates 2024-03-04T07:29:47.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:47 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:47.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:47 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:47.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:47 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:47.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:47 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:47.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:47 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:47.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:47 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:47.771 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:47 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:47.771 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:47 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:49.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:49 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:49.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:49 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:49.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:49 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:49.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:49 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:49.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:49 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:49.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:49 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:49.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:49 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:49.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:49 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:49.771 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:49 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:49.771 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:49 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:49.771 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:49 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:49.771 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:49 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:49.771 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:49 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:49.771 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:49 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:50.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:50 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:50.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:50 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:50.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:50 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:50.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:50 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:50.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:50 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:50.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:50 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:51.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:51.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:51.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:51.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:51.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:51.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:51.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:51.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:51.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:51.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:51.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:51.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:51 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:51.772 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:51.772 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:51.772 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:51.772 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:51.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:51 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:52.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:52 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:52.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:52 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:52.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:52 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:52.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:52 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:52.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:52 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:52.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:52 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:53.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:53.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:53.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:53.755 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:53.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:53.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:53.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:53.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:53.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:53.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:53.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:53.756 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:53.756 INFO:teuthology.orchestra.run.smithi113.stdout: 4 Mar 07:29:53 ntpd[17191]: ntpd: time slew +0.000426 s 2024-03-04T07:29:53.756 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd: time slew +0.000426s 2024-03-04T07:29:53.757 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-04T07:29:53.757 INFO:teuthology.orchestra.run.smithi113.stderr: 4 Mar 07:29:53 ntpd[17191]: can't open /var/log/ntpstats/loopstats.20240304: Permission denied 2024-03-04T07:29:53.772 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:53.772 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:53.772 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:53.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:53.774 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:53.774 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:53 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:53.858 INFO:teuthology.orchestra.run.smithi113.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T07:29:53.859 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================== 2024-03-04T07:29:53.859 INFO:teuthology.orchestra.run.smithi113.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T07:29:53.859 INFO:teuthology.orchestra.run.smithi113.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T07:29:53.859 INFO:teuthology.orchestra.run.smithi113.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T07:29:53.859 INFO:teuthology.orchestra.run.smithi113.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T07:29:55.772 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:55.772 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T07:29:55.773 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T07:29:55.774 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T07:29:55.774 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T07:29:55.774 INFO:teuthology.orchestra.run.smithi136.stdout: 4 Mar 07:29:55 ntpd[17601]: ntpd: time slew -0.006851 s 2024-03-04T07:29:55.774 INFO:teuthology.orchestra.run.smithi136.stdout:ntpd: time slew -0.006851s 2024-03-04T07:29:55.774 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-04T07:29:55.775 INFO:teuthology.orchestra.run.smithi136.stderr: 4 Mar 07:29:55 ntpd[17601]: can't open /var/log/ntpstats/loopstats.20240304: Permission denied 2024-03-04T07:29:55.920 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T07:29:55.920 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2024-03-04T07:29:55.920 INFO:teuthology.orchestra.run.smithi136.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T07:29:55.920 INFO:teuthology.orchestra.run.smithi136.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T07:29:55.920 INFO:teuthology.orchestra.run.smithi136.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T07:29:55.920 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T07:29:55.921 INFO:teuthology.run_tasks:Running task install... 2024-03-04T07:29:55.932 DEBUG:teuthology.task.install:project ceph 2024-03-04T07:29:55.932 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}} 2024-03-04T07:29:55.932 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'} 2024-03-04T07:29:55.933 INFO:teuthology.task.install:Using flavor: default 2024-03-04T07:29:55.938 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-04T07:29:55.938 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-04T07:29:55.939 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-key list | grep Ceph 2024-03-04T07:29:55.939 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-key list | grep Ceph 2024-03-04T07:29:56.030 INFO:teuthology.orchestra.run.smithi136.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-04T07:29:56.042 INFO:teuthology.orchestra.run.smithi113.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-04T07:29:56.056 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-04T07:29:56.056 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph.com (release key) 2024-03-04T07:29:56.057 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-04T07:29:56.057 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-04T07:29:56.058 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T07:29:56.093 INFO:teuthology.orchestra.run.smithi113.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-04T07:29:56.094 INFO:teuthology.orchestra.run.smithi113.stdout:uid [ unknown] Ceph.com (release key) 2024-03-04T07:29:56.095 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-04T07:29:56.095 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-04T07:29:56.095 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T07:29:56.277 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default/ 2024-03-04T07:29:56.277 INFO:teuthology.task.install.deb:Package version is 19.0.0-1739-g1bc250c7-1jammy 2024-03-04T07:29:56.311 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default/ 2024-03-04T07:29:56.311 INFO:teuthology.task.install.deb:Package version is 19.0.0-1739-g1bc250c7-1jammy 2024-03-04T07:29:56.451 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:29:56.451 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-04T07:29:56.466 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-03-04T07:29:56.483 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:29:56.483 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-04T07:29:56.502 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get update 2024-03-04T07:29:56.693 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T07:29:56.701 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T07:29:56.721 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T07:29:56.759 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T07:29:56.759 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy InRelease 2024-03-04T07:29:56.772 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T07:29:56.796 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-04T07:29:56.798 INFO:teuthology.orchestra.run.smithi113.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy InRelease 2024-03-04T07:29:56.806 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T07:29:56.831 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T07:29:56.833 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-04T07:29:56.835 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-04T07:29:56.870 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-03-04T07:29:56.872 INFO:teuthology.orchestra.run.smithi113.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-04T07:29:56.908 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-03-04T07:29:56.913 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T07:29:56.913 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-04T07:29:56.955 INFO:teuthology.orchestra.run.smithi113.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-04T07:29:57.126 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 28.9 kB in 0s (59.8 kB/s) 2024-03-04T07:29:57.275 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 28.9 kB in 1s (52.6 kB/s) 2024-03-04T07:29:58.052 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-03-04T07:29:58.067 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1739-g1bc250c7-1jammy cephadm=19.0.0-1739-g1bc250c7-1jammy ceph-mds=19.0.0-1739-g1bc250c7-1jammy ceph-mgr=19.0.0-1739-g1bc250c7-1jammy ceph-common=19.0.0-1739-g1bc250c7-1jammy ceph-fuse=19.0.0-1739-g1bc250c7-1jammy ceph-test=19.0.0-1739-g1bc250c7-1jammy ceph-volume=19.0.0-1739-g1bc250c7-1jammy radosgw=19.0.0-1739-g1bc250c7-1jammy python3-rados=19.0.0-1739-g1bc250c7-1jammy python3-rgw=19.0.0-1739-g1bc250c7-1jammy python3-cephfs=19.0.0-1739-g1bc250c7-1jammy python3-rbd=19.0.0-1739-g1bc250c7-1jammy libcephfs2=19.0.0-1739-g1bc250c7-1jammy libcephfs-dev=19.0.0-1739-g1bc250c7-1jammy librados2=19.0.0-1739-g1bc250c7-1jammy librbd1=19.0.0-1739-g1bc250c7-1jammy rbd-fuse=19.0.0-1739-g1bc250c7-1jammy python3-cephfs=19.0.0-1739-g1bc250c7-1jammy cephfs-shell=19.0.0-1739-g1bc250c7-1jammy cephfs-top=19.0.0-1739-g1bc250c7-1jammy cephfs-mirror=19.0.0-1739-g1bc250c7-1jammy 2024-03-04T07:29:58.128 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-03-04T07:29:58.214 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-03-04T07:29:58.229 DEBUG:teuthology.orchestra.run.smithi113:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1739-g1bc250c7-1jammy cephadm=19.0.0-1739-g1bc250c7-1jammy ceph-mds=19.0.0-1739-g1bc250c7-1jammy ceph-mgr=19.0.0-1739-g1bc250c7-1jammy ceph-common=19.0.0-1739-g1bc250c7-1jammy ceph-fuse=19.0.0-1739-g1bc250c7-1jammy ceph-test=19.0.0-1739-g1bc250c7-1jammy ceph-volume=19.0.0-1739-g1bc250c7-1jammy radosgw=19.0.0-1739-g1bc250c7-1jammy python3-rados=19.0.0-1739-g1bc250c7-1jammy python3-rgw=19.0.0-1739-g1bc250c7-1jammy python3-cephfs=19.0.0-1739-g1bc250c7-1jammy python3-rbd=19.0.0-1739-g1bc250c7-1jammy libcephfs2=19.0.0-1739-g1bc250c7-1jammy libcephfs-dev=19.0.0-1739-g1bc250c7-1jammy librados2=19.0.0-1739-g1bc250c7-1jammy librbd1=19.0.0-1739-g1bc250c7-1jammy rbd-fuse=19.0.0-1739-g1bc250c7-1jammy python3-cephfs=19.0.0-1739-g1bc250c7-1jammy cephfs-shell=19.0.0-1739-g1bc250c7-1jammy cephfs-top=19.0.0-1739-g1bc250c7-1jammy cephfs-mirror=19.0.0-1739-g1bc250c7-1jammy 2024-03-04T07:29:58.302 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-03-04T07:29:58.321 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-03-04T07:29:58.322 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-03-04T07:29:58.469 INFO:teuthology.orchestra.run.smithi113.stdout:Building dependency tree... 2024-03-04T07:29:58.470 INFO:teuthology.orchestra.run.smithi113.stdout:Reading state information... 2024-03-04T07:29:58.499 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T07:29:58.500 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T07:29:58.500 INFO:teuthology.orchestra.run.smithi136.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T07:29:58.500 INFO:teuthology.orchestra.run.smithi136.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T07:29:58.500 INFO:teuthology.orchestra.run.smithi136.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-04T07:29:58.501 INFO:teuthology.orchestra.run.smithi136.stdout: python2.7-minimal 2024-03-04T07:29:58.501 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T07:29:58.501 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2024-03-04T07:29:58.502 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-04T07:29:58.502 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-04T07:29:58.502 INFO:teuthology.orchestra.run.smithi136.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-04T07:29:58.502 INFO:teuthology.orchestra.run.smithi136.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-04T07:29:58.503 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-04T07:29:58.503 INFO:teuthology.orchestra.run.smithi136.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-04T07:29:58.503 INFO:teuthology.orchestra.run.smithi136.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-04T07:29:58.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-04T07:29:58.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-04T07:29:58.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-04T07:29:58.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-04T07:29:58.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-04T07:29:58.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-04T07:29:58.503 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-04T07:29:58.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-04T07:29:58.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-04T07:29:58.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-04T07:29:58.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-04T07:29:58.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-04T07:29:58.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-04T07:29:58.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-04T07:29:58.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-04T07:29:58.504 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-04T07:29:58.504 INFO:teuthology.orchestra.run.smithi136.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-04T07:29:58.505 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2024-03-04T07:29:58.505 INFO:teuthology.orchestra.run.smithi136.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-04T07:29:58.505 INFO:teuthology.orchestra.run.smithi136.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-04T07:29:58.505 INFO:teuthology.orchestra.run.smithi136.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-04T07:29:58.505 INFO:teuthology.orchestra.run.smithi136.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-04T07:29:58.505 INFO:teuthology.orchestra.run.smithi136.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-04T07:29:58.505 INFO:teuthology.orchestra.run.smithi136.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-04T07:29:58.505 INFO:teuthology.orchestra.run.smithi136.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-04T07:29:58.505 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2024-03-04T07:29:58.505 INFO:teuthology.orchestra.run.smithi136.stdout: btrfs-tools 2024-03-04T07:29:58.613 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2024-03-04T07:29:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-04T07:29:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-04T07:29:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-04T07:29:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-04T07:29:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-04T07:29:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-04T07:29:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-04T07:29:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-04T07:29:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-04T07:29:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-04T07:29:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-04T07:29:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-04T07:29:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-04T07:29:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-04T07:29:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: socat unzip xmlstarlet zip 2024-03-04T07:29:58.625 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T07:29:58.626 INFO:teuthology.orchestra.run.smithi113.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T07:29:58.626 INFO:teuthology.orchestra.run.smithi113.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T07:29:58.626 INFO:teuthology.orchestra.run.smithi113.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T07:29:58.626 INFO:teuthology.orchestra.run.smithi113.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T07:29:58.626 INFO:teuthology.orchestra.run.smithi113.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T07:29:58.627 INFO:teuthology.orchestra.run.smithi113.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T07:29:58.627 INFO:teuthology.orchestra.run.smithi113.stdout:The following additional packages will be installed: 2024-03-04T07:29:58.627 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-04T07:29:58.627 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-04T07:29:58.628 INFO:teuthology.orchestra.run.smithi113.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-04T07:29:58.628 INFO:teuthology.orchestra.run.smithi113.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-04T07:29:58.628 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-04T07:29:58.628 INFO:teuthology.orchestra.run.smithi113.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-04T07:29:58.629 INFO:teuthology.orchestra.run.smithi113.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-04T07:29:58.630 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-04T07:29:58.630 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-04T07:29:58.630 INFO:teuthology.orchestra.run.smithi113.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-04T07:29:58.630 INFO:teuthology.orchestra.run.smithi113.stdout:Suggested packages: 2024-03-04T07:29:58.630 INFO:teuthology.orchestra.run.smithi113.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-04T07:29:58.630 INFO:teuthology.orchestra.run.smithi113.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-04T07:29:58.630 INFO:teuthology.orchestra.run.smithi113.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-04T07:29:58.630 INFO:teuthology.orchestra.run.smithi113.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-04T07:29:58.630 INFO:teuthology.orchestra.run.smithi113.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-04T07:29:58.630 INFO:teuthology.orchestra.run.smithi113.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-04T07:29:58.631 INFO:teuthology.orchestra.run.smithi113.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-04T07:29:58.631 INFO:teuthology.orchestra.run.smithi113.stdout:Recommended packages: 2024-03-04T07:29:58.631 INFO:teuthology.orchestra.run.smithi113.stdout: btrfs-tools 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout:The following NEW packages will be installed: 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-04T07:29:58.780 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-04T07:29:58.781 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-04T07:29:58.782 INFO:teuthology.orchestra.run.smithi113.stdout: socat unzip xmlstarlet zip 2024-03-04T07:29:58.801 INFO:teuthology.orchestra.run.smithi136.stdout:0 upgraded, 124 newly installed, 0 to remove and 278 not upgraded. 2024-03-04T07:29:58.801 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 177 MB of archives. 2024-03-04T07:29:58.801 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 735 MB of additional disk space will be used. 2024-03-04T07:29:58.801 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-04T07:29:58.831 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1739-g1bc250c7-1jammy [3,555 kB] 2024-03-04T07:29:58.857 INFO:teuthology.orchestra.run.smithi113.stdout:0 upgraded, 124 newly installed, 0 to remove and 278 not upgraded. 2024-03-04T07:29:58.858 INFO:teuthology.orchestra.run.smithi113.stdout:Need to get 177 MB of archives. 2024-03-04T07:29:58.858 INFO:teuthology.orchestra.run.smithi113.stdout:After this operation, 735 MB of additional disk space will be used. 2024-03-04T07:29:58.858 INFO:teuthology.orchestra.run.smithi113.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-04T07:29:58.878 INFO:teuthology.orchestra.run.smithi113.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-04T07:29:58.883 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-04T07:29:58.889 INFO:teuthology.orchestra.run.smithi113.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-04T07:29:58.929 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-04T07:29:58.936 INFO:teuthology.orchestra.run.smithi113.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-04T07:29:58.960 INFO:teuthology.orchestra.run.smithi113.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-04T07:29:58.962 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-04T07:29:58.977 INFO:teuthology.orchestra.run.smithi113.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-04T07:29:59.003 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1739-g1bc250c7-1jammy [3,555 kB] 2024-03-04T07:29:59.023 INFO:teuthology.orchestra.run.smithi113.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-04T07:29:59.024 INFO:teuthology.orchestra.run.smithi113.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-04T07:29:59.032 INFO:teuthology.orchestra.run.smithi113.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-04T07:29:59.035 INFO:teuthology.orchestra.run.smithi113.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-04T07:29:59.036 INFO:teuthology.orchestra.run.smithi113.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-04T07:29:59.043 INFO:teuthology.orchestra.run.smithi113.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-04T07:29:59.045 INFO:teuthology.orchestra.run.smithi113.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-04T07:29:59.047 INFO:teuthology.orchestra.run.smithi113.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-04T07:29:59.048 INFO:teuthology.orchestra.run.smithi113.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-04T07:29:59.048 INFO:teuthology.orchestra.run.smithi113.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-04T07:29:59.064 INFO:teuthology.orchestra.run.smithi113.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-04T07:29:59.066 INFO:teuthology.orchestra.run.smithi113.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-04T07:29:59.068 INFO:teuthology.orchestra.run.smithi113.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-04T07:29:59.068 INFO:teuthology.orchestra.run.smithi113.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-04T07:29:59.084 INFO:teuthology.orchestra.run.smithi113.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-04T07:29:59.085 INFO:teuthology.orchestra.run.smithi113.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-04T07:29:59.085 INFO:teuthology.orchestra.run.smithi113.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-04T07:29:59.086 INFO:teuthology.orchestra.run.smithi113.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-04T07:29:59.087 INFO:teuthology.orchestra.run.smithi113.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-04T07:29:59.087 INFO:teuthology.orchestra.run.smithi113.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-04T07:29:59.104 INFO:teuthology.orchestra.run.smithi113.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-04T07:29:59.104 INFO:teuthology.orchestra.run.smithi113.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-04T07:29:59.106 INFO:teuthology.orchestra.run.smithi113.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-04T07:29:59.107 INFO:teuthology.orchestra.run.smithi113.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-04T07:29:59.123 INFO:teuthology.orchestra.run.smithi113.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-04T07:29:59.123 INFO:teuthology.orchestra.run.smithi113.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-04T07:29:59.124 INFO:teuthology.orchestra.run.smithi113.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-04T07:29:59.125 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-04T07:29:59.125 INFO:teuthology.orchestra.run.smithi113.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-04T07:29:59.126 INFO:teuthology.orchestra.run.smithi113.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-04T07:29:59.127 INFO:teuthology.orchestra.run.smithi113.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-04T07:29:59.142 INFO:teuthology.orchestra.run.smithi113.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-04T07:29:59.143 INFO:teuthology.orchestra.run.smithi113.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-04T07:29:59.148 INFO:teuthology.orchestra.run.smithi113.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-04T07:29:59.148 INFO:teuthology.orchestra.run.smithi113.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-04T07:29:59.162 INFO:teuthology.orchestra.run.smithi113.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-04T07:29:59.163 INFO:teuthology.orchestra.run.smithi113.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-04T07:29:59.164 INFO:teuthology.orchestra.run.smithi113.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-04T07:29:59.166 INFO:teuthology.orchestra.run.smithi113.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-04T07:29:59.167 INFO:teuthology.orchestra.run.smithi113.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-04T07:29:59.168 INFO:teuthology.orchestra.run.smithi113.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-04T07:29:59.182 INFO:teuthology.orchestra.run.smithi113.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-04T07:29:59.184 INFO:teuthology.orchestra.run.smithi113.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-04T07:29:59.185 INFO:teuthology.orchestra.run.smithi113.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-04T07:29:59.186 INFO:teuthology.orchestra.run.smithi113.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-04T07:29:59.201 INFO:teuthology.orchestra.run.smithi113.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-04T07:29:59.223 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-04T07:29:59.236 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-04T07:29:59.255 INFO:teuthology.orchestra.run.smithi113.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-04T07:29:59.256 INFO:teuthology.orchestra.run.smithi113.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-04T07:29:59.257 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1739-g1bc250c7-1jammy [3,217 kB] 2024-03-04T07:29:59.257 INFO:teuthology.orchestra.run.smithi113.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-04T07:29:59.258 INFO:teuthology.orchestra.run.smithi113.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-04T07:29:59.259 INFO:teuthology.orchestra.run.smithi113.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-04T07:29:59.282 INFO:teuthology.orchestra.run.smithi113.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-04T07:29:59.284 INFO:teuthology.orchestra.run.smithi113.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-04T07:29:59.285 INFO:teuthology.orchestra.run.smithi113.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-04T07:29:59.288 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-04T07:29:59.306 INFO:teuthology.orchestra.run.smithi113.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-04T07:29:59.307 INFO:teuthology.orchestra.run.smithi113.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-04T07:29:59.315 INFO:teuthology.orchestra.run.smithi113.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-04T07:29:59.315 INFO:teuthology.orchestra.run.smithi113.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-04T07:29:59.316 INFO:teuthology.orchestra.run.smithi113.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-04T07:29:59.317 INFO:teuthology.orchestra.run.smithi113.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-04T07:29:59.317 INFO:teuthology.orchestra.run.smithi113.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-04T07:29:59.319 INFO:teuthology.orchestra.run.smithi113.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-04T07:29:59.322 INFO:teuthology.orchestra.run.smithi113.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-04T07:29:59.333 INFO:teuthology.orchestra.run.smithi113.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-04T07:29:59.335 INFO:teuthology.orchestra.run.smithi113.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-04T07:29:59.336 INFO:teuthology.orchestra.run.smithi113.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-04T07:29:59.353 INFO:teuthology.orchestra.run.smithi113.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-04T07:29:59.357 INFO:teuthology.orchestra.run.smithi113.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-04T07:29:59.357 INFO:teuthology.orchestra.run.smithi113.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-04T07:29:59.359 INFO:teuthology.orchestra.run.smithi113.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-04T07:29:59.359 INFO:teuthology.orchestra.run.smithi113.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-04T07:29:59.365 INFO:teuthology.orchestra.run.smithi113.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-04T07:29:59.365 INFO:teuthology.orchestra.run.smithi113.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-04T07:29:59.372 INFO:teuthology.orchestra.run.smithi113.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-04T07:29:59.373 INFO:teuthology.orchestra.run.smithi113.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-04T07:29:59.374 INFO:teuthology.orchestra.run.smithi113.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-04T07:29:59.392 INFO:teuthology.orchestra.run.smithi113.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-04T07:29:59.394 INFO:teuthology.orchestra.run.smithi113.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-04T07:29:59.395 INFO:teuthology.orchestra.run.smithi113.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-04T07:29:59.403 INFO:teuthology.orchestra.run.smithi113.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-04T07:29:59.403 INFO:teuthology.orchestra.run.smithi113.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-04T07:29:59.403 INFO:teuthology.orchestra.run.smithi113.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-04T07:29:59.406 INFO:teuthology.orchestra.run.smithi113.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-04T07:29:59.412 INFO:teuthology.orchestra.run.smithi113.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-04T07:29:59.461 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1739-g1bc250c7-1jammy [711 kB] 2024-03-04T07:29:59.462 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-04T07:29:59.471 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-04T07:29:59.480 INFO:teuthology.orchestra.run.smithi113.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1739-g1bc250c7-1jammy [3,217 kB] 2024-03-04T07:29:59.497 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1739-g1bc250c7-1jammy [356 kB] 2024-03-04T07:29:59.499 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-04T07:29:59.508 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-04T07:29:59.508 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1739-g1bc250c7-1jammy [32.8 kB] 2024-03-04T07:29:59.510 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1739-g1bc250c7-1jammy [180 kB] 2024-03-04T07:29:59.533 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1739-g1bc250c7-1jammy [64.8 kB] 2024-03-04T07:29:59.535 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1739-g1bc250c7-1jammy [330 kB] 2024-03-04T07:29:59.546 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1739-g1bc250c7-1jammy [6,573 kB] 2024-03-04T07:29:59.672 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-04T07:29:59.710 INFO:teuthology.orchestra.run.smithi113.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1739-g1bc250c7-1jammy [711 kB] 2024-03-04T07:29:59.764 INFO:teuthology.orchestra.run.smithi113.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1739-g1bc250c7-1jammy [356 kB] 2024-03-04T07:29:59.775 INFO:teuthology.orchestra.run.smithi113.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1739-g1bc250c7-1jammy [32.8 kB] 2024-03-04T07:29:59.776 INFO:teuthology.orchestra.run.smithi113.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1739-g1bc250c7-1jammy [180 kB] 2024-03-04T07:29:59.815 INFO:teuthology.orchestra.run.smithi113.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1739-g1bc250c7-1jammy [64.8 kB] 2024-03-04T07:29:59.817 INFO:teuthology.orchestra.run.smithi113.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1739-g1bc250c7-1jammy [330 kB] 2024-03-04T07:29:59.829 INFO:teuthology.orchestra.run.smithi113.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1739-g1bc250c7-1jammy [6,573 kB] 2024-03-04T07:29:59.926 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1739-g1bc250c7-1jammy [112 kB] 2024-03-04T07:29:59.930 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1739-g1bc250c7-1jammy [466 kB] 2024-03-04T07:29:59.950 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1739-g1bc250c7-1jammy [24.9 MB] 2024-03-04T07:30:00.242 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-04T07:30:00.257 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-04T07:30:00.273 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-04T07:30:00.281 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-04T07:30:00.282 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-04T07:30:00.298 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-04T07:30:00.336 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-04T07:30:00.348 INFO:teuthology.orchestra.run.smithi113.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1739-g1bc250c7-1jammy [112 kB] 2024-03-04T07:30:00.349 INFO:teuthology.orchestra.run.smithi113.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1739-g1bc250c7-1jammy [466 kB] 2024-03-04T07:30:00.349 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-04T07:30:00.354 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-04T07:30:00.354 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-04T07:30:00.382 INFO:teuthology.orchestra.run.smithi113.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1739-g1bc250c7-1jammy [24.9 MB] 2024-03-04T07:30:00.524 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-04T07:30:00.607 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-04T07:30:00.614 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-04T07:30:00.689 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-04T07:30:00.715 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-04T07:30:00.730 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-04T07:30:00.748 INFO:teuthology.orchestra.run.smithi136.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-04T07:30:00.937 INFO:teuthology.orchestra.run.smithi136.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-04T07:30:00.953 INFO:teuthology.orchestra.run.smithi136.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-04T07:30:00.960 INFO:teuthology.orchestra.run.smithi136.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-04T07:30:00.965 INFO:teuthology.orchestra.run.smithi136.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-04T07:30:01.150 INFO:teuthology.orchestra.run.smithi136.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-04T07:30:01.233 INFO:teuthology.orchestra.run.smithi136.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-04T07:30:01.248 INFO:teuthology.orchestra.run.smithi136.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-04T07:30:01.434 INFO:teuthology.orchestra.run.smithi136.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-04T07:30:01.485 INFO:teuthology.orchestra.run.smithi136.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-04T07:30:01.498 INFO:teuthology.orchestra.run.smithi136.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-04T07:30:01.657 INFO:teuthology.orchestra.run.smithi136.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-04T07:30:01.657 INFO:teuthology.orchestra.run.smithi136.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-04T07:30:01.662 INFO:teuthology.orchestra.run.smithi136.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-04T07:30:01.667 INFO:teuthology.orchestra.run.smithi136.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-04T07:30:01.682 INFO:teuthology.orchestra.run.smithi136.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-04T07:30:01.831 INFO:teuthology.orchestra.run.smithi136.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1739-g1bc250c7-1jammy [5,078 kB] 2024-03-04T07:30:02.235 INFO:teuthology.orchestra.run.smithi136.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-04T07:30:02.312 INFO:teuthology.orchestra.run.smithi136.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1739-g1bc250c7-1jammy [244 kB] 2024-03-04T07:30:02.338 INFO:teuthology.orchestra.run.smithi136.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1739-g1bc250c7-1jammy [124 kB] 2024-03-04T07:30:02.347 INFO:teuthology.orchestra.run.smithi136.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1739-g1bc250c7-1jammy [1,524 kB] 2024-03-04T07:30:02.499 INFO:teuthology.orchestra.run.smithi136.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-04T07:30:02.547 INFO:teuthology.orchestra.run.smithi136.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1739-g1bc250c7-1jammy [6,241 kB] 2024-03-04T07:30:02.616 INFO:teuthology.orchestra.run.smithi136.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-04T07:30:02.660 INFO:teuthology.orchestra.run.smithi136.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-04T07:30:02.743 INFO:teuthology.orchestra.run.smithi136.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-04T07:30:02.754 INFO:teuthology.orchestra.run.smithi136.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-04T07:30:02.767 INFO:teuthology.orchestra.run.smithi136.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-04T07:30:02.808 INFO:teuthology.orchestra.run.smithi136.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-04T07:30:03.070 INFO:teuthology.orchestra.run.smithi136.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-04T07:30:03.071 INFO:teuthology.orchestra.run.smithi136.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-04T07:30:03.074 INFO:teuthology.orchestra.run.smithi136.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-04T07:30:03.341 INFO:teuthology.orchestra.run.smithi136.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-04T07:30:03.344 INFO:teuthology.orchestra.run.smithi136.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1739-g1bc250c7-1jammy [22.7 MB] 2024-03-04T07:30:03.516 INFO:teuthology.orchestra.run.smithi113.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1739-g1bc250c7-1jammy [5,078 kB] 2024-03-04T07:30:03.676 INFO:teuthology.orchestra.run.smithi136.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-04T07:30:04.073 INFO:teuthology.orchestra.run.smithi136.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-04T07:30:04.082 INFO:teuthology.orchestra.run.smithi136.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-04T07:30:04.083 INFO:teuthology.orchestra.run.smithi136.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-04T07:30:04.152 INFO:teuthology.orchestra.run.smithi136.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-04T07:30:04.153 INFO:teuthology.orchestra.run.smithi136.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-04T07:30:04.154 INFO:teuthology.orchestra.run.smithi136.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-04T07:30:04.156 INFO:teuthology.orchestra.run.smithi136.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-04T07:30:04.157 INFO:teuthology.orchestra.run.smithi136.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-04T07:30:04.262 INFO:teuthology.orchestra.run.smithi113.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1739-g1bc250c7-1jammy [244 kB] 2024-03-04T07:30:04.287 INFO:teuthology.orchestra.run.smithi113.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1739-g1bc250c7-1jammy [124 kB] 2024-03-04T07:30:04.295 INFO:teuthology.orchestra.run.smithi113.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1739-g1bc250c7-1jammy [1,524 kB] 2024-03-04T07:30:04.323 INFO:teuthology.orchestra.run.smithi136.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-04T07:30:04.419 INFO:teuthology.orchestra.run.smithi136.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-04T07:30:04.513 INFO:teuthology.orchestra.run.smithi136.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-04T07:30:04.541 INFO:teuthology.orchestra.run.smithi113.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1739-g1bc250c7-1jammy [6,241 kB] 2024-03-04T07:30:04.759 INFO:teuthology.orchestra.run.smithi136.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-04T07:30:04.802 INFO:teuthology.orchestra.run.smithi136.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-04T07:30:04.826 INFO:teuthology.orchestra.run.smithi136.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-04T07:30:04.835 INFO:teuthology.orchestra.run.smithi136.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-04T07:30:04.878 INFO:teuthology.orchestra.run.smithi136.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-04T07:30:04.904 INFO:teuthology.orchestra.run.smithi136.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-04T07:30:04.906 INFO:teuthology.orchestra.run.smithi136.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-04T07:30:04.912 INFO:teuthology.orchestra.run.smithi136.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-04T07:30:05.079 INFO:teuthology.orchestra.run.smithi136.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-04T07:30:05.509 INFO:teuthology.orchestra.run.smithi113.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1739-g1bc250c7-1jammy [22.7 MB] 2024-03-04T07:30:05.611 INFO:teuthology.orchestra.run.smithi136.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-04T07:30:05.616 INFO:teuthology.orchestra.run.smithi136.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-04T07:30:05.656 INFO:teuthology.orchestra.run.smithi136.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-04T07:30:05.656 INFO:teuthology.orchestra.run.smithi136.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-04T07:30:05.663 INFO:teuthology.orchestra.run.smithi136.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-04T07:30:05.666 INFO:teuthology.orchestra.run.smithi136.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-04T07:30:05.707 INFO:teuthology.orchestra.run.smithi136.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-04T07:30:05.716 INFO:teuthology.orchestra.run.smithi136.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-04T07:30:05.789 INFO:teuthology.orchestra.run.smithi136.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-04T07:30:05.789 INFO:teuthology.orchestra.run.smithi136.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-04T07:30:05.955 INFO:teuthology.orchestra.run.smithi136.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-04T07:30:06.150 INFO:teuthology.orchestra.run.smithi136.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1739-g1bc250c7-1jammy [14.0 kB] 2024-03-04T07:30:06.151 INFO:teuthology.orchestra.run.smithi136.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1739-g1bc250c7-1jammy [869 kB] 2024-03-04T07:30:06.225 INFO:teuthology.orchestra.run.smithi136.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1739-g1bc250c7-1jammy [2,384 kB] 2024-03-04T07:30:06.399 INFO:teuthology.orchestra.run.smithi136.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-04T07:30:06.412 INFO:teuthology.orchestra.run.smithi136.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-04T07:30:06.555 INFO:teuthology.orchestra.run.smithi136.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1739-g1bc250c7-1jammy [776 kB] 2024-03-04T07:30:06.667 INFO:teuthology.orchestra.run.smithi136.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1739-g1bc250c7-1jammy [144 kB] 2024-03-04T07:30:06.710 INFO:teuthology.orchestra.run.smithi136.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1739-g1bc250c7-1jammy [1,921 kB] 2024-03-04T07:30:06.968 INFO:teuthology.orchestra.run.smithi136.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1739-g1bc250c7-1jammy [8,625 kB] 2024-03-04T07:30:08.304 INFO:teuthology.orchestra.run.smithi136.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1739-g1bc250c7-1jammy [14.2 kB] 2024-03-04T07:30:08.304 INFO:teuthology.orchestra.run.smithi136.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1739-g1bc250c7-1jammy [49.9 MB] 2024-03-04T07:30:08.803 INFO:teuthology.orchestra.run.smithi113.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1739-g1bc250c7-1jammy [14.0 kB] 2024-03-04T07:30:08.803 INFO:teuthology.orchestra.run.smithi113.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1739-g1bc250c7-1jammy [869 kB] 2024-03-04T07:30:08.951 INFO:teuthology.orchestra.run.smithi113.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1739-g1bc250c7-1jammy [2,384 kB] 2024-03-04T07:30:09.308 INFO:teuthology.orchestra.run.smithi113.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1739-g1bc250c7-1jammy [776 kB] 2024-03-04T07:30:09.406 INFO:teuthology.orchestra.run.smithi113.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1739-g1bc250c7-1jammy [144 kB] 2024-03-04T07:30:09.420 INFO:teuthology.orchestra.run.smithi113.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1739-g1bc250c7-1jammy [1,921 kB] 2024-03-04T07:30:09.731 INFO:teuthology.orchestra.run.smithi113.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1739-g1bc250c7-1jammy [8,625 kB] 2024-03-04T07:30:11.137 INFO:teuthology.orchestra.run.smithi113.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1739-g1bc250c7-1jammy [14.2 kB] 2024-03-04T07:30:11.155 INFO:teuthology.orchestra.run.smithi113.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1739-g1bc250c7-1jammy [49.9 MB] 2024-03-04T07:30:19.852 INFO:teuthology.orchestra.run.smithi136.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1739-g1bc250c7-1jammy [121 kB] 2024-03-04T07:30:19.854 INFO:teuthology.orchestra.run.smithi136.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1739-g1bc250c7-1jammy [228 kB] 2024-03-04T07:30:19.859 INFO:teuthology.orchestra.run.smithi136.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1739-g1bc250c7-1jammy [34.0 kB] 2024-03-04T07:30:19.860 INFO:teuthology.orchestra.run.smithi136.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1739-g1bc250c7-1jammy [23.4 kB] 2024-03-04T07:30:19.860 INFO:teuthology.orchestra.run.smithi136.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1739-g1bc250c7-1jammy [38.5 kB] 2024-03-04T07:30:19.862 INFO:teuthology.orchestra.run.smithi136.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1739-g1bc250c7-1jammy [13.0 MB] 2024-03-04T07:30:20.386 INFO:teuthology.orchestra.run.smithi113.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1739-g1bc250c7-1jammy [121 kB] 2024-03-04T07:30:20.399 INFO:teuthology.orchestra.run.smithi113.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1739-g1bc250c7-1jammy [228 kB] 2024-03-04T07:30:20.413 INFO:teuthology.orchestra.run.smithi113.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1739-g1bc250c7-1jammy [34.0 kB] 2024-03-04T07:30:20.431 INFO:teuthology.orchestra.run.smithi113.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1739-g1bc250c7-1jammy [23.4 kB] 2024-03-04T07:30:20.454 INFO:teuthology.orchestra.run.smithi113.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1739-g1bc250c7-1jammy [38.5 kB] 2024-03-04T07:30:20.455 INFO:teuthology.orchestra.run.smithi113.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1739-g1bc250c7-1jammy [13.0 MB] 2024-03-04T07:30:23.030 INFO:teuthology.orchestra.run.smithi136.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1739-g1bc250c7-1jammy [91.3 kB] 2024-03-04T07:30:23.125 INFO:teuthology.orchestra.run.smithi113.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-lusov-qdb/1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1739-g1bc250c7-1jammy [91.3 kB] 2024-03-04T07:30:23.644 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 177 MB in 24s (7,268 kB/s) 2024-03-04T07:30:23.760 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 177 MB in 24s (7,289 kB/s) 2024-03-04T07:30:23.785 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-04T07:30:23.868 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 130504 files and directories currently installed.) 2024-03-04T07:30:23.873 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-04T07:30:23.916 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-04T07:30:23.918 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-04T07:30:23.984 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132923 files and directories currently installed.) 2024-03-04T07:30:23.988 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-04T07:30:24.042 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-04T07:30:24.252 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-04T07:30:24.273 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-04T07:30:24.311 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T07:30:24.345 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-04T07:30:24.358 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-04T07:30:24.403 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T07:30:24.605 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cmd2. 2024-03-04T07:30:24.626 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-04T07:30:24.663 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-04T07:30:24.723 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cmd2. 2024-03-04T07:30:24.736 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-04T07:30:24.797 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-04T07:30:24.983 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-04T07:30:25.004 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-04T07:30:25.083 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T07:30:25.151 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-04T07:30:25.164 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-04T07:30:25.251 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T07:30:25.420 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-04T07:30:25.441 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-04T07:30:25.478 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T07:30:25.638 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-04T07:30:25.651 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-04T07:30:25.696 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T07:30:25.815 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-04T07:30:25.836 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-04T07:30:25.873 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T07:30:25.982 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-04T07:30:25.997 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-04T07:30:26.032 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T07:30:26.310 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-04T07:30:26.331 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T07:30:26.376 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:30:26.536 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-04T07:30:26.551 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T07:30:26.602 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:30:26.772 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-04T07:30:26.793 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T07:30:26.830 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:30:27.006 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-04T07:30:27.020 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T07:30:27.089 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:30:27.183 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-04T07:30:27.204 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T07:30:27.241 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:30:27.417 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-04T07:30:27.432 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T07:30:27.485 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:30:27.603 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-04T07:30:27.624 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-04T07:30:27.653 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T07:30:27.887 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-04T07:30:27.902 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-04T07:30:27.945 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T07:30:27.972 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librados2. 2024-03-04T07:30:27.993 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../010-librados2_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:28.030 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librados2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:28.290 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librados2. 2024-03-04T07:30:28.305 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../010-librados2_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:28.348 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librados2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:28.525 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librbd1. 2024-03-04T07:30:28.546 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../011-librbd1_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:28.583 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librbd1 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:28.976 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librbd1. 2024-03-04T07:30:28.990 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../011-librbd1_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:29.026 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librbd1 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:29.269 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs2. 2024-03-04T07:30:29.281 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:29.320 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:29.445 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libcephfs2. 2024-03-04T07:30:29.459 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:29.520 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libcephfs2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:29.581 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rados. 2024-03-04T07:30:29.601 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:29.639 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rados (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:29.798 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rados. 2024-03-04T07:30:29.811 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:29.950 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-04T07:30:29.971 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:30:30.008 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-argparse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:30.261 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cephfs. 2024-03-04T07:30:30.273 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:30.310 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cephfs (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:30.510 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rados (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:30.630 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-04T07:30:30.651 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:30:30.688 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-common (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:30.926 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-04T07:30:30.940 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:30:30.966 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-prettytable. 2024-03-04T07:30:30.976 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-ceph-argparse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:30.987 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-04T07:30:31.024 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-04T07:30:31.228 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cephfs. 2024-03-04T07:30:31.243 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:31.286 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cephfs (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:31.302 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rbd. 2024-03-04T07:30:31.315 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:31.352 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rbd (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:31.614 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-04T07:30:31.629 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:30:31.672 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-ceph-common (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:32.034 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-prettytable. 2024-03-04T07:30:32.049 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-04T07:30:32.092 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-04T07:30:32.114 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-04T07:30:32.135 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-04T07:30:32.345 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rbd. 2024-03-04T07:30:32.359 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:32.395 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rbd (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:32.489 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T07:30:32.748 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-04T07:30:32.763 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-04T07:30:32.775 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-04T07:30:32.787 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-04T07:30:32.806 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T07:30:32.816 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T07:30:33.100 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-04T07:30:33.115 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-04T07:30:33.119 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-04T07:30:33.131 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-04T07:30:33.167 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T07:30:33.170 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T07:30:33.498 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua5.1. 2024-03-04T07:30:33.503 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-04T07:30:33.517 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-04T07:30:33.518 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-04T07:30:33.553 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T07:30:33.556 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-04T07:30:33.859 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua-any. 2024-03-04T07:30:33.879 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-04T07:30:33.890 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua5.1. 2024-03-04T07:30:33.906 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-04T07:30:33.917 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-04T07:30:33.948 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-04T07:30:34.147 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package zip. 2024-03-04T07:30:34.159 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-04T07:30:34.197 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking zip (3.0-12build2) ... 2024-03-04T07:30:34.309 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua-any. 2024-03-04T07:30:34.324 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-04T07:30:34.368 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-04T07:30:34.525 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package unzip. 2024-03-04T07:30:34.546 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-04T07:30:34.583 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-04T07:30:34.620 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package zip. 2024-03-04T07:30:34.635 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-04T07:30:34.670 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking zip (3.0-12build2) ... 2024-03-04T07:30:34.946 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package luarocks. 2024-03-04T07:30:34.966 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-04T07:30:34.996 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-04T07:30:35.007 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package unzip. 2024-03-04T07:30:35.022 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-04T07:30:35.065 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-04T07:30:35.395 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librgw2. 2024-03-04T07:30:35.416 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../027-librgw2_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:35.453 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librgw2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:35.469 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package luarocks. 2024-03-04T07:30:35.484 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-04T07:30:35.527 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-04T07:30:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librgw2. 2024-03-04T07:30:35.957 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../027-librgw2_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:35.989 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rgw. 2024-03-04T07:30:36.001 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:36.001 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librgw2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:36.039 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rgw (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:36.367 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-04T07:30:36.387 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-04T07:30:36.425 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T07:30:36.495 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rgw. 2024-03-04T07:30:36.511 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:36.545 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rgw (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:36.686 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libradosstriper1. 2024-03-04T07:30:36.707 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:36.736 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libradosstriper1 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:36.931 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-04T07:30:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-04T07:30:36.989 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T07:30:37.198 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-common. 2024-03-04T07:30:37.219 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:37.256 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-common (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:37.310 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libradosstriper1. 2024-03-04T07:30:37.325 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:37.393 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libradosstriper1 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:37.779 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-common. 2024-03-04T07:30:37.794 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:37.829 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-common (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:38.711 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-base. 2024-03-04T07:30:38.732 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:38.778 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-base (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:39.201 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-base. 2024-03-04T07:30:39.216 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:39.269 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-base (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:39.607 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-04T07:30:39.628 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-04T07:30:39.674 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-04T07:30:39.872 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-04T07:30:39.886 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-04T07:30:39.930 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-04T07:30:40.060 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cheroot. 2024-03-04T07:30:40.081 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-04T07:30:40.126 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T07:30:40.200 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cheroot. 2024-03-04T07:30:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-04T07:30:40.258 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T07:30:40.505 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-04T07:30:40.526 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-04T07:30:40.571 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-04T07:30:40.572 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-04T07:30:40.586 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-04T07:30:40.663 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-04T07:30:40.933 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-04T07:30:40.941 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-04T07:30:40.953 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-04T07:30:40.956 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-04T07:30:40.991 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-04T07:30:40.991 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-04T07:30:41.253 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-04T07:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-04T07:30:41.311 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-04T07:30:41.318 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-04T07:30:41.339 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-04T07:30:41.385 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-04T07:30:41.656 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-tempora. 2024-03-04T07:30:41.670 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-04T07:30:41.714 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-04T07:30:41.788 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-tempora. 2024-03-04T07:30:41.809 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-04T07:30:41.855 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-04T07:30:42.000 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-portend. 2024-03-04T07:30:42.015 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-04T07:30:42.058 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-04T07:30:42.224 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-portend. 2024-03-04T07:30:42.245 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-04T07:30:42.291 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-04T07:30:42.361 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-04T07:30:42.376 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-04T07:30:42.419 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-04T07:30:42.619 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-04T07:30:42.639 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-04T07:30:42.677 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-04T07:30:42.723 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-04T07:30:42.737 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-04T07:30:42.781 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-04T07:30:43.055 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-04T07:30:43.076 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-04T07:30:43.118 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-natsort. 2024-03-04T07:30:43.122 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-04T07:30:43.133 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-04T07:30:43.168 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-04T07:30:43.475 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-natsort. 2024-03-04T07:30:43.479 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-logutils. 2024-03-04T07:30:43.494 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-04T07:30:43.496 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-04T07:30:43.533 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-04T07:30:43.538 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-04T07:30:43.845 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-logutils. 2024-03-04T07:30:43.866 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-04T07:30:43.866 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-04T07:30:43.879 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-04T07:30:43.912 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-04T07:30:43.924 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-04T07:30:44.202 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-mako. 2024-03-04T07:30:44.213 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-04T07:30:44.214 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-04T07:30:44.235 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-04T07:30:44.252 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T07:30:44.275 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-04T07:30:44.534 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-mako. 2024-03-04T07:30:44.546 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-04T07:30:44.575 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T07:30:44.580 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-04T07:30:44.591 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-04T07:30:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-04T07:30:44.887 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-04T07:30:44.908 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-04T07:30:44.916 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-04T07:30:44.930 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-04T07:30:44.945 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-04T07:30:44.974 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-04T07:30:45.248 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-04T07:30:45.269 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-04T07:30:45.277 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-webob. 2024-03-04T07:30:45.292 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-04T07:30:45.306 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-04T07:30:45.369 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-04T07:30:45.609 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webob. 2024-03-04T07:30:45.630 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-04T07:30:45.668 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-04T07:30:45.731 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-waitress. 2024-03-04T07:30:45.743 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-04T07:30:45.791 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T07:30:46.029 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-waitress. 2024-03-04T07:30:46.042 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-04T07:30:46.082 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T07:30:46.100 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-tempita. 2024-03-04T07:30:46.116 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-04T07:30:46.142 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T07:30:46.324 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-tempita. 2024-03-04T07:30:46.336 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-04T07:30:46.365 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T07:30:46.437 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-paste. 2024-03-04T07:30:46.450 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-04T07:30:46.495 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T07:30:46.643 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-paste. 2024-03-04T07:30:46.664 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-04T07:30:46.693 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T07:30:46.781 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-04T07:30:46.794 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-04T07:30:46.857 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T07:30:47.013 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-04T07:30:47.025 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-04T07:30:47.063 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T07:30:47.151 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-04T07:30:47.164 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-04T07:30:47.201 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-04T07:30:47.333 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-04T07:30:47.353 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-04T07:30:47.382 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-04T07:30:47.512 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-webtest. 2024-03-04T07:30:47.526 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-04T07:30:47.571 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-04T07:30:47.686 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webtest. 2024-03-04T07:30:47.706 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-04T07:30:47.744 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-04T07:30:47.916 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pecan. 2024-03-04T07:30:47.929 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-04T07:30:47.973 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T07:30:48.080 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pecan. 2024-03-04T07:30:48.101 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-04T07:30:48.138 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T07:30:48.312 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-certifi. 2024-03-04T07:30:48.325 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-04T07:30:48.362 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-04T07:30:48.518 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-certifi. 2024-03-04T07:30:48.539 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-04T07:30:48.576 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-04T07:30:48.665 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-idna. 2024-03-04T07:30:48.678 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-04T07:30:48.723 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-04T07:30:48.905 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-idna. 2024-03-04T07:30:48.926 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-04T07:30:48.962 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-04T07:30:49.042 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-urllib3. 2024-03-04T07:30:49.059 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-04T07:30:49.101 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T07:30:49.257 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-urllib3. 2024-03-04T07:30:49.278 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-04T07:30:49.315 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T07:30:49.471 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-requests. 2024-03-04T07:30:49.485 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-04T07:30:49.529 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T07:30:49.738 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-requests. 2024-03-04T07:30:49.759 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-04T07:30:49.824 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-04T07:30:49.838 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-04T07:30:49.843 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T07:30:49.882 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T07:30:50.172 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-04T07:30:50.192 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-04T07:30:50.202 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-04T07:30:50.216 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:30:50.230 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T07:30:50.260 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:50.524 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-04T07:30:50.546 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:30:50.583 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:50.624 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-04T07:30:50.637 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:50.682 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:50.980 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-04T07:30:51.001 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:51.001 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-04T07:30:51.015 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-04T07:30:51.038 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:51.060 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-04T07:30:51.371 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-openssl. 2024-03-04T07:30:51.385 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-04T07:30:51.429 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-04T07:30:51.482 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-04T07:30:51.503 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-04T07:30:51.540 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-04T07:30:51.799 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr. 2024-03-04T07:30:51.813 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:51.818 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-openssl. 2024-03-04T07:30:51.839 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-04T07:30:51.857 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:51.868 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-04T07:30:52.194 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mon. 2024-03-04T07:30:52.208 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:52.213 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr. 2024-03-04T07:30:52.234 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:52.243 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mon (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:52.271 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:52.674 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mon. 2024-03-04T07:30:52.696 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:52.732 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mon (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:52.856 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-osd. 2024-03-04T07:30:52.870 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:52.914 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-osd (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:53.336 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-osd. 2024-03-04T07:30:53.357 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:53.386 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-osd (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:53.717 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph. 2024-03-04T07:30:53.731 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../068-ceph_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:53.775 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:54.078 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-fuse. 2024-03-04T07:30:54.093 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:54.128 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-fuse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:54.448 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph. 2024-03-04T07:30:54.460 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../068-ceph_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:54.523 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:54.665 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mds. 2024-03-04T07:30:54.684 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:54.722 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mds (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:54.842 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-fuse. 2024-03-04T07:30:54.862 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:54.900 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-fuse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:55.134 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package cephadm. 2024-03-04T07:30:55.147 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../071-cephadm_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:55.184 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking cephadm (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:55.262 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mds. 2024-03-04T07:30:55.282 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:55.320 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mds (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:55.514 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-04T07:30:55.527 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-04T07:30:55.570 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-04T07:30:55.706 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package cephadm. 2024-03-04T07:30:55.727 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../071-cephadm_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:30:55.764 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking cephadm (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:55.898 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-04T07:30:55.911 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T07:30:55.964 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T07:30:56.084 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-04T07:30:56.105 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-04T07:30:56.142 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-04T07:30:56.470 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-04T07:30:56.482 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T07:30:56.520 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T07:30:57.020 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-babel. 2024-03-04T07:30:57.032 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T07:30:57.095 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T07:30:57.423 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jinja2. 2024-03-04T07:30:57.434 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-04T07:30:57.472 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T07:30:57.575 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-babel. 2024-03-04T07:30:57.596 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T07:30:57.641 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T07:30:57.851 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-04T07:30:57.863 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:30:57.901 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:57.961 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jinja2. 2024-03-04T07:30:57.982 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-04T07:30:58.019 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T07:30:58.196 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-04T07:30:58.208 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-04T07:30:58.246 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-04T07:30:58.398 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-04T07:30:58.419 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:30:58.456 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:58.549 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-routes. 2024-03-04T07:30:58.561 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-04T07:30:58.632 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T07:30:58.801 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-04T07:30:58.822 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-04T07:30:58.876 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-04T07:30:58.960 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-04T07:30:58.973 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:30:59.018 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:30:59.188 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-routes. 2024-03-04T07:30:59.208 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-04T07:30:59.245 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T07:30:59.488 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-04T07:30:59.501 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-04T07:30:59.538 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T07:30:59.591 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-04T07:30:59.612 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:30:59.649 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:00.017 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-joblib. 2024-03-04T07:31:00.029 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-04T07:31:00.058 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T07:31:00.077 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-04T07:31:00.098 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-04T07:31:00.135 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T07:31:00.437 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-04T07:31:00.449 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-04T07:31:00.572 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-04T07:31:00.631 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-joblib. 2024-03-04T07:31:00.652 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-04T07:31:00.689 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T07:31:00.867 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-sklearn. 2024-03-04T07:31:00.879 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-04T07:31:00.950 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T07:31:01.059 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-04T07:31:01.080 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-04T07:31:01.211 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-04T07:31:01.522 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn. 2024-03-04T07:31:01.544 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-04T07:31:01.580 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T07:31:01.835 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-04T07:31:01.847 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:31:01.885 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:02.306 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-04T07:31:02.328 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:31:02.373 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:02.946 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cachetools. 2024-03-04T07:31:02.960 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-04T07:31:03.005 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-04T07:31:03.266 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-04T07:31:03.279 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-04T07:31:03.316 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-04T07:31:03.401 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cachetools. 2024-03-04T07:31:03.422 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-04T07:31:03.458 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-04T07:31:03.653 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-04T07:31:03.665 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-04T07:31:03.703 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T07:31:03.754 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-04T07:31:03.775 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-04T07:31:03.812 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-04T07:31:04.006 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rsa. 2024-03-04T07:31:04.019 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-04T07:31:04.064 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-04T07:31:04.132 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-04T07:31:04.153 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-04T07:31:04.190 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T07:31:04.410 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-google-auth. 2024-03-04T07:31:04.423 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-04T07:31:04.468 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-04T07:31:04.502 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rsa. 2024-03-04T07:31:04.523 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-04T07:31:04.560 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-04T07:31:04.779 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-04T07:31:04.793 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-04T07:31:04.838 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T07:31:04.872 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-google-auth. 2024-03-04T07:31:04.893 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-04T07:31:04.930 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-04T07:31:05.141 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-websocket. 2024-03-04T07:31:05.154 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-04T07:31:05.216 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-04T07:31:05.251 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-04T07:31:05.272 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-04T07:31:05.308 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T07:31:05.528 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-04T07:31:05.543 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-04T07:31:05.594 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T07:31:05.595 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-websocket. 2024-03-04T07:31:05.616 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-04T07:31:05.653 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-04T07:31:05.982 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-04T07:31:06.003 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-04T07:31:06.040 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T07:31:06.333 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-04T07:31:06.346 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:31:06.391 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:06.711 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-04T07:31:06.724 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-04T07:31:06.746 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-04T07:31:06.752 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T07:31:06.759 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:31:06.797 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:07.097 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-04T07:31:07.111 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T07:31:07.115 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-04T07:31:07.137 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-04T07:31:07.147 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T07:31:07.173 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T07:31:07.400 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package jq. 2024-03-04T07:31:07.414 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T07:31:07.450 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-04T07:31:07.551 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-04T07:31:07.573 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T07:31:07.618 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T07:31:07.686 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package socat. 2024-03-04T07:31:07.700 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-04T07:31:07.744 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-04T07:31:07.854 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package jq. 2024-03-04T07:31:07.866 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T07:31:07.896 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-04T07:31:08.064 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package xmlstarlet. 2024-03-04T07:31:08.079 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-04T07:31:08.115 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package socat. 2024-03-04T07:31:08.136 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-04T07:31:08.139 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-04T07:31:08.173 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-04T07:31:08.493 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package xmlstarlet. 2024-03-04T07:31:08.502 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-test. 2024-03-04T07:31:08.505 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-04T07:31:08.516 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:31:08.543 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-04T07:31:08.560 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-test (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:08.923 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-test. 2024-03-04T07:31:08.944 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:31:08.989 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-test (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:10.856 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-volume. 2024-03-04T07:31:10.871 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:31:10.905 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-volume (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:11.235 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-04T07:31:11.250 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:31:11.284 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking cephfs-mirror (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:11.360 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-volume. 2024-03-04T07:31:11.381 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:31:11.426 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-volume (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:11.630 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-colorama. 2024-03-04T07:31:11.644 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-04T07:31:11.688 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-04T07:31:11.797 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-04T07:31:11.818 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:31:11.855 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking cephfs-mirror (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:11.949 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package cephfs-shell. 2024-03-04T07:31:11.964 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:31:12.007 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking cephfs-shell (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:12.192 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-colorama. 2024-03-04T07:31:12.213 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-04T07:31:12.250 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-04T07:31:12.301 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package cephfs-top. 2024-03-04T07:31:12.316 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:31:12.351 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking cephfs-top (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:12.511 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package cephfs-shell. 2024-03-04T07:31:12.523 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:31:12.560 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking cephfs-shell (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:12.562 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-04T07:31:12.577 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:31:12.620 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libcephfs-dev (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:12.814 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package cephfs-top. 2024-03-04T07:31:12.835 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1739-g1bc250c7-1jammy_all.deb ... 2024-03-04T07:31:12.872 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking cephfs-top (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:12.957 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-04T07:31:12.972 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-04T07:31:13.015 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T07:31:13.124 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-04T07:31:13.146 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:31:13.191 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs-dev (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:13.387 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-04T07:31:13.401 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-04T07:31:13.444 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-04T07:31:13.519 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-04T07:31:13.540 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-04T07:31:13.585 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T07:31:13.757 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package nvme-cli. 2024-03-04T07:31:13.772 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-04T07:31:13.840 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T07:31:13.900 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-04T07:31:13.921 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-04T07:31:13.959 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-04T07:31:14.271 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package pkg-config. 2024-03-04T07:31:14.286 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-04T07:31:14.329 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package nvme-cli. 2024-03-04T07:31:14.329 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T07:31:14.350 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-04T07:31:14.395 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T07:31:14.623 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-04T07:31:14.638 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-04T07:31:14.681 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-04T07:31:14.842 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package pkg-config. 2024-03-04T07:31:14.864 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-04T07:31:14.908 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T07:31:15.102 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-04T07:31:15.116 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-04T07:31:15.159 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-04T07:31:15.169 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-04T07:31:15.191 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-04T07:31:15.236 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-04T07:31:15.463 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pastescript. 2024-03-04T07:31:15.477 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-04T07:31:15.512 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-04T07:31:15.680 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-04T07:31:15.702 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-04T07:31:15.747 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-04T07:31:15.774 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pluggy. 2024-03-04T07:31:15.789 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-04T07:31:15.824 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-04T07:31:16.008 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastescript. 2024-03-04T07:31:16.020 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-04T07:31:16.049 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-04T07:31:16.131 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-psutil. 2024-03-04T07:31:16.145 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-04T07:31:16.193 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-04T07:31:16.353 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pluggy. 2024-03-04T07:31:16.374 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-04T07:31:16.427 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-04T07:31:16.505 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-py. 2024-03-04T07:31:16.520 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-04T07:31:16.563 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-04T07:31:16.762 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-psutil. 2024-03-04T07:31:16.783 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-04T07:31:16.830 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-04T07:31:16.883 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pygments. 2024-03-04T07:31:16.898 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-04T07:31:16.975 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T07:31:17.166 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-py. 2024-03-04T07:31:17.188 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-04T07:31:17.224 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-04T07:31:17.420 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-04T07:31:17.435 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-04T07:31:17.479 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-04T07:31:17.544 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pygments. 2024-03-04T07:31:17.566 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-04T07:31:17.610 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T07:31:17.831 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-toml. 2024-03-04T07:31:17.844 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-04T07:31:17.881 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-04T07:31:18.074 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-04T07:31:18.095 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-04T07:31:18.139 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-04T07:31:18.159 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pytest. 2024-03-04T07:31:18.172 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-04T07:31:18.234 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T07:31:18.451 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-toml. 2024-03-04T07:31:18.472 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-04T07:31:18.508 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-04T07:31:18.606 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-simplejson. 2024-03-04T07:31:18.620 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-04T07:31:18.668 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-04T07:31:18.786 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pytest. 2024-03-04T07:31:18.808 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-04T07:31:18.853 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T07:31:18.917 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-04T07:31:18.932 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-04T07:31:18.967 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-04T07:31:19.215 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplejson. 2024-03-04T07:31:19.236 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-04T07:31:19.281 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-04T07:31:19.542 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-04T07:31:19.564 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-04T07:31:19.600 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-04T07:31:19.671 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package radosgw. 2024-03-04T07:31:19.684 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../122-radosgw_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:31:19.729 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking radosgw (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:20.321 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package radosgw. 2024-03-04T07:31:20.343 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../122-radosgw_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:31:20.357 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package rbd-fuse. 2024-03-04T07:31:20.371 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:31:20.387 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking radosgw (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:20.416 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking rbd-fuse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:20.709 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-04T07:31:20.988 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T07:31:21.114 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T07:31:21.182 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package rbd-fuse. 2024-03-04T07:31:21.194 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1739-g1bc250c7-1jammy_amd64.deb ... 2024-03-04T07:31:21.231 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking rbd-fuse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:21.377 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-04T07:31:21.533 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-04T07:31:21.762 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-04T07:31:21.794 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T07:31:21.920 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T07:31:22.181 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-04T07:31:22.364 INFO:teuthology.orchestra.run.smithi113.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-04T07:31:22.403 INFO:teuthology.orchestra.run.smithi113.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-03-04T07:31:22.473 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up cephadm (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:22.572 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-04T07:31:22.967 INFO:teuthology.orchestra.run.smithi113.stdout:Adding system user cephadm....done 2024-03-04T07:31:23.040 INFO:teuthology.orchestra.run.smithi136.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-04T07:31:23.121 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T07:31:23.139 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up cephadm (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:23.431 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-04T07:31:23.621 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user cephadm....done 2024-03-04T07:31:23.691 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-04T07:31:23.777 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T07:31:23.816 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-04T07:31:24.054 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-04T07:31:24.084 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-04T07:31:24.304 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-04T07:31:24.353 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T07:31:24.431 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-04T07:31:24.478 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-04T07:31:24.690 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-04T07:31:24.805 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T07:31:24.933 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T07:31:25.051 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-04T07:31:25.174 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-04T07:31:25.344 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T07:31:25.450 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-04T07:31:25.735 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-04T07:31:25.815 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-04T07:31:26.020 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-04T07:31:26.255 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-04T07:31:26.272 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-04T07:31:26.523 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-04T07:31:26.556 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-04T07:31:26.791 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-04T07:31:26.823 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-04T07:31:27.067 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-04T07:31:27.141 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-ceph-argparse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:27.343 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-04T07:31:27.417 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T07:31:27.568 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T07:31:27.594 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-04T07:31:27.711 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-04T07:31:27.845 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-argparse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:27.987 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T07:31:28.104 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T07:31:28.130 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-04T07:31:28.231 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T07:31:28.356 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-04T07:31:28.398 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-04T07:31:28.608 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T07:31:28.708 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-04T07:31:28.796 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-04T07:31:28.845 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-04T07:31:28.918 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T07:31:28.951 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-04T07:31:29.068 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-04T07:31:29.369 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-04T07:31:29.399 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-04T07:31:29.445 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T07:31:29.462 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T07:31:29.621 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-04T07:31:29.630 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-04T07:31:29.880 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T07:31:29.990 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T07:31:30.116 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-04T07:31:30.148 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-04T07:31:30.383 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T07:31:30.466 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up zip (3.0-12build2) ... 2024-03-04T07:31:30.617 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T07:31:30.634 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-04T07:31:30.936 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up zip (3.0-12build2) ... 2024-03-04T07:31:31.062 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T07:31:31.319 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T07:31:31.595 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T07:31:31.730 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-04T07:31:31.747 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T07:31:31.881 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T07:31:32.015 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T07:31:32.141 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-04T07:31:32.207 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-04T07:31:32.267 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T07:31:32.551 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-04T07:31:32.641 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-04T07:31:32.917 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T07:31:32.952 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-04T07:31:33.203 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T07:31:33.311 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T07:31:33.580 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T07:31:33.646 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-04T07:31:33.864 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-04T07:31:33.963 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T07:31:34.199 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T07:31:34.332 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-04T07:31:34.542 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-04T07:31:34.599 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-04T07:31:34.750 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-ceph-common (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:34.784 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-04T07:31:34.885 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-common (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:35.068 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-idna (3.3-1) ... 2024-03-04T07:31:35.211 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-idna (3.3-1) ... 2024-03-04T07:31:35.403 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T07:31:35.529 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T07:31:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T07:31:36.734 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T07:31:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:31:36.978 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:31:37.028 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-04T07:31:37.120 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-04T07:31:37.313 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T07:31:37.388 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T07:31:37.464 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-04T07:31:37.514 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-04T07:31:37.615 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T07:31:37.640 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T07:31:37.933 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-04T07:31:37.941 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-04T07:31:38.184 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-04T07:31:38.201 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-04T07:31:38.452 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:31:38.468 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:31:38.578 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-04T07:31:38.603 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-04T07:31:38.888 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-04T07:31:38.896 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-04T07:31:39.139 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-04T07:31:39.172 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-04T07:31:39.407 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-04T07:31:39.448 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-04T07:31:39.650 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-04T07:31:39.716 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-04T07:31:39.917 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-04T07:31:39.992 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-04T07:31:40.202 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T07:31:40.268 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T07:31:40.319 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-04T07:31:40.436 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-04T07:31:40.595 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T07:31:40.730 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T07:31:40.738 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T07:31:40.838 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T07:31:41.014 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-04T07:31:41.156 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-04T07:31:41.359 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-04T07:31:41.474 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-04T07:31:41.759 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T07:31:41.884 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-04T07:31:41.899 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T07:31:42.017 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-04T07:31:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T07:31:42.118 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T07:31:42.124 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-04T07:31:42.199 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-04T07:31:42.370 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T07:31:42.436 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T07:31:42.521 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T07:31:42.562 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T07:31:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-04T07:31:42.964 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-04T07:31:43.208 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-04T07:31:43.223 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-04T07:31:43.349 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-04T07:31:43.358 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-04T07:31:43.609 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:31:43.651 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T07:31:43.735 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T07:31:43.786 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T07:31:44.061 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T07:31:44.095 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T07:31:44.378 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T07:31:44.422 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T07:31:44.655 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-04T07:31:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-04T07:31:44.874 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-04T07:31:44.959 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-04T07:31:45.142 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-04T07:31:45.277 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-04T07:31:45.401 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T07:31:45.554 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T07:31:45.669 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-04T07:31:46.046 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-04T07:31:46.164 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-04T07:31:46.322 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T07:31:46.591 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-04T07:31:46.658 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T07:31:46.785 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librados2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:46.908 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T07:31:46.910 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:47.061 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-04T07:31:47.285 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T07:31:47.355 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:47.445 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librados2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:47.456 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libradosstriper1 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:47.574 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-04T07:31:47.596 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:47.748 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-04T07:31:47.850 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librbd1 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:47.960 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-modules-core (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:48.086 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-fuse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:48.167 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libcephfs2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:48.299 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T07:31:48.299 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T07:31:48.302 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libradosstriper1 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:48.444 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-04T07:31:48.754 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librbd1 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:48.812 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs-dev (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:48.905 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-modules-core (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:48.938 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rados (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:49.031 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-fuse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:49.064 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librgw2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:49.190 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up cephfs-top (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:49.240 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T07:31:49.240 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T07:31:49.316 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T07:31:49.874 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libcephfs-dev (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:50.000 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rados (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:50.126 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librgw2 (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:50.261 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up cephfs-top (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:50.437 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T07:31:50.511 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rbd (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:50.637 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up rbd-fuse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:50.738 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rgw (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:50.839 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cephfs (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:50.940 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up cephfs-shell (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:51.066 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-common (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:51.650 INFO:teuthology.orchestra.run.smithi136.stdout:Adding group ceph....done 2024-03-04T07:31:51.724 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rbd (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:51.850 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up rbd-fuse (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:51.984 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user ceph....done 2024-03-04T07:31:52.001 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rgw (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:52.135 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cephfs (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:52.170 INFO:teuthology.orchestra.run.smithi136.stdout:Setting system user ceph properties....done 2024-03-04T07:31:52.188 INFO:teuthology.orchestra.run.smithi136.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-04T07:31:52.236 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up cephfs-shell (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:52.326 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-04T07:31:52.346 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-common (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:52.720 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-04T07:31:52.924 INFO:teuthology.orchestra.run.smithi113.stdout:Adding group ceph....done 2024-03-04T07:31:53.233 INFO:teuthology.orchestra.run.smithi113.stdout:Adding system user ceph....done 2024-03-04T07:31:53.291 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up cephfs-mirror (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:53.403 INFO:teuthology.orchestra.run.smithi113.stdout:Setting system user ceph properties....done 2024-03-04T07:31:53.417 INFO:teuthology.orchestra.run.smithi113.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-04T07:31:53.559 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-04T07:31:53.621 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-04T07:31:53.621 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-04T07:31:53.966 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-04T07:31:54.151 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-test (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:54.277 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up radosgw (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:54.639 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up cephfs-mirror (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:54.760 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T07:31:54.760 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T07:31:54.847 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-04T07:31:54.847 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-04T07:31:55.262 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-base (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:55.473 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-test (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:55.599 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up radosgw (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:55.629 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-04T07:31:56.148 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T07:31:56.148 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T07:31:56.201 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mds (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:56.468 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T07:31:56.468 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T07:31:56.785 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-base (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:57.101 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:57.113 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-04T07:31:57.321 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T07:31:57.321 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T07:31:57.737 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mds (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:57.860 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-osd (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:57.977 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T07:31:57.978 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T07:31:58.199 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T07:31:58.199 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T07:31:58.605 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:58.730 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:58.818 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T07:31:58.818 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T07:31:58.996 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:59.147 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mon (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:59.341 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T07:31:59.341 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T07:31:59.415 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-osd (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:31:59.720 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T07:31:59.720 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T07:31:59.857 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-cephadm (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:00.033 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:00.159 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-dashboard (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:00.310 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-volume (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:00.341 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:00.484 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:00.619 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mon (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:00.671 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-04T07:32:00.772 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T07:32:00.827 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T07:32:00.827 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T07:32:00.916 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T07:32:01.462 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-cephadm (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:01.582 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-04T07:32:01.605 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:01.731 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-dashboard (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:01.873 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-volume (19.0.0-1739-g1bc250c7-1jammy) ... 2024-03-04T07:32:02.225 INFO:teuthology.orchestra.run.smithi136.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T07:32:02.228 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-04T07:32:02.268 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-04T07:32:02.281 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-03-04T07:32:02.369 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T07:32:02.474 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-03-04T07:32:02.475 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-03-04T07:32:02.504 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T07:32:02.644 INFO:teuthology.orchestra.run.smithi136.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-04T07:32:02.644 INFO:teuthology.orchestra.run.smithi136.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-04T07:32:02.645 INFO:teuthology.orchestra.run.smithi136.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-04T07:32:02.645 INFO:teuthology.orchestra.run.smithi136.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-04T07:32:02.645 INFO:teuthology.orchestra.run.smithi136.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-04T07:32:02.645 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T07:32:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T07:32:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T07:32:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T07:32:02.645 INFO:teuthology.orchestra.run.smithi136.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-04T07:32:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: python2.7-minimal 2024-03-04T07:32:02.646 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T07:32:02.646 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2024-03-04T07:32:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-04T07:32:02.646 INFO:teuthology.orchestra.run.smithi136.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-04T07:32:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-04T07:32:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-04T07:32:02.647 INFO:teuthology.orchestra.run.smithi136.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-04T07:32:02.648 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2024-03-04T07:32:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-04T07:32:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-04T07:32:02.648 INFO:teuthology.orchestra.run.smithi136.stdout: libengine-pkcs11-openssl 2024-03-04T07:32:02.688 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2024-03-04T07:32:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-04T07:32:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-04T07:32:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-04T07:32:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-04T07:32:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-04T07:32:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-04T07:32:02.688 INFO:teuthology.orchestra.run.smithi136.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-04T07:32:02.688 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages will be upgraded: 2024-03-04T07:32:02.689 INFO:teuthology.orchestra.run.smithi136.stdout: libssl3 2024-03-04T07:32:02.745 INFO:teuthology.orchestra.run.smithi136.stdout:1 upgraded, 39 newly installed, 0 to remove and 277 not upgraded. 2024-03-04T07:32:02.745 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 54.6 MB of archives. 2024-03-04T07:32:02.745 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-04T07:32:02.745 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-04T07:32:02.861 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-04T07:32:02.928 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-04T07:32:02.944 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-04T07:32:02.945 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-04T07:32:02.947 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-04T07:32:02.949 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-04T07:32:02.950 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-04T07:32:02.952 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-04T07:32:02.953 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-04T07:32:02.953 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-04T07:32:02.954 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-04T07:32:02.973 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-04T07:32:03.307 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-04T07:32:03.309 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-04T07:32:03.309 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-04T07:32:03.312 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-04T07:32:03.312 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-04T07:32:03.316 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-04T07:32:03.317 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-04T07:32:03.325 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-04T07:32:03.325 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-04T07:32:03.422 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-04T07:32:03.422 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-04T07:32:03.435 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-04T07:32:03.460 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-04T07:32:03.465 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-04T07:32:03.465 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-04T07:32:03.466 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-04T07:32:03.466 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-04T07:32:03.479 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-04T07:32:03.481 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-04T07:32:03.664 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-04T07:32:03.723 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-04T07:32:03.723 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-04T07:32:03.723 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-04T07:32:03.723 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-04T07:32:03.723 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-04T07:32:03.723 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-04T07:32:03.735 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-04T07:32:03.735 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-04T07:32:04.127 INFO:teuthology.orchestra.run.smithi136.stdout:Preconfiguring packages ... 2024-03-04T07:32:04.284 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 54.6 MB in 1s (52.5 MB/s) 2024-03-04T07:32:04.353 INFO:teuthology.orchestra.run.smithi113.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T07:32:04.357 DEBUG:teuthology.orchestra.run.smithi113:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-04T07:32:04.429 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-03-04T07:32:04.572 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package flex. 2024-03-04T07:32:04.599 INFO:teuthology.orchestra.run.smithi113.stdout:Building dependency tree... 2024-03-04T07:32:04.600 INFO:teuthology.orchestra.run.smithi113.stdout:Reading state information... 2024-03-04T07:32:04.662 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138164 files and directories currently installed.) 2024-03-04T07:32:04.667 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-04T07:32:04.711 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-04T07:32:04.757 INFO:teuthology.orchestra.run.smithi113.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-04T07:32:04.757 INFO:teuthology.orchestra.run.smithi113.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-04T07:32:04.757 INFO:teuthology.orchestra.run.smithi113.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-04T07:32:04.757 INFO:teuthology.orchestra.run.smithi113.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-04T07:32:04.757 INFO:teuthology.orchestra.run.smithi113.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-04T07:32:04.757 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T07:32:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T07:32:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T07:32:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T07:32:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T07:32:04.758 INFO:teuthology.orchestra.run.smithi113.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T07:32:04.758 INFO:teuthology.orchestra.run.smithi113.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T07:32:04.759 INFO:teuthology.orchestra.run.smithi113.stdout:The following additional packages will be installed: 2024-03-04T07:32:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-04T07:32:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-04T07:32:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-04T07:32:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-04T07:32:04.759 INFO:teuthology.orchestra.run.smithi113.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-04T07:32:04.760 INFO:teuthology.orchestra.run.smithi113.stdout:Suggested packages: 2024-03-04T07:32:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-04T07:32:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-04T07:32:04.760 INFO:teuthology.orchestra.run.smithi113.stdout: libengine-pkcs11-openssl 2024-03-04T07:32:04.810 INFO:teuthology.orchestra.run.smithi113.stdout:The following NEW packages will be installed: 2024-03-04T07:32:04.811 INFO:teuthology.orchestra.run.smithi113.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-04T07:32:04.811 INFO:teuthology.orchestra.run.smithi113.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-04T07:32:04.811 INFO:teuthology.orchestra.run.smithi113.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-04T07:32:04.811 INFO:teuthology.orchestra.run.smithi113.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-04T07:32:04.811 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-04T07:32:04.811 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-04T07:32:04.811 INFO:teuthology.orchestra.run.smithi113.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-04T07:32:04.812 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages will be upgraded: 2024-03-04T07:32:04.812 INFO:teuthology.orchestra.run.smithi113.stdout: libssl3 2024-03-04T07:32:04.996 INFO:teuthology.orchestra.run.smithi113.stdout:1 upgraded, 39 newly installed, 0 to remove and 277 not upgraded. 2024-03-04T07:32:04.996 INFO:teuthology.orchestra.run.smithi113.stdout:Need to get 54.6 MB of archives. 2024-03-04T07:32:04.996 INFO:teuthology.orchestra.run.smithi113.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-04T07:32:04.996 INFO:teuthology.orchestra.run.smithi113.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-04T07:32:05.271 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-04T07:32:05.450 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-04T07:32:05.490 INFO:teuthology.orchestra.run.smithi113.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-04T07:32:05.748 INFO:teuthology.orchestra.run.smithi113.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-04T07:32:05.782 INFO:teuthology.orchestra.run.smithi113.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-04T07:32:05.784 INFO:teuthology.orchestra.run.smithi113.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-04T07:32:05.790 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-04T07:32:05.792 INFO:teuthology.orchestra.run.smithi113.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-04T07:32:05.793 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-04T07:32:05.796 INFO:teuthology.orchestra.run.smithi113.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-04T07:32:05.798 INFO:teuthology.orchestra.run.smithi113.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-04T07:32:05.799 INFO:teuthology.orchestra.run.smithi113.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-04T07:32:05.869 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-04T07:32:05.963 INFO:teuthology.orchestra.run.smithi113.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-04T07:32:06.302 INFO:teuthology.orchestra.run.smithi113.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-04T07:32:06.387 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package bison. 2024-03-04T07:32:06.476 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138237 files and directories currently installed.) 2024-03-04T07:32:06.481 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-04T07:32:06.522 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-04T07:32:06.890 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-04T07:32:06.912 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-04T07:32:06.948 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-04T07:32:07.374 INFO:teuthology.orchestra.run.smithi113.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-04T07:32:07.376 INFO:teuthology.orchestra.run.smithi113.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-04T07:32:07.377 INFO:teuthology.orchestra.run.smithi113.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-04T07:32:07.379 INFO:teuthology.orchestra.run.smithi113.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-04T07:32:07.379 INFO:teuthology.orchestra.run.smithi113.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-04T07:32:07.384 INFO:teuthology.orchestra.run.smithi113.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-04T07:32:07.386 INFO:teuthology.orchestra.run.smithi113.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-04T07:32:07.404 INFO:teuthology.orchestra.run.smithi113.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-04T07:32:07.405 INFO:teuthology.orchestra.run.smithi113.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-04T07:32:07.420 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package dump. 2024-03-04T07:32:07.443 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-04T07:32:07.486 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking dump (0.4b47-3) ... 2024-03-04T07:32:07.649 INFO:teuthology.orchestra.run.smithi113.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-04T07:32:07.731 INFO:teuthology.orchestra.run.smithi113.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-04T07:32:07.774 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package indent. 2024-03-04T07:32:07.796 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:07.840 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-04T07:32:08.127 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-04T07:32:08.149 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-04T07:32:08.193 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-04T07:32:08.211 INFO:teuthology.orchestra.run.smithi113.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-04T07:32:08.341 INFO:teuthology.orchestra.run.smithi113.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-04T07:32:08.353 INFO:teuthology.orchestra.run.smithi113.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-04T07:32:08.354 INFO:teuthology.orchestra.run.smithi113.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-04T07:32:08.355 INFO:teuthology.orchestra.run.smithi113.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-04T07:32:08.357 INFO:teuthology.orchestra.run.smithi113.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-04T07:32:08.396 INFO:teuthology.orchestra.run.smithi113.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-04T07:32:08.409 INFO:teuthology.orchestra.run.smithi113.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-04T07:32:08.455 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-04T07:32:08.478 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-04T07:32:08.513 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-04T07:32:08.775 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-04T07:32:08.797 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-04T07:32:08.841 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-04T07:32:08.847 INFO:teuthology.orchestra.run.smithi113.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-04T07:32:09.010 INFO:teuthology.orchestra.run.smithi113.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-04T07:32:09.092 INFO:teuthology.orchestra.run.smithi113.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-04T07:32:09.093 INFO:teuthology.orchestra.run.smithi113.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-04T07:32:09.178 INFO:teuthology.orchestra.run.smithi113.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-04T07:32:09.186 INFO:teuthology.orchestra.run.smithi113.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-04T07:32:09.187 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-04T07:32:09.209 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-04T07:32:09.253 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-04T07:32:09.481 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-04T07:32:09.494 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-04T07:32:09.522 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-04T07:32:09.650 INFO:teuthology.orchestra.run.smithi113.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-04T07:32:09.651 INFO:teuthology.orchestra.run.smithi113.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-04T07:32:09.717 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjson-perl. 2024-03-04T07:32:09.730 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-04T07:32:09.767 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-04T07:32:10.036 INFO:teuthology.orchestra.run.smithi113.stdout:Preconfiguring packages ... 2024-03-04T07:32:10.087 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-04T07:32:10.109 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-04T07:32:10.153 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-04T07:32:10.196 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 54.6 MB in 5s (11.3 MB/s) 2024-03-04T07:32:10.372 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package flex. 2024-03-04T07:32:10.439 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140583 files and directories currently installed.) 2024-03-04T07:32:10.443 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-04T07:32:10.487 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-04T07:32:11.173 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-04T07:32:11.401 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-04T07:32:11.884 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-04T07:32:11.898 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-04T07:32:11.920 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-04T07:32:11.965 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-04T07:32:12.192 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-04T07:32:12.205 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-04T07:32:12.234 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-04T07:32:12.411 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package bison. 2024-03-04T07:32:12.482 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140656 files and directories currently installed.) 2024-03-04T07:32:12.485 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-04T07:32:12.530 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-04T07:32:12.575 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-04T07:32:12.597 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-04T07:32:12.912 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-04T07:32:12.981 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-04T07:32:12.995 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-04T07:32:13.039 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-04T07:32:13.343 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package dump. 2024-03-04T07:32:13.357 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-04T07:32:13.393 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking dump (0.4b47-3) ... 2024-03-04T07:32:13.491 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-04T07:32:13.512 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-04T07:32:13.557 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-04T07:32:13.715 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package indent. 2024-03-04T07:32:13.728 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:13.771 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-04T07:32:13.835 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-04T07:32:13.857 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-04T07:32:13.893 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-04T07:32:14.040 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-04T07:32:14.056 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-04T07:32:14.098 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-04T07:32:14.264 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-04T07:32:14.285 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:14.330 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:14.394 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-04T07:32:14.410 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-04T07:32:14.470 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-04T07:32:14.683 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-04T07:32:14.705 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:14.705 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-04T07:32:14.720 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-04T07:32:14.749 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-04T07:32:14.763 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-04T07:32:15.027 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-04T07:32:15.040 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:15.069 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-04T07:32:15.100 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-04T07:32:15.115 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-04T07:32:15.152 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-04T07:32:15.305 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-04T07:32:15.327 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-04T07:32:15.372 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-04T07:32:15.463 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-04T07:32:15.478 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-04T07:32:15.521 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-04T07:32:15.757 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libjson-perl. 2024-03-04T07:32:15.773 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-04T07:32:15.815 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-04T07:32:15.892 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-04T07:32:15.913 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-04T07:32:15.958 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-04T07:32:16.185 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-04T07:32:16.200 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-04T07:32:16.243 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-04T07:32:16.303 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package modemmanager. 2024-03-04T07:32:16.325 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-04T07:32:16.369 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-04T07:32:16.977 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package network-manager. 2024-03-04T07:32:16.999 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-04T07:32:17.051 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-04T07:32:17.768 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ppp. 2024-03-04T07:32:17.790 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-04T07:32:17.947 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-04T07:32:17.963 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-04T07:32:18.013 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-04T07:32:18.088 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-04T07:32:18.308 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-04T07:32:18.323 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-04T07:32:18.366 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-04T07:32:18.482 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package pptp-linux. 2024-03-04T07:32:18.495 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-04T07:32:18.532 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-04T07:32:18.661 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-04T07:32:18.676 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-04T07:32:18.710 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-04T07:32:18.785 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-04T07:32:18.807 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-04T07:32:18.851 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-04T07:32:19.071 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-04T07:32:19.087 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-04T07:32:19.129 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-04T07:32:19.146 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-04T07:32:19.169 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-04T07:32:19.213 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking postgresql-client-common (238) ... 2024-03-04T07:32:19.457 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-04T07:32:19.473 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-04T07:32:19.516 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-04T07:32:19.517 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-04T07:32:19.539 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:19.575 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:19.869 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-04T07:32:19.884 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:19.918 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:20.180 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package postgresql-common. 2024-03-04T07:32:20.203 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-04T07:32:20.255 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-04T07:32:20.265 INFO:teuthology.orchestra.run.smithi136.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-04T07:32:20.270 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:20.295 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking postgresql-common (238) ... 2024-03-04T07:32:20.313 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-04T07:32:20.641 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-04T07:32:20.656 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:20.715 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-04T07:32:20.763 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package postgresql-14. 2024-03-04T07:32:20.786 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:20.824 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:20.959 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-04T07:32:20.975 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-04T07:32:21.017 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-04T07:32:21.497 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-04T07:32:21.512 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-04T07:32:21.555 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-04T07:32:21.908 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package modemmanager. 2024-03-04T07:32:21.923 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-04T07:32:22.049 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-04T07:32:22.234 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package postgresql. 2024-03-04T07:32:22.257 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-04T07:32:22.300 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking postgresql (14+238) ... 2024-03-04T07:32:22.544 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package postgresql-client. 2024-03-04T07:32:22.566 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-04T07:32:22.610 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking postgresql-client (14+238) ... 2024-03-04T07:32:22.765 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package network-manager. 2024-03-04T07:32:22.782 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-04T07:32:22.831 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-04T07:32:22.854 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-04T07:32:22.877 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-04T07:32:22.929 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-04T07:32:23.199 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package tcl8.6. 2024-03-04T07:32:23.221 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-04T07:32:23.265 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-04T07:32:23.472 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ppp. 2024-03-04T07:32:23.489 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-04T07:32:23.837 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package tcl. 2024-03-04T07:32:23.859 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-04T07:32:23.864 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-04T07:32:23.908 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-04T07:32:24.202 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package pptp-linux. 2024-03-04T07:32:24.217 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-04T07:32:24.240 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package wpasupplicant. 2024-03-04T07:32:24.260 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-04T07:32:24.262 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-04T07:32:24.306 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-04T07:32:24.538 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-04T07:32:24.553 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-04T07:32:24.596 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-04T07:32:24.686 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-04T07:32:24.708 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-04T07:32:24.753 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-04T07:32:24.899 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-04T07:32:24.914 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-04T07:32:24.957 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-client-common (238) ... 2024-03-04T07:32:25.131 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-04T07:32:25.153 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-04T07:32:25.205 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-04T07:32:25.211 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-04T07:32:25.227 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:25.387 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:25.483 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up postgresql-client-common (238) ... 2024-03-04T07:32:25.696 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-04T07:32:25.822 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-04T07:32:25.948 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:25.975 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-common. 2024-03-04T07:32:25.990 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-04T07:32:26.074 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-04T07:32:26.084 INFO:teuthology.orchestra.run.smithi113.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-04T07:32:26.115 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-common (238) ... 2024-03-04T07:32:26.200 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-04T07:32:26.326 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:26.583 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-14. 2024-03-04T07:32:26.597 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-04T07:32:26.634 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:26.885 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-04T07:32:26.964 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-04T07:32:27.360 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-04T07:32:27.361 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-04T07:32:27.803 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql. 2024-03-04T07:32:27.813 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-04T07:32:27.844 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql (14+238) ... 2024-03-04T07:32:27.926 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-04T07:32:28.044 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-04T07:32:28.079 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-client. 2024-03-04T07:32:28.093 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-04T07:32:28.129 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-client (14+238) ... 2024-03-04T07:32:28.170 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-04T07:32:28.305 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-04T07:32:28.340 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-04T07:32:28.354 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-04T07:32:28.431 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-04T07:32:28.431 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-04T07:32:28.556 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-04T07:32:28.691 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-04T07:32:28.759 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package tcl8.6. 2024-03-04T07:32:28.772 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-04T07:32:28.809 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-04T07:32:29.071 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package tcl. 2024-03-04T07:32:29.083 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-04T07:32:29.125 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-04T07:32:29.322 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-04T07:32:29.490 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package wpasupplicant. 2024-03-04T07:32:29.503 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-04T07:32:29.514 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up dump (0.4b47-3) ... 2024-03-04T07:32:29.548 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-04T07:32:29.666 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-04T07:32:29.733 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-04T07:32:29.859 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-04T07:32:29.951 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-04T07:32:29.964 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-04T07:32:29.968 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-04T07:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-04T07:32:30.086 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-04T07:32:30.212 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-04T07:32:30.303 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-04T07:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-04T07:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-04T07:32:30.372 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-04T07:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-04T07:32:30.498 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-04T07:32:30.672 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-client-common (238) ... 2024-03-04T07:32:30.674 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-04T07:32:30.826 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-04T07:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-04T07:32:30.935 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-04T07:32:31.047 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-04T07:32:31.095 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up postgresql-client (14+238) ... 2024-03-04T07:32:31.165 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:31.221 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-04T07:32:31.291 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-04T07:32:31.392 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-04T07:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:31.731 INFO:teuthology.orchestra.run.smithi136.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-04T07:32:31.731 INFO:teuthology.orchestra.run.smithi136.stdout:which means they are currently configured by ifupdown: 2024-03-04T07:32:31.731 INFO:teuthology.orchestra.run.smithi136.stdout:- enp3s0f1 2024-03-04T07:32:31.731 INFO:teuthology.orchestra.run.smithi136.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-04T07:32:31.731 INFO:teuthology.orchestra.run.smithi136.stdout:remove their configuration from /etc/network/interfaces. 2024-03-04T07:32:31.731 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:31.859 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-04T07:32:32.221 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-04T07:32:32.528 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-04T07:32:32.567 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-04T07:32:32.624 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-04T07:32:32.975 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-04T07:32:32.976 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-04T07:32:33.132 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-04T07:32:33.275 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-04T07:32:33.385 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-04T07:32:33.502 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up postgresql-common (238) ... 2024-03-04T07:32:33.660 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-04T07:32:33.786 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-04T07:32:33.937 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-04T07:32:34.063 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-04T07:32:34.189 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-04T07:32:34.290 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-04T07:32:34.391 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-04T07:32:34.836 INFO:teuthology.orchestra.run.smithi136.stdout:Adding user postgres to group ssl-cert 2024-03-04T07:32:35.020 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:35.020 INFO:teuthology.orchestra.run.smithi136.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-04T07:32:35.163 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-04T07:32:35.212 INFO:teuthology.orchestra.run.smithi136.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-04T07:32:35.212 INFO:teuthology.orchestra.run.smithi136.stdout:Removing obsolete dictionary files: 2024-03-04T07:32:35.289 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up dump (0.4b47-3) ... 2024-03-04T07:32:35.368 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-04T07:32:35.424 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-04T07:32:35.576 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-04T07:32:35.702 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-04T07:32:35.828 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-04T07:32:35.836 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-04T07:32:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-04T07:32:36.068 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-04T07:32:36.130 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-04T07:32:36.256 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:36.289 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-04T07:32:36.574 INFO:teuthology.orchestra.run.smithi136.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-04T07:32:36.587 INFO:teuthology.orchestra.run.smithi136.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-04T07:32:36.648 INFO:teuthology.orchestra.run.smithi136.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-04T07:32:36.648 INFO:teuthology.orchestra.run.smithi136.stdout:This user must also own the server process. 2024-03-04T07:32:36.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:36.648 INFO:teuthology.orchestra.run.smithi136.stdout:The database cluster will be initialized with locale "en_US". 2024-03-04T07:32:36.648 INFO:teuthology.orchestra.run.smithi136.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-04T07:32:36.648 INFO:teuthology.orchestra.run.smithi136.stdout:The default text search configuration will be set to "english". 2024-03-04T07:32:36.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:36.648 INFO:teuthology.orchestra.run.smithi136.stdout:Data page checksums are disabled. 2024-03-04T07:32:36.648 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:36.649 INFO:teuthology.orchestra.run.smithi136.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-04T07:32:36.649 INFO:teuthology.orchestra.run.smithi136.stdout:creating subdirectories ... ok 2024-03-04T07:32:36.649 INFO:teuthology.orchestra.run.smithi136.stdout:selecting dynamic shared memory implementation ... posix 2024-03-04T07:32:36.664 INFO:teuthology.orchestra.run.smithi136.stdout:selecting default max_connections ... 100 2024-03-04T07:32:36.685 INFO:teuthology.orchestra.run.smithi136.stdout:selecting default shared_buffers ... 128MB 2024-03-04T07:32:36.695 INFO:teuthology.orchestra.run.smithi136.stdout:selecting default time zone ... Etc/UTC 2024-03-04T07:32:36.696 INFO:teuthology.orchestra.run.smithi136.stdout:creating configuration files ... ok 2024-03-04T07:32:36.800 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-04T07:32:36.811 INFO:teuthology.orchestra.run.smithi136.stdout:running bootstrap script ... ok 2024-03-04T07:32:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-04T07:32:37.077 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-04T07:32:37.245 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-client (14+238) ... 2024-03-04T07:32:37.371 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-04T07:32:37.443 INFO:teuthology.orchestra.run.smithi136.stdout:performing post-bootstrap initialization ... ok 2024-03-04T07:32:37.931 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-04T07:32:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:which means they are currently configured by ifupdown: 2024-03-04T07:32:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:- enp3s0f1 2024-03-04T07:32:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-04T07:32:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:remove their configuration from /etc/network/interfaces. 2024-03-04T07:32:37.931 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:38.063 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-04T07:32:38.230 INFO:teuthology.orchestra.run.smithi136.stdout:syncing data to disk ... ok 2024-03-04T07:32:38.478 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-04T07:32:38.652 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-04T07:32:38.886 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-04T07:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-04T07:32:39.724 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-04T07:32:39.826 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-04T07:32:39.935 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-common (238) ... 2024-03-04T07:32:40.915 INFO:teuthology.orchestra.run.smithi113.stdout:Adding user postgres to group ssl-cert 2024-03-04T07:32:41.169 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:41.170 INFO:teuthology.orchestra.run.smithi113.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-04T07:32:41.421 INFO:teuthology.orchestra.run.smithi113.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-04T07:32:41.421 INFO:teuthology.orchestra.run.smithi113.stdout:Removing obsolete dictionary files: 2024-03-04T07:32:41.487 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-04T07:32:41.614 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-04T07:32:41.740 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up postgresql (14+238) ... 2024-03-04T07:32:41.866 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-04T07:32:41.992 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-04T07:32:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-04T07:32:42.248 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-04T07:32:42.248 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-04T07:32:42.603 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-04T07:32:42.820 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-04T07:32:42.954 INFO:teuthology.orchestra.run.smithi113.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-04T07:32:42.973 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-04T07:32:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-04T07:32:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:This user must also own the server process. 2024-03-04T07:32:43.027 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:The database cluster will be initialized with locale "en_US". 2024-03-04T07:32:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-04T07:32:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:The default text search configuration will be set to "english". 2024-03-04T07:32:43.028 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:43.028 INFO:teuthology.orchestra.run.smithi113.stdout:Data page checksums are disabled. 2024-03-04T07:32:43.028 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:43.028 INFO:teuthology.orchestra.run.smithi113.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-04T07:32:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:creating subdirectories ... ok 2024-03-04T07:32:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:selecting dynamic shared memory implementation ... posix 2024-03-04T07:32:43.052 INFO:teuthology.orchestra.run.smithi113.stdout:selecting default max_connections ... 100 2024-03-04T07:32:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:selecting default shared_buffers ... 128MB 2024-03-04T07:32:43.099 INFO:teuthology.orchestra.run.smithi113.stdout:selecting default time zone ... Etc/UTC 2024-03-04T07:32:43.100 INFO:teuthology.orchestra.run.smithi113.stdout:creating configuration files ... ok 2024-03-04T07:32:43.122 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T07:32:43.263 INFO:teuthology.orchestra.run.smithi113.stdout:running bootstrap script ... ok 2024-03-04T07:32:43.274 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T07:32:43.899 INFO:teuthology.orchestra.run.smithi113.stdout:performing post-bootstrap initialization ... ok 2024-03-04T07:32:44.454 INFO:teuthology.orchestra.run.smithi113.stdout:syncing data to disk ... ok 2024-03-04T07:32:44.593 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-04T07:32:44.719 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-04T07:32:44.956 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-04T07:32:45.584 INFO:teuthology.orchestra.run.smithi136.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T07:32:45.589 DEBUG:teuthology.parallel:result is None 2024-03-04T07:32:48.035 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-04T07:32:48.161 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-04T07:32:48.287 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql (14+238) ... 2024-03-04T07:32:48.397 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-04T07:32:48.498 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-04T07:32:48.741 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-04T07:32:48.741 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-04T07:32:49.517 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-04T07:32:49.819 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T07:32:49.971 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T07:32:51.724 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-04T07:32:51.849 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-04T07:32:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T07:32:52.381 DEBUG:teuthology.parallel:result is None 2024-03-04T07:32:52.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T07:32:52.597 DEBUG:teuthology.orchestra.run.smithi113:> dpkg-query -W -f '${Version}' ceph 2024-03-04T07:32:52.618 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-1739-g1bc250c7-1jammy 2024-03-04T07:32:52.619 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739-g1bc250c7-1jammy 2024-03-04T07:32:52.619 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739-g1bc250c7-1jammy is installed. 2024-03-04T07:32:52.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T07:32:52.838 DEBUG:teuthology.orchestra.run.smithi136:> dpkg-query -W -f '${Version}' ceph 2024-03-04T07:32:52.857 INFO:teuthology.orchestra.run.smithi136.stdout:19.0.0-1739-g1bc250c7-1jammy 2024-03-04T07:32:52.858 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1739-g1bc250c7-1jammy 2024-03-04T07:32:52.858 INFO:teuthology.task.install:The correct ceph version 19.0.0-1739-g1bc250c7-1jammy is installed. 2024-03-04T07:32:52.859 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-04T07:32:52.860 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:52.860 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T07:32:52.880 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:32:52.880 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T07:32:52.914 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-04T07:32:52.914 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:52.914 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T07:32:52.942 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T07:32:53.004 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:32:53.005 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T07:32:53.018 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T07:32:53.072 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-04T07:32:53.073 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:53.073 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T07:32:53.091 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T07:32:53.154 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:32:53.154 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T07:32:53.167 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T07:32:53.226 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-04T07:32:53.227 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:53.227 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T07:32:53.245 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T07:32:53.305 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:32:53.306 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T07:32:53.318 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T07:32:53.373 INFO:teuthology.run_tasks:Running task ceph... 2024-03-04T07:32:53.456 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-04T07:32:53.457 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /var/log/ceph 2024-03-04T07:32:53.460 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /var/log/ceph 2024-03-04T07:32:53.475 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-04T07:32:53.475 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-04T07:32:53.522 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-04T07:32:53.537 INFO:tasks.ceph:Creating extra log directories... 2024-03-04T07:32:53.537 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-04T07:32:53.586 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-04T07:32:53.609 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-04T07:32:53.609 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-04T07:32:53.609 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-04T07:32:53.609 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:53.609 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-04T07:32:53.652 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-04T07:32:53.743 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:32:53.743 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-04T07:32:53.749 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-04T07:32:53.832 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-04T07:32:53.832 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-04T07:32:53.832 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671', 'branch': 'wip-lusov-qdb', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}}', 'email': None, 'first_in_suite': False, 'job_id': '7580671', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-lusov-qdb'}, 'ceph': {'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'Replacing daemon mds'], 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-qdb', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 5072, 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'sleep_before_teardown': 0, 'subset': '246/10000', 'suite': 'fs', 'suite_branch': 'wip-lusov-qdb', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi113.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBmgs9EG1bFVgO8kH7WX4asSSzQG1qyixcVqkKDDu/xE7ar5Bhw6/tXfttArGJLoicuMCFuzGZILnHVQy85oZMw=', 'smithi136.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpg8Cq3BIkk2zVUoyQhy1vZvBoVz0qwOK3jmJBEFlxWyAijHvNwPk1r1ggibX/1BPQ1/s/AHDBSaPaBhWHGLIk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'Replacing daemon mds'], 'sha1': '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd', 'cluster': 'ceph'}}, {'kclient': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b1dac5519c57c269ea98a22fb7729016a1d0e4d2', 'timestamp': '2024-03-04_07:05:03', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-04T07:32:53.833 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-04T07:32:53.836 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-04T07:32:53.874 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/run/ceph 2024-03-04T07:32:53.886 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/run/ceph 2024-03-04T07:32:53.929 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:53.930 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T07:32:53.951 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T07:32:53.951 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-03-04T07:32:53.999 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T07:32:54.000 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T07:32:54.000 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 885 Links: 1 2024-03-04T07:32:54.000 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T07:32:54.000 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-03-04 07:26:20.360372006 +0000 2024-03-04T07:32:54.000 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-03-04 07:26:20.088374437 +0000 2024-03-04T07:32:54.000 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-03-04 07:26:20.088374437 +0000 2024-03-04T07:32:54.000 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-03-04 07:26:20.088374437 +0000 2024-03-04T07:32:54.000 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T07:32:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-03-04T07:32:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-03-04T07:32:54.057 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.00040258 s, 1.3 MB/s 2024-03-04T07:32:54.058 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T07:32:54.109 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-03-04T07:32:54.163 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T07:32:54.163 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T07:32:54.163 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 902 Links: 1 2024-03-04T07:32:54.163 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T07:32:54.163 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-03-04 07:26:20.900367183 +0000 2024-03-04T07:32:54.164 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-03-04 07:26:20.648369434 +0000 2024-03-04T07:32:54.164 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-03-04 07:26:20.648369434 +0000 2024-03-04T07:32:54.164 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-03-04 07:26:20.648369434 +0000 2024-03-04T07:32:54.164 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T07:32:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-03-04T07:32:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-03-04T07:32:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.00034912 s, 1.5 MB/s 2024-03-04T07:32:54.222 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T07:32:54.273 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-03-04T07:32:54.323 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T07:32:54.323 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T07:32:54.324 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 918 Links: 1 2024-03-04T07:32:54.324 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T07:32:54.324 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-03-04 07:26:21.412362610 +0000 2024-03-04T07:32:54.324 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-03-04 07:26:21.148364967 +0000 2024-03-04T07:32:54.324 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-03-04 07:26:21.148364967 +0000 2024-03-04T07:32:54.324 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-03-04 07:26:21.148364967 +0000 2024-03-04T07:32:54.324 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T07:32:54.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-03-04T07:32:54.380 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-03-04T07:32:54.380 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000366534 s, 1.4 MB/s 2024-03-04T07:32:54.382 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T07:32:54.433 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-03-04T07:32:54.483 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T07:32:54.483 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T07:32:54.483 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 936 Links: 1 2024-03-04T07:32:54.484 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T07:32:54.484 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-03-04 07:26:21.932357965 +0000 2024-03-04T07:32:54.484 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-03-04 07:26:21.664360359 +0000 2024-03-04T07:32:54.484 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-03-04 07:26:21.664360359 +0000 2024-03-04T07:32:54.484 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-03-04 07:26:21.664360359 +0000 2024-03-04T07:32:54.484 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T07:32:54.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-03-04T07:32:54.540 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-03-04T07:32:54.540 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000672856 s, 761 kB/s 2024-03-04T07:32:54.542 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T07:32:54.592 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T07:32:54.593 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:32:54.593 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T07:32:54.599 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T07:32:54.599 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-03-04T07:32:54.648 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T07:32:54.648 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T07:32:54.648 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-03-04T07:32:54.648 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T07:32:54.648 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-03-04 07:26:01.091487121 +0000 2024-03-04T07:32:54.648 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-03-04 07:26:00.859489193 +0000 2024-03-04T07:32:54.648 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-03-04 07:26:00.859489193 +0000 2024-03-04T07:32:54.649 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-03-04T07:32:54.649 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T07:32:54.700 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-03-04T07:32:54.700 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-03-04T07:32:54.700 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000595419 s, 860 kB/s 2024-03-04T07:32:54.702 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T07:32:54.751 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-03-04T07:32:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T07:32:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T07:32:54.802 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-03-04T07:32:54.802 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T07:32:54.802 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-03-04 07:26:01.607482512 +0000 2024-03-04T07:32:54.802 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-03-04 07:26:01.335484941 +0000 2024-03-04T07:32:54.802 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-03-04 07:26:01.335484941 +0000 2024-03-04T07:32:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-03-04T07:32:54.803 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T07:32:54.855 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-03-04T07:32:54.855 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-03-04T07:32:54.856 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000576333 s, 888 kB/s 2024-03-04T07:32:54.857 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T07:32:54.908 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-03-04T07:32:54.953 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T07:32:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T07:32:54.954 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-03-04T07:32:54.954 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T07:32:54.954 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-03-04 07:26:02.295476366 +0000 2024-03-04T07:32:54.954 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-03-04 07:26:01.851480333 +0000 2024-03-04T07:32:54.954 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-03-04 07:26:01.851480333 +0000 2024-03-04T07:32:54.954 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-03-04T07:32:54.954 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T07:32:55.007 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-03-04T07:32:55.007 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-03-04T07:32:55.007 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000584392 s, 876 kB/s 2024-03-04T07:32:55.008 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T07:32:55.055 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-03-04T07:32:55.101 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T07:32:55.101 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T07:32:55.101 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 940 Links: 1 2024-03-04T07:32:55.102 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T07:32:55.102 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-03-04 07:26:02.823471649 +0000 2024-03-04T07:32:55.102 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-03-04 07:26:02.539474187 +0000 2024-03-04T07:32:55.102 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-03-04 07:26:02.539474187 +0000 2024-03-04T07:32:55.102 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-03-04T07:32:55.102 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T07:32:55.155 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-03-04T07:32:55.156 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-03-04T07:32:55.156 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.00048379 s, 1.1 MB/s 2024-03-04T07:32:55.157 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T07:32:55.206 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T07:32:55.207 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-04T07:32:55.207 INFO:tasks.ceph:Generating config... 2024-03-04T07:32:55.208 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-04T07:32:55.208 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-03-04T07:32:55.208 INFO:tasks.ceph:[client] debug client = 20 2024-03-04T07:32:55.208 INFO:tasks.ceph:[client] debug ms = 1 2024-03-04T07:32:55.208 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-04T07:32:55.208 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-04T07:32:55.208 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-04T07:32:55.208 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-04T07:32:55.208 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-04T07:32:55.208 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-04T07:32:55.208 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = False 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-04T07:32:55.209 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-04T07:32:55.210 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-04T07:32:55.210 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-04T07:32:55.210 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-04T07:32:55.210 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-04T07:32:55.210 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-04T07:32:55.210 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-04T07:32:55.210 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-04T07:32:55.210 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-04T07:32:55.210 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-04T07:32:55.210 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-04T07:32:55.210 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-04T07:32:55.210 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-04T07:32:55.210 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-04T07:32:55.210 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-04T07:32:55.210 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-04T07:32:55.211 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-04T07:32:55.212 INFO:tasks.ceph:Setting up mon.a... 2024-03-04T07:32:55.212 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-04T07:32:55.253 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.keyring 2024-03-04T07:32:55.271 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-04T07:32:55.357 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T07:32:55.376 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.113'), ('mon.b', '172.21.15.136'), ('mon.c', '[v2:172.21.15.136:3301,v1:172.21.15.136:6790]')] 2024-03-04T07:32:55.377 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.113,172.21.15.136,[v2:172.21.15.136:3301,v1:172.21.15.136:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': False, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-04T07:32:55.378 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:55.378 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-04T07:32:55.427 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.113 --add b 172.21.15.136 --addv c '[v2:172.21.15.136:3301,v1:172.21.15.136:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:32:55.513 INFO:teuthology.orchestra.run.smithi113.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:32:55.513 INFO:teuthology.orchestra.run.smithi113.stdout:monmaptool: generated fsid 43451c9f-4d9f-4f78-a5ec-a82e7d344007 2024-03-04T07:32:55.513 INFO:teuthology.orchestra.run.smithi113.stdout:setting min_mon_release = quincy 2024-03-04T07:32:55.513 INFO:teuthology.orchestra.run.smithi113.stdout:epoch 0 2024-03-04T07:32:55.513 INFO:teuthology.orchestra.run.smithi113.stdout:fsid 43451c9f-4d9f-4f78-a5ec-a82e7d344007 2024-03-04T07:32:55.513 INFO:teuthology.orchestra.run.smithi113.stdout:last_changed 2024-03-04T07:32:55.517303+0000 2024-03-04T07:32:55.513 INFO:teuthology.orchestra.run.smithi113.stdout:created 2024-03-04T07:32:55.517303+0000 2024-03-04T07:32:55.513 INFO:teuthology.orchestra.run.smithi113.stdout:min_mon_release 17 (quincy) 2024-03-04T07:32:55.513 INFO:teuthology.orchestra.run.smithi113.stdout:election_strategy: 1 2024-03-04T07:32:55.514 INFO:teuthology.orchestra.run.smithi113.stdout:0: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.a 2024-03-04T07:32:55.514 INFO:teuthology.orchestra.run.smithi113.stdout:1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-03-04T07:32:55.514 INFO:teuthology.orchestra.run.smithi113.stdout:2: [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] mon.c 2024-03-04T07:32:55.514 INFO:teuthology.orchestra.run.smithi113.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-04T07:32:55.516 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-04T07:32:55.523 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 43451c9f-4d9f-4f78-a5ec-a82e7d344007... 2024-03-04T07:32:55.524 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-04T07:32:55.570 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-04T07:32:55.600 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2024-03-04T07:32:55.600 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2024-03-04T07:32:55.600 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-04T07:32:55.600 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2024-03-04T07:32:55.600 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.600 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2024-03-04T07:32:55.600 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.601 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2024-03-04T07:32:55.602 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: mon_allow_pool_size_one = true 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2024-03-04T07:32:55.603 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.113,172.21.15.136,[v2:172.21.15.136:3301,v1:172.21.15.136:6790]" 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on skipped message = False 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = 43451c9f-4d9f-4f78-a5ec-a82e7d344007 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 600 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: osd mclock profile = high_recovery_ops 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.604 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug trim objects = true 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2024-03-04T07:32:55.605 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: bdev async discard = True 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: bdev enable discard = True 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore allocator = bitmap 2024-03-04T07:32:55.606 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 1/20 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 1/20 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 20 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 4/10 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: mon client directed command retry = 5 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject delay max = 1 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject delay probability = 0.005 2024-03-04T07:32:55.607 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject delay type = client mds 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject socket failures = 2500 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: osd blocked scrub grace period = 3600 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: osd max markdown count = 1000 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: osd op complaint time = 180 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-03-04T07:32:55.608 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2024-03-04T07:32:55.609 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: auth mon ticket ttl = 660 # 11m 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: auth service ticket ttl = 240 # 4m 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: # don't complain about insecure global_id in the test suite 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: # 1m isn't quite enough 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: mon_down_mkfs_grace = 2m 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.610 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_filestore_osds = false 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout: mon op complaint time = 120 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout: client mount timeout = 600 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout: client_shutdown_timeout = 120 2024-03-04T07:32:55.611 INFO:teuthology.orchestra.run.smithi136.stdout: debug client = 20 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout: rados mon op timeout = 900 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout: rados osd op timeout = 900 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.a] 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.c] 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.e] 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.b] 2024-03-04T07:32:55.612 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.d] 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.f] 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout:[mds] 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout: debug mds = 20 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout: debug mds balancer = 20 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal fragment size max = 10000 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal merge size = 5 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal split bits = 3 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout: mds bal split size = 100 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout: mds debug frag = True 2024-03-04T07:32:55.613 INFO:teuthology.orchestra.run.smithi136.stdout: mds debug scatterstat = True 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: mds oft prefetch dirfrags = False 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: mds op complaint time = 180 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: mds verify scatter = True 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: mds_dir_prefetch = True 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: mds_sleep_rank_change = 5000000.0 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: mon client directed command retry = 5 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject delay max = 1 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject delay probability = 0.005 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject delay type = client mds osd 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject socket failures = 2500 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: osd op complaint time = 180 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: rados mon op timeout = 900 2024-03-04T07:32:55.614 INFO:teuthology.orchestra.run.smithi136.stdout: rados osd op timeout = 900 2024-03-04T07:32:55.615 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-03-04T07:32:55.615 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-03-04T07:32:55.615 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-04T07:32:55.615 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-03-04T07:32:55.615 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.615 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-04T07:32:55.616 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-03-04T07:32:55.617 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-03-04T07:32:55.618 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.113,172.21.15.136,[v2:172.21.15.136:3301,v1:172.21.15.136:6790]" 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on skipped message = False 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = 43451c9f-4d9f-4f78-a5ec-a82e7d344007 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-03-04T07:32:55.619 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-03-04T07:32:55.620 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.620 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-03-04T07:32:55.620 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-03-04T07:32:55.620 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-03-04T07:32:55.620 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2024-03-04T07:32:55.620 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.620 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-03-04T07:32:55.620 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-03-04T07:32:55.620 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.620 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: bdev async discard = True 2024-03-04T07:32:55.621 INFO:teuthology.orchestra.run.smithi113.stdout: bdev enable discard = True 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore allocator = bitmap 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 1/20 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 1/20 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 4/10 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: mon client directed command retry = 5 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-03-04T07:32:55.622 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: ms inject delay max = 1 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: ms inject delay probability = 0.005 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: ms inject delay type = client mds 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: ms inject socket failures = 2500 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: osd blocked scrub grace period = 3600 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: osd max markdown count = 1000 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.623 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-03-04T07:32:55.624 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.625 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-03-04T07:32:55.626 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout: client_shutdown_timeout = 120 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-03-04T07:32:55.627 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.e] 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.f] 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout: mds bal fragment size max = 10000 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout: mds bal merge size = 5 2024-03-04T07:32:55.628 INFO:teuthology.orchestra.run.smithi113.stdout: mds bal split bits = 3 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: mds bal split size = 100 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = True 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: mds oft prefetch dirfrags = False 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = True 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: mds_dir_prefetch = True 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: mds_sleep_rank_change = 5000000.0 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: mon client directed command retry = 5 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: ms inject delay max = 1 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: ms inject delay probability = 0.005 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: ms inject delay type = client mds osd 2024-03-04T07:32:55.629 INFO:teuthology.orchestra.run.smithi113.stdout: ms inject socket failures = 2500 2024-03-04T07:32:55.630 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-03-04T07:32:55.630 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-03-04T07:32:55.630 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-03-04T07:32:55.631 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-04T07:32:55.632 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-04T07:32:55.727 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-04T07:32:55.728 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:55.728 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-04T07:32:55.736 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:55.736 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-04T07:32:55.787 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T07:32:55.787 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:55.788 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-04T07:32:55.788 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T07:32:55.858 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:32:55.858 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:32:55.907 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T07:32:55.907 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:32:55.907 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-04T07:32:55.907 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-04T07:32:55.929 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:32:55.929 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:32:55.978 INFO:tasks.ceph:Setting up mon nodes... 2024-03-04T07:32:55.978 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-04T07:32:55.978 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-04T07:32:56.031 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-04T07:32:56.036 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-04T07:32:56.079 INFO:teuthology.orchestra.run.smithi136.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-04T07:32:56.093 INFO:tasks.ceph:Setting up mds nodes... 2024-03-04T07:32:56.093 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-04T07:32:56.147 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-04T07:32:56.151 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-04T07:32:56.170 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-04T07:32:56.264 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-04T07:32:56.268 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-04T07:32:56.288 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-03-04T07:32:56.388 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-03-04T07:32:56.392 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-03-04T07:32:56.412 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-04T07:32:56.457 INFO:teuthology.orchestra.run.smithi136.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-04T07:32:56.460 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-04T07:32:56.472 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-04T07:32:56.556 INFO:teuthology.orchestra.run.smithi136.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-04T07:32:56.559 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-04T07:32:56.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-03-04T07:32:56.702 INFO:teuthology.orchestra.run.smithi136.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-03-04T07:32:56.705 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-03-04T07:32:56.766 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-04T07:32:56.766 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-04T07:32:56.808 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-04T07:32:56.824 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-04T07:32:56.824 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-04T07:32:56.825 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-04T07:32:56.886 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T07:32:56.886 INFO:tasks.ceph:role: osd.0 2024-03-04T07:32:56.887 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T07:32:56.887 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-04T07:32:56.957 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T07:32:56.957 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T07:32:56.958 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T07:32:56.958 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T07:32:56.958 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T07:32:56.958 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-03-04T07:32:56.958 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T07:32:56.958 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T07:32:56.958 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T07:32:56.958 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T07:32:57.458 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-03-04T07:32:57.461 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-03-04T07:32:57.461 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-04T07:32:57.571 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-04T07:32:57.586 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T07:32:57.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:32:57.587 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-04T07:32:57.646 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T07:32:57.646 INFO:tasks.ceph:role: osd.1 2024-03-04T07:32:57.647 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T07:32:57.647 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-04T07:32:57.714 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T07:32:57.714 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T07:32:57.714 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T07:32:57.714 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T07:32:57.714 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T07:32:57.714 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-03-04T07:32:57.714 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T07:32:57.714 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T07:32:57.714 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T07:32:57.714 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T07:32:58.164 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-03-04T07:32:58.167 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-03-04T07:32:58.167 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-04T07:32:58.275 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-04T07:32:58.290 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T07:32:58.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:32:58.292 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-04T07:32:58.354 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T07:32:58.355 INFO:tasks.ceph:role: osd.2 2024-03-04T07:32:58.355 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T07:32:58.355 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-04T07:32:58.422 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T07:32:58.423 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T07:32:58.423 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T07:32:58.423 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T07:32:58.423 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T07:32:58.423 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-03-04T07:32:58.423 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T07:32:58.423 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T07:32:58.423 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T07:32:58.423 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T07:32:58.940 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-03-04T07:32:58.942 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-03-04T07:32:58.943 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-04T07:32:59.055 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-04T07:32:59.071 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T07:32:59.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:32:59.072 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-04T07:32:59.135 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-04T07:32:59.136 INFO:tasks.ceph:role: osd.3 2024-03-04T07:32:59.136 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T07:32:59.136 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-04T07:32:59.205 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T07:32:59.205 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T07:32:59.205 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T07:32:59.205 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T07:32:59.205 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T07:32:59.205 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-03-04T07:32:59.205 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T07:32:59.205 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T07:32:59.205 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T07:32:59.205 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T07:32:59.699 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-03-04T07:32:59.702 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-03-04T07:32:59.702 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-04T07:32:59.817 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-04T07:32:59.833 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T07:32:59.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:32:59.834 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:32:59.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:32:59.966+0000 7f5b0356e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-04T07:32:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:32:59.970+0000 7f5b0356e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-04T07:32:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:32:59.970+0000 7f5b0356e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T07:32:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:32:59.970+0000 7f5b0356e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T07:32:59.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:32:59.970+0000 7f5b0356e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-04T07:32:59.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:32:59.974+0000 7f5b0356e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-04T07:33:02.962 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-04T07:33:02.982 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:33:03.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:03.078+0000 7f2e3ae707c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-04T07:33:03.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:03.078+0000 7f2e3ae707c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-04T07:33:03.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:03.078+0000 7f2e3ae707c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T07:33:03.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:03.078+0000 7f2e3ae707c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T07:33:03.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:03.078+0000 7f2e3ae707c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-04T07:33:03.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:03.078+0000 7f2e3ae707c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-04T07:33:06.042 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-04T07:33:06.062 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:33:06.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:06.154+0000 7f12870547c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-04T07:33:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:06.154+0000 7f12870547c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-04T07:33:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:06.154+0000 7f12870547c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T07:33:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:06.154+0000 7f12870547c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T07:33:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:06.154+0000 7f12870547c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-04T07:33:06.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:06.154+0000 7f12870547c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-04T07:33:09.118 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-04T07:33:09.137 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:33:09.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:09.230+0000 7f89b2dda7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-04T07:33:09.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:09.230+0000 7f89b2dda7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-04T07:33:09.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:09.230+0000 7f89b2dda7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T07:33:09.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:09.230+0000 7f89b2dda7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T07:33:09.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:09.230+0000 7f89b2dda7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-04T07:33:09.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:09.230+0000 7f89b2dda7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-04T07:33:12.310 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-04T07:33:12.330 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-04T07:33:12.345 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T07:33:12.345 INFO:tasks.ceph:role: osd.4 2024-03-04T07:33:12.345 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T07:33:12.345 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-04T07:33:12.409 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T07:33:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T07:33:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T07:33:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T07:33:12.409 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T07:33:12.409 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-03-04T07:33:12.409 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T07:33:12.409 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T07:33:12.410 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T07:33:12.410 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T07:33:12.919 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-03-04T07:33:12.921 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-03-04T07:33:12.921 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-04T07:33:13.057 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-04T07:33:13.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:33:13.068 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T07:33:13.068 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-04T07:33:13.120 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T07:33:13.120 INFO:tasks.ceph:role: osd.5 2024-03-04T07:33:13.120 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T07:33:13.120 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-04T07:33:13.184 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T07:33:13.184 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T07:33:13.184 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T07:33:13.184 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T07:33:13.185 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T07:33:13.185 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-03-04T07:33:13.185 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T07:33:13.185 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T07:33:13.185 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T07:33:13.185 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T07:33:13.600 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-03-04T07:33:13.602 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-03-04T07:33:13.602 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-04T07:33:13.662 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-04T07:33:13.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:33:13.718 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T07:33:13.718 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-04T07:33:13.771 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T07:33:13.771 INFO:tasks.ceph:role: osd.6 2024-03-04T07:33:13.771 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T07:33:13.771 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-04T07:33:13.833 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T07:33:13.833 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T07:33:13.833 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T07:33:13.833 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T07:33:13.833 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T07:33:13.833 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-03-04T07:33:13.833 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T07:33:13.833 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T07:33:13.833 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T07:33:13.833 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T07:33:14.326 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-03-04T07:33:14.328 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-03-04T07:33:14.328 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-04T07:33:14.433 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-04T07:33:14.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:33:14.443 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T07:33:14.444 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-04T07:33:14.494 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-04T07:33:14.494 INFO:tasks.ceph:role: osd.7 2024-03-04T07:33:14.495 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T07:33:14.495 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-04T07:33:14.558 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-04T07:33:14.559 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-04T07:33:14.559 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-04T07:33:14.559 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-04T07:33:14.559 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-04T07:33:14.559 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-03-04T07:33:14.559 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-04T07:33:14.559 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-04T07:33:14.559 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-04T07:33:14.559 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-04T07:33:15.060 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-03-04T07:33:15.062 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-03-04T07:33:15.062 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-04T07:33:15.174 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-04T07:33:15.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:33:15.187 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-03-04T07:33:15.188 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:33:15.301 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:15.302+0000 7f03cb92c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-04T07:33:15.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:15.302+0000 7f03cb92c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-04T07:33:15.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:15.302+0000 7f03cb92c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T07:33:15.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:15.302+0000 7f03cb92c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T07:33:15.303 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:15.302+0000 7f03cb92c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-04T07:33:15.307 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:15.306+0000 7f03cb92c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-04T07:33:18.309 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-04T07:33:18.323 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:33:18.407 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:18.406+0000 7fc7415267c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-04T07:33:18.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:18.406+0000 7fc7415267c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-04T07:33:18.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:18.406+0000 7fc7415267c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T07:33:18.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:18.406+0000 7fc7415267c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T07:33:18.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:18.406+0000 7fc7415267c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-04T07:33:18.408 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:18.406+0000 7fc7415267c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-04T07:33:21.383 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-04T07:33:21.442 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:33:21.523 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:21.522+0000 7f3f061437c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-04T07:33:21.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:21.522+0000 7f3f061437c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-04T07:33:21.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:21.522+0000 7f3f061437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T07:33:21.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:21.522+0000 7f3f061437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T07:33:21.524 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:21.522+0000 7f3f061437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-04T07:33:21.525 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:21.522+0000 7f3f061437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-04T07:33:23.834 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:33:23.838 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:33:24.525 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-04T07:33:24.539 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:33:24.619 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:24.618+0000 7f65c79cd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-04T07:33:24.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:24.618+0000 7f65c79cd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-04T07:33:24.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:24.618+0000 7f65c79cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T07:33:24.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:24.618+0000 7f65c79cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T07:33:24.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:24.618+0000 7f65c79cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-04T07:33:24.621 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:24.622+0000 7f65c79cd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-04T07:33:27.585 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-04T07:33:27.642 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-04T07:33:27.642 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:27.642 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-04T07:33:27.664 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:27.664 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-04T07:33:27.726 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:27.726 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-04T07:33:27.789 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:27.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-03-04T07:33:27.857 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:27.858 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-04T07:33:27.925 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:27.925 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-04T07:33:27.985 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:27.985 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-04T07:33:28.053 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:28.054 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-04T07:33:28.117 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:28.117 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-04T07:33:28.131 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:28.131 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-04T07:33:28.186 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:28.186 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-04T07:33:28.244 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:28.244 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-03-04T07:33:28.296 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:28.296 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-04T07:33:28.348 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:28.348 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-04T07:33:28.401 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:28.401 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-04T07:33:28.457 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:28.457 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-04T07:33:28.517 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:28.517 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-04T07:33:28.526 INFO:tasks.ceph:Adding keys to all mons... 2024-03-04T07:33:28.526 DEBUG:teuthology.orchestra.run.smithi113:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-04T07:33:28.572 DEBUG:teuthology.orchestra.run.smithi136:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-04T07:33:28.583 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.y] 2024-03-04T07:33:28.583 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAoeeVl7gAcAhAAp06m8EaqO2EHurC2HPwWyA== 2024-03-04T07:33:28.583 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.a] 2024-03-04T07:33:28.583 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAoeeVlA58DCRAAcfLraoT/j4Zf7iKZKuyWOg== 2024-03-04T07:33:28.583 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.c] 2024-03-04T07:33:28.583 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAoeeVlaTwBEBAAU6EMdNJBHUFOxn7ORaBJ8Q== 2024-03-04T07:33:28.583 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.e] 2024-03-04T07:33:28.583 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAoeeVlFM1nFxAAheyWce2h5X5Q1aRKTaz7cA== 2024-03-04T07:33:28.583 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.0] 2024-03-04T07:33:28.583 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAreeVlssTxORAAbpdPu44q6OvidOVtxnZiaw== 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.1] 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAveeVlpiTkBBAAIeCkGMbTTne+Yv4xNyuzCg== 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.2] 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAyeeVlDE1mCRAAwBmsheYi2/VoKaFMLY64Lg== 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.3] 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA1eeVlk1HoDRAAqLVbgbLxzajV0AF8GdvSuQ== 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.x] 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAoeeVllRTXBBAAyL4yDzv5o17hLkvSO1T1cg== 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.b] 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAoeeVlPshkGxAAT+GJaRbczvuWo825Sy7Bzg== 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.d] 2024-03-04T07:33:28.584 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAoeeVlUDBHIRAADewPdRzM+F8CaSwuoc9MIQ== 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout:[mds.f] 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAoeeVlqNQFKhAACoXejsVcGah5BJ5YiFuakg== 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.4] 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA7eeVl/MUWEhAActh/TkpUJwYUrzDjBFOxyw== 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.5] 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA+eeVlTkBqGBAAF6ajfcaWjRUBn7fKOHSxeQ== 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.6] 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBBeeVllVRSHxAAwOl7m+sBfDaVIx/a8q6O6w== 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.7] 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBEeeVlRCYQJRAAcrJ+FdQD0ftfb3Xy+uSFvQ== 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout:[client.0] 2024-03-04T07:33:28.585 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAoeeVlK9xrMBAADeYq8VD4knXvlv79Vf62yg== 2024-03-04T07:33:28.588 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.y] 2024-03-04T07:33:28.588 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAoeeVl7gAcAhAAp06m8EaqO2EHurC2HPwWyA== 2024-03-04T07:33:28.588 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-03-04T07:33:28.588 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAoeeVlA58DCRAAcfLraoT/j4Zf7iKZKuyWOg== 2024-03-04T07:33:28.588 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-03-04T07:33:28.588 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAoeeVlaTwBEBAAU6EMdNJBHUFOxn7ORaBJ8Q== 2024-03-04T07:33:28.588 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.e] 2024-03-04T07:33:28.588 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAoeeVlFM1nFxAAheyWce2h5X5Q1aRKTaz7cA== 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.0] 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAreeVlssTxORAAbpdPu44q6OvidOVtxnZiaw== 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.1] 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAveeVlpiTkBBAAIeCkGMbTTne+Yv4xNyuzCg== 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.2] 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAyeeVlDE1mCRAAwBmsheYi2/VoKaFMLY64Lg== 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.3] 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQA1eeVlk1HoDRAAqLVbgbLxzajV0AF8GdvSuQ== 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.x] 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAoeeVllRTXBBAAyL4yDzv5o17hLkvSO1T1cg== 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAoeeVlPshkGxAAT+GJaRbczvuWo825Sy7Bzg== 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAoeeVlUDBHIRAADewPdRzM+F8CaSwuoc9MIQ== 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.f] 2024-03-04T07:33:28.589 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAoeeVlqNQFKhAACoXejsVcGah5BJ5YiFuakg== 2024-03-04T07:33:28.590 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.4] 2024-03-04T07:33:28.590 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQA7eeVl/MUWEhAActh/TkpUJwYUrzDjBFOxyw== 2024-03-04T07:33:28.590 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.5] 2024-03-04T07:33:28.590 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQA+eeVlTkBqGBAAF6ajfcaWjRUBn7fKOHSxeQ== 2024-03-04T07:33:28.590 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.6] 2024-03-04T07:33:28.590 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBBeeVllVRSHxAAwOl7m+sBfDaVIx/a8q6O6w== 2024-03-04T07:33:28.590 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.7] 2024-03-04T07:33:28.590 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBEeeVlRCYQJRAAcrJ+FdQD0ftfb3Xy+uSFvQ== 2024-03-04T07:33:28.590 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-03-04T07:33:28.590 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAoeeVlK9xrMBAADeYq8VD4knXvlv79Vf62yg== 2024-03-04T07:33:28.591 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T07:33:28.637 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T07:33:28.691 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:28.694 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:28.773 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:28.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:28.829 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:28.832 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:28.903 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:28.906 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:28.989 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:28.992 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.160 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.163 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.213 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.216 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.287 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T07:33:29.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-04T07:33:29.369 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:29.372 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:29.457 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:29.460 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:29.543 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:29.546 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-04T07:33:29.629 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.632 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.718 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.721 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.771 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.774 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.840 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.843 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-04T07:33:29.920 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T07:33:29.923 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-04T07:33:30.002 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-04T07:33:30.002 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-04T07:33:30.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T07:33:30.129 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-04T07:33:30.148 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-04T07:33:30.162 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T07:33:30.254 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-04T07:33:30.264 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-04T07:33:30.314 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-04T07:33:30.404 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-04T07:33:30.414 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:33:30.417 DEBUG:teuthology.orchestra.run.smithi136:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-04T07:33:30.461 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-04T07:33:30.461 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-04T07:33:30.461 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-04T07:33:30.468 INFO:tasks.ceph.mon.a:Started 2024-03-04T07:33:30.468 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-04T07:33:30.468 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-04T07:33:30.507 INFO:tasks.ceph.mon.b:Started 2024-03-04T07:33:30.507 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-04T07:33:30.507 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-04T07:33:30.509 INFO:tasks.ceph.mon.c:Started 2024-03-04T07:33:30.509 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-04T07:33:30.509 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-04T07:33:30.510 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-04T07:33:30.512 INFO:tasks.ceph.mgr.y:Started 2024-03-04T07:33:30.513 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-04T07:33:30.513 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-04T07:33:30.515 INFO:tasks.ceph.mgr.x:Started 2024-03-04T07:33:30.515 INFO:tasks.ceph:Setting crush tunables to default 2024-03-04T07:33:30.515 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd crush tunables default 2024-03-04T07:33:30.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbd5200640 1 Processor -- start 2024-03-04T07:33:30.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbd5200640 1 -- start start 2024-03-04T07:33:30.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbd5200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbd00cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:30.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbd5200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 0x7ffbd00ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:30.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbd5200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd015ae10 con 0x7ffbd015e630 2024-03-04T07:33:30.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbd5200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd015e1a0 con 0x7ffbd00c79b0 2024-03-04T07:33:30.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbd5200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd015abb0 con 0x7ffbd00c75d0 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcd400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 0x7ffbd00ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcd400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 0x7ffbd00ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54046/0 (socket says 172.21.15.113:54046) 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbce800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ffbd015e630 0x7ffbd0154910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36366/0 (socket says 172.21.15.113:36366) 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcd400640 1 -- 172.21.15.113:0/2525456494 learned_addr learned my addr 172.21.15.113:0/2525456494 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcde00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbd00cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcde00640 1 -- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 msgr2=0x7ffbd00cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcde00640 1 -- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 msgr2=0x7ffbd00cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcde00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbd00cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcde00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbd00cc880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcd400640 1 -- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 msgr2=0x7ffbd00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcd400640 1 -- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 msgr2=0x7ffbd00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcd400640 1 --2- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 0x7ffbd00ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T07:33:30.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.656+0000 7ffbcd400640 1 --2- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 0x7ffbd00ccdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T07:33:30.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.860+0000 7ffbcd400640 1 --2- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 0x7ffbd00ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:30.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.860+0000 7ffbcd400640 1 -- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 msgr2=0x7ffbd00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-04T07:33:30.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.860+0000 7ffbcd400640 1 -- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 msgr2=0x7ffbd00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-04T07:33:30.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.860+0000 7ffbcd400640 1 --2- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 0x7ffbd00ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T07:33:30.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.860+0000 7ffbcde00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbd00cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:30.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.860+0000 7ffbcd400640 1 --2- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 0x7ffbd00ccdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T07:33:30.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.860+0000 7ffbcde00640 1 -- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 msgr2=0x7ffbd00cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-04T07:33:30.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.860+0000 7ffbcde00640 1 -- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 msgr2=0x7ffbd00cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-04T07:33:30.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.860+0000 7ffbcde00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbd00cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T07:33:30.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.860+0000 7ffbcde00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbd00cc880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T07:33:30.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340110685 0 0) 0x7ffbd015ae10 con 0x7ffbd015e630 2024-03-04T07:33:30.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbb4003530 con 0x7ffbd015e630 2024-03-04T07:33:30.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 732475882 0 0) 0x7ffbb4003530 con 0x7ffbd015e630 2024-03-04T07:33:30.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 msgr2=0x7ffbd00cc880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T07:33:30.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbd00cc880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:30.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 msgr2=0x7ffbd00ccdc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T07:33:30.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 --2- 172.21.15.113:0/2525456494 >> v2:172.21.15.113:3300/0 conn(0x7ffbd00c79b0 0x7ffbd00ccdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:30.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd015b0d0 con 0x7ffbd015e630 2024-03-04T07:33:30.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7ffbc8001680 con 0x7ffbd015e630 2024-03-04T07:33:30.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 >> v1:172.21.15.113:6789/0 conn(0x7ffbd015e630 legacy=0x7ffbd0154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbb40039f0 0x7ffbb4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4004810 0x7ffbb4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbb40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd015abb0 con 0x7ffbb4004810 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb400ba80 con 0x7ffbb40039f0 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb400bc30 con 0x7ffbd00c75d0 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcd400640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4004810 0x7ffbb4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcde00640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbb40039f0 0x7ffbb4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbce800640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbb40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcd400640 1 -- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 msgr2=0x7ffbb40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcd400640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd00c75d0 0x7ffbb40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcd400640 1 -- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbb40039f0 msgr2=0x7ffbb4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcd400640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbb40039f0 0x7ffbb4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:30.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.868+0000 7ffbcd400640 1 -- 172.21.15.113:0/2525456494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb4003530 con 0x7ffbb4004810 2024-03-04T07:33:30.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbcd400640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4004810 0x7ffbb4004c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00c7dd0 tx=0x7ffbbc01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98e2834a1ae2df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:30.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc00f040 con 0x7ffbb4004810 2024-03-04T07:33:30.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc0a66f0 con 0x7ffbb4004810 2024-03-04T07:33:30.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbcca00640 1 -- 172.21.15.113:0/2525456494 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0a6ae0 con 0x7ffbb4004810 2024-03-04T07:33:30.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbd5200640 1 -- 172.21.15.113:0/2525456494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4004810 msgr2=0x7ffbb4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:30.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbd5200640 1 --2- 172.21.15.113:0/2525456494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb4004810 0x7ffbb4004c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00c7dd0 tx=0x7ffbbc01c8f0 comp rx=0 tx=0).stop 2024-03-04T07:33:30.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbce800640 1 -- 172.21.15.113:0/2525456494 reap_dead start 2024-03-04T07:33:30.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbd5200640 1 -- 172.21.15.113:0/2525456494 shutdown_connections 2024-03-04T07:33:30.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbd5200640 1 -- 172.21.15.113:0/2525456494 >> 172.21.15.113:0/2525456494 conn(0x7ffbd006b060 msgr2=0x7ffbd006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:30.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbd5200640 1 -- 172.21.15.113:0/2525456494 shutdown_connections 2024-03-04T07:33:30.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbd5200640 1 -- 172.21.15.113:0/2525456494 wait complete. 2024-03-04T07:33:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbd5200640 1 Processor -- start 2024-03-04T07:33:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.872+0000 7ffbd5200640 1 -- start start 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbd5200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbb4004810 0x7ffbd015d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbd5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbd015e050 0x7ffbd0156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbd5200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd01572a0 0x7ffbd0155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbd5200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd015b970 con 0x7ffbd015e050 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbd5200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd015b690 con 0x7ffbb4004810 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbd5200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd0072a80 con 0x7ffbd01572a0 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbd015e050 0x7ffbd0156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbd015e050 0x7ffbd0156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54090/0 (socket says 172.21.15.113:54090) 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcd400640 1 -- 172.21.15.113:0/1179337347 learned_addr learned my addr 172.21.15.113:0/1179337347 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbce800640 1 --2- 172.21.15.113:0/1179337347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd01572a0 0x7ffbd0155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcde00640 1 --2- 172.21.15.113:0/1179337347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbb4004810 0x7ffbd015d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcd400640 1 -- 172.21.15.113:0/1179337347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd01572a0 msgr2=0x7ffbd0155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcd400640 1 --2- 172.21.15.113:0/1179337347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd01572a0 0x7ffbd0155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcd400640 1 -- 172.21.15.113:0/1179337347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbb4004810 msgr2=0x7ffbd015d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcd400640 1 --2- 172.21.15.113:0/1179337347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbb4004810 0x7ffbd015d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcd400640 1 -- 172.21.15.113:0/1179337347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd0157d10 con 0x7ffbd015e050 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcde00640 1 --2- 172.21.15.113:0/1179337347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbb4004810 0x7ffbd015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbcd400640 1 --2- 172.21.15.113:0/1179337347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbd015e050 0x7ffbd0156d60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc01cdd0 tx=0x7ffbb4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbc6000640 1 -- 172.21.15.113:0/1179337347 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0b0600 con 0x7ffbd015e050 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbc6000640 1 -- 172.21.15.113:0/1179337347 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc0bd070 con 0x7ffbd015e050 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbd5200640 1 -- 172.21.15.113:0/1179337347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd0158570 con 0x7ffbd015e050 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbc6000640 1 -- 172.21.15.113:0/1179337347 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0b0a00 con 0x7ffbd015e050 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbd5200640 1 -- 172.21.15.113:0/1179337347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd0158290 con 0x7ffbd015e050 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbc6000640 1 -- 172.21.15.113:0/1179337347 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ffbbc0a65b0 con 0x7ffbd015e050 2024-03-04T07:33:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbc6000640 1 -- 172.21.15.113:0/1179337347 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffbbc0c2c80 con 0x7ffbd015e050 2024-03-04T07:33:30.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.876+0000 7ffbd5200640 1 -- 172.21.15.113:0/1179337347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd0075c00 con 0x7ffbd015e050 2024-03-04T07:33:30.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.880+0000 7ffbc6000640 1 -- 172.21.15.113:0/1179337347 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ffbbc016640 con 0x7ffbd015e050 2024-03-04T07:33:30.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbd5200640 1 -- 172.21.15.113:0/1179337347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ffbd00cd7f0 con 0x7ffbd015e050 2024-03-04T07:33:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbc6000640 1 -- 172.21.15.113:0/1179337347 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ffbbc0b0ca0 con 0x7ffbd015e050 2024-03-04T07:33:30.941 INFO:teuthology.orchestra.run.smithi113.stderr:adjusted tunables profile to default 2024-03-04T07:33:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbd5200640 1 -- 172.21.15.113:0/1179337347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbd015e050 msgr2=0x7ffbd0156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbd5200640 1 --2- 172.21.15.113:0/1179337347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbd015e050 0x7ffbd0156d60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc01cdd0 tx=0x7ffbb4005840 comp rx=0 tx=0).stop 2024-03-04T07:33:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbd5200640 1 -- 172.21.15.113:0/1179337347 shutdown_connections 2024-03-04T07:33:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbd5200640 1 --2- 172.21.15.113:0/1179337347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbd01572a0 0x7ffbd0155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbd5200640 1 --2- 172.21.15.113:0/1179337347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbd015e050 0x7ffbd0156d60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbd5200640 1 --2- 172.21.15.113:0/1179337347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbb4004810 0x7ffbd015d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbd5200640 1 -- 172.21.15.113:0/1179337347 >> 172.21.15.113:0/1179337347 conn(0x7ffbd006b060 msgr2=0x7ffbd006efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:30.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbd5200640 1 -- 172.21.15.113:0/1179337347 shutdown_connections 2024-03-04T07:33:30.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:30.940+0000 7ffbd5200640 1 -- 172.21.15.113:0/1179337347 wait complete. 2024-03-04T07:33:30.960 INFO:tasks.ceph:check_enable_crimson: False 2024-03-04T07:33:30.960 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-04T07:33:30.961 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:30.961 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-04T07:33:30.980 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:30.980 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-04T07:33:31.040 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:31.040 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-04T07:33:31.056 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:31.053+0000 7fc375c73180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T07:33:31.099 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:33:31.100 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-04T07:33:31.159 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:31.159 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-04T07:33:31.173 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:31.173 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-04T07:33:31.229 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:31.229 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-04T07:33:31.284 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-03-04T07:33:31.284 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-04T07:33:31.336 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new cbf2ee75-ec28-483b-bb34-349b80272fcf 0 2024-03-04T07:33:31.427 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:31.428+0000 7f02fca55180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T07:33:31.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.493+0000 7f06c8431640 1 Processor -- start 2024-03-04T07:33:31.496 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.493+0000 7f06c8431640 1 -- start start 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.493+0000 7f06c8431640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f06c0071360 0x7f06c0072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.493+0000 7f06c8431640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06c00cd330 0x7f06c00c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.493+0000 7f06c8431640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c00cc890 con 0x7f06c0151180 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c8431640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00c6d90 con 0x7f06c00cd330 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c8431640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c0157850 con 0x7f06c0071360 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c59cc640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f06c0071360 0x7f06c0072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c59cc640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f06c0071360 0x7f06c0072400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:56532/0 (socket says 172.21.15.136:56532) 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c59cc640 1 -- 172.21.15.136:0/2434443021 learned_addr learned my addr 172.21.15.136:0/2434443021 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c59cc640 1 -- 172.21.15.136:0/2434443021 >> v2:172.21.15.113:3300/0 conn(0x7f06c00cd330 msgr2=0x7f06c00c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c59cc640 1 --2- 172.21.15.136:0/2434443021 >> v2:172.21.15.113:3300/0 conn(0x7f06c00cd330 0x7f06c00c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c59cc640 1 -- 172.21.15.136:0/2434443021 >> v1:172.21.15.113:6789/0 conn(0x7f06c0151180 legacy=0x7f06c0156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:31.498 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c59cc640 1 -- 172.21.15.136:0/2434443021 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c0157b30 con 0x7f06c0071360 2024-03-04T07:33:31.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c59cc640 1 --2- 172.21.15.136:0/2434443021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 0x7f06c0072400 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f06c00c49c0 tx=0x7f06b001c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96f4e3b6f352de49 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:31.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c51cb640 1 -- 172.21.15.136:0/2434443021 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00b0070 con 0x7f06c0071360 2024-03-04T07:33:31.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c51cb640 1 -- 172.21.15.136:0/2434443021 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b00ab600 con 0x7f06c0071360 2024-03-04T07:33:31.500 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c51cb640 1 -- 172.21.15.136:0/2434443021 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b00ab9d0 con 0x7f06c0071360 2024-03-04T07:33:31.501 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c8431640 1 -- 172.21.15.136:0/2434443021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 msgr2=0x7f06c0072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.501 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.497+0000 7f06c8431640 1 --2- 172.21.15.136:0/2434443021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 0x7f06c0072400 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f06c00c49c0 tx=0x7f06b001c4e0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.501 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- 172.21.15.136:0/2434443021 shutdown_connections 2024-03-04T07:33:31.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 --2- 172.21.15.136:0/2434443021 >> v2:172.21.15.113:3300/0 conn(0x7f06c00cd330 0x7f06c00c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 --2- 172.21.15.136:0/2434443021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 0x7f06c0072400 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- 172.21.15.136:0/2434443021 >> 172.21.15.136:0/2434443021 conn(0x7f06c006b060 msgr2=0x7f06c006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:31.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- 172.21.15.136:0/2434443021 shutdown_connections 2024-03-04T07:33:31.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- 172.21.15.136:0/2434443021 wait complete. 2024-03-04T07:33:31.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 Processor -- start 2024-03-04T07:33:31.502 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- start start 2024-03-04T07:33:31.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 0x7f06c01f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c00cd330 0x7f06c0145b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06c0151180 0x7f06c01f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0157e10 con 0x7f06c00cd330 2024-03-04T07:33:31.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00c38d0 con 0x7f06c0071360 2024-03-04T07:33:31.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0151560 con 0x7f06c0151180 2024-03-04T07:33:31.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c61cd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 0x7f06c01f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.505 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c61cd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 0x7f06c01f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:56544/0 (socket says 172.21.15.136:56544) 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c61cd640 1 -- 172.21.15.136:0/1438127997 learned_addr learned my addr 172.21.15.136:0/1438127997 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c69ce640 1 --2- 172.21.15.136:0/1438127997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06c0151180 0x7f06c01f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c59cc640 1 --2- 172.21.15.136:0/1438127997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c00cd330 0x7f06c0145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c61cd640 1 -- 172.21.15.136:0/1438127997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06c0151180 msgr2=0x7f06c01f0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c61cd640 1 --2- 172.21.15.136:0/1438127997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06c0151180 0x7f06c01f0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c61cd640 1 -- 172.21.15.136:0/1438127997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c00cd330 msgr2=0x7f06c0145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c61cd640 1 --2- 172.21.15.136:0/1438127997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c00cd330 0x7f06c0145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c61cd640 1 -- 172.21.15.136:0/1438127997 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c014c130 con 0x7f06c0071360 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c59cc640 1 --2- 172.21.15.136:0/1438127997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c00cd330 0x7f06c0145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c61cd640 1 --2- 172.21.15.136:0/1438127997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 0x7f06c01f4600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f06b800ba80 tx=0x7f06b800bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06b6ffd640 1 -- 172.21.15.136:0/1438127997 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8019070 con 0x7f06c0071360 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06b6ffd640 1 -- 172.21.15.136:0/1438127997 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b800cc90 con 0x7f06c0071360 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- 172.21.15.136:0/1438127997 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c0154e40 con 0x7f06c0071360 2024-03-04T07:33:31.506 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06b6ffd640 1 -- 172.21.15.136:0/1438127997 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8014680 con 0x7f06c0071360 2024-03-04T07:33:31.507 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- 172.21.15.136:0/1438127997 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c01f3e00 con 0x7f06c0071360 2024-03-04T07:33:31.507 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06b6ffd640 1 -- 172.21.15.136:0/1438127997 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f06b801c070 con 0x7f06c0071360 2024-03-04T07:33:31.507 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06b6ffd640 1 -- 172.21.15.136:0/1438127997 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f06b801eca0 con 0x7f06c0071360 2024-03-04T07:33:31.507 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.501+0000 7f06c8431640 1 -- 172.21.15.136:0/1438127997 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c0075c00 con 0x7f06c0071360 2024-03-04T07:33:31.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.513+0000 7f06b6ffd640 1 -- 172.21.15.136:0/1438127997 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f06b8023070 con 0x7f06c0071360 2024-03-04T07:33:31.514 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:31.513+0000 7fc375c73180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T07:33:31.582 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.581+0000 7f06c8431640 1 -- 172.21.15.136:0/1438127997 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cbf2ee75-ec28-483b-bb34-349b80272fcf", "id": 0} v 0) v1 -- 0x7f06c0068d90 con 0x7f06c0071360 2024-03-04T07:33:31.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.589+0000 7f06b6ffd640 1 -- 172.21.15.136:0/1438127997 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cbf2ee75-ec28-483b-bb34-349b80272fcf", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f06b801e440 con 0x7f06c0071360 2024-03-04T07:33:31.591 INFO:teuthology.orchestra.run.smithi136.stdout:0 2024-03-04T07:33:31.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.589+0000 7f06c8431640 1 -- 172.21.15.136:0/1438127997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 msgr2=0x7f06c01f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.589+0000 7f06c8431640 1 --2- 172.21.15.136:0/1438127997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 0x7f06c01f4600 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f06b800ba80 tx=0x7f06b800bf40 comp rx=0 tx=0).stop 2024-03-04T07:33:31.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.589+0000 7f06c8431640 1 -- 172.21.15.136:0/1438127997 shutdown_connections 2024-03-04T07:33:31.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.589+0000 7f06c8431640 1 --2- 172.21.15.136:0/1438127997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06c0151180 0x7f06c01f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.592 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.589+0000 7f06c8431640 1 --2- 172.21.15.136:0/1438127997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06c00cd330 0x7f06c0145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.589+0000 7f06c8431640 1 --2- 172.21.15.136:0/1438127997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06c0071360 0x7f06c01f4600 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.589+0000 7f06c8431640 1 -- 172.21.15.136:0/1438127997 >> 172.21.15.136:0/1438127997 conn(0x7f06c006b060 msgr2=0x7f06c00cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:31.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.589+0000 7f06c8431640 1 -- 172.21.15.136:0/1438127997 shutdown_connections 2024-03-04T07:33:31.593 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.589+0000 7f06c8431640 1 -- 172.21.15.136:0/1438127997 wait complete. 2024-03-04T07:33:31.607 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 49418806-cd59-4b3f-a3f1-7ea1736b54fd 1 2024-03-04T07:33:31.705 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:31.705+0000 7fc375c73180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T07:33:31.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db7fb3640 1 Processor -- start 2024-03-04T07:33:31.718 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db7fb3640 1 -- start start 2024-03-04T07:33:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db7fb3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3db006fb00 0x7f3db00ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db7fb3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db015b460 con 0x7f3db00ca7c0 2024-03-04T07:33:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db7fb3640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db015b720 con 0x7f3db006fee0 2024-03-04T07:33:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db7fb3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db015e7b0 con 0x7f3db006fb00 2024-03-04T07:33:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db554e640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f3db006fee0 0x7f3db015aaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.136:39762/0 (socket says 172.21.15.136:39762) 2024-03-04T07:33:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db554e640 1 -- 172.21.15.136:0/419805179 learned_addr learned my addr 172.21.15.136:0/419805179 (peer_addr_for_me v1:172.21.15.136:0/0) 2024-03-04T07:33:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db5d4f640 1 --2- 172.21.15.136:0/419805179 >> v2:172.21.15.113:3300/0 conn(0x7f3db006fb00 0x7f3db00ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616956513 0 0) 0x7f3db015b720 con 0x7f3db006fee0 2024-03-04T07:33:31.721 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d9c003530 con 0x7f3db006fee0 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 <== mon.1 v1:172.21.15.136:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3004712195 0 0) 0x7f3d9c003530 con 0x7f3db006fee0 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 >> v2:172.21.15.113:3300/0 conn(0x7f3db006fb00 msgr2=0x7f3db00ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 --2- 172.21.15.136:0/419805179 >> v2:172.21.15.113:3300/0 conn(0x7f3db006fb00 0x7f3db00ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 >> v1:172.21.15.113:6789/0 conn(0x7f3db00ca7c0 legacy=0x7f3db0154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 --> v1:172.21.15.136:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db015ba00 con 0x7f3db006fee0 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 <== mon.1 v1:172.21.15.136:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f3da0001680 con 0x7f3db006fee0 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 >> v1:172.21.15.136:6789/0 conn(0x7f3db006fee0 legacy=0x7f3db015aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d9c0039c0 0x7f3d9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d9c004810 0x7f3d9c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0051d0 0x7f3d9c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db015e7b0 con 0x7f3d9c0051d0 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c00c170 con 0x7f3d9c0039c0 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c00c370 con 0x7f3d9c004810 2024-03-04T07:33:31.722 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db5d4f640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d9c0039c0 0x7f3d9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db554e640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d9c004810 0x7f3d9c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db6550640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0051d0 0x7f3d9c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db5d4f640 1 -- 172.21.15.136:0/419805179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d9c004810 msgr2=0x7f3d9c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:31.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db5d4f640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d9c004810 0x7f3d9c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db5d4f640 1 -- 172.21.15.136:0/419805179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0051d0 msgr2=0x7f3d9c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:31.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db5d4f640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0051d0 0x7f3d9c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db5d4f640 1 -- 172.21.15.136:0/419805179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c003530 con 0x7f3d9c0039c0 2024-03-04T07:33:31.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db6550640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d9c0051d0 0x7f3d9c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:31.723 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.717+0000 7f3db6550640 1 -- 172.21.15.136:0/419805179 reap_dead start 2024-03-04T07:33:31.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db5d4f640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d9c0039c0 0x7f3d9c003de0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009ae0 tx=0x7f3dac01ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d8b358686bd3d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:31.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0096b0 con 0x7f3d9c0039c0 2024-03-04T07:33:31.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dac0a6480 con 0x7f3d9c0039c0 2024-03-04T07:33:31.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db4d4d640 1 -- 172.21.15.136:0/419805179 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0a6870 con 0x7f3d9c0039c0 2024-03-04T07:33:31.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 -- 172.21.15.136:0/419805179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d9c0039c0 msgr2=0x7f3d9c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d9c0039c0 0x7f3d9c003de0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3dac009ae0 tx=0x7f3dac01ca70 comp rx=0 tx=0).stop 2024-03-04T07:33:31.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 -- 172.21.15.136:0/419805179 shutdown_connections 2024-03-04T07:33:31.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 --2- 172.21.15.136:0/419805179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d9c0039c0 0x7f3d9c003de0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 -- 172.21.15.136:0/419805179 >> 172.21.15.136:0/419805179 conn(0x7f3db006e240 msgr2=0x7f3db006e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:31.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 -- 172.21.15.136:0/419805179 shutdown_connections 2024-03-04T07:33:31.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 -- 172.21.15.136:0/419805179 wait complete. 2024-03-04T07:33:31.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 Processor -- start 2024-03-04T07:33:31.725 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 -- start start 2024-03-04T07:33:31.727 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d9c0039c0 0x7f3db00b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.721+0000 7f3db7fb3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00c8160 0x7f3db01f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db7fb3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db01f0630 0x7f3db01562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db7fb3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db006b140 con 0x7f3db01f0630 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db7fb3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db015e440 con 0x7f3db00c8160 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db7fb3640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00ae8e0 con 0x7f3d9c0039c0 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db5d4f640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d9c0039c0 0x7f3db00b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db554e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00c8160 0x7f3db01f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db5d4f640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d9c0039c0 0x7f3db00b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:53750/0 (socket says 172.21.15.136:53750) 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db5d4f640 1 -- 172.21.15.136:0/8174753 learned_addr learned my addr 172.21.15.136:0/8174753 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db6550640 1 --2- 172.21.15.136:0/8174753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db01f0630 0x7f3db01562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db554e640 1 -- 172.21.15.136:0/8174753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d9c0039c0 msgr2=0x7f3db00b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db554e640 1 --2- 172.21.15.136:0/8174753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d9c0039c0 0x7f3db00b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db554e640 1 -- 172.21.15.136:0/8174753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db01f0630 msgr2=0x7f3db01562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.728 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db554e640 1 --2- 172.21.15.136:0/8174753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db01f0630 0x7f3db01562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db554e640 1 -- 172.21.15.136:0/8174753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db00b2290 con 0x7f3db00c8160 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db6550640 1 --2- 172.21.15.136:0/8174753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db01f0630 0x7f3db01562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db554e640 1 --2- 172.21.15.136:0/8174753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00c8160 0x7f3db01f00f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3da0012570 tx=0x7f3da0012a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3da67fc640 1 -- 172.21.15.136:0/8174753 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da001c070 con 0x7f3db00c8160 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3da67fc640 1 -- 172.21.15.136:0/8174753 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da0014810 con 0x7f3db00c8160 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db7fb3640 1 -- 172.21.15.136:0/8174753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db00b2550 con 0x7f3db00c8160 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3da67fc640 1 -- 172.21.15.136:0/8174753 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da0014c00 con 0x7f3db00c8160 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db7fb3640 1 -- 172.21.15.136:0/8174753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db01f38e0 con 0x7f3db00c8160 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3da67fc640 1 -- 172.21.15.136:0/8174753 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3da001f070 con 0x7f3db00c8160 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3da67fc640 1 -- 172.21.15.136:0/8174753 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f3da00167f0 con 0x7f3db00c8160 2024-03-04T07:33:31.729 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.725+0000 7f3db7fb3640 1 -- 172.21.15.136:0/8174753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db0075c00 con 0x7f3db00c8160 2024-03-04T07:33:31.730 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.729+0000 7f3da67fc640 1 -- 172.21.15.136:0/8174753 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3da0016a40 con 0x7f3db00c8160 2024-03-04T07:33:31.797 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.797+0000 7f3db7fb3640 1 -- 172.21.15.136:0/8174753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd new", "uuid": "49418806-cd59-4b3f-a3f1-7ea1736b54fd", "id": 1} v 0) v1 -- 0x7f3db01f0a10 con 0x7f3db00c8160 2024-03-04T07:33:31.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.805+0000 7f3da67fc640 1 -- 172.21.15.136:0/8174753 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "49418806-cd59-4b3f-a3f1-7ea1736b54fd", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3da0016cf0 con 0x7f3db00c8160 2024-03-04T07:33:31.805 INFO:teuthology.orchestra.run.smithi136.stdout:1 2024-03-04T07:33:31.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.805+0000 7f3db7fb3640 1 -- 172.21.15.136:0/8174753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00c8160 msgr2=0x7f3db01f00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.805+0000 7f3db7fb3640 1 --2- 172.21.15.136:0/8174753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00c8160 0x7f3db01f00f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3da0012570 tx=0x7f3da0012a30 comp rx=0 tx=0).stop 2024-03-04T07:33:31.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.805+0000 7f3db7fb3640 1 -- 172.21.15.136:0/8174753 shutdown_connections 2024-03-04T07:33:31.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.805+0000 7f3db7fb3640 1 --2- 172.21.15.136:0/8174753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3db01f0630 0x7f3db01562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.805+0000 7f3db7fb3640 1 --2- 172.21.15.136:0/8174753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3db00c8160 0x7f3db01f00f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.805+0000 7f3db7fb3640 1 --2- 172.21.15.136:0/8174753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d9c0039c0 0x7f3db00b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.805+0000 7f3db7fb3640 1 -- 172.21.15.136:0/8174753 >> 172.21.15.136:0/8174753 conn(0x7f3db006e240 msgr2=0x7f3db006e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:31.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.805+0000 7f3db7fb3640 1 -- 172.21.15.136:0/8174753 shutdown_connections 2024-03-04T07:33:31.807 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.805+0000 7f3db7fb3640 1 -- 172.21.15.136:0/8174753 wait complete. 2024-03-04T07:33:31.821 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 0513256b-863e-406e-88da-d1ad6902ed24 2 2024-03-04T07:33:31.924 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:31.924+0000 7f02fca55180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T07:33:31.939 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f05480f8640 1 Processor -- start 2024-03-04T07:33:31.940 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f05480f8640 1 -- start start 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f05480f8640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0540072f00 0x7f05400c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f05480f8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05400c5f20 0x7f0540151600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f05480f8640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0540066eb0 con 0x7f05400732e0 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f05480f8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0540071360 con 0x7f05400c5f20 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f05480f8640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0540071640 con 0x7f0540072f00 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f0545693640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f05400732e0 0x7f05400c5810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.136:39776/0 (socket says 172.21.15.136:39776) 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f0545e94640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0540072f00 0x7f05400c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f0545693640 1 -- 172.21.15.136:0/3060537280 learned_addr learned my addr 172.21.15.136:0/3060537280 (peer_addr_for_me v1:172.21.15.136:0/0) 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f0544e92640 1 -- 172.21.15.136:0/3060537280 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035037252 0 0) 0x7f0540066eb0 con 0x7f05400732e0 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f0544e92640 1 -- 172.21.15.136:0/3060537280 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f052c005740 con 0x7f05400732e0 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f0545e94640 1 -- 172.21.15.136:0/3060537280 >> v2:172.21.15.113:3300/0 conn(0x7f05400c5f20 msgr2=0x7f0540151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.937+0000 7f0545e94640 1 --2- 172.21.15.136:0/3060537280 >> v2:172.21.15.113:3300/0 conn(0x7f05400c5f20 0x7f0540151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.942 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f0545e94640 1 -- 172.21.15.136:0/3060537280 >> v1:172.21.15.136:6789/0 conn(0x7f05400732e0 legacy=0x7f05400c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f0545e94640 1 -- 172.21.15.136:0/3060537280 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0540071920 con 0x7f0540072f00 2024-03-04T07:33:31.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f0545e94640 1 --2- 172.21.15.136:0/3060537280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0540072f00 0x7f05400c3d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0538009d90 tx=0x7f053801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50ac2e836dccc23 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:31.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f0544e92640 1 -- 172.21.15.136:0/3060537280 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05380b2070 con 0x7f0540072f00 2024-03-04T07:33:31.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f0544e92640 1 -- 172.21.15.136:0/3060537280 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05380a5790 con 0x7f0540072f00 2024-03-04T07:33:31.943 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f0544e92640 1 -- 172.21.15.136:0/3060537280 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05380a5b60 con 0x7f0540072f00 2024-03-04T07:33:31.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f05480f8640 1 -- 172.21.15.136:0/3060537280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0540072f00 msgr2=0x7f05400c3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.944 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f05480f8640 1 --2- 172.21.15.136:0/3060537280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0540072f00 0x7f05400c3d60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0538009d90 tx=0x7f053801c8e0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f05480f8640 1 -- 172.21.15.136:0/3060537280 shutdown_connections 2024-03-04T07:33:31.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f05480f8640 1 --2- 172.21.15.136:0/3060537280 >> v2:172.21.15.113:3300/0 conn(0x7f05400c5f20 0x7f0540151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f05480f8640 1 --2- 172.21.15.136:0/3060537280 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0540072f00 0x7f05400c3d60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f05480f8640 1 -- 172.21.15.136:0/3060537280 >> 172.21.15.136:0/3060537280 conn(0x7f054006b060 msgr2=0x7f054006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:31.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f05480f8640 1 -- 172.21.15.136:0/3060537280 shutdown_connections 2024-03-04T07:33:31.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.941+0000 7f05480f8640 1 -- 172.21.15.136:0/3060537280 wait complete. 2024-03-04T07:33:31.945 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 Processor -- start 2024-03-04T07:33:31.946 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 -- start start 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0540072f00 0x7f05400ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05400732e0 0x7f0540157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400c5f20 0x7f05401584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0540072910 con 0x7f05400c5f20 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0540072790 con 0x7f0540072f00 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05400cd070 con 0x7f05400732e0 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545693640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05400732e0 0x7f0540157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545e94640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0540072f00 0x7f05400ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545e94640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0540072f00 0x7f05400ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:56566/0 (socket says 172.21.15.136:56566) 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545693640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05400732e0 0x7f0540157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:53768/0 (socket says 172.21.15.136:53768) 2024-03-04T07:33:31.948 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545e94640 1 -- 172.21.15.136:0/3268232459 learned_addr learned my addr 172.21.15.136:0/3268232459 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0546695640 1 --2- 172.21.15.136:0/3268232459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400c5f20 0x7f05401584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545e94640 1 -- 172.21.15.136:0/3268232459 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05400732e0 msgr2=0x7f0540157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545e94640 1 --2- 172.21.15.136:0/3268232459 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05400732e0 0x7f0540157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545e94640 1 -- 172.21.15.136:0/3268232459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400c5f20 msgr2=0x7f05401584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545e94640 1 --2- 172.21.15.136:0/3268232459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400c5f20 0x7f05401584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545e94640 1 -- 172.21.15.136:0/3268232459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05401f3fb0 con 0x7f0540072f00 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545693640 1 --2- 172.21.15.136:0/3268232459 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05400732e0 0x7f0540157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0546695640 1 --2- 172.21.15.136:0/3268232459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400c5f20 0x7f05401584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f0545e94640 1 --2- 172.21.15.136:0/3268232459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0540072f00 0x7f05400ca8c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f053801a510 tx=0x7f0538019680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05367fc640 1 -- 172.21.15.136:0/3268232459 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05380bd020 con 0x7f0540072f00 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05367fc640 1 -- 172.21.15.136:0/3268232459 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05380b8470 con 0x7f0540072f00 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 -- 172.21.15.136:0/3268232459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05401f4290 con 0x7f0540072f00 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05367fc640 1 -- 172.21.15.136:0/3268232459 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05380b2040 con 0x7f0540072f00 2024-03-04T07:33:31.949 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 -- 172.21.15.136:0/3268232459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05401f4850 con 0x7f0540072f00 2024-03-04T07:33:31.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05367fc640 1 -- 172.21.15.136:0/3268232459 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f05380c0020 con 0x7f0540072f00 2024-03-04T07:33:31.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05367fc640 1 -- 172.21.15.136:0/3268232459 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f05380b9420 con 0x7f0540072f00 2024-03-04T07:33:31.950 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.945+0000 7f05480f8640 1 -- 172.21.15.136:0/3268232459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0540075c00 con 0x7f0540072f00 2024-03-04T07:33:31.951 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:31.949+0000 7f05367fc640 1 -- 172.21.15.136:0/3268232459 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f05380c7030 con 0x7f0540072f00 2024-03-04T07:33:32.019 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.017+0000 7f05480f8640 1 -- 172.21.15.136:0/3268232459 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0513256b-863e-406e-88da-d1ad6902ed24", "id": 2} v 0) v1 -- 0x7f05401f7360 con 0x7f0540072f00 2024-03-04T07:33:32.025 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.021+0000 7f05367fc640 1 -- 172.21.15.136:0/3268232459 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0513256b-863e-406e-88da-d1ad6902ed24", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0538016560 con 0x7f0540072f00 2024-03-04T07:33:32.025 INFO:teuthology.orchestra.run.smithi136.stdout:2 2024-03-04T07:33:32.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.025+0000 7f05480f8640 1 -- 172.21.15.136:0/3268232459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0540072f00 msgr2=0x7f05400ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.025+0000 7f05480f8640 1 --2- 172.21.15.136:0/3268232459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0540072f00 0x7f05400ca8c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f053801a510 tx=0x7f0538019680 comp rx=0 tx=0).stop 2024-03-04T07:33:32.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.025+0000 7f05480f8640 1 -- 172.21.15.136:0/3268232459 shutdown_connections 2024-03-04T07:33:32.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.025+0000 7f05480f8640 1 --2- 172.21.15.136:0/3268232459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400c5f20 0x7f05401584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.025+0000 7f05480f8640 1 --2- 172.21.15.136:0/3268232459 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05400732e0 0x7f0540157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.025+0000 7f05480f8640 1 --2- 172.21.15.136:0/3268232459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0540072f00 0x7f05400ca8c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.025+0000 7f05480f8640 1 -- 172.21.15.136:0/3268232459 >> 172.21.15.136:0/3268232459 conn(0x7f054006b060 msgr2=0x7f054006b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:32.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.025+0000 7f05480f8640 1 -- 172.21.15.136:0/3268232459 shutdown_connections 2024-03-04T07:33:32.027 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.025+0000 7f05480f8640 1 -- 172.21.15.136:0/3268232459 wait complete. 2024-03-04T07:33:32.042 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 32c27a18-acb8-4089-a959-5a8545aabf68 3 2024-03-04T07:33:32.147 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:32.148+0000 7f02fca55180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T07:33:32.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.153+0000 7fb70ab84640 1 Processor -- start 2024-03-04T07:33:32.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.153+0000 7fb70ab84640 1 -- start start 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb704152b30 0x7fb7040c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb704152f10 con 0x7fb7040ccf00 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7040cd4b0 con 0x7fb7040c8f20 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040c42e0 con 0x7fb704152b30 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb703fff640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fb7040c8f20 0x7fb7040cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.136:39780/0 (socket says 172.21.15.136:39780) 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb708920640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb704152b30 0x7fb7040c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb703fff640 1 -- 172.21.15.136:0/1339764511 learned_addr learned my addr 172.21.15.136:0/1339764511 (peer_addr_for_me v1:172.21.15.136:0/0) 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb708920640 1 -- 172.21.15.136:0/1339764511 >> v1:172.21.15.136:6789/0 conn(0x7fb7040c8f20 legacy=0x7fb7040cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb708920640 1 -- 172.21.15.136:0/1339764511 >> v1:172.21.15.113:6789/0 conn(0x7fb7040ccf00 legacy=0x7fb7041631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb7037fe640 1 -- 172.21.15.136:0/1339764511 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253630778 0 0) 0x7fb7040cd4b0 con 0x7fb7040c8f20 2024-03-04T07:33:32.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb708920640 1 -- 172.21.15.136:0/1339764511 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb704159430 con 0x7fb704152b30 2024-03-04T07:33:32.160 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb708920640 1 --2- 172.21.15.136:0/1339764511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb704152b30 0x7fb7040c4770 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00097f0 tx=0x7fb6f001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61575bfad7537200 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:32.160 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb7037fe640 1 -- 172.21.15.136:0/1339764511 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00b5070 con 0x7fb704152b30 2024-03-04T07:33:32.160 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb7037fe640 1 -- 172.21.15.136:0/1339764511 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f00a8990 con 0x7fb704152b30 2024-03-04T07:33:32.160 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb7037fe640 1 -- 172.21.15.136:0/1339764511 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f00a8d60 con 0x7fb704152b30 2024-03-04T07:33:32.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 -- 172.21.15.136:0/1339764511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb704152b30 msgr2=0x7fb7040c4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 --2- 172.21.15.136:0/1339764511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb704152b30 0x7fb7040c4770 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6f00097f0 tx=0x7fb6f001ec90 comp rx=0 tx=0).stop 2024-03-04T07:33:32.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 -- 172.21.15.136:0/1339764511 shutdown_connections 2024-03-04T07:33:32.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 --2- 172.21.15.136:0/1339764511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb704152b30 0x7fb7040c4770 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 -- 172.21.15.136:0/1339764511 >> 172.21.15.136:0/1339764511 conn(0x7fb70406b060 msgr2=0x7fb70406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:32.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 -- 172.21.15.136:0/1339764511 shutdown_connections 2024-03-04T07:33:32.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.157+0000 7fb70ab84640 1 -- 172.21.15.136:0/1339764511 wait complete. 2024-03-04T07:33:32.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 Processor -- start 2024-03-04T07:33:32.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 -- start start 2024-03-04T07:33:32.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7040c8f20 0x7fb704071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7040ccf00 0x7fb70414ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb704152b30 0x7fb7041566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040c3f70 con 0x7fb7040ccf00 2024-03-04T07:33:32.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70406be50 con 0x7fb704152b30 2024-03-04T07:33:32.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb704067180 con 0x7fb7040c8f20 2024-03-04T07:33:32.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb708920640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7040c8f20 0x7fb704071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb708920640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7040c8f20 0x7fb704071360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:53772/0 (socket says 172.21.15.136:53772) 2024-03-04T07:33:32.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb709121640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb704152b30 0x7fb7041566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb708920640 1 -- 172.21.15.136:0/3148198593 learned_addr learned my addr 172.21.15.136:0/3148198593 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb703fff640 1 --2- 172.21.15.136:0/3148198593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7040ccf00 0x7fb70414ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb709121640 1 -- 172.21.15.136:0/3148198593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7040c8f20 msgr2=0x7fb704071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb709121640 1 --2- 172.21.15.136:0/3148198593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7040c8f20 0x7fb704071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb709121640 1 -- 172.21.15.136:0/3148198593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7040ccf00 msgr2=0x7fb70414ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb709121640 1 --2- 172.21.15.136:0/3148198593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7040ccf00 0x7fb70414ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb709121640 1 -- 172.21.15.136:0/3148198593 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7040cc010 con 0x7fb704152b30 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb708920640 1 --2- 172.21.15.136:0/3148198593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7040c8f20 0x7fb704071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb703fff640 1 --2- 172.21.15.136:0/3148198593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7040ccf00 0x7fb70414ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb709121640 1 --2- 172.21.15.136:0/3148198593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb704152b30 0x7fb7041566a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00c7b0 tx=0x7fb6fc00cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb7017fa640 1 -- 172.21.15.136:0/3148198593 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc016020 con 0x7fb704152b30 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb7017fa640 1 -- 172.21.15.136:0/3148198593 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00aa50 con 0x7fb704152b30 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 -- 172.21.15.136:0/3148198593 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7040cc2f0 con 0x7fb704152b30 2024-03-04T07:33:32.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb7017fa640 1 -- 172.21.15.136:0/3148198593 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00ce60 con 0x7fb704152b30 2024-03-04T07:33:32.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 -- 172.21.15.136:0/3148198593 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb704155a50 con 0x7fb704152b30 2024-03-04T07:33:32.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb7017fa640 1 -- 172.21.15.136:0/3148198593 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb6fc019020 con 0x7fb704152b30 2024-03-04T07:33:32.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb7017fa640 1 -- 172.21.15.136:0/3148198593 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fb6fc0128b0 con 0x7fb704152b30 2024-03-04T07:33:32.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.161+0000 7fb70ab84640 1 -- 172.21.15.136:0/3148198593 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb704075c00 con 0x7fb704152b30 2024-03-04T07:33:32.167 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.165+0000 7fb7017fa640 1 -- 172.21.15.136:0/3148198593 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb6fc020030 con 0x7fb704152b30 2024-03-04T07:33:32.235 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.233+0000 7fb70ab84640 1 -- 172.21.15.136:0/3148198593 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd new", "uuid": "32c27a18-acb8-4089-a959-5a8545aabf68", "id": 3} v 0) v1 -- 0x7fb70406df60 con 0x7fb704152b30 2024-03-04T07:33:32.241 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.237+0000 7fb7017fa640 1 -- 172.21.15.136:0/3148198593 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "32c27a18-acb8-4089-a959-5a8545aabf68", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb6fc012b90 con 0x7fb704152b30 2024-03-04T07:33:32.241 INFO:teuthology.orchestra.run.smithi136.stdout:3 2024-03-04T07:33:32.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.241+0000 7fb70ab84640 1 -- 172.21.15.136:0/3148198593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb704152b30 msgr2=0x7fb7041566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.241+0000 7fb70ab84640 1 --2- 172.21.15.136:0/3148198593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb704152b30 0x7fb7041566a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00c7b0 tx=0x7fb6fc00cc70 comp rx=0 tx=0).stop 2024-03-04T07:33:32.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.241+0000 7fb70ab84640 1 -- 172.21.15.136:0/3148198593 shutdown_connections 2024-03-04T07:33:32.242 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.241+0000 7fb70ab84640 1 --2- 172.21.15.136:0/3148198593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb704152b30 0x7fb7041566a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.241+0000 7fb70ab84640 1 --2- 172.21.15.136:0/3148198593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7040ccf00 0x7fb70414ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.241+0000 7fb70ab84640 1 --2- 172.21.15.136:0/3148198593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7040c8f20 0x7fb704071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.241+0000 7fb70ab84640 1 -- 172.21.15.136:0/3148198593 >> 172.21.15.136:0/3148198593 conn(0x7fb70406b060 msgr2=0x7fb70414ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:32.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.241+0000 7fb70ab84640 1 -- 172.21.15.136:0/3148198593 shutdown_connections 2024-03-04T07:33:32.243 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.241+0000 7fb70ab84640 1 -- 172.21.15.136:0/3148198593 wait complete. 2024-03-04T07:33:32.257 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1 4 2024-03-04T07:33:32.275 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:32.273+0000 7fc375c73180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T07:33:32.331 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:32.329+0000 7fc375c73180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T07:33:32.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.369+0000 7ff6e4ead640 1 Processor -- start 2024-03-04T07:33:32.372 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.369+0000 7ff6e4ead640 1 -- start start 2024-03-04T07:33:32.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff6e00c4e80 0x7ff6e01641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e0152f10 con 0x7ff6e0152b30 2024-03-04T07:33:32.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e00cd4b0 con 0x7ff6e00c8f20 2024-03-04T07:33:32.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e00c42e0 con 0x7ff6e00c4e80 2024-03-04T07:33:32.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6ded9d640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff6e00c4e80 0x7ff6e01641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6ded9d640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff6e00c4e80 0x7ff6e01641c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:52270/0 (socket says 172.21.15.136:52270) 2024-03-04T07:33:32.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6ded9d640 1 -- 172.21.15.136:0/2256501833 learned_addr learned my addr 172.21.15.136:0/2256501833 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:32.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6ded9d640 1 -- 172.21.15.136:0/2256501833 >> v1:172.21.15.136:6789/0 conn(0x7ff6e00c8f20 legacy=0x7ff6e00cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.374 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6ded9d640 1 -- 172.21.15.136:0/2256501833 >> v1:172.21.15.113:6789/0 conn(0x7ff6e0152b30 legacy=0x7ff6e00c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:32.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6ded9d640 1 -- 172.21.15.136:0/2256501833 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e0159430 con 0x7ff6e00c4e80 2024-03-04T07:33:32.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6ded9d640 1 --2- 172.21.15.136:0/2256501833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e00c4e80 0x7ff6e01641c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009e50 tx=0x7ff6d801f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ca0bf1748420b60 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:32.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6dd59a640 1 -- 172.21.15.136:0/2256501833 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80b4070 con 0x7ff6e00c4e80 2024-03-04T07:33:32.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6dd59a640 1 -- 172.21.15.136:0/2256501833 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d80af400 con 0x7ff6e00c4e80 2024-03-04T07:33:32.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6dd59a640 1 -- 172.21.15.136:0/2256501833 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80af7d0 con 0x7ff6e00c4e80 2024-03-04T07:33:32.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 -- 172.21.15.136:0/2256501833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e00c4e80 msgr2=0x7ff6e01641c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 --2- 172.21.15.136:0/2256501833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e00c4e80 0x7ff6e01641c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009e50 tx=0x7ff6d801f1e0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 -- 172.21.15.136:0/2256501833 shutdown_connections 2024-03-04T07:33:32.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 --2- 172.21.15.136:0/2256501833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e00c4e80 0x7ff6e01641c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 -- 172.21.15.136:0/2256501833 >> 172.21.15.136:0/2256501833 conn(0x7ff6e006b060 msgr2=0x7ff6e006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:32.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 -- 172.21.15.136:0/2256501833 shutdown_connections 2024-03-04T07:33:32.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.373+0000 7ff6e4ead640 1 -- 172.21.15.136:0/2256501833 wait complete. 2024-03-04T07:33:32.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 Processor -- start 2024-03-04T07:33:32.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 -- start start 2024-03-04T07:33:32.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e00c4e80 0x7ff6e0150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff6e00c8f20 0x7ff6e01f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e0152b30 0x7ff6e00cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.380 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e00c3f70 con 0x7ff6e00c4e80 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e006be50 con 0x7ff6e0152b30 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0067180 con 0x7ff6e00c8f20 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ded9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e0152b30 0x7ff6e00cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ded9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e0152b30 0x7ff6e00cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:52276/0 (socket says 172.21.15.136:52276) 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ded9d640 1 -- 172.21.15.136:0/840647836 learned_addr learned my addr 172.21.15.136:0/840647836 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ddd9b640 1 --2- 172.21.15.136:0/840647836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff6e00c8f20 0x7ff6e01f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6de59c640 1 --2- 172.21.15.136:0/840647836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e00c4e80 0x7ff6e0150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ded9d640 1 -- 172.21.15.136:0/840647836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff6e00c8f20 msgr2=0x7ff6e01f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ded9d640 1 --2- 172.21.15.136:0/840647836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff6e00c8f20 0x7ff6e01f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ded9d640 1 -- 172.21.15.136:0/840647836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e00c4e80 msgr2=0x7ff6e0150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ded9d640 1 --2- 172.21.15.136:0/840647836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e00c4e80 0x7ff6e0150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ded9d640 1 -- 172.21.15.136:0/840647836 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e01f1e10 con 0x7ff6e0152b30 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ded9d640 1 --2- 172.21.15.136:0/840647836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e0152b30 0x7ff6e00cbaa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff6d801bdf0 tx=0x7ff6d801bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ceffd640 1 -- 172.21.15.136:0/840647836 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80b4040 con 0x7ff6e0152b30 2024-03-04T07:33:32.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ceffd640 1 -- 172.21.15.136:0/840647836 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d80c1070 con 0x7ff6e0152b30 2024-03-04T07:33:32.382 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 -- 172.21.15.136:0/840647836 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e01447e0 con 0x7ff6e0152b30 2024-03-04T07:33:32.382 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ceffd640 1 -- 172.21.15.136:0/840647836 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d8009510 con 0x7ff6e0152b30 2024-03-04T07:33:32.382 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 -- 172.21.15.136:0/840647836 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e01e4420 con 0x7ff6e0152b30 2024-03-04T07:33:32.382 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ceffd640 1 -- 172.21.15.136:0/840647836 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff6d80b7070 con 0x7ff6e0152b30 2024-03-04T07:33:32.382 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6ceffd640 1 -- 172.21.15.136:0/840647836 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff6d80c6d70 con 0x7ff6e0152b30 2024-03-04T07:33:32.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.377+0000 7ff6e4ead640 1 -- 172.21.15.136:0/840647836 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e0075c00 con 0x7ff6e0152b30 2024-03-04T07:33:32.383 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.381+0000 7ff6ceffd640 1 -- 172.21.15.136:0/840647836 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff6d8018d40 con 0x7ff6e0152b30 2024-03-04T07:33:32.386 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:32.385+0000 7fc375c73180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T07:33:32.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.449+0000 7ff6e4ead640 1 -- 172.21.15.136:0/840647836 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd new", "uuid": "94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1", "id": 4} v 0) v1 -- 0x7ff6e0156250 con 0x7ff6e0152b30 2024-03-04T07:33:32.459 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.457+0000 7ff6ceffd640 1 -- 172.21.15.136:0/840647836 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff6d80c6460 con 0x7ff6e0152b30 2024-03-04T07:33:32.459 INFO:teuthology.orchestra.run.smithi136.stdout:4 2024-03-04T07:33:32.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.457+0000 7ff6e4ead640 1 -- 172.21.15.136:0/840647836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e0152b30 msgr2=0x7ff6e00cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.457+0000 7ff6e4ead640 1 --2- 172.21.15.136:0/840647836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e0152b30 0x7ff6e00cbaa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff6d801bdf0 tx=0x7ff6d801bf80 comp rx=0 tx=0).stop 2024-03-04T07:33:32.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.457+0000 7ff6e4ead640 1 -- 172.21.15.136:0/840647836 shutdown_connections 2024-03-04T07:33:32.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.457+0000 7ff6e4ead640 1 --2- 172.21.15.136:0/840647836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6e0152b30 0x7ff6e00cbaa0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.457+0000 7ff6e4ead640 1 --2- 172.21.15.136:0/840647836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff6e00c8f20 0x7ff6e01f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.457+0000 7ff6e4ead640 1 --2- 172.21.15.136:0/840647836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6e00c4e80 0x7ff6e0150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.457+0000 7ff6e4ead640 1 -- 172.21.15.136:0/840647836 >> 172.21.15.136:0/840647836 conn(0x7ff6e006b060 msgr2=0x7ff6e01513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:32.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.457+0000 7ff6e4ead640 1 -- 172.21.15.136:0/840647836 shutdown_connections 2024-03-04T07:33:32.461 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.457+0000 7ff6e4ead640 1 -- 172.21.15.136:0/840647836 wait complete. 2024-03-04T07:33:32.466 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:32.465+0000 7fc375c73180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T07:33:32.475 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 2949efcf-b78e-411c-87da-f27685b009bc 5 2024-03-04T07:33:32.520 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:32.517+0000 7fc375c73180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T07:33:32.599 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:32.597+0000 7fc375c73180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T07:33:32.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.629+0000 7f333b5c1640 1 Processor -- start 2024-03-04T07:33:32.632 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.629+0000 7f333b5c1640 1 -- start start 2024-03-04T07:33:32.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.629+0000 7f333b5c1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f333406fb00 0x7f33340ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.629+0000 7f333b5c1640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f333406fee0 0x7f3334071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.629+0000 7f333b5c1640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33340c7c90 0x7f3334152520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.629+0000 7f333b5c1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33340b0040 con 0x7f333406fb00 2024-03-04T07:33:32.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.629+0000 7f333b5c1640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333406b600 con 0x7f333406fee0 2024-03-04T07:33:32.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.629+0000 7f333b5c1640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333406b8c0 con 0x7f33340c7c90 2024-03-04T07:33:32.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3339b5e640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33340c7c90 0x7f3334152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3338b5c640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f333406fee0 0x7f3334071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.634 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3339b5e640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33340c7c90 0x7f3334152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:55102/0 (socket says 172.21.15.136:55102) 2024-03-04T07:33:32.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3339b5e640 1 -- 172.21.15.136:0/1302891207 learned_addr learned my addr 172.21.15.136:0/1302891207 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:32.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333935d640 1 --2- 172.21.15.136:0/1302891207 >> v2:172.21.15.113:3300/0 conn(0x7f333406fb00 0x7f33340ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3339b5e640 1 -- 172.21.15.136:0/1302891207 >> v2:172.21.15.136:3300/0 conn(0x7f333406fee0 msgr2=0x7f3334071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:32.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3339b5e640 1 --2- 172.21.15.136:0/1302891207 >> v2:172.21.15.136:3300/0 conn(0x7f333406fee0 0x7f3334071cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3339b5e640 1 -- 172.21.15.136:0/1302891207 >> v2:172.21.15.113:3300/0 conn(0x7f333406fb00 msgr2=0x7f33340ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:32.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3339b5e640 1 --2- 172.21.15.136:0/1302891207 >> v2:172.21.15.113:3300/0 conn(0x7f333406fb00 0x7f33340ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3339b5e640 1 -- 172.21.15.136:0/1302891207 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333406bba0 con 0x7f33340c7c90 2024-03-04T07:33:32.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3338b5c640 1 --2- 172.21.15.136:0/1302891207 >> v2:172.21.15.136:3300/0 conn(0x7f333406fee0 0x7f3334071cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:33:32.635 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333935d640 1 --2- 172.21.15.136:0/1302891207 >> v2:172.21.15.113:3300/0 conn(0x7f333406fb00 0x7f33340ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:32.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f3339b5e640 1 --2- 172.21.15.136:0/1302891207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33340c7c90 0x7f3334152520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f333406f5d0 tx=0x7f332c01c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc53c1874c446b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:32.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f332bfff640 1 -- 172.21.15.136:0/1302891207 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c0a5640 con 0x7f33340c7c90 2024-03-04T07:33:32.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f332bfff640 1 -- 172.21.15.136:0/1302891207 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c0a57e0 con 0x7f33340c7c90 2024-03-04T07:33:32.636 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f332bfff640 1 -- 172.21.15.136:0/1302891207 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c0a5bb0 con 0x7f33340c7c90 2024-03-04T07:33:32.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333b5c1640 1 -- 172.21.15.136:0/1302891207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33340c7c90 msgr2=0x7f3334152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333b5c1640 1 --2- 172.21.15.136:0/1302891207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33340c7c90 0x7f3334152520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f333406f5d0 tx=0x7f332c01c9c0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333b5c1640 1 -- 172.21.15.136:0/1302891207 shutdown_connections 2024-03-04T07:33:32.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333b5c1640 1 --2- 172.21.15.136:0/1302891207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f33340c7c90 0x7f3334152520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333b5c1640 1 --2- 172.21.15.136:0/1302891207 >> v2:172.21.15.136:3300/0 conn(0x7f333406fee0 0x7f3334071cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333b5c1640 1 --2- 172.21.15.136:0/1302891207 >> v2:172.21.15.113:3300/0 conn(0x7f333406fb00 0x7f33340ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333b5c1640 1 -- 172.21.15.136:0/1302891207 >> 172.21.15.136:0/1302891207 conn(0x7f333406e240 msgr2=0x7f333406e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:32.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333b5c1640 1 -- 172.21.15.136:0/1302891207 shutdown_connections 2024-03-04T07:33:32.637 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.633+0000 7f333b5c1640 1 -- 172.21.15.136:0/1302891207 wait complete. 2024-03-04T07:33:32.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 Processor -- start 2024-03-04T07:33:32.638 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 -- start start 2024-03-04T07:33:32.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f333406fb00 0x7f33340cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333406fee0 0x7f33341f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33340c7c90 0x7f33341f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334073500 con 0x7f33340c7c90 2024-03-04T07:33:32.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340721f0 con 0x7f333406fee0 2024-03-04T07:33:32.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333414c530 con 0x7f333406fb00 2024-03-04T07:33:32.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f3338b5c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333406fee0 0x7f33341f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333935d640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f333406fb00 0x7f33340cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333935d640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f333406fb00 0x7f33340cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:55114/0 (socket says 172.21.15.136:55114) 2024-03-04T07:33:32.640 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333935d640 1 -- 172.21.15.136:0/2112038930 learned_addr learned my addr 172.21.15.136:0/2112038930 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f3339b5e640 1 --2- 172.21.15.136:0/2112038930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33340c7c90 0x7f33341f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333935d640 1 -- 172.21.15.136:0/2112038930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333406fee0 msgr2=0x7f33341f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333935d640 1 --2- 172.21.15.136:0/2112038930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333406fee0 0x7f33341f4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333935d640 1 -- 172.21.15.136:0/2112038930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33340c7c90 msgr2=0x7f33341f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333935d640 1 --2- 172.21.15.136:0/2112038930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33340c7c90 0x7f33341f4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333935d640 1 -- 172.21.15.136:0/2112038930 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33341f13c0 con 0x7f333406fb00 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f3338b5c640 1 --2- 172.21.15.136:0/2112038930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333406fee0 0x7f33341f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333935d640 1 --2- 172.21.15.136:0/2112038930 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f333406fb00 0x7f33340cc470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f333000d830 tx=0x7f333000dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f3329ffb640 1 -- 172.21.15.136:0/2112038930 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330018070 con 0x7f333406fb00 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f3329ffb640 1 -- 172.21.15.136:0/2112038930 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333000be10 con 0x7f333406fb00 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f3329ffb640 1 -- 172.21.15.136:0/2112038930 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330004250 con 0x7f333406fb00 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 -- 172.21.15.136:0/2112038930 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3334151d60 con 0x7f333406fb00 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 -- 172.21.15.136:0/2112038930 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33341f1c00 con 0x7f333406fb00 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f3329ffb640 1 -- 172.21.15.136:0/2112038930 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3330009c40 con 0x7f333406fb00 2024-03-04T07:33:32.641 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f3329ffb640 1 -- 172.21.15.136:0/2112038930 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f333001d9d0 con 0x7f333406fb00 2024-03-04T07:33:32.642 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.637+0000 7f333b5c1640 1 -- 172.21.15.136:0/2112038930 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3334075c00 con 0x7f333406fb00 2024-03-04T07:33:32.643 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.641+0000 7f3329ffb640 1 -- 172.21.15.136:0/2112038930 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f33341f13c0 con 0x7f333406fb00 2024-03-04T07:33:32.656 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:32.653+0000 7fc375c73180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T07:33:32.712 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.709+0000 7f333b5c1640 1 -- 172.21.15.136:0/2112038930 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2949efcf-b78e-411c-87da-f27685b009bc", "id": 5} v 0) v1 -- 0x7f3334149e00 con 0x7f333406fb00 2024-03-04T07:33:32.717 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.717+0000 7f3329ffb640 1 -- 172.21.15.136:0/2112038930 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2949efcf-b78e-411c-87da-f27685b009bc", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f333001d450 con 0x7f333406fb00 2024-03-04T07:33:32.718 INFO:teuthology.orchestra.run.smithi136.stdout:5 2024-03-04T07:33:32.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.717+0000 7f333b5c1640 1 -- 172.21.15.136:0/2112038930 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f333406fb00 msgr2=0x7f33340cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.717+0000 7f333b5c1640 1 --2- 172.21.15.136:0/2112038930 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f333406fb00 0x7f33340cc470 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f333000d830 tx=0x7f333000dcf0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.717+0000 7f333b5c1640 1 -- 172.21.15.136:0/2112038930 shutdown_connections 2024-03-04T07:33:32.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.717+0000 7f333b5c1640 1 --2- 172.21.15.136:0/2112038930 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f33340c7c90 0x7f33341f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.717+0000 7f333b5c1640 1 --2- 172.21.15.136:0/2112038930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333406fee0 0x7f33341f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.717+0000 7f333b5c1640 1 --2- 172.21.15.136:0/2112038930 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f333406fb00 0x7f33340cc470 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.717+0000 7f333b5c1640 1 -- 172.21.15.136:0/2112038930 >> 172.21.15.136:0/2112038930 conn(0x7f333406e240 msgr2=0x7f333414db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:32.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.717+0000 7f333b5c1640 1 -- 172.21.15.136:0/2112038930 shutdown_connections 2024-03-04T07:33:32.719 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.717+0000 7f333b5c1640 1 -- 172.21.15.136:0/2112038930 wait complete. 2024-03-04T07:33:32.734 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new 9121e020-dcbd-4aea-af25-ce8e580d3968 6 2024-03-04T07:33:32.761 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:32.760+0000 7f02fca55180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T07:33:32.808 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:32.805+0000 7fc375c73180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T07:33:32.822 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:32.824+0000 7f02fca55180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T07:33:32.872 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:32.869+0000 7fc375c73180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T07:33:32.883 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:32.884+0000 7f02fca55180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T07:33:32.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.889+0000 7f29ab1da640 1 Processor -- start 2024-03-04T07:33:32.892 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.889+0000 7f29ab1da640 1 -- start start 2024-03-04T07:33:32.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29ab1da640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29a406fee0 0x7f29a40ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29ab1da640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f29a4073370 0x7f29a4154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29ab1da640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a4071f00 con 0x7f29a406fb00 2024-03-04T07:33:32.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29ab1da640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a4071cb0 con 0x7f29a406fee0 2024-03-04T07:33:32.894 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29ab1da640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a40cd3a0 con 0x7f29a4073370 2024-03-04T07:33:32.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29a9777640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f29a4073370 0x7f29a4154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29a9777640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f29a4073370 0x7f29a4154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:52306/0 (socket says 172.21.15.136:52306) 2024-03-04T07:33:32.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29a9777640 1 -- 172.21.15.136:0/4067713865 learned_addr learned my addr 172.21.15.136:0/4067713865 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:32.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29a3fff640 1 --2- 172.21.15.136:0/4067713865 >> v2:172.21.15.113:3300/0 conn(0x7f29a406fee0 0x7f29a40ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29a9777640 1 -- 172.21.15.136:0/4067713865 >> v2:172.21.15.113:3300/0 conn(0x7f29a406fee0 msgr2=0x7f29a40ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:32.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29a9777640 1 --2- 172.21.15.136:0/4067713865 >> v2:172.21.15.113:3300/0 conn(0x7f29a406fee0 0x7f29a40ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29a9777640 1 -- 172.21.15.136:0/4067713865 >> v1:172.21.15.113:6789/0 conn(0x7f29a406fb00 legacy=0x7f29a415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.895 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.893+0000 7f29a9777640 1 -- 172.21.15.136:0/4067713865 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a40cd680 con 0x7f29a4073370 2024-03-04T07:33:32.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a9777640 1 --2- 172.21.15.136:0/4067713865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 0x7f29a4154ad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f299c009e50 tx=0x7f299c01f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10fceafc555b7591 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:32.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a37fe640 1 -- 172.21.15.136:0/4067713865 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c0b4070 con 0x7f29a4073370 2024-03-04T07:33:32.897 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a37fe640 1 -- 172.21.15.136:0/4067713865 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299c0af400 con 0x7f29a4073370 2024-03-04T07:33:32.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a37fe640 1 -- 172.21.15.136:0/4067713865 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c0af7d0 con 0x7f29a4073370 2024-03-04T07:33:32.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- 172.21.15.136:0/4067713865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 msgr2=0x7f29a4154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 --2- 172.21.15.136:0/4067713865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 0x7f29a4154ad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f299c009e50 tx=0x7f299c01f1e0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.898 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- 172.21.15.136:0/4067713865 shutdown_connections 2024-03-04T07:33:32.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 --2- 172.21.15.136:0/4067713865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 0x7f29a4154ad0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 --2- 172.21.15.136:0/4067713865 >> v2:172.21.15.113:3300/0 conn(0x7f29a406fee0 0x7f29a40ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- 172.21.15.136:0/4067713865 >> 172.21.15.136:0/4067713865 conn(0x7f29a406e240 msgr2=0x7f29a406e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:32.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- 172.21.15.136:0/4067713865 shutdown_connections 2024-03-04T07:33:32.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- 172.21.15.136:0/4067713865 wait complete. 2024-03-04T07:33:32.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 Processor -- start 2024-03-04T07:33:32.899 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- start start 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a406fb00 0x7f29a40aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a406fee0 0x7f29a40c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 0x7f29a41ee640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40c9c90 con 0x7f29a406fb00 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40cdc40 con 0x7f29a4073370 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40c9e10 con 0x7f29a406fee0 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a9777640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 0x7f29a41ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a3fff640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a406fee0 0x7f29a40c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a9777640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 0x7f29a41ee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:52316/0 (socket says 172.21.15.136:52316) 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a9777640 1 -- 172.21.15.136:0/1059278579 learned_addr learned my addr 172.21.15.136:0/1059278579 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:32.902 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a8f76640 1 --2- 172.21.15.136:0/1059278579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a406fb00 0x7f29a40aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a3fff640 1 -- 172.21.15.136:0/1059278579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 msgr2=0x7f29a41ee640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a3fff640 1 --2- 172.21.15.136:0/1059278579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 0x7f29a41ee640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a3fff640 1 -- 172.21.15.136:0/1059278579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a406fb00 msgr2=0x7f29a40aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a3fff640 1 --2- 172.21.15.136:0/1059278579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a406fb00 0x7f29a40aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a3fff640 1 -- 172.21.15.136:0/1059278579 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a40ccfd0 con 0x7f29a406fee0 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a9777640 1 --2- 172.21.15.136:0/1059278579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 0x7f29a41ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a3fff640 1 --2- 172.21.15.136:0/1059278579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a406fee0 0x7f29a40c80a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f299000e820 tx=0x7f299000ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a17fa640 1 -- 172.21.15.136:0/1059278579 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990017070 con 0x7f29a406fee0 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a17fa640 1 -- 172.21.15.136:0/1059278579 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990004020 con 0x7f29a406fee0 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- 172.21.15.136:0/1059278579 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a41edeb0 con 0x7f29a406fee0 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29a17fa640 1 -- 172.21.15.136:0/1059278579 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990004410 con 0x7f29a406fee0 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.897+0000 7f29ab1da640 1 -- 172.21.15.136:0/1059278579 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a40ab120 con 0x7f29a406fee0 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.901+0000 7f29a17fa640 1 -- 172.21.15.136:0/1059278579 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f29900059a0 con 0x7f29a406fee0 2024-03-04T07:33:32.903 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.901+0000 7f29a17fa640 1 -- 172.21.15.136:0/1059278579 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f2990003df0 con 0x7f29a406fee0 2024-03-04T07:33:32.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.901+0000 7f29ab1da640 1 -- 172.21.15.136:0/1059278579 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a4075c00 con 0x7f29a406fee0 2024-03-04T07:33:32.904 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.901+0000 7f29a17fa640 1 -- 172.21.15.136:0/1059278579 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f299000cd00 con 0x7f29a406fee0 2024-03-04T07:33:32.971 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.969+0000 7f29ab1da640 1 -- 172.21.15.136:0/1059278579 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9121e020-dcbd-4aea-af25-ce8e580d3968", "id": 6} v 0) v1 -- 0x7f29a40c8960 con 0x7f29a406fee0 2024-03-04T07:33:32.972 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:32.972+0000 7f02fca55180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T07:33:32.979 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.977+0000 7f29a17fa640 1 -- 172.21.15.136:0/1059278579 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9121e020-dcbd-4aea-af25-ce8e580d3968", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2990003800 con 0x7f29a406fee0 2024-03-04T07:33:32.979 INFO:teuthology.orchestra.run.smithi136.stdout:6 2024-03-04T07:33:32.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.977+0000 7f29ab1da640 1 -- 172.21.15.136:0/1059278579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a406fee0 msgr2=0x7f29a40c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:32.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.977+0000 7f29ab1da640 1 --2- 172.21.15.136:0/1059278579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a406fee0 0x7f29a40c80a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f299000e820 tx=0x7f299000ece0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.977+0000 7f29ab1da640 1 -- 172.21.15.136:0/1059278579 shutdown_connections 2024-03-04T07:33:32.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.977+0000 7f29ab1da640 1 --2- 172.21.15.136:0/1059278579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a4073370 0x7f29a41ee640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.977+0000 7f29ab1da640 1 --2- 172.21.15.136:0/1059278579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a406fee0 0x7f29a40c80a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.977+0000 7f29ab1da640 1 --2- 172.21.15.136:0/1059278579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a406fb00 0x7f29a40aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:32.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.977+0000 7f29ab1da640 1 -- 172.21.15.136:0/1059278579 >> 172.21.15.136:0/1059278579 conn(0x7f29a406e240 msgr2=0x7f29a40b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:32.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.977+0000 7f29ab1da640 1 -- 172.21.15.136:0/1059278579 shutdown_connections 2024-03-04T07:33:32.981 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:32.977+0000 7f29ab1da640 1 -- 172.21.15.136:0/1059278579 wait complete. 2024-03-04T07:33:32.996 DEBUG:teuthology.orchestra.run.smithi136:> sudo ceph --cluster ceph osd new c73d46d6-a743-462e-913a-2cb43fec5a6f 7 2024-03-04T07:33:33.032 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:33.032+0000 7f02fca55180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T07:33:33.119 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:33.120+0000 7f02fca55180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T07:33:33.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.149+0000 7f31a1889640 1 Processor -- start 2024-03-04T07:33:33.153 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.149+0000 7f31a1889640 1 -- start start 2024-03-04T07:33:33.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f31a1889640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f31a1889640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f319c0c8a70 0x7f319c0c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f31a1889640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319c0cb4d0 con 0x7f319c0c7960 2024-03-04T07:33:33.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f31a1889640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319c15ba40 con 0x7f319c0c8a70 2024-03-04T07:33:33.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f31a1889640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319c0c93f0 con 0x7f319c154d80 2024-03-04T07:33:33.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f319affd640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f319affd640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:55130/0 (socket says 172.21.15.136:55130) 2024-03-04T07:33:33.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f319b7fe640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f319c0c8a70 0x7f319c0c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.155 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f319affd640 1 -- 172.21.15.136:0/2589645205 learned_addr learned my addr 172.21.15.136:0/2589645205 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:33.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f319b7fe640 1 -- 172.21.15.136:0/2589645205 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 msgr2=0x7f319c15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:33.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f319b7fe640 1 --2- 172.21.15.136:0/2589645205 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f319b7fe640 1 -- 172.21.15.136:0/2589645205 >> v1:172.21.15.136:6789/0 conn(0x7f319c0c7960 legacy=0x7f319c0d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f319b7fe640 1 -- 172.21.15.136:0/2589645205 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c0cba50 con 0x7f319c0c8a70 2024-03-04T07:33:33.156 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f319affd640 1 --2- 172.21.15.136:0/2589645205 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:33:33.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f319b7fe640 1 --2- 172.21.15.136:0/2589645205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f319c0c8a70 0x7f319c0c8eb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3194009d90 tx=0x7f319401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1728969e770c071 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:33.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f3199ffb640 1 -- 172.21.15.136:0/2589645205 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31940b2070 con 0x7f319c0c8a70 2024-03-04T07:33:33.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f3199ffb640 1 -- 172.21.15.136:0/2589645205 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31940a5910 con 0x7f319c0c8a70 2024-03-04T07:33:33.157 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.153+0000 7f3199ffb640 1 -- 172.21.15.136:0/2589645205 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31940a5ce0 con 0x7f319c0c8a70 2024-03-04T07:33:33.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- 172.21.15.136:0/2589645205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f319c0c8a70 msgr2=0x7f319c0c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 --2- 172.21.15.136:0/2589645205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f319c0c8a70 0x7f319c0c8eb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3194009d90 tx=0x7f319401c940 comp rx=0 tx=0).stop 2024-03-04T07:33:33.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- 172.21.15.136:0/2589645205 shutdown_connections 2024-03-04T07:33:33.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 --2- 172.21.15.136:0/2589645205 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f319c0c8a70 0x7f319c0c8eb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 --2- 172.21.15.136:0/2589645205 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- 172.21.15.136:0/2589645205 >> 172.21.15.136:0/2589645205 conn(0x7f319c06c090 msgr2=0x7f319c06d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:33.158 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- 172.21.15.136:0/2589645205 shutdown_connections 2024-03-04T07:33:33.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- 172.21.15.136:0/2589645205 wait complete. 2024-03-04T07:33:33.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 Processor -- start 2024-03-04T07:33:33.159 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- start start 2024-03-04T07:33:33.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f319c0c7960 0x7f319c0c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f319c0c8a70 0x7f319c0c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c155160 con 0x7f319c0c7960 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c0c8550 con 0x7f319c0c8a70 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319c0b60d0 con 0x7f319c154d80 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319a7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f319c0c8a70 0x7f319c0c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319b7fe640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319a7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f319c0c8a70 0x7f319c0c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:52336/0 (socket says 172.21.15.136:52336) 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319a7fc640 1 -- 172.21.15.136:0/2589029086 learned_addr learned my addr 172.21.15.136:0/2589029086 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319b7fe640 1 --2- 172.21.15.136:0/2589029086 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.136:55146/0 (socket says 172.21.15.136:55146) 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319affd640 1 --2- 172.21.15.136:0/2589029086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f319c0c7960 0x7f319c0c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319b7fe640 1 -- 172.21.15.136:0/2589029086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f319c0c8a70 msgr2=0x7f319c0c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319b7fe640 1 --2- 172.21.15.136:0/2589029086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f319c0c8a70 0x7f319c0c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319b7fe640 1 -- 172.21.15.136:0/2589029086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f319c0c7960 msgr2=0x7f319c0c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319b7fe640 1 --2- 172.21.15.136:0/2589029086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f319c0c7960 0x7f319c0c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319b7fe640 1 -- 172.21.15.136:0/2589029086 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c0cfb50 con 0x7f319c154d80 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319affd640 1 --2- 172.21.15.136:0/2589029086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f319c0c7960 0x7f319c0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f319b7fe640 1 --2- 172.21.15.136:0/2589029086 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15d810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f31940ac000 tx=0x7f31940196e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f3183fff640 1 -- 172.21.15.136:0/2589029086 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31940b2050 con 0x7f319c154d80 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f3183fff640 1 -- 172.21.15.136:0/2589029086 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31940b3040 con 0x7f319c154d80 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f3183fff640 1 -- 172.21.15.136:0/2589029086 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31940ad870 con 0x7f319c154d80 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- 172.21.15.136:0/2589029086 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319c0c4ea0 con 0x7f319c154d80 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f31a1889640 1 -- 172.21.15.136:0/2589029086 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319c0c5440 con 0x7f319c154d80 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f3183fff640 1 -- 172.21.15.136:0/2589029086 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f31940c0020 con 0x7f319c154d80 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.157+0000 7f3183fff640 1 -- 172.21.15.136:0/2589029086 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f31940b9d80 con 0x7f319c154d80 2024-03-04T07:33:33.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.161+0000 7f31a1889640 1 -- 172.21.15.136:0/2589029086 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319c075c00 con 0x7f319c154d80 2024-03-04T07:33:33.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.161+0000 7f3183fff640 1 -- 172.21.15.136:0/2589029086 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f31940c7020 con 0x7f319c154d80 2024-03-04T07:33:33.181 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:33.180+0000 7f02fca55180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T07:33:33.279 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.277+0000 7f31a1889640 1 -- 172.21.15.136:0/2589029086 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c73d46d6-a743-462e-913a-2cb43fec5a6f", "id": 7} v 0) v1 -- 0x7f319c165140 con 0x7f319c154d80 2024-03-04T07:33:33.283 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.281+0000 7f3183fff640 1 -- 172.21.15.136:0/2589029086 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c73d46d6-a743-462e-913a-2cb43fec5a6f", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3194016560 con 0x7f319c154d80 2024-03-04T07:33:33.283 INFO:teuthology.orchestra.run.smithi136.stdout:7 2024-03-04T07:33:33.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.281+0000 7f31a1889640 1 -- 172.21.15.136:0/2589029086 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 msgr2=0x7f319c15d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.281+0000 7f31a1889640 1 --2- 172.21.15.136:0/2589029086 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15d810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f31940ac000 tx=0x7f31940196e0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.281+0000 7f31a1889640 1 -- 172.21.15.136:0/2589029086 shutdown_connections 2024-03-04T07:33:33.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.281+0000 7f31a1889640 1 --2- 172.21.15.136:0/2589029086 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f319c154d80 0x7f319c15d810 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.281+0000 7f31a1889640 1 --2- 172.21.15.136:0/2589029086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f319c0c8a70 0x7f319c0c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.281+0000 7f31a1889640 1 --2- 172.21.15.136:0/2589029086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f319c0c7960 0x7f319c0c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.281+0000 7f31a1889640 1 -- 172.21.15.136:0/2589029086 >> 172.21.15.136:0/2589029086 conn(0x7f319c06c090 msgr2=0x7f319c14e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:33.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.281+0000 7f31a1889640 1 -- 172.21.15.136:0/2589029086 shutdown_connections 2024-03-04T07:33:33.284 INFO:teuthology.orchestra.run.smithi136.stderr:2024-03-04T07:33:33.281+0000 7f31a1889640 1 -- 172.21.15.136:0/2589029086 wait complete. 2024-03-04T07:33:33.299 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-04T07:33:33.299 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-04T07:33:33.302 INFO:tasks.ceph.osd.0:Started 2024-03-04T07:33:33.302 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-04T07:33:33.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-04T07:33:33.306 INFO:tasks.ceph.osd.1:Started 2024-03-04T07:33:33.306 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-04T07:33:33.306 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-04T07:33:33.309 INFO:tasks.ceph.osd.2:Started 2024-03-04T07:33:33.309 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-04T07:33:33.309 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-04T07:33:33.313 INFO:tasks.ceph.osd.3:Started 2024-03-04T07:33:33.313 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-04T07:33:33.313 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-04T07:33:33.315 INFO:tasks.ceph.osd.4:Started 2024-03-04T07:33:33.315 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-04T07:33:33.315 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-04T07:33:33.317 INFO:tasks.ceph.osd.5:Started 2024-03-04T07:33:33.317 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-04T07:33:33.317 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-04T07:33:33.319 INFO:tasks.ceph.osd.6:Started 2024-03-04T07:33:33.319 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-04T07:33:33.319 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-04T07:33:33.322 INFO:tasks.ceph.osd.7:Started 2024-03-04T07:33:33.322 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:33:33.351 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:33.352+0000 7f02fca55180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T07:33:33.469 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:33.468+0000 7f02fca55180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T07:33:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09f2800640 1 Processor -- start 2024-03-04T07:33:33.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09f2800640 1 -- start start 2024-03-04T07:33:33.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09f2800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec1527f0 0x7f09ec070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09f2800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec1591f0 con 0x7f09ec0c8200 2024-03-04T07:33:33.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09f2800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec1594d0 con 0x7f09ec152bd0 2024-03-04T07:33:33.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09f2800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec071df0 con 0x7f09ec1527f0 2024-03-04T07:33:33.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09eaa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f09ec0c8200 0x7f09ec072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36422/0 (socket says 172.21.15.113:36422) 2024-03-04T07:33:33.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09eaa00640 1 -- 172.21.15.113:0/447680195 learned_addr learned my addr 172.21.15.113:0/447680195 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:33:33.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09eb400640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec1527f0 0x7f09ec070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593335651 0 0) 0x7f09ec1591f0 con 0x7f09ec0c8200 2024-03-04T07:33:33.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09d0003530 con 0x7f09ec0c8200 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930196585 0 0) 0x7f09ec1594d0 con 0x7f09ec152bd0 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09ec1591f0 con 0x7f09ec152bd0 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234663077 0 0) 0x7f09d0003530 con 0x7f09ec0c8200 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec1527f0 msgr2=0x7f09ec070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec1527f0 0x7f09ec070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 >> v1:172.21.15.136:6789/0 conn(0x7f09ec152bd0 legacy=0x7f09ec1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec0717a0 con 0x7f09ec0c8200 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f09d8001680 con 0x7f09ec0c8200 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 >> v1:172.21.15.113:6789/0 conn(0x7f09ec0c8200 legacy=0x7f09ec072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec1527f0 0x7f09d0003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 0x7f09d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09d0004d50 0x7f09d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec071df0 con 0x7f09d0004130 2024-03-04T07:33:33.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d000bb10 con 0x7f09d0004d50 2024-03-04T07:33:33.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d000bd10 con 0x7f09ec1527f0 2024-03-04T07:33:33.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09eaa00640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 0x7f09d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09ebe00640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09d0004d50 0x7f09d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09eb400640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec1527f0 0x7f09d0003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09eaa00640 1 -- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec1527f0 msgr2=0x7f09d0003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:33.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09eaa00640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec1527f0 0x7f09d0003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09eaa00640 1 -- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09d0004d50 msgr2=0x7f09d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:33.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09eaa00640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09d0004d50 0x7f09d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.516+0000 7f09eaa00640 1 -- 172.21.15.113:0/447680195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d0003530 con 0x7f09d0004130 2024-03-04T07:33:33.521 INFO:tasks.ceph.mgr.x.smithi136.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-04T07:33:33.521 INFO:tasks.ceph.mgr.x.smithi136.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-04T07:33:33.521 INFO:tasks.ceph.mgr.x.smithi136.stderr: from numpy import show_config as show_numpy_config 2024-03-04T07:33:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09eaa00640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 0x7f09d0004810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09d8007f80 tx=0x7f09d801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd86c4fbdfc54827 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8007bd0 con 0x7f09d0004130 2024-03-04T07:33:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d80a9020 con 0x7f09d0004130 2024-03-04T07:33:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09ea000640 1 -- 172.21.15.113:0/447680195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d8005b00 con 0x7f09d0004130 2024-03-04T07:33:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09f2800640 1 -- 172.21.15.113:0/447680195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 msgr2=0x7f09d0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09f2800640 1 --2- 172.21.15.113:0/447680195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 0x7f09d0004810 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09d8007f80 tx=0x7f09d801ca20 comp rx=0 tx=0).stop 2024-03-04T07:33:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09ebe00640 1 -- 172.21.15.113:0/447680195 reap_dead start 2024-03-04T07:33:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09f2800640 1 -- 172.21.15.113:0/447680195 shutdown_connections 2024-03-04T07:33:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09f2800640 1 -- 172.21.15.113:0/447680195 >> 172.21.15.113:0/447680195 conn(0x7f09ec06e820 msgr2=0x7f09ec06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:33.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09f2800640 1 -- 172.21.15.113:0/447680195 shutdown_connections 2024-03-04T07:33:33.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.520+0000 7f09f2800640 1 -- 172.21.15.113:0/447680195 wait complete. 2024-03-04T07:33:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09f2800640 1 Processor -- start 2024-03-04T07:33:33.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09f2800640 1 -- start start 2024-03-04T07:33:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09f2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 0x7f09ec0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09f2800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec0c1de0 0x7f09ec1f1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09f2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09ec1f3d30 0x7f09ec150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09f2800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec158ee0 con 0x7f09d0004130 2024-03-04T07:33:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09f2800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec1597b0 con 0x7f09ec1f3d30 2024-03-04T07:33:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09f2800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec070d40 con 0x7f09ec0c1de0 2024-03-04T07:33:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09eb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 0x7f09ec0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09eb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 0x7f09ec0c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54150/0 (socket says 172.21.15.113:54150) 2024-03-04T07:33:33.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09eb400640 1 -- 172.21.15.113:0/917527597 learned_addr learned my addr 172.21.15.113:0/917527597 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:33.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09ebe00640 1 --2- 172.21.15.113:0/917527597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09ec1f3d30 0x7f09ec150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09eaa00640 1 --2- 172.21.15.113:0/917527597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec0c1de0 0x7f09ec1f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:33.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09eb400640 1 -- 172.21.15.113:0/917527597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec0c1de0 msgr2=0x7f09ec1f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09eb400640 1 --2- 172.21.15.113:0/917527597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec0c1de0 0x7f09ec1f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09eb400640 1 -- 172.21.15.113:0/917527597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09ec1f3d30 msgr2=0x7f09ec150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09eb400640 1 --2- 172.21.15.113:0/917527597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09ec1f3d30 0x7f09ec150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09eb400640 1 -- 172.21.15.113:0/917527597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec1f08d0 con 0x7f09d0004130 2024-03-04T07:33:33.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09eb400640 1 --2- 172.21.15.113:0/917527597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 0x7f09ec0c8f80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f09dc011a00 tx=0x7f09dc011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:33.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09e3400640 1 -- 172.21.15.113:0/917527597 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00fb50 con 0x7f09d0004130 2024-03-04T07:33:33.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09e3400640 1 -- 172.21.15.113:0/917527597 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc00fcf0 con 0x7f09d0004130 2024-03-04T07:33:33.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09e3400640 1 -- 172.21.15.113:0/917527597 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00fea0 con 0x7f09d0004130 2024-03-04T07:33:33.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.524+0000 7f09f2800640 1 -- 172.21.15.113:0/917527597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ec1f4300 con 0x7f09d0004130 2024-03-04T07:33:33.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.528+0000 7f09f2800640 1 -- 172.21.15.113:0/917527597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec1f48c0 con 0x7f09d0004130 2024-03-04T07:33:33.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.528+0000 7f09e3400640 1 -- 172.21.15.113:0/917527597 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f09dc012040 con 0x7f09d0004130 2024-03-04T07:33:33.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.528+0000 7f09e3400640 1 -- 172.21.15.113:0/917527597 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f09dc01ddf0 con 0x7f09d0004130 2024-03-04T07:33:33.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.528+0000 7f09f2800640 1 -- 172.21.15.113:0/917527597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ec075c00 con 0x7f09d0004130 2024-03-04T07:33:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.552+0000 7f09e3400640 1 -- 172.21.15.113:0/917527597 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f09dc019030 con 0x7f09d0004130 2024-03-04T07:33:33.561 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:33.557+0000 7fc375c73180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T07:33:33.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09f2800640 1 -- 172.21.15.113:0/917527597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09ec0cbc30 con 0x7f09d0004130 2024-03-04T07:33:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09e3400640 1 -- 172.21.15.113:0/917527597 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f09dc01d720 con 0x7f09d0004130 2024-03-04T07:33:33.608 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:33:33.608 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":10,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:33.280839+0000","last_up_change":"0.000000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:33:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09f2800640 1 -- 172.21.15.113:0/917527597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 msgr2=0x7f09ec0c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09f2800640 1 --2- 172.21.15.113:0/917527597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 0x7f09ec0c8f80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f09dc011a00 tx=0x7f09dc011ec0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09f2800640 1 -- 172.21.15.113:0/917527597 shutdown_connections 2024-03-04T07:33:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09f2800640 1 --2- 172.21.15.113:0/917527597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09ec1f3d30 0x7f09ec150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09f2800640 1 --2- 172.21.15.113:0/917527597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09ec0c1de0 0x7f09ec1f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09f2800640 1 --2- 172.21.15.113:0/917527597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09d0004130 0x7f09ec0c8f80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09f2800640 1 -- 172.21.15.113:0/917527597 >> 172.21.15.113:0/917527597 conn(0x7f09ec06e820 msgr2=0x7f09ec0c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09f2800640 1 -- 172.21.15.113:0/917527597 shutdown_connections 2024-03-04T07:33:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:33.608+0000 7f09f2800640 1 -- 172.21.15.113:0/917527597 wait complete. 2024-03-04T07:33:33.627 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-04T07:33:33.627 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-04T07:33:33.722 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:33.721+0000 7fc375c73180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T07:33:33.895 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:33.893+0000 7fc375c73180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T07:33:33.923 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-03-04T07:33:33.921+0000 7fa1703a37c0 -1 Falling back to public interface 2024-03-04T07:33:33.938 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-03-04T07:33:33.937+0000 7fbceb4d17c0 -1 Falling back to public interface 2024-03-04T07:33:33.942 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-03-04T07:33:33.941+0000 7fe7b56c17c0 -1 Falling back to public interface 2024-03-04T07:33:33.962 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-03-04T07:33:33.961+0000 7f85534ee7c0 -1 Falling back to public interface 2024-03-04T07:33:33.964 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-03-04T07:33:33.964+0000 7f6881ca47c0 -1 Falling back to public interface 2024-03-04T07:33:33.964 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:33.961+0000 7fc375c73180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T07:33:33.983 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-03-04T07:33:33.984+0000 7f0824aba7c0 -1 Falling back to public interface 2024-03-04T07:33:33.987 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-03-04T07:33:33.988+0000 7f05b407e7c0 -1 Falling back to public interface 2024-03-04T07:33:33.999 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-03-04T07:33:34.000+0000 7fdf1835c7c0 -1 Falling back to public interface 2024-03-04T07:33:34.099 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:34.097+0000 7fc375c73180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T07:33:34.123 INFO:tasks.ceph.mgr.y.smithi113.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-04T07:33:34.123 INFO:tasks.ceph.mgr.y.smithi113.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-04T07:33:34.123 INFO:tasks.ceph.mgr.y.smithi113.stderr: from numpy import show_config as show_numpy_config 2024-03-04T07:33:34.170 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:34.168+0000 7f02fca55180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T07:33:34.317 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:34.316+0000 7f02fca55180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T07:33:34.432 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T07:33:34.492 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:34.492+0000 7f02fca55180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T07:33:34.558 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.556+0000 7f7527a00640 1 Processor -- start 2024-03-04T07:33:34.558 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.556+0000 7f7527a00640 1 -- start start 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7527a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7520152770 0x7f75201588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7527a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7520158de0 0x7f75200c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7527a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75200c3570 con 0x7f7520152b50 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7527a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75200ca290 con 0x7f7520152770 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7527a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75200c3810 con 0x7f7520158de0 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7525000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7520152770 0x7f75201588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7525000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7520152770 0x7f75201588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54160/0 (socket says 172.21.15.113:54160) 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7525000640 1 -- 172.21.15.113:0/872507849 learned_addr learned my addr 172.21.15.113:0/872507849 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7525a00640 1 --2- 172.21.15.113:0/872507849 >> v2:172.21.15.136:3300/0 conn(0x7f7520158de0 0x7f75200c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7525000640 1 -- 172.21.15.113:0/872507849 >> v2:172.21.15.136:3300/0 conn(0x7f7520158de0 msgr2=0x7f75200c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7525000640 1 --2- 172.21.15.113:0/872507849 >> v2:172.21.15.136:3300/0 conn(0x7f7520158de0 0x7f75200c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:34.560 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7525000640 1 -- 172.21.15.113:0/872507849 >> v1:172.21.15.136:6789/0 conn(0x7f7520152b50 legacy=0x7f75200ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:34.561 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7525000640 1 -- 172.21.15.113:0/872507849 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75200c3af0 con 0x7f7520152770 2024-03-04T07:33:34.561 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.560+0000 7f7525a00640 1 --2- 172.21.15.113:0/872507849 >> v2:172.21.15.136:3300/0 conn(0x7f7520158de0 0x7f75200c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:34.563 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f7525000640 1 --2- 172.21.15.113:0/872507849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520152770 0x7f75201588a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f75200ca410 tx=0x7f751401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8becd76e24b5eade server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:34.563 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f751f400640 1 -- 172.21.15.113:0/872507849 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75140b5070 con 0x7f7520152770 2024-03-04T07:33:34.563 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f751f400640 1 -- 172.21.15.113:0/872507849 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75140a8a10 con 0x7f7520152770 2024-03-04T07:33:34.563 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f751f400640 1 -- 172.21.15.113:0/872507849 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75140a8de0 con 0x7f7520152770 2024-03-04T07:33:34.564 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f7527a00640 1 -- 172.21.15.113:0/872507849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520152770 msgr2=0x7f75201588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:34.564 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f7527a00640 1 --2- 172.21.15.113:0/872507849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520152770 0x7f75201588a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f75200ca410 tx=0x7f751401ed10 comp rx=0 tx=0).stop 2024-03-04T07:33:34.564 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f7527a00640 1 -- 172.21.15.113:0/872507849 shutdown_connections 2024-03-04T07:33:34.564 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f7527a00640 1 --2- 172.21.15.113:0/872507849 >> v2:172.21.15.136:3300/0 conn(0x7f7520158de0 0x7f75200c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:34.564 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f7527a00640 1 --2- 172.21.15.113:0/872507849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520152770 0x7f75201588a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:34.564 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f7527a00640 1 -- 172.21.15.113:0/872507849 >> 172.21.15.113:0/872507849 conn(0x7f752006b470 msgr2=0x7f7520066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:34.564 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f7527a00640 1 -- 172.21.15.113:0/872507849 shutdown_connections 2024-03-04T07:33:34.565 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f7527a00640 1 -- 172.21.15.113:0/872507849 wait complete. 2024-03-04T07:33:34.566 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.564+0000 7f7527a00640 1 Processor -- start 2024-03-04T07:33:34.566 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7527a00640 1 -- start start 2024-03-04T07:33:34.569 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7527a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7520152770 0x7f75200c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:34.569 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7527a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7520152b50 0x7f75201f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7527a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520158de0 0x7f75201f1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7527a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75200cbcf0 con 0x7f7520158de0 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7527a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75200ca7c0 con 0x7f7520152b50 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7527a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75200ceb10 con 0x7f7520152770 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7525a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520158de0 0x7f75201f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7525a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520158de0 0x7f75201f1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54164/0 (socket says 172.21.15.113:54164) 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7525a00640 1 -- 172.21.15.113:0/9645907 learned_addr learned my addr 172.21.15.113:0/9645907 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f751fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7520152b50 0x7f75201f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7525000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7520152770 0x7f75200c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7525a00640 1 -- 172.21.15.113:0/9645907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7520152770 msgr2=0x7f75200c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7525a00640 1 --2- 172.21.15.113:0/9645907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7520152770 0x7f75200c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7525a00640 1 -- 172.21.15.113:0/9645907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7520152b50 msgr2=0x7f75201f0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7525a00640 1 --2- 172.21.15.113:0/9645907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7520152b50 0x7f75201f0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7525a00640 1 -- 172.21.15.113:0/9645907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752006dfe0 con 0x7f7520158de0 2024-03-04T07:33:34.570 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7525a00640 1 --2- 172.21.15.113:0/9645907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520158de0 0x7f75201f1050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7518007e20 tx=0x7f751800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:34.571 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f751cc00640 1 -- 172.21.15.113:0/9645907 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751801a070 con 0x7f7520158de0 2024-03-04T07:33:34.571 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f751cc00640 1 -- 172.21.15.113:0/9645907 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7518003ba0 con 0x7f7520158de0 2024-03-04T07:33:34.571 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f751cc00640 1 -- 172.21.15.113:0/9645907 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751800de80 con 0x7f7520158de0 2024-03-04T07:33:34.571 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7527a00640 1 -- 172.21.15.113:0/9645907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75200c79e0 con 0x7f7520158de0 2024-03-04T07:33:34.571 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f7527a00640 1 -- 172.21.15.113:0/9645907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75200c7fa0 con 0x7f7520158de0 2024-03-04T07:33:34.571 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f02fca55180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T07:33:34.572 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f751cc00640 1 -- 172.21.15.113:0/9645907 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f751801d070 con 0x7f7520158de0 2024-03-04T07:33:34.572 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.568+0000 7f751cc00640 1 -- 172.21.15.113:0/9645907 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f7518004700 con 0x7f7520158de0 2024-03-04T07:33:34.572 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.572+0000 7f7527a00640 1 -- 172.21.15.113:0/9645907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7520068c80 con 0x7f7520158de0 2024-03-04T07:33:34.572 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.572+0000 7f751cc00640 1 -- 172.21.15.113:0/9645907 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7518024020 con 0x7f7520158de0 2024-03-04T07:33:34.631 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f7527a00640 1 -- 172.21.15.113:0/9645907 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75200c4dc0 con 0x7f7520158de0 2024-03-04T07:33:34.631 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f751cc00640 1 -- 172.21.15.113:0/9645907 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f7518010070 con 0x7f7520158de0 2024-03-04T07:33:34.632 INFO:teuthology.misc.health.smithi113.stdout: 2024-03-04T07:33:34.632 INFO:teuthology.misc.health.smithi113.stdout:{"epoch":10,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:33.280839+0000","last_up_change":"0.000000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:33:34.633 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f7527a00640 1 -- 172.21.15.113:0/9645907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520158de0 msgr2=0x7f75201f1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:34.633 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f7527a00640 1 --2- 172.21.15.113:0/9645907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520158de0 0x7f75201f1050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7518007e20 tx=0x7f751800d4a0 comp rx=0 tx=0).stop 2024-03-04T07:33:34.633 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f7527a00640 1 -- 172.21.15.113:0/9645907 shutdown_connections 2024-03-04T07:33:34.633 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f7527a00640 1 --2- 172.21.15.113:0/9645907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7520158de0 0x7f75201f1050 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:34.633 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f7527a00640 1 --2- 172.21.15.113:0/9645907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7520152b50 0x7f75201f0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:34.633 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f7527a00640 1 --2- 172.21.15.113:0/9645907 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7520152770 0x7f75200c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:34.633 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f7527a00640 1 -- 172.21.15.113:0/9645907 >> 172.21.15.113:0/9645907 conn(0x7f752006b470 msgr2=0x7f7520066f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:34.633 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f7527a00640 1 -- 172.21.15.113:0/9645907 shutdown_connections 2024-03-04T07:33:34.633 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:34.632+0000 7f7527a00640 1 -- 172.21.15.113:0/9645907 wait complete. 2024-03-04T07:33:34.648 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-04T07:33:34.715 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:34.713+0000 7fc375c73180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T07:33:34.721 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:34.720+0000 7f02fca55180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T07:33:34.790 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:34.789+0000 7fc375c73180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T07:33:34.972 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:34.969+0000 7fc375c73180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T07:33:35.045 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:35.045+0000 7fc375c73180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T07:33:35.336 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:35.333+0000 7fc375c73180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T07:33:35.396 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:35.393+0000 7fc375c73180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T07:33:35.406 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:35.407+0000 7f02fca55180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T07:33:35.499 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:35.499+0000 7f02fca55180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T07:33:35.700 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:35.699+0000 7f02fca55180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T07:33:35.784 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:35.783+0000 7f02fca55180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T07:33:36.111 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:36.111+0000 7f02fca55180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T07:33:36.181 INFO:tasks.ceph.mgr.y.smithi113.stderr:2024-03-04T07:33:36.179+0000 7f02fca55180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T07:33:36.436 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:36.433+0000 7fc31d0c1640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-04T07:33:36.441 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T07:33:36.441+0000 7fc31d0c1640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-04T07:33:37.732 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-03-04T07:33:37.729+0000 7fbceb4d17c0 -1 osd.4 0 log_to_monitors true 2024-03-04T07:33:37.826 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-03-04T07:33:37.823+0000 7f05b407e7c0 -1 osd.2 0 log_to_monitors true 2024-03-04T07:33:37.834 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-03-04T07:33:37.833+0000 7fe7b56c17c0 -1 osd.6 0 log_to_monitors true 2024-03-04T07:33:37.847 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-03-04T07:33:37.847+0000 7fdf1835c7c0 -1 osd.0 0 log_to_monitors true 2024-03-04T07:33:37.855 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-03-04T07:33:37.853+0000 7f85534ee7c0 -1 osd.5 0 log_to_monitors true 2024-03-04T07:33:37.937 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-03-04T07:33:37.935+0000 7f0824aba7c0 -1 osd.3 0 log_to_monitors true 2024-03-04T07:33:37.939 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-03-04T07:33:37.939+0000 7f6881ca47c0 -1 osd.1 0 log_to_monitors true 2024-03-04T07:33:37.947 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-03-04T07:33:37.945+0000 7fa1703a37c0 -1 osd.7 0 log_to_monitors true 2024-03-04T07:33:40.116 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-03-04T07:33:40.115+0000 7f05ae600640 -1 osd.2 0 waiting for initial osdmap 2024-03-04T07:33:40.125 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-03-04T07:33:40.123+0000 7fdf12800640 -1 osd.0 0 waiting for initial osdmap 2024-03-04T07:33:40.169 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-03-04T07:33:40.167+0000 7fdf09200640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T07:33:40.170 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-03-04T07:33:40.167+0000 7f05a5000640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T07:33:40.251 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-03-04T07:33:40.249+0000 7fe7b0e47640 -1 osd.6 0 waiting for initial osdmap 2024-03-04T07:33:40.253 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-03-04T07:33:40.249+0000 7f85504aa640 -1 osd.5 0 waiting for initial osdmap 2024-03-04T07:33:40.256 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-03-04T07:33:40.253+0000 7fbce848d640 -1 osd.4 0 waiting for initial osdmap 2024-03-04T07:33:40.283 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-03-04T07:33:40.281+0000 7fe7a948e640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T07:33:40.288 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-03-04T07:33:40.285+0000 7f85472bb640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T07:33:40.306 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-03-04T07:33:40.305+0000 7fbcdf29e640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T07:33:40.826 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-03-04T07:33:40.825+0000 7fa16bb29640 -1 osd.7 0 waiting for initial osdmap 2024-03-04T07:33:40.855 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-03-04T07:33:40.853+0000 7fa164170640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T07:33:41.146 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-03-04T07:33:41.143+0000 7f687e000640 -1 osd.1 0 waiting for initial osdmap 2024-03-04T07:33:41.179 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-03-04T07:33:41.179+0000 7f0820e00640 -1 osd.3 0 waiting for initial osdmap 2024-03-04T07:33:41.184 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-03-04T07:33:41.183+0000 7f6872c00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T07:33:41.204 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-03-04T07:33:41.203+0000 7f0815a00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-04T07:33:41.455 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T07:33:41.580 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb39400640 1 Processor -- start 2024-03-04T07:33:41.580 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb39400640 1 -- start start 2024-03-04T07:33:41.581 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb39400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 0x7fbb340c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:41.581 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb39400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb340c6990 con 0x7fbb34152e10 2024-03-04T07:33:41.581 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb39400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb340c6c50 con 0x7fbb340c8570 2024-03-04T07:33:41.581 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb39400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c3fa0 con 0x7fbb340c6120 2024-03-04T07:33:41.582 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb32800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbb34152e10 0x7fbb340c3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36674/0 (socket says 172.21.15.113:36674) 2024-03-04T07:33:41.582 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb32800640 1 -- 172.21.15.113:0/3872595960 learned_addr learned my addr 172.21.15.113:0/3872595960 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:33:41.582 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb2be00640 1 --2- 172.21.15.113:0/3872595960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 0x7fbb340c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:41.583 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb31e00640 1 -- 172.21.15.113:0/3872595960 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657905209 0 0) 0x7fbb340c6990 con 0x7fbb34152e10 2024-03-04T07:33:41.583 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb31e00640 1 -- 172.21.15.113:0/3872595960 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb18003530 con 0x7fbb34152e10 2024-03-04T07:33:41.583 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb31e00640 1 -- 172.21.15.113:0/3872595960 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766166254 0 0) 0x7fbb340c6c50 con 0x7fbb340c8570 2024-03-04T07:33:41.583 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb31e00640 1 -- 172.21.15.113:0/3872595960 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb340c6990 con 0x7fbb340c8570 2024-03-04T07:33:41.583 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb2be00640 1 -- 172.21.15.113:0/3872595960 >> v1:172.21.15.136:6789/0 conn(0x7fbb340c8570 legacy=0x7fbb34158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:41.583 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb2be00640 1 -- 172.21.15.113:0/3872595960 >> v1:172.21.15.113:6789/0 conn(0x7fbb34152e10 legacy=0x7fbb340c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:41.583 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb31e00640 1 -- 172.21.15.113:0/3872595960 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3410675772 0 0) 0x7fbb18003530 con 0x7fbb34152e10 2024-03-04T07:33:41.583 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.579+0000 7fbb2be00640 1 -- 172.21.15.113:0/3872595960 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340c6f30 con 0x7fbb340c6120 2024-03-04T07:33:41.585 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.583+0000 7fbb2be00640 1 --2- 172.21.15.113:0/3872595960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 0x7fbb340c8030 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0097f0 tx=0x7fbb1c01ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec49fbaaa92ec34b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:41.586 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.583+0000 7fbb31e00640 1 -- 172.21.15.113:0/3872595960 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0b5070 con 0x7fbb340c6120 2024-03-04T07:33:41.586 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.583+0000 7fbb31e00640 1 -- 172.21.15.113:0/3872595960 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c0a89d0 con 0x7fbb340c6120 2024-03-04T07:33:41.586 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.583+0000 7fbb31e00640 1 -- 172.21.15.113:0/3872595960 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c0a8d80 con 0x7fbb340c6120 2024-03-04T07:33:41.587 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.583+0000 7fbb39400640 1 -- 172.21.15.113:0/3872595960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 msgr2=0x7fbb340c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:41.587 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.583+0000 7fbb39400640 1 --2- 172.21.15.113:0/3872595960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 0x7fbb340c8030 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0097f0 tx=0x7fbb1c01ecd0 comp rx=0 tx=0).stop 2024-03-04T07:33:41.587 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.583+0000 7fbb39400640 1 -- 172.21.15.113:0/3872595960 shutdown_connections 2024-03-04T07:33:41.587 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.583+0000 7fbb39400640 1 --2- 172.21.15.113:0/3872595960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 0x7fbb340c8030 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:41.587 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.583+0000 7fbb39400640 1 -- 172.21.15.113:0/3872595960 >> 172.21.15.113:0/3872595960 conn(0x7fbb340cc090 msgr2=0x7fbb340cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:41.587 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb39400640 1 -- 172.21.15.113:0/3872595960 shutdown_connections 2024-03-04T07:33:41.587 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb39400640 1 -- 172.21.15.113:0/3872595960 wait complete. 2024-03-04T07:33:41.588 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb39400640 1 Processor -- start 2024-03-04T07:33:41.588 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb39400640 1 -- start start 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb39400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 0x7fbb34072800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb39400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb340c8570 0x7fbb34145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb39400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34152e10 0x7fbb341f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb39400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c47b0 con 0x7fbb34152e10 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb39400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c44d0 con 0x7fbb340c8570 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb39400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34159080 con 0x7fbb340c6120 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb33200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34152e10 0x7fbb341f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb33200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34152e10 0x7fbb341f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33186/0 (socket says 172.21.15.113:33186) 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb33200640 1 -- 172.21.15.113:0/42348682 learned_addr learned my addr 172.21.15.113:0/42348682 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb2be00640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb340c8570 0x7fbb34145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb32800640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 0x7fbb34072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb33200640 1 -- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 msgr2=0x7fbb34072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:41.591 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb33200640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 0x7fbb34072800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:41.592 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb33200640 1 -- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb340c8570 msgr2=0x7fbb34145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:41.592 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb33200640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb340c8570 0x7fbb34145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:41.592 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb33200640 1 -- 172.21.15.113:0/42348682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3414e2e0 con 0x7fbb34152e10 2024-03-04T07:33:41.592 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb33200640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34152e10 0x7fbb341f44e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00c7c0 tx=0x7fbb2c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:41.592 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb2aa00640 1 -- 172.21.15.113:0/42348682 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c003b80 con 0x7fbb34152e10 2024-03-04T07:33:41.592 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb2aa00640 1 -- 172.21.15.113:0/42348682 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c003d20 con 0x7fbb34152e10 2024-03-04T07:33:41.592 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.587+0000 7fbb2aa00640 1 -- 172.21.15.113:0/42348682 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00a9f0 con 0x7fbb34152e10 2024-03-04T07:33:41.592 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.591+0000 7fbb39400640 1 -- 172.21.15.113:0/42348682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3414fa70 con 0x7fbb34152e10 2024-03-04T07:33:41.592 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.591+0000 7fbb39400640 1 -- 172.21.15.113:0/42348682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb34150030 con 0x7fbb34152e10 2024-03-04T07:33:41.592 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.591+0000 7fbb39400640 1 -- 172.21.15.113:0/42348682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb34068c80 con 0x7fbb34152e10 2024-03-04T07:33:41.593 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.591+0000 7fbb2aa00640 1 -- 172.21.15.113:0/42348682 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7fbb2c00ab90 con 0x7fbb34152e10 2024-03-04T07:33:41.594 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.591+0000 7fbb2aa00640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb10066f00 0x7fbb100693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:41.594 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.591+0000 7fbb2aa00640 1 -- 172.21.15.113:0/42348682 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3911+0+0 (secure 0 0 0) 0x7fbb2c0846e0 con 0x7fbb34152e10 2024-03-04T07:33:41.594 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.595+0000 7fbb32800640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb10066f00 0x7fbb100693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:41.595 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.595+0000 7fbb32800640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb10066f00 0x7fbb100693c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb24005890 tx=0x7fbb24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:41.599 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.599+0000 7fbb2aa00640 1 -- 172.21.15.113:0/42348682 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbb2c055ec0 con 0x7fbb34152e10 2024-03-04T07:33:41.753 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.751+0000 7fbb39400640 1 -- 172.21.15.113:0/42348682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb341594c0 con 0x7fbb34152e10 2024-03-04T07:33:41.754 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.751+0000 7fbb2aa00640 1 -- 172.21.15.113:0/42348682 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9973 (secure 0 0 0) 0x7fbb2c0557c0 con 0x7fbb34152e10 2024-03-04T07:33:41.754 INFO:teuthology.misc.health.smithi113.stdout: 2024-03-04T07:33:41.754 INFO:teuthology.misc.health.smithi113.stdout:{"epoch":14,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:41.132300+0000","last_up_change":"2024-03-04T07:33:41.132300+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:33:41.756 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 -- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb10066f00 msgr2=0x7fbb100693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:41.756 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb10066f00 0x7fbb100693c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb24005890 tx=0x7fbb24009040 comp rx=0 tx=0).stop 2024-03-04T07:33:41.756 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 -- 172.21.15.113:0/42348682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34152e10 msgr2=0x7fbb341f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:41.756 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34152e10 0x7fbb341f44e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00c7c0 tx=0x7fbb2c00cc80 comp rx=0 tx=0).stop 2024-03-04T07:33:41.757 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 -- 172.21.15.113:0/42348682 shutdown_connections 2024-03-04T07:33:41.757 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb10066f00 0x7fbb100693c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:41.757 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb34152e10 0x7fbb341f44e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:41.757 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb340c8570 0x7fbb34145b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:41.757 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 --2- 172.21.15.113:0/42348682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb340c6120 0x7fbb34072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:41.757 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 -- 172.21.15.113:0/42348682 >> 172.21.15.113:0/42348682 conn(0x7fbb340cc090 msgr2=0x7fbb3406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:41.757 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 -- 172.21.15.113:0/42348682 shutdown_connections 2024-03-04T07:33:41.757 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:41.755+0000 7fbb39400640 1 -- 172.21.15.113:0/42348682 wait complete. 2024-03-04T07:33:41.773 DEBUG:teuthology.misc:6 of 8 OSDs are up 2024-03-04T07:33:48.579 DEBUG:teuthology.orchestra.run.smithi113:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-04T07:33:48.715 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.711+0000 7fcbde800640 1 Processor -- start 2024-03-04T07:33:48.715 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.711+0000 7fcbde800640 1 -- start start 2024-03-04T07:33:48.716 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbde800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcbd80c6120 0x7fcbd80c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:48.716 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbde800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd80c6990 con 0x7fcbd80c8570 2024-03-04T07:33:48.716 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbde800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd80c6c50 con 0x7fcbd8152e10 2024-03-04T07:33:48.717 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbde800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd80c3fa0 con 0x7fcbd80c6120 2024-03-04T07:33:48.717 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcbd80c8570 0x7fcbd8158970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36766/0 (socket says 172.21.15.113:36766) 2024-03-04T07:33:48.717 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd7e00640 1 -- 172.21.15.113:0/2975090557 learned_addr learned my addr 172.21.15.113:0/2975090557 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:33:48.717 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6a00640 1 --2- 172.21.15.113:0/2975090557 >> v2:172.21.15.136:3300/0 conn(0x7fcbd80c6120 0x7fcbd80c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:48.718 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990300801 0 0) 0x7fcbd80c6990 con 0x7fcbd80c8570 2024-03-04T07:33:48.718 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbbc003530 con 0x7fcbd80c8570 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3090057051 0 0) 0x7fcbbc003530 con 0x7fcbd80c8570 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 >> v2:172.21.15.136:3300/0 conn(0x7fcbd80c6120 msgr2=0x7fcbd80c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 --2- 172.21.15.113:0/2975090557 >> v2:172.21.15.136:3300/0 conn(0x7fcbd80c6120 0x7fcbd80c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 >> v1:172.21.15.136:6789/0 conn(0x7fcbd8152e10 legacy=0x7fcbd80c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd80c6f30 con 0x7fcbd80c8570 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fcbd0001a00 con 0x7fcbd80c8570 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 >> v1:172.21.15.113:6789/0 conn(0x7fcbd80c8570 legacy=0x7fcbd8158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc0039f0 0x7fcbbc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbbc004810 0x7fcbbc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0051d0 0x7fcbbc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80c3fa0 con 0x7fcbbc0039f0 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc00c080 con 0x7fcbbc004810 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc00c280 con 0x7fcbbc0051d0 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd7400640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc0039f0 0x7fcbbc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:48.719 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6a00640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbbc004810 0x7fcbbc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:48.720 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd7e00640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0051d0 0x7fcbbc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:48.720 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd7400640 1 -- 172.21.15.113:0/2975090557 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0051d0 msgr2=0x7fcbbc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:48.720 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd7400640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0051d0 0x7fcbbc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.720 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd7400640 1 -- 172.21.15.113:0/2975090557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbbc004810 msgr2=0x7fcbbc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:48.720 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd7400640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbbc004810 0x7fcbbc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.720 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd7400640 1 -- 172.21.15.113:0/2975090557 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc003530 con 0x7fcbbc0039f0 2024-03-04T07:33:48.720 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd7e00640 1 -- 172.21.15.113:0/2975090557 reap_dead start 2024-03-04T07:33:48.720 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.715+0000 7fcbd6a00640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbbc004810 0x7fcbbc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:48.721 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbd7400640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc0039f0 0x7fcbbc003e10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8001780 tx=0x7fcbc801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a7c2d5b3334f0a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:48.721 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80013a0 con 0x7fcbbc0039f0 2024-03-04T07:33:48.721 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc80052f0 con 0x7fcbbc0039f0 2024-03-04T07:33:48.721 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbd6000640 1 -- 172.21.15.113:0/2975090557 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc8003840 con 0x7fcbbc0039f0 2024-03-04T07:33:48.721 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbde800640 1 -- 172.21.15.113:0/2975090557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc0039f0 msgr2=0x7fcbbc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:48.722 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbde800640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc0039f0 0x7fcbbc003e10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8001780 tx=0x7fcbc801c420 comp rx=0 tx=0).stop 2024-03-04T07:33:48.722 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbde800640 1 -- 172.21.15.113:0/2975090557 shutdown_connections 2024-03-04T07:33:48.722 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbde800640 1 --2- 172.21.15.113:0/2975090557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc0039f0 0x7fcbbc003e10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.722 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbde800640 1 -- 172.21.15.113:0/2975090557 >> 172.21.15.113:0/2975090557 conn(0x7fcbd80cc090 msgr2=0x7fcbd80cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:48.722 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbde800640 1 -- 172.21.15.113:0/2975090557 shutdown_connections 2024-03-04T07:33:48.722 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.719+0000 7fcbde800640 1 -- 172.21.15.113:0/2975090557 wait complete. 2024-03-04T07:33:48.723 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 Processor -- start 2024-03-04T07:33:48.723 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 -- start start 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0039f0 0x7fcbd8155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd8140410 0x7fcbd813e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbd8140870 0x7fcbd8142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80c47b0 con 0x7fcbd8140410 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80c44d0 con 0x7fcbd8140870 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80c8030 con 0x7fcbbc0039f0 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd8140410 0x7fcbd813e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd8140410 0x7fcbd813e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50306/0 (socket says 172.21.15.113:50306) 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd6a00640 1 -- 172.21.15.113:0/522873950 learned_addr learned my addr 172.21.15.113:0/522873950 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd7400640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0039f0 0x7fcbd8155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd7e00640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbd8140870 0x7fcbd8142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd6a00640 1 -- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0039f0 msgr2=0x7fcbd8155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd6a00640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0039f0 0x7fcbd8155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.726 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd6a00640 1 -- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbd8140870 msgr2=0x7fcbd8142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:48.727 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd6a00640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbd8140870 0x7fcbd8142180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.727 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd6a00640 1 -- 172.21.15.113:0/522873950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd8153530 con 0x7fcbd8140410 2024-03-04T07:33:48.727 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbd6a00640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd8140410 0x7fcbd813e990 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4011510 tx=0x7fcbc40119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:48.727 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbcf400640 1 -- 172.21.15.113:0/522873950 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4004220 con 0x7fcbd8140410 2024-03-04T07:33:48.727 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbcf400640 1 -- 172.21.15.113:0/522873950 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc40043c0 con 0x7fcbd8140410 2024-03-04T07:33:48.727 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 -- 172.21.15.113:0/522873950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd8153810 con 0x7fcbd8140410 2024-03-04T07:33:48.727 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbcf400640 1 -- 172.21.15.113:0/522873950 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4015890 con 0x7fcbd8140410 2024-03-04T07:33:48.727 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 -- 172.21.15.113:0/522873950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd813e020 con 0x7fcbd8140410 2024-03-04T07:33:48.727 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.723+0000 7fcbde800640 1 -- 172.21.15.113:0/522873950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd8068c80 con 0x7fcbd8140410 2024-03-04T07:33:48.728 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.727+0000 7fcbcf400640 1 -- 172.21.15.113:0/522873950 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbc4015a30 con 0x7fcbd8140410 2024-03-04T07:33:48.729 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.727+0000 7fcbcf400640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcbb8066f90 0x7fcbb8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:48.729 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.727+0000 7fcbcf400640 1 -- 172.21.15.113:0/522873950 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcbc40852b0 con 0x7fcbd8140410 2024-03-04T07:33:48.729 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.727+0000 7fcbd7400640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcbb8066f90 0x7fcbb8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:48.729 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.727+0000 7fcbd7400640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcbb8066f90 0x7fcbb8069450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8001750 tx=0x7fcbc80011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:48.733 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.731+0000 7fcbcf400640 1 -- 172.21.15.113:0/522873950 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcbc4019030 con 0x7fcbd8140410 2024-03-04T07:33:48.890 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.887+0000 7fcbde800640 1 -- 172.21.15.113:0/522873950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbd80c6990 con 0x7fcbd8140410 2024-03-04T07:33:48.891 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbcf400640 1 -- 172.21.15.113:0/522873950 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12920 (secure 0 0 0) 0x7fcbc4056570 con 0x7fcbd8140410 2024-03-04T07:33:48.891 INFO:teuthology.misc.health.smithi113.stdout: 2024-03-04T07:33:48.891 INFO:teuthology.misc.health.smithi113.stdout:{"epoch":17,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:44.191289+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 -- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcbb8066f90 msgr2=0x7fcbb8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcbb8066f90 0x7fcbb8069450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8001750 tx=0x7fcbc80011a0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 -- 172.21.15.113:0/522873950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd8140410 msgr2=0x7fcbd813e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd8140410 0x7fcbd813e990 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4011510 tx=0x7fcbc40119d0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 -- 172.21.15.113:0/522873950 shutdown_connections 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbd8140870 0x7fcbd8142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcbb8066f90 0x7fcbb8069450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbd8140410 0x7fcbd813e990 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 --2- 172.21.15.113:0/522873950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0039f0 0x7fcbd8155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 -- 172.21.15.113:0/522873950 >> 172.21.15.113:0/522873950 conn(0x7fcbd80cc090 msgr2=0x7fcbd80c3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 -- 172.21.15.113:0/522873950 shutdown_connections 2024-03-04T07:33:48.894 INFO:teuthology.misc.health.smithi113.stderr:2024-03-04T07:33:48.891+0000 7fcbde800640 1 -- 172.21.15.113:0/522873950 wait complete. 2024-03-04T07:33:48.911 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-04T07:33:48.911 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-04T07:33:48.911 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-04T07:33:48.911 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-04T07:33:48.914 INFO:tasks.ceph.mds.a:Started 2024-03-04T07:33:48.914 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-04T07:33:48.914 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-04T07:33:48.917 INFO:tasks.ceph.mds.c:Started 2024-03-04T07:33:48.917 INFO:tasks.ceph.mds.e:Restarting daemon 2024-03-04T07:33:48.918 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-03-04T07:33:48.920 INFO:tasks.ceph.mds.e:Started 2024-03-04T07:33:48.920 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-04T07:33:48.920 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-04T07:33:48.923 INFO:tasks.ceph.mds.b:Started 2024-03-04T07:33:48.923 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-04T07:33:48.924 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-04T07:33:48.926 INFO:tasks.ceph.mds.d:Started 2024-03-04T07:33:48.926 INFO:tasks.ceph.mds.f:Restarting daemon 2024-03-04T07:33:48.926 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-03-04T07:33:48.929 INFO:tasks.ceph.mds.f:Started 2024-03-04T07:33:48.929 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-04T07:33:48.929 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:33:48.996 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2024-03-04T07:33:48.998 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2024-03-04T07:33:49.003 INFO:tasks.ceph.mds.f.smithi136.stdout:starting mds.f at 2024-03-04T07:33:49.018 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-03-04T07:33:49.032 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-03-04T07:33:49.034 INFO:tasks.ceph.mds.e.smithi113.stdout:starting mds.e at 2024-03-04T07:33:49.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.075+0000 7f6920400640 1 Processor -- start 2024-03-04T07:33:49.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.075+0000 7f6920400640 1 -- start start 2024-03-04T07:33:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f6920400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f69181527f0 0x7f6918070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f6920400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6918152bd0 0x7f69181587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f6920400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69181591f0 con 0x7f69180c8200 2024-03-04T07:33:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f6920400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6918071df0 con 0x7f6918152bd0 2024-03-04T07:33:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f6920400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69181597b0 con 0x7f69181527f0 2024-03-04T07:33:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f691e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6918152bd0 0x7f69181587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f691e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6918152bd0 0x7f69181587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50380/0 (socket says 172.21.15.113:50380) 2024-03-04T07:33:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f691e400640 1 -- 172.21.15.113:0/1858763369 learned_addr learned my addr 172.21.15.113:0/1858763369 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f691da00640 1 --2- 172.21.15.113:0/1858763369 >> v2:172.21.15.136:3300/0 conn(0x7f69181527f0 0x7f6918070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f691e400640 1 -- 172.21.15.113:0/1858763369 >> v2:172.21.15.136:3300/0 conn(0x7f69181527f0 msgr2=0x7f6918070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:49.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f691e400640 1 --2- 172.21.15.113:0/1858763369 >> v2:172.21.15.136:3300/0 conn(0x7f69181527f0 0x7f6918070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f691e400640 1 -- 172.21.15.113:0/1858763369 >> v1:172.21.15.136:6789/0 conn(0x7f69180c8200 legacy=0x7f6918072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.079+0000 7f691e400640 1 -- 172.21.15.113:0/1858763369 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69181594d0 con 0x7f6918152bd0 2024-03-04T07:33:49.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f691e400640 1 --2- 172.21.15.113:0/1858763369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6918152bd0 0x7f69181587d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f691806f400 tx=0x7f691001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a44d7661226df7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:49.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6917e00640 1 -- 172.21.15.113:0/1858763369 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69100a57a0 con 0x7f6918152bd0 2024-03-04T07:33:49.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6917e00640 1 -- 172.21.15.113:0/1858763369 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69100a5940 con 0x7f6918152bd0 2024-03-04T07:33:49.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6917e00640 1 -- 172.21.15.113:0/1858763369 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69100a5d10 con 0x7f6918152bd0 2024-03-04T07:33:49.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 -- 172.21.15.113:0/1858763369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6918152bd0 msgr2=0x7f69181587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 --2- 172.21.15.113:0/1858763369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6918152bd0 0x7f69181587d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f691806f400 tx=0x7f691001c9f0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 -- 172.21.15.113:0/1858763369 shutdown_connections 2024-03-04T07:33:49.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 --2- 172.21.15.113:0/1858763369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6918152bd0 0x7f69181587d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 --2- 172.21.15.113:0/1858763369 >> v2:172.21.15.136:3300/0 conn(0x7f69181527f0 0x7f6918070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 -- 172.21.15.113:0/1858763369 >> 172.21.15.113:0/1858763369 conn(0x7f691806e820 msgr2=0x7f691806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:49.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 -- 172.21.15.113:0/1858763369 shutdown_connections 2024-03-04T07:33:49.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 -- 172.21.15.113:0/1858763369 wait complete. 2024-03-04T07:33:49.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 Processor -- start 2024-03-04T07:33:49.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 -- start start 2024-03-04T07:33:49.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69180c8200 0x7f69181533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181527f0 0x7f691814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6918152bd0 0x7f69181f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69181654f0 con 0x7f69181527f0 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6918165370 con 0x7f6918152bd0 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f6920400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6918165670 con 0x7f69180c8200 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f691d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181527f0 0x7f691814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f691d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181527f0 0x7f691814a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50390/0 (socket says 172.21.15.113:50390) 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f691d000640 1 -- 172.21.15.113:0/3209846491 learned_addr learned my addr 172.21.15.113:0/3209846491 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f691e400640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6918152bd0 0x7f69181f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.083+0000 7f691da00640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69180c8200 0x7f69181533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f691d000640 1 -- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69180c8200 msgr2=0x7f69181533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f691d000640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69180c8200 0x7f69181533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f691d000640 1 -- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6918152bd0 msgr2=0x7f69181f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f691d000640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6918152bd0 0x7f69181f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f691d000640 1 -- 172.21.15.113:0/3209846491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69181f5a90 con 0x7f69181527f0 2024-03-04T07:33:49.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f691e400640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6918152bd0 0x7f69181f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:49.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f691d000640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181527f0 0x7f691814a540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f690400c430 tx=0x7f690400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:49.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f6915600640 1 -- 172.21.15.113:0/3209846491 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69040052d0 con 0x7f69181527f0 2024-03-04T07:33:49.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f6915600640 1 -- 172.21.15.113:0/3209846491 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6904005470 con 0x7f69181527f0 2024-03-04T07:33:49.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f6920400640 1 -- 172.21.15.113:0/3209846491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69181f5d50 con 0x7f69181527f0 2024-03-04T07:33:49.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f6915600640 1 -- 172.21.15.113:0/3209846491 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904007260 con 0x7f69181527f0 2024-03-04T07:33:49.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f6920400640 1 -- 172.21.15.113:0/3209846491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69181f0680 con 0x7f69181527f0 2024-03-04T07:33:49.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f6920400640 1 -- 172.21.15.113:0/3209846491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69180c79d0 con 0x7f69181527f0 2024-03-04T07:33:49.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f6915600640 1 -- 172.21.15.113:0/3209846491 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6904007400 con 0x7f69181527f0 2024-03-04T07:33:49.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f6915600640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f68f8066f90 0x7f68f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f6915600640 1 -- 172.21.15.113:0/3209846491 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f69040841d0 con 0x7f69181527f0 2024-03-04T07:33:49.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.087+0000 7f691da00640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f68f8066f90 0x7f68f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.091+0000 7f691da00640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f68f8066f90 0x7f68f8069450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f690c009af0 tx=0x7f690c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:49.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.091+0000 7f6915600640 1 -- 172.21.15.113:0/3209846491 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6904055520 con 0x7f69181527f0 2024-03-04T07:33:49.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.247+0000 7f6920400640 1 -- 172.21.15.113:0/3209846491 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6918075be0 con 0x7f69181527f0 2024-03-04T07:33:49.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6915600640 1 -- 172.21.15.113:0/3209846491 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12920 (secure 0 0 0) 0x7f6904054e20 con 0x7f69181527f0 2024-03-04T07:33:49.251 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:33:49.252 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":17,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:44.191289+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 -- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f68f8066f90 msgr2=0x7f68f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f68f8066f90 0x7f68f8069450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f690c009af0 tx=0x7f690c008040 comp rx=0 tx=0).stop 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 -- 172.21.15.113:0/3209846491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181527f0 msgr2=0x7f691814a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181527f0 0x7f691814a540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f690400c430 tx=0x7f690400c8f0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 -- 172.21.15.113:0/3209846491 shutdown_connections 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6918152bd0 0x7f69181f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f68f8066f90 0x7f68f8069450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69181527f0 0x7f691814a540 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 --2- 172.21.15.113:0/3209846491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69180c8200 0x7f69181533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 -- 172.21.15.113:0/3209846491 >> 172.21.15.113:0/3209846491 conn(0x7f691806e820 msgr2=0x7f69181581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 -- 172.21.15.113:0/3209846491 shutdown_connections 2024-03-04T07:33:49.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.251+0000 7f6920400640 1 -- 172.21.15.113:0/3209846491 wait complete. 2024-03-04T07:33:49.273 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T07:33:41.507700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T07:33:49.273 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T07:33:49.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.407+0000 7fb9c8000640 1 Processor -- start 2024-03-04T07:33:49.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c8000640 1 -- start start 2024-03-04T07:33:49.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c8000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9c00c19e0 0x7fb9c00c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c8000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c01577d0 con 0x7fb9c01519b0 2024-03-04T07:33:49.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c8000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c00c7760 con 0x7fb9c0157cd0 2024-03-04T07:33:49.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c8000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00c7a40 con 0x7fb9c00c19e0 2024-03-04T07:33:49.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c5600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb9c01519b0 0x7fb9c00cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36788/0 (socket says 172.21.15.113:36788) 2024-03-04T07:33:49.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c5600640 1 -- 172.21.15.113:0/1675902019 learned_addr learned my addr 172.21.15.113:0/1675902019 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:33:49.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c4c00640 1 --2- 172.21.15.113:0/1675902019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9c00c19e0 0x7fb9c00c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9bfe00640 1 -- 172.21.15.113:0/1675902019 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258442784 0 0) 0x7fb9c01577d0 con 0x7fb9c01519b0 2024-03-04T07:33:49.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9bfe00640 1 -- 172.21.15.113:0/1675902019 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9a8003530 con 0x7fb9c01519b0 2024-03-04T07:33:49.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c4c00640 1 -- 172.21.15.113:0/1675902019 >> v1:172.21.15.136:6789/0 conn(0x7fb9c0157cd0 legacy=0x7fb9c0158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9bfe00640 1 -- 172.21.15.113:0/1675902019 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1612087781 0 0) 0x7fb9a8003530 con 0x7fb9c01519b0 2024-03-04T07:33:49.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c4c00640 1 -- 172.21.15.113:0/1675902019 >> v1:172.21.15.113:6789/0 conn(0x7fb9c01519b0 legacy=0x7fb9c00cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.411+0000 7fb9c4c00640 1 -- 172.21.15.113:0/1675902019 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c00c1090 con 0x7fb9c00c19e0 2024-03-04T07:33:49.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9c4c00640 1 --2- 172.21.15.113:0/1675902019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9c00c19e0 0x7fb9c00c8370 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40097f0 tx=0x7fb9b401c2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676a1d7eb1c5f463 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:49.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9bfe00640 1 -- 172.21.15.113:0/1675902019 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40b0070 con 0x7fb9c00c19e0 2024-03-04T07:33:49.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9bfe00640 1 -- 172.21.15.113:0/1675902019 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b40ab400 con 0x7fb9c00c19e0 2024-03-04T07:33:49.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9bfe00640 1 -- 172.21.15.113:0/1675902019 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0001680 con 0x7fb9c00c19e0 2024-03-04T07:33:49.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9c8000640 1 -- 172.21.15.113:0/1675902019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9c00c19e0 msgr2=0x7fb9c00c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9c8000640 1 --2- 172.21.15.113:0/1675902019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9c00c19e0 0x7fb9c00c8370 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40097f0 tx=0x7fb9b401c2b0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9c8000640 1 -- 172.21.15.113:0/1675902019 shutdown_connections 2024-03-04T07:33:49.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9c8000640 1 --2- 172.21.15.113:0/1675902019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9c00c19e0 0x7fb9c00c8370 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9c8000640 1 -- 172.21.15.113:0/1675902019 >> 172.21.15.113:0/1675902019 conn(0x7fb9c0066db0 msgr2=0x7fb9c00671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:49.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9c8000640 1 -- 172.21.15.113:0/1675902019 shutdown_connections 2024-03-04T07:33:49.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9c8000640 1 -- 172.21.15.113:0/1675902019 wait complete. 2024-03-04T07:33:49.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9c8000640 1 Processor -- start 2024-03-04T07:33:49.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.415+0000 7fb9c8000640 1 -- start start 2024-03-04T07:33:49.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c8000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9c00c19e0 0x7fb9c0156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c8000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c01519b0 0x7fb9c014fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c8000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9c0157cd0 0x7fb9c0144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c8000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00ccc80 con 0x7fb9c01519b0 2024-03-04T07:33:49.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c8000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00ccb00 con 0x7fb9c0157cd0 2024-03-04T07:33:49.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c8000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0151d90 con 0x7fb9c00c19e0 2024-03-04T07:33:49.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c01519b0 0x7fb9c014fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c01519b0 0x7fb9c014fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50404/0 (socket says 172.21.15.113:50404) 2024-03-04T07:33:49.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c4c00640 1 -- 172.21.15.113:0/3961244423 learned_addr learned my addr 172.21.15.113:0/3961244423 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:49.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c4c00640 1 -- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9c00c19e0 msgr2=0x7fb9c0156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c4c00640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9c00c19e0 0x7fb9c0156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c4c00640 1 -- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9c0157cd0 msgr2=0x7fb9c0144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c4c00640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9c0157cd0 0x7fb9c0144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c4c00640 1 -- 172.21.15.113:0/3961244423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c01f6cd0 con 0x7fb9c01519b0 2024-03-04T07:33:49.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c4c00640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c01519b0 0x7fb9c014fde0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40ae040 tx=0x7fb9b401aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:49.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9bd600640 1 -- 172.21.15.113:0/3961244423 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40b0050 con 0x7fb9c01519b0 2024-03-04T07:33:49.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9bd600640 1 -- 172.21.15.113:0/3961244423 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b40b1040 con 0x7fb9c01519b0 2024-03-04T07:33:49.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9bd600640 1 -- 172.21.15.113:0/3961244423 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4004280 con 0x7fb9c01519b0 2024-03-04T07:33:49.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c8000640 1 -- 172.21.15.113:0/3961244423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c01f6f90 con 0x7fb9c01519b0 2024-03-04T07:33:49.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c8000640 1 -- 172.21.15.113:0/3961244423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c01f7550 con 0x7fb9c01519b0 2024-03-04T07:33:49.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9c8000640 1 -- 172.21.15.113:0/3961244423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c0075c00 con 0x7fb9c01519b0 2024-03-04T07:33:49.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.419+0000 7fb9bd600640 1 -- 172.21.15.113:0/3961244423 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9b40be020 con 0x7fb9c01519b0 2024-03-04T07:33:49.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.423+0000 7fb9bd600640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9a4066f60 0x7fb9a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.423+0000 7fb9bd600640 1 -- 172.21.15.113:0/3961244423 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb9b411ebe0 con 0x7fb9c01519b0 2024-03-04T07:33:49.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.423+0000 7fb9c5600640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9a4066f60 0x7fb9a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.423+0000 7fb9c5600640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9a4066f60 0x7fb9a4069420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0000c10 tx=0x7fb9b0001a10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:49.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.427+0000 7fb9bd600640 1 -- 172.21.15.113:0/3961244423 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb9b4015ff0 con 0x7fb9c01519b0 2024-03-04T07:33:49.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.579+0000 7fb9c8000640 1 -- 172.21.15.113:0/3961244423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9c0073280 con 0x7fb9c01519b0 2024-03-04T07:33:49.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9bd600640 1 -- 172.21.15.113:0/3961244423 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9b40efe10 con 0x7fb9c01519b0 2024-03-04T07:33:49.583 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-03-04T07:33:49.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 -- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9a4066f60 msgr2=0x7fb9a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9a4066f60 0x7fb9a4069420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0000c10 tx=0x7fb9b0001a10 comp rx=0 tx=0).stop 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 -- 172.21.15.113:0/3961244423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c01519b0 msgr2=0x7fb9c014fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c01519b0 0x7fb9c014fde0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40ae040 tx=0x7fb9b401aaf0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 -- 172.21.15.113:0/3961244423 shutdown_connections 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9c0157cd0 0x7fb9c0144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9a4066f60 0x7fb9a4069420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9c01519b0 0x7fb9c014fde0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 --2- 172.21.15.113:0/3961244423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9c00c19e0 0x7fb9c0156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 -- 172.21.15.113:0/3961244423 >> 172.21.15.113:0/3961244423 conn(0x7fb9c0066db0 msgr2=0x7fb9c014e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 -- 172.21.15.113:0/3961244423 shutdown_connections 2024-03-04T07:33:49.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.583+0000 7fb9c8000640 1 -- 172.21.15.113:0/3961244423 wait complete. 2024-03-04T07:33:49.603 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:33:49.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff95b200640 1 Processor -- start 2024-03-04T07:33:49.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff95b200640 1 -- start start 2024-03-04T07:33:49.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff95b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff95414e4c0 0x7ff954072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff95b200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff9540c22e0 0x7ff9540731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff95b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9540c27f0 con 0x7ff9540c2d80 2024-03-04T07:33:49.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff95b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95415ad60 con 0x7ff95414e4c0 2024-03-04T07:33:49.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff95b200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9540c2a90 con 0x7ff9540c22e0 2024-03-04T07:33:49.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff953e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff95414e4c0 0x7ff954072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff959200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff9540c2d80 0x7ff95415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36804/0 (socket says 172.21.15.113:36804) 2024-03-04T07:33:49.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff959200640 1 -- 172.21.15.113:0/1898646534 learned_addr learned my addr 172.21.15.113:0/1898646534 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:33:49.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff94b400640 1 --2- 172.21.15.113:0/1898646534 >> v2:172.21.15.136:3300/0 conn(0x7ff9540c22e0 0x7ff9540731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff953e00640 1 -- 172.21.15.113:0/1898646534 >> v2:172.21.15.136:3300/0 conn(0x7ff9540c22e0 msgr2=0x7ff9540731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff953e00640 1 --2- 172.21.15.113:0/1898646534 >> v2:172.21.15.136:3300/0 conn(0x7ff9540c22e0 0x7ff9540731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff953400640 1 -- 172.21.15.113:0/1898646534 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486695232 0 0) 0x7ff9540c27f0 con 0x7ff9540c2d80 2024-03-04T07:33:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff953e00640 1 -- 172.21.15.113:0/1898646534 >> v1:172.21.15.113:6789/0 conn(0x7ff9540c2d80 legacy=0x7ff95415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff953e00640 1 -- 172.21.15.113:0/1898646534 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff954070c60 con 0x7ff95414e4c0 2024-03-04T07:33:49.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.735+0000 7ff94b400640 1 --2- 172.21.15.113:0/1898646534 >> v2:172.21.15.136:3300/0 conn(0x7ff9540c22e0 0x7ff9540731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff953e00640 1 --2- 172.21.15.113:0/1898646534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff95414e4c0 0x7ff954072c80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff9540cd230 tx=0x7ff94401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d988246b8f74ec server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff953400640 1 -- 172.21.15.113:0/1898646534 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9440b5070 con 0x7ff95414e4c0 2024-03-04T07:33:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff953400640 1 -- 172.21.15.113:0/1898646534 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9440a8970 con 0x7ff95414e4c0 2024-03-04T07:33:49.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff953400640 1 -- 172.21.15.113:0/1898646534 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9440a8d40 con 0x7ff95414e4c0 2024-03-04T07:33:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff95b200640 1 -- 172.21.15.113:0/1898646534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff95414e4c0 msgr2=0x7ff954072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff95b200640 1 --2- 172.21.15.113:0/1898646534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff95414e4c0 0x7ff954072c80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff9540cd230 tx=0x7ff94401ec70 comp rx=0 tx=0).stop 2024-03-04T07:33:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff95b200640 1 -- 172.21.15.113:0/1898646534 shutdown_connections 2024-03-04T07:33:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff95b200640 1 --2- 172.21.15.113:0/1898646534 >> v2:172.21.15.136:3300/0 conn(0x7ff9540c22e0 0x7ff9540731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff95b200640 1 --2- 172.21.15.113:0/1898646534 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff95414e4c0 0x7ff954072c80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff95b200640 1 -- 172.21.15.113:0/1898646534 >> 172.21.15.113:0/1898646534 conn(0x7ff95406e820 msgr2=0x7ff95406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff95b200640 1 -- 172.21.15.113:0/1898646534 shutdown_connections 2024-03-04T07:33:49.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.739+0000 7ff95b200640 1 -- 172.21.15.113:0/1898646534 wait complete. 2024-03-04T07:33:49.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 Processor -- start 2024-03-04T07:33:49.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 -- start start 2024-03-04T07:33:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9540c22e0 0x7ff9541e7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9540c2d80 0x7ff95414a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff95414e4c0 0x7ff9540c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff954071500 con 0x7ff9540c2d80 2024-03-04T07:33:49.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff954071220 con 0x7ff95414e4c0 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95414e8a0 con 0x7ff9540c22e0 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff94b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9540c2d80 0x7ff95414a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff94b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9540c2d80 0x7ff95414a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50420/0 (socket says 172.21.15.113:50420) 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff94b400640 1 -- 172.21.15.113:0/3701130690 learned_addr learned my addr 172.21.15.113:0/3701130690 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff959200640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff95414e4c0 0x7ff9540c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff953e00640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9540c22e0 0x7ff9541e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff94b400640 1 -- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9540c22e0 msgr2=0x7ff9541e7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff94b400640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9540c22e0 0x7ff9541e7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff94b400640 1 -- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff95414e4c0 msgr2=0x7ff9540c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff94b400640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff95414e4c0 0x7ff9540c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff94b400640 1 -- 172.21.15.113:0/3701130690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9540af780 con 0x7ff9540c2d80 2024-03-04T07:33:49.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff94b400640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9540c2d80 0x7ff95414a660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00d800 tx=0x7ff93c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:49.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff950c00640 1 -- 172.21.15.113:0/3701130690 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c00b6c0 con 0x7ff9540c2d80 2024-03-04T07:33:49.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff950c00640 1 -- 172.21.15.113:0/3701130690 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c019070 con 0x7ff9540c2d80 2024-03-04T07:33:49.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 -- 172.21.15.113:0/3701130690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9540b0eb0 con 0x7ff9540c2d80 2024-03-04T07:33:49.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff950c00640 1 -- 172.21.15.113:0/3701130690 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c00baa0 con 0x7ff9540c2d80 2024-03-04T07:33:49.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 -- 172.21.15.113:0/3701130690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9540b1470 con 0x7ff9540c2d80 2024-03-04T07:33:49.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.743+0000 7ff95b200640 1 -- 172.21.15.113:0/3701130690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff954075c00 con 0x7ff9540c2d80 2024-03-04T07:33:49.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.747+0000 7ff950c00640 1 -- 172.21.15.113:0/3701130690 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff93c00bc40 con 0x7ff9540c2d80 2024-03-04T07:33:49.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.747+0000 7ff950c00640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff930066f90 0x7ff930069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:49.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.747+0000 7ff950c00640 1 -- 172.21.15.113:0/3701130690 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff93c0850f0 con 0x7ff9540c2d80 2024-03-04T07:33:49.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.747+0000 7ff953e00640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff930066f90 0x7ff930069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:49.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.747+0000 7ff953e00640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff930066f90 0x7ff930069450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9440b2000 tx=0x7ff944008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:49.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.751+0000 7ff950c00640 1 -- 172.21.15.113:0/3701130690 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff93c0563b0 con 0x7ff9540c2d80 2024-03-04T07:33:49.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.939+0000 7ff95b200640 1 -- 172.21.15.113:0/3701130690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9540c1260 con 0x7ff9540c2d80 2024-03-04T07:33:49.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.939+0000 7ff950c00640 1 -- 172.21.15.113:0/3701130690 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7ff93c055cb0 con 0x7ff9540c2d80 2024-03-04T07:33:49.942 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:33:49.942 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-04T07:33:49.942 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 2 2024-03-04T07:33:49.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 -- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff930066f90 msgr2=0x7ff930069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff930066f90 0x7ff930069450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9440b2000 tx=0x7ff944008040 comp rx=0 tx=0).stop 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 -- 172.21.15.113:0/3701130690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9540c2d80 msgr2=0x7ff95414a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9540c2d80 0x7ff95414a660 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00d800 tx=0x7ff93c00dcc0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 -- 172.21.15.113:0/3701130690 shutdown_connections 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff95414e4c0 0x7ff9540c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff930066f90 0x7ff930069450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9540c2d80 0x7ff95414a660 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 --2- 172.21.15.113:0/3701130690 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9540c22e0 0x7ff9541e7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 -- 172.21.15.113:0/3701130690 >> 172.21.15.113:0/3701130690 conn(0x7ff95406e820 msgr2=0x7ff9540c1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 -- 172.21.15.113:0/3701130690 shutdown_connections 2024-03-04T07:33:49.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:49.943+0000 7ff95b200640 1 -- 172.21.15.113:0/3701130690 wait complete. 2024-03-04T07:33:51.962 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:33:52.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd537200640 1 Processor -- start 2024-03-04T07:33:52.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd537200640 1 -- start start 2024-03-04T07:33:52.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd537200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd538157ef0 0x7fd5380c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd537200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5380c9c50 con 0x7fd53806fb70 2024-03-04T07:33:52.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd537200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53815a9a0 con 0x7fd538070df0 2024-03-04T07:33:52.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd537200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53806de30 con 0x7fd538157ef0 2024-03-04T07:33:52.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd536800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd53806fb70 0x7fd5380cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36818/0 (socket says 172.21.15.113:36818) 2024-03-04T07:33:52.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd536800640 1 -- 172.21.15.113:0/3589896539 learned_addr learned my addr 172.21.15.113:0/3589896539 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd535e00640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd538157ef0 0x7fd5380c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661351745 0 0) 0x7fd5380c9c50 con 0x7fd53806fb70 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd51c003530 con 0x7fd53806fb70 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2036162440 0 0) 0x7fd51c003530 con 0x7fd53806fb70 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd538157ef0 msgr2=0x7fd5380c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd538157ef0 0x7fd5380c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 >> v1:172.21.15.136:6789/0 conn(0x7fd538070df0 legacy=0x7fd53815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53806c210 con 0x7fd53806fb70 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fd530001a00 con 0x7fd53806fb70 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 >> v1:172.21.15.113:6789/0 conn(0x7fd53806fb70 legacy=0x7fd5380cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 0x7fd51c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd538157ef0 0x7fd51c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd51c004d50 0x7fd51c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53806de30 con 0x7fd51c0039f0 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c00ba80 con 0x7fd51c004d50 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c00bc80 con 0x7fd538157ef0 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd535e00640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 0x7fd51c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd535400640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd538157ef0 0x7fd51c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd536800640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd51c004d50 0x7fd51c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd535e00640 1 -- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd538157ef0 msgr2=0x7fd51c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd535e00640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd538157ef0 0x7fd51c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd535e00640 1 -- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd51c004d50 msgr2=0x7fd51c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:52.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd535e00640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd51c004d50 0x7fd51c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.106+0000 7fd535e00640 1 -- 172.21.15.113:0/3589896539 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c003530 con 0x7fd51c0039f0 2024-03-04T07:33:52.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd535e00640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 0x7fd51c003e10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd528009ab0 tx=0x7fd52801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5769476224c556 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:52.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5280b2070 con 0x7fd51c0039f0 2024-03-04T07:33:52.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5280a5530 con 0x7fd51c0039f0 2024-03-04T07:33:52.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd534a00640 1 -- 172.21.15.113:0/3589896539 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5280096b0 con 0x7fd51c0039f0 2024-03-04T07:33:52.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd537200640 1 -- 172.21.15.113:0/3589896539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 msgr2=0x7fd51c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd537200640 1 --2- 172.21.15.113:0/3589896539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 0x7fd51c003e10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd528009ab0 tx=0x7fd52801ca50 comp rx=0 tx=0).stop 2024-03-04T07:33:52.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd536800640 1 -- 172.21.15.113:0/3589896539 reap_dead start 2024-03-04T07:33:52.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd537200640 1 -- 172.21.15.113:0/3589896539 shutdown_connections 2024-03-04T07:33:52.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd537200640 1 -- 172.21.15.113:0/3589896539 >> 172.21.15.113:0/3589896539 conn(0x7fd538072f40 msgr2=0x7fd5380706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:52.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd537200640 1 -- 172.21.15.113:0/3589896539 shutdown_connections 2024-03-04T07:33:52.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.110+0000 7fd537200640 1 -- 172.21.15.113:0/3589896539 wait complete. 2024-03-04T07:33:52.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd537200640 1 Processor -- start 2024-03-04T07:33:52.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd537200640 1 -- start start 2024-03-04T07:33:52.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd537200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 0x7fd5381f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd537200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53815d5a0 0x7fd5380ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd537200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5381f6e60 0x7fd5381f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd537200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5381582d0 con 0x7fd51c0039f0 2024-03-04T07:33:52.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd537200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53815b890 con 0x7fd53815d5a0 2024-03-04T07:33:52.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd537200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd538158450 con 0x7fd5381f6e60 2024-03-04T07:33:52.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 0x7fd5381f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 0x7fd5381f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50436/0 (socket says 172.21.15.113:50436) 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535e00640 1 -- 172.21.15.113:0/475233374 learned_addr learned my addr 172.21.15.113:0/475233374 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd536800640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5381f6e60 0x7fd5381f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535400640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53815d5a0 0x7fd5380ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535e00640 1 -- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5381f6e60 msgr2=0x7fd5381f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535e00640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5381f6e60 0x7fd5381f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535e00640 1 -- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53815d5a0 msgr2=0x7fd5380ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535e00640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53815d5a0 0x7fd5380ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535e00640 1 -- 172.21.15.113:0/475233374 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53815e910 con 0x7fd51c0039f0 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd536800640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5381f6e60 0x7fd5381f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535400640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53815d5a0 0x7fd5380ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd535e00640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 0x7fd5381f4370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd528009ab0 tx=0x7fd51c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd52e000640 1 -- 172.21.15.113:0/475233374 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5280b2040 con 0x7fd51c0039f0 2024-03-04T07:33:52.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd52e000640 1 -- 172.21.15.113:0/475233374 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52800f000 con 0x7fd51c0039f0 2024-03-04T07:33:52.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd537200640 1 -- 172.21.15.113:0/475233374 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53815eb90 con 0x7fd51c0039f0 2024-03-04T07:33:52.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd52e000640 1 -- 172.21.15.113:0/475233374 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528004020 con 0x7fd51c0039f0 2024-03-04T07:33:52.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.114+0000 7fd537200640 1 -- 172.21.15.113:0/475233374 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53815cda0 con 0x7fd51c0039f0 2024-03-04T07:33:52.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.118+0000 7fd537200640 1 -- 172.21.15.113:0/475233374 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd538075a80 con 0x7fd51c0039f0 2024-03-04T07:33:52.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.118+0000 7fd52e000640 1 -- 172.21.15.113:0/475233374 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5280c0030 con 0x7fd51c0039f0 2024-03-04T07:33:52.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.118+0000 7fd52e000640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd518066f90 0x7fd518069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.118+0000 7fd52e000640 1 -- 172.21.15.113:0/475233374 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd52811e750 con 0x7fd51c0039f0 2024-03-04T07:33:52.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.118+0000 7fd535400640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd518066f90 0x7fd518069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.118+0000 7fd535400640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd518066f90 0x7fd518069450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd538166200 tx=0x7fd524009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:52.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.122+0000 7fd52e000640 1 -- 172.21.15.113:0/475233374 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd5280c7060 con 0x7fd51c0039f0 2024-03-04T07:33:52.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 -- 172.21.15.113:0/475233374 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd53806f950 con 0x7fd51c0039f0 2024-03-04T07:33:52.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd52e000640 1 -- 172.21.15.113:0/475233374 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5292 (secure 0 0 0) 0x7fd53815e910 con 0x7fd51c0039f0 2024-03-04T07:33:52.313 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:33:52.313 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-04T07:33:52.313 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 3 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 -- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd518066f90 msgr2=0x7fd518069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd518066f90 0x7fd518069450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd538166200 tx=0x7fd524009040 comp rx=0 tx=0).stop 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 -- 172.21.15.113:0/475233374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 msgr2=0x7fd5381f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 0x7fd5381f4370 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd528009ab0 tx=0x7fd51c005840 comp rx=0 tx=0).stop 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 -- 172.21.15.113:0/475233374 shutdown_connections 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5381f6e60 0x7fd5381f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd53815d5a0 0x7fd5380ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd518066f90 0x7fd518069450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 --2- 172.21.15.113:0/475233374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd51c0039f0 0x7fd5381f4370 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 -- 172.21.15.113:0/475233374 >> 172.21.15.113:0/475233374 conn(0x7fd538072f40 msgr2=0x7fd5381557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 -- 172.21.15.113:0/475233374 shutdown_connections 2024-03-04T07:33:52.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.310+0000 7fd537200640 1 -- 172.21.15.113:0/475233374 wait complete. 2024-03-04T07:33:52.335 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:33:52.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.466+0000 7fb3b6600640 1 Processor -- start 2024-03-04T07:33:52.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b6600640 1 -- start start 2024-03-04T07:33:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b6600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3b0154960 0x7fb3b0066f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b6600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3b00c68e0 0x7fb3b015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b6600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b006ea50 con 0x7fb3b0154d40 2024-03-04T07:33:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b6600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b0067480 con 0x7fb3b0154960 2024-03-04T07:33:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b6600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00c8c10 con 0x7fb3b00c68e0 2024-03-04T07:33:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b5200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3b0154960 0x7fb3b0066f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b5c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb3b0154d40 0x7fb3b006f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36826/0 (socket says 172.21.15.113:36826) 2024-03-04T07:33:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b5c00640 1 -- 172.21.15.113:0/4055230371 learned_addr learned my addr 172.21.15.113:0/4055230371 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:33:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3afe00640 1 --2- 172.21.15.113:0/4055230371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3b00c68e0 0x7fb3b015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b5200640 1 -- 172.21.15.113:0/4055230371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3b00c68e0 msgr2=0x7fb3b015dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:52.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b5200640 1 --2- 172.21.15.113:0/4055230371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3b00c68e0 0x7fb3b015dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b5200640 1 -- 172.21.15.113:0/4055230371 >> v1:172.21.15.113:6789/0 conn(0x7fb3b0154d40 legacy=0x7fb3b006f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3af400640 1 -- 172.21.15.113:0/4055230371 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247126154 0 0) 0x7fb3b006ea50 con 0x7fb3b0154d40 2024-03-04T07:33:52.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.470+0000 7fb3b5200640 1 -- 172.21.15.113:0/4055230371 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00c7b10 con 0x7fb3b0154960 2024-03-04T07:33:52.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b5200640 1 --2- 172.21.15.113:0/4055230371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 0x7fb3b0066f40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40097c0 tx=0x7fb3a401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7944e09d4ed604 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:52.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3af400640 1 -- 172.21.15.113:0/4055230371 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40b5070 con 0x7fb3b0154960 2024-03-04T07:33:52.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3af400640 1 -- 172.21.15.113:0/4055230371 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a40a8990 con 0x7fb3b0154960 2024-03-04T07:33:52.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3af400640 1 -- 172.21.15.113:0/4055230371 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40a8d60 con 0x7fb3b0154960 2024-03-04T07:33:52.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b6600640 1 -- 172.21.15.113:0/4055230371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 msgr2=0x7fb3b0066f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b6600640 1 --2- 172.21.15.113:0/4055230371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 0x7fb3b0066f40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40097c0 tx=0x7fb3a401ec90 comp rx=0 tx=0).stop 2024-03-04T07:33:52.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b6600640 1 -- 172.21.15.113:0/4055230371 shutdown_connections 2024-03-04T07:33:52.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b6600640 1 --2- 172.21.15.113:0/4055230371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3b00c68e0 0x7fb3b015dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b6600640 1 --2- 172.21.15.113:0/4055230371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 0x7fb3b0066f40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b6600640 1 -- 172.21.15.113:0/4055230371 >> 172.21.15.113:0/4055230371 conn(0x7fb3b00cc660 msgr2=0x7fb3b00cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:52.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b6600640 1 -- 172.21.15.113:0/4055230371 shutdown_connections 2024-03-04T07:33:52.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b6600640 1 -- 172.21.15.113:0/4055230371 wait complete. 2024-03-04T07:33:52.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b6600640 1 Processor -- start 2024-03-04T07:33:52.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.474+0000 7fb3b6600640 1 -- start start 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b6600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3b00c68e0 0x7fb3b00ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b6600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 0x7fb3b01f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b6600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3b0154d40 0x7fb3b00724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b6600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b00c80d0 con 0x7fb3b0154960 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b6600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b006ecf0 con 0x7fb3b00c68e0 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b6600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b006f920 con 0x7fb3b0154d40 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3afe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 0x7fb3b01f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3afe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 0x7fb3b01f0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50460/0 (socket says 172.21.15.113:50460) 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3afe00640 1 -- 172.21.15.113:0/2358421104 learned_addr learned my addr 172.21.15.113:0/2358421104 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b5200640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3b00c68e0 0x7fb3b00ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b5c00640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3b0154d40 0x7fb3b00724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3afe00640 1 -- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3b0154d40 msgr2=0x7fb3b00724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3afe00640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3b0154d40 0x7fb3b00724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3afe00640 1 -- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3b00c68e0 msgr2=0x7fb3b00ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3afe00640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3b00c68e0 0x7fb3b00ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3afe00640 1 -- 172.21.15.113:0/2358421104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b01f0060 con 0x7fb3b0154960 2024-03-04T07:33:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3afe00640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 0x7fb3b01f0b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00d800 tx=0x7fb39c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3acc00640 1 -- 172.21.15.113:0/2358421104 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c00b6c0 con 0x7fb3b0154960 2024-03-04T07:33:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3acc00640 1 -- 172.21.15.113:0/2358421104 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39c019070 con 0x7fb3b0154960 2024-03-04T07:33:52.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b6600640 1 -- 172.21.15.113:0/2358421104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b00cbef0 con 0x7fb3b0154960 2024-03-04T07:33:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3acc00640 1 -- 172.21.15.113:0/2358421104 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39c00ba60 con 0x7fb3b0154960 2024-03-04T07:33:52.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b6600640 1 -- 172.21.15.113:0/2358421104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b01f0320 con 0x7fb3b0154960 2024-03-04T07:33:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3b6600640 1 -- 172.21.15.113:0/2358421104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b0075a80 con 0x7fb3b0154960 2024-03-04T07:33:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.478+0000 7fb3acc00640 1 -- 172.21.15.113:0/2358421104 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb39c00bc00 con 0x7fb3b0154960 2024-03-04T07:33:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.482+0000 7fb3acc00640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb390066f90 0x7fb390069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.482+0000 7fb3acc00640 1 -- 172.21.15.113:0/2358421104 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb39c0851e0 con 0x7fb3b0154960 2024-03-04T07:33:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.482+0000 7fb3b5200640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb390066f90 0x7fb390069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.482+0000 7fb3b5200640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb390066f90 0x7fb390069450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40b2000 tx=0x7fb3a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:52.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.486+0000 7fb3acc00640 1 -- 172.21.15.113:0/2358421104 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb39c0564a0 con 0x7fb3b0154960 2024-03-04T07:33:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.638+0000 7fb3b6600640 1 -- 172.21.15.113:0/2358421104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3b014e920 con 0x7fb3b0154960 2024-03-04T07:33:52.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.638+0000 7fb3acc00640 1 -- 172.21.15.113:0/2358421104 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12920 (secure 0 0 0) 0x7fb39c055da0 con 0x7fb3b0154960 2024-03-04T07:33:52.643 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:33:52.643 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":17,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:44.191289+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:33:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 -- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb390066f90 msgr2=0x7fb390069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb390066f90 0x7fb390069450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40b2000 tx=0x7fb3a4008040 comp rx=0 tx=0).stop 2024-03-04T07:33:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 -- 172.21.15.113:0/2358421104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 msgr2=0x7fb3b01f0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 0x7fb3b01f0b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00d800 tx=0x7fb39c00dcc0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 -- 172.21.15.113:0/2358421104 shutdown_connections 2024-03-04T07:33:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3b0154d40 0x7fb3b00724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb390066f90 0x7fb390069450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3b0154960 0x7fb3b01f0b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 --2- 172.21.15.113:0/2358421104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3b00c68e0 0x7fb3b00ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 -- 172.21.15.113:0/2358421104 >> 172.21.15.113:0/2358421104 conn(0x7fb3b00cc660 msgr2=0x7fb3b014d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:52.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 -- 172.21.15.113:0/2358421104 shutdown_connections 2024-03-04T07:33:52.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.642+0000 7fb3b6600640 1 -- 172.21.15.113:0/2358421104 wait complete. 2024-03-04T07:33:52.663 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T07:33:41.507700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T07:33:52.663 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T07:33:52.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.798+0000 7f007ee00640 1 Processor -- start 2024-03-04T07:33:52.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.798+0000 7f007ee00640 1 -- start start 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.798+0000 7f007ee00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f007806ff60 0x7f00780ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.798+0000 7f007ee00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00780c9ce0 con 0x7f0078158320 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.798+0000 7f007ee00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007806e110 con 0x7f00780ce770 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.798+0000 7f007ee00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00780c9160 con 0x7f007806ff60 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.798+0000 7f0077e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0078158320 0x7f007806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36830/0 (socket says 172.21.15.113:36830) 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.798+0000 7f0077e00640 1 -- 172.21.15.113:0/3581316209 learned_addr learned my addr 172.21.15.113:0/3581316209 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0077400640 1 --2- 172.21.15.113:0/3581316209 >> v2:172.21.15.136:3300/0 conn(0x7f007806ff60 0x7f00780ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560084821 0 0) 0x7f00780c9ce0 con 0x7f0078158320 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f005c003530 con 0x7f0078158320 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126697799 0 0) 0x7f007806e110 con 0x7f00780ce770 2024-03-04T07:33:52.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00780c9ce0 con 0x7f00780ce770 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2022400678 0 0) 0x7f005c003530 con 0x7f0078158320 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 >> v2:172.21.15.136:3300/0 conn(0x7f007806ff60 msgr2=0x7f00780ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 --2- 172.21.15.113:0/3581316209 >> v2:172.21.15.136:3300/0 conn(0x7f007806ff60 0x7f00780ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 >> v1:172.21.15.136:6789/0 conn(0x7f00780ce770 legacy=0x7f007815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00780c93b0 con 0x7f0078158320 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f0068001680 con 0x7f0078158320 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 >> v1:172.21.15.113:6789/0 conn(0x7f0078158320 legacy=0x7f007806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f005c003b50 0x7f005c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005c004810 0x7f005c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f005c0051d0 0x7f005c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c9160 con 0x7f005c0051d0 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005c00c050 con 0x7f005c004810 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005c00c250 con 0x7f005c003b50 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f007ce00640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f005c0051d0 0x7f005c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0077e00640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f005c003b50 0x7f005c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0077400640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005c004810 0x7f005c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f007ce00640 1 -- 172.21.15.113:0/3581316209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f005c003b50 msgr2=0x7f005c003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:52.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f007ce00640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f005c003b50 0x7f005c003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f007ce00640 1 -- 172.21.15.113:0/3581316209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005c004810 msgr2=0x7f005c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:52.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f007ce00640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005c004810 0x7f005c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f007ce00640 1 -- 172.21.15.113:0/3581316209 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005c003530 con 0x7f005c0051d0 2024-03-04T07:33:52.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.802+0000 7f0077400640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005c004810 0x7f005c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:52.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ce00640 1 -- 172.21.15.113:0/3581316209 reap_dead start 2024-03-04T07:33:52.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ce00640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f005c0051d0 0x7f005c00b9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0070001fd0 tx=0x7f007001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6b16ed2db34ab4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:52.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070001c20 con 0x7f005c0051d0 2024-03-04T07:33:52.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0070003b80 con 0x7f005c0051d0 2024-03-04T07:33:52.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f0076a00640 1 -- 172.21.15.113:0/3581316209 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00700050d0 con 0x7f005c0051d0 2024-03-04T07:33:52.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 -- 172.21.15.113:0/3581316209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f005c0051d0 msgr2=0x7f005c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f005c0051d0 0x7f005c00b9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0070001fd0 tx=0x7f007001c900 comp rx=0 tx=0).stop 2024-03-04T07:33:52.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 -- 172.21.15.113:0/3581316209 shutdown_connections 2024-03-04T07:33:52.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 --2- 172.21.15.113:0/3581316209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f005c0051d0 0x7f005c00b9d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 -- 172.21.15.113:0/3581316209 >> 172.21.15.113:0/3581316209 conn(0x7f007806b820 msgr2=0x7f0078070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:52.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 -- 172.21.15.113:0/3581316209 shutdown_connections 2024-03-04T07:33:52.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 -- 172.21.15.113:0/3581316209 wait complete. 2024-03-04T07:33:52.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 Processor -- start 2024-03-04T07:33:52.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 -- start start 2024-03-04T07:33:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005c0051d0 0x7f00781f0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00780b07b0 0x7f00780c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00780cdf30 0x7f0078159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007806dcb0 con 0x7f00780cdf30 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c9970 con 0x7f005c0051d0 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.806+0000 7f007ee00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007806de30 con 0x7f00780b07b0 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00780cdf30 0x7f0078159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00780cdf30 0x7f0078159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50474/0 (socket says 172.21.15.113:50474) 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ce00640 1 -- 172.21.15.113:0/1395134504 learned_addr learned my addr 172.21.15.113:0/1395134504 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f0077e00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005c0051d0 0x7f00781f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f0077400640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00780b07b0 0x7f00780c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ce00640 1 -- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00780b07b0 msgr2=0x7f00780c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ce00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00780b07b0 0x7f00780c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ce00640 1 -- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005c0051d0 msgr2=0x7f00781f0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ce00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005c0051d0 0x7f00781f0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ce00640 1 -- 172.21.15.113:0/1395134504 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00780cc4a0 con 0x7f00780cdf30 2024-03-04T07:33:52.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ce00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00780cdf30 0x7f0078159a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f007001d1a0 tx=0x7f005c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:52.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f006fe00640 1 -- 172.21.15.113:0/1395134504 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070001c00 con 0x7f00780cdf30 2024-03-04T07:33:52.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f006fe00640 1 -- 172.21.15.113:0/1395134504 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0070003980 con 0x7f00780cdf30 2024-03-04T07:33:52.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ee00640 1 -- 172.21.15.113:0/1395134504 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00780cc780 con 0x7f00780cdf30 2024-03-04T07:33:52.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f006fe00640 1 -- 172.21.15.113:0/1395134504 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070006d60 con 0x7f00780cdf30 2024-03-04T07:33:52.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ee00640 1 -- 172.21.15.113:0/1395134504 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00781f3d00 con 0x7f00780cdf30 2024-03-04T07:33:52.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f007ee00640 1 -- 172.21.15.113:0/1395134504 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0078075c00 con 0x7f00780cdf30 2024-03-04T07:33:52.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.810+0000 7f006fe00640 1 -- 172.21.15.113:0/1395134504 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00700a64f0 con 0x7f00780cdf30 2024-03-04T07:33:52.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.814+0000 7f006fe00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0058066f60 0x7f0058069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:52.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.814+0000 7f006fe00640 1 -- 172.21.15.113:0/1395134504 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f007011eb70 con 0x7f00780cdf30 2024-03-04T07:33:52.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.814+0000 7f0077e00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0058066f60 0x7f0058069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:52.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.814+0000 7f0077e00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0058066f60 0x7f0058069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c10 tx=0x7f0068001c80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:52.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.814+0000 7f006fe00640 1 -- 172.21.15.113:0/1395134504 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f00700be060 con 0x7f00780cdf30 2024-03-04T07:33:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.970+0000 7f007ee00640 1 -- 172.21.15.113:0/1395134504 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f007806fd40 con 0x7f00780cdf30 2024-03-04T07:33:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f006fe00640 1 -- 172.21.15.113:0/1395134504 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0070016610 con 0x7f00780cdf30 2024-03-04T07:33:52.976 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-03-04T07:33:52.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 -- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0058066f60 msgr2=0x7f0058069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0058066f60 0x7f0058069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0068000c10 tx=0x7f0068001c80 comp rx=0 tx=0).stop 2024-03-04T07:33:52.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 -- 172.21.15.113:0/1395134504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00780cdf30 msgr2=0x7f0078159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00780cdf30 0x7f0078159a00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f007001d1a0 tx=0x7f005c005840 comp rx=0 tx=0).stop 2024-03-04T07:33:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 -- 172.21.15.113:0/1395134504 shutdown_connections 2024-03-04T07:33:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0058066f60 0x7f0058069420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00780cdf30 0x7f0078159a00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00780b07b0 0x7f00780c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 --2- 172.21.15.113:0/1395134504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005c0051d0 0x7f00781f0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 -- 172.21.15.113:0/1395134504 >> 172.21.15.113:0/1395134504 conn(0x7f007806b820 msgr2=0x7f0078156d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 -- 172.21.15.113:0/1395134504 shutdown_connections 2024-03-04T07:33:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:52.974+0000 7f007ee00640 1 -- 172.21.15.113:0/1395134504 wait complete. 2024-03-04T07:33:52.996 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-04T07:33:53.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbe1800640 1 Processor -- start 2024-03-04T07:33:53.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbe1800640 1 -- start start 2024-03-04T07:33:53.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbe1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c8db0 0x7fdbdc1526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbe1800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdbdc0c6840 0x7fdbdc152c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbe1800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbdc0c6da0 con 0x7fdbdc0c89d0 2024-03-04T07:33:53.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbe1800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbdc0c7080 con 0x7fdbdc0c6840 2024-03-04T07:33:53.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbe1800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc0c7c10 con 0x7fdbdc0c8db0 2024-03-04T07:33:53.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbd2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c8db0 0x7fdbdc1526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbdb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdbdc0c6840 0x7fdbdc152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbdb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdbdc0c6840 0x7fdbdc152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:50334/0 (socket says 172.21.15.113:50334) 2024-03-04T07:33:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbdaa00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fdbdc0c89d0 0x7fdbdc0ca030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:42204/0 (socket says 172.21.15.113:42204) 2024-03-04T07:33:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbdaa00640 1 -- 172.21.15.113:0/3213642033 learned_addr learned my addr 172.21.15.113:0/3213642033 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:33:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbd2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c8db0 0x7fdbdc1526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:57260/0 (socket says 172.21.15.113:57260) 2024-03-04T07:33:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbdb400640 1 -- 172.21.15.113:0/3213642033 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c8db0 msgr2=0x7fdbdc1526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbdb400640 1 --2- 172.21.15.113:0/3213642033 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c8db0 0x7fdbdc1526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbdb400640 1 -- 172.21.15.113:0/3213642033 >> v1:172.21.15.136:6789/0 conn(0x7fdbdc0c89d0 legacy=0x7fdbdc0ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:53.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.130+0000 7fdbdb400640 1 -- 172.21.15.113:0/3213642033 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc0c7ef0 con 0x7fdbdc0c6840 2024-03-04T07:33:53.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbdb400640 1 --2- 172.21.15.113:0/3213642033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbdc0c6840 0x7fdbdc152c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009d90 tx=0x7fdbd401f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b7687f6d7d772b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:53.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbda000640 1 -- 172.21.15.113:0/3213642033 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40b4070 con 0x7fdbdc0c6840 2024-03-04T07:33:53.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbda000640 1 -- 172.21.15.113:0/3213642033 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd40af600 con 0x7fdbdc0c6840 2024-03-04T07:33:53.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbda000640 1 -- 172.21.15.113:0/3213642033 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd40af9d0 con 0x7fdbdc0c6840 2024-03-04T07:33:53.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbe1800640 1 -- 172.21.15.113:0/3213642033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbdc0c6840 msgr2=0x7fdbdc152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:53.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbe1800640 1 --2- 172.21.15.113:0/3213642033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbdc0c6840 0x7fdbdc152c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009d90 tx=0x7fdbd401f150 comp rx=0 tx=0).stop 2024-03-04T07:33:53.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbe1800640 1 -- 172.21.15.113:0/3213642033 shutdown_connections 2024-03-04T07:33:53.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbe1800640 1 --2- 172.21.15.113:0/3213642033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbdc0c6840 0x7fdbdc152c30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbe1800640 1 --2- 172.21.15.113:0/3213642033 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c8db0 0x7fdbdc1526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbe1800640 1 -- 172.21.15.113:0/3213642033 >> 172.21.15.113:0/3213642033 conn(0x7fdbdc06deb0 msgr2=0x7fdbdc06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:53.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbe1800640 1 -- 172.21.15.113:0/3213642033 shutdown_connections 2024-03-04T07:33:53.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.134+0000 7fdbe1800640 1 -- 172.21.15.113:0/3213642033 wait complete. 2024-03-04T07:33:53.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbe1800640 1 Processor -- start 2024-03-04T07:33:53.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbe1800640 1 -- start start 2024-03-04T07:33:53.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbe1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc0c6840 0x7fdbdc1e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbe1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c89d0 0x7fdbdc146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbe1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbdc0c8db0 0x7fdbdc146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbe1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc06d590 con 0x7fdbdc0c6840 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbe1800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc06d410 con 0x7fdbdc0c8db0 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbe1800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc06b060 con 0x7fdbdc0c89d0 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc0c6840 0x7fdbdc1e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc0c6840 0x7fdbdc1e5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50488/0 (socket says 172.21.15.113:50488) 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdaa00640 1 -- 172.21.15.113:0/3119881564 learned_addr learned my addr 172.21.15.113:0/3119881564 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdb400640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbdc0c8db0 0x7fdbdc146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbd2000640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c89d0 0x7fdbdc146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdaa00640 1 -- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c89d0 msgr2=0x7fdbdc146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdaa00640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c89d0 0x7fdbdc146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdaa00640 1 -- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbdc0c8db0 msgr2=0x7fdbdc146f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdaa00640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbdc0c8db0 0x7fdbdc146f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdaa00640 1 -- 172.21.15.113:0/3119881564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc14ebe0 con 0x7fdbdc0c6840 2024-03-04T07:33:53.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdb400640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbdc0c8db0 0x7fdbdc146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:53.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbdaa00640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc0c6840 0x7fdbdc1e5a80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c450 tx=0x7fdbcc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:53.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbd3400640 1 -- 172.21.15.113:0/3119881564 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc017070 con 0x7fdbdc0c6840 2024-03-04T07:33:53.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbd3400640 1 -- 172.21.15.113:0/3119881564 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc005cf0 con 0x7fdbdc0c6840 2024-03-04T07:33:53.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbd3400640 1 -- 172.21.15.113:0/3119881564 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc007290 con 0x7fdbdc0c6840 2024-03-04T07:33:53.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbe1800640 1 -- 172.21.15.113:0/3119881564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbdc1f38d0 con 0x7fdbdc0c6840 2024-03-04T07:33:53.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.138+0000 7fdbe1800640 1 -- 172.21.15.113:0/3119881564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbdc1f3e90 con 0x7fdbdc0c6840 2024-03-04T07:33:53.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.142+0000 7fdbe1800640 1 -- 172.21.15.113:0/3119881564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbdc075c00 con 0x7fdbdc0c6840 2024-03-04T07:33:53.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.142+0000 7fdbd3400640 1 -- 172.21.15.113:0/3119881564 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdbcc01a070 con 0x7fdbdc0c6840 2024-03-04T07:33:53.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.142+0000 7fdbd3400640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbb8066f60 0x7fdbb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.142+0000 7fdbd3400640 1 -- 172.21.15.113:0/3119881564 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdbcc083f40 con 0x7fdbdc0c6840 2024-03-04T07:33:53.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.142+0000 7fdbd2000640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbb8066f60 0x7fdbb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.142+0000 7fdbd2000640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbb8066f60 0x7fdbb8069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc068580 tx=0x7fdbc4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:53.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.146+0000 7fdbd3400640 1 -- 172.21.15.113:0/3119881564 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdbcc054c40 con 0x7fdbdc0c6840 2024-03-04T07:33:53.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 -- 172.21.15.113:0/3119881564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdbdc06f9f0 con 0x7fdbdc0c6840 2024-03-04T07:33:53.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbd3400640 1 -- 172.21.15.113:0/3119881564 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdbcc058ce0 con 0x7fdbdc0c6840 2024-03-04T07:33:53.309 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:33:53.309 INFO:teuthology.orchestra.run.smithi113.stdout:[ 2024-03-04T07:33:53.309 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-03-04T07:33:53.309 INFO:teuthology.orchestra.run.smithi113.stdout: "poolnum": 1, 2024-03-04T07:33:53.309 INFO:teuthology.orchestra.run.smithi113.stdout: "poolname": ".mgr" 2024-03-04T07:33:53.309 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-03-04T07:33:53.309 INFO:teuthology.orchestra.run.smithi113.stdout:] 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 -- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbb8066f60 msgr2=0x7fdbb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbb8066f60 0x7fdbb8069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc068580 tx=0x7fdbc4009190 comp rx=0 tx=0).stop 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 -- 172.21.15.113:0/3119881564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc0c6840 msgr2=0x7fdbdc1e5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc0c6840 0x7fdbdc1e5a80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c450 tx=0x7fdbcc00c910 comp rx=0 tx=0).stop 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 -- 172.21.15.113:0/3119881564 shutdown_connections 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbdc0c8db0 0x7fdbdc146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbdc0c89d0 0x7fdbdc146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbb8066f60 0x7fdbb8069420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 --2- 172.21.15.113:0/3119881564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbdc0c6840 0x7fdbdc1e5a80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 -- 172.21.15.113:0/3119881564 >> 172.21.15.113:0/3119881564 conn(0x7fdbdc06deb0 msgr2=0x7fdbdc06b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 -- 172.21.15.113:0/3119881564 shutdown_connections 2024-03-04T07:33:53.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.306+0000 7fdbe1800640 1 -- 172.21.15.113:0/3119881564 wait complete. 2024-03-04T07:33:53.329 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-04T07:33:53.329 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-04T07:33:53.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.462+0000 7ff4be200640 1 Processor -- start 2024-03-04T07:33:53.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.462+0000 7ff4be200640 1 -- start start 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4be200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4b8152cf0 0x7ff4b80c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4be200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff4b8070cd0 0x7ff4b815be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4be200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b8158910 con 0x7ff4b815c3c0 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4be200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b81530d0 con 0x7ff4b8152cf0 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4be200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b80c88c0 con 0x7ff4b8070cd0 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4b8152cf0 0x7ff4b80c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4b8152cf0 0x7ff4b80c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50504/0 (socket says 172.21.15.113:50504) 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b7400640 1 -- 172.21.15.113:0/4089514882 learned_addr learned my addr 172.21.15.113:0/4089514882 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b7e00640 1 --1- 172.21.15.113:0/4089514882 >> v1:172.21.15.113:6789/0 conn(0x7ff4b815c3c0 0x7ff4b80c8d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36844/0 (socket says 172.21.15.113:36844) 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4aea00640 1 --2- 172.21.15.113:0/4089514882 >> v2:172.21.15.136:3300/0 conn(0x7ff4b8070cd0 0x7ff4b815be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b7400640 1 -- 172.21.15.113:0/4089514882 >> v2:172.21.15.136:3300/0 conn(0x7ff4b8070cd0 msgr2=0x7ff4b815be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b7400640 1 --2- 172.21.15.113:0/4089514882 >> v2:172.21.15.136:3300/0 conn(0x7ff4b8070cd0 0x7ff4b815be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b7400640 1 -- 172.21.15.113:0/4089514882 >> v1:172.21.15.113:6789/0 conn(0x7ff4b815c3c0 legacy=0x7ff4b80c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:53.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b7400640 1 -- 172.21.15.113:0/4089514882 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b8158bd0 con 0x7ff4b8152cf0 2024-03-04T07:33:53.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b7400640 1 --2- 172.21.15.113:0/4089514882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 0x7ff4b80c8210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80af1d0 tx=0x7ff4a801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249e495195468eeb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:53.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b6a00640 1 -- 172.21.15.113:0/4089514882 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80b5070 con 0x7ff4b8152cf0 2024-03-04T07:33:53.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.466+0000 7ff4b6a00640 1 -- 172.21.15.113:0/4089514882 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a80a8a10 con 0x7ff4b8152cf0 2024-03-04T07:33:53.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4b6a00640 1 -- 172.21.15.113:0/4089514882 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80a8de0 con 0x7ff4b8152cf0 2024-03-04T07:33:53.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 -- 172.21.15.113:0/4089514882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 msgr2=0x7ff4b80c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:53.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 --2- 172.21.15.113:0/4089514882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 0x7ff4b80c8210 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4b80af1d0 tx=0x7ff4a801ed10 comp rx=0 tx=0).stop 2024-03-04T07:33:53.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 -- 172.21.15.113:0/4089514882 shutdown_connections 2024-03-04T07:33:53.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 --2- 172.21.15.113:0/4089514882 >> v2:172.21.15.136:3300/0 conn(0x7ff4b8070cd0 0x7ff4b815be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 --2- 172.21.15.113:0/4089514882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 0x7ff4b80c8210 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 -- 172.21.15.113:0/4089514882 >> 172.21.15.113:0/4089514882 conn(0x7ff4b806c8c0 msgr2=0x7ff4b806ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:53.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 -- 172.21.15.113:0/4089514882 shutdown_connections 2024-03-04T07:33:53.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 -- 172.21.15.113:0/4089514882 wait complete. 2024-03-04T07:33:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 Processor -- start 2024-03-04T07:33:53.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 -- start start 2024-03-04T07:33:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4b8070cd0 0x7ff4b8157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 0x7ff4b80cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4b815c3c0 0x7ff4b81eec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8159470 con 0x7ff4b8152cf0 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8159190 con 0x7ff4b8070cd0 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4be200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80c9420 con 0x7ff4b815c3c0 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4aea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 0x7ff4b80cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4aea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 0x7ff4b80cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50518/0 (socket says 172.21.15.113:50518) 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4aea00640 1 -- 172.21.15.113:0/3700833198 learned_addr learned my addr 172.21.15.113:0/3700833198 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4b7400640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4b8070cd0 0x7ff4b8157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4b7e00640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4b815c3c0 0x7ff4b81eec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4aea00640 1 -- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4b815c3c0 msgr2=0x7ff4b81eec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4aea00640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4b815c3c0 0x7ff4b81eec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4aea00640 1 -- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4b8070cd0 msgr2=0x7ff4b8157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:53.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4aea00640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4b8070cd0 0x7ff4b8157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.470+0000 7ff4aea00640 1 -- 172.21.15.113:0/3700833198 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80cca20 con 0x7ff4b8152cf0 2024-03-04T07:33:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4aea00640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 0x7ff4b80cbd40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000d800 tx=0x7ff4a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4afe00640 1 -- 172.21.15.113:0/3700833198 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000b6c0 con 0x7ff4b8152cf0 2024-03-04T07:33:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4afe00640 1 -- 172.21.15.113:0/3700833198 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a0019070 con 0x7ff4b8152cf0 2024-03-04T07:33:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4be200640 1 -- 172.21.15.113:0/3700833198 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b80cb2a0 con 0x7ff4b8152cf0 2024-03-04T07:33:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4afe00640 1 -- 172.21.15.113:0/3700833198 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000ba60 con 0x7ff4b8152cf0 2024-03-04T07:33:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4be200640 1 -- 172.21.15.113:0/3700833198 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b80cb540 con 0x7ff4b8152cf0 2024-03-04T07:33:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4be200640 1 -- 172.21.15.113:0/3700833198 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b8075c00 con 0x7ff4b8152cf0 2024-03-04T07:33:53.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4afe00640 1 -- 172.21.15.113:0/3700833198 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4a000bc00 con 0x7ff4b8152cf0 2024-03-04T07:33:53.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4afe00640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff494066f60 0x7ff494069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:53.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4afe00640 1 -- 172.21.15.113:0/3700833198 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff4a0085180 con 0x7ff4b8152cf0 2024-03-04T07:33:53.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.474+0000 7ff4b7400640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff494066f60 0x7ff494069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:53.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.478+0000 7ff4b7400640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff494066f60 0x7ff494069420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80b2000 tx=0x7ff4a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:53.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.478+0000 7ff4afe00640 1 -- 172.21.15.113:0/3700833198 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff4a00564e0 con 0x7ff4b8152cf0 2024-03-04T07:33:53.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:53.634+0000 7ff4be200640 1 -- 172.21.15.113:0/3700833198 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff4b80c71d0 con 0x7ff4b8152cf0 2024-03-04T07:33:53.910 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:33:53.915 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:33:54.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.498+0000 7ff4afe00640 1 -- 172.21.15.113:0/3700833198 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff4a0083e20 con 0x7ff4b8152cf0 2024-03-04T07:33:54.503 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_metadata' created 2024-03-04T07:33:54.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 -- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff494066f60 msgr2=0x7ff494069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:54.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff494066f60 0x7ff494069420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4a80b2000 tx=0x7ff4a8008040 comp rx=0 tx=0).stop 2024-03-04T07:33:54.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 -- 172.21.15.113:0/3700833198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 msgr2=0x7ff4b80cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:54.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 0x7ff4b80cbd40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000d800 tx=0x7ff4a000dcc0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 -- 172.21.15.113:0/3700833198 shutdown_connections 2024-03-04T07:33:54.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4b815c3c0 0x7ff4b81eec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff494066f60 0x7ff494069420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4b8152cf0 0x7ff4b80cbd40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 --2- 172.21.15.113:0/3700833198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4b8070cd0 0x7ff4b8157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 -- 172.21.15.113:0/3700833198 >> 172.21.15.113:0/3700833198 conn(0x7ff4b806c8c0 msgr2=0x7ff4b815bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:54.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 -- 172.21.15.113:0/3700833198 shutdown_connections 2024-03-04T07:33:54.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.502+0000 7ff4be200640 1 -- 172.21.15.113:0/3700833198 wait complete. 2024-03-04T07:33:54.532 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-04T07:33:54.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.670+0000 7f807b000640 1 Processor -- start 2024-03-04T07:33:54.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.670+0000 7f807b000640 1 -- start start 2024-03-04T07:33:54.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f807b000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f80740cc500 0x7f80740c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:54.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f807b000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8074073360 0x7f807406bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:54.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f807b000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80740cc8e0 0x7f8074070f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:54.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f807b000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80741588b0 con 0x7f80740cc8e0 2024-03-04T07:33:54.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f807b000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8074158b70 con 0x7f80740cc500 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f807b000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8074158e50 con 0x7f8074073360 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f8079000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80740cc8e0 0x7f8074070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f8079000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f80740cc8e0 0x7f8074070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50530/0 (socket says 172.21.15.113:50530) 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f8079000640 1 -- 172.21.15.113:0/3482045075 learned_addr learned my addr 172.21.15.113:0/3482045075 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f806b400640 1 --2- 172.21.15.113:0/3482045075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8074073360 0x7f807406bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f8073e00640 1 --2- 172.21.15.113:0/3482045075 >> v2:172.21.15.136:3300/0 conn(0x7f80740cc500 0x7f80740c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f8079000640 1 -- 172.21.15.113:0/3482045075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8074073360 msgr2=0x7f807406bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f8079000640 1 --2- 172.21.15.113:0/3482045075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8074073360 0x7f807406bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f8079000640 1 -- 172.21.15.113:0/3482045075 >> v2:172.21.15.136:3300/0 conn(0x7f80740cc500 msgr2=0x7f80740c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f8079000640 1 --2- 172.21.15.113:0/3482045075 >> v2:172.21.15.136:3300/0 conn(0x7f80740cc500 0x7f80740c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.674+0000 7f8079000640 1 -- 172.21.15.113:0/3482045075 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8074159130 con 0x7f80740cc8e0 2024-03-04T07:33:54.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f8079000640 1 --2- 172.21.15.113:0/3482045075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 0x7f8074070f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f806c009e50 tx=0x7f806c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c66946bf3eb73 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:54.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f8073400640 1 -- 172.21.15.113:0/3482045075 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c0b4070 con 0x7f80740cc8e0 2024-03-04T07:33:54.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f8073400640 1 -- 172.21.15.113:0/3482045075 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c0af400 con 0x7f80740cc8e0 2024-03-04T07:33:54.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f8073400640 1 -- 172.21.15.113:0/3482045075 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c0af7d0 con 0x7f80740cc8e0 2024-03-04T07:33:54.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 -- 172.21.15.113:0/3482045075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 msgr2=0x7f8074070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:54.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 --2- 172.21.15.113:0/3482045075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 0x7f8074070f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f806c009e50 tx=0x7f806c01f1e0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 -- 172.21.15.113:0/3482045075 shutdown_connections 2024-03-04T07:33:54.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 --2- 172.21.15.113:0/3482045075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 0x7f8074070f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 --2- 172.21.15.113:0/3482045075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8074073360 0x7f807406bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 --2- 172.21.15.113:0/3482045075 >> v2:172.21.15.136:3300/0 conn(0x7f80740cc500 0x7f80740c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 -- 172.21.15.113:0/3482045075 >> 172.21.15.113:0/3482045075 conn(0x7f8074066db0 msgr2=0x7f80740671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:54.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 -- 172.21.15.113:0/3482045075 shutdown_connections 2024-03-04T07:33:54.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 -- 172.21.15.113:0/3482045075 wait complete. 2024-03-04T07:33:54.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 Processor -- start 2024-03-04T07:33:54.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 -- start start 2024-03-04T07:33:54.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8074073360 0x7f80740b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80740cc500 0x7f8074157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 0x7f80741f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80740715e0 con 0x7f80740cc8e0 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8074071460 con 0x7f8074073360 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.678+0000 7f807b000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8074071760 con 0x7f80740cc500 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8079000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 0x7f80741f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8079000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 0x7f80741f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50546/0 (socket says 172.21.15.113:50546) 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8079000640 1 -- 172.21.15.113:0/3277463077 learned_addr learned my addr 172.21.15.113:0/3277463077 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f806b400640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80740cc500 0x7f8074157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8073e00640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8074073360 0x7f80740b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8079000640 1 -- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80740cc500 msgr2=0x7f8074157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8079000640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80740cc500 0x7f8074157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8079000640 1 -- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8074073360 msgr2=0x7f80740b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:54.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8079000640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8074073360 0x7f80740b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:54.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8079000640 1 -- 172.21.15.113:0/3277463077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80740c2e80 con 0x7f80740cc8e0 2024-03-04T07:33:54.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8073e00640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8074073360 0x7f80740b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:54.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8079000640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 0x7f80741f02a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f806c01d890 tx=0x7f806c01bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:54.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8070c00640 1 -- 172.21.15.113:0/3277463077 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c0b4040 con 0x7f80740cc8e0 2024-03-04T07:33:54.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8070c00640 1 -- 172.21.15.113:0/3277463077 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c0afd30 con 0x7f80740cc8e0 2024-03-04T07:33:54.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8070c00640 1 -- 172.21.15.113:0/3277463077 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c009510 con 0x7f80740cc8e0 2024-03-04T07:33:54.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f807b000640 1 -- 172.21.15.113:0/3277463077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80741f39b0 con 0x7f80740cc8e0 2024-03-04T07:33:54.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f807b000640 1 -- 172.21.15.113:0/3277463077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80741f3f70 con 0x7f80740cc8e0 2024-03-04T07:33:54.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f807b000640 1 -- 172.21.15.113:0/3277463077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8074075c00 con 0x7f80740cc8e0 2024-03-04T07:33:54.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.682+0000 7f8070c00640 1 -- 172.21.15.113:0/3277463077 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f806c0c2020 con 0x7f80740cc8e0 2024-03-04T07:33:54.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.686+0000 7f8070c00640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8050066f90 0x7f8050069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:54.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.686+0000 7f8070c00640 1 -- 172.21.15.113:0/3277463077 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f806c122810 con 0x7f80740cc8e0 2024-03-04T07:33:54.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.686+0000 7f8073e00640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8050066f90 0x7f8050069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:54.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.686+0000 7f8073e00640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8050066f90 0x7f8050069450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8064007a70 tx=0x7f8064008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:54.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.690+0000 7f8070c00640 1 -- 172.21.15.113:0/3277463077 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f806c0c9020 con 0x7f80740cc8e0 2024-03-04T07:33:54.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:54.846+0000 7f807b000640 1 -- 172.21.15.113:0/3277463077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f807406ba20 con 0x7f80740cc8e0 2024-03-04T07:33:55.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.510+0000 7f8070c00640 1 -- 172.21.15.113:0/3277463077 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f806c018d40 con 0x7f80740cc8e0 2024-03-04T07:33:55.514 INFO:teuthology.orchestra.run.smithi113.stderr:pool 'cephfs_data' created 2024-03-04T07:33:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 -- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8050066f90 msgr2=0x7f8050069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8050066f90 0x7f8050069450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8064007a70 tx=0x7f8064008040 comp rx=0 tx=0).stop 2024-03-04T07:33:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 -- 172.21.15.113:0/3277463077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 msgr2=0x7f80741f02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 0x7f80741f02a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f806c01d890 tx=0x7f806c01bf80 comp rx=0 tx=0).stop 2024-03-04T07:33:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 -- 172.21.15.113:0/3277463077 shutdown_connections 2024-03-04T07:33:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8050066f90 0x7f8050069450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80740cc8e0 0x7f80741f02a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f80740cc500 0x7f8074157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 --2- 172.21.15.113:0/3277463077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8074073360 0x7f80740b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:55.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 -- 172.21.15.113:0/3277463077 >> 172.21.15.113:0/3277463077 conn(0x7f8074066db0 msgr2=0x7f80740ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:55.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 -- 172.21.15.113:0/3277463077 shutdown_connections 2024-03-04T07:33:55.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.514+0000 7f807b000640 1 -- 172.21.15.113:0/3277463077 wait complete. 2024-03-04T07:33:55.555 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-04T07:33:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.694+0000 7f5f3d200640 1 Processor -- start 2024-03-04T07:33:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.694+0000 7f5f3d200640 1 -- start start 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f3d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f380c82e0 0x7f5f380c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f3d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f380cb6e0 0x7f5f3815c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f3d200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380c4bf0 con 0x7f5f381528c0 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f3d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f38152ca0 con 0x7f5f380cb6e0 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f3d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380c31c0 con 0x7f5f380c82e0 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f37e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f380cb6e0 0x7f5f3815c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f37e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f380cb6e0 0x7f5f3815c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50548/0 (socket says 172.21.15.113:50548) 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f37e00640 1 -- 172.21.15.113:0/3533641007 learned_addr learned my addr 172.21.15.113:0/3533641007 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f36a00640 1 --2- 172.21.15.113:0/3533641007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f380c82e0 0x7f5f380c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f37e00640 1 -- 172.21.15.113:0/3533641007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f380c82e0 msgr2=0x7f5f380c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f37e00640 1 --2- 172.21.15.113:0/3533641007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f380c82e0 0x7f5f380c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f37e00640 1 -- 172.21.15.113:0/3533641007 >> v1:172.21.15.136:6789/0 conn(0x7f5f381528c0 legacy=0x7f5f38072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f37e00640 1 -- 172.21.15.113:0/3533641007 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380c34a0 con 0x7f5f380cb6e0 2024-03-04T07:33:55.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.698+0000 7f5f37e00640 1 --2- 172.21.15.113:0/3533641007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380cb6e0 0x7f5f3815c0b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009e50 tx=0x7f5f3001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0ec7b5b67bb252 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:55.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f36000640 1 -- 172.21.15.113:0/3533641007 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300b4070 con 0x7f5f380cb6e0 2024-03-04T07:33:55.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f36000640 1 -- 172.21.15.113:0/3533641007 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f300af400 con 0x7f5f380cb6e0 2024-03-04T07:33:55.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f36000640 1 -- 172.21.15.113:0/3533641007 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300af7d0 con 0x7f5f380cb6e0 2024-03-04T07:33:55.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 -- 172.21.15.113:0/3533641007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380cb6e0 msgr2=0x7f5f3815c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:55.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 --2- 172.21.15.113:0/3533641007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380cb6e0 0x7f5f3815c0b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5f30009e50 tx=0x7f5f3001f210 comp rx=0 tx=0).stop 2024-03-04T07:33:55.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 -- 172.21.15.113:0/3533641007 shutdown_connections 2024-03-04T07:33:55.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 --2- 172.21.15.113:0/3533641007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380cb6e0 0x7f5f3815c0b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:55.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 --2- 172.21.15.113:0/3533641007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f380c82e0 0x7f5f380c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:55.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 -- 172.21.15.113:0/3533641007 >> 172.21.15.113:0/3533641007 conn(0x7f5f3806f2d0 msgr2=0x7f5f3806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:55.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 -- 172.21.15.113:0/3533641007 shutdown_connections 2024-03-04T07:33:55.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 -- 172.21.15.113:0/3533641007 wait complete. 2024-03-04T07:33:55.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 Processor -- start 2024-03-04T07:33:55.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 -- start start 2024-03-04T07:33:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f380c82e0 0x7f5f3815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f380cb6e0 0x7f5f38147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f381528c0 0x7f5f38147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3815bb90 con 0x7f5f381528c0 2024-03-04T07:33:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380c5140 con 0x7f5f380cb6e0 2024-03-04T07:33:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f3d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f38073250 con 0x7f5f380c82e0 2024-03-04T07:33:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f37e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f381528c0 0x7f5f38147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f37e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f381528c0 0x7f5f38147e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50558/0 (socket says 172.21.15.113:50558) 2024-03-04T07:33:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f37e00640 1 -- 172.21.15.113:0/1178008171 learned_addr learned my addr 172.21.15.113:0/1178008171 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f37400640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f380c82e0 0x7f5f3815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.702+0000 7f5f36a00640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f380cb6e0 0x7f5f38147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f37e00640 1 -- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f380c82e0 msgr2=0x7f5f3815b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f37e00640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f380c82e0 0x7f5f3815b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f37e00640 1 -- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f380cb6e0 msgr2=0x7f5f38147930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f37e00640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f380cb6e0 0x7f5f38147930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f37e00640 1 -- 172.21.15.113:0/1178008171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f381f0990 con 0x7f5f381528c0 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f36a00640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f380cb6e0 0x7f5f38147930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f37e00640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f381528c0 0x7f5f38147e70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f3001ce40 tx=0x7f5f3001bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f2f400640 1 -- 172.21.15.113:0/1178008171 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300b4040 con 0x7f5f381528c0 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f2f400640 1 -- 172.21.15.113:0/1178008171 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f300c0070 con 0x7f5f381528c0 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f3d200640 1 -- 172.21.15.113:0/1178008171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f381f0c70 con 0x7f5f381528c0 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f2f400640 1 -- 172.21.15.113:0/1178008171 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30009510 con 0x7f5f381528c0 2024-03-04T07:33:55.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f3d200640 1 -- 172.21.15.113:0/1178008171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f381f1230 con 0x7f5f381528c0 2024-03-04T07:33:55.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f3d200640 1 -- 172.21.15.113:0/1178008171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f38075a80 con 0x7f5f381528c0 2024-03-04T07:33:55.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f2f400640 1 -- 172.21.15.113:0/1178008171 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f300a8d10 con 0x7f5f381528c0 2024-03-04T07:33:55.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f2f400640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f14066f90 0x7f5f14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:55.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.706+0000 7f5f2f400640 1 -- 172.21.15.113:0/1178008171 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5f30123990 con 0x7f5f381528c0 2024-03-04T07:33:55.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.710+0000 7f5f37400640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f14066f90 0x7f5f14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:55.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.710+0000 7f5f37400640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f14066f90 0x7f5f14069450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f28000f70 tx=0x7f5f28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:55.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.710+0000 7f5f2f400640 1 -- 172.21.15.113:0/1178008171 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5f30018d70 con 0x7f5f381528c0 2024-03-04T07:33:55.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:55.898+0000 7f5f3d200640 1 -- 172.21.15.113:0/1178008171 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5f3814e040 con 0x7f5f381528c0 2024-03-04T07:33:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.514+0000 7f5f2f400640 1 -- 172.21.15.113:0/1178008171 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5f300f4aa0 con 0x7f5f381528c0 2024-03-04T07:33:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:new fs with metadata pool 2 and data pool 3 2024-03-04T07:33:56.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 -- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f14066f90 msgr2=0x7f5f14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:56.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f14066f90 0x7f5f14069450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f28000f70 tx=0x7f5f28009040 comp rx=0 tx=0).stop 2024-03-04T07:33:56.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 -- 172.21.15.113:0/1178008171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f381528c0 msgr2=0x7f5f38147e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:56.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f381528c0 0x7f5f38147e70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f3001ce40 tx=0x7f5f3001bfb0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 -- 172.21.15.113:0/1178008171 shutdown_connections 2024-03-04T07:33:56.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f14066f90 0x7f5f14069450 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f28000f70 tx=0x7f5f28009040 comp rx=0 tx=0).stop 2024-03-04T07:33:56.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f381528c0 0x7f5f38147e70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f380cb6e0 0x7f5f38147930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 --2- 172.21.15.113:0/1178008171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f380c82e0 0x7f5f3815b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 -- 172.21.15.113:0/1178008171 >> 172.21.15.113:0/1178008171 conn(0x7f5f3806f2d0 msgr2=0x7f5f381500f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:56.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 -- 172.21.15.113:0/1178008171 shutdown_connections 2024-03-04T07:33:56.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.518+0000 7f5f3d200640 1 -- 172.21.15.113:0/1178008171 wait complete. 2024-03-04T07:33:56.561 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:33:56.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f542d200640 1 Processor -- start 2024-03-04T07:33:56.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f542d200640 1 -- start start 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f542d200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5428152a50 0x7f54280ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f542d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f542806ff20 0x7f54280ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f542d200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5428152e30 con 0x7f54280c8b90 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f542d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54280c31b0 con 0x7f542806ff20 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f542d200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542806f840 con 0x7f5428152a50 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f5425400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f542806ff20 0x7f54280ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f5425400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f542806ff20 0x7f54280ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50572/0 (socket says 172.21.15.113:50572) 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f5425400640 1 -- 172.21.15.113:0/62223124 learned_addr learned my addr 172.21.15.113:0/62223124 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f5425e00640 1 --2- 172.21.15.113:0/62223124 >> v2:172.21.15.136:3300/0 conn(0x7f5428152a50 0x7f54280ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f5425400640 1 -- 172.21.15.113:0/62223124 >> v2:172.21.15.136:3300/0 conn(0x7f5428152a50 msgr2=0x7f54280ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:56.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f5425400640 1 --2- 172.21.15.113:0/62223124 >> v2:172.21.15.136:3300/0 conn(0x7f5428152a50 0x7f54280ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f5425400640 1 -- 172.21.15.113:0/62223124 >> v1:172.21.15.113:6789/0 conn(0x7f54280c8b90 legacy=0x7f542815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:56.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f5425400640 1 -- 172.21.15.113:0/62223124 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542806f280 con 0x7f542806ff20 2024-03-04T07:33:56.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.694+0000 7f5424a00640 1 -- 172.21.15.113:0/62223124 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070254621 0 0) 0x7f5428152e30 con 0x7f54280c8b90 2024-03-04T07:33:56.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f5425400640 1 --2- 172.21.15.113:0/62223124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f542806ff20 0x7f54280ce660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5428067cd0 tx=0x7f541001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c24561f3a936220 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:56.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f5424a00640 1 -- 172.21.15.113:0/62223124 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54100aa680 con 0x7f542806ff20 2024-03-04T07:33:56.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f5424a00640 1 -- 172.21.15.113:0/62223124 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54100af070 con 0x7f542806ff20 2024-03-04T07:33:56.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f5424a00640 1 -- 172.21.15.113:0/62223124 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54100aaa40 con 0x7f542806ff20 2024-03-04T07:33:56.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f542d200640 1 -- 172.21.15.113:0/62223124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f542806ff20 msgr2=0x7f54280ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:56.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f542d200640 1 --2- 172.21.15.113:0/62223124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f542806ff20 0x7f54280ce660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5428067cd0 tx=0x7f541001c520 comp rx=0 tx=0).stop 2024-03-04T07:33:56.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f542d200640 1 -- 172.21.15.113:0/62223124 shutdown_connections 2024-03-04T07:33:56.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f542d200640 1 --2- 172.21.15.113:0/62223124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f542806ff20 0x7f54280ce660 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f542d200640 1 --2- 172.21.15.113:0/62223124 >> v2:172.21.15.136:3300/0 conn(0x7f5428152a50 0x7f54280ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f542d200640 1 -- 172.21.15.113:0/62223124 >> 172.21.15.113:0/62223124 conn(0x7f54280734d0 msgr2=0x7f5428066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:56.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f542d200640 1 -- 172.21.15.113:0/62223124 shutdown_connections 2024-03-04T07:33:56.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f542d200640 1 -- 172.21.15.113:0/62223124 wait complete. 2024-03-04T07:33:56.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.698+0000 7f542d200640 1 Processor -- start 2024-03-04T07:33:56.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f542d200640 1 -- start start 2024-03-04T07:33:56.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f542d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f542806ff20 0x7f54281570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:56.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f542d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54280c8b90 0x7f5428162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:56.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f542d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5428152a50 0x7f542806c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:56.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f542d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5428165a00 con 0x7f5428152a50 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f542d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5428165880 con 0x7f542806ff20 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f542d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5428165b80 con 0x7f54280c8b90 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5426800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5428152a50 0x7f542806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5426800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5428152a50 0x7f542806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50580/0 (socket says 172.21.15.113:50580) 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5426800640 1 -- 172.21.15.113:0/3940916655 learned_addr learned my addr 172.21.15.113:0/3940916655 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5425400640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54280c8b90 0x7f5428162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5425e00640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f542806ff20 0x7f54281570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5426800640 1 -- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54280c8b90 msgr2=0x7f5428162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5426800640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54280c8b90 0x7f5428162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5426800640 1 -- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f542806ff20 msgr2=0x7f54281570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5426800640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f542806ff20 0x7f54281570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5426800640 1 -- 172.21.15.113:0/3940916655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5428157ad0 con 0x7f5428152a50 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5425400640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54280c8b90 0x7f5428162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:56.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f5426800640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5428152a50 0x7f542806c170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f542000c7b0 tx=0x7f542000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:56.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f541e000640 1 -- 172.21.15.113:0/3940916655 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420003db0 con 0x7f5428152a50 2024-03-04T07:33:56.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f541e000640 1 -- 172.21.15.113:0/3940916655 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5420005070 con 0x7f5428152a50 2024-03-04T07:33:56.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f542d200640 1 -- 172.21.15.113:0/3940916655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542806dbd0 con 0x7f5428152a50 2024-03-04T07:33:56.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f541e000640 1 -- 172.21.15.113:0/3940916655 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420005460 con 0x7f5428152a50 2024-03-04T07:33:56.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f542d200640 1 -- 172.21.15.113:0/3940916655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542806d8f0 con 0x7f5428152a50 2024-03-04T07:33:56.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f542d200640 1 -- 172.21.15.113:0/3940916655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5428075c00 con 0x7f5428152a50 2024-03-04T07:33:56.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.702+0000 7f541e000640 1 -- 172.21.15.113:0/3940916655 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5420005600 con 0x7f5428152a50 2024-03-04T07:33:56.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.706+0000 7f541e000640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5404066f60 0x7f5404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:56.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.706+0000 7f541e000640 1 -- 172.21.15.113:0/3940916655 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5420084fb0 con 0x7f5428152a50 2024-03-04T07:33:56.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.706+0000 7f5425e00640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5404066f60 0x7f5404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:56.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.706+0000 7f5425e00640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5404066f60 0x7f5404069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f541800b9d0 tx=0x7f5418008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:56.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.710+0000 7f541e000640 1 -- 172.21.15.113:0/3940916655 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5420055fb0 con 0x7f5428152a50 2024-03-04T07:33:56.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 -- 172.21.15.113:0/3940916655 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54280c3680 con 0x7f5428152a50 2024-03-04T07:33:56.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f541e000640 1 -- 172.21.15.113:0/3940916655 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16654 (secure 0 0 0) 0x7f54200558b0 con 0x7f5428152a50 2024-03-04T07:33:56.869 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:33:56.869 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:55.904524+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:33:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 -- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5404066f60 msgr2=0x7f5404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5404066f60 0x7f5404069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f541800b9d0 tx=0x7f5418008040 comp rx=0 tx=0).stop 2024-03-04T07:33:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 -- 172.21.15.113:0/3940916655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5428152a50 msgr2=0x7f542806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5428152a50 0x7f542806c170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f542000c7b0 tx=0x7f542000cc70 comp rx=0 tx=0).stop 2024-03-04T07:33:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 -- 172.21.15.113:0/3940916655 shutdown_connections 2024-03-04T07:33:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5404066f60 0x7f5404069420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5428152a50 0x7f542806c170 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54280c8b90 0x7f5428162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 --2- 172.21.15.113:0/3940916655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f542806ff20 0x7f54281570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:56.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 -- 172.21.15.113:0/3940916655 >> 172.21.15.113:0/3940916655 conn(0x7f54280734d0 msgr2=0x7f54280c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:56.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 -- 172.21.15.113:0/3940916655 shutdown_connections 2024-03-04T07:33:56.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:56.866+0000 7f542d200640 1 -- 172.21.15.113:0/3940916655 wait complete. 2024-03-04T07:33:56.890 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:33:57.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f210b000640 1 Processor -- start 2024-03-04T07:33:57.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f210b000640 1 -- start start 2024-03-04T07:33:57.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f210b000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2104158010 0x7f210406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f210b000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21040ca0e0 con 0x7f21040ce120 2024-03-04T07:33:57.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f210b000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210415aa30 con 0x7f210406ff20 2024-03-04T07:33:57.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f210b000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210406f450 con 0x7f2104158010 2024-03-04T07:33:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2109000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f21040ce120 0x7f21041652d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36870/0 (socket says 172.21.15.113:36870) 2024-03-04T07:33:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2109000640 1 -- 172.21.15.113:0/1882027588 learned_addr learned my addr 172.21.15.113:0/1882027588 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:33:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2103e00640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2104158010 0x7f210406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986452890 0 0) 0x7f21040ca0e0 con 0x7f21040ce120 2024-03-04T07:33:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e8003530 con 0x7f21040ce120 2024-03-04T07:33:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342311742 0 0) 0x7f210415aa30 con 0x7f210406ff20 2024-03-04T07:33:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21040ca0e0 con 0x7f210406ff20 2024-03-04T07:33:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2219771446 0 0) 0x7f20e8003530 con 0x7f21040ce120 2024-03-04T07:33:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2104158010 msgr2=0x7f210406dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2104158010 0x7f210406dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 >> v1:172.21.15.136:6789/0 conn(0x7f210406ff20 legacy=0x7f210415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210406c590 con 0x7f21040ce120 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f20fc001a00 con 0x7f21040ce120 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 >> v1:172.21.15.113:6789/0 conn(0x7f21040ce120 legacy=0x7f21041652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2104158010 0x7f20e8003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20e8004100 0x7f20e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20e8004d50 0x7f20e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210406f450 con 0x7f20e8004100 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e800bab0 con 0x7f20e8004d50 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.022+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e800bcb0 con 0x7f2104158010 2024-03-04T07:33:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2103400640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20e8004100 0x7f20e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2109000640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20e8004d50 0x7f20e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2103e00640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2104158010 0x7f20e8003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2103400640 1 -- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2104158010 msgr2=0x7f20e8003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:57.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2103400640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2104158010 0x7f20e8003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2103400640 1 -- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20e8004d50 msgr2=0x7f20e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:57.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2103400640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20e8004d50 0x7f20e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2103400640 1 -- 172.21.15.113:0/1882027588 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e8003530 con 0x7f20e8004100 2024-03-04T07:33:57.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2103400640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20e8004100 0x7f20e8004810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f21040cf250 tx=0x7f20ec01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6f0649a852f0d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:57.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec001520 con 0x7f20e8004100 2024-03-04T07:33:57.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec0052f0 con 0x7f20e8004100 2024-03-04T07:33:57.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.026+0000 7f2102a00640 1 -- 172.21.15.113:0/1882027588 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0a8930 con 0x7f20e8004100 2024-03-04T07:33:57.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 -- 172.21.15.113:0/1882027588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20e8004100 msgr2=0x7f20e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 --2- 172.21.15.113:0/1882027588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20e8004100 0x7f20e8004810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f21040cf250 tx=0x7f20ec01ecd0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2109000640 1 -- 172.21.15.113:0/1882027588 reap_dead start 2024-03-04T07:33:57.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 -- 172.21.15.113:0/1882027588 shutdown_connections 2024-03-04T07:33:57.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 -- 172.21.15.113:0/1882027588 >> 172.21.15.113:0/1882027588 conn(0x7f21040734d0 msgr2=0x7f2104066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:57.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 -- 172.21.15.113:0/1882027588 shutdown_connections 2024-03-04T07:33:57.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 -- 172.21.15.113:0/1882027588 wait complete. 2024-03-04T07:33:57.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 Processor -- start 2024-03-04T07:33:57.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 -- start start 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20e8004100 0x7f21040cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21040af210 0x7f21040b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21040b25f0 0x7f21040b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210406f980 con 0x7f21040b25f0 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210415ad80 con 0x7f20e8004100 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f210b000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210415b920 con 0x7f21040af210 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2109000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21040b25f0 0x7f21040b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2109000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21040b25f0 0x7f21040b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50592/0 (socket says 172.21.15.113:50592) 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2109000640 1 -- 172.21.15.113:0/1072604964 learned_addr learned my addr 172.21.15.113:0/1072604964 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2103e00640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20e8004100 0x7f21040cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2103400640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21040af210 0x7f21040b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2109000640 1 -- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21040af210 msgr2=0x7f21040b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2109000640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21040af210 0x7f21040b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2109000640 1 -- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20e8004100 msgr2=0x7f21040cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2109000640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20e8004100 0x7f21040cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2109000640 1 -- 172.21.15.113:0/1072604964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21041f1940 con 0x7f21040b25f0 2024-03-04T07:33:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2103e00640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20e8004100 0x7f21040cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2103400640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21040af210 0x7f21040b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:33:57.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f2109000640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21040b25f0 0x7f21040b0e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2104070570 tx=0x7f20fc007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:57.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.030+0000 7f20fbe00640 1 -- 172.21.15.113:0/1072604964 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00eb00 con 0x7f21040b25f0 2024-03-04T07:33:57.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.034+0000 7f20fbe00640 1 -- 172.21.15.113:0/1072604964 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc018070 con 0x7f21040b25f0 2024-03-04T07:33:57.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.034+0000 7f210b000640 1 -- 172.21.15.113:0/1072604964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2104150560 con 0x7f21040b25f0 2024-03-04T07:33:57.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.034+0000 7f20fbe00640 1 -- 172.21.15.113:0/1072604964 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc013420 con 0x7f21040b25f0 2024-03-04T07:33:57.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.034+0000 7f210b000640 1 -- 172.21.15.113:0/1072604964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21041f0dc0 con 0x7f21040b25f0 2024-03-04T07:33:57.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.034+0000 7f210b000640 1 -- 172.21.15.113:0/1072604964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2104075c00 con 0x7f21040b25f0 2024-03-04T07:33:57.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.034+0000 7f20fbe00640 1 -- 172.21.15.113:0/1072604964 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20fc0135c0 con 0x7f21040b25f0 2024-03-04T07:33:57.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.034+0000 7f20fbe00640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20e0066f60 0x7f20e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.034+0000 7f20fbe00640 1 -- 172.21.15.113:0/1072604964 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20fc085320 con 0x7f21040b25f0 2024-03-04T07:33:57.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.034+0000 7f2103e00640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20e0066f60 0x7f20e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.038+0000 7f2103e00640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20e0066f60 0x7f20e0069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009a10 tx=0x7f20f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:57.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.038+0000 7f20fbe00640 1 -- 172.21.15.113:0/1072604964 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f20fc056330 con 0x7f21040b25f0 2024-03-04T07:33:57.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 -- 172.21.15.113:0/1072604964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21040b0b70 con 0x7f21040b25f0 2024-03-04T07:33:57.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f20fbe00640 1 -- 172.21.15.113:0/1072604964 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16654 (secure 0 0 0) 0x7f20fc082120 con 0x7f21040b25f0 2024-03-04T07:33:57.197 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:33:57.197 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:55.904524+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:33:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 -- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20e0066f60 msgr2=0x7f20e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20e0066f60 0x7f20e0069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009a10 tx=0x7f20f4008040 comp rx=0 tx=0).stop 2024-03-04T07:33:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 -- 172.21.15.113:0/1072604964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21040b25f0 msgr2=0x7f21040b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21040b25f0 0x7f21040b0e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2104070570 tx=0x7f20fc007bd0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 -- 172.21.15.113:0/1072604964 shutdown_connections 2024-03-04T07:33:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20e0066f60 0x7f20e0069420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21040b25f0 0x7f21040b0e10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21040af210 0x7f21040b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 --2- 172.21.15.113:0/1072604964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20e8004100 0x7f21040cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 -- 172.21.15.113:0/1072604964 >> 172.21.15.113:0/1072604964 conn(0x7f21040734d0 msgr2=0x7f21040cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:57.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 -- 172.21.15.113:0/1072604964 shutdown_connections 2024-03-04T07:33:57.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.194+0000 7f210b000640 1 -- 172.21.15.113:0/1072604964 wait complete. 2024-03-04T07:33:57.217 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-04T07:33:57.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.350+0000 7f717a200640 1 Processor -- start 2024-03-04T07:33:57.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.350+0000 7f717a200640 1 -- start start 2024-03-04T07:33:57.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f717a200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f71740cc500 0x7f71740c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f717a200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71741588b0 con 0x7f7174073360 2024-03-04T07:33:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f717a200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7174158b70 con 0x7f71740cc8e0 2024-03-04T07:33:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f717a200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7174158e50 con 0x7f71740cc500 2024-03-04T07:33:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f7173400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f71740cc500 0x7f71740c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f7173400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f71740cc500 0x7f71740c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56912/0 (socket says 172.21.15.113:56912) 2024-03-04T07:33:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f7173400640 1 -- 172.21.15.113:0/1061346370 learned_addr learned my addr 172.21.15.113:0/1061346370 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f716aa00640 1 --1- 172.21.15.113:0/1061346370 >> v1:172.21.15.113:6789/0 conn(0x7f7174073360 0x7f7174070f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49458/0 (socket says 172.21.15.113:49458) 2024-03-04T07:33:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f7173400640 1 -- 172.21.15.113:0/1061346370 >> v1:172.21.15.136:6789/0 conn(0x7f71740cc8e0 legacy=0x7f71741524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f7173400640 1 -- 172.21.15.113:0/1061346370 >> v1:172.21.15.113:6789/0 conn(0x7f7174073360 legacy=0x7f7174070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.354+0000 7f7173400640 1 -- 172.21.15.113:0/1061346370 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7174159130 con 0x7f71740cc500 2024-03-04T07:33:57.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f7173400640 1 --2- 172.21.15.113:0/1061346370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71740cc500 0x7f71740c7480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7164009870 tx=0x7f716401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aab7b33159d64cf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:57.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f7172a00640 1 -- 172.21.15.113:0/1061346370 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640b5070 con 0x7f71740cc500 2024-03-04T07:33:57.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f7172a00640 1 -- 172.21.15.113:0/1061346370 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71640a8a50 con 0x7f71740cc500 2024-03-04T07:33:57.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f7172a00640 1 -- 172.21.15.113:0/1061346370 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640a8e20 con 0x7f71740cc500 2024-03-04T07:33:57.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 -- 172.21.15.113:0/1061346370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71740cc500 msgr2=0x7f71740c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 --2- 172.21.15.113:0/1061346370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71740cc500 0x7f71740c7480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7164009870 tx=0x7f716401ed50 comp rx=0 tx=0).stop 2024-03-04T07:33:57.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 -- 172.21.15.113:0/1061346370 shutdown_connections 2024-03-04T07:33:57.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 --2- 172.21.15.113:0/1061346370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71740cc500 0x7f71740c7480 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 -- 172.21.15.113:0/1061346370 >> 172.21.15.113:0/1061346370 conn(0x7f7174066db0 msgr2=0x7f71740671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:57.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 -- 172.21.15.113:0/1061346370 shutdown_connections 2024-03-04T07:33:57.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 -- 172.21.15.113:0/1061346370 wait complete. 2024-03-04T07:33:57.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 Processor -- start 2024-03-04T07:33:57.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 -- start start 2024-03-04T07:33:57.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7174073360 0x7f71741ee350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71740cc500 0x7f717406e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71740cc8e0 0x7f71741e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71740717b0 con 0x7f7174073360 2024-03-04T07:33:57.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7174071630 con 0x7f71740cc500 2024-03-04T07:33:57.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f717a200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71741f2090 con 0x7f71740cc8e0 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.358+0000 7f7173400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7174073360 0x7f71741ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f7173400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7174073360 0x7f71741ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56918/0 (socket says 172.21.15.113:56918) 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f7173400640 1 -- 172.21.15.113:0/1235107381 learned_addr learned my addr 172.21.15.113:0/1235107381 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f7173e00640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71740cc8e0 0x7f71741e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f716aa00640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71740cc500 0x7f717406e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f7173400640 1 -- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71740cc8e0 msgr2=0x7f71741e4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f7173400640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71740cc8e0 0x7f71741e4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f7173400640 1 -- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71740cc500 msgr2=0x7f717406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f7173400640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71740cc500 0x7f717406e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f7173400640 1 -- 172.21.15.113:0/1235107381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71740c3890 con 0x7f7174073360 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f716aa00640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71740cc500 0x7f717406e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f7173400640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7174073360 0x7f71741ee350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71640b3040 tx=0x7f716401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f716be00640 1 -- 172.21.15.113:0/1235107381 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640c0070 con 0x7f7174073360 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f716be00640 1 -- 172.21.15.113:0/1235107381 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71640bb660 con 0x7f7174073360 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f716be00640 1 -- 172.21.15.113:0/1235107381 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71640b5070 con 0x7f7174073360 2024-03-04T07:33:57.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f717a200640 1 -- 172.21.15.113:0/1235107381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71740c3fc0 con 0x7f7174073360 2024-03-04T07:33:57.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f717a200640 1 -- 172.21.15.113:0/1235107381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71741f1a40 con 0x7f7174073360 2024-03-04T07:33:57.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f717a200640 1 -- 172.21.15.113:0/1235107381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7174075c00 con 0x7f7174073360 2024-03-04T07:33:57.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f716be00640 1 -- 172.21.15.113:0/1235107381 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71640a8910 con 0x7f7174073360 2024-03-04T07:33:57.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f716be00640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7150066f90 0x7f7150069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:57.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.362+0000 7f716be00640 1 -- 172.21.15.113:0/1235107381 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71641227a0 con 0x7f7174073360 2024-03-04T07:33:57.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.366+0000 7f716aa00640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7150066f90 0x7f7150069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:57.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.366+0000 7f716aa00640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7150066f90 0x7f7150069450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f717406ef60 tx=0x7f715c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:57.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.366+0000 7f716be00640 1 -- 172.21.15.113:0/1235107381 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f71640188b0 con 0x7f7174073360 2024-03-04T07:33:57.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:57.562+0000 7f717a200640 1 -- 172.21.15.113:0/1235107381 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f717406ba20 con 0x7f7174073360 2024-03-04T07:33:58.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.538+0000 7f716be00640 1 -- 172.21.15.113:0/1235107381 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f71640f37a0 con 0x7f7174073360 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 -- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7150066f90 msgr2=0x7f7150069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7150066f90 0x7f7150069450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f717406ef60 tx=0x7f715c009040 comp rx=0 tx=0).stop 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 -- 172.21.15.113:0/1235107381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7174073360 msgr2=0x7f71741ee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7174073360 0x7f71741ee350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71640b3040 tx=0x7f716401baf0 comp rx=0 tx=0).stop 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 -- 172.21.15.113:0/1235107381 shutdown_connections 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71740cc8e0 0x7f71741e4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71740cc500 0x7f717406e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7150066f90 0x7f7150069450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 --2- 172.21.15.113:0/1235107381 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7174073360 0x7f71741ee350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 -- 172.21.15.113:0/1235107381 >> 172.21.15.113:0/1235107381 conn(0x7f7174066db0 msgr2=0x7f71740cafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 -- 172.21.15.113:0/1235107381 shutdown_connections 2024-03-04T07:33:58.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.542+0000 7f717a200640 1 -- 172.21.15.113:0/1235107381 wait complete. 2024-03-04T07:33:58.564 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300} 2024-03-04T07:33:58.565 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-03-04T07:33:58.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25bd400640 1 Processor -- start 2024-03-04T07:33:58.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25bd400640 1 -- start start 2024-03-04T07:33:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25bd400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f25b80cc500 0x7f25b80c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25bd400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b8073360 0x7f25b806bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25bd400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b81588b0 con 0x7f25b80cc8e0 2024-03-04T07:33:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25bd400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b8158b70 con 0x7f25b80cc500 2024-03-04T07:33:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25bd400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8158e50 con 0x7f25b8073360 2024-03-04T07:33:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b8073360 0x7f25b806bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b6800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f25b80cc500 0x7f25b80c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b8073360 0x7f25b806bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:46858/0 (socket says 172.21.15.113:46858) 2024-03-04T07:33:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b5e00640 1 -- 172.21.15.113:0/3737032969 learned_addr learned my addr 172.21.15.113:0/3737032969 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b7200640 1 --1- 172.21.15.113:0/3737032969 >> v1:172.21.15.136:6789/0 conn(0x7f25b80cc8e0 0x7f25b8070f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55012/0 (socket says 172.21.15.113:55012) 2024-03-04T07:33:58.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b5400640 1 -- 172.21.15.113:0/3737032969 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291706390 0 0) 0x7f25b81588b0 con 0x7f25b80cc8e0 2024-03-04T07:33:58.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b5400640 1 -- 172.21.15.113:0/3737032969 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f259c003530 con 0x7f25b80cc8e0 2024-03-04T07:33:58.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b5e00640 1 -- 172.21.15.113:0/3737032969 >> v2:172.21.15.136:3300/0 conn(0x7f25b80cc500 msgr2=0x7f25b80c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:58.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b5e00640 1 --2- 172.21.15.113:0/3737032969 >> v2:172.21.15.136:3300/0 conn(0x7f25b80cc500 0x7f25b80c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b5e00640 1 -- 172.21.15.113:0/3737032969 >> v1:172.21.15.136:6789/0 conn(0x7f25b80cc8e0 legacy=0x7f25b8070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.702+0000 7f25b5e00640 1 -- 172.21.15.113:0/3737032969 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b8159130 con 0x7f25b8073360 2024-03-04T07:33:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25b5e00640 1 --2- 172.21.15.113:0/3737032969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b8073360 0x7f25b806bc60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f25a40098a0 tx=0x7f25a401ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c0dabf05b0a6c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25b5400640 1 -- 172.21.15.113:0/3737032969 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40b5070 con 0x7f25b8073360 2024-03-04T07:33:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25b5400640 1 -- 172.21.15.113:0/3737032969 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a40a8a30 con 0x7f25b8073360 2024-03-04T07:33:58.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25b5400640 1 -- 172.21.15.113:0/3737032969 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a40a8e00 con 0x7f25b8073360 2024-03-04T07:33:58.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25bd400640 1 -- 172.21.15.113:0/3737032969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b8073360 msgr2=0x7f25b806bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:58.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25bd400640 1 --2- 172.21.15.113:0/3737032969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b8073360 0x7f25b806bc60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f25a40098a0 tx=0x7f25a401ed30 comp rx=0 tx=0).stop 2024-03-04T07:33:58.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25bd400640 1 -- 172.21.15.113:0/3737032969 shutdown_connections 2024-03-04T07:33:58.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25bd400640 1 --2- 172.21.15.113:0/3737032969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b8073360 0x7f25b806bc60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:58.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25bd400640 1 --2- 172.21.15.113:0/3737032969 >> v2:172.21.15.136:3300/0 conn(0x7f25b80cc500 0x7f25b80c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:58.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25bd400640 1 -- 172.21.15.113:0/3737032969 >> 172.21.15.113:0/3737032969 conn(0x7f25b8066db0 msgr2=0x7f25b80671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:58.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25bd400640 1 -- 172.21.15.113:0/3737032969 shutdown_connections 2024-03-04T07:33:58.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.706+0000 7f25bd400640 1 -- 172.21.15.113:0/3737032969 wait complete. 2024-03-04T07:33:58.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25bd400640 1 Processor -- start 2024-03-04T07:33:58.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25bd400640 1 -- start start 2024-03-04T07:33:58.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25bd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25b8073360 0x7f25b80c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:58.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25bd400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b80cc500 0x7f25b81ee980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:58.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25bd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80cc8e0 0x7f25b81eeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:58.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25bd400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b80717b0 con 0x7f25b80cc8e0 2024-03-04T07:33:58.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25bd400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8071630 con 0x7f25b8073360 2024-03-04T07:33:58.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25bd400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b81ee4f0 con 0x7f25b80cc500 2024-03-04T07:33:58.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80cc8e0 0x7f25b81eeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:58.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80cc8e0 0x7f25b81eeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56934/0 (socket says 172.21.15.113:56934) 2024-03-04T07:33:58.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b7200640 1 -- 172.21.15.113:0/1048081170 learned_addr learned my addr 172.21.15.113:0/1048081170 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:58.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b6800640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25b8073360 0x7f25b80c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:58.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b5e00640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b80cc500 0x7f25b81ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:58.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b7200640 1 -- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b80cc500 msgr2=0x7f25b81ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b7200640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b80cc500 0x7f25b81ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b7200640 1 -- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25b8073360 msgr2=0x7f25b80c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b7200640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25b8073360 0x7f25b80c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b7200640 1 -- 172.21.15.113:0/1048081170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b81f1bc0 con 0x7f25b80cc8e0 2024-03-04T07:33:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25b7200640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80cc8e0 0x7f25b81eeec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f25b000c3c0 tx=0x7f25b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25aea00640 1 -- 172.21.15.113:0/1048081170 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b00074e0 con 0x7f25b80cc8e0 2024-03-04T07:33:58.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25aea00640 1 -- 172.21.15.113:0/1048081170 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b0007680 con 0x7f25b80cc8e0 2024-03-04T07:33:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25aea00640 1 -- 172.21.15.113:0/1048081170 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0004a10 con 0x7f25b80cc8e0 2024-03-04T07:33:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.710+0000 7f25bd400640 1 -- 172.21.15.113:0/1048081170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b81f1ea0 con 0x7f25b80cc8e0 2024-03-04T07:33:58.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.714+0000 7f25bd400640 1 -- 172.21.15.113:0/1048081170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b81f2460 con 0x7f25b80cc8e0 2024-03-04T07:33:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.714+0000 7f25bd400640 1 -- 172.21.15.113:0/1048081170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b8071930 con 0x7f25b80cc8e0 2024-03-04T07:33:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.714+0000 7f25aea00640 1 -- 172.21.15.113:0/1048081170 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25b001a020 con 0x7f25b80cc8e0 2024-03-04T07:33:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.714+0000 7f25aea00640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2598066f60 0x7f2598069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.714+0000 7f25aea00640 1 -- 172.21.15.113:0/1048081170 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25b0084750 con 0x7f25b80cc8e0 2024-03-04T07:33:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.714+0000 7f25b6800640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2598066f60 0x7f2598069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.714+0000 7f25b6800640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2598066f60 0x7f2598069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25a80113e0 tx=0x7f25a800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.718+0000 7f25aea00640 1 -- 172.21.15.113:0/1048081170 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f25b8071930 con 0x7f25b80cc8e0 2024-03-04T07:33:58.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:58.910+0000 7f25bd400640 1 -- 172.21.15.113:0/1048081170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f25b8075c00 con 0x7f25b80cc8e0 2024-03-04T07:33:59.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25aea00640 1 -- 172.21.15.113:0/1048081170 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f25b0004c70 con 0x7f25b80cc8e0 2024-03-04T07:33:59.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 -- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2598066f60 msgr2=0x7f2598069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:59.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2598066f60 0x7f2598069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25a80113e0 tx=0x7f25a800b040 comp rx=0 tx=0).stop 2024-03-04T07:33:59.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 -- 172.21.15.113:0/1048081170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80cc8e0 msgr2=0x7f25b81eeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:59.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80cc8e0 0x7f25b81eeec0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f25b000c3c0 tx=0x7f25b000c880 comp rx=0 tx=0).stop 2024-03-04T07:33:59.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 -- 172.21.15.113:0/1048081170 shutdown_connections 2024-03-04T07:33:59.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2598066f60 0x7f2598069420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b80cc8e0 0x7f25b81eeec0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b80cc500 0x7f25b81ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 --2- 172.21.15.113:0/1048081170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25b8073360 0x7f25b80c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 -- 172.21.15.113:0/1048081170 >> 172.21.15.113:0/1048081170 conn(0x7f25b8066db0 msgr2=0x7f25b80ca950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:59.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 -- 172.21.15.113:0/1048081170 shutdown_connections 2024-03-04T07:33:59.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.554+0000 7f25bd400640 1 -- 172.21.15.113:0/1048081170 wait complete. 2024-03-04T07:33:59.579 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-04T07:33:59.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5137200640 1 Processor -- start 2024-03-04T07:33:59.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5137200640 1 -- start start 2024-03-04T07:33:59.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5137200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51300cc500 0x7f51300c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:59.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5137200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5130073360 0x7f513006bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:59.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5137200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51300cc8e0 0x7f5130070f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:59.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5137200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51301588b0 con 0x7f51300cc8e0 2024-03-04T07:33:59.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5137200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5130158b70 con 0x7f5130073360 2024-03-04T07:33:59.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5137200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5130158e50 con 0x7f51300cc500 2024-03-04T07:33:59.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5135200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51300cc8e0 0x7f5130070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:59.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5135200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51300cc8e0 0x7f5130070f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56942/0 (socket says 172.21.15.113:56942) 2024-03-04T07:33:59.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5135200640 1 -- 172.21.15.113:0/3394194736 learned_addr learned my addr 172.21.15.113:0/3394194736 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:59.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f512fe00640 1 --2- 172.21.15.113:0/3394194736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51300cc500 0x7f51300c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:59.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5127400640 1 --2- 172.21.15.113:0/3394194736 >> v2:172.21.15.136:3300/0 conn(0x7f5130073360 0x7f513006bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:59.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5135200640 1 -- 172.21.15.113:0/3394194736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51300cc500 msgr2=0x7f51300c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:59.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5135200640 1 --2- 172.21.15.113:0/3394194736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51300cc500 0x7f51300c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5135200640 1 -- 172.21.15.113:0/3394194736 >> v2:172.21.15.136:3300/0 conn(0x7f5130073360 msgr2=0x7f513006bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:33:59.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5135200640 1 --2- 172.21.15.113:0/3394194736 >> v2:172.21.15.136:3300/0 conn(0x7f5130073360 0x7f513006bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.714+0000 7f5135200640 1 -- 172.21.15.113:0/3394194736 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130159130 con 0x7f51300cc8e0 2024-03-04T07:33:59.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f5135200640 1 --2- 172.21.15.113:0/3394194736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 0x7f5130070f20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51300c9b80 tx=0x7f512801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ab40e4c5b6fd9c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:59.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f512f400640 1 -- 172.21.15.113:0/3394194736 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280b5070 con 0x7f51300cc8e0 2024-03-04T07:33:59.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f512f400640 1 -- 172.21.15.113:0/3394194736 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51280a89d0 con 0x7f51300cc8e0 2024-03-04T07:33:59.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f512f400640 1 -- 172.21.15.113:0/3394194736 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280a8da0 con 0x7f51300cc8e0 2024-03-04T07:33:59.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f5137200640 1 -- 172.21.15.113:0/3394194736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 msgr2=0x7f5130070f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:59.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f5137200640 1 --2- 172.21.15.113:0/3394194736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 0x7f5130070f20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51300c9b80 tx=0x7f512801ed10 comp rx=0 tx=0).stop 2024-03-04T07:33:59.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f5137200640 1 -- 172.21.15.113:0/3394194736 shutdown_connections 2024-03-04T07:33:59.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f5137200640 1 --2- 172.21.15.113:0/3394194736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 0x7f5130070f20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f5137200640 1 --2- 172.21.15.113:0/3394194736 >> v2:172.21.15.136:3300/0 conn(0x7f5130073360 0x7f513006bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f5137200640 1 --2- 172.21.15.113:0/3394194736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51300cc500 0x7f51300c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f5137200640 1 -- 172.21.15.113:0/3394194736 >> 172.21.15.113:0/3394194736 conn(0x7f5130066db0 msgr2=0x7f51300671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:33:59.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f5137200640 1 -- 172.21.15.113:0/3394194736 shutdown_connections 2024-03-04T07:33:59.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.718+0000 7f5137200640 1 -- 172.21.15.113:0/3394194736 wait complete. 2024-03-04T07:33:59.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 Processor -- start 2024-03-04T07:33:59.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 -- start start 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5130073360 0x7f51300c2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51300cc500 0x7f51300c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 0x7f51300c3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51300715e0 con 0x7f51300cc8e0 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130071460 con 0x7f51300cc500 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5130071760 con 0x7f5130073360 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5135200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 0x7f51300c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f512fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5130073360 0x7f51300c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5135200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 0x7f51300c3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56958/0 (socket says 172.21.15.113:56958) 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5135200640 1 -- 172.21.15.113:0/1667107022 learned_addr learned my addr 172.21.15.113:0/1667107022 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5127400640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51300cc500 0x7f51300c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:59.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5135200640 1 -- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5130073360 msgr2=0x7f51300c2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5135200640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5130073360 0x7f51300c2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5135200640 1 -- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51300cc500 msgr2=0x7f51300c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:33:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5135200640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51300cc500 0x7f51300c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:33:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5135200640 1 -- 172.21.15.113:0/1667107022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51301582f0 con 0x7f51300cc8e0 2024-03-04T07:33:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5135200640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 0x7f51300c3970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f51280b3040 tx=0x7f512801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f512cc00640 1 -- 172.21.15.113:0/1667107022 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280b5070 con 0x7f51300cc8e0 2024-03-04T07:33:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f512cc00640 1 -- 172.21.15.113:0/1667107022 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51280b0a10 con 0x7f51300cc8e0 2024-03-04T07:33:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 -- 172.21.15.113:0/1667107022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51300c54c0 con 0x7f51300cc8e0 2024-03-04T07:33:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f512cc00640 1 -- 172.21.15.113:0/1667107022 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280b0e00 con 0x7f51300cc8e0 2024-03-04T07:33:59.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 -- 172.21.15.113:0/1667107022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51301f0610 con 0x7f51300cc8e0 2024-03-04T07:33:59.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.722+0000 7f5137200640 1 -- 172.21.15.113:0/1667107022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130075c00 con 0x7f51300cc8e0 2024-03-04T07:33:59.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.726+0000 7f512cc00640 1 -- 172.21.15.113:0/1667107022 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51280c2060 con 0x7f51300cc8e0 2024-03-04T07:33:59.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.726+0000 7f512cc00640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f510c066f60 0x7f510c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:33:59.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.726+0000 7f512cc00640 1 -- 172.21.15.113:0/1667107022 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5128122180 con 0x7f51300cc8e0 2024-03-04T07:33:59.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.726+0000 7f512fe00640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f510c066f60 0x7f510c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:33:59.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.726+0000 7f512fe00640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f510c066f60 0x7f510c069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5120007a60 tx=0x7f5120008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:33:59.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.730+0000 7f512cc00640 1 -- 172.21.15.113:0/1667107022 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5128018870 con 0x7f51300cc8e0 2024-03-04T07:33:59.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:33:59.918+0000 7f5137200640 1 -- 172.21.15.113:0/1667107022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f513006ba20 con 0x7f51300cc8e0 2024-03-04T07:34:00.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f512cc00640 1 -- 172.21.15.113:0/1667107022 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51280f30f0 con 0x7f51300cc8e0 2024-03-04T07:34:00.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 -- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f510c066f60 msgr2=0x7f510c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:00.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f510c066f60 0x7f510c069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5120007a60 tx=0x7f5120008040 comp rx=0 tx=0).stop 2024-03-04T07:34:00.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 -- 172.21.15.113:0/1667107022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 msgr2=0x7f51300c3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:00.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 0x7f51300c3970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f51280b3040 tx=0x7f512801bab0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 -- 172.21.15.113:0/1667107022 shutdown_connections 2024-03-04T07:34:00.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f510c066f60 0x7f510c069420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51300cc8e0 0x7f51300c3970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51300cc500 0x7f51300c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 --2- 172.21.15.113:0/1667107022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5130073360 0x7f51300c2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 -- 172.21.15.113:0/1667107022 >> 172.21.15.113:0/1667107022 conn(0x7f5130066db0 msgr2=0x7f51300b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:00.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 -- 172.21.15.113:0/1667107022 shutdown_connections 2024-03-04T07:34:00.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.562+0000 7f5137200640 1 -- 172.21.15.113:0/1667107022 wait complete. 2024-03-04T07:34:00.586 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-04T07:34:00.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6583a00640 1 Processor -- start 2024-03-04T07:34:00.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6583a00640 1 -- start start 2024-03-04T07:34:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6583a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f657c1519b0 0x7f657c0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6583a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c1577d0 con 0x7f657c157cd0 2024-03-04T07:34:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6583a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657c0c7760 con 0x7f657c0c19e0 2024-03-04T07:34:00.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6583a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657c0c7a40 con 0x7f657c1519b0 2024-03-04T07:34:00.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6581a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f657c157cd0 0x7f657c158100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49462/0 (socket says 172.21.15.113:49462) 2024-03-04T07:34:00.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6581a00640 1 -- 172.21.15.113:0/2064675220 learned_addr learned my addr 172.21.15.113:0/2064675220 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:00.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6581000640 1 --2- 172.21.15.113:0/2064675220 >> v2:172.21.15.136:3300/0 conn(0x7f657c1519b0 0x7f657c0c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111642699 0 0) 0x7f657c1577d0 con 0x7f657c157cd0 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6564005740 con 0x7f657c157cd0 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2747087405 0 0) 0x7f6564005740 con 0x7f657c157cd0 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 >> v2:172.21.15.136:3300/0 conn(0x7f657c1519b0 msgr2=0x7f657c0c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 --2- 172.21.15.113:0/2064675220 >> v2:172.21.15.136:3300/0 conn(0x7f657c1519b0 0x7f657c0c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 >> v1:172.21.15.136:6789/0 conn(0x7f657c0c19e0 legacy=0x7f657c0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0c1090 con 0x7f657c157cd0 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f6574001a00 con 0x7f657c157cd0 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 >> v1:172.21.15.113:6789/0 conn(0x7f657c157cd0 legacy=0x7f657c158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6564005bd0 0x7f6564005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6564006a20 0x7f6564006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65640073e0 0x7f656400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0c7a40 con 0x7f65640073e0 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656400e280 con 0x7f6564006a20 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656400e480 con 0x7f6564005bd0 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6581a00640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65640073e0 0x7f656400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6581000640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6564005bd0 0x7f6564005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657be00640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6564006a20 0x7f6564006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6581a00640 1 -- 172.21.15.113:0/2064675220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6564005bd0 msgr2=0x7f6564005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6581a00640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6564005bd0 0x7f6564005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6581a00640 1 -- 172.21.15.113:0/2064675220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6564006a20 msgr2=0x7f6564006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6581a00640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6564006a20 0x7f6564006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f6581a00640 1 -- 172.21.15.113:0/2064675220 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6564005740 con 0x7f65640073e0 2024-03-04T07:34:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.722+0000 7f657be00640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6564006a20 0x7f6564006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:00.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f6581a00640 1 -- 172.21.15.113:0/2064675220 reap_dead start 2024-03-04T07:34:00.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f6581a00640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65640073e0 0x7f656400dbe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f657400a840 tx=0x7f657401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d510d7d7b555577b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:00.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657400a460 con 0x7f65640073e0 2024-03-04T07:34:00.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65740ac640 con 0x7f65640073e0 2024-03-04T07:34:00.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f657b400640 1 -- 172.21.15.113:0/2064675220 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65740aca30 con 0x7f65640073e0 2024-03-04T07:34:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f6583a00640 1 -- 172.21.15.113:0/2064675220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65640073e0 msgr2=0x7f656400dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f6583a00640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65640073e0 0x7f656400dbe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f657400a840 tx=0x7f657401d360 comp rx=0 tx=0).stop 2024-03-04T07:34:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f6583a00640 1 -- 172.21.15.113:0/2064675220 shutdown_connections 2024-03-04T07:34:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f6583a00640 1 --2- 172.21.15.113:0/2064675220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65640073e0 0x7f656400dbe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f6583a00640 1 -- 172.21.15.113:0/2064675220 >> 172.21.15.113:0/2064675220 conn(0x7f657c066db0 msgr2=0x7f657c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f6583a00640 1 -- 172.21.15.113:0/2064675220 shutdown_connections 2024-03-04T07:34:00.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.726+0000 7f6583a00640 1 -- 172.21.15.113:0/2064675220 wait complete. 2024-03-04T07:34:00.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6583a00640 1 Processor -- start 2024-03-04T07:34:00.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6583a00640 1 -- start start 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6583a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65640073e0 0x7f657c0c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6583a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f657c1542a0 0x7f657c145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6583a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f657c146910 0x7f657c146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6583a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0c1650 con 0x7f657c1542a0 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6583a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0c75c0 con 0x7f657c146910 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6583a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c0c7dd0 con 0x7f65640073e0 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f657be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f657c1542a0 0x7f657c145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f657be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f657c1542a0 0x7f657c145130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56980/0 (socket says 172.21.15.113:56980) 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f657be00640 1 -- 172.21.15.113:0/4114920747 learned_addr learned my addr 172.21.15.113:0/4114920747 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6581a00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f657c146910 0x7f657c146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:00.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6581000640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65640073e0 0x7f657c0c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f657be00640 1 -- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65640073e0 msgr2=0x7f657c0c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f657be00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65640073e0 0x7f657c0c1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f657be00640 1 -- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f657c146910 msgr2=0x7f657c146d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f657be00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f657c146910 0x7f657c146d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f657be00640 1 -- 172.21.15.113:0/4114920747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c1472d0 con 0x7f657c1542a0 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f657be00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f657c1542a0 0x7f657c145130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f656800e440 tx=0x7f656800e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6578c00640 1 -- 172.21.15.113:0/4114920747 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568019070 con 0x7f657c1542a0 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6578c00640 1 -- 172.21.15.113:0/4114920747 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65680052e0 con 0x7f657c1542a0 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6583a00640 1 -- 172.21.15.113:0/4114920747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c1ec240 con 0x7f657c1542a0 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6578c00640 1 -- 172.21.15.113:0/4114920747 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568003840 con 0x7f657c1542a0 2024-03-04T07:34:00.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.730+0000 7f6583a00640 1 -- 172.21.15.113:0/4114920747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c1ec800 con 0x7f657c1542a0 2024-03-04T07:34:00.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.734+0000 7f6583a00640 1 -- 172.21.15.113:0/4114920747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c075c00 con 0x7f657c1542a0 2024-03-04T07:34:00.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.734+0000 7f6578c00640 1 -- 172.21.15.113:0/4114920747 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f656801c070 con 0x7f657c1542a0 2024-03-04T07:34:00.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.734+0000 7f6578c00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f655c066f90 0x7f655c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:00.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.734+0000 7f6578c00640 1 -- 172.21.15.113:0/4114920747 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6568084570 con 0x7f657c1542a0 2024-03-04T07:34:00.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.734+0000 7f6581000640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f655c066f90 0x7f655c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:00.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.734+0000 7f6581000640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f655c066f90 0x7f655c069450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f65700099b0 tx=0x7f6570008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:00.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.738+0000 7f6578c00640 1 -- 172.21.15.113:0/4114920747 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f657c1472d0 con 0x7f657c1542a0 2024-03-04T07:34:00.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:00.926+0000 7f6583a00640 1 -- 172.21.15.113:0/4114920747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f657c073280 con 0x7f657c1542a0 2024-03-04T07:34:01.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.570+0000 7f6578c00640 1 -- 172.21.15.113:0/4114920747 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) v1 ==== 115+0+0 (secure 0 0 0) 0x7f6568082e20 con 0x7f657c1542a0 2024-03-04T07:34:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 -- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f655c066f90 msgr2=0x7f655c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f655c066f90 0x7f655c069450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f65700099b0 tx=0x7f6570008040 comp rx=0 tx=0).stop 2024-03-04T07:34:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 -- 172.21.15.113:0/4114920747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f657c1542a0 msgr2=0x7f657c145130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f657c1542a0 0x7f657c145130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f656800e440 tx=0x7f656800e900 comp rx=0 tx=0).stop 2024-03-04T07:34:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 -- 172.21.15.113:0/4114920747 shutdown_connections 2024-03-04T07:34:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f657c146910 0x7f657c146d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f655c066f90 0x7f655c069450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f657c1542a0 0x7f657c145130 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 --2- 172.21.15.113:0/4114920747 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f65640073e0 0x7f657c0c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 -- 172.21.15.113:0/4114920747 >> 172.21.15.113:0/4114920747 conn(0x7f657c066db0 msgr2=0x7f657c0c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 -- 172.21.15.113:0/4114920747 shutdown_connections 2024-03-04T07:34:01.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.574+0000 7f6583a00640 1 -- 172.21.15.113:0/4114920747 wait complete. 2024-03-04T07:34:01.596 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-04T07:34:01.596 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:34:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0543200640 1 Processor -- start 2024-03-04T07:34:01.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0543200640 1 -- start start 2024-03-04T07:34:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0543200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544157df0 0x7f054406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:01.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0543200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05440ce020 0x7f05440ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0543200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0544070fd0 con 0x7f05440c8a90 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0543200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054415e090 con 0x7f05440ce020 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0543200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054406c6b0 con 0x7f0544157df0 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0542800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05440ce020 0x7f05440ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0542800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05440ce020 0x7f05440ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56984/0 (socket says 172.21.15.113:56984) 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0542800640 1 -- 172.21.15.113:0/2103185435 learned_addr learned my addr 172.21.15.113:0/2103185435 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0541e00640 1 --2- 172.21.15.113:0/2103185435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544157df0 0x7f054406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0542800640 1 -- 172.21.15.113:0/2103185435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544157df0 msgr2=0x7f054406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0542800640 1 --2- 172.21.15.113:0/2103185435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544157df0 0x7f054406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0542800640 1 -- 172.21.15.113:0/2103185435 >> v1:172.21.15.136:6789/0 conn(0x7f05440c8a90 legacy=0x7f054415e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:01.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.730+0000 7f0542800640 1 -- 172.21.15.113:0/2103185435 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054406dfd0 con 0x7f05440ce020 2024-03-04T07:34:01.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0542800640 1 --2- 172.21.15.113:0/2103185435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce020 0x7f05440ce460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f053c009e50 tx=0x7f053c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4efa8bdf0a3ce4d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0540a00640 1 -- 172.21.15.113:0/2103185435 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c0a55f0 con 0x7f05440ce020 2024-03-04T07:34:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0540a00640 1 -- 172.21.15.113:0/2103185435 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c0a5790 con 0x7f05440ce020 2024-03-04T07:34:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0540a00640 1 -- 172.21.15.113:0/2103185435 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c0a5b60 con 0x7f05440ce020 2024-03-04T07:34:01.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0543200640 1 -- 172.21.15.113:0/2103185435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce020 msgr2=0x7f05440ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0543200640 1 --2- 172.21.15.113:0/2103185435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce020 0x7f05440ce460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f053c009e50 tx=0x7f053c01ca30 comp rx=0 tx=0).stop 2024-03-04T07:34:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0543200640 1 -- 172.21.15.113:0/2103185435 shutdown_connections 2024-03-04T07:34:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0543200640 1 --2- 172.21.15.113:0/2103185435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce020 0x7f05440ce460 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0543200640 1 --2- 172.21.15.113:0/2103185435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544157df0 0x7f054406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0543200640 1 -- 172.21.15.113:0/2103185435 >> 172.21.15.113:0/2103185435 conn(0x7f0544072f40 msgr2=0x7f05440706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0543200640 1 -- 172.21.15.113:0/2103185435 shutdown_connections 2024-03-04T07:34:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0543200640 1 -- 172.21.15.113:0/2103185435 wait complete. 2024-03-04T07:34:01.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0543200640 1 Processor -- start 2024-03-04T07:34:01.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.734+0000 7f0543200640 1 -- start start 2024-03-04T07:34:01.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0543200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440c8a90 0x7f0544156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:01.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0543200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05440ce020 0x7f0544152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:01.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0543200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544157df0 0x7f0544153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0543200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440c30b0 con 0x7f05440c8a90 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0543200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054415ec30 con 0x7f05440ce020 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0543200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054406dc00 con 0x7f0544157df0 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0541e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440c8a90 0x7f0544156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0541e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440c8a90 0x7f0544156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56998/0 (socket says 172.21.15.113:56998) 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0541e00640 1 -- 172.21.15.113:0/517738416 learned_addr learned my addr 172.21.15.113:0/517738416 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0542800640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544157df0 0x7f0544153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0541400640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05440ce020 0x7f0544152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0541e00640 1 -- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544157df0 msgr2=0x7f0544153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0541e00640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544157df0 0x7f0544153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0541e00640 1 -- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05440ce020 msgr2=0x7f0544152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0541e00640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05440ce020 0x7f0544152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0541e00640 1 -- 172.21.15.113:0/517738416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0544157170 con 0x7f05440c8a90 2024-03-04T07:34:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0541e00640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440c8a90 0x7f0544156740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f053400d830 tx=0x7f053400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f053a000640 1 -- 172.21.15.113:0/517738416 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053400b6c0 con 0x7f05440c8a90 2024-03-04T07:34:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f053a000640 1 -- 172.21.15.113:0/517738416 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0534019070 con 0x7f05440c8a90 2024-03-04T07:34:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f053a000640 1 -- 172.21.15.113:0/517738416 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053400ba60 con 0x7f05440c8a90 2024-03-04T07:34:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0543200640 1 -- 172.21.15.113:0/517738416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0544156c80 con 0x7f05440c8a90 2024-03-04T07:34:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0543200640 1 -- 172.21.15.113:0/517738416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0544157410 con 0x7f05440c8a90 2024-03-04T07:34:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f0543200640 1 -- 172.21.15.113:0/517738416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0544075a80 con 0x7f05440c8a90 2024-03-04T07:34:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.738+0000 7f053a000640 1 -- 172.21.15.113:0/517738416 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f053401e460 con 0x7f05440c8a90 2024-03-04T07:34:01.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.742+0000 7f053a000640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:01.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.742+0000 7f053a000640 1 -- 172.21.15.113:0/517738416 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05340856c0 con 0x7f05440c8a90 2024-03-04T07:34:01.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.742+0000 7f0541400640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.742+0000 7f0541400640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f05440cd550 tx=0x7f0530009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:01.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.746+0000 7f053a000640 1 -- 172.21.15.113:0/517738416 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f05340566c0 con 0x7f05440c8a90 2024-03-04T07:34:01.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 -- 172.21.15.113:0/517738416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05440c33f0 con 0x7f05440c8a90 2024-03-04T07:34:01.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f053a000640 1 -- 172.21.15.113:0/517738416 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6565 (secure 0 0 0) 0x7f0534055fc0 con 0x7f05440c8a90 2024-03-04T07:34:01.937 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:01.937 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4256,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:00.929090+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-03-04T07:34:01.937 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 8 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 -- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 msgr2=0x7f0524069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f05440cd550 tx=0x7f0530009040 comp rx=0 tx=0).stop 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 -- 172.21.15.113:0/517738416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440c8a90 msgr2=0x7f0544156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440c8a90 0x7f0544156740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f053400d830 tx=0x7f053400dcf0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 -- 172.21.15.113:0/517738416 shutdown_connections 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544157df0 0x7f0544153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05440ce020 0x7f0544152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 --2- 172.21.15.113:0/517738416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440c8a90 0x7f0544156740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 -- 172.21.15.113:0/517738416 >> 172.21.15.113:0/517738416 conn(0x7f0544072f40 msgr2=0x7f05440c3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 -- 172.21.15.113:0/517738416 shutdown_connections 2024-03-04T07:34:01.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:01.934+0000 7f0543200640 1 -- 172.21.15.113:0/517738416 wait complete. 2024-03-04T07:34:01.958 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:34:02.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56bcc00640 1 Processor -- start 2024-03-04T07:34:02.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56bcc00640 1 -- start start 2024-03-04T07:34:02.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56bcc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56b814e4c0 0x7f56b8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56bcc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f56b80c22e0 0x7f56b80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56bcc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b80c27f0 con 0x7f56b80c2d80 2024-03-04T07:34:02.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56bcc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b815ad60 con 0x7f56b814e4c0 2024-03-04T07:34:02.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56bcc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b80c2a90 con 0x7f56b80c22e0 2024-03-04T07:34:02.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56b6800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f56b80c2d80 0x7f56b815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49472/0 (socket says 172.21.15.113:49472) 2024-03-04T07:34:02.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56b814e4c0 0x7f56b8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56b6800640 1 -- 172.21.15.113:0/1406280085 learned_addr learned my addr 172.21.15.113:0/1406280085 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:02.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56ad400640 1 --2- 172.21.15.113:0/1406280085 >> v2:172.21.15.136:3300/0 conn(0x7f56b80c22e0 0x7f56b80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.090+0000 7f56b5400640 1 -- 172.21.15.113:0/1406280085 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317995301 0 0) 0x7f56b80c27f0 con 0x7f56b80c2d80 2024-03-04T07:34:02.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56b5400640 1 -- 172.21.15.113:0/1406280085 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f569c003530 con 0x7f56b80c2d80 2024-03-04T07:34:02.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56b5e00640 1 -- 172.21.15.113:0/1406280085 >> v2:172.21.15.136:3300/0 conn(0x7f56b80c22e0 msgr2=0x7f56b80731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:02.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56b5e00640 1 --2- 172.21.15.113:0/1406280085 >> v2:172.21.15.136:3300/0 conn(0x7f56b80c22e0 0x7f56b80731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56b5e00640 1 -- 172.21.15.113:0/1406280085 >> v1:172.21.15.113:6789/0 conn(0x7f56b80c2d80 legacy=0x7f56b815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56b5e00640 1 -- 172.21.15.113:0/1406280085 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b8070c60 con 0x7f56b814e4c0 2024-03-04T07:34:02.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56b5e00640 1 --2- 172.21.15.113:0/1406280085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 0x7f56b8072c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f56b80cd230 tx=0x7f56a801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556021665ee2794d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:02.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56b5400640 1 -- 172.21.15.113:0/1406280085 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80b5070 con 0x7f56b814e4c0 2024-03-04T07:34:02.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56b5400640 1 -- 172.21.15.113:0/1406280085 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a80a8970 con 0x7f56b814e4c0 2024-03-04T07:34:02.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56b5400640 1 -- 172.21.15.113:0/1406280085 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80a8d40 con 0x7f56b814e4c0 2024-03-04T07:34:02.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56bcc00640 1 -- 172.21.15.113:0/1406280085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 msgr2=0x7f56b8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56bcc00640 1 --2- 172.21.15.113:0/1406280085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 0x7f56b8072c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f56b80cd230 tx=0x7f56a801ec70 comp rx=0 tx=0).stop 2024-03-04T07:34:02.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56bcc00640 1 -- 172.21.15.113:0/1406280085 shutdown_connections 2024-03-04T07:34:02.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56bcc00640 1 --2- 172.21.15.113:0/1406280085 >> v2:172.21.15.136:3300/0 conn(0x7f56b80c22e0 0x7f56b80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56bcc00640 1 --2- 172.21.15.113:0/1406280085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 0x7f56b8072c80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56bcc00640 1 -- 172.21.15.113:0/1406280085 >> 172.21.15.113:0/1406280085 conn(0x7f56b806e820 msgr2=0x7f56b806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:02.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56bcc00640 1 -- 172.21.15.113:0/1406280085 shutdown_connections 2024-03-04T07:34:02.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.094+0000 7f56bcc00640 1 -- 172.21.15.113:0/1406280085 wait complete. 2024-03-04T07:34:02.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56bcc00640 1 Processor -- start 2024-03-04T07:34:02.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56bcc00640 1 -- start start 2024-03-04T07:34:02.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56bcc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56b80c22e0 0x7f56b81528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56bcc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56b80c2d80 0x7f56b8153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56bcc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 0x7f56b81f0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56bcc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b8071500 con 0x7f56b814e4c0 2024-03-04T07:34:02.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56bcc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b8071220 con 0x7f56b80c2d80 2024-03-04T07:34:02.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56bcc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b814e8a0 con 0x7f56b80c22e0 2024-03-04T07:34:02.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56b6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 0x7f56b81f0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56b6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 0x7f56b81f0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57026/0 (socket says 172.21.15.113:57026) 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56b6800640 1 -- 172.21.15.113:0/406415116 learned_addr learned my addr 172.21.15.113:0/406415116 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56ad400640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56b80c2d80 0x7f56b8153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56b5e00640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56b80c22e0 0x7f56b81528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56b6800640 1 -- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56b80c22e0 msgr2=0x7f56b81528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56b6800640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56b80c22e0 0x7f56b81528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56b6800640 1 -- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56b80c2d80 msgr2=0x7f56b8153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56b6800640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56b80c2d80 0x7f56b8153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56b6800640 1 -- 172.21.15.113:0/406415116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80b0d30 con 0x7f56b814e4c0 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56b6800640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 0x7f56b81f0fd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f56b000caa0 tx=0x7f56b000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56aea00640 1 -- 172.21.15.113:0/406415116 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0017070 con 0x7f56b814e4c0 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56aea00640 1 -- 172.21.15.113:0/406415116 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b000ac00 con 0x7f56b814e4c0 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56aea00640 1 -- 172.21.15.113:0/406415116 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0012400 con 0x7f56b814e4c0 2024-03-04T07:34:02.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.098+0000 7f56bcc00640 1 -- 172.21.15.113:0/406415116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b81522b0 con 0x7f56b814e4c0 2024-03-04T07:34:02.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.102+0000 7f56bcc00640 1 -- 172.21.15.113:0/406415116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b80b09f0 con 0x7f56b814e4c0 2024-03-04T07:34:02.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.102+0000 7f56bcc00640 1 -- 172.21.15.113:0/406415116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b8075c00 con 0x7f56b814e4c0 2024-03-04T07:34:02.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.102+0000 7f56aea00640 1 -- 172.21.15.113:0/406415116 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56b001a070 con 0x7f56b814e4c0 2024-03-04T07:34:02.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.102+0000 7f56aea00640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5694066f60 0x7f5694069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.102+0000 7f56aea00640 1 -- 172.21.15.113:0/406415116 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56b0084d20 con 0x7f56b814e4c0 2024-03-04T07:34:02.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.102+0000 7f56b5e00640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5694066f60 0x7f5694069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.102+0000 7f56b5e00640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5694066f60 0x7f5694069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f56a80b2000 tx=0x7f56a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:02.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.106+0000 7f56aea00640 1 -- 172.21.15.113:0/406415116 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f56b00556e0 con 0x7f56b814e4c0 2024-03-04T07:34:02.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.262+0000 7f56bcc00640 1 -- 172.21.15.113:0/406415116 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56b8155190 con 0x7f56b814e4c0 2024-03-04T07:34:02.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.262+0000 7f56aea00640 1 -- 172.21.15.113:0/406415116 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f56b0059780 con 0x7f56b814e4c0 2024-03-04T07:34:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:02.268 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:57.517935+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 -- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5694066f60 msgr2=0x7f5694069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5694066f60 0x7f5694069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f56a80b2000 tx=0x7f56a8008040 comp rx=0 tx=0).stop 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 -- 172.21.15.113:0/406415116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 msgr2=0x7f56b81f0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 0x7f56b81f0fd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f56b000caa0 tx=0x7f56b000cf60 comp rx=0 tx=0).stop 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 -- 172.21.15.113:0/406415116 shutdown_connections 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5694066f60 0x7f5694069420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b814e4c0 0x7f56b81f0fd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56b80c2d80 0x7f56b8153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 --2- 172.21.15.113:0/406415116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56b80c22e0 0x7f56b81528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 -- 172.21.15.113:0/406415116 >> 172.21.15.113:0/406415116 conn(0x7f56b806e820 msgr2=0x7f56b80c8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 -- 172.21.15.113:0/406415116 shutdown_connections 2024-03-04T07:34:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.266+0000 7f56bcc00640 1 -- 172.21.15.113:0/406415116 wait complete. 2024-03-04T07:34:02.289 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-04T07:34:02.290 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T07:34:02.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1ebc400640 1 Processor -- start 2024-03-04T07:34:02.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1ebc400640 1 -- start start 2024-03-04T07:34:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1ebc400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1eb40c86e0 0x7f1eb4157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1ebc400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1eb4151c10 0x7f1eb4157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1ebc400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb40cc980 con 0x7f1eb40c8ac0 2024-03-04T07:34:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1ebc400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb40ccc20 con 0x7f1eb4151c10 2024-03-04T07:34:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1ebc400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb40ccec0 con 0x7f1eb40c86e0 2024-03-04T07:34:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1eba400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1eb4151c10 0x7f1eb4157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1eba400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1eb4151c10 0x7f1eb4157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57036/0 (socket says 172.21.15.113:57036) 2024-03-04T07:34:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1eba400640 1 -- 172.21.15.113:0/3475433614 learned_addr learned my addr 172.21.15.113:0/3475433614 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:02.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1eb9a00640 1 --2- 172.21.15.113:0/3475433614 >> v2:172.21.15.136:3300/0 conn(0x7f1eb40c86e0 0x7f1eb4157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1eba400640 1 -- 172.21.15.113:0/3475433614 >> v2:172.21.15.136:3300/0 conn(0x7f1eb40c86e0 msgr2=0x7f1eb4157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:02.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1eba400640 1 --2- 172.21.15.113:0/3475433614 >> v2:172.21.15.136:3300/0 conn(0x7f1eb40c86e0 0x7f1eb4157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1eba400640 1 -- 172.21.15.113:0/3475433614 >> v1:172.21.15.136:6789/0 conn(0x7f1eb40c8ac0 legacy=0x7f1eb4151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.422+0000 7f1eba400640 1 -- 172.21.15.113:0/3475433614 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40cd160 con 0x7f1eb4151c10 2024-03-04T07:34:02.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1eba400640 1 --2- 172.21.15.113:0/3475433614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 0x7f1eb4157ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40cda00 tx=0x7f1eac01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b45139f3b04b56b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:02.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1eb3e00640 1 -- 172.21.15.113:0/3475433614 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0b4070 con 0x7f1eb4151c10 2024-03-04T07:34:02.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1eb3e00640 1 -- 172.21.15.113:0/3475433614 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac0af400 con 0x7f1eb4151c10 2024-03-04T07:34:02.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1eb3e00640 1 -- 172.21.15.113:0/3475433614 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0af7d0 con 0x7f1eb4151c10 2024-03-04T07:34:02.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1ebc400640 1 -- 172.21.15.113:0/3475433614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 msgr2=0x7f1eb4157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1ebc400640 1 --2- 172.21.15.113:0/3475433614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 0x7f1eb4157ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40cda00 tx=0x7f1eac01f1e0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1ebc400640 1 -- 172.21.15.113:0/3475433614 shutdown_connections 2024-03-04T07:34:02.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1ebc400640 1 --2- 172.21.15.113:0/3475433614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 0x7f1eb4157ff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1ebc400640 1 --2- 172.21.15.113:0/3475433614 >> v2:172.21.15.136:3300/0 conn(0x7f1eb40c86e0 0x7f1eb4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1ebc400640 1 -- 172.21.15.113:0/3475433614 >> 172.21.15.113:0/3475433614 conn(0x7f1eb406b060 msgr2=0x7f1eb406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:02.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1ebc400640 1 -- 172.21.15.113:0/3475433614 shutdown_connections 2024-03-04T07:34:02.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.426+0000 7f1ebc400640 1 -- 172.21.15.113:0/3475433614 wait complete. 2024-03-04T07:34:02.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 Processor -- start 2024-03-04T07:34:02.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 -- start start 2024-03-04T07:34:02.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1eb40c86e0 0x7f1eb41483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1eb40c8ac0 0x7f1eb414de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 0x7f1eb4158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40c5940 con 0x7f1eb4151c10 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40c40e0 con 0x7f1eb40c86e0 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40c5ac0 con 0x7f1eb40c8ac0 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eba400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 0x7f1eb4158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eba400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 0x7f1eb4158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57038/0 (socket says 172.21.15.113:57038) 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eba400640 1 -- 172.21.15.113:0/1254632075 learned_addr learned my addr 172.21.15.113:0/1254632075 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eb9a00640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1eb40c86e0 0x7f1eb41483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eb9000640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1eb40c8ac0 0x7f1eb414de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eba400640 1 -- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1eb40c8ac0 msgr2=0x7f1eb414de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eba400640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1eb40c8ac0 0x7f1eb414de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eba400640 1 -- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1eb40c86e0 msgr2=0x7f1eb41483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eba400640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1eb40c86e0 0x7f1eb41483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eba400640 1 -- 172.21.15.113:0/1254632075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb414e830 con 0x7f1eb4151c10 2024-03-04T07:34:02.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eb9000640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1eb40c8ac0 0x7f1eb414de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:02.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eba400640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 0x7f1eb4158900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1eac01ce10 tx=0x7f1eac01bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:02.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eb1600640 1 -- 172.21.15.113:0/1254632075 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0c0070 con 0x7f1eb4151c10 2024-03-04T07:34:02.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eb1600640 1 -- 172.21.15.113:0/1254632075 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac0afcc0 con 0x7f1eb4151c10 2024-03-04T07:34:02.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 -- 172.21.15.113:0/1254632075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb41f1020 con 0x7f1eb4151c10 2024-03-04T07:34:02.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1eb1600640 1 -- 172.21.15.113:0/1254632075 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0b4040 con 0x7f1eb4151c10 2024-03-04T07:34:02.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 -- 172.21.15.113:0/1254632075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb41f3cd0 con 0x7f1eb4151c10 2024-03-04T07:34:02.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.430+0000 7f1ebc400640 1 -- 172.21.15.113:0/1254632075 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb4075c00 con 0x7f1eb4151c10 2024-03-04T07:34:02.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.434+0000 7f1eb1600640 1 -- 172.21.15.113:0/1254632075 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1eac0c3070 con 0x7f1eb4151c10 2024-03-04T07:34:02.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.434+0000 7f1eb1600640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e94066f90 0x7f1e94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.434+0000 7f1eb1600640 1 -- 172.21.15.113:0/1254632075 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1eac122790 con 0x7f1eb4151c10 2024-03-04T07:34:02.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.434+0000 7f1eb9a00640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e94066f90 0x7f1e94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.434+0000 7f1eb9a00640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e94066f90 0x7f1e94069450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009b60 tx=0x7f1ea8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:02.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.438+0000 7f1eb1600640 1 -- 172.21.15.113:0/1254632075 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1eac018d40 con 0x7f1eb4151c10 2024-03-04T07:34:02.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.610+0000 7f1ebc400640 1 -- 172.21.15.113:0/1254632075 --> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1eb41f37a0 con 0x7f1e94066f90 2024-03-04T07:34:02.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.626+0000 7f1eb1600640 1 -- 172.21.15.113:0/1254632075 <== mgr.4103 v2:172.21.15.136:6832/3353359815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251798 (secure 0 0 0) 0x7f1eb41f37a0 con 0x7f1e94066f90 2024-03-04T07:34:02.632 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:02.636 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 -- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e94066f90 msgr2=0x7f1e94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e94066f90 0x7f1e94069450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009b60 tx=0x7f1ea8009190 comp rx=0 tx=0).stop 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 -- 172.21.15.113:0/1254632075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 msgr2=0x7f1eb4158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 0x7f1eb4158900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1eac01ce10 tx=0x7f1eac01bf80 comp rx=0 tx=0).stop 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 -- 172.21.15.113:0/1254632075 shutdown_connections 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e94066f90 0x7f1e94069450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1eb4151c10 0x7f1eb4158900 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1eb40c8ac0 0x7f1eb414de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 --2- 172.21.15.113:0/1254632075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1eb40c86e0 0x7f1eb41483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 -- 172.21.15.113:0/1254632075 >> 172.21.15.113:0/1254632075 conn(0x7f1eb406b060 msgr2=0x7f1eb406b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 -- 172.21.15.113:0/1254632075 shutdown_connections 2024-03-04T07:34:02.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.630+0000 7f1ebc400640 1 -- 172.21.15.113:0/1254632075 wait complete. 2024-03-04T07:34:02.653 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-04T07:34:01.434431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223024,"kb_used_data":2684,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751696,"statfs":{"total":773094113280,"available":772865736704,"internally_reserved":0,"allocated":2748416,"data_stored":1711318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002150"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.565710+0000","last_change":"2024-03-04T07:33:56.559432+0000","last_active":"2024-03-04T07:33:56.565710+0000","last_peered":"2024-03-04T07:33:56.565710+0000","last_clean":"2024-03-04T07:33:56.565710+0000","last_became_active":"2024-03-04T07:33:56.559031+0000","last_became_peered":"2024-03-04T07:33:56.559031+0000","last_unstale":"2024-03-04T07:33:56.565710+0000","last_undegraded":"2024-03-04T07:33:56.565710+0000","last_fullsized":"2024-03-04T07:33:56.565710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:50:27.941814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750282+0000","last_change":"2024-03-04T07:33:56.546822+0000","last_active":"2024-03-04T07:33:57.750282+0000","last_peered":"2024-03-04T07:33:57.750282+0000","last_clean":"2024-03-04T07:33:57.750282+0000","last_became_active":"2024-03-04T07:33:56.546547+0000","last_became_peered":"2024-03-04T07:33:56.546547+0000","last_unstale":"2024-03-04T07:33:57.750282+0000","last_undegraded":"2024-03-04T07:33:57.750282+0000","last_fullsized":"2024-03-04T07:33:57.750282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:06.332337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.541032+0000","last_change":"2024-03-04T07:33:56.530803+0000","last_active":"2024-03-04T07:33:56.541032+0000","last_peered":"2024-03-04T07:33:56.541032+0000","last_clean":"2024-03-04T07:33:56.541032+0000","last_became_active":"2024-03-04T07:33:56.530488+0000","last_became_peered":"2024-03-04T07:33:56.530488+0000","last_unstale":"2024-03-04T07:33:56.541032+0000","last_undegraded":"2024-03-04T07:33:56.541032+0000","last_fullsized":"2024-03-04T07:33:56.541032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:15:19.032824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.561762+0000","last_change":"2024-03-04T07:33:56.558669+0000","last_active":"2024-03-04T07:33:56.561762+0000","last_peered":"2024-03-04T07:33:56.561762+0000","last_clean":"2024-03-04T07:33:56.561762+0000","last_became_active":"2024-03-04T07:33:56.558350+0000","last_became_peered":"2024-03-04T07:33:56.558350+0000","last_unstale":"2024-03-04T07:33:56.561762+0000","last_undegraded":"2024-03-04T07:33:56.561762+0000","last_fullsized":"2024-03-04T07:33:56.561762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:37:17.358787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.551730+0000","last_change":"2024-03-04T07:33:56.540760+0000","last_active":"2024-03-04T07:33:56.551730+0000","last_peered":"2024-03-04T07:33:56.551730+0000","last_clean":"2024-03-04T07:33:56.551730+0000","last_became_active":"2024-03-04T07:33:56.540351+0000","last_became_peered":"2024-03-04T07:33:56.540351+0000","last_unstale":"2024-03-04T07:33:56.551730+0000","last_undegraded":"2024-03-04T07:33:56.551730+0000","last_fullsized":"2024-03-04T07:33:56.551730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:36:55.087150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.559912+0000","last_change":"2024-03-04T07:33:56.558333+0000","last_active":"2024-03-04T07:33:56.559912+0000","last_peered":"2024-03-04T07:33:56.559912+0000","last_clean":"2024-03-04T07:33:56.559912+0000","last_became_active":"2024-03-04T07:33:56.558046+0000","last_became_peered":"2024-03-04T07:33:56.558046+0000","last_unstale":"2024-03-04T07:33:56.559912+0000","last_undegraded":"2024-03-04T07:33:56.559912+0000","last_fullsized":"2024-03-04T07:33:56.559912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:04:00.752661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.551431+0000","last_change":"2024-03-04T07:33:56.543476+0000","last_active":"2024-03-04T07:33:56.551431+0000","last_peered":"2024-03-04T07:33:56.551431+0000","last_clean":"2024-03-04T07:33:56.551431+0000","last_became_active":"2024-03-04T07:33:56.543176+0000","last_became_peered":"2024-03-04T07:33:56.543176+0000","last_unstale":"2024-03-04T07:33:56.551431+0000","last_undegraded":"2024-03-04T07:33:56.551431+0000","last_fullsized":"2024-03-04T07:33:56.551431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:42:32.313196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523950+0000","last_change":"2024-03-04T07:33:56.559488+0000","last_active":"2024-03-04T07:33:57.523950+0000","last_peered":"2024-03-04T07:33:57.523950+0000","last_clean":"2024-03-04T07:33:57.523950+0000","last_became_active":"2024-03-04T07:33:56.559172+0000","last_became_peered":"2024-03-04T07:33:56.559172+0000","last_unstale":"2024-03-04T07:33:57.523950+0000","last_undegraded":"2024-03-04T07:33:57.523950+0000","last_fullsized":"2024-03-04T07:33:57.523950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:53.461697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.565952+0000","last_change":"2024-03-04T07:33:56.560587+0000","last_active":"2024-03-04T07:33:56.565952+0000","last_peered":"2024-03-04T07:33:56.565952+0000","last_clean":"2024-03-04T07:33:56.565952+0000","last_became_active":"2024-03-04T07:33:56.560306+0000","last_became_peered":"2024-03-04T07:33:56.560306+0000","last_unstale":"2024-03-04T07:33:56.565952+0000","last_undegraded":"2024-03-04T07:33:56.565952+0000","last_fullsized":"2024-03-04T07:33:56.565952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:17.459841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.562088+0000","last_change":"2024-03-04T07:33:56.560852+0000","last_active":"2024-03-04T07:33:56.562088+0000","last_peered":"2024-03-04T07:33:56.562088+0000","last_clean":"2024-03-04T07:33:56.562088+0000","last_became_active":"2024-03-04T07:33:56.560592+0000","last_became_peered":"2024-03-04T07:33:56.560592+0000","last_unstale":"2024-03-04T07:33:56.562088+0000","last_undegraded":"2024-03-04T07:33:56.562088+0000","last_fullsized":"2024-03-04T07:33:56.562088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:01:50.310438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.562094+0000","last_change":"2024-03-04T07:33:56.561040+0000","last_active":"2024-03-04T07:33:56.562094+0000","last_peered":"2024-03-04T07:33:56.562094+0000","last_clean":"2024-03-04T07:33:56.562094+0000","last_became_active":"2024-03-04T07:33:56.560719+0000","last_became_peered":"2024-03-04T07:33:56.560719+0000","last_unstale":"2024-03-04T07:33:56.562094+0000","last_undegraded":"2024-03-04T07:33:56.562094+0000","last_fullsized":"2024-03-04T07:33:56.562094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:51:07.869500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.563152+0000","last_change":"2024-03-04T07:33:56.550648+0000","last_active":"2024-03-04T07:33:56.563152+0000","last_peered":"2024-03-04T07:33:56.563152+0000","last_clean":"2024-03-04T07:33:56.563152+0000","last_became_active":"2024-03-04T07:33:56.549568+0000","last_became_peered":"2024-03-04T07:33:56.549568+0000","last_unstale":"2024-03-04T07:33:56.563152+0000","last_undegraded":"2024-03-04T07:33:56.563152+0000","last_fullsized":"2024-03-04T07:33:56.563152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:12:46.537347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523777+0000","last_change":"2024-03-04T07:33:56.558880+0000","last_active":"2024-03-04T07:33:57.523777+0000","last_peered":"2024-03-04T07:33:57.523777+0000","last_clean":"2024-03-04T07:33:57.523777+0000","last_became_active":"2024-03-04T07:33:56.558567+0000","last_became_peered":"2024-03-04T07:33:56.558567+0000","last_unstale":"2024-03-04T07:33:57.523777+0000","last_undegraded":"2024-03-04T07:33:57.523777+0000","last_fullsized":"2024-03-04T07:33:57.523777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:16.354787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.546040+0000","last_change":"2024-03-04T07:33:56.544222+0000","last_active":"2024-03-04T07:33:56.546040+0000","last_peered":"2024-03-04T07:33:56.546040+0000","last_clean":"2024-03-04T07:33:56.546040+0000","last_became_active":"2024-03-04T07:33:56.543936+0000","last_became_peered":"2024-03-04T07:33:56.543936+0000","last_unstale":"2024-03-04T07:33:56.546040+0000","last_undegraded":"2024-03-04T07:33:56.546040+0000","last_fullsized":"2024-03-04T07:33:56.546040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:20:34.741592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749980+0000","last_change":"2024-03-04T07:33:56.545195+0000","last_active":"2024-03-04T07:33:57.749980+0000","last_peered":"2024-03-04T07:33:57.749980+0000","last_clean":"2024-03-04T07:33:57.749980+0000","last_became_active":"2024-03-04T07:33:56.544837+0000","last_became_peered":"2024-03-04T07:33:56.544837+0000","last_unstale":"2024-03-04T07:33:57.749980+0000","last_undegraded":"2024-03-04T07:33:57.749980+0000","last_fullsized":"2024-03-04T07:33:57.749980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:24:59.643746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.559283+0000","last_change":"2024-03-04T07:33:56.558398+0000","last_active":"2024-03-04T07:33:56.559283+0000","last_peered":"2024-03-04T07:33:56.559283+0000","last_clean":"2024-03-04T07:33:56.559283+0000","last_became_active":"2024-03-04T07:33:56.557962+0000","last_became_peered":"2024-03-04T07:33:56.557962+0000","last_unstale":"2024-03-04T07:33:56.559283+0000","last_undegraded":"2024-03-04T07:33:56.559283+0000","last_fullsized":"2024-03-04T07:33:56.559283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:47:39.055655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.563056+0000","last_change":"2024-03-04T07:33:56.559597+0000","last_active":"2024-03-04T07:33:56.563056+0000","last_peered":"2024-03-04T07:33:56.563056+0000","last_clean":"2024-03-04T07:33:56.563056+0000","last_became_active":"2024-03-04T07:33:56.558719+0000","last_became_peered":"2024-03-04T07:33:56.558719+0000","last_unstale":"2024-03-04T07:33:56.563056+0000","last_undegraded":"2024-03-04T07:33:56.563056+0000","last_fullsized":"2024-03-04T07:33:56.563056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:27:12.896606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.532751+0000","last_change":"2024-03-04T07:33:56.528479+0000","last_active":"2024-03-04T07:33:56.532751+0000","last_peered":"2024-03-04T07:33:56.532751+0000","last_clean":"2024-03-04T07:33:56.532751+0000","last_became_active":"2024-03-04T07:33:56.528213+0000","last_became_peered":"2024-03-04T07:33:56.528213+0000","last_unstale":"2024-03-04T07:33:56.532751+0000","last_undegraded":"2024-03-04T07:33:56.532751+0000","last_fullsized":"2024-03-04T07:33:56.532751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:01:21.785006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.552549+0000","last_change":"2024-03-04T07:33:56.545890+0000","last_active":"2024-03-04T07:33:56.552549+0000","last_peered":"2024-03-04T07:33:56.552549+0000","last_clean":"2024-03-04T07:33:56.552549+0000","last_became_active":"2024-03-04T07:33:56.545467+0000","last_became_peered":"2024-03-04T07:33:56.545467+0000","last_unstale":"2024-03-04T07:33:56.552549+0000","last_undegraded":"2024-03-04T07:33:56.552549+0000","last_fullsized":"2024-03-04T07:33:56.552549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:14:59.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.558357+0000","last_change":"2024-03-04T07:33:56.549059+0000","last_active":"2024-03-04T07:33:56.558357+0000","last_peered":"2024-03-04T07:33:56.558357+0000","last_clean":"2024-03-04T07:33:56.558357+0000","last_became_active":"2024-03-04T07:33:56.548783+0000","last_became_peered":"2024-03-04T07:33:56.548783+0000","last_unstale":"2024-03-04T07:33:56.558357+0000","last_undegraded":"2024-03-04T07:33:56.558357+0000","last_fullsized":"2024-03-04T07:33:56.558357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:57:39.713413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.545819+0000","last_change":"2024-03-04T07:33:56.540558+0000","last_active":"2024-03-04T07:33:56.545819+0000","last_peered":"2024-03-04T07:33:56.545819+0000","last_clean":"2024-03-04T07:33:56.545819+0000","last_became_active":"2024-03-04T07:33:56.539403+0000","last_became_peered":"2024-03-04T07:33:56.539403+0000","last_unstale":"2024-03-04T07:33:56.545819+0000","last_undegraded":"2024-03-04T07:33:56.545819+0000","last_fullsized":"2024-03-04T07:33:56.545819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:58:50.040860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.559037+0000","last_change":"2024-03-04T07:33:56.554982+0000","last_active":"2024-03-04T07:33:56.559037+0000","last_peered":"2024-03-04T07:33:56.559037+0000","last_clean":"2024-03-04T07:33:56.559037+0000","last_became_active":"2024-03-04T07:33:56.554470+0000","last_became_peered":"2024-03-04T07:33:56.554470+0000","last_unstale":"2024-03-04T07:33:56.559037+0000","last_undegraded":"2024-03-04T07:33:56.559037+0000","last_fullsized":"2024-03-04T07:33:56.559037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:00:11.441784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523496+0000","last_change":"2024-03-04T07:33:56.545465+0000","last_active":"2024-03-04T07:33:57.523496+0000","last_peered":"2024-03-04T07:33:57.523496+0000","last_clean":"2024-03-04T07:33:57.523496+0000","last_became_active":"2024-03-04T07:33:56.545270+0000","last_became_peered":"2024-03-04T07:33:56.545270+0000","last_unstale":"2024-03-04T07:33:57.523496+0000","last_undegraded":"2024-03-04T07:33:57.523496+0000","last_fullsized":"2024-03-04T07:33:57.523496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:08:45.244868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.559074+0000","last_change":"2024-03-04T07:33:56.555428+0000","last_active":"2024-03-04T07:33:56.559074+0000","last_peered":"2024-03-04T07:33:56.559074+0000","last_clean":"2024-03-04T07:33:56.559074+0000","last_became_active":"2024-03-04T07:33:56.554942+0000","last_became_peered":"2024-03-04T07:33:56.554942+0000","last_unstale":"2024-03-04T07:33:56.559074+0000","last_undegraded":"2024-03-04T07:33:56.559074+0000","last_fullsized":"2024-03-04T07:33:56.559074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:55:57.341319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523610+0000","last_change":"2024-03-04T07:33:56.549563+0000","last_active":"2024-03-04T07:33:57.523610+0000","last_peered":"2024-03-04T07:33:57.523610+0000","last_clean":"2024-03-04T07:33:57.523610+0000","last_became_active":"2024-03-04T07:33:56.548790+0000","last_became_peered":"2024-03-04T07:33:56.548790+0000","last_unstale":"2024-03-04T07:33:57.523610+0000","last_undegraded":"2024-03-04T07:33:57.523610+0000","last_fullsized":"2024-03-04T07:33:57.523610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:30:53.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.558883+0000","last_change":"2024-03-04T07:33:56.553791+0000","last_active":"2024-03-04T07:33:56.558883+0000","last_peered":"2024-03-04T07:33:56.558883+0000","last_clean":"2024-03-04T07:33:56.558883+0000","last_became_active":"2024-03-04T07:33:56.553012+0000","last_became_peered":"2024-03-04T07:33:56.553012+0000","last_unstale":"2024-03-04T07:33:56.558883+0000","last_undegraded":"2024-03-04T07:33:56.558883+0000","last_fullsized":"2024-03-04T07:33:56.558883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:53:21.626735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750163+0000","last_change":"2024-03-04T07:33:56.551341+0000","last_active":"2024-03-04T07:33:57.750163+0000","last_peered":"2024-03-04T07:33:57.750163+0000","last_clean":"2024-03-04T07:33:57.750163+0000","last_became_active":"2024-03-04T07:33:56.550616+0000","last_became_peered":"2024-03-04T07:33:56.550616+0000","last_unstale":"2024-03-04T07:33:57.750163+0000","last_undegraded":"2024-03-04T07:33:57.750163+0000","last_fullsized":"2024-03-04T07:33:57.750163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:35:45.801510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523840+0000","last_change":"2024-03-04T07:33:56.543903+0000","last_active":"2024-03-04T07:33:57.523840+0000","last_peered":"2024-03-04T07:33:57.523840+0000","last_clean":"2024-03-04T07:33:57.523840+0000","last_became_active":"2024-03-04T07:33:56.543632+0000","last_became_peered":"2024-03-04T07:33:56.543632+0000","last_unstale":"2024-03-04T07:33:57.523840+0000","last_undegraded":"2024-03-04T07:33:57.523840+0000","last_fullsized":"2024-03-04T07:33:57.523840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:26:10.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.550195+0000","last_change":"2024-03-04T07:33:56.547091+0000","last_active":"2024-03-04T07:33:56.550195+0000","last_peered":"2024-03-04T07:33:56.550195+0000","last_clean":"2024-03-04T07:33:56.550195+0000","last_became_active":"2024-03-04T07:33:56.546652+0000","last_became_peered":"2024-03-04T07:33:56.546652+0000","last_unstale":"2024-03-04T07:33:56.550195+0000","last_undegraded":"2024-03-04T07:33:56.550195+0000","last_fullsized":"2024-03-04T07:33:56.550195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:02:20.861575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.559018+0000","last_change":"2024-03-04T07:33:56.552493+0000","last_active":"2024-03-04T07:33:56.559018+0000","last_peered":"2024-03-04T07:33:56.559018+0000","last_clean":"2024-03-04T07:33:56.559018+0000","last_became_active":"2024-03-04T07:33:56.552189+0000","last_became_peered":"2024-03-04T07:33:56.552189+0000","last_unstale":"2024-03-04T07:33:56.559018+0000","last_undegraded":"2024-03-04T07:33:56.559018+0000","last_fullsized":"2024-03-04T07:33:56.559018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:49:36.688567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.552470+0000","last_change":"2024-03-04T07:33:56.549661+0000","last_active":"2024-03-04T07:33:56.552470+0000","last_peered":"2024-03-04T07:33:56.552470+0000","last_clean":"2024-03-04T07:33:56.552470+0000","last_became_active":"2024-03-04T07:33:56.549351+0000","last_became_peered":"2024-03-04T07:33:56.549351+0000","last_unstale":"2024-03-04T07:33:56.552470+0000","last_undegraded":"2024-03-04T07:33:56.552470+0000","last_fullsized":"2024-03-04T07:33:56.552470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:29:48.841517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.556093+0000","last_change":"2024-03-04T07:33:56.533132+0000","last_active":"2024-03-04T07:33:56.556093+0000","last_peered":"2024-03-04T07:33:56.556093+0000","last_clean":"2024-03-04T07:33:56.556093+0000","last_became_active":"2024-03-04T07:33:56.532251+0000","last_became_peered":"2024-03-04T07:33:56.532251+0000","last_unstale":"2024-03-04T07:33:56.556093+0000","last_undegraded":"2024-03-04T07:33:56.556093+0000","last_fullsized":"2024-03-04T07:33:56.556093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:30:00.806315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.545050+0000","last_change":"2024-03-04T07:33:56.543382+0000","last_active":"2024-03-04T07:33:56.545050+0000","last_peered":"2024-03-04T07:33:56.545050+0000","last_clean":"2024-03-04T07:33:56.545050+0000","last_became_active":"2024-03-04T07:33:56.542937+0000","last_became_peered":"2024-03-04T07:33:56.542937+0000","last_unstale":"2024-03-04T07:33:56.545050+0000","last_undegraded":"2024-03-04T07:33:56.545050+0000","last_fullsized":"2024-03-04T07:33:56.545050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:32:05.211873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523876+0000","last_change":"2024-03-04T07:33:55.566130+0000","last_active":"2024-03-04T07:33:57.523876+0000","last_peered":"2024-03-04T07:33:57.523876+0000","last_clean":"2024-03-04T07:33:57.523876+0000","last_became_active":"2024-03-04T07:33:55.565894+0000","last_became_peered":"2024-03-04T07:33:55.565894+0000","last_unstale":"2024-03-04T07:33:57.523876+0000","last_undegraded":"2024-03-04T07:33:57.523876+0000","last_fullsized":"2024-03-04T07:33:57.523876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:36.530949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.545797+0000","last_change":"2024-03-04T07:33:56.540113+0000","last_active":"2024-03-04T07:33:56.545797+0000","last_peered":"2024-03-04T07:33:56.545797+0000","last_clean":"2024-03-04T07:33:56.545797+0000","last_became_active":"2024-03-04T07:33:56.539738+0000","last_became_peered":"2024-03-04T07:33:56.539738+0000","last_unstale":"2024-03-04T07:33:56.545797+0000","last_undegraded":"2024-03-04T07:33:56.545797+0000","last_fullsized":"2024-03-04T07:33:56.545797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:46:17.799528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.524724+0000","last_change":"2024-03-04T07:33:55.554956+0000","last_active":"2024-03-04T07:33:56.524724+0000","last_peered":"2024-03-04T07:33:56.524724+0000","last_clean":"2024-03-04T07:33:56.524724+0000","last_became_active":"2024-03-04T07:33:55.554151+0000","last_became_peered":"2024-03-04T07:33:55.554151+0000","last_unstale":"2024-03-04T07:33:56.524724+0000","last_undegraded":"2024-03-04T07:33:56.524724+0000","last_fullsized":"2024-03-04T07:33:56.524724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:06:19.621510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523529+0000","last_change":"2024-03-04T07:33:55.542200+0000","last_active":"2024-03-04T07:33:57.523529+0000","last_peered":"2024-03-04T07:33:57.523529+0000","last_clean":"2024-03-04T07:33:57.523529+0000","last_became_active":"2024-03-04T07:33:55.541902+0000","last_became_peered":"2024-03-04T07:33:55.541902+0000","last_unstale":"2024-03-04T07:33:57.523529+0000","last_undegraded":"2024-03-04T07:33:57.523529+0000","last_fullsized":"2024-03-04T07:33:57.523529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:55:50.251288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523733+0000","last_change":"2024-03-04T07:33:56.548380+0000","last_active":"2024-03-04T07:33:57.523733+0000","last_peered":"2024-03-04T07:33:57.523733+0000","last_clean":"2024-03-04T07:33:57.523733+0000","last_became_active":"2024-03-04T07:33:56.546087+0000","last_became_peered":"2024-03-04T07:33:56.546087+0000","last_unstale":"2024-03-04T07:33:57.523733+0000","last_undegraded":"2024-03-04T07:33:57.523733+0000","last_fullsized":"2024-03-04T07:33:57.523733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:06:43.538687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.549228+0000","last_change":"2024-03-04T07:33:55.568508+0000","last_active":"2024-03-04T07:33:56.549228+0000","last_peered":"2024-03-04T07:33:56.549228+0000","last_clean":"2024-03-04T07:33:56.549228+0000","last_became_active":"2024-03-04T07:33:55.566073+0000","last_became_peered":"2024-03-04T07:33:55.566073+0000","last_unstale":"2024-03-04T07:33:56.549228+0000","last_undegraded":"2024-03-04T07:33:56.549228+0000","last_fullsized":"2024-03-04T07:33:56.549228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:34:17.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749969+0000","last_change":"2024-03-04T07:33:56.552785+0000","last_active":"2024-03-04T07:33:57.749969+0000","last_peered":"2024-03-04T07:33:57.749969+0000","last_clean":"2024-03-04T07:33:57.749969+0000","last_became_active":"2024-03-04T07:33:56.552533+0000","last_became_peered":"2024-03-04T07:33:56.552533+0000","last_unstale":"2024-03-04T07:33:57.749969+0000","last_undegraded":"2024-03-04T07:33:57.749969+0000","last_fullsized":"2024-03-04T07:33:57.749969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:08.582237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.524288+0000","last_change":"2024-03-04T07:33:55.559395+0000","last_active":"2024-03-04T07:33:56.524288+0000","last_peered":"2024-03-04T07:33:56.524288+0000","last_clean":"2024-03-04T07:33:56.524288+0000","last_became_active":"2024-03-04T07:33:55.558593+0000","last_became_peered":"2024-03-04T07:33:55.558593+0000","last_unstale":"2024-03-04T07:33:56.524288+0000","last_undegraded":"2024-03-04T07:33:56.524288+0000","last_fullsized":"2024-03-04T07:33:56.524288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:00:20.205445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.558878+0000","last_change":"2024-03-04T07:33:56.553260+0000","last_active":"2024-03-04T07:33:56.558878+0000","last_peered":"2024-03-04T07:33:56.558878+0000","last_clean":"2024-03-04T07:33:56.558878+0000","last_became_active":"2024-03-04T07:33:56.552952+0000","last_became_peered":"2024-03-04T07:33:56.552952+0000","last_unstale":"2024-03-04T07:33:56.558878+0000","last_undegraded":"2024-03-04T07:33:56.558878+0000","last_fullsized":"2024-03-04T07:33:56.558878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:41:27.660960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.550348+0000","last_change":"2024-03-04T07:33:55.568220+0000","last_active":"2024-03-04T07:33:56.550348+0000","last_peered":"2024-03-04T07:33:56.550348+0000","last_clean":"2024-03-04T07:33:56.550348+0000","last_became_active":"2024-03-04T07:33:55.567569+0000","last_became_peered":"2024-03-04T07:33:55.567569+0000","last_unstale":"2024-03-04T07:33:56.550348+0000","last_undegraded":"2024-03-04T07:33:56.550348+0000","last_fullsized":"2024-03-04T07:33:56.550348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:40:49.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.561115+0000","last_change":"2024-03-04T07:33:56.559745+0000","last_active":"2024-03-04T07:33:56.561115+0000","last_peered":"2024-03-04T07:33:56.561115+0000","last_clean":"2024-03-04T07:33:56.561115+0000","last_became_active":"2024-03-04T07:33:56.559456+0000","last_became_peered":"2024-03-04T07:33:56.559456+0000","last_unstale":"2024-03-04T07:33:56.561115+0000","last_undegraded":"2024-03-04T07:33:56.561115+0000","last_fullsized":"2024-03-04T07:33:56.561115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:43.565789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750174+0000","last_change":"2024-03-04T07:33:55.550625+0000","last_active":"2024-03-04T07:33:57.750174+0000","last_peered":"2024-03-04T07:33:57.750174+0000","last_clean":"2024-03-04T07:33:57.750174+0000","last_became_active":"2024-03-04T07:33:55.550327+0000","last_became_peered":"2024-03-04T07:33:55.550327+0000","last_unstale":"2024-03-04T07:33:57.750174+0000","last_undegraded":"2024-03-04T07:33:57.750174+0000","last_fullsized":"2024-03-04T07:33:57.750174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:20.282585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.540025+0000","last_change":"2024-03-04T07:33:56.537570+0000","last_active":"2024-03-04T07:33:56.540025+0000","last_peered":"2024-03-04T07:33:56.540025+0000","last_clean":"2024-03-04T07:33:56.540025+0000","last_became_active":"2024-03-04T07:33:56.536870+0000","last_became_peered":"2024-03-04T07:33:56.536870+0000","last_unstale":"2024-03-04T07:33:56.540025+0000","last_undegraded":"2024-03-04T07:33:56.540025+0000","last_fullsized":"2024-03-04T07:33:56.540025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:16.802084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.525030+0000","last_change":"2024-03-04T07:33:55.538915+0000","last_active":"2024-03-04T07:33:56.525030+0000","last_peered":"2024-03-04T07:33:56.525030+0000","last_clean":"2024-03-04T07:33:56.525030+0000","last_became_active":"2024-03-04T07:33:55.536263+0000","last_became_peered":"2024-03-04T07:33:55.536263+0000","last_unstale":"2024-03-04T07:33:56.525030+0000","last_undegraded":"2024-03-04T07:33:56.525030+0000","last_fullsized":"2024-03-04T07:33:56.525030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:09.612704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.555498+0000","last_change":"2024-03-04T07:33:56.551087+0000","last_active":"2024-03-04T07:33:56.555498+0000","last_peered":"2024-03-04T07:33:56.555498+0000","last_clean":"2024-03-04T07:33:56.555498+0000","last_became_active":"2024-03-04T07:33:56.550767+0000","last_became_peered":"2024-03-04T07:33:56.550767+0000","last_unstale":"2024-03-04T07:33:56.555498+0000","last_undegraded":"2024-03-04T07:33:56.555498+0000","last_fullsized":"2024-03-04T07:33:56.555498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:09:49.333705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.528919+0000","last_change":"2024-03-04T07:33:55.560382+0000","last_active":"2024-03-04T07:33:56.528919+0000","last_peered":"2024-03-04T07:33:56.528919+0000","last_clean":"2024-03-04T07:33:56.528919+0000","last_became_active":"2024-03-04T07:33:55.560078+0000","last_became_peered":"2024-03-04T07:33:55.560078+0000","last_unstale":"2024-03-04T07:33:56.528919+0000","last_undegraded":"2024-03-04T07:33:56.528919+0000","last_fullsized":"2024-03-04T07:33:56.528919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:41:10.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.546023+0000","last_change":"2024-03-04T07:33:56.537920+0000","last_active":"2024-03-04T07:33:56.546023+0000","last_peered":"2024-03-04T07:33:56.546023+0000","last_clean":"2024-03-04T07:33:56.546023+0000","last_became_active":"2024-03-04T07:33:56.535960+0000","last_became_peered":"2024-03-04T07:33:56.535960+0000","last_unstale":"2024-03-04T07:33:56.546023+0000","last_undegraded":"2024-03-04T07:33:56.546023+0000","last_fullsized":"2024-03-04T07:33:56.546023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:55:31.800424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.550145+0000","last_change":"2024-03-04T07:33:55.567682+0000","last_active":"2024-03-04T07:33:56.550145+0000","last_peered":"2024-03-04T07:33:56.550145+0000","last_clean":"2024-03-04T07:33:56.550145+0000","last_became_active":"2024-03-04T07:33:55.566915+0000","last_became_peered":"2024-03-04T07:33:55.566915+0000","last_unstale":"2024-03-04T07:33:56.550145+0000","last_undegraded":"2024-03-04T07:33:56.550145+0000","last_fullsized":"2024-03-04T07:33:56.550145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:33:47.010346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.553459+0000","last_change":"2024-03-04T07:33:56.547846+0000","last_active":"2024-03-04T07:33:56.553459+0000","last_peered":"2024-03-04T07:33:56.553459+0000","last_clean":"2024-03-04T07:33:56.553459+0000","last_became_active":"2024-03-04T07:33:56.547459+0000","last_became_peered":"2024-03-04T07:33:56.547459+0000","last_unstale":"2024-03-04T07:33:56.553459+0000","last_undegraded":"2024-03-04T07:33:56.553459+0000","last_fullsized":"2024-03-04T07:33:56.553459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:45:24.163852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.523896+0000","last_change":"2024-03-04T07:33:55.557041+0000","last_active":"2024-03-04T07:33:56.523896+0000","last_peered":"2024-03-04T07:33:56.523896+0000","last_clean":"2024-03-04T07:33:56.523896+0000","last_became_active":"2024-03-04T07:33:55.556416+0000","last_became_peered":"2024-03-04T07:33:55.556416+0000","last_unstale":"2024-03-04T07:33:56.523896+0000","last_undegraded":"2024-03-04T07:33:56.523896+0000","last_fullsized":"2024-03-04T07:33:56.523896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:31.472235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.555917+0000","last_change":"2024-03-04T07:33:56.531986+0000","last_active":"2024-03-04T07:33:56.555917+0000","last_peered":"2024-03-04T07:33:56.555917+0000","last_clean":"2024-03-04T07:33:56.555917+0000","last_became_active":"2024-03-04T07:33:56.531565+0000","last_became_peered":"2024-03-04T07:33:56.531565+0000","last_unstale":"2024-03-04T07:33:56.555917+0000","last_undegraded":"2024-03-04T07:33:56.555917+0000","last_fullsized":"2024-03-04T07:33:56.555917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:37:17.358787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.521077+0000","last_change":"2024-03-04T07:33:55.538469+0000","last_active":"2024-03-04T07:33:56.521077+0000","last_peered":"2024-03-04T07:33:56.521077+0000","last_clean":"2024-03-04T07:33:56.521077+0000","last_became_active":"2024-03-04T07:33:55.538165+0000","last_became_peered":"2024-03-04T07:33:55.538165+0000","last_unstale":"2024-03-04T07:33:56.521077+0000","last_undegraded":"2024-03-04T07:33:56.521077+0000","last_fullsized":"2024-03-04T07:33:56.521077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:00:01.263768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.552449+0000","last_change":"2024-03-04T07:33:56.547881+0000","last_active":"2024-03-04T07:33:56.552449+0000","last_peered":"2024-03-04T07:33:56.552449+0000","last_clean":"2024-03-04T07:33:56.552449+0000","last_became_active":"2024-03-04T07:33:56.547461+0000","last_became_peered":"2024-03-04T07:33:56.547461+0000","last_unstale":"2024-03-04T07:33:56.552449+0000","last_undegraded":"2024-03-04T07:33:56.552449+0000","last_fullsized":"2024-03-04T07:33:56.552449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:30:35.809552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.533836+0000","last_change":"2024-03-04T07:33:55.544356+0000","last_active":"2024-03-04T07:33:56.533836+0000","last_peered":"2024-03-04T07:33:56.533836+0000","last_clean":"2024-03-04T07:33:56.533836+0000","last_became_active":"2024-03-04T07:33:55.541295+0000","last_became_peered":"2024-03-04T07:33:55.541295+0000","last_unstale":"2024-03-04T07:33:56.533836+0000","last_undegraded":"2024-03-04T07:33:56.533836+0000","last_fullsized":"2024-03-04T07:33:56.533836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:22.953982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.552488+0000","last_change":"2024-03-04T07:33:56.549038+0000","last_active":"2024-03-04T07:33:56.552488+0000","last_peered":"2024-03-04T07:33:56.552488+0000","last_clean":"2024-03-04T07:33:56.552488+0000","last_became_active":"2024-03-04T07:33:56.548113+0000","last_became_peered":"2024-03-04T07:33:56.548113+0000","last_unstale":"2024-03-04T07:33:56.552488+0000","last_undegraded":"2024-03-04T07:33:56.552488+0000","last_fullsized":"2024-03-04T07:33:56.552488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:43:47.784030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.521591+0000","last_change":"2024-03-04T07:33:55.538871+0000","last_active":"2024-03-04T07:33:56.521591+0000","last_peered":"2024-03-04T07:33:56.521591+0000","last_clean":"2024-03-04T07:33:56.521591+0000","last_became_active":"2024-03-04T07:33:55.537221+0000","last_became_peered":"2024-03-04T07:33:55.537221+0000","last_unstale":"2024-03-04T07:33:56.521591+0000","last_undegraded":"2024-03-04T07:33:56.521591+0000","last_fullsized":"2024-03-04T07:33:56.521591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:02:34.533330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.551403+0000","last_change":"2024-03-04T07:33:56.543760+0000","last_active":"2024-03-04T07:33:56.551403+0000","last_peered":"2024-03-04T07:33:56.551403+0000","last_clean":"2024-03-04T07:33:56.551403+0000","last_became_active":"2024-03-04T07:33:56.543354+0000","last_became_peered":"2024-03-04T07:33:56.543354+0000","last_unstale":"2024-03-04T07:33:56.551403+0000","last_undegraded":"2024-03-04T07:33:56.551403+0000","last_fullsized":"2024-03-04T07:33:56.551403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:06.936208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750256+0000","last_change":"2024-03-04T07:33:55.552856+0000","last_active":"2024-03-04T07:33:57.750256+0000","last_peered":"2024-03-04T07:33:57.750256+0000","last_clean":"2024-03-04T07:33:57.750256+0000","last_became_active":"2024-03-04T07:33:55.552472+0000","last_became_peered":"2024-03-04T07:33:55.552472+0000","last_unstale":"2024-03-04T07:33:57.750256+0000","last_undegraded":"2024-03-04T07:33:57.750256+0000","last_fullsized":"2024-03-04T07:33:57.750256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:36:50.409567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.561741+0000","last_change":"2024-03-04T07:33:56.558497+0000","last_active":"2024-03-04T07:33:56.561741+0000","last_peered":"2024-03-04T07:33:56.561741+0000","last_clean":"2024-03-04T07:33:56.561741+0000","last_became_active":"2024-03-04T07:33:56.557989+0000","last_became_peered":"2024-03-04T07:33:56.557989+0000","last_unstale":"2024-03-04T07:33:56.561741+0000","last_undegraded":"2024-03-04T07:33:56.561741+0000","last_fullsized":"2024-03-04T07:33:56.561741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:31:13.406101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.520930+0000","last_change":"2024-03-04T07:33:55.560332+0000","last_active":"2024-03-04T07:33:56.520930+0000","last_peered":"2024-03-04T07:33:56.520930+0000","last_clean":"2024-03-04T07:33:56.520930+0000","last_became_active":"2024-03-04T07:33:55.560086+0000","last_became_peered":"2024-03-04T07:33:55.560086+0000","last_unstale":"2024-03-04T07:33:56.520930+0000","last_undegraded":"2024-03-04T07:33:56.520930+0000","last_fullsized":"2024-03-04T07:33:56.520930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:25:46.280657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.565836+0000","last_change":"2024-03-04T07:33:56.559881+0000","last_active":"2024-03-04T07:33:56.565836+0000","last_peered":"2024-03-04T07:33:56.565836+0000","last_clean":"2024-03-04T07:33:56.565836+0000","last_became_active":"2024-03-04T07:33:56.559589+0000","last_became_peered":"2024-03-04T07:33:56.559589+0000","last_unstale":"2024-03-04T07:33:56.565836+0000","last_undegraded":"2024-03-04T07:33:56.565836+0000","last_fullsized":"2024-03-04T07:33:56.565836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:59:39.122143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.531971+0000","last_change":"2024-03-04T07:33:43.559630+0000","last_active":"2024-03-04T07:33:56.531971+0000","last_peered":"2024-03-04T07:33:56.531971+0000","last_clean":"2024-03-04T07:33:56.531971+0000","last_became_active":"2024-03-04T07:33:43.557489+0000","last_became_peered":"2024-03-04T07:33:43.557489+0000","last_unstale":"2024-03-04T07:33:56.531971+0000","last_undegraded":"2024-03-04T07:33:56.531971+0000","last_fullsized":"2024-03-04T07:33:56.531971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_clean_scrub_stamp":"2024-03-04T07:33:42.183823+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:55:35.762857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.550284+0000","last_change":"2024-03-04T07:33:55.548616+0000","last_active":"2024-03-04T07:33:56.550284+0000","last_peered":"2024-03-04T07:33:56.550284+0000","last_clean":"2024-03-04T07:33:56.550284+0000","last_became_active":"2024-03-04T07:33:55.547322+0000","last_became_peered":"2024-03-04T07:33:55.547322+0000","last_unstale":"2024-03-04T07:33:56.550284+0000","last_undegraded":"2024-03-04T07:33:56.550284+0000","last_fullsized":"2024-03-04T07:33:56.550284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:20:16.717483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523877+0000","last_change":"2024-03-04T07:33:56.559966+0000","last_active":"2024-03-04T07:33:57.523877+0000","last_peered":"2024-03-04T07:33:57.523877+0000","last_clean":"2024-03-04T07:33:57.523877+0000","last_became_active":"2024-03-04T07:33:56.559581+0000","last_became_peered":"2024-03-04T07:33:56.559581+0000","last_unstale":"2024-03-04T07:33:57.523877+0000","last_undegraded":"2024-03-04T07:33:57.523877+0000","last_fullsized":"2024-03-04T07:33:57.523877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.531915+0000","last_change":"2024-03-04T07:33:55.563905+0000","last_active":"2024-03-04T07:33:56.531915+0000","last_peered":"2024-03-04T07:33:56.531915+0000","last_clean":"2024-03-04T07:33:56.531915+0000","last_became_active":"2024-03-04T07:33:55.563117+0000","last_became_peered":"2024-03-04T07:33:55.563117+0000","last_unstale":"2024-03-04T07:33:56.531915+0000","last_undegraded":"2024-03-04T07:33:56.531915+0000","last_fullsized":"2024-03-04T07:33:56.531915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:53:02.522757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.540990+0000","last_change":"2024-03-04T07:33:56.531080+0000","last_active":"2024-03-04T07:33:56.540990+0000","last_peered":"2024-03-04T07:33:56.540990+0000","last_clean":"2024-03-04T07:33:56.540990+0000","last_became_active":"2024-03-04T07:33:56.530776+0000","last_became_peered":"2024-03-04T07:33:56.530776+0000","last_unstale":"2024-03-04T07:33:56.540990+0000","last_undegraded":"2024-03-04T07:33:56.540990+0000","last_fullsized":"2024-03-04T07:33:56.540990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:45:39.713736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.520916+0000","last_change":"2024-03-04T07:33:55.538631+0000","last_active":"2024-03-04T07:33:56.520916+0000","last_peered":"2024-03-04T07:33:56.520916+0000","last_clean":"2024-03-04T07:33:56.520916+0000","last_became_active":"2024-03-04T07:33:55.537104+0000","last_became_peered":"2024-03-04T07:33:55.537104+0000","last_unstale":"2024-03-04T07:33:56.520916+0000","last_undegraded":"2024-03-04T07:33:56.520916+0000","last_fullsized":"2024-03-04T07:33:56.520916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:27:06.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749962+0000","last_change":"2024-03-04T07:33:56.541335+0000","last_active":"2024-03-04T07:33:57.749962+0000","last_peered":"2024-03-04T07:33:57.749962+0000","last_clean":"2024-03-04T07:33:57.749962+0000","last_became_active":"2024-03-04T07:33:56.540536+0000","last_became_peered":"2024-03-04T07:33:56.540536+0000","last_unstale":"2024-03-04T07:33:57.749962+0000","last_undegraded":"2024-03-04T07:33:57.749962+0000","last_fullsized":"2024-03-04T07:33:57.749962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:31:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.550343+0000","last_change":"2024-03-04T07:33:55.574740+0000","last_active":"2024-03-04T07:33:56.550343+0000","last_peered":"2024-03-04T07:33:56.550343+0000","last_clean":"2024-03-04T07:33:56.550343+0000","last_became_active":"2024-03-04T07:33:55.574129+0000","last_became_peered":"2024-03-04T07:33:55.574129+0000","last_unstale":"2024-03-04T07:33:56.550343+0000","last_undegraded":"2024-03-04T07:33:56.550343+0000","last_fullsized":"2024-03-04T07:33:56.550343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:40:57.891803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.543405+0000","last_change":"2024-03-04T07:33:56.541654+0000","last_active":"2024-03-04T07:33:56.543405+0000","last_peered":"2024-03-04T07:33:56.543405+0000","last_clean":"2024-03-04T07:33:56.543405+0000","last_became_active":"2024-03-04T07:33:56.541422+0000","last_became_peered":"2024-03-04T07:33:56.541422+0000","last_unstale":"2024-03-04T07:33:56.543405+0000","last_undegraded":"2024-03-04T07:33:56.543405+0000","last_fullsized":"2024-03-04T07:33:56.543405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:19:03.722779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750122+0000","last_change":"2024-03-04T07:33:55.539317+0000","last_active":"2024-03-04T07:33:57.750122+0000","last_peered":"2024-03-04T07:33:57.750122+0000","last_clean":"2024-03-04T07:33:57.750122+0000","last_became_active":"2024-03-04T07:33:55.539017+0000","last_became_peered":"2024-03-04T07:33:55.539017+0000","last_unstale":"2024-03-04T07:33:57.750122+0000","last_undegraded":"2024-03-04T07:33:57.750122+0000","last_fullsized":"2024-03-04T07:33:57.750122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:10.787316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.558954+0000","last_change":"2024-03-04T07:33:56.554103+0000","last_active":"2024-03-04T07:33:56.558954+0000","last_peered":"2024-03-04T07:33:56.558954+0000","last_clean":"2024-03-04T07:33:56.558954+0000","last_became_active":"2024-03-04T07:33:56.553742+0000","last_became_peered":"2024-03-04T07:33:56.553742+0000","last_unstale":"2024-03-04T07:33:56.558954+0000","last_undegraded":"2024-03-04T07:33:56.558954+0000","last_fullsized":"2024-03-04T07:33:56.558954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:10:17.027074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.536790+0000","last_change":"2024-03-04T07:33:55.569442+0000","last_active":"2024-03-04T07:33:56.536790+0000","last_peered":"2024-03-04T07:33:56.536790+0000","last_clean":"2024-03-04T07:33:56.536790+0000","last_became_active":"2024-03-04T07:33:55.567809+0000","last_became_peered":"2024-03-04T07:33:55.567809+0000","last_unstale":"2024-03-04T07:33:56.536790+0000","last_undegraded":"2024-03-04T07:33:56.536790+0000","last_fullsized":"2024-03-04T07:33:56.536790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:58.551866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750083+0000","last_change":"2024-03-04T07:33:56.542997+0000","last_active":"2024-03-04T07:33:57.750083+0000","last_peered":"2024-03-04T07:33:57.750083+0000","last_clean":"2024-03-04T07:33:57.750083+0000","last_became_active":"2024-03-04T07:33:56.542775+0000","last_became_peered":"2024-03-04T07:33:56.542775+0000","last_unstale":"2024-03-04T07:33:57.750083+0000","last_undegraded":"2024-03-04T07:33:57.750083+0000","last_fullsized":"2024-03-04T07:33:57.750083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:32.090568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750225+0000","last_change":"2024-03-04T07:33:55.545598+0000","last_active":"2024-03-04T07:33:57.750225+0000","last_peered":"2024-03-04T07:33:57.750225+0000","last_clean":"2024-03-04T07:33:57.750225+0000","last_became_active":"2024-03-04T07:33:55.545262+0000","last_became_peered":"2024-03-04T07:33:55.545262+0000","last_unstale":"2024-03-04T07:33:57.750225+0000","last_undegraded":"2024-03-04T07:33:57.750225+0000","last_fullsized":"2024-03-04T07:33:57.750225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:01.667243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523716+0000","last_change":"2024-03-04T07:33:56.550943+0000","last_active":"2024-03-04T07:33:57.523716+0000","last_peered":"2024-03-04T07:33:57.523716+0000","last_clean":"2024-03-04T07:33:57.523716+0000","last_became_active":"2024-03-04T07:33:56.550604+0000","last_became_peered":"2024-03-04T07:33:56.550604+0000","last_unstale":"2024-03-04T07:33:57.523716+0000","last_undegraded":"2024-03-04T07:33:57.523716+0000","last_fullsized":"2024-03-04T07:33:57.523716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:01:05.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.524536+0000","last_change":"2024-03-04T07:33:55.567220+0000","last_active":"2024-03-04T07:33:56.524536+0000","last_peered":"2024-03-04T07:33:56.524536+0000","last_clean":"2024-03-04T07:33:56.524536+0000","last_became_active":"2024-03-04T07:33:55.566600+0000","last_became_peered":"2024-03-04T07:33:55.566600+0000","last_unstale":"2024-03-04T07:33:56.524536+0000","last_undegraded":"2024-03-04T07:33:56.524536+0000","last_fullsized":"2024-03-04T07:33:56.524536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.876938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.545896+0000","last_change":"2024-03-04T07:33:56.544263+0000","last_active":"2024-03-04T07:33:56.545896+0000","last_peered":"2024-03-04T07:33:56.545896+0000","last_clean":"2024-03-04T07:33:56.545896+0000","last_became_active":"2024-03-04T07:33:56.543836+0000","last_became_peered":"2024-03-04T07:33:56.543836+0000","last_unstale":"2024-03-04T07:33:56.545896+0000","last_undegraded":"2024-03-04T07:33:56.545896+0000","last_fullsized":"2024-03-04T07:33:56.545896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:13:15.958853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.531709+0000","last_change":"2024-03-04T07:33:55.566886+0000","last_active":"2024-03-04T07:33:56.531709+0000","last_peered":"2024-03-04T07:33:56.531709+0000","last_clean":"2024-03-04T07:33:56.531709+0000","last_became_active":"2024-03-04T07:33:55.566048+0000","last_became_peered":"2024-03-04T07:33:55.566048+0000","last_unstale":"2024-03-04T07:33:56.531709+0000","last_undegraded":"2024-03-04T07:33:56.531709+0000","last_fullsized":"2024-03-04T07:33:56.531709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:45:27.190438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.539368+0000","last_change":"2024-03-04T07:33:56.532666+0000","last_active":"2024-03-04T07:33:56.539368+0000","last_peered":"2024-03-04T07:33:56.539368+0000","last_clean":"2024-03-04T07:33:56.539368+0000","last_became_active":"2024-03-04T07:33:56.531901+0000","last_became_peered":"2024-03-04T07:33:56.531901+0000","last_unstale":"2024-03-04T07:33:56.539368+0000","last_undegraded":"2024-03-04T07:33:56.539368+0000","last_fullsized":"2024-03-04T07:33:56.539368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:50:56.653702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750078+0000","last_change":"2024-03-04T07:33:55.557568+0000","last_active":"2024-03-04T07:33:57.750078+0000","last_peered":"2024-03-04T07:33:57.750078+0000","last_clean":"2024-03-04T07:33:57.750078+0000","last_became_active":"2024-03-04T07:33:55.557330+0000","last_became_peered":"2024-03-04T07:33:55.557330+0000","last_unstale":"2024-03-04T07:33:57.750078+0000","last_undegraded":"2024-03-04T07:33:57.750078+0000","last_fullsized":"2024-03-04T07:33:57.750078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:34.956318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523904+0000","last_change":"2024-03-04T07:33:56.552963+0000","last_active":"2024-03-04T07:33:57.523904+0000","last_peered":"2024-03-04T07:33:57.523904+0000","last_clean":"2024-03-04T07:33:57.523904+0000","last_became_active":"2024-03-04T07:33:56.552658+0000","last_became_peered":"2024-03-04T07:33:56.552658+0000","last_unstale":"2024-03-04T07:33:57.523904+0000","last_undegraded":"2024-03-04T07:33:57.523904+0000","last_fullsized":"2024-03-04T07:33:57.523904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:05.685496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750081+0000","last_change":"2024-03-04T07:33:55.536480+0000","last_active":"2024-03-04T07:33:57.750081+0000","last_peered":"2024-03-04T07:33:57.750081+0000","last_clean":"2024-03-04T07:33:57.750081+0000","last_became_active":"2024-03-04T07:33:55.536139+0000","last_became_peered":"2024-03-04T07:33:55.536139+0000","last_unstale":"2024-03-04T07:33:57.750081+0000","last_undegraded":"2024-03-04T07:33:57.750081+0000","last_fullsized":"2024-03-04T07:33:57.750081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:12.778026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523811+0000","last_change":"2024-03-04T07:33:56.546546+0000","last_active":"2024-03-04T07:33:57.523811+0000","last_peered":"2024-03-04T07:33:57.523811+0000","last_clean":"2024-03-04T07:33:57.523811+0000","last_became_active":"2024-03-04T07:33:56.546280+0000","last_became_peered":"2024-03-04T07:33:56.546280+0000","last_unstale":"2024-03-04T07:33:57.523811+0000","last_undegraded":"2024-03-04T07:33:57.523811+0000","last_fullsized":"2024-03-04T07:33:57.523811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:19:55.950163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.535401+0000","last_change":"2024-03-04T07:33:55.572540+0000","last_active":"2024-03-04T07:33:56.535401+0000","last_peered":"2024-03-04T07:33:56.535401+0000","last_clean":"2024-03-04T07:33:56.535401+0000","last_became_active":"2024-03-04T07:33:55.571612+0000","last_became_peered":"2024-03-04T07:33:55.571612+0000","last_unstale":"2024-03-04T07:33:56.535401+0000","last_undegraded":"2024-03-04T07:33:56.535401+0000","last_fullsized":"2024-03-04T07:33:56.535401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:01:43.741423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.556085+0000","last_change":"2024-03-04T07:33:56.532855+0000","last_active":"2024-03-04T07:33:56.556085+0000","last_peered":"2024-03-04T07:33:56.556085+0000","last_clean":"2024-03-04T07:33:56.556085+0000","last_became_active":"2024-03-04T07:33:56.532245+0000","last_became_peered":"2024-03-04T07:33:56.532245+0000","last_unstale":"2024-03-04T07:33:56.556085+0000","last_undegraded":"2024-03-04T07:33:56.556085+0000","last_fullsized":"2024-03-04T07:33:56.556085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:52:07.456511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.568498+0000","last_change":"2024-03-04T07:33:56.563803+0000","last_active":"2024-03-04T07:33:56.568498+0000","last_peered":"2024-03-04T07:33:56.568498+0000","last_clean":"2024-03-04T07:33:56.568498+0000","last_became_active":"2024-03-04T07:33:56.563473+0000","last_became_peered":"2024-03-04T07:33:56.563473+0000","last_unstale":"2024-03-04T07:33:56.568498+0000","last_undegraded":"2024-03-04T07:33:56.568498+0000","last_fullsized":"2024-03-04T07:33:56.568498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:41.267667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.533193+0000","last_change":"2024-03-04T07:33:55.575688+0000","last_active":"2024-03-04T07:33:56.533193+0000","last_peered":"2024-03-04T07:33:56.533193+0000","last_clean":"2024-03-04T07:33:56.533193+0000","last_became_active":"2024-03-04T07:33:55.574766+0000","last_became_peered":"2024-03-04T07:33:55.574766+0000","last_unstale":"2024-03-04T07:33:56.533193+0000","last_undegraded":"2024-03-04T07:33:56.533193+0000","last_fullsized":"2024-03-04T07:33:56.533193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:32:19.708016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.568497+0000","last_change":"2024-03-04T07:33:56.563798+0000","last_active":"2024-03-04T07:33:56.568497+0000","last_peered":"2024-03-04T07:33:56.568497+0000","last_clean":"2024-03-04T07:33:56.568497+0000","last_became_active":"2024-03-04T07:33:56.563474+0000","last_became_peered":"2024-03-04T07:33:56.563474+0000","last_unstale":"2024-03-04T07:33:56.568497+0000","last_undegraded":"2024-03-04T07:33:56.568497+0000","last_fullsized":"2024-03-04T07:33:56.568497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:18:24.399645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.535855+0000","last_change":"2024-03-04T07:33:55.570016+0000","last_active":"2024-03-04T07:33:56.535855+0000","last_peered":"2024-03-04T07:33:56.535855+0000","last_clean":"2024-03-04T07:33:56.535855+0000","last_became_active":"2024-03-04T07:33:55.568749+0000","last_became_peered":"2024-03-04T07:33:55.568749+0000","last_unstale":"2024-03-04T07:33:56.535855+0000","last_undegraded":"2024-03-04T07:33:56.535855+0000","last_fullsized":"2024-03-04T07:33:56.535855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:01.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.563345+0000","last_change":"2024-03-04T07:33:56.562015+0000","last_active":"2024-03-04T07:33:56.563345+0000","last_peered":"2024-03-04T07:33:56.563345+0000","last_clean":"2024-03-04T07:33:56.563345+0000","last_became_active":"2024-03-04T07:33:56.561675+0000","last_became_peered":"2024-03-04T07:33:56.561675+0000","last_unstale":"2024-03-04T07:33:56.563345+0000","last_undegraded":"2024-03-04T07:33:56.563345+0000","last_fullsized":"2024-03-04T07:33:56.563345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:05:03.773019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.524108+0000","last_change":"2024-03-04T07:33:55.538337+0000","last_active":"2024-03-04T07:33:57.524108+0000","last_peered":"2024-03-04T07:33:57.524108+0000","last_clean":"2024-03-04T07:33:57.524108+0000","last_became_active":"2024-03-04T07:33:55.538010+0000","last_became_peered":"2024-03-04T07:33:55.538010+0000","last_unstale":"2024-03-04T07:33:57.524108+0000","last_undegraded":"2024-03-04T07:33:57.524108+0000","last_fullsized":"2024-03-04T07:33:57.524108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:47:15.711205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.524079+0000","last_change":"2024-03-04T07:33:56.559051+0000","last_active":"2024-03-04T07:33:57.524079+0000","last_peered":"2024-03-04T07:33:57.524079+0000","last_clean":"2024-03-04T07:33:57.524079+0000","last_became_active":"2024-03-04T07:33:56.558799+0000","last_became_peered":"2024-03-04T07:33:56.558799+0000","last_unstale":"2024-03-04T07:33:57.524079+0000","last_undegraded":"2024-03-04T07:33:57.524079+0000","last_fullsized":"2024-03-04T07:33:57.524079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:16.516631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.528987+0000","last_change":"2024-03-04T07:33:55.566294+0000","last_active":"2024-03-04T07:33:56.528987+0000","last_peered":"2024-03-04T07:33:56.528987+0000","last_clean":"2024-03-04T07:33:56.528987+0000","last_became_active":"2024-03-04T07:33:55.565941+0000","last_became_peered":"2024-03-04T07:33:55.565941+0000","last_unstale":"2024-03-04T07:33:56.528987+0000","last_undegraded":"2024-03-04T07:33:56.528987+0000","last_fullsized":"2024-03-04T07:33:56.528987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:03.115085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":67229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":790528,"data_stored":657597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":67229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":67229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":657597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T07:34:02.658 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T07:34:02.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a8e400640 1 Processor -- start 2024-03-04T07:34:02.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a8e400640 1 -- start start 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a8e400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1a88152a50 0x7f1a880ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a8e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a8806ff20 0x7f1a880ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a8e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a880c8b90 0x7f1a8815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a8e400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a880c31b0 con 0x7f1a880c8b90 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a8e400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a8806f840 con 0x7f1a88152a50 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a8e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a88152e30 con 0x7f1a8806ff20 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a87e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a880c8b90 0x7f1a8815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a87e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a880c8b90 0x7f1a8815bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57048/0 (socket says 172.21.15.113:57048) 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.826+0000 7f1a87e00640 1 -- 172.21.15.113:0/3629212367 learned_addr learned my addr 172.21.15.113:0/3629212367 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a7ea00640 1 --2- 172.21.15.113:0/3629212367 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a8806ff20 0x7f1a880ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a87400640 1 --2- 172.21.15.113:0/3629212367 >> v2:172.21.15.136:3300/0 conn(0x7f1a88152a50 0x7f1a880ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a87e00640 1 -- 172.21.15.113:0/3629212367 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a8806ff20 msgr2=0x7f1a880ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:02.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a87e00640 1 --2- 172.21.15.113:0/3629212367 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a8806ff20 0x7f1a880ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a87e00640 1 -- 172.21.15.113:0/3629212367 >> v2:172.21.15.136:3300/0 conn(0x7f1a88152a50 msgr2=0x7f1a880ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:02.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a87e00640 1 --2- 172.21.15.113:0/3629212367 >> v2:172.21.15.136:3300/0 conn(0x7f1a88152a50 0x7f1a880ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a87e00640 1 -- 172.21.15.113:0/3629212367 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8806f280 con 0x7f1a880c8b90 2024-03-04T07:34:02.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a87e00640 1 --2- 172.21.15.113:0/3629212367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 0x7f1a8815bdd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a88067cd0 tx=0x7f1a8001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e31281b01ced3e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:02.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a86a00640 1 -- 172.21.15.113:0/3629212367 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a800a57e0 con 0x7f1a880c8b90 2024-03-04T07:34:02.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a86a00640 1 -- 172.21.15.113:0/3629212367 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a800a5980 con 0x7f1a880c8b90 2024-03-04T07:34:02.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a86a00640 1 -- 172.21.15.113:0/3629212367 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a800a5d50 con 0x7f1a880c8b90 2024-03-04T07:34:02.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a8e400640 1 -- 172.21.15.113:0/3629212367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 msgr2=0x7f1a8815bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a8e400640 1 --2- 172.21.15.113:0/3629212367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 0x7f1a8815bdd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1a88067cd0 tx=0x7f1a8001c960 comp rx=0 tx=0).stop 2024-03-04T07:34:02.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a8e400640 1 -- 172.21.15.113:0/3629212367 shutdown_connections 2024-03-04T07:34:02.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a8e400640 1 --2- 172.21.15.113:0/3629212367 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 0x7f1a8815bdd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a8e400640 1 --2- 172.21.15.113:0/3629212367 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a8806ff20 0x7f1a880ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a8e400640 1 --2- 172.21.15.113:0/3629212367 >> v2:172.21.15.136:3300/0 conn(0x7f1a88152a50 0x7f1a880ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.830+0000 7f1a8e400640 1 -- 172.21.15.113:0/3629212367 >> 172.21.15.113:0/3629212367 conn(0x7f1a880734d0 msgr2=0x7f1a88066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:02.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 -- 172.21.15.113:0/3629212367 shutdown_connections 2024-03-04T07:34:02.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 -- 172.21.15.113:0/3629212367 wait complete. 2024-03-04T07:34:02.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 Processor -- start 2024-03-04T07:34:02.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 -- start start 2024-03-04T07:34:02.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a8806ff20 0x7f1a880cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 0x7f1a880c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a88152a50 0x7f1a881f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a88158f10 con 0x7f1a880c8b90 2024-03-04T07:34:02.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a880c3400 con 0x7f1a8806ff20 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a88159090 con 0x7f1a88152a50 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 0x7f1a880c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 0x7f1a880c6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57062/0 (socket says 172.21.15.113:57062) 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7ea00640 1 -- 172.21.15.113:0/3773474629 learned_addr learned my addr 172.21.15.113:0/3773474629 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a87e00640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a88152a50 0x7f1a881f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a87400640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a8806ff20 0x7f1a880cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7ea00640 1 -- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a88152a50 msgr2=0x7f1a881f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7ea00640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a88152a50 0x7f1a881f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7ea00640 1 -- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a8806ff20 msgr2=0x7f1a880cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7ea00640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a8806ff20 0x7f1a880cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7ea00640 1 -- 172.21.15.113:0/3773474629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a880cd8c0 con 0x7f1a880c8b90 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a87400640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a8806ff20 0x7f1a880cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7ea00640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 0x7f1a880c6be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000d800 tx=0x7f1a7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7fe00640 1 -- 172.21.15.113:0/3773474629 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70018020 con 0x7f1a880c8b90 2024-03-04T07:34:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7fe00640 1 -- 172.21.15.113:0/3773474629 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a700096a0 con 0x7f1a880c8b90 2024-03-04T07:34:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 -- 172.21.15.113:0/3773474629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a880cdba0 con 0x7f1a880c8b90 2024-03-04T07:34:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a7fe00640 1 -- 172.21.15.113:0/3773474629 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a70009a50 con 0x7f1a880c8b90 2024-03-04T07:34:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.834+0000 7f1a8e400640 1 -- 172.21.15.113:0/3773474629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a881f3cb0 con 0x7f1a880c8b90 2024-03-04T07:34:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.838+0000 7f1a8e400640 1 -- 172.21.15.113:0/3773474629 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a88075c00 con 0x7f1a880c8b90 2024-03-04T07:34:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.838+0000 7f1a7fe00640 1 -- 172.21.15.113:0/3773474629 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a7001b020 con 0x7f1a880c8b90 2024-03-04T07:34:02.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.838+0000 7f1a7fe00640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a64066f60 0x7f1a64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:02.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.838+0000 7f1a7fe00640 1 -- 172.21.15.113:0/3773474629 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a70085740 con 0x7f1a880c8b90 2024-03-04T07:34:02.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.838+0000 7f1a87400640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a64066f60 0x7f1a64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:02.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.838+0000 7f1a87400640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a64066f60 0x7f1a64069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1a780097a0 tx=0x7f1a78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:02.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.842+0000 7f1a7fe00640 1 -- 172.21.15.113:0/3773474629 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1a70022020 con 0x7f1a880c8b90 2024-03-04T07:34:02.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:02.994+0000 7f1a8e400640 1 -- 172.21.15.113:0/3773474629 --> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a8815caf0 con 0x7f1a64066f60 2024-03-04T07:34:03.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.014+0000 7f1a7fe00640 1 -- 172.21.15.113:0/3773474629 <== mgr.4103 v2:172.21.15.136:6832/3353359815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251798 (secure 0 0 0) 0x7f1a8815caf0 con 0x7f1a64066f60 2024-03-04T07:34:03.019 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 -- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a64066f60 msgr2=0x7f1a64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a64066f60 0x7f1a64069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1a780097a0 tx=0x7f1a78008040 comp rx=0 tx=0).stop 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 -- 172.21.15.113:0/3773474629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 msgr2=0x7f1a880c6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 0x7f1a880c6be0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1a7000d800 tx=0x7f1a7000dcc0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 -- 172.21.15.113:0/3773474629 shutdown_connections 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a88152a50 0x7f1a881f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a64066f60 0x7f1a64069420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a880c8b90 0x7f1a880c6be0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 --2- 172.21.15.113:0/3773474629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a8806ff20 0x7f1a880cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 -- 172.21.15.113:0/3773474629 >> 172.21.15.113:0/3773474629 conn(0x7f1a880734d0 msgr2=0x7f1a88066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 -- 172.21.15.113:0/3773474629 shutdown_connections 2024-03-04T07:34:03.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.018+0000 7f1a8e400640 1 -- 172.21.15.113:0/3773474629 wait complete. 2024-03-04T07:34:03.039 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-04T07:34:01.434431+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223024,"kb_used_data":2684,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751696,"statfs":{"total":773094113280,"available":772865736704,"internally_reserved":0,"allocated":2748416,"data_stored":1711318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002150"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.565710+0000","last_change":"2024-03-04T07:33:56.559432+0000","last_active":"2024-03-04T07:33:56.565710+0000","last_peered":"2024-03-04T07:33:56.565710+0000","last_clean":"2024-03-04T07:33:56.565710+0000","last_became_active":"2024-03-04T07:33:56.559031+0000","last_became_peered":"2024-03-04T07:33:56.559031+0000","last_unstale":"2024-03-04T07:33:56.565710+0000","last_undegraded":"2024-03-04T07:33:56.565710+0000","last_fullsized":"2024-03-04T07:33:56.565710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:50:27.941814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750282+0000","last_change":"2024-03-04T07:33:56.546822+0000","last_active":"2024-03-04T07:33:57.750282+0000","last_peered":"2024-03-04T07:33:57.750282+0000","last_clean":"2024-03-04T07:33:57.750282+0000","last_became_active":"2024-03-04T07:33:56.546547+0000","last_became_peered":"2024-03-04T07:33:56.546547+0000","last_unstale":"2024-03-04T07:33:57.750282+0000","last_undegraded":"2024-03-04T07:33:57.750282+0000","last_fullsized":"2024-03-04T07:33:57.750282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:06.332337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.541032+0000","last_change":"2024-03-04T07:33:56.530803+0000","last_active":"2024-03-04T07:33:56.541032+0000","last_peered":"2024-03-04T07:33:56.541032+0000","last_clean":"2024-03-04T07:33:56.541032+0000","last_became_active":"2024-03-04T07:33:56.530488+0000","last_became_peered":"2024-03-04T07:33:56.530488+0000","last_unstale":"2024-03-04T07:33:56.541032+0000","last_undegraded":"2024-03-04T07:33:56.541032+0000","last_fullsized":"2024-03-04T07:33:56.541032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:15:19.032824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.561762+0000","last_change":"2024-03-04T07:33:56.558669+0000","last_active":"2024-03-04T07:33:56.561762+0000","last_peered":"2024-03-04T07:33:56.561762+0000","last_clean":"2024-03-04T07:33:56.561762+0000","last_became_active":"2024-03-04T07:33:56.558350+0000","last_became_peered":"2024-03-04T07:33:56.558350+0000","last_unstale":"2024-03-04T07:33:56.561762+0000","last_undegraded":"2024-03-04T07:33:56.561762+0000","last_fullsized":"2024-03-04T07:33:56.561762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:37:17.358787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.551730+0000","last_change":"2024-03-04T07:33:56.540760+0000","last_active":"2024-03-04T07:33:56.551730+0000","last_peered":"2024-03-04T07:33:56.551730+0000","last_clean":"2024-03-04T07:33:56.551730+0000","last_became_active":"2024-03-04T07:33:56.540351+0000","last_became_peered":"2024-03-04T07:33:56.540351+0000","last_unstale":"2024-03-04T07:33:56.551730+0000","last_undegraded":"2024-03-04T07:33:56.551730+0000","last_fullsized":"2024-03-04T07:33:56.551730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:36:55.087150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.559912+0000","last_change":"2024-03-04T07:33:56.558333+0000","last_active":"2024-03-04T07:33:56.559912+0000","last_peered":"2024-03-04T07:33:56.559912+0000","last_clean":"2024-03-04T07:33:56.559912+0000","last_became_active":"2024-03-04T07:33:56.558046+0000","last_became_peered":"2024-03-04T07:33:56.558046+0000","last_unstale":"2024-03-04T07:33:56.559912+0000","last_undegraded":"2024-03-04T07:33:56.559912+0000","last_fullsized":"2024-03-04T07:33:56.559912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:04:00.752661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.551431+0000","last_change":"2024-03-04T07:33:56.543476+0000","last_active":"2024-03-04T07:33:56.551431+0000","last_peered":"2024-03-04T07:33:56.551431+0000","last_clean":"2024-03-04T07:33:56.551431+0000","last_became_active":"2024-03-04T07:33:56.543176+0000","last_became_peered":"2024-03-04T07:33:56.543176+0000","last_unstale":"2024-03-04T07:33:56.551431+0000","last_undegraded":"2024-03-04T07:33:56.551431+0000","last_fullsized":"2024-03-04T07:33:56.551431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:42:32.313196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523950+0000","last_change":"2024-03-04T07:33:56.559488+0000","last_active":"2024-03-04T07:33:57.523950+0000","last_peered":"2024-03-04T07:33:57.523950+0000","last_clean":"2024-03-04T07:33:57.523950+0000","last_became_active":"2024-03-04T07:33:56.559172+0000","last_became_peered":"2024-03-04T07:33:56.559172+0000","last_unstale":"2024-03-04T07:33:57.523950+0000","last_undegraded":"2024-03-04T07:33:57.523950+0000","last_fullsized":"2024-03-04T07:33:57.523950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:53.461697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.565952+0000","last_change":"2024-03-04T07:33:56.560587+0000","last_active":"2024-03-04T07:33:56.565952+0000","last_peered":"2024-03-04T07:33:56.565952+0000","last_clean":"2024-03-04T07:33:56.565952+0000","last_became_active":"2024-03-04T07:33:56.560306+0000","last_became_peered":"2024-03-04T07:33:56.560306+0000","last_unstale":"2024-03-04T07:33:56.565952+0000","last_undegraded":"2024-03-04T07:33:56.565952+0000","last_fullsized":"2024-03-04T07:33:56.565952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:17.459841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.562088+0000","last_change":"2024-03-04T07:33:56.560852+0000","last_active":"2024-03-04T07:33:56.562088+0000","last_peered":"2024-03-04T07:33:56.562088+0000","last_clean":"2024-03-04T07:33:56.562088+0000","last_became_active":"2024-03-04T07:33:56.560592+0000","last_became_peered":"2024-03-04T07:33:56.560592+0000","last_unstale":"2024-03-04T07:33:56.562088+0000","last_undegraded":"2024-03-04T07:33:56.562088+0000","last_fullsized":"2024-03-04T07:33:56.562088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:01:50.310438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.562094+0000","last_change":"2024-03-04T07:33:56.561040+0000","last_active":"2024-03-04T07:33:56.562094+0000","last_peered":"2024-03-04T07:33:56.562094+0000","last_clean":"2024-03-04T07:33:56.562094+0000","last_became_active":"2024-03-04T07:33:56.560719+0000","last_became_peered":"2024-03-04T07:33:56.560719+0000","last_unstale":"2024-03-04T07:33:56.562094+0000","last_undegraded":"2024-03-04T07:33:56.562094+0000","last_fullsized":"2024-03-04T07:33:56.562094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:51:07.869500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.563152+0000","last_change":"2024-03-04T07:33:56.550648+0000","last_active":"2024-03-04T07:33:56.563152+0000","last_peered":"2024-03-04T07:33:56.563152+0000","last_clean":"2024-03-04T07:33:56.563152+0000","last_became_active":"2024-03-04T07:33:56.549568+0000","last_became_peered":"2024-03-04T07:33:56.549568+0000","last_unstale":"2024-03-04T07:33:56.563152+0000","last_undegraded":"2024-03-04T07:33:56.563152+0000","last_fullsized":"2024-03-04T07:33:56.563152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:12:46.537347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523777+0000","last_change":"2024-03-04T07:33:56.558880+0000","last_active":"2024-03-04T07:33:57.523777+0000","last_peered":"2024-03-04T07:33:57.523777+0000","last_clean":"2024-03-04T07:33:57.523777+0000","last_became_active":"2024-03-04T07:33:56.558567+0000","last_became_peered":"2024-03-04T07:33:56.558567+0000","last_unstale":"2024-03-04T07:33:57.523777+0000","last_undegraded":"2024-03-04T07:33:57.523777+0000","last_fullsized":"2024-03-04T07:33:57.523777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:16.354787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.546040+0000","last_change":"2024-03-04T07:33:56.544222+0000","last_active":"2024-03-04T07:33:56.546040+0000","last_peered":"2024-03-04T07:33:56.546040+0000","last_clean":"2024-03-04T07:33:56.546040+0000","last_became_active":"2024-03-04T07:33:56.543936+0000","last_became_peered":"2024-03-04T07:33:56.543936+0000","last_unstale":"2024-03-04T07:33:56.546040+0000","last_undegraded":"2024-03-04T07:33:56.546040+0000","last_fullsized":"2024-03-04T07:33:56.546040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:20:34.741592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749980+0000","last_change":"2024-03-04T07:33:56.545195+0000","last_active":"2024-03-04T07:33:57.749980+0000","last_peered":"2024-03-04T07:33:57.749980+0000","last_clean":"2024-03-04T07:33:57.749980+0000","last_became_active":"2024-03-04T07:33:56.544837+0000","last_became_peered":"2024-03-04T07:33:56.544837+0000","last_unstale":"2024-03-04T07:33:57.749980+0000","last_undegraded":"2024-03-04T07:33:57.749980+0000","last_fullsized":"2024-03-04T07:33:57.749980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:24:59.643746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.559283+0000","last_change":"2024-03-04T07:33:56.558398+0000","last_active":"2024-03-04T07:33:56.559283+0000","last_peered":"2024-03-04T07:33:56.559283+0000","last_clean":"2024-03-04T07:33:56.559283+0000","last_became_active":"2024-03-04T07:33:56.557962+0000","last_became_peered":"2024-03-04T07:33:56.557962+0000","last_unstale":"2024-03-04T07:33:56.559283+0000","last_undegraded":"2024-03-04T07:33:56.559283+0000","last_fullsized":"2024-03-04T07:33:56.559283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:47:39.055655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.563056+0000","last_change":"2024-03-04T07:33:56.559597+0000","last_active":"2024-03-04T07:33:56.563056+0000","last_peered":"2024-03-04T07:33:56.563056+0000","last_clean":"2024-03-04T07:33:56.563056+0000","last_became_active":"2024-03-04T07:33:56.558719+0000","last_became_peered":"2024-03-04T07:33:56.558719+0000","last_unstale":"2024-03-04T07:33:56.563056+0000","last_undegraded":"2024-03-04T07:33:56.563056+0000","last_fullsized":"2024-03-04T07:33:56.563056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:27:12.896606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.532751+0000","last_change":"2024-03-04T07:33:56.528479+0000","last_active":"2024-03-04T07:33:56.532751+0000","last_peered":"2024-03-04T07:33:56.532751+0000","last_clean":"2024-03-04T07:33:56.532751+0000","last_became_active":"2024-03-04T07:33:56.528213+0000","last_became_peered":"2024-03-04T07:33:56.528213+0000","last_unstale":"2024-03-04T07:33:56.532751+0000","last_undegraded":"2024-03-04T07:33:56.532751+0000","last_fullsized":"2024-03-04T07:33:56.532751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:01:21.785006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.552549+0000","last_change":"2024-03-04T07:33:56.545890+0000","last_active":"2024-03-04T07:33:56.552549+0000","last_peered":"2024-03-04T07:33:56.552549+0000","last_clean":"2024-03-04T07:33:56.552549+0000","last_became_active":"2024-03-04T07:33:56.545467+0000","last_became_peered":"2024-03-04T07:33:56.545467+0000","last_unstale":"2024-03-04T07:33:56.552549+0000","last_undegraded":"2024-03-04T07:33:56.552549+0000","last_fullsized":"2024-03-04T07:33:56.552549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:14:59.100678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.558357+0000","last_change":"2024-03-04T07:33:56.549059+0000","last_active":"2024-03-04T07:33:56.558357+0000","last_peered":"2024-03-04T07:33:56.558357+0000","last_clean":"2024-03-04T07:33:56.558357+0000","last_became_active":"2024-03-04T07:33:56.548783+0000","last_became_peered":"2024-03-04T07:33:56.548783+0000","last_unstale":"2024-03-04T07:33:56.558357+0000","last_undegraded":"2024-03-04T07:33:56.558357+0000","last_fullsized":"2024-03-04T07:33:56.558357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:57:39.713413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.545819+0000","last_change":"2024-03-04T07:33:56.540558+0000","last_active":"2024-03-04T07:33:56.545819+0000","last_peered":"2024-03-04T07:33:56.545819+0000","last_clean":"2024-03-04T07:33:56.545819+0000","last_became_active":"2024-03-04T07:33:56.539403+0000","last_became_peered":"2024-03-04T07:33:56.539403+0000","last_unstale":"2024-03-04T07:33:56.545819+0000","last_undegraded":"2024-03-04T07:33:56.545819+0000","last_fullsized":"2024-03-04T07:33:56.545819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:58:50.040860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.559037+0000","last_change":"2024-03-04T07:33:56.554982+0000","last_active":"2024-03-04T07:33:56.559037+0000","last_peered":"2024-03-04T07:33:56.559037+0000","last_clean":"2024-03-04T07:33:56.559037+0000","last_became_active":"2024-03-04T07:33:56.554470+0000","last_became_peered":"2024-03-04T07:33:56.554470+0000","last_unstale":"2024-03-04T07:33:56.559037+0000","last_undegraded":"2024-03-04T07:33:56.559037+0000","last_fullsized":"2024-03-04T07:33:56.559037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:00:11.441784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523496+0000","last_change":"2024-03-04T07:33:56.545465+0000","last_active":"2024-03-04T07:33:57.523496+0000","last_peered":"2024-03-04T07:33:57.523496+0000","last_clean":"2024-03-04T07:33:57.523496+0000","last_became_active":"2024-03-04T07:33:56.545270+0000","last_became_peered":"2024-03-04T07:33:56.545270+0000","last_unstale":"2024-03-04T07:33:57.523496+0000","last_undegraded":"2024-03-04T07:33:57.523496+0000","last_fullsized":"2024-03-04T07:33:57.523496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:08:45.244868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.559074+0000","last_change":"2024-03-04T07:33:56.555428+0000","last_active":"2024-03-04T07:33:56.559074+0000","last_peered":"2024-03-04T07:33:56.559074+0000","last_clean":"2024-03-04T07:33:56.559074+0000","last_became_active":"2024-03-04T07:33:56.554942+0000","last_became_peered":"2024-03-04T07:33:56.554942+0000","last_unstale":"2024-03-04T07:33:56.559074+0000","last_undegraded":"2024-03-04T07:33:56.559074+0000","last_fullsized":"2024-03-04T07:33:56.559074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:55:57.341319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523610+0000","last_change":"2024-03-04T07:33:56.549563+0000","last_active":"2024-03-04T07:33:57.523610+0000","last_peered":"2024-03-04T07:33:57.523610+0000","last_clean":"2024-03-04T07:33:57.523610+0000","last_became_active":"2024-03-04T07:33:56.548790+0000","last_became_peered":"2024-03-04T07:33:56.548790+0000","last_unstale":"2024-03-04T07:33:57.523610+0000","last_undegraded":"2024-03-04T07:33:57.523610+0000","last_fullsized":"2024-03-04T07:33:57.523610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:30:53.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.558883+0000","last_change":"2024-03-04T07:33:56.553791+0000","last_active":"2024-03-04T07:33:56.558883+0000","last_peered":"2024-03-04T07:33:56.558883+0000","last_clean":"2024-03-04T07:33:56.558883+0000","last_became_active":"2024-03-04T07:33:56.553012+0000","last_became_peered":"2024-03-04T07:33:56.553012+0000","last_unstale":"2024-03-04T07:33:56.558883+0000","last_undegraded":"2024-03-04T07:33:56.558883+0000","last_fullsized":"2024-03-04T07:33:56.558883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:53:21.626735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750163+0000","last_change":"2024-03-04T07:33:56.551341+0000","last_active":"2024-03-04T07:33:57.750163+0000","last_peered":"2024-03-04T07:33:57.750163+0000","last_clean":"2024-03-04T07:33:57.750163+0000","last_became_active":"2024-03-04T07:33:56.550616+0000","last_became_peered":"2024-03-04T07:33:56.550616+0000","last_unstale":"2024-03-04T07:33:57.750163+0000","last_undegraded":"2024-03-04T07:33:57.750163+0000","last_fullsized":"2024-03-04T07:33:57.750163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:35:45.801510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523840+0000","last_change":"2024-03-04T07:33:56.543903+0000","last_active":"2024-03-04T07:33:57.523840+0000","last_peered":"2024-03-04T07:33:57.523840+0000","last_clean":"2024-03-04T07:33:57.523840+0000","last_became_active":"2024-03-04T07:33:56.543632+0000","last_became_peered":"2024-03-04T07:33:56.543632+0000","last_unstale":"2024-03-04T07:33:57.523840+0000","last_undegraded":"2024-03-04T07:33:57.523840+0000","last_fullsized":"2024-03-04T07:33:57.523840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:26:10.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.550195+0000","last_change":"2024-03-04T07:33:56.547091+0000","last_active":"2024-03-04T07:33:56.550195+0000","last_peered":"2024-03-04T07:33:56.550195+0000","last_clean":"2024-03-04T07:33:56.550195+0000","last_became_active":"2024-03-04T07:33:56.546652+0000","last_became_peered":"2024-03-04T07:33:56.546652+0000","last_unstale":"2024-03-04T07:33:56.550195+0000","last_undegraded":"2024-03-04T07:33:56.550195+0000","last_fullsized":"2024-03-04T07:33:56.550195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:02:20.861575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.559018+0000","last_change":"2024-03-04T07:33:56.552493+0000","last_active":"2024-03-04T07:33:56.559018+0000","last_peered":"2024-03-04T07:33:56.559018+0000","last_clean":"2024-03-04T07:33:56.559018+0000","last_became_active":"2024-03-04T07:33:56.552189+0000","last_became_peered":"2024-03-04T07:33:56.552189+0000","last_unstale":"2024-03-04T07:33:56.559018+0000","last_undegraded":"2024-03-04T07:33:56.559018+0000","last_fullsized":"2024-03-04T07:33:56.559018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:49:36.688567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.552470+0000","last_change":"2024-03-04T07:33:56.549661+0000","last_active":"2024-03-04T07:33:56.552470+0000","last_peered":"2024-03-04T07:33:56.552470+0000","last_clean":"2024-03-04T07:33:56.552470+0000","last_became_active":"2024-03-04T07:33:56.549351+0000","last_became_peered":"2024-03-04T07:33:56.549351+0000","last_unstale":"2024-03-04T07:33:56.552470+0000","last_undegraded":"2024-03-04T07:33:56.552470+0000","last_fullsized":"2024-03-04T07:33:56.552470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:29:48.841517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.556093+0000","last_change":"2024-03-04T07:33:56.533132+0000","last_active":"2024-03-04T07:33:56.556093+0000","last_peered":"2024-03-04T07:33:56.556093+0000","last_clean":"2024-03-04T07:33:56.556093+0000","last_became_active":"2024-03-04T07:33:56.532251+0000","last_became_peered":"2024-03-04T07:33:56.532251+0000","last_unstale":"2024-03-04T07:33:56.556093+0000","last_undegraded":"2024-03-04T07:33:56.556093+0000","last_fullsized":"2024-03-04T07:33:56.556093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:30:00.806315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.545050+0000","last_change":"2024-03-04T07:33:56.543382+0000","last_active":"2024-03-04T07:33:56.545050+0000","last_peered":"2024-03-04T07:33:56.545050+0000","last_clean":"2024-03-04T07:33:56.545050+0000","last_became_active":"2024-03-04T07:33:56.542937+0000","last_became_peered":"2024-03-04T07:33:56.542937+0000","last_unstale":"2024-03-04T07:33:56.545050+0000","last_undegraded":"2024-03-04T07:33:56.545050+0000","last_fullsized":"2024-03-04T07:33:56.545050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:32:05.211873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523876+0000","last_change":"2024-03-04T07:33:55.566130+0000","last_active":"2024-03-04T07:33:57.523876+0000","last_peered":"2024-03-04T07:33:57.523876+0000","last_clean":"2024-03-04T07:33:57.523876+0000","last_became_active":"2024-03-04T07:33:55.565894+0000","last_became_peered":"2024-03-04T07:33:55.565894+0000","last_unstale":"2024-03-04T07:33:57.523876+0000","last_undegraded":"2024-03-04T07:33:57.523876+0000","last_fullsized":"2024-03-04T07:33:57.523876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:36.530949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.545797+0000","last_change":"2024-03-04T07:33:56.540113+0000","last_active":"2024-03-04T07:33:56.545797+0000","last_peered":"2024-03-04T07:33:56.545797+0000","last_clean":"2024-03-04T07:33:56.545797+0000","last_became_active":"2024-03-04T07:33:56.539738+0000","last_became_peered":"2024-03-04T07:33:56.539738+0000","last_unstale":"2024-03-04T07:33:56.545797+0000","last_undegraded":"2024-03-04T07:33:56.545797+0000","last_fullsized":"2024-03-04T07:33:56.545797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:46:17.799528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.524724+0000","last_change":"2024-03-04T07:33:55.554956+0000","last_active":"2024-03-04T07:33:56.524724+0000","last_peered":"2024-03-04T07:33:56.524724+0000","last_clean":"2024-03-04T07:33:56.524724+0000","last_became_active":"2024-03-04T07:33:55.554151+0000","last_became_peered":"2024-03-04T07:33:55.554151+0000","last_unstale":"2024-03-04T07:33:56.524724+0000","last_undegraded":"2024-03-04T07:33:56.524724+0000","last_fullsized":"2024-03-04T07:33:56.524724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:06:19.621510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523529+0000","last_change":"2024-03-04T07:33:55.542200+0000","last_active":"2024-03-04T07:33:57.523529+0000","last_peered":"2024-03-04T07:33:57.523529+0000","last_clean":"2024-03-04T07:33:57.523529+0000","last_became_active":"2024-03-04T07:33:55.541902+0000","last_became_peered":"2024-03-04T07:33:55.541902+0000","last_unstale":"2024-03-04T07:33:57.523529+0000","last_undegraded":"2024-03-04T07:33:57.523529+0000","last_fullsized":"2024-03-04T07:33:57.523529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:55:50.251288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523733+0000","last_change":"2024-03-04T07:33:56.548380+0000","last_active":"2024-03-04T07:33:57.523733+0000","last_peered":"2024-03-04T07:33:57.523733+0000","last_clean":"2024-03-04T07:33:57.523733+0000","last_became_active":"2024-03-04T07:33:56.546087+0000","last_became_peered":"2024-03-04T07:33:56.546087+0000","last_unstale":"2024-03-04T07:33:57.523733+0000","last_undegraded":"2024-03-04T07:33:57.523733+0000","last_fullsized":"2024-03-04T07:33:57.523733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:06:43.538687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.549228+0000","last_change":"2024-03-04T07:33:55.568508+0000","last_active":"2024-03-04T07:33:56.549228+0000","last_peered":"2024-03-04T07:33:56.549228+0000","last_clean":"2024-03-04T07:33:56.549228+0000","last_became_active":"2024-03-04T07:33:55.566073+0000","last_became_peered":"2024-03-04T07:33:55.566073+0000","last_unstale":"2024-03-04T07:33:56.549228+0000","last_undegraded":"2024-03-04T07:33:56.549228+0000","last_fullsized":"2024-03-04T07:33:56.549228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:34:17.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749969+0000","last_change":"2024-03-04T07:33:56.552785+0000","last_active":"2024-03-04T07:33:57.749969+0000","last_peered":"2024-03-04T07:33:57.749969+0000","last_clean":"2024-03-04T07:33:57.749969+0000","last_became_active":"2024-03-04T07:33:56.552533+0000","last_became_peered":"2024-03-04T07:33:56.552533+0000","last_unstale":"2024-03-04T07:33:57.749969+0000","last_undegraded":"2024-03-04T07:33:57.749969+0000","last_fullsized":"2024-03-04T07:33:57.749969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:08.582237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.524288+0000","last_change":"2024-03-04T07:33:55.559395+0000","last_active":"2024-03-04T07:33:56.524288+0000","last_peered":"2024-03-04T07:33:56.524288+0000","last_clean":"2024-03-04T07:33:56.524288+0000","last_became_active":"2024-03-04T07:33:55.558593+0000","last_became_peered":"2024-03-04T07:33:55.558593+0000","last_unstale":"2024-03-04T07:33:56.524288+0000","last_undegraded":"2024-03-04T07:33:56.524288+0000","last_fullsized":"2024-03-04T07:33:56.524288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:00:20.205445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.558878+0000","last_change":"2024-03-04T07:33:56.553260+0000","last_active":"2024-03-04T07:33:56.558878+0000","last_peered":"2024-03-04T07:33:56.558878+0000","last_clean":"2024-03-04T07:33:56.558878+0000","last_became_active":"2024-03-04T07:33:56.552952+0000","last_became_peered":"2024-03-04T07:33:56.552952+0000","last_unstale":"2024-03-04T07:33:56.558878+0000","last_undegraded":"2024-03-04T07:33:56.558878+0000","last_fullsized":"2024-03-04T07:33:56.558878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:41:27.660960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.550348+0000","last_change":"2024-03-04T07:33:55.568220+0000","last_active":"2024-03-04T07:33:56.550348+0000","last_peered":"2024-03-04T07:33:56.550348+0000","last_clean":"2024-03-04T07:33:56.550348+0000","last_became_active":"2024-03-04T07:33:55.567569+0000","last_became_peered":"2024-03-04T07:33:55.567569+0000","last_unstale":"2024-03-04T07:33:56.550348+0000","last_undegraded":"2024-03-04T07:33:56.550348+0000","last_fullsized":"2024-03-04T07:33:56.550348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:40:49.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.561115+0000","last_change":"2024-03-04T07:33:56.559745+0000","last_active":"2024-03-04T07:33:56.561115+0000","last_peered":"2024-03-04T07:33:56.561115+0000","last_clean":"2024-03-04T07:33:56.561115+0000","last_became_active":"2024-03-04T07:33:56.559456+0000","last_became_peered":"2024-03-04T07:33:56.559456+0000","last_unstale":"2024-03-04T07:33:56.561115+0000","last_undegraded":"2024-03-04T07:33:56.561115+0000","last_fullsized":"2024-03-04T07:33:56.561115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:43.565789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750174+0000","last_change":"2024-03-04T07:33:55.550625+0000","last_active":"2024-03-04T07:33:57.750174+0000","last_peered":"2024-03-04T07:33:57.750174+0000","last_clean":"2024-03-04T07:33:57.750174+0000","last_became_active":"2024-03-04T07:33:55.550327+0000","last_became_peered":"2024-03-04T07:33:55.550327+0000","last_unstale":"2024-03-04T07:33:57.750174+0000","last_undegraded":"2024-03-04T07:33:57.750174+0000","last_fullsized":"2024-03-04T07:33:57.750174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:20.282585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.540025+0000","last_change":"2024-03-04T07:33:56.537570+0000","last_active":"2024-03-04T07:33:56.540025+0000","last_peered":"2024-03-04T07:33:56.540025+0000","last_clean":"2024-03-04T07:33:56.540025+0000","last_became_active":"2024-03-04T07:33:56.536870+0000","last_became_peered":"2024-03-04T07:33:56.536870+0000","last_unstale":"2024-03-04T07:33:56.540025+0000","last_undegraded":"2024-03-04T07:33:56.540025+0000","last_fullsized":"2024-03-04T07:33:56.540025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:05:16.802084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.525030+0000","last_change":"2024-03-04T07:33:55.538915+0000","last_active":"2024-03-04T07:33:56.525030+0000","last_peered":"2024-03-04T07:33:56.525030+0000","last_clean":"2024-03-04T07:33:56.525030+0000","last_became_active":"2024-03-04T07:33:55.536263+0000","last_became_peered":"2024-03-04T07:33:55.536263+0000","last_unstale":"2024-03-04T07:33:56.525030+0000","last_undegraded":"2024-03-04T07:33:56.525030+0000","last_fullsized":"2024-03-04T07:33:56.525030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:09.612704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.555498+0000","last_change":"2024-03-04T07:33:56.551087+0000","last_active":"2024-03-04T07:33:56.555498+0000","last_peered":"2024-03-04T07:33:56.555498+0000","last_clean":"2024-03-04T07:33:56.555498+0000","last_became_active":"2024-03-04T07:33:56.550767+0000","last_became_peered":"2024-03-04T07:33:56.550767+0000","last_unstale":"2024-03-04T07:33:56.555498+0000","last_undegraded":"2024-03-04T07:33:56.555498+0000","last_fullsized":"2024-03-04T07:33:56.555498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:09:49.333705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.528919+0000","last_change":"2024-03-04T07:33:55.560382+0000","last_active":"2024-03-04T07:33:56.528919+0000","last_peered":"2024-03-04T07:33:56.528919+0000","last_clean":"2024-03-04T07:33:56.528919+0000","last_became_active":"2024-03-04T07:33:55.560078+0000","last_became_peered":"2024-03-04T07:33:55.560078+0000","last_unstale":"2024-03-04T07:33:56.528919+0000","last_undegraded":"2024-03-04T07:33:56.528919+0000","last_fullsized":"2024-03-04T07:33:56.528919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:41:10.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.546023+0000","last_change":"2024-03-04T07:33:56.537920+0000","last_active":"2024-03-04T07:33:56.546023+0000","last_peered":"2024-03-04T07:33:56.546023+0000","last_clean":"2024-03-04T07:33:56.546023+0000","last_became_active":"2024-03-04T07:33:56.535960+0000","last_became_peered":"2024-03-04T07:33:56.535960+0000","last_unstale":"2024-03-04T07:33:56.546023+0000","last_undegraded":"2024-03-04T07:33:56.546023+0000","last_fullsized":"2024-03-04T07:33:56.546023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:55:31.800424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.550145+0000","last_change":"2024-03-04T07:33:55.567682+0000","last_active":"2024-03-04T07:33:56.550145+0000","last_peered":"2024-03-04T07:33:56.550145+0000","last_clean":"2024-03-04T07:33:56.550145+0000","last_became_active":"2024-03-04T07:33:55.566915+0000","last_became_peered":"2024-03-04T07:33:55.566915+0000","last_unstale":"2024-03-04T07:33:56.550145+0000","last_undegraded":"2024-03-04T07:33:56.550145+0000","last_fullsized":"2024-03-04T07:33:56.550145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:33:47.010346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.553459+0000","last_change":"2024-03-04T07:33:56.547846+0000","last_active":"2024-03-04T07:33:56.553459+0000","last_peered":"2024-03-04T07:33:56.553459+0000","last_clean":"2024-03-04T07:33:56.553459+0000","last_became_active":"2024-03-04T07:33:56.547459+0000","last_became_peered":"2024-03-04T07:33:56.547459+0000","last_unstale":"2024-03-04T07:33:56.553459+0000","last_undegraded":"2024-03-04T07:33:56.553459+0000","last_fullsized":"2024-03-04T07:33:56.553459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:45:24.163852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.523896+0000","last_change":"2024-03-04T07:33:55.557041+0000","last_active":"2024-03-04T07:33:56.523896+0000","last_peered":"2024-03-04T07:33:56.523896+0000","last_clean":"2024-03-04T07:33:56.523896+0000","last_became_active":"2024-03-04T07:33:55.556416+0000","last_became_peered":"2024-03-04T07:33:55.556416+0000","last_unstale":"2024-03-04T07:33:56.523896+0000","last_undegraded":"2024-03-04T07:33:56.523896+0000","last_fullsized":"2024-03-04T07:33:56.523896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:31.472235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.555917+0000","last_change":"2024-03-04T07:33:56.531986+0000","last_active":"2024-03-04T07:33:56.555917+0000","last_peered":"2024-03-04T07:33:56.555917+0000","last_clean":"2024-03-04T07:33:56.555917+0000","last_became_active":"2024-03-04T07:33:56.531565+0000","last_became_peered":"2024-03-04T07:33:56.531565+0000","last_unstale":"2024-03-04T07:33:56.555917+0000","last_undegraded":"2024-03-04T07:33:56.555917+0000","last_fullsized":"2024-03-04T07:33:56.555917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:37:17.358787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.521077+0000","last_change":"2024-03-04T07:33:55.538469+0000","last_active":"2024-03-04T07:33:56.521077+0000","last_peered":"2024-03-04T07:33:56.521077+0000","last_clean":"2024-03-04T07:33:56.521077+0000","last_became_active":"2024-03-04T07:33:55.538165+0000","last_became_peered":"2024-03-04T07:33:55.538165+0000","last_unstale":"2024-03-04T07:33:56.521077+0000","last_undegraded":"2024-03-04T07:33:56.521077+0000","last_fullsized":"2024-03-04T07:33:56.521077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:00:01.263768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.552449+0000","last_change":"2024-03-04T07:33:56.547881+0000","last_active":"2024-03-04T07:33:56.552449+0000","last_peered":"2024-03-04T07:33:56.552449+0000","last_clean":"2024-03-04T07:33:56.552449+0000","last_became_active":"2024-03-04T07:33:56.547461+0000","last_became_peered":"2024-03-04T07:33:56.547461+0000","last_unstale":"2024-03-04T07:33:56.552449+0000","last_undegraded":"2024-03-04T07:33:56.552449+0000","last_fullsized":"2024-03-04T07:33:56.552449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:30:35.809552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.533836+0000","last_change":"2024-03-04T07:33:55.544356+0000","last_active":"2024-03-04T07:33:56.533836+0000","last_peered":"2024-03-04T07:33:56.533836+0000","last_clean":"2024-03-04T07:33:56.533836+0000","last_became_active":"2024-03-04T07:33:55.541295+0000","last_became_peered":"2024-03-04T07:33:55.541295+0000","last_unstale":"2024-03-04T07:33:56.533836+0000","last_undegraded":"2024-03-04T07:33:56.533836+0000","last_fullsized":"2024-03-04T07:33:56.533836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:22.953982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.552488+0000","last_change":"2024-03-04T07:33:56.549038+0000","last_active":"2024-03-04T07:33:56.552488+0000","last_peered":"2024-03-04T07:33:56.552488+0000","last_clean":"2024-03-04T07:33:56.552488+0000","last_became_active":"2024-03-04T07:33:56.548113+0000","last_became_peered":"2024-03-04T07:33:56.548113+0000","last_unstale":"2024-03-04T07:33:56.552488+0000","last_undegraded":"2024-03-04T07:33:56.552488+0000","last_fullsized":"2024-03-04T07:33:56.552488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:43:47.784030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.521591+0000","last_change":"2024-03-04T07:33:55.538871+0000","last_active":"2024-03-04T07:33:56.521591+0000","last_peered":"2024-03-04T07:33:56.521591+0000","last_clean":"2024-03-04T07:33:56.521591+0000","last_became_active":"2024-03-04T07:33:55.537221+0000","last_became_peered":"2024-03-04T07:33:55.537221+0000","last_unstale":"2024-03-04T07:33:56.521591+0000","last_undegraded":"2024-03-04T07:33:56.521591+0000","last_fullsized":"2024-03-04T07:33:56.521591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:02:34.533330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.551403+0000","last_change":"2024-03-04T07:33:56.543760+0000","last_active":"2024-03-04T07:33:56.551403+0000","last_peered":"2024-03-04T07:33:56.551403+0000","last_clean":"2024-03-04T07:33:56.551403+0000","last_became_active":"2024-03-04T07:33:56.543354+0000","last_became_peered":"2024-03-04T07:33:56.543354+0000","last_unstale":"2024-03-04T07:33:56.551403+0000","last_undegraded":"2024-03-04T07:33:56.551403+0000","last_fullsized":"2024-03-04T07:33:56.551403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:06.936208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750256+0000","last_change":"2024-03-04T07:33:55.552856+0000","last_active":"2024-03-04T07:33:57.750256+0000","last_peered":"2024-03-04T07:33:57.750256+0000","last_clean":"2024-03-04T07:33:57.750256+0000","last_became_active":"2024-03-04T07:33:55.552472+0000","last_became_peered":"2024-03-04T07:33:55.552472+0000","last_unstale":"2024-03-04T07:33:57.750256+0000","last_undegraded":"2024-03-04T07:33:57.750256+0000","last_fullsized":"2024-03-04T07:33:57.750256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:36:50.409567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.561741+0000","last_change":"2024-03-04T07:33:56.558497+0000","last_active":"2024-03-04T07:33:56.561741+0000","last_peered":"2024-03-04T07:33:56.561741+0000","last_clean":"2024-03-04T07:33:56.561741+0000","last_became_active":"2024-03-04T07:33:56.557989+0000","last_became_peered":"2024-03-04T07:33:56.557989+0000","last_unstale":"2024-03-04T07:33:56.561741+0000","last_undegraded":"2024-03-04T07:33:56.561741+0000","last_fullsized":"2024-03-04T07:33:56.561741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:31:13.406101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.520930+0000","last_change":"2024-03-04T07:33:55.560332+0000","last_active":"2024-03-04T07:33:56.520930+0000","last_peered":"2024-03-04T07:33:56.520930+0000","last_clean":"2024-03-04T07:33:56.520930+0000","last_became_active":"2024-03-04T07:33:55.560086+0000","last_became_peered":"2024-03-04T07:33:55.560086+0000","last_unstale":"2024-03-04T07:33:56.520930+0000","last_undegraded":"2024-03-04T07:33:56.520930+0000","last_fullsized":"2024-03-04T07:33:56.520930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:25:46.280657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.565836+0000","last_change":"2024-03-04T07:33:56.559881+0000","last_active":"2024-03-04T07:33:56.565836+0000","last_peered":"2024-03-04T07:33:56.565836+0000","last_clean":"2024-03-04T07:33:56.565836+0000","last_became_active":"2024-03-04T07:33:56.559589+0000","last_became_peered":"2024-03-04T07:33:56.559589+0000","last_unstale":"2024-03-04T07:33:56.565836+0000","last_undegraded":"2024-03-04T07:33:56.565836+0000","last_fullsized":"2024-03-04T07:33:56.565836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:59:39.122143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.531971+0000","last_change":"2024-03-04T07:33:43.559630+0000","last_active":"2024-03-04T07:33:56.531971+0000","last_peered":"2024-03-04T07:33:56.531971+0000","last_clean":"2024-03-04T07:33:56.531971+0000","last_became_active":"2024-03-04T07:33:43.557489+0000","last_became_peered":"2024-03-04T07:33:43.557489+0000","last_unstale":"2024-03-04T07:33:56.531971+0000","last_undegraded":"2024-03-04T07:33:56.531971+0000","last_fullsized":"2024-03-04T07:33:56.531971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_clean_scrub_stamp":"2024-03-04T07:33:42.183823+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:55:35.762857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.550284+0000","last_change":"2024-03-04T07:33:55.548616+0000","last_active":"2024-03-04T07:33:56.550284+0000","last_peered":"2024-03-04T07:33:56.550284+0000","last_clean":"2024-03-04T07:33:56.550284+0000","last_became_active":"2024-03-04T07:33:55.547322+0000","last_became_peered":"2024-03-04T07:33:55.547322+0000","last_unstale":"2024-03-04T07:33:56.550284+0000","last_undegraded":"2024-03-04T07:33:56.550284+0000","last_fullsized":"2024-03-04T07:33:56.550284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:20:16.717483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523877+0000","last_change":"2024-03-04T07:33:56.559966+0000","last_active":"2024-03-04T07:33:57.523877+0000","last_peered":"2024-03-04T07:33:57.523877+0000","last_clean":"2024-03-04T07:33:57.523877+0000","last_became_active":"2024-03-04T07:33:56.559581+0000","last_became_peered":"2024-03-04T07:33:56.559581+0000","last_unstale":"2024-03-04T07:33:57.523877+0000","last_undegraded":"2024-03-04T07:33:57.523877+0000","last_fullsized":"2024-03-04T07:33:57.523877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.531915+0000","last_change":"2024-03-04T07:33:55.563905+0000","last_active":"2024-03-04T07:33:56.531915+0000","last_peered":"2024-03-04T07:33:56.531915+0000","last_clean":"2024-03-04T07:33:56.531915+0000","last_became_active":"2024-03-04T07:33:55.563117+0000","last_became_peered":"2024-03-04T07:33:55.563117+0000","last_unstale":"2024-03-04T07:33:56.531915+0000","last_undegraded":"2024-03-04T07:33:56.531915+0000","last_fullsized":"2024-03-04T07:33:56.531915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:53:02.522757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.540990+0000","last_change":"2024-03-04T07:33:56.531080+0000","last_active":"2024-03-04T07:33:56.540990+0000","last_peered":"2024-03-04T07:33:56.540990+0000","last_clean":"2024-03-04T07:33:56.540990+0000","last_became_active":"2024-03-04T07:33:56.530776+0000","last_became_peered":"2024-03-04T07:33:56.530776+0000","last_unstale":"2024-03-04T07:33:56.540990+0000","last_undegraded":"2024-03-04T07:33:56.540990+0000","last_fullsized":"2024-03-04T07:33:56.540990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:45:39.713736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.520916+0000","last_change":"2024-03-04T07:33:55.538631+0000","last_active":"2024-03-04T07:33:56.520916+0000","last_peered":"2024-03-04T07:33:56.520916+0000","last_clean":"2024-03-04T07:33:56.520916+0000","last_became_active":"2024-03-04T07:33:55.537104+0000","last_became_peered":"2024-03-04T07:33:55.537104+0000","last_unstale":"2024-03-04T07:33:56.520916+0000","last_undegraded":"2024-03-04T07:33:56.520916+0000","last_fullsized":"2024-03-04T07:33:56.520916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:27:06.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749962+0000","last_change":"2024-03-04T07:33:56.541335+0000","last_active":"2024-03-04T07:33:57.749962+0000","last_peered":"2024-03-04T07:33:57.749962+0000","last_clean":"2024-03-04T07:33:57.749962+0000","last_became_active":"2024-03-04T07:33:56.540536+0000","last_became_peered":"2024-03-04T07:33:56.540536+0000","last_unstale":"2024-03-04T07:33:57.749962+0000","last_undegraded":"2024-03-04T07:33:57.749962+0000","last_fullsized":"2024-03-04T07:33:57.749962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:31:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.550343+0000","last_change":"2024-03-04T07:33:55.574740+0000","last_active":"2024-03-04T07:33:56.550343+0000","last_peered":"2024-03-04T07:33:56.550343+0000","last_clean":"2024-03-04T07:33:56.550343+0000","last_became_active":"2024-03-04T07:33:55.574129+0000","last_became_peered":"2024-03-04T07:33:55.574129+0000","last_unstale":"2024-03-04T07:33:56.550343+0000","last_undegraded":"2024-03-04T07:33:56.550343+0000","last_fullsized":"2024-03-04T07:33:56.550343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:40:57.891803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.543405+0000","last_change":"2024-03-04T07:33:56.541654+0000","last_active":"2024-03-04T07:33:56.543405+0000","last_peered":"2024-03-04T07:33:56.543405+0000","last_clean":"2024-03-04T07:33:56.543405+0000","last_became_active":"2024-03-04T07:33:56.541422+0000","last_became_peered":"2024-03-04T07:33:56.541422+0000","last_unstale":"2024-03-04T07:33:56.543405+0000","last_undegraded":"2024-03-04T07:33:56.543405+0000","last_fullsized":"2024-03-04T07:33:56.543405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:19:03.722779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750122+0000","last_change":"2024-03-04T07:33:55.539317+0000","last_active":"2024-03-04T07:33:57.750122+0000","last_peered":"2024-03-04T07:33:57.750122+0000","last_clean":"2024-03-04T07:33:57.750122+0000","last_became_active":"2024-03-04T07:33:55.539017+0000","last_became_peered":"2024-03-04T07:33:55.539017+0000","last_unstale":"2024-03-04T07:33:57.750122+0000","last_undegraded":"2024-03-04T07:33:57.750122+0000","last_fullsized":"2024-03-04T07:33:57.750122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:10.787316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.558954+0000","last_change":"2024-03-04T07:33:56.554103+0000","last_active":"2024-03-04T07:33:56.558954+0000","last_peered":"2024-03-04T07:33:56.558954+0000","last_clean":"2024-03-04T07:33:56.558954+0000","last_became_active":"2024-03-04T07:33:56.553742+0000","last_became_peered":"2024-03-04T07:33:56.553742+0000","last_unstale":"2024-03-04T07:33:56.558954+0000","last_undegraded":"2024-03-04T07:33:56.558954+0000","last_fullsized":"2024-03-04T07:33:56.558954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:10:17.027074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.536790+0000","last_change":"2024-03-04T07:33:55.569442+0000","last_active":"2024-03-04T07:33:56.536790+0000","last_peered":"2024-03-04T07:33:56.536790+0000","last_clean":"2024-03-04T07:33:56.536790+0000","last_became_active":"2024-03-04T07:33:55.567809+0000","last_became_peered":"2024-03-04T07:33:55.567809+0000","last_unstale":"2024-03-04T07:33:56.536790+0000","last_undegraded":"2024-03-04T07:33:56.536790+0000","last_fullsized":"2024-03-04T07:33:56.536790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:58.551866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750083+0000","last_change":"2024-03-04T07:33:56.542997+0000","last_active":"2024-03-04T07:33:57.750083+0000","last_peered":"2024-03-04T07:33:57.750083+0000","last_clean":"2024-03-04T07:33:57.750083+0000","last_became_active":"2024-03-04T07:33:56.542775+0000","last_became_peered":"2024-03-04T07:33:56.542775+0000","last_unstale":"2024-03-04T07:33:57.750083+0000","last_undegraded":"2024-03-04T07:33:57.750083+0000","last_fullsized":"2024-03-04T07:33:57.750083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:32.090568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750225+0000","last_change":"2024-03-04T07:33:55.545598+0000","last_active":"2024-03-04T07:33:57.750225+0000","last_peered":"2024-03-04T07:33:57.750225+0000","last_clean":"2024-03-04T07:33:57.750225+0000","last_became_active":"2024-03-04T07:33:55.545262+0000","last_became_peered":"2024-03-04T07:33:55.545262+0000","last_unstale":"2024-03-04T07:33:57.750225+0000","last_undegraded":"2024-03-04T07:33:57.750225+0000","last_fullsized":"2024-03-04T07:33:57.750225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:01.667243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523716+0000","last_change":"2024-03-04T07:33:56.550943+0000","last_active":"2024-03-04T07:33:57.523716+0000","last_peered":"2024-03-04T07:33:57.523716+0000","last_clean":"2024-03-04T07:33:57.523716+0000","last_became_active":"2024-03-04T07:33:56.550604+0000","last_became_peered":"2024-03-04T07:33:56.550604+0000","last_unstale":"2024-03-04T07:33:57.523716+0000","last_undegraded":"2024-03-04T07:33:57.523716+0000","last_fullsized":"2024-03-04T07:33:57.523716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:01:05.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.524536+0000","last_change":"2024-03-04T07:33:55.567220+0000","last_active":"2024-03-04T07:33:56.524536+0000","last_peered":"2024-03-04T07:33:56.524536+0000","last_clean":"2024-03-04T07:33:56.524536+0000","last_became_active":"2024-03-04T07:33:55.566600+0000","last_became_peered":"2024-03-04T07:33:55.566600+0000","last_unstale":"2024-03-04T07:33:56.524536+0000","last_undegraded":"2024-03-04T07:33:56.524536+0000","last_fullsized":"2024-03-04T07:33:56.524536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.876938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.545896+0000","last_change":"2024-03-04T07:33:56.544263+0000","last_active":"2024-03-04T07:33:56.545896+0000","last_peered":"2024-03-04T07:33:56.545896+0000","last_clean":"2024-03-04T07:33:56.545896+0000","last_became_active":"2024-03-04T07:33:56.543836+0000","last_became_peered":"2024-03-04T07:33:56.543836+0000","last_unstale":"2024-03-04T07:33:56.545896+0000","last_undegraded":"2024-03-04T07:33:56.545896+0000","last_fullsized":"2024-03-04T07:33:56.545896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:13:15.958853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.531709+0000","last_change":"2024-03-04T07:33:55.566886+0000","last_active":"2024-03-04T07:33:56.531709+0000","last_peered":"2024-03-04T07:33:56.531709+0000","last_clean":"2024-03-04T07:33:56.531709+0000","last_became_active":"2024-03-04T07:33:55.566048+0000","last_became_peered":"2024-03-04T07:33:55.566048+0000","last_unstale":"2024-03-04T07:33:56.531709+0000","last_undegraded":"2024-03-04T07:33:56.531709+0000","last_fullsized":"2024-03-04T07:33:56.531709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:45:27.190438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.539368+0000","last_change":"2024-03-04T07:33:56.532666+0000","last_active":"2024-03-04T07:33:56.539368+0000","last_peered":"2024-03-04T07:33:56.539368+0000","last_clean":"2024-03-04T07:33:56.539368+0000","last_became_active":"2024-03-04T07:33:56.531901+0000","last_became_peered":"2024-03-04T07:33:56.531901+0000","last_unstale":"2024-03-04T07:33:56.539368+0000","last_undegraded":"2024-03-04T07:33:56.539368+0000","last_fullsized":"2024-03-04T07:33:56.539368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:50:56.653702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750078+0000","last_change":"2024-03-04T07:33:55.557568+0000","last_active":"2024-03-04T07:33:57.750078+0000","last_peered":"2024-03-04T07:33:57.750078+0000","last_clean":"2024-03-04T07:33:57.750078+0000","last_became_active":"2024-03-04T07:33:55.557330+0000","last_became_peered":"2024-03-04T07:33:55.557330+0000","last_unstale":"2024-03-04T07:33:57.750078+0000","last_undegraded":"2024-03-04T07:33:57.750078+0000","last_fullsized":"2024-03-04T07:33:57.750078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:34.956318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523904+0000","last_change":"2024-03-04T07:33:56.552963+0000","last_active":"2024-03-04T07:33:57.523904+0000","last_peered":"2024-03-04T07:33:57.523904+0000","last_clean":"2024-03-04T07:33:57.523904+0000","last_became_active":"2024-03-04T07:33:56.552658+0000","last_became_peered":"2024-03-04T07:33:56.552658+0000","last_unstale":"2024-03-04T07:33:57.523904+0000","last_undegraded":"2024-03-04T07:33:57.523904+0000","last_fullsized":"2024-03-04T07:33:57.523904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:05.685496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750081+0000","last_change":"2024-03-04T07:33:55.536480+0000","last_active":"2024-03-04T07:33:57.750081+0000","last_peered":"2024-03-04T07:33:57.750081+0000","last_clean":"2024-03-04T07:33:57.750081+0000","last_became_active":"2024-03-04T07:33:55.536139+0000","last_became_peered":"2024-03-04T07:33:55.536139+0000","last_unstale":"2024-03-04T07:33:57.750081+0000","last_undegraded":"2024-03-04T07:33:57.750081+0000","last_fullsized":"2024-03-04T07:33:57.750081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:12.778026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523811+0000","last_change":"2024-03-04T07:33:56.546546+0000","last_active":"2024-03-04T07:33:57.523811+0000","last_peered":"2024-03-04T07:33:57.523811+0000","last_clean":"2024-03-04T07:33:57.523811+0000","last_became_active":"2024-03-04T07:33:56.546280+0000","last_became_peered":"2024-03-04T07:33:56.546280+0000","last_unstale":"2024-03-04T07:33:57.523811+0000","last_undegraded":"2024-03-04T07:33:57.523811+0000","last_fullsized":"2024-03-04T07:33:57.523811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:19:55.950163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.535401+0000","last_change":"2024-03-04T07:33:55.572540+0000","last_active":"2024-03-04T07:33:56.535401+0000","last_peered":"2024-03-04T07:33:56.535401+0000","last_clean":"2024-03-04T07:33:56.535401+0000","last_became_active":"2024-03-04T07:33:55.571612+0000","last_became_peered":"2024-03-04T07:33:55.571612+0000","last_unstale":"2024-03-04T07:33:56.535401+0000","last_undegraded":"2024-03-04T07:33:56.535401+0000","last_fullsized":"2024-03-04T07:33:56.535401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:01:43.741423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.556085+0000","last_change":"2024-03-04T07:33:56.532855+0000","last_active":"2024-03-04T07:33:56.556085+0000","last_peered":"2024-03-04T07:33:56.556085+0000","last_clean":"2024-03-04T07:33:56.556085+0000","last_became_active":"2024-03-04T07:33:56.532245+0000","last_became_peered":"2024-03-04T07:33:56.532245+0000","last_unstale":"2024-03-04T07:33:56.556085+0000","last_undegraded":"2024-03-04T07:33:56.556085+0000","last_fullsized":"2024-03-04T07:33:56.556085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:52:07.456511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.568498+0000","last_change":"2024-03-04T07:33:56.563803+0000","last_active":"2024-03-04T07:33:56.568498+0000","last_peered":"2024-03-04T07:33:56.568498+0000","last_clean":"2024-03-04T07:33:56.568498+0000","last_became_active":"2024-03-04T07:33:56.563473+0000","last_became_peered":"2024-03-04T07:33:56.563473+0000","last_unstale":"2024-03-04T07:33:56.568498+0000","last_undegraded":"2024-03-04T07:33:56.568498+0000","last_fullsized":"2024-03-04T07:33:56.568498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:45:41.267667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.533193+0000","last_change":"2024-03-04T07:33:55.575688+0000","last_active":"2024-03-04T07:33:56.533193+0000","last_peered":"2024-03-04T07:33:56.533193+0000","last_clean":"2024-03-04T07:33:56.533193+0000","last_became_active":"2024-03-04T07:33:55.574766+0000","last_became_peered":"2024-03-04T07:33:55.574766+0000","last_unstale":"2024-03-04T07:33:56.533193+0000","last_undegraded":"2024-03-04T07:33:56.533193+0000","last_fullsized":"2024-03-04T07:33:56.533193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:32:19.708016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.568497+0000","last_change":"2024-03-04T07:33:56.563798+0000","last_active":"2024-03-04T07:33:56.568497+0000","last_peered":"2024-03-04T07:33:56.568497+0000","last_clean":"2024-03-04T07:33:56.568497+0000","last_became_active":"2024-03-04T07:33:56.563474+0000","last_became_peered":"2024-03-04T07:33:56.563474+0000","last_unstale":"2024-03-04T07:33:56.568497+0000","last_undegraded":"2024-03-04T07:33:56.568497+0000","last_fullsized":"2024-03-04T07:33:56.568497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:18:24.399645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.535855+0000","last_change":"2024-03-04T07:33:55.570016+0000","last_active":"2024-03-04T07:33:56.535855+0000","last_peered":"2024-03-04T07:33:56.535855+0000","last_clean":"2024-03-04T07:33:56.535855+0000","last_became_active":"2024-03-04T07:33:55.568749+0000","last_became_peered":"2024-03-04T07:33:55.568749+0000","last_unstale":"2024-03-04T07:33:56.535855+0000","last_undegraded":"2024-03-04T07:33:56.535855+0000","last_fullsized":"2024-03-04T07:33:56.535855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:01.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.563345+0000","last_change":"2024-03-04T07:33:56.562015+0000","last_active":"2024-03-04T07:33:56.563345+0000","last_peered":"2024-03-04T07:33:56.563345+0000","last_clean":"2024-03-04T07:33:56.563345+0000","last_became_active":"2024-03-04T07:33:56.561675+0000","last_became_peered":"2024-03-04T07:33:56.561675+0000","last_unstale":"2024-03-04T07:33:56.563345+0000","last_undegraded":"2024-03-04T07:33:56.563345+0000","last_fullsized":"2024-03-04T07:33:56.563345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:05:03.773019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.524108+0000","last_change":"2024-03-04T07:33:55.538337+0000","last_active":"2024-03-04T07:33:57.524108+0000","last_peered":"2024-03-04T07:33:57.524108+0000","last_clean":"2024-03-04T07:33:57.524108+0000","last_became_active":"2024-03-04T07:33:55.538010+0000","last_became_peered":"2024-03-04T07:33:55.538010+0000","last_unstale":"2024-03-04T07:33:57.524108+0000","last_undegraded":"2024-03-04T07:33:57.524108+0000","last_fullsized":"2024-03-04T07:33:57.524108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:47:15.711205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.524079+0000","last_change":"2024-03-04T07:33:56.559051+0000","last_active":"2024-03-04T07:33:57.524079+0000","last_peered":"2024-03-04T07:33:57.524079+0000","last_clean":"2024-03-04T07:33:57.524079+0000","last_became_active":"2024-03-04T07:33:56.558799+0000","last_became_peered":"2024-03-04T07:33:56.558799+0000","last_unstale":"2024-03-04T07:33:57.524079+0000","last_undegraded":"2024-03-04T07:33:57.524079+0000","last_fullsized":"2024-03-04T07:33:57.524079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:16.516631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-04T07:33:56.528987+0000","last_change":"2024-03-04T07:33:55.566294+0000","last_active":"2024-03-04T07:33:56.528987+0000","last_peered":"2024-03-04T07:33:56.528987+0000","last_clean":"2024-03-04T07:33:56.528987+0000","last_became_active":"2024-03-04T07:33:55.565941+0000","last_became_peered":"2024-03-04T07:33:55.565941+0000","last_unstale":"2024-03-04T07:33:56.528987+0000","last_undegraded":"2024-03-04T07:33:56.528987+0000","last_fullsized":"2024-03-04T07:33:56.528987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:03.115085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":67229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339884,"statfs":{"total":96636764160,"available":96604041216,"internally_reserved":0,"allocated":790528,"data_stored":657597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":67229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":196608,"data_stored":67229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":657597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T07:34:03.042 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-04T07:34:03.043 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-04T07:34:03.043 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-04T07:34:03.043 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-04T07:34:03.043 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-04T07:34:03.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.218+0000 7fa16ce00640 1 Processor -- start 2024-03-04T07:34:03.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.218+0000 7fa16ce00640 1 -- start start 2024-03-04T07:34:03.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.218+0000 7fa16ce00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1680c22e0 0x7fa16815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.218+0000 7fa16ce00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa1680c2d80 0x7fa16815b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.218+0000 7fa16ce00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1680c27f0 con 0x7fa16814e4c0 2024-03-04T07:34:03.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.218+0000 7fa16ce00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16815ad60 con 0x7fa1680c2d80 2024-03-04T07:34:03.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa16ce00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680c2a90 con 0x7fa1680c22e0 2024-03-04T07:34:03.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa16814e4c0 0x7fa168072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49486/0 (socket says 172.21.15.113:49486) 2024-03-04T07:34:03.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165e00640 1 -- 172.21.15.113:0/2229285585 learned_addr learned my addr 172.21.15.113:0/2229285585 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:03.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa15d400640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1680c22e0 0x7fa16815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa166800640 1 --2- 172.21.15.113:0/2229285585 >> v2:172.21.15.136:3300/0 conn(0x7fa1680c2d80 0x7fa16815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824677271 0 0) 0x7fa1680c27f0 con 0x7fa16814e4c0 2024-03-04T07:34:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa14c003530 con 0x7fa16814e4c0 2024-03-04T07:34:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2734460069 0 0) 0x7fa14c003530 con 0x7fa16814e4c0 2024-03-04T07:34:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1680c22e0 msgr2=0x7fa16815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1680c22e0 0x7fa16815b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 >> v2:172.21.15.136:3300/0 conn(0x7fa1680c2d80 msgr2=0x7fa16815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 --2- 172.21.15.113:0/2229285585 >> v2:172.21.15.136:3300/0 conn(0x7fa1680c2d80 0x7fa16815b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168070c60 con 0x7fa16814e4c0 2024-03-04T07:34:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fa158001680 con 0x7fa16814e4c0 2024-03-04T07:34:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 >> v1:172.21.15.113:6789/0 conn(0x7fa16814e4c0 legacy=0x7fa168072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa14c0039c0 0x7fa14c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa14c004810 0x7fa14c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1680c22e0 0x7fa14c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680c2a90 con 0x7fa14c004810 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c00b9d0 con 0x7fa14c0039c0 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c00bb50 con 0x7fa1680c22e0 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa15d400640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa14c004810 0x7fa14c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa165e00640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa14c0039c0 0x7fa14c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa166800640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1680c22e0 0x7fa14c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa15d400640 1 -- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1680c22e0 msgr2=0x7fa14c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa15d400640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1680c22e0 0x7fa14c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa15d400640 1 -- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa14c0039c0 msgr2=0x7fa14c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa15d400640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa14c0039c0 0x7fa14c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.222+0000 7fa15d400640 1 -- 172.21.15.113:0/2229285585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c003530 con 0x7fa14c004810 2024-03-04T07:34:03.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa15d400640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa14c004810 0x7fa14c004c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa150009ae0 tx=0x7fa15001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9558941dff688f68 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:03.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1500b2070 con 0x7fa14c004810 2024-03-04T07:34:03.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1500a5580 con 0x7fa14c004810 2024-03-04T07:34:03.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa165400640 1 -- 172.21.15.113:0/2229285585 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1500096b0 con 0x7fa14c004810 2024-03-04T07:34:03.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa16ce00640 1 -- 172.21.15.113:0/2229285585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa14c004810 msgr2=0x7fa14c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa16ce00640 1 --2- 172.21.15.113:0/2229285585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa14c004810 0x7fa14c004c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa150009ae0 tx=0x7fa15001ca70 comp rx=0 tx=0).stop 2024-03-04T07:34:03.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa166800640 1 -- 172.21.15.113:0/2229285585 reap_dead start 2024-03-04T07:34:03.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa16ce00640 1 -- 172.21.15.113:0/2229285585 shutdown_connections 2024-03-04T07:34:03.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa16ce00640 1 -- 172.21.15.113:0/2229285585 >> 172.21.15.113:0/2229285585 conn(0x7fa16806e820 msgr2=0x7fa16806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:03.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa16ce00640 1 -- 172.21.15.113:0/2229285585 shutdown_connections 2024-03-04T07:34:03.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa16ce00640 1 -- 172.21.15.113:0/2229285585 wait complete. 2024-03-04T07:34:03.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa16ce00640 1 Processor -- start 2024-03-04T07:34:03.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.226+0000 7fa16ce00640 1 -- start start 2024-03-04T07:34:03.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa16ce00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa14c004810 0x7fa1681f0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa16ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1681f4b30 0x7fa168159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa16ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa16814d1f0 0x7fa1680caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa16ce00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa168071500 con 0x7fa1681f4b30 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa16ce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa168071220 con 0x7fa16814d1f0 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa16ce00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16814e8a0 con 0x7fa14c004810 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1681f4b30 0x7fa168159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1681f4b30 0x7fa168159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57084/0 (socket says 172.21.15.113:57084) 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15d400640 1 -- 172.21.15.113:0/258098681 learned_addr learned my addr 172.21.15.113:0/258098681 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa165e00640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa14c004810 0x7fa1681f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa166800640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa16814d1f0 0x7fa1680caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15d400640 1 -- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa14c004810 msgr2=0x7fa1681f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15d400640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa14c004810 0x7fa1681f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15d400640 1 -- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa16814d1f0 msgr2=0x7fa1680caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15d400640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa16814d1f0 0x7fa1680caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15d400640 1 -- 172.21.15.113:0/258098681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168163f60 con 0x7fa1681f4b30 2024-03-04T07:34:03.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15d400640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1681f4b30 0x7fa168159a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa150009ae0 tx=0x7fa14c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:03.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15ea00640 1 -- 172.21.15.113:0/258098681 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1500b2040 con 0x7fa1681f4b30 2024-03-04T07:34:03.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15ea00640 1 -- 172.21.15.113:0/258098681 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa150004020 con 0x7fa1681f4b30 2024-03-04T07:34:03.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15ea00640 1 -- 172.21.15.113:0/258098681 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1500043d0 con 0x7fa1681f4b30 2024-03-04T07:34:03.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa16ce00640 1 -- 172.21.15.113:0/258098681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16814d750 con 0x7fa1681f4b30 2024-03-04T07:34:03.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa16ce00640 1 -- 172.21.15.113:0/258098681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1681f4050 con 0x7fa1681f4b30 2024-03-04T07:34:03.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa16ce00640 1 -- 172.21.15.113:0/258098681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa168075c00 con 0x7fa1681f4b30 2024-03-04T07:34:03.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15ea00640 1 -- 172.21.15.113:0/258098681 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1500c0060 con 0x7fa1681f4b30 2024-03-04T07:34:03.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15ea00640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa144066f60 0x7fa144069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.230+0000 7fa15ea00640 1 -- 172.21.15.113:0/258098681 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa15011ec50 con 0x7fa1681f4b30 2024-03-04T07:34:03.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.234+0000 7fa165e00640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa144066f60 0x7fa144069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.234+0000 7fa165e00640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa144066f60 0x7fa144069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa158005a40 tx=0x7fa158009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:03.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.234+0000 7fa15ea00640 1 -- 172.21.15.113:0/258098681 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa150016780 con 0x7fa1681f4b30 2024-03-04T07:34:03.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.426+0000 7fa16ce00640 1 -- 172.21.15.113:0/258098681 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa168155190 con 0x7fa1681f4b30 2024-03-04T07:34:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.430+0000 7fa15ea00640 1 -- 172.21.15.113:0/258098681 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165740 (secure 0 0 0) 0x7fa1500efb30 con 0x7fa1681f4b30 2024-03-04T07:34:03.436 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:03.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 -- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa144066f60 msgr2=0x7fa144069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa144066f60 0x7fa144069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa158005a40 tx=0x7fa158009040 comp rx=0 tx=0).stop 2024-03-04T07:34:03.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 -- 172.21.15.113:0/258098681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1681f4b30 msgr2=0x7fa168159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1681f4b30 0x7fa168159a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa150009ae0 tx=0x7fa14c005840 comp rx=0 tx=0).stop 2024-03-04T07:34:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 -- 172.21.15.113:0/258098681 shutdown_connections 2024-03-04T07:34:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa16814d1f0 0x7fa1680caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa144066f60 0x7fa144069420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1681f4b30 0x7fa168159a00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 --2- 172.21.15.113:0/258098681 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa14c004810 0x7fa1681f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 -- 172.21.15.113:0/258098681 >> 172.21.15.113:0/258098681 conn(0x7fa16806e820 msgr2=0x7fa16806bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 -- 172.21.15.113:0/258098681 shutdown_connections 2024-03-04T07:34:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.434+0000 7fa16ce00640 1 -- 172.21.15.113:0/258098681 wait complete. 2024-03-04T07:34:03.456 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":3353359815},{"type":"v1","addr":"172.21.15.136:6833","nonce":3353359815}]},"active_addr":"172.21.15.136:6833/3353359815","active_change":"2024-03-04T07:33:35.418398+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.136:0","nonce":3489619216}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.136:0","nonce":2011452771}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.136:0","nonce":4050048470}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.136:0","nonce":2521273566}]}]} 2024-03-04T07:34:03.459 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-04T07:34:03.459 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-04T07:34:03.459 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:34:03.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be4800640 1 Processor -- start 2024-03-04T07:34:03.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be4800640 1 -- start start 2024-03-04T07:34:03.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be4800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9bdc0ce2d0 0x7f9bdc15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be4800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9bdc0ce6b0 0x7f9bdc06f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be4800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9bdc06de80 0x7f9bdc154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be4800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bdc15bbf0 con 0x7f9bdc06de80 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be4800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bdc06eb80 con 0x7f9bdc0ce2d0 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be4800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc06c210 con 0x7f9bdc0ce6b0 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9bdc06de80 0x7f9bdc154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9bdc06de80 0x7f9bdc154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57086/0 (socket says 172.21.15.113:57086) 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be2800640 1 -- 172.21.15.113:0/1256962620 learned_addr learned my addr 172.21.15.113:0/1256962620 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be1400640 1 --2- 172.21.15.113:0/1256962620 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9bdc0ce6b0 0x7f9bdc06f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be1e00640 1 --2- 172.21.15.113:0/1256962620 >> v2:172.21.15.136:3300/0 conn(0x7f9bdc0ce2d0 0x7f9bdc15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be2800640 1 -- 172.21.15.113:0/1256962620 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9bdc0ce6b0 msgr2=0x7f9bdc06f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be2800640 1 --2- 172.21.15.113:0/1256962620 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9bdc0ce6b0 0x7f9bdc06f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be2800640 1 -- 172.21.15.113:0/1256962620 >> v2:172.21.15.136:3300/0 conn(0x7f9bdc0ce2d0 msgr2=0x7f9bdc15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be2800640 1 --2- 172.21.15.113:0/1256962620 >> v2:172.21.15.136:3300/0 conn(0x7f9bdc0ce2d0 0x7f9bdc15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.594+0000 7f9be2800640 1 -- 172.21.15.113:0/1256962620 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc06c4f0 con 0x7f9bdc06de80 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be2800640 1 --2- 172.21.15.113:0/1256962620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 0x7f9bdc154b10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0b3ca0 tx=0x7f9bd801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833805bc99237021 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be0a00640 1 -- 172.21.15.113:0/1256962620 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80b4070 con 0x7f9bdc06de80 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be0a00640 1 -- 172.21.15.113:0/1256962620 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd80af400 con 0x7f9bdc06de80 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be0a00640 1 -- 172.21.15.113:0/1256962620 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80af7d0 con 0x7f9bdc06de80 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be4800640 1 -- 172.21.15.113:0/1256962620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 msgr2=0x7f9bdc154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be4800640 1 --2- 172.21.15.113:0/1256962620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 0x7f9bdc154b10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0b3ca0 tx=0x7f9bd801f1e0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be4800640 1 -- 172.21.15.113:0/1256962620 shutdown_connections 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be4800640 1 --2- 172.21.15.113:0/1256962620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 0x7f9bdc154b10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be4800640 1 --2- 172.21.15.113:0/1256962620 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9bdc0ce6b0 0x7f9bdc06f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be4800640 1 --2- 172.21.15.113:0/1256962620 >> v2:172.21.15.136:3300/0 conn(0x7f9bdc0ce2d0 0x7f9bdc15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be4800640 1 -- 172.21.15.113:0/1256962620 >> 172.21.15.113:0/1256962620 conn(0x7f9bdc0709a0 msgr2=0x7f9bdc070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be4800640 1 -- 172.21.15.113:0/1256962620 shutdown_connections 2024-03-04T07:34:03.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.598+0000 7f9be4800640 1 -- 172.21.15.113:0/1256962620 wait complete. 2024-03-04T07:34:03.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be4800640 1 Processor -- start 2024-03-04T07:34:03.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be4800640 1 -- start start 2024-03-04T07:34:03.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be4800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 0x7f9bdc0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be4800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bdc0ce2d0 0x7f9bdc1f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be4800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9bdc0ce6b0 0x7f9bdc14d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be4800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc06c7d0 con 0x7f9bdc06de80 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be4800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc06e810 con 0x7f9bdc0ce2d0 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be4800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bdc070010 con 0x7f9bdc0ce6b0 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 0x7f9bdc0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 0x7f9bdc0c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57102/0 (socket says 172.21.15.113:57102) 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be1e00640 1 -- 172.21.15.113:0/1121475598 learned_addr learned my addr 172.21.15.113:0/1121475598 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be2800640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9bdc0ce6b0 0x7f9bdc14d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be1400640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bdc0ce2d0 0x7f9bdc1f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be1e00640 1 -- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9bdc0ce6b0 msgr2=0x7f9bdc14d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be1e00640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9bdc0ce6b0 0x7f9bdc14d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be1e00640 1 -- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bdc0ce2d0 msgr2=0x7f9bdc1f17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be1e00640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bdc0ce2d0 0x7f9bdc1f17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be1e00640 1 -- 172.21.15.113:0/1121475598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc0c9180 con 0x7f9bdc06de80 2024-03-04T07:34:03.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be1e00640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 0x7f9bdc0c9c60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000d830 tx=0x7f9bd000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:03.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9bd6000640 1 -- 172.21.15.113:0/1121475598 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0018020 con 0x7f9bdc06de80 2024-03-04T07:34:03.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9bd6000640 1 -- 172.21.15.113:0/1121475598 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd00096a0 con 0x7f9bdc06de80 2024-03-04T07:34:03.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be4800640 1 -- 172.21.15.113:0/1121475598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bdc1f0d20 con 0x7f9bdc06de80 2024-03-04T07:34:03.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9bd6000640 1 -- 172.21.15.113:0/1121475598 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0009a50 con 0x7f9bdc06de80 2024-03-04T07:34:03.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.602+0000 7f9be4800640 1 -- 172.21.15.113:0/1121475598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bdc0c9700 con 0x7f9bdc06de80 2024-03-04T07:34:03.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.606+0000 7f9be4800640 1 -- 172.21.15.113:0/1121475598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bdc075c00 con 0x7f9bdc06de80 2024-03-04T07:34:03.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.606+0000 7f9bd6000640 1 -- 172.21.15.113:0/1121475598 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bd001b020 con 0x7f9bdc06de80 2024-03-04T07:34:03.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.606+0000 7f9bd6000640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9bc0066f90 0x7f9bc0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.606+0000 7f9bd6000640 1 -- 172.21.15.113:0/1121475598 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bd0085740 con 0x7f9bdc06de80 2024-03-04T07:34:03.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.606+0000 7f9be1400640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9bc0066f90 0x7f9bc0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.606+0000 7f9be1400640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9bc0066f90 0x7f9bc0069450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc15b080 tx=0x7f9bcc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:03.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.610+0000 7f9bd6000640 1 -- 172.21.15.113:0/1121475598 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9bd0022020 con 0x7f9bdc06de80 2024-03-04T07:34:03.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.766+0000 7f9be4800640 1 -- 172.21.15.113:0/1121475598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bdc06cb40 con 0x7f9bdc06de80 2024-03-04T07:34:03.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.766+0000 7f9bd6000640 1 -- 172.21.15.113:0/1121475598 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f9bd0011340 con 0x7f9bdc06de80 2024-03-04T07:34:03.770 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:03.770 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:57.517935+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:34:03.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 -- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9bc0066f90 msgr2=0x7f9bc0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9bc0066f90 0x7f9bc0069450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc15b080 tx=0x7f9bcc008040 comp rx=0 tx=0).stop 2024-03-04T07:34:03.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 -- 172.21.15.113:0/1121475598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 msgr2=0x7f9bdc0c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 0x7f9bdc0c9c60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000d830 tx=0x7f9bd000dcf0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 -- 172.21.15.113:0/1121475598 shutdown_connections 2024-03-04T07:34:03.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9bdc0ce6b0 0x7f9bdc14d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bdc0ce2d0 0x7f9bdc1f17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9bc0066f90 0x7f9bc0069450 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 --2- 172.21.15.113:0/1121475598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9bdc06de80 0x7f9bdc0c9c60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 -- 172.21.15.113:0/1121475598 >> 172.21.15.113:0/1121475598 conn(0x7f9bdc0709a0 msgr2=0x7f9bdc1548e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:03.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 -- 172.21.15.113:0/1121475598 shutdown_connections 2024-03-04T07:34:03.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.770+0000 7f9be4800640 1 -- 172.21.15.113:0/1121475598 wait complete. 2024-03-04T07:34:03.790 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-04T07:34:03.791 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:34:03.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f014da00640 1 Processor -- start 2024-03-04T07:34:03.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f014da00640 1 -- start start 2024-03-04T07:34:03.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f014da00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f014806ff20 0x7f014806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f014da00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01480ce120 0x7f01480ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f014da00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01480ca0e0 con 0x7f0148158010 2024-03-04T07:34:03.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f014da00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014806f450 con 0x7f01480ce120 2024-03-04T07:34:03.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f014da00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014815ad80 con 0x7f014806ff20 2024-03-04T07:34:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f0147400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01480ce120 0x7f01480ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f0146a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0148158010 0x7f014815b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49498/0 (socket says 172.21.15.113:49498) 2024-03-04T07:34:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f0147400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01480ce120 0x7f01480ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57112/0 (socket says 172.21.15.113:57112) 2024-03-04T07:34:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f0146a00640 1 -- 172.21.15.113:0/2754950683 learned_addr learned my addr 172.21.15.113:0/2754950683 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.922+0000 7f013e000640 1 --2- 172.21.15.113:0/2754950683 >> v2:172.21.15.136:3300/0 conn(0x7f014806ff20 0x7f014806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f0147400640 1 -- 172.21.15.113:0/2754950683 >> v2:172.21.15.136:3300/0 conn(0x7f014806ff20 msgr2=0x7f014806dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f0147400640 1 --2- 172.21.15.113:0/2754950683 >> v2:172.21.15.136:3300/0 conn(0x7f014806ff20 0x7f014806dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f0147400640 1 -- 172.21.15.113:0/2754950683 >> v1:172.21.15.113:6789/0 conn(0x7f0148158010 legacy=0x7f014815b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f0147400640 1 -- 172.21.15.113:0/2754950683 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014815aa30 con 0x7f01480ce120 2024-03-04T07:34:03.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f0146000640 1 -- 172.21.15.113:0/2754950683 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687715637 0 0) 0x7f01480ca0e0 con 0x7f0148158010 2024-03-04T07:34:03.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f0147400640 1 --2- 172.21.15.113:0/2754950683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01480ce120 0x7f01480ce5d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f01480cc040 tx=0x7f014001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef748209d67c58 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:03.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f0146000640 1 -- 172.21.15.113:0/2754950683 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01400b2070 con 0x7f01480ce120 2024-03-04T07:34:03.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f0146000640 1 -- 172.21.15.113:0/2754950683 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01400a57f0 con 0x7f01480ce120 2024-03-04T07:34:03.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f0146000640 1 -- 172.21.15.113:0/2754950683 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01400a5bc0 con 0x7f01480ce120 2024-03-04T07:34:03.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f014da00640 1 -- 172.21.15.113:0/2754950683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01480ce120 msgr2=0x7f01480ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f014da00640 1 --2- 172.21.15.113:0/2754950683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01480ce120 0x7f01480ce5d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f01480cc040 tx=0x7f014001c930 comp rx=0 tx=0).stop 2024-03-04T07:34:03.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f014da00640 1 -- 172.21.15.113:0/2754950683 shutdown_connections 2024-03-04T07:34:03.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f014da00640 1 --2- 172.21.15.113:0/2754950683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01480ce120 0x7f01480ce5d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f014da00640 1 --2- 172.21.15.113:0/2754950683 >> v2:172.21.15.136:3300/0 conn(0x7f014806ff20 0x7f014806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f014da00640 1 -- 172.21.15.113:0/2754950683 >> 172.21.15.113:0/2754950683 conn(0x7f01480734d0 msgr2=0x7f0148066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:03.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f014da00640 1 -- 172.21.15.113:0/2754950683 shutdown_connections 2024-03-04T07:34:03.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.926+0000 7f014da00640 1 -- 172.21.15.113:0/2754950683 wait complete. 2024-03-04T07:34:03.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f014da00640 1 Processor -- start 2024-03-04T07:34:03.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f014da00640 1 -- start start 2024-03-04T07:34:03.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f014da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f014806ff20 0x7f01481f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f014da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480ce120 0x7f01481ed410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f014da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0148158010 0x7f014814c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f014da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014815b920 con 0x7f014806ff20 2024-03-04T07:34:03.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f014da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014806e290 con 0x7f01480ce120 2024-03-04T07:34:03.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f014da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01481583f0 con 0x7f0148158010 2024-03-04T07:34:03.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f0146a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f014806ff20 0x7f01481f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f0146a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f014806ff20 0x7f01481f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57128/0 (socket says 172.21.15.113:57128) 2024-03-04T07:34:03.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f0146a00640 1 -- 172.21.15.113:0/317791739 learned_addr learned my addr 172.21.15.113:0/317791739 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:03.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f0147400640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0148158010 0x7f014814c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f013e000640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480ce120 0x7f01481ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f0146a00640 1 -- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0148158010 msgr2=0x7f014814c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f0146a00640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0148158010 0x7f014814c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f0146a00640 1 -- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480ce120 msgr2=0x7f01481ed410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f0146a00640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480ce120 0x7f01481ed410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f0146a00640 1 -- 172.21.15.113:0/317791739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01481ed950 con 0x7f014806ff20 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f0146a00640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f014806ff20 0x7f01481f3c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f013800c520 tx=0x7f013800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f013f400640 1 -- 172.21.15.113:0/317791739 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0138018070 con 0x7f014806ff20 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f013f400640 1 -- 172.21.15.113:0/317791739 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01380074e0 con 0x7f014806ff20 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f013f400640 1 -- 172.21.15.113:0/317791739 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01380049f0 con 0x7f014806ff20 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f014da00640 1 -- 172.21.15.113:0/317791739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01480b0260 con 0x7f014806ff20 2024-03-04T07:34:03.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.930+0000 7f014da00640 1 -- 172.21.15.113:0/317791739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01481ec930 con 0x7f014806ff20 2024-03-04T07:34:03.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.934+0000 7f014da00640 1 -- 172.21.15.113:0/317791739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01480ca0e0 con 0x7f014806ff20 2024-03-04T07:34:03.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.934+0000 7f013f400640 1 -- 172.21.15.113:0/317791739 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f013801b070 con 0x7f014806ff20 2024-03-04T07:34:03.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.934+0000 7f013f400640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0124066f60 0x7f0124069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:03.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.934+0000 7f013f400640 1 -- 172.21.15.113:0/317791739 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0138084440 con 0x7f014806ff20 2024-03-04T07:34:03.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.934+0000 7f013e000640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0124066f60 0x7f0124069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:03.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.938+0000 7f013e000640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0124066f60 0x7f0124069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f01480cc190 tx=0x7f0130008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:03.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:03.938+0000 7f013f400640 1 -- 172.21.15.113:0/317791739 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0138055440 con 0x7f014806ff20 2024-03-04T07:34:04.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.094+0000 7f014da00640 1 -- 172.21.15.113:0/317791739 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f014806fd00 con 0x7f014806ff20 2024-03-04T07:34:04.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.094+0000 7f013f400640 1 -- 172.21.15.113:0/317791739 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f0138054d40 con 0x7f014806ff20 2024-03-04T07:34:04.099 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:04.099 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:57.517935+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:34:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 -- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0124066f60 msgr2=0x7f0124069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0124066f60 0x7f0124069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f01480cc190 tx=0x7f0130008040 comp rx=0 tx=0).stop 2024-03-04T07:34:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 -- 172.21.15.113:0/317791739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f014806ff20 msgr2=0x7f01481f3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f014806ff20 0x7f01481f3c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f013800c520 tx=0x7f013800c9e0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 -- 172.21.15.113:0/317791739 shutdown_connections 2024-03-04T07:34:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0148158010 0x7f014814c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01480ce120 0x7f01481ed410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0124066f60 0x7f0124069420 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 --2- 172.21.15.113:0/317791739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f014806ff20 0x7f01481f3c80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 -- 172.21.15.113:0/317791739 >> 172.21.15.113:0/317791739 conn(0x7f01480734d0 msgr2=0x7f0148070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 -- 172.21.15.113:0/317791739 shutdown_connections 2024-03-04T07:34:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.098+0000 7f014da00640 1 -- 172.21.15.113:0/317791739 wait complete. 2024-03-04T07:34:04.123 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-04T07:34:04.124 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-04T07:34:04.124 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-04T07:34:04.125 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-04T07:34:04.125 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-04T07:34:04.125 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-04T07:34:04.126 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-04T07:34:04.126 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-04T07:34:04.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.362+0000 7f606ee00640 1 Processor -- start 2024-03-04T07:34:04.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.362+0000 7fa5d9800640 1 Processor -- start 2024-03-04T07:34:04.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.362+0000 7fa5d9800640 1 -- start start 2024-03-04T07:34:04.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.362+0000 7f606ee00640 1 -- start start 2024-03-04T07:34:04.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.362+0000 7fa5d9800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5d406ed90 0x7fa5d40c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.362+0000 7f606ee00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6060050cb0 0x7f606004c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.362+0000 7fa5d9800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d4071e30 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6067e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6060050cb0 0x7f606004c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.362+0000 7fa5d9800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d4072110 con 0x7fa5d4154140 2024-03-04T07:34:04.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.362+0000 7fa5d9800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d4072a40 con 0x7fa5d406ed90 2024-03-04T07:34:04.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fa5d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5d406ed90 0x7fa5d40c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fa5d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5d406ed90 0x7fa5d40c19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57138/0 (socket says 172.21.15.113:57138) 2024-03-04T07:34:04.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fa5d2a00640 1 -- 172.21.15.113:0/3485208526 learned_addr learned my addr 172.21.15.113:0/3485208526 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f606ee00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606004c820 con 0x7f60600d4f70 2024-03-04T07:34:04.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f606ee00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606004b990 con 0x7f60600dad30 2024-03-04T07:34:04.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f606ee00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f606004bc70 con 0x7f6060050cb0 2024-03-04T07:34:04.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6067400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f60600d4f70 0x7f60600347a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49514/0 (socket says 172.21.15.113:49514) 2024-03-04T07:34:04.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6067400640 1 -- 172.21.15.113:0/2914220979 learned_addr learned my addr 172.21.15.113:0/2914220979 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:04.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fa5d2a00640 1 -- 172.21.15.113:0/3485208526 >> v1:172.21.15.136:6789/0 conn(0x7fa5d4154140 legacy=0x7fa5d406c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fa5d2a00640 1 -- 172.21.15.113:0/3485208526 >> v1:172.21.15.113:6789/0 conn(0x7fa5d40c7ce0 legacy=0x7fa5d4156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fa5d2a00640 1 -- 172.21.15.113:0/3485208526 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40723f0 con 0x7fa5d406ed90 2024-03-04T07:34:04.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6066a00640 1 -- 172.21.15.113:0/2914220979 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142942147 0 0) 0x7f606004b990 con 0x7f60600dad30 2024-03-04T07:34:04.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6066a00640 1 -- 172.21.15.113:0/2914220979 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6050003530 con 0x7f60600dad30 2024-03-04T07:34:04.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6066a00640 1 -- 172.21.15.113:0/2914220979 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229480554 0 0) 0x7f6054001120 con 0x7f60600d4f70 2024-03-04T07:34:04.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6066a00640 1 -- 172.21.15.113:0/2914220979 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f606004b990 con 0x7f60600d4f70 2024-03-04T07:34:04.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fdb25000640 1 Processor -- start 2024-03-04T07:34:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fdb25000640 1 -- start start 2024-03-04T07:34:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fdb25000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdb20155d80 0x7fdb200c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fdb25000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 0x7fdb20162c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fdb25000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdb201631b0 0x7fdb200c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6067e00640 1 -- 172.21.15.113:0/2914220979 >> v1:172.21.15.136:6789/0 conn(0x7f60600dad30 legacy=0x7f60600db140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6066a00640 1 -- 172.21.15.113:0/2914220979 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4179825935 0 0) 0x7f606004b990 con 0x7f60600d4f70 2024-03-04T07:34:04.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fdb1d400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 0x7fdb20162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7fdb1d400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 0x7fdb20162c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47008/0 (socket says 172.21.15.113:47008) 2024-03-04T07:34:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdb20155d80 0x7fdb200c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdb20155d80 0x7fdb200c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57154/0 (socket says 172.21.15.113:57154) 2024-03-04T07:34:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1d400640 1 -- 172.21.15.113:0/3700896161 learned_addr learned my addr 172.21.15.113:0/3700896161 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb25000640 1 -- 172.21.15.113:0/3700896161 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb20162500 con 0x7fdb20155d80 2024-03-04T07:34:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb25000640 1 -- 172.21.15.113:0/3700896161 --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb20156160 con 0x7fdb201631b0 2024-03-04T07:34:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb25000640 1 -- 172.21.15.113:0/3700896161 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2015ef10 con 0x7fdb200c1860 2024-03-04T07:34:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1e800640 1 --2- 172.21.15.113:0/3700896161 >> v2:172.21.15.136:3300/0 conn(0x7fdb201631b0 0x7fdb200c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6067e00640 1 -- 172.21.15.113:0/2914220979 >> v1:172.21.15.113:6789/0 conn(0x7f60600d4f70 legacy=0x7f60600347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.366+0000 7f6067e00640 1 -- 172.21.15.113:0/2914220979 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6060006a50 con 0x7f6060050cb0 2024-03-04T07:34:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1e800640 1 -- 172.21.15.113:0/3700896161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 msgr2=0x7fdb20162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fa5d2a00640 1 --2- 172.21.15.113:0/3485208526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d406ed90 0x7fa5d40c19a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009870 tx=0x7fa5c401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb238321e4892c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1e800640 1 --2- 172.21.15.113:0/3700896161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 0x7fdb20162c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1e800640 1 -- 172.21.15.113:0/3700896161 >> v2:172.21.15.113:3300/0 conn(0x7fdb20155d80 msgr2=0x7fdb200c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1e800640 1 --2- 172.21.15.113:0/3700896161 >> v2:172.21.15.113:3300/0 conn(0x7fdb20155d80 0x7fdb200c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1e800640 1 -- 172.21.15.113:0/3700896161 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2015f1b0 con 0x7fdb201631b0 2024-03-04T07:34:04.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fa5d1600640 1 -- 172.21.15.113:0/3485208526 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40b5070 con 0x7fa5d406ed90 2024-03-04T07:34:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fa5d1600640 1 -- 172.21.15.113:0/3485208526 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40a89d0 con 0x7fa5d406ed90 2024-03-04T07:34:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fa5d1600640 1 -- 172.21.15.113:0/3485208526 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40a8da0 con 0x7fa5d406ed90 2024-03-04T07:34:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7f6067e00640 1 --2- 172.21.15.113:0/2914220979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6060050cb0 0x7f606004c2e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6068050930 tx=0x7f606809b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59530422cd1e7d57 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1de00640 1 --2- 172.21.15.113:0/3700896161 >> v2:172.21.15.113:3300/0 conn(0x7fdb20155d80 0x7fdb200c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:34:04.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.370+0000 7fdb1d400640 1 --2- 172.21.15.113:0/3700896161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 0x7fdb20162c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:34:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f6066a00640 1 -- 172.21.15.113:0/2914220979 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068130070 con 0x7f6060050cb0 2024-03-04T07:34:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7fa5d9800640 1 -- 172.21.15.113:0/3485208526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d406ed90 msgr2=0x7fa5d40c19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7fa5d9800640 1 --2- 172.21.15.113:0/3485208526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d406ed90 0x7fa5d40c19a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009870 tx=0x7fa5c401ecd0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f101ea00640 1 Processor -- start 2024-03-04T07:34:04.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f101ea00640 1 -- start start 2024-03-04T07:34:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f6066a00640 1 -- 172.21.15.113:0/2914220979 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606812b400 con 0x7f6060050cb0 2024-03-04T07:34:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7fa5d9800640 1 -- 172.21.15.113:0/3485208526 shutdown_connections 2024-03-04T07:34:04.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f6066a00640 1 -- 172.21.15.113:0/2914220979 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068130070 con 0x7f6060050cb0 2024-03-04T07:34:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7fa5d9800640 1 --2- 172.21.15.113:0/3485208526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d406ed90 0x7fa5d40c19a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7fa5d9800640 1 -- 172.21.15.113:0/3485208526 >> 172.21.15.113:0/3485208526 conn(0x7fa5d406e2a0 msgr2=0x7fa5d406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7fa5d9800640 1 -- 172.21.15.113:0/3485208526 shutdown_connections 2024-03-04T07:34:04.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7fa5d9800640 1 -- 172.21.15.113:0/3485208526 wait complete. 2024-03-04T07:34:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f606ee00640 1 -- 172.21.15.113:0/2914220979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6060050cb0 msgr2=0x7f606004c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f606ee00640 1 --2- 172.21.15.113:0/2914220979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6060050cb0 0x7f606004c2e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6068050930 tx=0x7f606809b400 comp rx=0 tx=0).stop 2024-03-04T07:34:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f606ee00640 1 -- 172.21.15.113:0/2914220979 shutdown_connections 2024-03-04T07:34:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f606ee00640 1 --2- 172.21.15.113:0/2914220979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6060050cb0 0x7f606004c2e0 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6068050930 tx=0x7f606809b400 comp rx=0 tx=0).stop 2024-03-04T07:34:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f606ee00640 1 -- 172.21.15.113:0/2914220979 >> 172.21.15.113:0/2914220979 conn(0x7f6060050190 msgr2=0x7f60600505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f606ee00640 1 -- 172.21.15.113:0/2914220979 shutdown_connections 2024-03-04T07:34:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f606ee00640 1 -- 172.21.15.113:0/2914220979 wait complete. 2024-03-04T07:34:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f101ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 0x7f10180c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f101ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f101806b060 0x7f101806b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f101ea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10180c1ec0 con 0x7f10180c85f0 2024-03-04T07:34:04.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f101ea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10180c9b30 con 0x7f101806b060 2024-03-04T07:34:04.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f101ea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10180c12a0 con 0x7f10180b2020 2024-03-04T07:34:04.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f2c9c600640 1 Processor -- start 2024-03-04T07:34:04.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f2c9c600640 1 -- start start 2024-03-04T07:34:04.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f1017e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 0x7f10180c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f18ee200640 1 Processor -- start 2024-03-04T07:34:04.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7fdb1e800640 1 --2- 172.21.15.113:0/3700896161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb201631b0 0x7fdb200c7050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb2006e4a0 tx=0x7fdb0c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6ab5f48ff507a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d9800640 1 Processor -- start 2024-03-04T07:34:04.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f1017e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 0x7f10180c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47022/0 (socket says 172.21.15.113:47022) 2024-03-04T07:34:04.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f1017e00640 1 -- 172.21.15.113:0/1962440963 learned_addr learned my addr 172.21.15.113:0/1962440963 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.374+0000 7f2c9c600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2c94070c60 0x7f2c9415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c99c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2c94070c60 0x7f2c9415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f18ee200640 1 -- start start 2024-03-04T07:34:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb1ca00640 1 -- 172.21.15.113:0/3700896161 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0b0070 con 0x7fdb201631b0 2024-03-04T07:34:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c99c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2c94070c60 0x7f2c9415ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47818/0 (socket says 172.21.15.113:47818) 2024-03-04T07:34:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c99200640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f2c940c72a0 0x7f2c940c2970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55072/0 (socket says 172.21.15.113:55072) 2024-03-04T07:34:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c9a600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2c940c1980 0x7f2c940c8910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49536/0 (socket says 172.21.15.113:49536) 2024-03-04T07:34:04.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f606ee00640 1 Processor -- start 2024-03-04T07:34:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb1ca00640 1 -- 172.21.15.113:0/3700896161 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c0ab600 con 0x7fdb201631b0 2024-03-04T07:34:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb1ca00640 1 -- 172.21.15.113:0/3700896161 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0ab9d0 con 0x7fdb201631b0 2024-03-04T07:34:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f101ca00640 1 --2- 172.21.15.113:0/1962440963 >> v2:172.21.15.113:3300/0 conn(0x7f101806b060 0x7f101806b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c9c600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c940c7e70 con 0x7f2c940c1980 2024-03-04T07:34:04.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c9c600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c940c8130 con 0x7f2c940c72a0 2024-03-04T07:34:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c9c600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c940c9640 con 0x7f2c94070c60 2024-03-04T07:34:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c99c00640 1 -- 172.21.15.113:0/2344768395 learned_addr learned my addr 172.21.15.113:0/2344768395 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f606ee00640 1 -- start start 2024-03-04T07:34:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c93e00640 1 -- 172.21.15.113:0/2344768395 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690359288 0 0) 0x7f2c940c7e70 con 0x7f2c940c1980 2024-03-04T07:34:04.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f1016a00640 1 -- 172.21.15.113:0/1962440963 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881051750 0 0) 0x7f10180c1ec0 con 0x7f10180c85f0 2024-03-04T07:34:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f1017e00640 1 -- 172.21.15.113:0/1962440963 >> v2:172.21.15.113:3300/0 conn(0x7f101806b060 msgr2=0x7f101806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f1017e00640 1 --2- 172.21.15.113:0/1962440963 >> v2:172.21.15.113:3300/0 conn(0x7f101806b060 0x7f101806b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f1017e00640 1 -- 172.21.15.113:0/1962440963 >> v1:172.21.15.113:6789/0 conn(0x7f10180c85f0 legacy=0x7f1018157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f1017e00640 1 -- 172.21.15.113:0/1962440963 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10180c1580 con 0x7f10180b2020 2024-03-04T07:34:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb25000640 1 -- 172.21.15.113:0/3700896161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb201631b0 msgr2=0x7fdb200c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb25000640 1 --2- 172.21.15.113:0/3700896161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb201631b0 0x7fdb200c7050 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb2006e4a0 tx=0x7fdb0c01c480 comp rx=0 tx=0).stop 2024-03-04T07:34:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d9800640 1 -- start start 2024-03-04T07:34:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d9800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5d406ed90 0x7fa5d40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c7ce0 0x7fa5d4155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d9800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5d4154140 0x7fa5d41486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d9800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d406ccf0 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d9800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d406d090 con 0x7fa5d4154140 2024-03-04T07:34:04.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d9800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d406ce70 con 0x7fa5d406ed90 2024-03-04T07:34:04.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb25000640 1 -- 172.21.15.113:0/3700896161 shutdown_connections 2024-03-04T07:34:04.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb25000640 1 --2- 172.21.15.113:0/3700896161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb201631b0 0x7fdb200c7050 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb25000640 1 --2- 172.21.15.113:0/3700896161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 0x7fdb20162c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb25000640 1 --2- 172.21.15.113:0/3700896161 >> v2:172.21.15.113:3300/0 conn(0x7fdb20155d80 0x7fdb200c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb25000640 1 -- 172.21.15.113:0/3700896161 >> 172.21.15.113:0/3700896161 conn(0x7fdb2006e820 msgr2=0x7fdb2006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f18ee200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f18e806cde0 0x7f18e81663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f18ee200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e8072710 con 0x7f18e806b6b0 2024-03-04T07:34:04.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f18ee200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e815b710 con 0x7f18e8154d50 2024-03-04T07:34:04.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f18ee200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e8072c90 con 0x7f18e806cde0 2024-03-04T07:34:04.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c93e00640 1 -- 172.21.15.113:0/2344768395 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c7c003530 con 0x7f2c940c1980 2024-03-04T07:34:04.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c99c00640 1 -- 172.21.15.113:0/2344768395 >> v1:172.21.15.136:6789/0 conn(0x7f2c940c72a0 legacy=0x7f2c940c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c99c00640 1 -- 172.21.15.113:0/2344768395 >> v1:172.21.15.113:6789/0 conn(0x7f2c940c1980 legacy=0x7f2c940c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c99c00640 1 -- 172.21.15.113:0/2344768395 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940c8410 con 0x7f2c94070c60 2024-03-04T07:34:04.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f2c99200640 1 --1- 172.21.15.113:0/2344768395 >> v1:172.21.15.136:6789/0 conn(0x7f2c940c72a0 0x7f2c940c2970 :-1 s=CLOSED pgs=0 cs=0 l=0).handle_server_banner_and_identify state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-04T07:34:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5d4154140 0x7fa5d41486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb25000640 1 -- 172.21.15.113:0/3700896161 shutdown_connections 2024-03-04T07:34:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f18e7400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f18e8154d50 0x7f18e80c20a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55074/0 (socket says 172.21.15.113:55074) 2024-03-04T07:34:04.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f18e7400640 1 -- 172.21.15.113:0/4093947726 learned_addr learned my addr 172.21.15.113:0/4093947726 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5d4154140 0x7fa5d41486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47830/0 (socket says 172.21.15.113:47830) 2024-03-04T07:34:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fa5d3400640 1 -- 172.21.15.113:0/1966814966 learned_addr learned my addr 172.21.15.113:0/1966814966 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fdb25000640 1 -- 172.21.15.113:0/3700896161 wait complete. 2024-03-04T07:34:04.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fd96b400640 1 Processor -- start 2024-03-04T07:34:04.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f18e7e00640 1 --2- 172.21.15.113:0/4093947726 >> v2:172.21.15.136:3300/0 conn(0x7f18e806cde0 0x7f18e81663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7fd96b400640 1 -- start start 2024-03-04T07:34:04.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.378+0000 7f606ee00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60600d4f70 0x7f60600e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ee00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60600dad30 0x7f60600d9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60600e4220 0x7f606017c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6060045130 con 0x7f60600e4220 2024-03-04T07:34:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ee00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6060007010 con 0x7f60600dad30 2024-03-04T07:34:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ee00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60600d53e0 con 0x7f60600d4f70 2024-03-04T07:34:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60600e4220 0x7f606017c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f18e6000640 1 -- 172.21.15.113:0/4093947726 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284153908 0 0) 0x7f18e815b710 con 0x7f18e8154d50 2024-03-04T07:34:04.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60600e4220 0x7f606017c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57182/0 (socket says 172.21.15.113:57182) 2024-03-04T07:34:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ce00640 1 -- 172.21.15.113:0/2155575912 learned_addr learned my addr 172.21.15.113:0/2155575912 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f18e6000640 1 -- 172.21.15.113:0/4093947726 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18cc003530 con 0x7f18e8154d50 2024-03-04T07:34:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f18e6000640 1 -- 172.21.15.113:0/4093947726 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665743962 0 0) 0x7f18e8072710 con 0x7f18e806b6b0 2024-03-04T07:34:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f18e6000640 1 -- 172.21.15.113:0/4093947726 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e815b710 con 0x7f18e806b6b0 2024-03-04T07:34:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f18e7e00640 1 -- 172.21.15.113:0/4093947726 >> v1:172.21.15.136:6789/0 conn(0x7f18e8154d50 legacy=0x7f18e80c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f18e7e00640 1 -- 172.21.15.113:0/4093947726 >> v1:172.21.15.113:6789/0 conn(0x7f18e806b6b0 legacy=0x7f18e80cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f18e7e00640 1 -- 172.21.15.113:0/4093947726 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e80729b0 con 0x7f18e806cde0 2024-03-04T07:34:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fd96b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd964154c00 0x7fd9640c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f6067400640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60600dad30 0x7f60600d9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fd96b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154fe0 0x7fd9640c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ce00640 1 -- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60600d4f70 msgr2=0x7f60600e2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f99f6600640 1 Processor -- start 2024-03-04T07:34:04.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f6067e00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60600d4f70 0x7f60600e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ce00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60600d4f70 0x7f60600e2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ce00640 1 -- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60600dad30 msgr2=0x7f60600d9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ce00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60600dad30 0x7f60600d9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ce00640 1 -- 172.21.15.113:0/2155575912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60600e3410 con 0x7f60600e4220 2024-03-04T07:34:04.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fd963e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154fe0 0x7fd9640c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f99f6600640 1 -- start start 2024-03-04T07:34:04.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fd963e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154fe0 0x7fd9640c1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47054/0 (socket says 172.21.15.113:47054) 2024-03-04T07:34:04.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fd963e00640 1 -- 172.21.15.113:0/3285586275 learned_addr learned my addr 172.21.15.113:0/3285586275 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5d2000640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c7ce0 0x7fa5d4155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5d2a00640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5d406ed90 0x7fa5d40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5d2000640 1 -- 172.21.15.113:0/1966814966 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5d406ed90 msgr2=0x7fa5d40b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5d2000640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5d406ed90 0x7fa5d40b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5d2000640 1 -- 172.21.15.113:0/1966814966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5d4154140 msgr2=0x7fa5d41486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5d2000640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5d4154140 0x7fa5d41486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5d2000640 1 -- 172.21.15.113:0/1966814966 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d41f51b0 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5d2000640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c7ce0 0x7fa5d4155c80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00b440 tx=0x7fa5bc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5caa00640 1 -- 172.21.15.113:0/1966814966 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc011830 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5caa00640 1 -- 172.21.15.113:0/1966814966 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc016070 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5caa00640 1 -- 172.21.15.113:0/1966814966 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc011c10 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f6067400640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60600dad30 0x7f60600d9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:34:04.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f6067e00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60600d4f70 0x7f60600e2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:34:04.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5d9800640 1 -- 172.21.15.113:0/1966814966 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d41f4630 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fa5d9800640 1 -- 172.21.15.113:0/1966814966 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d41f4bf0 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fd96b400640 1 -- 172.21.15.113:0/3285586275 --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9640c7d80 con 0x7fd9640c7700 2024-03-04T07:34:04.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fd96b400640 1 -- 172.21.15.113:0/3285586275 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96415efb0 con 0x7fd964154c00 2024-03-04T07:34:04.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fd96b400640 1 -- 172.21.15.113:0/3285586275 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640c6b70 con 0x7fd964154fe0 2024-03-04T07:34:04.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7fd968a00640 1 --2- 172.21.15.113:0/3285586275 >> v2:172.21.15.113:3300/0 conn(0x7fd964154c00 0x7fd9640c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fd963400640 1 -- 172.21.15.113:0/3285586275 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987048928 0 0) 0x7fd9640c7d80 con 0x7fd9640c7700 2024-03-04T07:34:04.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.382+0000 7f606ce00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60600e4220 0x7f606017c2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f605c00c410 tx=0x7f605c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f1017e00640 1 --2- 172.21.15.113:0/1962440963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 0x7f10180c8c70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1008009870 tx=0x7f100801ed90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25df391cacc0c1e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f1016a00640 1 -- 172.21.15.113:0/1962440963 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080b5070 con 0x7f10180b2020 2024-03-04T07:34:04.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c99c00640 1 --2- 172.21.15.113:0/2344768395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c94070c60 0x7f2c9415ac60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009e50 tx=0x7f2c8c01c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f71ce32b36aca2f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c93e00640 1 -- 172.21.15.113:0/2344768395 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c0a5620 con 0x7f2c94070c60 2024-03-04T07:34:04.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c93e00640 1 -- 172.21.15.113:0/2344768395 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c0a57c0 con 0x7f2c94070c60 2024-03-04T07:34:04.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c93e00640 1 -- 172.21.15.113:0/2344768395 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c0a5b70 con 0x7f2c94070c60 2024-03-04T07:34:04.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fd963400640 1 -- 172.21.15.113:0/3285586275 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd948003530 con 0x7fd9640c7700 2024-03-04T07:34:04.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fd963e00640 1 -- 172.21.15.113:0/3285586275 >> v2:172.21.15.113:3300/0 conn(0x7fd964154c00 msgr2=0x7fd9640c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fd963e00640 1 --2- 172.21.15.113:0/3285586275 >> v2:172.21.15.113:3300/0 conn(0x7fd964154c00 0x7fd9640c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fd963e00640 1 -- 172.21.15.113:0/3285586275 >> v1:172.21.15.136:6789/0 conn(0x7fd9640c7700 legacy=0x7fd96415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fd963e00640 1 -- 172.21.15.113:0/3285586275 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9640c8060 con 0x7fd964154fe0 2024-03-04T07:34:04.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fdb25000640 1 Processor -- start 2024-03-04T07:34:04.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f1016a00640 1 -- 172.21.15.113:0/1962440963 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10080a8a50 con 0x7f10180b2020 2024-03-04T07:34:04.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f1016a00640 1 -- 172.21.15.113:0/1962440963 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080a8e20 con 0x7f10180b2020 2024-03-04T07:34:04.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fa5c3e00640 1 -- 172.21.15.113:0/1966814966 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa5ac001040 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f605be00640 1 -- 172.21.15.113:0/2155575912 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c017070 con 0x7f60600e4220 2024-03-04T07:34:04.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f605be00640 1 -- 172.21.15.113:0/2155575912 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c0052d0 con 0x7f60600e4220 2024-03-04T07:34:04.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f605be00640 1 -- 172.21.15.113:0/2155575912 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c0054c0 con 0x7f60600e4220 2024-03-04T07:34:04.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60600d40e0 con 0x7f60600e4220 2024-03-04T07:34:04.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60600e3990 con 0x7f60600e4220 2024-03-04T07:34:04.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99f6600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f99f01549b0 0x7f99f015b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c9c600640 1 -- 172.21.15.113:0/2344768395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c94070c60 msgr2=0x7f2c9415ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c9c600640 1 --2- 172.21.15.113:0/2344768395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c94070c60 0x7f2c9415ac60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009e50 tx=0x7f2c8c01c980 comp rx=0 tx=0).stop 2024-03-04T07:34:04.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f101ea00640 1 -- 172.21.15.113:0/1962440963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 msgr2=0x7f10180c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f101ea00640 1 --2- 172.21.15.113:0/1962440963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 0x7f10180c8c70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1008009870 tx=0x7f100801ed90 comp rx=0 tx=0).stop 2024-03-04T07:34:04.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fa5caa00640 1 -- 172.21.15.113:0/1966814966 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5bc004020 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c9c600640 1 -- 172.21.15.113:0/2344768395 shutdown_connections 2024-03-04T07:34:04.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c9c600640 1 --2- 172.21.15.113:0/2344768395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c94070c60 0x7f2c9415ac60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99f6600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f0154d90 0x7f99f00d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fd968a00640 1 --2- 172.21.15.113:0/3285586275 >> v2:172.21.15.113:3300/0 conn(0x7fd964154c00 0x7fd9640c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:34:04.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99f6600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f00cae80 con 0x7f99f006cb30 2024-03-04T07:34:04.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c9c600640 1 -- 172.21.15.113:0/2344768395 >> 172.21.15.113:0/2344768395 conn(0x7f2c9406e820 msgr2=0x7f2c9406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99f6600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f015b840 con 0x7f99f01549b0 2024-03-04T07:34:04.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99f6600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f015e170 con 0x7f99f0154d90 2024-03-04T07:34:04.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f101ea00640 1 -- 172.21.15.113:0/1962440963 shutdown_connections 2024-03-04T07:34:04.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f101ea00640 1 --2- 172.21.15.113:0/1962440963 >> v2:172.21.15.113:3300/0 conn(0x7f101806b060 0x7f101806b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f101ea00640 1 --2- 172.21.15.113:0/1962440963 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 0x7f10180c8c70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f101ea00640 1 -- 172.21.15.113:0/1962440963 >> 172.21.15.113:0/1962440963 conn(0x7f101806e820 msgr2=0x7f101806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6044001040 con 0x7f60600e4220 2024-03-04T07:34:04.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f101ea00640 1 -- 172.21.15.113:0/1962440963 shutdown_connections 2024-03-04T07:34:04.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99efe00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f99f006cb30 0x7f99f00c58a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55084/0 (socket says 172.21.15.113:55084) 2024-03-04T07:34:04.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99efe00640 1 -- 172.21.15.113:0/1945966045 learned_addr learned my addr 172.21.15.113:0/1945966045 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:04.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99ef400640 1 --2- 172.21.15.113:0/1945966045 >> v2:172.21.15.136:3300/0 conn(0x7f99f01549b0 0x7f99f015b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99eea00640 1 --2- 172.21.15.113:0/1945966045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f0154d90 0x7f99f00d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f18e7e00640 1 --2- 172.21.15.113:0/4093947726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e806cde0 0x7f18e81663c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18d80097f0 tx=0x7f18d801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e33444336c1091d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c9c600640 1 -- 172.21.15.113:0/2344768395 shutdown_connections 2024-03-04T07:34:04.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f101ea00640 1 -- 172.21.15.113:0/1962440963 wait complete. 2024-03-04T07:34:04.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f18e6000640 1 -- 172.21.15.113:0/4093947726 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d80b0070 con 0x7f18e806cde0 2024-03-04T07:34:04.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f2c9c600640 1 -- 172.21.15.113:0/2344768395 wait complete. 2024-03-04T07:34:04.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f18e6000640 1 -- 172.21.15.113:0/4093947726 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d80ab600 con 0x7f18e806cde0 2024-03-04T07:34:04.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f18e6000640 1 -- 172.21.15.113:0/4093947726 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d80ab9d0 con 0x7f18e806cde0 2024-03-04T07:34:04.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99eea00640 1 -- 172.21.15.113:0/1945966045 >> v2:172.21.15.136:3300/0 conn(0x7f99f01549b0 msgr2=0x7f99f015b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99eea00640 1 --2- 172.21.15.113:0/1945966045 >> v2:172.21.15.136:3300/0 conn(0x7f99f01549b0 0x7f99f015b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99eea00640 1 -- 172.21.15.113:0/1945966045 >> v1:172.21.15.136:6789/0 conn(0x7f99f006cb30 legacy=0x7f99f00c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7f99eea00640 1 -- 172.21.15.113:0/1945966045 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f015e450 con 0x7f99f0154d90 2024-03-04T07:34:04.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f99ef400640 1 --2- 172.21.15.113:0/1945966045 >> v2:172.21.15.136:3300/0 conn(0x7f99f01549b0 0x7f99f015b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:04.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fdb25000640 1 -- start start 2024-03-04T07:34:04.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.386+0000 7fa5caa00640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b0066fb0 0x7fa5b0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fa5caa00640 1 -- 172.21.15.113:0/1966814966 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5bc084810 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fa5caa00640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.113:6800/3516786891,v1:172.21.15.113:6801/3516786891] conn(0x7fa5b006f1d0 0x7fa5b00715f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fa5caa00640 1 -- 172.21.15.113:0/1966814966 --> [v2:172.21.15.113:6800/3516786891,v1:172.21.15.113:6801/3516786891] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5b0071ca0 con 0x7fa5b006f1d0 2024-03-04T07:34:04.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fa5caa00640 1 -- 172.21.15.113:0/1966814966 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5bc0892c0 con 0x7fa5d40c7ce0 2024-03-04T07:34:04.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fa5d2a00640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b0066fb0 0x7fa5b0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fa5d3400640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.113:6800/3516786891,v1:172.21.15.113:6801/3516786891] conn(0x7fa5b006f1d0 0x7fa5b00715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f101ea00640 1 Processor -- start 2024-03-04T07:34:04.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f605be00640 1 -- 172.21.15.113:0/2155575912 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f605c01a070 con 0x7f60600e4220 2024-03-04T07:34:04.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f18ee200640 1 -- 172.21.15.113:0/4093947726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e806cde0 msgr2=0x7f18e81663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f18ee200640 1 --2- 172.21.15.113:0/4093947726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e806cde0 0x7f18e81663c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18d80097f0 tx=0x7f18d801c4a0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f101ea00640 1 -- start start 2024-03-04T07:34:04.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f2c9c600640 1 Processor -- start 2024-03-04T07:34:04.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f18ee200640 1 -- 172.21.15.113:0/4093947726 shutdown_connections 2024-03-04T07:34:04.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f18ee200640 1 --2- 172.21.15.113:0/4093947726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e806cde0 0x7f18e81663c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f18ee200640 1 -- 172.21.15.113:0/4093947726 >> 172.21.15.113:0/4093947726 conn(0x7f18e80671b0 msgr2=0x7f18e806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fa5d3400640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.113:6800/3516786891,v1:172.21.15.113:6801/3516786891] conn(0x7fa5b006f1d0 0x7fa5b00715f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb25000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 0x7fdb201f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f18ee200640 1 -- 172.21.15.113:0/4093947726 shutdown_connections 2024-03-04T07:34:04.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f18ee200640 1 -- 172.21.15.113:0/4093947726 wait complete. 2024-03-04T07:34:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f2c9c600640 1 -- start start 2024-03-04T07:34:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb25000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb20155d80 0x7fdb20150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb25000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb201631b0 0x7fdb20150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb25000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2015fcd0 con 0x7fdb201631b0 2024-03-04T07:34:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb25000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2015f9f0 con 0x7fdb20155d80 2024-03-04T07:34:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb25000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb200c1e10 con 0x7fdb200c1860 2024-03-04T07:34:04.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fa5d2a00640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b0066fb0 0x7fa5b0069470 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40b2000 tx=0x7fa5c40aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb1e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb201631b0 0x7fdb20150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb1e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb201631b0 0x7fdb20150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57204/0 (socket says 172.21.15.113:57204) 2024-03-04T07:34:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb1d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb20155d80 0x7fdb20150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb1d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb20155d80 0x7fdb20150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47860/0 (socket says 172.21.15.113:47860) 2024-03-04T07:34:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fdb1d400640 1 -- 172.21.15.113:0/1685416104 learned_addr learned my addr 172.21.15.113:0/1685416104 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7fd963e00640 1 --2- 172.21.15.113:0/3285586275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154fe0 0x7fd9640c1b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd950009870 tx=0x7fd95001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f0be387eee9f6f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f605be00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6048067060 0x7f6048069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f605be00640 1 -- 172.21.15.113:0/2155575912 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f605c0842d0 con 0x7f60600e4220 2024-03-04T07:34:04.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f101ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101806b060 0x7f10181338a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f605be00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.113:6824/596399521,v1:172.21.15.113:6825/596399521] conn(0x7f604806f280 0x7f60480716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f2c9c600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c94070c60 0x7f2c94150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f2c9c600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c940c1980 0x7f2c9414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f2c9c600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c940c72a0 0x7f2c940b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f2c9c600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c94157b10 con 0x7f2c940c1980 2024-03-04T07:34:04.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f2c9c600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c94157680 con 0x7f2c940c72a0 2024-03-04T07:34:04.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.390+0000 7f2c9c600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c94157c90 con 0x7f2c94070c60 2024-03-04T07:34:04.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f101ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 0x7f1018143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f101ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10180c85f0 0x7f1018154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f101ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1018157e20 con 0x7f101806b060 2024-03-04T07:34:04.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fdb1de00640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 0x7fdb201f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f101ea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10181571f0 con 0x7f10180c85f0 2024-03-04T07:34:04.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f1017e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101806b060 0x7f10181338a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f1017e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101806b060 0x7f10181338a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57220/0 (socket says 172.21.15.113:57220) 2024-03-04T07:34:04.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f1017400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 0x7f1018143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f1017e00640 1 -- 172.21.15.113:0/1722854079 learned_addr learned my addr 172.21.15.113:0/1722854079 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10180c2400 con 0x7f10180b2020 2024-03-04T07:34:04.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c940c1980 0x7f2c9414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c940c1980 0x7f2c9414e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57236/0 (socket says 172.21.15.113:57236) 2024-03-04T07:34:04.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99200640 1 -- 172.21.15.113:0/2288133724 learned_addr learned my addr 172.21.15.113:0/2288133724 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99c00640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c94070c60 0x7f2c94150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fdb1e800640 1 -- 172.21.15.113:0/1685416104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 msgr2=0x7fdb201f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fdb1e800640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdb200c1860 0x7fdb201f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fdb1e800640 1 -- 172.21.15.113:0/1685416104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb20155d80 msgr2=0x7fdb20150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fdb1e800640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb20155d80 0x7fdb20150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fdb1e800640 1 -- 172.21.15.113:0/1685416104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb201f87c0 con 0x7fdb201631b0 2024-03-04T07:34:04.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c9a600640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c940c72a0 0x7f2c940b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f605be00640 1 -- 172.21.15.113:0/2155575912 --> [v2:172.21.15.113:6824/596399521,v1:172.21.15.113:6825/596399521] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6048071d50 con 0x7f604806f280 2024-03-04T07:34:04.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f605be00640 1 -- 172.21.15.113:0/2155575912 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f605c08b050 con 0x7f60600e4220 2024-03-04T07:34:04.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f6067e00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6048067060 0x7f6048069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99200640 1 -- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c94070c60 msgr2=0x7f2c94150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18ee200640 1 Processor -- start 2024-03-04T07:34:04.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f6067400640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.113:6824/596399521,v1:172.21.15.113:6825/596399521] conn(0x7f604806f280 0x7f60480716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f99eea00640 1 --2- 172.21.15.113:0/1945966045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f0154d90 0x7f99f00d0270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0097c0 tx=0x7f99dc01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0b6edc890c9723 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99200640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c94070c60 0x7f2c94150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99200640 1 -- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c940c72a0 msgr2=0x7f2c940b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99200640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c940c72a0 0x7f2c940b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99200640 1 -- 172.21.15.113:0/2288133724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c941554d0 con 0x7f2c940c1980 2024-03-04T07:34:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c9a600640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c940c72a0 0x7f2c940b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd963400640 1 -- 172.21.15.113:0/3285586275 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9500b1070 con 0x7fd964154fe0 2024-03-04T07:34:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd963400640 1 -- 172.21.15.113:0/3285586275 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9500ac620 con 0x7fd964154fe0 2024-03-04T07:34:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd963400640 1 -- 172.21.15.113:0/3285586275 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9500ac9f0 con 0x7fd964154fe0 2024-03-04T07:34:04.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f101ca00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10180c85f0 0x7f1018154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fa5caa00640 1 -- 172.21.15.113:0/1966814966 <== osd.1 v2:172.21.15.113:6800/3516786891 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa5b0071ca0 con 0x7fa5b006f1d0 2024-03-04T07:34:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fdb1e800640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb201631b0 0x7fdb20150ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0ab000 tx=0x7fdb0c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99c00640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c94070c60 0x7f2c94150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:34:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f1017e00640 1 -- 172.21.15.113:0/1722854079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 msgr2=0x7f1018143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f1017e00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10180b2020 0x7f1018143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f1017e00640 1 -- 172.21.15.113:0/1722854079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10180c85f0 msgr2=0x7f1018154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f1017e00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10180c85f0 0x7f1018154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f1017e00640 1 -- 172.21.15.113:0/1722854079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1018153f10 con 0x7f101806b060 2024-03-04T07:34:04.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c99200640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c940c1980 0x7f2c9414e050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c4b0 tx=0x7f2c8800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f99ee000640 1 -- 172.21.15.113:0/1945966045 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc0b5070 con 0x7f99f0154d90 2024-03-04T07:34:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f99ee000640 1 -- 172.21.15.113:0/1945966045 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc0a8990 con 0x7f99f0154d90 2024-03-04T07:34:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18ee200640 1 -- start start 2024-03-04T07:34:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f99ee000640 1 -- 172.21.15.113:0/1945966045 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc0a8d60 con 0x7f99f0154d90 2024-03-04T07:34:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd96b400640 1 -- 172.21.15.113:0/3285586275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154fe0 msgr2=0x7fd9640c1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd96b400640 1 --2- 172.21.15.113:0/3285586275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154fe0 0x7fd9640c1b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd950009870 tx=0x7fd95001c4d0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f6067400640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.113:6824/596399521,v1:172.21.15.113:6825/596399521] conn(0x7f604806f280 0x7f60480716a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f1017e00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101806b060 0x7f10181338a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f100801d440 tx=0x7f100801bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18ee200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e806cde0 0x7f18e8152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18ee200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18e8154d50 0x7f18e8164f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18ee200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e814eb80 0x7f18e8165470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f6067e00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6048067060 0x7f6048069520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f606000dc10 tx=0x7f6068126000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c91600640 1 -- 172.21.15.113:0/2288133724 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88018070 con 0x7f2c940c1980 2024-03-04T07:34:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fdb16000640 1 -- 172.21.15.113:0/1685416104 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0b0050 con 0x7fdb201631b0 2024-03-04T07:34:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fdb16000640 1 -- 172.21.15.113:0/1685416104 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c0bc070 con 0x7fdb201631b0 2024-03-04T07:34:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fdb16000640 1 -- 172.21.15.113:0/1685416104 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0aabc0 con 0x7fdb201631b0 2024-03-04T07:34:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c91600640 1 -- 172.21.15.113:0/2288133724 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c88007500 con 0x7f2c940c1980 2024-03-04T07:34:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c91600640 1 -- 172.21.15.113:0/2288133724 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88004a30 con 0x7f2c940c1980 2024-03-04T07:34:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c9c600640 1 -- 172.21.15.113:0/2288133724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c94155770 con 0x7f2c940c1980 2024-03-04T07:34:04.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18e7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e806cde0 0x7f18e8152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18e7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e806cde0 0x7f18e8152d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47894/0 (socket says 172.21.15.113:47894) 2024-03-04T07:34:04.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18e7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e814eb80 0x7f18e8165470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18e6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18e8154d50 0x7f18e8164f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18e7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e814eb80 0x7f18e8165470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57242/0 (socket says 172.21.15.113:57242) 2024-03-04T07:34:04.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18e7400640 1 -- 172.21.15.113:0/1093092879 learned_addr learned my addr 172.21.15.113:0/1093092879 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8169120 con 0x7f18e814eb80 2024-03-04T07:34:04.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8155340 con 0x7f18e806cde0 2024-03-04T07:34:04.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e81692a0 con 0x7f18e8154d50 2024-03-04T07:34:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f99f6600640 1 -- 172.21.15.113:0/1945966045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f0154d90 msgr2=0x7f99f00d0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f99f6600640 1 --2- 172.21.15.113:0/1945966045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f0154d90 0x7f99f00d0270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0097c0 tx=0x7f99dc01ec90 comp rx=0 tx=0).stop 2024-03-04T07:34:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd96b400640 1 -- 172.21.15.113:0/3285586275 shutdown_connections 2024-03-04T07:34:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd96b400640 1 --2- 172.21.15.113:0/3285586275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154fe0 0x7fd9640c1b50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd96b400640 1 --2- 172.21.15.113:0/3285586275 >> v2:172.21.15.113:3300/0 conn(0x7fd964154c00 0x7fd9640c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd96b400640 1 -- 172.21.15.113:0/3285586275 >> 172.21.15.113:0/3285586275 conn(0x7fd96406e820 msgr2=0x7fd96406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd96b400640 1 -- 172.21.15.113:0/3285586275 shutdown_connections 2024-03-04T07:34:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb200b2100 con 0x7fdb201631b0 2024-03-04T07:34:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f100fe00640 1 -- 172.21.15.113:0/1722854079 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080b5070 con 0x7f101806b060 2024-03-04T07:34:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f2c9c600640 1 -- 172.21.15.113:0/2288133724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c94150320 con 0x7f2c940c1980 2024-03-04T07:34:04.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18e7e00640 1 -- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18e8154d50 msgr2=0x7f18e8164f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7f18e7e00640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18e8154d50 0x7f18e8164f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f18e7e00640 1 -- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e806cde0 msgr2=0x7f18e8152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f18e7e00640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18e806cde0 0x7f18e8152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f18e7e00640 1 -- 172.21.15.113:0/1093092879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e80c36c0 con 0x7f18e814eb80 2024-03-04T07:34:04.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1018156dc0 con 0x7f101806b060 2024-03-04T07:34:04.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1018153150 con 0x7f101806b060 2024-03-04T07:34:04.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f18e7e00640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e814eb80 0x7f18e8165470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f18d80ab000 tx=0x7f18d8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f100fe00640 1 -- 172.21.15.113:0/1722854079 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10080a8a50 con 0x7f101806b060 2024-03-04T07:34:04.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f2c9c600640 1 -- 172.21.15.113:0/2288133724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c94075c00 con 0x7f2c940c1980 2024-03-04T07:34:04.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb201513e0 con 0x7fdb201631b0 2024-03-04T07:34:04.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f605be00640 1 -- 172.21.15.113:0/2155575912 <== osd.0 v2:172.21.15.113:6824/596399521 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6048071d50 con 0x7f604806f280 2024-03-04T07:34:04.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f100fe00640 1 -- 172.21.15.113:0/1722854079 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080b5070 con 0x7f101806b060 2024-03-04T07:34:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ff0001040 con 0x7f101806b060 2024-03-04T07:34:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f99f6600640 1 -- 172.21.15.113:0/1945966045 shutdown_connections 2024-03-04T07:34:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f99f6600640 1 --2- 172.21.15.113:0/1945966045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f0154d90 0x7f99f00d0270 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f99f6600640 1 --2- 172.21.15.113:0/1945966045 >> v2:172.21.15.136:3300/0 conn(0x7f99f01549b0 0x7f99f015b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f99f6600640 1 -- 172.21.15.113:0/1945966045 >> 172.21.15.113:0/1945966045 conn(0x7f99f006dc50 msgr2=0x7f99f0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.394+0000 7fd96b400640 1 -- 172.21.15.113:0/3285586275 wait complete. 2024-03-04T07:34:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7fd96b400640 1 Processor -- start 2024-03-04T07:34:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb20075c00 con 0x7fdb201631b0 2024-03-04T07:34:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f18df400640 1 -- 172.21.15.113:0/1093092879 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d80b0040 con 0x7f18e814eb80 2024-03-04T07:34:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e81f91a0 con 0x7f18e814eb80 2024-03-04T07:34:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e8164730 con 0x7f18e814eb80 2024-03-04T07:34:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f18df400640 1 -- 172.21.15.113:0/1093092879 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d80ab400 con 0x7f18e814eb80 2024-03-04T07:34:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7fd96b400640 1 -- start start 2024-03-04T07:34:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7fd96b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9640c7700 0x7fd96415a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7fd96b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154c00 0x7fd96414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7fd96b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964154fe0 0x7fd96414f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f18df400640 1 -- 172.21.15.113:0/1093092879 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d80b0040 con 0x7f18e814eb80 2024-03-04T07:34:04.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.398+0000 7f2c91600640 1 -- 172.21.15.113:0/2288133724 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c88004bd0 con 0x7f2c940c1980 2024-03-04T07:34:04.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f2c91600640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c74066f90 0x7f2c74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd963e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154c00 0x7fd96414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd969400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964154fe0 0x7fd96414f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd969400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964154fe0 0x7fd96414f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47904/0 (socket says 172.21.15.113:47904) 2024-03-04T07:34:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd969400640 1 -- 172.21.15.113:0/4028918701 learned_addr learned my addr 172.21.15.113:0/4028918701 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fdb16000640 1 -- 172.21.15.113:0/1685416104 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb0c0c1460 con 0x7fdb201631b0 2024-03-04T07:34:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd96b400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640c2460 con 0x7fd9640c7700 2024-03-04T07:34:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640c22e0 con 0x7fd964154fe0 2024-03-04T07:34:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd964167c10 con 0x7fd964154c00 2024-03-04T07:34:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f2c91600640 1 -- 172.21.15.113:0/2288133724 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c88084870 con 0x7f2c940c1980 2024-03-04T07:34:04.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18c0001040 con 0x7f18e814eb80 2024-03-04T07:34:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f2c91600640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:6808/196578205,v1:172.21.15.136:6809/196578205] conn(0x7f2c7406f130 0x7f2c74071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f2c91600640 1 -- 172.21.15.113:0/2288133724 --> [v2:172.21.15.136:6808/196578205,v1:172.21.15.136:6809/196578205] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c74071c00 con 0x7f2c7406f130 2024-03-04T07:34:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f2c91600640 1 -- 172.21.15.113:0/2288133724 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c88051240 con 0x7f2c940c1980 2024-03-04T07:34:04.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f99f6600640 1 -- 172.21.15.113:0/1945966045 shutdown_connections 2024-03-04T07:34:04.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fdb16000640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdb00066fe0 0x7fdb000694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd969400640 1 -- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154c00 msgr2=0x7fd96414f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd969400640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd964154c00 0x7fd96414f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f2c9a600640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:6808/196578205,v1:172.21.15.136:6809/196578205] conn(0x7f2c7406f130 0x7f2c74071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd968a00640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9640c7700 0x7fd96415a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd969400640 1 -- 172.21.15.113:0/4028918701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9640c7700 msgr2=0x7fd96415a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd969400640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9640c7700 0x7fd96415a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fdb16000640 1 -- 172.21.15.113:0/1685416104 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb0c11fa20 con 0x7fdb201631b0 2024-03-04T07:34:04.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fdb1de00640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdb00066fe0 0x7fdb000694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fdb16000640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.113:6815/4286907884,v1:172.21.15.113:6817/4286907884] conn(0x7fdb0006f200 0x7fdb00071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fdb16000640 1 -- 172.21.15.113:0/1685416104 --> [v2:172.21.15.113:6815/4286907884,v1:172.21.15.113:6817/4286907884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb0c0bfe80 con 0x7fdb0006f200 2024-03-04T07:34:04.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fdb16000640 1 -- 172.21.15.113:0/1685416104 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb0c1242a0 con 0x7fdb201631b0 2024-03-04T07:34:04.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd969400640 1 -- 172.21.15.113:0/4028918701 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96415c610 con 0x7fd964154fe0 2024-03-04T07:34:04.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fd968a00640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9640c7700 0x7fd96415a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:34:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f2c99c00640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c74066f90 0x7f2c74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f99f6600640 1 -- 172.21.15.113:0/1945966045 wait complete. 2024-03-04T07:34:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f100fe00640 1 -- 172.21.15.113:0/1722854079 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10080c2060 con 0x7f101806b060 2024-03-04T07:34:04.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f2c9a600640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:6808/196578205,v1:172.21.15.136:6809/196578205] conn(0x7f2c7406f130 0x7f2c74071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7f18df400640 1 -- 172.21.15.113:0/1093092879 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18d80bb020 con 0x7f18e814eb80 2024-03-04T07:34:04.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fdb1d400640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.113:6815/4286907884,v1:172.21.15.113:6817/4286907884] conn(0x7fdb0006f200 0x7fdb00071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.402+0000 7fdb1de00640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdb00066fe0 0x7fdb000694a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb10006fd0 tx=0x7fdb10008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7fdb1d400640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.113:6815/4286907884,v1:172.21.15.113:6817/4286907884] conn(0x7fdb0006f200 0x7fdb00071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7fd969400640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964154fe0 0x7fd96414f7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd95400c390 tx=0x7fd95400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f100fe00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ff8067390 0x7f0ff8069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f100fe00640 1 -- 172.21.15.113:0/1722854079 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10080a8850 con 0x7f101806b060 2024-03-04T07:34:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f2c99c00640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c74066f90 0x7f2c74069450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0ae000 tx=0x7f2c8c0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7fd960c00640 1 -- 172.21.15.113:0/4028918701 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954017070 con 0x7fd964154fe0 2024-03-04T07:34:04.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f1017400640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ff8067390 0x7f0ff8069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f18df400640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18c8067390 0x7f18c8069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f18df400640 1 -- 172.21.15.113:0/1093092879 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18d80b9020 con 0x7f18e814eb80 2024-03-04T07:34:04.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7fd960c00640 1 -- 172.21.15.113:0/4028918701 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9540052d0 con 0x7fd964154fe0 2024-03-04T07:34:04.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f100fe00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.113:6808/2541121417,v1:172.21.15.113:6809/2541121417] conn(0x7f0ff806f530 0x7f0ff8071950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f100fe00640 1 -- 172.21.15.113:0/1722854079 --> [v2:172.21.15.113:6808/2541121417,v1:172.21.15.113:6809/2541121417] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1008001d40 con 0x7f0ff806f530 2024-03-04T07:34:04.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f100fe00640 1 -- 172.21.15.113:0/1722854079 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10081226d0 con 0x7f101806b060 2024-03-04T07:34:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f18df400640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:6824/1056870180,v1:172.21.15.136:6825/1056870180] conn(0x7f18c806f530 0x7f18c8071950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f18df400640 1 -- 172.21.15.113:0/1093092879 --> [v2:172.21.15.136:6824/1056870180,v1:172.21.15.136:6825/1056870180] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18d80afe90 con 0x7f18c806f530 2024-03-04T07:34:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f18df400640 1 -- 172.21.15.113:0/1093092879 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18d81222c0 con 0x7f18e814eb80 2024-03-04T07:34:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f18e7400640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18c8067390 0x7f18c8069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f99f6600640 1 Processor -- start 2024-03-04T07:34:04.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9640afba0 con 0x7fd964154fe0 2024-03-04T07:34:04.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9640c3470 con 0x7fd964154fe0 2024-03-04T07:34:04.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7fdb16000640 1 -- 172.21.15.113:0/1685416104 <== osd.2 v2:172.21.15.113:6815/4286907884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdb0c0bfe80 con 0x7fdb0006f200 2024-03-04T07:34:04.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f99f6600640 1 -- start start 2024-03-04T07:34:04.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f101ca00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.113:6808/2541121417,v1:172.21.15.113:6809/2541121417] conn(0x7f0ff806f530 0x7f0ff8071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f2c91600640 1 -- 172.21.15.113:0/2288133724 <== osd.4 v2:172.21.15.136:6808/196578205 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2c74071c00 con 0x7f2c7406f130 2024-03-04T07:34:04.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f18e6a00640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:6824/1056870180,v1:172.21.15.136:6825/1056870180] conn(0x7f18c806f530 0x7f18c8071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7fd960c00640 1 -- 172.21.15.113:0/4028918701 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9540054c0 con 0x7fd964154fe0 2024-03-04T07:34:04.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd93c001040 con 0x7fd964154fe0 2024-03-04T07:34:04.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f18e6a00640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:6824/1056870180,v1:172.21.15.136:6825/1056870180] conn(0x7f18c806f530 0x7f18c8071950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f101ca00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.113:6808/2541121417,v1:172.21.15.113:6809/2541121417] conn(0x7f0ff806f530 0x7f0ff8071950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f99f6600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f006cb30 0x7f99f015c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f99f6600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99f01549b0 0x7f99f015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f99f6600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f0154d90 0x7f99f0146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f99f6600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f015ecf0 con 0x7f99f0154d90 2024-03-04T07:34:04.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f99f6600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f015ea10 con 0x7f99f01549b0 2024-03-04T07:34:04.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.406+0000 7f99f6600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00c5380 con 0x7f99f006cb30 2024-03-04T07:34:04.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f18e7400640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18c8067390 0x7f18c8069850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18d4000f70 tx=0x7f18d4007e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99ef400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f006cb30 0x7f99f015c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99ef400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f006cb30 0x7f99f015c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47100/0 (socket says 172.21.15.113:47100) 2024-03-04T07:34:04.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99ef400640 1 -- 172.21.15.113:0/3079048422 learned_addr learned my addr 172.21.15.113:0/3079048422 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99efe00640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f0154d90 0x7f99f0146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f1017400640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ff8067390 0x7f0ff8069850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f10180c7940 tx=0x7f1004009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7fd960c00640 1 -- 172.21.15.113:0/4028918701 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd95401a070 con 0x7fd964154fe0 2024-03-04T07:34:04.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f100fe00640 1 -- 172.21.15.113:0/1722854079 <== osd.3 v2:172.21.15.113:6808/2541121417 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1008001d40 con 0x7f0ff806f530 2024-03-04T07:34:04.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f18df400640 1 -- 172.21.15.113:0/1093092879 <== osd.5 v2:172.21.15.136:6824/1056870180 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f18d80afe90 con 0x7f18c806f530 2024-03-04T07:34:04.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7fd960c00640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 0x7fd944069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7fd960c00640 1 -- 172.21.15.113:0/4028918701 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9540849c0 con 0x7fd964154fe0 2024-03-04T07:34:04.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7fd960c00640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:6800/1911155961,v1:172.21.15.136:6801/1911155961] conn(0x7fd94406f130 0x7fd944071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7fd960c00640 1 -- 172.21.15.113:0/4028918701 --> [v2:172.21.15.136:6800/1911155961,v1:172.21.15.136:6801/1911155961] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd954008e50 con 0x7fd94406f130 2024-03-04T07:34:04.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99eea00640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99f01549b0 0x7f99f015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99efe00640 1 -- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f006cb30 msgr2=0x7f99f015c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99efe00640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f99f006cb30 0x7f99f015c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99efe00640 1 -- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99f01549b0 msgr2=0x7f99f015cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99efe00640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99f01549b0 0x7f99f015cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99efe00640 1 -- 172.21.15.113:0/3079048422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0150500 con 0x7f99f0154d90 2024-03-04T07:34:04.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7fd960c00640 1 -- 172.21.15.113:0/4028918701 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd954084d70 con 0x7fd964154fe0 2024-03-04T07:34:04.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99eea00640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f99f01549b0 0x7f99f015cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:04.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7f99efe00640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f0154d90 0x7f99f0146b70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99e000c3c0 tx=0x7f99e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7fd963e00640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:6800/1911155961,v1:172.21.15.136:6801/1911155961] conn(0x7fd94406f130 0x7fd944071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.410+0000 7fd968a00640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 0x7fd944069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.414+0000 7f99e7400640 1 -- 172.21.15.113:0/3079048422 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0017070 con 0x7f99f0154d90 2024-03-04T07:34:04.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.414+0000 7f99e7400640 1 -- 172.21.15.113:0/3079048422 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e0005cf0 con 0x7f99f0154d90 2024-03-04T07:34:04.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.414+0000 7f99e7400640 1 -- 172.21.15.113:0/3079048422 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e00072d0 con 0x7f99f0154d90 2024-03-04T07:34:04.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.414+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f014ff40 con 0x7f99f0154d90 2024-03-04T07:34:04.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.414+0000 7fd963e00640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:6800/1911155961,v1:172.21.15.136:6801/1911155961] conn(0x7fd94406f130 0x7fd944071550 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.414+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f015bf00 con 0x7f99f0154d90 2024-03-04T07:34:04.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.414+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99f0075c00 con 0x7f99f0154d90 2024-03-04T07:34:04.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.414+0000 7fd968a00640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 0x7fd944069450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd95c007f40 tx=0x7fd95c007490 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.414+0000 7fd960c00640 1 -- 172.21.15.113:0/4028918701 <== osd.7 v2:172.21.15.136:6800/1911155961 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd954008e50 con 0x7fd94406f130 2024-03-04T07:34:04.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.418+0000 7f99e7400640 1 -- 172.21.15.113:0/3079048422 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99e001a070 con 0x7f99f0154d90 2024-03-04T07:34:04.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.418+0000 7f99e7400640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f99d0066fb0 0x7f99d0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.418+0000 7f99ef400640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f99d0066fb0 0x7f99d0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.418+0000 7f99e7400640 1 -- 172.21.15.113:0/3079048422 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99e00537f0 con 0x7f99f0154d90 2024-03-04T07:34:04.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.418+0000 7f99e7400640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:6816/1386829518,v1:172.21.15.136:6817/1386829518] conn(0x7f99d006fea0 0x7f99d0072340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.418+0000 7f99e7400640 1 -- 172.21.15.113:0/3079048422 --> [v2:172.21.15.136:6816/1386829518,v1:172.21.15.136:6817/1386829518] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99d0072a10 con 0x7f99d006fea0 2024-03-04T07:34:04.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.418+0000 7f99e7400640 1 -- 172.21.15.113:0/3079048422 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e0052180 con 0x7f99f0154d90 2024-03-04T07:34:04.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.418+0000 7f99eea00640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:6816/1386829518,v1:172.21.15.136:6817/1386829518] conn(0x7f99d006fea0 0x7f99d0072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.418+0000 7f99ef400640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f99d0066fb0 0x7f99d0069470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009bd0 tx=0x7f99e800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 -- 172.21.15.113:0/1966814966 --> [v2:172.21.15.113:6800/3516786891,v1:172.21.15.113:6801/3516786891] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa5ac002de0 con 0x7fa5b006f1d0 2024-03-04T07:34:04.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5caa00640 1 -- 172.21.15.113:0/1966814966 <== osd.1 v2:172.21.15.113:6800/3516786891 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa5ac002de0 con 0x7fa5b006f1d0 2024-03-04T07:34:04.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7f99eea00640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:6816/1386829518,v1:172.21.15.136:6817/1386829518] conn(0x7f99d006fea0 0x7f99d0072340 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 -- 172.21.15.113:0/1966814966 >> [v2:172.21.15.113:6800/3516786891,v1:172.21.15.113:6801/3516786891] conn(0x7fa5b006f1d0 msgr2=0x7fa5b00715f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.113:6800/3516786891,v1:172.21.15.113:6801/3516786891] conn(0x7fa5b006f1d0 0x7fa5b00715f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 -- 172.21.15.113:0/1966814966 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b0066fb0 msgr2=0x7fa5b0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b0066fb0 0x7fa5b0069470 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa5c40b2000 tx=0x7fa5c40aa070 comp rx=0 tx=0).stop 2024-03-04T07:34:04.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 -- 172.21.15.113:0/1966814966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c7ce0 msgr2=0x7fa5d4155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 --2- 172.21.15.113:0/1966814966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c7ce0 0x7fa5d4155c80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00b440 tx=0x7fa5bc00b900 comp rx=0 tx=0).stop 2024-03-04T07:34:04.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5d3400640 1 -- 172.21.15.113:0/1966814966 reap_dead start 2024-03-04T07:34:04.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 -- 172.21.15.113:0/1966814966 shutdown_connections 2024-03-04T07:34:04.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 -- 172.21.15.113:0/1966814966 >> 172.21.15.113:0/1966814966 conn(0x7fa5d406e2a0 msgr2=0x7fa5d406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 -- 172.21.15.113:0/1966814966 shutdown_connections 2024-03-04T07:34:04.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7f99e7400640 1 -- 172.21.15.113:0/3079048422 <== osd.6 v2:172.21.15.136:6816/1386829518 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f99d0072a10 con 0x7f99d006fea0 2024-03-04T07:34:04.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.422+0000 7fa5c3e00640 1 -- 172.21.15.113:0/1966814966 wait complete. 2024-03-04T07:34:04.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.430+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 --> [v2:172.21.15.113:6824/596399521,v1:172.21.15.113:6825/596399521] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6044002de0 con 0x7f604806f280 2024-03-04T07:34:04.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f605be00640 1 -- 172.21.15.113:0/2155575912 <== osd.0 v2:172.21.15.113:6824/596399521 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6044002de0 con 0x7f604806f280 2024-03-04T07:34:04.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 --> [v2:172.21.15.113:6815/4286907884,v1:172.21.15.113:6817/4286907884] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdb200c7bb0 con 0x7fdb0006f200 2024-03-04T07:34:04.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 >> [v2:172.21.15.113:6824/596399521,v1:172.21.15.113:6825/596399521] conn(0x7f604806f280 msgr2=0x7f60480716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ee00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.113:6824/596399521,v1:172.21.15.113:6825/596399521] conn(0x7f604806f280 0x7f60480716a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6048067060 msgr2=0x7f6048069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ee00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6048067060 0x7f6048069520 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f606000dc10 tx=0x7f6068126000 comp rx=0 tx=0).stop 2024-03-04T07:34:04.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60600e4220 msgr2=0x7f606017c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ee00640 1 --2- 172.21.15.113:0/2155575912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60600e4220 0x7f606017c2e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f605c00c410 tx=0x7f605c00c8d0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ce00640 1 -- 172.21.15.113:0/2155575912 reap_dead start 2024-03-04T07:34:04.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 shutdown_connections 2024-03-04T07:34:04.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 >> 172.21.15.113:0/2155575912 conn(0x7f6060050190 msgr2=0x7f60600d92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 shutdown_connections 2024-03-04T07:34:04.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.434+0000 7f606ee00640 1 -- 172.21.15.113:0/2155575912 wait complete. 2024-03-04T07:34:04.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c9c600640 1 -- 172.21.15.113:0/2288133724 --> [v2:172.21.15.136:6808/196578205,v1:172.21.15.136:6809/196578205] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c940cb560 con 0x7f2c7406f130 2024-03-04T07:34:04.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb16000640 1 -- 172.21.15.113:0/1685416104 <== osd.2 v2:172.21.15.113:6815/4286907884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdb200c7bb0 con 0x7fdb0006f200 2024-03-04T07:34:04.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c91600640 1 -- 172.21.15.113:0/2288133724 <== osd.4 v2:172.21.15.136:6808/196578205 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2c940cb560 con 0x7f2c7406f130 2024-03-04T07:34:04.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 >> [v2:172.21.15.113:6815/4286907884,v1:172.21.15.113:6817/4286907884] conn(0x7fdb0006f200 msgr2=0x7fdb00071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb25000640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.113:6815/4286907884,v1:172.21.15.113:6817/4286907884] conn(0x7fdb0006f200 0x7fdb00071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdb00066fe0 msgr2=0x7fdb000694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb25000640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdb00066fe0 0x7fdb000694a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb10006fd0 tx=0x7fdb10008040 comp rx=0 tx=0).stop 2024-03-04T07:34:04.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb201631b0 msgr2=0x7fdb20150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb25000640 1 --2- 172.21.15.113:0/1685416104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb201631b0 0x7fdb20150ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0ab000 tx=0x7fdb0c01acc0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 --> [v2:172.21.15.136:6824/1056870180,v1:172.21.15.136:6825/1056870180] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18c0002de0 con 0x7f18c806f530 2024-03-04T07:34:04.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb1e800640 1 -- 172.21.15.113:0/1685416104 reap_dead start 2024-03-04T07:34:04.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 shutdown_connections 2024-03-04T07:34:04.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 >> 172.21.15.113:0/1685416104 conn(0x7fdb2006e820 msgr2=0x7fdb200ccaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 --> [v2:172.21.15.113:6808/2541121417,v1:172.21.15.113:6809/2541121417] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ff0002de0 con 0x7f0ff806f530 2024-03-04T07:34:04.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c86a00640 1 -- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:6808/196578205,v1:172.21.15.136:6809/196578205] conn(0x7f2c7406f130 msgr2=0x7f2c74071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c86a00640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:6808/196578205,v1:172.21.15.136:6809/196578205] conn(0x7f2c7406f130 0x7f2c74071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 shutdown_connections 2024-03-04T07:34:04.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7fdb25000640 1 -- 172.21.15.113:0/1685416104 wait complete. 2024-03-04T07:34:04.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f18df400640 1 -- 172.21.15.113:0/1093092879 <== osd.5 v2:172.21.15.136:6824/1056870180 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f18c0002de0 con 0x7f18c806f530 2024-03-04T07:34:04.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c86a00640 1 -- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c74066f90 msgr2=0x7f2c74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c86a00640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c74066f90 0x7f2c74069450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c0ae000 tx=0x7f2c8c0a7070 comp rx=0 tx=0).stop 2024-03-04T07:34:04.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c86a00640 1 -- 172.21.15.113:0/2288133724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c940c1980 msgr2=0x7f2c9414e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c86a00640 1 --2- 172.21.15.113:0/2288133724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c940c1980 0x7f2c9414e050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c4b0 tx=0x7f2c8800c970 comp rx=0 tx=0).stop 2024-03-04T07:34:04.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c9a600640 1 -- 172.21.15.113:0/2288133724 reap_dead start 2024-03-04T07:34:04.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c86a00640 1 -- 172.21.15.113:0/2288133724 shutdown_connections 2024-03-04T07:34:04.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.438+0000 7f2c86a00640 1 -- 172.21.15.113:0/2288133724 >> 172.21.15.113:0/2288133724 conn(0x7f2c9406e820 msgr2=0x7f2c94073130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f2c86a00640 1 -- 172.21.15.113:0/2288133724 shutdown_connections 2024-03-04T07:34:04.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f100fe00640 1 -- 172.21.15.113:0/1722854079 <== osd.3 v2:172.21.15.113:6808/2541121417 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ff0002de0 con 0x7f0ff806f530 2024-03-04T07:34:04.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f2c86a00640 1 -- 172.21.15.113:0/2288133724 wait complete. 2024-03-04T07:34:04.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:6824/1056870180,v1:172.21.15.136:6825/1056870180] conn(0x7f18c806f530 msgr2=0x7f18c8071950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18ee200640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:6824/1056870180,v1:172.21.15.136:6825/1056870180] conn(0x7f18c806f530 0x7f18c8071950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18c8067390 msgr2=0x7f18c8069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18ee200640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18c8067390 0x7f18c8069850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f18d4000f70 tx=0x7f18d4007e20 comp rx=0 tx=0).stop 2024-03-04T07:34:04.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e814eb80 msgr2=0x7f18e8165470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18ee200640 1 --2- 172.21.15.113:0/1093092879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18e814eb80 0x7f18e8165470 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f18d80ab000 tx=0x7f18d8019240 comp rx=0 tx=0).stop 2024-03-04T07:34:04.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 >> [v2:172.21.15.113:6808/2541121417,v1:172.21.15.113:6809/2541121417] conn(0x7f0ff806f530 msgr2=0x7f0ff8071950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18e7e00640 1 -- 172.21.15.113:0/1093092879 reap_dead start 2024-03-04T07:34:04.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ea00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.113:6808/2541121417,v1:172.21.15.113:6809/2541121417] conn(0x7f0ff806f530 0x7f0ff8071950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ff8067390 msgr2=0x7f0ff8069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ea00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ff8067390 0x7f0ff8069850 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f10180c7940 tx=0x7f1004009040 comp rx=0 tx=0).stop 2024-03-04T07:34:04.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101806b060 msgr2=0x7f10181338a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ea00640 1 --2- 172.21.15.113:0/1722854079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f101806b060 0x7f10181338a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f100801d440 tx=0x7f100801bb30 comp rx=0 tx=0).stop 2024-03-04T07:34:04.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ca00640 1 -- 172.21.15.113:0/1722854079 reap_dead start 2024-03-04T07:34:04.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 shutdown_connections 2024-03-04T07:34:04.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 >> 172.21.15.113:0/1093092879 conn(0x7f18e80671b0 msgr2=0x7f18e80cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 shutdown_connections 2024-03-04T07:34:04.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 >> 172.21.15.113:0/1722854079 conn(0x7f101806e820 msgr2=0x7f10180cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 --> [v2:172.21.15.136:6800/1911155961,v1:172.21.15.136:6801/1911155961] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd93c002de0 con 0x7fd94406f130 2024-03-04T07:34:04.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 shutdown_connections 2024-03-04T07:34:04.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 shutdown_connections 2024-03-04T07:34:04.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f18ee200640 1 -- 172.21.15.113:0/1093092879 wait complete. 2024-03-04T07:34:04.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.442+0000 7f101ea00640 1 -- 172.21.15.113:0/1722854079 wait complete. 2024-03-04T07:34:04.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd960c00640 1 -- 172.21.15.113:0/4028918701 <== osd.7 v2:172.21.15.136:6800/1911155961 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd93c002de0 con 0x7fd94406f130 2024-03-04T07:34:04.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:6800/1911155961,v1:172.21.15.136:6801/1911155961] conn(0x7fd94406f130 msgr2=0x7fd944071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd96b400640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:6800/1911155961,v1:172.21.15.136:6801/1911155961] conn(0x7fd94406f130 0x7fd944071550 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 msgr2=0x7fd944069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd96b400640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 0x7fd944069450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd95c007f40 tx=0x7fd95c007490 comp rx=0 tx=0).stop 2024-03-04T07:34:04.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964154fe0 msgr2=0x7fd96414f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd96b400640 1 --2- 172.21.15.113:0/4028918701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964154fe0 0x7fd96414f7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd95400c390 tx=0x7fd95400c850 comp rx=0 tx=0).stop 2024-03-04T07:34:04.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd969400640 1 -- 172.21.15.113:0/4028918701 reap_dead start 2024-03-04T07:34:04.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 shutdown_connections 2024-03-04T07:34:04.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 >> 172.21.15.113:0/4028918701 conn(0x7fd96406e820 msgr2=0x7fd9641591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 shutdown_connections 2024-03-04T07:34:04.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.446+0000 7fd96b400640 1 -- 172.21.15.113:0/4028918701 wait complete. 2024-03-04T07:34:04.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.450+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 --> [v2:172.21.15.136:6816/1386829518,v1:172.21.15.136:6817/1386829518] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99f015a8b0 con 0x7f99d006fea0 2024-03-04T07:34:04.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99e7400640 1 -- 172.21.15.113:0/3079048422 <== osd.6 v2:172.21.15.136:6816/1386829518 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f99f015a8b0 con 0x7f99d006fea0 2024-03-04T07:34:04.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:6816/1386829518,v1:172.21.15.136:6817/1386829518] conn(0x7f99d006fea0 msgr2=0x7f99d0072340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99f6600640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:6816/1386829518,v1:172.21.15.136:6817/1386829518] conn(0x7f99d006fea0 0x7f99d0072340 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f99d0066fb0 msgr2=0x7f99d0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99f6600640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f99d0066fb0 0x7f99d0069470 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009bd0 tx=0x7f99e800b040 comp rx=0 tx=0).stop 2024-03-04T07:34:04.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f0154d90 msgr2=0x7f99f0146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99f6600640 1 --2- 172.21.15.113:0/3079048422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99f0154d90 0x7f99f0146b70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99e000c3c0 tx=0x7f99e000c880 comp rx=0 tx=0).stop 2024-03-04T07:34:04.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99efe00640 1 -- 172.21.15.113:0/3079048422 reap_dead start 2024-03-04T07:34:04.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 shutdown_connections 2024-03-04T07:34:04.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 >> 172.21.15.113:0/3079048422 conn(0x7f99f006dc50 msgr2=0x7f99f0159310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 shutdown_connections 2024-03-04T07:34:04.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.454+0000 7f99f6600640 1 -- 172.21.15.113:0/3079048422 wait complete. 2024-03-04T07:34:04.490 INFO:teuthology.orchestra.run.smithi113.stdout:64424509448 2024-03-04T07:34:04.490 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-04T07:34:04.491 INFO:teuthology.orchestra.run.smithi113.stdout:60129542152 2024-03-04T07:34:04.492 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-04T07:34:04.492 INFO:teuthology.orchestra.run.smithi113.stdout:60129542151 2024-03-04T07:34:04.493 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-04T07:34:04.493 INFO:teuthology.orchestra.run.smithi113.stdout:64424509448 2024-03-04T07:34:04.494 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-04T07:34:04.495 INFO:teuthology.orchestra.run.smithi113.stdout:60129542151 2024-03-04T07:34:04.495 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-04T07:34:04.496 INFO:teuthology.orchestra.run.smithi113.stdout:60129542152 2024-03-04T07:34:04.496 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-04T07:34:04.497 INFO:teuthology.orchestra.run.smithi113.stdout:60129542152 2024-03-04T07:34:04.497 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-04T07:34:04.500 INFO:teuthology.orchestra.run.smithi113.stdout:60129542151 2024-03-04T07:34:04.501 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-04T07:34:04.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.730+0000 7f2205000640 1 Processor -- start 2024-03-04T07:34:04.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f2205000640 1 -- start start 2024-03-04T07:34:04.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f50800640 1 Processor -- start 2024-03-04T07:34:04.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f50800640 1 -- start start 2024-03-04T07:34:04.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7fee1ac00640 1 Processor -- start 2024-03-04T07:34:04.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7fee1ac00640 1 -- start start 2024-03-04T07:34:04.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f2205000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2200159010 0x7f22000cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f2205000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22000cb690 0x7f22000cc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f50800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5f481549b0 0x7f5f4815b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f50800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 0x7f5f480c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f50800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f480cae80 con 0x7f5f48154d90 2024-03-04T07:34:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f50800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4815b840 con 0x7f5f481549b0 2024-03-04T07:34:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f50800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815e170 con 0x7f5f4806cb30 2024-03-04T07:34:04.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f4e800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 0x7f5f480c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f4e800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 0x7f5f480c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47116/0 (socket says 172.21.15.113:47116) 2024-03-04T07:34:04.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f4e800640 1 -- 172.21.15.113:0/2020306034 learned_addr learned my addr 172.21.15.113:0/2020306034 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f21ff400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2200159010 0x7f22000cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f5f4de00640 1 --2- 172.21.15.113:0/2020306034 >> v2:172.21.15.136:3300/0 conn(0x7f5f481549b0 0x7f5f4815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f5f4e800640 1 -- 172.21.15.113:0/2020306034 >> v2:172.21.15.136:3300/0 conn(0x7f5f481549b0 msgr2=0x7f5f4815b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f5f4e800640 1 --2- 172.21.15.113:0/2020306034 >> v2:172.21.15.136:3300/0 conn(0x7f5f481549b0 0x7f5f4815b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f5f4e800640 1 -- 172.21.15.113:0/2020306034 >> v1:172.21.15.113:6789/0 conn(0x7f5f48154d90 legacy=0x7f5f480d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.734+0000 7f21ffe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22000cb690 0x7f22000cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f21fea00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f22000c5430 0x7f22000caf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55090/0 (socket says 172.21.15.113:55090) 2024-03-04T07:34:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f21fea00640 1 -- 172.21.15.113:0/504470085 learned_addr learned my addr 172.21.15.113:0/504470085 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f2205000640 1 -- 172.21.15.113:0/504470085 --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220015f130 con 0x7f22000c5430 2024-03-04T07:34:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f2205000640 1 -- 172.21.15.113:0/504470085 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220015fd40 con 0x7f22000cb690 2024-03-04T07:34:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f2205000640 1 -- 172.21.15.113:0/504470085 --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220015f410 con 0x7f2200159010 2024-03-04T07:34:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f5f4e800640 1 -- 172.21.15.113:0/2020306034 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4815e450 con 0x7f5f4806cb30 2024-03-04T07:34:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f21fe000640 1 -- 172.21.15.113:0/504470085 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605692949 0 0) 0x7f220015f130 con 0x7f22000c5430 2024-03-04T07:34:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f21fe000640 1 -- 172.21.15.113:0/504470085 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e4003530 con 0x7f22000c5430 2024-03-04T07:34:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f21ff400640 1 -- 172.21.15.113:0/504470085 >> v2:172.21.15.113:3300/0 conn(0x7f22000cb690 msgr2=0x7f22000cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f21ff400640 1 --2- 172.21.15.113:0/504470085 >> v2:172.21.15.113:3300/0 conn(0x7f22000cb690 0x7f22000cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f21ff400640 1 -- 172.21.15.113:0/504470085 >> v1:172.21.15.136:6789/0 conn(0x7f22000c5430 legacy=0x7f22000caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7f21ff400640 1 -- 172.21.15.113:0/504470085 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220015f6f0 con 0x7f2200159010 2024-03-04T07:34:04.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee1ac00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee141549b0 0x7fee1415b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee1ac00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fee1406cb30 0x7fee140c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee1ac00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee140cae80 con 0x7fee14154d90 2024-03-04T07:34:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee1ac00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1415b840 con 0x7fee1406cb30 2024-03-04T07:34:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee1ac00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1415e170 con 0x7fee141549b0 2024-03-04T07:34:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee18c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fee1406cb30 0x7fee140c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee18c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fee1406cb30 0x7fee140c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47954/0 (socket says 172.21.15.113:47954) 2024-03-04T07:34:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee18c00640 1 -- 172.21.15.113:0/1289778116 learned_addr learned my addr 172.21.15.113:0/1289778116 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee13e00640 1 --2- 172.21.15.113:0/1289778116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee141549b0 0x7fee1415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee18c00640 1 -- 172.21.15.113:0/1289778116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee141549b0 msgr2=0x7fee1415b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee18c00640 1 --2- 172.21.15.113:0/1289778116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee141549b0 0x7fee1415b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.738+0000 7fee18c00640 1 -- 172.21.15.113:0/1289778116 >> v1:172.21.15.113:6789/0 conn(0x7fee14154d90 legacy=0x7fee140d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f21ffe00640 1 --2- 172.21.15.113:0/504470085 >> v2:172.21.15.113:3300/0 conn(0x7f22000cb690 0x7f22000cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:04.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7fee18c00640 1 -- 172.21.15.113:0/1289778116 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1415e450 con 0x7fee1406cb30 2024-03-04T07:34:04.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f3e25400640 1 Processor -- start 2024-03-04T07:34:04.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f3e25400640 1 -- start start 2024-03-04T07:34:04.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f3e25400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 0x7f3e2015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f5f4e800640 1 --2- 172.21.15.113:0/2020306034 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 0x7f5f480c58a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009e50 tx=0x7f5f4401f210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb0db8c4b7bcefa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f3e25400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e200c2d50 0x7f3e200c1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f3e1e800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 0x7f3e2015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f3e1e800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 0x7f3e2015ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47144/0 (socket says 172.21.15.113:47144) 2024-03-04T07:34:04.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f3e1e800640 1 -- 172.21.15.113:0/2059485829 learned_addr learned my addr 172.21.15.113:0/2059485829 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f21ff400640 1 --2- 172.21.15.113:0/504470085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2200159010 0x7f22000cc030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f22000d0160 tx=0x7f21f001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2f96f41e028ae2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f3e1de00640 1 --2- 172.21.15.113:0/2059485829 >> v2:172.21.15.113:3300/0 conn(0x7f3e200c2d50 0x7f3e200c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f3e25400640 1 -- 172.21.15.113:0/2059485829 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e200c7e70 con 0x7f3e20070c60 2024-03-04T07:34:04.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f5f4ca00640 1 -- 172.21.15.113:0/2020306034 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440b4070 con 0x7f5f4806cb30 2024-03-04T07:34:04.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f5f4ca00640 1 -- 172.21.15.113:0/2020306034 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f440af400 con 0x7f5f4806cb30 2024-03-04T07:34:04.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f5f4ca00640 1 -- 172.21.15.113:0/2020306034 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440af7d0 con 0x7f5f4806cb30 2024-03-04T07:34:04.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.742+0000 7f3e25400640 1 -- 172.21.15.113:0/2059485829 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e200c9640 con 0x7f3e200c2d50 2024-03-04T07:34:04.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f3e25400640 1 -- 172.21.15.113:0/2059485829 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2015b1a0 con 0x7f3e200c2970 2024-03-04T07:34:04.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f21fe000640 1 -- 172.21.15.113:0/504470085 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f00b2070 con 0x7f2200159010 2024-03-04T07:34:04.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee18c00640 1 --2- 172.21.15.113:0/1289778116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee1406cb30 0x7fee140c58a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009e50 tx=0x7fee0c01f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63ce47c5e85c7e35 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f21fe000640 1 -- 172.21.15.113:0/504470085 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f00a9070 con 0x7f2200159010 2024-03-04T07:34:04.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f21fe000640 1 -- 172.21.15.113:0/504470085 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f00ad880 con 0x7f2200159010 2024-03-04T07:34:04.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f5f50800640 1 -- 172.21.15.113:0/2020306034 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 msgr2=0x7f5f480c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f5f50800640 1 --2- 172.21.15.113:0/2020306034 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 0x7f5f480c58a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5f44009e50 tx=0x7f5f4401f210 comp rx=0 tx=0).stop 2024-03-04T07:34:04.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f3e1d400640 1 -- 172.21.15.113:0/2059485829 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220636860 0 0) 0x7f3e200c7e70 con 0x7f3e20070c60 2024-03-04T07:34:04.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f3e1d400640 1 -- 172.21.15.113:0/2059485829 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e04005740 con 0x7f3e20070c60 2024-03-04T07:34:04.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee12a00640 1 -- 172.21.15.113:0/1289778116 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0b4070 con 0x7fee1406cb30 2024-03-04T07:34:04.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee12a00640 1 -- 172.21.15.113:0/1289778116 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c0af400 con 0x7fee1406cb30 2024-03-04T07:34:04.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee12a00640 1 -- 172.21.15.113:0/1289778116 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c0af7d0 con 0x7fee1406cb30 2024-03-04T07:34:04.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f5f50800640 1 -- 172.21.15.113:0/2020306034 shutdown_connections 2024-03-04T07:34:04.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f5f50800640 1 --2- 172.21.15.113:0/2020306034 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 0x7f5f480c58a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f5f50800640 1 --2- 172.21.15.113:0/2020306034 >> v2:172.21.15.136:3300/0 conn(0x7f5f481549b0 0x7f5f4815b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f5f50800640 1 -- 172.21.15.113:0/2020306034 >> 172.21.15.113:0/2020306034 conn(0x7f5f4806dc50 msgr2=0x7f5f48066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f5f50800640 1 -- 172.21.15.113:0/2020306034 shutdown_connections 2024-03-04T07:34:04.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f3e1e800640 1 -- 172.21.15.113:0/2059485829 >> v2:172.21.15.113:3300/0 conn(0x7f3e200c2d50 msgr2=0x7f3e200c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f3e1e800640 1 --2- 172.21.15.113:0/2059485829 >> v2:172.21.15.113:3300/0 conn(0x7f3e200c2d50 0x7f3e200c1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f3e1e800640 1 -- 172.21.15.113:0/2059485829 >> v1:172.21.15.113:6789/0 conn(0x7f3e20070c60 legacy=0x7f3e200c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee1ac00640 1 -- 172.21.15.113:0/1289778116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee1406cb30 msgr2=0x7fee140c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee1ac00640 1 --2- 172.21.15.113:0/1289778116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee1406cb30 0x7fee140c58a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009e50 tx=0x7fee0c01f1e0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f3e1e800640 1 -- 172.21.15.113:0/2059485829 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e200c8130 con 0x7f3e200c2970 2024-03-04T07:34:04.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f2205000640 1 -- 172.21.15.113:0/504470085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2200159010 msgr2=0x7f22000cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f2205000640 1 --2- 172.21.15.113:0/504470085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2200159010 0x7f22000cc030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f22000d0160 tx=0x7f21f001c470 comp rx=0 tx=0).stop 2024-03-04T07:34:04.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fd8ade00640 1 Processor -- start 2024-03-04T07:34:04.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fd8ade00640 1 -- start start 2024-03-04T07:34:04.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee1ac00640 1 -- 172.21.15.113:0/1289778116 shutdown_connections 2024-03-04T07:34:04.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee1ac00640 1 --2- 172.21.15.113:0/1289778116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee1406cb30 0x7fee140c58a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee1ac00640 1 --2- 172.21.15.113:0/1289778116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee141549b0 0x7fee1415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee1ac00640 1 -- 172.21.15.113:0/1289778116 >> 172.21.15.113:0/1289778116 conn(0x7fee1406dc50 msgr2=0x7fee14066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f017dc00640 1 Processor -- start 2024-03-04T07:34:04.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f5f50800640 1 -- 172.21.15.113:0/2020306034 wait complete. 2024-03-04T07:34:04.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7fee1ac00640 1 -- 172.21.15.113:0/1289778116 shutdown_connections 2024-03-04T07:34:04.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f2205000640 1 -- 172.21.15.113:0/504470085 shutdown_connections 2024-03-04T07:34:04.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f2205000640 1 --2- 172.21.15.113:0/504470085 >> v2:172.21.15.113:3300/0 conn(0x7f22000cb690 0x7f22000cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f2205000640 1 --2- 172.21.15.113:0/504470085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2200159010 0x7f22000cc030 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f2205000640 1 -- 172.21.15.113:0/504470085 >> 172.21.15.113:0/504470085 conn(0x7f220006d3e0 msgr2=0x7f220006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.746+0000 7f017dc00640 1 -- start start 2024-03-04T07:34:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f2205000640 1 -- 172.21.15.113:0/504470085 shutdown_connections 2024-03-04T07:34:04.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f5f50800640 1 Processor -- start 2024-03-04T07:34:04.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f2c48c00640 1 Processor -- start 2024-03-04T07:34:04.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8ade00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a815ead0 0x7fd8a80cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8ade00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a80c7c00 con 0x7fd8a8155090 2024-03-04T07:34:04.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8ade00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a80c7ee0 con 0x7fd8a80c20a0 2024-03-04T07:34:04.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8ade00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a8072fa0 con 0x7fd8a815ead0 2024-03-04T07:34:04.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f2c48c00640 1 -- start start 2024-03-04T07:34:04.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f017dc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f017806cb70 0x7f0178162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f017dc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f01781601e0 0x7f0178073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f017dc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0178071dd0 con 0x7f017806c790 2024-03-04T07:34:04.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f017dc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0178073690 con 0x7f017806cb70 2024-03-04T07:34:04.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f017dc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01780720b0 con 0x7f01781601e0 2024-03-04T07:34:04.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f5f50800640 1 -- start start 2024-03-04T07:34:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8a6000640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fd8a80c20a0 0x7fd8a80cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55096/0 (socket says 172.21.15.113:55096) 2024-03-04T07:34:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8a6000640 1 -- 172.21.15.113:0/1972842617 learned_addr learned my addr 172.21.15.113:0/1972842617 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8a7400640 1 --2- 172.21.15.113:0/1972842617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a815ead0 0x7fd8a80cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f017d200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f01781601e0 0x7f0178073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f0177e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f017806c790 0x7f017806d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55110/0 (socket says 172.21.15.113:55110) 2024-03-04T07:34:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f0177e00640 1 -- 172.21.15.113:0/4036229495 learned_addr learned my addr 172.21.15.113:0/4036229495 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f2205000640 1 -- 172.21.15.113:0/504470085 wait complete. 2024-03-04T07:34:04.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fee1ac00640 1 -- 172.21.15.113:0/1289778116 wait complete. 2024-03-04T07:34:04.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fee1ac00640 1 Processor -- start 2024-03-04T07:34:04.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f2205000640 1 Processor -- start 2024-03-04T07:34:04.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f2205000640 1 -- start start 2024-03-04T07:34:04.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f3e1e800640 1 --2- 172.21.15.113:0/2059485829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 0x7f3e2015ac60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0083f0 tx=0x7f3e0c01c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=496b5e192a315022 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f0177400640 1 --2- 172.21.15.113:0/4036229495 >> v2:172.21.15.113:3300/0 conn(0x7f017806cb70 0x7f0178162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f2c48c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406c790 0x7f2c44162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f0176a00640 1 -- 172.21.15.113:0/4036229495 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818438186 0 0) 0x7f0178071dd0 con 0x7f017806c790 2024-03-04T07:34:04.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f0176a00640 1 -- 172.21.15.113:0/4036229495 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f015c003530 con 0x7f017806c790 2024-03-04T07:34:04.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8a5600640 1 -- 172.21.15.113:0/1972842617 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390686779 0 0) 0x7fd8a80c7ee0 con 0x7fd8a80c20a0 2024-03-04T07:34:04.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8a7400640 1 -- 172.21.15.113:0/1972842617 >> v1:172.21.15.136:6789/0 conn(0x7fd8a80c20a0 legacy=0x7fd8a80cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8a7400640 1 -- 172.21.15.113:0/1972842617 >> v1:172.21.15.113:6789/0 conn(0x7fd8a8155090 legacy=0x7fd8a806b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f3e1d400640 1 -- 172.21.15.113:0/2059485829 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0b0070 con 0x7f3e200c2970 2024-03-04T07:34:04.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7fd8a7400640 1 -- 172.21.15.113:0/1972842617 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a8072950 con 0x7fd8a815ead0 2024-03-04T07:34:04.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f3e1d400640 1 -- 172.21.15.113:0/2059485829 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c008be0 con 0x7f3e200c2970 2024-03-04T07:34:04.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f3e1d400640 1 -- 172.21.15.113:0/2059485829 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00a0d0 con 0x7f3e200c2970 2024-03-04T07:34:04.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c43400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406c790 0x7f2c44162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c43400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406c790 0x7f2c44162e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47150/0 (socket says 172.21.15.113:47150) 2024-03-04T07:34:04.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c43400640 1 -- 172.21.15.113:0/893173432 learned_addr learned my addr 172.21.15.113:0/893173432 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c48c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c44071dd0 con 0x7f2c441601e0 2024-03-04T07:34:04.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f50800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 0x7f5f481f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f50800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f481549b0 0x7f5f48158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f50800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f48154d90 0x7f5f4814e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f50800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815ecf0 con 0x7f5f48154d90 2024-03-04T07:34:04.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f50800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4815ea10 con 0x7f5f481549b0 2024-03-04T07:34:04.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f50800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480c5380 con 0x7f5f4806cb30 2024-03-04T07:34:04.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f017d200640 1 -- 172.21.15.113:0/4036229495 >> v2:172.21.15.113:3300/0 conn(0x7f017806cb70 msgr2=0x7f0178162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f017d200640 1 --2- 172.21.15.113:0/4036229495 >> v2:172.21.15.113:3300/0 conn(0x7f017806cb70 0x7f0178162e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f017d200640 1 -- 172.21.15.113:0/4036229495 >> v1:172.21.15.136:6789/0 conn(0x7f017806c790 legacy=0x7f017806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.750+0000 7f017d200640 1 -- 172.21.15.113:0/4036229495 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0178072390 con 0x7f01781601e0 2024-03-04T07:34:04.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c48c00640 1 -- 172.21.15.113:0/893173432 --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c440720b0 con 0x7f2c4406cb70 2024-03-04T07:34:04.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c48c00640 1 -- 172.21.15.113:0/893173432 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c44072390 con 0x7f2c4406c790 2024-03-04T07:34:04.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f4e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f48154d90 0x7f5f4814e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2205000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22000c5430 0x7f2200142160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2205000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22000cb690 0x7f22001426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2205000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2200159010 0x7f2200148ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2205000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220016b750 con 0x7f22000c5430 2024-03-04T07:34:04.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2205000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220016b8d0 con 0x7f22000cb690 2024-03-04T07:34:04.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2205000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220016ba50 con 0x7f2200159010 2024-03-04T07:34:04.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f4e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f48154d90 0x7f5f4814e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57302/0 (socket says 172.21.15.113:57302) 2024-03-04T07:34:04.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f4e800640 1 -- 172.21.15.113:0/2618873117 learned_addr learned my addr 172.21.15.113:0/2618873117 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f4d400640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f481549b0 0x7f5f48158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c42000640 1 -- 172.21.15.113:0/893173432 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178074509 0 0) 0x7f2c440720b0 con 0x7f2c4406cb70 2024-03-04T07:34:04.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c42000640 1 -- 172.21.15.113:0/893173432 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c28003530 con 0x7f2c4406cb70 2024-03-04T07:34:04.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f3e25400640 1 -- 172.21.15.113:0/2059485829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 msgr2=0x7f3e2015ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f3e25400640 1 --2- 172.21.15.113:0/2059485829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 0x7f3e2015ac60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0083f0 tx=0x7f3e0c01c3a0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21fea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22000cb690 0x7f22001426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f3e25400640 1 -- 172.21.15.113:0/2059485829 shutdown_connections 2024-03-04T07:34:04.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f3e25400640 1 --2- 172.21.15.113:0/2059485829 >> v2:172.21.15.113:3300/0 conn(0x7f3e200c2d50 0x7f3e200c1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f3e25400640 1 --2- 172.21.15.113:0/2059485829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 0x7f3e2015ac60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f3e25400640 1 -- 172.21.15.113:0/2059485829 >> 172.21.15.113:0/2059485829 conn(0x7f3e2006e820 msgr2=0x7f3e2006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21ffe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2200159010 0x7f2200148ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21ffe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2200159010 0x7f2200148ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47160/0 (socket says 172.21.15.113:47160) 2024-03-04T07:34:04.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21ffe00640 1 -- 172.21.15.113:0/2821219919 learned_addr learned my addr 172.21.15.113:0/2821219919 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f4e800640 1 -- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 msgr2=0x7f5f481f0e10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-04T07:34:04.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f4e800640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 0x7f5f481f0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f4e800640 1 -- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f481549b0 msgr2=0x7f5f48158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f4e800640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f481549b0 0x7f5f48158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f4e800640 1 -- 172.21.15.113:0/2618873117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f481f0370 con 0x7f5f48154d90 2024-03-04T07:34:04.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c43400640 1 -- 172.21.15.113:0/893173432 >> v1:172.21.15.136:6789/0 conn(0x7f2c4406cb70 legacy=0x7f2c4406d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c43400640 1 -- 172.21.15.113:0/893173432 >> v1:172.21.15.113:6789/0 conn(0x7f2c441601e0 legacy=0x7f2c44158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21ff400640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22000c5430 0x7f2200142160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f3e25400640 1 -- 172.21.15.113:0/2059485829 shutdown_connections 2024-03-04T07:34:04.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21fea00640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22000cb690 0x7f22001426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47988/0 (socket says 172.21.15.113:47988) 2024-03-04T07:34:04.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f2c43400640 1 -- 172.21.15.113:0/893173432 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c44072670 con 0x7f2c4406c790 2024-03-04T07:34:04.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21ffe00640 1 -- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22000cb690 msgr2=0x7f22001426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21ffe00640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22000cb690 0x7f22001426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21ffe00640 1 -- 172.21.15.113:0/2821219919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22000c5430 msgr2=0x7f2200142160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21ffe00640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22000c5430 0x7f2200142160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21ffe00640 1 -- 172.21.15.113:0/2821219919 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2200146300 con 0x7f2200159010 2024-03-04T07:34:04.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21ffe00640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2200159010 0x7f2200148ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009f00 tx=0x7f21f800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f5f4e800640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f48154d90 0x7f5f4814e290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f4401d8c0 tx=0x7f5f4401bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f21f7400640 1 -- 172.21.15.113:0/2821219919 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8018030 con 0x7f2200159010 2024-03-04T07:34:04.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f21f7400640 1 -- 172.21.15.113:0/2821219919 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f800ce50 con 0x7f2200159010 2024-03-04T07:34:04.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f21f7400640 1 -- 172.21.15.113:0/2821219919 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8003bc0 con 0x7f2200159010 2024-03-04T07:34:04.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f5f42000640 1 -- 172.21.15.113:0/2618873117 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440b4040 con 0x7f5f48154d90 2024-03-04T07:34:04.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f5f42000640 1 -- 172.21.15.113:0/2618873117 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f440bf070 con 0x7f5f48154d90 2024-03-04T07:34:04.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f5f42000640 1 -- 172.21.15.113:0/2618873117 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440afde0 con 0x7f5f48154d90 2024-03-04T07:34:04.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f5f50800640 1 -- 172.21.15.113:0/2618873117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f481f4360 con 0x7f5f48154d90 2024-03-04T07:34:04.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7fee1ac00640 1 -- start start 2024-03-04T07:34:04.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7fee1ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee1406cb30 0x7fee141f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7fee1ac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee141549b0 0x7fee14148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7fee1ac00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee14154d90 0x7fee14148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee13400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee141549b0 0x7fee14148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.754+0000 7f3e25400640 1 -- 172.21.15.113:0/2059485829 wait complete. 2024-03-04T07:34:04.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f3e25400640 1 Processor -- start 2024-03-04T07:34:04.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee13400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee141549b0 0x7fee14148920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47992/0 (socket says 172.21.15.113:47992) 2024-03-04T07:34:04.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f2205000640 1 -- 172.21.15.113:0/2821219919 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2200146a10 con 0x7f2200159010 2024-03-04T07:34:04.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f2205000640 1 -- 172.21.15.113:0/2821219919 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2200144b10 con 0x7f2200159010 2024-03-04T07:34:04.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f5f50800640 1 -- 172.21.15.113:0/2618873117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f481f08f0 con 0x7f5f48154d90 2024-03-04T07:34:04.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee13e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee1406cb30 0x7fee141f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee13e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee1406cb30 0x7fee141f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57316/0 (socket says 172.21.15.113:57316) 2024-03-04T07:34:04.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fd8a7400640 1 --2- 172.21.15.113:0/1972842617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a815ead0 0x7fd8a80cd720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009dc0 tx=0x7fd8a001f140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ee2b7a83188698 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f017d200640 1 --2- 172.21.15.113:0/4036229495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01781601e0 0x7f0178073150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0178071b30 tx=0x7f016401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a80e9c840ffc2e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee1ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1415ecf0 con 0x7fee1406cb30 2024-03-04T07:34:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee1ac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1415ea10 con 0x7fee141549b0 2024-03-04T07:34:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee1ac00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee140c5380 con 0x7fee14154d90 2024-03-04T07:34:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee13400640 1 -- 172.21.15.113:0/835526395 learned_addr learned my addr 172.21.15.113:0/835526395 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f2c43400640 1 --2- 172.21.15.113:0/893173432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406c790 0x7f2c44162e80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009820 tx=0x7f2c3001eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e6bcb0e94434ed server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f2205000640 1 -- 172.21.15.113:0/2821219919 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2200075a80 con 0x7f2200159010 2024-03-04T07:34:04.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fd8a5600640 1 -- 172.21.15.113:0/1972842617 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00b4070 con 0x7fd8a815ead0 2024-03-04T07:34:04.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fd8a5600640 1 -- 172.21.15.113:0/1972842617 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a00af600 con 0x7fd8a815ead0 2024-03-04T07:34:04.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee18c00640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee14154d90 0x7fee14148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f3e25400640 1 -- start start 2024-03-04T07:34:04.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7f5f50800640 1 -- 172.21.15.113:0/2618873117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f48075c00 con 0x7f5f48154d90 2024-03-04T07:34:04.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fd8a5600640 1 -- 172.21.15.113:0/1972842617 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00b4070 con 0x7fd8a815ead0 2024-03-04T07:34:04.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee13400640 1 -- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee14154d90 msgr2=0x7fee14148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.758+0000 7fee13400640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee14154d90 0x7fee14148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fee13400640 1 -- 172.21.15.113:0/835526395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee1406cb30 msgr2=0x7fee141f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fee13400640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee1406cb30 0x7fee141f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fee13400640 1 -- 172.21.15.113:0/835526395 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee141f44d0 con 0x7fee141549b0 2024-03-04T07:34:04.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f0176a00640 1 -- 172.21.15.113:0/4036229495 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640ab640 con 0x7f01781601e0 2024-03-04T07:34:04.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fd8ade00640 1 -- 172.21.15.113:0/1972842617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a815ead0 msgr2=0x7fd8a80cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fd8ade00640 1 --2- 172.21.15.113:0/1972842617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a815ead0 0x7fd8a80cd720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009dc0 tx=0x7fd8a001f140 comp rx=0 tx=0).stop 2024-03-04T07:34:04.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f2c42000640 1 -- 172.21.15.113:0/893173432 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c300b5070 con 0x7f2c4406c790 2024-03-04T07:34:04.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f2c42000640 1 -- 172.21.15.113:0/893173432 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c300a8650 con 0x7f2c4406c790 2024-03-04T07:34:04.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fd8ade00640 1 -- 172.21.15.113:0/1972842617 shutdown_connections 2024-03-04T07:34:04.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fd8ade00640 1 --2- 172.21.15.113:0/1972842617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a815ead0 0x7fd8a80cd720 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fd8ade00640 1 -- 172.21.15.113:0/1972842617 >> 172.21.15.113:0/1972842617 conn(0x7fd8a80671e0 msgr2=0x7fd8a806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f0176a00640 1 -- 172.21.15.113:0/4036229495 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01640b0070 con 0x7f01781601e0 2024-03-04T07:34:04.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f0176a00640 1 -- 172.21.15.113:0/4036229495 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01640aba00 con 0x7f01781601e0 2024-03-04T07:34:04.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f2c42000640 1 -- 172.21.15.113:0/893173432 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34001680 con 0x7f2c4406c790 2024-03-04T07:34:04.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fee13400640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee141549b0 0x7fee14148920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c450 tx=0x7fee0000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f5f42000640 1 -- 172.21.15.113:0/2618873117 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f440a8d10 con 0x7f5f48154d90 2024-03-04T07:34:04.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fd8ade00640 1 -- 172.21.15.113:0/1972842617 shutdown_connections 2024-03-04T07:34:04.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e25400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e20070c60 0x7f3e20155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e25400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 0x7f3e200b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fd8ade00640 1 -- 172.21.15.113:0/1972842617 wait complete. 2024-03-04T07:34:04.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e25400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e200c2d50 0x7f3e200c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e25400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20157b10 con 0x7f3e20070c60 2024-03-04T07:34:04.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e25400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20156ce0 con 0x7f3e200c2d50 2024-03-04T07:34:04.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fee0be00640 1 -- 172.21.15.113:0/835526395 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00012830 con 0x7fee141549b0 2024-03-04T07:34:04.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e25400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e20157c90 con 0x7f3e200c2970 2024-03-04T07:34:04.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fee1ac00640 1 -- 172.21.15.113:0/835526395 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee14150560 con 0x7fee141549b0 2024-03-04T07:34:04.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fee1ac00640 1 -- 172.21.15.113:0/835526395 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee14150b00 con 0x7fee141549b0 2024-03-04T07:34:04.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fee0be00640 1 -- 172.21.15.113:0/835526395 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee00017070 con 0x7fee141549b0 2024-03-04T07:34:04.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f2c48c00640 1 -- 172.21.15.113:0/893173432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406c790 msgr2=0x7f2c44162e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f2c48c00640 1 --2- 172.21.15.113:0/893173432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406c790 0x7f2c44162e80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009820 tx=0x7f2c3001eb50 comp rx=0 tx=0).stop 2024-03-04T07:34:04.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e200c2d50 0x7f3e200c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e200c2d50 0x7f3e200c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48002/0 (socket says 172.21.15.113:48002) 2024-03-04T07:34:04.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1f200640 1 -- 172.21.15.113:0/3911849527 learned_addr learned my addr 172.21.15.113:0/3911849527 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1e800640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e20070c60 0x7f3e20155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fee0be00640 1 -- 172.21.15.113:0/835526395 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00012830 con 0x7fee141549b0 2024-03-04T07:34:04.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f5f42000640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f2c066fb0 0x7f5f2c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f2c48c00640 1 -- 172.21.15.113:0/893173432 shutdown_connections 2024-03-04T07:34:04.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1de00640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 0x7f3e200b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f2c48c00640 1 --2- 172.21.15.113:0/893173432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406c790 0x7f2c44162e80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f2c48c00640 1 -- 172.21.15.113:0/893173432 >> 172.21.15.113:0/893173432 conn(0x7f2c4406e2c0 msgr2=0x7f2c4406e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1f200640 1 -- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 msgr2=0x7f3e200b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1f200640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 0x7f3e200b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f5f42000640 1 -- 172.21.15.113:0/2618873117 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f44002020 con 0x7f5f48154d90 2024-03-04T07:34:04.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f2c48c00640 1 -- 172.21.15.113:0/893173432 shutdown_connections 2024-03-04T07:34:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1f200640 1 -- 172.21.15.113:0/3911849527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e20070c60 msgr2=0x7f3e20155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1f200640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e20070c60 0x7f3e20155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1f200640 1 -- 172.21.15.113:0/3911849527 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e200adfe0 con 0x7f3e200c2d50 2024-03-04T07:34:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f5f4de00640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f2c066fb0 0x7f5f2c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f2c48c00640 1 -- 172.21.15.113:0/893173432 wait complete. 2024-03-04T07:34:04.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fd8ade00640 1 Processor -- start 2024-03-04T07:34:04.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1e800640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e20070c60 0x7f3e20155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:34:04.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fee1ac00640 1 -- 172.21.15.113:0/835526395 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedec0052a0 con 0x7fee141549b0 2024-03-04T07:34:04.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f21f7400640 1 -- 172.21.15.113:0/2821219919 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21f801b070 con 0x7f2200159010 2024-03-04T07:34:04.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7fd8ade00640 1 -- start start 2024-03-04T07:34:04.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.762+0000 7f3e1f200640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e200c2d50 0x7f3e200c5b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c3c0 tx=0x7f3e10001e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f5f4de00640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f2c066fb0 0x7f5f2c069470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f380099f0 tx=0x7f5f38009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f017dc00640 1 -- 172.21.15.113:0/4036229495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01781601e0 msgr2=0x7f0178073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f017dc00640 1 --2- 172.21.15.113:0/4036229495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01781601e0 0x7f0178073150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0178071b30 tx=0x7f016401c440 comp rx=0 tx=0).stop 2024-03-04T07:34:04.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f3e16a00640 1 -- 172.21.15.113:0/3911849527 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10017070 con 0x7f3e200c2d50 2024-03-04T07:34:04.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f017dc00640 1 -- 172.21.15.113:0/4036229495 shutdown_connections 2024-03-04T07:34:04.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f017dc00640 1 --2- 172.21.15.113:0/4036229495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01781601e0 0x7f0178073150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f017dc00640 1 --2- 172.21.15.113:0/4036229495 >> v2:172.21.15.113:3300/0 conn(0x7f017806cb70 0x7f0178162e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f017dc00640 1 -- 172.21.15.113:0/4036229495 >> 172.21.15.113:0/4036229495 conn(0x7f017806e2c0 msgr2=0x7f017806e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8ade00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a80c20a0 0x7fd8a815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8ade00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a8155090 0x7fd8a80b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f3e16a00640 1 -- 172.21.15.113:0/3911849527 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e100052d0 con 0x7f3e200c2d50 2024-03-04T07:34:04.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f3e16a00640 1 -- 172.21.15.113:0/3911849527 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10003840 con 0x7f3e200c2d50 2024-03-04T07:34:04.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f3e25400640 1 -- 172.21.15.113:0/3911849527 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e200ae2a0 con 0x7f3e200c2d50 2024-03-04T07:34:04.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f3e25400640 1 -- 172.21.15.113:0/3911849527 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e200b0f20 con 0x7f3e200c2d50 2024-03-04T07:34:04.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f21f7400640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21e0067030 0x7f21e00694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f21f7400640 1 -- 172.21.15.113:0/2821219919 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21f80860c0 con 0x7f2200159010 2024-03-04T07:34:04.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f21ff400640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21e0067030 0x7f21e00694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a80c20a0 0x7fd8a815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8ade00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8a815ead0 0x7fd8a80c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8ade00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80cdc60 con 0x7fd8a80c20a0 2024-03-04T07:34:04.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8ade00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80734d0 con 0x7fd8a815ead0 2024-03-04T07:34:04.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8ade00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80cdde0 con 0x7fd8a8155090 2024-03-04T07:34:04.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f2c48c00640 1 Processor -- start 2024-03-04T07:34:04.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f2c48c00640 1 -- start start 2024-03-04T07:34:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a80c20a0 0x7fd8a815ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57336/0 (socket says 172.21.15.113:57336) 2024-03-04T07:34:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8a815ead0 0x7fd8a80c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a6000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a8155090 0x7fd8a80b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a6a00640 1 -- 172.21.15.113:0/2129362258 learned_addr learned my addr 172.21.15.113:0/2129362258 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a6a00640 1 -- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a8155090 msgr2=0x7fd8a80b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f3e25400640 1 -- 172.21.15.113:0/3911849527 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df80052a0 con 0x7f3e200c2d50 2024-03-04T07:34:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f21ff400640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21e0067030 0x7f21e00694f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f21f00af000 tx=0x7f21f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a6a00640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a8155090 0x7fd8a80b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a6a00640 1 -- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8a815ead0 msgr2=0x7fd8a80c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a6a00640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8a815ead0 0x7fd8a80c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a6a00640 1 -- 172.21.15.113:0/2129362258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a81537b0 con 0x7fd8a80c20a0 2024-03-04T07:34:04.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f017dc00640 1 -- 172.21.15.113:0/4036229495 shutdown_connections 2024-03-04T07:34:04.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a6a00640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a80c20a0 0x7fd8a815ca50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd89800c490 tx=0x7fd898001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7f017dc00640 1 -- 172.21.15.113:0/4036229495 wait complete. 2024-03-04T07:34:04.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fd8a7400640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8a815ead0 0x7fd8a80c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:34:04.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.766+0000 7fee0be00640 1 -- 172.21.15.113:0/835526395 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee000129d0 con 0x7fee141549b0 2024-03-04T07:34:04.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7fee0be00640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedf4067390 0x7fedf4069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7fee0be00640 1 -- 172.21.15.113:0/835526395 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee00012630 con 0x7fee141549b0 2024-03-04T07:34:04.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c48c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c4406c790 0x7f2c441fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c48c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406cb70 0x7f2c440c3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c48c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c441601e0 0x7f2c44161f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c43400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c4406c790 0x7f2c441fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c43400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c4406c790 0x7f2c441fcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57350/0 (socket says 172.21.15.113:57350) 2024-03-04T07:34:04.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7fee13e00640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedf4067390 0x7fedf4069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c42a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406cb70 0x7f2c440c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c48c00640 1 -- 172.21.15.113:0/3369216327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4406dbb0 con 0x7f2c4406c790 2024-03-04T07:34:04.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c43400640 1 -- 172.21.15.113:0/3369216327 learned_addr learned my addr 172.21.15.113:0/3369216327 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c48c00640 1 -- 172.21.15.113:0/3369216327 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c44072c30 con 0x7f2c441601e0 2024-03-04T07:34:04.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c48c00640 1 -- 172.21.15.113:0/3369216327 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c44159280 con 0x7f2c4406cb70 2024-03-04T07:34:04.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c43e00640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c441601e0 0x7f2c44161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c43400640 1 -- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406cb70 msgr2=0x7f2c440c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c43400640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406cb70 0x7f2c440c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c43400640 1 -- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c441601e0 msgr2=0x7f2c44161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c43400640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c441601e0 0x7f2c44161f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c43400640 1 -- 172.21.15.113:0/3369216327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c440c7300 con 0x7f2c4406c790 2024-03-04T07:34:04.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7fee13e00640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedf4067390 0x7fedf4069850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee0400b9d0 tx=0x7fee04008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c43400640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c4406c790 0x7f2c441fcf80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c3001c780 tx=0x7f2c3001b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f017dc00640 1 Processor -- start 2024-03-04T07:34:04.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7fd89f400640 1 -- 172.21.15.113:0/2129362258 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898017070 con 0x7fd8a80c20a0 2024-03-04T07:34:04.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7fd89f400640 1 -- 172.21.15.113:0/2129362258 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd898003600 con 0x7fd8a80c20a0 2024-03-04T07:34:04.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7fd89f400640 1 -- 172.21.15.113:0/2129362258 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8980039f0 con 0x7fd8a80c20a0 2024-03-04T07:34:04.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7fd8ade00640 1 -- 172.21.15.113:0/2129362258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a81f0990 con 0x7fd8a80c20a0 2024-03-04T07:34:04.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f017dc00640 1 -- start start 2024-03-04T07:34:04.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c3b400640 1 -- 172.21.15.113:0/3369216327 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c300c0070 con 0x7f2c4406c790 2024-03-04T07:34:04.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f5f42000640 1 -- 172.21.15.113:0/2618873117 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5f44018d90 con 0x7f5f48154d90 2024-03-04T07:34:04.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c3b400640 1 -- 172.21.15.113:0/3369216327 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c300b6040 con 0x7f2c4406c790 2024-03-04T07:34:04.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7fd8ade00640 1 -- 172.21.15.113:0/2129362258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a81f0f50 con 0x7fd8a80c20a0 2024-03-04T07:34:04.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c48c00640 1 -- 172.21.15.113:0/3369216327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c440c75e0 con 0x7f2c4406c790 2024-03-04T07:34:04.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c48c00640 1 -- 172.21.15.113:0/3369216327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c440c7ba0 con 0x7f2c4406c790 2024-03-04T07:34:04.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.770+0000 7f2c3b400640 1 -- 172.21.15.113:0/3369216327 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c300b5070 con 0x7f2c4406c790 2024-03-04T07:34:04.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7fd8ade00640 1 -- 172.21.15.113:0/2129362258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a8075c00 con 0x7fd8a80c20a0 2024-03-04T07:34:04.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f017dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806c790 0x7f0178151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f017dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f017806cb70 0x7f017814f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f2c48c00640 1 -- 172.21.15.113:0/3369216327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c44075a80 con 0x7f2c4406c790 2024-03-04T07:34:04.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f017dc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01781601e0 0x7f017814fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f017dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017816b680 con 0x7f017806c790 2024-03-04T07:34:04.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f017dc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017806dbb0 con 0x7f017806cb70 2024-03-04T07:34:04.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f017dc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017816b800 con 0x7f01781601e0 2024-03-04T07:34:04.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f0177400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f017806cb70 0x7f017814f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f0177e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806c790 0x7f0178151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7fd89f400640 1 -- 172.21.15.113:0/2129362258 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd89801a070 con 0x7fd8a80c20a0 2024-03-04T07:34:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f0177e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806c790 0x7f0178151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57352/0 (socket says 172.21.15.113:57352) 2024-03-04T07:34:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f0177e00640 1 -- 172.21.15.113:0/2003357649 learned_addr learned my addr 172.21.15.113:0/2003357649 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f017d200640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01781601e0 0x7f017814fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f0177e00640 1 -- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01781601e0 msgr2=0x7f017814fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f0177e00640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01781601e0 0x7f017814fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f0177e00640 1 -- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f017806cb70 msgr2=0x7f017814f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f0177e00640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f017806cb70 0x7f017814f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f0177e00640 1 -- 172.21.15.113:0/2003357649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01780c84a0 con 0x7f017806c790 2024-03-04T07:34:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f017d200640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01781601e0 0x7f017814fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:04.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f0177e00640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806c790 0x7f0178151c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f017000c8d0 tx=0x7f017000cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7f2c3b400640 1 -- 172.21.15.113:0/3369216327 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c300b0a90 con 0x7f2c4406c790 2024-03-04T07:34:04.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7fd89f400640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd888067030 0x7fd8880694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7fd89f400640 1 -- 172.21.15.113:0/2129362258 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8980842b0 con 0x7fd8a80c20a0 2024-03-04T07:34:04.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.774+0000 7fd8a6000640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd888067030 0x7fd8880694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.778+0000 7f2c3b400640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c24067060 0x7f2c24069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.778+0000 7f2c3b400640 1 -- 172.21.15.113:0/3369216327 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c30121ac0 con 0x7f2c4406c790 2024-03-04T07:34:04.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.778+0000 7f21f7400640 1 -- 172.21.15.113:0/2821219919 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f21f8010040 con 0x7f2200159010 2024-03-04T07:34:04.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.778+0000 7f2c42a00640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c24067060 0x7f2c24069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.778+0000 7fd8a6000640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd888067030 0x7fd8880694f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd8a81546e0 tx=0x7fd894007bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.778+0000 7f2c42a00640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c24067060 0x7f2c24069520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c441567a0 tx=0x7f2c34009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.778+0000 7f016fe00640 1 -- 172.21.15.113:0/2003357649 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170018070 con 0x7f017806c790 2024-03-04T07:34:04.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.782+0000 7f017dc00640 1 -- 172.21.15.113:0/2003357649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01780c8740 con 0x7f017806c790 2024-03-04T07:34:04.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.782+0000 7f017dc00640 1 -- 172.21.15.113:0/2003357649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017814cab0 con 0x7f017806c790 2024-03-04T07:34:04.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.782+0000 7fd89f400640 1 -- 172.21.15.113:0/2129362258 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd898054ce0 con 0x7fd8a80c20a0 2024-03-04T07:34:04.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.782+0000 7fee0be00640 1 -- 172.21.15.113:0/835526395 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fee00056240 con 0x7fee141549b0 2024-03-04T07:34:04.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.782+0000 7f017dc00640 1 -- 172.21.15.113:0/2003357649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0178075a80 con 0x7f017806c790 2024-03-04T07:34:04.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.782+0000 7f3e16a00640 1 -- 172.21.15.113:0/3911849527 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e1001a070 con 0x7f3e200c2d50 2024-03-04T07:34:04.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.786+0000 7f3e16a00640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e00066f60 0x7f3e00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.786+0000 7f3e16a00640 1 -- 172.21.15.113:0/3911849527 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e100846f0 con 0x7f3e200c2d50 2024-03-04T07:34:04.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.786+0000 7f3e1e800640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e00066f60 0x7f3e00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.786+0000 7f3e1e800640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e00066f60 0x7f3e00069420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0ad000 tx=0x7f3e0c004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.786+0000 7f016fe00640 1 -- 172.21.15.113:0/2003357649 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017000a640 con 0x7f017806c790 2024-03-04T07:34:04.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.786+0000 7f016fe00640 1 -- 172.21.15.113:0/2003357649 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170022060 con 0x7f017806c790 2024-03-04T07:34:04.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.790+0000 7f2c3b400640 1 -- 172.21.15.113:0/3369216327 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2c30018770 con 0x7f2c4406c790 2024-03-04T07:34:04.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.790+0000 7f016fe00640 1 -- 172.21.15.113:0/2003357649 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f017001b070 con 0x7f017806c790 2024-03-04T07:34:04.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.794+0000 7f016fe00640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0158066f90 0x7f0158069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.794+0000 7f016fe00640 1 -- 172.21.15.113:0/2003357649 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01700849d0 con 0x7f017806c790 2024-03-04T07:34:04.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.794+0000 7f3e16a00640 1 -- 172.21.15.113:0/3911849527 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3e10021020 con 0x7f3e200c2d50 2024-03-04T07:34:04.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.794+0000 7f0177400640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0158066f90 0x7f0158069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.798+0000 7f016fe00640 1 -- 172.21.15.113:0/2003357649 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0170055400 con 0x7f017806c790 2024-03-04T07:34:04.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.802+0000 7f0177400640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0158066f90 0x7f0158069450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0178155150 tx=0x7f0168008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.802+0000 7f5ccc200640 1 Processor -- start 2024-03-04T07:34:04.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5ccc200640 1 -- start start 2024-03-04T07:34:04.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5ccc200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4158860 0x7f5cc415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5ccc200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc4158c60 con 0x7f5cc4152b70 2024-03-04T07:34:04.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5ccc200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc4152f50 con 0x7f5cc40c6e20 2024-03-04T07:34:04.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5ccc200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc415c260 con 0x7f5cc4158860 2024-03-04T07:34:04.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5cc9800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5cc4152b70 0x7f5cc40c82e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49594/0 (socket says 172.21.15.113:49594) 2024-03-04T07:34:04.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5cc9800640 1 -- 172.21.15.113:0/915198365 learned_addr learned my addr 172.21.15.113:0/915198365 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:04.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5cc8e00640 1 --2- 172.21.15.113:0/915198365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4158860 0x7f5cc415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5cc8e00640 1 -- 172.21.15.113:0/915198365 >> v1:172.21.15.136:6789/0 conn(0x7f5cc40c6e20 legacy=0x7f5cc41631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:04.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5cc8e00640 1 -- 172.21.15.113:0/915198365 >> v1:172.21.15.113:6789/0 conn(0x7f5cc4152b70 legacy=0x7f5cc40c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5cc8e00640 1 -- 172.21.15.113:0/915198365 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc4158ef0 con 0x7f5cc4158860 2024-03-04T07:34:04.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.806+0000 7f5cc3e00640 1 -- 172.21.15.113:0/915198365 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488093243 0 0) 0x7f5cc4158c60 con 0x7f5cc4152b70 2024-03-04T07:34:04.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5cc8e00640 1 --2- 172.21.15.113:0/915198365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4158860 0x7f5cc415c6f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80097f0 tx=0x7f5cb801ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be495a7a8a9afd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5cc3e00640 1 -- 172.21.15.113:0/915198365 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80b5070 con 0x7f5cc4158860 2024-03-04T07:34:04.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5cc3e00640 1 -- 172.21.15.113:0/915198365 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb80a8990 con 0x7f5cc4158860 2024-03-04T07:34:04.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5cc3e00640 1 -- 172.21.15.113:0/915198365 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80a8d60 con 0x7f5cc4158860 2024-03-04T07:34:04.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5ccc200640 1 -- 172.21.15.113:0/915198365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4158860 msgr2=0x7f5cc415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5ccc200640 1 --2- 172.21.15.113:0/915198365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4158860 0x7f5cc415c6f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5cb80097f0 tx=0x7f5cb801ec90 comp rx=0 tx=0).stop 2024-03-04T07:34:04.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5ccc200640 1 -- 172.21.15.113:0/915198365 shutdown_connections 2024-03-04T07:34:04.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5ccc200640 1 --2- 172.21.15.113:0/915198365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4158860 0x7f5cc415c6f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5ccc200640 1 -- 172.21.15.113:0/915198365 >> 172.21.15.113:0/915198365 conn(0x7f5cc406e820 msgr2=0x7f5cc406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:04.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5ccc200640 1 -- 172.21.15.113:0/915198365 shutdown_connections 2024-03-04T07:34:04.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.818+0000 7f5ccc200640 1 -- 172.21.15.113:0/915198365 wait complete. 2024-03-04T07:34:04.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.822+0000 7f5ccc200640 1 Processor -- start 2024-03-04T07:34:04.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.822+0000 7f5ccc200640 1 -- start start 2024-03-04T07:34:04.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5ccc200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc40c6e20 0x7f5cc40c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5ccc200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4152b70 0x7f5cc40cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5ccc200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc4158860 0x7f5cc40b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cc8e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4152b70 0x7f5cc40cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cc8e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4152b70 0x7f5cc40cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47206/0 (socket says 172.21.15.113:47206) 2024-03-04T07:34:04.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cca200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc4158860 0x7f5cc40b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cc9800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc40c6e20 0x7f5cc40c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cc9800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc40c6e20 0x7f5cc40c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48020/0 (socket says 172.21.15.113:48020) 2024-03-04T07:34:04.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cca200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc4158860 0x7f5cc40b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57354/0 (socket says 172.21.15.113:57354) 2024-03-04T07:34:04.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cc8e00640 1 -- 172.21.15.113:0/1889180261 learned_addr learned my addr 172.21.15.113:0/1889180261 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:04.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4159790 con 0x7f5cc4158860 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc41594b0 con 0x7f5cc40c6e20 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc415bef0 con 0x7f5cc4152b70 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cca200640 1 -- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4152b70 msgr2=0x7f5cc40cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cca200640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4152b70 0x7f5cc40cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cca200640 1 -- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc40c6e20 msgr2=0x7f5cc40c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cca200640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc40c6e20 0x7f5cc40c4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cca200640 1 -- 172.21.15.113:0/1889180261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc40724f0 con 0x7f5cc4158860 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.826+0000 7f5cca200640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc4158860 0x7f5cc40b0b90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00be30 tx=0x7f5cbc00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.830+0000 7f5cc1600640 1 -- 172.21.15.113:0/1889180261 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc019070 con 0x7f5cc4158860 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.830+0000 7f5cc1600640 1 -- 172.21.15.113:0/1889180261 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc014440 con 0x7f5cc4158860 2024-03-04T07:34:04.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.830+0000 7f5cc1600640 1 -- 172.21.15.113:0/1889180261 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc014810 con 0x7f5cc4158860 2024-03-04T07:34:04.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.830+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc40ccbc0 con 0x7f5cc4158860 2024-03-04T07:34:04.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.830+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc41ee960 con 0x7f5cc4158860 2024-03-04T07:34:04.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.830+0000 7f5cc1600640 1 -- 172.21.15.113:0/1889180261 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cbc01c070 con 0x7f5cc4158860 2024-03-04T07:34:04.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.834+0000 7f5cc1600640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ca4066fb0 0x7f5ca4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:04.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.834+0000 7f5cc1600640 1 -- 172.21.15.113:0/1889180261 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cbc053990 con 0x7f5cc4158860 2024-03-04T07:34:04.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.834+0000 7f5cc9800640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ca4066fb0 0x7f5ca4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:04.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.834+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc4075c00 con 0x7f5cc4158860 2024-03-04T07:34:04.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.834+0000 7f5cc9800640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ca4066fb0 0x7f5ca4069470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0000f70 tx=0x7f5cb0007240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:04.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:04.842+0000 7f5cc1600640 1 -- 172.21.15.113:0/1889180261 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5cbc0558f0 con 0x7f5cc4158860 2024-03-04T07:34:05.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.102+0000 7f5f50800640 1 -- 172.21.15.113:0/2618873117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5f4814e7d0 con 0x7f5f48154d90 2024-03-04T07:34:05.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.106+0000 7f5f42000640 1 -- 172.21.15.113:0/2618873117 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f440f3830 con 0x7f5f48154d90 2024-03-04T07:34:05.108 INFO:teuthology.orchestra.run.smithi113.stdout:60129542151 2024-03-04T07:34:05.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 -- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f2c066fb0 msgr2=0x7f5f2c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f2c066fb0 0x7f5f2c069470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f380099f0 tx=0x7f5f38009190 comp rx=0 tx=0).stop 2024-03-04T07:34:05.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 -- 172.21.15.113:0/2618873117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f48154d90 msgr2=0x7f5f4814e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f48154d90 0x7f5f4814e290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5f4401d8c0 tx=0x7f5f4401bfb0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7fee1ac00640 1 -- 172.21.15.113:0/835526395 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fedec005860 con 0x7fee141549b0 2024-03-04T07:34:05.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f2205000640 1 -- 172.21.15.113:0/2821219919 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f22001621b0 con 0x7f2200159010 2024-03-04T07:34:05.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 -- 172.21.15.113:0/2618873117 shutdown_connections 2024-03-04T07:34:05.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f2c066fb0 0x7f5f2c069470 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f48154d90 0x7f5f4814e290 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f481549b0 0x7f5f48158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 --2- 172.21.15.113:0/2618873117 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f4806cb30 0x7f5f481f0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 -- 172.21.15.113:0/2618873117 >> 172.21.15.113:0/2618873117 conn(0x7f5f4806dc50 msgr2=0x7f5f480c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:05.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 -- 172.21.15.113:0/2618873117 shutdown_connections 2024-03-04T07:34:05.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f5f50800640 1 -- 172.21.15.113:0/2618873117 wait complete. 2024-03-04T07:34:05.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7f21f7400640 1 -- 172.21.15.113:0/2821219919 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f21f80570b0 con 0x7f2200159010 2024-03-04T07:34:05.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.110+0000 7fee0be00640 1 -- 172.21.15.113:0/835526395 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fee00055b40 con 0x7fee141549b0 2024-03-04T07:34:05.114 INFO:teuthology.orchestra.run.smithi113.stdout:64424509447 2024-03-04T07:34:05.115 INFO:teuthology.orchestra.run.smithi113.stdout:60129542150 2024-03-04T07:34:05.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 -- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedf4067390 msgr2=0x7fedf4069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedf4067390 0x7fedf4069850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fee0400b9d0 tx=0x7fee04008040 comp rx=0 tx=0).stop 2024-03-04T07:34:05.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 -- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee141549b0 msgr2=0x7fee14148920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee141549b0 0x7fee14148920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c450 tx=0x7fee0000c910 comp rx=0 tx=0).stop 2024-03-04T07:34:05.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 -- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21e0067030 msgr2=0x7f21e00694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21e0067030 0x7f21e00694f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f21f00af000 tx=0x7f21f0008040 comp rx=0 tx=0).stop 2024-03-04T07:34:05.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 -- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2200159010 msgr2=0x7f2200148ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2200159010 0x7f2200148ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009f00 tx=0x7f21f800c490 comp rx=0 tx=0).stop 2024-03-04T07:34:05.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 -- 172.21.15.113:0/2821219919 shutdown_connections 2024-03-04T07:34:05.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2200159010 0x7f2200148ab0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22000cb690 0x7f22001426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21e0067030 0x7f21e00694f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 --2- 172.21.15.113:0/2821219919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22000c5430 0x7f2200142160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 -- 172.21.15.113:0/2821219919 >> 172.21.15.113:0/2821219919 conn(0x7f220006d3e0 msgr2=0x7f2200157d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:05.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 -- 172.21.15.113:0/835526395 shutdown_connections 2024-03-04T07:34:05.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee14154d90 0x7fee14148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee141549b0 0x7fee14148920 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedf4067390 0x7fedf4069850 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 --2- 172.21.15.113:0/835526395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee1406cb30 0x7fee141f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 -- 172.21.15.113:0/835526395 >> 172.21.15.113:0/835526395 conn(0x7fee1406dc50 msgr2=0x7fee14159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:05.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 -- 172.21.15.113:0/835526395 shutdown_connections 2024-03-04T07:34:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 -- 172.21.15.113:0/2821219919 shutdown_connections 2024-03-04T07:34:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7f2205000640 1 -- 172.21.15.113:0/2821219919 wait complete. 2024-03-04T07:34:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.114+0000 7fee1ac00640 1 -- 172.21.15.113:0/835526395 wait complete. 2024-03-04T07:34:05.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.122+0000 7fd8ade00640 1 -- 172.21.15.113:0/2129362258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd8a8073240 con 0x7fd8a80c20a0 2024-03-04T07:34:05.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.122+0000 7fd89f400640 1 -- 172.21.15.113:0/2129362258 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd898058d80 con 0x7fd8a80c20a0 2024-03-04T07:34:05.128 INFO:teuthology.orchestra.run.smithi113.stdout:64424509447 2024-03-04T07:34:05.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 -- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd888067030 msgr2=0x7fd8880694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd888067030 0x7fd8880694f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd8a81546e0 tx=0x7fd894007bc0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 -- 172.21.15.113:0/2129362258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a80c20a0 msgr2=0x7fd8a815ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a80c20a0 0x7fd8a815ca50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd89800c490 tx=0x7fd898001e80 comp rx=0 tx=0).stop 2024-03-04T07:34:05.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7f2c48c00640 1 -- 172.21.15.113:0/3369216327 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2c441533e0 con 0x7f2c4406c790 2024-03-04T07:34:05.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 -- 172.21.15.113:0/2129362258 shutdown_connections 2024-03-04T07:34:05.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8a815ead0 0x7fd8a80c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd8a8155090 0x7fd8a80b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd888067030 0x7fd8880694f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 --2- 172.21.15.113:0/2129362258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a80c20a0 0x7fd8a815ca50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 -- 172.21.15.113:0/2129362258 >> 172.21.15.113:0/2129362258 conn(0x7fd8a80671e0 msgr2=0x7fd8a80c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:05.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 -- 172.21.15.113:0/2129362258 shutdown_connections 2024-03-04T07:34:05.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.126+0000 7fd89cc00640 1 -- 172.21.15.113:0/2129362258 wait complete. 2024-03-04T07:34:05.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.130+0000 7f2c3b400640 1 -- 172.21.15.113:0/3369216327 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c300f2460 con 0x7f2c4406c790 2024-03-04T07:34:05.135 INFO:teuthology.orchestra.run.smithi113.stdout:60129542150 2024-03-04T07:34:05.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.138+0000 7f2c38c00640 1 -- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c24067060 msgr2=0x7f2c24069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.138+0000 7f2c38c00640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c24067060 0x7f2c24069520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c441567a0 tx=0x7f2c34009040 comp rx=0 tx=0).stop 2024-03-04T07:34:05.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.138+0000 7f2c38c00640 1 -- 172.21.15.113:0/3369216327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c4406c790 msgr2=0x7f2c441fcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.138+0000 7f2c38c00640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c4406c790 0x7f2c441fcf80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2c3001c780 tx=0x7f2c3001b8f0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.138+0000 7f2c38c00640 1 -- 172.21.15.113:0/3369216327 shutdown_connections 2024-03-04T07:34:05.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.138+0000 7f2c38c00640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c441601e0 0x7f2c44161f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.138+0000 7f2c38c00640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2c4406cb70 0x7f2c440c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.138+0000 7f2c38c00640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2c24067060 0x7f2c24069520 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c441567a0 tx=0x7f2c34009040 comp rx=0 tx=0).stop 2024-03-04T07:34:05.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.138+0000 7f2c38c00640 1 --2- 172.21.15.113:0/3369216327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c4406c790 0x7f2c441fcf80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.138+0000 7f2c38c00640 1 -- 172.21.15.113:0/3369216327 >> 172.21.15.113:0/3369216327 conn(0x7f2c4406e2c0 msgr2=0x7f2c44153900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:05.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.142+0000 7f017dc00640 1 -- 172.21.15.113:0/2003357649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0178155180 con 0x7f017806c790 2024-03-04T07:34:05.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.142+0000 7f3e25400640 1 -- 172.21.15.113:0/3911849527 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3df8005860 con 0x7f3e200c2d50 2024-03-04T07:34:05.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.2 2024-03-04T07:34:05.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.142+0000 7f2c38c00640 1 -- 172.21.15.113:0/3369216327 shutdown_connections 2024-03-04T07:34:05.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.142+0000 7f2c38c00640 1 -- 172.21.15.113:0/3369216327 wait complete. 2024-03-04T07:34:05.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.142+0000 7f016fe00640 1 -- 172.21.15.113:0/2003357649 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01700594a0 con 0x7f017806c790 2024-03-04T07:34:05.149 INFO:teuthology.orchestra.run.smithi113.stdout:60129542151 2024-03-04T07:34:05.150 INFO:teuthology.orchestra.run.smithi113.stdout:60129542151 2024-03-04T07:34:05.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.142+0000 7f3e16a00640 1 -- 172.21.15.113:0/3911849527 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e10005480 con 0x7f3e200c2d50 2024-03-04T07:34:05.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 -- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0158066f90 msgr2=0x7f0158069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0158066f90 0x7f0158069450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0178155150 tx=0x7f0168008040 comp rx=0 tx=0).stop 2024-03-04T07:34:05.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 -- 172.21.15.113:0/2003357649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806c790 msgr2=0x7f0178151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806c790 0x7f0178151c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f017000c8d0 tx=0x7f017000cd90 comp rx=0 tx=0).stop 2024-03-04T07:34:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 -- 172.21.15.113:0/2003357649 shutdown_connections 2024-03-04T07:34:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01781601e0 0x7f017814fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f017806cb70 0x7f017814f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0158066f90 0x7f0158069450 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 --2- 172.21.15.113:0/2003357649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f017806c790 0x7f0178151c10 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f017000c8d0 tx=0x7f017000cd90 comp rx=0 tx=0).stop 2024-03-04T07:34:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 -- 172.21.15.113:0/2003357649 >> 172.21.15.113:0/2003357649 conn(0x7f017806e2c0 msgr2=0x7f0178157b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 -- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e00066f60 msgr2=0x7f3e00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e00066f60 0x7f3e00069420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0ad000 tx=0x7f3e0c004740 comp rx=0 tx=0).stop 2024-03-04T07:34:05.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 -- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e200c2d50 msgr2=0x7f3e200c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e200c2d50 0x7f3e200c5b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000c3c0 tx=0x7f3e10001e80 comp rx=0 tx=0).stop 2024-03-04T07:34:05.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 -- 172.21.15.113:0/3911849527 shutdown_connections 2024-03-04T07:34:05.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e200c2d50 0x7f3e200c5b80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e200c2970 0x7f3e200b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e00066f60 0x7f3e00069420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 --2- 172.21.15.113:0/3911849527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e20070c60 0x7f3e20155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 -- 172.21.15.113:0/3911849527 >> 172.21.15.113:0/3911849527 conn(0x7f3e2006e820 msgr2=0x7f3e200ca550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:05.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 -- 172.21.15.113:0/2003357649 shutdown_connections 2024-03-04T07:34:05.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f017dc00640 1 -- 172.21.15.113:0/2003357649 wait complete. 2024-03-04T07:34:05.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 -- 172.21.15.113:0/3911849527 shutdown_connections 2024-03-04T07:34:05.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.146+0000 7f3e25400640 1 -- 172.21.15.113:0/3911849527 wait complete. 2024-03-04T07:34:05.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.1 2024-03-04T07:34:05.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.0 2024-03-04T07:34:05.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.3 2024-03-04T07:34:05.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.166+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5cc40c79d0 con 0x7f5cc4158860 2024-03-04T07:34:05.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.166+0000 7f5cc1600640 1 -- 172.21.15.113:0/1889180261 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5cbc059990 con 0x7f5cc4158860 2024-03-04T07:34:05.171 INFO:teuthology.orchestra.run.smithi113.stdout:60129542150 2024-03-04T07:34:05.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.4 2024-03-04T07:34:05.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ca4066fb0 msgr2=0x7f5ca4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ca4066fb0 0x7f5ca4069470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0000f70 tx=0x7f5cb0007240 comp rx=0 tx=0).stop 2024-03-04T07:34:05.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc4158860 msgr2=0x7f5cc40b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:05.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc4158860 0x7f5cc40b0b90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00be30 tx=0x7f5cbc00bf20 comp rx=0 tx=0).stop 2024-03-04T07:34:05.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 shutdown_connections 2024-03-04T07:34:05.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ca4066fb0 0x7f5ca4069470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cc4158860 0x7f5cc40b0b90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cc4152b70 0x7f5cc40cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 --2- 172.21.15.113:0/1889180261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cc40c6e20 0x7f5cc40c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:05.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 >> 172.21.15.113:0/1889180261 conn(0x7f5cc406e820 msgr2=0x7f5cc41580d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:05.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 shutdown_connections 2024-03-04T07:34:05.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:05.170+0000 7f5ccc200640 1 -- 172.21.15.113:0/1889180261 wait complete. 2024-03-04T07:34:05.186 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.5 2024-03-04T07:34:05.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.7 2024-03-04T07:34:05.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.6 2024-03-04T07:34:06.149 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-04T07:34:06.158 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-04T07:34:06.159 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-04T07:34:06.165 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-04T07:34:06.173 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-04T07:34:06.187 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-04T07:34:06.187 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-04T07:34:06.194 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-04T07:34:06.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.370+0000 7f4c77000640 1 Processor -- start 2024-03-04T07:34:06.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.374+0000 7f4c77000640 1 -- start start 2024-03-04T07:34:06.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.374+0000 7f4c77000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4c7006ed90 0x7f4c70159f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.374+0000 7f4c77000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4c7006d300 0x7f4c700c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.374+0000 7f4c77000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7015a5b0 con 0x7f4c70150bc0 2024-03-04T07:34:06.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.374+0000 7f4c77000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7015a890 con 0x7f4c7006ed90 2024-03-04T07:34:06.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.374+0000 7f4c77000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7015ab70 con 0x7f4c7006d300 2024-03-04T07:34:06.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.374+0000 7f4c76600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4c7006d300 0x7f4c700c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.374+0000 7f4c76600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4c7006d300 0x7f4c700c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48022/0 (socket says 172.21.15.113:48022) 2024-03-04T07:34:06.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.374+0000 7f4c76600640 1 -- 172.21.15.113:0/2539404473 learned_addr learned my addr 172.21.15.113:0/2539404473 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.378+0000 7f4c76600640 1 -- 172.21.15.113:0/2539404473 >> v2:172.21.15.113:3300/0 conn(0x7f4c7006ed90 msgr2=0x7f4c70159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.378+0000 7f4c76600640 1 --2- 172.21.15.113:0/2539404473 >> v2:172.21.15.113:3300/0 conn(0x7f4c7006ed90 0x7f4c70159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.378+0000 7f4c76600640 1 -- 172.21.15.113:0/2539404473 >> v1:172.21.15.136:6789/0 conn(0x7f4c70150bc0 legacy=0x7f4c70157320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T07:34:06.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.378+0000 7f4c76600640 1 -- 172.21.15.113:0/2539404473 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c70156b20 con 0x7f4c7006d300 2024-03-04T07:34:06.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.382+0000 7f4c76600640 1 --2- 172.21.15.113:0/2539404473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 0x7f4c700c7eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4c7006e230 tx=0x7f4c6801ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcd20bdd90ccd70c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.382+0000 7f4c6fe00640 1 -- 172.21.15.113:0/2539404473 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c680ad070 con 0x7f4c7006d300 2024-03-04T07:34:06.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.382+0000 7f4c6fe00640 1 -- 172.21.15.113:0/2539404473 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c680a7bf0 con 0x7f4c7006d300 2024-03-04T07:34:06.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.382+0000 7f4c6fe00640 1 -- 172.21.15.113:0/2539404473 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c680b1400 con 0x7f4c7006d300 2024-03-04T07:34:06.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.382+0000 7f4c77000640 1 -- 172.21.15.113:0/2539404473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 msgr2=0x7f4c700c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.382+0000 7f4c77000640 1 --2- 172.21.15.113:0/2539404473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 0x7f4c700c7eb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4c7006e230 tx=0x7f4c6801ec10 comp rx=0 tx=0).stop 2024-03-04T07:34:06.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.382+0000 7f4c77000640 1 -- 172.21.15.113:0/2539404473 shutdown_connections 2024-03-04T07:34:06.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.382+0000 7f4c77000640 1 --2- 172.21.15.113:0/2539404473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 0x7f4c700c7eb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.386+0000 7f4c77000640 1 --2- 172.21.15.113:0/2539404473 >> v2:172.21.15.113:3300/0 conn(0x7f4c7006ed90 0x7f4c70159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.386+0000 7f4c77000640 1 -- 172.21.15.113:0/2539404473 >> 172.21.15.113:0/2539404473 conn(0x7f4c7006e2a0 msgr2=0x7f4c7006e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.386+0000 7f4c77000640 1 -- 172.21.15.113:0/2539404473 shutdown_connections 2024-03-04T07:34:06.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.386+0000 7f4c77000640 1 -- 172.21.15.113:0/2539404473 wait complete. 2024-03-04T07:34:06.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.386+0000 7f4c77000640 1 Processor -- start 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.386+0000 7f4c77000640 1 -- start start 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c77000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 0x7f4c70151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c77000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c7006ed90 0x7f4c7014ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c77000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c70150bc0 0x7f4c70200c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c77000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c70163b20 con 0x7f4c7006ed90 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c75c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 0x7f4c70151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c75c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 0x7f4c70151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48034/0 (socket says 172.21.15.113:48034) 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c75c00640 1 -- 172.21.15.113:0/1011554187 learned_addr learned my addr 172.21.15.113:0/1011554187 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c76600640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c70150bc0 0x7f4c70200c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c70163ca0 con 0x7f4c7006d300 2024-03-04T07:34:06.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c70163e20 con 0x7f4c70150bc0 2024-03-04T07:34:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c76600640 1 -- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 msgr2=0x7f4c70151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c76600640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 0x7f4c70151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c76600640 1 -- 172.21.15.113:0/1011554187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c7006ed90 msgr2=0x7f4c7014ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c76600640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c7006ed90 0x7f4c7014ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c76600640 1 -- 172.21.15.113:0/1011554187 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c700caf30 con 0x7f4c70150bc0 2024-03-04T07:34:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.390+0000 7f4c75c00640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 0x7f4c70151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.394+0000 7f4c76600640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c70150bc0 0x7f4c70200c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4c680af000 tx=0x7f4c6801b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.394+0000 7f4c6d600640 1 -- 172.21.15.113:0/1011554187 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c680ad040 con 0x7f4c70150bc0 2024-03-04T07:34:06.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.394+0000 7f4c6d600640 1 -- 172.21.15.113:0/1011554187 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c680a79f0 con 0x7f4c70150bc0 2024-03-04T07:34:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.394+0000 7f4c6d600640 1 -- 172.21.15.113:0/1011554187 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c680bb5f0 con 0x7f4c70150bc0 2024-03-04T07:34:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.394+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c70204190 con 0x7f4c70150bc0 2024-03-04T07:34:06.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.394+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c700cba80 con 0x7f4c70150bc0 2024-03-04T07:34:06.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.394+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c700726c0 con 0x7f4c70150bc0 2024-03-04T07:34:06.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.398+0000 7f4c6d600640 1 -- 172.21.15.113:0/1011554187 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c680c2060 con 0x7f4c70150bc0 2024-03-04T07:34:06.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.398+0000 7f4c6d600640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4c50066f60 0x7f4c50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.398+0000 7f4c75c00640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4c50066f60 0x7f4c50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.398+0000 7f4c6d600640 1 -- 172.21.15.113:0/1011554187 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c6811f9b0 con 0x7f4c70150bc0 2024-03-04T07:34:06.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.402+0000 7f4c75c00640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4c50066f60 0x7f4c50069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009e20 tx=0x7f4c64009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.410+0000 7f4c6d600640 1 -- 172.21.15.113:0/1011554187 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4c700caf30 con 0x7f4c70150bc0 2024-03-04T07:34:06.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.426+0000 7f55e1800640 1 Processor -- start 2024-03-04T07:34:06.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.426+0000 7f55e1800640 1 -- start start 2024-03-04T07:34:06.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55e1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc158f10 0x7f55dc071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55e1800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55dc0c20a0 0x7f55dc15f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55e1800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc0c3830 con 0x7f55dc0c33e0 2024-03-04T07:34:06.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55e1800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc1592f0 con 0x7f55dc0c20a0 2024-03-04T07:34:06.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55e1800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c81e0 con 0x7f55dc158f10 2024-03-04T07:34:06.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55e0e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f55dc0c33e0 0x7f55dc0c8b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55156/0 (socket says 172.21.15.113:55156) 2024-03-04T07:34:06.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55e0e00640 1 -- 172.21.15.113:0/3306891528 learned_addr learned my addr 172.21.15.113:0/3306891528 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:06.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55dbe00640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc158f10 0x7f55dc071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50314160 0 0) 0x7f55cc001120 con 0x7f55dc0c33e0 2024-03-04T07:34:06.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c0003530 con 0x7f55dc0c33e0 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55db400640 1 --2- 172.21.15.113:0/3306891528 >> v2:172.21.15.113:3300/0 conn(0x7f55dc0c20a0 0x7f55dc15f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 <== mon.1 v1:172.21.15.136:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449690459 0 0) 0x7f55c0003530 con 0x7f55dc0c33e0 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc158f10 msgr2=0x7f55dc071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55daa00640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc158f10 0x7f55dc071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 >> v2:172.21.15.113:3300/0 conn(0x7f55dc0c20a0 msgr2=0x7f55dc15f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55daa00640 1 --2- 172.21.15.113:0/3306891528 >> v2:172.21.15.113:3300/0 conn(0x7f55dc0c20a0 0x7f55dc15f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.430+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 --> v1:172.21.15.136:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc0c7c20 con 0x7f55dc0c33e0 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 <== mon.1 v1:172.21.15.136:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f55cc001850 con 0x7f55dc0c33e0 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 >> v1:172.21.15.136:6789/0 conn(0x7f55dc0c33e0 legacy=0x7f55dc0c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55daa00640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 0x7f55c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55daa00640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc158f10 0x7f55c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55daa00640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0004d50 0x7f55c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc1592f0 con 0x7f55c00039e0 2024-03-04T07:34:06.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c81e0 con 0x7f55c0004d50 2024-03-04T07:34:06.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c000bb50 con 0x7f55dc158f10 2024-03-04T07:34:06.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55dbe00640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 0x7f55c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55db400640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc158f10 0x7f55c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55e0e00640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0004d50 0x7f55c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55dbe00640 1 -- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc158f10 msgr2=0x7f55c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55dbe00640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc158f10 0x7f55c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55dbe00640 1 -- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0004d50 msgr2=0x7f55c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55dbe00640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55c0004d50 0x7f55c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.434+0000 7f55dbe00640 1 -- 172.21.15.113:0/3306891528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc001120 con 0x7f55c00039e0 2024-03-04T07:34:06.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.438+0000 7f55dbe00640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 0x7f55c0003e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009ae0 tx=0x7f55c801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b120956cdfef80a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.438+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80096b0 con 0x7f55c00039e0 2024-03-04T07:34:06.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.438+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c80a6450 con 0x7f55c00039e0 2024-03-04T07:34:06.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.438+0000 7f55daa00640 1 -- 172.21.15.113:0/3306891528 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80a6840 con 0x7f55c00039e0 2024-03-04T07:34:06.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.438+0000 7f55e1800640 1 -- 172.21.15.113:0/3306891528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 msgr2=0x7f55c0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.438+0000 7f55e1800640 1 --2- 172.21.15.113:0/3306891528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 0x7f55c0003e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009ae0 tx=0x7f55c801ca40 comp rx=0 tx=0).stop 2024-03-04T07:34:06.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.438+0000 7f55e0e00640 1 -- 172.21.15.113:0/3306891528 reap_dead start 2024-03-04T07:34:06.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f55e1800640 1 -- 172.21.15.113:0/3306891528 shutdown_connections 2024-03-04T07:34:06.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f55e1800640 1 -- 172.21.15.113:0/3306891528 >> 172.21.15.113:0/3306891528 conn(0x7f55dc0cd080 msgr2=0x7f55dc0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f8230a00640 1 Processor -- start 2024-03-04T07:34:06.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f55e1800640 1 -- 172.21.15.113:0/3306891528 shutdown_connections 2024-03-04T07:34:06.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f45dd600640 1 Processor -- start 2024-03-04T07:34:06.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f8230a00640 1 -- start start 2024-03-04T07:34:06.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f55e1800640 1 -- 172.21.15.113:0/3306891528 wait complete. 2024-03-04T07:34:06.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f45dd600640 1 -- start start 2024-03-04T07:34:06.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f55e1800640 1 Processor -- start 2024-03-04T07:34:06.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f55e1800640 1 -- start start 2024-03-04T07:34:06.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f45dd600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d8152ac0 0x7f45d80c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f45dd600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d8159340 con 0x7f45d80c3ff0 2024-03-04T07:34:06.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f45dd600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d8159620 con 0x7f45d80ca8c0 2024-03-04T07:34:06.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f45dd600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d806f900 con 0x7f45d8152ac0 2024-03-04T07:34:06.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f45d6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d8152ac0 0x7f45d80c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f45d6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d8152ac0 0x7f45d80c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47240/0 (socket says 172.21.15.113:47240) 2024-03-04T07:34:06.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f45d6a00640 1 -- 172.21.15.113:0/3900295723 learned_addr learned my addr 172.21.15.113:0/3900295723 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f8229e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f822c1591a0 0x7f822c15eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55168/0 (socket says 172.21.15.113:55168) 2024-03-04T07:34:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f8229e00640 1 -- 172.21.15.113:0/1924040462 learned_addr learned my addr 172.21.15.113:0/1924040462 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.446+0000 7f8230a00640 1 --2- 172.21.15.113:0/1924040462 >> v2:172.21.15.136:3300/0 conn(0x7f822c168790 0x7f822c16ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8230a00640 1 -- 172.21.15.113:0/1924040462 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822c0c8e50 con 0x7f822c06cde0 2024-03-04T07:34:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f45d6a00640 1 -- 172.21.15.113:0/3900295723 >> v1:172.21.15.136:6789/0 conn(0x7f45d80ca8c0 legacy=0x7f45d8158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f822a800640 1 --2- 172.21.15.113:0/1924040462 >> v2:172.21.15.136:3300/0 conn(0x7f822c168790 0x7f822c16ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55e1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 0x7f55dc157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55e1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc1655c0 0x7f55dc0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8230a00640 1 -- 172.21.15.113:0/1924040462 --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822c072bc0 con 0x7f822c1591a0 2024-03-04T07:34:06.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8230a00640 1 -- 172.21.15.113:0/1924040462 --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822c0c8c20 con 0x7f822c168790 2024-03-04T07:34:06.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55e1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc2046b0 0x7f55dc200ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55e1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c9620 con 0x7f55c00039e0 2024-03-04T07:34:06.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55e1800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc15f2a0 con 0x7f55dc2046b0 2024-03-04T07:34:06.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55e1800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c97a0 con 0x7f55dc1655c0 2024-03-04T07:34:06.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc1655c0 0x7f55dc0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc1655c0 0x7f55dc0cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47250/0 (socket says 172.21.15.113:47250) 2024-03-04T07:34:06.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55db400640 1 -- 172.21.15.113:0/2601998083 learned_addr learned my addr 172.21.15.113:0/2601998083 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55e0e00640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc2046b0 0x7f55dc200ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695933745 0 0) 0x7f822c0c8e50 con 0x7f822c06cde0 2024-03-04T07:34:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8210003530 con 0x7f822c06cde0 2024-03-04T07:34:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f3dc5000640 1 Processor -- start 2024-03-04T07:34:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364796870 0 0) 0x7f822c072bc0 con 0x7f822c1591a0 2024-03-04T07:34:06.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82100048f0 con 0x7f822c1591a0 2024-03-04T07:34:06.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f3dc5000640 1 -- start start 2024-03-04T07:34:06.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55dbe00640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 0x7f55dc157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55dbe00640 1 -- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc1655c0 msgr2=0x7f55dc0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55dbe00640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc1655c0 0x7f55dc0cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55dbe00640 1 -- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc2046b0 msgr2=0x7f55dc200ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55dbe00640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc2046b0 0x7f55dc200ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f45d6a00640 1 -- 172.21.15.113:0/3900295723 >> v1:172.21.15.113:6789/0 conn(0x7f45d80c3ff0 legacy=0x7f45d80ce290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T07:34:06.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1436656437 0 0) 0x7f8210003530 con 0x7f822c06cde0 2024-03-04T07:34:06.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 >> v2:172.21.15.136:3300/0 conn(0x7f822c168790 msgr2=0x7f822c16ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 --2- 172.21.15.113:0/1924040462 >> v2:172.21.15.136:3300/0 conn(0x7f822c168790 0x7f822c16ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 >> v1:172.21.15.136:6789/0 conn(0x7f822c1591a0 legacy=0x7f822c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c06b860 con 0x7f822c06cde0 2024-03-04T07:34:06.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f821c001660 con 0x7f822c06cde0 2024-03-04T07:34:06.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 >> v1:172.21.15.113:6789/0 conn(0x7f822c06cde0 legacy=0x7f822c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 0x7f8210005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8210003820 0x7f8210003ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8210004310 0x7f821000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821000c270 con 0x7f8210004df0 2024-03-04T07:34:06.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821000c420 con 0x7f8210003820 2024-03-04T07:34:06.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821000c620 con 0x7f8210004310 2024-03-04T07:34:06.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f45d6a00640 1 -- 172.21.15.113:0/3900295723 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80c3510 con 0x7f45d8152ac0 2024-03-04T07:34:06.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8229e00640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 0x7f8210005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f1c7aa00640 1 Processor -- start 2024-03-04T07:34:06.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55dbe00640 1 -- 172.21.15.113:0/2601998083 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc0c5880 con 0x7f55c00039e0 2024-03-04T07:34:06.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55db400640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc1655c0 0x7f55dc0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:06.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55e0e00640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc2046b0 0x7f55dc200ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:06.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f822a800640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8210004310 0x7f821000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8229400640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8210003820 0x7f8210003ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8229e00640 1 -- 172.21.15.113:0/1924040462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8210004310 msgr2=0x7f821000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8229e00640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8210004310 0x7f821000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55dbe00640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 0x7f55dc157b80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f55c000baa0 tx=0x7f55c0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8229e00640 1 -- 172.21.15.113:0/1924040462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8210003820 msgr2=0x7f8210003ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8229e00640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8210003820 0x7f8210003ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f822a800640 1 -- 172.21.15.113:0/1924040462 reap_dead start 2024-03-04T07:34:06.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8229e00640 1 -- 172.21.15.113:0/1924040462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8210003530 con 0x7f8210004df0 2024-03-04T07:34:06.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f8229400640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8210003820 0x7f8210003ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:06.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55d3e00640 1 -- 172.21.15.113:0/2601998083 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80096b0 con 0x7f55c00039e0 2024-03-04T07:34:06.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55d3e00640 1 -- 172.21.15.113:0/2601998083 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c80a6c20 con 0x7f55c00039e0 2024-03-04T07:34:06.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55d3e00640 1 -- 172.21.15.113:0/2601998083 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80059a0 con 0x7f55c00039e0 2024-03-04T07:34:06.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f55e1800640 1 -- 172.21.15.113:0/2601998083 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc0c5b60 con 0x7f55c00039e0 2024-03-04T07:34:06.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f3dc5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc00c2d50 0x7f3dc015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f3dc5000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc00c7e70 con 0x7f3dc00c2970 2024-03-04T07:34:06.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f3dc5000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc00c8130 con 0x7f3dc00c72a0 2024-03-04T07:34:06.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f3dc5000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00c9640 con 0x7f3dc00c2d50 2024-03-04T07:34:06.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f3dbde00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3dc00c2970 0x7f3dc00c1980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49606/0 (socket says 172.21.15.113:49606) 2024-03-04T07:34:06.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f55e1800640 1 -- 172.21.15.113:0/2601998083 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc0c6120 con 0x7f55c00039e0 2024-03-04T07:34:06.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.450+0000 7f1c7aa00640 1 -- start start 2024-03-04T07:34:06.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f1c7aa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c740c0ed0 0x7f1c74157230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f1c7aa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1c740729e0 0x7f1c74157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f1c7aa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c740c83b0 con 0x7f1c74150e70 2024-03-04T07:34:06.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f1c7aa00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c740c1b00 con 0x7f1c740729e0 2024-03-04T07:34:06.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f1c7aa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c1540 con 0x7f1c740c0ed0 2024-03-04T07:34:06.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f3dbde00640 1 -- 172.21.15.113:0/3635381193 learned_addr learned my addr 172.21.15.113:0/3635381193 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:06.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f45d6a00640 1 --2- 172.21.15.113:0/3900295723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d8152ac0 0x7f45d80c8c00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f45c8013180 tx=0x7f45c8024c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=225fe39a813080b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f3dbd400640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc00c2d50 0x7f3dc015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f45d6000640 1 -- 172.21.15.113:0/3900295723 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80adb20 con 0x7f45d8152ac0 2024-03-04T07:34:06.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f45d6000640 1 -- 172.21.15.113:0/3900295723 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c80adcc0 con 0x7f45d8152ac0 2024-03-04T07:34:06.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f45d6000640 1 -- 172.21.15.113:0/3900295723 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80112f0 con 0x7f45d8152ac0 2024-03-04T07:34:06.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125256897 0 0) 0x7f3dc00c8130 con 0x7f3dc00c72a0 2024-03-04T07:34:06.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f45dd600640 1 -- 172.21.15.113:0/3900295723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d8152ac0 msgr2=0x7f45d80c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f45dd600640 1 --2- 172.21.15.113:0/3900295723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d8152ac0 0x7f45d80c8c00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f45c8013180 tx=0x7f45c8024c70 comp rx=0 tx=0).stop 2024-03-04T07:34:06.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3da4003530 con 0x7f3dc00c72a0 2024-03-04T07:34:06.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f1c78c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c740c0ed0 0x7f1c74157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f1c78c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c740c0ed0 0x7f1c74157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47286/0 (socket says 172.21.15.113:47286) 2024-03-04T07:34:06.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f1c78c00640 1 -- 172.21.15.113:0/1796040798 learned_addr learned my addr 172.21.15.113:0/1796040798 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f8229e00640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 0x7f8210005210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82180019c0 tx=0x7f821801ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ef44c6955d30f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616789584 0 0) 0x7f3dc00c7e70 con 0x7f3dc00c2970 2024-03-04T07:34:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dc00c8130 con 0x7f3dc00c2970 2024-03-04T07:34:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.454+0000 7f45dd600640 1 -- 172.21.15.113:0/3900295723 shutdown_connections 2024-03-04T07:34:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f45dd600640 1 --2- 172.21.15.113:0/3900295723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d8152ac0 0x7f45d80c8c00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f45dd600640 1 -- 172.21.15.113:0/3900295723 >> 172.21.15.113:0/3900295723 conn(0x7f45d8066db0 msgr2=0x7f45d80671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f1c7a000640 1 --2- 172.21.15.113:0/1796040798 >> v2:172.21.15.136:3300/0 conn(0x7f1c740729e0 0x7f1c74157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 <== mon.1 v1:172.21.15.136:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3282516132 0 0) 0x7f3da4003530 con 0x7f3dc00c72a0 2024-03-04T07:34:06.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f45dd600640 1 -- 172.21.15.113:0/3900295723 shutdown_connections 2024-03-04T07:34:06.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc00c2d50 msgr2=0x7f3dc015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc00c2d50 0x7f3dc015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f1c78c00640 1 -- 172.21.15.113:0/1796040798 >> v2:172.21.15.136:3300/0 conn(0x7f1c740729e0 msgr2=0x7f1c74157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f1c78c00640 1 --2- 172.21.15.113:0/1796040798 >> v2:172.21.15.136:3300/0 conn(0x7f1c740729e0 0x7f1c74157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f1c78c00640 1 -- 172.21.15.113:0/1796040798 >> v1:172.21.15.113:6789/0 conn(0x7f1c74150e70 legacy=0x7f1c7406c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 >> v1:172.21.15.113:6789/0 conn(0x7f3dc00c2970 legacy=0x7f3dc00c1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f1c78c00640 1 -- 172.21.15.113:0/1796040798 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740c1820 con 0x7f1c740c0ed0 2024-03-04T07:34:06.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f1c7a000640 1 --2- 172.21.15.113:0/1796040798 >> v2:172.21.15.136:3300/0 conn(0x7f1c740729e0 0x7f1c74157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:06.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 --> v1:172.21.15.136:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc00c8410 con 0x7f3dc00c72a0 2024-03-04T07:34:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f55d3e00640 1 -- 172.21.15.113:0/2601998083 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55c80ba440 con 0x7f55c00039e0 2024-03-04T07:34:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f55e1800640 1 -- 172.21.15.113:0/2601998083 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55dc075a80 con 0x7f55c00039e0 2024-03-04T07:34:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f55d3e00640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f55bc067030 0x7f55bc0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f55d3e00640 1 -- 172.21.15.113:0/2601998083 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55c8123060 con 0x7f55c00039e0 2024-03-04T07:34:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f55db400640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f55bc067030 0x7f55bc0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f55db400640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f55bc067030 0x7f55bc0694f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55d4003fd0 tx=0x7f55d4003dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f45dd600640 1 -- 172.21.15.113:0/3900295723 wait complete. 2024-03-04T07:34:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f45dd600640 1 Processor -- start 2024-03-04T07:34:06.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f45dd600640 1 -- start start 2024-03-04T07:34:06.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45dd600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d80c3ff0 0x7f45d80c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45dd600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45d80ca8c0 0x7f45d8149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45dd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45d8152ac0 0x7f45d80b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180015e0 con 0x7f8210004df0 2024-03-04T07:34:06.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8218004ab0 con 0x7f8210004df0 2024-03-04T07:34:06.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f8228a00640 1 -- 172.21.15.113:0/1924040462 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180070d0 con 0x7f8210004df0 2024-03-04T07:34:06.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f8230a00640 1 -- 172.21.15.113:0/1924040462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 msgr2=0x7f8210005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f8230a00640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 0x7f8210005210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82180019c0 tx=0x7f821801ed70 comp rx=0 tx=0).stop 2024-03-04T07:34:06.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f8230a00640 1 -- 172.21.15.113:0/1924040462 shutdown_connections 2024-03-04T07:34:06.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f8230a00640 1 --2- 172.21.15.113:0/1924040462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 0x7f8210005210 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f8230a00640 1 -- 172.21.15.113:0/1924040462 >> 172.21.15.113:0/1924040462 conn(0x7f822c0671b0 msgr2=0x7f822c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f8230a00640 1 -- 172.21.15.113:0/1924040462 shutdown_connections 2024-03-04T07:34:06.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f8230a00640 1 -- 172.21.15.113:0/1924040462 wait complete. 2024-03-04T07:34:06.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f8230a00640 1 Processor -- start 2024-03-04T07:34:06.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f8230a00640 1 -- start start 2024-03-04T07:34:06.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f82e1400640 1 Processor -- start 2024-03-04T07:34:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f82e1400640 1 -- start start 2024-03-04T07:34:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f82e1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06f050 0x7f82dc0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f82e1400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc06d550 con 0x7f82dc06ec70 2024-03-04T07:34:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f82e1400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc15eca0 con 0x7f82dc0c8b90 2024-03-04T07:34:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f82e1400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0c3640 con 0x7f82dc06f050 2024-03-04T07:34:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f82db200640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f82dc0c8b90 0x7f82dc1548e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55200/0 (socket says 172.21.15.113:55200) 2024-03-04T07:34:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f82db200640 1 -- 172.21.15.113:0/3116690338 learned_addr learned my addr 172.21.15.113:0/3116690338 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f82d9e00640 1 --2- 172.21.15.113:0/3116690338 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06f050 0x7f82dc0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 <== mon.1 v1:172.21.15.136:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f3dac001680 con 0x7f3dc00c72a0 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 >> v1:172.21.15.136:6789/0 conn(0x7f3dc00c72a0 legacy=0x7f3dc00c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc00c2d50 0x7f3da4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da40040c0 0x7f3da4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da4004d50 0x7f3da40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00c9640 con 0x7f3da4004d50 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4007af0 con 0x7f3da40040c0 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4007cf0 con 0x7f3dc00c2d50 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbe800640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da4004d50 0x7f3da40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbd400640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da40040c0 0x7f3da4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbde00640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc00c2d50 0x7f3da4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbe800640 1 -- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc00c2d50 msgr2=0x7f3da4003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbe800640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc00c2d50 0x7f3da4003b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbe800640 1 -- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da40040c0 msgr2=0x7f3da4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbe800640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da40040c0 0x7f3da4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.458+0000 7f3dbe800640 1 -- 172.21.15.113:0/3635381193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da4003530 con 0x7f3da4004d50 2024-03-04T07:34:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45cfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45d80ca8c0 0x7f45d8149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45d6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d80c3ff0 0x7f45d80c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45d6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d80c3ff0 0x7f45d80c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47312/0 (socket says 172.21.15.113:47312) 2024-03-04T07:34:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45cfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45d80ca8c0 0x7f45d8149430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48088/0 (socket says 172.21.15.113:48088) 2024-03-04T07:34:06.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45dd600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80c4460 con 0x7f45d8152ac0 2024-03-04T07:34:06.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45dd600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d8159900 con 0x7f45d80ca8c0 2024-03-04T07:34:06.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45dd600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d806e180 con 0x7f45d80c3ff0 2024-03-04T07:34:06.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f8230a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 0x7f822c15f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f8230a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f822c1588d0 0x7f822c1f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45d6a00640 1 -- 172.21.15.113:0/2408760524 learned_addr learned my addr 172.21.15.113:0/2408760524 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f8230a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f822c15edb0 0x7f822c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f8230a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0c7b20 con 0x7f8210004df0 2024-03-04T07:34:06.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f8230a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0c7e00 con 0x7f822c15edb0 2024-03-04T07:34:06.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f8230a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c06b060 con 0x7f822c1588d0 2024-03-04T07:34:06.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f8229400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f822c1588d0 0x7f822c1f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f82d9400640 1 -- 172.21.15.113:0/3116690338 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407062305 0 0) 0x7f82dc15eca0 con 0x7f82dc0c8b90 2024-03-04T07:34:06.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f82d9400640 1 -- 172.21.15.113:0/3116690338 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c0003530 con 0x7f82dc0c8b90 2024-03-04T07:34:06.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f1c78c00640 1 --2- 172.21.15.113:0/1796040798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c740c0ed0 0x7f1c74157230 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c60008870 tx=0x7f1c6001ea70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2eab6137307b2ea server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7faccbe00640 1 Processor -- start 2024-03-04T07:34:06.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7faccbe00640 1 -- start start 2024-03-04T07:34:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f822a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f822c15edb0 0x7f822c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f822a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f822c15edb0 0x7f822c154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48096/0 (socket says 172.21.15.113:48096) 2024-03-04T07:34:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f822a800640 1 -- 172.21.15.113:0/1491599068 learned_addr learned my addr 172.21.15.113:0/1491599068 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45d7400640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45d8152ac0 0x7f45d80b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c73e00640 1 -- 172.21.15.113:0/1796040798 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c600ad070 con 0x7f1c740c0ed0 2024-03-04T07:34:06.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f82d9e00640 1 -- 172.21.15.113:0/3116690338 >> v1:172.21.15.136:6789/0 conn(0x7f82dc0c8b90 legacy=0x7f82dc1548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f82d9e00640 1 -- 172.21.15.113:0/3116690338 >> v1:172.21.15.113:6789/0 conn(0x7f82dc06ec70 legacy=0x7f82dc15e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f82d9e00640 1 -- 172.21.15.113:0/3116690338 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc15ef80 con 0x7f82dc06f050 2024-03-04T07:34:06.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f8229e00640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 0x7f822c15f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45d6a00640 1 -- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45d80ca8c0 msgr2=0x7f45d8149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.462+0000 7f45d6a00640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45d80ca8c0 0x7f45d8149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f822a800640 1 -- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f822c1588d0 msgr2=0x7f822c1f8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f822a800640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f822c1588d0 0x7f822c1f8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f822a800640 1 -- 172.21.15.113:0/1491599068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 msgr2=0x7f822c15f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f822a800640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 0x7f822c15f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f822a800640 1 -- 172.21.15.113:0/1491599068 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c1f9a50 con 0x7f822c15edb0 2024-03-04T07:34:06.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f3dbe800640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da4004d50 0x7f3da40051d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3dac007f40 tx=0x7f3dac01cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33f81a02fb4137c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f45d6a00640 1 -- 172.21.15.113:0/2408760524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45d8152ac0 msgr2=0x7f45d80b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f45d6a00640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45d8152ac0 0x7f45d80b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f45d6a00640 1 -- 172.21.15.113:0/2408760524 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d814bc30 con 0x7f45d80c3ff0 2024-03-04T07:34:06.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f45d7400640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45d8152ac0 0x7f45d80b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:06.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c73e00640 1 -- 172.21.15.113:0/1796040798 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c600a7a50 con 0x7f1c740c0ed0 2024-03-04T07:34:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f822a800640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f822c15edb0 0x7f822c154330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f822c0cdbd0 tx=0x7f822400f1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f8222000640 1 -- 172.21.15.113:0/1491599068 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822400f8d0 con 0x7f822c15edb0 2024-03-04T07:34:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f8222000640 1 -- 172.21.15.113:0/1491599068 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822401c070 con 0x7f822c15edb0 2024-03-04T07:34:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f8222000640 1 -- 172.21.15.113:0/1491599068 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822400fcf0 con 0x7f822c15edb0 2024-03-04T07:34:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f8230a00640 1 -- 172.21.15.113:0/1491599068 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c15def0 con 0x7f822c15edb0 2024-03-04T07:34:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faccc154c40 0x7faccc15b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7faccc0d0030 0x7faccc0d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccbe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc0cb4c0 con 0x7faccc0cc2d0 2024-03-04T07:34:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c7aa00640 1 -- 172.21.15.113:0/1796040798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c740c0ed0 msgr2=0x7f1c74157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccbe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc155020 con 0x7faccc154c40 2024-03-04T07:34:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccbe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc15df00 con 0x7faccc0d0030 2024-03-04T07:34:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7faccc0d0030 0x7faccc0d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7faccc0d0030 0x7faccc0d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48112/0 (socket says 172.21.15.113:48112) 2024-03-04T07:34:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccb400640 1 -- 172.21.15.113:0/353627313 learned_addr learned my addr 172.21.15.113:0/353627313 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccaa00640 1 --2- 172.21.15.113:0/353627313 >> v2:172.21.15.113:3300/0 conn(0x7faccc154c40 0x7faccc15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c73e00640 1 -- 172.21.15.113:0/1796040798 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c600a7e20 con 0x7f1c740c0ed0 2024-03-04T07:34:06.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c7aa00640 1 --2- 172.21.15.113:0/1796040798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c740c0ed0 0x7f1c74157230 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c60008870 tx=0x7f1c6001ea70 comp rx=0 tx=0).stop 2024-03-04T07:34:06.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f8230a00640 1 -- 172.21.15.113:0/1491599068 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c153e10 con 0x7f822c15edb0 2024-03-04T07:34:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccb400640 1 -- 172.21.15.113:0/353627313 >> v2:172.21.15.113:3300/0 conn(0x7faccc154c40 msgr2=0x7faccc15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccb400640 1 --2- 172.21.15.113:0/353627313 >> v2:172.21.15.113:3300/0 conn(0x7faccc154c40 0x7faccc15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccb400640 1 -- 172.21.15.113:0/353627313 >> v1:172.21.15.113:6789/0 conn(0x7faccc0cc2d0 legacy=0x7faccc0c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7faccb400640 1 -- 172.21.15.113:0/353627313 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc15e1e0 con 0x7faccc0d0030 2024-03-04T07:34:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c7aa00640 1 -- 172.21.15.113:0/1796040798 shutdown_connections 2024-03-04T07:34:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c7aa00640 1 --2- 172.21.15.113:0/1796040798 >> v2:172.21.15.136:3300/0 conn(0x7f1c740729e0 0x7f1c74157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c7aa00640 1 --2- 172.21.15.113:0/1796040798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c740c0ed0 0x7f1c74157230 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c7aa00640 1 -- 172.21.15.113:0/1796040798 >> 172.21.15.113:0/1796040798 conn(0x7f1c7406e2c0 msgr2=0x7f1c7406e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c7aa00640 1 -- 172.21.15.113:0/1796040798 shutdown_connections 2024-03-04T07:34:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.466+0000 7f1c7aa00640 1 -- 172.21.15.113:0/1796040798 wait complete. 2024-03-04T07:34:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f45d6a00640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d80c3ff0 0x7f45d80c5f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f45c80228a0 tx=0x7f45c80234b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f8222000640 1 -- 172.21.15.113:0/1491599068 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8224004030 con 0x7f822c15edb0 2024-03-04T07:34:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f8230a00640 1 -- 172.21.15.113:0/1491599068 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822c154010 con 0x7f822c15edb0 2024-03-04T07:34:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f45cea00640 1 -- 172.21.15.113:0/2408760524 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80b56c0 con 0x7f45d80c3ff0 2024-03-04T07:34:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f45cea00640 1 -- 172.21.15.113:0/2408760524 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c80b5860 con 0x7f45d80c3ff0 2024-03-04T07:34:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f45dd600640 1 -- 172.21.15.113:0/2408760524 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d8204480 con 0x7f45d80c3ff0 2024-03-04T07:34:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f82d9e00640 1 --2- 172.21.15.113:0/3116690338 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06f050 0x7f82dc0c9e60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82c80097f0 tx=0x7f82c801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240a4408abc8ac39 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f45dd600640 1 -- 172.21.15.113:0/2408760524 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d8203900 con 0x7f45d80c3ff0 2024-03-04T07:34:06.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f45cea00640 1 -- 172.21.15.113:0/2408760524 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80b5c50 con 0x7f45d80c3ff0 2024-03-04T07:34:06.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f8222000640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f820c066f90 0x7f820c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f8222000640 1 -- 172.21.15.113:0/1491599068 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82240859d0 con 0x7f822c15edb0 2024-03-04T07:34:06.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f82d9400640 1 -- 172.21.15.113:0/3116690338 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c80b0070 con 0x7f82dc06f050 2024-03-04T07:34:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f8229e00640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f820c066f90 0x7f820c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f82d9400640 1 -- 172.21.15.113:0/3116690338 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c80a4e50 con 0x7f82dc06f050 2024-03-04T07:34:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f1c7aa00640 1 Processor -- start 2024-03-04T07:34:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f82d9400640 1 -- 172.21.15.113:0/3116690338 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c80ab800 con 0x7f82dc06f050 2024-03-04T07:34:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7faccb400640 1 --2- 172.21.15.113:0/353627313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faccc0d0030 0x7faccc0d0440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faccc0c71a0 tx=0x7facc401f180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43764f998aefdaab server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f45dd600640 1 -- 172.21.15.113:0/2408760524 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d8075c00 con 0x7f45d80c3ff0 2024-03-04T07:34:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f1c7aa00640 1 -- start start 2024-03-04T07:34:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f82e1400640 1 -- 172.21.15.113:0/3116690338 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06f050 msgr2=0x7f82dc0c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.470+0000 7f82e1400640 1 --2- 172.21.15.113:0/3116690338 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06f050 0x7f82dc0c9e60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f82c80097f0 tx=0x7f82c801c3c0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f55d3e00640 1 -- 172.21.15.113:0/2601998083 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f55c80ac020 con 0x7f55c00039e0 2024-03-04T07:34:06.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f82e1400640 1 -- 172.21.15.113:0/3116690338 shutdown_connections 2024-03-04T07:34:06.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f82e1400640 1 --2- 172.21.15.113:0/3116690338 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06f050 0x7f82dc0c9e60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f82e1400640 1 -- 172.21.15.113:0/3116690338 >> 172.21.15.113:0/3116690338 conn(0x7f82dc066db0 msgr2=0x7f82dc0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f82e1400640 1 -- 172.21.15.113:0/3116690338 shutdown_connections 2024-03-04T07:34:06.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f82e1400640 1 -- 172.21.15.113:0/3116690338 wait complete. 2024-03-04T07:34:06.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0b2070 con 0x7f3da4004d50 2024-03-04T07:34:06.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dac0a59e0 con 0x7f3da4004d50 2024-03-04T07:34:06.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f3dbca00640 1 -- 172.21.15.113:0/3635381193 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac007b60 con 0x7f3da4004d50 2024-03-04T07:34:06.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f8229e00640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f820c066f90 0x7f820c069450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82180ab000 tx=0x7f8218001350 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c7aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c740c0ed0 0x7f1c741fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c7aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c74150e70 0x7f1c74160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c7aa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c74156720 0x7f1c741e8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f3dc5000640 1 -- 172.21.15.113:0/3635381193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da4004d50 msgr2=0x7f3da40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f3dc5000640 1 --2- 172.21.15.113:0/3635381193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3da4004d50 0x7f3da40051d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3dac007f40 tx=0x7f3dac01cb00 comp rx=0 tx=0).stop 2024-03-04T07:34:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c7aa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c7df0 con 0x7f1c74150e70 2024-03-04T07:34:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f3dbe800640 1 -- 172.21.15.113:0/3635381193 reap_dead start 2024-03-04T07:34:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c7aa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c7c70 con 0x7f1c740c0ed0 2024-03-04T07:34:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c79600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c740c0ed0 0x7f1c741fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c7aa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7406d8a0 con 0x7f1c74156720 2024-03-04T07:34:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c79600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c740c0ed0 0x7f1c741fb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48126/0 (socket says 172.21.15.113:48126) 2024-03-04T07:34:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c79600640 1 -- 172.21.15.113:0/2250061837 learned_addr learned my addr 172.21.15.113:0/2250061837 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c7a000640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c74156720 0x7f1c741e8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f45cea00640 1 -- 172.21.15.113:0/2408760524 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45c80bc020 con 0x7f45d80c3ff0 2024-03-04T07:34:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f45cea00640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f45b4066fb0 0x7f45b4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c78c00640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c74150e70 0x7f1c74160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f82e1400640 1 Processor -- start 2024-03-04T07:34:06.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f3dc5000640 1 -- 172.21.15.113:0/3635381193 shutdown_connections 2024-03-04T07:34:06.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f3dc5000640 1 -- 172.21.15.113:0/3635381193 >> 172.21.15.113:0/3635381193 conn(0x7f3dc006e820 msgr2=0x7f3dc006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f45cfe00640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f45b4066fb0 0x7f45b4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f82e1400640 1 -- start start 2024-03-04T07:34:06.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c79600640 1 -- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c74156720 msgr2=0x7f1c741e8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c79600640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c74156720 0x7f1c741e8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c79600640 1 -- 172.21.15.113:0/2250061837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c74150e70 msgr2=0x7f1c74160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c79600640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c74150e70 0x7f1c74160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c79600640 1 -- 172.21.15.113:0/2250061837 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c741fb5e0 con 0x7f1c740c0ed0 2024-03-04T07:34:06.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f45cea00640 1 -- 172.21.15.113:0/2408760524 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45c80b5df0 con 0x7f45d80c3ff0 2024-03-04T07:34:06.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f3dc5000640 1 -- 172.21.15.113:0/3635381193 shutdown_connections 2024-03-04T07:34:06.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f1c79600640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c740c0ed0 0x7f1c741fb0a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c7f0 tx=0x7f1c6c00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f3dc5000640 1 -- 172.21.15.113:0/3635381193 wait complete. 2024-03-04T07:34:06.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.474+0000 7f45cfe00640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f45b4066fb0 0x7f45b4069470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f45d80cb120 tx=0x7f45c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82e1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06ec70 0x7f82dc156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82e1400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc06f050 0x7f82dc1569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82e1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c8b90 0x7f82dc14dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82e1400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0c99d0 con 0x7f82dc0c8b90 2024-03-04T07:34:06.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82e1400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0c9130 con 0x7f82dc06f050 2024-03-04T07:34:06.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82e1400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc15e900 con 0x7f82dc06ec70 2024-03-04T07:34:06.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82db200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c8b90 0x7f82dc14dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82db200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c8b90 0x7f82dc14dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57458/0 (socket says 172.21.15.113:57458) 2024-03-04T07:34:06.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82db200640 1 -- 172.21.15.113:0/3779186929 learned_addr learned my addr 172.21.15.113:0/3779186929 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82da800640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06ec70 0x7f82dc156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82d9e00640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc06f050 0x7f82dc1569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82db200640 1 -- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06ec70 msgr2=0x7f82dc156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82db200640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06ec70 0x7f82dc156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82db200640 1 -- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc06f050 msgr2=0x7f82dc1569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82db200640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc06f050 0x7f82dc1569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82db200640 1 -- 172.21.15.113:0/3779186929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc157c00 con 0x7f82dc0c8b90 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82db200640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c8b90 0x7f82dc14dcb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82d400cad0 tx=0x7f82d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82d2a00640 1 -- 172.21.15.113:0/3779186929 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4017070 con 0x7f82dc0c8b90 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82d2a00640 1 -- 172.21.15.113:0/3779186929 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d400ac30 con 0x7f82dc0c8b90 2024-03-04T07:34:06.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82d2a00640 1 -- 172.21.15.113:0/3779186929 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d4012400 con 0x7f82dc0c8b90 2024-03-04T07:34:06.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f3dc5000640 1 Processor -- start 2024-03-04T07:34:06.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82e1400640 1 -- 172.21.15.113:0/3779186929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc157ee0 con 0x7f82dc0c8b90 2024-03-04T07:34:06.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82e1400640 1 -- 172.21.15.113:0/3779186929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc1577e0 con 0x7f82dc0c8b90 2024-03-04T07:34:06.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f3dc5000640 1 -- start start 2024-03-04T07:34:06.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f82e1400640 1 -- 172.21.15.113:0/3779186929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc1572f0 con 0x7f82dc0c8b90 2024-03-04T07:34:06.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f3dc5000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da4004d50 0x7f3dc0150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f3dc5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc0151da0 0x7f3dc014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f3dc5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc01def90 0x7f3dc01e1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f3dc5000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0157310 con 0x7f3dc0151da0 2024-03-04T07:34:06.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f3dc5000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0157680 con 0x7f3da4004d50 2024-03-04T07:34:06.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.478+0000 7f3dc5000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0156ce0 con 0x7f3dc01def90 2024-03-04T07:34:06.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da4004d50 0x7f3dc0150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da4004d50 0x7f3dc0150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48152/0 (socket says 172.21.15.113:48152) 2024-03-04T07:34:06.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbde00640 1 -- 172.21.15.113:0/1323081822 learned_addr learned my addr 172.21.15.113:0/1323081822 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f82d2a00640 1 -- 172.21.15.113:0/3779186929 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82d401a070 con 0x7f82dc0c8b90 2024-03-04T07:34:06.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbd400640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc0151da0 0x7f3dc014ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbe800640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc01def90 0x7f3dc01e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbde00640 1 -- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc01def90 msgr2=0x7f3dc01e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbde00640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc01def90 0x7f3dc01e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbde00640 1 -- 172.21.15.113:0/1323081822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc0151da0 msgr2=0x7f3dc014ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbde00640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc0151da0 0x7f3dc014ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbde00640 1 -- 172.21.15.113:0/1323081822 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc01510e0 con 0x7f3da4004d50 2024-03-04T07:34:06.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f82d2a00640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82bc066f60 0x7f82bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f3dbde00640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da4004d50 0x7f3dc0150240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3db800fba0 tx=0x7f3db800d430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f82d2a00640 1 -- 172.21.15.113:0/3779186929 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82d4084d80 con 0x7f82dc0c8b90 2024-03-04T07:34:06.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f82da800640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82bc066f60 0x7f82bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.482+0000 7f82da800640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82bc066f60 0x7f82bc069420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000f70 tx=0x7f82cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.486+0000 7f82d2a00640 1 -- 172.21.15.113:0/3779186929 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f82d40556c0 con 0x7f82dc0c8b90 2024-03-04T07:34:06.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f3db6000640 1 -- 172.21.15.113:0/1323081822 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db801b070 con 0x7f3da4004d50 2024-03-04T07:34:06.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f1c71600640 1 -- 172.21.15.113:0/2250061837 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c003b80 con 0x7f1c740c0ed0 2024-03-04T07:34:06.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f3dc5000640 1 -- 172.21.15.113:0/1323081822 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc01513c0 con 0x7f3da4004d50 2024-03-04T07:34:06.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f1c71600640 1 -- 172.21.15.113:0/2250061837 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c003d20 con 0x7f1c740c0ed0 2024-03-04T07:34:06.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f1c71600640 1 -- 172.21.15.113:0/2250061837 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00aa90 con 0x7f1c740c0ed0 2024-03-04T07:34:06.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f8222000640 1 -- 172.21.15.113:0/1491599068 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f822c154010 con 0x7f822c15edb0 2024-03-04T07:34:06.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7facc9600640 1 -- 172.21.15.113:0/353627313 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40b4070 con 0x7faccc0d0030 2024-03-04T07:34:06.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7facc9600640 1 -- 172.21.15.113:0/353627313 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc40af400 con 0x7faccc0d0030 2024-03-04T07:34:06.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7facc9600640 1 -- 172.21.15.113:0/353627313 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40af7d0 con 0x7faccc0d0030 2024-03-04T07:34:06.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f3dc5000640 1 -- 172.21.15.113:0/1323081822 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc0151980 con 0x7f3da4004d50 2024-03-04T07:34:06.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f3db6000640 1 -- 172.21.15.113:0/1323081822 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db8004d70 con 0x7f3da4004d50 2024-03-04T07:34:06.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f3db6000640 1 -- 172.21.15.113:0/1323081822 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db800dce0 con 0x7f3da4004d50 2024-03-04T07:34:06.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7faccbe00640 1 -- 172.21.15.113:0/353627313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faccc0d0030 msgr2=0x7faccc0d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7faccbe00640 1 --2- 172.21.15.113:0/353627313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faccc0d0030 0x7faccc0d0440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faccc0c71a0 tx=0x7facc401f180 comp rx=0 tx=0).stop 2024-03-04T07:34:06.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7faccbe00640 1 -- 172.21.15.113:0/353627313 shutdown_connections 2024-03-04T07:34:06.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7faccbe00640 1 --2- 172.21.15.113:0/353627313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faccc0d0030 0x7faccc0d0440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7faccbe00640 1 --2- 172.21.15.113:0/353627313 >> v2:172.21.15.113:3300/0 conn(0x7faccc154c40 0x7faccc15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7faccbe00640 1 -- 172.21.15.113:0/353627313 >> 172.21.15.113:0/353627313 conn(0x7faccc06d3e0 msgr2=0x7faccc06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f1c7aa00640 1 -- 172.21.15.113:0/2250061837 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c74154b70 con 0x7f1c740c0ed0 2024-03-04T07:34:06.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f1c7aa00640 1 -- 172.21.15.113:0/2250061837 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c741f7d20 con 0x7f1c740c0ed0 2024-03-04T07:34:06.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7faccbe00640 1 -- 172.21.15.113:0/353627313 shutdown_connections 2024-03-04T07:34:06.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7faccbe00640 1 -- 172.21.15.113:0/353627313 wait complete. 2024-03-04T07:34:06.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f3dc5000640 1 -- 172.21.15.113:0/1323081822 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d980052a0 con 0x7f3da4004d50 2024-03-04T07:34:06.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.494+0000 7f1c7aa00640 1 -- 172.21.15.113:0/2250061837 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c741f7b30 con 0x7f1c740c0ed0 2024-03-04T07:34:06.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.498+0000 7faccbe00640 1 Processor -- start 2024-03-04T07:34:06.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.498+0000 7f45cea00640 1 -- 172.21.15.113:0/2408760524 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f45d814bc30 con 0x7f45d80c3ff0 2024-03-04T07:34:06.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.498+0000 7faccbe00640 1 -- start start 2024-03-04T07:34:06.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.498+0000 7f3db6000640 1 -- 172.21.15.113:0/1323081822 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3db801e070 con 0x7f3da4004d50 2024-03-04T07:34:06.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.498+0000 7f3db6000640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3da0066f90 0x7f3da0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.498+0000 7f3db6000640 1 -- 172.21.15.113:0/1323081822 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3db80878f0 con 0x7f3da4004d50 2024-03-04T07:34:06.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.498+0000 7f1c71600640 1 -- 172.21.15.113:0/2250061837 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c6c00ac30 con 0x7f1c740c0ed0 2024-03-04T07:34:06.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.498+0000 7f3dbd400640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3da0066f90 0x7f3da0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.498+0000 7faccbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faccc0cc2d0 0x7faccc15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faccc0d0030 0x7faccc1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc154c40 0x7faccc148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc15ecc0 con 0x7faccc154c40 2024-03-04T07:34:06.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccbe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc15e7a0 con 0x7faccc0cc2d0 2024-03-04T07:34:06.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0cb180 con 0x7faccc0d0030 2024-03-04T07:34:06.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7f1c71600640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1c54066f60 0x7f1c54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7f1c71600640 1 -- 172.21.15.113:0/2250061837 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c6c085990 con 0x7f1c740c0ed0 2024-03-04T07:34:06.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faccc0cc2d0 0x7faccc15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7facca000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faccc0d0030 0x7faccc1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7facca000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faccc0d0030 0x7faccc1599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47364/0 (socket says 172.21.15.113:47364) 2024-03-04T07:34:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7facca000640 1 -- 172.21.15.113:0/44503465 learned_addr learned my addr 172.21.15.113:0/44503465 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccb400640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc154c40 0x7faccc148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7f3dbd400640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3da0066f90 0x7f3da0069450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3dc014ee90 tx=0x7f3db0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7f1c78c00640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1c54066f60 0x7f1c54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccb400640 1 -- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faccc0d0030 msgr2=0x7faccc1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccb400640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faccc0d0030 0x7faccc1599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccb400640 1 -- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faccc0cc2d0 msgr2=0x7faccc15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccb400640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faccc0cc2d0 0x7faccc15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccb400640 1 -- 172.21.15.113:0/44503465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc15a770 con 0x7faccc154c40 2024-03-04T07:34:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccb400640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc154c40 0x7faccc148840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faccc15e080 tx=0x7facc401bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7facc2a00640 1 -- 172.21.15.113:0/44503465 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40be020 con 0x7faccc154c40 2024-03-04T07:34:06.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7f1c78c00640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1c54066f60 0x7f1c54069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c600b2000 tx=0x7f1c6000a040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7facc2a00640 1 -- 172.21.15.113:0/44503465 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc40090d0 con 0x7faccc154c40 2024-03-04T07:34:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccbe00640 1 -- 172.21.15.113:0/44503465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc151740 con 0x7faccc154c40 2024-03-04T07:34:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7facc2a00640 1 -- 172.21.15.113:0/44503465 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40b4040 con 0x7faccc154c40 2024-03-04T07:34:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.502+0000 7faccbe00640 1 -- 172.21.15.113:0/44503465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc15d120 con 0x7faccc154c40 2024-03-04T07:34:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.506+0000 7facb7e00640 1 -- 172.21.15.113:0/44503465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca00052a0 con 0x7faccc154c40 2024-03-04T07:34:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.506+0000 7facc2a00640 1 -- 172.21.15.113:0/44503465 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7facc40c1020 con 0x7faccc154c40 2024-03-04T07:34:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.510+0000 7facc2a00640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facac066fb0 0x7facac069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.510+0000 7facc2a00640 1 -- 172.21.15.113:0/44503465 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facc4122f00 con 0x7faccc154c40 2024-03-04T07:34:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.510+0000 7faccaa00640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facac066fb0 0x7facac069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:06.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.510+0000 7faccaa00640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facac066fb0 0x7facac069470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7facbc009b60 tx=0x7facbc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.510+0000 7f1c71600640 1 -- 172.21.15.113:0/2250061837 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1c6c056330 con 0x7f1c740c0ed0 2024-03-04T07:34:06.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.518+0000 7facc2a00640 1 -- 172.21.15.113:0/44503465 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7facc40c8030 con 0x7faccc154c40 2024-03-04T07:34:06.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.526+0000 7f3db6000640 1 -- 172.21.15.113:0/1323081822 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3db80582b0 con 0x7f3da4004d50 2024-03-04T07:34:06.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.726+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4c70075a80 con 0x7f4c70150bc0 2024-03-04T07:34:06.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.726+0000 7f4c6d600640 1 -- 172.21.15.113:0/1011554187 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c68018920 con 0x7f4c70150bc0 2024-03-04T07:34:06.731 INFO:teuthology.orchestra.run.smithi113.stdout:60129542152 2024-03-04T07:34:06.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.730+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4c50066f60 msgr2=0x7f4c50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4c50066f60 0x7f4c50069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009e20 tx=0x7f4c64009190 comp rx=0 tx=0).stop 2024-03-04T07:34:06.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c70150bc0 msgr2=0x7f4c70200c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c70150bc0 0x7f4c70200c20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4c680af000 tx=0x7f4c6801b9b0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 shutdown_connections 2024-03-04T07:34:06.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4c70150bc0 0x7f4c70200c20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4c50066f60 0x7f4c50069420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c7006ed90 0x7f4c7014ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 --2- 172.21.15.113:0/1011554187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4c7006d300 0x7f4c70151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 >> 172.21.15.113:0/1011554187 conn(0x7f4c7006e2a0 msgr2=0x7f4c700c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 shutdown_connections 2024-03-04T07:34:06.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.734+0000 7f4c77000640 1 -- 172.21.15.113:0/1011554187 wait complete. 2024-03-04T07:34:06.766 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.2 2024-03-04T07:34:06.767 DEBUG:teuthology.parallel:result is None 2024-03-04T07:34:06.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.794+0000 7f55e1800640 1 -- 172.21.15.113:0/2601998083 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f55dc0c8400 con 0x7f55c00039e0 2024-03-04T07:34:06.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.794+0000 7f55d3e00640 1 -- 172.21.15.113:0/2601998083 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55c8016780 con 0x7f55c00039e0 2024-03-04T07:34:06.798 INFO:teuthology.orchestra.run.smithi113.stdout:64424509448 2024-03-04T07:34:06.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 -- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f55bc067030 msgr2=0x7f55bc0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f55bc067030 0x7f55bc0694f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55d4003fd0 tx=0x7f55d4003dc0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 -- 172.21.15.113:0/2601998083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 msgr2=0x7f55dc157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 0x7f55dc157b80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f55c000baa0 tx=0x7f55c0005840 comp rx=0 tx=0).stop 2024-03-04T07:34:06.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 -- 172.21.15.113:0/2601998083 shutdown_connections 2024-03-04T07:34:06.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc2046b0 0x7f55dc200ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f55dc1655c0 0x7f55dc0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f55bc067030 0x7f55bc0694f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 --2- 172.21.15.113:0/2601998083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c00039e0 0x7f55dc157b80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 -- 172.21.15.113:0/2601998083 >> 172.21.15.113:0/2601998083 conn(0x7f55dc0cd080 msgr2=0x7f55dc06c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 -- 172.21.15.113:0/2601998083 shutdown_connections 2024-03-04T07:34:06.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.798+0000 7f55e1800640 1 -- 172.21.15.113:0/2601998083 wait complete. 2024-03-04T07:34:06.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.806+0000 7f82e1400640 1 -- 172.21.15.113:0/3779186929 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f82dc075c00 con 0x7f82dc0c8b90 2024-03-04T07:34:06.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82d2a00640 1 -- 172.21.15.113:0/3779186929 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f82d4059760 con 0x7f82dc0c8b90 2024-03-04T07:34:06.814 INFO:teuthology.orchestra.run.smithi113.stdout:60129542151 2024-03-04T07:34:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82e1400640 1 -- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82bc066f60 msgr2=0x7f82bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82e1400640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82bc066f60 0x7f82bc069420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f82cc000f70 tx=0x7f82cc009040 comp rx=0 tx=0).stop 2024-03-04T07:34:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82e1400640 1 -- 172.21.15.113:0/3779186929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c8b90 msgr2=0x7f82dc14dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82e1400640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c8b90 0x7f82dc14dcb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82d400cad0 tx=0x7f82d400cf90 comp rx=0 tx=0).stop 2024-03-04T07:34:06.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82e1400640 1 -- 172.21.15.113:0/3779186929 shutdown_connections 2024-03-04T07:34:06.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82e1400640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82bc066f60 0x7f82bc069420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82e1400640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc0c8b90 0x7f82dc14dcb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82e1400640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc06f050 0x7f82dc1569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82e1400640 1 --2- 172.21.15.113:0/3779186929 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82dc06ec70 0x7f82dc156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.810+0000 7f82e1400640 1 -- 172.21.15.113:0/3779186929 >> 172.21.15.113:0/3779186929 conn(0x7f82dc066db0 msgr2=0x7f82dc15c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.814+0000 7f82e1400640 1 -- 172.21.15.113:0/3779186929 shutdown_connections 2024-03-04T07:34:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.814+0000 7f3dc5000640 1 -- 172.21.15.113:0/1323081822 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3d98005860 con 0x7f3da4004d50 2024-03-04T07:34:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.814+0000 7f82e1400640 1 -- 172.21.15.113:0/3779186929 wait complete. 2024-03-04T07:34:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.814+0000 7f3db6000640 1 -- 172.21.15.113:0/1323081822 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3db805c350 con 0x7f3da4004d50 2024-03-04T07:34:06.817 INFO:teuthology.orchestra.run.smithi113.stdout:60129542152 2024-03-04T07:34:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 -- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3da0066f90 msgr2=0x7f3da0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3da0066f90 0x7f3da0069450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3dc014ee90 tx=0x7f3db0008040 comp rx=0 tx=0).stop 2024-03-04T07:34:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 -- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da4004d50 msgr2=0x7f3dc0150240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da4004d50 0x7f3dc0150240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3db800fba0 tx=0x7f3db800d430 comp rx=0 tx=0).stop 2024-03-04T07:34:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 -- 172.21.15.113:0/1323081822 shutdown_connections 2024-03-04T07:34:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3dc01def90 0x7f3dc01e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3da0066f90 0x7f3da0069450 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dc0151da0 0x7f3dc014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 --2- 172.21.15.113:0/1323081822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3da4004d50 0x7f3dc0150240 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 -- 172.21.15.113:0/1323081822 >> 172.21.15.113:0/1323081822 conn(0x7f3dc006e820 msgr2=0x7f3dc00cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 -- 172.21.15.113:0/1323081822 shutdown_connections 2024-03-04T07:34:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.818+0000 7f3dc5000640 1 -- 172.21.15.113:0/1323081822 wait complete. 2024-03-04T07:34:06.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.826+0000 7f45dd600640 1 -- 172.21.15.113:0/2408760524 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f45d80b18d0 con 0x7f45d80c3ff0 2024-03-04T07:34:06.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.826+0000 7f45cea00640 1 -- 172.21.15.113:0/2408760524 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f45c801e860 con 0x7f45d80c3ff0 2024-03-04T07:34:06.831 INFO:teuthology.orchestra.run.smithi113.stdout:60129542151 2024-03-04T07:34:06.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f8230a00640 1 -- 172.21.15.113:0/1491599068 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f822c075c00 con 0x7f822c15edb0 2024-03-04T07:34:06.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f8222000640 1 -- 172.21.15.113:0/1491599068 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8224010040 con 0x7f822c15edb0 2024-03-04T07:34:06.834 INFO:teuthology.orchestra.run.smithi113.stdout:64424509448 2024-03-04T07:34:06.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f45dd600640 1 -- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f45b4066fb0 msgr2=0x7f45b4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f45dd600640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f45b4066fb0 0x7f45b4069470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f45d80cb120 tx=0x7f45c0009040 comp rx=0 tx=0).stop 2024-03-04T07:34:06.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f45dd600640 1 -- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d80c3ff0 msgr2=0x7f45d80c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f45dd600640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d80c3ff0 0x7f45d80c5f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f45c80228a0 tx=0x7f45c80234b0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f45dd600640 1 -- 172.21.15.113:0/2408760524 shutdown_connections 2024-03-04T07:34:06.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f45dd600640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f45b4066fb0 0x7f45b4069470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f45dd600640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45d8152ac0 0x7f45d80b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f45dd600640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45d80ca8c0 0x7f45d8149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f45dd600640 1 --2- 172.21.15.113:0/2408760524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45d80c3ff0 0x7f45d80c5f50 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f45c80228a0 tx=0x7f45c80234b0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.830+0000 7f45dd600640 1 -- 172.21.15.113:0/2408760524 >> 172.21.15.113:0/2408760524 conn(0x7f45d8066db0 msgr2=0x7f45d814a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.1 2024-03-04T07:34:06.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f1c7aa00640 1 -- 172.21.15.113:0/2250061837 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1c74146790 con 0x7f1c740c0ed0 2024-03-04T07:34:06.837 DEBUG:teuthology.parallel:result is None 2024-03-04T07:34:06.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f45dd600640 1 -- 172.21.15.113:0/2408760524 shutdown_connections 2024-03-04T07:34:06.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f45dd600640 1 -- 172.21.15.113:0/2408760524 wait complete. 2024-03-04T07:34:06.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f1c71600640 1 -- 172.21.15.113:0/2250061837 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c6c05a3d0 con 0x7f1c740c0ed0 2024-03-04T07:34:06.838 INFO:teuthology.orchestra.run.smithi113.stdout:60129542152 2024-03-04T07:34:06.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 -- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f820c066f90 msgr2=0x7f820c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f820c066f90 0x7f820c069450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82180ab000 tx=0x7f8218001350 comp rx=0 tx=0).stop 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 -- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f822c15edb0 msgr2=0x7f822c154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f822c15edb0 0x7f822c154330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f822c0cdbd0 tx=0x7f822400f1b0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 -- 172.21.15.113:0/1491599068 shutdown_connections 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f822c15edb0 0x7f822c154330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f822c1588d0 0x7f822c1f8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f820c066f90 0x7f820c069450 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 --2- 172.21.15.113:0/1491599068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8210004df0 0x7f822c15f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 -- 172.21.15.113:0/1491599068 >> 172.21.15.113:0/1491599068 conn(0x7f822c0671b0 msgr2=0x7f822c0cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 -- 172.21.15.113:0/1491599068 shutdown_connections 2024-03-04T07:34:06.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.834+0000 7f8230a00640 1 -- 172.21.15.113:0/1491599068 wait complete. 2024-03-04T07:34:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 -- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1c54066f60 msgr2=0x7f1c54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1c54066f60 0x7f1c54069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c600b2000 tx=0x7f1c6000a040 comp rx=0 tx=0).stop 2024-03-04T07:34:06.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 -- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c740c0ed0 msgr2=0x7f1c741fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c740c0ed0 0x7f1c741fb0a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c00c7f0 tx=0x7f1c6c00ccb0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 -- 172.21.15.113:0/2250061837 shutdown_connections 2024-03-04T07:34:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1c74156720 0x7f1c741e8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1c54066f60 0x7f1c54069420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c74150e70 0x7f1c74160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 --2- 172.21.15.113:0/2250061837 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1c740c0ed0 0x7f1c741fb0a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 -- 172.21.15.113:0/2250061837 >> 172.21.15.113:0/2250061837 conn(0x7f1c7406e2c0 msgr2=0x7f1c740c1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 -- 172.21.15.113:0/2250061837 shutdown_connections 2024-03-04T07:34:06.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.838+0000 7f1c7aa00640 1 -- 172.21.15.113:0/2250061837 wait complete. 2024-03-04T07:34:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.846+0000 7facb7e00640 1 -- 172.21.15.113:0/44503465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7faca0005590 con 0x7faccc154c40 2024-03-04T07:34:06.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.4 2024-03-04T07:34:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.846+0000 7facc2a00640 1 -- 172.21.15.113:0/44503465 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7facc4018ce0 con 0x7faccc154c40 2024-03-04T07:34:06.850 DEBUG:teuthology.parallel:result is None 2024-03-04T07:34:06.850 INFO:teuthology.orchestra.run.smithi113.stdout:60129542151 2024-03-04T07:34:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.846+0000 7facb7e00640 1 -- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facac066fb0 msgr2=0x7facac069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.846+0000 7facb7e00640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facac066fb0 0x7facac069470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7facbc009b60 tx=0x7facbc009190 comp rx=0 tx=0).stop 2024-03-04T07:34:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.846+0000 7facb7e00640 1 -- 172.21.15.113:0/44503465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc154c40 msgr2=0x7faccc148840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.846+0000 7facb7e00640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc154c40 0x7faccc148840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faccc15e080 tx=0x7facc401bf20 comp rx=0 tx=0).stop 2024-03-04T07:34:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.850+0000 7facb7e00640 1 -- 172.21.15.113:0/44503465 shutdown_connections 2024-03-04T07:34:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.850+0000 7facb7e00640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facac066fb0 0x7facac069470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.850+0000 7facb7e00640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc154c40 0x7faccc148840 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.850+0000 7facb7e00640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faccc0d0030 0x7faccc1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.850+0000 7facb7e00640 1 --2- 172.21.15.113:0/44503465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faccc0cc2d0 0x7faccc15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.850+0000 7facb7e00640 1 -- 172.21.15.113:0/44503465 >> 172.21.15.113:0/44503465 conn(0x7faccc06d3e0 msgr2=0x7faccc0ceb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.850+0000 7facb7e00640 1 -- 172.21.15.113:0/44503465 shutdown_connections 2024-03-04T07:34:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:06.850+0000 7facb7e00640 1 -- 172.21.15.113:0/44503465 wait complete. 2024-03-04T07:34:06.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.5 2024-03-04T07:34:06.854 DEBUG:teuthology.parallel:result is None 2024-03-04T07:34:06.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.3 2024-03-04T07:34:06.866 DEBUG:teuthology.parallel:result is None 2024-03-04T07:34:06.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-03-04T07:34:06.873 DEBUG:teuthology.parallel:result is None 2024-03-04T07:34:06.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.7 2024-03-04T07:34:06.875 DEBUG:teuthology.parallel:result is None 2024-03-04T07:34:06.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.6 2024-03-04T07:34:06.876 DEBUG:teuthology.parallel:result is None 2024-03-04T07:34:06.876 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-04T07:34:06.876 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T07:34:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b37400640 1 Processor -- start 2024-03-04T07:34:07.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b37400640 1 -- start start 2024-03-04T07:34:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b37400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b3014e4c0 0x7f9b30072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b37400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b300c22e0 0x7f9b300731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b37400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9b300c2d80 0x7f9b3015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b37400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3015ad60 con 0x7f9b300c22e0 2024-03-04T07:34:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b37400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b300c27f0 con 0x7f9b300c2d80 2024-03-04T07:34:07.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b37400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b300c2a90 con 0x7f9b3014e4c0 2024-03-04T07:34:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b27e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b300c22e0 0x7f9b300731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b27e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b300c22e0 0x7f9b300731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57488/0 (socket says 172.21.15.113:57488) 2024-03-04T07:34:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b27e00640 1 -- 172.21.15.113:0/3999953313 learned_addr learned my addr 172.21.15.113:0/3999953313 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b34a00640 1 --2- 172.21.15.113:0/3999953313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b3014e4c0 0x7f9b30072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b35400640 1 --2- 172.21.15.113:0/3999953313 >> v2:172.21.15.136:3300/0 conn(0x7f9b300c2d80 0x7f9b3015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b27e00640 1 -- 172.21.15.113:0/3999953313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b3014e4c0 msgr2=0x7f9b30072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:07.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b27e00640 1 --2- 172.21.15.113:0/3999953313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b3014e4c0 0x7f9b30072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b27e00640 1 -- 172.21.15.113:0/3999953313 >> v2:172.21.15.136:3300/0 conn(0x7f9b300c2d80 msgr2=0x7f9b3015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:07.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b27e00640 1 --2- 172.21.15.113:0/3999953313 >> v2:172.21.15.136:3300/0 conn(0x7f9b300c2d80 0x7f9b3015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.054+0000 7f9b27e00640 1 -- 172.21.15.113:0/3999953313 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b30070c60 con 0x7f9b300c22e0 2024-03-04T07:34:07.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b27e00640 1 --2- 172.21.15.113:0/3999953313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b300c22e0 0x7f9b300731c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9b300c2700 tx=0x7f9b1801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fcdf8cdf7e8383 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:07.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b2fe00640 1 -- 172.21.15.113:0/3999953313 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b180b5070 con 0x7f9b300c22e0 2024-03-04T07:34:07.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b2fe00640 1 -- 172.21.15.113:0/3999953313 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b180a89d0 con 0x7f9b300c22e0 2024-03-04T07:34:07.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b2fe00640 1 -- 172.21.15.113:0/3999953313 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b180a8da0 con 0x7f9b300c22e0 2024-03-04T07:34:07.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b37400640 1 -- 172.21.15.113:0/3999953313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b300c22e0 msgr2=0x7f9b300731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b37400640 1 --2- 172.21.15.113:0/3999953313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b300c22e0 0x7f9b300731c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9b300c2700 tx=0x7f9b1801ed10 comp rx=0 tx=0).stop 2024-03-04T07:34:07.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b37400640 1 -- 172.21.15.113:0/3999953313 shutdown_connections 2024-03-04T07:34:07.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b37400640 1 --2- 172.21.15.113:0/3999953313 >> v2:172.21.15.136:3300/0 conn(0x7f9b300c2d80 0x7f9b3015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b37400640 1 --2- 172.21.15.113:0/3999953313 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b300c22e0 0x7f9b300731c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b37400640 1 --2- 172.21.15.113:0/3999953313 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b3014e4c0 0x7f9b30072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b37400640 1 -- 172.21.15.113:0/3999953313 >> 172.21.15.113:0/3999953313 conn(0x7f9b3006e820 msgr2=0x7f9b3006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:07.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b37400640 1 -- 172.21.15.113:0/3999953313 shutdown_connections 2024-03-04T07:34:07.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.058+0000 7f9b37400640 1 -- 172.21.15.113:0/3999953313 wait complete. 2024-03-04T07:34:07.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b37400640 1 Processor -- start 2024-03-04T07:34:07.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b37400640 1 -- start start 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b37400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b300c22e0 0x7f9b30154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b37400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b300c2d80 0x7f9b300b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b37400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b3014e4c0 0x7f9b30145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b37400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b30071500 con 0x7f9b3014e4c0 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b37400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b30071220 con 0x7f9b300c2d80 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b37400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3014e8a0 con 0x7f9b300c22e0 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b35400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b3014e4c0 0x7f9b30145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b35400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b3014e4c0 0x7f9b30145d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57500/0 (socket says 172.21.15.113:57500) 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b35400640 1 -- 172.21.15.113:0/1074548761 learned_addr learned my addr 172.21.15.113:0/1074548761 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b34a00640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b300c22e0 0x7f9b30154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b27e00640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b300c2d80 0x7f9b300b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b35400640 1 -- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b300c22e0 msgr2=0x7f9b30154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b35400640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b300c22e0 0x7f9b30154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b35400640 1 -- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b300c2d80 msgr2=0x7f9b300b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b35400640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b300c2d80 0x7f9b300b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b35400640 1 -- 172.21.15.113:0/1074548761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b30154d10 con 0x7f9b3014e4c0 2024-03-04T07:34:07.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b35400640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b3014e4c0 0x7f9b30145d60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9b28007e60 tx=0x7f9b2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:07.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b2d600640 1 -- 172.21.15.113:0/1074548761 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28014620 con 0x7f9b3014e4c0 2024-03-04T07:34:07.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b2d600640 1 -- 172.21.15.113:0/1074548761 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b28019070 con 0x7f9b3014e4c0 2024-03-04T07:34:07.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b37400640 1 -- 172.21.15.113:0/1074548761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b300af470 con 0x7f9b3014e4c0 2024-03-04T07:34:07.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b2d600640 1 -- 172.21.15.113:0/1074548761 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b28014a00 con 0x7f9b3014e4c0 2024-03-04T07:34:07.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.062+0000 7f9b37400640 1 -- 172.21.15.113:0/1074548761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b300b11c0 con 0x7f9b3014e4c0 2024-03-04T07:34:07.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.066+0000 7f9b37400640 1 -- 172.21.15.113:0/1074548761 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b30075c00 con 0x7f9b3014e4c0 2024-03-04T07:34:07.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.066+0000 7f9b2d600640 1 -- 172.21.15.113:0/1074548761 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b28014ba0 con 0x7f9b3014e4c0 2024-03-04T07:34:07.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.066+0000 7f9b2d600640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b0c066f60 0x7f9b0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.066+0000 7f9b2d600640 1 -- 172.21.15.113:0/1074548761 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b280857e0 con 0x7f9b3014e4c0 2024-03-04T07:34:07.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.066+0000 7f9b34a00640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b0c066f60 0x7f9b0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.066+0000 7f9b34a00640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b0c066f60 0x7f9b0c069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b20007940 tx=0x7f9b20008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.070+0000 7f9b2d600640 1 -- 172.21.15.113:0/1074548761 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9b280561a0 con 0x7f9b3014e4c0 2024-03-04T07:34:07.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.222+0000 7f9b37400640 1 -- 172.21.15.113:0/1074548761 --> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b300b1c20 con 0x7f9b0c066f60 2024-03-04T07:34:07.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.238+0000 7f9b2d600640 1 -- 172.21.15.113:0/1074548761 <== mgr.4103 v2:172.21.15.136:6832/3353359815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251835 (secure 0 0 0) 0x7f9b20007590 con 0x7f9b0c066f60 2024-03-04T07:34:07.245 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:07.246 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-03-04T07:34:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 -- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b0c066f60 msgr2=0x7f9b0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b0c066f60 0x7f9b0c069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b20007940 tx=0x7f9b20008040 comp rx=0 tx=0).stop 2024-03-04T07:34:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 -- 172.21.15.113:0/1074548761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b3014e4c0 msgr2=0x7f9b30145d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b3014e4c0 0x7f9b30145d60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9b28007e60 tx=0x7f9b2800e4b0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 -- 172.21.15.113:0/1074548761 shutdown_connections 2024-03-04T07:34:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b0c066f60 0x7f9b0c069420 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b3014e4c0 0x7f9b30145d60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b300c2d80 0x7f9b300b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 --2- 172.21.15.113:0/1074548761 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b300c22e0 0x7f9b30154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 -- 172.21.15.113:0/1074548761 >> 172.21.15.113:0/1074548761 conn(0x7f9b3006e820 msgr2=0x7f9b300c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 -- 172.21.15.113:0/1074548761 shutdown_connections 2024-03-04T07:34:07.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.242+0000 7f9b37400640 1 -- 172.21.15.113:0/1074548761 wait complete. 2024-03-04T07:34:07.265 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-03-04T07:34:05.435938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223144,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751576,"statfs":{"total":773094113280,"available":772865613824,"internally_reserved":0,"allocated":2924544,"data_stored":1773788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.914973"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645580+0000","last_change":"2024-03-04T07:33:56.559432+0000","last_active":"2024-03-04T07:33:58.645580+0000","last_peered":"2024-03-04T07:33:58.645580+0000","last_clean":"2024-03-04T07:33:58.645580+0000","last_became_active":"2024-03-04T07:33:56.559031+0000","last_became_peered":"2024-03-04T07:33:56.559031+0000","last_unstale":"2024-03-04T07:33:58.645580+0000","last_undegraded":"2024-03-04T07:33:58.645580+0000","last_fullsized":"2024-03-04T07:33:58.645580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:43:55.037090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750282+0000","last_change":"2024-03-04T07:33:56.546822+0000","last_active":"2024-03-04T07:33:57.750282+0000","last_peered":"2024-03-04T07:33:57.750282+0000","last_clean":"2024-03-04T07:33:57.750282+0000","last_became_active":"2024-03-04T07:33:56.546547+0000","last_became_peered":"2024-03-04T07:33:56.546547+0000","last_unstale":"2024-03-04T07:33:57.750282+0000","last_undegraded":"2024-03-04T07:33:57.750282+0000","last_fullsized":"2024-03-04T07:33:57.750282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:06.332337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859174+0000","last_change":"2024-03-04T07:33:56.530803+0000","last_active":"2024-03-04T07:33:57.859174+0000","last_peered":"2024-03-04T07:33:57.859174+0000","last_clean":"2024-03-04T07:33:57.859174+0000","last_became_active":"2024-03-04T07:33:56.530488+0000","last_became_peered":"2024-03-04T07:33:56.530488+0000","last_unstale":"2024-03-04T07:33:57.859174+0000","last_undegraded":"2024-03-04T07:33:57.859174+0000","last_fullsized":"2024-03-04T07:33:57.859174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:00:42.147951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642296+0000","last_change":"2024-03-04T07:33:56.558669+0000","last_active":"2024-03-04T07:33:58.642296+0000","last_peered":"2024-03-04T07:33:58.642296+0000","last_clean":"2024-03-04T07:33:58.642296+0000","last_became_active":"2024-03-04T07:33:56.558350+0000","last_became_peered":"2024-03-04T07:33:56.558350+0000","last_unstale":"2024-03-04T07:33:58.642296+0000","last_undegraded":"2024-03-04T07:33:58.642296+0000","last_fullsized":"2024-03-04T07:33:58.642296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:35:53.460563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644296+0000","last_change":"2024-03-04T07:33:56.540760+0000","last_active":"2024-03-04T07:33:58.644296+0000","last_peered":"2024-03-04T07:33:58.644296+0000","last_clean":"2024-03-04T07:33:58.644296+0000","last_became_active":"2024-03-04T07:33:56.540351+0000","last_became_peered":"2024-03-04T07:33:56.540351+0000","last_unstale":"2024-03-04T07:33:58.644296+0000","last_undegraded":"2024-03-04T07:33:58.644296+0000","last_fullsized":"2024-03-04T07:33:58.644296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:24:21.708228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641368+0000","last_change":"2024-03-04T07:33:56.558333+0000","last_active":"2024-03-04T07:33:58.641368+0000","last_peered":"2024-03-04T07:33:58.641368+0000","last_clean":"2024-03-04T07:33:58.641368+0000","last_became_active":"2024-03-04T07:33:56.558046+0000","last_became_peered":"2024-03-04T07:33:56.558046+0000","last_unstale":"2024-03-04T07:33:58.641368+0000","last_undegraded":"2024-03-04T07:33:58.641368+0000","last_fullsized":"2024-03-04T07:33:58.641368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:30:46.375184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643038+0000","last_change":"2024-03-04T07:33:56.543476+0000","last_active":"2024-03-04T07:33:58.643038+0000","last_peered":"2024-03-04T07:33:58.643038+0000","last_clean":"2024-03-04T07:33:58.643038+0000","last_became_active":"2024-03-04T07:33:56.543176+0000","last_became_peered":"2024-03-04T07:33:56.543176+0000","last_unstale":"2024-03-04T07:33:58.643038+0000","last_undegraded":"2024-03-04T07:33:58.643038+0000","last_fullsized":"2024-03-04T07:33:58.643038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:26:34.736870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523950+0000","last_change":"2024-03-04T07:33:56.559488+0000","last_active":"2024-03-04T07:33:57.523950+0000","last_peered":"2024-03-04T07:33:57.523950+0000","last_clean":"2024-03-04T07:33:57.523950+0000","last_became_active":"2024-03-04T07:33:56.559172+0000","last_became_peered":"2024-03-04T07:33:56.559172+0000","last_unstale":"2024-03-04T07:33:57.523950+0000","last_undegraded":"2024-03-04T07:33:57.523950+0000","last_fullsized":"2024-03-04T07:33:57.523950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:53.461697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645566+0000","last_change":"2024-03-04T07:33:56.560587+0000","last_active":"2024-03-04T07:33:58.645566+0000","last_peered":"2024-03-04T07:33:58.645566+0000","last_clean":"2024-03-04T07:33:58.645566+0000","last_became_active":"2024-03-04T07:33:56.560306+0000","last_became_peered":"2024-03-04T07:33:56.560306+0000","last_unstale":"2024-03-04T07:33:58.645566+0000","last_undegraded":"2024-03-04T07:33:58.645566+0000","last_fullsized":"2024-03-04T07:33:58.645566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:06:17.011327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641979+0000","last_change":"2024-03-04T07:33:56.560852+0000","last_active":"2024-03-04T07:33:58.641979+0000","last_peered":"2024-03-04T07:33:58.641979+0000","last_clean":"2024-03-04T07:33:58.641979+0000","last_became_active":"2024-03-04T07:33:56.560592+0000","last_became_peered":"2024-03-04T07:33:56.560592+0000","last_unstale":"2024-03-04T07:33:58.641979+0000","last_undegraded":"2024-03-04T07:33:58.641979+0000","last_fullsized":"2024-03-04T07:33:58.641979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:39:31.496432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641631+0000","last_change":"2024-03-04T07:33:56.561040+0000","last_active":"2024-03-04T07:33:58.641631+0000","last_peered":"2024-03-04T07:33:58.641631+0000","last_clean":"2024-03-04T07:33:58.641631+0000","last_became_active":"2024-03-04T07:33:56.560719+0000","last_became_peered":"2024-03-04T07:33:56.560719+0000","last_unstale":"2024-03-04T07:33:58.641631+0000","last_undegraded":"2024-03-04T07:33:58.641631+0000","last_fullsized":"2024-03-04T07:33:58.641631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:14:13.007587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526511+0000","last_change":"2024-03-04T07:33:56.550648+0000","last_active":"2024-03-04T07:33:57.526511+0000","last_peered":"2024-03-04T07:33:57.526511+0000","last_clean":"2024-03-04T07:33:57.526511+0000","last_became_active":"2024-03-04T07:33:56.549568+0000","last_became_peered":"2024-03-04T07:33:56.549568+0000","last_unstale":"2024-03-04T07:33:57.526511+0000","last_undegraded":"2024-03-04T07:33:57.526511+0000","last_fullsized":"2024-03-04T07:33:57.526511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:34:35.299480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523777+0000","last_change":"2024-03-04T07:33:56.558880+0000","last_active":"2024-03-04T07:33:57.523777+0000","last_peered":"2024-03-04T07:33:57.523777+0000","last_clean":"2024-03-04T07:33:57.523777+0000","last_became_active":"2024-03-04T07:33:56.558567+0000","last_became_peered":"2024-03-04T07:33:56.558567+0000","last_unstale":"2024-03-04T07:33:57.523777+0000","last_undegraded":"2024-03-04T07:33:57.523777+0000","last_fullsized":"2024-03-04T07:33:57.523777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:16.354787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.858969+0000","last_change":"2024-03-04T07:33:56.544222+0000","last_active":"2024-03-04T07:33:57.858969+0000","last_peered":"2024-03-04T07:33:57.858969+0000","last_clean":"2024-03-04T07:33:57.858969+0000","last_became_active":"2024-03-04T07:33:56.543936+0000","last_became_peered":"2024-03-04T07:33:56.543936+0000","last_unstale":"2024-03-04T07:33:57.858969+0000","last_undegraded":"2024-03-04T07:33:57.858969+0000","last_fullsized":"2024-03-04T07:33:57.858969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:42.788232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749980+0000","last_change":"2024-03-04T07:33:56.545195+0000","last_active":"2024-03-04T07:33:57.749980+0000","last_peered":"2024-03-04T07:33:57.749980+0000","last_clean":"2024-03-04T07:33:57.749980+0000","last_became_active":"2024-03-04T07:33:56.544837+0000","last_became_peered":"2024-03-04T07:33:56.544837+0000","last_unstale":"2024-03-04T07:33:57.749980+0000","last_undegraded":"2024-03-04T07:33:57.749980+0000","last_fullsized":"2024-03-04T07:33:57.749980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:24:59.643746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641197+0000","last_change":"2024-03-04T07:33:56.558398+0000","last_active":"2024-03-04T07:33:58.641197+0000","last_peered":"2024-03-04T07:33:58.641197+0000","last_clean":"2024-03-04T07:33:58.641197+0000","last_became_active":"2024-03-04T07:33:56.557962+0000","last_became_peered":"2024-03-04T07:33:56.557962+0000","last_unstale":"2024-03-04T07:33:58.641197+0000","last_undegraded":"2024-03-04T07:33:58.641197+0000","last_fullsized":"2024-03-04T07:33:58.641197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:21:54.588819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643019+0000","last_change":"2024-03-04T07:33:56.559597+0000","last_active":"2024-03-04T07:33:58.643019+0000","last_peered":"2024-03-04T07:33:58.643019+0000","last_clean":"2024-03-04T07:33:58.643019+0000","last_became_active":"2024-03-04T07:33:56.558719+0000","last_became_peered":"2024-03-04T07:33:56.558719+0000","last_unstale":"2024-03-04T07:33:58.643019+0000","last_undegraded":"2024-03-04T07:33:58.643019+0000","last_fullsized":"2024-03-04T07:33:58.643019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:43:56.063344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522942+0000","last_change":"2024-03-04T07:33:56.528479+0000","last_active":"2024-03-04T07:33:57.522942+0000","last_peered":"2024-03-04T07:33:57.522942+0000","last_clean":"2024-03-04T07:33:57.522942+0000","last_became_active":"2024-03-04T07:33:56.528213+0000","last_became_peered":"2024-03-04T07:33:56.528213+0000","last_unstale":"2024-03-04T07:33:57.522942+0000","last_undegraded":"2024-03-04T07:33:57.522942+0000","last_fullsized":"2024-03-04T07:33:57.522942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:38:48.062007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644805+0000","last_change":"2024-03-04T07:33:56.545890+0000","last_active":"2024-03-04T07:33:58.644805+0000","last_peered":"2024-03-04T07:33:58.644805+0000","last_clean":"2024-03-04T07:33:58.644805+0000","last_became_active":"2024-03-04T07:33:56.545467+0000","last_became_peered":"2024-03-04T07:33:56.545467+0000","last_unstale":"2024-03-04T07:33:58.644805+0000","last_undegraded":"2024-03-04T07:33:58.644805+0000","last_fullsized":"2024-03-04T07:33:58.644805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:43:56.049152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641702+0000","last_change":"2024-03-04T07:33:56.549059+0000","last_active":"2024-03-04T07:33:58.641702+0000","last_peered":"2024-03-04T07:33:58.641702+0000","last_clean":"2024-03-04T07:33:58.641702+0000","last_became_active":"2024-03-04T07:33:56.548783+0000","last_became_peered":"2024-03-04T07:33:56.548783+0000","last_unstale":"2024-03-04T07:33:58.641702+0000","last_undegraded":"2024-03-04T07:33:58.641702+0000","last_fullsized":"2024-03-04T07:33:58.641702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:02.330428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859045+0000","last_change":"2024-03-04T07:33:56.540558+0000","last_active":"2024-03-04T07:33:57.859045+0000","last_peered":"2024-03-04T07:33:57.859045+0000","last_clean":"2024-03-04T07:33:57.859045+0000","last_became_active":"2024-03-04T07:33:56.539403+0000","last_became_peered":"2024-03-04T07:33:56.539403+0000","last_unstale":"2024-03-04T07:33:57.859045+0000","last_undegraded":"2024-03-04T07:33:57.859045+0000","last_fullsized":"2024-03-04T07:33:57.859045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:28:19.413591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641763+0000","last_change":"2024-03-04T07:33:56.554982+0000","last_active":"2024-03-04T07:33:58.641763+0000","last_peered":"2024-03-04T07:33:58.641763+0000","last_clean":"2024-03-04T07:33:58.641763+0000","last_became_active":"2024-03-04T07:33:56.554470+0000","last_became_peered":"2024-03-04T07:33:56.554470+0000","last_unstale":"2024-03-04T07:33:58.641763+0000","last_undegraded":"2024-03-04T07:33:58.641763+0000","last_fullsized":"2024-03-04T07:33:58.641763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:32.767329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523496+0000","last_change":"2024-03-04T07:33:56.545465+0000","last_active":"2024-03-04T07:33:57.523496+0000","last_peered":"2024-03-04T07:33:57.523496+0000","last_clean":"2024-03-04T07:33:57.523496+0000","last_became_active":"2024-03-04T07:33:56.545270+0000","last_became_peered":"2024-03-04T07:33:56.545270+0000","last_unstale":"2024-03-04T07:33:57.523496+0000","last_undegraded":"2024-03-04T07:33:57.523496+0000","last_fullsized":"2024-03-04T07:33:57.523496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:08:45.244868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643452+0000","last_change":"2024-03-04T07:33:56.555428+0000","last_active":"2024-03-04T07:33:58.643452+0000","last_peered":"2024-03-04T07:33:58.643452+0000","last_clean":"2024-03-04T07:33:58.643452+0000","last_became_active":"2024-03-04T07:33:56.554942+0000","last_became_peered":"2024-03-04T07:33:56.554942+0000","last_unstale":"2024-03-04T07:33:58.643452+0000","last_undegraded":"2024-03-04T07:33:58.643452+0000","last_fullsized":"2024-03-04T07:33:58.643452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:04.724517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523610+0000","last_change":"2024-03-04T07:33:56.549563+0000","last_active":"2024-03-04T07:33:57.523610+0000","last_peered":"2024-03-04T07:33:57.523610+0000","last_clean":"2024-03-04T07:33:57.523610+0000","last_became_active":"2024-03-04T07:33:56.548790+0000","last_became_peered":"2024-03-04T07:33:56.548790+0000","last_unstale":"2024-03-04T07:33:57.523610+0000","last_undegraded":"2024-03-04T07:33:57.523610+0000","last_fullsized":"2024-03-04T07:33:57.523610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:30:53.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642385+0000","last_change":"2024-03-04T07:33:56.553791+0000","last_active":"2024-03-04T07:33:58.642385+0000","last_peered":"2024-03-04T07:33:58.642385+0000","last_clean":"2024-03-04T07:33:58.642385+0000","last_became_active":"2024-03-04T07:33:56.553012+0000","last_became_peered":"2024-03-04T07:33:56.553012+0000","last_unstale":"2024-03-04T07:33:58.642385+0000","last_undegraded":"2024-03-04T07:33:58.642385+0000","last_fullsized":"2024-03-04T07:33:58.642385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:38.440189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750163+0000","last_change":"2024-03-04T07:33:56.551341+0000","last_active":"2024-03-04T07:33:57.750163+0000","last_peered":"2024-03-04T07:33:57.750163+0000","last_clean":"2024-03-04T07:33:57.750163+0000","last_became_active":"2024-03-04T07:33:56.550616+0000","last_became_peered":"2024-03-04T07:33:56.550616+0000","last_unstale":"2024-03-04T07:33:57.750163+0000","last_undegraded":"2024-03-04T07:33:57.750163+0000","last_fullsized":"2024-03-04T07:33:57.750163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:35:45.801510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523840+0000","last_change":"2024-03-04T07:33:56.543903+0000","last_active":"2024-03-04T07:33:57.523840+0000","last_peered":"2024-03-04T07:33:57.523840+0000","last_clean":"2024-03-04T07:33:57.523840+0000","last_became_active":"2024-03-04T07:33:56.543632+0000","last_became_peered":"2024-03-04T07:33:56.543632+0000","last_unstale":"2024-03-04T07:33:57.523840+0000","last_undegraded":"2024-03-04T07:33:57.523840+0000","last_fullsized":"2024-03-04T07:33:57.523840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:26:10.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859232+0000","last_change":"2024-03-04T07:33:56.547091+0000","last_active":"2024-03-04T07:33:57.859232+0000","last_peered":"2024-03-04T07:33:57.859232+0000","last_clean":"2024-03-04T07:33:57.859232+0000","last_became_active":"2024-03-04T07:33:56.546652+0000","last_became_peered":"2024-03-04T07:33:56.546652+0000","last_unstale":"2024-03-04T07:33:57.859232+0000","last_undegraded":"2024-03-04T07:33:57.859232+0000","last_fullsized":"2024-03-04T07:33:57.859232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:26:03.091219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642136+0000","last_change":"2024-03-04T07:33:56.552493+0000","last_active":"2024-03-04T07:33:58.642136+0000","last_peered":"2024-03-04T07:33:58.642136+0000","last_clean":"2024-03-04T07:33:58.642136+0000","last_became_active":"2024-03-04T07:33:56.552189+0000","last_became_peered":"2024-03-04T07:33:56.552189+0000","last_unstale":"2024-03-04T07:33:58.642136+0000","last_undegraded":"2024-03-04T07:33:58.642136+0000","last_fullsized":"2024-03-04T07:33:58.642136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:10.313129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522710+0000","last_change":"2024-03-04T07:33:56.549661+0000","last_active":"2024-03-04T07:33:57.522710+0000","last_peered":"2024-03-04T07:33:57.522710+0000","last_clean":"2024-03-04T07:33:57.522710+0000","last_became_active":"2024-03-04T07:33:56.549351+0000","last_became_peered":"2024-03-04T07:33:56.549351+0000","last_unstale":"2024-03-04T07:33:57.522710+0000","last_undegraded":"2024-03-04T07:33:57.522710+0000","last_fullsized":"2024-03-04T07:33:57.522710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:55.684272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640651+0000","last_change":"2024-03-04T07:33:56.533132+0000","last_active":"2024-03-04T07:33:58.640651+0000","last_peered":"2024-03-04T07:33:58.640651+0000","last_clean":"2024-03-04T07:33:58.640651+0000","last_became_active":"2024-03-04T07:33:56.532251+0000","last_became_peered":"2024-03-04T07:33:56.532251+0000","last_unstale":"2024-03-04T07:33:58.640651+0000","last_undegraded":"2024-03-04T07:33:58.640651+0000","last_fullsized":"2024-03-04T07:33:58.640651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:17:47.018585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.525697+0000","last_change":"2024-03-04T07:33:56.543382+0000","last_active":"2024-03-04T07:33:57.525697+0000","last_peered":"2024-03-04T07:33:57.525697+0000","last_clean":"2024-03-04T07:33:57.525697+0000","last_became_active":"2024-03-04T07:33:56.542937+0000","last_became_peered":"2024-03-04T07:33:56.542937+0000","last_unstale":"2024-03-04T07:33:57.525697+0000","last_undegraded":"2024-03-04T07:33:57.525697+0000","last_fullsized":"2024-03-04T07:33:57.525697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:18:28.332590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.918391+0000","last_change":"2024-03-04T07:33:55.566130+0000","last_active":"2024-03-04T07:33:57.918391+0000","last_peered":"2024-03-04T07:33:57.918391+0000","last_clean":"2024-03-04T07:33:57.918391+0000","last_became_active":"2024-03-04T07:33:55.565894+0000","last_became_peered":"2024-03-04T07:33:55.565894+0000","last_unstale":"2024-03-04T07:33:57.918391+0000","last_undegraded":"2024-03-04T07:33:57.918391+0000","last_fullsized":"2024-03-04T07:33:57.918391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:36.530949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859235+0000","last_change":"2024-03-04T07:33:56.540113+0000","last_active":"2024-03-04T07:33:57.859235+0000","last_peered":"2024-03-04T07:33:57.859235+0000","last_clean":"2024-03-04T07:33:57.859235+0000","last_became_active":"2024-03-04T07:33:56.539738+0000","last_became_peered":"2024-03-04T07:33:56.539738+0000","last_unstale":"2024-03-04T07:33:57.859235+0000","last_undegraded":"2024-03-04T07:33:57.859235+0000","last_fullsized":"2024-03-04T07:33:57.859235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:44.949754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641120+0000","last_change":"2024-03-04T07:33:55.554956+0000","last_active":"2024-03-04T07:33:58.641120+0000","last_peered":"2024-03-04T07:33:58.641120+0000","last_clean":"2024-03-04T07:33:58.641120+0000","last_became_active":"2024-03-04T07:33:55.554151+0000","last_became_peered":"2024-03-04T07:33:55.554151+0000","last_unstale":"2024-03-04T07:33:58.641120+0000","last_undegraded":"2024-03-04T07:33:58.641120+0000","last_fullsized":"2024-03-04T07:33:58.641120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:06:19.621510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523529+0000","last_change":"2024-03-04T07:33:55.542200+0000","last_active":"2024-03-04T07:33:57.523529+0000","last_peered":"2024-03-04T07:33:57.523529+0000","last_clean":"2024-03-04T07:33:57.523529+0000","last_became_active":"2024-03-04T07:33:55.541902+0000","last_became_peered":"2024-03-04T07:33:55.541902+0000","last_unstale":"2024-03-04T07:33:57.523529+0000","last_undegraded":"2024-03-04T07:33:57.523529+0000","last_fullsized":"2024-03-04T07:33:57.523529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:55:50.251288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523733+0000","last_change":"2024-03-04T07:33:56.548380+0000","last_active":"2024-03-04T07:33:57.523733+0000","last_peered":"2024-03-04T07:33:57.523733+0000","last_clean":"2024-03-04T07:33:57.523733+0000","last_became_active":"2024-03-04T07:33:56.546087+0000","last_became_peered":"2024-03-04T07:33:56.546087+0000","last_unstale":"2024-03-04T07:33:57.523733+0000","last_undegraded":"2024-03-04T07:33:57.523733+0000","last_fullsized":"2024-03-04T07:33:57.523733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:06:43.538687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643489+0000","last_change":"2024-03-04T07:33:55.568508+0000","last_active":"2024-03-04T07:33:58.643489+0000","last_peered":"2024-03-04T07:33:58.643489+0000","last_clean":"2024-03-04T07:33:58.643489+0000","last_became_active":"2024-03-04T07:33:55.566073+0000","last_became_peered":"2024-03-04T07:33:55.566073+0000","last_unstale":"2024-03-04T07:33:58.643489+0000","last_undegraded":"2024-03-04T07:33:58.643489+0000","last_fullsized":"2024-03-04T07:33:58.643489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:34:17.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749969+0000","last_change":"2024-03-04T07:33:56.552785+0000","last_active":"2024-03-04T07:33:57.749969+0000","last_peered":"2024-03-04T07:33:57.749969+0000","last_clean":"2024-03-04T07:33:57.749969+0000","last_became_active":"2024-03-04T07:33:56.552533+0000","last_became_peered":"2024-03-04T07:33:56.552533+0000","last_unstale":"2024-03-04T07:33:57.749969+0000","last_undegraded":"2024-03-04T07:33:57.749969+0000","last_fullsized":"2024-03-04T07:33:57.749969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:08.582237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642106+0000","last_change":"2024-03-04T07:33:55.559395+0000","last_active":"2024-03-04T07:33:58.642106+0000","last_peered":"2024-03-04T07:33:58.642106+0000","last_clean":"2024-03-04T07:33:58.642106+0000","last_became_active":"2024-03-04T07:33:55.558593+0000","last_became_peered":"2024-03-04T07:33:55.558593+0000","last_unstale":"2024-03-04T07:33:58.642106+0000","last_undegraded":"2024-03-04T07:33:58.642106+0000","last_fullsized":"2024-03-04T07:33:58.642106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:00:20.205445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642007+0000","last_change":"2024-03-04T07:33:56.553260+0000","last_active":"2024-03-04T07:33:58.642007+0000","last_peered":"2024-03-04T07:33:58.642007+0000","last_clean":"2024-03-04T07:33:58.642007+0000","last_became_active":"2024-03-04T07:33:56.552952+0000","last_became_peered":"2024-03-04T07:33:56.552952+0000","last_unstale":"2024-03-04T07:33:58.642007+0000","last_undegraded":"2024-03-04T07:33:58.642007+0000","last_fullsized":"2024-03-04T07:33:58.642007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:25:51.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642257+0000","last_change":"2024-03-04T07:33:55.568220+0000","last_active":"2024-03-04T07:33:58.642257+0000","last_peered":"2024-03-04T07:33:58.642257+0000","last_clean":"2024-03-04T07:33:58.642257+0000","last_became_active":"2024-03-04T07:33:55.567569+0000","last_became_peered":"2024-03-04T07:33:55.567569+0000","last_unstale":"2024-03-04T07:33:58.642257+0000","last_undegraded":"2024-03-04T07:33:58.642257+0000","last_fullsized":"2024-03-04T07:33:58.642257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:40:49.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859421+0000","last_change":"2024-03-04T07:33:56.559745+0000","last_active":"2024-03-04T07:33:57.859421+0000","last_peered":"2024-03-04T07:33:57.859421+0000","last_clean":"2024-03-04T07:33:57.859421+0000","last_became_active":"2024-03-04T07:33:56.559456+0000","last_became_peered":"2024-03-04T07:33:56.559456+0000","last_unstale":"2024-03-04T07:33:57.859421+0000","last_undegraded":"2024-03-04T07:33:57.859421+0000","last_fullsized":"2024-03-04T07:33:57.859421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:18:43.135906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.531554+0000","last_change":"2024-03-04T07:33:55.550625+0000","last_active":"2024-03-04T07:34:01.531554+0000","last_peered":"2024-03-04T07:34:01.531554+0000","last_clean":"2024-03-04T07:34:01.531554+0000","last_became_active":"2024-03-04T07:33:55.550327+0000","last_became_peered":"2024-03-04T07:33:55.550327+0000","last_unstale":"2024-03-04T07:34:01.531554+0000","last_undegraded":"2024-03-04T07:34:01.531554+0000","last_fullsized":"2024-03-04T07:34:01.531554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:20.282585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526558+0000","last_change":"2024-03-04T07:33:56.537570+0000","last_active":"2024-03-04T07:33:57.526558+0000","last_peered":"2024-03-04T07:33:57.526558+0000","last_clean":"2024-03-04T07:33:57.526558+0000","last_became_active":"2024-03-04T07:33:56.536870+0000","last_became_peered":"2024-03-04T07:33:56.536870+0000","last_unstale":"2024-03-04T07:33:57.526558+0000","last_undegraded":"2024-03-04T07:33:57.526558+0000","last_fullsized":"2024-03-04T07:33:57.526558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:59:23.807432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.536570+0000","last_change":"2024-03-04T07:33:55.538915+0000","last_active":"2024-03-04T07:34:01.536570+0000","last_peered":"2024-03-04T07:34:01.536570+0000","last_clean":"2024-03-04T07:34:01.536570+0000","last_became_active":"2024-03-04T07:33:55.536263+0000","last_became_peered":"2024-03-04T07:33:55.536263+0000","last_unstale":"2024-03-04T07:34:01.536570+0000","last_undegraded":"2024-03-04T07:34:01.536570+0000","last_fullsized":"2024-03-04T07:34:01.536570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:09.612704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643901+0000","last_change":"2024-03-04T07:33:56.551087+0000","last_active":"2024-03-04T07:33:58.643901+0000","last_peered":"2024-03-04T07:33:58.643901+0000","last_clean":"2024-03-04T07:33:58.643901+0000","last_became_active":"2024-03-04T07:33:56.550767+0000","last_became_peered":"2024-03-04T07:33:56.550767+0000","last_unstale":"2024-03-04T07:33:58.643901+0000","last_undegraded":"2024-03-04T07:33:58.643901+0000","last_fullsized":"2024-03-04T07:33:58.643901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:15.603800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.532771+0000","last_change":"2024-03-04T07:33:55.560382+0000","last_active":"2024-03-04T07:34:01.532771+0000","last_peered":"2024-03-04T07:34:01.532771+0000","last_clean":"2024-03-04T07:34:01.532771+0000","last_became_active":"2024-03-04T07:33:55.560078+0000","last_became_peered":"2024-03-04T07:33:55.560078+0000","last_unstale":"2024-03-04T07:34:01.532771+0000","last_undegraded":"2024-03-04T07:34:01.532771+0000","last_fullsized":"2024-03-04T07:34:01.532771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:41:10.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859054+0000","last_change":"2024-03-04T07:33:56.537920+0000","last_active":"2024-03-04T07:33:57.859054+0000","last_peered":"2024-03-04T07:33:57.859054+0000","last_clean":"2024-03-04T07:33:57.859054+0000","last_became_active":"2024-03-04T07:33:56.535960+0000","last_became_peered":"2024-03-04T07:33:56.535960+0000","last_unstale":"2024-03-04T07:33:57.859054+0000","last_undegraded":"2024-03-04T07:33:57.859054+0000","last_fullsized":"2024-03-04T07:33:57.859054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:37:09.344120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642382+0000","last_change":"2024-03-04T07:33:55.567682+0000","last_active":"2024-03-04T07:33:58.642382+0000","last_peered":"2024-03-04T07:33:58.642382+0000","last_clean":"2024-03-04T07:33:58.642382+0000","last_became_active":"2024-03-04T07:33:55.566915+0000","last_became_peered":"2024-03-04T07:33:55.566915+0000","last_unstale":"2024-03-04T07:33:58.642382+0000","last_undegraded":"2024-03-04T07:33:58.642382+0000","last_fullsized":"2024-03-04T07:33:58.642382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:33:47.010346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644757+0000","last_change":"2024-03-04T07:33:56.547846+0000","last_active":"2024-03-04T07:33:58.644757+0000","last_peered":"2024-03-04T07:33:58.644757+0000","last_clean":"2024-03-04T07:33:58.644757+0000","last_became_active":"2024-03-04T07:33:56.547459+0000","last_became_peered":"2024-03-04T07:33:56.547459+0000","last_unstale":"2024-03-04T07:33:58.644757+0000","last_undegraded":"2024-03-04T07:33:58.644757+0000","last_fullsized":"2024-03-04T07:33:58.644757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:23:57.623123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640665+0000","last_change":"2024-03-04T07:33:55.557041+0000","last_active":"2024-03-04T07:33:58.640665+0000","last_peered":"2024-03-04T07:33:58.640665+0000","last_clean":"2024-03-04T07:33:58.640665+0000","last_became_active":"2024-03-04T07:33:55.556416+0000","last_became_peered":"2024-03-04T07:33:55.556416+0000","last_unstale":"2024-03-04T07:33:58.640665+0000","last_undegraded":"2024-03-04T07:33:58.640665+0000","last_fullsized":"2024-03-04T07:33:58.640665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:31.472235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641078+0000","last_change":"2024-03-04T07:33:56.531986+0000","last_active":"2024-03-04T07:33:58.641078+0000","last_peered":"2024-03-04T07:33:58.641078+0000","last_clean":"2024-03-04T07:33:58.641078+0000","last_became_active":"2024-03-04T07:33:56.531565+0000","last_became_peered":"2024-03-04T07:33:56.531565+0000","last_unstale":"2024-03-04T07:33:58.641078+0000","last_undegraded":"2024-03-04T07:33:58.641078+0000","last_fullsized":"2024-03-04T07:33:58.641078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:42:35.040979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.533357+0000","last_change":"2024-03-04T07:33:55.538469+0000","last_active":"2024-03-04T07:34:01.533357+0000","last_peered":"2024-03-04T07:34:01.533357+0000","last_clean":"2024-03-04T07:34:01.533357+0000","last_became_active":"2024-03-04T07:33:55.538165+0000","last_became_peered":"2024-03-04T07:33:55.538165+0000","last_unstale":"2024-03-04T07:34:01.533357+0000","last_undegraded":"2024-03-04T07:34:01.533357+0000","last_fullsized":"2024-03-04T07:34:01.533357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:00:01.263768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643331+0000","last_change":"2024-03-04T07:33:56.547881+0000","last_active":"2024-03-04T07:33:58.643331+0000","last_peered":"2024-03-04T07:33:58.643331+0000","last_clean":"2024-03-04T07:33:58.643331+0000","last_became_active":"2024-03-04T07:33:56.547461+0000","last_became_peered":"2024-03-04T07:33:56.547461+0000","last_unstale":"2024-03-04T07:33:58.643331+0000","last_undegraded":"2024-03-04T07:33:58.643331+0000","last_fullsized":"2024-03-04T07:33:58.643331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:34:32.709430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526523+0000","last_change":"2024-03-04T07:33:55.544356+0000","last_active":"2024-03-04T07:33:57.526523+0000","last_peered":"2024-03-04T07:33:57.526523+0000","last_clean":"2024-03-04T07:33:57.526523+0000","last_became_active":"2024-03-04T07:33:55.541295+0000","last_became_peered":"2024-03-04T07:33:55.541295+0000","last_unstale":"2024-03-04T07:33:57.526523+0000","last_undegraded":"2024-03-04T07:33:57.526523+0000","last_fullsized":"2024-03-04T07:33:57.526523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:22.953982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645470+0000","last_change":"2024-03-04T07:33:56.549038+0000","last_active":"2024-03-04T07:33:58.645470+0000","last_peered":"2024-03-04T07:33:58.645470+0000","last_clean":"2024-03-04T07:33:58.645470+0000","last_became_active":"2024-03-04T07:33:56.548113+0000","last_became_peered":"2024-03-04T07:33:56.548113+0000","last_unstale":"2024-03-04T07:33:58.645470+0000","last_undegraded":"2024-03-04T07:33:58.645470+0000","last_fullsized":"2024-03-04T07:33:58.645470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:53:04.601075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.530724+0000","last_change":"2024-03-04T07:33:55.538871+0000","last_active":"2024-03-04T07:34:01.530724+0000","last_peered":"2024-03-04T07:34:01.530724+0000","last_clean":"2024-03-04T07:34:01.530724+0000","last_became_active":"2024-03-04T07:33:55.537221+0000","last_became_peered":"2024-03-04T07:33:55.537221+0000","last_unstale":"2024-03-04T07:34:01.530724+0000","last_undegraded":"2024-03-04T07:34:01.530724+0000","last_fullsized":"2024-03-04T07:34:01.530724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:02:34.533330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645792+0000","last_change":"2024-03-04T07:33:56.543760+0000","last_active":"2024-03-04T07:33:58.645792+0000","last_peered":"2024-03-04T07:33:58.645792+0000","last_clean":"2024-03-04T07:33:58.645792+0000","last_became_active":"2024-03-04T07:33:56.543354+0000","last_became_peered":"2024-03-04T07:33:56.543354+0000","last_unstale":"2024-03-04T07:33:58.645792+0000","last_undegraded":"2024-03-04T07:33:58.645792+0000","last_fullsized":"2024-03-04T07:33:58.645792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:19:37.564790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.913391+0000","last_change":"2024-03-04T07:33:55.552856+0000","last_active":"2024-03-04T07:33:57.913391+0000","last_peered":"2024-03-04T07:33:57.913391+0000","last_clean":"2024-03-04T07:33:57.913391+0000","last_became_active":"2024-03-04T07:33:55.552472+0000","last_became_peered":"2024-03-04T07:33:55.552472+0000","last_unstale":"2024-03-04T07:33:57.913391+0000","last_undegraded":"2024-03-04T07:33:57.913391+0000","last_fullsized":"2024-03-04T07:33:57.913391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:36:50.409567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643132+0000","last_change":"2024-03-04T07:33:56.558497+0000","last_active":"2024-03-04T07:33:58.643132+0000","last_peered":"2024-03-04T07:33:58.643132+0000","last_clean":"2024-03-04T07:33:58.643132+0000","last_became_active":"2024-03-04T07:33:56.557989+0000","last_became_peered":"2024-03-04T07:33:56.557989+0000","last_unstale":"2024-03-04T07:33:58.643132+0000","last_undegraded":"2024-03-04T07:33:58.643132+0000","last_fullsized":"2024-03-04T07:33:58.643132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:57:05.380755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522764+0000","last_change":"2024-03-04T07:33:55.560332+0000","last_active":"2024-03-04T07:33:57.522764+0000","last_peered":"2024-03-04T07:33:57.522764+0000","last_clean":"2024-03-04T07:33:57.522764+0000","last_became_active":"2024-03-04T07:33:55.560086+0000","last_became_peered":"2024-03-04T07:33:55.560086+0000","last_unstale":"2024-03-04T07:33:57.522764+0000","last_undegraded":"2024-03-04T07:33:57.522764+0000","last_fullsized":"2024-03-04T07:33:57.522764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:25:46.280657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644296+0000","last_change":"2024-03-04T07:33:56.559881+0000","last_active":"2024-03-04T07:33:58.644296+0000","last_peered":"2024-03-04T07:33:58.644296+0000","last_clean":"2024-03-04T07:33:58.644296+0000","last_became_active":"2024-03-04T07:33:56.559589+0000","last_became_peered":"2024-03-04T07:33:56.559589+0000","last_unstale":"2024-03-04T07:33:58.644296+0000","last_undegraded":"2024-03-04T07:33:58.644296+0000","last_fullsized":"2024-03-04T07:33:58.644296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:47:35.123163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640654+0000","last_change":"2024-03-04T07:33:43.559630+0000","last_active":"2024-03-04T07:33:58.640654+0000","last_peered":"2024-03-04T07:33:58.640654+0000","last_clean":"2024-03-04T07:33:58.640654+0000","last_became_active":"2024-03-04T07:33:43.557489+0000","last_became_peered":"2024-03-04T07:33:43.557489+0000","last_unstale":"2024-03-04T07:33:58.640654+0000","last_undegraded":"2024-03-04T07:33:58.640654+0000","last_fullsized":"2024-03-04T07:33:58.640654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_clean_scrub_stamp":"2024-03-04T07:33:42.183823+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:55:35.762857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641306+0000","last_change":"2024-03-04T07:33:55.548616+0000","last_active":"2024-03-04T07:33:58.641306+0000","last_peered":"2024-03-04T07:33:58.641306+0000","last_clean":"2024-03-04T07:33:58.641306+0000","last_became_active":"2024-03-04T07:33:55.547322+0000","last_became_peered":"2024-03-04T07:33:55.547322+0000","last_unstale":"2024-03-04T07:33:58.641306+0000","last_undegraded":"2024-03-04T07:33:58.641306+0000","last_fullsized":"2024-03-04T07:33:58.641306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:20:16.717483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523877+0000","last_change":"2024-03-04T07:33:56.559966+0000","last_active":"2024-03-04T07:33:57.523877+0000","last_peered":"2024-03-04T07:33:57.523877+0000","last_clean":"2024-03-04T07:33:57.523877+0000","last_became_active":"2024-03-04T07:33:56.559581+0000","last_became_peered":"2024-03-04T07:33:56.559581+0000","last_unstale":"2024-03-04T07:33:57.523877+0000","last_undegraded":"2024-03-04T07:33:57.523877+0000","last_fullsized":"2024-03-04T07:33:57.523877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640360+0000","last_change":"2024-03-04T07:33:55.563905+0000","last_active":"2024-03-04T07:33:58.640360+0000","last_peered":"2024-03-04T07:33:58.640360+0000","last_clean":"2024-03-04T07:33:58.640360+0000","last_became_active":"2024-03-04T07:33:55.563117+0000","last_became_peered":"2024-03-04T07:33:55.563117+0000","last_unstale":"2024-03-04T07:33:58.640360+0000","last_undegraded":"2024-03-04T07:33:58.640360+0000","last_fullsized":"2024-03-04T07:33:58.640360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:53:02.522757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.858965+0000","last_change":"2024-03-04T07:33:56.531080+0000","last_active":"2024-03-04T07:33:57.858965+0000","last_peered":"2024-03-04T07:33:57.858965+0000","last_clean":"2024-03-04T07:33:57.858965+0000","last_became_active":"2024-03-04T07:33:56.530776+0000","last_became_peered":"2024-03-04T07:33:56.530776+0000","last_unstale":"2024-03-04T07:33:57.858965+0000","last_undegraded":"2024-03-04T07:33:57.858965+0000","last_fullsized":"2024-03-04T07:33:57.858965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:29:14.385057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522732+0000","last_change":"2024-03-04T07:33:55.538631+0000","last_active":"2024-03-04T07:33:57.522732+0000","last_peered":"2024-03-04T07:33:57.522732+0000","last_clean":"2024-03-04T07:33:57.522732+0000","last_became_active":"2024-03-04T07:33:55.537104+0000","last_became_peered":"2024-03-04T07:33:55.537104+0000","last_unstale":"2024-03-04T07:33:57.522732+0000","last_undegraded":"2024-03-04T07:33:57.522732+0000","last_fullsized":"2024-03-04T07:33:57.522732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:27:06.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749962+0000","last_change":"2024-03-04T07:33:56.541335+0000","last_active":"2024-03-04T07:33:57.749962+0000","last_peered":"2024-03-04T07:33:57.749962+0000","last_clean":"2024-03-04T07:33:57.749962+0000","last_became_active":"2024-03-04T07:33:56.540536+0000","last_became_peered":"2024-03-04T07:33:56.540536+0000","last_unstale":"2024-03-04T07:33:57.749962+0000","last_undegraded":"2024-03-04T07:33:57.749962+0000","last_fullsized":"2024-03-04T07:33:57.749962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:31:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642643+0000","last_change":"2024-03-04T07:33:55.574740+0000","last_active":"2024-03-04T07:33:58.642643+0000","last_peered":"2024-03-04T07:33:58.642643+0000","last_clean":"2024-03-04T07:33:58.642643+0000","last_became_active":"2024-03-04T07:33:55.574129+0000","last_became_peered":"2024-03-04T07:33:55.574129+0000","last_unstale":"2024-03-04T07:33:58.642643+0000","last_undegraded":"2024-03-04T07:33:58.642643+0000","last_fullsized":"2024-03-04T07:33:58.642643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:40:57.891803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526225+0000","last_change":"2024-03-04T07:33:56.541654+0000","last_active":"2024-03-04T07:33:57.526225+0000","last_peered":"2024-03-04T07:33:57.526225+0000","last_clean":"2024-03-04T07:33:57.526225+0000","last_became_active":"2024-03-04T07:33:56.541422+0000","last_became_peered":"2024-03-04T07:33:56.541422+0000","last_unstale":"2024-03-04T07:33:57.526225+0000","last_undegraded":"2024-03-04T07:33:57.526225+0000","last_fullsized":"2024-03-04T07:33:57.526225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:38.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750122+0000","last_change":"2024-03-04T07:33:55.539317+0000","last_active":"2024-03-04T07:33:57.750122+0000","last_peered":"2024-03-04T07:33:57.750122+0000","last_clean":"2024-03-04T07:33:57.750122+0000","last_became_active":"2024-03-04T07:33:55.539017+0000","last_became_peered":"2024-03-04T07:33:55.539017+0000","last_unstale":"2024-03-04T07:33:57.750122+0000","last_undegraded":"2024-03-04T07:33:57.750122+0000","last_fullsized":"2024-03-04T07:33:57.750122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:10.787316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642464+0000","last_change":"2024-03-04T07:33:56.554103+0000","last_active":"2024-03-04T07:33:58.642464+0000","last_peered":"2024-03-04T07:33:58.642464+0000","last_clean":"2024-03-04T07:33:58.642464+0000","last_became_active":"2024-03-04T07:33:56.553742+0000","last_became_peered":"2024-03-04T07:33:56.553742+0000","last_unstale":"2024-03-04T07:33:58.642464+0000","last_undegraded":"2024-03-04T07:33:58.642464+0000","last_fullsized":"2024-03-04T07:33:58.642464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:53.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644822+0000","last_change":"2024-03-04T07:33:55.569442+0000","last_active":"2024-03-04T07:33:58.644822+0000","last_peered":"2024-03-04T07:33:58.644822+0000","last_clean":"2024-03-04T07:33:58.644822+0000","last_became_active":"2024-03-04T07:33:55.567809+0000","last_became_peered":"2024-03-04T07:33:55.567809+0000","last_unstale":"2024-03-04T07:33:58.644822+0000","last_undegraded":"2024-03-04T07:33:58.644822+0000","last_fullsized":"2024-03-04T07:33:58.644822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:58.551866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750083+0000","last_change":"2024-03-04T07:33:56.542997+0000","last_active":"2024-03-04T07:33:57.750083+0000","last_peered":"2024-03-04T07:33:57.750083+0000","last_clean":"2024-03-04T07:33:57.750083+0000","last_became_active":"2024-03-04T07:33:56.542775+0000","last_became_peered":"2024-03-04T07:33:56.542775+0000","last_unstale":"2024-03-04T07:33:57.750083+0000","last_undegraded":"2024-03-04T07:33:57.750083+0000","last_fullsized":"2024-03-04T07:33:57.750083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:32.090568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750225+0000","last_change":"2024-03-04T07:33:55.545598+0000","last_active":"2024-03-04T07:33:57.750225+0000","last_peered":"2024-03-04T07:33:57.750225+0000","last_clean":"2024-03-04T07:33:57.750225+0000","last_became_active":"2024-03-04T07:33:55.545262+0000","last_became_peered":"2024-03-04T07:33:55.545262+0000","last_unstale":"2024-03-04T07:33:57.750225+0000","last_undegraded":"2024-03-04T07:33:57.750225+0000","last_fullsized":"2024-03-04T07:33:57.750225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:01.667243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523716+0000","last_change":"2024-03-04T07:33:56.550943+0000","last_active":"2024-03-04T07:33:57.523716+0000","last_peered":"2024-03-04T07:33:57.523716+0000","last_clean":"2024-03-04T07:33:57.523716+0000","last_became_active":"2024-03-04T07:33:56.550604+0000","last_became_peered":"2024-03-04T07:33:56.550604+0000","last_unstale":"2024-03-04T07:33:57.523716+0000","last_undegraded":"2024-03-04T07:33:57.523716+0000","last_fullsized":"2024-03-04T07:33:57.523716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:01:05.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.536925+0000","last_change":"2024-03-04T07:33:55.567220+0000","last_active":"2024-03-04T07:34:01.536925+0000","last_peered":"2024-03-04T07:34:01.536925+0000","last_clean":"2024-03-04T07:34:01.536925+0000","last_became_active":"2024-03-04T07:33:55.566600+0000","last_became_peered":"2024-03-04T07:33:55.566600+0000","last_unstale":"2024-03-04T07:34:01.536925+0000","last_undegraded":"2024-03-04T07:34:01.536925+0000","last_fullsized":"2024-03-04T07:34:01.536925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.876938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859267+0000","last_change":"2024-03-04T07:33:56.544263+0000","last_active":"2024-03-04T07:33:57.859267+0000","last_peered":"2024-03-04T07:33:57.859267+0000","last_clean":"2024-03-04T07:33:57.859267+0000","last_became_active":"2024-03-04T07:33:56.543836+0000","last_became_peered":"2024-03-04T07:33:56.543836+0000","last_unstale":"2024-03-04T07:33:57.859267+0000","last_undegraded":"2024-03-04T07:33:57.859267+0000","last_fullsized":"2024-03-04T07:33:57.859267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:49.839824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640953+0000","last_change":"2024-03-04T07:33:55.566886+0000","last_active":"2024-03-04T07:33:58.640953+0000","last_peered":"2024-03-04T07:33:58.640953+0000","last_clean":"2024-03-04T07:33:58.640953+0000","last_became_active":"2024-03-04T07:33:55.566048+0000","last_became_peered":"2024-03-04T07:33:55.566048+0000","last_unstale":"2024-03-04T07:33:58.640953+0000","last_undegraded":"2024-03-04T07:33:58.640953+0000","last_fullsized":"2024-03-04T07:33:58.640953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:45:27.190438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.525876+0000","last_change":"2024-03-04T07:33:56.532666+0000","last_active":"2024-03-04T07:33:57.525876+0000","last_peered":"2024-03-04T07:33:57.525876+0000","last_clean":"2024-03-04T07:33:57.525876+0000","last_became_active":"2024-03-04T07:33:56.531901+0000","last_became_peered":"2024-03-04T07:33:56.531901+0000","last_unstale":"2024-03-04T07:33:57.525876+0000","last_undegraded":"2024-03-04T07:33:57.525876+0000","last_fullsized":"2024-03-04T07:33:57.525876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:42:04.545881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750078+0000","last_change":"2024-03-04T07:33:55.557568+0000","last_active":"2024-03-04T07:33:57.750078+0000","last_peered":"2024-03-04T07:33:57.750078+0000","last_clean":"2024-03-04T07:33:57.750078+0000","last_became_active":"2024-03-04T07:33:55.557330+0000","last_became_peered":"2024-03-04T07:33:55.557330+0000","last_unstale":"2024-03-04T07:33:57.750078+0000","last_undegraded":"2024-03-04T07:33:57.750078+0000","last_fullsized":"2024-03-04T07:33:57.750078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:34.956318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523904+0000","last_change":"2024-03-04T07:33:56.552963+0000","last_active":"2024-03-04T07:33:57.523904+0000","last_peered":"2024-03-04T07:33:57.523904+0000","last_clean":"2024-03-04T07:33:57.523904+0000","last_became_active":"2024-03-04T07:33:56.552658+0000","last_became_peered":"2024-03-04T07:33:56.552658+0000","last_unstale":"2024-03-04T07:33:57.523904+0000","last_undegraded":"2024-03-04T07:33:57.523904+0000","last_fullsized":"2024-03-04T07:33:57.523904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:05.685496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750081+0000","last_change":"2024-03-04T07:33:55.536480+0000","last_active":"2024-03-04T07:33:57.750081+0000","last_peered":"2024-03-04T07:33:57.750081+0000","last_clean":"2024-03-04T07:33:57.750081+0000","last_became_active":"2024-03-04T07:33:55.536139+0000","last_became_peered":"2024-03-04T07:33:55.536139+0000","last_unstale":"2024-03-04T07:33:57.750081+0000","last_undegraded":"2024-03-04T07:33:57.750081+0000","last_fullsized":"2024-03-04T07:33:57.750081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:12.778026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523811+0000","last_change":"2024-03-04T07:33:56.546546+0000","last_active":"2024-03-04T07:33:57.523811+0000","last_peered":"2024-03-04T07:33:57.523811+0000","last_clean":"2024-03-04T07:33:57.523811+0000","last_became_active":"2024-03-04T07:33:56.546280+0000","last_became_peered":"2024-03-04T07:33:56.546280+0000","last_unstale":"2024-03-04T07:33:57.523811+0000","last_undegraded":"2024-03-04T07:33:57.523811+0000","last_fullsized":"2024-03-04T07:33:57.523811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:19:55.950163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.534127+0000","last_change":"2024-03-04T07:33:55.572540+0000","last_active":"2024-03-04T07:34:01.534127+0000","last_peered":"2024-03-04T07:34:01.534127+0000","last_clean":"2024-03-04T07:34:01.534127+0000","last_became_active":"2024-03-04T07:33:55.571612+0000","last_became_peered":"2024-03-04T07:33:55.571612+0000","last_unstale":"2024-03-04T07:34:01.534127+0000","last_undegraded":"2024-03-04T07:34:01.534127+0000","last_fullsized":"2024-03-04T07:34:01.534127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:01:43.741423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640728+0000","last_change":"2024-03-04T07:33:56.532855+0000","last_active":"2024-03-04T07:33:58.640728+0000","last_peered":"2024-03-04T07:33:58.640728+0000","last_clean":"2024-03-04T07:33:58.640728+0000","last_became_active":"2024-03-04T07:33:56.532245+0000","last_became_peered":"2024-03-04T07:33:56.532245+0000","last_unstale":"2024-03-04T07:33:58.640728+0000","last_undegraded":"2024-03-04T07:33:58.640728+0000","last_fullsized":"2024-03-04T07:33:58.640728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:26.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642056+0000","last_change":"2024-03-04T07:33:56.563803+0000","last_active":"2024-03-04T07:33:58.642056+0000","last_peered":"2024-03-04T07:33:58.642056+0000","last_clean":"2024-03-04T07:33:58.642056+0000","last_became_active":"2024-03-04T07:33:56.563473+0000","last_became_peered":"2024-03-04T07:33:56.563473+0000","last_unstale":"2024-03-04T07:33:58.642056+0000","last_undegraded":"2024-03-04T07:33:58.642056+0000","last_fullsized":"2024-03-04T07:33:58.642056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:26.904436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.531549+0000","last_change":"2024-03-04T07:33:55.575688+0000","last_active":"2024-03-04T07:34:01.531549+0000","last_peered":"2024-03-04T07:34:01.531549+0000","last_clean":"2024-03-04T07:34:01.531549+0000","last_became_active":"2024-03-04T07:33:55.574766+0000","last_became_peered":"2024-03-04T07:33:55.574766+0000","last_unstale":"2024-03-04T07:34:01.531549+0000","last_undegraded":"2024-03-04T07:34:01.531549+0000","last_fullsized":"2024-03-04T07:34:01.531549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:32:19.708016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642750+0000","last_change":"2024-03-04T07:33:56.563798+0000","last_active":"2024-03-04T07:33:58.642750+0000","last_peered":"2024-03-04T07:33:58.642750+0000","last_clean":"2024-03-04T07:33:58.642750+0000","last_became_active":"2024-03-04T07:33:56.563474+0000","last_became_peered":"2024-03-04T07:33:56.563474+0000","last_unstale":"2024-03-04T07:33:58.642750+0000","last_undegraded":"2024-03-04T07:33:58.642750+0000","last_fullsized":"2024-03-04T07:33:58.642750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:47:39.763001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.535125+0000","last_change":"2024-03-04T07:33:55.570016+0000","last_active":"2024-03-04T07:34:01.535125+0000","last_peered":"2024-03-04T07:34:01.535125+0000","last_clean":"2024-03-04T07:34:01.535125+0000","last_became_active":"2024-03-04T07:33:55.568749+0000","last_became_peered":"2024-03-04T07:33:55.568749+0000","last_unstale":"2024-03-04T07:34:01.535125+0000","last_undegraded":"2024-03-04T07:34:01.535125+0000","last_fullsized":"2024-03-04T07:34:01.535125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:01.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859463+0000","last_change":"2024-03-04T07:33:56.562015+0000","last_active":"2024-03-04T07:33:57.859463+0000","last_peered":"2024-03-04T07:33:57.859463+0000","last_clean":"2024-03-04T07:33:57.859463+0000","last_became_active":"2024-03-04T07:33:56.561675+0000","last_became_peered":"2024-03-04T07:33:56.561675+0000","last_unstale":"2024-03-04T07:33:57.859463+0000","last_undegraded":"2024-03-04T07:33:57.859463+0000","last_fullsized":"2024-03-04T07:33:57.859463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:36:20.241414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.918424+0000","last_change":"2024-03-04T07:33:55.538337+0000","last_active":"2024-03-04T07:33:57.918424+0000","last_peered":"2024-03-04T07:33:57.918424+0000","last_clean":"2024-03-04T07:33:57.918424+0000","last_became_active":"2024-03-04T07:33:55.538010+0000","last_became_peered":"2024-03-04T07:33:55.538010+0000","last_unstale":"2024-03-04T07:33:57.918424+0000","last_undegraded":"2024-03-04T07:33:57.918424+0000","last_fullsized":"2024-03-04T07:33:57.918424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:47:15.711205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.524079+0000","last_change":"2024-03-04T07:33:56.559051+0000","last_active":"2024-03-04T07:33:57.524079+0000","last_peered":"2024-03-04T07:33:57.524079+0000","last_clean":"2024-03-04T07:33:57.524079+0000","last_became_active":"2024-03-04T07:33:56.558799+0000","last_became_peered":"2024-03-04T07:33:56.558799+0000","last_unstale":"2024-03-04T07:33:57.524079+0000","last_undegraded":"2024-03-04T07:33:57.524079+0000","last_fullsized":"2024-03-04T07:33:57.524079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:16.516631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.536473+0000","last_change":"2024-03-04T07:33:55.566294+0000","last_active":"2024-03-04T07:34:01.536473+0000","last_peered":"2024-03-04T07:34:01.536473+0000","last_clean":"2024-03-04T07:34:01.536473+0000","last_became_active":"2024-03-04T07:33:55.565941+0000","last_became_peered":"2024-03-04T07:33:55.565941+0000","last_unstale":"2024-03-04T07:34:01.536473+0000","last_undegraded":"2024-03-04T07:34:01.536473+0000","last_fullsized":"2024-03-04T07:34:01.536473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:03.115085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":73689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":806912,"data_stored":664479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":73619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":73655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":815104,"data_stored":664535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T07:34:07.269 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T07:34:07.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b3400640 1 Processor -- start 2024-03-04T07:34:07.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b3400640 1 -- start start 2024-03-04T07:34:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b3400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f15ac0c8200 0x7f15ac070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15ac152bd0 0x7f15ac1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b3400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ac1591f0 con 0x7f15ac1527f0 2024-03-04T07:34:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b3400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac071df0 con 0x7f15ac152bd0 2024-03-04T07:34:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b3400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac1597b0 con 0x7f15ac0c8200 2024-03-04T07:34:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15ac152bd0 0x7f15ac1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15ac152bd0 0x7f15ac1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59218/0 (socket says 172.21.15.113:59218) 2024-03-04T07:34:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b1400640 1 -- 172.21.15.113:0/2069766007 learned_addr learned my addr 172.21.15.113:0/2069766007 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:07.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15a3e00640 1 --2- 172.21.15.113:0/2069766007 >> v2:172.21.15.136:3300/0 conn(0x7f15ac0c8200 0x7f15ac070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b1400640 1 -- 172.21.15.113:0/2069766007 >> v2:172.21.15.136:3300/0 conn(0x7f15ac0c8200 msgr2=0x7f15ac070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:07.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b1400640 1 --2- 172.21.15.113:0/2069766007 >> v2:172.21.15.136:3300/0 conn(0x7f15ac0c8200 0x7f15ac070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b1400640 1 -- 172.21.15.113:0/2069766007 >> v1:172.21.15.113:6789/0 conn(0x7f15ac1527f0 legacy=0x7f15ac072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15b1400640 1 -- 172.21.15.113:0/2069766007 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac1594d0 con 0x7f15ac152bd0 2024-03-04T07:34:07.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.438+0000 7f15abe00640 1 -- 172.21.15.113:0/2069766007 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119749458 0 0) 0x7f15ac1591f0 con 0x7f15ac1527f0 2024-03-04T07:34:07.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15b1400640 1 --2- 172.21.15.113:0/2069766007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 0x7f15ac1587d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0cd230 tx=0x7f15a401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9067d2b3ded763b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:07.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15abe00640 1 -- 172.21.15.113:0/2069766007 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40a57a0 con 0x7f15ac152bd0 2024-03-04T07:34:07.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15abe00640 1 -- 172.21.15.113:0/2069766007 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a40a5940 con 0x7f15ac152bd0 2024-03-04T07:34:07.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15abe00640 1 -- 172.21.15.113:0/2069766007 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40a5d10 con 0x7f15ac152bd0 2024-03-04T07:34:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15b3400640 1 -- 172.21.15.113:0/2069766007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 msgr2=0x7f15ac1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15b3400640 1 --2- 172.21.15.113:0/2069766007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 0x7f15ac1587d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0cd230 tx=0x7f15a401c9f0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15b3400640 1 -- 172.21.15.113:0/2069766007 shutdown_connections 2024-03-04T07:34:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15b3400640 1 --2- 172.21.15.113:0/2069766007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 0x7f15ac1587d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15b3400640 1 --2- 172.21.15.113:0/2069766007 >> v2:172.21.15.136:3300/0 conn(0x7f15ac0c8200 0x7f15ac070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15b3400640 1 -- 172.21.15.113:0/2069766007 >> 172.21.15.113:0/2069766007 conn(0x7f15ac06e820 msgr2=0x7f15ac06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15b3400640 1 -- 172.21.15.113:0/2069766007 shutdown_connections 2024-03-04T07:34:07.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.442+0000 7f15b3400640 1 -- 172.21.15.113:0/2069766007 wait complete. 2024-03-04T07:34:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b3400640 1 Processor -- start 2024-03-04T07:34:07.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b3400640 1 -- start start 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15ac0c8200 0x7f15ac1f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15ac1527f0 0x7f15ac151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 0x7f15ac15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b3400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1654f0 con 0x7f15ac152bd0 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b3400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac165370 con 0x7f15ac1527f0 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b3400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac165670 con 0x7f15ac0c8200 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 0x7f15ac15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 0x7f15ac15ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59234/0 (socket says 172.21.15.113:59234) 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b1400640 1 -- 172.21.15.113:0/387116533 learned_addr learned my addr 172.21.15.113:0/387116533 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15a3e00640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15ac1527f0 0x7f15ac151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b0a00640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15ac0c8200 0x7f15ac1f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b1400640 1 -- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15ac0c8200 msgr2=0x7f15ac1f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b1400640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15ac0c8200 0x7f15ac1f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b1400640 1 -- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15ac1527f0 msgr2=0x7f15ac151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b1400640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15ac1527f0 0x7f15ac151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b1400640 1 -- 172.21.15.113:0/387116533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac1f2520 con 0x7f15ac152bd0 2024-03-04T07:34:07.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b1400640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 0x7f15ac15ba30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f15a4019600 tx=0x7f15a4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:07.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15a9600640 1 -- 172.21.15.113:0/387116533 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40b8670 con 0x7f15ac152bd0 2024-03-04T07:34:07.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15a9600640 1 -- 172.21.15.113:0/387116533 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a40be070 con 0x7f15ac152bd0 2024-03-04T07:34:07.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15a9600640 1 -- 172.21.15.113:0/387116533 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40b8a50 con 0x7f15ac152bd0 2024-03-04T07:34:07.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b3400640 1 -- 172.21.15.113:0/387116533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac1f2800 con 0x7f15ac152bd0 2024-03-04T07:34:07.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.446+0000 7f15b3400640 1 -- 172.21.15.113:0/387116533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac15b230 con 0x7f15ac152bd0 2024-03-04T07:34:07.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.450+0000 7f15b3400640 1 -- 172.21.15.113:0/387116533 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac075c00 con 0x7f15ac152bd0 2024-03-04T07:34:07.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.450+0000 7f15a9600640 1 -- 172.21.15.113:0/387116533 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15a40b8bf0 con 0x7f15ac152bd0 2024-03-04T07:34:07.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.450+0000 7f15a9600640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1588066f90 0x7f1588069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.450+0000 7f15a9600640 1 -- 172.21.15.113:0/387116533 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15a4120150 con 0x7f15ac152bd0 2024-03-04T07:34:07.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.450+0000 7f15b0a00640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1588066f90 0x7f1588069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.450+0000 7f15b0a00640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1588066f90 0x7f1588069450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f159c000f70 tx=0x7f159c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:07.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.454+0000 7f15a9600640 1 -- 172.21.15.113:0/387116533 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f15a40165e0 con 0x7f15ac152bd0 2024-03-04T07:34:07.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.614+0000 7f15b3400640 1 -- 172.21.15.113:0/387116533 --> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15ac0cae50 con 0x7f1588066f90 2024-03-04T07:34:07.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.622+0000 7f15a9600640 1 -- 172.21.15.113:0/387116533 <== mgr.4103 v2:172.21.15.136:6832/3353359815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f15ac0cae50 con 0x7f1588066f90 2024-03-04T07:34:07.629 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:07.630 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-03-04T07:34:07.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 -- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1588066f90 msgr2=0x7f1588069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1588066f90 0x7f1588069450 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f159c000f70 tx=0x7f159c009040 comp rx=0 tx=0).stop 2024-03-04T07:34:07.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 -- 172.21.15.113:0/387116533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 msgr2=0x7f15ac15ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 0x7f15ac15ba30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f15a4019600 tx=0x7f15a4019790 comp rx=0 tx=0).stop 2024-03-04T07:34:07.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 -- 172.21.15.113:0/387116533 shutdown_connections 2024-03-04T07:34:07.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1588066f90 0x7f1588069450 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15ac152bd0 0x7f15ac15ba30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15ac1527f0 0x7f15ac151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 --2- 172.21.15.113:0/387116533 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15ac0c8200 0x7f15ac1f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 -- 172.21.15.113:0/387116533 >> 172.21.15.113:0/387116533 conn(0x7f15ac06e820 msgr2=0x7f15ac0ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:07.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 -- 172.21.15.113:0/387116533 shutdown_connections 2024-03-04T07:34:07.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.626+0000 7f15b3400640 1 -- 172.21.15.113:0/387116533 wait complete. 2024-03-04T07:34:07.649 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-03-04T07:34:07.436863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223144,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751576,"statfs":{"total":773094113280,"available":772865613824,"internally_reserved":0,"allocated":2924544,"data_stored":1773788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004001"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645580+0000","last_change":"2024-03-04T07:33:56.559432+0000","last_active":"2024-03-04T07:33:58.645580+0000","last_peered":"2024-03-04T07:33:58.645580+0000","last_clean":"2024-03-04T07:33:58.645580+0000","last_became_active":"2024-03-04T07:33:56.559031+0000","last_became_peered":"2024-03-04T07:33:56.559031+0000","last_unstale":"2024-03-04T07:33:58.645580+0000","last_undegraded":"2024-03-04T07:33:58.645580+0000","last_fullsized":"2024-03-04T07:33:58.645580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:43:55.037090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750282+0000","last_change":"2024-03-04T07:33:56.546822+0000","last_active":"2024-03-04T07:33:57.750282+0000","last_peered":"2024-03-04T07:33:57.750282+0000","last_clean":"2024-03-04T07:33:57.750282+0000","last_became_active":"2024-03-04T07:33:56.546547+0000","last_became_peered":"2024-03-04T07:33:56.546547+0000","last_unstale":"2024-03-04T07:33:57.750282+0000","last_undegraded":"2024-03-04T07:33:57.750282+0000","last_fullsized":"2024-03-04T07:33:57.750282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:06.332337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859174+0000","last_change":"2024-03-04T07:33:56.530803+0000","last_active":"2024-03-04T07:33:57.859174+0000","last_peered":"2024-03-04T07:33:57.859174+0000","last_clean":"2024-03-04T07:33:57.859174+0000","last_became_active":"2024-03-04T07:33:56.530488+0000","last_became_peered":"2024-03-04T07:33:56.530488+0000","last_unstale":"2024-03-04T07:33:57.859174+0000","last_undegraded":"2024-03-04T07:33:57.859174+0000","last_fullsized":"2024-03-04T07:33:57.859174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:00:42.147951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642296+0000","last_change":"2024-03-04T07:33:56.558669+0000","last_active":"2024-03-04T07:33:58.642296+0000","last_peered":"2024-03-04T07:33:58.642296+0000","last_clean":"2024-03-04T07:33:58.642296+0000","last_became_active":"2024-03-04T07:33:56.558350+0000","last_became_peered":"2024-03-04T07:33:56.558350+0000","last_unstale":"2024-03-04T07:33:58.642296+0000","last_undegraded":"2024-03-04T07:33:58.642296+0000","last_fullsized":"2024-03-04T07:33:58.642296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:35:53.460563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644296+0000","last_change":"2024-03-04T07:33:56.540760+0000","last_active":"2024-03-04T07:33:58.644296+0000","last_peered":"2024-03-04T07:33:58.644296+0000","last_clean":"2024-03-04T07:33:58.644296+0000","last_became_active":"2024-03-04T07:33:56.540351+0000","last_became_peered":"2024-03-04T07:33:56.540351+0000","last_unstale":"2024-03-04T07:33:58.644296+0000","last_undegraded":"2024-03-04T07:33:58.644296+0000","last_fullsized":"2024-03-04T07:33:58.644296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:24:21.708228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641368+0000","last_change":"2024-03-04T07:33:56.558333+0000","last_active":"2024-03-04T07:33:58.641368+0000","last_peered":"2024-03-04T07:33:58.641368+0000","last_clean":"2024-03-04T07:33:58.641368+0000","last_became_active":"2024-03-04T07:33:56.558046+0000","last_became_peered":"2024-03-04T07:33:56.558046+0000","last_unstale":"2024-03-04T07:33:58.641368+0000","last_undegraded":"2024-03-04T07:33:58.641368+0000","last_fullsized":"2024-03-04T07:33:58.641368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:30:46.375184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643038+0000","last_change":"2024-03-04T07:33:56.543476+0000","last_active":"2024-03-04T07:33:58.643038+0000","last_peered":"2024-03-04T07:33:58.643038+0000","last_clean":"2024-03-04T07:33:58.643038+0000","last_became_active":"2024-03-04T07:33:56.543176+0000","last_became_peered":"2024-03-04T07:33:56.543176+0000","last_unstale":"2024-03-04T07:33:58.643038+0000","last_undegraded":"2024-03-04T07:33:58.643038+0000","last_fullsized":"2024-03-04T07:33:58.643038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:26:34.736870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523950+0000","last_change":"2024-03-04T07:33:56.559488+0000","last_active":"2024-03-04T07:33:57.523950+0000","last_peered":"2024-03-04T07:33:57.523950+0000","last_clean":"2024-03-04T07:33:57.523950+0000","last_became_active":"2024-03-04T07:33:56.559172+0000","last_became_peered":"2024-03-04T07:33:56.559172+0000","last_unstale":"2024-03-04T07:33:57.523950+0000","last_undegraded":"2024-03-04T07:33:57.523950+0000","last_fullsized":"2024-03-04T07:33:57.523950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:53.461697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645566+0000","last_change":"2024-03-04T07:33:56.560587+0000","last_active":"2024-03-04T07:33:58.645566+0000","last_peered":"2024-03-04T07:33:58.645566+0000","last_clean":"2024-03-04T07:33:58.645566+0000","last_became_active":"2024-03-04T07:33:56.560306+0000","last_became_peered":"2024-03-04T07:33:56.560306+0000","last_unstale":"2024-03-04T07:33:58.645566+0000","last_undegraded":"2024-03-04T07:33:58.645566+0000","last_fullsized":"2024-03-04T07:33:58.645566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:06:17.011327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641979+0000","last_change":"2024-03-04T07:33:56.560852+0000","last_active":"2024-03-04T07:33:58.641979+0000","last_peered":"2024-03-04T07:33:58.641979+0000","last_clean":"2024-03-04T07:33:58.641979+0000","last_became_active":"2024-03-04T07:33:56.560592+0000","last_became_peered":"2024-03-04T07:33:56.560592+0000","last_unstale":"2024-03-04T07:33:58.641979+0000","last_undegraded":"2024-03-04T07:33:58.641979+0000","last_fullsized":"2024-03-04T07:33:58.641979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:39:31.496432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641631+0000","last_change":"2024-03-04T07:33:56.561040+0000","last_active":"2024-03-04T07:33:58.641631+0000","last_peered":"2024-03-04T07:33:58.641631+0000","last_clean":"2024-03-04T07:33:58.641631+0000","last_became_active":"2024-03-04T07:33:56.560719+0000","last_became_peered":"2024-03-04T07:33:56.560719+0000","last_unstale":"2024-03-04T07:33:58.641631+0000","last_undegraded":"2024-03-04T07:33:58.641631+0000","last_fullsized":"2024-03-04T07:33:58.641631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:14:13.007587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526511+0000","last_change":"2024-03-04T07:33:56.550648+0000","last_active":"2024-03-04T07:33:57.526511+0000","last_peered":"2024-03-04T07:33:57.526511+0000","last_clean":"2024-03-04T07:33:57.526511+0000","last_became_active":"2024-03-04T07:33:56.549568+0000","last_became_peered":"2024-03-04T07:33:56.549568+0000","last_unstale":"2024-03-04T07:33:57.526511+0000","last_undegraded":"2024-03-04T07:33:57.526511+0000","last_fullsized":"2024-03-04T07:33:57.526511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:34:35.299480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523777+0000","last_change":"2024-03-04T07:33:56.558880+0000","last_active":"2024-03-04T07:33:57.523777+0000","last_peered":"2024-03-04T07:33:57.523777+0000","last_clean":"2024-03-04T07:33:57.523777+0000","last_became_active":"2024-03-04T07:33:56.558567+0000","last_became_peered":"2024-03-04T07:33:56.558567+0000","last_unstale":"2024-03-04T07:33:57.523777+0000","last_undegraded":"2024-03-04T07:33:57.523777+0000","last_fullsized":"2024-03-04T07:33:57.523777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:16.354787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.858969+0000","last_change":"2024-03-04T07:33:56.544222+0000","last_active":"2024-03-04T07:33:57.858969+0000","last_peered":"2024-03-04T07:33:57.858969+0000","last_clean":"2024-03-04T07:33:57.858969+0000","last_became_active":"2024-03-04T07:33:56.543936+0000","last_became_peered":"2024-03-04T07:33:56.543936+0000","last_unstale":"2024-03-04T07:33:57.858969+0000","last_undegraded":"2024-03-04T07:33:57.858969+0000","last_fullsized":"2024-03-04T07:33:57.858969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:42.788232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749980+0000","last_change":"2024-03-04T07:33:56.545195+0000","last_active":"2024-03-04T07:33:57.749980+0000","last_peered":"2024-03-04T07:33:57.749980+0000","last_clean":"2024-03-04T07:33:57.749980+0000","last_became_active":"2024-03-04T07:33:56.544837+0000","last_became_peered":"2024-03-04T07:33:56.544837+0000","last_unstale":"2024-03-04T07:33:57.749980+0000","last_undegraded":"2024-03-04T07:33:57.749980+0000","last_fullsized":"2024-03-04T07:33:57.749980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:24:59.643746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641197+0000","last_change":"2024-03-04T07:33:56.558398+0000","last_active":"2024-03-04T07:33:58.641197+0000","last_peered":"2024-03-04T07:33:58.641197+0000","last_clean":"2024-03-04T07:33:58.641197+0000","last_became_active":"2024-03-04T07:33:56.557962+0000","last_became_peered":"2024-03-04T07:33:56.557962+0000","last_unstale":"2024-03-04T07:33:58.641197+0000","last_undegraded":"2024-03-04T07:33:58.641197+0000","last_fullsized":"2024-03-04T07:33:58.641197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:21:54.588819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643019+0000","last_change":"2024-03-04T07:33:56.559597+0000","last_active":"2024-03-04T07:33:58.643019+0000","last_peered":"2024-03-04T07:33:58.643019+0000","last_clean":"2024-03-04T07:33:58.643019+0000","last_became_active":"2024-03-04T07:33:56.558719+0000","last_became_peered":"2024-03-04T07:33:56.558719+0000","last_unstale":"2024-03-04T07:33:58.643019+0000","last_undegraded":"2024-03-04T07:33:58.643019+0000","last_fullsized":"2024-03-04T07:33:58.643019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:43:56.063344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522942+0000","last_change":"2024-03-04T07:33:56.528479+0000","last_active":"2024-03-04T07:33:57.522942+0000","last_peered":"2024-03-04T07:33:57.522942+0000","last_clean":"2024-03-04T07:33:57.522942+0000","last_became_active":"2024-03-04T07:33:56.528213+0000","last_became_peered":"2024-03-04T07:33:56.528213+0000","last_unstale":"2024-03-04T07:33:57.522942+0000","last_undegraded":"2024-03-04T07:33:57.522942+0000","last_fullsized":"2024-03-04T07:33:57.522942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:38:48.062007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644805+0000","last_change":"2024-03-04T07:33:56.545890+0000","last_active":"2024-03-04T07:33:58.644805+0000","last_peered":"2024-03-04T07:33:58.644805+0000","last_clean":"2024-03-04T07:33:58.644805+0000","last_became_active":"2024-03-04T07:33:56.545467+0000","last_became_peered":"2024-03-04T07:33:56.545467+0000","last_unstale":"2024-03-04T07:33:58.644805+0000","last_undegraded":"2024-03-04T07:33:58.644805+0000","last_fullsized":"2024-03-04T07:33:58.644805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:43:56.049152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641702+0000","last_change":"2024-03-04T07:33:56.549059+0000","last_active":"2024-03-04T07:33:58.641702+0000","last_peered":"2024-03-04T07:33:58.641702+0000","last_clean":"2024-03-04T07:33:58.641702+0000","last_became_active":"2024-03-04T07:33:56.548783+0000","last_became_peered":"2024-03-04T07:33:56.548783+0000","last_unstale":"2024-03-04T07:33:58.641702+0000","last_undegraded":"2024-03-04T07:33:58.641702+0000","last_fullsized":"2024-03-04T07:33:58.641702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:02.330428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859045+0000","last_change":"2024-03-04T07:33:56.540558+0000","last_active":"2024-03-04T07:33:57.859045+0000","last_peered":"2024-03-04T07:33:57.859045+0000","last_clean":"2024-03-04T07:33:57.859045+0000","last_became_active":"2024-03-04T07:33:56.539403+0000","last_became_peered":"2024-03-04T07:33:56.539403+0000","last_unstale":"2024-03-04T07:33:57.859045+0000","last_undegraded":"2024-03-04T07:33:57.859045+0000","last_fullsized":"2024-03-04T07:33:57.859045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:28:19.413591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641763+0000","last_change":"2024-03-04T07:33:56.554982+0000","last_active":"2024-03-04T07:33:58.641763+0000","last_peered":"2024-03-04T07:33:58.641763+0000","last_clean":"2024-03-04T07:33:58.641763+0000","last_became_active":"2024-03-04T07:33:56.554470+0000","last_became_peered":"2024-03-04T07:33:56.554470+0000","last_unstale":"2024-03-04T07:33:58.641763+0000","last_undegraded":"2024-03-04T07:33:58.641763+0000","last_fullsized":"2024-03-04T07:33:58.641763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:32.767329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523496+0000","last_change":"2024-03-04T07:33:56.545465+0000","last_active":"2024-03-04T07:33:57.523496+0000","last_peered":"2024-03-04T07:33:57.523496+0000","last_clean":"2024-03-04T07:33:57.523496+0000","last_became_active":"2024-03-04T07:33:56.545270+0000","last_became_peered":"2024-03-04T07:33:56.545270+0000","last_unstale":"2024-03-04T07:33:57.523496+0000","last_undegraded":"2024-03-04T07:33:57.523496+0000","last_fullsized":"2024-03-04T07:33:57.523496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:08:45.244868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643452+0000","last_change":"2024-03-04T07:33:56.555428+0000","last_active":"2024-03-04T07:33:58.643452+0000","last_peered":"2024-03-04T07:33:58.643452+0000","last_clean":"2024-03-04T07:33:58.643452+0000","last_became_active":"2024-03-04T07:33:56.554942+0000","last_became_peered":"2024-03-04T07:33:56.554942+0000","last_unstale":"2024-03-04T07:33:58.643452+0000","last_undegraded":"2024-03-04T07:33:58.643452+0000","last_fullsized":"2024-03-04T07:33:58.643452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:04.724517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523610+0000","last_change":"2024-03-04T07:33:56.549563+0000","last_active":"2024-03-04T07:33:57.523610+0000","last_peered":"2024-03-04T07:33:57.523610+0000","last_clean":"2024-03-04T07:33:57.523610+0000","last_became_active":"2024-03-04T07:33:56.548790+0000","last_became_peered":"2024-03-04T07:33:56.548790+0000","last_unstale":"2024-03-04T07:33:57.523610+0000","last_undegraded":"2024-03-04T07:33:57.523610+0000","last_fullsized":"2024-03-04T07:33:57.523610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:30:53.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642385+0000","last_change":"2024-03-04T07:33:56.553791+0000","last_active":"2024-03-04T07:33:58.642385+0000","last_peered":"2024-03-04T07:33:58.642385+0000","last_clean":"2024-03-04T07:33:58.642385+0000","last_became_active":"2024-03-04T07:33:56.553012+0000","last_became_peered":"2024-03-04T07:33:56.553012+0000","last_unstale":"2024-03-04T07:33:58.642385+0000","last_undegraded":"2024-03-04T07:33:58.642385+0000","last_fullsized":"2024-03-04T07:33:58.642385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:38.440189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750163+0000","last_change":"2024-03-04T07:33:56.551341+0000","last_active":"2024-03-04T07:33:57.750163+0000","last_peered":"2024-03-04T07:33:57.750163+0000","last_clean":"2024-03-04T07:33:57.750163+0000","last_became_active":"2024-03-04T07:33:56.550616+0000","last_became_peered":"2024-03-04T07:33:56.550616+0000","last_unstale":"2024-03-04T07:33:57.750163+0000","last_undegraded":"2024-03-04T07:33:57.750163+0000","last_fullsized":"2024-03-04T07:33:57.750163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:35:45.801510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523840+0000","last_change":"2024-03-04T07:33:56.543903+0000","last_active":"2024-03-04T07:33:57.523840+0000","last_peered":"2024-03-04T07:33:57.523840+0000","last_clean":"2024-03-04T07:33:57.523840+0000","last_became_active":"2024-03-04T07:33:56.543632+0000","last_became_peered":"2024-03-04T07:33:56.543632+0000","last_unstale":"2024-03-04T07:33:57.523840+0000","last_undegraded":"2024-03-04T07:33:57.523840+0000","last_fullsized":"2024-03-04T07:33:57.523840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:26:10.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859232+0000","last_change":"2024-03-04T07:33:56.547091+0000","last_active":"2024-03-04T07:33:57.859232+0000","last_peered":"2024-03-04T07:33:57.859232+0000","last_clean":"2024-03-04T07:33:57.859232+0000","last_became_active":"2024-03-04T07:33:56.546652+0000","last_became_peered":"2024-03-04T07:33:56.546652+0000","last_unstale":"2024-03-04T07:33:57.859232+0000","last_undegraded":"2024-03-04T07:33:57.859232+0000","last_fullsized":"2024-03-04T07:33:57.859232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:26:03.091219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642136+0000","last_change":"2024-03-04T07:33:56.552493+0000","last_active":"2024-03-04T07:33:58.642136+0000","last_peered":"2024-03-04T07:33:58.642136+0000","last_clean":"2024-03-04T07:33:58.642136+0000","last_became_active":"2024-03-04T07:33:56.552189+0000","last_became_peered":"2024-03-04T07:33:56.552189+0000","last_unstale":"2024-03-04T07:33:58.642136+0000","last_undegraded":"2024-03-04T07:33:58.642136+0000","last_fullsized":"2024-03-04T07:33:58.642136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:10.313129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522710+0000","last_change":"2024-03-04T07:33:56.549661+0000","last_active":"2024-03-04T07:33:57.522710+0000","last_peered":"2024-03-04T07:33:57.522710+0000","last_clean":"2024-03-04T07:33:57.522710+0000","last_became_active":"2024-03-04T07:33:56.549351+0000","last_became_peered":"2024-03-04T07:33:56.549351+0000","last_unstale":"2024-03-04T07:33:57.522710+0000","last_undegraded":"2024-03-04T07:33:57.522710+0000","last_fullsized":"2024-03-04T07:33:57.522710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:55.684272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640651+0000","last_change":"2024-03-04T07:33:56.533132+0000","last_active":"2024-03-04T07:33:58.640651+0000","last_peered":"2024-03-04T07:33:58.640651+0000","last_clean":"2024-03-04T07:33:58.640651+0000","last_became_active":"2024-03-04T07:33:56.532251+0000","last_became_peered":"2024-03-04T07:33:56.532251+0000","last_unstale":"2024-03-04T07:33:58.640651+0000","last_undegraded":"2024-03-04T07:33:58.640651+0000","last_fullsized":"2024-03-04T07:33:58.640651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:17:47.018585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.525697+0000","last_change":"2024-03-04T07:33:56.543382+0000","last_active":"2024-03-04T07:33:57.525697+0000","last_peered":"2024-03-04T07:33:57.525697+0000","last_clean":"2024-03-04T07:33:57.525697+0000","last_became_active":"2024-03-04T07:33:56.542937+0000","last_became_peered":"2024-03-04T07:33:56.542937+0000","last_unstale":"2024-03-04T07:33:57.525697+0000","last_undegraded":"2024-03-04T07:33:57.525697+0000","last_fullsized":"2024-03-04T07:33:57.525697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:18:28.332590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.918391+0000","last_change":"2024-03-04T07:33:55.566130+0000","last_active":"2024-03-04T07:33:57.918391+0000","last_peered":"2024-03-04T07:33:57.918391+0000","last_clean":"2024-03-04T07:33:57.918391+0000","last_became_active":"2024-03-04T07:33:55.565894+0000","last_became_peered":"2024-03-04T07:33:55.565894+0000","last_unstale":"2024-03-04T07:33:57.918391+0000","last_undegraded":"2024-03-04T07:33:57.918391+0000","last_fullsized":"2024-03-04T07:33:57.918391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:36.530949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859235+0000","last_change":"2024-03-04T07:33:56.540113+0000","last_active":"2024-03-04T07:33:57.859235+0000","last_peered":"2024-03-04T07:33:57.859235+0000","last_clean":"2024-03-04T07:33:57.859235+0000","last_became_active":"2024-03-04T07:33:56.539738+0000","last_became_peered":"2024-03-04T07:33:56.539738+0000","last_unstale":"2024-03-04T07:33:57.859235+0000","last_undegraded":"2024-03-04T07:33:57.859235+0000","last_fullsized":"2024-03-04T07:33:57.859235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:44.949754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641120+0000","last_change":"2024-03-04T07:33:55.554956+0000","last_active":"2024-03-04T07:33:58.641120+0000","last_peered":"2024-03-04T07:33:58.641120+0000","last_clean":"2024-03-04T07:33:58.641120+0000","last_became_active":"2024-03-04T07:33:55.554151+0000","last_became_peered":"2024-03-04T07:33:55.554151+0000","last_unstale":"2024-03-04T07:33:58.641120+0000","last_undegraded":"2024-03-04T07:33:58.641120+0000","last_fullsized":"2024-03-04T07:33:58.641120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:06:19.621510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523529+0000","last_change":"2024-03-04T07:33:55.542200+0000","last_active":"2024-03-04T07:33:57.523529+0000","last_peered":"2024-03-04T07:33:57.523529+0000","last_clean":"2024-03-04T07:33:57.523529+0000","last_became_active":"2024-03-04T07:33:55.541902+0000","last_became_peered":"2024-03-04T07:33:55.541902+0000","last_unstale":"2024-03-04T07:33:57.523529+0000","last_undegraded":"2024-03-04T07:33:57.523529+0000","last_fullsized":"2024-03-04T07:33:57.523529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:55:50.251288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523733+0000","last_change":"2024-03-04T07:33:56.548380+0000","last_active":"2024-03-04T07:33:57.523733+0000","last_peered":"2024-03-04T07:33:57.523733+0000","last_clean":"2024-03-04T07:33:57.523733+0000","last_became_active":"2024-03-04T07:33:56.546087+0000","last_became_peered":"2024-03-04T07:33:56.546087+0000","last_unstale":"2024-03-04T07:33:57.523733+0000","last_undegraded":"2024-03-04T07:33:57.523733+0000","last_fullsized":"2024-03-04T07:33:57.523733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:06:43.538687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643489+0000","last_change":"2024-03-04T07:33:55.568508+0000","last_active":"2024-03-04T07:33:58.643489+0000","last_peered":"2024-03-04T07:33:58.643489+0000","last_clean":"2024-03-04T07:33:58.643489+0000","last_became_active":"2024-03-04T07:33:55.566073+0000","last_became_peered":"2024-03-04T07:33:55.566073+0000","last_unstale":"2024-03-04T07:33:58.643489+0000","last_undegraded":"2024-03-04T07:33:58.643489+0000","last_fullsized":"2024-03-04T07:33:58.643489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:34:17.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749969+0000","last_change":"2024-03-04T07:33:56.552785+0000","last_active":"2024-03-04T07:33:57.749969+0000","last_peered":"2024-03-04T07:33:57.749969+0000","last_clean":"2024-03-04T07:33:57.749969+0000","last_became_active":"2024-03-04T07:33:56.552533+0000","last_became_peered":"2024-03-04T07:33:56.552533+0000","last_unstale":"2024-03-04T07:33:57.749969+0000","last_undegraded":"2024-03-04T07:33:57.749969+0000","last_fullsized":"2024-03-04T07:33:57.749969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:08.582237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642106+0000","last_change":"2024-03-04T07:33:55.559395+0000","last_active":"2024-03-04T07:33:58.642106+0000","last_peered":"2024-03-04T07:33:58.642106+0000","last_clean":"2024-03-04T07:33:58.642106+0000","last_became_active":"2024-03-04T07:33:55.558593+0000","last_became_peered":"2024-03-04T07:33:55.558593+0000","last_unstale":"2024-03-04T07:33:58.642106+0000","last_undegraded":"2024-03-04T07:33:58.642106+0000","last_fullsized":"2024-03-04T07:33:58.642106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:00:20.205445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642007+0000","last_change":"2024-03-04T07:33:56.553260+0000","last_active":"2024-03-04T07:33:58.642007+0000","last_peered":"2024-03-04T07:33:58.642007+0000","last_clean":"2024-03-04T07:33:58.642007+0000","last_became_active":"2024-03-04T07:33:56.552952+0000","last_became_peered":"2024-03-04T07:33:56.552952+0000","last_unstale":"2024-03-04T07:33:58.642007+0000","last_undegraded":"2024-03-04T07:33:58.642007+0000","last_fullsized":"2024-03-04T07:33:58.642007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:25:51.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642257+0000","last_change":"2024-03-04T07:33:55.568220+0000","last_active":"2024-03-04T07:33:58.642257+0000","last_peered":"2024-03-04T07:33:58.642257+0000","last_clean":"2024-03-04T07:33:58.642257+0000","last_became_active":"2024-03-04T07:33:55.567569+0000","last_became_peered":"2024-03-04T07:33:55.567569+0000","last_unstale":"2024-03-04T07:33:58.642257+0000","last_undegraded":"2024-03-04T07:33:58.642257+0000","last_fullsized":"2024-03-04T07:33:58.642257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:40:49.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859421+0000","last_change":"2024-03-04T07:33:56.559745+0000","last_active":"2024-03-04T07:33:57.859421+0000","last_peered":"2024-03-04T07:33:57.859421+0000","last_clean":"2024-03-04T07:33:57.859421+0000","last_became_active":"2024-03-04T07:33:56.559456+0000","last_became_peered":"2024-03-04T07:33:56.559456+0000","last_unstale":"2024-03-04T07:33:57.859421+0000","last_undegraded":"2024-03-04T07:33:57.859421+0000","last_fullsized":"2024-03-04T07:33:57.859421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:18:43.135906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.531554+0000","last_change":"2024-03-04T07:33:55.550625+0000","last_active":"2024-03-04T07:34:01.531554+0000","last_peered":"2024-03-04T07:34:01.531554+0000","last_clean":"2024-03-04T07:34:01.531554+0000","last_became_active":"2024-03-04T07:33:55.550327+0000","last_became_peered":"2024-03-04T07:33:55.550327+0000","last_unstale":"2024-03-04T07:34:01.531554+0000","last_undegraded":"2024-03-04T07:34:01.531554+0000","last_fullsized":"2024-03-04T07:34:01.531554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:20.282585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526558+0000","last_change":"2024-03-04T07:33:56.537570+0000","last_active":"2024-03-04T07:33:57.526558+0000","last_peered":"2024-03-04T07:33:57.526558+0000","last_clean":"2024-03-04T07:33:57.526558+0000","last_became_active":"2024-03-04T07:33:56.536870+0000","last_became_peered":"2024-03-04T07:33:56.536870+0000","last_unstale":"2024-03-04T07:33:57.526558+0000","last_undegraded":"2024-03-04T07:33:57.526558+0000","last_fullsized":"2024-03-04T07:33:57.526558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:59:23.807432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.536570+0000","last_change":"2024-03-04T07:33:55.538915+0000","last_active":"2024-03-04T07:34:01.536570+0000","last_peered":"2024-03-04T07:34:01.536570+0000","last_clean":"2024-03-04T07:34:01.536570+0000","last_became_active":"2024-03-04T07:33:55.536263+0000","last_became_peered":"2024-03-04T07:33:55.536263+0000","last_unstale":"2024-03-04T07:34:01.536570+0000","last_undegraded":"2024-03-04T07:34:01.536570+0000","last_fullsized":"2024-03-04T07:34:01.536570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:09.612704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643901+0000","last_change":"2024-03-04T07:33:56.551087+0000","last_active":"2024-03-04T07:33:58.643901+0000","last_peered":"2024-03-04T07:33:58.643901+0000","last_clean":"2024-03-04T07:33:58.643901+0000","last_became_active":"2024-03-04T07:33:56.550767+0000","last_became_peered":"2024-03-04T07:33:56.550767+0000","last_unstale":"2024-03-04T07:33:58.643901+0000","last_undegraded":"2024-03-04T07:33:58.643901+0000","last_fullsized":"2024-03-04T07:33:58.643901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:15.603800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.532771+0000","last_change":"2024-03-04T07:33:55.560382+0000","last_active":"2024-03-04T07:34:01.532771+0000","last_peered":"2024-03-04T07:34:01.532771+0000","last_clean":"2024-03-04T07:34:01.532771+0000","last_became_active":"2024-03-04T07:33:55.560078+0000","last_became_peered":"2024-03-04T07:33:55.560078+0000","last_unstale":"2024-03-04T07:34:01.532771+0000","last_undegraded":"2024-03-04T07:34:01.532771+0000","last_fullsized":"2024-03-04T07:34:01.532771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:41:10.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859054+0000","last_change":"2024-03-04T07:33:56.537920+0000","last_active":"2024-03-04T07:33:57.859054+0000","last_peered":"2024-03-04T07:33:57.859054+0000","last_clean":"2024-03-04T07:33:57.859054+0000","last_became_active":"2024-03-04T07:33:56.535960+0000","last_became_peered":"2024-03-04T07:33:56.535960+0000","last_unstale":"2024-03-04T07:33:57.859054+0000","last_undegraded":"2024-03-04T07:33:57.859054+0000","last_fullsized":"2024-03-04T07:33:57.859054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:37:09.344120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642382+0000","last_change":"2024-03-04T07:33:55.567682+0000","last_active":"2024-03-04T07:33:58.642382+0000","last_peered":"2024-03-04T07:33:58.642382+0000","last_clean":"2024-03-04T07:33:58.642382+0000","last_became_active":"2024-03-04T07:33:55.566915+0000","last_became_peered":"2024-03-04T07:33:55.566915+0000","last_unstale":"2024-03-04T07:33:58.642382+0000","last_undegraded":"2024-03-04T07:33:58.642382+0000","last_fullsized":"2024-03-04T07:33:58.642382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:33:47.010346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644757+0000","last_change":"2024-03-04T07:33:56.547846+0000","last_active":"2024-03-04T07:33:58.644757+0000","last_peered":"2024-03-04T07:33:58.644757+0000","last_clean":"2024-03-04T07:33:58.644757+0000","last_became_active":"2024-03-04T07:33:56.547459+0000","last_became_peered":"2024-03-04T07:33:56.547459+0000","last_unstale":"2024-03-04T07:33:58.644757+0000","last_undegraded":"2024-03-04T07:33:58.644757+0000","last_fullsized":"2024-03-04T07:33:58.644757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:23:57.623123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640665+0000","last_change":"2024-03-04T07:33:55.557041+0000","last_active":"2024-03-04T07:33:58.640665+0000","last_peered":"2024-03-04T07:33:58.640665+0000","last_clean":"2024-03-04T07:33:58.640665+0000","last_became_active":"2024-03-04T07:33:55.556416+0000","last_became_peered":"2024-03-04T07:33:55.556416+0000","last_unstale":"2024-03-04T07:33:58.640665+0000","last_undegraded":"2024-03-04T07:33:58.640665+0000","last_fullsized":"2024-03-04T07:33:58.640665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:31.472235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641078+0000","last_change":"2024-03-04T07:33:56.531986+0000","last_active":"2024-03-04T07:33:58.641078+0000","last_peered":"2024-03-04T07:33:58.641078+0000","last_clean":"2024-03-04T07:33:58.641078+0000","last_became_active":"2024-03-04T07:33:56.531565+0000","last_became_peered":"2024-03-04T07:33:56.531565+0000","last_unstale":"2024-03-04T07:33:58.641078+0000","last_undegraded":"2024-03-04T07:33:58.641078+0000","last_fullsized":"2024-03-04T07:33:58.641078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:42:35.040979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.533357+0000","last_change":"2024-03-04T07:33:55.538469+0000","last_active":"2024-03-04T07:34:01.533357+0000","last_peered":"2024-03-04T07:34:01.533357+0000","last_clean":"2024-03-04T07:34:01.533357+0000","last_became_active":"2024-03-04T07:33:55.538165+0000","last_became_peered":"2024-03-04T07:33:55.538165+0000","last_unstale":"2024-03-04T07:34:01.533357+0000","last_undegraded":"2024-03-04T07:34:01.533357+0000","last_fullsized":"2024-03-04T07:34:01.533357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:00:01.263768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643331+0000","last_change":"2024-03-04T07:33:56.547881+0000","last_active":"2024-03-04T07:33:58.643331+0000","last_peered":"2024-03-04T07:33:58.643331+0000","last_clean":"2024-03-04T07:33:58.643331+0000","last_became_active":"2024-03-04T07:33:56.547461+0000","last_became_peered":"2024-03-04T07:33:56.547461+0000","last_unstale":"2024-03-04T07:33:58.643331+0000","last_undegraded":"2024-03-04T07:33:58.643331+0000","last_fullsized":"2024-03-04T07:33:58.643331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:34:32.709430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526523+0000","last_change":"2024-03-04T07:33:55.544356+0000","last_active":"2024-03-04T07:33:57.526523+0000","last_peered":"2024-03-04T07:33:57.526523+0000","last_clean":"2024-03-04T07:33:57.526523+0000","last_became_active":"2024-03-04T07:33:55.541295+0000","last_became_peered":"2024-03-04T07:33:55.541295+0000","last_unstale":"2024-03-04T07:33:57.526523+0000","last_undegraded":"2024-03-04T07:33:57.526523+0000","last_fullsized":"2024-03-04T07:33:57.526523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:22.953982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645470+0000","last_change":"2024-03-04T07:33:56.549038+0000","last_active":"2024-03-04T07:33:58.645470+0000","last_peered":"2024-03-04T07:33:58.645470+0000","last_clean":"2024-03-04T07:33:58.645470+0000","last_became_active":"2024-03-04T07:33:56.548113+0000","last_became_peered":"2024-03-04T07:33:56.548113+0000","last_unstale":"2024-03-04T07:33:58.645470+0000","last_undegraded":"2024-03-04T07:33:58.645470+0000","last_fullsized":"2024-03-04T07:33:58.645470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:53:04.601075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.530724+0000","last_change":"2024-03-04T07:33:55.538871+0000","last_active":"2024-03-04T07:34:01.530724+0000","last_peered":"2024-03-04T07:34:01.530724+0000","last_clean":"2024-03-04T07:34:01.530724+0000","last_became_active":"2024-03-04T07:33:55.537221+0000","last_became_peered":"2024-03-04T07:33:55.537221+0000","last_unstale":"2024-03-04T07:34:01.530724+0000","last_undegraded":"2024-03-04T07:34:01.530724+0000","last_fullsized":"2024-03-04T07:34:01.530724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:02:34.533330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645792+0000","last_change":"2024-03-04T07:33:56.543760+0000","last_active":"2024-03-04T07:33:58.645792+0000","last_peered":"2024-03-04T07:33:58.645792+0000","last_clean":"2024-03-04T07:33:58.645792+0000","last_became_active":"2024-03-04T07:33:56.543354+0000","last_became_peered":"2024-03-04T07:33:56.543354+0000","last_unstale":"2024-03-04T07:33:58.645792+0000","last_undegraded":"2024-03-04T07:33:58.645792+0000","last_fullsized":"2024-03-04T07:33:58.645792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:19:37.564790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.913391+0000","last_change":"2024-03-04T07:33:55.552856+0000","last_active":"2024-03-04T07:33:57.913391+0000","last_peered":"2024-03-04T07:33:57.913391+0000","last_clean":"2024-03-04T07:33:57.913391+0000","last_became_active":"2024-03-04T07:33:55.552472+0000","last_became_peered":"2024-03-04T07:33:55.552472+0000","last_unstale":"2024-03-04T07:33:57.913391+0000","last_undegraded":"2024-03-04T07:33:57.913391+0000","last_fullsized":"2024-03-04T07:33:57.913391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:36:50.409567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643132+0000","last_change":"2024-03-04T07:33:56.558497+0000","last_active":"2024-03-04T07:33:58.643132+0000","last_peered":"2024-03-04T07:33:58.643132+0000","last_clean":"2024-03-04T07:33:58.643132+0000","last_became_active":"2024-03-04T07:33:56.557989+0000","last_became_peered":"2024-03-04T07:33:56.557989+0000","last_unstale":"2024-03-04T07:33:58.643132+0000","last_undegraded":"2024-03-04T07:33:58.643132+0000","last_fullsized":"2024-03-04T07:33:58.643132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:57:05.380755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522764+0000","last_change":"2024-03-04T07:33:55.560332+0000","last_active":"2024-03-04T07:33:57.522764+0000","last_peered":"2024-03-04T07:33:57.522764+0000","last_clean":"2024-03-04T07:33:57.522764+0000","last_became_active":"2024-03-04T07:33:55.560086+0000","last_became_peered":"2024-03-04T07:33:55.560086+0000","last_unstale":"2024-03-04T07:33:57.522764+0000","last_undegraded":"2024-03-04T07:33:57.522764+0000","last_fullsized":"2024-03-04T07:33:57.522764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:25:46.280657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644296+0000","last_change":"2024-03-04T07:33:56.559881+0000","last_active":"2024-03-04T07:33:58.644296+0000","last_peered":"2024-03-04T07:33:58.644296+0000","last_clean":"2024-03-04T07:33:58.644296+0000","last_became_active":"2024-03-04T07:33:56.559589+0000","last_became_peered":"2024-03-04T07:33:56.559589+0000","last_unstale":"2024-03-04T07:33:58.644296+0000","last_undegraded":"2024-03-04T07:33:58.644296+0000","last_fullsized":"2024-03-04T07:33:58.644296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:47:35.123163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640654+0000","last_change":"2024-03-04T07:33:43.559630+0000","last_active":"2024-03-04T07:33:58.640654+0000","last_peered":"2024-03-04T07:33:58.640654+0000","last_clean":"2024-03-04T07:33:58.640654+0000","last_became_active":"2024-03-04T07:33:43.557489+0000","last_became_peered":"2024-03-04T07:33:43.557489+0000","last_unstale":"2024-03-04T07:33:58.640654+0000","last_undegraded":"2024-03-04T07:33:58.640654+0000","last_fullsized":"2024-03-04T07:33:58.640654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_clean_scrub_stamp":"2024-03-04T07:33:42.183823+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:55:35.762857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641306+0000","last_change":"2024-03-04T07:33:55.548616+0000","last_active":"2024-03-04T07:33:58.641306+0000","last_peered":"2024-03-04T07:33:58.641306+0000","last_clean":"2024-03-04T07:33:58.641306+0000","last_became_active":"2024-03-04T07:33:55.547322+0000","last_became_peered":"2024-03-04T07:33:55.547322+0000","last_unstale":"2024-03-04T07:33:58.641306+0000","last_undegraded":"2024-03-04T07:33:58.641306+0000","last_fullsized":"2024-03-04T07:33:58.641306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:20:16.717483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523877+0000","last_change":"2024-03-04T07:33:56.559966+0000","last_active":"2024-03-04T07:33:57.523877+0000","last_peered":"2024-03-04T07:33:57.523877+0000","last_clean":"2024-03-04T07:33:57.523877+0000","last_became_active":"2024-03-04T07:33:56.559581+0000","last_became_peered":"2024-03-04T07:33:56.559581+0000","last_unstale":"2024-03-04T07:33:57.523877+0000","last_undegraded":"2024-03-04T07:33:57.523877+0000","last_fullsized":"2024-03-04T07:33:57.523877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640360+0000","last_change":"2024-03-04T07:33:55.563905+0000","last_active":"2024-03-04T07:33:58.640360+0000","last_peered":"2024-03-04T07:33:58.640360+0000","last_clean":"2024-03-04T07:33:58.640360+0000","last_became_active":"2024-03-04T07:33:55.563117+0000","last_became_peered":"2024-03-04T07:33:55.563117+0000","last_unstale":"2024-03-04T07:33:58.640360+0000","last_undegraded":"2024-03-04T07:33:58.640360+0000","last_fullsized":"2024-03-04T07:33:58.640360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:53:02.522757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.858965+0000","last_change":"2024-03-04T07:33:56.531080+0000","last_active":"2024-03-04T07:33:57.858965+0000","last_peered":"2024-03-04T07:33:57.858965+0000","last_clean":"2024-03-04T07:33:57.858965+0000","last_became_active":"2024-03-04T07:33:56.530776+0000","last_became_peered":"2024-03-04T07:33:56.530776+0000","last_unstale":"2024-03-04T07:33:57.858965+0000","last_undegraded":"2024-03-04T07:33:57.858965+0000","last_fullsized":"2024-03-04T07:33:57.858965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:29:14.385057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522732+0000","last_change":"2024-03-04T07:33:55.538631+0000","last_active":"2024-03-04T07:33:57.522732+0000","last_peered":"2024-03-04T07:33:57.522732+0000","last_clean":"2024-03-04T07:33:57.522732+0000","last_became_active":"2024-03-04T07:33:55.537104+0000","last_became_peered":"2024-03-04T07:33:55.537104+0000","last_unstale":"2024-03-04T07:33:57.522732+0000","last_undegraded":"2024-03-04T07:33:57.522732+0000","last_fullsized":"2024-03-04T07:33:57.522732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:27:06.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749962+0000","last_change":"2024-03-04T07:33:56.541335+0000","last_active":"2024-03-04T07:33:57.749962+0000","last_peered":"2024-03-04T07:33:57.749962+0000","last_clean":"2024-03-04T07:33:57.749962+0000","last_became_active":"2024-03-04T07:33:56.540536+0000","last_became_peered":"2024-03-04T07:33:56.540536+0000","last_unstale":"2024-03-04T07:33:57.749962+0000","last_undegraded":"2024-03-04T07:33:57.749962+0000","last_fullsized":"2024-03-04T07:33:57.749962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:31:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642643+0000","last_change":"2024-03-04T07:33:55.574740+0000","last_active":"2024-03-04T07:33:58.642643+0000","last_peered":"2024-03-04T07:33:58.642643+0000","last_clean":"2024-03-04T07:33:58.642643+0000","last_became_active":"2024-03-04T07:33:55.574129+0000","last_became_peered":"2024-03-04T07:33:55.574129+0000","last_unstale":"2024-03-04T07:33:58.642643+0000","last_undegraded":"2024-03-04T07:33:58.642643+0000","last_fullsized":"2024-03-04T07:33:58.642643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:40:57.891803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526225+0000","last_change":"2024-03-04T07:33:56.541654+0000","last_active":"2024-03-04T07:33:57.526225+0000","last_peered":"2024-03-04T07:33:57.526225+0000","last_clean":"2024-03-04T07:33:57.526225+0000","last_became_active":"2024-03-04T07:33:56.541422+0000","last_became_peered":"2024-03-04T07:33:56.541422+0000","last_unstale":"2024-03-04T07:33:57.526225+0000","last_undegraded":"2024-03-04T07:33:57.526225+0000","last_fullsized":"2024-03-04T07:33:57.526225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:38.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750122+0000","last_change":"2024-03-04T07:33:55.539317+0000","last_active":"2024-03-04T07:33:57.750122+0000","last_peered":"2024-03-04T07:33:57.750122+0000","last_clean":"2024-03-04T07:33:57.750122+0000","last_became_active":"2024-03-04T07:33:55.539017+0000","last_became_peered":"2024-03-04T07:33:55.539017+0000","last_unstale":"2024-03-04T07:33:57.750122+0000","last_undegraded":"2024-03-04T07:33:57.750122+0000","last_fullsized":"2024-03-04T07:33:57.750122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:10.787316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642464+0000","last_change":"2024-03-04T07:33:56.554103+0000","last_active":"2024-03-04T07:33:58.642464+0000","last_peered":"2024-03-04T07:33:58.642464+0000","last_clean":"2024-03-04T07:33:58.642464+0000","last_became_active":"2024-03-04T07:33:56.553742+0000","last_became_peered":"2024-03-04T07:33:56.553742+0000","last_unstale":"2024-03-04T07:33:58.642464+0000","last_undegraded":"2024-03-04T07:33:58.642464+0000","last_fullsized":"2024-03-04T07:33:58.642464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:53.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644822+0000","last_change":"2024-03-04T07:33:55.569442+0000","last_active":"2024-03-04T07:33:58.644822+0000","last_peered":"2024-03-04T07:33:58.644822+0000","last_clean":"2024-03-04T07:33:58.644822+0000","last_became_active":"2024-03-04T07:33:55.567809+0000","last_became_peered":"2024-03-04T07:33:55.567809+0000","last_unstale":"2024-03-04T07:33:58.644822+0000","last_undegraded":"2024-03-04T07:33:58.644822+0000","last_fullsized":"2024-03-04T07:33:58.644822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:58.551866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750083+0000","last_change":"2024-03-04T07:33:56.542997+0000","last_active":"2024-03-04T07:33:57.750083+0000","last_peered":"2024-03-04T07:33:57.750083+0000","last_clean":"2024-03-04T07:33:57.750083+0000","last_became_active":"2024-03-04T07:33:56.542775+0000","last_became_peered":"2024-03-04T07:33:56.542775+0000","last_unstale":"2024-03-04T07:33:57.750083+0000","last_undegraded":"2024-03-04T07:33:57.750083+0000","last_fullsized":"2024-03-04T07:33:57.750083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:32.090568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750225+0000","last_change":"2024-03-04T07:33:55.545598+0000","last_active":"2024-03-04T07:33:57.750225+0000","last_peered":"2024-03-04T07:33:57.750225+0000","last_clean":"2024-03-04T07:33:57.750225+0000","last_became_active":"2024-03-04T07:33:55.545262+0000","last_became_peered":"2024-03-04T07:33:55.545262+0000","last_unstale":"2024-03-04T07:33:57.750225+0000","last_undegraded":"2024-03-04T07:33:57.750225+0000","last_fullsized":"2024-03-04T07:33:57.750225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:01.667243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523716+0000","last_change":"2024-03-04T07:33:56.550943+0000","last_active":"2024-03-04T07:33:57.523716+0000","last_peered":"2024-03-04T07:33:57.523716+0000","last_clean":"2024-03-04T07:33:57.523716+0000","last_became_active":"2024-03-04T07:33:56.550604+0000","last_became_peered":"2024-03-04T07:33:56.550604+0000","last_unstale":"2024-03-04T07:33:57.523716+0000","last_undegraded":"2024-03-04T07:33:57.523716+0000","last_fullsized":"2024-03-04T07:33:57.523716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:01:05.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.536925+0000","last_change":"2024-03-04T07:33:55.567220+0000","last_active":"2024-03-04T07:34:01.536925+0000","last_peered":"2024-03-04T07:34:01.536925+0000","last_clean":"2024-03-04T07:34:01.536925+0000","last_became_active":"2024-03-04T07:33:55.566600+0000","last_became_peered":"2024-03-04T07:33:55.566600+0000","last_unstale":"2024-03-04T07:34:01.536925+0000","last_undegraded":"2024-03-04T07:34:01.536925+0000","last_fullsized":"2024-03-04T07:34:01.536925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.876938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859267+0000","last_change":"2024-03-04T07:33:56.544263+0000","last_active":"2024-03-04T07:33:57.859267+0000","last_peered":"2024-03-04T07:33:57.859267+0000","last_clean":"2024-03-04T07:33:57.859267+0000","last_became_active":"2024-03-04T07:33:56.543836+0000","last_became_peered":"2024-03-04T07:33:56.543836+0000","last_unstale":"2024-03-04T07:33:57.859267+0000","last_undegraded":"2024-03-04T07:33:57.859267+0000","last_fullsized":"2024-03-04T07:33:57.859267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:49.839824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640953+0000","last_change":"2024-03-04T07:33:55.566886+0000","last_active":"2024-03-04T07:33:58.640953+0000","last_peered":"2024-03-04T07:33:58.640953+0000","last_clean":"2024-03-04T07:33:58.640953+0000","last_became_active":"2024-03-04T07:33:55.566048+0000","last_became_peered":"2024-03-04T07:33:55.566048+0000","last_unstale":"2024-03-04T07:33:58.640953+0000","last_undegraded":"2024-03-04T07:33:58.640953+0000","last_fullsized":"2024-03-04T07:33:58.640953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:45:27.190438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.525876+0000","last_change":"2024-03-04T07:33:56.532666+0000","last_active":"2024-03-04T07:33:57.525876+0000","last_peered":"2024-03-04T07:33:57.525876+0000","last_clean":"2024-03-04T07:33:57.525876+0000","last_became_active":"2024-03-04T07:33:56.531901+0000","last_became_peered":"2024-03-04T07:33:56.531901+0000","last_unstale":"2024-03-04T07:33:57.525876+0000","last_undegraded":"2024-03-04T07:33:57.525876+0000","last_fullsized":"2024-03-04T07:33:57.525876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:42:04.545881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750078+0000","last_change":"2024-03-04T07:33:55.557568+0000","last_active":"2024-03-04T07:33:57.750078+0000","last_peered":"2024-03-04T07:33:57.750078+0000","last_clean":"2024-03-04T07:33:57.750078+0000","last_became_active":"2024-03-04T07:33:55.557330+0000","last_became_peered":"2024-03-04T07:33:55.557330+0000","last_unstale":"2024-03-04T07:33:57.750078+0000","last_undegraded":"2024-03-04T07:33:57.750078+0000","last_fullsized":"2024-03-04T07:33:57.750078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:34.956318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523904+0000","last_change":"2024-03-04T07:33:56.552963+0000","last_active":"2024-03-04T07:33:57.523904+0000","last_peered":"2024-03-04T07:33:57.523904+0000","last_clean":"2024-03-04T07:33:57.523904+0000","last_became_active":"2024-03-04T07:33:56.552658+0000","last_became_peered":"2024-03-04T07:33:56.552658+0000","last_unstale":"2024-03-04T07:33:57.523904+0000","last_undegraded":"2024-03-04T07:33:57.523904+0000","last_fullsized":"2024-03-04T07:33:57.523904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:05.685496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750081+0000","last_change":"2024-03-04T07:33:55.536480+0000","last_active":"2024-03-04T07:33:57.750081+0000","last_peered":"2024-03-04T07:33:57.750081+0000","last_clean":"2024-03-04T07:33:57.750081+0000","last_became_active":"2024-03-04T07:33:55.536139+0000","last_became_peered":"2024-03-04T07:33:55.536139+0000","last_unstale":"2024-03-04T07:33:57.750081+0000","last_undegraded":"2024-03-04T07:33:57.750081+0000","last_fullsized":"2024-03-04T07:33:57.750081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:12.778026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523811+0000","last_change":"2024-03-04T07:33:56.546546+0000","last_active":"2024-03-04T07:33:57.523811+0000","last_peered":"2024-03-04T07:33:57.523811+0000","last_clean":"2024-03-04T07:33:57.523811+0000","last_became_active":"2024-03-04T07:33:56.546280+0000","last_became_peered":"2024-03-04T07:33:56.546280+0000","last_unstale":"2024-03-04T07:33:57.523811+0000","last_undegraded":"2024-03-04T07:33:57.523811+0000","last_fullsized":"2024-03-04T07:33:57.523811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:19:55.950163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.534127+0000","last_change":"2024-03-04T07:33:55.572540+0000","last_active":"2024-03-04T07:34:01.534127+0000","last_peered":"2024-03-04T07:34:01.534127+0000","last_clean":"2024-03-04T07:34:01.534127+0000","last_became_active":"2024-03-04T07:33:55.571612+0000","last_became_peered":"2024-03-04T07:33:55.571612+0000","last_unstale":"2024-03-04T07:34:01.534127+0000","last_undegraded":"2024-03-04T07:34:01.534127+0000","last_fullsized":"2024-03-04T07:34:01.534127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:01:43.741423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640728+0000","last_change":"2024-03-04T07:33:56.532855+0000","last_active":"2024-03-04T07:33:58.640728+0000","last_peered":"2024-03-04T07:33:58.640728+0000","last_clean":"2024-03-04T07:33:58.640728+0000","last_became_active":"2024-03-04T07:33:56.532245+0000","last_became_peered":"2024-03-04T07:33:56.532245+0000","last_unstale":"2024-03-04T07:33:58.640728+0000","last_undegraded":"2024-03-04T07:33:58.640728+0000","last_fullsized":"2024-03-04T07:33:58.640728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:26.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642056+0000","last_change":"2024-03-04T07:33:56.563803+0000","last_active":"2024-03-04T07:33:58.642056+0000","last_peered":"2024-03-04T07:33:58.642056+0000","last_clean":"2024-03-04T07:33:58.642056+0000","last_became_active":"2024-03-04T07:33:56.563473+0000","last_became_peered":"2024-03-04T07:33:56.563473+0000","last_unstale":"2024-03-04T07:33:58.642056+0000","last_undegraded":"2024-03-04T07:33:58.642056+0000","last_fullsized":"2024-03-04T07:33:58.642056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:26.904436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.531549+0000","last_change":"2024-03-04T07:33:55.575688+0000","last_active":"2024-03-04T07:34:01.531549+0000","last_peered":"2024-03-04T07:34:01.531549+0000","last_clean":"2024-03-04T07:34:01.531549+0000","last_became_active":"2024-03-04T07:33:55.574766+0000","last_became_peered":"2024-03-04T07:33:55.574766+0000","last_unstale":"2024-03-04T07:34:01.531549+0000","last_undegraded":"2024-03-04T07:34:01.531549+0000","last_fullsized":"2024-03-04T07:34:01.531549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:32:19.708016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642750+0000","last_change":"2024-03-04T07:33:56.563798+0000","last_active":"2024-03-04T07:33:58.642750+0000","last_peered":"2024-03-04T07:33:58.642750+0000","last_clean":"2024-03-04T07:33:58.642750+0000","last_became_active":"2024-03-04T07:33:56.563474+0000","last_became_peered":"2024-03-04T07:33:56.563474+0000","last_unstale":"2024-03-04T07:33:58.642750+0000","last_undegraded":"2024-03-04T07:33:58.642750+0000","last_fullsized":"2024-03-04T07:33:58.642750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:47:39.763001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.535125+0000","last_change":"2024-03-04T07:33:55.570016+0000","last_active":"2024-03-04T07:34:01.535125+0000","last_peered":"2024-03-04T07:34:01.535125+0000","last_clean":"2024-03-04T07:34:01.535125+0000","last_became_active":"2024-03-04T07:33:55.568749+0000","last_became_peered":"2024-03-04T07:33:55.568749+0000","last_unstale":"2024-03-04T07:34:01.535125+0000","last_undegraded":"2024-03-04T07:34:01.535125+0000","last_fullsized":"2024-03-04T07:34:01.535125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:01.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859463+0000","last_change":"2024-03-04T07:33:56.562015+0000","last_active":"2024-03-04T07:33:57.859463+0000","last_peered":"2024-03-04T07:33:57.859463+0000","last_clean":"2024-03-04T07:33:57.859463+0000","last_became_active":"2024-03-04T07:33:56.561675+0000","last_became_peered":"2024-03-04T07:33:56.561675+0000","last_unstale":"2024-03-04T07:33:57.859463+0000","last_undegraded":"2024-03-04T07:33:57.859463+0000","last_fullsized":"2024-03-04T07:33:57.859463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:36:20.241414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.918424+0000","last_change":"2024-03-04T07:33:55.538337+0000","last_active":"2024-03-04T07:33:57.918424+0000","last_peered":"2024-03-04T07:33:57.918424+0000","last_clean":"2024-03-04T07:33:57.918424+0000","last_became_active":"2024-03-04T07:33:55.538010+0000","last_became_peered":"2024-03-04T07:33:55.538010+0000","last_unstale":"2024-03-04T07:33:57.918424+0000","last_undegraded":"2024-03-04T07:33:57.918424+0000","last_fullsized":"2024-03-04T07:33:57.918424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:47:15.711205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.524079+0000","last_change":"2024-03-04T07:33:56.559051+0000","last_active":"2024-03-04T07:33:57.524079+0000","last_peered":"2024-03-04T07:33:57.524079+0000","last_clean":"2024-03-04T07:33:57.524079+0000","last_became_active":"2024-03-04T07:33:56.558799+0000","last_became_peered":"2024-03-04T07:33:56.558799+0000","last_unstale":"2024-03-04T07:33:57.524079+0000","last_undegraded":"2024-03-04T07:33:57.524079+0000","last_fullsized":"2024-03-04T07:33:57.524079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:16.516631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.536473+0000","last_change":"2024-03-04T07:33:55.566294+0000","last_active":"2024-03-04T07:34:01.536473+0000","last_peered":"2024-03-04T07:34:01.536473+0000","last_clean":"2024-03-04T07:34:01.536473+0000","last_became_active":"2024-03-04T07:33:55.565941+0000","last_became_peered":"2024-03-04T07:33:55.565941+0000","last_unstale":"2024-03-04T07:34:01.536473+0000","last_undegraded":"2024-03-04T07:34:01.536473+0000","last_fullsized":"2024-03-04T07:34:01.536473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:03.115085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":217088,"data_stored":73689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":806912,"data_stored":664479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":212992,"data_stored":73619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":73655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":815104,"data_stored":664535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T07:34:07.653 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-04T07:34:07.653 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-04T07:34:07.653 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-04T07:34:07.653 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-04T07:34:07.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe19b200640 1 Processor -- start 2024-03-04T07:34:07.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe19b200640 1 -- start start 2024-03-04T07:34:07.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe19b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 0x7fe1940c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe19b200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe1940cb2e0 0x7fe1940cb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe19b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1941598c0 con 0x7fe19406f540 2024-03-04T07:34:07.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe19b200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1940ca3a0 con 0x7fe1940cb2e0 2024-03-04T07:34:07.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe19b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940ca680 con 0x7fe194152b10 2024-03-04T07:34:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe193400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe19406f540 0x7fe1941591b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33276/0 (socket says 172.21.15.113:33276) 2024-03-04T07:34:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe193400640 1 -- 172.21.15.113:0/3638127722 learned_addr learned my addr 172.21.15.113:0/3638127722 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe193e00640 1 --2- 172.21.15.113:0/3638127722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 0x7fe1940c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe199200640 1 --2- 172.21.15.113:0/3638127722 >> v2:172.21.15.136:3300/0 conn(0x7fe1940cb2e0 0x7fe1940cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe192a00640 1 -- 172.21.15.113:0/3638127722 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872044201 0 0) 0x7fe1941598c0 con 0x7fe19406f540 2024-03-04T07:34:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe192a00640 1 -- 172.21.15.113:0/3638127722 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe178003530 con 0x7fe19406f540 2024-03-04T07:34:07.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe193e00640 1 -- 172.21.15.113:0/3638127722 >> v2:172.21.15.136:3300/0 conn(0x7fe1940cb2e0 msgr2=0x7fe1940cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:07.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe193e00640 1 --2- 172.21.15.113:0/3638127722 >> v2:172.21.15.136:3300/0 conn(0x7fe1940cb2e0 0x7fe1940cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe193e00640 1 -- 172.21.15.113:0/3638127722 >> v1:172.21.15.113:6789/0 conn(0x7fe19406f540 legacy=0x7fe1941591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.822+0000 7fe193e00640 1 -- 172.21.15.113:0/3638127722 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1940ca960 con 0x7fe194152b10 2024-03-04T07:34:07.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe193e00640 1 --2- 172.21.15.113:0/3638127722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 0x7fe1940c9860 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1840097c0 tx=0x7fe18401ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75b78504f05c8a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:07.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe192a00640 1 -- 172.21.15.113:0/3638127722 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840b5070 con 0x7fe194152b10 2024-03-04T07:34:07.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe192a00640 1 -- 172.21.15.113:0/3638127722 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1840a8990 con 0x7fe194152b10 2024-03-04T07:34:07.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe192a00640 1 -- 172.21.15.113:0/3638127722 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840a8d40 con 0x7fe194152b10 2024-03-04T07:34:07.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe19b200640 1 -- 172.21.15.113:0/3638127722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 msgr2=0x7fe1940c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe19b200640 1 --2- 172.21.15.113:0/3638127722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 0x7fe1940c9860 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe1840097c0 tx=0x7fe18401ec90 comp rx=0 tx=0).stop 2024-03-04T07:34:07.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe19b200640 1 -- 172.21.15.113:0/3638127722 shutdown_connections 2024-03-04T07:34:07.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe19b200640 1 --2- 172.21.15.113:0/3638127722 >> v2:172.21.15.136:3300/0 conn(0x7fe1940cb2e0 0x7fe1940cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe19b200640 1 --2- 172.21.15.113:0/3638127722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 0x7fe1940c9860 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe19b200640 1 -- 172.21.15.113:0/3638127722 >> 172.21.15.113:0/3638127722 conn(0x7fe194066f80 msgr2=0x7fe194067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:07.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe19b200640 1 -- 172.21.15.113:0/3638127722 shutdown_connections 2024-03-04T07:34:07.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.826+0000 7fe19b200640 1 -- 172.21.15.113:0/3638127722 wait complete. 2024-03-04T07:34:07.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 Processor -- start 2024-03-04T07:34:07.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 -- start start 2024-03-04T07:34:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe19406f540 0x7fe194150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1940cb2e0 0x7fe194150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 0x7fe1941f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe194158940 con 0x7fe19406f540 2024-03-04T07:34:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe194158c90 con 0x7fe1940cb2e0 2024-03-04T07:34:07.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe194070690 con 0x7fe194152b10 2024-03-04T07:34:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe193e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe19406f540 0x7fe194150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe193e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe19406f540 0x7fe194150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59248/0 (socket says 172.21.15.113:59248) 2024-03-04T07:34:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe193e00640 1 -- 172.21.15.113:0/599695597 learned_addr learned my addr 172.21.15.113:0/599695597 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe193400640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1940cb2e0 0x7fe194150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe199200640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 0x7fe1941f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe193e00640 1 -- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 msgr2=0x7fe1941f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe193e00640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 0x7fe1941f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe193e00640 1 -- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1940cb2e0 msgr2=0x7fe194150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe193e00640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1940cb2e0 0x7fe194150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:07.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe193e00640 1 -- 172.21.15.113:0/599695597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1940b0620 con 0x7fe19406f540 2024-03-04T07:34:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe193e00640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe19406f540 0x7fe194150a40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe1840b3040 tx=0x7fe18401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe18be00640 1 -- 172.21.15.113:0/599695597 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840c1070 con 0x7fe19406f540 2024-03-04T07:34:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe18be00640 1 -- 172.21.15.113:0/599695597 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe184004240 con 0x7fe19406f540 2024-03-04T07:34:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe18be00640 1 -- 172.21.15.113:0/599695597 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840b5040 con 0x7fe19406f540 2024-03-04T07:34:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 -- 172.21.15.113:0/599695597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1940b0890 con 0x7fe19406f540 2024-03-04T07:34:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 -- 172.21.15.113:0/599695597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe194150240 con 0x7fe19406f540 2024-03-04T07:34:07.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.830+0000 7fe19b200640 1 -- 172.21.15.113:0/599695597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe194075c00 con 0x7fe19406f540 2024-03-04T07:34:07.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.834+0000 7fe18be00640 1 -- 172.21.15.113:0/599695597 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1840c4070 con 0x7fe19406f540 2024-03-04T07:34:07.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.834+0000 7fe18be00640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe174066f90 0x7fe174069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:07.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.834+0000 7fe18be00640 1 -- 172.21.15.113:0/599695597 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe184122120 con 0x7fe19406f540 2024-03-04T07:34:07.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.834+0000 7fe193400640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe174066f90 0x7fe174069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:07.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.834+0000 7fe193400640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe174066f90 0x7fe174069450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1941586d0 tx=0x7fe180009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:07.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:07.838+0000 7fe18be00640 1 -- 172.21.15.113:0/599695597 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe1840187f0 con 0x7fe19406f540 2024-03-04T07:34:08.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.022+0000 7fe19b200640 1 -- 172.21.15.113:0/599695597 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe1940b0300 con 0x7fe19406f540 2024-03-04T07:34:08.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.022+0000 7fe18be00640 1 -- 172.21.15.113:0/599695597 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe1840f31b0 con 0x7fe19406f540 2024-03-04T07:34:08.026 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:08.026 INFO:teuthology.orchestra.run.smithi113.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-04T07:34:08.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 -- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe174066f90 msgr2=0x7fe174069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe174066f90 0x7fe174069450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1941586d0 tx=0x7fe180009040 comp rx=0 tx=0).stop 2024-03-04T07:34:08.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 -- 172.21.15.113:0/599695597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe19406f540 msgr2=0x7fe194150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe19406f540 0x7fe194150a40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe1840b3040 tx=0x7fe18401ba30 comp rx=0 tx=0).stop 2024-03-04T07:34:08.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 -- 172.21.15.113:0/599695597 shutdown_connections 2024-03-04T07:34:08.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe194152b10 0x7fe1941f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1940cb2e0 0x7fe194150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe174066f90 0x7fe174069450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 --2- 172.21.15.113:0/599695597 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe19406f540 0x7fe194150a40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 -- 172.21.15.113:0/599695597 >> 172.21.15.113:0/599695597 conn(0x7fe194066f80 msgr2=0x7fe194067360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:08.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 -- 172.21.15.113:0/599695597 shutdown_connections 2024-03-04T07:34:08.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.026+0000 7fe19b200640 1 -- 172.21.15.113:0/599695597 wait complete. 2024-03-04T07:34:08.047 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-04T07:34:08.047 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:34:08.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f24800640 1 Processor -- start 2024-03-04T07:34:08.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f24800640 1 -- start start 2024-03-04T07:34:08.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f24800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9f1c155100 0x7f9f1c0c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f24800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1c15ad30 0x7f9f1c15b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f24800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f1c15b6c0 con 0x7f9f1c06b060 2024-03-04T07:34:08.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f24800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1c0c8890 con 0x7f9f1c155100 2024-03-04T07:34:08.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f24800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c06b4b0 con 0x7f9f1c15ad30 2024-03-04T07:34:08.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f21e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9f1c155100 0x7f9f1c0c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f21400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9f1c06b060 0x7f9f1c0cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33290/0 (socket says 172.21.15.113:33290) 2024-03-04T07:34:08.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f21400640 1 -- 172.21.15.113:0/381842116 learned_addr learned my addr 172.21.15.113:0/381842116 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:08.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f22800640 1 --2- 172.21.15.113:0/381842116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1c15ad30 0x7f9f1c15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f21e00640 1 -- 172.21.15.113:0/381842116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1c15ad30 msgr2=0x7f9f1c15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:08.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f21e00640 1 --2- 172.21.15.113:0/381842116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1c15ad30 0x7f9f1c15b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f21e00640 1 -- 172.21.15.113:0/381842116 >> v1:172.21.15.113:6789/0 conn(0x7f9f1c06b060 legacy=0x7f9f1c0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f21e00640 1 -- 172.21.15.113:0/381842116 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c15efd0 con 0x7f9f1c155100 2024-03-04T07:34:08.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.178+0000 7f9f20a00640 1 -- 172.21.15.113:0/381842116 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772820877 0 0) 0x7f9f1c15b6c0 con 0x7f9f1c06b060 2024-03-04T07:34:08.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f21e00640 1 --2- 172.21.15.113:0/381842116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c155100 0x7f9f1c0c08d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097f0 tx=0x7f9f1001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de60d0d5d2dd5474 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:08.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f20a00640 1 -- 172.21.15.113:0/381842116 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100b2070 con 0x7f9f1c155100 2024-03-04T07:34:08.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f20a00640 1 -- 172.21.15.113:0/381842116 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f100ad640 con 0x7f9f1c155100 2024-03-04T07:34:08.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f20a00640 1 -- 172.21.15.113:0/381842116 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100ada10 con 0x7f9f1c155100 2024-03-04T07:34:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f24800640 1 -- 172.21.15.113:0/381842116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c155100 msgr2=0x7f9f1c0c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f24800640 1 --2- 172.21.15.113:0/381842116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c155100 0x7f9f1c0c08d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097f0 tx=0x7f9f1001c4a0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f24800640 1 -- 172.21.15.113:0/381842116 shutdown_connections 2024-03-04T07:34:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f24800640 1 --2- 172.21.15.113:0/381842116 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1c15ad30 0x7f9f1c15b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f24800640 1 --2- 172.21.15.113:0/381842116 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c155100 0x7f9f1c0c08d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f24800640 1 -- 172.21.15.113:0/381842116 >> 172.21.15.113:0/381842116 conn(0x7f9f1c06deb0 msgr2=0x7f9f1c06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f24800640 1 -- 172.21.15.113:0/381842116 shutdown_connections 2024-03-04T07:34:08.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.182+0000 7f9f24800640 1 -- 172.21.15.113:0/381842116 wait complete. 2024-03-04T07:34:08.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f24800640 1 Processor -- start 2024-03-04T07:34:08.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f24800640 1 -- start start 2024-03-04T07:34:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f24800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c06b060 0x7f9f1c148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f24800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1c155100 0x7f9f1c06f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f24800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f1c15ad30 0x7f9f1c06fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f24800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c15ebb0 con 0x7f9f1c06b060 2024-03-04T07:34:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f24800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0c9610 con 0x7f9f1c15ad30 2024-03-04T07:34:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f24800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c15ed30 con 0x7f9f1c155100 2024-03-04T07:34:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f21e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c06b060 0x7f9f1c148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f21e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c06b060 0x7f9f1c148e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59268/0 (socket says 172.21.15.113:59268) 2024-03-04T07:34:08.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f21e00640 1 -- 172.21.15.113:0/2501804427 learned_addr learned my addr 172.21.15.113:0/2501804427 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f21400640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1c155100 0x7f9f1c06f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f22800640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f1c15ad30 0x7f9f1c06fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f21e00640 1 -- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1c155100 msgr2=0x7f9f1c06f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f21e00640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1c155100 0x7f9f1c06f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f21e00640 1 -- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f1c15ad30 msgr2=0x7f9f1c06fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f21e00640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f1c15ad30 0x7f9f1c06fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f21e00640 1 -- 172.21.15.113:0/2501804427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0cb970 con 0x7f9f1c06b060 2024-03-04T07:34:08.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f22800640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f1c15ad30 0x7f9f1c06fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:08.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f21e00640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c06b060 0x7f9f1c148e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9f100b0040 tx=0x7f9f10019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:08.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f16000640 1 -- 172.21.15.113:0/2501804427 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100b2040 con 0x7f9f1c06b060 2024-03-04T07:34:08.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f16000640 1 -- 172.21.15.113:0/2501804427 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f100bc070 con 0x7f9f1c06b060 2024-03-04T07:34:08.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f24800640 1 -- 172.21.15.113:0/2501804427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c0cbc50 con 0x7f9f1c06b060 2024-03-04T07:34:08.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f16000640 1 -- 172.21.15.113:0/2501804427 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100b6670 con 0x7f9f1c06b060 2024-03-04T07:34:08.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.186+0000 7f9f24800640 1 -- 172.21.15.113:0/2501804427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c0cc210 con 0x7f9f1c06b060 2024-03-04T07:34:08.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.190+0000 7f9f24800640 1 -- 172.21.15.113:0/2501804427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1c075c00 con 0x7f9f1c06b060 2024-03-04T07:34:08.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.190+0000 7f9f16000640 1 -- 172.21.15.113:0/2501804427 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f100b6810 con 0x7f9f1c06b060 2024-03-04T07:34:08.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.190+0000 7f9f16000640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f00066f60 0x7f9f00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.190+0000 7f9f16000640 1 -- 172.21.15.113:0/2501804427 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f1011efb0 con 0x7f9f1c06b060 2024-03-04T07:34:08.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.190+0000 7f9f21400640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f00066f60 0x7f9f00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.190+0000 7f9f21400640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f00066f60 0x7f9f00069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c151c00 tx=0x7f9f0c007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:08.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.194+0000 7f9f16000640 1 -- 172.21.15.113:0/2501804427 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9f10016090 con 0x7f9f1c06b060 2024-03-04T07:34:08.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.350+0000 7f9f24800640 1 -- 172.21.15.113:0/2501804427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f1c0c8a20 con 0x7f9f1c06b060 2024-03-04T07:34:08.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.350+0000 7f9f16000640 1 -- 172.21.15.113:0/2501804427 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f9f100eff20 con 0x7f9f1c06b060 2024-03-04T07:34:08.354 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:08.354 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:57.517935+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:34:08.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.350+0000 7f9f24800640 1 -- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f00066f60 msgr2=0x7f9f00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.350+0000 7f9f24800640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f00066f60 0x7f9f00069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c151c00 tx=0x7f9f0c007d60 comp rx=0 tx=0).stop 2024-03-04T07:34:08.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.350+0000 7f9f24800640 1 -- 172.21.15.113:0/2501804427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c06b060 msgr2=0x7f9f1c148e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.350+0000 7f9f24800640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c06b060 0x7f9f1c148e40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9f100b0040 tx=0x7f9f10019240 comp rx=0 tx=0).stop 2024-03-04T07:34:08.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.354+0000 7f9f24800640 1 -- 172.21.15.113:0/2501804427 shutdown_connections 2024-03-04T07:34:08.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.354+0000 7f9f24800640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f1c15ad30 0x7f9f1c06fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.354+0000 7f9f24800640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1c155100 0x7f9f1c06f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.354+0000 7f9f24800640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f00066f60 0x7f9f00069420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.354+0000 7f9f24800640 1 --2- 172.21.15.113:0/2501804427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1c06b060 0x7f9f1c148e40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.354+0000 7f9f24800640 1 -- 172.21.15.113:0/2501804427 >> 172.21.15.113:0/2501804427 conn(0x7f9f1c06deb0 msgr2=0x7f9f1c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:08.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.354+0000 7f9f24800640 1 -- 172.21.15.113:0/2501804427 shutdown_connections 2024-03-04T07:34:08.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.354+0000 7f9f24800640 1 -- 172.21.15.113:0/2501804427 wait complete. 2024-03-04T07:34:08.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T07:33:41.507700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T07:33:53.680465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T07:33:54.892043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T07:34:08.375 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T07:34:08.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.510+0000 7fc6c7400640 1 Processor -- start 2024-03-04T07:34:08.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.510+0000 7fc6c7400640 1 -- start start 2024-03-04T07:34:08.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6c7400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc6c01519b0 0x7fc6c00c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6c7400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c01577d0 con 0x7fc6c0157cd0 2024-03-04T07:34:08.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6c7400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c00c7760 con 0x7fc6c00c19e0 2024-03-04T07:34:08.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6c7400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c00c7a40 con 0x7fc6c01519b0 2024-03-04T07:34:08.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6c4a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc6c01519b0 0x7fc6c00c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6c5400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc6c0157cd0 0x7fc6c0158100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33304/0 (socket says 172.21.15.113:33304) 2024-03-04T07:34:08.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6c5400640 1 -- 172.21.15.113:0/1873614322 learned_addr learned my addr 172.21.15.113:0/1873614322 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:08.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6bfe00640 1 -- 172.21.15.113:0/1873614322 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924726578 0 0) 0x7fc6c01577d0 con 0x7fc6c0157cd0 2024-03-04T07:34:08.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6bfe00640 1 -- 172.21.15.113:0/1873614322 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6a4003530 con 0x7fc6c0157cd0 2024-03-04T07:34:08.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6c4a00640 1 -- 172.21.15.113:0/1873614322 >> v1:172.21.15.136:6789/0 conn(0x7fc6c00c19e0 legacy=0x7fc6c00cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6c4a00640 1 -- 172.21.15.113:0/1873614322 >> v1:172.21.15.113:6789/0 conn(0x7fc6c0157cd0 legacy=0x7fc6c0158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.514+0000 7fc6c4a00640 1 -- 172.21.15.113:0/1873614322 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c00c1090 con 0x7fc6c01519b0 2024-03-04T07:34:08.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c4a00640 1 --2- 172.21.15.113:0/1873614322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c01519b0 0x7fc6c00c8370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00097c0 tx=0x7fc6b001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad28d8a1f4ac43a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:08.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6bfe00640 1 -- 172.21.15.113:0/1873614322 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00aa600 con 0x7fc6c01519b0 2024-03-04T07:34:08.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6bfe00640 1 -- 172.21.15.113:0/1873614322 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b00af070 con 0x7fc6c01519b0 2024-03-04T07:34:08.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6bfe00640 1 -- 172.21.15.113:0/1873614322 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00aa9a0 con 0x7fc6c01519b0 2024-03-04T07:34:08.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 -- 172.21.15.113:0/1873614322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c01519b0 msgr2=0x7fc6c00c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 --2- 172.21.15.113:0/1873614322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c01519b0 0x7fc6c00c8370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00097c0 tx=0x7fc6b001c4a0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 -- 172.21.15.113:0/1873614322 shutdown_connections 2024-03-04T07:34:08.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 --2- 172.21.15.113:0/1873614322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c01519b0 0x7fc6c00c8370 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 -- 172.21.15.113:0/1873614322 >> 172.21.15.113:0/1873614322 conn(0x7fc6c0066db0 msgr2=0x7fc6c00671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:08.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 -- 172.21.15.113:0/1873614322 shutdown_connections 2024-03-04T07:34:08.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 -- 172.21.15.113:0/1873614322 wait complete. 2024-03-04T07:34:08.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 Processor -- start 2024-03-04T07:34:08.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 -- start start 2024-03-04T07:34:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6c00c19e0 0x7fc6c00c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.518+0000 7fc6c7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c01519b0 0x7fc6c01f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6c0157cd0 0x7fc6c01f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c7400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00ccc80 con 0x7fc6c00c19e0 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c7400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c00ccb00 con 0x7fc6c01519b0 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c7400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c0151d90 con 0x7fc6c0157cd0 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c4a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6c00c19e0 0x7fc6c00c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c4a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6c00c19e0 0x7fc6c00c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59280/0 (socket says 172.21.15.113:59280) 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c4a00640 1 -- 172.21.15.113:0/1333951443 learned_addr learned my addr 172.21.15.113:0/1333951443 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6b7e00640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c01519b0 0x7fc6c01f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c5400640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6c0157cd0 0x7fc6c01f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c4a00640 1 -- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6c0157cd0 msgr2=0x7fc6c01f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c4a00640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6c0157cd0 0x7fc6c01f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c4a00640 1 -- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c01519b0 msgr2=0x7fc6c01f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c4a00640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c01519b0 0x7fc6c01f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c4a00640 1 -- 172.21.15.113:0/1333951443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c014bc60 con 0x7fc6c00c19e0 2024-03-04T07:34:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c4a00640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6c00c19e0 0x7fc6c00c6770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc6b001ab50 tx=0x7fc6b0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6bd600640 1 -- 172.21.15.113:0/1333951443 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00c0070 con 0x7fc6c00c19e0 2024-03-04T07:34:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6bd600640 1 -- 172.21.15.113:0/1333951443 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b0003820 con 0x7fc6c00c19e0 2024-03-04T07:34:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6bd600640 1 -- 172.21.15.113:0/1333951443 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b0003c30 con 0x7fc6c00c19e0 2024-03-04T07:34:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c7400640 1 -- 172.21.15.113:0/1333951443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c014bf00 con 0x7fc6c00c19e0 2024-03-04T07:34:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c7400640 1 -- 172.21.15.113:0/1333951443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c0154840 con 0x7fc6c00c19e0 2024-03-04T07:34:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6c7400640 1 -- 172.21.15.113:0/1333951443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c0075c00 con 0x7fc6c00c19e0 2024-03-04T07:34:08.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6bd600640 1 -- 172.21.15.113:0/1333951443 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6b00a9070 con 0x7fc6c00c19e0 2024-03-04T07:34:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6bd600640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc69c066f90 0x7fc69c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.522+0000 7fc6bd600640 1 -- 172.21.15.113:0/1333951443 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6b011ec20 con 0x7fc6c00c19e0 2024-03-04T07:34:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.526+0000 7fc6b7e00640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc69c066f90 0x7fc69c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.526+0000 7fc6b7e00640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc69c066f90 0x7fc69c069450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00ca730 tx=0x7fc6a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:08.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.526+0000 7fc6bd600640 1 -- 172.21.15.113:0/1333951443 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc6b0016090 con 0x7fc6c00c19e0 2024-03-04T07:34:08.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.682+0000 7fc6c7400640 1 -- 172.21.15.113:0/1333951443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6c014b810 con 0x7fc6c00c19e0 2024-03-04T07:34:08.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6bd600640 1 -- 172.21.15.113:0/1333951443 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6b00ef540 con 0x7fc6c00c19e0 2024-03-04T07:34:08.688 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 -- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc69c066f90 msgr2=0x7fc69c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc69c066f90 0x7fc69c069450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6c00ca730 tx=0x7fc6a8009040 comp rx=0 tx=0).stop 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 -- 172.21.15.113:0/1333951443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6c00c19e0 msgr2=0x7fc6c00c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6c00c19e0 0x7fc6c00c6770 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc6b001ab50 tx=0x7fc6b0019240 comp rx=0 tx=0).stop 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 -- 172.21.15.113:0/1333951443 shutdown_connections 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6c0157cd0 0x7fc6c01f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6c01519b0 0x7fc6c01f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc69c066f90 0x7fc69c069450 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 --2- 172.21.15.113:0/1333951443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6c00c19e0 0x7fc6c00c6770 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 -- 172.21.15.113:0/1333951443 >> 172.21.15.113:0/1333951443 conn(0x7fc6c0066db0 msgr2=0x7fc6c00ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 -- 172.21.15.113:0/1333951443 shutdown_connections 2024-03-04T07:34:08.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.686+0000 7fc6c7400640 1 -- 172.21.15.113:0/1333951443 wait complete. 2024-03-04T07:34:08.708 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T07:34:08.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97ac200640 1 Processor -- start 2024-03-04T07:34:08.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97ac200640 1 -- start start 2024-03-04T07:34:08.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97ac200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97a406fac0 0x7f97a406b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97ac200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97a40ca800 0x7f97a4154560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97ac200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a40ce080 con 0x7f97a406fea0 2024-03-04T07:34:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97ac200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a406b5a0 con 0x7f97a406fac0 2024-03-04T07:34:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97ac200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40afd40 con 0x7f97a40ca800 2024-03-04T07:34:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97a9800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97a406fac0 0x7f97a406b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97a9800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97a406fac0 0x7f97a406b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59292/0 (socket says 172.21.15.113:59292) 2024-03-04T07:34:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97a9800640 1 -- 172.21.15.113:0/3004511347 learned_addr learned my addr 172.21.15.113:0/3004511347 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97aa200640 1 --2- 172.21.15.113:0/3004511347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97a40ca800 0x7f97a4154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97a9800640 1 -- 172.21.15.113:0/3004511347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97a40ca800 msgr2=0x7f97a4154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97a9800640 1 --2- 172.21.15.113:0/3004511347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97a40ca800 0x7f97a4154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97a9800640 1 -- 172.21.15.113:0/3004511347 >> v1:172.21.15.136:6789/0 conn(0x7f97a406fea0 legacy=0x7f97a415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.846+0000 7f97a9800640 1 -- 172.21.15.113:0/3004511347 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40affe0 con 0x7f97a406fac0 2024-03-04T07:34:08.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97a9800640 1 --2- 172.21.15.113:0/3004511347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fac0 0x7f97a406b060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9798009870 tx=0x7f979801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d15a3677ae19f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:08.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97a3e00640 1 -- 172.21.15.113:0/3004511347 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980b5070 con 0x7f97a406fac0 2024-03-04T07:34:08.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97a3e00640 1 -- 172.21.15.113:0/3004511347 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97980a8a50 con 0x7f97a406fac0 2024-03-04T07:34:08.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97a3e00640 1 -- 172.21.15.113:0/3004511347 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980a8e20 con 0x7f97a406fac0 2024-03-04T07:34:08.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97ac200640 1 -- 172.21.15.113:0/3004511347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fac0 msgr2=0x7f97a406b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97ac200640 1 --2- 172.21.15.113:0/3004511347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fac0 0x7f97a406b060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9798009870 tx=0x7f979801ed50 comp rx=0 tx=0).stop 2024-03-04T07:34:08.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97ac200640 1 -- 172.21.15.113:0/3004511347 shutdown_connections 2024-03-04T07:34:08.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97ac200640 1 --2- 172.21.15.113:0/3004511347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97a40ca800 0x7f97a4154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97ac200640 1 --2- 172.21.15.113:0/3004511347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fac0 0x7f97a406b060 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97ac200640 1 -- 172.21.15.113:0/3004511347 >> 172.21.15.113:0/3004511347 conn(0x7f97a406e240 msgr2=0x7f97a406e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:08.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97ac200640 1 -- 172.21.15.113:0/3004511347 shutdown_connections 2024-03-04T07:34:08.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.850+0000 7f97ac200640 1 -- 172.21.15.113:0/3004511347 wait complete. 2024-03-04T07:34:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97ac200640 1 Processor -- start 2024-03-04T07:34:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97ac200640 1 -- start start 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97ac200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97a406fac0 0x7f97a40b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97ac200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fea0 0x7f97a414d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97ac200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97a40ca800 0x7f97a40cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97ac200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40cd880 con 0x7f97a406fea0 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97ac200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40cd5a0 con 0x7f97a406fac0 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97ac200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40cdb60 con 0x7f97a40ca800 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fea0 0x7f97a414d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fea0 0x7f97a414d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59294/0 (socket says 172.21.15.113:59294) 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a8e00640 1 -- 172.21.15.113:0/1787812480 learned_addr learned my addr 172.21.15.113:0/1787812480 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97aa200640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97a40ca800 0x7f97a40cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a9800640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97a406fac0 0x7f97a40b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a8e00640 1 -- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97a40ca800 msgr2=0x7f97a40cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a8e00640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97a40ca800 0x7f97a40cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a8e00640 1 -- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97a406fac0 msgr2=0x7f97a40b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a8e00640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97a406fac0 0x7f97a40b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a8e00640 1 -- 172.21.15.113:0/1787812480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40c9b70 con 0x7f97a406fea0 2024-03-04T07:34:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a8e00640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fea0 0x7f97a414d6d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f979000c430 tx=0x7f979000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a1600640 1 -- 172.21.15.113:0/1787812480 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97900052d0 con 0x7f97a406fea0 2024-03-04T07:34:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a1600640 1 -- 172.21.15.113:0/1787812480 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9790005470 con 0x7f97a406fea0 2024-03-04T07:34:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97ac200640 1 -- 172.21.15.113:0/1787812480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a40ca3a0 con 0x7f97a406fea0 2024-03-04T07:34:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97a1600640 1 -- 172.21.15.113:0/1787812480 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790007260 con 0x7f97a406fea0 2024-03-04T07:34:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.854+0000 7f97ac200640 1 -- 172.21.15.113:0/1787812480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a414dc10 con 0x7f97a406fea0 2024-03-04T07:34:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.858+0000 7f97ac200640 1 -- 172.21.15.113:0/1787812480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a4075c00 con 0x7f97a406fea0 2024-03-04T07:34:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.858+0000 7f97a1600640 1 -- 172.21.15.113:0/1787812480 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9790007400 con 0x7f97a406fea0 2024-03-04T07:34:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.858+0000 7f97a1600640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9784066f60 0x7f9784069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.858+0000 7f97a1600640 1 -- 172.21.15.113:0/1787812480 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9790084400 con 0x7f97a406fea0 2024-03-04T07:34:08.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.858+0000 7f97a9800640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9784066f60 0x7f9784069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:08.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.858+0000 7f97a9800640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9784066f60 0x7f9784069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f97980b2000 tx=0x7f9798009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:08.862+0000 7f97a1600640 1 -- 172.21.15.113:0/1787812480 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9790054e50 con 0x7f97a406fea0 2024-03-04T07:34:09.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.014+0000 7f97ac200640 1 -- 172.21.15.113:0/1787812480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97a40b11f0 con 0x7f97a406fea0 2024-03-04T07:34:09.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97a1600640 1 -- 172.21.15.113:0/1787812480 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9790058ef0 con 0x7f97a406fea0 2024-03-04T07:34:09.020 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-03-04T07:34:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 -- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9784066f60 msgr2=0x7f9784069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9784066f60 0x7f9784069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f97980b2000 tx=0x7f9798009040 comp rx=0 tx=0).stop 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 -- 172.21.15.113:0/1787812480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fea0 msgr2=0x7f97a414d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fea0 0x7f97a414d6d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f979000c430 tx=0x7f979000c8f0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 -- 172.21.15.113:0/1787812480 shutdown_connections 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97a40ca800 0x7f97a40cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9784066f60 0x7f9784069420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97a406fea0 0x7f97a414d6d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 --2- 172.21.15.113:0/1787812480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97a406fac0 0x7f97a40b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 -- 172.21.15.113:0/1787812480 >> 172.21.15.113:0/1787812480 conn(0x7f97a406e240 msgr2=0x7f97a406e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 -- 172.21.15.113:0/1787812480 shutdown_connections 2024-03-04T07:34:09.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.018+0000 7f97ac200640 1 -- 172.21.15.113:0/1787812480 wait complete. 2024-03-04T07:34:09.027 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T07:34:09.025+0000 7f2bac933640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T07:34:09.027 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T07:34:09.025+0000 7f2bac933640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T07:34:09.040 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T07:34:09.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.174+0000 7f59f4e00640 1 Processor -- start 2024-03-04T07:34:09.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59f4e00640 1 -- start start 2024-03-04T07:34:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59f4e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f59f01582e0 0x7f59f00ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59f4e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59f00c9f70 0x7f59f00ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59f4e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f0070460 con 0x7f59f006def0 2024-03-04T07:34:09.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59f4e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f006f580 con 0x7f59f00c9f70 2024-03-04T07:34:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59f4e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f00c8fb0 con 0x7f59f01582e0 2024-03-04T07:34:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59f00c9f70 0x7f59f00ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59f00c9f70 0x7f59f00ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59300/0 (socket says 172.21.15.113:59300) 2024-03-04T07:34:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59e7e00640 1 -- 172.21.15.113:0/245397770 learned_addr learned my addr 172.21.15.113:0/245397770 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59ede00640 1 --2- 172.21.15.113:0/245397770 >> v2:172.21.15.136:3300/0 conn(0x7f59f01582e0 0x7f59f00ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59e7e00640 1 -- 172.21.15.113:0/245397770 >> v2:172.21.15.136:3300/0 conn(0x7f59f01582e0 msgr2=0x7f59f00ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59e7e00640 1 --2- 172.21.15.113:0/245397770 >> v2:172.21.15.136:3300/0 conn(0x7f59f01582e0 0x7f59f00ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59e7e00640 1 -- 172.21.15.113:0/245397770 >> v1:172.21.15.136:6789/0 conn(0x7f59f006def0 legacy=0x7f59f015e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.178+0000 7f59e7e00640 1 -- 172.21.15.113:0/245397770 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f006c210 con 0x7f59f00c9f70 2024-03-04T07:34:09.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59e7e00640 1 --2- 172.21.15.113:0/245397770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c9f70 0x7f59f00ce8c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f59f00ce350 tx=0x7f59d801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f2b7ecae4784f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:09.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59ed400640 1 -- 172.21.15.113:0/245397770 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d80b5070 con 0x7f59f00c9f70 2024-03-04T07:34:09.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59ed400640 1 -- 172.21.15.113:0/245397770 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d80a8a10 con 0x7f59f00c9f70 2024-03-04T07:34:09.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59ed400640 1 -- 172.21.15.113:0/245397770 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d80a8de0 con 0x7f59f00c9f70 2024-03-04T07:34:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59f4e00640 1 -- 172.21.15.113:0/245397770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c9f70 msgr2=0x7f59f00ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59f4e00640 1 --2- 172.21.15.113:0/245397770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c9f70 0x7f59f00ce8c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f59f00ce350 tx=0x7f59d801ed10 comp rx=0 tx=0).stop 2024-03-04T07:34:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59f4e00640 1 -- 172.21.15.113:0/245397770 shutdown_connections 2024-03-04T07:34:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59f4e00640 1 --2- 172.21.15.113:0/245397770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c9f70 0x7f59f00ce8c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59f4e00640 1 --2- 172.21.15.113:0/245397770 >> v2:172.21.15.136:3300/0 conn(0x7f59f01582e0 0x7f59f00ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59f4e00640 1 -- 172.21.15.113:0/245397770 >> 172.21.15.113:0/245397770 conn(0x7f59f0066db0 msgr2=0x7f59f00671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59f4e00640 1 -- 172.21.15.113:0/245397770 shutdown_connections 2024-03-04T07:34:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59f4e00640 1 -- 172.21.15.113:0/245397770 wait complete. 2024-03-04T07:34:09.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59f4e00640 1 Processor -- start 2024-03-04T07:34:09.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.182+0000 7f59f4e00640 1 -- start start 2024-03-04T07:34:09.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59f4e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f006def0 0x7f59f01f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59f4e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f00c9f70 0x7f59f015c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59f4e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01582e0 0x7f59f00c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59f4e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f01586c0 con 0x7f59f01582e0 2024-03-04T07:34:09.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59f4e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f006f7d0 con 0x7f59f00c9f70 2024-03-04T07:34:09.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59f4e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f0158840 con 0x7f59f006def0 2024-03-04T07:34:09.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59ee800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01582e0 0x7f59f00c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59ee800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01582e0 0x7f59f00c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59314/0 (socket says 172.21.15.113:59314) 2024-03-04T07:34:09.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59ee800640 1 -- 172.21.15.113:0/2686321897 learned_addr learned my addr 172.21.15.113:0/2686321897 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:09.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59ede00640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f006def0 0x7f59f01f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59e7e00640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f00c9f70 0x7f59f015c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59ee800640 1 -- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f006def0 msgr2=0x7f59f01f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59ee800640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f006def0 0x7f59f01f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59ee800640 1 -- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f00c9f70 msgr2=0x7f59f015c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59ee800640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f00c9f70 0x7f59f015c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59ee800640 1 -- 172.21.15.113:0/2686321897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00c8860 con 0x7f59f01582e0 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59ee800640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01582e0 0x7f59f00c6820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f59e800c6c0 tx=0x7f59e800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59e6000640 1 -- 172.21.15.113:0/2686321897 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8003b80 con 0x7f59f01582e0 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59e6000640 1 -- 172.21.15.113:0/2686321897 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e8003d20 con 0x7f59f01582e0 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59f4e00640 1 -- 172.21.15.113:0/2686321897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f0155630 con 0x7f59f01582e0 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59e6000640 1 -- 172.21.15.113:0/2686321897 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e80052f0 con 0x7f59f01582e0 2024-03-04T07:34:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59f4e00640 1 -- 172.21.15.113:0/2686321897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f0154870 con 0x7f59f01582e0 2024-03-04T07:34:09.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59f4e00640 1 -- 172.21.15.113:0/2686321897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f0068eb0 con 0x7f59f01582e0 2024-03-04T07:34:09.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.186+0000 7f59e6000640 1 -- 172.21.15.113:0/2686321897 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59e8005490 con 0x7f59f01582e0 2024-03-04T07:34:09.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.190+0000 7f59e6000640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59cc066f60 0x7f59cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.190+0000 7f59e6000640 1 -- 172.21.15.113:0/2686321897 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59e8084e60 con 0x7f59f01582e0 2024-03-04T07:34:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.190+0000 7f59ede00640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59cc066f60 0x7f59cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.190+0000 7f59ede00640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59cc066f60 0x7f59cc069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009ac0 tx=0x7f59e0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.194+0000 7f59e6000640 1 -- 172.21.15.113:0/2686321897 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f59e80556f0 con 0x7f59f01582e0 2024-03-04T07:34:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.346+0000 7f59f4e00640 1 -- 172.21.15.113:0/2686321897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59f00700c0 con 0x7f59f01582e0 2024-03-04T07:34:09.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59e6000640 1 -- 172.21.15.113:0/2686321897 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f59e8059790 con 0x7f59f01582e0 2024-03-04T07:34:09.352 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-03-04T07:34:09.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 -- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59cc066f60 msgr2=0x7f59cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59cc066f60 0x7f59cc069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009ac0 tx=0x7f59e0009190 comp rx=0 tx=0).stop 2024-03-04T07:34:09.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 -- 172.21.15.113:0/2686321897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01582e0 msgr2=0x7f59f00c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01582e0 0x7f59f00c6820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f59e800c6c0 tx=0x7f59e800cb80 comp rx=0 tx=0).stop 2024-03-04T07:34:09.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 -- 172.21.15.113:0/2686321897 shutdown_connections 2024-03-04T07:34:09.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59cc066f60 0x7f59cc069420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01582e0 0x7f59f00c6820 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f00c9f70 0x7f59f015c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 --2- 172.21.15.113:0/2686321897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f006def0 0x7f59f01f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 -- 172.21.15.113:0/2686321897 >> 172.21.15.113:0/2686321897 conn(0x7f59f0066db0 msgr2=0x7f59f00ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:09.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 -- 172.21.15.113:0/2686321897 shutdown_connections 2024-03-04T07:34:09.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.350+0000 7f59f4e00640 1 -- 172.21.15.113:0/2686321897 wait complete. 2024-03-04T07:34:09.372 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:34:09.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3ea800640 1 Processor -- start 2024-03-04T07:34:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3ea800640 1 -- start start 2024-03-04T07:34:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3ea800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 0x7fb3e40ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3ea800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3e415c4e0 0x7fb3e40ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3ea800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e4152e30 con 0x7fb3e406ff20 2024-03-04T07:34:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3ea800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e40c31b0 con 0x7fb3e415c4e0 2024-03-04T07:34:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3ea800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e406f840 con 0x7fb3e4152a50 2024-03-04T07:34:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3e3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3e415c4e0 0x7fb3e40ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3e3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 0x7fb3e40ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3e3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 0x7fb3e40ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:44882/0 (socket says 172.21.15.113:44882) 2024-03-04T07:34:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3e3400640 1 -- 172.21.15.113:0/1480278659 learned_addr learned my addr 172.21.15.113:0/1480278659 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3e3e00640 1 -- 172.21.15.113:0/1480278659 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 msgr2=0x7fb3e40ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3e3e00640 1 --2- 172.21.15.113:0/1480278659 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 0x7fb3e40ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3e3e00640 1 -- 172.21.15.113:0/1480278659 >> v1:172.21.15.136:6789/0 conn(0x7fb3e406ff20 legacy=0x7fb3e415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.506+0000 7fb3e3e00640 1 -- 172.21.15.113:0/1480278659 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e406f280 con 0x7fb3e415c4e0 2024-03-04T07:34:09.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3e3e00640 1 --2- 172.21.15.113:0/1480278659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 0x7fb3e40ce660 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009d90 tx=0x7fb3dc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4b0d9241a4a2f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:09.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3e2000640 1 -- 172.21.15.113:0/1480278659 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc0b2070 con 0x7fb3e415c4e0 2024-03-04T07:34:09.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3e2000640 1 -- 172.21.15.113:0/1480278659 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc0a5a30 con 0x7fb3e415c4e0 2024-03-04T07:34:09.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3e2000640 1 -- 172.21.15.113:0/1480278659 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc0a5e00 con 0x7fb3e415c4e0 2024-03-04T07:34:09.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3ea800640 1 -- 172.21.15.113:0/1480278659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 msgr2=0x7fb3e40ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3ea800640 1 --2- 172.21.15.113:0/1480278659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 0x7fb3e40ce660 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009d90 tx=0x7fb3dc01c970 comp rx=0 tx=0).stop 2024-03-04T07:34:09.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3ea800640 1 -- 172.21.15.113:0/1480278659 shutdown_connections 2024-03-04T07:34:09.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3ea800640 1 --2- 172.21.15.113:0/1480278659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 0x7fb3e40ce660 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3ea800640 1 --2- 172.21.15.113:0/1480278659 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 0x7fb3e40ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3ea800640 1 -- 172.21.15.113:0/1480278659 >> 172.21.15.113:0/1480278659 conn(0x7fb3e40734d0 msgr2=0x7fb3e4066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:09.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3ea800640 1 -- 172.21.15.113:0/1480278659 shutdown_connections 2024-03-04T07:34:09.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3ea800640 1 -- 172.21.15.113:0/1480278659 wait complete. 2024-03-04T07:34:09.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3ea800640 1 Processor -- start 2024-03-04T07:34:09.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.510+0000 7fb3ea800640 1 -- start start 2024-03-04T07:34:09.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3ea800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e406ff20 0x7fb3e415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3ea800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 0x7fb3e41eac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3ea800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 0x7fb3e4155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3ea800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4165a00 con 0x7fb3e415c4e0 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3ea800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4165880 con 0x7fb3e406ff20 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3ea800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4165b80 con 0x7fb3e4152a50 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 0x7fb3e4155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 0x7fb3e4155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59332/0 (socket says 172.21.15.113:59332) 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e3e00640 1 -- 172.21.15.113:0/1633092986 learned_addr learned my addr 172.21.15.113:0/1633092986 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e3400640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e406ff20 0x7fb3e415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e2a00640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 0x7fb3e41eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e3e00640 1 -- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 msgr2=0x7fb3e41eac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e3e00640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 0x7fb3e41eac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e3e00640 1 -- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e406ff20 msgr2=0x7fb3e415b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e3e00640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e406ff20 0x7fb3e415b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e3e00640 1 -- 172.21.15.113:0/1633092986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e40cdf50 con 0x7fb3e415c4e0 2024-03-04T07:34:09.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3e3e00640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 0x7fb3e4155170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc01b020 tx=0x7fb3dc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:09.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3db400640 1 -- 172.21.15.113:0/1633092986 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc0b2040 con 0x7fb3e415c4e0 2024-03-04T07:34:09.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3db400640 1 -- 172.21.15.113:0/1633092986 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc0bf070 con 0x7fb3e415c4e0 2024-03-04T07:34:09.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3db400640 1 -- 172.21.15.113:0/1633092986 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc0ad8c0 con 0x7fb3e415c4e0 2024-03-04T07:34:09.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3ea800640 1 -- 172.21.15.113:0/1633092986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e4153260 con 0x7fb3e415c4e0 2024-03-04T07:34:09.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3ea800640 1 -- 172.21.15.113:0/1633092986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e415ac30 con 0x7fb3e415c4e0 2024-03-04T07:34:09.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3ea800640 1 -- 172.21.15.113:0/1633092986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4075c00 con 0x7fb3e415c4e0 2024-03-04T07:34:09.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.514+0000 7fb3db400640 1 -- 172.21.15.113:0/1633092986 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3dc0ada60 con 0x7fb3e415c4e0 2024-03-04T07:34:09.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.518+0000 7fb3db400640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3c4066f90 0x7fb3c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.518+0000 7fb3db400640 1 -- 172.21.15.113:0/1633092986 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3dc11f8f0 con 0x7fb3e415c4e0 2024-03-04T07:34:09.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.518+0000 7fb3e3400640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3c4066f90 0x7fb3c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.518+0000 7fb3e3400640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3c4066f90 0x7fb3c4069450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009b50 tx=0x7fb3d4009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:09.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.522+0000 7fb3db400640 1 -- 172.21.15.113:0/1633092986 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb3dc0c8020 con 0x7fb3e415c4e0 2024-03-04T07:34:09.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.710+0000 7fb3ea800640 1 -- 172.21.15.113:0/1633092986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3e415b930 con 0x7fb3e415c4e0 2024-03-04T07:34:09.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.710+0000 7fb3db400640 1 -- 172.21.15.113:0/1633092986 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6609 (secure 0 0 0) 0x7fb3dc016560 con 0x7fb3e415c4e0 2024-03-04T07:34:09.715 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:09.715 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:07.656635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:creating","state_seq":1,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:34:09.715 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 12 2024-03-04T07:34:09.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 -- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3c4066f90 msgr2=0x7fb3c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3c4066f90 0x7fb3c4069450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4009b50 tx=0x7fb3d4009240 comp rx=0 tx=0).stop 2024-03-04T07:34:09.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 -- 172.21.15.113:0/1633092986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 msgr2=0x7fb3e4155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 0x7fb3e4155170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc01b020 tx=0x7fb3dc019710 comp rx=0 tx=0).stop 2024-03-04T07:34:09.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 -- 172.21.15.113:0/1633092986 shutdown_connections 2024-03-04T07:34:09.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3c4066f90 0x7fb3c4069450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3e415c4e0 0x7fb3e4155170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3e4152a50 0x7fb3e41eac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 --2- 172.21.15.113:0/1633092986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3e406ff20 0x7fb3e415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 -- 172.21.15.113:0/1633092986 >> 172.21.15.113:0/1633092986 conn(0x7fb3e40734d0 msgr2=0x7fb3e40cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:09.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 -- 172.21.15.113:0/1633092986 shutdown_connections 2024-03-04T07:34:09.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.714+0000 7fb3ea800640 1 -- 172.21.15.113:0/1633092986 wait complete. 2024-03-04T07:34:09.736 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:34:09.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1accc00640 1 Processor -- start 2024-03-04T07:34:09.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1accc00640 1 -- start start 2024-03-04T07:34:09.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1accc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1ac8151ac0 0x7f1ac806b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1accc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 0x7f1ac815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1accc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac815b7e0 con 0x7f1ac80c3d40 2024-03-04T07:34:09.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1accc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac8072d00 con 0x7f1ac8151ac0 2024-03-04T07:34:09.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1accc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8072a20 con 0x7f1ac815ae10 2024-03-04T07:34:09.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1ac7400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1ac8151ac0 0x7f1ac806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1ac6a00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f1ac80c3d40 0x7f1ac80c63d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:35460/0 (socket says 172.21.15.113:35460) 2024-03-04T07:34:09.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1ac7400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1ac8151ac0 0x7f1ac806b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:37998/0 (socket says 172.21.15.113:37998) 2024-03-04T07:34:09.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1ac6a00640 1 -- 172.21.15.113:0/899983424 learned_addr learned my addr 172.21.15.113:0/899983424 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:09.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1ac7e00640 1 --2- 172.21.15.113:0/899983424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 0x7f1ac815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1ac7400640 1 -- 172.21.15.113:0/899983424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 msgr2=0x7f1ac815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:09.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1ac7400640 1 --2- 172.21.15.113:0/899983424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 0x7f1ac815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1ac7400640 1 -- 172.21.15.113:0/899983424 >> v1:172.21.15.136:6789/0 conn(0x7f1ac80c3d40 legacy=0x7f1ac80c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.870+0000 7f1ac7400640 1 -- 172.21.15.113:0/899983424 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac80c56c0 con 0x7f1ac8151ac0 2024-03-04T07:34:09.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1ac7400640 1 --2- 172.21.15.113:0/899983424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8151ac0 0x7f1ac806b820 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80097c0 tx=0x7f1ab801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=763c91a8ac0d5761 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:09.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1ac6000640 1 -- 172.21.15.113:0/899983424 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80a7070 con 0x7f1ac8151ac0 2024-03-04T07:34:09.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1ac6000640 1 -- 172.21.15.113:0/899983424 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab80b0070 con 0x7f1ac8151ac0 2024-03-04T07:34:09.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1ac6000640 1 -- 172.21.15.113:0/899983424 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80ab890 con 0x7f1ac8151ac0 2024-03-04T07:34:09.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1accc00640 1 -- 172.21.15.113:0/899983424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8151ac0 msgr2=0x7f1ac806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1accc00640 1 --2- 172.21.15.113:0/899983424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8151ac0 0x7f1ac806b820 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80097c0 tx=0x7f1ab801c470 comp rx=0 tx=0).stop 2024-03-04T07:34:09.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1accc00640 1 -- 172.21.15.113:0/899983424 shutdown_connections 2024-03-04T07:34:09.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1accc00640 1 --2- 172.21.15.113:0/899983424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 0x7f1ac815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1accc00640 1 --2- 172.21.15.113:0/899983424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8151ac0 0x7f1ac806b820 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1accc00640 1 -- 172.21.15.113:0/899983424 >> 172.21.15.113:0/899983424 conn(0x7f1ac8067810 msgr2=0x7f1ac806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:09.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1accc00640 1 -- 172.21.15.113:0/899983424 shutdown_connections 2024-03-04T07:34:09.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.874+0000 7f1accc00640 1 -- 172.21.15.113:0/899983424 wait complete. 2024-03-04T07:34:09.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 Processor -- start 2024-03-04T07:34:09.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 -- start start 2024-03-04T07:34:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c3d40 0x7f1ac8145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8151ac0 0x7f1ac8145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 0x7f1ac8138210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80c6ae0 con 0x7f1ac80c3d40 2024-03-04T07:34:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac806bd60 con 0x7f1ac8151ac0 2024-03-04T07:34:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80c6c60 con 0x7f1ac815ae10 2024-03-04T07:34:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c3d40 0x7f1ac8145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c3d40 0x7f1ac8145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59348/0 (socket says 172.21.15.113:59348) 2024-03-04T07:34:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7400640 1 -- 172.21.15.113:0/3710055821 learned_addr learned my addr 172.21.15.113:0/3710055821 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:09.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac6a00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8151ac0 0x7f1ac8145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7e00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 0x7f1ac8138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7400640 1 -- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 msgr2=0x7f1ac8138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7400640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 0x7f1ac8138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7400640 1 -- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8151ac0 msgr2=0x7f1ac8145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7400640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8151ac0 0x7f1ac8145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7400640 1 -- 172.21.15.113:0/3710055821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac81f43c0 con 0x7f1ac80c3d40 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7e00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 0x7f1ac8138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1ac7400640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c3d40 0x7f1ac8145800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1ab801cd10 tx=0x7f1ab801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1abf400640 1 -- 172.21.15.113:0/3710055821 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8006e20 con 0x7f1ac80c3d40 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1abf400640 1 -- 172.21.15.113:0/3710055821 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab80bf600 con 0x7f1ac80c3d40 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 -- 172.21.15.113:0/3710055821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac81f46a0 con 0x7f1ac80c3d40 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1abf400640 1 -- 172.21.15.113:0/3710055821 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80bf9d0 con 0x7f1ac80c3d40 2024-03-04T07:34:09.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 -- 172.21.15.113:0/3710055821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac81f4c60 con 0x7f1ac80c3d40 2024-03-04T07:34:09.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.878+0000 7f1accc00640 1 -- 172.21.15.113:0/3710055821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8075a80 con 0x7f1ac80c3d40 2024-03-04T07:34:09.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.882+0000 7f1abf400640 1 -- 172.21.15.113:0/3710055821 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ab80b5400 con 0x7f1ac80c3d40 2024-03-04T07:34:09.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.882+0000 7f1abf400640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:09.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.882+0000 7f1abf400640 1 -- 172.21.15.113:0/3710055821 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ab811f5a0 con 0x7f1ac80c3d40 2024-03-04T07:34:09.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.882+0000 7f1ac6a00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:09.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.882+0000 7f1ac6a00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 0x7f1aa8069420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1ac815ac00 tx=0x7f1ab4007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:09.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:09.886+0000 7f1abf400640 1 -- 172.21.15.113:0/3710055821 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1ab8016090 con 0x7f1ac80c3d40 2024-03-04T07:34:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.042+0000 7f1accc00640 1 -- 172.21.15.113:0/3710055821 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ac81f2190 con 0x7f1ac80c3d40 2024-03-04T07:34:10.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.042+0000 7f1abf400640 1 -- 172.21.15.113:0/3710055821 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f1ab80efec0 con 0x7f1ac80c3d40 2024-03-04T07:34:10.047 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:10.047 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:57.517935+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:34:10.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 -- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 msgr2=0x7f1aa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 0x7f1aa8069420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1ac815ac00 tx=0x7f1ab4007e90 comp rx=0 tx=0).stop 2024-03-04T07:34:10.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 -- 172.21.15.113:0/3710055821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c3d40 msgr2=0x7f1ac8145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c3d40 0x7f1ac8145800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1ab801cd10 tx=0x7f1ab801acb0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 -- 172.21.15.113:0/3710055821 shutdown_connections 2024-03-04T07:34:10.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac815ae10 0x7f1ac8138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac8151ac0 0x7f1ac8145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 --2- 172.21.15.113:0/3710055821 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c3d40 0x7f1ac8145800 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 -- 172.21.15.113:0/3710055821 >> 172.21.15.113:0/3710055821 conn(0x7f1ac8067810 msgr2=0x7f1ac80c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:10.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 -- 172.21.15.113:0/3710055821 shutdown_connections 2024-03-04T07:34:10.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.046+0000 7f1accc00640 1 -- 172.21.15.113:0/3710055821 wait complete. 2024-03-04T07:34:10.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T07:33:41.507700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T07:33:53.680465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T07:33:54.892043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T07:34:10.068 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T07:34:10.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.202+0000 7f14dea00640 1 Processor -- start 2024-03-04T07:34:10.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.202+0000 7f14dea00640 1 -- start start 2024-03-04T07:34:10.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.202+0000 7f14dea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f14d8158320 0x7f14d806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.202+0000 7f14dea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d806c6b0 con 0x7f14d806ff60 2024-03-04T07:34:10.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.202+0000 7f14dea00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d80c9ce0 con 0x7f14d815ae90 2024-03-04T07:34:10.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.202+0000 7f14dea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d80c9160 con 0x7f14d8158320 2024-03-04T07:34:10.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.202+0000 7f14d7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f14d8158320 0x7f14d806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.202+0000 7f14cf400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f14d806ff60 0x7f14d80ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33308/0 (socket says 172.21.15.113:33308) 2024-03-04T07:34:10.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.202+0000 7f14cf400640 1 -- 172.21.15.113:0/4029424580 learned_addr learned my addr 172.21.15.113:0/4029424580 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:10.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14d7e00640 1 -- 172.21.15.113:0/4029424580 >> v1:172.21.15.136:6789/0 conn(0x7f14d815ae90 legacy=0x7f14d815b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14d7e00640 1 -- 172.21.15.113:0/4029424580 >> v1:172.21.15.113:6789/0 conn(0x7f14d806ff60 legacy=0x7f14d80ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14d7e00640 1 -- 172.21.15.113:0/4029424580 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d806e110 con 0x7f14d8158320 2024-03-04T07:34:10.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14d7400640 1 -- 172.21.15.113:0/4029424580 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270901796 0 0) 0x7f14d806c6b0 con 0x7f14d806ff60 2024-03-04T07:34:10.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14d7e00640 1 --2- 172.21.15.113:0/4029424580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d8158320 0x7f14d806c170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009d90 tx=0x7f14c801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12758f02fe2a3474 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:10.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14d7400640 1 -- 172.21.15.113:0/4029424580 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c80b2070 con 0x7f14d8158320 2024-03-04T07:34:10.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14d7400640 1 -- 172.21.15.113:0/4029424580 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c80a5920 con 0x7f14d8158320 2024-03-04T07:34:10.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14d7400640 1 -- 172.21.15.113:0/4029424580 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c80a5cf0 con 0x7f14d8158320 2024-03-04T07:34:10.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14dea00640 1 -- 172.21.15.113:0/4029424580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d8158320 msgr2=0x7f14d806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14dea00640 1 --2- 172.21.15.113:0/4029424580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d8158320 0x7f14d806c170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009d90 tx=0x7f14c801c970 comp rx=0 tx=0).stop 2024-03-04T07:34:10.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14dea00640 1 -- 172.21.15.113:0/4029424580 shutdown_connections 2024-03-04T07:34:10.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14dea00640 1 --2- 172.21.15.113:0/4029424580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d8158320 0x7f14d806c170 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14dea00640 1 -- 172.21.15.113:0/4029424580 >> 172.21.15.113:0/4029424580 conn(0x7f14d806b820 msgr2=0x7f14d8070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:10.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14dea00640 1 -- 172.21.15.113:0/4029424580 shutdown_connections 2024-03-04T07:34:10.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.206+0000 7f14dea00640 1 -- 172.21.15.113:0/4029424580 wait complete. 2024-03-04T07:34:10.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dea00640 1 Processor -- start 2024-03-04T07:34:10.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dea00640 1 -- start start 2024-03-04T07:34:10.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14d806ff60 0x7f14d81f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d8158320 0x7f14d815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d815ae90 0x7f14d8158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80ceac0 con 0x7f14d815ae90 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d80ce940 con 0x7f14d8158320 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d806dcb0 con 0x7f14d806ff60 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d815ae90 0x7f14d8158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d815ae90 0x7f14d8158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59366/0 (socket says 172.21.15.113:59366) 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dca00640 1 -- 172.21.15.113:0/4218135871 learned_addr learned my addr 172.21.15.113:0/4218135871 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14d7e00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14d806ff60 0x7f14d81f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14cf400640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d8158320 0x7f14d815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dca00640 1 -- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14d806ff60 msgr2=0x7f14d81f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dca00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14d806ff60 0x7f14d81f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dca00640 1 -- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d8158320 msgr2=0x7f14d815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dca00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d8158320 0x7f14d815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dca00640 1 -- 172.21.15.113:0/4218135871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d814ff60 con 0x7f14d815ae90 2024-03-04T07:34:10.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dca00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d815ae90 0x7f14d8158bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c430 tx=0x7f14d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:10.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14d4c00640 1 -- 172.21.15.113:0/4218135871 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0005220 con 0x7f14d815ae90 2024-03-04T07:34:10.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14d4c00640 1 -- 172.21.15.113:0/4218135871 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d00053c0 con 0x7f14d815ae90 2024-03-04T07:34:10.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14d4c00640 1 -- 172.21.15.113:0/4218135871 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0007270 con 0x7f14d815ae90 2024-03-04T07:34:10.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dea00640 1 -- 172.21.15.113:0/4218135871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d8150200 con 0x7f14d815ae90 2024-03-04T07:34:10.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.210+0000 7f14dea00640 1 -- 172.21.15.113:0/4218135871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d8159b00 con 0x7f14d815ae90 2024-03-04T07:34:10.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.214+0000 7f14dea00640 1 -- 172.21.15.113:0/4218135871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d8075c00 con 0x7f14d815ae90 2024-03-04T07:34:10.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.214+0000 7f14d4c00640 1 -- 172.21.15.113:0/4218135871 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14d0019020 con 0x7f14d815ae90 2024-03-04T07:34:10.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.214+0000 7f14d4c00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14b4066f60 0x7f14b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.214+0000 7f14d4c00640 1 -- 172.21.15.113:0/4218135871 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14d00843d0 con 0x7f14d815ae90 2024-03-04T07:34:10.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.214+0000 7f14d7e00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14b4066f60 0x7f14b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.214+0000 7f14d7e00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14b4066f60 0x7f14b4069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14c80af000 tx=0x7f14c80a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:10.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.218+0000 7f14d4c00640 1 -- 172.21.15.113:0/4218135871 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f14d0055340 con 0x7f14d815ae90 2024-03-04T07:34:10.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 -- 172.21.15.113:0/4218135871 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14d806fd40 con 0x7f14d815ae90 2024-03-04T07:34:10.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14d4c00640 1 -- 172.21.15.113:0/4218135871 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14d0054c40 con 0x7f14d815ae90 2024-03-04T07:34:10.376 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-03-04T07:34:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 -- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14b4066f60 msgr2=0x7f14b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14b4066f60 0x7f14b4069420 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f14c80af000 tx=0x7f14c80a7070 comp rx=0 tx=0).stop 2024-03-04T07:34:10.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 -- 172.21.15.113:0/4218135871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d815ae90 msgr2=0x7f14d8158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d815ae90 0x7f14d8158bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c430 tx=0x7f14d000c8f0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 -- 172.21.15.113:0/4218135871 shutdown_connections 2024-03-04T07:34:10.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14b4066f60 0x7f14b4069420 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14d815ae90 0x7f14d8158bf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d8158320 0x7f14d815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 --2- 172.21.15.113:0/4218135871 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14d806ff60 0x7f14d81f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 -- 172.21.15.113:0/4218135871 >> 172.21.15.113:0/4218135871 conn(0x7f14d806b820 msgr2=0x7f14d806f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:10.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 -- 172.21.15.113:0/4218135871 shutdown_connections 2024-03-04T07:34:10.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.374+0000 7f14dea00640 1 -- 172.21.15.113:0/4218135871 wait complete. 2024-03-04T07:34:10.397 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T07:34:10.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.530+0000 7fd6bf000640 1 Processor -- start 2024-03-04T07:34:10.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.530+0000 7fd6bf000640 1 -- start start 2024-03-04T07:34:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6bf000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd6b8152950 0x7fd6b80c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6bf000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd6b815bb60 0x7fd6b8162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6bf000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b80c52a0 con 0x7fd6b80c8a90 2024-03-04T07:34:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6bf000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b815cb30 con 0x7fd6b815bb60 2024-03-04T07:34:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6bf000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b80c3230 con 0x7fd6b8152950 2024-03-04T07:34:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6be600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd6b815bb60 0x7fd6b8162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6bd200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd6b80c8a90 0x7fd6b80cdfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33314/0 (socket says 172.21.15.113:33314) 2024-03-04T07:34:10.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6bd200640 1 -- 172.21.15.113:0/1661921796 learned_addr learned my addr 172.21.15.113:0/1661921796 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:10.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6bdc00640 1 --2- 172.21.15.113:0/1661921796 >> v2:172.21.15.136:3300/0 conn(0x7fd6b8152950 0x7fd6b80c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6be600640 1 -- 172.21.15.113:0/1661921796 >> v2:172.21.15.136:3300/0 conn(0x7fd6b8152950 msgr2=0x7fd6b80c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:10.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6be600640 1 --2- 172.21.15.113:0/1661921796 >> v2:172.21.15.136:3300/0 conn(0x7fd6b8152950 0x7fd6b80c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6be600640 1 -- 172.21.15.113:0/1661921796 >> v1:172.21.15.113:6789/0 conn(0x7fd6b80c8a90 legacy=0x7fd6b80cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6be600640 1 -- 172.21.15.113:0/1661921796 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b81594e0 con 0x7fd6b815bb60 2024-03-04T07:34:10.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6b7e00640 1 -- 172.21.15.113:0/1661921796 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759877188 0 0) 0x7fd6b80c52a0 con 0x7fd6b80c8a90 2024-03-04T07:34:10.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6bdc00640 1 --2- 172.21.15.113:0/1661921796 >> v2:172.21.15.136:3300/0 conn(0x7fd6b8152950 0x7fd6b80c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6be600640 1 --2- 172.21.15.113:0/1661921796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b815bb60 0x7fd6b8162ef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8073880 tx=0x7fd6b001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e3fad11d43315c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6b7e00640 1 -- 172.21.15.113:0/1661921796 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00b4070 con 0x7fd6b815bb60 2024-03-04T07:34:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6b7e00640 1 -- 172.21.15.113:0/1661921796 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b00af600 con 0x7fd6b815bb60 2024-03-04T07:34:10.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.534+0000 7fd6b7e00640 1 -- 172.21.15.113:0/1661921796 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00af9d0 con 0x7fd6b815bb60 2024-03-04T07:34:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 -- 172.21.15.113:0/1661921796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b815bb60 msgr2=0x7fd6b8162ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 --2- 172.21.15.113:0/1661921796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b815bb60 0x7fd6b8162ef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8073880 tx=0x7fd6b001f110 comp rx=0 tx=0).stop 2024-03-04T07:34:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 -- 172.21.15.113:0/1661921796 shutdown_connections 2024-03-04T07:34:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 --2- 172.21.15.113:0/1661921796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b815bb60 0x7fd6b8162ef0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 --2- 172.21.15.113:0/1661921796 >> v2:172.21.15.136:3300/0 conn(0x7fd6b8152950 0x7fd6b80c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 -- 172.21.15.113:0/1661921796 >> 172.21.15.113:0/1661921796 conn(0x7fd6b8072f40 msgr2=0x7fd6b8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 -- 172.21.15.113:0/1661921796 shutdown_connections 2024-03-04T07:34:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 -- 172.21.15.113:0/1661921796 wait complete. 2024-03-04T07:34:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 Processor -- start 2024-03-04T07:34:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 -- start start 2024-03-04T07:34:10.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b80c8a90 0x7fd6b81e19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8152950 0x7fd6b81f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b815bb60 0x7fd6b81e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b806e820 con 0x7fd6b8152950 2024-03-04T07:34:10.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b806e540 con 0x7fd6b815bb60 2024-03-04T07:34:10.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bf000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b80ce6b0 con 0x7fd6b80c8a90 2024-03-04T07:34:10.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bd200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8152950 0x7fd6b81f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bd200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8152950 0x7fd6b81f0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59380/0 (socket says 172.21.15.113:59380) 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bd200640 1 -- 172.21.15.113:0/3855696194 learned_addr learned my addr 172.21.15.113:0/3855696194 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bdc00640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b80c8a90 0x7fd6b81e19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6be600640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b815bb60 0x7fd6b81e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bd200640 1 -- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b80c8a90 msgr2=0x7fd6b81e19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bd200640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b80c8a90 0x7fd6b81e19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bd200640 1 -- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b815bb60 msgr2=0x7fd6b81e41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bd200640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b815bb60 0x7fd6b81e41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.538+0000 7fd6bd200640 1 -- 172.21.15.113:0/3855696194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b81f0320 con 0x7fd6b8152950 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6bd200640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8152950 0x7fd6b81f0de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00c4e0 tx=0x7fd6ac00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6b5600640 1 -- 172.21.15.113:0/3855696194 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac018070 con 0x7fd6b8152950 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6b5600640 1 -- 172.21.15.113:0/3855696194 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ac007500 con 0x7fd6b8152950 2024-03-04T07:34:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6b5600640 1 -- 172.21.15.113:0/3855696194 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ac004a30 con 0x7fd6b8152950 2024-03-04T07:34:10.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6bf000640 1 -- 172.21.15.113:0/3855696194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b80cdb50 con 0x7fd6b8152950 2024-03-04T07:34:10.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6bf000640 1 -- 172.21.15.113:0/3855696194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b80ce110 con 0x7fd6b8152950 2024-03-04T07:34:10.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6bf000640 1 -- 172.21.15.113:0/3855696194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b8075a80 con 0x7fd6b8152950 2024-03-04T07:34:10.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6b5600640 1 -- 172.21.15.113:0/3855696194 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6ac004bd0 con 0x7fd6b8152950 2024-03-04T07:34:10.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6b5600640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd69c066f90 0x7fd69c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6b5600640 1 -- 172.21.15.113:0/3855696194 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6ac084300 con 0x7fd6b8152950 2024-03-04T07:34:10.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.542+0000 7fd6bdc00640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd69c066f90 0x7fd69c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.546+0000 7fd6bdc00640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd69c066f90 0x7fd69c069450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009950 tx=0x7fd6a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.546+0000 7fd6b5600640 1 -- 172.21.15.113:0/3855696194 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd6ac054cf0 con 0x7fd6b8152950 2024-03-04T07:34:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 -- 172.21.15.113:0/3855696194 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6b80c3bb0 con 0x7fd6b8152950 2024-03-04T07:34:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6b5600640 1 -- 172.21.15.113:0/3855696194 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6ac058d90 con 0x7fd6b8152950 2024-03-04T07:34:10.705 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-03-04T07:34:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 -- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd69c066f90 msgr2=0x7fd69c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd69c066f90 0x7fd69c069450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009950 tx=0x7fd6a8008040 comp rx=0 tx=0).stop 2024-03-04T07:34:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 -- 172.21.15.113:0/3855696194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8152950 msgr2=0x7fd6b81f0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8152950 0x7fd6b81f0de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac00c4e0 tx=0x7fd6ac00c9a0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 -- 172.21.15.113:0/3855696194 shutdown_connections 2024-03-04T07:34:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b815bb60 0x7fd6b81e41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd69c066f90 0x7fd69c069450 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8152950 0x7fd6b81f0de0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 --2- 172.21.15.113:0/3855696194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b80c8a90 0x7fd6b81e19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 -- 172.21.15.113:0/3855696194 >> 172.21.15.113:0/3855696194 conn(0x7fd6b8072f40 msgr2=0x7fd6b80c8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 -- 172.21.15.113:0/3855696194 shutdown_connections 2024-03-04T07:34:10.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.702+0000 7fd6bf000640 1 -- 172.21.15.113:0/3855696194 wait complete. 2024-03-04T07:34:10.726 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T07:34:10.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0548800640 1 Processor -- start 2024-03-04T07:34:10.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0548800640 1 -- start start 2024-03-04T07:34:10.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0548800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f054006fac0 0x7f054006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0548800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f05400ca800 0x7f0540154560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0548800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05400ce080 con 0x7f054006fea0 2024-03-04T07:34:10.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0548800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054006b5a0 con 0x7f054006fac0 2024-03-04T07:34:10.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0548800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05400afd40 con 0x7f05400ca800 2024-03-04T07:34:10.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0545e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f054006fac0 0x7f054006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0545e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f054006fac0 0x7f054006b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59382/0 (socket says 172.21.15.113:59382) 2024-03-04T07:34:10.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0545e00640 1 -- 172.21.15.113:0/2618870178 learned_addr learned my addr 172.21.15.113:0/2618870178 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:10.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0545400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f054006fea0 0x7f054015aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33326/0 (socket says 172.21.15.113:33326) 2024-03-04T07:34:10.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0546800640 1 --2- 172.21.15.113:0/2618870178 >> v2:172.21.15.136:3300/0 conn(0x7f05400ca800 0x7f0540154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0545e00640 1 -- 172.21.15.113:0/2618870178 >> v2:172.21.15.136:3300/0 conn(0x7f05400ca800 msgr2=0x7f0540154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:10.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0545e00640 1 --2- 172.21.15.113:0/2618870178 >> v2:172.21.15.136:3300/0 conn(0x7f05400ca800 0x7f0540154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0545e00640 1 -- 172.21.15.113:0/2618870178 >> v1:172.21.15.113:6789/0 conn(0x7f054006fea0 legacy=0x7f054015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0545e00640 1 -- 172.21.15.113:0/2618870178 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05400affe0 con 0x7f054006fac0 2024-03-04T07:34:10.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.862+0000 7f0544a00640 1 -- 172.21.15.113:0/2618870178 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664510581 0 0) 0x7f05400ce080 con 0x7f054006fea0 2024-03-04T07:34:10.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0545e00640 1 --2- 172.21.15.113:0/2618870178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f054006fac0 0x7f054006b060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f054006f5d0 tx=0x7f053401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38c81dccd13e30 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:10.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0544a00640 1 -- 172.21.15.113:0/2618870178 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340b5070 con 0x7f054006fac0 2024-03-04T07:34:10.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0544a00640 1 -- 172.21.15.113:0/2618870178 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05340a8a10 con 0x7f054006fac0 2024-03-04T07:34:10.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0544a00640 1 -- 172.21.15.113:0/2618870178 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340a8de0 con 0x7f054006fac0 2024-03-04T07:34:10.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0548800640 1 -- 172.21.15.113:0/2618870178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f054006fac0 msgr2=0x7f054006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0548800640 1 --2- 172.21.15.113:0/2618870178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f054006fac0 0x7f054006b060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f054006f5d0 tx=0x7f053401ed10 comp rx=0 tx=0).stop 2024-03-04T07:34:10.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0548800640 1 -- 172.21.15.113:0/2618870178 shutdown_connections 2024-03-04T07:34:10.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0548800640 1 --2- 172.21.15.113:0/2618870178 >> v2:172.21.15.136:3300/0 conn(0x7f05400ca800 0x7f0540154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0548800640 1 --2- 172.21.15.113:0/2618870178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f054006fac0 0x7f054006b060 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0548800640 1 -- 172.21.15.113:0/2618870178 >> 172.21.15.113:0/2618870178 conn(0x7f054006e240 msgr2=0x7f054006e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:10.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0548800640 1 -- 172.21.15.113:0/2618870178 shutdown_connections 2024-03-04T07:34:10.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.866+0000 7f0548800640 1 -- 172.21.15.113:0/2618870178 wait complete. 2024-03-04T07:34:10.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 Processor -- start 2024-03-04T07:34:10.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 -- start start 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f054006fac0 0x7f05401561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f054006fea0 0x7f0540157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400ca800 0x7f054015a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05400cd880 con 0x7f05400ca800 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05400cd5a0 con 0x7f054006fac0 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05400cdb60 con 0x7f054006fea0 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0546800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400ca800 0x7f054015a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0546800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400ca800 0x7f054015a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59384/0 (socket says 172.21.15.113:59384) 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0546800640 1 -- 172.21.15.113:0/789926215 learned_addr learned my addr 172.21.15.113:0/789926215 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0545400640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f054006fea0 0x7f0540157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0545e00640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f054006fac0 0x7f05401561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0546800640 1 -- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f054006fea0 msgr2=0x7f0540157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0546800640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f054006fea0 0x7f0540157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0546800640 1 -- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f054006fac0 msgr2=0x7f05401561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0546800640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f054006fac0 0x7f05401561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0546800640 1 -- 172.21.15.113:0/789926215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05400c9060 con 0x7f05400ca800 2024-03-04T07:34:10.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0546800640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400ca800 0x7f054015a500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f053c007e60 tx=0x7f053c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f053a000640 1 -- 172.21.15.113:0/789926215 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c014420 con 0x7f05400ca800 2024-03-04T07:34:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f053a000640 1 -- 172.21.15.113:0/789926215 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c019070 con 0x7f05400ca800 2024-03-04T07:34:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 -- 172.21.15.113:0/789926215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05400c9340 con 0x7f05400ca800 2024-03-04T07:34:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f053a000640 1 -- 172.21.15.113:0/789926215 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c014820 con 0x7f05400ca800 2024-03-04T07:34:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 -- 172.21.15.113:0/789926215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05400c9900 con 0x7f05400ca800 2024-03-04T07:34:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.870+0000 7f0548800640 1 -- 172.21.15.113:0/789926215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0540075c00 con 0x7f05400ca800 2024-03-04T07:34:10.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.874+0000 7f053a000640 1 -- 172.21.15.113:0/789926215 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f053c0149c0 con 0x7f05400ca800 2024-03-04T07:34:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.874+0000 7f053a000640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.874+0000 7f053a000640 1 -- 172.21.15.113:0/789926215 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f053c0857b0 con 0x7f05400ca800 2024-03-04T07:34:10.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.874+0000 7f0545e00640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:10.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.874+0000 7f0545e00640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f05340b2000 tx=0x7f05340aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:10.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:10.878+0000 7f053a000640 1 -- 172.21.15.113:0/789926215 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f053c056160 con 0x7f05400ca800 2024-03-04T07:34:11.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.030+0000 7f0548800640 1 -- 172.21.15.113:0/789926215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f054006ed90 con 0x7f05400ca800 2024-03-04T07:34:11.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.030+0000 7f053a000640 1 -- 172.21.15.113:0/789926215 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f053c05a200 con 0x7f05400ca800 2024-03-04T07:34:11.035 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-03-04T07:34:11.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 -- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 msgr2=0x7f0524069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f05340b2000 tx=0x7f05340aa070 comp rx=0 tx=0).stop 2024-03-04T07:34:11.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 -- 172.21.15.113:0/789926215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400ca800 msgr2=0x7f054015a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400ca800 0x7f054015a500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f053c007e60 tx=0x7f053c00e4b0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 -- 172.21.15.113:0/789926215 shutdown_connections 2024-03-04T07:34:11.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05400ca800 0x7f054015a500 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f054006fea0 0x7f0540157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 --2- 172.21.15.113:0/789926215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f054006fac0 0x7f05401561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 -- 172.21.15.113:0/789926215 >> 172.21.15.113:0/789926215 conn(0x7f054006e240 msgr2=0x7f054006c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:11.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 -- 172.21.15.113:0/789926215 shutdown_connections 2024-03-04T07:34:11.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.034+0000 7f0548800640 1 -- 172.21.15.113:0/789926215 wait complete. 2024-03-04T07:34:11.055 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:34:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe189a00640 1 Processor -- start 2024-03-04T07:34:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe189a00640 1 -- start start 2024-03-04T07:34:11.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe189a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1840c22e0 0x7fe18415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe189a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1840c2d80 0x7fe18415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe189a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1840c27f0 con 0x7fe18414e4c0 2024-03-04T07:34:11.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe189a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18415ad60 con 0x7fe1840c2d80 2024-03-04T07:34:11.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe189a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1840c2a90 con 0x7fe1840c22e0 2024-03-04T07:34:11.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe183400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1840c2d80 0x7fe18415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe183400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1840c2d80 0x7fe18415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59390/0 (socket says 172.21.15.113:59390) 2024-03-04T07:34:11.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe183400640 1 -- 172.21.15.113:0/2344315301 learned_addr learned my addr 172.21.15.113:0/2344315301 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:11.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe182000640 1 --2- 172.21.15.113:0/2344315301 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1840c22e0 0x7fe18415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe183400640 1 -- 172.21.15.113:0/2344315301 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1840c22e0 msgr2=0x7fe18415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:11.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe183400640 1 --2- 172.21.15.113:0/2344315301 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1840c22e0 0x7fe18415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe183400640 1 -- 172.21.15.113:0/2344315301 >> v1:172.21.15.136:6789/0 conn(0x7fe18414e4c0 legacy=0x7fe184072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.186+0000 7fe183400640 1 -- 172.21.15.113:0/2344315301 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe184070c60 con 0x7fe1840c2d80 2024-03-04T07:34:11.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe183400640 1 --2- 172.21.15.113:0/2344315301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 0x7fe18415b730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009e50 tx=0x7fe17c01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513176bec8116a18 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:11.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe181600640 1 -- 172.21.15.113:0/2344315301 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c0b4070 con 0x7fe1840c2d80 2024-03-04T07:34:11.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe181600640 1 -- 172.21.15.113:0/2344315301 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c0af400 con 0x7fe1840c2d80 2024-03-04T07:34:11.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe181600640 1 -- 172.21.15.113:0/2344315301 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c0af7d0 con 0x7fe1840c2d80 2024-03-04T07:34:11.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe189a00640 1 -- 172.21.15.113:0/2344315301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 msgr2=0x7fe18415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe189a00640 1 --2- 172.21.15.113:0/2344315301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 0x7fe18415b730 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009e50 tx=0x7fe17c01f210 comp rx=0 tx=0).stop 2024-03-04T07:34:11.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe189a00640 1 -- 172.21.15.113:0/2344315301 shutdown_connections 2024-03-04T07:34:11.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe189a00640 1 --2- 172.21.15.113:0/2344315301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 0x7fe18415b730 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe189a00640 1 --2- 172.21.15.113:0/2344315301 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1840c22e0 0x7fe18415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe189a00640 1 -- 172.21.15.113:0/2344315301 >> 172.21.15.113:0/2344315301 conn(0x7fe18406e820 msgr2=0x7fe18406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:11.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe189a00640 1 -- 172.21.15.113:0/2344315301 shutdown_connections 2024-03-04T07:34:11.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.190+0000 7fe189a00640 1 -- 172.21.15.113:0/2344315301 wait complete. 2024-03-04T07:34:11.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe189a00640 1 Processor -- start 2024-03-04T07:34:11.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe189a00640 1 -- start start 2024-03-04T07:34:11.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe189a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1840c22e0 0x7fe1840cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe189a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 0x7fe1841e2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe189a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe18414e4c0 0x7fe1841e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe189a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe184071500 con 0x7fe1840c2d80 2024-03-04T07:34:11.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe189a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe184071220 con 0x7fe1840c22e0 2024-03-04T07:34:11.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe189a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18414e8a0 con 0x7fe18414e4c0 2024-03-04T07:34:11.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 0x7fe1841e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 0x7fe1841e2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59392/0 (socket says 172.21.15.113:59392) 2024-03-04T07:34:11.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182000640 1 -- 172.21.15.113:0/1452575847 learned_addr learned my addr 172.21.15.113:0/1452575847 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:11.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe183400640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe18414e4c0 0x7fe1841e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182a00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1840c22e0 0x7fe1840cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182000640 1 -- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe18414e4c0 msgr2=0x7fe1841e28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182000640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe18414e4c0 0x7fe1841e28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182000640 1 -- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1840c22e0 msgr2=0x7fe1840cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182000640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1840c22e0 0x7fe1840cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182000640 1 -- 172.21.15.113:0/1452575847 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18415ab20 con 0x7fe1840c2d80 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182a00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1840c22e0 0x7fe1840cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe182000640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 0x7fe1841e2360 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe17000e8a0 tx=0x7fe17000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe17aa00640 1 -- 172.21.15.113:0/1452575847 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170017070 con 0x7fe1840c2d80 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe17aa00640 1 -- 172.21.15.113:0/1452575847 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17000c940 con 0x7fe1840c2d80 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe189a00640 1 -- 172.21.15.113:0/1452575847 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1841487f0 con 0x7fe1840c2d80 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe17aa00640 1 -- 172.21.15.113:0/1452575847 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000ccf0 con 0x7fe1840c2d80 2024-03-04T07:34:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.194+0000 7fe189a00640 1 -- 172.21.15.113:0/1452575847 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe184148db0 con 0x7fe1840c2d80 2024-03-04T07:34:11.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.198+0000 7fe189a00640 1 -- 172.21.15.113:0/1452575847 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe184075c00 con 0x7fe1840c2d80 2024-03-04T07:34:11.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.198+0000 7fe17aa00640 1 -- 172.21.15.113:0/1452575847 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe17001a070 con 0x7fe1840c2d80 2024-03-04T07:34:11.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.198+0000 7fe17aa00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe164066f60 0x7fe164069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.198+0000 7fe17aa00640 1 -- 172.21.15.113:0/1452575847 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe170085250 con 0x7fe1840c2d80 2024-03-04T07:34:11.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.198+0000 7fe182a00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe164066f60 0x7fe164069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.198+0000 7fe182a00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe164066f60 0x7fe164069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe174001710 tx=0x7fe174009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:11.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.202+0000 7fe17aa00640 1 -- 172.21.15.113:0/1452575847 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe170055b70 con 0x7fe1840c2d80 2024-03-04T07:34:11.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 -- 172.21.15.113:0/1452575847 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe184149470 con 0x7fe1840c2d80 2024-03-04T07:34:11.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe17aa00640 1 -- 172.21.15.113:0/1452575847 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6609 (secure 0 0 0) 0x7fe170059c10 con 0x7fe1840c2d80 2024-03-04T07:34:11.393 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:11.393 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:07.656635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:creating","state_seq":1,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:34:11.393 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 12 2024-03-04T07:34:11.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 -- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe164066f60 msgr2=0x7fe164069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe164066f60 0x7fe164069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe174001710 tx=0x7fe174009040 comp rx=0 tx=0).stop 2024-03-04T07:34:11.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 -- 172.21.15.113:0/1452575847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 msgr2=0x7fe1841e2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 0x7fe1841e2360 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe17000e8a0 tx=0x7fe17000eca0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 -- 172.21.15.113:0/1452575847 shutdown_connections 2024-03-04T07:34:11.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe18414e4c0 0x7fe1841e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe164066f60 0x7fe164069420 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1840c2d80 0x7fe1841e2360 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 --2- 172.21.15.113:0/1452575847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1840c22e0 0x7fe1840cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 -- 172.21.15.113:0/1452575847 >> 172.21.15.113:0/1452575847 conn(0x7fe18406e820 msgr2=0x7fe184072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:11.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 -- 172.21.15.113:0/1452575847 shutdown_connections 2024-03-04T07:34:11.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.390+0000 7fe189a00640 1 -- 172.21.15.113:0/1452575847 wait complete. 2024-03-04T07:34:11.414 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:34:11.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.546+0000 7f977b200640 1 Processor -- start 2024-03-04T07:34:11.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.546+0000 7f977b200640 1 -- start start 2024-03-04T07:34:11.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f977b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c06fdb0 0x7f977c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f977b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977c15cb20 con 0x7f977c0c31b0 2024-03-04T07:34:11.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f977b200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977c06f280 con 0x7f977c152b70 2024-03-04T07:34:11.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f977b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c152f50 con 0x7f977c06fdb0 2024-03-04T07:34:11.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f9779400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c06fdb0 0x7f977c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f9779e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f977c152b70 0x7f977c0ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:35486/0 (socket says 172.21.15.113:35486) 2024-03-04T07:34:11.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f9779e00640 1 -- 172.21.15.113:0/1464772067 learned_addr learned my addr 172.21.15.113:0/1464772067 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:11.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f9778a00640 1 -- 172.21.15.113:0/1464772067 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857331046 0 0) 0x7f977c15cb20 con 0x7f977c0c31b0 2024-03-04T07:34:11.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f9778a00640 1 -- 172.21.15.113:0/1464772067 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9760003530 con 0x7f977c0c31b0 2024-03-04T07:34:11.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f9778a00640 1 -- 172.21.15.113:0/1464772067 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079458946 0 0) 0x7f977c06f280 con 0x7f977c152b70 2024-03-04T07:34:11.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f9778a00640 1 -- 172.21.15.113:0/1464772067 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f977c15cb20 con 0x7f977c152b70 2024-03-04T07:34:11.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f9779400640 1 -- 172.21.15.113:0/1464772067 >> v1:172.21.15.136:6789/0 conn(0x7f977c152b70 legacy=0x7f977c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f9779400640 1 -- 172.21.15.113:0/1464772067 >> v1:172.21.15.113:6789/0 conn(0x7f977c0c31b0 legacy=0x7f977c1630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.550+0000 7f9779400640 1 -- 172.21.15.113:0/1464772067 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c06f560 con 0x7f977c06fdb0 2024-03-04T07:34:11.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f9779400640 1 --2- 172.21.15.113:0/1464772067 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c06fdb0 0x7f977c0c8dd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97680097f0 tx=0x7f976801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5b37c033e5a1906 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:11.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f9778a00640 1 -- 172.21.15.113:0/1464772067 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97680b0070 con 0x7f977c06fdb0 2024-03-04T07:34:11.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f9778a00640 1 -- 172.21.15.113:0/1464772067 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97680ab600 con 0x7f977c06fdb0 2024-03-04T07:34:11.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f9778a00640 1 -- 172.21.15.113:0/1464772067 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97680ab9b0 con 0x7f977c06fdb0 2024-03-04T07:34:11.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 -- 172.21.15.113:0/1464772067 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c06fdb0 msgr2=0x7f977c0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 --2- 172.21.15.113:0/1464772067 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c06fdb0 0x7f977c0c8dd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97680097f0 tx=0x7f976801c450 comp rx=0 tx=0).stop 2024-03-04T07:34:11.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 -- 172.21.15.113:0/1464772067 shutdown_connections 2024-03-04T07:34:11.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 --2- 172.21.15.113:0/1464772067 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c06fdb0 0x7f977c0c8dd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 -- 172.21.15.113:0/1464772067 >> 172.21.15.113:0/1464772067 conn(0x7f977c072e70 msgr2=0x7f977c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:11.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 -- 172.21.15.113:0/1464772067 shutdown_connections 2024-03-04T07:34:11.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 -- 172.21.15.113:0/1464772067 wait complete. 2024-03-04T07:34:11.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 Processor -- start 2024-03-04T07:34:11.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 -- start start 2024-03-04T07:34:11.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f977c06fdb0 0x7f977c1f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f977c0c31b0 0x7f977c14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c152b70 0x7f977c0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c15c730 con 0x7f977c06fdb0 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c06f030 con 0x7f977c0c31b0 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f977b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c15c8b0 con 0x7f977c152b70 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f9779e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f977c06fdb0 0x7f977c1f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f9779e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f977c06fdb0 0x7f977c1f11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59402/0 (socket says 172.21.15.113:59402) 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f9779e00640 1 -- 172.21.15.113:0/4236589239 learned_addr learned my addr 172.21.15.113:0/4236589239 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.554+0000 7f9779400640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f977c0c31b0 0x7f977c14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f977a800640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c152b70 0x7f977c0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9779e00640 1 -- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c152b70 msgr2=0x7f977c0c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9779e00640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c152b70 0x7f977c0c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9779e00640 1 -- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f977c0c31b0 msgr2=0x7f977c14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9779e00640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f977c0c31b0 0x7f977c14dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9779e00640 1 -- 172.21.15.113:0/4236589239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c1f19f0 con 0x7f977c06fdb0 2024-03-04T07:34:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f977a800640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c152b70 0x7f977c0c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9779e00640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f977c06fdb0 0x7f977c1f11b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c480 tx=0x7f976c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9772000640 1 -- 172.21.15.113:0/4236589239 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c017070 con 0x7f977c06fdb0 2024-03-04T07:34:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9772000640 1 -- 172.21.15.113:0/4236589239 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c005cf0 con 0x7f977c06fdb0 2024-03-04T07:34:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f977b200640 1 -- 172.21.15.113:0/4236589239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c1f1cd0 con 0x7f977c06fdb0 2024-03-04T07:34:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9772000640 1 -- 172.21.15.113:0/4236589239 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c007290 con 0x7f977c06fdb0 2024-03-04T07:34:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f977b200640 1 -- 172.21.15.113:0/4236589239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c0c6620 con 0x7f977c06fdb0 2024-03-04T07:34:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f977b200640 1 -- 172.21.15.113:0/4236589239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977c075a80 con 0x7f977c06fdb0 2024-03-04T07:34:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9772000640 1 -- 172.21.15.113:0/4236589239 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f976c01a070 con 0x7f977c06fdb0 2024-03-04T07:34:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9772000640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f975c066f90 0x7f975c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.558+0000 7f9772000640 1 -- 172.21.15.113:0/4236589239 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f976c084390 con 0x7f977c06fdb0 2024-03-04T07:34:11.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.562+0000 7f9779400640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f975c066f90 0x7f975c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.562+0000 7f9779400640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f975c066f90 0x7f975c069450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97680ad000 tx=0x7f97680a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:11.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.562+0000 7f9772000640 1 -- 172.21.15.113:0/4236589239 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f976c054d40 con 0x7f977c06fdb0 2024-03-04T07:34:11.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.722+0000 7f977b200640 1 -- 172.21.15.113:0/4236589239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f977c06fb90 con 0x7f977c06fdb0 2024-03-04T07:34:11.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.722+0000 7f9772000640 1 -- 172.21.15.113:0/4236589239 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f976c058de0 con 0x7f977c06fdb0 2024-03-04T07:34:11.727 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:11.727 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:57.517935+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 -- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f975c066f90 msgr2=0x7f975c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f975c066f90 0x7f975c069450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97680ad000 tx=0x7f97680a6000 comp rx=0 tx=0).stop 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 -- 172.21.15.113:0/4236589239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f977c06fdb0 msgr2=0x7f977c1f11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f977c06fdb0 0x7f977c1f11b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c480 tx=0x7f976c00c940 comp rx=0 tx=0).stop 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 -- 172.21.15.113:0/4236589239 shutdown_connections 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f977c152b70 0x7f977c0c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f977c0c31b0 0x7f977c14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f975c066f90 0x7f975c069450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 --2- 172.21.15.113:0/4236589239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f977c06fdb0 0x7f977c1f11b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 -- 172.21.15.113:0/4236589239 >> 172.21.15.113:0/4236589239 conn(0x7f977c072e70 msgr2=0x7f977c0c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 -- 172.21.15.113:0/4236589239 shutdown_connections 2024-03-04T07:34:11.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.726+0000 7f977b200640 1 -- 172.21.15.113:0/4236589239 wait complete. 2024-03-04T07:34:11.747 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:34:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.878+0000 7f76ada00640 1 Processor -- start 2024-03-04T07:34:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.878+0000 7f76ada00640 1 -- start start 2024-03-04T07:34:11.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f76ada00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76a814e4c0 0x7f76a8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f76ada00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f76a80c22e0 0x7f76a80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f76ada00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80c27f0 con 0x7f76a80c2d80 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f76ada00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a815ad60 con 0x7f76a80c22e0 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f76ada00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80c2a90 con 0x7f76a814e4c0 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f769e000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f76a80c22e0 0x7f76a80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f76a7400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f76a80c2d80 0x7f76a815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33358/0 (socket says 172.21.15.113:33358) 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f76a7400640 1 -- 172.21.15.113:0/3216172267 learned_addr learned my addr 172.21.15.113:0/3216172267 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f76a6a00640 1 --2- 172.21.15.113:0/3216172267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76a814e4c0 0x7f76a8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f769e000640 1 -- 172.21.15.113:0/3216172267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76a814e4c0 msgr2=0x7f76a8072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f769e000640 1 --2- 172.21.15.113:0/3216172267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76a814e4c0 0x7f76a8072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f769e000640 1 -- 172.21.15.113:0/3216172267 >> v1:172.21.15.113:6789/0 conn(0x7f76a80c2d80 legacy=0x7f76a815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f769e000640 1 -- 172.21.15.113:0/3216172267 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a8070c60 con 0x7f76a80c22e0 2024-03-04T07:34:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.882+0000 7f76a6000640 1 -- 172.21.15.113:0/3216172267 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10404025 0 0) 0x7f76a80c27f0 con 0x7f76a80c2d80 2024-03-04T07:34:11.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f769e000640 1 --2- 172.21.15.113:0/3216172267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c22e0 0x7f76a80731c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76900097c0 tx=0x7f769001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd02ceb41299f357 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:11.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76a6000640 1 -- 172.21.15.113:0/3216172267 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900b5070 con 0x7f76a80c22e0 2024-03-04T07:34:11.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76a6000640 1 -- 172.21.15.113:0/3216172267 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76900a8990 con 0x7f76a80c22e0 2024-03-04T07:34:11.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76a6000640 1 -- 172.21.15.113:0/3216172267 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900a8d60 con 0x7f76a80c22e0 2024-03-04T07:34:11.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 -- 172.21.15.113:0/3216172267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c22e0 msgr2=0x7f76a80731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 --2- 172.21.15.113:0/3216172267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c22e0 0x7f76a80731c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f76900097c0 tx=0x7f769001ec90 comp rx=0 tx=0).stop 2024-03-04T07:34:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 -- 172.21.15.113:0/3216172267 shutdown_connections 2024-03-04T07:34:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 --2- 172.21.15.113:0/3216172267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c22e0 0x7f76a80731c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 --2- 172.21.15.113:0/3216172267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76a814e4c0 0x7f76a8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 -- 172.21.15.113:0/3216172267 >> 172.21.15.113:0/3216172267 conn(0x7f76a806e820 msgr2=0x7f76a806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 -- 172.21.15.113:0/3216172267 shutdown_connections 2024-03-04T07:34:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 -- 172.21.15.113:0/3216172267 wait complete. 2024-03-04T07:34:11.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 Processor -- start 2024-03-04T07:34:11.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 -- start start 2024-03-04T07:34:11.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76a80c22e0 0x7f76a806b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c2d80 0x7f76a8147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76a814e4c0 0x7f76a8148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8071500 con 0x7f76a80c2d80 2024-03-04T07:34:11.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8071220 con 0x7f76a814e4c0 2024-03-04T07:34:11.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76ada00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a814e8a0 con 0x7f76a80c22e0 2024-03-04T07:34:11.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f769e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c2d80 0x7f76a8147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f769e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c2d80 0x7f76a8147b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59416/0 (socket says 172.21.15.113:59416) 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f769e000640 1 -- 172.21.15.113:0/3714302511 learned_addr learned my addr 172.21.15.113:0/3714302511 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.886+0000 7f76a6a00640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76a80c22e0 0x7f76a806b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f76a7400640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76a814e4c0 0x7f76a8148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f769e000640 1 -- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76a80c22e0 msgr2=0x7f76a806b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f769e000640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76a80c22e0 0x7f76a806b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f769e000640 1 -- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76a814e4c0 msgr2=0x7f76a8148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f769e000640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76a814e4c0 0x7f76a8148040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f769e000640 1 -- 172.21.15.113:0/3714302511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a8152170 con 0x7f76a80c2d80 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f76a7400640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76a814e4c0 0x7f76a8148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f769e000640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c2d80 0x7f76a8147b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76900b3040 tx=0x7f769001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f769f400640 1 -- 172.21.15.113:0/3714302511 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900b5040 con 0x7f76a80c2d80 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f769f400640 1 -- 172.21.15.113:0/3714302511 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7690004220 con 0x7f76a80c2d80 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f769f400640 1 -- 172.21.15.113:0/3714302511 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690003620 con 0x7f76a80c2d80 2024-03-04T07:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f76ada00640 1 -- 172.21.15.113:0/3714302511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a8154f30 con 0x7f76a80c2d80 2024-03-04T07:34:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f76ada00640 1 -- 172.21.15.113:0/3714302511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a80c5c70 con 0x7f76a80c2d80 2024-03-04T07:34:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f76ada00640 1 -- 172.21.15.113:0/3714302511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a8075c00 con 0x7f76a80c2d80 2024-03-04T07:34:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.890+0000 7f769f400640 1 -- 172.21.15.113:0/3714302511 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76900037c0 con 0x7f76a80c2d80 2024-03-04T07:34:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.894+0000 7f769f400640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7684066f60 0x7f7684069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.894+0000 7f769f400640 1 -- 172.21.15.113:0/3714302511 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7690122490 con 0x7f76a80c2d80 2024-03-04T07:34:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.894+0000 7f76a6a00640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7684066f60 0x7f7684069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:11.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.894+0000 7f76a6a00640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7684066f60 0x7f7684069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7698009b60 tx=0x7f7698009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:11.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:11.898+0000 7f769f400640 1 -- 172.21.15.113:0/3714302511 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f76900c6030 con 0x7f76a80c2d80 2024-03-04T07:34:12.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.082+0000 7f76ada00640 1 -- 172.21.15.113:0/3714302511 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76a8155190 con 0x7f76a80c2d80 2024-03-04T07:34:12.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.082+0000 7f769f400640 1 -- 172.21.15.113:0/3714302511 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6609 (secure 0 0 0) 0x7f76900bd030 con 0x7f76a80c2d80 2024-03-04T07:34:12.086 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:12.086 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:07.656635+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:creating","state_seq":1,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:34:12.087 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 12 2024-03-04T07:34:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 -- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7684066f60 msgr2=0x7f7684069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7684066f60 0x7f7684069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7698009b60 tx=0x7f7698009190 comp rx=0 tx=0).stop 2024-03-04T07:34:12.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 -- 172.21.15.113:0/3714302511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c2d80 msgr2=0x7f76a8147b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c2d80 0x7f76a8147b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f76900b3040 tx=0x7f769001ba30 comp rx=0 tx=0).stop 2024-03-04T07:34:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 -- 172.21.15.113:0/3714302511 shutdown_connections 2024-03-04T07:34:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76a814e4c0 0x7f76a8148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7684066f60 0x7f7684069420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a80c2d80 0x7f76a8147b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 --2- 172.21.15.113:0/3714302511 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76a80c22e0 0x7f76a806b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 -- 172.21.15.113:0/3714302511 >> 172.21.15.113:0/3714302511 conn(0x7f76a806e820 msgr2=0x7f76a80c1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 -- 172.21.15.113:0/3714302511 shutdown_connections 2024-03-04T07:34:12.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:12.086+0000 7f76ada00640 1 -- 172.21.15.113:0/3714302511 wait complete. 2024-03-04T07:34:12.710 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-04T07:33:55.904916+0000', 'modified': '2024-03-04T07:34:07.656635+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4276, 'mds_1': 4256, 'mds_2': 4253}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4253': {'gid': 4253, 'name': 'a', 'rank': 2, 'incarnation': 12, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.113:6836/4038214481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 4038214481}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 4038214481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4256': {'gid': 4256, 'name': 'e', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.113:6837/1310042629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 1310042629}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 1310042629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4276': {'gid': 4276, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.136:6835/3348592059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 3348592059}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 3348592059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4276, 'qdb_cluster': [4276, 4256]} 2024-03-04T07:34:12.710 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4253:up:creating 2024-03-04T07:34:13.711 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:34:13.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ea00640 1 Processor -- start 2024-03-04T07:34:13.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ea00640 1 -- start start 2024-03-04T07:34:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf7806ff20 0x7fbf7815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbf780c8b90 0x7fbf7815c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ea00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf78152e30 con 0x7fbf78152a50 2024-03-04T07:34:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ea00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf780c31b0 con 0x7fbf780c8b90 2024-03-04T07:34:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7806f840 con 0x7fbf7806ff20 2024-03-04T07:34:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbf780c8b90 0x7fbf7815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbf780c8b90 0x7fbf7815c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:38106/0 (socket says 172.21.15.113:38106) 2024-03-04T07:34:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ca00640 1 -- 172.21.15.113:0/4196255628 learned_addr learned my addr 172.21.15.113:0/4196255628 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf77e00640 1 --1- 172.21.15.113:0/4196255628 >> v1:172.21.15.136:6789/0 conn(0x7fbf78152a50 0x7fbf780ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:35500/0 (socket says 172.21.15.113:35500) 2024-03-04T07:34:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf77400640 1 --2- 172.21.15.113:0/4196255628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf7806ff20 0x7fbf7815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:13.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ca00640 1 -- 172.21.15.113:0/4196255628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf7806ff20 msgr2=0x7fbf7815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:13.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ca00640 1 --2- 172.21.15.113:0/4196255628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf7806ff20 0x7fbf7815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:13.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ca00640 1 -- 172.21.15.113:0/4196255628 >> v1:172.21.15.136:6789/0 conn(0x7fbf78152a50 legacy=0x7fbf780ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:13.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.846+0000 7fbf7ca00640 1 -- 172.21.15.113:0/4196255628 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7806f280 con 0x7fbf780c8b90 2024-03-04T07:34:13.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf7ca00640 1 --2- 172.21.15.113:0/4196255628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf780c8b90 0x7fbf7815c310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009e50 tx=0x7fbf7001c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e501834cf8a4184c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:13.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf76a00640 1 -- 172.21.15.113:0/4196255628 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf700a5880 con 0x7fbf780c8b90 2024-03-04T07:34:13.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf76a00640 1 -- 172.21.15.113:0/4196255628 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf700a5a20 con 0x7fbf780c8b90 2024-03-04T07:34:13.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf76a00640 1 -- 172.21.15.113:0/4196255628 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf700a5df0 con 0x7fbf780c8b90 2024-03-04T07:34:13.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf7ea00640 1 -- 172.21.15.113:0/4196255628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf780c8b90 msgr2=0x7fbf7815c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:13.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf7ea00640 1 --2- 172.21.15.113:0/4196255628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf780c8b90 0x7fbf7815c310 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009e50 tx=0x7fbf7001c9a0 comp rx=0 tx=0).stop 2024-03-04T07:34:13.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf7ea00640 1 -- 172.21.15.113:0/4196255628 shutdown_connections 2024-03-04T07:34:13.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf7ea00640 1 --2- 172.21.15.113:0/4196255628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf780c8b90 0x7fbf7815c310 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:13.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf7ea00640 1 --2- 172.21.15.113:0/4196255628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf7806ff20 0x7fbf7815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:13.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf7ea00640 1 -- 172.21.15.113:0/4196255628 >> 172.21.15.113:0/4196255628 conn(0x7fbf780734d0 msgr2=0x7fbf78066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:13.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf7ea00640 1 -- 172.21.15.113:0/4196255628 shutdown_connections 2024-03-04T07:34:13.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.850+0000 7fbf7ea00640 1 -- 172.21.15.113:0/4196255628 wait complete. 2024-03-04T07:34:13.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 Processor -- start 2024-03-04T07:34:13.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 -- start start 2024-03-04T07:34:13.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf7806ff20 0x7fbf780c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:13.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf780c8b90 0x7fbf7815a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:13.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf78152a50 0x7fbf780c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:13.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf780ce9b0 con 0x7fbf7806ff20 2024-03-04T07:34:13.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf780ce830 con 0x7fbf78152a50 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf78165880 con 0x7fbf780c8b90 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf77e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf7806ff20 0x7fbf780c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf77e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf7806ff20 0x7fbf780c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59422/0 (socket says 172.21.15.113:59422) 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf78152a50 0x7fbf780c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf77e00640 1 -- 172.21.15.113:0/1161456926 learned_addr learned my addr 172.21.15.113:0/1161456926 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf77400640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf780c8b90 0x7fbf7815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf77e00640 1 -- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf780c8b90 msgr2=0x7fbf7815a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf77e00640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf780c8b90 0x7fbf7815a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf77e00640 1 -- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf78152a50 msgr2=0x7fbf780c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf77e00640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf78152a50 0x7fbf780c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf77e00640 1 -- 172.21.15.113:0/1161456926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7815b940 con 0x7fbf7806ff20 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf77e00640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf7806ff20 0x7fbf780c6a20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800c4c0 tx=0x7fbf6800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:13.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf6fe00640 1 -- 172.21.15.113:0/1161456926 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf68018070 con 0x7fbf7806ff20 2024-03-04T07:34:13.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf6fe00640 1 -- 172.21.15.113:0/1161456926 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf68007480 con 0x7fbf7806ff20 2024-03-04T07:34:13.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 -- 172.21.15.113:0/1161456926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf78159a40 con 0x7fbf7806ff20 2024-03-04T07:34:13.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf6fe00640 1 -- 172.21.15.113:0/1161456926 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680047f0 con 0x7fbf7806ff20 2024-03-04T07:34:13.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 -- 172.21.15.113:0/1161456926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7815a000 con 0x7fbf7806ff20 2024-03-04T07:34:13.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.854+0000 7fbf7ea00640 1 -- 172.21.15.113:0/1161456926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7815a200 con 0x7fbf7806ff20 2024-03-04T07:34:13.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.858+0000 7fbf6fe00640 1 -- 172.21.15.113:0/1161456926 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf6801b070 con 0x7fbf7806ff20 2024-03-04T07:34:13.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.858+0000 7fbf6fe00640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf58066f60 0x7fbf58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:13.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.858+0000 7fbf6fe00640 1 -- 172.21.15.113:0/1161456926 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf68084250 con 0x7fbf7806ff20 2024-03-04T07:34:13.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.858+0000 7fbf77400640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf58066f60 0x7fbf58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:13.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.858+0000 7fbf77400640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf58066f60 0x7fbf58069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf781562b0 tx=0x7fbf64009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:13.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:13.862+0000 7fbf6fe00640 1 -- 172.21.15.113:0/1161456926 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbf7815a200 con 0x7fbf7806ff20 2024-03-04T07:34:14.027 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T07:34:14.024+0000 7f2bac933640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-04T07:34:14.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.058+0000 7fbf7ea00640 1 -- 172.21.15.113:0/1161456926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf78075c00 con 0x7fbf7806ff20 2024-03-04T07:34:14.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.058+0000 7fbf6fe00640 1 -- 172.21.15.113:0/1161456926 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fbf68054c00 con 0x7fbf7806ff20 2024-03-04T07:34:14.063 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:14.064 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:34:14.064 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 -- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf58066f60 msgr2=0x7fbf58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf58066f60 0x7fbf58069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf781562b0 tx=0x7fbf64009190 comp rx=0 tx=0).stop 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 -- 172.21.15.113:0/1161456926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf7806ff20 msgr2=0x7fbf780c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf7806ff20 0x7fbf780c6a20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbf6800c4c0 tx=0x7fbf6800c980 comp rx=0 tx=0).stop 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 -- 172.21.15.113:0/1161456926 shutdown_connections 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf78152a50 0x7fbf780c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf780c8b90 0x7fbf7815a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf58066f60 0x7fbf58069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 --2- 172.21.15.113:0/1161456926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf7806ff20 0x7fbf780c6a20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 -- 172.21.15.113:0/1161456926 >> 172.21.15.113:0/1161456926 conn(0x7fbf780734d0 msgr2=0x7fbf780cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 -- 172.21.15.113:0/1161456926 shutdown_connections 2024-03-04T07:34:14.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.062+0000 7fbf7ea00640 1 -- 172.21.15.113:0/1161456926 wait complete. 2024-03-04T07:34:14.689 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-04T07:33:55.904916+0000', 'modified': '2024-03-04T07:34:13.676159+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4276, 'mds_1': 4256, 'mds_2': 4253}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4253': {'gid': 4253, 'name': 'a', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.113:6836/4038214481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 4038214481}, {'type': 'v1', 'addr': '172.21.15.113:6836', 'nonce': 4038214481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4256': {'gid': 4256, 'name': 'e', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.113:6837/1310042629', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6835', 'nonce': 1310042629}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 1310042629}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4276': {'gid': 4276, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.136:6835/3348592059', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 3348592059}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 3348592059}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4276, 'qdb_cluster': [4253, 4256, 4276]} 2024-03-04T07:34:14.689 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-03-04T07:34:14.689 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-04T07:34:14.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.822+0000 7ff663400640 1 Processor -- start 2024-03-04T07:34:14.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.822+0000 7ff663400640 1 -- start start 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff663400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff65c152b20 0x7ff65c070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff663400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff65c15c510 0x7ff65c163190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff663400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65c152f00 con 0x7ff65c0c8b00 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff663400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65c0c8f90 con 0x7ff65c152b20 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff663400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0c5320 con 0x7ff65c15c510 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff660a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff65c152b20 0x7ff65c070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff660a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff65c152b20 0x7ff65c070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59436/0 (socket says 172.21.15.113:59436) 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff661400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff65c15c510 0x7ff65c163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff660a00640 1 -- 172.21.15.113:0/434055498 learned_addr learned my addr 172.21.15.113:0/434055498 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff660a00640 1 -- 172.21.15.113:0/434055498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff65c15c510 msgr2=0x7ff65c163190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff660a00640 1 --2- 172.21.15.113:0/434055498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff65c15c510 0x7ff65c163190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff660a00640 1 -- 172.21.15.113:0/434055498 >> v1:172.21.15.136:6789/0 conn(0x7ff65c0c8b00 legacy=0x7ff65c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:14.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.826+0000 7ff660a00640 1 -- 172.21.15.113:0/434055498 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c0c5040 con 0x7ff65c152b20 2024-03-04T07:34:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff660a00640 1 --2- 172.21.15.113:0/434055498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 0x7ff65c070010 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff650009870 tx=0x7ff65001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2c552c5e6a3a31 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff65b400640 1 -- 172.21.15.113:0/434055498 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6500ab650 con 0x7ff65c152b20 2024-03-04T07:34:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff65b400640 1 -- 172.21.15.113:0/434055498 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6500b0070 con 0x7ff65c152b20 2024-03-04T07:34:14.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff65b400640 1 -- 172.21.15.113:0/434055498 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6500aba10 con 0x7ff65c152b20 2024-03-04T07:34:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff663400640 1 -- 172.21.15.113:0/434055498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 msgr2=0x7ff65c070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff663400640 1 --2- 172.21.15.113:0/434055498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 0x7ff65c070010 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff650009870 tx=0x7ff65001c560 comp rx=0 tx=0).stop 2024-03-04T07:34:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff663400640 1 -- 172.21.15.113:0/434055498 shutdown_connections 2024-03-04T07:34:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff663400640 1 --2- 172.21.15.113:0/434055498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff65c15c510 0x7ff65c163190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff663400640 1 --2- 172.21.15.113:0/434055498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 0x7ff65c070010 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff663400640 1 -- 172.21.15.113:0/434055498 >> 172.21.15.113:0/434055498 conn(0x7ff65c0709a0 msgr2=0x7ff65c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff663400640 1 -- 172.21.15.113:0/434055498 shutdown_connections 2024-03-04T07:34:14.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff663400640 1 -- 172.21.15.113:0/434055498 wait complete. 2024-03-04T07:34:14.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff663400640 1 Processor -- start 2024-03-04T07:34:14.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.830+0000 7ff663400640 1 -- start start 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff663400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff65c0c8b00 0x7ff65c0c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff663400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 0x7ff65c0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff663400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff65c15c510 0x7ff65c0cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff663400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c158810 con 0x7ff65c152b20 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff663400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0c3810 con 0x7ff65c0c8b00 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff663400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0c8490 con 0x7ff65c15c510 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff65be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 0x7ff65c0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff65be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 0x7ff65c0c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59446/0 (socket says 172.21.15.113:59446) 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff65be00640 1 -- 172.21.15.113:0/1896622493 learned_addr learned my addr 172.21.15.113:0/1896622493 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff660a00640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff65c0c8b00 0x7ff65c0c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff661400640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff65c15c510 0x7ff65c0cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff65be00640 1 -- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff65c15c510 msgr2=0x7ff65c0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:14.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff65be00640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff65c15c510 0x7ff65c0cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff65be00640 1 -- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff65c0c8b00 msgr2=0x7ff65c0c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff65be00640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff65c0c8b00 0x7ff65c0c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff65be00640 1 -- 172.21.15.113:0/1896622493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c1f39d0 con 0x7ff65c152b20 2024-03-04T07:34:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff65be00640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 0x7ff65c0c5fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff64800c3c0 tx=0x7ff64800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff658c00640 1 -- 172.21.15.113:0/1896622493 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648017070 con 0x7ff65c152b20 2024-03-04T07:34:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff658c00640 1 -- 172.21.15.113:0/1896622493 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6480052f0 con 0x7ff65c152b20 2024-03-04T07:34:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff658c00640 1 -- 172.21.15.113:0/1896622493 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648003840 con 0x7ff65c152b20 2024-03-04T07:34:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff663400640 1 -- 172.21.15.113:0/1896622493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65c1f4530 con 0x7ff65c152b20 2024-03-04T07:34:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.834+0000 7ff663400640 1 -- 172.21.15.113:0/1896622493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c1f3f70 con 0x7ff65c152b20 2024-03-04T07:34:14.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff658c00640 1 -- 172.21.15.113:0/1896622493 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff64801a070 con 0x7ff65c152b20 2024-03-04T07:34:14.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff66696f000 1 Processor -- start 2024-03-04T07:34:14.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff66696f000 1 -- start start 2024-03-04T07:34:14.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff658c00640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff63c066f90 0x7ff63c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff658c00640 1 -- 172.21.15.113:0/1896622493 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff648084320 con 0x7ff65c152b20 2024-03-04T07:34:14.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff63c066f90 0x7ff63c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff66696f000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d3fa0 0x559ebe7d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff66696f000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d48c0 0x559ebe7d4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff66696f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7df230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff66696f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebe5e0120 con 0x559ebe7d51e0 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff66696f000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebe567dc0 con 0x559ebe7d48c0 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff66696f000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ebe5dc1c0 con 0x559ebe7d3fa0 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7df230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7df230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59456/0 (socket says 172.21.15.113:59456) 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 -- 172.21.15.113:0/3494492815 learned_addr learned my addr 172.21.15.113:0/3494492815 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff65be00640 1 --2- 172.21.15.113:0/3494492815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d48c0 0x559ebe7d4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff660a00640 1 --2- 172.21.15.113:0/3494492815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d3fa0 0x559ebe7d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 -- 172.21.15.113:0/3494492815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d3fa0 msgr2=0x559ebe7d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 --2- 172.21.15.113:0/3494492815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d3fa0 0x559ebe7d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 -- 172.21.15.113:0/3494492815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d48c0 msgr2=0x559ebe7d4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 --2- 172.21.15.113:0/3494492815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d48c0 0x559ebe7d4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 -- 172.21.15.113:0/3494492815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ebe56bcc0 con 0x559ebe7d51e0 2024-03-04T07:34:14.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff63c066f90 0x7ff63c069450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff654015820 tx=0x7ff654015c20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:14.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff661400640 1 --2- 172.21.15.113:0/3494492815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7df230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff654018730 tx=0x7ff654020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dec02e82bb69386 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:14.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff64d600640 1 -- 172.21.15.113:0/3494492815 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65401ace0 con 0x559ebe7d51e0 2024-03-04T07:34:14.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff64d600640 1 -- 172.21.15.113:0/3494492815 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff654028070 con 0x559ebe7d51e0 2024-03-04T07:34:14.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.838+0000 7ff64d600640 1 -- 172.21.15.113:0/3494492815 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6540235f0 con 0x559ebe7d51e0 2024-03-04T07:34:14.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- 172.21.15.113:0/3494492815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 msgr2=0x559ebe7df230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:14.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 --2- 172.21.15.113:0/3494492815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7df230 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff654018730 tx=0x7ff654020000 comp rx=0 tx=0).stop 2024-03-04T07:34:14.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- 172.21.15.113:0/3494492815 shutdown_connections 2024-03-04T07:34:14.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 --2- 172.21.15.113:0/3494492815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7df230 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 --2- 172.21.15.113:0/3494492815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d48c0 0x559ebe7d4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 --2- 172.21.15.113:0/3494492815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d3fa0 0x559ebe7d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- 172.21.15.113:0/3494492815 >> 172.21.15.113:0/3494492815 conn(0x559ebe67a990 msgr2=0x559ebe7d1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:14.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- 172.21.15.113:0/3494492815 shutdown_connections 2024-03-04T07:34:14.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- 172.21.15.113:0/3494492815 wait complete. 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- start start 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 Processor -- start 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d3fa0 0x559ebe793390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d48c0 0x559ebe7938d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d51e0 0x559ebe793e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebe59a9c0 con 0x559ebe7d48c0 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebe69df90 con 0x559ebe7d3fa0 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ebe5da5e0 con 0x559ebe7d51e0 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 10 client.0 _ll_register_callbacks cb 0x559ebe77e4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff660a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d3fa0 0x559ebe793390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff661400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d51e0 0x559ebe793e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff660a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d3fa0 0x559ebe793390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:38130/0 (socket says 172.21.15.113:38130) 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff65be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d48c0 0x559ebe7938d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff660a00640 1 -- 172.21.15.113:0/733282063 learned_addr learned my addr 172.21.15.113:0/733282063 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff65be00640 1 -- 172.21.15.113:0/733282063 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d51e0 msgr2=0x559ebe793e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff65be00640 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d51e0 0x559ebe793e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff65be00640 1 -- 172.21.15.113:0/733282063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d3fa0 msgr2=0x559ebe793390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff65be00640 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d3fa0 0x559ebe793390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff65be00640 1 -- 172.21.15.113:0/733282063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ebe59c550 con 0x559ebe7d48c0 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff65be00640 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d48c0 0x559ebe7938d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff6480827b0 tx=0x7ff648054bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2ab7048a030407 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff642000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff642000640 1 -- 172.21.15.113:0/733282063 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648051220 con 0x559ebe7d48c0 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff642000640 1 -- 172.21.15.113:0/733282063 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff648068b50 con 0x559ebe7d48c0 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- 172.21.15.113:0/733282063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff654028070 con 0x559ebe7d48c0 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff642000640 1 -- 172.21.15.113:0/733282063 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6480513c0 con 0x559ebe7d48c0 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 10 client.4650 fetch_fsmap learned FSMap version 13 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff66696f000 1 -- 172.21.15.113:0/733282063 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x559ebe568010 con 0x559ebe7d48c0 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.842+0000 7ff642000640 1 -- 172.21.15.113:0/733282063 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff654028070 con 0x559ebe7d48c0 2024-03-04T07:34:14.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.846+0000 7ff642000640 1 -- 172.21.15.113:0/733282063 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 13) v1 ==== 4504+0+0 (secure 0 0 0) 0x7ff648054000 con 0x559ebe7d48c0 2024-03-04T07:34:14.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.846+0000 7ff66696f000 10 client.4650 fetch_fsmap finished waiting for FSMap version 13 2024-03-04T07:34:14.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.846+0000 7ff66696f000 10 client.4650 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-04T07:34:14.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.846+0000 7ff66696f000 20 client.4650 populate_metadata read hostname 'smithi113' 2024-03-04T07:34:14.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.846+0000 7ff66696f000 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.113:6834/4038214481,v1:172.21.15.113:6836/4038214481] conn(0x559ebe7e8390 0x559ebe7ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:14.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.846+0000 7ff66696f000 4 client.4650 mds_command: new command op to 4253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T07:34:14.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.846+0000 7ff66696f000 1 -- 172.21.15.113:0/733282063 --> [v2:172.21.15.113:6834/4038214481,v1:172.21.15.113:6836/4038214481] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ebe57ba50 con 0x559ebe7e8390 2024-03-04T07:34:14.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.846+0000 7ff660a00640 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.113:6834/4038214481,v1:172.21.15.113:6836/4038214481] conn(0x559ebe7e8390 0x559ebe7ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:14.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.846+0000 7ff660a00640 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.113:6834/4038214481,v1:172.21.15.113:6836/4038214481] conn(0x559ebe7e8390 0x559ebe7ea780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df6897c919eb7266 server_cookie=84cccd683051ebd3 in_seq=0 out_seq=0 2024-03-04T07:34:14.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:14.846+0000 7ff642000640 10 client.4650 ms_handle_connect on v2:172.21.15.113:6834/4038214481 2024-03-04T07:34:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff642000640 1 -- 172.21.15.113:0/733282063 <== mds.2 v2:172.21.15.113:6834/4038214481 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x559ebe57ba50 con 0x559ebe7e8390 2024-03-04T07:34:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff642000640 10 client.4650 handle_command_reply: tid=0 2024-03-04T07:34:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 client.4650 shutdown 2024-03-04T07:34:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 -- 172.21.15.113:0/733282063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d48c0 msgr2=0x559ebe7938d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d48c0 0x559ebe7938d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff6480827b0 tx=0x7ff648054bb0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 -- 172.21.15.113:0/733282063 shutdown_connections 2024-03-04T07:34:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.113:6834/4038214481,v1:172.21.15.113:6836/4038214481] conn(0x559ebe7e8390 0x559ebe7ea780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d51e0 0x559ebe793e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d48c0 0x559ebe7938d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 --2- 172.21.15.113:0/733282063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d3fa0 0x559ebe793390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 -- 172.21.15.113:0/733282063 >> 172.21.15.113:0/733282063 conn(0x559ebe67a990 msgr2=0x559ebe7d1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff642000640 0 client.4650 ms_handle_reset on v2:172.21.15.113:6834/4038214481 2024-03-04T07:34:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 -- 172.21.15.113:0/733282063 shutdown_connections 2024-03-04T07:34:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 1 -- 172.21.15.113:0/733282063 wait complete. 2024-03-04T07:34:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.681+0000 7ff66696f000 20 client.4650 trim_cache size 0 max 16384 2024-03-04T07:34:18.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 Processor -- start 2024-03-04T07:34:18.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 -- start start 2024-03-04T07:34:18.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d3fa0 0x559ebe7e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d48c0 0x559ebe7e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c1f7930 con 0x559ebe7d51e0 2024-03-04T07:34:18.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c1f6d30 con 0x559ebe7d48c0 2024-03-04T07:34:18.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c1f7330 con 0x559ebe7d3fa0 2024-03-04T07:34:18.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff661400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff661400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7e9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56556/0 (socket says 172.21.15.113:56556) 2024-03-04T07:34:18.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff661400640 1 -- 172.21.15.113:0/1396045771 learned_addr learned my addr 172.21.15.113:0/1396045771 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:18.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff65be00640 1 --2- 172.21.15.113:0/1396045771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d48c0 0x559ebe7e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff660a00640 1 --2- 172.21.15.113:0/1396045771 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d3fa0 0x559ebe7e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff661400640 1 -- 172.21.15.113:0/1396045771 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d3fa0 msgr2=0x559ebe7e93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff661400640 1 --2- 172.21.15.113:0/1396045771 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d3fa0 0x559ebe7e93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff661400640 1 -- 172.21.15.113:0/1396045771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d48c0 msgr2=0x559ebe7e98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff661400640 1 --2- 172.21.15.113:0/1396045771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d48c0 0x559ebe7e98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff661400640 1 -- 172.21.15.113:0/1396045771 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ebe7d46f0 con 0x559ebe7d51e0 2024-03-04T07:34:18.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff661400640 1 --2- 172.21.15.113:0/1396045771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7e9e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff654024000 tx=0x7ff65401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e802587543f530f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:18.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff64cc00640 1 -- 172.21.15.113:0/1396045771 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654037480 con 0x559ebe7d51e0 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff64cc00640 1 -- 172.21.15.113:0/1396045771 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff654028040 con 0x559ebe7d51e0 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff64cc00640 1 -- 172.21.15.113:0/1396045771 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654037820 con 0x559ebe7d51e0 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 -- 172.21.15.113:0/1396045771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 msgr2=0x559ebe7e9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 --2- 172.21.15.113:0/1396045771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7e9e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff654024000 tx=0x7ff65401f000 comp rx=0 tx=0).stop 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 -- 172.21.15.113:0/1396045771 shutdown_connections 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 --2- 172.21.15.113:0/1396045771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d51e0 0x559ebe7e9e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 --2- 172.21.15.113:0/1396045771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d48c0 0x559ebe7e98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 --2- 172.21.15.113:0/1396045771 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d3fa0 0x559ebe7e93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 -- 172.21.15.113:0/1396045771 >> 172.21.15.113:0/1396045771 conn(0x559ebe67a990 msgr2=0x559ebe7cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 -- 172.21.15.113:0/1396045771 shutdown_connections 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.705+0000 7ff66696f000 1 -- 172.21.15.113:0/1396045771 wait complete. 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 -- start start 2024-03-04T07:34:18.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 Processor -- start 2024-03-04T07:34:18.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d3fa0 0x559ebe7fda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d48c0 0x559ebe7fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d51e0 0x559ebe7fe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebe585860 con 0x559ebe7d3fa0 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ebe5887c0 con 0x559ebe7d51e0 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ebe5df430 con 0x559ebe7d48c0 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 10 client.0 _ll_register_callbacks cb 0x7ff648054780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff660a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d3fa0 0x559ebe7fda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff660a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d3fa0 0x559ebe7fda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56562/0 (socket says 172.21.15.113:56562) 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff660a00640 1 -- 172.21.15.113:0/2183978135 learned_addr learned my addr 172.21.15.113:0/2183978135 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff661400640 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d51e0 0x559ebe7fe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff65be00640 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d48c0 0x559ebe7fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff660a00640 1 -- 172.21.15.113:0/2183978135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d48c0 msgr2=0x559ebe7fdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff660a00640 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d48c0 0x559ebe7fdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff660a00640 1 -- 172.21.15.113:0/2183978135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d51e0 msgr2=0x559ebe7fe4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff660a00640 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d51e0 0x559ebe7fe4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff660a00640 1 -- 172.21.15.113:0/2183978135 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ebe5ade30 con 0x559ebe7d3fa0 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff660a00640 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d3fa0 0x559ebe7fda70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff65001c5a0 tx=0x7ff6500046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c04dd9556cd552c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff642000640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff642000640 1 -- 172.21.15.113:0/2183978135 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6500c8070 con 0x559ebe7d3fa0 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff642000640 1 -- 172.21.15.113:0/2183978135 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6500c4840 con 0x559ebe7d3fa0 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 -- 172.21.15.113:0/2183978135 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff654028040 con 0x559ebe7d3fa0 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff642000640 1 -- 172.21.15.113:0/2183978135 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6500d2050 con 0x559ebe7d3fa0 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 10 client.4656 fetch_fsmap learned FSMap version 13 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 -- 172.21.15.113:0/2183978135 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x559ebe6390c0 con 0x559ebe7d3fa0 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff642000640 1 -- 172.21.15.113:0/2183978135 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff654028040 con 0x559ebe7d3fa0 2024-03-04T07:34:18.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff642000640 1 -- 172.21.15.113:0/2183978135 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 13) v1 ==== 4504+0+0 (secure 0 0 0) 0x7ff6500d5020 con 0x559ebe7d3fa0 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 10 client.4656 fetch_fsmap finished waiting for FSMap version 13 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 10 client.4656 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 20 client.4656 populate_metadata read hostname 'smithi113' 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.113:6834/4038214481,v1:172.21.15.113:6836/4038214481] conn(0x559ebe843530 0x559ebe845920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 4 client.4656 mds_command: new command op to 4253 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.709+0000 7ff66696f000 1 -- 172.21.15.113:0/2183978135 --> [v2:172.21.15.113:6834/4038214481,v1:172.21.15.113:6836/4038214481] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559ebe5ed660 con 0x559ebe843530 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff65be00640 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.113:6834/4038214481,v1:172.21.15.113:6836/4038214481] conn(0x559ebe843530 0x559ebe845920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff65be00640 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.113:6834/4038214481,v1:172.21.15.113:6836/4038214481] conn(0x559ebe843530 0x559ebe845920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=325a0ea323865b10 server_cookie=f7cf044b910db0a6 in_seq=0 out_seq=0 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff642000640 10 client.4656 ms_handle_connect on v2:172.21.15.113:6834/4038214481 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff642000640 1 -- 172.21.15.113:0/2183978135 <== mds.2 v2:172.21.15.113:6834/4038214481 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x559ebe5ed660 con 0x559ebe843530 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff642000640 10 client.4656 handle_command_reply: tid=0 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 client.4656 shutdown 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 -- 172.21.15.113:0/2183978135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d3fa0 msgr2=0x559ebe7fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d3fa0 0x559ebe7fda70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff65001c5a0 tx=0x7ff6500046d0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 -- 172.21.15.113:0/2183978135 shutdown_connections 2024-03-04T07:34:18.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.113:6834/4038214481,v1:172.21.15.113:6836/4038214481] conn(0x559ebe843530 0x559ebe845920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ebe7d51e0 0x559ebe7fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x559ebe7d48c0 0x559ebe7fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 --2- 172.21.15.113:0/2183978135 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559ebe7d3fa0 0x559ebe7fda70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff642000640 0 client.4656 ms_handle_reset on v2:172.21.15.113:6834/4038214481 2024-03-04T07:34:18.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 -- 172.21.15.113:0/2183978135 >> 172.21.15.113:0/2183978135 conn(0x559ebe67a990 msgr2=0x559ebe7e8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:18.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 -- 172.21.15.113:0/2183978135 shutdown_connections 2024-03-04T07:34:18.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 1 -- 172.21.15.113:0/2183978135 wait complete. 2024-03-04T07:34:18.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff66696f000 20 client.4656 trim_cache size 0 max 16384 2024-03-04T07:34:18.721 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-03-04T07:34:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "43451c9f-4d9f-4f78-a5ec-a82e7d344007", 2024-03-04T07:34:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 2, 2024-03-04T07:34:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 4253, 2024-03-04T07:34:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-03-04T07:34:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-03-04T07:34:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-03-04T07:34:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 11.0574314, 2024-03-04T07:34:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 13, 2024-03-04T07:34:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 21, 2024-03-04T07:34:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 21, 2024-03-04T07:34:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 29.679174654000001 2024-03-04T07:34:18.722 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-03-04T07:34:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 -- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff63c066f90 msgr2=0x7ff63c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff63c066f90 0x7ff63c069450 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff654015820 tx=0x7ff654015c20 comp rx=0 tx=0).stop 2024-03-04T07:34:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 -- 172.21.15.113:0/1896622493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 msgr2=0x7ff65c0c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 0x7ff65c0c5fd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff64800c3c0 tx=0x7ff64800c880 comp rx=0 tx=0).stop 2024-03-04T07:34:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 -- 172.21.15.113:0/1896622493 shutdown_connections 2024-03-04T07:34:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff65c15c510 0x7ff65c0cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff63c066f90 0x7ff63c069450 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff65c152b20 0x7ff65c0c5fd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 --2- 172.21.15.113:0/1896622493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff65c0c8b00 0x7ff65c0c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 -- 172.21.15.113:0/1896622493 >> 172.21.15.113:0/1896622493 conn(0x7ff65c0709a0 msgr2=0x7ff65c157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:18.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 -- 172.21.15.113:0/1896622493 shutdown_connections 2024-03-04T07:34:18.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.713+0000 7ff663400640 1 -- 172.21.15.113:0/1896622493 wait complete. 2024-03-04T07:34:18.736 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-04T07:34:18.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f16a4a00640 1 Processor -- start 2024-03-04T07:34:18.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f16a4a00640 1 -- start start 2024-03-04T07:34:18.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f16a4a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f16a00c8200 0x7f16a0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f16a4a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a0152bd0 0x7f16a01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f16a4a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a01591f0 con 0x7f16a01527f0 2024-03-04T07:34:18.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f16a4a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a0071df0 con 0x7f16a00c8200 2024-03-04T07:34:18.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f16a4a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01597b0 con 0x7f16a0152bd0 2024-03-04T07:34:18.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f169de00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f16a01527f0 0x7f16a0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:60662/0 (socket says 172.21.15.113:60662) 2024-03-04T07:34:18.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f169de00640 1 -- 172.21.15.113:0/244367757 learned_addr learned my addr 172.21.15.113:0/244367757 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:18.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f1697e00640 1 --2- 172.21.15.113:0/244367757 >> v2:172.21.15.136:3300/0 conn(0x7f16a00c8200 0x7f16a0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.869+0000 7f169e800640 1 --2- 172.21.15.113:0/244367757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a0152bd0 0x7f16a01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f169d400640 1 -- 172.21.15.113:0/244367757 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669656969 0 0) 0x7f16a01591f0 con 0x7f16a01527f0 2024-03-04T07:34:18.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f169d400640 1 -- 172.21.15.113:0/244367757 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1684003530 con 0x7f16a01527f0 2024-03-04T07:34:18.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f169e800640 1 -- 172.21.15.113:0/244367757 >> v2:172.21.15.136:3300/0 conn(0x7f16a00c8200 msgr2=0x7f16a0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f169e800640 1 --2- 172.21.15.113:0/244367757 >> v2:172.21.15.136:3300/0 conn(0x7f16a00c8200 0x7f16a0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f169e800640 1 -- 172.21.15.113:0/244367757 >> v1:172.21.15.136:6789/0 conn(0x7f16a01527f0 legacy=0x7f16a0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f169e800640 1 -- 172.21.15.113:0/244367757 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a01594d0 con 0x7f16a0152bd0 2024-03-04T07:34:18.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f169e800640 1 --2- 172.21.15.113:0/244367757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a0152bd0 0x7f16a01587d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1698009d90 tx=0x7f169801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39105ce85e46928a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:18.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f169d400640 1 -- 172.21.15.113:0/244367757 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16980b2070 con 0x7f16a0152bd0 2024-03-04T07:34:18.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f169d400640 1 -- 172.21.15.113:0/244367757 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16980a5920 con 0x7f16a0152bd0 2024-03-04T07:34:18.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f169d400640 1 -- 172.21.15.113:0/244367757 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16980a5cd0 con 0x7f16a0152bd0 2024-03-04T07:34:18.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f16a4a00640 1 -- 172.21.15.113:0/244367757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a0152bd0 msgr2=0x7f16a01587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.873+0000 7f16a4a00640 1 --2- 172.21.15.113:0/244367757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a0152bd0 0x7f16a01587d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1698009d90 tx=0x7f169801c970 comp rx=0 tx=0).stop 2024-03-04T07:34:18.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 -- 172.21.15.113:0/244367757 shutdown_connections 2024-03-04T07:34:18.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 --2- 172.21.15.113:0/244367757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a0152bd0 0x7f16a01587d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 --2- 172.21.15.113:0/244367757 >> v2:172.21.15.136:3300/0 conn(0x7f16a00c8200 0x7f16a0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 -- 172.21.15.113:0/244367757 >> 172.21.15.113:0/244367757 conn(0x7f16a006e820 msgr2=0x7f16a006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:18.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 -- 172.21.15.113:0/244367757 shutdown_connections 2024-03-04T07:34:18.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 -- 172.21.15.113:0/244367757 wait complete. 2024-03-04T07:34:18.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 Processor -- start 2024-03-04T07:34:18.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 -- start start 2024-03-04T07:34:18.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a00c8200 0x7f16a01f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a01527f0 0x7f16a00cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a0152bd0 0x7f16a00af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01654f0 con 0x7f16a01527f0 2024-03-04T07:34:18.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0165370 con 0x7f16a0152bd0 2024-03-04T07:34:18.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0165670 con 0x7f16a00c8200 2024-03-04T07:34:18.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1697e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a01527f0 0x7f16a00cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1697e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a01527f0 0x7f16a00cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56574/0 (socket says 172.21.15.113:56574) 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1697e00640 1 -- 172.21.15.113:0/2113201168 learned_addr learned my addr 172.21.15.113:0/2113201168 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f169de00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a00c8200 0x7f16a01f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f169e800640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a0152bd0 0x7f16a00af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1697e00640 1 -- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a00c8200 msgr2=0x7f16a01f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1697e00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a00c8200 0x7f16a01f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1697e00640 1 -- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a0152bd0 msgr2=0x7f16a00af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1697e00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a0152bd0 0x7f16a00af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1697e00640 1 -- 172.21.15.113:0/2113201168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0148220 con 0x7f16a01527f0 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f169e800640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a0152bd0 0x7f16a00af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1697e00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a01527f0 0x7f16a00cb430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f168800e930 tx=0x7f168800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1696000640 1 -- 172.21.15.113:0/2113201168 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688017070 con 0x7f16a01527f0 2024-03-04T07:34:18.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1696000640 1 -- 172.21.15.113:0/2113201168 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168800ca00 con 0x7f16a01527f0 2024-03-04T07:34:18.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 -- 172.21.15.113:0/2113201168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a01484c0 con 0x7f16a01527f0 2024-03-04T07:34:18.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f1696000640 1 -- 172.21.15.113:0/2113201168 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168800cdd0 con 0x7f16a01527f0 2024-03-04T07:34:18.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.877+0000 7f16a4a00640 1 -- 172.21.15.113:0/2113201168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a01f3a40 con 0x7f16a01527f0 2024-03-04T07:34:18.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.881+0000 7f1696000640 1 -- 172.21.15.113:0/2113201168 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f168801a070 con 0x7f16a01527f0 2024-03-04T07:34:18.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.881+0000 7f1696000640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f167c066f60 0x7f167c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.881+0000 7f1696000640 1 -- 172.21.15.113:0/2113201168 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16880850b0 con 0x7f16a01527f0 2024-03-04T07:34:18.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.881+0000 7f169de00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f167c066f60 0x7f167c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.881+0000 7f16a7eab000 1 Processor -- start 2024-03-04T07:34:18.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.881+0000 7f16a7eab000 1 -- start start 2024-03-04T07:34:18.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.881+0000 7f169de00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f167c066f60 0x7f167c069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16900011f0 tx=0x7f1690007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:18.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b37438fa0 0x557b37439380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b374398c0 0x557b37439ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b3743a1e0 0x557b37444230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b37245120 con 0x557b3743a1e0 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b371ccdc0 con 0x557b37438fa0 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b372411c0 con 0x557b374398c0 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f169de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b37438fa0 0x557b37439380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f169e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b3743a1e0 0x557b37444230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f1697e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b374398c0 0x557b37439ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f169de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b37438fa0 0x557b37439380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:49052/0 (socket says 172.21.15.113:49052) 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f169de00640 1 -- 172.21.15.113:0/1565628614 learned_addr learned my addr 172.21.15.113:0/1565628614 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f169de00640 1 -- 172.21.15.113:0/1565628614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b374398c0 msgr2=0x557b37439ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f169de00640 1 --2- 172.21.15.113:0/1565628614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b374398c0 0x557b37439ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f169de00640 1 -- 172.21.15.113:0/1565628614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b3743a1e0 msgr2=0x557b37444230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f169de00640 1 --2- 172.21.15.113:0/1565628614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b3743a1e0 0x557b37444230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f169de00640 1 -- 172.21.15.113:0/1565628614 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b371d0cc0 con 0x557b37438fa0 2024-03-04T07:34:18.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f169de00640 1 --2- 172.21.15.113:0/1565628614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b37438fa0 0x557b37439380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1690006a00 tx=0x7f1690012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad2299d8c0f7371 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:18.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f168ea00640 1 -- 172.21.15.113:0/1565628614 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690026070 con 0x557b37438fa0 2024-03-04T07:34:18.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f168ea00640 1 -- 172.21.15.113:0/1565628614 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1690005e40 con 0x557b37438fa0 2024-03-04T07:34:18.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f168ea00640 1 -- 172.21.15.113:0/1565628614 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690001940 con 0x557b37438fa0 2024-03-04T07:34:18.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 -- 172.21.15.113:0/1565628614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b37438fa0 msgr2=0x557b37439380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1565628614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b37438fa0 0x557b37439380 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1690006a00 tx=0x7f1690012000 comp rx=0 tx=0).stop 2024-03-04T07:34:18.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 -- 172.21.15.113:0/1565628614 shutdown_connections 2024-03-04T07:34:18.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1565628614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b3743a1e0 0x557b37444230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1565628614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b374398c0 0x557b37439ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1565628614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b37438fa0 0x557b37439380 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 -- 172.21.15.113:0/1565628614 >> 172.21.15.113:0/1565628614 conn(0x557b372df990 msgr2=0x557b37436d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:18.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 -- 172.21.15.113:0/1565628614 shutdown_connections 2024-03-04T07:34:18.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 -- 172.21.15.113:0/1565628614 wait complete. 2024-03-04T07:34:18.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 -- start start 2024-03-04T07:34:18.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.885+0000 7f16a7eab000 1 Processor -- start 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 0x557b373f8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b373f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b373f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b371ff9c0 con 0x557b374398c0 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b37302f90 con 0x557b3743a1e0 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b3723f5e0 con 0x557b37438fa0 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 10 client.0 _ll_register_callbacks cb 0x557b373e34f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1697e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b373f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1697e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b373f88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56580/0 (socket says 172.21.15.113:56580) 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1697e00640 1 -- 172.21.15.113:0/1687066110 learned_addr learned my addr 172.21.15.113:0/1687066110 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f169de00640 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 0x557b373f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f169e800640 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b373f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1697e00640 1 -- 172.21.15.113:0/1687066110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 msgr2=0x557b373f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1697e00640 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 0x557b373f8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1697e00640 1 -- 172.21.15.113:0/1687066110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 msgr2=0x557b373f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1697e00640 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b373f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1697e00640 1 -- 172.21.15.113:0/1687066110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b37201550 con 0x557b374398c0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f169de00640 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 0x557b373f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1697e00640 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b373f88b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f168800c520 tx=0x7f1688059bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9256084c535129d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1683400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1683400640 1 -- 172.21.15.113:0/1687066110 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688080070 con 0x557b374398c0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1683400640 1 -- 172.21.15.113:0/1687066110 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16880517e0 con 0x557b374398c0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 -- 172.21.15.113:0/1687066110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b371e0a50 con 0x557b374398c0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1683400640 1 -- 172.21.15.113:0/1687066110 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1688021040 con 0x557b374398c0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 10 client.4665 fetch_fsmap learned FSMap version 13 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 -- 172.21.15.113:0/1687066110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x557b371cd010 con 0x557b374398c0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1683400640 1 -- 172.21.15.113:0/1687066110 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b371e0a50 con 0x557b374398c0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1683400640 1 -- 172.21.15.113:0/1687066110 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 13) v1 ==== 4504+0+0 (secure 0 0 0) 0x7f1688024060 con 0x557b374398c0 2024-03-04T07:34:18.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 10 client.4665 fetch_fsmap finished waiting for FSMap version 13 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 10 client.4665 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 20 client.4665 populate_metadata read hostname 'smithi113' 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.113:6835/1310042629,v1:172.21.15.113:6837/1310042629] conn(0x557b3744d390 0x557b3744f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 4 client.4665 mds_command: new command op to 4256 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 -- 172.21.15.113:0/1687066110 --> [v2:172.21.15.113:6835/1310042629,v1:172.21.15.113:6837/1310042629] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b37252660 con 0x557b3744d390 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f169de00640 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.113:6835/1310042629,v1:172.21.15.113:6837/1310042629] conn(0x557b3744d390 0x557b3744f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1683400640 10 client.4665 ms_handle_connect on v2:172.21.15.113:6835/1310042629 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f169de00640 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.113:6835/1310042629,v1:172.21.15.113:6837/1310042629] conn(0x557b3744d390 0x557b3744f780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b129598c5ec98b07 server_cookie=1688a02fed7736a4 in_seq=0 out_seq=0 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1683400640 1 -- 172.21.15.113:0/1687066110 <== mds.1 v2:172.21.15.113:6835/1310042629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x557b37252660 con 0x557b3744d390 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f1683400640 10 client.4665 handle_command_reply: tid=0 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.889+0000 7f16a7eab000 1 client.4665 shutdown 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 1 -- 172.21.15.113:0/1687066110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 msgr2=0x557b373f88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b373f88b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f168800c520 tx=0x7f1688059bf0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 1 -- 172.21.15.113:0/1687066110 shutdown_connections 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.113:6835/1310042629,v1:172.21.15.113:6837/1310042629] conn(0x557b3744d390 0x557b3744f780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b373f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b373f88b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1687066110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 0x557b373f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 1 -- 172.21.15.113:0/1687066110 >> 172.21.15.113:0/1687066110 conn(0x557b372df990 msgr2=0x557b37436730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:18.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f1683400640 0 client.4665 ms_handle_reset on v2:172.21.15.113:6835/1310042629 2024-03-04T07:34:18.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 1 -- 172.21.15.113:0/1687066110 shutdown_connections 2024-03-04T07:34:18.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 1 -- 172.21.15.113:0/1687066110 wait complete. 2024-03-04T07:34:18.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.893+0000 7f16a7eab000 20 client.4665 trim_cache size 0 max 16384 2024-03-04T07:34:18.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 Processor -- start 2024-03-04T07:34:18.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 -- start start 2024-03-04T07:34:18.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 0x557b37451e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b3744e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b3744e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0145f40 con 0x557b374398c0 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01f7d00 con 0x557b3743a1e0 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0145940 con 0x557b37438fa0 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f1697e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b3744e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f1697e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b3744e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56596/0 (socket says 172.21.15.113:56596) 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f1697e00640 1 -- 172.21.15.113:0/1483611247 learned_addr learned my addr 172.21.15.113:0/1483611247 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f169de00640 1 --2- 172.21.15.113:0/1483611247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 0x557b37451e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f169e800640 1 --2- 172.21.15.113:0/1483611247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b3744e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f1697e00640 1 -- 172.21.15.113:0/1483611247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 msgr2=0x557b37451e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f1697e00640 1 --2- 172.21.15.113:0/1483611247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 0x557b37451e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f1697e00640 1 -- 172.21.15.113:0/1483611247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 msgr2=0x557b3744e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f1697e00640 1 --2- 172.21.15.113:0/1483611247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b3744e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f1697e00640 1 -- 172.21.15.113:0/1483611247 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b374396f0 con 0x557b374398c0 2024-03-04T07:34:18.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f1697e00640 1 --2- 172.21.15.113:0/1483611247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b3744e3a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1688055470 tx=0x7f1688058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780727595b35efbc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:18.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f168e000640 1 -- 172.21.15.113:0/1483611247 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688080050 con 0x557b374398c0 2024-03-04T07:34:18.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f168e000640 1 -- 172.21.15.113:0/1483611247 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1688053b30 con 0x557b374398c0 2024-03-04T07:34:18.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f168e000640 1 -- 172.21.15.113:0/1483611247 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688051a40 con 0x557b374398c0 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 -- 172.21.15.113:0/1483611247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 msgr2=0x557b3744e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1483611247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b3744e3a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1688055470 tx=0x7f1688058000 comp rx=0 tx=0).stop 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 -- 172.21.15.113:0/1483611247 shutdown_connections 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1483611247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b3744e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1483611247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b374398c0 0x557b3744e3a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1483611247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b37438fa0 0x557b37451e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 -- 172.21.15.113:0/1483611247 >> 172.21.15.113:0/1483611247 conn(0x557b372df990 msgr2=0x557b37432b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 -- 172.21.15.113:0/1483611247 shutdown_connections 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.909+0000 7f16a7eab000 1 -- 172.21.15.113:0/1483611247 wait complete. 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f16a7eab000 1 -- start start 2024-03-04T07:34:18.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f16a7eab000 1 Processor -- start 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b37438fa0 0x557b374519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b374398c0 0x557b3749fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f16a7eab000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b374a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f16a7eab000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b371ea860 con 0x557b37438fa0 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f16a7eab000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b371ed7c0 con 0x557b3743a1e0 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f16a7eab000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b37244430 con 0x557b374398c0 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f16a7eab000 10 client.0 _ll_register_callbacks cb 0x7f16880046d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f169de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b37438fa0 0x557b374519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f169de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b37438fa0 0x557b374519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56602/0 (socket says 172.21.15.113:56602) 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f169de00640 1 -- 172.21.15.113:0/1601071049 learned_addr learned my addr 172.21.15.113:0/1601071049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f169e800640 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b374a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f1697e00640 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b374398c0 0x557b3749fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f169de00640 1 -- 172.21.15.113:0/1601071049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b374398c0 msgr2=0x557b3749fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f169de00640 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b374398c0 0x557b3749fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f169de00640 1 -- 172.21.15.113:0/1601071049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 msgr2=0x557b374a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f169de00640 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b374a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f169de00640 1 -- 172.21.15.113:0/1601071049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b37212e30 con 0x557b37438fa0 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f169de00640 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b37438fa0 0x557b374519d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1690024040 tx=0x7f1690000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc5d663cde7f3bb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.913+0000 7f1683400640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1683400640 1 -- 172.21.15.113:0/1601071049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690026040 con 0x557b37438fa0 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1683400640 1 -- 172.21.15.113:0/1601071049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1690001700 con 0x557b37438fa0 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 -- 172.21.15.113:0/1601071049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1688053b30 con 0x557b37438fa0 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1683400640 1 -- 172.21.15.113:0/1601071049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1690042050 con 0x557b37438fa0 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 10 client.4671 fetch_fsmap learned FSMap version 13 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 -- 172.21.15.113:0/1601071049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x557b3729e0c0 con 0x557b37438fa0 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1683400640 1 -- 172.21.15.113:0/1601071049 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1688053b30 con 0x557b37438fa0 2024-03-04T07:34:18.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1683400640 1 -- 172.21.15.113:0/1601071049 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 13) v1 ==== 4504+0+0 (secure 0 0 0) 0x7f1690045020 con 0x557b37438fa0 2024-03-04T07:34:18.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 10 client.4671 fetch_fsmap finished waiting for FSMap version 13 2024-03-04T07:34:18.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 10 client.4671 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-04T07:34:18.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 20 client.4671 populate_metadata read hostname 'smithi113' 2024-03-04T07:34:18.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.113:6835/1310042629,v1:172.21.15.113:6837/1310042629] conn(0x557b3749ea40 0x557b3749ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:18.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 4 client.4671 mds_command: new command op to 4256 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T07:34:18.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 -- 172.21.15.113:0/1601071049 --> [v2:172.21.15.113:6835/1310042629,v1:172.21.15.113:6837/1310042629] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557b3724c9b0 con 0x557b3749ea40 2024-03-04T07:34:18.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1697e00640 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.113:6835/1310042629,v1:172.21.15.113:6837/1310042629] conn(0x557b3749ea40 0x557b3749ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:18.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1697e00640 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.113:6835/1310042629,v1:172.21.15.113:6837/1310042629] conn(0x557b3749ea40 0x557b3749ee20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=338b15f892733b80 server_cookie=a922acfea28ef7bf in_seq=0 out_seq=0 2024-03-04T07:34:18.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1683400640 10 client.4671 ms_handle_connect on v2:172.21.15.113:6835/1310042629 2024-03-04T07:34:18.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1683400640 1 -- 172.21.15.113:0/1601071049 <== mds.1 v2:172.21.15.113:6835/1310042629 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557b3724c9b0 con 0x557b3749ea40 2024-03-04T07:34:18.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1683400640 10 client.4671 handle_command_reply: tid=0 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 client.4671 shutdown 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 -- 172.21.15.113:0/1601071049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b37438fa0 msgr2=0x557b374519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b37438fa0 0x557b374519d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1690024040 tx=0x7f1690000ee0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 -- 172.21.15.113:0/1601071049 shutdown_connections 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.113:6835/1310042629,v1:172.21.15.113:6837/1310042629] conn(0x557b3749ea40 0x557b3749ee20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557b3743a1e0 0x557b374a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x557b374398c0 0x557b3749fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f1683400640 0 client.4671 ms_handle_reset on v2:172.21.15.113:6835/1310042629 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 --2- 172.21.15.113:0/1601071049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557b37438fa0 0x557b374519d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 -- 172.21.15.113:0/1601071049 >> 172.21.15.113:0/1601071049 conn(0x557b372df990 msgr2=0x557b374333d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 -- 172.21.15.113:0/1601071049 shutdown_connections 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 1 -- 172.21.15.113:0/1601071049 wait complete. 2024-03-04T07:34:18.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a7eab000 20 client.4671 trim_cache size 0 max 16384 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "43451c9f-4d9f-4f78-a5ec-a82e7d344007", 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 1, 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 4256, 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 16.331929379000002, 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 13, 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 21, 2024-03-04T07:34:18.926 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 21, 2024-03-04T07:34:18.927 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 29.882792867999999 2024-03-04T07:34:18.927 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-03-04T07:34:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 -- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f167c066f60 msgr2=0x7f167c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f167c066f60 0x7f167c069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f16900011f0 tx=0x7f1690007d40 comp rx=0 tx=0).stop 2024-03-04T07:34:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 -- 172.21.15.113:0/2113201168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a01527f0 msgr2=0x7f16a00cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:18.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a01527f0 0x7f16a00cb430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f168800e930 tx=0x7f168800edf0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 -- 172.21.15.113:0/2113201168 shutdown_connections 2024-03-04T07:34:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16a0152bd0 0x7f16a00af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f167c066f60 0x7f167c069420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16a01527f0 0x7f16a00cb430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 --2- 172.21.15.113:0/2113201168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16a00c8200 0x7f16a01f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 -- 172.21.15.113:0/2113201168 >> 172.21.15.113:0/2113201168 conn(0x7f16a006e820 msgr2=0x7f16a0156440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 -- 172.21.15.113:0/2113201168 shutdown_connections 2024-03-04T07:34:18.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:18.917+0000 7f16a4a00640 1 -- 172.21.15.113:0/2113201168 wait complete. 2024-03-04T07:34:18.947 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-04T07:34:19.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.081+0000 7f9acd400640 1 Processor -- start 2024-03-04T07:34:19.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.081+0000 7f9acd400640 1 -- start start 2024-03-04T07:34:19.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9acd400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9ac8158390 0x7f9ac8070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9acd400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9ac806de10 0x7f9ac80ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9acd400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac80c8da0 con 0x7f9ac80ce7b0 2024-03-04T07:34:19.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9acd400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac8158770 con 0x7f9ac806de10 2024-03-04T07:34:19.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9acd400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac806f770 con 0x7f9ac8158390 2024-03-04T07:34:19.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9abfe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9ac806de10 0x7f9ac80ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9abfe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9ac806de10 0x7f9ac80ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56604/0 (socket says 172.21.15.113:56604) 2024-03-04T07:34:19.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9abfe00640 1 -- 172.21.15.113:0/1385573159 learned_addr learned my addr 172.21.15.113:0/1385573159 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:19.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9ac6800640 1 --2- 172.21.15.113:0/1385573159 >> v2:172.21.15.136:3300/0 conn(0x7f9ac8158390 0x7f9ac8070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9abfe00640 1 -- 172.21.15.113:0/1385573159 >> v2:172.21.15.136:3300/0 conn(0x7f9ac8158390 msgr2=0x7f9ac8070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:19.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9abfe00640 1 --2- 172.21.15.113:0/1385573159 >> v2:172.21.15.136:3300/0 conn(0x7f9ac8158390 0x7f9ac8070130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9abfe00640 1 -- 172.21.15.113:0/1385573159 >> v1:172.21.15.136:6789/0 conn(0x7f9ac80ce7b0 legacy=0x7f9ac815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.085+0000 7f9abfe00640 1 -- 172.21.15.113:0/1385573159 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac806fa50 con 0x7f9ac806de10 2024-03-04T07:34:19.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9abfe00640 1 --2- 172.21.15.113:0/1385573159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 0x7f9ac80ce270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80cf090 tx=0x7f9ab001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a113e3d7c0867e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:19.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9ac5e00640 1 -- 172.21.15.113:0/1385573159 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00b0070 con 0x7f9ac806de10 2024-03-04T07:34:19.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9ac5e00640 1 -- 172.21.15.113:0/1385573159 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab00ab600 con 0x7f9ac806de10 2024-03-04T07:34:19.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9ac5e00640 1 -- 172.21.15.113:0/1385573159 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00ab9d0 con 0x7f9ac806de10 2024-03-04T07:34:19.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9acd400640 1 -- 172.21.15.113:0/1385573159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 msgr2=0x7f9ac80ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9acd400640 1 --2- 172.21.15.113:0/1385573159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 0x7f9ac80ce270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ac80cf090 tx=0x7f9ab001c490 comp rx=0 tx=0).stop 2024-03-04T07:34:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9acd400640 1 -- 172.21.15.113:0/1385573159 shutdown_connections 2024-03-04T07:34:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9acd400640 1 --2- 172.21.15.113:0/1385573159 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 0x7f9ac80ce270 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9acd400640 1 --2- 172.21.15.113:0/1385573159 >> v2:172.21.15.136:3300/0 conn(0x7f9ac8158390 0x7f9ac8070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9acd400640 1 -- 172.21.15.113:0/1385573159 >> 172.21.15.113:0/1385573159 conn(0x7f9ac8066db0 msgr2=0x7f9ac80671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9acd400640 1 -- 172.21.15.113:0/1385573159 shutdown_connections 2024-03-04T07:34:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9acd400640 1 -- 172.21.15.113:0/1385573159 wait complete. 2024-03-04T07:34:19.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9acd400640 1 Processor -- start 2024-03-04T07:34:19.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.089+0000 7f9acd400640 1 -- start start 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9acd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 0x7f9ac814f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9acd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac80ce7b0 0x7f9ac80c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9acd400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ac8158390 0x7f9ac81f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9acd400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8167dc0 con 0x7f9ac806de10 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9acd400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b1100 con 0x7f9ac80ce7b0 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9acd400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8167f40 con 0x7f9ac8158390 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9ac6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 0x7f9ac814f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9ac6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 0x7f9ac814f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56606/0 (socket says 172.21.15.113:56606) 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9ac6800640 1 -- 172.21.15.113:0/3120204401 learned_addr learned my addr 172.21.15.113:0/3120204401 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9abfe00640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac80ce7b0 0x7f9ac80c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9ac7200640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ac8158390 0x7f9ac81f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9ac6800640 1 -- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ac8158390 msgr2=0x7f9ac81f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9ac6800640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ac8158390 0x7f9ac81f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9ac6800640 1 -- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac80ce7b0 msgr2=0x7f9ac80c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9ac6800640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac80ce7b0 0x7f9ac80c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9ac6800640 1 -- 172.21.15.113:0/3120204401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80affb0 con 0x7f9ac806de10 2024-03-04T07:34:19.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9ac6800640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 0x7f9ac814f110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800e930 tx=0x7f9ab800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:19.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9abea00640 1 -- 172.21.15.113:0/3120204401 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8017070 con 0x7f9ac806de10 2024-03-04T07:34:19.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9abea00640 1 -- 172.21.15.113:0/3120204401 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab800c9a0 con 0x7f9ac806de10 2024-03-04T07:34:19.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9abea00640 1 -- 172.21.15.113:0/3120204401 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800cd50 con 0x7f9ac806de10 2024-03-04T07:34:19.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9acd400640 1 -- 172.21.15.113:0/3120204401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac80b0290 con 0x7f9ac806de10 2024-03-04T07:34:19.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.093+0000 7f9acd400640 1 -- 172.21.15.113:0/3120204401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac80c37b0 con 0x7f9ac806de10 2024-03-04T07:34:19.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.097+0000 7f9abea00640 1 -- 172.21.15.113:0/3120204401 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ab801a070 con 0x7f9ac806de10 2024-03-04T07:34:19.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.097+0000 7f9abea00640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9aa4066f60 0x7f9aa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.097+0000 7f9abea00640 1 -- 172.21.15.113:0/3120204401 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ab8084f70 con 0x7f9ac806de10 2024-03-04T07:34:19.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.097+0000 7f9abfe00640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9aa4066f60 0x7f9aa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.097+0000 7f9abfe00640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9aa4066f60 0x7f9aa4069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00ad000 tx=0x7f9ab0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:19.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.097+0000 7f9ad087c000 1 Processor -- start 2024-03-04T07:34:19.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 -- start start 2024-03-04T07:34:19.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca5686fa0 0x55eca5687380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56878c0 0x55eca5687ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56881e0 0x55eca5692230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca5493120 con 0x55eca56881e0 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca541adc0 con 0x55eca56878c0 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eca548f1c0 con 0x55eca5686fa0 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ac7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56881e0 0x55eca5692230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ac7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56881e0 0x55eca5692230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56622/0 (socket says 172.21.15.113:56622) 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ac7200640 1 -- 172.21.15.113:0/2501765391 learned_addr learned my addr 172.21.15.113:0/2501765391 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ac6800640 1 --2- 172.21.15.113:0/2501765391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca5686fa0 0x55eca5687380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9abfe00640 1 --2- 172.21.15.113:0/2501765391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56878c0 0x55eca5687ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ac7200640 1 -- 172.21.15.113:0/2501765391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca5686fa0 msgr2=0x55eca5687380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ac7200640 1 --2- 172.21.15.113:0/2501765391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca5686fa0 0x55eca5687380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ac7200640 1 -- 172.21.15.113:0/2501765391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56878c0 msgr2=0x55eca5687ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ac7200640 1 --2- 172.21.15.113:0/2501765391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56878c0 0x55eca5687ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ac7200640 1 -- 172.21.15.113:0/2501765391 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eca541ecc0 con 0x55eca56881e0 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9abfe00640 1 --2- 172.21.15.113:0/2501765391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56878c0 0x55eca5687ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ac7200640 1 --2- 172.21.15.113:0/2501765391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56881e0 0x55eca5692230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0001c60 tx=0x7f9ac0002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1439ab7d5b6d212d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:19.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ab7400640 1 -- 172.21.15.113:0/2501765391 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0019070 con 0x55eca56881e0 2024-03-04T07:34:19.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ab7400640 1 -- 172.21.15.113:0/2501765391 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0003b90 con 0x55eca56881e0 2024-03-04T07:34:19.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ab7400640 1 -- 172.21.15.113:0/2501765391 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000e860 con 0x55eca56881e0 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 -- 172.21.15.113:0/2501765391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56881e0 msgr2=0x55eca5692230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 --2- 172.21.15.113:0/2501765391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56881e0 0x55eca5692230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0001c60 tx=0x7f9ac0002120 comp rx=0 tx=0).stop 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 -- 172.21.15.113:0/2501765391 shutdown_connections 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 --2- 172.21.15.113:0/2501765391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56881e0 0x55eca5692230 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 --2- 172.21.15.113:0/2501765391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56878c0 0x55eca5687ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 --2- 172.21.15.113:0/2501765391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca5686fa0 0x55eca5687380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 -- 172.21.15.113:0/2501765391 >> 172.21.15.113:0/2501765391 conn(0x55eca552d990 msgr2=0x55eca5684d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 -- 172.21.15.113:0/2501765391 shutdown_connections 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.101+0000 7f9ad087c000 1 -- 172.21.15.113:0/2501765391 wait complete. 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ad087c000 1 -- start start 2024-03-04T07:34:19.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ad087c000 1 Processor -- start 2024-03-04T07:34:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca5686fa0 0x55eca56463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56878c0 0x55eca5646930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56881e0 0x55eca5646e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ad087c000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca544d9c0 con 0x55eca5686fa0 2024-03-04T07:34:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ad087c000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca5550f90 con 0x55eca56881e0 2024-03-04T07:34:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ad087c000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eca548d5e0 con 0x55eca56878c0 2024-03-04T07:34:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ad087c000 10 client.0 _ll_register_callbacks cb 0x55eca56314f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T07:34:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ac6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca5686fa0 0x55eca56463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ac6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca5686fa0 0x55eca56463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56636/0 (socket says 172.21.15.113:56636) 2024-03-04T07:34:19.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ac6800640 1 -- 172.21.15.113:0/3626612382 learned_addr learned my addr 172.21.15.113:0/3626612382 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ac7200640 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56881e0 0x55eca5646e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9abfe00640 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56878c0 0x55eca5646930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ac6800640 1 -- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56878c0 msgr2=0x55eca5646930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ac6800640 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56878c0 0x55eca5646930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ac6800640 1 -- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56881e0 msgr2=0x55eca5646e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ac6800640 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56881e0 0x55eca5646e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ac6800640 1 -- 172.21.15.113:0/3626612382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eca544f550 con 0x55eca5686fa0 2024-03-04T07:34:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ac6800640 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca5686fa0 0x55eca56463f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8055970 tx=0x7f9ab8059b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1b526e25172e9c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:19.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9aabe00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-03-04T07:34:19.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9aabe00640 1 -- 172.21.15.113:0/3626612382 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8080070 con 0x55eca5686fa0 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9ad087c000 1 -- 172.21.15.113:0/3626612382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ac0003b90 con 0x55eca5686fa0 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.105+0000 7f9aabe00640 1 -- 172.21.15.113:0/3626612382 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ab8055030 con 0x55eca5686fa0 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9aabe00640 1 -- 172.21.15.113:0/3626612382 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ab8021040 con 0x55eca5686fa0 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 10 client.4683 fetch_fsmap learned FSMap version 13 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 -- 172.21.15.113:0/3626612382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55eca541b010 con 0x55eca5686fa0 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9aabe00640 1 -- 172.21.15.113:0/3626612382 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0003b90 con 0x55eca5686fa0 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9aabe00640 1 -- 172.21.15.113:0/3626612382 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 13) v1 ==== 4504+0+0 (secure 0 0 0) 0x7f9ab8024060 con 0x55eca5686fa0 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 10 client.4683 fetch_fsmap finished waiting for FSMap version 13 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 10 client.4683 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 20 client.4683 populate_metadata read hostname 'smithi113' 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:6834/3348592059,v1:172.21.15.136:6835/3348592059] conn(0x55eca569b390 0x55eca569d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 4 client.4683 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 -- 172.21.15.113:0/3626612382 --> [v2:172.21.15.136:6834/3348592059,v1:172.21.15.136:6835/3348592059] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eca542ea50 con 0x55eca569b390 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9abfe00640 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:6834/3348592059,v1:172.21.15.136:6835/3348592059] conn(0x55eca569b390 0x55eca569d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9abfe00640 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:6834/3348592059,v1:172.21.15.136:6835/3348592059] conn(0x55eca569b390 0x55eca569d780 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752f6e7c2541244a server_cookie=4f55ce22574a9573 in_seq=0 out_seq=0 2024-03-04T07:34:19.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9aabe00640 10 client.4683 ms_handle_connect on v2:172.21.15.136:6834/3348592059 2024-03-04T07:34:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9aabe00640 1 -- 172.21.15.113:0/3626612382 <== mds.0 v2:172.21.15.136:6834/3348592059 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55eca542ea50 con 0x55eca569b390 2024-03-04T07:34:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9aabe00640 10 client.4683 handle_command_reply: tid=0 2024-03-04T07:34:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 client.4683 shutdown 2024-03-04T07:34:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 -- 172.21.15.113:0/3626612382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca5686fa0 msgr2=0x55eca56463f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca5686fa0 0x55eca56463f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8055970 tx=0x7f9ab8059b40 comp rx=0 tx=0).stop 2024-03-04T07:34:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 -- 172.21.15.113:0/3626612382 shutdown_connections 2024-03-04T07:34:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:6834/3348592059,v1:172.21.15.136:6835/3348592059] conn(0x55eca569b390 0x55eca569d780 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca56881e0 0x55eca5646e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56878c0 0x55eca5646930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 --2- 172.21.15.113:0/3626612382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca5686fa0 0x55eca56463f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 -- 172.21.15.113:0/3626612382 >> 172.21.15.113:0/3626612382 conn(0x55eca552d990 msgr2=0x55eca5684730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:19.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9aabe00640 0 client.4683 ms_handle_reset on v2:172.21.15.136:6834/3348592059 2024-03-04T07:34:19.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 -- 172.21.15.113:0/3626612382 shutdown_connections 2024-03-04T07:34:19.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.109+0000 7f9ad087c000 1 -- 172.21.15.113:0/3626612382 wait complete. 2024-03-04T07:34:19.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.113+0000 7f9ad087c000 20 client.4683 trim_cache size 0 max 16384 2024-03-04T07:34:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 Processor -- start 2024-03-04T07:34:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 -- start start 2024-03-04T07:34:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 0x55eca56a0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca569e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 0x55eca569e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80ca7d0 con 0x55eca56878c0 2024-03-04T07:34:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac814ad10 con 0x55eca5686fa0 2024-03-04T07:34:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eca543b7c0 con 0x55eca56881e0 2024-03-04T07:34:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9abfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca569e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9abfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca569e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56646/0 (socket says 172.21.15.113:56646) 2024-03-04T07:34:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9abfe00640 1 -- 172.21.15.113:0/2240957371 learned_addr learned my addr 172.21.15.113:0/2240957371 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ac6800640 1 --2- 172.21.15.113:0/2240957371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 0x55eca56a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ac7200640 1 --2- 172.21.15.113:0/2240957371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 0x55eca569e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9abfe00640 1 -- 172.21.15.113:0/2240957371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 msgr2=0x55eca569e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9abfe00640 1 --2- 172.21.15.113:0/2240957371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 0x55eca569e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9abfe00640 1 -- 172.21.15.113:0/2240957371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 msgr2=0x55eca56a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9abfe00640 1 --2- 172.21.15.113:0/2240957371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 0x55eca56a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9abfe00640 1 -- 172.21.15.113:0/2240957371 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eca56876f0 con 0x55eca56878c0 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9abfe00640 1 --2- 172.21.15.113:0/2240957371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca569e3b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00ae040 tx=0x7f9ab0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c5b27d497fa111 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ab6a00640 1 -- 172.21.15.113:0/2240957371 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00d3070 con 0x55eca56878c0 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ab6a00640 1 -- 172.21.15.113:0/2240957371 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab00b1040 con 0x55eca56878c0 2024-03-04T07:34:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ab6a00640 1 -- 172.21.15.113:0/2240957371 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00c56c0 con 0x55eca56878c0 2024-03-04T07:34:19.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 -- 172.21.15.113:0/2240957371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 msgr2=0x55eca569e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 --2- 172.21.15.113:0/2240957371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca569e3b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00ae040 tx=0x7f9ab0004740 comp rx=0 tx=0).stop 2024-03-04T07:34:19.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 -- 172.21.15.113:0/2240957371 shutdown_connections 2024-03-04T07:34:19.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 --2- 172.21.15.113:0/2240957371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 0x55eca569e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 --2- 172.21.15.113:0/2240957371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca569e3b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 --2- 172.21.15.113:0/2240957371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 0x55eca56a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 -- 172.21.15.113:0/2240957371 >> 172.21.15.113:0/2240957371 conn(0x55eca552d990 msgr2=0x55eca5684b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 -- 172.21.15.113:0/2240957371 shutdown_connections 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 -- 172.21.15.113:0/2240957371 wait complete. 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 -- start start 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.129+0000 7f9ad087c000 1 Processor -- start 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 0x55eca56ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca56ed570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 0x55eca56edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca5492430 con 0x55eca56878c0 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca5438860 con 0x55eca5686fa0 2024-03-04T07:34:19.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eca5497750 con 0x55eca56881e0 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 10 client.0 _ll_register_callbacks cb 0x7f9ab80555d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9abfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca56ed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ac6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 0x55eca56ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ac6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 0x55eca56ed030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:49146/0 (socket says 172.21.15.113:49146) 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ac6800640 1 -- 172.21.15.113:0/1211003383 learned_addr learned my addr 172.21.15.113:0/1211003383 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ac7200640 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 0x55eca56edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9abfe00640 1 -- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 msgr2=0x55eca56edcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9abfe00640 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 0x55eca56edcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9abfe00640 1 -- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 msgr2=0x55eca56ed030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9abfe00640 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 0x55eca56ed030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9abfe00640 1 -- 172.21.15.113:0/1211003383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eca5460e30 con 0x55eca56878c0 2024-03-04T07:34:19.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ac7200640 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 0x55eca56edcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ac6800640 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 0x55eca56ed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9abfe00640 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca56ed570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00190a0 tx=0x7f9ab00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c988240cee7da7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9aabe00640 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9aabe00640 1 -- 172.21.15.113:0/1211003383 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00d3070 con 0x55eca56878c0 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9aabe00640 1 -- 172.21.15.113:0/1211003383 <== mon.0 v2:172.21.15.113:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ab00d8ab0 con 0x55eca56878c0 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 1 -- 172.21.15.113:0/1211003383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ab00b1040 con 0x55eca56878c0 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9aabe00640 1 -- 172.21.15.113:0/1211003383 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ab00d8cb0 con 0x55eca56878c0 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 10 client.4689 fetch_fsmap learned FSMap version 13 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 1 -- 172.21.15.113:0/1211003383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55eca54ec0c0 con 0x55eca56878c0 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9aabe00640 1 -- 172.21.15.113:0/1211003383 <== mon.0 v2:172.21.15.113:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab00b1040 con 0x55eca56878c0 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9aabe00640 1 -- 172.21.15.113:0/1211003383 <== mon.0 v2:172.21.15.113:3300/0 5 ==== fsmap(e 13) v1 ==== 4504+0+0 (secure 0 0 0) 0x7f9ab00cd070 con 0x55eca56878c0 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 10 client.4689 fetch_fsmap finished waiting for FSMap version 13 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 10 client.4689 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 20 client.4689 populate_metadata read hostname 'smithi113' 2024-03-04T07:34:19.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:6834/3348592059,v1:172.21.15.136:6835/3348592059] conn(0x55eca56ee630 0x55eca56eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 4 client.4689 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-04T07:34:19.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ad087c000 1 -- 172.21.15.113:0/1211003383 --> [v2:172.21.15.136:6834/3348592059,v1:172.21.15.136:6835/3348592059] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eca54a0660 con 0x55eca56ee630 2024-03-04T07:34:19.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ac6800640 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:6834/3348592059,v1:172.21.15.136:6835/3348592059] conn(0x55eca56ee630 0x55eca56eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9ac6800640 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:6834/3348592059,v1:172.21.15.136:6835/3348592059] conn(0x55eca56ee630 0x55eca56eea10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6ebc8b227faab5 server_cookie=2e45f559acc66b22 in_seq=0 out_seq=0 2024-03-04T07:34:19.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.133+0000 7f9aabe00640 10 client.4689 ms_handle_connect on v2:172.21.15.136:6834/3348592059 2024-03-04T07:34:19.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9aabe00640 1 -- 172.21.15.113:0/1211003383 <== mds.0 v2:172.21.15.136:6834/3348592059 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55eca54a0660 con 0x55eca56ee630 2024-03-04T07:34:19.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9aabe00640 10 client.4689 handle_command_reply: tid=0 2024-03-04T07:34:19.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 client.4689 shutdown 2024-03-04T07:34:19.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 -- 172.21.15.113:0/1211003383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 msgr2=0x55eca56ed570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca56ed570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00190a0 tx=0x7f9ab00046d0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 -- 172.21.15.113:0/1211003383 shutdown_connections 2024-03-04T07:34:19.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:6834/3348592059,v1:172.21.15.136:6835/3348592059] conn(0x55eca56ee630 0x55eca56eea10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x55eca56881e0 0x55eca56edcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55eca56878c0 0x55eca56ed570 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 --2- 172.21.15.113:0/1211003383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55eca5686fa0 0x55eca56ed030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 -- 172.21.15.113:0/1211003383 >> 172.21.15.113:0/1211003383 conn(0x55eca552d990 msgr2=0x55eca569d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9aabe00640 0 client.4689 ms_handle_reset on v2:172.21.15.136:6834/3348592059 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 -- 172.21.15.113:0/1211003383 shutdown_connections 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 1 -- 172.21.15.113:0/1211003383 wait complete. 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9ad087c000 20 client.4689 trim_cache size 0 max 16384 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: "cluster_fsid": "43451c9f-4d9f-4f78-a5ec-a82e7d344007", 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: "whoami": 0, 2024-03-04T07:34:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 4276, 2024-03-04T07:34:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "want_state": "up:active", 2024-03-04T07:34:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "state": "up:active", 2024-03-04T07:34:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs", 2024-03-04T07:34:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "rank_uptime": 22.621109728, 2024-03-04T07:34:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "mdsmap_epoch": 13, 2024-03-04T07:34:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch": 21, 2024-03-04T07:34:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "osdmap_epoch_barrier": 21, 2024-03-04T07:34:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "uptime": 30.141629322 2024-03-04T07:34:19.146 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 -- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9aa4066f60 msgr2=0x7f9aa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9aa4066f60 0x7f9aa4069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00ad000 tx=0x7f9ab0008040 comp rx=0 tx=0).stop 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 -- 172.21.15.113:0/3120204401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 msgr2=0x7f9ac814f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 0x7f9ac814f110 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800e930 tx=0x7f9ab800edf0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 -- 172.21.15.113:0/3120204401 shutdown_connections 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ac8158390 0x7f9ac81f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ac80ce7b0 0x7f9ac80c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9aa4066f60 0x7f9aa4069420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 --2- 172.21.15.113:0/3120204401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ac806de10 0x7f9ac814f110 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 -- 172.21.15.113:0/3120204401 >> 172.21.15.113:0/3120204401 conn(0x7f9ac8066db0 msgr2=0x7f9ac8067190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 -- 172.21.15.113:0/3120204401 shutdown_connections 2024-03-04T07:34:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.137+0000 7f9acd400640 1 -- 172.21.15.113:0/3120204401 wait complete. 2024-03-04T07:34:19.161 INFO:teuthology.run_tasks:Running task kclient... 2024-03-04T07:34:19.184 INFO:tasks.kclient:Mounting kernel clients... 2024-03-04T07:34:19.185 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-04T07:34:19.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:19.185 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-03-04T07:34:19.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:19.196 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-03-04T07:34:19.266 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-03-04T07:34:19.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T07:34:19.268 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-04T07:34:19.268 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-04T07:34:19.268 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T07:34:19.269 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-04T07:34:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b2bc00640 1 Processor -- start 2024-03-04T07:34:19.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b2bc00640 1 -- start start 2024-03-04T07:34:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b2bc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6b24155100 0x7f6b240c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b2bc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b2415ad30 0x7f6b2415b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b2bc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2415b6c0 con 0x7f6b2406b060 2024-03-04T07:34:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b2bc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b240c8890 con 0x7f6b24155100 2024-03-04T07:34:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b2bc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2406b4b0 con 0x7f6b2415ad30 2024-03-04T07:34:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b23e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6b2406b060 0x7f6b240cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39230/0 (socket says 172.21.15.113:39230) 2024-03-04T07:34:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b23e00640 1 -- 172.21.15.113:0/2723754710 learned_addr learned my addr 172.21.15.113:0/2723754710 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b29200640 1 --2- 172.21.15.113:0/2723754710 >> v2:172.21.15.136:3300/0 conn(0x7f6b24155100 0x7f6b240c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b29c00640 1 --2- 172.21.15.113:0/2723754710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b2415ad30 0x7f6b2415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b29200640 1 -- 172.21.15.113:0/2723754710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b2415ad30 msgr2=0x7f6b2415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b29200640 1 --2- 172.21.15.113:0/2723754710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b2415ad30 0x7f6b2415b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b23400640 1 -- 172.21.15.113:0/2723754710 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152193327 0 0) 0x7f6b2415b6c0 con 0x7f6b2406b060 2024-03-04T07:34:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b29200640 1 -- 172.21.15.113:0/2723754710 >> v1:172.21.15.113:6789/0 conn(0x7f6b2406b060 legacy=0x7f6b240cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.449+0000 7f6b29200640 1 -- 172.21.15.113:0/2723754710 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2415efd0 con 0x7f6b24155100 2024-03-04T07:34:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b29200640 1 --2- 172.21.15.113:0/2723754710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b24155100 0x7f6b240c08d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6b180097f0 tx=0x7f6b1801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9792de217821c134 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b23400640 1 -- 172.21.15.113:0/2723754710 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b180b2070 con 0x7f6b24155100 2024-03-04T07:34:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b23400640 1 -- 172.21.15.113:0/2723754710 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b180ad640 con 0x7f6b24155100 2024-03-04T07:34:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b23400640 1 -- 172.21.15.113:0/2723754710 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b180ada10 con 0x7f6b24155100 2024-03-04T07:34:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b2bc00640 1 -- 172.21.15.113:0/2723754710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b24155100 msgr2=0x7f6b240c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b2bc00640 1 --2- 172.21.15.113:0/2723754710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b24155100 0x7f6b240c08d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6b180097f0 tx=0x7f6b1801c4a0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b2bc00640 1 -- 172.21.15.113:0/2723754710 shutdown_connections 2024-03-04T07:34:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b2bc00640 1 --2- 172.21.15.113:0/2723754710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b2415ad30 0x7f6b2415b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b2bc00640 1 --2- 172.21.15.113:0/2723754710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b24155100 0x7f6b240c08d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b2bc00640 1 -- 172.21.15.113:0/2723754710 >> 172.21.15.113:0/2723754710 conn(0x7f6b2406deb0 msgr2=0x7f6b2406e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b2bc00640 1 -- 172.21.15.113:0/2723754710 shutdown_connections 2024-03-04T07:34:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.453+0000 7f6b2bc00640 1 -- 172.21.15.113:0/2723754710 wait complete. 2024-03-04T07:34:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b2bc00640 1 Processor -- start 2024-03-04T07:34:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b2bc00640 1 -- start start 2024-03-04T07:34:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b2bc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2406b060 0x7f6b240c1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b2bc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b24155100 0x7f6b2415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b2bc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b2415ad30 0x7f6b240c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b2bc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2415ebb0 con 0x7f6b2406b060 2024-03-04T07:34:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b2bc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240c9610 con 0x7f6b24155100 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b2bc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2415ed30 con 0x7f6b2415ad30 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b29200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2406b060 0x7f6b240c1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b29200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2406b060 0x7f6b240c1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56672/0 (socket says 172.21.15.113:56672) 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b29200640 1 -- 172.21.15.113:0/4119154424 learned_addr learned my addr 172.21.15.113:0/4119154424 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b23e00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b24155100 0x7f6b2415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b29c00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b2415ad30 0x7f6b240c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b29200640 1 -- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b2415ad30 msgr2=0x7f6b240c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b29200640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b2415ad30 0x7f6b240c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b29200640 1 -- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b24155100 msgr2=0x7f6b2415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b29200640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b24155100 0x7f6b2415d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b29200640 1 -- 172.21.15.113:0/4119154424 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2415c2f0 con 0x7f6b2406b060 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b29200640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2406b060 0x7f6b240c1b90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6b180097f0 tx=0x7f6b1801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b20c00640 1 -- 172.21.15.113:0/4119154424 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b180be070 con 0x7f6b2406b060 2024-03-04T07:34:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b20c00640 1 -- 172.21.15.113:0/4119154424 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b180b3040 con 0x7f6b2406b060 2024-03-04T07:34:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b2bc00640 1 -- 172.21.15.113:0/4119154424 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2415c5d0 con 0x7f6b2406b060 2024-03-04T07:34:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b20c00640 1 -- 172.21.15.113:0/4119154424 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b180b2050 con 0x7f6b2406b060 2024-03-04T07:34:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.457+0000 7f6b2bc00640 1 -- 172.21.15.113:0/4119154424 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2415cb90 con 0x7f6b2406b060 2024-03-04T07:34:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.461+0000 7f6b2bc00640 1 -- 172.21.15.113:0/4119154424 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b24075c00 con 0x7f6b2406b060 2024-03-04T07:34:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.461+0000 7f6b20c00640 1 -- 172.21.15.113:0/4119154424 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b180c1070 con 0x7f6b2406b060 2024-03-04T07:34:19.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.461+0000 7f6b20c00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b04066f90 0x7f6b04069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:19.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.461+0000 7f6b20c00640 1 -- 172.21.15.113:0/4119154424 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b1811ec80 con 0x7f6b2406b060 2024-03-04T07:34:19.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.461+0000 7f6b23e00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b04066f90 0x7f6b04069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:19.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.461+0000 7f6b23e00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b04066f90 0x7f6b04069450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6b2414e410 tx=0x7f6b10007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:19.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.465+0000 7f6b20c00640 1 -- 172.21.15.113:0/4119154424 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6b18016090 con 0x7f6b2406b060 2024-03-04T07:34:19.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.661+0000 7f6b2bc00640 1 -- 172.21.15.113:0/4119154424 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6b2415b310 con 0x7f6b2406b060 2024-03-04T07:34:19.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.661+0000 7f6b20c00640 1 -- 172.21.15.113:0/4119154424 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6b180ef6d0 con 0x7f6b2406b060 2024-03-04T07:34:19.668 INFO:teuthology.orchestra.run.smithi113.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-04T07:34:19.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 -- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b04066f90 msgr2=0x7f6b04069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b04066f90 0x7f6b04069450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6b2414e410 tx=0x7f6b10007d60 comp rx=0 tx=0).stop 2024-03-04T07:34:19.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 -- 172.21.15.113:0/4119154424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2406b060 msgr2=0x7f6b240c1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:19.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2406b060 0x7f6b240c1b90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6b180097f0 tx=0x7f6b1801ace0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 -- 172.21.15.113:0/4119154424 shutdown_connections 2024-03-04T07:34:19.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b2415ad30 0x7f6b240c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b24155100 0x7f6b2415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b04066f90 0x7f6b04069450 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 --2- 172.21.15.113:0/4119154424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b2406b060 0x7f6b240c1b90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:19.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 -- 172.21.15.113:0/4119154424 >> 172.21.15.113:0/4119154424 conn(0x7f6b2406deb0 msgr2=0x7f6b2406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 -- 172.21.15.113:0/4119154424 shutdown_connections 2024-03-04T07:34:19.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:19.665+0000 7f6b2bc00640 1 -- 172.21.15.113:0/4119154424 wait complete. 2024-03-04T07:34:19.689 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-04T07:34:19.689 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-04T07:34:19.689 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-03-04T07:34:19.689 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-04T07:34:19.689 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-04T07:34:19.689 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-04T07:34:19.689 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-04T07:34:19.689 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-04T07:34:19.689 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-04T07:34:19.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:19.690 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-03-04T07:34:19.700 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-04T07:34:19.700 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-04T07:34:19.700 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-03-04T07:34:19.700 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-04T07:34:19.701 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42443sec preferred_lft 42443sec 2024-03-04T07:34:19.702 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link 2024-03-04T07:34:19.702 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-04T07:34:19.702 INFO:teuthology.orchestra.run.smithi113.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-04T07:34:19.702 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 02:42:0a:36:72:10 brd ff:ff:ff:ff:ff:ff 2024-03-04T07:34:19.702 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-04T07:34:19.702 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-04T07:34:19.702 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-04T07:34:19.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:19.703 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-03-04T07:34:19.703 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-04T07:34:19.703 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add name ceph-brx type bridge 2024-03-04T07:34:19.703 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr flush dev ceph-brx 2024-03-04T07:34:19.703 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set ceph-brx up 2024-03-04T07:34:19.703 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-04T07:34:19.703 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-04T07:34:19.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:19.825 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-04T07:34:19.843 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-03-04T07:34:19.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:19.846 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-03-04T07:34:19.920 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-03-04T07:34:19.920 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-04T07:34:19.920 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-04T07:34:19.920 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-04T07:34:19.921 INFO:teuthology.orchestra.run.smithi113.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-04T07:34:19.921 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-04T07:34:19.921 INFO:teuthology.orchestra.run.smithi113.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-04T07:34:19.921 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-04T07:34:19.921 INFO:teuthology.orchestra.run.smithi113.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-04T07:34:19.921 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-04T07:34:19.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:19.921 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-03-04T07:34:19.922 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-04T07:34:19.922 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-04T07:34:19.922 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-04T07:34:19.922 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-04T07:34:19.922 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-04T07:34:20.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:20.012 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-03-04T07:34:20.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:20.068 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2024-03-04T07:34:20.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:20.119 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-03-04T07:34:20.119 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-04T07:34:20.119 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T07:34:20.120 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-04T07:34:20.120 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-04T07:34:20.203 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-04T07:34:20.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:20.203 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-03-04T07:34:20.203 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-04T07:34:20.203 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-04T07:34:20.203 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-04T07:34:20.203 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-04T07:34:20.203 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-04T07:34:20.203 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-04T07:34:20.204 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-04T07:34:20.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:20.378 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec bash -c ' 2024-03-04T07:34:20.378 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-04T07:34:20.378 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-03-04T07:34:20.378 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.0 master ceph-brx 2024-03-04T07:34:20.378 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-04T07:34:20.418 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T07:34:20.418 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-04T07:34:20.472 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-04T07:34:20.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-04T07:34:20.472 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-04T07:34:20.523 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-04T07:34:20.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:20.523 DEBUG:teuthology.orchestra.run.smithi113:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T07:34:20.626 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T07:34:20.626 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T07:34:21.136 INFO:teuthology.orchestra.run.smithi113.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-04T07:34:21.137 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-04T07:34:21.137 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: resolved to: "172.21.15.113:3300,172.21.15.136:3300,172.21.15.136:3301" 2024-03-04T07:34:21.137 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: trying mount with new device syntax: 0@43451c9f-4d9f-4f78-a5ec-a82e7d344007.cephfs=/ 2024-03-04T07:34:21.137 INFO:teuthology.orchestra.run.smithi113.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.113:3300/172.21.15.136:3300/172.21.15.136:3301" will pass to kernel 2024-03-04T07:34:21.141 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-04T07:34:21.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:21.141 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-04T07:34:21.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:21.169 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T07:34:21.257 INFO:teuthology.orchestra.run.smithi113.stdout:client4698 2024-03-04T07:34:21.257 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:34:21.257 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T07:34:21.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:21.308 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T07:34:22.849 INFO:teuthology.orchestra.run.smithi113.stdout:client4698 2024-03-04T07:34:22.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:22.849 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-04T07:34:22.861 INFO:teuthology.orchestra.run.smithi113.stdout:client4698 2024-03-04T07:34:22.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T07:34:22.861 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/43451c9f-4d9f-4f78-a5ec-a82e7d344007.client4698/status') 2024-03-04T07:34:22.922 INFO:teuthology.orchestra.run.smithi113.stdout:instance: client.4698 (3)192.168.0.1:0/3162252876 2024-03-04T07:34:22.922 INFO:teuthology.orchestra.run.smithi113.stdout:blocklisted: false 2024-03-04T07:34:22.922 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records in 2024-03-04T07:34:22.922 INFO:teuthology.orchestra.run.smithi113.stderr:0+1 records out 2024-03-04T07:34:22.923 INFO:teuthology.orchestra.run.smithi113.stderr:69 bytes copied, 7.2509e-05 s, 952 kB/s 2024-03-04T07:34:22.927 INFO:teuthology.run_tasks:Running task mds_thrash... 2024-03-04T07:34:22.939 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:34:23.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1934000640 1 Processor -- start 2024-03-04T07:34:23.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1934000640 1 -- start start 2024-03-04T07:34:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1934000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 0x7f192c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1934000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f192c152bd0 0x7f192c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1934000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192c1591f0 con 0x7f192c0c8200 2024-03-04T07:34:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1934000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192c071df0 con 0x7f192c152bd0 2024-03-04T07:34:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1934000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192c1597b0 con 0x7f192c1527f0 2024-03-04T07:34:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1930c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f192c0c8200 0x7f192c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39242/0 (socket says 172.21.15.113:39242) 2024-03-04T07:34:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1930c00640 1 -- 172.21.15.113:0/639592699 learned_addr learned my addr 172.21.15.113:0/639592699 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1931600640 1 --2- 172.21.15.113:0/639592699 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 0x7f192c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1932000640 1 --2- 172.21.15.113:0/639592699 >> v2:172.21.15.136:3300/0 conn(0x7f192c152bd0 0x7f192c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f192be00640 1 -- 172.21.15.113:0/639592699 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268421005 0 0) 0x7f192c1591f0 con 0x7f192c0c8200 2024-03-04T07:34:23.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f192be00640 1 -- 172.21.15.113:0/639592699 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1914003530 con 0x7f192c0c8200 2024-03-04T07:34:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1931600640 1 -- 172.21.15.113:0/639592699 >> v2:172.21.15.136:3300/0 conn(0x7f192c152bd0 msgr2=0x7f192c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1931600640 1 --2- 172.21.15.113:0/639592699 >> v2:172.21.15.136:3300/0 conn(0x7f192c152bd0 0x7f192c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1931600640 1 -- 172.21.15.113:0/639592699 >> v1:172.21.15.113:6789/0 conn(0x7f192c0c8200 legacy=0x7f192c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f192be00640 1 -- 172.21.15.113:0/639592699 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3771267442 0 0) 0x7f1914003530 con 0x7f192c0c8200 2024-03-04T07:34:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1931600640 1 -- 172.21.15.113:0/639592699 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c1594d0 con 0x7f192c1527f0 2024-03-04T07:34:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.101+0000 7f1932000640 1 --2- 172.21.15.113:0/639592699 >> v2:172.21.15.136:3300/0 conn(0x7f192c152bd0 0x7f192c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:23.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f1931600640 1 --2- 172.21.15.113:0/639592699 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 0x7f192c070d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f19200097c0 tx=0x7f192001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be74b12c4467ccea server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:23.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f192be00640 1 -- 172.21.15.113:0/639592699 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19200b0070 con 0x7f192c1527f0 2024-03-04T07:34:23.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f192be00640 1 -- 172.21.15.113:0/639592699 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19200ab400 con 0x7f192c1527f0 2024-03-04T07:34:23.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f192be00640 1 -- 172.21.15.113:0/639592699 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918001680 con 0x7f192c1527f0 2024-03-04T07:34:23.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f1934000640 1 -- 172.21.15.113:0/639592699 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 msgr2=0x7f192c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f1934000640 1 --2- 172.21.15.113:0/639592699 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 0x7f192c070d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f19200097c0 tx=0x7f192001c3c0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f1934000640 1 -- 172.21.15.113:0/639592699 shutdown_connections 2024-03-04T07:34:23.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f1934000640 1 --2- 172.21.15.113:0/639592699 >> v2:172.21.15.136:3300/0 conn(0x7f192c152bd0 0x7f192c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f1934000640 1 --2- 172.21.15.113:0/639592699 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 0x7f192c070d40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f1934000640 1 -- 172.21.15.113:0/639592699 >> 172.21.15.113:0/639592699 conn(0x7f192c06e820 msgr2=0x7f192c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:23.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f1934000640 1 -- 172.21.15.113:0/639592699 shutdown_connections 2024-03-04T07:34:23.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.105+0000 7f1934000640 1 -- 172.21.15.113:0/639592699 wait complete. 2024-03-04T07:34:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1934000640 1 Processor -- start 2024-03-04T07:34:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1934000640 1 -- start start 2024-03-04T07:34:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1934000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192c0c8200 0x7f192c0c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1934000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 0x7f192c153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1934000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f192c152bd0 0x7f192c1f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1934000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c1654f0 con 0x7f192c0c8200 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1934000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c165370 con 0x7f192c152bd0 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1934000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192c165670 con 0x7f192c1527f0 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1931600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192c0c8200 0x7f192c0c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1931600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192c0c8200 0x7f192c0c1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56682/0 (socket says 172.21.15.113:56682) 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1931600640 1 -- 172.21.15.113:0/3963036515 learned_addr learned my addr 172.21.15.113:0/3963036515 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1930c00640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 0x7f192c153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1932000640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f192c152bd0 0x7f192c1f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1931600640 1 -- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 msgr2=0x7f192c153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1931600640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 0x7f192c153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1931600640 1 -- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f192c152bd0 msgr2=0x7f192c1f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1931600640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f192c152bd0 0x7f192c1f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1931600640 1 -- 172.21.15.113:0/3963036515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c0c2050 con 0x7f192c0c8200 2024-03-04T07:34:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1931600640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192c0c8200 0x7f192c0c1b10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1920000c10 tx=0x7f1920019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1929600640 1 -- 172.21.15.113:0/3963036515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19200b0040 con 0x7f192c0c8200 2024-03-04T07:34:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1929600640 1 -- 172.21.15.113:0/3963036515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19200bb070 con 0x7f192c0c8200 2024-03-04T07:34:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1929600640 1 -- 172.21.15.113:0/3963036515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920003aa0 con 0x7f192c0c8200 2024-03-04T07:34:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1934000640 1 -- 172.21.15.113:0/3963036515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192c0c5ac0 con 0x7f192c0c8200 2024-03-04T07:34:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.109+0000 7f1934000640 1 -- 172.21.15.113:0/3963036515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192c0c2b30 con 0x7f192c0c8200 2024-03-04T07:34:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.113+0000 7f1934000640 1 -- 172.21.15.113:0/3963036515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192c075c00 con 0x7f192c0c8200 2024-03-04T07:34:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.113+0000 7f1929600640 1 -- 172.21.15.113:0/3963036515 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19200b3070 con 0x7f192c0c8200 2024-03-04T07:34:23.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.113+0000 7f1929600640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f190c066f60 0x7f190c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.113+0000 7f1929600640 1 -- 172.21.15.113:0/3963036515 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f192011ed40 con 0x7f192c0c8200 2024-03-04T07:34:23.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.113+0000 7f1930c00640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f190c066f60 0x7f190c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.113+0000 7f1930c00640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f190c066f60 0x7f190c069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f192c162b90 tx=0x7f1918009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.117+0000 7f1929600640 1 -- 172.21.15.113:0/3963036515 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1920015fb0 con 0x7f192c0c8200 2024-03-04T07:34:23.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.305+0000 7f1934000640 1 -- 172.21.15.113:0/3963036515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f192c0c79d0 con 0x7f192c0c8200 2024-03-04T07:34:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1929600640 1 -- 172.21.15.113:0/3963036515 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f19200efc20 con 0x7f192c0c8200 2024-03-04T07:34:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:23.312 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:57.517935+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:34:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 -- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f190c066f60 msgr2=0x7f190c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f190c066f60 0x7f190c069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f192c162b90 tx=0x7f1918009040 comp rx=0 tx=0).stop 2024-03-04T07:34:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 -- 172.21.15.113:0/3963036515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192c0c8200 msgr2=0x7f192c0c1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192c0c8200 0x7f192c0c1b10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1920000c10 tx=0x7f1920019160 comp rx=0 tx=0).stop 2024-03-04T07:34:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 -- 172.21.15.113:0/3963036515 shutdown_connections 2024-03-04T07:34:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f192c152bd0 0x7f192c1f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f192c1527f0 0x7f192c153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f190c066f60 0x7f190c069420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 --2- 172.21.15.113:0/3963036515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f192c0c8200 0x7f192c0c1b10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 -- 172.21.15.113:0/3963036515 >> 172.21.15.113:0/3963036515 conn(0x7f192c06e820 msgr2=0x7f192c0ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:23.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 -- 172.21.15.113:0/3963036515 shutdown_connections 2024-03-04T07:34:23.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.309+0000 7f1934000640 1 -- 172.21.15.113:0/3963036515 wait complete. 2024-03-04T07:34:23.333 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T07:33:41.507700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T07:33:53.680465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T07:33:54.892043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T07:34:23.334 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T07:34:23.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.469+0000 7f52c2600640 1 Processor -- start 2024-03-04T07:34:23.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.469+0000 7f52c2600640 1 -- start start 2024-03-04T07:34:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52c2600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 0x7f52bc0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52c2600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52bc0c9ce0 con 0x7f52bc0ce770 2024-03-04T07:34:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52c2600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52bc06e110 con 0x7f52bc158320 2024-03-04T07:34:23.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52c2600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc0c9160 con 0x7f52bc06ff60 2024-03-04T07:34:23.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52bbe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f52bc0ce770 0x7f52bc15ae90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39256/0 (socket says 172.21.15.113:39256) 2024-03-04T07:34:23.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52bbe00640 1 -- 172.21.15.113:0/3648833618 learned_addr learned my addr 172.21.15.113:0/3648833618 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:23.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52b2a00640 1 --2- 172.21.15.113:0/3648833618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 0x7f52bc0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52baa00640 1 -- 172.21.15.113:0/3648833618 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205021953 0 0) 0x7f52bc0c9ce0 con 0x7f52bc0ce770 2024-03-04T07:34:23.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52baa00640 1 -- 172.21.15.113:0/3648833618 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52a0003530 con 0x7f52bc0ce770 2024-03-04T07:34:23.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52baa00640 1 -- 172.21.15.113:0/3648833618 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906733546 0 0) 0x7f52bc06e110 con 0x7f52bc158320 2024-03-04T07:34:23.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52baa00640 1 -- 172.21.15.113:0/3648833618 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52bc0c9ce0 con 0x7f52bc158320 2024-03-04T07:34:23.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52b2a00640 1 -- 172.21.15.113:0/3648833618 >> v1:172.21.15.136:6789/0 conn(0x7f52bc158320 legacy=0x7f52bc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52b2a00640 1 -- 172.21.15.113:0/3648833618 >> v1:172.21.15.113:6789/0 conn(0x7f52bc0ce770 legacy=0x7f52bc15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52baa00640 1 -- 172.21.15.113:0/3648833618 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 841856508 0 0) 0x7f52a0003530 con 0x7f52bc0ce770 2024-03-04T07:34:23.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.473+0000 7f52b2a00640 1 -- 172.21.15.113:0/3648833618 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc0c93b0 con 0x7f52bc06ff60 2024-03-04T07:34:23.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.477+0000 7f52b2a00640 1 --2- 172.21.15.113:0/3648833618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 0x7f52bc0ce230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f52a40097f0 tx=0x7f52a401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23fa683f09bb076a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:23.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.477+0000 7f52baa00640 1 -- 172.21.15.113:0/3648833618 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40b0070 con 0x7f52bc06ff60 2024-03-04T07:34:23.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.477+0000 7f52baa00640 1 -- 172.21.15.113:0/3648833618 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a40ab400 con 0x7f52bc06ff60 2024-03-04T07:34:23.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.477+0000 7f52baa00640 1 -- 172.21.15.113:0/3648833618 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4001680 con 0x7f52bc06ff60 2024-03-04T07:34:23.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.481+0000 7f52c2600640 1 -- 172.21.15.113:0/3648833618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 msgr2=0x7f52bc0ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.481+0000 7f52c2600640 1 --2- 172.21.15.113:0/3648833618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 0x7f52bc0ce230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f52a40097f0 tx=0x7f52a401c400 comp rx=0 tx=0).stop 2024-03-04T07:34:23.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.481+0000 7f52c2600640 1 -- 172.21.15.113:0/3648833618 shutdown_connections 2024-03-04T07:34:23.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.481+0000 7f52c2600640 1 --2- 172.21.15.113:0/3648833618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 0x7f52bc0ce230 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.481+0000 7f52c2600640 1 -- 172.21.15.113:0/3648833618 >> 172.21.15.113:0/3648833618 conn(0x7f52bc06b820 msgr2=0x7f52bc070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:23.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.481+0000 7f52c2600640 1 -- 172.21.15.113:0/3648833618 shutdown_connections 2024-03-04T07:34:23.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.481+0000 7f52c2600640 1 -- 172.21.15.113:0/3648833618 wait complete. 2024-03-04T07:34:23.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.481+0000 7f52c2600640 1 Processor -- start 2024-03-04T07:34:23.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.481+0000 7f52c2600640 1 -- start start 2024-03-04T07:34:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52c2600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 0x7f52bc1f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52c2600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52bc0ce770 0x7f52bc0c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52c2600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc158320 0x7f52bc0c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52c2600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc168d90 con 0x7f52bc0ce770 2024-03-04T07:34:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52c2600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc168c10 con 0x7f52bc158320 2024-03-04T07:34:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52c2600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc168f10 con 0x7f52bc06ff60 2024-03-04T07:34:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52bc0ce770 0x7f52bc0c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52bc0ce770 0x7f52bc0c56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56690/0 (socket says 172.21.15.113:56690) 2024-03-04T07:34:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b2a00640 1 -- 172.21.15.113:0/2691441794 learned_addr learned my addr 172.21.15.113:0/2691441794 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:23.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52bb400640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 0x7f52bc1f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52bbe00640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc158320 0x7f52bc0c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b2a00640 1 -- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 msgr2=0x7f52bc1f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b2a00640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 0x7f52bc1f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b2a00640 1 -- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc158320 msgr2=0x7f52bc0c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b2a00640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc158320 0x7f52bc0c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b2a00640 1 -- 172.21.15.113:0/2691441794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc1f7330 con 0x7f52bc0ce770 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52bbe00640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc158320 0x7f52bc0c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b2a00640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52bc0ce770 0x7f52bc0c56a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f52a401a030 tx=0x7f52a40191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b3e00640 1 -- 172.21.15.113:0/2691441794 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40b9020 con 0x7f52bc0ce770 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b3e00640 1 -- 172.21.15.113:0/2691441794 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4003820 con 0x7f52bc0ce770 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52c2600640 1 -- 172.21.15.113:0/2691441794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52bc0b2310 con 0x7f52bc0ce770 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52b3e00640 1 -- 172.21.15.113:0/2691441794 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40b0040 con 0x7f52bc0ce770 2024-03-04T07:34:23.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52c2600640 1 -- 172.21.15.113:0/2691441794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52bc1f75d0 con 0x7f52bc0ce770 2024-03-04T07:34:23.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.485+0000 7f52c2600640 1 -- 172.21.15.113:0/2691441794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52bc075c00 con 0x7f52bc0ce770 2024-03-04T07:34:23.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.489+0000 7f52b3e00640 1 -- 172.21.15.113:0/2691441794 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52a40bc020 con 0x7f52bc0ce770 2024-03-04T07:34:23.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.489+0000 7f52b3e00640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5298066f60 0x7f5298069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.489+0000 7f52b3e00640 1 -- 172.21.15.113:0/2691441794 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52a411ee30 con 0x7f52bc0ce770 2024-03-04T07:34:23.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.489+0000 7f52bb400640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5298066f60 0x7f5298069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.489+0000 7f52bb400640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5298066f60 0x7f5298069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52ac001570 tx=0x7f52ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:23.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.493+0000 7f52b3e00640 1 -- 172.21.15.113:0/2691441794 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f52a40c3030 con 0x7f52bc0ce770 2024-03-04T07:34:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.745+0000 7f52c2600640 1 -- 172.21.15.113:0/2691441794 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52bc156b40 con 0x7f52bc0ce770 2024-03-04T07:34:23.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.745+0000 7f52b3e00640 1 -- 172.21.15.113:0/2691441794 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52a4015ff0 con 0x7f52bc0ce770 2024-03-04T07:34:23.752 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-03-04T07:34:23.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 -- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5298066f60 msgr2=0x7f5298069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5298066f60 0x7f5298069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f52ac001570 tx=0x7f52ac009040 comp rx=0 tx=0).stop 2024-03-04T07:34:23.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 -- 172.21.15.113:0/2691441794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52bc0ce770 msgr2=0x7f52bc0c56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52bc0ce770 0x7f52bc0c56a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f52a401a030 tx=0x7f52a40191a0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 -- 172.21.15.113:0/2691441794 shutdown_connections 2024-03-04T07:34:23.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc158320 0x7f52bc0c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5298066f60 0x7f5298069420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52bc0ce770 0x7f52bc0c56a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 --2- 172.21.15.113:0/2691441794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f52bc06ff60 0x7f52bc1f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 -- 172.21.15.113:0/2691441794 >> 172.21.15.113:0/2691441794 conn(0x7f52bc06b820 msgr2=0x7f52bc155060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:23.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 -- 172.21.15.113:0/2691441794 shutdown_connections 2024-03-04T07:34:23.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.749+0000 7f52c2600640 1 -- 172.21.15.113:0/2691441794 wait complete. 2024-03-04T07:34:23.775 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T07:34:23.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06d9a00640 1 Processor -- start 2024-03-04T07:34:23.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06d9a00640 1 -- start start 2024-03-04T07:34:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06d9a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06d41517b0 0x7f06d40732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06d9a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d4151b90 0x7f06d4071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06d9a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d40c8400 con 0x7f06d40c98e0 2024-03-04T07:34:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06d9a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d415ac50 con 0x7f06d41517b0 2024-03-04T07:34:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06d9a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40c86e0 con 0x7f06d4151b90 2024-03-04T07:34:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06d3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06d41517b0 0x7f06d40732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06d3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06d41517b0 0x7f06d40732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56704/0 (socket says 172.21.15.113:56704) 2024-03-04T07:34:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06d3e00640 1 -- 172.21.15.113:0/3557136518 learned_addr learned my addr 172.21.15.113:0/3557136518 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.929+0000 7f06cb400640 1 --2- 172.21.15.113:0/3557136518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d4151b90 0x7f06d4071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.933+0000 7f06d3e00640 1 -- 172.21.15.113:0/3557136518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d4151b90 msgr2=0x7f06d4071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:23.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.933+0000 7f06d3e00640 1 --2- 172.21.15.113:0/3557136518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d4151b90 0x7f06d4071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.933+0000 7f06d3e00640 1 -- 172.21.15.113:0/3557136518 >> v1:172.21.15.136:6789/0 conn(0x7f06d40c98e0 legacy=0x7f06d415b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.933+0000 7f06d3e00640 1 -- 172.21.15.113:0/3557136518 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d40c89c0 con 0x7f06d41517b0 2024-03-04T07:34:23.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.933+0000 7f06d3e00640 1 --2- 172.21.15.113:0/3557136518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d41517b0 0x7f06d40732d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009e50 tx=0x7f06c401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c574589d9fa66a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:23.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.933+0000 7f06d3400640 1 -- 172.21.15.113:0/3557136518 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c40b4070 con 0x7f06d41517b0 2024-03-04T07:34:23.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.933+0000 7f06d3400640 1 -- 172.21.15.113:0/3557136518 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c40af400 con 0x7f06d41517b0 2024-03-04T07:34:23.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.933+0000 7f06d3400640 1 -- 172.21.15.113:0/3557136518 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c40af7d0 con 0x7f06d41517b0 2024-03-04T07:34:23.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.933+0000 7f06d9a00640 1 -- 172.21.15.113:0/3557136518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d41517b0 msgr2=0x7f06d40732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.933+0000 7f06d9a00640 1 --2- 172.21.15.113:0/3557136518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d41517b0 0x7f06d40732d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06c4009e50 tx=0x7f06c401f210 comp rx=0 tx=0).stop 2024-03-04T07:34:23.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.937+0000 7f06d9a00640 1 -- 172.21.15.113:0/3557136518 shutdown_connections 2024-03-04T07:34:23.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.937+0000 7f06d9a00640 1 --2- 172.21.15.113:0/3557136518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d4151b90 0x7f06d4071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.937+0000 7f06d9a00640 1 --2- 172.21.15.113:0/3557136518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d41517b0 0x7f06d40732d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.937+0000 7f06d9a00640 1 -- 172.21.15.113:0/3557136518 >> 172.21.15.113:0/3557136518 conn(0x7f06d406f640 msgr2=0x7f06d406fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:23.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.937+0000 7f06d9a00640 1 -- 172.21.15.113:0/3557136518 shutdown_connections 2024-03-04T07:34:23.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.937+0000 7f06d9a00640 1 -- 172.21.15.113:0/3557136518 wait complete. 2024-03-04T07:34:23.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.937+0000 7f06d9a00640 1 Processor -- start 2024-03-04T07:34:23.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.937+0000 7f06d9a00640 1 -- start start 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40c98e0 0x7f06d4154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d9a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d41517b0 0x7f06d414a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d9a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d4151b90 0x7f06d414aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d9a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d4071fe0 con 0x7f06d40c98e0 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d9a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40c8f80 con 0x7f06d4151b90 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d9a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d40c7bf0 con 0x7f06d41517b0 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40c98e0 0x7f06d4154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40c98e0 0x7f06d4154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56716/0 (socket says 172.21.15.113:56716) 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d3e00640 1 -- 172.21.15.113:0/3667568675 learned_addr learned my addr 172.21.15.113:0/3667568675 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06cb400640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d41517b0 0x7f06d414a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d9000640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d4151b90 0x7f06d414aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d3e00640 1 -- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d41517b0 msgr2=0x7f06d414a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d3e00640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d41517b0 0x7f06d414a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d3e00640 1 -- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d4151b90 msgr2=0x7f06d414aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:23.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d3e00640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d4151b90 0x7f06d414aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:23.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d3e00640 1 -- 172.21.15.113:0/3667568675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d4163090 con 0x7f06d40c98e0 2024-03-04T07:34:23.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d3e00640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40c98e0 0x7f06d4154970 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f06c40b2040 tx=0x7f06c401bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:23.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d0c00640 1 -- 172.21.15.113:0/3667568675 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c40b4040 con 0x7f06d40c98e0 2024-03-04T07:34:23.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d0c00640 1 -- 172.21.15.113:0/3667568675 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c40afd60 con 0x7f06d40c98e0 2024-03-04T07:34:23.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d9a00640 1 -- 172.21.15.113:0/3667568675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d4150a00 con 0x7f06d40c98e0 2024-03-04T07:34:23.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d0c00640 1 -- 172.21.15.113:0/3667568675 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4009510 con 0x7f06d40c98e0 2024-03-04T07:34:23.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d9a00640 1 -- 172.21.15.113:0/3667568675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d41f45d0 con 0x7f06d40c98e0 2024-03-04T07:34:23.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.941+0000 7f06d9a00640 1 -- 172.21.15.113:0/3667568675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d4075a80 con 0x7f06d40c98e0 2024-03-04T07:34:23.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.945+0000 7f06d0c00640 1 -- 172.21.15.113:0/3667568675 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06c40c2020 con 0x7f06d40c98e0 2024-03-04T07:34:23.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.945+0000 7f06d0c00640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b0066f90 0x7f06b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:23.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.945+0000 7f06d0c00640 1 -- 172.21.15.113:0/3667568675 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06c4122bc0 con 0x7f06d40c98e0 2024-03-04T07:34:23.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.945+0000 7f06cb400640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b0066f90 0x7f06b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:23.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.945+0000 7f06cb400640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b0066f90 0x7f06b0069450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f06d4151030 tx=0x7f06bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:23.949+0000 7f06d0c00640 1 -- 172.21.15.113:0/3667568675 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f06c4018d70 con 0x7f06d40c98e0 2024-03-04T07:34:23.963 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:34:23.967 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:34:24.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.125+0000 7f06d9a00640 1 -- 172.21.15.113:0/3667568675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06d415b830 con 0x7f06d40c98e0 2024-03-04T07:34:24.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.125+0000 7f06d0c00640 1 -- 172.21.15.113:0/3667568675 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06c40f3c80 con 0x7f06d40c98e0 2024-03-04T07:34:24.130 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-03-04T07:34:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 -- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b0066f90 msgr2=0x7f06b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b0066f90 0x7f06b0069450 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f06d4151030 tx=0x7f06bc008040 comp rx=0 tx=0).stop 2024-03-04T07:34:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 -- 172.21.15.113:0/3667568675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40c98e0 msgr2=0x7f06d4154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40c98e0 0x7f06d4154970 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f06c40b2040 tx=0x7f06c401bfb0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 -- 172.21.15.113:0/3667568675 shutdown_connections 2024-03-04T07:34:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d4151b90 0x7f06d414aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d41517b0 0x7f06d414a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b0066f90 0x7f06b0069450 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 --2- 172.21.15.113:0/3667568675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d40c98e0 0x7f06d4154970 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 -- 172.21.15.113:0/3667568675 >> 172.21.15.113:0/3667568675 conn(0x7f06d406f640 msgr2=0x7f06d40c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 -- 172.21.15.113:0/3667568675 shutdown_connections 2024-03-04T07:34:24.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.129+0000 7f06d9a00640 1 -- 172.21.15.113:0/3667568675 wait complete. 2024-03-04T07:34:24.153 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T07:34:24.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.289+0000 7f876a600640 1 Processor -- start 2024-03-04T07:34:24.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f876a600640 1 -- start start 2024-03-04T07:34:24.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f876a600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f87640c6f80 0x7f87640c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f876a600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 0x7f87640c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f876a600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876415b190 con 0x7f8764157060 2024-03-04T07:34:24.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f876a600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640c1ab0 con 0x7f87640c6f80 2024-03-04T07:34:24.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f876a600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87640c1d90 con 0x7f87640c7360 2024-03-04T07:34:24.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f8763400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f87640c6f80 0x7f87640c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f8763e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 0x7f87640c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f875be00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f8764157060 0x7f8764150cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:60676/0 (socket says 172.21.15.113:60676) 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f875be00640 1 -- 172.21.15.113:0/4141419075 learned_addr learned my addr 172.21.15.113:0/4141419075 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f8763400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f87640c6f80 0x7f87640c2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:49204/0 (socket says 172.21.15.113:49204) 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f8762a00640 1 -- 172.21.15.113:0/4141419075 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347723782 0 0) 0x7f876415b190 con 0x7f8764157060 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f8762a00640 1 -- 172.21.15.113:0/4141419075 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8748003530 con 0x7f8764157060 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f8763e00640 1 -- 172.21.15.113:0/4141419075 >> v2:172.21.15.136:3300/0 conn(0x7f87640c6f80 msgr2=0x7f87640c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f8763e00640 1 --2- 172.21.15.113:0/4141419075 >> v2:172.21.15.136:3300/0 conn(0x7f87640c6f80 0x7f87640c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f8763e00640 1 -- 172.21.15.113:0/4141419075 >> v1:172.21.15.136:6789/0 conn(0x7f8764157060 legacy=0x7f8764150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f8763e00640 1 -- 172.21.15.113:0/4141419075 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87640c2070 con 0x7f87640c7360 2024-03-04T07:34:24.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.293+0000 7f8763400640 1 --2- 172.21.15.113:0/4141419075 >> v2:172.21.15.136:3300/0 conn(0x7f87640c6f80 0x7f87640c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:24.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f8763e00640 1 --2- 172.21.15.113:0/4141419075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 0x7f87640c7d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f875c00bfa0 tx=0x7f875c021350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae13dab07aa84845 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:24.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f8762a00640 1 -- 172.21.15.113:0/4141419075 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0b8070 con 0x7f87640c7360 2024-03-04T07:34:24.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f8762a00640 1 -- 172.21.15.113:0/4141419075 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c0aae50 con 0x7f87640c7360 2024-03-04T07:34:24.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f8762a00640 1 -- 172.21.15.113:0/4141419075 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0b3630 con 0x7f87640c7360 2024-03-04T07:34:24.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f876a600640 1 -- 172.21.15.113:0/4141419075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 msgr2=0x7f87640c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f876a600640 1 --2- 172.21.15.113:0/4141419075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 0x7f87640c7d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f875c00bfa0 tx=0x7f875c021350 comp rx=0 tx=0).stop 2024-03-04T07:34:24.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f876a600640 1 -- 172.21.15.113:0/4141419075 shutdown_connections 2024-03-04T07:34:24.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f876a600640 1 --2- 172.21.15.113:0/4141419075 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 0x7f87640c7d00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f876a600640 1 --2- 172.21.15.113:0/4141419075 >> v2:172.21.15.136:3300/0 conn(0x7f87640c6f80 0x7f87640c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f876a600640 1 -- 172.21.15.113:0/4141419075 >> 172.21.15.113:0/4141419075 conn(0x7f8764066db0 msgr2=0x7f87640671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:24.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f876a600640 1 -- 172.21.15.113:0/4141419075 shutdown_connections 2024-03-04T07:34:24.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.297+0000 7f876a600640 1 -- 172.21.15.113:0/4141419075 wait complete. 2024-03-04T07:34:24.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f876a600640 1 Processor -- start 2024-03-04T07:34:24.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f876a600640 1 -- start start 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f876a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87640c6f80 0x7f87641f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f876a600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 0x7f876406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f876a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764157060 0x7f8764158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f876a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8764151540 con 0x7f87640c6f80 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f876a600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87641513c0 con 0x7f8764157060 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f876a600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87641516c0 con 0x7f87640c7360 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87640c6f80 0x7f87641f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87640c6f80 0x7f87641f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56726/0 (socket says 172.21.15.113:56726) 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763400640 1 -- 172.21.15.113:0/3146469213 learned_addr learned my addr 172.21.15.113:0/3146469213 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f875be00640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 0x7f876406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763e00640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764157060 0x7f8764158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763400640 1 -- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 msgr2=0x7f876406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763400640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 0x7f876406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763400640 1 -- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764157060 msgr2=0x7f8764158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763400640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764157060 0x7f8764158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763400640 1 -- 172.21.15.113:0/3146469213 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87641e69e0 con 0x7f87640c6f80 2024-03-04T07:34:24.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763e00640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764157060 0x7f8764158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f8763400640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87640c6f80 0x7f87641f43c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f875400ebe0 tx=0x7f875400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f875b400640 1 -- 172.21.15.113:0/3146469213 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754017070 con 0x7f87640c6f80 2024-03-04T07:34:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f875b400640 1 -- 172.21.15.113:0/3146469213 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875400cc30 con 0x7f87640c6f80 2024-03-04T07:34:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f876a600640 1 -- 172.21.15.113:0/3146469213 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87641f4900 con 0x7f87640c6f80 2024-03-04T07:34:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f875b400640 1 -- 172.21.15.113:0/3146469213 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754013470 con 0x7f87640c6f80 2024-03-04T07:34:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.301+0000 7f876a600640 1 -- 172.21.15.113:0/3146469213 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87641e52c0 con 0x7f87640c6f80 2024-03-04T07:34:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.305+0000 7f876a600640 1 -- 172.21.15.113:0/3146469213 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8764075c00 con 0x7f87640c6f80 2024-03-04T07:34:24.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.305+0000 7f875b400640 1 -- 172.21.15.113:0/3146469213 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f875401a070 con 0x7f87640c6f80 2024-03-04T07:34:24.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.305+0000 7f875b400640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8740066f60 0x7f8740069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.305+0000 7f875b400640 1 -- 172.21.15.113:0/3146469213 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8754084f30 con 0x7f87640c6f80 2024-03-04T07:34:24.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.305+0000 7f875be00640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8740066f60 0x7f8740069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.309+0000 7f875be00640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8740066f60 0x7f8740069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f87640c66a0 tx=0x7f874c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:24.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.309+0000 7f875b400640 1 -- 172.21.15.113:0/3146469213 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8754055fd0 con 0x7f87640c6f80 2024-03-04T07:34:24.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.485+0000 7f876a600640 1 -- 172.21.15.113:0/3146469213 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8764150fb0 con 0x7f87640c6f80 2024-03-04T07:34:24.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.485+0000 7f875b400640 1 -- 172.21.15.113:0/3146469213 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87540558d0 con 0x7f87640c6f80 2024-03-04T07:34:24.490 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 -- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8740066f60 msgr2=0x7f8740069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8740066f60 0x7f8740069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f87640c66a0 tx=0x7f874c009040 comp rx=0 tx=0).stop 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 -- 172.21.15.113:0/3146469213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87640c6f80 msgr2=0x7f87641f43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87640c6f80 0x7f87641f43c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f875400ebe0 tx=0x7f875400c520 comp rx=0 tx=0).stop 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 -- 172.21.15.113:0/3146469213 shutdown_connections 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8764157060 0x7f8764158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87640c7360 0x7f876406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8740066f60 0x7f8740069420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 --2- 172.21.15.113:0/3146469213 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87640c6f80 0x7f87641f43c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 -- 172.21.15.113:0/3146469213 >> 172.21.15.113:0/3146469213 conn(0x7f8764066db0 msgr2=0x7f8764071830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:24.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 -- 172.21.15.113:0/3146469213 shutdown_connections 2024-03-04T07:34:24.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.489+0000 7f876a600640 1 -- 172.21.15.113:0/3146469213 wait complete. 2024-03-04T07:34:24.514 INFO:tasks.mds_thrash:mds thrasher using random seed: 1709537664 2024-03-04T07:34:24.514 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:34:24.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.661+0000 7fa1b7e00640 1 Processor -- start 2024-03-04T07:34:24.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.661+0000 7fa1b7e00640 1 -- start start 2024-03-04T07:34:24.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1b8152950 0x7fa1b815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa1b80cb540 0x7fa1b80c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b80c4ce0 con 0x7fa1b806faa0 2024-03-04T07:34:24.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b80c8370 con 0x7fa1b8152950 2024-03-04T07:34:24.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b80ca0a0 con 0x7fa1b80cb540 2024-03-04T07:34:24.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1b8152950 0x7fa1b815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa1b8152950 0x7fa1b815bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56738/0 (socket says 172.21.15.113:56738) 2024-03-04T07:34:24.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1ae000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa1b806faa0 0x7fa1b806ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39266/0 (socket says 172.21.15.113:39266) 2024-03-04T07:34:24.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b6a00640 1 -- 172.21.15.113:0/94666446 learned_addr learned my addr 172.21.15.113:0/94666446 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:24.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b7400640 1 --2- 172.21.15.113:0/94666446 >> v2:172.21.15.136:3300/0 conn(0x7fa1b80cb540 0x7fa1b80c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b6a00640 1 -- 172.21.15.113:0/94666446 >> v2:172.21.15.136:3300/0 conn(0x7fa1b80cb540 msgr2=0x7fa1b80c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:24.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b6a00640 1 --2- 172.21.15.113:0/94666446 >> v2:172.21.15.136:3300/0 conn(0x7fa1b80cb540 0x7fa1b80c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b6a00640 1 -- 172.21.15.113:0/94666446 >> v1:172.21.15.113:6789/0 conn(0x7fa1b806faa0 legacy=0x7fa1b806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.665+0000 7fa1b6a00640 1 -- 172.21.15.113:0/94666446 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b80c4f80 con 0x7fa1b8152950 2024-03-04T07:34:24.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b6a00640 1 --2- 172.21.15.113:0/94666446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b8152950 0x7fa1b815bb40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80c8530 tx=0x7fa1a801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d14c4ef48e2a78 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:24.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b6000640 1 -- 172.21.15.113:0/94666446 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80b1070 con 0x7fa1b8152950 2024-03-04T07:34:24.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b6000640 1 -- 172.21.15.113:0/94666446 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a80a8050 con 0x7fa1b8152950 2024-03-04T07:34:24.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b6000640 1 -- 172.21.15.113:0/94666446 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a80ac860 con 0x7fa1b8152950 2024-03-04T07:34:24.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b7e00640 1 -- 172.21.15.113:0/94666446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b8152950 msgr2=0x7fa1b815bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b7e00640 1 --2- 172.21.15.113:0/94666446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b8152950 0x7fa1b815bb40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80c8530 tx=0x7fa1a801c440 comp rx=0 tx=0).stop 2024-03-04T07:34:24.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b7e00640 1 -- 172.21.15.113:0/94666446 shutdown_connections 2024-03-04T07:34:24.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b7e00640 1 --2- 172.21.15.113:0/94666446 >> v2:172.21.15.136:3300/0 conn(0x7fa1b80cb540 0x7fa1b80c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b7e00640 1 --2- 172.21.15.113:0/94666446 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b8152950 0x7fa1b815bb40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b7e00640 1 -- 172.21.15.113:0/94666446 >> 172.21.15.113:0/94666446 conn(0x7fa1b8072f40 msgr2=0x7fa1b80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:24.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b7e00640 1 -- 172.21.15.113:0/94666446 shutdown_connections 2024-03-04T07:34:24.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.669+0000 7fa1b7e00640 1 -- 172.21.15.113:0/94666446 wait complete. 2024-03-04T07:34:24.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b7e00640 1 Processor -- start 2024-03-04T07:34:24.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b7e00640 1 -- start start 2024-03-04T07:34:24.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1b806faa0 0x7fa1b80cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b80cb540 0x7fa1b8148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b8152950 0x7fa1b814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c9a50 con 0x7fa1b8152950 2024-03-04T07:34:24.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b806fe80 con 0x7fa1b80cb540 2024-03-04T07:34:24.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c9d30 con 0x7fa1b806faa0 2024-03-04T07:34:24.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b8152950 0x7fa1b814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1b806faa0 0x7fa1b80cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1b806faa0 0x7fa1b80cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:40910/0 (socket says 172.21.15.113:40910) 2024-03-04T07:34:24.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1b6a00640 1 -- 172.21.15.113:0/1353861341 learned_addr learned my addr 172.21.15.113:0/1353861341 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:24.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.673+0000 7fa1ae000640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b80cb540 0x7fa1b8148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1b7400640 1 -- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1b806faa0 msgr2=0x7fa1b80cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1b7400640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1b806faa0 0x7fa1b80cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1b7400640 1 -- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b80cb540 msgr2=0x7fa1b8148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1b7400640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b80cb540 0x7fa1b8148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1b7400640 1 -- 172.21.15.113:0/1353861341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b80ce080 con 0x7fa1b8152950 2024-03-04T07:34:24.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1b7400640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b8152950 0x7fa1b814ef70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009d90 tx=0x7fa1b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:24.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1af400640 1 -- 172.21.15.113:0/1353861341 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0018020 con 0x7fa1b8152950 2024-03-04T07:34:24.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1af400640 1 -- 172.21.15.113:0/1353861341 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b0014480 con 0x7fa1b8152950 2024-03-04T07:34:24.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1b7e00640 1 -- 172.21.15.113:0/1353861341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b81ee820 con 0x7fa1b8152950 2024-03-04T07:34:24.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1af400640 1 -- 172.21.15.113:0/1353861341 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0014870 con 0x7fa1b8152950 2024-03-04T07:34:24.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1b7e00640 1 -- 172.21.15.113:0/1353861341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b81eed80 con 0x7fa1b8152950 2024-03-04T07:34:24.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.677+0000 7fa1b7e00640 1 -- 172.21.15.113:0/1353861341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b8075a80 con 0x7fa1b8152950 2024-03-04T07:34:24.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.681+0000 7fa1af400640 1 -- 172.21.15.113:0/1353861341 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1b001b020 con 0x7fa1b8152950 2024-03-04T07:34:24.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.681+0000 7fa1af400640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa194066f90 0x7fa194069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:24.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.681+0000 7fa1af400640 1 -- 172.21.15.113:0/1353861341 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1b0085be0 con 0x7fa1b8152950 2024-03-04T07:34:24.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.681+0000 7fa1b6a00640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa194066f90 0x7fa194069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:24.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.681+0000 7fa1b6a00640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa194066f90 0x7fa194069450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa1a801c480 tx=0x7fa1a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:24.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.689+0000 7fa1af400640 1 -- 172.21.15.113:0/1353861341 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa1b0022030 con 0x7fa1b8152950 2024-03-04T07:34:24.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.849+0000 7fa1b7e00640 1 -- 172.21.15.113:0/1353861341 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1b8156160 con 0x7fa1b8152950 2024-03-04T07:34:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1af400640 1 -- 172.21.15.113:0/1353861341 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fa1b81ee820 con 0x7fa1b8152950 2024-03-04T07:34:24.857 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:24.857 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:57.517935+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:34:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 -- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa194066f90 msgr2=0x7fa194069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa194066f90 0x7fa194069450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa1a801c480 tx=0x7fa1a8008040 comp rx=0 tx=0).stop 2024-03-04T07:34:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 -- 172.21.15.113:0/1353861341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b8152950 msgr2=0x7fa1b814ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b8152950 0x7fa1b814ef70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009d90 tx=0x7fa1b000c490 comp rx=0 tx=0).stop 2024-03-04T07:34:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 -- 172.21.15.113:0/1353861341 shutdown_connections 2024-03-04T07:34:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa194066f90 0x7fa194069450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1b8152950 0x7fa1b814ef70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b80cb540 0x7fa1b8148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 --2- 172.21.15.113:0/1353861341 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1b806faa0 0x7fa1b80cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 -- 172.21.15.113:0/1353861341 >> 172.21.15.113:0/1353861341 conn(0x7fa1b8072f40 msgr2=0x7fa1b80705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 -- 172.21.15.113:0/1353861341 shutdown_connections 2024-03-04T07:34:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:24.853+0000 7fa1b7e00640 1 -- 172.21.15.113:0/1353861341 wait complete. 2024-03-04T07:34:24.878 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T07:33:41.507700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T07:33:53.680465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T07:33:54.892043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T07:34:24.878 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T07:34:25.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.013+0000 7f0ee8200640 1 Processor -- start 2024-03-04T07:34:25.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.013+0000 7f0ee8200640 1 -- start start 2024-03-04T07:34:25.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.013+0000 7f0ee8200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ee015ae90 0x7f0ee015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.013+0000 7f0ee8200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee00c9ce0 con 0x7f0ee006ff60 2024-03-04T07:34:25.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.013+0000 7f0ee8200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee015b820 con 0x7f0ee0158320 2024-03-04T07:34:25.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.013+0000 7f0ee8200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee00c9160 con 0x7f0ee015ae90 2024-03-04T07:34:25.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.013+0000 7f0ee6200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ee015ae90 0x7f0ee015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.013+0000 7f0ee4e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0ee006ff60 0x7f0ee00ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39276/0 (socket says 172.21.15.113:39276) 2024-03-04T07:34:25.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.013+0000 7f0ee4e00640 1 -- 172.21.15.113:0/2741304912 learned_addr learned my addr 172.21.15.113:0/2741304912 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:25.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.017+0000 7f0ee6200640 1 -- 172.21.15.113:0/2741304912 >> v1:172.21.15.136:6789/0 conn(0x7f0ee0158320 legacy=0x7f0ee006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.017+0000 7f0ee6200640 1 -- 172.21.15.113:0/2741304912 >> v1:172.21.15.113:6789/0 conn(0x7f0ee006ff60 legacy=0x7f0ee00ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.017+0000 7f0ee6200640 1 -- 172.21.15.113:0/2741304912 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee006e110 con 0x7f0ee015ae90 2024-03-04T07:34:25.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.017+0000 7f0ee6200640 1 --2- 172.21.15.113:0/2741304912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee015ae90 0x7f0ee015b2e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009d90 tx=0x7f0ed801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb41ff08c69efd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:25.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.017+0000 7f0edfe00640 1 -- 172.21.15.113:0/2741304912 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80b2070 con 0x7f0ee015ae90 2024-03-04T07:34:25.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.017+0000 7f0edfe00640 1 -- 172.21.15.113:0/2741304912 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed80a5920 con 0x7f0ee015ae90 2024-03-04T07:34:25.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.017+0000 7f0edfe00640 1 -- 172.21.15.113:0/2741304912 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80a5cf0 con 0x7f0ee015ae90 2024-03-04T07:34:25.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 -- 172.21.15.113:0/2741304912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee015ae90 msgr2=0x7f0ee015b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 --2- 172.21.15.113:0/2741304912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee015ae90 0x7f0ee015b2e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009d90 tx=0x7f0ed801c970 comp rx=0 tx=0).stop 2024-03-04T07:34:25.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 -- 172.21.15.113:0/2741304912 shutdown_connections 2024-03-04T07:34:25.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 --2- 172.21.15.113:0/2741304912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee015ae90 0x7f0ee015b2e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 -- 172.21.15.113:0/2741304912 >> 172.21.15.113:0/2741304912 conn(0x7f0ee006b820 msgr2=0x7f0ee0070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:25.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 -- 172.21.15.113:0/2741304912 shutdown_connections 2024-03-04T07:34:25.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 -- 172.21.15.113:0/2741304912 wait complete. 2024-03-04T07:34:25.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 Processor -- start 2024-03-04T07:34:25.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 -- start start 2024-03-04T07:34:25.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ee006ff60 0x7f0ee014ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee0158320 0x7f0ee01f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ee015ae90 0x7f0ee01f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00ceac0 con 0x7f0ee0158320 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00ce940 con 0x7f0ee006ff60 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.021+0000 7f0ee8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee006dcb0 con 0x7f0ee015ae90 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee4e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee0158320 0x7f0ee01f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee4e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee0158320 0x7f0ee01f67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56772/0 (socket says 172.21.15.113:56772) 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee4e00640 1 -- 172.21.15.113:0/584862751 learned_addr learned my addr 172.21.15.113:0/584862751 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee6200640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ee015ae90 0x7f0ee01f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee5800640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ee006ff60 0x7f0ee014ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee4e00640 1 -- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ee015ae90 msgr2=0x7f0ee01f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee4e00640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ee015ae90 0x7f0ee01f6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee4e00640 1 -- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ee006ff60 msgr2=0x7f0ee014ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee4e00640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ee006ff60 0x7f0ee014ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee4e00640 1 -- 172.21.15.113:0/584862751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee01f3310 con 0x7f0ee0158320 2024-03-04T07:34:25.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee4e00640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee0158320 0x7f0ee01f67d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00c370 tx=0x7f0ecc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:25.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0edd600640 1 -- 172.21.15.113:0/584862751 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc017070 con 0x7f0ee0158320 2024-03-04T07:34:25.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0edd600640 1 -- 172.21.15.113:0/584862751 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ecc005cf0 con 0x7f0ee0158320 2024-03-04T07:34:25.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0edd600640 1 -- 172.21.15.113:0/584862751 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc0072f0 con 0x7f0ee0158320 2024-03-04T07:34:25.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee8200640 1 -- 172.21.15.113:0/584862751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee01f35b0 con 0x7f0ee0158320 2024-03-04T07:34:25.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee8200640 1 -- 172.21.15.113:0/584862751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee01f3b70 con 0x7f0ee0158320 2024-03-04T07:34:25.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0ee8200640 1 -- 172.21.15.113:0/584862751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee006fd40 con 0x7f0ee0158320 2024-03-04T07:34:25.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0edd600640 1 -- 172.21.15.113:0/584862751 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ecc01a070 con 0x7f0ee0158320 2024-03-04T07:34:25.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.025+0000 7f0edd600640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ec0066f60 0x7f0ec0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.029+0000 7f0edd600640 1 -- 172.21.15.113:0/584862751 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ecc084400 con 0x7f0ee0158320 2024-03-04T07:34:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.029+0000 7f0ee5800640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ec0066f60 0x7f0ec0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.029+0000 7f0ee5800640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ec0066f60 0x7f0ec0069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4000f70 tx=0x7f0ed4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:25.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.033+0000 7f0edd600640 1 -- 172.21.15.113:0/584862751 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0ecc054c90 con 0x7f0ee0158320 2024-03-04T07:34:25.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 -- 172.21.15.113:0/584862751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ee0075be0 con 0x7f0ee0158320 2024-03-04T07:34:25.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0edd600640 1 -- 172.21.15.113:0/584862751 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ecc058d30 con 0x7f0ee0158320 2024-03-04T07:34:25.345 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-03-04T07:34:25.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 -- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ec0066f60 msgr2=0x7f0ec0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ec0066f60 0x7f0ec0069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0ed4000f70 tx=0x7f0ed4009040 comp rx=0 tx=0).stop 2024-03-04T07:34:25.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 -- 172.21.15.113:0/584862751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee0158320 msgr2=0x7f0ee01f67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee0158320 0x7f0ee01f67d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00c370 tx=0x7f0ecc00c830 comp rx=0 tx=0).stop 2024-03-04T07:34:25.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 -- 172.21.15.113:0/584862751 shutdown_connections 2024-03-04T07:34:25.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ee015ae90 0x7f0ee01f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ec0066f60 0x7f0ec0069420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ee0158320 0x7f0ee01f67d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 --2- 172.21.15.113:0/584862751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ee006ff60 0x7f0ee014ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 -- 172.21.15.113:0/584862751 >> 172.21.15.113:0/584862751 conn(0x7f0ee006b820 msgr2=0x7f0ee00cb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:25.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.341+0000 7f0ee8200640 1 -- 172.21.15.113:0/584862751 shutdown_connections 2024-03-04T07:34:25.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.345+0000 7f0ee8200640 1 -- 172.21.15.113:0/584862751 wait complete. 2024-03-04T07:34:25.367 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T07:34:25.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb295000640 1 Processor -- start 2024-03-04T07:34:25.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb295000640 1 -- start start 2024-03-04T07:34:25.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb295000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb29015ad80 0x7fb29015b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb295000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2900c3780 con 0x7fb2900c5190 2024-03-04T07:34:25.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb295000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29015b6f0 con 0x7fb290152cb0 2024-03-04T07:34:25.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb295000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900c4ed0 con 0x7fb29015ad80 2024-03-04T07:34:25.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb285400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb2900c5190 0x7fb29006f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39290/0 (socket says 172.21.15.113:39290) 2024-03-04T07:34:25.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb285400640 1 -- 172.21.15.113:0/1135002812 learned_addr learned my addr 172.21.15.113:0/1135002812 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:25.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28e800640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb29015ad80 0x7fb29015b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580996586 0 0) 0x7fb2900c3780 con 0x7fb2900c5190 2024-03-04T07:34:25.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb274003530 con 0x7fb2900c5190 2024-03-04T07:34:25.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300405128 0 0) 0x7fb29015b6f0 con 0x7fb290152cb0 2024-03-04T07:34:25.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2900c3780 con 0x7fb290152cb0 2024-03-04T07:34:25.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2121980066 0 0) 0x7fb274003530 con 0x7fb2900c5190 2024-03-04T07:34:25.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb29015ad80 msgr2=0x7fb29015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:25.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb29015ad80 0x7fb29015b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 >> v1:172.21.15.136:6789/0 conn(0x7fb290152cb0 legacy=0x7fb2900c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2900c3a40 con 0x7fb2900c5190 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fb278001680 con 0x7fb2900c5190 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 >> v1:172.21.15.113:6789/0 conn(0x7fb2900c5190 legacy=0x7fb29006f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb274003b80 0x7fb274003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb274004810 0x7fb274004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.505+0000 7fb28d400640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2740051d0 0x7fb27400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900c4ed0 con 0x7fb2740051d0 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb274010380 con 0x7fb274003b80 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb274010580 con 0x7fb274004810 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28e800640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb29015ad80 0x7fb29015b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28de00640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb274003b80 0x7fb274003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28e800640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2740051d0 0x7fb27400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb285400640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb274004810 0x7fb274004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28e800640 1 -- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb274004810 msgr2=0x7fb274004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:25.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28e800640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb274004810 0x7fb274004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28e800640 1 -- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb274003b80 msgr2=0x7fb274003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:25.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28e800640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb274003b80 0x7fb274003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28e800640 1 -- 172.21.15.113:0/1135002812 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb274003530 con 0x7fb2740051d0 2024-03-04T07:34:25.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28e800640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2740051d0 0x7fb27400b9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb29015b160 tx=0x7fb28801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3354b1eb2dbab0a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:25.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880097f0 con 0x7fb2740051d0 2024-03-04T07:34:25.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb288004030 con 0x7fb2740051d0 2024-03-04T07:34:25.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.509+0000 7fb28d400640 1 -- 172.21.15.113:0/1135002812 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288004400 con 0x7fb2740051d0 2024-03-04T07:34:25.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 -- 172.21.15.113:0/1135002812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2740051d0 msgr2=0x7fb27400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 --2- 172.21.15.113:0/1135002812 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2740051d0 0x7fb27400b9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb29015b160 tx=0x7fb28801cab0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb28e800640 1 -- 172.21.15.113:0/1135002812 reap_dead start 2024-03-04T07:34:25.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 -- 172.21.15.113:0/1135002812 shutdown_connections 2024-03-04T07:34:25.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 -- 172.21.15.113:0/1135002812 >> 172.21.15.113:0/1135002812 conn(0x7fb290066db0 msgr2=0x7fb2900671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:25.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 -- 172.21.15.113:0/1135002812 shutdown_connections 2024-03-04T07:34:25.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 -- 172.21.15.113:0/1135002812 wait complete. 2024-03-04T07:34:25.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 Processor -- start 2024-03-04T07:34:25.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 -- start start 2024-03-04T07:34:25.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2740051d0 0x7fb29014de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2900c9820 0x7fb29006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb290150e60 0x7fb2901512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900705f0 con 0x7fb290150e60 2024-03-04T07:34:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29015a050 con 0x7fb2740051d0 2024-03-04T07:34:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb295000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29015a580 con 0x7fb2900c9820 2024-03-04T07:34:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb28de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2740051d0 0x7fb29014de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb28e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb290150e60 0x7fb2901512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb28de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2740051d0 0x7fb29014de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:49260/0 (socket says 172.21.15.113:49260) 2024-03-04T07:34:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb28de00640 1 -- 172.21.15.113:0/3022970318 learned_addr learned my addr 172.21.15.113:0/3022970318 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:25.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.513+0000 7fb285400640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2900c9820 0x7fb29006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb28de00640 1 -- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2900c9820 msgr2=0x7fb29006f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb28de00640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2900c9820 0x7fb29006f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb28de00640 1 -- 172.21.15.113:0/3022970318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb290150e60 msgr2=0x7fb2901512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb28de00640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb290150e60 0x7fb2901512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb28de00640 1 -- 172.21.15.113:0/3022970318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290151820 con 0x7fb2740051d0 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb28e800640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb290150e60 0x7fb2901512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb28de00640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2740051d0 0x7fb29014de20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb28000fb60 tx=0x7fb28000d520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb286a00640 1 -- 172.21.15.113:0/3022970318 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28000dbd0 con 0x7fb2740051d0 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb286a00640 1 -- 172.21.15.113:0/3022970318 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb280018070 con 0x7fb2740051d0 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb295000640 1 -- 172.21.15.113:0/3022970318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29014e440 con 0x7fb2740051d0 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb286a00640 1 -- 172.21.15.113:0/3022970318 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280003980 con 0x7fb2740051d0 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb295000640 1 -- 172.21.15.113:0/3022970318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2901500a0 con 0x7fb2740051d0 2024-03-04T07:34:25.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb295000640 1 -- 172.21.15.113:0/3022970318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb290075c00 con 0x7fb2740051d0 2024-03-04T07:34:25.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.517+0000 7fb286a00640 1 -- 172.21.15.113:0/3022970318 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb280003b20 con 0x7fb2740051d0 2024-03-04T07:34:25.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.521+0000 7fb286a00640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 0x7fb26c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.521+0000 7fb286a00640 1 -- 172.21.15.113:0/3022970318 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb280085d30 con 0x7fb2740051d0 2024-03-04T07:34:25.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.521+0000 7fb285400640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 0x7fb26c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.521+0000 7fb285400640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 0x7fb26c069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb2900cea70 tx=0x7fb278007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:25.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.533+0000 7fb286a00640 1 -- 172.21.15.113:0/3022970318 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb290151820 con 0x7fb2740051d0 2024-03-04T07:34:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.697+0000 7fb295000640 1 -- 172.21.15.113:0/3022970318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb290067550 con 0x7fb2740051d0 2024-03-04T07:34:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.697+0000 7fb286a00640 1 -- 172.21.15.113:0/3022970318 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb280056760 con 0x7fb2740051d0 2024-03-04T07:34:25.702 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 -- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 msgr2=0x7fb26c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 0x7fb26c069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb2900cea70 tx=0x7fb278007040 comp rx=0 tx=0).stop 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 -- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2740051d0 msgr2=0x7fb29014de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2740051d0 0x7fb29014de20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb28000fb60 tx=0x7fb28000d520 comp rx=0 tx=0).stop 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 -- 172.21.15.113:0/3022970318 shutdown_connections 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 0x7fb26c069420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb290150e60 0x7fb2901512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2900c9820 0x7fb29006f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 --2- 172.21.15.113:0/3022970318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2740051d0 0x7fb29014de20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 -- 172.21.15.113:0/3022970318 >> 172.21.15.113:0/3022970318 conn(0x7fb290066db0 msgr2=0x7fb29006c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 -- 172.21.15.113:0/3022970318 shutdown_connections 2024-03-04T07:34:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.701+0000 7fb295000640 1 -- 172.21.15.113:0/3022970318 wait complete. 2024-03-04T07:34:25.729 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T07:34:25.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.865+0000 7fc232200640 1 Processor -- start 2024-03-04T07:34:25.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.865+0000 7fc232200640 1 -- start start 2024-03-04T07:34:25.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc232200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc22c06e6a0 0x7fc22c0c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc232200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc22c06ea80 0x7fc22c0c2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc232200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c0c9530 con 0x7fc22c0c2a70 2024-03-04T07:34:25.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc232200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22c15b7d0 con 0x7fc22c06ea80 2024-03-04T07:34:25.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc232200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22c15abe0 con 0x7fc22c06e6a0 2024-03-04T07:34:25.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc223e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc22c06ea80 0x7fc22c0c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc223e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc22c06ea80 0x7fc22c0c2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56800/0 (socket says 172.21.15.113:56800) 2024-03-04T07:34:25.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc223e00640 1 -- 172.21.15.113:0/873858712 learned_addr learned my addr 172.21.15.113:0/873858712 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:25.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc223e00640 1 -- 172.21.15.113:0/873858712 >> v2:172.21.15.136:3300/0 conn(0x7fc22c06e6a0 msgr2=0x7fc22c0c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:25.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc223e00640 1 --2- 172.21.15.113:0/873858712 >> v2:172.21.15.136:3300/0 conn(0x7fc22c06e6a0 0x7fc22c0c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc223e00640 1 -- 172.21.15.113:0/873858712 >> v1:172.21.15.136:6789/0 conn(0x7fc22c0c2a70 legacy=0x7fc22c0c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:25.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.869+0000 7fc223e00640 1 -- 172.21.15.113:0/873858712 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c15aea0 con 0x7fc22c06ea80 2024-03-04T07:34:25.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc223e00640 1 --2- 172.21.15.113:0/873858712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06ea80 0x7fc22c0c2490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0c4fa0 tx=0x7fc21401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7758ea067f76bc0b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:25.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc22be00640 1 -- 172.21.15.113:0/873858712 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140b5070 con 0x7fc22c06ea80 2024-03-04T07:34:25.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc22be00640 1 -- 172.21.15.113:0/873858712 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2140a89d0 con 0x7fc22c06ea80 2024-03-04T07:34:25.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc22be00640 1 -- 172.21.15.113:0/873858712 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140a8da0 con 0x7fc22c06ea80 2024-03-04T07:34:25.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc232200640 1 -- 172.21.15.113:0/873858712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06ea80 msgr2=0x7fc22c0c2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc232200640 1 --2- 172.21.15.113:0/873858712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06ea80 0x7fc22c0c2490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0c4fa0 tx=0x7fc21401ecd0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc232200640 1 -- 172.21.15.113:0/873858712 shutdown_connections 2024-03-04T07:34:25.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc232200640 1 --2- 172.21.15.113:0/873858712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06ea80 0x7fc22c0c2490 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc232200640 1 --2- 172.21.15.113:0/873858712 >> v2:172.21.15.136:3300/0 conn(0x7fc22c06e6a0 0x7fc22c0c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc232200640 1 -- 172.21.15.113:0/873858712 >> 172.21.15.113:0/873858712 conn(0x7fc22c06f460 msgr2=0x7fc22c06f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:25.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc232200640 1 -- 172.21.15.113:0/873858712 shutdown_connections 2024-03-04T07:34:25.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.873+0000 7fc232200640 1 -- 172.21.15.113:0/873858712 wait complete. 2024-03-04T07:34:25.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc232200640 1 Processor -- start 2024-03-04T07:34:25.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc232200640 1 -- start start 2024-03-04T07:34:25.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc232200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc22c06e6a0 0x7fc22c0cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc232200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc22c06ea80 0x7fc22c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc232200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c0c2a70 0x7fc22c14b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc232200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0c97d0 con 0x7fc22c0c2a70 2024-03-04T07:34:25.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc232200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c15b460 con 0x7fc22c06ea80 2024-03-04T07:34:25.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc232200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0c9140 con 0x7fc22c06e6a0 2024-03-04T07:34:25.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc231800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c0c2a70 0x7fc22c14b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc230e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc22c06e6a0 0x7fc22c0cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc231800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c0c2a70 0x7fc22c14b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56816/0 (socket says 172.21.15.113:56816) 2024-03-04T07:34:25.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc231800640 1 -- 172.21.15.113:0/263036912 learned_addr learned my addr 172.21.15.113:0/263036912 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:25.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc223e00640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc22c06ea80 0x7fc22c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc231800640 1 -- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc22c06e6a0 msgr2=0x7fc22c0cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc231800640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc22c06e6a0 0x7fc22c0cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc231800640 1 -- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc22c06ea80 msgr2=0x7fc22c0af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc231800640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc22c06ea80 0x7fc22c0af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc231800640 1 -- 172.21.15.113:0/263036912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c0afc90 con 0x7fc22c0c2a70 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc231800640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c0c2a70 0x7fc22c14b300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc22400c7b0 tx=0x7fc22400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc229600640 1 -- 172.21.15.113:0/263036912 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224017070 con 0x7fc22c0c2a70 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc229600640 1 -- 172.21.15.113:0/263036912 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc224003db0 con 0x7fc22c0c2a70 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc232200640 1 -- 172.21.15.113:0/263036912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c0aff50 con 0x7fc22c0c2a70 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc229600640 1 -- 172.21.15.113:0/263036912 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2240052b0 con 0x7fc22c0c2a70 2024-03-04T07:34:25.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.877+0000 7fc232200640 1 -- 172.21.15.113:0/263036912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c1f41b0 con 0x7fc22c0c2a70 2024-03-04T07:34:25.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.881+0000 7fc232200640 1 -- 172.21.15.113:0/263036912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c075a80 con 0x7fc22c0c2a70 2024-03-04T07:34:25.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.881+0000 7fc229600640 1 -- 172.21.15.113:0/263036912 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc22401a070 con 0x7fc22c0c2a70 2024-03-04T07:34:25.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.881+0000 7fc229600640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc208066f90 0x7fc208069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:25.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.881+0000 7fc229600640 1 -- 172.21.15.113:0/263036912 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc224084650 con 0x7fc22c0c2a70 2024-03-04T07:34:25.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.881+0000 7fc230e00640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc208066f90 0x7fc208069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:25.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.885+0000 7fc230e00640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc208066f90 0x7fc208069450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098f0 tx=0x7fc21c006f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:25.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:25.885+0000 7fc229600640 1 -- 172.21.15.113:0/263036912 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc224054f80 con 0x7fc22c0c2a70 2024-03-04T07:34:26.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 -- 172.21.15.113:0/263036912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc22c0c8f20 con 0x7fc22c0c2a70 2024-03-04T07:34:26.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc229600640 1 -- 172.21.15.113:0/263036912 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc224059020 con 0x7fc22c0c2a70 2024-03-04T07:34:26.085 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-03-04T07:34:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 -- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc208066f90 msgr2=0x7fc208069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc208066f90 0x7fc208069450 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0098f0 tx=0x7fc21c006f90 comp rx=0 tx=0).stop 2024-03-04T07:34:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 -- 172.21.15.113:0/263036912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c0c2a70 msgr2=0x7fc22c14b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c0c2a70 0x7fc22c14b300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc22400c7b0 tx=0x7fc22400cc70 comp rx=0 tx=0).stop 2024-03-04T07:34:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 -- 172.21.15.113:0/263036912 shutdown_connections 2024-03-04T07:34:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc208066f90 0x7fc208069450 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c0c2a70 0x7fc22c14b300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc22c06ea80 0x7fc22c0af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 --2- 172.21.15.113:0/263036912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc22c06e6a0 0x7fc22c0cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 -- 172.21.15.113:0/263036912 >> 172.21.15.113:0/263036912 conn(0x7fc22c06f460 msgr2=0x7fc22c153e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:26.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 -- 172.21.15.113:0/263036912 shutdown_connections 2024-03-04T07:34:26.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.081+0000 7fc232200640 1 -- 172.21.15.113:0/263036912 wait complete. 2024-03-04T07:34:26.115 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2024-03-04T07:34:26.115 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:34:26.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6beae00640 1 Processor -- start 2024-03-04T07:34:26.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6beae00640 1 -- start start 2024-03-04T07:34:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6beae00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be406ff20 0x7f6be406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6beae00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6be40ce120 0x7f6be40ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6beae00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be40ca0e0 con 0x7f6be4158010 2024-03-04T07:34:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6beae00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be406f450 con 0x7f6be40ce120 2024-03-04T07:34:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6beae00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be415ad80 con 0x7f6be406ff20 2024-03-04T07:34:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6be8e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6be40ce120 0x7f6be40ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6be8e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6be40ce120 0x7f6be40ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56828/0 (socket says 172.21.15.113:56828) 2024-03-04T07:34:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6be8e00640 1 -- 172.21.15.113:0/992703324 learned_addr learned my addr 172.21.15.113:0/992703324 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:26.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6be3400640 1 --2- 172.21.15.113:0/992703324 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be406ff20 0x7f6be406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6be8e00640 1 -- 172.21.15.113:0/992703324 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be406ff20 msgr2=0x7f6be406dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:26.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6be8e00640 1 --2- 172.21.15.113:0/992703324 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be406ff20 0x7f6be406dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6be8e00640 1 -- 172.21.15.113:0/992703324 >> v1:172.21.15.136:6789/0 conn(0x7f6be4158010 legacy=0x7f6be415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.249+0000 7f6be8e00640 1 -- 172.21.15.113:0/992703324 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be415aa30 con 0x7f6be40ce120 2024-03-04T07:34:26.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6be8e00640 1 --2- 172.21.15.113:0/992703324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be40ce120 0x7f6be40ce5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009e50 tx=0x7f6bdc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bbc84d77c94bcf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:26.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6be2a00640 1 -- 172.21.15.113:0/992703324 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0a5700 con 0x7f6be40ce120 2024-03-04T07:34:26.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6be2a00640 1 -- 172.21.15.113:0/992703324 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc0a58a0 con 0x7f6be40ce120 2024-03-04T07:34:26.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6be2a00640 1 -- 172.21.15.113:0/992703324 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0a5c70 con 0x7f6be40ce120 2024-03-04T07:34:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6beae00640 1 -- 172.21.15.113:0/992703324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be40ce120 msgr2=0x7f6be40ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6beae00640 1 --2- 172.21.15.113:0/992703324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be40ce120 0x7f6be40ce5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009e50 tx=0x7f6bdc01ca30 comp rx=0 tx=0).stop 2024-03-04T07:34:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6beae00640 1 -- 172.21.15.113:0/992703324 shutdown_connections 2024-03-04T07:34:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6beae00640 1 --2- 172.21.15.113:0/992703324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be40ce120 0x7f6be40ce5a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6beae00640 1 --2- 172.21.15.113:0/992703324 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be406ff20 0x7f6be406dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6beae00640 1 -- 172.21.15.113:0/992703324 >> 172.21.15.113:0/992703324 conn(0x7f6be40734d0 msgr2=0x7f6be4066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:26.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6beae00640 1 -- 172.21.15.113:0/992703324 shutdown_connections 2024-03-04T07:34:26.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6beae00640 1 -- 172.21.15.113:0/992703324 wait complete. 2024-03-04T07:34:26.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6beae00640 1 Processor -- start 2024-03-04T07:34:26.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.253+0000 7f6beae00640 1 -- start start 2024-03-04T07:34:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6beae00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be406ff20 0x7f6be4139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6beae00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6be40ce120 0x7f6be4143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6beae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be4158010 0x7f6be414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6beae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be415b920 con 0x7f6be4158010 2024-03-04T07:34:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6beae00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be406e290 con 0x7f6be40ce120 2024-03-04T07:34:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6beae00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be41583f0 con 0x7f6be406ff20 2024-03-04T07:34:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be4158010 0x7f6be414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be4158010 0x7f6be414d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56842/0 (socket says 172.21.15.113:56842) 2024-03-04T07:34:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be8e00640 1 -- 172.21.15.113:0/364396129 learned_addr learned my addr 172.21.15.113:0/364396129 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be3400640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6be40ce120 0x7f6be4143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be3e00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be406ff20 0x7f6be4139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be8e00640 1 -- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be406ff20 msgr2=0x7f6be4139090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be8e00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be406ff20 0x7f6be4139090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be8e00640 1 -- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6be40ce120 msgr2=0x7f6be4143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be8e00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6be40ce120 0x7f6be4143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be8e00640 1 -- 172.21.15.113:0/364396129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be4143f70 con 0x7f6be4158010 2024-03-04T07:34:26.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6be8e00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be4158010 0x7f6be414d040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc01a660 tx=0x7f6bdc0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:26.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6bdbe00640 1 -- 172.21.15.113:0/364396129 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0bc020 con 0x7f6be4158010 2024-03-04T07:34:26.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6bdbe00640 1 -- 172.21.15.113:0/364396129 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc0b8670 con 0x7f6be4158010 2024-03-04T07:34:26.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6bdbe00640 1 -- 172.21.15.113:0/364396129 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc0b8a60 con 0x7f6be4158010 2024-03-04T07:34:26.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6beae00640 1 -- 172.21.15.113:0/364396129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be40cd1c0 con 0x7f6be4158010 2024-03-04T07:34:26.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6beae00640 1 -- 172.21.15.113:0/364396129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be4145950 con 0x7f6be4158010 2024-03-04T07:34:26.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.257+0000 7f6beae00640 1 -- 172.21.15.113:0/364396129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be4075c00 con 0x7f6be4158010 2024-03-04T07:34:26.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.261+0000 7f6bdbe00640 1 -- 172.21.15.113:0/364396129 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6bdc0bf020 con 0x7f6be4158010 2024-03-04T07:34:26.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.261+0000 7f6bdbe00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc4066f60 0x7f6bc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.261+0000 7f6bdbe00640 1 -- 172.21.15.113:0/364396129 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bdc120220 con 0x7f6be4158010 2024-03-04T07:34:26.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.261+0000 7f6be3e00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc4066f60 0x7f6bc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.261+0000 7f6be3e00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc4066f60 0x7f6bc4069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40015b0 tx=0x7f6bd4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:26.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.269+0000 7f6bdbe00640 1 -- 172.21.15.113:0/364396129 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6bdc0c6030 con 0x7f6be4158010 2024-03-04T07:34:26.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.529+0000 7f6beae00640 1 -- 172.21.15.113:0/364396129 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6be406fd00 con 0x7f6be4158010 2024-03-04T07:34:26.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.529+0000 7f6bdbe00640 1 -- 172.21.15.113:0/364396129 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f6bdc016620 con 0x7f6be4158010 2024-03-04T07:34:26.535 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:26.535 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:34:26.535 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:34:26.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 -- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc4066f60 msgr2=0x7f6bc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc4066f60 0x7f6bc4069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40015b0 tx=0x7f6bd4009040 comp rx=0 tx=0).stop 2024-03-04T07:34:26.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 -- 172.21.15.113:0/364396129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be4158010 msgr2=0x7f6be414d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be4158010 0x7f6be414d040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc01a660 tx=0x7f6bdc0197d0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 -- 172.21.15.113:0/364396129 shutdown_connections 2024-03-04T07:34:26.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc4066f60 0x7f6bc4069420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be4158010 0x7f6be414d040 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6be40ce120 0x7f6be4143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 --2- 172.21.15.113:0/364396129 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be406ff20 0x7f6be4139090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 -- 172.21.15.113:0/364396129 >> 172.21.15.113:0/364396129 conn(0x7f6be40734d0 msgr2=0x7f6be40ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:26.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 -- 172.21.15.113:0/364396129 shutdown_connections 2024-03-04T07:34:26.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.533+0000 7f6beae00640 1 -- 172.21.15.113:0/364396129 wait complete. 2024-03-04T07:34:26.556 INFO:tasks.mds_thrash:Ready to start thrashing 2024-03-04T07:34:26.557 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2024-03-04T07:34:26.557 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T07:34:26.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ecce00640 1 Processor -- start 2024-03-04T07:34:26.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ecce00640 1 -- start start 2024-03-04T07:34:26.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ecce00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6ec8152b20 0x7f6ec8070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ecce00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec8152f00 con 0x7f6ec815c510 2024-03-04T07:34:26.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ecce00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec80c5040 con 0x7f6ec80c8b00 2024-03-04T07:34:26.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ecce00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec80c8f90 con 0x7f6ec8152b20 2024-03-04T07:34:26.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ec6800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6ec815c510 0x7f6ec8163190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39306/0 (socket says 172.21.15.113:39306) 2024-03-04T07:34:26.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ec6800640 1 -- 172.21.15.113:0/1859710199 learned_addr learned my addr 172.21.15.113:0/1859710199 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:26.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ec5e00640 1 --2- 172.21.15.113:0/1859710199 >> v2:172.21.15.136:3300/0 conn(0x7f6ec8152b20 0x7f6ec8070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ec5400640 1 -- 172.21.15.113:0/1859710199 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365388231 0 0) 0x7f6ec8152f00 con 0x7f6ec815c510 2024-03-04T07:34:26.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ec5400640 1 -- 172.21.15.113:0/1859710199 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eac003530 con 0x7f6ec815c510 2024-03-04T07:34:26.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ec5e00640 1 -- 172.21.15.113:0/1859710199 >> v1:172.21.15.136:6789/0 conn(0x7f6ec80c8b00 legacy=0x7f6ec815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ec5e00640 1 -- 172.21.15.113:0/1859710199 >> v1:172.21.15.113:6789/0 conn(0x7f6ec815c510 legacy=0x7f6ec8163190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.701+0000 7f6ec5e00640 1 -- 172.21.15.113:0/1859710199 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec80c3250 con 0x7f6ec8152b20 2024-03-04T07:34:26.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ec5e00640 1 --2- 172.21.15.113:0/1859710199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec8152b20 0x7f6ec8070010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80097c0 tx=0x7f6eb801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fccec8f4e4b9b51 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:26.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ec5400640 1 -- 172.21.15.113:0/1859710199 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80b0070 con 0x7f6ec8152b20 2024-03-04T07:34:26.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ec5400640 1 -- 172.21.15.113:0/1859710199 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb80ab600 con 0x7f6ec8152b20 2024-03-04T07:34:26.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ec5400640 1 -- 172.21.15.113:0/1859710199 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80ab9b0 con 0x7f6ec8152b20 2024-03-04T07:34:26.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ecce00640 1 -- 172.21.15.113:0/1859710199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec8152b20 msgr2=0x7f6ec8070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ecce00640 1 --2- 172.21.15.113:0/1859710199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec8152b20 0x7f6ec8070010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80097c0 tx=0x7f6eb801c4a0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ecce00640 1 -- 172.21.15.113:0/1859710199 shutdown_connections 2024-03-04T07:34:26.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ecce00640 1 --2- 172.21.15.113:0/1859710199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec8152b20 0x7f6ec8070010 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ecce00640 1 -- 172.21.15.113:0/1859710199 >> 172.21.15.113:0/1859710199 conn(0x7f6ec80709a0 msgr2=0x7f6ec8070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:26.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ecce00640 1 -- 172.21.15.113:0/1859710199 shutdown_connections 2024-03-04T07:34:26.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.705+0000 7f6ecce00640 1 -- 172.21.15.113:0/1859710199 wait complete. 2024-03-04T07:34:26.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ecce00640 1 Processor -- start 2024-03-04T07:34:26.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ecce00640 1 -- start start 2024-03-04T07:34:26.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ecce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80c8b00 0x7f6ec8142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ecce00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ec8152b20 0x7f6ec8156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ecce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec815c510 0x7f6ec814cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ecce00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8158810 con 0x7f6ec80c8b00 2024-03-04T07:34:26.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ecce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80c5320 con 0x7f6ec815c510 2024-03-04T07:34:26.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ecce00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80c8490 con 0x7f6ec8152b20 2024-03-04T07:34:26.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ec5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80c8b00 0x7f6ec8142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ec5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80c8b00 0x7f6ec8142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56858/0 (socket says 172.21.15.113:56858) 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ec5e00640 1 -- 172.21.15.113:0/2849190585 learned_addr learned my addr 172.21.15.113:0/2849190585 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ec6800640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec815c510 0x7f6ec814cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ebfe00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ec8152b20 0x7f6ec8156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ec5e00640 1 -- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ec8152b20 msgr2=0x7f6ec8156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ec5e00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ec8152b20 0x7f6ec8156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ec5e00640 1 -- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec815c510 msgr2=0x7f6ec814cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ec5e00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec815c510 0x7f6ec814cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ec5e00640 1 -- 172.21.15.113:0/2849190585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec8156bb0 con 0x7f6ec80c8b00 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ec5e00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80c8b00 0x7f6ec8142060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80ae040 tx=0x7f6eb8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ebe000640 1 -- 172.21.15.113:0/2849190585 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80b9070 con 0x7f6ec80c8b00 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ebe000640 1 -- 172.21.15.113:0/2849190585 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb80ab9b0 con 0x7f6ec80c8b00 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ecce00640 1 -- 172.21.15.113:0/2849190585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec8156e90 con 0x7f6ec80c8b00 2024-03-04T07:34:26.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ebe000640 1 -- 172.21.15.113:0/2849190585 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80b0040 con 0x7f6ec80c8b00 2024-03-04T07:34:26.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.709+0000 7f6ecce00640 1 -- 172.21.15.113:0/2849190585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec814c460 con 0x7f6ec80c8b00 2024-03-04T07:34:26.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.713+0000 7f6ecce00640 1 -- 172.21.15.113:0/2849190585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec8075c00 con 0x7f6ec80c8b00 2024-03-04T07:34:26.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.713+0000 7f6ebe000640 1 -- 172.21.15.113:0/2849190585 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6eb80bc070 con 0x7f6ec80c8b00 2024-03-04T07:34:26.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.713+0000 7f6ebe000640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ea4066f90 0x7f6ea4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:26.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.713+0000 7f6ebe000640 1 -- 172.21.15.113:0/2849190585 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6eb811f120 con 0x7f6ec80c8b00 2024-03-04T07:34:26.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.713+0000 7f6ebfe00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ea4066f90 0x7f6ea4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:26.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.717+0000 7f6ebfe00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ea4066f90 0x7f6ea4069450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8164210 tx=0x7f6eb0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:26.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.717+0000 7f6ebe000640 1 -- 172.21.15.113:0/2849190585 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6eb8016090 con 0x7f6ec80c8b00 2024-03-04T07:34:26.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.881+0000 7f6ecce00640 1 -- 172.21.15.113:0/2849190585 --> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ec80ce130 con 0x7f6ea4066f90 2024-03-04T07:34:26.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.897+0000 7f6ebe000640 1 -- 172.21.15.113:0/2849190585 <== mgr.4103 v2:172.21.15.136:6832/3353359815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251931 (secure 0 0 0) 0x7f6ec80ce130 con 0x7f6ea4066f90 2024-03-04T07:34:26.905 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:26.907 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-03-04T07:34:26.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 -- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ea4066f90 msgr2=0x7f6ea4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ea4066f90 0x7f6ea4069450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8164210 tx=0x7f6eb0009040 comp rx=0 tx=0).stop 2024-03-04T07:34:26.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 -- 172.21.15.113:0/2849190585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80c8b00 msgr2=0x7f6ec8142060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:26.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80c8b00 0x7f6ec8142060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80ae040 tx=0x7f6eb8019240 comp rx=0 tx=0).stop 2024-03-04T07:34:26.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 -- 172.21.15.113:0/2849190585 shutdown_connections 2024-03-04T07:34:26.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec815c510 0x7f6ec814cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ec8152b20 0x7f6ec8156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ea4066f90 0x7f6ea4069450 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 --2- 172.21.15.113:0/2849190585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80c8b00 0x7f6ec8142060 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:26.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 -- 172.21.15.113:0/2849190585 >> 172.21.15.113:0/2849190585 conn(0x7f6ec80709a0 msgr2=0x7f6ec80ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:26.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 -- 172.21.15.113:0/2849190585 shutdown_connections 2024-03-04T07:34:26.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:26.901+0000 7f6ecce00640 1 -- 172.21.15.113:0/2849190585 wait complete. 2024-03-04T07:34:26.931 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-03-04T07:34:25.442656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596106,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":287,"num_write_kb":2457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223240,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751480,"statfs":{"total":773094113280,"available":772865515520,"internally_reserved":0,"allocated":3022848,"data_stored":1781001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1922,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004248"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645580+0000","last_change":"2024-03-04T07:33:56.559432+0000","last_active":"2024-03-04T07:33:58.645580+0000","last_peered":"2024-03-04T07:33:58.645580+0000","last_clean":"2024-03-04T07:33:58.645580+0000","last_became_active":"2024-03-04T07:33:56.559031+0000","last_became_peered":"2024-03-04T07:33:56.559031+0000","last_unstale":"2024-03-04T07:33:58.645580+0000","last_undegraded":"2024-03-04T07:33:58.645580+0000","last_fullsized":"2024-03-04T07:33:58.645580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:43:55.037090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750282+0000","last_change":"2024-03-04T07:33:56.546822+0000","last_active":"2024-03-04T07:33:57.750282+0000","last_peered":"2024-03-04T07:33:57.750282+0000","last_clean":"2024-03-04T07:33:57.750282+0000","last_became_active":"2024-03-04T07:33:56.546547+0000","last_became_peered":"2024-03-04T07:33:56.546547+0000","last_unstale":"2024-03-04T07:33:57.750282+0000","last_undegraded":"2024-03-04T07:33:57.750282+0000","last_fullsized":"2024-03-04T07:33:57.750282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:06.332337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859174+0000","last_change":"2024-03-04T07:33:56.530803+0000","last_active":"2024-03-04T07:33:57.859174+0000","last_peered":"2024-03-04T07:33:57.859174+0000","last_clean":"2024-03-04T07:33:57.859174+0000","last_became_active":"2024-03-04T07:33:56.530488+0000","last_became_peered":"2024-03-04T07:33:56.530488+0000","last_unstale":"2024-03-04T07:33:57.859174+0000","last_undegraded":"2024-03-04T07:33:57.859174+0000","last_fullsized":"2024-03-04T07:33:57.859174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:00:42.147951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642296+0000","last_change":"2024-03-04T07:33:56.558669+0000","last_active":"2024-03-04T07:33:58.642296+0000","last_peered":"2024-03-04T07:33:58.642296+0000","last_clean":"2024-03-04T07:33:58.642296+0000","last_became_active":"2024-03-04T07:33:56.558350+0000","last_became_peered":"2024-03-04T07:33:56.558350+0000","last_unstale":"2024-03-04T07:33:58.642296+0000","last_undegraded":"2024-03-04T07:33:58.642296+0000","last_fullsized":"2024-03-04T07:33:58.642296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:35:53.460563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644296+0000","last_change":"2024-03-04T07:33:56.540760+0000","last_active":"2024-03-04T07:33:58.644296+0000","last_peered":"2024-03-04T07:33:58.644296+0000","last_clean":"2024-03-04T07:33:58.644296+0000","last_became_active":"2024-03-04T07:33:56.540351+0000","last_became_peered":"2024-03-04T07:33:56.540351+0000","last_unstale":"2024-03-04T07:33:58.644296+0000","last_undegraded":"2024-03-04T07:33:58.644296+0000","last_fullsized":"2024-03-04T07:33:58.644296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:24:21.708228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641368+0000","last_change":"2024-03-04T07:33:56.558333+0000","last_active":"2024-03-04T07:33:58.641368+0000","last_peered":"2024-03-04T07:33:58.641368+0000","last_clean":"2024-03-04T07:33:58.641368+0000","last_became_active":"2024-03-04T07:33:56.558046+0000","last_became_peered":"2024-03-04T07:33:56.558046+0000","last_unstale":"2024-03-04T07:33:58.641368+0000","last_undegraded":"2024-03-04T07:33:58.641368+0000","last_fullsized":"2024-03-04T07:33:58.641368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:30:46.375184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643038+0000","last_change":"2024-03-04T07:33:56.543476+0000","last_active":"2024-03-04T07:33:58.643038+0000","last_peered":"2024-03-04T07:33:58.643038+0000","last_clean":"2024-03-04T07:33:58.643038+0000","last_became_active":"2024-03-04T07:33:56.543176+0000","last_became_peered":"2024-03-04T07:33:56.543176+0000","last_unstale":"2024-03-04T07:33:58.643038+0000","last_undegraded":"2024-03-04T07:33:58.643038+0000","last_fullsized":"2024-03-04T07:33:58.643038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:26:34.736870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523950+0000","last_change":"2024-03-04T07:33:56.559488+0000","last_active":"2024-03-04T07:33:57.523950+0000","last_peered":"2024-03-04T07:33:57.523950+0000","last_clean":"2024-03-04T07:33:57.523950+0000","last_became_active":"2024-03-04T07:33:56.559172+0000","last_became_peered":"2024-03-04T07:33:56.559172+0000","last_unstale":"2024-03-04T07:33:57.523950+0000","last_undegraded":"2024-03-04T07:33:57.523950+0000","last_fullsized":"2024-03-04T07:33:57.523950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:53.461697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645566+0000","last_change":"2024-03-04T07:33:56.560587+0000","last_active":"2024-03-04T07:33:58.645566+0000","last_peered":"2024-03-04T07:33:58.645566+0000","last_clean":"2024-03-04T07:33:58.645566+0000","last_became_active":"2024-03-04T07:33:56.560306+0000","last_became_peered":"2024-03-04T07:33:56.560306+0000","last_unstale":"2024-03-04T07:33:58.645566+0000","last_undegraded":"2024-03-04T07:33:58.645566+0000","last_fullsized":"2024-03-04T07:33:58.645566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:06:17.011327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641979+0000","last_change":"2024-03-04T07:33:56.560852+0000","last_active":"2024-03-04T07:33:58.641979+0000","last_peered":"2024-03-04T07:33:58.641979+0000","last_clean":"2024-03-04T07:33:58.641979+0000","last_became_active":"2024-03-04T07:33:56.560592+0000","last_became_peered":"2024-03-04T07:33:56.560592+0000","last_unstale":"2024-03-04T07:33:58.641979+0000","last_undegraded":"2024-03-04T07:33:58.641979+0000","last_fullsized":"2024-03-04T07:33:58.641979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:39:31.496432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641631+0000","last_change":"2024-03-04T07:33:56.561040+0000","last_active":"2024-03-04T07:33:58.641631+0000","last_peered":"2024-03-04T07:33:58.641631+0000","last_clean":"2024-03-04T07:33:58.641631+0000","last_became_active":"2024-03-04T07:33:56.560719+0000","last_became_peered":"2024-03-04T07:33:56.560719+0000","last_unstale":"2024-03-04T07:33:58.641631+0000","last_undegraded":"2024-03-04T07:33:58.641631+0000","last_fullsized":"2024-03-04T07:33:58.641631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:14:13.007587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526511+0000","last_change":"2024-03-04T07:33:56.550648+0000","last_active":"2024-03-04T07:33:57.526511+0000","last_peered":"2024-03-04T07:33:57.526511+0000","last_clean":"2024-03-04T07:33:57.526511+0000","last_became_active":"2024-03-04T07:33:56.549568+0000","last_became_peered":"2024-03-04T07:33:56.549568+0000","last_unstale":"2024-03-04T07:33:57.526511+0000","last_undegraded":"2024-03-04T07:33:57.526511+0000","last_fullsized":"2024-03-04T07:33:57.526511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:34:35.299480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523777+0000","last_change":"2024-03-04T07:33:56.558880+0000","last_active":"2024-03-04T07:33:57.523777+0000","last_peered":"2024-03-04T07:33:57.523777+0000","last_clean":"2024-03-04T07:33:57.523777+0000","last_became_active":"2024-03-04T07:33:56.558567+0000","last_became_peered":"2024-03-04T07:33:56.558567+0000","last_unstale":"2024-03-04T07:33:57.523777+0000","last_undegraded":"2024-03-04T07:33:57.523777+0000","last_fullsized":"2024-03-04T07:33:57.523777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:16.354787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.858969+0000","last_change":"2024-03-04T07:33:56.544222+0000","last_active":"2024-03-04T07:33:57.858969+0000","last_peered":"2024-03-04T07:33:57.858969+0000","last_clean":"2024-03-04T07:33:57.858969+0000","last_became_active":"2024-03-04T07:33:56.543936+0000","last_became_peered":"2024-03-04T07:33:56.543936+0000","last_unstale":"2024-03-04T07:33:57.858969+0000","last_undegraded":"2024-03-04T07:33:57.858969+0000","last_fullsized":"2024-03-04T07:33:57.858969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:42.788232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749980+0000","last_change":"2024-03-04T07:33:56.545195+0000","last_active":"2024-03-04T07:33:57.749980+0000","last_peered":"2024-03-04T07:33:57.749980+0000","last_clean":"2024-03-04T07:33:57.749980+0000","last_became_active":"2024-03-04T07:33:56.544837+0000","last_became_peered":"2024-03-04T07:33:56.544837+0000","last_unstale":"2024-03-04T07:33:57.749980+0000","last_undegraded":"2024-03-04T07:33:57.749980+0000","last_fullsized":"2024-03-04T07:33:57.749980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:24:59.643746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641197+0000","last_change":"2024-03-04T07:33:56.558398+0000","last_active":"2024-03-04T07:33:58.641197+0000","last_peered":"2024-03-04T07:33:58.641197+0000","last_clean":"2024-03-04T07:33:58.641197+0000","last_became_active":"2024-03-04T07:33:56.557962+0000","last_became_peered":"2024-03-04T07:33:56.557962+0000","last_unstale":"2024-03-04T07:33:58.641197+0000","last_undegraded":"2024-03-04T07:33:58.641197+0000","last_fullsized":"2024-03-04T07:33:58.641197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:21:54.588819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643019+0000","last_change":"2024-03-04T07:33:56.559597+0000","last_active":"2024-03-04T07:33:58.643019+0000","last_peered":"2024-03-04T07:33:58.643019+0000","last_clean":"2024-03-04T07:33:58.643019+0000","last_became_active":"2024-03-04T07:33:56.558719+0000","last_became_peered":"2024-03-04T07:33:56.558719+0000","last_unstale":"2024-03-04T07:33:58.643019+0000","last_undegraded":"2024-03-04T07:33:58.643019+0000","last_fullsized":"2024-03-04T07:33:58.643019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:43:56.063344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522942+0000","last_change":"2024-03-04T07:33:56.528479+0000","last_active":"2024-03-04T07:33:57.522942+0000","last_peered":"2024-03-04T07:33:57.522942+0000","last_clean":"2024-03-04T07:33:57.522942+0000","last_became_active":"2024-03-04T07:33:56.528213+0000","last_became_peered":"2024-03-04T07:33:56.528213+0000","last_unstale":"2024-03-04T07:33:57.522942+0000","last_undegraded":"2024-03-04T07:33:57.522942+0000","last_fullsized":"2024-03-04T07:33:57.522942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:38:48.062007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644805+0000","last_change":"2024-03-04T07:33:56.545890+0000","last_active":"2024-03-04T07:33:58.644805+0000","last_peered":"2024-03-04T07:33:58.644805+0000","last_clean":"2024-03-04T07:33:58.644805+0000","last_became_active":"2024-03-04T07:33:56.545467+0000","last_became_peered":"2024-03-04T07:33:56.545467+0000","last_unstale":"2024-03-04T07:33:58.644805+0000","last_undegraded":"2024-03-04T07:33:58.644805+0000","last_fullsized":"2024-03-04T07:33:58.644805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:43:56.049152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641702+0000","last_change":"2024-03-04T07:33:56.549059+0000","last_active":"2024-03-04T07:33:58.641702+0000","last_peered":"2024-03-04T07:33:58.641702+0000","last_clean":"2024-03-04T07:33:58.641702+0000","last_became_active":"2024-03-04T07:33:56.548783+0000","last_became_peered":"2024-03-04T07:33:56.548783+0000","last_unstale":"2024-03-04T07:33:58.641702+0000","last_undegraded":"2024-03-04T07:33:58.641702+0000","last_fullsized":"2024-03-04T07:33:58.641702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:02.330428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859045+0000","last_change":"2024-03-04T07:33:56.540558+0000","last_active":"2024-03-04T07:33:57.859045+0000","last_peered":"2024-03-04T07:33:57.859045+0000","last_clean":"2024-03-04T07:33:57.859045+0000","last_became_active":"2024-03-04T07:33:56.539403+0000","last_became_peered":"2024-03-04T07:33:56.539403+0000","last_unstale":"2024-03-04T07:33:57.859045+0000","last_undegraded":"2024-03-04T07:33:57.859045+0000","last_fullsized":"2024-03-04T07:33:57.859045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:28:19.413591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641763+0000","last_change":"2024-03-04T07:33:56.554982+0000","last_active":"2024-03-04T07:33:58.641763+0000","last_peered":"2024-03-04T07:33:58.641763+0000","last_clean":"2024-03-04T07:33:58.641763+0000","last_became_active":"2024-03-04T07:33:56.554470+0000","last_became_peered":"2024-03-04T07:33:56.554470+0000","last_unstale":"2024-03-04T07:33:58.641763+0000","last_undegraded":"2024-03-04T07:33:58.641763+0000","last_fullsized":"2024-03-04T07:33:58.641763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:32.767329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523496+0000","last_change":"2024-03-04T07:33:56.545465+0000","last_active":"2024-03-04T07:33:57.523496+0000","last_peered":"2024-03-04T07:33:57.523496+0000","last_clean":"2024-03-04T07:33:57.523496+0000","last_became_active":"2024-03-04T07:33:56.545270+0000","last_became_peered":"2024-03-04T07:33:56.545270+0000","last_unstale":"2024-03-04T07:33:57.523496+0000","last_undegraded":"2024-03-04T07:33:57.523496+0000","last_fullsized":"2024-03-04T07:33:57.523496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:08:45.244868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643452+0000","last_change":"2024-03-04T07:33:56.555428+0000","last_active":"2024-03-04T07:33:58.643452+0000","last_peered":"2024-03-04T07:33:58.643452+0000","last_clean":"2024-03-04T07:33:58.643452+0000","last_became_active":"2024-03-04T07:33:56.554942+0000","last_became_peered":"2024-03-04T07:33:56.554942+0000","last_unstale":"2024-03-04T07:33:58.643452+0000","last_undegraded":"2024-03-04T07:33:58.643452+0000","last_fullsized":"2024-03-04T07:33:58.643452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:04.724517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523610+0000","last_change":"2024-03-04T07:33:56.549563+0000","last_active":"2024-03-04T07:33:57.523610+0000","last_peered":"2024-03-04T07:33:57.523610+0000","last_clean":"2024-03-04T07:33:57.523610+0000","last_became_active":"2024-03-04T07:33:56.548790+0000","last_became_peered":"2024-03-04T07:33:56.548790+0000","last_unstale":"2024-03-04T07:33:57.523610+0000","last_undegraded":"2024-03-04T07:33:57.523610+0000","last_fullsized":"2024-03-04T07:33:57.523610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:30:53.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642385+0000","last_change":"2024-03-04T07:33:56.553791+0000","last_active":"2024-03-04T07:33:58.642385+0000","last_peered":"2024-03-04T07:33:58.642385+0000","last_clean":"2024-03-04T07:33:58.642385+0000","last_became_active":"2024-03-04T07:33:56.553012+0000","last_became_peered":"2024-03-04T07:33:56.553012+0000","last_unstale":"2024-03-04T07:33:58.642385+0000","last_undegraded":"2024-03-04T07:33:58.642385+0000","last_fullsized":"2024-03-04T07:33:58.642385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:38.440189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750163+0000","last_change":"2024-03-04T07:33:56.551341+0000","last_active":"2024-03-04T07:33:57.750163+0000","last_peered":"2024-03-04T07:33:57.750163+0000","last_clean":"2024-03-04T07:33:57.750163+0000","last_became_active":"2024-03-04T07:33:56.550616+0000","last_became_peered":"2024-03-04T07:33:56.550616+0000","last_unstale":"2024-03-04T07:33:57.750163+0000","last_undegraded":"2024-03-04T07:33:57.750163+0000","last_fullsized":"2024-03-04T07:33:57.750163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:35:45.801510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523840+0000","last_change":"2024-03-04T07:33:56.543903+0000","last_active":"2024-03-04T07:33:57.523840+0000","last_peered":"2024-03-04T07:33:57.523840+0000","last_clean":"2024-03-04T07:33:57.523840+0000","last_became_active":"2024-03-04T07:33:56.543632+0000","last_became_peered":"2024-03-04T07:33:56.543632+0000","last_unstale":"2024-03-04T07:33:57.523840+0000","last_undegraded":"2024-03-04T07:33:57.523840+0000","last_fullsized":"2024-03-04T07:33:57.523840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:26:10.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859232+0000","last_change":"2024-03-04T07:33:56.547091+0000","last_active":"2024-03-04T07:33:57.859232+0000","last_peered":"2024-03-04T07:33:57.859232+0000","last_clean":"2024-03-04T07:33:57.859232+0000","last_became_active":"2024-03-04T07:33:56.546652+0000","last_became_peered":"2024-03-04T07:33:56.546652+0000","last_unstale":"2024-03-04T07:33:57.859232+0000","last_undegraded":"2024-03-04T07:33:57.859232+0000","last_fullsized":"2024-03-04T07:33:57.859232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:26:03.091219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642136+0000","last_change":"2024-03-04T07:33:56.552493+0000","last_active":"2024-03-04T07:33:58.642136+0000","last_peered":"2024-03-04T07:33:58.642136+0000","last_clean":"2024-03-04T07:33:58.642136+0000","last_became_active":"2024-03-04T07:33:56.552189+0000","last_became_peered":"2024-03-04T07:33:56.552189+0000","last_unstale":"2024-03-04T07:33:58.642136+0000","last_undegraded":"2024-03-04T07:33:58.642136+0000","last_fullsized":"2024-03-04T07:33:58.642136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:10.313129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522710+0000","last_change":"2024-03-04T07:33:56.549661+0000","last_active":"2024-03-04T07:33:57.522710+0000","last_peered":"2024-03-04T07:33:57.522710+0000","last_clean":"2024-03-04T07:33:57.522710+0000","last_became_active":"2024-03-04T07:33:56.549351+0000","last_became_peered":"2024-03-04T07:33:56.549351+0000","last_unstale":"2024-03-04T07:33:57.522710+0000","last_undegraded":"2024-03-04T07:33:57.522710+0000","last_fullsized":"2024-03-04T07:33:57.522710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:55.684272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640651+0000","last_change":"2024-03-04T07:33:56.533132+0000","last_active":"2024-03-04T07:33:58.640651+0000","last_peered":"2024-03-04T07:33:58.640651+0000","last_clean":"2024-03-04T07:33:58.640651+0000","last_became_active":"2024-03-04T07:33:56.532251+0000","last_became_peered":"2024-03-04T07:33:56.532251+0000","last_unstale":"2024-03-04T07:33:58.640651+0000","last_undegraded":"2024-03-04T07:33:58.640651+0000","last_fullsized":"2024-03-04T07:33:58.640651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:17:47.018585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.525697+0000","last_change":"2024-03-04T07:33:56.543382+0000","last_active":"2024-03-04T07:33:57.525697+0000","last_peered":"2024-03-04T07:33:57.525697+0000","last_clean":"2024-03-04T07:33:57.525697+0000","last_became_active":"2024-03-04T07:33:56.542937+0000","last_became_peered":"2024-03-04T07:33:56.542937+0000","last_unstale":"2024-03-04T07:33:57.525697+0000","last_undegraded":"2024-03-04T07:33:57.525697+0000","last_fullsized":"2024-03-04T07:33:57.525697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:18:28.332590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.606616+0000","last_change":"2024-03-04T07:33:55.566130+0000","last_active":"2024-03-04T07:34:07.606616+0000","last_peered":"2024-03-04T07:34:07.606616+0000","last_clean":"2024-03-04T07:34:07.606616+0000","last_became_active":"2024-03-04T07:33:55.565894+0000","last_became_peered":"2024-03-04T07:33:55.565894+0000","last_unstale":"2024-03-04T07:34:07.606616+0000","last_undegraded":"2024-03-04T07:34:07.606616+0000","last_fullsized":"2024-03-04T07:34:07.606616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:36.530949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859235+0000","last_change":"2024-03-04T07:33:56.540113+0000","last_active":"2024-03-04T07:33:57.859235+0000","last_peered":"2024-03-04T07:33:57.859235+0000","last_clean":"2024-03-04T07:33:57.859235+0000","last_became_active":"2024-03-04T07:33:56.539738+0000","last_became_peered":"2024-03-04T07:33:56.539738+0000","last_unstale":"2024-03-04T07:33:57.859235+0000","last_undegraded":"2024-03-04T07:33:57.859235+0000","last_fullsized":"2024-03-04T07:33:57.859235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:44.949754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671613+0000","last_change":"2024-03-04T07:33:55.554956+0000","last_active":"2024-03-04T07:34:12.671613+0000","last_peered":"2024-03-04T07:34:12.671613+0000","last_clean":"2024-03-04T07:34:12.671613+0000","last_became_active":"2024-03-04T07:33:55.554151+0000","last_became_peered":"2024-03-04T07:33:55.554151+0000","last_unstale":"2024-03-04T07:34:12.671613+0000","last_undegraded":"2024-03-04T07:34:12.671613+0000","last_fullsized":"2024-03-04T07:34:12.671613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:06:19.621510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671450+0000","last_change":"2024-03-04T07:33:55.542200+0000","last_active":"2024-03-04T07:34:12.671450+0000","last_peered":"2024-03-04T07:34:12.671450+0000","last_clean":"2024-03-04T07:34:12.671450+0000","last_became_active":"2024-03-04T07:33:55.541902+0000","last_became_peered":"2024-03-04T07:33:55.541902+0000","last_unstale":"2024-03-04T07:34:12.671450+0000","last_undegraded":"2024-03-04T07:34:12.671450+0000","last_fullsized":"2024-03-04T07:34:12.671450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:55:50.251288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523733+0000","last_change":"2024-03-04T07:33:56.548380+0000","last_active":"2024-03-04T07:33:57.523733+0000","last_peered":"2024-03-04T07:33:57.523733+0000","last_clean":"2024-03-04T07:33:57.523733+0000","last_became_active":"2024-03-04T07:33:56.546087+0000","last_became_peered":"2024-03-04T07:33:56.546087+0000","last_unstale":"2024-03-04T07:33:57.523733+0000","last_undegraded":"2024-03-04T07:33:57.523733+0000","last_fullsized":"2024-03-04T07:33:57.523733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:06:43.538687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643611+0000","last_change":"2024-03-04T07:33:55.568508+0000","last_active":"2024-03-04T07:33:58.643611+0000","last_peered":"2024-03-04T07:33:58.643611+0000","last_clean":"2024-03-04T07:33:58.643611+0000","last_became_active":"2024-03-04T07:33:55.566073+0000","last_became_peered":"2024-03-04T07:33:55.566073+0000","last_unstale":"2024-03-04T07:33:58.643611+0000","last_undegraded":"2024-03-04T07:33:58.643611+0000","last_fullsized":"2024-03-04T07:33:58.643611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:34:17.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749969+0000","last_change":"2024-03-04T07:33:56.552785+0000","last_active":"2024-03-04T07:33:57.749969+0000","last_peered":"2024-03-04T07:33:57.749969+0000","last_clean":"2024-03-04T07:33:57.749969+0000","last_became_active":"2024-03-04T07:33:56.552533+0000","last_became_peered":"2024-03-04T07:33:56.552533+0000","last_unstale":"2024-03-04T07:33:57.749969+0000","last_undegraded":"2024-03-04T07:33:57.749969+0000","last_fullsized":"2024-03-04T07:33:57.749969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:08.582237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.603663+0000","last_change":"2024-03-04T07:33:55.559395+0000","last_active":"2024-03-04T07:34:07.603663+0000","last_peered":"2024-03-04T07:34:07.603663+0000","last_clean":"2024-03-04T07:34:07.603663+0000","last_became_active":"2024-03-04T07:33:55.558593+0000","last_became_peered":"2024-03-04T07:33:55.558593+0000","last_unstale":"2024-03-04T07:34:07.603663+0000","last_undegraded":"2024-03-04T07:34:07.603663+0000","last_fullsized":"2024-03-04T07:34:07.603663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:00:20.205445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642007+0000","last_change":"2024-03-04T07:33:56.553260+0000","last_active":"2024-03-04T07:33:58.642007+0000","last_peered":"2024-03-04T07:33:58.642007+0000","last_clean":"2024-03-04T07:33:58.642007+0000","last_became_active":"2024-03-04T07:33:56.552952+0000","last_became_peered":"2024-03-04T07:33:56.552952+0000","last_unstale":"2024-03-04T07:33:58.642007+0000","last_undegraded":"2024-03-04T07:33:58.642007+0000","last_fullsized":"2024-03-04T07:33:58.642007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:25:51.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.603045+0000","last_change":"2024-03-04T07:33:55.568220+0000","last_active":"2024-03-04T07:34:07.603045+0000","last_peered":"2024-03-04T07:34:07.603045+0000","last_clean":"2024-03-04T07:34:07.603045+0000","last_became_active":"2024-03-04T07:33:55.567569+0000","last_became_peered":"2024-03-04T07:33:55.567569+0000","last_unstale":"2024-03-04T07:34:07.603045+0000","last_undegraded":"2024-03-04T07:34:07.603045+0000","last_fullsized":"2024-03-04T07:34:07.603045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:40:49.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859421+0000","last_change":"2024-03-04T07:33:56.559745+0000","last_active":"2024-03-04T07:33:57.859421+0000","last_peered":"2024-03-04T07:33:57.859421+0000","last_clean":"2024-03-04T07:33:57.859421+0000","last_became_active":"2024-03-04T07:33:56.559456+0000","last_became_peered":"2024-03-04T07:33:56.559456+0000","last_unstale":"2024-03-04T07:33:57.859421+0000","last_undegraded":"2024-03-04T07:33:57.859421+0000","last_fullsized":"2024-03-04T07:33:57.859421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:18:43.135906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.532766+0000","last_change":"2024-03-04T07:33:55.550625+0000","last_active":"2024-03-04T07:34:01.532766+0000","last_peered":"2024-03-04T07:34:01.532766+0000","last_clean":"2024-03-04T07:34:01.532766+0000","last_became_active":"2024-03-04T07:33:55.550327+0000","last_became_peered":"2024-03-04T07:33:55.550327+0000","last_unstale":"2024-03-04T07:34:01.532766+0000","last_undegraded":"2024-03-04T07:34:01.532766+0000","last_fullsized":"2024-03-04T07:34:01.532766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:20.282585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526558+0000","last_change":"2024-03-04T07:33:56.537570+0000","last_active":"2024-03-04T07:33:57.526558+0000","last_peered":"2024-03-04T07:33:57.526558+0000","last_clean":"2024-03-04T07:33:57.526558+0000","last_became_active":"2024-03-04T07:33:56.536870+0000","last_became_peered":"2024-03-04T07:33:56.536870+0000","last_unstale":"2024-03-04T07:33:57.526558+0000","last_undegraded":"2024-03-04T07:33:57.526558+0000","last_fullsized":"2024-03-04T07:33:57.526558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:59:23.807432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'10","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:21.167311+0000","last_change":"2024-03-04T07:33:55.538915+0000","last_active":"2024-03-04T07:34:21.167311+0000","last_peered":"2024-03-04T07:34:21.167311+0000","last_clean":"2024-03-04T07:34:21.167311+0000","last_became_active":"2024-03-04T07:33:55.536263+0000","last_became_peered":"2024-03-04T07:33:55.536263+0000","last_unstale":"2024-03-04T07:34:21.167311+0000","last_undegraded":"2024-03-04T07:34:21.167311+0000","last_fullsized":"2024-03-04T07:34:21.167311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:09.612704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1870,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643901+0000","last_change":"2024-03-04T07:33:56.551087+0000","last_active":"2024-03-04T07:33:58.643901+0000","last_peered":"2024-03-04T07:33:58.643901+0000","last_clean":"2024-03-04T07:33:58.643901+0000","last_became_active":"2024-03-04T07:33:56.550767+0000","last_became_peered":"2024-03-04T07:33:56.550767+0000","last_unstale":"2024-03-04T07:33:58.643901+0000","last_undegraded":"2024-03-04T07:33:58.643901+0000","last_fullsized":"2024-03-04T07:33:58.643901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:15.603800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671768+0000","last_change":"2024-03-04T07:33:55.560382+0000","last_active":"2024-03-04T07:34:12.671768+0000","last_peered":"2024-03-04T07:34:12.671768+0000","last_clean":"2024-03-04T07:34:12.671768+0000","last_became_active":"2024-03-04T07:33:55.560078+0000","last_became_peered":"2024-03-04T07:33:55.560078+0000","last_unstale":"2024-03-04T07:34:12.671768+0000","last_undegraded":"2024-03-04T07:34:12.671768+0000","last_fullsized":"2024-03-04T07:34:12.671768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:41:10.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859054+0000","last_change":"2024-03-04T07:33:56.537920+0000","last_active":"2024-03-04T07:33:57.859054+0000","last_peered":"2024-03-04T07:33:57.859054+0000","last_clean":"2024-03-04T07:33:57.859054+0000","last_became_active":"2024-03-04T07:33:56.535960+0000","last_became_peered":"2024-03-04T07:33:56.535960+0000","last_unstale":"2024-03-04T07:33:57.859054+0000","last_undegraded":"2024-03-04T07:33:57.859054+0000","last_fullsized":"2024-03-04T07:33:57.859054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:37:09.344120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.604934+0000","last_change":"2024-03-04T07:33:55.567682+0000","last_active":"2024-03-04T07:34:07.604934+0000","last_peered":"2024-03-04T07:34:07.604934+0000","last_clean":"2024-03-04T07:34:07.604934+0000","last_became_active":"2024-03-04T07:33:55.566915+0000","last_became_peered":"2024-03-04T07:33:55.566915+0000","last_unstale":"2024-03-04T07:34:07.604934+0000","last_undegraded":"2024-03-04T07:34:07.604934+0000","last_fullsized":"2024-03-04T07:34:07.604934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:33:47.010346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644757+0000","last_change":"2024-03-04T07:33:56.547846+0000","last_active":"2024-03-04T07:33:58.644757+0000","last_peered":"2024-03-04T07:33:58.644757+0000","last_clean":"2024-03-04T07:33:58.644757+0000","last_became_active":"2024-03-04T07:33:56.547459+0000","last_became_peered":"2024-03-04T07:33:56.547459+0000","last_unstale":"2024-03-04T07:33:58.644757+0000","last_undegraded":"2024-03-04T07:33:58.644757+0000","last_fullsized":"2024-03-04T07:33:58.644757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:23:57.623123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.675235+0000","last_change":"2024-03-04T07:33:55.557041+0000","last_active":"2024-03-04T07:34:12.675235+0000","last_peered":"2024-03-04T07:34:12.675235+0000","last_clean":"2024-03-04T07:34:12.675235+0000","last_became_active":"2024-03-04T07:33:55.556416+0000","last_became_peered":"2024-03-04T07:33:55.556416+0000","last_unstale":"2024-03-04T07:34:12.675235+0000","last_undegraded":"2024-03-04T07:34:12.675235+0000","last_fullsized":"2024-03-04T07:34:12.675235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:31.472235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641078+0000","last_change":"2024-03-04T07:33:56.531986+0000","last_active":"2024-03-04T07:33:58.641078+0000","last_peered":"2024-03-04T07:33:58.641078+0000","last_clean":"2024-03-04T07:33:58.641078+0000","last_became_active":"2024-03-04T07:33:56.531565+0000","last_became_peered":"2024-03-04T07:33:56.531565+0000","last_unstale":"2024-03-04T07:33:58.641078+0000","last_undegraded":"2024-03-04T07:33:58.641078+0000","last_fullsized":"2024-03-04T07:33:58.641078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:42:35.040979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'15","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.675036+0000","last_change":"2024-03-04T07:33:55.538469+0000","last_active":"2024-03-04T07:34:12.675036+0000","last_peered":"2024-03-04T07:34:12.675036+0000","last_clean":"2024-03-04T07:34:12.675036+0000","last_became_active":"2024-03-04T07:33:55.538165+0000","last_became_peered":"2024-03-04T07:33:55.538165+0000","last_unstale":"2024-03-04T07:34:12.675036+0000","last_undegraded":"2024-03-04T07:34:12.675036+0000","last_fullsized":"2024-03-04T07:34:12.675036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:00:01.263768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643331+0000","last_change":"2024-03-04T07:33:56.547881+0000","last_active":"2024-03-04T07:33:58.643331+0000","last_peered":"2024-03-04T07:33:58.643331+0000","last_clean":"2024-03-04T07:33:58.643331+0000","last_became_active":"2024-03-04T07:33:56.547461+0000","last_became_peered":"2024-03-04T07:33:56.547461+0000","last_unstale":"2024-03-04T07:33:58.643331+0000","last_undegraded":"2024-03-04T07:33:58.643331+0000","last_fullsized":"2024-03-04T07:33:58.643331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:34:32.709430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.535127+0000","last_change":"2024-03-04T07:33:55.544356+0000","last_active":"2024-03-04T07:34:01.535127+0000","last_peered":"2024-03-04T07:34:01.535127+0000","last_clean":"2024-03-04T07:34:01.535127+0000","last_became_active":"2024-03-04T07:33:55.541295+0000","last_became_peered":"2024-03-04T07:33:55.541295+0000","last_unstale":"2024-03-04T07:34:01.535127+0000","last_undegraded":"2024-03-04T07:34:01.535127+0000","last_fullsized":"2024-03-04T07:34:01.535127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:22.953982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645470+0000","last_change":"2024-03-04T07:33:56.549038+0000","last_active":"2024-03-04T07:33:58.645470+0000","last_peered":"2024-03-04T07:33:58.645470+0000","last_clean":"2024-03-04T07:33:58.645470+0000","last_became_active":"2024-03-04T07:33:56.548113+0000","last_became_peered":"2024-03-04T07:33:56.548113+0000","last_unstale":"2024-03-04T07:33:58.645470+0000","last_undegraded":"2024-03-04T07:33:58.645470+0000","last_fullsized":"2024-03-04T07:33:58.645470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:53:04.601075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.530724+0000","last_change":"2024-03-04T07:33:55.538871+0000","last_active":"2024-03-04T07:34:01.530724+0000","last_peered":"2024-03-04T07:34:01.530724+0000","last_clean":"2024-03-04T07:34:01.530724+0000","last_became_active":"2024-03-04T07:33:55.537221+0000","last_became_peered":"2024-03-04T07:33:55.537221+0000","last_unstale":"2024-03-04T07:34:01.530724+0000","last_undegraded":"2024-03-04T07:34:01.530724+0000","last_fullsized":"2024-03-04T07:34:01.530724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:02:34.533330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645792+0000","last_change":"2024-03-04T07:33:56.543760+0000","last_active":"2024-03-04T07:33:58.645792+0000","last_peered":"2024-03-04T07:33:58.645792+0000","last_clean":"2024-03-04T07:33:58.645792+0000","last_became_active":"2024-03-04T07:33:56.543354+0000","last_became_peered":"2024-03-04T07:33:56.543354+0000","last_unstale":"2024-03-04T07:33:58.645792+0000","last_undegraded":"2024-03-04T07:33:58.645792+0000","last_fullsized":"2024-03-04T07:33:58.645792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:19:37.564790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.913391+0000","last_change":"2024-03-04T07:33:55.552856+0000","last_active":"2024-03-04T07:33:57.913391+0000","last_peered":"2024-03-04T07:33:57.913391+0000","last_clean":"2024-03-04T07:33:57.913391+0000","last_became_active":"2024-03-04T07:33:55.552472+0000","last_became_peered":"2024-03-04T07:33:55.552472+0000","last_unstale":"2024-03-04T07:33:57.913391+0000","last_undegraded":"2024-03-04T07:33:57.913391+0000","last_fullsized":"2024-03-04T07:33:57.913391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:36:50.409567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643132+0000","last_change":"2024-03-04T07:33:56.558497+0000","last_active":"2024-03-04T07:33:58.643132+0000","last_peered":"2024-03-04T07:33:58.643132+0000","last_clean":"2024-03-04T07:33:58.643132+0000","last_became_active":"2024-03-04T07:33:56.557989+0000","last_became_peered":"2024-03-04T07:33:56.557989+0000","last_unstale":"2024-03-04T07:33:58.643132+0000","last_undegraded":"2024-03-04T07:33:58.643132+0000","last_fullsized":"2024-03-04T07:33:58.643132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:57:05.380755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.673353+0000","last_change":"2024-03-04T07:33:55.560332+0000","last_active":"2024-03-04T07:34:12.673353+0000","last_peered":"2024-03-04T07:34:12.673353+0000","last_clean":"2024-03-04T07:34:12.673353+0000","last_became_active":"2024-03-04T07:33:55.560086+0000","last_became_peered":"2024-03-04T07:33:55.560086+0000","last_unstale":"2024-03-04T07:34:12.673353+0000","last_undegraded":"2024-03-04T07:34:12.673353+0000","last_fullsized":"2024-03-04T07:34:12.673353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:25:46.280657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644296+0000","last_change":"2024-03-04T07:33:56.559881+0000","last_active":"2024-03-04T07:33:58.644296+0000","last_peered":"2024-03-04T07:33:58.644296+0000","last_clean":"2024-03-04T07:33:58.644296+0000","last_became_active":"2024-03-04T07:33:56.559589+0000","last_became_peered":"2024-03-04T07:33:56.559589+0000","last_unstale":"2024-03-04T07:33:58.644296+0000","last_undegraded":"2024-03-04T07:33:58.644296+0000","last_fullsized":"2024-03-04T07:33:58.644296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:47:35.123163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640654+0000","last_change":"2024-03-04T07:33:43.559630+0000","last_active":"2024-03-04T07:33:58.640654+0000","last_peered":"2024-03-04T07:33:58.640654+0000","last_clean":"2024-03-04T07:33:58.640654+0000","last_became_active":"2024-03-04T07:33:43.557489+0000","last_became_peered":"2024-03-04T07:33:43.557489+0000","last_unstale":"2024-03-04T07:33:58.640654+0000","last_undegraded":"2024-03-04T07:33:58.640654+0000","last_fullsized":"2024-03-04T07:33:58.640654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_clean_scrub_stamp":"2024-03-04T07:33:42.183823+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:55:35.762857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.602970+0000","last_change":"2024-03-04T07:33:55.548616+0000","last_active":"2024-03-04T07:34:07.602970+0000","last_peered":"2024-03-04T07:34:07.602970+0000","last_clean":"2024-03-04T07:34:07.602970+0000","last_became_active":"2024-03-04T07:33:55.547322+0000","last_became_peered":"2024-03-04T07:33:55.547322+0000","last_unstale":"2024-03-04T07:34:07.602970+0000","last_undegraded":"2024-03-04T07:34:07.602970+0000","last_fullsized":"2024-03-04T07:34:07.602970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:20:16.717483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523877+0000","last_change":"2024-03-04T07:33:56.559966+0000","last_active":"2024-03-04T07:33:57.523877+0000","last_peered":"2024-03-04T07:33:57.523877+0000","last_clean":"2024-03-04T07:33:57.523877+0000","last_became_active":"2024-03-04T07:33:56.559581+0000","last_became_peered":"2024-03-04T07:33:56.559581+0000","last_unstale":"2024-03-04T07:33:57.523877+0000","last_undegraded":"2024-03-04T07:33:57.523877+0000","last_fullsized":"2024-03-04T07:33:57.523877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.670511+0000","last_change":"2024-03-04T07:33:55.563905+0000","last_active":"2024-03-04T07:34:12.670511+0000","last_peered":"2024-03-04T07:34:12.670511+0000","last_clean":"2024-03-04T07:34:12.670511+0000","last_became_active":"2024-03-04T07:33:55.563117+0000","last_became_peered":"2024-03-04T07:33:55.563117+0000","last_unstale":"2024-03-04T07:34:12.670511+0000","last_undegraded":"2024-03-04T07:34:12.670511+0000","last_fullsized":"2024-03-04T07:34:12.670511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:53:02.522757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.858965+0000","last_change":"2024-03-04T07:33:56.531080+0000","last_active":"2024-03-04T07:33:57.858965+0000","last_peered":"2024-03-04T07:33:57.858965+0000","last_clean":"2024-03-04T07:33:57.858965+0000","last_became_active":"2024-03-04T07:33:56.530776+0000","last_became_peered":"2024-03-04T07:33:56.530776+0000","last_unstale":"2024-03-04T07:33:57.858965+0000","last_undegraded":"2024-03-04T07:33:57.858965+0000","last_fullsized":"2024-03-04T07:33:57.858965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:29:14.385057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.673458+0000","last_change":"2024-03-04T07:33:55.538631+0000","last_active":"2024-03-04T07:34:12.673458+0000","last_peered":"2024-03-04T07:34:12.673458+0000","last_clean":"2024-03-04T07:34:12.673458+0000","last_became_active":"2024-03-04T07:33:55.537104+0000","last_became_peered":"2024-03-04T07:33:55.537104+0000","last_unstale":"2024-03-04T07:34:12.673458+0000","last_undegraded":"2024-03-04T07:34:12.673458+0000","last_fullsized":"2024-03-04T07:34:12.673458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:27:06.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749962+0000","last_change":"2024-03-04T07:33:56.541335+0000","last_active":"2024-03-04T07:33:57.749962+0000","last_peered":"2024-03-04T07:33:57.749962+0000","last_clean":"2024-03-04T07:33:57.749962+0000","last_became_active":"2024-03-04T07:33:56.540536+0000","last_became_peered":"2024-03-04T07:33:56.540536+0000","last_unstale":"2024-03-04T07:33:57.749962+0000","last_undegraded":"2024-03-04T07:33:57.749962+0000","last_fullsized":"2024-03-04T07:33:57.749962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:31:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642753+0000","last_change":"2024-03-04T07:33:55.574740+0000","last_active":"2024-03-04T07:33:58.642753+0000","last_peered":"2024-03-04T07:33:58.642753+0000","last_clean":"2024-03-04T07:33:58.642753+0000","last_became_active":"2024-03-04T07:33:55.574129+0000","last_became_peered":"2024-03-04T07:33:55.574129+0000","last_unstale":"2024-03-04T07:33:58.642753+0000","last_undegraded":"2024-03-04T07:33:58.642753+0000","last_fullsized":"2024-03-04T07:33:58.642753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:40:57.891803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526225+0000","last_change":"2024-03-04T07:33:56.541654+0000","last_active":"2024-03-04T07:33:57.526225+0000","last_peered":"2024-03-04T07:33:57.526225+0000","last_clean":"2024-03-04T07:33:57.526225+0000","last_became_active":"2024-03-04T07:33:56.541422+0000","last_became_peered":"2024-03-04T07:33:56.541422+0000","last_unstale":"2024-03-04T07:33:57.526225+0000","last_undegraded":"2024-03-04T07:33:57.526225+0000","last_fullsized":"2024-03-04T07:33:57.526225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:38.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750122+0000","last_change":"2024-03-04T07:33:55.539317+0000","last_active":"2024-03-04T07:33:57.750122+0000","last_peered":"2024-03-04T07:33:57.750122+0000","last_clean":"2024-03-04T07:33:57.750122+0000","last_became_active":"2024-03-04T07:33:55.539017+0000","last_became_peered":"2024-03-04T07:33:55.539017+0000","last_unstale":"2024-03-04T07:33:57.750122+0000","last_undegraded":"2024-03-04T07:33:57.750122+0000","last_fullsized":"2024-03-04T07:33:57.750122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:10.787316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642464+0000","last_change":"2024-03-04T07:33:56.554103+0000","last_active":"2024-03-04T07:33:58.642464+0000","last_peered":"2024-03-04T07:33:58.642464+0000","last_clean":"2024-03-04T07:33:58.642464+0000","last_became_active":"2024-03-04T07:33:56.553742+0000","last_became_peered":"2024-03-04T07:33:56.553742+0000","last_unstale":"2024-03-04T07:33:58.642464+0000","last_undegraded":"2024-03-04T07:33:58.642464+0000","last_fullsized":"2024-03-04T07:33:58.642464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:53.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644929+0000","last_change":"2024-03-04T07:33:55.569442+0000","last_active":"2024-03-04T07:33:58.644929+0000","last_peered":"2024-03-04T07:33:58.644929+0000","last_clean":"2024-03-04T07:33:58.644929+0000","last_became_active":"2024-03-04T07:33:55.567809+0000","last_became_peered":"2024-03-04T07:33:55.567809+0000","last_unstale":"2024-03-04T07:33:58.644929+0000","last_undegraded":"2024-03-04T07:33:58.644929+0000","last_fullsized":"2024-03-04T07:33:58.644929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:58.551866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750083+0000","last_change":"2024-03-04T07:33:56.542997+0000","last_active":"2024-03-04T07:33:57.750083+0000","last_peered":"2024-03-04T07:33:57.750083+0000","last_clean":"2024-03-04T07:33:57.750083+0000","last_became_active":"2024-03-04T07:33:56.542775+0000","last_became_peered":"2024-03-04T07:33:56.542775+0000","last_unstale":"2024-03-04T07:33:57.750083+0000","last_undegraded":"2024-03-04T07:33:57.750083+0000","last_fullsized":"2024-03-04T07:33:57.750083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:32.090568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671260+0000","last_change":"2024-03-04T07:33:55.545598+0000","last_active":"2024-03-04T07:34:12.671260+0000","last_peered":"2024-03-04T07:34:12.671260+0000","last_clean":"2024-03-04T07:34:12.671260+0000","last_became_active":"2024-03-04T07:33:55.545262+0000","last_became_peered":"2024-03-04T07:33:55.545262+0000","last_unstale":"2024-03-04T07:34:12.671260+0000","last_undegraded":"2024-03-04T07:34:12.671260+0000","last_fullsized":"2024-03-04T07:34:12.671260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:01.667243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523716+0000","last_change":"2024-03-04T07:33:56.550943+0000","last_active":"2024-03-04T07:33:57.523716+0000","last_peered":"2024-03-04T07:33:57.523716+0000","last_clean":"2024-03-04T07:33:57.523716+0000","last_became_active":"2024-03-04T07:33:56.550604+0000","last_became_peered":"2024-03-04T07:33:56.550604+0000","last_unstale":"2024-03-04T07:33:57.523716+0000","last_undegraded":"2024-03-04T07:33:57.523716+0000","last_fullsized":"2024-03-04T07:33:57.523716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:01:05.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.607065+0000","last_change":"2024-03-04T07:33:55.567220+0000","last_active":"2024-03-04T07:34:07.607065+0000","last_peered":"2024-03-04T07:34:07.607065+0000","last_clean":"2024-03-04T07:34:07.607065+0000","last_became_active":"2024-03-04T07:33:55.566600+0000","last_became_peered":"2024-03-04T07:33:55.566600+0000","last_unstale":"2024-03-04T07:34:07.607065+0000","last_undegraded":"2024-03-04T07:34:07.607065+0000","last_fullsized":"2024-03-04T07:34:07.607065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.876938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859267+0000","last_change":"2024-03-04T07:33:56.544263+0000","last_active":"2024-03-04T07:33:57.859267+0000","last_peered":"2024-03-04T07:33:57.859267+0000","last_clean":"2024-03-04T07:33:57.859267+0000","last_became_active":"2024-03-04T07:33:56.543836+0000","last_became_peered":"2024-03-04T07:33:56.543836+0000","last_unstale":"2024-03-04T07:33:57.859267+0000","last_undegraded":"2024-03-04T07:33:57.859267+0000","last_fullsized":"2024-03-04T07:33:57.859267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:49.839824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.602139+0000","last_change":"2024-03-04T07:33:55.566886+0000","last_active":"2024-03-04T07:34:07.602139+0000","last_peered":"2024-03-04T07:34:07.602139+0000","last_clean":"2024-03-04T07:34:07.602139+0000","last_became_active":"2024-03-04T07:33:55.566048+0000","last_became_peered":"2024-03-04T07:33:55.566048+0000","last_unstale":"2024-03-04T07:34:07.602139+0000","last_undegraded":"2024-03-04T07:34:07.602139+0000","last_fullsized":"2024-03-04T07:34:07.602139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:45:27.190438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.525876+0000","last_change":"2024-03-04T07:33:56.532666+0000","last_active":"2024-03-04T07:33:57.525876+0000","last_peered":"2024-03-04T07:33:57.525876+0000","last_clean":"2024-03-04T07:33:57.525876+0000","last_became_active":"2024-03-04T07:33:56.531901+0000","last_became_peered":"2024-03-04T07:33:56.531901+0000","last_unstale":"2024-03-04T07:33:57.525876+0000","last_undegraded":"2024-03-04T07:33:57.525876+0000","last_fullsized":"2024-03-04T07:33:57.525876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:42:04.545881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.602429+0000","last_change":"2024-03-04T07:33:55.557568+0000","last_active":"2024-03-04T07:34:07.602429+0000","last_peered":"2024-03-04T07:34:07.602429+0000","last_clean":"2024-03-04T07:34:07.602429+0000","last_became_active":"2024-03-04T07:33:55.557330+0000","last_became_peered":"2024-03-04T07:33:55.557330+0000","last_unstale":"2024-03-04T07:34:07.602429+0000","last_undegraded":"2024-03-04T07:34:07.602429+0000","last_fullsized":"2024-03-04T07:34:07.602429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:34.956318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523904+0000","last_change":"2024-03-04T07:33:56.552963+0000","last_active":"2024-03-04T07:33:57.523904+0000","last_peered":"2024-03-04T07:33:57.523904+0000","last_clean":"2024-03-04T07:33:57.523904+0000","last_became_active":"2024-03-04T07:33:56.552658+0000","last_became_peered":"2024-03-04T07:33:56.552658+0000","last_unstale":"2024-03-04T07:33:57.523904+0000","last_undegraded":"2024-03-04T07:33:57.523904+0000","last_fullsized":"2024-03-04T07:33:57.523904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:05.685496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.913351+0000","last_change":"2024-03-04T07:33:55.536480+0000","last_active":"2024-03-04T07:33:57.913351+0000","last_peered":"2024-03-04T07:33:57.913351+0000","last_clean":"2024-03-04T07:33:57.913351+0000","last_became_active":"2024-03-04T07:33:55.536139+0000","last_became_peered":"2024-03-04T07:33:55.536139+0000","last_unstale":"2024-03-04T07:33:57.913351+0000","last_undegraded":"2024-03-04T07:33:57.913351+0000","last_fullsized":"2024-03-04T07:33:57.913351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:12.778026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523811+0000","last_change":"2024-03-04T07:33:56.546546+0000","last_active":"2024-03-04T07:33:57.523811+0000","last_peered":"2024-03-04T07:33:57.523811+0000","last_clean":"2024-03-04T07:33:57.523811+0000","last_became_active":"2024-03-04T07:33:56.546280+0000","last_became_peered":"2024-03-04T07:33:56.546280+0000","last_unstale":"2024-03-04T07:33:57.523811+0000","last_undegraded":"2024-03-04T07:33:57.523811+0000","last_fullsized":"2024-03-04T07:33:57.523811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:19:55.950163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.603451+0000","last_change":"2024-03-04T07:33:55.572540+0000","last_active":"2024-03-04T07:34:07.603451+0000","last_peered":"2024-03-04T07:34:07.603451+0000","last_clean":"2024-03-04T07:34:07.603451+0000","last_became_active":"2024-03-04T07:33:55.571612+0000","last_became_peered":"2024-03-04T07:33:55.571612+0000","last_unstale":"2024-03-04T07:34:07.603451+0000","last_undegraded":"2024-03-04T07:34:07.603451+0000","last_fullsized":"2024-03-04T07:34:07.603451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:01:43.741423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640728+0000","last_change":"2024-03-04T07:33:56.532855+0000","last_active":"2024-03-04T07:33:58.640728+0000","last_peered":"2024-03-04T07:33:58.640728+0000","last_clean":"2024-03-04T07:33:58.640728+0000","last_became_active":"2024-03-04T07:33:56.532245+0000","last_became_peered":"2024-03-04T07:33:56.532245+0000","last_unstale":"2024-03-04T07:33:58.640728+0000","last_undegraded":"2024-03-04T07:33:58.640728+0000","last_fullsized":"2024-03-04T07:33:58.640728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:26.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642056+0000","last_change":"2024-03-04T07:33:56.563803+0000","last_active":"2024-03-04T07:33:58.642056+0000","last_peered":"2024-03-04T07:33:58.642056+0000","last_clean":"2024-03-04T07:33:58.642056+0000","last_became_active":"2024-03-04T07:33:56.563473+0000","last_became_peered":"2024-03-04T07:33:56.563473+0000","last_unstale":"2024-03-04T07:33:58.642056+0000","last_undegraded":"2024-03-04T07:33:58.642056+0000","last_fullsized":"2024-03-04T07:33:58.642056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:26.904436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.531549+0000","last_change":"2024-03-04T07:33:55.575688+0000","last_active":"2024-03-04T07:34:01.531549+0000","last_peered":"2024-03-04T07:34:01.531549+0000","last_clean":"2024-03-04T07:34:01.531549+0000","last_became_active":"2024-03-04T07:33:55.574766+0000","last_became_peered":"2024-03-04T07:33:55.574766+0000","last_unstale":"2024-03-04T07:34:01.531549+0000","last_undegraded":"2024-03-04T07:34:01.531549+0000","last_fullsized":"2024-03-04T07:34:01.531549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:32:19.708016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642750+0000","last_change":"2024-03-04T07:33:56.563798+0000","last_active":"2024-03-04T07:33:58.642750+0000","last_peered":"2024-03-04T07:33:58.642750+0000","last_clean":"2024-03-04T07:33:58.642750+0000","last_became_active":"2024-03-04T07:33:56.563474+0000","last_became_peered":"2024-03-04T07:33:56.563474+0000","last_unstale":"2024-03-04T07:33:58.642750+0000","last_undegraded":"2024-03-04T07:33:58.642750+0000","last_fullsized":"2024-03-04T07:33:58.642750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:47:39.763001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.670842+0000","last_change":"2024-03-04T07:33:55.570016+0000","last_active":"2024-03-04T07:34:12.670842+0000","last_peered":"2024-03-04T07:34:12.670842+0000","last_clean":"2024-03-04T07:34:12.670842+0000","last_became_active":"2024-03-04T07:33:55.568749+0000","last_became_peered":"2024-03-04T07:33:55.568749+0000","last_unstale":"2024-03-04T07:34:12.670842+0000","last_undegraded":"2024-03-04T07:34:12.670842+0000","last_fullsized":"2024-03-04T07:34:12.670842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:01.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859463+0000","last_change":"2024-03-04T07:33:56.562015+0000","last_active":"2024-03-04T07:33:57.859463+0000","last_peered":"2024-03-04T07:33:57.859463+0000","last_clean":"2024-03-04T07:33:57.859463+0000","last_became_active":"2024-03-04T07:33:56.561675+0000","last_became_peered":"2024-03-04T07:33:56.561675+0000","last_unstale":"2024-03-04T07:33:57.859463+0000","last_undegraded":"2024-03-04T07:33:57.859463+0000","last_fullsized":"2024-03-04T07:33:57.859463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:36:20.241414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.532052+0000","last_change":"2024-03-04T07:33:55.538337+0000","last_active":"2024-03-04T07:34:01.532052+0000","last_peered":"2024-03-04T07:34:01.532052+0000","last_clean":"2024-03-04T07:34:01.532052+0000","last_became_active":"2024-03-04T07:33:55.538010+0000","last_became_peered":"2024-03-04T07:33:55.538010+0000","last_unstale":"2024-03-04T07:34:01.532052+0000","last_undegraded":"2024-03-04T07:34:01.532052+0000","last_fullsized":"2024-03-04T07:34:01.532052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:47:15.711205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.524079+0000","last_change":"2024-03-04T07:33:56.559051+0000","last_active":"2024-03-04T07:33:57.524079+0000","last_peered":"2024-03-04T07:33:57.524079+0000","last_clean":"2024-03-04T07:33:57.524079+0000","last_became_active":"2024-03-04T07:33:56.558799+0000","last_became_peered":"2024-03-04T07:33:56.558799+0000","last_unstale":"2024-03-04T07:33:57.524079+0000","last_undegraded":"2024-03-04T07:33:57.524079+0000","last_fullsized":"2024-03-04T07:33:57.524079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:16.516631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671146+0000","last_change":"2024-03-04T07:33:55.566294+0000","last_active":"2024-03-04T07:34:12.671146+0000","last_peered":"2024-03-04T07:34:12.671146+0000","last_clean":"2024-03-04T07:34:12.671146+0000","last_became_active":"2024-03-04T07:33:55.565941+0000","last_became_peered":"2024-03-04T07:33:55.565941+0000","last_unstale":"2024-03-04T07:34:12.671146+0000","last_undegraded":"2024-03-04T07:34:12.671146+0000","last_fullsized":"2024-03-04T07:34:12.671146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:03.115085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5738,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":12291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":76926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":74473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":823296,"data_stored":665213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":76335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":74907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":74243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":74347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":819200,"data_stored":664557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T07:34:26.935 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T07:34:27.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f3661a00640 1 Processor -- start 2024-03-04T07:34:27.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f3661a00640 1 -- start start 2024-03-04T07:34:27.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f3661a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f365c152bd0 0x7f365c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f3661a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365c1591f0 con 0x7f365c1527f0 2024-03-04T07:34:27.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f3661a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365c1594d0 con 0x7f365c0c8200 2024-03-04T07:34:27.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f3661a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365c071df0 con 0x7f365c152bd0 2024-03-04T07:34:27.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f365aa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f365c1527f0 0x7f365c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39316/0 (socket says 172.21.15.113:39316) 2024-03-04T07:34:27.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f365aa00640 1 -- 172.21.15.113:0/1460122464 learned_addr learned my addr 172.21.15.113:0/1460122464 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:27.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f365b400640 1 --2- 172.21.15.113:0/1460122464 >> v2:172.21.15.136:3300/0 conn(0x7f365c152bd0 0x7f365c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f365a000640 1 -- 172.21.15.113:0/1460122464 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700589684 0 0) 0x7f365c1591f0 con 0x7f365c1527f0 2024-03-04T07:34:27.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f365a000640 1 -- 172.21.15.113:0/1460122464 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3640003530 con 0x7f365c1527f0 2024-03-04T07:34:27.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f365a000640 1 -- 172.21.15.113:0/1460122464 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033856716 0 0) 0x7f365c1594d0 con 0x7f365c0c8200 2024-03-04T07:34:27.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.105+0000 7f365a000640 1 -- 172.21.15.113:0/1460122464 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f365c1591f0 con 0x7f365c0c8200 2024-03-04T07:34:27.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.109+0000 7f365b400640 1 -- 172.21.15.113:0/1460122464 >> v1:172.21.15.136:6789/0 conn(0x7f365c0c8200 legacy=0x7f365c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.109+0000 7f365a000640 1 -- 172.21.15.113:0/1460122464 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2533652894 0 0) 0x7f3640003530 con 0x7f365c1527f0 2024-03-04T07:34:27.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.109+0000 7f365b400640 1 -- 172.21.15.113:0/1460122464 >> v1:172.21.15.113:6789/0 conn(0x7f365c1527f0 legacy=0x7f365c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.109+0000 7f365b400640 1 -- 172.21.15.113:0/1460122464 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0717a0 con 0x7f365c152bd0 2024-03-04T07:34:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.109+0000 7f365b400640 1 --2- 172.21.15.113:0/1460122464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f365c152bd0 0x7f365c070d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36540097c0 tx=0x7f365401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4879e0f5d159f867 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.109+0000 7f365a000640 1 -- 172.21.15.113:0/1460122464 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540b0070 con 0x7f365c152bd0 2024-03-04T07:34:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.109+0000 7f365a000640 1 -- 172.21.15.113:0/1460122464 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36540ab400 con 0x7f365c152bd0 2024-03-04T07:34:27.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.109+0000 7f365a000640 1 -- 172.21.15.113:0/1460122464 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c001a00 con 0x7f365c152bd0 2024-03-04T07:34:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.109+0000 7f3661a00640 1 -- 172.21.15.113:0/1460122464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f365c152bd0 msgr2=0x7f365c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.109+0000 7f3661a00640 1 --2- 172.21.15.113:0/1460122464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f365c152bd0 0x7f365c070d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f36540097c0 tx=0x7f365401c3c0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.113+0000 7f3661a00640 1 -- 172.21.15.113:0/1460122464 shutdown_connections 2024-03-04T07:34:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.113+0000 7f3661a00640 1 --2- 172.21.15.113:0/1460122464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f365c152bd0 0x7f365c070d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.113+0000 7f3661a00640 1 -- 172.21.15.113:0/1460122464 >> 172.21.15.113:0/1460122464 conn(0x7f365c06e820 msgr2=0x7f365c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.113+0000 7f3661a00640 1 -- 172.21.15.113:0/1460122464 shutdown_connections 2024-03-04T07:34:27.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.113+0000 7f3661a00640 1 -- 172.21.15.113:0/1460122464 wait complete. 2024-03-04T07:34:27.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.113+0000 7f3661a00640 1 Processor -- start 2024-03-04T07:34:27.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.113+0000 7f3661a00640 1 -- start start 2024-03-04T07:34:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3661a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f365c0c8200 0x7f365c14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3661a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f365c1527f0 0x7f365c1f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3661a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365c152bd0 0x7f365c0c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3661a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c166500 con 0x7f365c152bd0 2024-03-04T07:34:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3661a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c166380 con 0x7f365c1527f0 2024-03-04T07:34:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3661a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365c166680 con 0x7f365c0c8200 2024-03-04T07:34:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f365b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365c152bd0 0x7f365c0c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f365b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365c152bd0 0x7f365c0c59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56864/0 (socket says 172.21.15.113:56864) 2024-03-04T07:34:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f365b400640 1 -- 172.21.15.113:0/1118508221 learned_addr learned my addr 172.21.15.113:0/1118508221 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:27.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f365aa00640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f365c0c8200 0x7f365c14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3652000640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f365c1527f0 0x7f365c1f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f365b400640 1 -- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f365c0c8200 msgr2=0x7f365c14e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f365b400640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f365c0c8200 0x7f365c14e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f365b400640 1 -- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f365c1527f0 msgr2=0x7f365c1f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f365b400640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f365c1527f0 0x7f365c1f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f365b400640 1 -- 172.21.15.113:0/1118508221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0c3a70 con 0x7f365c152bd0 2024-03-04T07:34:27.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f365b400640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365c152bd0 0x7f365c0c59d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3654019ff0 tx=0x7f3654019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:27.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3653400640 1 -- 172.21.15.113:0/1118508221 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540b0040 con 0x7f365c152bd0 2024-03-04T07:34:27.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3653400640 1 -- 172.21.15.113:0/1118508221 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36540bb070 con 0x7f365c152bd0 2024-03-04T07:34:27.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3661a00640 1 -- 172.21.15.113:0/1118508221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365c0c3d50 con 0x7f365c152bd0 2024-03-04T07:34:27.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3653400640 1 -- 172.21.15.113:0/1118508221 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654003a20 con 0x7f365c152bd0 2024-03-04T07:34:27.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3661a00640 1 -- 172.21.15.113:0/1118508221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365c1f3d70 con 0x7f365c152bd0 2024-03-04T07:34:27.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.117+0000 7f3661a00640 1 -- 172.21.15.113:0/1118508221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365c075c00 con 0x7f365c152bd0 2024-03-04T07:34:27.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.121+0000 7f3653400640 1 -- 172.21.15.113:0/1118508221 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36540b3070 con 0x7f365c152bd0 2024-03-04T07:34:27.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.121+0000 7f3653400640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3638066f60 0x7f3638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.121+0000 7f3653400640 1 -- 172.21.15.113:0/1118508221 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f365411ec50 con 0x7f365c152bd0 2024-03-04T07:34:27.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.121+0000 7f365aa00640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3638066f60 0x7f3638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.121+0000 7f365aa00640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3638066f60 0x7f3638069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f364c001360 tx=0x7f364c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:27.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.125+0000 7f3653400640 1 -- 172.21.15.113:0/1118508221 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3654015fb0 con 0x7f365c152bd0 2024-03-04T07:34:27.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.313+0000 7f3661a00640 1 -- 172.21.15.113:0/1118508221 --> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f365c1584a0 con 0x7f3638066f60 2024-03-04T07:34:27.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.329+0000 7f3653400640 1 -- 172.21.15.113:0/1118508221 <== mgr.4103 v2:172.21.15.136:6832/3353359815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251931 (secure 0 0 0) 0x7f364c011370 con 0x7f3638066f60 2024-03-04T07:34:27.338 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:27.340 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-03-04T07:34:27.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.333+0000 7f3661a00640 1 -- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3638066f60 msgr2=0x7f3638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.333+0000 7f3661a00640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3638066f60 0x7f3638069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f364c001360 tx=0x7f364c009040 comp rx=0 tx=0).stop 2024-03-04T07:34:27.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.333+0000 7f3661a00640 1 -- 172.21.15.113:0/1118508221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365c152bd0 msgr2=0x7f365c0c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.333+0000 7f3661a00640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365c152bd0 0x7f365c0c59d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3654019ff0 tx=0x7f3654019160 comp rx=0 tx=0).stop 2024-03-04T07:34:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.337+0000 7f3661a00640 1 -- 172.21.15.113:0/1118508221 shutdown_connections 2024-03-04T07:34:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.337+0000 7f3661a00640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3638066f60 0x7f3638069420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.337+0000 7f3661a00640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f365c152bd0 0x7f365c0c59d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.337+0000 7f3661a00640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f365c1527f0 0x7f365c1f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.337+0000 7f3661a00640 1 --2- 172.21.15.113:0/1118508221 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f365c0c8200 0x7f365c14e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.337+0000 7f3661a00640 1 -- 172.21.15.113:0/1118508221 >> 172.21.15.113:0/1118508221 conn(0x7f365c06e820 msgr2=0x7f365c0c1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.337+0000 7f3661a00640 1 -- 172.21.15.113:0/1118508221 shutdown_connections 2024-03-04T07:34:27.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.337+0000 7f3661a00640 1 -- 172.21.15.113:0/1118508221 wait complete. 2024-03-04T07:34:27.359 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-03-04T07:34:25.442656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596106,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":287,"num_write_kb":2457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223240,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751480,"statfs":{"total":773094113280,"available":772865515520,"internally_reserved":0,"allocated":3022848,"data_stored":1781001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1922,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004248"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645580+0000","last_change":"2024-03-04T07:33:56.559432+0000","last_active":"2024-03-04T07:33:58.645580+0000","last_peered":"2024-03-04T07:33:58.645580+0000","last_clean":"2024-03-04T07:33:58.645580+0000","last_became_active":"2024-03-04T07:33:56.559031+0000","last_became_peered":"2024-03-04T07:33:56.559031+0000","last_unstale":"2024-03-04T07:33:58.645580+0000","last_undegraded":"2024-03-04T07:33:58.645580+0000","last_fullsized":"2024-03-04T07:33:58.645580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:43:55.037090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750282+0000","last_change":"2024-03-04T07:33:56.546822+0000","last_active":"2024-03-04T07:33:57.750282+0000","last_peered":"2024-03-04T07:33:57.750282+0000","last_clean":"2024-03-04T07:33:57.750282+0000","last_became_active":"2024-03-04T07:33:56.546547+0000","last_became_peered":"2024-03-04T07:33:56.546547+0000","last_unstale":"2024-03-04T07:33:57.750282+0000","last_undegraded":"2024-03-04T07:33:57.750282+0000","last_fullsized":"2024-03-04T07:33:57.750282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:06.332337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859174+0000","last_change":"2024-03-04T07:33:56.530803+0000","last_active":"2024-03-04T07:33:57.859174+0000","last_peered":"2024-03-04T07:33:57.859174+0000","last_clean":"2024-03-04T07:33:57.859174+0000","last_became_active":"2024-03-04T07:33:56.530488+0000","last_became_peered":"2024-03-04T07:33:56.530488+0000","last_unstale":"2024-03-04T07:33:57.859174+0000","last_undegraded":"2024-03-04T07:33:57.859174+0000","last_fullsized":"2024-03-04T07:33:57.859174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:00:42.147951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642296+0000","last_change":"2024-03-04T07:33:56.558669+0000","last_active":"2024-03-04T07:33:58.642296+0000","last_peered":"2024-03-04T07:33:58.642296+0000","last_clean":"2024-03-04T07:33:58.642296+0000","last_became_active":"2024-03-04T07:33:56.558350+0000","last_became_peered":"2024-03-04T07:33:56.558350+0000","last_unstale":"2024-03-04T07:33:58.642296+0000","last_undegraded":"2024-03-04T07:33:58.642296+0000","last_fullsized":"2024-03-04T07:33:58.642296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:35:53.460563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644296+0000","last_change":"2024-03-04T07:33:56.540760+0000","last_active":"2024-03-04T07:33:58.644296+0000","last_peered":"2024-03-04T07:33:58.644296+0000","last_clean":"2024-03-04T07:33:58.644296+0000","last_became_active":"2024-03-04T07:33:56.540351+0000","last_became_peered":"2024-03-04T07:33:56.540351+0000","last_unstale":"2024-03-04T07:33:58.644296+0000","last_undegraded":"2024-03-04T07:33:58.644296+0000","last_fullsized":"2024-03-04T07:33:58.644296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:24:21.708228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641368+0000","last_change":"2024-03-04T07:33:56.558333+0000","last_active":"2024-03-04T07:33:58.641368+0000","last_peered":"2024-03-04T07:33:58.641368+0000","last_clean":"2024-03-04T07:33:58.641368+0000","last_became_active":"2024-03-04T07:33:56.558046+0000","last_became_peered":"2024-03-04T07:33:56.558046+0000","last_unstale":"2024-03-04T07:33:58.641368+0000","last_undegraded":"2024-03-04T07:33:58.641368+0000","last_fullsized":"2024-03-04T07:33:58.641368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:30:46.375184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643038+0000","last_change":"2024-03-04T07:33:56.543476+0000","last_active":"2024-03-04T07:33:58.643038+0000","last_peered":"2024-03-04T07:33:58.643038+0000","last_clean":"2024-03-04T07:33:58.643038+0000","last_became_active":"2024-03-04T07:33:56.543176+0000","last_became_peered":"2024-03-04T07:33:56.543176+0000","last_unstale":"2024-03-04T07:33:58.643038+0000","last_undegraded":"2024-03-04T07:33:58.643038+0000","last_fullsized":"2024-03-04T07:33:58.643038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:26:34.736870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523950+0000","last_change":"2024-03-04T07:33:56.559488+0000","last_active":"2024-03-04T07:33:57.523950+0000","last_peered":"2024-03-04T07:33:57.523950+0000","last_clean":"2024-03-04T07:33:57.523950+0000","last_became_active":"2024-03-04T07:33:56.559172+0000","last_became_peered":"2024-03-04T07:33:56.559172+0000","last_unstale":"2024-03-04T07:33:57.523950+0000","last_undegraded":"2024-03-04T07:33:57.523950+0000","last_fullsized":"2024-03-04T07:33:57.523950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:53.461697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645566+0000","last_change":"2024-03-04T07:33:56.560587+0000","last_active":"2024-03-04T07:33:58.645566+0000","last_peered":"2024-03-04T07:33:58.645566+0000","last_clean":"2024-03-04T07:33:58.645566+0000","last_became_active":"2024-03-04T07:33:56.560306+0000","last_became_peered":"2024-03-04T07:33:56.560306+0000","last_unstale":"2024-03-04T07:33:58.645566+0000","last_undegraded":"2024-03-04T07:33:58.645566+0000","last_fullsized":"2024-03-04T07:33:58.645566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:06:17.011327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641979+0000","last_change":"2024-03-04T07:33:56.560852+0000","last_active":"2024-03-04T07:33:58.641979+0000","last_peered":"2024-03-04T07:33:58.641979+0000","last_clean":"2024-03-04T07:33:58.641979+0000","last_became_active":"2024-03-04T07:33:56.560592+0000","last_became_peered":"2024-03-04T07:33:56.560592+0000","last_unstale":"2024-03-04T07:33:58.641979+0000","last_undegraded":"2024-03-04T07:33:58.641979+0000","last_fullsized":"2024-03-04T07:33:58.641979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:39:31.496432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641631+0000","last_change":"2024-03-04T07:33:56.561040+0000","last_active":"2024-03-04T07:33:58.641631+0000","last_peered":"2024-03-04T07:33:58.641631+0000","last_clean":"2024-03-04T07:33:58.641631+0000","last_became_active":"2024-03-04T07:33:56.560719+0000","last_became_peered":"2024-03-04T07:33:56.560719+0000","last_unstale":"2024-03-04T07:33:58.641631+0000","last_undegraded":"2024-03-04T07:33:58.641631+0000","last_fullsized":"2024-03-04T07:33:58.641631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:14:13.007587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526511+0000","last_change":"2024-03-04T07:33:56.550648+0000","last_active":"2024-03-04T07:33:57.526511+0000","last_peered":"2024-03-04T07:33:57.526511+0000","last_clean":"2024-03-04T07:33:57.526511+0000","last_became_active":"2024-03-04T07:33:56.549568+0000","last_became_peered":"2024-03-04T07:33:56.549568+0000","last_unstale":"2024-03-04T07:33:57.526511+0000","last_undegraded":"2024-03-04T07:33:57.526511+0000","last_fullsized":"2024-03-04T07:33:57.526511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:34:35.299480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523777+0000","last_change":"2024-03-04T07:33:56.558880+0000","last_active":"2024-03-04T07:33:57.523777+0000","last_peered":"2024-03-04T07:33:57.523777+0000","last_clean":"2024-03-04T07:33:57.523777+0000","last_became_active":"2024-03-04T07:33:56.558567+0000","last_became_peered":"2024-03-04T07:33:56.558567+0000","last_unstale":"2024-03-04T07:33:57.523777+0000","last_undegraded":"2024-03-04T07:33:57.523777+0000","last_fullsized":"2024-03-04T07:33:57.523777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:16.354787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.858969+0000","last_change":"2024-03-04T07:33:56.544222+0000","last_active":"2024-03-04T07:33:57.858969+0000","last_peered":"2024-03-04T07:33:57.858969+0000","last_clean":"2024-03-04T07:33:57.858969+0000","last_became_active":"2024-03-04T07:33:56.543936+0000","last_became_peered":"2024-03-04T07:33:56.543936+0000","last_unstale":"2024-03-04T07:33:57.858969+0000","last_undegraded":"2024-03-04T07:33:57.858969+0000","last_fullsized":"2024-03-04T07:33:57.858969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:42.788232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749980+0000","last_change":"2024-03-04T07:33:56.545195+0000","last_active":"2024-03-04T07:33:57.749980+0000","last_peered":"2024-03-04T07:33:57.749980+0000","last_clean":"2024-03-04T07:33:57.749980+0000","last_became_active":"2024-03-04T07:33:56.544837+0000","last_became_peered":"2024-03-04T07:33:56.544837+0000","last_unstale":"2024-03-04T07:33:57.749980+0000","last_undegraded":"2024-03-04T07:33:57.749980+0000","last_fullsized":"2024-03-04T07:33:57.749980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:24:59.643746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641197+0000","last_change":"2024-03-04T07:33:56.558398+0000","last_active":"2024-03-04T07:33:58.641197+0000","last_peered":"2024-03-04T07:33:58.641197+0000","last_clean":"2024-03-04T07:33:58.641197+0000","last_became_active":"2024-03-04T07:33:56.557962+0000","last_became_peered":"2024-03-04T07:33:56.557962+0000","last_unstale":"2024-03-04T07:33:58.641197+0000","last_undegraded":"2024-03-04T07:33:58.641197+0000","last_fullsized":"2024-03-04T07:33:58.641197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:21:54.588819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643019+0000","last_change":"2024-03-04T07:33:56.559597+0000","last_active":"2024-03-04T07:33:58.643019+0000","last_peered":"2024-03-04T07:33:58.643019+0000","last_clean":"2024-03-04T07:33:58.643019+0000","last_became_active":"2024-03-04T07:33:56.558719+0000","last_became_peered":"2024-03-04T07:33:56.558719+0000","last_unstale":"2024-03-04T07:33:58.643019+0000","last_undegraded":"2024-03-04T07:33:58.643019+0000","last_fullsized":"2024-03-04T07:33:58.643019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:43:56.063344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522942+0000","last_change":"2024-03-04T07:33:56.528479+0000","last_active":"2024-03-04T07:33:57.522942+0000","last_peered":"2024-03-04T07:33:57.522942+0000","last_clean":"2024-03-04T07:33:57.522942+0000","last_became_active":"2024-03-04T07:33:56.528213+0000","last_became_peered":"2024-03-04T07:33:56.528213+0000","last_unstale":"2024-03-04T07:33:57.522942+0000","last_undegraded":"2024-03-04T07:33:57.522942+0000","last_fullsized":"2024-03-04T07:33:57.522942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:38:48.062007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644805+0000","last_change":"2024-03-04T07:33:56.545890+0000","last_active":"2024-03-04T07:33:58.644805+0000","last_peered":"2024-03-04T07:33:58.644805+0000","last_clean":"2024-03-04T07:33:58.644805+0000","last_became_active":"2024-03-04T07:33:56.545467+0000","last_became_peered":"2024-03-04T07:33:56.545467+0000","last_unstale":"2024-03-04T07:33:58.644805+0000","last_undegraded":"2024-03-04T07:33:58.644805+0000","last_fullsized":"2024-03-04T07:33:58.644805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:43:56.049152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641702+0000","last_change":"2024-03-04T07:33:56.549059+0000","last_active":"2024-03-04T07:33:58.641702+0000","last_peered":"2024-03-04T07:33:58.641702+0000","last_clean":"2024-03-04T07:33:58.641702+0000","last_became_active":"2024-03-04T07:33:56.548783+0000","last_became_peered":"2024-03-04T07:33:56.548783+0000","last_unstale":"2024-03-04T07:33:58.641702+0000","last_undegraded":"2024-03-04T07:33:58.641702+0000","last_fullsized":"2024-03-04T07:33:58.641702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:02.330428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859045+0000","last_change":"2024-03-04T07:33:56.540558+0000","last_active":"2024-03-04T07:33:57.859045+0000","last_peered":"2024-03-04T07:33:57.859045+0000","last_clean":"2024-03-04T07:33:57.859045+0000","last_became_active":"2024-03-04T07:33:56.539403+0000","last_became_peered":"2024-03-04T07:33:56.539403+0000","last_unstale":"2024-03-04T07:33:57.859045+0000","last_undegraded":"2024-03-04T07:33:57.859045+0000","last_fullsized":"2024-03-04T07:33:57.859045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:28:19.413591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641763+0000","last_change":"2024-03-04T07:33:56.554982+0000","last_active":"2024-03-04T07:33:58.641763+0000","last_peered":"2024-03-04T07:33:58.641763+0000","last_clean":"2024-03-04T07:33:58.641763+0000","last_became_active":"2024-03-04T07:33:56.554470+0000","last_became_peered":"2024-03-04T07:33:56.554470+0000","last_unstale":"2024-03-04T07:33:58.641763+0000","last_undegraded":"2024-03-04T07:33:58.641763+0000","last_fullsized":"2024-03-04T07:33:58.641763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:32.767329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523496+0000","last_change":"2024-03-04T07:33:56.545465+0000","last_active":"2024-03-04T07:33:57.523496+0000","last_peered":"2024-03-04T07:33:57.523496+0000","last_clean":"2024-03-04T07:33:57.523496+0000","last_became_active":"2024-03-04T07:33:56.545270+0000","last_became_peered":"2024-03-04T07:33:56.545270+0000","last_unstale":"2024-03-04T07:33:57.523496+0000","last_undegraded":"2024-03-04T07:33:57.523496+0000","last_fullsized":"2024-03-04T07:33:57.523496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:08:45.244868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643452+0000","last_change":"2024-03-04T07:33:56.555428+0000","last_active":"2024-03-04T07:33:58.643452+0000","last_peered":"2024-03-04T07:33:58.643452+0000","last_clean":"2024-03-04T07:33:58.643452+0000","last_became_active":"2024-03-04T07:33:56.554942+0000","last_became_peered":"2024-03-04T07:33:56.554942+0000","last_unstale":"2024-03-04T07:33:58.643452+0000","last_undegraded":"2024-03-04T07:33:58.643452+0000","last_fullsized":"2024-03-04T07:33:58.643452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:04.724517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523610+0000","last_change":"2024-03-04T07:33:56.549563+0000","last_active":"2024-03-04T07:33:57.523610+0000","last_peered":"2024-03-04T07:33:57.523610+0000","last_clean":"2024-03-04T07:33:57.523610+0000","last_became_active":"2024-03-04T07:33:56.548790+0000","last_became_peered":"2024-03-04T07:33:56.548790+0000","last_unstale":"2024-03-04T07:33:57.523610+0000","last_undegraded":"2024-03-04T07:33:57.523610+0000","last_fullsized":"2024-03-04T07:33:57.523610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:30:53.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642385+0000","last_change":"2024-03-04T07:33:56.553791+0000","last_active":"2024-03-04T07:33:58.642385+0000","last_peered":"2024-03-04T07:33:58.642385+0000","last_clean":"2024-03-04T07:33:58.642385+0000","last_became_active":"2024-03-04T07:33:56.553012+0000","last_became_peered":"2024-03-04T07:33:56.553012+0000","last_unstale":"2024-03-04T07:33:58.642385+0000","last_undegraded":"2024-03-04T07:33:58.642385+0000","last_fullsized":"2024-03-04T07:33:58.642385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:38.440189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750163+0000","last_change":"2024-03-04T07:33:56.551341+0000","last_active":"2024-03-04T07:33:57.750163+0000","last_peered":"2024-03-04T07:33:57.750163+0000","last_clean":"2024-03-04T07:33:57.750163+0000","last_became_active":"2024-03-04T07:33:56.550616+0000","last_became_peered":"2024-03-04T07:33:56.550616+0000","last_unstale":"2024-03-04T07:33:57.750163+0000","last_undegraded":"2024-03-04T07:33:57.750163+0000","last_fullsized":"2024-03-04T07:33:57.750163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:35:45.801510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523840+0000","last_change":"2024-03-04T07:33:56.543903+0000","last_active":"2024-03-04T07:33:57.523840+0000","last_peered":"2024-03-04T07:33:57.523840+0000","last_clean":"2024-03-04T07:33:57.523840+0000","last_became_active":"2024-03-04T07:33:56.543632+0000","last_became_peered":"2024-03-04T07:33:56.543632+0000","last_unstale":"2024-03-04T07:33:57.523840+0000","last_undegraded":"2024-03-04T07:33:57.523840+0000","last_fullsized":"2024-03-04T07:33:57.523840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:26:10.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859232+0000","last_change":"2024-03-04T07:33:56.547091+0000","last_active":"2024-03-04T07:33:57.859232+0000","last_peered":"2024-03-04T07:33:57.859232+0000","last_clean":"2024-03-04T07:33:57.859232+0000","last_became_active":"2024-03-04T07:33:56.546652+0000","last_became_peered":"2024-03-04T07:33:56.546652+0000","last_unstale":"2024-03-04T07:33:57.859232+0000","last_undegraded":"2024-03-04T07:33:57.859232+0000","last_fullsized":"2024-03-04T07:33:57.859232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:26:03.091219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642136+0000","last_change":"2024-03-04T07:33:56.552493+0000","last_active":"2024-03-04T07:33:58.642136+0000","last_peered":"2024-03-04T07:33:58.642136+0000","last_clean":"2024-03-04T07:33:58.642136+0000","last_became_active":"2024-03-04T07:33:56.552189+0000","last_became_peered":"2024-03-04T07:33:56.552189+0000","last_unstale":"2024-03-04T07:33:58.642136+0000","last_undegraded":"2024-03-04T07:33:58.642136+0000","last_fullsized":"2024-03-04T07:33:58.642136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:10.313129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522710+0000","last_change":"2024-03-04T07:33:56.549661+0000","last_active":"2024-03-04T07:33:57.522710+0000","last_peered":"2024-03-04T07:33:57.522710+0000","last_clean":"2024-03-04T07:33:57.522710+0000","last_became_active":"2024-03-04T07:33:56.549351+0000","last_became_peered":"2024-03-04T07:33:56.549351+0000","last_unstale":"2024-03-04T07:33:57.522710+0000","last_undegraded":"2024-03-04T07:33:57.522710+0000","last_fullsized":"2024-03-04T07:33:57.522710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:55.684272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640651+0000","last_change":"2024-03-04T07:33:56.533132+0000","last_active":"2024-03-04T07:33:58.640651+0000","last_peered":"2024-03-04T07:33:58.640651+0000","last_clean":"2024-03-04T07:33:58.640651+0000","last_became_active":"2024-03-04T07:33:56.532251+0000","last_became_peered":"2024-03-04T07:33:56.532251+0000","last_unstale":"2024-03-04T07:33:58.640651+0000","last_undegraded":"2024-03-04T07:33:58.640651+0000","last_fullsized":"2024-03-04T07:33:58.640651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:17:47.018585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.525697+0000","last_change":"2024-03-04T07:33:56.543382+0000","last_active":"2024-03-04T07:33:57.525697+0000","last_peered":"2024-03-04T07:33:57.525697+0000","last_clean":"2024-03-04T07:33:57.525697+0000","last_became_active":"2024-03-04T07:33:56.542937+0000","last_became_peered":"2024-03-04T07:33:56.542937+0000","last_unstale":"2024-03-04T07:33:57.525697+0000","last_undegraded":"2024-03-04T07:33:57.525697+0000","last_fullsized":"2024-03-04T07:33:57.525697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:18:28.332590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.606616+0000","last_change":"2024-03-04T07:33:55.566130+0000","last_active":"2024-03-04T07:34:07.606616+0000","last_peered":"2024-03-04T07:34:07.606616+0000","last_clean":"2024-03-04T07:34:07.606616+0000","last_became_active":"2024-03-04T07:33:55.565894+0000","last_became_peered":"2024-03-04T07:33:55.565894+0000","last_unstale":"2024-03-04T07:34:07.606616+0000","last_undegraded":"2024-03-04T07:34:07.606616+0000","last_fullsized":"2024-03-04T07:34:07.606616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:36.530949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859235+0000","last_change":"2024-03-04T07:33:56.540113+0000","last_active":"2024-03-04T07:33:57.859235+0000","last_peered":"2024-03-04T07:33:57.859235+0000","last_clean":"2024-03-04T07:33:57.859235+0000","last_became_active":"2024-03-04T07:33:56.539738+0000","last_became_peered":"2024-03-04T07:33:56.539738+0000","last_unstale":"2024-03-04T07:33:57.859235+0000","last_undegraded":"2024-03-04T07:33:57.859235+0000","last_fullsized":"2024-03-04T07:33:57.859235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:44.949754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671613+0000","last_change":"2024-03-04T07:33:55.554956+0000","last_active":"2024-03-04T07:34:12.671613+0000","last_peered":"2024-03-04T07:34:12.671613+0000","last_clean":"2024-03-04T07:34:12.671613+0000","last_became_active":"2024-03-04T07:33:55.554151+0000","last_became_peered":"2024-03-04T07:33:55.554151+0000","last_unstale":"2024-03-04T07:34:12.671613+0000","last_undegraded":"2024-03-04T07:34:12.671613+0000","last_fullsized":"2024-03-04T07:34:12.671613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:06:19.621510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671450+0000","last_change":"2024-03-04T07:33:55.542200+0000","last_active":"2024-03-04T07:34:12.671450+0000","last_peered":"2024-03-04T07:34:12.671450+0000","last_clean":"2024-03-04T07:34:12.671450+0000","last_became_active":"2024-03-04T07:33:55.541902+0000","last_became_peered":"2024-03-04T07:33:55.541902+0000","last_unstale":"2024-03-04T07:34:12.671450+0000","last_undegraded":"2024-03-04T07:34:12.671450+0000","last_fullsized":"2024-03-04T07:34:12.671450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:55:50.251288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523733+0000","last_change":"2024-03-04T07:33:56.548380+0000","last_active":"2024-03-04T07:33:57.523733+0000","last_peered":"2024-03-04T07:33:57.523733+0000","last_clean":"2024-03-04T07:33:57.523733+0000","last_became_active":"2024-03-04T07:33:56.546087+0000","last_became_peered":"2024-03-04T07:33:56.546087+0000","last_unstale":"2024-03-04T07:33:57.523733+0000","last_undegraded":"2024-03-04T07:33:57.523733+0000","last_fullsized":"2024-03-04T07:33:57.523733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:06:43.538687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643611+0000","last_change":"2024-03-04T07:33:55.568508+0000","last_active":"2024-03-04T07:33:58.643611+0000","last_peered":"2024-03-04T07:33:58.643611+0000","last_clean":"2024-03-04T07:33:58.643611+0000","last_became_active":"2024-03-04T07:33:55.566073+0000","last_became_peered":"2024-03-04T07:33:55.566073+0000","last_unstale":"2024-03-04T07:33:58.643611+0000","last_undegraded":"2024-03-04T07:33:58.643611+0000","last_fullsized":"2024-03-04T07:33:58.643611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:34:17.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749969+0000","last_change":"2024-03-04T07:33:56.552785+0000","last_active":"2024-03-04T07:33:57.749969+0000","last_peered":"2024-03-04T07:33:57.749969+0000","last_clean":"2024-03-04T07:33:57.749969+0000","last_became_active":"2024-03-04T07:33:56.552533+0000","last_became_peered":"2024-03-04T07:33:56.552533+0000","last_unstale":"2024-03-04T07:33:57.749969+0000","last_undegraded":"2024-03-04T07:33:57.749969+0000","last_fullsized":"2024-03-04T07:33:57.749969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:08.582237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.603663+0000","last_change":"2024-03-04T07:33:55.559395+0000","last_active":"2024-03-04T07:34:07.603663+0000","last_peered":"2024-03-04T07:34:07.603663+0000","last_clean":"2024-03-04T07:34:07.603663+0000","last_became_active":"2024-03-04T07:33:55.558593+0000","last_became_peered":"2024-03-04T07:33:55.558593+0000","last_unstale":"2024-03-04T07:34:07.603663+0000","last_undegraded":"2024-03-04T07:34:07.603663+0000","last_fullsized":"2024-03-04T07:34:07.603663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:00:20.205445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642007+0000","last_change":"2024-03-04T07:33:56.553260+0000","last_active":"2024-03-04T07:33:58.642007+0000","last_peered":"2024-03-04T07:33:58.642007+0000","last_clean":"2024-03-04T07:33:58.642007+0000","last_became_active":"2024-03-04T07:33:56.552952+0000","last_became_peered":"2024-03-04T07:33:56.552952+0000","last_unstale":"2024-03-04T07:33:58.642007+0000","last_undegraded":"2024-03-04T07:33:58.642007+0000","last_fullsized":"2024-03-04T07:33:58.642007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:25:51.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.603045+0000","last_change":"2024-03-04T07:33:55.568220+0000","last_active":"2024-03-04T07:34:07.603045+0000","last_peered":"2024-03-04T07:34:07.603045+0000","last_clean":"2024-03-04T07:34:07.603045+0000","last_became_active":"2024-03-04T07:33:55.567569+0000","last_became_peered":"2024-03-04T07:33:55.567569+0000","last_unstale":"2024-03-04T07:34:07.603045+0000","last_undegraded":"2024-03-04T07:34:07.603045+0000","last_fullsized":"2024-03-04T07:34:07.603045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:40:49.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859421+0000","last_change":"2024-03-04T07:33:56.559745+0000","last_active":"2024-03-04T07:33:57.859421+0000","last_peered":"2024-03-04T07:33:57.859421+0000","last_clean":"2024-03-04T07:33:57.859421+0000","last_became_active":"2024-03-04T07:33:56.559456+0000","last_became_peered":"2024-03-04T07:33:56.559456+0000","last_unstale":"2024-03-04T07:33:57.859421+0000","last_undegraded":"2024-03-04T07:33:57.859421+0000","last_fullsized":"2024-03-04T07:33:57.859421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:18:43.135906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.532766+0000","last_change":"2024-03-04T07:33:55.550625+0000","last_active":"2024-03-04T07:34:01.532766+0000","last_peered":"2024-03-04T07:34:01.532766+0000","last_clean":"2024-03-04T07:34:01.532766+0000","last_became_active":"2024-03-04T07:33:55.550327+0000","last_became_peered":"2024-03-04T07:33:55.550327+0000","last_unstale":"2024-03-04T07:34:01.532766+0000","last_undegraded":"2024-03-04T07:34:01.532766+0000","last_fullsized":"2024-03-04T07:34:01.532766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:20.282585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526558+0000","last_change":"2024-03-04T07:33:56.537570+0000","last_active":"2024-03-04T07:33:57.526558+0000","last_peered":"2024-03-04T07:33:57.526558+0000","last_clean":"2024-03-04T07:33:57.526558+0000","last_became_active":"2024-03-04T07:33:56.536870+0000","last_became_peered":"2024-03-04T07:33:56.536870+0000","last_unstale":"2024-03-04T07:33:57.526558+0000","last_undegraded":"2024-03-04T07:33:57.526558+0000","last_fullsized":"2024-03-04T07:33:57.526558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:59:23.807432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'10","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:21.167311+0000","last_change":"2024-03-04T07:33:55.538915+0000","last_active":"2024-03-04T07:34:21.167311+0000","last_peered":"2024-03-04T07:34:21.167311+0000","last_clean":"2024-03-04T07:34:21.167311+0000","last_became_active":"2024-03-04T07:33:55.536263+0000","last_became_peered":"2024-03-04T07:33:55.536263+0000","last_unstale":"2024-03-04T07:34:21.167311+0000","last_undegraded":"2024-03-04T07:34:21.167311+0000","last_fullsized":"2024-03-04T07:34:21.167311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:09.612704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1870,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643901+0000","last_change":"2024-03-04T07:33:56.551087+0000","last_active":"2024-03-04T07:33:58.643901+0000","last_peered":"2024-03-04T07:33:58.643901+0000","last_clean":"2024-03-04T07:33:58.643901+0000","last_became_active":"2024-03-04T07:33:56.550767+0000","last_became_peered":"2024-03-04T07:33:56.550767+0000","last_unstale":"2024-03-04T07:33:58.643901+0000","last_undegraded":"2024-03-04T07:33:58.643901+0000","last_fullsized":"2024-03-04T07:33:58.643901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:15.603800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671768+0000","last_change":"2024-03-04T07:33:55.560382+0000","last_active":"2024-03-04T07:34:12.671768+0000","last_peered":"2024-03-04T07:34:12.671768+0000","last_clean":"2024-03-04T07:34:12.671768+0000","last_became_active":"2024-03-04T07:33:55.560078+0000","last_became_peered":"2024-03-04T07:33:55.560078+0000","last_unstale":"2024-03-04T07:34:12.671768+0000","last_undegraded":"2024-03-04T07:34:12.671768+0000","last_fullsized":"2024-03-04T07:34:12.671768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:41:10.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859054+0000","last_change":"2024-03-04T07:33:56.537920+0000","last_active":"2024-03-04T07:33:57.859054+0000","last_peered":"2024-03-04T07:33:57.859054+0000","last_clean":"2024-03-04T07:33:57.859054+0000","last_became_active":"2024-03-04T07:33:56.535960+0000","last_became_peered":"2024-03-04T07:33:56.535960+0000","last_unstale":"2024-03-04T07:33:57.859054+0000","last_undegraded":"2024-03-04T07:33:57.859054+0000","last_fullsized":"2024-03-04T07:33:57.859054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:37:09.344120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.604934+0000","last_change":"2024-03-04T07:33:55.567682+0000","last_active":"2024-03-04T07:34:07.604934+0000","last_peered":"2024-03-04T07:34:07.604934+0000","last_clean":"2024-03-04T07:34:07.604934+0000","last_became_active":"2024-03-04T07:33:55.566915+0000","last_became_peered":"2024-03-04T07:33:55.566915+0000","last_unstale":"2024-03-04T07:34:07.604934+0000","last_undegraded":"2024-03-04T07:34:07.604934+0000","last_fullsized":"2024-03-04T07:34:07.604934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:33:47.010346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644757+0000","last_change":"2024-03-04T07:33:56.547846+0000","last_active":"2024-03-04T07:33:58.644757+0000","last_peered":"2024-03-04T07:33:58.644757+0000","last_clean":"2024-03-04T07:33:58.644757+0000","last_became_active":"2024-03-04T07:33:56.547459+0000","last_became_peered":"2024-03-04T07:33:56.547459+0000","last_unstale":"2024-03-04T07:33:58.644757+0000","last_undegraded":"2024-03-04T07:33:58.644757+0000","last_fullsized":"2024-03-04T07:33:58.644757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:23:57.623123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.675235+0000","last_change":"2024-03-04T07:33:55.557041+0000","last_active":"2024-03-04T07:34:12.675235+0000","last_peered":"2024-03-04T07:34:12.675235+0000","last_clean":"2024-03-04T07:34:12.675235+0000","last_became_active":"2024-03-04T07:33:55.556416+0000","last_became_peered":"2024-03-04T07:33:55.556416+0000","last_unstale":"2024-03-04T07:34:12.675235+0000","last_undegraded":"2024-03-04T07:34:12.675235+0000","last_fullsized":"2024-03-04T07:34:12.675235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:31.472235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641078+0000","last_change":"2024-03-04T07:33:56.531986+0000","last_active":"2024-03-04T07:33:58.641078+0000","last_peered":"2024-03-04T07:33:58.641078+0000","last_clean":"2024-03-04T07:33:58.641078+0000","last_became_active":"2024-03-04T07:33:56.531565+0000","last_became_peered":"2024-03-04T07:33:56.531565+0000","last_unstale":"2024-03-04T07:33:58.641078+0000","last_undegraded":"2024-03-04T07:33:58.641078+0000","last_fullsized":"2024-03-04T07:33:58.641078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:42:35.040979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'15","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.675036+0000","last_change":"2024-03-04T07:33:55.538469+0000","last_active":"2024-03-04T07:34:12.675036+0000","last_peered":"2024-03-04T07:34:12.675036+0000","last_clean":"2024-03-04T07:34:12.675036+0000","last_became_active":"2024-03-04T07:33:55.538165+0000","last_became_peered":"2024-03-04T07:33:55.538165+0000","last_unstale":"2024-03-04T07:34:12.675036+0000","last_undegraded":"2024-03-04T07:34:12.675036+0000","last_fullsized":"2024-03-04T07:34:12.675036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:00:01.263768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643331+0000","last_change":"2024-03-04T07:33:56.547881+0000","last_active":"2024-03-04T07:33:58.643331+0000","last_peered":"2024-03-04T07:33:58.643331+0000","last_clean":"2024-03-04T07:33:58.643331+0000","last_became_active":"2024-03-04T07:33:56.547461+0000","last_became_peered":"2024-03-04T07:33:56.547461+0000","last_unstale":"2024-03-04T07:33:58.643331+0000","last_undegraded":"2024-03-04T07:33:58.643331+0000","last_fullsized":"2024-03-04T07:33:58.643331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:34:32.709430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.535127+0000","last_change":"2024-03-04T07:33:55.544356+0000","last_active":"2024-03-04T07:34:01.535127+0000","last_peered":"2024-03-04T07:34:01.535127+0000","last_clean":"2024-03-04T07:34:01.535127+0000","last_became_active":"2024-03-04T07:33:55.541295+0000","last_became_peered":"2024-03-04T07:33:55.541295+0000","last_unstale":"2024-03-04T07:34:01.535127+0000","last_undegraded":"2024-03-04T07:34:01.535127+0000","last_fullsized":"2024-03-04T07:34:01.535127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:22.953982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645470+0000","last_change":"2024-03-04T07:33:56.549038+0000","last_active":"2024-03-04T07:33:58.645470+0000","last_peered":"2024-03-04T07:33:58.645470+0000","last_clean":"2024-03-04T07:33:58.645470+0000","last_became_active":"2024-03-04T07:33:56.548113+0000","last_became_peered":"2024-03-04T07:33:56.548113+0000","last_unstale":"2024-03-04T07:33:58.645470+0000","last_undegraded":"2024-03-04T07:33:58.645470+0000","last_fullsized":"2024-03-04T07:33:58.645470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:53:04.601075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.530724+0000","last_change":"2024-03-04T07:33:55.538871+0000","last_active":"2024-03-04T07:34:01.530724+0000","last_peered":"2024-03-04T07:34:01.530724+0000","last_clean":"2024-03-04T07:34:01.530724+0000","last_became_active":"2024-03-04T07:33:55.537221+0000","last_became_peered":"2024-03-04T07:33:55.537221+0000","last_unstale":"2024-03-04T07:34:01.530724+0000","last_undegraded":"2024-03-04T07:34:01.530724+0000","last_fullsized":"2024-03-04T07:34:01.530724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:02:34.533330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645792+0000","last_change":"2024-03-04T07:33:56.543760+0000","last_active":"2024-03-04T07:33:58.645792+0000","last_peered":"2024-03-04T07:33:58.645792+0000","last_clean":"2024-03-04T07:33:58.645792+0000","last_became_active":"2024-03-04T07:33:56.543354+0000","last_became_peered":"2024-03-04T07:33:56.543354+0000","last_unstale":"2024-03-04T07:33:58.645792+0000","last_undegraded":"2024-03-04T07:33:58.645792+0000","last_fullsized":"2024-03-04T07:33:58.645792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:19:37.564790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.913391+0000","last_change":"2024-03-04T07:33:55.552856+0000","last_active":"2024-03-04T07:33:57.913391+0000","last_peered":"2024-03-04T07:33:57.913391+0000","last_clean":"2024-03-04T07:33:57.913391+0000","last_became_active":"2024-03-04T07:33:55.552472+0000","last_became_peered":"2024-03-04T07:33:55.552472+0000","last_unstale":"2024-03-04T07:33:57.913391+0000","last_undegraded":"2024-03-04T07:33:57.913391+0000","last_fullsized":"2024-03-04T07:33:57.913391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:36:50.409567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643132+0000","last_change":"2024-03-04T07:33:56.558497+0000","last_active":"2024-03-04T07:33:58.643132+0000","last_peered":"2024-03-04T07:33:58.643132+0000","last_clean":"2024-03-04T07:33:58.643132+0000","last_became_active":"2024-03-04T07:33:56.557989+0000","last_became_peered":"2024-03-04T07:33:56.557989+0000","last_unstale":"2024-03-04T07:33:58.643132+0000","last_undegraded":"2024-03-04T07:33:58.643132+0000","last_fullsized":"2024-03-04T07:33:58.643132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:57:05.380755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.673353+0000","last_change":"2024-03-04T07:33:55.560332+0000","last_active":"2024-03-04T07:34:12.673353+0000","last_peered":"2024-03-04T07:34:12.673353+0000","last_clean":"2024-03-04T07:34:12.673353+0000","last_became_active":"2024-03-04T07:33:55.560086+0000","last_became_peered":"2024-03-04T07:33:55.560086+0000","last_unstale":"2024-03-04T07:34:12.673353+0000","last_undegraded":"2024-03-04T07:34:12.673353+0000","last_fullsized":"2024-03-04T07:34:12.673353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:25:46.280657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644296+0000","last_change":"2024-03-04T07:33:56.559881+0000","last_active":"2024-03-04T07:33:58.644296+0000","last_peered":"2024-03-04T07:33:58.644296+0000","last_clean":"2024-03-04T07:33:58.644296+0000","last_became_active":"2024-03-04T07:33:56.559589+0000","last_became_peered":"2024-03-04T07:33:56.559589+0000","last_unstale":"2024-03-04T07:33:58.644296+0000","last_undegraded":"2024-03-04T07:33:58.644296+0000","last_fullsized":"2024-03-04T07:33:58.644296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:47:35.123163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640654+0000","last_change":"2024-03-04T07:33:43.559630+0000","last_active":"2024-03-04T07:33:58.640654+0000","last_peered":"2024-03-04T07:33:58.640654+0000","last_clean":"2024-03-04T07:33:58.640654+0000","last_became_active":"2024-03-04T07:33:43.557489+0000","last_became_peered":"2024-03-04T07:33:43.557489+0000","last_unstale":"2024-03-04T07:33:58.640654+0000","last_undegraded":"2024-03-04T07:33:58.640654+0000","last_fullsized":"2024-03-04T07:33:58.640654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_clean_scrub_stamp":"2024-03-04T07:33:42.183823+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:55:35.762857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.602970+0000","last_change":"2024-03-04T07:33:55.548616+0000","last_active":"2024-03-04T07:34:07.602970+0000","last_peered":"2024-03-04T07:34:07.602970+0000","last_clean":"2024-03-04T07:34:07.602970+0000","last_became_active":"2024-03-04T07:33:55.547322+0000","last_became_peered":"2024-03-04T07:33:55.547322+0000","last_unstale":"2024-03-04T07:34:07.602970+0000","last_undegraded":"2024-03-04T07:34:07.602970+0000","last_fullsized":"2024-03-04T07:34:07.602970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:20:16.717483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523877+0000","last_change":"2024-03-04T07:33:56.559966+0000","last_active":"2024-03-04T07:33:57.523877+0000","last_peered":"2024-03-04T07:33:57.523877+0000","last_clean":"2024-03-04T07:33:57.523877+0000","last_became_active":"2024-03-04T07:33:56.559581+0000","last_became_peered":"2024-03-04T07:33:56.559581+0000","last_unstale":"2024-03-04T07:33:57.523877+0000","last_undegraded":"2024-03-04T07:33:57.523877+0000","last_fullsized":"2024-03-04T07:33:57.523877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.670511+0000","last_change":"2024-03-04T07:33:55.563905+0000","last_active":"2024-03-04T07:34:12.670511+0000","last_peered":"2024-03-04T07:34:12.670511+0000","last_clean":"2024-03-04T07:34:12.670511+0000","last_became_active":"2024-03-04T07:33:55.563117+0000","last_became_peered":"2024-03-04T07:33:55.563117+0000","last_unstale":"2024-03-04T07:34:12.670511+0000","last_undegraded":"2024-03-04T07:34:12.670511+0000","last_fullsized":"2024-03-04T07:34:12.670511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:53:02.522757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.858965+0000","last_change":"2024-03-04T07:33:56.531080+0000","last_active":"2024-03-04T07:33:57.858965+0000","last_peered":"2024-03-04T07:33:57.858965+0000","last_clean":"2024-03-04T07:33:57.858965+0000","last_became_active":"2024-03-04T07:33:56.530776+0000","last_became_peered":"2024-03-04T07:33:56.530776+0000","last_unstale":"2024-03-04T07:33:57.858965+0000","last_undegraded":"2024-03-04T07:33:57.858965+0000","last_fullsized":"2024-03-04T07:33:57.858965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:29:14.385057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.673458+0000","last_change":"2024-03-04T07:33:55.538631+0000","last_active":"2024-03-04T07:34:12.673458+0000","last_peered":"2024-03-04T07:34:12.673458+0000","last_clean":"2024-03-04T07:34:12.673458+0000","last_became_active":"2024-03-04T07:33:55.537104+0000","last_became_peered":"2024-03-04T07:33:55.537104+0000","last_unstale":"2024-03-04T07:34:12.673458+0000","last_undegraded":"2024-03-04T07:34:12.673458+0000","last_fullsized":"2024-03-04T07:34:12.673458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:27:06.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749962+0000","last_change":"2024-03-04T07:33:56.541335+0000","last_active":"2024-03-04T07:33:57.749962+0000","last_peered":"2024-03-04T07:33:57.749962+0000","last_clean":"2024-03-04T07:33:57.749962+0000","last_became_active":"2024-03-04T07:33:56.540536+0000","last_became_peered":"2024-03-04T07:33:56.540536+0000","last_unstale":"2024-03-04T07:33:57.749962+0000","last_undegraded":"2024-03-04T07:33:57.749962+0000","last_fullsized":"2024-03-04T07:33:57.749962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:31:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642753+0000","last_change":"2024-03-04T07:33:55.574740+0000","last_active":"2024-03-04T07:33:58.642753+0000","last_peered":"2024-03-04T07:33:58.642753+0000","last_clean":"2024-03-04T07:33:58.642753+0000","last_became_active":"2024-03-04T07:33:55.574129+0000","last_became_peered":"2024-03-04T07:33:55.574129+0000","last_unstale":"2024-03-04T07:33:58.642753+0000","last_undegraded":"2024-03-04T07:33:58.642753+0000","last_fullsized":"2024-03-04T07:33:58.642753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:40:57.891803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526225+0000","last_change":"2024-03-04T07:33:56.541654+0000","last_active":"2024-03-04T07:33:57.526225+0000","last_peered":"2024-03-04T07:33:57.526225+0000","last_clean":"2024-03-04T07:33:57.526225+0000","last_became_active":"2024-03-04T07:33:56.541422+0000","last_became_peered":"2024-03-04T07:33:56.541422+0000","last_unstale":"2024-03-04T07:33:57.526225+0000","last_undegraded":"2024-03-04T07:33:57.526225+0000","last_fullsized":"2024-03-04T07:33:57.526225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:38.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750122+0000","last_change":"2024-03-04T07:33:55.539317+0000","last_active":"2024-03-04T07:33:57.750122+0000","last_peered":"2024-03-04T07:33:57.750122+0000","last_clean":"2024-03-04T07:33:57.750122+0000","last_became_active":"2024-03-04T07:33:55.539017+0000","last_became_peered":"2024-03-04T07:33:55.539017+0000","last_unstale":"2024-03-04T07:33:57.750122+0000","last_undegraded":"2024-03-04T07:33:57.750122+0000","last_fullsized":"2024-03-04T07:33:57.750122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:10.787316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642464+0000","last_change":"2024-03-04T07:33:56.554103+0000","last_active":"2024-03-04T07:33:58.642464+0000","last_peered":"2024-03-04T07:33:58.642464+0000","last_clean":"2024-03-04T07:33:58.642464+0000","last_became_active":"2024-03-04T07:33:56.553742+0000","last_became_peered":"2024-03-04T07:33:56.553742+0000","last_unstale":"2024-03-04T07:33:58.642464+0000","last_undegraded":"2024-03-04T07:33:58.642464+0000","last_fullsized":"2024-03-04T07:33:58.642464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:53.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644929+0000","last_change":"2024-03-04T07:33:55.569442+0000","last_active":"2024-03-04T07:33:58.644929+0000","last_peered":"2024-03-04T07:33:58.644929+0000","last_clean":"2024-03-04T07:33:58.644929+0000","last_became_active":"2024-03-04T07:33:55.567809+0000","last_became_peered":"2024-03-04T07:33:55.567809+0000","last_unstale":"2024-03-04T07:33:58.644929+0000","last_undegraded":"2024-03-04T07:33:58.644929+0000","last_fullsized":"2024-03-04T07:33:58.644929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:58.551866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750083+0000","last_change":"2024-03-04T07:33:56.542997+0000","last_active":"2024-03-04T07:33:57.750083+0000","last_peered":"2024-03-04T07:33:57.750083+0000","last_clean":"2024-03-04T07:33:57.750083+0000","last_became_active":"2024-03-04T07:33:56.542775+0000","last_became_peered":"2024-03-04T07:33:56.542775+0000","last_unstale":"2024-03-04T07:33:57.750083+0000","last_undegraded":"2024-03-04T07:33:57.750083+0000","last_fullsized":"2024-03-04T07:33:57.750083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:32.090568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671260+0000","last_change":"2024-03-04T07:33:55.545598+0000","last_active":"2024-03-04T07:34:12.671260+0000","last_peered":"2024-03-04T07:34:12.671260+0000","last_clean":"2024-03-04T07:34:12.671260+0000","last_became_active":"2024-03-04T07:33:55.545262+0000","last_became_peered":"2024-03-04T07:33:55.545262+0000","last_unstale":"2024-03-04T07:34:12.671260+0000","last_undegraded":"2024-03-04T07:34:12.671260+0000","last_fullsized":"2024-03-04T07:34:12.671260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:01.667243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523716+0000","last_change":"2024-03-04T07:33:56.550943+0000","last_active":"2024-03-04T07:33:57.523716+0000","last_peered":"2024-03-04T07:33:57.523716+0000","last_clean":"2024-03-04T07:33:57.523716+0000","last_became_active":"2024-03-04T07:33:56.550604+0000","last_became_peered":"2024-03-04T07:33:56.550604+0000","last_unstale":"2024-03-04T07:33:57.523716+0000","last_undegraded":"2024-03-04T07:33:57.523716+0000","last_fullsized":"2024-03-04T07:33:57.523716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:01:05.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.607065+0000","last_change":"2024-03-04T07:33:55.567220+0000","last_active":"2024-03-04T07:34:07.607065+0000","last_peered":"2024-03-04T07:34:07.607065+0000","last_clean":"2024-03-04T07:34:07.607065+0000","last_became_active":"2024-03-04T07:33:55.566600+0000","last_became_peered":"2024-03-04T07:33:55.566600+0000","last_unstale":"2024-03-04T07:34:07.607065+0000","last_undegraded":"2024-03-04T07:34:07.607065+0000","last_fullsized":"2024-03-04T07:34:07.607065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.876938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859267+0000","last_change":"2024-03-04T07:33:56.544263+0000","last_active":"2024-03-04T07:33:57.859267+0000","last_peered":"2024-03-04T07:33:57.859267+0000","last_clean":"2024-03-04T07:33:57.859267+0000","last_became_active":"2024-03-04T07:33:56.543836+0000","last_became_peered":"2024-03-04T07:33:56.543836+0000","last_unstale":"2024-03-04T07:33:57.859267+0000","last_undegraded":"2024-03-04T07:33:57.859267+0000","last_fullsized":"2024-03-04T07:33:57.859267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:49.839824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.602139+0000","last_change":"2024-03-04T07:33:55.566886+0000","last_active":"2024-03-04T07:34:07.602139+0000","last_peered":"2024-03-04T07:34:07.602139+0000","last_clean":"2024-03-04T07:34:07.602139+0000","last_became_active":"2024-03-04T07:33:55.566048+0000","last_became_peered":"2024-03-04T07:33:55.566048+0000","last_unstale":"2024-03-04T07:34:07.602139+0000","last_undegraded":"2024-03-04T07:34:07.602139+0000","last_fullsized":"2024-03-04T07:34:07.602139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:45:27.190438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.525876+0000","last_change":"2024-03-04T07:33:56.532666+0000","last_active":"2024-03-04T07:33:57.525876+0000","last_peered":"2024-03-04T07:33:57.525876+0000","last_clean":"2024-03-04T07:33:57.525876+0000","last_became_active":"2024-03-04T07:33:56.531901+0000","last_became_peered":"2024-03-04T07:33:56.531901+0000","last_unstale":"2024-03-04T07:33:57.525876+0000","last_undegraded":"2024-03-04T07:33:57.525876+0000","last_fullsized":"2024-03-04T07:33:57.525876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:42:04.545881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.602429+0000","last_change":"2024-03-04T07:33:55.557568+0000","last_active":"2024-03-04T07:34:07.602429+0000","last_peered":"2024-03-04T07:34:07.602429+0000","last_clean":"2024-03-04T07:34:07.602429+0000","last_became_active":"2024-03-04T07:33:55.557330+0000","last_became_peered":"2024-03-04T07:33:55.557330+0000","last_unstale":"2024-03-04T07:34:07.602429+0000","last_undegraded":"2024-03-04T07:34:07.602429+0000","last_fullsized":"2024-03-04T07:34:07.602429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:34.956318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523904+0000","last_change":"2024-03-04T07:33:56.552963+0000","last_active":"2024-03-04T07:33:57.523904+0000","last_peered":"2024-03-04T07:33:57.523904+0000","last_clean":"2024-03-04T07:33:57.523904+0000","last_became_active":"2024-03-04T07:33:56.552658+0000","last_became_peered":"2024-03-04T07:33:56.552658+0000","last_unstale":"2024-03-04T07:33:57.523904+0000","last_undegraded":"2024-03-04T07:33:57.523904+0000","last_fullsized":"2024-03-04T07:33:57.523904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:05.685496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.913351+0000","last_change":"2024-03-04T07:33:55.536480+0000","last_active":"2024-03-04T07:33:57.913351+0000","last_peered":"2024-03-04T07:33:57.913351+0000","last_clean":"2024-03-04T07:33:57.913351+0000","last_became_active":"2024-03-04T07:33:55.536139+0000","last_became_peered":"2024-03-04T07:33:55.536139+0000","last_unstale":"2024-03-04T07:33:57.913351+0000","last_undegraded":"2024-03-04T07:33:57.913351+0000","last_fullsized":"2024-03-04T07:33:57.913351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:12.778026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523811+0000","last_change":"2024-03-04T07:33:56.546546+0000","last_active":"2024-03-04T07:33:57.523811+0000","last_peered":"2024-03-04T07:33:57.523811+0000","last_clean":"2024-03-04T07:33:57.523811+0000","last_became_active":"2024-03-04T07:33:56.546280+0000","last_became_peered":"2024-03-04T07:33:56.546280+0000","last_unstale":"2024-03-04T07:33:57.523811+0000","last_undegraded":"2024-03-04T07:33:57.523811+0000","last_fullsized":"2024-03-04T07:33:57.523811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:19:55.950163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.603451+0000","last_change":"2024-03-04T07:33:55.572540+0000","last_active":"2024-03-04T07:34:07.603451+0000","last_peered":"2024-03-04T07:34:07.603451+0000","last_clean":"2024-03-04T07:34:07.603451+0000","last_became_active":"2024-03-04T07:33:55.571612+0000","last_became_peered":"2024-03-04T07:33:55.571612+0000","last_unstale":"2024-03-04T07:34:07.603451+0000","last_undegraded":"2024-03-04T07:34:07.603451+0000","last_fullsized":"2024-03-04T07:34:07.603451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:01:43.741423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640728+0000","last_change":"2024-03-04T07:33:56.532855+0000","last_active":"2024-03-04T07:33:58.640728+0000","last_peered":"2024-03-04T07:33:58.640728+0000","last_clean":"2024-03-04T07:33:58.640728+0000","last_became_active":"2024-03-04T07:33:56.532245+0000","last_became_peered":"2024-03-04T07:33:56.532245+0000","last_unstale":"2024-03-04T07:33:58.640728+0000","last_undegraded":"2024-03-04T07:33:58.640728+0000","last_fullsized":"2024-03-04T07:33:58.640728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:26.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642056+0000","last_change":"2024-03-04T07:33:56.563803+0000","last_active":"2024-03-04T07:33:58.642056+0000","last_peered":"2024-03-04T07:33:58.642056+0000","last_clean":"2024-03-04T07:33:58.642056+0000","last_became_active":"2024-03-04T07:33:56.563473+0000","last_became_peered":"2024-03-04T07:33:56.563473+0000","last_unstale":"2024-03-04T07:33:58.642056+0000","last_undegraded":"2024-03-04T07:33:58.642056+0000","last_fullsized":"2024-03-04T07:33:58.642056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:26.904436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.531549+0000","last_change":"2024-03-04T07:33:55.575688+0000","last_active":"2024-03-04T07:34:01.531549+0000","last_peered":"2024-03-04T07:34:01.531549+0000","last_clean":"2024-03-04T07:34:01.531549+0000","last_became_active":"2024-03-04T07:33:55.574766+0000","last_became_peered":"2024-03-04T07:33:55.574766+0000","last_unstale":"2024-03-04T07:34:01.531549+0000","last_undegraded":"2024-03-04T07:34:01.531549+0000","last_fullsized":"2024-03-04T07:34:01.531549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:32:19.708016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642750+0000","last_change":"2024-03-04T07:33:56.563798+0000","last_active":"2024-03-04T07:33:58.642750+0000","last_peered":"2024-03-04T07:33:58.642750+0000","last_clean":"2024-03-04T07:33:58.642750+0000","last_became_active":"2024-03-04T07:33:56.563474+0000","last_became_peered":"2024-03-04T07:33:56.563474+0000","last_unstale":"2024-03-04T07:33:58.642750+0000","last_undegraded":"2024-03-04T07:33:58.642750+0000","last_fullsized":"2024-03-04T07:33:58.642750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:47:39.763001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.670842+0000","last_change":"2024-03-04T07:33:55.570016+0000","last_active":"2024-03-04T07:34:12.670842+0000","last_peered":"2024-03-04T07:34:12.670842+0000","last_clean":"2024-03-04T07:34:12.670842+0000","last_became_active":"2024-03-04T07:33:55.568749+0000","last_became_peered":"2024-03-04T07:33:55.568749+0000","last_unstale":"2024-03-04T07:34:12.670842+0000","last_undegraded":"2024-03-04T07:34:12.670842+0000","last_fullsized":"2024-03-04T07:34:12.670842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:01.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859463+0000","last_change":"2024-03-04T07:33:56.562015+0000","last_active":"2024-03-04T07:33:57.859463+0000","last_peered":"2024-03-04T07:33:57.859463+0000","last_clean":"2024-03-04T07:33:57.859463+0000","last_became_active":"2024-03-04T07:33:56.561675+0000","last_became_peered":"2024-03-04T07:33:56.561675+0000","last_unstale":"2024-03-04T07:33:57.859463+0000","last_undegraded":"2024-03-04T07:33:57.859463+0000","last_fullsized":"2024-03-04T07:33:57.859463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:36:20.241414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.532052+0000","last_change":"2024-03-04T07:33:55.538337+0000","last_active":"2024-03-04T07:34:01.532052+0000","last_peered":"2024-03-04T07:34:01.532052+0000","last_clean":"2024-03-04T07:34:01.532052+0000","last_became_active":"2024-03-04T07:33:55.538010+0000","last_became_peered":"2024-03-04T07:33:55.538010+0000","last_unstale":"2024-03-04T07:34:01.532052+0000","last_undegraded":"2024-03-04T07:34:01.532052+0000","last_fullsized":"2024-03-04T07:34:01.532052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:47:15.711205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.524079+0000","last_change":"2024-03-04T07:33:56.559051+0000","last_active":"2024-03-04T07:33:57.524079+0000","last_peered":"2024-03-04T07:33:57.524079+0000","last_clean":"2024-03-04T07:33:57.524079+0000","last_became_active":"2024-03-04T07:33:56.558799+0000","last_became_peered":"2024-03-04T07:33:56.558799+0000","last_unstale":"2024-03-04T07:33:57.524079+0000","last_undegraded":"2024-03-04T07:33:57.524079+0000","last_fullsized":"2024-03-04T07:33:57.524079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:16.516631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671146+0000","last_change":"2024-03-04T07:33:55.566294+0000","last_active":"2024-03-04T07:34:12.671146+0000","last_peered":"2024-03-04T07:34:12.671146+0000","last_clean":"2024-03-04T07:34:12.671146+0000","last_became_active":"2024-03-04T07:33:55.565941+0000","last_became_peered":"2024-03-04T07:33:55.565941+0000","last_unstale":"2024-03-04T07:34:12.671146+0000","last_undegraded":"2024-03-04T07:34:12.671146+0000","last_fullsized":"2024-03-04T07:34:12.671146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:03.115085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5738,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":12291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112,"ondisk_log_size":112,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":76926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":74473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":823296,"data_stored":665213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542155,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":76335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":74907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":74243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542156,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":74347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":819200,"data_stored":664557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T07:34:27.363 INFO:tasks.mds_thrash.ceph_manager:clean! 2024-03-04T07:34:27.363 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T07:34:27.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.541+0000 7f442be00640 1 Processor -- start 2024-03-04T07:34:27.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.541+0000 7f442be00640 1 -- start start 2024-03-04T07:34:27.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.541+0000 7f442be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f442c1528b0 0x7f442c15c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f442c158bb0 0x7f442c159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442be00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442c15cbf0 con 0x7f442c0c6b20 2024-03-04T07:34:27.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442be00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f442c159560 con 0x7f442c1528b0 2024-03-04T07:34:27.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442c06ce20 con 0x7f442c158bb0 2024-03-04T07:34:27.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f442c1528b0 0x7f442c15c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f442c1528b0 0x7f442c15c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46636/0 (socket says 172.21.15.113:46636) 2024-03-04T07:34:27.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442aa00640 1 -- 172.21.15.113:0/548872913 learned_addr learned my addr 172.21.15.113:0/548872913 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:27.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442b400640 1 --2- 172.21.15.113:0/548872913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f442c158bb0 0x7f442c159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442aa00640 1 -- 172.21.15.113:0/548872913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f442c158bb0 msgr2=0x7f442c159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:27.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442aa00640 1 --2- 172.21.15.113:0/548872913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f442c158bb0 0x7f442c159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442aa00640 1 -- 172.21.15.113:0/548872913 >> v1:172.21.15.136:6789/0 conn(0x7f442c0c6b20 legacy=0x7f442c0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.545+0000 7f442aa00640 1 -- 172.21.15.113:0/548872913 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442c1586b0 con 0x7f442c1528b0 2024-03-04T07:34:27.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f442aa00640 1 --2- 172.21.15.113:0/548872913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 0x7f442c15c6b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f441c009870 tx=0x7f441c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900fb0eb2c13c43e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:27.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f4429600640 1 -- 172.21.15.113:0/548872913 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c0b5070 con 0x7f442c1528b0 2024-03-04T07:34:27.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f4429600640 1 -- 172.21.15.113:0/548872913 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c0a8a50 con 0x7f442c1528b0 2024-03-04T07:34:27.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f4429600640 1 -- 172.21.15.113:0/548872913 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c0a8e20 con 0x7f442c1528b0 2024-03-04T07:34:27.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f442be00640 1 -- 172.21.15.113:0/548872913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 msgr2=0x7f442c15c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f442be00640 1 --2- 172.21.15.113:0/548872913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 0x7f442c15c6b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f441c009870 tx=0x7f441c01ed50 comp rx=0 tx=0).stop 2024-03-04T07:34:27.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f442be00640 1 -- 172.21.15.113:0/548872913 shutdown_connections 2024-03-04T07:34:27.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f442be00640 1 --2- 172.21.15.113:0/548872913 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f442c158bb0 0x7f442c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f442be00640 1 --2- 172.21.15.113:0/548872913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 0x7f442c15c6b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f442be00640 1 -- 172.21.15.113:0/548872913 >> 172.21.15.113:0/548872913 conn(0x7f442c06e9a0 msgr2=0x7f442c06edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:27.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f442be00640 1 -- 172.21.15.113:0/548872913 shutdown_connections 2024-03-04T07:34:27.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.549+0000 7f442be00640 1 -- 172.21.15.113:0/548872913 wait complete. 2024-03-04T07:34:27.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442be00640 1 Processor -- start 2024-03-04T07:34:27.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442be00640 1 -- start start 2024-03-04T07:34:27.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f442c0c6b20 0x7f442c0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 0x7f442c153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c158bb0 0x7f442c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c06d3a0 con 0x7f442c1528b0 2024-03-04T07:34:27.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c06d0c0 con 0x7f442c158bb0 2024-03-04T07:34:27.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442c06c900 con 0x7f442c0c6b20 2024-03-04T07:34:27.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 0x7f442c153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442aa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f442c0c6b20 0x7f442c0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 0x7f442c153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46644/0 (socket says 172.21.15.113:46644) 2024-03-04T07:34:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442a000640 1 -- 172.21.15.113:0/1059707663 learned_addr learned my addr 172.21.15.113:0/1059707663 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442b400640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c158bb0 0x7f442c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442a000640 1 -- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f442c0c6b20 msgr2=0x7f442c0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442a000640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f442c0c6b20 0x7f442c0cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442a000640 1 -- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c158bb0 msgr2=0x7f442c154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442a000640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c158bb0 0x7f442c154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442a000640 1 -- 172.21.15.113:0/1059707663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442c155f60 con 0x7f442c1528b0 2024-03-04T07:34:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442a000640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 0x7f442c153f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f441800c450 tx=0x7f441800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f4423400640 1 -- 172.21.15.113:0/1059707663 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418016020 con 0x7f442c1528b0 2024-03-04T07:34:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f4423400640 1 -- 172.21.15.113:0/1059707663 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44180052d0 con 0x7f442c1528b0 2024-03-04T07:34:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442be00640 1 -- 172.21.15.113:0/1059707663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442c14b9f0 con 0x7f442c1528b0 2024-03-04T07:34:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f4423400640 1 -- 172.21.15.113:0/1059707663 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418007260 con 0x7f442c1528b0 2024-03-04T07:34:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.553+0000 7f442be00640 1 -- 172.21.15.113:0/1059707663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442c1f47b0 con 0x7f442c1528b0 2024-03-04T07:34:27.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.557+0000 7f442be00640 1 -- 172.21.15.113:0/1059707663 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442c075a80 con 0x7f442c1528b0 2024-03-04T07:34:27.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.557+0000 7f4423400640 1 -- 172.21.15.113:0/1059707663 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4418007400 con 0x7f442c1528b0 2024-03-04T07:34:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.557+0000 7f4423400640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f440c066fb0 0x7f440c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.557+0000 7f4423400640 1 -- 172.21.15.113:0/1059707663 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4418084890 con 0x7f442c1528b0 2024-03-04T07:34:27.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.557+0000 7f442aa00640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f440c066fb0 0x7f440c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.561+0000 7f442aa00640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f440c066fb0 0x7f440c069470 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f441c0b2000 tx=0x7f441c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:27.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.561+0000 7f4423400640 1 -- 172.21.15.113:0/1059707663 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4418020020 con 0x7f442c1528b0 2024-03-04T07:34:27.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.721+0000 7f442be00640 1 -- 172.21.15.113:0/1059707663 --> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f442c14bcd0 con 0x7f440c066fb0 2024-03-04T07:34:27.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.741+0000 7f4423400640 1 -- 172.21.15.113:0/1059707663 <== mgr.4103 v2:172.21.15.136:6832/3353359815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251895 (secure 0 0 0) 0x7f442c14bcd0 con 0x7f440c066fb0 2024-03-04T07:34:27.747 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:27.749 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-03-04T07:34:27.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 -- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f440c066fb0 msgr2=0x7f440c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f440c066fb0 0x7f440c069470 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f441c0b2000 tx=0x7f441c009040 comp rx=0 tx=0).stop 2024-03-04T07:34:27.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 -- 172.21.15.113:0/1059707663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 msgr2=0x7f442c153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 0x7f442c153f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f441800c450 tx=0x7f441800c910 comp rx=0 tx=0).stop 2024-03-04T07:34:27.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 -- 172.21.15.113:0/1059707663 shutdown_connections 2024-03-04T07:34:27.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f442c158bb0 0x7f442c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f440c066fb0 0x7f440c069470 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f442c1528b0 0x7f442c153f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 --2- 172.21.15.113:0/1059707663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f442c0c6b20 0x7f442c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 -- 172.21.15.113:0/1059707663 >> 172.21.15.113:0/1059707663 conn(0x7f442c06e9a0 msgr2=0x7f442c1573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:27.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 -- 172.21.15.113:0/1059707663 shutdown_connections 2024-03-04T07:34:27.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.745+0000 7f442be00640 1 -- 172.21.15.113:0/1059707663 wait complete. 2024-03-04T07:34:27.769 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-03-04T07:34:27.443205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596921,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":288,"num_write_kb":2458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223240,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751480,"statfs":{"total":773094113280,"available":772865515520,"internally_reserved":0,"allocated":3022848,"data_stored":1782074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":225496630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2737,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003894"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645580+0000","last_change":"2024-03-04T07:33:56.559432+0000","last_active":"2024-03-04T07:33:58.645580+0000","last_peered":"2024-03-04T07:33:58.645580+0000","last_clean":"2024-03-04T07:33:58.645580+0000","last_became_active":"2024-03-04T07:33:56.559031+0000","last_became_peered":"2024-03-04T07:33:56.559031+0000","last_unstale":"2024-03-04T07:33:58.645580+0000","last_undegraded":"2024-03-04T07:33:58.645580+0000","last_fullsized":"2024-03-04T07:33:58.645580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:43:55.037090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750282+0000","last_change":"2024-03-04T07:33:56.546822+0000","last_active":"2024-03-04T07:33:57.750282+0000","last_peered":"2024-03-04T07:33:57.750282+0000","last_clean":"2024-03-04T07:33:57.750282+0000","last_became_active":"2024-03-04T07:33:56.546547+0000","last_became_peered":"2024-03-04T07:33:56.546547+0000","last_unstale":"2024-03-04T07:33:57.750282+0000","last_undegraded":"2024-03-04T07:33:57.750282+0000","last_fullsized":"2024-03-04T07:33:57.750282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:33:06.332337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859174+0000","last_change":"2024-03-04T07:33:56.530803+0000","last_active":"2024-03-04T07:33:57.859174+0000","last_peered":"2024-03-04T07:33:57.859174+0000","last_clean":"2024-03-04T07:33:57.859174+0000","last_became_active":"2024-03-04T07:33:56.530488+0000","last_became_peered":"2024-03-04T07:33:56.530488+0000","last_unstale":"2024-03-04T07:33:57.859174+0000","last_undegraded":"2024-03-04T07:33:57.859174+0000","last_fullsized":"2024-03-04T07:33:57.859174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:00:42.147951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642296+0000","last_change":"2024-03-04T07:33:56.558669+0000","last_active":"2024-03-04T07:33:58.642296+0000","last_peered":"2024-03-04T07:33:58.642296+0000","last_clean":"2024-03-04T07:33:58.642296+0000","last_became_active":"2024-03-04T07:33:56.558350+0000","last_became_peered":"2024-03-04T07:33:56.558350+0000","last_unstale":"2024-03-04T07:33:58.642296+0000","last_undegraded":"2024-03-04T07:33:58.642296+0000","last_fullsized":"2024-03-04T07:33:58.642296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:35:53.460563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644296+0000","last_change":"2024-03-04T07:33:56.540760+0000","last_active":"2024-03-04T07:33:58.644296+0000","last_peered":"2024-03-04T07:33:58.644296+0000","last_clean":"2024-03-04T07:33:58.644296+0000","last_became_active":"2024-03-04T07:33:56.540351+0000","last_became_peered":"2024-03-04T07:33:56.540351+0000","last_unstale":"2024-03-04T07:33:58.644296+0000","last_undegraded":"2024-03-04T07:33:58.644296+0000","last_fullsized":"2024-03-04T07:33:58.644296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:24:21.708228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641368+0000","last_change":"2024-03-04T07:33:56.558333+0000","last_active":"2024-03-04T07:33:58.641368+0000","last_peered":"2024-03-04T07:33:58.641368+0000","last_clean":"2024-03-04T07:33:58.641368+0000","last_became_active":"2024-03-04T07:33:56.558046+0000","last_became_peered":"2024-03-04T07:33:56.558046+0000","last_unstale":"2024-03-04T07:33:58.641368+0000","last_undegraded":"2024-03-04T07:33:58.641368+0000","last_fullsized":"2024-03-04T07:33:58.641368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:30:46.375184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643038+0000","last_change":"2024-03-04T07:33:56.543476+0000","last_active":"2024-03-04T07:33:58.643038+0000","last_peered":"2024-03-04T07:33:58.643038+0000","last_clean":"2024-03-04T07:33:58.643038+0000","last_became_active":"2024-03-04T07:33:56.543176+0000","last_became_peered":"2024-03-04T07:33:56.543176+0000","last_unstale":"2024-03-04T07:33:58.643038+0000","last_undegraded":"2024-03-04T07:33:58.643038+0000","last_fullsized":"2024-03-04T07:33:58.643038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:26:34.736870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523950+0000","last_change":"2024-03-04T07:33:56.559488+0000","last_active":"2024-03-04T07:33:57.523950+0000","last_peered":"2024-03-04T07:33:57.523950+0000","last_clean":"2024-03-04T07:33:57.523950+0000","last_became_active":"2024-03-04T07:33:56.559172+0000","last_became_peered":"2024-03-04T07:33:56.559172+0000","last_unstale":"2024-03-04T07:33:57.523950+0000","last_undegraded":"2024-03-04T07:33:57.523950+0000","last_fullsized":"2024-03-04T07:33:57.523950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:58:53.461697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645566+0000","last_change":"2024-03-04T07:33:56.560587+0000","last_active":"2024-03-04T07:33:58.645566+0000","last_peered":"2024-03-04T07:33:58.645566+0000","last_clean":"2024-03-04T07:33:58.645566+0000","last_became_active":"2024-03-04T07:33:56.560306+0000","last_became_peered":"2024-03-04T07:33:56.560306+0000","last_unstale":"2024-03-04T07:33:58.645566+0000","last_undegraded":"2024-03-04T07:33:58.645566+0000","last_fullsized":"2024-03-04T07:33:58.645566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:06:17.011327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641979+0000","last_change":"2024-03-04T07:33:56.560852+0000","last_active":"2024-03-04T07:33:58.641979+0000","last_peered":"2024-03-04T07:33:58.641979+0000","last_clean":"2024-03-04T07:33:58.641979+0000","last_became_active":"2024-03-04T07:33:56.560592+0000","last_became_peered":"2024-03-04T07:33:56.560592+0000","last_unstale":"2024-03-04T07:33:58.641979+0000","last_undegraded":"2024-03-04T07:33:58.641979+0000","last_fullsized":"2024-03-04T07:33:58.641979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:39:31.496432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641631+0000","last_change":"2024-03-04T07:33:56.561040+0000","last_active":"2024-03-04T07:33:58.641631+0000","last_peered":"2024-03-04T07:33:58.641631+0000","last_clean":"2024-03-04T07:33:58.641631+0000","last_became_active":"2024-03-04T07:33:56.560719+0000","last_became_peered":"2024-03-04T07:33:56.560719+0000","last_unstale":"2024-03-04T07:33:58.641631+0000","last_undegraded":"2024-03-04T07:33:58.641631+0000","last_fullsized":"2024-03-04T07:33:58.641631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:14:13.007587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526511+0000","last_change":"2024-03-04T07:33:56.550648+0000","last_active":"2024-03-04T07:33:57.526511+0000","last_peered":"2024-03-04T07:33:57.526511+0000","last_clean":"2024-03-04T07:33:57.526511+0000","last_became_active":"2024-03-04T07:33:56.549568+0000","last_became_peered":"2024-03-04T07:33:56.549568+0000","last_unstale":"2024-03-04T07:33:57.526511+0000","last_undegraded":"2024-03-04T07:33:57.526511+0000","last_fullsized":"2024-03-04T07:33:57.526511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:34:35.299480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523777+0000","last_change":"2024-03-04T07:33:56.558880+0000","last_active":"2024-03-04T07:33:57.523777+0000","last_peered":"2024-03-04T07:33:57.523777+0000","last_clean":"2024-03-04T07:33:57.523777+0000","last_became_active":"2024-03-04T07:33:56.558567+0000","last_became_peered":"2024-03-04T07:33:56.558567+0000","last_unstale":"2024-03-04T07:33:57.523777+0000","last_undegraded":"2024-03-04T07:33:57.523777+0000","last_fullsized":"2024-03-04T07:33:57.523777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:52:16.354787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.858969+0000","last_change":"2024-03-04T07:33:56.544222+0000","last_active":"2024-03-04T07:33:57.858969+0000","last_peered":"2024-03-04T07:33:57.858969+0000","last_clean":"2024-03-04T07:33:57.858969+0000","last_became_active":"2024-03-04T07:33:56.543936+0000","last_became_peered":"2024-03-04T07:33:56.543936+0000","last_unstale":"2024-03-04T07:33:57.858969+0000","last_undegraded":"2024-03-04T07:33:57.858969+0000","last_fullsized":"2024-03-04T07:33:57.858969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:12:42.788232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749980+0000","last_change":"2024-03-04T07:33:56.545195+0000","last_active":"2024-03-04T07:33:57.749980+0000","last_peered":"2024-03-04T07:33:57.749980+0000","last_clean":"2024-03-04T07:33:57.749980+0000","last_became_active":"2024-03-04T07:33:56.544837+0000","last_became_peered":"2024-03-04T07:33:56.544837+0000","last_unstale":"2024-03-04T07:33:57.749980+0000","last_undegraded":"2024-03-04T07:33:57.749980+0000","last_fullsized":"2024-03-04T07:33:57.749980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:24:59.643746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641197+0000","last_change":"2024-03-04T07:33:56.558398+0000","last_active":"2024-03-04T07:33:58.641197+0000","last_peered":"2024-03-04T07:33:58.641197+0000","last_clean":"2024-03-04T07:33:58.641197+0000","last_became_active":"2024-03-04T07:33:56.557962+0000","last_became_peered":"2024-03-04T07:33:56.557962+0000","last_unstale":"2024-03-04T07:33:58.641197+0000","last_undegraded":"2024-03-04T07:33:58.641197+0000","last_fullsized":"2024-03-04T07:33:58.641197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:21:54.588819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643019+0000","last_change":"2024-03-04T07:33:56.559597+0000","last_active":"2024-03-04T07:33:58.643019+0000","last_peered":"2024-03-04T07:33:58.643019+0000","last_clean":"2024-03-04T07:33:58.643019+0000","last_became_active":"2024-03-04T07:33:56.558719+0000","last_became_peered":"2024-03-04T07:33:56.558719+0000","last_unstale":"2024-03-04T07:33:58.643019+0000","last_undegraded":"2024-03-04T07:33:58.643019+0000","last_fullsized":"2024-03-04T07:33:58.643019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:43:56.063344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522942+0000","last_change":"2024-03-04T07:33:56.528479+0000","last_active":"2024-03-04T07:33:57.522942+0000","last_peered":"2024-03-04T07:33:57.522942+0000","last_clean":"2024-03-04T07:33:57.522942+0000","last_became_active":"2024-03-04T07:33:56.528213+0000","last_became_peered":"2024-03-04T07:33:56.528213+0000","last_unstale":"2024-03-04T07:33:57.522942+0000","last_undegraded":"2024-03-04T07:33:57.522942+0000","last_fullsized":"2024-03-04T07:33:57.522942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:38:48.062007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644805+0000","last_change":"2024-03-04T07:33:56.545890+0000","last_active":"2024-03-04T07:33:58.644805+0000","last_peered":"2024-03-04T07:33:58.644805+0000","last_clean":"2024-03-04T07:33:58.644805+0000","last_became_active":"2024-03-04T07:33:56.545467+0000","last_became_peered":"2024-03-04T07:33:56.545467+0000","last_unstale":"2024-03-04T07:33:58.644805+0000","last_undegraded":"2024-03-04T07:33:58.644805+0000","last_fullsized":"2024-03-04T07:33:58.644805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:43:56.049152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641702+0000","last_change":"2024-03-04T07:33:56.549059+0000","last_active":"2024-03-04T07:33:58.641702+0000","last_peered":"2024-03-04T07:33:58.641702+0000","last_clean":"2024-03-04T07:33:58.641702+0000","last_became_active":"2024-03-04T07:33:56.548783+0000","last_became_peered":"2024-03-04T07:33:56.548783+0000","last_unstale":"2024-03-04T07:33:58.641702+0000","last_undegraded":"2024-03-04T07:33:58.641702+0000","last_fullsized":"2024-03-04T07:33:58.641702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:24:02.330428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859045+0000","last_change":"2024-03-04T07:33:56.540558+0000","last_active":"2024-03-04T07:33:57.859045+0000","last_peered":"2024-03-04T07:33:57.859045+0000","last_clean":"2024-03-04T07:33:57.859045+0000","last_became_active":"2024-03-04T07:33:56.539403+0000","last_became_peered":"2024-03-04T07:33:56.539403+0000","last_unstale":"2024-03-04T07:33:57.859045+0000","last_undegraded":"2024-03-04T07:33:57.859045+0000","last_fullsized":"2024-03-04T07:33:57.859045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:28:19.413591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641763+0000","last_change":"2024-03-04T07:33:56.554982+0000","last_active":"2024-03-04T07:33:58.641763+0000","last_peered":"2024-03-04T07:33:58.641763+0000","last_clean":"2024-03-04T07:33:58.641763+0000","last_became_active":"2024-03-04T07:33:56.554470+0000","last_became_peered":"2024-03-04T07:33:56.554470+0000","last_unstale":"2024-03-04T07:33:58.641763+0000","last_undegraded":"2024-03-04T07:33:58.641763+0000","last_fullsized":"2024-03-04T07:33:58.641763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:59:32.767329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523496+0000","last_change":"2024-03-04T07:33:56.545465+0000","last_active":"2024-03-04T07:33:57.523496+0000","last_peered":"2024-03-04T07:33:57.523496+0000","last_clean":"2024-03-04T07:33:57.523496+0000","last_became_active":"2024-03-04T07:33:56.545270+0000","last_became_peered":"2024-03-04T07:33:56.545270+0000","last_unstale":"2024-03-04T07:33:57.523496+0000","last_undegraded":"2024-03-04T07:33:57.523496+0000","last_fullsized":"2024-03-04T07:33:57.523496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:08:45.244868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643452+0000","last_change":"2024-03-04T07:33:56.555428+0000","last_active":"2024-03-04T07:33:58.643452+0000","last_peered":"2024-03-04T07:33:58.643452+0000","last_clean":"2024-03-04T07:33:58.643452+0000","last_became_active":"2024-03-04T07:33:56.554942+0000","last_became_peered":"2024-03-04T07:33:56.554942+0000","last_unstale":"2024-03-04T07:33:58.643452+0000","last_undegraded":"2024-03-04T07:33:58.643452+0000","last_fullsized":"2024-03-04T07:33:58.643452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:09:04.724517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523610+0000","last_change":"2024-03-04T07:33:56.549563+0000","last_active":"2024-03-04T07:33:57.523610+0000","last_peered":"2024-03-04T07:33:57.523610+0000","last_clean":"2024-03-04T07:33:57.523610+0000","last_became_active":"2024-03-04T07:33:56.548790+0000","last_became_peered":"2024-03-04T07:33:56.548790+0000","last_unstale":"2024-03-04T07:33:57.523610+0000","last_undegraded":"2024-03-04T07:33:57.523610+0000","last_fullsized":"2024-03-04T07:33:57.523610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:30:53.665286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642385+0000","last_change":"2024-03-04T07:33:56.553791+0000","last_active":"2024-03-04T07:33:58.642385+0000","last_peered":"2024-03-04T07:33:58.642385+0000","last_clean":"2024-03-04T07:33:58.642385+0000","last_became_active":"2024-03-04T07:33:56.553012+0000","last_became_peered":"2024-03-04T07:33:56.553012+0000","last_unstale":"2024-03-04T07:33:58.642385+0000","last_undegraded":"2024-03-04T07:33:58.642385+0000","last_fullsized":"2024-03-04T07:33:58.642385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:38.440189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750163+0000","last_change":"2024-03-04T07:33:56.551341+0000","last_active":"2024-03-04T07:33:57.750163+0000","last_peered":"2024-03-04T07:33:57.750163+0000","last_clean":"2024-03-04T07:33:57.750163+0000","last_became_active":"2024-03-04T07:33:56.550616+0000","last_became_peered":"2024-03-04T07:33:56.550616+0000","last_unstale":"2024-03-04T07:33:57.750163+0000","last_undegraded":"2024-03-04T07:33:57.750163+0000","last_fullsized":"2024-03-04T07:33:57.750163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:35:45.801510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523840+0000","last_change":"2024-03-04T07:33:56.543903+0000","last_active":"2024-03-04T07:33:57.523840+0000","last_peered":"2024-03-04T07:33:57.523840+0000","last_clean":"2024-03-04T07:33:57.523840+0000","last_became_active":"2024-03-04T07:33:56.543632+0000","last_became_peered":"2024-03-04T07:33:56.543632+0000","last_unstale":"2024-03-04T07:33:57.523840+0000","last_undegraded":"2024-03-04T07:33:57.523840+0000","last_fullsized":"2024-03-04T07:33:57.523840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:26:10.826004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859232+0000","last_change":"2024-03-04T07:33:56.547091+0000","last_active":"2024-03-04T07:33:57.859232+0000","last_peered":"2024-03-04T07:33:57.859232+0000","last_clean":"2024-03-04T07:33:57.859232+0000","last_became_active":"2024-03-04T07:33:56.546652+0000","last_became_peered":"2024-03-04T07:33:56.546652+0000","last_unstale":"2024-03-04T07:33:57.859232+0000","last_undegraded":"2024-03-04T07:33:57.859232+0000","last_fullsized":"2024-03-04T07:33:57.859232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:26:03.091219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642136+0000","last_change":"2024-03-04T07:33:56.552493+0000","last_active":"2024-03-04T07:33:58.642136+0000","last_peered":"2024-03-04T07:33:58.642136+0000","last_clean":"2024-03-04T07:33:58.642136+0000","last_became_active":"2024-03-04T07:33:56.552189+0000","last_became_peered":"2024-03-04T07:33:56.552189+0000","last_unstale":"2024-03-04T07:33:58.642136+0000","last_undegraded":"2024-03-04T07:33:58.642136+0000","last_fullsized":"2024-03-04T07:33:58.642136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:29:10.313129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.522710+0000","last_change":"2024-03-04T07:33:56.549661+0000","last_active":"2024-03-04T07:33:57.522710+0000","last_peered":"2024-03-04T07:33:57.522710+0000","last_clean":"2024-03-04T07:33:57.522710+0000","last_became_active":"2024-03-04T07:33:56.549351+0000","last_became_peered":"2024-03-04T07:33:56.549351+0000","last_unstale":"2024-03-04T07:33:57.522710+0000","last_undegraded":"2024-03-04T07:33:57.522710+0000","last_fullsized":"2024-03-04T07:33:57.522710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:55.684272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640651+0000","last_change":"2024-03-04T07:33:56.533132+0000","last_active":"2024-03-04T07:33:58.640651+0000","last_peered":"2024-03-04T07:33:58.640651+0000","last_clean":"2024-03-04T07:33:58.640651+0000","last_became_active":"2024-03-04T07:33:56.532251+0000","last_became_peered":"2024-03-04T07:33:56.532251+0000","last_unstale":"2024-03-04T07:33:58.640651+0000","last_undegraded":"2024-03-04T07:33:58.640651+0000","last_fullsized":"2024-03-04T07:33:58.640651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:17:47.018585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.525697+0000","last_change":"2024-03-04T07:33:56.543382+0000","last_active":"2024-03-04T07:33:57.525697+0000","last_peered":"2024-03-04T07:33:57.525697+0000","last_clean":"2024-03-04T07:33:57.525697+0000","last_became_active":"2024-03-04T07:33:56.542937+0000","last_became_peered":"2024-03-04T07:33:56.542937+0000","last_unstale":"2024-03-04T07:33:57.525697+0000","last_undegraded":"2024-03-04T07:33:57.525697+0000","last_fullsized":"2024-03-04T07:33:57.525697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:18:28.332590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'5","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.606616+0000","last_change":"2024-03-04T07:33:55.566130+0000","last_active":"2024-03-04T07:34:07.606616+0000","last_peered":"2024-03-04T07:34:07.606616+0000","last_clean":"2024-03-04T07:34:07.606616+0000","last_became_active":"2024-03-04T07:33:55.565894+0000","last_became_peered":"2024-03-04T07:33:55.565894+0000","last_unstale":"2024-03-04T07:34:07.606616+0000","last_undegraded":"2024-03-04T07:34:07.606616+0000","last_fullsized":"2024-03-04T07:34:07.606616+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:36.530949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859235+0000","last_change":"2024-03-04T07:33:56.540113+0000","last_active":"2024-03-04T07:33:57.859235+0000","last_peered":"2024-03-04T07:33:57.859235+0000","last_clean":"2024-03-04T07:33:57.859235+0000","last_became_active":"2024-03-04T07:33:56.539738+0000","last_became_peered":"2024-03-04T07:33:56.539738+0000","last_unstale":"2024-03-04T07:33:57.859235+0000","last_undegraded":"2024-03-04T07:33:57.859235+0000","last_fullsized":"2024-03-04T07:33:57.859235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:24:44.949754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671613+0000","last_change":"2024-03-04T07:33:55.554956+0000","last_active":"2024-03-04T07:34:12.671613+0000","last_peered":"2024-03-04T07:34:12.671613+0000","last_clean":"2024-03-04T07:34:12.671613+0000","last_became_active":"2024-03-04T07:33:55.554151+0000","last_became_peered":"2024-03-04T07:33:55.554151+0000","last_unstale":"2024-03-04T07:34:12.671613+0000","last_undegraded":"2024-03-04T07:34:12.671613+0000","last_fullsized":"2024-03-04T07:34:12.671613+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:06:19.621510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671450+0000","last_change":"2024-03-04T07:33:55.542200+0000","last_active":"2024-03-04T07:34:12.671450+0000","last_peered":"2024-03-04T07:34:12.671450+0000","last_clean":"2024-03-04T07:34:12.671450+0000","last_became_active":"2024-03-04T07:33:55.541902+0000","last_became_peered":"2024-03-04T07:33:55.541902+0000","last_unstale":"2024-03-04T07:34:12.671450+0000","last_undegraded":"2024-03-04T07:34:12.671450+0000","last_fullsized":"2024-03-04T07:34:12.671450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:55:50.251288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523733+0000","last_change":"2024-03-04T07:33:56.548380+0000","last_active":"2024-03-04T07:33:57.523733+0000","last_peered":"2024-03-04T07:33:57.523733+0000","last_clean":"2024-03-04T07:33:57.523733+0000","last_became_active":"2024-03-04T07:33:56.546087+0000","last_became_peered":"2024-03-04T07:33:56.546087+0000","last_unstale":"2024-03-04T07:33:57.523733+0000","last_undegraded":"2024-03-04T07:33:57.523733+0000","last_fullsized":"2024-03-04T07:33:57.523733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:06:43.538687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643611+0000","last_change":"2024-03-04T07:33:55.568508+0000","last_active":"2024-03-04T07:33:58.643611+0000","last_peered":"2024-03-04T07:33:58.643611+0000","last_clean":"2024-03-04T07:33:58.643611+0000","last_became_active":"2024-03-04T07:33:55.566073+0000","last_became_peered":"2024-03-04T07:33:55.566073+0000","last_unstale":"2024-03-04T07:33:58.643611+0000","last_undegraded":"2024-03-04T07:33:58.643611+0000","last_fullsized":"2024-03-04T07:33:58.643611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:34:17.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749969+0000","last_change":"2024-03-04T07:33:56.552785+0000","last_active":"2024-03-04T07:33:57.749969+0000","last_peered":"2024-03-04T07:33:57.749969+0000","last_clean":"2024-03-04T07:33:57.749969+0000","last_became_active":"2024-03-04T07:33:56.552533+0000","last_became_peered":"2024-03-04T07:33:56.552533+0000","last_unstale":"2024-03-04T07:33:57.749969+0000","last_undegraded":"2024-03-04T07:33:57.749969+0000","last_fullsized":"2024-03-04T07:33:57.749969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:06:08.582237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.603663+0000","last_change":"2024-03-04T07:33:55.559395+0000","last_active":"2024-03-04T07:34:07.603663+0000","last_peered":"2024-03-04T07:34:07.603663+0000","last_clean":"2024-03-04T07:34:07.603663+0000","last_became_active":"2024-03-04T07:33:55.558593+0000","last_became_peered":"2024-03-04T07:33:55.558593+0000","last_unstale":"2024-03-04T07:34:07.603663+0000","last_undegraded":"2024-03-04T07:34:07.603663+0000","last_fullsized":"2024-03-04T07:34:07.603663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:00:20.205445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642007+0000","last_change":"2024-03-04T07:33:56.553260+0000","last_active":"2024-03-04T07:33:58.642007+0000","last_peered":"2024-03-04T07:33:58.642007+0000","last_clean":"2024-03-04T07:33:58.642007+0000","last_became_active":"2024-03-04T07:33:56.552952+0000","last_became_peered":"2024-03-04T07:33:56.552952+0000","last_unstale":"2024-03-04T07:33:58.642007+0000","last_undegraded":"2024-03-04T07:33:58.642007+0000","last_fullsized":"2024-03-04T07:33:58.642007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:25:51.076222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.603045+0000","last_change":"2024-03-04T07:33:55.568220+0000","last_active":"2024-03-04T07:34:07.603045+0000","last_peered":"2024-03-04T07:34:07.603045+0000","last_clean":"2024-03-04T07:34:07.603045+0000","last_became_active":"2024-03-04T07:33:55.567569+0000","last_became_peered":"2024-03-04T07:33:55.567569+0000","last_unstale":"2024-03-04T07:34:07.603045+0000","last_undegraded":"2024-03-04T07:34:07.603045+0000","last_fullsized":"2024-03-04T07:34:07.603045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:40:49.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859421+0000","last_change":"2024-03-04T07:33:56.559745+0000","last_active":"2024-03-04T07:33:57.859421+0000","last_peered":"2024-03-04T07:33:57.859421+0000","last_clean":"2024-03-04T07:33:57.859421+0000","last_became_active":"2024-03-04T07:33:56.559456+0000","last_became_peered":"2024-03-04T07:33:56.559456+0000","last_unstale":"2024-03-04T07:33:57.859421+0000","last_undegraded":"2024-03-04T07:33:57.859421+0000","last_fullsized":"2024-03-04T07:33:57.859421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:18:43.135906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.532766+0000","last_change":"2024-03-04T07:33:55.550625+0000","last_active":"2024-03-04T07:34:01.532766+0000","last_peered":"2024-03-04T07:34:01.532766+0000","last_clean":"2024-03-04T07:34:01.532766+0000","last_became_active":"2024-03-04T07:33:55.550327+0000","last_became_peered":"2024-03-04T07:33:55.550327+0000","last_unstale":"2024-03-04T07:34:01.532766+0000","last_undegraded":"2024-03-04T07:34:01.532766+0000","last_fullsized":"2024-03-04T07:34:01.532766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:20.282585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526558+0000","last_change":"2024-03-04T07:33:56.537570+0000","last_active":"2024-03-04T07:33:57.526558+0000","last_peered":"2024-03-04T07:33:57.526558+0000","last_clean":"2024-03-04T07:33:57.526558+0000","last_became_active":"2024-03-04T07:33:56.536870+0000","last_became_peered":"2024-03-04T07:33:56.536870+0000","last_unstale":"2024-03-04T07:33:57.526558+0000","last_undegraded":"2024-03-04T07:33:57.526558+0000","last_fullsized":"2024-03-04T07:33:57.526558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:59:23.807432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'11","reported_seq":25,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:21.167614+0000","last_change":"2024-03-04T07:33:55.538915+0000","last_active":"2024-03-04T07:34:21.167614+0000","last_peered":"2024-03-04T07:34:21.167614+0000","last_clean":"2024-03-04T07:34:21.167614+0000","last_became_active":"2024-03-04T07:33:55.536263+0000","last_became_peered":"2024-03-04T07:33:55.536263+0000","last_unstale":"2024-03-04T07:34:21.167614+0000","last_undegraded":"2024-03-04T07:34:21.167614+0000","last_fullsized":"2024-03-04T07:34:21.167614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:09.612704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2685,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643901+0000","last_change":"2024-03-04T07:33:56.551087+0000","last_active":"2024-03-04T07:33:58.643901+0000","last_peered":"2024-03-04T07:33:58.643901+0000","last_clean":"2024-03-04T07:33:58.643901+0000","last_became_active":"2024-03-04T07:33:56.550767+0000","last_became_peered":"2024-03-04T07:33:56.550767+0000","last_unstale":"2024-03-04T07:33:58.643901+0000","last_undegraded":"2024-03-04T07:33:58.643901+0000","last_fullsized":"2024-03-04T07:33:58.643901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:02:15.603800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671768+0000","last_change":"2024-03-04T07:33:55.560382+0000","last_active":"2024-03-04T07:34:12.671768+0000","last_peered":"2024-03-04T07:34:12.671768+0000","last_clean":"2024-03-04T07:34:12.671768+0000","last_became_active":"2024-03-04T07:33:55.560078+0000","last_became_peered":"2024-03-04T07:33:55.560078+0000","last_unstale":"2024-03-04T07:34:12.671768+0000","last_undegraded":"2024-03-04T07:34:12.671768+0000","last_fullsized":"2024-03-04T07:34:12.671768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:41:10.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859054+0000","last_change":"2024-03-04T07:33:56.537920+0000","last_active":"2024-03-04T07:33:57.859054+0000","last_peered":"2024-03-04T07:33:57.859054+0000","last_clean":"2024-03-04T07:33:57.859054+0000","last_became_active":"2024-03-04T07:33:56.535960+0000","last_became_peered":"2024-03-04T07:33:56.535960+0000","last_unstale":"2024-03-04T07:33:57.859054+0000","last_undegraded":"2024-03-04T07:33:57.859054+0000","last_fullsized":"2024-03-04T07:33:57.859054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:37:09.344120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.604934+0000","last_change":"2024-03-04T07:33:55.567682+0000","last_active":"2024-03-04T07:34:07.604934+0000","last_peered":"2024-03-04T07:34:07.604934+0000","last_clean":"2024-03-04T07:34:07.604934+0000","last_became_active":"2024-03-04T07:33:55.566915+0000","last_became_peered":"2024-03-04T07:33:55.566915+0000","last_unstale":"2024-03-04T07:34:07.604934+0000","last_undegraded":"2024-03-04T07:34:07.604934+0000","last_fullsized":"2024-03-04T07:34:07.604934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:33:47.010346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644757+0000","last_change":"2024-03-04T07:33:56.547846+0000","last_active":"2024-03-04T07:33:58.644757+0000","last_peered":"2024-03-04T07:33:58.644757+0000","last_clean":"2024-03-04T07:33:58.644757+0000","last_became_active":"2024-03-04T07:33:56.547459+0000","last_became_peered":"2024-03-04T07:33:56.547459+0000","last_unstale":"2024-03-04T07:33:58.644757+0000","last_undegraded":"2024-03-04T07:33:58.644757+0000","last_fullsized":"2024-03-04T07:33:58.644757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:23:57.623123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.675235+0000","last_change":"2024-03-04T07:33:55.557041+0000","last_active":"2024-03-04T07:34:12.675235+0000","last_peered":"2024-03-04T07:34:12.675235+0000","last_clean":"2024-03-04T07:34:12.675235+0000","last_became_active":"2024-03-04T07:33:55.556416+0000","last_became_peered":"2024-03-04T07:33:55.556416+0000","last_unstale":"2024-03-04T07:34:12.675235+0000","last_undegraded":"2024-03-04T07:34:12.675235+0000","last_fullsized":"2024-03-04T07:34:12.675235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:31.472235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":846,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.641078+0000","last_change":"2024-03-04T07:33:56.531986+0000","last_active":"2024-03-04T07:33:58.641078+0000","last_peered":"2024-03-04T07:33:58.641078+0000","last_clean":"2024-03-04T07:33:58.641078+0000","last_became_active":"2024-03-04T07:33:56.531565+0000","last_became_peered":"2024-03-04T07:33:56.531565+0000","last_unstale":"2024-03-04T07:33:58.641078+0000","last_undegraded":"2024-03-04T07:33:58.641078+0000","last_fullsized":"2024-03-04T07:33:58.641078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:42:35.040979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'15","reported_seq":29,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.675036+0000","last_change":"2024-03-04T07:33:55.538469+0000","last_active":"2024-03-04T07:34:12.675036+0000","last_peered":"2024-03-04T07:34:12.675036+0000","last_clean":"2024-03-04T07:34:12.675036+0000","last_became_active":"2024-03-04T07:33:55.538165+0000","last_became_peered":"2024-03-04T07:33:55.538165+0000","last_unstale":"2024-03-04T07:34:12.675036+0000","last_undegraded":"2024-03-04T07:34:12.675036+0000","last_fullsized":"2024-03-04T07:34:12.675036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:00:01.263768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643331+0000","last_change":"2024-03-04T07:33:56.547881+0000","last_active":"2024-03-04T07:33:58.643331+0000","last_peered":"2024-03-04T07:33:58.643331+0000","last_clean":"2024-03-04T07:33:58.643331+0000","last_became_active":"2024-03-04T07:33:56.547461+0000","last_became_peered":"2024-03-04T07:33:56.547461+0000","last_unstale":"2024-03-04T07:33:58.643331+0000","last_undegraded":"2024-03-04T07:33:58.643331+0000","last_fullsized":"2024-03-04T07:33:58.643331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:34:32.709430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.535127+0000","last_change":"2024-03-04T07:33:55.544356+0000","last_active":"2024-03-04T07:34:01.535127+0000","last_peered":"2024-03-04T07:34:01.535127+0000","last_clean":"2024-03-04T07:34:01.535127+0000","last_became_active":"2024-03-04T07:33:55.541295+0000","last_became_peered":"2024-03-04T07:33:55.541295+0000","last_unstale":"2024-03-04T07:34:01.535127+0000","last_undegraded":"2024-03-04T07:34:01.535127+0000","last_fullsized":"2024-03-04T07:34:01.535127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:22.953982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645470+0000","last_change":"2024-03-04T07:33:56.549038+0000","last_active":"2024-03-04T07:33:58.645470+0000","last_peered":"2024-03-04T07:33:58.645470+0000","last_clean":"2024-03-04T07:33:58.645470+0000","last_became_active":"2024-03-04T07:33:56.548113+0000","last_became_peered":"2024-03-04T07:33:56.548113+0000","last_unstale":"2024-03-04T07:33:58.645470+0000","last_undegraded":"2024-03-04T07:33:58.645470+0000","last_fullsized":"2024-03-04T07:33:58.645470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:53:04.601075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.530724+0000","last_change":"2024-03-04T07:33:55.538871+0000","last_active":"2024-03-04T07:34:01.530724+0000","last_peered":"2024-03-04T07:34:01.530724+0000","last_clean":"2024-03-04T07:34:01.530724+0000","last_became_active":"2024-03-04T07:33:55.537221+0000","last_became_peered":"2024-03-04T07:33:55.537221+0000","last_unstale":"2024-03-04T07:34:01.530724+0000","last_undegraded":"2024-03-04T07:34:01.530724+0000","last_fullsized":"2024-03-04T07:34:01.530724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:02:34.533330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.645792+0000","last_change":"2024-03-04T07:33:56.543760+0000","last_active":"2024-03-04T07:33:58.645792+0000","last_peered":"2024-03-04T07:33:58.645792+0000","last_clean":"2024-03-04T07:33:58.645792+0000","last_became_active":"2024-03-04T07:33:56.543354+0000","last_became_peered":"2024-03-04T07:33:56.543354+0000","last_unstale":"2024-03-04T07:33:58.645792+0000","last_undegraded":"2024-03-04T07:33:58.645792+0000","last_fullsized":"2024-03-04T07:33:58.645792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:19:37.564790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.913391+0000","last_change":"2024-03-04T07:33:55.552856+0000","last_active":"2024-03-04T07:33:57.913391+0000","last_peered":"2024-03-04T07:33:57.913391+0000","last_clean":"2024-03-04T07:33:57.913391+0000","last_became_active":"2024-03-04T07:33:55.552472+0000","last_became_peered":"2024-03-04T07:33:55.552472+0000","last_unstale":"2024-03-04T07:33:57.913391+0000","last_undegraded":"2024-03-04T07:33:57.913391+0000","last_fullsized":"2024-03-04T07:33:57.913391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:36:50.409567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.643132+0000","last_change":"2024-03-04T07:33:56.558497+0000","last_active":"2024-03-04T07:33:58.643132+0000","last_peered":"2024-03-04T07:33:58.643132+0000","last_clean":"2024-03-04T07:33:58.643132+0000","last_became_active":"2024-03-04T07:33:56.557989+0000","last_became_peered":"2024-03-04T07:33:56.557989+0000","last_unstale":"2024-03-04T07:33:58.643132+0000","last_undegraded":"2024-03-04T07:33:58.643132+0000","last_fullsized":"2024-03-04T07:33:58.643132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:57:05.380755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.673353+0000","last_change":"2024-03-04T07:33:55.560332+0000","last_active":"2024-03-04T07:34:12.673353+0000","last_peered":"2024-03-04T07:34:12.673353+0000","last_clean":"2024-03-04T07:34:12.673353+0000","last_became_active":"2024-03-04T07:33:55.560086+0000","last_became_peered":"2024-03-04T07:33:55.560086+0000","last_unstale":"2024-03-04T07:34:12.673353+0000","last_undegraded":"2024-03-04T07:34:12.673353+0000","last_fullsized":"2024-03-04T07:34:12.673353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:25:46.280657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644296+0000","last_change":"2024-03-04T07:33:56.559881+0000","last_active":"2024-03-04T07:33:58.644296+0000","last_peered":"2024-03-04T07:33:58.644296+0000","last_clean":"2024-03-04T07:33:58.644296+0000","last_became_active":"2024-03-04T07:33:56.559589+0000","last_became_peered":"2024-03-04T07:33:56.559589+0000","last_unstale":"2024-03-04T07:33:58.644296+0000","last_undegraded":"2024-03-04T07:33:58.644296+0000","last_fullsized":"2024-03-04T07:33:58.644296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:47:35.123163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640654+0000","last_change":"2024-03-04T07:33:43.559630+0000","last_active":"2024-03-04T07:33:58.640654+0000","last_peered":"2024-03-04T07:33:58.640654+0000","last_clean":"2024-03-04T07:33:58.640654+0000","last_became_active":"2024-03-04T07:33:43.557489+0000","last_became_peered":"2024-03-04T07:33:43.557489+0000","last_unstale":"2024-03-04T07:33:58.640654+0000","last_undegraded":"2024-03-04T07:33:58.640654+0000","last_fullsized":"2024-03-04T07:33:58.640654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_clean_scrub_stamp":"2024-03-04T07:33:42.183823+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:55:35.762857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.602970+0000","last_change":"2024-03-04T07:33:55.548616+0000","last_active":"2024-03-04T07:34:07.602970+0000","last_peered":"2024-03-04T07:34:07.602970+0000","last_clean":"2024-03-04T07:34:07.602970+0000","last_became_active":"2024-03-04T07:33:55.547322+0000","last_became_peered":"2024-03-04T07:33:55.547322+0000","last_unstale":"2024-03-04T07:34:07.602970+0000","last_undegraded":"2024-03-04T07:34:07.602970+0000","last_fullsized":"2024-03-04T07:34:07.602970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:20:16.717483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523877+0000","last_change":"2024-03-04T07:33:56.559966+0000","last_active":"2024-03-04T07:33:57.523877+0000","last_peered":"2024-03-04T07:33:57.523877+0000","last_clean":"2024-03-04T07:33:57.523877+0000","last_became_active":"2024-03-04T07:33:56.559581+0000","last_became_peered":"2024-03-04T07:33:56.559581+0000","last_unstale":"2024-03-04T07:33:57.523877+0000","last_undegraded":"2024-03-04T07:33:57.523877+0000","last_fullsized":"2024-03-04T07:33:57.523877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:54:26.031249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.670511+0000","last_change":"2024-03-04T07:33:55.563905+0000","last_active":"2024-03-04T07:34:12.670511+0000","last_peered":"2024-03-04T07:34:12.670511+0000","last_clean":"2024-03-04T07:34:12.670511+0000","last_became_active":"2024-03-04T07:33:55.563117+0000","last_became_peered":"2024-03-04T07:33:55.563117+0000","last_unstale":"2024-03-04T07:34:12.670511+0000","last_undegraded":"2024-03-04T07:34:12.670511+0000","last_fullsized":"2024-03-04T07:34:12.670511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:53:02.522757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.858965+0000","last_change":"2024-03-04T07:33:56.531080+0000","last_active":"2024-03-04T07:33:57.858965+0000","last_peered":"2024-03-04T07:33:57.858965+0000","last_clean":"2024-03-04T07:33:57.858965+0000","last_became_active":"2024-03-04T07:33:56.530776+0000","last_became_peered":"2024-03-04T07:33:56.530776+0000","last_unstale":"2024-03-04T07:33:57.858965+0000","last_undegraded":"2024-03-04T07:33:57.858965+0000","last_fullsized":"2024-03-04T07:33:57.858965+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:29:14.385057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.673458+0000","last_change":"2024-03-04T07:33:55.538631+0000","last_active":"2024-03-04T07:34:12.673458+0000","last_peered":"2024-03-04T07:34:12.673458+0000","last_clean":"2024-03-04T07:34:12.673458+0000","last_became_active":"2024-03-04T07:33:55.537104+0000","last_became_peered":"2024-03-04T07:33:55.537104+0000","last_unstale":"2024-03-04T07:34:12.673458+0000","last_undegraded":"2024-03-04T07:34:12.673458+0000","last_fullsized":"2024-03-04T07:34:12.673458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:27:06.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.749962+0000","last_change":"2024-03-04T07:33:56.541335+0000","last_active":"2024-03-04T07:33:57.749962+0000","last_peered":"2024-03-04T07:33:57.749962+0000","last_clean":"2024-03-04T07:33:57.749962+0000","last_became_active":"2024-03-04T07:33:56.540536+0000","last_became_peered":"2024-03-04T07:33:56.540536+0000","last_unstale":"2024-03-04T07:33:57.749962+0000","last_undegraded":"2024-03-04T07:33:57.749962+0000","last_fullsized":"2024-03-04T07:33:57.749962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:31:57.710826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642753+0000","last_change":"2024-03-04T07:33:55.574740+0000","last_active":"2024-03-04T07:33:58.642753+0000","last_peered":"2024-03-04T07:33:58.642753+0000","last_clean":"2024-03-04T07:33:58.642753+0000","last_became_active":"2024-03-04T07:33:55.574129+0000","last_became_peered":"2024-03-04T07:33:55.574129+0000","last_unstale":"2024-03-04T07:33:58.642753+0000","last_undegraded":"2024-03-04T07:33:58.642753+0000","last_fullsized":"2024-03-04T07:33:58.642753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:40:57.891803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.526225+0000","last_change":"2024-03-04T07:33:56.541654+0000","last_active":"2024-03-04T07:33:57.526225+0000","last_peered":"2024-03-04T07:33:57.526225+0000","last_clean":"2024-03-04T07:33:57.526225+0000","last_became_active":"2024-03-04T07:33:56.541422+0000","last_became_peered":"2024-03-04T07:33:56.541422+0000","last_unstale":"2024-03-04T07:33:57.526225+0000","last_undegraded":"2024-03-04T07:33:57.526225+0000","last_fullsized":"2024-03-04T07:33:57.526225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:23:38.605481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750122+0000","last_change":"2024-03-04T07:33:55.539317+0000","last_active":"2024-03-04T07:33:57.750122+0000","last_peered":"2024-03-04T07:33:57.750122+0000","last_clean":"2024-03-04T07:33:57.750122+0000","last_became_active":"2024-03-04T07:33:55.539017+0000","last_became_peered":"2024-03-04T07:33:55.539017+0000","last_unstale":"2024-03-04T07:33:57.750122+0000","last_undegraded":"2024-03-04T07:33:57.750122+0000","last_fullsized":"2024-03-04T07:33:57.750122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:10.787316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642464+0000","last_change":"2024-03-04T07:33:56.554103+0000","last_active":"2024-03-04T07:33:58.642464+0000","last_peered":"2024-03-04T07:33:58.642464+0000","last_clean":"2024-03-04T07:33:58.642464+0000","last_became_active":"2024-03-04T07:33:56.553742+0000","last_became_peered":"2024-03-04T07:33:56.553742+0000","last_unstale":"2024-03-04T07:33:58.642464+0000","last_undegraded":"2024-03-04T07:33:58.642464+0000","last_fullsized":"2024-03-04T07:33:58.642464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:53.851585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.644929+0000","last_change":"2024-03-04T07:33:55.569442+0000","last_active":"2024-03-04T07:33:58.644929+0000","last_peered":"2024-03-04T07:33:58.644929+0000","last_clean":"2024-03-04T07:33:58.644929+0000","last_became_active":"2024-03-04T07:33:55.567809+0000","last_became_peered":"2024-03-04T07:33:55.567809+0000","last_unstale":"2024-03-04T07:33:58.644929+0000","last_undegraded":"2024-03-04T07:33:58.644929+0000","last_fullsized":"2024-03-04T07:33:58.644929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:58.551866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.750083+0000","last_change":"2024-03-04T07:33:56.542997+0000","last_active":"2024-03-04T07:33:57.750083+0000","last_peered":"2024-03-04T07:33:57.750083+0000","last_clean":"2024-03-04T07:33:57.750083+0000","last_became_active":"2024-03-04T07:33:56.542775+0000","last_became_peered":"2024-03-04T07:33:56.542775+0000","last_unstale":"2024-03-04T07:33:57.750083+0000","last_undegraded":"2024-03-04T07:33:57.750083+0000","last_fullsized":"2024-03-04T07:33:57.750083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:32:32.090568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671260+0000","last_change":"2024-03-04T07:33:55.545598+0000","last_active":"2024-03-04T07:34:12.671260+0000","last_peered":"2024-03-04T07:34:12.671260+0000","last_clean":"2024-03-04T07:34:12.671260+0000","last_became_active":"2024-03-04T07:33:55.545262+0000","last_became_peered":"2024-03-04T07:33:55.545262+0000","last_unstale":"2024-03-04T07:34:12.671260+0000","last_undegraded":"2024-03-04T07:34:12.671260+0000","last_fullsized":"2024-03-04T07:34:12.671260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:01.667243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523716+0000","last_change":"2024-03-04T07:33:56.550943+0000","last_active":"2024-03-04T07:33:57.523716+0000","last_peered":"2024-03-04T07:33:57.523716+0000","last_clean":"2024-03-04T07:33:57.523716+0000","last_became_active":"2024-03-04T07:33:56.550604+0000","last_became_peered":"2024-03-04T07:33:56.550604+0000","last_unstale":"2024-03-04T07:33:57.523716+0000","last_undegraded":"2024-03-04T07:33:57.523716+0000","last_fullsized":"2024-03-04T07:33:57.523716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:01:05.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.607065+0000","last_change":"2024-03-04T07:33:55.567220+0000","last_active":"2024-03-04T07:34:07.607065+0000","last_peered":"2024-03-04T07:34:07.607065+0000","last_clean":"2024-03-04T07:34:07.607065+0000","last_became_active":"2024-03-04T07:33:55.566600+0000","last_became_peered":"2024-03-04T07:33:55.566600+0000","last_unstale":"2024-03-04T07:34:07.607065+0000","last_undegraded":"2024-03-04T07:34:07.607065+0000","last_fullsized":"2024-03-04T07:34:07.607065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.876938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859267+0000","last_change":"2024-03-04T07:33:56.544263+0000","last_active":"2024-03-04T07:33:57.859267+0000","last_peered":"2024-03-04T07:33:57.859267+0000","last_clean":"2024-03-04T07:33:57.859267+0000","last_became_active":"2024-03-04T07:33:56.543836+0000","last_became_peered":"2024-03-04T07:33:56.543836+0000","last_unstale":"2024-03-04T07:33:57.859267+0000","last_undegraded":"2024-03-04T07:33:57.859267+0000","last_fullsized":"2024-03-04T07:33:57.859267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:49.839824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.602139+0000","last_change":"2024-03-04T07:33:55.566886+0000","last_active":"2024-03-04T07:34:07.602139+0000","last_peered":"2024-03-04T07:34:07.602139+0000","last_clean":"2024-03-04T07:34:07.602139+0000","last_became_active":"2024-03-04T07:33:55.566048+0000","last_became_peered":"2024-03-04T07:33:55.566048+0000","last_unstale":"2024-03-04T07:34:07.602139+0000","last_undegraded":"2024-03-04T07:34:07.602139+0000","last_fullsized":"2024-03-04T07:34:07.602139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:45:27.190438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.525876+0000","last_change":"2024-03-04T07:33:56.532666+0000","last_active":"2024-03-04T07:33:57.525876+0000","last_peered":"2024-03-04T07:33:57.525876+0000","last_clean":"2024-03-04T07:33:57.525876+0000","last_became_active":"2024-03-04T07:33:56.531901+0000","last_became_peered":"2024-03-04T07:33:56.531901+0000","last_unstale":"2024-03-04T07:33:57.525876+0000","last_undegraded":"2024-03-04T07:33:57.525876+0000","last_fullsized":"2024-03-04T07:33:57.525876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:42:04.545881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"21'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.602429+0000","last_change":"2024-03-04T07:33:55.557568+0000","last_active":"2024-03-04T07:34:07.602429+0000","last_peered":"2024-03-04T07:34:07.602429+0000","last_clean":"2024-03-04T07:34:07.602429+0000","last_became_active":"2024-03-04T07:33:55.557330+0000","last_became_peered":"2024-03-04T07:33:55.557330+0000","last_unstale":"2024-03-04T07:34:07.602429+0000","last_undegraded":"2024-03-04T07:34:07.602429+0000","last_fullsized":"2024-03-04T07:34:07.602429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:34.956318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523904+0000","last_change":"2024-03-04T07:33:56.552963+0000","last_active":"2024-03-04T07:33:57.523904+0000","last_peered":"2024-03-04T07:33:57.523904+0000","last_clean":"2024-03-04T07:33:57.523904+0000","last_became_active":"2024-03-04T07:33:56.552658+0000","last_became_peered":"2024-03-04T07:33:56.552658+0000","last_unstale":"2024-03-04T07:33:57.523904+0000","last_undegraded":"2024-03-04T07:33:57.523904+0000","last_fullsized":"2024-03-04T07:33:57.523904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:05.685496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.913351+0000","last_change":"2024-03-04T07:33:55.536480+0000","last_active":"2024-03-04T07:33:57.913351+0000","last_peered":"2024-03-04T07:33:57.913351+0000","last_clean":"2024-03-04T07:33:57.913351+0000","last_became_active":"2024-03-04T07:33:55.536139+0000","last_became_peered":"2024-03-04T07:33:55.536139+0000","last_unstale":"2024-03-04T07:33:57.913351+0000","last_undegraded":"2024-03-04T07:33:57.913351+0000","last_fullsized":"2024-03-04T07:33:57.913351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:12.778026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.523811+0000","last_change":"2024-03-04T07:33:56.546546+0000","last_active":"2024-03-04T07:33:57.523811+0000","last_peered":"2024-03-04T07:33:57.523811+0000","last_clean":"2024-03-04T07:33:57.523811+0000","last_became_active":"2024-03-04T07:33:56.546280+0000","last_became_peered":"2024-03-04T07:33:56.546280+0000","last_unstale":"2024-03-04T07:33:57.523811+0000","last_undegraded":"2024-03-04T07:33:57.523811+0000","last_fullsized":"2024-03-04T07:33:57.523811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:19:55.950163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:07.603451+0000","last_change":"2024-03-04T07:33:55.572540+0000","last_active":"2024-03-04T07:34:07.603451+0000","last_peered":"2024-03-04T07:34:07.603451+0000","last_clean":"2024-03-04T07:34:07.603451+0000","last_became_active":"2024-03-04T07:33:55.571612+0000","last_became_peered":"2024-03-04T07:33:55.571612+0000","last_unstale":"2024-03-04T07:34:07.603451+0000","last_undegraded":"2024-03-04T07:34:07.603451+0000","last_fullsized":"2024-03-04T07:34:07.603451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:01:43.741423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.640728+0000","last_change":"2024-03-04T07:33:56.532855+0000","last_active":"2024-03-04T07:33:58.640728+0000","last_peered":"2024-03-04T07:33:58.640728+0000","last_clean":"2024-03-04T07:33:58.640728+0000","last_became_active":"2024-03-04T07:33:56.532245+0000","last_became_peered":"2024-03-04T07:33:56.532245+0000","last_unstale":"2024-03-04T07:33:58.640728+0000","last_undegraded":"2024-03-04T07:33:58.640728+0000","last_fullsized":"2024-03-04T07:33:58.640728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:26.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642056+0000","last_change":"2024-03-04T07:33:56.563803+0000","last_active":"2024-03-04T07:33:58.642056+0000","last_peered":"2024-03-04T07:33:58.642056+0000","last_clean":"2024-03-04T07:33:58.642056+0000","last_became_active":"2024-03-04T07:33:56.563473+0000","last_became_peered":"2024-03-04T07:33:56.563473+0000","last_unstale":"2024-03-04T07:33:58.642056+0000","last_undegraded":"2024-03-04T07:33:58.642056+0000","last_fullsized":"2024-03-04T07:33:58.642056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:26.904436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.531549+0000","last_change":"2024-03-04T07:33:55.575688+0000","last_active":"2024-03-04T07:34:01.531549+0000","last_peered":"2024-03-04T07:34:01.531549+0000","last_clean":"2024-03-04T07:34:01.531549+0000","last_became_active":"2024-03-04T07:33:55.574766+0000","last_became_peered":"2024-03-04T07:33:55.574766+0000","last_unstale":"2024-03-04T07:34:01.531549+0000","last_undegraded":"2024-03-04T07:34:01.531549+0000","last_fullsized":"2024-03-04T07:34:01.531549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:32:19.708016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:58.642750+0000","last_change":"2024-03-04T07:33:56.563798+0000","last_active":"2024-03-04T07:33:58.642750+0000","last_peered":"2024-03-04T07:33:58.642750+0000","last_clean":"2024-03-04T07:33:58.642750+0000","last_became_active":"2024-03-04T07:33:56.563474+0000","last_became_peered":"2024-03-04T07:33:56.563474+0000","last_unstale":"2024-03-04T07:33:58.642750+0000","last_undegraded":"2024-03-04T07:33:58.642750+0000","last_fullsized":"2024-03-04T07:33:58.642750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:47:39.763001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.670842+0000","last_change":"2024-03-04T07:33:55.570016+0000","last_active":"2024-03-04T07:34:12.670842+0000","last_peered":"2024-03-04T07:34:12.670842+0000","last_clean":"2024-03-04T07:34:12.670842+0000","last_became_active":"2024-03-04T07:33:55.568749+0000","last_became_peered":"2024-03-04T07:33:55.568749+0000","last_unstale":"2024-03-04T07:34:12.670842+0000","last_undegraded":"2024-03-04T07:34:12.670842+0000","last_fullsized":"2024-03-04T07:34:12.670842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:01.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.859463+0000","last_change":"2024-03-04T07:33:56.562015+0000","last_active":"2024-03-04T07:33:57.859463+0000","last_peered":"2024-03-04T07:33:57.859463+0000","last_clean":"2024-03-04T07:33:57.859463+0000","last_became_active":"2024-03-04T07:33:56.561675+0000","last_became_peered":"2024-03-04T07:33:56.561675+0000","last_unstale":"2024-03-04T07:33:57.859463+0000","last_undegraded":"2024-03-04T07:33:57.859463+0000","last_fullsized":"2024-03-04T07:33:57.859463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:36:20.241414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:01.532052+0000","last_change":"2024-03-04T07:33:55.538337+0000","last_active":"2024-03-04T07:34:01.532052+0000","last_peered":"2024-03-04T07:34:01.532052+0000","last_clean":"2024-03-04T07:34:01.532052+0000","last_became_active":"2024-03-04T07:33:55.538010+0000","last_became_peered":"2024-03-04T07:33:55.538010+0000","last_unstale":"2024-03-04T07:34:01.532052+0000","last_undegraded":"2024-03-04T07:34:01.532052+0000","last_fullsized":"2024-03-04T07:34:01.532052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:47:15.711205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:33:57.524079+0000","last_change":"2024-03-04T07:33:56.559051+0000","last_active":"2024-03-04T07:33:57.524079+0000","last_peered":"2024-03-04T07:33:57.524079+0000","last_clean":"2024-03-04T07:33:57.524079+0000","last_became_active":"2024-03-04T07:33:56.558799+0000","last_became_peered":"2024-03-04T07:33:56.558799+0000","last_unstale":"2024-03-04T07:33:57.524079+0000","last_undegraded":"2024-03-04T07:33:57.524079+0000","last_fullsized":"2024-03-04T07:33:57.524079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:16:16.516631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'9","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-04T07:34:12.671146+0000","last_change":"2024-03-04T07:33:55.566294+0000","last_active":"2024-03-04T07:34:12.671146+0000","last_peered":"2024-03-04T07:34:12.671146+0000","last_clean":"2024-03-04T07:34:12.671146+0000","last_became_active":"2024-03-04T07:33:55.565941+0000","last_became_peered":"2024-03-04T07:33:55.565941+0000","last_unstale":"2024-03-04T07:34:12.671146+0000","last_undegraded":"2024-03-04T07:34:12.671146+0000","last_fullsized":"2024-03-04T07:34:12.671146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:03.115085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6553,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":13106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":76926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":74473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":823296,"data_stored":665213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":77150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542157,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":74907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":225280,"data_stored":74501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542157,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":74347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":819200,"data_stored":664557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T07:34:27.773 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:34:27.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.949+0000 7f779f200640 1 Processor -- start 2024-03-04T07:34:27.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.949+0000 7f779f200640 1 -- start start 2024-03-04T07:34:27.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779f200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77980730d0 0x7f779815c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779f200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779806f1b0 con 0x7f779806d420 2024-03-04T07:34:27.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779f200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7798071580 con 0x7f7798152a90 2024-03-04T07:34:27.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779f200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7798152e70 con 0x7f77980730d0 2024-03-04T07:34:27.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779d400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f779806d420 0x7f779815bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35924/0 (socket says 172.21.15.113:35924) 2024-03-04T07:34:27.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779d400640 1 -- 172.21.15.113:0/3666363723 learned_addr learned my addr 172.21.15.113:0/3666363723 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:27.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779e800640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77980730d0 0x7f779815c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017043889 0 0) 0x7f779806f1b0 con 0x7f779806d420 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7780003530 con 0x7f779806d420 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304902503 0 0) 0x7f7798071580 con 0x7f7798152a90 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f779806f1b0 con 0x7f7798152a90 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 514318792 0 0) 0x7f7780003530 con 0x7f779806d420 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77980730d0 msgr2=0x7f779815c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77980730d0 0x7f779815c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 >> v1:172.21.15.136:6789/0 conn(0x7f7798152a90 legacy=0x7f779806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798071860 con 0x7f779806d420 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f7788001680 con 0x7f779806d420 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 >> v1:172.21.15.113:6789/0 conn(0x7f779806d420 legacy=0x7f779815bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7780003b80 0x7f7780003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77980730d0 0x7f7780004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780004d50 0x7f77800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778000ba80 con 0x7f7780004d50 2024-03-04T07:34:27.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778000bc00 con 0x7f7780003b80 2024-03-04T07:34:27.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778000bdb0 con 0x7f77980730d0 2024-03-04T07:34:27.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779e800640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780004d50 0x7f77800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779de00640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7780003b80 0x7f7780003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779d400640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77980730d0 0x7f7780004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779e800640 1 -- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77980730d0 msgr2=0x7f7780004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:27.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779e800640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77980730d0 0x7f7780004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779e800640 1 -- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7780003b80 msgr2=0x7f7780003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:27.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779e800640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7780003b80 0x7f7780003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.953+0000 7f779e800640 1 -- 172.21.15.113:0/3666363723 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7780003530 con 0x7f7780004d50 2024-03-04T07:34:27.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779e800640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780004d50 0x7f77800051d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7794009fc0 tx=0x7f779401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8469f00bfc77dad6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:27.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940b2070 con 0x7f7780004d50 2024-03-04T07:34:27.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77940a5b80 con 0x7f7780004d50 2024-03-04T07:34:27.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779ca00640 1 -- 172.21.15.113:0/3666363723 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794009c10 con 0x7f7780004d50 2024-03-04T07:34:27.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779f200640 1 -- 172.21.15.113:0/3666363723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780004d50 msgr2=0x7f77800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779f200640 1 --2- 172.21.15.113:0/3666363723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7780004d50 0x7f77800051d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7794009fc0 tx=0x7f779401cc80 comp rx=0 tx=0).stop 2024-03-04T07:34:27.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779e800640 1 -- 172.21.15.113:0/3666363723 reap_dead start 2024-03-04T07:34:27.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779f200640 1 -- 172.21.15.113:0/3666363723 shutdown_connections 2024-03-04T07:34:27.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779f200640 1 -- 172.21.15.113:0/3666363723 >> 172.21.15.113:0/3666363723 conn(0x7f779806e2a0 msgr2=0x7f779806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:27.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779f200640 1 -- 172.21.15.113:0/3666363723 shutdown_connections 2024-03-04T07:34:27.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779f200640 1 -- 172.21.15.113:0/3666363723 wait complete. 2024-03-04T07:34:27.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779f200640 1 Processor -- start 2024-03-04T07:34:27.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.957+0000 7f779f200640 1 -- start start 2024-03-04T07:34:27.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779f200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f779814b840 0x7f779815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f779814bc20 0x7f77981e2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77981598b0 0x7f77981e30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7798071330 con 0x7f77981598b0 2024-03-04T07:34:27.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779806ed90 con 0x7f779814bc20 2024-03-04T07:34:27.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779f200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779806ef10 con 0x7f779814b840 2024-03-04T07:34:27.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f779814b840 0x7f779815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f779814b840 0x7f779815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:45082/0 (socket says 172.21.15.113:45082) 2024-03-04T07:34:27.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f779814bc20 0x7f77981e2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779de00640 1 -- 172.21.15.113:0/2116068455 learned_addr learned my addr 172.21.15.113:0/2116068455 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779e800640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77981598b0 0x7f77981e30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779e800640 1 -- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f779814b840 msgr2=0x7f779815c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779e800640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f779814b840 0x7f779815c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779e800640 1 -- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f779814bc20 msgr2=0x7f77981e2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779e800640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f779814bc20 0x7f77981e2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779e800640 1 -- 172.21.15.113:0/2116068455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77981f0700 con 0x7f77981598b0 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779e800640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77981598b0 0x7f77981e30d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f77940ac000 tx=0x7f7794009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f7792000640 1 -- 172.21.15.113:0/2116068455 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940b2040 con 0x7f77981598b0 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f7792000640 1 -- 172.21.15.113:0/2116068455 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77940a5980 con 0x7f77981598b0 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f7792000640 1 -- 172.21.15.113:0/2116068455 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794009c10 con 0x7f77981598b0 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779f200640 1 -- 172.21.15.113:0/2116068455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77981f09e0 con 0x7f77981598b0 2024-03-04T07:34:27.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779f200640 1 -- 172.21.15.113:0/2116068455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77981f0fa0 con 0x7f77981598b0 2024-03-04T07:34:27.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f779f200640 1 -- 172.21.15.113:0/2116068455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779806c2d0 con 0x7f77981598b0 2024-03-04T07:34:27.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.961+0000 7f7792000640 1 -- 172.21.15.113:0/2116068455 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77940be020 con 0x7f77981598b0 2024-03-04T07:34:27.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.965+0000 7f7792000640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f777c066f80 0x7f777c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:27.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.965+0000 7f7792000640 1 -- 172.21.15.113:0/2116068455 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f779411efe0 con 0x7f77981598b0 2024-03-04T07:34:27.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.965+0000 7f779de00640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f777c066f80 0x7f777c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:27.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.965+0000 7f779de00640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f777c066f80 0x7f777c069440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f778c0019b0 tx=0x7f778c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:27.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:27.969+0000 7f7792000640 1 -- 172.21.15.113:0/2116068455 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f779806c2d0 con 0x7f77981598b0 2024-03-04T07:34:28.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.129+0000 7f779f200640 1 -- 172.21.15.113:0/2116068455 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7798075a80 con 0x7f77981598b0 2024-03-04T07:34:28.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.129+0000 7f7792000640 1 -- 172.21.15.113:0/2116068455 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f77940c5020 con 0x7f77981598b0 2024-03-04T07:34:28.135 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:28.136 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:57.517935+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:34:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 -- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f777c066f80 msgr2=0x7f777c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f777c066f80 0x7f777c069440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f778c0019b0 tx=0x7f778c009040 comp rx=0 tx=0).stop 2024-03-04T07:34:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 -- 172.21.15.113:0/2116068455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77981598b0 msgr2=0x7f77981e30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77981598b0 0x7f77981e30d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f77940ac000 tx=0x7f7794009750 comp rx=0 tx=0).stop 2024-03-04T07:34:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 -- 172.21.15.113:0/2116068455 shutdown_connections 2024-03-04T07:34:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f777c066f80 0x7f777c069440 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77981598b0 0x7f77981e30d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f779814bc20 0x7f77981e2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 --2- 172.21.15.113:0/2116068455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f779814b840 0x7f779815c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 -- 172.21.15.113:0/2116068455 >> 172.21.15.113:0/2116068455 conn(0x7f779806e2a0 msgr2=0x7f77980c8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:28.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 -- 172.21.15.113:0/2116068455 shutdown_connections 2024-03-04T07:34:28.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.133+0000 7f779f200640 1 -- 172.21.15.113:0/2116068455 wait complete. 2024-03-04T07:34:28.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T07:33:41.507700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T07:33:53.680465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T07:33:54.892043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T07:34:28.159 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T07:34:28.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d0ba00640 1 Processor -- start 2024-03-04T07:34:28.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d0ba00640 1 -- start start 2024-03-04T07:34:28.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d0ba00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d04158d60 0x7f0d041591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:28.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d0ba00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d040cd2c0 con 0x7f0d040ca6a0 2024-03-04T07:34:28.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d0ba00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d041596f0 con 0x7f0d04152b40 2024-03-04T07:34:28.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d0ba00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d040cd560 con 0x7f0d04158d60 2024-03-04T07:34:28.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d09a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d04158d60 0x7f0d041591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:28.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d09a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d04158d60 0x7f0d041591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46664/0 (socket says 172.21.15.113:46664) 2024-03-04T07:34:28.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0cfbe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0d040ca6a0 0x7f0d0406f860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35930/0 (socket says 172.21.15.113:35930) 2024-03-04T07:34:28.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d09a00640 1 -- 172.21.15.113:0/1693798946 learned_addr learned my addr 172.21.15.113:0/1693798946 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:28.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d09a00640 1 -- 172.21.15.113:0/1693798946 >> v1:172.21.15.136:6789/0 conn(0x7f0d04152b40 legacy=0x7f0d04071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d09a00640 1 -- 172.21.15.113:0/1693798946 >> v1:172.21.15.113:6789/0 conn(0x7f0d040ca6a0 legacy=0x7f0d0406f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.297+0000 7f0d09a00640 1 -- 172.21.15.113:0/1693798946 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d040cd840 con 0x7f0d04158d60 2024-03-04T07:34:28.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d09a00640 1 --2- 172.21.15.113:0/1693798946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04158d60 0x7f0d041591b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009e50 tx=0x7f0cfc01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c35fe4408ba134f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:28.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d03e00640 1 -- 172.21.15.113:0/1693798946 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0b4070 con 0x7f0d04158d60 2024-03-04T07:34:28.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d03e00640 1 -- 172.21.15.113:0/1693798946 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc0af400 con 0x7f0d04158d60 2024-03-04T07:34:28.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d03e00640 1 -- 172.21.15.113:0/1693798946 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0af7d0 con 0x7f0d04158d60 2024-03-04T07:34:28.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1693798946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04158d60 msgr2=0x7f0d041591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d0ba00640 1 --2- 172.21.15.113:0/1693798946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04158d60 0x7f0d041591b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009e50 tx=0x7f0cfc01f210 comp rx=0 tx=0).stop 2024-03-04T07:34:28.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1693798946 shutdown_connections 2024-03-04T07:34:28.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d0ba00640 1 --2- 172.21.15.113:0/1693798946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04158d60 0x7f0d041591b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1693798946 >> 172.21.15.113:0/1693798946 conn(0x7f0d0406e240 msgr2=0x7f0d0406e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:28.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1693798946 shutdown_connections 2024-03-04T07:34:28.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1693798946 wait complete. 2024-03-04T07:34:28.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d0ba00640 1 Processor -- start 2024-03-04T07:34:28.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.305+0000 7f0d0ba00640 1 -- start start 2024-03-04T07:34:28.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d0ba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d040ca6a0 0x7f0d0406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d0ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152b40 0x7f0d040aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d0ba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d04158d60 0x7f0d041f0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d0ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040723c0 con 0x7f0d04152b40 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d0ba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040cde00 con 0x7f0d04158d60 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d0ba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0406ff70 con 0x7f0d040ca6a0 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0cfbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152b40 0x7f0d040aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0cfbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152b40 0x7f0d040aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46680/0 (socket says 172.21.15.113:46680) 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1701463651 learned_addr learned my addr 172.21.15.113:0/1701463651 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d040ca6a0 msgr2=0x7f0d0406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0cfbe00640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d040ca6a0 0x7f0d0406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d04158d60 msgr2=0x7f0d041f0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0cfbe00640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d04158d60 0x7f0d041f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1701463651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d040b0e20 con 0x7f0d04152b40 2024-03-04T07:34:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0cfbe00640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152b40 0x7f0d040aece0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00c450 tx=0x7f0cec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d01600640 1 -- 172.21.15.113:0/1701463651 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec018070 con 0x7f0d04152b40 2024-03-04T07:34:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d01600640 1 -- 172.21.15.113:0/1701463651 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec007480 con 0x7f0d04152b40 2024-03-04T07:34:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d01600640 1 -- 172.21.15.113:0/1701463651 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0047f0 con 0x7f0d04152b40 2024-03-04T07:34:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1701463651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d040b1100 con 0x7f0d04152b40 2024-03-04T07:34:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1701463651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d040b16c0 con 0x7f0d04152b40 2024-03-04T07:34:28.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1701463651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d04075c00 con 0x7f0d04152b40 2024-03-04T07:34:28.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.309+0000 7f0d01600640 1 -- 172.21.15.113:0/1701463651 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0cec004990 con 0x7f0d04152b40 2024-03-04T07:34:28.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.313+0000 7f0d01600640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce0066f90 0x7f0ce0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:28.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.313+0000 7f0d01600640 1 -- 172.21.15.113:0/1701463651 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cec0841c0 con 0x7f0d04152b40 2024-03-04T07:34:28.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.313+0000 7f0d09000640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce0066f90 0x7f0ce0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:28.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.313+0000 7f0d09000640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce0066f90 0x7f0ce0069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000f70 tx=0x7f0cf40079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:28.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.317+0000 7f0d01600640 1 -- 172.21.15.113:0/1701463651 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0cec0551d0 con 0x7f0d04152b40 2024-03-04T07:34:28.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1701463651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d040cac60 con 0x7f0d04152b40 2024-03-04T07:34:28.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d01600640 1 -- 172.21.15.113:0/1701463651 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0cec054ad0 con 0x7f0d04152b40 2024-03-04T07:34:28.529 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-03-04T07:34:28.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce0066f90 msgr2=0x7f0ce0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce0066f90 0x7f0ce0069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000f70 tx=0x7f0cf40079c0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1701463651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152b40 msgr2=0x7f0d040aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152b40 0x7f0d040aece0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00c450 tx=0x7f0cec00c910 comp rx=0 tx=0).stop 2024-03-04T07:34:28.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1701463651 shutdown_connections 2024-03-04T07:34:28.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d04158d60 0x7f0d041f0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce0066f90 0x7f0ce0069450 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152b40 0x7f0d040aece0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 --2- 172.21.15.113:0/1701463651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d040ca6a0 0x7f0d0406c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1701463651 >> 172.21.15.113:0/1701463651 conn(0x7f0d0406e240 msgr2=0x7f0d04150340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:28.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1701463651 shutdown_connections 2024-03-04T07:34:28.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.525+0000 7f0d0ba00640 1 -- 172.21.15.113:0/1701463651 wait complete. 2024-03-04T07:34:28.553 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T07:34:28.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efcd7400640 1 Processor -- start 2024-03-04T07:34:28.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efcd7400640 1 -- start start 2024-03-04T07:34:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efcd7400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efcd00c19e0 0x7efcd00c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efcd7400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd01577d0 con 0x7efcd01519b0 2024-03-04T07:34:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efcd7400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd00c7760 con 0x7efcd0157cd0 2024-03-04T07:34:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efcd7400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd00c7a40 con 0x7efcd00c19e0 2024-03-04T07:34:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efcd4a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efcd01519b0 0x7efcd00cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35936/0 (socket says 172.21.15.113:35936) 2024-03-04T07:34:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efcd4a00640 1 -- 172.21.15.113:0/2278478930 learned_addr learned my addr 172.21.15.113:0/2278478930 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efccfe00640 1 --2- 172.21.15.113:0/2278478930 >> v2:172.21.15.136:3300/0 conn(0x7efcd00c19e0 0x7efcd00c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efccf400640 1 -- 172.21.15.113:0/2278478930 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818013843 0 0) 0x7efcd01577d0 con 0x7efcd01519b0 2024-03-04T07:34:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efccf400640 1 -- 172.21.15.113:0/2278478930 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcb8003530 con 0x7efcd01519b0 2024-03-04T07:34:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efccf400640 1 -- 172.21.15.113:0/2278478930 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 126967946 0 0) 0x7efcb8003530 con 0x7efcd01519b0 2024-03-04T07:34:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efccfe00640 1 -- 172.21.15.113:0/2278478930 >> v1:172.21.15.136:6789/0 conn(0x7efcd0157cd0 legacy=0x7efcd0158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efccfe00640 1 -- 172.21.15.113:0/2278478930 >> v1:172.21.15.113:6789/0 conn(0x7efcd01519b0 legacy=0x7efcd00cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.697+0000 7efccfe00640 1 -- 172.21.15.113:0/2278478930 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd00c1090 con 0x7efcd00c19e0 2024-03-04T07:34:28.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.701+0000 7efccfe00640 1 --2- 172.21.15.113:0/2278478930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcd00c19e0 0x7efcd00c8370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0097c0 tx=0x7efcbc01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2836fd83c3e9d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:28.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efccf400640 1 -- 172.21.15.113:0/2278478930 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0b0070 con 0x7efcd00c19e0 2024-03-04T07:34:28.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efccf400640 1 -- 172.21.15.113:0/2278478930 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc0ab400 con 0x7efcd00c19e0 2024-03-04T07:34:28.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efccf400640 1 -- 172.21.15.113:0/2278478930 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4001680 con 0x7efcd00c19e0 2024-03-04T07:34:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 -- 172.21.15.113:0/2278478930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcd00c19e0 msgr2=0x7efcd00c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 --2- 172.21.15.113:0/2278478930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcd00c19e0 0x7efcd00c8370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0097c0 tx=0x7efcbc01c3c0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 -- 172.21.15.113:0/2278478930 shutdown_connections 2024-03-04T07:34:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 --2- 172.21.15.113:0/2278478930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcd00c19e0 0x7efcd00c8370 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 -- 172.21.15.113:0/2278478930 >> 172.21.15.113:0/2278478930 conn(0x7efcd0066db0 msgr2=0x7efcd00671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 -- 172.21.15.113:0/2278478930 shutdown_connections 2024-03-04T07:34:28.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 -- 172.21.15.113:0/2278478930 wait complete. 2024-03-04T07:34:28.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 Processor -- start 2024-03-04T07:34:28.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 -- start start 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcd00c19e0 0x7efcd014c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd01519b0 0x7efcd0150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcd0157cd0 0x7efcd014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00ccc80 con 0x7efcd01519b0 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00ccb00 con 0x7efcd00c19e0 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd7400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd0151d90 con 0x7efcd0157cd0 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efccfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd01519b0 0x7efcd0150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efccfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd01519b0 0x7efcd0150000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46688/0 (socket says 172.21.15.113:46688) 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efccfe00640 1 -- 172.21.15.113:0/1515234443 learned_addr learned my addr 172.21.15.113:0/1515234443 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd4a00640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcd00c19e0 0x7efcd014c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:28.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.705+0000 7efcd5400640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcd0157cd0 0x7efcd014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efccfe00640 1 -- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcd0157cd0 msgr2=0x7efcd014cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efccfe00640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcd0157cd0 0x7efcd014cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efccfe00640 1 -- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcd00c19e0 msgr2=0x7efcd014c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efccfe00640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcd00c19e0 0x7efcd014c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efccfe00640 1 -- 172.21.15.113:0/1515234443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd014df30 con 0x7efcd01519b0 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efccfe00640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd01519b0 0x7efcd0150000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efcbc018fd0 tx=0x7efcbc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efcccc00640 1 -- 172.21.15.113:0/1515234443 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0b0040 con 0x7efcd01519b0 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efcccc00640 1 -- 172.21.15.113:0/1515234443 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc0abce0 con 0x7efcd01519b0 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efcccc00640 1 -- 172.21.15.113:0/1515234443 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0b5680 con 0x7efcd01519b0 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efcd7400640 1 -- 172.21.15.113:0/1515234443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd01f7e60 con 0x7efcd01519b0 2024-03-04T07:34:28.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efcd7400640 1 -- 172.21.15.113:0/1515234443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd0154e20 con 0x7efcd01519b0 2024-03-04T07:34:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efcc2000640 1 -- 172.21.15.113:0/1515234443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca40052a0 con 0x7efcd01519b0 2024-03-04T07:34:28.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efcccc00640 1 -- 172.21.15.113:0/1515234443 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efcbc0bd020 con 0x7efcd01519b0 2024-03-04T07:34:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efcccc00640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efcb0066f60 0x7efcb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efcccc00640 1 -- 172.21.15.113:0/1515234443 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcbc11e970 con 0x7efcd01519b0 2024-03-04T07:34:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.709+0000 7efcd4a00640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efcb0066f60 0x7efcb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:28.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.713+0000 7efcd4a00640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efcb0066f60 0x7efcb0069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efcc4005890 tx=0x7efcc4005800 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:28.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.721+0000 7efcccc00640 1 -- 172.21.15.113:0/1515234443 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efcbc015fb0 con 0x7efcd01519b0 2024-03-04T07:34:28.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.881+0000 7efcc2000640 1 -- 172.21.15.113:0/1515234443 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efca4005da0 con 0x7efcd01519b0 2024-03-04T07:34:28.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.881+0000 7efcccc00640 1 -- 172.21.15.113:0/1515234443 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7efcbc0ef850 con 0x7efcd01519b0 2024-03-04T07:34:28.886 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-03-04T07:34:28.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.881+0000 7efcc2000640 1 -- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efcb0066f60 msgr2=0x7efcb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.881+0000 7efcc2000640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efcb0066f60 0x7efcb0069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efcc4005890 tx=0x7efcc4005800 comp rx=0 tx=0).stop 2024-03-04T07:34:28.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.881+0000 7efcc2000640 1 -- 172.21.15.113:0/1515234443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd01519b0 msgr2=0x7efcd0150000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:28.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.881+0000 7efcc2000640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd01519b0 0x7efcd0150000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efcbc018fd0 tx=0x7efcbc019160 comp rx=0 tx=0).stop 2024-03-04T07:34:28.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.885+0000 7efcc2000640 1 -- 172.21.15.113:0/1515234443 shutdown_connections 2024-03-04T07:34:28.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.885+0000 7efcc2000640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcd0157cd0 0x7efcd014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.885+0000 7efcc2000640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efcb0066f60 0x7efcb0069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.885+0000 7efcc2000640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd01519b0 0x7efcd0150000 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.885+0000 7efcc2000640 1 --2- 172.21.15.113:0/1515234443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcd00c19e0 0x7efcd014c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:28.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.885+0000 7efcc2000640 1 -- 172.21.15.113:0/1515234443 >> 172.21.15.113:0/1515234443 conn(0x7efcd0066db0 msgr2=0x7efcd00c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:28.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.885+0000 7efcc2000640 1 -- 172.21.15.113:0/1515234443 shutdown_connections 2024-03-04T07:34:28.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:28.885+0000 7efcc2000640 1 -- 172.21.15.113:0/1515234443 wait complete. 2024-03-04T07:34:28.907 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T07:34:29.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.061+0000 7fa4bcc00640 1 Processor -- start 2024-03-04T07:34:29.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.061+0000 7fa4bcc00640 1 -- start start 2024-03-04T07:34:29.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4bcc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4b8154a60 0x7fa4b815b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4bcc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b815b7b0 con 0x7fa4b8072550 2024-03-04T07:34:29.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4bcc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b80ccf80 con 0x7fa4b806cab0 2024-03-04T07:34:29.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4bcc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b80717d0 con 0x7fa4b8154a60 2024-03-04T07:34:29.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4b7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4b8154a60 0x7fa4b815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4b7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa4b8154a60 0x7fa4b815b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46694/0 (socket says 172.21.15.113:46694) 2024-03-04T07:34:29.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4b7400640 1 -- 172.21.15.113:0/857266166 learned_addr learned my addr 172.21.15.113:0/857266166 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:29.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4aea00640 1 --1- 172.21.15.113:0/857266166 >> v1:172.21.15.113:6789/0 conn(0x7fa4b8072550 0x7fa4b815e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35946/0 (socket says 172.21.15.113:35946) 2024-03-04T07:34:29.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4b7400640 1 -- 172.21.15.113:0/857266166 >> v1:172.21.15.136:6789/0 conn(0x7fa4b806cab0 legacy=0x7fa4b8165170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4b7400640 1 -- 172.21.15.113:0/857266166 >> v1:172.21.15.113:6789/0 conn(0x7fa4b8072550 legacy=0x7fa4b815e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.065+0000 7fa4b7400640 1 -- 172.21.15.113:0/857266166 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b80cd260 con 0x7fa4b8154a60 2024-03-04T07:34:29.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4b7400640 1 --2- 172.21.15.113:0/857266166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b8154a60 0x7fa4b815b270 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009870 tx=0x7fa4a801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5acc22bab501063 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:29.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4b6a00640 1 -- 172.21.15.113:0/857266166 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80a7070 con 0x7fa4b8154a60 2024-03-04T07:34:29.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4b6a00640 1 -- 172.21.15.113:0/857266166 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a80b0070 con 0x7fa4b8154a60 2024-03-04T07:34:29.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4b6a00640 1 -- 172.21.15.113:0/857266166 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80ab890 con 0x7fa4b8154a60 2024-03-04T07:34:29.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4bcc00640 1 -- 172.21.15.113:0/857266166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b8154a60 msgr2=0x7fa4b815b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4bcc00640 1 --2- 172.21.15.113:0/857266166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b8154a60 0x7fa4b815b270 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009870 tx=0x7fa4a801c560 comp rx=0 tx=0).stop 2024-03-04T07:34:29.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4bcc00640 1 -- 172.21.15.113:0/857266166 shutdown_connections 2024-03-04T07:34:29.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4bcc00640 1 --2- 172.21.15.113:0/857266166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b8154a60 0x7fa4b815b270 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4bcc00640 1 -- 172.21.15.113:0/857266166 >> 172.21.15.113:0/857266166 conn(0x7fa4b806b7f0 msgr2=0x7fa4b80cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:29.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4bcc00640 1 -- 172.21.15.113:0/857266166 shutdown_connections 2024-03-04T07:34:29.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4bcc00640 1 -- 172.21.15.113:0/857266166 wait complete. 2024-03-04T07:34:29.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4bcc00640 1 Processor -- start 2024-03-04T07:34:29.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.069+0000 7fa4bcc00640 1 -- start start 2024-03-04T07:34:29.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4bcc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b806cab0 0x7fa4b8153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4bcc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa4b8072550 0x7fa4b814c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4bcc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8154a60 0x7fa4b814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4bcc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b8072960 con 0x7fa4b806cab0 2024-03-04T07:34:29.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4bcc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b8154e40 con 0x7fa4b8154a60 2024-03-04T07:34:29.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4bcc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b8072ae0 con 0x7fa4b8072550 2024-03-04T07:34:29.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4b7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b806cab0 0x7fa4b8153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4b7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b806cab0 0x7fa4b8153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46702/0 (socket says 172.21.15.113:46702) 2024-03-04T07:34:29.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4b7400640 1 -- 172.21.15.113:0/4271036456 learned_addr learned my addr 172.21.15.113:0/4271036456 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:29.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4aea00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa4b8072550 0x7fa4b814c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4b7e00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8154a60 0x7fa4b814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4b7400640 1 -- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa4b8072550 msgr2=0x7fa4b814c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4b7400640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa4b8072550 0x7fa4b814c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4b7400640 1 -- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8154a60 msgr2=0x7fa4b814ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4b7400640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8154a60 0x7fa4b814ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4b7400640 1 -- 172.21.15.113:0/4271036456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b81ee1d0 con 0x7fa4b806cab0 2024-03-04T07:34:29.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4b7400640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b806cab0 0x7fa4b8153170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80ad040 tx=0x7fa4a801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:29.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4afe00640 1 -- 172.21.15.113:0/4271036456 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80abe60 con 0x7fa4b806cab0 2024-03-04T07:34:29.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4afe00640 1 -- 172.21.15.113:0/4271036456 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a80b4820 con 0x7fa4b806cab0 2024-03-04T07:34:29.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4bcc00640 1 -- 172.21.15.113:0/4271036456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b81f7c20 con 0x7fa4b806cab0 2024-03-04T07:34:29.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4afe00640 1 -- 172.21.15.113:0/4271036456 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80b4c10 con 0x7fa4b806cab0 2024-03-04T07:34:29.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4bcc00640 1 -- 172.21.15.113:0/4271036456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b814c150 con 0x7fa4b806cab0 2024-03-04T07:34:29.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4bcc00640 1 -- 172.21.15.113:0/4271036456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b8075a80 con 0x7fa4b806cab0 2024-03-04T07:34:29.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.073+0000 7fa4afe00640 1 -- 172.21.15.113:0/4271036456 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa4a80c7390 con 0x7fa4b806cab0 2024-03-04T07:34:29.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.077+0000 7fa4afe00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa494066f60 0x7fa494069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.077+0000 7fa4afe00640 1 -- 172.21.15.113:0/4271036456 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4a811f740 con 0x7fa4b806cab0 2024-03-04T07:34:29.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.077+0000 7fa4aea00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa494066f60 0x7fa494069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.077+0000 7fa4aea00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa494066f60 0x7fa494069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8158360 tx=0x7fa4a0007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:29.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.081+0000 7fa4afe00640 1 -- 172.21.15.113:0/4271036456 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa4a80c4060 con 0x7fa4b806cab0 2024-03-04T07:34:29.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 -- 172.21.15.113:0/4271036456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4b80c8140 con 0x7fa4b806cab0 2024-03-04T07:34:29.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4afe00640 1 -- 172.21.15.113:0/4271036456 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4a8016150 con 0x7fa4b806cab0 2024-03-04T07:34:29.241 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-03-04T07:34:29.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 -- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa494066f60 msgr2=0x7fa494069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa494066f60 0x7fa494069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8158360 tx=0x7fa4a0007cb0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 -- 172.21.15.113:0/4271036456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b806cab0 msgr2=0x7fa4b8153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b806cab0 0x7fa4b8153170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80ad040 tx=0x7fa4a801ada0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 -- 172.21.15.113:0/4271036456 shutdown_connections 2024-03-04T07:34:29.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8154a60 0x7fa4b814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa4b8072550 0x7fa4b814c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa494066f60 0x7fa494069420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 --2- 172.21.15.113:0/4271036456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4b806cab0 0x7fa4b8153170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 -- 172.21.15.113:0/4271036456 >> 172.21.15.113:0/4271036456 conn(0x7fa4b806b7f0 msgr2=0x7fa4b80cac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:29.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 -- 172.21.15.113:0/4271036456 shutdown_connections 2024-03-04T07:34:29.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.237+0000 7fa4bcc00640 1 -- 172.21.15.113:0/4271036456 wait complete. 2024-03-04T07:34:29.262 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:34:29.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b4400640 1 Processor -- start 2024-03-04T07:34:29.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b4400640 1 -- start start 2024-03-04T07:34:29.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b4400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97ac154d00 0x7f97ac15b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b4400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97ac0c3bd0 0x7f97ac0c4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b4400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac06d480 con 0x7f97ac06f5b0 2024-03-04T07:34:29.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b4400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac15ba50 con 0x7f97ac154d00 2024-03-04T07:34:29.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b4400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0c4540 con 0x7f97ac0c3bd0 2024-03-04T07:34:29.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b1a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97ac154d00 0x7f97ac15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b2400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97ac0c3bd0 0x7f97ac0c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b1000640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f97ac06f5b0 0x7f97ac0ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:58232/0 (socket says 172.21.15.113:58232) 2024-03-04T07:34:29.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b1000640 1 -- 172.21.15.113:0/632381424 learned_addr learned my addr 172.21.15.113:0/632381424 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:34:29.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b1a00640 1 -- 172.21.15.113:0/632381424 >> v2:172.21.15.136:3300/0 conn(0x7f97ac0c3bd0 msgr2=0x7f97ac0c4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:29.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b1a00640 1 --2- 172.21.15.113:0/632381424 >> v2:172.21.15.136:3300/0 conn(0x7f97ac0c3bd0 0x7f97ac0c4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b1a00640 1 -- 172.21.15.113:0/632381424 >> v1:172.21.15.136:6789/0 conn(0x7f97ac06f5b0 legacy=0x7f97ac0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.393+0000 7f97b1a00640 1 -- 172.21.15.113:0/632381424 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac06c590 con 0x7f97ac154d00 2024-03-04T07:34:29.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97b1a00640 1 --2- 172.21.15.113:0/632381424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 0x7f97ac15b510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0c9100 tx=0x7f97a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b932a7bf747e4703 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:29.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97abe00640 1 -- 172.21.15.113:0/632381424 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a00b0070 con 0x7f97ac154d00 2024-03-04T07:34:29.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97abe00640 1 -- 172.21.15.113:0/632381424 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a00ab600 con 0x7f97ac154d00 2024-03-04T07:34:29.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97abe00640 1 -- 172.21.15.113:0/632381424 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a00ab9d0 con 0x7f97ac154d00 2024-03-04T07:34:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97b4400640 1 -- 172.21.15.113:0/632381424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 msgr2=0x7f97ac15b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97b4400640 1 --2- 172.21.15.113:0/632381424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 0x7f97ac15b510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0c9100 tx=0x7f97a001c480 comp rx=0 tx=0).stop 2024-03-04T07:34:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97b4400640 1 -- 172.21.15.113:0/632381424 shutdown_connections 2024-03-04T07:34:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97b4400640 1 --2- 172.21.15.113:0/632381424 >> v2:172.21.15.136:3300/0 conn(0x7f97ac0c3bd0 0x7f97ac0c4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97b4400640 1 --2- 172.21.15.113:0/632381424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 0x7f97ac15b510 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97b4400640 1 -- 172.21.15.113:0/632381424 >> 172.21.15.113:0/632381424 conn(0x7f97ac066db0 msgr2=0x7f97ac0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97b4400640 1 -- 172.21.15.113:0/632381424 shutdown_connections 2024-03-04T07:34:29.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.397+0000 7f97b4400640 1 -- 172.21.15.113:0/632381424 wait complete. 2024-03-04T07:34:29.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.401+0000 7f97b4400640 1 Processor -- start 2024-03-04T07:34:29.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.401+0000 7f97b4400640 1 -- start start 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.401+0000 7f97b4400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac06f5b0 0x7f97ac0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.401+0000 7f97b4400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97ac0c3bd0 0x7f97ac1504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.401+0000 7f97b4400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 0x7f97ac15d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.401+0000 7f97b4400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac1550e0 con 0x7f97ac154d00 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.401+0000 7f97b4400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac06d110 con 0x7f97ac06f5b0 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.401+0000 7f97b4400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac06c170 con 0x7f97ac0c3bd0 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 0x7f97ac15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 0x7f97ac15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46722/0 (socket says 172.21.15.113:46722) 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b2400640 1 -- 172.21.15.113:0/1785044284 learned_addr learned my addr 172.21.15.113:0/1785044284 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b1000640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97ac0c3bd0 0x7f97ac1504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b1a00640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac06f5b0 0x7f97ac0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b2400640 1 -- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97ac0c3bd0 msgr2=0x7f97ac1504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b2400640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97ac0c3bd0 0x7f97ac1504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b2400640 1 -- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac06f5b0 msgr2=0x7f97ac0b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b2400640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac06f5b0 0x7f97ac0b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b2400640 1 -- 172.21.15.113:0/1785044284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0b0220 con 0x7f97ac154d00 2024-03-04T07:34:29.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b2400640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 0x7f97ac15d360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f97a400efc0 tx=0x7f97a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:29.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97a9600640 1 -- 172.21.15.113:0/1785044284 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4018020 con 0x7f97ac154d00 2024-03-04T07:34:29.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97a9600640 1 -- 172.21.15.113:0/1785044284 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a4014480 con 0x7f97ac154d00 2024-03-04T07:34:29.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97a9600640 1 -- 172.21.15.113:0/1785044284 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4014870 con 0x7f97ac154d00 2024-03-04T07:34:29.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b4400640 1 -- 172.21.15.113:0/1785044284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac0b1350 con 0x7f97ac154d00 2024-03-04T07:34:29.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b4400640 1 -- 172.21.15.113:0/1785044284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac15c020 con 0x7f97ac154d00 2024-03-04T07:34:29.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97b4400640 1 -- 172.21.15.113:0/1785044284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac075c00 con 0x7f97ac154d00 2024-03-04T07:34:29.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97a9600640 1 -- 172.21.15.113:0/1785044284 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97a401b020 con 0x7f97ac154d00 2024-03-04T07:34:29.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.405+0000 7f97a9600640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f60 0x7f978c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.409+0000 7f97a9600640 1 -- 172.21.15.113:0/1785044284 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97a4085d80 con 0x7f97ac154d00 2024-03-04T07:34:29.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.409+0000 7f97b1a00640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f60 0x7f978c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.409+0000 7f97b1a00640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f60 0x7f978c069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97a00ad000 tx=0x7f97a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:29.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.413+0000 7f97a9600640 1 -- 172.21.15.113:0/1785044284 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f97a4022030 con 0x7f97ac154d00 2024-03-04T07:34:29.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.613+0000 7f97b4400640 1 -- 172.21.15.113:0/1785044284 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97ac06df20 con 0x7f97ac154d00 2024-03-04T07:34:29.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97a9600640 1 -- 172.21.15.113:0/1785044284 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f97a4010040 con 0x7f97ac154d00 2024-03-04T07:34:29.621 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:29.621 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:34:29.621 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 -- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f60 msgr2=0x7f978c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f60 0x7f978c069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97a00ad000 tx=0x7f97a0008040 comp rx=0 tx=0).stop 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 -- 172.21.15.113:0/1785044284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 msgr2=0x7f97ac15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 0x7f97ac15d360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f97a400efc0 tx=0x7f97a400c490 comp rx=0 tx=0).stop 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 -- 172.21.15.113:0/1785044284 shutdown_connections 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f60 0x7f978c069420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97ac154d00 0x7f97ac15d360 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97ac0c3bd0 0x7f97ac1504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 --2- 172.21.15.113:0/1785044284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97ac06f5b0 0x7f97ac0b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 -- 172.21.15.113:0/1785044284 >> 172.21.15.113:0/1785044284 conn(0x7f97ac066db0 msgr2=0x7f97ac0cd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 -- 172.21.15.113:0/1785044284 shutdown_connections 2024-03-04T07:34:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.617+0000 7f97b4400640 1 -- 172.21.15.113:0/1785044284 wait complete. 2024-03-04T07:34:29.641 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T07:34:29.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd04ea00640 1 Processor -- start 2024-03-04T07:34:29.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd04ea00640 1 -- start start 2024-03-04T07:34:29.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd04ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd04814e4c0 0x7fd048072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd04ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0480c22e0 0x7fd0480731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd04ea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd0480c2d80 0x7fd04815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd04ea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04815ad60 con 0x7fd04814e4c0 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd04ea00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0480c27f0 con 0x7fd0480c2d80 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd04ea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480c2a90 con 0x7fd0480c22e0 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd047e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd04814e4c0 0x7fd048072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd047e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd04814e4c0 0x7fd048072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46728/0 (socket says 172.21.15.113:46728) 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd047e00640 1 -- 172.21.15.113:0/3537115611 learned_addr learned my addr 172.21.15.113:0/3537115611 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd047400640 1 --2- 172.21.15.113:0/3537115611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0480c22e0 0x7fd0480731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd04ca00640 1 --2- 172.21.15.113:0/3537115611 >> v2:172.21.15.136:3300/0 conn(0x7fd0480c2d80 0x7fd04815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd047e00640 1 -- 172.21.15.113:0/3537115611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0480c22e0 msgr2=0x7fd0480731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd047e00640 1 --2- 172.21.15.113:0/3537115611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0480c22e0 0x7fd0480731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd047e00640 1 -- 172.21.15.113:0/3537115611 >> v2:172.21.15.136:3300/0 conn(0x7fd0480c2d80 msgr2=0x7fd04815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:34:29.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd047e00640 1 --2- 172.21.15.113:0/3537115611 >> v2:172.21.15.136:3300/0 conn(0x7fd0480c2d80 0x7fd04815b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd047e00640 1 -- 172.21.15.113:0/3537115611 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd048070c60 con 0x7fd04814e4c0 2024-03-04T07:34:29.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.777+0000 7fd04ca00640 1 --2- 172.21.15.113:0/3537115611 >> v2:172.21.15.136:3300/0 conn(0x7fd0480c2d80 0x7fd04815b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:34:29.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd047e00640 1 --2- 172.21.15.113:0/3537115611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 0x7fd048072c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0480c2700 tx=0x7fd03801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95b77c362bc7455 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:29.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd046a00640 1 -- 172.21.15.113:0/3537115611 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0380b5070 con 0x7fd04814e4c0 2024-03-04T07:34:29.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd046a00640 1 -- 172.21.15.113:0/3537115611 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0380a89d0 con 0x7fd04814e4c0 2024-03-04T07:34:29.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd046a00640 1 -- 172.21.15.113:0/3537115611 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0380a8da0 con 0x7fd04814e4c0 2024-03-04T07:34:29.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd04ea00640 1 -- 172.21.15.113:0/3537115611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 msgr2=0x7fd048072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd04ea00640 1 --2- 172.21.15.113:0/3537115611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 0x7fd048072c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0480c2700 tx=0x7fd03801ed10 comp rx=0 tx=0).stop 2024-03-04T07:34:29.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd04ea00640 1 -- 172.21.15.113:0/3537115611 shutdown_connections 2024-03-04T07:34:29.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd04ea00640 1 --2- 172.21.15.113:0/3537115611 >> v2:172.21.15.136:3300/0 conn(0x7fd0480c2d80 0x7fd04815b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd04ea00640 1 --2- 172.21.15.113:0/3537115611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0480c22e0 0x7fd0480731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd04ea00640 1 --2- 172.21.15.113:0/3537115611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 0x7fd048072c80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd04ea00640 1 -- 172.21.15.113:0/3537115611 >> 172.21.15.113:0/3537115611 conn(0x7fd04806e820 msgr2=0x7fd04806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:29.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd04ea00640 1 -- 172.21.15.113:0/3537115611 shutdown_connections 2024-03-04T07:34:29.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.781+0000 7fd04ea00640 1 -- 172.21.15.113:0/3537115611 wait complete. 2024-03-04T07:34:29.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.785+0000 7fd04ea00640 1 Processor -- start 2024-03-04T07:34:29.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.785+0000 7fd04ea00640 1 -- start start 2024-03-04T07:34:29.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.785+0000 7fd04ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0480c22e0 0x7fd0480c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.785+0000 7fd04ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0480c2d80 0x7fd0481e6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 0x7fd0481f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048071500 con 0x7fd04814e4c0 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048071220 con 0x7fd0480c22e0 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04814e8a0 con 0x7fd0480c2d80 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 0x7fd0481f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 0x7fd0481f4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46732/0 (socket says 172.21.15.113:46732) 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ca00640 1 -- 172.21.15.113:0/766876899 learned_addr learned my addr 172.21.15.113:0/766876899 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd047e00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0480c22e0 0x7fd0480c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd047400640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0480c2d80 0x7fd0481e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ca00640 1 -- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0480c2d80 msgr2=0x7fd0481e6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ca00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0480c2d80 0x7fd0481e6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ca00640 1 -- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0480c22e0 msgr2=0x7fd0480c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ca00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0480c22e0 0x7fd0480c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ca00640 1 -- 172.21.15.113:0/766876899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04815ab00 con 0x7fd04814e4c0 2024-03-04T07:34:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ca00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 0x7fd0481f4a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd040007ef0 tx=0x7fd04000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd03fe00640 1 -- 172.21.15.113:0/766876899 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04001a070 con 0x7fd04814e4c0 2024-03-04T07:34:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd03fe00640 1 -- 172.21.15.113:0/766876899 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd040015600 con 0x7fd04814e4c0 2024-03-04T07:34:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ea00640 1 -- 172.21.15.113:0/766876899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0481e55f0 con 0x7fd04814e4c0 2024-03-04T07:34:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd03fe00640 1 -- 172.21.15.113:0/766876899 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400159f0 con 0x7fd04814e4c0 2024-03-04T07:34:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ea00640 1 -- 172.21.15.113:0/766876899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0481e5ea0 con 0x7fd04814e4c0 2024-03-04T07:34:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.789+0000 7fd04ea00640 1 -- 172.21.15.113:0/766876899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd048075c00 con 0x7fd04814e4c0 2024-03-04T07:34:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.793+0000 7fd03fe00640 1 -- 172.21.15.113:0/766876899 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd04001d070 con 0x7fd04814e4c0 2024-03-04T07:34:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.793+0000 7fd03fe00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd028066f90 0x7fd028069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:34:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.793+0000 7fd03fe00640 1 -- 172.21.15.113:0/766876899 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0400851a0 con 0x7fd04814e4c0 2024-03-04T07:34:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.793+0000 7fd047e00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd028066f90 0x7fd028069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:34:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.793+0000 7fd047e00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd028066f90 0x7fd028069450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd0380b2000 tx=0x7fd038009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:34:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.797+0000 7fd03fe00640 1 -- 172.21.15.113:0/766876899 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd040010070 con 0x7fd04814e4c0 2024-03-04T07:34:29.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.961+0000 7fd04ea00640 1 -- 172.21.15.113:0/766876899 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd048147db0 con 0x7fd04814e4c0 2024-03-04T07:34:29.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd03fe00640 1 -- 172.21.15.113:0/766876899 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fd040055bf0 con 0x7fd04814e4c0 2024-03-04T07:34:29.969 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:34:29.970 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T07:33:57.517935+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T07:34:29.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 -- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd028066f90 msgr2=0x7fd028069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd028066f90 0x7fd028069450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd0380b2000 tx=0x7fd038009040 comp rx=0 tx=0).stop 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 -- 172.21.15.113:0/766876899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 msgr2=0x7fd0481f4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 0x7fd0481f4a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd040007ef0 tx=0x7fd04000d4a0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 -- 172.21.15.113:0/766876899 shutdown_connections 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd028066f90 0x7fd028069450 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd04814e4c0 0x7fd0481f4a50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0480c2d80 0x7fd0481e6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 --2- 172.21.15.113:0/766876899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0480c22e0 0x7fd0480c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 -- 172.21.15.113:0/766876899 >> 172.21.15.113:0/766876899 conn(0x7fd04806e820 msgr2=0x7fd0480c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 -- 172.21.15.113:0/766876899 shutdown_connections 2024-03-04T07:34:29.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:34:29.965+0000 7fd04ea00640 1 -- 172.21.15.113:0/766876899 wait complete. 2024-03-04T07:34:29.991 DEBUG:tasks.mds_thrash:Yielding 2024-03-04T07:34:29.991 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T07:34:30.005 INFO:tasks.workunit:Pulling workunits from ref 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T07:34:30.005 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T07:34:30.005 INFO:tasks.workunit:timeout=6h 2024-03-04T07:34:30.005 INFO:tasks.workunit:cleanup=True 2024-03-04T07:34:30.005 DEBUG:teuthology.orchestra.run.smithi113:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T07:34:30.006 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2024-03-04T07:34:30.006 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 79 secs before thrashing 2024-03-04T07:34:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-04T07:34:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-04T07:34:30.014 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 2bh/43d Inode: 1 Links: 2 2024-03-04T07:34:30.015 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T07:34:30.015 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-04T07:34:30.015 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-03-04 07:34:01.522460212 +0000 2024-03-04T07:34:30.015 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-03-04 07:34:21.157714057 +0000 2024-03-04T07:34:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-03-04 07:34:01.522460212 +0000 2024-03-04T07:34:30.015 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-04T07:34:30.015 DEBUG:teuthology.orchestra.run.smithi113:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-04T07:34:30.084 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd 2024-03-04T07:34:30.139 INFO:tasks.workunit.client.0.smithi113.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T07:34:54.020 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:34:54.025 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:35:24.085 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:35:24.090 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:35:49.008 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:35:49.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e7ce00640 1 Processor -- start 2024-03-04T07:35:49.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e7ce00640 1 -- start start 2024-03-04T07:35:49.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e7ce00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151840 0x7f4e780c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e7ce00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4e7806d140 0x7f4e78157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e7ce00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e78157da0 con 0x7f4e78151c20 2024-03-04T07:35:49.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e7ce00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e780c3440 con 0x7f4e7806d140 2024-03-04T07:35:49.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e7ce00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78158080 con 0x7f4e78151840 2024-03-04T07:35:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e77400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151840 0x7f4e780c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e76a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4e78151c20 0x7f4e7806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60726/0 (socket says 172.21.15.113:60726) 2024-03-04T07:35:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e76a00640 1 -- 172.21.15.113:0/3218776020 learned_addr learned my addr 172.21.15.113:0/3218776020 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:35:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e77e00640 1 --2- 172.21.15.113:0/3218776020 >> v2:172.21.15.136:3300/0 conn(0x7f4e7806d140 0x7f4e78157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e77400640 1 -- 172.21.15.113:0/3218776020 >> v2:172.21.15.136:3300/0 conn(0x7f4e7806d140 msgr2=0x7f4e78157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:35:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e77400640 1 --2- 172.21.15.113:0/3218776020 >> v2:172.21.15.136:3300/0 conn(0x7f4e7806d140 0x7f4e78157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e77400640 1 -- 172.21.15.113:0/3218776020 >> v1:172.21.15.113:6789/0 conn(0x7f4e78151c20 legacy=0x7f4e7806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:49.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.163+0000 7f4e77400640 1 -- 172.21.15.113:0/3218776020 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e78158360 con 0x7f4e78151840 2024-03-04T07:35:49.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e77400640 1 --2- 172.21.15.113:0/3218776020 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151840 0x7f4e780c1f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097c0 tx=0x7f4e6801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a448c3f0b94f6ab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:49.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e76000640 1 -- 172.21.15.113:0/3218776020 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e680b0070 con 0x7f4e78151840 2024-03-04T07:35:49.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e76000640 1 -- 172.21.15.113:0/3218776020 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e680ab600 con 0x7f4e78151840 2024-03-04T07:35:49.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e76000640 1 -- 172.21.15.113:0/3218776020 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e680ab9d0 con 0x7f4e78151840 2024-03-04T07:35:49.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3218776020 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151840 msgr2=0x7f4e780c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:49.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e7ce00640 1 --2- 172.21.15.113:0/3218776020 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151840 0x7f4e780c1f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e680097c0 tx=0x7f4e6801c3c0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3218776020 shutdown_connections 2024-03-04T07:35:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e7ce00640 1 --2- 172.21.15.113:0/3218776020 >> v2:172.21.15.136:3300/0 conn(0x7f4e7806d140 0x7f4e78157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e7ce00640 1 --2- 172.21.15.113:0/3218776020 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151840 0x7f4e780c1f50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3218776020 >> 172.21.15.113:0/3218776020 conn(0x7f4e780c9a30 msgr2=0x7f4e780c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.167+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3218776020 shutdown_connections 2024-03-04T07:35:49.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.171+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3218776020 wait complete. 2024-03-04T07:35:49.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.171+0000 7f4e7ce00640 1 Processor -- start 2024-03-04T07:35:49.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.171+0000 7f4e7ce00640 1 -- start start 2024-03-04T07:35:49.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e7ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7806d140 0x7f4e7815b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e7ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e78151840 0x7f4e781f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e7ce00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151c20 0x7f4e7815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e7ce00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78071db0 con 0x7f4e78151840 2024-03-04T07:35:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e7ce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78071ad0 con 0x7f4e7806d140 2024-03-04T07:35:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e7ce00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7806c880 con 0x7f4e78151c20 2024-03-04T07:35:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e77400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7806d140 0x7f4e7815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e77400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7806d140 0x7f4e7815b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:43366/0 (socket says 172.21.15.113:43366) 2024-03-04T07:35:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e77400640 1 -- 172.21.15.113:0/3480865131 learned_addr learned my addr 172.21.15.113:0/3480865131 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:35:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e77e00640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151c20 0x7f4e7815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e76a00640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e78151840 0x7f4e781f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e77400640 1 -- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151c20 msgr2=0x7f4e7815afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e77400640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151c20 0x7f4e7815afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e77400640 1 -- 172.21.15.113:0/3480865131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e78151840 msgr2=0x7f4e781f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e77400640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e78151840 0x7f4e781f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e77400640 1 -- 172.21.15.113:0/3480865131 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7815a520 con 0x7f4e7806d140 2024-03-04T07:35:49.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e77400640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7806d140 0x7f4e7815b610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e6801aa70 tx=0x7f4e68019160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:49.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e6f400640 1 -- 172.21.15.113:0/3480865131 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e680b0040 con 0x7f4e7806d140 2024-03-04T07:35:49.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e6f400640 1 -- 172.21.15.113:0/3480865131 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e680ba070 con 0x7f4e7806d140 2024-03-04T07:35:49.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e6f400640 1 -- 172.21.15.113:0/3480865131 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e680aac30 con 0x7f4e7806d140 2024-03-04T07:35:49.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.175+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3480865131 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e78158b20 con 0x7f4e7806d140 2024-03-04T07:35:49.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.179+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3480865131 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7815a7e0 con 0x7f4e7806d140 2024-03-04T07:35:49.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.179+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3480865131 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e78075a80 con 0x7f4e7806d140 2024-03-04T07:35:49.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.179+0000 7f4e6f400640 1 -- 172.21.15.113:0/3480865131 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e680ab4c0 con 0x7f4e7806d140 2024-03-04T07:35:49.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.183+0000 7f4e6f400640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e58066f60 0x7f4e58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.183+0000 7f4e6f400640 1 -- 172.21.15.113:0/3480865131 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4e6811fd10 con 0x7f4e7806d140 2024-03-04T07:35:49.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.183+0000 7f4e76a00640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e58066f60 0x7f4e58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.183+0000 7f4e76a00640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e58066f60 0x7f4e58069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009fd0 tx=0x7f4e6400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:49.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.191+0000 7f4e6f400640 1 -- 172.21.15.113:0/3480865131 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4e68015fb0 con 0x7f4e7806d140 2024-03-04T07:35:49.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.595+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3480865131 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e780c70a0 con 0x7f4e7806d140 2024-03-04T07:35:49.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.595+0000 7f4e6f400640 1 -- 172.21.15.113:0/3480865131 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f4e680f0630 con 0x7f4e7806d140 2024-03-04T07:35:49.602 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:35:49.603 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:35:49.604 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:35:49.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e58066f60 msgr2=0x7f4e58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:49.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e58066f60 0x7f4e58069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009fd0 tx=0x7f4e6400b040 comp rx=0 tx=0).stop 2024-03-04T07:35:49.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7806d140 msgr2=0x7f4e7815b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:49.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7806d140 0x7f4e7815b610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4e6801aa70 tx=0x7f4e68019160 comp rx=0 tx=0).stop 2024-03-04T07:35:49.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3480865131 shutdown_connections 2024-03-04T07:35:49.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e78151c20 0x7f4e7815afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e58066f60 0x7f4e58069420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e78151840 0x7f4e781f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 --2- 172.21.15.113:0/3480865131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e7806d140 0x7f4e7815b610 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3480865131 >> 172.21.15.113:0/3480865131 conn(0x7f4e780c9a30 msgr2=0x7f4e78156a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:49.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3480865131 shutdown_connections 2024-03-04T07:35:49.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.599+0000 7f4e7ce00640 1 -- 172.21.15.113:0/3480865131 wait complete. 2024-03-04T07:35:49.631 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=2) 2024-03-04T07:35:49.632 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-04T07:35:49.632 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:35:49.651 INFO:tasks.ceph.mds.a:Stopped 2024-03-04T07:35:49.651 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:35:49.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e7000640 1 Processor -- start 2024-03-04T07:35:49.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e7000640 1 -- start start 2024-03-04T07:35:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e7000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff8e0151c20 0x7ff8e00c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e7000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8e006d140 0x7ff8e0157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e7000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e0157da0 con 0x7ff8e0151840 2024-03-04T07:35:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e7000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e00c3440 con 0x7ff8e006d140 2024-03-04T07:35:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e7000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e0158080 con 0x7ff8e0151c20 2024-03-04T07:35:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e5200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff8e0151c20 0x7ff8e00c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e6600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8e006d140 0x7ff8e0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e5200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff8e0151c20 0x7ff8e00c1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:43372/0 (socket says 172.21.15.113:43372) 2024-03-04T07:35:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e5c00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7ff8e0151840 0x7ff8e006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:35094/0 (socket says 172.21.15.113:35094) 2024-03-04T07:35:49.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e5c00640 1 -- 172.21.15.113:0/3041359012 learned_addr learned my addr 172.21.15.113:0/3041359012 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:35:49.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e6600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8e006d140 0x7ff8e0157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49660/0 (socket says 172.21.15.113:49660) 2024-03-04T07:35:49.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e5200640 1 -- 172.21.15.113:0/3041359012 >> v2:172.21.15.113:3300/0 conn(0x7ff8e006d140 msgr2=0x7ff8e0157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:35:49.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e5200640 1 --2- 172.21.15.113:0/3041359012 >> v2:172.21.15.113:3300/0 conn(0x7ff8e006d140 0x7ff8e0157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e5200640 1 -- 172.21.15.113:0/3041359012 >> v1:172.21.15.136:6789/0 conn(0x7ff8e0151840 legacy=0x7ff8e006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:49.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.831+0000 7ff8e5200640 1 -- 172.21.15.113:0/3041359012 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e0158360 con 0x7ff8e0151c20 2024-03-04T07:35:49.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8e5200640 1 --2- 172.21.15.113:0/3041359012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8e0151c20 0x7ff8e00c1f50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00ccec0 tx=0x7ff8cc01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e85881dbf40cde0f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:49.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8dfe00640 1 -- 172.21.15.113:0/3041359012 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0b5070 con 0x7ff8e0151c20 2024-03-04T07:35:49.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8dfe00640 1 -- 172.21.15.113:0/3041359012 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc0a8970 con 0x7ff8e0151c20 2024-03-04T07:35:49.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8dfe00640 1 -- 172.21.15.113:0/3041359012 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0a8d40 con 0x7ff8e0151c20 2024-03-04T07:35:49.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8e7000640 1 -- 172.21.15.113:0/3041359012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8e0151c20 msgr2=0x7ff8e00c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:49.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8e7000640 1 --2- 172.21.15.113:0/3041359012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8e0151c20 0x7ff8e00c1f50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff8e00ccec0 tx=0x7ff8cc01ec70 comp rx=0 tx=0).stop 2024-03-04T07:35:49.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8e7000640 1 -- 172.21.15.113:0/3041359012 shutdown_connections 2024-03-04T07:35:49.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8e7000640 1 --2- 172.21.15.113:0/3041359012 >> v2:172.21.15.113:3300/0 conn(0x7ff8e006d140 0x7ff8e0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8e7000640 1 --2- 172.21.15.113:0/3041359012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8e0151c20 0x7ff8e00c1f50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8e7000640 1 -- 172.21.15.113:0/3041359012 >> 172.21.15.113:0/3041359012 conn(0x7ff8e00c9a30 msgr2=0x7ff8e00c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:49.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8e7000640 1 -- 172.21.15.113:0/3041359012 shutdown_connections 2024-03-04T07:35:49.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.839+0000 7ff8e7000640 1 -- 172.21.15.113:0/3041359012 wait complete. 2024-03-04T07:35:49.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e7000640 1 Processor -- start 2024-03-04T07:35:49.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e7000640 1 -- start start 2024-03-04T07:35:49.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e7000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8e006d140 0x7ff8e00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e7000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e0151840 0x7ff8e00c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e7000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8e0151c20 0x7ff8e00c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e7000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0071db0 con 0x7ff8e0151840 2024-03-04T07:35:49.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e7000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e0071ad0 con 0x7ff8e0151c20 2024-03-04T07:35:49.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e7000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e006c880 con 0x7ff8e006d140 2024-03-04T07:35:49.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e0151840 0x7ff8e00c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e5c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8e006d140 0x7ff8e00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e5c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8e006d140 0x7ff8e00cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:55588/0 (socket says 172.21.15.113:55588) 2024-03-04T07:35:49.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e5c00640 1 -- 172.21.15.113:0/4093559028 learned_addr learned my addr 172.21.15.113:0/4093559028 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:35:49.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e6600640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8e0151c20 0x7ff8e00c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e5200640 1 -- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8e006d140 msgr2=0x7ff8e00cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:49.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e5200640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8e006d140 0x7ff8e00cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e5200640 1 -- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8e0151c20 msgr2=0x7ff8e00c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:49.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e5200640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8e0151c20 0x7ff8e00c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:49.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.843+0000 7ff8e5200640 1 -- 172.21.15.113:0/4093559028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e014ee20 con 0x7ff8e0151840 2024-03-04T07:35:49.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.847+0000 7ff8e5200640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e0151840 0x7ff8e00c8b60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0af000 tx=0x7ff8cc01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:49.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.847+0000 7ff8dd600640 1 -- 172.21.15.113:0/4093559028 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0bf070 con 0x7ff8e0151840 2024-03-04T07:35:49.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.847+0000 7ff8dd600640 1 -- 172.21.15.113:0/4093559028 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc0b0720 con 0x7ff8e0151840 2024-03-04T07:35:49.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.847+0000 7ff8e7000640 1 -- 172.21.15.113:0/4093559028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e014f0e0 con 0x7ff8e0151840 2024-03-04T07:35:49.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.847+0000 7ff8dd600640 1 -- 172.21.15.113:0/4093559028 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0b5040 con 0x7ff8e0151840 2024-03-04T07:35:49.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.847+0000 7ff8e7000640 1 -- 172.21.15.113:0/4093559028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e00c7da0 con 0x7ff8e0151840 2024-03-04T07:35:49.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.847+0000 7ff8e7000640 1 -- 172.21.15.113:0/4093559028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e0075a80 con 0x7ff8e0151840 2024-03-04T07:35:49.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.851+0000 7ff8dd600640 1 -- 172.21.15.113:0/4093559028 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8cc0c2070 con 0x7ff8e0151840 2024-03-04T07:35:49.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.851+0000 7ff8dd600640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8c0066fb0 0x7ff8c0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:49.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.851+0000 7ff8dd600640 1 -- 172.21.15.113:0/4093559028 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff8cc121af0 con 0x7ff8e0151840 2024-03-04T07:35:49.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.851+0000 7ff8e5c00640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8c0066fb0 0x7ff8c0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.855+0000 7ff8e5c00640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8c0066fb0 0x7ff8c0069470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4007fd0 tx=0x7ff8d4007f20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:49.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:49.859+0000 7ff8dd600640 1 -- 172.21.15.113:0/4093559028 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff8cc0c9080 con 0x7ff8e0151840 2024-03-04T07:35:50.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.215+0000 7ff8e7000640 1 -- 172.21.15.113:0/4093559028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8e006cf20 con 0x7ff8e0151840 2024-03-04T07:35:50.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.215+0000 7ff8dd600640 1 -- 172.21.15.113:0/4093559028 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7ff8cc0187d0 con 0x7ff8e0151840 2024-03-04T07:35:50.220 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:35:50.220 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:35:50.220 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:35:50.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 -- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8c0066fb0 msgr2=0x7ff8c0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:50.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8c0066fb0 0x7ff8c0069470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4007fd0 tx=0x7ff8d4007f20 comp rx=0 tx=0).stop 2024-03-04T07:35:50.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 -- 172.21.15.113:0/4093559028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e0151840 msgr2=0x7ff8e00c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:50.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e0151840 0x7ff8e00c8b60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0af000 tx=0x7ff8cc01ba10 comp rx=0 tx=0).stop 2024-03-04T07:35:50.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 -- 172.21.15.113:0/4093559028 shutdown_connections 2024-03-04T07:35:50.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8e0151c20 0x7ff8e00c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:50.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8c0066fb0 0x7ff8c0069470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:50.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8e0151840 0x7ff8e00c8b60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:50.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 --2- 172.21.15.113:0/4093559028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8e006d140 0x7ff8e00cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:50.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 -- 172.21.15.113:0/4093559028 >> 172.21.15.113:0/4093559028 conn(0x7ff8e00c9a30 msgr2=0x7ff8e01573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:50.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 -- 172.21.15.113:0/4093559028 shutdown_connections 2024-03-04T07:35:50.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:50.219+0000 7ff8e7000640 1 -- 172.21.15.113:0/4093559028 wait complete. 2024-03-04T07:35:50.253 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:35:52.253 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:35:52.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3eff800640 1 Processor -- start 2024-03-04T07:35:52.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3eff800640 1 -- start start 2024-03-04T07:35:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3eff800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ef81527f0 0x7f3ef8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3eff800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef81591f0 con 0x7f3ef8152bd0 2024-03-04T07:35:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3eff800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef81594d0 con 0x7f3ef80c8200 2024-03-04T07:35:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3eff800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef8071df0 con 0x7f3ef81527f0 2024-03-04T07:35:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3efce00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ef81527f0 0x7f3ef8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3efce00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ef81527f0 0x7f3ef8070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49676/0 (socket says 172.21.15.113:49676) 2024-03-04T07:35:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3efce00640 1 -- 172.21.15.113:0/1512261247 learned_addr learned my addr 172.21.15.113:0/1512261247 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:35:52.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3efce00640 1 -- 172.21.15.113:0/1512261247 >> v1:172.21.15.136:6789/0 conn(0x7f3ef80c8200 legacy=0x7f3ef8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:52.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3efce00640 1 -- 172.21.15.113:0/1512261247 >> v1:172.21.15.113:6789/0 conn(0x7f3ef8152bd0 legacy=0x7f3ef81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:52.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3ef7400640 1 -- 172.21.15.113:0/1512261247 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220001561 0 0) 0x7f3ef81591f0 con 0x7f3ef8152bd0 2024-03-04T07:35:52.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.387+0000 7f3efce00640 1 -- 172.21.15.113:0/1512261247 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef80717a0 con 0x7f3ef81527f0 2024-03-04T07:35:52.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3efce00640 1 --2- 172.21.15.113:0/1512261247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef81527f0 0x7f3ef8070d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009e50 tx=0x7f3eec01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e06f8fa7f6df16d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:52.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3ef7400640 1 -- 172.21.15.113:0/1512261247 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0a5700 con 0x7f3ef81527f0 2024-03-04T07:35:52.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3ef7400640 1 -- 172.21.15.113:0/1512261247 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec0a58a0 con 0x7f3ef81527f0 2024-03-04T07:35:52.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3ef7400640 1 -- 172.21.15.113:0/1512261247 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0a5c70 con 0x7f3ef81527f0 2024-03-04T07:35:52.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3eff800640 1 -- 172.21.15.113:0/1512261247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef81527f0 msgr2=0x7f3ef8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:52.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3eff800640 1 --2- 172.21.15.113:0/1512261247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef81527f0 0x7f3ef8070d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009e50 tx=0x7f3eec01ca30 comp rx=0 tx=0).stop 2024-03-04T07:35:52.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3eff800640 1 -- 172.21.15.113:0/1512261247 shutdown_connections 2024-03-04T07:35:52.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3eff800640 1 --2- 172.21.15.113:0/1512261247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef81527f0 0x7f3ef8070d40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:52.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3eff800640 1 -- 172.21.15.113:0/1512261247 >> 172.21.15.113:0/1512261247 conn(0x7f3ef806e820 msgr2=0x7f3ef806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:52.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3eff800640 1 -- 172.21.15.113:0/1512261247 shutdown_connections 2024-03-04T07:35:52.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.391+0000 7f3eff800640 1 -- 172.21.15.113:0/1512261247 wait complete. 2024-03-04T07:35:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3eff800640 1 Processor -- start 2024-03-04T07:35:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3eff800640 1 -- start start 2024-03-04T07:35:52.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3eff800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ef80c8200 0x7f3ef814e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3eff800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ef81527f0 0x7f3ef8156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3eff800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef8152bd0 0x7f3ef8156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3eff800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef8166500 con 0x7f3ef8152bd0 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3eff800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef8166380 con 0x7f3ef80c8200 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3eff800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef8166680 con 0x7f3ef81527f0 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3efd800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef8152bd0 0x7f3ef8156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3efd800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef8152bd0 0x7f3ef8156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49684/0 (socket says 172.21.15.113:49684) 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3efd800640 1 -- 172.21.15.113:0/2060786596 learned_addr learned my addr 172.21.15.113:0/2060786596 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3ef7e00640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ef81527f0 0x7f3ef8156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3efce00640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ef80c8200 0x7f3ef814e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3efd800640 1 -- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ef81527f0 msgr2=0x7f3ef8156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3efd800640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ef81527f0 0x7f3ef8156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3efd800640 1 -- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ef80c8200 msgr2=0x7f3ef814e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3efd800640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ef80c8200 0x7f3ef814e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3efd800640 1 -- 172.21.15.113:0/2060786596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef80c6820 con 0x7f3ef8152bd0 2024-03-04T07:35:52.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3efd800640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef8152bd0 0x7f3ef8156eb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c440 tx=0x7f3ef000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:52.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3ef4c00640 1 -- 172.21.15.113:0/2060786596 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0017070 con 0x7f3ef8152bd0 2024-03-04T07:35:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3ef4c00640 1 -- 172.21.15.113:0/2060786596 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef0003600 con 0x7f3ef8152bd0 2024-03-04T07:35:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3ef4c00640 1 -- 172.21.15.113:0/2060786596 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00039f0 con 0x7f3ef8152bd0 2024-03-04T07:35:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3eff800640 1 -- 172.21.15.113:0/2060786596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef81500a0 con 0x7f3ef8152bd0 2024-03-04T07:35:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.395+0000 7f3eff800640 1 -- 172.21.15.113:0/2060786596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef814d9d0 con 0x7f3ef8152bd0 2024-03-04T07:35:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.399+0000 7f3eff800640 1 -- 172.21.15.113:0/2060786596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef8075c00 con 0x7f3ef8152bd0 2024-03-04T07:35:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.399+0000 7f3ef4c00640 1 -- 172.21.15.113:0/2060786596 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ef001a070 con 0x7f3ef8152bd0 2024-03-04T07:35:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.399+0000 7f3ef4c00640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ed8066f90 0x7f3ed8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:52.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.399+0000 7f3ef4c00640 1 -- 172.21.15.113:0/2060786596 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3ef0084280 con 0x7f3ef8152bd0 2024-03-04T07:35:52.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.399+0000 7f3efce00640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ed8066f90 0x7f3ed8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:52.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.403+0000 7f3efce00640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ed8066f90 0x7f3ed8069450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0ae000 tx=0x7f3eec0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:52.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.403+0000 7f3ef4c00640 1 -- 172.21.15.113:0/2060786596 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3ef0054c50 con 0x7f3ef8152bd0 2024-03-04T07:35:52.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.599+0000 7f3eff800640 1 -- 172.21.15.113:0/2060786596 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ef80c79d0 con 0x7f3ef8152bd0 2024-03-04T07:35:52.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.599+0000 7f3ef4c00640 1 -- 172.21.15.113:0/2060786596 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f3ef0058cf0 con 0x7f3ef8152bd0 2024-03-04T07:35:52.603 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:35:52.603 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:35:52.603 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:35:52.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 -- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ed8066f90 msgr2=0x7f3ed8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:52.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ed8066f90 0x7f3ed8069450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0ae000 tx=0x7f3eec0a7070 comp rx=0 tx=0).stop 2024-03-04T07:35:52.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 -- 172.21.15.113:0/2060786596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef8152bd0 msgr2=0x7f3ef8156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:52.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef8152bd0 0x7f3ef8156eb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c440 tx=0x7f3ef000c840 comp rx=0 tx=0).stop 2024-03-04T07:35:52.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 -- 172.21.15.113:0/2060786596 shutdown_connections 2024-03-04T07:35:52.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ed8066f90 0x7f3ed8069450 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:52.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ef8152bd0 0x7f3ef8156eb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:52.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ef81527f0 0x7f3ef8156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:52.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 --2- 172.21.15.113:0/2060786596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ef80c8200 0x7f3ef814e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:52.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 -- 172.21.15.113:0/2060786596 >> 172.21.15.113:0/2060786596 conn(0x7f3ef806e820 msgr2=0x7f3ef80c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:52.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 -- 172.21.15.113:0/2060786596 shutdown_connections 2024-03-04T07:35:52.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:52.603+0000 7f3eff800640 1 -- 172.21.15.113:0/2060786596 wait complete. 2024-03-04T07:35:52.635 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:35:54.173 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:35:54.176 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:35:54.382 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 66% (8137/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-04T07:35:54.636 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:35:54.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec6a00640 1 Processor -- start 2024-03-04T07:35:54.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec6a00640 1 -- start start 2024-03-04T07:35:54.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ec0152710 0x7f0ec00c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:54.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec6a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0ec00c71d0 0x7f0ec00c6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:54.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006eae0 0x7f0ec015be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:54.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec6a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec0152af0 con 0x7f0ec0152710 2024-03-04T07:35:54.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec6a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec0159570 con 0x7f0ec00c71d0 2024-03-04T07:35:54.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec6a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec006f3b0 con 0x7f0ec006eae0 2024-03-04T07:35:54.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec5600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ec0152710 0x7f0ec00c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:54.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec5600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ec0152710 0x7f0ec00c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49698/0 (socket says 172.21.15.113:49698) 2024-03-04T07:35:54.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec5600640 1 -- 172.21.15.113:0/642678530 learned_addr learned my addr 172.21.15.113:0/642678530 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:35:54.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec4c00640 1 --2- 172.21.15.113:0/642678530 >> v2:172.21.15.136:3300/0 conn(0x7f0ec00c71d0 0x7f0ec00c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:54.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec6000640 1 --2- 172.21.15.113:0/642678530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006eae0 0x7f0ec015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:54.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec5600640 1 -- 172.21.15.113:0/642678530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006eae0 msgr2=0x7f0ec015be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:35:54.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec5600640 1 --2- 172.21.15.113:0/642678530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006eae0 0x7f0ec015be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec5600640 1 -- 172.21.15.113:0/642678530 >> v2:172.21.15.136:3300/0 conn(0x7f0ec00c71d0 msgr2=0x7f0ec00c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:35:54.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec5600640 1 --2- 172.21.15.113:0/642678530 >> v2:172.21.15.136:3300/0 conn(0x7f0ec00c71d0 0x7f0ec00c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.775+0000 7f0ec5600640 1 -- 172.21.15.113:0/642678530 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec0159290 con 0x7f0ec0152710 2024-03-04T07:35:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.779+0000 7f0ec5600640 1 --2- 172.21.15.113:0/642678530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec0152710 0x7f0ec00c5ee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00c8140 tx=0x7f0eb401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25e103911808f37 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.779+0000 7f0ebfe00640 1 -- 172.21.15.113:0/642678530 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40b0070 con 0x7f0ec0152710 2024-03-04T07:35:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.779+0000 7f0ebfe00640 1 -- 172.21.15.113:0/642678530 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb40ab600 con 0x7f0ec0152710 2024-03-04T07:35:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.779+0000 7f0ebfe00640 1 -- 172.21.15.113:0/642678530 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40ab9d0 con 0x7f0ec0152710 2024-03-04T07:35:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.779+0000 7f0ec6a00640 1 -- 172.21.15.113:0/642678530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec0152710 msgr2=0x7f0ec00c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.779+0000 7f0ec6a00640 1 --2- 172.21.15.113:0/642678530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec0152710 0x7f0ec00c5ee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00c8140 tx=0x7f0eb401c520 comp rx=0 tx=0).stop 2024-03-04T07:35:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.779+0000 7f0ec6a00640 1 -- 172.21.15.113:0/642678530 shutdown_connections 2024-03-04T07:35:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.783+0000 7f0ec6a00640 1 --2- 172.21.15.113:0/642678530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec006eae0 0x7f0ec015be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.783+0000 7f0ec6a00640 1 --2- 172.21.15.113:0/642678530 >> v2:172.21.15.136:3300/0 conn(0x7f0ec00c71d0 0x7f0ec00c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.783+0000 7f0ec6a00640 1 --2- 172.21.15.113:0/642678530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec0152710 0x7f0ec00c5ee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.783+0000 7f0ec6a00640 1 -- 172.21.15.113:0/642678530 >> 172.21.15.113:0/642678530 conn(0x7f0ec0067810 msgr2=0x7f0ec006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:54.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.783+0000 7f0ec6a00640 1 -- 172.21.15.113:0/642678530 shutdown_connections 2024-03-04T07:35:54.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.783+0000 7f0ec6a00640 1 -- 172.21.15.113:0/642678530 wait complete. 2024-03-04T07:35:54.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.783+0000 7f0ec6a00640 1 Processor -- start 2024-03-04T07:35:54.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.783+0000 7f0ec6a00640 1 -- start start 2024-03-04T07:35:54.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec006eae0 0x7f0ec014d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:54.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec00c71d0 0x7f0ec00c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:54.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec0152710 0x7f0ec00c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:54.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec6a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00c9e40 con 0x7f0ec006eae0 2024-03-04T07:35:54.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec6a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00c9cc0 con 0x7f0ec00c71d0 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec6a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00c9fc0 con 0x7f0ec0152710 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec006eae0 0x7f0ec014d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec006eae0 0x7f0ec014d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49710/0 (socket says 172.21.15.113:49710) 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec5600640 1 -- 172.21.15.113:0/2041083828 learned_addr learned my addr 172.21.15.113:0/2041083828 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec6000640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec0152710 0x7f0ec00c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec4c00640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec00c71d0 0x7f0ec00c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec5600640 1 -- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec0152710 msgr2=0x7f0ec00c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec5600640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec0152710 0x7f0ec00c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec5600640 1 -- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec00c71d0 msgr2=0x7f0ec00c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec5600640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec00c71d0 0x7f0ec00c4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec5600640 1 -- 172.21.15.113:0/2041083828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec01f3d90 con 0x7f0ec006eae0 2024-03-04T07:35:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec5600640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec006eae0 0x7f0ec014d0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40ae040 tx=0x7f0eb401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ebd600640 1 -- 172.21.15.113:0/2041083828 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40b0040 con 0x7f0ec006eae0 2024-03-04T07:35:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ebd600640 1 -- 172.21.15.113:0/2041083828 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb40bc070 con 0x7f0ec006eae0 2024-03-04T07:35:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec6a00640 1 -- 172.21.15.113:0/2041083828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec01f4070 con 0x7f0ec006eae0 2024-03-04T07:35:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ebd600640 1 -- 172.21.15.113:0/2041083828 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40b6880 con 0x7f0ec006eae0 2024-03-04T07:35:54.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec6a00640 1 -- 172.21.15.113:0/2041083828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec01f4630 con 0x7f0ec006eae0 2024-03-04T07:35:54.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.787+0000 7f0ec6a00640 1 -- 172.21.15.113:0/2041083828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec01f4870 con 0x7f0ec006eae0 2024-03-04T07:35:54.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.791+0000 7f0ebd600640 1 -- 172.21.15.113:0/2041083828 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0eb40ab4c0 con 0x7f0ec006eae0 2024-03-04T07:35:54.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.791+0000 7f0ebd600640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ea0066f60 0x7f0ea0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:54.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.791+0000 7f0ebd600640 1 -- 172.21.15.113:0/2041083828 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0eb411f4b0 con 0x7f0ec006eae0 2024-03-04T07:35:54.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.791+0000 7f0ec4c00640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ea0066f60 0x7f0ea0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:54.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.791+0000 7f0ec4c00640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ea0066f60 0x7f0ea0069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ec01584f0 tx=0x7f0eac009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:54.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.795+0000 7f0ebd600640 1 -- 172.21.15.113:0/2041083828 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0ec01f4870 con 0x7f0ec006eae0 2024-03-04T07:35:54.799 INFO:tasks.workunit.client.0.smithi113.stderr:Note: switching to '1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd'. 2024-03-04T07:35:54.799 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-03-04T07:35:54.799 INFO:tasks.workunit.client.0.smithi113.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T07:35:54.799 INFO:tasks.workunit.client.0.smithi113.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T07:35:54.799 INFO:tasks.workunit.client.0.smithi113.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T07:35:54.799 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-03-04T07:35:54.799 INFO:tasks.workunit.client.0.smithi113.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr: git switch -c 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr:Or undo this operation with: 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr: git switch - 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr: 2024-03-04T07:35:54.800 INFO:tasks.workunit.client.0.smithi113.stderr:HEAD is now at 1bc250c7129 mds/quiesce-db: incorporate review comments 2024-03-04T07:35:54.804 DEBUG:teuthology.orchestra.run.smithi113:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T07:35:54.813 INFO:tasks.workunit.client.0.smithi113.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T07:35:54.816 INFO:tasks.workunit.client.0.smithi113.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T07:35:54.816 INFO:tasks.workunit.client.0.smithi113.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T07:35:54.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 -- 172.21.15.113:0/2041083828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ec0146e20 con 0x7f0ec006eae0 2024-03-04T07:35:54.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ebd600640 1 -- 172.21.15.113:0/2041083828 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f0eb4016110 con 0x7f0ec006eae0 2024-03-04T07:35:54.993 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:35:54.994 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:35:54.994 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 -- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ea0066f60 msgr2=0x7f0ea0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ea0066f60 0x7f0ea0069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0ec01584f0 tx=0x7f0eac009190 comp rx=0 tx=0).stop 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 -- 172.21.15.113:0/2041083828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec006eae0 msgr2=0x7f0ec014d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec006eae0 0x7f0ec014d0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40ae040 tx=0x7f0eb401ad60 comp rx=0 tx=0).stop 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 -- 172.21.15.113:0/2041083828 shutdown_connections 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ec0152710 0x7f0ec00c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ec00c71d0 0x7f0ec00c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ea0066f60 0x7f0ea0069420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 --2- 172.21.15.113:0/2041083828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec006eae0 0x7f0ec014d0a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 -- 172.21.15.113:0/2041083828 >> 172.21.15.113:0/2041083828 conn(0x7f0ec0067810 msgr2=0x7f0ec013fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:54.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 -- 172.21.15.113:0/2041083828 shutdown_connections 2024-03-04T07:35:54.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:54.991+0000 7f0ec6a00640 1 -- 172.21.15.113:0/2041083828 wait complete. 2024-03-04T07:35:55.021 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:35:55.533 INFO:tasks.workunit.client.0.smithi113.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T07:35:55.613 INFO:tasks.workunit.client.0.smithi113.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T07:35:55.675 INFO:tasks.workunit.client.0.smithi113.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T07:35:55.677 INFO:tasks.workunit.client.0.smithi113.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T07:35:55.678 INFO:tasks.workunit.client.0.smithi113.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T07:35:55.730 INFO:tasks.workunit.client.0.smithi113.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T07:35:55.737 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T07:35:55.737 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T07:35:55.745 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2024-03-04T07:35:55.746 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2024-03-04T07:35:55.747 DEBUG:teuthology.orchestra.run.smithi113:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2024-03-04T07:35:57.023 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:35:57.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.155+0000 7f7bd2c00640 1 Processor -- start 2024-03-04T07:35:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.155+0000 7f7bd2c00640 1 -- start start 2024-03-04T07:35:57.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.155+0000 7f7bd2c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc152a50 0x7f7bcc0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:57.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.155+0000 7f7bd2c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc152e30 con 0x7f7bcc06ff20 2024-03-04T07:35:57.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.155+0000 7f7bd2c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc06f280 con 0x7f7bcc15c4e0 2024-03-04T07:35:57.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.155+0000 7f7bd2c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0c31b0 con 0x7f7bcc152a50 2024-03-04T07:35:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bc3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7bcc06ff20 0x7f7bcc15bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60734/0 (socket says 172.21.15.113:60734) 2024-03-04T07:35:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bc3e00640 1 -- 172.21.15.113:0/3580611892 learned_addr learned my addr 172.21.15.113:0/3580611892 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:35:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcbe00640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc152a50 0x7f7bcc0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758374646 0 0) 0x7f7bcc152e30 con 0x7f7bcc06ff20 2024-03-04T07:35:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb0003530 con 0x7f7bcc06ff20 2024-03-04T07:35:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348049986 0 0) 0x7f7bcc06f280 con 0x7f7bcc15c4e0 2024-03-04T07:35:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bcc152e30 con 0x7f7bcc15c4e0 2024-03-04T07:35:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2922404522 0 0) 0x7f7bb0003530 con 0x7f7bcc06ff20 2024-03-04T07:35:57.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc152a50 msgr2=0x7f7bcc0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc152a50 0x7f7bcc0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 >> v1:172.21.15.136:6789/0 conn(0x7f7bcc15c4e0 legacy=0x7f7bcc163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc06f560 con 0x7f7bcc06ff20 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f7bb4001680 con 0x7f7bcc06ff20 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 >> v1:172.21.15.113:6789/0 conn(0x7f7bcc06ff20 legacy=0x7f7bcc15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bb0004be0 0x7f7bb0004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 0x7f7bb0003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc152a50 0x7f7bb0004170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0c31b0 con 0x7f7bb00037b0 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb000ba80 con 0x7f7bb0004be0 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb000bc50 con 0x7f7bcc152a50 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bc3e00640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 0x7f7bb0003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bcbe00640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bb0004be0 0x7f7bb0004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bd0c00640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc152a50 0x7f7bb0004170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:57.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bc3e00640 1 -- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc152a50 msgr2=0x7f7bb0004170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:35:57.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bc3e00640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc152a50 0x7f7bb0004170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:57.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bc3e00640 1 -- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bb0004be0 msgr2=0x7f7bb0004fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:35:57.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bc3e00640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bb0004be0 0x7f7bb0004fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:57.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bc3e00640 1 -- 172.21.15.113:0/3580611892 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0003530 con 0x7f7bb00037b0 2024-03-04T07:35:57.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.159+0000 7f7bd0c00640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc152a50 0x7f7bb0004170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:35:57.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bc3e00640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 0x7f7bb0003c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc070360 tx=0x7f7bb401cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214751c91d7b386 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:57.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4007cb0 con 0x7f7bb00037b0 2024-03-04T07:35:57.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb40a9020 con 0x7f7bb00037b0 2024-03-04T07:35:57.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bcb400640 1 -- 172.21.15.113:0/3580611892 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4005b20 con 0x7f7bb00037b0 2024-03-04T07:35:57.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bd2c00640 1 -- 172.21.15.113:0/3580611892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 msgr2=0x7f7bb0003c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:57.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bd2c00640 1 --2- 172.21.15.113:0/3580611892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 0x7f7bb0003c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc070360 tx=0x7f7bb401cb60 comp rx=0 tx=0).stop 2024-03-04T07:35:57.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bd0c00640 1 -- 172.21.15.113:0/3580611892 reap_dead start 2024-03-04T07:35:57.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bd2c00640 1 -- 172.21.15.113:0/3580611892 shutdown_connections 2024-03-04T07:35:57.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bd2c00640 1 -- 172.21.15.113:0/3580611892 >> 172.21.15.113:0/3580611892 conn(0x7f7bcc0734d0 msgr2=0x7f7bcc066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:57.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bd2c00640 1 -- 172.21.15.113:0/3580611892 shutdown_connections 2024-03-04T07:35:57.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bd2c00640 1 -- 172.21.15.113:0/3580611892 wait complete. 2024-03-04T07:35:57.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bd2c00640 1 Processor -- start 2024-03-04T07:35:57.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.163+0000 7f7bd2c00640 1 -- start start 2024-03-04T07:35:57.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bd2c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 0x7f7bcc06c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:57.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bd2c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc1e1af0 0x7f7bcc147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:57.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bd2c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bcc1e46d0 0x7f7bcc148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bd2c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0c3400 con 0x7f7bb00037b0 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bd2c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc06f840 con 0x7f7bcc1e46d0 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bd2c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0c8b90 con 0x7f7bcc1e1af0 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bcbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 0x7f7bcc06c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bcbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 0x7f7bcc06c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49722/0 (socket says 172.21.15.113:49722) 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bcbe00640 1 -- 172.21.15.113:0/2240279751 learned_addr learned my addr 172.21.15.113:0/2240279751 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bc3e00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc1e1af0 0x7f7bcc147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bcbe00640 1 -- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc1e1af0 msgr2=0x7f7bcc147b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bcbe00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc1e1af0 0x7f7bcc147b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bcbe00640 1 -- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bcc1e46d0 msgr2=0x7f7bcc148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bcbe00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bcc1e46d0 0x7f7bcc148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bcbe00640 1 -- 172.21.15.113:0/2240279751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc1e43d0 con 0x7f7bb00037b0 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bcbe00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 0x7f7bcc06c3d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0119d0 tx=0x7f7bbc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:57.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bc8c00640 1 -- 172.21.15.113:0/2240279751 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00fad0 con 0x7f7bb00037b0 2024-03-04T07:35:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bc8c00640 1 -- 172.21.15.113:0/2240279751 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc00fc70 con 0x7f7bb00037b0 2024-03-04T07:35:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bd2c00640 1 -- 172.21.15.113:0/2240279751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc06ce90 con 0x7f7bb00037b0 2024-03-04T07:35:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bc8c00640 1 -- 172.21.15.113:0/2240279751 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc014440 con 0x7f7bb00037b0 2024-03-04T07:35:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bd2c00640 1 -- 172.21.15.113:0/2240279751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc1e2ad0 con 0x7f7bb00037b0 2024-03-04T07:35:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bd2c00640 1 -- 172.21.15.113:0/2240279751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc1e2900 con 0x7f7bb00037b0 2024-03-04T07:35:57.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.167+0000 7f7bc8c00640 1 -- 172.21.15.113:0/2240279751 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7bbc012040 con 0x7f7bb00037b0 2024-03-04T07:35:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.171+0000 7f7bc8c00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba8066f90 0x7f7ba8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:57.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.171+0000 7f7bc8c00640 1 -- 172.21.15.113:0/2240279751 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7bbc085ad0 con 0x7f7bb00037b0 2024-03-04T07:35:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.171+0000 7f7bc3e00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba8066f90 0x7f7ba8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:57.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.171+0000 7f7bc3e00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba8066f90 0x7f7ba8069450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40aa000 tx=0x7f7bb40b0040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:57.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.171+0000 7f7bc8c00640 1 -- 172.21.15.113:0/2240279751 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7bbc085e20 con 0x7f7bb00037b0 2024-03-04T07:35:57.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 -- 172.21.15.113:0/2240279751 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bcc075c00 con 0x7f7bb00037b0 2024-03-04T07:35:57.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bc8c00640 1 -- 172.21.15.113:0/2240279751 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f7bbc02b020 con 0x7f7bb00037b0 2024-03-04T07:35:57.446 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:35:57.446 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:35:57.447 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:35:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 -- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba8066f90 msgr2=0x7f7ba8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba8066f90 0x7f7ba8069450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40aa000 tx=0x7f7bb40b0040 comp rx=0 tx=0).stop 2024-03-04T07:35:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 -- 172.21.15.113:0/2240279751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 msgr2=0x7f7bcc06c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 0x7f7bcc06c3d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0119d0 tx=0x7f7bbc011e90 comp rx=0 tx=0).stop 2024-03-04T07:35:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 -- 172.21.15.113:0/2240279751 shutdown_connections 2024-03-04T07:35:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bcc1e46d0 0x7f7bcc148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:57.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bcc1e1af0 0x7f7bcc147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba8066f90 0x7f7ba8069450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 --2- 172.21.15.113:0/2240279751 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bb00037b0 0x7f7bcc06c3d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.443+0000 7f7bd2c00640 1 -- 172.21.15.113:0/2240279751 >> 172.21.15.113:0/2240279751 conn(0x7f7bcc0734d0 msgr2=0x7f7bcc159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.447+0000 7f7bd2c00640 1 -- 172.21.15.113:0/2240279751 shutdown_connections 2024-03-04T07:35:57.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:57.447+0000 7f7bd2c00640 1 -- 172.21.15.113:0/2240279751 wait complete. 2024-03-04T07:35:57.467 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:35:59.468 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:35:59.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf52800640 1 Processor -- start 2024-03-04T07:35:59.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf52800640 1 -- start start 2024-03-04T07:35:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf52800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf52800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdf4c06de80 0x7fdf4c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf52800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4c06c210 con 0x7fdf4c0ce2d0 2024-03-04T07:35:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf52800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4c15bbf0 con 0x7fdf4c06de80 2024-03-04T07:35:59.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf52800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c06eb80 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:46462/0 (socket says 172.21.15.113:46462) 2024-03-04T07:35:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf43e00640 1 -- 172.21.15.113:0/3206574767 learned_addr learned my addr 172.21.15.113:0/3206574767 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:35:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf4be00640 1 --2- 172.21.15.113:0/3206574767 >> v2:172.21.15.136:3300/0 conn(0x7fdf4c06de80 0x7fdf4c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf43e00640 1 -- 172.21.15.113:0/3206574767 >> v2:172.21.15.136:3300/0 conn(0x7fdf4c06de80 msgr2=0x7fdf4c154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:35:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf43e00640 1 --2- 172.21.15.113:0/3206574767 >> v2:172.21.15.136:3300/0 conn(0x7fdf4c06de80 0x7fdf4c154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:59.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf43e00640 1 -- 172.21.15.113:0/3206574767 >> v1:172.21.15.136:6789/0 conn(0x7fdf4c0ce2d0 legacy=0x7fdf4c06f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:59.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.603+0000 7fdf43e00640 1 -- 172.21.15.113:0/3206574767 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c06c4f0 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf43e00640 1 --2- 172.21.15.113:0/3206574767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15b6b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009870 tx=0x7fdf3401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c338c2187a6ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:59.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf4aa00640 1 -- 172.21.15.113:0/3206574767 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf340b5070 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf4aa00640 1 -- 172.21.15.113:0/3206574767 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf340a8a50 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf4aa00640 1 -- 172.21.15.113:0/3206574767 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf340a8e20 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf52800640 1 -- 172.21.15.113:0/3206574767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c0ce6b0 msgr2=0x7fdf4c15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf52800640 1 --2- 172.21.15.113:0/3206574767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15b6b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009870 tx=0x7fdf3401ed50 comp rx=0 tx=0).stop 2024-03-04T07:35:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf52800640 1 -- 172.21.15.113:0/3206574767 shutdown_connections 2024-03-04T07:35:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf52800640 1 --2- 172.21.15.113:0/3206574767 >> v2:172.21.15.136:3300/0 conn(0x7fdf4c06de80 0x7fdf4c154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf52800640 1 --2- 172.21.15.113:0/3206574767 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15b6b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf52800640 1 -- 172.21.15.113:0/3206574767 >> 172.21.15.113:0/3206574767 conn(0x7fdf4c0709a0 msgr2=0x7fdf4c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf52800640 1 -- 172.21.15.113:0/3206574767 shutdown_connections 2024-03-04T07:35:59.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.607+0000 7fdf52800640 1 -- 172.21.15.113:0/3206574767 wait complete. 2024-03-04T07:35:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf52800640 1 Processor -- start 2024-03-04T07:35:59.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf52800640 1 -- start start 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf52800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c06de80 0x7fdf4c156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf52800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf4c0ce2d0 0x7fdf4c14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf52800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf52800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c06c7d0 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf52800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c06e810 con 0x7fdf4c0ce2d0 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf52800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c06f720 con 0x7fdf4c06de80 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf4be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf4b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c06de80 0x7fdf4c156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf4be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41720/0 (socket says 172.21.15.113:41720) 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf4be00640 1 -- 172.21.15.113:0/3494855195 learned_addr learned my addr 172.21.15.113:0/3494855195 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf43e00640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf4c0ce2d0 0x7fdf4c14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf4be00640 1 -- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c06de80 msgr2=0x7fdf4c156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:59.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf4be00640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c06de80 0x7fdf4c156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:59.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf4be00640 1 -- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf4c0ce2d0 msgr2=0x7fdf4c14bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:59.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf4be00640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf4c0ce2d0 0x7fdf4c14bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:59.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf4be00640 1 -- 172.21.15.113:0/3494855195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c1574f0 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf4be00640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15aca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400efc0 tx=0x7fdf4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:59.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf43400640 1 -- 172.21.15.113:0/3494855195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44018020 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf43400640 1 -- 172.21.15.113:0/3494855195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44014480 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf52800640 1 -- 172.21.15.113:0/3494855195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c1577b0 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf43400640 1 -- 172.21.15.113:0/3494855195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44014870 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.611+0000 7fdf52800640 1 -- 172.21.15.113:0/3494855195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c0c7b90 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.615+0000 7fdf52800640 1 -- 172.21.15.113:0/3494855195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4c0c76d0 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.615+0000 7fdf43400640 1 -- 172.21.15.113:0/3494855195 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf4401b020 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.615+0000 7fdf43400640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf28066f60 0x7fdf28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:35:59.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.615+0000 7fdf43400640 1 -- 172.21.15.113:0/3494855195 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdf44085a20 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.615+0000 7fdf4b400640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf28066f60 0x7fdf28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:35:59.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.619+0000 7fdf4b400640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf28066f60 0x7fdf28069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c001810 tx=0x7fdf3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:35:59.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.619+0000 7fdf43400640 1 -- 172.21.15.113:0/3494855195 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdf4c0c76d0 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 -- 172.21.15.113:0/3494855195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf4c075c00 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf43400640 1 -- 172.21.15.113:0/3494855195 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fdf44022070 con 0x7fdf4c0ce6b0 2024-03-04T07:35:59.810 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:35:59.810 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:35:59.810 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:35:59.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 -- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf28066f60 msgr2=0x7fdf28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf28066f60 0x7fdf28069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c001810 tx=0x7fdf3c009040 comp rx=0 tx=0).stop 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 -- 172.21.15.113:0/3494855195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf4c0ce6b0 msgr2=0x7fdf4c15aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15aca0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400efc0 tx=0x7fdf4400c490 comp rx=0 tx=0).stop 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 -- 172.21.15.113:0/3494855195 shutdown_connections 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf28066f60 0x7fdf28069420 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf4c0ce6b0 0x7fdf4c15aca0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf4c0ce2d0 0x7fdf4c14bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 --2- 172.21.15.113:0/3494855195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf4c06de80 0x7fdf4c156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 -- 172.21.15.113:0/3494855195 >> 172.21.15.113:0/3494855195 conn(0x7fdf4c0709a0 msgr2=0x7fdf4c0cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 -- 172.21.15.113:0/3494855195 shutdown_connections 2024-03-04T07:35:59.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:35:59.807+0000 7fdf52800640 1 -- 172.21.15.113:0/3494855195 wait complete. 2024-03-04T07:35:59.832 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:36:01.833 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:01.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f424cc00640 1 Processor -- start 2024-03-04T07:36:01.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f424cc00640 1 -- start start 2024-03-04T07:36:01.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f424cc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4248158010 0x7f424806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f424cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42480ce120 0x7f42480ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f424cc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42480ca0e0 con 0x7f424806ff20 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f424cc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424806f450 con 0x7f4248158010 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f424cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424815ad80 con 0x7f42480ce120 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f4245e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4248158010 0x7f424806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f4245400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f424806ff20 0x7f424815b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36142/0 (socket says 172.21.15.113:36142) 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f4245400640 1 -- 172.21.15.113:0/3740647902 learned_addr learned my addr 172.21.15.113:0/3740647902 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f4246800640 1 --2- 172.21.15.113:0/3740647902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42480ce120 0x7f42480ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f4245e00640 1 -- 172.21.15.113:0/3740647902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42480ce120 msgr2=0x7f42480ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f4245e00640 1 --2- 172.21.15.113:0/3740647902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42480ce120 0x7f42480ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f4245e00640 1 -- 172.21.15.113:0/3740647902 >> v1:172.21.15.113:6789/0 conn(0x7f424806ff20 legacy=0x7f424815b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f4245e00640 1 -- 172.21.15.113:0/3740647902 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424815aa30 con 0x7f4248158010 2024-03-04T07:36:01.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.971+0000 7f4244a00640 1 -- 172.21.15.113:0/3740647902 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176108227 0 0) 0x7f42480ca0e0 con 0x7f424806ff20 2024-03-04T07:36:01.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f4245e00640 1 --2- 172.21.15.113:0/3740647902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4248158010 0x7f424806dd50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f42380097c0 tx=0x7f423801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632acefe924dbf08 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:01.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f4244a00640 1 -- 172.21.15.113:0/3740647902 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380b0070 con 0x7f4248158010 2024-03-04T07:36:01.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f4244a00640 1 -- 172.21.15.113:0/3740647902 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42380ab600 con 0x7f4248158010 2024-03-04T07:36:01.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f4244a00640 1 -- 172.21.15.113:0/3740647902 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380ab9d0 con 0x7f4248158010 2024-03-04T07:36:01.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f424cc00640 1 -- 172.21.15.113:0/3740647902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4248158010 msgr2=0x7f424806dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:01.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f424cc00640 1 --2- 172.21.15.113:0/3740647902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4248158010 0x7f424806dd50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f42380097c0 tx=0x7f423801c4a0 comp rx=0 tx=0).stop 2024-03-04T07:36:01.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f424cc00640 1 -- 172.21.15.113:0/3740647902 shutdown_connections 2024-03-04T07:36:01.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f424cc00640 1 --2- 172.21.15.113:0/3740647902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f42480ce120 0x7f42480ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:01.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f424cc00640 1 --2- 172.21.15.113:0/3740647902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4248158010 0x7f424806dd50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:01.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f424cc00640 1 -- 172.21.15.113:0/3740647902 >> 172.21.15.113:0/3740647902 conn(0x7f42480734d0 msgr2=0x7f4248066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:01.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f424cc00640 1 -- 172.21.15.113:0/3740647902 shutdown_connections 2024-03-04T07:36:01.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.975+0000 7f424cc00640 1 -- 172.21.15.113:0/3740647902 wait complete. 2024-03-04T07:36:01.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f424cc00640 1 Processor -- start 2024-03-04T07:36:01.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f424cc00640 1 -- start start 2024-03-04T07:36:01.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f424cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f424806ff20 0x7f42481f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:01.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f424cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42480ce120 0x7f42481f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:01.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f424cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4248158010 0x7f4248141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:01.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f424cc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42481583f0 con 0x7f424806ff20 2024-03-04T07:36:01.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f424cc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424815b920 con 0x7f42480ce120 2024-03-04T07:36:01.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f424cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424806c170 con 0x7f4248158010 2024-03-04T07:36:01.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4245e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f424806ff20 0x7f42481f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:01.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4245e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f424806ff20 0x7f42481f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41738/0 (socket says 172.21.15.113:41738) 2024-03-04T07:36:01.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4245e00640 1 -- 172.21.15.113:0/1974928666 learned_addr learned my addr 172.21.15.113:0/1974928666 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:01.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4246800640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4248158010 0x7f4248141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:01.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4245400640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42480ce120 0x7f42481f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4245e00640 1 -- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4248158010 msgr2=0x7f4248141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4245e00640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4248158010 0x7f4248141c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4245e00640 1 -- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42480ce120 msgr2=0x7f42481f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4245e00640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42480ce120 0x7f42481f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4245e00640 1 -- 172.21.15.113:0/1974928666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480c85a0 con 0x7f424806ff20 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.979+0000 7f4245e00640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f424806ff20 0x7f42481f7430 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f423801a0d0 tx=0x7f423801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.983+0000 7f423e000640 1 -- 172.21.15.113:0/1974928666 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380b0050 con 0x7f424806ff20 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.983+0000 7f423e000640 1 -- 172.21.15.113:0/1974928666 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42380bc070 con 0x7f424806ff20 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.983+0000 7f424cc00640 1 -- 172.21.15.113:0/1974928666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4248140d60 con 0x7f424806ff20 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.983+0000 7f423e000640 1 -- 172.21.15.113:0/1974928666 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380b5850 con 0x7f424806ff20 2024-03-04T07:36:01.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.983+0000 7f424cc00640 1 -- 172.21.15.113:0/1974928666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4248159b50 con 0x7f424806ff20 2024-03-04T07:36:01.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.983+0000 7f424cc00640 1 -- 172.21.15.113:0/1974928666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424806e880 con 0x7f424806ff20 2024-03-04T07:36:01.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.983+0000 7f423e000640 1 -- 172.21.15.113:0/1974928666 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42380b59f0 con 0x7f424806ff20 2024-03-04T07:36:01.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.983+0000 7f423e000640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4228066f90 0x7f4228069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:01.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.983+0000 7f423e000640 1 -- 172.21.15.113:0/1974928666 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f423811f940 con 0x7f424806ff20 2024-03-04T07:36:01.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.983+0000 7f4245400640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4228066f90 0x7f4228069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:01.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.987+0000 7f4245400640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4228066f90 0x7f4228069450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f424815b8c0 tx=0x7f4234007c10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:01.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:01.987+0000 7f423e000640 1 -- 172.21.15.113:0/1974928666 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4238016090 con 0x7f424806ff20 2024-03-04T07:36:02.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.175+0000 7f424cc00640 1 -- 172.21.15.113:0/1974928666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4248075c00 con 0x7f424806ff20 2024-03-04T07:36:02.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.175+0000 7f423e000640 1 -- 172.21.15.113:0/1974928666 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f423811f030 con 0x7f424806ff20 2024-03-04T07:36:02.179 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:02.179 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:36:02.179 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:36:02.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 -- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4228066f90 msgr2=0x7f4228069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:02.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4228066f90 0x7f4228069450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f424815b8c0 tx=0x7f4234007c10 comp rx=0 tx=0).stop 2024-03-04T07:36:02.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 -- 172.21.15.113:0/1974928666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f424806ff20 msgr2=0x7f42481f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:02.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f424806ff20 0x7f42481f7430 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f423801a0d0 tx=0x7f423801ace0 comp rx=0 tx=0).stop 2024-03-04T07:36:02.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 -- 172.21.15.113:0/1974928666 shutdown_connections 2024-03-04T07:36:02.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4248158010 0x7f4248141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:02.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42480ce120 0x7f42481f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:02.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4228066f90 0x7f4228069450 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:02.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 --2- 172.21.15.113:0/1974928666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f424806ff20 0x7f42481f7430 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:02.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 -- 172.21.15.113:0/1974928666 >> 172.21.15.113:0/1974928666 conn(0x7f42480734d0 msgr2=0x7f424814dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:02.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 -- 172.21.15.113:0/1974928666 shutdown_connections 2024-03-04T07:36:02.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:02.179+0000 7f424cc00640 1 -- 172.21.15.113:0/1974928666 wait complete. 2024-03-04T07:36:02.201 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:36:04.203 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:04.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.335+0000 7f6716c00640 1 Processor -- start 2024-03-04T07:36:04.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.335+0000 7f6716c00640 1 -- start start 2024-03-04T07:36:04.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f6716c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6710152a50 0x7f67100ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:04.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f6716c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f671006ff20 0x7f67100ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:04.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f6716c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6710152e30 con 0x7f67100c8b90 2024-03-04T07:36:04.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f6716c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67100c31b0 con 0x7f671006ff20 2024-03-04T07:36:04.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f6716c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671006f840 con 0x7f6710152a50 2024-03-04T07:36:04.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f670f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f671006ff20 0x7f67100ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:04.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f670f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f671006ff20 0x7f67100ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41740/0 (socket says 172.21.15.113:41740) 2024-03-04T07:36:04.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f670f400640 1 -- 172.21.15.113:0/1387069508 learned_addr learned my addr 172.21.15.113:0/1387069508 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:04.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f6714c00640 1 --1- 172.21.15.113:0/1387069508 >> v1:172.21.15.113:6789/0 conn(0x7f67100c8b90 0x7f671015bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36150/0 (socket says 172.21.15.113:36150) 2024-03-04T07:36:04.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f670fe00640 1 --2- 172.21.15.113:0/1387069508 >> v2:172.21.15.136:3300/0 conn(0x7f6710152a50 0x7f67100ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:04.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f670f400640 1 -- 172.21.15.113:0/1387069508 >> v2:172.21.15.136:3300/0 conn(0x7f6710152a50 msgr2=0x7f67100ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:04.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f670f400640 1 --2- 172.21.15.113:0/1387069508 >> v2:172.21.15.136:3300/0 conn(0x7f6710152a50 0x7f67100ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:04.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f670f400640 1 -- 172.21.15.113:0/1387069508 >> v1:172.21.15.113:6789/0 conn(0x7f67100c8b90 legacy=0x7f671015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:04.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f670ea00640 1 -- 172.21.15.113:0/1387069508 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134062709 0 0) 0x7f6710152e30 con 0x7f67100c8b90 2024-03-04T07:36:04.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.339+0000 7f670f400640 1 -- 172.21.15.113:0/1387069508 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671006f280 con 0x7f671006ff20 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f670f400640 1 --2- 172.21.15.113:0/1387069508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 0x7f67100ce660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6710067cd0 tx=0x7f66fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613ea003a7fdb781 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f670ea00640 1 -- 172.21.15.113:0/1387069508 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0aa680 con 0x7f671006ff20 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f670ea00640 1 -- 172.21.15.113:0/1387069508 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66fc0af070 con 0x7f671006ff20 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f670ea00640 1 -- 172.21.15.113:0/1387069508 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66fc0aaa40 con 0x7f671006ff20 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 -- 172.21.15.113:0/1387069508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 msgr2=0x7f67100ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 --2- 172.21.15.113:0/1387069508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 0x7f67100ce660 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6710067cd0 tx=0x7f66fc01c520 comp rx=0 tx=0).stop 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 -- 172.21.15.113:0/1387069508 shutdown_connections 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 --2- 172.21.15.113:0/1387069508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 0x7f67100ce660 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 --2- 172.21.15.113:0/1387069508 >> v2:172.21.15.136:3300/0 conn(0x7f6710152a50 0x7f67100ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 -- 172.21.15.113:0/1387069508 >> 172.21.15.113:0/1387069508 conn(0x7f67100734d0 msgr2=0x7f6710066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 -- 172.21.15.113:0/1387069508 shutdown_connections 2024-03-04T07:36:04.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 -- 172.21.15.113:0/1387069508 wait complete. 2024-03-04T07:36:04.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 Processor -- start 2024-03-04T07:36:04.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 -- start start 2024-03-04T07:36:04.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 0x7f67101570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:04.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67100c8b90 0x7f6710162290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:04.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6710152a50 0x7f671006c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:04.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6710165a00 con 0x7f671006ff20 2024-03-04T07:36:04.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6710165880 con 0x7f67100c8b90 2024-03-04T07:36:04.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f6716c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6710165b80 con 0x7f6710152a50 2024-03-04T07:36:04.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f670fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 0x7f67101570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:04.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f670fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 0x7f67101570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41756/0 (socket says 172.21.15.113:41756) 2024-03-04T07:36:04.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.343+0000 7f670fe00640 1 -- 172.21.15.113:0/1177131094 learned_addr learned my addr 172.21.15.113:0/1177131094 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:04.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f670f400640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67100c8b90 0x7f6710162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:04.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f6714c00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6710152a50 0x7f671006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:04.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f670fe00640 1 -- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6710152a50 msgr2=0x7f671006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:04.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f670fe00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6710152a50 0x7f671006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:04.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f670fe00640 1 -- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67100c8b90 msgr2=0x7f6710162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:04.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f670fe00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67100c8b90 0x7f6710162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:04.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f670fe00640 1 -- 172.21.15.113:0/1177131094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6710157ad0 con 0x7f671006ff20 2024-03-04T07:36:04.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f670fe00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 0x7f67101570b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f670000e960 tx=0x7f670000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:04.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f6707e00640 1 -- 172.21.15.113:0/1177131094 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700017070 con 0x7f671006ff20 2024-03-04T07:36:04.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f6707e00640 1 -- 172.21.15.113:0/1177131094 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670000ca00 con 0x7f671006ff20 2024-03-04T07:36:04.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f6716c00640 1 -- 172.21.15.113:0/1177131094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671006dbd0 con 0x7f671006ff20 2024-03-04T07:36:04.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f6707e00640 1 -- 172.21.15.113:0/1177131094 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670000cdb0 con 0x7f671006ff20 2024-03-04T07:36:04.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f6716c00640 1 -- 172.21.15.113:0/1177131094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671006d8f0 con 0x7f671006ff20 2024-03-04T07:36:04.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f6716c00640 1 -- 172.21.15.113:0/1177131094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6710075c00 con 0x7f671006ff20 2024-03-04T07:36:04.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.347+0000 7f6707e00640 1 -- 172.21.15.113:0/1177131094 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f670001a070 con 0x7f671006ff20 2024-03-04T07:36:04.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.351+0000 7f6707e00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66f0066f60 0x7f66f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:04.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.351+0000 7f6707e00640 1 -- 172.21.15.113:0/1177131094 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6700085060 con 0x7f671006ff20 2024-03-04T07:36:04.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.351+0000 7f670f400640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66f0066f60 0x7f66f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:04.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.351+0000 7f670f400640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66f0066f60 0x7f66f0069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0ae000 tx=0x7f66fc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:04.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.355+0000 7f6707e00640 1 -- 172.21.15.113:0/1177131094 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6700056080 con 0x7f671006ff20 2024-03-04T07:36:04.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 -- 172.21.15.113:0/1177131094 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67100c3680 con 0x7f671006ff20 2024-03-04T07:36:04.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6707e00640 1 -- 172.21.15.113:0/1177131094 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f6700055980 con 0x7f671006ff20 2024-03-04T07:36:04.545 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:04.545 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:34:13.676159+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6836/4038214481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4038214481},{"type":"v1","addr":"172.21.15.113:6836","nonce":4038214481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4253,4256,4276]},"id":1}]} 2024-03-04T07:36:04.545 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 13 2024-03-04T07:36:04.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 -- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66f0066f60 msgr2=0x7f66f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66f0066f60 0x7f66f0069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0ae000 tx=0x7f66fc0a7000 comp rx=0 tx=0).stop 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 -- 172.21.15.113:0/1177131094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 msgr2=0x7f67101570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 0x7f67101570b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f670000e960 tx=0x7f670000ee20 comp rx=0 tx=0).stop 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 -- 172.21.15.113:0/1177131094 shutdown_connections 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6710152a50 0x7f671006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67100c8b90 0x7f6710162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66f0066f60 0x7f66f0069420 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 --2- 172.21.15.113:0/1177131094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f671006ff20 0x7f67101570b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 -- 172.21.15.113:0/1177131094 >> 172.21.15.113:0/1177131094 conn(0x7f67100734d0 msgr2=0x7f67100c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 -- 172.21.15.113:0/1177131094 shutdown_connections 2024-03-04T07:36:04.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:04.543+0000 7f6716c00640 1 -- 172.21.15.113:0/1177131094 wait complete. 2024-03-04T07:36:04.569 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:36:06.570 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:06.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.703+0000 7f5a2ba00640 1 Processor -- start 2024-03-04T07:36:06.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.703+0000 7f5a2ba00640 1 -- start start 2024-03-04T07:36:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a2ba00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5a24154e50 0x7f5a240c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a2ba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 0x7f5a241653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a2ba00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2415b590 con 0x7f5a240c25e0 2024-03-04T07:36:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a2ba00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2415b850 con 0x7f5a24154e50 2024-03-04T07:36:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a2ba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24072810 con 0x7f5a240733f0 2024-03-04T07:36:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a23e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5a240c25e0 0x7f5a240cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36152/0 (socket says 172.21.15.113:36152) 2024-03-04T07:36:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a23e00640 1 -- 172.21.15.113:0/2593901310 learned_addr learned my addr 172.21.15.113:0/2593901310 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:36:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a29000640 1 --2- 172.21.15.113:0/2593901310 >> v2:172.21.15.136:3300/0 conn(0x7f5a24154e50 0x7f5a240c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a29a00640 1 --2- 172.21.15.113:0/2593901310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 0x7f5a241653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:06.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a23400640 1 -- 172.21.15.113:0/2593901310 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294221068 0 0) 0x7f5a2415b590 con 0x7f5a240c25e0 2024-03-04T07:36:06.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a23400640 1 -- 172.21.15.113:0/2593901310 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a0c003530 con 0x7f5a240c25e0 2024-03-04T07:36:06.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a29a00640 1 -- 172.21.15.113:0/2593901310 >> v2:172.21.15.136:3300/0 conn(0x7f5a24154e50 msgr2=0x7f5a240c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:06.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a29a00640 1 --2- 172.21.15.113:0/2593901310 >> v2:172.21.15.136:3300/0 conn(0x7f5a24154e50 0x7f5a240c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:06.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a29a00640 1 -- 172.21.15.113:0/2593901310 >> v1:172.21.15.113:6789/0 conn(0x7f5a240c25e0 legacy=0x7f5a240cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:06.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.707+0000 7f5a29a00640 1 -- 172.21.15.113:0/2593901310 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a24072af0 con 0x7f5a240733f0 2024-03-04T07:36:06.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a29a00640 1 --2- 172.21.15.113:0/2593901310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 0x7f5a241653b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009d90 tx=0x7f5a1c01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c28affcf5f9f78 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:06.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a23400640 1 -- 172.21.15.113:0/2593901310 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0b4070 con 0x7f5a240733f0 2024-03-04T07:36:06.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a23400640 1 -- 172.21.15.113:0/2593901310 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c0af600 con 0x7f5a240733f0 2024-03-04T07:36:06.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a23400640 1 -- 172.21.15.113:0/2593901310 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0af9b0 con 0x7f5a240733f0 2024-03-04T07:36:06.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 -- 172.21.15.113:0/2593901310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 msgr2=0x7f5a241653b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:06.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 --2- 172.21.15.113:0/2593901310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 0x7f5a241653b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009d90 tx=0x7f5a1c01f150 comp rx=0 tx=0).stop 2024-03-04T07:36:06.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 -- 172.21.15.113:0/2593901310 shutdown_connections 2024-03-04T07:36:06.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 --2- 172.21.15.113:0/2593901310 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 0x7f5a241653b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:06.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 --2- 172.21.15.113:0/2593901310 >> v2:172.21.15.136:3300/0 conn(0x7f5a24154e50 0x7f5a240c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:06.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 -- 172.21.15.113:0/2593901310 >> 172.21.15.113:0/2593901310 conn(0x7f5a240671b0 msgr2=0x7f5a2406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:06.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 -- 172.21.15.113:0/2593901310 shutdown_connections 2024-03-04T07:36:06.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 -- 172.21.15.113:0/2593901310 wait complete. 2024-03-04T07:36:06.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 Processor -- start 2024-03-04T07:36:06.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 -- start start 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 0x7f5a241f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a24154e50 0x7f5a24150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a24155c50 0x7f5a2415bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2406b790 con 0x7f5a24155c50 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24155230 con 0x7f5a24154e50 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a2ba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a241f1aa0 con 0x7f5a240733f0 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a29a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a24155c50 0x7f5a2415bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a29a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a24155c50 0x7f5a2415bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41766/0 (socket says 172.21.15.113:41766) 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.711+0000 7f5a29a00640 1 -- 172.21.15.113:0/3378975587 learned_addr learned my addr 172.21.15.113:0/3378975587 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a23e00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a24154e50 0x7f5a24150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:06.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a29000640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 0x7f5a241f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a29a00640 1 -- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 msgr2=0x7f5a241f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a29a00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 0x7f5a241f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a29a00640 1 -- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a24154e50 msgr2=0x7f5a24150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a29a00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a24154e50 0x7f5a24150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a29a00640 1 -- 172.21.15.113:0/3378975587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a241f4f60 con 0x7f5a24155c50 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a29000640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 0x7f5a241f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a29a00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a24155c50 0x7f5a2415bc70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0b2040 tx=0x7f5a1c01bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a20c00640 1 -- 172.21.15.113:0/3378975587 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0b4040 con 0x7f5a24155c50 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a20c00640 1 -- 172.21.15.113:0/3378975587 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c0c1070 con 0x7f5a24155c50 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a20c00640 1 -- 172.21.15.113:0/3378975587 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c009450 con 0x7f5a24155c50 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a2ba00640 1 -- 172.21.15.113:0/3378975587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a24156590 con 0x7f5a24155c50 2024-03-04T07:36:06.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a2ba00640 1 -- 172.21.15.113:0/3378975587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a241f0fc0 con 0x7f5a24155c50 2024-03-04T07:36:06.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.715+0000 7f5a2ba00640 1 -- 172.21.15.113:0/3378975587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a24075c00 con 0x7f5a24155c50 2024-03-04T07:36:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.719+0000 7f5a20c00640 1 -- 172.21.15.113:0/3378975587 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a1c0ba690 con 0x7f5a24155c50 2024-03-04T07:36:06.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.719+0000 7f5a20c00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a04066f60 0x7f5a04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.719+0000 7f5a20c00640 1 -- 172.21.15.113:0/3378975587 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f5a1c123a20 con 0x7f5a24155c50 2024-03-04T07:36:06.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.719+0000 7f5a29000640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a04066f60 0x7f5a04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:06.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.719+0000 7f5a29000640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a04066f60 0x7f5a04069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009b10 tx=0x7f5a18008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:06.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.723+0000 7f5a20c00640 1 -- 172.21.15.113:0/3378975587 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5a1c018cb0 con 0x7f5a24155c50 2024-03-04T07:36:06.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.911+0000 7f5a2ba00640 1 -- 172.21.15.113:0/3378975587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a24158500 con 0x7f5a24155c50 2024-03-04T07:36:06.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.911+0000 7f5a20c00640 1 -- 172.21.15.113:0/3378975587 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f5a1c0f4990 con 0x7f5a24155c50 2024-03-04T07:36:06.916 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:06.916 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:05.584608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:36:06.916 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 14 2024-03-04T07:36:06.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 -- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a04066f60 msgr2=0x7f5a04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:06.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a04066f60 0x7f5a04069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009b10 tx=0x7f5a18008040 comp rx=0 tx=0).stop 2024-03-04T07:36:06.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 -- 172.21.15.113:0/3378975587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a24155c50 msgr2=0x7f5a2415bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:06.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a24155c50 0x7f5a2415bc70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0b2040 tx=0x7f5a1c01bef0 comp rx=0 tx=0).stop 2024-03-04T07:36:06.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 -- 172.21.15.113:0/3378975587 shutdown_connections 2024-03-04T07:36:06.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a04066f60 0x7f5a04069420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:06.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a24155c50 0x7f5a2415bc70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:06.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a24154e50 0x7f5a24150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:06.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 --2- 172.21.15.113:0/3378975587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a240733f0 0x7f5a241f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:06.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 -- 172.21.15.113:0/3378975587 >> 172.21.15.113:0/3378975587 conn(0x7f5a240671b0 msgr2=0x7f5a2406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:06.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 -- 172.21.15.113:0/3378975587 shutdown_connections 2024-03-04T07:36:06.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:06.915+0000 7f5a2ba00640 1 -- 172.21.15.113:0/3378975587 wait complete. 2024-03-04T07:36:06.937 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-03-04T07:36:06.937 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:36:06.937 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a7ca00640 1 Processor -- start 2024-03-04T07:36:07.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a7ca00640 1 -- start start 2024-03-04T07:36:07.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a7ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9a78152a50 0x7f9a780ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:07.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a7ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a7806ff20 0x7f9a780ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:07.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a7ca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9a780c8b90 0x7f9a7815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:07.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a7ca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a780c31b0 con 0x7f9a78152a50 2024-03-04T07:36:07.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a7ca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7806f840 con 0x7f9a780c8b90 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a7ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78152e30 con 0x7f9a7806ff20 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a75e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9a78152a50 0x7f9a780ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a75e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9a78152a50 0x7f9a780ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41778/0 (socket says 172.21.15.113:41778) 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a75e00640 1 -- 172.21.15.113:0/1768960222 learned_addr learned my addr 172.21.15.113:0/1768960222 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a6d400640 1 --2- 172.21.15.113:0/1768960222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a7806ff20 0x7f9a780ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a76800640 1 --2- 172.21.15.113:0/1768960222 >> v2:172.21.15.136:3300/0 conn(0x7f9a780c8b90 0x7f9a7815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a75e00640 1 -- 172.21.15.113:0/1768960222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a7806ff20 msgr2=0x7f9a780ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a75e00640 1 --2- 172.21.15.113:0/1768960222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a7806ff20 0x7f9a780ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a75e00640 1 -- 172.21.15.113:0/1768960222 >> v2:172.21.15.136:3300/0 conn(0x7f9a780c8b90 msgr2=0x7f9a7815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a75e00640 1 --2- 172.21.15.113:0/1768960222 >> v2:172.21.15.136:3300/0 conn(0x7f9a780c8b90 0x7f9a7815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.071+0000 7f9a75e00640 1 -- 172.21.15.113:0/1768960222 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7806f280 con 0x7f9a78152a50 2024-03-04T07:36:07.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a75e00640 1 --2- 172.21.15.113:0/1768960222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a78152a50 0x7f9a780ce120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9a78067bb0 tx=0x7f9a6801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bd6c348d20cda9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a75400640 1 -- 172.21.15.113:0/1768960222 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a680ab650 con 0x7f9a78152a50 2024-03-04T07:36:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a75400640 1 -- 172.21.15.113:0/1768960222 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a680b0070 con 0x7f9a78152a50 2024-03-04T07:36:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a75400640 1 -- 172.21.15.113:0/1768960222 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a680aba10 con 0x7f9a78152a50 2024-03-04T07:36:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a7ca00640 1 -- 172.21.15.113:0/1768960222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a78152a50 msgr2=0x7f9a780ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a7ca00640 1 --2- 172.21.15.113:0/1768960222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a78152a50 0x7f9a780ce120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9a78067bb0 tx=0x7f9a6801c490 comp rx=0 tx=0).stop 2024-03-04T07:36:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a7ca00640 1 -- 172.21.15.113:0/1768960222 shutdown_connections 2024-03-04T07:36:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a7ca00640 1 --2- 172.21.15.113:0/1768960222 >> v2:172.21.15.136:3300/0 conn(0x7f9a780c8b90 0x7f9a7815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a7ca00640 1 --2- 172.21.15.113:0/1768960222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a7806ff20 0x7f9a780ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a7ca00640 1 --2- 172.21.15.113:0/1768960222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a78152a50 0x7f9a780ce120 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a7ca00640 1 -- 172.21.15.113:0/1768960222 >> 172.21.15.113:0/1768960222 conn(0x7f9a780734d0 msgr2=0x7f9a78066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:07.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a7ca00640 1 -- 172.21.15.113:0/1768960222 shutdown_connections 2024-03-04T07:36:07.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.075+0000 7f9a7ca00640 1 -- 172.21.15.113:0/1768960222 wait complete. 2024-03-04T07:36:07.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 Processor -- start 2024-03-04T07:36:07.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 -- start start 2024-03-04T07:36:07.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a7806ff20 0x7f9a7814c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:07.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780c8b90 0x7f9a781f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:07.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a78152a50 0x7f9a781611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:07.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78158f10 con 0x7f9a780c8b90 2024-03-04T07:36:07.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780c3400 con 0x7f9a7806ff20 2024-03-04T07:36:07.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78159090 con 0x7f9a78152a50 2024-03-04T07:36:07.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780c8b90 0x7f9a781f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780c8b90 0x7f9a781f7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41794/0 (socket says 172.21.15.113:41794) 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6d400640 1 -- 172.21.15.113:0/3035188685 learned_addr learned my addr 172.21.15.113:0/3035188685 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a75e00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a7806ff20 0x7f9a7814c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a76800640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a78152a50 0x7f9a781611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6d400640 1 -- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a78152a50 msgr2=0x7f9a781611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6d400640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a78152a50 0x7f9a781611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6d400640 1 -- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a7806ff20 msgr2=0x7f9a7814c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6d400640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a7806ff20 0x7f9a7814c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6d400640 1 -- 172.21.15.113:0/3035188685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a781f5020 con 0x7f9a780c8b90 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a75e00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a7806ff20 0x7f9a7814c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:36:07.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6d400640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780c8b90 0x7f9a781f7cd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000e8b0 tx=0x7f9a6000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:07.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6ea00640 1 -- 172.21.15.113:0/3035188685 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60017070 con 0x7f9a780c8b90 2024-03-04T07:36:07.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6ea00640 1 -- 172.21.15.113:0/3035188685 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6000c940 con 0x7f9a780c8b90 2024-03-04T07:36:07.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a6ea00640 1 -- 172.21.15.113:0/3035188685 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6000cd10 con 0x7f9a780c8b90 2024-03-04T07:36:07.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 -- 172.21.15.113:0/3035188685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a78153b90 con 0x7f9a780c8b90 2024-03-04T07:36:07.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 -- 172.21.15.113:0/3035188685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7815a580 con 0x7f9a780c8b90 2024-03-04T07:36:07.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.079+0000 7f9a7ca00640 1 -- 172.21.15.113:0/3035188685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a78075c00 con 0x7f9a780c8b90 2024-03-04T07:36:07.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.083+0000 7f9a6ea00640 1 -- 172.21.15.113:0/3035188685 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a6001a070 con 0x7f9a780c8b90 2024-03-04T07:36:07.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.083+0000 7f9a6ea00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 0x7f9a54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:07.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.083+0000 7f9a6ea00640 1 -- 172.21.15.113:0/3035188685 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f9a600850e0 con 0x7f9a780c8b90 2024-03-04T07:36:07.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.083+0000 7f9a75e00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 0x7f9a54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:07.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.087+0000 7f9a75e00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 0x7f9a54069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a680af000 tx=0x7f9a68004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:07.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.087+0000 7f9a6ea00640 1 -- 172.21.15.113:0/3035188685 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9a781f5020 con 0x7f9a780c8b90 2024-03-04T07:36:07.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.275+0000 7f9a7ca00640 1 -- 172.21.15.113:0/3035188685 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a780cb700 con 0x7f9a780c8b90 2024-03-04T07:36:07.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.275+0000 7f9a6ea00640 1 -- 172.21.15.113:0/3035188685 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f9a60056010 con 0x7f9a780c8b90 2024-03-04T07:36:07.279 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:07.279 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:05.584608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:36:07.279 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 14 2024-03-04T07:36:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 -- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 msgr2=0x7f9a54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 0x7f9a54069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a680af000 tx=0x7f9a68004740 comp rx=0 tx=0).stop 2024-03-04T07:36:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 -- 172.21.15.113:0/3035188685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780c8b90 msgr2=0x7f9a781f7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780c8b90 0x7f9a781f7cd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000e8b0 tx=0x7f9a6000ed70 comp rx=0 tx=0).stop 2024-03-04T07:36:07.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 -- 172.21.15.113:0/3035188685 shutdown_connections 2024-03-04T07:36:07.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a78152a50 0x7f9a781611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 0x7f9a54069420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a780c8b90 0x7f9a781f7cd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 --2- 172.21.15.113:0/3035188685 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a7806ff20 0x7f9a7814c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:07.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 -- 172.21.15.113:0/3035188685 >> 172.21.15.113:0/3035188685 conn(0x7f9a780734d0 msgr2=0x7f9a78156570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:07.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 -- 172.21.15.113:0/3035188685 shutdown_connections 2024-03-04T07:36:07.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:07.279+0000 7f9a7ca00640 1 -- 172.21.15.113:0/3035188685 wait complete. 2024-03-04T07:36:07.306 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 14, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 14, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4250": { "addr": "172.21.15.113:6833/3005395426", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 3005395426, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 3005395426, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4250, "incarnation": 14, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:replay", "state_seq": 1 }, "gid_4256": { "addr": "172.21.15.113:6837/1310042629", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1310042629, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1310042629, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4256, "incarnation": 10, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4276": { "addr": "172.21.15.136:6835/3348592059", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3348592059, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3348592059, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4276, "incarnation": 4, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 24, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:36:05.584608+0000", "qdb_cluster": [ 4256, 4276 ], "qdb_leader": 4276, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4276, "mds_1": 4256, "mds_2": 4250 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:36:09.308 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:09.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe98a00640 1 Processor -- start 2024-03-04T07:36:09.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe98a00640 1 -- start start 2024-03-04T07:36:09.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe98a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 0x7fbe9406ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:09.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe98a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe940c18e0 0x7fbe940c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:09.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe98a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9415bac0 con 0x7fbe9406d520 2024-03-04T07:36:09.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe98a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9406f2d0 con 0x7fbe940c18e0 2024-03-04T07:36:09.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe98a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940c7e10 con 0x7fbe94154e20 2024-03-04T07:36:09.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe93e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe940c18e0 0x7fbe940c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:09.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe93e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe940c18e0 0x7fbe940c1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52110/0 (socket says 172.21.15.113:52110) 2024-03-04T07:36:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe93e00640 1 -- 172.21.15.113:0/3560088960 learned_addr learned my addr 172.21.15.113:0/3560088960 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe93400640 1 --2- 172.21.15.113:0/3560088960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 0x7fbe9406ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe93e00640 1 -- 172.21.15.113:0/3560088960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 msgr2=0x7fbe9406ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe93e00640 1 --2- 172.21.15.113:0/3560088960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 0x7fbe9406ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe93e00640 1 -- 172.21.15.113:0/3560088960 >> v1:172.21.15.136:6789/0 conn(0x7fbe9406d520 legacy=0x7fbe9415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:09.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.443+0000 7fbe93e00640 1 -- 172.21.15.113:0/3560088960 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe940c80f0 con 0x7fbe940c18e0 2024-03-04T07:36:09.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe93e00640 1 --2- 172.21.15.113:0/3560088960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe940c18e0 0x7fbe940c1d50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009e50 tx=0x7fbe8c01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb01534a9c04f63a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:09.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe92000640 1 -- 172.21.15.113:0/3560088960 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0b4070 con 0x7fbe940c18e0 2024-03-04T07:36:09.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe92000640 1 -- 172.21.15.113:0/3560088960 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c0af400 con 0x7fbe940c18e0 2024-03-04T07:36:09.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe92000640 1 -- 172.21.15.113:0/3560088960 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0af7d0 con 0x7fbe940c18e0 2024-03-04T07:36:09.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe98a00640 1 -- 172.21.15.113:0/3560088960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe940c18e0 msgr2=0x7fbe940c1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:09.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe98a00640 1 --2- 172.21.15.113:0/3560088960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe940c18e0 0x7fbe940c1d50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009e50 tx=0x7fbe8c01f210 comp rx=0 tx=0).stop 2024-03-04T07:36:09.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe98a00640 1 -- 172.21.15.113:0/3560088960 shutdown_connections 2024-03-04T07:36:09.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe98a00640 1 --2- 172.21.15.113:0/3560088960 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe940c18e0 0x7fbe940c1d50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:09.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe98a00640 1 --2- 172.21.15.113:0/3560088960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 0x7fbe9406ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:09.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe98a00640 1 -- 172.21.15.113:0/3560088960 >> 172.21.15.113:0/3560088960 conn(0x7fbe9406e2a0 msgr2=0x7fbe9406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:09.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe98a00640 1 -- 172.21.15.113:0/3560088960 shutdown_connections 2024-03-04T07:36:09.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.447+0000 7fbe98a00640 1 -- 172.21.15.113:0/3560088960 wait complete. 2024-03-04T07:36:09.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 Processor -- start 2024-03-04T07:36:09.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 -- start start 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9406d520 0x7fbe941f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940c18e0 0x7fbe9406e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 0x7fbe9415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9415ae90 con 0x7fbe9406d520 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940c86b0 con 0x7fbe940c18e0 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9406c230 con 0x7fbe94154e20 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9406d520 0x7fbe941f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9406d520 0x7fbe941f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52112/0 (socket says 172.21.15.113:52112) 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93400640 1 -- 172.21.15.113:0/3524792628 learned_addr learned my addr 172.21.15.113:0/3524792628 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe92a00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940c18e0 0x7fbe9406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93e00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 0x7fbe9415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93400640 1 -- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 msgr2=0x7fbe9415bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:09.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93400640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 0x7fbe9415bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93400640 1 -- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940c18e0 msgr2=0x7fbe9406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93400640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940c18e0 0x7fbe9406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93400640 1 -- 172.21.15.113:0/3524792628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe940b0a40 con 0x7fbe9406d520 2024-03-04T07:36:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93e00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 0x7fbe9415bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:36:09.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe93400640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9406d520 0x7fbe941f4840 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400d620 tx=0x7fbe8400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:09.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe8b400640 1 -- 172.21.15.113:0/3524792628 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400b620 con 0x7fbe9406d520 2024-03-04T07:36:09.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe8b400640 1 -- 172.21.15.113:0/3524792628 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe84019070 con 0x7fbe9406d520 2024-03-04T07:36:09.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe8b400640 1 -- 172.21.15.113:0/3524792628 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400b9e0 con 0x7fbe9406d520 2024-03-04T07:36:09.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 -- 172.21.15.113:0/3524792628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9415d2e0 con 0x7fbe9406d520 2024-03-04T07:36:09.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 -- 172.21.15.113:0/3524792628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9415d8a0 con 0x7fbe9406d520 2024-03-04T07:36:09.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.451+0000 7fbe98a00640 1 -- 172.21.15.113:0/3524792628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe94075a80 con 0x7fbe9406d520 2024-03-04T07:36:09.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.455+0000 7fbe8b400640 1 -- 172.21.15.113:0/3524792628 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe8400bb80 con 0x7fbe9406d520 2024-03-04T07:36:09.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.455+0000 7fbe8b400640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe74066f60 0x7fbe74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:09.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.455+0000 7fbe8b400640 1 -- 172.21.15.113:0/3524792628 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fbe84085f80 con 0x7fbe9406d520 2024-03-04T07:36:09.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.455+0000 7fbe92a00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe74066f60 0x7fbe74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:09.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.455+0000 7fbe92a00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe74066f60 0x7fbe74069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe94151f70 tx=0x7fbe80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:09.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.459+0000 7fbe8b400640 1 -- 172.21.15.113:0/3524792628 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbe84056f50 con 0x7fbe9406d520 2024-03-04T07:36:09.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.647+0000 7fbe98a00640 1 -- 172.21.15.113:0/3524792628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe94159e10 con 0x7fbe9406d520 2024-03-04T07:36:09.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.647+0000 7fbe8b400640 1 -- 172.21.15.113:0/3524792628 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5815 (secure 0 0 0) 0x7fbe84056850 con 0x7fbe9406d520 2024-03-04T07:36:09.651 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:09.651 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:05.584608+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:36:09.652 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 14 2024-03-04T07:36:09.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 -- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe74066f60 msgr2=0x7fbe74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:09.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe74066f60 0x7fbe74069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe94151f70 tx=0x7fbe80009040 comp rx=0 tx=0).stop 2024-03-04T07:36:09.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 -- 172.21.15.113:0/3524792628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9406d520 msgr2=0x7fbe941f4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:09.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9406d520 0x7fbe941f4840 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400d620 tx=0x7fbe8400dae0 comp rx=0 tx=0).stop 2024-03-04T07:36:09.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 -- 172.21.15.113:0/3524792628 shutdown_connections 2024-03-04T07:36:09.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe94154e20 0x7fbe9415bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:09.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe940c18e0 0x7fbe9406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:09.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe74066f60 0x7fbe74069420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:09.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 --2- 172.21.15.113:0/3524792628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9406d520 0x7fbe941f4840 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:09.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 -- 172.21.15.113:0/3524792628 >> 172.21.15.113:0/3524792628 conn(0x7fbe9406e2a0 msgr2=0x7fbe94159470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:09.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 -- 172.21.15.113:0/3524792628 shutdown_connections 2024-03-04T07:36:09.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:09.651+0000 7fbe98a00640 1 -- 172.21.15.113:0/3524792628 wait complete. 2024-03-04T07:36:09.674 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:36:11.675 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:11.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.807+0000 7f1a07000640 1 Processor -- start 2024-03-04T07:36:11.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.807+0000 7f1a07000640 1 -- start start 2024-03-04T07:36:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f1a07000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a001527f0 0x7f1a00070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f1a07000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1a000c8200 0x7f1a00072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f1a07000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a001591f0 con 0x7f1a00152bd0 2024-03-04T07:36:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f1a07000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a00071df0 con 0x7f1a000c8200 2024-03-04T07:36:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f1a07000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a001597b0 con 0x7f1a001527f0 2024-03-04T07:36:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f19ff400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1a000c8200 0x7f1a00072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f19ffe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a001527f0 0x7f1a00070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f1a05000640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f1a00152bd0 0x7f1a001587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:60274/0 (socket says 172.21.15.113:60274) 2024-03-04T07:36:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f19ffe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a001527f0 0x7f1a00070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:56386/0 (socket says 172.21.15.113:56386) 2024-03-04T07:36:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f1a05000640 1 -- 172.21.15.113:0/2829765384 learned_addr learned my addr 172.21.15.113:0/2829765384 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:36:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f19ff400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1a000c8200 0x7f1a00072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:34660/0 (socket says 172.21.15.113:34660) 2024-03-04T07:36:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f19ff400640 1 -- 172.21.15.113:0/2829765384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a001527f0 msgr2=0x7f1a00070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f19ff400640 1 --2- 172.21.15.113:0/2829765384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a001527f0 0x7f1a00070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f19ff400640 1 -- 172.21.15.113:0/2829765384 >> v1:172.21.15.136:6789/0 conn(0x7f1a00152bd0 legacy=0x7f1a001587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f19ff400640 1 -- 172.21.15.113:0/2829765384 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a001594d0 con 0x7f1a000c8200 2024-03-04T07:36:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.811+0000 7f19ffe00640 1 --2- 172.21.15.113:0/2829765384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a001527f0 0x7f1a00070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:36:11.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f19ff400640 1 --2- 172.21.15.113:0/2829765384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a000c8200 0x7f1a00072280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0097c0 tx=0x7f19ec01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5172ea98594533ba server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:11.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f19fea00640 1 -- 172.21.15.113:0/2829765384 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0b1070 con 0x7f1a000c8200 2024-03-04T07:36:11.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f19fea00640 1 -- 172.21.15.113:0/2829765384 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec0ac410 con 0x7f1a000c8200 2024-03-04T07:36:11.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f19fea00640 1 -- 172.21.15.113:0/2829765384 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0ac7e0 con 0x7f1a000c8200 2024-03-04T07:36:11.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f1a07000640 1 -- 172.21.15.113:0/2829765384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a000c8200 msgr2=0x7f1a00072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:11.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f1a07000640 1 --2- 172.21.15.113:0/2829765384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a000c8200 0x7f1a00072280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0097c0 tx=0x7f19ec01c4a0 comp rx=0 tx=0).stop 2024-03-04T07:36:11.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f1a07000640 1 -- 172.21.15.113:0/2829765384 shutdown_connections 2024-03-04T07:36:11.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f1a07000640 1 --2- 172.21.15.113:0/2829765384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a000c8200 0x7f1a00072280 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:11.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f1a07000640 1 --2- 172.21.15.113:0/2829765384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a001527f0 0x7f1a00070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:11.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f1a07000640 1 -- 172.21.15.113:0/2829765384 >> 172.21.15.113:0/2829765384 conn(0x7f1a0006e820 msgr2=0x7f1a0006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:11.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f1a07000640 1 -- 172.21.15.113:0/2829765384 shutdown_connections 2024-03-04T07:36:11.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f1a07000640 1 -- 172.21.15.113:0/2829765384 wait complete. 2024-03-04T07:36:11.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f1a07000640 1 Processor -- start 2024-03-04T07:36:11.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.815+0000 7f1a07000640 1 -- start start 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a07000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a000c8200 0x7f1a00158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a07000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a001527f0 0x7f1a0014a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a07000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a00152bd0 0x7f1a0014a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a07000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a00165160 con 0x7f1a001527f0 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a07000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a000727c0 con 0x7f1a000c8200 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a07000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a001652e0 con 0x7f1a00152bd0 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19ff400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a001527f0 0x7f1a0014a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19ff400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a001527f0 0x7f1a0014a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52118/0 (socket says 172.21.15.113:52118) 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19ff400640 1 -- 172.21.15.113:0/911466521 learned_addr learned my addr 172.21.15.113:0/911466521 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19ffe00640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a000c8200 0x7f1a00158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a05000640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a00152bd0 0x7f1a0014a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19ff400640 1 -- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a00152bd0 msgr2=0x7f1a0014a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19ff400640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a00152bd0 0x7f1a0014a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19ff400640 1 -- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a000c8200 msgr2=0x7f1a00158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19ff400640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a000c8200 0x7f1a00158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19ff400640 1 -- 172.21.15.113:0/911466521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a000b1500 con 0x7f1a001527f0 2024-03-04T07:36:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a05000640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a00152bd0 0x7f1a0014a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:36:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19ff400640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a001527f0 0x7f1a0014a170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f19ec01a0d0 tx=0x7f19ec019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19f7e00640 1 -- 172.21.15.113:0/911466521 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0b9020 con 0x7f1a001527f0 2024-03-04T07:36:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19f7e00640 1 -- 172.21.15.113:0/911466521 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec0acd90 con 0x7f1a001527f0 2024-03-04T07:36:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19f7e00640 1 -- 172.21.15.113:0/911466521 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0b1040 con 0x7f1a001527f0 2024-03-04T07:36:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a07000640 1 -- 172.21.15.113:0/911466521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a00158b40 con 0x7f1a001527f0 2024-03-04T07:36:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a07000640 1 -- 172.21.15.113:0/911466521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a000b1770 con 0x7f1a001527f0 2024-03-04T07:36:11.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f1a07000640 1 -- 172.21.15.113:0/911466521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a00075c00 con 0x7f1a001527f0 2024-03-04T07:36:11.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.819+0000 7f19f7e00640 1 -- 172.21.15.113:0/911466521 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19ec0bc020 con 0x7f1a001527f0 2024-03-04T07:36:11.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.823+0000 7f19f7e00640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19e0066f60 0x7f19e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:11.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.823+0000 7f19f7e00640 1 -- 172.21.15.113:0/911466521 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f19ec11e990 con 0x7f1a001527f0 2024-03-04T07:36:11.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.823+0000 7f19ffe00640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19e0066f60 0x7f19e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:11.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.823+0000 7f19ffe00640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19e0066f60 0x7f19e0069420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009580 tx=0x7f19f0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:11.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:11.827+0000 7f19f7e00640 1 -- 172.21.15.113:0/911466521 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f19ec0c3030 con 0x7f1a001527f0 2024-03-04T07:36:12.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 -- 172.21.15.113:0/911466521 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a0014e730 con 0x7f1a001527f0 2024-03-04T07:36:12.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f19f7e00640 1 -- 172.21.15.113:0/911466521 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+5817 (secure 0 0 0) 0x7f19ec016090 con 0x7f1a001527f0 2024-03-04T07:36:12.017 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:12.018 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:10.603476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:resolve","state_seq":37,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:36:12.018 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 15 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 -- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19e0066f60 msgr2=0x7f19e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19e0066f60 0x7f19e0069420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009580 tx=0x7f19f0009190 comp rx=0 tx=0).stop 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 -- 172.21.15.113:0/911466521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a001527f0 msgr2=0x7f1a0014a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a001527f0 0x7f1a0014a170 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f19ec01a0d0 tx=0x7f19ec019240 comp rx=0 tx=0).stop 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 -- 172.21.15.113:0/911466521 shutdown_connections 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a00152bd0 0x7f1a0014a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19e0066f60 0x7f19e0069420 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a001527f0 0x7f1a0014a170 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 --2- 172.21.15.113:0/911466521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a000c8200 0x7f1a00158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 -- 172.21.15.113:0/911466521 >> 172.21.15.113:0/911466521 conn(0x7f1a0006e820 msgr2=0x7f1a0006ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 -- 172.21.15.113:0/911466521 shutdown_connections 2024-03-04T07:36:12.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:12.015+0000 7f1a07000640 1 -- 172.21.15.113:0/911466521 wait complete. 2024-03-04T07:36:12.045 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:36:14.046 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:14.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.179+0000 7ff94e000640 1 Processor -- start 2024-03-04T07:36:14.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.179+0000 7ff94e000640 1 -- start start 2024-03-04T07:36:14.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff94e000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9481527f0 0x7ff948070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:14.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff94e000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff948152bd0 0x7ff9481587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:14.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff94e000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9481591f0 con 0x7ff9480c8200 2024-03-04T07:36:14.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff94e000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff948071df0 con 0x7ff948152bd0 2024-03-04T07:36:14.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff94e000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9481597b0 con 0x7ff9481527f0 2024-03-04T07:36:14.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff947e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff948152bd0 0x7ff9481587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:14.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff947400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9481527f0 0x7ff948070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:14.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff947e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff948152bd0 0x7ff9481587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:34684/0 (socket says 172.21.15.113:34684) 2024-03-04T07:36:14.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff947e00640 1 -- 172.21.15.113:0/1558791566 learned_addr learned my addr 172.21.15.113:0/1558791566 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:14.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff93fe00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7ff9480c8200 0x7ff948072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:60286/0 (socket says 172.21.15.113:60286) 2024-03-04T07:36:14.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff947e00640 1 -- 172.21.15.113:0/1558791566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9481527f0 msgr2=0x7ff948070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:14.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff947e00640 1 --2- 172.21.15.113:0/1558791566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9481527f0 0x7ff948070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:14.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff947e00640 1 -- 172.21.15.113:0/1558791566 >> v1:172.21.15.136:6789/0 conn(0x7ff9480c8200 legacy=0x7ff948072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:14.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff947e00640 1 -- 172.21.15.113:0/1558791566 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9481594d0 con 0x7ff948152bd0 2024-03-04T07:36:14.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.183+0000 7ff947400640 1 --2- 172.21.15.113:0/1558791566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9481527f0 0x7ff948070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:36:14.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff947e00640 1 --2- 172.21.15.113:0/1558791566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff948152bd0 0x7ff9481587d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff940009e50 tx=0x7ff94001ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b0296d8c205da server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:14.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff946a00640 1 -- 172.21.15.113:0/1558791566 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400a57f0 con 0x7ff948152bd0 2024-03-04T07:36:14.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff946a00640 1 -- 172.21.15.113:0/1558791566 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9400a5990 con 0x7ff948152bd0 2024-03-04T07:36:14.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff946a00640 1 -- 172.21.15.113:0/1558791566 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400a5d60 con 0x7ff948152bd0 2024-03-04T07:36:14.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff94e000640 1 -- 172.21.15.113:0/1558791566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff948152bd0 msgr2=0x7ff9481587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:14.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff94e000640 1 --2- 172.21.15.113:0/1558791566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff948152bd0 0x7ff9481587d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff940009e50 tx=0x7ff94001ca30 comp rx=0 tx=0).stop 2024-03-04T07:36:14.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff94e000640 1 -- 172.21.15.113:0/1558791566 shutdown_connections 2024-03-04T07:36:14.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff94e000640 1 --2- 172.21.15.113:0/1558791566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff948152bd0 0x7ff9481587d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:14.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff94e000640 1 --2- 172.21.15.113:0/1558791566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9481527f0 0x7ff948070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:14.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff94e000640 1 -- 172.21.15.113:0/1558791566 >> 172.21.15.113:0/1558791566 conn(0x7ff94806e820 msgr2=0x7ff94806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:14.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff94e000640 1 -- 172.21.15.113:0/1558791566 shutdown_connections 2024-03-04T07:36:14.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff94e000640 1 -- 172.21.15.113:0/1558791566 wait complete. 2024-03-04T07:36:14.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff94e000640 1 Processor -- start 2024-03-04T07:36:14.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.187+0000 7ff94e000640 1 -- start start 2024-03-04T07:36:14.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff94e000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9480c8200 0x7ff9480b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:14.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff94e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9481527f0 0x7ff948147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:14.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff94e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff948152bd0 0x7ff948148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff94e000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9481654f0 con 0x7ff9481527f0 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff94e000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff948165370 con 0x7ff948152bd0 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff94e000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff948165670 con 0x7ff9480c8200 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff93fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9481527f0 0x7ff948147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff947400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9480c8200 0x7ff9480b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff947400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9480c8200 0x7ff9480b1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:56420/0 (socket says 172.21.15.113:56420) 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff947400640 1 -- 172.21.15.113:0/2422899648 learned_addr learned my addr 172.21.15.113:0/2422899648 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff947e00640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff948152bd0 0x7ff948148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff947400640 1 -- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff948152bd0 msgr2=0x7ff948148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff947400640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff948152bd0 0x7ff948148170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff947400640 1 -- 172.21.15.113:0/2422899648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9481527f0 msgr2=0x7ff948147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff947400640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9481527f0 0x7ff948147c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff947400640 1 -- 172.21.15.113:0/2422899648 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480726c0 con 0x7ff9480c8200 2024-03-04T07:36:14.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff93fe00640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9481527f0 0x7ff948147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:36:14.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff947400640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9480c8200 0x7ff9480b1110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff93800ec10 tx=0x7ff93800c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:14.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff93f400640 1 -- 172.21.15.113:0/2422899648 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938017070 con 0x7ff9480c8200 2024-03-04T07:36:14.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff93f400640 1 -- 172.21.15.113:0/2422899648 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93800cc30 con 0x7ff9480c8200 2024-03-04T07:36:14.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff94e000640 1 -- 172.21.15.113:0/2422899648 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9480b0cf0 con 0x7ff9480c8200 2024-03-04T07:36:14.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff93f400640 1 -- 172.21.15.113:0/2422899648 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938013470 con 0x7ff9480c8200 2024-03-04T07:36:14.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff94e000640 1 -- 172.21.15.113:0/2422899648 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff948147190 con 0x7ff9480c8200 2024-03-04T07:36:14.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.191+0000 7ff94e000640 1 -- 172.21.15.113:0/2422899648 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff948075c00 con 0x7ff9480c8200 2024-03-04T07:36:14.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.195+0000 7ff93f400640 1 -- 172.21.15.113:0/2422899648 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff93801a070 con 0x7ff9480c8200 2024-03-04T07:36:14.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.195+0000 7ff93f400640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff924066f60 0x7ff924069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:14.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.195+0000 7ff93f400640 1 -- 172.21.15.113:0/2422899648 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7ff9380857b0 con 0x7ff9480c8200 2024-03-04T07:36:14.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.195+0000 7ff93fe00640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff924066f60 0x7ff924069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:14.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.195+0000 7ff93fe00640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff924066f60 0x7ff924069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff948148dd0 tx=0x7ff930007e50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:14.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.207+0000 7ff93f400640 1 -- 172.21.15.113:0/2422899648 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff9480b0cf0 con 0x7ff9480c8200 2024-03-04T07:36:14.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.395+0000 7ff94e000640 1 -- 172.21.15.113:0/2422899648 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff948158af0 con 0x7ff9480c8200 2024-03-04T07:36:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.395+0000 7ff93f400640 1 -- 172.21.15.113:0/2422899648 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+5817 (secure 0 0 0) 0x7ff938056110 con 0x7ff9480c8200 2024-03-04T07:36:14.400 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:14.400 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:10.603476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:resolve","state_seq":37,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:36:14.400 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 15 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 -- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff924066f60 msgr2=0x7ff924069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff924066f60 0x7ff924069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff948148dd0 tx=0x7ff930007e50 comp rx=0 tx=0).stop 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 -- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9480c8200 msgr2=0x7ff9480b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9480c8200 0x7ff9480b1110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff93800ec10 tx=0x7ff93800c520 comp rx=0 tx=0).stop 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 -- 172.21.15.113:0/2422899648 shutdown_connections 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff948152bd0 0x7ff948148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff924066f60 0x7ff924069420 secure :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff948148dd0 tx=0x7ff930007e50 comp rx=0 tx=0).stop 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9481527f0 0x7ff948147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 --2- 172.21.15.113:0/2422899648 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9480c8200 0x7ff9480b1110 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 -- 172.21.15.113:0/2422899648 >> 172.21.15.113:0/2422899648 conn(0x7ff94806e820 msgr2=0x7ff9480c2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 -- 172.21.15.113:0/2422899648 shutdown_connections 2024-03-04T07:36:14.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:14.399+0000 7ff94e000640 1 -- 172.21.15.113:0/2422899648 wait complete. 2024-03-04T07:36:14.432 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:36:16.433 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:16.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7eda00640 1 Processor -- start 2024-03-04T07:36:16.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7eda00640 1 -- start start 2024-03-04T07:36:16.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7eda00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff7e81527f0 0x7ff7e8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:16.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7eda00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff7e8152bd0 0x7ff7e81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:16.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7eda00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e81591f0 con 0x7ff7e80c8200 2024-03-04T07:36:16.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7eda00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e8071df0 con 0x7ff7e8152bd0 2024-03-04T07:36:16.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7eda00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e81597b0 con 0x7ff7e81527f0 2024-03-04T07:36:16.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7e7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff7e8152bd0 0x7ff7e81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:16.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7e7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff7e8152bd0 0x7ff7e81587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52138/0 (socket says 172.21.15.113:52138) 2024-03-04T07:36:16.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7e7400640 1 -- 172.21.15.113:0/4248854593 learned_addr learned my addr 172.21.15.113:0/4248854593 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:16.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7e6a00640 1 --2- 172.21.15.113:0/4248854593 >> v2:172.21.15.136:3300/0 conn(0x7ff7e81527f0 0x7ff7e8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:16.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7e7400640 1 -- 172.21.15.113:0/4248854593 >> v2:172.21.15.136:3300/0 conn(0x7ff7e81527f0 msgr2=0x7ff7e8070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:16.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7e7400640 1 --2- 172.21.15.113:0/4248854593 >> v2:172.21.15.136:3300/0 conn(0x7ff7e81527f0 0x7ff7e8070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:16.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7e7400640 1 -- 172.21.15.113:0/4248854593 >> v1:172.21.15.136:6789/0 conn(0x7ff7e80c8200 legacy=0x7ff7e8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:16.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.567+0000 7ff7e7400640 1 -- 172.21.15.113:0/4248854593 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e81594d0 con 0x7ff7e8152bd0 2024-03-04T07:36:16.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7e7400640 1 --2- 172.21.15.113:0/4248854593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 0x7ff7e81587d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff7e806f400 tx=0x7ff7e001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d6126951511c46 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:16.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7e5600640 1 -- 172.21.15.113:0/4248854593 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00a57a0 con 0x7ff7e8152bd0 2024-03-04T07:36:16.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7e5600640 1 -- 172.21.15.113:0/4248854593 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e00a5940 con 0x7ff7e8152bd0 2024-03-04T07:36:16.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7e5600640 1 -- 172.21.15.113:0/4248854593 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00a5d10 con 0x7ff7e8152bd0 2024-03-04T07:36:16.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7eda00640 1 -- 172.21.15.113:0/4248854593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 msgr2=0x7ff7e81587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:16.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7eda00640 1 --2- 172.21.15.113:0/4248854593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 0x7ff7e81587d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff7e806f400 tx=0x7ff7e001c9f0 comp rx=0 tx=0).stop 2024-03-04T07:36:16.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7eda00640 1 -- 172.21.15.113:0/4248854593 shutdown_connections 2024-03-04T07:36:16.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7eda00640 1 --2- 172.21.15.113:0/4248854593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 0x7ff7e81587d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:16.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7eda00640 1 --2- 172.21.15.113:0/4248854593 >> v2:172.21.15.136:3300/0 conn(0x7ff7e81527f0 0x7ff7e8070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:16.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7eda00640 1 -- 172.21.15.113:0/4248854593 >> 172.21.15.113:0/4248854593 conn(0x7ff7e806e820 msgr2=0x7ff7e806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:16.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7eda00640 1 -- 172.21.15.113:0/4248854593 shutdown_connections 2024-03-04T07:36:16.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.571+0000 7ff7eda00640 1 -- 172.21.15.113:0/4248854593 wait complete. 2024-03-04T07:36:16.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.575+0000 7ff7eda00640 1 Processor -- start 2024-03-04T07:36:16.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.575+0000 7ff7eda00640 1 -- start start 2024-03-04T07:36:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.575+0000 7ff7eda00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7e80c8200 0x7ff7e814fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.575+0000 7ff7eda00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7e81527f0 0x7ff7e81502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.575+0000 7ff7eda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 0x7ff7e8157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.575+0000 7ff7eda00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e81654f0 con 0x7ff7e8152bd0 2024-03-04T07:36:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.575+0000 7ff7eda00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e8165370 con 0x7ff7e81527f0 2024-03-04T07:36:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.575+0000 7ff7eda00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e8165670 con 0x7ff7e80c8200 2024-03-04T07:36:16.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.575+0000 7ff7e7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 0x7ff7e8157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7e7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 0x7ff7e8157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52144/0 (socket says 172.21.15.113:52144) 2024-03-04T07:36:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7e7400640 1 -- 172.21.15.113:0/938011857 learned_addr learned my addr 172.21.15.113:0/938011857 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.575+0000 7ff7e6000640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7e81527f0 0x7ff7e81502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7e6a00640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7e80c8200 0x7ff7e814fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7e7400640 1 -- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7e80c8200 msgr2=0x7ff7e814fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7e7400640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7e80c8200 0x7ff7e814fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7e7400640 1 -- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7e81527f0 msgr2=0x7ff7e81502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7e7400640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7e81527f0 0x7ff7e81502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:16.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7e7400640 1 -- 172.21.15.113:0/938011857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e81f49b0 con 0x7ff7e8152bd0 2024-03-04T07:36:16.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7e7400640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 0x7ff7e8157580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00af040 tx=0x7ff7e0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:16.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7dea00640 1 -- 172.21.15.113:0/938011857 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00b8670 con 0x7ff7e8152bd0 2024-03-04T07:36:16.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7dea00640 1 -- 172.21.15.113:0/938011857 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e00be070 con 0x7ff7e8152bd0 2024-03-04T07:36:16.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7dea00640 1 -- 172.21.15.113:0/938011857 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e00b8a70 con 0x7ff7e8152bd0 2024-03-04T07:36:16.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7eda00640 1 -- 172.21.15.113:0/938011857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e81f4c90 con 0x7ff7e8152bd0 2024-03-04T07:36:16.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7eda00640 1 -- 172.21.15.113:0/938011857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e8156d80 con 0x7ff7e8152bd0 2024-03-04T07:36:16.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7eda00640 1 -- 172.21.15.113:0/938011857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e8075c00 con 0x7ff7e8152bd0 2024-03-04T07:36:16.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.579+0000 7ff7dea00640 1 -- 172.21.15.113:0/938011857 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7e00b8c10 con 0x7ff7e8152bd0 2024-03-04T07:36:16.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.583+0000 7ff7dea00640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7c8066f60 0x7ff7c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:16.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.583+0000 7ff7dea00640 1 -- 172.21.15.113:0/938011857 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7ff7e0120100 con 0x7ff7e8152bd0 2024-03-04T07:36:16.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.583+0000 7ff7e6a00640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7c8066f60 0x7ff7c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:16.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.583+0000 7ff7e6a00640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7c8066f60 0x7ff7c8069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8007a60 tx=0x7ff7d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:16.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.587+0000 7ff7dea00640 1 -- 172.21.15.113:0/938011857 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff7e00165e0 con 0x7ff7e8152bd0 2024-03-04T07:36:16.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 -- 172.21.15.113:0/938011857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7e80c79d0 con 0x7ff7e8152bd0 2024-03-04T07:36:16.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7dea00640 1 -- 172.21.15.113:0/938011857 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5819 (secure 0 0 0) 0x7ff7e00f0a10 con 0x7ff7e8152bd0 2024-03-04T07:36:16.786 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:16.786 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:16.600436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:reconnect","state_seq":39,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:36:16.786 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 16 2024-03-04T07:36:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 -- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7c8066f60 msgr2=0x7ff7c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7c8066f60 0x7ff7c8069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8007a60 tx=0x7ff7d8008040 comp rx=0 tx=0).stop 2024-03-04T07:36:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 -- 172.21.15.113:0/938011857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 msgr2=0x7ff7e8157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 0x7ff7e8157580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00af040 tx=0x7ff7e0019790 comp rx=0 tx=0).stop 2024-03-04T07:36:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 -- 172.21.15.113:0/938011857 shutdown_connections 2024-03-04T07:36:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7c8066f60 0x7ff7c8069420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7e8152bd0 0x7ff7e8157580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7e81527f0 0x7ff7e81502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 --2- 172.21.15.113:0/938011857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7e80c8200 0x7ff7e814fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:16.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 -- 172.21.15.113:0/938011857 >> 172.21.15.113:0/938011857 conn(0x7ff7e806e820 msgr2=0x7ff7e80c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:16.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 -- 172.21.15.113:0/938011857 shutdown_connections 2024-03-04T07:36:16.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:16.783+0000 7ff7eda00640 1 -- 172.21.15.113:0/938011857 wait complete. 2024-03-04T07:36:16.806 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:36:18.807 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:18.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.943+0000 7f5d8ca00640 1 Processor -- start 2024-03-04T07:36:18.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.943+0000 7f5d8ca00640 1 -- start start 2024-03-04T07:36:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d8ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d88152a50 0x7f5d880ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d8ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806ff20 0x7f5d880ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d8ca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d88152e30 con 0x7f5d880c8b90 2024-03-04T07:36:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d8ca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d880c31b0 con 0x7f5d88152a50 2024-03-04T07:36:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d8ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8806f840 con 0x7f5d8806ff20 2024-03-04T07:36:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d85e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d88152a50 0x7f5d880ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d85e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d88152a50 0x7f5d880ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59900/0 (socket says 172.21.15.113:59900) 2024-03-04T07:36:18.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d85e00640 1 -- 172.21.15.113:0/1612386612 learned_addr learned my addr 172.21.15.113:0/1612386612 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:18.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d85400640 1 --2- 172.21.15.113:0/1612386612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806ff20 0x7f5d880ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:18.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d85e00640 1 -- 172.21.15.113:0/1612386612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806ff20 msgr2=0x7f5d880ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:18.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d85e00640 1 --2- 172.21.15.113:0/1612386612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806ff20 0x7f5d880ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:18.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d85e00640 1 -- 172.21.15.113:0/1612386612 >> v1:172.21.15.136:6789/0 conn(0x7f5d880c8b90 legacy=0x7f5d8815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:18.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.947+0000 7f5d85e00640 1 -- 172.21.15.113:0/1612386612 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8806f280 con 0x7f5d88152a50 2024-03-04T07:36:18.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d85e00640 1 --2- 172.21.15.113:0/1612386612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 0x7f5d880ce120 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009870 tx=0x7f5d7801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a469fa53c1c32f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:18.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d84a00640 1 -- 172.21.15.113:0/1612386612 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d780aa430 con 0x7f5d88152a50 2024-03-04T07:36:18.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d84a00640 1 -- 172.21.15.113:0/1612386612 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d780af070 con 0x7f5d88152a50 2024-03-04T07:36:18.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d84a00640 1 -- 172.21.15.113:0/1612386612 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d780aa7f0 con 0x7f5d88152a50 2024-03-04T07:36:18.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d8ca00640 1 -- 172.21.15.113:0/1612386612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 msgr2=0x7f5d880ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:18.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d8ca00640 1 --2- 172.21.15.113:0/1612386612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 0x7f5d880ce120 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009870 tx=0x7f5d7801c4d0 comp rx=0 tx=0).stop 2024-03-04T07:36:18.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d8ca00640 1 -- 172.21.15.113:0/1612386612 shutdown_connections 2024-03-04T07:36:18.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d8ca00640 1 --2- 172.21.15.113:0/1612386612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d8806ff20 0x7f5d880ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:18.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d8ca00640 1 --2- 172.21.15.113:0/1612386612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 0x7f5d880ce120 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:18.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d8ca00640 1 -- 172.21.15.113:0/1612386612 >> 172.21.15.113:0/1612386612 conn(0x7f5d880734d0 msgr2=0x7f5d88066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:18.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d8ca00640 1 -- 172.21.15.113:0/1612386612 shutdown_connections 2024-03-04T07:36:18.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d8ca00640 1 -- 172.21.15.113:0/1612386612 wait complete. 2024-03-04T07:36:18.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d8ca00640 1 Processor -- start 2024-03-04T07:36:18.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.951+0000 7f5d8ca00640 1 -- start start 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d8ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d8806ff20 0x7f5d880ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d8ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d880c8b90 0x7f5d88156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d8ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 0x7f5d881f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d8ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d88165a00 con 0x7f5d88152a50 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d8ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d88165880 con 0x7f5d8806ff20 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d8ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d88165b80 con 0x7f5d880c8b90 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d86800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 0x7f5d881f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d86800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 0x7f5d881f3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59906/0 (socket says 172.21.15.113:59906) 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d86800640 1 -- 172.21.15.113:0/4191903940 learned_addr learned my addr 172.21.15.113:0/4191903940 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d85400640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d880c8b90 0x7f5d88156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:18.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d85e00640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d8806ff20 0x7f5d880ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:18.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d86800640 1 -- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d880c8b90 msgr2=0x7f5d88156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:18.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d86800640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d880c8b90 0x7f5d88156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:18.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d86800640 1 -- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d8806ff20 msgr2=0x7f5d880ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:18.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d86800640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d8806ff20 0x7f5d880ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:18.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d86800640 1 -- 172.21.15.113:0/4191903940 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d88071730 con 0x7f5d88152a50 2024-03-04T07:36:18.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d86800640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 0x7f5d881f3ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000c7a0 tx=0x7f5d8000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:18.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d7e000640 1 -- 172.21.15.113:0/4191903940 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80003b80 con 0x7f5d88152a50 2024-03-04T07:36:18.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d7e000640 1 -- 172.21.15.113:0/4191903940 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d80003d20 con 0x7f5d88152a50 2024-03-04T07:36:18.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d8ca00640 1 -- 172.21.15.113:0/4191903940 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d881585d0 con 0x7f5d88152a50 2024-03-04T07:36:18.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d7e000640 1 -- 172.21.15.113:0/4191903940 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8000aa90 con 0x7f5d88152a50 2024-03-04T07:36:18.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d8ca00640 1 -- 172.21.15.113:0/4191903940 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d88156e70 con 0x7f5d88152a50 2024-03-04T07:36:18.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d8ca00640 1 -- 172.21.15.113:0/4191903940 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d88075c00 con 0x7f5d88152a50 2024-03-04T07:36:18.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.955+0000 7f5d7e000640 1 -- 172.21.15.113:0/4191903940 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d8000ac30 con 0x7f5d88152a50 2024-03-04T07:36:18.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.959+0000 7f5d7e000640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5d68066f90 0x7f5d68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:18.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.959+0000 7f5d7e000640 1 -- 172.21.15.113:0/4191903940 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f5d80084fc0 con 0x7f5d88152a50 2024-03-04T07:36:18.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.959+0000 7f5d85e00640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5d68066f90 0x7f5d68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:18.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.959+0000 7f5d85e00640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5d68066f90 0x7f5d68069450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5d780ae000 tx=0x7f5d780a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:18.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:18.963+0000 7f5d7e000640 1 -- 172.21.15.113:0/4191903940 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5d80055810 con 0x7f5d88152a50 2024-03-04T07:36:19.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 -- 172.21.15.113:0/4191903940 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d880cc310 con 0x7f5d88152a50 2024-03-04T07:36:19.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d7e000640 1 -- 172.21.15.113:0/4191903940 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f5d800598b0 con 0x7f5d88152a50 2024-03-04T07:36:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:19.229 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:16.600436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:reconnect","state_seq":39,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:36:19.230 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 16 2024-03-04T07:36:19.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 -- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5d68066f90 msgr2=0x7f5d68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:19.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5d68066f90 0x7f5d68069450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5d780ae000 tx=0x7f5d780a7000 comp rx=0 tx=0).stop 2024-03-04T07:36:19.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 -- 172.21.15.113:0/4191903940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 msgr2=0x7f5d881f3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:19.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 0x7f5d881f3ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000c7a0 tx=0x7f5d8000cc60 comp rx=0 tx=0).stop 2024-03-04T07:36:19.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 -- 172.21.15.113:0/4191903940 shutdown_connections 2024-03-04T07:36:19.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5d68066f90 0x7f5d68069450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:19.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d88152a50 0x7f5d881f3ed0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:19.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d880c8b90 0x7f5d88156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:19.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 --2- 172.21.15.113:0/4191903940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d8806ff20 0x7f5d880ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:19.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 -- 172.21.15.113:0/4191903940 >> 172.21.15.113:0/4191903940 conn(0x7f5d880734d0 msgr2=0x7f5d880ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:19.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 -- 172.21.15.113:0/4191903940 shutdown_connections 2024-03-04T07:36:19.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:19.227+0000 7f5d8ca00640 1 -- 172.21.15.113:0/4191903940 wait complete. 2024-03-04T07:36:19.251 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 16, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 16, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4250": { "addr": "172.21.15.113:6833/3005395426", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 3005395426, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 3005395426, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4250, "incarnation": 14, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:reconnect", "state_seq": 39 }, "gid_4256": { "addr": "172.21.15.113:6837/1310042629", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1310042629, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1310042629, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4256, "incarnation": 10, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4276": { "addr": "172.21.15.136:6835/3348592059", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3348592059, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3348592059, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4276, "incarnation": 4, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 5 } }, "last_failure": 0, "last_failure_osd_epoch": 24, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:36:16.600436+0000", "qdb_cluster": [ 4256, 4276 ], "qdb_leader": 4276, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4276, "mds_1": 4256, "mds_2": 4250 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:36:21.252 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:21.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.383+0000 7fa556e00640 1 Processor -- start 2024-03-04T07:36:21.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.383+0000 7fa556e00640 1 -- start start 2024-03-04T07:36:21.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa556e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa550158010 0x7fa55006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:21.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa556e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa55006ff20 0x7fa55015b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:21.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa556e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5500ca0e0 con 0x7fa55015b750 2024-03-04T07:36:21.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa556e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55006f450 con 0x7fa55006ff20 2024-03-04T07:36:21.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa556e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55015ad80 con 0x7fa550158010 2024-03-04T07:36:21.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa547e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa55006ff20 0x7fa55015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:21.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa547e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa55006ff20 0x7fa55015b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59920/0 (socket says 172.21.15.113:59920) 2024-03-04T07:36:21.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa547e00640 1 -- 172.21.15.113:0/1068732413 learned_addr learned my addr 172.21.15.113:0/1068732413 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:21.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa54fe00640 1 --2- 172.21.15.113:0/1068732413 >> v2:172.21.15.136:3300/0 conn(0x7fa550158010 0x7fa55006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:21.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa547e00640 1 -- 172.21.15.113:0/1068732413 >> v2:172.21.15.136:3300/0 conn(0x7fa550158010 msgr2=0x7fa55006dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:21.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa547e00640 1 --2- 172.21.15.113:0/1068732413 >> v2:172.21.15.136:3300/0 conn(0x7fa550158010 0x7fa55006dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:21.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa547e00640 1 -- 172.21.15.113:0/1068732413 >> v1:172.21.15.136:6789/0 conn(0x7fa55015b750 legacy=0x7fa5500ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:21.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.387+0000 7fa547e00640 1 -- 172.21.15.113:0/1068732413 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55015aa30 con 0x7fa55006ff20 2024-03-04T07:36:21.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa547e00640 1 --2- 172.21.15.113:0/1068732413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55006ff20 0x7fa55015b210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa55006d140 tx=0x7fa53801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe0c2ff5ac1d0c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:21.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa54f400640 1 -- 172.21.15.113:0/1068732413 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5380b0070 con 0x7fa55006ff20 2024-03-04T07:36:21.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa54f400640 1 -- 172.21.15.113:0/1068732413 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5380ab600 con 0x7fa55006ff20 2024-03-04T07:36:21.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa54f400640 1 -- 172.21.15.113:0/1068732413 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5380ab9d0 con 0x7fa55006ff20 2024-03-04T07:36:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa556e00640 1 -- 172.21.15.113:0/1068732413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55006ff20 msgr2=0x7fa55015b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa556e00640 1 --2- 172.21.15.113:0/1068732413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55006ff20 0x7fa55015b210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa55006d140 tx=0x7fa53801c520 comp rx=0 tx=0).stop 2024-03-04T07:36:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa556e00640 1 -- 172.21.15.113:0/1068732413 shutdown_connections 2024-03-04T07:36:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa556e00640 1 --2- 172.21.15.113:0/1068732413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55006ff20 0x7fa55015b210 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa556e00640 1 --2- 172.21.15.113:0/1068732413 >> v2:172.21.15.136:3300/0 conn(0x7fa550158010 0x7fa55006dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa556e00640 1 -- 172.21.15.113:0/1068732413 >> 172.21.15.113:0/1068732413 conn(0x7fa5500734d0 msgr2=0x7fa550066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa556e00640 1 -- 172.21.15.113:0/1068732413 shutdown_connections 2024-03-04T07:36:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa556e00640 1 -- 172.21.15.113:0/1068732413 wait complete. 2024-03-04T07:36:21.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa556e00640 1 Processor -- start 2024-03-04T07:36:21.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.391+0000 7fa556e00640 1 -- start start 2024-03-04T07:36:21.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa556e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa55006ff20 0x7fa55014c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:21.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa556e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa550158010 0x7fa55014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:21.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa556e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55015b750 0x7fa55014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:21.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa556e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55006c170 con 0x7fa55015b750 2024-03-04T07:36:21.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa556e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5501583f0 con 0x7fa550158010 2024-03-04T07:36:21.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa556e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55006c2f0 con 0x7fa55006ff20 2024-03-04T07:36:21.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa554e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55015b750 0x7fa55014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:21.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa547e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa550158010 0x7fa55014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:21.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa54fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa55006ff20 0x7fa55014c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:21.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa554e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55015b750 0x7fa55014ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59936/0 (socket says 172.21.15.113:59936) 2024-03-04T07:36:21.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa554e00640 1 -- 172.21.15.113:0/997919373 learned_addr learned my addr 172.21.15.113:0/997919373 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:21.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa554e00640 1 -- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa55006ff20 msgr2=0x7fa55014c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:21.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa554e00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa55006ff20 0x7fa55014c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:21.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa554e00640 1 -- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa550158010 msgr2=0x7fa55014cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:21.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa554e00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa550158010 0x7fa55014cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:21.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa554e00640 1 -- 172.21.15.113:0/997919373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5501f78a0 con 0x7fa55015b750 2024-03-04T07:36:21.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa554e00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55015b750 0x7fa55014ab60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa54800c730 tx=0x7fa54800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:21.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa54cc00640 1 -- 172.21.15.113:0/997919373 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548003b80 con 0x7fa55015b750 2024-03-04T07:36:21.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa54cc00640 1 -- 172.21.15.113:0/997919373 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa548003d20 con 0x7fa55015b750 2024-03-04T07:36:21.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa54cc00640 1 -- 172.21.15.113:0/997919373 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548005310 con 0x7fa55015b750 2024-03-04T07:36:21.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa556e00640 1 -- 172.21.15.113:0/997919373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5501f6b20 con 0x7fa55015b750 2024-03-04T07:36:21.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa556e00640 1 -- 172.21.15.113:0/997919373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55014b480 con 0x7fa55015b750 2024-03-04T07:36:21.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa556e00640 1 -- 172.21.15.113:0/997919373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa550075c00 con 0x7fa55015b750 2024-03-04T07:36:21.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.395+0000 7fa54cc00640 1 -- 172.21.15.113:0/997919373 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5480056c0 con 0x7fa55015b750 2024-03-04T07:36:21.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.399+0000 7fa54cc00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 0x7fa52c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:21.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.399+0000 7fa54cc00640 1 -- 172.21.15.113:0/997919373 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fa5480850d0 con 0x7fa55015b750 2024-03-04T07:36:21.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.399+0000 7fa54fe00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 0x7fa52c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:21.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.399+0000 7fa54fe00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 0x7fa52c069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa540009b60 tx=0x7fa540009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:21.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.407+0000 7fa54cc00640 1 -- 172.21.15.113:0/997919373 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa548055890 con 0x7fa55015b750 2024-03-04T07:36:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.595+0000 7fa556e00640 1 -- 172.21.15.113:0/997919373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5500cdeb0 con 0x7fa55015b750 2024-03-04T07:36:21.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.595+0000 7fa54cc00640 1 -- 172.21.15.113:0/997919373 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5819 (secure 0 0 0) 0x7fa548059930 con 0x7fa55015b750 2024-03-04T07:36:21.600 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:21.600 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:16.600436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:reconnect","state_seq":39,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:36:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 16 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 -- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 msgr2=0x7fa52c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 0x7fa52c069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa540009b60 tx=0x7fa540009190 comp rx=0 tx=0).stop 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 -- 172.21.15.113:0/997919373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55015b750 msgr2=0x7fa55014ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55015b750 0x7fa55014ab60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa54800c730 tx=0x7fa54800cbf0 comp rx=0 tx=0).stop 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 -- 172.21.15.113:0/997919373 shutdown_connections 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 0x7fa52c069420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa55015b750 0x7fa55014ab60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa550158010 0x7fa55014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 --2- 172.21.15.113:0/997919373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa55006ff20 0x7fa55014c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 -- 172.21.15.113:0/997919373 >> 172.21.15.113:0/997919373 conn(0x7fa5500734d0 msgr2=0x7fa5500c8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 -- 172.21.15.113:0/997919373 shutdown_connections 2024-03-04T07:36:21.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:21.599+0000 7fa556e00640 1 -- 172.21.15.113:0/997919373 wait complete. 2024-03-04T07:36:21.622 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:36:23.623 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:23.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.763+0000 7f5114a00640 1 Processor -- start 2024-03-04T07:36:23.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.763+0000 7f5114a00640 1 -- start start 2024-03-04T07:36:23.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.763+0000 7f5114a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f511006b060 0x7f51100c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:23.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.763+0000 7f5114a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51100b2230 con 0x7f51100733f0 2024-03-04T07:36:23.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.763+0000 7f5114a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51100c7bb0 con 0x7f5110158210 2024-03-04T07:36:23.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.763+0000 7f5114a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5110072fb0 con 0x7f511006b060 2024-03-04T07:36:23.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.763+0000 7f5107e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f51100733f0 0x7f51100ca400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50102/0 (socket says 172.21.15.113:50102) 2024-03-04T07:36:23.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.763+0000 7f510e800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f511006b060 0x7f51100c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:23.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.763+0000 7f5107e00640 1 -- 172.21.15.113:0/4292047116 learned_addr learned my addr 172.21.15.113:0/4292047116 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:36:23.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.767+0000 7f510d400640 1 -- 172.21.15.113:0/4292047116 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588296588 0 0) 0x7f51100b2230 con 0x7f51100733f0 2024-03-04T07:36:23.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.767+0000 7f510d400640 1 -- 172.21.15.113:0/4292047116 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50f4003530 con 0x7f51100733f0 2024-03-04T07:36:23.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.767+0000 7f510e800640 1 -- 172.21.15.113:0/4292047116 >> v1:172.21.15.136:6789/0 conn(0x7f5110158210 legacy=0x7f51100cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:23.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.767+0000 7f510e800640 1 -- 172.21.15.113:0/4292047116 >> v1:172.21.15.113:6789/0 conn(0x7f51100733f0 legacy=0x7f51100ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:23.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.767+0000 7f510e800640 1 -- 172.21.15.113:0/4292047116 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51100c7e50 con 0x7f511006b060 2024-03-04T07:36:23.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.767+0000 7f510e800640 1 --2- 172.21.15.113:0/4292047116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f511006b060 0x7f51100c8bb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5110073130 tx=0x7f510801c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aba8d7ff056268a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:23.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.767+0000 7f510d400640 1 -- 172.21.15.113:0/4292047116 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080b2070 con 0x7f511006b060 2024-03-04T07:36:23.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.767+0000 7f510d400640 1 -- 172.21.15.113:0/4292047116 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51080a5870 con 0x7f511006b060 2024-03-04T07:36:23.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.767+0000 7f510d400640 1 -- 172.21.15.113:0/4292047116 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080a5c40 con 0x7f511006b060 2024-03-04T07:36:23.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.771+0000 7f5114a00640 1 -- 172.21.15.113:0/4292047116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f511006b060 msgr2=0x7f51100c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.771+0000 7f5114a00640 1 --2- 172.21.15.113:0/4292047116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f511006b060 0x7f51100c8bb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5110073130 tx=0x7f510801c8c0 comp rx=0 tx=0).stop 2024-03-04T07:36:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.771+0000 7f5114a00640 1 -- 172.21.15.113:0/4292047116 shutdown_connections 2024-03-04T07:36:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.771+0000 7f5114a00640 1 --2- 172.21.15.113:0/4292047116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f511006b060 0x7f51100c8bb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.771+0000 7f5114a00640 1 -- 172.21.15.113:0/4292047116 >> 172.21.15.113:0/4292047116 conn(0x7f51100671b0 msgr2=0x7f511006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.771+0000 7f5114a00640 1 -- 172.21.15.113:0/4292047116 shutdown_connections 2024-03-04T07:36:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.771+0000 7f5114a00640 1 -- 172.21.15.113:0/4292047116 wait complete. 2024-03-04T07:36:23.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.771+0000 7f5114a00640 1 Processor -- start 2024-03-04T07:36:23.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.771+0000 7f5114a00640 1 -- start start 2024-03-04T07:36:23.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5114a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f511006b060 0x7f51100ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5114a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51100733f0 0x7f51101f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5114a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5110158210 0x7f5110142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5114a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51100cd8a0 con 0x7f511006b060 2024-03-04T07:36:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5114a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51100cd720 con 0x7f51100733f0 2024-03-04T07:36:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5114a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511006ce70 con 0x7f5110158210 2024-03-04T07:36:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f510de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f511006b060 0x7f51100ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f510de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f511006b060 0x7f51100ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59948/0 (socket says 172.21.15.113:59948) 2024-03-04T07:36:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f510de00640 1 -- 172.21.15.113:0/1968384752 learned_addr learned my addr 172.21.15.113:0/1968384752 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f510e800640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5110158210 0x7f5110142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5107e00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51100733f0 0x7f51101f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:23.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f510de00640 1 -- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5110158210 msgr2=0x7f5110142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f510de00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5110158210 0x7f5110142d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f510de00640 1 -- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51100733f0 msgr2=0x7f51101f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f510de00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51100733f0 0x7f51101f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f510de00640 1 -- 172.21.15.113:0/1968384752 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51100cc8c0 con 0x7f511006b060 2024-03-04T07:36:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f510de00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f511006b060 0x7f51100ccf70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f510000c450 tx=0x7f510000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5106000640 1 -- 172.21.15.113:0/1968384752 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100016020 con 0x7f511006b060 2024-03-04T07:36:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5106000640 1 -- 172.21.15.113:0/1968384752 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51000052d0 con 0x7f511006b060 2024-03-04T07:36:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5106000640 1 -- 172.21.15.113:0/1968384752 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100007260 con 0x7f511006b060 2024-03-04T07:36:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5114a00640 1 -- 172.21.15.113:0/1968384752 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51101467a0 con 0x7f511006b060 2024-03-04T07:36:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5114a00640 1 -- 172.21.15.113:0/1968384752 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51100b1bb0 con 0x7f511006b060 2024-03-04T07:36:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.775+0000 7f5114a00640 1 -- 172.21.15.113:0/1968384752 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5110075c00 con 0x7f511006b060 2024-03-04T07:36:23.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.779+0000 7f5106000640 1 -- 172.21.15.113:0/1968384752 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5100007400 con 0x7f511006b060 2024-03-04T07:36:23.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.779+0000 7f5106000640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50ec066f60 0x7f50ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:23.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.779+0000 7f5106000640 1 -- 172.21.15.113:0/1968384752 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f5100084940 con 0x7f511006b060 2024-03-04T07:36:23.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.779+0000 7f5107e00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50ec066f60 0x7f50ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.779+0000 7f5107e00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50ec066f60 0x7f50ec069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5110141a70 tx=0x7f50f8007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:23.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.787+0000 7f5106000640 1 -- 172.21.15.113:0/1968384752 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5100020020 con 0x7f511006b060 2024-03-04T07:36:23.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.995+0000 7f5114a00640 1 -- 172.21.15.113:0/1968384752 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f511014aac0 con 0x7f511006b060 2024-03-04T07:36:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.995+0000 7f5106000640 1 -- 172.21.15.113:0/1968384752 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f5100005480 con 0x7f511006b060 2024-03-04T07:36:24.000 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:24.000 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:21.612877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:rejoin","state_seq":41,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:36:24.001 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 17 2024-03-04T07:36:24.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 -- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50ec066f60 msgr2=0x7f50ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:24.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50ec066f60 0x7f50ec069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5110141a70 tx=0x7f50f8007d40 comp rx=0 tx=0).stop 2024-03-04T07:36:24.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 -- 172.21.15.113:0/1968384752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f511006b060 msgr2=0x7f51100ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:24.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f511006b060 0x7f51100ccf70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f510000c450 tx=0x7f510000c910 comp rx=0 tx=0).stop 2024-03-04T07:36:24.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 -- 172.21.15.113:0/1968384752 shutdown_connections 2024-03-04T07:36:24.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5110158210 0x7f5110142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:24.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51100733f0 0x7f51101f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:24.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50ec066f60 0x7f50ec069420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:24.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 --2- 172.21.15.113:0/1968384752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f511006b060 0x7f51100ccf70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:24.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 -- 172.21.15.113:0/1968384752 >> 172.21.15.113:0/1968384752 conn(0x7f51100671b0 msgr2=0x7f5110143be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:24.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 -- 172.21.15.113:0/1968384752 shutdown_connections 2024-03-04T07:36:24.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:23.999+0000 7f5114a00640 1 -- 172.21.15.113:0/1968384752 wait complete. 2024-03-04T07:36:24.021 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:36:24.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:36:24.306 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:36:26.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:26.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d65a00640 1 Processor -- start 2024-03-04T07:36:26.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d65a00640 1 -- start start 2024-03-04T07:36:26.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d65a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0d601527f0 0x7f0d60070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:26.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d65a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d60152bd0 0x7f0d601587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:26.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d65a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d601591f0 con 0x7f0d600c8200 2024-03-04T07:36:26.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d65a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d60071df0 con 0x7f0d60152bd0 2024-03-04T07:36:26.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d65a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d601597b0 con 0x7f0d601527f0 2024-03-04T07:36:26.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d5f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d60152bd0 0x7f0d601587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:26.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d5f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d60152bd0 0x7f0d601587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59952/0 (socket says 172.21.15.113:59952) 2024-03-04T07:36:26.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d5f400640 1 -- 172.21.15.113:0/331862808 learned_addr learned my addr 172.21.15.113:0/331862808 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:26.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.155+0000 7f0d5ea00640 1 --2- 172.21.15.113:0/331862808 >> v2:172.21.15.136:3300/0 conn(0x7f0d601527f0 0x7f0d60070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:26.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.159+0000 7f0d5f400640 1 -- 172.21.15.113:0/331862808 >> v2:172.21.15.136:3300/0 conn(0x7f0d601527f0 msgr2=0x7f0d60070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:26.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.159+0000 7f0d5f400640 1 --2- 172.21.15.113:0/331862808 >> v2:172.21.15.136:3300/0 conn(0x7f0d601527f0 0x7f0d60070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.159+0000 7f0d5f400640 1 -- 172.21.15.113:0/331862808 >> v1:172.21.15.136:6789/0 conn(0x7f0d600c8200 legacy=0x7f0d60072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.159+0000 7f0d5f400640 1 -- 172.21.15.113:0/331862808 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d601594d0 con 0x7f0d60152bd0 2024-03-04T07:36:26.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d5f400640 1 --2- 172.21.15.113:0/331862808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d60152bd0 0x7f0d601587d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0d6006f400 tx=0x7f0d5801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a75b983c3f84414 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:26.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d5e000640 1 -- 172.21.15.113:0/331862808 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d580a57a0 con 0x7f0d60152bd0 2024-03-04T07:36:26.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d5e000640 1 -- 172.21.15.113:0/331862808 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d580a5940 con 0x7f0d60152bd0 2024-03-04T07:36:26.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d5e000640 1 -- 172.21.15.113:0/331862808 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d580a5d10 con 0x7f0d60152bd0 2024-03-04T07:36:26.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d65a00640 1 -- 172.21.15.113:0/331862808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d60152bd0 msgr2=0x7f0d601587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:26.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d65a00640 1 --2- 172.21.15.113:0/331862808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d60152bd0 0x7f0d601587d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0d6006f400 tx=0x7f0d5801c9f0 comp rx=0 tx=0).stop 2024-03-04T07:36:26.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d65a00640 1 -- 172.21.15.113:0/331862808 shutdown_connections 2024-03-04T07:36:26.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d65a00640 1 --2- 172.21.15.113:0/331862808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d60152bd0 0x7f0d601587d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:26.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d65a00640 1 --2- 172.21.15.113:0/331862808 >> v2:172.21.15.136:3300/0 conn(0x7f0d601527f0 0x7f0d60070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:26.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d65a00640 1 -- 172.21.15.113:0/331862808 >> 172.21.15.113:0/331862808 conn(0x7f0d6006e820 msgr2=0x7f0d6006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:26.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d65a00640 1 -- 172.21.15.113:0/331862808 shutdown_connections 2024-03-04T07:36:26.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d65a00640 1 -- 172.21.15.113:0/331862808 wait complete. 2024-03-04T07:36:26.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d65a00640 1 Processor -- start 2024-03-04T07:36:26.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.163+0000 7f0d65a00640 1 -- start start 2024-03-04T07:36:26.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d65a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d600c8200 0x7f0d60150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:26.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d65a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d601527f0 0x7f0d60150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:26.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d65a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d60152bd0 0x7f0d60148970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d65a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d601654f0 con 0x7f0d601527f0 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d65a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d60165370 con 0x7f0d60152bd0 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d65a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d60165670 con 0x7f0d600c8200 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d57e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d601527f0 0x7f0d60150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d57e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d601527f0 0x7f0d60150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59964/0 (socket says 172.21.15.113:59964) 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d57e00640 1 -- 172.21.15.113:0/971044454 learned_addr learned my addr 172.21.15.113:0/971044454 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d5ea00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d600c8200 0x7f0d60150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d5f400640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d60152bd0 0x7f0d60148970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d57e00640 1 -- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d600c8200 msgr2=0x7f0d60150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d57e00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d600c8200 0x7f0d60150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d57e00640 1 -- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d60152bd0 msgr2=0x7f0d60148970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d57e00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d60152bd0 0x7f0d60148970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d57e00640 1 -- 172.21.15.113:0/971044454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d600c2140 con 0x7f0d601527f0 2024-03-04T07:36:26.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d5f400640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d60152bd0 0x7f0d60148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:36:26.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d57e00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d601527f0 0x7f0d60150d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800c460 tx=0x7f0d4800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:26.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d56a00640 1 -- 172.21.15.113:0/971044454 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48016020 con 0x7f0d601527f0 2024-03-04T07:36:26.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d56a00640 1 -- 172.21.15.113:0/971044454 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d480052d0 con 0x7f0d601527f0 2024-03-04T07:36:26.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d56a00640 1 -- 172.21.15.113:0/971044454 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d48007260 con 0x7f0d601527f0 2024-03-04T07:36:26.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d65a00640 1 -- 172.21.15.113:0/971044454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d600c2420 con 0x7f0d601527f0 2024-03-04T07:36:26.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d65a00640 1 -- 172.21.15.113:0/971044454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d600c29e0 con 0x7f0d601527f0 2024-03-04T07:36:26.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d65a00640 1 -- 172.21.15.113:0/971044454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d60075c00 con 0x7f0d601527f0 2024-03-04T07:36:26.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.167+0000 7f0d56a00640 1 -- 172.21.15.113:0/971044454 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d48007400 con 0x7f0d601527f0 2024-03-04T07:36:26.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.171+0000 7f0d56a00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d3c066f60 0x7f0d3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:26.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.171+0000 7f0d56a00640 1 -- 172.21.15.113:0/971044454 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f0d48084a80 con 0x7f0d601527f0 2024-03-04T07:36:26.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.171+0000 7f0d5ea00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d3c066f60 0x7f0d3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:26.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.171+0000 7f0d5ea00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d3c066f60 0x7f0d3c069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009b60 tx=0x7f0d50009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:26.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.175+0000 7f0d56a00640 1 -- 172.21.15.113:0/971044454 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0d48086e20 con 0x7f0d601527f0 2024-03-04T07:36:26.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.419+0000 7f0d65a00640 1 -- 172.21.15.113:0/971044454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d600c79d0 con 0x7f0d601527f0 2024-03-04T07:36:26.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.419+0000 7f0d56a00640 1 -- 172.21.15.113:0/971044454 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f0d48020020 con 0x7f0d601527f0 2024-03-04T07:36:26.423 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:26.423 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:21.612877+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:rejoin","state_seq":41,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:36:26.423 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 17 2024-03-04T07:36:26.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 -- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d3c066f60 msgr2=0x7f0d3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:26.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d3c066f60 0x7f0d3c069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009b60 tx=0x7f0d50009190 comp rx=0 tx=0).stop 2024-03-04T07:36:26.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 -- 172.21.15.113:0/971044454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d601527f0 msgr2=0x7f0d60150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:26.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d601527f0 0x7f0d60150d40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800c460 tx=0x7f0d4800c920 comp rx=0 tx=0).stop 2024-03-04T07:36:26.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 -- 172.21.15.113:0/971044454 shutdown_connections 2024-03-04T07:36:26.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d60152bd0 0x7f0d60148970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:26.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d3c066f60 0x7f0d3c069420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:26.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d601527f0 0x7f0d60150d40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:26.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 --2- 172.21.15.113:0/971044454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d600c8200 0x7f0d60150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:26.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 -- 172.21.15.113:0/971044454 >> 172.21.15.113:0/971044454 conn(0x7f0d6006e820 msgr2=0x7f0d6006ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:26.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 -- 172.21.15.113:0/971044454 shutdown_connections 2024-03-04T07:36:26.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:26.423+0000 7f0d65a00640 1 -- 172.21.15.113:0/971044454 wait complete. 2024-03-04T07:36:26.445 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:36:26.639 INFO:tasks.workunit.client.0.smithi113.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2024-03-04T07:36:28.446 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:28.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.587+0000 7fc21b600640 1 Processor -- start 2024-03-04T07:36:28.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.587+0000 7fc21b600640 1 -- start start 2024-03-04T07:36:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc21b600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 0x7fc214072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc21b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc2140731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc21b600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2140c27f0 con 0x7fc2140c2d80 2024-03-04T07:36:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc21b600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21415ad60 con 0x7fc21414e4c0 2024-03-04T07:36:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc21b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140c2a90 con 0x7fc2140c22e0 2024-03-04T07:36:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc219600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc2140c2d80 0x7fc21415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33890/0 (socket says 172.21.15.113:33890) 2024-03-04T07:36:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc219600640 1 -- 172.21.15.113:0/699484902 learned_addr learned my addr 172.21.15.113:0/699484902 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:36:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc213e00640 1 --2- 172.21.15.113:0/699484902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc2140731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc218c00640 1 --2- 172.21.15.113:0/699484902 >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 0x7fc214072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc213400640 1 -- 172.21.15.113:0/699484902 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901120519 0 0) 0x7fc2140c27f0 con 0x7fc2140c2d80 2024-03-04T07:36:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc213400640 1 -- 172.21.15.113:0/699484902 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1fc005740 con 0x7fc2140c2d80 2024-03-04T07:36:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc213e00640 1 -- 172.21.15.113:0/699484902 >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 msgr2=0x7fc214072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc213e00640 1 --2- 172.21.15.113:0/699484902 >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 0x7fc214072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc213e00640 1 -- 172.21.15.113:0/699484902 >> v1:172.21.15.113:6789/0 conn(0x7fc2140c2d80 legacy=0x7fc21415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.591+0000 7fc213e00640 1 -- 172.21.15.113:0/699484902 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214070c60 con 0x7fc2140c22e0 2024-03-04T07:36:28.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc213e00640 1 --2- 172.21.15.113:0/699484902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc2140731c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc2000097c0 tx=0x7fc20001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=177c40035eaeb28 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc213400640 1 -- 172.21.15.113:0/699484902 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000b5070 con 0x7fc2140c22e0 2024-03-04T07:36:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc213400640 1 -- 172.21.15.113:0/699484902 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2000a8990 con 0x7fc2140c22e0 2024-03-04T07:36:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc213400640 1 -- 172.21.15.113:0/699484902 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000a8d60 con 0x7fc2140c22e0 2024-03-04T07:36:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc21b600640 1 -- 172.21.15.113:0/699484902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 msgr2=0x7fc2140731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc21b600640 1 --2- 172.21.15.113:0/699484902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc2140731c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc2000097c0 tx=0x7fc20001ec90 comp rx=0 tx=0).stop 2024-03-04T07:36:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc21b600640 1 -- 172.21.15.113:0/699484902 shutdown_connections 2024-03-04T07:36:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc21b600640 1 --2- 172.21.15.113:0/699484902 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc2140731c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc21b600640 1 --2- 172.21.15.113:0/699484902 >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 0x7fc214072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc21b600640 1 -- 172.21.15.113:0/699484902 >> 172.21.15.113:0/699484902 conn(0x7fc21406e820 msgr2=0x7fc21406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc21b600640 1 -- 172.21.15.113:0/699484902 shutdown_connections 2024-03-04T07:36:28.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc21b600640 1 -- 172.21.15.113:0/699484902 wait complete. 2024-03-04T07:36:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc21b600640 1 Processor -- start 2024-03-04T07:36:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.595+0000 7fc21b600640 1 -- start start 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc21b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc214142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc21b600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c2d80 0x7fc2141536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc21b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc21b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc214071500 con 0x7fc21414e4c0 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc21b600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc214071220 con 0x7fc2140c2d80 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc21b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21414e8a0 con 0x7fc2140c22e0 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc219600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc219600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39440/0 (socket says 172.21.15.113:39440) 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc219600640 1 -- 172.21.15.113:0/2652272447 learned_addr learned my addr 172.21.15.113:0/2652272447 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc213e00640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c2d80 0x7fc2141536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc218c00640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc214142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:28.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc219600640 1 -- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 msgr2=0x7fc214142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:28.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc219600640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc214142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:28.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc219600640 1 -- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c2d80 msgr2=0x7fc2141536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:28.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc219600640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c2d80 0x7fc2141536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:28.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc219600640 1 -- 172.21.15.113:0/2652272447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214154100 con 0x7fc21414e4c0 2024-03-04T07:36:28.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc219600640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214143670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00cbb0 tx=0x7fc20c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:28.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc210c00640 1 -- 172.21.15.113:0/2652272447 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c017070 con 0x7fc21414e4c0 2024-03-04T07:36:28.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc210c00640 1 -- 172.21.15.113:0/2652272447 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00ad40 con 0x7fc21414e4c0 2024-03-04T07:36:28.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc21b600640 1 -- 172.21.15.113:0/2652272447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2141543e0 con 0x7fc21414e4c0 2024-03-04T07:36:28.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc210c00640 1 -- 172.21.15.113:0/2652272447 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c012600 con 0x7fc21414e4c0 2024-03-04T07:36:28.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc21b600640 1 -- 172.21.15.113:0/2652272447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2141549a0 con 0x7fc21414e4c0 2024-03-04T07:36:28.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc21b600640 1 -- 172.21.15.113:0/2652272447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc214075c00 con 0x7fc21414e4c0 2024-03-04T07:36:28.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.599+0000 7fc210c00640 1 -- 172.21.15.113:0/2652272447 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc20c01a070 con 0x7fc21414e4c0 2024-03-04T07:36:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.603+0000 7fc210c00640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f4066f60 0x7fc1f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.603+0000 7fc210c00640 1 -- 172.21.15.113:0/2652272447 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fc20c084e90 con 0x7fc21414e4c0 2024-03-04T07:36:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.603+0000 7fc218c00640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f4066f60 0x7fc1f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.603+0000 7fc218c00640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f4066f60 0x7fc1f4069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc208009ac0 tx=0x7fc208009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:28.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.607+0000 7fc210c00640 1 -- 172.21.15.113:0/2652272447 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc20c0556e0 con 0x7fc21414e4c0 2024-03-04T07:36:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 -- 172.21.15.113:0/2652272447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc214151dc0 con 0x7fc21414e4c0 2024-03-04T07:36:28.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc210c00640 1 -- 172.21.15.113:0/2652272447 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5821 (secure 0 0 0) 0x7fc20c086090 con 0x7fc21414e4c0 2024-03-04T07:36:28.877 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:28.877 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4250,4276,4256]},"id":1}]} 2024-03-04T07:36:28.877 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 18 2024-03-04T07:36:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 -- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f4066f60 msgr2=0x7fc1f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f4066f60 0x7fc1f4069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc208009ac0 tx=0x7fc208009190 comp rx=0 tx=0).stop 2024-03-04T07:36:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 -- 172.21.15.113:0/2652272447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 msgr2=0x7fc214143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:28.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214143670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00cbb0 tx=0x7fc20c00a430 comp rx=0 tx=0).stop 2024-03-04T07:36:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 -- 172.21.15.113:0/2652272447 shutdown_connections 2024-03-04T07:36:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f4066f60 0x7fc1f4069420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214143670 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c2d80 0x7fc2141536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 --2- 172.21.15.113:0/2652272447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc214142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:28.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 -- 172.21.15.113:0/2652272447 >> 172.21.15.113:0/2652272447 conn(0x7fc21406e820 msgr2=0x7fc2140c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 -- 172.21.15.113:0/2652272447 shutdown_connections 2024-03-04T07:36:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:28.875+0000 7fc21b600640 1 -- 172.21.15.113:0/2652272447 wait complete. 2024-03-04T07:36:28.899 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=2, replacing gid=4253 2024-03-04T07:36:28.899 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.a 2024-03-04T07:36:29.205 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir foo 2024-03-04T07:36:29.214 INFO:tasks.workunit.client.0.smithi113.stderr:+ cp linux-2.6.33.tar.bz2 foo 2024-03-04T07:36:29.342 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir foo/.snap/barsnap 2024-03-04T07:36:31.660 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm foo/linux-2.6.33.tar.bz2 2024-03-04T07:36:31.676 INFO:tasks.workunit.client.0.smithi113.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2024-03-04T07:36:32.039 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo passed: files are identical 2024-03-04T07:36:32.039 INFO:tasks.workunit.client.0.smithi113.stdout:passed: files are identical 2024-03-04T07:36:32.039 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir foo/.snap/barsnap 2024-03-04T07:36:32.067 INFO:tasks.workunit.client.0.smithi113.stdout:OK 2024-03-04T07:36:32.068 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo OK 2024-03-04T07:36:32.068 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T07:36:32.068 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T07:36:32.116 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2024-03-04T07:36:32.116 DEBUG:teuthology.orchestra.run.smithi113:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2024-03-04T07:36:32.178 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo 1 2024-03-04T07:36:32.180 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo 2 2024-03-04T07:36:32.181 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo 3 2024-03-04T07:36:32.181 INFO:tasks.workunit.client.0.smithi113.stderr:+ '[' -e file4 ']' 2024-03-04T07:36:32.181 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/snap1 2024-03-04T07:36:32.204 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo 4 2024-03-04T07:36:32.207 INFO:tasks.workunit.client.0.smithi113.stderr:++ ls 2024-03-04T07:36:32.213 INFO:tasks.workunit.client.0.smithi113.stderr:+ now='file1 2024-03-04T07:36:32.214 INFO:tasks.workunit.client.0.smithi113.stderr:file2 2024-03-04T07:36:32.214 INFO:tasks.workunit.client.0.smithi113.stderr:file3 2024-03-04T07:36:32.214 INFO:tasks.workunit.client.0.smithi113.stderr:file4' 2024-03-04T07:36:32.214 INFO:tasks.workunit.client.0.smithi113.stderr:++ ls .snap/snap1 2024-03-04T07:36:32.591 INFO:tasks.workunit.client.0.smithi113.stderr:+ then='file1 2024-03-04T07:36:32.591 INFO:tasks.workunit.client.0.smithi113.stderr:file2 2024-03-04T07:36:32.591 INFO:tasks.workunit.client.0.smithi113.stderr:file3' 2024-03-04T07:36:32.591 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/snap1 2024-03-04T07:36:33.008 INFO:tasks.workunit.client.0.smithi113.stderr:+ '[' 'file1 2024-03-04T07:36:33.008 INFO:tasks.workunit.client.0.smithi113.stderr:file2 2024-03-04T07:36:33.008 INFO:tasks.workunit.client.0.smithi113.stderr:file3 2024-03-04T07:36:33.008 INFO:tasks.workunit.client.0.smithi113.stderr:file4' = 'file1 2024-03-04T07:36:33.009 INFO:tasks.workunit.client.0.smithi113.stderr:file2 2024-03-04T07:36:33.009 INFO:tasks.workunit.client.0.smithi113.stderr:file3' ']' 2024-03-04T07:36:33.009 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo 1 2024-03-04T07:36:33.018 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo 2 2024-03-04T07:36:33.028 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo 3 2024-03-04T07:36:33.038 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/snap1 2024-03-04T07:36:33.048 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo 4 2024-03-04T07:36:33.059 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/snap1 2024-03-04T07:36:33.068 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm file1 file2 file3 file4 2024-03-04T07:36:33.088 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo OK 2024-03-04T07:36:33.089 INFO:tasks.workunit.client.0.smithi113.stdout:OK 2024-03-04T07:36:33.090 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T07:36:33.090 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T07:36:33.165 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2024-03-04T07:36:33.165 DEBUG:teuthology.orchestra.run.smithi113:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2024-03-04T07:36:33.230 INFO:tasks.workunit.client.0.smithi113.stdout:Create dir 100 to 199 ... 2024-03-04T07:36:33.231 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 100 2024-03-04T07:36:33.240 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 101 2024-03-04T07:36:33.259 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 102 2024-03-04T07:36:33.280 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 103 2024-03-04T07:36:33.301 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 104 2024-03-04T07:36:33.323 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 105 2024-03-04T07:36:33.342 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 106 2024-03-04T07:36:33.358 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 107 2024-03-04T07:36:33.375 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 108 2024-03-04T07:36:33.390 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 109 2024-03-04T07:36:33.404 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 110 2024-03-04T07:36:33.418 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 111 2024-03-04T07:36:33.433 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 112 2024-03-04T07:36:33.446 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 113 2024-03-04T07:36:33.460 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 114 2024-03-04T07:36:33.473 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 115 2024-03-04T07:36:33.508 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 116 2024-03-04T07:36:33.520 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 117 2024-03-04T07:36:33.535 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 118 2024-03-04T07:36:33.548 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 119 2024-03-04T07:36:33.812 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 120 2024-03-04T07:36:33.913 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 121 2024-03-04T07:36:33.913 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 122 2024-03-04T07:36:33.913 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 123 2024-03-04T07:36:33.913 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 124 2024-03-04T07:36:33.934 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 125 2024-03-04T07:36:33.953 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 126 2024-03-04T07:36:34.114 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 127 2024-03-04T07:36:34.813 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 128 2024-03-04T07:36:34.837 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 129 2024-03-04T07:36:34.861 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 130 2024-03-04T07:36:34.884 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 131 2024-03-04T07:36:34.900 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-03-04T07:36:34.900 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-04T07:36:34.900 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-04T07:36:34.904 INFO:tasks.ceph.mds.a:Started 2024-03-04T07:36:34.904 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:34.905 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 132 2024-03-04T07:36:34.990 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-03-04T07:36:35.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.047+0000 7f18da200640 1 Processor -- start 2024-03-04T07:36:35.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.047+0000 7f18da200640 1 -- start start 2024-03-04T07:36:35.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18da200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f18d406ff20 0x7f18d406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:35.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18da200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d40ca0e0 con 0x7f18d4158010 2024-03-04T07:36:35.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18da200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d415aa30 con 0x7f18d40ce120 2024-03-04T07:36:35.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18da200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d406f450 con 0x7f18d406ff20 2024-03-04T07:36:35.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18d3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f18d4158010 0x7f18d415b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33918/0 (socket says 172.21.15.113:33918) 2024-03-04T07:36:35.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18d3400640 1 -- 172.21.15.113:0/51330549 learned_addr learned my addr 172.21.15.113:0/51330549 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:36:35.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18cbe00640 1 --2- 172.21.15.113:0/51330549 >> v2:172.21.15.136:3300/0 conn(0x7f18d406ff20 0x7f18d406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:35.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18d2a00640 1 -- 172.21.15.113:0/51330549 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645878417 0 0) 0x7f18d40ca0e0 con 0x7f18d4158010 2024-03-04T07:36:35.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18d2a00640 1 -- 172.21.15.113:0/51330549 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b8003530 con 0x7f18d4158010 2024-03-04T07:36:35.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18cbe00640 1 -- 172.21.15.113:0/51330549 >> v1:172.21.15.136:6789/0 conn(0x7f18d40ce120 legacy=0x7f18d41652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:35.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18cbe00640 1 -- 172.21.15.113:0/51330549 >> v1:172.21.15.113:6789/0 conn(0x7f18d4158010 legacy=0x7f18d415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:35.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.051+0000 7f18cbe00640 1 -- 172.21.15.113:0/51330549 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d406c590 con 0x7f18d406ff20 2024-03-04T07:36:35.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18cbe00640 1 --2- 172.21.15.113:0/51330549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d406ff20 0x7f18d406dd50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0097c0 tx=0x7f18bc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c284571f72d963f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:35.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18d2a00640 1 -- 172.21.15.113:0/51330549 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0b0070 con 0x7f18d406ff20 2024-03-04T07:36:35.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18d2a00640 1 -- 172.21.15.113:0/51330549 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc0ab600 con 0x7f18d406ff20 2024-03-04T07:36:35.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18d2a00640 1 -- 172.21.15.113:0/51330549 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0ab9b0 con 0x7f18d406ff20 2024-03-04T07:36:35.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18da200640 1 -- 172.21.15.113:0/51330549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d406ff20 msgr2=0x7f18d406dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:35.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18da200640 1 --2- 172.21.15.113:0/51330549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d406ff20 0x7f18d406dd50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0097c0 tx=0x7f18bc01c4a0 comp rx=0 tx=0).stop 2024-03-04T07:36:35.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18da200640 1 -- 172.21.15.113:0/51330549 shutdown_connections 2024-03-04T07:36:35.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18da200640 1 --2- 172.21.15.113:0/51330549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d406ff20 0x7f18d406dd50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:35.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18da200640 1 -- 172.21.15.113:0/51330549 >> 172.21.15.113:0/51330549 conn(0x7f18d40734d0 msgr2=0x7f18d4066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:35.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18da200640 1 -- 172.21.15.113:0/51330549 shutdown_connections 2024-03-04T07:36:35.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18da200640 1 -- 172.21.15.113:0/51330549 wait complete. 2024-03-04T07:36:35.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18da200640 1 Processor -- start 2024-03-04T07:36:35.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.055+0000 7f18da200640 1 -- start start 2024-03-04T07:36:35.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18da200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d406ff20 0x7f18d4150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:35.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18da200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d40ce120 0x7f18d414f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:35.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18da200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d4158010 0x7f18d41532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:35.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18da200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d415b920 con 0x7f18d40ce120 2024-03-04T07:36:35.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18da200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d406e290 con 0x7f18d406ff20 2024-03-04T07:36:35.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18da200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d41583f0 con 0x7f18d4158010 2024-03-04T07:36:35.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d40ce120 0x7f18d414f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:35.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d40ce120 0x7f18d414f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39478/0 (socket says 172.21.15.113:39478) 2024-03-04T07:36:35.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cbe00640 1 -- 172.21.15.113:0/1035003393 learned_addr learned my addr 172.21.15.113:0/1035003393 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:35.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18d3400640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d406ff20 0x7f18d4150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:35.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18d3e00640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d4158010 0x7f18d41532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:35.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cbe00640 1 -- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d4158010 msgr2=0x7f18d41532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:35.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cbe00640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d4158010 0x7f18d41532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:35.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cbe00640 1 -- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d406ff20 msgr2=0x7f18d4150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:35.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cbe00640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d406ff20 0x7f18d4150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:35.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cbe00640 1 -- 172.21.15.113:0/1035003393 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d4152050 con 0x7f18d40ce120 2024-03-04T07:36:35.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cbe00640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d40ce120 0x7f18d414f360 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0ae040 tx=0x7f18bc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:35.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cb400640 1 -- 172.21.15.113:0/1035003393 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0b0050 con 0x7f18d40ce120 2024-03-04T07:36:35.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cb400640 1 -- 172.21.15.113:0/1035003393 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc0b1040 con 0x7f18d40ce120 2024-03-04T07:36:35.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18da200640 1 -- 172.21.15.113:0/1035003393 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d41522f0 con 0x7f18d40ce120 2024-03-04T07:36:35.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18cb400640 1 -- 172.21.15.113:0/1035003393 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc006e20 con 0x7f18d40ce120 2024-03-04T07:36:35.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18da200640 1 -- 172.21.15.113:0/1035003393 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d41528b0 con 0x7f18d40ce120 2024-03-04T07:36:35.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.059+0000 7f18da200640 1 -- 172.21.15.113:0/1035003393 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d4152af0 con 0x7f18d40ce120 2024-03-04T07:36:35.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.063+0000 7f18cb400640 1 -- 172.21.15.113:0/1035003393 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18bc0be060 con 0x7f18d40ce120 2024-03-04T07:36:35.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.063+0000 7f18cb400640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18b0066f60 0x7f18b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:35.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.063+0000 7f18cb400640 1 -- 172.21.15.113:0/1035003393 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f18bc11e930 con 0x7f18d40ce120 2024-03-04T07:36:35.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.063+0000 7f18d3400640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18b0066f60 0x7f18b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:35.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.063+0000 7f18d3400640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18b0066f60 0x7f18b0069420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f18c40011f0 tx=0x7f18c4007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:35.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.067+0000 7f18cb400640 1 -- 172.21.15.113:0/1035003393 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f18d4152af0 con 0x7f18d40ce120 2024-03-04T07:36:35.106 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 133 2024-03-04T07:36:35.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.267+0000 7f18da200640 1 -- 172.21.15.113:0/1035003393 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18d41f4610 con 0x7f18d40ce120 2024-03-04T07:36:35.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.267+0000 7f18cb400640 1 -- 172.21.15.113:0/1035003393 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+5821 (secure 0 0 0) 0x7f18bc016090 con 0x7f18d40ce120 2024-03-04T07:36:35.270 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:35.270 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4250,4276,4256]},"id":1}]} 2024-03-04T07:36:35.270 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 18 2024-03-04T07:36:35.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 -- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18b0066f60 msgr2=0x7f18b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:35.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18b0066f60 0x7f18b0069420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f18c40011f0 tx=0x7f18c4007d40 comp rx=0 tx=0).stop 2024-03-04T07:36:35.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 -- 172.21.15.113:0/1035003393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d40ce120 msgr2=0x7f18d414f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:35.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d40ce120 0x7f18d414f360 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0ae040 tx=0x7f18bc01ace0 comp rx=0 tx=0).stop 2024-03-04T07:36:35.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 -- 172.21.15.113:0/1035003393 shutdown_connections 2024-03-04T07:36:35.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18d4158010 0x7f18d41532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:35.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18b0066f60 0x7f18b0069420 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:35.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18d40ce120 0x7f18d414f360 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:35.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 --2- 172.21.15.113:0/1035003393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18d406ff20 0x7f18d4150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:35.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 -- 172.21.15.113:0/1035003393 >> 172.21.15.113:0/1035003393 conn(0x7f18d40734d0 msgr2=0x7f18d40cbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:35.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 -- 172.21.15.113:0/1035003393 shutdown_connections 2024-03-04T07:36:35.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:35.271+0000 7f18da200640 1 -- 172.21.15.113:0/1035003393 wait complete. 2024-03-04T07:36:35.291 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-03-04T07:36:35.355 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 134 2024-03-04T07:36:35.378 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 135 2024-03-04T07:36:35.400 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 136 2024-03-04T07:36:35.423 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 137 2024-03-04T07:36:35.445 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 138 2024-03-04T07:36:35.463 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 139 2024-03-04T07:36:36.130 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 140 2024-03-04T07:36:36.333 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 141 2024-03-04T07:36:37.078 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 142 2024-03-04T07:36:37.098 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 143 2024-03-04T07:36:37.122 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 144 2024-03-04T07:36:37.292 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:36:37.326 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 145 2024-03-04T07:36:37.347 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 146 2024-03-04T07:36:37.368 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 147 2024-03-04T07:36:37.389 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 148 2024-03-04T07:36:37.409 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 149 2024-03-04T07:36:37.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dda600640 1 Processor -- start 2024-03-04T07:36:37.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dda600640 1 -- start start 2024-03-04T07:36:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dda600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0dd4158230 0x7f0dd406f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dda600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0dd40703e0 0x7f0dd40ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dda600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd406c310 con 0x7f0dd406df10 2024-03-04T07:36:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dda600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd40c95f0 con 0x7f0dd40703e0 2024-03-04T07:36:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dda600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd406c5b0 con 0x7f0dd4158230 2024-03-04T07:36:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dd3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0dd40703e0 0x7f0dd40ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dcaa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0dd406df10 0x7f0dd415e190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34272/0 (socket says 172.21.15.113:34272) 2024-03-04T07:36:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dcaa00640 1 -- 172.21.15.113:0/2655603238 learned_addr learned my addr 172.21.15.113:0/2655603238 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:36:37.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dd3400640 1 --2- 172.21.15.113:0/2655603238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0dd4158230 0x7f0dd406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:37.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dd3e00640 1 -- 172.21.15.113:0/2655603238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0dd4158230 msgr2=0x7f0dd406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:36:37.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dd3e00640 1 --2- 172.21.15.113:0/2655603238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0dd4158230 0x7f0dd406f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:37.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dd3e00640 1 -- 172.21.15.113:0/2655603238 >> v1:172.21.15.113:6789/0 conn(0x7f0dd406df10 legacy=0x7f0dd415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:37.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.459+0000 7f0dd3e00640 1 -- 172.21.15.113:0/2655603238 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd40c9890 con 0x7f0dd40703e0 2024-03-04T07:36:37.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.463+0000 7f0dd3e00640 1 --2- 172.21.15.113:0/2655603238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd40703e0 0x7f0dd40ce3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009d90 tx=0x7f0dcc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b657e6aa3074b504 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:37.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.463+0000 7f0dd2a00640 1 -- 172.21.15.113:0/2655603238 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0b2070 con 0x7f0dd40703e0 2024-03-04T07:36:37.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.463+0000 7f0dd2a00640 1 -- 172.21.15.113:0/2655603238 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc0a5830 con 0x7f0dd40703e0 2024-03-04T07:36:37.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.463+0000 7f0dd2a00640 1 -- 172.21.15.113:0/2655603238 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0a5c00 con 0x7f0dd40703e0 2024-03-04T07:36:37.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.463+0000 7f0dda600640 1 -- 172.21.15.113:0/2655603238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd40703e0 msgr2=0x7f0dd40ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:37.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.463+0000 7f0dda600640 1 --2- 172.21.15.113:0/2655603238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd40703e0 0x7f0dd40ce3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009d90 tx=0x7f0dcc01c970 comp rx=0 tx=0).stop 2024-03-04T07:36:37.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.463+0000 7f0dda600640 1 -- 172.21.15.113:0/2655603238 shutdown_connections 2024-03-04T07:36:37.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.463+0000 7f0dda600640 1 --2- 172.21.15.113:0/2655603238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd40703e0 0x7f0dd40ce3c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:37.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.467+0000 7f0dda600640 1 --2- 172.21.15.113:0/2655603238 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0dd4158230 0x7f0dd406f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:37.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.467+0000 7f0dda600640 1 -- 172.21.15.113:0/2655603238 >> 172.21.15.113:0/2655603238 conn(0x7f0dd4066db0 msgr2=0x7f0dd40671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:37.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.467+0000 7f0dda600640 1 -- 172.21.15.113:0/2655603238 shutdown_connections 2024-03-04T07:36:37.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.467+0000 7f0dda600640 1 -- 172.21.15.113:0/2655603238 wait complete. 2024-03-04T07:36:37.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.467+0000 7f0dda600640 1 Processor -- start 2024-03-04T07:36:37.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.467+0000 7f0dda600640 1 -- start start 2024-03-04T07:36:37.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dda600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0dd406df10 0x7f0dd415d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:37.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dda600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dd40703e0 0x7f0dd40c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:37.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dda600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd4158230 0x7f0dd40b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dda600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd415ea20 con 0x7f0dd4158230 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dda600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd415e8a0 con 0x7f0dd40703e0 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dda600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd4167ce0 con 0x7f0dd406df10 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd4158230 0x7f0dd40b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dd3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd4158230 0x7f0dd40b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33094/0 (socket says 172.21.15.113:33094) 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dd3e00640 1 -- 172.21.15.113:0/2060521217 learned_addr learned my addr 172.21.15.113:0/2060521217 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dd3400640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0dd406df10 0x7f0dd415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dcaa00640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dd40703e0 0x7f0dd40c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dd3e00640 1 -- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0dd406df10 msgr2=0x7f0dd415d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dd3e00640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0dd406df10 0x7f0dd415d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dd3e00640 1 -- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dd40703e0 msgr2=0x7f0dd40c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dd3e00640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dd40703e0 0x7f0dd40c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dd3e00640 1 -- 172.21.15.113:0/2060521217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd41f6fa0 con 0x7f0dd4158230 2024-03-04T07:36:37.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dcaa00640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dd40703e0 0x7f0dd40c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:36:37.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dd3e00640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd4158230 0x7f0dd40b1a80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0b0040 tx=0x7f0dcc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:37.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dcbe00640 1 -- 172.21.15.113:0/2060521217 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0b2050 con 0x7f0dd4158230 2024-03-04T07:36:37.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dcbe00640 1 -- 172.21.15.113:0/2060521217 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc0be070 con 0x7f0dd4158230 2024-03-04T07:36:37.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dda600640 1 -- 172.21.15.113:0/2060521217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd41f7240 con 0x7f0dd4158230 2024-03-04T07:36:37.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dcbe00640 1 -- 172.21.15.113:0/2060521217 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc0ada90 con 0x7f0dd4158230 2024-03-04T07:36:37.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dda600640 1 -- 172.21.15.113:0/2060521217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd41f7800 con 0x7f0dd4158230 2024-03-04T07:36:37.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dda600640 1 -- 172.21.15.113:0/2060521217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd4075c00 con 0x7f0dd4158230 2024-03-04T07:36:37.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.471+0000 7f0dcbe00640 1 -- 172.21.15.113:0/2060521217 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0dcc0a56b0 con 0x7f0dd4158230 2024-03-04T07:36:37.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.475+0000 7f0dcbe00640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0db0066f90 0x7f0db0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:36:37.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.475+0000 7f0dcbe00640 1 -- 172.21.15.113:0/2060521217 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0dcc120100 con 0x7f0dd4158230 2024-03-04T07:36:37.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.475+0000 7f0dd3400640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0db0066f90 0x7f0db0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:36:37.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.475+0000 7f0dd3400640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0db0066f90 0x7f0db0069450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4007a60 tx=0x7f0dc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:36:37.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.479+0000 7f0dcbe00640 1 -- 172.21.15.113:0/2060521217 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0dcc016560 con 0x7f0dd4158230 2024-03-04T07:36:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 -- 172.21.15.113:0/2060521217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dd40701c0 con 0x7f0dd4158230 2024-03-04T07:36:37.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dcbe00640 1 -- 172.21.15.113:0/2060521217 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f0dcc0f1120 con 0x7f0dd4158230 2024-03-04T07:36:37.741 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:36:37.741 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4938,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4250]},"id":1}]} 2024-03-04T07:36:37.742 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 19 2024-03-04T07:36:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 -- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0db0066f90 msgr2=0x7f0db0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0db0066f90 0x7f0db0069450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4007a60 tx=0x7f0dc4008040 comp rx=0 tx=0).stop 2024-03-04T07:36:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 -- 172.21.15.113:0/2060521217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd4158230 msgr2=0x7f0dd40b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:36:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd4158230 0x7f0dd40b1a80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0b0040 tx=0x7f0dcc019710 comp rx=0 tx=0).stop 2024-03-04T07:36:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 -- 172.21.15.113:0/2060521217 shutdown_connections 2024-03-04T07:36:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0db0066f90 0x7f0db0069450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0dd4158230 0x7f0dd40b1a80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0dd40703e0 0x7f0dd40c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 --2- 172.21.15.113:0/2060521217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0dd406df10 0x7f0dd415d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:36:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 -- 172.21.15.113:0/2060521217 >> 172.21.15.113:0/2060521217 conn(0x7f0dd4066db0 msgr2=0x7f0dd4155790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:36:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.739+0000 7f0dda600640 1 -- 172.21.15.113:0/2060521217 shutdown_connections 2024-03-04T07:36:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:36:37.743+0000 7f0dda600640 1 -- 172.21.15.113:0/2060521217 wait complete. 2024-03-04T07:36:37.764 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-03-04T07:36:37.765 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 46 secs before thrashing 2024-03-04T07:36:38.114 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 150 2024-03-04T07:36:38.391 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 151 2024-03-04T07:36:38.905 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 152 2024-03-04T07:36:38.926 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 153 2024-03-04T07:36:38.952 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 154 2024-03-04T07:36:38.968 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 155 2024-03-04T07:36:38.989 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 156 2024-03-04T07:36:39.007 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 157 2024-03-04T07:36:39.025 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 158 2024-03-04T07:36:39.042 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 159 2024-03-04T07:36:39.058 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 160 2024-03-04T07:36:39.072 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 161 2024-03-04T07:36:39.085 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 162 2024-03-04T07:36:39.098 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 163 2024-03-04T07:36:39.120 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 164 2024-03-04T07:36:39.132 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 165 2024-03-04T07:36:39.145 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 166 2024-03-04T07:36:39.795 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 167 2024-03-04T07:36:39.810 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 168 2024-03-04T07:36:39.825 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 169 2024-03-04T07:36:39.846 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 170 2024-03-04T07:36:39.865 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 171 2024-03-04T07:36:39.883 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 172 2024-03-04T07:36:39.900 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 173 2024-03-04T07:36:39.916 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 174 2024-03-04T07:36:40.130 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 175 2024-03-04T07:36:40.269 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 176 2024-03-04T07:36:40.290 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 177 2024-03-04T07:36:40.312 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 178 2024-03-04T07:36:40.334 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 179 2024-03-04T07:36:40.354 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 180 2024-03-04T07:36:40.372 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 181 2024-03-04T07:36:40.390 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 182 2024-03-04T07:36:40.406 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 183 2024-03-04T07:36:40.421 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 184 2024-03-04T07:36:40.436 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 185 2024-03-04T07:36:40.451 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 186 2024-03-04T07:36:41.116 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 187 2024-03-04T07:36:41.139 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 188 2024-03-04T07:36:42.000 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 189 2024-03-04T07:36:42.011 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 190 2024-03-04T07:36:42.278 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 191 2024-03-04T07:36:42.303 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 192 2024-03-04T07:36:42.334 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 193 2024-03-04T07:36:42.366 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 194 2024-03-04T07:36:43.381 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 195 2024-03-04T07:36:43.410 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 196 2024-03-04T07:36:43.445 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 197 2024-03-04T07:36:43.476 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 198 2024-03-04T07:36:43.496 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 199 2024-03-04T07:36:43.510 INFO:tasks.workunit.client.0.smithi113.stdout:Take first snapshot .snap/test1 2024-03-04T07:36:43.848 INFO:tasks.workunit.client.0.smithi113.stdout:Create dir 200 to 299 ... 2024-03-04T07:36:43.850 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 200 2024-03-04T07:36:43.858 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 201 2024-03-04T07:36:44.130 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 202 2024-03-04T07:36:44.212 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 203 2024-03-04T07:36:44.230 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 204 2024-03-04T07:36:44.249 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 205 2024-03-04T07:36:44.268 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 206 2024-03-04T07:36:44.287 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 207 2024-03-04T07:36:44.303 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 208 2024-03-04T07:36:44.319 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 209 2024-03-04T07:36:44.334 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 210 2024-03-04T07:36:44.348 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 211 2024-03-04T07:36:44.362 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 212 2024-03-04T07:36:44.375 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 213 2024-03-04T07:36:44.387 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 214 2024-03-04T07:36:44.399 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 215 2024-03-04T07:36:44.411 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 216 2024-03-04T07:36:44.674 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 217 2024-03-04T07:36:44.686 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 218 2024-03-04T07:36:44.947 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 219 2024-03-04T07:36:44.968 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 220 2024-03-04T07:36:44.990 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 221 2024-03-04T07:36:45.012 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 222 2024-03-04T07:36:45.122 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 223 2024-03-04T07:36:46.000 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 224 2024-03-04T07:36:46.018 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 225 2024-03-04T07:36:46.115 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 226 2024-03-04T07:36:46.744 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 227 2024-03-04T07:36:46.768 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 228 2024-03-04T07:36:46.787 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 229 2024-03-04T07:36:46.808 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 230 2024-03-04T07:36:46.829 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 231 2024-03-04T07:36:46.847 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 232 2024-03-04T07:36:46.863 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 233 2024-03-04T07:36:46.879 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 234 2024-03-04T07:36:47.107 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 235 2024-03-04T07:36:47.913 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 236 2024-03-04T07:36:47.934 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 237 2024-03-04T07:36:47.954 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 238 2024-03-04T07:36:47.973 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 239 2024-03-04T07:36:47.992 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 240 2024-03-04T07:36:48.009 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 241 2024-03-04T07:36:48.024 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 242 2024-03-04T07:36:48.039 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 243 2024-03-04T07:36:48.129 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 244 2024-03-04T07:36:48.709 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 245 2024-03-04T07:36:48.728 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 246 2024-03-04T07:36:48.749 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 247 2024-03-04T07:36:48.768 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 248 2024-03-04T07:36:48.788 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 249 2024-03-04T07:36:48.806 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 250 2024-03-04T07:36:48.883 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 251 2024-03-04T07:36:52.619 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 252 2024-03-04T07:36:52.709 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 253 2024-03-04T07:36:52.732 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 254 2024-03-04T07:36:52.753 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 255 2024-03-04T07:36:52.774 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 256 2024-03-04T07:36:52.793 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 257 2024-03-04T07:36:52.811 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 258 2024-03-04T07:36:52.827 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 259 2024-03-04T07:36:52.843 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 260 2024-03-04T07:36:52.858 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 261 2024-03-04T07:36:52.873 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 262 2024-03-04T07:36:52.887 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 263 2024-03-04T07:36:52.900 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 264 2024-03-04T07:36:52.913 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 265 2024-03-04T07:36:52.926 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 266 2024-03-04T07:36:53.132 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 267 2024-03-04T07:36:53.401 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 268 2024-03-04T07:36:53.436 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 269 2024-03-04T07:36:53.458 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 270 2024-03-04T07:36:53.478 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 271 2024-03-04T07:36:53.497 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 272 2024-03-04T07:36:53.512 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 273 2024-03-04T07:36:53.529 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 274 2024-03-04T07:36:53.545 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 275 2024-03-04T07:36:53.560 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 276 2024-03-04T07:36:53.575 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 277 2024-03-04T07:36:53.588 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 278 2024-03-04T07:36:53.602 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 279 2024-03-04T07:36:53.622 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 280 2024-03-04T07:36:53.637 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 281 2024-03-04T07:36:53.649 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 282 2024-03-04T07:36:53.661 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 283 2024-03-04T07:36:53.672 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 284 2024-03-04T07:36:54.065 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 285 2024-03-04T07:36:54.078 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 286 2024-03-04T07:36:54.096 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 287 2024-03-04T07:36:54.116 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 288 2024-03-04T07:36:54.135 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 289 2024-03-04T07:36:54.605 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 290 2024-03-04T07:36:54.625 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 291 2024-03-04T07:36:54.648 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 292 2024-03-04T07:36:54.671 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 293 2024-03-04T07:36:54.691 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 294 2024-03-04T07:36:54.710 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 295 2024-03-04T07:36:54.741 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 296 2024-03-04T07:36:55.118 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 297 2024-03-04T07:36:55.140 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 298 2024-03-04T07:36:55.161 INFO:tasks.workunit.client.0.smithi113.stdout: create dir 299 2024-03-04T07:36:55.181 INFO:tasks.workunit.client.0.smithi113.stdout:Create a snapshot in every first level dir ... 2024-03-04T07:36:55.227 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:36:55.231 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:36:55.267 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/.snap/snap-subdir-test 2024-03-04T07:36:55.298 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/30 file after the snapshot 2024-03-04T07:36:55.303 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/31 file after the snapshot 2024-03-04T07:36:55.303 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/32 file after the snapshot 2024-03-04T07:36:55.303 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/33 file after the snapshot 2024-03-04T07:36:55.304 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/34 file after the snapshot 2024-03-04T07:36:55.304 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/35 file after the snapshot 2024-03-04T07:36:55.304 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/36 file after the snapshot 2024-03-04T07:36:55.304 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/37 file after the snapshot 2024-03-04T07:36:55.304 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/38 file after the snapshot 2024-03-04T07:36:55.304 INFO:tasks.workunit.client.0.smithi113.stdout: create 100/39 file after the snapshot 2024-03-04T07:36:55.304 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/.snap/snap-subdir-test 2024-03-04T07:36:55.357 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/30 file after the snapshot 2024-03-04T07:36:55.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/31 file after the snapshot 2024-03-04T07:36:55.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/32 file after the snapshot 2024-03-04T07:36:55.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/33 file after the snapshot 2024-03-04T07:36:55.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/34 file after the snapshot 2024-03-04T07:36:55.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/35 file after the snapshot 2024-03-04T07:36:55.363 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/36 file after the snapshot 2024-03-04T07:36:55.363 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/37 file after the snapshot 2024-03-04T07:36:55.363 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/38 file after the snapshot 2024-03-04T07:36:55.363 INFO:tasks.workunit.client.0.smithi113.stdout: create 101/39 file after the snapshot 2024-03-04T07:36:55.363 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/.snap/snap-subdir-test 2024-03-04T07:36:55.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/30 file after the snapshot 2024-03-04T07:36:55.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/31 file after the snapshot 2024-03-04T07:36:55.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/32 file after the snapshot 2024-03-04T07:36:55.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/33 file after the snapshot 2024-03-04T07:36:55.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/34 file after the snapshot 2024-03-04T07:36:55.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/35 file after the snapshot 2024-03-04T07:36:55.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/36 file after the snapshot 2024-03-04T07:36:55.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/37 file after the snapshot 2024-03-04T07:36:55.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/38 file after the snapshot 2024-03-04T07:36:55.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 102/39 file after the snapshot 2024-03-04T07:36:55.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/.snap/snap-subdir-test 2024-03-04T07:36:55.467 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/30 file after the snapshot 2024-03-04T07:36:55.470 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/31 file after the snapshot 2024-03-04T07:36:55.470 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/32 file after the snapshot 2024-03-04T07:36:55.470 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/33 file after the snapshot 2024-03-04T07:36:55.470 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/34 file after the snapshot 2024-03-04T07:36:55.470 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/35 file after the snapshot 2024-03-04T07:36:55.470 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/36 file after the snapshot 2024-03-04T07:36:55.470 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/37 file after the snapshot 2024-03-04T07:36:55.470 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/38 file after the snapshot 2024-03-04T07:36:55.470 INFO:tasks.workunit.client.0.smithi113.stdout: create 103/39 file after the snapshot 2024-03-04T07:36:55.470 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/.snap/snap-subdir-test 2024-03-04T07:36:55.941 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/30 file after the snapshot 2024-03-04T07:36:55.945 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/31 file after the snapshot 2024-03-04T07:36:55.946 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/32 file after the snapshot 2024-03-04T07:36:55.946 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/33 file after the snapshot 2024-03-04T07:36:55.946 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/34 file after the snapshot 2024-03-04T07:36:55.946 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/35 file after the snapshot 2024-03-04T07:36:55.946 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/36 file after the snapshot 2024-03-04T07:36:55.946 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/37 file after the snapshot 2024-03-04T07:36:55.946 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/38 file after the snapshot 2024-03-04T07:36:55.946 INFO:tasks.workunit.client.0.smithi113.stdout: create 104/39 file after the snapshot 2024-03-04T07:36:55.946 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/.snap/snap-subdir-test 2024-03-04T07:36:56.009 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/30 file after the snapshot 2024-03-04T07:36:56.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/31 file after the snapshot 2024-03-04T07:36:56.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/32 file after the snapshot 2024-03-04T07:36:56.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/33 file after the snapshot 2024-03-04T07:36:56.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/34 file after the snapshot 2024-03-04T07:36:56.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/35 file after the snapshot 2024-03-04T07:36:56.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/36 file after the snapshot 2024-03-04T07:36:56.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/37 file after the snapshot 2024-03-04T07:36:56.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/38 file after the snapshot 2024-03-04T07:36:56.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 105/39 file after the snapshot 2024-03-04T07:36:56.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/.snap/snap-subdir-test 2024-03-04T07:36:56.415 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/30 file after the snapshot 2024-03-04T07:36:56.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/31 file after the snapshot 2024-03-04T07:36:56.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/32 file after the snapshot 2024-03-04T07:36:56.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/33 file after the snapshot 2024-03-04T07:36:56.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/34 file after the snapshot 2024-03-04T07:36:56.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/35 file after the snapshot 2024-03-04T07:36:56.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/36 file after the snapshot 2024-03-04T07:36:56.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/37 file after the snapshot 2024-03-04T07:36:56.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/38 file after the snapshot 2024-03-04T07:36:56.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 106/39 file after the snapshot 2024-03-04T07:36:56.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/.snap/snap-subdir-test 2024-03-04T07:36:57.145 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/30 file after the snapshot 2024-03-04T07:36:57.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/31 file after the snapshot 2024-03-04T07:36:57.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/32 file after the snapshot 2024-03-04T07:36:57.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/33 file after the snapshot 2024-03-04T07:36:57.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/34 file after the snapshot 2024-03-04T07:36:57.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/35 file after the snapshot 2024-03-04T07:36:57.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/36 file after the snapshot 2024-03-04T07:36:57.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/37 file after the snapshot 2024-03-04T07:36:57.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/38 file after the snapshot 2024-03-04T07:36:57.151 INFO:tasks.workunit.client.0.smithi113.stdout: create 107/39 file after the snapshot 2024-03-04T07:36:57.151 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/.snap/snap-subdir-test 2024-03-04T07:36:57.203 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/30 file after the snapshot 2024-03-04T07:36:57.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/31 file after the snapshot 2024-03-04T07:36:57.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/32 file after the snapshot 2024-03-04T07:36:57.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/33 file after the snapshot 2024-03-04T07:36:57.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/34 file after the snapshot 2024-03-04T07:36:57.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/35 file after the snapshot 2024-03-04T07:36:57.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/36 file after the snapshot 2024-03-04T07:36:57.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/37 file after the snapshot 2024-03-04T07:36:57.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/38 file after the snapshot 2024-03-04T07:36:57.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 108/39 file after the snapshot 2024-03-04T07:36:57.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/.snap/snap-subdir-test 2024-03-04T07:36:57.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/30 file after the snapshot 2024-03-04T07:36:57.264 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/31 file after the snapshot 2024-03-04T07:36:57.265 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/32 file after the snapshot 2024-03-04T07:36:57.265 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/33 file after the snapshot 2024-03-04T07:36:57.265 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/34 file after the snapshot 2024-03-04T07:36:57.266 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/35 file after the snapshot 2024-03-04T07:36:57.266 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/36 file after the snapshot 2024-03-04T07:36:57.266 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/37 file after the snapshot 2024-03-04T07:36:57.266 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/38 file after the snapshot 2024-03-04T07:36:57.266 INFO:tasks.workunit.client.0.smithi113.stdout: create 109/39 file after the snapshot 2024-03-04T07:36:57.266 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/.snap/snap-subdir-test 2024-03-04T07:36:57.310 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/30 file after the snapshot 2024-03-04T07:36:57.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/31 file after the snapshot 2024-03-04T07:36:57.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/32 file after the snapshot 2024-03-04T07:36:57.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/33 file after the snapshot 2024-03-04T07:36:57.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/34 file after the snapshot 2024-03-04T07:36:57.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/35 file after the snapshot 2024-03-04T07:36:57.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/36 file after the snapshot 2024-03-04T07:36:57.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/37 file after the snapshot 2024-03-04T07:36:57.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/38 file after the snapshot 2024-03-04T07:36:57.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 110/39 file after the snapshot 2024-03-04T07:36:57.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/.snap/snap-subdir-test 2024-03-04T07:36:57.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/30 file after the snapshot 2024-03-04T07:36:57.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/31 file after the snapshot 2024-03-04T07:36:57.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/32 file after the snapshot 2024-03-04T07:36:57.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/33 file after the snapshot 2024-03-04T07:36:57.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/34 file after the snapshot 2024-03-04T07:36:57.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/35 file after the snapshot 2024-03-04T07:36:57.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/36 file after the snapshot 2024-03-04T07:36:57.363 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/37 file after the snapshot 2024-03-04T07:36:57.363 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/38 file after the snapshot 2024-03-04T07:36:57.363 INFO:tasks.workunit.client.0.smithi113.stdout: create 111/39 file after the snapshot 2024-03-04T07:36:57.363 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/.snap/snap-subdir-test 2024-03-04T07:36:57.437 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/30 file after the snapshot 2024-03-04T07:36:57.443 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/31 file after the snapshot 2024-03-04T07:36:57.443 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/32 file after the snapshot 2024-03-04T07:36:57.446 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/33 file after the snapshot 2024-03-04T07:36:57.446 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/34 file after the snapshot 2024-03-04T07:36:57.446 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/35 file after the snapshot 2024-03-04T07:36:57.446 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/36 file after the snapshot 2024-03-04T07:36:57.446 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/37 file after the snapshot 2024-03-04T07:36:57.446 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/38 file after the snapshot 2024-03-04T07:36:57.446 INFO:tasks.workunit.client.0.smithi113.stdout: create 112/39 file after the snapshot 2024-03-04T07:36:57.446 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/.snap/snap-subdir-test 2024-03-04T07:36:57.516 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/30 file after the snapshot 2024-03-04T07:36:57.532 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/31 file after the snapshot 2024-03-04T07:36:57.532 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/32 file after the snapshot 2024-03-04T07:36:57.532 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/33 file after the snapshot 2024-03-04T07:36:57.532 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/34 file after the snapshot 2024-03-04T07:36:57.533 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/35 file after the snapshot 2024-03-04T07:36:57.533 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/36 file after the snapshot 2024-03-04T07:36:57.533 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/37 file after the snapshot 2024-03-04T07:36:57.533 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/38 file after the snapshot 2024-03-04T07:36:57.533 INFO:tasks.workunit.client.0.smithi113.stdout: create 113/39 file after the snapshot 2024-03-04T07:36:57.533 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/.snap/snap-subdir-test 2024-03-04T07:36:57.577 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/30 file after the snapshot 2024-03-04T07:36:57.580 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/31 file after the snapshot 2024-03-04T07:36:57.580 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/32 file after the snapshot 2024-03-04T07:36:57.580 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/33 file after the snapshot 2024-03-04T07:36:57.580 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/34 file after the snapshot 2024-03-04T07:36:57.581 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/35 file after the snapshot 2024-03-04T07:36:57.581 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/36 file after the snapshot 2024-03-04T07:36:57.581 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/37 file after the snapshot 2024-03-04T07:36:57.581 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/38 file after the snapshot 2024-03-04T07:36:57.581 INFO:tasks.workunit.client.0.smithi113.stdout: create 114/39 file after the snapshot 2024-03-04T07:36:57.581 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/.snap/snap-subdir-test 2024-03-04T07:36:58.566 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/30 file after the snapshot 2024-03-04T07:36:58.571 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/31 file after the snapshot 2024-03-04T07:36:58.571 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/32 file after the snapshot 2024-03-04T07:36:58.571 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/33 file after the snapshot 2024-03-04T07:36:58.571 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/34 file after the snapshot 2024-03-04T07:36:58.571 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/35 file after the snapshot 2024-03-04T07:36:58.571 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/36 file after the snapshot 2024-03-04T07:36:58.571 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/37 file after the snapshot 2024-03-04T07:36:58.572 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/38 file after the snapshot 2024-03-04T07:36:58.572 INFO:tasks.workunit.client.0.smithi113.stdout: create 115/39 file after the snapshot 2024-03-04T07:36:58.572 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/.snap/snap-subdir-test 2024-03-04T07:36:58.630 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/30 file after the snapshot 2024-03-04T07:36:58.635 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/31 file after the snapshot 2024-03-04T07:36:58.635 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/32 file after the snapshot 2024-03-04T07:36:58.635 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/33 file after the snapshot 2024-03-04T07:36:58.635 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/34 file after the snapshot 2024-03-04T07:36:58.635 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/35 file after the snapshot 2024-03-04T07:36:58.636 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/36 file after the snapshot 2024-03-04T07:36:58.636 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/37 file after the snapshot 2024-03-04T07:36:58.636 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/38 file after the snapshot 2024-03-04T07:36:58.636 INFO:tasks.workunit.client.0.smithi113.stdout: create 116/39 file after the snapshot 2024-03-04T07:36:58.636 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/.snap/snap-subdir-test 2024-03-04T07:36:58.689 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/30 file after the snapshot 2024-03-04T07:36:58.694 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/31 file after the snapshot 2024-03-04T07:36:58.695 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/32 file after the snapshot 2024-03-04T07:36:58.695 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/33 file after the snapshot 2024-03-04T07:36:58.695 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/34 file after the snapshot 2024-03-04T07:36:58.695 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/35 file after the snapshot 2024-03-04T07:36:58.695 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/36 file after the snapshot 2024-03-04T07:36:58.695 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/37 file after the snapshot 2024-03-04T07:36:58.695 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/38 file after the snapshot 2024-03-04T07:36:58.695 INFO:tasks.workunit.client.0.smithi113.stdout: create 117/39 file after the snapshot 2024-03-04T07:36:58.695 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/.snap/snap-subdir-test 2024-03-04T07:36:58.998 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/30 file after the snapshot 2024-03-04T07:36:59.003 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/31 file after the snapshot 2024-03-04T07:36:59.003 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/32 file after the snapshot 2024-03-04T07:36:59.003 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/33 file after the snapshot 2024-03-04T07:36:59.003 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/34 file after the snapshot 2024-03-04T07:36:59.003 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/35 file after the snapshot 2024-03-04T07:36:59.004 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/36 file after the snapshot 2024-03-04T07:36:59.004 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/37 file after the snapshot 2024-03-04T07:36:59.004 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/38 file after the snapshot 2024-03-04T07:36:59.004 INFO:tasks.workunit.client.0.smithi113.stdout: create 118/39 file after the snapshot 2024-03-04T07:36:59.004 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/.snap/snap-subdir-test 2024-03-04T07:36:59.058 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/30 file after the snapshot 2024-03-04T07:36:59.064 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/31 file after the snapshot 2024-03-04T07:36:59.064 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/32 file after the snapshot 2024-03-04T07:36:59.064 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/33 file after the snapshot 2024-03-04T07:36:59.064 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/34 file after the snapshot 2024-03-04T07:36:59.064 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/35 file after the snapshot 2024-03-04T07:36:59.064 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/36 file after the snapshot 2024-03-04T07:36:59.064 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/37 file after the snapshot 2024-03-04T07:36:59.064 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/38 file after the snapshot 2024-03-04T07:36:59.064 INFO:tasks.workunit.client.0.smithi113.stdout: create 119/39 file after the snapshot 2024-03-04T07:36:59.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/.snap/snap-subdir-test 2024-03-04T07:36:59.127 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/30 file after the snapshot 2024-03-04T07:36:59.133 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/31 file after the snapshot 2024-03-04T07:36:59.133 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/32 file after the snapshot 2024-03-04T07:36:59.133 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/33 file after the snapshot 2024-03-04T07:36:59.133 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/34 file after the snapshot 2024-03-04T07:36:59.133 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/35 file after the snapshot 2024-03-04T07:36:59.133 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/36 file after the snapshot 2024-03-04T07:36:59.133 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/37 file after the snapshot 2024-03-04T07:36:59.133 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/38 file after the snapshot 2024-03-04T07:36:59.134 INFO:tasks.workunit.client.0.smithi113.stdout: create 120/39 file after the snapshot 2024-03-04T07:36:59.134 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/.snap/snap-subdir-test 2024-03-04T07:36:59.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/30 file after the snapshot 2024-03-04T07:36:59.182 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/31 file after the snapshot 2024-03-04T07:36:59.182 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/32 file after the snapshot 2024-03-04T07:36:59.182 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/33 file after the snapshot 2024-03-04T07:36:59.182 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/34 file after the snapshot 2024-03-04T07:36:59.182 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/35 file after the snapshot 2024-03-04T07:36:59.182 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/36 file after the snapshot 2024-03-04T07:36:59.182 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/37 file after the snapshot 2024-03-04T07:36:59.183 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/38 file after the snapshot 2024-03-04T07:36:59.183 INFO:tasks.workunit.client.0.smithi113.stdout: create 121/39 file after the snapshot 2024-03-04T07:36:59.183 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/.snap/snap-subdir-test 2024-03-04T07:37:00.131 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/30 file after the snapshot 2024-03-04T07:37:00.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/31 file after the snapshot 2024-03-04T07:37:00.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/32 file after the snapshot 2024-03-04T07:37:00.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/33 file after the snapshot 2024-03-04T07:37:00.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/34 file after the snapshot 2024-03-04T07:37:00.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/35 file after the snapshot 2024-03-04T07:37:00.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/36 file after the snapshot 2024-03-04T07:37:00.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/37 file after the snapshot 2024-03-04T07:37:00.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/38 file after the snapshot 2024-03-04T07:37:00.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 122/39 file after the snapshot 2024-03-04T07:37:00.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/.snap/snap-subdir-test 2024-03-04T07:37:00.225 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/30 file after the snapshot 2024-03-04T07:37:00.227 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/31 file after the snapshot 2024-03-04T07:37:00.227 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/32 file after the snapshot 2024-03-04T07:37:00.227 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/33 file after the snapshot 2024-03-04T07:37:00.227 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/34 file after the snapshot 2024-03-04T07:37:00.227 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/35 file after the snapshot 2024-03-04T07:37:00.227 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/36 file after the snapshot 2024-03-04T07:37:00.227 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/37 file after the snapshot 2024-03-04T07:37:00.227 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/38 file after the snapshot 2024-03-04T07:37:00.227 INFO:tasks.workunit.client.0.smithi113.stdout: create 123/39 file after the snapshot 2024-03-04T07:37:00.227 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/.snap/snap-subdir-test 2024-03-04T07:37:00.509 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/30 file after the snapshot 2024-03-04T07:37:00.512 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/31 file after the snapshot 2024-03-04T07:37:00.512 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/32 file after the snapshot 2024-03-04T07:37:00.512 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/33 file after the snapshot 2024-03-04T07:37:00.513 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/34 file after the snapshot 2024-03-04T07:37:00.513 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/35 file after the snapshot 2024-03-04T07:37:00.513 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/36 file after the snapshot 2024-03-04T07:37:00.513 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/37 file after the snapshot 2024-03-04T07:37:00.513 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/38 file after the snapshot 2024-03-04T07:37:00.513 INFO:tasks.workunit.client.0.smithi113.stdout: create 124/39 file after the snapshot 2024-03-04T07:37:00.513 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/.snap/snap-subdir-test 2024-03-04T07:37:01.141 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/30 file after the snapshot 2024-03-04T07:37:01.144 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/31 file after the snapshot 2024-03-04T07:37:01.144 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/32 file after the snapshot 2024-03-04T07:37:01.144 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/33 file after the snapshot 2024-03-04T07:37:01.144 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/34 file after the snapshot 2024-03-04T07:37:01.144 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/35 file after the snapshot 2024-03-04T07:37:01.144 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/36 file after the snapshot 2024-03-04T07:37:01.145 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/37 file after the snapshot 2024-03-04T07:37:01.145 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/38 file after the snapshot 2024-03-04T07:37:01.145 INFO:tasks.workunit.client.0.smithi113.stdout: create 125/39 file after the snapshot 2024-03-04T07:37:01.145 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/.snap/snap-subdir-test 2024-03-04T07:37:01.200 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/30 file after the snapshot 2024-03-04T07:37:01.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/31 file after the snapshot 2024-03-04T07:37:01.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/32 file after the snapshot 2024-03-04T07:37:01.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/33 file after the snapshot 2024-03-04T07:37:01.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/34 file after the snapshot 2024-03-04T07:37:01.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/35 file after the snapshot 2024-03-04T07:37:01.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/36 file after the snapshot 2024-03-04T07:37:01.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/37 file after the snapshot 2024-03-04T07:37:01.206 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/38 file after the snapshot 2024-03-04T07:37:01.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 126/39 file after the snapshot 2024-03-04T07:37:01.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/.snap/snap-subdir-test 2024-03-04T07:37:01.257 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/30 file after the snapshot 2024-03-04T07:37:01.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/31 file after the snapshot 2024-03-04T07:37:01.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/32 file after the snapshot 2024-03-04T07:37:01.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/33 file after the snapshot 2024-03-04T07:37:01.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/34 file after the snapshot 2024-03-04T07:37:01.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/35 file after the snapshot 2024-03-04T07:37:01.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/36 file after the snapshot 2024-03-04T07:37:01.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/37 file after the snapshot 2024-03-04T07:37:01.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/38 file after the snapshot 2024-03-04T07:37:01.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 127/39 file after the snapshot 2024-03-04T07:37:01.261 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/.snap/snap-subdir-test 2024-03-04T07:37:01.310 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/30 file after the snapshot 2024-03-04T07:37:01.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/31 file after the snapshot 2024-03-04T07:37:01.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/32 file after the snapshot 2024-03-04T07:37:01.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/33 file after the snapshot 2024-03-04T07:37:01.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/34 file after the snapshot 2024-03-04T07:37:01.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/35 file after the snapshot 2024-03-04T07:37:01.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/36 file after the snapshot 2024-03-04T07:37:01.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/37 file after the snapshot 2024-03-04T07:37:01.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/38 file after the snapshot 2024-03-04T07:37:01.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 128/39 file after the snapshot 2024-03-04T07:37:01.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/.snap/snap-subdir-test 2024-03-04T07:37:01.375 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/30 file after the snapshot 2024-03-04T07:37:01.379 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/31 file after the snapshot 2024-03-04T07:37:01.379 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/32 file after the snapshot 2024-03-04T07:37:01.379 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/33 file after the snapshot 2024-03-04T07:37:01.379 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/34 file after the snapshot 2024-03-04T07:37:01.379 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/35 file after the snapshot 2024-03-04T07:37:01.379 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/36 file after the snapshot 2024-03-04T07:37:01.379 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/37 file after the snapshot 2024-03-04T07:37:01.379 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/38 file after the snapshot 2024-03-04T07:37:01.379 INFO:tasks.workunit.client.0.smithi113.stdout: create 129/39 file after the snapshot 2024-03-04T07:37:01.379 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/.snap/snap-subdir-test 2024-03-04T07:37:01.444 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/30 file after the snapshot 2024-03-04T07:37:01.833 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/31 file after the snapshot 2024-03-04T07:37:01.833 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/32 file after the snapshot 2024-03-04T07:37:01.833 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/33 file after the snapshot 2024-03-04T07:37:01.833 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/34 file after the snapshot 2024-03-04T07:37:01.833 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/35 file after the snapshot 2024-03-04T07:37:01.833 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/36 file after the snapshot 2024-03-04T07:37:01.833 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/37 file after the snapshot 2024-03-04T07:37:01.833 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/38 file after the snapshot 2024-03-04T07:37:01.834 INFO:tasks.workunit.client.0.smithi113.stdout: create 130/39 file after the snapshot 2024-03-04T07:37:01.834 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/.snap/snap-subdir-test 2024-03-04T07:37:01.884 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/30 file after the snapshot 2024-03-04T07:37:01.889 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/31 file after the snapshot 2024-03-04T07:37:01.889 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/32 file after the snapshot 2024-03-04T07:37:01.890 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/33 file after the snapshot 2024-03-04T07:37:01.890 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/34 file after the snapshot 2024-03-04T07:37:01.890 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/35 file after the snapshot 2024-03-04T07:37:01.890 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/36 file after the snapshot 2024-03-04T07:37:01.890 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/37 file after the snapshot 2024-03-04T07:37:01.890 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/38 file after the snapshot 2024-03-04T07:37:01.890 INFO:tasks.workunit.client.0.smithi113.stdout: create 131/39 file after the snapshot 2024-03-04T07:37:01.890 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/.snap/snap-subdir-test 2024-03-04T07:37:01.948 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/30 file after the snapshot 2024-03-04T07:37:01.953 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/31 file after the snapshot 2024-03-04T07:37:01.953 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/32 file after the snapshot 2024-03-04T07:37:01.954 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/33 file after the snapshot 2024-03-04T07:37:01.954 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/34 file after the snapshot 2024-03-04T07:37:01.954 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/35 file after the snapshot 2024-03-04T07:37:01.954 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/36 file after the snapshot 2024-03-04T07:37:01.954 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/37 file after the snapshot 2024-03-04T07:37:01.954 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/38 file after the snapshot 2024-03-04T07:37:01.954 INFO:tasks.workunit.client.0.smithi113.stdout: create 132/39 file after the snapshot 2024-03-04T07:37:01.954 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/.snap/snap-subdir-test 2024-03-04T07:37:02.010 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/30 file after the snapshot 2024-03-04T07:37:02.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/31 file after the snapshot 2024-03-04T07:37:02.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/32 file after the snapshot 2024-03-04T07:37:02.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/33 file after the snapshot 2024-03-04T07:37:02.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/34 file after the snapshot 2024-03-04T07:37:02.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/35 file after the snapshot 2024-03-04T07:37:02.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/36 file after the snapshot 2024-03-04T07:37:02.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/37 file after the snapshot 2024-03-04T07:37:02.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/38 file after the snapshot 2024-03-04T07:37:02.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 133/39 file after the snapshot 2024-03-04T07:37:02.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/.snap/snap-subdir-test 2024-03-04T07:37:02.074 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/30 file after the snapshot 2024-03-04T07:37:02.079 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/31 file after the snapshot 2024-03-04T07:37:02.079 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/32 file after the snapshot 2024-03-04T07:37:02.079 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/33 file after the snapshot 2024-03-04T07:37:02.079 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/34 file after the snapshot 2024-03-04T07:37:02.079 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/35 file after the snapshot 2024-03-04T07:37:02.079 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/36 file after the snapshot 2024-03-04T07:37:02.080 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/37 file after the snapshot 2024-03-04T07:37:02.080 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/38 file after the snapshot 2024-03-04T07:37:02.080 INFO:tasks.workunit.client.0.smithi113.stdout: create 134/39 file after the snapshot 2024-03-04T07:37:02.080 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/.snap/snap-subdir-test 2024-03-04T07:37:02.124 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/30 file after the snapshot 2024-03-04T07:37:02.127 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/31 file after the snapshot 2024-03-04T07:37:02.127 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/32 file after the snapshot 2024-03-04T07:37:02.127 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/33 file after the snapshot 2024-03-04T07:37:02.127 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/34 file after the snapshot 2024-03-04T07:37:02.127 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/35 file after the snapshot 2024-03-04T07:37:02.128 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/36 file after the snapshot 2024-03-04T07:37:02.128 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/37 file after the snapshot 2024-03-04T07:37:02.128 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/38 file after the snapshot 2024-03-04T07:37:02.128 INFO:tasks.workunit.client.0.smithi113.stdout: create 135/39 file after the snapshot 2024-03-04T07:37:02.128 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/.snap/snap-subdir-test 2024-03-04T07:37:02.183 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/30 file after the snapshot 2024-03-04T07:37:02.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/31 file after the snapshot 2024-03-04T07:37:02.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/32 file after the snapshot 2024-03-04T07:37:02.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/33 file after the snapshot 2024-03-04T07:37:02.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/34 file after the snapshot 2024-03-04T07:37:02.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/35 file after the snapshot 2024-03-04T07:37:02.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/36 file after the snapshot 2024-03-04T07:37:02.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/37 file after the snapshot 2024-03-04T07:37:02.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/38 file after the snapshot 2024-03-04T07:37:02.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 136/39 file after the snapshot 2024-03-04T07:37:02.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/.snap/snap-subdir-test 2024-03-04T07:37:02.244 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/30 file after the snapshot 2024-03-04T07:37:02.247 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/31 file after the snapshot 2024-03-04T07:37:02.247 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/32 file after the snapshot 2024-03-04T07:37:02.247 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/33 file after the snapshot 2024-03-04T07:37:02.247 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/34 file after the snapshot 2024-03-04T07:37:02.247 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/35 file after the snapshot 2024-03-04T07:37:02.248 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/36 file after the snapshot 2024-03-04T07:37:02.248 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/37 file after the snapshot 2024-03-04T07:37:02.248 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/38 file after the snapshot 2024-03-04T07:37:02.248 INFO:tasks.workunit.client.0.smithi113.stdout: create 137/39 file after the snapshot 2024-03-04T07:37:02.248 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/.snap/snap-subdir-test 2024-03-04T07:37:02.295 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/30 file after the snapshot 2024-03-04T07:37:02.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/31 file after the snapshot 2024-03-04T07:37:02.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/32 file after the snapshot 2024-03-04T07:37:02.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/33 file after the snapshot 2024-03-04T07:37:02.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/34 file after the snapshot 2024-03-04T07:37:02.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/35 file after the snapshot 2024-03-04T07:37:02.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/36 file after the snapshot 2024-03-04T07:37:02.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/37 file after the snapshot 2024-03-04T07:37:02.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/38 file after the snapshot 2024-03-04T07:37:02.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 138/39 file after the snapshot 2024-03-04T07:37:02.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/.snap/snap-subdir-test 2024-03-04T07:37:02.350 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/30 file after the snapshot 2024-03-04T07:37:02.355 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/31 file after the snapshot 2024-03-04T07:37:02.355 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/32 file after the snapshot 2024-03-04T07:37:02.355 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/33 file after the snapshot 2024-03-04T07:37:02.355 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/34 file after the snapshot 2024-03-04T07:37:02.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/35 file after the snapshot 2024-03-04T07:37:02.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/36 file after the snapshot 2024-03-04T07:37:02.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/37 file after the snapshot 2024-03-04T07:37:02.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/38 file after the snapshot 2024-03-04T07:37:02.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 139/39 file after the snapshot 2024-03-04T07:37:02.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/.snap/snap-subdir-test 2024-03-04T07:37:02.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/30 file after the snapshot 2024-03-04T07:37:02.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/31 file after the snapshot 2024-03-04T07:37:02.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/32 file after the snapshot 2024-03-04T07:37:02.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/33 file after the snapshot 2024-03-04T07:37:02.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/34 file after the snapshot 2024-03-04T07:37:02.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/35 file after the snapshot 2024-03-04T07:37:02.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/36 file after the snapshot 2024-03-04T07:37:02.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/37 file after the snapshot 2024-03-04T07:37:02.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/38 file after the snapshot 2024-03-04T07:37:02.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 140/39 file after the snapshot 2024-03-04T07:37:02.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/.snap/snap-subdir-test 2024-03-04T07:37:03.145 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/30 file after the snapshot 2024-03-04T07:37:03.148 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/31 file after the snapshot 2024-03-04T07:37:03.148 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/32 file after the snapshot 2024-03-04T07:37:03.148 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/33 file after the snapshot 2024-03-04T07:37:03.148 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/34 file after the snapshot 2024-03-04T07:37:03.148 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/35 file after the snapshot 2024-03-04T07:37:03.148 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/36 file after the snapshot 2024-03-04T07:37:03.148 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/37 file after the snapshot 2024-03-04T07:37:03.149 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/38 file after the snapshot 2024-03-04T07:37:03.149 INFO:tasks.workunit.client.0.smithi113.stdout: create 141/39 file after the snapshot 2024-03-04T07:37:03.149 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/.snap/snap-subdir-test 2024-03-04T07:37:03.210 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/30 file after the snapshot 2024-03-04T07:37:03.215 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/31 file after the snapshot 2024-03-04T07:37:03.215 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/32 file after the snapshot 2024-03-04T07:37:03.215 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/33 file after the snapshot 2024-03-04T07:37:03.216 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/34 file after the snapshot 2024-03-04T07:37:03.216 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/35 file after the snapshot 2024-03-04T07:37:03.216 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/36 file after the snapshot 2024-03-04T07:37:03.216 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/37 file after the snapshot 2024-03-04T07:37:03.216 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/38 file after the snapshot 2024-03-04T07:37:03.216 INFO:tasks.workunit.client.0.smithi113.stdout: create 142/39 file after the snapshot 2024-03-04T07:37:03.216 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/.snap/snap-subdir-test 2024-03-04T07:37:03.276 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/30 file after the snapshot 2024-03-04T07:37:03.282 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/31 file after the snapshot 2024-03-04T07:37:03.282 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/32 file after the snapshot 2024-03-04T07:37:03.282 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/33 file after the snapshot 2024-03-04T07:37:03.282 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/34 file after the snapshot 2024-03-04T07:37:03.282 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/35 file after the snapshot 2024-03-04T07:37:03.282 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/36 file after the snapshot 2024-03-04T07:37:03.282 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/37 file after the snapshot 2024-03-04T07:37:03.282 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/38 file after the snapshot 2024-03-04T07:37:03.282 INFO:tasks.workunit.client.0.smithi113.stdout: create 143/39 file after the snapshot 2024-03-04T07:37:03.283 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/.snap/snap-subdir-test 2024-03-04T07:37:03.338 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/30 file after the snapshot 2024-03-04T07:37:03.342 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/31 file after the snapshot 2024-03-04T07:37:03.342 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/32 file after the snapshot 2024-03-04T07:37:03.342 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/33 file after the snapshot 2024-03-04T07:37:03.342 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/34 file after the snapshot 2024-03-04T07:37:03.342 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/35 file after the snapshot 2024-03-04T07:37:03.342 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/36 file after the snapshot 2024-03-04T07:37:03.342 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/37 file after the snapshot 2024-03-04T07:37:03.343 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/38 file after the snapshot 2024-03-04T07:37:03.343 INFO:tasks.workunit.client.0.smithi113.stdout: create 144/39 file after the snapshot 2024-03-04T07:37:03.343 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/.snap/snap-subdir-test 2024-03-04T07:37:03.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/30 file after the snapshot 2024-03-04T07:37:03.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/31 file after the snapshot 2024-03-04T07:37:03.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/32 file after the snapshot 2024-03-04T07:37:03.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/33 file after the snapshot 2024-03-04T07:37:03.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/34 file after the snapshot 2024-03-04T07:37:03.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/35 file after the snapshot 2024-03-04T07:37:03.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/36 file after the snapshot 2024-03-04T07:37:03.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/37 file after the snapshot 2024-03-04T07:37:03.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/38 file after the snapshot 2024-03-04T07:37:03.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 145/39 file after the snapshot 2024-03-04T07:37:03.412 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/.snap/snap-subdir-test 2024-03-04T07:37:03.465 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/30 file after the snapshot 2024-03-04T07:37:03.468 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/31 file after the snapshot 2024-03-04T07:37:03.468 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/32 file after the snapshot 2024-03-04T07:37:03.468 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/33 file after the snapshot 2024-03-04T07:37:03.468 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/34 file after the snapshot 2024-03-04T07:37:03.468 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/35 file after the snapshot 2024-03-04T07:37:03.468 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/36 file after the snapshot 2024-03-04T07:37:03.468 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/37 file after the snapshot 2024-03-04T07:37:03.468 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/38 file after the snapshot 2024-03-04T07:37:03.469 INFO:tasks.workunit.client.0.smithi113.stdout: create 146/39 file after the snapshot 2024-03-04T07:37:03.469 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/.snap/snap-subdir-test 2024-03-04T07:37:04.132 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/30 file after the snapshot 2024-03-04T07:37:04.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/31 file after the snapshot 2024-03-04T07:37:04.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/32 file after the snapshot 2024-03-04T07:37:04.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/33 file after the snapshot 2024-03-04T07:37:04.137 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/34 file after the snapshot 2024-03-04T07:37:04.137 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/35 file after the snapshot 2024-03-04T07:37:04.137 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/36 file after the snapshot 2024-03-04T07:37:04.137 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/37 file after the snapshot 2024-03-04T07:37:04.137 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/38 file after the snapshot 2024-03-04T07:37:04.137 INFO:tasks.workunit.client.0.smithi113.stdout: create 147/39 file after the snapshot 2024-03-04T07:37:04.137 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/.snap/snap-subdir-test 2024-03-04T07:37:04.201 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/30 file after the snapshot 2024-03-04T07:37:04.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/31 file after the snapshot 2024-03-04T07:37:04.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/32 file after the snapshot 2024-03-04T07:37:04.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/33 file after the snapshot 2024-03-04T07:37:04.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/34 file after the snapshot 2024-03-04T07:37:04.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/35 file after the snapshot 2024-03-04T07:37:04.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/36 file after the snapshot 2024-03-04T07:37:04.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/37 file after the snapshot 2024-03-04T07:37:04.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/38 file after the snapshot 2024-03-04T07:37:04.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 148/39 file after the snapshot 2024-03-04T07:37:04.207 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/.snap/snap-subdir-test 2024-03-04T07:37:04.269 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/30 file after the snapshot 2024-03-04T07:37:04.272 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/31 file after the snapshot 2024-03-04T07:37:04.272 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/32 file after the snapshot 2024-03-04T07:37:04.272 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/33 file after the snapshot 2024-03-04T07:37:04.272 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/34 file after the snapshot 2024-03-04T07:37:04.272 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/35 file after the snapshot 2024-03-04T07:37:04.273 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/36 file after the snapshot 2024-03-04T07:37:04.273 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/37 file after the snapshot 2024-03-04T07:37:04.273 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/38 file after the snapshot 2024-03-04T07:37:04.273 INFO:tasks.workunit.client.0.smithi113.stdout: create 149/39 file after the snapshot 2024-03-04T07:37:04.273 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/.snap/snap-subdir-test 2024-03-04T07:37:04.330 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/30 file after the snapshot 2024-03-04T07:37:04.352 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/31 file after the snapshot 2024-03-04T07:37:04.355 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/32 file after the snapshot 2024-03-04T07:37:04.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/33 file after the snapshot 2024-03-04T07:37:04.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/34 file after the snapshot 2024-03-04T07:37:04.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/35 file after the snapshot 2024-03-04T07:37:04.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/36 file after the snapshot 2024-03-04T07:37:04.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/37 file after the snapshot 2024-03-04T07:37:04.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/38 file after the snapshot 2024-03-04T07:37:04.357 INFO:tasks.workunit.client.0.smithi113.stdout: create 150/39 file after the snapshot 2024-03-04T07:37:04.357 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/.snap/snap-subdir-test 2024-03-04T07:37:04.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/30 file after the snapshot 2024-03-04T07:37:04.896 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/31 file after the snapshot 2024-03-04T07:37:04.896 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/32 file after the snapshot 2024-03-04T07:37:04.896 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/33 file after the snapshot 2024-03-04T07:37:04.896 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/34 file after the snapshot 2024-03-04T07:37:04.896 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/35 file after the snapshot 2024-03-04T07:37:04.896 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/36 file after the snapshot 2024-03-04T07:37:04.896 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/37 file after the snapshot 2024-03-04T07:37:04.896 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/38 file after the snapshot 2024-03-04T07:37:04.896 INFO:tasks.workunit.client.0.smithi113.stdout: create 151/39 file after the snapshot 2024-03-04T07:37:04.896 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/.snap/snap-subdir-test 2024-03-04T07:37:05.124 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/30 file after the snapshot 2024-03-04T07:37:05.129 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/31 file after the snapshot 2024-03-04T07:37:05.129 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/32 file after the snapshot 2024-03-04T07:37:05.129 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/33 file after the snapshot 2024-03-04T07:37:05.129 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/34 file after the snapshot 2024-03-04T07:37:05.130 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/35 file after the snapshot 2024-03-04T07:37:05.130 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/36 file after the snapshot 2024-03-04T07:37:05.130 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/37 file after the snapshot 2024-03-04T07:37:05.130 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/38 file after the snapshot 2024-03-04T07:37:05.130 INFO:tasks.workunit.client.0.smithi113.stdout: create 152/39 file after the snapshot 2024-03-04T07:37:05.130 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/.snap/snap-subdir-test 2024-03-04T07:37:05.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/30 file after the snapshot 2024-03-04T07:37:05.189 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/31 file after the snapshot 2024-03-04T07:37:05.189 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/32 file after the snapshot 2024-03-04T07:37:05.189 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/33 file after the snapshot 2024-03-04T07:37:05.189 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/34 file after the snapshot 2024-03-04T07:37:05.189 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/35 file after the snapshot 2024-03-04T07:37:05.189 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/36 file after the snapshot 2024-03-04T07:37:05.190 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/37 file after the snapshot 2024-03-04T07:37:05.190 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/38 file after the snapshot 2024-03-04T07:37:05.190 INFO:tasks.workunit.client.0.smithi113.stdout: create 153/39 file after the snapshot 2024-03-04T07:37:05.190 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/.snap/snap-subdir-test 2024-03-04T07:37:05.250 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/30 file after the snapshot 2024-03-04T07:37:05.253 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/31 file after the snapshot 2024-03-04T07:37:05.253 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/32 file after the snapshot 2024-03-04T07:37:05.253 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/33 file after the snapshot 2024-03-04T07:37:05.254 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/34 file after the snapshot 2024-03-04T07:37:05.254 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/35 file after the snapshot 2024-03-04T07:37:05.254 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/36 file after the snapshot 2024-03-04T07:37:05.254 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/37 file after the snapshot 2024-03-04T07:37:05.254 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/38 file after the snapshot 2024-03-04T07:37:05.254 INFO:tasks.workunit.client.0.smithi113.stdout: create 154/39 file after the snapshot 2024-03-04T07:37:05.254 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/.snap/snap-subdir-test 2024-03-04T07:37:05.312 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/30 file after the snapshot 2024-03-04T07:37:05.317 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/31 file after the snapshot 2024-03-04T07:37:05.317 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/32 file after the snapshot 2024-03-04T07:37:05.317 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/33 file after the snapshot 2024-03-04T07:37:05.317 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/34 file after the snapshot 2024-03-04T07:37:05.317 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/35 file after the snapshot 2024-03-04T07:37:05.317 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/36 file after the snapshot 2024-03-04T07:37:05.318 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/37 file after the snapshot 2024-03-04T07:37:05.318 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/38 file after the snapshot 2024-03-04T07:37:05.318 INFO:tasks.workunit.client.0.smithi113.stdout: create 155/39 file after the snapshot 2024-03-04T07:37:05.318 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/.snap/snap-subdir-test 2024-03-04T07:37:05.378 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/30 file after the snapshot 2024-03-04T07:37:05.382 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/31 file after the snapshot 2024-03-04T07:37:05.382 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/32 file after the snapshot 2024-03-04T07:37:05.382 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/33 file after the snapshot 2024-03-04T07:37:05.382 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/34 file after the snapshot 2024-03-04T07:37:05.382 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/35 file after the snapshot 2024-03-04T07:37:05.382 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/36 file after the snapshot 2024-03-04T07:37:05.382 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/37 file after the snapshot 2024-03-04T07:37:05.382 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/38 file after the snapshot 2024-03-04T07:37:05.383 INFO:tasks.workunit.client.0.smithi113.stdout: create 156/39 file after the snapshot 2024-03-04T07:37:05.383 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/.snap/snap-subdir-test 2024-03-04T07:37:05.449 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/30 file after the snapshot 2024-03-04T07:37:05.454 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/31 file after the snapshot 2024-03-04T07:37:05.454 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/32 file after the snapshot 2024-03-04T07:37:05.454 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/33 file after the snapshot 2024-03-04T07:37:05.454 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/34 file after the snapshot 2024-03-04T07:37:05.454 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/35 file after the snapshot 2024-03-04T07:37:05.454 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/36 file after the snapshot 2024-03-04T07:37:05.454 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/37 file after the snapshot 2024-03-04T07:37:05.454 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/38 file after the snapshot 2024-03-04T07:37:05.454 INFO:tasks.workunit.client.0.smithi113.stdout: create 157/39 file after the snapshot 2024-03-04T07:37:05.455 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/.snap/snap-subdir-test 2024-03-04T07:37:06.151 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/30 file after the snapshot 2024-03-04T07:37:06.406 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/31 file after the snapshot 2024-03-04T07:37:06.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/32 file after the snapshot 2024-03-04T07:37:06.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/33 file after the snapshot 2024-03-04T07:37:06.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/34 file after the snapshot 2024-03-04T07:37:06.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/35 file after the snapshot 2024-03-04T07:37:06.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/36 file after the snapshot 2024-03-04T07:37:06.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/37 file after the snapshot 2024-03-04T07:37:06.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/38 file after the snapshot 2024-03-04T07:37:06.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 158/39 file after the snapshot 2024-03-04T07:37:06.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/.snap/snap-subdir-test 2024-03-04T07:37:06.469 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/30 file after the snapshot 2024-03-04T07:37:06.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/31 file after the snapshot 2024-03-04T07:37:06.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/32 file after the snapshot 2024-03-04T07:37:06.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/33 file after the snapshot 2024-03-04T07:37:06.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/34 file after the snapshot 2024-03-04T07:37:06.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/35 file after the snapshot 2024-03-04T07:37:06.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/36 file after the snapshot 2024-03-04T07:37:06.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/37 file after the snapshot 2024-03-04T07:37:06.474 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/38 file after the snapshot 2024-03-04T07:37:06.474 INFO:tasks.workunit.client.0.smithi113.stdout: create 159/39 file after the snapshot 2024-03-04T07:37:06.474 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/.snap/snap-subdir-test 2024-03-04T07:37:07.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/30 file after the snapshot 2024-03-04T07:37:07.180 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/31 file after the snapshot 2024-03-04T07:37:07.180 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/32 file after the snapshot 2024-03-04T07:37:07.181 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/33 file after the snapshot 2024-03-04T07:37:07.181 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/34 file after the snapshot 2024-03-04T07:37:07.181 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/35 file after the snapshot 2024-03-04T07:37:07.181 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/36 file after the snapshot 2024-03-04T07:37:07.181 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/37 file after the snapshot 2024-03-04T07:37:07.181 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/38 file after the snapshot 2024-03-04T07:37:07.181 INFO:tasks.workunit.client.0.smithi113.stdout: create 160/39 file after the snapshot 2024-03-04T07:37:07.181 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/.snap/snap-subdir-test 2024-03-04T07:37:07.246 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/30 file after the snapshot 2024-03-04T07:37:07.250 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/31 file after the snapshot 2024-03-04T07:37:07.250 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/32 file after the snapshot 2024-03-04T07:37:07.250 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/33 file after the snapshot 2024-03-04T07:37:07.251 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/34 file after the snapshot 2024-03-04T07:37:07.251 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/35 file after the snapshot 2024-03-04T07:37:07.252 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/36 file after the snapshot 2024-03-04T07:37:07.252 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/37 file after the snapshot 2024-03-04T07:37:07.252 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/38 file after the snapshot 2024-03-04T07:37:07.252 INFO:tasks.workunit.client.0.smithi113.stdout: create 161/39 file after the snapshot 2024-03-04T07:37:07.252 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/.snap/snap-subdir-test 2024-03-04T07:37:07.309 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/30 file after the snapshot 2024-03-04T07:37:07.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/31 file after the snapshot 2024-03-04T07:37:07.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/32 file after the snapshot 2024-03-04T07:37:07.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/33 file after the snapshot 2024-03-04T07:37:07.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/34 file after the snapshot 2024-03-04T07:37:07.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/35 file after the snapshot 2024-03-04T07:37:07.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/36 file after the snapshot 2024-03-04T07:37:07.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/37 file after the snapshot 2024-03-04T07:37:07.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/38 file after the snapshot 2024-03-04T07:37:07.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 162/39 file after the snapshot 2024-03-04T07:37:07.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/.snap/snap-subdir-test 2024-03-04T07:37:07.996 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/30 file after the snapshot 2024-03-04T07:37:08.196 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/31 file after the snapshot 2024-03-04T07:37:08.196 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/32 file after the snapshot 2024-03-04T07:37:08.196 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/33 file after the snapshot 2024-03-04T07:37:08.196 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/34 file after the snapshot 2024-03-04T07:37:08.196 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/35 file after the snapshot 2024-03-04T07:37:08.197 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/36 file after the snapshot 2024-03-04T07:37:08.197 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/37 file after the snapshot 2024-03-04T07:37:08.197 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/38 file after the snapshot 2024-03-04T07:37:08.197 INFO:tasks.workunit.client.0.smithi113.stdout: create 163/39 file after the snapshot 2024-03-04T07:37:08.197 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/.snap/snap-subdir-test 2024-03-04T07:37:09.407 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/30 file after the snapshot 2024-03-04T07:37:09.412 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/31 file after the snapshot 2024-03-04T07:37:09.412 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/32 file after the snapshot 2024-03-04T07:37:09.412 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/33 file after the snapshot 2024-03-04T07:37:09.412 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/34 file after the snapshot 2024-03-04T07:37:09.413 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/35 file after the snapshot 2024-03-04T07:37:09.413 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/36 file after the snapshot 2024-03-04T07:37:09.413 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/37 file after the snapshot 2024-03-04T07:37:09.413 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/38 file after the snapshot 2024-03-04T07:37:09.413 INFO:tasks.workunit.client.0.smithi113.stdout: create 164/39 file after the snapshot 2024-03-04T07:37:09.413 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/.snap/snap-subdir-test 2024-03-04T07:37:10.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/30 file after the snapshot 2024-03-04T07:37:10.120 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/31 file after the snapshot 2024-03-04T07:37:10.120 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/32 file after the snapshot 2024-03-04T07:37:10.120 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/33 file after the snapshot 2024-03-04T07:37:10.121 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/34 file after the snapshot 2024-03-04T07:37:10.121 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/35 file after the snapshot 2024-03-04T07:37:10.121 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/36 file after the snapshot 2024-03-04T07:37:10.121 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/37 file after the snapshot 2024-03-04T07:37:10.121 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/38 file after the snapshot 2024-03-04T07:37:10.121 INFO:tasks.workunit.client.0.smithi113.stdout: create 165/39 file after the snapshot 2024-03-04T07:37:10.121 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/.snap/snap-subdir-test 2024-03-04T07:37:11.107 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/30 file after the snapshot 2024-03-04T07:37:11.110 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/31 file after the snapshot 2024-03-04T07:37:11.110 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/32 file after the snapshot 2024-03-04T07:37:11.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/33 file after the snapshot 2024-03-04T07:37:11.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/34 file after the snapshot 2024-03-04T07:37:11.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/35 file after the snapshot 2024-03-04T07:37:11.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/36 file after the snapshot 2024-03-04T07:37:11.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/37 file after the snapshot 2024-03-04T07:37:11.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/38 file after the snapshot 2024-03-04T07:37:11.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 166/39 file after the snapshot 2024-03-04T07:37:11.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/.snap/snap-subdir-test 2024-03-04T07:37:11.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/30 file after the snapshot 2024-03-04T07:37:11.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/31 file after the snapshot 2024-03-04T07:37:11.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/32 file after the snapshot 2024-03-04T07:37:11.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/33 file after the snapshot 2024-03-04T07:37:11.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/34 file after the snapshot 2024-03-04T07:37:11.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/35 file after the snapshot 2024-03-04T07:37:11.187 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/36 file after the snapshot 2024-03-04T07:37:11.188 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/37 file after the snapshot 2024-03-04T07:37:11.188 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/38 file after the snapshot 2024-03-04T07:37:11.188 INFO:tasks.workunit.client.0.smithi113.stdout: create 167/39 file after the snapshot 2024-03-04T07:37:11.188 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/.snap/snap-subdir-test 2024-03-04T07:37:11.252 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/30 file after the snapshot 2024-03-04T07:37:11.255 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/31 file after the snapshot 2024-03-04T07:37:11.255 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/32 file after the snapshot 2024-03-04T07:37:11.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/33 file after the snapshot 2024-03-04T07:37:11.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/34 file after the snapshot 2024-03-04T07:37:11.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/35 file after the snapshot 2024-03-04T07:37:11.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/36 file after the snapshot 2024-03-04T07:37:11.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/37 file after the snapshot 2024-03-04T07:37:11.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/38 file after the snapshot 2024-03-04T07:37:11.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 168/39 file after the snapshot 2024-03-04T07:37:11.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/.snap/snap-subdir-test 2024-03-04T07:37:11.318 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/30 file after the snapshot 2024-03-04T07:37:11.323 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/31 file after the snapshot 2024-03-04T07:37:11.323 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/32 file after the snapshot 2024-03-04T07:37:11.324 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/33 file after the snapshot 2024-03-04T07:37:11.324 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/34 file after the snapshot 2024-03-04T07:37:11.324 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/35 file after the snapshot 2024-03-04T07:37:11.324 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/36 file after the snapshot 2024-03-04T07:37:11.324 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/37 file after the snapshot 2024-03-04T07:37:11.324 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/38 file after the snapshot 2024-03-04T07:37:11.324 INFO:tasks.workunit.client.0.smithi113.stdout: create 169/39 file after the snapshot 2024-03-04T07:37:11.324 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/.snap/snap-subdir-test 2024-03-04T07:37:11.384 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/30 file after the snapshot 2024-03-04T07:37:11.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/31 file after the snapshot 2024-03-04T07:37:11.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/32 file after the snapshot 2024-03-04T07:37:11.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/33 file after the snapshot 2024-03-04T07:37:11.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/34 file after the snapshot 2024-03-04T07:37:11.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/35 file after the snapshot 2024-03-04T07:37:11.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/36 file after the snapshot 2024-03-04T07:37:11.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/37 file after the snapshot 2024-03-04T07:37:11.390 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/38 file after the snapshot 2024-03-04T07:37:11.390 INFO:tasks.workunit.client.0.smithi113.stdout: create 170/39 file after the snapshot 2024-03-04T07:37:11.390 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/.snap/snap-subdir-test 2024-03-04T07:37:11.445 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/30 file after the snapshot 2024-03-04T07:37:11.449 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/31 file after the snapshot 2024-03-04T07:37:11.449 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/32 file after the snapshot 2024-03-04T07:37:11.449 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/33 file after the snapshot 2024-03-04T07:37:11.449 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/34 file after the snapshot 2024-03-04T07:37:11.450 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/35 file after the snapshot 2024-03-04T07:37:11.450 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/36 file after the snapshot 2024-03-04T07:37:11.450 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/37 file after the snapshot 2024-03-04T07:37:11.450 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/38 file after the snapshot 2024-03-04T07:37:11.450 INFO:tasks.workunit.client.0.smithi113.stdout: create 171/39 file after the snapshot 2024-03-04T07:37:11.450 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/.snap/snap-subdir-test 2024-03-04T07:37:12.299 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/30 file after the snapshot 2024-03-04T07:37:12.308 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/31 file after the snapshot 2024-03-04T07:37:12.308 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/32 file after the snapshot 2024-03-04T07:37:12.308 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/33 file after the snapshot 2024-03-04T07:37:12.308 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/34 file after the snapshot 2024-03-04T07:37:12.309 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/35 file after the snapshot 2024-03-04T07:37:12.309 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/36 file after the snapshot 2024-03-04T07:37:12.309 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/37 file after the snapshot 2024-03-04T07:37:12.309 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/38 file after the snapshot 2024-03-04T07:37:12.309 INFO:tasks.workunit.client.0.smithi113.stdout: create 172/39 file after the snapshot 2024-03-04T07:37:12.309 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/.snap/snap-subdir-test 2024-03-04T07:37:12.381 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/30 file after the snapshot 2024-03-04T07:37:12.386 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/31 file after the snapshot 2024-03-04T07:37:12.386 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/32 file after the snapshot 2024-03-04T07:37:12.386 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/33 file after the snapshot 2024-03-04T07:37:12.386 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/34 file after the snapshot 2024-03-04T07:37:12.386 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/35 file after the snapshot 2024-03-04T07:37:12.386 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/36 file after the snapshot 2024-03-04T07:37:12.386 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/37 file after the snapshot 2024-03-04T07:37:12.387 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/38 file after the snapshot 2024-03-04T07:37:12.387 INFO:tasks.workunit.client.0.smithi113.stdout: create 173/39 file after the snapshot 2024-03-04T07:37:12.387 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/.snap/snap-subdir-test 2024-03-04T07:37:12.437 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/30 file after the snapshot 2024-03-04T07:37:12.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/31 file after the snapshot 2024-03-04T07:37:12.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/32 file after the snapshot 2024-03-04T07:37:12.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/33 file after the snapshot 2024-03-04T07:37:12.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/34 file after the snapshot 2024-03-04T07:37:12.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/35 file after the snapshot 2024-03-04T07:37:12.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/36 file after the snapshot 2024-03-04T07:37:12.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/37 file after the snapshot 2024-03-04T07:37:12.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/38 file after the snapshot 2024-03-04T07:37:12.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 174/39 file after the snapshot 2024-03-04T07:37:12.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/.snap/snap-subdir-test 2024-03-04T07:37:12.478 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/30 file after the snapshot 2024-03-04T07:37:12.484 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/31 file after the snapshot 2024-03-04T07:37:12.484 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/32 file after the snapshot 2024-03-04T07:37:12.484 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/33 file after the snapshot 2024-03-04T07:37:12.484 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/34 file after the snapshot 2024-03-04T07:37:12.484 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/35 file after the snapshot 2024-03-04T07:37:12.484 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/36 file after the snapshot 2024-03-04T07:37:12.484 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/37 file after the snapshot 2024-03-04T07:37:12.485 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/38 file after the snapshot 2024-03-04T07:37:12.485 INFO:tasks.workunit.client.0.smithi113.stdout: create 175/39 file after the snapshot 2024-03-04T07:37:12.485 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/.snap/snap-subdir-test 2024-03-04T07:37:12.538 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/30 file after the snapshot 2024-03-04T07:37:12.541 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/31 file after the snapshot 2024-03-04T07:37:12.541 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/32 file after the snapshot 2024-03-04T07:37:12.541 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/33 file after the snapshot 2024-03-04T07:37:12.541 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/34 file after the snapshot 2024-03-04T07:37:12.541 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/35 file after the snapshot 2024-03-04T07:37:12.541 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/36 file after the snapshot 2024-03-04T07:37:12.542 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/37 file after the snapshot 2024-03-04T07:37:12.542 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/38 file after the snapshot 2024-03-04T07:37:12.542 INFO:tasks.workunit.client.0.smithi113.stdout: create 176/39 file after the snapshot 2024-03-04T07:37:12.542 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/.snap/snap-subdir-test 2024-03-04T07:37:13.092 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/30 file after the snapshot 2024-03-04T07:37:13.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/31 file after the snapshot 2024-03-04T07:37:13.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/32 file after the snapshot 2024-03-04T07:37:13.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/33 file after the snapshot 2024-03-04T07:37:13.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/34 file after the snapshot 2024-03-04T07:37:13.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/35 file after the snapshot 2024-03-04T07:37:13.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/36 file after the snapshot 2024-03-04T07:37:13.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/37 file after the snapshot 2024-03-04T07:37:13.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/38 file after the snapshot 2024-03-04T07:37:13.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 177/39 file after the snapshot 2024-03-04T07:37:13.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/.snap/snap-subdir-test 2024-03-04T07:37:13.288 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/30 file after the snapshot 2024-03-04T07:37:13.292 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/31 file after the snapshot 2024-03-04T07:37:13.292 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/32 file after the snapshot 2024-03-04T07:37:13.292 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/33 file after the snapshot 2024-03-04T07:37:13.292 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/34 file after the snapshot 2024-03-04T07:37:13.292 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/35 file after the snapshot 2024-03-04T07:37:13.292 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/36 file after the snapshot 2024-03-04T07:37:13.292 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/37 file after the snapshot 2024-03-04T07:37:13.292 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/38 file after the snapshot 2024-03-04T07:37:13.292 INFO:tasks.workunit.client.0.smithi113.stdout: create 178/39 file after the snapshot 2024-03-04T07:37:13.292 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/.snap/snap-subdir-test 2024-03-04T07:37:13.336 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/30 file after the snapshot 2024-03-04T07:37:13.339 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/31 file after the snapshot 2024-03-04T07:37:13.339 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/32 file after the snapshot 2024-03-04T07:37:13.339 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/33 file after the snapshot 2024-03-04T07:37:13.339 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/34 file after the snapshot 2024-03-04T07:37:13.339 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/35 file after the snapshot 2024-03-04T07:37:13.339 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/36 file after the snapshot 2024-03-04T07:37:13.339 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/37 file after the snapshot 2024-03-04T07:37:13.340 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/38 file after the snapshot 2024-03-04T07:37:13.340 INFO:tasks.workunit.client.0.smithi113.stdout: create 179/39 file after the snapshot 2024-03-04T07:37:13.340 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/.snap/snap-subdir-test 2024-03-04T07:37:13.385 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/30 file after the snapshot 2024-03-04T07:37:13.388 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/31 file after the snapshot 2024-03-04T07:37:13.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/32 file after the snapshot 2024-03-04T07:37:13.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/33 file after the snapshot 2024-03-04T07:37:13.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/34 file after the snapshot 2024-03-04T07:37:13.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/35 file after the snapshot 2024-03-04T07:37:13.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/36 file after the snapshot 2024-03-04T07:37:13.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/37 file after the snapshot 2024-03-04T07:37:13.390 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/38 file after the snapshot 2024-03-04T07:37:13.390 INFO:tasks.workunit.client.0.smithi113.stdout: create 180/39 file after the snapshot 2024-03-04T07:37:13.390 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/.snap/snap-subdir-test 2024-03-04T07:37:13.435 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/30 file after the snapshot 2024-03-04T07:37:13.437 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/31 file after the snapshot 2024-03-04T07:37:13.437 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/32 file after the snapshot 2024-03-04T07:37:13.437 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/33 file after the snapshot 2024-03-04T07:37:13.437 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/34 file after the snapshot 2024-03-04T07:37:13.437 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/35 file after the snapshot 2024-03-04T07:37:13.438 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/36 file after the snapshot 2024-03-04T07:37:13.438 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/37 file after the snapshot 2024-03-04T07:37:13.438 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/38 file after the snapshot 2024-03-04T07:37:13.439 INFO:tasks.workunit.client.0.smithi113.stdout: create 181/39 file after the snapshot 2024-03-04T07:37:13.439 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/.snap/snap-subdir-test 2024-03-04T07:37:13.471 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/30 file after the snapshot 2024-03-04T07:37:13.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/31 file after the snapshot 2024-03-04T07:37:13.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/32 file after the snapshot 2024-03-04T07:37:13.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/33 file after the snapshot 2024-03-04T07:37:13.473 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/34 file after the snapshot 2024-03-04T07:37:13.474 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/35 file after the snapshot 2024-03-04T07:37:13.474 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/36 file after the snapshot 2024-03-04T07:37:13.474 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/37 file after the snapshot 2024-03-04T07:37:13.475 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/38 file after the snapshot 2024-03-04T07:37:13.475 INFO:tasks.workunit.client.0.smithi113.stdout: create 182/39 file after the snapshot 2024-03-04T07:37:13.475 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/.snap/snap-subdir-test 2024-03-04T07:37:13.513 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/30 file after the snapshot 2024-03-04T07:37:13.516 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/31 file after the snapshot 2024-03-04T07:37:13.516 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/32 file after the snapshot 2024-03-04T07:37:13.516 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/33 file after the snapshot 2024-03-04T07:37:13.516 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/34 file after the snapshot 2024-03-04T07:37:13.517 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/35 file after the snapshot 2024-03-04T07:37:13.517 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/36 file after the snapshot 2024-03-04T07:37:13.517 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/37 file after the snapshot 2024-03-04T07:37:13.517 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/38 file after the snapshot 2024-03-04T07:37:13.517 INFO:tasks.workunit.client.0.smithi113.stdout: create 183/39 file after the snapshot 2024-03-04T07:37:13.517 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/.snap/snap-subdir-test 2024-03-04T07:37:13.556 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/30 file after the snapshot 2024-03-04T07:37:13.559 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/31 file after the snapshot 2024-03-04T07:37:13.559 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/32 file after the snapshot 2024-03-04T07:37:13.559 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/33 file after the snapshot 2024-03-04T07:37:13.560 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/34 file after the snapshot 2024-03-04T07:37:13.560 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/35 file after the snapshot 2024-03-04T07:37:13.560 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/36 file after the snapshot 2024-03-04T07:37:13.560 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/37 file after the snapshot 2024-03-04T07:37:13.560 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/38 file after the snapshot 2024-03-04T07:37:13.560 INFO:tasks.workunit.client.0.smithi113.stdout: create 184/39 file after the snapshot 2024-03-04T07:37:13.560 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/.snap/snap-subdir-test 2024-03-04T07:37:13.603 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/30 file after the snapshot 2024-03-04T07:37:13.605 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/31 file after the snapshot 2024-03-04T07:37:13.605 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/32 file after the snapshot 2024-03-04T07:37:13.605 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/33 file after the snapshot 2024-03-04T07:37:13.605 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/34 file after the snapshot 2024-03-04T07:37:13.605 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/35 file after the snapshot 2024-03-04T07:37:13.606 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/36 file after the snapshot 2024-03-04T07:37:13.606 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/37 file after the snapshot 2024-03-04T07:37:13.606 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/38 file after the snapshot 2024-03-04T07:37:13.606 INFO:tasks.workunit.client.0.smithi113.stdout: create 185/39 file after the snapshot 2024-03-04T07:37:13.606 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/.snap/snap-subdir-test 2024-03-04T07:37:13.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/30 file after the snapshot 2024-03-04T07:37:13.650 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/31 file after the snapshot 2024-03-04T07:37:13.650 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/32 file after the snapshot 2024-03-04T07:37:13.650 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/33 file after the snapshot 2024-03-04T07:37:13.650 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/34 file after the snapshot 2024-03-04T07:37:13.650 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/35 file after the snapshot 2024-03-04T07:37:13.650 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/36 file after the snapshot 2024-03-04T07:37:13.651 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/37 file after the snapshot 2024-03-04T07:37:13.651 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/38 file after the snapshot 2024-03-04T07:37:13.651 INFO:tasks.workunit.client.0.smithi113.stdout: create 186/39 file after the snapshot 2024-03-04T07:37:13.651 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/.snap/snap-subdir-test 2024-03-04T07:37:13.705 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/30 file after the snapshot 2024-03-04T07:37:13.708 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/31 file after the snapshot 2024-03-04T07:37:13.709 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/32 file after the snapshot 2024-03-04T07:37:13.709 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/33 file after the snapshot 2024-03-04T07:37:13.709 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/34 file after the snapshot 2024-03-04T07:37:13.709 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/35 file after the snapshot 2024-03-04T07:37:13.709 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/36 file after the snapshot 2024-03-04T07:37:13.709 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/37 file after the snapshot 2024-03-04T07:37:13.709 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/38 file after the snapshot 2024-03-04T07:37:13.709 INFO:tasks.workunit.client.0.smithi113.stdout: create 187/39 file after the snapshot 2024-03-04T07:37:13.709 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/.snap/snap-subdir-test 2024-03-04T07:37:13.761 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/30 file after the snapshot 2024-03-04T07:37:13.764 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/31 file after the snapshot 2024-03-04T07:37:13.764 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/32 file after the snapshot 2024-03-04T07:37:13.764 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/33 file after the snapshot 2024-03-04T07:37:13.765 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/34 file after the snapshot 2024-03-04T07:37:13.765 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/35 file after the snapshot 2024-03-04T07:37:13.765 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/36 file after the snapshot 2024-03-04T07:37:13.765 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/37 file after the snapshot 2024-03-04T07:37:13.765 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/38 file after the snapshot 2024-03-04T07:37:13.765 INFO:tasks.workunit.client.0.smithi113.stdout: create 188/39 file after the snapshot 2024-03-04T07:37:13.765 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/.snap/snap-subdir-test 2024-03-04T07:37:13.809 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/30 file after the snapshot 2024-03-04T07:37:13.813 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/31 file after the snapshot 2024-03-04T07:37:13.813 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/32 file after the snapshot 2024-03-04T07:37:13.813 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/33 file after the snapshot 2024-03-04T07:37:13.813 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/34 file after the snapshot 2024-03-04T07:37:13.813 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/35 file after the snapshot 2024-03-04T07:37:13.813 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/36 file after the snapshot 2024-03-04T07:37:13.813 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/37 file after the snapshot 2024-03-04T07:37:13.813 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/38 file after the snapshot 2024-03-04T07:37:13.813 INFO:tasks.workunit.client.0.smithi113.stdout: create 189/39 file after the snapshot 2024-03-04T07:37:13.813 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/.snap/snap-subdir-test 2024-03-04T07:37:13.855 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/30 file after the snapshot 2024-03-04T07:37:13.858 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/31 file after the snapshot 2024-03-04T07:37:13.858 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/32 file after the snapshot 2024-03-04T07:37:13.858 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/33 file after the snapshot 2024-03-04T07:37:13.858 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/34 file after the snapshot 2024-03-04T07:37:13.858 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/35 file after the snapshot 2024-03-04T07:37:13.858 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/36 file after the snapshot 2024-03-04T07:37:13.859 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/37 file after the snapshot 2024-03-04T07:37:13.859 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/38 file after the snapshot 2024-03-04T07:37:13.859 INFO:tasks.workunit.client.0.smithi113.stdout: create 190/39 file after the snapshot 2024-03-04T07:37:13.859 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/.snap/snap-subdir-test 2024-03-04T07:37:13.889 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/30 file after the snapshot 2024-03-04T07:37:13.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/31 file after the snapshot 2024-03-04T07:37:13.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/32 file after the snapshot 2024-03-04T07:37:13.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/33 file after the snapshot 2024-03-04T07:37:13.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/34 file after the snapshot 2024-03-04T07:37:13.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/35 file after the snapshot 2024-03-04T07:37:13.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/36 file after the snapshot 2024-03-04T07:37:13.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/37 file after the snapshot 2024-03-04T07:37:13.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/38 file after the snapshot 2024-03-04T07:37:13.892 INFO:tasks.workunit.client.0.smithi113.stdout: create 191/39 file after the snapshot 2024-03-04T07:37:13.892 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/.snap/snap-subdir-test 2024-03-04T07:37:13.925 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/30 file after the snapshot 2024-03-04T07:37:13.927 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/31 file after the snapshot 2024-03-04T07:37:13.927 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/32 file after the snapshot 2024-03-04T07:37:13.927 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/33 file after the snapshot 2024-03-04T07:37:13.927 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/34 file after the snapshot 2024-03-04T07:37:13.927 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/35 file after the snapshot 2024-03-04T07:37:13.928 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/36 file after the snapshot 2024-03-04T07:37:13.928 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/37 file after the snapshot 2024-03-04T07:37:13.928 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/38 file after the snapshot 2024-03-04T07:37:13.928 INFO:tasks.workunit.client.0.smithi113.stdout: create 192/39 file after the snapshot 2024-03-04T07:37:13.928 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/.snap/snap-subdir-test 2024-03-04T07:37:14.351 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/30 file after the snapshot 2024-03-04T07:37:14.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/31 file after the snapshot 2024-03-04T07:37:14.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/32 file after the snapshot 2024-03-04T07:37:14.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/33 file after the snapshot 2024-03-04T07:37:14.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/34 file after the snapshot 2024-03-04T07:37:14.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/35 file after the snapshot 2024-03-04T07:37:14.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/36 file after the snapshot 2024-03-04T07:37:14.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/37 file after the snapshot 2024-03-04T07:37:14.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/38 file after the snapshot 2024-03-04T07:37:14.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 193/39 file after the snapshot 2024-03-04T07:37:14.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/.snap/snap-subdir-test 2024-03-04T07:37:14.410 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/30 file after the snapshot 2024-03-04T07:37:14.415 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/31 file after the snapshot 2024-03-04T07:37:14.415 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/32 file after the snapshot 2024-03-04T07:37:14.415 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/33 file after the snapshot 2024-03-04T07:37:14.415 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/34 file after the snapshot 2024-03-04T07:37:14.415 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/35 file after the snapshot 2024-03-04T07:37:14.415 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/36 file after the snapshot 2024-03-04T07:37:14.415 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/37 file after the snapshot 2024-03-04T07:37:14.415 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/38 file after the snapshot 2024-03-04T07:37:14.416 INFO:tasks.workunit.client.0.smithi113.stdout: create 194/39 file after the snapshot 2024-03-04T07:37:14.416 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/.snap/snap-subdir-test 2024-03-04T07:37:15.029 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/30 file after the snapshot 2024-03-04T07:37:15.034 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/31 file after the snapshot 2024-03-04T07:37:15.034 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/32 file after the snapshot 2024-03-04T07:37:15.034 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/33 file after the snapshot 2024-03-04T07:37:15.034 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/34 file after the snapshot 2024-03-04T07:37:15.034 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/35 file after the snapshot 2024-03-04T07:37:15.034 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/36 file after the snapshot 2024-03-04T07:37:15.034 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/37 file after the snapshot 2024-03-04T07:37:15.034 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/38 file after the snapshot 2024-03-04T07:37:15.034 INFO:tasks.workunit.client.0.smithi113.stdout: create 195/39 file after the snapshot 2024-03-04T07:37:15.034 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/.snap/snap-subdir-test 2024-03-04T07:37:15.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/30 file after the snapshot 2024-03-04T07:37:15.116 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/31 file after the snapshot 2024-03-04T07:37:15.116 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/32 file after the snapshot 2024-03-04T07:37:15.116 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/33 file after the snapshot 2024-03-04T07:37:15.116 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/34 file after the snapshot 2024-03-04T07:37:15.116 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/35 file after the snapshot 2024-03-04T07:37:15.116 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/36 file after the snapshot 2024-03-04T07:37:15.116 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/37 file after the snapshot 2024-03-04T07:37:15.117 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/38 file after the snapshot 2024-03-04T07:37:15.117 INFO:tasks.workunit.client.0.smithi113.stdout: create 196/39 file after the snapshot 2024-03-04T07:37:15.117 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/.snap/snap-subdir-test 2024-03-04T07:37:15.172 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/30 file after the snapshot 2024-03-04T07:37:15.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/31 file after the snapshot 2024-03-04T07:37:15.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/32 file after the snapshot 2024-03-04T07:37:15.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/33 file after the snapshot 2024-03-04T07:37:15.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/34 file after the snapshot 2024-03-04T07:37:15.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/35 file after the snapshot 2024-03-04T07:37:15.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/36 file after the snapshot 2024-03-04T07:37:15.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/37 file after the snapshot 2024-03-04T07:37:15.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/38 file after the snapshot 2024-03-04T07:37:15.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 197/39 file after the snapshot 2024-03-04T07:37:15.178 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/.snap/snap-subdir-test 2024-03-04T07:37:15.234 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/30 file after the snapshot 2024-03-04T07:37:15.237 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/31 file after the snapshot 2024-03-04T07:37:15.237 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/32 file after the snapshot 2024-03-04T07:37:15.237 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/33 file after the snapshot 2024-03-04T07:37:15.237 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/34 file after the snapshot 2024-03-04T07:37:15.238 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/35 file after the snapshot 2024-03-04T07:37:15.238 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/36 file after the snapshot 2024-03-04T07:37:15.238 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/37 file after the snapshot 2024-03-04T07:37:15.238 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/38 file after the snapshot 2024-03-04T07:37:15.238 INFO:tasks.workunit.client.0.smithi113.stdout: create 198/39 file after the snapshot 2024-03-04T07:37:15.238 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/.snap/snap-subdir-test 2024-03-04T07:37:15.537 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/30 file after the snapshot 2024-03-04T07:37:15.540 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/31 file after the snapshot 2024-03-04T07:37:15.540 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/32 file after the snapshot 2024-03-04T07:37:15.540 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/33 file after the snapshot 2024-03-04T07:37:15.540 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/34 file after the snapshot 2024-03-04T07:37:15.540 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/35 file after the snapshot 2024-03-04T07:37:15.540 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/36 file after the snapshot 2024-03-04T07:37:15.541 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/37 file after the snapshot 2024-03-04T07:37:15.541 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/38 file after the snapshot 2024-03-04T07:37:15.541 INFO:tasks.workunit.client.0.smithi113.stdout: create 199/39 file after the snapshot 2024-03-04T07:37:15.541 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/.snap/snap-subdir-test 2024-03-04T07:37:15.607 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/30 file after the snapshot 2024-03-04T07:37:15.610 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/31 file after the snapshot 2024-03-04T07:37:15.610 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/32 file after the snapshot 2024-03-04T07:37:15.610 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/33 file after the snapshot 2024-03-04T07:37:15.610 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/34 file after the snapshot 2024-03-04T07:37:15.610 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/35 file after the snapshot 2024-03-04T07:37:15.610 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/36 file after the snapshot 2024-03-04T07:37:15.610 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/37 file after the snapshot 2024-03-04T07:37:15.610 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/38 file after the snapshot 2024-03-04T07:37:15.610 INFO:tasks.workunit.client.0.smithi113.stdout: create 200/39 file after the snapshot 2024-03-04T07:37:15.611 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/.snap/snap-subdir-test 2024-03-04T07:37:15.661 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/30 file after the snapshot 2024-03-04T07:37:15.663 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/31 file after the snapshot 2024-03-04T07:37:15.664 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/32 file after the snapshot 2024-03-04T07:37:15.664 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/33 file after the snapshot 2024-03-04T07:37:15.664 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/34 file after the snapshot 2024-03-04T07:37:15.664 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/35 file after the snapshot 2024-03-04T07:37:15.664 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/36 file after the snapshot 2024-03-04T07:37:15.664 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/37 file after the snapshot 2024-03-04T07:37:15.664 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/38 file after the snapshot 2024-03-04T07:37:15.664 INFO:tasks.workunit.client.0.smithi113.stdout: create 201/39 file after the snapshot 2024-03-04T07:37:15.665 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/.snap/snap-subdir-test 2024-03-04T07:37:15.707 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/30 file after the snapshot 2024-03-04T07:37:15.710 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/31 file after the snapshot 2024-03-04T07:37:15.710 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/32 file after the snapshot 2024-03-04T07:37:15.710 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/33 file after the snapshot 2024-03-04T07:37:15.710 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/34 file after the snapshot 2024-03-04T07:37:15.710 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/35 file after the snapshot 2024-03-04T07:37:15.710 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/36 file after the snapshot 2024-03-04T07:37:15.710 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/37 file after the snapshot 2024-03-04T07:37:15.711 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/38 file after the snapshot 2024-03-04T07:37:15.711 INFO:tasks.workunit.client.0.smithi113.stdout: create 202/39 file after the snapshot 2024-03-04T07:37:15.711 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/.snap/snap-subdir-test 2024-03-04T07:37:15.953 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/30 file after the snapshot 2024-03-04T07:37:15.956 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/31 file after the snapshot 2024-03-04T07:37:15.956 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/32 file after the snapshot 2024-03-04T07:37:15.956 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/33 file after the snapshot 2024-03-04T07:37:15.956 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/34 file after the snapshot 2024-03-04T07:37:15.956 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/35 file after the snapshot 2024-03-04T07:37:15.957 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/36 file after the snapshot 2024-03-04T07:37:15.957 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/37 file after the snapshot 2024-03-04T07:37:15.957 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/38 file after the snapshot 2024-03-04T07:37:15.957 INFO:tasks.workunit.client.0.smithi113.stdout: create 203/39 file after the snapshot 2024-03-04T07:37:15.957 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/.snap/snap-subdir-test 2024-03-04T07:37:16.008 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/30 file after the snapshot 2024-03-04T07:37:16.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/31 file after the snapshot 2024-03-04T07:37:16.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/32 file after the snapshot 2024-03-04T07:37:16.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/33 file after the snapshot 2024-03-04T07:37:16.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/34 file after the snapshot 2024-03-04T07:37:16.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/35 file after the snapshot 2024-03-04T07:37:16.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/36 file after the snapshot 2024-03-04T07:37:16.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/37 file after the snapshot 2024-03-04T07:37:16.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/38 file after the snapshot 2024-03-04T07:37:16.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 204/39 file after the snapshot 2024-03-04T07:37:16.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/.snap/snap-subdir-test 2024-03-04T07:37:16.062 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/30 file after the snapshot 2024-03-04T07:37:16.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/31 file after the snapshot 2024-03-04T07:37:16.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/32 file after the snapshot 2024-03-04T07:37:16.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/33 file after the snapshot 2024-03-04T07:37:16.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/34 file after the snapshot 2024-03-04T07:37:16.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/35 file after the snapshot 2024-03-04T07:37:16.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/36 file after the snapshot 2024-03-04T07:37:16.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/37 file after the snapshot 2024-03-04T07:37:16.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/38 file after the snapshot 2024-03-04T07:37:16.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 205/39 file after the snapshot 2024-03-04T07:37:16.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/.snap/snap-subdir-test 2024-03-04T07:37:16.176 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/30 file after the snapshot 2024-03-04T07:37:16.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/31 file after the snapshot 2024-03-04T07:37:16.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/32 file after the snapshot 2024-03-04T07:37:16.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/33 file after the snapshot 2024-03-04T07:37:16.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/34 file after the snapshot 2024-03-04T07:37:16.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/35 file after the snapshot 2024-03-04T07:37:16.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/36 file after the snapshot 2024-03-04T07:37:16.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/37 file after the snapshot 2024-03-04T07:37:16.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/38 file after the snapshot 2024-03-04T07:37:16.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 206/39 file after the snapshot 2024-03-04T07:37:16.180 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/.snap/snap-subdir-test 2024-03-04T07:37:17.146 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/30 file after the snapshot 2024-03-04T07:37:17.149 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/31 file after the snapshot 2024-03-04T07:37:17.149 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/32 file after the snapshot 2024-03-04T07:37:17.149 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/33 file after the snapshot 2024-03-04T07:37:17.149 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/34 file after the snapshot 2024-03-04T07:37:17.149 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/35 file after the snapshot 2024-03-04T07:37:17.149 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/36 file after the snapshot 2024-03-04T07:37:17.149 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/37 file after the snapshot 2024-03-04T07:37:17.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/38 file after the snapshot 2024-03-04T07:37:17.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 207/39 file after the snapshot 2024-03-04T07:37:17.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/.snap/snap-subdir-test 2024-03-04T07:37:17.201 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/30 file after the snapshot 2024-03-04T07:37:17.205 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/31 file after the snapshot 2024-03-04T07:37:17.205 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/32 file after the snapshot 2024-03-04T07:37:17.205 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/33 file after the snapshot 2024-03-04T07:37:17.205 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/34 file after the snapshot 2024-03-04T07:37:17.205 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/35 file after the snapshot 2024-03-04T07:37:17.205 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/36 file after the snapshot 2024-03-04T07:37:17.205 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/37 file after the snapshot 2024-03-04T07:37:17.205 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/38 file after the snapshot 2024-03-04T07:37:17.205 INFO:tasks.workunit.client.0.smithi113.stdout: create 208/39 file after the snapshot 2024-03-04T07:37:17.205 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/.snap/snap-subdir-test 2024-03-04T07:37:17.257 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/30 file after the snapshot 2024-03-04T07:37:17.259 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/31 file after the snapshot 2024-03-04T07:37:17.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/32 file after the snapshot 2024-03-04T07:37:17.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/33 file after the snapshot 2024-03-04T07:37:17.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/34 file after the snapshot 2024-03-04T07:37:17.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/35 file after the snapshot 2024-03-04T07:37:17.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/36 file after the snapshot 2024-03-04T07:37:17.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/37 file after the snapshot 2024-03-04T07:37:17.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/38 file after the snapshot 2024-03-04T07:37:17.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 209/39 file after the snapshot 2024-03-04T07:37:17.260 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/.snap/snap-subdir-test 2024-03-04T07:37:17.743 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/30 file after the snapshot 2024-03-04T07:37:17.746 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/31 file after the snapshot 2024-03-04T07:37:17.746 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/32 file after the snapshot 2024-03-04T07:37:17.746 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/33 file after the snapshot 2024-03-04T07:37:17.746 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/34 file after the snapshot 2024-03-04T07:37:17.746 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/35 file after the snapshot 2024-03-04T07:37:17.746 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/36 file after the snapshot 2024-03-04T07:37:17.746 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/37 file after the snapshot 2024-03-04T07:37:17.746 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/38 file after the snapshot 2024-03-04T07:37:17.746 INFO:tasks.workunit.client.0.smithi113.stdout: create 210/39 file after the snapshot 2024-03-04T07:37:17.746 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/.snap/snap-subdir-test 2024-03-04T07:37:17.799 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/30 file after the snapshot 2024-03-04T07:37:17.802 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/31 file after the snapshot 2024-03-04T07:37:17.803 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/32 file after the snapshot 2024-03-04T07:37:17.803 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/33 file after the snapshot 2024-03-04T07:37:17.803 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/34 file after the snapshot 2024-03-04T07:37:17.803 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/35 file after the snapshot 2024-03-04T07:37:17.803 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/36 file after the snapshot 2024-03-04T07:37:17.803 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/37 file after the snapshot 2024-03-04T07:37:17.803 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/38 file after the snapshot 2024-03-04T07:37:17.803 INFO:tasks.workunit.client.0.smithi113.stdout: create 211/39 file after the snapshot 2024-03-04T07:37:17.803 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/.snap/snap-subdir-test 2024-03-04T07:37:17.860 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/30 file after the snapshot 2024-03-04T07:37:17.863 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/31 file after the snapshot 2024-03-04T07:37:17.863 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/32 file after the snapshot 2024-03-04T07:37:17.863 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/33 file after the snapshot 2024-03-04T07:37:17.864 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/34 file after the snapshot 2024-03-04T07:37:17.864 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/35 file after the snapshot 2024-03-04T07:37:17.864 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/36 file after the snapshot 2024-03-04T07:37:17.864 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/37 file after the snapshot 2024-03-04T07:37:17.864 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/38 file after the snapshot 2024-03-04T07:37:17.864 INFO:tasks.workunit.client.0.smithi113.stdout: create 212/39 file after the snapshot 2024-03-04T07:37:17.864 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/.snap/snap-subdir-test 2024-03-04T07:37:17.972 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/30 file after the snapshot 2024-03-04T07:37:18.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/31 file after the snapshot 2024-03-04T07:37:18.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/32 file after the snapshot 2024-03-04T07:37:18.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/33 file after the snapshot 2024-03-04T07:37:18.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/34 file after the snapshot 2024-03-04T07:37:18.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/35 file after the snapshot 2024-03-04T07:37:18.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/36 file after the snapshot 2024-03-04T07:37:18.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/37 file after the snapshot 2024-03-04T07:37:18.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/38 file after the snapshot 2024-03-04T07:37:18.256 INFO:tasks.workunit.client.0.smithi113.stdout: create 213/39 file after the snapshot 2024-03-04T07:37:18.257 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/.snap/snap-subdir-test 2024-03-04T07:37:19.108 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/30 file after the snapshot 2024-03-04T07:37:19.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/31 file after the snapshot 2024-03-04T07:37:19.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/32 file after the snapshot 2024-03-04T07:37:19.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/33 file after the snapshot 2024-03-04T07:37:19.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/34 file after the snapshot 2024-03-04T07:37:19.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/35 file after the snapshot 2024-03-04T07:37:19.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/36 file after the snapshot 2024-03-04T07:37:19.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/37 file after the snapshot 2024-03-04T07:37:19.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/38 file after the snapshot 2024-03-04T07:37:19.112 INFO:tasks.workunit.client.0.smithi113.stdout: create 214/39 file after the snapshot 2024-03-04T07:37:19.112 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/.snap/snap-subdir-test 2024-03-04T07:37:19.645 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/30 file after the snapshot 2024-03-04T07:37:19.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/31 file after the snapshot 2024-03-04T07:37:19.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/32 file after the snapshot 2024-03-04T07:37:19.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/33 file after the snapshot 2024-03-04T07:37:19.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/34 file after the snapshot 2024-03-04T07:37:19.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/35 file after the snapshot 2024-03-04T07:37:19.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/36 file after the snapshot 2024-03-04T07:37:19.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/37 file after the snapshot 2024-03-04T07:37:19.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/38 file after the snapshot 2024-03-04T07:37:19.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 215/39 file after the snapshot 2024-03-04T07:37:19.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/.snap/snap-subdir-test 2024-03-04T07:37:19.740 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/30 file after the snapshot 2024-03-04T07:37:19.743 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/31 file after the snapshot 2024-03-04T07:37:19.743 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/32 file after the snapshot 2024-03-04T07:37:19.743 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/33 file after the snapshot 2024-03-04T07:37:19.743 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/34 file after the snapshot 2024-03-04T07:37:19.743 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/35 file after the snapshot 2024-03-04T07:37:19.743 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/36 file after the snapshot 2024-03-04T07:37:19.743 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/37 file after the snapshot 2024-03-04T07:37:19.744 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/38 file after the snapshot 2024-03-04T07:37:19.744 INFO:tasks.workunit.client.0.smithi113.stdout: create 216/39 file after the snapshot 2024-03-04T07:37:19.744 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/.snap/snap-subdir-test 2024-03-04T07:37:20.009 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/30 file after the snapshot 2024-03-04T07:37:20.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/31 file after the snapshot 2024-03-04T07:37:20.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/32 file after the snapshot 2024-03-04T07:37:20.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/33 file after the snapshot 2024-03-04T07:37:20.011 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/34 file after the snapshot 2024-03-04T07:37:20.012 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/35 file after the snapshot 2024-03-04T07:37:20.012 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/36 file after the snapshot 2024-03-04T07:37:20.012 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/37 file after the snapshot 2024-03-04T07:37:20.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/38 file after the snapshot 2024-03-04T07:37:20.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 217/39 file after the snapshot 2024-03-04T07:37:20.013 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/.snap/snap-subdir-test 2024-03-04T07:37:20.065 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/30 file after the snapshot 2024-03-04T07:37:20.067 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/31 file after the snapshot 2024-03-04T07:37:20.067 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/32 file after the snapshot 2024-03-04T07:37:20.067 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/33 file after the snapshot 2024-03-04T07:37:20.068 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/34 file after the snapshot 2024-03-04T07:37:20.068 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/35 file after the snapshot 2024-03-04T07:37:20.068 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/36 file after the snapshot 2024-03-04T07:37:20.068 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/37 file after the snapshot 2024-03-04T07:37:20.068 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/38 file after the snapshot 2024-03-04T07:37:20.068 INFO:tasks.workunit.client.0.smithi113.stdout: create 218/39 file after the snapshot 2024-03-04T07:37:20.068 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/.snap/snap-subdir-test 2024-03-04T07:37:20.112 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/30 file after the snapshot 2024-03-04T07:37:20.114 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/31 file after the snapshot 2024-03-04T07:37:20.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/32 file after the snapshot 2024-03-04T07:37:20.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/33 file after the snapshot 2024-03-04T07:37:20.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/34 file after the snapshot 2024-03-04T07:37:20.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/35 file after the snapshot 2024-03-04T07:37:20.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/36 file after the snapshot 2024-03-04T07:37:20.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/37 file after the snapshot 2024-03-04T07:37:20.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/38 file after the snapshot 2024-03-04T07:37:20.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 219/39 file after the snapshot 2024-03-04T07:37:20.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/.snap/snap-subdir-test 2024-03-04T07:37:20.158 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/30 file after the snapshot 2024-03-04T07:37:20.161 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/31 file after the snapshot 2024-03-04T07:37:20.161 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/32 file after the snapshot 2024-03-04T07:37:20.161 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/33 file after the snapshot 2024-03-04T07:37:20.161 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/34 file after the snapshot 2024-03-04T07:37:20.161 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/35 file after the snapshot 2024-03-04T07:37:20.161 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/36 file after the snapshot 2024-03-04T07:37:20.161 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/37 file after the snapshot 2024-03-04T07:37:20.161 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/38 file after the snapshot 2024-03-04T07:37:20.162 INFO:tasks.workunit.client.0.smithi113.stdout: create 220/39 file after the snapshot 2024-03-04T07:37:20.162 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/.snap/snap-subdir-test 2024-03-04T07:37:20.214 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/30 file after the snapshot 2024-03-04T07:37:20.217 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/31 file after the snapshot 2024-03-04T07:37:20.217 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/32 file after the snapshot 2024-03-04T07:37:20.218 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/33 file after the snapshot 2024-03-04T07:37:20.218 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/34 file after the snapshot 2024-03-04T07:37:20.218 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/35 file after the snapshot 2024-03-04T07:37:20.218 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/36 file after the snapshot 2024-03-04T07:37:20.218 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/37 file after the snapshot 2024-03-04T07:37:20.218 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/38 file after the snapshot 2024-03-04T07:37:20.218 INFO:tasks.workunit.client.0.smithi113.stdout: create 221/39 file after the snapshot 2024-03-04T07:37:20.218 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/.snap/snap-subdir-test 2024-03-04T07:37:20.271 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/30 file after the snapshot 2024-03-04T07:37:20.274 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/31 file after the snapshot 2024-03-04T07:37:20.274 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/32 file after the snapshot 2024-03-04T07:37:20.274 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/33 file after the snapshot 2024-03-04T07:37:20.274 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/34 file after the snapshot 2024-03-04T07:37:20.274 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/35 file after the snapshot 2024-03-04T07:37:20.274 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/36 file after the snapshot 2024-03-04T07:37:20.275 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/37 file after the snapshot 2024-03-04T07:37:20.275 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/38 file after the snapshot 2024-03-04T07:37:20.275 INFO:tasks.workunit.client.0.smithi113.stdout: create 222/39 file after the snapshot 2024-03-04T07:37:20.275 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/.snap/snap-subdir-test 2024-03-04T07:37:20.326 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/30 file after the snapshot 2024-03-04T07:37:20.329 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/31 file after the snapshot 2024-03-04T07:37:20.329 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/32 file after the snapshot 2024-03-04T07:37:20.329 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/33 file after the snapshot 2024-03-04T07:37:20.329 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/34 file after the snapshot 2024-03-04T07:37:20.329 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/35 file after the snapshot 2024-03-04T07:37:20.329 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/36 file after the snapshot 2024-03-04T07:37:20.329 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/37 file after the snapshot 2024-03-04T07:37:20.330 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/38 file after the snapshot 2024-03-04T07:37:20.330 INFO:tasks.workunit.client.0.smithi113.stdout: create 223/39 file after the snapshot 2024-03-04T07:37:20.330 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/.snap/snap-subdir-test 2024-03-04T07:37:20.366 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/30 file after the snapshot 2024-03-04T07:37:20.369 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/31 file after the snapshot 2024-03-04T07:37:20.370 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/32 file after the snapshot 2024-03-04T07:37:20.370 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/33 file after the snapshot 2024-03-04T07:37:20.370 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/34 file after the snapshot 2024-03-04T07:37:20.370 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/35 file after the snapshot 2024-03-04T07:37:20.370 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/36 file after the snapshot 2024-03-04T07:37:20.370 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/37 file after the snapshot 2024-03-04T07:37:20.370 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/38 file after the snapshot 2024-03-04T07:37:20.370 INFO:tasks.workunit.client.0.smithi113.stdout: create 224/39 file after the snapshot 2024-03-04T07:37:20.370 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/.snap/snap-subdir-test 2024-03-04T07:37:20.416 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/30 file after the snapshot 2024-03-04T07:37:20.419 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/31 file after the snapshot 2024-03-04T07:37:20.419 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/32 file after the snapshot 2024-03-04T07:37:20.419 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/33 file after the snapshot 2024-03-04T07:37:20.419 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/34 file after the snapshot 2024-03-04T07:37:20.419 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/35 file after the snapshot 2024-03-04T07:37:20.419 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/36 file after the snapshot 2024-03-04T07:37:20.419 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/37 file after the snapshot 2024-03-04T07:37:20.419 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/38 file after the snapshot 2024-03-04T07:37:20.420 INFO:tasks.workunit.client.0.smithi113.stdout: create 225/39 file after the snapshot 2024-03-04T07:37:20.420 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/.snap/snap-subdir-test 2024-03-04T07:37:21.082 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/30 file after the snapshot 2024-03-04T07:37:21.084 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/31 file after the snapshot 2024-03-04T07:37:21.085 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/32 file after the snapshot 2024-03-04T07:37:21.085 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/33 file after the snapshot 2024-03-04T07:37:21.085 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/34 file after the snapshot 2024-03-04T07:37:21.085 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/35 file after the snapshot 2024-03-04T07:37:21.085 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/36 file after the snapshot 2024-03-04T07:37:21.085 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/37 file after the snapshot 2024-03-04T07:37:21.085 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/38 file after the snapshot 2024-03-04T07:37:21.085 INFO:tasks.workunit.client.0.smithi113.stdout: create 226/39 file after the snapshot 2024-03-04T07:37:21.085 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/.snap/snap-subdir-test 2024-03-04T07:37:21.140 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/30 file after the snapshot 2024-03-04T07:37:21.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/31 file after the snapshot 2024-03-04T07:37:21.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/32 file after the snapshot 2024-03-04T07:37:21.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/33 file after the snapshot 2024-03-04T07:37:21.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/34 file after the snapshot 2024-03-04T07:37:21.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/35 file after the snapshot 2024-03-04T07:37:21.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/36 file after the snapshot 2024-03-04T07:37:21.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/37 file after the snapshot 2024-03-04T07:37:21.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/38 file after the snapshot 2024-03-04T07:37:21.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 227/39 file after the snapshot 2024-03-04T07:37:21.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/.snap/snap-subdir-test 2024-03-04T07:37:22.181 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/30 file after the snapshot 2024-03-04T07:37:22.183 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/31 file after the snapshot 2024-03-04T07:37:22.183 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/32 file after the snapshot 2024-03-04T07:37:22.183 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/33 file after the snapshot 2024-03-04T07:37:22.184 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/34 file after the snapshot 2024-03-04T07:37:22.184 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/35 file after the snapshot 2024-03-04T07:37:22.184 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/36 file after the snapshot 2024-03-04T07:37:22.184 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/37 file after the snapshot 2024-03-04T07:37:22.184 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/38 file after the snapshot 2024-03-04T07:37:22.185 INFO:tasks.workunit.client.0.smithi113.stdout: create 228/39 file after the snapshot 2024-03-04T07:37:22.185 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/.snap/snap-subdir-test 2024-03-04T07:37:22.620 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/30 file after the snapshot 2024-03-04T07:37:22.623 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/31 file after the snapshot 2024-03-04T07:37:22.623 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/32 file after the snapshot 2024-03-04T07:37:22.623 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/33 file after the snapshot 2024-03-04T07:37:22.623 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/34 file after the snapshot 2024-03-04T07:37:22.624 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/35 file after the snapshot 2024-03-04T07:37:22.624 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/36 file after the snapshot 2024-03-04T07:37:22.624 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/37 file after the snapshot 2024-03-04T07:37:22.624 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/38 file after the snapshot 2024-03-04T07:37:22.624 INFO:tasks.workunit.client.0.smithi113.stdout: create 229/39 file after the snapshot 2024-03-04T07:37:22.624 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/.snap/snap-subdir-test 2024-03-04T07:37:22.888 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/30 file after the snapshot 2024-03-04T07:37:22.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/31 file after the snapshot 2024-03-04T07:37:22.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/32 file after the snapshot 2024-03-04T07:37:22.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/33 file after the snapshot 2024-03-04T07:37:22.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/34 file after the snapshot 2024-03-04T07:37:22.892 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/35 file after the snapshot 2024-03-04T07:37:22.892 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/36 file after the snapshot 2024-03-04T07:37:22.892 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/37 file after the snapshot 2024-03-04T07:37:22.892 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/38 file after the snapshot 2024-03-04T07:37:22.892 INFO:tasks.workunit.client.0.smithi113.stdout: create 230/39 file after the snapshot 2024-03-04T07:37:22.892 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/.snap/snap-subdir-test 2024-03-04T07:37:23.317 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/30 file after the snapshot 2024-03-04T07:37:23.596 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/31 file after the snapshot 2024-03-04T07:37:23.597 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/32 file after the snapshot 2024-03-04T07:37:23.597 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/33 file after the snapshot 2024-03-04T07:37:23.597 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/34 file after the snapshot 2024-03-04T07:37:23.597 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/35 file after the snapshot 2024-03-04T07:37:23.597 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/36 file after the snapshot 2024-03-04T07:37:23.597 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/37 file after the snapshot 2024-03-04T07:37:23.597 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/38 file after the snapshot 2024-03-04T07:37:23.597 INFO:tasks.workunit.client.0.smithi113.stdout: create 231/39 file after the snapshot 2024-03-04T07:37:23.597 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/.snap/snap-subdir-test 2024-03-04T07:37:23.646 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/30 file after the snapshot 2024-03-04T07:37:23.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/31 file after the snapshot 2024-03-04T07:37:23.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/32 file after the snapshot 2024-03-04T07:37:23.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/33 file after the snapshot 2024-03-04T07:37:23.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/34 file after the snapshot 2024-03-04T07:37:23.648 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/35 file after the snapshot 2024-03-04T07:37:23.649 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/36 file after the snapshot 2024-03-04T07:37:23.649 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/37 file after the snapshot 2024-03-04T07:37:23.649 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/38 file after the snapshot 2024-03-04T07:37:23.649 INFO:tasks.workunit.client.0.smithi113.stdout: create 232/39 file after the snapshot 2024-03-04T07:37:23.649 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/.snap/snap-subdir-test 2024-03-04T07:37:23.684 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/30 file after the snapshot 2024-03-04T07:37:23.688 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/31 file after the snapshot 2024-03-04T07:37:23.688 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/32 file after the snapshot 2024-03-04T07:37:23.688 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/33 file after the snapshot 2024-03-04T07:37:23.688 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/34 file after the snapshot 2024-03-04T07:37:23.688 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/35 file after the snapshot 2024-03-04T07:37:23.688 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/36 file after the snapshot 2024-03-04T07:37:23.688 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/37 file after the snapshot 2024-03-04T07:37:23.689 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/38 file after the snapshot 2024-03-04T07:37:23.689 INFO:tasks.workunit.client.0.smithi113.stdout: create 233/39 file after the snapshot 2024-03-04T07:37:23.689 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/.snap/snap-subdir-test 2024-03-04T07:37:23.745 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/30 file after the snapshot 2024-03-04T07:37:23.747 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/31 file after the snapshot 2024-03-04T07:37:23.748 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/32 file after the snapshot 2024-03-04T07:37:23.748 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/33 file after the snapshot 2024-03-04T07:37:23.748 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/34 file after the snapshot 2024-03-04T07:37:23.748 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/35 file after the snapshot 2024-03-04T07:37:23.748 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/36 file after the snapshot 2024-03-04T07:37:23.748 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/37 file after the snapshot 2024-03-04T07:37:23.748 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/38 file after the snapshot 2024-03-04T07:37:23.748 INFO:tasks.workunit.client.0.smithi113.stdout: create 234/39 file after the snapshot 2024-03-04T07:37:23.748 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/.snap/snap-subdir-test 2024-03-04T07:37:23.766 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:23.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.902+0000 7f5e4da00640 1 Processor -- start 2024-03-04T07:37:23.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e4da00640 1 -- start start 2024-03-04T07:37:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e4da00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e480b20b0 0x7f5e48157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e4da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e48157c50 0x7f5e4806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e4da00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e480c1ec0 con 0x7f5e480c85f0 2024-03-04T07:37:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e4da00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e480c9b30 con 0x7f5e480b20b0 2024-03-04T07:37:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e4da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480c12a0 con 0x7f5e48157c50 2024-03-04T07:37:23.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e46a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e480b20b0 0x7f5e48157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e46a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5e480b20b0 0x7f5e48157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43846/0 (socket says 172.21.15.113:43846) 2024-03-04T07:37:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e46a00640 1 -- 172.21.15.113:0/2497547836 learned_addr learned my addr 172.21.15.113:0/2497547836 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e46000640 1 --2- 172.21.15.113:0/2497547836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e48157c50 0x7f5e4806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e46a00640 1 -- 172.21.15.113:0/2497547836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e48157c50 msgr2=0x7f5e4806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e46a00640 1 --2- 172.21.15.113:0/2497547836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e48157c50 0x7f5e4806b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:23.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e46a00640 1 -- 172.21.15.113:0/2497547836 >> v1:172.21.15.136:6789/0 conn(0x7f5e480c85f0 legacy=0x7f5e4815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:23.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.906+0000 7f5e46a00640 1 -- 172.21.15.113:0/2497547836 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480c1580 con 0x7f5e480b20b0 2024-03-04T07:37:23.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e46a00640 1 --2- 172.21.15.113:0/2497547836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e480b20b0 0x7f5e48157710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009870 tx=0x7f5e3801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1401b698a91ac00 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:23.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e45600640 1 -- 172.21.15.113:0/2497547836 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380b1070 con 0x7f5e480b20b0 2024-03-04T07:37:23.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e45600640 1 -- 172.21.15.113:0/2497547836 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e380ac620 con 0x7f5e480b20b0 2024-03-04T07:37:23.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e45600640 1 -- 172.21.15.113:0/2497547836 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e380ac9f0 con 0x7f5e480b20b0 2024-03-04T07:37:23.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e4da00640 1 -- 172.21.15.113:0/2497547836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e480b20b0 msgr2=0x7f5e48157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:23.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e4da00640 1 --2- 172.21.15.113:0/2497547836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e480b20b0 0x7f5e48157710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009870 tx=0x7f5e3801c530 comp rx=0 tx=0).stop 2024-03-04T07:37:23.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e4da00640 1 -- 172.21.15.113:0/2497547836 shutdown_connections 2024-03-04T07:37:23.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e4da00640 1 --2- 172.21.15.113:0/2497547836 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e48157c50 0x7f5e4806b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:23.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e4da00640 1 --2- 172.21.15.113:0/2497547836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e480b20b0 0x7f5e48157710 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e4da00640 1 -- 172.21.15.113:0/2497547836 >> 172.21.15.113:0/2497547836 conn(0x7f5e4806e820 msgr2=0x7f5e4806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e4da00640 1 -- 172.21.15.113:0/2497547836 shutdown_connections 2024-03-04T07:37:23.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.910+0000 7f5e4da00640 1 -- 172.21.15.113:0/2497547836 wait complete. 2024-03-04T07:37:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.914+0000 7f5e4da00640 1 Processor -- start 2024-03-04T07:37:23.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.914+0000 7f5e4da00640 1 -- start start 2024-03-04T07:37:23.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.914+0000 7f5e4da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e480b20b0 0x7f5e480c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:23.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.914+0000 7f5e4da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e480c85f0 0x7f5e480c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:23.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.914+0000 7f5e4da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e48157c50 0x7f5e48150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:23.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.914+0000 7f5e4da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480c1b80 con 0x7f5e48157c50 2024-03-04T07:37:23.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.914+0000 7f5e4da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48156ec0 con 0x7f5e480c85f0 2024-03-04T07:37:23.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.914+0000 7f5e4da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480c80c0 con 0x7f5e480b20b0 2024-03-04T07:37:23.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.914+0000 7f5e47400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e48157c50 0x7f5e48150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:23.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e46a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e480b20b0 0x7f5e480c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:23.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e47400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e48157c50 0x7f5e48150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43862/0 (socket says 172.21.15.113:43862) 2024-03-04T07:37:23.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e47400640 1 -- 172.21.15.113:0/2258019022 learned_addr learned my addr 172.21.15.113:0/2258019022 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e46000640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e480c85f0 0x7f5e480c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e47400640 1 -- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e480b20b0 msgr2=0x7f5e480c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e47400640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e480b20b0 0x7f5e480c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e47400640 1 -- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e480c85f0 msgr2=0x7f5e480c3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e47400640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e480c85f0 0x7f5e480c3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:23.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e47400640 1 -- 172.21.15.113:0/2258019022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e481f3c10 con 0x7f5e48157c50 2024-03-04T07:37:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e47400640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e48157c50 0x7f5e48150de0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c730 tx=0x7f5e4000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e3ea00640 1 -- 172.21.15.113:0/2258019022 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40003b80 con 0x7f5e48157c50 2024-03-04T07:37:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e3ea00640 1 -- 172.21.15.113:0/2258019022 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e40003d20 con 0x7f5e48157c50 2024-03-04T07:37:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e4da00640 1 -- 172.21.15.113:0/2258019022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e480af9d0 con 0x7f5e48157c50 2024-03-04T07:37:23.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e3ea00640 1 -- 172.21.15.113:0/2258019022 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4000aa30 con 0x7f5e48157c50 2024-03-04T07:37:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e4da00640 1 -- 172.21.15.113:0/2258019022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e480aff90 con 0x7f5e48157c50 2024-03-04T07:37:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e4da00640 1 -- 172.21.15.113:0/2258019022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e48075c00 con 0x7f5e48157c50 2024-03-04T07:37:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.918+0000 7f5e3ea00640 1 -- 172.21.15.113:0/2258019022 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e4000abd0 con 0x7f5e48157c50 2024-03-04T07:37:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.922+0000 7f5e3ea00640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e28066f60 0x7f5e28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.922+0000 7f5e3ea00640 1 -- 172.21.15.113:0/2258019022 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f5e40084ee0 con 0x7f5e48157c50 2024-03-04T07:37:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.922+0000 7f5e46a00640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e28066f60 0x7f5e28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:23.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.922+0000 7f5e46a00640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e28066f60 0x7f5e28069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5e380ae000 tx=0x7f5e38008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:23.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:23.926+0000 7f5e3ea00640 1 -- 172.21.15.113:0/2258019022 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5e40055710 con 0x7f5e48157c50 2024-03-04T07:37:24.165 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/30 file after the snapshot 2024-03-04T07:37:24.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 -- 172.21.15.113:0/2258019022 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e48072c90 con 0x7f5e48157c50 2024-03-04T07:37:24.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e3ea00640 1 -- 172.21.15.113:0/2258019022 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f5e400597b0 con 0x7f5e48157c50 2024-03-04T07:37:24.229 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:24.229 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4938,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4250]},"id":1}]} 2024-03-04T07:37:24.229 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 19 2024-03-04T07:37:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 -- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e28066f60 msgr2=0x7f5e28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e28066f60 0x7f5e28069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5e380ae000 tx=0x7f5e38008040 comp rx=0 tx=0).stop 2024-03-04T07:37:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 -- 172.21.15.113:0/2258019022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e48157c50 msgr2=0x7f5e48150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e48157c50 0x7f5e48150de0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5e4000c730 tx=0x7f5e4000cbf0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 -- 172.21.15.113:0/2258019022 shutdown_connections 2024-03-04T07:37:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e28066f60 0x7f5e28069420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e48157c50 0x7f5e48150de0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e480c85f0 0x7f5e480c3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 --2- 172.21.15.113:0/2258019022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5e480b20b0 0x7f5e480c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 -- 172.21.15.113:0/2258019022 >> 172.21.15.113:0/2258019022 conn(0x7f5e4806e820 msgr2=0x7f5e480c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 -- 172.21.15.113:0/2258019022 shutdown_connections 2024-03-04T07:37:24.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.226+0000 7f5e4da00640 1 -- 172.21.15.113:0/2258019022 wait complete. 2024-03-04T07:37:24.250 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=2) 2024-03-04T07:37:24.251 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-04T07:37:24.251 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:37:24.265 INFO:tasks.ceph.mds.c:Stopped 2024-03-04T07:37:24.265 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:24.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.438+0000 7ff5f5c00640 1 Processor -- start 2024-03-04T07:37:24.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.438+0000 7ff5f5c00640 1 -- start start 2024-03-04T07:37:24.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5f5c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 0x7ff5f00ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:24.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5f5c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff5f006ff20 0x7ff5f00ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:24.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5f5c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f0152e30 con 0x7ff5f00c8b90 2024-03-04T07:37:24.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5f5c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f00c31b0 con 0x7ff5f006ff20 2024-03-04T07:37:24.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5f5c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f006f840 con 0x7ff5f0152a50 2024-03-04T07:37:24.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5ef400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff5f00c8b90 0x7ff5f015bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40866/0 (socket says 172.21.15.113:40866) 2024-03-04T07:37:24.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5ef400640 1 -- 172.21.15.113:0/185183582 learned_addr learned my addr 172.21.15.113:0/185183582 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:37:24.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5eea00640 1 --2- 172.21.15.113:0/185183582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 0x7ff5f00ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:24.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5e7e00640 1 --2- 172.21.15.113:0/185183582 >> v2:172.21.15.136:3300/0 conn(0x7ff5f006ff20 0x7ff5f00ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5ee000640 1 -- 172.21.15.113:0/185183582 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535539593 0 0) 0x7ff5f0152e30 con 0x7ff5f00c8b90 2024-03-04T07:37:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5ee000640 1 -- 172.21.15.113:0/185183582 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5d4003530 con 0x7ff5f00c8b90 2024-03-04T07:37:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5eea00640 1 -- 172.21.15.113:0/185183582 >> v2:172.21.15.136:3300/0 conn(0x7ff5f006ff20 msgr2=0x7ff5f00ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5eea00640 1 --2- 172.21.15.113:0/185183582 >> v2:172.21.15.136:3300/0 conn(0x7ff5f006ff20 0x7ff5f00ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5eea00640 1 -- 172.21.15.113:0/185183582 >> v1:172.21.15.113:6789/0 conn(0x7ff5f00c8b90 legacy=0x7ff5f015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5eea00640 1 -- 172.21.15.113:0/185183582 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f006f280 con 0x7ff5f0152a50 2024-03-04T07:37:24.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.442+0000 7ff5e7e00640 1 --2- 172.21.15.113:0/185183582 >> v2:172.21.15.136:3300/0 conn(0x7ff5f006ff20 0x7ff5f00ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:37:24.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.446+0000 7ff5eea00640 1 --2- 172.21.15.113:0/185183582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 0x7ff5f00ce120 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00097c0 tx=0x7ff5e001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548e22a1acd00483 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:24.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.446+0000 7ff5ee000640 1 -- 172.21.15.113:0/185183582 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00aa600 con 0x7ff5f0152a50 2024-03-04T07:37:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.446+0000 7ff5ee000640 1 -- 172.21.15.113:0/185183582 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e00af070 con 0x7ff5f0152a50 2024-03-04T07:37:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.446+0000 7ff5ee000640 1 -- 172.21.15.113:0/185183582 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00aa9a0 con 0x7ff5f0152a50 2024-03-04T07:37:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.446+0000 7ff5f5c00640 1 -- 172.21.15.113:0/185183582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 msgr2=0x7ff5f00ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.446+0000 7ff5f5c00640 1 --2- 172.21.15.113:0/185183582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 0x7ff5f00ce120 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00097c0 tx=0x7ff5e001c4a0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.446+0000 7ff5f5c00640 1 -- 172.21.15.113:0/185183582 shutdown_connections 2024-03-04T07:37:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.446+0000 7ff5f5c00640 1 --2- 172.21.15.113:0/185183582 >> v2:172.21.15.136:3300/0 conn(0x7ff5f006ff20 0x7ff5f00ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.446+0000 7ff5f5c00640 1 --2- 172.21.15.113:0/185183582 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 0x7ff5f00ce120 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.446+0000 7ff5f5c00640 1 -- 172.21.15.113:0/185183582 >> 172.21.15.113:0/185183582 conn(0x7ff5f00734d0 msgr2=0x7ff5f0066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5f5c00640 1 -- 172.21.15.113:0/185183582 shutdown_connections 2024-03-04T07:37:24.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5f5c00640 1 -- 172.21.15.113:0/185183582 wait complete. 2024-03-04T07:37:24.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5f5c00640 1 Processor -- start 2024-03-04T07:37:24.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5f5c00640 1 -- start start 2024-03-04T07:37:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5f5c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f006ff20 0x7ff5f0154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5f5c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f00c8b90 0x7ff5f014e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5f5c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 0x7ff5f01f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5f5c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0165a00 con 0x7ff5f006ff20 2024-03-04T07:37:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5f5c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0165880 con 0x7ff5f00c8b90 2024-03-04T07:37:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5f5c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0165b80 con 0x7ff5f0152a50 2024-03-04T07:37:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5eea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f006ff20 0x7ff5f0154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5eea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f006ff20 0x7ff5f0154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43876/0 (socket says 172.21.15.113:43876) 2024-03-04T07:37:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5eea00640 1 -- 172.21.15.113:0/1210892695 learned_addr learned my addr 172.21.15.113:0/1210892695 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5ef400640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 0x7ff5f01f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5e7e00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f00c8b90 0x7ff5f014e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5eea00640 1 -- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 msgr2=0x7ff5f01f3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5eea00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 0x7ff5f01f3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5eea00640 1 -- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f00c8b90 msgr2=0x7ff5f014e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5eea00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f00c8b90 0x7ff5f014e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5eea00640 1 -- 172.21.15.113:0/1210892695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f0155c00 con 0x7ff5f006ff20 2024-03-04T07:37:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5eea00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f006ff20 0x7ff5f0154f50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00ac040 tx=0x7ff5e0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.450+0000 7ff5e6a00640 1 -- 172.21.15.113:0/1210892695 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0003ab0 con 0x7ff5f006ff20 2024-03-04T07:37:24.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.454+0000 7ff5e6a00640 1 -- 172.21.15.113:0/1210892695 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e0003c50 con 0x7ff5f006ff20 2024-03-04T07:37:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.454+0000 7ff5f5c00640 1 -- 172.21.15.113:0/1210892695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f0155ee0 con 0x7ff5f006ff20 2024-03-04T07:37:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.454+0000 7ff5e6a00640 1 -- 172.21.15.113:0/1210892695 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00bf480 con 0x7ff5f006ff20 2024-03-04T07:37:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.454+0000 7ff5f5c00640 1 -- 172.21.15.113:0/1210892695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f00c8450 con 0x7ff5f006ff20 2024-03-04T07:37:24.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.454+0000 7ff5f5c00640 1 -- 172.21.15.113:0/1210892695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f01f2da0 con 0x7ff5f006ff20 2024-03-04T07:37:24.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.454+0000 7ff5e6a00640 1 -- 172.21.15.113:0/1210892695 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5e00a9070 con 0x7ff5f006ff20 2024-03-04T07:37:24.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.454+0000 7ff5e6a00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5cc066f60 0x7ff5cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:24.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.454+0000 7ff5e6a00640 1 -- 172.21.15.113:0/1210892695 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff5e011fa70 con 0x7ff5f006ff20 2024-03-04T07:37:24.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.458+0000 7ff5e7e00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5cc066f60 0x7ff5cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:24.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.458+0000 7ff5e7e00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5cc066f60 0x7ff5cc069420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009580 tx=0x7ff5d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:24.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.462+0000 7ff5e6a00640 1 -- 172.21.15.113:0/1210892695 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff5f01f2da0 con 0x7ff5f006ff20 2024-03-04T07:37:24.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.654+0000 7ff5f5c00640 1 -- 172.21.15.113:0/1210892695 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5f006e880 con 0x7ff5f006ff20 2024-03-04T07:37:24.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.654+0000 7ff5e6a00640 1 -- 172.21.15.113:0/1210892695 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6615 (secure 0 0 0) 0x7ff5e0016090 con 0x7ff5f006ff20 2024-03-04T07:37:24.659 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:24.659 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4938,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4250]},"id":1}]} 2024-03-04T07:37:24.659 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 19 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 -- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5cc066f60 msgr2=0x7ff5cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5cc066f60 0x7ff5cc069420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009580 tx=0x7ff5d8008040 comp rx=0 tx=0).stop 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 -- 172.21.15.113:0/1210892695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f006ff20 msgr2=0x7ff5f0154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f006ff20 0x7ff5f0154f50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00ac040 tx=0x7ff5e0019240 comp rx=0 tx=0).stop 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 -- 172.21.15.113:0/1210892695 shutdown_connections 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f0152a50 0x7ff5f01f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f00c8b90 0x7ff5f014e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5cc066f60 0x7ff5cc069420 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 --2- 172.21.15.113:0/1210892695 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f006ff20 0x7ff5f0154f50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 -- 172.21.15.113:0/1210892695 >> 172.21.15.113:0/1210892695 conn(0x7ff5f00734d0 msgr2=0x7ff5f00ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:24.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 -- 172.21.15.113:0/1210892695 shutdown_connections 2024-03-04T07:37:24.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:24.658+0000 7ff5f5c00640 1 -- 172.21.15.113:0/1210892695 wait complete. 2024-03-04T07:37:24.680 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:37:24.705 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/31 file after the snapshot 2024-03-04T07:37:24.705 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/32 file after the snapshot 2024-03-04T07:37:24.705 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/33 file after the snapshot 2024-03-04T07:37:24.705 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/34 file after the snapshot 2024-03-04T07:37:24.705 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/35 file after the snapshot 2024-03-04T07:37:24.706 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/36 file after the snapshot 2024-03-04T07:37:24.706 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/37 file after the snapshot 2024-03-04T07:37:24.706 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/38 file after the snapshot 2024-03-04T07:37:24.706 INFO:tasks.workunit.client.0.smithi113.stdout: create 235/39 file after the snapshot 2024-03-04T07:37:24.706 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/.snap/snap-subdir-test 2024-03-04T07:37:25.278 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:37:25.281 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:37:26.681 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:26.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.814+0000 7ff429600640 1 Processor -- start 2024-03-04T07:37:26.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.814+0000 7ff429600640 1 -- start start 2024-03-04T07:37:26.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.814+0000 7ff429600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff424152920 0x7ff424071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:26.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.814+0000 7ff429600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff42406c580 0x7ff4240c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.814+0000 7ff429600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42406bb10 con 0x7ff42415bf20 2024-03-04T07:37:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.814+0000 7ff429600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4240c7a40 con 0x7ff424152920 2024-03-04T07:37:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.814+0000 7ff429600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240c7d20 con 0x7ff42406c580 2024-03-04T07:37:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.814+0000 7ff423e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff424152920 0x7ff424071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.814+0000 7ff428c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff42415bf20 0x7ff42415c3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40878/0 (socket says 172.21.15.113:40878) 2024-03-04T07:37:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.814+0000 7ff428c00640 1 -- 172.21.15.113:0/801407112 learned_addr learned my addr 172.21.15.113:0/801407112 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:37:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff423e00640 1 -- 172.21.15.113:0/801407112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff42406c580 msgr2=0x7ff4240c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff423e00640 1 --2- 172.21.15.113:0/801407112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff42406c580 0x7ff4240c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff423e00640 1 -- 172.21.15.113:0/801407112 >> v1:172.21.15.113:6789/0 conn(0x7ff42415bf20 legacy=0x7ff42415c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:26.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff423e00640 1 -- 172.21.15.113:0/801407112 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42406bd80 con 0x7ff424152920 2024-03-04T07:37:26.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff423e00640 1 --2- 172.21.15.113:0/801407112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff424152920 0x7ff424071a30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff4140097c0 tx=0x7ff41401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a8c82d63398adb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff423400640 1 -- 172.21.15.113:0/801407112 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4140ab600 con 0x7ff424152920 2024-03-04T07:37:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff423400640 1 -- 172.21.15.113:0/801407112 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4140b0070 con 0x7ff424152920 2024-03-04T07:37:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff423400640 1 -- 172.21.15.113:0/801407112 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4140ab9c0 con 0x7ff424152920 2024-03-04T07:37:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff429600640 1 -- 172.21.15.113:0/801407112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff424152920 msgr2=0x7ff424071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff429600640 1 --2- 172.21.15.113:0/801407112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff424152920 0x7ff424071a30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff4140097c0 tx=0x7ff41401c3e0 comp rx=0 tx=0).stop 2024-03-04T07:37:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff429600640 1 -- 172.21.15.113:0/801407112 shutdown_connections 2024-03-04T07:37:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff429600640 1 --2- 172.21.15.113:0/801407112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff42406c580 0x7ff4240c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff429600640 1 --2- 172.21.15.113:0/801407112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff424152920 0x7ff424071a30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff429600640 1 -- 172.21.15.113:0/801407112 >> 172.21.15.113:0/801407112 conn(0x7ff4240c9a30 msgr2=0x7ff4240c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:26.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff429600640 1 -- 172.21.15.113:0/801407112 shutdown_connections 2024-03-04T07:37:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.818+0000 7ff429600640 1 -- 172.21.15.113:0/801407112 wait complete. 2024-03-04T07:37:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff429600640 1 Processor -- start 2024-03-04T07:37:26.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff429600640 1 -- start start 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff429600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff42406c580 0x7ff424153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff429600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff424152920 0x7ff424151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff429600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff42415bf20 0x7ff4240c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff429600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240c95a0 con 0x7ff42406c580 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff429600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff424152d00 con 0x7ff42415bf20 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff429600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240c9720 con 0x7ff424152920 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff423e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff42406c580 0x7ff424153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff423e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff42406c580 0x7ff424153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43896/0 (socket says 172.21.15.113:43896) 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff423e00640 1 -- 172.21.15.113:0/3207646012 learned_addr learned my addr 172.21.15.113:0/3207646012 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff41be00640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff424152920 0x7ff424151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:26.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff423e00640 1 -- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff424152920 msgr2=0x7ff424151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff423e00640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff424152920 0x7ff424151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff423e00640 1 -- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff42415bf20 msgr2=0x7ff4240c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff423e00640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff42415bf20 0x7ff4240c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff423e00640 1 -- 172.21.15.113:0/3207646012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff424154110 con 0x7ff42406c580 2024-03-04T07:37:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff423e00640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff42406c580 0x7ff424153860 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff4140ad040 tx=0x7ff41401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff420c00640 1 -- 172.21.15.113:0/3207646012 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4140bf610 con 0x7ff42406c580 2024-03-04T07:37:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff420c00640 1 -- 172.21.15.113:0/3207646012 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4140bf7b0 con 0x7ff42406c580 2024-03-04T07:37:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff429600640 1 -- 172.21.15.113:0/3207646012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4241543f0 con 0x7ff42406c580 2024-03-04T07:37:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff420c00640 1 -- 172.21.15.113:0/3207646012 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4140bfba0 con 0x7ff42406c580 2024-03-04T07:37:26.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.822+0000 7ff429600640 1 -- 172.21.15.113:0/3207646012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241549b0 con 0x7ff42406c580 2024-03-04T07:37:26.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.826+0000 7ff429600640 1 -- 172.21.15.113:0/3207646012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff424075a80 con 0x7ff42406c580 2024-03-04T07:37:26.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.826+0000 7ff420c00640 1 -- 172.21.15.113:0/3207646012 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4140bd020 con 0x7ff42406c580 2024-03-04T07:37:26.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.826+0000 7ff420c00640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff400066f60 0x7ff400069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:26.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.826+0000 7ff420c00640 1 -- 172.21.15.113:0/3207646012 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff41411f8c0 con 0x7ff42406c580 2024-03-04T07:37:26.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.830+0000 7ff41be00640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff400066f60 0x7ff400069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.834+0000 7ff41be00640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff400066f60 0x7ff400069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff40c006fd0 tx=0x7ff40c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:26.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:26.834+0000 7ff420c00640 1 -- 172.21.15.113:0/3207646012 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff414016090 con 0x7ff42406c580 2024-03-04T07:37:27.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.086+0000 7ff429600640 1 -- 172.21.15.113:0/3207646012 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff424071b80 con 0x7ff42406c580 2024-03-04T07:37:27.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff420c00640 1 -- 172.21.15.113:0/3207646012 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6615 (secure 0 0 0) 0x7ff4140f07d0 con 0x7ff42406c580 2024-03-04T07:37:27.091 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:27.092 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4938,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4250]},"id":1}]} 2024-03-04T07:37:27.092 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 19 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 -- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff400066f60 msgr2=0x7ff400069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff400066f60 0x7ff400069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff40c006fd0 tx=0x7ff40c008040 comp rx=0 tx=0).stop 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 -- 172.21.15.113:0/3207646012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff42406c580 msgr2=0x7ff424153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff42406c580 0x7ff424153860 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff4140ad040 tx=0x7ff41401ac20 comp rx=0 tx=0).stop 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 -- 172.21.15.113:0/3207646012 shutdown_connections 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff42415bf20 0x7ff4240c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff424152920 0x7ff424151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff400066f60 0x7ff400069420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 --2- 172.21.15.113:0/3207646012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff42406c580 0x7ff424153860 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 -- 172.21.15.113:0/3207646012 >> 172.21.15.113:0/3207646012 conn(0x7ff4240c9a30 msgr2=0x7ff424072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 -- 172.21.15.113:0/3207646012 shutdown_connections 2024-03-04T07:37:27.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:27.090+0000 7ff429600640 1 -- 172.21.15.113:0/3207646012 wait complete. 2024-03-04T07:37:27.113 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:37:29.114 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:29.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab78a00640 1 Processor -- start 2024-03-04T07:37:29.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab78a00640 1 -- start start 2024-03-04T07:37:29.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab78a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab74157f80 0x7fab7406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:29.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab78a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab740c8a90 0x7fab7406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:29.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab78a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fab7406f2c0 0x7fab740c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:29.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab78a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7406e1e0 con 0x7fab74157f80 2024-03-04T07:37:29.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab78a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7406df00 con 0x7fab7406f2c0 2024-03-04T07:37:29.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab78a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab740ce1a0 con 0x7fab740c8a90 2024-03-04T07:37:29.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab73400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab74157f80 0x7fab7406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:29.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab73400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab74157f80 0x7fab7406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55962/0 (socket says 172.21.15.113:55962) 2024-03-04T07:37:29.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab73400640 1 -- 172.21.15.113:0/2183070172 learned_addr learned my addr 172.21.15.113:0/2183070172 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:29.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab6be00640 1 --2- 172.21.15.113:0/2183070172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab740c8a90 0x7fab7406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:29.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab73e00640 1 --2- 172.21.15.113:0/2183070172 >> v2:172.21.15.136:3300/0 conn(0x7fab7406f2c0 0x7fab740c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:29.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab73400640 1 -- 172.21.15.113:0/2183070172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab740c8a90 msgr2=0x7fab7406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:29.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab73400640 1 --2- 172.21.15.113:0/2183070172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab740c8a90 0x7fab7406ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab73400640 1 -- 172.21.15.113:0/2183070172 >> v2:172.21.15.136:3300/0 conn(0x7fab7406f2c0 msgr2=0x7fab740c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:29.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab73400640 1 --2- 172.21.15.113:0/2183070172 >> v2:172.21.15.136:3300/0 conn(0x7fab7406f2c0 0x7fab740c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.310+0000 7fab73400640 1 -- 172.21.15.113:0/2183070172 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab740ce480 con 0x7fab74157f80 2024-03-04T07:37:29.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab73400640 1 --2- 172.21.15.113:0/2183070172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 0x7fab7406c170 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab7415efb0 tx=0x7fab6401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c507de6ffb24a87 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:29.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab72a00640 1 -- 172.21.15.113:0/2183070172 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab640b0070 con 0x7fab74157f80 2024-03-04T07:37:29.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab72a00640 1 -- 172.21.15.113:0/2183070172 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab640ab600 con 0x7fab74157f80 2024-03-04T07:37:29.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab72a00640 1 -- 172.21.15.113:0/2183070172 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab640ab9d0 con 0x7fab74157f80 2024-03-04T07:37:29.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab78a00640 1 -- 172.21.15.113:0/2183070172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 msgr2=0x7fab7406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:29.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab78a00640 1 --2- 172.21.15.113:0/2183070172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 0x7fab7406c170 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab7415efb0 tx=0x7fab6401c520 comp rx=0 tx=0).stop 2024-03-04T07:37:29.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab78a00640 1 -- 172.21.15.113:0/2183070172 shutdown_connections 2024-03-04T07:37:29.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab78a00640 1 --2- 172.21.15.113:0/2183070172 >> v2:172.21.15.136:3300/0 conn(0x7fab7406f2c0 0x7fab740c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab78a00640 1 --2- 172.21.15.113:0/2183070172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab740c8a90 0x7fab7406ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab78a00640 1 --2- 172.21.15.113:0/2183070172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 0x7fab7406c170 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab78a00640 1 -- 172.21.15.113:0/2183070172 >> 172.21.15.113:0/2183070172 conn(0x7fab74072f40 msgr2=0x7fab740706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:29.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab78a00640 1 -- 172.21.15.113:0/2183070172 shutdown_connections 2024-03-04T07:37:29.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.314+0000 7fab78a00640 1 -- 172.21.15.113:0/2183070172 wait complete. 2024-03-04T07:37:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 Processor -- start 2024-03-04T07:37:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 -- start start 2024-03-04T07:37:29.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7406f2c0 0x7fab7414c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:29.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab740c8a90 0x7fab741f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:29.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 0x7fab74155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:29.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7406fc30 con 0x7fab74157f80 2024-03-04T07:37:29.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740c8e70 con 0x7fab7406f2c0 2024-03-04T07:37:29.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7406fdb0 con 0x7fab740c8a90 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab73e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 0x7fab74155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab73e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 0x7fab74155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55978/0 (socket says 172.21.15.113:55978) 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab73e00640 1 -- 172.21.15.113:0/2479902683 learned_addr learned my addr 172.21.15.113:0/2479902683 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab6be00640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab740c8a90 0x7fab741f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab73400640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7406f2c0 0x7fab7414c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab73e00640 1 -- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab740c8a90 msgr2=0x7fab741f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab73e00640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab740c8a90 0x7fab741f4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab73e00640 1 -- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7406f2c0 msgr2=0x7fab7414c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab73e00640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7406f2c0 0x7fab7414c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab73e00640 1 -- 172.21.15.113:0/2479902683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7414a950 con 0x7fab74157f80 2024-03-04T07:37:29.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab73e00640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 0x7fab74155920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00ef00 tx=0x7fab6c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:29.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab6b400640 1 -- 172.21.15.113:0/2479902683 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c018020 con 0x7fab74157f80 2024-03-04T07:37:29.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab6b400640 1 -- 172.21.15.113:0/2479902683 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c00bd20 con 0x7fab74157f80 2024-03-04T07:37:29.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 -- 172.21.15.113:0/2479902683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab74150370 con 0x7fab74157f80 2024-03-04T07:37:29.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab6b400640 1 -- 172.21.15.113:0/2479902683 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c014640 con 0x7fab74157f80 2024-03-04T07:37:29.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 -- 172.21.15.113:0/2479902683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab74150930 con 0x7fab74157f80 2024-03-04T07:37:29.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.318+0000 7fab78a00640 1 -- 172.21.15.113:0/2479902683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab74075a80 con 0x7fab74157f80 2024-03-04T07:37:29.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.322+0000 7fab6b400640 1 -- 172.21.15.113:0/2479902683 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab6c01b020 con 0x7fab74157f80 2024-03-04T07:37:29.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.322+0000 7fab6b400640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab50066f60 0x7fab50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:29.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.322+0000 7fab6b400640 1 -- 172.21.15.113:0/2479902683 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fab6c0854e0 con 0x7fab74157f80 2024-03-04T07:37:29.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.322+0000 7fab73400640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab50066f60 0x7fab50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:29.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.322+0000 7fab73400640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab50066f60 0x7fab50069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab640ad000 tx=0x7fab64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:29.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.326+0000 7fab6b400640 1 -- 172.21.15.113:0/2479902683 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fab6c022070 con 0x7fab74157f80 2024-03-04T07:37:29.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.526+0000 7fab78a00640 1 -- 172.21.15.113:0/2479902683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab740b2920 con 0x7fab74157f80 2024-03-04T07:37:29.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab6b400640 1 -- 172.21.15.113:0/2479902683 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6615 (secure 0 0 0) 0x7fab6c055e40 con 0x7fab74157f80 2024-03-04T07:37:29.531 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:29.532 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4938,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4250]},"id":1}]} 2024-03-04T07:37:29.532 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 19 2024-03-04T07:37:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 -- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab50066f60 msgr2=0x7fab50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab50066f60 0x7fab50069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab640ad000 tx=0x7fab64008040 comp rx=0 tx=0).stop 2024-03-04T07:37:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 -- 172.21.15.113:0/2479902683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 msgr2=0x7fab74155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 0x7fab74155920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00ef00 tx=0x7fab6c00b480 comp rx=0 tx=0).stop 2024-03-04T07:37:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 -- 172.21.15.113:0/2479902683 shutdown_connections 2024-03-04T07:37:29.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab50066f60 0x7fab50069420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab74157f80 0x7fab74155920 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab740c8a90 0x7fab741f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 --2- 172.21.15.113:0/2479902683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab7406f2c0 0x7fab7414c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:29.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 -- 172.21.15.113:0/2479902683 >> 172.21.15.113:0/2479902683 conn(0x7fab74072f40 msgr2=0x7fab740c3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:29.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 -- 172.21.15.113:0/2479902683 shutdown_connections 2024-03-04T07:37:29.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:29.530+0000 7fab78a00640 1 -- 172.21.15.113:0/2479902683 wait complete. 2024-03-04T07:37:29.553 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:37:31.554 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:31.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.694+0000 7fdd85e00640 1 Processor -- start 2024-03-04T07:37:31.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.694+0000 7fdd85e00640 1 -- start start 2024-03-04T07:37:31.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd85e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdd80151af0 0x7fdd800cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:31.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd85e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdd800c7af0 0x7fdd800cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:31.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd85e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 0x7fdd80157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:31.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd85e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd80151ed0 con 0x7fdd800c7af0 2024-03-04T07:37:31.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd85e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd800c9970 con 0x7fdd80151af0 2024-03-04T07:37:31.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd85e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800c8400 con 0x7fdd8006cc50 2024-03-04T07:37:31.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd77e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdd800c7af0 0x7fdd800cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:31.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd7f400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 0x7fdd80157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:31.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd7ea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdd80151af0 0x7fdd800cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:31.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd7f400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 0x7fdd80157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:48122/0 (socket says 172.21.15.113:48122) 2024-03-04T07:37:31.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd7f400640 1 -- 172.21.15.113:0/3734332977 learned_addr learned my addr 172.21.15.113:0/3734332977 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:31.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd77e00640 1 -- 172.21.15.113:0/3734332977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 msgr2=0x7fdd80157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:31.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd77e00640 1 --2- 172.21.15.113:0/3734332977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 0x7fdd80157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd77e00640 1 -- 172.21.15.113:0/3734332977 >> v2:172.21.15.136:3300/0 conn(0x7fdd80151af0 msgr2=0x7fdd800cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:31.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd77e00640 1 --2- 172.21.15.113:0/3734332977 >> v2:172.21.15.136:3300/0 conn(0x7fdd80151af0 0x7fdd800cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.698+0000 7fdd77e00640 1 -- 172.21.15.113:0/3734332977 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd800c8650 con 0x7fdd800c7af0 2024-03-04T07:37:31.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd77e00640 1 --2- 172.21.15.113:0/3734332977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd800c7af0 0x7fdd800cd550 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdd80067810 tx=0x7fdd6801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc1847aa0beb083 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:31.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd7e000640 1 -- 172.21.15.113:0/3734332977 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680b0070 con 0x7fdd800c7af0 2024-03-04T07:37:31.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd7e000640 1 -- 172.21.15.113:0/3734332977 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd680ab600 con 0x7fdd800c7af0 2024-03-04T07:37:31.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd7e000640 1 -- 172.21.15.113:0/3734332977 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680ab9d0 con 0x7fdd800c7af0 2024-03-04T07:37:31.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd85e00640 1 -- 172.21.15.113:0/3734332977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd800c7af0 msgr2=0x7fdd800cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:31.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd85e00640 1 --2- 172.21.15.113:0/3734332977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd800c7af0 0x7fdd800cd550 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdd80067810 tx=0x7fdd6801c480 comp rx=0 tx=0).stop 2024-03-04T07:37:31.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd85e00640 1 -- 172.21.15.113:0/3734332977 shutdown_connections 2024-03-04T07:37:31.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd85e00640 1 --2- 172.21.15.113:0/3734332977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 0x7fdd80157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd85e00640 1 --2- 172.21.15.113:0/3734332977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd800c7af0 0x7fdd800cd550 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd85e00640 1 --2- 172.21.15.113:0/3734332977 >> v2:172.21.15.136:3300/0 conn(0x7fdd80151af0 0x7fdd800cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.702+0000 7fdd85e00640 1 -- 172.21.15.113:0/3734332977 >> 172.21.15.113:0/3734332977 conn(0x7fdd800671e0 msgr2=0x7fdd8006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:31.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 -- 172.21.15.113:0/3734332977 shutdown_connections 2024-03-04T07:37:31.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 -- 172.21.15.113:0/3734332977 wait complete. 2024-03-04T07:37:31.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 Processor -- start 2024-03-04T07:37:31.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 -- start start 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 0x7fdd80161750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd800c7af0 0x7fdd801f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd80151af0 0x7fdd801560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8015b510 con 0x7fdd80151af0 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8015b690 con 0x7fdd800c7af0 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8015b810 con 0x7fdd8006cc50 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd7f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd80151af0 0x7fdd801560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd7f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd80151af0 0x7fdd801560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55996/0 (socket says 172.21.15.113:55996) 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd7f400640 1 -- 172.21.15.113:0/2057212172 learned_addr learned my addr 172.21.15.113:0/2057212172 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd7ea00640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 0x7fdd80161750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:31.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd77e00640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd800c7af0 0x7fdd801f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd7f400640 1 -- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 msgr2=0x7fdd80161750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd7f400640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 0x7fdd80161750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd7f400640 1 -- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd800c7af0 msgr2=0x7fdd801f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd7f400640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd800c7af0 0x7fdd801f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd7f400640 1 -- 172.21.15.113:0/2057212172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80159380 con 0x7fdd80151af0 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd7f400640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd80151af0 0x7fdd801560f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800eed0 tx=0x7fdd7800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd77400640 1 -- 172.21.15.113:0/2057212172 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7801a070 con 0x7fdd80151af0 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd77400640 1 -- 172.21.15.113:0/2057212172 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7800bd10 con 0x7fdd80151af0 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd77400640 1 -- 172.21.15.113:0/2057212172 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78009e60 con 0x7fdd80151af0 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.706+0000 7fdd85e00640 1 -- 172.21.15.113:0/2057212172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd801f81f0 con 0x7fdd80151af0 2024-03-04T07:37:31.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.710+0000 7fdd85e00640 1 -- 172.21.15.113:0/2057212172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd801607e0 con 0x7fdd80151af0 2024-03-04T07:37:31.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.710+0000 7fdd85e00640 1 -- 172.21.15.113:0/2057212172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd801f7910 con 0x7fdd80151af0 2024-03-04T07:37:31.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.710+0000 7fdd77400640 1 -- 172.21.15.113:0/2057212172 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd7801d070 con 0x7fdd80151af0 2024-03-04T07:37:31.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.710+0000 7fdd77400640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd5c066f60 0x7fdd5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:31.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.710+0000 7fdd77400640 1 -- 172.21.15.113:0/2057212172 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fdd78085070 con 0x7fdd80151af0 2024-03-04T07:37:31.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.710+0000 7fdd7ea00640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd5c066f60 0x7fdd5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:31.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.714+0000 7fdd7ea00640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd5c066f60 0x7fdd5c069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009ac0 tx=0x7fdd70009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.714+0000 7fdd77400640 1 -- 172.21.15.113:0/2057212172 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdd78055940 con 0x7fdd80151af0 2024-03-04T07:37:31.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.926+0000 7fdd85e00640 1 -- 172.21.15.113:0/2057212172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd80075c00 con 0x7fdd80151af0 2024-03-04T07:37:31.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.926+0000 7fdd77400640 1 -- 172.21.15.113:0/2057212172 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6615 (secure 0 0 0) 0x7fdd780599e0 con 0x7fdd80151af0 2024-03-04T07:37:31.930 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:31.930 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4938,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4250]},"id":1}]} 2024-03-04T07:37:31.930 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 19 2024-03-04T07:37:31.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 -- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd5c066f60 msgr2=0x7fdd5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:31.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd5c066f60 0x7fdd5c069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009ac0 tx=0x7fdd70009190 comp rx=0 tx=0).stop 2024-03-04T07:37:31.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 -- 172.21.15.113:0/2057212172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd80151af0 msgr2=0x7fdd801560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:31.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd80151af0 0x7fdd801560f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800eed0 tx=0x7fdd7800b480 comp rx=0 tx=0).stop 2024-03-04T07:37:31.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 -- 172.21.15.113:0/2057212172 shutdown_connections 2024-03-04T07:37:31.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd5c066f60 0x7fdd5c069420 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd80151af0 0x7fdd801560f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd800c7af0 0x7fdd801f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 --2- 172.21.15.113:0/2057212172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd8006cc50 0x7fdd80161750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:31.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 -- 172.21.15.113:0/2057212172 >> 172.21.15.113:0/2057212172 conn(0x7fdd800671e0 msgr2=0x7fdd800c3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:31.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 -- 172.21.15.113:0/2057212172 shutdown_connections 2024-03-04T07:37:31.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:31.930+0000 7fdd85e00640 1 -- 172.21.15.113:0/2057212172 wait complete. 2024-03-04T07:37:31.961 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:37:33.962 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd384200640 1 Processor -- start 2024-03-04T07:37:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd384200640 1 -- start start 2024-03-04T07:37:34.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd384200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c0c8200 0x7fd37c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:34.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd384200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c1591f0 con 0x7fd37c152bd0 2024-03-04T07:37:34.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd384200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c1594d0 con 0x7fd37c1527f0 2024-03-04T07:37:34.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd384200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c071df0 con 0x7fd37c0c8200 2024-03-04T07:37:34.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd382200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd37c152bd0 0x7fd37c1587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44768/0 (socket says 172.21.15.113:44768) 2024-03-04T07:37:34.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd382200640 1 -- 172.21.15.113:0/2856598333 learned_addr learned my addr 172.21.15.113:0/2856598333 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:37:34.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd380e00640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c0c8200 0x7fd37c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:34.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696341767 0 0) 0x7fd37c1591f0 con 0x7fd37c152bd0 2024-03-04T07:37:34.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd364003530 con 0x7fd37c152bd0 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277334776 0 0) 0x7fd37c1594d0 con 0x7fd37c1527f0 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd37c1591f0 con 0x7fd37c1527f0 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3527054375 0 0) 0x7fd364003530 con 0x7fd37c152bd0 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c0c8200 msgr2=0x7fd37c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c0c8200 0x7fd37c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 >> v1:172.21.15.136:6789/0 conn(0x7fd37c1527f0 legacy=0x7fd37c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0717a0 con 0x7fd37c152bd0 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fd374001a00 con 0x7fd37c152bd0 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 >> v1:172.21.15.113:6789/0 conn(0x7fd37c152bd0 legacy=0x7fd37c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd364003b80 0x7fd364003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 0x7fd364004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c0c8200 0x7fd3640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c071df0 con 0x7fd364004810 2024-03-04T07:37:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36400ba80 con 0x7fd364003b80 2024-03-04T07:37:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36400bc50 con 0x7fd37c0c8200 2024-03-04T07:37:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd380e00640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 0x7fd364004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd381800640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd364003b80 0x7fd364003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd382200640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c0c8200 0x7fd3640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd380e00640 1 -- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c0c8200 msgr2=0x7fd3640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd380e00640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c0c8200 0x7fd3640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd380e00640 1 -- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd364003b80 msgr2=0x7fd364003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.098+0000 7fd380e00640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd364003b80 0x7fd364003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd380e00640 1 -- 172.21.15.113:0/2856598333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364003530 con 0x7fd364004810 2024-03-04T07:37:34.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd380e00640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 0x7fd364004c90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd368009ab0 tx=0x7fd36801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf08a1b9a343bee9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:34.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680b2070 con 0x7fd364004810 2024-03-04T07:37:34.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3680a55e0 con 0x7fd364004810 2024-03-04T07:37:34.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd37be00640 1 -- 172.21.15.113:0/2856598333 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680096b0 con 0x7fd364004810 2024-03-04T07:37:34.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd384200640 1 -- 172.21.15.113:0/2856598333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 msgr2=0x7fd364004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:34.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd384200640 1 --2- 172.21.15.113:0/2856598333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 0x7fd364004c90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd368009ab0 tx=0x7fd36801c9f0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd382200640 1 -- 172.21.15.113:0/2856598333 reap_dead start 2024-03-04T07:37:34.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd384200640 1 -- 172.21.15.113:0/2856598333 shutdown_connections 2024-03-04T07:37:34.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd384200640 1 -- 172.21.15.113:0/2856598333 >> 172.21.15.113:0/2856598333 conn(0x7fd37c06e820 msgr2=0x7fd37c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:34.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd384200640 1 -- 172.21.15.113:0/2856598333 shutdown_connections 2024-03-04T07:37:34.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.102+0000 7fd384200640 1 -- 172.21.15.113:0/2856598333 wait complete. 2024-03-04T07:37:34.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd384200640 1 Processor -- start 2024-03-04T07:37:34.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd384200640 1 -- start start 2024-03-04T07:37:34.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd384200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 0x7fd37c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:34.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd384200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd37c0c2ed0 0x7fd37c1f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:34.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd384200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c154340 0x7fd37c14ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:34.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd384200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c158ee0 con 0x7fd364004810 2024-03-04T07:37:34.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd384200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c1597b0 con 0x7fd37c0c2ed0 2024-03-04T07:37:34.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd384200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c070d40 con 0x7fd37c154340 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd381800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 0x7fd37c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd381800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 0x7fd37c155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56024/0 (socket says 172.21.15.113:56024) 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd381800640 1 -- 172.21.15.113:0/498659876 learned_addr learned my addr 172.21.15.113:0/498659876 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd382200640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c154340 0x7fd37c14ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd380e00640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd37c0c2ed0 0x7fd37c1f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd381800640 1 -- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c154340 msgr2=0x7fd37c14ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd381800640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c154340 0x7fd37c14ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd381800640 1 -- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd37c0c2ed0 msgr2=0x7fd37c1f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd381800640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd37c0c2ed0 0x7fd37c1f07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd381800640 1 -- 172.21.15.113:0/498659876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0b18b0 con 0x7fd364004810 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd380e00640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd37c0c2ed0 0x7fd37c1f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd381800640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 0x7fd37c155940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd3700079b0 tx=0x7fd370012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:34.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd379600640 1 -- 172.21.15.113:0/498659876 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370010b70 con 0x7fd364004810 2024-03-04T07:37:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd379600640 1 -- 172.21.15.113:0/498659876 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37001b070 con 0x7fd364004810 2024-03-04T07:37:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd384200640 1 -- 172.21.15.113:0/498659876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c0c2b20 con 0x7fd364004810 2024-03-04T07:37:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd379600640 1 -- 172.21.15.113:0/498659876 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370016480 con 0x7fd364004810 2024-03-04T07:37:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.106+0000 7fd384200640 1 -- 172.21.15.113:0/498659876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c155180 con 0x7fd364004810 2024-03-04T07:37:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.110+0000 7fd384200640 1 -- 172.21.15.113:0/498659876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c075c00 con 0x7fd364004810 2024-03-04T07:37:34.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.110+0000 7fd379600640 1 -- 172.21.15.113:0/498659876 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd370016620 con 0x7fd364004810 2024-03-04T07:37:34.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.110+0000 7fd379600640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd35c066f60 0x7fd35c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:34.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.110+0000 7fd379600640 1 -- 172.21.15.113:0/498659876 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd370088920 con 0x7fd364004810 2024-03-04T07:37:34.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.110+0000 7fd380e00640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd35c066f60 0x7fd35c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:34.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.114+0000 7fd380e00640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd35c066f60 0x7fd35c069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd3680af000 tx=0x7fd3680046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:34.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.114+0000 7fd379600640 1 -- 172.21.15.113:0/498659876 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd3700597b0 con 0x7fd364004810 2024-03-04T07:37:34.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 -- 172.21.15.113:0/498659876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd37c0c79d0 con 0x7fd364004810 2024-03-04T07:37:34.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd379600640 1 -- 172.21.15.113:0/498659876 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6615 (secure 0 0 0) 0x7fd3700590b0 con 0x7fd364004810 2024-03-04T07:37:34.308 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:34.309 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4938,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4250]},"id":1}]} 2024-03-04T07:37:34.309 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 19 2024-03-04T07:37:34.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 -- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd35c066f60 msgr2=0x7fd35c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:34.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd35c066f60 0x7fd35c069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd3680af000 tx=0x7fd3680046d0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 -- 172.21.15.113:0/498659876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 msgr2=0x7fd37c155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 0x7fd37c155940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd3700079b0 tx=0x7fd370012d00 comp rx=0 tx=0).stop 2024-03-04T07:37:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 -- 172.21.15.113:0/498659876 shutdown_connections 2024-03-04T07:37:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd37c154340 0x7fd37c14ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd37c0c2ed0 0x7fd37c1f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd35c066f60 0x7fd35c069420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 --2- 172.21.15.113:0/498659876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd364004810 0x7fd37c155940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 -- 172.21.15.113:0/498659876 >> 172.21.15.113:0/498659876 conn(0x7fd37c06e820 msgr2=0x7fd37c0caa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 -- 172.21.15.113:0/498659876 shutdown_connections 2024-03-04T07:37:34.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:34.306+0000 7fd384200640 1 -- 172.21.15.113:0/498659876 wait complete. 2024-03-04T07:37:34.329 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:37:36.330 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:36.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.462+0000 7f5ee7200640 1 Processor -- start 2024-03-04T07:37:36.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.462+0000 7f5ee7200640 1 -- start start 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.462+0000 7f5ee7200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5ee00c8200 0x7f5ee0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.462+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 0x7f5ee01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.462+0000 7f5ee7200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee01591f0 con 0x7f5ee01527f0 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.462+0000 7f5ee7200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee0071df0 con 0x7f5ee00c8200 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.462+0000 7f5ee7200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee01597b0 con 0x7f5ee0152bd0 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.466+0000 7f5edf400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5ee00c8200 0x7f5ee0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.466+0000 7f5edf400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5ee00c8200 0x7f5ee0070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56038/0 (socket says 172.21.15.113:56038) 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.466+0000 7f5edf400640 1 -- 172.21.15.113:0/4071091411 learned_addr learned my addr 172.21.15.113:0/4071091411 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.466+0000 7f5ee5200640 1 --2- 172.21.15.113:0/4071091411 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 0x7f5ee01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.466+0000 7f5edf400640 1 -- 172.21.15.113:0/4071091411 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 msgr2=0x7f5ee01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.466+0000 7f5edf400640 1 --2- 172.21.15.113:0/4071091411 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 0x7f5ee01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:36.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.466+0000 7f5edf400640 1 -- 172.21.15.113:0/4071091411 >> v1:172.21.15.136:6789/0 conn(0x7f5ee01527f0 legacy=0x7f5ee0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:36.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.466+0000 7f5edf400640 1 -- 172.21.15.113:0/4071091411 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee01594d0 con 0x7f5ee00c8200 2024-03-04T07:37:36.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.466+0000 7f5ee5200640 1 --2- 172.21.15.113:0/4071091411 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 0x7f5ee01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:37:36.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.466+0000 7f5edf400640 1 --2- 172.21.15.113:0/4071091411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00c8200 0x7f5ee0070d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009870 tx=0x7f5ecc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7e45dae44211de server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:36.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edea00640 1 -- 172.21.15.113:0/4071091411 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc0a7070 con 0x7f5ee00c8200 2024-03-04T07:37:36.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edea00640 1 -- 172.21.15.113:0/4071091411 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ecc0b0070 con 0x7f5ee00c8200 2024-03-04T07:37:36.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edea00640 1 -- 172.21.15.113:0/4071091411 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc0ab890 con 0x7f5ee00c8200 2024-03-04T07:37:36.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 -- 172.21.15.113:0/4071091411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00c8200 msgr2=0x7f5ee0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:36.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 --2- 172.21.15.113:0/4071091411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00c8200 0x7f5ee0070d40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009870 tx=0x7f5ecc01c560 comp rx=0 tx=0).stop 2024-03-04T07:37:36.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 -- 172.21.15.113:0/4071091411 shutdown_connections 2024-03-04T07:37:36.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 --2- 172.21.15.113:0/4071091411 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 0x7f5ee01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:36.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 --2- 172.21.15.113:0/4071091411 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee00c8200 0x7f5ee0070d40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:36.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 -- 172.21.15.113:0/4071091411 >> 172.21.15.113:0/4071091411 conn(0x7f5ee006e820 msgr2=0x7f5ee006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:36.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 -- 172.21.15.113:0/4071091411 shutdown_connections 2024-03-04T07:37:36.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 -- 172.21.15.113:0/4071091411 wait complete. 2024-03-04T07:37:36.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 Processor -- start 2024-03-04T07:37:36.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 -- start start 2024-03-04T07:37:36.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ee00c8200 0x7f5ee00b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:36.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee01527f0 0x7f5ee01571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:36.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 0x7f5ee014ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:36.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee01654f0 con 0x7f5ee01527f0 2024-03-04T07:37:36.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee0165370 con 0x7f5ee00c8200 2024-03-04T07:37:36.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee7200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee0165670 con 0x7f5ee0152bd0 2024-03-04T07:37:36.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edf400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee01527f0 0x7f5ee01571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:36.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edf400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee01527f0 0x7f5ee01571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56054/0 (socket says 172.21.15.113:56054) 2024-03-04T07:37:36.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edf400640 1 -- 172.21.15.113:0/2682732919 learned_addr learned my addr 172.21.15.113:0/2682732919 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:36.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edfe00640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ee00c8200 0x7f5ee00b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:36.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5ee5200640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 0x7f5ee014ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:36.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edf400640 1 -- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 msgr2=0x7f5ee014ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:36.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edf400640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 0x7f5ee014ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:36.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edf400640 1 -- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ee00c8200 msgr2=0x7f5ee00b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:36.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.470+0000 7f5edf400640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ee00c8200 0x7f5ee00b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:36.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5edf400640 1 -- 172.21.15.113:0/2682732919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee0156ed0 con 0x7f5ee01527f0 2024-03-04T07:37:36.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5edf400640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee01527f0 0x7f5ee01571f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0ad040 tx=0x7f5ecc01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:36.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5ed7e00640 1 -- 172.21.15.113:0/2682732919 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc0be5f0 con 0x7f5ee01527f0 2024-03-04T07:37:36.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5ed7e00640 1 -- 172.21.15.113:0/2682732919 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ecc0be7b0 con 0x7f5ee01527f0 2024-03-04T07:37:36.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5ed7e00640 1 -- 172.21.15.113:0/2682732919 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc0beba0 con 0x7f5ee01527f0 2024-03-04T07:37:36.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5ee7200640 1 -- 172.21.15.113:0/2682732919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee01465f0 con 0x7f5ee01527f0 2024-03-04T07:37:36.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5ee7200640 1 -- 172.21.15.113:0/2682732919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee0146bb0 con 0x7f5ee01527f0 2024-03-04T07:37:36.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5ee7200640 1 -- 172.21.15.113:0/2682732919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee0075c00 con 0x7f5ee01527f0 2024-03-04T07:37:36.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5ed7e00640 1 -- 172.21.15.113:0/2682732919 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ecc0b43f0 con 0x7f5ee01527f0 2024-03-04T07:37:36.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5ed7e00640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ec0066f60 0x7f5ec0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:36.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5ed7e00640 1 -- 172.21.15.113:0/2682732919 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f5ecc11f6b0 con 0x7f5ee01527f0 2024-03-04T07:37:36.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.474+0000 7f5edfe00640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ec0066f60 0x7f5ec0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:36.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.478+0000 7f5edfe00640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ec0066f60 0x7f5ec0069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0001810 tx=0x7f5ed0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:36.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.478+0000 7f5ed7e00640 1 -- 172.21.15.113:0/2682732919 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5ecc0c4060 con 0x7f5ee01527f0 2024-03-04T07:37:36.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 -- 172.21.15.113:0/2682732919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ee00732f0 con 0x7f5ee01527f0 2024-03-04T07:37:36.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ed7e00640 1 -- 172.21.15.113:0/2682732919 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f5ecc016150 con 0x7f5ee01527f0 2024-03-04T07:37:36.672 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:36.672 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4938,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4250]},"id":1}]} 2024-03-04T07:37:36.672 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 19 2024-03-04T07:37:36.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 -- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ec0066f60 msgr2=0x7f5ec0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:36.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ec0066f60 0x7f5ec0069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0001810 tx=0x7f5ed0009040 comp rx=0 tx=0).stop 2024-03-04T07:37:36.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 -- 172.21.15.113:0/2682732919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee01527f0 msgr2=0x7f5ee01571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:36.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee01527f0 0x7f5ee01571f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0ad040 tx=0x7f5ecc01ada0 comp rx=0 tx=0).stop 2024-03-04T07:37:36.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 -- 172.21.15.113:0/2682732919 shutdown_connections 2024-03-04T07:37:36.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ee0152bd0 0x7f5ee014ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:36.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ec0066f60 0x7f5ec0069420 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:36.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ee01527f0 0x7f5ee01571f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:36.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 --2- 172.21.15.113:0/2682732919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ee00c8200 0x7f5ee00b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:36.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 -- 172.21.15.113:0/2682732919 >> 172.21.15.113:0/2682732919 conn(0x7f5ee006e820 msgr2=0x7f5ee00c9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:36.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 -- 172.21.15.113:0/2682732919 shutdown_connections 2024-03-04T07:37:36.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:36.670+0000 7f5ee7200640 1 -- 172.21.15.113:0/2682732919 wait complete. 2024-03-04T07:37:36.692 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:37:38.693 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:38.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fdc800640 1 Processor -- start 2024-03-04T07:37:38.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fdc800640 1 -- start start 2024-03-04T07:37:38.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fdc800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7fd40c8200 0x7f7fd4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:38.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fdc800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7fd4152bd0 0x7f7fd41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fdc800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd41591f0 con 0x7f7fd41527f0 2024-03-04T07:37:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fdc800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd4071df0 con 0x7f7fd40c8200 2024-03-04T07:37:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fdc800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd41597b0 con 0x7f7fd4152bd0 2024-03-04T07:37:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fd9400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7fd40c8200 0x7f7fd4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fd9400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7fd40c8200 0x7f7fd4070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52144/0 (socket says 172.21.15.113:52144) 2024-03-04T07:37:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fd9400640 1 -- 172.21.15.113:0/1419594726 learned_addr learned my addr 172.21.15.113:0/1419594726 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fda800640 1 --2- 172.21.15.113:0/1419594726 >> v2:172.21.15.136:3300/0 conn(0x7f7fd4152bd0 0x7f7fd41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fd9400640 1 -- 172.21.15.113:0/1419594726 >> v2:172.21.15.136:3300/0 conn(0x7f7fd4152bd0 msgr2=0x7f7fd41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:38.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fd9400640 1 --2- 172.21.15.113:0/1419594726 >> v2:172.21.15.136:3300/0 conn(0x7f7fd4152bd0 0x7f7fd41587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:38.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.826+0000 7f7fd9400640 1 -- 172.21.15.113:0/1419594726 >> v1:172.21.15.136:6789/0 conn(0x7f7fd41527f0 legacy=0x7f7fd4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:38.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fd9400640 1 -- 172.21.15.113:0/1419594726 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd41594d0 con 0x7f7fd40c8200 2024-03-04T07:37:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fd9400640 1 --2- 172.21.15.113:0/1419594726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 0x7f7fd4070d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40712b0 tx=0x7f7fc401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d383c1de904e4f44 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fd8a00640 1 -- 172.21.15.113:0/1419594726 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40ab430 con 0x7f7fd40c8200 2024-03-04T07:37:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fd8a00640 1 -- 172.21.15.113:0/1419594726 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc40b0070 con 0x7f7fd40c8200 2024-03-04T07:37:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fd8a00640 1 -- 172.21.15.113:0/1419594726 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40ab7f0 con 0x7f7fd40c8200 2024-03-04T07:37:38.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fdc800640 1 -- 172.21.15.113:0/1419594726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 msgr2=0x7f7fd4070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:38.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fdc800640 1 --2- 172.21.15.113:0/1419594726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 0x7f7fd4070d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40712b0 tx=0x7f7fc401c520 comp rx=0 tx=0).stop 2024-03-04T07:37:38.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fdc800640 1 -- 172.21.15.113:0/1419594726 shutdown_connections 2024-03-04T07:37:38.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fdc800640 1 --2- 172.21.15.113:0/1419594726 >> v2:172.21.15.136:3300/0 conn(0x7f7fd4152bd0 0x7f7fd41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:38.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fdc800640 1 --2- 172.21.15.113:0/1419594726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 0x7f7fd4070d40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:38.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fdc800640 1 -- 172.21.15.113:0/1419594726 >> 172.21.15.113:0/1419594726 conn(0x7f7fd406e820 msgr2=0x7f7fd406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:38.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fdc800640 1 -- 172.21.15.113:0/1419594726 shutdown_connections 2024-03-04T07:37:38.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.830+0000 7f7fdc800640 1 -- 172.21.15.113:0/1419594726 wait complete. 2024-03-04T07:37:38.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fdc800640 1 Processor -- start 2024-03-04T07:37:38.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fdc800640 1 -- start start 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fdc800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 0x7f7fd4148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fdc800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd41527f0 0x7f7fd414dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fdc800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd4152bd0 0x7f7fd414e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fdc800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd41654f0 con 0x7f7fd40c8200 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fdc800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4165370 con 0x7f7fd41527f0 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fdc800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4165670 con 0x7f7fd4152bd0 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fd9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 0x7f7fd4148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fd9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 0x7f7fd4148520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52158/0 (socket says 172.21.15.113:52158) 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fd9e00640 1 -- 172.21.15.113:0/189478801 learned_addr learned my addr 172.21.15.113:0/189478801 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fd9400640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd41527f0 0x7f7fd414dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fda800640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd4152bd0 0x7f7fd414e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:38.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fd9e00640 1 -- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd4152bd0 msgr2=0x7f7fd414e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fd9e00640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd4152bd0 0x7f7fd414e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fd9e00640 1 -- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd41527f0 msgr2=0x7f7fd414dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fd9e00640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd41527f0 0x7f7fd414dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fd9e00640 1 -- 172.21.15.113:0/189478801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd41f4880 con 0x7f7fd40c8200 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fda800640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd4152bd0 0x7f7fd414e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fd9e00640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 0x7f7fd4148520 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c450 tx=0x7f7fc800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fce000640 1 -- 172.21.15.113:0/189478801 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8016020 con 0x7f7fd40c8200 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fce000640 1 -- 172.21.15.113:0/189478801 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc80052d0 con 0x7f7fd40c8200 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fdc800640 1 -- 172.21.15.113:0/189478801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd4151650 con 0x7f7fd40c8200 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fce000640 1 -- 172.21.15.113:0/189478801 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8007260 con 0x7f7fd40c8200 2024-03-04T07:37:38.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.834+0000 7f7fdc800640 1 -- 172.21.15.113:0/189478801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd4151c10 con 0x7f7fd40c8200 2024-03-04T07:37:38.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.838+0000 7f7fdc800640 1 -- 172.21.15.113:0/189478801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd4075c00 con 0x7f7fd40c8200 2024-03-04T07:37:38.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.838+0000 7f7fce000640 1 -- 172.21.15.113:0/189478801 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fc8007400 con 0x7f7fd40c8200 2024-03-04T07:37:38.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.838+0000 7f7fce000640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fb8066f60 0x7f7fb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:38.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.838+0000 7f7fce000640 1 -- 172.21.15.113:0/189478801 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f7fc8084a20 con 0x7f7fd40c8200 2024-03-04T07:37:38.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.838+0000 7f7fd9400640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fb8066f60 0x7f7fb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:38.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.842+0000 7f7fd9400640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fb8066f60 0x7f7fb8069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40af000 tx=0x7f7fc40a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:38.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:38.842+0000 7f7fce000640 1 -- 172.21.15.113:0/189478801 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7fc8020020 con 0x7f7fd40c8200 2024-03-04T07:37:39.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 -- 172.21.15.113:0/189478801 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fd40afa90 con 0x7f7fd40c8200 2024-03-04T07:37:39.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fce000640 1 -- 172.21.15.113:0/189478801 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f7fc8005480 con 0x7f7fd40c8200 2024-03-04T07:37:39.036 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:39.036 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4938,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:36:26.641361+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4250},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4250":{"gid":4250,"name":"c","rank":2,"incarnation":14,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6833/3005395426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3005395426},{"type":"v1","addr":"172.21.15.113:6833","nonce":3005395426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4250]},"id":1}]} 2024-03-04T07:37:39.037 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 19 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 -- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fb8066f60 msgr2=0x7f7fb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fb8066f60 0x7f7fb8069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40af000 tx=0x7f7fc40a7000 comp rx=0 tx=0).stop 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 -- 172.21.15.113:0/189478801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 msgr2=0x7f7fd4148520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 0x7f7fd4148520 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c450 tx=0x7f7fc800c910 comp rx=0 tx=0).stop 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 -- 172.21.15.113:0/189478801 shutdown_connections 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fd4152bd0 0x7f7fd414e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fd41527f0 0x7f7fd414dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fb8066f60 0x7f7fb8069420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 --2- 172.21.15.113:0/189478801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fd40c8200 0x7f7fd4148520 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 -- 172.21.15.113:0/189478801 >> 172.21.15.113:0/189478801 conn(0x7f7fd406e820 msgr2=0x7f7fd4158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 -- 172.21.15.113:0/189478801 shutdown_connections 2024-03-04T07:37:39.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:39.034+0000 7f7fdc800640 1 -- 172.21.15.113:0/189478801 wait complete. 2024-03-04T07:37:39.057 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:37:40.656 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/30 file after the snapshot 2024-03-04T07:37:40.658 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/31 file after the snapshot 2024-03-04T07:37:40.658 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/32 file after the snapshot 2024-03-04T07:37:40.658 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/33 file after the snapshot 2024-03-04T07:37:40.658 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/34 file after the snapshot 2024-03-04T07:37:40.659 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/35 file after the snapshot 2024-03-04T07:37:40.659 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/36 file after the snapshot 2024-03-04T07:37:40.659 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/37 file after the snapshot 2024-03-04T07:37:40.659 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/38 file after the snapshot 2024-03-04T07:37:40.659 INFO:tasks.workunit.client.0.smithi113.stdout: create 236/39 file after the snapshot 2024-03-04T07:37:40.659 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/.snap/snap-subdir-test 2024-03-04T07:37:40.699 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/30 file after the snapshot 2024-03-04T07:37:40.702 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/31 file after the snapshot 2024-03-04T07:37:40.702 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/32 file after the snapshot 2024-03-04T07:37:40.702 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/33 file after the snapshot 2024-03-04T07:37:40.702 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/34 file after the snapshot 2024-03-04T07:37:40.702 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/35 file after the snapshot 2024-03-04T07:37:40.702 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/36 file after the snapshot 2024-03-04T07:37:40.702 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/37 file after the snapshot 2024-03-04T07:37:40.702 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/38 file after the snapshot 2024-03-04T07:37:40.703 INFO:tasks.workunit.client.0.smithi113.stdout: create 237/39 file after the snapshot 2024-03-04T07:37:40.703 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/.snap/snap-subdir-test 2024-03-04T07:37:40.749 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/30 file after the snapshot 2024-03-04T07:37:40.752 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/31 file after the snapshot 2024-03-04T07:37:40.753 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/32 file after the snapshot 2024-03-04T07:37:40.753 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/33 file after the snapshot 2024-03-04T07:37:40.753 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/34 file after the snapshot 2024-03-04T07:37:40.753 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/35 file after the snapshot 2024-03-04T07:37:40.753 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/36 file after the snapshot 2024-03-04T07:37:40.753 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/37 file after the snapshot 2024-03-04T07:37:40.753 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/38 file after the snapshot 2024-03-04T07:37:40.754 INFO:tasks.workunit.client.0.smithi113.stdout: create 238/39 file after the snapshot 2024-03-04T07:37:40.754 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/.snap/snap-subdir-test 2024-03-04T07:37:40.798 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/30 file after the snapshot 2024-03-04T07:37:40.801 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/31 file after the snapshot 2024-03-04T07:37:40.801 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/32 file after the snapshot 2024-03-04T07:37:40.801 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/33 file after the snapshot 2024-03-04T07:37:40.801 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/34 file after the snapshot 2024-03-04T07:37:40.801 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/35 file after the snapshot 2024-03-04T07:37:40.801 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/36 file after the snapshot 2024-03-04T07:37:40.802 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/37 file after the snapshot 2024-03-04T07:37:40.802 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/38 file after the snapshot 2024-03-04T07:37:40.802 INFO:tasks.workunit.client.0.smithi113.stdout: create 239/39 file after the snapshot 2024-03-04T07:37:40.802 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/.snap/snap-subdir-test 2024-03-04T07:37:40.850 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/30 file after the snapshot 2024-03-04T07:37:41.058 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:41.122 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/31 file after the snapshot 2024-03-04T07:37:41.122 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/32 file after the snapshot 2024-03-04T07:37:41.122 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/33 file after the snapshot 2024-03-04T07:37:41.122 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/34 file after the snapshot 2024-03-04T07:37:41.123 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/35 file after the snapshot 2024-03-04T07:37:41.123 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/36 file after the snapshot 2024-03-04T07:37:41.123 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/37 file after the snapshot 2024-03-04T07:37:41.123 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/38 file after the snapshot 2024-03-04T07:37:41.123 INFO:tasks.workunit.client.0.smithi113.stdout: create 240/39 file after the snapshot 2024-03-04T07:37:41.123 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/.snap/snap-subdir-test 2024-03-04T07:37:41.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.190+0000 7f5cf4800640 1 Processor -- start 2024-03-04T07:37:41.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.190+0000 7f5cf4800640 1 -- start start 2024-03-04T07:37:41.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf4800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5cec15b900 0x7f5cec163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf4800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec0c27f0 con 0x7f5cec0c22e0 2024-03-04T07:37:41.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf4800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec0c2a90 con 0x7f5cec14e4c0 2024-03-04T07:37:41.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf4800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cec15ad60 con 0x7f5cec15b900 2024-03-04T07:37:41.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5cec15b900 0x7f5cec163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf1400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5cec0c22e0 0x7f5cec15b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36676/0 (socket says 172.21.15.113:36676) 2024-03-04T07:37:41.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf1400640 1 -- 172.21.15.113:0/2054837773 learned_addr learned my addr 172.21.15.113:0/2054837773 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:37:41.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf2800640 1 -- 172.21.15.113:0/2054837773 >> v1:172.21.15.136:6789/0 conn(0x7f5cec14e4c0 legacy=0x7f5cec072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf2800640 1 -- 172.21.15.113:0/2054837773 >> v1:172.21.15.113:6789/0 conn(0x7f5cec0c22e0 legacy=0x7f5cec15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf0a00640 1 -- 172.21.15.113:0/2054837773 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158638460 0 0) 0x7f5cec0c27f0 con 0x7f5cec0c22e0 2024-03-04T07:37:41.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.194+0000 7f5cf2800640 1 -- 172.21.15.113:0/2054837773 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec070c60 con 0x7f5cec15b900 2024-03-04T07:37:41.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf2800640 1 --2- 172.21.15.113:0/2054837773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec15b900 0x7f5cec163100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009d90 tx=0x7f5ce801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea368429999f1a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:41.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf0a00640 1 -- 172.21.15.113:0/2054837773 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80b4070 con 0x7f5cec15b900 2024-03-04T07:37:41.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf0a00640 1 -- 172.21.15.113:0/2054837773 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce80af600 con 0x7f5cec15b900 2024-03-04T07:37:41.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf0a00640 1 -- 172.21.15.113:0/2054837773 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80af9d0 con 0x7f5cec15b900 2024-03-04T07:37:41.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf4800640 1 -- 172.21.15.113:0/2054837773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec15b900 msgr2=0x7f5cec163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf4800640 1 --2- 172.21.15.113:0/2054837773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec15b900 0x7f5cec163100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009d90 tx=0x7f5ce801f150 comp rx=0 tx=0).stop 2024-03-04T07:37:41.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf4800640 1 -- 172.21.15.113:0/2054837773 shutdown_connections 2024-03-04T07:37:41.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf4800640 1 --2- 172.21.15.113:0/2054837773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec15b900 0x7f5cec163100 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf4800640 1 -- 172.21.15.113:0/2054837773 >> 172.21.15.113:0/2054837773 conn(0x7f5cec06e820 msgr2=0x7f5cec06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:41.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf4800640 1 -- 172.21.15.113:0/2054837773 shutdown_connections 2024-03-04T07:37:41.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.198+0000 7f5cf4800640 1 -- 172.21.15.113:0/2054837773 wait complete. 2024-03-04T07:37:41.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf4800640 1 Processor -- start 2024-03-04T07:37:41.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf4800640 1 -- start start 2024-03-04T07:37:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf4800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec0c22e0 0x7f5cec152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf4800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cec14e4c0 0x7f5cec14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf4800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec15b900 0x7f5cec159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf4800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec071500 con 0x7f5cec0c22e0 2024-03-04T07:37:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf4800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec071220 con 0x7f5cec15b900 2024-03-04T07:37:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf4800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec14e8a0 con 0x7f5cec14e4c0 2024-03-04T07:37:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec0c22e0 0x7f5cec152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec0c22e0 0x7f5cec152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52174/0 (socket says 172.21.15.113:52174) 2024-03-04T07:37:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1e00640 1 -- 172.21.15.113:0/412552478 learned_addr learned my addr 172.21.15.113:0/412552478 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf2800640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec15b900 0x7f5cec159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1400640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cec14e4c0 0x7f5cec14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1e00640 1 -- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cec14e4c0 msgr2=0x7f5cec14bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1e00640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cec14e4c0 0x7f5cec14bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1e00640 1 -- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec15b900 msgr2=0x7f5cec159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1e00640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec15b900 0x7f5cec159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1e00640 1 -- 172.21.15.113:0/412552478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec1589d0 con 0x7f5cec0c22e0 2024-03-04T07:37:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1400640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cec14e4c0 0x7f5cec14bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:37:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf1e00640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec0c22e0 0x7f5cec152060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c530 tx=0x7f5ce000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5ce6000640 1 -- 172.21.15.113:0/412552478 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0016020 con 0x7f5cec0c22e0 2024-03-04T07:37:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5ce6000640 1 -- 172.21.15.113:0/412552478 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce0005220 con 0x7f5cec0c22e0 2024-03-04T07:37:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf4800640 1 -- 172.21.15.113:0/412552478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec158c70 con 0x7f5cec0c22e0 2024-03-04T07:37:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5ce6000640 1 -- 172.21.15.113:0/412552478 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0007030 con 0x7f5cec0c22e0 2024-03-04T07:37:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.202+0000 7f5cf4800640 1 -- 172.21.15.113:0/412552478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec159230 con 0x7f5cec0c22e0 2024-03-04T07:37:41.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.206+0000 7f5cf4800640 1 -- 172.21.15.113:0/412552478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec075c00 con 0x7f5cec0c22e0 2024-03-04T07:37:41.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.206+0000 7f5ce6000640 1 -- 172.21.15.113:0/412552478 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ce00071d0 con 0x7f5cec0c22e0 2024-03-04T07:37:41.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.206+0000 7f5ce6000640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cd0066f60 0x7f5cd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.206+0000 7f5ce6000640 1 -- 172.21.15.113:0/412552478 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f5ce0084a10 con 0x7f5cec0c22e0 2024-03-04T07:37:41.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.206+0000 7f5cf1400640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cd0066f60 0x7f5cd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.210+0000 7f5cf1400640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cd0066f60 0x7f5cd0069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5cec155160 tx=0x7f5cdc007dd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:41.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.210+0000 7f5ce6000640 1 -- 172.21.15.113:0/412552478 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5ce0020020 con 0x7f5cec0c22e0 2024-03-04T07:37:41.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 -- 172.21.15.113:0/412552478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cec155190 con 0x7f5cec0c22e0 2024-03-04T07:37:41.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5ce6000640 1 -- 172.21.15.113:0/412552478 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f5ce00074c0 con 0x7f5cec0c22e0 2024-03-04T07:37:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:37:40.596640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:37:41.404 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 20 2024-03-04T07:37:41.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 -- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cd0066f60 msgr2=0x7f5cd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cd0066f60 0x7f5cd0069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5cec155160 tx=0x7f5cdc007dd0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 -- 172.21.15.113:0/412552478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec0c22e0 msgr2=0x7f5cec152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec0c22e0 0x7f5cec152060 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000c530 tx=0x7f5ce000c9f0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 -- 172.21.15.113:0/412552478 shutdown_connections 2024-03-04T07:37:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cec15b900 0x7f5cec159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5cec14e4c0 0x7f5cec14bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cd0066f60 0x7f5cd0069420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 --2- 172.21.15.113:0/412552478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cec0c22e0 0x7f5cec152060 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 -- 172.21.15.113:0/412552478 >> 172.21.15.113:0/412552478 conn(0x7f5cec06e820 msgr2=0x7f5cec0c8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 -- 172.21.15.113:0/412552478 shutdown_connections 2024-03-04T07:37:41.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.402+0000 7f5cf4800640 1 -- 172.21.15.113:0/412552478 wait complete. 2024-03-04T07:37:41.425 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-03-04T07:37:41.425 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:37:41.426 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:41.457 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/30 file after the snapshot 2024-03-04T07:37:41.460 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/31 file after the snapshot 2024-03-04T07:37:41.460 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/32 file after the snapshot 2024-03-04T07:37:41.460 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/33 file after the snapshot 2024-03-04T07:37:41.460 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/34 file after the snapshot 2024-03-04T07:37:41.460 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/35 file after the snapshot 2024-03-04T07:37:41.461 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/36 file after the snapshot 2024-03-04T07:37:41.461 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/37 file after the snapshot 2024-03-04T07:37:41.461 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/38 file after the snapshot 2024-03-04T07:37:41.461 INFO:tasks.workunit.client.0.smithi113.stdout: create 241/39 file after the snapshot 2024-03-04T07:37:41.461 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/.snap/snap-subdir-test 2024-03-04T07:37:41.500 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/30 file after the snapshot 2024-03-04T07:37:41.503 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/31 file after the snapshot 2024-03-04T07:37:41.503 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/32 file after the snapshot 2024-03-04T07:37:41.503 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/33 file after the snapshot 2024-03-04T07:37:41.503 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/34 file after the snapshot 2024-03-04T07:37:41.503 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/35 file after the snapshot 2024-03-04T07:37:41.503 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/36 file after the snapshot 2024-03-04T07:37:41.503 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/37 file after the snapshot 2024-03-04T07:37:41.503 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/38 file after the snapshot 2024-03-04T07:37:41.504 INFO:tasks.workunit.client.0.smithi113.stdout: create 242/39 file after the snapshot 2024-03-04T07:37:41.504 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/.snap/snap-subdir-test 2024-03-04T07:37:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.562+0000 7f618f200640 1 Processor -- start 2024-03-04T07:37:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618f200640 1 -- start start 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618f200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619015b450 0x7f61900c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618f200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619015b850 con 0x7f6190154c40 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618f200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61900cb4c0 con 0x7f61900d0030 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618f200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6190155020 con 0x7f619015b450 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618de00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6190154c40 0x7f61900cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36692/0 (socket says 172.21.15.113:36692) 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618de00640 1 -- 172.21.15.113:0/3622264337 learned_addr learned my addr 172.21.15.113:0/3622264337 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f6187e00640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619015b450 0x7f61900c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654578023 0 0) 0x7f6180001120 con 0x7f6190154c40 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6174003530 con 0x7f6190154c40 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3455680029 0 0) 0x7f6174003530 con 0x7f6190154c40 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619015b450 msgr2=0x7f61900c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619015b450 0x7f61900c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 >> v1:172.21.15.136:6789/0 conn(0x7f61900d0030 legacy=0x7f61901651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619015df00 con 0x7f6190154c40 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f6180001850 con 0x7f6190154c40 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 >> v1:172.21.15.113:6789/0 conn(0x7f6190154c40 legacy=0x7f61900cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61740039f0 0x7f6174003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619015b450 0x7f6174004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174004d50 0x7f61740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6190155020 con 0x7f61740039f0 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617400b9d0 con 0x7f6174004d50 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617400bbd0 con 0x7f619015b450 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618de00640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61740039f0 0x7f6174003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618e800640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174004d50 0x7f61740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f6187e00640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619015b450 0x7f6174004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618de00640 1 -- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619015b450 msgr2=0x7f6174004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:41.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618de00640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619015b450 0x7f6174004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618de00640 1 -- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174004d50 msgr2=0x7f61740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618de00640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6174004d50 0x7f61740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.566+0000 7f618de00640 1 -- 172.21.15.113:0/3622264337 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6174003530 con 0x7f61740039f0 2024-03-04T07:37:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618de00640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61740039f0 0x7f6174003dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f618000c4c0 tx=0x7f618001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a337ca84bd17bc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000c0e0 con 0x7f61740039f0 2024-03-04T07:37:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61800a9070 con 0x7f61740039f0 2024-03-04T07:37:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618d400640 1 -- 172.21.15.113:0/3622264337 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180005d20 con 0x7f61740039f0 2024-03-04T07:37:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618f200640 1 -- 172.21.15.113:0/3622264337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61740039f0 msgr2=0x7f6174003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618f200640 1 --2- 172.21.15.113:0/3622264337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61740039f0 0x7f6174003dd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f618000c4c0 tx=0x7f618001cbe0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618e800640 1 -- 172.21.15.113:0/3622264337 reap_dead start 2024-03-04T07:37:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618f200640 1 -- 172.21.15.113:0/3622264337 shutdown_connections 2024-03-04T07:37:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618f200640 1 -- 172.21.15.113:0/3622264337 >> 172.21.15.113:0/3622264337 conn(0x7f619006d3e0 msgr2=0x7f619006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618f200640 1 -- 172.21.15.113:0/3622264337 shutdown_connections 2024-03-04T07:37:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.570+0000 7f618f200640 1 -- 172.21.15.113:0/3622264337 wait complete. 2024-03-04T07:37:41.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618f200640 1 Processor -- start 2024-03-04T07:37:41.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618f200640 1 -- start start 2024-03-04T07:37:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618f200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61740039f0 0x7f6190158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61900c6a90 0x7f61901e7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f619015aa00 0x7f61900c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619015e7a0 con 0x7f61900c6a90 2024-03-04T07:37:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619015e4c0 con 0x7f619015aa00 2024-03-04T07:37:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618f200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619015ecc0 con 0x7f61740039f0 2024-03-04T07:37:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6187e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61900c6a90 0x7f61901e7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6187e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61900c6a90 0x7f61901e7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52192/0 (socket says 172.21.15.113:52192) 2024-03-04T07:37:41.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6187e00640 1 -- 172.21.15.113:0/2006991543 learned_addr learned my addr 172.21.15.113:0/2006991543 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:41.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618e800640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f619015aa00 0x7f61900c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618de00640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61740039f0 0x7f6190158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6187e00640 1 -- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61740039f0 msgr2=0x7f6190158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6187e00640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61740039f0 0x7f6190158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6187e00640 1 -- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f619015aa00 msgr2=0x7f61900c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6187e00640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f619015aa00 0x7f61900c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6187e00640 1 -- 172.21.15.113:0/2006991543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6190150d50 con 0x7f61900c6a90 2024-03-04T07:37:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6187e00640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61900c6a90 0x7f61901e7000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6178011a00 tx=0x7f6178011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6186000640 1 -- 172.21.15.113:0/2006991543 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617800fb50 con 0x7f61900c6a90 2024-03-04T07:37:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6186000640 1 -- 172.21.15.113:0/2006991543 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617800fcf0 con 0x7f61900c6a90 2024-03-04T07:37:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618f200640 1 -- 172.21.15.113:0/2006991543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61901f0480 con 0x7f61900c6a90 2024-03-04T07:37:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f6186000640 1 -- 172.21.15.113:0/2006991543 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617800fea0 con 0x7f61900c6a90 2024-03-04T07:37:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.574+0000 7f618f200640 1 -- 172.21.15.113:0/2006991543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61901f0a20 con 0x7f61900c6a90 2024-03-04T07:37:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.578+0000 7f618f200640 1 -- 172.21.15.113:0/2006991543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6190075a80 con 0x7f61900c6a90 2024-03-04T07:37:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.578+0000 7f6186000640 1 -- 172.21.15.113:0/2006991543 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6178012040 con 0x7f61900c6a90 2024-03-04T07:37:41.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.578+0000 7f6186000640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f616c066f90 0x7f616c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:41.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.578+0000 7f6186000640 1 -- 172.21.15.113:0/2006991543 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f6178085510 con 0x7f61900c6a90 2024-03-04T07:37:41.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.578+0000 7f618de00640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f616c066f90 0x7f616c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:41.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.578+0000 7f618de00640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f616c066f90 0x7f616c069450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f61800ab000 tx=0x7f61800b1040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:41.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.586+0000 7f6186000640 1 -- 172.21.15.113:0/2006991543 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6178019030 con 0x7f61900c6a90 2024-03-04T07:37:41.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 -- 172.21.15.113:0/2006991543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61900b2a70 con 0x7f61900c6a90 2024-03-04T07:37:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f6186000640 1 -- 172.21.15.113:0/2006991543 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f6178055f00 con 0x7f61900c6a90 2024-03-04T07:37:41.781 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:41.781 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:37:40.596640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:37:41.781 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 20 2024-03-04T07:37:41.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 -- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f616c066f90 msgr2=0x7f616c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f616c066f90 0x7f616c069450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f61800ab000 tx=0x7f61800b1040 comp rx=0 tx=0).stop 2024-03-04T07:37:41.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 -- 172.21.15.113:0/2006991543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61900c6a90 msgr2=0x7f61901e7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:41.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61900c6a90 0x7f61901e7000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6178011a00 tx=0x7f6178011ec0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 -- 172.21.15.113:0/2006991543 shutdown_connections 2024-03-04T07:37:41.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f619015aa00 0x7f61900c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f616c066f90 0x7f616c069450 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61900c6a90 0x7f61901e7000 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 --2- 172.21.15.113:0/2006991543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61740039f0 0x7f6190158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:41.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.778+0000 7f618f200640 1 -- 172.21.15.113:0/2006991543 >> 172.21.15.113:0/2006991543 conn(0x7f619006d3e0 msgr2=0x7f6190152420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:41.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.782+0000 7f618f200640 1 -- 172.21.15.113:0/2006991543 shutdown_connections 2024-03-04T07:37:41.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:41.782+0000 7f618f200640 1 -- 172.21.15.113:0/2006991543 wait complete. 2024-03-04T07:37:41.802 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 20, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 20, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4256": { "addr": "172.21.15.113:6837/1310042629", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1310042629, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1310042629, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4256, "incarnation": 10, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4276": { "addr": "172.21.15.136:6835/3348592059", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3348592059, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3348592059, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4276, "incarnation": 4, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4938": { "addr": "172.21.15.113:6836/2997246507", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2997246507, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2997246507, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4938, "incarnation": 20, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:37:40.596640+0000", "qdb_cluster": [ 4256, 4276 ], "qdb_leader": 4276, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4276, "mds_1": 4256, "mds_2": 4938 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:37:42.423 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/30 file after the snapshot 2024-03-04T07:37:42.425 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/31 file after the snapshot 2024-03-04T07:37:42.426 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/32 file after the snapshot 2024-03-04T07:37:42.426 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/33 file after the snapshot 2024-03-04T07:37:42.426 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/34 file after the snapshot 2024-03-04T07:37:42.426 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/35 file after the snapshot 2024-03-04T07:37:42.427 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/36 file after the snapshot 2024-03-04T07:37:42.427 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/37 file after the snapshot 2024-03-04T07:37:42.427 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/38 file after the snapshot 2024-03-04T07:37:42.427 INFO:tasks.workunit.client.0.smithi113.stdout: create 243/39 file after the snapshot 2024-03-04T07:37:42.427 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/.snap/snap-subdir-test 2024-03-04T07:37:42.478 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/30 file after the snapshot 2024-03-04T07:37:42.479 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/31 file after the snapshot 2024-03-04T07:37:42.479 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/32 file after the snapshot 2024-03-04T07:37:42.479 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/33 file after the snapshot 2024-03-04T07:37:42.480 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/34 file after the snapshot 2024-03-04T07:37:42.480 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/35 file after the snapshot 2024-03-04T07:37:42.480 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/36 file after the snapshot 2024-03-04T07:37:42.480 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/37 file after the snapshot 2024-03-04T07:37:42.480 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/38 file after the snapshot 2024-03-04T07:37:42.480 INFO:tasks.workunit.client.0.smithi113.stdout: create 244/39 file after the snapshot 2024-03-04T07:37:42.480 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/.snap/snap-subdir-test 2024-03-04T07:37:42.515 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/30 file after the snapshot 2024-03-04T07:37:42.518 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/31 file after the snapshot 2024-03-04T07:37:42.518 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/32 file after the snapshot 2024-03-04T07:37:42.518 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/33 file after the snapshot 2024-03-04T07:37:42.518 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/34 file after the snapshot 2024-03-04T07:37:42.518 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/35 file after the snapshot 2024-03-04T07:37:42.518 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/36 file after the snapshot 2024-03-04T07:37:42.518 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/37 file after the snapshot 2024-03-04T07:37:42.519 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/38 file after the snapshot 2024-03-04T07:37:42.519 INFO:tasks.workunit.client.0.smithi113.stdout: create 245/39 file after the snapshot 2024-03-04T07:37:42.519 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/.snap/snap-subdir-test 2024-03-04T07:37:42.562 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/30 file after the snapshot 2024-03-04T07:37:42.565 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/31 file after the snapshot 2024-03-04T07:37:42.565 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/32 file after the snapshot 2024-03-04T07:37:42.565 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/33 file after the snapshot 2024-03-04T07:37:42.565 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/34 file after the snapshot 2024-03-04T07:37:42.565 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/35 file after the snapshot 2024-03-04T07:37:42.566 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/36 file after the snapshot 2024-03-04T07:37:42.566 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/37 file after the snapshot 2024-03-04T07:37:42.566 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/38 file after the snapshot 2024-03-04T07:37:42.566 INFO:tasks.workunit.client.0.smithi113.stdout: create 246/39 file after the snapshot 2024-03-04T07:37:42.566 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/.snap/snap-subdir-test 2024-03-04T07:37:43.107 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/30 file after the snapshot 2024-03-04T07:37:43.361 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/31 file after the snapshot 2024-03-04T07:37:43.361 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/32 file after the snapshot 2024-03-04T07:37:43.361 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/33 file after the snapshot 2024-03-04T07:37:43.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/34 file after the snapshot 2024-03-04T07:37:43.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/35 file after the snapshot 2024-03-04T07:37:43.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/36 file after the snapshot 2024-03-04T07:37:43.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/37 file after the snapshot 2024-03-04T07:37:43.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/38 file after the snapshot 2024-03-04T07:37:43.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 247/39 file after the snapshot 2024-03-04T07:37:43.362 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/.snap/snap-subdir-test 2024-03-04T07:37:43.408 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/30 file after the snapshot 2024-03-04T07:37:43.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/31 file after the snapshot 2024-03-04T07:37:43.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/32 file after the snapshot 2024-03-04T07:37:43.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/33 file after the snapshot 2024-03-04T07:37:43.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/34 file after the snapshot 2024-03-04T07:37:43.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/35 file after the snapshot 2024-03-04T07:37:43.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/36 file after the snapshot 2024-03-04T07:37:43.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/37 file after the snapshot 2024-03-04T07:37:43.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/38 file after the snapshot 2024-03-04T07:37:43.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 248/39 file after the snapshot 2024-03-04T07:37:43.411 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/.snap/snap-subdir-test 2024-03-04T07:37:43.619 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/30 file after the snapshot 2024-03-04T07:37:43.803 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:43.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd06a400640 1 Processor -- start 2024-03-04T07:37:43.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd06a400640 1 -- start start 2024-03-04T07:37:43.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd06a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd06414e4c0 0x7fd064072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:43.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd06a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd06415b900 0x7fd0640731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:43.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd06a400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0640c27f0 con 0x7fd0640c22e0 2024-03-04T07:37:43.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd06a400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06415ad60 con 0x7fd06415b900 2024-03-04T07:37:43.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd06a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640c2a90 con 0x7fd06414e4c0 2024-03-04T07:37:43.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd062a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd0640c22e0 0x7fd06415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36704/0 (socket says 172.21.15.113:36704) 2024-03-04T07:37:43.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd062a00640 1 -- 172.21.15.113:0/835658069 learned_addr learned my addr 172.21.15.113:0/835658069 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:37:43.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd063e00640 1 --2- 172.21.15.113:0/835658069 >> v2:172.21.15.113:3300/0 conn(0x7fd06415b900 0x7fd0640731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:43.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd063400640 1 --2- 172.21.15.113:0/835658069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd06414e4c0 0x7fd064072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:43.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd062000640 1 -- 172.21.15.113:0/835658069 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680810340 0 0) 0x7fd0640c27f0 con 0x7fd0640c22e0 2024-03-04T07:37:43.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd063e00640 1 -- 172.21.15.113:0/835658069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd06414e4c0 msgr2=0x7fd064072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:43.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd063e00640 1 --2- 172.21.15.113:0/835658069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd06414e4c0 0x7fd064072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:43.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd063e00640 1 -- 172.21.15.113:0/835658069 >> v1:172.21.15.113:6789/0 conn(0x7fd0640c22e0 legacy=0x7fd06415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:43.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.938+0000 7fd063e00640 1 -- 172.21.15.113:0/835658069 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064070c60 con 0x7fd06415b900 2024-03-04T07:37:43.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd063e00640 1 --2- 172.21.15.113:0/835658069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 0x7fd0640731c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009d90 tx=0x7fd05c01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9e41393f20c036 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:43.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd062000640 1 -- 172.21.15.113:0/835658069 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0b4070 con 0x7fd06415b900 2024-03-04T07:37:43.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd062000640 1 -- 172.21.15.113:0/835658069 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c0af600 con 0x7fd06415b900 2024-03-04T07:37:43.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd062000640 1 -- 172.21.15.113:0/835658069 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0af9d0 con 0x7fd06415b900 2024-03-04T07:37:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd06a400640 1 -- 172.21.15.113:0/835658069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 msgr2=0x7fd0640731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd06a400640 1 --2- 172.21.15.113:0/835658069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 0x7fd0640731c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009d90 tx=0x7fd05c01f150 comp rx=0 tx=0).stop 2024-03-04T07:37:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd06a400640 1 -- 172.21.15.113:0/835658069 shutdown_connections 2024-03-04T07:37:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd06a400640 1 --2- 172.21.15.113:0/835658069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 0x7fd0640731c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd06a400640 1 --2- 172.21.15.113:0/835658069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd06414e4c0 0x7fd064072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd06a400640 1 -- 172.21.15.113:0/835658069 >> 172.21.15.113:0/835658069 conn(0x7fd06406e820 msgr2=0x7fd06406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd06a400640 1 -- 172.21.15.113:0/835658069 shutdown_connections 2024-03-04T07:37:43.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd06a400640 1 -- 172.21.15.113:0/835658069 wait complete. 2024-03-04T07:37:43.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd06a400640 1 Processor -- start 2024-03-04T07:37:43.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.942+0000 7fd06a400640 1 -- start start 2024-03-04T07:37:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd06a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0640c22e0 0x7fd0641ef300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd06a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd06414e4c0 0x7fd0640c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd06a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 0x7fd064145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:43.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd06a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064071500 con 0x7fd06415b900 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd06a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064071220 con 0x7fd0640c22e0 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd06a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06414e8a0 con 0x7fd06414e4c0 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd063e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 0x7fd064145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd063e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 0x7fd064145720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52198/0 (socket says 172.21.15.113:52198) 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd063e00640 1 -- 172.21.15.113:0/4161806944 learned_addr learned my addr 172.21.15.113:0/4161806944 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd062a00640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd06414e4c0 0x7fd0640c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd063400640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0640c22e0 0x7fd0641ef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd063e00640 1 -- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd06414e4c0 msgr2=0x7fd0640c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd063e00640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd06414e4c0 0x7fd0640c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd063e00640 1 -- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0640c22e0 msgr2=0x7fd0641ef300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd063e00640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0640c22e0 0x7fd0641ef300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd063e00640 1 -- 172.21.15.113:0/4161806944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0641522c0 con 0x7fd06415b900 2024-03-04T07:37:43.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd063e00640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 0x7fd064145720 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009d90 tx=0x7fd05c01bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd05b400640 1 -- 172.21.15.113:0/4161806944 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c0b4040 con 0x7fd06415b900 2024-03-04T07:37:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd05b400640 1 -- 172.21.15.113:0/4161806944 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05c0039a0 con 0x7fd06415b900 2024-03-04T07:37:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd06a400640 1 -- 172.21.15.113:0/4161806944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0641525a0 con 0x7fd06415b900 2024-03-04T07:37:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd05b400640 1 -- 172.21.15.113:0/4161806944 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05c009450 con 0x7fd06415b900 2024-03-04T07:37:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd06a400640 1 -- 172.21.15.113:0/4161806944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0641eeb00 con 0x7fd06415b900 2024-03-04T07:37:43.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.946+0000 7fd06a400640 1 -- 172.21.15.113:0/4161806944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd064075c00 con 0x7fd06415b900 2024-03-04T07:37:43.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.950+0000 7fd05b400640 1 -- 172.21.15.113:0/4161806944 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd05c0c4060 con 0x7fd06415b900 2024-03-04T07:37:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.950+0000 7fd05b400640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd044066f60 0x7fd044069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.950+0000 7fd05b400640 1 -- 172.21.15.113:0/4161806944 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fd05c122a10 con 0x7fd06415b900 2024-03-04T07:37:43.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.950+0000 7fd063400640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd044066f60 0x7fd044069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:43.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.950+0000 7fd063400640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd044066f60 0x7fd044069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd054009ac0 tx=0x7fd054009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:43.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:43.958+0000 7fd05b400640 1 -- 172.21.15.113:0/4161806944 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd05c018cb0 con 0x7fd06415b900 2024-03-04T07:37:43.989 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/31 file after the snapshot 2024-03-04T07:37:43.989 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/32 file after the snapshot 2024-03-04T07:37:43.989 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/33 file after the snapshot 2024-03-04T07:37:43.989 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/34 file after the snapshot 2024-03-04T07:37:43.989 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/35 file after the snapshot 2024-03-04T07:37:43.990 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/36 file after the snapshot 2024-03-04T07:37:43.990 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/37 file after the snapshot 2024-03-04T07:37:43.990 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/38 file after the snapshot 2024-03-04T07:37:43.990 INFO:tasks.workunit.client.0.smithi113.stdout: create 249/39 file after the snapshot 2024-03-04T07:37:43.990 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/.snap/snap-subdir-test 2024-03-04T07:37:44.041 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/30 file after the snapshot 2024-03-04T07:37:44.044 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/31 file after the snapshot 2024-03-04T07:37:44.044 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/32 file after the snapshot 2024-03-04T07:37:44.044 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/33 file after the snapshot 2024-03-04T07:37:44.044 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/34 file after the snapshot 2024-03-04T07:37:44.044 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/35 file after the snapshot 2024-03-04T07:37:44.044 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/36 file after the snapshot 2024-03-04T07:37:44.045 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/37 file after the snapshot 2024-03-04T07:37:44.045 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/38 file after the snapshot 2024-03-04T07:37:44.045 INFO:tasks.workunit.client.0.smithi113.stdout: create 250/39 file after the snapshot 2024-03-04T07:37:44.045 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/.snap/snap-subdir-test 2024-03-04T07:37:44.097 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/30 file after the snapshot 2024-03-04T07:37:44.100 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/31 file after the snapshot 2024-03-04T07:37:44.100 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/32 file after the snapshot 2024-03-04T07:37:44.100 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/33 file after the snapshot 2024-03-04T07:37:44.100 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/34 file after the snapshot 2024-03-04T07:37:44.100 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/35 file after the snapshot 2024-03-04T07:37:44.101 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/36 file after the snapshot 2024-03-04T07:37:44.101 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/37 file after the snapshot 2024-03-04T07:37:44.101 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/38 file after the snapshot 2024-03-04T07:37:44.101 INFO:tasks.workunit.client.0.smithi113.stdout: create 251/39 file after the snapshot 2024-03-04T07:37:44.101 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/.snap/snap-subdir-test 2024-03-04T07:37:44.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 -- 172.21.15.113:0/4161806944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd064155190 con 0x7fd06415b900 2024-03-04T07:37:44.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd05b400640 1 -- 172.21.15.113:0/4161806944 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+5815 (secure 0 0 0) 0x7fd05c0f3210 con 0x7fd06415b900 2024-03-04T07:37:44.156 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:44.156 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:37:40.596640+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:37:44.157 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 20 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 -- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd044066f60 msgr2=0x7fd044069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd044066f60 0x7fd044069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd054009ac0 tx=0x7fd054009190 comp rx=0 tx=0).stop 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 -- 172.21.15.113:0/4161806944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 msgr2=0x7fd064145720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 0x7fd064145720 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009d90 tx=0x7fd05c01bef0 comp rx=0 tx=0).stop 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 -- 172.21.15.113:0/4161806944 shutdown_connections 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd044066f60 0x7fd044069420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd06415b900 0x7fd064145720 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd06414e4c0 0x7fd0640c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 --2- 172.21.15.113:0/4161806944 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0640c22e0 0x7fd0641ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 -- 172.21.15.113:0/4161806944 >> 172.21.15.113:0/4161806944 conn(0x7fd06406e820 msgr2=0x7fd06406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 -- 172.21.15.113:0/4161806944 shutdown_connections 2024-03-04T07:37:44.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:44.154+0000 7fd06a400640 1 -- 172.21.15.113:0/4161806944 wait complete. 2024-03-04T07:37:44.177 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:37:44.414 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/30 file after the snapshot 2024-03-04T07:37:44.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/31 file after the snapshot 2024-03-04T07:37:44.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/32 file after the snapshot 2024-03-04T07:37:44.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/33 file after the snapshot 2024-03-04T07:37:44.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/34 file after the snapshot 2024-03-04T07:37:44.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/35 file after the snapshot 2024-03-04T07:37:44.417 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/36 file after the snapshot 2024-03-04T07:37:44.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/37 file after the snapshot 2024-03-04T07:37:44.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/38 file after the snapshot 2024-03-04T07:37:44.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 252/39 file after the snapshot 2024-03-04T07:37:44.418 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/.snap/snap-subdir-test 2024-03-04T07:37:44.476 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/30 file after the snapshot 2024-03-04T07:37:44.478 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/31 file after the snapshot 2024-03-04T07:37:44.478 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/32 file after the snapshot 2024-03-04T07:37:44.478 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/33 file after the snapshot 2024-03-04T07:37:44.478 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/34 file after the snapshot 2024-03-04T07:37:44.478 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/35 file after the snapshot 2024-03-04T07:37:44.479 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/36 file after the snapshot 2024-03-04T07:37:44.479 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/37 file after the snapshot 2024-03-04T07:37:44.479 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/38 file after the snapshot 2024-03-04T07:37:44.479 INFO:tasks.workunit.client.0.smithi113.stdout: create 253/39 file after the snapshot 2024-03-04T07:37:44.479 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/.snap/snap-subdir-test 2024-03-04T07:37:44.534 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/30 file after the snapshot 2024-03-04T07:37:44.536 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/31 file after the snapshot 2024-03-04T07:37:44.536 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/32 file after the snapshot 2024-03-04T07:37:44.536 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/33 file after the snapshot 2024-03-04T07:37:44.536 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/34 file after the snapshot 2024-03-04T07:37:44.536 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/35 file after the snapshot 2024-03-04T07:37:44.537 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/36 file after the snapshot 2024-03-04T07:37:44.537 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/37 file after the snapshot 2024-03-04T07:37:44.537 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/38 file after the snapshot 2024-03-04T07:37:44.537 INFO:tasks.workunit.client.0.smithi113.stdout: create 254/39 file after the snapshot 2024-03-04T07:37:44.537 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/.snap/snap-subdir-test 2024-03-04T07:37:44.580 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/30 file after the snapshot 2024-03-04T07:37:44.583 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/31 file after the snapshot 2024-03-04T07:37:44.583 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/32 file after the snapshot 2024-03-04T07:37:44.583 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/33 file after the snapshot 2024-03-04T07:37:44.583 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/34 file after the snapshot 2024-03-04T07:37:44.584 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/35 file after the snapshot 2024-03-04T07:37:44.584 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/36 file after the snapshot 2024-03-04T07:37:44.584 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/37 file after the snapshot 2024-03-04T07:37:44.584 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/38 file after the snapshot 2024-03-04T07:37:44.584 INFO:tasks.workunit.client.0.smithi113.stdout: create 255/39 file after the snapshot 2024-03-04T07:37:44.584 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/.snap/snap-subdir-test 2024-03-04T07:37:44.619 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/30 file after the snapshot 2024-03-04T07:37:44.621 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/31 file after the snapshot 2024-03-04T07:37:44.622 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/32 file after the snapshot 2024-03-04T07:37:44.622 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/33 file after the snapshot 2024-03-04T07:37:44.622 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/34 file after the snapshot 2024-03-04T07:37:44.622 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/35 file after the snapshot 2024-03-04T07:37:44.622 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/36 file after the snapshot 2024-03-04T07:37:44.622 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/37 file after the snapshot 2024-03-04T07:37:44.622 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/38 file after the snapshot 2024-03-04T07:37:44.622 INFO:tasks.workunit.client.0.smithi113.stdout: create 256/39 file after the snapshot 2024-03-04T07:37:44.622 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/.snap/snap-subdir-test 2024-03-04T07:37:44.674 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/30 file after the snapshot 2024-03-04T07:37:44.678 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/31 file after the snapshot 2024-03-04T07:37:44.679 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/32 file after the snapshot 2024-03-04T07:37:44.679 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/33 file after the snapshot 2024-03-04T07:37:44.679 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/34 file after the snapshot 2024-03-04T07:37:44.679 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/35 file after the snapshot 2024-03-04T07:37:44.679 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/36 file after the snapshot 2024-03-04T07:37:44.679 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/37 file after the snapshot 2024-03-04T07:37:44.679 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/38 file after the snapshot 2024-03-04T07:37:44.679 INFO:tasks.workunit.client.0.smithi113.stdout: create 257/39 file after the snapshot 2024-03-04T07:37:44.679 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/.snap/snap-subdir-test 2024-03-04T07:37:44.726 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/30 file after the snapshot 2024-03-04T07:37:44.729 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/31 file after the snapshot 2024-03-04T07:37:44.729 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/32 file after the snapshot 2024-03-04T07:37:44.729 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/33 file after the snapshot 2024-03-04T07:37:44.729 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/34 file after the snapshot 2024-03-04T07:37:44.729 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/35 file after the snapshot 2024-03-04T07:37:44.729 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/36 file after the snapshot 2024-03-04T07:37:44.729 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/37 file after the snapshot 2024-03-04T07:37:44.729 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/38 file after the snapshot 2024-03-04T07:37:44.729 INFO:tasks.workunit.client.0.smithi113.stdout: create 258/39 file after the snapshot 2024-03-04T07:37:44.730 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/.snap/snap-subdir-test 2024-03-04T07:37:44.779 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/30 file after the snapshot 2024-03-04T07:37:44.781 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/31 file after the snapshot 2024-03-04T07:37:44.782 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/32 file after the snapshot 2024-03-04T07:37:44.782 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/33 file after the snapshot 2024-03-04T07:37:44.782 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/34 file after the snapshot 2024-03-04T07:37:44.782 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/35 file after the snapshot 2024-03-04T07:37:44.782 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/36 file after the snapshot 2024-03-04T07:37:44.782 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/37 file after the snapshot 2024-03-04T07:37:44.782 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/38 file after the snapshot 2024-03-04T07:37:44.782 INFO:tasks.workunit.client.0.smithi113.stdout: create 259/39 file after the snapshot 2024-03-04T07:37:44.782 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/.snap/snap-subdir-test 2024-03-04T07:37:44.818 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/30 file after the snapshot 2024-03-04T07:37:44.820 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/31 file after the snapshot 2024-03-04T07:37:44.820 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/32 file after the snapshot 2024-03-04T07:37:44.821 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/33 file after the snapshot 2024-03-04T07:37:44.821 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/34 file after the snapshot 2024-03-04T07:37:44.821 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/35 file after the snapshot 2024-03-04T07:37:44.821 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/36 file after the snapshot 2024-03-04T07:37:44.821 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/37 file after the snapshot 2024-03-04T07:37:44.821 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/38 file after the snapshot 2024-03-04T07:37:44.821 INFO:tasks.workunit.client.0.smithi113.stdout: create 260/39 file after the snapshot 2024-03-04T07:37:44.821 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/.snap/snap-subdir-test 2024-03-04T07:37:44.982 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/30 file after the snapshot 2024-03-04T07:37:44.986 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/31 file after the snapshot 2024-03-04T07:37:44.986 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/32 file after the snapshot 2024-03-04T07:37:44.986 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/33 file after the snapshot 2024-03-04T07:37:44.986 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/34 file after the snapshot 2024-03-04T07:37:44.986 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/35 file after the snapshot 2024-03-04T07:37:44.986 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/36 file after the snapshot 2024-03-04T07:37:44.986 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/37 file after the snapshot 2024-03-04T07:37:44.986 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/38 file after the snapshot 2024-03-04T07:37:44.986 INFO:tasks.workunit.client.0.smithi113.stdout: create 261/39 file after the snapshot 2024-03-04T07:37:44.986 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/.snap/snap-subdir-test 2024-03-04T07:37:45.643 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/30 file after the snapshot 2024-03-04T07:37:45.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/31 file after the snapshot 2024-03-04T07:37:45.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/32 file after the snapshot 2024-03-04T07:37:45.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/33 file after the snapshot 2024-03-04T07:37:45.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/34 file after the snapshot 2024-03-04T07:37:45.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/35 file after the snapshot 2024-03-04T07:37:45.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/36 file after the snapshot 2024-03-04T07:37:45.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/37 file after the snapshot 2024-03-04T07:37:45.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/38 file after the snapshot 2024-03-04T07:37:45.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 262/39 file after the snapshot 2024-03-04T07:37:45.647 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/.snap/snap-subdir-test 2024-03-04T07:37:46.178 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:46.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.310+0000 7fd295000640 1 Processor -- start 2024-03-04T07:37:46.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.310+0000 7fd295000640 1 -- start start 2024-03-04T07:37:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd295000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd2901527f0 0x7fd290070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd295000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2901591f0 con 0x7fd2900c8200 2024-03-04T07:37:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd295000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2901594d0 con 0x7fd290152bd0 2024-03-04T07:37:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd295000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd290071df0 con 0x7fd2901527f0 2024-03-04T07:37:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28d400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd2900c8200 0x7fd290072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36712/0 (socket says 172.21.15.113:36712) 2024-03-04T07:37:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28d400640 1 -- 172.21.15.113:0/512088271 learned_addr learned my addr 172.21.15.113:0/512088271 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:37:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28de00640 1 --2- 172.21.15.113:0/512088271 >> v2:172.21.15.136:3300/0 conn(0x7fd2901527f0 0x7fd290070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:46.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667206782 0 0) 0x7fd2901591f0 con 0x7fd2900c8200 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd274003530 con 0x7fd2900c8200 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 795810051 0 0) 0x7fd274003530 con 0x7fd2900c8200 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 >> v2:172.21.15.136:3300/0 conn(0x7fd2901527f0 msgr2=0x7fd290070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 --2- 172.21.15.113:0/512088271 >> v2:172.21.15.136:3300/0 conn(0x7fd2901527f0 0x7fd290070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 >> v1:172.21.15.136:6789/0 conn(0x7fd290152bd0 legacy=0x7fd2901587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900717a0 con 0x7fd2900c8200 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fd27c001680 con 0x7fd2900c8200 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 >> v1:172.21.15.113:6789/0 conn(0x7fd2900c8200 legacy=0x7fd290072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2740039c0 0x7fd274003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274004810 0x7fd274004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2740051d0 0x7fd27400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd290071df0 con 0x7fd2740051d0 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27400c070 con 0x7fd274004810 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27400c270 con 0x7fd2740039c0 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28e800640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2740051d0 0x7fd27400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28de00640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2740039c0 0x7fd274003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:46.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28d400640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274004810 0x7fd274004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:46.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28e800640 1 -- 172.21.15.113:0/512088271 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2740039c0 msgr2=0x7fd274003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:46.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28e800640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2740039c0 0x7fd274003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:46.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28e800640 1 -- 172.21.15.113:0/512088271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274004810 msgr2=0x7fd274004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:46.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28e800640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274004810 0x7fd274004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:46.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.314+0000 7fd28e800640 1 -- 172.21.15.113:0/512088271 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274003530 con 0x7fd2740051d0 2024-03-04T07:37:46.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd28e800640 1 -- 172.21.15.113:0/512088271 reap_dead start 2024-03-04T07:37:46.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd28e800640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2740051d0 0x7fd27400b9d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd288001e70 tx=0x7fd28801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580a116399d0522a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:46.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288001a90 con 0x7fd2740051d0 2024-03-04T07:37:46.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd288003ba0 con 0x7fd2740051d0 2024-03-04T07:37:46.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd28ca00640 1 -- 172.21.15.113:0/512088271 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288005020 con 0x7fd2740051d0 2024-03-04T07:37:46.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd295000640 1 -- 172.21.15.113:0/512088271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2740051d0 msgr2=0x7fd27400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:46.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd295000640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2740051d0 0x7fd27400b9d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd288001e70 tx=0x7fd28801c780 comp rx=0 tx=0).stop 2024-03-04T07:37:46.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd295000640 1 -- 172.21.15.113:0/512088271 shutdown_connections 2024-03-04T07:37:46.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd295000640 1 --2- 172.21.15.113:0/512088271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2740051d0 0x7fd27400b9d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:46.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd295000640 1 -- 172.21.15.113:0/512088271 >> 172.21.15.113:0/512088271 conn(0x7fd29006e820 msgr2=0x7fd29006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:46.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd295000640 1 -- 172.21.15.113:0/512088271 shutdown_connections 2024-03-04T07:37:46.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd295000640 1 -- 172.21.15.113:0/512088271 wait complete. 2024-03-04T07:37:46.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd295000640 1 Processor -- start 2024-03-04T07:37:46.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.318+0000 7fd295000640 1 -- start start 2024-03-04T07:37:46.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd295000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2740051d0 0x7fd290153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:46.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd295000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2900c2cf0 0x7fd2900c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:46.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd295000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2901e4df0 0x7fd29014ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd295000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd290070d40 con 0x7fd2901e4df0 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd295000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2901597b0 con 0x7fd2900c2cf0 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd295000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd290070ec0 con 0x7fd2740051d0 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2901e4df0 0x7fd29014ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2901e4df0 0x7fd29014ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52208/0 (socket says 172.21.15.113:52208) 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28e800640 1 -- 172.21.15.113:0/2578708224 learned_addr learned my addr 172.21.15.113:0/2578708224 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28de00640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2740051d0 0x7fd290153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28d400640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2900c2cf0 0x7fd2900c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28e800640 1 -- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2740051d0 msgr2=0x7fd290153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28e800640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2740051d0 0x7fd290153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28e800640 1 -- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2900c2cf0 msgr2=0x7fd2900c1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28e800640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2900c2cf0 0x7fd2900c1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28e800640 1 -- 172.21.15.113:0/2578708224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd290068660 con 0x7fd2901e4df0 2024-03-04T07:37:46.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd28e800640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2901e4df0 0x7fd29014ef40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd28801cc60 tx=0x7fd274005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:46.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd286000640 1 -- 172.21.15.113:0/2578708224 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288007dc0 con 0x7fd2901e4df0 2024-03-04T07:37:46.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd286000640 1 -- 172.21.15.113:0/2578708224 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd288001260 con 0x7fd2901e4df0 2024-03-04T07:37:46.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd295000640 1 -- 172.21.15.113:0/2578708224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2901567a0 con 0x7fd2901e4df0 2024-03-04T07:37:46.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd286000640 1 -- 172.21.15.113:0/2578708224 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288006b60 con 0x7fd2901e4df0 2024-03-04T07:37:46.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd295000640 1 -- 172.21.15.113:0/2578708224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2901f1240 con 0x7fd2901e4df0 2024-03-04T07:37:46.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.322+0000 7fd295000640 1 -- 172.21.15.113:0/2578708224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd290075c00 con 0x7fd2901e4df0 2024-03-04T07:37:46.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.326+0000 7fd286000640 1 -- 172.21.15.113:0/2578708224 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2880af030 con 0x7fd2901e4df0 2024-03-04T07:37:46.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.326+0000 7fd286000640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd270066f60 0x7fd270069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:46.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.326+0000 7fd286000640 1 -- 172.21.15.113:0/2578708224 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fd28811e190 con 0x7fd2901e4df0 2024-03-04T07:37:46.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.326+0000 7fd28de00640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd270066f60 0x7fd270069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:46.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.326+0000 7fd28de00640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd270066f60 0x7fd270069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd280009b60 tx=0x7fd280009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:46.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.334+0000 7fd286000640 1 -- 172.21.15.113:0/2578708224 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd288016490 con 0x7fd2901e4df0 2024-03-04T07:37:46.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.522+0000 7fd295000640 1 -- 172.21.15.113:0/2578708224 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2900c79d0 con 0x7fd2901e4df0 2024-03-04T07:37:46.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.522+0000 7fd286000640 1 -- 172.21.15.113:0/2578708224 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5817 (secure 0 0 0) 0x7fd2880eeae0 con 0x7fd2901e4df0 2024-03-04T07:37:46.526 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:46.526 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:37:45.608068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:resolve","state_seq":19,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:37:46.526 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-03-04T07:37:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 -- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd270066f60 msgr2=0x7fd270069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd270066f60 0x7fd270069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd280009b60 tx=0x7fd280009190 comp rx=0 tx=0).stop 2024-03-04T07:37:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 -- 172.21.15.113:0/2578708224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2901e4df0 msgr2=0x7fd29014ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2901e4df0 0x7fd29014ef40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd28801cc60 tx=0x7fd274005840 comp rx=0 tx=0).stop 2024-03-04T07:37:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 -- 172.21.15.113:0/2578708224 shutdown_connections 2024-03-04T07:37:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd270066f60 0x7fd270069420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2901e4df0 0x7fd29014ef40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:46.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2900c2cf0 0x7fd2900c1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:46.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 --2- 172.21.15.113:0/2578708224 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2740051d0 0x7fd290153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:46.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 -- 172.21.15.113:0/2578708224 >> 172.21.15.113:0/2578708224 conn(0x7fd29006e820 msgr2=0x7fd2900af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:46.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 -- 172.21.15.113:0/2578708224 shutdown_connections 2024-03-04T07:37:46.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:46.526+0000 7fd295000640 1 -- 172.21.15.113:0/2578708224 wait complete. 2024-03-04T07:37:46.547 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:37:48.548 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:48.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.738+0000 7fbbf1800640 1 Processor -- start 2024-03-04T07:37:48.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.738+0000 7fbbf1800640 1 -- start start 2024-03-04T07:37:48.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.738+0000 7fbbf1800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbec0c8200 0x7fbbec070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:48.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.738+0000 7fbbf1800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbbec152bd0 0x7fbbec1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:48.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.738+0000 7fbbf1800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbec1591f0 con 0x7fbbec1527f0 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.738+0000 7fbbf1800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbec071df0 con 0x7fbbec0c8200 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.738+0000 7fbbf1800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbec1597b0 con 0x7fbbec152bd0 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbea000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbec0c8200 0x7fbbec070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbea000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbec0c8200 0x7fbbec070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39602/0 (socket says 172.21.15.113:39602) 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbeaa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbbec1527f0 0x7fbbec072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46698/0 (socket says 172.21.15.113:46698) 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbea000640 1 -- 172.21.15.113:0/2472821750 learned_addr learned my addr 172.21.15.113:0/2472821750 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbeb400640 1 --2- 172.21.15.113:0/2472821750 >> v2:172.21.15.136:3300/0 conn(0x7fbbec152bd0 0x7fbbec1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbea000640 1 -- 172.21.15.113:0/2472821750 >> v2:172.21.15.136:3300/0 conn(0x7fbbec152bd0 msgr2=0x7fbbec1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbea000640 1 --2- 172.21.15.113:0/2472821750 >> v2:172.21.15.136:3300/0 conn(0x7fbbec152bd0 0x7fbbec1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbea000640 1 -- 172.21.15.113:0/2472821750 >> v1:172.21.15.113:6789/0 conn(0x7fbbec1527f0 legacy=0x7fbbec072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbea000640 1 -- 172.21.15.113:0/2472821750 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec1594d0 con 0x7fbbec0c8200 2024-03-04T07:37:48.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbe9600640 1 -- 172.21.15.113:0/2472821750 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78985907 0 0) 0x7fbbec1591f0 con 0x7fbbec1527f0 2024-03-04T07:37:48.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbea000640 1 --2- 172.21.15.113:0/2472821750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c8200 0x7fbbec070d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0712b0 tx=0x7fbbd801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349ca237dc4a4574 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:48.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbe9600640 1 -- 172.21.15.113:0/2472821750 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80ab430 con 0x7fbbec0c8200 2024-03-04T07:37:48.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbe9600640 1 -- 172.21.15.113:0/2472821750 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd80b0070 con 0x7fbbec0c8200 2024-03-04T07:37:48.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.742+0000 7fbbe9600640 1 -- 172.21.15.113:0/2472821750 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80ab7f0 con 0x7fbbec0c8200 2024-03-04T07:37:48.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 -- 172.21.15.113:0/2472821750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c8200 msgr2=0x7fbbec070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:48.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 --2- 172.21.15.113:0/2472821750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c8200 0x7fbbec070d40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0712b0 tx=0x7fbbd801c520 comp rx=0 tx=0).stop 2024-03-04T07:37:48.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 -- 172.21.15.113:0/2472821750 shutdown_connections 2024-03-04T07:37:48.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 --2- 172.21.15.113:0/2472821750 >> v2:172.21.15.136:3300/0 conn(0x7fbbec152bd0 0x7fbbec1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:48.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 --2- 172.21.15.113:0/2472821750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c8200 0x7fbbec070d40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:48.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 -- 172.21.15.113:0/2472821750 >> 172.21.15.113:0/2472821750 conn(0x7fbbec06e820 msgr2=0x7fbbec06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:48.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 -- 172.21.15.113:0/2472821750 shutdown_connections 2024-03-04T07:37:48.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 -- 172.21.15.113:0/2472821750 wait complete. 2024-03-04T07:37:48.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 Processor -- start 2024-03-04T07:37:48.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 -- start start 2024-03-04T07:37:48.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c8200 0x7fbbec157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec1527f0 0x7fbbec1f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbec152bd0 0x7fbbec1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec1654f0 con 0x7fbbec0c8200 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec165370 con 0x7fbbec1527f0 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbf1800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec165670 con 0x7fbbec152bd0 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbeaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c8200 0x7fbbec157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbea000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec1527f0 0x7fbbec1f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbeb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbec152bd0 0x7fbbec1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbea000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec1527f0 0x7fbbec1f7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:45378/0 (socket says 172.21.15.113:45378) 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbea000640 1 -- 172.21.15.113:0/4021850209 learned_addr learned my addr 172.21.15.113:0/4021850209 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbea000640 1 -- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbec152bd0 msgr2=0x7fbbec1540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbea000640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbec152bd0 0x7fbbec1540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbea000640 1 -- 172.21.15.113:0/4021850209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c8200 msgr2=0x7fbbec157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbea000640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c8200 0x7fbbec157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.746+0000 7fbbea000640 1 -- 172.21.15.113:0/4021850209 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec1f3290 con 0x7fbbec1527f0 2024-03-04T07:37:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbea000640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec1527f0 0x7fbbec1f7290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80ad040 tx=0x7fbbd801ad60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbe2a00640 1 -- 172.21.15.113:0/4021850209 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80be5f0 con 0x7fbbec1527f0 2024-03-04T07:37:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbe2a00640 1 -- 172.21.15.113:0/4021850209 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd80be790 con 0x7fbbec1527f0 2024-03-04T07:37:48.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbe2a00640 1 -- 172.21.15.113:0/4021850209 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd80beb60 con 0x7fbbec1527f0 2024-03-04T07:37:48.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbf1800640 1 -- 172.21.15.113:0/4021850209 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbec1545f0 con 0x7fbbec1527f0 2024-03-04T07:37:48.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbf1800640 1 -- 172.21.15.113:0/4021850209 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbec158390 con 0x7fbbec1527f0 2024-03-04T07:37:48.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbf1800640 1 -- 172.21.15.113:0/4021850209 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec075c00 con 0x7fbbec1527f0 2024-03-04T07:37:48.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbe2a00640 1 -- 172.21.15.113:0/4021850209 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbd80dd6b0 con 0x7fbbec1527f0 2024-03-04T07:37:48.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbe2a00640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbcc066f60 0x7fbbcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:48.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbe2a00640 1 -- 172.21.15.113:0/4021850209 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fbbd80a8030 con 0x7fbbec1527f0 2024-03-04T07:37:48.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.750+0000 7fbbeaa00640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbcc066f60 0x7fbbcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:48.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.754+0000 7fbbeaa00640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbcc066f60 0x7fbbcc069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc007fd0 tx=0x7fbbdc007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:48.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.758+0000 7fbbe2a00640 1 -- 172.21.15.113:0/4021850209 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbbec1f3290 con 0x7fbbec1527f0 2024-03-04T07:37:48.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 -- 172.21.15.113:0/4021850209 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbec0c19d0 con 0x7fbbec1527f0 2024-03-04T07:37:48.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbe2a00640 1 -- 172.21.15.113:0/4021850209 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5817 (secure 0 0 0) 0x7fbbd8016110 con 0x7fbbec1527f0 2024-03-04T07:37:48.956 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:48.956 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:37:45.608068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:resolve","state_seq":19,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:37:48.957 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 21 2024-03-04T07:37:48.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 -- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbcc066f60 msgr2=0x7fbbcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:48.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbcc066f60 0x7fbbcc069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc007fd0 tx=0x7fbbdc007d60 comp rx=0 tx=0).stop 2024-03-04T07:37:48.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 -- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec1527f0 msgr2=0x7fbbec1f7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:48.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec1527f0 0x7fbbec1f7290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbbd80ad040 tx=0x7fbbd801ad60 comp rx=0 tx=0).stop 2024-03-04T07:37:48.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 -- 172.21.15.113:0/4021850209 shutdown_connections 2024-03-04T07:37:48.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbec152bd0 0x7fbbec1540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:48.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec1527f0 0x7fbbec1f7290 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:48.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbcc066f60 0x7fbbcc069420 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:48.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 --2- 172.21.15.113:0/4021850209 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c8200 0x7fbbec157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:48.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 -- 172.21.15.113:0/4021850209 >> 172.21.15.113:0/4021850209 conn(0x7fbbec06e820 msgr2=0x7fbbec0c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:48.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 -- 172.21.15.113:0/4021850209 shutdown_connections 2024-03-04T07:37:48.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:48.954+0000 7fbbf1800640 1 -- 172.21.15.113:0/4021850209 wait complete. 2024-03-04T07:37:48.978 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:37:50.950 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/30 file after the snapshot 2024-03-04T07:37:50.952 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/31 file after the snapshot 2024-03-04T07:37:50.953 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/32 file after the snapshot 2024-03-04T07:37:50.954 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/33 file after the snapshot 2024-03-04T07:37:50.955 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/34 file after the snapshot 2024-03-04T07:37:50.956 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/35 file after the snapshot 2024-03-04T07:37:50.979 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:51.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.114+0000 7f72e0400640 1 Processor -- start 2024-03-04T07:37:51.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.114+0000 7f72e0400640 1 -- start start 2024-03-04T07:37:51.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.114+0000 7f72e0400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72d8152a50 0x7f72d80ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:51.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.114+0000 7f72e0400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d8152e30 con 0x7f72d815c4e0 2024-03-04T07:37:51.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.114+0000 7f72e0400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d806f280 con 0x7f72d806ff20 2024-03-04T07:37:51.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.114+0000 7f72e0400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d80c31b0 con 0x7f72d8152a50 2024-03-04T07:37:51.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72dda00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72d8152a50 0x7f72d80ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:51.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72dda00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f72d8152a50 0x7f72d80ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39614/0 (socket says 172.21.15.113:39614) 2024-03-04T07:37:51.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72dda00640 1 -- 172.21.15.113:0/3747919197 learned_addr learned my addr 172.21.15.113:0/3747919197 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:51.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72de400640 1 --1- 172.21.15.113:0/3747919197 >> v1:172.21.15.113:6789/0 conn(0x7f72d815c4e0 0x7f72d8163060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46700/0 (socket says 172.21.15.113:46700) 2024-03-04T07:37:51.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72dda00640 1 -- 172.21.15.113:0/3747919197 >> v1:172.21.15.136:6789/0 conn(0x7f72d806ff20 legacy=0x7f72d815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:51.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72dda00640 1 -- 172.21.15.113:0/3747919197 >> v1:172.21.15.113:6789/0 conn(0x7f72d815c4e0 legacy=0x7f72d8163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:51.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72dda00640 1 -- 172.21.15.113:0/3747919197 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d806f560 con 0x7f72d8152a50 2024-03-04T07:37:51.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72dda00640 1 --2- 172.21.15.113:0/3747919197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 0x7f72d80ce120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009870 tx=0x7f72cc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ed0a99f0f5663 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72d7e00640 1 -- 172.21.15.113:0/3747919197 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0a7070 con 0x7f72d8152a50 2024-03-04T07:37:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72d7e00640 1 -- 172.21.15.113:0/3747919197 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc0b0070 con 0x7f72d8152a50 2024-03-04T07:37:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.118+0000 7f72d7e00640 1 -- 172.21.15.113:0/3747919197 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0ab890 con 0x7f72d8152a50 2024-03-04T07:37:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 -- 172.21.15.113:0/3747919197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 msgr2=0x7f72d80ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 --2- 172.21.15.113:0/3747919197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 0x7f72d80ce120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009870 tx=0x7f72cc01c560 comp rx=0 tx=0).stop 2024-03-04T07:37:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 -- 172.21.15.113:0/3747919197 shutdown_connections 2024-03-04T07:37:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 --2- 172.21.15.113:0/3747919197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 0x7f72d80ce120 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 -- 172.21.15.113:0/3747919197 >> 172.21.15.113:0/3747919197 conn(0x7f72d80734d0 msgr2=0x7f72d8066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 -- 172.21.15.113:0/3747919197 shutdown_connections 2024-03-04T07:37:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 -- 172.21.15.113:0/3747919197 wait complete. 2024-03-04T07:37:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 Processor -- start 2024-03-04T07:37:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 -- start start 2024-03-04T07:37:51.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72d806ff20 0x7f72d814dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:51.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 0x7f72d81f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:51.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72d815c4e0 0x7f72d81f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:51.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80ce9f0 con 0x7f72d8152a50 2024-03-04T07:37:51.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80ce870 con 0x7f72d806ff20 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72e0400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d8166fa0 con 0x7f72d815c4e0 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72dd000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 0x7f72d81f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72dd000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 0x7f72d81f1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39628/0 (socket says 172.21.15.113:39628) 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72dd000640 1 -- 172.21.15.113:0/2310811333 learned_addr learned my addr 172.21.15.113:0/2310811333 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72dda00640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72d806ff20 0x7f72d814dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72de400640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72d815c4e0 0x7f72d81f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72dd000640 1 -- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72d815c4e0 msgr2=0x7f72d81f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72dd000640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72d815c4e0 0x7f72d81f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72dd000640 1 -- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72d806ff20 msgr2=0x7f72d814dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72dd000640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72d806ff20 0x7f72d814dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72dd000640 1 -- 172.21.15.113:0/2310811333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d815b810 con 0x7f72d8152a50 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.122+0000 7f72de400640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72d815c4e0 0x7f72d81f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:37:51.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.126+0000 7f72dd000640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 0x7f72d81f1660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c3f0 tx=0x7f72c400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:51.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.126+0000 7f72d5600640 1 -- 172.21.15.113:0/2310811333 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4017070 con 0x7f72d8152a50 2024-03-04T07:37:51.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.126+0000 7f72d5600640 1 -- 172.21.15.113:0/2310811333 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c40052d0 con 0x7f72d8152a50 2024-03-04T07:37:51.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.126+0000 7f72e0400640 1 -- 172.21.15.113:0/2310811333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d80c5730 con 0x7f72d8152a50 2024-03-04T07:37:51.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.126+0000 7f72d5600640 1 -- 172.21.15.113:0/2310811333 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4003840 con 0x7f72d8152a50 2024-03-04T07:37:51.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.126+0000 7f72e0400640 1 -- 172.21.15.113:0/2310811333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d80c5cf0 con 0x7f72d8152a50 2024-03-04T07:37:51.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.126+0000 7f72e0400640 1 -- 172.21.15.113:0/2310811333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d8075be0 con 0x7f72d8152a50 2024-03-04T07:37:51.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.126+0000 7f72d5600640 1 -- 172.21.15.113:0/2310811333 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72c401a070 con 0x7f72d8152a50 2024-03-04T07:37:51.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.126+0000 7f72d5600640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f72b8066f90 0x7f72b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:51.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.126+0000 7f72d5600640 1 -- 172.21.15.113:0/2310811333 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f72c4084540 con 0x7f72d8152a50 2024-03-04T07:37:51.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.130+0000 7f72dda00640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f72b8066f90 0x7f72b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:51.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.130+0000 7f72dda00640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f72b8066f90 0x7f72b8069450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0af000 tx=0x7f72cc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:51.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.130+0000 7f72d5600640 1 -- 172.21.15.113:0/2310811333 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f72c4005480 con 0x7f72d8152a50 2024-03-04T07:37:51.208 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/36 file after the snapshot 2024-03-04T07:37:51.210 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/37 file after the snapshot 2024-03-04T07:37:51.212 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/38 file after the snapshot 2024-03-04T07:37:51.213 INFO:tasks.workunit.client.0.smithi113.stdout: create 263/39 file after the snapshot 2024-03-04T07:37:51.215 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/.snap/snap-subdir-test 2024-03-04T07:37:51.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.322+0000 7f72e0400640 1 -- 172.21.15.113:0/2310811333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72d80cad90 con 0x7f72d8152a50 2024-03-04T07:37:51.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.322+0000 7f72d5600640 1 -- 172.21.15.113:0/2310811333 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f72c4054cb0 con 0x7f72d8152a50 2024-03-04T07:37:51.326 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:51.326 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:37:50.728646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:37:51.326 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-03-04T07:37:51.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 -- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f72b8066f90 msgr2=0x7f72b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:51.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f72b8066f90 0x7f72b8069450 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0af000 tx=0x7f72cc0a7000 comp rx=0 tx=0).stop 2024-03-04T07:37:51.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 -- 172.21.15.113:0/2310811333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 msgr2=0x7f72d81f1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:51.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 0x7f72d81f1660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f72c400c3f0 tx=0x7f72c400c8b0 comp rx=0 tx=0).stop 2024-03-04T07:37:51.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 -- 172.21.15.113:0/2310811333 shutdown_connections 2024-03-04T07:37:51.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f72d815c4e0 0x7f72d81f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:51.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f72b8066f90 0x7f72b8069450 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:51.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72d8152a50 0x7f72d81f1660 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:51.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 --2- 172.21.15.113:0/2310811333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72d806ff20 0x7f72d814dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:51.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 -- 172.21.15.113:0/2310811333 >> 172.21.15.113:0/2310811333 conn(0x7f72d80734d0 msgr2=0x7f72d80ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:51.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 -- 172.21.15.113:0/2310811333 shutdown_connections 2024-03-04T07:37:51.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:51.326+0000 7f72e0400640 1 -- 172.21.15.113:0/2310811333 wait complete. 2024-03-04T07:37:51.348 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:37:53.349 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:53.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f054a800640 1 Processor -- start 2024-03-04T07:37:53.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f054a800640 1 -- start start 2024-03-04T07:37:53.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f054a800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05440ce570 0x7f0544154c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:53.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f054a800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f05440c8d80 0x7f05441551d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:53.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f054a800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544070120 0x7f054415b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:53.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f054a800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054406c860 con 0x7f05440ce570 2024-03-04T07:37:53.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f054a800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054406f8d0 con 0x7f05440c8d80 2024-03-04T07:37:53.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f054a800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054406f310 con 0x7f0544070120 2024-03-04T07:37:53.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f0543400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05440ce570 0x7f0544154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:53.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f0543400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05440ce570 0x7f0544154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39642/0 (socket says 172.21.15.113:39642) 2024-03-04T07:37:53.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f0543400640 1 -- 172.21.15.113:0/3597534422 learned_addr learned my addr 172.21.15.113:0/3597534422 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:53.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.482+0000 7f0542a00640 1 --2- 172.21.15.113:0/3597534422 >> v2:172.21.15.136:3300/0 conn(0x7f05440c8d80 0x7f05441551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:53.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.486+0000 7f0543e00640 1 --2- 172.21.15.113:0/3597534422 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544070120 0x7f054415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:53.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.486+0000 7f0543400640 1 -- 172.21.15.113:0/3597534422 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544070120 msgr2=0x7f054415b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:53.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.486+0000 7f0543400640 1 --2- 172.21.15.113:0/3597534422 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544070120 0x7f054415b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.486+0000 7f0543400640 1 -- 172.21.15.113:0/3597534422 >> v2:172.21.15.136:3300/0 conn(0x7f05440c8d80 msgr2=0x7f05441551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:53.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.486+0000 7f0543400640 1 --2- 172.21.15.113:0/3597534422 >> v2:172.21.15.136:3300/0 conn(0x7f05440c8d80 0x7f05441551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.486+0000 7f0543400640 1 -- 172.21.15.113:0/3597534422 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054406f5f0 con 0x7f05440ce570 2024-03-04T07:37:53.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f0543400640 1 --2- 172.21.15.113:0/3597534422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 0x7f0544154c90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f05440706d0 tx=0x7f053401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d451ab512a661b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:53.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f0542000640 1 -- 172.21.15.113:0/3597534422 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340b5070 con 0x7f05440ce570 2024-03-04T07:37:53.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f0542000640 1 -- 172.21.15.113:0/3597534422 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05340a89d0 con 0x7f05440ce570 2024-03-04T07:37:53.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f0542000640 1 -- 172.21.15.113:0/3597534422 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340a8da0 con 0x7f05440ce570 2024-03-04T07:37:53.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f054a800640 1 -- 172.21.15.113:0/3597534422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 msgr2=0x7f0544154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:53.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f054a800640 1 --2- 172.21.15.113:0/3597534422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 0x7f0544154c90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f05440706d0 tx=0x7f053401ed10 comp rx=0 tx=0).stop 2024-03-04T07:37:53.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f054a800640 1 -- 172.21.15.113:0/3597534422 shutdown_connections 2024-03-04T07:37:53.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f054a800640 1 --2- 172.21.15.113:0/3597534422 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0544070120 0x7f054415b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f054a800640 1 --2- 172.21.15.113:0/3597534422 >> v2:172.21.15.136:3300/0 conn(0x7f05440c8d80 0x7f05441551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f054a800640 1 --2- 172.21.15.113:0/3597534422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 0x7f0544154c90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f054a800640 1 -- 172.21.15.113:0/3597534422 >> 172.21.15.113:0/3597534422 conn(0x7f0544066db0 msgr2=0x7f05440671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:53.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f054a800640 1 -- 172.21.15.113:0/3597534422 shutdown_connections 2024-03-04T07:37:53.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.490+0000 7f054a800640 1 -- 172.21.15.113:0/3597534422 wait complete. 2024-03-04T07:37:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f054a800640 1 Processor -- start 2024-03-04T07:37:53.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f054a800640 1 -- start start 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f054a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0544070120 0x7f054414ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f054a800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05440c8d80 0x7f0544157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f054a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 0x7f05441f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f054a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054406fb80 con 0x7f05440ce570 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f054a800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054406c4f0 con 0x7f0544070120 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f054a800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054406fd00 con 0x7f05440c8d80 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0543e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 0x7f05441f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0543e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 0x7f05441f0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39648/0 (socket says 172.21.15.113:39648) 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0543e00640 1 -- 172.21.15.113:0/1618763246 learned_addr learned my addr 172.21.15.113:0/1618763246 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0543400640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0544070120 0x7f054414ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0542a00640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05440c8d80 0x7f0544157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:53.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0543e00640 1 -- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05440c8d80 msgr2=0x7f0544157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0543e00640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05440c8d80 0x7f0544157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0543e00640 1 -- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0544070120 msgr2=0x7f054414ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0543e00640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0544070120 0x7f054414ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0543e00640 1 -- 172.21.15.113:0/1618763246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05441501c0 con 0x7f05440ce570 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0542a00640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05440c8d80 0x7f0544157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f0543e00640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 0x7f05441f0ee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f053c007d70 tx=0x7f053c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f053b400640 1 -- 172.21.15.113:0/1618763246 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c014420 con 0x7f05440ce570 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f053b400640 1 -- 172.21.15.113:0/1618763246 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c019070 con 0x7f05440ce570 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f054a800640 1 -- 172.21.15.113:0/1618763246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05441504a0 con 0x7f05440ce570 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f053b400640 1 -- 172.21.15.113:0/1618763246 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c014800 con 0x7f05440ce570 2024-03-04T07:37:53.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.494+0000 7f054a800640 1 -- 172.21.15.113:0/1618763246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054414f4a0 con 0x7f05440ce570 2024-03-04T07:37:53.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.498+0000 7f054a800640 1 -- 172.21.15.113:0/1618763246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0544075c00 con 0x7f05440ce570 2024-03-04T07:37:53.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.498+0000 7f053b400640 1 -- 172.21.15.113:0/1618763246 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f053c0149a0 con 0x7f05440ce570 2024-03-04T07:37:53.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.498+0000 7f053b400640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:53.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.498+0000 7f053b400640 1 -- 172.21.15.113:0/1618763246 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f053c0857b0 con 0x7f05440ce570 2024-03-04T07:37:53.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.498+0000 7f0543400640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:53.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.498+0000 7f0543400640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f05340b2000 tx=0x7f0534009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:53.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.502+0000 7f053b400640 1 -- 172.21.15.113:0/1618763246 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f053c056730 con 0x7f05440ce570 2024-03-04T07:37:53.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.690+0000 7f054a800640 1 -- 172.21.15.113:0/1618763246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0544153e70 con 0x7f05440ce570 2024-03-04T07:37:53.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.690+0000 7f053b400640 1 -- 172.21.15.113:0/1618763246 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f053c056030 con 0x7f05440ce570 2024-03-04T07:37:53.694 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:53.695 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:37:50.728646+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276]},"id":1}]} 2024-03-04T07:37:53.695 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 22 2024-03-04T07:37:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 -- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 msgr2=0x7f0524069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f05340b2000 tx=0x7f0534009040 comp rx=0 tx=0).stop 2024-03-04T07:37:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 -- 172.21.15.113:0/1618763246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 msgr2=0x7f05441f0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:53.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 0x7f05441f0ee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f053c007d70 tx=0x7f053c00e4b0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 -- 172.21.15.113:0/1618763246 shutdown_connections 2024-03-04T07:37:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0524066f60 0x7f0524069420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05440ce570 0x7f05441f0ee0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05440c8d80 0x7f0544157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 --2- 172.21.15.113:0/1618763246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0544070120 0x7f054414ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 -- 172.21.15.113:0/1618763246 >> 172.21.15.113:0/1618763246 conn(0x7f0544066db0 msgr2=0x7f0544153250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 -- 172.21.15.113:0/1618763246 shutdown_connections 2024-03-04T07:37:53.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:53.694+0000 7f054a800640 1 -- 172.21.15.113:0/1618763246 wait complete. 2024-03-04T07:37:53.716 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 22, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 22, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4256": { "addr": "172.21.15.113:6837/1310042629", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1310042629, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1310042629, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4256, "incarnation": 10, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4276": { "addr": "172.21.15.136:6835/3348592059", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3348592059, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3348592059, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4276, "incarnation": 4, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4938": { "addr": "172.21.15.113:6836/2997246507", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2997246507, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2997246507, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4938, "incarnation": 20, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:reconnect", "state_seq": 21 } }, "last_failure": 0, "last_failure_osd_epoch": 29, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:37:50.728646+0000", "qdb_cluster": [ 4256, 4276 ], "qdb_leader": 4276, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4276, "mds_1": 4256, "mds_2": 4938 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:37:55.403 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:37:55.406 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:37:55.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:55.826 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/30 file after the snapshot 2024-03-04T07:37:55.828 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/31 file after the snapshot 2024-03-04T07:37:55.829 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/32 file after the snapshot 2024-03-04T07:37:55.831 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/33 file after the snapshot 2024-03-04T07:37:55.832 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/34 file after the snapshot 2024-03-04T07:37:55.834 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/35 file after the snapshot 2024-03-04T07:37:55.835 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/36 file after the snapshot 2024-03-04T07:37:55.837 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/37 file after the snapshot 2024-03-04T07:37:55.839 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/38 file after the snapshot 2024-03-04T07:37:55.840 INFO:tasks.workunit.client.0.smithi113.stdout: create 264/39 file after the snapshot 2024-03-04T07:37:55.842 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/.snap/snap-subdir-test 2024-03-04T07:37:55.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.858+0000 7f3a9d000640 1 Processor -- start 2024-03-04T07:37:55.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.858+0000 7f3a9d000640 1 -- start start 2024-03-04T07:37:55.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a9d000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3a980c2180 0x7f3a9815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:55.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a9d000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a980c9160 con 0x7f3a980b2de0 2024-03-04T07:37:55.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a9d000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a980c9400 con 0x7f3a981547a0 2024-03-04T07:37:55.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a9d000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a980c96e0 con 0x7f3a980c2180 2024-03-04T07:37:55.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a96a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3a980c2180 0x7f3a9815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:55.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a96a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3a980c2180 0x7f3a9815ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39654/0 (socket says 172.21.15.113:39654) 2024-03-04T07:37:55.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a96a00640 1 -- 172.21.15.113:0/3229809684 learned_addr learned my addr 172.21.15.113:0/3229809684 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:55.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a97e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3a980b2de0 0x7f3a98165160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46710/0 (socket says 172.21.15.113:46710) 2024-03-04T07:37:55.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a96a00640 1 -- 172.21.15.113:0/3229809684 >> v1:172.21.15.136:6789/0 conn(0x7f3a981547a0 legacy=0x7f3a980c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:55.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a96a00640 1 -- 172.21.15.113:0/3229809684 >> v1:172.21.15.113:6789/0 conn(0x7f3a980b2de0 legacy=0x7f3a98165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:55.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a96a00640 1 -- 172.21.15.113:0/3229809684 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a980c7c00 con 0x7f3a980c2180 2024-03-04T07:37:55.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.862+0000 7f3a96000640 1 -- 172.21.15.113:0/3229809684 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045225594 0 0) 0x7f3a980c9160 con 0x7f3a980b2de0 2024-03-04T07:37:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a96a00640 1 --2- 172.21.15.113:0/3229809684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c2180 0x7f3a9815ab30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009870 tx=0x7f3a8401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0f33bfb7e1ff0d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a96000640 1 -- 172.21.15.113:0/3229809684 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a840b5070 con 0x7f3a980c2180 2024-03-04T07:37:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a96000640 1 -- 172.21.15.113:0/3229809684 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a840a8a50 con 0x7f3a980c2180 2024-03-04T07:37:55.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a96000640 1 -- 172.21.15.113:0/3229809684 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a840a8e20 con 0x7f3a980c2180 2024-03-04T07:37:55.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a9d000640 1 -- 172.21.15.113:0/3229809684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c2180 msgr2=0x7f3a9815ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:55.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a9d000640 1 --2- 172.21.15.113:0/3229809684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c2180 0x7f3a9815ab30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009870 tx=0x7f3a8401ed50 comp rx=0 tx=0).stop 2024-03-04T07:37:55.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a9d000640 1 -- 172.21.15.113:0/3229809684 shutdown_connections 2024-03-04T07:37:55.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a9d000640 1 --2- 172.21.15.113:0/3229809684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a980c2180 0x7f3a9815ab30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:55.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a9d000640 1 -- 172.21.15.113:0/3229809684 >> 172.21.15.113:0/3229809684 conn(0x7f3a980cd080 msgr2=0x7f3a980cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:55.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a9d000640 1 -- 172.21.15.113:0/3229809684 shutdown_connections 2024-03-04T07:37:55.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.866+0000 7f3a9d000640 1 -- 172.21.15.113:0/3229809684 wait complete. 2024-03-04T07:37:55.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 Processor -- start 2024-03-04T07:37:55.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 -- start start 2024-03-04T07:37:55.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3a980b2de0 0x7f3a981f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:55.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a980c2180 0x7f3a9814c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a981547a0 0x7f3a980cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98154b80 con 0x7f3a981547a0 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a980c81c0 con 0x7f3a980c2180 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98154d00 con 0x7f3a980b2de0 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a97e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a981547a0 0x7f3a980cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a97e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a981547a0 0x7f3a980cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39666/0 (socket says 172.21.15.113:39666) 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a97e00640 1 -- 172.21.15.113:0/3694193046 learned_addr learned my addr 172.21.15.113:0/3694193046 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a97400640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3a980b2de0 0x7f3a981f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a96a00640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a980c2180 0x7f3a9814c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a97e00640 1 -- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3a980b2de0 msgr2=0x7f3a981f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a97e00640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3a980b2de0 0x7f3a981f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a97e00640 1 -- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a980c2180 msgr2=0x7f3a9814c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:55.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a97e00640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a980c2180 0x7f3a9814c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a97e00640 1 -- 172.21.15.113:0/3694193046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a98150e80 con 0x7f3a981547a0 2024-03-04T07:37:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a97e00640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a981547a0 0x7f3a980cbcc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000c7a0 tx=0x7f3a9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a8f400640 1 -- 172.21.15.113:0/3694193046 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a90018070 con 0x7f3a981547a0 2024-03-04T07:37:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a8f400640 1 -- 172.21.15.113:0/3694193046 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9000a640 con 0x7f3a981547a0 2024-03-04T07:37:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a8f400640 1 -- 172.21.15.113:0/3694193046 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000aab0 con 0x7f3a981547a0 2024-03-04T07:37:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 -- 172.21.15.113:0/3694193046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a98151e60 con 0x7f3a981547a0 2024-03-04T07:37:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 -- 172.21.15.113:0/3694193046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a98152420 con 0x7f3a981547a0 2024-03-04T07:37:55.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.870+0000 7f3a9d000640 1 -- 172.21.15.113:0/3694193046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a98075a80 con 0x7f3a981547a0 2024-03-04T07:37:55.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.874+0000 7f3a8f400640 1 -- 172.21.15.113:0/3694193046 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a9001b070 con 0x7f3a981547a0 2024-03-04T07:37:55.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.874+0000 7f3a8f400640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3a78066f60 0x7f3a78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:55.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.874+0000 7f3a8f400640 1 -- 172.21.15.113:0/3694193046 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f3a90084790 con 0x7f3a981547a0 2024-03-04T07:37:55.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.874+0000 7f3a97400640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3a78066f60 0x7f3a78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:55.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.874+0000 7f3a97400640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3a78066f60 0x7f3a78069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007fd0 tx=0x7f3a88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:55.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:55.878+0000 7f3a8f400640 1 -- 172.21.15.113:0/3694193046 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3a90054f90 con 0x7f3a981547a0 2024-03-04T07:37:56.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.066+0000 7f3a9d000640 1 -- 172.21.15.113:0/3694193046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a98153b20 con 0x7f3a981547a0 2024-03-04T07:37:56.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a8f400640 1 -- 172.21.15.113:0/3694193046 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f3a90059030 con 0x7f3a981547a0 2024-03-04T07:37:56.071 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:56.071 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:37:55.751681+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:37:56.072 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 23 2024-03-04T07:37:56.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 -- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3a78066f60 msgr2=0x7f3a78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:56.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3a78066f60 0x7f3a78069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3a88007fd0 tx=0x7f3a88009040 comp rx=0 tx=0).stop 2024-03-04T07:37:56.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 -- 172.21.15.113:0/3694193046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a981547a0 msgr2=0x7f3a980cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:56.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a981547a0 0x7f3a980cbcc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000c7a0 tx=0x7f3a9000cc60 comp rx=0 tx=0).stop 2024-03-04T07:37:56.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 -- 172.21.15.113:0/3694193046 shutdown_connections 2024-03-04T07:37:56.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3a78066f60 0x7f3a78069420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:56.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3a981547a0 0x7f3a980cbcc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:56.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a980c2180 0x7f3a9814c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:56.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 --2- 172.21.15.113:0/3694193046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3a980b2de0 0x7f3a981f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:56.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 -- 172.21.15.113:0/3694193046 >> 172.21.15.113:0/3694193046 conn(0x7f3a980cd080 msgr2=0x7f3a98071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:56.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 -- 172.21.15.113:0/3694193046 shutdown_connections 2024-03-04T07:37:56.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:56.070+0000 7f3a9d000640 1 -- 172.21.15.113:0/3694193046 wait complete. 2024-03-04T07:37:56.092 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:37:58.094 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:37:58.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.226+0000 7fdfefe00640 1 Processor -- start 2024-03-04T07:37:58.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.226+0000 7fdfefe00640 1 -- start start 2024-03-04T07:37:58.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfefe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdfe8154d00 0x7fdfe815b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:58.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfefe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdfe806f5b0 0x7fdfe80ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:58.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfefe00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe806d480 con 0x7fdfe80ce7d0 2024-03-04T07:37:58.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfefe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe815ba50 con 0x7fdfe806f5b0 2024-03-04T07:37:58.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfefe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe806c590 con 0x7fdfe8154d00 2024-03-04T07:37:58.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfeca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdfe806f5b0 0x7fdfe80ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:58.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfed400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdfe8154d00 0x7fdfe815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:58.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfed400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdfe8154d00 0x7fdfe815b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:58576/0 (socket says 172.21.15.113:58576) 2024-03-04T07:37:58.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfed400640 1 -- 172.21.15.113:0/1371055389 learned_addr learned my addr 172.21.15.113:0/1371055389 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:58.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfeca00640 1 -- 172.21.15.113:0/1371055389 >> v2:172.21.15.136:3300/0 conn(0x7fdfe8154d00 msgr2=0x7fdfe815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:37:58.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfeca00640 1 --2- 172.21.15.113:0/1371055389 >> v2:172.21.15.136:3300/0 conn(0x7fdfe8154d00 0x7fdfe815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:58.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfeca00640 1 -- 172.21.15.113:0/1371055389 >> v1:172.21.15.136:6789/0 conn(0x7fdfe80ce7d0 legacy=0x7fdfe80c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:58.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.230+0000 7fdfeca00640 1 -- 172.21.15.113:0/1371055389 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe806c870 con 0x7fdfe806f5b0 2024-03-04T07:37:58.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfeca00640 1 --2- 172.21.15.113:0/1371055389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 0x7fdfe80ce290 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8067d90 tx=0x7fdfd401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2a155fc31c6665 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:58.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfe7e00640 1 -- 172.21.15.113:0/1371055389 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40b0070 con 0x7fdfe806f5b0 2024-03-04T07:37:58.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfe7e00640 1 -- 172.21.15.113:0/1371055389 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd40ab600 con 0x7fdfe806f5b0 2024-03-04T07:37:58.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfe7e00640 1 -- 172.21.15.113:0/1371055389 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40ab9d0 con 0x7fdfe806f5b0 2024-03-04T07:37:58.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfefe00640 1 -- 172.21.15.113:0/1371055389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 msgr2=0x7fdfe80ce290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:58.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfefe00640 1 --2- 172.21.15.113:0/1371055389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 0x7fdfe80ce290 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8067d90 tx=0x7fdfd401c480 comp rx=0 tx=0).stop 2024-03-04T07:37:58.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfefe00640 1 -- 172.21.15.113:0/1371055389 shutdown_connections 2024-03-04T07:37:58.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfefe00640 1 --2- 172.21.15.113:0/1371055389 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 0x7fdfe80ce290 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:58.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfefe00640 1 --2- 172.21.15.113:0/1371055389 >> v2:172.21.15.136:3300/0 conn(0x7fdfe8154d00 0x7fdfe815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:58.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfefe00640 1 -- 172.21.15.113:0/1371055389 >> 172.21.15.113:0/1371055389 conn(0x7fdfe8066db0 msgr2=0x7fdfe80671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:58.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfefe00640 1 -- 172.21.15.113:0/1371055389 shutdown_connections 2024-03-04T07:37:58.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.234+0000 7fdfefe00640 1 -- 172.21.15.113:0/1371055389 wait complete. 2024-03-04T07:37:58.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfefe00640 1 Processor -- start 2024-03-04T07:37:58.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfefe00640 1 -- start start 2024-03-04T07:37:58.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfefe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 0x7fdfe80afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:58.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfefe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdfe80ce7d0 0x7fdfe814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:58.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfefe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe8154d00 0x7fdfe814c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:58.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfefe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe806c170 con 0x7fdfe806f5b0 2024-03-04T07:37:58.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfefe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe81550e0 con 0x7fdfe8154d00 2024-03-04T07:37:58.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfefe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe806c2f0 con 0x7fdfe80ce7d0 2024-03-04T07:37:58.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfed400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 0x7fdfe80afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:58.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfed400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 0x7fdfe80afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35448/0 (socket says 172.21.15.113:35448) 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfed400640 1 -- 172.21.15.113:0/899687605 learned_addr learned my addr 172.21.15.113:0/899687605 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfede00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe8154d00 0x7fdfe814c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfeca00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdfe80ce7d0 0x7fdfe814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfed400640 1 -- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdfe80ce7d0 msgr2=0x7fdfe814c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfed400640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdfe80ce7d0 0x7fdfe814c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfed400640 1 -- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe8154d00 msgr2=0x7fdfe814c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfed400640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe8154d00 0x7fdfe814c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfed400640 1 -- 172.21.15.113:0/899687605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8152470 con 0x7fdfe806f5b0 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfeca00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdfe80ce7d0 0x7fdfe814c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfed400640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 0x7fdfe80afed0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00e8a0 tx=0x7fdfdc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfe5600640 1 -- 172.21.15.113:0/899687605 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc017070 con 0x7fdfe806f5b0 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfe5600640 1 -- 172.21.15.113:0/899687605 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc00c940 con 0x7fdfe806f5b0 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfe5600640 1 -- 172.21.15.113:0/899687605 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc00cd50 con 0x7fdfe806f5b0 2024-03-04T07:37:58.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfefe00640 1 -- 172.21.15.113:0/899687605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe8152750 con 0x7fdfe806f5b0 2024-03-04T07:37:58.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.238+0000 7fdfefe00640 1 -- 172.21.15.113:0/899687605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe814da80 con 0x7fdfe806f5b0 2024-03-04T07:37:58.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.242+0000 7fdfefe00640 1 -- 172.21.15.113:0/899687605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe8075c00 con 0x7fdfe806f5b0 2024-03-04T07:37:58.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.242+0000 7fdfe5600640 1 -- 172.21.15.113:0/899687605 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfdc01a070 con 0x7fdfe806f5b0 2024-03-04T07:37:58.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.242+0000 7fdfe5600640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdfc8066f90 0x7fdfc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:37:58.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.242+0000 7fdfe5600640 1 -- 172.21.15.113:0/899687605 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fdfdc0853d0 con 0x7fdfe806f5b0 2024-03-04T07:37:58.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.242+0000 7fdfeca00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdfc8066f90 0x7fdfc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:37:58.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.242+0000 7fdfeca00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdfc8066f90 0x7fdfc8069450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40ad000 tx=0x7fdfd4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:37:58.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.246+0000 7fdfe5600640 1 -- 172.21.15.113:0/899687605 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdfdc055bd0 con 0x7fdfe806f5b0 2024-03-04T07:37:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 -- 172.21.15.113:0/899687605 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfe8151570 con 0x7fdfe806f5b0 2024-03-04T07:37:58.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfe5600640 1 -- 172.21.15.113:0/899687605 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5816 (secure 0 0 0) 0x7fdfdc059c70 con 0x7fdfe806f5b0 2024-03-04T07:37:58.440 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:37:58.441 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:37:55.751681+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:37:58.441 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 23 2024-03-04T07:37:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 -- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdfc8066f90 msgr2=0x7fdfc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdfc8066f90 0x7fdfc8069450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40ad000 tx=0x7fdfd4008040 comp rx=0 tx=0).stop 2024-03-04T07:37:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 -- 172.21.15.113:0/899687605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 msgr2=0x7fdfe80afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:37:58.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 0x7fdfe80afed0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00e8a0 tx=0x7fdfdc00ed60 comp rx=0 tx=0).stop 2024-03-04T07:37:58.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 -- 172.21.15.113:0/899687605 shutdown_connections 2024-03-04T07:37:58.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe8154d00 0x7fdfe814c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:58.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdfe80ce7d0 0x7fdfe814c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:58.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdfc8066f90 0x7fdfc8069450 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:58.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 --2- 172.21.15.113:0/899687605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe806f5b0 0x7fdfe80afed0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:37:58.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 -- 172.21.15.113:0/899687605 >> 172.21.15.113:0/899687605 conn(0x7fdfe8066db0 msgr2=0x7fdfe80ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:37:58.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 -- 172.21.15.113:0/899687605 shutdown_connections 2024-03-04T07:37:58.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:37:58.438+0000 7fdfefe00640 1 -- 172.21.15.113:0/899687605 wait complete. 2024-03-04T07:37:58.461 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:38:00.462 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.594+0000 7f0e04a00640 1 Processor -- start 2024-03-04T07:38:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.594+0000 7f0e04a00640 1 -- start start 2024-03-04T07:38:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0e04a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e0006ff20 0x7f0e0015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0e04a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e000c8b90 0x7f0e0015c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0e04a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e00152e30 con 0x7f0e00152a50 2024-03-04T07:38:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0e04a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e000c31b0 con 0x7f0e0006ff20 2024-03-04T07:38:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0e04a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0006f840 con 0x7f0e000c8b90 2024-03-04T07:38:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0dfd400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e0006ff20 0x7f0e0015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0dfde00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0e00152a50 0x7f0e000ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52176/0 (socket says 172.21.15.113:52176) 2024-03-04T07:38:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0dfde00640 1 -- 172.21.15.113:0/1143412247 learned_addr learned my addr 172.21.15.113:0/1143412247 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:38:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0dfe800640 1 --2- 172.21.15.113:0/1143412247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e000c8b90 0x7f0e0015c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0dfd400640 1 -- 172.21.15.113:0/1143412247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e000c8b90 msgr2=0x7f0e0015c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0dfd400640 1 --2- 172.21.15.113:0/1143412247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e000c8b90 0x7f0e0015c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0dfd400640 1 -- 172.21.15.113:0/1143412247 >> v1:172.21.15.113:6789/0 conn(0x7f0e00152a50 legacy=0x7f0e000ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0dfd400640 1 -- 172.21.15.113:0/1143412247 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0006f280 con 0x7f0e0006ff20 2024-03-04T07:38:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.598+0000 7f0dfca00640 1 -- 172.21.15.113:0/1143412247 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019313177 0 0) 0x7f0e00152e30 con 0x7f0e00152a50 2024-03-04T07:38:00.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0dfd400640 1 --2- 172.21.15.113:0/1143412247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0006ff20 0x7f0e0015bdd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0097c0 tx=0x7f0dec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c7bd8975bb7679 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:00.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0dfca00640 1 -- 172.21.15.113:0/1143412247 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0aa620 con 0x7f0e0006ff20 2024-03-04T07:38:00.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0dfca00640 1 -- 172.21.15.113:0/1143412247 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec0af070 con 0x7f0e0006ff20 2024-03-04T07:38:00.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0dfca00640 1 -- 172.21.15.113:0/1143412247 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0aa9e0 con 0x7f0e0006ff20 2024-03-04T07:38:00.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0e04a00640 1 -- 172.21.15.113:0/1143412247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0006ff20 msgr2=0x7f0e0015bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0e04a00640 1 --2- 172.21.15.113:0/1143412247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0006ff20 0x7f0e0015bdd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0097c0 tx=0x7f0dec01c4a0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0e04a00640 1 -- 172.21.15.113:0/1143412247 shutdown_connections 2024-03-04T07:38:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0e04a00640 1 --2- 172.21.15.113:0/1143412247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e000c8b90 0x7f0e0015c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0e04a00640 1 --2- 172.21.15.113:0/1143412247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e0006ff20 0x7f0e0015bdd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0e04a00640 1 -- 172.21.15.113:0/1143412247 >> 172.21.15.113:0/1143412247 conn(0x7f0e000734d0 msgr2=0x7f0e00066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0e04a00640 1 -- 172.21.15.113:0/1143412247 shutdown_connections 2024-03-04T07:38:00.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0e04a00640 1 -- 172.21.15.113:0/1143412247 wait complete. 2024-03-04T07:38:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0e04a00640 1 Processor -- start 2024-03-04T07:38:00.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.602+0000 7f0e04a00640 1 -- start start 2024-03-04T07:38:00.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0e04a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e0006ff20 0x7f0e001568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:00.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0e04a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e000c8b90 0x7f0e00146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:00.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0e04a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e00152a50 0x7f0e001f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:00.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0e04a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000ce9b0 con 0x7f0e00152a50 2024-03-04T07:38:00.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0e04a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000ce830 con 0x7f0e0006ff20 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0e04a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e00165880 con 0x7f0e000c8b90 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfe800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e00152a50 0x7f0e001f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfe800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e00152a50 0x7f0e001f2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35456/0 (socket says 172.21.15.113:35456) 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfe800640 1 -- 172.21.15.113:0/4213582149 learned_addr learned my addr 172.21.15.113:0/4213582149 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfd400640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e000c8b90 0x7f0e00146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfde00640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e0006ff20 0x7f0e001568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfe800640 1 -- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e000c8b90 msgr2=0x7f0e00146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfe800640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e000c8b90 0x7f0e00146720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfe800640 1 -- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e0006ff20 msgr2=0x7f0e001568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfe800640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e0006ff20 0x7f0e001568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfe800640 1 -- 172.21.15.113:0/4213582149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0014ff00 con 0x7f0e00152a50 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfde00640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e0006ff20 0x7f0e001568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0dfe800640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e00152a50 0x7f0e001f2f50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009dd0 tx=0x7f0df800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0df6000640 1 -- 172.21.15.113:0/4213582149 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8018020 con 0x7f0e00152a50 2024-03-04T07:38:00.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0df6000640 1 -- 172.21.15.113:0/4213582149 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df8014480 con 0x7f0e00152a50 2024-03-04T07:38:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0e04a00640 1 -- 172.21.15.113:0/4213582149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e001501e0 con 0x7f0e00152a50 2024-03-04T07:38:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0df6000640 1 -- 172.21.15.113:0/4213582149 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8014870 con 0x7f0e00152a50 2024-03-04T07:38:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0e04a00640 1 -- 172.21.15.113:0/4213582149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e001507a0 con 0x7f0e00152a50 2024-03-04T07:38:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0e04a00640 1 -- 172.21.15.113:0/4213582149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e001509e0 con 0x7f0e00152a50 2024-03-04T07:38:00.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.606+0000 7f0df6000640 1 -- 172.21.15.113:0/4213582149 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0df801b020 con 0x7f0e00152a50 2024-03-04T07:38:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.610+0000 7f0df6000640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0de0066f60 0x7f0de0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.610+0000 7f0df6000640 1 -- 172.21.15.113:0/4213582149 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f0df8085c70 con 0x7f0e00152a50 2024-03-04T07:38:00.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.610+0000 7f0dfde00640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0de0066f60 0x7f0de0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:00.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.610+0000 7f0dfde00640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0de0066f60 0x7f0de0069420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0df0000f70 tx=0x7f0df0007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:00.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.618+0000 7f0df6000640 1 -- 172.21.15.113:0/4213582149 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0e001509e0 con 0x7f0e00152a50 2024-03-04T07:38:00.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.810+0000 7f0e04a00640 1 -- 172.21.15.113:0/4213582149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e00075c00 con 0x7f0e00152a50 2024-03-04T07:38:00.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.810+0000 7f0df6000640 1 -- 172.21.15.113:0/4213582149 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f0df8022070 con 0x7f0e00152a50 2024-03-04T07:38:00.814 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:00.814 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:37:55.751681+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4256]},"id":1}]} 2024-03-04T07:38:00.814 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 23 2024-03-04T07:38:00.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 -- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0de0066f60 msgr2=0x7f0de0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0de0066f60 0x7f0de0069420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0df0000f70 tx=0x7f0df0007cb0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 -- 172.21.15.113:0/4213582149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e00152a50 msgr2=0x7f0e001f2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e00152a50 0x7f0e001f2f50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009dd0 tx=0x7f0df800c490 comp rx=0 tx=0).stop 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 -- 172.21.15.113:0/4213582149 shutdown_connections 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0de0066f60 0x7f0de0069420 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e00152a50 0x7f0e001f2f50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e000c8b90 0x7f0e00146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 --2- 172.21.15.113:0/4213582149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e0006ff20 0x7f0e001568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 -- 172.21.15.113:0/4213582149 >> 172.21.15.113:0/4213582149 conn(0x7f0e000734d0 msgr2=0x7f0e00159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 -- 172.21.15.113:0/4213582149 shutdown_connections 2024-03-04T07:38:00.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:00.814+0000 7f0e04a00640 1 -- 172.21.15.113:0/4213582149 wait complete. 2024-03-04T07:38:00.835 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:38:00.884 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/30 file after the snapshot 2024-03-04T07:38:00.887 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/31 file after the snapshot 2024-03-04T07:38:00.889 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/32 file after the snapshot 2024-03-04T07:38:00.890 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/33 file after the snapshot 2024-03-04T07:38:00.891 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/34 file after the snapshot 2024-03-04T07:38:00.892 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/35 file after the snapshot 2024-03-04T07:38:00.894 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/36 file after the snapshot 2024-03-04T07:38:00.895 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/37 file after the snapshot 2024-03-04T07:38:00.897 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/38 file after the snapshot 2024-03-04T07:38:00.898 INFO:tasks.workunit.client.0.smithi113.stdout: create 265/39 file after the snapshot 2024-03-04T07:38:00.899 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/.snap/snap-subdir-test 2024-03-04T07:38:02.836 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.970+0000 7fc34ca00640 1 Processor -- start 2024-03-04T07:38:02.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.970+0000 7fc34ca00640 1 -- start start 2024-03-04T07:38:02.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.970+0000 7fc34ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3480c89d0 0x7fc3480ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.970+0000 7fc34ca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc3480c8db0 0x7fc3481526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.970+0000 7fc34ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3480c6840 0x7fc348152c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.970+0000 7fc34ca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3480c6da0 con 0x7fc3480c89d0 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.970+0000 7fc34ca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3480c7080 con 0x7fc3480c8db0 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.970+0000 7fc34ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480c7c10 con 0x7fc3480c6840 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc345e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3480c89d0 0x7fc3480ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc33fe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc3480c8db0 0x7fc3481526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc345e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3480c89d0 0x7fc3480ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35472/0 (socket says 172.21.15.113:35472) 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc345e00640 1 -- 172.21.15.113:0/1810915637 learned_addr learned my addr 172.21.15.113:0/1810915637 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc346800640 1 --2- 172.21.15.113:0/1810915637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3480c6840 0x7fc348152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc345e00640 1 -- 172.21.15.113:0/1810915637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3480c6840 msgr2=0x7fc348152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc345e00640 1 --2- 172.21.15.113:0/1810915637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3480c6840 0x7fc348152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:02.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc345e00640 1 -- 172.21.15.113:0/1810915637 >> v2:172.21.15.136:3300/0 conn(0x7fc3480c8db0 msgr2=0x7fc3481526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:02.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc345e00640 1 --2- 172.21.15.113:0/1810915637 >> v2:172.21.15.136:3300/0 conn(0x7fc3480c8db0 0x7fc3481526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:02.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc345e00640 1 -- 172.21.15.113:0/1810915637 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3480c7ef0 con 0x7fc3480c89d0 2024-03-04T07:38:02.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.974+0000 7fc33fe00640 1 --2- 172.21.15.113:0/1810915637 >> v2:172.21.15.136:3300/0 conn(0x7fc3480c8db0 0x7fc3481526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:38:02.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc345e00640 1 --2- 172.21.15.113:0/1810915637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 0x7fc3480ca030 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc348071730 tx=0x7fc33801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e943b4cd1323e56 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc345400640 1 -- 172.21.15.113:0/1810915637 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3380b5070 con 0x7fc3480c89d0 2024-03-04T07:38:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc345400640 1 -- 172.21.15.113:0/1810915637 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3380a89d0 con 0x7fc3480c89d0 2024-03-04T07:38:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc345400640 1 -- 172.21.15.113:0/1810915637 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3380a8da0 con 0x7fc3480c89d0 2024-03-04T07:38:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc34ca00640 1 -- 172.21.15.113:0/1810915637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 msgr2=0x7fc3480ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:02.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc34ca00640 1 --2- 172.21.15.113:0/1810915637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 0x7fc3480ca030 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc348071730 tx=0x7fc33801ed10 comp rx=0 tx=0).stop 2024-03-04T07:38:02.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc34ca00640 1 -- 172.21.15.113:0/1810915637 shutdown_connections 2024-03-04T07:38:02.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc34ca00640 1 --2- 172.21.15.113:0/1810915637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3480c6840 0x7fc348152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:02.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc34ca00640 1 --2- 172.21.15.113:0/1810915637 >> v2:172.21.15.136:3300/0 conn(0x7fc3480c8db0 0x7fc3481526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:02.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc34ca00640 1 --2- 172.21.15.113:0/1810915637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 0x7fc3480ca030 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:02.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc34ca00640 1 -- 172.21.15.113:0/1810915637 >> 172.21.15.113:0/1810915637 conn(0x7fc34806deb0 msgr2=0x7fc34806e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:02.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc34ca00640 1 -- 172.21.15.113:0/1810915637 shutdown_connections 2024-03-04T07:38:02.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.978+0000 7fc34ca00640 1 -- 172.21.15.113:0/1810915637 wait complete. 2024-03-04T07:38:02.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 Processor -- start 2024-03-04T07:38:02.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 -- start start 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3480c6840 0x7fc34814bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 0x7fc348152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3480c8db0 0x7fc3481f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34806d410 con 0x7fc3480c89d0 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480ca570 con 0x7fc3480c8db0 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34806d590 con 0x7fc3480c6840 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 0x7fc348152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 0x7fc348152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35484/0 (socket says 172.21.15.113:35484) 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33fe00640 1 -- 172.21.15.113:0/1830081680 learned_addr learned my addr 172.21.15.113:0/1830081680 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc346800640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3480c8db0 0x7fc3481f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:02.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc345e00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3480c6840 0x7fc34814bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33fe00640 1 -- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3480c6840 msgr2=0x7fc34814bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33fe00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3480c6840 0x7fc34814bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33fe00640 1 -- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3480c8db0 msgr2=0x7fc3481f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33fe00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3480c8db0 0x7fc3481f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33fe00640 1 -- 172.21.15.113:0/1830081680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34814c370 con 0x7fc3480c89d0 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33fe00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 0x7fc348152720 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc33000e980 tx=0x7fc33000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33e000640 1 -- 172.21.15.113:0/1830081680 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330017070 con 0x7fc3480c89d0 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33e000640 1 -- 172.21.15.113:0/1830081680 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33000ca00 con 0x7fc3480c89d0 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc33e000640 1 -- 172.21.15.113:0/1830081680 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33000cdd0 con 0x7fc3480c89d0 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 -- 172.21.15.113:0/1830081680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34814c650 con 0x7fc3480c89d0 2024-03-04T07:38:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 -- 172.21.15.113:0/1830081680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3481f4490 con 0x7fc3480c89d0 2024-03-04T07:38:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.982+0000 7fc34ca00640 1 -- 172.21.15.113:0/1830081680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc348075c00 con 0x7fc3480c89d0 2024-03-04T07:38:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.986+0000 7fc33e000640 1 -- 172.21.15.113:0/1830081680 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc33001a070 con 0x7fc3480c89d0 2024-03-04T07:38:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.986+0000 7fc33e000640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc324066f60 0x7fc324069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.986+0000 7fc33e000640 1 -- 172.21.15.113:0/1830081680 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fc330085370 con 0x7fc3480c89d0 2024-03-04T07:38:02.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.986+0000 7fc345e00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc324066f60 0x7fc324069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:02.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.990+0000 7fc345e00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc324066f60 0x7fc324069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc3380b2000 tx=0x7fc338009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:02.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:02.990+0000 7fc33e000640 1 -- 172.21.15.113:0/1830081680 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc330055c10 con 0x7fc3480c89d0 2024-03-04T07:38:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.178+0000 7fc34ca00640 1 -- 172.21.15.113:0/1830081680 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc34806b0f0 con 0x7fc3480c89d0 2024-03-04T07:38:03.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.178+0000 7fc33e000640 1 -- 172.21.15.113:0/1830081680 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5821 (secure 0 0 0) 0x7fc330059cb0 con 0x7fc3480c89d0 2024-03-04T07:38:03.182 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:03.182 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4938]},"id":1}]} 2024-03-04T07:38:03.183 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 24 2024-03-04T07:38:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 -- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc324066f60 msgr2=0x7fc324069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc324066f60 0x7fc324069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc3380b2000 tx=0x7fc338009040 comp rx=0 tx=0).stop 2024-03-04T07:38:03.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 -- 172.21.15.113:0/1830081680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 msgr2=0x7fc348152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 0x7fc348152720 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc33000e980 tx=0x7fc33000ee40 comp rx=0 tx=0).stop 2024-03-04T07:38:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 -- 172.21.15.113:0/1830081680 shutdown_connections 2024-03-04T07:38:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3480c8db0 0x7fc3481f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc324066f60 0x7fc324069420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3480c89d0 0x7fc348152720 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 --2- 172.21.15.113:0/1830081680 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3480c6840 0x7fc34814bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 -- 172.21.15.113:0/1830081680 >> 172.21.15.113:0/1830081680 conn(0x7fc34806deb0 msgr2=0x7fc348157c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 -- 172.21.15.113:0/1830081680 shutdown_connections 2024-03-04T07:38:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.182+0000 7fc34ca00640 1 -- 172.21.15.113:0/1830081680 wait complete. 2024-03-04T07:38:03.213 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=2, replacing gid=4250 2024-03-04T07:38:03.213 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.c 2024-03-04T07:38:03.213 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-03-04T07:38:03.213 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-04T07:38:03.213 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-04T07:38:03.216 INFO:tasks.ceph.mds.c:Started 2024-03-04T07:38:03.216 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:03.301 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-03-04T07:38:03.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.350+0000 7fc0a2400640 1 Processor -- start 2024-03-04T07:38:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a2400640 1 -- start start 2024-03-04T07:38:03.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a2400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc09c0c6b20 0x7fc09c0c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:03.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a2400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc09c0c5e70 0x7fc09c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:03.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a2400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09c06ce20 con 0x7fc09c1528b0 2024-03-04T07:38:03.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a2400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09c1586b0 con 0x7fc09c0c6b20 2024-03-04T07:38:03.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a2400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09c06c620 con 0x7fc09c0c5e70 2024-03-04T07:38:03.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a1a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc09c0c5e70 0x7fc09c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:03.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a1000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc09c1528b0 0x7fc09c15c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52192/0 (socket says 172.21.15.113:52192) 2024-03-04T07:38:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a1000640 1 -- 172.21.15.113:0/3961724999 learned_addr learned my addr 172.21.15.113:0/3961724999 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:38:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc09be00640 1 --2- 172.21.15.113:0/3961724999 >> v2:172.21.15.113:3300/0 conn(0x7fc09c0c6b20 0x7fc09c0c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a1a00640 1 -- 172.21.15.113:0/3961724999 >> v2:172.21.15.113:3300/0 conn(0x7fc09c0c6b20 msgr2=0x7fc09c0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a1a00640 1 --2- 172.21.15.113:0/3961724999 >> v2:172.21.15.113:3300/0 conn(0x7fc09c0c6b20 0x7fc09c0c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a1a00640 1 -- 172.21.15.113:0/3961724999 >> v1:172.21.15.113:6789/0 conn(0x7fc09c1528b0 legacy=0x7fc09c15c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:03.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.354+0000 7fc0a1a00640 1 -- 172.21.15.113:0/3961724999 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c06d0c0 con 0x7fc09c0c5e70 2024-03-04T07:38:03.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc0a1a00640 1 --2- 172.21.15.113:0/3961724999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc09c0c5e70 0x7fc09c158bb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc09c06fea0 tx=0x7fc09401f110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c73ac260aae5d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:03.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc09b400640 1 -- 172.21.15.113:0/3961724999 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0940b4070 con 0x7fc09c0c5e70 2024-03-04T07:38:03.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc09b400640 1 -- 172.21.15.113:0/3961724999 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0940af600 con 0x7fc09c0c5e70 2024-03-04T07:38:03.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc09b400640 1 -- 172.21.15.113:0/3961724999 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0940af9d0 con 0x7fc09c0c5e70 2024-03-04T07:38:03.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc0a2400640 1 -- 172.21.15.113:0/3961724999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc09c0c5e70 msgr2=0x7fc09c158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:03.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc0a2400640 1 --2- 172.21.15.113:0/3961724999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc09c0c5e70 0x7fc09c158bb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc09c06fea0 tx=0x7fc09401f110 comp rx=0 tx=0).stop 2024-03-04T07:38:03.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc0a2400640 1 -- 172.21.15.113:0/3961724999 shutdown_connections 2024-03-04T07:38:03.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc0a2400640 1 --2- 172.21.15.113:0/3961724999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc09c0c5e70 0x7fc09c158bb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc0a2400640 1 --2- 172.21.15.113:0/3961724999 >> v2:172.21.15.113:3300/0 conn(0x7fc09c0c6b20 0x7fc09c0c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc0a2400640 1 -- 172.21.15.113:0/3961724999 >> 172.21.15.113:0/3961724999 conn(0x7fc09c06e9a0 msgr2=0x7fc09c06edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:03.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc0a2400640 1 -- 172.21.15.113:0/3961724999 shutdown_connections 2024-03-04T07:38:03.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.358+0000 7fc0a2400640 1 -- 172.21.15.113:0/3961724999 wait complete. 2024-03-04T07:38:03.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a2400640 1 Processor -- start 2024-03-04T07:38:03.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a2400640 1 -- start start 2024-03-04T07:38:03.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc09c0c5e70 0x7fc09c14ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:03.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a2400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc09c0c6b20 0x7fc09c14ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:03.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a2400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc09c1528b0 0x7fc09c1e1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:03.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a2400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c152c90 con 0x7fc09c0c5e70 2024-03-04T07:38:03.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a2400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c06c900 con 0x7fc09c0c6b20 2024-03-04T07:38:03.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a2400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c14e410 con 0x7fc09c1528b0 2024-03-04T07:38:03.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a1000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc09c0c5e70 0x7fc09c14ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:03.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a1000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc09c0c5e70 0x7fc09c14ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35520/0 (socket says 172.21.15.113:35520) 2024-03-04T07:38:03.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc0a1000640 1 -- 172.21.15.113:0/4259389637 learned_addr learned my addr 172.21.15.113:0/4259389637 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:03.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.362+0000 7fc09be00640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc09c0c6b20 0x7fc09c14ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:03.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc0a1000640 1 -- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc09c1528b0 msgr2=0x7fc09c1e1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:03.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc0a1000640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc09c1528b0 0x7fc09c1e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc0a1000640 1 -- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc09c0c6b20 msgr2=0x7fc09c14ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:03.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc0a1000640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc09c0c6b20 0x7fc09c14ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc0a1000640 1 -- 172.21.15.113:0/4259389637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c14cc70 con 0x7fc09c0c5e70 2024-03-04T07:38:03.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc0a1000640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc09c0c5e70 0x7fc09c14ded0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc09000c3a0 tx=0x7fc09000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:03.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc098c00640 1 -- 172.21.15.113:0/4259389637 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090017070 con 0x7fc09c0c5e70 2024-03-04T07:38:03.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc098c00640 1 -- 172.21.15.113:0/4259389637 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc090005cf0 con 0x7fc09c0c5e70 2024-03-04T07:38:03.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc098c00640 1 -- 172.21.15.113:0/4259389637 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0900072f0 con 0x7fc09c0c5e70 2024-03-04T07:38:03.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc0a2400640 1 -- 172.21.15.113:0/4259389637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09c14cf30 con 0x7fc09c0c5e70 2024-03-04T07:38:03.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc0a2400640 1 -- 172.21.15.113:0/4259389637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09c1e3c80 con 0x7fc09c0c5e70 2024-03-04T07:38:03.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc0a2400640 1 -- 172.21.15.113:0/4259389637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09c075a80 con 0x7fc09c0c5e70 2024-03-04T07:38:03.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.366+0000 7fc098c00640 1 -- 172.21.15.113:0/4259389637 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc09001a070 con 0x7fc09c0c5e70 2024-03-04T07:38:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.370+0000 7fc098c00640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc07c066f60 0x7fc07c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.370+0000 7fc098c00640 1 -- 172.21.15.113:0/4259389637 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fc090084480 con 0x7fc09c0c5e70 2024-03-04T07:38:03.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.370+0000 7fc09be00640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc07c066f60 0x7fc07c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:03.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.370+0000 7fc09be00640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc07c066f60 0x7fc07c069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc09c1f4820 tx=0x7fc088009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:03.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.374+0000 7fc098c00640 1 -- 172.21.15.113:0/4259389637 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc090054c90 con 0x7fc09c0c5e70 2024-03-04T07:38:03.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.578+0000 7fc0a2400640 1 -- 172.21.15.113:0/4259389637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc09c06cb60 con 0x7fc09c0c5e70 2024-03-04T07:38:03.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc098c00640 1 -- 172.21.15.113:0/4259389637 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5821 (secure 0 0 0) 0x7fc090058d30 con 0x7fc09c0c5e70 2024-03-04T07:38:03.583 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:03.583 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4256,4276,4938]},"id":1}]} 2024-03-04T07:38:03.584 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 24 2024-03-04T07:38:03.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 -- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc07c066f60 msgr2=0x7fc07c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc07c066f60 0x7fc07c069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc09c1f4820 tx=0x7fc088009190 comp rx=0 tx=0).stop 2024-03-04T07:38:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 -- 172.21.15.113:0/4259389637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc09c0c5e70 msgr2=0x7fc09c14ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc09c0c5e70 0x7fc09c14ded0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc09000c3a0 tx=0x7fc09000c860 comp rx=0 tx=0).stop 2024-03-04T07:38:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 -- 172.21.15.113:0/4259389637 shutdown_connections 2024-03-04T07:38:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc09c1528b0 0x7fc09c1e1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc09c0c6b20 0x7fc09c14ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc07c066f60 0x7fc07c069420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 --2- 172.21.15.113:0/4259389637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc09c0c5e70 0x7fc09c14ded0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:03.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 -- 172.21.15.113:0/4259389637 >> 172.21.15.113:0/4259389637 conn(0x7fc09c06e9a0 msgr2=0x7fc09c071fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:03.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 -- 172.21.15.113:0/4259389637 shutdown_connections 2024-03-04T07:38:03.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:03.582+0000 7fc0a2400640 1 -- 172.21.15.113:0/4259389637 wait complete. 2024-03-04T07:38:03.608 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-03-04T07:38:05.608 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:05.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.742+0000 7f95d4400640 1 Processor -- start 2024-03-04T07:38:05.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.742+0000 7f95d4400640 1 -- start start 2024-03-04T07:38:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95d4400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f95cc152bd0 0x7f95cc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95d4400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc1591f0 con 0x7f95cc0c8200 2024-03-04T07:38:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95d4400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95cc1594d0 con 0x7f95cc1527f0 2024-03-04T07:38:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95d4400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95cc071df0 con 0x7f95cc152bd0 2024-03-04T07:38:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95d1000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f95cc0c8200 0x7f95cc1587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52194/0 (socket says 172.21.15.113:52194) 2024-03-04T07:38:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95d1000640 1 -- 172.21.15.113:0/1690203703 learned_addr learned my addr 172.21.15.113:0/1690203703 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:38:05.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95d2400640 1 --2- 172.21.15.113:0/1690203703 >> v2:172.21.15.136:3300/0 conn(0x7f95cc152bd0 0x7f95cc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95cbe00640 1 -- 172.21.15.113:0/1690203703 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518584157 0 0) 0x7f95cc1591f0 con 0x7f95cc0c8200 2024-03-04T07:38:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95cbe00640 1 -- 172.21.15.113:0/1690203703 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b4003530 con 0x7f95cc0c8200 2024-03-04T07:38:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95d2400640 1 -- 172.21.15.113:0/1690203703 >> v1:172.21.15.136:6789/0 conn(0x7f95cc1527f0 legacy=0x7f95cc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95d2400640 1 -- 172.21.15.113:0/1690203703 >> v1:172.21.15.113:6789/0 conn(0x7f95cc0c8200 legacy=0x7f95cc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.746+0000 7f95d2400640 1 -- 172.21.15.113:0/1690203703 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0717a0 con 0x7f95cc152bd0 2024-03-04T07:38:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95d2400640 1 --2- 172.21.15.113:0/1690203703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc152bd0 0x7f95cc070d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009d90 tx=0x7f95c401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43503ce7045d1b6b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95cbe00640 1 -- 172.21.15.113:0/1690203703 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40b2070 con 0x7f95cc152bd0 2024-03-04T07:38:05.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95cbe00640 1 -- 172.21.15.113:0/1690203703 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c40a5830 con 0x7f95cc152bd0 2024-03-04T07:38:05.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95cbe00640 1 -- 172.21.15.113:0/1690203703 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40a5be0 con 0x7f95cc152bd0 2024-03-04T07:38:05.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95d4400640 1 -- 172.21.15.113:0/1690203703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc152bd0 msgr2=0x7f95cc070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:05.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95d4400640 1 --2- 172.21.15.113:0/1690203703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc152bd0 0x7f95cc070d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f95c4009d90 tx=0x7f95c401c970 comp rx=0 tx=0).stop 2024-03-04T07:38:05.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95d4400640 1 -- 172.21.15.113:0/1690203703 shutdown_connections 2024-03-04T07:38:05.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95d4400640 1 --2- 172.21.15.113:0/1690203703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc152bd0 0x7f95cc070d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:05.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95d4400640 1 -- 172.21.15.113:0/1690203703 >> 172.21.15.113:0/1690203703 conn(0x7f95cc06e820 msgr2=0x7f95cc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:05.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95d4400640 1 -- 172.21.15.113:0/1690203703 shutdown_connections 2024-03-04T07:38:05.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.750+0000 7f95d4400640 1 -- 172.21.15.113:0/1690203703 wait complete. 2024-03-04T07:38:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d4400640 1 Processor -- start 2024-03-04T07:38:05.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d4400640 1 -- start start 2024-03-04T07:38:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d4400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc0c8200 0x7f95cc143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d4400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95cc1527f0 0x7f95cc0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d4400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc152bd0 0x7f95cc14a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d4400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc166500 con 0x7f95cc152bd0 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d4400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc166380 con 0x7f95cc0c8200 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d4400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc166680 con 0x7f95cc1527f0 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc152bd0 0x7f95cc14a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d1a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc0c8200 0x7f95cc143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc152bd0 0x7f95cc14a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35524/0 (socket says 172.21.15.113:35524) 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d2400640 1 -- 172.21.15.113:0/52613416 learned_addr learned my addr 172.21.15.113:0/52613416 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d1000640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95cc1527f0 0x7f95cc0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d2400640 1 -- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95cc1527f0 msgr2=0x7f95cc0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d2400640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95cc1527f0 0x7f95cc0afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d2400640 1 -- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc0c8200 msgr2=0x7f95cc143ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d2400640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc0c8200 0x7f95cc143ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:05.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d2400640 1 -- 172.21.15.113:0/52613416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0cbd20 con 0x7f95cc152bd0 2024-03-04T07:38:05.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d1000640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95cc1527f0 0x7f95cc0afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:38:05.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d2400640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc152bd0 0x7f95cc14a910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f95c40b0040 tx=0x7f95c4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:05.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95c9600640 1 -- 172.21.15.113:0/52613416 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40b2050 con 0x7f95cc152bd0 2024-03-04T07:38:05.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95c9600640 1 -- 172.21.15.113:0/52613416 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c40be070 con 0x7f95cc152bd0 2024-03-04T07:38:05.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d4400640 1 -- 172.21.15.113:0/52613416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc0cab20 con 0x7f95cc152bd0 2024-03-04T07:38:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95c9600640 1 -- 172.21.15.113:0/52613416 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4003db0 con 0x7f95cc152bd0 2024-03-04T07:38:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.754+0000 7f95d4400640 1 -- 172.21.15.113:0/52613416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc1441e0 con 0x7f95cc152bd0 2024-03-04T07:38:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.758+0000 7f95d4400640 1 -- 172.21.15.113:0/52613416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc075c00 con 0x7f95cc152bd0 2024-03-04T07:38:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.758+0000 7f95c9600640 1 -- 172.21.15.113:0/52613416 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95c4004040 con 0x7f95cc152bd0 2024-03-04T07:38:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.758+0000 7f95c9600640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95ac066f60 0x7f95ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.758+0000 7f95c9600640 1 -- 172.21.15.113:0/52613416 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f95c41200d0 con 0x7f95cc152bd0 2024-03-04T07:38:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.758+0000 7f95d1a00640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95ac066f60 0x7f95ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.758+0000 7f95d1a00640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95ac066f60 0x7f95ac069420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f95c0001810 tx=0x7f95c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.762+0000 7f95c9600640 1 -- 172.21.15.113:0/52613416 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f95c4016560 con 0x7f95cc152bd0 2024-03-04T07:38:05.912 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/30 file after the snapshot 2024-03-04T07:38:05.912 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/31 file after the snapshot 2024-03-04T07:38:05.912 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/32 file after the snapshot 2024-03-04T07:38:05.912 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/33 file after the snapshot 2024-03-04T07:38:05.912 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/34 file after the snapshot 2024-03-04T07:38:05.912 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/35 file after the snapshot 2024-03-04T07:38:05.912 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/36 file after the snapshot 2024-03-04T07:38:05.912 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/37 file after the snapshot 2024-03-04T07:38:05.912 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/38 file after the snapshot 2024-03-04T07:38:05.913 INFO:tasks.workunit.client.0.smithi113.stdout: create 266/39 file after the snapshot 2024-03-04T07:38:05.913 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/.snap/snap-subdir-test 2024-03-04T07:38:05.913 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/30 file after the snapshot 2024-03-04T07:38:05.913 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/31 file after the snapshot 2024-03-04T07:38:05.913 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/32 file after the snapshot 2024-03-04T07:38:06.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.998+0000 7f95d4400640 1 -- 172.21.15.113:0/52613416 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95cc0b1cc0 con 0x7f95cc152bd0 2024-03-04T07:38:06.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:05.998+0000 7f95c9600640 1 -- 172.21.15.113:0/52613416 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f95c40f0fb0 con 0x7f95cc152bd0 2024-03-04T07:38:06.001 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:06.001 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276,4256]},"id":1}]} 2024-03-04T07:38:06.002 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-03-04T07:38:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 -- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95ac066f60 msgr2=0x7f95ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95ac066f60 0x7f95ac069420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f95c0001810 tx=0x7f95c0009040 comp rx=0 tx=0).stop 2024-03-04T07:38:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 -- 172.21.15.113:0/52613416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc152bd0 msgr2=0x7f95cc14a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:06.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc152bd0 0x7f95cc14a910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f95c40b0040 tx=0x7f95c4019710 comp rx=0 tx=0).stop 2024-03-04T07:38:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 -- 172.21.15.113:0/52613416 shutdown_connections 2024-03-04T07:38:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95ac066f60 0x7f95ac069420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95cc152bd0 0x7f95cc14a910 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95cc1527f0 0x7f95cc0afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 --2- 172.21.15.113:0/52613416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95cc0c8200 0x7f95cc143ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 -- 172.21.15.113:0/52613416 >> 172.21.15.113:0/52613416 conn(0x7f95cc06e820 msgr2=0x7f95cc1563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 -- 172.21.15.113:0/52613416 shutdown_connections 2024-03-04T07:38:06.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:06.002+0000 7f95d4400640 1 -- 172.21.15.113:0/52613416 wait complete. 2024-03-04T07:38:06.032 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-03-04T07:38:06.032 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 28 secs before thrashing 2024-03-04T07:38:06.132 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/33 file after the snapshot 2024-03-04T07:38:06.133 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/34 file after the snapshot 2024-03-04T07:38:06.135 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/35 file after the snapshot 2024-03-04T07:38:06.136 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/36 file after the snapshot 2024-03-04T07:38:06.138 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/37 file after the snapshot 2024-03-04T07:38:06.139 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/38 file after the snapshot 2024-03-04T07:38:06.141 INFO:tasks.workunit.client.0.smithi113.stdout: create 267/39 file after the snapshot 2024-03-04T07:38:06.143 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/.snap/snap-subdir-test 2024-03-04T07:38:06.167 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/30 file after the snapshot 2024-03-04T07:38:06.169 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/31 file after the snapshot 2024-03-04T07:38:06.171 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/32 file after the snapshot 2024-03-04T07:38:06.172 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/33 file after the snapshot 2024-03-04T07:38:06.174 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/34 file after the snapshot 2024-03-04T07:38:06.176 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/35 file after the snapshot 2024-03-04T07:38:06.177 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/36 file after the snapshot 2024-03-04T07:38:06.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/37 file after the snapshot 2024-03-04T07:38:06.180 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/38 file after the snapshot 2024-03-04T07:38:06.182 INFO:tasks.workunit.client.0.smithi113.stdout: create 268/39 file after the snapshot 2024-03-04T07:38:06.184 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/.snap/snap-subdir-test 2024-03-04T07:38:06.210 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/30 file after the snapshot 2024-03-04T07:38:06.212 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/31 file after the snapshot 2024-03-04T07:38:06.214 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/32 file after the snapshot 2024-03-04T07:38:06.215 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/33 file after the snapshot 2024-03-04T07:38:06.217 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/34 file after the snapshot 2024-03-04T07:38:06.219 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/35 file after the snapshot 2024-03-04T07:38:06.220 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/36 file after the snapshot 2024-03-04T07:38:06.222 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/37 file after the snapshot 2024-03-04T07:38:06.223 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/38 file after the snapshot 2024-03-04T07:38:06.224 INFO:tasks.workunit.client.0.smithi113.stdout: create 269/39 file after the snapshot 2024-03-04T07:38:06.226 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/.snap/snap-subdir-test 2024-03-04T07:38:06.259 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/30 file after the snapshot 2024-03-04T07:38:06.261 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/31 file after the snapshot 2024-03-04T07:38:06.262 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/32 file after the snapshot 2024-03-04T07:38:06.264 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/33 file after the snapshot 2024-03-04T07:38:06.265 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/34 file after the snapshot 2024-03-04T07:38:06.267 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/35 file after the snapshot 2024-03-04T07:38:06.268 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/36 file after the snapshot 2024-03-04T07:38:06.270 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/37 file after the snapshot 2024-03-04T07:38:06.271 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/38 file after the snapshot 2024-03-04T07:38:06.273 INFO:tasks.workunit.client.0.smithi113.stdout: create 270/39 file after the snapshot 2024-03-04T07:38:06.275 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/.snap/snap-subdir-test 2024-03-04T07:38:06.300 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/30 file after the snapshot 2024-03-04T07:38:06.302 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/31 file after the snapshot 2024-03-04T07:38:06.303 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/32 file after the snapshot 2024-03-04T07:38:06.305 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/33 file after the snapshot 2024-03-04T07:38:06.306 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/34 file after the snapshot 2024-03-04T07:38:06.308 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/35 file after the snapshot 2024-03-04T07:38:06.309 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/36 file after the snapshot 2024-03-04T07:38:06.311 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/37 file after the snapshot 2024-03-04T07:38:06.312 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/38 file after the snapshot 2024-03-04T07:38:06.314 INFO:tasks.workunit.client.0.smithi113.stdout: create 271/39 file after the snapshot 2024-03-04T07:38:06.315 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/.snap/snap-subdir-test 2024-03-04T07:38:06.341 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/30 file after the snapshot 2024-03-04T07:38:06.343 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/31 file after the snapshot 2024-03-04T07:38:06.344 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/32 file after the snapshot 2024-03-04T07:38:06.346 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/33 file after the snapshot 2024-03-04T07:38:06.347 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/34 file after the snapshot 2024-03-04T07:38:06.349 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/35 file after the snapshot 2024-03-04T07:38:06.350 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/36 file after the snapshot 2024-03-04T07:38:06.352 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/37 file after the snapshot 2024-03-04T07:38:06.353 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/38 file after the snapshot 2024-03-04T07:38:06.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 272/39 file after the snapshot 2024-03-04T07:38:06.356 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/.snap/snap-subdir-test 2024-03-04T07:38:06.381 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/30 file after the snapshot 2024-03-04T07:38:06.383 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/31 file after the snapshot 2024-03-04T07:38:06.385 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/32 file after the snapshot 2024-03-04T07:38:06.387 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/33 file after the snapshot 2024-03-04T07:38:06.388 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/34 file after the snapshot 2024-03-04T07:38:06.390 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/35 file after the snapshot 2024-03-04T07:38:06.391 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/36 file after the snapshot 2024-03-04T07:38:06.393 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/37 file after the snapshot 2024-03-04T07:38:06.395 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/38 file after the snapshot 2024-03-04T07:38:06.396 INFO:tasks.workunit.client.0.smithi113.stdout: create 273/39 file after the snapshot 2024-03-04T07:38:06.398 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/.snap/snap-subdir-test 2024-03-04T07:38:06.423 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/30 file after the snapshot 2024-03-04T07:38:06.425 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/31 file after the snapshot 2024-03-04T07:38:06.426 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/32 file after the snapshot 2024-03-04T07:38:06.428 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/33 file after the snapshot 2024-03-04T07:38:06.430 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/34 file after the snapshot 2024-03-04T07:38:06.431 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/35 file after the snapshot 2024-03-04T07:38:06.433 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/36 file after the snapshot 2024-03-04T07:38:06.435 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/37 file after the snapshot 2024-03-04T07:38:06.436 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/38 file after the snapshot 2024-03-04T07:38:06.438 INFO:tasks.workunit.client.0.smithi113.stdout: create 274/39 file after the snapshot 2024-03-04T07:38:06.440 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/.snap/snap-subdir-test 2024-03-04T07:38:06.465 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/30 file after the snapshot 2024-03-04T07:38:06.467 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/31 file after the snapshot 2024-03-04T07:38:06.469 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/32 file after the snapshot 2024-03-04T07:38:06.471 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/33 file after the snapshot 2024-03-04T07:38:06.472 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/34 file after the snapshot 2024-03-04T07:38:06.474 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/35 file after the snapshot 2024-03-04T07:38:06.476 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/36 file after the snapshot 2024-03-04T07:38:06.477 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/37 file after the snapshot 2024-03-04T07:38:06.479 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/38 file after the snapshot 2024-03-04T07:38:06.480 INFO:tasks.workunit.client.0.smithi113.stdout: create 275/39 file after the snapshot 2024-03-04T07:38:06.482 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/.snap/snap-subdir-test 2024-03-04T07:38:06.508 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/30 file after the snapshot 2024-03-04T07:38:06.510 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/31 file after the snapshot 2024-03-04T07:38:06.512 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/32 file after the snapshot 2024-03-04T07:38:06.513 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/33 file after the snapshot 2024-03-04T07:38:06.515 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/34 file after the snapshot 2024-03-04T07:38:06.516 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/35 file after the snapshot 2024-03-04T07:38:06.518 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/36 file after the snapshot 2024-03-04T07:38:06.519 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/37 file after the snapshot 2024-03-04T07:38:06.521 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/38 file after the snapshot 2024-03-04T07:38:06.522 INFO:tasks.workunit.client.0.smithi113.stdout: create 276/39 file after the snapshot 2024-03-04T07:38:06.524 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/.snap/snap-subdir-test 2024-03-04T07:38:06.558 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/30 file after the snapshot 2024-03-04T07:38:06.560 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/31 file after the snapshot 2024-03-04T07:38:06.562 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/32 file after the snapshot 2024-03-04T07:38:06.564 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/33 file after the snapshot 2024-03-04T07:38:06.566 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/34 file after the snapshot 2024-03-04T07:38:06.567 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/35 file after the snapshot 2024-03-04T07:38:06.569 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/36 file after the snapshot 2024-03-04T07:38:06.571 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/37 file after the snapshot 2024-03-04T07:38:06.572 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/38 file after the snapshot 2024-03-04T07:38:06.574 INFO:tasks.workunit.client.0.smithi113.stdout: create 277/39 file after the snapshot 2024-03-04T07:38:06.575 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/.snap/snap-subdir-test 2024-03-04T07:38:06.605 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/30 file after the snapshot 2024-03-04T07:38:06.608 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/31 file after the snapshot 2024-03-04T07:38:06.609 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/32 file after the snapshot 2024-03-04T07:38:06.611 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/33 file after the snapshot 2024-03-04T07:38:06.612 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/34 file after the snapshot 2024-03-04T07:38:06.614 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/35 file after the snapshot 2024-03-04T07:38:06.616 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/36 file after the snapshot 2024-03-04T07:38:06.619 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/37 file after the snapshot 2024-03-04T07:38:06.621 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/38 file after the snapshot 2024-03-04T07:38:06.624 INFO:tasks.workunit.client.0.smithi113.stdout: create 278/39 file after the snapshot 2024-03-04T07:38:06.626 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/.snap/snap-subdir-test 2024-03-04T07:38:06.655 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/30 file after the snapshot 2024-03-04T07:38:06.658 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/31 file after the snapshot 2024-03-04T07:38:06.660 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/32 file after the snapshot 2024-03-04T07:38:06.663 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/33 file after the snapshot 2024-03-04T07:38:06.665 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/34 file after the snapshot 2024-03-04T07:38:06.668 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/35 file after the snapshot 2024-03-04T07:38:06.671 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/36 file after the snapshot 2024-03-04T07:38:06.673 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/37 file after the snapshot 2024-03-04T07:38:06.676 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/38 file after the snapshot 2024-03-04T07:38:06.678 INFO:tasks.workunit.client.0.smithi113.stdout: create 279/39 file after the snapshot 2024-03-04T07:38:06.680 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/.snap/snap-subdir-test 2024-03-04T07:38:06.710 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/30 file after the snapshot 2024-03-04T07:38:06.712 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/31 file after the snapshot 2024-03-04T07:38:06.713 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/32 file after the snapshot 2024-03-04T07:38:06.715 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/33 file after the snapshot 2024-03-04T07:38:06.716 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/34 file after the snapshot 2024-03-04T07:38:06.718 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/35 file after the snapshot 2024-03-04T07:38:06.720 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/36 file after the snapshot 2024-03-04T07:38:06.721 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/37 file after the snapshot 2024-03-04T07:38:06.723 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/38 file after the snapshot 2024-03-04T07:38:06.724 INFO:tasks.workunit.client.0.smithi113.stdout: create 280/39 file after the snapshot 2024-03-04T07:38:06.726 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/.snap/snap-subdir-test 2024-03-04T07:38:06.753 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/30 file after the snapshot 2024-03-04T07:38:06.755 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/31 file after the snapshot 2024-03-04T07:38:06.757 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/32 file after the snapshot 2024-03-04T07:38:06.759 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/33 file after the snapshot 2024-03-04T07:38:06.760 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/34 file after the snapshot 2024-03-04T07:38:06.762 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/35 file after the snapshot 2024-03-04T07:38:06.763 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/36 file after the snapshot 2024-03-04T07:38:06.765 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/37 file after the snapshot 2024-03-04T07:38:06.766 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/38 file after the snapshot 2024-03-04T07:38:06.768 INFO:tasks.workunit.client.0.smithi113.stdout: create 281/39 file after the snapshot 2024-03-04T07:38:06.770 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/.snap/snap-subdir-test 2024-03-04T07:38:06.796 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/30 file after the snapshot 2024-03-04T07:38:06.798 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/31 file after the snapshot 2024-03-04T07:38:06.800 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/32 file after the snapshot 2024-03-04T07:38:06.802 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/33 file after the snapshot 2024-03-04T07:38:06.803 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/34 file after the snapshot 2024-03-04T07:38:06.804 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/35 file after the snapshot 2024-03-04T07:38:06.806 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/36 file after the snapshot 2024-03-04T07:38:06.807 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/37 file after the snapshot 2024-03-04T07:38:06.809 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/38 file after the snapshot 2024-03-04T07:38:06.810 INFO:tasks.workunit.client.0.smithi113.stdout: create 282/39 file after the snapshot 2024-03-04T07:38:06.812 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/.snap/snap-subdir-test 2024-03-04T07:38:07.111 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/30 file after the snapshot 2024-03-04T07:38:07.112 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/31 file after the snapshot 2024-03-04T07:38:07.114 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/32 file after the snapshot 2024-03-04T07:38:07.115 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/33 file after the snapshot 2024-03-04T07:38:07.117 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/34 file after the snapshot 2024-03-04T07:38:07.118 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/35 file after the snapshot 2024-03-04T07:38:07.119 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/36 file after the snapshot 2024-03-04T07:38:07.121 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/37 file after the snapshot 2024-03-04T07:38:07.122 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/38 file after the snapshot 2024-03-04T07:38:07.123 INFO:tasks.workunit.client.0.smithi113.stdout: create 283/39 file after the snapshot 2024-03-04T07:38:07.125 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/.snap/snap-subdir-test 2024-03-04T07:38:07.151 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/30 file after the snapshot 2024-03-04T07:38:07.153 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/31 file after the snapshot 2024-03-04T07:38:07.154 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/32 file after the snapshot 2024-03-04T07:38:07.156 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/33 file after the snapshot 2024-03-04T07:38:07.157 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/34 file after the snapshot 2024-03-04T07:38:07.158 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/35 file after the snapshot 2024-03-04T07:38:08.129 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/36 file after the snapshot 2024-03-04T07:38:08.145 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/37 file after the snapshot 2024-03-04T07:38:08.146 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/38 file after the snapshot 2024-03-04T07:38:08.148 INFO:tasks.workunit.client.0.smithi113.stdout: create 284/39 file after the snapshot 2024-03-04T07:38:08.150 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/.snap/snap-subdir-test 2024-03-04T07:38:08.177 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/30 file after the snapshot 2024-03-04T07:38:08.179 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/31 file after the snapshot 2024-03-04T07:38:08.180 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/32 file after the snapshot 2024-03-04T07:38:08.182 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/33 file after the snapshot 2024-03-04T07:38:08.183 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/34 file after the snapshot 2024-03-04T07:38:08.185 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/35 file after the snapshot 2024-03-04T07:38:08.186 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/36 file after the snapshot 2024-03-04T07:38:08.188 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/37 file after the snapshot 2024-03-04T07:38:08.189 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/38 file after the snapshot 2024-03-04T07:38:08.190 INFO:tasks.workunit.client.0.smithi113.stdout: create 285/39 file after the snapshot 2024-03-04T07:38:08.192 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/.snap/snap-subdir-test 2024-03-04T07:38:08.210 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/30 file after the snapshot 2024-03-04T07:38:08.213 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/31 file after the snapshot 2024-03-04T07:38:08.214 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/32 file after the snapshot 2024-03-04T07:38:08.215 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/33 file after the snapshot 2024-03-04T07:38:08.217 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/34 file after the snapshot 2024-03-04T07:38:08.218 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/35 file after the snapshot 2024-03-04T07:38:08.220 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/36 file after the snapshot 2024-03-04T07:38:08.222 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/37 file after the snapshot 2024-03-04T07:38:08.223 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/38 file after the snapshot 2024-03-04T07:38:08.225 INFO:tasks.workunit.client.0.smithi113.stdout: create 286/39 file after the snapshot 2024-03-04T07:38:08.349 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/.snap/snap-subdir-test 2024-03-04T07:38:08.381 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/30 file after the snapshot 2024-03-04T07:38:08.383 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/31 file after the snapshot 2024-03-04T07:38:08.384 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/32 file after the snapshot 2024-03-04T07:38:08.385 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/33 file after the snapshot 2024-03-04T07:38:08.387 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/34 file after the snapshot 2024-03-04T07:38:08.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/35 file after the snapshot 2024-03-04T07:38:08.390 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/36 file after the snapshot 2024-03-04T07:38:08.392 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/37 file after the snapshot 2024-03-04T07:38:08.393 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/38 file after the snapshot 2024-03-04T07:38:08.395 INFO:tasks.workunit.client.0.smithi113.stdout: create 287/39 file after the snapshot 2024-03-04T07:38:08.396 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/.snap/snap-subdir-test 2024-03-04T07:38:08.430 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/30 file after the snapshot 2024-03-04T07:38:08.433 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/31 file after the snapshot 2024-03-04T07:38:08.434 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/32 file after the snapshot 2024-03-04T07:38:08.436 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/33 file after the snapshot 2024-03-04T07:38:08.437 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/34 file after the snapshot 2024-03-04T07:38:08.439 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/35 file after the snapshot 2024-03-04T07:38:08.441 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/36 file after the snapshot 2024-03-04T07:38:08.443 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/37 file after the snapshot 2024-03-04T07:38:08.445 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/38 file after the snapshot 2024-03-04T07:38:08.446 INFO:tasks.workunit.client.0.smithi113.stdout: create 288/39 file after the snapshot 2024-03-04T07:38:08.448 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/.snap/snap-subdir-test 2024-03-04T07:38:08.483 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/30 file after the snapshot 2024-03-04T07:38:08.485 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/31 file after the snapshot 2024-03-04T07:38:08.486 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/32 file after the snapshot 2024-03-04T07:38:08.488 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/33 file after the snapshot 2024-03-04T07:38:08.489 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/34 file after the snapshot 2024-03-04T07:38:08.490 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/35 file after the snapshot 2024-03-04T07:38:08.492 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/36 file after the snapshot 2024-03-04T07:38:08.493 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/37 file after the snapshot 2024-03-04T07:38:08.494 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/38 file after the snapshot 2024-03-04T07:38:08.495 INFO:tasks.workunit.client.0.smithi113.stdout: create 289/39 file after the snapshot 2024-03-04T07:38:08.497 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/.snap/snap-subdir-test 2024-03-04T07:38:08.967 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/30 file after the snapshot 2024-03-04T07:38:08.969 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/31 file after the snapshot 2024-03-04T07:38:08.970 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/32 file after the snapshot 2024-03-04T07:38:08.972 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/33 file after the snapshot 2024-03-04T07:38:08.974 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/34 file after the snapshot 2024-03-04T07:38:08.975 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/35 file after the snapshot 2024-03-04T07:38:08.977 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/36 file after the snapshot 2024-03-04T07:38:08.979 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/37 file after the snapshot 2024-03-04T07:38:08.980 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/38 file after the snapshot 2024-03-04T07:38:08.982 INFO:tasks.workunit.client.0.smithi113.stdout: create 290/39 file after the snapshot 2024-03-04T07:38:08.984 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/.snap/snap-subdir-test 2024-03-04T07:38:09.012 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/30 file after the snapshot 2024-03-04T07:38:09.014 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/31 file after the snapshot 2024-03-04T07:38:09.016 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/32 file after the snapshot 2024-03-04T07:38:09.018 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/33 file after the snapshot 2024-03-04T07:38:09.020 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/34 file after the snapshot 2024-03-04T07:38:09.021 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/35 file after the snapshot 2024-03-04T07:38:09.023 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/36 file after the snapshot 2024-03-04T07:38:09.025 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/37 file after the snapshot 2024-03-04T07:38:09.026 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/38 file after the snapshot 2024-03-04T07:38:09.028 INFO:tasks.workunit.client.0.smithi113.stdout: create 291/39 file after the snapshot 2024-03-04T07:38:09.030 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/.snap/snap-subdir-test 2024-03-04T07:38:09.058 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/30 file after the snapshot 2024-03-04T07:38:09.061 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/31 file after the snapshot 2024-03-04T07:38:09.062 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/32 file after the snapshot 2024-03-04T07:38:09.064 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/33 file after the snapshot 2024-03-04T07:38:09.066 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/34 file after the snapshot 2024-03-04T07:38:09.067 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/35 file after the snapshot 2024-03-04T07:38:09.069 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/36 file after the snapshot 2024-03-04T07:38:09.071 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/37 file after the snapshot 2024-03-04T07:38:09.072 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/38 file after the snapshot 2024-03-04T07:38:09.074 INFO:tasks.workunit.client.0.smithi113.stdout: create 292/39 file after the snapshot 2024-03-04T07:38:09.076 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/.snap/snap-subdir-test 2024-03-04T07:38:09.846 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/30 file after the snapshot 2024-03-04T07:38:09.848 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/31 file after the snapshot 2024-03-04T07:38:09.849 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/32 file after the snapshot 2024-03-04T07:38:09.851 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/33 file after the snapshot 2024-03-04T07:38:09.853 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/34 file after the snapshot 2024-03-04T07:38:09.854 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/35 file after the snapshot 2024-03-04T07:38:09.856 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/36 file after the snapshot 2024-03-04T07:38:09.857 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/37 file after the snapshot 2024-03-04T07:38:09.859 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/38 file after the snapshot 2024-03-04T07:38:09.861 INFO:tasks.workunit.client.0.smithi113.stdout: create 293/39 file after the snapshot 2024-03-04T07:38:09.862 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/.snap/snap-subdir-test 2024-03-04T07:38:09.899 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/30 file after the snapshot 2024-03-04T07:38:09.902 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/31 file after the snapshot 2024-03-04T07:38:09.904 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/32 file after the snapshot 2024-03-04T07:38:09.906 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/33 file after the snapshot 2024-03-04T07:38:09.909 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/34 file after the snapshot 2024-03-04T07:38:09.911 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/35 file after the snapshot 2024-03-04T07:38:09.914 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/36 file after the snapshot 2024-03-04T07:38:09.916 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/37 file after the snapshot 2024-03-04T07:38:09.918 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/38 file after the snapshot 2024-03-04T07:38:09.919 INFO:tasks.workunit.client.0.smithi113.stdout: create 294/39 file after the snapshot 2024-03-04T07:38:09.921 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/.snap/snap-subdir-test 2024-03-04T07:38:09.946 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/30 file after the snapshot 2024-03-04T07:38:09.948 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/31 file after the snapshot 2024-03-04T07:38:09.950 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/32 file after the snapshot 2024-03-04T07:38:09.952 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/33 file after the snapshot 2024-03-04T07:38:09.954 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/34 file after the snapshot 2024-03-04T07:38:09.956 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/35 file after the snapshot 2024-03-04T07:38:09.958 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/36 file after the snapshot 2024-03-04T07:38:09.959 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/37 file after the snapshot 2024-03-04T07:38:09.961 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/38 file after the snapshot 2024-03-04T07:38:09.963 INFO:tasks.workunit.client.0.smithi113.stdout: create 295/39 file after the snapshot 2024-03-04T07:38:09.965 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/.snap/snap-subdir-test 2024-03-04T07:38:09.995 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/30 file after the snapshot 2024-03-04T07:38:09.997 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/31 file after the snapshot 2024-03-04T07:38:09.999 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/32 file after the snapshot 2024-03-04T07:38:10.001 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/33 file after the snapshot 2024-03-04T07:38:10.002 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/34 file after the snapshot 2024-03-04T07:38:10.004 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/35 file after the snapshot 2024-03-04T07:38:10.006 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/36 file after the snapshot 2024-03-04T07:38:10.007 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/37 file after the snapshot 2024-03-04T07:38:10.009 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/38 file after the snapshot 2024-03-04T07:38:10.010 INFO:tasks.workunit.client.0.smithi113.stdout: create 296/39 file after the snapshot 2024-03-04T07:38:10.012 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/.snap/snap-subdir-test 2024-03-04T07:38:10.291 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/30 file after the snapshot 2024-03-04T07:38:10.294 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/31 file after the snapshot 2024-03-04T07:38:10.295 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/32 file after the snapshot 2024-03-04T07:38:10.297 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/33 file after the snapshot 2024-03-04T07:38:10.299 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/34 file after the snapshot 2024-03-04T07:38:10.300 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/35 file after the snapshot 2024-03-04T07:38:10.302 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/36 file after the snapshot 2024-03-04T07:38:10.304 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/37 file after the snapshot 2024-03-04T07:38:10.305 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/38 file after the snapshot 2024-03-04T07:38:10.307 INFO:tasks.workunit.client.0.smithi113.stdout: create 297/39 file after the snapshot 2024-03-04T07:38:10.309 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/.snap/snap-subdir-test 2024-03-04T07:38:10.336 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/30 file after the snapshot 2024-03-04T07:38:10.339 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/31 file after the snapshot 2024-03-04T07:38:10.340 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/32 file after the snapshot 2024-03-04T07:38:10.342 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/33 file after the snapshot 2024-03-04T07:38:10.344 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/34 file after the snapshot 2024-03-04T07:38:10.348 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/35 file after the snapshot 2024-03-04T07:38:10.348 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/36 file after the snapshot 2024-03-04T07:38:10.349 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/37 file after the snapshot 2024-03-04T07:38:10.351 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/38 file after the snapshot 2024-03-04T07:38:10.352 INFO:tasks.workunit.client.0.smithi113.stdout: create 298/39 file after the snapshot 2024-03-04T07:38:10.354 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/.snap/snap-subdir-test 2024-03-04T07:38:10.382 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/30 file after the snapshot 2024-03-04T07:38:10.385 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/31 file after the snapshot 2024-03-04T07:38:10.386 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/32 file after the snapshot 2024-03-04T07:38:10.388 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/33 file after the snapshot 2024-03-04T07:38:10.389 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/34 file after the snapshot 2024-03-04T07:38:10.391 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/35 file after the snapshot 2024-03-04T07:38:10.393 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/36 file after the snapshot 2024-03-04T07:38:10.395 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/37 file after the snapshot 2024-03-04T07:38:10.396 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/38 file after the snapshot 2024-03-04T07:38:10.398 INFO:tasks.workunit.client.0.smithi113.stdout: create 299/39 file after the snapshot 2024-03-04T07:38:10.399 INFO:tasks.workunit.client.0.smithi113.stdout:Take second snapshot .snap/test2 2024-03-04T07:38:10.433 INFO:tasks.workunit.client.0.smithi113.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2024-03-04T07:38:13.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2024-03-04T07:38:13.228 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2024-03-04T07:38:13.229 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2024-03-04T07:38:13.230 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2024-03-04T07:38:13.231 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2024-03-04T07:38:13.232 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2024-03-04T07:38:13.233 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2024-03-04T07:38:13.234 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2024-03-04T07:38:13.234 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2024-03-04T07:38:13.234 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2024-03-04T07:38:14.100 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2024-03-04T07:38:14.100 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2024-03-04T07:38:14.100 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2024-03-04T07:38:14.100 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2024-03-04T07:38:14.100 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2024-03-04T07:38:14.100 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2024-03-04T07:38:14.100 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2024-03-04T07:38:14.100 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2024-03-04T07:38:14.101 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2024-03-04T07:38:14.102 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2024-03-04T07:38:14.103 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2024-03-04T07:38:14.104 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2024-03-04T07:38:14.105 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2024-03-04T07:38:14.106 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2024-03-04T07:38:14.106 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2024-03-04T07:38:14.106 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2024-03-04T07:38:14.106 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2024-03-04T07:38:14.863 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2024-03-04T07:38:14.864 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2024-03-04T07:38:14.865 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2024-03-04T07:38:14.866 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2024-03-04T07:38:14.867 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2024-03-04T07:38:14.868 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2024-03-04T07:38:14.868 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2024-03-04T07:38:14.868 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2024-03-04T07:38:14.868 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2024-03-04T07:38:14.868 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2024-03-04T07:38:14.868 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2024-03-04T07:38:14.868 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2024-03-04T07:38:14.868 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2024-03-04T07:38:14.868 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2024-03-04T07:38:14.868 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2024-03-04T07:38:17.157 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2024-03-04T07:38:17.157 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2024-03-04T07:38:17.158 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2024-03-04T07:38:17.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2024-03-04T07:38:17.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2024-03-04T07:38:17.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2024-03-04T07:38:17.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2024-03-04T07:38:17.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2024-03-04T07:38:17.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2024-03-04T07:38:17.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2024-03-04T07:38:17.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2024-03-04T07:38:17.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2024-03-04T07:38:17.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2024-03-04T07:38:17.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2024-03-04T07:38:19.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2024-03-04T07:38:19.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2024-03-04T07:38:19.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2024-03-04T07:38:19.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2024-03-04T07:38:19.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2024-03-04T07:38:19.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2024-03-04T07:38:19.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2024-03-04T07:38:19.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2024-03-04T07:38:19.159 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2024-03-04T07:38:19.160 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2024-03-04T07:38:19.161 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2024-03-04T07:38:19.162 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2024-03-04T07:38:19.163 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2024-03-04T07:38:19.164 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2024-03-04T07:38:19.165 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2024-03-04T07:38:21.087 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2024-03-04T07:38:21.087 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2024-03-04T07:38:21.087 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2024-03-04T07:38:21.087 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2024-03-04T07:38:21.087 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2024-03-04T07:38:21.088 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2024-03-04T07:38:21.089 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2024-03-04T07:38:21.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2024-03-04T07:38:21.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2024-03-04T07:38:21.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2024-03-04T07:38:21.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2024-03-04T07:38:21.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2024-03-04T07:38:21.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2024-03-04T07:38:21.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2024-03-04T07:38:21.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2024-03-04T07:38:21.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2024-03-04T07:38:21.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2024-03-04T07:38:21.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2024-03-04T07:38:21.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2024-03-04T07:38:21.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2024-03-04T07:38:21.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2024-03-04T07:38:21.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2024-03-04T07:38:21.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2024-03-04T07:38:21.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2024-03-04T07:38:21.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2024-03-04T07:38:21.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2024-03-04T07:38:21.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2024-03-04T07:38:22.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2024-03-04T07:38:22.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2024-03-04T07:38:22.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2024-03-04T07:38:22.090 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2024-03-04T07:38:22.091 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2024-03-04T07:38:22.092 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2024-03-04T07:38:22.093 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2024-03-04T07:38:22.094 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2024-03-04T07:38:22.094 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2024-03-04T07:38:22.094 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2024-03-04T07:38:22.094 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2024-03-04T07:38:22.094 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2024-03-04T07:38:22.094 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2024-03-04T07:38:22.094 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2024-03-04T07:38:22.094 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2024-03-04T07:38:22.094 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2024-03-04T07:38:22.095 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2024-03-04T07:38:22.096 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2024-03-04T07:38:22.097 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2024-03-04T07:38:22.097 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2024-03-04T07:38:22.097 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2024-03-04T07:38:23.150 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2024-03-04T07:38:23.150 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2024-03-04T07:38:23.150 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2024-03-04T07:38:23.151 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2024-03-04T07:38:23.152 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2024-03-04T07:38:23.153 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2024-03-04T07:38:23.154 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2024-03-04T07:38:23.155 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2024-03-04T07:38:23.156 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2024-03-04T07:38:23.156 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2024-03-04T07:38:23.156 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2024-03-04T07:38:23.156 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2024-03-04T07:38:23.156 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2024-03-04T07:38:23.156 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2024-03-04T07:38:23.156 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2024-03-04T07:38:23.156 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2024-03-04T07:38:23.156 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2024-03-04T07:38:24.348 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2024-03-04T07:38:24.349 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2024-03-04T07:38:24.350 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2024-03-04T07:38:24.351 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2024-03-04T07:38:24.352 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2024-03-04T07:38:24.353 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2024-03-04T07:38:24.353 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2024-03-04T07:38:24.353 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2024-03-04T07:38:24.353 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2024-03-04T07:38:24.353 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2024-03-04T07:38:24.353 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2024-03-04T07:38:24.353 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2024-03-04T07:38:24.353 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2024-03-04T07:38:25.448 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:38:25.453 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2024-03-04T07:38:26.133 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2024-03-04T07:38:26.134 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2024-03-04T07:38:26.135 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2024-03-04T07:38:26.136 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2024-03-04T07:38:26.137 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2024-03-04T07:38:26.138 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2024-03-04T07:38:29.012 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2024-03-04T07:38:29.012 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2024-03-04T07:38:29.013 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2024-03-04T07:38:29.014 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2024-03-04T07:38:29.015 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2024-03-04T07:38:29.016 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2024-03-04T07:38:29.017 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2024-03-04T07:38:29.018 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2024-03-04T07:38:29.018 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2024-03-04T07:38:29.018 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2024-03-04T07:38:29.018 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2024-03-04T07:38:29.018 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2024-03-04T07:38:29.018 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2024-03-04T07:38:30.221 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2024-03-04T07:38:30.222 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2024-03-04T07:38:30.223 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2024-03-04T07:38:30.224 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2024-03-04T07:38:30.225 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2024-03-04T07:38:30.226 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2024-03-04T07:38:30.227 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2024-03-04T07:38:32.181 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2024-03-04T07:38:32.181 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2024-03-04T07:38:32.181 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2024-03-04T07:38:32.181 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2024-03-04T07:38:32.181 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2024-03-04T07:38:32.181 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2024-03-04T07:38:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2024-03-04T07:38:32.183 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2024-03-04T07:38:32.184 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2024-03-04T07:38:32.185 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2024-03-04T07:38:32.186 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2024-03-04T07:38:32.187 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2024-03-04T07:38:32.187 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2024-03-04T07:38:32.187 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2024-03-04T07:38:32.187 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2024-03-04T07:38:32.187 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2024-03-04T07:38:32.187 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2024-03-04T07:38:32.187 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2024-03-04T07:38:32.187 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2024-03-04T07:38:32.187 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2024-03-04T07:38:34.033 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:34.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc12fc00640 1 Processor -- start 2024-03-04T07:38:34.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc12fc00640 1 -- start start 2024-03-04T07:38:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc12fc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc128070120 0x7fc128158930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc12fc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1280c5380 con 0x7fc128152d60 2024-03-04T07:38:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc12fc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1280c3440 con 0x7fc1280c9db0 2024-03-04T07:38:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc12fc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12806f620 con 0x7fc128070120 2024-03-04T07:38:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc127e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc128070120 0x7fc128158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc127e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc128070120 0x7fc128158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46234/0 (socket says 172.21.15.113:46234) 2024-03-04T07:38:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc127e00640 1 -- 172.21.15.113:0/1009548362 learned_addr learned my addr 172.21.15.113:0/1009548362 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc12d200640 1 --1- 172.21.15.113:0/1009548362 >> v1:172.21.15.113:6789/0 conn(0x7fc128152d60 0x7fc1280ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51904/0 (socket says 172.21.15.113:51904) 2024-03-04T07:38:34.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc127e00640 1 -- 172.21.15.113:0/1009548362 >> v1:172.21.15.136:6789/0 conn(0x7fc1280c9db0 legacy=0x7fc1281632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc127e00640 1 -- 172.21.15.113:0/1009548362 >> v1:172.21.15.113:6789/0 conn(0x7fc128152d60 legacy=0x7fc1280ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.169+0000 7fc127e00640 1 -- 172.21.15.113:0/1009548362 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1280c36e0 con 0x7fc128070120 2024-03-04T07:38:34.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc127e00640 1 --2- 172.21.15.113:0/1009548362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc128070120 0x7fc128158930 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc114009870 tx=0x7fc11401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c079a0ec60849e23 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:34.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc127400640 1 -- 172.21.15.113:0/1009548362 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140a7070 con 0x7fc128070120 2024-03-04T07:38:34.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc127400640 1 -- 172.21.15.113:0/1009548362 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1140b0070 con 0x7fc128070120 2024-03-04T07:38:34.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc127400640 1 -- 172.21.15.113:0/1009548362 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140ab890 con 0x7fc128070120 2024-03-04T07:38:34.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc12fc00640 1 -- 172.21.15.113:0/1009548362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc128070120 msgr2=0x7fc128158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc12fc00640 1 --2- 172.21.15.113:0/1009548362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc128070120 0x7fc128158930 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc114009870 tx=0x7fc11401c560 comp rx=0 tx=0).stop 2024-03-04T07:38:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc12fc00640 1 -- 172.21.15.113:0/1009548362 shutdown_connections 2024-03-04T07:38:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc12fc00640 1 --2- 172.21.15.113:0/1009548362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc128070120 0x7fc128158930 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc12fc00640 1 -- 172.21.15.113:0/1009548362 >> 172.21.15.113:0/1009548362 conn(0x7fc128066db0 msgr2=0x7fc1280671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc12fc00640 1 -- 172.21.15.113:0/1009548362 shutdown_connections 2024-03-04T07:38:34.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc12fc00640 1 -- 172.21.15.113:0/1009548362 wait complete. 2024-03-04T07:38:34.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc12fc00640 1 Processor -- start 2024-03-04T07:38:34.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.177+0000 7fc12fc00640 1 -- start start 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12fc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc128070120 0x7fc1281510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12fc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1280c9db0 0x7fc128159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12fc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc128152d60 0x7fc12806d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12fc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280cea20 con 0x7fc128152d60 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12fc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc128159280 con 0x7fc1280c9db0 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12fc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280ceba0 con 0x7fc128070120 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc128152d60 0x7fc12806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc127e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1280c9db0 0x7fc128159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc128070120 0x7fc1281510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc127e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1280c9db0 0x7fc128159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48512/0 (socket says 172.21.15.113:48512) 2024-03-04T07:38:34.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc127e00640 1 -- 172.21.15.113:0/1636390117 learned_addr learned my addr 172.21.15.113:0/1636390117 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12dc00640 1 -- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc128070120 msgr2=0x7fc1281510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12dc00640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc128070120 0x7fc1281510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12dc00640 1 -- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1280c9db0 msgr2=0x7fc128159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12dc00640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1280c9db0 0x7fc128159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12dc00640 1 -- 172.21.15.113:0/1636390117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12806ca70 con 0x7fc128152d60 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12dc00640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc128152d60 0x7fc12806d230 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc12000c700 tx=0x7fc12000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc124c00640 1 -- 172.21.15.113:0/1636390117 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120003b80 con 0x7fc128152d60 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc124c00640 1 -- 172.21.15.113:0/1636390117 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc120003d20 con 0x7fc128152d60 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12fc00640 1 -- 172.21.15.113:0/1636390117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12815c2f0 con 0x7fc128152d60 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc124c00640 1 -- 172.21.15.113:0/1636390117 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120005310 con 0x7fc128152d60 2024-03-04T07:38:34.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12fc00640 1 -- 172.21.15.113:0/1636390117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12815c8b0 con 0x7fc128152d60 2024-03-04T07:38:34.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.181+0000 7fc12fc00640 1 -- 172.21.15.113:0/1636390117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12815caf0 con 0x7fc128152d60 2024-03-04T07:38:34.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.185+0000 7fc124c00640 1 -- 172.21.15.113:0/1636390117 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1200054b0 con 0x7fc128152d60 2024-03-04T07:38:34.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.185+0000 7fc124c00640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc108066f60 0x7fc108069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.185+0000 7fc124c00640 1 -- 172.21.15.113:0/1636390117 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fc120084e30 con 0x7fc128152d60 2024-03-04T07:38:34.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.185+0000 7fc12d200640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc108066f60 0x7fc108069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.185+0000 7fc12d200640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc108066f60 0x7fc108069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc11c000f70 tx=0x7fc11c007e50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:34.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.189+0000 7fc124c00640 1 -- 172.21.15.113:0/1636390117 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc120055640 con 0x7fc128152d60 2024-03-04T07:38:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.389+0000 7fc12fc00640 1 -- 172.21.15.113:0/1636390117 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1280c8a60 con 0x7fc128152d60 2024-03-04T07:38:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.389+0000 7fc124c00640 1 -- 172.21.15.113:0/1636390117 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fc1200596e0 con 0x7fc128152d60 2024-03-04T07:38:34.394 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:34.394 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276,4256]},"id":1}]} 2024-03-04T07:38:34.394 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 -- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc108066f60 msgr2=0x7fc108069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc108066f60 0x7fc108069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc11c000f70 tx=0x7fc11c007e50 comp rx=0 tx=0).stop 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 -- 172.21.15.113:0/1636390117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc128152d60 msgr2=0x7fc12806d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc128152d60 0x7fc12806d230 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc12000c700 tx=0x7fc12000cbc0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 -- 172.21.15.113:0/1636390117 shutdown_connections 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc108066f60 0x7fc108069420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc128152d60 0x7fc12806d230 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1280c9db0 0x7fc128159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 --2- 172.21.15.113:0/1636390117 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc128070120 0x7fc1281510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 -- 172.21.15.113:0/1636390117 >> 172.21.15.113:0/1636390117 conn(0x7fc128066db0 msgr2=0x7fc1280ce310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 -- 172.21.15.113:0/1636390117 shutdown_connections 2024-03-04T07:38:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.393+0000 7fc12fc00640 1 -- 172.21.15.113:0/1636390117 wait complete. 2024-03-04T07:38:34.414 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=1) 2024-03-04T07:38:34.415 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-03-04T07:38:34.415 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:38:34.430 INFO:tasks.ceph.mds.e:Stopped 2024-03-04T07:38:34.430 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:34.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.605+0000 7fcf43e00640 1 Processor -- start 2024-03-04T07:38:34.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.605+0000 7fcf43e00640 1 -- start start 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 0x7fcf3c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf43e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcf3c152bd0 0x7fcf3c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf43e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3c1591f0 con 0x7fcf3c1527f0 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf43e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3c071df0 con 0x7fcf3c152bd0 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf43e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c1597b0 con 0x7fcf3c0c8200 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf41400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcf3c1527f0 0x7fcf3c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51914/0 (socket says 172.21.15.113:51914) 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf41400640 1 -- 172.21.15.113:0/3357979252 learned_addr learned my addr 172.21.15.113:0/3357979252 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf40a00640 1 --2- 172.21.15.113:0/3357979252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 0x7fcf3c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf41e00640 1 --2- 172.21.15.113:0/3357979252 >> v2:172.21.15.136:3300/0 conn(0x7fcf3c152bd0 0x7fcf3c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf3be00640 1 -- 172.21.15.113:0/3357979252 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919539252 0 0) 0x7fcf3c1591f0 con 0x7fcf3c1527f0 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf3be00640 1 -- 172.21.15.113:0/3357979252 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf24003530 con 0x7fcf3c1527f0 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf40a00640 1 -- 172.21.15.113:0/3357979252 >> v2:172.21.15.136:3300/0 conn(0x7fcf3c152bd0 msgr2=0x7fcf3c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:34.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf40a00640 1 --2- 172.21.15.113:0/3357979252 >> v2:172.21.15.136:3300/0 conn(0x7fcf3c152bd0 0x7fcf3c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf40a00640 1 -- 172.21.15.113:0/3357979252 >> v1:172.21.15.113:6789/0 conn(0x7fcf3c1527f0 legacy=0x7fcf3c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.609+0000 7fcf40a00640 1 -- 172.21.15.113:0/3357979252 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c1594d0 con 0x7fcf3c0c8200 2024-03-04T07:38:34.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf40a00640 1 --2- 172.21.15.113:0/3357979252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 0x7fcf3c070d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097c0 tx=0x7fcf2801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5f6cf3b2597c09 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:34.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf3be00640 1 -- 172.21.15.113:0/3357979252 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280b1070 con 0x7fcf3c0c8200 2024-03-04T07:38:34.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf3be00640 1 -- 172.21.15.113:0/3357979252 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf280ac410 con 0x7fcf3c0c8200 2024-03-04T07:38:34.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf3be00640 1 -- 172.21.15.113:0/3357979252 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280ac7c0 con 0x7fcf3c0c8200 2024-03-04T07:38:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 -- 172.21.15.113:0/3357979252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 msgr2=0x7fcf3c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 --2- 172.21.15.113:0/3357979252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 0x7fcf3c070d40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097c0 tx=0x7fcf2801c4a0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 -- 172.21.15.113:0/3357979252 shutdown_connections 2024-03-04T07:38:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 --2- 172.21.15.113:0/3357979252 >> v2:172.21.15.136:3300/0 conn(0x7fcf3c152bd0 0x7fcf3c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 --2- 172.21.15.113:0/3357979252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 0x7fcf3c070d40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 -- 172.21.15.113:0/3357979252 >> 172.21.15.113:0/3357979252 conn(0x7fcf3c06e820 msgr2=0x7fcf3c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 -- 172.21.15.113:0/3357979252 shutdown_connections 2024-03-04T07:38:34.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 -- 172.21.15.113:0/3357979252 wait complete. 2024-03-04T07:38:34.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 Processor -- start 2024-03-04T07:38:34.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 -- start start 2024-03-04T07:38:34.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 0x7fcf3c0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf3c1527f0 0x7fcf3c157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf3c152bd0 0x7fcf3c149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c1654f0 con 0x7fcf3c152bd0 2024-03-04T07:38:34.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c165370 con 0x7fcf3c1527f0 2024-03-04T07:38:34.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.613+0000 7fcf43e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c165670 con 0x7fcf3c0c8200 2024-03-04T07:38:34.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf41e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf3c152bd0 0x7fcf3c149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf41e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf3c152bd0 0x7fcf3c149d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46250/0 (socket says 172.21.15.113:46250) 2024-03-04T07:38:34.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf41e00640 1 -- 172.21.15.113:0/2931059015 learned_addr learned my addr 172.21.15.113:0/2931059015 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:34.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf41400640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 0x7fcf3c0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf40a00640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf3c1527f0 0x7fcf3c157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf41e00640 1 -- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 msgr2=0x7fcf3c0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf41e00640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 0x7fcf3c0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf41e00640 1 -- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf3c1527f0 msgr2=0x7fcf3c157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf41e00640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf3c1527f0 0x7fcf3c157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf41e00640 1 -- 172.21.15.113:0/2931059015 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c1f4650 con 0x7fcf3c152bd0 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf41e00640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf3c152bd0 0x7fcf3c149d40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009d30 tx=0x7fcf3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf39600640 1 -- 172.21.15.113:0/2931059015 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34018020 con 0x7fcf3c152bd0 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf39600640 1 -- 172.21.15.113:0/2931059015 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf34014480 con 0x7fcf3c152bd0 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf43e00640 1 -- 172.21.15.113:0/2931059015 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c0b1c90 con 0x7fcf3c152bd0 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf39600640 1 -- 172.21.15.113:0/2931059015 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34014870 con 0x7fcf3c152bd0 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf43e00640 1 -- 172.21.15.113:0/2931059015 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c0af8f0 con 0x7fcf3c152bd0 2024-03-04T07:38:34.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.617+0000 7fcf43e00640 1 -- 172.21.15.113:0/2931059015 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c075c00 con 0x7fcf3c152bd0 2024-03-04T07:38:34.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.621+0000 7fcf39600640 1 -- 172.21.15.113:0/2931059015 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf3401b020 con 0x7fcf3c152bd0 2024-03-04T07:38:34.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.621+0000 7fcf39600640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf1c066f90 0x7fcf1c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:34.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.621+0000 7fcf39600640 1 -- 172.21.15.113:0/2931059015 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fcf34085d50 con 0x7fcf3c152bd0 2024-03-04T07:38:34.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.621+0000 7fcf41400640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf1c066f90 0x7fcf1c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:34.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.621+0000 7fcf41400640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf1c066f90 0x7fcf1c069450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcf30007fd0 tx=0x7fcf30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:34.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.625+0000 7fcf39600640 1 -- 172.21.15.113:0/2931059015 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcf34089030 con 0x7fcf3c152bd0 2024-03-04T07:38:34.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.813+0000 7fcf43e00640 1 -- 172.21.15.113:0/2931059015 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf3c0c1ae0 con 0x7fcf3c152bd0 2024-03-04T07:38:34.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.813+0000 7fcf39600640 1 -- 172.21.15.113:0/2931059015 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fcf34022030 con 0x7fcf3c152bd0 2024-03-04T07:38:34.818 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:34.818 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276,4256]},"id":1}]} 2024-03-04T07:38:34.818 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 -- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf1c066f90 msgr2=0x7fcf1c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf1c066f90 0x7fcf1c069450 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcf30007fd0 tx=0x7fcf30009040 comp rx=0 tx=0).stop 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 -- 172.21.15.113:0/2931059015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf3c152bd0 msgr2=0x7fcf3c149d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf3c152bd0 0x7fcf3c149d40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009d30 tx=0x7fcf3400c490 comp rx=0 tx=0).stop 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 -- 172.21.15.113:0/2931059015 shutdown_connections 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf1c066f90 0x7fcf1c069450 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf3c152bd0 0x7fcf3c149d40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf3c1527f0 0x7fcf3c157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 --2- 172.21.15.113:0/2931059015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf3c0c8200 0x7fcf3c0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 -- 172.21.15.113:0/2931059015 >> 172.21.15.113:0/2931059015 conn(0x7fcf3c06e820 msgr2=0x7fcf3c06ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 -- 172.21.15.113:0/2931059015 shutdown_connections 2024-03-04T07:38:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:34.817+0000 7fcf43e00640 1 -- 172.21.15.113:0/2931059015 wait complete. 2024-03-04T07:38:34.839 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:38:36.840 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f0883c00640 1 Processor -- start 2024-03-04T07:38:36.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f0883c00640 1 -- start start 2024-03-04T07:38:36.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f0883c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f087c155100 0x7f087c0c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:36.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f0883c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f087c06b060 0x7f087c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:36.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f0883c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c15efd0 con 0x7f087c0cd600 2024-03-04T07:38:36.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f0883c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c0c8890 con 0x7f087c06b060 2024-03-04T07:38:36.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f0883c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c06b4a0 con 0x7f087c155100 2024-03-04T07:38:36.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f087be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f087c06b060 0x7f087c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:36.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f087be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f087c06b060 0x7f087c0cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46254/0 (socket says 172.21.15.113:46254) 2024-03-04T07:38:36.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f087be00640 1 -- 172.21.15.113:0/3962996357 learned_addr learned my addr 172.21.15.113:0/3962996357 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:36.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f0881200640 1 --2- 172.21.15.113:0/3962996357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f087c155100 0x7f087c0c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:36.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f087be00640 1 -- 172.21.15.113:0/3962996357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f087c155100 msgr2=0x7f087c0c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:36.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f087be00640 1 --2- 172.21.15.113:0/3962996357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f087c155100 0x7f087c0c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:36.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f087be00640 1 -- 172.21.15.113:0/3962996357 >> v1:172.21.15.136:6789/0 conn(0x7f087c0cd600 legacy=0x7f087c15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:36.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.989+0000 7f087be00640 1 -- 172.21.15.113:0/3962996357 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c0c7ef0 con 0x7f087c06b060 2024-03-04T07:38:36.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f087be00640 1 --2- 172.21.15.113:0/3962996357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 0x7f087c0cd0c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0868009870 tx=0x7f086801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2618845f8f100fe server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:36.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f087b400640 1 -- 172.21.15.113:0/3962996357 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680b2070 con 0x7f087c06b060 2024-03-04T07:38:36.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f087b400640 1 -- 172.21.15.113:0/3962996357 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08680a9070 con 0x7f087c06b060 2024-03-04T07:38:36.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f087b400640 1 -- 172.21.15.113:0/3962996357 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680ad880 con 0x7f087c06b060 2024-03-04T07:38:36.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f0883c00640 1 -- 172.21.15.113:0/3962996357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 msgr2=0x7f087c0cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:36.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f0883c00640 1 --2- 172.21.15.113:0/3962996357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 0x7f087c0cd0c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0868009870 tx=0x7f086801c560 comp rx=0 tx=0).stop 2024-03-04T07:38:36.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f0883c00640 1 -- 172.21.15.113:0/3962996357 shutdown_connections 2024-03-04T07:38:36.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f0883c00640 1 --2- 172.21.15.113:0/3962996357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 0x7f087c0cd0c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:36.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f0883c00640 1 --2- 172.21.15.113:0/3962996357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f087c155100 0x7f087c0c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:36.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f0883c00640 1 -- 172.21.15.113:0/3962996357 >> 172.21.15.113:0/3962996357 conn(0x7f087c06deb0 msgr2=0x7f087c06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:36.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f0883c00640 1 -- 172.21.15.113:0/3962996357 shutdown_connections 2024-03-04T07:38:36.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.993+0000 7f0883c00640 1 -- 172.21.15.113:0/3962996357 wait complete. 2024-03-04T07:38:36.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0883c00640 1 Processor -- start 2024-03-04T07:38:36.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0883c00640 1 -- start start 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0883c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 0x7f087c15ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0883c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f087c0cd600 0x7f087c1f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0883c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f087c155100 0x7f087c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0883c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c15ed30 con 0x7f087c06b060 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0883c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c15ebb0 con 0x7f087c0cd600 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0883c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c167d30 con 0x7f087c155100 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0881200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 0x7f087c15ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0881200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 0x7f087c15ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46256/0 (socket says 172.21.15.113:46256) 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0881200640 1 -- 172.21.15.113:0/3982826460 learned_addr learned my addr 172.21.15.113:0/3982826460 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0881c00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f087c155100 0x7f087c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f087be00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f087c0cd600 0x7f087c1f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:37.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0881200640 1 -- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f087c155100 msgr2=0x7f087c15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:37.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0881200640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f087c155100 0x7f087c15bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:37.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0881200640 1 -- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f087c0cd600 msgr2=0x7f087c1f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:37.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0881200640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f087c0cd600 0x7f087c1f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:37.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0881200640 1 -- 172.21.15.113:0/3982826460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c1f8570 con 0x7f087c06b060 2024-03-04T07:38:37.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0881200640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 0x7f087c15ded0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f087000e870 tx=0x7f087000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:37.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0878c00640 1 -- 172.21.15.113:0/3982826460 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870017070 con 0x7f087c06b060 2024-03-04T07:38:37.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0878c00640 1 -- 172.21.15.113:0/3982826460 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000c940 con 0x7f087c06b060 2024-03-04T07:38:37.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0878c00640 1 -- 172.21.15.113:0/3982826460 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000ccf0 con 0x7f087c06b060 2024-03-04T07:38:37.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0883c00640 1 -- 172.21.15.113:0/3982826460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c1f4100 con 0x7f087c06b060 2024-03-04T07:38:37.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:36.997+0000 7f0883c00640 1 -- 172.21.15.113:0/3982826460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c1f46a0 con 0x7f087c06b060 2024-03-04T07:38:37.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.001+0000 7f0883c00640 1 -- 172.21.15.113:0/3982826460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c075c00 con 0x7f087c06b060 2024-03-04T07:38:37.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.001+0000 7f0878c00640 1 -- 172.21.15.113:0/3982826460 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f087001a070 con 0x7f087c06b060 2024-03-04T07:38:37.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.001+0000 7f0878c00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f085c066f60 0x7f085c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:37.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.001+0000 7f0878c00640 1 -- 172.21.15.113:0/3982826460 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f08700853d0 con 0x7f087c06b060 2024-03-04T07:38:37.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.001+0000 7f087be00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f085c066f60 0x7f085c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:37.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.001+0000 7f087be00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f085c066f60 0x7f085c069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f08680af000 tx=0x7f0868008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:37.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.005+0000 7f0878c00640 1 -- 172.21.15.113:0/3982826460 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0870055bd0 con 0x7f087c06b060 2024-03-04T07:38:37.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.193+0000 7f0883c00640 1 -- 172.21.15.113:0/3982826460 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f087c0c8a20 con 0x7f087c06b060 2024-03-04T07:38:37.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0878c00640 1 -- 172.21.15.113:0/3982826460 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f0870059c70 con 0x7f087c06b060 2024-03-04T07:38:37.199 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:37.199 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276,4256]},"id":1}]} 2024-03-04T07:38:37.200 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-03-04T07:38:37.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 -- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f085c066f60 msgr2=0x7f085c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:37.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f085c066f60 0x7f085c069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f08680af000 tx=0x7f0868008040 comp rx=0 tx=0).stop 2024-03-04T07:38:37.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 -- 172.21.15.113:0/3982826460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 msgr2=0x7f087c15ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:37.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 0x7f087c15ded0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f087000e870 tx=0x7f087000ec70 comp rx=0 tx=0).stop 2024-03-04T07:38:37.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 -- 172.21.15.113:0/3982826460 shutdown_connections 2024-03-04T07:38:37.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f087c155100 0x7f087c15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:37.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f087c0cd600 0x7f087c1f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:37.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f085c066f60 0x7f085c069420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:37.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 --2- 172.21.15.113:0/3982826460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f087c06b060 0x7f087c15ded0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:37.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 -- 172.21.15.113:0/3982826460 >> 172.21.15.113:0/3982826460 conn(0x7f087c06deb0 msgr2=0x7f087c15a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:37.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 -- 172.21.15.113:0/3982826460 shutdown_connections 2024-03-04T07:38:37.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:37.197+0000 7f0883c00640 1 -- 172.21.15.113:0/3982826460 wait complete. 2024-03-04T07:38:37.220 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:38:39.221 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.353+0000 7f2029200640 1 Processor -- start 2024-03-04T07:38:39.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.353+0000 7f2029200640 1 -- start start 2024-03-04T07:38:39.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2029200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2024152a50 0x7f20240ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:39.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2029200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f202406ff20 0x7f20240ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:39.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2029200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2024152e30 con 0x7f20240c8b90 2024-03-04T07:38:39.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2029200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20240c31b0 con 0x7f2024152a50 2024-03-04T07:38:39.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2029200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202406f840 con 0x7f202406ff20 2024-03-04T07:38:39.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2021e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2024152a50 0x7f20240ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:39.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2021400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f202406ff20 0x7f20240ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:39.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2021e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2024152a50 0x7f20240ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48802/0 (socket says 172.21.15.113:48802) 2024-03-04T07:38:39.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2021e00640 1 -- 172.21.15.113:0/1833407373 learned_addr learned my addr 172.21.15.113:0/1833407373 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:39.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2022800640 1 --1- 172.21.15.113:0/1833407373 >> v1:172.21.15.136:6789/0 conn(0x7f20240c8b90 0x7f202415bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:48056/0 (socket says 172.21.15.113:48056) 2024-03-04T07:38:39.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2021e00640 1 -- 172.21.15.113:0/1833407373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f202406ff20 msgr2=0x7f20240ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:39.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2021e00640 1 --2- 172.21.15.113:0/1833407373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f202406ff20 0x7f20240ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2021e00640 1 -- 172.21.15.113:0/1833407373 >> v1:172.21.15.136:6789/0 conn(0x7f20240c8b90 legacy=0x7f202415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:39.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.357+0000 7f2021e00640 1 -- 172.21.15.113:0/1833407373 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202406f280 con 0x7f2024152a50 2024-03-04T07:38:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2021e00640 1 --2- 172.21.15.113:0/1833407373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2024152a50 0x7f20240ce120 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2014009870 tx=0x7f201401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dd7ca2cad0d8e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2020a00640 1 -- 172.21.15.113:0/1833407373 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20140aa430 con 0x7f2024152a50 2024-03-04T07:38:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2020a00640 1 -- 172.21.15.113:0/1833407373 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20140af070 con 0x7f2024152a50 2024-03-04T07:38:39.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2020a00640 1 -- 172.21.15.113:0/1833407373 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20140aa7f0 con 0x7f2024152a50 2024-03-04T07:38:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2029200640 1 -- 172.21.15.113:0/1833407373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2024152a50 msgr2=0x7f20240ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2029200640 1 --2- 172.21.15.113:0/1833407373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2024152a50 0x7f20240ce120 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2014009870 tx=0x7f201401c4d0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2029200640 1 -- 172.21.15.113:0/1833407373 shutdown_connections 2024-03-04T07:38:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2029200640 1 --2- 172.21.15.113:0/1833407373 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f202406ff20 0x7f20240ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2029200640 1 --2- 172.21.15.113:0/1833407373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2024152a50 0x7f20240ce120 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2029200640 1 -- 172.21.15.113:0/1833407373 >> 172.21.15.113:0/1833407373 conn(0x7f20240734d0 msgr2=0x7f2024066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2029200640 1 -- 172.21.15.113:0/1833407373 shutdown_connections 2024-03-04T07:38:39.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2029200640 1 -- 172.21.15.113:0/1833407373 wait complete. 2024-03-04T07:38:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2029200640 1 Processor -- start 2024-03-04T07:38:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.361+0000 7f2029200640 1 -- start start 2024-03-04T07:38:39.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2029200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202406ff20 0x7f202406db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:39.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2029200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20240c8b90 0x7f20241f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2029200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2024152a50 0x7f202415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2029200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2024165a00 con 0x7f20240c8b90 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2029200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2024165880 con 0x7f202406ff20 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2029200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2024165b80 con 0x7f2024152a50 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2021400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20240c8b90 0x7f20241f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2021400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20240c8b90 0x7f20241f77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48810/0 (socket says 172.21.15.113:48810) 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2021400640 1 -- 172.21.15.113:0/2747069297 learned_addr learned my addr 172.21.15.113:0/2747069297 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2021e00640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202406ff20 0x7f202406db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2022800640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2024152a50 0x7f202415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2021400640 1 -- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2024152a50 msgr2=0x7f202415a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2021400640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2024152a50 0x7f202415a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2021400640 1 -- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202406ff20 msgr2=0x7f202406db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2021400640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202406ff20 0x7f202406db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2021400640 1 -- 172.21.15.113:0/2747069297 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20241f7cf0 con 0x7f20240c8b90 2024-03-04T07:38:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2022800640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2024152a50 0x7f202415a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:38:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2021400640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20240c8b90 0x7f20241f77b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f201000e8a0 tx=0x7f201000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f201a000640 1 -- 172.21.15.113:0/2747069297 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010017070 con 0x7f20240c8b90 2024-03-04T07:38:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f201a000640 1 -- 172.21.15.113:0/2747069297 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201000c940 con 0x7f20240c8b90 2024-03-04T07:38:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2029200640 1 -- 172.21.15.113:0/2747069297 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2024150060 con 0x7f20240c8b90 2024-03-04T07:38:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f201a000640 1 -- 172.21.15.113:0/2747069297 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201000ccf0 con 0x7f20240c8b90 2024-03-04T07:38:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2029200640 1 -- 172.21.15.113:0/2747069297 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2024150620 con 0x7f20240c8b90 2024-03-04T07:38:39.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f2029200640 1 -- 172.21.15.113:0/2747069297 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2024075c00 con 0x7f20240c8b90 2024-03-04T07:38:39.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.365+0000 7f201a000640 1 -- 172.21.15.113:0/2747069297 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f201001a070 con 0x7f20240c8b90 2024-03-04T07:38:39.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.369+0000 7f201a000640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2004066f60 0x7f2004069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:39.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.369+0000 7f201a000640 1 -- 172.21.15.113:0/2747069297 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f20100853a0 con 0x7f20240c8b90 2024-03-04T07:38:39.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.369+0000 7f2021e00640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2004066f60 0x7f2004069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:39.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.369+0000 7f2021e00640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2004066f60 0x7f2004069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f20140ae000 tx=0x7f20140a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:39.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.373+0000 7f201a000640 1 -- 172.21.15.113:0/2747069297 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f20241f7cf0 con 0x7f20240c8b90 2024-03-04T07:38:39.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.569+0000 7f2029200640 1 -- 172.21.15.113:0/2747069297 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20240cc310 con 0x7f20240c8b90 2024-03-04T07:38:39.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.569+0000 7f201a000640 1 -- 172.21.15.113:0/2747069297 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f2010055ba0 con 0x7f20240c8b90 2024-03-04T07:38:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:39.573 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276,4256]},"id":1}]} 2024-03-04T07:38:39.573 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-03-04T07:38:39.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 -- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2004066f60 msgr2=0x7f2004069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2004066f60 0x7f2004069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f20140ae000 tx=0x7f20140a7000 comp rx=0 tx=0).stop 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 -- 172.21.15.113:0/2747069297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20240c8b90 msgr2=0x7f20241f77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20240c8b90 0x7f20241f77b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f201000e8a0 tx=0x7f201000eca0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 -- 172.21.15.113:0/2747069297 shutdown_connections 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2024152a50 0x7f202415a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2004066f60 0x7f2004069420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20240c8b90 0x7f20241f77b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 --2- 172.21.15.113:0/2747069297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f202406ff20 0x7f202406db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 -- 172.21.15.113:0/2747069297 >> 172.21.15.113:0/2747069297 conn(0x7f20240734d0 msgr2=0x7f20240ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 -- 172.21.15.113:0/2747069297 shutdown_connections 2024-03-04T07:38:39.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:39.573+0000 7f2029200640 1 -- 172.21.15.113:0/2747069297 wait complete. 2024-03-04T07:38:39.594 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:38:41.595 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:41.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f9752a00640 1 Processor -- start 2024-03-04T07:38:41.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f9752a00640 1 -- start start 2024-03-04T07:38:41.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f9752a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f974c154fe0 0x7f974c0c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:41.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f9752a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974c0c7d80 con 0x7f974c0c7700 2024-03-04T07:38:41.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f9752a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974c0c8060 con 0x7f974c154c00 2024-03-04T07:38:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f9752a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974c15efb0 con 0x7f974c154fe0 2024-03-04T07:38:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f9750a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f974c0c7700 0x7f974c15abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60556/0 (socket says 172.21.15.113:60556) 2024-03-04T07:38:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f9750a00640 1 -- 172.21.15.113:0/588024719 learned_addr learned my addr 172.21.15.113:0/588024719 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:38:41.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974b400640 1 --2- 172.21.15.113:0/588024719 >> v2:172.21.15.136:3300/0 conn(0x7f974c154fe0 0x7f974c0c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:41.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107038850 0 0) 0x7f974c0c7d80 con 0x7f974c0c7700 2024-03-04T07:38:41.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9730003530 con 0x7f974c0c7700 2024-03-04T07:38:41.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 749048437 0 0) 0x7f9730003530 con 0x7f974c0c7700 2024-03-04T07:38:41.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 >> v2:172.21.15.136:3300/0 conn(0x7f974c154fe0 msgr2=0x7f974c0c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:41.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 --2- 172.21.15.113:0/588024719 >> v2:172.21.15.136:3300/0 conn(0x7f974c154fe0 0x7f974c0c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 >> v1:172.21.15.136:6789/0 conn(0x7f974c154c00 legacy=0x7f974c0c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:41.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c0c6dc0 con 0x7f974c0c7700 2024-03-04T07:38:41.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f9744001a00 con 0x7f974c0c7700 2024-03-04T07:38:41.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 >> v1:172.21.15.113:6789/0 conn(0x7f974c0c7700 legacy=0x7f974c15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97300039c0 0x7f9730003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9730004810 0x7f9730004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97300051d0 0x7f973000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c15efb0 con 0x7f97300039c0 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973000c170 con 0x7f9730004810 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973000c370 con 0x7f97300051d0 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974be00640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97300039c0 0x7f9730003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974b400640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9730004810 0x7f9730004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f9750a00640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97300051d0 0x7f973000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974be00640 1 -- 172.21.15.113:0/588024719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97300051d0 msgr2=0x7f973000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974be00640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97300051d0 0x7f973000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974be00640 1 -- 172.21.15.113:0/588024719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9730004810 msgr2=0x7f9730004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974be00640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9730004810 0x7f9730004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f974be00640 1 -- 172.21.15.113:0/588024719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730003530 con 0x7f97300039c0 2024-03-04T07:38:41.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.729+0000 7f9750a00640 1 -- 172.21.15.113:0/588024719 reap_dead start 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f974be00640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97300039c0 0x7f9730003de0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f973c001780 tx=0x7f973c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bcb5f781adafe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c0013a0 con 0x7f97300039c0 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973c0052f0 con 0x7f97300039c0 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f974aa00640 1 -- 172.21.15.113:0/588024719 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c003840 con 0x7f97300039c0 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f9752a00640 1 -- 172.21.15.113:0/588024719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97300039c0 msgr2=0x7f9730003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f9752a00640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97300039c0 0x7f9730003de0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f973c001780 tx=0x7f973c01c400 comp rx=0 tx=0).stop 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f9752a00640 1 -- 172.21.15.113:0/588024719 shutdown_connections 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f9752a00640 1 --2- 172.21.15.113:0/588024719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97300039c0 0x7f9730003de0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f9752a00640 1 -- 172.21.15.113:0/588024719 >> 172.21.15.113:0/588024719 conn(0x7f974c06e820 msgr2=0x7f974c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f9752a00640 1 -- 172.21.15.113:0/588024719 shutdown_connections 2024-03-04T07:38:41.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.733+0000 7f9752a00640 1 -- 172.21.15.113:0/588024719 wait complete. 2024-03-04T07:38:41.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.737+0000 7f9752a00640 1 Processor -- start 2024-03-04T07:38:41.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.737+0000 7f9752a00640 1 -- start start 2024-03-04T07:38:41.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.737+0000 7f9752a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97300039c0 0x7f974c155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.737+0000 7f9752a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f974c1e94b0 0x7f974c152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.737+0000 7f9752a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974c1e98f0 0x7f974c151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.737+0000 7f9752a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c15ad60 con 0x7f974c1e94b0 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.737+0000 7f9752a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c15abe0 con 0x7f974c1e98f0 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.737+0000 7f9752a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c15aee0 con 0x7f97300039c0 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.737+0000 7f974b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f974c1e94b0 0x7f974c152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f974b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f974c1e94b0 0x7f974c152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48834/0 (socket says 172.21.15.113:48834) 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f974b400640 1 -- 172.21.15.113:0/1556940562 learned_addr learned my addr 172.21.15.113:0/1556940562 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f974be00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97300039c0 0x7f974c155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f9750a00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974c1e98f0 0x7f974c151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f974b400640 1 -- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97300039c0 msgr2=0x7f974c155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:41.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f974b400640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97300039c0 0x7f974c155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f974b400640 1 -- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974c1e98f0 msgr2=0x7f974c151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:41.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f974b400640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974c1e98f0 0x7f974c151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f974b400640 1 -- 172.21.15.113:0/1556940562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c158440 con 0x7f974c1e94b0 2024-03-04T07:38:41.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f974b400640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f974c1e94b0 0x7f974c152c80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97380116c0 tx=0x7f9738011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:41.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f9743e00640 1 -- 172.21.15.113:0/1556940562 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973800f9c0 con 0x7f974c1e94b0 2024-03-04T07:38:41.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f9743e00640 1 -- 172.21.15.113:0/1556940562 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973800fb60 con 0x7f974c1e94b0 2024-03-04T07:38:41.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f9752a00640 1 -- 172.21.15.113:0/1556940562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c153560 con 0x7f974c1e94b0 2024-03-04T07:38:41.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f9743e00640 1 -- 172.21.15.113:0/1556940562 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738015460 con 0x7f974c1e94b0 2024-03-04T07:38:41.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f9752a00640 1 -- 172.21.15.113:0/1556940562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c1f4290 con 0x7f974c1e94b0 2024-03-04T07:38:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f9752a00640 1 -- 172.21.15.113:0/1556940562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c075c00 con 0x7f974c1e94b0 2024-03-04T07:38:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.741+0000 7f9743e00640 1 -- 172.21.15.113:0/1556940562 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9738015600 con 0x7f974c1e94b0 2024-03-04T07:38:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.745+0000 7f9743e00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f972c066f90 0x7f972c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:41.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.745+0000 7f9743e00640 1 -- 172.21.15.113:0/1556940562 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f97380859a0 con 0x7f974c1e94b0 2024-03-04T07:38:41.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.745+0000 7f974be00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f972c066f90 0x7f972c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:41.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.745+0000 7f974be00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f972c066f90 0x7f972c069450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f973c001750 tx=0x7f973c0011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:41.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.749+0000 7f9743e00640 1 -- 172.21.15.113:0/1556940562 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f974c158440 con 0x7f974c1e94b0 2024-03-04T07:38:41.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.937+0000 7f9752a00640 1 -- 172.21.15.113:0/1556940562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f974c154410 con 0x7f974c1e94b0 2024-03-04T07:38:41.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.937+0000 7f9743e00640 1 -- 172.21.15.113:0/1556940562 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f9738019030 con 0x7f974c1e94b0 2024-03-04T07:38:41.942 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:41.942 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276,4256]},"id":1}]} 2024-03-04T07:38:41.942 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-03-04T07:38:41.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 -- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f972c066f90 msgr2=0x7f972c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:41.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f972c066f90 0x7f972c069450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f973c001750 tx=0x7f973c0011a0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 -- 172.21.15.113:0/1556940562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f974c1e94b0 msgr2=0x7f974c152c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:41.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f974c1e94b0 0x7f974c152c80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97380116c0 tx=0x7f9738011b80 comp rx=0 tx=0).stop 2024-03-04T07:38:41.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 -- 172.21.15.113:0/1556940562 shutdown_connections 2024-03-04T07:38:41.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974c1e98f0 0x7f974c151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f972c066f90 0x7f972c069450 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f974c1e94b0 0x7f974c152c80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 --2- 172.21.15.113:0/1556940562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97300039c0 0x7f974c155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:41.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 -- 172.21.15.113:0/1556940562 >> 172.21.15.113:0/1556940562 conn(0x7f974c06e820 msgr2=0x7f974c15c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:41.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 -- 172.21.15.113:0/1556940562 shutdown_connections 2024-03-04T07:38:41.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:41.941+0000 7f9752a00640 1 -- 172.21.15.113:0/1556940562 wait complete. 2024-03-04T07:38:41.963 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:38:43.963 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:44.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.093+0000 7fb09aa00640 1 Processor -- start 2024-03-04T07:38:44.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.093+0000 7fb09aa00640 1 -- start start 2024-03-04T07:38:44.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb09aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb0940730d0 0x7fb09415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:44.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb09aa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09406f1b0 con 0x7fb09406d420 2024-03-04T07:38:44.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb09aa00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb094071580 con 0x7fb094152a90 2024-03-04T07:38:44.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb09aa00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb094152e70 con 0x7fb0940730d0 2024-03-04T07:38:44.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb098c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb09406d420 0x7fb09415bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60562/0 (socket says 172.21.15.113:60562) 2024-03-04T07:38:44.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb098c00640 1 -- 172.21.15.113:0/1796031407 learned_addr learned my addr 172.21.15.113:0/1796031407 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:38:44.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb09a000640 1 --2- 172.21.15.113:0/1796031407 >> v2:172.21.15.113:3300/0 conn(0x7fb0940730d0 0x7fb09415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:44.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb093e00640 1 -- 172.21.15.113:0/1796031407 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729229251 0 0) 0x7fb09406f1b0 con 0x7fb09406d420 2024-03-04T07:38:44.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb093e00640 1 -- 172.21.15.113:0/1796031407 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb07c003530 con 0x7fb09406d420 2024-03-04T07:38:44.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb09a000640 1 -- 172.21.15.113:0/1796031407 >> v1:172.21.15.136:6789/0 conn(0x7fb094152a90 legacy=0x7fb09406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:44.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb09a000640 1 -- 172.21.15.113:0/1796031407 >> v1:172.21.15.113:6789/0 conn(0x7fb09406d420 legacy=0x7fb09415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:44.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.097+0000 7fb09a000640 1 -- 172.21.15.113:0/1796031407 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb094071860 con 0x7fb0940730d0 2024-03-04T07:38:44.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb09a000640 1 --2- 172.21.15.113:0/1796031407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 0x7fb09415c3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009d90 tx=0x7fb08c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e493eb2dc7ce36 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:44.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb093e00640 1 -- 172.21.15.113:0/1796031407 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0b2070 con 0x7fb0940730d0 2024-03-04T07:38:44.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb093e00640 1 -- 172.21.15.113:0/1796031407 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c0a5830 con 0x7fb0940730d0 2024-03-04T07:38:44.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb093e00640 1 -- 172.21.15.113:0/1796031407 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0a5be0 con 0x7fb0940730d0 2024-03-04T07:38:44.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb09aa00640 1 -- 172.21.15.113:0/1796031407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 msgr2=0x7fb09415c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:44.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb09aa00640 1 --2- 172.21.15.113:0/1796031407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 0x7fb09415c3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009d90 tx=0x7fb08c01c970 comp rx=0 tx=0).stop 2024-03-04T07:38:44.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb09aa00640 1 -- 172.21.15.113:0/1796031407 shutdown_connections 2024-03-04T07:38:44.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb09aa00640 1 --2- 172.21.15.113:0/1796031407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 0x7fb09415c3f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:44.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb09aa00640 1 -- 172.21.15.113:0/1796031407 >> 172.21.15.113:0/1796031407 conn(0x7fb09406e2a0 msgr2=0x7fb09406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:44.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb09aa00640 1 -- 172.21.15.113:0/1796031407 shutdown_connections 2024-03-04T07:38:44.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb09aa00640 1 -- 172.21.15.113:0/1796031407 wait complete. 2024-03-04T07:38:44.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb09aa00640 1 Processor -- start 2024-03-04T07:38:44.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.101+0000 7fb09aa00640 1 -- start start 2024-03-04T07:38:44.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09aa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb09406d420 0x7fb09414bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:44.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 0x7fb0940afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:44.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb094152a90 0x7fb09414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:44.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09aa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb094071330 con 0x7fb0940730d0 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09aa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0941668a0 con 0x7fb094152a90 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09aa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb094166a20 con 0x7fb09406d420 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb098c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 0x7fb0940afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb098c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 0x7fb0940afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48850/0 (socket says 172.21.15.113:48850) 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb098c00640 1 -- 172.21.15.113:0/3488703211 learned_addr learned my addr 172.21.15.113:0/3488703211 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb099600640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb09406d420 0x7fb09414bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09a000640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb094152a90 0x7fb09414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb098c00640 1 -- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb09406d420 msgr2=0x7fb09414bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb098c00640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb09406d420 0x7fb09414bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb098c00640 1 -- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb094152a90 msgr2=0x7fb09414e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb098c00640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb094152a90 0x7fb09414e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb098c00640 1 -- 172.21.15.113:0/3488703211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0941f34b0 con 0x7fb0940730d0 2024-03-04T07:38:44.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09a000640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb094152a90 0x7fb09414e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:38:44.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb098c00640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 0x7fb0940afb90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c450 tx=0x7fb08000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:44.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb091600640 1 -- 172.21.15.113:0/3488703211 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080017070 con 0x7fb0940730d0 2024-03-04T07:38:44.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb091600640 1 -- 172.21.15.113:0/3488703211 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080005cf0 con 0x7fb0940730d0 2024-03-04T07:38:44.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb091600640 1 -- 172.21.15.113:0/3488703211 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080007270 con 0x7fb0940730d0 2024-03-04T07:38:44.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09aa00640 1 -- 172.21.15.113:0/3488703211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09414ce50 con 0x7fb0940730d0 2024-03-04T07:38:44.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09aa00640 1 -- 172.21.15.113:0/3488703211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09414d410 con 0x7fb0940730d0 2024-03-04T07:38:44.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.105+0000 7fb09aa00640 1 -- 172.21.15.113:0/3488703211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb094075a80 con 0x7fb0940730d0 2024-03-04T07:38:44.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.109+0000 7fb091600640 1 -- 172.21.15.113:0/3488703211 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb08001a070 con 0x7fb0940730d0 2024-03-04T07:38:44.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.109+0000 7fb091600640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:44.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.109+0000 7fb091600640 1 -- 172.21.15.113:0/3488703211 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fb080084810 con 0x7fb0940730d0 2024-03-04T07:38:44.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.109+0000 7fb099600640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:44.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.109+0000 7fb099600640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb074066f60 0x7fb074069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb088007fd0 tx=0x7fb088009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:44.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.113+0000 7fb091600640 1 -- 172.21.15.113:0/3488703211 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb080055110 con 0x7fb0940730d0 2024-03-04T07:38:44.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.321+0000 7fb09aa00640 1 -- 172.21.15.113:0/3488703211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0940c5f30 con 0x7fb0940730d0 2024-03-04T07:38:44.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.321+0000 7fb091600640 1 -- 172.21.15.113:0/3488703211 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fb0800591b0 con 0x7fb0940730d0 2024-03-04T07:38:44.325 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:44.325 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276,4256]},"id":1}]} 2024-03-04T07:38:44.325 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-03-04T07:38:44.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.321+0000 7fb09aa00640 1 -- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb074066f60 msgr2=0x7fb074069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:44.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb074066f60 0x7fb074069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb088007fd0 tx=0x7fb088009040 comp rx=0 tx=0).stop 2024-03-04T07:38:44.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 -- 172.21.15.113:0/3488703211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 msgr2=0x7fb0940afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:44.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 0x7fb0940afb90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c450 tx=0x7fb08000c910 comp rx=0 tx=0).stop 2024-03-04T07:38:44.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 -- 172.21.15.113:0/3488703211 shutdown_connections 2024-03-04T07:38:44.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb094152a90 0x7fb09414e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:44.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:44.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0940730d0 0x7fb0940afb90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:44.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 --2- 172.21.15.113:0/3488703211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb09406d420 0x7fb09414bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:44.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 -- 172.21.15.113:0/3488703211 >> 172.21.15.113:0/3488703211 conn(0x7fb09406e2a0 msgr2=0x7fb0940cb770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:44.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 -- 172.21.15.113:0/3488703211 shutdown_connections 2024-03-04T07:38:44.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:44.325+0000 7fb09aa00640 1 -- 172.21.15.113:0/3488703211 wait complete. 2024-03-04T07:38:44.350 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:38:46.350 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:46.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.481+0000 7f2188400640 1 Processor -- start 2024-03-04T07:38:46.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.481+0000 7f2188400640 1 -- start start 2024-03-04T07:38:46.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2188400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21800ccb40 0x7f21800c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:46.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2188400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f218015aa40 0x7f2180154800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:46.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2188400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f21800ccf20 0x7f2180154d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:46.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2188400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218015aed0 con 0x7f218015aa40 2024-03-04T07:38:46.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2188400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218015b170 con 0x7f21800ccf20 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2188400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218015b410 con 0x7f21800ccb40 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2185000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f218015aa40 0x7f2180154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2185000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f218015aa40 0x7f2180154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48854/0 (socket says 172.21.15.113:48854) 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2185000640 1 -- 172.21.15.113:0/569785157 learned_addr learned my addr 172.21.15.113:0/569785157 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2185a00640 1 --2- 172.21.15.113:0/569785157 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21800ccb40 0x7f21800c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2186400640 1 --2- 172.21.15.113:0/569785157 >> v2:172.21.15.136:3300/0 conn(0x7f21800ccf20 0x7f2180154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2185000640 1 -- 172.21.15.113:0/569785157 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21800ccb40 msgr2=0x7f21800c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2185000640 1 --2- 172.21.15.113:0/569785157 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21800ccb40 0x7f21800c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2185000640 1 -- 172.21.15.113:0/569785157 >> v2:172.21.15.136:3300/0 conn(0x7f21800ccf20 msgr2=0x7f2180154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2185000640 1 --2- 172.21.15.113:0/569785157 >> v2:172.21.15.136:3300/0 conn(0x7f21800ccf20 0x7f2180154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2185000640 1 -- 172.21.15.113:0/569785157 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218015b9b0 con 0x7f218015aa40 2024-03-04T07:38:46.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2186400640 1 --2- 172.21.15.113:0/569785157 >> v2:172.21.15.136:3300/0 conn(0x7f21800ccf20 0x7f2180154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:38:46.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.485+0000 7f2185000640 1 --2- 172.21.15.113:0/569785157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f218015aa40 0x7f2180154800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f21800c9320 tx=0x7f216c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f8a135a05faf59 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:46.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f217fe00640 1 -- 172.21.15.113:0/569785157 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c0b5070 con 0x7f218015aa40 2024-03-04T07:38:46.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f217fe00640 1 -- 172.21.15.113:0/569785157 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c0a89d0 con 0x7f218015aa40 2024-03-04T07:38:46.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f217fe00640 1 -- 172.21.15.113:0/569785157 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c0a8da0 con 0x7f218015aa40 2024-03-04T07:38:46.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 -- 172.21.15.113:0/569785157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f218015aa40 msgr2=0x7f2180154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:46.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 --2- 172.21.15.113:0/569785157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f218015aa40 0x7f2180154800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f21800c9320 tx=0x7f216c01ed10 comp rx=0 tx=0).stop 2024-03-04T07:38:46.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 -- 172.21.15.113:0/569785157 shutdown_connections 2024-03-04T07:38:46.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 --2- 172.21.15.113:0/569785157 >> v2:172.21.15.136:3300/0 conn(0x7f21800ccf20 0x7f2180154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 --2- 172.21.15.113:0/569785157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f218015aa40 0x7f2180154800 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 --2- 172.21.15.113:0/569785157 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21800ccb40 0x7f21800c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 -- 172.21.15.113:0/569785157 >> 172.21.15.113:0/569785157 conn(0x7f218006b060 msgr2=0x7f218006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:46.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 -- 172.21.15.113:0/569785157 shutdown_connections 2024-03-04T07:38:46.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 -- 172.21.15.113:0/569785157 wait complete. 2024-03-04T07:38:46.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 Processor -- start 2024-03-04T07:38:46.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 -- start start 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21800ccb40 0x7f2180151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21800ccf20 0x7f218014add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21800c69f0 0x7f21801f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800c78b0 con 0x7f21800ccf20 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218015e0d0 con 0x7f21800c69f0 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2188400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21800c9fd0 con 0x7f21800ccb40 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2185000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21800ccf20 0x7f218014add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2185000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21800ccf20 0x7f218014add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48866/0 (socket says 172.21.15.113:48866) 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2185000640 1 -- 172.21.15.113:0/3919746184 learned_addr learned my addr 172.21.15.113:0/3919746184 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2186400640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21800c69f0 0x7f21801f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.489+0000 7f2185a00640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21800ccb40 0x7f2180151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f2185000640 1 -- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21800ccb40 msgr2=0x7f2180151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f2185000640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21800ccb40 0x7f2180151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f2185000640 1 -- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21800c69f0 msgr2=0x7f21801f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:46.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f2185000640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21800c69f0 0x7f21801f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f2185000640 1 -- 172.21.15.113:0/3919746184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21801515f0 con 0x7f21800ccf20 2024-03-04T07:38:46.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f2185000640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21800ccf20 0x7f218014add0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f216c0b3040 tx=0x7f216c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:46.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f217d600640 1 -- 172.21.15.113:0/3919746184 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c0b5070 con 0x7f21800ccf20 2024-03-04T07:38:46.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f217d600640 1 -- 172.21.15.113:0/3919746184 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c0b0a30 con 0x7f21800ccf20 2024-03-04T07:38:46.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f217d600640 1 -- 172.21.15.113:0/3919746184 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c0b0e20 con 0x7f21800ccf20 2024-03-04T07:38:46.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f2188400640 1 -- 172.21.15.113:0/3919746184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218015a270 con 0x7f21800ccf20 2024-03-04T07:38:46.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f2188400640 1 -- 172.21.15.113:0/3919746184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21801f3970 con 0x7f21800ccf20 2024-03-04T07:38:46.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f2188400640 1 -- 172.21.15.113:0/3919746184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2180075c00 con 0x7f21800ccf20 2024-03-04T07:38:46.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f217d600640 1 -- 172.21.15.113:0/3919746184 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f216c0c2060 con 0x7f21800ccf20 2024-03-04T07:38:46.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f217d600640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2160066f90 0x7f2160069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:46.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f217d600640 1 -- 172.21.15.113:0/3919746184 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f216c1220d0 con 0x7f21800ccf20 2024-03-04T07:38:46.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.493+0000 7f2185a00640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2160066f90 0x7f2160069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:46.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.497+0000 7f2185a00640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2160066f90 0x7f2160069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2174009ae0 tx=0x7f2174009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:46.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.501+0000 7f217d600640 1 -- 172.21.15.113:0/3919746184 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f216c018870 con 0x7f21800ccf20 2024-03-04T07:38:46.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.709+0000 7f2188400640 1 -- 172.21.15.113:0/3919746184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21800730d0 con 0x7f21800ccf20 2024-03-04T07:38:46.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.709+0000 7f217d600640 1 -- 172.21.15.113:0/3919746184 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f216c0a89a0 con 0x7f21800ccf20 2024-03-04T07:38:46.713 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:46.713 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276,4256]},"id":1}]} 2024-03-04T07:38:46.713 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-03-04T07:38:46.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 -- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2160066f90 msgr2=0x7f2160069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:46.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2160066f90 0x7f2160069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2174009ae0 tx=0x7f2174009190 comp rx=0 tx=0).stop 2024-03-04T07:38:46.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 -- 172.21.15.113:0/3919746184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21800ccf20 msgr2=0x7f218014add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:46.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21800ccf20 0x7f218014add0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f216c0b3040 tx=0x7f216c01bab0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 -- 172.21.15.113:0/3919746184 shutdown_connections 2024-03-04T07:38:46.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21800c69f0 0x7f21801f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2160066f90 0x7f2160069450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21800ccf20 0x7f218014add0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 --2- 172.21.15.113:0/3919746184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21800ccb40 0x7f2180151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:46.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 -- 172.21.15.113:0/3919746184 >> 172.21.15.113:0/3919746184 conn(0x7f218006b060 msgr2=0x7f21800ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:46.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 -- 172.21.15.113:0/3919746184 shutdown_connections 2024-03-04T07:38:46.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:46.713+0000 7f2188400640 1 -- 172.21.15.113:0/3919746184 wait complete. 2024-03-04T07:38:46.733 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:38:48.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:48.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.865+0000 7fa0aa400640 1 Processor -- start 2024-03-04T07:38:48.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.865+0000 7fa0aa400640 1 -- start start 2024-03-04T07:38:48.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0aa400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa0a415c4e0 0x7fa0a4164070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:48.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0aa400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a4152e30 con 0x7fa0a4152a50 2024-03-04T07:38:48.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0aa400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a406f280 con 0x7fa0a406ff20 2024-03-04T07:38:48.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0aa400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a40c31b0 con 0x7fa0a415c4e0 2024-03-04T07:38:48.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa0a4152a50 0x7fa0a40ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53488/0 (socket says 172.21.15.113:53488) 2024-03-04T07:38:48.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3400640 1 -- 172.21.15.113:0/2337259479 learned_addr learned my addr 172.21.15.113:0/2337259479 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:38:48.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/2337259479 >> v2:172.21.15.136:3300/0 conn(0x7fa0a415c4e0 0x7fa0a4164070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487828536 0 0) 0x7fa0a4152e30 con 0x7fa0a4152a50 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa088003530 con 0x7fa0a4152a50 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 122426583 0 0) 0x7fa088003530 con 0x7fa0a4152a50 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 >> v2:172.21.15.136:3300/0 conn(0x7fa0a415c4e0 msgr2=0x7fa0a4164070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 --2- 172.21.15.113:0/2337259479 >> v2:172.21.15.136:3300/0 conn(0x7fa0a415c4e0 0x7fa0a4164070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 >> v1:172.21.15.136:6789/0 conn(0x7fa0a406ff20 legacy=0x7fa0a415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a406f560 con 0x7fa0a4152a50 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fa094001680 con 0x7fa0a4152a50 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 >> v1:172.21.15.113:6789/0 conn(0x7fa0a4152a50 legacy=0x7fa0a40ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa088003a10 0x7fa088003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa088004810 0x7fa088004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0880051d0 0x7fa08800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:48.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a40c31b0 con 0x7fa0880051d0 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08800c0d0 con 0x7fa088004810 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08800c2d0 con 0x7fa088003a10 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0880051d0 0x7fa08800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3400640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa088003a10 0x7fa088003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2a00640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa088004810 0x7fa088004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3e00640 1 -- 172.21.15.113:0/2337259479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa088003a10 msgr2=0x7fa088003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa088003a10 0x7fa088003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3e00640 1 -- 172.21.15.113:0/2337259479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa088004810 msgr2=0x7fa088004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa088004810 0x7fa088004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3e00640 1 -- 172.21.15.113:0/2337259479 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa088003530 con 0x7fa0880051d0 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a2a00640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa088004810 0x7fa088004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:38:48.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.869+0000 7fa0a3400640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa088003a10 0x7fa088003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0a3e00640 1 -- 172.21.15.113:0/2337259479 reap_dead start 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0880051d0 0x7fa08800b9d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009fc0 tx=0x7fa09c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce490d8d031075d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c009c10 con 0x7fa0880051d0 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c0a5cd0 con 0x7fa0880051d0 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0a2000640 1 -- 172.21.15.113:0/2337259479 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c003be0 con 0x7fa0880051d0 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0aa400640 1 -- 172.21.15.113:0/2337259479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0880051d0 msgr2=0x7fa08800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0aa400640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0880051d0 0x7fa08800b9d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009fc0 tx=0x7fa09c01ccc0 comp rx=0 tx=0).stop 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0aa400640 1 -- 172.21.15.113:0/2337259479 shutdown_connections 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0aa400640 1 --2- 172.21.15.113:0/2337259479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0880051d0 0x7fa08800b9d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0aa400640 1 -- 172.21.15.113:0/2337259479 >> 172.21.15.113:0/2337259479 conn(0x7fa0a40734d0 msgr2=0x7fa0a4066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0aa400640 1 -- 172.21.15.113:0/2337259479 shutdown_connections 2024-03-04T07:38:48.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0aa400640 1 -- 172.21.15.113:0/2337259479 wait complete. 2024-03-04T07:38:48.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.873+0000 7fa0aa400640 1 Processor -- start 2024-03-04T07:38:48.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0aa400640 1 -- start start 2024-03-04T07:38:48.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0aa400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0880051d0 0x7fa0a40c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:48.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0aa400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a406c170 0x7fa0a415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:48.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0aa400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a41f0390 0x7fa0a41f0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:48.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0aa400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a40c3330 con 0x7fa0a41f0390 2024-03-04T07:38:48.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0aa400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a406f840 con 0x7fa0880051d0 2024-03-04T07:38:48.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0aa400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a40c34b0 con 0x7fa0a406c170 2024-03-04T07:38:48.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a41f0390 0x7fa0a41f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:48.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a41f0390 0x7fa0a41f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47504/0 (socket says 172.21.15.113:47504) 2024-03-04T07:38:48.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a3e00640 1 -- 172.21.15.113:0/894157498 learned_addr learned my addr 172.21.15.113:0/894157498 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:48.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a3400640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0880051d0 0x7fa0a40c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a2a00640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a406c170 0x7fa0a415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a3e00640 1 -- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a406c170 msgr2=0x7fa0a415aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a406c170 0x7fa0a415aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a3e00640 1 -- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0880051d0 msgr2=0x7fa0a40c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0880051d0 0x7fa0a40c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a3e00640 1 -- 172.21.15.113:0/894157498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a40c5630 con 0x7fa0a41f0390 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a41f0390 0x7fa0a41f0810 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0987f0 tx=0x7fa09c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa09b400640 1 -- 172.21.15.113:0/894157498 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c0aed70 con 0x7fa0a41f0390 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa09b400640 1 -- 172.21.15.113:0/894157498 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c0bd070 con 0x7fa0a41f0390 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0aa400640 1 -- 172.21.15.113:0/894157498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a40c80f0 con 0x7fa0a41f0390 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa09b400640 1 -- 172.21.15.113:0/894157498 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c003ba0 con 0x7fa0a41f0390 2024-03-04T07:38:48.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0aa400640 1 -- 172.21.15.113:0/894157498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a40c5bb0 con 0x7fa0a41f0390 2024-03-04T07:38:48.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.877+0000 7fa0aa400640 1 -- 172.21.15.113:0/894157498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a4075c00 con 0x7fa0a41f0390 2024-03-04T07:38:48.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.881+0000 7fa09b400640 1 -- 172.21.15.113:0/894157498 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa09c004030 con 0x7fa0a41f0390 2024-03-04T07:38:48.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.881+0000 7fa09b400640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa084066f90 0x7fa084069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:48.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.881+0000 7fa09b400640 1 -- 172.21.15.113:0/894157498 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fa09c120f40 con 0x7fa0a41f0390 2024-03-04T07:38:48.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.881+0000 7fa0a3400640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa084066f90 0x7fa084069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:48.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.881+0000 7fa0a3400640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa084066f90 0x7fa084069450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa094000c10 tx=0x7fa094009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:48.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:48.885+0000 7fa09b400640 1 -- 172.21.15.113:0/894157498 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa09c0c7020 con 0x7fa0a41f0390 2024-03-04T07:38:49.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 -- 172.21.15.113:0/894157498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0a40cc1e0 con 0x7fa0a41f0390 2024-03-04T07:38:49.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa09b400640 1 -- 172.21.15.113:0/894157498 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fa09c0169d0 con 0x7fa0a41f0390 2024-03-04T07:38:49.080 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:49.080 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:00.806973+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":29,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":4256,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4256":{"gid":4256,"name":"e","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.113:6837/1310042629","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1310042629},{"type":"v1","addr":"172.21.15.113:6837","nonce":1310042629}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276,4256]},"id":1}]} 2024-03-04T07:38:49.080 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 25 2024-03-04T07:38:49.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 -- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa084066f90 msgr2=0x7fa084069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:49.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa084066f90 0x7fa084069450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa094000c10 tx=0x7fa094009040 comp rx=0 tx=0).stop 2024-03-04T07:38:49.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 -- 172.21.15.113:0/894157498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a41f0390 msgr2=0x7fa0a41f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:49.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a41f0390 0x7fa0a41f0810 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0987f0 tx=0x7fa09c009750 comp rx=0 tx=0).stop 2024-03-04T07:38:49.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 -- 172.21.15.113:0/894157498 shutdown_connections 2024-03-04T07:38:49.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa084066f90 0x7fa084069450 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:49.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a41f0390 0x7fa0a41f0810 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:49.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a406c170 0x7fa0a415aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:49.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 --2- 172.21.15.113:0/894157498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0880051d0 0x7fa0a40c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:49.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 -- 172.21.15.113:0/894157498 >> 172.21.15.113:0/894157498 conn(0x7fa0a40734d0 msgr2=0x7fa0a40c91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:49.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 -- 172.21.15.113:0/894157498 shutdown_connections 2024-03-04T07:38:49.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:49.077+0000 7fa0aa400640 1 -- 172.21.15.113:0/894157498 wait complete. 2024-03-04T07:38:49.101 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:38:51.102 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:51.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dfc00640 1 Processor -- start 2024-03-04T07:38:51.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dfc00640 1 -- start start 2024-03-04T07:38:51.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dfc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7d81527f0 0x7fe7d8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dfc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d80c8200 0x7fe7d8072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dfc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe7d8152bd0 0x7fe7d81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dfc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d8071df0 con 0x7fe7d81527f0 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dfc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d81597b0 con 0x7fe7d8152bd0 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dfc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d81591f0 con 0x7fe7d80c8200 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dd200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7d81527f0 0x7fe7d8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dd200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7d81527f0 0x7fe7d8070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47512/0 (socket says 172.21.15.113:47512) 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dd200640 1 -- 172.21.15.113:0/2788410602 learned_addr learned my addr 172.21.15.113:0/2788410602 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7ddc00640 1 --2- 172.21.15.113:0/2788410602 >> v2:172.21.15.136:3300/0 conn(0x7fe7d8152bd0 0x7fe7d81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7d7e00640 1 --2- 172.21.15.113:0/2788410602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d80c8200 0x7fe7d8072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dd200640 1 -- 172.21.15.113:0/2788410602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d80c8200 msgr2=0x7fe7d8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dd200640 1 --2- 172.21.15.113:0/2788410602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d80c8200 0x7fe7d8072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dd200640 1 -- 172.21.15.113:0/2788410602 >> v2:172.21.15.136:3300/0 conn(0x7fe7d8152bd0 msgr2=0x7fe7d81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dd200640 1 --2- 172.21.15.113:0/2788410602 >> v2:172.21.15.136:3300/0 conn(0x7fe7d8152bd0 0x7fe7d81587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.237+0000 7fe7dd200640 1 -- 172.21.15.113:0/2788410602 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d81594d0 con 0x7fe7d81527f0 2024-03-04T07:38:51.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dd200640 1 --2- 172.21.15.113:0/2788410602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d81527f0 0x7fe7d8070d40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ca020 tx=0x7fe7cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abbbe1c2c8e9b59 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7d7400640 1 -- 172.21.15.113:0/2788410602 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc0ab5f0 con 0x7fe7d81527f0 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7d7400640 1 -- 172.21.15.113:0/2788410602 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7cc0b0070 con 0x7fe7d81527f0 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7d7400640 1 -- 172.21.15.113:0/2788410602 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc0ab9b0 con 0x7fe7d81527f0 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dfc00640 1 -- 172.21.15.113:0/2788410602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d81527f0 msgr2=0x7fe7d8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dfc00640 1 --2- 172.21.15.113:0/2788410602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d81527f0 0x7fe7d8070d40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ca020 tx=0x7fe7cc01c520 comp rx=0 tx=0).stop 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dfc00640 1 -- 172.21.15.113:0/2788410602 shutdown_connections 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dfc00640 1 --2- 172.21.15.113:0/2788410602 >> v2:172.21.15.136:3300/0 conn(0x7fe7d8152bd0 0x7fe7d81587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dfc00640 1 --2- 172.21.15.113:0/2788410602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d80c8200 0x7fe7d8072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dfc00640 1 --2- 172.21.15.113:0/2788410602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d81527f0 0x7fe7d8070d40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dfc00640 1 -- 172.21.15.113:0/2788410602 >> 172.21.15.113:0/2788410602 conn(0x7fe7d806e820 msgr2=0x7fe7d806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:51.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dfc00640 1 -- 172.21.15.113:0/2788410602 shutdown_connections 2024-03-04T07:38:51.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dfc00640 1 -- 172.21.15.113:0/2788410602 wait complete. 2024-03-04T07:38:51.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.241+0000 7fe7dfc00640 1 Processor -- start 2024-03-04T07:38:51.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dfc00640 1 -- start start 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dfc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d80c8200 0x7fe7d8157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dfc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d81527f0 0x7fe7d81583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dfc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d8152bd0 0x7fe7d81609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dfc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d815bf40 con 0x7fe7d8152bd0 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dfc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d815bdc0 con 0x7fe7d80c8200 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dfc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d815c0c0 con 0x7fe7d81527f0 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7ddc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d8152bd0 0x7fe7d81609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7ddc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d8152bd0 0x7fe7d81609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47520/0 (socket says 172.21.15.113:47520) 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7ddc00640 1 -- 172.21.15.113:0/1723623281 learned_addr learned my addr 172.21.15.113:0/1723623281 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7d7e00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d81527f0 0x7fe7d81583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dd200640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d80c8200 0x7fe7d8157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7ddc00640 1 -- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d81527f0 msgr2=0x7fe7d81583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7ddc00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d81527f0 0x7fe7d81583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7ddc00640 1 -- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d80c8200 msgr2=0x7fe7d8157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7ddc00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d80c8200 0x7fe7d8157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7ddc00640 1 -- 172.21.15.113:0/1723623281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d80b13a0 con 0x7fe7d8152bd0 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7d7e00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d81527f0 0x7fe7d81583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dd200640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d80c8200 0x7fe7d8157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7ddc00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d8152bd0 0x7fe7d81609b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0007e90 tx=0x7fe7d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7d4c00640 1 -- 172.21.15.113:0/1723623281 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0014420 con 0x7fe7d8152bd0 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7d4c00640 1 -- 172.21.15.113:0/1723623281 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d0019070 con 0x7fe7d8152bd0 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dfc00640 1 -- 172.21.15.113:0/1723623281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d80b0420 con 0x7fe7d8152bd0 2024-03-04T07:38:51.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7d4c00640 1 -- 172.21.15.113:0/1723623281 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0014800 con 0x7fe7d8152bd0 2024-03-04T07:38:51.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dfc00640 1 -- 172.21.15.113:0/1723623281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d81f4840 con 0x7fe7d8152bd0 2024-03-04T07:38:51.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.245+0000 7fe7dfc00640 1 -- 172.21.15.113:0/1723623281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d8075c00 con 0x7fe7d8152bd0 2024-03-04T07:38:51.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.249+0000 7fe7d4c00640 1 -- 172.21.15.113:0/1723623281 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7d00149a0 con 0x7fe7d8152bd0 2024-03-04T07:38:51.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.249+0000 7fe7d4c00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe7b8066f90 0x7fe7b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.249+0000 7fe7d4c00640 1 -- 172.21.15.113:0/1723623281 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fe7d00859f0 con 0x7fe7d8152bd0 2024-03-04T07:38:51.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.249+0000 7fe7dd200640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe7b8066f90 0x7fe7b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.249+0000 7fe7dd200640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe7b8066f90 0x7fe7b8069450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0af000 tx=0x7fe7cc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:51.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.257+0000 7fe7d4c00640 1 -- 172.21.15.113:0/1723623281 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe7d0056890 con 0x7fe7d8152bd0 2024-03-04T07:38:51.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.449+0000 7fe7dfc00640 1 -- 172.21.15.113:0/1723623281 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7d80c79d0 con 0x7fe7d8152bd0 2024-03-04T07:38:51.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.449+0000 7fe7d4c00640 1 -- 172.21.15.113:0/1723623281 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5813 (secure 0 0 0) 0x7fe7d0056190 con 0x7fe7d8152bd0 2024-03-04T07:38:51.455 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:51.455 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:50.604872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4938]},"id":1}]} 2024-03-04T07:38:51.455 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 26 2024-03-04T07:38:51.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 -- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe7b8066f90 msgr2=0x7fe7b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe7b8066f90 0x7fe7b8069450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0af000 tx=0x7fe7cc0a7000 comp rx=0 tx=0).stop 2024-03-04T07:38:51.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 -- 172.21.15.113:0/1723623281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d8152bd0 msgr2=0x7fe7d81609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d8152bd0 0x7fe7d81609b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0007e90 tx=0x7fe7d000e4b0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 -- 172.21.15.113:0/1723623281 shutdown_connections 2024-03-04T07:38:51.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe7b8066f90 0x7fe7b8069450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d8152bd0 0x7fe7d81609b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7d81527f0 0x7fe7d81583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 --2- 172.21.15.113:0/1723623281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7d80c8200 0x7fe7d8157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 -- 172.21.15.113:0/1723623281 >> 172.21.15.113:0/1723623281 conn(0x7fe7d806e820 msgr2=0x7fe7d814fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:51.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 -- 172.21.15.113:0/1723623281 shutdown_connections 2024-03-04T07:38:51.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.453+0000 7fe7dfc00640 1 -- 172.21.15.113:0/1723623281 wait complete. 2024-03-04T07:38:51.475 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-03-04T07:38:51.475 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:38:51.476 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f5481a00640 1 Processor -- start 2024-03-04T07:38:51.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f5481a00640 1 -- start start 2024-03-04T07:38:51.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f5481a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f547c06ff20 0x7f547c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f5481a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f547c0c8b90 0x7f547c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f5481a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547c152e30 con 0x7f547c152a50 2024-03-04T07:38:51.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f5481a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547c0c31b0 con 0x7f547c0c8b90 2024-03-04T07:38:51.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f5481a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c06f840 con 0x7f547c06ff20 2024-03-04T07:38:51.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f547b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f547c0c8b90 0x7f547c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f5473e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f547c06ff20 0x7f547c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f547b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f547c0c8b90 0x7f547c15c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47528/0 (socket says 172.21.15.113:47528) 2024-03-04T07:38:51.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.613+0000 7f547b400640 1 -- 172.21.15.113:0/1943500467 learned_addr learned my addr 172.21.15.113:0/1943500467 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:51.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.617+0000 7f547b400640 1 -- 172.21.15.113:0/1943500467 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f547c06ff20 msgr2=0x7f547c15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:51.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.617+0000 7f547b400640 1 --2- 172.21.15.113:0/1943500467 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f547c06ff20 0x7f547c15bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.617+0000 7f547b400640 1 -- 172.21.15.113:0/1943500467 >> v1:172.21.15.136:6789/0 conn(0x7f547c152a50 legacy=0x7f547c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.617+0000 7f547b400640 1 -- 172.21.15.113:0/1943500467 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c06f280 con 0x7f547c0c8b90 2024-03-04T07:38:51.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f547b400640 1 --2- 172.21.15.113:0/1943500467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c0c8b90 0x7f547c15c310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5474009e50 tx=0x7f547401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86293cd0620c5948 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:51.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f547a000640 1 -- 172.21.15.113:0/1943500467 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54740a5880 con 0x7f547c0c8b90 2024-03-04T07:38:51.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f547a000640 1 -- 172.21.15.113:0/1943500467 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54740a5a20 con 0x7f547c0c8b90 2024-03-04T07:38:51.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f547a000640 1 -- 172.21.15.113:0/1943500467 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54740a5df0 con 0x7f547c0c8b90 2024-03-04T07:38:51.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f5481a00640 1 -- 172.21.15.113:0/1943500467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c0c8b90 msgr2=0x7f547c15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f5481a00640 1 --2- 172.21.15.113:0/1943500467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c0c8b90 0x7f547c15c310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5474009e50 tx=0x7f547401c9a0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f5481a00640 1 -- 172.21.15.113:0/1943500467 shutdown_connections 2024-03-04T07:38:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f5481a00640 1 --2- 172.21.15.113:0/1943500467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c0c8b90 0x7f547c15c310 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f5481a00640 1 --2- 172.21.15.113:0/1943500467 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f547c06ff20 0x7f547c15bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f5481a00640 1 -- 172.21.15.113:0/1943500467 >> 172.21.15.113:0/1943500467 conn(0x7f547c0734d0 msgr2=0x7f547c066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f5481a00640 1 -- 172.21.15.113:0/1943500467 shutdown_connections 2024-03-04T07:38:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f5481a00640 1 -- 172.21.15.113:0/1943500467 wait complete. 2024-03-04T07:38:51.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f5481a00640 1 Processor -- start 2024-03-04T07:38:51.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.621+0000 7f5481a00640 1 -- start start 2024-03-04T07:38:51.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5481a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c06ff20 0x7f547c162590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5481a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f547c0c8b90 0x7f547c0ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5481a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f547c152a50 0x7f547c1f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5481a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0ce9b0 con 0x7f547c06ff20 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5481a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c0ce830 con 0x7f547c152a50 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5481a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c165880 con 0x7f547c0c8b90 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f547aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c06ff20 0x7f547c162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f547aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c06ff20 0x7f547c162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47530/0 (socket says 172.21.15.113:47530) 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f547aa00640 1 -- 172.21.15.113:0/616148660 learned_addr learned my addr 172.21.15.113:0/616148660 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5473e00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f547c0c8b90 0x7f547c0ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f547b400640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f547c152a50 0x7f547c1f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f547aa00640 1 -- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f547c0c8b90 msgr2=0x7f547c0ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f547aa00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f547c0c8b90 0x7f547c0ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f547aa00640 1 -- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f547c152a50 msgr2=0x7f547c1f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f547aa00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f547c152a50 0x7f547c1f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f547aa00640 1 -- 172.21.15.113:0/616148660 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c162cd0 con 0x7f547c06ff20 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f547aa00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c06ff20 0x7f547c162590 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f546c00c420 tx=0x7f546c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5472a00640 1 -- 172.21.15.113:0/616148660 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0052d0 con 0x7f547c06ff20 2024-03-04T07:38:51.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5472a00640 1 -- 172.21.15.113:0/616148660 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c005470 con 0x7f547c06ff20 2024-03-04T07:38:51.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5472a00640 1 -- 172.21.15.113:0/616148660 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c007260 con 0x7f547c06ff20 2024-03-04T07:38:51.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5481a00640 1 -- 172.21.15.113:0/616148660 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547c1f6f60 con 0x7f547c06ff20 2024-03-04T07:38:51.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5481a00640 1 -- 172.21.15.113:0/616148660 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c1f7520 con 0x7f547c06ff20 2024-03-04T07:38:51.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5481a00640 1 -- 172.21.15.113:0/616148660 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547c075c00 con 0x7f547c06ff20 2024-03-04T07:38:51.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.625+0000 7f5472a00640 1 -- 172.21.15.113:0/616148660 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f546c007400 con 0x7f547c06ff20 2024-03-04T07:38:51.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.629+0000 7f5472a00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5458066f60 0x7f5458069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:51.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.629+0000 7f5472a00640 1 -- 172.21.15.113:0/616148660 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f546c084750 con 0x7f547c06ff20 2024-03-04T07:38:51.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.629+0000 7f5473e00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5458066f60 0x7f5458069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:51.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.629+0000 7f5473e00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5458066f60 0x7f5458069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f547c0b2ef0 tx=0x7f5464008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:51.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.637+0000 7f5472a00640 1 -- 172.21.15.113:0/616148660 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f546c055540 con 0x7f547c06ff20 2024-03-04T07:38:51.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.821+0000 7f5481a00640 1 -- 172.21.15.113:0/616148660 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f547c14f560 con 0x7f547c06ff20 2024-03-04T07:38:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.821+0000 7f5472a00640 1 -- 172.21.15.113:0/616148660 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5813 (secure 0 0 0) 0x7f546c054e40 con 0x7f547c06ff20 2024-03-04T07:38:51.826 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:51.826 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:50.604872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4938]},"id":1}]} 2024-03-04T07:38:51.827 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 26 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 -- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5458066f60 msgr2=0x7f5458069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5458066f60 0x7f5458069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f547c0b2ef0 tx=0x7f5464008040 comp rx=0 tx=0).stop 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 -- 172.21.15.113:0/616148660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c06ff20 msgr2=0x7f547c162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c06ff20 0x7f547c162590 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f546c00c420 tx=0x7f546c00c8e0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 -- 172.21.15.113:0/616148660 shutdown_connections 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f547c152a50 0x7f547c1f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f547c0c8b90 0x7f547c0ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5458066f60 0x7f5458069420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 --2- 172.21.15.113:0/616148660 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f547c06ff20 0x7f547c162590 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 -- 172.21.15.113:0/616148660 >> 172.21.15.113:0/616148660 conn(0x7f547c0734d0 msgr2=0x7f547c159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 -- 172.21.15.113:0/616148660 shutdown_connections 2024-03-04T07:38:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:51.825+0000 7f5481a00640 1 -- 172.21.15.113:0/616148660 wait complete. 2024-03-04T07:38:51.849 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 26, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 26, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4276": { "addr": "172.21.15.136:6835/3348592059", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3348592059, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3348592059, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4276, "incarnation": 4, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4938": { "addr": "172.21.15.113:6836/2997246507", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2997246507, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2997246507, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4938, "incarnation": 20, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 25 }, "gid_5094": { "addr": "172.21.15.113:6833/605142559", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 605142559, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 605142559, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5094, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 30, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:38:50.604872+0000", "qdb_cluster": [ 4276, 4938 ], "qdb_leader": 4276, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4276, "mds_1": 5094, "mds_2": 4938 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:38:53.851 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:53.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e8200640 1 Processor -- start 2024-03-04T07:38:53.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e8200640 1 -- start start 2024-03-04T07:38:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e8200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f79e01527f0 0x7f79e0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 0x7f79e01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e8200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e01591f0 con 0x7f79e00c8200 2024-03-04T07:38:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e8200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e0071df0 con 0x7f79e01527f0 2024-03-04T07:38:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e01597b0 con 0x7f79e0152bd0 2024-03-04T07:38:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e6200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 0x7f79e01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e6200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 0x7f79e01587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:36642/0 (socket says 172.21.15.113:36642) 2024-03-04T07:38:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e5800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f79e01527f0 0x7f79e0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e6200640 1 -- 172.21.15.113:0/742197568 learned_addr learned my addr 172.21.15.113:0/742197568 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e4e00640 1 --1- 172.21.15.113:0/742197568 >> v1:172.21.15.136:6789/0 conn(0x7f79e00c8200 0x7f79e0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:45882/0 (socket says 172.21.15.113:45882) 2024-03-04T07:38:53.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79dfe00640 1 -- 172.21.15.113:0/742197568 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199142010 0 0) 0x7f79e01591f0 con 0x7f79e00c8200 2024-03-04T07:38:53.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79dfe00640 1 -- 172.21.15.113:0/742197568 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c8003530 con 0x7f79e00c8200 2024-03-04T07:38:53.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e6200640 1 -- 172.21.15.113:0/742197568 >> v2:172.21.15.136:3300/0 conn(0x7f79e01527f0 msgr2=0x7f79e0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:53.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e6200640 1 --2- 172.21.15.113:0/742197568 >> v2:172.21.15.136:3300/0 conn(0x7f79e01527f0 0x7f79e0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:53.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e6200640 1 -- 172.21.15.113:0/742197568 >> v1:172.21.15.136:6789/0 conn(0x7f79e00c8200 legacy=0x7f79e0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:53.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.985+0000 7f79e6200640 1 -- 172.21.15.113:0/742197568 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e01594d0 con 0x7f79e0152bd0 2024-03-04T07:38:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79e6200640 1 --2- 172.21.15.113:0/742197568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 0x7f79e01587d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009e50 tx=0x7f79d801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6506118fcebf31d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79dfe00640 1 -- 172.21.15.113:0/742197568 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d80a57f0 con 0x7f79e0152bd0 2024-03-04T07:38:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79dfe00640 1 -- 172.21.15.113:0/742197568 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d80a5990 con 0x7f79e0152bd0 2024-03-04T07:38:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79dfe00640 1 -- 172.21.15.113:0/742197568 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d80a5d60 con 0x7f79e0152bd0 2024-03-04T07:38:53.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79e8200640 1 -- 172.21.15.113:0/742197568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 msgr2=0x7f79e01587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:53.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79e8200640 1 --2- 172.21.15.113:0/742197568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 0x7f79e01587d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009e50 tx=0x7f79d801ca30 comp rx=0 tx=0).stop 2024-03-04T07:38:53.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79e8200640 1 -- 172.21.15.113:0/742197568 shutdown_connections 2024-03-04T07:38:53.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79e8200640 1 --2- 172.21.15.113:0/742197568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 0x7f79e01587d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:53.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79e8200640 1 --2- 172.21.15.113:0/742197568 >> v2:172.21.15.136:3300/0 conn(0x7f79e01527f0 0x7f79e0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:53.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79e8200640 1 -- 172.21.15.113:0/742197568 >> 172.21.15.113:0/742197568 conn(0x7f79e006e820 msgr2=0x7f79e006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:53.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79e8200640 1 -- 172.21.15.113:0/742197568 shutdown_connections 2024-03-04T07:38:53.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.989+0000 7f79e8200640 1 -- 172.21.15.113:0/742197568 wait complete. 2024-03-04T07:38:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.993+0000 7f79e8200640 1 Processor -- start 2024-03-04T07:38:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.993+0000 7f79e8200640 1 -- start start 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.993+0000 7f79e8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79e00c8200 0x7f79e0150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.993+0000 7f79e8200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e01527f0 0x7f79e014bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.993+0000 7f79e8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 0x7f79e01f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.993+0000 7f79e8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e01654f0 con 0x7f79e00c8200 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.993+0000 7f79e8200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0165370 con 0x7f79e01527f0 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.993+0000 7f79e8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0165670 con 0x7f79e0152bd0 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79e00c8200 0x7f79e0150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79e00c8200 0x7f79e0150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47544/0 (socket says 172.21.15.113:47544) 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e5800640 1 -- 172.21.15.113:0/3748228197 learned_addr learned my addr 172.21.15.113:0/3748228197 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e4e00640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e01527f0 0x7f79e014bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e6200640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 0x7f79e01f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:54.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e5800640 1 -- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 msgr2=0x7f79e01f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:54.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e5800640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 0x7f79e01f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:54.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e5800640 1 -- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e01527f0 msgr2=0x7f79e014bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:54.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e5800640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e01527f0 0x7f79e014bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:54.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e5800640 1 -- 172.21.15.113:0/3748228197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0150660 con 0x7f79e00c8200 2024-03-04T07:38:54.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e4e00640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e01527f0 0x7f79e014bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:38:54.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e5800640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79e00c8200 0x7f79e0150120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f79d400e930 tx=0x7f79d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:54.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79dd600640 1 -- 172.21.15.113:0/3748228197 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4017070 con 0x7f79e00c8200 2024-03-04T07:38:54.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79dd600640 1 -- 172.21.15.113:0/3748228197 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d400ca00 con 0x7f79e00c8200 2024-03-04T07:38:54.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e8200640 1 -- 172.21.15.113:0/3748228197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e00c2d60 con 0x7f79e00c8200 2024-03-04T07:38:54.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79dd600640 1 -- 172.21.15.113:0/3748228197 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400cdb0 con 0x7f79e00c8200 2024-03-04T07:38:54.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e8200640 1 -- 172.21.15.113:0/3748228197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e00c55e0 con 0x7f79e00c8200 2024-03-04T07:38:54.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79e8200640 1 -- 172.21.15.113:0/3748228197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e0075c00 con 0x7f79e00c8200 2024-03-04T07:38:54.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:53.997+0000 7f79dd600640 1 -- 172.21.15.113:0/3748228197 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79d401a070 con 0x7f79e00c8200 2024-03-04T07:38:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.001+0000 7f79dd600640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c0066f60 0x7f79c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.001+0000 7f79dd600640 1 -- 172.21.15.113:0/3748228197 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f79d40851f0 con 0x7f79e00c8200 2024-03-04T07:38:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.001+0000 7f79e4e00640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c0066f60 0x7f79c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:54.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.001+0000 7f79e4e00640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c0066f60 0x7f79c0069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f79cc007fd0 tx=0x7f79cc007ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:54.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.001+0000 7f79dd600640 1 -- 172.21.15.113:0/3748228197 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f79d4056130 con 0x7f79e00c8200 2024-03-04T07:38:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 -- 172.21.15.113:0/3748228197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79e00c79d0 con 0x7f79e00c8200 2024-03-04T07:38:54.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79dd600640 1 -- 172.21.15.113:0/3748228197 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5813 (secure 0 0 0) 0x7f79d4055a30 con 0x7f79e00c8200 2024-03-04T07:38:54.196 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:54.196 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:50.604872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4938]},"id":1}]} 2024-03-04T07:38:54.196 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 26 2024-03-04T07:38:54.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 -- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c0066f60 msgr2=0x7f79c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:54.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c0066f60 0x7f79c0069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f79cc007fd0 tx=0x7f79cc007ba0 comp rx=0 tx=0).stop 2024-03-04T07:38:54.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 -- 172.21.15.113:0/3748228197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79e00c8200 msgr2=0x7f79e0150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:54.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79e00c8200 0x7f79e0150120 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f79d400e930 tx=0x7f79d400edf0 comp rx=0 tx=0).stop 2024-03-04T07:38:54.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 -- 172.21.15.113:0/3748228197 shutdown_connections 2024-03-04T07:38:54.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79e0152bd0 0x7f79e01f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:54.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e01527f0 0x7f79e014bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:54.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c0066f60 0x7f79c0069420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:54.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 --2- 172.21.15.113:0/3748228197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79e00c8200 0x7f79e0150120 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:54.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 -- 172.21.15.113:0/3748228197 >> 172.21.15.113:0/3748228197 conn(0x7f79e006e820 msgr2=0x7f79e006ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:54.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 -- 172.21.15.113:0/3748228197 shutdown_connections 2024-03-04T07:38:54.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:54.193+0000 7f79e8200640 1 -- 172.21.15.113:0/3748228197 wait complete. 2024-03-04T07:38:54.219 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:38:55.560 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:38:55.564 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:38:56.219 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f9445800640 1 Processor -- start 2024-03-04T07:38:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f9445800640 1 -- start start 2024-03-04T07:38:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f9445800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f94400cb540 0x7f944015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f9445800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f944006faa0 0x7f94400c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f9445800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400c4ce0 con 0x7f9440152950 2024-03-04T07:38:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f9445800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944006ff20 con 0x7f94400cb540 2024-03-04T07:38:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f9445800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94400c8370 con 0x7f944006faa0 2024-03-04T07:38:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f943f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f94400cb540 0x7f944015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f943f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f94400cb540 0x7f944015bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47550/0 (socket says 172.21.15.113:47550) 2024-03-04T07:38:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f943f400640 1 -- 172.21.15.113:0/2132398753 learned_addr learned my addr 172.21.15.113:0/2132398753 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.353+0000 7f9444e00640 1 --2- 172.21.15.113:0/2132398753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f944006faa0 0x7f94400c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.357+0000 7f943f400640 1 -- 172.21.15.113:0/2132398753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f944006faa0 msgr2=0x7f94400c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:38:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.357+0000 7f943f400640 1 --2- 172.21.15.113:0/2132398753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f944006faa0 0x7f94400c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.357+0000 7f943f400640 1 -- 172.21.15.113:0/2132398753 >> v1:172.21.15.136:6789/0 conn(0x7f9440152950 legacy=0x7f944006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.357+0000 7f943f400640 1 -- 172.21.15.113:0/2132398753 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400c4f80 con 0x7f94400cb540 2024-03-04T07:38:56.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.357+0000 7f943f400640 1 --2- 172.21.15.113:0/2132398753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94400cb540 0x7f944015bb40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f942c009870 tx=0x7f942c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d00a490569084c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.357+0000 7f943ea00640 1 -- 172.21.15.113:0/2132398753 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0b5070 con 0x7f94400cb540 2024-03-04T07:38:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.357+0000 7f943ea00640 1 -- 172.21.15.113:0/2132398753 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c0a8a50 con 0x7f94400cb540 2024-03-04T07:38:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.357+0000 7f943ea00640 1 -- 172.21.15.113:0/2132398753 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0a8e20 con 0x7f94400cb540 2024-03-04T07:38:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 -- 172.21.15.113:0/2132398753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94400cb540 msgr2=0x7f944015bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 --2- 172.21.15.113:0/2132398753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94400cb540 0x7f944015bb40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f942c009870 tx=0x7f942c01ed50 comp rx=0 tx=0).stop 2024-03-04T07:38:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 -- 172.21.15.113:0/2132398753 shutdown_connections 2024-03-04T07:38:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 --2- 172.21.15.113:0/2132398753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f944006faa0 0x7f94400c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 --2- 172.21.15.113:0/2132398753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94400cb540 0x7f944015bb40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 -- 172.21.15.113:0/2132398753 >> 172.21.15.113:0/2132398753 conn(0x7f9440072f40 msgr2=0x7f94400706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 -- 172.21.15.113:0/2132398753 shutdown_connections 2024-03-04T07:38:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 -- 172.21.15.113:0/2132398753 wait complete. 2024-03-04T07:38:56.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 Processor -- start 2024-03-04T07:38:56.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 -- start start 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f944006faa0 0x7f94400cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94400cb540 0x7f94400cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9440152950 0x7f94401e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400c5220 con 0x7f94400cb540 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400ca0a0 con 0x7f944006faa0 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9445800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94400c9a50 con 0x7f9440152950 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9444e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9440152950 0x7f94401e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f943f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94400cb540 0x7f94400cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f943fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f944006faa0 0x7f94400cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9444e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9440152950 0x7f94401e23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:36672/0 (socket says 172.21.15.113:36672) 2024-03-04T07:38:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f9444e00640 1 -- 172.21.15.113:0/3263347985 learned_addr learned my addr 172.21.15.113:0/3263347985 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:56.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f943f400640 1 -- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9440152950 msgr2=0x7f94401e23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:56.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f943f400640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9440152950 0x7f94401e23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:56.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f943f400640 1 -- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f944006faa0 msgr2=0x7f94400cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:56.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f943f400640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f944006faa0 0x7f94400cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:56.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f943f400640 1 -- 172.21.15.113:0/3263347985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94401f1150 con 0x7f94400cb540 2024-03-04T07:38:56.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.361+0000 7f943f400640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94400cb540 0x7f94400cca90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f942c0af000 tx=0x7f942c01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:56.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.365+0000 7f9437e00640 1 -- 172.21.15.113:0/3263347985 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0c0070 con 0x7f94400cb540 2024-03-04T07:38:56.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.365+0000 7f9437e00640 1 -- 172.21.15.113:0/3263347985 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c0ae550 con 0x7f94400cb540 2024-03-04T07:38:56.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.365+0000 7f9437e00640 1 -- 172.21.15.113:0/3263347985 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0b5070 con 0x7f94400cb540 2024-03-04T07:38:56.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.365+0000 7f9445800640 1 -- 172.21.15.113:0/3263347985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94401f1410 con 0x7f94400cb540 2024-03-04T07:38:56.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.365+0000 7f9445800640 1 -- 172.21.15.113:0/3263347985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94400cc030 con 0x7f94400cb540 2024-03-04T07:38:56.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.365+0000 7f9445800640 1 -- 172.21.15.113:0/3263347985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94400cdfa0 con 0x7f94400cb540 2024-03-04T07:38:56.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.365+0000 7f9437e00640 1 -- 172.21.15.113:0/3263347985 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f942c0b9070 con 0x7f94400cb540 2024-03-04T07:38:56.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.365+0000 7f9437e00640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9420066f60 0x7f9420069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:56.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.365+0000 7f9437e00640 1 -- 172.21.15.113:0/3263347985 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f942c122760 con 0x7f94400cb540 2024-03-04T07:38:56.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.365+0000 7f943fe00640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9420066f60 0x7f9420069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:56.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.369+0000 7f943fe00640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9420066f60 0x7f9420069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9430007fd0 tx=0x7f9430009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:56.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.369+0000 7f9437e00640 1 -- 172.21.15.113:0/3263347985 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f94400cdfa0 con 0x7f94400cb540 2024-03-04T07:38:56.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.557+0000 7f9445800640 1 -- 172.21.15.113:0/3263347985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9440075a80 con 0x7f94400cb540 2024-03-04T07:38:56.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.557+0000 7f9437e00640 1 -- 172.21.15.113:0/3263347985 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f942c0188b0 con 0x7f94400cb540 2024-03-04T07:38:56.561 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:56.561 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:55.815255+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:resolve","state_seq":15,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276]},"id":1}]} 2024-03-04T07:38:56.561 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 27 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 -- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9420066f60 msgr2=0x7f9420069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9420066f60 0x7f9420069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9430007fd0 tx=0x7f9430009040 comp rx=0 tx=0).stop 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 -- 172.21.15.113:0/3263347985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94400cb540 msgr2=0x7f94400cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94400cb540 0x7f94400cca90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f942c0af000 tx=0x7f942c01baf0 comp rx=0 tx=0).stop 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 -- 172.21.15.113:0/3263347985 shutdown_connections 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9440152950 0x7f94401e23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9420066f60 0x7f9420069420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94400cb540 0x7f94400cca90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 --2- 172.21.15.113:0/3263347985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f944006faa0 0x7f94400cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 -- 172.21.15.113:0/3263347985 >> 172.21.15.113:0/3263347985 conn(0x7f9440072f40 msgr2=0x7f94400705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 -- 172.21.15.113:0/3263347985 shutdown_connections 2024-03-04T07:38:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:56.561+0000 7f9445800640 1 -- 172.21.15.113:0/3263347985 wait complete. 2024-03-04T07:38:56.581 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:38:58.582 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:38:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d0b200640 1 Processor -- start 2024-03-04T07:38:58.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d0b200640 1 -- start start 2024-03-04T07:38:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d0b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d041527f0 0x7f0d04070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d0b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d041591f0 con 0x7f0d040c8200 2024-03-04T07:38:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d0b200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d041594d0 con 0x7f0d04152bd0 2024-03-04T07:38:58.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d0b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d04071df0 con 0x7f0d041527f0 2024-03-04T07:38:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d03e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d041527f0 0x7f0d04070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d03400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0d040c8200 0x7f0d04072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33150/0 (socket says 172.21.15.113:33150) 2024-03-04T07:38:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d03400640 1 -- 172.21.15.113:0/3789125273 learned_addr learned my addr 172.21.15.113:0/3789125273 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:38:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d03e00640 1 -- 172.21.15.113:0/3789125273 >> v1:172.21.15.136:6789/0 conn(0x7f0d04152bd0 legacy=0x7f0d041587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d03e00640 1 -- 172.21.15.113:0/3789125273 >> v1:172.21.15.113:6789/0 conn(0x7f0d040c8200 legacy=0x7f0d04072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d03e00640 1 -- 172.21.15.113:0/3789125273 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d040717a0 con 0x7f0d041527f0 2024-03-04T07:38:58.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.717+0000 7f0d02a00640 1 -- 172.21.15.113:0/3789125273 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799421080 0 0) 0x7f0d041591f0 con 0x7f0d040c8200 2024-03-04T07:38:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d03e00640 1 --2- 172.21.15.113:0/3789125273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d041527f0 0x7f0d04070d40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40097c0 tx=0x7f0cf401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f7b9e005bd2d4a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d02a00640 1 -- 172.21.15.113:0/3789125273 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf40b1070 con 0x7f0d041527f0 2024-03-04T07:38:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d02a00640 1 -- 172.21.15.113:0/3789125273 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf40ac410 con 0x7f0d041527f0 2024-03-04T07:38:58.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d02a00640 1 -- 172.21.15.113:0/3789125273 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf40ac7e0 con 0x7f0d041527f0 2024-03-04T07:38:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d0b200640 1 -- 172.21.15.113:0/3789125273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d041527f0 msgr2=0x7f0d04070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d0b200640 1 --2- 172.21.15.113:0/3789125273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d041527f0 0x7f0d04070d40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40097c0 tx=0x7f0cf401c4a0 comp rx=0 tx=0).stop 2024-03-04T07:38:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d0b200640 1 -- 172.21.15.113:0/3789125273 shutdown_connections 2024-03-04T07:38:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d0b200640 1 --2- 172.21.15.113:0/3789125273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d041527f0 0x7f0d04070d40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d0b200640 1 -- 172.21.15.113:0/3789125273 >> 172.21.15.113:0/3789125273 conn(0x7f0d0406e820 msgr2=0x7f0d0406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d0b200640 1 -- 172.21.15.113:0/3789125273 shutdown_connections 2024-03-04T07:38:58.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d0b200640 1 -- 172.21.15.113:0/3789125273 wait complete. 2024-03-04T07:38:58.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.721+0000 7f0d0b200640 1 Processor -- start 2024-03-04T07:38:58.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d0b200640 1 -- start start 2024-03-04T07:38:58.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d0b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040c8200 0x7f0d0414e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:58.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d0b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d041527f0 0x7f0d04155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:58.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d0b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152bd0 0x7f0d040c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d0b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04166380 con 0x7f0d04152bd0 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d0b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d041597b0 con 0x7f0d040c8200 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d0b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04166500 con 0x7f0d041527f0 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d09200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152bd0 0x7f0d040c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d09200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152bd0 0x7f0d040c13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58736/0 (socket says 172.21.15.113:58736) 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d09200640 1 -- 172.21.15.113:0/1546374946 learned_addr learned my addr 172.21.15.113:0/1546374946 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d03e00640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040c8200 0x7f0d0414e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d03400640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d041527f0 0x7f0d04155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d09200640 1 -- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d041527f0 msgr2=0x7f0d04155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d09200640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d041527f0 0x7f0d04155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d09200640 1 -- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040c8200 msgr2=0x7f0d0414e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d09200640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040c8200 0x7f0d0414e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d09200640 1 -- 172.21.15.113:0/1546374946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d040c3b00 con 0x7f0d04152bd0 2024-03-04T07:38:58.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d09200640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152bd0 0x7f0d040c13d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00c760 tx=0x7f0cfc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:58.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1546374946 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc003b80 con 0x7f0d04152bd0 2024-03-04T07:38:58.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1546374946 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cfc003d20 con 0x7f0d04152bd0 2024-03-04T07:38:58.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1546374946 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc00aa30 con 0x7f0d04152bd0 2024-03-04T07:38:58.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d0b200640 1 -- 172.21.15.113:0/1546374946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d041f5d50 con 0x7f0d04152bd0 2024-03-04T07:38:58.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d0b200640 1 -- 172.21.15.113:0/1546374946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d041f6310 con 0x7f0d04152bd0 2024-03-04T07:38:58.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.725+0000 7f0d0b200640 1 -- 172.21.15.113:0/1546374946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d04075c00 con 0x7f0d04152bd0 2024-03-04T07:38:58.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.729+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1546374946 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0cfc00abd0 con 0x7f0d04152bd0 2024-03-04T07:38:58.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.729+0000 7f0cfbe00640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce4066f60 0x7f0ce4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:38:58.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.729+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1546374946 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f0cfc084f30 con 0x7f0d04152bd0 2024-03-04T07:38:58.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.729+0000 7f0d03e00640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce4066f60 0x7f0ce4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:38:58.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.729+0000 7f0d03e00640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce4066f60 0x7f0ce4069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40ae000 tx=0x7f0cf4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:38:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.733+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1546374946 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0d040c3b00 con 0x7f0d04152bd0 2024-03-04T07:38:58.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.921+0000 7f0d0b200640 1 -- 172.21.15.113:0/1546374946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d04158cc0 con 0x7f0d04152bd0 2024-03-04T07:38:58.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.921+0000 7f0cfbe00640 1 -- 172.21.15.113:0/1546374946 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f0cfc055e50 con 0x7f0d04152bd0 2024-03-04T07:38:58.925 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:38:58.925 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:38:55.815255+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:resolve","state_seq":15,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276]},"id":1}]} 2024-03-04T07:38:58.925 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 27 2024-03-04T07:38:58.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 -- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce4066f60 msgr2=0x7f0ce4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:58.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce4066f60 0x7f0ce4069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40ae000 tx=0x7f0cf4008040 comp rx=0 tx=0).stop 2024-03-04T07:38:58.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 -- 172.21.15.113:0/1546374946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152bd0 msgr2=0x7f0d040c13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:38:58.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152bd0 0x7f0d040c13d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00c760 tx=0x7f0cfc00cc20 comp rx=0 tx=0).stop 2024-03-04T07:38:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 -- 172.21.15.113:0/1546374946 shutdown_connections 2024-03-04T07:38:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0ce4066f60 0x7f0ce4069420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d04152bd0 0x7f0d040c13d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d041527f0 0x7f0d04155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 --2- 172.21.15.113:0/1546374946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d040c8200 0x7f0d0414e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:38:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 -- 172.21.15.113:0/1546374946 >> 172.21.15.113:0/1546374946 conn(0x7f0d0406e820 msgr2=0x7f0d040c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:38:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 -- 172.21.15.113:0/1546374946 shutdown_connections 2024-03-04T07:38:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:38:58.925+0000 7f0d0b200640 1 -- 172.21.15.113:0/1546374946 wait complete. 2024-03-04T07:38:58.945 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:39:00.946 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:39:01.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa01ae00640 1 Processor -- start 2024-03-04T07:39:01.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa01ae00640 1 -- start start 2024-03-04T07:39:01.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa01ae00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014152c50 0x7fa01406b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa01ae00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01415bda0 con 0x7fa0140c7930 2024-03-04T07:39:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa01ae00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01415c040 con 0x7fa014071a30 2024-03-04T07:39:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa01ae00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140c7fd0 con 0x7fa014152c50 2024-03-04T07:39:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa019000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa0140c7930 0x7fa0140c8cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33152/0 (socket says 172.21.15.113:33152) 2024-03-04T07:39:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa019000640 1 -- 172.21.15.113:0/749880108 learned_addr learned my addr 172.21.15.113:0/749880108 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:39:01.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa019a00640 1 --2- 172.21.15.113:0/749880108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014152c50 0x7fa01406b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa013e00640 1 -- 172.21.15.113:0/749880108 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854619787 0 0) 0x7fa01415bda0 con 0x7fa0140c7930 2024-03-04T07:39:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa013e00640 1 -- 172.21.15.113:0/749880108 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ffc003530 con 0x7fa0140c7930 2024-03-04T07:39:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa019a00640 1 -- 172.21.15.113:0/749880108 >> v1:172.21.15.136:6789/0 conn(0x7fa014071a30 legacy=0x7fa014071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa019a00640 1 -- 172.21.15.113:0/749880108 >> v1:172.21.15.113:6789/0 conn(0x7fa0140c7930 legacy=0x7fa0140c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:01.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.081+0000 7fa019a00640 1 -- 172.21.15.113:0/749880108 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01415c320 con 0x7fa014152c50 2024-03-04T07:39:01.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa019a00640 1 --2- 172.21.15.113:0/749880108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014152c50 0x7fa01406b060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa0080097d0 tx=0x7fa00801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51b27ef0748f1d6b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:01.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa013e00640 1 -- 172.21.15.113:0/749880108 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080b5070 con 0x7fa014152c50 2024-03-04T07:39:01.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa013e00640 1 -- 172.21.15.113:0/749880108 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0080a89b0 con 0x7fa014152c50 2024-03-04T07:39:01.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa013e00640 1 -- 172.21.15.113:0/749880108 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080a8d60 con 0x7fa014152c50 2024-03-04T07:39:01.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa01ae00640 1 -- 172.21.15.113:0/749880108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014152c50 msgr2=0x7fa01406b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:01.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa01ae00640 1 --2- 172.21.15.113:0/749880108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014152c50 0x7fa01406b060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa0080097d0 tx=0x7fa00801ecb0 comp rx=0 tx=0).stop 2024-03-04T07:39:01.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa01ae00640 1 -- 172.21.15.113:0/749880108 shutdown_connections 2024-03-04T07:39:01.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa01ae00640 1 --2- 172.21.15.113:0/749880108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014152c50 0x7fa01406b060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:01.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa01ae00640 1 -- 172.21.15.113:0/749880108 >> 172.21.15.113:0/749880108 conn(0x7fa0140c9a30 msgr2=0x7fa0140c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:01.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa01ae00640 1 -- 172.21.15.113:0/749880108 shutdown_connections 2024-03-04T07:39:01.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.085+0000 7fa01ae00640 1 -- 172.21.15.113:0/749880108 wait complete. 2024-03-04T07:39:01.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01ae00640 1 Processor -- start 2024-03-04T07:39:01.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01ae00640 1 -- start start 2024-03-04T07:39:01.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01ae00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014071a30 0x7fa01415a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:01.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01ae00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0140c7930 0x7fa01415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:01.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01ae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa014152c50 0x7fa01414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:01.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01ae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01406b710 con 0x7fa014152c50 2024-03-04T07:39:01.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01ae00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01415c8e0 con 0x7fa0140c7930 2024-03-04T07:39:01.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01ae00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140c9400 con 0x7fa014071a30 2024-03-04T07:39:01.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa014152c50 0x7fa01414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:01.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa014152c50 0x7fa01414bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58742/0 (socket says 172.21.15.113:58742) 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01a400640 1 -- 172.21.15.113:0/2317895883 learned_addr learned my addr 172.21.15.113:0/2317895883 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa019a00640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014071a30 0x7fa01415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa019000640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0140c7930 0x7fa01415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01a400640 1 -- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014071a30 msgr2=0x7fa01415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01a400640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014071a30 0x7fa01415a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01a400640 1 -- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0140c7930 msgr2=0x7fa01415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01a400640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0140c7930 0x7fa01415a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01a400640 1 -- 172.21.15.113:0/2317895883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0141f11f0 con 0x7fa014152c50 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa019a00640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014071a30 0x7fa01415a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01a400640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa014152c50 0x7fa01414bfd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00c760 tx=0x7fa00c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa011600640 1 -- 172.21.15.113:0/2317895883 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c003b80 con 0x7fa014152c50 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa011600640 1 -- 172.21.15.113:0/2317895883 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c003d20 con 0x7fa014152c50 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01ae00640 1 -- 172.21.15.113:0/2317895883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0141f0f10 con 0x7fa014152c50 2024-03-04T07:39:01.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa011600640 1 -- 172.21.15.113:0/2317895883 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00aa30 con 0x7fa014152c50 2024-03-04T07:39:01.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.089+0000 7fa01ae00640 1 -- 172.21.15.113:0/2317895883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa014159c50 con 0x7fa014152c50 2024-03-04T07:39:01.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.093+0000 7fa01ae00640 1 -- 172.21.15.113:0/2317895883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa014075a80 con 0x7fa014152c50 2024-03-04T07:39:01.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.093+0000 7fa011600640 1 -- 172.21.15.113:0/2317895883 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa00c00abd0 con 0x7fa014152c50 2024-03-04T07:39:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.093+0000 7fa011600640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9ff4066f90 0x7f9ff4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.093+0000 7fa011600640 1 -- 172.21.15.113:0/2317895883 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fa00c0850f0 con 0x7fa014152c50 2024-03-04T07:39:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.093+0000 7fa019a00640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9ff4066f90 0x7f9ff4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.093+0000 7fa019a00640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9ff4066f90 0x7f9ff4069450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0080b2000 tx=0x7fa0080aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.097+0000 7fa011600640 1 -- 172.21.15.113:0/2317895883 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa00c055e50 con 0x7fa014152c50 2024-03-04T07:39:01.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.361+0000 7fa01ae00640 1 -- 172.21.15.113:0/2317895883 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa01414fc50 con 0x7fa014152c50 2024-03-04T07:39:01.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.361+0000 7fa011600640 1 -- 172.21.15.113:0/2317895883 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5817 (secure 0 0 0) 0x7fa00c055750 con 0x7fa014152c50 2024-03-04T07:39:01.366 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:39:01.366 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:00.943129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4938]},"id":1}]} 2024-03-04T07:39:01.366 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 28 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 -- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9ff4066f90 msgr2=0x7f9ff4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9ff4066f90 0x7f9ff4069450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0080b2000 tx=0x7fa0080aa070 comp rx=0 tx=0).stop 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 -- 172.21.15.113:0/2317895883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa014152c50 msgr2=0x7fa01414bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa014152c50 0x7fa01414bfd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00c760 tx=0x7fa00c00cc20 comp rx=0 tx=0).stop 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 -- 172.21.15.113:0/2317895883 shutdown_connections 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9ff4066f90 0x7f9ff4069450 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa014152c50 0x7fa01414bfd0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0140c7930 0x7fa01415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 --2- 172.21.15.113:0/2317895883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa014071a30 0x7fa01415a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 -- 172.21.15.113:0/2317895883 >> 172.21.15.113:0/2317895883 conn(0x7fa0140c9a30 msgr2=0x7fa0140c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 -- 172.21.15.113:0/2317895883 shutdown_connections 2024-03-04T07:39:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:01.365+0000 7fa01ae00640 1 -- 172.21.15.113:0/2317895883 wait complete. 2024-03-04T07:39:01.387 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:39:03.387 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:39:03.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a1e400640 1 Processor -- start 2024-03-04T07:39:03.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a1e400640 1 -- start start 2024-03-04T07:39:03.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a1e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a1814e4c0 0x7f5a18072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:03.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a1e400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a180c27f0 con 0x7f5a180c22e0 2024-03-04T07:39:03.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a1e400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a180c2a90 con 0x7f5a1815b900 2024-03-04T07:39:03.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a1e400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1815ad60 con 0x7f5a1814e4c0 2024-03-04T07:39:03.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a17400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a1814e4c0 0x7f5a18072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:03.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a17400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a1814e4c0 0x7f5a18072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58746/0 (socket says 172.21.15.113:58746) 2024-03-04T07:39:03.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a17400640 1 -- 172.21.15.113:0/643989680 learned_addr learned my addr 172.21.15.113:0/643989680 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:03.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a17400640 1 -- 172.21.15.113:0/643989680 >> v1:172.21.15.136:6789/0 conn(0x7f5a1815b900 legacy=0x7f5a181620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:03.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a17400640 1 -- 172.21.15.113:0/643989680 >> v1:172.21.15.113:6789/0 conn(0x7f5a180c22e0 legacy=0x7f5a1815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:03.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.573+0000 7f5a17400640 1 -- 172.21.15.113:0/643989680 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a18070c60 con 0x7f5a1814e4c0 2024-03-04T07:39:03.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a17400640 1 --2- 172.21.15.113:0/643989680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 0x7f5a18072c80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800bb10 tx=0x7f5a08021420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21560f46acde809 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:03.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a16000640 1 -- 172.21.15.113:0/643989680 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a080b6070 con 0x7f5a1814e4c0 2024-03-04T07:39:03.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a16000640 1 -- 172.21.15.113:0/643989680 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a080b1600 con 0x7f5a1814e4c0 2024-03-04T07:39:03.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a16000640 1 -- 172.21.15.113:0/643989680 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a080b19d0 con 0x7f5a1814e4c0 2024-03-04T07:39:03.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a1e400640 1 -- 172.21.15.113:0/643989680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 msgr2=0x7f5a18072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:03.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a1e400640 1 --2- 172.21.15.113:0/643989680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 0x7f5a18072c80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800bb10 tx=0x7f5a08021420 comp rx=0 tx=0).stop 2024-03-04T07:39:03.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a1e400640 1 -- 172.21.15.113:0/643989680 shutdown_connections 2024-03-04T07:39:03.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a1e400640 1 --2- 172.21.15.113:0/643989680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 0x7f5a18072c80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:03.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a1e400640 1 -- 172.21.15.113:0/643989680 >> 172.21.15.113:0/643989680 conn(0x7f5a1806e820 msgr2=0x7f5a1806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:03.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a1e400640 1 -- 172.21.15.113:0/643989680 shutdown_connections 2024-03-04T07:39:03.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.577+0000 7f5a1e400640 1 -- 172.21.15.113:0/643989680 wait complete. 2024-03-04T07:39:03.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 Processor -- start 2024-03-04T07:39:03.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 -- start start 2024-03-04T07:39:03.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a180c22e0 0x7f5a18159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:03.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 0x7f5a18149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:03.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a1815b900 0x7f5a181499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:03.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a18071500 con 0x7f5a1814e4c0 2024-03-04T07:39:03.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a18071220 con 0x7f5a1815b900 2024-03-04T07:39:03.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1814e8a0 con 0x7f5a180c22e0 2024-03-04T07:39:03.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a16a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 0x7f5a18149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a16a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 0x7f5a18149490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58760/0 (socket says 172.21.15.113:58760) 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a16a00640 1 -- 172.21.15.113:0/2020012774 learned_addr learned my addr 172.21.15.113:0/2020012774 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a17e00640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a1815b900 0x7f5a181499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a17400640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a180c22e0 0x7f5a18159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a16a00640 1 -- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a180c22e0 msgr2=0x7f5a18159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a16a00640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a180c22e0 0x7f5a18159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a16a00640 1 -- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a1815b900 msgr2=0x7f5a181499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a16a00640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a1815b900 0x7f5a181499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a16a00640 1 -- 172.21.15.113:0/2020012774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a180c7bb0 con 0x7f5a1814e4c0 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a17e00640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a1815b900 0x7f5a181499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a16a00640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 0x7f5a18149490 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c3f0 tx=0x7f5a0400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a0f400640 1 -- 172.21.15.113:0/2020012774 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04017070 con 0x7f5a1814e4c0 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a0f400640 1 -- 172.21.15.113:0/2020012774 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a040052d0 con 0x7f5a1814e4c0 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 -- 172.21.15.113:0/2020012774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a181f13d0 con 0x7f5a1814e4c0 2024-03-04T07:39:03.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a0f400640 1 -- 172.21.15.113:0/2020012774 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04003840 con 0x7f5a1814e4c0 2024-03-04T07:39:03.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 -- 172.21.15.113:0/2020012774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1806bbf0 con 0x7f5a1814e4c0 2024-03-04T07:39:03.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.581+0000 7f5a1e400640 1 -- 172.21.15.113:0/2020012774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a18075c00 con 0x7f5a1814e4c0 2024-03-04T07:39:03.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.585+0000 7f5a0f400640 1 -- 172.21.15.113:0/2020012774 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a0401a070 con 0x7f5a1814e4c0 2024-03-04T07:39:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.585+0000 7f5a0f400640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59f8066f90 0x7f59f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.585+0000 7f5a0f400640 1 -- 172.21.15.113:0/2020012774 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f5a04084660 con 0x7f5a1814e4c0 2024-03-04T07:39:03.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.585+0000 7f5a17400640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59f8066f90 0x7f59f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:03.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.585+0000 7f5a17400640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59f8066f90 0x7f59f8069450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a080b3000 tx=0x7f5a08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:03.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.589+0000 7f5a0f400640 1 -- 172.21.15.113:0/2020012774 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5a04005480 con 0x7f5a1814e4c0 2024-03-04T07:39:03.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.785+0000 7f5a1e400640 1 -- 172.21.15.113:0/2020012774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a1806bdf0 con 0x7f5a1814e4c0 2024-03-04T07:39:03.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.785+0000 7f5a0f400640 1 -- 172.21.15.113:0/2020012774 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5817 (secure 0 0 0) 0x7f5a040553c0 con 0x7f5a1814e4c0 2024-03-04T07:39:03.789 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:39:03.789 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:00.943129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4938]},"id":1}]} 2024-03-04T07:39:03.790 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 28 2024-03-04T07:39:03.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 -- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59f8066f90 msgr2=0x7f59f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:03.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59f8066f90 0x7f59f8069450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a080b3000 tx=0x7f5a08008040 comp rx=0 tx=0).stop 2024-03-04T07:39:03.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 -- 172.21.15.113:0/2020012774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 msgr2=0x7f5a18149490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:03.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 0x7f5a18149490 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c3f0 tx=0x7f5a0400c8b0 comp rx=0 tx=0).stop 2024-03-04T07:39:03.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 -- 172.21.15.113:0/2020012774 shutdown_connections 2024-03-04T07:39:03.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a1815b900 0x7f5a181499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:03.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59f8066f90 0x7f59f8069450 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:03.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a1814e4c0 0x7f5a18149490 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:03.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 --2- 172.21.15.113:0/2020012774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a180c22e0 0x7f5a18159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:03.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 -- 172.21.15.113:0/2020012774 >> 172.21.15.113:0/2020012774 conn(0x7f5a1806e820 msgr2=0x7f5a18072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:03.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 -- 172.21.15.113:0/2020012774 shutdown_connections 2024-03-04T07:39:03.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:03.789+0000 7f5a1e400640 1 -- 172.21.15.113:0/2020012774 wait complete. 2024-03-04T07:39:03.810 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 28, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 28, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4276": { "addr": "172.21.15.136:6835/3348592059", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3348592059, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3348592059, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4276, "incarnation": 4, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 5 }, "gid_4938": { "addr": "172.21.15.113:6836/2997246507", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2997246507, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2997246507, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4938, "incarnation": 20, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 25 }, "gid_5094": { "addr": "172.21.15.113:6833/605142559", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 605142559, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 605142559, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5094, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:reconnect", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 30, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:39:00.943129+0000", "qdb_cluster": [ 4276, 4938 ], "qdb_leader": 4276, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4276, "mds_1": 5094, "mds_2": 4938 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:39:05.812 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:39:05.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e35400640 1 Processor -- start 2024-03-04T07:39:05.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e35400640 1 -- start start 2024-03-04T07:39:05.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e35400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e300c22e0 0x7f8e3015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:05.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e35400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8e300c2d80 0x7f8e3015b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:05.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e35400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e300c27f0 con 0x7f8e3014e4c0 2024-03-04T07:39:05.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e35400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3015ad60 con 0x7f8e300c2d80 2024-03-04T07:39:05.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e35400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300c2a90 con 0x7f8e300c22e0 2024-03-04T07:39:05.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2e800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8e3014e4c0 0x7f8e30072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33172/0 (socket says 172.21.15.113:33172) 2024-03-04T07:39:05.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2e800640 1 -- 172.21.15.113:0/736421651 learned_addr learned my addr 172.21.15.113:0/736421651 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:39:05.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2f200640 1 --2- 172.21.15.113:0/736421651 >> v2:172.21.15.136:3300/0 conn(0x7f8e300c2d80 0x7f8e3015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:05.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e25e00640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e300c22e0 0x7f8e3015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:05.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958244259 0 0) 0x7f8e300c27f0 con 0x7f8e3014e4c0 2024-03-04T07:39:05.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e14003530 con 0x7f8e3014e4c0 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3666630586 0 0) 0x7f8e14003530 con 0x7f8e3014e4c0 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e300c22e0 msgr2=0x7f8e3015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e300c22e0 0x7f8e3015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 >> v2:172.21.15.136:3300/0 conn(0x7f8e300c2d80 msgr2=0x7f8e3015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 --2- 172.21.15.113:0/736421651 >> v2:172.21.15.136:3300/0 conn(0x7f8e300c2d80 0x7f8e3015b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e30070c60 con 0x7f8e3014e4c0 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f8e20001680 con 0x7f8e3014e4c0 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 >> v1:172.21.15.113:6789/0 conn(0x7f8e3014e4c0 legacy=0x7f8e30072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e140039c0 0x7f8e14003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 0x7f8e14004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e300c22e0 0x7f8e140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300c2a90 con 0x7f8e14004810 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1400b9d0 con 0x7f8e140039c0 2024-03-04T07:39:05.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1400bb80 con 0x7f8e300c22e0 2024-03-04T07:39:05.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e25e00640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 0x7f8e14004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:05.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2e800640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e140039c0 0x7f8e14003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:05.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e2f200640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e300c22e0 0x7f8e140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:05.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e25e00640 1 -- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e300c22e0 msgr2=0x7f8e140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:05.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e25e00640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e300c22e0 0x7f8e140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:05.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e25e00640 1 -- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e140039c0 msgr2=0x7f8e14003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:05.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e25e00640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e140039c0 0x7f8e14003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:05.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.945+0000 7f8e25e00640 1 -- 172.21.15.113:0/736421651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14003530 con 0x7f8e14004810 2024-03-04T07:39:05.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e25e00640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 0x7f8e14004c90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009ae0 tx=0x7f8e1801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdec976597e3f05 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:05.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180b2070 con 0x7f8e14004810 2024-03-04T07:39:05.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e180a5580 con 0x7f8e14004810 2024-03-04T07:39:05.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e2de00640 1 -- 172.21.15.113:0/736421651 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180096b0 con 0x7f8e14004810 2024-03-04T07:39:05.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e35400640 1 -- 172.21.15.113:0/736421651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 msgr2=0x7f8e14004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:05.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e35400640 1 --2- 172.21.15.113:0/736421651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 0x7f8e14004c90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009ae0 tx=0x7f8e1801ca70 comp rx=0 tx=0).stop 2024-03-04T07:39:05.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e2f200640 1 -- 172.21.15.113:0/736421651 reap_dead start 2024-03-04T07:39:05.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e35400640 1 -- 172.21.15.113:0/736421651 shutdown_connections 2024-03-04T07:39:05.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e35400640 1 -- 172.21.15.113:0/736421651 >> 172.21.15.113:0/736421651 conn(0x7f8e3006e820 msgr2=0x7f8e3006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:05.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e35400640 1 -- 172.21.15.113:0/736421651 shutdown_connections 2024-03-04T07:39:05.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e35400640 1 -- 172.21.15.113:0/736421651 wait complete. 2024-03-04T07:39:05.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e35400640 1 Processor -- start 2024-03-04T07:39:05.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.953+0000 7f8e35400640 1 -- start start 2024-03-04T07:39:05.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e35400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 0x7f8e3014d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:05.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e35400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e3014b0e0 0x7f8e30159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:05.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e35400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e301f3c50 0x7f8e301468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:05.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e35400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30071500 con 0x7f8e14004810 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e35400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30071220 con 0x7f8e301f3c50 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e35400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3014e8a0 con 0x7f8e3014b0e0 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e2e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 0x7f8e3014d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e2e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 0x7f8e3014d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58780/0 (socket says 172.21.15.113:58780) 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e2e800640 1 -- 172.21.15.113:0/2832960630 learned_addr learned my addr 172.21.15.113:0/2832960630 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e25e00640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e3014b0e0 0x7f8e30159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e2f200640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e301f3c50 0x7f8e301468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e2e800640 1 -- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e3014b0e0 msgr2=0x7f8e30159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e2e800640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e3014b0e0 0x7f8e30159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e2e800640 1 -- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e301f3c50 msgr2=0x7f8e301468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e2e800640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e301f3c50 0x7f8e301468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e2e800640 1 -- 172.21.15.113:0/2832960630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e301494e0 con 0x7f8e14004810 2024-03-04T07:39:05.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e2e800640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 0x7f8e3014d2d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8e20004f80 tx=0x7f8e200074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:05.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e27400640 1 -- 172.21.15.113:0/2832960630 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2000f750 con 0x7f8e14004810 2024-03-04T07:39:05.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e27400640 1 -- 172.21.15.113:0/2832960630 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e20018070 con 0x7f8e14004810 2024-03-04T07:39:05.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e27400640 1 -- 172.21.15.113:0/2832960630 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2000fb30 con 0x7f8e14004810 2024-03-04T07:39:05.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e35400640 1 -- 172.21.15.113:0/2832960630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e30149780 con 0x7f8e14004810 2024-03-04T07:39:05.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e35400640 1 -- 172.21.15.113:0/2832960630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3014cad0 con 0x7f8e14004810 2024-03-04T07:39:05.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.957+0000 7f8e35400640 1 -- 172.21.15.113:0/2832960630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e30075c00 con 0x7f8e14004810 2024-03-04T07:39:05.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.961+0000 7f8e27400640 1 -- 172.21.15.113:0/2832960630 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e2001c410 con 0x7f8e14004810 2024-03-04T07:39:05.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.961+0000 7f8e27400640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e0c066f90 0x7f8e0c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:05.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.961+0000 7f8e27400640 1 -- 172.21.15.113:0/2832960630 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f8e20084e30 con 0x7f8e14004810 2024-03-04T07:39:05.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.961+0000 7f8e25e00640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e0c066f90 0x7f8e0c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:05.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.961+0000 7f8e25e00640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e0c066f90 0x7f8e0c069450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8e180af000 tx=0x7f8e180046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:05.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:05.965+0000 7f8e27400640 1 -- 172.21.15.113:0/2832960630 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8e20055d50 con 0x7f8e14004810 2024-03-04T07:39:06.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.153+0000 7f8e35400640 1 -- 172.21.15.113:0/2832960630 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e30155190 con 0x7f8e14004810 2024-03-04T07:39:06.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.153+0000 7f8e27400640 1 -- 172.21.15.113:0/2832960630 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5817 (secure 0 0 0) 0x7f8e20055650 con 0x7f8e14004810 2024-03-04T07:39:06.157 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:39:06.157 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:00.943129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4938]},"id":1}]} 2024-03-04T07:39:06.157 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 28 2024-03-04T07:39:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 -- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e0c066f90 msgr2=0x7f8e0c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e0c066f90 0x7f8e0c069450 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8e180af000 tx=0x7f8e180046d0 comp rx=0 tx=0).stop 2024-03-04T07:39:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 -- 172.21.15.113:0/2832960630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 msgr2=0x7f8e3014d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 0x7f8e3014d2d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8e20004f80 tx=0x7f8e200074f0 comp rx=0 tx=0).stop 2024-03-04T07:39:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 -- 172.21.15.113:0/2832960630 shutdown_connections 2024-03-04T07:39:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e301f3c50 0x7f8e301468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e3014b0e0 0x7f8e30159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e0c066f90 0x7f8e0c069450 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 --2- 172.21.15.113:0/2832960630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e14004810 0x7f8e3014d2d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:06.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 -- 172.21.15.113:0/2832960630 >> 172.21.15.113:0/2832960630 conn(0x7f8e3006e820 msgr2=0x7f8e300c6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:06.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 -- 172.21.15.113:0/2832960630 shutdown_connections 2024-03-04T07:39:06.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:06.157+0000 7f8e35400640 1 -- 172.21.15.113:0/2832960630 wait complete. 2024-03-04T07:39:06.179 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:39:08.181 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:39:08.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.313+0000 7fc567200640 1 Processor -- start 2024-03-04T07:39:08.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.313+0000 7fc567200640 1 -- start start 2024-03-04T07:39:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc567200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc56006b060 0x7fc5600c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc567200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5600b2230 con 0x7fc5600733f0 2024-03-04T07:39:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc567200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5600c7bb0 con 0x7fc560158210 2024-03-04T07:39:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc567200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc560072fb0 con 0x7fc56006b060 2024-03-04T07:39:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc557e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc5600733f0 0x7fc5600ca400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53954/0 (socket says 172.21.15.113:53954) 2024-03-04T07:39:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc557e00640 1 -- 172.21.15.113:0/1393584610 learned_addr learned my addr 172.21.15.113:0/1393584610 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:39:08.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc565200640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc56006b060 0x7fc5600c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:08.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595847545 0 0) 0x7fc5600b2230 con 0x7fc5600733f0 2024-03-04T07:39:08.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc544003530 con 0x7fc5600733f0 2024-03-04T07:39:08.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195967653 0 0) 0x7fc5600c7bb0 con 0x7fc560158210 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5600b2230 con 0x7fc560158210 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2791490275 0 0) 0x7fc544003530 con 0x7fc5600733f0 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc56006b060 msgr2=0x7fc5600c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc56006b060 0x7fc5600c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 >> v1:172.21.15.136:6789/0 conn(0x7fc560158210 legacy=0x7fc5600cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5600c7e50 con 0x7fc5600733f0 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fc548001680 con 0x7fc5600733f0 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc565200640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc56006b060 0x7fc5600c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 >> v1:172.21.15.113:6789/0 conn(0x7fc5600733f0 legacy=0x7fc5600ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc544003b80 0x7fc544003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc544004810 0x7fc544004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc56006b060 0x7fc5440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc560072fb0 con 0x7fc544003b80 2024-03-04T07:39:08.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54400ba80 con 0x7fc544004810 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54400bc30 con 0x7fc56006b060 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55fe00640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc544003b80 0x7fc544003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc565200640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc56006b060 0x7fc5440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc557e00640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc544004810 0x7fc544004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55fe00640 1 -- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc56006b060 msgr2=0x7fc5440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55fe00640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc56006b060 0x7fc5440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55fe00640 1 -- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc544004810 msgr2=0x7fc544004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55fe00640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc544004810 0x7fc544004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc55fe00640 1 -- 172.21.15.113:0/1393584610 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc544003530 con 0x7fc544003b80 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc565200640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc56006b060 0x7fc5440051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:39:08.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.317+0000 7fc557e00640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc544004810 0x7fc544004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:39:08.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc55fe00640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc544003b80 0x7fc544003fa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc550007fd0 tx=0x7fc55001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fc83f54086714 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550001520 con 0x7fc544003b80 2024-03-04T07:39:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5500052f0 con 0x7fc544003b80 2024-03-04T07:39:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc55f400640 1 -- 172.21.15.113:0/1393584610 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550003840 con 0x7fc544003b80 2024-03-04T07:39:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc567200640 1 -- 172.21.15.113:0/1393584610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc544003b80 msgr2=0x7fc544003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc567200640 1 --2- 172.21.15.113:0/1393584610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc544003b80 0x7fc544003fa0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc550007fd0 tx=0x7fc55001c4e0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc565200640 1 -- 172.21.15.113:0/1393584610 reap_dead start 2024-03-04T07:39:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc567200640 1 -- 172.21.15.113:0/1393584610 shutdown_connections 2024-03-04T07:39:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc567200640 1 -- 172.21.15.113:0/1393584610 >> 172.21.15.113:0/1393584610 conn(0x7fc5600671b0 msgr2=0x7fc56006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc567200640 1 -- 172.21.15.113:0/1393584610 shutdown_connections 2024-03-04T07:39:08.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc567200640 1 -- 172.21.15.113:0/1393584610 wait complete. 2024-03-04T07:39:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc567200640 1 Processor -- start 2024-03-04T07:39:08.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.321+0000 7fc567200640 1 -- start start 2024-03-04T07:39:08.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc567200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc544003b80 0x7fc5601493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:08.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc567200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5601f7620 0x7fc560156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc567200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc56014d570 0x7fc56014d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc567200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5600cd8a0 con 0x7fc56014d570 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc567200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5600cd720 con 0x7fc5601f7620 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc567200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56006ce70 con 0x7fc544003b80 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc565200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc56014d570 0x7fc56014d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc565200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc56014d570 0x7fc56014d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53438/0 (socket says 172.21.15.113:53438) 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc55fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc544003b80 0x7fc5601493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc565200640 1 -- 172.21.15.113:0/3723476979 learned_addr learned my addr 172.21.15.113:0/3723476979 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc557e00640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5601f7620 0x7fc560156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc565200640 1 -- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc544003b80 msgr2=0x7fc5601493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc565200640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc544003b80 0x7fc5601493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc565200640 1 -- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5601f7620 msgr2=0x7fc560156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc565200640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5601f7620 0x7fc560156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc565200640 1 -- 172.21.15.113:0/3723476979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56014df30 con 0x7fc56014d570 2024-03-04T07:39:08.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc565200640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc56014d570 0x7fc56014d9f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc558009ed0 tx=0x7fc5580115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:08.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc55cc00640 1 -- 172.21.15.113:0/3723476979 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558011de0 con 0x7fc56014d570 2024-03-04T07:39:08.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc55cc00640 1 -- 172.21.15.113:0/3723476979 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55801b070 con 0x7fc56014d570 2024-03-04T07:39:08.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc55cc00640 1 -- 172.21.15.113:0/3723476979 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5580039a0 con 0x7fc56014d570 2024-03-04T07:39:08.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc567200640 1 -- 172.21.15.113:0/3723476979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc560146950 con 0x7fc56014d570 2024-03-04T07:39:08.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc567200640 1 -- 172.21.15.113:0/3723476979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc560146f10 con 0x7fc56014d570 2024-03-04T07:39:08.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.325+0000 7fc567200640 1 -- 172.21.15.113:0/3723476979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc560075c00 con 0x7fc56014d570 2024-03-04T07:39:08.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.329+0000 7fc55cc00640 1 -- 172.21.15.113:0/3723476979 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc558003b40 con 0x7fc56014d570 2024-03-04T07:39:08.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.329+0000 7fc55cc00640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc53c066f60 0x7fc53c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:08.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.329+0000 7fc55cc00640 1 -- 172.21.15.113:0/3723476979 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7fc5580856f0 con 0x7fc56014d570 2024-03-04T07:39:08.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.329+0000 7fc55fe00640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc53c066f60 0x7fc53c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:08.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.329+0000 7fc55fe00640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc53c066f60 0x7fc53c069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5500ac000 tx=0x7fc5500a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:08.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.333+0000 7fc55cc00640 1 -- 172.21.15.113:0/3723476979 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc56014df30 con 0x7fc56014d570 2024-03-04T07:39:08.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.521+0000 7fc567200640 1 -- 172.21.15.113:0/3723476979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5600731d0 con 0x7fc56014d570 2024-03-04T07:39:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.521+0000 7fc55cc00640 1 -- 172.21.15.113:0/3723476979 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5814 (secure 0 0 0) 0x7fc558083e20 con 0x7fc56014d570 2024-03-04T07:39:08.525 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:39:08.525 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:05.963977+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276]},"id":1}]} 2024-03-04T07:39:08.525 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 29 2024-03-04T07:39:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 -- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc53c066f60 msgr2=0x7fc53c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc53c066f60 0x7fc53c069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc5500ac000 tx=0x7fc5500a6000 comp rx=0 tx=0).stop 2024-03-04T07:39:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 -- 172.21.15.113:0/3723476979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc56014d570 msgr2=0x7fc56014d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc56014d570 0x7fc56014d9f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc558009ed0 tx=0x7fc5580115a0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 -- 172.21.15.113:0/3723476979 shutdown_connections 2024-03-04T07:39:08.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc53c066f60 0x7fc53c069420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc56014d570 0x7fc56014d9f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5601f7620 0x7fc560156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 --2- 172.21.15.113:0/3723476979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc544003b80 0x7fc5601493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:08.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 -- 172.21.15.113:0/3723476979 >> 172.21.15.113:0/3723476979 conn(0x7fc5600671b0 msgr2=0x7fc5600c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:08.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 -- 172.21.15.113:0/3723476979 shutdown_connections 2024-03-04T07:39:08.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:08.525+0000 7fc567200640 1 -- 172.21.15.113:0/3723476979 wait complete. 2024-03-04T07:39:08.547 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:39:10.548 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:39:10.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.681+0000 7f2aa1c00640 1 Processor -- start 2024-03-04T07:39:10.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.681+0000 7f2aa1c00640 1 -- start start 2024-03-04T07:39:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2aa1c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2a9c0c22e0 0x7f2a9c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2aa1c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a9c0c2d80 0x7f2a9c15b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2aa1c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9c0c27f0 con 0x7f2a9c14e4c0 2024-03-04T07:39:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2aa1c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c15ad60 con 0x7f2a9c0c2d80 2024-03-04T07:39:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2aa1c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c0c2a90 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a9b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a9c0c2d80 0x7f2a9c15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a9aa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2a9c14e4c0 0x7f2a9c072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53962/0 (socket says 172.21.15.113:53962) 2024-03-04T07:39:10.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a9b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a9c0c2d80 0x7f2a9c15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53450/0 (socket says 172.21.15.113:53450) 2024-03-04T07:39:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a9aa00640 1 -- 172.21.15.113:0/2538869443 learned_addr learned my addr 172.21.15.113:0/2538869443 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:39:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a93e00640 1 --2- 172.21.15.113:0/2538869443 >> v2:172.21.15.136:3300/0 conn(0x7f2a9c0c22e0 0x7f2a9c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a9b400640 1 -- 172.21.15.113:0/2538869443 >> v2:172.21.15.136:3300/0 conn(0x7f2a9c0c22e0 msgr2=0x7f2a9c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a9b400640 1 --2- 172.21.15.113:0/2538869443 >> v2:172.21.15.136:3300/0 conn(0x7f2a9c0c22e0 0x7f2a9c15b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a9b400640 1 -- 172.21.15.113:0/2538869443 >> v1:172.21.15.113:6789/0 conn(0x7f2a9c14e4c0 legacy=0x7f2a9c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a9b400640 1 -- 172.21.15.113:0/2538869443 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c070c60 con 0x7f2a9c0c2d80 2024-03-04T07:39:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a9a000640 1 -- 172.21.15.113:0/2538869443 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203979240 0 0) 0x7f2a9c0c27f0 con 0x7f2a9c14e4c0 2024-03-04T07:39:10.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.685+0000 7f2a93e00640 1 --2- 172.21.15.113:0/2538869443 >> v2:172.21.15.136:3300/0 conn(0x7f2a9c0c22e0 0x7f2a9c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:39:10.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2a9b400640 1 --2- 172.21.15.113:0/2538869443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c2d80 0x7f2a9c15b730 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0cd230 tx=0x7f2a9401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8306f799f0b970d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:10.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2a9a000640 1 -- 172.21.15.113:0/2538869443 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940b2070 con 0x7f2a9c0c2d80 2024-03-04T07:39:10.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2a9a000640 1 -- 172.21.15.113:0/2538869443 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a940a5800 con 0x7f2a9c0c2d80 2024-03-04T07:39:10.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2a9a000640 1 -- 172.21.15.113:0/2538869443 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940a5bd0 con 0x7f2a9c0c2d80 2024-03-04T07:39:10.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2aa1c00640 1 -- 172.21.15.113:0/2538869443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c2d80 msgr2=0x7f2a9c15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:10.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2aa1c00640 1 --2- 172.21.15.113:0/2538869443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c2d80 0x7f2a9c15b730 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0cd230 tx=0x7f2a9401c930 comp rx=0 tx=0).stop 2024-03-04T07:39:10.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2aa1c00640 1 -- 172.21.15.113:0/2538869443 shutdown_connections 2024-03-04T07:39:10.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2aa1c00640 1 --2- 172.21.15.113:0/2538869443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c2d80 0x7f2a9c15b730 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:10.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2aa1c00640 1 --2- 172.21.15.113:0/2538869443 >> v2:172.21.15.136:3300/0 conn(0x7f2a9c0c22e0 0x7f2a9c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:10.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2aa1c00640 1 -- 172.21.15.113:0/2538869443 >> 172.21.15.113:0/2538869443 conn(0x7f2a9c06e820 msgr2=0x7f2a9c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:10.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2aa1c00640 1 -- 172.21.15.113:0/2538869443 shutdown_connections 2024-03-04T07:39:10.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.689+0000 7f2aa1c00640 1 -- 172.21.15.113:0/2538869443 wait complete. 2024-03-04T07:39:10.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2aa1c00640 1 Processor -- start 2024-03-04T07:39:10.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2aa1c00640 1 -- start start 2024-03-04T07:39:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2aa1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c22e0 0x7f2a9c15a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2aa1c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c0c2d80 0x7f2a9c1f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2aa1c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a9c14e4c0 0x7f2a9c157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2aa1c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c0aede0 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2aa1c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c14e8a0 con 0x7f2a9c0c2d80 2024-03-04T07:39:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2aa1c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c073390 con 0x7f2a9c14e4c0 2024-03-04T07:39:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a9aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c22e0 0x7f2a9c15a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a9aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c22e0 0x7f2a9c15a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53466/0 (socket says 172.21.15.113:53466) 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a9aa00640 1 -- 172.21.15.113:0/497675813 learned_addr learned my addr 172.21.15.113:0/497675813 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a9b400640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a9c14e4c0 0x7f2a9c157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a93e00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c0c2d80 0x7f2a9c1f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a9aa00640 1 -- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a9c14e4c0 msgr2=0x7f2a9c157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a9aa00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a9c14e4c0 0x7f2a9c157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a9aa00640 1 -- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c0c2d80 msgr2=0x7f2a9c1f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a9aa00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c0c2d80 0x7f2a9c1f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a9aa00640 1 -- 172.21.15.113:0/497675813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c0b1c80 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a9aa00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c22e0 0x7f2a9c15a2b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0011f0 tx=0x7f2a8c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a92a00640 1 -- 172.21.15.113:0/497675813 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c018070 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a92a00640 1 -- 172.21.15.113:0/497675813 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c0074e0 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2aa1c00640 1 -- 172.21.15.113:0/497675813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c14b350 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2a92a00640 1 -- 172.21.15.113:0/497675813 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c004a10 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.693+0000 7f2aa1c00640 1 -- 172.21.15.113:0/497675813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c1487b0 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.697+0000 7f2aa1c00640 1 -- 172.21.15.113:0/497675813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c075c00 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.697+0000 7f2a92a00640 1 -- 172.21.15.113:0/497675813 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a8c01b070 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.697+0000 7f2a92a00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a78066f60 0x7f2a78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:10.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.697+0000 7f2a92a00640 1 -- 172.21.15.113:0/497675813 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f2a8c084560 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.697+0000 7f2a93e00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a78066f60 0x7f2a78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:10.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.697+0000 7f2a93e00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a78066f60 0x7f2a78069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c14a210 tx=0x7f2a84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.701+0000 7f2a92a00640 1 -- 172.21.15.113:0/497675813 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2a8c055360 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 -- 172.21.15.113:0/497675813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a9c0b2500 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2a92a00640 1 -- 172.21.15.113:0/497675813 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5814 (secure 0 0 0) 0x7f2a8c054c60 con 0x7f2a9c0c22e0 2024-03-04T07:39:10.896 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:39:10.896 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:05.963977+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4938,4276]},"id":1}]} 2024-03-04T07:39:10.897 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 29 2024-03-04T07:39:10.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 -- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a78066f60 msgr2=0x7f2a78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:10.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a78066f60 0x7f2a78069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c14a210 tx=0x7f2a84008040 comp rx=0 tx=0).stop 2024-03-04T07:39:10.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 -- 172.21.15.113:0/497675813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c22e0 msgr2=0x7f2a9c15a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:10.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c22e0 0x7f2a9c15a2b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0011f0 tx=0x7f2a8c00c9b0 comp rx=0 tx=0).stop 2024-03-04T07:39:10.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 -- 172.21.15.113:0/497675813 shutdown_connections 2024-03-04T07:39:10.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a9c14e4c0 0x7f2a9c157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:10.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c0c2d80 0x7f2a9c1f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:10.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a78066f60 0x7f2a78069420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:10.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 --2- 172.21.15.113:0/497675813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a9c0c22e0 0x7f2a9c15a2b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:10.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 -- 172.21.15.113:0/497675813 >> 172.21.15.113:0/497675813 conn(0x7f2a9c06e820 msgr2=0x7f2a9c0c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:10.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 -- 172.21.15.113:0/497675813 shutdown_connections 2024-03-04T07:39:10.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:10.893+0000 7f2aa1c00640 1 -- 172.21.15.113:0/497675813 wait complete. 2024-03-04T07:39:10.919 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:39:12.919 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:39:13.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.053+0000 7f22aaa00640 1 Processor -- start 2024-03-04T07:39:13.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.053+0000 7f22aaa00640 1 -- start start 2024-03-04T07:39:13.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f22aaa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c22e0 0x7f22a415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:13.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f22aaa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a40c27f0 con 0x7f22a40c2d80 2024-03-04T07:39:13.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f22aaa00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a40c2a90 con 0x7f22a414e4c0 2024-03-04T07:39:13.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f22aaa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a415ad60 con 0x7f22a40c22e0 2024-03-04T07:39:13.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f22a8a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f22a40c2d80 0x7f22a41620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53976/0 (socket says 172.21.15.113:53976) 2024-03-04T07:39:13.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f22a8a00640 1 -- 172.21.15.113:0/673017585 learned_addr learned my addr 172.21.15.113:0/673017585 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:39:13.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f229be00640 1 --2- 172.21.15.113:0/673017585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c22e0 0x7f22a415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:13.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f22a3400640 1 -- 172.21.15.113:0/673017585 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447400152 0 0) 0x7f22a40c27f0 con 0x7f22a40c2d80 2024-03-04T07:39:13.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f22a3400640 1 -- 172.21.15.113:0/673017585 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2288005740 con 0x7f22a40c2d80 2024-03-04T07:39:13.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f229be00640 1 -- 172.21.15.113:0/673017585 >> v1:172.21.15.136:6789/0 conn(0x7f22a414e4c0 legacy=0x7f22a4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:13.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f22a3400640 1 -- 172.21.15.113:0/673017585 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453731774 0 0) 0x7f22a40c2a90 con 0x7f22a414e4c0 2024-03-04T07:39:13.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f229be00640 1 -- 172.21.15.113:0/673017585 >> v1:172.21.15.113:6789/0 conn(0x7f22a40c2d80 legacy=0x7f22a41620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:13.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.057+0000 7f229be00640 1 -- 172.21.15.113:0/673017585 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4070c60 con 0x7f22a40c22e0 2024-03-04T07:39:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f229be00640 1 --2- 172.21.15.113:0/673017585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c22e0 0x7f22a415b1f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f228c0097f0 tx=0x7f228c01ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ce1ec52153cf70a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22a3400640 1 -- 172.21.15.113:0/673017585 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c0b5070 con 0x7f22a40c22e0 2024-03-04T07:39:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22a3400640 1 -- 172.21.15.113:0/673017585 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c0a89d0 con 0x7f22a40c22e0 2024-03-04T07:39:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22a3400640 1 -- 172.21.15.113:0/673017585 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c0a8da0 con 0x7f22a40c22e0 2024-03-04T07:39:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 -- 172.21.15.113:0/673017585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c22e0 msgr2=0x7f22a415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 --2- 172.21.15.113:0/673017585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c22e0 0x7f22a415b1f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f228c0097f0 tx=0x7f228c01ecd0 comp rx=0 tx=0).stop 2024-03-04T07:39:13.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 -- 172.21.15.113:0/673017585 shutdown_connections 2024-03-04T07:39:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 --2- 172.21.15.113:0/673017585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c22e0 0x7f22a415b1f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 -- 172.21.15.113:0/673017585 >> 172.21.15.113:0/673017585 conn(0x7f22a406e820 msgr2=0x7f22a406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 -- 172.21.15.113:0/673017585 shutdown_connections 2024-03-04T07:39:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 -- 172.21.15.113:0/673017585 wait complete. 2024-03-04T07:39:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 Processor -- start 2024-03-04T07:39:13.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 -- start start 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a40c22e0 0x7f22a40c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c2d80 0x7f22a4154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a414e4c0 0x7f22a40c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4071500 con 0x7f22a40c22e0 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4071220 con 0x7f22a414e4c0 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22aaa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a414e8a0 con 0x7f22a40c2d80 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a40c22e0 0x7f22a40c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a40c22e0 0x7f22a40c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53474/0 (socket says 172.21.15.113:53474) 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22a3e00640 1 -- 172.21.15.113:0/3009146098 learned_addr learned my addr 172.21.15.113:0/3009146098 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f229be00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c2d80 0x7f22a4154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:13.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.061+0000 7f22a8a00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a414e4c0 0x7f22a40c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a3e00640 1 -- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c2d80 msgr2=0x7f22a4154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a3e00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c2d80 0x7f22a4154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a3e00640 1 -- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a414e4c0 msgr2=0x7f22a40c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a3e00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a414e4c0 0x7f22a40c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a3e00640 1 -- 172.21.15.113:0/3009146098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a40c5de0 con 0x7f22a40c22e0 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a3e00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a40c22e0 0x7f22a40c5680 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f229400c4d0 tx=0x7f229400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a0c00640 1 -- 172.21.15.113:0/3009146098 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294017070 con 0x7f22a40c22e0 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a0c00640 1 -- 172.21.15.113:0/3009146098 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2294005480 con 0x7f22a40c22e0 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a0c00640 1 -- 172.21.15.113:0/3009146098 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294003840 con 0x7f22a40c22e0 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22aaa00640 1 -- 172.21.15.113:0/3009146098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a4152670 con 0x7f22a40c22e0 2024-03-04T07:39:13.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22aaa00640 1 -- 172.21.15.113:0/3009146098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a4152c30 con 0x7f22a40c22e0 2024-03-04T07:39:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22aaa00640 1 -- 172.21.15.113:0/3009146098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a4153010 con 0x7f22a40c22e0 2024-03-04T07:39:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a0c00640 1 -- 172.21.15.113:0/3009146098 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f229401a070 con 0x7f22a40c22e0 2024-03-04T07:39:13.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a0c00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2280066f90 0x7f2280069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:13.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.065+0000 7f22a0c00640 1 -- 172.21.15.113:0/3009146098 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f2294084580 con 0x7f22a40c22e0 2024-03-04T07:39:13.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.069+0000 7f229be00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2280066f90 0x7f2280069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:13.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.069+0000 7f229be00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2280066f90 0x7f2280069450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f228c0b2000 tx=0x7f228c0aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:13.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.069+0000 7f22a0c00640 1 -- 172.21.15.113:0/3009146098 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f22a4153010 con 0x7f22a40c22e0 2024-03-04T07:39:13.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 -- 172.21.15.113:0/3009146098 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22a4075c00 con 0x7f22a40c22e0 2024-03-04T07:39:13.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22a0c00640 1 -- 172.21.15.113:0/3009146098 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f2294054de0 con 0x7f22a40c22e0 2024-03-04T07:39:13.267 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:39:13.267 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[4276,4938,5094]},"id":1}]} 2024-03-04T07:39:13.268 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 30 2024-03-04T07:39:13.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 -- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2280066f90 msgr2=0x7f2280069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:13.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2280066f90 0x7f2280069450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f228c0b2000 tx=0x7f228c0aa070 comp rx=0 tx=0).stop 2024-03-04T07:39:13.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 -- 172.21.15.113:0/3009146098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a40c22e0 msgr2=0x7f22a40c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:13.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a40c22e0 0x7f22a40c5680 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f229400c4d0 tx=0x7f229400c990 comp rx=0 tx=0).stop 2024-03-04T07:39:13.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 -- 172.21.15.113:0/3009146098 shutdown_connections 2024-03-04T07:39:13.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a414e4c0 0x7f22a40c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:13.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a40c2d80 0x7f22a4154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:13.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2280066f90 0x7f2280069450 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:13.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 --2- 172.21.15.113:0/3009146098 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a40c22e0 0x7f22a40c5680 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:13.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 -- 172.21.15.113:0/3009146098 >> 172.21.15.113:0/3009146098 conn(0x7f22a406e820 msgr2=0x7f22a40c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:13.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 -- 172.21.15.113:0/3009146098 shutdown_connections 2024-03-04T07:39:13.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:13.265+0000 7f22aaa00640 1 -- 172.21.15.113:0/3009146098 wait complete. 2024-03-04T07:39:13.288 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=1, replacing gid=4256 2024-03-04T07:39:13.288 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.e 2024-03-04T07:39:15.289 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-03-04T07:39:15.289 INFO:tasks.ceph.mds.e:Restarting daemon 2024-03-04T07:39:15.289 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-03-04T07:39:15.293 INFO:tasks.ceph.mds.e:Started 2024-03-04T07:39:15.293 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:39:15.377 INFO:tasks.ceph.mds.e.smithi113.stdout:starting mds.e at 2024-03-04T07:39:15.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.429+0000 7f06d3400640 1 Processor -- start 2024-03-04T07:39:15.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.429+0000 7f06d3400640 1 -- start start 2024-03-04T07:39:15.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06d3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 0x7f06cc06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:15.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06d3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06cc06ff20 0x7f06cc15b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:15.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06d3400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06cc0ca0e0 con 0x7f06cc15b750 2024-03-04T07:39:15.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06d3400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06cc06f450 con 0x7f06cc06ff20 2024-03-04T07:39:15.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06d3400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc15ad80 con 0x7f06cc158010 2024-03-04T07:39:15.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06cbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06cc06ff20 0x7f06cc15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:15.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06cbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06cc06ff20 0x7f06cc15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53496/0 (socket says 172.21.15.113:53496) 2024-03-04T07:39:15.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06cbe00640 1 -- 172.21.15.113:0/2136967667 learned_addr learned my addr 172.21.15.113:0/2136967667 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:15.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06d0a00640 1 --2- 172.21.15.113:0/2136967667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 0x7f06cc06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:15.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06cbe00640 1 -- 172.21.15.113:0/2136967667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 msgr2=0x7f06cc06dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:15.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06cbe00640 1 --2- 172.21.15.113:0/2136967667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 0x7f06cc06dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:15.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06cbe00640 1 -- 172.21.15.113:0/2136967667 >> v1:172.21.15.136:6789/0 conn(0x7f06cc15b750 legacy=0x7f06cc0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:15.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06cbe00640 1 -- 172.21.15.113:0/2136967667 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc15aa30 con 0x7f06cc06ff20 2024-03-04T07:39:15.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.433+0000 7f06d0a00640 1 --2- 172.21.15.113:0/2136967667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 0x7f06cc06dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:39:15.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06cbe00640 1 --2- 172.21.15.113:0/2136967667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 0x7f06cc15b210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009870 tx=0x7f06b801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa357d1fb2bb10f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:15.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06cb400640 1 -- 172.21.15.113:0/2136967667 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b80b2070 con 0x7f06cc06ff20 2024-03-04T07:39:15.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06cb400640 1 -- 172.21.15.113:0/2136967667 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b80a9070 con 0x7f06cc06ff20 2024-03-04T07:39:15.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06cb400640 1 -- 172.21.15.113:0/2136967667 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b80ad880 con 0x7f06cc06ff20 2024-03-04T07:39:15.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06d3400640 1 -- 172.21.15.113:0/2136967667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 msgr2=0x7f06cc15b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:15.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06d3400640 1 --2- 172.21.15.113:0/2136967667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 0x7f06cc15b210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009870 tx=0x7f06b801c560 comp rx=0 tx=0).stop 2024-03-04T07:39:15.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06d3400640 1 -- 172.21.15.113:0/2136967667 shutdown_connections 2024-03-04T07:39:15.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06d3400640 1 --2- 172.21.15.113:0/2136967667 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 0x7f06cc15b210 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:15.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06d3400640 1 --2- 172.21.15.113:0/2136967667 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 0x7f06cc06dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:15.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06d3400640 1 -- 172.21.15.113:0/2136967667 >> 172.21.15.113:0/2136967667 conn(0x7f06cc0734d0 msgr2=0x7f06cc066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:15.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06d3400640 1 -- 172.21.15.113:0/2136967667 shutdown_connections 2024-03-04T07:39:15.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06d3400640 1 -- 172.21.15.113:0/2136967667 wait complete. 2024-03-04T07:39:15.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06d3400640 1 Processor -- start 2024-03-04T07:39:15.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.437+0000 7f06d3400640 1 -- start start 2024-03-04T07:39:15.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 0x7f06cc1f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:15.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 0x7f06cc1eca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06cc15b750 0x7f06cc0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d3400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc06c170 con 0x7f06cc06ff20 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d3400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc1583f0 con 0x7f06cc15b750 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d3400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc06c2f0 con 0x7f06cc158010 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 0x7f06cc1f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 0x7f06cc1f3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53500/0 (socket says 172.21.15.113:53500) 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d0a00640 1 -- 172.21.15.113:0/498742195 learned_addr learned my addr 172.21.15.113:0/498742195 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06cbe00640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 0x7f06cc1eca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d1400640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06cc15b750 0x7f06cc0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d0a00640 1 -- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 msgr2=0x7f06cc1eca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d0a00640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 0x7f06cc1eca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d0a00640 1 -- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06cc15b750 msgr2=0x7f06cc0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:15.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d0a00640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06cc15b750 0x7f06cc0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:15.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d0a00640 1 -- 172.21.15.113:0/498742195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc159310 con 0x7f06cc06ff20 2024-03-04T07:39:15.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d1400640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06cc15b750 0x7f06cc0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:39:15.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d0a00640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 0x7f06cc1f3920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f06c000e840 tx=0x7f06c000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:15.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06c8c00640 1 -- 172.21.15.113:0/498742195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0017070 con 0x7f06cc06ff20 2024-03-04T07:39:15.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06c8c00640 1 -- 172.21.15.113:0/498742195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c000c8d0 con 0x7f06cc06ff20 2024-03-04T07:39:15.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06c8c00640 1 -- 172.21.15.113:0/498742195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c000f040 con 0x7f06cc06ff20 2024-03-04T07:39:15.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d3400640 1 -- 172.21.15.113:0/498742195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc1595f0 con 0x7f06cc06ff20 2024-03-04T07:39:15.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d3400640 1 -- 172.21.15.113:0/498742195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06cc159bb0 con 0x7f06cc06ff20 2024-03-04T07:39:15.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06d3400640 1 -- 172.21.15.113:0/498742195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06cc06fd00 con 0x7f06cc06ff20 2024-03-04T07:39:15.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.441+0000 7f06c8c00640 1 -- 172.21.15.113:0/498742195 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06c001a070 con 0x7f06cc06ff20 2024-03-04T07:39:15.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.445+0000 7f06c8c00640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06ac066f90 0x7f06ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:15.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.445+0000 7f06c8c00640 1 -- 172.21.15.113:0/498742195 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6083+0+0 (secure 0 0 0) 0x7f06c0084db0 con 0x7f06cc06ff20 2024-03-04T07:39:15.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.445+0000 7f06cbe00640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06ac066f90 0x7f06ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:15.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.445+0000 7f06cbe00640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06ac066f90 0x7f06ac069450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f06b80af000 tx=0x7f06b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:15.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.449+0000 7f06c8c00640 1 -- 172.21.15.113:0/498742195 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f06c0055540 con 0x7f06cc06ff20 2024-03-04T07:39:15.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.641+0000 7f06d3400640 1 -- 172.21.15.113:0/498742195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06cc075be0 con 0x7f06cc06ff20 2024-03-04T07:39:15.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.641+0000 7f06c8c00640 1 -- 172.21.15.113:0/498742195 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6610 (secure 0 0 0) 0x7f06c00595e0 con 0x7f06cc06ff20 2024-03-04T07:39:15.645 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:39:15.645 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[5094,4938,4276]},"id":1}]} 2024-03-04T07:39:15.645 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-03-04T07:39:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 -- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06ac066f90 msgr2=0x7f06ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06ac066f90 0x7f06ac069450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f06b80af000 tx=0x7f06b8008040 comp rx=0 tx=0).stop 2024-03-04T07:39:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 -- 172.21.15.113:0/498742195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 msgr2=0x7f06cc1f3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 0x7f06cc1f3920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f06c000e840 tx=0x7f06c000ed00 comp rx=0 tx=0).stop 2024-03-04T07:39:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 -- 172.21.15.113:0/498742195 shutdown_connections 2024-03-04T07:39:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06cc15b750 0x7f06cc0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06cc158010 0x7f06cc1eca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06ac066f90 0x7f06ac069450 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:15.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 --2- 172.21.15.113:0/498742195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06cc06ff20 0x7f06cc1f3920 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 -- 172.21.15.113:0/498742195 >> 172.21.15.113:0/498742195 conn(0x7f06cc0734d0 msgr2=0x7f06cc1569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 -- 172.21.15.113:0/498742195 shutdown_connections 2024-03-04T07:39:15.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:15.645+0000 7f06d3400640 1 -- 172.21.15.113:0/498742195 wait complete. 2024-03-04T07:39:15.666 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-03-04T07:39:15.666 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 42 secs before thrashing 2024-03-04T07:39:16.976 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2024-03-04T07:39:16.976 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2024-03-04T07:39:16.976 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2024-03-04T07:39:16.976 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2024-03-04T07:39:16.976 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2024-03-04T07:39:16.976 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2024-03-04T07:39:16.977 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2024-03-04T07:39:16.978 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2024-03-04T07:39:16.979 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2024-03-04T07:39:16.980 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2024-03-04T07:39:16.981 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2024-03-04T07:39:16.982 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2024-03-04T07:39:16.982 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2024-03-04T07:39:16.982 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2024-03-04T07:39:18.046 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2024-03-04T07:39:18.046 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2024-03-04T07:39:18.047 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2024-03-04T07:39:18.048 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2024-03-04T07:39:18.049 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2024-03-04T07:39:18.050 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2024-03-04T07:39:18.051 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2024-03-04T07:39:18.051 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2024-03-04T07:39:18.051 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2024-03-04T07:39:18.051 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2024-03-04T07:39:18.051 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2024-03-04T07:39:18.051 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2024-03-04T07:39:18.051 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2024-03-04T07:39:18.051 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2024-03-04T07:39:18.051 INFO:tasks.workunit.client.0.smithi113.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2024-03-04T07:39:18.051 INFO:tasks.workunit.client.0.smithi113.stdout:Take third snapshot .snap/test3 2024-03-04T07:39:18.065 INFO:tasks.workunit.client.0.smithi113.stdout:Delete the snapshots... 2024-03-04T07:39:20.457 INFO:tasks.workunit.client.0.smithi113.stdout:.//.snap/test1 2024-03-04T07:39:20.457 INFO:tasks.workunit.client.0.smithi113.stdout:.//.snap/test2 2024-03-04T07:39:20.457 INFO:tasks.workunit.client.0.smithi113.stdout:.//.snap/test3 2024-03-04T07:39:20.464 INFO:tasks.workunit.client.0.smithi113.stdout:./274/.snap/snap-subdir-test 2024-03-04T07:39:20.474 INFO:tasks.workunit.client.0.smithi113.stdout:./166/.snap/snap-subdir-test 2024-03-04T07:39:20.482 INFO:tasks.workunit.client.0.smithi113.stdout:./256/.snap/snap-subdir-test 2024-03-04T07:39:21.087 INFO:tasks.workunit.client.0.smithi113.stdout:./247/.snap/snap-subdir-test 2024-03-04T07:39:21.907 INFO:tasks.workunit.client.0.smithi113.stdout:./248/.snap/snap-subdir-test 2024-03-04T07:39:21.918 INFO:tasks.workunit.client.0.smithi113.stdout:./147/.snap/snap-subdir-test 2024-03-04T07:39:21.930 INFO:tasks.workunit.client.0.smithi113.stdout:./151/.snap/snap-subdir-test 2024-03-04T07:39:23.316 INFO:tasks.workunit.client.0.smithi113.stdout:./170/.snap/snap-subdir-test 2024-03-04T07:39:23.328 INFO:tasks.workunit.client.0.smithi113.stdout:./167/.snap/snap-subdir-test 2024-03-04T07:39:23.340 INFO:tasks.workunit.client.0.smithi113.stdout:./140/.snap/snap-subdir-test 2024-03-04T07:39:23.352 INFO:tasks.workunit.client.0.smithi113.stdout:./142/.snap/snap-subdir-test 2024-03-04T07:39:23.364 INFO:tasks.workunit.client.0.smithi113.stdout:./150/.snap/snap-subdir-test 2024-03-04T07:39:23.372 INFO:tasks.workunit.client.0.smithi113.stdout:./273/.snap/snap-subdir-test 2024-03-04T07:39:23.380 INFO:tasks.workunit.client.0.smithi113.stdout:./269/.snap/snap-subdir-test 2024-03-04T07:39:23.384 INFO:tasks.workunit.client.0.smithi113.stdout:./105/.snap/snap-subdir-test 2024-03-04T07:39:23.392 INFO:tasks.workunit.client.0.smithi113.stdout:./210/.snap/snap-subdir-test 2024-03-04T07:39:23.400 INFO:tasks.workunit.client.0.smithi113.stdout:./265/.snap/snap-subdir-test 2024-03-04T07:39:23.410 INFO:tasks.workunit.client.0.smithi113.stdout:./218/.snap/snap-subdir-test 2024-03-04T07:39:23.419 INFO:tasks.workunit.client.0.smithi113.stdout:./290/.snap/snap-subdir-test 2024-03-04T07:39:23.430 INFO:tasks.workunit.client.0.smithi113.stdout:./149/.snap/snap-subdir-test 2024-03-04T07:39:23.438 INFO:tasks.workunit.client.0.smithi113.stdout:./109/.snap/snap-subdir-test 2024-03-04T07:39:23.446 INFO:tasks.workunit.client.0.smithi113.stdout:./292/.snap/snap-subdir-test 2024-03-04T07:39:24.329 INFO:tasks.workunit.client.0.smithi113.stdout:./242/.snap/snap-subdir-test 2024-03-04T07:39:24.341 INFO:tasks.workunit.client.0.smithi113.stdout:./158/.snap/snap-subdir-test 2024-03-04T07:39:25.164 INFO:tasks.workunit.client.0.smithi113.stdout:./243/.snap/snap-subdir-test 2024-03-04T07:39:25.175 INFO:tasks.workunit.client.0.smithi113.stdout:./106/.snap/snap-subdir-test 2024-03-04T07:39:25.183 INFO:tasks.workunit.client.0.smithi113.stdout:./250/.snap/snap-subdir-test 2024-03-04T07:39:25.193 INFO:tasks.workunit.client.0.smithi113.stdout:./221/.snap/snap-subdir-test 2024-03-04T07:39:25.204 INFO:tasks.workunit.client.0.smithi113.stdout:./244/.snap/snap-subdir-test 2024-03-04T07:39:25.214 INFO:tasks.workunit.client.0.smithi113.stdout:./258/.snap/snap-subdir-test 2024-03-04T07:39:25.408 INFO:tasks.workunit.client.0.smithi113.stdout:./268/.snap/snap-subdir-test 2024-03-04T07:39:25.420 INFO:tasks.workunit.client.0.smithi113.stdout:./189/.snap/snap-subdir-test 2024-03-04T07:39:25.428 INFO:tasks.workunit.client.0.smithi113.stdout:./285/.snap/snap-subdir-test 2024-03-04T07:39:25.436 INFO:tasks.workunit.client.0.smithi113.stdout:./208/.snap/snap-subdir-test 2024-03-04T07:39:25.444 INFO:tasks.workunit.client.0.smithi113.stdout:./288/.snap/snap-subdir-test 2024-03-04T07:39:25.454 INFO:tasks.workunit.client.0.smithi113.stdout:./209/.snap/snap-subdir-test 2024-03-04T07:39:25.465 INFO:tasks.workunit.client.0.smithi113.stdout:./253/.snap/snap-subdir-test 2024-03-04T07:39:25.623 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:39:25.626 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:39:26.154 INFO:tasks.workunit.client.0.smithi113.stdout:./249/.snap/snap-subdir-test 2024-03-04T07:39:26.163 INFO:tasks.workunit.client.0.smithi113.stdout:./133/.snap/snap-subdir-test 2024-03-04T07:39:26.913 INFO:tasks.workunit.client.0.smithi113.stdout:./291/.snap/snap-subdir-test 2024-03-04T07:39:26.923 INFO:tasks.workunit.client.0.smithi113.stdout:./226/.snap/snap-subdir-test 2024-03-04T07:39:26.935 INFO:tasks.workunit.client.0.smithi113.stdout:./173/.snap/snap-subdir-test 2024-03-04T07:39:26.945 INFO:tasks.workunit.client.0.smithi113.stdout:./174/.snap/snap-subdir-test 2024-03-04T07:39:26.955 INFO:tasks.workunit.client.0.smithi113.stdout:./299/.snap/snap-subdir-test 2024-03-04T07:39:26.964 INFO:tasks.workunit.client.0.smithi113.stdout:./262/.snap/snap-subdir-test 2024-03-04T07:39:26.974 INFO:tasks.workunit.client.0.smithi113.stdout:./231/.snap/snap-subdir-test 2024-03-04T07:39:26.983 INFO:tasks.workunit.client.0.smithi113.stdout:./214/.snap/snap-subdir-test 2024-03-04T07:39:26.992 INFO:tasks.workunit.client.0.smithi113.stdout:./281/.snap/snap-subdir-test 2024-03-04T07:39:27.002 INFO:tasks.workunit.client.0.smithi113.stdout:./289/.snap/snap-subdir-test 2024-03-04T07:39:27.011 INFO:tasks.workunit.client.0.smithi113.stdout:./283/.snap/snap-subdir-test 2024-03-04T07:39:27.020 INFO:tasks.workunit.client.0.smithi113.stdout:./232/.snap/snap-subdir-test 2024-03-04T07:39:27.029 INFO:tasks.workunit.client.0.smithi113.stdout:./298/.snap/snap-subdir-test 2024-03-04T07:39:27.037 INFO:tasks.workunit.client.0.smithi113.stdout:./233/.snap/snap-subdir-test 2024-03-04T07:39:27.200 INFO:tasks.workunit.client.0.smithi113.stdout:./155/.snap/snap-subdir-test 2024-03-04T07:39:27.907 INFO:tasks.workunit.client.0.smithi113.stdout:./114/.snap/snap-subdir-test 2024-03-04T07:39:27.919 INFO:tasks.workunit.client.0.smithi113.stdout:./159/.snap/snap-subdir-test 2024-03-04T07:39:27.932 INFO:tasks.workunit.client.0.smithi113.stdout:./187/.snap/snap-subdir-test 2024-03-04T07:39:27.942 INFO:tasks.workunit.client.0.smithi113.stdout:./185/.snap/snap-subdir-test 2024-03-04T07:39:27.954 INFO:tasks.workunit.client.0.smithi113.stdout:./293/.snap/snap-subdir-test 2024-03-04T07:39:27.962 INFO:tasks.workunit.client.0.smithi113.stdout:./228/.snap/snap-subdir-test 2024-03-04T07:39:27.972 INFO:tasks.workunit.client.0.smithi113.stdout:./120/.snap/snap-subdir-test 2024-03-04T07:39:27.981 INFO:tasks.workunit.client.0.smithi113.stdout:./110/.snap/snap-subdir-test 2024-03-04T07:39:27.992 INFO:tasks.workunit.client.0.smithi113.stdout:./267/.snap/snap-subdir-test 2024-03-04T07:39:28.000 INFO:tasks.workunit.client.0.smithi113.stdout:./202/.snap/snap-subdir-test 2024-03-04T07:39:28.013 INFO:tasks.workunit.client.0.smithi113.stdout:./137/.snap/snap-subdir-test 2024-03-04T07:39:28.023 INFO:tasks.workunit.client.0.smithi113.stdout:./246/.snap/snap-subdir-test 2024-03-04T07:39:28.710 INFO:tasks.workunit.client.0.smithi113.stdout:./219/.snap/snap-subdir-test 2024-03-04T07:39:28.721 INFO:tasks.workunit.client.0.smithi113.stdout:./198/.snap/snap-subdir-test 2024-03-04T07:39:29.158 INFO:tasks.workunit.client.0.smithi113.stdout:./136/.snap/snap-subdir-test 2024-03-04T07:39:29.167 INFO:tasks.workunit.client.0.smithi113.stdout:./270/.snap/snap-subdir-test 2024-03-04T07:39:29.179 INFO:tasks.workunit.client.0.smithi113.stdout:./125/.snap/snap-subdir-test 2024-03-04T07:39:30.022 INFO:tasks.workunit.client.0.smithi113.stdout:./194/.snap/snap-subdir-test 2024-03-04T07:39:30.550 INFO:tasks.workunit.client.0.smithi113.stdout:./119/.snap/snap-subdir-test 2024-03-04T07:39:30.559 INFO:tasks.workunit.client.0.smithi113.stdout:./297/.snap/snap-subdir-test 2024-03-04T07:39:30.569 INFO:tasks.workunit.client.0.smithi113.stdout:./230/.snap/snap-subdir-test 2024-03-04T07:39:31.556 INFO:tasks.workunit.client.0.smithi113.stdout:./260/.snap/snap-subdir-test 2024-03-04T07:39:31.566 INFO:tasks.workunit.client.0.smithi113.stdout:./157/.snap/snap-subdir-test 2024-03-04T07:39:31.577 INFO:tasks.workunit.client.0.smithi113.stdout:./172/.snap/snap-subdir-test 2024-03-04T07:39:31.925 INFO:tasks.workunit.client.0.smithi113.stdout:./277/.snap/snap-subdir-test 2024-03-04T07:39:31.937 INFO:tasks.workunit.client.0.smithi113.stdout:./186/.snap/snap-subdir-test 2024-03-04T07:39:31.947 INFO:tasks.workunit.client.0.smithi113.stdout:./237/.snap/snap-subdir-test 2024-03-04T07:39:31.958 INFO:tasks.workunit.client.0.smithi113.stdout:./190/.snap/snap-subdir-test 2024-03-04T07:39:31.967 INFO:tasks.workunit.client.0.smithi113.stdout:./213/.snap/snap-subdir-test 2024-03-04T07:39:31.978 INFO:tasks.workunit.client.0.smithi113.stdout:./104/.snap/snap-subdir-test 2024-03-04T07:39:31.988 INFO:tasks.workunit.client.0.smithi113.stdout:./164/.snap/snap-subdir-test 2024-03-04T07:39:31.992 INFO:tasks.workunit.client.0.smithi113.stdout:./112/.snap/snap-subdir-test 2024-03-04T07:39:32.000 INFO:tasks.workunit.client.0.smithi113.stdout:./295/.snap/snap-subdir-test 2024-03-04T07:39:32.009 INFO:tasks.workunit.client.0.smithi113.stdout:./272/.snap/snap-subdir-test 2024-03-04T07:39:32.020 INFO:tasks.workunit.client.0.smithi113.stdout:./100/.snap/snap-subdir-test 2024-03-04T07:39:32.031 INFO:tasks.workunit.client.0.smithi113.stdout:./134/.snap/snap-subdir-test 2024-03-04T07:39:32.040 INFO:tasks.workunit.client.0.smithi113.stdout:./200/.snap/snap-subdir-test 2024-03-04T07:39:32.051 INFO:tasks.workunit.client.0.smithi113.stdout:./115/.snap/snap-subdir-test 2024-03-04T07:39:32.061 INFO:tasks.workunit.client.0.smithi113.stdout:./284/.snap/snap-subdir-test 2024-03-04T07:39:32.070 INFO:tasks.workunit.client.0.smithi113.stdout:./280/.snap/snap-subdir-test 2024-03-04T07:39:32.080 INFO:tasks.workunit.client.0.smithi113.stdout:./116/.snap/snap-subdir-test 2024-03-04T07:39:32.141 INFO:tasks.workunit.client.0.smithi113.stdout:./245/.snap/snap-subdir-test 2024-03-04T07:39:32.151 INFO:tasks.workunit.client.0.smithi113.stdout:./195/.snap/snap-subdir-test 2024-03-04T07:39:32.163 INFO:tasks.workunit.client.0.smithi113.stdout:./138/.snap/snap-subdir-test 2024-03-04T07:39:32.173 INFO:tasks.workunit.client.0.smithi113.stdout:./135/.snap/snap-subdir-test 2024-03-04T07:39:32.182 INFO:tasks.workunit.client.0.smithi113.stdout:./117/.snap/snap-subdir-test 2024-03-04T07:39:32.192 INFO:tasks.workunit.client.0.smithi113.stdout:./241/.snap/snap-subdir-test 2024-03-04T07:39:32.204 INFO:tasks.workunit.client.0.smithi113.stdout:./143/.snap/snap-subdir-test 2024-03-04T07:39:32.213 INFO:tasks.workunit.client.0.smithi113.stdout:./180/.snap/snap-subdir-test 2024-03-04T07:39:32.223 INFO:tasks.workunit.client.0.smithi113.stdout:./126/.snap/snap-subdir-test 2024-03-04T07:39:32.234 INFO:tasks.workunit.client.0.smithi113.stdout:./161/.snap/snap-subdir-test 2024-03-04T07:39:32.244 INFO:tasks.workunit.client.0.smithi113.stdout:./204/.snap/snap-subdir-test 2024-03-04T07:39:32.256 INFO:tasks.workunit.client.0.smithi113.stdout:./176/.snap/snap-subdir-test 2024-03-04T07:39:32.263 INFO:tasks.workunit.client.0.smithi113.stdout:./255/.snap/snap-subdir-test 2024-03-04T07:39:32.272 INFO:tasks.workunit.client.0.smithi113.stdout:./215/.snap/snap-subdir-test 2024-03-04T07:39:32.275 INFO:tasks.workunit.client.0.smithi113.stdout:./111/.snap/snap-subdir-test 2024-03-04T07:39:32.958 INFO:tasks.workunit.client.0.smithi113.stdout:./238/.snap/snap-subdir-test 2024-03-04T07:39:32.971 INFO:tasks.workunit.client.0.smithi113.stdout:./191/.snap/snap-subdir-test 2024-03-04T07:39:32.983 INFO:tasks.workunit.client.0.smithi113.stdout:./184/.snap/snap-subdir-test 2024-03-04T07:39:32.995 INFO:tasks.workunit.client.0.smithi113.stdout:./192/.snap/snap-subdir-test 2024-03-04T07:39:33.119 INFO:tasks.workunit.client.0.smithi113.stdout:./103/.snap/snap-subdir-test 2024-03-04T07:39:33.801 INFO:tasks.workunit.client.0.smithi113.stdout:./128/.snap/snap-subdir-test 2024-03-04T07:39:33.815 INFO:tasks.workunit.client.0.smithi113.stdout:./229/.snap/snap-subdir-test 2024-03-04T07:39:33.830 INFO:tasks.workunit.client.0.smithi113.stdout:./144/.snap/snap-subdir-test 2024-03-04T07:39:33.846 INFO:tasks.workunit.client.0.smithi113.stdout:./146/.snap/snap-subdir-test 2024-03-04T07:39:33.860 INFO:tasks.workunit.client.0.smithi113.stdout:./227/.snap/snap-subdir-test 2024-03-04T07:39:33.870 INFO:tasks.workunit.client.0.smithi113.stdout:./278/.snap/snap-subdir-test 2024-03-04T07:39:33.883 INFO:tasks.workunit.client.0.smithi113.stdout:./141/.snap/snap-subdir-test 2024-03-04T07:39:33.894 INFO:tasks.workunit.client.0.smithi113.stdout:./234/.snap/snap-subdir-test 2024-03-04T07:39:33.904 INFO:tasks.workunit.client.0.smithi113.stdout:./225/.snap/snap-subdir-test 2024-03-04T07:39:33.914 INFO:tasks.workunit.client.0.smithi113.stdout:./182/.snap/snap-subdir-test 2024-03-04T07:39:33.924 INFO:tasks.workunit.client.0.smithi113.stdout:./107/.snap/snap-subdir-test 2024-03-04T07:39:33.936 INFO:tasks.workunit.client.0.smithi113.stdout:./178/.snap/snap-subdir-test 2024-03-04T07:39:33.946 INFO:tasks.workunit.client.0.smithi113.stdout:./266/.snap/snap-subdir-test 2024-03-04T07:39:33.957 INFO:tasks.workunit.client.0.smithi113.stdout:./240/.snap/snap-subdir-test 2024-03-04T07:39:33.966 INFO:tasks.workunit.client.0.smithi113.stdout:./156/.snap/snap-subdir-test 2024-03-04T07:39:33.977 INFO:tasks.workunit.client.0.smithi113.stdout:./131/.snap/snap-subdir-test 2024-03-04T07:39:34.010 INFO:tasks.workunit.client.0.smithi113.stdout:./123/.snap/snap-subdir-test 2024-03-04T07:39:34.019 INFO:tasks.workunit.client.0.smithi113.stdout:./160/.snap/snap-subdir-test 2024-03-04T07:39:34.032 INFO:tasks.workunit.client.0.smithi113.stdout:./165/.snap/snap-subdir-test 2024-03-04T07:39:34.044 INFO:tasks.workunit.client.0.smithi113.stdout:./101/.snap/snap-subdir-test 2024-03-04T07:39:34.055 INFO:tasks.workunit.client.0.smithi113.stdout:./148/.snap/snap-subdir-test 2024-03-04T07:39:34.067 INFO:tasks.workunit.client.0.smithi113.stdout:./168/.snap/snap-subdir-test 2024-03-04T07:39:34.077 INFO:tasks.workunit.client.0.smithi113.stdout:./179/.snap/snap-subdir-test 2024-03-04T07:39:34.088 INFO:tasks.workunit.client.0.smithi113.stdout:./257/.snap/snap-subdir-test 2024-03-04T07:39:34.097 INFO:tasks.workunit.client.0.smithi113.stdout:./252/.snap/snap-subdir-test 2024-03-04T07:39:34.108 INFO:tasks.workunit.client.0.smithi113.stdout:./132/.snap/snap-subdir-test 2024-03-04T07:39:34.119 INFO:tasks.workunit.client.0.smithi113.stdout:./236/.snap/snap-subdir-test 2024-03-04T07:39:34.130 INFO:tasks.workunit.client.0.smithi113.stdout:./211/.snap/snap-subdir-test 2024-03-04T07:39:34.140 INFO:tasks.workunit.client.0.smithi113.stdout:./199/.snap/snap-subdir-test 2024-03-04T07:39:34.149 INFO:tasks.workunit.client.0.smithi113.stdout:./205/.snap/snap-subdir-test 2024-03-04T07:39:34.413 INFO:tasks.workunit.client.0.smithi113.stdout:./163/.snap/snap-subdir-test 2024-03-04T07:39:34.779 INFO:tasks.workunit.client.0.smithi113.stdout:./261/.snap/snap-subdir-test 2024-03-04T07:39:34.816 INFO:tasks.workunit.client.0.smithi113.stdout:./201/.snap/snap-subdir-test 2024-03-04T07:39:34.899 INFO:tasks.workunit.client.0.smithi113.stdout:./282/.snap/snap-subdir-test 2024-03-04T07:39:34.907 INFO:tasks.workunit.client.0.smithi113.stdout:./296/.snap/snap-subdir-test 2024-03-04T07:39:34.918 INFO:tasks.workunit.client.0.smithi113.stdout:./217/.snap/snap-subdir-test 2024-03-04T07:39:34.928 INFO:tasks.workunit.client.0.smithi113.stdout:./153/.snap/snap-subdir-test 2024-03-04T07:39:35.820 INFO:tasks.workunit.client.0.smithi113.stdout:./206/.snap/snap-subdir-test 2024-03-04T07:39:36.832 INFO:tasks.workunit.client.0.smithi113.stdout:./122/.snap/snap-subdir-test 2024-03-04T07:39:36.844 INFO:tasks.workunit.client.0.smithi113.stdout:./162/.snap/snap-subdir-test 2024-03-04T07:39:36.848 INFO:tasks.workunit.client.0.smithi113.stdout:./197/.snap/snap-subdir-test 2024-03-04T07:39:36.860 INFO:tasks.workunit.client.0.smithi113.stdout:./216/.snap/snap-subdir-test 2024-03-04T07:39:39.282 INFO:tasks.workunit.client.0.smithi113.stdout:./254/.snap/snap-subdir-test 2024-03-04T07:39:39.293 INFO:tasks.workunit.client.0.smithi113.stdout:./113/.snap/snap-subdir-test 2024-03-04T07:39:39.304 INFO:tasks.workunit.client.0.smithi113.stdout:./152/.snap/snap-subdir-test 2024-03-04T07:39:39.315 INFO:tasks.workunit.client.0.smithi113.stdout:./121/.snap/snap-subdir-test 2024-03-04T07:39:39.324 INFO:tasks.workunit.client.0.smithi113.stdout:./118/.snap/snap-subdir-test 2024-03-04T07:39:39.335 INFO:tasks.workunit.client.0.smithi113.stdout:./196/.snap/snap-subdir-test 2024-03-04T07:39:40.129 INFO:tasks.workunit.client.0.smithi113.stdout:./124/.snap/snap-subdir-test 2024-03-04T07:39:40.141 INFO:tasks.workunit.client.0.smithi113.stdout:./239/.snap/snap-subdir-test 2024-03-04T07:39:40.152 INFO:tasks.workunit.client.0.smithi113.stdout:./188/.snap/snap-subdir-test 2024-03-04T07:39:40.163 INFO:tasks.workunit.client.0.smithi113.stdout:./287/.snap/snap-subdir-test 2024-03-04T07:39:40.171 INFO:tasks.workunit.client.0.smithi113.stdout:./275/.snap/snap-subdir-test 2024-03-04T07:39:40.180 INFO:tasks.workunit.client.0.smithi113.stdout:./259/.snap/snap-subdir-test 2024-03-04T07:39:40.447 INFO:tasks.workunit.client.0.smithi113.stdout:./224/.snap/snap-subdir-test 2024-03-04T07:39:40.457 INFO:tasks.workunit.client.0.smithi113.stdout:./264/.snap/snap-subdir-test 2024-03-04T07:39:40.469 INFO:tasks.workunit.client.0.smithi113.stdout:./145/.snap/snap-subdir-test 2024-03-04T07:39:40.478 INFO:tasks.workunit.client.0.smithi113.stdout:./279/.snap/snap-subdir-test 2024-03-04T07:39:40.632 INFO:tasks.workunit.client.0.smithi113.stdout:./263/.snap/snap-subdir-test 2024-03-04T07:39:41.076 INFO:tasks.workunit.client.0.smithi113.stdout:./130/.snap/snap-subdir-test 2024-03-04T07:39:41.086 INFO:tasks.workunit.client.0.smithi113.stdout:./193/.snap/snap-subdir-test 2024-03-04T07:39:41.096 INFO:tasks.workunit.client.0.smithi113.stdout:./223/.snap/snap-subdir-test 2024-03-04T07:39:41.105 INFO:tasks.workunit.client.0.smithi113.stdout:./207/.snap/snap-subdir-test 2024-03-04T07:39:41.113 INFO:tasks.workunit.client.0.smithi113.stdout:./276/.snap/snap-subdir-test 2024-03-04T07:39:41.122 INFO:tasks.workunit.client.0.smithi113.stdout:./271/.snap/snap-subdir-test 2024-03-04T07:39:41.131 INFO:tasks.workunit.client.0.smithi113.stdout:./251/.snap/snap-subdir-test 2024-03-04T07:39:41.144 INFO:tasks.workunit.client.0.smithi113.stdout:./175/.snap/snap-subdir-test 2024-03-04T07:39:41.153 INFO:tasks.workunit.client.0.smithi113.stdout:./222/.snap/snap-subdir-test 2024-03-04T07:39:41.164 INFO:tasks.workunit.client.0.smithi113.stdout:./220/.snap/snap-subdir-test 2024-03-04T07:39:41.173 INFO:tasks.workunit.client.0.smithi113.stdout:./203/.snap/snap-subdir-test 2024-03-04T07:39:41.184 INFO:tasks.workunit.client.0.smithi113.stdout:./181/.snap/snap-subdir-test 2024-03-04T07:39:41.316 INFO:tasks.workunit.client.0.smithi113.stdout:./286/.snap/snap-subdir-test 2024-03-04T07:39:41.692 INFO:tasks.workunit.client.0.smithi113.stdout:./127/.snap/snap-subdir-test 2024-03-04T07:39:41.695 INFO:tasks.workunit.client.0.smithi113.stdout:./108/.snap/snap-subdir-test 2024-03-04T07:39:41.705 INFO:tasks.workunit.client.0.smithi113.stdout:./183/.snap/snap-subdir-test 2024-03-04T07:39:41.715 INFO:tasks.workunit.client.0.smithi113.stdout:./294/.snap/snap-subdir-test 2024-03-04T07:39:41.724 INFO:tasks.workunit.client.0.smithi113.stdout:./235/.snap/snap-subdir-test 2024-03-04T07:39:41.734 INFO:tasks.workunit.client.0.smithi113.stdout:./177/.snap/snap-subdir-test 2024-03-04T07:39:41.746 INFO:tasks.workunit.client.0.smithi113.stdout:./169/.snap/snap-subdir-test 2024-03-04T07:39:41.756 INFO:tasks.workunit.client.0.smithi113.stdout:./212/.snap/snap-subdir-test 2024-03-04T07:39:41.766 INFO:tasks.workunit.client.0.smithi113.stdout:./154/.snap/snap-subdir-test 2024-03-04T07:39:41.778 INFO:tasks.workunit.client.0.smithi113.stdout:./171/.snap/snap-subdir-test 2024-03-04T07:39:41.787 INFO:tasks.workunit.client.0.smithi113.stdout:./139/.snap/snap-subdir-test 2024-03-04T07:39:41.798 INFO:tasks.workunit.client.0.smithi113.stdout:./102/.snap/snap-subdir-test 2024-03-04T07:39:41.805 INFO:tasks.workunit.client.0.smithi113.stdout:./129/.snap/snap-subdir-test 2024-03-04T07:39:49.694 INFO:tasks.workunit.client.0.smithi113.stdout:Delete all the files and directories ... 2024-03-04T07:39:50.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/19' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/15' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/31' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/36' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/20' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/12' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/39' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/10' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/34' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/38' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/11' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/16' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/32' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/30' 2024-03-04T07:39:50.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/14' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/18' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/13' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/33' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/17' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/35' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './100/37' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './100' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/19' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/15' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/31' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/36' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/20' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/12' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/39' 2024-03-04T07:39:50.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/10' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/34' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/38' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/11' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/16' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/32' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/30' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/14' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/18' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/13' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/33' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/17' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/35' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './101/37' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './101' 2024-03-04T07:39:50.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/19' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/15' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/31' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/36' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/20' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/12' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/39' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/10' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/34' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/38' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/11' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/16' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/32' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/30' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/14' 2024-03-04T07:39:50.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/18' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/13' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/33' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/17' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/35' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './102/37' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './102' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/19' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/15' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/31' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/36' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/20' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/12' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/39' 2024-03-04T07:39:50.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/10' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/34' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/38' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/11' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/16' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/32' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/30' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/14' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/18' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/13' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/33' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/17' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/35' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './103/37' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './103' 2024-03-04T07:39:50.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/19' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/15' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/31' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/36' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/20' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/12' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/39' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/10' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/34' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/38' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/11' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/16' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/32' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/30' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/14' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/18' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/13' 2024-03-04T07:39:50.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/33' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/17' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/35' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './104/37' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './104' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/19' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/15' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/31' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/36' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/20' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/12' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/39' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/10' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/34' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/38' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/11' 2024-03-04T07:39:50.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/16' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/32' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/30' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/14' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/18' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/13' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/33' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/17' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/35' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './105/37' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './105' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/19' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/15' 2024-03-04T07:39:50.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/31' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/36' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/20' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/12' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/39' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/10' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/34' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/38' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/11' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/16' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/32' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/30' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/14' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/18' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/13' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/33' 2024-03-04T07:39:50.532 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/17' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/35' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './106/37' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './106' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/19' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/15' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/31' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/36' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/20' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/12' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/39' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/10' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/34' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/38' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/11' 2024-03-04T07:39:50.533 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/16' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/32' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/30' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/14' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/18' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/13' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/33' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/17' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/35' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './107/37' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './107' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/19' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/15' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/31' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/36' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/20' 2024-03-04T07:39:50.534 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/12' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/39' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/10' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/34' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/38' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/11' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/16' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/32' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/30' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/14' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/18' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/13' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/33' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/17' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/35' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed './108/37' 2024-03-04T07:39:50.535 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './108' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/19' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/15' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/31' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/36' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/20' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/12' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/39' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/10' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/34' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/38' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/11' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/16' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/32' 2024-03-04T07:39:50.536 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/30' 2024-03-04T07:39:53.157 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/14' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/18' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/13' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/33' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/17' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/35' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './109/37' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './109' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/19' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/15' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/31' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/36' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/20' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/12' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/39' 2024-03-04T07:39:53.158 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/10' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/34' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/38' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/11' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/16' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/32' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/30' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/14' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/18' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/13' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/33' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/17' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/35' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './110/37' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './110' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/19' 2024-03-04T07:39:53.159 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/15' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/31' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/36' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/20' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/12' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/39' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/10' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/34' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/38' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/11' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/16' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/32' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/30' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/14' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/18' 2024-03-04T07:39:53.160 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/13' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/33' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/17' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/35' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './111/37' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './111' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/19' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/15' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/31' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/36' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/20' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/12' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/39' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/10' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/34' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/38' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/11' 2024-03-04T07:39:53.161 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/16' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/32' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/30' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/14' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/18' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/13' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/33' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/17' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/35' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './112/37' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './112' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/19' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/15' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/31' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/36' 2024-03-04T07:39:53.162 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/20' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/12' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/39' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/10' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/34' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/38' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/11' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/16' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/32' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/30' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/14' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/18' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/13' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/33' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/17' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/35' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed './113/37' 2024-03-04T07:39:53.163 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './113' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/19' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/15' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/31' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/36' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/20' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/12' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/39' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/10' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/34' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/38' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/11' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/16' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/32' 2024-03-04T07:39:53.164 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/30' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/14' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/18' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/13' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/33' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/17' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/35' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './114/37' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './114' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/19' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/15' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/31' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/36' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/20' 2024-03-04T07:39:53.165 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/12' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/39' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/10' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/34' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/38' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/11' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/16' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/32' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/30' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/14' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/18' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/13' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/33' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/17' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/35' 2024-03-04T07:39:53.166 INFO:tasks.workunit.client.0.smithi113.stdout:removed './115/37' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './115' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/19' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/15' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/31' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/36' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/20' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/12' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/39' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/10' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/34' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/38' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/11' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/16' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/32' 2024-03-04T07:39:53.167 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/30' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/14' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/18' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/13' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/33' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/17' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/35' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './116/37' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './116' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/19' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/15' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/31' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/36' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/20' 2024-03-04T07:39:53.168 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/12' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/39' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/10' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/34' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/38' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/11' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/16' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/32' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/30' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/14' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/18' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/13' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/33' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/17' 2024-03-04T07:39:53.169 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/35' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './117/37' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './117' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/19' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/15' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/31' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/36' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/20' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/12' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/39' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/10' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/34' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/38' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/11' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/16' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/32' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/30' 2024-03-04T07:39:53.170 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/14' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/18' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/13' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/33' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/17' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/35' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './118/37' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './118' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/19' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/15' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/31' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/36' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/20' 2024-03-04T07:39:53.171 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/12' 2024-03-04T07:39:55.726 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:39:55.731 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:39:55.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/39' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/10' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/34' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/38' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/11' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/16' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/32' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/30' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/14' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/18' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/13' 2024-03-04T07:39:55.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/33' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/17' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/35' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './119/37' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './119' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/19' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/15' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/31' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/36' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/20' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/12' 2024-03-04T07:39:55.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/39' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/10' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/34' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/38' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/11' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/16' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/32' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/30' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/14' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/18' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/13' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/33' 2024-03-04T07:39:55.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/17' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/35' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './120/37' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './120' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/19' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/15' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/31' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/36' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/20' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/12' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/39' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/10' 2024-03-04T07:39:55.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/34' 2024-03-04T07:39:55.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/38' 2024-03-04T07:39:55.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/11' 2024-03-04T07:39:55.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/16' 2024-03-04T07:39:55.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/32' 2024-03-04T07:39:55.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/30' 2024-03-04T07:39:55.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/14' 2024-03-04T07:39:55.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/18' 2024-03-04T07:39:55.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/13' 2024-03-04T07:39:55.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/33' 2024-03-04T07:39:55.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/17' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/35' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './121/37' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './121' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/19' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/15' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/31' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/36' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/20' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/12' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/39' 2024-03-04T07:39:55.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/10' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/34' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/38' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/11' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/16' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/32' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/30' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/14' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/18' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/13' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/33' 2024-03-04T07:39:55.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/17' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/35' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './122/37' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './122' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/19' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/15' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/31' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/36' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/20' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/12' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/39' 2024-03-04T07:39:55.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/10' 2024-03-04T07:39:55.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/34' 2024-03-04T07:39:55.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/38' 2024-03-04T07:39:55.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/11' 2024-03-04T07:39:55.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/16' 2024-03-04T07:39:55.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/32' 2024-03-04T07:39:55.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/30' 2024-03-04T07:39:55.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/14' 2024-03-04T07:39:55.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/18' 2024-03-04T07:39:55.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/13' 2024-03-04T07:39:55.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/33' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/17' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/35' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './123/37' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './123' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/19' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/15' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/31' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/36' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/20' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/12' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/39' 2024-03-04T07:39:55.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/10' 2024-03-04T07:39:55.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/34' 2024-03-04T07:39:55.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/38' 2024-03-04T07:39:55.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/11' 2024-03-04T07:39:55.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/16' 2024-03-04T07:39:55.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/32' 2024-03-04T07:39:55.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/30' 2024-03-04T07:39:55.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/14' 2024-03-04T07:39:55.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/18' 2024-03-04T07:39:55.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/13' 2024-03-04T07:39:55.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/33' 2024-03-04T07:39:55.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/17' 2024-03-04T07:39:55.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/35' 2024-03-04T07:39:55.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './124/37' 2024-03-04T07:39:55.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './124' 2024-03-04T07:39:55.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/19' 2024-03-04T07:39:55.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/15' 2024-03-04T07:39:55.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/31' 2024-03-04T07:39:55.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/36' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/20' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/12' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/39' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/10' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/34' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/38' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/11' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/16' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/32' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/30' 2024-03-04T07:39:55.992 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/14' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/18' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/13' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/33' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/17' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/35' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './125/37' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './125' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/19' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/15' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/31' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/36' 2024-03-04T07:39:55.993 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/20' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/12' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/39' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/10' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/34' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/38' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/11' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/16' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/32' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/30' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/14' 2024-03-04T07:39:55.994 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/18' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/13' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/33' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/17' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/35' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './126/37' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './126' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/19' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/15' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/31' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/36' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/20' 2024-03-04T07:39:55.995 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/12' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/39' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/10' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/34' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/38' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/11' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/16' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/32' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/30' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/14' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/18' 2024-03-04T07:39:55.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/13' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/33' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/17' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/35' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './127/37' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './127' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/19' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/15' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/31' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/36' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/20' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/12' 2024-03-04T07:39:55.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/39' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/10' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/34' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/38' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/11' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/16' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/32' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/30' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/14' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/18' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/13' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/33' 2024-03-04T07:39:55.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/17' 2024-03-04T07:39:55.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/35' 2024-03-04T07:39:57.667 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:39:57.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.833+0000 7f4b90a00640 1 Processor -- start 2024-03-04T07:39:57.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.833+0000 7f4b90a00640 1 -- start start 2024-03-04T07:39:57.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.833+0000 7f4b90a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 0x7f4b8c0ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:57.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.833+0000 7f4b90a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8c0c5470 con 0x7f4b8c0703e0 2024-03-04T07:39:57.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.833+0000 7f4b90a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8c0cba40 con 0x7f4b8c152d00 2024-03-04T07:39:57.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.833+0000 7f4b90a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c0c5220 con 0x7f4b8c0ca0d0 2024-03-04T07:39:57.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.837+0000 7f4b8a800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4b8c0703e0 0x7f4b8c1631a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57548/0 (socket says 172.21.15.113:57548) 2024-03-04T07:39:57.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.837+0000 7f4b8a800640 1 -- 172.21.15.113:0/888617574 learned_addr learned my addr 172.21.15.113:0/888617574 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:39:57.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.837+0000 7f4b83e00640 1 --2- 172.21.15.113:0/888617574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 0x7f4b8c0ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:57.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.837+0000 7f4b89400640 1 -- 172.21.15.113:0/888617574 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261168925 0 0) 0x7f4b8c0c5470 con 0x7f4b8c0703e0 2024-03-04T07:39:57.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.837+0000 7f4b89400640 1 -- 172.21.15.113:0/888617574 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b70003530 con 0x7f4b8c0703e0 2024-03-04T07:39:57.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.837+0000 7f4b89400640 1 -- 172.21.15.113:0/888617574 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242420560 0 0) 0x7f4b8c0cba40 con 0x7f4b8c152d00 2024-03-04T07:39:57.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.837+0000 7f4b83e00640 1 -- 172.21.15.113:0/888617574 >> v1:172.21.15.136:6789/0 conn(0x7f4b8c152d00 legacy=0x7f4b8c0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:57.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.837+0000 7f4b83e00640 1 -- 172.21.15.113:0/888617574 >> v1:172.21.15.113:6789/0 conn(0x7f4b8c0703e0 legacy=0x7f4b8c1631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:57.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.837+0000 7f4b83e00640 1 -- 172.21.15.113:0/888617574 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c06f690 con 0x7f4b8c0ca0d0 2024-03-04T07:39:57.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b83e00640 1 --2- 172.21.15.113:0/888617574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 0x7f4b8c0ce3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b740097f0 tx=0x7f4b7401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58e05b65ffa0ff3f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:57.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b89400640 1 -- 172.21.15.113:0/888617574 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b740b0070 con 0x7f4b8c0ca0d0 2024-03-04T07:39:57.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b89400640 1 -- 172.21.15.113:0/888617574 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b740ab600 con 0x7f4b8c0ca0d0 2024-03-04T07:39:57.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b89400640 1 -- 172.21.15.113:0/888617574 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b740ab9b0 con 0x7f4b8c0ca0d0 2024-03-04T07:39:57.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b90a00640 1 -- 172.21.15.113:0/888617574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 msgr2=0x7f4b8c0ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:57.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b90a00640 1 --2- 172.21.15.113:0/888617574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 0x7f4b8c0ce3c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4b740097f0 tx=0x7f4b7401c450 comp rx=0 tx=0).stop 2024-03-04T07:39:57.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b90a00640 1 -- 172.21.15.113:0/888617574 shutdown_connections 2024-03-04T07:39:57.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b90a00640 1 --2- 172.21.15.113:0/888617574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 0x7f4b8c0ce3c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:57.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b90a00640 1 -- 172.21.15.113:0/888617574 >> 172.21.15.113:0/888617574 conn(0x7f4b8c066db0 msgr2=0x7f4b8c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:57.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b90a00640 1 -- 172.21.15.113:0/888617574 shutdown_connections 2024-03-04T07:39:57.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.841+0000 7f4b90a00640 1 -- 172.21.15.113:0/888617574 wait complete. 2024-03-04T07:39:57.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 Processor -- start 2024-03-04T07:39:57.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 -- start start 2024-03-04T07:39:57.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b8c0703e0 0x7f4b8c1f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:57.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 0x7f4b8c1f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:57.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b8c152d00 0x7f4b8c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:57.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c167260 con 0x7f4b8c0703e0 2024-03-04T07:39:57.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c1670e0 con 0x7f4b8c152d00 2024-03-04T07:39:57.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c1673e0 con 0x7f4b8c0ca0d0 2024-03-04T07:39:57.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b89e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b8c0703e0 0x7f4b8c1f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:57.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b89e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b8c0703e0 0x7f4b8c1f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57064/0 (socket says 172.21.15.113:57064) 2024-03-04T07:39:57.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b89e00640 1 -- 172.21.15.113:0/3338979430 learned_addr learned my addr 172.21.15.113:0/3338979430 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:57.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b83e00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 0x7f4b8c1f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:57.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b8a800640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b8c152d00 0x7f4b8c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:57.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b89e00640 1 -- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 msgr2=0x7f4b8c1f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:57.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b89e00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 0x7f4b8c1f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:57.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b89e00640 1 -- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b8c152d00 msgr2=0x7f4b8c155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:57.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b89e00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b8c152d00 0x7f4b8c155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:57.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b89e00640 1 -- 172.21.15.113:0/3338979430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c0cb9e0 con 0x7f4b8c0703e0 2024-03-04T07:39:57.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b89e00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b8c0703e0 0x7f4b8c1f5280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c440 tx=0x7f4b7c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:57.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b82000640 1 -- 172.21.15.113:0/3338979430 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c017070 con 0x7f4b8c0703e0 2024-03-04T07:39:57.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b82000640 1 -- 172.21.15.113:0/3338979430 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c003600 con 0x7f4b8c0703e0 2024-03-04T07:39:57.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b82000640 1 -- 172.21.15.113:0/3338979430 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0039f0 con 0x7f4b8c0703e0 2024-03-04T07:39:57.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 -- 172.21.15.113:0/3338979430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c06c5a0 con 0x7f4b8c0703e0 2024-03-04T07:39:57.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 -- 172.21.15.113:0/3338979430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c06c840 con 0x7f4b8c0703e0 2024-03-04T07:39:57.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.845+0000 7f4b90a00640 1 -- 172.21.15.113:0/3338979430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c075c00 con 0x7f4b8c0703e0 2024-03-04T07:39:57.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.849+0000 7f4b82000640 1 -- 172.21.15.113:0/3338979430 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b7c01a070 con 0x7f4b8c0703e0 2024-03-04T07:39:57.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.849+0000 7f4b82000640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b68066f60 0x7f4b68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:57.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.849+0000 7f4b82000640 1 -- 172.21.15.113:0/3338979430 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6731+0+0 (secure 0 0 0) 0x7f4b7c084870 con 0x7f4b8c0703e0 2024-03-04T07:39:57.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.849+0000 7f4b83e00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b68066f60 0x7f4b68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:57.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.849+0000 7f4b83e00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b68066f60 0x7f4b68069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b740ad000 tx=0x7f4b740a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:57.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:57.857+0000 7f4b82000640 1 -- 172.21.15.113:0/3338979430 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4b8c0cb9e0 con 0x7f4b8c0703e0 2024-03-04T07:39:58.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.109+0000 7f4b90a00640 1 -- 172.21.15.113:0/3338979430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b8c0701c0 con 0x7f4b8c0703e0 2024-03-04T07:39:58.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.109+0000 7f4b82000640 1 -- 172.21.15.113:0/3338979430 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6610 (secure 0 0 0) 0x7f4b7c054d00 con 0x7f4b8c0703e0 2024-03-04T07:39:58.113 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:39:58.113 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[5094,4938,4276]},"id":1}]} 2024-03-04T07:39:58.113 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-03-04T07:39:58.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 -- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b68066f60 msgr2=0x7f4b68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:58.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b68066f60 0x7f4b68069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b740ad000 tx=0x7f4b740a6000 comp rx=0 tx=0).stop 2024-03-04T07:39:58.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 -- 172.21.15.113:0/3338979430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b8c0703e0 msgr2=0x7f4b8c1f5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:58.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b8c0703e0 0x7f4b8c1f5280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c440 tx=0x7f4b7c00c900 comp rx=0 tx=0).stop 2024-03-04T07:39:58.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 -- 172.21.15.113:0/3338979430 shutdown_connections 2024-03-04T07:39:58.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b8c152d00 0x7f4b8c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b8c0ca0d0 0x7f4b8c1f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b68066f60 0x7f4b68069420 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 --2- 172.21.15.113:0/3338979430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b8c0703e0 0x7f4b8c1f5280 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 -- 172.21.15.113:0/3338979430 >> 172.21.15.113:0/3338979430 conn(0x7f4b8c066db0 msgr2=0x7f4b8c0ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:58.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 -- 172.21.15.113:0/3338979430 shutdown_connections 2024-03-04T07:39:58.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.113+0000 7f4b90a00640 1 -- 172.21.15.113:0/3338979430 wait complete. 2024-03-04T07:39:58.140 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=0) 2024-03-04T07:39:58.140 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-04T07:39:58.141 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:39:58.160 INFO:tasks.ceph.mds.b:Stopped 2024-03-04T07:39:58.160 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:39:58.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f5766a00640 1 Processor -- start 2024-03-04T07:39:58.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f5766a00640 1 -- start start 2024-03-04T07:39:58.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f5766a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f576006b060 0x7f57600c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:58.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f5766a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57600b2230 con 0x7f5760158210 2024-03-04T07:39:58.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f5766a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57600c7bb0 con 0x7f57600733f0 2024-03-04T07:39:58.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f5766a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5760072fb0 con 0x7f576006b060 2024-03-04T07:39:58.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5760158210 0x7f57600cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57554/0 (socket says 172.21.15.113:57554) 2024-03-04T07:39:58.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575fe00640 1 -- 172.21.15.113:0/375286690 learned_addr learned my addr 172.21.15.113:0/375286690 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f5764a00640 1 --2- 172.21.15.113:0/375286690 >> v2:172.21.15.136:3300/0 conn(0x7f576006b060 0x7f57600c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290242879 0 0) 0x7f57600b2230 con 0x7f5760158210 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5744003530 con 0x7f5760158210 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3082532541 0 0) 0x7f5744003530 con 0x7f5760158210 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 >> v2:172.21.15.136:3300/0 conn(0x7f576006b060 msgr2=0x7f57600c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 --2- 172.21.15.113:0/375286690 >> v2:172.21.15.136:3300/0 conn(0x7f576006b060 0x7f57600c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 >> v1:172.21.15.136:6789/0 conn(0x7f57600733f0 legacy=0x7f57600ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600c7e50 con 0x7f5760158210 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f5750001680 con 0x7f5760158210 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 >> v1:172.21.15.113:6789/0 conn(0x7f5760158210 legacy=0x7f57600cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57440039c0 0x7f5744003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:58.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5744004810 0x7f5744004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5744005220 0x7f574400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5760072fb0 con 0x7f57440039c0 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574400c150 con 0x7f5744004810 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.301+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574400c350 con 0x7f5744005220 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f575fe00640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57440039c0 0x7f5744003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f5764a00640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5744005220 0x7f574400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f5757e00640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5744004810 0x7f5744004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f575fe00640 1 -- 172.21.15.113:0/375286690 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5744005220 msgr2=0x7f574400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f575fe00640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5744005220 0x7f574400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f575fe00640 1 -- 172.21.15.113:0/375286690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5744004810 msgr2=0x7f5744004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f575fe00640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5744004810 0x7f5744004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f575fe00640 1 -- 172.21.15.113:0/375286690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5744003530 con 0x7f57440039c0 2024-03-04T07:39:58.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f5764a00640 1 -- 172.21.15.113:0/375286690 reap_dead start 2024-03-04T07:39:58.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f575fe00640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57440039c0 0x7f5744003e00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f575000c210 tx=0x7f575001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e07debfd548356 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:58.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750007d40 con 0x7f57440039c0 2024-03-04T07:39:58.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57500a9070 con 0x7f57440039c0 2024-03-04T07:39:58.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.305+0000 7f575f400640 1 -- 172.21.15.113:0/375286690 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750005d80 con 0x7f57440039c0 2024-03-04T07:39:58.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.309+0000 7f5766a00640 1 -- 172.21.15.113:0/375286690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57440039c0 msgr2=0x7f5744003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:58.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.309+0000 7f5766a00640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57440039c0 0x7f5744003e00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f575000c210 tx=0x7f575001cc20 comp rx=0 tx=0).stop 2024-03-04T07:39:58.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.309+0000 7f5766a00640 1 -- 172.21.15.113:0/375286690 shutdown_connections 2024-03-04T07:39:58.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.309+0000 7f5766a00640 1 --2- 172.21.15.113:0/375286690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57440039c0 0x7f5744003e00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.309+0000 7f5766a00640 1 -- 172.21.15.113:0/375286690 >> 172.21.15.113:0/375286690 conn(0x7f57600671b0 msgr2=0x7f576006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:58.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.309+0000 7f5766a00640 1 -- 172.21.15.113:0/375286690 shutdown_connections 2024-03-04T07:39:58.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.309+0000 7f5766a00640 1 -- 172.21.15.113:0/375286690 wait complete. 2024-03-04T07:39:58.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.309+0000 7f5766a00640 1 Processor -- start 2024-03-04T07:39:58.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.309+0000 7f5766a00640 1 -- start start 2024-03-04T07:39:58.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5766a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57440039c0 0x7f57601513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:58.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5766a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f576015b530 0x7f57601e9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:58.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5766a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f576014f7e0 0x7f57600b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:58.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5766a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576006cff0 con 0x7f576014f7e0 2024-03-04T07:39:58.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5766a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576006ce70 con 0x7f57440039c0 2024-03-04T07:39:58.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5766a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57600c93d0 con 0x7f576015b530 2024-03-04T07:39:58.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5764a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f576014f7e0 0x7f57600b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:58.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5764a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f576014f7e0 0x7f57600b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57082/0 (socket says 172.21.15.113:57082) 2024-03-04T07:39:58.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5764a00640 1 -- 172.21.15.113:0/3199940413 learned_addr learned my addr 172.21.15.113:0/3199940413 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:39:58.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5757e00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f576015b530 0x7f57601e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:58.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f575fe00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57440039c0 0x7f57601513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:58.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5764a00640 1 -- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f576015b530 msgr2=0x7f57601e9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:58.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5764a00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f576015b530 0x7f57601e9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5764a00640 1 -- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57440039c0 msgr2=0x7f57601513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:58.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5764a00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57440039c0 0x7f57601513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5764a00640 1 -- 172.21.15.113:0/3199940413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600aef90 con 0x7f576014f7e0 2024-03-04T07:39:58.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5757e00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f576015b530 0x7f57601e9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:39:58.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5764a00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f576014f7e0 0x7f57600b0880 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5758007d10 tx=0x7f57580115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:58.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f575cc00640 1 -- 172.21.15.113:0/3199940413 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758011de0 con 0x7f576014f7e0 2024-03-04T07:39:58.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f575cc00640 1 -- 172.21.15.113:0/3199940413 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575801b070 con 0x7f576014f7e0 2024-03-04T07:39:58.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5766a00640 1 -- 172.21.15.113:0/3199940413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57601f3d50 con 0x7f576014f7e0 2024-03-04T07:39:58.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f575cc00640 1 -- 172.21.15.113:0/3199940413 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580039a0 con 0x7f576014f7e0 2024-03-04T07:39:58.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5766a00640 1 -- 172.21.15.113:0/3199940413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57600aecb0 con 0x7f576014f7e0 2024-03-04T07:39:58.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.313+0000 7f5766a00640 1 -- 172.21.15.113:0/3199940413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5760075c00 con 0x7f576014f7e0 2024-03-04T07:39:58.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.317+0000 7f575cc00640 1 -- 172.21.15.113:0/3199940413 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5758003b40 con 0x7f576014f7e0 2024-03-04T07:39:58.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.317+0000 7f575cc00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f573c066f60 0x7f573c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:39:58.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.317+0000 7f575cc00640 1 -- 172.21.15.113:0/3199940413 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6731+0+0 (secure 0 0 0) 0x7f5758085c00 con 0x7f576014f7e0 2024-03-04T07:39:58.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.317+0000 7f575fe00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f573c066f60 0x7f573c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:39:58.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.317+0000 7f575fe00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f573c066f60 0x7f573c069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f57500ab000 tx=0x7f57500b0040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:39:58.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.325+0000 7f575cc00640 1 -- 172.21.15.113:0/3199940413 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f575808c080 con 0x7f576014f7e0 2024-03-04T07:39:58.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.693+0000 7f5766a00640 1 -- 172.21.15.113:0/3199940413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5760151900 con 0x7f576014f7e0 2024-03-04T07:39:58.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.693+0000 7f575cc00640 1 -- 172.21.15.113:0/3199940413 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6610 (secure 0 0 0) 0x7f5758056810 con 0x7f576014f7e0 2024-03-04T07:39:58.697 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:39:58.697 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[5094,4938,4276]},"id":1}]} 2024-03-04T07:39:58.697 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-03-04T07:39:58.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 -- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f573c066f60 msgr2=0x7f573c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:58.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f573c066f60 0x7f573c069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f57500ab000 tx=0x7f57500b0040 comp rx=0 tx=0).stop 2024-03-04T07:39:58.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 -- 172.21.15.113:0/3199940413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f576014f7e0 msgr2=0x7f57600b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:39:58.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f576014f7e0 0x7f57600b0880 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5758007d10 tx=0x7f57580115a0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 -- 172.21.15.113:0/3199940413 shutdown_connections 2024-03-04T07:39:58.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f573c066f60 0x7f573c069420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f576014f7e0 0x7f57600b0880 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f576015b530 0x7f57601e9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 --2- 172.21.15.113:0/3199940413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57440039c0 0x7f57601513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:39:58.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 -- 172.21.15.113:0/3199940413 >> 172.21.15.113:0/3199940413 conn(0x7f57600671b0 msgr2=0x7f5760155a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:39:58.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 -- 172.21.15.113:0/3199940413 shutdown_connections 2024-03-04T07:39:58.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:39:58.697+0000 7f5766a00640 1 -- 172.21.15.113:0/3199940413 wait complete. 2024-03-04T07:39:58.726 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:40:00.727 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:00.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34ffa00640 1 Processor -- start 2024-03-04T07:40:00.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34ffa00640 1 -- start start 2024-03-04T07:40:00.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34ffa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34f806ff20 0x7f34f815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:00.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34ffa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f34f80c8b90 0x7f34f815c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:00.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34ffa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f8152e30 con 0x7f34f8152a50 2024-03-04T07:40:00.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34ffa00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f80c31b0 con 0x7f34f80c8b90 2024-03-04T07:40:00.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34ffa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f806f840 con 0x7f34f806ff20 2024-03-04T07:40:00.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34fd000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f34f8152a50 0x7f34f80ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57556/0 (socket says 172.21.15.113:57556) 2024-03-04T07:40:00.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34fd000640 1 -- 172.21.15.113:0/2497914231 learned_addr learned my addr 172.21.15.113:0/2497914231 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:00.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34fda00640 1 --2- 172.21.15.113:0/2497914231 >> v2:172.21.15.113:3300/0 conn(0x7f34f80c8b90 0x7f34f815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:00.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34efe00640 1 --2- 172.21.15.113:0/2497914231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34f806ff20 0x7f34f815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:00.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34fda00640 1 -- 172.21.15.113:0/2497914231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34f806ff20 msgr2=0x7f34f815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:00.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34f7e00640 1 -- 172.21.15.113:0/2497914231 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244542314 0 0) 0x7f34f8152e30 con 0x7f34f8152a50 2024-03-04T07:40:00.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34fda00640 1 --2- 172.21.15.113:0/2497914231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34f806ff20 0x7f34f815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:00.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34fda00640 1 -- 172.21.15.113:0/2497914231 >> v1:172.21.15.113:6789/0 conn(0x7f34f8152a50 legacy=0x7f34f80ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:00.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.897+0000 7f34fda00640 1 -- 172.21.15.113:0/2497914231 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f806f280 con 0x7f34f80c8b90 2024-03-04T07:40:00.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34fda00640 1 --2- 172.21.15.113:0/2497914231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 0x7f34f815c310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009d90 tx=0x7f34f001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a392f2c37ab6ebfd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:00.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34f7e00640 1 -- 172.21.15.113:0/2497914231 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f00b2070 con 0x7f34f80c8b90 2024-03-04T07:40:00.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34f7e00640 1 -- 172.21.15.113:0/2497914231 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f00a59c0 con 0x7f34f80c8b90 2024-03-04T07:40:00.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34f7e00640 1 -- 172.21.15.113:0/2497914231 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f00a5d90 con 0x7f34f80c8b90 2024-03-04T07:40:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34ffa00640 1 -- 172.21.15.113:0/2497914231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 msgr2=0x7f34f815c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34ffa00640 1 --2- 172.21.15.113:0/2497914231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 0x7f34f815c310 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009d90 tx=0x7f34f001c8e0 comp rx=0 tx=0).stop 2024-03-04T07:40:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34ffa00640 1 -- 172.21.15.113:0/2497914231 shutdown_connections 2024-03-04T07:40:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34ffa00640 1 --2- 172.21.15.113:0/2497914231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 0x7f34f815c310 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34ffa00640 1 --2- 172.21.15.113:0/2497914231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34f806ff20 0x7f34f815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:00.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34ffa00640 1 -- 172.21.15.113:0/2497914231 >> 172.21.15.113:0/2497914231 conn(0x7f34f80734d0 msgr2=0x7f34f8066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:00.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34ffa00640 1 -- 172.21.15.113:0/2497914231 shutdown_connections 2024-03-04T07:40:00.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.905+0000 7f34ffa00640 1 -- 172.21.15.113:0/2497914231 wait complete. 2024-03-04T07:40:00.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34ffa00640 1 Processor -- start 2024-03-04T07:40:00.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34ffa00640 1 -- start start 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34ffa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34f806ff20 0x7f34f8162770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34ffa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 0x7f34f815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34ffa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34f8152a50 0x7f34f80ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34ffa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f80ce9b0 con 0x7f34f80c8b90 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34ffa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f80ce830 con 0x7f34f806ff20 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34ffa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f8165880 con 0x7f34f8152a50 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34efe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 0x7f34f815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34efe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 0x7f34f815ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57106/0 (socket says 172.21.15.113:57106) 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34efe00640 1 -- 172.21.15.113:0/3810195274 learned_addr learned my addr 172.21.15.113:0/3810195274 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34fd000640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34f806ff20 0x7f34f8162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34fda00640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34f8152a50 0x7f34f80ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34efe00640 1 -- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34f8152a50 msgr2=0x7f34f80ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34efe00640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34f8152a50 0x7f34f80ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:00.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34efe00640 1 -- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34f806ff20 msgr2=0x7f34f8162770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:00.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34efe00640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34f806ff20 0x7f34f8162770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:00.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34efe00640 1 -- 172.21.15.113:0/3810195274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f81611f0 con 0x7f34f80c8b90 2024-03-04T07:40:00.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34fd000640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34f806ff20 0x7f34f8162770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:00.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34efe00640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 0x7f34f815ab30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f34e000e960 tx=0x7f34e000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:00.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34f5600640 1 -- 172.21.15.113:0/3810195274 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0017070 con 0x7f34f80c8b90 2024-03-04T07:40:00.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34f5600640 1 -- 172.21.15.113:0/3810195274 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e000ca00 con 0x7f34f80c8b90 2024-03-04T07:40:00.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34ffa00640 1 -- 172.21.15.113:0/3810195274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f806d740 con 0x7f34f80c8b90 2024-03-04T07:40:00.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34f5600640 1 -- 172.21.15.113:0/3810195274 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e000cdb0 con 0x7f34f80c8b90 2024-03-04T07:40:00.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.909+0000 7f34ffa00640 1 -- 172.21.15.113:0/3810195274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f8161770 con 0x7f34f80c8b90 2024-03-04T07:40:00.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.913+0000 7f34ffa00640 1 -- 172.21.15.113:0/3810195274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f8075c00 con 0x7f34f80c8b90 2024-03-04T07:40:00.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.913+0000 7f34f5600640 1 -- 172.21.15.113:0/3810195274 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34e001a070 con 0x7f34f80c8b90 2024-03-04T07:40:00.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.913+0000 7f34f5600640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f34d4066f90 0x7f34d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:00.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.913+0000 7f34f5600640 1 -- 172.21.15.113:0/3810195274 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6319+0+0 (secure 0 0 0) 0x7f34e0085760 con 0x7f34f80c8b90 2024-03-04T07:40:00.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.913+0000 7f34fd000640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f34d4066f90 0x7f34d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:00.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.913+0000 7f34fd000640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f34d4066f90 0x7f34d4069450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34e8000f70 tx=0x7f34e8007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:00.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:00.917+0000 7f34f5600640 1 -- 172.21.15.113:0/3810195274 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f34e00563e0 con 0x7f34f80c8b90 2024-03-04T07:40:01.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.113+0000 7f34ffa00640 1 -- 172.21.15.113:0/3810195274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34f80cc310 con 0x7f34f80c8b90 2024-03-04T07:40:01.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.113+0000 7f34f5600640 1 -- 172.21.15.113:0/3810195274 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6610 (secure 0 0 0) 0x7f34e0055ce0 con 0x7f34f80c8b90 2024-03-04T07:40:01.117 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:01.117 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[5094,4938,4276]},"id":1}]} 2024-03-04T07:40:01.117 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-03-04T07:40:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 -- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f34d4066f90 msgr2=0x7f34d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f34d4066f90 0x7f34d4069450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34e8000f70 tx=0x7f34e8007cb0 comp rx=0 tx=0).stop 2024-03-04T07:40:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 -- 172.21.15.113:0/3810195274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 msgr2=0x7f34f815ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 0x7f34f815ab30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f34e000e960 tx=0x7f34e000ee20 comp rx=0 tx=0).stop 2024-03-04T07:40:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 -- 172.21.15.113:0/3810195274 shutdown_connections 2024-03-04T07:40:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f34f8152a50 0x7f34f80ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f34d4066f90 0x7f34d4069450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f34f80c8b90 0x7f34f815ab30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 --2- 172.21.15.113:0/3810195274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34f806ff20 0x7f34f8162770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:01.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 -- 172.21.15.113:0/3810195274 >> 172.21.15.113:0/3810195274 conn(0x7f34f80734d0 msgr2=0x7f34f80c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 -- 172.21.15.113:0/3810195274 shutdown_connections 2024-03-04T07:40:01.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:01.117+0000 7f34ffa00640 1 -- 172.21.15.113:0/3810195274 wait complete. 2024-03-04T07:40:01.147 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:40:03.147 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d55800640 1 Processor -- start 2024-03-04T07:40:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d55800640 1 -- start start 2024-03-04T07:40:03.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d55800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6d50157df0 0x7f6d5006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:03.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d55800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d50070fd0 con 0x7f6d500c8a90 2024-03-04T07:40:03.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d55800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5006c6b0 con 0x7f6d500ce020 2024-03-04T07:40:03.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d55800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5015e090 con 0x7f6d50157df0 2024-03-04T07:40:03.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d4fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6d50157df0 0x7f6d5006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:03.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d4f400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6d500c8a90 0x7f6d5015e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57562/0 (socket says 172.21.15.113:57562) 2024-03-04T07:40:03.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d4f400640 1 -- 172.21.15.113:0/1321710058 learned_addr learned my addr 172.21.15.113:0/1321710058 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:03.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d4fe00640 1 -- 172.21.15.113:0/1321710058 >> v1:172.21.15.136:6789/0 conn(0x7f6d500ce020 legacy=0x7f6d501650b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d4fe00640 1 -- 172.21.15.113:0/1321710058 >> v1:172.21.15.113:6789/0 conn(0x7f6d500c8a90 legacy=0x7f6d5015e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.301+0000 7f6d4fe00640 1 -- 172.21.15.113:0/1321710058 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5006dfd0 con 0x7f6d50157df0 2024-03-04T07:40:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d4fe00640 1 --2- 172.21.15.113:0/1321710058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d50157df0 0x7f6d5006c170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d400097c0 tx=0x7f6d4001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af061088b23961ad server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:03.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d4ea00640 1 -- 172.21.15.113:0/1321710058 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d400b0070 con 0x7f6d50157df0 2024-03-04T07:40:03.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d4ea00640 1 -- 172.21.15.113:0/1321710058 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d400ab600 con 0x7f6d50157df0 2024-03-04T07:40:03.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d4ea00640 1 -- 172.21.15.113:0/1321710058 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d400ab9d0 con 0x7f6d50157df0 2024-03-04T07:40:03.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d55800640 1 -- 172.21.15.113:0/1321710058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d50157df0 msgr2=0x7f6d5006c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:03.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d55800640 1 --2- 172.21.15.113:0/1321710058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d50157df0 0x7f6d5006c170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d400097c0 tx=0x7f6d4001c4a0 comp rx=0 tx=0).stop 2024-03-04T07:40:03.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d55800640 1 -- 172.21.15.113:0/1321710058 shutdown_connections 2024-03-04T07:40:03.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d55800640 1 --2- 172.21.15.113:0/1321710058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d50157df0 0x7f6d5006c170 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:03.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d55800640 1 -- 172.21.15.113:0/1321710058 >> 172.21.15.113:0/1321710058 conn(0x7f6d50072f40 msgr2=0x7f6d500706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:03.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d55800640 1 -- 172.21.15.113:0/1321710058 shutdown_connections 2024-03-04T07:40:03.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.305+0000 7f6d55800640 1 -- 172.21.15.113:0/1321710058 wait complete. 2024-03-04T07:40:03.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d55800640 1 Processor -- start 2024-03-04T07:40:03.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d55800640 1 -- start start 2024-03-04T07:40:03.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d55800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d500c8a90 0x7f6d50155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:03.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d55800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d500ce020 0x7f6d50156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:03.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d55800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d50157df0 0x7f6d501f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:03.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d55800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500c30b0 con 0x7f6d500c8a90 2024-03-04T07:40:03.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d55800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5015ec30 con 0x7f6d500ce020 2024-03-04T07:40:03.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d55800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d500ce840 con 0x7f6d50157df0 2024-03-04T07:40:03.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d500c8a90 0x7f6d50155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:03.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d4fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d500c8a90 0x7f6d50155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57128/0 (socket says 172.21.15.113:57128) 2024-03-04T07:40:03.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d4fe00640 1 -- 172.21.15.113:0/1529642754 learned_addr learned my addr 172.21.15.113:0/1529642754 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:03.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d4f400640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d500ce020 0x7f6d50156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d54e00640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d50157df0 0x7f6d501f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d4fe00640 1 -- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d50157df0 msgr2=0x7f6d501f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d4fe00640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d50157df0 0x7f6d501f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d4fe00640 1 -- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d500ce020 msgr2=0x7f6d50156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d4fe00640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d500ce020 0x7f6d50156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d4fe00640 1 -- 172.21.15.113:0/1529642754 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d50158470 con 0x7f6d500c8a90 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d4fe00640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d500c8a90 0x7f6d50155cc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6d400097c0 tx=0x7f6d4001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d47e00640 1 -- 172.21.15.113:0/1529642754 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d400b0050 con 0x7f6d500c8a90 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.309+0000 7f6d47e00640 1 -- 172.21.15.113:0/1529642754 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d400bc070 con 0x7f6d500c8a90 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.313+0000 7f6d55800640 1 -- 172.21.15.113:0/1529642754 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d50152290 con 0x7f6d500c8a90 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.313+0000 7f6d47e00640 1 -- 172.21.15.113:0/1529642754 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d400b5a70 con 0x7f6d500c8a90 2024-03-04T07:40:03.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.313+0000 7f6d55800640 1 -- 172.21.15.113:0/1529642754 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d50152550 con 0x7f6d500c8a90 2024-03-04T07:40:03.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.313+0000 7f6d55800640 1 -- 172.21.15.113:0/1529642754 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d50075a80 con 0x7f6d500c8a90 2024-03-04T07:40:03.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.313+0000 7f6d47e00640 1 -- 172.21.15.113:0/1529642754 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d400ab480 con 0x7f6d500c8a90 2024-03-04T07:40:03.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.317+0000 7f6d47e00640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d30066f60 0x7f6d30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:03.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.317+0000 7f6d47e00640 1 -- 172.21.15.113:0/1529642754 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6319+0+0 (secure 0 0 0) 0x7f6d4011fb80 con 0x7f6d500c8a90 2024-03-04T07:40:03.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.317+0000 7f6d4f400640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d30066f60 0x7f6d30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:03.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.317+0000 7f6d4f400640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d30066f60 0x7f6d30069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d5015d810 tx=0x7f6d3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:03.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.321+0000 7f6d47e00640 1 -- 172.21.15.113:0/1529642754 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6d50158470 con 0x7f6d500c8a90 2024-03-04T07:40:03.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.529+0000 7f6d55800640 1 -- 172.21.15.113:0/1529642754 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d5006f8a0 con 0x7f6d500c8a90 2024-03-04T07:40:03.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.529+0000 7f6d47e00640 1 -- 172.21.15.113:0/1529642754 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6610 (secure 0 0 0) 0x7f6d40016090 con 0x7f6d500c8a90 2024-03-04T07:40:03.532 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:03.532 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[5094,4938,4276]},"id":1}]} 2024-03-04T07:40:03.533 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-03-04T07:40:03.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 -- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d30066f60 msgr2=0x7f6d30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:03.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d30066f60 0x7f6d30069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d5015d810 tx=0x7f6d3c009040 comp rx=0 tx=0).stop 2024-03-04T07:40:03.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 -- 172.21.15.113:0/1529642754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d500c8a90 msgr2=0x7f6d50155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:03.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d500c8a90 0x7f6d50155cc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6d400097c0 tx=0x7f6d4001ace0 comp rx=0 tx=0).stop 2024-03-04T07:40:03.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 -- 172.21.15.113:0/1529642754 shutdown_connections 2024-03-04T07:40:03.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d50157df0 0x7f6d501f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:03.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d500ce020 0x7f6d50156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:03.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d30066f60 0x7f6d30069420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:03.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 --2- 172.21.15.113:0/1529642754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d500c8a90 0x7f6d50155cc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:03.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 -- 172.21.15.113:0/1529642754 >> 172.21.15.113:0/1529642754 conn(0x7f6d50072f40 msgr2=0x7f6d50157130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:03.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 -- 172.21.15.113:0/1529642754 shutdown_connections 2024-03-04T07:40:03.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:03.533+0000 7f6d55800640 1 -- 172.21.15.113:0/1529642754 wait complete. 2024-03-04T07:40:03.555 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:40:05.556 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:05.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97b5a00640 1 Processor -- start 2024-03-04T07:40:05.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97b5a00640 1 -- start start 2024-03-04T07:40:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97b5a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97b00b20b0 0x7f97b0157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97b5a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b0157c50 0x7f97b006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97b5a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b00c1ec0 con 0x7f97b00c85f0 2024-03-04T07:40:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97b5a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b00c9b30 con 0x7f97b00b20b0 2024-03-04T07:40:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97b5a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00c12a0 con 0x7f97b0157c50 2024-03-04T07:40:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b0157c50 0x7f97b006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97aea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97b00b20b0 0x7f97b0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97aea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97b00b20b0 0x7f97b0157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:37652/0 (socket says 172.21.15.113:37652) 2024-03-04T07:40:05.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97a7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b0157c50 0x7f97b006b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:39992/0 (socket says 172.21.15.113:39992) 2024-03-04T07:40:05.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.692+0000 7f97aea00640 1 -- 172.21.15.113:0/1235013407 learned_addr learned my addr 172.21.15.113:0/1235013407 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:05.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.696+0000 7f97aea00640 1 -- 172.21.15.113:0/1235013407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b0157c50 msgr2=0x7f97b006b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:05.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.696+0000 7f97aea00640 1 --2- 172.21.15.113:0/1235013407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b0157c50 0x7f97b006b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:05.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.696+0000 7f97aea00640 1 -- 172.21.15.113:0/1235013407 >> v1:172.21.15.136:6789/0 conn(0x7f97b00c85f0 legacy=0x7f97b015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:05.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.696+0000 7f97aea00640 1 -- 172.21.15.113:0/1235013407 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b00c1580 con 0x7f97b00b20b0 2024-03-04T07:40:05.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.696+0000 7f97a7e00640 1 --2- 172.21.15.113:0/1235013407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b0157c50 0x7f97b006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:40:05.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.696+0000 7f97aea00640 1 --2- 172.21.15.113:0/1235013407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b00b20b0 0x7f97b0157710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f97a000aa10 tx=0x7f97a001ea10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40ba2bb27a2e0071 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:05.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.696+0000 7f97ae000640 1 -- 172.21.15.113:0/1235013407 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a00b4070 con 0x7f97b00b20b0 2024-03-04T07:40:05.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.696+0000 7f97ae000640 1 -- 172.21.15.113:0/1235013407 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a00a7970 con 0x7f97b00b20b0 2024-03-04T07:40:05.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.696+0000 7f97ae000640 1 -- 172.21.15.113:0/1235013407 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000b040 con 0x7f97b00b20b0 2024-03-04T07:40:05.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.700+0000 7f97b5a00640 1 -- 172.21.15.113:0/1235013407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b00b20b0 msgr2=0x7f97b0157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:05.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.700+0000 7f97b5a00640 1 --2- 172.21.15.113:0/1235013407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b00b20b0 0x7f97b0157710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f97a000aa10 tx=0x7f97a001ea10 comp rx=0 tx=0).stop 2024-03-04T07:40:05.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.700+0000 7f97b5a00640 1 -- 172.21.15.113:0/1235013407 shutdown_connections 2024-03-04T07:40:05.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.700+0000 7f97b5a00640 1 --2- 172.21.15.113:0/1235013407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b0157c50 0x7f97b006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:05.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.700+0000 7f97b5a00640 1 --2- 172.21.15.113:0/1235013407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b00b20b0 0x7f97b0157710 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:05.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.700+0000 7f97b5a00640 1 -- 172.21.15.113:0/1235013407 >> 172.21.15.113:0/1235013407 conn(0x7f97b006e820 msgr2=0x7f97b006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:05.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.700+0000 7f97b5a00640 1 -- 172.21.15.113:0/1235013407 shutdown_connections 2024-03-04T07:40:05.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.700+0000 7f97b5a00640 1 -- 172.21.15.113:0/1235013407 wait complete. 2024-03-04T07:40:05.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.700+0000 7f97b5a00640 1 Processor -- start 2024-03-04T07:40:05.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.700+0000 7f97b5a00640 1 -- start start 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97b5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b00b20b0 0x7f97b0142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97b5a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b00c85f0 0x7f97b01444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97b5a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b0157c50 0x7f97b0144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97b5a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00c1b80 con 0x7f97b00b20b0 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97b5a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b0156ec0 con 0x7f97b0157c50 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97b5a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b00c80c0 con 0x7f97b00c85f0 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97aea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b00b20b0 0x7f97b0142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97aea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b00b20b0 0x7f97b0142970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57138/0 (socket says 172.21.15.113:57138) 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97aea00640 1 -- 172.21.15.113:0/2037632344 learned_addr learned my addr 172.21.15.113:0/2037632344 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97a7e00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b00c85f0 0x7f97b01444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97af400640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b0157c50 0x7f97b0144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97aea00640 1 -- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b00c85f0 msgr2=0x7f97b01444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97aea00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b00c85f0 0x7f97b01444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:05.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97aea00640 1 -- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b0157c50 msgr2=0x7f97b0144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:05.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97aea00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b0157c50 0x7f97b0144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:05.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97aea00640 1 -- 172.21.15.113:0/2037632344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b01efbb0 con 0x7f97b00b20b0 2024-03-04T07:40:05.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97aea00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b00b20b0 0x7f97b0142970 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f97a000aa10 tx=0x7f97a001d250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:05.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97a6a00640 1 -- 172.21.15.113:0/2037632344 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a00bf070 con 0x7f97b00b20b0 2024-03-04T07:40:05.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97a6a00640 1 -- 172.21.15.113:0/2037632344 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a00af660 con 0x7f97b00b20b0 2024-03-04T07:40:05.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97a6a00640 1 -- 172.21.15.113:0/2037632344 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a00b4040 con 0x7f97b00b20b0 2024-03-04T07:40:05.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97b5a00640 1 -- 172.21.15.113:0/2037632344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b01efe90 con 0x7f97b00b20b0 2024-03-04T07:40:05.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97b5a00640 1 -- 172.21.15.113:0/2037632344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b01f0450 con 0x7f97b00b20b0 2024-03-04T07:40:05.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97b5a00640 1 -- 172.21.15.113:0/2037632344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b0075c00 con 0x7f97b00b20b0 2024-03-04T07:40:05.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.704+0000 7f97a6a00640 1 -- 172.21.15.113:0/2037632344 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97a00afa50 con 0x7f97b00b20b0 2024-03-04T07:40:05.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.708+0000 7f97a6a00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f90 0x7f978c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:05.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.708+0000 7f97a6a00640 1 -- 172.21.15.113:0/2037632344 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6319+0+0 (secure 0 0 0) 0x7f97a0121280 con 0x7f97b00b20b0 2024-03-04T07:40:05.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.708+0000 7f97a7e00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f90 0x7f978c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:05.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.708+0000 7f97a7e00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f90 0x7f978c069450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f97b0143330 tx=0x7f9798008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:05.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.712+0000 7f97a6a00640 1 -- 172.21.15.113:0/2037632344 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f97a00c8080 con 0x7f97b00b20b0 2024-03-04T07:40:05.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.904+0000 7f97b5a00640 1 -- 172.21.15.113:0/2037632344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97b00c83d0 con 0x7f97b00b20b0 2024-03-04T07:40:05.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.904+0000 7f97a6a00640 1 -- 172.21.15.113:0/2037632344 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6610 (secure 0 0 0) 0x7f97a0018630 con 0x7f97b00b20b0 2024-03-04T07:40:05.909 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:05.909 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[5094,4938,4276]},"id":1}]} 2024-03-04T07:40:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-03-04T07:40:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 -- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f90 msgr2=0x7f978c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f90 0x7f978c069450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f97b0143330 tx=0x7f9798008040 comp rx=0 tx=0).stop 2024-03-04T07:40:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 -- 172.21.15.113:0/2037632344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b00b20b0 msgr2=0x7f97b0142970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b00b20b0 0x7f97b0142970 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f97a000aa10 tx=0x7f97a001d250 comp rx=0 tx=0).stop 2024-03-04T07:40:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 -- 172.21.15.113:0/2037632344 shutdown_connections 2024-03-04T07:40:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97b0157c50 0x7f97b0144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97b00c85f0 0x7f97b01444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f978c066f90 0x7f978c069450 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 --2- 172.21.15.113:0/2037632344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b00b20b0 0x7f97b0142970 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 -- 172.21.15.113:0/2037632344 >> 172.21.15.113:0/2037632344 conn(0x7f97b006e820 msgr2=0x7f97b00c7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 -- 172.21.15.113:0/2037632344 shutdown_connections 2024-03-04T07:40:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:05.908+0000 7f97b5a00640 1 -- 172.21.15.113:0/2037632344 wait complete. 2024-03-04T07:40:05.931 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:40:07.932 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:08.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe27d200640 1 Processor -- start 2024-03-04T07:40:08.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe27d200640 1 -- start start 2024-03-04T07:40:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe27d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe278155100 0x7fe2780c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe27d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe27815ad30 0x7fe27815b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe27d200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27815efd0 con 0x7fe27806b060 2024-03-04T07:40:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe27d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2780c8890 con 0x7fe27815ad30 2024-03-04T07:40:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe27d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27806b4a0 con 0x7fe278155100 2024-03-04T07:40:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe276800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe27815ad30 0x7fe27815b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe276800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe27815ad30 0x7fe27815b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58534/0 (socket says 172.21.15.113:58534) 2024-03-04T07:40:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe276800640 1 -- 172.21.15.113:0/583825334 learned_addr learned my addr 172.21.15.113:0/583825334 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe26d400640 1 --1- 172.21.15.113:0/583825334 >> v1:172.21.15.113:6789/0 conn(0x7fe27806b060 0x7fe2780cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55870/0 (socket says 172.21.15.113:55870) 2024-03-04T07:40:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe275e00640 1 --2- 172.21.15.113:0/583825334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe278155100 0x7fe2780c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:08.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe276800640 1 -- 172.21.15.113:0/583825334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe278155100 msgr2=0x7fe2780c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:08.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe276800640 1 --2- 172.21.15.113:0/583825334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe278155100 0x7fe2780c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:08.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe276800640 1 -- 172.21.15.113:0/583825334 >> v1:172.21.15.113:6789/0 conn(0x7fe27806b060 legacy=0x7fe2780cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:08.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe276800640 1 -- 172.21.15.113:0/583825334 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2780c7ef0 con 0x7fe27815ad30 2024-03-04T07:40:08.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.084+0000 7fe275400640 1 -- 172.21.15.113:0/583825334 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984678138 0 0) 0x7fe27815efd0 con 0x7fe27806b060 2024-03-04T07:40:08.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.088+0000 7fe276800640 1 --2- 172.21.15.113:0/583825334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27815ad30 0x7fe27815b140 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe270009870 tx=0x7fe27001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee685e4a526f27b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.088+0000 7fe275400640 1 -- 172.21.15.113:0/583825334 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700ab650 con 0x7fe27815ad30 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.088+0000 7fe275400640 1 -- 172.21.15.113:0/583825334 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2700b0070 con 0x7fe27815ad30 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.088+0000 7fe275400640 1 -- 172.21.15.113:0/583825334 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700aba10 con 0x7fe27815ad30 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.092+0000 7fe27d200640 1 -- 172.21.15.113:0/583825334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27815ad30 msgr2=0x7fe27815b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.092+0000 7fe27d200640 1 --2- 172.21.15.113:0/583825334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27815ad30 0x7fe27815b140 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe270009870 tx=0x7fe27001c560 comp rx=0 tx=0).stop 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.092+0000 7fe27d200640 1 -- 172.21.15.113:0/583825334 shutdown_connections 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.092+0000 7fe27d200640 1 --2- 172.21.15.113:0/583825334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27815ad30 0x7fe27815b140 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.092+0000 7fe27d200640 1 --2- 172.21.15.113:0/583825334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe278155100 0x7fe2780c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.092+0000 7fe27d200640 1 -- 172.21.15.113:0/583825334 >> 172.21.15.113:0/583825334 conn(0x7fe27806deb0 msgr2=0x7fe27806e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.092+0000 7fe27d200640 1 -- 172.21.15.113:0/583825334 shutdown_connections 2024-03-04T07:40:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.092+0000 7fe27d200640 1 -- 172.21.15.113:0/583825334 wait complete. 2024-03-04T07:40:08.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.092+0000 7fe27d200640 1 Processor -- start 2024-03-04T07:40:08.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.092+0000 7fe27d200640 1 -- start start 2024-03-04T07:40:08.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe27d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27806b060 0x7fe2781532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:08.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe27d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe278155100 0x7fe27815da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:08.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe27d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe27815ad30 0x7fe2780c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:08.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe27d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27815ebb0 con 0x7fe27806b060 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe27d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27815b680 con 0x7fe278155100 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe27d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27815ed30 con 0x7fe27815ad30 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe275e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27806b060 0x7fe2781532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe275e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27806b060 0x7fe2781532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58546/0 (socket says 172.21.15.113:58546) 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe275e00640 1 -- 172.21.15.113:0/196456621 learned_addr learned my addr 172.21.15.113:0/196456621 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe276800640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe27815ad30 0x7fe2780c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe26d400640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe278155100 0x7fe27815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe275e00640 1 -- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe27815ad30 msgr2=0x7fe2780c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe275e00640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe27815ad30 0x7fe2780c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe275e00640 1 -- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe278155100 msgr2=0x7fe27815da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe275e00640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe278155100 0x7fe27815da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe275e00640 1 -- 172.21.15.113:0/196456621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27815be30 con 0x7fe27806b060 2024-03-04T07:40:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe275e00640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27806b060 0x7fe2781532f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe268007f20 tx=0x7fe26800e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe26ea00640 1 -- 172.21.15.113:0/196456621 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268014620 con 0x7fe27806b060 2024-03-04T07:40:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe26ea00640 1 -- 172.21.15.113:0/196456621 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe268019070 con 0x7fe27806b060 2024-03-04T07:40:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe27d200640 1 -- 172.21.15.113:0/196456621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe278153910 con 0x7fe27806b060 2024-03-04T07:40:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe26ea00640 1 -- 172.21.15.113:0/196456621 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268014a00 con 0x7fe27806b060 2024-03-04T07:40:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe27d200640 1 -- 172.21.15.113:0/196456621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe278153ed0 con 0x7fe27806b060 2024-03-04T07:40:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe27d200640 1 -- 172.21.15.113:0/196456621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe278075c00 con 0x7fe27806b060 2024-03-04T07:40:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.096+0000 7fe26ea00640 1 -- 172.21.15.113:0/196456621 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe268014ba0 con 0x7fe27806b060 2024-03-04T07:40:08.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.100+0000 7fe26ea00640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe254066f90 0x7fe254069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:08.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.100+0000 7fe26ea00640 1 -- 172.21.15.113:0/196456621 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6319+0+0 (secure 0 0 0) 0x7fe268085ca0 con 0x7fe27806b060 2024-03-04T07:40:08.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.100+0000 7fe26d400640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe254066f90 0x7fe254069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:08.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.100+0000 7fe26d400640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe254066f90 0x7fe254069450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe27806f280 tx=0x7fe260007b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.104+0000 7fe26ea00640 1 -- 172.21.15.113:0/196456621 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe268056920 con 0x7fe27806b060 2024-03-04T07:40:08.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe27d200640 1 -- 172.21.15.113:0/196456621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe278152300 con 0x7fe27806b060 2024-03-04T07:40:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe26ea00640 1 -- 172.21.15.113:0/196456621 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6610 (secure 0 0 0) 0x7fe268056220 con 0x7fe27806b060 2024-03-04T07:40:08.303 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:08.303 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[5094,4938,4276]},"id":1}]} 2024-03-04T07:40:08.303 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe27d200640 1 -- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe254066f90 msgr2=0x7fe254069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe27d200640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe254066f90 0x7fe254069450 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe27806f280 tx=0x7fe260007b60 comp rx=0 tx=0).stop 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe27d200640 1 -- 172.21.15.113:0/196456621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27806b060 msgr2=0x7fe2781532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe27d200640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27806b060 0x7fe2781532f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe268007f20 tx=0x7fe26800e570 comp rx=0 tx=0).stop 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe27d200640 1 -- 172.21.15.113:0/196456621 shutdown_connections 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe27d200640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe27815ad30 0x7fe2780c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe27d200640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe278155100 0x7fe27815da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe27d200640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe254066f90 0x7fe254069450 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.300+0000 7fe27d200640 1 --2- 172.21.15.113:0/196456621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe27806b060 0x7fe2781532f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.304+0000 7fe27d200640 1 -- 172.21.15.113:0/196456621 >> 172.21.15.113:0/196456621 conn(0x7fe27806deb0 msgr2=0x7fe278159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.304+0000 7fe27d200640 1 -- 172.21.15.113:0/196456621 shutdown_connections 2024-03-04T07:40:08.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:08.304+0000 7fe27d200640 1 -- 172.21.15.113:0/196456621 wait complete. 2024-03-04T07:40:08.327 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:40:10.328 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:10.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.460+0000 7f7bcd600640 1 Processor -- start 2024-03-04T07:40:10.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.460+0000 7f7bcd600640 1 -- start start 2024-03-04T07:40:10.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.460+0000 7f7bcd600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bc81582e0 0x7f7bc80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:10.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.460+0000 7f7bcd600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc81586c0 con 0x7f7bc806ddf0 2024-03-04T07:40:10.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.460+0000 7f7bcd600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc806c210 con 0x7f7bc80b0ed0 2024-03-04T07:40:10.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.460+0000 7f7bcd600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc806c4f0 con 0x7f7bc81582e0 2024-03-04T07:40:10.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.460+0000 7f7bc7400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7bc806ddf0 0x7f7bc815af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55882/0 (socket says 172.21.15.113:55882) 2024-03-04T07:40:10.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.460+0000 7f7bc7400640 1 -- 172.21.15.113:0/1376614480 learned_addr learned my addr 172.21.15.113:0/1376614480 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:10.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.460+0000 7f7bc6a00640 1 --2- 172.21.15.113:0/1376614480 >> v2:172.21.15.136:3300/0 conn(0x7f7bc81582e0 0x7f7bc80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:10.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684908080 0 0) 0x7f7bc81586c0 con 0x7f7bc806ddf0 2024-03-04T07:40:10.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bac003530 con 0x7f7bc806ddf0 2024-03-04T07:40:10.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1428082750 0 0) 0x7f7bac003530 con 0x7f7bc806ddf0 2024-03-04T07:40:10.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 >> v2:172.21.15.136:3300/0 conn(0x7f7bc81582e0 msgr2=0x7f7bc80701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:10.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 --2- 172.21.15.113:0/1376614480 >> v2:172.21.15.136:3300/0 conn(0x7f7bc81582e0 0x7f7bc80701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:10.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 >> v1:172.21.15.136:6789/0 conn(0x7f7bc80b0ed0 legacy=0x7f7bc80ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:10.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc806f6a0 con 0x7f7bc806ddf0 2024-03-04T07:40:10.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f7bc0001a00 con 0x7f7bc806ddf0 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 >> v1:172.21.15.113:6789/0 conn(0x7f7bc806ddf0 legacy=0x7f7bc815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bac003a10 0x7f7bac003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bac004810 0x7f7bac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0051d0 0x7f7bac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc806c4f0 con 0x7f7bac004810 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac00c0d0 con 0x7f7bac003a10 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac00c2d0 con 0x7f7bac0051d0 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bbe000640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bac004810 0x7f7bac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc6a00640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bac003a10 0x7f7bac003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc7400640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0051d0 0x7f7bac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bbe000640 1 -- 172.21.15.113:0/1376614480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0051d0 msgr2=0x7f7bac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bbe000640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0051d0 0x7f7bac00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bbe000640 1 -- 172.21.15.113:0/1376614480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bac003a10 msgr2=0x7f7bac003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bbe000640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bac003a10 0x7f7bac003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:10.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bbe000640 1 -- 172.21.15.113:0/1376614480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac003530 con 0x7f7bac004810 2024-03-04T07:40:10.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.464+0000 7f7bc7400640 1 -- 172.21.15.113:0/1376614480 reap_dead start 2024-03-04T07:40:10.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bbe000640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bac004810 0x7f7bac004c90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0001780 tx=0x7f7bb001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804be3ad6b73a626 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:10.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb00013a0 con 0x7f7bac004810 2024-03-04T07:40:10.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb00052f0 con 0x7f7bac004810 2024-03-04T07:40:10.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bc6000640 1 -- 172.21.15.113:0/1376614480 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb0003840 con 0x7f7bac004810 2024-03-04T07:40:10.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bcd600640 1 -- 172.21.15.113:0/1376614480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bac004810 msgr2=0x7f7bac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:10.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bcd600640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bac004810 0x7f7bac004c90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0001780 tx=0x7f7bb001c400 comp rx=0 tx=0).stop 2024-03-04T07:40:10.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bcd600640 1 -- 172.21.15.113:0/1376614480 shutdown_connections 2024-03-04T07:40:10.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bcd600640 1 --2- 172.21.15.113:0/1376614480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bac004810 0x7f7bac004c90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:10.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bcd600640 1 -- 172.21.15.113:0/1376614480 >> 172.21.15.113:0/1376614480 conn(0x7f7bc8066db0 msgr2=0x7f7bc80671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:10.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bcd600640 1 -- 172.21.15.113:0/1376614480 shutdown_connections 2024-03-04T07:40:10.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.468+0000 7f7bcd600640 1 -- 172.21.15.113:0/1376614480 wait complete. 2024-03-04T07:40:10.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.472+0000 7f7bcd600640 1 Processor -- start 2024-03-04T07:40:10.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.472+0000 7f7bcd600640 1 -- start start 2024-03-04T07:40:10.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.472+0000 7f7bcd600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bac004810 0x7f7bc80aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:10.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bcd600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bc8152310 0x7f7bc80b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bcd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b0260 0x7f7bc81f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bcd600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc806c7d0 con 0x7f7bc80b0260 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bcd600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc806fc60 con 0x7f7bac004810 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bcd600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80701c0 con 0x7f7bc8152310 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bc7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b0260 0x7f7bc81f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bc7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b0260 0x7f7bc81f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58570/0 (socket says 172.21.15.113:58570) 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bc7400640 1 -- 172.21.15.113:0/3075050904 learned_addr learned my addr 172.21.15.113:0/3075050904 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bbe000640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bc8152310 0x7f7bc80b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bc6a00640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bac004810 0x7f7bc80aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bc7400640 1 -- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bc8152310 msgr2=0x7f7bc80b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bc7400640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bc8152310 0x7f7bc80b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bc7400640 1 -- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bac004810 msgr2=0x7f7bc80aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bc7400640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bac004810 0x7f7bc80aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:10.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bc7400640 1 -- 172.21.15.113:0/3075050904 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80b28e0 con 0x7f7bc80b0260 2024-03-04T07:40:10.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bc7400640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b0260 0x7f7bc81f3b30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7bc815bdc0 tx=0x7f7bc0007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:10.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bbf400640 1 -- 172.21.15.113:0/3075050904 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0010040 con 0x7f7bc80b0260 2024-03-04T07:40:10.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bbf400640 1 -- 172.21.15.113:0/3075050904 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc001a070 con 0x7f7bc80b0260 2024-03-04T07:40:10.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bbf400640 1 -- 172.21.15.113:0/3075050904 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0015830 con 0x7f7bc80b0260 2024-03-04T07:40:10.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bcd600640 1 -- 172.21.15.113:0/3075050904 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc80af780 con 0x7f7bc80b0260 2024-03-04T07:40:10.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bcd600640 1 -- 172.21.15.113:0/3075050904 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc80b25d0 con 0x7f7bc80b0260 2024-03-04T07:40:10.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bcd600640 1 -- 172.21.15.113:0/3075050904 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc8068eb0 con 0x7f7bc80b0260 2024-03-04T07:40:10.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.476+0000 7f7bbf400640 1 -- 172.21.15.113:0/3075050904 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7bc00159d0 con 0x7f7bc80b0260 2024-03-04T07:40:10.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.480+0000 7f7bbf400640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba4066f90 0x7f7ba4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:10.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.480+0000 7f7bbf400640 1 -- 172.21.15.113:0/3075050904 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6319+0+0 (secure 0 0 0) 0x7f7bc0087fc0 con 0x7f7bc80b0260 2024-03-04T07:40:10.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.480+0000 7f7bc6a00640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba4066f90 0x7f7ba4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:10.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.480+0000 7f7bc6a00640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba4066f90 0x7f7ba4069450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8050c80 tx=0x7f7bb8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:10.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.484+0000 7f7bbf400640 1 -- 172.21.15.113:0/3075050904 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7bc000f070 con 0x7f7bc80b0260 2024-03-04T07:40:10.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 -- 172.21.15.113:0/3075050904 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bc8075c00 con 0x7f7bc80b0260 2024-03-04T07:40:10.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bbf400640 1 -- 172.21.15.113:0/3075050904 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6610 (secure 0 0 0) 0x7f7bc0058d60 con 0x7f7bc80b0260 2024-03-04T07:40:10.679 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:10.679 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[5094,4938,4276]},"id":1}]} 2024-03-04T07:40:10.680 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-03-04T07:40:10.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 -- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba4066f90 msgr2=0x7f7ba4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:10.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba4066f90 0x7f7ba4069450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8050c80 tx=0x7f7bb8009190 comp rx=0 tx=0).stop 2024-03-04T07:40:10.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 -- 172.21.15.113:0/3075050904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b0260 msgr2=0x7f7bc81f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:10.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b0260 0x7f7bc81f3b30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7bc815bdc0 tx=0x7f7bc0007640 comp rx=0 tx=0).stop 2024-03-04T07:40:10.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 -- 172.21.15.113:0/3075050904 shutdown_connections 2024-03-04T07:40:10.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ba4066f90 0x7f7ba4069450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:10.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7bc80b0260 0x7f7bc81f3b30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:10.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bc8152310 0x7f7bc80b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:10.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 --2- 172.21.15.113:0/3075050904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7bac004810 0x7f7bc80aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:10.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 -- 172.21.15.113:0/3075050904 >> 172.21.15.113:0/3075050904 conn(0x7f7bc8066db0 msgr2=0x7f7bc80cb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:10.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 -- 172.21.15.113:0/3075050904 shutdown_connections 2024-03-04T07:40:10.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:10.676+0000 7f7bcd600640 1 -- 172.21.15.113:0/3075050904 wait complete. 2024-03-04T07:40:10.701 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:40:12.702 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:12.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87d0c00640 1 Processor -- start 2024-03-04T07:40:12.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87d0c00640 1 -- start start 2024-03-04T07:40:12.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87d0c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87cc154e20 0x7f87cc06ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:12.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87d0c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc15bac0 con 0x7f87cc0c18e0 2024-03-04T07:40:12.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87d0c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87cc06f2d0 con 0x7f87cc06d520 2024-03-04T07:40:12.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87d0c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87cc0c7e10 con 0x7f87cc154e20 2024-03-04T07:40:12.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87cb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87cc154e20 0x7f87cc06ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:12.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87cbe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f87cc0c18e0 0x7f87cc165400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55890/0 (socket says 172.21.15.113:55890) 2024-03-04T07:40:12.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87cbe00640 1 -- 172.21.15.113:0/1421448780 learned_addr learned my addr 172.21.15.113:0/1421448780 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:12.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87cb400640 1 -- 172.21.15.113:0/1421448780 >> v1:172.21.15.136:6789/0 conn(0x7f87cc06d520 legacy=0x7f87cc15b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:12.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87cb400640 1 -- 172.21.15.113:0/1421448780 >> v1:172.21.15.113:6789/0 conn(0x7f87cc0c18e0 legacy=0x7f87cc165400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:12.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87cb400640 1 -- 172.21.15.113:0/1421448780 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc0c80f0 con 0x7f87cc154e20 2024-03-04T07:40:12.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.852+0000 7f87ca000640 1 -- 172.21.15.113:0/1421448780 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542903368 0 0) 0x7f87cc15bac0 con 0x7f87cc0c18e0 2024-03-04T07:40:12.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.856+0000 7f87cb400640 1 --2- 172.21.15.113:0/1421448780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc154e20 0x7f87cc06ed90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0097c0 tx=0x7f87bc01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cf445445e3c9b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:12.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.856+0000 7f87ca000640 1 -- 172.21.15.113:0/1421448780 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0b5070 con 0x7f87cc154e20 2024-03-04T07:40:12.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.856+0000 7f87ca000640 1 -- 172.21.15.113:0/1421448780 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc0a8990 con 0x7f87cc154e20 2024-03-04T07:40:12.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.856+0000 7f87ca000640 1 -- 172.21.15.113:0/1421448780 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc0a8d60 con 0x7f87cc154e20 2024-03-04T07:40:12.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.860+0000 7f87d0c00640 1 -- 172.21.15.113:0/1421448780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc154e20 msgr2=0x7f87cc06ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:12.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.860+0000 7f87d0c00640 1 --2- 172.21.15.113:0/1421448780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc154e20 0x7f87cc06ed90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0097c0 tx=0x7f87bc01ec90 comp rx=0 tx=0).stop 2024-03-04T07:40:12.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.860+0000 7f87d0c00640 1 -- 172.21.15.113:0/1421448780 shutdown_connections 2024-03-04T07:40:12.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.860+0000 7f87d0c00640 1 --2- 172.21.15.113:0/1421448780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc154e20 0x7f87cc06ed90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:12.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.860+0000 7f87d0c00640 1 -- 172.21.15.113:0/1421448780 >> 172.21.15.113:0/1421448780 conn(0x7f87cc06e2a0 msgr2=0x7f87cc06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:12.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.860+0000 7f87d0c00640 1 -- 172.21.15.113:0/1421448780 shutdown_connections 2024-03-04T07:40:12.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.860+0000 7f87d0c00640 1 -- 172.21.15.113:0/1421448780 wait complete. 2024-03-04T07:40:12.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.860+0000 7f87d0c00640 1 Processor -- start 2024-03-04T07:40:12.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.860+0000 7f87d0c00640 1 -- start start 2024-03-04T07:40:12.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87d0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87cc06d520 0x7f87cc1f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:12.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87d0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc0c18e0 0x7f87cc072080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:12.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87d0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc154e20 0x7f87cc070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:12.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87d0c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc15ae90 con 0x7f87cc0c18e0 2024-03-04T07:40:12.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87d0c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc0c86b0 con 0x7f87cc154e20 2024-03-04T07:40:12.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87d0c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87cc06c230 con 0x7f87cc06d520 2024-03-04T07:40:12.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87caa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc0c18e0 0x7f87cc072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:12.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87caa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc0c18e0 0x7f87cc072080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58588/0 (socket says 172.21.15.113:58588) 2024-03-04T07:40:12.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87caa00640 1 -- 172.21.15.113:0/4046774585 learned_addr learned my addr 172.21.15.113:0/4046774585 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:12.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87cb400640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87cc06d520 0x7f87cc1f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:12.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87cbe00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc154e20 0x7f87cc070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:12.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87caa00640 1 -- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87cc06d520 msgr2=0x7f87cc1f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:12.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87caa00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87cc06d520 0x7f87cc1f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:12.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87caa00640 1 -- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc154e20 msgr2=0x7f87cc070b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:12.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87caa00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc154e20 0x7f87cc070b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:12.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87caa00640 1 -- 172.21.15.113:0/4046774585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc071610 con 0x7f87cc0c18e0 2024-03-04T07:40:12.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87cbe00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc154e20 0x7f87cc070b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:12.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87caa00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc0c18e0 0x7f87cc072080 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f87b800c530 tx=0x7f87b800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:12.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87c3400640 1 -- 172.21.15.113:0/4046774585 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8016020 con 0x7f87cc0c18e0 2024-03-04T07:40:12.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87c3400640 1 -- 172.21.15.113:0/4046774585 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b8005220 con 0x7f87cc0c18e0 2024-03-04T07:40:12.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87d0c00640 1 -- 172.21.15.113:0/4046774585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87cc150fb0 con 0x7f87cc0c18e0 2024-03-04T07:40:12.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87c3400640 1 -- 172.21.15.113:0/4046774585 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b8007030 con 0x7f87cc0c18e0 2024-03-04T07:40:12.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87d0c00640 1 -- 172.21.15.113:0/4046774585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc071880 con 0x7f87cc0c18e0 2024-03-04T07:40:12.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.864+0000 7f87d0c00640 1 -- 172.21.15.113:0/4046774585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc075a80 con 0x7f87cc0c18e0 2024-03-04T07:40:12.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.868+0000 7f87c3400640 1 -- 172.21.15.113:0/4046774585 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87b80071d0 con 0x7f87cc0c18e0 2024-03-04T07:40:12.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.868+0000 7f87c3400640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f87ac066f60 0x7f87ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:12.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.868+0000 7f87c3400640 1 -- 172.21.15.113:0/4046774585 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6319+0+0 (secure 0 0 0) 0x7f87b8084c20 con 0x7f87cc0c18e0 2024-03-04T07:40:12.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.868+0000 7f87cb400640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f87ac066f60 0x7f87ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:12.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.868+0000 7f87cb400640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f87ac066f60 0x7f87ac069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0b2000 tx=0x7f87bc0aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:12.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:12.876+0000 7f87c3400640 1 -- 172.21.15.113:0/4046774585 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f87b8020020 con 0x7f87cc0c18e0 2024-03-04T07:40:13.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.132+0000 7f87d0c00640 1 -- 172.21.15.113:0/4046774585 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87cc0c1200 con 0x7f87cc0c18e0 2024-03-04T07:40:13.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87c3400640 1 -- 172.21.15.113:0/4046774585 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6610 (secure 0 0 0) 0x7f87b80074c0 con 0x7f87cc0c18e0 2024-03-04T07:40:13.138 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:13.138 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[5094,4938,4276]},"id":1}]} 2024-03-04T07:40:13.138 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-03-04T07:40:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 -- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f87ac066f60 msgr2=0x7f87ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f87ac066f60 0x7f87ac069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0b2000 tx=0x7f87bc0aa070 comp rx=0 tx=0).stop 2024-03-04T07:40:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 -- 172.21.15.113:0/4046774585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc0c18e0 msgr2=0x7f87cc072080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc0c18e0 0x7f87cc072080 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f87b800c530 tx=0x7f87b800c9f0 comp rx=0 tx=0).stop 2024-03-04T07:40:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 -- 172.21.15.113:0/4046774585 shutdown_connections 2024-03-04T07:40:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87cc154e20 0x7f87cc070b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:13.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f87ac066f60 0x7f87ac069420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:13.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87cc0c18e0 0x7f87cc072080 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:13.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 --2- 172.21.15.113:0/4046774585 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87cc06d520 0x7f87cc1f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:13.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 -- 172.21.15.113:0/4046774585 >> 172.21.15.113:0/4046774585 conn(0x7f87cc06e2a0 msgr2=0x7f87cc1593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:13.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 -- 172.21.15.113:0/4046774585 shutdown_connections 2024-03-04T07:40:13.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:13.136+0000 7f87d0c00640 1 -- 172.21.15.113:0/4046774585 wait complete. 2024-03-04T07:40:13.168 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:40:15.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:15.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbce0e00640 1 Processor -- start 2024-03-04T07:40:15.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbce0e00640 1 -- start start 2024-03-04T07:40:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbce0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbcdc151840 0x7fbcdc0c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbce0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbcdc151c20 0x7fbcdc157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbce0e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcdc157da0 con 0x7fbcdc06d140 2024-03-04T07:40:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbce0e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc0c3440 con 0x7fbcdc151c20 2024-03-04T07:40:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbce0e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc158080 con 0x7fbcdc151840 2024-03-04T07:40:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbcd3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbcdc151c20 0x7fbcdc157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbcd3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbcdc151c20 0x7fbcdc157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58604/0 (socket says 172.21.15.113:58604) 2024-03-04T07:40:15.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbcd3e00640 1 -- 172.21.15.113:0/634358416 learned_addr learned my addr 172.21.15.113:0/634358416 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:15.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbcdb400640 1 --2- 172.21.15.113:0/634358416 >> v2:172.21.15.136:3300/0 conn(0x7fbcdc151840 0x7fbcdc0c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:15.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbcd3e00640 1 -- 172.21.15.113:0/634358416 >> v2:172.21.15.136:3300/0 conn(0x7fbcdc151840 msgr2=0x7fbcdc0c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:15.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbcd3e00640 1 --2- 172.21.15.113:0/634358416 >> v2:172.21.15.136:3300/0 conn(0x7fbcdc151840 0x7fbcdc0c1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:15.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbcd3e00640 1 -- 172.21.15.113:0/634358416 >> v1:172.21.15.136:6789/0 conn(0x7fbcdc06d140 legacy=0x7fbcdc06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:15.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.308+0000 7fbcd3e00640 1 -- 172.21.15.113:0/634358416 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc158360 con 0x7fbcdc151c20 2024-03-04T07:40:15.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbcd3e00640 1 --2- 172.21.15.113:0/634358416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc151c20 0x7fbcdc157710 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc070ba0 tx=0x7fbcc401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616b0dda710a3b2f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbcdaa00640 1 -- 172.21.15.113:0/634358416 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40b5070 con 0x7fbcdc151c20 2024-03-04T07:40:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbcdaa00640 1 -- 172.21.15.113:0/634358416 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc40a8a10 con 0x7fbcdc151c20 2024-03-04T07:40:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbcdaa00640 1 -- 172.21.15.113:0/634358416 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40a8de0 con 0x7fbcdc151c20 2024-03-04T07:40:15.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbce0e00640 1 -- 172.21.15.113:0/634358416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc151c20 msgr2=0x7fbcdc157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:15.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbce0e00640 1 --2- 172.21.15.113:0/634358416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc151c20 0x7fbcdc157710 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc070ba0 tx=0x7fbcc401ed10 comp rx=0 tx=0).stop 2024-03-04T07:40:15.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbce0e00640 1 -- 172.21.15.113:0/634358416 shutdown_connections 2024-03-04T07:40:15.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbce0e00640 1 --2- 172.21.15.113:0/634358416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc151c20 0x7fbcdc157710 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:15.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbce0e00640 1 --2- 172.21.15.113:0/634358416 >> v2:172.21.15.136:3300/0 conn(0x7fbcdc151840 0x7fbcdc0c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:15.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbce0e00640 1 -- 172.21.15.113:0/634358416 >> 172.21.15.113:0/634358416 conn(0x7fbcdc0c9a30 msgr2=0x7fbcdc0c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:15.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbce0e00640 1 -- 172.21.15.113:0/634358416 shutdown_connections 2024-03-04T07:40:15.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.312+0000 7fbce0e00640 1 -- 172.21.15.113:0/634358416 wait complete. 2024-03-04T07:40:15.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbce0e00640 1 Processor -- start 2024-03-04T07:40:15.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbce0e00640 1 -- start start 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbce0e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc06d140 0x7fbcdc13eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbce0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbcdc151840 0x7fbcdc1f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbce0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcdc151c20 0x7fbcdc1f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbce0e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc071db0 con 0x7fbcdc06d140 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbce0e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc071ad0 con 0x7fbcdc151c20 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbce0e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc06c880 con 0x7fbcdc151840 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcdb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc06d140 0x7fbcdc13eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcdb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc06d140 0x7fbcdc13eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58606/0 (socket says 172.21.15.113:58606) 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcdb400640 1 -- 172.21.15.113:0/3860317212 learned_addr learned my addr 172.21.15.113:0/3860317212 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcd3e00640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbcdc151840 0x7fbcdc1f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcdbe00640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcdc151c20 0x7fbcdc1f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcdb400640 1 -- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbcdc151840 msgr2=0x7fbcdc1f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcdb400640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbcdc151840 0x7fbcdc1f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcdb400640 1 -- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcdc151c20 msgr2=0x7fbcdc1f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcdb400640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcdc151c20 0x7fbcdc1f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcdb400640 1 -- 172.21.15.113:0/3860317212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc143be0 con 0x7fbcdc06d140 2024-03-04T07:40:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcdb400640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc06d140 0x7fbcdc13eff0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00e930 tx=0x7fbccc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcd3400640 1 -- 172.21.15.113:0/3860317212 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc017070 con 0x7fbcdc06d140 2024-03-04T07:40:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcd3400640 1 -- 172.21.15.113:0/3860317212 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc00ca00 con 0x7fbcdc06d140 2024-03-04T07:40:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbce0e00640 1 -- 172.21.15.113:0/3860317212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcdc0c77c0 con 0x7fbcdc06d140 2024-03-04T07:40:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbcd3400640 1 -- 172.21.15.113:0/3860317212 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00cdb0 con 0x7fbcdc06d140 2024-03-04T07:40:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.316+0000 7fbce0e00640 1 -- 172.21.15.113:0/3860317212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc1f4090 con 0x7fbcdc06d140 2024-03-04T07:40:15.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.320+0000 7fbce0e00640 1 -- 172.21.15.113:0/3860317212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcdc075a80 con 0x7fbcdc06d140 2024-03-04T07:40:15.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.320+0000 7fbcd3400640 1 -- 172.21.15.113:0/3860317212 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbccc01a070 con 0x7fbcdc06d140 2024-03-04T07:40:15.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.320+0000 7fbcd3400640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbcb8066f60 0x7fbcb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:15.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.320+0000 7fbcd3400640 1 -- 172.21.15.113:0/3860317212 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6319+0+0 (secure 0 0 0) 0x7fbccc0859d0 con 0x7fbcdc06d140 2024-03-04T07:40:15.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.320+0000 7fbcd3e00640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbcb8066f60 0x7fbcb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:15.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.324+0000 7fbcd3e00640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbcb8066f60 0x7fbcb8069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40b2000 tx=0x7fbcc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:15.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.324+0000 7fbcd3400640 1 -- 172.21.15.113:0/3860317212 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbccc0569b0 con 0x7fbcdc06d140 2024-03-04T07:40:15.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.516+0000 7fbce0e00640 1 -- 172.21.15.113:0/3860317212 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcdc073520 con 0x7fbcdc06d140 2024-03-04T07:40:15.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.516+0000 7fbcd3400640 1 -- 172.21.15.113:0/3860317212 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+6610 (secure 0 0 0) 0x7fbccc0562b0 con 0x7fbcdc06d140 2024-03-04T07:40:15.521 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:15.522 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5244,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:39:11.619579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":30,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4276,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.136:6835/3348592059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3348592059},{"type":"v1","addr":"172.21.15.136:6835","nonce":3348592059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4276,"qdb_cluster":[5094,4938,4276]},"id":1}]} 2024-03-04T07:40:15.522 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 31 2024-03-04T07:40:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 -- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbcb8066f60 msgr2=0x7fbcb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbcb8066f60 0x7fbcb8069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40b2000 tx=0x7fbcc4008040 comp rx=0 tx=0).stop 2024-03-04T07:40:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 -- 172.21.15.113:0/3860317212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc06d140 msgr2=0x7fbcdc13eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc06d140 0x7fbcdc13eff0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00e930 tx=0x7fbccc00edf0 comp rx=0 tx=0).stop 2024-03-04T07:40:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 -- 172.21.15.113:0/3860317212 shutdown_connections 2024-03-04T07:40:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcdc151c20 0x7fbcdc1f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbcdc151840 0x7fbcdc1f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbcb8066f60 0x7fbcb8069420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 --2- 172.21.15.113:0/3860317212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbcdc06d140 0x7fbcdc13eff0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 -- 172.21.15.113:0/3860317212 >> 172.21.15.113:0/3860317212 conn(0x7fbcdc0c9a30 msgr2=0x7fbcdc0c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 -- 172.21.15.113:0/3860317212 shutdown_connections 2024-03-04T07:40:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:15.520+0000 7fbce0e00640 1 -- 172.21.15.113:0/3860317212 wait complete. 2024-03-04T07:40:15.543 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:40:17.544 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:17.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.680+0000 7ff0ad000640 1 Processor -- start 2024-03-04T07:40:17.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0ad000640 1 -- start start 2024-03-04T07:40:17.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0ad000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff0a81528d0 0x7ff0a8071380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:17.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0ad000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff0a80c5db0 0x7ff0a80c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:17.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0ad000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a815cd10 con 0x7ff0a80c7120 2024-03-04T07:40:17.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0ad000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a815be40 con 0x7ff0a81528d0 2024-03-04T07:40:17.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0ad000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0a80c7c90 con 0x7ff0a80c5db0 2024-03-04T07:40:17.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0a5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff0a81528d0 0x7ff0a8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:17.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0a5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff0a81528d0 0x7ff0a8071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55788/0 (socket says 172.21.15.113:55788) 2024-03-04T07:40:17.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0a5e00640 1 -- 172.21.15.113:0/1637640829 learned_addr learned my addr 172.21.15.113:0/1637640829 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:17.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0a6800640 1 --2- 172.21.15.113:0/1637640829 >> v2:172.21.15.136:3300/0 conn(0x7ff0a80c5db0 0x7ff0a80c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:17.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0a5e00640 1 -- 172.21.15.113:0/1637640829 >> v2:172.21.15.136:3300/0 conn(0x7ff0a80c5db0 msgr2=0x7ff0a80c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:17.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0a5e00640 1 --2- 172.21.15.113:0/1637640829 >> v2:172.21.15.136:3300/0 conn(0x7ff0a80c5db0 0x7ff0a80c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:17.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0a5e00640 1 -- 172.21.15.113:0/1637640829 >> v1:172.21.15.113:6789/0 conn(0x7ff0a80c7120 legacy=0x7ff0a815c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:17.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.684+0000 7ff0a5e00640 1 -- 172.21.15.113:0/1637640829 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8159570 con 0x7ff0a81528d0 2024-03-04T07:40:17.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0a5e00640 1 --2- 172.21.15.113:0/1637640829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a81528d0 0x7ff0a8071380 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80c6da0 tx=0x7ff09801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c8d9ecdd95abb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:17.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0a4a00640 1 -- 172.21.15.113:0/1637640829 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0980b5070 con 0x7ff0a81528d0 2024-03-04T07:40:17.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0a4a00640 1 -- 172.21.15.113:0/1637640829 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0980a8a10 con 0x7ff0a81528d0 2024-03-04T07:40:17.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0a4a00640 1 -- 172.21.15.113:0/1637640829 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0980a8de0 con 0x7ff0a81528d0 2024-03-04T07:40:17.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0ad000640 1 -- 172.21.15.113:0/1637640829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a81528d0 msgr2=0x7ff0a8071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:17.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0ad000640 1 --2- 172.21.15.113:0/1637640829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a81528d0 0x7ff0a8071380 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff0a80c6da0 tx=0x7ff09801ed10 comp rx=0 tx=0).stop 2024-03-04T07:40:17.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0ad000640 1 -- 172.21.15.113:0/1637640829 shutdown_connections 2024-03-04T07:40:17.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0ad000640 1 --2- 172.21.15.113:0/1637640829 >> v2:172.21.15.136:3300/0 conn(0x7ff0a80c5db0 0x7ff0a80c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:17.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0ad000640 1 --2- 172.21.15.113:0/1637640829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a81528d0 0x7ff0a8071380 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:17.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0ad000640 1 -- 172.21.15.113:0/1637640829 >> 172.21.15.113:0/1637640829 conn(0x7ff0a8067a40 msgr2=0x7ff0a806ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:17.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0ad000640 1 -- 172.21.15.113:0/1637640829 shutdown_connections 2024-03-04T07:40:17.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.688+0000 7ff0ad000640 1 -- 172.21.15.113:0/1637640829 wait complete. 2024-03-04T07:40:17.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 Processor -- start 2024-03-04T07:40:17.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 -- start start 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0a80c5db0 0x7ff0a80ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a80c7120 0x7ff0a814a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0a81528d0 0x7ff0a814a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a80731b0 con 0x7ff0a80c7120 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a8072ed0 con 0x7ff0a80c5db0 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a815c0e0 con 0x7ff0a81528d0 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a80c7120 0x7ff0a814a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a80c7120 0x7ff0a814a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55790/0 (socket says 172.21.15.113:55790) 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a5400640 1 -- 172.21.15.113:0/3176697412 learned_addr learned my addr 172.21.15.113:0/3176697412 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a5e00640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0a80c5db0 0x7ff0a80ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:17.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a6800640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0a81528d0 0x7ff0a814a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a5400640 1 -- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0a81528d0 msgr2=0x7ff0a814a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a5400640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0a81528d0 0x7ff0a814a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a5400640 1 -- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0a80c5db0 msgr2=0x7ff0a80ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a5400640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0a80c5db0 0x7ff0a80ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a5400640 1 -- 172.21.15.113:0/3176697412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a81532c0 con 0x7ff0a80c7120 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0a5400640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a80c7120 0x7ff0a814a1b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff09400c370 tx=0x7ff09400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff09e000640 1 -- 172.21.15.113:0/3176697412 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0940052d0 con 0x7ff0a80c7120 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff09e000640 1 -- 172.21.15.113:0/3176697412 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff094005470 con 0x7ff0a80c7120 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff09e000640 1 -- 172.21.15.113:0/3176697412 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094007260 con 0x7ff0a80c7120 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 -- 172.21.15.113:0/3176697412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a80cba70 con 0x7ff0a80c7120 2024-03-04T07:40:17.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 -- 172.21.15.113:0/3176697412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a80cc030 con 0x7ff0a80c7120 2024-03-04T07:40:17.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.692+0000 7ff0ad000640 1 -- 172.21.15.113:0/3176697412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a8075c00 con 0x7ff0a80c7120 2024-03-04T07:40:17.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.696+0000 7ff09e000640 1 -- 172.21.15.113:0/3176697412 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff094007400 con 0x7ff0a80c7120 2024-03-04T07:40:17.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.696+0000 7ff09e000640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff088066f60 0x7ff088069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:17.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.696+0000 7ff09e000640 1 -- 172.21.15.113:0/3176697412 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7ff094084690 con 0x7ff0a80c7120 2024-03-04T07:40:17.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.696+0000 7ff0a5e00640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff088066f60 0x7ff088069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:17.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.696+0000 7ff0a5e00640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff088066f60 0x7ff088069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0980b2000 tx=0x7ff098008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:17.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.700+0000 7ff09e000640 1 -- 172.21.15.113:0/3176697412 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff094055430 con 0x7ff0a80c7120 2024-03-04T07:40:17.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.904+0000 7ff0ad000640 1 -- 172.21.15.113:0/3176697412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0a80c9ab0 con 0x7ff0a80c7120 2024-03-04T07:40:17.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.904+0000 7ff09e000640 1 -- 172.21.15.113:0/3176697412 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5812 (secure 0 0 0) 0x7ff094054d30 con 0x7ff0a80c7120 2024-03-04T07:40:17.908 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:17.908 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:15.620167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[4938,5094]},"id":1}]} 2024-03-04T07:40:17.908 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 32 2024-03-04T07:40:17.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 -- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff088066f60 msgr2=0x7ff088069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:17.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff088066f60 0x7ff088069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0980b2000 tx=0x7ff098008040 comp rx=0 tx=0).stop 2024-03-04T07:40:17.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 -- 172.21.15.113:0/3176697412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a80c7120 msgr2=0x7ff0a814a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:17.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a80c7120 0x7ff0a814a1b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff09400c370 tx=0x7ff09400c830 comp rx=0 tx=0).stop 2024-03-04T07:40:17.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 -- 172.21.15.113:0/3176697412 shutdown_connections 2024-03-04T07:40:17.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0a81528d0 0x7ff0a814a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:17.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff088066f60 0x7ff088069420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:17.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0a80c7120 0x7ff0a814a1b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:17.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 --2- 172.21.15.113:0/3176697412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0a80c5db0 0x7ff0a80ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:17.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 -- 172.21.15.113:0/3176697412 >> 172.21.15.113:0/3176697412 conn(0x7ff0a8067a40 msgr2=0x7ff0a80c86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:17.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 -- 172.21.15.113:0/3176697412 shutdown_connections 2024-03-04T07:40:17.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:17.908+0000 7ff0ad000640 1 -- 172.21.15.113:0/3176697412 wait complete. 2024-03-04T07:40:17.939 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2024-03-04T07:40:17.939 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:40:17.939 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.076+0000 7f11cd000640 1 Processor -- start 2024-03-04T07:40:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.076+0000 7f11cd000640 1 -- start start 2024-03-04T07:40:18.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.076+0000 7f11cd000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c81527f0 0x7f11c8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:18.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.076+0000 7f11cd000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c81591f0 con 0x7f11c8152bd0 2024-03-04T07:40:18.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.076+0000 7f11cd000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c81594d0 con 0x7f11c80c8200 2024-03-04T07:40:18.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.076+0000 7f11cd000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8071df0 con 0x7f11c81527f0 2024-03-04T07:40:18.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.076+0000 7f11c6800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f11c8152bd0 0x7f11c81587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45850/0 (socket says 172.21.15.113:45850) 2024-03-04T07:40:18.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.076+0000 7f11c6800640 1 -- 172.21.15.113:0/2380656766 learned_addr learned my addr 172.21.15.113:0/2380656766 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:18.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.076+0000 7f11c5e00640 1 --2- 172.21.15.113:0/2380656766 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c81527f0 0x7f11c8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:18.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.080+0000 7f11c4a00640 1 -- 172.21.15.113:0/2380656766 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387704277 0 0) 0x7f11c81591f0 con 0x7f11c8152bd0 2024-03-04T07:40:18.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.080+0000 7f11c4a00640 1 -- 172.21.15.113:0/2380656766 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11ac003530 con 0x7f11c8152bd0 2024-03-04T07:40:18.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.080+0000 7f11c4a00640 1 -- 172.21.15.113:0/2380656766 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478339996 0 0) 0x7f11c81594d0 con 0x7f11c80c8200 2024-03-04T07:40:18.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.080+0000 7f11c4a00640 1 -- 172.21.15.113:0/2380656766 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11c81591f0 con 0x7f11c80c8200 2024-03-04T07:40:18.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.080+0000 7f11c5e00640 1 -- 172.21.15.113:0/2380656766 >> v1:172.21.15.136:6789/0 conn(0x7f11c80c8200 legacy=0x7f11c8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:18.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.080+0000 7f11c5e00640 1 -- 172.21.15.113:0/2380656766 >> v1:172.21.15.113:6789/0 conn(0x7f11c8152bd0 legacy=0x7f11c81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:18.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.080+0000 7f11c5e00640 1 -- 172.21.15.113:0/2380656766 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c80717a0 con 0x7f11c81527f0 2024-03-04T07:40:18.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11c5e00640 1 --2- 172.21.15.113:0/2380656766 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c81527f0 0x7f11c8070d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f11b80097f0 tx=0x7f11b801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=279d34e7e584aeff server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:18.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11c4a00640 1 -- 172.21.15.113:0/2380656766 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80ab5f0 con 0x7f11c81527f0 2024-03-04T07:40:18.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11c4a00640 1 -- 172.21.15.113:0/2380656766 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b80b0070 con 0x7f11c81527f0 2024-03-04T07:40:18.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11c4a00640 1 -- 172.21.15.113:0/2380656766 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80ab990 con 0x7f11c81527f0 2024-03-04T07:40:18.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11cd000640 1 -- 172.21.15.113:0/2380656766 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c81527f0 msgr2=0x7f11c8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:18.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11cd000640 1 --2- 172.21.15.113:0/2380656766 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c81527f0 0x7f11c8070d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f11b80097f0 tx=0x7f11b801c4e0 comp rx=0 tx=0).stop 2024-03-04T07:40:18.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11cd000640 1 -- 172.21.15.113:0/2380656766 shutdown_connections 2024-03-04T07:40:18.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11cd000640 1 --2- 172.21.15.113:0/2380656766 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c81527f0 0x7f11c8070d40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:18.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11cd000640 1 -- 172.21.15.113:0/2380656766 >> 172.21.15.113:0/2380656766 conn(0x7f11c806e820 msgr2=0x7f11c806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:18.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11cd000640 1 -- 172.21.15.113:0/2380656766 shutdown_connections 2024-03-04T07:40:18.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.084+0000 7f11cd000640 1 -- 172.21.15.113:0/2380656766 wait complete. 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 Processor -- start 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 -- start start 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c80c8200 0x7f11c80c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c81527f0 0x7f11c81f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c8152bd0 0x7f11c814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8166380 con 0x7f11c80c8200 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c81597b0 con 0x7f11c81527f0 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c8166500 con 0x7f11c8152bd0 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c80c8200 0x7f11c80c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c80c8200 0x7f11c80c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55792/0 (socket says 172.21.15.113:55792) 2024-03-04T07:40:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5e00640 1 -- 172.21.15.113:0/2572696546 learned_addr learned my addr 172.21.15.113:0/2572696546 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c6800640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c8152bd0 0x7f11c814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5400640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c81527f0 0x7f11c81f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5e00640 1 -- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c8152bd0 msgr2=0x7f11c814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5e00640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c8152bd0 0x7f11c814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5e00640 1 -- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c81527f0 msgr2=0x7f11c81f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5e00640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c81527f0 0x7f11c81f37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5e00640 1 -- 172.21.15.113:0/2572696546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c80c46b0 con 0x7f11c80c8200 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5400640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c81527f0 0x7f11c81f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11c5e00640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c80c8200 0x7f11c80c34f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f11b80190f0 tx=0x7f11b8019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11be000640 1 -- 172.21.15.113:0/2572696546 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8003820 con 0x7f11c80c8200 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11be000640 1 -- 172.21.15.113:0/2572696546 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b80039c0 con 0x7f11c80c8200 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 -- 172.21.15.113:0/2572696546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c814d010 con 0x7f11c80c8200 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11be000640 1 -- 172.21.15.113:0/2572696546 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b8003d90 con 0x7f11c80c8200 2024-03-04T07:40:18.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 -- 172.21.15.113:0/2572696546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c814d5d0 con 0x7f11c80c8200 2024-03-04T07:40:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.088+0000 7f11cd000640 1 -- 172.21.15.113:0/2572696546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c814d810 con 0x7f11c80c8200 2024-03-04T07:40:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.092+0000 7f11be000640 1 -- 172.21.15.113:0/2572696546 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11b80c3060 con 0x7f11c80c8200 2024-03-04T07:40:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.092+0000 7f11be000640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a8066f60 0x7f11a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.092+0000 7f11be000640 1 -- 172.21.15.113:0/2572696546 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f11b811ec20 con 0x7f11c80c8200 2024-03-04T07:40:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.092+0000 7f11c5400640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a8066f60 0x7f11a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:18.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.092+0000 7f11c5400640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a8066f60 0x7f11a8069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11c80c76c0 tx=0x7f11b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:18.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.096+0000 7f11be000640 1 -- 172.21.15.113:0/2572696546 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f11c814d810 con 0x7f11c80c8200 2024-03-04T07:40:18.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.292+0000 7f11cd000640 1 -- 172.21.15.113:0/2572696546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11c8075c00 con 0x7f11c80c8200 2024-03-04T07:40:18.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.292+0000 7f11be000640 1 -- 172.21.15.113:0/2572696546 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f11b80160d0 con 0x7f11c80c8200 2024-03-04T07:40:18.297 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:18.297 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:15.620167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[4938,5094]},"id":1}]} 2024-03-04T07:40:18.297 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 32 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 -- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a8066f60 msgr2=0x7f11a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a8066f60 0x7f11a8069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f11c80c76c0 tx=0x7f11b4009040 comp rx=0 tx=0).stop 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 -- 172.21.15.113:0/2572696546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c80c8200 msgr2=0x7f11c80c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c80c8200 0x7f11c80c34f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f11b80190f0 tx=0x7f11b8019280 comp rx=0 tx=0).stop 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 -- 172.21.15.113:0/2572696546 shutdown_connections 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c8152bd0 0x7f11c814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c81527f0 0x7f11c81f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a8066f60 0x7f11a8069420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 --2- 172.21.15.113:0/2572696546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c80c8200 0x7f11c80c34f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 -- 172.21.15.113:0/2572696546 >> 172.21.15.113:0/2572696546 conn(0x7f11c806e820 msgr2=0x7f11c8158520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 -- 172.21.15.113:0/2572696546 shutdown_connections 2024-03-04T07:40:18.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:18.296+0000 7f11cd000640 1 -- 172.21.15.113:0/2572696546 wait complete. 2024-03-04T07:40:18.318 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 32, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 32, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4938": { "addr": "172.21.15.113:6836/2997246507", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2997246507, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2997246507, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4938, "incarnation": 20, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 25 }, "gid_5094": { "addr": "172.21.15.113:6833/605142559", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 605142559, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 605142559, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5094, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 21 }, "gid_5244": { "addr": "172.21.15.113:6837/742633123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 742633123, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 742633123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5244, "incarnation": 32, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:40:15.620167+0000", "qdb_cluster": [ 4938, 5094 ], "qdb_leader": 5094, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5244, "mds_1": 5094, "mds_2": 4938 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:40:20.319 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:20.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.452+0000 7fcbb6000640 1 Processor -- start 2024-03-04T07:40:20.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.452+0000 7fcbb6000640 1 -- start start 2024-03-04T07:40:20.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbb6000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbb00b20b0 0x7fcbb0157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:20.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbb6000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcbb0157c50 0x7fcbb006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:20.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbb6000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcbb00c85f0 0x7fcbb006b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbb6000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb00c1ec0 con 0x7fcbb00c85f0 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbb6000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb00c9b30 con 0x7fcbb0157c50 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbb6000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb00c12a0 con 0x7fcbb00b20b0 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbafe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcbb00c85f0 0x7fcbb006b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbafe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcbb00c85f0 0x7fcbb006b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55798/0 (socket says 172.21.15.113:55798) 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbafe00640 1 -- 172.21.15.113:0/4032656 learned_addr learned my addr 172.21.15.113:0/4032656 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbaf400640 1 --2- 172.21.15.113:0/4032656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbb00b20b0 0x7fcbb0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbaea00640 1 --2- 172.21.15.113:0/4032656 >> v2:172.21.15.136:3300/0 conn(0x7fcbb0157c50 0x7fcbb006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbafe00640 1 -- 172.21.15.113:0/4032656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbb00b20b0 msgr2=0x7fcbb0157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbafe00640 1 --2- 172.21.15.113:0/4032656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbb00b20b0 0x7fcbb0157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbafe00640 1 -- 172.21.15.113:0/4032656 >> v2:172.21.15.136:3300/0 conn(0x7fcbb0157c50 msgr2=0x7fcbb006b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbafe00640 1 --2- 172.21.15.113:0/4032656 >> v2:172.21.15.136:3300/0 conn(0x7fcbb0157c50 0x7fcbb006b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.456+0000 7fcbafe00640 1 -- 172.21.15.113:0/4032656 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb00c1580 con 0x7fcbb00c85f0 2024-03-04T07:40:20.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbafe00640 1 --2- 172.21.15.113:0/4032656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb00c85f0 0x7fcbb006b5a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcbb006f4c0 tx=0x7fcba801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4350af17beeceef0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:20.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbae000640 1 -- 172.21.15.113:0/4032656 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80a57c0 con 0x7fcbb00c85f0 2024-03-04T07:40:20.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbae000640 1 -- 172.21.15.113:0/4032656 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba80a5960 con 0x7fcbb00c85f0 2024-03-04T07:40:20.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbae000640 1 -- 172.21.15.113:0/4032656 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80a5d30 con 0x7fcbb00c85f0 2024-03-04T07:40:20.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbb6000640 1 -- 172.21.15.113:0/4032656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb00c85f0 msgr2=0x7fcbb006b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbb6000640 1 --2- 172.21.15.113:0/4032656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb00c85f0 0x7fcbb006b5a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcbb006f4c0 tx=0x7fcba801c9c0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbb6000640 1 -- 172.21.15.113:0/4032656 shutdown_connections 2024-03-04T07:40:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbb6000640 1 --2- 172.21.15.113:0/4032656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb00c85f0 0x7fcbb006b5a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbb6000640 1 --2- 172.21.15.113:0/4032656 >> v2:172.21.15.136:3300/0 conn(0x7fcbb0157c50 0x7fcbb006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbb6000640 1 --2- 172.21.15.113:0/4032656 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbb00b20b0 0x7fcbb0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbb6000640 1 -- 172.21.15.113:0/4032656 >> 172.21.15.113:0/4032656 conn(0x7fcbb006e820 msgr2=0x7fcbb006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbb6000640 1 -- 172.21.15.113:0/4032656 shutdown_connections 2024-03-04T07:40:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.460+0000 7fcbb6000640 1 -- 172.21.15.113:0/4032656 wait complete. 2024-03-04T07:40:20.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbb6000640 1 Processor -- start 2024-03-04T07:40:20.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbb6000640 1 -- start start 2024-03-04T07:40:20.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbb6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbb00b20b0 0x7fcbb0149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:20.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbb6000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbb00c85f0 0x7fcbb00affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:20.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbb6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb0157c50 0x7fcbb00b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:20.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbb6000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb00c1b80 con 0x7fcbb0157c50 2024-03-04T07:40:20.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbb6000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb0156ec0 con 0x7fcbb00b20b0 2024-03-04T07:40:20.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbb6000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb00c80c0 con 0x7fcbb00c85f0 2024-03-04T07:40:20.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb0157c50 0x7fcbb00b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:20.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbafe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb0157c50 0x7fcbb00b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55800/0 (socket says 172.21.15.113:55800) 2024-03-04T07:40:20.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbafe00640 1 -- 172.21.15.113:0/1627163882 learned_addr learned my addr 172.21.15.113:0/1627163882 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbaea00640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbb00c85f0 0x7fcbb00affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbaf400640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbb00b20b0 0x7fcbb0149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbafe00640 1 -- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbb00c85f0 msgr2=0x7fcbb00affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbafe00640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbb00c85f0 0x7fcbb00affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbafe00640 1 -- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbb00b20b0 msgr2=0x7fcbb0149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbafe00640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbb00b20b0 0x7fcbb0149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbafe00640 1 -- 172.21.15.113:0/1627163882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb00c29d0 con 0x7fcbb0157c50 2024-03-04T07:40:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbaf400640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbb00b20b0 0x7fcbb0149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbafe00640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb0157c50 0x7fcbb00b04f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcba801b070 tx=0x7fcba8019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:20.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcba7400640 1 -- 172.21.15.113:0/1627163882 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80b8670 con 0x7fcbb0157c50 2024-03-04T07:40:20.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcba7400640 1 -- 172.21.15.113:0/1627163882 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba80be070 con 0x7fcbb0157c50 2024-03-04T07:40:20.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbb6000640 1 -- 172.21.15.113:0/1627163882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb00af820 con 0x7fcbb0157c50 2024-03-04T07:40:20.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcba7400640 1 -- 172.21.15.113:0/1627163882 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80b8a70 con 0x7fcbb0157c50 2024-03-04T07:40:20.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.464+0000 7fcbb6000640 1 -- 172.21.15.113:0/1627163882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb014c560 con 0x7fcbb0157c50 2024-03-04T07:40:20.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.468+0000 7fcba4c00640 1 -- 172.21.15.113:0/1627163882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb0075c00 con 0x7fcbb0157c50 2024-03-04T07:40:20.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.468+0000 7fcba7400640 1 -- 172.21.15.113:0/1627163882 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcba80b8c10 con 0x7fcbb0157c50 2024-03-04T07:40:20.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.468+0000 7fcba7400640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb90066f60 0x7fcb90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:20.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.468+0000 7fcba7400640 1 -- 172.21.15.113:0/1627163882 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fcba8120410 con 0x7fcbb0157c50 2024-03-04T07:40:20.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.468+0000 7fcbaf400640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb90066f60 0x7fcb90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.472+0000 7fcbaf400640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb90066f60 0x7fcb90069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009a90 tx=0x7fcba0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.472+0000 7fcba7400640 1 -- 172.21.15.113:0/1627163882 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcba80165e0 con 0x7fcbb0157c50 2024-03-04T07:40:20.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.696+0000 7fcba4c00640 1 -- 172.21.15.113:0/1627163882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbb00c83d0 con 0x7fcbb0157c50 2024-03-04T07:40:20.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.696+0000 7fcba7400640 1 -- 172.21.15.113:0/1627163882 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5812 (secure 0 0 0) 0x7fcba80f1120 con 0x7fcbb0157c50 2024-03-04T07:40:20.701 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:20.702 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:15.620167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[4938,5094]},"id":1}]} 2024-03-04T07:40:20.702 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 32 2024-03-04T07:40:20.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 -- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb90066f60 msgr2=0x7fcb90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:20.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb90066f60 0x7fcb90069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009a90 tx=0x7fcba0008040 comp rx=0 tx=0).stop 2024-03-04T07:40:20.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 -- 172.21.15.113:0/1627163882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb0157c50 msgr2=0x7fcbb00b04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:20.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb0157c50 0x7fcbb00b04f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcba801b070 tx=0x7fcba8019760 comp rx=0 tx=0).stop 2024-03-04T07:40:20.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 -- 172.21.15.113:0/1627163882 shutdown_connections 2024-03-04T07:40:20.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb90066f60 0x7fcb90069420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbb0157c50 0x7fcbb00b04f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbb00c85f0 0x7fcbb00affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 --2- 172.21.15.113:0/1627163882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbb00b20b0 0x7fcbb0149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:20.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 -- 172.21.15.113:0/1627163882 >> 172.21.15.113:0/1627163882 conn(0x7fcbb006e820 msgr2=0x7fcbb01566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:20.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 -- 172.21.15.113:0/1627163882 shutdown_connections 2024-03-04T07:40:20.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:20.700+0000 7fcba4c00640 1 -- 172.21.15.113:0/1627163882 wait complete. 2024-03-04T07:40:20.728 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:40:22.729 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.876+0000 7f7daa200640 1 Processor -- start 2024-03-04T07:40:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.876+0000 7f7daa200640 1 -- start start 2024-03-04T07:40:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7daa200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7da4157f80 0x7f7da406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7daa200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da40c8a90 0x7f7da406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7daa200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da406df00 con 0x7f7da406f2c0 2024-03-04T07:40:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7daa200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da406e1e0 con 0x7f7da4157f80 2024-03-04T07:40:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7daa200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40ce1a0 con 0x7f7da40c8a90 2024-03-04T07:40:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da9800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7da406f2c0 0x7f7da40c9ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45864/0 (socket says 172.21.15.113:45864) 2024-03-04T07:40:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da9800640 1 -- 172.21.15.113:0/843913954 learned_addr learned my addr 172.21.15.113:0/843913954 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:22.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da3e00640 1 --2- 172.21.15.113:0/843913954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da40c8a90 0x7f7da406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:22.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da8e00640 1 --2- 172.21.15.113:0/843913954 >> v2:172.21.15.136:3300/0 conn(0x7f7da4157f80 0x7f7da406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:22.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da3400640 1 -- 172.21.15.113:0/843913954 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316033085 0 0) 0x7f7da406df00 con 0x7f7da406f2c0 2024-03-04T07:40:22.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da3400640 1 -- 172.21.15.113:0/843913954 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d8c003530 con 0x7f7da406f2c0 2024-03-04T07:40:22.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da3e00640 1 -- 172.21.15.113:0/843913954 >> v2:172.21.15.136:3300/0 conn(0x7f7da4157f80 msgr2=0x7f7da406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:22.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da3e00640 1 --2- 172.21.15.113:0/843913954 >> v2:172.21.15.136:3300/0 conn(0x7f7da4157f80 0x7f7da406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:22.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da3e00640 1 -- 172.21.15.113:0/843913954 >> v1:172.21.15.113:6789/0 conn(0x7f7da406f2c0 legacy=0x7f7da40c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:22.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da3e00640 1 -- 172.21.15.113:0/843913954 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da40ce480 con 0x7f7da40c8a90 2024-03-04T07:40:22.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.880+0000 7f7da8e00640 1 --2- 172.21.15.113:0/843913954 >> v2:172.21.15.136:3300/0 conn(0x7f7da4157f80 0x7f7da406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:40:22.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7da3e00640 1 --2- 172.21.15.113:0/843913954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da40c8a90 0x7f7da406ed80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d900097c0 tx=0x7f7d9001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e824f2a602dcd63e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:22.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7da3400640 1 -- 172.21.15.113:0/843913954 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d900b0070 con 0x7f7da40c8a90 2024-03-04T07:40:22.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7da3400640 1 -- 172.21.15.113:0/843913954 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d900ab600 con 0x7f7da40c8a90 2024-03-04T07:40:22.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7da3400640 1 -- 172.21.15.113:0/843913954 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d900ab9b0 con 0x7f7da40c8a90 2024-03-04T07:40:22.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7daa200640 1 -- 172.21.15.113:0/843913954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da40c8a90 msgr2=0x7f7da406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:22.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7daa200640 1 --2- 172.21.15.113:0/843913954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da40c8a90 0x7f7da406ed80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7d900097c0 tx=0x7f7d9001c4a0 comp rx=0 tx=0).stop 2024-03-04T07:40:22.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7daa200640 1 -- 172.21.15.113:0/843913954 shutdown_connections 2024-03-04T07:40:22.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7daa200640 1 --2- 172.21.15.113:0/843913954 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da40c8a90 0x7f7da406ed80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:22.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7daa200640 1 --2- 172.21.15.113:0/843913954 >> v2:172.21.15.136:3300/0 conn(0x7f7da4157f80 0x7f7da406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:22.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7daa200640 1 -- 172.21.15.113:0/843913954 >> 172.21.15.113:0/843913954 conn(0x7f7da4072f40 msgr2=0x7f7da40706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:22.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7daa200640 1 -- 172.21.15.113:0/843913954 shutdown_connections 2024-03-04T07:40:22.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7daa200640 1 -- 172.21.15.113:0/843913954 wait complete. 2024-03-04T07:40:22.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7daa200640 1 Processor -- start 2024-03-04T07:40:22.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.884+0000 7f7daa200640 1 -- start start 2024-03-04T07:40:22.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7daa200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7da406f2c0 0x7f7da41549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:22.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7daa200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da40c8a90 0x7f7da41506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:22.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7daa200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da4157f80 0x7f7da40cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:22.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7daa200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da406fdb0 con 0x7f7da40c8a90 2024-03-04T07:40:22.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7daa200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da406fc30 con 0x7f7da406f2c0 2024-03-04T07:40:22.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7daa200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da4167a60 con 0x7f7da4157f80 2024-03-04T07:40:22.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da40c8a90 0x7f7da41506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:22.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da40c8a90 0x7f7da41506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55806/0 (socket says 172.21.15.113:55806) 2024-03-04T07:40:22.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da3e00640 1 -- 172.21.15.113:0/2827391047 learned_addr learned my addr 172.21.15.113:0/2827391047 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:22.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da9800640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da4157f80 0x7f7da40cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:22.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da8e00640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7da406f2c0 0x7f7da41549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:22.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da9800640 1 -- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7da406f2c0 msgr2=0x7f7da41549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:22.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da9800640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7da406f2c0 0x7f7da41549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:22.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da9800640 1 -- 172.21.15.113:0/2827391047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da40c8a90 msgr2=0x7f7da41506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:22.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da9800640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da40c8a90 0x7f7da41506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:22.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da9800640 1 -- 172.21.15.113:0/2827391047 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da41f6b70 con 0x7f7da4157f80 2024-03-04T07:40:22.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da3e00640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da40c8a90 0x7f7da41506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:40:22.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da9800640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da4157f80 0x7f7da40cd810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00ca80 tx=0x7f7d9c00cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:22.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da0c00640 1 -- 172.21.15.113:0/2827391047 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c017070 con 0x7f7da4157f80 2024-03-04T07:40:22.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7daa200640 1 -- 172.21.15.113:0/2827391047 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da4152b30 con 0x7f7da4157f80 2024-03-04T07:40:22.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7daa200640 1 -- 172.21.15.113:0/2827391047 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da41533f0 con 0x7f7da4157f80 2024-03-04T07:40:22.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da0c00640 1 -- 172.21.15.113:0/2827391047 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c00ac30 con 0x7f7da4157f80 2024-03-04T07:40:22.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7da0c00640 1 -- 172.21.15.113:0/2827391047 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c012400 con 0x7f7da4157f80 2024-03-04T07:40:22.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.888+0000 7f7daa200640 1 -- 172.21.15.113:0/2827391047 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d800052a0 con 0x7f7da4157f80 2024-03-04T07:40:22.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.892+0000 7f7da0c00640 1 -- 172.21.15.113:0/2827391047 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d9c01a070 con 0x7f7da4157f80 2024-03-04T07:40:22.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.892+0000 7f7da0c00640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d84066f60 0x7f7d84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:22.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.892+0000 7f7da0c00640 1 -- 172.21.15.113:0/2827391047 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f7d9c0856d0 con 0x7f7da4157f80 2024-03-04T07:40:22.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.896+0000 7f7da8e00640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d84066f60 0x7f7d84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:22.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.896+0000 7f7da8e00640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d84066f60 0x7f7d84069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009580 tx=0x7f7d98009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:22.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:22.904+0000 7f7da0c00640 1 -- 172.21.15.113:0/2827391047 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7d9c055e30 con 0x7f7da4157f80 2024-03-04T07:40:23.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.116+0000 7f7daa200640 1 -- 172.21.15.113:0/2827391047 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d80005590 con 0x7f7da4157f80 2024-03-04T07:40:23.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.116+0000 7f7da0c00640 1 -- 172.21.15.113:0/2827391047 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f7d9c059ed0 con 0x7f7da4157f80 2024-03-04T07:40:23.121 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:23.121 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:15.620167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[4938,5094]},"id":1}]} 2024-03-04T07:40:23.122 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 32 2024-03-04T07:40:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 -- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d84066f60 msgr2=0x7f7d84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d84066f60 0x7f7d84069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009580 tx=0x7f7d98009190 comp rx=0 tx=0).stop 2024-03-04T07:40:23.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 -- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da4157f80 msgr2=0x7f7da40cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da4157f80 0x7f7da40cd810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00ca80 tx=0x7f7d9c00cf40 comp rx=0 tx=0).stop 2024-03-04T07:40:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 -- 172.21.15.113:0/2827391047 shutdown_connections 2024-03-04T07:40:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7da4157f80 0x7f7da40cd810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d84066f60 0x7f7d84069420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7da40c8a90 0x7f7da41506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 --2- 172.21.15.113:0/2827391047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7da406f2c0 0x7f7da41549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 -- 172.21.15.113:0/2827391047 >> 172.21.15.113:0/2827391047 conn(0x7f7da4072f40 msgr2=0x7f7da41519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 -- 172.21.15.113:0/2827391047 shutdown_connections 2024-03-04T07:40:23.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:23.120+0000 7f7daa200640 1 -- 172.21.15.113:0/2827391047 wait complete. 2024-03-04T07:40:23.143 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:40:25.144 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:25.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f2ba00640 1 Processor -- start 2024-03-04T07:40:25.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f2ba00640 1 -- start start 2024-03-04T07:40:25.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f2ba00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f241527f0 0x7f5f24070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:25.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f2ba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f24152bd0 0x7f5f241587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f2ba00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f241591f0 con 0x7f5f240c8200 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f2ba00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f24071df0 con 0x7f5f241527f0 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f2ba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f241597b0 con 0x7f5f24152bd0 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f29000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f241527f0 0x7f5f24070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f29000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f241527f0 0x7f5f24070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55816/0 (socket says 172.21.15.113:55816) 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f29000640 1 -- 172.21.15.113:0/4121649564 learned_addr learned my addr 172.21.15.113:0/4121649564 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.280+0000 7f5f29a00640 1 --2- 172.21.15.113:0/4121649564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f24152bd0 0x7f5f241587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.284+0000 7f5f29000640 1 -- 172.21.15.113:0/4121649564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f24152bd0 msgr2=0x7f5f241587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.284+0000 7f5f29000640 1 --2- 172.21.15.113:0/4121649564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f24152bd0 0x7f5f241587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.284+0000 7f5f29000640 1 -- 172.21.15.113:0/4121649564 >> v1:172.21.15.136:6789/0 conn(0x7f5f240c8200 legacy=0x7f5f24072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:25.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.284+0000 7f5f29000640 1 -- 172.21.15.113:0/4121649564 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f241594d0 con 0x7f5f241527f0 2024-03-04T07:40:25.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f29000640 1 --2- 172.21.15.113:0/4121649564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 0x7f5f24070d40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009870 tx=0x7f5f1801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f8ca8e699339f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:25.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f23400640 1 -- 172.21.15.113:0/4121649564 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180a7070 con 0x7f5f241527f0 2024-03-04T07:40:25.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f23400640 1 -- 172.21.15.113:0/4121649564 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f180b0070 con 0x7f5f241527f0 2024-03-04T07:40:25.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f23400640 1 -- 172.21.15.113:0/4121649564 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f180ab890 con 0x7f5f241527f0 2024-03-04T07:40:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f2ba00640 1 -- 172.21.15.113:0/4121649564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 msgr2=0x7f5f24070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f2ba00640 1 --2- 172.21.15.113:0/4121649564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 0x7f5f24070d40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009870 tx=0x7f5f1801c560 comp rx=0 tx=0).stop 2024-03-04T07:40:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f2ba00640 1 -- 172.21.15.113:0/4121649564 shutdown_connections 2024-03-04T07:40:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f2ba00640 1 --2- 172.21.15.113:0/4121649564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f24152bd0 0x7f5f241587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f2ba00640 1 --2- 172.21.15.113:0/4121649564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 0x7f5f24070d40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f2ba00640 1 -- 172.21.15.113:0/4121649564 >> 172.21.15.113:0/4121649564 conn(0x7f5f2406e820 msgr2=0x7f5f2406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f2ba00640 1 -- 172.21.15.113:0/4121649564 shutdown_connections 2024-03-04T07:40:25.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.288+0000 7f5f2ba00640 1 -- 172.21.15.113:0/4121649564 wait complete. 2024-03-04T07:40:25.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f2ba00640 1 Processor -- start 2024-03-04T07:40:25.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f2ba00640 1 -- start start 2024-03-04T07:40:25.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f2ba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f240c8200 0x7f5f2414b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f2ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 0x7f5f240c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f2ba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f24152bd0 0x7f5f240c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f2ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f241654f0 con 0x7f5f241527f0 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f2ba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f24165370 con 0x7f5f24152bd0 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f2ba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f24165670 con 0x7f5f240c8200 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 0x7f5f240c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 0x7f5f240c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55820/0 (socket says 172.21.15.113:55820) 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f23e00640 1 -- 172.21.15.113:0/3569725155 learned_addr learned my addr 172.21.15.113:0/3569725155 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f29000640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f240c8200 0x7f5f2414b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f29a00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f24152bd0 0x7f5f240c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f23e00640 1 -- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f240c8200 msgr2=0x7f5f2414b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f23e00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f240c8200 0x7f5f2414b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:25.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f23e00640 1 -- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f24152bd0 msgr2=0x7f5f240c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f23e00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f24152bd0 0x7f5f240c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f23e00640 1 -- 172.21.15.113:0/3569725155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240c1230 con 0x7f5f241527f0 2024-03-04T07:40:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f29a00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f24152bd0 0x7f5f240c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.292+0000 7f5f23e00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 0x7f5f240c3920 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000c430 tx=0x7f5f1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.296+0000 7f5f20c00640 1 -- 172.21.15.113:0/3569725155 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f100052d0 con 0x7f5f241527f0 2024-03-04T07:40:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.296+0000 7f5f20c00640 1 -- 172.21.15.113:0/3569725155 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f10005470 con 0x7f5f241527f0 2024-03-04T07:40:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.296+0000 7f5f2ba00640 1 -- 172.21.15.113:0/3569725155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f241f43d0 con 0x7f5f241527f0 2024-03-04T07:40:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.296+0000 7f5f20c00640 1 -- 172.21.15.113:0/3569725155 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10007260 con 0x7f5f241527f0 2024-03-04T07:40:25.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.296+0000 7f5f2ba00640 1 -- 172.21.15.113:0/3569725155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f241f46b0 con 0x7f5f241527f0 2024-03-04T07:40:25.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.296+0000 7f5f2ba00640 1 -- 172.21.15.113:0/3569725155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f24075c00 con 0x7f5f241527f0 2024-03-04T07:40:25.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.296+0000 7f5f20c00640 1 -- 172.21.15.113:0/3569725155 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f10007400 con 0x7f5f241527f0 2024-03-04T07:40:25.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.296+0000 7f5f20c00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f04066f60 0x7f5f04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:25.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.300+0000 7f5f20c00640 1 -- 172.21.15.113:0/3569725155 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f5f10084770 con 0x7f5f241527f0 2024-03-04T07:40:25.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.300+0000 7f5f29000640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f04066f60 0x7f5f04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:25.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.300+0000 7f5f29000640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f04066f60 0x7f5f04069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5f180af000 tx=0x7f5f180a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:25.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.308+0000 7f5f20c00640 1 -- 172.21.15.113:0/3569725155 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5f10054e30 con 0x7f5f241527f0 2024-03-04T07:40:25.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.504+0000 7f5f2ba00640 1 -- 172.21.15.113:0/3569725155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f240c79d0 con 0x7f5f241527f0 2024-03-04T07:40:25.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.504+0000 7f5f20c00640 1 -- 172.21.15.113:0/3569725155 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5814 (secure 0 0 0) 0x7f5f10058ed0 con 0x7f5f241527f0 2024-03-04T07:40:25.508 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:25.508 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:22.927270+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:resolve","state_seq":18,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[5094,4938]},"id":1}]} 2024-03-04T07:40:25.508 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 33 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.504+0000 7f5f2ba00640 1 -- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f04066f60 msgr2=0x7f5f04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.504+0000 7f5f2ba00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f04066f60 0x7f5f04069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5f180af000 tx=0x7f5f180a7000 comp rx=0 tx=0).stop 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.504+0000 7f5f2ba00640 1 -- 172.21.15.113:0/3569725155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 msgr2=0x7f5f240c3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.504+0000 7f5f2ba00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 0x7f5f240c3920 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000c430 tx=0x7f5f1000c8f0 comp rx=0 tx=0).stop 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.508+0000 7f5f2ba00640 1 -- 172.21.15.113:0/3569725155 shutdown_connections 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.508+0000 7f5f2ba00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f24152bd0 0x7f5f240c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.508+0000 7f5f2ba00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f04066f60 0x7f5f04069420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.508+0000 7f5f2ba00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f241527f0 0x7f5f240c3920 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.508+0000 7f5f2ba00640 1 --2- 172.21.15.113:0/3569725155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f240c8200 0x7f5f2414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.508+0000 7f5f2ba00640 1 -- 172.21.15.113:0/3569725155 >> 172.21.15.113:0/3569725155 conn(0x7f5f2406e820 msgr2=0x7f5f24156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.508+0000 7f5f2ba00640 1 -- 172.21.15.113:0/3569725155 shutdown_connections 2024-03-04T07:40:25.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:25.508+0000 7f5f2ba00640 1 -- 172.21.15.113:0/3569725155 wait complete. 2024-03-04T07:40:25.529 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:40:26.084 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:40:26.088 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:40:27.529 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:27.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.720+0000 7fb1ce400640 1 Processor -- start 2024-03-04T07:40:27.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.720+0000 7fb1ce400640 1 -- start start 2024-03-04T07:40:27.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1ce400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1c8154960 0x7fb1c8066f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:27.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1ce400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c80c68e0 0x7fb1c815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:27.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1ce400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c806ea50 con 0x7fb1c8154d40 2024-03-04T07:40:27.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1ce400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c8067480 con 0x7fb1c8154960 2024-03-04T07:40:27.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1ce400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c80c8c10 con 0x7fb1c80c68e0 2024-03-04T07:40:27.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1cd000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1c8154960 0x7fb1c8066f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:27.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1cda00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb1c8154d40 0x7fb1c806f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52646/0 (socket says 172.21.15.113:52646) 2024-03-04T07:40:27.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1cda00640 1 -- 172.21.15.113:0/3725082094 learned_addr learned my addr 172.21.15.113:0/3725082094 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:27.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1c7e00640 1 --2- 172.21.15.113:0/3725082094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c80c68e0 0x7fb1c815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:27.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1cd000640 1 -- 172.21.15.113:0/3725082094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c80c68e0 msgr2=0x7fb1c815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:27.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1cd000640 1 --2- 172.21.15.113:0/3725082094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c80c68e0 0x7fb1c815dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:27.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1cd000640 1 -- 172.21.15.113:0/3725082094 >> v1:172.21.15.113:6789/0 conn(0x7fb1c8154d40 legacy=0x7fb1c806f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:27.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.724+0000 7fb1cd000640 1 -- 172.21.15.113:0/3725082094 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c80c7b10 con 0x7fb1c8154960 2024-03-04T07:40:27.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1cd000640 1 --2- 172.21.15.113:0/3725082094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c8154960 0x7fb1c8066f40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0097c0 tx=0x7fb1bc01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232d798a70936a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:27.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1c7400640 1 -- 172.21.15.113:0/3725082094 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0b5070 con 0x7fb1c8154960 2024-03-04T07:40:27.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1c7400640 1 -- 172.21.15.113:0/3725082094 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc0a8990 con 0x7fb1c8154960 2024-03-04T07:40:27.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1c7400640 1 -- 172.21.15.113:0/3725082094 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0a8d60 con 0x7fb1c8154960 2024-03-04T07:40:27.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1ce400640 1 -- 172.21.15.113:0/3725082094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c8154960 msgr2=0x7fb1c8066f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:27.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1ce400640 1 --2- 172.21.15.113:0/3725082094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c8154960 0x7fb1c8066f40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0097c0 tx=0x7fb1bc01ec90 comp rx=0 tx=0).stop 2024-03-04T07:40:27.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1ce400640 1 -- 172.21.15.113:0/3725082094 shutdown_connections 2024-03-04T07:40:27.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1ce400640 1 --2- 172.21.15.113:0/3725082094 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c80c68e0 0x7fb1c815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:27.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1ce400640 1 --2- 172.21.15.113:0/3725082094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c8154960 0x7fb1c8066f40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:27.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1ce400640 1 -- 172.21.15.113:0/3725082094 >> 172.21.15.113:0/3725082094 conn(0x7fb1c80cc660 msgr2=0x7fb1c80cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:27.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1ce400640 1 -- 172.21.15.113:0/3725082094 shutdown_connections 2024-03-04T07:40:27.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.728+0000 7fb1ce400640 1 -- 172.21.15.113:0/3725082094 wait complete. 2024-03-04T07:40:27.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1ce400640 1 Processor -- start 2024-03-04T07:40:27.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1ce400640 1 -- start start 2024-03-04T07:40:27.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1ce400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c80c68e0 0x7fb1c80c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:27.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1ce400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8154960 0x7fb1c80cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:27.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1ce400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c8154d40 0x7fb1c81f0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:27.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1ce400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c80c80d0 con 0x7fb1c80c68e0 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1ce400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c806ecf0 con 0x7fb1c8154960 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1ce400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c806f920 con 0x7fb1c8154d40 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cd000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c80c68e0 0x7fb1c80c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cd000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c80c68e0 0x7fb1c80c5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49524/0 (socket says 172.21.15.113:49524) 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cd000640 1 -- 172.21.15.113:0/3281699701 learned_addr learned my addr 172.21.15.113:0/3281699701 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cda00640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c8154d40 0x7fb1c81f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1c7e00640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8154960 0x7fb1c80cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cd000640 1 -- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c8154d40 msgr2=0x7fb1c81f0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cd000640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c8154d40 0x7fb1c81f0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cd000640 1 -- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8154960 msgr2=0x7fb1c80cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cd000640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8154960 0x7fb1c80cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cd000640 1 -- 172.21.15.113:0/3281699701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c81507c0 con 0x7fb1c80c68e0 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cda00640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c8154d40 0x7fb1c81f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1c7e00640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8154960 0x7fb1c80cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:27.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1cd000640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c80c68e0 0x7fb1c80c5c60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc01c8c0 tx=0x7fb1bc01ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:27.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1c4c00640 1 -- 172.21.15.113:0/3281699701 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0bf070 con 0x7fb1c80c68e0 2024-03-04T07:40:27.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1c4c00640 1 -- 172.21.15.113:0/3281699701 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc0b0820 con 0x7fb1c80c68e0 2024-03-04T07:40:27.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1ce400640 1 -- 172.21.15.113:0/3281699701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c815bde0 con 0x7fb1c80c68e0 2024-03-04T07:40:27.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1c4c00640 1 -- 172.21.15.113:0/3281699701 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0b5040 con 0x7fb1c80c68e0 2024-03-04T07:40:27.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.732+0000 7fb1ce400640 1 -- 172.21.15.113:0/3281699701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c81f3d70 con 0x7fb1c80c68e0 2024-03-04T07:40:27.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.736+0000 7fb1ce400640 1 -- 172.21.15.113:0/3281699701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c8075a80 con 0x7fb1c80c68e0 2024-03-04T07:40:27.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.736+0000 7fb1c4c00640 1 -- 172.21.15.113:0/3281699701 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1bc0c2070 con 0x7fb1c80c68e0 2024-03-04T07:40:27.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.736+0000 7fb1c4c00640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1a8066f90 0x7fb1a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:27.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.736+0000 7fb1c4c00640 1 -- 172.21.15.113:0/3281699701 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fb1bc121a40 con 0x7fb1c80c68e0 2024-03-04T07:40:27.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.736+0000 7fb1c7e00640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1a8066f90 0x7fb1a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:27.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.740+0000 7fb1c7e00640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1a8066f90 0x7fb1a8069450 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8154120 tx=0x7fb1b4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:27.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.740+0000 7fb1c4c00640 1 -- 172.21.15.113:0/3281699701 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb1bc0187f0 con 0x7fb1c80c68e0 2024-03-04T07:40:27.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.932+0000 7fb1ce400640 1 -- 172.21.15.113:0/3281699701 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1c814f900 con 0x7fb1c80c68e0 2024-03-04T07:40:27.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1c4c00640 1 -- 172.21.15.113:0/3281699701 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+5814 (secure 0 0 0) 0x7fb1bc0f2800 con 0x7fb1c80c68e0 2024-03-04T07:40:27.938 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:27.938 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:22.927270+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:resolve","state_seq":18,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[5094,4938]},"id":1}]} 2024-03-04T07:40:27.938 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 33 2024-03-04T07:40:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 -- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1a8066f90 msgr2=0x7fb1a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1a8066f90 0x7fb1a8069450 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8154120 tx=0x7fb1b4008040 comp rx=0 tx=0).stop 2024-03-04T07:40:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 -- 172.21.15.113:0/3281699701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c80c68e0 msgr2=0x7fb1c80c5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c80c68e0 0x7fb1c80c5c60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc01c8c0 tx=0x7fb1bc01ba30 comp rx=0 tx=0).stop 2024-03-04T07:40:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 -- 172.21.15.113:0/3281699701 shutdown_connections 2024-03-04T07:40:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1c8154d40 0x7fb1c81f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:27.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8154960 0x7fb1c80cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:27.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1a8066f90 0x7fb1a8069450 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:27.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 --2- 172.21.15.113:0/3281699701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1c80c68e0 0x7fb1c80c5c60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:27.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 -- 172.21.15.113:0/3281699701 >> 172.21.15.113:0/3281699701 conn(0x7fb1c80cc660 msgr2=0x7fb1c814e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:27.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 -- 172.21.15.113:0/3281699701 shutdown_connections 2024-03-04T07:40:27.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:27.936+0000 7fb1ce400640 1 -- 172.21.15.113:0/3281699701 wait complete. 2024-03-04T07:40:27.968 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:40:29.970 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:30.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.108+0000 7fcf8b400640 1 Processor -- start 2024-03-04T07:40:30.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.108+0000 7fcf8b400640 1 -- start start 2024-03-04T07:40:30.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.108+0000 7fcf8b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcf8414e4c0 0x7fcf84072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:30.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.108+0000 7fcf8b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf840c22e0 0x7fcf840731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:30.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.108+0000 7fcf8b400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf840c27f0 con 0x7fcf840c2d80 2024-03-04T07:40:30.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.108+0000 7fcf8b400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf8415ad60 con 0x7fcf8414e4c0 2024-03-04T07:40:30.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.108+0000 7fcf8b400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840c2a90 con 0x7fcf840c22e0 2024-03-04T07:40:30.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.112+0000 7fcf88a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcf8414e4c0 0x7fcf84072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:30.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.112+0000 7fcf89400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcf840c2d80 0x7fcf8415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52660/0 (socket says 172.21.15.113:52660) 2024-03-04T07:40:30.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.112+0000 7fcf89400640 1 -- 172.21.15.113:0/3993483717 learned_addr learned my addr 172.21.15.113:0/3993483717 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:30.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.112+0000 7fcf7be00640 1 --2- 172.21.15.113:0/3993483717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf840c22e0 0x7fcf840731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:30.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.112+0000 7fcf88a00640 1 -- 172.21.15.113:0/3993483717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf840c22e0 msgr2=0x7fcf840731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:30.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.112+0000 7fcf88a00640 1 --2- 172.21.15.113:0/3993483717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf840c22e0 0x7fcf840731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:30.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.112+0000 7fcf88a00640 1 -- 172.21.15.113:0/3993483717 >> v1:172.21.15.113:6789/0 conn(0x7fcf840c2d80 legacy=0x7fcf8415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:30.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.112+0000 7fcf88a00640 1 -- 172.21.15.113:0/3993483717 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf84070c60 con 0x7fcf8414e4c0 2024-03-04T07:40:30.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.112+0000 7fcf83e00640 1 -- 172.21.15.113:0/3993483717 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948997962 0 0) 0x7fcf840c27f0 con 0x7fcf840c2d80 2024-03-04T07:40:30.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf88a00640 1 --2- 172.21.15.113:0/3993483717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf8414e4c0 0x7fcf84072c80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcf740097c0 tx=0x7fcf7401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a9418c8d798382 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:30.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf83e00640 1 -- 172.21.15.113:0/3993483717 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf740b5070 con 0x7fcf8414e4c0 2024-03-04T07:40:30.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf83e00640 1 -- 172.21.15.113:0/3993483717 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf740a8990 con 0x7fcf8414e4c0 2024-03-04T07:40:30.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf83e00640 1 -- 172.21.15.113:0/3993483717 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf740a8d60 con 0x7fcf8414e4c0 2024-03-04T07:40:30.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf8b400640 1 -- 172.21.15.113:0/3993483717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf8414e4c0 msgr2=0x7fcf84072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:30.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf8b400640 1 --2- 172.21.15.113:0/3993483717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf8414e4c0 0x7fcf84072c80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fcf740097c0 tx=0x7fcf7401ec90 comp rx=0 tx=0).stop 2024-03-04T07:40:30.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf8b400640 1 -- 172.21.15.113:0/3993483717 shutdown_connections 2024-03-04T07:40:30.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf8b400640 1 --2- 172.21.15.113:0/3993483717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf840c22e0 0x7fcf840731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:30.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf8b400640 1 --2- 172.21.15.113:0/3993483717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf8414e4c0 0x7fcf84072c80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:30.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf8b400640 1 -- 172.21.15.113:0/3993483717 >> 172.21.15.113:0/3993483717 conn(0x7fcf8406e820 msgr2=0x7fcf8406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:30.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf8b400640 1 -- 172.21.15.113:0/3993483717 shutdown_connections 2024-03-04T07:40:30.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.116+0000 7fcf8b400640 1 -- 172.21.15.113:0/3993483717 wait complete. 2024-03-04T07:40:30.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 Processor -- start 2024-03-04T07:40:30.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 -- start start 2024-03-04T07:40:30.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf840c22e0 0x7fcf8415f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:30.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf840c2d80 0x7fcf8414d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:30.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf8414e4c0 0x7fcf841640c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84071500 con 0x7fcf840c22e0 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84071220 con 0x7fcf840c2d80 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8414e8a0 con 0x7fcf8414e4c0 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf88a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf840c22e0 0x7fcf8415f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf88a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf840c22e0 0x7fcf8415f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49546/0 (socket says 172.21.15.113:49546) 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf88a00640 1 -- 172.21.15.113:0/3001297227 learned_addr learned my addr 172.21.15.113:0/3001297227 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf89400640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf8414e4c0 0x7fcf841640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf7be00640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf840c2d80 0x7fcf8414d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf88a00640 1 -- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf8414e4c0 msgr2=0x7fcf841640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf88a00640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf8414e4c0 0x7fcf841640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf88a00640 1 -- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf840c2d80 msgr2=0x7fcf8414d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf88a00640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf840c2d80 0x7fcf8414d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:30.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf88a00640 1 -- 172.21.15.113:0/3001297227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf84159c50 con 0x7fcf840c22e0 2024-03-04T07:40:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf88a00640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf840c22e0 0x7fcf8415f600 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcf7401c8c0 tx=0x7fcf7401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf81600640 1 -- 172.21.15.113:0/3001297227 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf740bf070 con 0x7fcf840c22e0 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf81600640 1 -- 172.21.15.113:0/3001297227 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf740b0720 con 0x7fcf840c22e0 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 -- 172.21.15.113:0/3001297227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8414b7d0 con 0x7fcf840c22e0 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf81600640 1 -- 172.21.15.113:0/3001297227 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf740b5040 con 0x7fcf840c22e0 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 -- 172.21.15.113:0/3001297227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8415a1d0 con 0x7fcf840c22e0 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.120+0000 7fcf8b400640 1 -- 172.21.15.113:0/3001297227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf84075c00 con 0x7fcf840c22e0 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.124+0000 7fcf81600640 1 -- 172.21.15.113:0/3001297227 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf740c2070 con 0x7fcf840c22e0 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.124+0000 7fcf81600640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf60066f60 0x7fcf60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.124+0000 7fcf81600640 1 -- 172.21.15.113:0/3001297227 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fcf74121a20 con 0x7fcf840c22e0 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.124+0000 7fcf7be00640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf60066f60 0x7fcf60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.124+0000 7fcf7be00640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf60066f60 0x7fcf60069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcf841652e0 tx=0x7fcf6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:30.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.128+0000 7fcf81600640 1 -- 172.21.15.113:0/3001297227 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcf740187f0 con 0x7fcf840c22e0 2024-03-04T07:40:30.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.324+0000 7fcf8b400640 1 -- 172.21.15.113:0/3001297227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf84155190 con 0x7fcf840c22e0 2024-03-04T07:40:30.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.324+0000 7fcf81600640 1 -- 172.21.15.113:0/3001297227 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5816 (secure 0 0 0) 0x7fcf740f2730 con 0x7fcf840c22e0 2024-03-04T07:40:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:30.328 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:28.743092+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[4938,5094]},"id":1}]} 2024-03-04T07:40:30.328 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 34 2024-03-04T07:40:30.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 -- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf60066f60 msgr2=0x7fcf60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:30.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf60066f60 0x7fcf60069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcf841652e0 tx=0x7fcf6c008040 comp rx=0 tx=0).stop 2024-03-04T07:40:30.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 -- 172.21.15.113:0/3001297227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf840c22e0 msgr2=0x7fcf8415f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:30.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf840c22e0 0x7fcf8415f600 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcf7401c8c0 tx=0x7fcf7401ba30 comp rx=0 tx=0).stop 2024-03-04T07:40:30.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 -- 172.21.15.113:0/3001297227 shutdown_connections 2024-03-04T07:40:30.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf8414e4c0 0x7fcf841640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:30.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf840c2d80 0x7fcf8414d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:30.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf60066f60 0x7fcf60069420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:30.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 --2- 172.21.15.113:0/3001297227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf840c22e0 0x7fcf8415f600 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:30.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 -- 172.21.15.113:0/3001297227 >> 172.21.15.113:0/3001297227 conn(0x7fcf8406e820 msgr2=0x7fcf840c8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:30.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 -- 172.21.15.113:0/3001297227 shutdown_connections 2024-03-04T07:40:30.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:30.328+0000 7fcf8b400640 1 -- 172.21.15.113:0/3001297227 wait complete. 2024-03-04T07:40:30.351 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 34, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 34, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4938": { "addr": "172.21.15.113:6836/2997246507", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2997246507, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2997246507, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4938, "incarnation": 20, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 25 }, "gid_5094": { "addr": "172.21.15.113:6833/605142559", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 605142559, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 605142559, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5094, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 21 }, "gid_5244": { "addr": "172.21.15.113:6837/742633123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 742633123, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 742633123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5244, "incarnation": 32, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:reconnect", "state_seq": 20 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:40:28.743092+0000", "qdb_cluster": [ 4938, 5094 ], "qdb_leader": 5094, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5244, "mds_1": 5094, "mds_2": 4938 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:40:32.352 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:32.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba75a00640 1 Processor -- start 2024-03-04T07:40:32.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba75a00640 1 -- start start 2024-03-04T07:40:32.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba75a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba700ce570 0x7fba70154c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:32.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba75a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fba70070120 0x7fba701551d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:32.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba75a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7006c860 con 0x7fba700c8d80 2024-03-04T07:40:32.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba75a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7006f8d0 con 0x7fba700ce570 2024-03-04T07:40:32.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba75a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7006f310 con 0x7fba70070120 2024-03-04T07:40:32.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba6ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba700ce570 0x7fba70154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba6ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba700ce570 0x7fba70154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49550/0 (socket says 172.21.15.113:49550) 2024-03-04T07:40:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba6ea00640 1 -- 172.21.15.113:0/1350547140 learned_addr learned my addr 172.21.15.113:0/1350547140 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba66000640 1 --1- 172.21.15.113:0/1350547140 >> v1:172.21.15.113:6789/0 conn(0x7fba700c8d80 0x7fba7015b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52672/0 (socket says 172.21.15.113:52672) 2024-03-04T07:40:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba6f400640 1 --2- 172.21.15.113:0/1350547140 >> v2:172.21.15.136:3300/0 conn(0x7fba70070120 0x7fba701551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba6ea00640 1 -- 172.21.15.113:0/1350547140 >> v2:172.21.15.136:3300/0 conn(0x7fba70070120 msgr2=0x7fba701551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba6ea00640 1 --2- 172.21.15.113:0/1350547140 >> v2:172.21.15.136:3300/0 conn(0x7fba70070120 0x7fba701551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba6ea00640 1 -- 172.21.15.113:0/1350547140 >> v1:172.21.15.113:6789/0 conn(0x7fba700c8d80 legacy=0x7fba7015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.492+0000 7fba6ea00640 1 -- 172.21.15.113:0/1350547140 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7006f5f0 con 0x7fba700ce570 2024-03-04T07:40:32.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.496+0000 7fba6ea00640 1 --2- 172.21.15.113:0/1350547140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba700ce570 0x7fba70154c90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fba700708d0 tx=0x7fba6001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dd9b2bcc77bf72 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:32.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.496+0000 7fba6e000640 1 -- 172.21.15.113:0/1350547140 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba600b4070 con 0x7fba700ce570 2024-03-04T07:40:32.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.496+0000 7fba6e000640 1 -- 172.21.15.113:0/1350547140 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba600af400 con 0x7fba700ce570 2024-03-04T07:40:32.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.496+0000 7fba6e000640 1 -- 172.21.15.113:0/1350547140 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba600af7d0 con 0x7fba700ce570 2024-03-04T07:40:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 -- 172.21.15.113:0/1350547140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba700ce570 msgr2=0x7fba70154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 --2- 172.21.15.113:0/1350547140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba700ce570 0x7fba70154c90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fba700708d0 tx=0x7fba6001f1e0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 -- 172.21.15.113:0/1350547140 shutdown_connections 2024-03-04T07:40:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 --2- 172.21.15.113:0/1350547140 >> v2:172.21.15.136:3300/0 conn(0x7fba70070120 0x7fba701551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 --2- 172.21.15.113:0/1350547140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba700ce570 0x7fba70154c90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 -- 172.21.15.113:0/1350547140 >> 172.21.15.113:0/1350547140 conn(0x7fba70066db0 msgr2=0x7fba700671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 -- 172.21.15.113:0/1350547140 shutdown_connections 2024-03-04T07:40:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 -- 172.21.15.113:0/1350547140 wait complete. 2024-03-04T07:40:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 Processor -- start 2024-03-04T07:40:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 -- start start 2024-03-04T07:40:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba70070120 0x7fba700c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba700c8d80 0x7fba701504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba700ce570 0x7fba701f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7006fb80 con 0x7fba70070120 2024-03-04T07:40:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7006c4f0 con 0x7fba700ce570 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba75a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7006fd00 con 0x7fba700c8d80 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba6ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba70070120 0x7fba700c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba6ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba70070120 0x7fba700c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49552/0 (socket says 172.21.15.113:49552) 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba6ea00640 1 -- 172.21.15.113:0/2798659599 learned_addr learned my addr 172.21.15.113:0/2798659599 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba66000640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba700c8d80 0x7fba701504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba6f400640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba700ce570 0x7fba701f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba6ea00640 1 -- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba700c8d80 msgr2=0x7fba701504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba6ea00640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba700c8d80 0x7fba701504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba6ea00640 1 -- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba700ce570 msgr2=0x7fba701f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba6ea00640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba700ce570 0x7fba701f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.500+0000 7fba6ea00640 1 -- 172.21.15.113:0/2798659599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba70156110 con 0x7fba70070120 2024-03-04T07:40:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.504+0000 7fba6ea00640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba70070120 0x7fba700c4f10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fba6001bdf0 tx=0x7fba6001bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:32.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.504+0000 7fba67400640 1 -- 172.21.15.113:0/2798659599 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba600c0070 con 0x7fba70070120 2024-03-04T07:40:32.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.504+0000 7fba67400640 1 -- 172.21.15.113:0/2798659599 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba600afe30 con 0x7fba70070120 2024-03-04T07:40:32.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.504+0000 7fba75a00640 1 -- 172.21.15.113:0/2798659599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba70155840 con 0x7fba70070120 2024-03-04T07:40:32.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.504+0000 7fba67400640 1 -- 172.21.15.113:0/2798659599 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba600b4040 con 0x7fba70070120 2024-03-04T07:40:32.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.504+0000 7fba75a00640 1 -- 172.21.15.113:0/2798659599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba701f4c80 con 0x7fba70070120 2024-03-04T07:40:32.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.504+0000 7fba75a00640 1 -- 172.21.15.113:0/2798659599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba70075c00 con 0x7fba70070120 2024-03-04T07:40:32.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.504+0000 7fba67400640 1 -- 172.21.15.113:0/2798659599 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba600c3070 con 0x7fba70070120 2024-03-04T07:40:32.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.508+0000 7fba67400640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fba4c066f60 0x7fba4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:32.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.508+0000 7fba67400640 1 -- 172.21.15.113:0/2798659599 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fba60122a10 con 0x7fba70070120 2024-03-04T07:40:32.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.508+0000 7fba66000640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fba4c066f60 0x7fba4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:32.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.508+0000 7fba66000640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fba4c066f60 0x7fba4c069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fba70150e30 tx=0x7fba58007cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:32.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.512+0000 7fba67400640 1 -- 172.21.15.113:0/2798659599 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fba60018d40 con 0x7fba70070120 2024-03-04T07:40:32.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.752+0000 7fba75a00640 1 -- 172.21.15.113:0/2798659599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba70151b60 con 0x7fba70070120 2024-03-04T07:40:32.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.752+0000 7fba67400640 1 -- 172.21.15.113:0/2798659599 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5816 (secure 0 0 0) 0x7fba600f3910 con 0x7fba70070120 2024-03-04T07:40:32.756 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:32.756 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:28.743092+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[4938,5094]},"id":1}]} 2024-03-04T07:40:32.756 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 34 2024-03-04T07:40:32.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 -- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fba4c066f60 msgr2=0x7fba4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:32.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fba4c066f60 0x7fba4c069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fba70150e30 tx=0x7fba58007cd0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 -- 172.21.15.113:0/2798659599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba70070120 msgr2=0x7fba700c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:32.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba70070120 0x7fba700c4f10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fba6001bdf0 tx=0x7fba6001bf80 comp rx=0 tx=0).stop 2024-03-04T07:40:32.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 -- 172.21.15.113:0/2798659599 shutdown_connections 2024-03-04T07:40:32.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba700ce570 0x7fba701f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fba700c8d80 0x7fba701504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fba4c066f60 0x7fba4c069420 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 --2- 172.21.15.113:0/2798659599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba70070120 0x7fba700c4f10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 -- 172.21.15.113:0/2798659599 >> 172.21.15.113:0/2798659599 conn(0x7fba70066db0 msgr2=0x7fba70153130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 -- 172.21.15.113:0/2798659599 shutdown_connections 2024-03-04T07:40:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:32.756+0000 7fba75a00640 1 -- 172.21.15.113:0/2798659599 wait complete. 2024-03-04T07:40:32.789 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:40:34.789 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f978b400640 1 Processor -- start 2024-03-04T07:40:34.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f978b400640 1 -- start start 2024-03-04T07:40:34.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f978b400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97840c8200 0x7f9784070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:34.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f978b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 0x7f97841587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:34.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f978b400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97841591f0 con 0x7f97841527f0 2024-03-04T07:40:34.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f978b400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9784071df0 con 0x7f97840c8200 2024-03-04T07:40:34.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f978b400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97841597b0 con 0x7f9784152bd0 2024-03-04T07:40:34.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f9788a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f97841527f0 0x7f9784072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52674/0 (socket says 172.21.15.113:52674) 2024-03-04T07:40:34.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f9788a00640 1 -- 172.21.15.113:0/556754448 learned_addr learned my addr 172.21.15.113:0/556754448 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:34.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f9789400640 1 --2- 172.21.15.113:0/556754448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 0x7f97841587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:34.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.928+0000 7f977be00640 1 --2- 172.21.15.113:0/556754448 >> v2:172.21.15.136:3300/0 conn(0x7f97840c8200 0x7f9784070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:34.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.932+0000 7f9783e00640 1 -- 172.21.15.113:0/556754448 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370978445 0 0) 0x7f97841591f0 con 0x7f97841527f0 2024-03-04T07:40:34.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.932+0000 7f9783e00640 1 -- 172.21.15.113:0/556754448 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9768003530 con 0x7f97841527f0 2024-03-04T07:40:34.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.932+0000 7f9789400640 1 -- 172.21.15.113:0/556754448 >> v2:172.21.15.136:3300/0 conn(0x7f97840c8200 msgr2=0x7f9784070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:34.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.932+0000 7f9789400640 1 --2- 172.21.15.113:0/556754448 >> v2:172.21.15.136:3300/0 conn(0x7f97840c8200 0x7f9784070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:34.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.932+0000 7f9789400640 1 -- 172.21.15.113:0/556754448 >> v1:172.21.15.113:6789/0 conn(0x7f97841527f0 legacy=0x7f9784072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:34.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.932+0000 7f9789400640 1 -- 172.21.15.113:0/556754448 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97841594d0 con 0x7f9784152bd0 2024-03-04T07:40:34.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.932+0000 7f977be00640 1 --2- 172.21.15.113:0/556754448 >> v2:172.21.15.136:3300/0 conn(0x7f97840c8200 0x7f9784070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:40:34.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f9789400640 1 --2- 172.21.15.113:0/556754448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 0x7f97841587d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f977c009d90 tx=0x7f977c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb97529fa9088fdd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f9783e00640 1 -- 172.21.15.113:0/556754448 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0b2070 con 0x7f9784152bd0 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f9783e00640 1 -- 172.21.15.113:0/556754448 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c0a5920 con 0x7f9784152bd0 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f9783e00640 1 -- 172.21.15.113:0/556754448 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0a5cd0 con 0x7f9784152bd0 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f978b400640 1 -- 172.21.15.113:0/556754448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 msgr2=0x7f97841587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f978b400640 1 --2- 172.21.15.113:0/556754448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 0x7f97841587d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f977c009d90 tx=0x7f977c01c970 comp rx=0 tx=0).stop 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f978b400640 1 -- 172.21.15.113:0/556754448 shutdown_connections 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f978b400640 1 --2- 172.21.15.113:0/556754448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 0x7f97841587d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f978b400640 1 --2- 172.21.15.113:0/556754448 >> v2:172.21.15.136:3300/0 conn(0x7f97840c8200 0x7f9784070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f978b400640 1 -- 172.21.15.113:0/556754448 >> 172.21.15.113:0/556754448 conn(0x7f978406e820 msgr2=0x7f978406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f978b400640 1 -- 172.21.15.113:0/556754448 shutdown_connections 2024-03-04T07:40:34.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f978b400640 1 -- 172.21.15.113:0/556754448 wait complete. 2024-03-04T07:40:34.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.936+0000 7f978b400640 1 Processor -- start 2024-03-04T07:40:34.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f978b400640 1 -- start start 2024-03-04T07:40:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f978b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97840c8200 0x7f97841f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f978b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97841527f0 0x7f97840c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f978b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 0x7f9784151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f978b400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97841654f0 con 0x7f97841527f0 2024-03-04T07:40:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f978b400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9784165370 con 0x7f97840c8200 2024-03-04T07:40:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f978b400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9784165670 con 0x7f9784152bd0 2024-03-04T07:40:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f977be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97841527f0 0x7f97840c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:34.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f977be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97841527f0 0x7f97840c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49556/0 (socket says 172.21.15.113:49556) 2024-03-04T07:40:34.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f977be00640 1 -- 172.21.15.113:0/1188304133 learned_addr learned my addr 172.21.15.113:0/1188304133 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:34.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f9789400640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 0x7f9784151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:34.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f9788a00640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97840c8200 0x7f97841f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:34.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f977be00640 1 -- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 msgr2=0x7f9784151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:34.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f977be00640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 0x7f9784151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:34.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f977be00640 1 -- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97840c8200 msgr2=0x7f97841f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:34.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f977be00640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97840c8200 0x7f97841f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:34.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f977be00640 1 -- 172.21.15.113:0/1188304133 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978414fcf0 con 0x7f97841527f0 2024-03-04T07:40:34.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f977be00640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97841527f0 0x7f97840c5f50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f976c00ea00 tx=0x7f976c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:34.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f9781600640 1 -- 172.21.15.113:0/1188304133 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c017070 con 0x7f97841527f0 2024-03-04T07:40:34.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f9781600640 1 -- 172.21.15.113:0/1188304133 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c00ca60 con 0x7f97841527f0 2024-03-04T07:40:34.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f9781600640 1 -- 172.21.15.113:0/1188304133 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c00ce50 con 0x7f97841527f0 2024-03-04T07:40:34.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f978b400640 1 -- 172.21.15.113:0/1188304133 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978414ff90 con 0x7f97841527f0 2024-03-04T07:40:34.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f978b400640 1 -- 172.21.15.113:0/1188304133 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9784150510 con 0x7f97841527f0 2024-03-04T07:40:34.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.940+0000 7f978b400640 1 -- 172.21.15.113:0/1188304133 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9784075c00 con 0x7f97841527f0 2024-03-04T07:40:34.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.944+0000 7f9781600640 1 -- 172.21.15.113:0/1188304133 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f976c01a070 con 0x7f97841527f0 2024-03-04T07:40:34.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.944+0000 7f9781600640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9760066f60 0x7f9760069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:34.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.944+0000 7f9781600640 1 -- 172.21.15.113:0/1188304133 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f976c084d20 con 0x7f97841527f0 2024-03-04T07:40:34.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.944+0000 7f9788a00640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9760066f60 0x7f9760069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:34.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.944+0000 7f9788a00640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9760066f60 0x7f9760069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9774007fd0 tx=0x7f9774009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:34.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:34.948+0000 7f9781600640 1 -- 172.21.15.113:0/1188304133 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f976c055500 con 0x7f97841527f0 2024-03-04T07:40:35.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.140+0000 7f978b400640 1 -- 172.21.15.113:0/1188304133 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97840c79d0 con 0x7f97841527f0 2024-03-04T07:40:35.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.140+0000 7f9781600640 1 -- 172.21.15.113:0/1188304133 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5813 (secure 0 0 0) 0x7f976c0595a0 con 0x7f97841527f0 2024-03-04T07:40:35.144 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:35.144 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:34.735336+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[5094,4938]},"id":1}]} 2024-03-04T07:40:35.144 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 35 2024-03-04T07:40:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 -- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9760066f60 msgr2=0x7f9760069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9760066f60 0x7f9760069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9774007fd0 tx=0x7f9774009040 comp rx=0 tx=0).stop 2024-03-04T07:40:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 -- 172.21.15.113:0/1188304133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97841527f0 msgr2=0x7f97840c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97841527f0 0x7f97840c5f50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f976c00ea00 tx=0x7f976c00eec0 comp rx=0 tx=0).stop 2024-03-04T07:40:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 -- 172.21.15.113:0/1188304133 shutdown_connections 2024-03-04T07:40:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9784152bd0 0x7f9784151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9760066f60 0x7f9760069420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:35.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97841527f0 0x7f97840c5f50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:35.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 --2- 172.21.15.113:0/1188304133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97840c8200 0x7f97841f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:35.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 -- 172.21.15.113:0/1188304133 >> 172.21.15.113:0/1188304133 conn(0x7f978406e820 msgr2=0x7f9784158390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:35.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 -- 172.21.15.113:0/1188304133 shutdown_connections 2024-03-04T07:40:35.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:35.144+0000 7f978b400640 1 -- 172.21.15.113:0/1188304133 wait complete. 2024-03-04T07:40:35.167 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:40:37.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:37.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf17800640 1 Processor -- start 2024-03-04T07:40:37.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf17800640 1 -- start start 2024-03-04T07:40:37.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf17800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf1006ff20 0x7faf1015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:37.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf17800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7faf100c8b90 0x7faf1015c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:37.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf17800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf10152e30 con 0x7faf10152a50 2024-03-04T07:40:37.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf17800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf100c31b0 con 0x7faf1006ff20 2024-03-04T07:40:37.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf17800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1006f840 con 0x7faf100c8b90 2024-03-04T07:40:37.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf0fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf1006ff20 0x7faf1015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:37.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf0fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7faf1006ff20 0x7faf1015bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39026/0 (socket says 172.21.15.113:39026) 2024-03-04T07:40:37.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf0fe00640 1 -- 172.21.15.113:0/2677701433 learned_addr learned my addr 172.21.15.113:0/2677701433 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:37.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf15800640 1 --2- 172.21.15.113:0/2677701433 >> v2:172.21.15.136:3300/0 conn(0x7faf100c8b90 0x7faf1015c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:37.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf0fe00640 1 -- 172.21.15.113:0/2677701433 >> v2:172.21.15.136:3300/0 conn(0x7faf100c8b90 msgr2=0x7faf1015c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:37.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf0fe00640 1 --2- 172.21.15.113:0/2677701433 >> v2:172.21.15.136:3300/0 conn(0x7faf100c8b90 0x7faf1015c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf0fe00640 1 -- 172.21.15.113:0/2677701433 >> v1:172.21.15.136:6789/0 conn(0x7faf10152a50 legacy=0x7faf100ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:37.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.304+0000 7faf0fe00640 1 -- 172.21.15.113:0/2677701433 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1006f280 con 0x7faf1006ff20 2024-03-04T07:40:37.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf0fe00640 1 --2- 172.21.15.113:0/2677701433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 0x7faf1015bdd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faf1006ef00 tx=0x7faefc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f822a4ac744bb006 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:37.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf0f400640 1 -- 172.21.15.113:0/2677701433 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0aa660 con 0x7faf1006ff20 2024-03-04T07:40:37.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf0f400640 1 -- 172.21.15.113:0/2677701433 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc0af070 con 0x7faf1006ff20 2024-03-04T07:40:37.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf0f400640 1 -- 172.21.15.113:0/2677701433 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0aaa20 con 0x7faf1006ff20 2024-03-04T07:40:37.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf17800640 1 -- 172.21.15.113:0/2677701433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 msgr2=0x7faf1015bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:37.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf17800640 1 --2- 172.21.15.113:0/2677701433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 0x7faf1015bdd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7faf1006ef00 tx=0x7faefc01c4e0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf17800640 1 -- 172.21.15.113:0/2677701433 shutdown_connections 2024-03-04T07:40:37.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf17800640 1 --2- 172.21.15.113:0/2677701433 >> v2:172.21.15.136:3300/0 conn(0x7faf100c8b90 0x7faf1015c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf17800640 1 --2- 172.21.15.113:0/2677701433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 0x7faf1015bdd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf17800640 1 -- 172.21.15.113:0/2677701433 >> 172.21.15.113:0/2677701433 conn(0x7faf100734d0 msgr2=0x7faf10066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:37.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf17800640 1 -- 172.21.15.113:0/2677701433 shutdown_connections 2024-03-04T07:40:37.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.312+0000 7faf17800640 1 -- 172.21.15.113:0/2677701433 wait complete. 2024-03-04T07:40:37.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf17800640 1 Processor -- start 2024-03-04T07:40:37.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf17800640 1 -- start start 2024-03-04T07:40:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf17800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 0x7faf101554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf17800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf100c8b90 0x7faf100c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf17800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf10152a50 0x7faf100c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf17800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf100ce9b0 con 0x7faf1006ff20 2024-03-04T07:40:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf17800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf100ce830 con 0x7faf10152a50 2024-03-04T07:40:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf17800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf10165880 con 0x7faf100c8b90 2024-03-04T07:40:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf14e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 0x7faf101554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf14e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 0x7faf101554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39030/0 (socket says 172.21.15.113:39030) 2024-03-04T07:40:37.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf14e00640 1 -- 172.21.15.113:0/3773668246 learned_addr learned my addr 172.21.15.113:0/3773668246 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:37.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf0fe00640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf100c8b90 0x7faf100c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:37.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf15800640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf10152a50 0x7faf100c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:37.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf14e00640 1 -- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf100c8b90 msgr2=0x7faf100c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:37.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf14e00640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf100c8b90 0x7faf100c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf14e00640 1 -- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf10152a50 msgr2=0x7faf100c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:37.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf14e00640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf10152a50 0x7faf100c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf14e00640 1 -- 172.21.15.113:0/3773668246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf100c76c0 con 0x7faf1006ff20 2024-03-04T07:40:37.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf14e00640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 0x7faf101554a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b410 tx=0x7faf0400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:37.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf0cc00640 1 -- 172.21.15.113:0/3773668246 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04015030 con 0x7faf1006ff20 2024-03-04T07:40:37.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf0cc00640 1 -- 172.21.15.113:0/3773668246 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf04004020 con 0x7faf1006ff20 2024-03-04T07:40:37.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf17800640 1 -- 172.21.15.113:0/3773668246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf101f50a0 con 0x7faf1006ff20 2024-03-04T07:40:37.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf0cc00640 1 -- 172.21.15.113:0/3773668246 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04004410 con 0x7faf1006ff20 2024-03-04T07:40:37.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.316+0000 7faf17800640 1 -- 172.21.15.113:0/3773668246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf101f5660 con 0x7faf1006ff20 2024-03-04T07:40:37.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.320+0000 7faf17800640 1 -- 172.21.15.113:0/3773668246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf10075c00 con 0x7faf1006ff20 2024-03-04T07:40:37.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.320+0000 7faf0cc00640 1 -- 172.21.15.113:0/3773668246 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf04018070 con 0x7faf1006ff20 2024-03-04T07:40:37.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.320+0000 7faf0cc00640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faef0066f60 0x7faef0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:37.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.320+0000 7faf0cc00640 1 -- 172.21.15.113:0/3773668246 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7faf04084cf0 con 0x7faf1006ff20 2024-03-04T07:40:37.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.320+0000 7faf0fe00640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faef0066f60 0x7faef0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:37.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.320+0000 7faf0fe00640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faef0066f60 0x7faef0069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faefc0ae000 tx=0x7faefc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:37.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.328+0000 7faf0cc00640 1 -- 172.21.15.113:0/3773668246 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7faf04055440 con 0x7faf1006ff20 2024-03-04T07:40:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.536+0000 7faf17800640 1 -- 172.21.15.113:0/3773668246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf100cd510 con 0x7faf1006ff20 2024-03-04T07:40:37.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.536+0000 7faf0cc00640 1 -- 172.21.15.113:0/3773668246 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5813 (secure 0 0 0) 0x7faf040594e0 con 0x7faf1006ff20 2024-03-04T07:40:37.536 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:37.536 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:34.735336+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[5094,4938]},"id":1}]} 2024-03-04T07:40:37.536 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 35 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 -- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faef0066f60 msgr2=0x7faef0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faef0066f60 0x7faef0069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faefc0ae000 tx=0x7faefc0a7000 comp rx=0 tx=0).stop 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 -- 172.21.15.113:0/3773668246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 msgr2=0x7faf101554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 0x7faf101554a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7faf0400b410 tx=0x7faf0400b8d0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 -- 172.21.15.113:0/3773668246 shutdown_connections 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf10152a50 0x7faf100c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf100c8b90 0x7faf100c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faef0066f60 0x7faef0069420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 --2- 172.21.15.113:0/3773668246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf1006ff20 0x7faf101554a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 -- 172.21.15.113:0/3773668246 >> 172.21.15.113:0/3773668246 conn(0x7faf100734d0 msgr2=0x7faf100cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:37.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 -- 172.21.15.113:0/3773668246 shutdown_connections 2024-03-04T07:40:37.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:37.540+0000 7faf17800640 1 -- 172.21.15.113:0/3773668246 wait complete. 2024-03-04T07:40:37.557 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:40:39.558 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02da800640 1 Processor -- start 2024-03-04T07:40:39.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02da800640 1 -- start start 2024-03-04T07:40:39.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02da800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d40cb540 0x7f02d415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:39.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02da800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d40c4ce0 con 0x7f02d4152950 2024-03-04T07:40:39.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02da800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d40c4f80 con 0x7f02d406faa0 2024-03-04T07:40:39.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02da800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40c8370 con 0x7f02d40cb540 2024-03-04T07:40:39.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02d9400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f02d4152950 0x7f02d406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48208/0 (socket says 172.21.15.113:48208) 2024-03-04T07:40:39.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02d9400640 1 -- 172.21.15.113:0/3510517199 learned_addr learned my addr 172.21.15.113:0/3510517199 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:39.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02d8a00640 1 --2- 172.21.15.113:0/3510517199 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d40cb540 0x7f02d415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:39.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02d3e00640 1 -- 172.21.15.113:0/3510517199 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805760892 0 0) 0x7f02d40c4ce0 con 0x7f02d4152950 2024-03-04T07:40:39.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02d3e00640 1 -- 172.21.15.113:0/3510517199 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02bc003530 con 0x7f02d4152950 2024-03-04T07:40:39.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02d8a00640 1 -- 172.21.15.113:0/3510517199 >> v1:172.21.15.136:6789/0 conn(0x7f02d406faa0 legacy=0x7f02d4162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:39.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02d3e00640 1 -- 172.21.15.113:0/3510517199 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289638202 0 0) 0x7f02d40c4f80 con 0x7f02d406faa0 2024-03-04T07:40:39.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02d8a00640 1 -- 172.21.15.113:0/3510517199 >> v1:172.21.15.113:6789/0 conn(0x7f02d4152950 legacy=0x7f02d406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:39.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.700+0000 7f02d8a00640 1 -- 172.21.15.113:0/3510517199 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40cb920 con 0x7f02d40cb540 2024-03-04T07:40:39.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02d8a00640 1 --2- 172.21.15.113:0/3510517199 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d40cb540 0x7f02d415bb40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02c00097f0 tx=0x7f02c001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=596673b2c4d36d0b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:39.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02d3e00640 1 -- 172.21.15.113:0/3510517199 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c00b5070 con 0x7f02d40cb540 2024-03-04T07:40:39.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02d3e00640 1 -- 172.21.15.113:0/3510517199 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c00a89d0 con 0x7f02d40cb540 2024-03-04T07:40:39.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02d3e00640 1 -- 172.21.15.113:0/3510517199 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c00a8d80 con 0x7f02d40cb540 2024-03-04T07:40:39.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02da800640 1 -- 172.21.15.113:0/3510517199 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d40cb540 msgr2=0x7f02d415bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:39.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02da800640 1 --2- 172.21.15.113:0/3510517199 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d40cb540 0x7f02d415bb40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f02c00097f0 tx=0x7f02c001ecd0 comp rx=0 tx=0).stop 2024-03-04T07:40:39.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02da800640 1 -- 172.21.15.113:0/3510517199 shutdown_connections 2024-03-04T07:40:39.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02da800640 1 --2- 172.21.15.113:0/3510517199 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d40cb540 0x7f02d415bb40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:39.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02da800640 1 -- 172.21.15.113:0/3510517199 >> 172.21.15.113:0/3510517199 conn(0x7f02d4072f40 msgr2=0x7f02d40706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:39.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02da800640 1 -- 172.21.15.113:0/3510517199 shutdown_connections 2024-03-04T07:40:39.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.704+0000 7f02da800640 1 -- 172.21.15.113:0/3510517199 wait complete. 2024-03-04T07:40:39.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02da800640 1 Processor -- start 2024-03-04T07:40:39.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02da800640 1 -- start start 2024-03-04T07:40:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02da800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d406faa0 0x7f02d40cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02da800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d40cb540 0x7f02d4159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02da800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d4152950 0x7f02d4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:39.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02da800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40c9a50 con 0x7f02d406faa0 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02da800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40ca0a0 con 0x7f02d40cb540 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02da800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40c9d30 con 0x7f02d4152950 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d406faa0 0x7f02d40cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d406faa0 0x7f02d40cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39044/0 (socket says 172.21.15.113:39044) 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d9400640 1 -- 172.21.15.113:0/283286215 learned_addr learned my addr 172.21.15.113:0/283286215 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d8a00640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d40cb540 0x7f02d4159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d9e00640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d4152950 0x7f02d4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d9400640 1 -- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d4152950 msgr2=0x7f02d4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d9400640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d4152950 0x7f02d4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d9400640 1 -- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d40cb540 msgr2=0x7f02d4159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d9400640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d40cb540 0x7f02d4159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d9400640 1 -- 172.21.15.113:0/283286215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d41f0270 con 0x7f02d406faa0 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d8a00640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d40cb540 0x7f02d4159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:40:39.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d9400640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d406faa0 0x7f02d40cd650 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f02c80010f0 tx=0x7f02c800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:39.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d1600640 1 -- 172.21.15.113:0/283286215 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8017070 con 0x7f02d406faa0 2024-03-04T07:40:39.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d1600640 1 -- 172.21.15.113:0/283286215 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c8005d10 con 0x7f02d406faa0 2024-03-04T07:40:39.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02d1600640 1 -- 172.21.15.113:0/283286215 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8007310 con 0x7f02d406faa0 2024-03-04T07:40:39.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02da800640 1 -- 172.21.15.113:0/283286215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d41f0550 con 0x7f02d406faa0 2024-03-04T07:40:39.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.708+0000 7f02da800640 1 -- 172.21.15.113:0/283286215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d41f0b10 con 0x7f02d406faa0 2024-03-04T07:40:39.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.712+0000 7f02da800640 1 -- 172.21.15.113:0/283286215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d4075a80 con 0x7f02d406faa0 2024-03-04T07:40:39.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.712+0000 7f02d1600640 1 -- 172.21.15.113:0/283286215 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02c801a070 con 0x7f02d406faa0 2024-03-04T07:40:39.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.712+0000 7f02d1600640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f02b4066f60 0x7f02b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:39.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.712+0000 7f02d1600640 1 -- 172.21.15.113:0/283286215 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f02c8084530 con 0x7f02d406faa0 2024-03-04T07:40:39.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.712+0000 7f02d8a00640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f02b4066f60 0x7f02b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.712+0000 7f02d8a00640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f02b4066f60 0x7f02b4069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f02c00b2000 tx=0x7f02c00aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:39.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.716+0000 7f02d1600640 1 -- 172.21.15.113:0/283286215 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f02c8054d10 con 0x7f02d406faa0 2024-03-04T07:40:39.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.988+0000 7f02da800640 1 -- 172.21.15.113:0/283286215 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02d406f5b0 con 0x7f02d406faa0 2024-03-04T07:40:39.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.988+0000 7f02d1600640 1 -- 172.21.15.113:0/283286215 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5813 (secure 0 0 0) 0x7f02c8058db0 con 0x7f02d406faa0 2024-03-04T07:40:39.989 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:39.989 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:34.735336+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[5094,4938]},"id":1}]} 2024-03-04T07:40:39.989 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 35 2024-03-04T07:40:39.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 -- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f02b4066f60 msgr2=0x7f02b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f02b4066f60 0x7f02b4069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f02c00b2000 tx=0x7f02c00aa070 comp rx=0 tx=0).stop 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 -- 172.21.15.113:0/283286215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d406faa0 msgr2=0x7f02d40cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d406faa0 0x7f02d40cd650 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f02c80010f0 tx=0x7f02c800c890 comp rx=0 tx=0).stop 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 -- 172.21.15.113:0/283286215 shutdown_connections 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f02d4152950 0x7f02d4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02d40cb540 0x7f02d4159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f02b4066f60 0x7f02b4069420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 --2- 172.21.15.113:0/283286215 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02d406faa0 0x7f02d40cd650 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 -- 172.21.15.113:0/283286215 >> 172.21.15.113:0/283286215 conn(0x7f02d4072f40 msgr2=0x7f02d41585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 -- 172.21.15.113:0/283286215 shutdown_connections 2024-03-04T07:40:39.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:39.992+0000 7f02da800640 1 -- 172.21.15.113:0/283286215 wait complete. 2024-03-04T07:40:40.010 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:40:42.010 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:42.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23e5800640 1 Processor -- start 2024-03-04T07:40:42.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23e5800640 1 -- start start 2024-03-04T07:40:42.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23e5800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e0155100 0x7f23e00c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:42.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23e5800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e015efd0 con 0x7f23e006b060 2024-03-04T07:40:42.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23e5800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e00c7ef0 con 0x7f23e015ad30 2024-03-04T07:40:42.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23e5800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00c8890 con 0x7f23e0155100 2024-03-04T07:40:42.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23d6000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f23e006b060 0x7f23e00cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48210/0 (socket says 172.21.15.113:48210) 2024-03-04T07:40:42.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23d6000640 1 -- 172.21.15.113:0/3725053975 learned_addr learned my addr 172.21.15.113:0/3725053975 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:42.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23dea00640 1 --2- 172.21.15.113:0/3725053975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e0155100 0x7f23e00c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:42.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23de000640 1 -- 172.21.15.113:0/3725053975 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667433238 0 0) 0x7f23e015efd0 con 0x7f23e006b060 2024-03-04T07:40:42.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.152+0000 7f23de000640 1 -- 172.21.15.113:0/3725053975 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c4003530 con 0x7f23e006b060 2024-03-04T07:40:42.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.156+0000 7f23de000640 1 -- 172.21.15.113:0/3725053975 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613218304 0 0) 0x7f23c4003530 con 0x7f23e006b060 2024-03-04T07:40:42.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.156+0000 7f23dea00640 1 -- 172.21.15.113:0/3725053975 >> v1:172.21.15.136:6789/0 conn(0x7f23e015ad30 legacy=0x7f23e015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:42.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.156+0000 7f23dea00640 1 -- 172.21.15.113:0/3725053975 >> v1:172.21.15.113:6789/0 conn(0x7f23e006b060 legacy=0x7f23e00cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:42.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.156+0000 7f23dea00640 1 -- 172.21.15.113:0/3725053975 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e00c81d0 con 0x7f23e0155100 2024-03-04T07:40:42.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.156+0000 7f23dea00640 1 --2- 172.21.15.113:0/3725053975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e0155100 0x7f23e00c08d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23d00097f0 tx=0x7f23d001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6d9e99fbd965bf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:42.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.156+0000 7f23de000640 1 -- 172.21.15.113:0/3725053975 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00b0070 con 0x7f23e0155100 2024-03-04T07:40:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.156+0000 7f23de000640 1 -- 172.21.15.113:0/3725053975 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d00ab400 con 0x7f23e0155100 2024-03-04T07:40:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.156+0000 7f23de000640 1 -- 172.21.15.113:0/3725053975 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8001680 con 0x7f23e0155100 2024-03-04T07:40:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.160+0000 7f23e5800640 1 -- 172.21.15.113:0/3725053975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e0155100 msgr2=0x7f23e00c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.160+0000 7f23e5800640 1 --2- 172.21.15.113:0/3725053975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e0155100 0x7f23e00c08d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23d00097f0 tx=0x7f23d001c400 comp rx=0 tx=0).stop 2024-03-04T07:40:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.160+0000 7f23e5800640 1 -- 172.21.15.113:0/3725053975 shutdown_connections 2024-03-04T07:40:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.160+0000 7f23e5800640 1 --2- 172.21.15.113:0/3725053975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e0155100 0x7f23e00c08d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.160+0000 7f23e5800640 1 -- 172.21.15.113:0/3725053975 >> 172.21.15.113:0/3725053975 conn(0x7f23e006deb0 msgr2=0x7f23e006e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.160+0000 7f23e5800640 1 -- 172.21.15.113:0/3725053975 shutdown_connections 2024-03-04T07:40:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.160+0000 7f23e5800640 1 -- 172.21.15.113:0/3725053975 wait complete. 2024-03-04T07:40:42.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.160+0000 7f23e5800640 1 Processor -- start 2024-03-04T07:40:42.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.160+0000 7f23e5800640 1 -- start start 2024-03-04T07:40:42.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23e5800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e006b060 0x7f23e006c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:42.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23e5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e0155100 0x7f23e0151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:42.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23e5800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e015ad30 0x7f23e014ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:42.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23e5800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00cd950 con 0x7f23e0155100 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23e5800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00cd7d0 con 0x7f23e006b060 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23e5800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e0168d40 con 0x7f23e015ad30 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e0155100 0x7f23e0151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e0155100 0x7f23e0151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39056/0 (socket says 172.21.15.113:39056) 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d6000640 1 -- 172.21.15.113:0/1031782377 learned_addr learned my addr 172.21.15.113:0/1031782377 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23df400640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e015ad30 0x7f23e014ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d6000640 1 -- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e015ad30 msgr2=0x7f23e014ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d6000640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e015ad30 0x7f23e014ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d6000640 1 -- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e006b060 msgr2=0x7f23e006c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d6000640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e006b060 0x7f23e006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d6000640 1 -- 172.21.15.113:0/1031782377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e0155a00 con 0x7f23e0155100 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d6000640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e0155100 0x7f23e0151ca0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f23c800c720 tx=0x7f23c800cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d7400640 1 -- 172.21.15.113:0/1031782377 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8005ae0 con 0x7f23e0155100 2024-03-04T07:40:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d7400640 1 -- 172.21.15.113:0/1031782377 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c8005c80 con 0x7f23e0155100 2024-03-04T07:40:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23e5800640 1 -- 172.21.15.113:0/1031782377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e01f44a0 con 0x7f23e0155100 2024-03-04T07:40:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23d7400640 1 -- 172.21.15.113:0/1031782377 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8007270 con 0x7f23e0155100 2024-03-04T07:40:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23e5800640 1 -- 172.21.15.113:0/1031782377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e0154400 con 0x7f23e0155100 2024-03-04T07:40:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.164+0000 7f23e5800640 1 -- 172.21.15.113:0/1031782377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e0075c00 con 0x7f23e0155100 2024-03-04T07:40:42.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.168+0000 7f23d7400640 1 -- 172.21.15.113:0/1031782377 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23c8007410 con 0x7f23e0155100 2024-03-04T07:40:42.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.168+0000 7f23d7400640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23bc066f60 0x7f23bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:42.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.168+0000 7f23d7400640 1 -- 172.21.15.113:0/1031782377 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f23c8084f70 con 0x7f23e0155100 2024-03-04T07:40:42.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.168+0000 7f23dea00640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23bc066f60 0x7f23bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:42.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.168+0000 7f23dea00640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23bc066f60 0x7f23bc069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f23d00ad000 tx=0x7f23d0004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:42.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.172+0000 7f23d7400640 1 -- 172.21.15.113:0/1031782377 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f23e0155a00 con 0x7f23e0155100 2024-03-04T07:40:42.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 -- 172.21.15.113:0/1031782377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23e00c8a20 con 0x7f23e0155100 2024-03-04T07:40:42.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23d7400640 1 -- 172.21.15.113:0/1031782377 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f23c8055e40 con 0x7f23e0155100 2024-03-04T07:40:42.366 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:42.366 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:39.846406+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:clientreplay","state_seq":24,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[4938,5094]},"id":1}]} 2024-03-04T07:40:42.366 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 36 2024-03-04T07:40:42.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 -- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23bc066f60 msgr2=0x7f23bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:42.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23bc066f60 0x7f23bc069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f23d00ad000 tx=0x7f23d0004740 comp rx=0 tx=0).stop 2024-03-04T07:40:42.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 -- 172.21.15.113:0/1031782377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e0155100 msgr2=0x7f23e0151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:42.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e0155100 0x7f23e0151ca0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f23c800c720 tx=0x7f23c800cb20 comp rx=0 tx=0).stop 2024-03-04T07:40:42.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 -- 172.21.15.113:0/1031782377 shutdown_connections 2024-03-04T07:40:42.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e015ad30 0x7f23e014ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:42.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23bc066f60 0x7f23bc069420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:42.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e0155100 0x7f23e0151ca0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:42.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 --2- 172.21.15.113:0/1031782377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e006b060 0x7f23e006c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:42.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 -- 172.21.15.113:0/1031782377 >> 172.21.15.113:0/1031782377 conn(0x7f23e006deb0 msgr2=0x7f23e00cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:42.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 -- 172.21.15.113:0/1031782377 shutdown_connections 2024-03-04T07:40:42.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:42.368+0000 7f23e5800640 1 -- 172.21.15.113:0/1031782377 wait complete. 2024-03-04T07:40:42.397 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 36, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 36, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4938": { "addr": "172.21.15.113:6836/2997246507", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2997246507, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2997246507, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4938, "incarnation": 20, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 25 }, "gid_5094": { "addr": "172.21.15.113:6833/605142559", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 605142559, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 605142559, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5094, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 21 }, "gid_5244": { "addr": "172.21.15.113:6837/742633123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 742633123, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 742633123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5244, "incarnation": 32, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:clientreplay", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:40:39.846406+0000", "qdb_cluster": [ 4938, 5094 ], "qdb_leader": 5094, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5244, "mds_1": 5094, "mds_2": 4938 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:40:44.399 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:44.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97fb000640 1 Processor -- start 2024-03-04T07:40:44.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97fb000640 1 -- start start 2024-03-04T07:40:44.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97fb000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f414e4c0 0x7f97f4072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:44.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97fb000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97f40c22e0 0x7f97f40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:44.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97fb000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f40c27f0 con 0x7f97f40c2d80 2024-03-04T07:40:44.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97fb000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f415ad60 con 0x7f97f40c22e0 2024-03-04T07:40:44.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97fb000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f40c2a90 con 0x7f97f414e4c0 2024-03-04T07:40:44.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f414e4c0 0x7f97f4072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:44.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97f3400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97f40c22e0 0x7f97f40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:44.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97f3400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97f40c22e0 0x7f97f40731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:38930/0 (socket says 172.21.15.113:38930) 2024-03-04T07:40:44.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97f3400640 1 -- 172.21.15.113:0/14828848 learned_addr learned my addr 172.21.15.113:0/14828848 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:44.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97f3400640 1 -- 172.21.15.113:0/14828848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f414e4c0 msgr2=0x7f97f4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:44.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97f3400640 1 --2- 172.21.15.113:0/14828848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f414e4c0 0x7f97f4072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:44.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97f3400640 1 -- 172.21.15.113:0/14828848 >> v1:172.21.15.136:6789/0 conn(0x7f97f40c2d80 legacy=0x7f97f415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:44.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97f3400640 1 -- 172.21.15.113:0/14828848 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4070c60 con 0x7f97f40c22e0 2024-03-04T07:40:44.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.568+0000 7f97f3e00640 1 --2- 172.21.15.113:0/14828848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f414e4c0 0x7f97f4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:40:44.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97f3400640 1 --2- 172.21.15.113:0/14828848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f40c22e0 0x7f97f40731c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009870 tx=0x7f97e001ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f6c4863fe587aca server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:44.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97f2a00640 1 -- 172.21.15.113:0/14828848 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00b5070 con 0x7f97f40c22e0 2024-03-04T07:40:44.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97f2a00640 1 -- 172.21.15.113:0/14828848 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e00a8a50 con 0x7f97f40c22e0 2024-03-04T07:40:44.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97f2a00640 1 -- 172.21.15.113:0/14828848 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00a8e20 con 0x7f97f40c22e0 2024-03-04T07:40:44.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97fb000640 1 -- 172.21.15.113:0/14828848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f40c22e0 msgr2=0x7f97f40731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:44.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97fb000640 1 --2- 172.21.15.113:0/14828848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f40c22e0 0x7f97f40731c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009870 tx=0x7f97e001ed50 comp rx=0 tx=0).stop 2024-03-04T07:40:44.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97fb000640 1 -- 172.21.15.113:0/14828848 shutdown_connections 2024-03-04T07:40:44.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97fb000640 1 --2- 172.21.15.113:0/14828848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f40c22e0 0x7f97f40731c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:44.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97fb000640 1 --2- 172.21.15.113:0/14828848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f414e4c0 0x7f97f4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:44.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97fb000640 1 -- 172.21.15.113:0/14828848 >> 172.21.15.113:0/14828848 conn(0x7f97f406e820 msgr2=0x7f97f406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:44.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97fb000640 1 -- 172.21.15.113:0/14828848 shutdown_connections 2024-03-04T07:40:44.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.572+0000 7f97fb000640 1 -- 172.21.15.113:0/14828848 wait complete. 2024-03-04T07:40:44.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97fb000640 1 Processor -- start 2024-03-04T07:40:44.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97fb000640 1 -- start start 2024-03-04T07:40:44.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97fb000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f40c22e0 0x7f97f40cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:44.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97fb000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f40c2d80 0x7f97f41ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:44.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97fb000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f414e4c0 0x7f97f41eee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:44.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97fb000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4071500 con 0x7f97f40c22e0 2024-03-04T07:40:44.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97fb000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4071220 con 0x7f97f414e4c0 2024-03-04T07:40:44.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97fb000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f414e8a0 con 0x7f97f40c2d80 2024-03-04T07:40:44.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f40c22e0 0x7f97f40cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:44.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f40c22e0 0x7f97f40cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39064/0 (socket says 172.21.15.113:39064) 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f3e00640 1 -- 172.21.15.113:0/4160082219 learned_addr learned my addr 172.21.15.113:0/4160082219 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f3400640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f40c2d80 0x7f97f41ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f9000640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f414e4c0 0x7f97f41eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f3e00640 1 -- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f40c2d80 msgr2=0x7f97f41ee8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f3e00640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f40c2d80 0x7f97f41ee8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f3e00640 1 -- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f414e4c0 msgr2=0x7f97f41eee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f3e00640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f414e4c0 0x7f97f41eee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f3e00640 1 -- 172.21.15.113:0/4160082219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4142c30 con 0x7f97f40c22e0 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f9000640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f414e4c0 0x7f97f41eee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97f3e00640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f40c22e0 0x7f97f40cb350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f97e400e980 tx=0x7f97e400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97ebe00640 1 -- 172.21.15.113:0/4160082219 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4017070 con 0x7f97f40c22e0 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97ebe00640 1 -- 172.21.15.113:0/4160082219 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e400c9a0 con 0x7f97f40c22e0 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97fb000640 1 -- 172.21.15.113:0/4160082219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f40c5c80 con 0x7f97f40c22e0 2024-03-04T07:40:44.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97ebe00640 1 -- 172.21.15.113:0/4160082219 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e400cd70 con 0x7f97f40c22e0 2024-03-04T07:40:44.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.576+0000 7f97fb000640 1 -- 172.21.15.113:0/4160082219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f41f2370 con 0x7f97f40c22e0 2024-03-04T07:40:44.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.580+0000 7f97fb000640 1 -- 172.21.15.113:0/4160082219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f4075c00 con 0x7f97f40c22e0 2024-03-04T07:40:44.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.580+0000 7f97ebe00640 1 -- 172.21.15.113:0/4160082219 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97e401a070 con 0x7f97f40c22e0 2024-03-04T07:40:44.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.580+0000 7f97ebe00640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 0x7f97d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:44.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.580+0000 7f97ebe00640 1 -- 172.21.15.113:0/4160082219 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f97e40852b0 con 0x7f97f40c22e0 2024-03-04T07:40:44.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.580+0000 7f97f3400640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 0x7f97d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:44.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.584+0000 7f97f3400640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 0x7f97d4069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97e00b2000 tx=0x7f97e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:44.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.584+0000 7f97ebe00640 1 -- 172.21.15.113:0/4160082219 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f97e4055b30 con 0x7f97f40c22e0 2024-03-04T07:40:44.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.788+0000 7f97fb000640 1 -- 172.21.15.113:0/4160082219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97f4155190 con 0x7f97f40c22e0 2024-03-04T07:40:44.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.788+0000 7f97ebe00640 1 -- 172.21.15.113:0/4160082219 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f97e4059bd0 con 0x7f97f40c22e0 2024-03-04T07:40:44.788 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:39.846406+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:clientreplay","state_seq":24,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5094,"qdb_cluster":[4938,5094]},"id":1}]} 2024-03-04T07:40:44.788 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 36 2024-03-04T07:40:44.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 -- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 msgr2=0x7f97d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:44.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 0x7f97d4069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f97e00b2000 tx=0x7f97e0009040 comp rx=0 tx=0).stop 2024-03-04T07:40:44.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 -- 172.21.15.113:0/4160082219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f40c22e0 msgr2=0x7f97f40cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:44.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f40c22e0 0x7f97f40cb350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f97e400e980 tx=0x7f97e400ee40 comp rx=0 tx=0).stop 2024-03-04T07:40:44.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 -- 172.21.15.113:0/4160082219 shutdown_connections 2024-03-04T07:40:44.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f414e4c0 0x7f97f41eee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:44.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f40c2d80 0x7f97f41ee8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:44.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 0x7f97d4069420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:44.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 --2- 172.21.15.113:0/4160082219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f40c22e0 0x7f97f40cb350 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:44.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 -- 172.21.15.113:0/4160082219 >> 172.21.15.113:0/4160082219 conn(0x7f97f406e820 msgr2=0x7f97f40c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:44.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 -- 172.21.15.113:0/4160082219 shutdown_connections 2024-03-04T07:40:44.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:44.792+0000 7f97fb000640 1 -- 172.21.15.113:0/4160082219 wait complete. 2024-03-04T07:40:44.819 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:40:46.820 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:46.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f55600640 1 Processor -- start 2024-03-04T07:40:46.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f55600640 1 -- start start 2024-03-04T07:40:46.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f55600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0f500c8200 0x7f0f50070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:46.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f55600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f50152bd0 0x7f0f501587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:46.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f55600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f501591f0 con 0x7f0f501527f0 2024-03-04T07:40:46.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f55600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f50071df0 con 0x7f0f50152bd0 2024-03-04T07:40:46.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f55600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f501597b0 con 0x7f0f500c8200 2024-03-04T07:40:46.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f4ea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0f501527f0 0x7f0f50072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48216/0 (socket says 172.21.15.113:48216) 2024-03-04T07:40:46.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f4ea00640 1 -- 172.21.15.113:0/357138584 learned_addr learned my addr 172.21.15.113:0/357138584 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:40:46.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f4f400640 1 --2- 172.21.15.113:0/357138584 >> v2:172.21.15.113:3300/0 conn(0x7f0f50152bd0 0x7f0f501587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:46.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f47e00640 1 --2- 172.21.15.113:0/357138584 >> v2:172.21.15.136:3300/0 conn(0x7f0f500c8200 0x7f0f50070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:46.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f4e000640 1 -- 172.21.15.113:0/357138584 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034405025 0 0) 0x7f0f501591f0 con 0x7f0f501527f0 2024-03-04T07:40:46.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f4e000640 1 -- 172.21.15.113:0/357138584 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f34003530 con 0x7f0f501527f0 2024-03-04T07:40:46.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f4f400640 1 -- 172.21.15.113:0/357138584 >> v2:172.21.15.136:3300/0 conn(0x7f0f500c8200 msgr2=0x7f0f50070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:46.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f4f400640 1 --2- 172.21.15.113:0/357138584 >> v2:172.21.15.136:3300/0 conn(0x7f0f500c8200 0x7f0f50070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:46.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f4f400640 1 -- 172.21.15.113:0/357138584 >> v1:172.21.15.113:6789/0 conn(0x7f0f501527f0 legacy=0x7f0f50072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:46.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f4f400640 1 -- 172.21.15.113:0/357138584 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f501594d0 con 0x7f0f50152bd0 2024-03-04T07:40:46.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.964+0000 7f0f47e00640 1 --2- 172.21.15.113:0/357138584 >> v2:172.21.15.136:3300/0 conn(0x7f0f500c8200 0x7f0f50070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:46.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f4f400640 1 --2- 172.21.15.113:0/357138584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f50152bd0 0x7f0f501587d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0f500cd230 tx=0x7f0f4801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c628d1602e781c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:46.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f4e000640 1 -- 172.21.15.113:0/357138584 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480b2070 con 0x7f0f50152bd0 2024-03-04T07:40:46.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f4e000640 1 -- 172.21.15.113:0/357138584 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f480a58e0 con 0x7f0f50152bd0 2024-03-04T07:40:46.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f4e000640 1 -- 172.21.15.113:0/357138584 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480a5cb0 con 0x7f0f50152bd0 2024-03-04T07:40:46.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f55600640 1 -- 172.21.15.113:0/357138584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f50152bd0 msgr2=0x7f0f501587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:46.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f55600640 1 --2- 172.21.15.113:0/357138584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f50152bd0 0x7f0f501587d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0f500cd230 tx=0x7f0f4801c930 comp rx=0 tx=0).stop 2024-03-04T07:40:46.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f55600640 1 -- 172.21.15.113:0/357138584 shutdown_connections 2024-03-04T07:40:46.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f55600640 1 --2- 172.21.15.113:0/357138584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f50152bd0 0x7f0f501587d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:46.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f55600640 1 --2- 172.21.15.113:0/357138584 >> v2:172.21.15.136:3300/0 conn(0x7f0f500c8200 0x7f0f50070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:46.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f55600640 1 -- 172.21.15.113:0/357138584 >> 172.21.15.113:0/357138584 conn(0x7f0f5006e820 msgr2=0x7f0f5006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:46.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f55600640 1 -- 172.21.15.113:0/357138584 shutdown_connections 2024-03-04T07:40:46.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f55600640 1 -- 172.21.15.113:0/357138584 wait complete. 2024-03-04T07:40:46.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f55600640 1 Processor -- start 2024-03-04T07:40:46.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.968+0000 7f0f55600640 1 -- start start 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f55600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f500c8200 0x7f0f500b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f55600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f501527f0 0x7f0f5014a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f55600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f50152bd0 0x7f0f50150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f55600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f501654f0 con 0x7f0f500c8200 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f55600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50165370 con 0x7f0f501527f0 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f55600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50165670 con 0x7f0f50152bd0 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f4ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f500c8200 0x7f0f500b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f4ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f500c8200 0x7f0f500b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39086/0 (socket says 172.21.15.113:39086) 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f4ea00640 1 -- 172.21.15.113:0/2262527200 learned_addr learned my addr 172.21.15.113:0/2262527200 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f47e00640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f501527f0 0x7f0f5014a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f4f400640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f50152bd0 0x7f0f50150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:46.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f4ea00640 1 -- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f50152bd0 msgr2=0x7f0f50150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:46.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f4ea00640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f50152bd0 0x7f0f50150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:46.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f4ea00640 1 -- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f501527f0 msgr2=0x7f0f5014a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:46.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f4ea00640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f501527f0 0x7f0f5014a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:46.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f4ea00640 1 -- 172.21.15.113:0/2262527200 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f501f4e60 con 0x7f0f500c8200 2024-03-04T07:40:46.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f4ea00640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f500c8200 0x7f0f500b1130 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000c480 tx=0x7f0f4000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:46.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f46a00640 1 -- 172.21.15.113:0/2262527200 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40017070 con 0x7f0f500c8200 2024-03-04T07:40:46.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f46a00640 1 -- 172.21.15.113:0/2262527200 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f40005cf0 con 0x7f0f500c8200 2024-03-04T07:40:46.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f46a00640 1 -- 172.21.15.113:0/2262527200 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40007290 con 0x7f0f500c8200 2024-03-04T07:40:46.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f55600640 1 -- 172.21.15.113:0/2262527200 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f501f5120 con 0x7f0f500c8200 2024-03-04T07:40:46.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f55600640 1 -- 172.21.15.113:0/2262527200 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f501f56e0 con 0x7f0f500c8200 2024-03-04T07:40:46.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.972+0000 7f0f55600640 1 -- 172.21.15.113:0/2262527200 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f50075c00 con 0x7f0f500c8200 2024-03-04T07:40:46.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.976+0000 7f0f46a00640 1 -- 172.21.15.113:0/2262527200 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f4001a070 con 0x7f0f500c8200 2024-03-04T07:40:46.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.976+0000 7f0f46a00640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f2c066f90 0x7f0f2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:46.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.976+0000 7f0f46a00640 1 -- 172.21.15.113:0/2262527200 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f0f40084680 con 0x7f0f500c8200 2024-03-04T07:40:46.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.976+0000 7f0f47e00640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f2c066f90 0x7f0f2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:46.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.976+0000 7f0f47e00640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f2c066f90 0x7f0f2c069450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f500ca6a0 tx=0x7f0f38008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:46.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:46.984+0000 7f0f46a00640 1 -- 172.21.15.113:0/2262527200 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0f400554b0 con 0x7f0f500c8200 2024-03-04T07:40:47.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.180+0000 7f0f55600640 1 -- 172.21.15.113:0/2262527200 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f500c34c0 con 0x7f0f500c8200 2024-03-04T07:40:47.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.180+0000 7f0f46a00640 1 -- 172.21.15.113:0/2262527200 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f0f40059e20 con 0x7f0f500c8200 2024-03-04T07:40:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:47.181 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:46.796316+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[4938,5094,5244]},"id":1}]} 2024-03-04T07:40:47.181 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 37 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 -- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f2c066f90 msgr2=0x7f0f2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f2c066f90 0x7f0f2c069450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f500ca6a0 tx=0x7f0f38008040 comp rx=0 tx=0).stop 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 -- 172.21.15.113:0/2262527200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f500c8200 msgr2=0x7f0f500b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f500c8200 0x7f0f500b1130 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000c480 tx=0x7f0f4000c940 comp rx=0 tx=0).stop 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 -- 172.21.15.113:0/2262527200 shutdown_connections 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f50152bd0 0x7f0f50150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f501527f0 0x7f0f5014a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f2c066f90 0x7f0f2c069450 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 --2- 172.21.15.113:0/2262527200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f500c8200 0x7f0f500b1130 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 -- 172.21.15.113:0/2262527200 >> 172.21.15.113:0/2262527200 conn(0x7f0f5006e820 msgr2=0x7f0f500c6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:47.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 -- 172.21.15.113:0/2262527200 shutdown_connections 2024-03-04T07:40:47.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:47.184+0000 7f0f55600640 1 -- 172.21.15.113:0/2262527200 wait complete. 2024-03-04T07:40:47.202 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=0, replacing gid=4276 2024-03-04T07:40:47.203 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.b 2024-03-04T07:40:52.311 INFO:tasks.workunit.client.0.smithi113.stdout:removed './128/37' 2024-03-04T07:40:52.311 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './128' 2024-03-04T07:40:52.311 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/19' 2024-03-04T07:40:52.311 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/15' 2024-03-04T07:40:52.311 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/31' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/36' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/20' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/12' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/39' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/10' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/34' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/38' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/11' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/16' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/32' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/30' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/14' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/18' 2024-03-04T07:40:52.312 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/13' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/33' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/17' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/35' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './129/37' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './129' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/19' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/15' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/31' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/36' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/20' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/12' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/39' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/10' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/34' 2024-03-04T07:40:52.313 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/38' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/11' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/16' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/32' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/30' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/14' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/18' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/13' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/33' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/17' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/35' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './130/37' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './130' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/19' 2024-03-04T07:40:52.314 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/15' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/31' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/36' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/20' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/12' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/39' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/10' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/34' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/38' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/11' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/16' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/32' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/30' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/14' 2024-03-04T07:40:52.315 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/18' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/13' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/33' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/17' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/35' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './131/37' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './131' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/19' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/15' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/31' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/36' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/20' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/12' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/39' 2024-03-04T07:40:52.316 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/10' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/34' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/38' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/11' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/16' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/32' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/30' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/14' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/18' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/13' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/33' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/17' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/35' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './132/37' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './132' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/19' 2024-03-04T07:40:52.317 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/15' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/31' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/36' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/20' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/12' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/39' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/10' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/34' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/38' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/11' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/16' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/32' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/30' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/14' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/18' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/13' 2024-03-04T07:40:52.318 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/33' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/17' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/35' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './133/37' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './133' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/19' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/15' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/31' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/36' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/20' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/12' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/39' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/10' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/34' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/38' 2024-03-04T07:40:52.319 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/11' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/16' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/32' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/30' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/14' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/18' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/13' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/33' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/17' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/35' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './134/37' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './134' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/19' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/15' 2024-03-04T07:40:52.320 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/31' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/36' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/20' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/12' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/39' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/10' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/34' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/38' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/11' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/16' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/32' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/30' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/14' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/18' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/13' 2024-03-04T07:40:52.321 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/33' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/17' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/35' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './135/37' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './135' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/19' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/15' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/31' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/36' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/20' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/12' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/39' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/10' 2024-03-04T07:40:52.322 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/34' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/38' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/11' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/16' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/32' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/30' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/14' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/18' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/13' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/33' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/17' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/35' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './136/37' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './136' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/19' 2024-03-04T07:40:52.323 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/15' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/31' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/36' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/20' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/12' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/39' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/10' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/34' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/38' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/11' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/16' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/32' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/30' 2024-03-04T07:40:52.324 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/14' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/18' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/13' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/33' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/17' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/35' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './137/37' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './137' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/19' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/15' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/31' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/36' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/20' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/12' 2024-03-04T07:40:52.325 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/39' 2024-03-04T07:40:52.326 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/10' 2024-03-04T07:40:52.326 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/34' 2024-03-04T07:40:52.326 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/38' 2024-03-04T07:40:52.326 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/11' 2024-03-04T07:40:52.326 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/16' 2024-03-04T07:40:53.204 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2024-03-04T07:40:53.204 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-04T07:40:53.204 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-04T07:40:53.207 INFO:tasks.ceph.mds.b:Started 2024-03-04T07:40:53.207 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:53.285 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2024-03-04T07:40:53.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.348+0000 7f64ef800640 1 Processor -- start 2024-03-04T07:40:53.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.348+0000 7f64ef800640 1 -- start start 2024-03-04T07:40:53.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.348+0000 7f64ef800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64e81527f0 0x7f64e8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:53.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.348+0000 7f64ef800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e8152bd0 0x7f64e81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:53.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.348+0000 7f64ef800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e81591f0 con 0x7f64e80c8200 2024-03-04T07:40:53.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.348+0000 7f64ef800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e8071df0 con 0x7f64e81527f0 2024-03-04T07:40:53.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.348+0000 7f64ef800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e81597b0 con 0x7f64e8152bd0 2024-03-04T07:40:53.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.348+0000 7f64ece00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64e81527f0 0x7f64e8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:53.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64ece00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f64e81527f0 0x7f64e8070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44290/0 (socket says 172.21.15.113:44290) 2024-03-04T07:40:53.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64ece00640 1 -- 172.21.15.113:0/1982206160 learned_addr learned my addr 172.21.15.113:0/1982206160 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:53.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64e7e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f64e80c8200 0x7f64e8072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:49100/0 (socket says 172.21.15.113:49100) 2024-03-04T07:40:53.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64ed800640 1 --2- 172.21.15.113:0/1982206160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e8152bd0 0x7f64e81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:53.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64ece00640 1 -- 172.21.15.113:0/1982206160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e8152bd0 msgr2=0x7f64e81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:53.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64ece00640 1 --2- 172.21.15.113:0/1982206160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e8152bd0 0x7f64e81587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:53.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64ece00640 1 -- 172.21.15.113:0/1982206160 >> v1:172.21.15.136:6789/0 conn(0x7f64e80c8200 legacy=0x7f64e8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:53.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64ece00640 1 -- 172.21.15.113:0/1982206160 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e81594d0 con 0x7f64e81527f0 2024-03-04T07:40:53.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64ece00640 1 --2- 172.21.15.113:0/1982206160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 0x7f64e8070d40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009870 tx=0x7f64dc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b003db2cdd262 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64e7400640 1 -- 172.21.15.113:0/1982206160 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0a7070 con 0x7f64e81527f0 2024-03-04T07:40:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64e7400640 1 -- 172.21.15.113:0/1982206160 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc0b0070 con 0x7f64e81527f0 2024-03-04T07:40:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.352+0000 7f64e7400640 1 -- 172.21.15.113:0/1982206160 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0ab890 con 0x7f64e81527f0 2024-03-04T07:40:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 -- 172.21.15.113:0/1982206160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 msgr2=0x7f64e8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:53.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 --2- 172.21.15.113:0/1982206160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 0x7f64e8070d40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f64dc009870 tx=0x7f64dc01c560 comp rx=0 tx=0).stop 2024-03-04T07:40:53.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 -- 172.21.15.113:0/1982206160 shutdown_connections 2024-03-04T07:40:53.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 --2- 172.21.15.113:0/1982206160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e8152bd0 0x7f64e81587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:53.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 --2- 172.21.15.113:0/1982206160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 0x7f64e8070d40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:53.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 -- 172.21.15.113:0/1982206160 >> 172.21.15.113:0/1982206160 conn(0x7f64e806e820 msgr2=0x7f64e806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:53.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 -- 172.21.15.113:0/1982206160 shutdown_connections 2024-03-04T07:40:53.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 -- 172.21.15.113:0/1982206160 wait complete. 2024-03-04T07:40:53.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 Processor -- start 2024-03-04T07:40:53.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 -- start start 2024-03-04T07:40:53.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e80c8200 0x7f64e8153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:53.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 0x7f64e8157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:53.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e8152bd0 0x7f64e814ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:53.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e81654f0 con 0x7f64e81527f0 2024-03-04T07:40:53.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e8165370 con 0x7f64e8152bd0 2024-03-04T07:40:53.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.356+0000 7f64ef800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e8165670 con 0x7f64e80c8200 2024-03-04T07:40:53.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 0x7f64e8157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:53.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 0x7f64e8157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44300/0 (socket says 172.21.15.113:44300) 2024-03-04T07:40:53.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e7e00640 1 -- 172.21.15.113:0/1824716331 learned_addr learned my addr 172.21.15.113:0/1824716331 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:53.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64ed800640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e8152bd0 0x7f64e814ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64ece00640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e80c8200 0x7f64e8153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e7e00640 1 -- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e80c8200 msgr2=0x7f64e8153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e7e00640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e80c8200 0x7f64e8153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e7e00640 1 -- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e8152bd0 msgr2=0x7f64e814ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e7e00640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e8152bd0 0x7f64e814ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e7e00640 1 -- 172.21.15.113:0/1824716331 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80b1b90 con 0x7f64e81527f0 2024-03-04T07:40:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e7e00640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 0x7f64e8157840 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f64d400c430 tx=0x7f64d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e4c00640 1 -- 172.21.15.113:0/1824716331 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d40074e0 con 0x7f64e81527f0 2024-03-04T07:40:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e4c00640 1 -- 172.21.15.113:0/1824716331 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d4007680 con 0x7f64e81527f0 2024-03-04T07:40:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64ef800640 1 -- 172.21.15.113:0/1824716331 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e80b12f0 con 0x7f64e81527f0 2024-03-04T07:40:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64e4c00640 1 -- 172.21.15.113:0/1824716331 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d4004a30 con 0x7f64e81527f0 2024-03-04T07:40:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64ef800640 1 -- 172.21.15.113:0/1824716331 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e80b18b0 con 0x7f64e81527f0 2024-03-04T07:40:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.360+0000 7f64ef800640 1 -- 172.21.15.113:0/1824716331 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e8075c00 con 0x7f64e81527f0 2024-03-04T07:40:53.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.364+0000 7f64e4c00640 1 -- 172.21.15.113:0/1824716331 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64d401a020 con 0x7f64e81527f0 2024-03-04T07:40:53.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.364+0000 7f64e4c00640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f64c8066f90 0x7f64c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:53.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.364+0000 7f64e4c00640 1 -- 172.21.15.113:0/1824716331 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f64d40847f0 con 0x7f64e81527f0 2024-03-04T07:40:53.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.364+0000 7f64ece00640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f64c8066f90 0x7f64c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:53.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.364+0000 7f64ece00640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f64c8066f90 0x7f64c8069450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0af000 tx=0x7f64dc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:53.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.368+0000 7f64e4c00640 1 -- 172.21.15.113:0/1824716331 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f64d4055070 con 0x7f64e81527f0 2024-03-04T07:40:53.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.564+0000 7f64ef800640 1 -- 172.21.15.113:0/1824716331 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64e80c1f10 con 0x7f64e81527f0 2024-03-04T07:40:53.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64e4c00640 1 -- 172.21.15.113:0/1824716331 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f64d4059110 con 0x7f64e81527f0 2024-03-04T07:40:53.565 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:53.565 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:46.796316+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[4938,5094,5244]},"id":1}]} 2024-03-04T07:40:53.566 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 37 2024-03-04T07:40:53.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 -- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f64c8066f90 msgr2=0x7f64c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:53.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f64c8066f90 0x7f64c8069450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0af000 tx=0x7f64dc0a7000 comp rx=0 tx=0).stop 2024-03-04T07:40:53.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 -- 172.21.15.113:0/1824716331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 msgr2=0x7f64e8157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:53.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 0x7f64e8157840 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f64d400c430 tx=0x7f64d400c8f0 comp rx=0 tx=0).stop 2024-03-04T07:40:53.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 -- 172.21.15.113:0/1824716331 shutdown_connections 2024-03-04T07:40:53.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64e8152bd0 0x7f64e814ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:53.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f64c8066f90 0x7f64c8069450 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:53.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64e81527f0 0x7f64e8157840 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:53.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 --2- 172.21.15.113:0/1824716331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f64e80c8200 0x7f64e8153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:53.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 -- 172.21.15.113:0/1824716331 >> 172.21.15.113:0/1824716331 conn(0x7f64e806e820 msgr2=0x7f64e814d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:53.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 -- 172.21.15.113:0/1824716331 shutdown_connections 2024-03-04T07:40:53.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:53.568+0000 7f64ef800640 1 -- 172.21.15.113:0/1824716331 wait complete. 2024-03-04T07:40:53.588 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2024-03-04T07:40:55.588 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:40:55.651 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T07:40:55.652+0000 7f8872400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T07:40:55.651 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T07:40:55.652+0000 7f8872400640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T07:40:55.651 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T07:40:55.652+0000 7f8872400640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-04T07:40:55.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb990e00640 1 Processor -- start 2024-03-04T07:40:55.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb990e00640 1 -- start start 2024-03-04T07:40:55.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb990e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 0x7fb98c072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:55.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb990e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb98c0c22e0 0x7fb98c0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:55.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb990e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb98c0c2d80 0x7fb98c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:55.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb990e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98c15ad60 con 0x7fb98c0c22e0 2024-03-04T07:40:55.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb990e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98c0c27f0 con 0x7fb98c0c2d80 2024-03-04T07:40:55.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb990e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c0c2a90 con 0x7fb98c14e4c0 2024-03-04T07:40:55.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb981400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb98c0c22e0 0x7fb98c0731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:55.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb981400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb98c0c22e0 0x7fb98c0731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44314/0 (socket says 172.21.15.113:44314) 2024-03-04T07:40:55.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb981400640 1 -- 172.21.15.113:0/1672511420 learned_addr learned my addr 172.21.15.113:0/1672511420 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:55.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb989e00640 1 --2- 172.21.15.113:0/1672511420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 0x7fb98c072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:55.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.756+0000 7fb98a800640 1 --2- 172.21.15.113:0/1672511420 >> v2:172.21.15.136:3300/0 conn(0x7fb98c0c2d80 0x7fb98c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:55.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb981400640 1 -- 172.21.15.113:0/1672511420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 msgr2=0x7fb98c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:55.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb981400640 1 --2- 172.21.15.113:0/1672511420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 0x7fb98c072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:55.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb981400640 1 -- 172.21.15.113:0/1672511420 >> v2:172.21.15.136:3300/0 conn(0x7fb98c0c2d80 msgr2=0x7fb98c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:40:55.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb981400640 1 --2- 172.21.15.113:0/1672511420 >> v2:172.21.15.136:3300/0 conn(0x7fb98c0c2d80 0x7fb98c15b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:55.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb981400640 1 -- 172.21.15.113:0/1672511420 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c070c60 con 0x7fb98c0c22e0 2024-03-04T07:40:55.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb981400640 1 --2- 172.21.15.113:0/1672511420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c22e0 0x7fb98c0731c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0c2700 tx=0x7fb97401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f858c97838c6b736 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:55.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb989400640 1 -- 172.21.15.113:0/1672511420 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9740b5070 con 0x7fb98c0c22e0 2024-03-04T07:40:55.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb989400640 1 -- 172.21.15.113:0/1672511420 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9740a89d0 con 0x7fb98c0c22e0 2024-03-04T07:40:55.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb989400640 1 -- 172.21.15.113:0/1672511420 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9740a8da0 con 0x7fb98c0c22e0 2024-03-04T07:40:55.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb990e00640 1 -- 172.21.15.113:0/1672511420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c22e0 msgr2=0x7fb98c0731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:55.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.760+0000 7fb990e00640 1 --2- 172.21.15.113:0/1672511420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c22e0 0x7fb98c0731c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0c2700 tx=0x7fb97401ed10 comp rx=0 tx=0).stop 2024-03-04T07:40:55.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.764+0000 7fb990e00640 1 -- 172.21.15.113:0/1672511420 shutdown_connections 2024-03-04T07:40:55.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.764+0000 7fb990e00640 1 --2- 172.21.15.113:0/1672511420 >> v2:172.21.15.136:3300/0 conn(0x7fb98c0c2d80 0x7fb98c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:55.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.764+0000 7fb990e00640 1 --2- 172.21.15.113:0/1672511420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c22e0 0x7fb98c0731c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:55.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.764+0000 7fb990e00640 1 --2- 172.21.15.113:0/1672511420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 0x7fb98c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:55.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.764+0000 7fb990e00640 1 -- 172.21.15.113:0/1672511420 >> 172.21.15.113:0/1672511420 conn(0x7fb98c06e820 msgr2=0x7fb98c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:55.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.764+0000 7fb990e00640 1 -- 172.21.15.113:0/1672511420 shutdown_connections 2024-03-04T07:40:55.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.764+0000 7fb990e00640 1 -- 172.21.15.113:0/1672511420 wait complete. 2024-03-04T07:40:55.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.764+0000 7fb990e00640 1 Processor -- start 2024-03-04T07:40:55.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.764+0000 7fb990e00640 1 -- start start 2024-03-04T07:40:55.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb990e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb98c0c22e0 0x7fb98c1f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:55.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb990e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c2d80 0x7fb98c06b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:55.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb990e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 0x7fb98c145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:55.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb990e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c071500 con 0x7fb98c0c2d80 2024-03-04T07:40:55.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb990e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c071220 con 0x7fb98c0c22e0 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb990e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98c14e8a0 con 0x7fb98c14e4c0 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb981400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c2d80 0x7fb98c06b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb981400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c2d80 0x7fb98c06b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44326/0 (socket says 172.21.15.113:44326) 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb981400640 1 -- 172.21.15.113:0/3925426859 learned_addr learned my addr 172.21.15.113:0/3925426859 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb989e00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb98c0c22e0 0x7fb98c1f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb98a800640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 0x7fb98c145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb981400640 1 -- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 msgr2=0x7fb98c145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb981400640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 0x7fb98c145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb981400640 1 -- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb98c0c22e0 msgr2=0x7fb98c1f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb981400640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb98c0c22e0 0x7fb98c1f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb981400640 1 -- 172.21.15.113:0/3925426859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c15a830 con 0x7fb98c0c2d80 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb98a800640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 0x7fb98c145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:55.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb989e00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb98c0c22e0 0x7fb98c1f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:40:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb981400640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c2d80 0x7fb98c06b2c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb97401d3c0 tx=0x7fb97401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb982a00640 1 -- 172.21.15.113:0/3925426859 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9740b5070 con 0x7fb98c0c2d80 2024-03-04T07:40:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb982a00640 1 -- 172.21.15.113:0/3925426859 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9740b0950 con 0x7fb98c0c2d80 2024-03-04T07:40:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb982a00640 1 -- 172.21.15.113:0/3925426859 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9740b0d40 con 0x7fb98c0c2d80 2024-03-04T07:40:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb990e00640 1 -- 172.21.15.113:0/3925426859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98c1589d0 con 0x7fb98c0c2d80 2024-03-04T07:40:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb990e00640 1 -- 172.21.15.113:0/3925426859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98c1f4640 con 0x7fb98c0c2d80 2024-03-04T07:40:55.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.768+0000 7fb990e00640 1 -- 172.21.15.113:0/3925426859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98c075c00 con 0x7fb98c0c2d80 2024-03-04T07:40:55.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.772+0000 7fb982a00640 1 -- 172.21.15.113:0/3925426859 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9740c2060 con 0x7fb98c0c2d80 2024-03-04T07:40:55.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.772+0000 7fb982a00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb968066f60 0x7fb968069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:40:55.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.772+0000 7fb982a00640 1 -- 172.21.15.113:0/3925426859 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fb974122300 con 0x7fb98c0c2d80 2024-03-04T07:40:55.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.772+0000 7fb989e00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb968066f60 0x7fb968069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:40:55.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.776+0000 7fb989e00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb968066f60 0x7fb968069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009a90 tx=0x7fb97c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:40:55.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:55.776+0000 7fb982a00640 1 -- 172.21.15.113:0/3925426859 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb974018870 con 0x7fb98c0c2d80 2024-03-04T07:40:56.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.024+0000 7fb990e00640 1 -- 172.21.15.113:0/3925426859 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb98c0b2350 con 0x7fb98c0c2d80 2024-03-04T07:40:56.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.024+0000 7fb982a00640 1 -- 172.21.15.113:0/3925426859 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fb9740f30a0 con 0x7fb98c0c2d80 2024-03-04T07:40:56.025 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:40:56.025 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:46.796316+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094,4938]},"id":1}]} 2024-03-04T07:40:56.025 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 38 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 -- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb968066f60 msgr2=0x7fb968069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb968066f60 0x7fb968069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009a90 tx=0x7fb97c008040 comp rx=0 tx=0).stop 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 -- 172.21.15.113:0/3925426859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c2d80 msgr2=0x7fb98c06b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c2d80 0x7fb98c06b2c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb97401d3c0 tx=0x7fb97401bab0 comp rx=0 tx=0).stop 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 -- 172.21.15.113:0/3925426859 shutdown_connections 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb98c14e4c0 0x7fb98c145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb968066f60 0x7fb968069420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb98c0c2d80 0x7fb98c06b2c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 --2- 172.21.15.113:0/3925426859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb98c0c22e0 0x7fb98c1f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 -- 172.21.15.113:0/3925426859 >> 172.21.15.113:0/3925426859 conn(0x7fb98c06e820 msgr2=0x7fb98c0c8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 -- 172.21.15.113:0/3925426859 shutdown_connections 2024-03-04T07:40:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:40:56.028+0000 7fb990e00640 1 -- 172.21.15.113:0/3925426859 wait complete. 2024-03-04T07:40:56.046 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2024-03-04T07:40:56.046 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 32 secs before thrashing 2024-03-04T07:40:56.138 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:40:56.141 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:40:56.755 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/32' 2024-03-04T07:40:56.755 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/30' 2024-03-04T07:40:56.755 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/14' 2024-03-04T07:40:56.755 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/18' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/13' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/33' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/17' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/35' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './138/37' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './138' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/19' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/15' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/31' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/36' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/20' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/12' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/39' 2024-03-04T07:40:56.756 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/10' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/34' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/38' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/11' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/16' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/32' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/30' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/14' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/18' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/13' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/33' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/17' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/35' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './139/37' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './139' 2024-03-04T07:40:56.757 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/19' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/15' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/31' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/36' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/20' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/12' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/39' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/10' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/34' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/38' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/11' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/16' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/32' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/30' 2024-03-04T07:40:56.758 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/14' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/18' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/13' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/33' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/17' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/35' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './140/37' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './140' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/19' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/15' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/31' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/36' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/20' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/12' 2024-03-04T07:40:56.759 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/39' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/10' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/34' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/38' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/11' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/16' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/32' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/30' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/14' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/18' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/13' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/33' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/17' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/35' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed './141/37' 2024-03-04T07:40:56.760 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './141' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/19' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/15' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/31' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/36' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/20' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/12' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/39' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/10' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/34' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/38' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/11' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/16' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/32' 2024-03-04T07:40:56.761 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/30' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/14' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/18' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/13' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/33' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/17' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/35' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './142/37' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './142' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/19' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/15' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/31' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/36' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/20' 2024-03-04T07:40:56.762 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/12' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/39' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/10' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/34' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/38' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/11' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/16' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/32' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/30' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/14' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/18' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/13' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/33' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/17' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/35' 2024-03-04T07:40:56.763 INFO:tasks.workunit.client.0.smithi113.stdout:removed './143/37' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './143' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/19' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/15' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/31' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/36' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/20' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/12' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/39' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/10' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/34' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/38' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/11' 2024-03-04T07:40:56.764 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/16' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/32' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/30' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/14' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/18' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/13' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/33' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/17' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/35' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './144/37' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './144' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/19' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/15' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/31' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/36' 2024-03-04T07:40:56.765 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/20' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/12' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/39' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/10' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/34' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/38' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/11' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/16' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/32' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/30' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/14' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/18' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/13' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/33' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/17' 2024-03-04T07:40:56.766 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/35' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './145/37' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './145' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/19' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/15' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/31' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/36' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/20' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/12' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/39' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/10' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/34' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/38' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/11' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/16' 2024-03-04T07:40:56.767 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/32' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/30' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/14' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/18' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/13' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/33' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/17' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/35' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './146/37' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './146' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/19' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/15' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/31' 2024-03-04T07:40:56.768 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/36' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/20' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/12' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/39' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/10' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/34' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/38' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/11' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/16' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/32' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/30' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/14' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/18' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/13' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/33' 2024-03-04T07:40:56.769 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/17' 2024-03-04T07:40:56.770 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/35' 2024-03-04T07:40:56.770 INFO:tasks.workunit.client.0.smithi113.stdout:removed './147/37' 2024-03-04T07:40:56.770 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './147' 2024-03-04T07:40:56.770 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/19' 2024-03-04T07:40:56.770 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/15' 2024-03-04T07:40:56.770 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/31' 2024-03-04T07:40:56.770 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/36' 2024-03-04T07:40:59.452 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/20' 2024-03-04T07:40:59.452 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/12' 2024-03-04T07:40:59.452 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/39' 2024-03-04T07:40:59.452 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/10' 2024-03-04T07:40:59.452 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/34' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/38' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/11' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/16' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/32' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/30' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/14' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/18' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/13' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/33' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/17' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/35' 2024-03-04T07:40:59.453 INFO:tasks.workunit.client.0.smithi113.stdout:removed './148/37' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './148' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/19' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/15' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/31' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/36' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/20' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/12' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/39' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/10' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/34' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/38' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/11' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/16' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/32' 2024-03-04T07:40:59.454 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/30' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/14' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/18' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/13' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/33' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/17' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/35' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './149/37' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './149' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/19' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/15' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/31' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/36' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/20' 2024-03-04T07:40:59.455 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/12' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/39' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/10' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/34' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/38' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/11' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/16' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/32' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/30' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/14' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/18' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/13' 2024-03-04T07:40:59.456 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/33' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/17' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/35' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './150/37' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './150' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/19' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/15' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/31' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/36' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/20' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/12' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/39' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/10' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/34' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/38' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/11' 2024-03-04T07:40:59.457 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/16' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/32' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/30' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/14' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/18' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/13' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/33' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/17' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/35' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './151/37' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './151' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/19' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/15' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/31' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/36' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/20' 2024-03-04T07:40:59.458 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/12' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/39' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/10' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/34' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/38' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/11' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/16' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/32' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/30' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/14' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/18' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/13' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/33' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/17' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/35' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed './152/37' 2024-03-04T07:40:59.459 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './152' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/19' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/15' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/31' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/36' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/20' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/12' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/39' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/10' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/34' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/38' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/11' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/16' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/32' 2024-03-04T07:40:59.460 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/30' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/14' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/18' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/13' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/33' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/17' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/35' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './153/37' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './153' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/19' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/15' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/31' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/36' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/20' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/12' 2024-03-04T07:40:59.461 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/39' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/10' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/34' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/38' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/11' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/16' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/32' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/30' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/14' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/18' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/13' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/33' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/17' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/35' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed './154/37' 2024-03-04T07:40:59.462 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './154' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/19' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/15' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/31' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/36' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/20' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/12' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/39' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/10' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/34' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/38' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/11' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/16' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/32' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/30' 2024-03-04T07:40:59.463 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/14' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/18' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/13' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/33' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/17' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/35' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './155/37' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './155' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/19' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/15' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/31' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/36' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/20' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/12' 2024-03-04T07:40:59.464 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/39' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/10' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/34' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/38' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/11' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/16' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/32' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/30' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/14' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/18' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/13' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/33' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/17' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/35' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed './156/37' 2024-03-04T07:40:59.465 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './156' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/19' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/15' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/31' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/36' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/20' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/12' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/39' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/10' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/34' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/38' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/11' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/16' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/32' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/30' 2024-03-04T07:40:59.466 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/14' 2024-03-04T07:40:59.467 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/18' 2024-03-04T07:40:59.467 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/13' 2024-03-04T07:40:59.467 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/33' 2024-03-04T07:41:02.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/17' 2024-03-04T07:41:02.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/35' 2024-03-04T07:41:02.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './157/37' 2024-03-04T07:41:02.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './157' 2024-03-04T07:41:02.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/19' 2024-03-04T07:41:02.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/15' 2024-03-04T07:41:02.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/31' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/36' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/20' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/12' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/39' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/10' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/34' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/38' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/11' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/16' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/32' 2024-03-04T07:41:02.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/30' 2024-03-04T07:41:02.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/14' 2024-03-04T07:41:02.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/18' 2024-03-04T07:41:02.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/13' 2024-03-04T07:41:02.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/33' 2024-03-04T07:41:02.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/17' 2024-03-04T07:41:02.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/35' 2024-03-04T07:41:02.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './158/37' 2024-03-04T07:41:02.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './158' 2024-03-04T07:41:02.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/19' 2024-03-04T07:41:02.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/15' 2024-03-04T07:41:02.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/31' 2024-03-04T07:41:02.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/36' 2024-03-04T07:41:02.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/20' 2024-03-04T07:41:02.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/12' 2024-03-04T07:41:02.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/39' 2024-03-04T07:41:02.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/10' 2024-03-04T07:41:02.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/34' 2024-03-04T07:41:02.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/38' 2024-03-04T07:41:02.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/11' 2024-03-04T07:41:02.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/16' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/32' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/30' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/14' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/18' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/13' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/33' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/17' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/35' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './159/37' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './159' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/19' 2024-03-04T07:41:02.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/15' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/31' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/36' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/20' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/12' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/39' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/10' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/34' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/38' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/11' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/16' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/32' 2024-03-04T07:41:02.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/30' 2024-03-04T07:41:02.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/14' 2024-03-04T07:41:02.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/18' 2024-03-04T07:41:02.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/13' 2024-03-04T07:41:02.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/33' 2024-03-04T07:41:02.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/17' 2024-03-04T07:41:02.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/35' 2024-03-04T07:41:02.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './160/37' 2024-03-04T07:41:02.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './160' 2024-03-04T07:41:02.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/19' 2024-03-04T07:41:02.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/15' 2024-03-04T07:41:02.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/31' 2024-03-04T07:41:02.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/36' 2024-03-04T07:41:02.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/20' 2024-03-04T07:41:02.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/12' 2024-03-04T07:41:02.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/39' 2024-03-04T07:41:02.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/10' 2024-03-04T07:41:02.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/34' 2024-03-04T07:41:02.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/38' 2024-03-04T07:41:02.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/11' 2024-03-04T07:41:02.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/16' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/32' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/30' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/14' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/18' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/13' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/33' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/17' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/35' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './161/37' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './161' 2024-03-04T07:41:02.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/19' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/15' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/31' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/36' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/20' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/12' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/39' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/10' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/34' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/38' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/11' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/16' 2024-03-04T07:41:02.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/32' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/30' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/14' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/18' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/13' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/33' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/17' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/35' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './162/37' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './162' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/19' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/15' 2024-03-04T07:41:02.523 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/31' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/36' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/20' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/12' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/39' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/10' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/34' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/38' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/11' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/16' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/32' 2024-03-04T07:41:02.524 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/30' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/14' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/18' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/13' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/33' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/17' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/35' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './163/37' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './163' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/19' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/15' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/31' 2024-03-04T07:41:02.525 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/36' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/20' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/12' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/39' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/10' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/34' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/38' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/11' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/16' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/32' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/30' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/14' 2024-03-04T07:41:02.526 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/18' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/13' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/33' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/17' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/35' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './164/37' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './164' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/19' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/15' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/31' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/36' 2024-03-04T07:41:02.527 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/20' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/12' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/39' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/10' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/34' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/38' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/11' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/16' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/32' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/30' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/14' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/18' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/13' 2024-03-04T07:41:02.528 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/33' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/17' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/35' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './165/37' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './165' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/19' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/15' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/31' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/36' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/20' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/12' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/39' 2024-03-04T07:41:02.529 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/10' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/34' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/38' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/11' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/16' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/32' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/30' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/14' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/18' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/13' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/33' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/17' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/35' 2024-03-04T07:41:02.530 INFO:tasks.workunit.client.0.smithi113.stdout:removed './166/37' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './166' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/19' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/15' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/31' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/36' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/20' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/12' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/39' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/10' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/34' 2024-03-04T07:41:02.531 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/38' 2024-03-04T07:41:06.239 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/11' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/16' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/32' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/30' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/14' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/18' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/13' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/33' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/17' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/35' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './167/37' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './167' 2024-03-04T07:41:06.240 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/19' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/15' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/31' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/36' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/20' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/12' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/39' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/10' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/34' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/38' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/11' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/16' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/32' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/30' 2024-03-04T07:41:06.241 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/14' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/18' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/13' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/33' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/17' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/35' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './168/37' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './168' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/19' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/15' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/31' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/36' 2024-03-04T07:41:06.242 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/20' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/12' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/39' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/10' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/34' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/38' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/11' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/16' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/32' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/30' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/14' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/18' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/13' 2024-03-04T07:41:06.243 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/33' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/17' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/35' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './169/37' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './169' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/19' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/15' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/31' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/36' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/20' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/12' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/39' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/10' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/34' 2024-03-04T07:41:06.244 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/38' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/11' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/16' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/32' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/30' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/14' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/18' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/13' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/33' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/17' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/35' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './170/37' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './170' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/19' 2024-03-04T07:41:06.245 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/15' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/31' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/36' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/20' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/12' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/39' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/10' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/34' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/38' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/11' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/16' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/32' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/30' 2024-03-04T07:41:06.246 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/14' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/18' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/13' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/33' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/17' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/35' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './171/37' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './171' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/19' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/15' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/31' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/36' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/20' 2024-03-04T07:41:06.247 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/12' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/39' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/10' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/34' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/38' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/11' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/16' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/32' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/30' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/14' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/18' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/13' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/33' 2024-03-04T07:41:06.248 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/17' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/35' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './172/37' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './172' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/19' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/15' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/31' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/36' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/20' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/12' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/39' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/10' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/34' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/38' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/11' 2024-03-04T07:41:06.249 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/16' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/32' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/30' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/14' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/18' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/13' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/33' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/17' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/35' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './173/37' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './173' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/19' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/15' 2024-03-04T07:41:06.250 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/31' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/36' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/20' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/12' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/39' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/10' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/34' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/38' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/11' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/16' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/32' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/30' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/14' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/18' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/13' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/33' 2024-03-04T07:41:06.251 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/17' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/35' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './174/37' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './174' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/19' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/15' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/31' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/36' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/20' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/12' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/39' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/10' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/34' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/38' 2024-03-04T07:41:06.252 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/11' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/16' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/32' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/30' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/14' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/18' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/13' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/33' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/17' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/35' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './175/37' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './175' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/19' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/15' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/31' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/36' 2024-03-04T07:41:06.253 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/20' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/12' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/39' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/10' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/34' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/38' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/11' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/16' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/32' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/30' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/14' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/18' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/13' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/33' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/17' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/35' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed './176/37' 2024-03-04T07:41:06.254 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './176' 2024-03-04T07:41:06.255 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/19' 2024-03-04T07:41:06.255 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/15' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/31' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/36' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/20' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/12' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/39' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/10' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/34' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/38' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/11' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/16' 2024-03-04T07:41:12.487 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/32' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/30' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/14' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/18' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/13' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/33' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/17' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/35' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './177/37' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './177' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/19' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/15' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/31' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/36' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/20' 2024-03-04T07:41:12.488 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/12' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/39' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/10' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/34' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/38' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/11' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/16' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/32' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/30' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/14' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/18' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/13' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/33' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/17' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/35' 2024-03-04T07:41:12.489 INFO:tasks.workunit.client.0.smithi113.stdout:removed './178/37' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './178' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/19' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/15' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/31' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/36' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/20' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/12' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/39' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/10' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/34' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/38' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/11' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/16' 2024-03-04T07:41:12.490 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/32' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/30' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/14' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/18' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/13' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/33' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/17' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/35' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './179/37' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './179' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/19' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/15' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/31' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/36' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/20' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/12' 2024-03-04T07:41:12.491 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/39' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/10' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/34' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/38' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/11' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/16' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/32' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/30' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/14' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/18' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/13' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/33' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/17' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/35' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './180/37' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './180' 2024-03-04T07:41:12.492 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/19' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/15' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/31' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/36' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/20' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/12' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/39' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/10' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/34' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/38' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/11' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/16' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/32' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/30' 2024-03-04T07:41:12.493 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/14' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/18' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/13' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/33' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/17' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/35' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './181/37' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './181' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/19' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/15' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/31' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/36' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/20' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/12' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/39' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/10' 2024-03-04T07:41:12.494 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/34' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/38' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/11' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/16' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/32' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/30' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/14' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/18' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/13' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/33' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/17' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/35' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './182/37' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './182' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/19' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/15' 2024-03-04T07:41:12.495 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/31' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/36' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/20' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/12' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/39' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/10' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/34' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/38' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/11' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/16' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/32' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/30' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/14' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/18' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/13' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/33' 2024-03-04T07:41:12.496 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/17' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/35' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './183/37' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './183' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/19' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/15' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/31' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/36' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/20' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/12' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/39' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/10' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/34' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/38' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/11' 2024-03-04T07:41:12.497 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/16' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/32' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/30' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/14' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/18' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/13' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/33' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/17' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/35' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './184/37' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './184' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/19' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/15' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/31' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/36' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/20' 2024-03-04T07:41:12.498 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/12' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/39' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/10' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/34' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/38' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/11' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/16' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/32' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/30' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/14' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/18' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/13' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/33' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/17' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/35' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed './185/37' 2024-03-04T07:41:12.499 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './185' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/19' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/15' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/31' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/36' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/20' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/12' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/39' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/10' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/34' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/38' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/11' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/16' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/32' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/30' 2024-03-04T07:41:12.500 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/14' 2024-03-04T07:41:12.501 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/18' 2024-03-04T07:41:18.540 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/13' 2024-03-04T07:41:18.540 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/33' 2024-03-04T07:41:18.540 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/17' 2024-03-04T07:41:18.540 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/35' 2024-03-04T07:41:18.540 INFO:tasks.workunit.client.0.smithi113.stdout:removed './186/37' 2024-03-04T07:41:18.540 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './186' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/19' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/15' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/31' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/36' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/20' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/12' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/39' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/10' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/34' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/38' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/11' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/16' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/32' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/30' 2024-03-04T07:41:18.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/14' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/18' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/13' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/33' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/17' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/35' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './187/37' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './187' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/19' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/15' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/31' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/36' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/20' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/12' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/39' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/10' 2024-03-04T07:41:18.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/34' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/38' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/11' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/16' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/32' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/30' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/14' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/18' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/13' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/33' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/17' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/35' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './188/37' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './188' 2024-03-04T07:41:18.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/19' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/15' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/31' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/36' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/20' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/12' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/39' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/10' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/34' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/38' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/11' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/16' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/32' 2024-03-04T07:41:18.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/30' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/14' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/18' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/13' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/33' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/17' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/35' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './189/37' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './189' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/19' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/15' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/31' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/36' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/20' 2024-03-04T07:41:18.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/12' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/39' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/10' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/34' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/38' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/11' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/16' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/32' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/30' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/14' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/18' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/13' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/33' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/17' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/35' 2024-03-04T07:41:18.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './190/37' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './190' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/19' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/15' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/31' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/36' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/20' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/12' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/39' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/10' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/34' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/38' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/11' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/16' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/32' 2024-03-04T07:41:18.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/30' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/14' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/18' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/13' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/33' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/17' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/35' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './191/37' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './191' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/19' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/15' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/31' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/36' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/20' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/12' 2024-03-04T07:41:18.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/39' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/10' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/34' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/38' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/11' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/16' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/32' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/30' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/14' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/18' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/13' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/33' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/17' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/35' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './192/37' 2024-03-04T07:41:18.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './192' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/19' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/15' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/31' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/36' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/20' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/12' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/39' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/10' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/34' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/38' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/11' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/16' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/32' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/30' 2024-03-04T07:41:18.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/14' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/18' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/13' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/33' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/17' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/35' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './193/37' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './193' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/19' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/15' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/31' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/36' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/20' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/12' 2024-03-04T07:41:18.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/39' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/10' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/34' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/38' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/11' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/16' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/32' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/30' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/14' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/18' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/13' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/33' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/17' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/35' 2024-03-04T07:41:18.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './194/37' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './194' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/19' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/15' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/31' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/36' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/20' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/12' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/39' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/10' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/34' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/38' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/11' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/16' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/32' 2024-03-04T07:41:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/30' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/14' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/18' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/13' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/33' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/17' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/35' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './195/37' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './195' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/19' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/15' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/31' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/36' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/20' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/12' 2024-03-04T07:41:18.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/39' 2024-03-04T07:41:18.555 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/10' 2024-03-04T07:41:24.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/34' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/38' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/11' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/16' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/32' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/30' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/14' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/18' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/13' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/33' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/17' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/35' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './196/37' 2024-03-04T07:41:24.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './196' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/19' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/15' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/31' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/36' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/20' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/12' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/39' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/10' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/34' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/38' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/11' 2024-03-04T07:41:24.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/16' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/32' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/30' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/14' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/18' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/13' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/33' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/17' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/35' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './197/37' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './197' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/19' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/15' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/31' 2024-03-04T07:41:24.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/36' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/20' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/12' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/39' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/10' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/34' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/38' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/11' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/16' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/32' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/30' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/14' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/18' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/13' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/33' 2024-03-04T07:41:24.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/17' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/35' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './198/37' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './198' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/19' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/15' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/31' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/36' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/20' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/12' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/39' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/10' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/34' 2024-03-04T07:41:24.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/38' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/11' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/16' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/32' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/30' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/14' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/18' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/13' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/33' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/17' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/35' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './199/37' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './199' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/21' 2024-03-04T07:41:24.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/27' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/31' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/36' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/20' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/39' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/34' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/38' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/28' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/24' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/29' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/32' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/22' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/25' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/30' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/33' 2024-03-04T07:41:24.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/23' 2024-03-04T07:41:24.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/35' 2024-03-04T07:41:24.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/26' 2024-03-04T07:41:24.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './200/37' 2024-03-04T07:41:24.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './200' 2024-03-04T07:41:24.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/21' 2024-03-04T07:41:24.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/27' 2024-03-04T07:41:24.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/31' 2024-03-04T07:41:24.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/36' 2024-03-04T07:41:24.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/20' 2024-03-04T07:41:24.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/39' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/34' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/38' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/28' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/24' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/29' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/32' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/22' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/25' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/30' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/33' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/23' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/35' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/26' 2024-03-04T07:41:24.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './201/37' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './201' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/21' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/27' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/31' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/36' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/20' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/39' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/34' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/38' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/28' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/24' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/29' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/32' 2024-03-04T07:41:24.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/22' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/25' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/30' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/33' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/23' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/35' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/26' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './202/37' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './202' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/21' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/27' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/31' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/36' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/20' 2024-03-04T07:41:24.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/39' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/34' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/38' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/28' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/24' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/29' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/32' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/22' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/25' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/30' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/33' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/23' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/35' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/26' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './203/37' 2024-03-04T07:41:24.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './203' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/21' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/27' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/31' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/36' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/20' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/39' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/34' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/38' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/28' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/24' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/29' 2024-03-04T07:41:24.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/32' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/22' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/25' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/30' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/33' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/23' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/35' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/26' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './204/37' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './204' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/21' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/27' 2024-03-04T07:41:24.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/31' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/36' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/20' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/39' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/34' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/38' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/28' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/24' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/29' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/32' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/22' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/25' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/30' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/33' 2024-03-04T07:41:24.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/23' 2024-03-04T07:41:24.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/35' 2024-03-04T07:41:24.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/26' 2024-03-04T07:41:24.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './205/37' 2024-03-04T07:41:24.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './205' 2024-03-04T07:41:24.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/21' 2024-03-04T07:41:24.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/27' 2024-03-04T07:41:24.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/31' 2024-03-04T07:41:24.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/36' 2024-03-04T07:41:24.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/20' 2024-03-04T07:41:24.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/39' 2024-03-04T07:41:26.212 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:41:26.217 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:41:28.047 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:28.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.224+0000 7f6ee9600640 1 Processor -- start 2024-03-04T07:41:28.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.224+0000 7f6ee9600640 1 -- start start 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6ee40c85f0 0x7f6ee406b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee406b5a0 0x7f6ee4161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee40c1ec0 con 0x7f6ee40b20b0 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee40c9b30 con 0x7f6ee40c85f0 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40c12a0 con 0x7f6ee406b5a0 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6ee40b20b0 0x7f6ee4157710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38474/0 (socket says 172.21.15.113:38474) 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2a00640 1 -- 172.21.15.113:0/1150152911 learned_addr learned my addr 172.21.15.113:0/1150152911 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6eda000640 1 --2- 172.21.15.113:0/1150152911 >> v2:172.21.15.136:3300/0 conn(0x7f6ee40c85f0 0x7f6ee406b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee3400640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee406b5a0 0x7f6ee4161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200401175 0 0) 0x7f6ee40c1ec0 con 0x7f6ee40b20b0 2024-03-04T07:41:28.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec8003530 con 0x7f6ee40b20b0 2024-03-04T07:41:28.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1929279055 0 0) 0x7f6ec8003530 con 0x7f6ee40b20b0 2024-03-04T07:41:28.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee406b5a0 msgr2=0x7f6ee4161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:28.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee406b5a0 0x7f6ee4161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 >> v2:172.21.15.136:3300/0 conn(0x7f6ee40c85f0 msgr2=0x7f6ee406b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:28.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 --2- 172.21.15.113:0/1150152911 >> v2:172.21.15.136:3300/0 conn(0x7f6ee40c85f0 0x7f6ee406b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee40c1580 con 0x7f6ee40b20b0 2024-03-04T07:41:28.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f6ed4001680 con 0x7f6ee40b20b0 2024-03-04T07:41:28.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6eda000640 1 --2- 172.21.15.113:0/1150152911 >> v2:172.21.15.136:3300/0 conn(0x7f6ee40c85f0 0x7f6ee406b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:41:28.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 >> v1:172.21.15.113:6789/0 conn(0x7f6ee40b20b0 legacy=0x7f6ee4157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:28.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80039c0 0x7f6ec8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee406b5a0 0x7f6ec8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec8004d50 0x7f6ec80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40c9b30 con 0x7f6ec80039c0 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40c12a0 con 0x7f6ec8004d50 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec800b9d0 con 0x7f6ee406b5a0 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2a00640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80039c0 0x7f6ec8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6eda000640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee406b5a0 0x7f6ec8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee3400640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec8004d50 0x7f6ec80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2a00640 1 -- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee406b5a0 msgr2=0x7f6ec8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2a00640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ee406b5a0 0x7f6ec8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2a00640 1 -- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec8004d50 msgr2=0x7f6ec80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2a00640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ec8004d50 0x7f6ec80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.228+0000 7f6ee2a00640 1 -- 172.21.15.113:0/1150152911 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec8003530 con 0x7f6ec80039c0 2024-03-04T07:41:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee2a00640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80039c0 0x7f6ec8003de0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4001060 tx=0x7f6ed401f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa95e122a18bd04b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4007d80 con 0x7f6ec80039c0 2024-03-04T07:41:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed4005b00 con 0x7f6ec80039c0 2024-03-04T07:41:28.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee2000640 1 -- 172.21.15.113:0/1150152911 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4007070 con 0x7f6ec80039c0 2024-03-04T07:41:28.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee9600640 1 -- 172.21.15.113:0/1150152911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80039c0 msgr2=0x7f6ec8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:28.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee9600640 1 --2- 172.21.15.113:0/1150152911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ec80039c0 0x7f6ec8003de0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4001060 tx=0x7f6ed401f410 comp rx=0 tx=0).stop 2024-03-04T07:41:28.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee3400640 1 -- 172.21.15.113:0/1150152911 reap_dead start 2024-03-04T07:41:28.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee9600640 1 -- 172.21.15.113:0/1150152911 shutdown_connections 2024-03-04T07:41:28.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee9600640 1 -- 172.21.15.113:0/1150152911 >> 172.21.15.113:0/1150152911 conn(0x7f6ee406e820 msgr2=0x7f6ee406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:28.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee9600640 1 -- 172.21.15.113:0/1150152911 shutdown_connections 2024-03-04T07:41:28.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee9600640 1 -- 172.21.15.113:0/1150152911 wait complete. 2024-03-04T07:41:28.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.232+0000 7f6ee9600640 1 Processor -- start 2024-03-04T07:41:28.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee9600640 1 -- start start 2024-03-04T07:41:28.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ec80039c0 0x7f6ee4146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee9600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ee4163e80 0x7f6ee40c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ee414a450 0x7f6ee40c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee9600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4157e20 con 0x7f6ee414a450 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee9600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee41571f0 con 0x7f6ee4163e80 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee40c2400 con 0x7f6ec80039c0 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ee414a450 0x7f6ee40c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ec80039c0 0x7f6ee4146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ee414a450 0x7f6ee40c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48078/0 (socket says 172.21.15.113:48078) 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee3400640 1 -- 172.21.15.113:0/1433149335 learned_addr learned my addr 172.21.15.113:0/1433149335 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6eda000640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ee4163e80 0x7f6ee40c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee3400640 1 -- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ec80039c0 msgr2=0x7f6ee4146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee3400640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ec80039c0 0x7f6ee4146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee3400640 1 -- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ee4163e80 msgr2=0x7f6ee40c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee3400640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ee4163e80 0x7f6ee40c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee3400640 1 -- 172.21.15.113:0/1433149335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee4164400 con 0x7f6ee414a450 2024-03-04T07:41:28.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee3400640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ee414a450 0x7f6ee40c60d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40ca020 tx=0x7f6edc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:28.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6edb400640 1 -- 172.21.15.113:0/1433149335 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00fd80 con 0x7f6ee414a450 2024-03-04T07:41:28.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6edb400640 1 -- 172.21.15.113:0/1433149335 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc01d070 con 0x7f6ee414a450 2024-03-04T07:41:28.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee9600640 1 -- 172.21.15.113:0/1433149335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee41646f0 con 0x7f6ee414a450 2024-03-04T07:41:28.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6edb400640 1 -- 172.21.15.113:0/1433149335 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc015670 con 0x7f6ee414a450 2024-03-04T07:41:28.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.236+0000 7f6ee9600640 1 -- 172.21.15.113:0/1433149335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee4164cb0 con 0x7f6ee414a450 2024-03-04T07:41:28.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.240+0000 7f6ee9600640 1 -- 172.21.15.113:0/1433149335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee4075c00 con 0x7f6ee414a450 2024-03-04T07:41:28.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.240+0000 7f6edb400640 1 -- 172.21.15.113:0/1433149335 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6edc015810 con 0x7f6ee414a450 2024-03-04T07:41:28.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.240+0000 7f6edb400640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ec0066f90 0x7f6ec0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.240+0000 7f6edb400640 1 -- 172.21.15.113:0/1433149335 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f6edc0859c0 con 0x7f6ee414a450 2024-03-04T07:41:28.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.240+0000 7f6ee2a00640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ec0066f90 0x7f6ec0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.240+0000 7f6ee2a00640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ec0066f90 0x7f6ec0069450 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40ae000 tx=0x7f6ed4001a50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:28.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.248+0000 7f6edb400640 1 -- 172.21.15.113:0/1433149335 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6edc010040 con 0x7f6ee414a450 2024-03-04T07:41:28.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.592+0000 7f6ee9600640 1 -- 172.21.15.113:0/1433149335 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ee40c83d0 con 0x7f6ee414a450 2024-03-04T07:41:28.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.592+0000 7f6edb400640 1 -- 172.21.15.113:0/1433149335 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f6edc056110 con 0x7f6ee414a450 2024-03-04T07:41:28.591 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:28.591 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:46.796316+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094,4938]},"id":1}]} 2024-03-04T07:41:28.591 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 38 2024-03-04T07:41:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 -- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ec0066f90 msgr2=0x7f6ec0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ec0066f90 0x7f6ec0069450 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40ae000 tx=0x7f6ed4001a50 comp rx=0 tx=0).stop 2024-03-04T07:41:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 -- 172.21.15.113:0/1433149335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ee414a450 msgr2=0x7f6ee40c60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:28.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ee414a450 0x7f6ee40c60d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40ca020 tx=0x7f6edc00f580 comp rx=0 tx=0).stop 2024-03-04T07:41:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 -- 172.21.15.113:0/1433149335 shutdown_connections 2024-03-04T07:41:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ec0066f90 0x7f6ec0069450 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ee414a450 0x7f6ee40c60d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ee4163e80 0x7f6ee40c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 --2- 172.21.15.113:0/1433149335 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ec80039c0 0x7f6ee4146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 -- 172.21.15.113:0/1433149335 >> 172.21.15.113:0/1433149335 conn(0x7f6ee406e820 msgr2=0x7f6ee4154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 -- 172.21.15.113:0/1433149335 shutdown_connections 2024-03-04T07:41:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.596+0000 7f6ee9600640 1 -- 172.21.15.113:0/1433149335 wait complete. 2024-03-04T07:41:28.622 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=2) 2024-03-04T07:41:28.622 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-04T07:41:28.622 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:41:28.637 INFO:tasks.ceph.mds.a:Stopped 2024-03-04T07:41:28.637 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.820+0000 7ff5ff400640 1 Processor -- start 2024-03-04T07:41:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.820+0000 7ff5ff400640 1 -- start start 2024-03-04T07:41:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f806ff20 0x7ff5f815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5ff400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5f80c8b90 0x7ff5f815c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5ff400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f8152e30 con 0x7ff5f8152a50 2024-03-04T07:41:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5ff400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f80c31b0 con 0x7ff5f80c8b90 2024-03-04T07:41:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5ff400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f806f840 con 0x7ff5f806ff20 2024-03-04T07:41:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5fd400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5f80c8b90 0x7ff5f815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5fd400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5f80c8b90 0x7ff5f815c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48080/0 (socket says 172.21.15.113:48080) 2024-03-04T07:41:28.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5fd400640 1 -- 172.21.15.113:0/234107170 learned_addr learned my addr 172.21.15.113:0/234107170 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5f7e00640 1 --2- 172.21.15.113:0/234107170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f806ff20 0x7ff5f815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5fd400640 1 -- 172.21.15.113:0/234107170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f806ff20 msgr2=0x7ff5f815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5fd400640 1 --2- 172.21.15.113:0/234107170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f806ff20 0x7ff5f815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5fd400640 1 -- 172.21.15.113:0/234107170 >> v1:172.21.15.136:6789/0 conn(0x7ff5f8152a50 legacy=0x7ff5f80ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:28.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.824+0000 7ff5fd400640 1 -- 172.21.15.113:0/234107170 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f806f280 con 0x7ff5f80c8b90 2024-03-04T07:41:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5fd400640 1 --2- 172.21.15.113:0/234107170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f80c8b90 0x7ff5f815c310 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009e50 tx=0x7ff5f001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a4abc56f81691f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5f7400640 1 -- 172.21.15.113:0/234107170 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00a5880 con 0x7ff5f80c8b90 2024-03-04T07:41:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5f7400640 1 -- 172.21.15.113:0/234107170 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f00a5a20 con 0x7ff5f80c8b90 2024-03-04T07:41:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5f7400640 1 -- 172.21.15.113:0/234107170 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00a5df0 con 0x7ff5f80c8b90 2024-03-04T07:41:28.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5ff400640 1 -- 172.21.15.113:0/234107170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f80c8b90 msgr2=0x7ff5f815c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:28.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5ff400640 1 --2- 172.21.15.113:0/234107170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f80c8b90 0x7ff5f815c310 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009e50 tx=0x7ff5f001c9a0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5ff400640 1 -- 172.21.15.113:0/234107170 shutdown_connections 2024-03-04T07:41:28.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5ff400640 1 --2- 172.21.15.113:0/234107170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f80c8b90 0x7ff5f815c310 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5ff400640 1 --2- 172.21.15.113:0/234107170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f806ff20 0x7ff5f815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5ff400640 1 -- 172.21.15.113:0/234107170 >> 172.21.15.113:0/234107170 conn(0x7ff5f80734d0 msgr2=0x7ff5f8066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:28.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5ff400640 1 -- 172.21.15.113:0/234107170 shutdown_connections 2024-03-04T07:41:28.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.828+0000 7ff5ff400640 1 -- 172.21.15.113:0/234107170 wait complete. 2024-03-04T07:41:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.832+0000 7ff5ff400640 1 Processor -- start 2024-03-04T07:41:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.832+0000 7ff5ff400640 1 -- start start 2024-03-04T07:41:28.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.832+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f806ff20 0x7ff5f80caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.832+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f80c8b90 0x7ff5f8156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.832+0000 7ff5ff400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f8152a50 0x7ff5f81f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.832+0000 7ff5ff400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f80ce9b0 con 0x7ff5f8152a50 2024-03-04T07:41:28.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.832+0000 7ff5ff400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f80ce830 con 0x7ff5f80c8b90 2024-03-04T07:41:28.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.832+0000 7ff5ff400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f8165880 con 0x7ff5f806ff20 2024-03-04T07:41:28.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5fd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f8152a50 0x7ff5f81f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5fd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f8152a50 0x7ff5f81f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48088/0 (socket says 172.21.15.113:48088) 2024-03-04T07:41:28.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5fd400640 1 -- 172.21.15.113:0/1799571555 learned_addr learned my addr 172.21.15.113:0/1799571555 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5fca00640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f806ff20 0x7ff5f80caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5f7e00640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f80c8b90 0x7ff5f8156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5fd400640 1 -- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f806ff20 msgr2=0x7ff5f80caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5fd400640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f806ff20 0x7ff5f80caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5fd400640 1 -- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f80c8b90 msgr2=0x7ff5f8156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5fd400640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f80c8b90 0x7ff5f8156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5fd400640 1 -- 172.21.15.113:0/1799571555 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f8158680 con 0x7ff5f8152a50 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5fd400640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f8152a50 0x7ff5f81f7260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff5f001a5d0 tx=0x7ff5f0019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5f4c00640 1 -- 172.21.15.113:0/1799571555 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00bc030 con 0x7ff5f8152a50 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5f4c00640 1 -- 172.21.15.113:0/1799571555 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f00b8470 con 0x7ff5f8152a50 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5f4c00640 1 -- 172.21.15.113:0/1799571555 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00b8860 con 0x7ff5f8152a50 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5ff400640 1 -- 172.21.15.113:0/1799571555 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f8158960 con 0x7ff5f8152a50 2024-03-04T07:41:28.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5ff400640 1 -- 172.21.15.113:0/1799571555 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f8158f20 con 0x7ff5f8152a50 2024-03-04T07:41:28.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.836+0000 7ff5ff400640 1 -- 172.21.15.113:0/1799571555 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f8075c00 con 0x7ff5f8152a50 2024-03-04T07:41:28.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.840+0000 7ff5f4c00640 1 -- 172.21.15.113:0/1799571555 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5f00bf070 con 0x7ff5f8152a50 2024-03-04T07:41:28.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.840+0000 7ff5f4c00640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5d8066f90 0x7ff5d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:28.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.840+0000 7ff5f4c00640 1 -- 172.21.15.113:0/1799571555 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7ff5f011f980 con 0x7ff5f8152a50 2024-03-04T07:41:28.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.840+0000 7ff5fca00640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5d8066f90 0x7ff5d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:28.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.840+0000 7ff5fca00640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5d8066f90 0x7ff5d8069450 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec001810 tx=0x7ff5ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:28.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:28.844+0000 7ff5f4c00640 1 -- 172.21.15.113:0/1799571555 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff5f0016620 con 0x7ff5f8152a50 2024-03-04T07:41:29.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.064+0000 7ff5ff400640 1 -- 172.21.15.113:0/1799571555 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5f814c3d0 con 0x7ff5f8152a50 2024-03-04T07:41:29.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.064+0000 7ff5f4c00640 1 -- 172.21.15.113:0/1799571555 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6612 (secure 0 0 0) 0x7ff5f00f0350 con 0x7ff5f8152a50 2024-03-04T07:41:29.065 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:29.065 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:46.796316+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094,4938]},"id":1}]} 2024-03-04T07:41:29.065 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 38 2024-03-04T07:41:29.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 -- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5d8066f90 msgr2=0x7ff5d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:29.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5d8066f90 0x7ff5d8069450 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec001810 tx=0x7ff5ec009040 comp rx=0 tx=0).stop 2024-03-04T07:41:29.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 -- 172.21.15.113:0/1799571555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f8152a50 msgr2=0x7ff5f81f7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:29.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f8152a50 0x7ff5f81f7260 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff5f001a5d0 tx=0x7ff5f0019740 comp rx=0 tx=0).stop 2024-03-04T07:41:29.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 -- 172.21.15.113:0/1799571555 shutdown_connections 2024-03-04T07:41:29.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5d8066f90 0x7ff5d8069450 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:29.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f8152a50 0x7ff5f81f7260 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:29.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f80c8b90 0x7ff5f8156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:29.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 --2- 172.21.15.113:0/1799571555 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5f806ff20 0x7ff5f80caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:29.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 -- 172.21.15.113:0/1799571555 >> 172.21.15.113:0/1799571555 conn(0x7ff5f80734d0 msgr2=0x7ff5f8159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:29.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 -- 172.21.15.113:0/1799571555 shutdown_connections 2024-03-04T07:41:29.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:29.068+0000 7ff5ff400640 1 -- 172.21.15.113:0/1799571555 wait complete. 2024-03-04T07:41:29.086 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:41:31.087 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:31.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17f0200640 1 Processor -- start 2024-03-04T07:41:31.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17f0200640 1 -- start start 2024-03-04T07:41:31.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17f0200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17e814e4c0 0x7f17e8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:31.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17f0200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e815b900 0x7f17e80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:31.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17f0200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e80c27f0 con 0x7f17e80c22e0 2024-03-04T07:41:31.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17f0200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e815ad60 con 0x7f17e814e4c0 2024-03-04T07:41:31.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17f0200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e80c2a90 con 0x7f17e815b900 2024-03-04T07:41:31.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17ed800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17e814e4c0 0x7f17e8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17ed800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17e814e4c0 0x7f17e8072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48090/0 (socket says 172.21.15.113:48090) 2024-03-04T07:41:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17ed800640 1 -- 172.21.15.113:0/3651769942 learned_addr learned my addr 172.21.15.113:0/3651769942 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17ee200640 1 --2- 172.21.15.113:0/3651769942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e815b900 0x7f17e80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17ed800640 1 -- 172.21.15.113:0/3651769942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e815b900 msgr2=0x7f17e80731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17ed800640 1 --2- 172.21.15.113:0/3651769942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e815b900 0x7f17e80731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17ed800640 1 -- 172.21.15.113:0/3651769942 >> v1:172.21.15.136:6789/0 conn(0x7f17e80c22e0 legacy=0x7f17e815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:31.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17ed800640 1 -- 172.21.15.113:0/3651769942 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e8070c60 con 0x7f17e814e4c0 2024-03-04T07:41:31.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.228+0000 7f17ee200640 1 --2- 172.21.15.113:0/3651769942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e815b900 0x7f17e80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:31.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17ed800640 1 --2- 172.21.15.113:0/3651769942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 0x7f17e8072c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009870 tx=0x7f17dc01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c854913aaec64863 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:31.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17e7e00640 1 -- 172.21.15.113:0/3651769942 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0b5070 con 0x7f17e814e4c0 2024-03-04T07:41:31.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17e7e00640 1 -- 172.21.15.113:0/3651769942 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc0a8a50 con 0x7f17e814e4c0 2024-03-04T07:41:31.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17e7e00640 1 -- 172.21.15.113:0/3651769942 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0a8e20 con 0x7f17e814e4c0 2024-03-04T07:41:31.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 -- 172.21.15.113:0/3651769942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 msgr2=0x7f17e8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:31.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 --2- 172.21.15.113:0/3651769942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 0x7f17e8072c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009870 tx=0x7f17dc01ed50 comp rx=0 tx=0).stop 2024-03-04T07:41:31.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 -- 172.21.15.113:0/3651769942 shutdown_connections 2024-03-04T07:41:31.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 --2- 172.21.15.113:0/3651769942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e815b900 0x7f17e80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:31.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 --2- 172.21.15.113:0/3651769942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 0x7f17e8072c80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:31.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 -- 172.21.15.113:0/3651769942 >> 172.21.15.113:0/3651769942 conn(0x7f17e806e820 msgr2=0x7f17e806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:31.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 -- 172.21.15.113:0/3651769942 shutdown_connections 2024-03-04T07:41:31.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 -- 172.21.15.113:0/3651769942 wait complete. 2024-03-04T07:41:31.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 Processor -- start 2024-03-04T07:41:31.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 -- start start 2024-03-04T07:41:31.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e80c22e0 0x7f17e80cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:31.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 0x7f17e8154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:31.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.236+0000 7f17f0200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e815b900 0x7f17e81548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:31.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17f0200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e8071500 con 0x7f17e814e4c0 2024-03-04T07:41:31.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17f0200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e8071220 con 0x7f17e815b900 2024-03-04T07:41:31.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17f0200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e814e8a0 con 0x7f17e80c22e0 2024-03-04T07:41:31.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ece00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 0x7f17e8154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:31.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ece00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 0x7f17e8154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48094/0 (socket says 172.21.15.113:48094) 2024-03-04T07:41:31.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ece00640 1 -- 172.21.15.113:0/3354810172 learned_addr learned my addr 172.21.15.113:0/3354810172 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:31.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ee200640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e815b900 0x7f17e81548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:31.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ed800640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e80c22e0 0x7f17e80cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:31.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ece00640 1 -- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e80c22e0 msgr2=0x7f17e80cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:31.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ece00640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e80c22e0 0x7f17e80cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:31.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ece00640 1 -- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e815b900 msgr2=0x7f17e81548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:31.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ece00640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e815b900 0x7f17e81548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:31.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ece00640 1 -- 172.21.15.113:0/3354810172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e80af220 con 0x7f17e814e4c0 2024-03-04T07:41:31.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17ece00640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 0x7f17e8154380 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f17d400c3a0 tx=0x7f17d400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:31.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17e5600640 1 -- 172.21.15.113:0/3354810172 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d40052d0 con 0x7f17e814e4c0 2024-03-04T07:41:31.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17e5600640 1 -- 172.21.15.113:0/3354810172 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d4005470 con 0x7f17e814e4c0 2024-03-04T07:41:31.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17e5600640 1 -- 172.21.15.113:0/3354810172 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4007260 con 0x7f17e814e4c0 2024-03-04T07:41:31.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17f0200640 1 -- 172.21.15.113:0/3354810172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e80b0370 con 0x7f17e814e4c0 2024-03-04T07:41:31.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17f0200640 1 -- 172.21.15.113:0/3354810172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e8158bb0 con 0x7f17e814e4c0 2024-03-04T07:41:31.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17f0200640 1 -- 172.21.15.113:0/3354810172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e8075c00 con 0x7f17e814e4c0 2024-03-04T07:41:31.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.240+0000 7f17e5600640 1 -- 172.21.15.113:0/3354810172 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17d4007400 con 0x7f17e814e4c0 2024-03-04T07:41:31.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.244+0000 7f17e5600640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f17c8066f60 0x7f17c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:31.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.244+0000 7f17e5600640 1 -- 172.21.15.113:0/3354810172 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f17d4084560 con 0x7f17e814e4c0 2024-03-04T07:41:31.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.244+0000 7f17ed800640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f17c8066f60 0x7f17c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:31.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.244+0000 7f17ed800640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f17c8066f60 0x7f17c8069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0b2000 tx=0x7f17dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:31.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.248+0000 7f17e5600640 1 -- 172.21.15.113:0/3354810172 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f17d4054d40 con 0x7f17e814e4c0 2024-03-04T07:41:31.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.440+0000 7f17f0200640 1 -- 172.21.15.113:0/3354810172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17e8074790 con 0x7f17e814e4c0 2024-03-04T07:41:31.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17e5600640 1 -- 172.21.15.113:0/3354810172 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f17d4058de0 con 0x7f17e814e4c0 2024-03-04T07:41:31.441 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:31.442 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:46.796316+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094,4938]},"id":1}]} 2024-03-04T07:41:31.442 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 38 2024-03-04T07:41:31.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 -- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f17c8066f60 msgr2=0x7f17c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:31.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f17c8066f60 0x7f17c8069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0b2000 tx=0x7f17dc009040 comp rx=0 tx=0).stop 2024-03-04T07:41:31.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 -- 172.21.15.113:0/3354810172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 msgr2=0x7f17e8154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:31.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 0x7f17e8154380 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f17d400c3a0 tx=0x7f17d400c860 comp rx=0 tx=0).stop 2024-03-04T07:41:31.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 -- 172.21.15.113:0/3354810172 shutdown_connections 2024-03-04T07:41:31.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17e815b900 0x7f17e81548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:31.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f17c8066f60 0x7f17c8069420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:31.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17e814e4c0 0x7f17e8154380 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:31.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 --2- 172.21.15.113:0/3354810172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17e80c22e0 0x7f17e80cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:31.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 -- 172.21.15.113:0/3354810172 >> 172.21.15.113:0/3354810172 conn(0x7f17e806e820 msgr2=0x7f17e8072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:31.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 -- 172.21.15.113:0/3354810172 shutdown_connections 2024-03-04T07:41:31.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:31.444+0000 7f17f0200640 1 -- 172.21.15.113:0/3354810172 wait complete. 2024-03-04T07:41:31.463 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:41:33.464 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.604+0000 7f43d3400640 1 Processor -- start 2024-03-04T07:41:33.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.604+0000 7f43d3400640 1 -- start start 2024-03-04T07:41:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.604+0000 7f43d3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43cc1582e0 0x7f43cc0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.604+0000 7f43d3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43cc0b0ed0 0x7f43cc0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:33.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.604+0000 7f43d3400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc1586c0 con 0x7f43cc06ddf0 2024-03-04T07:41:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.604+0000 7f43d3400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc06c210 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.604+0000 7f43d3400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc06c4f0 con 0x7f43cc1582e0 2024-03-04T07:41:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43c3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43cc0b0ed0 0x7f43cc0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43c3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f43cc0b0ed0 0x7f43cc0ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48098/0 (socket says 172.21.15.113:48098) 2024-03-04T07:41:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43c3e00640 1 -- 172.21.15.113:0/2436629938 learned_addr learned my addr 172.21.15.113:0/2436629938 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43d0a00640 1 --2- 172.21.15.113:0/2436629938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43cc1582e0 0x7f43cc0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43c3e00640 1 -- 172.21.15.113:0/2436629938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43cc1582e0 msgr2=0x7f43cc0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43c3e00640 1 --2- 172.21.15.113:0/2436629938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43cc1582e0 0x7f43cc0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43c3e00640 1 -- 172.21.15.113:0/2436629938 >> v1:172.21.15.136:6789/0 conn(0x7f43cc06ddf0 legacy=0x7f43cc15af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:33.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43c3e00640 1 -- 172.21.15.113:0/2436629938 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc06f6a0 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43c3e00640 1 --2- 172.21.15.113:0/2436629938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 0x7f43cc0ce2d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009870 tx=0x7f43b401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51dad18ea2b4fbb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:33.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43cbe00640 1 -- 172.21.15.113:0/2436629938 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b40b2070 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43cbe00640 1 -- 172.21.15.113:0/2436629938 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b40a9070 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.608+0000 7f43cbe00640 1 -- 172.21.15.113:0/2436629938 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b40ad880 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.612+0000 7f43d3400640 1 -- 172.21.15.113:0/2436629938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 msgr2=0x7f43cc0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.612+0000 7f43d3400640 1 --2- 172.21.15.113:0/2436629938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 0x7f43cc0ce2d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009870 tx=0x7f43b401c560 comp rx=0 tx=0).stop 2024-03-04T07:41:33.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.612+0000 7f43d3400640 1 -- 172.21.15.113:0/2436629938 shutdown_connections 2024-03-04T07:41:33.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.612+0000 7f43d3400640 1 --2- 172.21.15.113:0/2436629938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 0x7f43cc0ce2d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:33.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.612+0000 7f43d3400640 1 --2- 172.21.15.113:0/2436629938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43cc1582e0 0x7f43cc0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:33.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.612+0000 7f43d3400640 1 -- 172.21.15.113:0/2436629938 >> 172.21.15.113:0/2436629938 conn(0x7f43cc066db0 msgr2=0x7f43cc0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:33.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.612+0000 7f43d3400640 1 -- 172.21.15.113:0/2436629938 shutdown_connections 2024-03-04T07:41:33.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.612+0000 7f43d3400640 1 -- 172.21.15.113:0/2436629938 wait complete. 2024-03-04T07:41:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.612+0000 7f43d3400640 1 Processor -- start 2024-03-04T07:41:33.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.612+0000 7f43d3400640 1 -- start start 2024-03-04T07:41:33.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43cc06ddf0 0x7f43cc1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:33.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 0x7f43cc1f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:33.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43cc1582e0 0x7f43cc159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:33.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d3400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc167ed0 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d3400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc167d50 con 0x7f43cc1582e0 2024-03-04T07:41:33.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d3400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc168050 con 0x7f43cc06ddf0 2024-03-04T07:41:33.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 0x7f43cc1f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:33.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 0x7f43cc1f6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48104/0 (socket says 172.21.15.113:48104) 2024-03-04T07:41:33.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c3e00640 1 -- 172.21.15.113:0/1801571074 learned_addr learned my addr 172.21.15.113:0/1801571074 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d1400640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43cc1582e0 0x7f43cc159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d0a00640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43cc06ddf0 0x7f43cc1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c3e00640 1 -- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43cc06ddf0 msgr2=0x7f43cc1f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c3e00640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43cc06ddf0 0x7f43cc1f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c3e00640 1 -- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43cc1582e0 msgr2=0x7f43cc159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c3e00640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43cc1582e0 0x7f43cc159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c3e00640 1 -- 172.21.15.113:0/1801571074 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc1ecee0 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c3e00640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 0x7f43cc1f6cd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f43b40b0040 tx=0x7f43b4019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c9600640 1 -- 172.21.15.113:0/1801571074 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b40bb070 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c9600640 1 -- 172.21.15.113:0/1801571074 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b40adce0 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d3400640 1 -- 172.21.15.113:0/1801571074 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc1f72d0 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43c9600640 1 -- 172.21.15.113:0/1801571074 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b40b2040 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d3400640 1 -- 172.21.15.113:0/1801571074 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc1f7890 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.616+0000 7f43d3400640 1 -- 172.21.15.113:0/1801571074 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc068eb0 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.620+0000 7f43c9600640 1 -- 172.21.15.113:0/1801571074 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43b40be070 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.620+0000 7f43c9600640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43a8066f60 0x7f43a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:33.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.620+0000 7f43c9600640 1 -- 172.21.15.113:0/1801571074 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f43b411e120 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.620+0000 7f43d0a00640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43a8066f60 0x7f43a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:33.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.620+0000 7f43d0a00640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43a8066f60 0x7f43a8069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009ac0 tx=0x7f43bc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:33.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.624+0000 7f43c9600640 1 -- 172.21.15.113:0/1801571074 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f43b4016150 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.884+0000 7f43d3400640 1 -- 172.21.15.113:0/1801571074 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43cc075c00 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43c9600640 1 -- 172.21.15.113:0/1801571074 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f43b40ee870 con 0x7f43cc0b0ed0 2024-03-04T07:41:33.885 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:33.885 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:46.796316+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094,4938]},"id":1}]} 2024-03-04T07:41:33.885 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 38 2024-03-04T07:41:33.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 -- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43a8066f60 msgr2=0x7f43a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:33.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43a8066f60 0x7f43a8069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009ac0 tx=0x7f43bc009190 comp rx=0 tx=0).stop 2024-03-04T07:41:33.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 -- 172.21.15.113:0/1801571074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 msgr2=0x7f43cc1f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:33.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 0x7f43cc1f6cd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f43b40b0040 tx=0x7f43b4019300 comp rx=0 tx=0).stop 2024-03-04T07:41:33.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 -- 172.21.15.113:0/1801571074 shutdown_connections 2024-03-04T07:41:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43cc1582e0 0x7f43cc159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43a8066f60 0x7f43a8069420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43cc0b0ed0 0x7f43cc1f6cd0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 --2- 172.21.15.113:0/1801571074 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43cc06ddf0 0x7f43cc1f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 -- 172.21.15.113:0/1801571074 >> 172.21.15.113:0/1801571074 conn(0x7f43cc066db0 msgr2=0x7f43cc156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 -- 172.21.15.113:0/1801571074 shutdown_connections 2024-03-04T07:41:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:33.888+0000 7f43d3400640 1 -- 172.21.15.113:0/1801571074 wait complete. 2024-03-04T07:41:33.917 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:41:35.918 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:36.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.060+0000 7fdbf8c00640 1 Processor -- start 2024-03-04T07:41:36.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.060+0000 7fdbf8c00640 1 -- start start 2024-03-04T07:41:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.060+0000 7fdbf8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf40c7a80 0x7fdbf40cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.060+0000 7fdbf8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdbf406cb60 0x7fdbf40cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.060+0000 7fdbf8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf4072d80 con 0x7fdbf4152c60 2024-03-04T07:41:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.060+0000 7fdbf8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf406b060 con 0x7fdbf406cb60 2024-03-04T07:41:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.060+0000 7fdbf8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4073510 con 0x7fdbf40c7a80 2024-03-04T07:41:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.064+0000 7fdbf1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf40c7a80 0x7fdbf40cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.064+0000 7fdbf2800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdbf406cb60 0x7fdbf40cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.064+0000 7fdbf2800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdbf406cb60 0x7fdbf40cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:56932/0 (socket says 172.21.15.113:56932) 2024-03-04T07:41:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.064+0000 7fdbf1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf40c7a80 0x7fdbf40cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:55670/0 (socket says 172.21.15.113:55670) 2024-03-04T07:41:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.064+0000 7fdbf2800640 1 -- 172.21.15.113:0/2940140315 learned_addr learned my addr 172.21.15.113:0/2940140315 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:36.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.064+0000 7fdbf2800640 1 -- 172.21.15.113:0/2940140315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf40c7a80 msgr2=0x7fdbf40cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:36.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.064+0000 7fdbf2800640 1 --2- 172.21.15.113:0/2940140315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf40c7a80 0x7fdbf40cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:36.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.064+0000 7fdbf2800640 1 -- 172.21.15.113:0/2940140315 >> v1:172.21.15.136:6789/0 conn(0x7fdbf4152c60 legacy=0x7fdbf415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:36.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.064+0000 7fdbf2800640 1 -- 172.21.15.113:0/2940140315 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf406b2b0 con 0x7fdbf406cb60 2024-03-04T07:41:36.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.064+0000 7fdbf2800640 1 --2- 172.21.15.113:0/2940140315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf406cb60 0x7fdbf40cd550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009e50 tx=0x7fdbec01ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=413a9006d2a23ee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:36.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf0a00640 1 -- 172.21.15.113:0/2940140315 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0a58d0 con 0x7fdbf406cb60 2024-03-04T07:41:36.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf0a00640 1 -- 172.21.15.113:0/2940140315 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec0a5a70 con 0x7fdbf406cb60 2024-03-04T07:41:36.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf0a00640 1 -- 172.21.15.113:0/2940140315 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0a5e40 con 0x7fdbf406cb60 2024-03-04T07:41:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 -- 172.21.15.113:0/2940140315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf406cb60 msgr2=0x7fdbf40cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 --2- 172.21.15.113:0/2940140315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf406cb60 0x7fdbf40cd550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009e50 tx=0x7fdbec01ca00 comp rx=0 tx=0).stop 2024-03-04T07:41:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 -- 172.21.15.113:0/2940140315 shutdown_connections 2024-03-04T07:41:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 --2- 172.21.15.113:0/2940140315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf406cb60 0x7fdbf40cd550 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 --2- 172.21.15.113:0/2940140315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf40c7a80 0x7fdbf40cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 -- 172.21.15.113:0/2940140315 >> 172.21.15.113:0/2940140315 conn(0x7fdbf40671b0 msgr2=0x7fdbf406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 -- 172.21.15.113:0/2940140315 shutdown_connections 2024-03-04T07:41:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 -- 172.21.15.113:0/2940140315 wait complete. 2024-03-04T07:41:36.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 Processor -- start 2024-03-04T07:41:36.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 -- start start 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf406cb60 0x7fdbf41499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf40c7a80 0x7fdbf414e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf4152c60 0x7fdbf40c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40731d0 con 0x7fdbf40c7a80 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf406b590 con 0x7fdbf406cb60 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40c8ac0 con 0x7fdbf4152c60 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf40c7a80 0x7fdbf414e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf40c7a80 0x7fdbf414e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48106/0 (socket says 172.21.15.113:48106) 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf1400640 1 -- 172.21.15.113:0/1460507180 learned_addr learned my addr 172.21.15.113:0/1460507180 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf2800640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf4152c60 0x7fdbf40c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.068+0000 7fdbf1e00640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf406cb60 0x7fdbf41499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:36.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf1400640 1 -- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf4152c60 msgr2=0x7fdbf40c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:36.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf1400640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf4152c60 0x7fdbf40c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:36.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf1400640 1 -- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf406cb60 msgr2=0x7fdbf41499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:36.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf1400640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf406cb60 0x7fdbf41499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:36.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf1400640 1 -- 172.21.15.113:0/1460507180 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf414e720 con 0x7fdbf40c7a80 2024-03-04T07:41:36.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf1e00640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf406cb60 0x7fdbf41499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:36.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf1400640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf40c7a80 0x7fdbf414e1e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000e870 tx=0x7fdbe000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:36.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbea000640 1 -- 172.21.15.113:0/1460507180 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0017070 con 0x7fdbf40c7a80 2024-03-04T07:41:36.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbea000640 1 -- 172.21.15.113:0/1460507180 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe000c940 con 0x7fdbf40c7a80 2024-03-04T07:41:36.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbea000640 1 -- 172.21.15.113:0/1460507180 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe000ccf0 con 0x7fdbf40c7a80 2024-03-04T07:41:36.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf8c00640 1 -- 172.21.15.113:0/1460507180 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf4149f80 con 0x7fdbf40c7a80 2024-03-04T07:41:36.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf8c00640 1 -- 172.21.15.113:0/1460507180 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf41ea850 con 0x7fdbf40c7a80 2024-03-04T07:41:36.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf8c00640 1 -- 172.21.15.113:0/1460507180 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf4075c00 con 0x7fdbf40c7a80 2024-03-04T07:41:36.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbea000640 1 -- 172.21.15.113:0/1460507180 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdbe001a070 con 0x7fdbf40c7a80 2024-03-04T07:41:36.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbea000640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbd4066f90 0x7fdbd4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:36.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbea000640 1 -- 172.21.15.113:0/1460507180 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fdbe0084fe0 con 0x7fdbf40c7a80 2024-03-04T07:41:36.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.072+0000 7fdbf1e00640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbd4066f90 0x7fdbd4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:36.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.076+0000 7fdbf1e00640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbd4066f90 0x7fdbd4069450 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4001710 tx=0x7fdbe4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:36.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.084+0000 7fdbea000640 1 -- 172.21.15.113:0/1460507180 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdbe00557d0 con 0x7fdbf40c7a80 2024-03-04T07:41:36.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.364+0000 7fdbf8c00640 1 -- 172.21.15.113:0/1460507180 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbf4072fb0 con 0x7fdbf40c7a80 2024-03-04T07:41:36.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.364+0000 7fdbea000640 1 -- 172.21.15.113:0/1460507180 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fdbe0059870 con 0x7fdbf40c7a80 2024-03-04T07:41:36.364 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:36.365 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:46.796316+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094,4938]},"id":1}]} 2024-03-04T07:41:36.365 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 38 2024-03-04T07:41:36.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 -- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbd4066f90 msgr2=0x7fdbd4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbd4066f90 0x7fdbd4069450 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4001710 tx=0x7fdbe4009040 comp rx=0 tx=0).stop 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 -- 172.21.15.113:0/1460507180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf40c7a80 msgr2=0x7fdbf414e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf40c7a80 0x7fdbf414e1e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000e870 tx=0x7fdbe000ec70 comp rx=0 tx=0).stop 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 -- 172.21.15.113:0/1460507180 shutdown_connections 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdbf4152c60 0x7fdbf40c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdbd4066f90 0x7fdbd4069450 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf40c7a80 0x7fdbf414e1e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 --2- 172.21.15.113:0/1460507180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbf406cb60 0x7fdbf41499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 -- 172.21.15.113:0/1460507180 >> 172.21.15.113:0/1460507180 conn(0x7fdbf40671b0 msgr2=0x7fdbf40c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 -- 172.21.15.113:0/1460507180 shutdown_connections 2024-03-04T07:41:36.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:36.368+0000 7fdbf8c00640 1 -- 172.21.15.113:0/1460507180 wait complete. 2024-03-04T07:41:36.387 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:41:38.388 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:38.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.544+0000 7fab4a400640 1 Processor -- start 2024-03-04T07:41:38.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.544+0000 7fab4a400640 1 -- start start 2024-03-04T07:41:38.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab4a400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fab44154d00 0x7fab44071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:38.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab4a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 0x7fab4415ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:38.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab4a400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4415b720 con 0x7fab440c9640 2024-03-04T07:41:38.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab4a400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab440c8630 con 0x7fab44154d00 2024-03-04T07:41:38.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab4a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab441550e0 con 0x7fab4415a990 2024-03-04T07:41:38.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab43e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fab440c9640 0x7fab440c2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56974/0 (socket says 172.21.15.113:56974) 2024-03-04T07:41:38.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab43e00640 1 -- 172.21.15.113:0/217255194 learned_addr learned my addr 172.21.15.113:0/217255194 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:41:38.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab49000640 1 --2- 172.21.15.113:0/217255194 >> v2:172.21.15.136:3300/0 conn(0x7fab44154d00 0x7fab44071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:38.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab49a00640 1 --2- 172.21.15.113:0/217255194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 0x7fab4415ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:38.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab43400640 1 -- 172.21.15.113:0/217255194 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648031394 0 0) 0x7fab4415b720 con 0x7fab440c9640 2024-03-04T07:41:38.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab43400640 1 -- 172.21.15.113:0/217255194 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab2c003530 con 0x7fab440c9640 2024-03-04T07:41:38.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab49000640 1 -- 172.21.15.113:0/217255194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 msgr2=0x7fab4415ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:38.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab49000640 1 --2- 172.21.15.113:0/217255194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 0x7fab4415ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:38.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab49000640 1 -- 172.21.15.113:0/217255194 >> v1:172.21.15.113:6789/0 conn(0x7fab440c9640 legacy=0x7fab440c2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:38.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.548+0000 7fab49000640 1 -- 172.21.15.113:0/217255194 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440c8880 con 0x7fab44154d00 2024-03-04T07:41:38.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab49000640 1 --2- 172.21.15.113:0/217255194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44154d00 0x7fab44071a30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab380097c0 tx=0x7fab3801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a2a0f0835db2574 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:38.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab43400640 1 -- 172.21.15.113:0/217255194 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380b5070 con 0x7fab44154d00 2024-03-04T07:41:38.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab43400640 1 -- 172.21.15.113:0/217255194 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab380a8990 con 0x7fab44154d00 2024-03-04T07:41:38.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab43400640 1 -- 172.21.15.113:0/217255194 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380a8d60 con 0x7fab44154d00 2024-03-04T07:41:38.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab4a400640 1 -- 172.21.15.113:0/217255194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44154d00 msgr2=0x7fab44071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:38.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab4a400640 1 --2- 172.21.15.113:0/217255194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44154d00 0x7fab44071a30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab380097c0 tx=0x7fab3801ec90 comp rx=0 tx=0).stop 2024-03-04T07:41:38.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab4a400640 1 -- 172.21.15.113:0/217255194 shutdown_connections 2024-03-04T07:41:38.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab4a400640 1 --2- 172.21.15.113:0/217255194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 0x7fab4415ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:38.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab4a400640 1 --2- 172.21.15.113:0/217255194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44154d00 0x7fab44071a30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:38.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab4a400640 1 -- 172.21.15.113:0/217255194 >> 172.21.15.113:0/217255194 conn(0x7fab440c9a30 msgr2=0x7fab440c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:38.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab4a400640 1 -- 172.21.15.113:0/217255194 shutdown_connections 2024-03-04T07:41:38.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab4a400640 1 -- 172.21.15.113:0/217255194 wait complete. 2024-03-04T07:41:38.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.552+0000 7fab4a400640 1 Processor -- start 2024-03-04T07:41:38.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab4a400640 1 -- start start 2024-03-04T07:41:38.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab4a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44154d00 0x7fab441f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:38.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab4a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 0x7fab4415c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:38.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab4a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab441583b0 0x7fab4415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:38.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab4a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44071f70 con 0x7fab441583b0 2024-03-04T07:41:38.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab4a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440c8e40 con 0x7fab44154d00 2024-03-04T07:41:38.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab4a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4415b320 con 0x7fab4415a990 2024-03-04T07:41:38.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab441583b0 0x7fab4415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:38.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab441583b0 0x7fab4415bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58688/0 (socket says 172.21.15.113:58688) 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49a00640 1 -- 172.21.15.113:0/2802168163 learned_addr learned my addr 172.21.15.113:0/2802168163 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab43e00640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 0x7fab4415c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49000640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44154d00 0x7fab441f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49a00640 1 -- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 msgr2=0x7fab4415c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49a00640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 0x7fab4415c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49a00640 1 -- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44154d00 msgr2=0x7fab441f0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49a00640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44154d00 0x7fab441f0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49a00640 1 -- 172.21.15.113:0/2802168163 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab441f0170 con 0x7fab441583b0 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49000640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44154d00 0x7fab441f0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab43e00640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 0x7fab4415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab49a00640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab441583b0 0x7fab4415bb50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00efc0 tx=0x7fab3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab40c00640 1 -- 172.21.15.113:0/2802168163 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c018020 con 0x7fab441583b0 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab40c00640 1 -- 172.21.15.113:0/2802168163 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c014480 con 0x7fab441583b0 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab4a400640 1 -- 172.21.15.113:0/2802168163 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4415cdb0 con 0x7fab441583b0 2024-03-04T07:41:38.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab40c00640 1 -- 172.21.15.113:0/2802168163 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c014870 con 0x7fab441583b0 2024-03-04T07:41:38.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab4a400640 1 -- 172.21.15.113:0/2802168163 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4415d3e0 con 0x7fab441583b0 2024-03-04T07:41:38.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.556+0000 7fab4a400640 1 -- 172.21.15.113:0/2802168163 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab44075a80 con 0x7fab441583b0 2024-03-04T07:41:38.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.560+0000 7fab40c00640 1 -- 172.21.15.113:0/2802168163 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab3c01b020 con 0x7fab441583b0 2024-03-04T07:41:38.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.560+0000 7fab40c00640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab24066f90 0x7fab24069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:38.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.560+0000 7fab40c00640 1 -- 172.21.15.113:0/2802168163 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fab3c086140 con 0x7fab441583b0 2024-03-04T07:41:38.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.560+0000 7fab49000640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab24066f90 0x7fab24069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:38.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.560+0000 7fab49000640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab24066f90 0x7fab24069450 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fab380b2000 tx=0x7fab38008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:38.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.564+0000 7fab40c00640 1 -- 172.21.15.113:0/2802168163 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fab3c022020 con 0x7fab441583b0 2024-03-04T07:41:38.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 -- 172.21.15.113:0/2802168163 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab440c8410 con 0x7fab441583b0 2024-03-04T07:41:38.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab40c00640 1 -- 172.21.15.113:0/2802168163 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fab441f0170 con 0x7fab441583b0 2024-03-04T07:41:38.786 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:38.786 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5099,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:40:46.796316+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":2,"incarnation":20,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6836/2997246507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2997246507},{"type":"v1","addr":"172.21.15.113:6836","nonce":2997246507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094,4938]},"id":1}]} 2024-03-04T07:41:38.786 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 38 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 -- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab24066f90 msgr2=0x7fab24069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab24066f90 0x7fab24069450 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fab380b2000 tx=0x7fab38008040 comp rx=0 tx=0).stop 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 -- 172.21.15.113:0/2802168163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab441583b0 msgr2=0x7fab4415bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab441583b0 0x7fab4415bb50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00efc0 tx=0x7fab3c00c490 comp rx=0 tx=0).stop 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 -- 172.21.15.113:0/2802168163 shutdown_connections 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab24066f90 0x7fab24069450 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab441583b0 0x7fab4415bb50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab4415a990 0x7fab4415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 --2- 172.21.15.113:0/2802168163 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44154d00 0x7fab441f0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 -- 172.21.15.113:0/2802168163 >> 172.21.15.113:0/2802168163 conn(0x7fab440c9a30 msgr2=0x7fab44152130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:38.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 -- 172.21.15.113:0/2802168163 shutdown_connections 2024-03-04T07:41:38.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:38.788+0000 7fab4a400640 1 -- 172.21.15.113:0/2802168163 wait complete. 2024-03-04T07:41:38.808 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:41:40.809 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:40.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfbb000640 1 Processor -- start 2024-03-04T07:41:40.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfbb000640 1 -- start start 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfbb000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcfb4152920 0x7fcfb4071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfbb000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcfb415bf20 0x7fcfb415c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfbb000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb406bb10 con 0x7fcfb406c580 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfbb000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb40c7a40 con 0x7fcfb415bf20 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfbb000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb40c7d20 con 0x7fcfb4152920 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfba600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcfb415bf20 0x7fcfb415c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfb9200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcfb406c580 0x7fcfb40c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56976/0 (socket says 172.21.15.113:56976) 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfba600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcfb415bf20 0x7fcfb415c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58702/0 (socket says 172.21.15.113:58702) 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.956+0000 7fcfb9200640 1 -- 172.21.15.113:0/2360307165 learned_addr learned my addr 172.21.15.113:0/2360307165 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfb9c00640 1 --2- 172.21.15.113:0/2360307165 >> v2:172.21.15.136:3300/0 conn(0x7fcfb4152920 0x7fcfb4071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:40.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfba600640 1 -- 172.21.15.113:0/2360307165 >> v2:172.21.15.136:3300/0 conn(0x7fcfb4152920 msgr2=0x7fcfb4071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:40.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfba600640 1 --2- 172.21.15.113:0/2360307165 >> v2:172.21.15.136:3300/0 conn(0x7fcfb4152920 0x7fcfb4071a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:40.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfba600640 1 -- 172.21.15.113:0/2360307165 >> v1:172.21.15.113:6789/0 conn(0x7fcfb406c580 legacy=0x7fcfb40c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:40.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfba600640 1 -- 172.21.15.113:0/2360307165 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb406bd80 con 0x7fcfb415bf20 2024-03-04T07:41:40.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfb9c00640 1 --2- 172.21.15.113:0/2360307165 >> v2:172.21.15.136:3300/0 conn(0x7fcfb4152920 0x7fcfb4071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:40.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfba600640 1 --2- 172.21.15.113:0/2360307165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb415bf20 0x7fcfb415c3b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40ca670 tx=0x7fcfac01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d355eeb990035763 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:40.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfb3e00640 1 -- 172.21.15.113:0/2360307165 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0b2070 con 0x7fcfb415bf20 2024-03-04T07:41:40.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfb3e00640 1 -- 172.21.15.113:0/2360307165 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac0a5760 con 0x7fcfb415bf20 2024-03-04T07:41:40.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfb3e00640 1 -- 172.21.15.113:0/2360307165 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0a5b30 con 0x7fcfb415bf20 2024-03-04T07:41:40.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfbb000640 1 -- 172.21.15.113:0/2360307165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb415bf20 msgr2=0x7fcfb415c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:40.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.960+0000 7fcfbb000640 1 --2- 172.21.15.113:0/2360307165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb415bf20 0x7fcfb415c3b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcfb40ca670 tx=0x7fcfac01c8a0 comp rx=0 tx=0).stop 2024-03-04T07:41:40.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 -- 172.21.15.113:0/2360307165 shutdown_connections 2024-03-04T07:41:40.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 --2- 172.21.15.113:0/2360307165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb415bf20 0x7fcfb415c3b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:40.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 --2- 172.21.15.113:0/2360307165 >> v2:172.21.15.136:3300/0 conn(0x7fcfb4152920 0x7fcfb4071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:40.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 -- 172.21.15.113:0/2360307165 >> 172.21.15.113:0/2360307165 conn(0x7fcfb40c9a30 msgr2=0x7fcfb40c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:40.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 -- 172.21.15.113:0/2360307165 shutdown_connections 2024-03-04T07:41:40.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 -- 172.21.15.113:0/2360307165 wait complete. 2024-03-04T07:41:40.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 Processor -- start 2024-03-04T07:41:40.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 -- start start 2024-03-04T07:41:40.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfb406c580 0x7fcfb4161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:40.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb4152920 0x7fcfb4148430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:40.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcfb415bf20 0x7fcfb4072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:40.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40c4430 con 0x7fcfb4152920 2024-03-04T07:41:40.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb4152d00 con 0x7fcfb406c580 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfbb000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40c45b0 con 0x7fcfb415bf20 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfb9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb4152920 0x7fcfb4148430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfb9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb4152920 0x7fcfb4148430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58708/0 (socket says 172.21.15.113:58708) 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfb9200640 1 -- 172.21.15.113:0/3912299870 learned_addr learned my addr 172.21.15.113:0/3912299870 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfb9c00640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfb406c580 0x7fcfb4161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfba600640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcfb415bf20 0x7fcfb4072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfb9200640 1 -- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcfb415bf20 msgr2=0x7fcfb4072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfb9200640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcfb415bf20 0x7fcfb4072440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfb9200640 1 -- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfb406c580 msgr2=0x7fcfb4161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfb9200640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfb406c580 0x7fcfb4161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfb9200640 1 -- 172.21.15.113:0/3912299870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40c20f0 con 0x7fcfb4152920 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfba600640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcfb415bf20 0x7fcfb4072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:40.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.964+0000 7fcfb9200640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb4152920 0x7fcfb4148430 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c420 tx=0x7fcfa000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:40.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.968+0000 7fcfb1600640 1 -- 172.21.15.113:0/3912299870 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0017070 con 0x7fcfb4152920 2024-03-04T07:41:40.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.968+0000 7fcfb1600640 1 -- 172.21.15.113:0/3912299870 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa0005cf0 con 0x7fcfb4152920 2024-03-04T07:41:40.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.968+0000 7fcfbb000640 1 -- 172.21.15.113:0/3912299870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb4147950 con 0x7fcfb4152920 2024-03-04T07:41:40.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.968+0000 7fcfb1600640 1 -- 172.21.15.113:0/3912299870 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00072f0 con 0x7fcfb4152920 2024-03-04T07:41:40.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.968+0000 7fcfbb000640 1 -- 172.21.15.113:0/3912299870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb4147f10 con 0x7fcfb4152920 2024-03-04T07:41:40.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.968+0000 7fcfbb000640 1 -- 172.21.15.113:0/3912299870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb4148110 con 0x7fcfb4152920 2024-03-04T07:41:40.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.968+0000 7fcfb1600640 1 -- 172.21.15.113:0/3912299870 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfa001a070 con 0x7fcfb4152920 2024-03-04T07:41:40.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.968+0000 7fcfb1600640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf94066f90 0x7fcf94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:40.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.968+0000 7fcfb1600640 1 -- 172.21.15.113:0/3912299870 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fcfa0084610 con 0x7fcfb4152920 2024-03-04T07:41:40.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.972+0000 7fcfb9c00640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf94066f90 0x7fcf94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:40.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.972+0000 7fcfb9c00640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf94066f90 0x7fcf94069450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009950 tx=0x7fcfa8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:40.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:40.972+0000 7fcfb1600640 1 -- 172.21.15.113:0/3912299870 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcfb4148110 con 0x7fcfb4152920 2024-03-04T07:41:41.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.172+0000 7fcfbb000640 1 -- 172.21.15.113:0/3912299870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfb4075a80 con 0x7fcfb4152920 2024-03-04T07:41:41.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.172+0000 7fcfb1600640 1 -- 172.21.15.113:0/3912299870 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5812 (secure 0 0 0) 0x7fcfa0054d00 con 0x7fcfb4152920 2024-03-04T07:41:41.171 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:41.171 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:41:40.631226+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5094,5244]},"id":1}]} 2024-03-04T07:41:41.172 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 39 2024-03-04T07:41:41.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 -- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf94066f90 msgr2=0x7fcf94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:41.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf94066f90 0x7fcf94069450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009950 tx=0x7fcfa8008040 comp rx=0 tx=0).stop 2024-03-04T07:41:41.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 -- 172.21.15.113:0/3912299870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb4152920 msgr2=0x7fcfb4148430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:41.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb4152920 0x7fcfb4148430 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c420 tx=0x7fcfa000c8e0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 -- 172.21.15.113:0/3912299870 shutdown_connections 2024-03-04T07:41:41.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcfb415bf20 0x7fcfb4072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcf94066f90 0x7fcf94069450 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfb4152920 0x7fcfb4148430 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 --2- 172.21.15.113:0/3912299870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfb406c580 0x7fcfb4161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 -- 172.21.15.113:0/3912299870 >> 172.21.15.113:0/3912299870 conn(0x7fcfb40c9a30 msgr2=0x7fcfb40cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:41.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 -- 172.21.15.113:0/3912299870 shutdown_connections 2024-03-04T07:41:41.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.176+0000 7fcfbb000640 1 -- 172.21.15.113:0/3912299870 wait complete. 2024-03-04T07:41:41.192 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-03-04T07:41:41.192 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:41:41.193 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.352+0000 7f0d81200640 1 Processor -- start 2024-03-04T07:41:41.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.352+0000 7f0d81200640 1 -- start start 2024-03-04T07:41:41.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d81200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 0x7f0d7c157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:41.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d81200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0d7c0c85f0 0x7f0d7c161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:41.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d81200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c0c1ec0 con 0x7f0d7c157c50 2024-03-04T07:41:41.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d81200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c0c9b30 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d81200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0c12a0 con 0x7f0d7c0b20b0 2024-03-04T07:41:41.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d79e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 0x7f0d7c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:41.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d7a800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0d7c0c85f0 0x7f0d7c161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:41.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d73e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f0d7c157c50 0x7f0d7c06b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:38236/0 (socket says 172.21.15.113:38236) 2024-03-04T07:41:41.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d7a800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0d7c0c85f0 0x7f0d7c161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:36980/0 (socket says 172.21.15.113:36980) 2024-03-04T07:41:41.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d79e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 0x7f0d7c157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:41324/0 (socket says 172.21.15.113:41324) 2024-03-04T07:41:41.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d7a800640 1 -- 172.21.15.113:0/3489409327 learned_addr learned my addr 172.21.15.113:0/3489409327 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:41.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d79e00640 1 -- 172.21.15.113:0/3489409327 >> v2:172.21.15.136:3300/0 conn(0x7f0d7c0c85f0 msgr2=0x7f0d7c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:41.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d79e00640 1 --2- 172.21.15.113:0/3489409327 >> v2:172.21.15.136:3300/0 conn(0x7f0d7c0c85f0 0x7f0d7c161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d79e00640 1 -- 172.21.15.113:0/3489409327 >> v1:172.21.15.136:6789/0 conn(0x7f0d7c157c50 legacy=0x7f0d7c06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:41.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d79e00640 1 -- 172.21.15.113:0/3489409327 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0c1580 con 0x7f0d7c0b20b0 2024-03-04T07:41:41.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.356+0000 7f0d7a800640 1 --2- 172.21.15.113:0/3489409327 >> v2:172.21.15.136:3300/0 conn(0x7f0d7c0c85f0 0x7f0d7c161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:41:41.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d79e00640 1 --2- 172.21.15.113:0/3489409327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 0x7f0d7c157710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0097c0 tx=0x7f0d6c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f59b4a2e75dd0b02 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:41.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d79400640 1 -- 172.21.15.113:0/3489409327 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0ab5f0 con 0x7f0d7c0b20b0 2024-03-04T07:41:41.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d79400640 1 -- 172.21.15.113:0/3489409327 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c0b0070 con 0x7f0d7c0b20b0 2024-03-04T07:41:41.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d79400640 1 -- 172.21.15.113:0/3489409327 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c0ab9b0 con 0x7f0d7c0b20b0 2024-03-04T07:41:41.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d81200640 1 -- 172.21.15.113:0/3489409327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 msgr2=0x7f0d7c157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:41.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d81200640 1 --2- 172.21.15.113:0/3489409327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 0x7f0d7c157710 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0097c0 tx=0x7f0d6c01c470 comp rx=0 tx=0).stop 2024-03-04T07:41:41.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d81200640 1 -- 172.21.15.113:0/3489409327 shutdown_connections 2024-03-04T07:41:41.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d81200640 1 --2- 172.21.15.113:0/3489409327 >> v2:172.21.15.136:3300/0 conn(0x7f0d7c0c85f0 0x7f0d7c161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d81200640 1 --2- 172.21.15.113:0/3489409327 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 0x7f0d7c157710 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d81200640 1 -- 172.21.15.113:0/3489409327 >> 172.21.15.113:0/3489409327 conn(0x7f0d7c06e820 msgr2=0x7f0d7c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:41.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d81200640 1 -- 172.21.15.113:0/3489409327 shutdown_connections 2024-03-04T07:41:41.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.360+0000 7f0d81200640 1 -- 172.21.15.113:0/3489409327 wait complete. 2024-03-04T07:41:41.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d81200640 1 Processor -- start 2024-03-04T07:41:41.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d81200640 1 -- start start 2024-03-04T07:41:41.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d81200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 0x7f0d7c1f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:41.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d81200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d7c0c85f0 0x7f0d7c14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:41.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d81200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d7c157c50 0x7f0d7c14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:41.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d81200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0c1b80 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d81200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c156ec0 con 0x7f0d7c157c50 2024-03-04T07:41:41.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d81200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0c80c0 con 0x7f0d7c0b20b0 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d73e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d7c0c85f0 0x7f0d7c14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d73e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d7c0c85f0 0x7f0d7c14cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58724/0 (socket says 172.21.15.113:58724) 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d79e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 0x7f0d7c1f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d73e00640 1 -- 172.21.15.113:0/3240449318 learned_addr learned my addr 172.21.15.113:0/3240449318 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d7a800640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d7c157c50 0x7f0d7c14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d73e00640 1 -- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 msgr2=0x7f0d7c1f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d73e00640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 0x7f0d7c1f4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d73e00640 1 -- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d7c157c50 msgr2=0x7f0d7c14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.364+0000 7f0d73e00640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d7c157c50 0x7f0d7c14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d73e00640 1 -- 172.21.15.113:0/3240449318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0c6550 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d73e00640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d7c0c85f0 0x7f0d7c14cf60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400c430 tx=0x7f0d6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d72000640 1 -- 172.21.15.113:0/3240449318 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d640074e0 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d72000640 1 -- 172.21.15.113:0/3240449318 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d64007680 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d81200640 1 -- 172.21.15.113:0/3240449318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c1535e0 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d72000640 1 -- 172.21.15.113:0/3240449318 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64004a30 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d81200640 1 -- 172.21.15.113:0/3240449318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c153ba0 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d81200640 1 -- 172.21.15.113:0/3240449318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c075c00 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d72000640 1 -- 172.21.15.113:0/3240449318 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d6401a020 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d72000640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d58066f60 0x7f0d58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:41.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.368+0000 7f0d72000640 1 -- 172.21.15.113:0/3240449318 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f0d640850b0 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.372+0000 7f0d79e00640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d58066f60 0x7f0d58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:41.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.372+0000 7f0d79e00640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d58066f60 0x7f0d58069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0af000 tx=0x7f0d6c0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:41.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.376+0000 7f0d72000640 1 -- 172.21.15.113:0/3240449318 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0d64021070 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.580+0000 7f0d81200640 1 -- 172.21.15.113:0/3240449318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d7c0c83d0 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.580+0000 7f0d72000640 1 -- 172.21.15.113:0/3240449318 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f0d64055830 con 0x7f0d7c0c85f0 2024-03-04T07:41:41.580 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:41.580 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:41:40.631226+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5094,5244]},"id":1}]} 2024-03-04T07:41:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 39 2024-03-04T07:41:41.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 -- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d58066f60 msgr2=0x7f0d58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:41.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d58066f60 0x7f0d58069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0af000 tx=0x7f0d6c0a7000 comp rx=0 tx=0).stop 2024-03-04T07:41:41.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 -- 172.21.15.113:0/3240449318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d7c0c85f0 msgr2=0x7f0d7c14cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:41.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d7c0c85f0 0x7f0d7c14cf60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400c430 tx=0x7f0d6400c8f0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 -- 172.21.15.113:0/3240449318 shutdown_connections 2024-03-04T07:41:41.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d7c157c50 0x7f0d7c14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d58066f60 0x7f0d58069420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d7c0c85f0 0x7f0d7c14cf60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 --2- 172.21.15.113:0/3240449318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d7c0b20b0 0x7f0d7c1f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:41.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 -- 172.21.15.113:0/3240449318 >> 172.21.15.113:0/3240449318 conn(0x7f0d7c06e820 msgr2=0x7f0d7c1507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:41.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 -- 172.21.15.113:0/3240449318 shutdown_connections 2024-03-04T07:41:41.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:41.584+0000 7f0d81200640 1 -- 172.21.15.113:0/3240449318 wait complete. 2024-03-04T07:41:41.603 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 39, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 39, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5094": { "addr": "172.21.15.113:6833/605142559", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 605142559, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 605142559, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5094, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 21 }, "gid_5099": { "addr": "172.21.15.136:6835/3944001911", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3944001911, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3944001911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5099, "incarnation": 39, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:replay", "state_seq": 1 }, "gid_5244": { "addr": "172.21.15.113:6837/742633123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 742633123, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 742633123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5244, "incarnation": 32, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 43, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:41:40.631226+0000", "qdb_cluster": [ 5094, 5244 ], "qdb_leader": 5244, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5244, "mds_1": 5094, "mds_2": 5099 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:41:43.604 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:43.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb76c00640 1 Processor -- start 2024-03-04T07:41:43.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb76c00640 1 -- start start 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb76c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb701527f0 0x7ffb70070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb76c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffb700c8200 0x7ffb70072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb76c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb70152bd0 0x7ffb701587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb76c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb70071df0 con 0x7ffb701527f0 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb76c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb701597b0 con 0x7ffb700c8200 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb76c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb701591f0 con 0x7ffb70152bd0 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb6fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb701527f0 0x7ffb70070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb6fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb701527f0 0x7ffb70070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58740/0 (socket says 172.21.15.113:58740) 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb6fe00640 1 -- 172.21.15.113:0/3626440961 learned_addr learned my addr 172.21.15.113:0/3626440961 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb74c00640 1 --2- 172.21.15.113:0/3626440961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb70152bd0 0x7ffb701587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb67400640 1 --2- 172.21.15.113:0/3626440961 >> v2:172.21.15.136:3300/0 conn(0x7ffb700c8200 0x7ffb70072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:43.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb6fe00640 1 -- 172.21.15.113:0/3626440961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb70152bd0 msgr2=0x7ffb701587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:43.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb6fe00640 1 --2- 172.21.15.113:0/3626440961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb70152bd0 0x7ffb701587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb6fe00640 1 -- 172.21.15.113:0/3626440961 >> v2:172.21.15.136:3300/0 conn(0x7ffb700c8200 msgr2=0x7ffb70072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:43.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb6fe00640 1 --2- 172.21.15.113:0/3626440961 >> v2:172.21.15.136:3300/0 conn(0x7ffb700c8200 0x7ffb70072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.744+0000 7ffb6fe00640 1 -- 172.21.15.113:0/3626440961 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb701594d0 con 0x7ffb701527f0 2024-03-04T07:41:43.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb6fe00640 1 --2- 172.21.15.113:0/3626440961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb701527f0 0x7ffb70070d40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ffb700cd230 tx=0x7ffb6001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8243f00aeb7b13b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:43.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb6f400640 1 -- 172.21.15.113:0/3626440961 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600a5780 con 0x7ffb701527f0 2024-03-04T07:41:43.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb6f400640 1 -- 172.21.15.113:0/3626440961 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb600a5920 con 0x7ffb701527f0 2024-03-04T07:41:43.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb6f400640 1 -- 172.21.15.113:0/3626440961 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb600a5cf0 con 0x7ffb701527f0 2024-03-04T07:41:43.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb76c00640 1 -- 172.21.15.113:0/3626440961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb701527f0 msgr2=0x7ffb70070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:43.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb76c00640 1 --2- 172.21.15.113:0/3626440961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb701527f0 0x7ffb70070d40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ffb700cd230 tx=0x7ffb6001c9f0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb76c00640 1 -- 172.21.15.113:0/3626440961 shutdown_connections 2024-03-04T07:41:43.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb76c00640 1 --2- 172.21.15.113:0/3626440961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb70152bd0 0x7ffb701587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb76c00640 1 --2- 172.21.15.113:0/3626440961 >> v2:172.21.15.136:3300/0 conn(0x7ffb700c8200 0x7ffb70072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb76c00640 1 --2- 172.21.15.113:0/3626440961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb701527f0 0x7ffb70070d40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb76c00640 1 -- 172.21.15.113:0/3626440961 >> 172.21.15.113:0/3626440961 conn(0x7ffb7006e820 msgr2=0x7ffb7006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:43.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb76c00640 1 -- 172.21.15.113:0/3626440961 shutdown_connections 2024-03-04T07:41:43.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.748+0000 7ffb76c00640 1 -- 172.21.15.113:0/3626440961 wait complete. 2024-03-04T07:41:43.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.752+0000 7ffb76c00640 1 Processor -- start 2024-03-04T07:41:43.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.752+0000 7ffb76c00640 1 -- start start 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.752+0000 7ffb76c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb700c8200 0x7ffb700c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.752+0000 7ffb76c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb701527f0 0x7ffb701f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.752+0000 7ffb76c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb70152bd0 0x7ffb701f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.752+0000 7ffb76c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7015bf40 con 0x7ffb701527f0 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.752+0000 7ffb76c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7015bdc0 con 0x7ffb70152bd0 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.752+0000 7ffb76c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7015c0c0 con 0x7ffb700c8200 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb67400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb701527f0 0x7ffb701f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb6fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb700c8200 0x7ffb700c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb6fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb700c8200 0x7ffb700c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:41356/0 (socket says 172.21.15.113:41356) 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb6fe00640 1 -- 172.21.15.113:0/642598307 learned_addr learned my addr 172.21.15.113:0/642598307 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb74c00640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb70152bd0 0x7ffb701f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:43.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb67400640 1 -- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb700c8200 msgr2=0x7ffb700c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:43.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb67400640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb700c8200 0x7ffb700c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb67400640 1 -- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb70152bd0 msgr2=0x7ffb701f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:43.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb67400640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb70152bd0 0x7ffb701f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb67400640 1 -- 172.21.15.113:0/642598307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb701f78e0 con 0x7ffb701527f0 2024-03-04T07:41:43.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb67400640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb701527f0 0x7ffb701f3e60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800d5f0 tx=0x7ffb5800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:43.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb6cc00640 1 -- 172.21.15.113:0/642598307 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58018020 con 0x7ffb701527f0 2024-03-04T07:41:43.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb6cc00640 1 -- 172.21.15.113:0/642598307 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb58014620 con 0x7ffb701527f0 2024-03-04T07:41:43.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb6cc00640 1 -- 172.21.15.113:0/642598307 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58014a10 con 0x7ffb701527f0 2024-03-04T07:41:43.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb76c00640 1 -- 172.21.15.113:0/642598307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb70158030 con 0x7ffb701527f0 2024-03-04T07:41:43.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb76c00640 1 -- 172.21.15.113:0/642598307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb701f3940 con 0x7ffb701527f0 2024-03-04T07:41:43.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.756+0000 7ffb76c00640 1 -- 172.21.15.113:0/642598307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb70075c00 con 0x7ffb701527f0 2024-03-04T07:41:43.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.760+0000 7ffb6cc00640 1 -- 172.21.15.113:0/642598307 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb5801b020 con 0x7ffb701527f0 2024-03-04T07:41:43.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.760+0000 7ffb6cc00640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb4c066f60 0x7ffb4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:43.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.760+0000 7ffb6cc00640 1 -- 172.21.15.113:0/642598307 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7ffb58085500 con 0x7ffb701527f0 2024-03-04T07:41:43.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.760+0000 7ffb6fe00640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb4c066f60 0x7ffb4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:43.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.760+0000 7ffb6fe00640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb4c066f60 0x7ffb4c069420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb60000f90 tx=0x7ffb600a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:43.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.764+0000 7ffb6cc00640 1 -- 172.21.15.113:0/642598307 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ffb58022020 con 0x7ffb701527f0 2024-03-04T07:41:43.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 -- 172.21.15.113:0/642598307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb700c79d0 con 0x7ffb701527f0 2024-03-04T07:41:43.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb6cc00640 1 -- 172.21.15.113:0/642598307 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5812 (secure 0 0 0) 0x7ffb58056380 con 0x7ffb701527f0 2024-03-04T07:41:43.954 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:43.954 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:41:40.631226+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5094,5244]},"id":1}]} 2024-03-04T07:41:43.955 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 39 2024-03-04T07:41:43.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 -- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb4c066f60 msgr2=0x7ffb4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:43.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb4c066f60 0x7ffb4c069420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb60000f90 tx=0x7ffb600a7070 comp rx=0 tx=0).stop 2024-03-04T07:41:43.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 -- 172.21.15.113:0/642598307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb701527f0 msgr2=0x7ffb701f3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb701527f0 0x7ffb701f3e60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800d5f0 tx=0x7ffb5800d9f0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 -- 172.21.15.113:0/642598307 shutdown_connections 2024-03-04T07:41:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb70152bd0 0x7ffb701f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb4c066f60 0x7ffb4c069420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb701527f0 0x7ffb701f3e60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 --2- 172.21.15.113:0/642598307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb700c8200 0x7ffb700c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 -- 172.21.15.113:0/642598307 >> 172.21.15.113:0/642598307 conn(0x7ffb7006e820 msgr2=0x7ffb700c9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 -- 172.21.15.113:0/642598307 shutdown_connections 2024-03-04T07:41:43.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:43.956+0000 7ffb76c00640 1 -- 172.21.15.113:0/642598307 wait complete. 2024-03-04T07:41:43.975 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:41:45.976 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:46.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7e00640 1 Processor -- start 2024-03-04T07:41:46.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7e00640 1 -- start start 2024-03-04T07:41:46.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f01e80c8a90 0x7f01e815e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:46.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e815ea60 0x7f01e80ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:46.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e8070fd0 con 0x7f01e8157df0 2024-03-04T07:41:46.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e815e090 con 0x7f01e80c8a90 2024-03-04T07:41:46.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e806dfd0 con 0x7f01e815ea60 2024-03-04T07:41:46.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e815ea60 0x7f01e80ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:46.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e6000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f01e80c8a90 0x7f01e815e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:46.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e815ea60 0x7f01e80ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:41360/0 (socket says 172.21.15.113:41360) 2024-03-04T07:41:46.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7400640 1 -- 172.21.15.113:0/4095310053 learned_addr learned my addr 172.21.15.113:0/4095310053 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:46.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e6000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f01e80c8a90 0x7f01e815e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:37022/0 (socket says 172.21.15.113:37022) 2024-03-04T07:41:46.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e6000640 1 -- 172.21.15.113:0/4095310053 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e815ea60 msgr2=0x7f01e80ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e6000640 1 --2- 172.21.15.113:0/4095310053 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e815ea60 0x7f01e80ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e6000640 1 -- 172.21.15.113:0/4095310053 >> v1:172.21.15.136:6789/0 conn(0x7f01e8157df0 legacy=0x7f01e806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e6000640 1 -- 172.21.15.113:0/4095310053 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e806fcd0 con 0x7f01e80c8a90 2024-03-04T07:41:46.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.120+0000 7f01e7400640 1 --2- 172.21.15.113:0/4095310053 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e815ea60 0x7f01e80ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:41:46.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e6000640 1 --2- 172.21.15.113:0/4095310053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e80c8a90 0x7f01e815e520 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f01d40097c0 tx=0x7f01d401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1d184cd7d8945ac server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e5600640 1 -- 172.21.15.113:0/4095310053 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40b0070 con 0x7f01e80c8a90 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e5600640 1 -- 172.21.15.113:0/4095310053 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d40ab600 con 0x7f01e80c8a90 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e5600640 1 -- 172.21.15.113:0/4095310053 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d40ab9d0 con 0x7f01e80c8a90 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e7e00640 1 -- 172.21.15.113:0/4095310053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e80c8a90 msgr2=0x7f01e815e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e7e00640 1 --2- 172.21.15.113:0/4095310053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e80c8a90 0x7f01e815e520 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f01d40097c0 tx=0x7f01d401c4a0 comp rx=0 tx=0).stop 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e7e00640 1 -- 172.21.15.113:0/4095310053 shutdown_connections 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e7e00640 1 --2- 172.21.15.113:0/4095310053 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e815ea60 0x7f01e80ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e7e00640 1 --2- 172.21.15.113:0/4095310053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e80c8a90 0x7f01e815e520 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e7e00640 1 -- 172.21.15.113:0/4095310053 >> 172.21.15.113:0/4095310053 conn(0x7f01e8072f40 msgr2=0x7f01e80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e7e00640 1 -- 172.21.15.113:0/4095310053 shutdown_connections 2024-03-04T07:41:46.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.124+0000 7f01e7e00640 1 -- 172.21.15.113:0/4095310053 wait complete. 2024-03-04T07:41:46.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7e00640 1 Processor -- start 2024-03-04T07:41:46.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7e00640 1 -- start start 2024-03-04T07:41:46.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e80c8a90 0x7f01e814c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:46.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e8157df0 0x7f01e81563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:46.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e815ea60 0x7f01e8156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:46.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e806dd80 con 0x7f01e815ea60 2024-03-04T07:41:46.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e806dc00 con 0x7f01e80c8a90 2024-03-04T07:41:46.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e81581d0 con 0x7f01e8157df0 2024-03-04T07:41:46.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e815ea60 0x7f01e8156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:46.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e815ea60 0x7f01e8156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58758/0 (socket says 172.21.15.113:58758) 2024-03-04T07:41:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7400640 1 -- 172.21.15.113:0/2186910882 learned_addr learned my addr 172.21.15.113:0/2186910882 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e6a00640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e80c8a90 0x7f01e814c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e6000640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e8157df0 0x7f01e81563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7400640 1 -- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e8157df0 msgr2=0x7f01e81563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7400640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e8157df0 0x7f01e81563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7400640 1 -- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e80c8a90 msgr2=0x7f01e814c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7400640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e80c8a90 0x7f01e814c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7400640 1 -- 172.21.15.113:0/2186910882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e815d870 con 0x7f01e815ea60 2024-03-04T07:41:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e6000640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e8157df0 0x7f01e81563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:46.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7400640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e815ea60 0x7f01e8156920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f01e000efc0 tx=0x7f01e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:46.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01dea00640 1 -- 172.21.15.113:0/2186910882 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e000cbe0 con 0x7f01e815ea60 2024-03-04T07:41:46.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01dea00640 1 -- 172.21.15.113:0/2186910882 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e000cd80 con 0x7f01e815ea60 2024-03-04T07:41:46.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7e00640 1 -- 172.21.15.113:0/2186910882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e81f38d0 con 0x7f01e815ea60 2024-03-04T07:41:46.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01dea00640 1 -- 172.21.15.113:0/2186910882 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0003be0 con 0x7f01e815ea60 2024-03-04T07:41:46.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.128+0000 7f01e7e00640 1 -- 172.21.15.113:0/2186910882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e81f3e70 con 0x7f01e815ea60 2024-03-04T07:41:46.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.132+0000 7f01e7e00640 1 -- 172.21.15.113:0/2186910882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e81f4810 con 0x7f01e815ea60 2024-03-04T07:41:46.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.132+0000 7f01dea00640 1 -- 172.21.15.113:0/2186910882 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01e001b030 con 0x7f01e815ea60 2024-03-04T07:41:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.132+0000 7f01dea00640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c8066f60 0x7f01c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.132+0000 7f01dea00640 1 -- 172.21.15.113:0/2186910882 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f01e00854f0 con 0x7f01e815ea60 2024-03-04T07:41:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.132+0000 7f01e6a00640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c8066f60 0x7f01c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:46.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.132+0000 7f01e6a00640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c8066f60 0x7f01c8069420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f01d8001810 tx=0x7f01d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:46.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.140+0000 7f01dea00640 1 -- 172.21.15.113:0/2186910882 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f01e81f4810 con 0x7f01e815ea60 2024-03-04T07:41:46.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.352+0000 7f01e7e00640 1 -- 172.21.15.113:0/2186910882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01e81561e0 con 0x7f01e815ea60 2024-03-04T07:41:46.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.352+0000 7f01dea00640 1 -- 172.21.15.113:0/2186910882 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5814 (secure 0 0 0) 0x7f01e0022020 con 0x7f01e815ea60 2024-03-04T07:41:46.351 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:46.351 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:41:45.646826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:resolve","state_seq":15,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094]},"id":1}]} 2024-03-04T07:41:46.351 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 40 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.352+0000 7f01e7e00640 1 -- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c8066f60 msgr2=0x7f01c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.352+0000 7f01e7e00640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c8066f60 0x7f01c8069420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f01d8001810 tx=0x7f01d8009040 comp rx=0 tx=0).stop 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.352+0000 7f01e7e00640 1 -- 172.21.15.113:0/2186910882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e815ea60 msgr2=0x7f01e8156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.352+0000 7f01e7e00640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e815ea60 0x7f01e8156920 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f01e000efc0 tx=0x7f01e000c490 comp rx=0 tx=0).stop 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.356+0000 7f01e7e00640 1 -- 172.21.15.113:0/2186910882 shutdown_connections 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.356+0000 7f01e7e00640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c8066f60 0x7f01c8069420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.356+0000 7f01e7e00640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e815ea60 0x7f01e8156920 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.356+0000 7f01e7e00640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e8157df0 0x7f01e81563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.356+0000 7f01e7e00640 1 --2- 172.21.15.113:0/2186910882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e80c8a90 0x7f01e814c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.356+0000 7f01e7e00640 1 -- 172.21.15.113:0/2186910882 >> 172.21.15.113:0/2186910882 conn(0x7f01e8072f40 msgr2=0x7f01e81508d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.356+0000 7f01e7e00640 1 -- 172.21.15.113:0/2186910882 shutdown_connections 2024-03-04T07:41:46.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:46.356+0000 7f01e7e00640 1 -- 172.21.15.113:0/2186910882 wait complete. 2024-03-04T07:41:46.373 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:41:48.374 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:48.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.516+0000 7f0f4e000640 1 Processor -- start 2024-03-04T07:41:48.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.516+0000 7f0f4e000640 1 -- start start 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f4e000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f481527f0 0x7f0f48070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f4e000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f48152bd0 0x7f0f481587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f4e000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f481591f0 con 0x7f0f480c8200 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f4e000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f48071df0 con 0x7f0f48152bd0 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f4e000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f481597b0 con 0x7f0f481527f0 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f47e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f48152bd0 0x7f0f481587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f47e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f48152bd0 0x7f0f481587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46532/0 (socket says 172.21.15.113:46532) 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f3ea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0f480c8200 0x7f0f48072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43576/0 (socket says 172.21.15.113:43576) 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f47e00640 1 -- 172.21.15.113:0/1758788870 learned_addr learned my addr 172.21.15.113:0/1758788870 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f47400640 1 --2- 172.21.15.113:0/1758788870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f481527f0 0x7f0f48070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f47e00640 1 -- 172.21.15.113:0/1758788870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f481527f0 msgr2=0x7f0f48070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f47e00640 1 --2- 172.21.15.113:0/1758788870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f481527f0 0x7f0f48070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:48.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f47e00640 1 -- 172.21.15.113:0/1758788870 >> v1:172.21.15.113:6789/0 conn(0x7f0f480c8200 legacy=0x7f0f48072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:48.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.520+0000 7f0f47e00640 1 -- 172.21.15.113:0/1758788870 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f481594d0 con 0x7f0f48152bd0 2024-03-04T07:41:48.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f47e00640 1 --2- 172.21.15.113:0/1758788870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f48152bd0 0x7f0f481587d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009e50 tx=0x7f0f4001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a06839d3b74ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:48.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f46a00640 1 -- 172.21.15.113:0/1758788870 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400a57f0 con 0x7f0f48152bd0 2024-03-04T07:41:48.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f46a00640 1 -- 172.21.15.113:0/1758788870 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f400a5990 con 0x7f0f48152bd0 2024-03-04T07:41:48.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f46a00640 1 -- 172.21.15.113:0/1758788870 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400a5d60 con 0x7f0f48152bd0 2024-03-04T07:41:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 -- 172.21.15.113:0/1758788870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f48152bd0 msgr2=0x7f0f481587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 --2- 172.21.15.113:0/1758788870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f48152bd0 0x7f0f481587d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009e50 tx=0x7f0f4001ca30 comp rx=0 tx=0).stop 2024-03-04T07:41:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 -- 172.21.15.113:0/1758788870 shutdown_connections 2024-03-04T07:41:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 --2- 172.21.15.113:0/1758788870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f48152bd0 0x7f0f481587d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 --2- 172.21.15.113:0/1758788870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f481527f0 0x7f0f48070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 -- 172.21.15.113:0/1758788870 >> 172.21.15.113:0/1758788870 conn(0x7f0f4806e820 msgr2=0x7f0f4806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:48.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 -- 172.21.15.113:0/1758788870 shutdown_connections 2024-03-04T07:41:48.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 -- 172.21.15.113:0/1758788870 wait complete. 2024-03-04T07:41:48.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 Processor -- start 2024-03-04T07:41:48.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 -- start start 2024-03-04T07:41:48.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f480c8200 0x7f0f4814dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:48.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f481527f0 0x7f0f481f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:48.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f48152bd0 0x7f0f481f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:48.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f481654f0 con 0x7f0f480c8200 2024-03-04T07:41:48.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48165370 con 0x7f0f481527f0 2024-03-04T07:41:48.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f4e000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48165670 con 0x7f0f48152bd0 2024-03-04T07:41:48.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f47400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f480c8200 0x7f0f4814dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:48.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f47400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f480c8200 0x7f0f4814dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46538/0 (socket says 172.21.15.113:46538) 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f47400640 1 -- 172.21.15.113:0/1122215777 learned_addr learned my addr 172.21.15.113:0/1122215777 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.524+0000 7f0f3ea00640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f481527f0 0x7f0f481f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f47e00640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f48152bd0 0x7f0f481f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f47400640 1 -- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f48152bd0 msgr2=0x7f0f481f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f47400640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f48152bd0 0x7f0f481f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f47400640 1 -- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f481527f0 msgr2=0x7f0f481f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f47400640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f481527f0 0x7f0f481f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f47400640 1 -- 172.21.15.113:0/1122215777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f48153330 con 0x7f0f480c8200 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f3ea00640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f481527f0 0x7f0f481f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f47e00640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f48152bd0 0x7f0f481f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:48.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f47400640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f480c8200 0x7f0f4814dfe0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800d830 tx=0x7f0f3800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:48.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f3fe00640 1 -- 172.21.15.113:0/1122215777 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800b6c0 con 0x7f0f480c8200 2024-03-04T07:41:48.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f3fe00640 1 -- 172.21.15.113:0/1122215777 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f38019070 con 0x7f0f480c8200 2024-03-04T07:41:48.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f3fe00640 1 -- 172.21.15.113:0/1122215777 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800ba60 con 0x7f0f480c8200 2024-03-04T07:41:48.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f4e000640 1 -- 172.21.15.113:0/1122215777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4814b7e0 con 0x7f0f480c8200 2024-03-04T07:41:48.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f4e000640 1 -- 172.21.15.113:0/1122215777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f48153610 con 0x7f0f480c8200 2024-03-04T07:41:48.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.528+0000 7f0f4e000640 1 -- 172.21.15.113:0/1122215777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f48075c00 con 0x7f0f480c8200 2024-03-04T07:41:48.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.532+0000 7f0f3fe00640 1 -- 172.21.15.113:0/1122215777 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f3801e460 con 0x7f0f480c8200 2024-03-04T07:41:48.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.532+0000 7f0f3fe00640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f24066f60 0x7f0f24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:48.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.532+0000 7f0f3fe00640 1 -- 172.21.15.113:0/1122215777 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f0f380862d0 con 0x7f0f480c8200 2024-03-04T07:41:48.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.532+0000 7f0f3ea00640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f24066f60 0x7f0f24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:48.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.532+0000 7f0f3ea00640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f24066f60 0x7f0f24069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0f4815ce60 tx=0x7f0f30007d70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:48.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.532+0000 7f0f3fe00640 1 -- 172.21.15.113:0/1122215777 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0f38057230 con 0x7f0f480c8200 2024-03-04T07:41:48.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.732+0000 7f0f4e000640 1 -- 172.21.15.113:0/1122215777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f480c79d0 con 0x7f0f480c8200 2024-03-04T07:41:48.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.732+0000 7f0f3fe00640 1 -- 172.21.15.113:0/1122215777 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5814 (secure 0 0 0) 0x7f0f38056b30 con 0x7f0f480c8200 2024-03-04T07:41:48.732 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:48.732 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:41:45.646826+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:resolve","state_seq":15,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094]},"id":1}]} 2024-03-04T07:41:48.732 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 40 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 -- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f24066f60 msgr2=0x7f0f24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f24066f60 0x7f0f24069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0f4815ce60 tx=0x7f0f30007d70 comp rx=0 tx=0).stop 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 -- 172.21.15.113:0/1122215777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f480c8200 msgr2=0x7f0f4814dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f480c8200 0x7f0f4814dfe0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800d830 tx=0x7f0f3800dcf0 comp rx=0 tx=0).stop 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 -- 172.21.15.113:0/1122215777 shutdown_connections 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f48152bd0 0x7f0f481f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f481527f0 0x7f0f481f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f24066f60 0x7f0f24069420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 --2- 172.21.15.113:0/1122215777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f480c8200 0x7f0f4814dfe0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 -- 172.21.15.113:0/1122215777 >> 172.21.15.113:0/1122215777 conn(0x7f0f4806e820 msgr2=0x7f0f48158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 -- 172.21.15.113:0/1122215777 shutdown_connections 2024-03-04T07:41:48.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:48.736+0000 7f0f4e000640 1 -- 172.21.15.113:0/1122215777 wait complete. 2024-03-04T07:41:48.754 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:41:50.755 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.911+0000 7fa8b7e00640 1 Processor -- start 2024-03-04T07:41:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.911+0000 7fa8b7e00640 1 -- start start 2024-03-04T07:41:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa8b8152ad0 0x7fa8b80c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b8066fb0 0x7fa8b80cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8b80c65b0 con 0x7fa8b81526f0 2024-03-04T07:41:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b80c9530 con 0x7fa8b8152ad0 2024-03-04T07:41:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b80c8c80 con 0x7fa8b8066fb0 2024-03-04T07:41:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b6000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa8b8152ad0 0x7fa8b80c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b8066fb0 0x7fa8b80cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b6a00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fa8b81526f0 0x7fa8b80c8e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:60582/0 (socket says 172.21.15.113:60582) 2024-03-04T07:41:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b6000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa8b8152ad0 0x7fa8b80c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:43902/0 (socket says 172.21.15.113:43902) 2024-03-04T07:41:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b6a00640 1 -- 172.21.15.113:0/565981997 learned_addr learned my addr 172.21.15.113:0/565981997 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:41:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b8066fb0 0x7fa8b80cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:40628/0 (socket says 172.21.15.113:40628) 2024-03-04T07:41:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b6000640 1 -- 172.21.15.113:0/565981997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b8066fb0 msgr2=0x7fa8b80cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b6000640 1 --2- 172.21.15.113:0/565981997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b8066fb0 0x7fa8b80cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b6000640 1 -- 172.21.15.113:0/565981997 >> v1:172.21.15.136:6789/0 conn(0x7fa8b81526f0 legacy=0x7fa8b80c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b6000640 1 -- 172.21.15.113:0/565981997 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b80c6890 con 0x7fa8b8152ad0 2024-03-04T07:41:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b7400640 1 --2- 172.21.15.113:0/565981997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b8066fb0 0x7fa8b80cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:41:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.915+0000 7fa8b5600640 1 -- 172.21.15.113:0/565981997 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411055197 0 0) 0x7fa8b80c65b0 con 0x7fa8b81526f0 2024-03-04T07:41:50.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b6000640 1 --2- 172.21.15.113:0/565981997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8b8152ad0 0x7fa8b80c8740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40097c0 tx=0x7fa8a401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cd465982566b00 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:50.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b5600640 1 -- 172.21.15.113:0/565981997 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40b5070 con 0x7fa8b8152ad0 2024-03-04T07:41:50.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b5600640 1 -- 172.21.15.113:0/565981997 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a40a8990 con 0x7fa8b8152ad0 2024-03-04T07:41:50.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b5600640 1 -- 172.21.15.113:0/565981997 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a40a8d60 con 0x7fa8b8152ad0 2024-03-04T07:41:50.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b7e00640 1 -- 172.21.15.113:0/565981997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8b8152ad0 msgr2=0x7fa8b80c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:50.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b7e00640 1 --2- 172.21.15.113:0/565981997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8b8152ad0 0x7fa8b80c8740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40097c0 tx=0x7fa8a401ec90 comp rx=0 tx=0).stop 2024-03-04T07:41:50.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b7e00640 1 -- 172.21.15.113:0/565981997 shutdown_connections 2024-03-04T07:41:50.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b7e00640 1 --2- 172.21.15.113:0/565981997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b8066fb0 0x7fa8b80cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:50.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b7e00640 1 --2- 172.21.15.113:0/565981997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8b8152ad0 0x7fa8b80c8740 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:50.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b7e00640 1 -- 172.21.15.113:0/565981997 >> 172.21.15.113:0/565981997 conn(0x7fa8b806b060 msgr2=0x7fa8b806b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:50.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b7e00640 1 -- 172.21.15.113:0/565981997 shutdown_connections 2024-03-04T07:41:50.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b7e00640 1 -- 172.21.15.113:0/565981997 wait complete. 2024-03-04T07:41:50.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b7e00640 1 Processor -- start 2024-03-04T07:41:50.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.919+0000 7fa8b7e00640 1 -- start start 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8b8066fb0 0x7fa8b806ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b81526f0 0x7fa8b81f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b8152ad0 0x7fa8b81f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b8158ed0 con 0x7fa8b8152ad0 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b8158bf0 con 0x7fa8b8066fb0 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b81585c0 con 0x7fa8b81526f0 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b8152ad0 0x7fa8b81f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b8152ad0 0x7fa8b81f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46544/0 (socket says 172.21.15.113:46544) 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7400640 1 -- 172.21.15.113:0/1372328544 learned_addr learned my addr 172.21.15.113:0/1372328544 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b6a00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8b8066fb0 0x7fa8b806ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b6000640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b81526f0 0x7fa8b81f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7400640 1 -- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b81526f0 msgr2=0x7fa8b81f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7400640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b81526f0 0x7fa8b81f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7400640 1 -- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8b8066fb0 msgr2=0x7fa8b806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7400640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8b8066fb0 0x7fa8b806ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7400640 1 -- 172.21.15.113:0/1372328544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b81575a0 con 0x7fa8b8152ad0 2024-03-04T07:41:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b6a00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8b8066fb0 0x7fa8b806ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b6000640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b81526f0 0x7fa8b81f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7400640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b8152ad0 0x7fa8b81f3bf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000efc0 tx=0x7fa8b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8aea00640 1 -- 172.21.15.113:0/1372328544 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0018030 con 0x7fa8b8152ad0 2024-03-04T07:41:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8aea00640 1 -- 172.21.15.113:0/1372328544 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b000cdf0 con 0x7fa8b8152ad0 2024-03-04T07:41:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7e00640 1 -- 172.21.15.113:0/1372328544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b81e3e40 con 0x7fa8b8152ad0 2024-03-04T07:41:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8aea00640 1 -- 172.21.15.113:0/1372328544 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0003bc0 con 0x7fa8b8152ad0 2024-03-04T07:41:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7e00640 1 -- 172.21.15.113:0/1372328544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b81e4100 con 0x7fa8b8152ad0 2024-03-04T07:41:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8b7e00640 1 -- 172.21.15.113:0/1372328544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b81e36f0 con 0x7fa8b8152ad0 2024-03-04T07:41:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.923+0000 7fa8aea00640 1 -- 172.21.15.113:0/1372328544 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8b001b070 con 0x7fa8b8152ad0 2024-03-04T07:41:50.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.927+0000 7fa8aea00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa898066f60 0x7fa898069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:50.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.927+0000 7fa8aea00640 1 -- 172.21.15.113:0/1372328544 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fa8b0085d80 con 0x7fa8b8152ad0 2024-03-04T07:41:50.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.927+0000 7fa8b6a00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa898066f60 0x7fa898069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:50.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.927+0000 7fa8b6a00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa898066f60 0x7fa898069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8000f70 tx=0x7fa8a8007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:50.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:50.931+0000 7fa8aea00640 1 -- 172.21.15.113:0/1372328544 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa8b81575a0 con 0x7fa8b8152ad0 2024-03-04T07:41:51.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 -- 172.21.15.113:0/1372328544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8b8075a80 con 0x7fa8b8152ad0 2024-03-04T07:41:51.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8aea00640 1 -- 172.21.15.113:0/1372328544 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5816 (secure 0 0 0) 0x7fa8b0010040 con 0x7fa8b8152ad0 2024-03-04T07:41:51.131 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:51.131 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:41:50.944317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5094,5244]},"id":1}]} 2024-03-04T07:41:51.131 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 41 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 -- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa898066f60 msgr2=0x7fa898069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa898066f60 0x7fa898069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8000f70 tx=0x7fa8a8007cb0 comp rx=0 tx=0).stop 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 -- 172.21.15.113:0/1372328544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b8152ad0 msgr2=0x7fa8b81f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b8152ad0 0x7fa8b81f3bf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000efc0 tx=0x7fa8b000c490 comp rx=0 tx=0).stop 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 -- 172.21.15.113:0/1372328544 shutdown_connections 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa898066f60 0x7fa898069420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b8152ad0 0x7fa8b81f3bf0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8b81526f0 0x7fa8b81f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 --2- 172.21.15.113:0/1372328544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8b8066fb0 0x7fa8b806ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 -- 172.21.15.113:0/1372328544 >> 172.21.15.113:0/1372328544 conn(0x7fa8b806b060 msgr2=0x7fa8b8151920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 -- 172.21.15.113:0/1372328544 shutdown_connections 2024-03-04T07:41:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:51.131+0000 7fa8b7e00640 1 -- 172.21.15.113:0/1372328544 wait complete. 2024-03-04T07:41:51.152 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:41:53.153 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:53.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa593200640 1 Processor -- start 2024-03-04T07:41:53.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa593200640 1 -- start start 2024-03-04T07:41:53.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa593200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa5940c9640 0x7fa5940c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:53.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa593200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5940c2e40 0x7fa59415a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:53.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa593200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59415b720 con 0x7fa594154d00 2024-03-04T07:41:53.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa593200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940c8630 con 0x7fa5940c2e40 2024-03-04T07:41:53.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa593200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5941550e0 con 0x7fa5940c9640 2024-03-04T07:41:53.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa592800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5940c2e40 0x7fa59415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:53.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa592800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5940c2e40 0x7fa59415a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46554/0 (socket says 172.21.15.113:46554) 2024-03-04T07:41:53.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa591400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa5940c9640 0x7fa5940c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:53.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.311+0000 7fa592800640 1 -- 172.21.15.113:0/2558101958 learned_addr learned my addr 172.21.15.113:0/2558101958 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:53.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.315+0000 7fa592800640 1 -- 172.21.15.113:0/2558101958 >> v2:172.21.15.136:3300/0 conn(0x7fa5940c9640 msgr2=0x7fa5940c2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:53.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.315+0000 7fa592800640 1 --2- 172.21.15.113:0/2558101958 >> v2:172.21.15.136:3300/0 conn(0x7fa5940c9640 0x7fa5940c2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.315+0000 7fa592800640 1 -- 172.21.15.113:0/2558101958 >> v1:172.21.15.136:6789/0 conn(0x7fa594154d00 legacy=0x7fa594071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:53.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.315+0000 7fa592800640 1 -- 172.21.15.113:0/2558101958 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5940c8880 con 0x7fa5940c2e40 2024-03-04T07:41:53.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.315+0000 7fa592800640 1 --2- 172.21.15.113:0/2558101958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5940c2e40 0x7fa59415a990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa5940c2850 tx=0x7fa58c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382d59bf7209acc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:53.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.315+0000 7fa590a00640 1 -- 172.21.15.113:0/2558101958 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0b4070 con 0x7fa5940c2e40 2024-03-04T07:41:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.315+0000 7fa590a00640 1 -- 172.21.15.113:0/2558101958 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c0af400 con 0x7fa5940c2e40 2024-03-04T07:41:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.315+0000 7fa590a00640 1 -- 172.21.15.113:0/2558101958 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0af7d0 con 0x7fa5940c2e40 2024-03-04T07:41:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.315+0000 7fa593200640 1 -- 172.21.15.113:0/2558101958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5940c2e40 msgr2=0x7fa59415a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.315+0000 7fa593200640 1 --2- 172.21.15.113:0/2558101958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5940c2e40 0x7fa59415a990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa5940c2850 tx=0x7fa58c01f1e0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 -- 172.21.15.113:0/2558101958 shutdown_connections 2024-03-04T07:41:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 --2- 172.21.15.113:0/2558101958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5940c2e40 0x7fa59415a990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 --2- 172.21.15.113:0/2558101958 >> v2:172.21.15.136:3300/0 conn(0x7fa5940c9640 0x7fa5940c2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 -- 172.21.15.113:0/2558101958 >> 172.21.15.113:0/2558101958 conn(0x7fa5940c9a30 msgr2=0x7fa5940c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:53.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 -- 172.21.15.113:0/2558101958 shutdown_connections 2024-03-04T07:41:53.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 -- 172.21.15.113:0/2558101958 wait complete. 2024-03-04T07:41:53.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 Processor -- start 2024-03-04T07:41:53.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 -- start start 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5940c2e40 0x7fa594072510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa594154d00 0x7fa5941f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5941f1590 0x7fa5940c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c3500 con 0x7fa594154d00 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c8e40 con 0x7fa5940c2e40 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa593200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c3680 con 0x7fa5941f1590 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa591400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa594154d00 0x7fa5941f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa591400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa594154d00 0x7fa5941f0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46562/0 (socket says 172.21.15.113:46562) 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa591400640 1 -- 172.21.15.113:0/3540563651 learned_addr learned my addr 172.21.15.113:0/3540563651 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa591e00640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5940c2e40 0x7fa594072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:53.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa592800640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5941f1590 0x7fa5940c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa591400640 1 -- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5941f1590 msgr2=0x7fa5940c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa591400640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5941f1590 0x7fa5940c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa591400640 1 -- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5940c2e40 msgr2=0x7fa594072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa591400640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5940c2e40 0x7fa594072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa591400640 1 -- 172.21.15.113:0/3540563651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa594072ea0 con 0x7fa594154d00 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.319+0000 7fa591400640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa594154d00 0x7fa5941f0b20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e930 tx=0x7fa58000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.323+0000 7fa58a000640 1 -- 172.21.15.113:0/3540563651 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580017070 con 0x7fa594154d00 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.323+0000 7fa58a000640 1 -- 172.21.15.113:0/3540563651 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58000ca00 con 0x7fa594154d00 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.323+0000 7fa593200640 1 -- 172.21.15.113:0/3540563651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5940aed40 con 0x7fa594154d00 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.323+0000 7fa58a000640 1 -- 172.21.15.113:0/3540563651 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58000cdd0 con 0x7fa594154d00 2024-03-04T07:41:53.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.323+0000 7fa593200640 1 -- 172.21.15.113:0/3540563651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5940c43e0 con 0x7fa594154d00 2024-03-04T07:41:53.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.323+0000 7fa593200640 1 -- 172.21.15.113:0/3540563651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa594075a80 con 0x7fa594154d00 2024-03-04T07:41:53.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.323+0000 7fa58a000640 1 -- 172.21.15.113:0/3540563651 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa58001a070 con 0x7fa594154d00 2024-03-04T07:41:53.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.327+0000 7fa58a000640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa574066f60 0x7fa574069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:53.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.327+0000 7fa58a000640 1 -- 172.21.15.113:0/3540563651 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fa5800852a0 con 0x7fa594154d00 2024-03-04T07:41:53.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.327+0000 7fa591e00640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa574066f60 0x7fa574069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:53.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.327+0000 7fa591e00640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa574066f60 0x7fa574069420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa584007fd0 tx=0x7fa584009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:53.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.331+0000 7fa58a000640 1 -- 172.21.15.113:0/3540563651 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa5800559d0 con 0x7fa594154d00 2024-03-04T07:41:53.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa593200640 1 -- 172.21.15.113:0/3540563651 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5940c8410 con 0x7fa594154d00 2024-03-04T07:41:53.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa58a000640 1 -- 172.21.15.113:0/3540563651 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+5816 (secure 0 0 0) 0x7fa580059a70 con 0x7fa594154d00 2024-03-04T07:41:53.663 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:53.663 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:41:50.944317+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5094,5244]},"id":1}]} 2024-03-04T07:41:53.663 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 41 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa593200640 1 -- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa574066f60 msgr2=0x7fa574069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa593200640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa574066f60 0x7fa574069420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa584007fd0 tx=0x7fa584009040 comp rx=0 tx=0).stop 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa593200640 1 -- 172.21.15.113:0/3540563651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa594154d00 msgr2=0x7fa5941f0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa593200640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa594154d00 0x7fa5941f0b20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa58000e930 tx=0x7fa58000edf0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa593200640 1 -- 172.21.15.113:0/3540563651 shutdown_connections 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa593200640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5941f1590 0x7fa5940c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa593200640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa574066f60 0x7fa574069420 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa593200640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa594154d00 0x7fa5941f0b20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.663+0000 7fa593200640 1 --2- 172.21.15.113:0/3540563651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5940c2e40 0x7fa594072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.667+0000 7fa593200640 1 -- 172.21.15.113:0/3540563651 >> 172.21.15.113:0/3540563651 conn(0x7fa5940c9a30 msgr2=0x7fa594152a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.667+0000 7fa593200640 1 -- 172.21.15.113:0/3540563651 shutdown_connections 2024-03-04T07:41:53.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:53.667+0000 7fa593200640 1 -- 172.21.15.113:0/3540563651 wait complete. 2024-03-04T07:41:53.690 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 41, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 41, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5094": { "addr": "172.21.15.113:6833/605142559", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 605142559, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 605142559, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5094, "incarnation": 26, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 21 }, "gid_5099": { "addr": "172.21.15.136:6835/3944001911", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3944001911, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3944001911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5099, "incarnation": 39, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:reconnect", "state_seq": 17 }, "gid_5244": { "addr": "172.21.15.113:6837/742633123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 742633123, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 742633123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5244, "incarnation": 32, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 43, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:41:50.944317+0000", "qdb_cluster": [ 5094, 5244 ], "qdb_leader": 5244, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5244, "mds_1": 5094, "mds_2": 5099 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:41:55.691 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:55.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.827+0000 7fbf97c00640 1 Processor -- start 2024-03-04T07:41:55.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.827+0000 7fbf97c00640 1 -- start start 2024-03-04T07:41:55.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf97c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf900cd650 0x7fbf900c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:55.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf97c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf90157830 con 0x7fbf900cd270 2024-03-04T07:41:55.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf97c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf90157af0 con 0x7fbf900c7a80 2024-03-04T07:41:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf97c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf90157dd0 con 0x7fbf900cd650 2024-03-04T07:41:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf95200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbf900cd270 0x7fbf90151290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43582/0 (socket says 172.21.15.113:43582) 2024-03-04T07:41:55.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf95200640 1 -- 172.21.15.113:0/2927514442 learned_addr learned my addr 172.21.15.113:0/2927514442 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:41:55.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8fe00640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf900cd650 0x7fbf900c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1841324404 0 0) 0x7fbf90157830 con 0x7fbf900cd270 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf78005740 con 0x7fbf900cd270 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1312136075 0 0) 0x7fbf78005740 con 0x7fbf900cd270 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf900cd650 msgr2=0x7fbf900c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf900cd650 0x7fbf900c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 >> v1:172.21.15.136:6789/0 conn(0x7fbf900c7a80 legacy=0x7fbf9015b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf901580b0 con 0x7fbf900cd270 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fbf84001680 con 0x7fbf900cd270 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 >> v1:172.21.15.113:6789/0 conn(0x7fbf900cd270 legacy=0x7fbf90151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf78005c00 0x7fbf78005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf780065b0 0x7fbf78006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:55.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf900cd650 0x7fbf780072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf90157dd0 con 0x7fbf780065b0 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7800dc10 con 0x7fbf78005c00 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7800de10 con 0x7fbf900cd650 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8fe00640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf780065b0 0x7fbf78006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf95200640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf78005c00 0x7fbf78005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf95c00640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf900cd650 0x7fbf780072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8fe00640 1 -- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf900cd650 msgr2=0x7fbf780072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8fe00640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf900cd650 0x7fbf780072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8fe00640 1 -- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf78005c00 msgr2=0x7fbf78005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8fe00640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf78005c00 0x7fbf78005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:55.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.831+0000 7fbf8fe00640 1 -- 172.21.15.113:0/2927514442 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf78005740 con 0x7fbf780065b0 2024-03-04T07:41:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf8fe00640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf780065b0 0x7fbf78006a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009ab0 tx=0x7fbf7c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8305f4ba6bf63db4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0b2070 con 0x7fbf780065b0 2024-03-04T07:41:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c0a5640 con 0x7fbf780065b0 2024-03-04T07:41:55.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf8f400640 1 -- 172.21.15.113:0/2927514442 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0096b0 con 0x7fbf780065b0 2024-03-04T07:41:55.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf97c00640 1 -- 172.21.15.113:0/2927514442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf780065b0 msgr2=0x7fbf78006a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf97c00640 1 --2- 172.21.15.113:0/2927514442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf780065b0 0x7fbf78006a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009ab0 tx=0x7fbf7c01ca50 comp rx=0 tx=0).stop 2024-03-04T07:41:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf95c00640 1 -- 172.21.15.113:0/2927514442 reap_dead start 2024-03-04T07:41:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf97c00640 1 -- 172.21.15.113:0/2927514442 shutdown_connections 2024-03-04T07:41:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf97c00640 1 -- 172.21.15.113:0/2927514442 >> 172.21.15.113:0/2927514442 conn(0x7fbf900671b0 msgr2=0x7fbf9006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf97c00640 1 -- 172.21.15.113:0/2927514442 shutdown_connections 2024-03-04T07:41:55.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.835+0000 7fbf97c00640 1 -- 172.21.15.113:0/2927514442 wait complete. 2024-03-04T07:41:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf97c00640 1 Processor -- start 2024-03-04T07:41:55.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf97c00640 1 -- start start 2024-03-04T07:41:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf97c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf780065b0 0x7fbf90159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf97c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf90151fa0 0x7fbf901ef580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf97c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf901523e0 0x7fbf901efac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf97c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf90151b20 con 0x7fbf90151fa0 2024-03-04T07:41:55.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf97c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf901519a0 con 0x7fbf780065b0 2024-03-04T07:41:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf97c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9015b720 con 0x7fbf901523e0 2024-03-04T07:41:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf90151fa0 0x7fbf901ef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf90151fa0 0x7fbf901ef580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46578/0 (socket says 172.21.15.113:46578) 2024-03-04T07:41:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8fe00640 1 -- 172.21.15.113:0/4007986977 learned_addr learned my addr 172.21.15.113:0/4007986977 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf95200640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf780065b0 0x7fbf90159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:55.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf95c00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf901523e0 0x7fbf901efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8fe00640 1 -- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf901523e0 msgr2=0x7fbf901efac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8fe00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf901523e0 0x7fbf901efac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8fe00640 1 -- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf780065b0 msgr2=0x7fbf90159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8fe00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf780065b0 0x7fbf90159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8fe00640 1 -- 172.21.15.113:0/4007986977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf901ec070 con 0x7fbf90151fa0 2024-03-04T07:41:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8fe00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf90151fa0 0x7fbf901ef580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009ab0 tx=0x7fbf78007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:55.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8cc00640 1 -- 172.21.15.113:0/4007986977 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0b2040 con 0x7fbf90151fa0 2024-03-04T07:41:55.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8cc00640 1 -- 172.21.15.113:0/4007986977 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c00f000 con 0x7fbf90151fa0 2024-03-04T07:41:55.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf8cc00640 1 -- 172.21.15.113:0/4007986977 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c004220 con 0x7fbf90151fa0 2024-03-04T07:41:55.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf97c00640 1 -- 172.21.15.113:0/4007986977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf901ec350 con 0x7fbf90151fa0 2024-03-04T07:41:55.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.839+0000 7fbf97c00640 1 -- 172.21.15.113:0/4007986977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf901ec910 con 0x7fbf90151fa0 2024-03-04T07:41:55.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.843+0000 7fbf97c00640 1 -- 172.21.15.113:0/4007986977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf90075c00 con 0x7fbf90151fa0 2024-03-04T07:41:55.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.843+0000 7fbf8cc00640 1 -- 172.21.15.113:0/4007986977 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf7c0c0030 con 0x7fbf90151fa0 2024-03-04T07:41:55.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.843+0000 7fbf8cc00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf70066f60 0x7fbf70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:55.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.843+0000 7fbf8cc00640 1 -- 172.21.15.113:0/4007986977 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fbf7c11f130 con 0x7fbf90151fa0 2024-03-04T07:41:55.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.843+0000 7fbf95200640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf70066f60 0x7fbf70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:55.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.843+0000 7fbf95200640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf70066f60 0x7fbf70069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbf84001480 tx=0x7fbf84007900 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:55.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:55.847+0000 7fbf8cc00640 1 -- 172.21.15.113:0/4007986977 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbf7c0c7060 con 0x7fbf90151fa0 2024-03-04T07:41:56.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.115+0000 7fbf97c00640 1 -- 172.21.15.113:0/4007986977 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf9014e8f0 con 0x7fbf90151fa0 2024-03-04T07:41:56.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.115+0000 7fbf8cc00640 1 -- 172.21.15.113:0/4007986977 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5813 (secure 0 0 0) 0x7fbf7c016760 con 0x7fbf90151fa0 2024-03-04T07:41:56.115 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:56.115 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:41:55.959176+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094]},"id":1}]} 2024-03-04T07:41:56.115 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 42 2024-03-04T07:41:56.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 -- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf70066f60 msgr2=0x7fbf70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:56.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf70066f60 0x7fbf70069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbf84001480 tx=0x7fbf84007900 comp rx=0 tx=0).stop 2024-03-04T07:41:56.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 -- 172.21.15.113:0/4007986977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf90151fa0 msgr2=0x7fbf901ef580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:56.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf90151fa0 0x7fbf901ef580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009ab0 tx=0x7fbf78007a50 comp rx=0 tx=0).stop 2024-03-04T07:41:56.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 -- 172.21.15.113:0/4007986977 shutdown_connections 2024-03-04T07:41:56.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf901523e0 0x7fbf901efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:56.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf70066f60 0x7fbf70069420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:56.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf90151fa0 0x7fbf901ef580 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:56.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 --2- 172.21.15.113:0/4007986977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf780065b0 0x7fbf90159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:56.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 -- 172.21.15.113:0/4007986977 >> 172.21.15.113:0/4007986977 conn(0x7fbf900671b0 msgr2=0x7fbf9006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:56.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 -- 172.21.15.113:0/4007986977 shutdown_connections 2024-03-04T07:41:56.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:56.119+0000 7fbf97c00640 1 -- 172.21.15.113:0/4007986977 wait complete. 2024-03-04T07:41:56.136 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:41:56.307 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:41:56.310 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:41:58.137 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:41:58.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.275+0000 7fbfc2000640 1 Processor -- start 2024-03-04T07:41:58.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.275+0000 7fbfc2000640 1 -- start start 2024-03-04T07:41:58.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfc2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0b20b0 0x7fbfbc157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:58.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfc2000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfbc0c1ec0 con 0x7fbfbc157c50 2024-03-04T07:41:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfc2000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfbc0c9b30 con 0x7fbfbc0c85f0 2024-03-04T07:41:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfc2000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0c12a0 con 0x7fbfbc0b20b0 2024-03-04T07:41:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfb3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbfbc157c50 0x7fbfbc06b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58902/0 (socket says 172.21.15.113:58902) 2024-03-04T07:41:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfb3e00640 1 -- 172.21.15.113:0/3722499879 learned_addr learned my addr 172.21.15.113:0/3722499879 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:41:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbb400640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0b20b0 0x7fbfbc157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190635527 0 0) 0x7fbfbc0c1ec0 con 0x7fbfbc157c50 2024-03-04T07:41:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfa0003530 con 0x7fbfbc157c50 2024-03-04T07:41:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807744516 0 0) 0x7fbfbc0c9b30 con 0x7fbfbc0c85f0 2024-03-04T07:41:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfbc0c1ec0 con 0x7fbfbc0c85f0 2024-03-04T07:41:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 974998403 0 0) 0x7fbfa0003530 con 0x7fbfbc157c50 2024-03-04T07:41:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0b20b0 msgr2=0x7fbfbc157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0b20b0 0x7fbfbc157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 >> v1:172.21.15.136:6789/0 conn(0x7fbfbc0c85f0 legacy=0x7fbfbc161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0c1580 con 0x7fbfbc157c50 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fbfa4001600 con 0x7fbfbc157c50 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 >> v1:172.21.15.113:6789/0 conn(0x7fbfbc157c50 legacy=0x7fbfbc06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfa0003b80 0x7fbfa0003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 0x7fbfa0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0b20b0 0x7fbfa00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0c12a0 con 0x7fbfa0004810 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa000dc90 con 0x7fbfa0003b80 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa000de90 con 0x7fbfbc0b20b0 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfb3e00640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 0x7fbfa0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbb400640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfa0003b80 0x7fbfa0003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbbe00640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0b20b0 0x7fbfa00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfb3e00640 1 -- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0b20b0 msgr2=0x7fbfa00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfb3e00640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0b20b0 0x7fbfa00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfb3e00640 1 -- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfa0003b80 msgr2=0x7fbfa0003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:41:58.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfb3e00640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfa0003b80 0x7fbfa0003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:58.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfb3e00640 1 -- 172.21.15.113:0/3722499879 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa0003530 con 0x7fbfa0004810 2024-03-04T07:41:58.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.279+0000 7fbfbbe00640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0b20b0 0x7fbfa00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:41:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfb3e00640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 0x7fbfa0004c90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4007f60 tx=0x7fbfa401cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66a164c84176682 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4007bb0 con 0x7fbfa0004810 2024-03-04T07:41:58.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa40a9070 con 0x7fbfa0004810 2024-03-04T07:41:58.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfbaa00640 1 -- 172.21.15.113:0/3722499879 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4005e00 con 0x7fbfa0004810 2024-03-04T07:41:58.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfc2000640 1 -- 172.21.15.113:0/3722499879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 msgr2=0x7fbfa0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:58.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfc2000640 1 --2- 172.21.15.113:0/3722499879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 0x7fbfa0004c90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4007f60 tx=0x7fbfa401cac0 comp rx=0 tx=0).stop 2024-03-04T07:41:58.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfbbe00640 1 -- 172.21.15.113:0/3722499879 reap_dead start 2024-03-04T07:41:58.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfc2000640 1 -- 172.21.15.113:0/3722499879 shutdown_connections 2024-03-04T07:41:58.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfc2000640 1 -- 172.21.15.113:0/3722499879 >> 172.21.15.113:0/3722499879 conn(0x7fbfbc06e820 msgr2=0x7fbfbc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:58.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfc2000640 1 -- 172.21.15.113:0/3722499879 shutdown_connections 2024-03-04T07:41:58.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.283+0000 7fbfc2000640 1 -- 172.21.15.113:0/3722499879 wait complete. 2024-03-04T07:41:58.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 Processor -- start 2024-03-04T07:41:58.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 -- start start 2024-03-04T07:41:58.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 0x7fbfbc14ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:58.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfbc1ef930 0x7fbfbc0c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:58.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0c3ab0 0x7fbfbc148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:58.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0c2400 con 0x7fbfa0004810 2024-03-04T07:41:58.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc1571f0 con 0x7fbfbc1ef930 2024-03-04T07:41:58.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc156ec0 con 0x7fbfbc0c3ab0 2024-03-04T07:41:58.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfbb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 0x7fbfbc14ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:58.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfbb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 0x7fbfbc14ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50192/0 (socket says 172.21.15.113:50192) 2024-03-04T07:41:58.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfbb400640 1 -- 172.21.15.113:0/877799721 learned_addr learned my addr 172.21.15.113:0/877799721 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:41:58.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfb3e00640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfbc1ef930 0x7fbfbc0c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:58.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfbbe00640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0c3ab0 0x7fbfbc148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:58.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfbb400640 1 -- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0c3ab0 msgr2=0x7fbfbc148b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:58.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfbb400640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0c3ab0 0x7fbfbc148b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:58.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfbb400640 1 -- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfbc1ef930 msgr2=0x7fbfbc0c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:58.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfbb400640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfbc1ef930 0x7fbfbc0c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:58.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfbb400640 1 -- 172.21.15.113:0/877799721 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc14e5e0 con 0x7fbfa0004810 2024-03-04T07:41:58.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfbb400640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 0x7fbfbc14ded0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfac011a60 tx=0x7fbfac011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:58.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfb3400640 1 -- 172.21.15.113:0/877799721 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00b040 con 0x7fbfa0004810 2024-03-04T07:41:58.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfb3400640 1 -- 172.21.15.113:0/877799721 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac00fdf0 con 0x7fbfa0004810 2024-03-04T07:41:58.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfb3400640 1 -- 172.21.15.113:0/877799721 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac01c6c0 con 0x7fbfa0004810 2024-03-04T07:41:58.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 -- 172.21.15.113:0/877799721 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc14e8c0 con 0x7fbfa0004810 2024-03-04T07:41:58.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 -- 172.21.15.113:0/877799721 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc14ee80 con 0x7fbfa0004810 2024-03-04T07:41:58.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.287+0000 7fbfc2000640 1 -- 172.21.15.113:0/877799721 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc075c00 con 0x7fbfa0004810 2024-03-04T07:41:58.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.291+0000 7fbfb3400640 1 -- 172.21.15.113:0/877799721 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfac01c860 con 0x7fbfa0004810 2024-03-04T07:41:58.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.291+0000 7fbfb3400640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf98066f60 0x7fbf98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:41:58.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.291+0000 7fbfb3400640 1 -- 172.21.15.113:0/877799721 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fbfac0860e0 con 0x7fbfa0004810 2024-03-04T07:41:58.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.291+0000 7fbfb3e00640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf98066f60 0x7fbf98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:41:58.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.291+0000 7fbfb3e00640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf98066f60 0x7fbf98069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc158030 tx=0x7fbfa40b1040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:41:58.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.295+0000 7fbfb3400640 1 -- 172.21.15.113:0/877799721 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbfac056da0 con 0x7fbfa0004810 2024-03-04T07:41:58.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 -- 172.21.15.113:0/877799721 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfbc074c70 con 0x7fbfa0004810 2024-03-04T07:41:58.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfb3400640 1 -- 172.21.15.113:0/877799721 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5813 (secure 0 0 0) 0x7fbfac0566a0 con 0x7fbfa0004810 2024-03-04T07:41:58.486 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:41:58.486 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:41:55.959176+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094]},"id":1}]} 2024-03-04T07:41:58.487 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 42 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 -- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf98066f60 msgr2=0x7fbf98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf98066f60 0x7fbf98069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc158030 tx=0x7fbfa40b1040 comp rx=0 tx=0).stop 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 -- 172.21.15.113:0/877799721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 msgr2=0x7fbfbc14ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 0x7fbfbc14ded0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfac011a60 tx=0x7fbfac011f20 comp rx=0 tx=0).stop 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 -- 172.21.15.113:0/877799721 shutdown_connections 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbfbc0c3ab0 0x7fbfbc148b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfbc1ef930 0x7fbfbc0c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf98066f60 0x7fbf98069420 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 --2- 172.21.15.113:0/877799721 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbfa0004810 0x7fbfbc14ded0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 -- 172.21.15.113:0/877799721 >> 172.21.15.113:0/877799721 conn(0x7fbfbc06e820 msgr2=0x7fbfbc072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 -- 172.21.15.113:0/877799721 shutdown_connections 2024-03-04T07:41:58.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:41:58.487+0000 7fbfc2000640 1 -- 172.21.15.113:0/877799721 wait complete. 2024-03-04T07:41:58.511 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:42:00.512 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:42:00.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.651+0000 7fc4e5200640 1 Processor -- start 2024-03-04T07:42:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.651+0000 7fc4e5200640 1 -- start start 2024-03-04T07:42:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4e5200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4e014e4c0 0x7fc4e0072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4e5200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4e00c22e0 0x7fc4e00731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4e5200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e00c27f0 con 0x7fc4e00c2d80 2024-03-04T07:42:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4e5200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e015ad60 con 0x7fc4e014e4c0 2024-03-04T07:42:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4e5200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00c2a90 con 0x7fc4e00c22e0 2024-03-04T07:42:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4dde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4e014e4c0 0x7fc4e0072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4de800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc4e00c2d80 0x7fc4e015b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58916/0 (socket says 172.21.15.113:58916) 2024-03-04T07:42:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4dde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4e014e4c0 0x7fc4e0072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50204/0 (socket says 172.21.15.113:50204) 2024-03-04T07:42:00.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4de800640 1 -- 172.21.15.113:0/2783577 learned_addr learned my addr 172.21.15.113:0/2783577 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:42:00.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4d5400640 1 --2- 172.21.15.113:0/2783577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4e00c22e0 0x7fc4e00731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:00.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4dde00640 1 -- 172.21.15.113:0/2783577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4e00c22e0 msgr2=0x7fc4e00731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:42:00.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4dde00640 1 --2- 172.21.15.113:0/2783577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4e00c22e0 0x7fc4e00731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:00.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4dde00640 1 -- 172.21.15.113:0/2783577 >> v1:172.21.15.113:6789/0 conn(0x7fc4e00c2d80 legacy=0x7fc4e015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:00.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.655+0000 7fc4dde00640 1 -- 172.21.15.113:0/2783577 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e0070c60 con 0x7fc4e014e4c0 2024-03-04T07:42:00.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4dde00640 1 --2- 172.21.15.113:0/2783577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e014e4c0 0x7fc4e0072c80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009d90 tx=0x7fc4d001f170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea80dc05b9ddc3e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:00.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4dd400640 1 -- 172.21.15.113:0/2783577 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00b4070 con 0x7fc4e014e4c0 2024-03-04T07:42:00.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4dd400640 1 -- 172.21.15.113:0/2783577 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00af600 con 0x7fc4e014e4c0 2024-03-04T07:42:00.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4dd400640 1 -- 172.21.15.113:0/2783577 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00af9d0 con 0x7fc4e014e4c0 2024-03-04T07:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4e5200640 1 -- 172.21.15.113:0/2783577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e014e4c0 msgr2=0x7fc4e0072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4e5200640 1 --2- 172.21.15.113:0/2783577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e014e4c0 0x7fc4e0072c80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009d90 tx=0x7fc4d001f170 comp rx=0 tx=0).stop 2024-03-04T07:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4e5200640 1 -- 172.21.15.113:0/2783577 shutdown_connections 2024-03-04T07:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4e5200640 1 --2- 172.21.15.113:0/2783577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4e00c22e0 0x7fc4e00731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4e5200640 1 --2- 172.21.15.113:0/2783577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e014e4c0 0x7fc4e0072c80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4e5200640 1 -- 172.21.15.113:0/2783577 >> 172.21.15.113:0/2783577 conn(0x7fc4e006e820 msgr2=0x7fc4e006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4e5200640 1 -- 172.21.15.113:0/2783577 shutdown_connections 2024-03-04T07:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.659+0000 7fc4e5200640 1 -- 172.21.15.113:0/2783577 wait complete. 2024-03-04T07:42:00.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4e5200640 1 Processor -- start 2024-03-04T07:42:00.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4e5200640 1 -- start start 2024-03-04T07:42:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4e5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e00c22e0 0x7fc4e014b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4e5200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4e00c2d80 0x7fc4e00c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4e5200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4e014e4c0 0x7fc4e00c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4e5200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e0071500 con 0x7fc4e00c22e0 2024-03-04T07:42:00.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4e5200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e0071220 con 0x7fc4e014e4c0 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4e5200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e014e8a0 con 0x7fc4e00c2d80 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4dde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e00c22e0 0x7fc4e014b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4dde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e00c22e0 0x7fc4e014b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50220/0 (socket says 172.21.15.113:50220) 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4dde00640 1 -- 172.21.15.113:0/2233139179 learned_addr learned my addr 172.21.15.113:0/2233139179 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4de800640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4e014e4c0 0x7fc4e00c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4d5400640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4e00c2d80 0x7fc4e00c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4dde00640 1 -- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4e00c2d80 msgr2=0x7fc4e00c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4dde00640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4e00c2d80 0x7fc4e00c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4dde00640 1 -- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4e014e4c0 msgr2=0x7fc4e00c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4dde00640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4e014e4c0 0x7fc4e00c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4dde00640 1 -- 172.21.15.113:0/2233139179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e0143730 con 0x7fc4e00c22e0 2024-03-04T07:42:00.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4dde00640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e00c22e0 0x7fc4e014b290 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc4d001cda0 tx=0x7fc4d001bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:00.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4d6a00640 1 -- 172.21.15.113:0/2233139179 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00b4040 con 0x7fc4e00c22e0 2024-03-04T07:42:00.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4d6a00640 1 -- 172.21.15.113:0/2233139179 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00c1070 con 0x7fc4e00c22e0 2024-03-04T07:42:00.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4d6a00640 1 -- 172.21.15.113:0/2233139179 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0009450 con 0x7fc4e00c22e0 2024-03-04T07:42:00.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4e5200640 1 -- 172.21.15.113:0/2233139179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e014a4a0 con 0x7fc4e00c22e0 2024-03-04T07:42:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.663+0000 7fc4e5200640 1 -- 172.21.15.113:0/2233139179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e00cb3d0 con 0x7fc4e00c22e0 2024-03-04T07:42:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.667+0000 7fc4e5200640 1 -- 172.21.15.113:0/2233139179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e0075c00 con 0x7fc4e00c22e0 2024-03-04T07:42:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.667+0000 7fc4d6a00640 1 -- 172.21.15.113:0/2233139179 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4d00ba690 con 0x7fc4e00c22e0 2024-03-04T07:42:00.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.667+0000 7fc4d6a00640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4bc066f60 0x7fc4bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:00.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.667+0000 7fc4d6a00640 1 -- 172.21.15.113:0/2233139179 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fc4d0123de0 con 0x7fc4e00c22e0 2024-03-04T07:42:00.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.667+0000 7fc4d5400640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4bc066f60 0x7fc4bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:00.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.667+0000 7fc4d5400640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4bc066f60 0x7fc4bc069420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc4e015ad30 tx=0x7fc4c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:00.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.675+0000 7fc4d6a00640 1 -- 172.21.15.113:0/2233139179 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc4d0018cd0 con 0x7fc4e00c22e0 2024-03-04T07:42:00.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.907+0000 7fc4e5200640 1 -- 172.21.15.113:0/2233139179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4e0155190 con 0x7fc4e00c22e0 2024-03-04T07:42:00.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.907+0000 7fc4d6a00640 1 -- 172.21.15.113:0/2233139179 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5813 (secure 0 0 0) 0x7fc4d00f4d60 con 0x7fc4e00c22e0 2024-03-04T07:42:00.908 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:42:00.908 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:41:55.959176+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5094]},"id":1}]} 2024-03-04T07:42:00.908 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 42 2024-03-04T07:42:00.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 -- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4bc066f60 msgr2=0x7fc4bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:00.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4bc066f60 0x7fc4bc069420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc4e015ad30 tx=0x7fc4c8008040 comp rx=0 tx=0).stop 2024-03-04T07:42:00.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 -- 172.21.15.113:0/2233139179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e00c22e0 msgr2=0x7fc4e014b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:00.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e00c22e0 0x7fc4e014b290 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc4d001cda0 tx=0x7fc4d001bf10 comp rx=0 tx=0).stop 2024-03-04T07:42:00.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 -- 172.21.15.113:0/2233139179 shutdown_connections 2024-03-04T07:42:00.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4e014e4c0 0x7fc4e00c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:00.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4e00c2d80 0x7fc4e00c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:00.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4bc066f60 0x7fc4bc069420 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:00.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 --2- 172.21.15.113:0/2233139179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4e00c22e0 0x7fc4e014b290 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:00.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 -- 172.21.15.113:0/2233139179 >> 172.21.15.113:0/2233139179 conn(0x7fc4e006e820 msgr2=0x7fc4e00c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:42:00.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 -- 172.21.15.113:0/2233139179 shutdown_connections 2024-03-04T07:42:00.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:00.911+0000 7fc4e5200640 1 -- 172.21.15.113:0/2233139179 wait complete. 2024-03-04T07:42:00.938 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:42:02.940 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:42:03.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.079+0000 7fde31a00640 1 Processor -- start 2024-03-04T07:42:03.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.079+0000 7fde31a00640 1 -- start start 2024-03-04T07:42:03.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde31a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde2c06ff20 0x7fde2c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:03.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde31a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde2c0c8b90 0x7fde2c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:03.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde31a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde2c152e30 con 0x7fde2c152a50 2024-03-04T07:42:03.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde31a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2c0c31b0 con 0x7fde2c0c8b90 2024-03-04T07:42:03.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde31a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c06f840 con 0x7fde2c06ff20 2024-03-04T07:42:03.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde2b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde2c0c8b90 0x7fde2c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:03.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde2b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde2c0c8b90 0x7fde2c15c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50234/0 (socket says 172.21.15.113:50234) 2024-03-04T07:42:03.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde2b400640 1 -- 172.21.15.113:0/1776709193 learned_addr learned my addr 172.21.15.113:0/1776709193 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:42:03.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde2a000640 1 --2- 172.21.15.113:0/1776709193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde2c06ff20 0x7fde2c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:03.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde2b400640 1 -- 172.21.15.113:0/1776709193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde2c06ff20 msgr2=0x7fde2c15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:42:03.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde2b400640 1 --2- 172.21.15.113:0/1776709193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde2c06ff20 0x7fde2c15bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:03.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde2b400640 1 -- 172.21.15.113:0/1776709193 >> v1:172.21.15.136:6789/0 conn(0x7fde2c152a50 legacy=0x7fde2c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:03.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.083+0000 7fde2b400640 1 -- 172.21.15.113:0/1776709193 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c06f280 con 0x7fde2c0c8b90 2024-03-04T07:42:03.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde2b400640 1 --2- 172.21.15.113:0/1776709193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0c8b90 0x7fde2c15c310 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fde24009e50 tx=0x7fde2401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1892fc7dd1b1d148 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:03.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde29600640 1 -- 172.21.15.113:0/1776709193 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde240a5880 con 0x7fde2c0c8b90 2024-03-04T07:42:03.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde29600640 1 -- 172.21.15.113:0/1776709193 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde240a5a20 con 0x7fde2c0c8b90 2024-03-04T07:42:03.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde29600640 1 -- 172.21.15.113:0/1776709193 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde240a5df0 con 0x7fde2c0c8b90 2024-03-04T07:42:03.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde31a00640 1 -- 172.21.15.113:0/1776709193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0c8b90 msgr2=0x7fde2c15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:03.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde31a00640 1 --2- 172.21.15.113:0/1776709193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0c8b90 0x7fde2c15c310 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fde24009e50 tx=0x7fde2401c9a0 comp rx=0 tx=0).stop 2024-03-04T07:42:03.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde31a00640 1 -- 172.21.15.113:0/1776709193 shutdown_connections 2024-03-04T07:42:03.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde31a00640 1 --2- 172.21.15.113:0/1776709193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c0c8b90 0x7fde2c15c310 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:03.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde31a00640 1 --2- 172.21.15.113:0/1776709193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde2c06ff20 0x7fde2c15bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:03.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde31a00640 1 -- 172.21.15.113:0/1776709193 >> 172.21.15.113:0/1776709193 conn(0x7fde2c0734d0 msgr2=0x7fde2c066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:42:03.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde31a00640 1 -- 172.21.15.113:0/1776709193 shutdown_connections 2024-03-04T07:42:03.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.087+0000 7fde31a00640 1 -- 172.21.15.113:0/1776709193 wait complete. 2024-03-04T07:42:03.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde31a00640 1 Processor -- start 2024-03-04T07:42:03.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde31a00640 1 -- start start 2024-03-04T07:42:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde31a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde2c06ff20 0x7fde2c14ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde31a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde2c0c8b90 0x7fde2c150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde31a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c152a50 0x7fde2c150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde31a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c0ce9b0 con 0x7fde2c152a50 2024-03-04T07:42:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde31a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c0ce830 con 0x7fde2c0c8b90 2024-03-04T07:42:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde31a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c165880 con 0x7fde2c06ff20 2024-03-04T07:42:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c152a50 0x7fde2c150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c152a50 0x7fde2c150fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50242/0 (socket says 172.21.15.113:50242) 2024-03-04T07:42:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2b400640 1 -- 172.21.15.113:0/2006619941 learned_addr learned my addr 172.21.15.113:0/2006619941 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2a000640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde2c0c8b90 0x7fde2c150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2aa00640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde2c06ff20 0x7fde2c14ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2b400640 1 -- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde2c06ff20 msgr2=0x7fde2c14ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2b400640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde2c06ff20 0x7fde2c14ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2b400640 1 -- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde2c0c8b90 msgr2=0x7fde2c150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2b400640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde2c0c8b90 0x7fde2c150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2b400640 1 -- 172.21.15.113:0/2006619941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c1514f0 con 0x7fde2c152a50 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde2b400640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c152a50 0x7fde2c150fb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fde2401a5d0 tx=0x7fde24019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde22a00640 1 -- 172.21.15.113:0/2006619941 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde240bc020 con 0x7fde2c152a50 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde22a00640 1 -- 172.21.15.113:0/2006619941 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde240b8690 con 0x7fde2c152a50 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde31a00640 1 -- 172.21.15.113:0/2006619941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2c1f3e50 con 0x7fde2c152a50 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde22a00640 1 -- 172.21.15.113:0/2006619941 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde240b8a80 con 0x7fde2c152a50 2024-03-04T07:42:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.091+0000 7fde31a00640 1 -- 172.21.15.113:0/2006619941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2c150270 con 0x7fde2c152a50 2024-03-04T07:42:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.095+0000 7fde31a00640 1 -- 172.21.15.113:0/2006619941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2c075c00 con 0x7fde2c152a50 2024-03-04T07:42:03.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.095+0000 7fde22a00640 1 -- 172.21.15.113:0/2006619941 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde240bf020 con 0x7fde2c152a50 2024-03-04T07:42:03.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.095+0000 7fde22a00640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde0c066f90 0x7fde0c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:03.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.095+0000 7fde22a00640 1 -- 172.21.15.113:0/2006619941 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fde2411fda0 con 0x7fde2c152a50 2024-03-04T07:42:03.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.099+0000 7fde2aa00640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde0c066f90 0x7fde0c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:03.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.099+0000 7fde2aa00640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde0c066f90 0x7fde0c069450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde1c001650 tx=0x7fde1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:03.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.103+0000 7fde22a00640 1 -- 172.21.15.113:0/2006619941 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fde240c6070 con 0x7fde2c152a50 2024-03-04T07:42:03.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 -- 172.21.15.113:0/2006619941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde2c0b2bd0 con 0x7fde2c152a50 2024-03-04T07:42:03.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde22a00640 1 -- 172.21.15.113:0/2006619941 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5818 (secure 0 0 0) 0x7fde24016620 con 0x7fde2c152a50 2024-03-04T07:42:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:42:03.347 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:42:01.001453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5094,5099,5244]},"id":1}]} 2024-03-04T07:42:03.347 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 43 2024-03-04T07:42:03.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 -- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde0c066f90 msgr2=0x7fde0c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:03.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde0c066f90 0x7fde0c069450 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde1c001650 tx=0x7fde1c009040 comp rx=0 tx=0).stop 2024-03-04T07:42:03.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 -- 172.21.15.113:0/2006619941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c152a50 msgr2=0x7fde2c150fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:03.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c152a50 0x7fde2c150fb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fde2401a5d0 tx=0x7fde24019740 comp rx=0 tx=0).stop 2024-03-04T07:42:03.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 -- 172.21.15.113:0/2006619941 shutdown_connections 2024-03-04T07:42:03.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde0c066f90 0x7fde0c069450 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:03.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde2c152a50 0x7fde2c150fb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:03.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde2c0c8b90 0x7fde2c150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:03.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 --2- 172.21.15.113:0/2006619941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde2c06ff20 0x7fde2c14ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:03.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.347+0000 7fde31a00640 1 -- 172.21.15.113:0/2006619941 >> 172.21.15.113:0/2006619941 conn(0x7fde2c0734d0 msgr2=0x7fde2c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:42:03.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.351+0000 7fde31a00640 1 -- 172.21.15.113:0/2006619941 shutdown_connections 2024-03-04T07:42:03.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:03.351+0000 7fde31a00640 1 -- 172.21.15.113:0/2006619941 wait complete. 2024-03-04T07:42:03.368 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=2, replacing gid=4938 2024-03-04T07:42:03.368 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.a 2024-03-04T07:42:04.369 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-03-04T07:42:04.369 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-04T07:42:04.370 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-04T07:42:04.373 INFO:tasks.ceph.mds.a:Started 2024-03-04T07:42:04.373 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:42:04.461 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-03-04T07:42:04.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f1566400640 1 Processor -- start 2024-03-04T07:42:04.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f1566400640 1 -- start start 2024-03-04T07:42:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f1566400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1560152a50 0x7f15600ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f1566400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f156015c4e0 0x7f15600ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f1566400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1560152e30 con 0x7f156006ff20 2024-03-04T07:42:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f1566400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15600c31b0 con 0x7f156015c4e0 2024-03-04T07:42:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f1566400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156006f840 con 0x7f1560152a50 2024-03-04T07:42:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f155fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f156015c4e0 0x7f15600ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f155f400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1560152a50 0x7f15600ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f155fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f156015c4e0 0x7f15600ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50274/0 (socket says 172.21.15.113:50274) 2024-03-04T07:42:04.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.571+0000 7f155fe00640 1 -- 172.21.15.113:0/1060078780 learned_addr learned my addr 172.21.15.113:0/1060078780 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:42:04.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.575+0000 7f155fe00640 1 -- 172.21.15.113:0/1060078780 >> v2:172.21.15.136:3300/0 conn(0x7f1560152a50 msgr2=0x7f15600ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:42:04.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.575+0000 7f155fe00640 1 --2- 172.21.15.113:0/1060078780 >> v2:172.21.15.136:3300/0 conn(0x7f1560152a50 0x7f15600ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:04.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.575+0000 7f155fe00640 1 -- 172.21.15.113:0/1060078780 >> v1:172.21.15.136:6789/0 conn(0x7f156006ff20 legacy=0x7f156015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:04.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.575+0000 7f155fe00640 1 -- 172.21.15.113:0/1060078780 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156006f280 con 0x7f156015c4e0 2024-03-04T07:42:04.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f155fe00640 1 --2- 172.21.15.113:0/1060078780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 0x7f15600ce660 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1560067cd0 tx=0x7f155801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1b355d6c2fa761 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:04.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f155e000640 1 -- 172.21.15.113:0/1060078780 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580a58b0 con 0x7f156015c4e0 2024-03-04T07:42:04.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f155e000640 1 -- 172.21.15.113:0/1060078780 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15580a5a50 con 0x7f156015c4e0 2024-03-04T07:42:04.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f155e000640 1 -- 172.21.15.113:0/1060078780 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580a5e20 con 0x7f156015c4e0 2024-03-04T07:42:04.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f1566400640 1 -- 172.21.15.113:0/1060078780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 msgr2=0x7f15600ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:04.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f1566400640 1 --2- 172.21.15.113:0/1060078780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 0x7f15600ce660 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1560067cd0 tx=0x7f155801c9f0 comp rx=0 tx=0).stop 2024-03-04T07:42:04.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f1566400640 1 -- 172.21.15.113:0/1060078780 shutdown_connections 2024-03-04T07:42:04.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f1566400640 1 --2- 172.21.15.113:0/1060078780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 0x7f15600ce660 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:04.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f1566400640 1 --2- 172.21.15.113:0/1060078780 >> v2:172.21.15.136:3300/0 conn(0x7f1560152a50 0x7f15600ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:04.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f1566400640 1 -- 172.21.15.113:0/1060078780 >> 172.21.15.113:0/1060078780 conn(0x7f15600734d0 msgr2=0x7f1560066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:42:04.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f1566400640 1 -- 172.21.15.113:0/1060078780 shutdown_connections 2024-03-04T07:42:04.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f1566400640 1 -- 172.21.15.113:0/1060078780 wait complete. 2024-03-04T07:42:04.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f1566400640 1 Processor -- start 2024-03-04T07:42:04.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.579+0000 7f1566400640 1 -- start start 2024-03-04T07:42:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1566400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156006ff20 0x7f15601f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1566400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1560152a50 0x7f15601f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1566400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 0x7f1560143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1566400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560165a00 con 0x7f156015c4e0 2024-03-04T07:42:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1566400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560165880 con 0x7f1560152a50 2024-03-04T07:42:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1566400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1560165b80 con 0x7f156006ff20 2024-03-04T07:42:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 0x7f1560143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 0x7f1560143680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50278/0 (socket says 172.21.15.113:50278) 2024-03-04T07:42:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155fe00640 1 -- 172.21.15.113:0/4282787263 learned_addr learned my addr 172.21.15.113:0/4282787263 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:42:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155f400640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156006ff20 0x7f15601f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155ea00640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1560152a50 0x7f15601f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155fe00640 1 -- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156006ff20 msgr2=0x7f15601f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155fe00640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156006ff20 0x7f15601f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155fe00640 1 -- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1560152a50 msgr2=0x7f15601f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155fe00640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1560152a50 0x7f15601f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155fe00640 1 -- 172.21.15.113:0/4282787263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15600cb090 con 0x7f156015c4e0 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f155fe00640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 0x7f1560143680 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f15580af040 tx=0x7f1558019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1557400640 1 -- 172.21.15.113:0/4282787263 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580a58b0 con 0x7f156015c4e0 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1557400640 1 -- 172.21.15.113:0/4282787263 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15580be070 con 0x7f156015c4e0 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1566400640 1 -- 172.21.15.113:0/4282787263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15601599d0 con 0x7f156015c4e0 2024-03-04T07:42:04.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1557400640 1 -- 172.21.15.113:0/4282787263 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558009510 con 0x7f156015c4e0 2024-03-04T07:42:04.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1566400640 1 -- 172.21.15.113:0/4282787263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1560159f90 con 0x7f156015c4e0 2024-03-04T07:42:04.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.583+0000 7f1566400640 1 -- 172.21.15.113:0/4282787263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1560075c00 con 0x7f156015c4e0 2024-03-04T07:42:04.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.587+0000 7f1557400640 1 -- 172.21.15.113:0/4282787263 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15580b8470 con 0x7f156015c4e0 2024-03-04T07:42:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.587+0000 7f1557400640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1540066f60 0x7f1540069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:04.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.587+0000 7f1557400640 1 -- 172.21.15.113:0/4282787263 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f15581202b0 con 0x7f156015c4e0 2024-03-04T07:42:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.587+0000 7f155f400640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1540066f60 0x7f1540069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:04.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.587+0000 7f155f400640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1540066f60 0x7f1540069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1550009b60 tx=0x7f1550009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.591+0000 7f1557400640 1 -- 172.21.15.113:0/4282787263 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f15580c8020 con 0x7f156015c4e0 2024-03-04T07:42:04.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.795+0000 7f1566400640 1 -- 172.21.15.113:0/4282787263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15600c9660 con 0x7f156015c4e0 2024-03-04T07:42:04.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.795+0000 7f1557400640 1 -- 172.21.15.113:0/4282787263 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f15580165e0 con 0x7f156015c4e0 2024-03-04T07:42:04.795 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:42:04.795 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:42:01.001453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5094,5099,5244]},"id":1}]} 2024-03-04T07:42:04.795 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 43 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 -- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1540066f60 msgr2=0x7f1540069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1540066f60 0x7f1540069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1550009b60 tx=0x7f1550009190 comp rx=0 tx=0).stop 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 -- 172.21.15.113:0/4282787263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 msgr2=0x7f1560143680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 0x7f1560143680 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f15580af040 tx=0x7f1558019790 comp rx=0 tx=0).stop 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 -- 172.21.15.113:0/4282787263 shutdown_connections 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1540066f60 0x7f1540069420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f156015c4e0 0x7f1560143680 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1560152a50 0x7f15601f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 --2- 172.21.15.113:0/4282787263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f156006ff20 0x7f15601f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 -- 172.21.15.113:0/4282787263 >> 172.21.15.113:0/4282787263 conn(0x7f15600734d0 msgr2=0x7f15600ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:42:04.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 -- 172.21.15.113:0/4282787263 shutdown_connections 2024-03-04T07:42:04.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:04.799+0000 7f1566400640 1 -- 172.21.15.113:0/4282787263 wait complete. 2024-03-04T07:42:04.817 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/34' 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/38' 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/28' 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/24' 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/29' 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/32' 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/22' 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/25' 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/30' 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/33' 2024-03-04T07:42:06.359 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/23' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/35' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/26' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './206/37' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './206' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/21' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/27' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/31' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/36' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/20' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/39' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/34' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/38' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/28' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/24' 2024-03-04T07:42:06.360 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/29' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/32' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/22' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/25' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/30' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/33' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/23' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/35' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/26' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './207/37' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './207' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/21' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/27' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/31' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/36' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/20' 2024-03-04T07:42:06.361 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/39' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/34' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/38' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/28' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/24' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/29' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/32' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/22' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/25' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/30' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/33' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/23' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/35' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/26' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed './208/37' 2024-03-04T07:42:06.362 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './208' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/21' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/27' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/31' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/36' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/20' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/39' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/34' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/38' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/28' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/24' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/29' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/32' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/22' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/25' 2024-03-04T07:42:06.363 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/30' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/33' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/23' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/35' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/26' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './209/37' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './209' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/21' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/27' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/31' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/36' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/20' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/39' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/34' 2024-03-04T07:42:06.364 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/38' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/28' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/24' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/29' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/32' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/22' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/25' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/30' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/33' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/23' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/35' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/26' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './210/37' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './210' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/21' 2024-03-04T07:42:06.365 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/27' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/31' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/36' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/20' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/39' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/34' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/38' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/28' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/24' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/29' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/32' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/22' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/25' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/30' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/33' 2024-03-04T07:42:06.366 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/23' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/35' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/26' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './211/37' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './211' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/21' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/27' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/31' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/36' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/20' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/39' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/34' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/38' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/28' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/24' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/29' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/32' 2024-03-04T07:42:06.367 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/22' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/25' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/30' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/33' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/23' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/35' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/26' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './212/37' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './212' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/21' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/27' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/31' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/36' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/20' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/39' 2024-03-04T07:42:06.368 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/34' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/38' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/28' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/24' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/29' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/32' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/22' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/25' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/30' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/33' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/23' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/35' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/26' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './213/37' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './213' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/21' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/27' 2024-03-04T07:42:06.369 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/31' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/36' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/20' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/39' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/34' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/38' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/28' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/24' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/29' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/32' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/22' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/25' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/30' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/33' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/23' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/35' 2024-03-04T07:42:06.370 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/26' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './214/37' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './214' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/21' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/27' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/31' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/36' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/20' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/39' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/34' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/38' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/28' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/24' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/29' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/32' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/22' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/25' 2024-03-04T07:42:06.371 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/30' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/33' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/23' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/35' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/26' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './215/37' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './215' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/21' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/27' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/31' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/36' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/20' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/39' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/34' 2024-03-04T07:42:06.372 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/38' 2024-03-04T07:42:06.818 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:42:06.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c7e00640 1 Processor -- start 2024-03-04T07:42:06.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c7e00640 1 -- start start 2024-03-04T07:42:06.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0152d60 0x7f85c00ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:06.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f85c0070120 0x7f85c00ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f85c00c9db0 0x7f85c0158930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c00c5380 con 0x7f85c0070120 2024-03-04T07:42:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c00c3440 con 0x7f85c00c9db0 2024-03-04T07:42:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c006f620 con 0x7f85c0152d60 2024-03-04T07:42:06.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0152d60 0x7f85c00ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c4a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f85c0070120 0x7f85c00ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0152d60 0x7f85c00ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:56592/0 (socket says 172.21.15.113:56592) 2024-03-04T07:42:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c5400640 1 -- 172.21.15.113:0/2210147742 learned_addr learned my addr 172.21.15.113:0/2210147742 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:42:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.979+0000 7f85c5e00640 1 --2- 172.21.15.113:0/2210147742 >> v2:172.21.15.136:3300/0 conn(0x7f85c00c9db0 0x7f85c0158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.983+0000 7f85c4a00640 1 -- 172.21.15.113:0/2210147742 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0152d60 msgr2=0x7f85c00ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:42:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.983+0000 7f85c4a00640 1 --2- 172.21.15.113:0/2210147742 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0152d60 0x7f85c00ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.983+0000 7f85c4a00640 1 -- 172.21.15.113:0/2210147742 >> v2:172.21.15.136:3300/0 conn(0x7f85c00c9db0 msgr2=0x7f85c0158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:42:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.983+0000 7f85c4a00640 1 --2- 172.21.15.113:0/2210147742 >> v2:172.21.15.136:3300/0 conn(0x7f85c00c9db0 0x7f85c0158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:06.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.983+0000 7f85c4a00640 1 -- 172.21.15.113:0/2210147742 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c00c36e0 con 0x7f85c0070120 2024-03-04T07:42:06.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85c4a00640 1 --2- 172.21.15.113:0/2210147742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0070120 0x7f85c00ce850 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f85c00706d0 tx=0x7f85b4022f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe9bb98548fa5f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:06.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85bfe00640 1 -- 172.21.15.113:0/2210147742 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400d2d0 con 0x7f85c0070120 2024-03-04T07:42:06.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85bfe00640 1 -- 172.21.15.113:0/2210147742 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b40acb30 con 0x7f85c0070120 2024-03-04T07:42:06.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85bfe00640 1 -- 172.21.15.113:0/2210147742 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b400b450 con 0x7f85c0070120 2024-03-04T07:42:06.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85c7e00640 1 -- 172.21.15.113:0/2210147742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0070120 msgr2=0x7f85c00ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:06.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85c7e00640 1 --2- 172.21.15.113:0/2210147742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0070120 0x7f85c00ce850 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f85c00706d0 tx=0x7f85b4022f80 comp rx=0 tx=0).stop 2024-03-04T07:42:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85c7e00640 1 -- 172.21.15.113:0/2210147742 shutdown_connections 2024-03-04T07:42:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85c7e00640 1 --2- 172.21.15.113:0/2210147742 >> v2:172.21.15.136:3300/0 conn(0x7f85c00c9db0 0x7f85c0158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85c7e00640 1 --2- 172.21.15.113:0/2210147742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0070120 0x7f85c00ce850 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85c7e00640 1 --2- 172.21.15.113:0/2210147742 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0152d60 0x7f85c00ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:06.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.987+0000 7f85c7e00640 1 -- 172.21.15.113:0/2210147742 >> 172.21.15.113:0/2210147742 conn(0x7f85c0066db0 msgr2=0x7f85c00671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:42:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.991+0000 7f85c7e00640 1 -- 172.21.15.113:0/2210147742 shutdown_connections 2024-03-04T07:42:06.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.991+0000 7f85c7e00640 1 -- 172.21.15.113:0/2210147742 wait complete. 2024-03-04T07:42:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.991+0000 7f85c7e00640 1 Processor -- start 2024-03-04T07:42:06.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.991+0000 7f85c7e00640 1 -- start start 2024-03-04T07:42:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.991+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0070120 0x7f85c00c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:06.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85c00c9db0 0x7f85c01ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0152d60 0x7f85c0071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00c39c0 con 0x7f85c0152d60 2024-03-04T07:42:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00c5020 con 0x7f85c00c9db0 2024-03-04T07:42:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c00c3b40 con 0x7f85c0070120 2024-03-04T07:42:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0070120 0x7f85c00c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0152d60 0x7f85c0071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85c00c9db0 0x7f85c01ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0070120 0x7f85c00c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:56600/0 (socket says 172.21.15.113:56600) 2024-03-04T07:42:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5400640 1 -- 172.21.15.113:0/2536875109 learned_addr learned my addr 172.21.15.113:0/2536875109 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:42:06.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0152d60 0x7f85c0071090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50288/0 (socket says 172.21.15.113:50288) 2024-03-04T07:42:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5e00640 1 -- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0070120 msgr2=0x7f85c00c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5e00640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0070120 0x7f85c00c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5e00640 1 -- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85c00c9db0 msgr2=0x7f85c01ecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5e00640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85c00c9db0 0x7f85c01ecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:06.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5e00640 1 -- 172.21.15.113:0/2536875109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c01ec4e0 con 0x7f85c0152d60 2024-03-04T07:42:06.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c5e00640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0152d60 0x7f85c0071090 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f85b800eed0 tx=0x7f85b800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:06.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85bd600640 1 -- 172.21.15.113:0/2536875109 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8018020 con 0x7f85c0152d60 2024-03-04T07:42:06.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85bd600640 1 -- 172.21.15.113:0/2536875109 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b800bd30 con 0x7f85c0152d60 2024-03-04T07:42:06.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c7e00640 1 -- 172.21.15.113:0/2536875109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c00b34c0 con 0x7f85c0152d60 2024-03-04T07:42:06.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85c7e00640 1 -- 172.21.15.113:0/2536875109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c01ec7a0 con 0x7f85c0152d60 2024-03-04T07:42:06.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.995+0000 7f85bd600640 1 -- 172.21.15.113:0/2536875109 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8014640 con 0x7f85c0152d60 2024-03-04T07:42:06.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.999+0000 7f85c7e00640 1 -- 172.21.15.113:0/2536875109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859c0052a0 con 0x7f85c0152d60 2024-03-04T07:42:06.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.999+0000 7f85bd600640 1 -- 172.21.15.113:0/2536875109 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85b801b020 con 0x7f85c0152d60 2024-03-04T07:42:07.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.999+0000 7f85bd600640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f85a4066f60 0x7f85a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:42:07.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:06.999+0000 7f85bd600640 1 -- 172.21.15.113:0/2536875109 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f85b80859c0 con 0x7f85c0152d60 2024-03-04T07:42:07.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.003+0000 7f85c5400640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f85a4066f60 0x7f85a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:42:07.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.003+0000 7f85c5400640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f85a4066f60 0x7f85a4069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85b0006aa0 tx=0x7f85b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:42:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.007+0000 7f85bd600640 1 -- 172.21.15.113:0/2536875109 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f85b8022030 con 0x7f85c0152d60 2024-03-04T07:42:07.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.239+0000 7f85c7e00640 1 -- 172.21.15.113:0/2536875109 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f859c005590 con 0x7f85c0152d60 2024-03-04T07:42:07.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.239+0000 7f85bd600640 1 -- 172.21.15.113:0/2536875109 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f85b80560c0 con 0x7f85c0152d60 2024-03-04T07:42:07.239 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:42:07.239 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:42:01.001453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099,5094]},"id":1}]} 2024-03-04T07:42:07.239 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.239+0000 7f85c7e00640 1 -- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f85a4066f60 msgr2=0x7f85a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.239+0000 7f85c7e00640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f85a4066f60 0x7f85a4069420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85b0006aa0 tx=0x7f85b0008040 comp rx=0 tx=0).stop 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.239+0000 7f85c7e00640 1 -- 172.21.15.113:0/2536875109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0152d60 msgr2=0x7f85c0071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.239+0000 7f85c7e00640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0152d60 0x7f85c0071090 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f85b800eed0 tx=0x7f85b800b480 comp rx=0 tx=0).stop 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.243+0000 7f85c7e00640 1 -- 172.21.15.113:0/2536875109 shutdown_connections 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.243+0000 7f85c7e00640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f85a4066f60 0x7f85a4069420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.243+0000 7f85c7e00640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f85c0152d60 0x7f85c0071090 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.243+0000 7f85c7e00640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85c00c9db0 0x7f85c01ecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.243+0000 7f85c7e00640 1 --2- 172.21.15.113:0/2536875109 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85c0070120 0x7f85c00c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.243+0000 7f85c7e00640 1 -- 172.21.15.113:0/2536875109 >> 172.21.15.113:0/2536875109 conn(0x7f85c0066db0 msgr2=0x7f85c006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.243+0000 7f85c7e00640 1 -- 172.21.15.113:0/2536875109 shutdown_connections 2024-03-04T07:42:07.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:42:07.243+0000 7f85c7e00640 1 -- 172.21.15.113:0/2536875109 wait complete. 2024-03-04T07:42:07.259 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-03-04T07:42:07.260 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 117 secs before thrashing 2024-03-04T07:42:09.976 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/28' 2024-03-04T07:42:09.976 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/24' 2024-03-04T07:42:09.976 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/29' 2024-03-04T07:42:09.976 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/32' 2024-03-04T07:42:09.976 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/22' 2024-03-04T07:42:09.976 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/25' 2024-03-04T07:42:09.976 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/30' 2024-03-04T07:42:09.976 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/33' 2024-03-04T07:42:09.976 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/23' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/35' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/26' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './216/37' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './216' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/21' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/27' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/31' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/36' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/20' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/39' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/34' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/38' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/28' 2024-03-04T07:42:09.977 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/24' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/29' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/32' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/22' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/25' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/30' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/33' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/23' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/35' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/26' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './217/37' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './217' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/21' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/27' 2024-03-04T07:42:09.978 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/31' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/36' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/20' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/39' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/34' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/38' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/28' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/24' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/29' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/32' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/22' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/25' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/30' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/33' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/23' 2024-03-04T07:42:09.979 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/35' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/26' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './218/37' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './218' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/21' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/27' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/31' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/36' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/20' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/39' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/34' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/38' 2024-03-04T07:42:09.980 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/28' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/24' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/29' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/32' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/22' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/25' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/30' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/33' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/23' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/35' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/26' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './219/37' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './219' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/21' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/27' 2024-03-04T07:42:09.981 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/31' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/36' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/20' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/39' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/34' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/38' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/28' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/24' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/29' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/32' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/22' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/25' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/30' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/33' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/23' 2024-03-04T07:42:09.982 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/35' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/26' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './220/37' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './220' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/21' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/27' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/31' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/36' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/20' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/39' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/34' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/38' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/28' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/24' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/29' 2024-03-04T07:42:09.983 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/32' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/22' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/25' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/30' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/33' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/23' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/35' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/26' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './221/37' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './221' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/21' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/27' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/31' 2024-03-04T07:42:09.984 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/36' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/20' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/39' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/34' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/38' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/28' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/24' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/29' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/32' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/22' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/25' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/30' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/33' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/23' 2024-03-04T07:42:09.985 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/35' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/26' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './222/37' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './222' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/21' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/27' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/31' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/36' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/20' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/39' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/34' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/38' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/28' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/24' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/29' 2024-03-04T07:42:09.986 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/32' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/22' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/25' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/30' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/33' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/23' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/35' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/26' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './223/37' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './223' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/21' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/27' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/31' 2024-03-04T07:42:09.987 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/36' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/20' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/39' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/34' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/38' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/28' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/24' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/29' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/32' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/22' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/25' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/30' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/33' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/23' 2024-03-04T07:42:09.988 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/35' 2024-03-04T07:42:09.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/26' 2024-03-04T07:42:09.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './224/37' 2024-03-04T07:42:09.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './224' 2024-03-04T07:42:09.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/21' 2024-03-04T07:42:09.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/27' 2024-03-04T07:42:09.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/31' 2024-03-04T07:42:09.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/36' 2024-03-04T07:42:09.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/20' 2024-03-04T07:42:09.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/39' 2024-03-04T07:42:09.989 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/34' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/38' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/28' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/24' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/29' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/32' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/22' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/25' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/30' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/33' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/23' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/35' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/26' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './225/37' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './225' 2024-03-04T07:42:09.990 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/21' 2024-03-04T07:42:09.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/27' 2024-03-04T07:42:09.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/31' 2024-03-04T07:42:09.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/36' 2024-03-04T07:42:09.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/20' 2024-03-04T07:42:09.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/39' 2024-03-04T07:42:09.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/34' 2024-03-04T07:42:09.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/38' 2024-03-04T07:42:09.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/28' 2024-03-04T07:42:09.991 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/24' 2024-03-04T07:42:11.893 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/29' 2024-03-04T07:42:11.893 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/32' 2024-03-04T07:42:11.893 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/22' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/25' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/30' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/33' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/23' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/35' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/26' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './226/37' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './226' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/21' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/27' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/31' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/36' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/20' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/39' 2024-03-04T07:42:11.894 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/34' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/38' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/28' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/24' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/29' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/32' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/22' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/25' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/30' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/33' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/23' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/35' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/26' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed './227/37' 2024-03-04T07:42:11.895 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './227' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/21' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/27' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/31' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/36' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/20' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/39' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/34' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/38' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/28' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/24' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/29' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/32' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/22' 2024-03-04T07:42:11.896 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/25' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/30' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/33' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/23' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/35' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/26' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './228/37' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './228' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/21' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/27' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/31' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/36' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/20' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/39' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/34' 2024-03-04T07:42:11.897 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/38' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/28' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/24' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/29' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/32' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/22' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/25' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/30' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/33' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/23' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/35' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/26' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './229/37' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './229' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/21' 2024-03-04T07:42:11.898 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/27' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/31' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/36' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/20' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/39' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/34' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/38' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/28' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/24' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/29' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/32' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/22' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/25' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/30' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/33' 2024-03-04T07:42:11.899 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/23' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/35' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/26' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './230/37' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './230' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/21' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/27' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/31' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/36' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/20' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/39' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/34' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/38' 2024-03-04T07:42:11.900 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/28' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/24' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/29' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/32' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/22' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/25' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/30' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/33' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/23' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/35' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/26' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './231/37' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './231' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/21' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/27' 2024-03-04T07:42:11.901 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/31' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/36' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/20' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/39' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/34' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/38' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/28' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/24' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/29' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/32' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/22' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/25' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/30' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/33' 2024-03-04T07:42:11.902 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/23' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/35' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/26' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './232/37' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './232' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/21' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/27' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/31' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/36' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/20' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/39' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/34' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/38' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/28' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/24' 2024-03-04T07:42:11.903 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/29' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/32' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/22' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/25' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/30' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/33' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/23' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/35' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/26' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './233/37' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './233' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/21' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/27' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/31' 2024-03-04T07:42:11.904 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/36' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/20' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/39' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/34' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/38' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/28' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/24' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/29' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/32' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/22' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/25' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/30' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/33' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/23' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/35' 2024-03-04T07:42:11.905 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/26' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './234/37' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './234' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/21' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/27' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/31' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/36' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/20' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/39' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/34' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/38' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/28' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/24' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/29' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/32' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/22' 2024-03-04T07:42:11.906 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/25' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/30' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/33' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/23' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/35' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/26' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './235/37' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './235' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/21' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/27' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/31' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/36' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/20' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/39' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/34' 2024-03-04T07:42:11.907 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/38' 2024-03-04T07:42:11.908 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/28' 2024-03-04T07:42:11.908 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/24' 2024-03-04T07:42:11.908 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/29' 2024-03-04T07:42:11.908 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/32' 2024-03-04T07:42:16.727 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/22' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/25' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/30' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/33' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/23' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/35' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/26' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './236/37' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './236' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/21' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/27' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/31' 2024-03-04T07:42:16.728 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/36' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/20' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/39' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/34' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/38' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/28' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/24' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/29' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/32' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/22' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/25' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/30' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/33' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/23' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/35' 2024-03-04T07:42:16.729 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/26' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './237/37' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './237' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/21' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/27' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/31' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/36' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/20' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/39' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/34' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/38' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/28' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/24' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/29' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/32' 2024-03-04T07:42:16.730 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/22' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/25' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/30' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/33' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/23' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/35' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/26' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './238/37' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './238' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/21' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/27' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/31' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/36' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/20' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/39' 2024-03-04T07:42:16.731 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/34' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/38' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/28' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/24' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/29' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/32' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/22' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/25' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/30' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/33' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/23' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/35' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/26' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './239/37' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './239' 2024-03-04T07:42:16.732 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/21' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/27' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/31' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/36' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/20' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/39' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/34' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/38' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/28' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/24' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/29' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/32' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/22' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/25' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/30' 2024-03-04T07:42:16.733 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/33' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/23' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/35' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/26' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './240/37' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './240' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/21' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/27' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/31' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/36' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/20' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/39' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/34' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/38' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/28' 2024-03-04T07:42:16.734 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/24' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/29' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/32' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/22' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/25' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/30' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/33' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/23' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/35' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/26' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './241/37' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './241' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/21' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/27' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/31' 2024-03-04T07:42:16.735 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/36' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/20' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/39' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/34' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/38' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/28' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/24' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/29' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/32' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/22' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/25' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/30' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/33' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/23' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/35' 2024-03-04T07:42:16.736 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/26' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './242/37' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './242' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/21' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/27' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/31' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/36' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/20' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/39' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/34' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/38' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/28' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/24' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/29' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/32' 2024-03-04T07:42:16.737 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/22' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/25' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/30' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/33' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/23' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/35' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/26' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './243/37' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './243' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/21' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/27' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/31' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/36' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/20' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/39' 2024-03-04T07:42:16.738 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/34' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/38' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/28' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/24' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/29' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/32' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/22' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/25' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/30' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/33' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/23' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/35' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/26' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './244/37' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './244' 2024-03-04T07:42:16.739 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/21' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/27' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/31' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/36' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/20' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/39' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/34' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/38' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/28' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/24' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/29' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/32' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/22' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/25' 2024-03-04T07:42:16.740 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/30' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/33' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/23' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/35' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/26' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './245/37' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './245' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/21' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/27' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/31' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/36' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/20' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/39' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/34' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/38' 2024-03-04T07:42:16.741 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/28' 2024-03-04T07:42:16.742 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/24' 2024-03-04T07:42:16.742 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/29' 2024-03-04T07:42:16.742 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/32' 2024-03-04T07:42:16.742 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/22' 2024-03-04T07:42:19.035 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/25' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/30' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/33' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/23' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/35' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/26' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './246/37' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './246' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/21' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/27' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/31' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/36' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/20' 2024-03-04T07:42:19.036 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/39' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/34' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/38' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/28' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/24' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/29' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/32' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/22' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/25' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/30' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/33' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/23' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/35' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/26' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed './247/37' 2024-03-04T07:42:19.037 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './247' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/21' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/27' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/31' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/36' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/20' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/39' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/34' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/38' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/28' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/24' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/29' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/32' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/22' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/25' 2024-03-04T07:42:19.038 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/30' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/33' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/23' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/35' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/26' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './248/37' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './248' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/21' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/27' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/31' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/36' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/20' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/39' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/34' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/38' 2024-03-04T07:42:19.039 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/28' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/24' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/29' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/32' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/22' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/25' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/30' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/33' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/23' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/35' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/26' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './249/37' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './249' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/21' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/27' 2024-03-04T07:42:19.040 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/31' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/36' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/20' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/39' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/34' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/38' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/28' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/24' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/29' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/32' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/22' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/25' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/30' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/33' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/23' 2024-03-04T07:42:19.041 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/35' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/26' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './250/37' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './250' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/21' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/27' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/31' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/36' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/20' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/39' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/34' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/38' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/28' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/24' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/29' 2024-03-04T07:42:19.042 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/32' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/22' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/25' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/30' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/33' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/23' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/35' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/26' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './251/37' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './251' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/21' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/27' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/31' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/36' 2024-03-04T07:42:19.043 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/20' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/39' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/34' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/38' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/28' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/24' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/29' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/32' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/22' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/25' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/30' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/33' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/23' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/35' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/26' 2024-03-04T07:42:19.044 INFO:tasks.workunit.client.0.smithi113.stdout:removed './252/37' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './252' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/21' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/27' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/31' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/36' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/20' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/39' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/34' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/38' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/28' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/24' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/29' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/32' 2024-03-04T07:42:19.045 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/22' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/25' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/30' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/33' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/23' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/35' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/26' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './253/37' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './253' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/21' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/27' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/31' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/36' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/20' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/39' 2024-03-04T07:42:19.046 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/34' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/38' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/28' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/24' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/29' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/32' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/22' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/25' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/30' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/33' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/23' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/35' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/26' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './254/37' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './254' 2024-03-04T07:42:19.047 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/21' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/27' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/31' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/36' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/20' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/39' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/34' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/38' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/28' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/24' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/29' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/32' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/22' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/25' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/30' 2024-03-04T07:42:19.048 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/33' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/23' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/35' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/26' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './255/37' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './255' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/21' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/27' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/31' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/36' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/20' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/39' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/34' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/38' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/28' 2024-03-04T07:42:19.049 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/24' 2024-03-04T07:42:19.050 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/29' 2024-03-04T07:42:19.050 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/32' 2024-03-04T07:42:19.050 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/22' 2024-03-04T07:42:19.050 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/25' 2024-03-04T07:42:19.050 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/30' 2024-03-04T07:42:26.411 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:42:26.416 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:42:26.594 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/33' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/23' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/35' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/26' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './256/37' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './256' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/21' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/27' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/31' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/36' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/20' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/39' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/34' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/38' 2024-03-04T07:42:26.595 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/28' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/24' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/29' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/32' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/22' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/25' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/30' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/33' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/23' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/35' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/26' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './257/37' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './257' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/21' 2024-03-04T07:42:26.596 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/27' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/31' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/36' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/20' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/39' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/34' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/38' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/28' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/24' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/29' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/32' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/22' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/25' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/30' 2024-03-04T07:42:26.597 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/33' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/23' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/35' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/26' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './258/37' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './258' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/21' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/27' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/31' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/36' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/20' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/39' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/34' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/38' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/28' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/24' 2024-03-04T07:42:26.598 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/29' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/32' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/22' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/25' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/30' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/33' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/23' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/35' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/26' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './259/37' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './259' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/21' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/27' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/31' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/36' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/20' 2024-03-04T07:42:26.599 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/39' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/34' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/38' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/28' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/24' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/29' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/32' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/22' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/25' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/30' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/33' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/23' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/35' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/26' 2024-03-04T07:42:26.600 INFO:tasks.workunit.client.0.smithi113.stdout:removed './260/37' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './260' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/21' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/27' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/31' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/36' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/20' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/39' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/34' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/38' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/28' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/24' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/29' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/32' 2024-03-04T07:42:26.601 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/22' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/25' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/30' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/33' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/23' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/35' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/26' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './261/37' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './261' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/21' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/27' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/31' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/36' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/20' 2024-03-04T07:42:26.602 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/39' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/34' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/38' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/28' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/24' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/29' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/32' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/22' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/25' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/30' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/33' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/23' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/35' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/26' 2024-03-04T07:42:26.603 INFO:tasks.workunit.client.0.smithi113.stdout:removed './262/37' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './262' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/21' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/27' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/31' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/36' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/20' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/39' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/34' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/38' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/28' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/24' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/29' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/32' 2024-03-04T07:42:26.604 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/22' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/25' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/30' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/33' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/23' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/35' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/26' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './263/37' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './263' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/21' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/27' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/31' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/36' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/20' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/39' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/34' 2024-03-04T07:42:26.605 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/38' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/28' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/24' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/29' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/32' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/22' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/25' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/30' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/33' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/23' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/35' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/26' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './264/37' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './264' 2024-03-04T07:42:26.606 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/21' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/27' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/31' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/36' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/20' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/39' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/34' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/38' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/28' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/24' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/29' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/32' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/22' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/25' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/30' 2024-03-04T07:42:26.607 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/33' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/23' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/35' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/26' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './265/37' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './265' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/21' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/27' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/31' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/36' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/20' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/39' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/34' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/38' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/28' 2024-03-04T07:42:26.608 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/24' 2024-03-04T07:42:26.609 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/29' 2024-03-04T07:42:26.609 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/32' 2024-03-04T07:42:26.609 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/22' 2024-03-04T07:42:26.609 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/25' 2024-03-04T07:42:26.609 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/30' 2024-03-04T07:42:26.609 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/33' 2024-03-04T07:42:26.609 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/23' 2024-03-04T07:42:29.505 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/35' 2024-03-04T07:42:29.505 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/26' 2024-03-04T07:42:29.505 INFO:tasks.workunit.client.0.smithi113.stdout:removed './266/37' 2024-03-04T07:42:29.505 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './266' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/21' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/27' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/31' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/36' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/20' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/39' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/34' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/38' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/28' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/24' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/29' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/32' 2024-03-04T07:42:29.506 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/22' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/25' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/30' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/33' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/23' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/35' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/26' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './267/37' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './267' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/21' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/27' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/31' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/36' 2024-03-04T07:42:29.507 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/20' 2024-03-04T07:42:29.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/39' 2024-03-04T07:42:29.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/34' 2024-03-04T07:42:29.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/38' 2024-03-04T07:42:29.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/28' 2024-03-04T07:42:29.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/24' 2024-03-04T07:42:29.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/29' 2024-03-04T07:42:29.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/32' 2024-03-04T07:42:29.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/22' 2024-03-04T07:42:29.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/25' 2024-03-04T07:42:29.508 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/30' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/33' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/23' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/35' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/26' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './268/37' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './268' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/21' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/27' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/31' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/36' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/20' 2024-03-04T07:42:29.509 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/39' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/34' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/38' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/28' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/24' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/29' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/32' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/22' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/25' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/30' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/33' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/23' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/35' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/26' 2024-03-04T07:42:29.510 INFO:tasks.workunit.client.0.smithi113.stdout:removed './269/37' 2024-03-04T07:42:29.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './269' 2024-03-04T07:42:29.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/21' 2024-03-04T07:42:29.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/27' 2024-03-04T07:42:29.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/31' 2024-03-04T07:42:29.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/36' 2024-03-04T07:42:29.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/20' 2024-03-04T07:42:29.511 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/39' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/34' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/38' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/28' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/24' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/29' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/32' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/22' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/25' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/30' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/33' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/23' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/35' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/26' 2024-03-04T07:42:29.512 INFO:tasks.workunit.client.0.smithi113.stdout:removed './270/37' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './270' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/21' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/27' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/31' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/36' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/20' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/39' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/34' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/38' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/28' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/24' 2024-03-04T07:42:29.513 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/29' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/32' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/22' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/25' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/30' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/33' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/23' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/35' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/26' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './271/37' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './271' 2024-03-04T07:42:29.514 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/21' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/27' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/31' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/36' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/20' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/39' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/34' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/38' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/28' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/24' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/29' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/32' 2024-03-04T07:42:29.515 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/22' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/25' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/30' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/33' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/23' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/35' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/26' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './272/37' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './272' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/21' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/27' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/31' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/36' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/20' 2024-03-04T07:42:29.516 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/39' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/34' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/38' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/28' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/24' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/29' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/32' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/22' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/25' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/30' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/33' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/23' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/35' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/26' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed './273/37' 2024-03-04T07:42:29.517 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './273' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/21' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/27' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/31' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/36' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/20' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/39' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/34' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/38' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/28' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/24' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/29' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/32' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/22' 2024-03-04T07:42:29.518 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/25' 2024-03-04T07:42:29.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/30' 2024-03-04T07:42:29.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/33' 2024-03-04T07:42:29.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/23' 2024-03-04T07:42:29.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/35' 2024-03-04T07:42:29.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/26' 2024-03-04T07:42:29.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './274/37' 2024-03-04T07:42:29.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './274' 2024-03-04T07:42:29.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/21' 2024-03-04T07:42:29.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/27' 2024-03-04T07:42:29.519 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/31' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/36' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/20' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/39' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/34' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/38' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/28' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/24' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/29' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/32' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/22' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/25' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/30' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/33' 2024-03-04T07:42:29.520 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/23' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/35' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/26' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './275/37' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './275' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/21' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/27' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/31' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/36' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/20' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/39' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/34' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/38' 2024-03-04T07:42:29.521 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/28' 2024-03-04T07:42:29.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/24' 2024-03-04T07:42:29.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/29' 2024-03-04T07:42:29.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/32' 2024-03-04T07:42:29.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/22' 2024-03-04T07:42:29.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/25' 2024-03-04T07:42:29.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/30' 2024-03-04T07:42:29.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/33' 2024-03-04T07:42:29.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/23' 2024-03-04T07:42:29.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/35' 2024-03-04T07:42:29.522 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/26' 2024-03-04T07:42:32.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './276/37' 2024-03-04T07:42:32.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './276' 2024-03-04T07:42:32.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/21' 2024-03-04T07:42:32.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/27' 2024-03-04T07:42:32.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/31' 2024-03-04T07:42:32.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/36' 2024-03-04T07:42:32.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/20' 2024-03-04T07:42:32.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/39' 2024-03-04T07:42:32.541 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/34' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/38' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/28' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/24' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/29' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/32' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/22' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/25' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/30' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/33' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/23' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/35' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/26' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './277/37' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './277' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/21' 2024-03-04T07:42:32.542 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/27' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/31' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/36' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/20' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/39' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/34' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/38' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/28' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/24' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/29' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/32' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/22' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/25' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/30' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/33' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/23' 2024-03-04T07:42:32.543 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/35' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/26' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './278/37' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './278' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/21' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/27' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/31' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/36' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/20' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/39' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/34' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/38' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/28' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/24' 2024-03-04T07:42:32.544 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/29' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/32' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/22' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/25' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/30' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/33' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/23' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/35' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/26' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './279/37' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './279' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/21' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/27' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/31' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/36' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/20' 2024-03-04T07:42:32.545 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/39' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/34' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/38' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/28' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/24' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/29' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/32' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/22' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/25' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/30' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/33' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/23' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/35' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/26' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed './280/37' 2024-03-04T07:42:32.546 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './280' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/21' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/27' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/31' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/36' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/20' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/39' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/34' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/38' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/28' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/24' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/29' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/32' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/22' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/25' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/30' 2024-03-04T07:42:32.547 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/33' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/23' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/35' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/26' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './281/37' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './281' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/21' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/27' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/31' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/36' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/20' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/39' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/34' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/38' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/28' 2024-03-04T07:42:32.548 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/24' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/29' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/32' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/22' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/25' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/30' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/33' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/23' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/35' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/26' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './282/37' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './282' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/21' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/27' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/31' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/36' 2024-03-04T07:42:32.549 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/20' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/39' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/34' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/38' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/28' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/24' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/29' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/32' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/22' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/25' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/30' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/33' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/23' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/35' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/26' 2024-03-04T07:42:32.550 INFO:tasks.workunit.client.0.smithi113.stdout:removed './283/37' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './283' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/21' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/27' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/31' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/36' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/20' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/39' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/34' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/38' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/28' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/24' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/29' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/32' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/22' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/25' 2024-03-04T07:42:32.551 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/30' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/33' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/23' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/35' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/26' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './284/37' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './284' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/21' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/27' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/31' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/36' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/20' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/39' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/34' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/38' 2024-03-04T07:42:32.552 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/28' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/24' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/29' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/32' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/22' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/25' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/30' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/33' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/23' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/35' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/26' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './285/37' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './285' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/21' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/27' 2024-03-04T07:42:32.553 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/31' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/36' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/20' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/39' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/34' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/38' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/28' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/24' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/29' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/32' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/22' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/25' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/30' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/33' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/23' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/35' 2024-03-04T07:42:32.554 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/26' 2024-03-04T07:42:32.555 INFO:tasks.workunit.client.0.smithi113.stdout:removed './286/37' 2024-03-04T07:42:32.555 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './286' 2024-03-04T07:42:40.646 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/21' 2024-03-04T07:42:40.646 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/27' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/31' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/36' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/20' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/39' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/34' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/38' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/28' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/24' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/29' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/32' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/22' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/25' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/30' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/33' 2024-03-04T07:42:40.647 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/23' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/35' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/26' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './287/37' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './287' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/21' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/27' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/31' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/36' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/20' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/39' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/34' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/38' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/28' 2024-03-04T07:42:40.648 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/24' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/29' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/32' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/22' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/25' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/30' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/33' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/23' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/35' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/26' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './288/37' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './288' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/21' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/27' 2024-03-04T07:42:40.649 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/31' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/36' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/20' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/39' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/34' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/38' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/28' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/24' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/29' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/32' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/22' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/25' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/30' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/33' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/23' 2024-03-04T07:42:40.650 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/35' 2024-03-04T07:42:40.651 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/26' 2024-03-04T07:42:40.651 INFO:tasks.workunit.client.0.smithi113.stdout:removed './289/37' 2024-03-04T07:42:40.651 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './289' 2024-03-04T07:42:40.651 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/21' 2024-03-04T07:42:40.651 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/27' 2024-03-04T07:42:40.651 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/31' 2024-03-04T07:42:40.651 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/36' 2024-03-04T07:42:40.651 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/20' 2024-03-04T07:42:40.651 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/39' 2024-03-04T07:42:40.651 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/34' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/38' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/28' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/24' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/29' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/32' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/22' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/25' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/30' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/33' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/23' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/35' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/26' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed './290/37' 2024-03-04T07:42:40.652 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './290' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/21' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/27' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/31' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/36' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/20' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/39' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/34' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/38' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/28' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/24' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/29' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/32' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/22' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/25' 2024-03-04T07:42:40.653 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/30' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/33' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/23' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/35' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/26' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './291/37' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './291' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/21' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/27' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/31' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/36' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/20' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/39' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/34' 2024-03-04T07:42:40.654 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/38' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/28' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/24' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/29' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/32' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/22' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/25' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/30' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/33' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/23' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/35' 2024-03-04T07:42:40.655 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/26' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './292/37' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './292' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/21' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/27' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/31' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/36' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/20' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/39' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/34' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/38' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/28' 2024-03-04T07:42:40.656 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/24' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/29' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/32' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/22' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/25' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/30' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/33' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/23' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/35' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/26' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './293/37' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './293' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/21' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/27' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/31' 2024-03-04T07:42:40.657 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/36' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/20' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/39' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/34' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/38' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/28' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/24' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/29' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/32' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/22' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/25' 2024-03-04T07:42:40.658 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/30' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/33' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/23' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/35' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/26' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './294/37' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './294' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/21' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/27' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/31' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/36' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/20' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/39' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/34' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/38' 2024-03-04T07:42:40.659 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/28' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/24' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/29' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/32' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/22' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/25' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/30' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/33' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/23' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/35' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/26' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed './295/37' 2024-03-04T07:42:40.660 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './295' 2024-03-04T07:42:40.661 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/21' 2024-03-04T07:42:40.661 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/27' 2024-03-04T07:42:40.661 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/31' 2024-03-04T07:42:40.661 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/36' 2024-03-04T07:42:40.661 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/20' 2024-03-04T07:42:40.661 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/39' 2024-03-04T07:42:40.661 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/34' 2024-03-04T07:42:40.661 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/38' 2024-03-04T07:42:40.661 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/28' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/24' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/29' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/32' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/22' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/25' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/30' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/33' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/23' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/35' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/26' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed './296/37' 2024-03-04T07:42:40.662 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './296' 2024-03-04T07:42:40.663 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/21' 2024-03-04T07:42:40.663 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/27' 2024-03-04T07:42:41.190 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/31' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/36' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/20' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/39' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/34' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/38' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/28' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/24' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/29' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/32' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/22' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/25' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/30' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/33' 2024-03-04T07:42:41.191 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/23' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/35' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/26' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './297/37' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './297' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/21' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/27' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/31' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/36' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/20' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/39' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/34' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/38' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/28' 2024-03-04T07:42:41.192 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/24' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/29' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/32' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/22' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/25' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/30' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/33' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/23' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/35' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/26' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './298/37' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './298' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/21' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/27' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/31' 2024-03-04T07:42:41.193 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/36' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/20' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/39' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/34' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/38' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/28' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/24' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/29' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/32' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/22' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/25' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/30' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/33' 2024-03-04T07:42:41.194 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/23' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/35' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/26' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './299/37' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './299' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/19' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/15' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/20' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/12' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/10' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/11' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/16' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/14' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/18' 2024-03-04T07:42:41.195 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/13' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/166/17' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/166' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/19' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/15' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/20' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/12' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/10' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/11' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/16' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/14' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/18' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/13' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/147/17' 2024-03-04T07:42:41.196 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/147' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/19' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/15' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/20' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/12' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/10' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/11' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/16' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/14' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/18' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/13' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/151/17' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/151' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/19' 2024-03-04T07:42:41.197 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/15' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/20' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/12' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/10' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/11' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/16' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/14' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/18' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/13' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/170/17' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/170' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/19' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/15' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/20' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/12' 2024-03-04T07:42:41.198 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/10' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/11' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/16' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/14' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/18' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/13' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/167/17' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/167' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/19' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/15' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/20' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/12' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/10' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/11' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/16' 2024-03-04T07:42:41.199 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/14' 2024-03-04T07:42:41.200 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/18' 2024-03-04T07:42:41.200 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/13' 2024-03-04T07:42:41.200 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/140/17' 2024-03-04T07:42:41.200 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/140' 2024-03-04T07:42:41.200 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/19' 2024-03-04T07:42:41.200 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/15' 2024-03-04T07:42:41.200 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/20' 2024-03-04T07:42:41.200 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/12' 2024-03-04T07:42:41.200 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/10' 2024-03-04T07:42:42.271 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/11' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/16' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/14' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/18' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/13' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/142/17' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/142' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/19' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/15' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/20' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/12' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/10' 2024-03-04T07:42:42.272 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/11' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/16' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/14' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/18' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/13' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/150/17' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/150' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/19' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/15' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/20' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/12' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/10' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/11' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/16' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/14' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/18' 2024-03-04T07:42:42.273 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/13' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/105/17' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/105' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/19' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/15' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/20' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/12' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/10' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/11' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/16' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/14' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/18' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/13' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/149/17' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/149' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/19' 2024-03-04T07:42:42.274 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/15' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/20' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/12' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/10' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/11' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/16' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/14' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/18' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/13' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/109/17' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/109' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/19' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/15' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/20' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/12' 2024-03-04T07:42:42.275 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/10' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/11' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/16' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/14' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/18' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/13' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/158/17' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/158' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/19' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/15' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/20' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/12' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/10' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/11' 2024-03-04T07:42:42.276 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/16' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/14' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/18' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/13' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/106/17' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/106' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/19' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/15' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/20' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/12' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/10' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/11' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/16' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/14' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/18' 2024-03-04T07:42:42.277 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/13' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/189/17' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/189' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/19' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/15' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/20' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/12' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/10' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/11' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/16' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/14' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/18' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/13' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/133/17' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/133' 2024-03-04T07:42:42.278 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/19' 2024-03-04T07:42:42.279 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/15' 2024-03-04T07:42:42.279 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/20' 2024-03-04T07:42:42.279 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/12' 2024-03-04T07:42:42.279 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/10' 2024-03-04T07:42:42.279 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/11' 2024-03-04T07:42:47.374 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/16' 2024-03-04T07:42:47.374 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/14' 2024-03-04T07:42:47.374 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/18' 2024-03-04T07:42:47.374 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/13' 2024-03-04T07:42:47.374 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/173/17' 2024-03-04T07:42:47.374 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/173' 2024-03-04T07:42:47.374 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/19' 2024-03-04T07:42:47.374 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/15' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/20' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/12' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/10' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/11' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/16' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/14' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/18' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/13' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/174/17' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/174' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/19' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/15' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/20' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/12' 2024-03-04T07:42:47.375 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/10' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/11' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/16' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/14' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/18' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/13' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/155/17' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/155' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/19' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/15' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/20' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/12' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/10' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/11' 2024-03-04T07:42:47.376 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/16' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/14' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/18' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/13' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/114/17' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/114' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/19' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/15' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/20' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/12' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/10' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/11' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/16' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/14' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/18' 2024-03-04T07:42:47.377 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/13' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/159/17' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/159' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/19' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/15' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/20' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/12' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/10' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/11' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/16' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/14' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/18' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/13' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/187/17' 2024-03-04T07:42:47.378 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/187' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/19' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/15' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/20' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/12' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/10' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/11' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/16' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/14' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/18' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/13' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/185/17' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/185' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/19' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/15' 2024-03-04T07:42:47.379 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/20' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/12' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/10' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/11' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/16' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/14' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/18' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/13' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/120/17' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/120' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/19' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/15' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/20' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/12' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/10' 2024-03-04T07:42:47.380 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/11' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/16' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/14' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/18' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/13' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/110/17' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/110' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/19' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/15' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/20' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/12' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/10' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/11' 2024-03-04T07:42:47.381 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/16' 2024-03-04T07:42:51.859 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/14' 2024-03-04T07:42:51.859 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/18' 2024-03-04T07:42:51.859 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/13' 2024-03-04T07:42:51.859 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/137/17' 2024-03-04T07:42:51.859 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/137' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/19' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/15' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/20' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/12' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/10' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/11' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/16' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/14' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/18' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/13' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/198/17' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/198' 2024-03-04T07:42:51.860 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/19' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/15' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/20' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/12' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/10' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/11' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/16' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/14' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/18' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/13' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/136/17' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/136' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/19' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/15' 2024-03-04T07:42:51.861 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/20' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/12' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/10' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/11' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/16' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/14' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/18' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/13' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/125/17' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/125' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/19' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/15' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/20' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/12' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/10' 2024-03-04T07:42:51.862 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/11' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/16' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/14' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/18' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/13' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/194/17' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/194' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/19' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/15' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/20' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/12' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/10' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/11' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/16' 2024-03-04T07:42:51.863 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/14' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/18' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/13' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/119/17' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/119' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/19' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/15' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/20' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/12' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/10' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/11' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/16' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/14' 2024-03-04T07:42:51.864 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/18' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/13' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/157/17' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/157' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/19' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/15' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/20' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/12' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/10' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/11' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/16' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/14' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/18' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/13' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/172/17' 2024-03-04T07:42:51.865 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/172' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/19' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/15' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/20' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/12' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/10' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/11' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/16' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/14' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/18' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/13' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/186/17' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/186' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/19' 2024-03-04T07:42:51.866 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/15' 2024-03-04T07:42:51.867 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/20' 2024-03-04T07:42:51.867 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/12' 2024-03-04T07:42:51.867 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/10' 2024-03-04T07:42:51.867 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/11' 2024-03-04T07:42:51.867 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/16' 2024-03-04T07:42:51.867 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/14' 2024-03-04T07:42:52.682 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/18' 2024-03-04T07:42:52.682 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/13' 2024-03-04T07:42:52.682 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/190/17' 2024-03-04T07:42:52.682 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/190' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/19' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/15' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/20' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/12' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/10' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/11' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/16' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/14' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/18' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/13' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/104/17' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/104' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/19' 2024-03-04T07:42:52.683 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/15' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/20' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/12' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/10' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/11' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/16' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/14' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/18' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/13' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/164/17' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/164' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/19' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/15' 2024-03-04T07:42:52.684 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/20' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/12' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/10' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/11' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/16' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/14' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/18' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/13' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/112/17' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/112' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/19' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/15' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/20' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/12' 2024-03-04T07:42:52.685 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/10' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/11' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/16' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/14' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/18' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/13' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/100/17' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/100' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/19' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/15' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/20' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/12' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/10' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/11' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/16' 2024-03-04T07:42:52.686 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/14' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/18' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/13' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/134/17' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/134' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/19' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/15' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/20' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/12' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/10' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/11' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/16' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/14' 2024-03-04T07:42:52.687 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/18' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/13' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/115/17' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/115' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/19' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/15' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/20' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/12' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/10' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/11' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/16' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/14' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/18' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/13' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/116/17' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/116' 2024-03-04T07:42:52.688 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/19' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/15' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/20' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/12' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/10' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/11' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/16' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/14' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/18' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/13' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/195/17' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/195' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/19' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/15' 2024-03-04T07:42:52.689 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/20' 2024-03-04T07:42:52.690 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/12' 2024-03-04T07:42:52.690 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/10' 2024-03-04T07:42:52.690 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/11' 2024-03-04T07:42:52.690 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/16' 2024-03-04T07:42:52.690 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/14' 2024-03-04T07:42:52.690 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/18' 2024-03-04T07:42:54.104 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/13' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/138/17' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/138' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/19' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/15' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/20' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/12' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/10' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/11' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/16' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/14' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/18' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/13' 2024-03-04T07:42:54.105 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/135/17' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/135' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/19' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/15' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/20' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/12' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/10' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/11' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/16' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/14' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/18' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/13' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/117/17' 2024-03-04T07:42:54.106 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/117' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/19' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/15' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/20' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/12' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/10' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/11' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/16' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/14' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/18' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/13' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/143/17' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/143' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/19' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/15' 2024-03-04T07:42:54.107 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/20' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/12' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/10' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/11' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/16' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/14' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/18' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/13' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/180/17' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/180' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/19' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/15' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/20' 2024-03-04T07:42:54.108 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/12' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/10' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/11' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/16' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/14' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/18' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/13' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/126/17' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/126' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/19' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/15' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/20' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/12' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/10' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/11' 2024-03-04T07:42:54.109 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/16' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/14' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/18' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/13' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/161/17' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/161' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/19' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/15' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/20' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/12' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/10' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/11' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/16' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/14' 2024-03-04T07:42:54.110 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/18' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/13' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/176/17' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/176' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/19' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/15' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/20' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/12' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/10' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/11' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/16' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/14' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/18' 2024-03-04T07:42:54.111 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/13' 2024-03-04T07:42:54.112 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/111/17' 2024-03-04T07:42:54.112 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/111' 2024-03-04T07:42:54.112 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/19' 2024-03-04T07:42:54.112 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/15' 2024-03-04T07:42:54.112 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/20' 2024-03-04T07:42:54.112 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/12' 2024-03-04T07:42:54.112 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/10' 2024-03-04T07:42:54.112 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/11' 2024-03-04T07:42:54.113 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/16' 2024-03-04T07:42:54.113 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/14' 2024-03-04T07:42:54.113 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/18' 2024-03-04T07:42:54.113 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/13' 2024-03-04T07:42:54.996 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/191/17' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/191' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/19' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/15' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/20' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/12' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/10' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/11' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/16' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/14' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/18' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/13' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/184/17' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/184' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/19' 2024-03-04T07:42:54.997 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/15' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/20' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/12' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/10' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/11' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/16' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/14' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/18' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/13' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/192/17' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/192' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/19' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/15' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/20' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/12' 2024-03-04T07:42:54.998 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/10' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/11' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/16' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/14' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/18' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/13' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/103/17' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/103' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/19' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/15' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/20' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/12' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/10' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/11' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/16' 2024-03-04T07:42:54.999 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/14' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/18' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/13' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/128/17' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/128' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/19' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/15' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/20' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/12' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/10' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/11' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/16' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/14' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/18' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/13' 2024-03-04T07:42:55.000 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/144/17' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/144' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/19' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/15' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/20' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/12' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/10' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/11' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/16' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/14' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/18' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/13' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/146/17' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/146' 2024-03-04T07:42:55.001 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/19' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/15' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/20' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/12' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/10' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/11' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/16' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/14' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/18' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/13' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/141/17' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/141' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/19' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/15' 2024-03-04T07:42:55.002 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/20' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/12' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/10' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/11' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/16' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/14' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/18' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/13' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/182/17' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/182' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/19' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/15' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/20' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/12' 2024-03-04T07:42:55.003 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/10' 2024-03-04T07:42:55.004 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/11' 2024-03-04T07:42:55.004 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/16' 2024-03-04T07:42:55.004 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/14' 2024-03-04T07:42:55.004 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/18' 2024-03-04T07:42:55.004 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/13' 2024-03-04T07:42:55.004 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/107/17' 2024-03-04T07:42:56.475 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:42:56.479 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:42:58.440 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/107' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/19' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/15' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/20' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/12' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/10' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/11' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/16' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/14' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/18' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/13' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/178/17' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/178' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/19' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/15' 2024-03-04T07:42:58.441 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/20' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/12' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/10' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/11' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/16' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/14' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/18' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/13' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/156/17' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/156' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/19' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/15' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/20' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/12' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/10' 2024-03-04T07:42:58.442 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/11' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/16' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/14' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/18' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/13' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/131/17' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/131' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/19' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/15' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/20' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/12' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/10' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/11' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/16' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/14' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/18' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/13' 2024-03-04T07:42:58.443 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/123/17' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/123' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/19' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/15' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/20' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/12' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/10' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/11' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/16' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/14' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/18' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/13' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/160/17' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/160' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/19' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/15' 2024-03-04T07:42:58.444 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/20' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/12' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/10' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/11' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/16' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/14' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/18' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/13' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/165/17' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/165' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/19' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/15' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/20' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/12' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/10' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/11' 2024-03-04T07:42:58.445 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/16' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/14' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/18' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/13' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/101/17' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/101' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/19' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/15' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/20' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/12' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/10' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/11' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/16' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/14' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/18' 2024-03-04T07:42:58.446 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/13' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/148/17' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/148' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/19' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/15' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/20' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/12' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/10' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/11' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/16' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/14' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/18' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/13' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/168/17' 2024-03-04T07:42:58.447 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/168' 2024-03-04T07:42:59.024 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/19' 2024-03-04T07:42:59.024 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/15' 2024-03-04T07:42:59.024 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/20' 2024-03-04T07:42:59.024 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/12' 2024-03-04T07:42:59.024 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/10' 2024-03-04T07:42:59.024 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/11' 2024-03-04T07:42:59.024 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/16' 2024-03-04T07:42:59.024 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/14' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/18' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/13' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/179/17' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/179' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/19' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/15' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/20' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/12' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/10' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/11' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/16' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/14' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/18' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/13' 2024-03-04T07:42:59.025 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/132/17' 2024-03-04T07:42:59.026 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/132' 2024-03-04T07:42:59.026 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/19' 2024-03-04T07:42:59.026 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/15' 2024-03-04T07:42:59.026 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/20' 2024-03-04T07:42:59.026 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/12' 2024-03-04T07:42:59.026 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/10' 2024-03-04T07:42:59.026 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/11' 2024-03-04T07:42:59.026 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/16' 2024-03-04T07:42:59.026 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/14' 2024-03-04T07:42:59.026 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/18' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/13' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/199/17' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/199' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/19' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/15' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/20' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/12' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/10' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/11' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/16' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/14' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/18' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/13' 2024-03-04T07:42:59.027 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/163/17' 2024-03-04T07:42:59.028 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/163' 2024-03-04T07:42:59.028 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/19' 2024-03-04T07:42:59.028 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/15' 2024-03-04T07:42:59.028 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/20' 2024-03-04T07:42:59.028 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/12' 2024-03-04T07:42:59.028 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/10' 2024-03-04T07:42:59.028 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/11' 2024-03-04T07:42:59.028 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/16' 2024-03-04T07:42:59.028 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/14' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/18' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/13' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/153/17' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/153' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/19' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/15' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/20' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/12' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/10' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/11' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/16' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/14' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/18' 2024-03-04T07:42:59.029 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/13' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/122/17' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/122' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/19' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/15' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/20' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/12' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/10' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/11' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/16' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/14' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/18' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/13' 2024-03-04T07:42:59.030 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/162/17' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/162' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/19' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/15' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/20' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/12' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/10' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/11' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/16' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/14' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/18' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/13' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/197/17' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/197' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/19' 2024-03-04T07:42:59.031 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/15' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/20' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/12' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/10' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/11' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/16' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/14' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/18' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/13' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/113/17' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/113' 2024-03-04T07:42:59.032 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/19' 2024-03-04T07:43:02.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/15' 2024-03-04T07:43:02.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/20' 2024-03-04T07:43:02.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/12' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/10' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/11' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/16' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/14' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/18' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/13' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/152/17' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/152' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/19' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/15' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/20' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/12' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/10' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/11' 2024-03-04T07:43:02.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/16' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/14' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/18' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/13' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/121/17' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/121' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/19' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/15' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/20' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/12' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/10' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/11' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/16' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/14' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/18' 2024-03-04T07:43:02.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/13' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/118/17' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/118' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/19' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/15' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/20' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/12' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/10' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/11' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/16' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/14' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/18' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/13' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/196/17' 2024-03-04T07:43:02.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/196' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/19' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/15' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/20' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/12' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/10' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/11' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/16' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/14' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/18' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/13' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/124/17' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/124' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/19' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/15' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/20' 2024-03-04T07:43:02.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/12' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/10' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/11' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/16' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/14' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/18' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/13' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/188/17' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/188' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/19' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/15' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/20' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/12' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/10' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/11' 2024-03-04T07:43:02.958 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/16' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/14' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/18' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/13' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/145/17' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/145' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/19' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/15' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/20' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/12' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/10' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/11' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/16' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/14' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/18' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/13' 2024-03-04T07:43:02.959 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/130/17' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/130' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/19' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/15' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/20' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/12' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/10' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/11' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/16' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/14' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/18' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/13' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/193/17' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/193' 2024-03-04T07:43:02.960 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/19' 2024-03-04T07:43:02.961 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/15' 2024-03-04T07:43:04.950 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/20' 2024-03-04T07:43:04.950 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/12' 2024-03-04T07:43:04.950 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/10' 2024-03-04T07:43:04.950 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/11' 2024-03-04T07:43:04.950 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/16' 2024-03-04T07:43:04.950 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/14' 2024-03-04T07:43:04.950 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/18' 2024-03-04T07:43:04.950 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/13' 2024-03-04T07:43:04.950 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/175/17' 2024-03-04T07:43:04.950 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/175' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/19' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/15' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/20' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/12' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/10' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/11' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/16' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/14' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/18' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/13' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/181/17' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/181' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/19' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/15' 2024-03-04T07:43:04.951 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/20' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/12' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/10' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/11' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/16' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/14' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/18' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/13' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/127/17' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/127' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/19' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/15' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/20' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/12' 2024-03-04T07:43:04.952 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/10' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/11' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/16' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/14' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/18' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/13' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/108/17' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/108' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/19' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/15' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/20' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/12' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/10' 2024-03-04T07:43:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/11' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/16' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/14' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/18' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/13' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/183/17' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/183' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/19' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/15' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/20' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/12' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/10' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/11' 2024-03-04T07:43:04.954 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/16' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/14' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/18' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/13' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/177/17' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/177' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/19' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/15' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/20' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/12' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/10' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/11' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/16' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/14' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/18' 2024-03-04T07:43:04.955 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/13' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/169/17' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/169' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/19' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/15' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/20' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/12' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/10' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/11' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/16' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/14' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/18' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/13' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/154/17' 2024-03-04T07:43:04.956 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/154' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/19' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/15' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/20' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/12' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/10' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/11' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/16' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/14' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/18' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/13' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/171/17' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/171' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/19' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/15' 2024-03-04T07:43:04.957 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/20' 2024-03-04T07:43:05.341 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/12' 2024-03-04T07:43:05.341 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/10' 2024-03-04T07:43:05.341 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/11' 2024-03-04T07:43:05.341 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/16' 2024-03-04T07:43:05.341 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/14' 2024-03-04T07:43:05.341 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/18' 2024-03-04T07:43:05.341 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/13' 2024-03-04T07:43:05.341 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/139/17' 2024-03-04T07:43:05.341 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/139' 2024-03-04T07:43:05.341 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/19' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/15' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/20' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/12' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/10' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/11' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/16' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/14' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/18' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/13' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/102/17' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/102' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/19' 2024-03-04T07:43:05.342 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/15' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/20' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/12' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/10' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/11' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/16' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/14' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/18' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/13' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed './copyofsnap1/test1/129/17' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1/129' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1/test1' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:removed directory './copyofsnap1' 2024-03-04T07:43:05.343 INFO:tasks.workunit.client.0.smithi113.stdout:OK 2024-03-04T07:43:05.345 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T07:43:05.345 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T07:43:06.185 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2024-03-04T07:43:06.186 DEBUG:teuthology.orchestra.run.smithi113:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2024-03-04T07:43:06.838 INFO:tasks.workunit.client.0.smithi113.stderr:+ set -e 2024-03-04T07:43:06.838 INFO:tasks.workunit.client.0.smithi113.stderr:+ touch foo 2024-03-04T07:43:06.841 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod +x foo 2024-03-04T07:43:06.841 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/s 2024-03-04T07:43:07.150 INFO:tasks.workunit.client.0.smithi113.stderr:+ find .snap/s/foo -executable 2024-03-04T07:43:07.150 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep foo 2024-03-04T07:43:07.153 INFO:tasks.workunit.client.0.smithi113.stdout:.snap/s/foo 2024-03-04T07:43:07.153 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/s 2024-03-04T07:43:07.162 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm foo 2024-03-04T07:43:07.167 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo OK 2024-03-04T07:43:07.168 INFO:tasks.workunit.client.0.smithi113.stdout:OK 2024-03-04T07:43:07.168 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T07:43:07.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T07:43:07.199 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2024-03-04T07:43:07.199 DEBUG:teuthology.orchestra.run.smithi113:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2024-03-04T07:43:11.829 INFO:tasks.workunit.client.0.smithi113.stderr:+ set -e 2024-03-04T07:43:11.829 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir foo 2024-03-04T07:43:11.832 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo x 2024-03-04T07:43:11.832 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:12.954 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir foo/.snap/ss 2024-03-04T07:43:12.965 INFO:tasks.workunit.client.0.smithi113.stderr:+ ln foo/x foo/xx 2024-03-04T07:43:12.967 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat foo/.snap/ss/x 2024-03-04T07:43:12.973 INFO:tasks.workunit.client.0.smithi113.stdout:x 2024-03-04T07:43:12.974 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir foo/.snap/ss 2024-03-04T07:43:12.982 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo a 2024-03-04T07:43:12.982 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo b 2024-03-04T07:43:12.982 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir foo/.snap/s 2024-03-04T07:43:12.999 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat foo/.snap/s/a 2024-03-04T07:43:13.007 INFO:tasks.workunit.client.0.smithi113.stderr:+ r=a 2024-03-04T07:43:13.007 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ -z a ] 2024-03-04T07:43:13.007 INFO:tasks.workunit.client.0.smithi113.stderr:+ ln foo/b foo/b2 2024-03-04T07:43:13.009 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat foo/.snap/s/b 2024-03-04T07:43:13.018 INFO:tasks.workunit.client.0.smithi113.stdout:b 2024-03-04T07:43:13.018 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo this used to hang: 2024-03-04T07:43:13.018 INFO:tasks.workunit.client.0.smithi113.stdout:this used to hang: 2024-03-04T07:43:13.018 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo more 2024-03-04T07:43:13.019 INFO:tasks.workunit.client.0.smithi113.stdout:oh, it didn't hang! good job. 2024-03-04T07:43:13.019 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo oh, it didn't hang! good job. 2024-03-04T07:43:13.019 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat foo/b 2024-03-04T07:43:13.019 INFO:tasks.workunit.client.0.smithi113.stdout:b 2024-03-04T07:43:13.019 INFO:tasks.workunit.client.0.smithi113.stdout:more 2024-03-04T07:43:13.020 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir foo/.snap/s 2024-03-04T07:43:13.026 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm -r foo 2024-03-04T07:43:13.045 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo OK 2024-03-04T07:43:13.046 INFO:tasks.workunit.client.0.smithi113.stdout:OK 2024-03-04T07:43:13.046 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T07:43:13.046 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T07:43:13.076 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2024-03-04T07:43:13.076 DEBUG:teuthology.orchestra.run.smithi113:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2024-03-04T07:43:16.836 INFO:tasks.workunit.client.0.smithi113.stderr:+ set -e 2024-03-04T07:43:16.837 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:16.838 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/bar 2024-03-04T07:43:16.849 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/foo 2024-03-04T07:43:16.857 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/foo 2024-03-04T07:43:17.109 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:17.112 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/bar/a 2024-03-04T07:43:17.116 INFO:tasks.workunit.client.0.smithi113.stdout: File: .snap/bar/a 2024-03-04T07:43:17.116 INFO:tasks.workunit.client.0.smithi113.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-04T07:43:17.116 INFO:tasks.workunit.client.0.smithi113.stdout:Device: fah/250d Inode: 1099511633796 Links: 2 2024-03-04T07:43:17.116 INFO:tasks.workunit.client.0.smithi113.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1271/ ubuntu) 2024-03-04T07:43:17.116 INFO:tasks.workunit.client.0.smithi113.stdout:Access: 2024-03-04 07:43:16.838982042 +0000 2024-03-04T07:43:17.116 INFO:tasks.workunit.client.0.smithi113.stdout:Modify: 2024-03-04 07:43:16.838982042 +0000 2024-03-04T07:43:17.116 INFO:tasks.workunit.client.0.smithi113.stdout:Change: 2024-03-04 07:43:16.838982042 +0000 2024-03-04T07:43:17.116 INFO:tasks.workunit.client.0.smithi113.stdout: Birth: 2024-03-04 07:43:16.838982042 +0000 2024-03-04T07:43:17.117 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/bar 2024-03-04T07:43:17.127 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo OK 2024-03-04T07:43:17.128 INFO:tasks.workunit.client.0.smithi113.stdout:OK 2024-03-04T07:43:17.128 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T07:43:17.128 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T07:43:17.152 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2024-03-04T07:43:17.153 DEBUG:teuthology.orchestra.run.smithi113:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2024-03-04T07:43:21.830 INFO:tasks.workunit.client.0.smithi113.stderr:+ set -e 2024-03-04T07:43:21.830 INFO:tasks.workunit.client.0.smithi113.stderr:+ seq 1 20 2024-03-04T07:43:21.831 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:21.834 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:21.835 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:21.845 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:21.852 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:21.853 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:22.202 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:22.216 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:22.217 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:22.217 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:22.217 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:22.217 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:22.217 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:22.217 INFO:tasks.workunit.client.0.smithi113.stdout: 38816 ? 00:00:00 ps 2024-03-04T07:43:22.217 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:22.229 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:22.232 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:22.239 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:22.241 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:22.243 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:22.253 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:22.259 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:22.260 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:22.343 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:22.358 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:22.358 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:22.358 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:22.358 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:22.358 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:22.359 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:22.359 INFO:tasks.workunit.client.0.smithi113.stdout: 38826 ? 00:00:00 ps 2024-03-04T07:43:22.359 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:22.370 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:22.374 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:22.382 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:22.384 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:22.386 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:22.395 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:22.402 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:22.403 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:22.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:22.491 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:22.491 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:22.491 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:22.491 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:22.491 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:22.491 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:22.491 INFO:tasks.workunit.client.0.smithi113.stdout: 38836 ? 00:00:00 ps 2024-03-04T07:43:22.492 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:22.503 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:22.506 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:22.514 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:22.517 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:22.519 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:22.529 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:22.537 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:22.537 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:22.592 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:22.606 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:22.606 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:22.606 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:22.606 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:22.606 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:22.606 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:22.606 INFO:tasks.workunit.client.0.smithi113.stdout: 38846 ? 00:00:00 ps 2024-03-04T07:43:22.607 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:22.618 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:22.621 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:22.806 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:22.808 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:22.810 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:22.822 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:22.831 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:22.831 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:22.894 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:22.908 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:22.908 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:22.908 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:22.908 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:22.908 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:22.908 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:22.908 INFO:tasks.workunit.client.0.smithi113.stdout: 38856 ? 00:00:00 ps 2024-03-04T07:43:22.909 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:22.921 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:22.925 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:22.933 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:22.935 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:22.936 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:22.947 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:22.954 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:22.955 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:23.011 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:23.025 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:23.025 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:23.025 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:23.025 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:23.025 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:23.025 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:23.025 INFO:tasks.workunit.client.0.smithi113.stdout: 38866 ? 00:00:00 ps 2024-03-04T07:43:23.026 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:23.038 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:23.040 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:23.048 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:23.052 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:23.053 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:23.063 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:23.070 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:23.071 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:23.136 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:23.150 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:23.150 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:23.150 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:23.150 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:23.150 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:23.150 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:23.150 INFO:tasks.workunit.client.0.smithi113.stdout: 38876 ? 00:00:00 ps 2024-03-04T07:43:23.151 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:23.163 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:23.166 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:23.174 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:23.177 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:23.178 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:23.190 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:23.196 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:23.197 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:23.253 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:23.274 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:23.274 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:23.274 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:23.275 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:23.275 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:23.275 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:23.275 INFO:tasks.workunit.client.0.smithi113.stdout: 38886 ? 00:00:00 ps 2024-03-04T07:43:23.275 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:23.287 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:23.290 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:23.298 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:23.301 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:23.302 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:23.312 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:23.319 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:23.320 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:23.386 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:23.400 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:23.400 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:23.400 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:23.400 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:23.400 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:23.400 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:23.400 INFO:tasks.workunit.client.0.smithi113.stdout: 38896 ? 00:00:00 ps 2024-03-04T07:43:23.401 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:23.413 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:23.417 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:23.426 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:23.429 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:23.430 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:23.441 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:23.449 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:23.449 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:23.504 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:23.518 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:23.518 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:23.518 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:23.519 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:23.519 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:23.519 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:23.519 INFO:tasks.workunit.client.0.smithi113.stdout: 38906 ? 00:00:00 ps 2024-03-04T07:43:23.520 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:23.531 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:23.535 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:23.544 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:23.548 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:23.549 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:23.559 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:23.567 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:23.568 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:23.629 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:23.643 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:23.643 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:23.643 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:23.643 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:23.643 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:23.643 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:23.644 INFO:tasks.workunit.client.0.smithi113.stdout: 38916 ? 00:00:00 ps 2024-03-04T07:43:23.644 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:23.656 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:23.660 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:23.667 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:23.669 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:23.670 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:23.682 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:23.689 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:23.690 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:23.764 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:23.786 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:23.786 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:23.786 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:23.786 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:23.786 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:23.786 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:23.786 INFO:tasks.workunit.client.0.smithi113.stdout: 38926 ? 00:00:00 ps 2024-03-04T07:43:23.787 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:23.799 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:23.802 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:23.809 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:23.811 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:23.813 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:23.824 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:23.833 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:23.834 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:23.896 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:23.910 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:23.910 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:23.910 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:23.910 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:23.910 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:23.910 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:23.910 INFO:tasks.workunit.client.0.smithi113.stdout: 38936 ? 00:00:00 ps 2024-03-04T07:43:23.911 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:23.924 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:23.927 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:23.936 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:23.939 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:23.941 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:23.952 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:23.959 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:23.960 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:24.022 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:24.036 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:24.036 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:24.036 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:24.036 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:24.037 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:24.037 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:24.037 INFO:tasks.workunit.client.0.smithi113.stdout: 38946 ? 00:00:00 ps 2024-03-04T07:43:24.037 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:24.050 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:24.054 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:24.062 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:24.067 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:24.067 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:24.080 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:24.087 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:24.088 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:24.146 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:24.160 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:24.161 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:24.161 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:24.161 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:24.161 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:24.161 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:24.161 INFO:tasks.workunit.client.0.smithi113.stdout: 38956 ? 00:00:00 ps 2024-03-04T07:43:24.161 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:24.174 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:24.178 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:24.187 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:24.191 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:24.192 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:24.203 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:24.211 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:24.212 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:24.281 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:24.295 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:24.295 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:24.295 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:24.295 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:24.295 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:24.295 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:24.295 INFO:tasks.workunit.client.0.smithi113.stdout: 38966 ? 00:00:00 ps 2024-03-04T07:43:24.295 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:24.308 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:24.313 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:24.323 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:24.323 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:24.324 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:24.335 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:24.343 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:24.344 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:24.414 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:24.428 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:24.428 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:24.428 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:24.428 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:24.428 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:24.428 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:24.428 INFO:tasks.workunit.client.0.smithi113.stdout: 38976 ? 00:00:00 ps 2024-03-04T07:43:24.429 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:24.441 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:24.444 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:24.453 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:24.456 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:24.457 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:24.469 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:24.476 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:24.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:24.540 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:24.556 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:24.556 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:24.556 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:24.556 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:24.556 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:24.557 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:24.557 INFO:tasks.workunit.client.0.smithi113.stdout: 38986 ? 00:00:00 ps 2024-03-04T07:43:24.557 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:24.570 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:24.573 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:24.579 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:24.582 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:24.584 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:24.595 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:24.602 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:24.603 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:24.665 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:24.679 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:24.679 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:24.679 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:24.679 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:24.679 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:24.680 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:24.680 INFO:tasks.workunit.client.0.smithi113.stdout: 38996 ? 00:00:00 ps 2024-03-04T07:43:24.680 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:25.579 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:25.582 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:25.590 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a 2024-03-04T07:43:25.593 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/one 2024-03-04T07:43:25.594 INFO:tasks.workunit.client.0.smithi113.stderr:+ cat 2024-03-04T07:43:25.605 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir a/.snap/two 2024-03-04T07:43:25.828 INFO:tasks.workunit.client.0.smithi113.stderr:+ chmod 777 a/foo 2024-03-04T07:43:25.830 INFO:tasks.workunit.client.0.smithi113.stderr:+ sync 2024-03-04T07:43:25.932 INFO:tasks.workunit.client.0.smithi113.stderr:+ ps 2024-03-04T07:43:25.946 INFO:tasks.workunit.client.0.smithi113.stdout: PID TTY TIME CMD 2024-03-04T07:43:25.946 INFO:tasks.workunit.client.0.smithi113.stdout: 1317 ? 00:00:00 systemd 2024-03-04T07:43:25.946 INFO:tasks.workunit.client.0.smithi113.stdout: 1318 ? 00:00:00 (sd-pam) 2024-03-04T07:43:25.946 INFO:tasks.workunit.client.0.smithi113.stdout: 17183 ? 00:00:01 sshd 2024-03-04T07:43:25.946 INFO:tasks.workunit.client.0.smithi113.stdout: 38805 ? 00:00:00 timeout 2024-03-04T07:43:25.946 INFO:tasks.workunit.client.0.smithi113.stdout: 38808 ? 00:00:00 snaptest-double 2024-03-04T07:43:25.946 INFO:tasks.workunit.client.0.smithi113.stdout: 39006 ? 00:00:00 ps 2024-03-04T07:43:25.947 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a/.snap/one a/.snap/two 2024-03-04T07:43:25.960 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm a/foo 2024-03-04T07:43:25.964 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir a 2024-03-04T07:43:25.973 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo OK 2024-03-04T07:43:25.974 INFO:tasks.workunit.client.0.smithi113.stdout:OK 2024-03-04T07:43:25.974 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T07:43:25.974 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T07:43:26.530 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:43:26.533 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:43:27.242 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2024-03-04T07:43:27.242 DEBUG:teuthology.orchestra.run.smithi113:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2024-03-04T07:43:27.258 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/foo 2024-03-04T07:43:27.269 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo We want ENOENT, not ESTALE, here. 2024-03-04T07:43:27.269 INFO:tasks.workunit.client.0.smithi113.stdout:We want ENOENT, not ESTALE, here. 2024-03-04T07:43:27.270 INFO:tasks.workunit.client.0.smithi113.stderr:+ seq 1 100 2024-03-04T07:43:27.270 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/1 2024-03-04T07:43:27.270 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.273 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/1': No such file or directory 2024-03-04T07:43:27.274 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/2 2024-03-04T07:43:27.275 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.276 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/2': No such file or directory 2024-03-04T07:43:27.277 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/3 2024-03-04T07:43:27.277 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.278 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/3': No such file or directory 2024-03-04T07:43:27.279 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/4 2024-03-04T07:43:27.279 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.280 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/4': No such file or directory 2024-03-04T07:43:27.281 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/5 2024-03-04T07:43:27.281 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.283 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/5': No such file or directory 2024-03-04T07:43:27.285 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/6 2024-03-04T07:43:27.285 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.287 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/6': No such file or directory 2024-03-04T07:43:27.288 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/7 2024-03-04T07:43:27.288 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.289 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/7': No such file or directory 2024-03-04T07:43:27.290 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/8 2024-03-04T07:43:27.290 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.292 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/8': No such file or directory 2024-03-04T07:43:27.293 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/9 2024-03-04T07:43:27.293 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.295 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/9': No such file or directory 2024-03-04T07:43:27.296 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/10 2024-03-04T07:43:27.296 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.313 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/10': No such file or directory 2024-03-04T07:43:27.314 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/11 2024-03-04T07:43:27.314 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.316 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/11': No such file or directory 2024-03-04T07:43:27.317 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/12 2024-03-04T07:43:27.317 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.319 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/12': No such file or directory 2024-03-04T07:43:27.321 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/13 2024-03-04T07:43:27.321 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.322 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/13': No such file or directory 2024-03-04T07:43:27.323 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/14 2024-03-04T07:43:27.323 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.324 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/14': No such file or directory 2024-03-04T07:43:27.325 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/15 2024-03-04T07:43:27.325 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.327 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/15': No such file or directory 2024-03-04T07:43:27.328 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/16 2024-03-04T07:43:27.328 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.330 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/16': No such file or directory 2024-03-04T07:43:27.331 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/17 2024-03-04T07:43:27.331 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.333 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/17': No such file or directory 2024-03-04T07:43:27.335 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/18 2024-03-04T07:43:27.335 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.336 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/18': No such file or directory 2024-03-04T07:43:27.337 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/19 2024-03-04T07:43:27.337 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.338 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/19': No such file or directory 2024-03-04T07:43:27.339 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/20 2024-03-04T07:43:27.340 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.343 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/20': No such file or directory 2024-03-04T07:43:27.343 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/21 2024-03-04T07:43:27.344 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.344 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/21': No such file or directory 2024-03-04T07:43:27.344 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/22 2024-03-04T07:43:27.344 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.346 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/22': No such file or directory 2024-03-04T07:43:27.347 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/23 2024-03-04T07:43:27.347 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.348 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/23': No such file or directory 2024-03-04T07:43:27.349 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/24 2024-03-04T07:43:27.350 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.351 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/24': No such file or directory 2024-03-04T07:43:27.351 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/25 2024-03-04T07:43:27.352 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.354 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/25': No such file or directory 2024-03-04T07:43:27.355 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/26 2024-03-04T07:43:27.355 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.357 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/26': No such file or directory 2024-03-04T07:43:27.357 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/27 2024-03-04T07:43:27.357 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.359 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/27': No such file or directory 2024-03-04T07:43:27.360 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/28 2024-03-04T07:43:27.360 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.361 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/28': No such file or directory 2024-03-04T07:43:27.362 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/29 2024-03-04T07:43:27.362 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.364 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/29': No such file or directory 2024-03-04T07:43:27.365 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/30 2024-03-04T07:43:27.365 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.366 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/30': No such file or directory 2024-03-04T07:43:27.367 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/31 2024-03-04T07:43:27.368 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.369 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/31': No such file or directory 2024-03-04T07:43:27.370 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/32 2024-03-04T07:43:27.370 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.371 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/32': No such file or directory 2024-03-04T07:43:27.372 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/33 2024-03-04T07:43:27.373 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.374 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/33': No such file or directory 2024-03-04T07:43:27.375 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/34 2024-03-04T07:43:27.375 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.377 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/34': No such file or directory 2024-03-04T07:43:27.378 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/35 2024-03-04T07:43:27.378 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.380 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/35': No such file or directory 2024-03-04T07:43:27.381 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/36 2024-03-04T07:43:27.381 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.382 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/36': No such file or directory 2024-03-04T07:43:27.383 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/37 2024-03-04T07:43:27.383 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.384 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/37': No such file or directory 2024-03-04T07:43:27.385 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/38 2024-03-04T07:43:27.385 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.387 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/38': No such file or directory 2024-03-04T07:43:27.388 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/39 2024-03-04T07:43:27.388 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.389 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/39': No such file or directory 2024-03-04T07:43:27.390 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/40 2024-03-04T07:43:27.390 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.391 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/40': No such file or directory 2024-03-04T07:43:27.392 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/41 2024-03-04T07:43:27.392 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.393 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/41': No such file or directory 2024-03-04T07:43:27.394 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/42 2024-03-04T07:43:27.394 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.395 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/42': No such file or directory 2024-03-04T07:43:27.396 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/43 2024-03-04T07:43:27.396 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.398 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/43': No such file or directory 2024-03-04T07:43:27.398 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/44 2024-03-04T07:43:27.399 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.400 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/44': No such file or directory 2024-03-04T07:43:27.401 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/45 2024-03-04T07:43:27.401 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.403 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/45': No such file or directory 2024-03-04T07:43:27.403 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/46 2024-03-04T07:43:27.404 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.405 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/46': No such file or directory 2024-03-04T07:43:27.406 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/47 2024-03-04T07:43:27.406 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.407 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/47': No such file or directory 2024-03-04T07:43:27.408 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/48 2024-03-04T07:43:27.408 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.411 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/48': No such file or directory 2024-03-04T07:43:27.412 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/49 2024-03-04T07:43:27.412 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.413 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/49': No such file or directory 2024-03-04T07:43:27.414 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/50 2024-03-04T07:43:27.414 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.416 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/50': No such file or directory 2024-03-04T07:43:27.417 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/51 2024-03-04T07:43:27.418 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.419 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/51': No such file or directory 2024-03-04T07:43:27.420 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/52 2024-03-04T07:43:27.420 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.421 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/52': No such file or directory 2024-03-04T07:43:27.422 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/53 2024-03-04T07:43:27.422 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.424 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/53': No such file or directory 2024-03-04T07:43:27.424 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/54 2024-03-04T07:43:27.424 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.425 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/54': No such file or directory 2024-03-04T07:43:27.426 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/55 2024-03-04T07:43:27.426 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.428 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/55': No such file or directory 2024-03-04T07:43:27.429 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/56 2024-03-04T07:43:27.429 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.430 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/56': No such file or directory 2024-03-04T07:43:27.431 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/57 2024-03-04T07:43:27.431 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.434 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/57': No such file or directory 2024-03-04T07:43:27.435 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/58 2024-03-04T07:43:27.435 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.437 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/58': No such file or directory 2024-03-04T07:43:27.438 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/59 2024-03-04T07:43:27.438 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.439 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/59': No such file or directory 2024-03-04T07:43:27.440 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/60 2024-03-04T07:43:27.440 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.442 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/60': No such file or directory 2024-03-04T07:43:27.442 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/61 2024-03-04T07:43:27.443 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.444 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/61': No such file or directory 2024-03-04T07:43:27.445 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/62 2024-03-04T07:43:27.445 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.446 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/62': No such file or directory 2024-03-04T07:43:27.447 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/63 2024-03-04T07:43:27.447 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.448 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/63': No such file or directory 2024-03-04T07:43:27.449 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/64 2024-03-04T07:43:27.449 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.451 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/64': No such file or directory 2024-03-04T07:43:27.451 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/65 2024-03-04T07:43:27.451 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.453 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/65': No such file or directory 2024-03-04T07:43:27.454 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/66 2024-03-04T07:43:27.454 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.455 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/66': No such file or directory 2024-03-04T07:43:27.456 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/67 2024-03-04T07:43:27.456 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.457 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/67': No such file or directory 2024-03-04T07:43:27.458 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/68 2024-03-04T07:43:27.459 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.460 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/68': No such file or directory 2024-03-04T07:43:27.460 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/69 2024-03-04T07:43:27.461 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.462 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/69': No such file or directory 2024-03-04T07:43:27.463 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/70 2024-03-04T07:43:27.463 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.465 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/70': No such file or directory 2024-03-04T07:43:27.466 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/71 2024-03-04T07:43:27.466 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.467 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/71': No such file or directory 2024-03-04T07:43:27.469 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/72 2024-03-04T07:43:27.469 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.470 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/72': No such file or directory 2024-03-04T07:43:27.471 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/73 2024-03-04T07:43:27.471 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.472 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/73': No such file or directory 2024-03-04T07:43:27.473 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/74 2024-03-04T07:43:27.474 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.476 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/74': No such file or directory 2024-03-04T07:43:27.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/75 2024-03-04T07:43:27.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.478 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/75': No such file or directory 2024-03-04T07:43:27.479 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/76 2024-03-04T07:43:27.479 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.481 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/76': No such file or directory 2024-03-04T07:43:27.482 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/77 2024-03-04T07:43:27.482 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.484 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/77': No such file or directory 2024-03-04T07:43:27.485 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/78 2024-03-04T07:43:27.485 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.486 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/78': No such file or directory 2024-03-04T07:43:27.487 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/79 2024-03-04T07:43:27.487 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.489 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/79': No such file or directory 2024-03-04T07:43:27.490 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/80 2024-03-04T07:43:27.490 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.492 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/80': No such file or directory 2024-03-04T07:43:27.493 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/81 2024-03-04T07:43:27.493 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.494 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/81': No such file or directory 2024-03-04T07:43:27.495 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/82 2024-03-04T07:43:27.495 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.497 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/82': No such file or directory 2024-03-04T07:43:27.498 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/83 2024-03-04T07:43:27.498 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.499 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/83': No such file or directory 2024-03-04T07:43:27.500 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/84 2024-03-04T07:43:27.500 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.501 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/84': No such file or directory 2024-03-04T07:43:27.502 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/85 2024-03-04T07:43:27.502 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.503 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/85': No such file or directory 2024-03-04T07:43:27.505 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/86 2024-03-04T07:43:27.505 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.505 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/86': No such file or directory 2024-03-04T07:43:27.507 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/87 2024-03-04T07:43:27.507 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.507 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/87': No such file or directory 2024-03-04T07:43:27.509 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/88 2024-03-04T07:43:27.509 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.510 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/88': No such file or directory 2024-03-04T07:43:27.511 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/89 2024-03-04T07:43:27.511 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.512 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/89': No such file or directory 2024-03-04T07:43:27.513 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/90 2024-03-04T07:43:27.513 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.514 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/90': No such file or directory 2024-03-04T07:43:27.515 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/91 2024-03-04T07:43:27.515 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.516 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/91': No such file or directory 2024-03-04T07:43:27.517 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/92 2024-03-04T07:43:27.517 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.519 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/92': No such file or directory 2024-03-04T07:43:27.520 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/93 2024-03-04T07:43:27.520 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.521 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/93': No such file or directory 2024-03-04T07:43:27.522 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/94 2024-03-04T07:43:27.522 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.523 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/94': No such file or directory 2024-03-04T07:43:27.524 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/95 2024-03-04T07:43:27.525 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.526 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/95': No such file or directory 2024-03-04T07:43:27.527 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/96 2024-03-04T07:43:27.527 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.529 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/96': No such file or directory 2024-03-04T07:43:27.530 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/97 2024-03-04T07:43:27.530 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.531 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/97': No such file or directory 2024-03-04T07:43:27.532 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/98 2024-03-04T07:43:27.532 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.534 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/98': No such file or directory 2024-03-04T07:43:27.534 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/99 2024-03-04T07:43:27.534 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.535 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/99': No such file or directory 2024-03-04T07:43:27.536 INFO:tasks.workunit.client.0.smithi113.stderr:+ stat .snap/foo/100 2024-03-04T07:43:27.536 INFO:tasks.workunit.client.0.smithi113.stderr:+ grep No such file 2024-03-04T07:43:27.537 INFO:tasks.workunit.client.0.smithi113.stdout:stat: cannot statx '.snap/foo/100': No such file or directory 2024-03-04T07:43:27.538 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/foo 2024-03-04T07:43:27.549 INFO:tasks.workunit.client.0.smithi113.stdout:OK 2024-03-04T07:43:27.550 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo OK 2024-03-04T07:43:27.550 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T07:43:27.550 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T07:43:27.616 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2024-03-04T07:43:27.617 DEBUG:teuthology.orchestra.run.smithi113:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2024-03-04T07:43:27.684 INFO:tasks.workunit.client.0.smithi113.stderr:+ set -e 2024-03-04T07:43:27.684 INFO:tasks.workunit.client.0.smithi113.stderr:+ sudo git config --global http.sslVerify false 2024-03-04T07:43:27.700 INFO:tasks.workunit.client.0.smithi113.stderr:+ sudo git config --global http.postBuffer 1048576000 2024-03-04T07:43:27.714 INFO:tasks.workunit.client.0.smithi113.stderr:+ retried=false 2024-03-04T07:43:27.714 INFO:tasks.workunit.client.0.smithi113.stderr:+ trap -- retry EXIT 2024-03-04T07:43:27.714 INFO:tasks.workunit.client.0.smithi113.stderr:+ rm -rf ceph 2024-03-04T07:43:27.715 INFO:tasks.workunit.client.0.smithi113.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2024-03-04T07:43:27.718 INFO:tasks.workunit.client.0.smithi113.stderr:Cloning into 'ceph'... 2024-03-04T07:43:56.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:43:56.794 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:44:04.260 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:04.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd9c00640 1 Processor -- start 2024-03-04T07:44:04.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd9c00640 1 -- start start 2024-03-04T07:44:04.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd9c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8dd40c22e0 0x7f8dd415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:04.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd9c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 0x7f8dd415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:04.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd9c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd40c27f0 con 0x7f8dd414e4c0 2024-03-04T07:44:04.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd9c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd415ad60 con 0x7f8dd40c22e0 2024-03-04T07:44:04.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd9c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40c2a90 con 0x7f8dd40c2d80 2024-03-04T07:44:04.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8dd40c22e0 0x7f8dd415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:04.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8dd40c22e0 0x7f8dd415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54122/0 (socket says 172.21.15.113:54122) 2024-03-04T07:44:04.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd2000640 1 -- 172.21.15.113:0/235634049 learned_addr learned my addr 172.21.15.113:0/235634049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:04.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd2a00640 1 --1- 172.21.15.113:0/235634049 >> v1:172.21.15.136:6789/0 conn(0x7f8dd414e4c0 0x7f8dd4072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:38716/0 (socket says 172.21.15.113:38716) 2024-03-04T07:44:04.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd3400640 1 --2- 172.21.15.113:0/235634049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 0x7f8dd415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:04.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd2000640 1 -- 172.21.15.113:0/235634049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 msgr2=0x7f8dd415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:04.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd2000640 1 --2- 172.21.15.113:0/235634049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 0x7f8dd415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:04.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd2000640 1 -- 172.21.15.113:0/235634049 >> v1:172.21.15.136:6789/0 conn(0x7f8dd414e4c0 legacy=0x7f8dd4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:04.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.522+0000 7f8dd2000640 1 -- 172.21.15.113:0/235634049 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd4070c60 con 0x7f8dd40c22e0 2024-03-04T07:44:04.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd2000640 1 --2- 172.21.15.113:0/235634049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd40c22e0 0x7f8dd415b1f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009870 tx=0x7f8dc001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3699f55e1110c27a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:04.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd1600640 1 -- 172.21.15.113:0/235634049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00b5070 con 0x7f8dd40c22e0 2024-03-04T07:44:04.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd1600640 1 -- 172.21.15.113:0/235634049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc00a8a50 con 0x7f8dd40c22e0 2024-03-04T07:44:04.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd1600640 1 -- 172.21.15.113:0/235634049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00a8e20 con 0x7f8dd40c22e0 2024-03-04T07:44:04.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd9c00640 1 -- 172.21.15.113:0/235634049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd40c22e0 msgr2=0x7f8dd415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:04.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd9c00640 1 --2- 172.21.15.113:0/235634049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd40c22e0 0x7f8dd415b1f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0009870 tx=0x7f8dc001ed50 comp rx=0 tx=0).stop 2024-03-04T07:44:04.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd9c00640 1 -- 172.21.15.113:0/235634049 shutdown_connections 2024-03-04T07:44:04.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd9c00640 1 --2- 172.21.15.113:0/235634049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 0x7f8dd415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:04.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd9c00640 1 --2- 172.21.15.113:0/235634049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd40c22e0 0x7f8dd415b1f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:04.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd9c00640 1 -- 172.21.15.113:0/235634049 >> 172.21.15.113:0/235634049 conn(0x7f8dd406e820 msgr2=0x7f8dd406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:04.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd9c00640 1 -- 172.21.15.113:0/235634049 shutdown_connections 2024-03-04T07:44:04.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.530+0000 7f8dd9c00640 1 -- 172.21.15.113:0/235634049 wait complete. 2024-03-04T07:44:04.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.534+0000 7f8dd9c00640 1 Processor -- start 2024-03-04T07:44:04.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.534+0000 7f8dd9c00640 1 -- start start 2024-03-04T07:44:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.534+0000 7f8dd9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd40c22e0 0x7f8dd41582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.534+0000 7f8dd9c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 0x7f8dd41530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.534+0000 7f8dd9c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd414e4c0 0x7f8dd40b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd40c22e0 0x7f8dd41582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd9c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4071500 con 0x7f8dd414e4c0 2024-03-04T07:44:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd9c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd4071220 con 0x7f8dd40c22e0 2024-03-04T07:44:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd9c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd414e8a0 con 0x7f8dd40c2d80 2024-03-04T07:44:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd40c22e0 0x7f8dd41582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:36618/0 (socket says 172.21.15.113:36618) 2024-03-04T07:44:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd2a00640 1 -- 172.21.15.113:0/2010669056 learned_addr learned my addr 172.21.15.113:0/2010669056 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd2000640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 0x7f8dd41530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd2000640 1 -- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd40c22e0 msgr2=0x7f8dd41582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd2000640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd40c22e0 0x7f8dd41582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd2000640 1 -- 172.21.15.113:0/2010669056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd414e4c0 msgr2=0x7f8dd40b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:04.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd2000640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd414e4c0 0x7f8dd40b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:04.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd2000640 1 -- 172.21.15.113:0/2010669056 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40b2090 con 0x7f8dd40c2d80 2024-03-04T07:44:04.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.538+0000 7f8dd2000640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 0x7f8dd41530b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00af000 tx=0x7f8dc001baf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:04.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.542+0000 7f8dcaa00640 1 -- 172.21.15.113:0/2010669056 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00b5070 con 0x7f8dd40c2d80 2024-03-04T07:44:04.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.542+0000 7f8dcaa00640 1 -- 172.21.15.113:0/2010669056 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc00bb410 con 0x7f8dd40c2d80 2024-03-04T07:44:04.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.542+0000 7f8dcaa00640 1 -- 172.21.15.113:0/2010669056 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00bb820 con 0x7f8dd40c2d80 2024-03-04T07:44:04.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.542+0000 7f8dd9c00640 1 -- 172.21.15.113:0/2010669056 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd4156b20 con 0x7f8dd40c2d80 2024-03-04T07:44:04.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.542+0000 7f8dd9c00640 1 -- 172.21.15.113:0/2010669056 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd40af220 con 0x7f8dd40c2d80 2024-03-04T07:44:04.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.546+0000 7f8dcaa00640 1 -- 172.21.15.113:0/2010669056 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8dc00b9070 con 0x7f8dd40c2d80 2024-03-04T07:44:04.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.546+0000 7f8dd9c00640 1 -- 172.21.15.113:0/2010669056 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd41578c0 con 0x7f8dd40c2d80 2024-03-04T07:44:04.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.550+0000 7f8dcaa00640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8db4067030 0x7f8db40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:04.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.550+0000 7f8dcaa00640 1 -- 172.21.15.113:0/2010669056 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6283+0+0 (secure 0 0 0) 0x7f8dc0122880 con 0x7f8dd40c2d80 2024-03-04T07:44:04.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.550+0000 7f8dd2a00640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8db4067030 0x7f8db40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:04.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.550+0000 7f8dd2a00640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8db4067030 0x7f8db40694f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4007fd0 tx=0x7f8dc4007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:04.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.554+0000 7f8dcaa00640 1 -- 172.21.15.113:0/2010669056 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8dc00188b0 con 0x7f8dd40c2d80 2024-03-04T07:44:04.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.934+0000 7f8dd9c00640 1 -- 172.21.15.113:0/2010669056 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dd4075c00 con 0x7f8dd40c2d80 2024-03-04T07:44:04.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.934+0000 7f8dcaa00640 1 -- 172.21.15.113:0/2010669056 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f8dc00f36c0 con 0x7f8dd40c2d80 2024-03-04T07:44:04.934 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:04.935 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:42:01.001453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099,5094]},"id":1}]} 2024-03-04T07:44:04.935 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 -- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8db4067030 msgr2=0x7f8db40694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8db4067030 0x7f8db40694f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4007fd0 tx=0x7f8dc4007d60 comp rx=0 tx=0).stop 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 -- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 msgr2=0x7f8dd41530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 0x7f8dd41530b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00af000 tx=0x7f8dc001baf0 comp rx=0 tx=0).stop 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 -- 172.21.15.113:0/2010669056 shutdown_connections 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8db4067030 0x7f8db40694f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8dd414e4c0 0x7f8dd40b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8dd40c2d80 0x7f8dd41530b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 --2- 172.21.15.113:0/2010669056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8dd40c22e0 0x7f8dd41582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 -- 172.21.15.113:0/2010669056 >> 172.21.15.113:0/2010669056 conn(0x7f8dd406e820 msgr2=0x7f8dd40cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 -- 172.21.15.113:0/2010669056 shutdown_connections 2024-03-04T07:44:04.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:04.938+0000 7f8dd9c00640 1 -- 172.21.15.113:0/2010669056 wait complete. 2024-03-04T07:44:04.961 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=1) 2024-03-04T07:44:04.962 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-04T07:44:04.962 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:44:04.981 INFO:tasks.ceph.mds.c:Stopped 2024-03-04T07:44:04.981 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:05.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facf7800640 1 Processor -- start 2024-03-04T07:44:05.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facf7800640 1 -- start start 2024-03-04T07:44:05.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facf7800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face80062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:05.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facf7800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face8007130 con 0x7face80dd5a0 2024-03-04T07:44:05.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facf7800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face80dd980 con 0x7face80e3cd0 2024-03-04T07:44:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facf7800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face80073d0 con 0x7face8008190 2024-03-04T07:44:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facf5800640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7face80e3cd0 0x7face8046370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:38720/0 (socket says 172.21.15.113:38720) 2024-03-04T07:44:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facf4e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7face80dd5a0 0x7face804b3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35634/0 (socket says 172.21.15.113:35634) 2024-03-04T07:44:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facf5800640 1 -- 172.21.15.113:0/458619042 learned_addr learned my addr 172.21.15.113:0/458619042 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:05.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facefe00640 1 --2- 172.21.15.113:0/458619042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face80062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facef400640 1 -- 172.21.15.113:0/458619042 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722260269 0 0) 0x7face80dd980 con 0x7face80e3cd0 2024-03-04T07:44:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facef400640 1 -- 172.21.15.113:0/458619042 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd8003530 con 0x7face80e3cd0 2024-03-04T07:44:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.266+0000 7facef400640 1 -- 172.21.15.113:0/458619042 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291453445 0 0) 0x7face8007130 con 0x7face80dd5a0 2024-03-04T07:44:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.270+0000 7facef400640 1 -- 172.21.15.113:0/458619042 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facd80048f0 con 0x7face80dd5a0 2024-03-04T07:44:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.270+0000 7facefe00640 1 -- 172.21.15.113:0/458619042 >> v1:172.21.15.136:6789/0 conn(0x7face80e3cd0 legacy=0x7face8046370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:05.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.270+0000 7facefe00640 1 -- 172.21.15.113:0/458619042 >> v1:172.21.15.113:6789/0 conn(0x7face80dd5a0 legacy=0x7face804b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:05.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.270+0000 7facefe00640 1 -- 172.21.15.113:0/458619042 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face80076b0 con 0x7face8008190 2024-03-04T07:44:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facefe00640 1 --2- 172.21.15.113:0/458619042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face80062a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097f0 tx=0x7facdc01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6607147f1d8a9a9c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facef400640 1 -- 172.21.15.113:0/458619042 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc0aa610 con 0x7face8008190 2024-03-04T07:44:05.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facef400640 1 -- 172.21.15.113:0/458619042 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc0af070 con 0x7face8008190 2024-03-04T07:44:05.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facef400640 1 -- 172.21.15.113:0/458619042 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc0aa9d0 con 0x7face8008190 2024-03-04T07:44:05.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facf7800640 1 -- 172.21.15.113:0/458619042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 msgr2=0x7face80062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:05.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facf7800640 1 --2- 172.21.15.113:0/458619042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face80062a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097f0 tx=0x7facdc01c4b0 comp rx=0 tx=0).stop 2024-03-04T07:44:05.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facf7800640 1 -- 172.21.15.113:0/458619042 shutdown_connections 2024-03-04T07:44:05.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facf7800640 1 --2- 172.21.15.113:0/458619042 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face80062a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:05.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facf7800640 1 -- 172.21.15.113:0/458619042 >> 172.21.15.113:0/458619042 conn(0x7face8050a50 msgr2=0x7face8050e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:05.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facf7800640 1 -- 172.21.15.113:0/458619042 shutdown_connections 2024-03-04T07:44:05.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.274+0000 7facf7800640 1 -- 172.21.15.113:0/458619042 wait complete. 2024-03-04T07:44:05.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.278+0000 7facf7800640 1 Processor -- start 2024-03-04T07:44:05.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.278+0000 7facf7800640 1 -- start start 2024-03-04T07:44:05.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.278+0000 7facf7800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face8047dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:05.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.278+0000 7facf7800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face80dd5a0 0x7face80e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:05.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.278+0000 7facf7800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face80e3cd0 0x7face8048930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:05.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.278+0000 7facf7800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face8046a80 con 0x7face80dd5a0 2024-03-04T07:44:05.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.278+0000 7facf7800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80e40e0 con 0x7face80e3cd0 2024-03-04T07:44:05.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.278+0000 7facf7800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face8046c00 con 0x7face8008190 2024-03-04T07:44:05.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.282+0000 7facf4e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face8047dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:05.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.282+0000 7facf4e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face8047dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:38612/0 (socket says 172.21.15.113:38612) 2024-03-04T07:44:05.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.282+0000 7facf4e00640 1 -- 172.21.15.113:0/1039332333 learned_addr learned my addr 172.21.15.113:0/1039332333 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:05.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.282+0000 7facefe00640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face80dd5a0 0x7face80e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:05.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.282+0000 7facf4e00640 1 -- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face80e3cd0 msgr2=0x7face8048930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:05.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.282+0000 7facf4e00640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face80e3cd0 0x7face8048930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:05.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.282+0000 7facf4e00640 1 -- 172.21.15.113:0/1039332333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face80dd5a0 msgr2=0x7face80e1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:05.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.282+0000 7facf4e00640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face80dd5a0 0x7face80e1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:05.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.282+0000 7facf4e00640 1 -- 172.21.15.113:0/1039332333 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face8180300 con 0x7face8008190 2024-03-04T07:44:05.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.286+0000 7facf4e00640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face8047dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7face400c530 tx=0x7face400c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:05.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.286+0000 7facecc00640 1 -- 172.21.15.113:0/1039332333 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4018070 con 0x7face8008190 2024-03-04T07:44:05.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.286+0000 7facecc00640 1 -- 172.21.15.113:0/1039332333 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4007480 con 0x7face8008190 2024-03-04T07:44:05.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.286+0000 7facecc00640 1 -- 172.21.15.113:0/1039332333 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40047f0 con 0x7face8008190 2024-03-04T07:44:05.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.286+0000 7facf7800640 1 -- 172.21.15.113:0/1039332333 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face8036620 con 0x7face8008190 2024-03-04T07:44:05.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.286+0000 7facf7800640 1 -- 172.21.15.113:0/1039332333 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face80343f0 con 0x7face8008190 2024-03-04T07:44:05.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.286+0000 7facf7800640 1 -- 172.21.15.113:0/1039332333 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face80053d0 con 0x7face8008190 2024-03-04T07:44:05.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.290+0000 7facecc00640 1 -- 172.21.15.113:0/1039332333 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7face4004990 con 0x7face8008190 2024-03-04T07:44:05.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.290+0000 7facecc00640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facd0066f90 0x7facd0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:05.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.290+0000 7facecc00640 1 -- 172.21.15.113:0/1039332333 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6283+0+0 (secure 0 0 0) 0x7face4084580 con 0x7face8008190 2024-03-04T07:44:05.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.290+0000 7facefe00640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facd0066f90 0x7facd0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:05.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.294+0000 7facefe00640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facd0066f90 0x7facd0069450 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7facdc0ae000 tx=0x7facdc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:05.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.298+0000 7facecc00640 1 -- 172.21.15.113:0/1039332333 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7face4054e70 con 0x7face8008190 2024-03-04T07:44:05.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.538+0000 7facf7800640 1 -- 172.21.15.113:0/1039332333 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7face8006e70 con 0x7face8008190 2024-03-04T07:44:05.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.542+0000 7facecc00640 1 -- 172.21.15.113:0/1039332333 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6612 (secure 0 0 0) 0x7face4058f10 con 0x7face8008190 2024-03-04T07:44:05.545 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:05.545 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:42:01.001453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099,5094]},"id":1}]} 2024-03-04T07:44:05.545 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-03-04T07:44:05.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.550+0000 7facf7800640 1 -- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facd0066f90 msgr2=0x7facd0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:05.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.550+0000 7facf7800640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facd0066f90 0x7facd0069450 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7facdc0ae000 tx=0x7facdc0a7000 comp rx=0 tx=0).stop 2024-03-04T07:44:05.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.550+0000 7facf7800640 1 -- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 msgr2=0x7face8047dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:05.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.550+0000 7facf7800640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face8047dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7face400c530 tx=0x7face400c9f0 comp rx=0 tx=0).stop 2024-03-04T07:44:05.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.554+0000 7facf7800640 1 -- 172.21.15.113:0/1039332333 shutdown_connections 2024-03-04T07:44:05.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.554+0000 7facf7800640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7face80e3cd0 0x7face8048930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:05.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.554+0000 7facf7800640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facd0066f90 0x7facd0069450 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:05.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.554+0000 7facf7800640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face80dd5a0 0x7face80e1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:05.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.554+0000 7facf7800640 1 --2- 172.21.15.113:0/1039332333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7face8008190 0x7face8047dd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:05.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.554+0000 7facf7800640 1 -- 172.21.15.113:0/1039332333 >> 172.21.15.113:0/1039332333 conn(0x7face8050a50 msgr2=0x7face804fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:05.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.554+0000 7facf7800640 1 -- 172.21.15.113:0/1039332333 shutdown_connections 2024-03-04T07:44:05.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:05.554+0000 7facf7800640 1 -- 172.21.15.113:0/1039332333 wait complete. 2024-03-04T07:44:05.582 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:44:07.583 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:07.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.746+0000 7f9692c00640 1 Processor -- start 2024-03-04T07:44:07.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.746+0000 7f9692c00640 1 -- start start 2024-03-04T07:44:07.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.746+0000 7f9692c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f968c0c22e0 0x7f968c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:07.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f9692c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c0c27f0 con 0x7f968c0c2d80 2024-03-04T07:44:07.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f9692c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c0c2a90 con 0x7f968c14e4c0 2024-03-04T07:44:07.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f9692c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c15ad60 con 0x7f968c0c22e0 2024-03-04T07:44:07.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f968b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f968c0c22e0 0x7f968c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:07.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f968b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f968c0c22e0 0x7f968c15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34310/0 (socket says 172.21.15.113:34310) 2024-03-04T07:44:07.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f968b400640 1 -- 172.21.15.113:0/1499882816 learned_addr learned my addr 172.21.15.113:0/1499882816 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:07.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f968b400640 1 -- 172.21.15.113:0/1499882816 >> v1:172.21.15.136:6789/0 conn(0x7f968c14e4c0 legacy=0x7f968c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:07.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f968b400640 1 -- 172.21.15.113:0/1499882816 >> v1:172.21.15.113:6789/0 conn(0x7f968c0c2d80 legacy=0x7f968c1620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:07.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f968b400640 1 -- 172.21.15.113:0/1499882816 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c070c60 con 0x7f968c0c22e0 2024-03-04T07:44:07.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f968b400640 1 --2- 172.21.15.113:0/1499882816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c22e0 0x7f968c15b1f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9678009870 tx=0x7f967801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24213ba1731f898b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:07.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f968aa00640 1 -- 172.21.15.113:0/1499882816 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780ab650 con 0x7f968c0c22e0 2024-03-04T07:44:07.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f968aa00640 1 -- 172.21.15.113:0/1499882816 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96780b0070 con 0x7f968c0c22e0 2024-03-04T07:44:07.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.750+0000 7f968aa00640 1 -- 172.21.15.113:0/1499882816 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780aba10 con 0x7f968c0c22e0 2024-03-04T07:44:07.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.754+0000 7f9692c00640 1 -- 172.21.15.113:0/1499882816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c22e0 msgr2=0x7f968c15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:07.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.754+0000 7f9692c00640 1 --2- 172.21.15.113:0/1499882816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c22e0 0x7f968c15b1f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9678009870 tx=0x7f967801c560 comp rx=0 tx=0).stop 2024-03-04T07:44:07.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.754+0000 7f9692c00640 1 -- 172.21.15.113:0/1499882816 shutdown_connections 2024-03-04T07:44:07.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.754+0000 7f9692c00640 1 --2- 172.21.15.113:0/1499882816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c22e0 0x7f968c15b1f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:07.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.754+0000 7f9692c00640 1 -- 172.21.15.113:0/1499882816 >> 172.21.15.113:0/1499882816 conn(0x7f968c06e820 msgr2=0x7f968c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:07.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.754+0000 7f9692c00640 1 -- 172.21.15.113:0/1499882816 shutdown_connections 2024-03-04T07:44:07.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.754+0000 7f9692c00640 1 -- 172.21.15.113:0/1499882816 wait complete. 2024-03-04T07:44:07.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.754+0000 7f9692c00640 1 Processor -- start 2024-03-04T07:44:07.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.754+0000 7f9692c00640 1 -- start start 2024-03-04T07:44:07.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9692c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f968c0c22e0 0x7f968c143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:07.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9692c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c2d80 0x7f968c1f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:07.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9692c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f968c14e4c0 0x7f968c1543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:07.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9692c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c14e8a0 con 0x7f968c0c2d80 2024-03-04T07:44:07.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9692c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c15bb40 con 0x7f968c14e4c0 2024-03-04T07:44:07.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9692c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c0aede0 con 0x7f968c0c22e0 2024-03-04T07:44:07.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f968b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c2d80 0x7f968c1f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:07.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f968be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f968c0c22e0 0x7f968c143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:07.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f968b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c2d80 0x7f968c1f42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34316/0 (socket says 172.21.15.113:34316) 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f968b400640 1 -- 172.21.15.113:0/1894983639 learned_addr learned my addr 172.21.15.113:0/1894983639 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9690c00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f968c14e4c0 0x7f968c1543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f968b400640 1 -- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f968c0c22e0 msgr2=0x7f968c143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f968b400640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f968c0c22e0 0x7f968c143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f968b400640 1 -- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f968c14e4c0 msgr2=0x7f968c1543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f968b400640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f968c14e4c0 0x7f968c1543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f968b400640 1 -- 172.21.15.113:0/1894983639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c06ec00 con 0x7f968c0c2d80 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9690c00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f968c14e4c0 0x7f968c1543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f968b400640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c2d80 0x7f968c1f42e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9678019170 tx=0x7f967801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9683e00640 1 -- 172.21.15.113:0/1894983639 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780abe60 con 0x7f968c0c2d80 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9683e00640 1 -- 172.21.15.113:0/1894983639 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96780b4800 con 0x7f968c0c2d80 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9683e00640 1 -- 172.21.15.113:0/1894983639 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780b4bf0 con 0x7f968c0c2d80 2024-03-04T07:44:07.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9692c00640 1 -- 172.21.15.113:0/1894983639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c159a40 con 0x7f968c0c2d80 2024-03-04T07:44:07.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9692c00640 1 -- 172.21.15.113:0/1894983639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c1f7d80 con 0x7f968c0c2d80 2024-03-04T07:44:07.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.758+0000 7f9692c00640 1 -- 172.21.15.113:0/1894983639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c075c00 con 0x7f968c0c2d80 2024-03-04T07:44:07.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.762+0000 7f9683e00640 1 -- 172.21.15.113:0/1894983639 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96780c7440 con 0x7f968c0c2d80 2024-03-04T07:44:07.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.762+0000 7f9683e00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f966c066f60 0x7f966c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:07.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.762+0000 7f9683e00640 1 -- 172.21.15.113:0/1894983639 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6283+0+0 (secure 0 0 0) 0x7f967811fb70 con 0x7f968c0c2d80 2024-03-04T07:44:07.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.762+0000 7f968be00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f966c066f60 0x7f966c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:07.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.762+0000 7f968be00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f966c066f60 0x7f966c069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f967c007fd0 tx=0x7f967c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:07.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:07.766+0000 7f9683e00640 1 -- 172.21.15.113:0/1894983639 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9678016150 con 0x7f968c0c2d80 2024-03-04T07:44:08.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 -- 172.21.15.113:0/1894983639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f968c0c8ec0 con 0x7f968c0c2d80 2024-03-04T07:44:08.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9683e00640 1 -- 172.21.15.113:0/1894983639 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f96780f0380 con 0x7f968c0c2d80 2024-03-04T07:44:08.001 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:08.001 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:42:01.001453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099,5094]},"id":1}]} 2024-03-04T07:44:08.001 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-03-04T07:44:08.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 -- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f966c066f60 msgr2=0x7f966c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:08.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f966c066f60 0x7f966c069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f967c007fd0 tx=0x7f967c009040 comp rx=0 tx=0).stop 2024-03-04T07:44:08.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 -- 172.21.15.113:0/1894983639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c2d80 msgr2=0x7f968c1f42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:08.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c2d80 0x7f968c1f42e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9678019170 tx=0x7f967801ada0 comp rx=0 tx=0).stop 2024-03-04T07:44:08.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 -- 172.21.15.113:0/1894983639 shutdown_connections 2024-03-04T07:44:08.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f968c14e4c0 0x7f968c1543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:08.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f966c066f60 0x7f966c069420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:08.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f968c0c2d80 0x7f968c1f42e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:08.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 --2- 172.21.15.113:0/1894983639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f968c0c22e0 0x7f968c143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:08.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 -- 172.21.15.113:0/1894983639 >> 172.21.15.113:0/1894983639 conn(0x7f968c06e820 msgr2=0x7f968c14d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:08.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 -- 172.21.15.113:0/1894983639 shutdown_connections 2024-03-04T07:44:08.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:08.002+0000 7f9692c00640 1 -- 172.21.15.113:0/1894983639 wait complete. 2024-03-04T07:44:08.025 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:44:10.026 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:10.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.174+0000 7f63ad800640 1 Processor -- start 2024-03-04T07:44:10.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63ad800640 1 -- start start 2024-03-04T07:44:10.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63ad800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f63a8070df0 0x7f63a80c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:10.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63ad800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 0x7f63a80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:10.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63ad800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a80c9c50 con 0x7f63a8157ef0 2024-03-04T07:44:10.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63ad800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a815a9a0 con 0x7f63a8070df0 2024-03-04T07:44:10.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63ad800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a806de30 con 0x7f63a806fb70 2024-03-04T07:44:10.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63a7400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f63a8070df0 0x7f63a80c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:10.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63a7e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f63a8157ef0 0x7f63a815b180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:51610/0 (socket says 172.21.15.113:51610) 2024-03-04T07:44:10.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63a7e00640 1 -- 172.21.15.113:0/3296990819 learned_addr learned my addr 172.21.15.113:0/3296990819 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:10.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63ace00640 1 --2- 172.21.15.113:0/3296990819 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 0x7f63a80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:10.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63a7400640 1 -- 172.21.15.113:0/3296990819 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 msgr2=0x7f63a80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:10.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63a7400640 1 --2- 172.21.15.113:0/3296990819 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 0x7f63a80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63a7400640 1 -- 172.21.15.113:0/3296990819 >> v1:172.21.15.136:6789/0 conn(0x7f63a8157ef0 legacy=0x7f63a815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:10.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63a7400640 1 -- 172.21.15.113:0/3296990819 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a806c210 con 0x7f63a8070df0 2024-03-04T07:44:10.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63a6a00640 1 -- 172.21.15.113:0/3296990819 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348369011 0 0) 0x7f63a80c9c50 con 0x7f63a8157ef0 2024-03-04T07:44:10.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.178+0000 7f63ace00640 1 --2- 172.21.15.113:0/3296990819 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 0x7f63a80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:44:10.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63a7400640 1 --2- 172.21.15.113:0/3296990819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a8070df0 0x7f63a80c8a90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f63940097c0 tx=0x7f639401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b89763fcd4052fe server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:10.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63a6a00640 1 -- 172.21.15.113:0/3296990819 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940b1070 con 0x7f63a8070df0 2024-03-04T07:44:10.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63a6a00640 1 -- 172.21.15.113:0/3296990819 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63940ac410 con 0x7f63a8070df0 2024-03-04T07:44:10.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63a6a00640 1 -- 172.21.15.113:0/3296990819 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940ac7e0 con 0x7f63a8070df0 2024-03-04T07:44:10.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63ad800640 1 -- 172.21.15.113:0/3296990819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a8070df0 msgr2=0x7f63a80c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:10.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63ad800640 1 --2- 172.21.15.113:0/3296990819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a8070df0 0x7f63a80c8a90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f63940097c0 tx=0x7f639401c4a0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63ad800640 1 -- 172.21.15.113:0/3296990819 shutdown_connections 2024-03-04T07:44:10.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63ad800640 1 --2- 172.21.15.113:0/3296990819 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 0x7f63a80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63ad800640 1 --2- 172.21.15.113:0/3296990819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a8070df0 0x7f63a80c8a90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63ad800640 1 -- 172.21.15.113:0/3296990819 >> 172.21.15.113:0/3296990819 conn(0x7f63a8072f40 msgr2=0x7f63a80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:10.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63ad800640 1 -- 172.21.15.113:0/3296990819 shutdown_connections 2024-03-04T07:44:10.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.182+0000 7f63ad800640 1 -- 172.21.15.113:0/3296990819 wait complete. 2024-03-04T07:44:10.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ad800640 1 Processor -- start 2024-03-04T07:44:10.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ad800640 1 -- start start 2024-03-04T07:44:10.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ad800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 0x7f63a8150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:10.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ad800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a8070df0 0x7f63a8151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:10.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ad800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a8157ef0 0x7f63a8153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:10.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ad800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a81582d0 con 0x7f63a8157ef0 2024-03-04T07:44:10.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ad800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a815b890 con 0x7f63a8070df0 2024-03-04T07:44:10.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ad800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a8158450 con 0x7f63a806fb70 2024-03-04T07:44:10.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ace00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a8157ef0 0x7f63a8153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:10.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ace00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a8157ef0 0x7f63a8153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34330/0 (socket says 172.21.15.113:34330) 2024-03-04T07:44:10.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63a7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 0x7f63a8150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:10.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ace00640 1 -- 172.21.15.113:0/1616846742 learned_addr learned my addr 172.21.15.113:0/1616846742 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:10.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63a7400640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a8070df0 0x7f63a8151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:10.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ace00640 1 -- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 msgr2=0x7f63a8150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:10.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ace00640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 0x7f63a8150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ace00640 1 -- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a8070df0 msgr2=0x7f63a8151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:10.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ace00640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a8070df0 0x7f63a8151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ace00640 1 -- 172.21.15.113:0/1616846742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a8151f80 con 0x7f63a8157ef0 2024-03-04T07:44:10.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ace00640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a8157ef0 0x7f63a8153590 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f63a000efc0 tx=0x7f63a0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:10.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f639fe00640 1 -- 172.21.15.113:0/1616846742 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0018030 con 0x7f63a8157ef0 2024-03-04T07:44:10.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f639fe00640 1 -- 172.21.15.113:0/1616846742 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a000cde0 con 0x7f63a8157ef0 2024-03-04T07:44:10.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ad800640 1 -- 172.21.15.113:0/1616846742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a8152ab0 con 0x7f63a8157ef0 2024-03-04T07:44:10.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f63ad800640 1 -- 172.21.15.113:0/1616846742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a81573c0 con 0x7f63a8157ef0 2024-03-04T07:44:10.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.186+0000 7f639fe00640 1 -- 172.21.15.113:0/1616846742 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0003bc0 con 0x7f63a8157ef0 2024-03-04T07:44:10.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.190+0000 7f63ad800640 1 -- 172.21.15.113:0/1616846742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a8075a80 con 0x7f63a8157ef0 2024-03-04T07:44:10.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.190+0000 7f639fe00640 1 -- 172.21.15.113:0/1616846742 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63a001b070 con 0x7f63a8157ef0 2024-03-04T07:44:10.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.190+0000 7f639fe00640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6388066f60 0x7f6388069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:10.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.190+0000 7f639fe00640 1 -- 172.21.15.113:0/1616846742 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6283+0+0 (secure 0 0 0) 0x7f63a0085bd0 con 0x7f63a8157ef0 2024-03-04T07:44:10.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.190+0000 7f63a7e00640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6388066f60 0x7f6388069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:10.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.194+0000 7f63a7e00640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6388066f60 0x7f6388069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6398000f70 tx=0x7f6398007df0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:10.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.198+0000 7f639fe00640 1 -- 172.21.15.113:0/1616846742 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f63a0022070 con 0x7f63a8157ef0 2024-03-04T07:44:10.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.426+0000 7f63ad800640 1 -- 172.21.15.113:0/1616846742 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63a8152410 con 0x7f63a8157ef0 2024-03-04T07:44:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.426+0000 7f639fe00640 1 -- 172.21.15.113:0/1616846742 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f63a0010040 con 0x7f63a8157ef0 2024-03-04T07:44:10.428 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:10.428 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:42:01.001453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099,5094]},"id":1}]} 2024-03-04T07:44:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-03-04T07:44:10.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 -- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6388066f60 msgr2=0x7f6388069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:10.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6388066f60 0x7f6388069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6398000f70 tx=0x7f6398007df0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 -- 172.21.15.113:0/1616846742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a8157ef0 msgr2=0x7f63a8153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:10.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a8157ef0 0x7f63a8153590 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f63a000efc0 tx=0x7f63a0009f90 comp rx=0 tx=0).stop 2024-03-04T07:44:10.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 -- 172.21.15.113:0/1616846742 shutdown_connections 2024-03-04T07:44:10.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6388066f60 0x7f6388069420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63a8157ef0 0x7f63a8153590 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63a8070df0 0x7f63a8151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 --2- 172.21.15.113:0/1616846742 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63a806fb70 0x7f63a8150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:10.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 -- 172.21.15.113:0/1616846742 >> 172.21.15.113:0/1616846742 conn(0x7f63a8072f40 msgr2=0x7f63a8158910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:10.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 -- 172.21.15.113:0/1616846742 shutdown_connections 2024-03-04T07:44:10.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:10.430+0000 7f63ad800640 1 -- 172.21.15.113:0/1616846742 wait complete. 2024-03-04T07:44:10.451 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:44:12.451 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.654+0000 7fb759c00640 1 Processor -- start 2024-03-04T07:44:12.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.654+0000 7fb759c00640 1 -- start start 2024-03-04T07:44:12.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb759c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb754155050 0x7fb75415b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:12.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb759c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb7540c77e0 0x7fb75415c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:12.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb759c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb754071a70 con 0x7fb754154c70 2024-03-04T07:44:12.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb759c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb754071d10 con 0x7fb7540c77e0 2024-03-04T07:44:12.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb759c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb754071ff0 con 0x7fb754155050 2024-03-04T07:44:12.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb753400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb7540c77e0 0x7fb75415c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:12.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb752a00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fb754154c70 0x7fb7540cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:51614/0 (socket says 172.21.15.113:51614) 2024-03-04T07:44:12.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb752a00640 1 -- 172.21.15.113:0/3140673663 learned_addr learned my addr 172.21.15.113:0/3140673663 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:12.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb752000640 1 --2- 172.21.15.113:0/3140673663 >> v2:172.21.15.136:3300/0 conn(0x7fb754155050 0x7fb75415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:12.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb753400640 1 -- 172.21.15.113:0/3140673663 >> v2:172.21.15.136:3300/0 conn(0x7fb754155050 msgr2=0x7fb75415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:12.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb753400640 1 --2- 172.21.15.113:0/3140673663 >> v2:172.21.15.136:3300/0 conn(0x7fb754155050 0x7fb75415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:12.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb753400640 1 -- 172.21.15.113:0/3140673663 >> v1:172.21.15.136:6789/0 conn(0x7fb754154c70 legacy=0x7fb7540cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:12.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.658+0000 7fb753400640 1 -- 172.21.15.113:0/3140673663 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7540722d0 con 0x7fb7540c77e0 2024-03-04T07:44:12.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb753400640 1 --2- 172.21.15.113:0/3140673663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 0x7fb75415c1f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009d90 tx=0x7fb74c01c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad5e37395392d3f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:12.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb751600640 1 -- 172.21.15.113:0/3140673663 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0ac070 con 0x7fb7540c77e0 2024-03-04T07:44:12.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb751600640 1 -- 172.21.15.113:0/3140673663 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74c0a6650 con 0x7fb7540c77e0 2024-03-04T07:44:12.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb751600640 1 -- 172.21.15.113:0/3140673663 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0a6a20 con 0x7fb7540c77e0 2024-03-04T07:44:12.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb759c00640 1 -- 172.21.15.113:0/3140673663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 msgr2=0x7fb75415c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:12.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb759c00640 1 --2- 172.21.15.113:0/3140673663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 0x7fb75415c1f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009d90 tx=0x7fb74c01c7c0 comp rx=0 tx=0).stop 2024-03-04T07:44:12.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb759c00640 1 -- 172.21.15.113:0/3140673663 shutdown_connections 2024-03-04T07:44:12.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb759c00640 1 --2- 172.21.15.113:0/3140673663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 0x7fb75415c1f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:12.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb759c00640 1 --2- 172.21.15.113:0/3140673663 >> v2:172.21.15.136:3300/0 conn(0x7fb754155050 0x7fb75415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:12.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb759c00640 1 -- 172.21.15.113:0/3140673663 >> 172.21.15.113:0/3140673663 conn(0x7fb75406ef70 msgr2=0x7fb75406f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:12.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb759c00640 1 -- 172.21.15.113:0/3140673663 shutdown_connections 2024-03-04T07:44:12.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.662+0000 7fb759c00640 1 -- 172.21.15.113:0/3140673663 wait complete. 2024-03-04T07:44:12.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb759c00640 1 Processor -- start 2024-03-04T07:44:12.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb759c00640 1 -- start start 2024-03-04T07:44:12.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb759c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 0x7fb7540c2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:12.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb759c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb754154c70 0x7fb7540c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:12.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb759c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb754155050 0x7fb754150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:12.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb759c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540ccf20 con 0x7fb7540c77e0 2024-03-04T07:44:12.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb759c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75415cb40 con 0x7fb754154c70 2024-03-04T07:44:12.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb759c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540cd0a0 con 0x7fb754155050 2024-03-04T07:44:12.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb752a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 0x7fb7540c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:12.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb752a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 0x7fb7540c2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34336/0 (socket says 172.21.15.113:34336) 2024-03-04T07:44:12.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb752a00640 1 -- 172.21.15.113:0/3809108227 learned_addr learned my addr 172.21.15.113:0/3809108227 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:12.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb753400640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb754155050 0x7fb754150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:12.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb752000640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb754154c70 0x7fb7540c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:12.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb752a00640 1 -- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb754155050 msgr2=0x7fb754150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:12.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb752a00640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb754155050 0x7fb754150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:12.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb752a00640 1 -- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb754154c70 msgr2=0x7fb7540c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:12.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb752a00640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb754154c70 0x7fb7540c2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:12.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb752a00640 1 -- 172.21.15.113:0/3809108227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75406f580 con 0x7fb7540c77e0 2024-03-04T07:44:12.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb752a00640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 0x7fb7540c2430 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb74400c780 tx=0x7fb74400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:12.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb74aa00640 1 -- 172.21.15.113:0/3809108227 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744017070 con 0x7fb7540c77e0 2024-03-04T07:44:12.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb74aa00640 1 -- 172.21.15.113:0/3809108227 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb744005a60 con 0x7fb7540c77e0 2024-03-04T07:44:12.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb74aa00640 1 -- 172.21.15.113:0/3809108227 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744007070 con 0x7fb7540c77e0 2024-03-04T07:44:12.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb759c00640 1 -- 172.21.15.113:0/3809108227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb754159930 con 0x7fb7540c77e0 2024-03-04T07:44:12.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.666+0000 7fb759c00640 1 -- 172.21.15.113:0/3809108227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7540c9390 con 0x7fb7540c77e0 2024-03-04T07:44:12.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.670+0000 7fb759c00640 1 -- 172.21.15.113:0/3809108227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb754075c00 con 0x7fb7540c77e0 2024-03-04T07:44:12.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.670+0000 7fb74aa00640 1 -- 172.21.15.113:0/3809108227 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb74401a070 con 0x7fb7540c77e0 2024-03-04T07:44:12.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.674+0000 7fb74aa00640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb734066f60 0x7fb734069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:12.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.674+0000 7fb74aa00640 1 -- 172.21.15.113:0/3809108227 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6283+0+0 (secure 0 0 0) 0x7fb744084680 con 0x7fb7540c77e0 2024-03-04T07:44:12.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.674+0000 7fb752000640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb734066f60 0x7fb734069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:12.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.674+0000 7fb752000640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb734066f60 0x7fb734069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb75415c600 tx=0x7fb740009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:12.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:12.682+0000 7fb74aa00640 1 -- 172.21.15.113:0/3809108227 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb744054e00 con 0x7fb7540c77e0 2024-03-04T07:44:13.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.006+0000 7fb759c00640 1 -- 172.21.15.113:0/3809108227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7540733d0 con 0x7fb7540c77e0 2024-03-04T07:44:13.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.006+0000 7fb74aa00640 1 -- 172.21.15.113:0/3809108227 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fb744058ea0 con 0x7fb7540c77e0 2024-03-04T07:44:13.007 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:13.008 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:42:01.001453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099,5094]},"id":1}]} 2024-03-04T07:44:13.008 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-03-04T07:44:13.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 -- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb734066f60 msgr2=0x7fb734069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb734066f60 0x7fb734069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb75415c600 tx=0x7fb740009190 comp rx=0 tx=0).stop 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 -- 172.21.15.113:0/3809108227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 msgr2=0x7fb7540c2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 0x7fb7540c2430 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb74400c780 tx=0x7fb74400cc40 comp rx=0 tx=0).stop 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 -- 172.21.15.113:0/3809108227 shutdown_connections 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb754155050 0x7fb754150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb754154c70 0x7fb7540c2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb734066f60 0x7fb734069420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 --2- 172.21.15.113:0/3809108227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7540c77e0 0x7fb7540c2430 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 -- 172.21.15.113:0/3809108227 >> 172.21.15.113:0/3809108227 conn(0x7fb75406ef70 msgr2=0x7fb754159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 -- 172.21.15.113:0/3809108227 shutdown_connections 2024-03-04T07:44:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:13.010+0000 7fb759c00640 1 -- 172.21.15.113:0/3809108227 wait complete. 2024-03-04T07:44:13.038 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:44:15.039 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:15.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.246+0000 7fc3b0400640 1 Processor -- start 2024-03-04T07:44:15.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.246+0000 7fc3b0400640 1 -- start start 2024-03-04T07:44:15.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3b0400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc3a8155100 0x7fc3a80c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:15.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3b0400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a815ad30 0x7fc3a815b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:15.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3b0400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a815b6c0 con 0x7fc3a806b060 2024-03-04T07:44:15.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3b0400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a80c8890 con 0x7fc3a8155100 2024-03-04T07:44:15.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3b0400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a806b4b0 con 0x7fc3a815ad30 2024-03-04T07:44:15.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3ada00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc3a8155100 0x7fc3a80c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:15.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3ae400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a815ad30 0x7fc3a815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:15.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3ad000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc3a806b060 0x7fc3a80cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49918/0 (socket says 172.21.15.113:49918) 2024-03-04T07:44:15.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3ada00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc3a8155100 0x7fc3a80c08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:49422/0 (socket says 172.21.15.113:49422) 2024-03-04T07:44:15.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3ad000640 1 -- 172.21.15.113:0/2866118787 learned_addr learned my addr 172.21.15.113:0/2866118787 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:15.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3a7e00640 1 -- 172.21.15.113:0/2866118787 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795157070 0 0) 0x7fc3a815b6c0 con 0x7fc3a806b060 2024-03-04T07:44:15.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3a7e00640 1 -- 172.21.15.113:0/2866118787 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc390003530 con 0x7fc3a806b060 2024-03-04T07:44:15.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3ada00640 1 -- 172.21.15.113:0/2866118787 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a815ad30 msgr2=0x7fc3a815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:15.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3ada00640 1 --2- 172.21.15.113:0/2866118787 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a815ad30 0x7fc3a815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:15.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3ada00640 1 -- 172.21.15.113:0/2866118787 >> v1:172.21.15.113:6789/0 conn(0x7fc3a806b060 legacy=0x7fc3a80cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:15.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3ada00640 1 -- 172.21.15.113:0/2866118787 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a815efd0 con 0x7fc3a8155100 2024-03-04T07:44:15.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.250+0000 7fc3ae400640 1 --2- 172.21.15.113:0/2866118787 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a815ad30 0x7fc3a815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:44:15.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.254+0000 7fc3ada00640 1 --2- 172.21.15.113:0/2866118787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a8155100 0x7fc3a80c08d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009dc0 tx=0x7fc39c01c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=958a89349e43eb9c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:15.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.254+0000 7fc3a7e00640 1 -- 172.21.15.113:0/2866118787 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0b2070 con 0x7fc3a8155100 2024-03-04T07:44:15.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.254+0000 7fc3a7e00640 1 -- 172.21.15.113:0/2866118787 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c0a5880 con 0x7fc3a8155100 2024-03-04T07:44:15.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.254+0000 7fc3a7e00640 1 -- 172.21.15.113:0/2866118787 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0a5c30 con 0x7fc3a8155100 2024-03-04T07:44:15.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.258+0000 7fc3b0400640 1 -- 172.21.15.113:0/2866118787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a8155100 msgr2=0x7fc3a80c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:15.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.258+0000 7fc3b0400640 1 --2- 172.21.15.113:0/2866118787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a8155100 0x7fc3a80c08d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc39c009dc0 tx=0x7fc39c01c910 comp rx=0 tx=0).stop 2024-03-04T07:44:15.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.258+0000 7fc3b0400640 1 -- 172.21.15.113:0/2866118787 shutdown_connections 2024-03-04T07:44:15.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.258+0000 7fc3b0400640 1 --2- 172.21.15.113:0/2866118787 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a815ad30 0x7fc3a815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:15.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.258+0000 7fc3b0400640 1 --2- 172.21.15.113:0/2866118787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a8155100 0x7fc3a80c08d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:15.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.258+0000 7fc3b0400640 1 -- 172.21.15.113:0/2866118787 >> 172.21.15.113:0/2866118787 conn(0x7fc3a806deb0 msgr2=0x7fc3a806e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:15.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.258+0000 7fc3b0400640 1 -- 172.21.15.113:0/2866118787 shutdown_connections 2024-03-04T07:44:15.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.258+0000 7fc3b0400640 1 -- 172.21.15.113:0/2866118787 wait complete. 2024-03-04T07:44:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.258+0000 7fc3b0400640 1 Processor -- start 2024-03-04T07:44:15.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.258+0000 7fc3b0400640 1 -- start start 2024-03-04T07:44:15.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3b0400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a806b060 0x7fc3a81f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:15.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3b0400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a8155100 0x7fc3a80cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:15.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3b0400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3a815ad30 0x7fc3a814db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:15.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3b0400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a815ebb0 con 0x7fc3a815ad30 2024-03-04T07:44:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3b0400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80c9610 con 0x7fc3a806b060 2024-03-04T07:44:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3b0400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a815ed30 con 0x7fc3a8155100 2024-03-04T07:44:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ae400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3a815ad30 0x7fc3a814db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ad000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a8155100 0x7fc3a80cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ada00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a806b060 0x7fc3a81f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ae400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3a815ad30 0x7fc3a814db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34340/0 (socket says 172.21.15.113:34340) 2024-03-04T07:44:15.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ae400640 1 -- 172.21.15.113:0/3029428678 learned_addr learned my addr 172.21.15.113:0/3029428678 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ae400640 1 -- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a8155100 msgr2=0x7fc3a80cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ae400640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a8155100 0x7fc3a80cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ae400640 1 -- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a806b060 msgr2=0x7fc3a81f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ae400640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a806b060 0x7fc3a81f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ae400640 1 -- 172.21.15.113:0/3029428678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a81541f0 con 0x7fc3a815ad30 2024-03-04T07:44:15.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3ae400640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3a815ad30 0x7fc3a814db10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000e930 tx=0x7fc3a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:15.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3a5600640 1 -- 172.21.15.113:0/3029428678 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0017070 con 0x7fc3a815ad30 2024-03-04T07:44:15.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3b0400640 1 -- 172.21.15.113:0/3029428678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a8154490 con 0x7fc3a815ad30 2024-03-04T07:44:15.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3a5600640 1 -- 172.21.15.113:0/3029428678 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a0004020 con 0x7fc3a815ad30 2024-03-04T07:44:15.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3b0400640 1 -- 172.21.15.113:0/3029428678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a81f4240 con 0x7fc3a815ad30 2024-03-04T07:44:15.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc3a5600640 1 -- 172.21.15.113:0/3029428678 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0004430 con 0x7fc3a815ad30 2024-03-04T07:44:15.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.262+0000 7fc39aa00640 1 -- 172.21.15.113:0/3029428678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a8075c00 con 0x7fc3a815ad30 2024-03-04T07:44:15.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.266+0000 7fc3a5600640 1 -- 172.21.15.113:0/3029428678 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3a000c940 con 0x7fc3a815ad30 2024-03-04T07:44:15.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.266+0000 7fc3a5600640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc388066f90 0x7fc388069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:15.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.266+0000 7fc3a5600640 1 -- 172.21.15.113:0/3029428678 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6283+0+0 (secure 0 0 0) 0x7fc3a0085320 con 0x7fc3a815ad30 2024-03-04T07:44:15.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.270+0000 7fc3ada00640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc388066f90 0x7fc388069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:15.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.270+0000 7fc3ada00640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc388066f90 0x7fc388069450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0af000 tx=0x7fc39c0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:15.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.274+0000 7fc3a5600640 1 -- 172.21.15.113:0/3029428678 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc3a0055a90 con 0x7fc3a815ad30 2024-03-04T07:44:15.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.518+0000 7fc39aa00640 1 -- 172.21.15.113:0/3029428678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3a815b310 con 0x7fc3a815ad30 2024-03-04T07:44:15.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.518+0000 7fc3a5600640 1 -- 172.21.15.113:0/3029428678 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fc3a00850e0 con 0x7fc3a815ad30 2024-03-04T07:44:15.519 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:15.519 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:42:01.001453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099,5094]},"id":1}]} 2024-03-04T07:44:15.520 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-03-04T07:44:15.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 -- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc388066f90 msgr2=0x7fc388069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc388066f90 0x7fc388069450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0af000 tx=0x7fc39c0a7070 comp rx=0 tx=0).stop 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 -- 172.21.15.113:0/3029428678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3a815ad30 msgr2=0x7fc3a814db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3a815ad30 0x7fc3a814db10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000e930 tx=0x7fc3a000edf0 comp rx=0 tx=0).stop 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 -- 172.21.15.113:0/3029428678 shutdown_connections 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc388066f90 0x7fc388069450 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3a815ad30 0x7fc3a814db10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3a8155100 0x7fc3a80cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 --2- 172.21.15.113:0/3029428678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3a806b060 0x7fc3a81f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 -- 172.21.15.113:0/3029428678 >> 172.21.15.113:0/3029428678 conn(0x7fc3a806deb0 msgr2=0x7fc3a8151dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 -- 172.21.15.113:0/3029428678 shutdown_connections 2024-03-04T07:44:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:15.522+0000 7fc39aa00640 1 -- 172.21.15.113:0/3029428678 wait complete. 2024-03-04T07:44:15.543 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:44:17.543 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:17.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.769+0000 7fe29a000640 1 Processor -- start 2024-03-04T07:44:17.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.769+0000 7fe29a000640 1 -- start start 2024-03-04T07:44:17.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe29a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe29406faa0 0x7fe29415f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:17.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe29a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe29406c170 0x7fe2940c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:17.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe29a000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29406dc00 con 0x7fe294158f10 2024-03-04T07:44:17.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe29a000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2941592f0 con 0x7fe29406faa0 2024-03-04T07:44:17.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe29a000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29406e150 con 0x7fe29406c170 2024-03-04T07:44:17.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe298c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe294158f10 0x7fe29406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55114/0 (socket says 172.21.15.113:55114) 2024-03-04T07:44:17.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe298c00640 1 -- 172.21.15.113:0/1546035628 learned_addr learned my addr 172.21.15.113:0/1546035628 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:17.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe299600640 1 --2- 172.21.15.113:0/1546035628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe29406c170 0x7fe2940c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:17.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe293e00640 1 --2- 172.21.15.113:0/1546035628 >> v2:172.21.15.136:3300/0 conn(0x7fe29406faa0 0x7fe29415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:17.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe293400640 1 -- 172.21.15.113:0/1546035628 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826659219 0 0) 0x7fe29406dc00 con 0x7fe294158f10 2024-03-04T07:44:17.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe293400640 1 -- 172.21.15.113:0/1546035628 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe27c003530 con 0x7fe294158f10 2024-03-04T07:44:17.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe293e00640 1 -- 172.21.15.113:0/1546035628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe29406c170 msgr2=0x7fe2940c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:17.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe293e00640 1 --2- 172.21.15.113:0/1546035628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe29406c170 0x7fe2940c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:17.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe293e00640 1 -- 172.21.15.113:0/1546035628 >> v1:172.21.15.113:6789/0 conn(0x7fe294158f10 legacy=0x7fe29406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:17.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.773+0000 7fe293e00640 1 -- 172.21.15.113:0/1546035628 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29406de70 con 0x7fe29406faa0 2024-03-04T07:44:17.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.777+0000 7fe299600640 1 --2- 172.21.15.113:0/1546035628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe29406c170 0x7fe2940c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:44:17.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe293e00640 1 --2- 172.21.15.113:0/1546035628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 0x7fe29415f760 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2800097c0 tx=0x7fe28001ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ae069b69b23ee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:17.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe293400640 1 -- 172.21.15.113:0/1546035628 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2800b5070 con 0x7fe29406faa0 2024-03-04T07:44:17.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe293400640 1 -- 172.21.15.113:0/1546035628 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2800a89b0 con 0x7fe29406faa0 2024-03-04T07:44:17.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe293400640 1 -- 172.21.15.113:0/1546035628 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2800a8dc0 con 0x7fe29406faa0 2024-03-04T07:44:17.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe29a000640 1 -- 172.21.15.113:0/1546035628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 msgr2=0x7fe29415f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:17.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe29a000640 1 --2- 172.21.15.113:0/1546035628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 0x7fe29415f760 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe2800097c0 tx=0x7fe28001ecb0 comp rx=0 tx=0).stop 2024-03-04T07:44:17.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe29a000640 1 -- 172.21.15.113:0/1546035628 shutdown_connections 2024-03-04T07:44:17.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe29a000640 1 --2- 172.21.15.113:0/1546035628 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe29406c170 0x7fe2940c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:17.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe29a000640 1 --2- 172.21.15.113:0/1546035628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 0x7fe29415f760 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:17.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe29a000640 1 -- 172.21.15.113:0/1546035628 >> 172.21.15.113:0/1546035628 conn(0x7fe294072f40 msgr2=0x7fe2940706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:17.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe29a000640 1 -- 172.21.15.113:0/1546035628 shutdown_connections 2024-03-04T07:44:17.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.781+0000 7fe29a000640 1 -- 172.21.15.113:0/1546035628 wait complete. 2024-03-04T07:44:17.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe29a000640 1 Processor -- start 2024-03-04T07:44:17.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe29a000640 1 -- start start 2024-03-04T07:44:17.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe29a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe29406c170 0x7fe2940c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:17.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe29a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 0x7fe2940ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:17.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe29a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe294158f10 0x7fe2940cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:17.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe29a000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29415f240 con 0x7fe29406c170 2024-03-04T07:44:17.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe29a000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29415ef60 con 0x7fe29406faa0 2024-03-04T07:44:17.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe29a000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29406f490 con 0x7fe294158f10 2024-03-04T07:44:17.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe293e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 0x7fe2940ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:17.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe298c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe29406c170 0x7fe2940c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:17.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe293e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 0x7fe2940ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:36668/0 (socket says 172.21.15.113:36668) 2024-03-04T07:44:17.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe293e00640 1 -- 172.21.15.113:0/1043377916 learned_addr learned my addr 172.21.15.113:0/1043377916 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:17.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe299600640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe294158f10 0x7fe2940cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:17.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe298c00640 1 -- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe294158f10 msgr2=0x7fe2940cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:17.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe298c00640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe294158f10 0x7fe2940cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:17.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe298c00640 1 -- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 msgr2=0x7fe2940ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:17.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe298c00640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 0x7fe2940ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:17.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe298c00640 1 -- 172.21.15.113:0/1043377916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2941548e0 con 0x7fe29406c170 2024-03-04T07:44:17.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe293e00640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 0x7fe2940ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:44:17.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.785+0000 7fe299600640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe294158f10 0x7fe2940cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:44:17.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.789+0000 7fe298c00640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe29406c170 0x7fe2940c9c60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe288007fd0 tx=0x7fe28800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:17.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.789+0000 7fe290c00640 1 -- 172.21.15.113:0/1043377916 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288017070 con 0x7fe29406c170 2024-03-04T07:44:17.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.789+0000 7fe290c00640 1 -- 172.21.15.113:0/1043377916 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe288005cf0 con 0x7fe29406c170 2024-03-04T07:44:17.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.789+0000 7fe290c00640 1 -- 172.21.15.113:0/1043377916 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2880072f0 con 0x7fe29406c170 2024-03-04T07:44:17.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.789+0000 7fe29a000640 1 -- 172.21.15.113:0/1043377916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2941502b0 con 0x7fe29406c170 2024-03-04T07:44:17.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.789+0000 7fe29a000640 1 -- 172.21.15.113:0/1043377916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe294150870 con 0x7fe29406c170 2024-03-04T07:44:17.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.789+0000 7fe29a000640 1 -- 172.21.15.113:0/1043377916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe294075a80 con 0x7fe29406c170 2024-03-04T07:44:17.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.793+0000 7fe290c00640 1 -- 172.21.15.113:0/1043377916 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe28801a070 con 0x7fe29406c170 2024-03-04T07:44:17.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.793+0000 7fe290c00640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe274066fe0 0x7fe2740694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:17.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.793+0000 7fe290c00640 1 -- 172.21.15.113:0/1043377916 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6283+0+0 (secure 0 0 0) 0x7fe288084960 con 0x7fe29406c170 2024-03-04T07:44:17.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.793+0000 7fe293e00640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe274066fe0 0x7fe2740694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:17.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.797+0000 7fe293e00640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe274066fe0 0x7fe2740694a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe2800097c0 tx=0x7fe280008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:17.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:17.801+0000 7fe290c00640 1 -- 172.21.15.113:0/1043377916 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe288021020 con 0x7fe29406c170 2024-03-04T07:44:18.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.193+0000 7fe29a000640 1 -- 172.21.15.113:0/1043377916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe294154d70 con 0x7fe29406c170 2024-03-04T07:44:18.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.193+0000 7fe290c00640 1 -- 172.21.15.113:0/1043377916 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fe2941548e0 con 0x7fe29406c170 2024-03-04T07:44:18.194 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:18.194 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:42:01.001453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5094,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5094":{"gid":5094,"name":"c","rank":1,"incarnation":26,"state":"up:active","state_seq":21,"addr":"172.21.15.113:6833/605142559","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":605142559},{"type":"v1","addr":"172.21.15.113:6833","nonce":605142559}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099,5094]},"id":1}]} 2024-03-04T07:44:18.194 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 44 2024-03-04T07:44:18.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 -- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe274066fe0 msgr2=0x7fe2740694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:18.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe274066fe0 0x7fe2740694a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe2800097c0 tx=0x7fe280008040 comp rx=0 tx=0).stop 2024-03-04T07:44:18.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 -- 172.21.15.113:0/1043377916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe29406c170 msgr2=0x7fe2940c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:18.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe29406c170 0x7fe2940c9c60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe288007fd0 tx=0x7fe28800c7d0 comp rx=0 tx=0).stop 2024-03-04T07:44:18.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 -- 172.21.15.113:0/1043377916 shutdown_connections 2024-03-04T07:44:18.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe294158f10 0x7fe2940cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:18.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe29406faa0 0x7fe2940ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:18.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe274066fe0 0x7fe2740694a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:18.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 --2- 172.21.15.113:0/1043377916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe29406c170 0x7fe2940c9c60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:18.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 -- 172.21.15.113:0/1043377916 >> 172.21.15.113:0/1043377916 conn(0x7fe294072f40 msgr2=0x7fe2940cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:18.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 -- 172.21.15.113:0/1043377916 shutdown_connections 2024-03-04T07:44:18.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:18.197+0000 7fe29a000640 1 -- 172.21.15.113:0/1043377916 wait complete. 2024-03-04T07:44:18.229 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:44:20.229 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:20.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.457+0000 7fa32be00640 1 Processor -- start 2024-03-04T07:44:20.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.457+0000 7fa32be00640 1 -- start start 2024-03-04T07:44:20.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.457+0000 7fa32be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa32c06c580 0x7fa32c0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:20.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.457+0000 7fa32be00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa32c15bf20 0x7fa32c15c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:20.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.457+0000 7fa32be00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32c06bb10 con 0x7fa32c152920 2024-03-04T07:44:20.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.457+0000 7fa32be00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c0c7a40 con 0x7fa32c06c580 2024-03-04T07:44:20.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.457+0000 7fa32be00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32c0c7d20 con 0x7fa32c15bf20 2024-03-04T07:44:20.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.457+0000 7fa32a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa32c06c580 0x7fa32c0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:20.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.461+0000 7fa32a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa32c06c580 0x7fa32c0c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40196/0 (socket says 172.21.15.113:40196) 2024-03-04T07:44:20.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.461+0000 7fa32a000640 1 -- 172.21.15.113:0/3148907526 learned_addr learned my addr 172.21.15.113:0/3148907526 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:20.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.461+0000 7fa32b400640 1 --2- 172.21.15.113:0/3148907526 >> v2:172.21.15.136:3300/0 conn(0x7fa32c15bf20 0x7fa32c15c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:20.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.461+0000 7fa32a000640 1 -- 172.21.15.113:0/3148907526 >> v2:172.21.15.136:3300/0 conn(0x7fa32c15bf20 msgr2=0x7fa32c15c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:20.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.461+0000 7fa32a000640 1 --2- 172.21.15.113:0/3148907526 >> v2:172.21.15.136:3300/0 conn(0x7fa32c15bf20 0x7fa32c15c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.461+0000 7fa32a000640 1 -- 172.21.15.113:0/3148907526 >> v1:172.21.15.136:6789/0 conn(0x7fa32c152920 legacy=0x7fa32c071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:20.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.461+0000 7fa32a000640 1 -- 172.21.15.113:0/3148907526 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c06bd80 con 0x7fa32c06c580 2024-03-04T07:44:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa32a000640 1 --2- 172.21.15.113:0/3148907526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa32c06c580 0x7fa32c0c9060 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0c9a00 tx=0x7fa31801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67df70635d1ad868 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa329600640 1 -- 172.21.15.113:0/3148907526 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180b0070 con 0x7fa32c06c580 2024-03-04T07:44:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa329600640 1 -- 172.21.15.113:0/3148907526 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3180ab600 con 0x7fa32c06c580 2024-03-04T07:44:20.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa329600640 1 -- 172.21.15.113:0/3148907526 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180ab9d0 con 0x7fa32c06c580 2024-03-04T07:44:20.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa32be00640 1 -- 172.21.15.113:0/3148907526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa32c06c580 msgr2=0x7fa32c0c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:20.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa32be00640 1 --2- 172.21.15.113:0/3148907526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa32c06c580 0x7fa32c0c9060 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0c9a00 tx=0x7fa31801c460 comp rx=0 tx=0).stop 2024-03-04T07:44:20.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa32be00640 1 -- 172.21.15.113:0/3148907526 shutdown_connections 2024-03-04T07:44:20.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa32be00640 1 --2- 172.21.15.113:0/3148907526 >> v2:172.21.15.136:3300/0 conn(0x7fa32c15bf20 0x7fa32c15c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:20.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa32be00640 1 --2- 172.21.15.113:0/3148907526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa32c06c580 0x7fa32c0c9060 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:20.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa32be00640 1 -- 172.21.15.113:0/3148907526 >> 172.21.15.113:0/3148907526 conn(0x7fa32c0c9a30 msgr2=0x7fa32c0c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:20.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa32be00640 1 -- 172.21.15.113:0/3148907526 shutdown_connections 2024-03-04T07:44:20.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.465+0000 7fa32be00640 1 -- 172.21.15.113:0/3148907526 wait complete. 2024-03-04T07:44:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32be00640 1 Processor -- start 2024-03-04T07:44:20.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32be00640 1 -- start start 2024-03-04T07:44:20.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa32c06c580 0x7fa32c159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:20.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32c152920 0x7fa32c1f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:20.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa32c15bf20 0x7fa32c1f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:20.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c95a0 con 0x7fa32c15bf20 2024-03-04T07:44:20.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c3f60 con 0x7fa32c152920 2024-03-04T07:44:20.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c0c9720 con 0x7fa32c06c580 2024-03-04T07:44:20.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32aa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa32c06c580 0x7fa32c159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:20.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa32c15bf20 0x7fa32c1f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.469+0000 7fa32a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32c152920 0x7fa32c1f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa32c15bf20 0x7fa32c1f3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40202/0 (socket says 172.21.15.113:40202) 2024-03-04T07:44:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32c152920 0x7fa32c1f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:36688/0 (socket says 172.21.15.113:36688) 2024-03-04T07:44:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32b400640 1 -- 172.21.15.113:0/980020738 learned_addr learned my addr 172.21.15.113:0/980020738 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32a000640 1 -- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa32c06c580 msgr2=0x7fa32c159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32a000640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa32c06c580 0x7fa32c159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32a000640 1 -- 172.21.15.113:0/980020738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa32c15bf20 msgr2=0x7fa32c1f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32a000640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa32c15bf20 0x7fa32c1f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32a000640 1 -- 172.21.15.113:0/980020738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c153840 con 0x7fa32c152920 2024-03-04T07:44:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32a000640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32c152920 0x7fa32c1f3830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa3180ab000 tx=0x7fa318019200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:20.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa322a00640 1 -- 172.21.15.113:0/980020738 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180b0040 con 0x7fa32c152920 2024-03-04T07:44:20.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa322a00640 1 -- 172.21.15.113:0/980020738 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3180b4420 con 0x7fa32c152920 2024-03-04T07:44:20.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa322a00640 1 -- 172.21.15.113:0/980020738 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180b4810 con 0x7fa32c152920 2024-03-04T07:44:20.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32be00640 1 -- 172.21.15.113:0/980020738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c153560 con 0x7fa32c152920 2024-03-04T07:44:20.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32be00640 1 -- 172.21.15.113:0/980020738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c1f7220 con 0x7fa32c152920 2024-03-04T07:44:20.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.473+0000 7fa32be00640 1 -- 172.21.15.113:0/980020738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c075a80 con 0x7fa32c152920 2024-03-04T07:44:20.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.477+0000 7fa322a00640 1 -- 172.21.15.113:0/980020738 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3180b49b0 con 0x7fa32c152920 2024-03-04T07:44:20.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.477+0000 7fa322a00640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa30c066f60 0x7fa30c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:20.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.477+0000 7fa322a00640 1 -- 172.21.15.113:0/980020738 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6283+0+0 (secure 0 0 0) 0x7fa31811f800 con 0x7fa32c152920 2024-03-04T07:44:20.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.477+0000 7fa32aa00640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa30c066f60 0x7fa30c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:20.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.481+0000 7fa32aa00640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa30c066f60 0x7fa30c069420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa31c000f70 tx=0x7fa31c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:20.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.489+0000 7fa322a00640 1 -- 172.21.15.113:0/980020738 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa32c153840 con 0x7fa32c152920 2024-03-04T07:44:20.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.881+0000 7fa32be00640 1 -- 172.21.15.113:0/980020738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa32c0c8300 con 0x7fa32c152920 2024-03-04T07:44:20.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa322a00640 1 -- 172.21.15.113:0/980020738 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5815 (secure 0 0 0) 0x7fa318016110 con 0x7fa32c152920 2024-03-04T07:44:20.885 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:20.886 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:20.656319+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5099,5244]},"id":1}]} 2024-03-04T07:44:20.886 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 45 2024-03-04T07:44:20.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa32be00640 1 -- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa30c066f60 msgr2=0x7fa30c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:20.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa32be00640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa30c066f60 0x7fa30c069420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa31c000f70 tx=0x7fa31c009040 comp rx=0 tx=0).stop 2024-03-04T07:44:20.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa32be00640 1 -- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32c152920 msgr2=0x7fa32c1f3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:20.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa32be00640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32c152920 0x7fa32c1f3830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa3180ab000 tx=0x7fa318019200 comp rx=0 tx=0).stop 2024-03-04T07:44:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa32be00640 1 -- 172.21.15.113:0/980020738 shutdown_connections 2024-03-04T07:44:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa32be00640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa30c066f60 0x7fa30c069420 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa32be00640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa32c15bf20 0x7fa32c1f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa32be00640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa32c152920 0x7fa32c1f3830 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa32be00640 1 --2- 172.21.15.113:0/980020738 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa32c06c580 0x7fa32c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.885+0000 7fa32be00640 1 -- 172.21.15.113:0/980020738 >> 172.21.15.113:0/980020738 conn(0x7fa32c0c9a30 msgr2=0x7fa32c15a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.889+0000 7fa32be00640 1 -- 172.21.15.113:0/980020738 shutdown_connections 2024-03-04T07:44:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:20.889+0000 7fa32be00640 1 -- 172.21.15.113:0/980020738 wait complete. 2024-03-04T07:44:20.916 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-03-04T07:44:20.917 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:44:20.917 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:21.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.145+0000 7f1df5000640 1 Processor -- start 2024-03-04T07:44:21.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.145+0000 7f1df5000640 1 -- start start 2024-03-04T07:44:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.145+0000 7f1df5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df006c580 0x7f1df00c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.145+0000 7f1df5000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df006bb10 con 0x7f1df015bf20 2024-03-04T07:44:21.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.145+0000 7f1df5000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df00c7a40 con 0x7f1df0152920 2024-03-04T07:44:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.145+0000 7f1df5000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00c7d20 con 0x7f1df006c580 2024-03-04T07:44:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.149+0000 7f1deea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df006c580 0x7f1df00c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.149+0000 7f1def400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f1df0152920 0x7f1df0071a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:51250/0 (socket says 172.21.15.113:51250) 2024-03-04T07:44:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.149+0000 7f1def400640 1 -- 172.21.15.113:0/4190058287 learned_addr learned my addr 172.21.15.113:0/4190058287 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:21.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.149+0000 7f1dee000640 1 -- 172.21.15.113:0/4190058287 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493896077 0 0) 0x7f1df00c7a40 con 0x7f1df0152920 2024-03-04T07:44:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.149+0000 7f1dee000640 1 -- 172.21.15.113:0/4190058287 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd4003530 con 0x7f1df0152920 2024-03-04T07:44:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.149+0000 7f1deea00640 1 -- 172.21.15.113:0/4190058287 >> v1:172.21.15.136:6789/0 conn(0x7f1df0152920 legacy=0x7f1df0071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.149+0000 7f1deea00640 1 -- 172.21.15.113:0/4190058287 >> v1:172.21.15.113:6789/0 conn(0x7f1df015bf20 legacy=0x7f1df015c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:21.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.149+0000 7f1deea00640 1 -- 172.21.15.113:0/4190058287 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df006bd80 con 0x7f1df006c580 2024-03-04T07:44:21.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1deea00640 1 --2- 172.21.15.113:0/4190058287 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df006c580 0x7f1df00c9060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0097f0 tx=0x7f1ddc01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e92137a0e31538cf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:21.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1dee000640 1 -- 172.21.15.113:0/4190058287 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc0ab640 con 0x7f1df006c580 2024-03-04T07:44:21.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1dee000640 1 -- 172.21.15.113:0/4190058287 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc0b0070 con 0x7f1df006c580 2024-03-04T07:44:21.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1dee000640 1 -- 172.21.15.113:0/4190058287 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc0ab9e0 con 0x7f1df006c580 2024-03-04T07:44:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1df5000640 1 -- 172.21.15.113:0/4190058287 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df006c580 msgr2=0x7f1df00c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1df5000640 1 --2- 172.21.15.113:0/4190058287 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df006c580 0x7f1df00c9060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0097f0 tx=0x7f1ddc01c420 comp rx=0 tx=0).stop 2024-03-04T07:44:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1df5000640 1 -- 172.21.15.113:0/4190058287 shutdown_connections 2024-03-04T07:44:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1df5000640 1 --2- 172.21.15.113:0/4190058287 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df006c580 0x7f1df00c9060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1df5000640 1 -- 172.21.15.113:0/4190058287 >> 172.21.15.113:0/4190058287 conn(0x7f1df00c9a30 msgr2=0x7f1df00c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1df5000640 1 -- 172.21.15.113:0/4190058287 shutdown_connections 2024-03-04T07:44:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.153+0000 7f1df5000640 1 -- 172.21.15.113:0/4190058287 wait complete. 2024-03-04T07:44:21.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.157+0000 7f1df5000640 1 Processor -- start 2024-03-04T07:44:21.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.157+0000 7f1df5000640 1 -- start start 2024-03-04T07:44:21.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.157+0000 7f1df5000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df006c580 0x7f1df0150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:21.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.157+0000 7f1df5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df0152920 0x7f1df014d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:21.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.157+0000 7f1df5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df015bf20 0x7f1df014fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:21.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.157+0000 7f1df5000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00c9720 con 0x7f1df015bf20 2024-03-04T07:44:21.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.157+0000 7f1df5000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00c95a0 con 0x7f1df006c580 2024-03-04T07:44:21.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.157+0000 7f1df5000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00c4430 con 0x7f1df0152920 2024-03-04T07:44:21.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1defe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df015bf20 0x7f1df014fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:21.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1def400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df006c580 0x7f1df0150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:21.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1defe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df015bf20 0x7f1df014fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40204/0 (socket says 172.21.15.113:40204) 2024-03-04T07:44:21.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1defe00640 1 -- 172.21.15.113:0/363885784 learned_addr learned my addr 172.21.15.113:0/363885784 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:21.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1deea00640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df0152920 0x7f1df014d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1def400640 1 -- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df0152920 msgr2=0x7f1df014d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1def400640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df0152920 0x7f1df014d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1def400640 1 -- 172.21.15.113:0/363885784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df015bf20 msgr2=0x7f1df014fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1def400640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df015bf20 0x7f1df014fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1def400640 1 -- 172.21.15.113:0/363885784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df014e5e0 con 0x7f1df006c580 2024-03-04T07:44:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1deea00640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df0152920 0x7f1df014d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:44:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1def400640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df006c580 0x7f1df0150fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c400 tx=0x7f1de000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1de7400640 1 -- 172.21.15.113:0/363885784 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0017070 con 0x7f1df006c580 2024-03-04T07:44:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1de7400640 1 -- 172.21.15.113:0/363885784 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0005cf0 con 0x7f1df006c580 2024-03-04T07:44:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1de7400640 1 -- 172.21.15.113:0/363885784 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00072f0 con 0x7f1df006c580 2024-03-04T07:44:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1df5000640 1 -- 172.21.15.113:0/363885784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df014e8a0 con 0x7f1df006c580 2024-03-04T07:44:21.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1df5000640 1 -- 172.21.15.113:0/363885784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df014ee60 con 0x7f1df006c580 2024-03-04T07:44:21.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.161+0000 7f1df5000640 1 -- 172.21.15.113:0/363885784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df0075a80 con 0x7f1df006c580 2024-03-04T07:44:21.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.165+0000 7f1de7400640 1 -- 172.21.15.113:0/363885784 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1de001a070 con 0x7f1df006c580 2024-03-04T07:44:21.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.165+0000 7f1de7400640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1dd0066fb0 0x7f1dd0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:21.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.165+0000 7f1de7400640 1 -- 172.21.15.113:0/363885784 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f1de0084a80 con 0x7f1df006c580 2024-03-04T07:44:21.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.165+0000 7f1deea00640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1dd0066fb0 0x7f1dd0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:21.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.169+0000 7f1deea00640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1dd0066fb0 0x7f1dd0069470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0af000 tx=0x7f1ddc0b6040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:21.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.177+0000 7f1de7400640 1 -- 172.21.15.113:0/363885784 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1de0021020 con 0x7f1df006c580 2024-03-04T07:44:21.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.569+0000 7f1df5000640 1 -- 172.21.15.113:0/363885784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1df01640a0 con 0x7f1df006c580 2024-03-04T07:44:21.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.569+0000 7f1de7400640 1 -- 172.21.15.113:0/363885784 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f1de0082e20 con 0x7f1df006c580 2024-03-04T07:44:21.570 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:21.570 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:20.656319+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5099,5244]},"id":1}]} 2024-03-04T07:44:21.570 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 45 2024-03-04T07:44:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 -- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1dd0066fb0 msgr2=0x7f1dd0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1dd0066fb0 0x7f1dd0069470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc0af000 tx=0x7f1ddc0b6040 comp rx=0 tx=0).stop 2024-03-04T07:44:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 -- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df006c580 msgr2=0x7f1df0150fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:21.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df006c580 0x7f1df0150fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c400 tx=0x7f1de000c8c0 comp rx=0 tx=0).stop 2024-03-04T07:44:21.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 -- 172.21.15.113:0/363885784 shutdown_connections 2024-03-04T07:44:21.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1dd0066fb0 0x7f1dd0069470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:21.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df015bf20 0x7f1df014fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:21.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1df0152920 0x7f1df014d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:21.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 --2- 172.21.15.113:0/363885784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1df006c580 0x7f1df0150fc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:21.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 -- 172.21.15.113:0/363885784 >> 172.21.15.113:0/363885784 conn(0x7f1df00c9a30 msgr2=0x7f1df006b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:21.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 -- 172.21.15.113:0/363885784 shutdown_connections 2024-03-04T07:44:21.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:21.573+0000 7f1df5000640 1 -- 172.21.15.113:0/363885784 wait complete. 2024-03-04T07:44:21.601 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 45, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 45, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5099": { "addr": "172.21.15.136:6835/3944001911", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3944001911, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3944001911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5099, "incarnation": 39, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 21 }, "gid_5244": { "addr": "172.21.15.113:6837/742633123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 742633123, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 742633123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5244, "incarnation": 32, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 26 }, "gid_5529": { "addr": "172.21.15.113:6836/2299268553", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2299268553, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2299268553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5529, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:44:20.656319+0000", "qdb_cluster": [ 5099, 5244 ], "qdb_leader": 5244, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5244, "mds_1": 5529, "mds_2": 5099 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:44:23.602 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.829+0000 7fe2db200640 1 Processor -- start 2024-03-04T07:44:23.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.829+0000 7fe2db200640 1 -- start start 2024-03-04T07:44:23.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2db200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d4157ef0 0x7fe2d40c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:23.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2db200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe2d406fb70 0x7fe2d40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:23.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2db200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d40c9c50 con 0x7fe2d4070df0 2024-03-04T07:44:23.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2db200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d415a9a0 con 0x7fe2d406fb70 2024-03-04T07:44:23.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2db200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d406de30 con 0x7fe2d4157ef0 2024-03-04T07:44:23.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2d9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d4157ef0 0x7fe2d40c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:23.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2d9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d4157ef0 0x7fe2d40c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:40648/0 (socket says 172.21.15.113:40648) 2024-03-04T07:44:23.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2d9e00640 1 -- 172.21.15.113:0/2966800442 learned_addr learned my addr 172.21.15.113:0/2966800442 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:23.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2da800640 1 --2- 172.21.15.113:0/2966800442 >> v2:172.21.15.136:3300/0 conn(0x7fe2d406fb70 0x7fe2d40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:23.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2d9e00640 1 -- 172.21.15.113:0/2966800442 >> v2:172.21.15.136:3300/0 conn(0x7fe2d406fb70 msgr2=0x7fe2d40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:23.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2d9e00640 1 --2- 172.21.15.113:0/2966800442 >> v2:172.21.15.136:3300/0 conn(0x7fe2d406fb70 0x7fe2d40cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:23.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2d9e00640 1 -- 172.21.15.113:0/2966800442 >> v1:172.21.15.113:6789/0 conn(0x7fe2d4070df0 legacy=0x7fe2d415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:23.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.833+0000 7fe2d9e00640 1 -- 172.21.15.113:0/2966800442 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d406c210 con 0x7fe2d4157ef0 2024-03-04T07:44:23.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2d9e00640 1 --2- 172.21.15.113:0/2966800442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d4157ef0 0x7fe2d40c8a90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009870 tx=0x7fe2c801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f2f93a6bc5453e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:23.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2d8a00640 1 -- 172.21.15.113:0/2966800442 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80ab5f0 con 0x7fe2d4157ef0 2024-03-04T07:44:23.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2d8a00640 1 -- 172.21.15.113:0/2966800442 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c80b0070 con 0x7fe2d4157ef0 2024-03-04T07:44:23.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2d8a00640 1 -- 172.21.15.113:0/2966800442 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80ab9b0 con 0x7fe2d4157ef0 2024-03-04T07:44:23.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2db200640 1 -- 172.21.15.113:0/2966800442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d4157ef0 msgr2=0x7fe2d40c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:23.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2db200640 1 --2- 172.21.15.113:0/2966800442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d4157ef0 0x7fe2d40c8a90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8009870 tx=0x7fe2c801c540 comp rx=0 tx=0).stop 2024-03-04T07:44:23.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2db200640 1 -- 172.21.15.113:0/2966800442 shutdown_connections 2024-03-04T07:44:23.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2db200640 1 --2- 172.21.15.113:0/2966800442 >> v2:172.21.15.136:3300/0 conn(0x7fe2d406fb70 0x7fe2d40cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:23.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2db200640 1 --2- 172.21.15.113:0/2966800442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d4157ef0 0x7fe2d40c8a90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:23.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2db200640 1 -- 172.21.15.113:0/2966800442 >> 172.21.15.113:0/2966800442 conn(0x7fe2d4072f40 msgr2=0x7fe2d40706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:23.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2db200640 1 -- 172.21.15.113:0/2966800442 shutdown_connections 2024-03-04T07:44:23.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.837+0000 7fe2db200640 1 -- 172.21.15.113:0/2966800442 wait complete. 2024-03-04T07:44:23.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.841+0000 7fe2db200640 1 Processor -- start 2024-03-04T07:44:23.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.841+0000 7fe2db200640 1 -- start start 2024-03-04T07:44:23.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2db200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d406fb70 0x7fe2d40c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:23.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2db200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2d4070df0 0x7fe2d40c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:23.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2db200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2d4157ef0 0x7fe2d40cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:23.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2db200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d41582d0 con 0x7fe2d4157ef0 2024-03-04T07:44:23.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2db200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d415b890 con 0x7fe2d4070df0 2024-03-04T07:44:23.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2db200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d4158450 con 0x7fe2d406fb70 2024-03-04T07:44:23.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2d9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d406fb70 0x7fe2d40c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:23.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2d9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d406fb70 0x7fe2d40c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:40662/0 (socket says 172.21.15.113:40662) 2024-03-04T07:44:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2d9e00640 1 -- 172.21.15.113:0/713276076 learned_addr learned my addr 172.21.15.113:0/713276076 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2d9400640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2d4070df0 0x7fe2d40c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2da800640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2d4157ef0 0x7fe2d40cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2d9e00640 1 -- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2d4070df0 msgr2=0x7fe2d40c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2d9e00640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2d4070df0 0x7fe2d40c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2d9e00640 1 -- 172.21.15.113:0/713276076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2d4157ef0 msgr2=0x7fe2d40cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2d9e00640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2d4157ef0 0x7fe2d40cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2d9e00640 1 -- 172.21.15.113:0/713276076 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d40cd820 con 0x7fe2d406fb70 2024-03-04T07:44:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2d9e00640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d406fb70 0x7fe2d40c5d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80ad040 tx=0x7fe2c801ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2ce000640 1 -- 172.21.15.113:0/713276076 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80be3f0 con 0x7fe2d406fb70 2024-03-04T07:44:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2ce000640 1 -- 172.21.15.113:0/713276076 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c80be590 con 0x7fe2d406fb70 2024-03-04T07:44:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2ce000640 1 -- 172.21.15.113:0/713276076 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80be980 con 0x7fe2d406fb70 2024-03-04T07:44:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2db200640 1 -- 172.21.15.113:0/713276076 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d41f87c0 con 0x7fe2d406fb70 2024-03-04T07:44:23.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2db200640 1 -- 172.21.15.113:0/713276076 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d40c5050 con 0x7fe2d406fb70 2024-03-04T07:44:23.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.845+0000 7fe2db200640 1 -- 172.21.15.113:0/713276076 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d4075a80 con 0x7fe2d406fb70 2024-03-04T07:44:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.849+0000 7fe2ce000640 1 -- 172.21.15.113:0/713276076 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2c80beb20 con 0x7fe2d406fb70 2024-03-04T07:44:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.849+0000 7fe2ce000640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2b8066f60 0x7fe2b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.849+0000 7fe2ce000640 1 -- 172.21.15.113:0/713276076 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fe2c811f320 con 0x7fe2d406fb70 2024-03-04T07:44:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.849+0000 7fe2d9400640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2b8066f60 0x7fe2b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:23.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.853+0000 7fe2d9400640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2b8066f60 0x7fe2b8069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe2d41f8d40 tx=0x7fe2c4007e40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:23.861+0000 7fe2ce000640 1 -- 172.21.15.113:0/713276076 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe2c80c4060 con 0x7fe2d406fb70 2024-03-04T07:44:24.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.249+0000 7fe2db200640 1 -- 172.21.15.113:0/713276076 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2d40b2d50 con 0x7fe2d406fb70 2024-03-04T07:44:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.253+0000 7fe2ce000640 1 -- 172.21.15.113:0/713276076 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+5815 (secure 0 0 0) 0x7fe2c8016130 con 0x7fe2d406fb70 2024-03-04T07:44:24.253 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:24.253 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:20.656319+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5099,5244]},"id":1}]} 2024-03-04T07:44:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 45 2024-03-04T07:44:24.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.253+0000 7fe2db200640 1 -- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2b8066f60 msgr2=0x7fe2b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:24.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.253+0000 7fe2db200640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2b8066f60 0x7fe2b8069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe2d41f8d40 tx=0x7fe2c4007e40 comp rx=0 tx=0).stop 2024-03-04T07:44:24.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.253+0000 7fe2db200640 1 -- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d406fb70 msgr2=0x7fe2d40c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:24.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.253+0000 7fe2db200640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d406fb70 0x7fe2d40c5d10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80ad040 tx=0x7fe2c801ad80 comp rx=0 tx=0).stop 2024-03-04T07:44:24.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.257+0000 7fe2db200640 1 -- 172.21.15.113:0/713276076 shutdown_connections 2024-03-04T07:44:24.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.257+0000 7fe2db200640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2b8066f60 0x7fe2b8069420 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:24.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.257+0000 7fe2db200640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2d4157ef0 0x7fe2d40cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:24.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.257+0000 7fe2db200640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2d4070df0 0x7fe2d40c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:24.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.257+0000 7fe2db200640 1 --2- 172.21.15.113:0/713276076 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2d406fb70 0x7fe2d40c5d10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:24.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.257+0000 7fe2db200640 1 -- 172.21.15.113:0/713276076 >> 172.21.15.113:0/713276076 conn(0x7fe2d4072f40 msgr2=0x7fe2d406ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:24.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.257+0000 7fe2db200640 1 -- 172.21.15.113:0/713276076 shutdown_connections 2024-03-04T07:44:24.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:24.257+0000 7fe2db200640 1 -- 172.21.15.113:0/713276076 wait complete. 2024-03-04T07:44:24.287 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:44:26.288 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:26.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd8a00640 1 Processor -- start 2024-03-04T07:44:26.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd8a00640 1 -- start start 2024-03-04T07:44:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd8a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2fd40c19b0 0x7f2fd406ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd8a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c1d90 0x7f2fd415b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd8a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd406f2d0 con 0x7f2fd406d550 2024-03-04T07:44:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd8a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd406d320 con 0x7f2fd40c19b0 2024-03-04T07:44:26.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd8a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd406c7a0 con 0x7f2fd40c1d90 2024-03-04T07:44:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c1d90 0x7f2fd415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c1d90 0x7f2fd415b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:40666/0 (socket says 172.21.15.113:40666) 2024-03-04T07:44:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd1e00640 1 -- 172.21.15.113:0/748465102 learned_addr learned my addr 172.21.15.113:0/748465102 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:26.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd2800640 1 --2- 172.21.15.113:0/748465102 >> v2:172.21.15.113:3300/0 conn(0x7f2fd40c19b0 0x7f2fd406ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.569+0000 7f2fd1400640 1 -- 172.21.15.113:0/748465102 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216293757 0 0) 0x7f2fd406f2d0 con 0x7f2fd406d550 2024-03-04T07:44:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.573+0000 7f2fd1400640 1 -- 172.21.15.113:0/748465102 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fb8003530 con 0x7f2fd406d550 2024-03-04T07:44:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.573+0000 7f2fd1e00640 1 -- 172.21.15.113:0/748465102 >> v2:172.21.15.113:3300/0 conn(0x7f2fd40c19b0 msgr2=0x7f2fd406ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.573+0000 7f2fd1e00640 1 --2- 172.21.15.113:0/748465102 >> v2:172.21.15.113:3300/0 conn(0x7f2fd40c19b0 0x7f2fd406ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.573+0000 7f2fd1e00640 1 -- 172.21.15.113:0/748465102 >> v1:172.21.15.136:6789/0 conn(0x7f2fd406d550 legacy=0x7f2fd4154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:26.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.573+0000 7f2fd1e00640 1 -- 172.21.15.113:0/748465102 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd406ca80 con 0x7f2fd40c1d90 2024-03-04T07:44:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.581+0000 7f2fd1e00640 1 --2- 172.21.15.113:0/748465102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c1d90 0x7f2fd415b220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009870 tx=0x7f2fc401ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81372928943947 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.581+0000 7f2fd1400640 1 -- 172.21.15.113:0/748465102 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40b5070 con 0x7f2fd40c1d90 2024-03-04T07:44:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.581+0000 7f2fd1400640 1 -- 172.21.15.113:0/748465102 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc40a8a50 con 0x7f2fd40c1d90 2024-03-04T07:44:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.581+0000 7f2fd1400640 1 -- 172.21.15.113:0/748465102 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40a8e20 con 0x7f2fd40c1d90 2024-03-04T07:44:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.581+0000 7f2fd8a00640 1 -- 172.21.15.113:0/748465102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c1d90 msgr2=0x7f2fd415b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.581+0000 7f2fd8a00640 1 --2- 172.21.15.113:0/748465102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c1d90 0x7f2fd415b220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009870 tx=0x7f2fc401ed70 comp rx=0 tx=0).stop 2024-03-04T07:44:26.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.597+0000 7f2fd8a00640 1 -- 172.21.15.113:0/748465102 shutdown_connections 2024-03-04T07:44:26.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.597+0000 7f2fd8a00640 1 --2- 172.21.15.113:0/748465102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c1d90 0x7f2fd415b220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:26.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.597+0000 7f2fd8a00640 1 --2- 172.21.15.113:0/748465102 >> v2:172.21.15.113:3300/0 conn(0x7f2fd40c19b0 0x7f2fd406ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:26.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.597+0000 7f2fd8a00640 1 -- 172.21.15.113:0/748465102 >> 172.21.15.113:0/748465102 conn(0x7f2fd406e2a0 msgr2=0x7f2fd406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:26.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.597+0000 7f2fd8a00640 1 -- 172.21.15.113:0/748465102 shutdown_connections 2024-03-04T07:44:26.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.597+0000 7f2fd8a00640 1 -- 172.21.15.113:0/748465102 wait complete. 2024-03-04T07:44:26.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.597+0000 7f2fd8a00640 1 Processor -- start 2024-03-04T07:44:26.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.601+0000 7f2fd8a00640 1 -- start start 2024-03-04T07:44:26.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fd406d550 0x7f2fd41e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:26.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd8a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c19b0 0x7f2fd41f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:26.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fd406d550 0x7f2fd41e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:26.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fd406d550 0x7f2fd41e1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40228/0 (socket says 172.21.15.113:40228) 2024-03-04T07:44:26.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd2800640 1 -- 172.21.15.113:0/4230486738 learned_addr learned my addr 172.21.15.113:0/4230486738 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:26.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd1e00640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c19b0 0x7f2fd41f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:26.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd8a00640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fd40c1d90 0x7f2fd41e3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:26.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd415ad00 con 0x7f2fd406d550 2024-03-04T07:44:26.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd3200640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fd40c1d90 0x7f2fd41e3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:26.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd406d040 con 0x7f2fd40c1d90 2024-03-04T07:44:26.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.605+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd406c0e0 con 0x7f2fd40c19b0 2024-03-04T07:44:26.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fd3200640 1 -- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c19b0 msgr2=0x7f2fd41f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:26.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fd3200640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c19b0 0x7f2fd41f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:26.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fd3200640 1 -- 172.21.15.113:0/4230486738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fd406d550 msgr2=0x7f2fd41e1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:26.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fd3200640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fd406d550 0x7f2fd41e1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:26.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fd3200640 1 -- 172.21.15.113:0/4230486738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd41537d0 con 0x7f2fd40c1d90 2024-03-04T07:44:26.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fd3200640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fd40c1d90 0x7f2fd41e3330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000c390 tx=0x7f2fc000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:26.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fcaa00640 1 -- 172.21.15.113:0/4230486738 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0017070 con 0x7f2fd40c1d90 2024-03-04T07:44:26.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fcaa00640 1 -- 172.21.15.113:0/4230486738 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc0003600 con 0x7f2fd40c1d90 2024-03-04T07:44:26.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fcaa00640 1 -- 172.21.15.113:0/4230486738 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00039f0 con 0x7f2fd40c1d90 2024-03-04T07:44:26.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd41e4ae0 con 0x7f2fd40c1d90 2024-03-04T07:44:26.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.609+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd41e0d10 con 0x7f2fd40c1d90 2024-03-04T07:44:26.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.613+0000 7f2fcaa00640 1 -- 172.21.15.113:0/4230486738 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2fc001a070 con 0x7f2fd40c1d90 2024-03-04T07:44:26.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.613+0000 7f2fcaa00640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2fb4066f60 0x7f2fb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:26.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.613+0000 7f2fcaa00640 1 -- 172.21.15.113:0/4230486738 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f2fc00844e0 con 0x7f2fd40c1d90 2024-03-04T07:44:26.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.617+0000 7f2fd2800640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2fb4066f60 0x7f2fb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:26.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.617+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd41e1040 con 0x7f2fd40c1d90 2024-03-04T07:44:26.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.617+0000 7f2fd2800640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2fb4066f60 0x7f2fb4069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009b40 tx=0x7f2fcc003a60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:26.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:26.633+0000 7f2fcaa00640 1 -- 172.21.15.113:0/4230486738 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2fc0054c10 con 0x7f2fd40c1d90 2024-03-04T07:44:26.928 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:44:26.940 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:44:27.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.153+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fd4075a80 con 0x7f2fd40c1d90 2024-03-04T07:44:27.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.153+0000 7f2fcaa00640 1 -- 172.21.15.113:0/4230486738 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5817 (secure 0 0 0) 0x7f2fc0058cb0 con 0x7f2fd40c1d90 2024-03-04T07:44:27.154 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:27.154 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:25.673512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:resolve","state_seq":37,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099]},"id":1}]} 2024-03-04T07:44:27.154 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 46 2024-03-04T07:44:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2fb4066f60 msgr2=0x7f2fb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2fb4066f60 0x7f2fb4069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009b40 tx=0x7f2fcc003a60 comp rx=0 tx=0).stop 2024-03-04T07:44:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fd40c1d90 msgr2=0x7f2fd41e3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fd40c1d90 0x7f2fd41e3330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000c390 tx=0x7f2fc000c850 comp rx=0 tx=0).stop 2024-03-04T07:44:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 shutdown_connections 2024-03-04T07:44:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fd40c1d90 0x7f2fd41e3330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2fd40c19b0 0x7f2fd41f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2fb4066f60 0x7f2fb4069420 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 --2- 172.21.15.113:0/4230486738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2fd406d550 0x7f2fd41e1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 >> 172.21.15.113:0/4230486738 conn(0x7f2fd406e2a0 msgr2=0x7f2fd40c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 shutdown_connections 2024-03-04T07:44:27.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:27.157+0000 7f2fd8a00640 1 -- 172.21.15.113:0/4230486738 wait complete. 2024-03-04T07:44:27.187 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:44:29.188 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:29.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.413+0000 7f1ea9600640 1 Processor -- start 2024-03-04T07:44:29.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea9600640 1 -- start start 2024-03-04T07:44:29.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1ea4157ef0 0x7f1ea40c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:29.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea406fb70 0x7f1ea40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:29.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea40c9c50 con 0x7f1ea4070df0 2024-03-04T07:44:29.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea415a9a0 con 0x7f1ea4157ef0 2024-03-04T07:44:29.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea406de30 con 0x7f1ea406fb70 2024-03-04T07:44:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1ea4157ef0 0x7f1ea40c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea406fb70 0x7f1ea40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1ea4157ef0 0x7f1ea40c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:42894/0 (socket says 172.21.15.113:42894) 2024-03-04T07:44:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea3e00640 1 -- 172.21.15.113:0/1558684371 learned_addr learned my addr 172.21.15.113:0/1558684371 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea8c00640 1 -- 172.21.15.113:0/1558684371 >> v2:172.21.15.136:3300/0 conn(0x7f1ea4157ef0 msgr2=0x7f1ea40c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea8c00640 1 --2- 172.21.15.113:0/1558684371 >> v2:172.21.15.136:3300/0 conn(0x7f1ea4157ef0 0x7f1ea40c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:29.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea8c00640 1 -- 172.21.15.113:0/1558684371 >> v1:172.21.15.113:6789/0 conn(0x7f1ea4070df0 legacy=0x7f1ea415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:29.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.417+0000 7f1ea8c00640 1 -- 172.21.15.113:0/1558684371 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea406c210 con 0x7f1ea406fb70 2024-03-04T07:44:29.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea8c00640 1 --2- 172.21.15.113:0/1558684371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea406fb70 0x7f1ea40cdfe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009d90 tx=0x7f1e9c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da844e8e5e751153 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:29.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea2a00640 1 -- 172.21.15.113:0/1558684371 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0b2070 con 0x7f1ea406fb70 2024-03-04T07:44:29.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea2a00640 1 -- 172.21.15.113:0/1558684371 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c0a5940 con 0x7f1ea406fb70 2024-03-04T07:44:29.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea2a00640 1 -- 172.21.15.113:0/1558684371 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0a5d10 con 0x7f1ea406fb70 2024-03-04T07:44:29.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea9600640 1 -- 172.21.15.113:0/1558684371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea406fb70 msgr2=0x7f1ea40cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:29.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea9600640 1 --2- 172.21.15.113:0/1558684371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea406fb70 0x7f1ea40cdfe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009d90 tx=0x7f1e9c01c970 comp rx=0 tx=0).stop 2024-03-04T07:44:29.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea9600640 1 -- 172.21.15.113:0/1558684371 shutdown_connections 2024-03-04T07:44:29.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea9600640 1 --2- 172.21.15.113:0/1558684371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea406fb70 0x7f1ea40cdfe0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:29.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea9600640 1 --2- 172.21.15.113:0/1558684371 >> v2:172.21.15.136:3300/0 conn(0x7f1ea4157ef0 0x7f1ea40c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:29.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea9600640 1 -- 172.21.15.113:0/1558684371 >> 172.21.15.113:0/1558684371 conn(0x7f1ea4072f40 msgr2=0x7f1ea40706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:29.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea9600640 1 -- 172.21.15.113:0/1558684371 shutdown_connections 2024-03-04T07:44:29.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.425+0000 7f1ea9600640 1 -- 172.21.15.113:0/1558684371 wait complete. 2024-03-04T07:44:29.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea9600640 1 Processor -- start 2024-03-04T07:44:29.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea9600640 1 -- start start 2024-03-04T07:44:29.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ea406fb70 0x7f1ea4152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:29.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea4070df0 0x7f1ea41530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:29.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea9600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ea4157ef0 0x7f1ea4153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:29.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea9600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea41582d0 con 0x7f1ea406fb70 2024-03-04T07:44:29.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea9600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea415b890 con 0x7f1ea4157ef0 2024-03-04T07:44:29.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea4158450 con 0x7f1ea4070df0 2024-03-04T07:44:29.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea4070df0 0x7f1ea41530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:29.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea4070df0 0x7f1ea41530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:40332/0 (socket says 172.21.15.113:40332) 2024-03-04T07:44:29.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea3400640 1 -- 172.21.15.113:0/2277579175 learned_addr learned my addr 172.21.15.113:0/2277579175 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:29.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea8c00640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ea4157ef0 0x7f1ea4153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:29.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea3e00640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ea406fb70 0x7f1ea4152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:29.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea8c00640 1 -- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea4070df0 msgr2=0x7f1ea41530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:29.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea8c00640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea4070df0 0x7f1ea41530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:29.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea8c00640 1 -- 172.21.15.113:0/2277579175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ea406fb70 msgr2=0x7f1ea4152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:29.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea8c00640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ea406fb70 0x7f1ea4152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:29.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea8c00640 1 -- 172.21.15.113:0/2277579175 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea40c6430 con 0x7f1ea4157ef0 2024-03-04T07:44:29.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.429+0000 7f1ea8c00640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ea4157ef0 0x7f1ea4153620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40cbf40 tx=0x7f1e9c019710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:29.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.433+0000 7f1e9be00640 1 -- 172.21.15.113:0/2277579175 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0b2050 con 0x7f1ea4157ef0 2024-03-04T07:44:29.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.433+0000 7f1e9be00640 1 -- 172.21.15.113:0/2277579175 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c0be070 con 0x7f1ea4157ef0 2024-03-04T07:44:29.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.433+0000 7f1ea9600640 1 -- 172.21.15.113:0/2277579175 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea40c58b0 con 0x7f1ea4157ef0 2024-03-04T07:44:29.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.433+0000 7f1e9be00640 1 -- 172.21.15.113:0/2277579175 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0ada50 con 0x7f1ea4157ef0 2024-03-04T07:44:29.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.433+0000 7f1ea9600640 1 -- 172.21.15.113:0/2277579175 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea40c5e70 con 0x7f1ea4157ef0 2024-03-04T07:44:29.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.433+0000 7f1ea9600640 1 -- 172.21.15.113:0/2277579175 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea4075a80 con 0x7f1ea4157ef0 2024-03-04T07:44:29.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.437+0000 7f1e9be00640 1 -- 172.21.15.113:0/2277579175 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e9c0adbf0 con 0x7f1ea4157ef0 2024-03-04T07:44:29.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.437+0000 7f1e9be00640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e84066f60 0x7f1e84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:29.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.437+0000 7f1e9be00640 1 -- 172.21.15.113:0/2277579175 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f1e9c120cc0 con 0x7f1ea4157ef0 2024-03-04T07:44:29.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.437+0000 7f1ea3e00640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e84066f60 0x7f1e84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:29.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.437+0000 7f1ea3e00640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e84066f60 0x7f1e84069420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009ab0 tx=0x7f1e94009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:29.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.445+0000 7f1e9be00640 1 -- 172.21.15.113:0/2277579175 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1e9c016560 con 0x7f1ea4157ef0 2024-03-04T07:44:29.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.837+0000 7f1ea9600640 1 -- 172.21.15.113:0/2277579175 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ea415a780 con 0x7f1ea4157ef0 2024-03-04T07:44:29.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1e9be00640 1 -- 172.21.15.113:0/2277579175 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5817 (secure 0 0 0) 0x7f1e9c0f13e0 con 0x7f1ea4157ef0 2024-03-04T07:44:29.840 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:29.841 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:25.673512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:resolve","state_seq":37,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099]},"id":1}]} 2024-03-04T07:44:29.841 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 46 2024-03-04T07:44:29.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1ea9600640 1 -- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e84066f60 msgr2=0x7f1e84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1ea9600640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e84066f60 0x7f1e84069420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009ab0 tx=0x7f1e94009240 comp rx=0 tx=0).stop 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1ea9600640 1 -- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ea4157ef0 msgr2=0x7f1ea4153620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1ea9600640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ea4157ef0 0x7f1ea4153620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40cbf40 tx=0x7f1e9c019710 comp rx=0 tx=0).stop 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1ea9600640 1 -- 172.21.15.113:0/2277579175 shutdown_connections 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1ea9600640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ea4157ef0 0x7f1ea4153620 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1ea9600640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ea4070df0 0x7f1ea41530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1ea9600640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e84066f60 0x7f1e84069420 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1ea9600640 1 --2- 172.21.15.113:0/2277579175 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ea406fb70 0x7f1ea4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.841+0000 7f1ea9600640 1 -- 172.21.15.113:0/2277579175 >> 172.21.15.113:0/2277579175 conn(0x7f1ea4072f40 msgr2=0x7f1ea4159690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.845+0000 7f1ea9600640 1 -- 172.21.15.113:0/2277579175 shutdown_connections 2024-03-04T07:44:29.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:29.845+0000 7f1ea9600640 1 -- 172.21.15.113:0/2277579175 wait complete. 2024-03-04T07:44:29.872 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:44:31.873 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:32.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.101+0000 7f831ca00640 1 Processor -- start 2024-03-04T07:44:32.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.101+0000 7f831ca00640 1 -- start start 2024-03-04T07:44:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f831ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151c20 0x7f83180c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f831ca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8318157da0 con 0x7f831806d140 2024-03-04T07:44:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f831ca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8318158080 con 0x7f8318151840 2024-03-04T07:44:32.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f831ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83180c3440 con 0x7f8318151c20 2024-03-04T07:44:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f8317400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f8318151840 0x7f831806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:53560/0 (socket says 172.21.15.113:53560) 2024-03-04T07:44:32.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f8317400640 1 -- 172.21.15.113:0/1551933870 learned_addr learned my addr 172.21.15.113:0/1551933870 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f8316a00640 1 --2- 172.21.15.113:0/1551933870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151c20 0x7f83180c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f8316000640 1 -- 172.21.15.113:0/1551933870 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412808838 0 0) 0x7f8318158080 con 0x7f8318151840 2024-03-04T07:44:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f8316000640 1 -- 172.21.15.113:0/1551933870 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82fc007940 con 0x7f8318151840 2024-03-04T07:44:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f8316000640 1 -- 172.21.15.113:0/1551933870 <== mon.1 v1:172.21.15.136:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1572239311 0 0) 0x7f82fc007940 con 0x7f8318151840 2024-03-04T07:44:32.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f8316a00640 1 -- 172.21.15.113:0/1551933870 >> v1:172.21.15.136:6789/0 conn(0x7f8318151840 legacy=0x7f831806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:32.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f8316a00640 1 -- 172.21.15.113:0/1551933870 >> v1:172.21.15.113:6789/0 conn(0x7f831806d140 legacy=0x7f83180722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:32.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.105+0000 7f8316a00640 1 -- 172.21.15.113:0/1551933870 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8318158360 con 0x7f8318151c20 2024-03-04T07:44:32.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.109+0000 7f8316a00640 1 --2- 172.21.15.113:0/1551933870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151c20 0x7f83180c1f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8304009770 tx=0x7f830401eb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38d3ecbcdf9a266 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:32.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.109+0000 7f8316000640 1 -- 172.21.15.113:0/1551933870 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83040b5070 con 0x7f8318151c20 2024-03-04T07:44:32.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.113+0000 7f8316000640 1 -- 172.21.15.113:0/1551933870 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83040a8630 con 0x7f8318151c20 2024-03-04T07:44:32.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.113+0000 7f8316000640 1 -- 172.21.15.113:0/1551933870 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308001680 con 0x7f8318151c20 2024-03-04T07:44:32.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.113+0000 7f831ca00640 1 -- 172.21.15.113:0/1551933870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151c20 msgr2=0x7f83180c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:32.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.113+0000 7f831ca00640 1 --2- 172.21.15.113:0/1551933870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151c20 0x7f83180c1f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8304009770 tx=0x7f830401eb30 comp rx=0 tx=0).stop 2024-03-04T07:44:32.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.113+0000 7f831ca00640 1 -- 172.21.15.113:0/1551933870 shutdown_connections 2024-03-04T07:44:32.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.113+0000 7f831ca00640 1 --2- 172.21.15.113:0/1551933870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151c20 0x7f83180c1f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:32.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.113+0000 7f831ca00640 1 -- 172.21.15.113:0/1551933870 >> 172.21.15.113:0/1551933870 conn(0x7f83180c9a30 msgr2=0x7f83180c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:32.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.113+0000 7f831ca00640 1 -- 172.21.15.113:0/1551933870 shutdown_connections 2024-03-04T07:44:32.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.113+0000 7f831ca00640 1 -- 172.21.15.113:0/1551933870 wait complete. 2024-03-04T07:44:32.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.113+0000 7f831ca00640 1 Processor -- start 2024-03-04T07:44:32.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f831ca00640 1 -- start start 2024-03-04T07:44:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f831ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f831806d140 0x7f83181e1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f831ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151840 0x7f831814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f831ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8318151c20 0x7f83180cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f831ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8318071db0 con 0x7f8318151c20 2024-03-04T07:44:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f831ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8318071ad0 con 0x7f831806d140 2024-03-04T07:44:32.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f831ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831806c880 con 0x7f8318151840 2024-03-04T07:44:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8316a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151840 0x7f831814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8316a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151840 0x7f831814afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:40350/0 (socket says 172.21.15.113:40350) 2024-03-04T07:44:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8316a00640 1 -- 172.21.15.113:0/3435799558 learned_addr learned my addr 172.21.15.113:0/3435799558 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8317400640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f831806d140 0x7f83181e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:32.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8317e00640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8318151c20 0x7f83180cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8316a00640 1 -- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f831806d140 msgr2=0x7f83181e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8316a00640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f831806d140 0x7f83181e1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8316a00640 1 -- 172.21.15.113:0/3435799558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8318151c20 msgr2=0x7f83180cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:32.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8316a00640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8318151c20 0x7f83180cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8316a00640 1 -- 172.21.15.113:0/3435799558 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83181ed0c0 con 0x7f8318151840 2024-03-04T07:44:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f8316a00640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151840 0x7f831814afb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f83040b3040 tx=0x7f830401b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f830f400640 1 -- 172.21.15.113:0/3435799558 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83040b5040 con 0x7f8318151840 2024-03-04T07:44:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f830f400640 1 -- 172.21.15.113:0/3435799558 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83040bf070 con 0x7f8318151840 2024-03-04T07:44:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f831ca00640 1 -- 172.21.15.113:0/3435799558 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83181ec260 con 0x7f8318151840 2024-03-04T07:44:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.117+0000 7f830f400640 1 -- 172.21.15.113:0/3435799558 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83040a9d30 con 0x7f8318151840 2024-03-04T07:44:32.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.121+0000 7f831ca00640 1 -- 172.21.15.113:0/3435799558 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83181ec820 con 0x7f8318151840 2024-03-04T07:44:32.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.121+0000 7f830cc00640 1 -- 172.21.15.113:0/3435799558 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83180c6d80 con 0x7f8318151840 2024-03-04T07:44:32.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.121+0000 7f830f400640 1 -- 172.21.15.113:0/3435799558 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8304003620 con 0x7f8318151840 2024-03-04T07:44:32.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.125+0000 7f830f400640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82f8066f60 0x7f82f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:32.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.125+0000 7f830f400640 1 -- 172.21.15.113:0/3435799558 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f8304122580 con 0x7f8318151840 2024-03-04T07:44:32.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.125+0000 7f8317400640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82f8066f60 0x7f82f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:32.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.125+0000 7f8317400640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82f8066f60 0x7f82f8069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8308005890 tx=0x7f8308005800 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:32.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.141+0000 7f830f400640 1 -- 172.21.15.113:0/3435799558 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f83180c6d80 con 0x7f8318151840 2024-03-04T07:44:32.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.529+0000 7f830cc00640 1 -- 172.21.15.113:0/3435799558 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8318075a80 con 0x7f8318151840 2024-03-04T07:44:32.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.533+0000 7f830f400640 1 -- 172.21.15.113:0/3435799558 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f83040186d0 con 0x7f8318151840 2024-03-04T07:44:32.534 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:32.534 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:30.768700+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:reconnect","state_seq":39,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5099,5244]},"id":1}]} 2024-03-04T07:44:32.535 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 47 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 -- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82f8066f60 msgr2=0x7f82f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82f8066f60 0x7f82f8069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8308005890 tx=0x7f8308005800 comp rx=0 tx=0).stop 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 -- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151840 msgr2=0x7f831814afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151840 0x7f831814afb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f83040b3040 tx=0x7f830401b8d0 comp rx=0 tx=0).stop 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 -- 172.21.15.113:0/3435799558 shutdown_connections 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82f8066f60 0x7f82f8069420 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8318151c20 0x7f83180cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8318151840 0x7f831814afb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 --2- 172.21.15.113:0/3435799558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f831806d140 0x7f83181e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 -- 172.21.15.113:0/3435799558 >> 172.21.15.113:0/3435799558 conn(0x7f83180c9a30 msgr2=0x7f831814fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 -- 172.21.15.113:0/3435799558 shutdown_connections 2024-03-04T07:44:32.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:32.537+0000 7f830cc00640 1 -- 172.21.15.113:0/3435799558 wait complete. 2024-03-04T07:44:32.568 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:44:34.569 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:34.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.797+0000 7fe33a200640 1 Processor -- start 2024-03-04T07:44:34.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.797+0000 7fe33a200640 1 -- start start 2024-03-04T07:44:34.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.797+0000 7fe33a200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe334157ef0 0x7fe3340c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:34.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.797+0000 7fe33a200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3340c9c50 con 0x7fe33406fb70 2024-03-04T07:44:34.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.797+0000 7fe33a200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33415a9a0 con 0x7fe334070df0 2024-03-04T07:44:34.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.797+0000 7fe33a200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33406de30 con 0x7fe334157ef0 2024-03-04T07:44:34.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe338e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe334157ef0 0x7fe3340c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:34.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe333e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fe334070df0 0x7fe33415b180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:53564/0 (socket says 172.21.15.113:53564) 2024-03-04T07:44:34.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe333e00640 1 -- 172.21.15.113:0/2309753543 learned_addr learned my addr 172.21.15.113:0/2309753543 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:34.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe339800640 1 --1- 172.21.15.113:0/2309753543 >> v1:172.21.15.113:6789/0 conn(0x7fe33406fb70 0x7fe3340cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39082/0 (socket says 172.21.15.113:39082) 2024-03-04T07:44:34.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe333400640 1 -- 172.21.15.113:0/2309753543 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809490164 0 0) 0x7fe3340c9c50 con 0x7fe33406fb70 2024-03-04T07:44:34.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe333400640 1 -- 172.21.15.113:0/2309753543 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe31c003530 con 0x7fe33406fb70 2024-03-04T07:44:34.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe333400640 1 -- 172.21.15.113:0/2309753543 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306266501 0 0) 0x7fe33415a9a0 con 0x7fe334070df0 2024-03-04T07:44:34.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe333400640 1 -- 172.21.15.113:0/2309753543 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3340c9c50 con 0x7fe334070df0 2024-03-04T07:44:34.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe338e00640 1 -- 172.21.15.113:0/2309753543 >> v1:172.21.15.136:6789/0 conn(0x7fe334070df0 legacy=0x7fe33415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:34.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe338e00640 1 -- 172.21.15.113:0/2309753543 >> v1:172.21.15.113:6789/0 conn(0x7fe33406fb70 legacy=0x7fe3340cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:34.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.801+0000 7fe338e00640 1 -- 172.21.15.113:0/2309753543 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33406c210 con 0x7fe334157ef0 2024-03-04T07:44:34.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe338e00640 1 --2- 172.21.15.113:0/2309753543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe334157ef0 0x7fe3340c8a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3280097f0 tx=0x7fe32801c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbaf23e18000251a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:34.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe333400640 1 -- 172.21.15.113:0/2309753543 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3280ab5f0 con 0x7fe334157ef0 2024-03-04T07:44:34.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe333400640 1 -- 172.21.15.113:0/2309753543 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3280b0070 con 0x7fe334157ef0 2024-03-04T07:44:34.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe333400640 1 -- 172.21.15.113:0/2309753543 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3280ab9b0 con 0x7fe334157ef0 2024-03-04T07:44:34.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe33a200640 1 -- 172.21.15.113:0/2309753543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe334157ef0 msgr2=0x7fe3340c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:34.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe33a200640 1 --2- 172.21.15.113:0/2309753543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe334157ef0 0x7fe3340c8a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3280097f0 tx=0x7fe32801c500 comp rx=0 tx=0).stop 2024-03-04T07:44:34.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe33a200640 1 -- 172.21.15.113:0/2309753543 shutdown_connections 2024-03-04T07:44:34.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe33a200640 1 --2- 172.21.15.113:0/2309753543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe334157ef0 0x7fe3340c8a90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:34.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe33a200640 1 -- 172.21.15.113:0/2309753543 >> 172.21.15.113:0/2309753543 conn(0x7fe334072f40 msgr2=0x7fe3340706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:34.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe33a200640 1 -- 172.21.15.113:0/2309753543 shutdown_connections 2024-03-04T07:44:34.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.805+0000 7fe33a200640 1 -- 172.21.15.113:0/2309753543 wait complete. 2024-03-04T07:44:34.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.809+0000 7fe33a200640 1 Processor -- start 2024-03-04T07:44:34.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.809+0000 7fe33a200640 1 -- start start 2024-03-04T07:44:34.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.809+0000 7fe33a200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe33406fb70 0x7fe3341488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:34.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.809+0000 7fe33a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe334070df0 0x7fe33414ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:34.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.809+0000 7fe33a200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe334157ef0 0x7fe334165130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:34.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.809+0000 7fe33a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3341582d0 con 0x7fe334070df0 2024-03-04T07:44:34.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.809+0000 7fe33a200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33415b890 con 0x7fe334157ef0 2024-03-04T07:44:34.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.809+0000 7fe33a200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe334158450 con 0x7fe33406fb70 2024-03-04T07:44:34.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe339800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe334157ef0 0x7fe334165130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:34.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe338e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe33406fb70 0x7fe3341488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:34.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe339800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe334157ef0 0x7fe334165130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:42926/0 (socket says 172.21.15.113:42926) 2024-03-04T07:44:34.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe339800640 1 -- 172.21.15.113:0/1059957808 learned_addr learned my addr 172.21.15.113:0/1059957808 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:34.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe333e00640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe334070df0 0x7fe33414ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:34.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe339800640 1 -- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe33406fb70 msgr2=0x7fe3341488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:34.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe339800640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe33406fb70 0x7fe3341488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:34.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe339800640 1 -- 172.21.15.113:0/1059957808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe334070df0 msgr2=0x7fe33414ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:34.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe339800640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe334070df0 0x7fe33414ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:34.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe339800640 1 -- 172.21.15.113:0/1059957808 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe334149a50 con 0x7fe334157ef0 2024-03-04T07:44:34.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe333e00640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe334070df0 0x7fe33414ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:44:34.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe339800640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe334157ef0 0x7fe334165130 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00c800 tx=0x7fe32c00ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:34.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe330c00640 1 -- 172.21.15.113:0/1059957808 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c017070 con 0x7fe334157ef0 2024-03-04T07:44:34.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe330c00640 1 -- 172.21.15.113:0/1059957808 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c00a640 con 0x7fe334157ef0 2024-03-04T07:44:34.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe330c00640 1 -- 172.21.15.113:0/1059957808 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c00a9f0 con 0x7fe334157ef0 2024-03-04T07:44:34.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe33a200640 1 -- 172.21.15.113:0/1059957808 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe334149d10 con 0x7fe334157ef0 2024-03-04T07:44:34.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe33a200640 1 -- 172.21.15.113:0/1059957808 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe334164930 con 0x7fe334157ef0 2024-03-04T07:44:34.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.813+0000 7fe33a200640 1 -- 172.21.15.113:0/1059957808 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe334075a80 con 0x7fe334157ef0 2024-03-04T07:44:34.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.817+0000 7fe330c00640 1 -- 172.21.15.113:0/1059957808 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe32c01a070 con 0x7fe334157ef0 2024-03-04T07:44:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.817+0000 7fe330c00640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 0x7fe314069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.817+0000 7fe330c00640 1 -- 172.21.15.113:0/1059957808 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fe32c084810 con 0x7fe334157ef0 2024-03-04T07:44:34.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.817+0000 7fe338e00640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 0x7fe314069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:34.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.821+0000 7fe338e00640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 0x7fe314069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe3280af000 tx=0x7fe3280a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:34.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:34.837+0000 7fe330c00640 1 -- 172.21.15.113:0/1059957808 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe32c054f50 con 0x7fe334157ef0 2024-03-04T07:44:35.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.229+0000 7fe33a200640 1 -- 172.21.15.113:0/1059957808 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe334157720 con 0x7fe334157ef0 2024-03-04T07:44:35.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.229+0000 7fe330c00640 1 -- 172.21.15.113:0/1059957808 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5819 (secure 0 0 0) 0x7fe32c058ff0 con 0x7fe334157ef0 2024-03-04T07:44:35.231 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:35.231 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:30.768700+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:reconnect","state_seq":39,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5099,5244]},"id":1}]} 2024-03-04T07:44:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 47 2024-03-04T07:44:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 -- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 msgr2=0x7fe314069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 0x7fe314069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe3280af000 tx=0x7fe3280a7000 comp rx=0 tx=0).stop 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 -- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe334157ef0 msgr2=0x7fe334165130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe334157ef0 0x7fe334165130 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00c800 tx=0x7fe32c00ccc0 comp rx=0 tx=0).stop 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 -- 172.21.15.113:0/1059957808 shutdown_connections 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe334157ef0 0x7fe334165130 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 0x7fe314069420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe334070df0 0x7fe33414ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 --2- 172.21.15.113:0/1059957808 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe33406fb70 0x7fe3341488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 -- 172.21.15.113:0/1059957808 >> 172.21.15.113:0/1059957808 conn(0x7fe334072f40 msgr2=0x7fe33415b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 -- 172.21.15.113:0/1059957808 shutdown_connections 2024-03-04T07:44:35.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:35.233+0000 7fe33a200640 1 -- 172.21.15.113:0/1059957808 wait complete. 2024-03-04T07:44:35.264 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 47, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 47, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5099": { "addr": "172.21.15.136:6835/3944001911", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3944001911, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3944001911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5099, "incarnation": 39, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 21 }, "gid_5244": { "addr": "172.21.15.113:6837/742633123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 742633123, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 742633123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5244, "incarnation": 32, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 26 }, "gid_5529": { "addr": "172.21.15.113:6836/2299268553", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2299268553, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2299268553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5529, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:reconnect", "state_seq": 39 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:44:30.768700+0000", "qdb_cluster": [ 5099, 5244 ], "qdb_leader": 5244, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5244, "mds_1": 5529, "mds_2": 5099 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:44:37.266 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:37.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646be00640 1 Processor -- start 2024-03-04T07:44:37.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646be00640 1 -- start start 2024-03-04T07:44:37.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646be00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f646c06faa0 0x7f646c15f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:37.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 0x7f646c0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:37.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646be00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646c06dc00 con 0x7f646c158f10 2024-03-04T07:44:37.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646be00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646c1592f0 con 0x7f646c06faa0 2024-03-04T07:44:37.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c06e150 con 0x7f646c06c170 2024-03-04T07:44:37.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 0x7f646c0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:37.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646aa00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f646c158f10 0x7f646c06ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:51686/0 (socket says 172.21.15.113:51686) 2024-03-04T07:44:37.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646aa00640 1 -- 172.21.15.113:0/1966567363 learned_addr learned my addr 172.21.15.113:0/1966567363 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:37.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646a000640 1 --2- 172.21.15.113:0/1966567363 >> v2:172.21.15.136:3300/0 conn(0x7f646c06faa0 0x7f646c15f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:37.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f6469600640 1 -- 172.21.15.113:0/1966567363 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624945543 0 0) 0x7f646c06dc00 con 0x7f646c158f10 2024-03-04T07:44:37.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f6469600640 1 -- 172.21.15.113:0/1966567363 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6450003530 con 0x7f646c158f10 2024-03-04T07:44:37.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646a000640 1 -- 172.21.15.113:0/1966567363 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 msgr2=0x7f646c0c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:37.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646a000640 1 --2- 172.21.15.113:0/1966567363 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 0x7f646c0c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:37.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646a000640 1 -- 172.21.15.113:0/1966567363 >> v1:172.21.15.136:6789/0 conn(0x7f646c158f10 legacy=0x7f646c06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:37.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646a000640 1 -- 172.21.15.113:0/1966567363 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c06de70 con 0x7f646c06faa0 2024-03-04T07:44:37.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.409+0000 7f646b400640 1 --2- 172.21.15.113:0/1966567363 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 0x7f646c0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:44:37.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f646a000640 1 --2- 172.21.15.113:0/1966567363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f646c06faa0 0x7f646c15f760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f64580097c0 tx=0x7f645801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aadde0b4ee27f41 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:37.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f6469600640 1 -- 172.21.15.113:0/1966567363 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64580b5070 con 0x7f646c06faa0 2024-03-04T07:44:37.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f6469600640 1 -- 172.21.15.113:0/1966567363 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64580a8990 con 0x7f646c06faa0 2024-03-04T07:44:37.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f6469600640 1 -- 172.21.15.113:0/1966567363 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64580a8d40 con 0x7f646c06faa0 2024-03-04T07:44:37.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f646be00640 1 -- 172.21.15.113:0/1966567363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f646c06faa0 msgr2=0x7f646c15f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:37.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f646be00640 1 --2- 172.21.15.113:0/1966567363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f646c06faa0 0x7f646c15f760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f64580097c0 tx=0x7f645801ec90 comp rx=0 tx=0).stop 2024-03-04T07:44:37.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f646be00640 1 -- 172.21.15.113:0/1966567363 shutdown_connections 2024-03-04T07:44:37.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f646be00640 1 --2- 172.21.15.113:0/1966567363 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 0x7f646c0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:37.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f646be00640 1 --2- 172.21.15.113:0/1966567363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f646c06faa0 0x7f646c15f760 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:37.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f646be00640 1 -- 172.21.15.113:0/1966567363 >> 172.21.15.113:0/1966567363 conn(0x7f646c072f40 msgr2=0x7f646c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:37.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f646be00640 1 -- 172.21.15.113:0/1966567363 shutdown_connections 2024-03-04T07:44:37.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.413+0000 7f646be00640 1 -- 172.21.15.113:0/1966567363 wait complete. 2024-03-04T07:44:37.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646be00640 1 Processor -- start 2024-03-04T07:44:37.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646be00640 1 -- start start 2024-03-04T07:44:37.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 0x7f646c0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:37.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f646c06faa0 0x7f646c15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:37.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f646c158f10 0x7f646c15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:37.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c15f240 con 0x7f646c06faa0 2024-03-04T07:44:37.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c15ef60 con 0x7f646c158f10 2024-03-04T07:44:37.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c06f490 con 0x7f646c06c170 2024-03-04T07:44:37.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646aa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 0x7f646c0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646aa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 0x7f646c0c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:40850/0 (socket says 172.21.15.113:40850) 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646aa00640 1 -- 172.21.15.113:0/3492223579 learned_addr learned my addr 172.21.15.113:0/3492223579 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646b400640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f646c158f10 0x7f646c15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646a000640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f646c06faa0 0x7f646c15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646b400640 1 -- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 msgr2=0x7f646c0c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646b400640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 0x7f646c0c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.417+0000 7f646b400640 1 -- 172.21.15.113:0/3492223579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f646c06faa0 msgr2=0x7f646c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f646b400640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f646c06faa0 0x7f646c15b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f646b400640 1 -- 172.21.15.113:0/3492223579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c1f4be0 con 0x7f646c158f10 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f646a000640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f646c06faa0 0x7f646c15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:44:37.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f646b400640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f646c158f10 0x7f646c15b7e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f646400efc0 tx=0x7f646400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:37.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f6462a00640 1 -- 172.21.15.113:0/3492223579 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464018030 con 0x7f646c158f10 2024-03-04T07:44:37.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f646be00640 1 -- 172.21.15.113:0/3492223579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646c1f4e80 con 0x7f646c158f10 2024-03-04T07:44:37.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f6462a00640 1 -- 172.21.15.113:0/3492223579 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646400cd30 con 0x7f646c158f10 2024-03-04T07:44:37.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f646be00640 1 -- 172.21.15.113:0/3492223579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c160950 con 0x7f646c158f10 2024-03-04T07:44:37.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f6462a00640 1 -- 172.21.15.113:0/3492223579 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464003bc0 con 0x7f646c158f10 2024-03-04T07:44:37.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f646be00640 1 -- 172.21.15.113:0/3492223579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646c075a80 con 0x7f646c158f10 2024-03-04T07:44:37.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.421+0000 7f6462a00640 1 -- 172.21.15.113:0/3492223579 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f646401b070 con 0x7f646c158f10 2024-03-04T07:44:37.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.425+0000 7f6462a00640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f644c066f60 0x7f644c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:37.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.425+0000 7f6462a00640 1 -- 172.21.15.113:0/3492223579 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f6464086300 con 0x7f646c158f10 2024-03-04T07:44:37.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.425+0000 7f646aa00640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f644c066f60 0x7f644c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:37.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.425+0000 7f646aa00640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f644c066f60 0x7f644c069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f645c007fd0 tx=0x7f645c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:37.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.433+0000 7f6462a00640 1 -- 172.21.15.113:0/3492223579 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6464010040 con 0x7f646c158f10 2024-03-04T07:44:37.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.629+0000 7f646be00640 1 -- 172.21.15.113:0/3492223579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f646c15c510 con 0x7f646c158f10 2024-03-04T07:44:37.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f6462a00640 1 -- 172.21.15.113:0/3492223579 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f64640860c0 con 0x7f646c158f10 2024-03-04T07:44:37.632 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:37.633 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:35.811511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:rejoin","state_seq":41,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099]},"id":1}]} 2024-03-04T07:44:37.633 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 48 2024-03-04T07:44:37.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 -- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f644c066f60 msgr2=0x7f644c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:37.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f644c066f60 0x7f644c069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f645c007fd0 tx=0x7f645c009040 comp rx=0 tx=0).stop 2024-03-04T07:44:37.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 -- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f646c158f10 msgr2=0x7f646c15b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:37.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f646c158f10 0x7f646c15b7e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f646400efc0 tx=0x7f646400c490 comp rx=0 tx=0).stop 2024-03-04T07:44:37.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 -- 172.21.15.113:0/3492223579 shutdown_connections 2024-03-04T07:44:37.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f646c158f10 0x7f646c15b7e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:37.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f644c066f60 0x7f644c069420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:37.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f646c06faa0 0x7f646c15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:37.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 --2- 172.21.15.113:0/3492223579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f646c06c170 0x7f646c0c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:37.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 -- 172.21.15.113:0/3492223579 >> 172.21.15.113:0/3492223579 conn(0x7f646c072f40 msgr2=0x7f646c15c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:37.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 -- 172.21.15.113:0/3492223579 shutdown_connections 2024-03-04T07:44:37.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:37.633+0000 7f646be00640 1 -- 172.21.15.113:0/3492223579 wait complete. 2024-03-04T07:44:37.662 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:44:39.664 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.829+0000 7f76bf200640 1 Processor -- start 2024-03-04T07:44:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.829+0000 7f76bf200640 1 -- start start 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f76b8155100 0x7f76b80c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bf200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76b806b060 0x7f76b80cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f76b80cd600 0x7f76b815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bf200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b80c8890 con 0x7f76b80cd600 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bf200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b815efd0 con 0x7f76b8155100 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bf200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80c7ef0 con 0x7f76b806b060 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bd200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f76b80cd600 0x7f76b815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bd200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f76b80cd600 0x7f76b815ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39886/0 (socket says 172.21.15.113:39886) 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bd200640 1 -- 172.21.15.113:0/884355102 learned_addr learned my addr 172.21.15.113:0/884355102 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76b7e00640 1 --2- 172.21.15.113:0/884355102 >> v2:172.21.15.136:3300/0 conn(0x7f76b8155100 0x7f76b80c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:39.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76b7400640 1 --2- 172.21.15.113:0/884355102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76b806b060 0x7f76b80cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:39.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bd200640 1 -- 172.21.15.113:0/884355102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76b806b060 msgr2=0x7f76b80cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:39.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bd200640 1 --2- 172.21.15.113:0/884355102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76b806b060 0x7f76b80cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:39.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bd200640 1 -- 172.21.15.113:0/884355102 >> v2:172.21.15.136:3300/0 conn(0x7f76b8155100 msgr2=0x7f76b80c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:39.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bd200640 1 --2- 172.21.15.113:0/884355102 >> v2:172.21.15.136:3300/0 conn(0x7f76b8155100 0x7f76b80c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:39.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.833+0000 7f76bd200640 1 -- 172.21.15.113:0/884355102 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b80c81d0 con 0x7f76b80cd600 2024-03-04T07:44:39.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.837+0000 7f76bd200640 1 --2- 172.21.15.113:0/884355102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b80cd600 0x7f76b815ad30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f76b806f3c0 tx=0x7f76b001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3888263606916c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:39.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.837+0000 7f76b6a00640 1 -- 172.21.15.113:0/884355102 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00a5670 con 0x7f76b80cd600 2024-03-04T07:44:39.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76b6a00640 1 -- 172.21.15.113:0/884355102 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b00a5810 con 0x7f76b80cd600 2024-03-04T07:44:39.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76b6a00640 1 -- 172.21.15.113:0/884355102 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b00a5be0 con 0x7f76b80cd600 2024-03-04T07:44:39.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 -- 172.21.15.113:0/884355102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b80cd600 msgr2=0x7f76b815ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:39.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 --2- 172.21.15.113:0/884355102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b80cd600 0x7f76b815ad30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f76b806f3c0 tx=0x7f76b001c9f0 comp rx=0 tx=0).stop 2024-03-04T07:44:39.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 -- 172.21.15.113:0/884355102 shutdown_connections 2024-03-04T07:44:39.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 --2- 172.21.15.113:0/884355102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b80cd600 0x7f76b815ad30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:39.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 --2- 172.21.15.113:0/884355102 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76b806b060 0x7f76b80cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:39.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 --2- 172.21.15.113:0/884355102 >> v2:172.21.15.136:3300/0 conn(0x7f76b8155100 0x7f76b80c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:39.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 -- 172.21.15.113:0/884355102 >> 172.21.15.113:0/884355102 conn(0x7f76b806deb0 msgr2=0x7f76b806e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:39.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 -- 172.21.15.113:0/884355102 shutdown_connections 2024-03-04T07:44:39.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 -- 172.21.15.113:0/884355102 wait complete. 2024-03-04T07:44:39.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 Processor -- start 2024-03-04T07:44:39.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.841+0000 7f76bf200640 1 -- start start 2024-03-04T07:44:39.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76bf200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b806b060 0x7f76b81543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:39.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76bf200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76b80cd600 0x7f76b814de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:39.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76bf200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b8155100 0x7f76b815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:39.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76bf200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b815ebb0 con 0x7f76b806b060 2024-03-04T07:44:39.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76bf200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b806b470 con 0x7f76b8155100 2024-03-04T07:44:39.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76bf200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b815ed30 con 0x7f76b80cd600 2024-03-04T07:44:39.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b806b060 0x7f76b81543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:39.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b806b060 0x7f76b81543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39888/0 (socket says 172.21.15.113:39888) 2024-03-04T07:44:39.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76b7e00640 1 -- 172.21.15.113:0/484210863 learned_addr learned my addr 172.21.15.113:0/484210863 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:39.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76bd200640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b8155100 0x7f76b815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:39.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76b7400640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76b80cd600 0x7f76b814de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:39.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76b7e00640 1 -- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76b80cd600 msgr2=0x7f76b814de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:39.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76b7e00640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76b80cd600 0x7f76b814de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:39.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76b7e00640 1 -- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b8155100 msgr2=0x7f76b815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:39.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76b7e00640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b8155100 0x7f76b815c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:39.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76b7e00640 1 -- 172.21.15.113:0/484210863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b815d940 con 0x7f76b806b060 2024-03-04T07:44:39.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76b7e00640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b806b060 0x7f76b81543a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e7f0 tx=0x7f76a000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:39.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76afe00640 1 -- 172.21.15.113:0/484210863 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0024070 con 0x7f76b806b060 2024-03-04T07:44:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76afe00640 1 -- 172.21.15.113:0/484210863 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a0017c60 con 0x7f76b806b060 2024-03-04T07:44:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76afe00640 1 -- 172.21.15.113:0/484210863 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a000c930 con 0x7f76b806b060 2024-03-04T07:44:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76bf200640 1 -- 172.21.15.113:0/484210863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b81548e0 con 0x7f76b806b060 2024-03-04T07:44:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76bf200640 1 -- 172.21.15.113:0/484210863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b815bec0 con 0x7f76b806b060 2024-03-04T07:44:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.845+0000 7f76bf200640 1 -- 172.21.15.113:0/484210863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b8075c00 con 0x7f76b806b060 2024-03-04T07:44:39.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.849+0000 7f76afe00640 1 -- 172.21.15.113:0/484210863 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76a0027070 con 0x7f76b806b060 2024-03-04T07:44:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.849+0000 7f76afe00640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7694066f90 0x7f7694069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.849+0000 7f76afe00640 1 -- 172.21.15.113:0/484210863 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f76a008d330 con 0x7f76b806b060 2024-03-04T07:44:39.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.849+0000 7f76b7400640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7694066f90 0x7f7694069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:39.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.849+0000 7f76b7400640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7694066f90 0x7f7694069450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f76b815d660 tx=0x7f76a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:39.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:39.853+0000 7f76afe00640 1 -- 172.21.15.113:0/484210863 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f76a005da70 con 0x7f76b806b060 2024-03-04T07:44:40.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 -- 172.21.15.113:0/484210863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76b80c73b0 con 0x7f76b806b060 2024-03-04T07:44:40.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76afe00640 1 -- 172.21.15.113:0/484210863 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f76a0061b10 con 0x7f76b806b060 2024-03-04T07:44:40.061 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:35.811511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:rejoin","state_seq":41,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5099]},"id":1}]} 2024-03-04T07:44:40.061 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 48 2024-03-04T07:44:40.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 -- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7694066f90 msgr2=0x7f7694069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:40.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7694066f90 0x7f7694069450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f76b815d660 tx=0x7f76a8008040 comp rx=0 tx=0).stop 2024-03-04T07:44:40.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 -- 172.21.15.113:0/484210863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b806b060 msgr2=0x7f76b81543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:40.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b806b060 0x7f76b81543a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f76a000e7f0 tx=0x7f76a000ecb0 comp rx=0 tx=0).stop 2024-03-04T07:44:40.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 -- 172.21.15.113:0/484210863 shutdown_connections 2024-03-04T07:44:40.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76b8155100 0x7f76b815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:40.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76b80cd600 0x7f76b814de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:40.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7694066f90 0x7f7694069450 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:40.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 --2- 172.21.15.113:0/484210863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76b806b060 0x7f76b81543a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:40.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 -- 172.21.15.113:0/484210863 >> 172.21.15.113:0/484210863 conn(0x7f76b806deb0 msgr2=0x7f76b806ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:40.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 -- 172.21.15.113:0/484210863 shutdown_connections 2024-03-04T07:44:40.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:40.061+0000 7f76bf200640 1 -- 172.21.15.113:0/484210863 wait complete. 2024-03-04T07:44:40.082 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:44:42.083 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:42.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.217+0000 7f22af400640 1 Processor -- start 2024-03-04T07:44:42.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.217+0000 7f22af400640 1 -- start start 2024-03-04T07:44:42.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22af400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22a8152b20 0x7f22a8070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:42.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22af400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f22a815c510 0x7f22a8163190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:42.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22af400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a8152f00 con 0x7f22a80c8b00 2024-03-04T07:44:42.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22af400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a80c8f90 con 0x7f22a8152b20 2024-03-04T07:44:42.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22af400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a80c5320 con 0x7f22a815c510 2024-03-04T07:44:42.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22aca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22a8152b20 0x7f22a8070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:42.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22aca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22a8152b20 0x7f22a8070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39900/0 (socket says 172.21.15.113:39900) 2024-03-04T07:44:42.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22aca00640 1 -- 172.21.15.113:0/2488466916 learned_addr learned my addr 172.21.15.113:0/2488466916 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:42.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22ad400640 1 --2- 172.21.15.113:0/2488466916 >> v2:172.21.15.136:3300/0 conn(0x7f22a815c510 0x7f22a8163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:42.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22aca00640 1 -- 172.21.15.113:0/2488466916 >> v2:172.21.15.136:3300/0 conn(0x7f22a815c510 msgr2=0x7f22a8163190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:42.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22aca00640 1 --2- 172.21.15.113:0/2488466916 >> v2:172.21.15.136:3300/0 conn(0x7f22a815c510 0x7f22a8163190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:42.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22aca00640 1 -- 172.21.15.113:0/2488466916 >> v1:172.21.15.136:6789/0 conn(0x7f22a80c8b00 legacy=0x7f22a815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:42.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.221+0000 7f22aca00640 1 -- 172.21.15.113:0/2488466916 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a80c5040 con 0x7f22a8152b20 2024-03-04T07:44:42.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22aca00640 1 --2- 172.21.15.113:0/2488466916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a8152b20 0x7f22a8070010 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f22a8072800 tx=0x7f229c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b191434caa957a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:42.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22a7400640 1 -- 172.21.15.113:0/2488466916 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0b0070 con 0x7f22a8152b20 2024-03-04T07:44:42.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22a7400640 1 -- 172.21.15.113:0/2488466916 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c0ab600 con 0x7f22a8152b20 2024-03-04T07:44:42.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22a7400640 1 -- 172.21.15.113:0/2488466916 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0ab9d0 con 0x7f22a8152b20 2024-03-04T07:44:42.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 -- 172.21.15.113:0/2488466916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a8152b20 msgr2=0x7f22a8070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:42.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 --2- 172.21.15.113:0/2488466916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a8152b20 0x7f22a8070010 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f22a8072800 tx=0x7f229c01c520 comp rx=0 tx=0).stop 2024-03-04T07:44:42.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 -- 172.21.15.113:0/2488466916 shutdown_connections 2024-03-04T07:44:42.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 --2- 172.21.15.113:0/2488466916 >> v2:172.21.15.136:3300/0 conn(0x7f22a815c510 0x7f22a8163190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:42.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 --2- 172.21.15.113:0/2488466916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a8152b20 0x7f22a8070010 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:42.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 -- 172.21.15.113:0/2488466916 >> 172.21.15.113:0/2488466916 conn(0x7f22a80709a0 msgr2=0x7f22a8070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:42.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 -- 172.21.15.113:0/2488466916 shutdown_connections 2024-03-04T07:44:42.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 -- 172.21.15.113:0/2488466916 wait complete. 2024-03-04T07:44:42.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 Processor -- start 2024-03-04T07:44:42.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 -- start start 2024-03-04T07:44:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a80c8b00 0x7f22a815b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a8152b20 0x7f22a806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a815c510 0x7f22a806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a8158810 con 0x7f22a815c510 2024-03-04T07:44:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80c3810 con 0x7f22a8152b20 2024-03-04T07:44:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.225+0000 7f22af400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80c8490 con 0x7f22a80c8b00 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22ad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a815c510 0x7f22a806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22ad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a815c510 0x7f22a806da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39908/0 (socket says 172.21.15.113:39908) 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22ad400640 1 -- 172.21.15.113:0/1653904379 learned_addr learned my addr 172.21.15.113:0/1653904379 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22aca00640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a80c8b00 0x7f22a815b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22a7e00640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a8152b20 0x7f22a806d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22ad400640 1 -- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a80c8b00 msgr2=0x7f22a815b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22ad400640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a80c8b00 0x7f22a815b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22ad400640 1 -- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a8152b20 msgr2=0x7f22a806d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22ad400640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a8152b20 0x7f22a806d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22ad400640 1 -- 172.21.15.113:0/1653904379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a81f7d10 con 0x7f22a815c510 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22ad400640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a815c510 0x7f22a806da10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f22a000efc0 tx=0x7f22a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22a4c00640 1 -- 172.21.15.113:0/1653904379 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0018020 con 0x7f22a815c510 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22a4c00640 1 -- 172.21.15.113:0/1653904379 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a0014480 con 0x7f22a815c510 2024-03-04T07:44:42.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22af400640 1 -- 172.21.15.113:0/1653904379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a80cca40 con 0x7f22a815c510 2024-03-04T07:44:42.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22a4c00640 1 -- 172.21.15.113:0/1653904379 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0014870 con 0x7f22a815c510 2024-03-04T07:44:42.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22af400640 1 -- 172.21.15.113:0/1653904379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a806ccd0 con 0x7f22a815c510 2024-03-04T07:44:42.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.229+0000 7f22af400640 1 -- 172.21.15.113:0/1653904379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a8075c00 con 0x7f22a815c510 2024-03-04T07:44:42.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.233+0000 7f22a4c00640 1 -- 172.21.15.113:0/1653904379 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22a001b020 con 0x7f22a815c510 2024-03-04T07:44:42.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.233+0000 7f22a4c00640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2288066f60 0x7f2288069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:42.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.233+0000 7f22a4c00640 1 -- 172.21.15.113:0/1653904379 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f22a0085be0 con 0x7f22a815c510 2024-03-04T07:44:42.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.233+0000 7f22aca00640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2288066f60 0x7f2288069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:42.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.233+0000 7f22aca00640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2288066f60 0x7f2288069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f229c0ad000 tx=0x7f229c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:42.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.237+0000 7f22a4c00640 1 -- 172.21.15.113:0/1653904379 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f22a0022070 con 0x7f22a815c510 2024-03-04T07:44:42.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.433+0000 7f22af400640 1 -- 172.21.15.113:0/1653904379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22a806c750 con 0x7f22a815c510 2024-03-04T07:44:42.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.433+0000 7f22a4c00640 1 -- 172.21.15.113:0/1653904379 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5821 (secure 0 0 0) 0x7f22a0010040 con 0x7f22a815c510 2024-03-04T07:44:42.434 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:42.435 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:40.846373+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5099,5244,5529]},"id":1}]} 2024-03-04T07:44:42.435 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 49 2024-03-04T07:44:42.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 -- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2288066f60 msgr2=0x7f2288069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:42.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2288066f60 0x7f2288069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f229c0ad000 tx=0x7f229c008040 comp rx=0 tx=0).stop 2024-03-04T07:44:42.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 -- 172.21.15.113:0/1653904379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a815c510 msgr2=0x7f22a806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:42.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a815c510 0x7f22a806da10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f22a000efc0 tx=0x7f22a000c490 comp rx=0 tx=0).stop 2024-03-04T07:44:42.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 -- 172.21.15.113:0/1653904379 shutdown_connections 2024-03-04T07:44:42.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2288066f60 0x7f2288069420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:42.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22a815c510 0x7f22a806da10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:42.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22a8152b20 0x7f22a806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:42.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 --2- 172.21.15.113:0/1653904379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22a80c8b00 0x7f22a815b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:42.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 -- 172.21.15.113:0/1653904379 >> 172.21.15.113:0/1653904379 conn(0x7f22a80709a0 msgr2=0x7f22a814f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:42.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 -- 172.21.15.113:0/1653904379 shutdown_connections 2024-03-04T07:44:42.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:42.437+0000 7f22af400640 1 -- 172.21.15.113:0/1653904379 wait complete. 2024-03-04T07:44:42.457 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=1, replacing gid=5094 2024-03-04T07:44:42.457 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.c 2024-03-04T07:44:49.458 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-03-04T07:44:49.459 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-04T07:44:49.459 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-04T07:44:49.463 INFO:tasks.ceph.mds.c:Started 2024-03-04T07:44:49.463 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:49.570 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-03-04T07:44:49.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.629+0000 7f770a400640 1 Processor -- start 2024-03-04T07:44:49.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.629+0000 7f770a400640 1 -- start start 2024-03-04T07:44:49.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f770a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77040c18e0 0x7f77040c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:49.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f770a400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770415bac0 con 0x7f7704154e20 2024-03-04T07:44:49.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f770a400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77040c7e10 con 0x7f770406d520 2024-03-04T07:44:49.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f770a400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77040c80f0 con 0x7f77040c18e0 2024-03-04T07:44:49.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f7709000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7704154e20 0x7f770406ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53810/0 (socket says 172.21.15.113:53810) 2024-03-04T07:44:49.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f7709000640 1 -- 172.21.15.113:0/4199150524 learned_addr learned my addr 172.21.15.113:0/4199150524 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:49.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f7709a00640 1 --2- 172.21.15.113:0/4199150524 >> v2:172.21.15.113:3300/0 conn(0x7f77040c18e0 0x7f77040c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:49.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f7709a00640 1 -- 172.21.15.113:0/4199150524 >> v1:172.21.15.136:6789/0 conn(0x7f770406d520 legacy=0x7f770415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:49.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f7709a00640 1 -- 172.21.15.113:0/4199150524 >> v1:172.21.15.113:6789/0 conn(0x7f7704154e20 legacy=0x7f770406ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:49.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f7709a00640 1 -- 172.21.15.113:0/4199150524 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040c83d0 con 0x7f77040c18e0 2024-03-04T07:44:49.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.633+0000 7f7703400640 1 -- 172.21.15.113:0/4199150524 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232074877 0 0) 0x7f770415bac0 con 0x7f7704154e20 2024-03-04T07:44:49.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f7709a00640 1 --2- 172.21.15.113:0/4199150524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77040c18e0 0x7f77040c1d10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009d90 tx=0x7f76fc01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23393cca78dc6b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:49.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f7703400640 1 -- 172.21.15.113:0/4199150524 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0b4070 con 0x7f77040c18e0 2024-03-04T07:44:49.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f7703400640 1 -- 172.21.15.113:0/4199150524 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76fc0af600 con 0x7f77040c18e0 2024-03-04T07:44:49.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f7703400640 1 -- 172.21.15.113:0/4199150524 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc0af9d0 con 0x7f77040c18e0 2024-03-04T07:44:49.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f770a400640 1 -- 172.21.15.113:0/4199150524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77040c18e0 msgr2=0x7f77040c1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:49.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f770a400640 1 --2- 172.21.15.113:0/4199150524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77040c18e0 0x7f77040c1d10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009d90 tx=0x7f76fc01f150 comp rx=0 tx=0).stop 2024-03-04T07:44:49.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f770a400640 1 -- 172.21.15.113:0/4199150524 shutdown_connections 2024-03-04T07:44:49.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f770a400640 1 --2- 172.21.15.113:0/4199150524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77040c18e0 0x7f77040c1d10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:49.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f770a400640 1 -- 172.21.15.113:0/4199150524 >> 172.21.15.113:0/4199150524 conn(0x7f770406e2a0 msgr2=0x7f770406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:49.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f770a400640 1 -- 172.21.15.113:0/4199150524 shutdown_connections 2024-03-04T07:44:49.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f770a400640 1 -- 172.21.15.113:0/4199150524 wait complete. 2024-03-04T07:44:49.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f770a400640 1 Processor -- start 2024-03-04T07:44:49.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.637+0000 7f770a400640 1 -- start start 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f770a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f770406d520 0x7f7704071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f770a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77040c18e0 0x7f7704071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f770a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7704154e20 0x7f77040cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f770a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770406c230 con 0x7f770406d520 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f770a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770415ae90 con 0x7f7704154e20 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f770a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770415ab60 con 0x7f77040c18e0 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f770406d520 0x7f7704071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7703e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77040c18e0 0x7f7704071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f770406d520 0x7f7704071d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35562/0 (socket says 172.21.15.113:35562) 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709000640 1 -- 172.21.15.113:0/2788255783 learned_addr learned my addr 172.21.15.113:0/2788255783 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709a00640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7704154e20 0x7f77040cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709000640 1 -- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77040c18e0 msgr2=0x7f7704071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709000640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77040c18e0 0x7f7704071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709000640 1 -- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7704154e20 msgr2=0x7f77040cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709000640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7704154e20 0x7f77040cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709000640 1 -- 172.21.15.113:0/2788255783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77041f1330 con 0x7f770406d520 2024-03-04T07:44:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709a00640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7704154e20 0x7f77040cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:44:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7709000640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f770406d520 0x7f7704071d30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f76f800c440 tx=0x7f76f800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7700c00640 1 -- 172.21.15.113:0/2788255783 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8018070 con 0x7f770406d520 2024-03-04T07:44:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7700c00640 1 -- 172.21.15.113:0/2788255783 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f80074e0 con 0x7f770406d520 2024-03-04T07:44:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7700c00640 1 -- 172.21.15.113:0/2788255783 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8004a10 con 0x7f770406d520 2024-03-04T07:44:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f770a400640 1 -- 172.21.15.113:0/2788255783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77041f15d0 con 0x7f770406d520 2024-03-04T07:44:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f770a400640 1 -- 172.21.15.113:0/2788255783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7704072840 con 0x7f770406d520 2024-03-04T07:44:49.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f770a400640 1 -- 172.21.15.113:0/2788255783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7704075a80 con 0x7f770406d520 2024-03-04T07:44:49.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.641+0000 7f7700c00640 1 -- 172.21.15.113:0/2788255783 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76f8004bb0 con 0x7f770406d520 2024-03-04T07:44:49.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.645+0000 7f7700c00640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76e4066f60 0x7f76e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:49.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.645+0000 7f7700c00640 1 -- 172.21.15.113:0/2788255783 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f76f8084460 con 0x7f770406d520 2024-03-04T07:44:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.645+0000 7f7703e00640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76e4066f60 0x7f76e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.645+0000 7f7703e00640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76e4066f60 0x7f76e4069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f77040cb800 tx=0x7f76f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:49.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.649+0000 7f7700c00640 1 -- 172.21.15.113:0/2788255783 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f76f8054b90 con 0x7f770406d520 2024-03-04T07:44:49.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 -- 172.21.15.113:0/2788255783 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7704159a30 con 0x7f770406d520 2024-03-04T07:44:49.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f7700c00640 1 -- 172.21.15.113:0/2788255783 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5821 (secure 0 0 0) 0x7f76f8058c30 con 0x7f770406d520 2024-03-04T07:44:49.853 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:49.853 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:40.846373+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5099,5244,5529]},"id":1}]} 2024-03-04T07:44:49.853 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 49 2024-03-04T07:44:49.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 -- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76e4066f60 msgr2=0x7f76e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:49.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76e4066f60 0x7f76e4069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f77040cb800 tx=0x7f76f0009040 comp rx=0 tx=0).stop 2024-03-04T07:44:49.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 -- 172.21.15.113:0/2788255783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f770406d520 msgr2=0x7f7704071d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:49.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f770406d520 0x7f7704071d30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f76f800c440 tx=0x7f76f800c900 comp rx=0 tx=0).stop 2024-03-04T07:44:49.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 -- 172.21.15.113:0/2788255783 shutdown_connections 2024-03-04T07:44:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7704154e20 0x7f77040cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77040c18e0 0x7f7704071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76e4066f60 0x7f76e4069420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 --2- 172.21.15.113:0/2788255783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f770406d520 0x7f7704071d30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 -- 172.21.15.113:0/2788255783 >> 172.21.15.113:0/2788255783 conn(0x7f770406e2a0 msgr2=0x7f770406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 -- 172.21.15.113:0/2788255783 shutdown_connections 2024-03-04T07:44:49.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:49.853+0000 7f770a400640 1 -- 172.21.15.113:0/2788255783 wait complete. 2024-03-04T07:44:49.879 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-03-04T07:44:51.880 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:44:52.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.053+0000 7f11cb800640 1 Processor -- start 2024-03-04T07:44:52.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.053+0000 7f11cb800640 1 -- start start 2024-03-04T07:44:52.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11cb800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f11c40ce2d0 0x7f11c415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:52.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11cb800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f11c406de80 0x7f11c4154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11cb800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c406c210 con 0x7f11c40ce6b0 2024-03-04T07:44:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11cb800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c415bbf0 con 0x7f11c40ce2d0 2024-03-04T07:44:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11cb800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c406eb80 con 0x7f11c406de80 2024-03-04T07:44:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11c8e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f11c40ce2d0 0x7f11c415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11c3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f11c40ce6b0 0x7f11c406f010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53818/0 (socket says 172.21.15.113:53818) 2024-03-04T07:44:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11c3e00640 1 -- 172.21.15.113:0/3621836121 learned_addr learned my addr 172.21.15.113:0/3621836121 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:44:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11c9800640 1 --2- 172.21.15.113:0/3621836121 >> v2:172.21.15.136:3300/0 conn(0x7f11c406de80 0x7f11c4154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11c8e00640 1 -- 172.21.15.113:0/3621836121 >> v2:172.21.15.136:3300/0 conn(0x7f11c406de80 msgr2=0x7f11c4154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:44:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11c8e00640 1 --2- 172.21.15.113:0/3621836121 >> v2:172.21.15.136:3300/0 conn(0x7f11c406de80 0x7f11c4154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11c8e00640 1 -- 172.21.15.113:0/3621836121 >> v1:172.21.15.113:6789/0 conn(0x7f11c40ce6b0 legacy=0x7f11c406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11c8e00640 1 -- 172.21.15.113:0/3621836121 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c406c4f0 con 0x7f11c40ce2d0 2024-03-04T07:44:52.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.057+0000 7f11c3400640 1 -- 172.21.15.113:0/3621836121 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228457183 0 0) 0x7f11c406c210 con 0x7f11c40ce6b0 2024-03-04T07:44:52.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11c8e00640 1 --2- 172.21.15.113:0/3621836121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 0x7f11c415b6b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f11c4072830 tx=0x7f11b801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527bcc22b14004ad server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:52.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11c3400640 1 -- 172.21.15.113:0/3621836121 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80b5070 con 0x7f11c40ce2d0 2024-03-04T07:44:52.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11c3400640 1 -- 172.21.15.113:0/3621836121 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b80a8970 con 0x7f11c40ce2d0 2024-03-04T07:44:52.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11c3400640 1 -- 172.21.15.113:0/3621836121 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b80a8d40 con 0x7f11c40ce2d0 2024-03-04T07:44:52.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11cb800640 1 -- 172.21.15.113:0/3621836121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 msgr2=0x7f11c415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:52.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11cb800640 1 --2- 172.21.15.113:0/3621836121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 0x7f11c415b6b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f11c4072830 tx=0x7f11b801ec70 comp rx=0 tx=0).stop 2024-03-04T07:44:52.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11cb800640 1 -- 172.21.15.113:0/3621836121 shutdown_connections 2024-03-04T07:44:52.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11cb800640 1 --2- 172.21.15.113:0/3621836121 >> v2:172.21.15.136:3300/0 conn(0x7f11c406de80 0x7f11c4154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:52.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11cb800640 1 --2- 172.21.15.113:0/3621836121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 0x7f11c415b6b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:52.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11cb800640 1 -- 172.21.15.113:0/3621836121 >> 172.21.15.113:0/3621836121 conn(0x7f11c40709a0 msgr2=0x7f11c4070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:52.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11cb800640 1 -- 172.21.15.113:0/3621836121 shutdown_connections 2024-03-04T07:44:52.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.061+0000 7f11cb800640 1 -- 172.21.15.113:0/3621836121 wait complete. 2024-03-04T07:44:52.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11cb800640 1 Processor -- start 2024-03-04T07:44:52.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11cb800640 1 -- start start 2024-03-04T07:44:52.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11cb800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c406de80 0x7f11c41f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:52.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11cb800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 0x7f11c41577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:52.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11cb800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c40ce6b0 0x7f11c414e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:52.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11cb800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c406c7d0 con 0x7f11c40ce2d0 2024-03-04T07:44:52.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11cb800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c406e810 con 0x7f11c406de80 2024-03-04T07:44:52.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11cb800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c406f720 con 0x7f11c40ce6b0 2024-03-04T07:44:52.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 0x7f11c41577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:52.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c406de80 0x7f11c41f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:52.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 0x7f11c41577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35588/0 (socket says 172.21.15.113:35588) 2024-03-04T07:44:52.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c3e00640 1 -- 172.21.15.113:0/2316169526 learned_addr learned my addr 172.21.15.113:0/2316169526 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:44:52.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c9800640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c40ce6b0 0x7f11c414e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c3e00640 1 -- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c40ce6b0 msgr2=0x7f11c414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c3e00640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c40ce6b0 0x7f11c414e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c3e00640 1 -- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c406de80 msgr2=0x7f11c41f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c3e00640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c406de80 0x7f11c41f4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c3e00640 1 -- 172.21.15.113:0/2316169526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c414e150 con 0x7f11c40ce2d0 2024-03-04T07:44:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c3e00640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 0x7f11c41577c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f11b00011f0 tx=0x7f11b000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c0c00640 1 -- 172.21.15.113:0/2316169526 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0018070 con 0x7f11c40ce2d0 2024-03-04T07:44:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c0c00640 1 -- 172.21.15.113:0/2316169526 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b00074e0 con 0x7f11c40ce2d0 2024-03-04T07:44:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11c0c00640 1 -- 172.21.15.113:0/2316169526 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0004a10 con 0x7f11c40ce2d0 2024-03-04T07:44:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.065+0000 7f11cb800640 1 -- 172.21.15.113:0/2316169526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c4153300 con 0x7f11c40ce2d0 2024-03-04T07:44:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.069+0000 7f11cb800640 1 -- 172.21.15.113:0/2316169526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c414e3f0 con 0x7f11c40ce2d0 2024-03-04T07:44:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.069+0000 7f11cb800640 1 -- 172.21.15.113:0/2316169526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c4075c00 con 0x7f11c40ce2d0 2024-03-04T07:44:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.069+0000 7f11c0c00640 1 -- 172.21.15.113:0/2316169526 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11b0004bb0 con 0x7f11c40ce2d0 2024-03-04T07:44:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.073+0000 7f11c0c00640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a4066f60 0x7f11a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:44:52.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.073+0000 7f11c0c00640 1 -- 172.21.15.113:0/2316169526 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f11b00846b0 con 0x7f11c40ce2d0 2024-03-04T07:44:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.073+0000 7f11c8e00640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a4066f60 0x7f11a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:44:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.073+0000 7f11c8e00640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a4066f60 0x7f11a4069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f11b80b2000 tx=0x7f11b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:44:52.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.077+0000 7f11c0c00640 1 -- 172.21.15.113:0/2316169526 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f11b0054cc0 con 0x7f11c40ce2d0 2024-03-04T07:44:52.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.273+0000 7f11cb800640 1 -- 172.21.15.113:0/2316169526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11c406cb40 con 0x7f11c40ce2d0 2024-03-04T07:44:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.273+0000 7f11c0c00640 1 -- 172.21.15.113:0/2316169526 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f11b0058d60 con 0x7f11c40ce2d0 2024-03-04T07:44:52.274 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:44:52.274 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:40.846373+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244,5099]},"id":1}]} 2024-03-04T07:44:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-03-04T07:44:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 -- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a4066f60 msgr2=0x7f11a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a4066f60 0x7f11a4069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f11b80b2000 tx=0x7f11b8008040 comp rx=0 tx=0).stop 2024-03-04T07:44:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 -- 172.21.15.113:0/2316169526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 msgr2=0x7f11c41577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:44:52.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 0x7f11c41577c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f11b00011f0 tx=0x7f11b000c9b0 comp rx=0 tx=0).stop 2024-03-04T07:44:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 -- 172.21.15.113:0/2316169526 shutdown_connections 2024-03-04T07:44:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11c40ce6b0 0x7f11c414e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f11a4066f60 0x7f11a4069420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11c40ce2d0 0x7f11c41577c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 --2- 172.21.15.113:0/2316169526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11c406de80 0x7f11c41f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:44:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 -- 172.21.15.113:0/2316169526 >> 172.21.15.113:0/2316169526 conn(0x7f11c40709a0 msgr2=0x7f11c415ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:44:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 -- 172.21.15.113:0/2316169526 shutdown_connections 2024-03-04T07:44:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:44:52.277+0000 7f11cb800640 1 -- 172.21.15.113:0/2316169526 wait complete. 2024-03-04T07:44:52.295 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-03-04T07:44:52.295 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 16 secs before thrashing 2024-03-04T07:44:57.197 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:44:57.200 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:45:08.296 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bd7000640 1 Processor -- start 2024-03-04T07:45:08.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bd7000640 1 -- start start 2024-03-04T07:45:08.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bd7000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 0x7f6bd0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bd7000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd01591f0 con 0x7f6bd00c8200 2024-03-04T07:45:08.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bd7000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd01594d0 con 0x7f6bd0152bd0 2024-03-04T07:45:08.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bd7000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd0071df0 con 0x7f6bd01527f0 2024-03-04T07:45:08.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bc7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6bd00c8200 0x7f6bd0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55784/0 (socket says 172.21.15.113:55784) 2024-03-04T07:45:08.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bc7e00640 1 -- 172.21.15.113:0/3030837550 learned_addr learned my addr 172.21.15.113:0/3030837550 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:45:08.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bcfe00640 1 --2- 172.21.15.113:0/3030837550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 0x7f6bd0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bcf400640 1 -- 172.21.15.113:0/3030837550 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719114607 0 0) 0x7f6bd01591f0 con 0x7f6bd00c8200 2024-03-04T07:45:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bcf400640 1 -- 172.21.15.113:0/3030837550 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb4003530 con 0x7f6bd00c8200 2024-03-04T07:45:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bcf400640 1 -- 172.21.15.113:0/3030837550 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157902860 0 0) 0x7f6bd01594d0 con 0x7f6bd0152bd0 2024-03-04T07:45:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bcf400640 1 -- 172.21.15.113:0/3030837550 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd01591f0 con 0x7f6bd0152bd0 2024-03-04T07:45:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bcfe00640 1 -- 172.21.15.113:0/3030837550 >> v1:172.21.15.136:6789/0 conn(0x7f6bd0152bd0 legacy=0x7f6bd01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bcfe00640 1 -- 172.21.15.113:0/3030837550 >> v1:172.21.15.113:6789/0 conn(0x7f6bd00c8200 legacy=0x7f6bd0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.437+0000 7f6bcfe00640 1 -- 172.21.15.113:0/3030837550 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd00717a0 con 0x7f6bd01527f0 2024-03-04T07:45:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bcfe00640 1 --2- 172.21.15.113:0/3030837550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 0x7f6bd0070d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00097f0 tx=0x7f6bc001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e371539bbc8e78e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bcf400640 1 -- 172.21.15.113:0/3030837550 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc00ab5f0 con 0x7f6bd01527f0 2024-03-04T07:45:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bcf400640 1 -- 172.21.15.113:0/3030837550 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc00b0070 con 0x7f6bd01527f0 2024-03-04T07:45:08.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bcf400640 1 -- 172.21.15.113:0/3030837550 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc00ab990 con 0x7f6bd01527f0 2024-03-04T07:45:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bd7000640 1 -- 172.21.15.113:0/3030837550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 msgr2=0x7f6bd0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bd7000640 1 --2- 172.21.15.113:0/3030837550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 0x7f6bd0070d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00097f0 tx=0x7f6bc001c4e0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bd7000640 1 -- 172.21.15.113:0/3030837550 shutdown_connections 2024-03-04T07:45:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bd7000640 1 --2- 172.21.15.113:0/3030837550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 0x7f6bd0070d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bd7000640 1 -- 172.21.15.113:0/3030837550 >> 172.21.15.113:0/3030837550 conn(0x7f6bd006e820 msgr2=0x7f6bd006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bd7000640 1 -- 172.21.15.113:0/3030837550 shutdown_connections 2024-03-04T07:45:08.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.441+0000 7f6bd7000640 1 -- 172.21.15.113:0/3030837550 wait complete. 2024-03-04T07:45:08.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd7000640 1 Processor -- start 2024-03-04T07:45:08.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd7000640 1 -- start start 2024-03-04T07:45:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd7000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bd00c8200 0x7f6bd0158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd7000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 0x7f6bd01f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd7000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0152bd0 0x7f6bd0162dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd7000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd0166380 con 0x7f6bd0152bd0 2024-03-04T07:45:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd7000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd01597b0 con 0x7f6bd00c8200 2024-03-04T07:45:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd7000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd0166500 con 0x7f6bd01527f0 2024-03-04T07:45:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0152bd0 0x7f6bd0162dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bcfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bd00c8200 0x7f6bd0158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0152bd0 0x7f6bd0162dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60986/0 (socket says 172.21.15.113:60986) 2024-03-04T07:45:08.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bd5000640 1 -- 172.21.15.113:0/2479216832 learned_addr learned my addr 172.21.15.113:0/2479216832 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.445+0000 7f6bc7e00640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 0x7f6bd01f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bd5000640 1 -- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 msgr2=0x7f6bd01f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bd5000640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 0x7f6bd01f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bd5000640 1 -- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bd00c8200 msgr2=0x7f6bd0158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bd5000640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bd00c8200 0x7f6bd0158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bd5000640 1 -- 172.21.15.113:0/2479216832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd0157780 con 0x7f6bd0152bd0 2024-03-04T07:45:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bd5000640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0152bd0 0x7f6bd0162dc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800c760 tx=0x7f6bc800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bccc00640 1 -- 172.21.15.113:0/2479216832 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8003b80 con 0x7f6bd0152bd0 2024-03-04T07:45:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bccc00640 1 -- 172.21.15.113:0/2479216832 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc8003d20 con 0x7f6bd0152bd0 2024-03-04T07:45:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bd7000640 1 -- 172.21.15.113:0/2479216832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd0157a60 con 0x7f6bd0152bd0 2024-03-04T07:45:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bccc00640 1 -- 172.21.15.113:0/2479216832 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800aa90 con 0x7f6bd0152bd0 2024-03-04T07:45:08.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bd7000640 1 -- 172.21.15.113:0/2479216832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd0158020 con 0x7f6bd0152bd0 2024-03-04T07:45:08.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bd7000640 1 -- 172.21.15.113:0/2479216832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd0075c00 con 0x7f6bd0152bd0 2024-03-04T07:45:08.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.449+0000 7f6bccc00640 1 -- 172.21.15.113:0/2479216832 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6bc800ac30 con 0x7f6bd0152bd0 2024-03-04T07:45:08.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.453+0000 7f6bccc00640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bac066f60 0x7f6bac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.453+0000 7f6bccc00640 1 -- 172.21.15.113:0/2479216832 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f6bc80850d0 con 0x7f6bd0152bd0 2024-03-04T07:45:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.453+0000 7f6bcfe00640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bac066f60 0x7f6bac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.453+0000 7f6bcfe00640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bac066f60 0x7f6bac069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00af000 tx=0x7f6bc00a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:08.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.461+0000 7f6bccc00640 1 -- 172.21.15.113:0/2479216832 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6bc8055810 con 0x7f6bd0152bd0 2024-03-04T07:45:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.721+0000 7f6bd7000640 1 -- 172.21.15.113:0/2479216832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bd014c8b0 con 0x7f6bd0152bd0 2024-03-04T07:45:08.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.721+0000 7f6bccc00640 1 -- 172.21.15.113:0/2479216832 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f6bc80598b0 con 0x7f6bd0152bd0 2024-03-04T07:45:08.722 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:08.722 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:40.846373+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244,5099]},"id":1}]} 2024-03-04T07:45:08.722 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-03-04T07:45:08.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 -- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bac066f60 msgr2=0x7f6bac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bac066f60 0x7f6bac069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00af000 tx=0x7f6bc00a7000 comp rx=0 tx=0).stop 2024-03-04T07:45:08.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 -- 172.21.15.113:0/2479216832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0152bd0 msgr2=0x7f6bd0162dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0152bd0 0x7f6bd0162dc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800c760 tx=0x7f6bc800cc20 comp rx=0 tx=0).stop 2024-03-04T07:45:08.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 -- 172.21.15.113:0/2479216832 shutdown_connections 2024-03-04T07:45:08.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bac066f60 0x7f6bac069420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6bd0152bd0 0x7f6bd0162dc0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6bd01527f0 0x7f6bd01f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 --2- 172.21.15.113:0/2479216832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6bd00c8200 0x7f6bd0158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 -- 172.21.15.113:0/2479216832 >> 172.21.15.113:0/2479216832 conn(0x7f6bd006e820 msgr2=0x7f6bd00c1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:08.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 -- 172.21.15.113:0/2479216832 shutdown_connections 2024-03-04T07:45:08.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.725+0000 7f6bd7000640 1 -- 172.21.15.113:0/2479216832 wait complete. 2024-03-04T07:45:08.744 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=2) 2024-03-04T07:45:08.745 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-04T07:45:08.745 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:45:08.758 INFO:tasks.ceph.mds.b:Stopped 2024-03-04T07:45:08.759 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:08.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efe05600640 1 Processor -- start 2024-03-04T07:45:08.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efe05600640 1 -- start start 2024-03-04T07:45:08.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efe05600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efe0006ff20 0x7efe0015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efe05600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe000c8b90 0x7efe0015c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efe05600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe00152e30 con 0x7efe00152a50 2024-03-04T07:45:08.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efe05600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe000c31b0 con 0x7efe0006ff20 2024-03-04T07:45:08.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efe05600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0006f840 con 0x7efe000c8b90 2024-03-04T07:45:08.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efdfea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efe00152a50 0x7efe000ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55792/0 (socket says 172.21.15.113:55792) 2024-03-04T07:45:08.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efdfea00640 1 -- 172.21.15.113:0/553118016 learned_addr learned my addr 172.21.15.113:0/553118016 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:45:08.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efdff400640 1 --2- 172.21.15.113:0/553118016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe000c8b90 0x7efe0015c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efdfe000640 1 --2- 172.21.15.113:0/553118016 >> v2:172.21.15.136:3300/0 conn(0x7efe0006ff20 0x7efe0015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.901+0000 7efdfd600640 1 -- 172.21.15.113:0/553118016 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212038785 0 0) 0x7efe00152e30 con 0x7efe00152a50 2024-03-04T07:45:08.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.905+0000 7efdfd600640 1 -- 172.21.15.113:0/553118016 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde4003530 con 0x7efe00152a50 2024-03-04T07:45:08.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.905+0000 7efdff400640 1 -- 172.21.15.113:0/553118016 >> v2:172.21.15.136:3300/0 conn(0x7efe0006ff20 msgr2=0x7efe0015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:08.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.905+0000 7efdff400640 1 --2- 172.21.15.113:0/553118016 >> v2:172.21.15.136:3300/0 conn(0x7efe0006ff20 0x7efe0015bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.905+0000 7efdff400640 1 -- 172.21.15.113:0/553118016 >> v1:172.21.15.113:6789/0 conn(0x7efe00152a50 legacy=0x7efe000ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.905+0000 7efdff400640 1 -- 172.21.15.113:0/553118016 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0006f280 con 0x7efe000c8b90 2024-03-04T07:45:08.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.905+0000 7efdff400640 1 --2- 172.21.15.113:0/553118016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe000c8b90 0x7efe0015c310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009d90 tx=0x7efdf801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ef0ee68bbd9da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:08.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efdfd600640 1 -- 172.21.15.113:0/553118016 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80b2070 con 0x7efe000c8b90 2024-03-04T07:45:08.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efdfd600640 1 -- 172.21.15.113:0/553118016 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf80a5850 con 0x7efe000c8b90 2024-03-04T07:45:08.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efdfd600640 1 -- 172.21.15.113:0/553118016 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80a5c00 con 0x7efe000c8b90 2024-03-04T07:45:08.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efe05600640 1 -- 172.21.15.113:0/553118016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe000c8b90 msgr2=0x7efe0015c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efe05600640 1 --2- 172.21.15.113:0/553118016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe000c8b90 0x7efe0015c310 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009d90 tx=0x7efdf801c970 comp rx=0 tx=0).stop 2024-03-04T07:45:08.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efe05600640 1 -- 172.21.15.113:0/553118016 shutdown_connections 2024-03-04T07:45:08.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efe05600640 1 --2- 172.21.15.113:0/553118016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe000c8b90 0x7efe0015c310 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efe05600640 1 --2- 172.21.15.113:0/553118016 >> v2:172.21.15.136:3300/0 conn(0x7efe0006ff20 0x7efe0015bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efe05600640 1 -- 172.21.15.113:0/553118016 >> 172.21.15.113:0/553118016 conn(0x7efe000734d0 msgr2=0x7efe00066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:08.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efe05600640 1 -- 172.21.15.113:0/553118016 shutdown_connections 2024-03-04T07:45:08.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efe05600640 1 -- 172.21.15.113:0/553118016 wait complete. 2024-03-04T07:45:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.913+0000 7efe05600640 1 Processor -- start 2024-03-04T07:45:08.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efe05600640 1 -- start start 2024-03-04T07:45:08.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efe05600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe0006ff20 0x7efe000c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efe05600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe000c8b90 0x7efe0014e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efe05600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe00152a50 0x7efe0014ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efe05600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000ce9b0 con 0x7efe00152a50 2024-03-04T07:45:08.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efe05600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000ce830 con 0x7efe000c8b90 2024-03-04T07:45:08.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efe05600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe00163870 con 0x7efe0006ff20 2024-03-04T07:45:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdff400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe00152a50 0x7efe0014ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdfea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe0006ff20 0x7efe000c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdff400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe00152a50 0x7efe0014ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60996/0 (socket says 172.21.15.113:60996) 2024-03-04T07:45:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdfe000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe000c8b90 0x7efe0014e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdff400640 1 -- 172.21.15.113:0/2438476969 learned_addr learned my addr 172.21.15.113:0/2438476969 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdff400640 1 -- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe0006ff20 msgr2=0x7efe000c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdff400640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe0006ff20 0x7efe000c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdff400640 1 -- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe000c8b90 msgr2=0x7efe0014e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdff400640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe000c8b90 0x7efe0014e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdff400640 1 -- 172.21.15.113:0/2438476969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe001eee40 con 0x7efe00152a50 2024-03-04T07:45:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdff400640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe00152a50 0x7efe0014ebd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efdf80b0040 tx=0x7efdf8019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdf6a00640 1 -- 172.21.15.113:0/2438476969 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80b2050 con 0x7efe00152a50 2024-03-04T07:45:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdf6a00640 1 -- 172.21.15.113:0/2438476969 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf80be070 con 0x7efe00152a50 2024-03-04T07:45:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efdf6a00640 1 -- 172.21.15.113:0/2438476969 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf80ada70 con 0x7efe00152a50 2024-03-04T07:45:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efe05600640 1 -- 172.21.15.113:0/2438476969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe001ef100 con 0x7efe00152a50 2024-03-04T07:45:08.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.917+0000 7efe05600640 1 -- 172.21.15.113:0/2438476969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe001ef6c0 con 0x7efe00152a50 2024-03-04T07:45:08.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.921+0000 7efe05600640 1 -- 172.21.15.113:0/2438476969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe00075c00 con 0x7efe00152a50 2024-03-04T07:45:08.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.921+0000 7efdf6a00640 1 -- 172.21.15.113:0/2438476969 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efdf80adc10 con 0x7efe00152a50 2024-03-04T07:45:08.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.921+0000 7efdf6a00640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efde0066f60 0x7efde0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:08.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.921+0000 7efdf6a00640 1 -- 172.21.15.113:0/2438476969 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7efdf8120c60 con 0x7efe00152a50 2024-03-04T07:45:08.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.921+0000 7efdfea00640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efde0066f60 0x7efde0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:08.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.925+0000 7efdfea00640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efde0066f60 0x7efde0069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efdf0007fd0 tx=0x7efdf0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:08.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:08.929+0000 7efdf6a00640 1 -- 172.21.15.113:0/2438476969 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efdf8016560 con 0x7efe00152a50 2024-03-04T07:45:09.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.181+0000 7efe05600640 1 -- 172.21.15.113:0/2438476969 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe000ca440 con 0x7efe00152a50 2024-03-04T07:45:09.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.181+0000 7efdf6a00640 1 -- 172.21.15.113:0/2438476969 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6612 (secure 0 0 0) 0x7efdf80f14a0 con 0x7efe00152a50 2024-03-04T07:45:09.183 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:09.183 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:40.846373+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244,5099]},"id":1}]} 2024-03-04T07:45:09.183 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-03-04T07:45:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 -- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efde0066f60 msgr2=0x7efde0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efde0066f60 0x7efde0069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efdf0007fd0 tx=0x7efdf0009040 comp rx=0 tx=0).stop 2024-03-04T07:45:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 -- 172.21.15.113:0/2438476969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe00152a50 msgr2=0x7efe0014ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe00152a50 0x7efe0014ebd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efdf80b0040 tx=0x7efdf8019710 comp rx=0 tx=0).stop 2024-03-04T07:45:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 -- 172.21.15.113:0/2438476969 shutdown_connections 2024-03-04T07:45:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efde0066f60 0x7efde0069420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe00152a50 0x7efe0014ebd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe000c8b90 0x7efe0014e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 --2- 172.21.15.113:0/2438476969 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe0006ff20 0x7efe000c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 -- 172.21.15.113:0/2438476969 >> 172.21.15.113:0/2438476969 conn(0x7efe000734d0 msgr2=0x7efe0015a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:09.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 -- 172.21.15.113:0/2438476969 shutdown_connections 2024-03-04T07:45:09.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:09.185+0000 7efe05600640 1 -- 172.21.15.113:0/2438476969 wait complete. 2024-03-04T07:45:09.204 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:45:11.205 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:11.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.413+0000 7ffa34a00640 1 Processor -- start 2024-03-04T07:45:11.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.413+0000 7ffa34a00640 1 -- start start 2024-03-04T07:45:11.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.413+0000 7ffa34a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffa30152920 0x7ffa30071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:11.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa34a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa3006c580 0x7ffa300c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:11.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa34a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa3006bb10 con 0x7ffa3015bf20 2024-03-04T07:45:11.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa34a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa300c7a40 con 0x7ffa3006c580 2024-03-04T07:45:11.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa34a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa300c7d20 con 0x7ffa30152920 2024-03-04T07:45:11.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa2de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa3006c580 0x7ffa300c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:11.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa2de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffa3006c580 0x7ffa300c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32774/0 (socket says 172.21.15.113:32774) 2024-03-04T07:45:11.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa2de00640 1 -- 172.21.15.113:0/3365467983 learned_addr learned my addr 172.21.15.113:0/3365467983 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:11.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa2e800640 1 --2- 172.21.15.113:0/3365467983 >> v2:172.21.15.136:3300/0 conn(0x7ffa30152920 0x7ffa30071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:11.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa2de00640 1 -- 172.21.15.113:0/3365467983 >> v2:172.21.15.136:3300/0 conn(0x7ffa30152920 msgr2=0x7ffa30071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:11.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa2de00640 1 --2- 172.21.15.113:0/3365467983 >> v2:172.21.15.136:3300/0 conn(0x7ffa30152920 0x7ffa30071a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:11.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa2de00640 1 -- 172.21.15.113:0/3365467983 >> v1:172.21.15.136:6789/0 conn(0x7ffa3015bf20 legacy=0x7ffa3015c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:11.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.417+0000 7ffa2de00640 1 -- 172.21.15.113:0/3365467983 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3006bd80 con 0x7ffa3006c580 2024-03-04T07:45:11.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa2de00640 1 --2- 172.21.15.113:0/3365467983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 0x7ffa300c9060 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffa300ca670 tx=0x7ffa1c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53e0c88245d0501 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:11.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa2d400640 1 -- 172.21.15.113:0/3365467983 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c0ab680 con 0x7ffa3006c580 2024-03-04T07:45:11.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa2d400640 1 -- 172.21.15.113:0/3365467983 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c0b0070 con 0x7ffa3006c580 2024-03-04T07:45:11.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa2d400640 1 -- 172.21.15.113:0/3365467983 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c0aba40 con 0x7ffa3006c580 2024-03-04T07:45:11.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa34a00640 1 -- 172.21.15.113:0/3365467983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 msgr2=0x7ffa300c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:11.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa34a00640 1 --2- 172.21.15.113:0/3365467983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 0x7ffa300c9060 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ffa300ca670 tx=0x7ffa1c01c460 comp rx=0 tx=0).stop 2024-03-04T07:45:11.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa34a00640 1 -- 172.21.15.113:0/3365467983 shutdown_connections 2024-03-04T07:45:11.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa34a00640 1 --2- 172.21.15.113:0/3365467983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 0x7ffa300c9060 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:11.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa34a00640 1 --2- 172.21.15.113:0/3365467983 >> v2:172.21.15.136:3300/0 conn(0x7ffa30152920 0x7ffa30071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:11.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa34a00640 1 -- 172.21.15.113:0/3365467983 >> 172.21.15.113:0/3365467983 conn(0x7ffa300c9a30 msgr2=0x7ffa300c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:11.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa34a00640 1 -- 172.21.15.113:0/3365467983 shutdown_connections 2024-03-04T07:45:11.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.421+0000 7ffa34a00640 1 -- 172.21.15.113:0/3365467983 wait complete. 2024-03-04T07:45:11.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa34a00640 1 Processor -- start 2024-03-04T07:45:11.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa34a00640 1 -- start start 2024-03-04T07:45:11.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa34a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 0x7ffa30150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa34a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa30152920 0x7ffa3014a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa34a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3015bf20 0x7ffa3014aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa34a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300c95a0 con 0x7ffa3006c580 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa34a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa30152d00 con 0x7ffa3015bf20 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa34a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa300c9720 con 0x7ffa30152920 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 0x7ffa30150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 0x7ffa30150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32782/0 (socket says 172.21.15.113:32782) 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2e800640 1 -- 172.21.15.113:0/3359780562 learned_addr learned my addr 172.21.15.113:0/3359780562 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2de00640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa30152920 0x7ffa3014a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2f200640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3015bf20 0x7ffa3014aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2e800640 1 -- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa30152920 msgr2=0x7ffa3014a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2e800640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa30152920 0x7ffa3014a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:11.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2e800640 1 -- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3015bf20 msgr2=0x7ffa3014aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:11.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2e800640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3015bf20 0x7ffa3014aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:11.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2e800640 1 -- 172.21.15.113:0/3359780562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa301f0dd0 con 0x7ffa3006c580 2024-03-04T07:45:11.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.425+0000 7ffa2e800640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 0x7ffa30150e10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000d800 tx=0x7ffa2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:11.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.429+0000 7ffa26a00640 1 -- 172.21.15.113:0/3359780562 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2000b6c0 con 0x7ffa3006c580 2024-03-04T07:45:11.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.429+0000 7ffa26a00640 1 -- 172.21.15.113:0/3359780562 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa20019070 con 0x7ffa3006c580 2024-03-04T07:45:11.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.429+0000 7ffa34a00640 1 -- 172.21.15.113:0/3359780562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa301f10b0 con 0x7ffa3006c580 2024-03-04T07:45:11.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.429+0000 7ffa26a00640 1 -- 172.21.15.113:0/3359780562 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2000ba60 con 0x7ffa3006c580 2024-03-04T07:45:11.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.429+0000 7ffa34a00640 1 -- 172.21.15.113:0/3359780562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa301f1670 con 0x7ffa3006c580 2024-03-04T07:45:11.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.429+0000 7ffa34a00640 1 -- 172.21.15.113:0/3359780562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa30075a80 con 0x7ffa3006c580 2024-03-04T07:45:11.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.429+0000 7ffa26a00640 1 -- 172.21.15.113:0/3359780562 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa2000bc00 con 0x7ffa3006c580 2024-03-04T07:45:11.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.429+0000 7ffa26a00640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa10066f60 0x7ffa10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:11.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.429+0000 7ffa26a00640 1 -- 172.21.15.113:0/3359780562 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7ffa20085840 con 0x7ffa3006c580 2024-03-04T07:45:11.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.429+0000 7ffa2de00640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa10066f60 0x7ffa10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:11.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.433+0000 7ffa2de00640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa10066f60 0x7ffa10069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0af000 tx=0x7ffa1c0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:11.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.433+0000 7ffa26a00640 1 -- 172.21.15.113:0/3359780562 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ffa200564a0 con 0x7ffa3006c580 2024-03-04T07:45:11.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.821+0000 7ffa34a00640 1 -- 172.21.15.113:0/3359780562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa30159d00 con 0x7ffa3006c580 2024-03-04T07:45:11.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.821+0000 7ffa26a00640 1 -- 172.21.15.113:0/3359780562 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6612 (secure 0 0 0) 0x7ffa20055da0 con 0x7ffa3006c580 2024-03-04T07:45:11.823 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:11.823 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:40.846373+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244,5099]},"id":1}]} 2024-03-04T07:45:11.823 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-03-04T07:45:11.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 -- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa10066f60 msgr2=0x7ffa10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:11.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa10066f60 0x7ffa10069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0af000 tx=0x7ffa1c0a7000 comp rx=0 tx=0).stop 2024-03-04T07:45:11.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 -- 172.21.15.113:0/3359780562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 msgr2=0x7ffa30150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:11.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 0x7ffa30150e10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000d800 tx=0x7ffa2000dcc0 comp rx=0 tx=0).stop 2024-03-04T07:45:11.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 -- 172.21.15.113:0/3359780562 shutdown_connections 2024-03-04T07:45:11.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa3015bf20 0x7ffa3014aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:11.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa30152920 0x7ffa3014a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:11.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa10066f60 0x7ffa10069420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:11.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 --2- 172.21.15.113:0/3359780562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa3006c580 0x7ffa30150e10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:11.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 -- 172.21.15.113:0/3359780562 >> 172.21.15.113:0/3359780562 conn(0x7ffa300c9a30 msgr2=0x7ffa3015a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:11.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 -- 172.21.15.113:0/3359780562 shutdown_connections 2024-03-04T07:45:11.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:11.825+0000 7ffa34a00640 1 -- 172.21.15.113:0/3359780562 wait complete. 2024-03-04T07:45:11.852 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:45:13.853 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:13.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.993+0000 7f2371a00640 1 Processor -- start 2024-03-04T07:45:13.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.993+0000 7f2371a00640 1 -- start start 2024-03-04T07:45:13.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f2371a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f236c1527f0 0x7f236c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:13.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f2371a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f236c0c8200 0x7f236c072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:13.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f2371a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f236c152bd0 0x7f236c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:13.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f2371a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c071df0 con 0x7f236c0c8200 2024-03-04T07:45:13.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f2371a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c1597b0 con 0x7f236c152bd0 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f2371a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c1591f0 con 0x7f236c1527f0 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f236a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f236c0c8200 0x7f236c072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f236a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f236c0c8200 0x7f236c072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32788/0 (socket says 172.21.15.113:32788) 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f236a000640 1 -- 172.21.15.113:0/3568937037 learned_addr learned my addr 172.21.15.113:0/3568937037 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f236aa00640 1 --2- 172.21.15.113:0/3568937037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f236c1527f0 0x7f236c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f236b400640 1 --2- 172.21.15.113:0/3568937037 >> v2:172.21.15.136:3300/0 conn(0x7f236c152bd0 0x7f236c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f236a000640 1 -- 172.21.15.113:0/3568937037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f236c1527f0 msgr2=0x7f236c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f236a000640 1 --2- 172.21.15.113:0/3568937037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f236c1527f0 0x7f236c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f236a000640 1 -- 172.21.15.113:0/3568937037 >> v2:172.21.15.136:3300/0 conn(0x7f236c152bd0 msgr2=0x7f236c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f236a000640 1 --2- 172.21.15.113:0/3568937037 >> v2:172.21.15.136:3300/0 conn(0x7f236c152bd0 0x7f236c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:13.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:13.997+0000 7f236a000640 1 -- 172.21.15.113:0/3568937037 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c1594d0 con 0x7f236c0c8200 2024-03-04T07:45:14.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f236a000640 1 --2- 172.21.15.113:0/3568937037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c0c8200 0x7f236c072280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f236c0ca020 tx=0x7f235801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c0e0be0ac5978a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:14.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2369600640 1 -- 172.21.15.113:0/3568937037 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580ab5f0 con 0x7f236c0c8200 2024-03-04T07:45:14.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2369600640 1 -- 172.21.15.113:0/3568937037 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23580b0070 con 0x7f236c0c8200 2024-03-04T07:45:14.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2369600640 1 -- 172.21.15.113:0/3568937037 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580ab9b0 con 0x7f236c0c8200 2024-03-04T07:45:14.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2371a00640 1 -- 172.21.15.113:0/3568937037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c0c8200 msgr2=0x7f236c072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:14.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2371a00640 1 --2- 172.21.15.113:0/3568937037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c0c8200 0x7f236c072280 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f236c0ca020 tx=0x7f235801c520 comp rx=0 tx=0).stop 2024-03-04T07:45:14.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2371a00640 1 -- 172.21.15.113:0/3568937037 shutdown_connections 2024-03-04T07:45:14.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2371a00640 1 --2- 172.21.15.113:0/3568937037 >> v2:172.21.15.136:3300/0 conn(0x7f236c152bd0 0x7f236c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:14.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2371a00640 1 --2- 172.21.15.113:0/3568937037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c0c8200 0x7f236c072280 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:14.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2371a00640 1 --2- 172.21.15.113:0/3568937037 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f236c1527f0 0x7f236c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:14.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2371a00640 1 -- 172.21.15.113:0/3568937037 >> 172.21.15.113:0/3568937037 conn(0x7f236c06e820 msgr2=0x7f236c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:14.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2371a00640 1 -- 172.21.15.113:0/3568937037 shutdown_connections 2024-03-04T07:45:14.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.001+0000 7f2371a00640 1 -- 172.21.15.113:0/3568937037 wait complete. 2024-03-04T07:45:14.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2371a00640 1 Processor -- start 2024-03-04T07:45:14.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2371a00640 1 -- start start 2024-03-04T07:45:14.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2371a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f236c0c8200 0x7f236c0c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:14.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2371a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c1527f0 0x7f236c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:14.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2371a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f236c152bd0 0x7f236c1f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:14.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2371a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15bf40 con 0x7f236c1527f0 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2371a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15bdc0 con 0x7f236c152bd0 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2371a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15c0c0 con 0x7f236c0c8200 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c1527f0 0x7f236c0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c1527f0 0x7f236c0b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32796/0 (socket says 172.21.15.113:32796) 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236a000640 1 -- 172.21.15.113:0/863486839 learned_addr learned my addr 172.21.15.113:0/863486839 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236aa00640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f236c0c8200 0x7f236c0c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236b400640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f236c152bd0 0x7f236c1f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236a000640 1 -- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f236c0c8200 msgr2=0x7f236c0c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236a000640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f236c0c8200 0x7f236c0c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236a000640 1 -- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f236c152bd0 msgr2=0x7f236c1f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:14.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236a000640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f236c152bd0 0x7f236c1f1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:14.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236a000640 1 -- 172.21.15.113:0/863486839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0c4310 con 0x7f236c1527f0 2024-03-04T07:45:14.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f236a000640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c1527f0 0x7f236c0b0e70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f235801abd0 tx=0x7f235801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:14.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2362a00640 1 -- 172.21.15.113:0/863486839 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580be3f0 con 0x7f236c1527f0 2024-03-04T07:45:14.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2362a00640 1 -- 172.21.15.113:0/863486839 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23580be590 con 0x7f236c1527f0 2024-03-04T07:45:14.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2362a00640 1 -- 172.21.15.113:0/863486839 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23580be940 con 0x7f236c1527f0 2024-03-04T07:45:14.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2371a00640 1 -- 172.21.15.113:0/863486839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c0c74e0 con 0x7f236c1527f0 2024-03-04T07:45:14.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.005+0000 7f2371a00640 1 -- 172.21.15.113:0/863486839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c1f1fe0 con 0x7f236c1527f0 2024-03-04T07:45:14.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.009+0000 7f2371a00640 1 -- 172.21.15.113:0/863486839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c075c00 con 0x7f236c1527f0 2024-03-04T07:45:14.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.009+0000 7f2362a00640 1 -- 172.21.15.113:0/863486839 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23580b43f0 con 0x7f236c1527f0 2024-03-04T07:45:14.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.009+0000 7f2362a00640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f234c066f60 0x7f234c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:14.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.009+0000 7f2362a00640 1 -- 172.21.15.113:0/863486839 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f235811ee50 con 0x7f236c1527f0 2024-03-04T07:45:14.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.009+0000 7f236aa00640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f234c066f60 0x7f234c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:14.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.009+0000 7f236aa00640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f234c066f60 0x7f234c069420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f235c009ae0 tx=0x7f235c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:14.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.017+0000 7f2362a00640 1 -- 172.21.15.113:0/863486839 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2358016110 con 0x7f236c1527f0 2024-03-04T07:45:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.285+0000 7f2371a00640 1 -- 172.21.15.113:0/863486839 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f236c162f30 con 0x7f236c1527f0 2024-03-04T07:45:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.285+0000 7f2362a00640 1 -- 172.21.15.113:0/863486839 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f23580ef610 con 0x7f236c1527f0 2024-03-04T07:45:14.287 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:14.287 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:40.846373+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244,5099]},"id":1}]} 2024-03-04T07:45:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-03-04T07:45:14.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 -- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f234c066f60 msgr2=0x7f234c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:14.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f234c066f60 0x7f234c069420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f235c009ae0 tx=0x7f235c009190 comp rx=0 tx=0).stop 2024-03-04T07:45:14.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 -- 172.21.15.113:0/863486839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c1527f0 msgr2=0x7f236c0b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c1527f0 0x7f236c0b0e70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f235801abd0 tx=0x7f235801ad60 comp rx=0 tx=0).stop 2024-03-04T07:45:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 -- 172.21.15.113:0/863486839 shutdown_connections 2024-03-04T07:45:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f236c152bd0 0x7f236c1f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f234c066f60 0x7f234c069420 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f236c1527f0 0x7f236c0b0e70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 --2- 172.21.15.113:0/863486839 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f236c0c8200 0x7f236c0c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 -- 172.21.15.113:0/863486839 >> 172.21.15.113:0/863486839 conn(0x7f236c06e820 msgr2=0x7f236c0c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 -- 172.21.15.113:0/863486839 shutdown_connections 2024-03-04T07:45:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:14.289+0000 7f2371a00640 1 -- 172.21.15.113:0/863486839 wait complete. 2024-03-04T07:45:14.310 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:45:16.311 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:16.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.449+0000 7f635da00640 1 Processor -- start 2024-03-04T07:45:16.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.449+0000 7f635da00640 1 -- start start 2024-03-04T07:45:16.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.449+0000 7f635da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 0x7f635806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:16.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.449+0000 7f635da00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f63580ce120 0x7f63580ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:16.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.449+0000 7f635da00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63580ca0e0 con 0x7f635806ff20 2024-03-04T07:45:16.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.449+0000 7f635da00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f635806f450 con 0x7f63580ce120 2024-03-04T07:45:16.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.449+0000 7f635da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635815ad80 con 0x7f6358158010 2024-03-04T07:45:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.449+0000 7f6356a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 0x7f635806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f6357400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f63580ce120 0x7f63580ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f6356a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 0x7f635806dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:48690/0 (socket says 172.21.15.113:48690) 2024-03-04T07:45:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f634fe00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f635806ff20 0x7f635815b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:33384/0 (socket says 172.21.15.113:33384) 2024-03-04T07:45:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f6356a00640 1 -- 172.21.15.113:0/3869913424 learned_addr learned my addr 172.21.15.113:0/3869913424 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f6357400640 1 -- 172.21.15.113:0/3869913424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 msgr2=0x7f635806dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f6357400640 1 --2- 172.21.15.113:0/3869913424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 0x7f635806dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f6357400640 1 -- 172.21.15.113:0/3869913424 >> v1:172.21.15.136:6789/0 conn(0x7f635806ff20 legacy=0x7f635815b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f6357400640 1 -- 172.21.15.113:0/3869913424 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635815aa30 con 0x7f63580ce120 2024-03-04T07:45:16.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f6356a00640 1 --2- 172.21.15.113:0/3869913424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 0x7f635806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:45:16.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f6357400640 1 --2- 172.21.15.113:0/3869913424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63580ce120 0x7f63580ce5a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f635806f5d0 tx=0x7f635001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47522adba9b005b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:16.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.453+0000 7f6356000640 1 -- 172.21.15.113:0/3869913424 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500b2070 con 0x7f63580ce120 2024-03-04T07:45:16.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f6356000640 1 -- 172.21.15.113:0/3869913424 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63500a5810 con 0x7f63580ce120 2024-03-04T07:45:16.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f6356000640 1 -- 172.21.15.113:0/3869913424 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500a5be0 con 0x7f63580ce120 2024-03-04T07:45:16.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 -- 172.21.15.113:0/3869913424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63580ce120 msgr2=0x7f63580ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:16.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 --2- 172.21.15.113:0/3869913424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63580ce120 0x7f63580ce5a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f635806f5d0 tx=0x7f635001c950 comp rx=0 tx=0).stop 2024-03-04T07:45:16.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 -- 172.21.15.113:0/3869913424 shutdown_connections 2024-03-04T07:45:16.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 --2- 172.21.15.113:0/3869913424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63580ce120 0x7f63580ce5a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:16.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 --2- 172.21.15.113:0/3869913424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 0x7f635806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:16.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 -- 172.21.15.113:0/3869913424 >> 172.21.15.113:0/3869913424 conn(0x7f63580734d0 msgr2=0x7f6358066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:16.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 -- 172.21.15.113:0/3869913424 shutdown_connections 2024-03-04T07:45:16.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 -- 172.21.15.113:0/3869913424 wait complete. 2024-03-04T07:45:16.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 Processor -- start 2024-03-04T07:45:16.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 -- start start 2024-03-04T07:45:16.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f635806ff20 0x7f635814edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:16.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63580ce120 0x7f63581f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 0x7f63580cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635815b920 con 0x7f63580ce120 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635806e290 con 0x7f635806ff20 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f635da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63581583f0 con 0x7f6358158010 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f634fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63580ce120 0x7f63581f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f634fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63580ce120 0x7f63581f6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32808/0 (socket says 172.21.15.113:32808) 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f634fe00640 1 -- 172.21.15.113:0/637256525 learned_addr learned my addr 172.21.15.113:0/637256525 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f6356a00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f635806ff20 0x7f635814edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.457+0000 7f6357400640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 0x7f63580cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634fe00640 1 -- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 msgr2=0x7f63580cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634fe00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 0x7f63580cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634fe00640 1 -- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f635806ff20 msgr2=0x7f635814edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634fe00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f635806ff20 0x7f635814edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634fe00640 1 -- 172.21.15.113:0/637256525 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63581f6cd0 con 0x7f63580ce120 2024-03-04T07:45:16.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634fe00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63580ce120 0x7f63581f6620 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f634000c460 tx=0x7f634000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:16.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634ea00640 1 -- 172.21.15.113:0/637256525 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6340018070 con 0x7f63580ce120 2024-03-04T07:45:16.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634ea00640 1 -- 172.21.15.113:0/637256525 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6340007480 con 0x7f63580ce120 2024-03-04T07:45:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f635da00640 1 -- 172.21.15.113:0/637256525 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63581f6fb0 con 0x7f63580ce120 2024-03-04T07:45:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634ea00640 1 -- 172.21.15.113:0/637256525 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63400047f0 con 0x7f63580ce120 2024-03-04T07:45:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f635da00640 1 -- 172.21.15.113:0/637256525 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63581ec9a0 con 0x7f63580ce120 2024-03-04T07:45:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f635da00640 1 -- 172.21.15.113:0/637256525 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63581f74d0 con 0x7f63580ce120 2024-03-04T07:45:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634ea00640 1 -- 172.21.15.113:0/637256525 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f634001b070 con 0x7f63580ce120 2024-03-04T07:45:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634ea00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6334066f60 0x7f6334069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.461+0000 7f634ea00640 1 -- 172.21.15.113:0/637256525 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f6340084600 con 0x7f63580ce120 2024-03-04T07:45:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.465+0000 7f6356a00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6334066f60 0x7f6334069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:16.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.465+0000 7f6356a00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6334066f60 0x7f6334069420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6348006cb0 tx=0x7f6348009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:16.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.473+0000 7f634ea00640 1 -- 172.21.15.113:0/637256525 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f63581f74d0 con 0x7f63580ce120 2024-03-04T07:45:16.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.681+0000 7f635da00640 1 -- 172.21.15.113:0/637256525 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6358075c00 con 0x7f63580ce120 2024-03-04T07:45:16.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f634ea00640 1 -- 172.21.15.113:0/637256525 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f6340054c10 con 0x7f63580ce120 2024-03-04T07:45:16.684 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:16.684 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:40.846373+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244,5099]},"id":1}]} 2024-03-04T07:45:16.684 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 -- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6334066f60 msgr2=0x7f6334069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6334066f60 0x7f6334069420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6348006cb0 tx=0x7f6348009240 comp rx=0 tx=0).stop 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 -- 172.21.15.113:0/637256525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63580ce120 msgr2=0x7f63581f6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63580ce120 0x7f63581f6620 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f634000c460 tx=0x7f634000c920 comp rx=0 tx=0).stop 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 -- 172.21.15.113:0/637256525 shutdown_connections 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6358158010 0x7f63580cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6334066f60 0x7f6334069420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63580ce120 0x7f63581f6620 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 --2- 172.21.15.113:0/637256525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f635806ff20 0x7f635814edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 -- 172.21.15.113:0/637256525 >> 172.21.15.113:0/637256525 conn(0x7f63580734d0 msgr2=0x7f6358066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 -- 172.21.15.113:0/637256525 shutdown_connections 2024-03-04T07:45:16.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:16.685+0000 7f635da00640 1 -- 172.21.15.113:0/637256525 wait complete. 2024-03-04T07:45:16.705 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:45:18.706 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:18.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.845+0000 7fa1fbc00640 1 Processor -- start 2024-03-04T07:45:18.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.845+0000 7fa1fbc00640 1 -- start start 2024-03-04T07:45:18.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1fbc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f406df10 0x7fa1f415e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:18.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1fbc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f406c310 con 0x7fa1f4158230 2024-03-04T07:45:18.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1fbc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f40c95f0 con 0x7fa1f40703e0 2024-03-04T07:45:18.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1fbc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f406c5b0 con 0x7fa1f406df10 2024-03-04T07:45:18.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f9200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa1f4158230 0x7fa1f406f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33300/0 (socket says 172.21.15.113:33300) 2024-03-04T07:45:18.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f9200640 1 -- 172.21.15.113:0/3241723535 learned_addr learned my addr 172.21.15.113:0/3241723535 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:45:18.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1ebe00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f406df10 0x7fa1f415e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:18.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561504261 0 0) 0x7fa1f406c310 con 0x7fa1f4158230 2024-03-04T07:45:18.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d8003530 con 0x7fa1f4158230 2024-03-04T07:45:18.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2983335213 0 0) 0x7fa1d8003530 con 0x7fa1f4158230 2024-03-04T07:45:18.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f406df10 msgr2=0x7fa1f415e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:18.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f406df10 0x7fa1f415e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:18.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 >> v1:172.21.15.136:6789/0 conn(0x7fa1f40703e0 legacy=0x7fa1f40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:18.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f40c9890 con 0x7fa1f4158230 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fa1e4001680 con 0x7fa1f4158230 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 >> v1:172.21.15.113:6789/0 conn(0x7fa1f4158230 legacy=0x7fa1f406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1d80039f0 0x7fa1d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f406df10 0x7fa1d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8004d50 0x7fa1d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f406c5b0 con 0x7fa1d8004d50 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d800b9d0 con 0x7fa1d80039f0 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d800bbd0 con 0x7fa1f406df10 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1ebe00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f406df10 0x7fa1d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f9200640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1d80039f0 0x7fa1d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f9c00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8004d50 0x7fa1d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f9c00640 1 -- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f406df10 msgr2=0x7fa1d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f9c00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f406df10 0x7fa1d8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:18.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f9c00640 1 -- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1d80039f0 msgr2=0x7fa1d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f9c00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1d80039f0 0x7fa1d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.849+0000 7fa1f9c00640 1 -- 172.21.15.113:0/3241723535 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8003530 con 0x7fa1d8004d50 2024-03-04T07:45:18.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1f9c00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8004d50 0x7fa1d80051d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec001f60 tx=0x7fa1ec01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f15c504601e0b67 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0b2070 con 0x7fa1d8004d50 2024-03-04T07:45:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec0a5770 con 0x7fa1d8004d50 2024-03-04T07:45:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1f3e00640 1 -- 172.21.15.113:0/3241723535 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec001b60 con 0x7fa1d8004d50 2024-03-04T07:45:18.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1fbc00640 1 -- 172.21.15.113:0/3241723535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8004d50 msgr2=0x7fa1d80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:18.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1fbc00640 1 --2- 172.21.15.113:0/3241723535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1d8004d50 0x7fa1d80051d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec001f60 tx=0x7fa1ec01c880 comp rx=0 tx=0).stop 2024-03-04T07:45:18.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1f9c00640 1 -- 172.21.15.113:0/3241723535 reap_dead start 2024-03-04T07:45:18.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1fbc00640 1 -- 172.21.15.113:0/3241723535 shutdown_connections 2024-03-04T07:45:18.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1fbc00640 1 -- 172.21.15.113:0/3241723535 >> 172.21.15.113:0/3241723535 conn(0x7fa1f4066db0 msgr2=0x7fa1f40671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:18.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1fbc00640 1 -- 172.21.15.113:0/3241723535 shutdown_connections 2024-03-04T07:45:18.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1fbc00640 1 -- 172.21.15.113:0/3241723535 wait complete. 2024-03-04T07:45:18.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.853+0000 7fa1fbc00640 1 Processor -- start 2024-03-04T07:45:18.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1fbc00640 1 -- start start 2024-03-04T07:45:18.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1fbc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f4150be0 0x7fa1f4154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1fbc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f4150fc0 0x7fa1f40cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1fbc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1f40ce5b0 0x7fa1f40cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1fbc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f406ca10 con 0x7fa1f4150fc0 2024-03-04T07:45:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1fbc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f406c890 con 0x7fa1f40ce5b0 2024-03-04T07:45:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1fbc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40cead0 con 0x7fa1f4150be0 2024-03-04T07:45:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1ebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f4150fc0 0x7fa1f40cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1ebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f4150fc0 0x7fa1f40cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52886/0 (socket says 172.21.15.113:52886) 2024-03-04T07:45:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1ebe00640 1 -- 172.21.15.113:0/1330766476 learned_addr learned my addr 172.21.15.113:0/1330766476 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1f9200640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f4150be0 0x7fa1f4154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:18.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1f9c00640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1f40ce5b0 0x7fa1f40cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:18.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1ebe00640 1 -- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f4150be0 msgr2=0x7fa1f4154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:18.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1ebe00640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f4150be0 0x7fa1f4154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1ebe00640 1 -- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1f40ce5b0 msgr2=0x7fa1f40cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1ebe00640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1f40ce5b0 0x7fa1f40cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1ebe00640 1 -- 172.21.15.113:0/1330766476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f4153e30 con 0x7fa1f4150fc0 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1ebe00640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f4150fc0 0x7fa1f40cd2b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc011940 tx=0x7fa1dc011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1f1600640 1 -- 172.21.15.113:0/1330766476 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc019070 con 0x7fa1f4150fc0 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1f1600640 1 -- 172.21.15.113:0/1330766476 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc004300 con 0x7fa1f4150fc0 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1f1600640 1 -- 172.21.15.113:0/1330766476 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc00f9e0 con 0x7fa1f4150fc0 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1fbc00640 1 -- 172.21.15.113:0/1330766476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f4151a80 con 0x7fa1f4150fc0 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1fbc00640 1 -- 172.21.15.113:0/1330766476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f415a5e0 con 0x7fa1f4150fc0 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.857+0000 7fa1fbc00640 1 -- 172.21.15.113:0/1330766476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f4075c00 con 0x7fa1f4150fc0 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.861+0000 7fa1f1600640 1 -- 172.21.15.113:0/1330766476 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1dc01c070 con 0x7fa1f4150fc0 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.861+0000 7fa1f1600640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa1d0066f90 0x7fa1d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:18.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.861+0000 7fa1f1600640 1 -- 172.21.15.113:0/1330766476 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fa1dc0857c0 con 0x7fa1f4150fc0 2024-03-04T07:45:18.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.861+0000 7fa1f9200640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa1d0066f90 0x7fa1d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:18.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.861+0000 7fa1f9200640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa1d0066f90 0x7fa1d0069450 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4007c80 tx=0x7fa1e4001e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:18.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:18.865+0000 7fa1f1600640 1 -- 172.21.15.113:0/1330766476 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa1f4153e30 con 0x7fa1f4150fc0 2024-03-04T07:45:19.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 -- 172.21.15.113:0/1330766476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1f40701c0 con 0x7fa1f4150fc0 2024-03-04T07:45:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1f1600640 1 -- 172.21.15.113:0/1330766476 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fa1dc055e70 con 0x7fa1f4150fc0 2024-03-04T07:45:19.129 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:19.129 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5631,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:44:40.846373+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5099":{"gid":5099,"name":"b","rank":2,"incarnation":39,"state":"up:active","state_seq":21,"addr":"172.21.15.136:6835/3944001911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3944001911},{"type":"v1","addr":"172.21.15.136:6835","nonce":3944001911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244,5099]},"id":1}]} 2024-03-04T07:45:19.130 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 50 2024-03-04T07:45:19.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 -- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa1d0066f90 msgr2=0x7fa1d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:19.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa1d0066f90 0x7fa1d0069450 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4007c80 tx=0x7fa1e4001e30 comp rx=0 tx=0).stop 2024-03-04T07:45:19.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 -- 172.21.15.113:0/1330766476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f4150fc0 msgr2=0x7fa1f40cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:19.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f4150fc0 0x7fa1f40cd2b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc011940 tx=0x7fa1dc011e00 comp rx=0 tx=0).stop 2024-03-04T07:45:19.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 -- 172.21.15.113:0/1330766476 shutdown_connections 2024-03-04T07:45:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1f40ce5b0 0x7fa1f40cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa1d0066f90 0x7fa1d0069450 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1f4150fc0 0x7fa1f40cd2b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 --2- 172.21.15.113:0/1330766476 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1f4150be0 0x7fa1f4154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 -- 172.21.15.113:0/1330766476 >> 172.21.15.113:0/1330766476 conn(0x7fa1f4066db0 msgr2=0x7fa1f41573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 -- 172.21.15.113:0/1330766476 shutdown_connections 2024-03-04T07:45:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:19.129+0000 7fa1fbc00640 1 -- 172.21.15.113:0/1330766476 wait complete. 2024-03-04T07:45:19.152 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:45:21.153 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:21.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.293+0000 7fb9fd200640 1 Processor -- start 2024-03-04T07:45:21.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.293+0000 7fb9fd200640 1 -- start start 2024-03-04T07:45:21.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.293+0000 7fb9fd200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb9f8152950 0x7fb9f815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9fd200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f80c4ce0 con 0x7fb9f806faa0 2024-03-04T07:45:21.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9fd200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f80c4f80 con 0x7fb9f80cb540 2024-03-04T07:45:21.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9fd200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f80c8370 con 0x7fb9f8152950 2024-03-04T07:45:21.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb9f806faa0 0x7fb9f806ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33306/0 (socket says 172.21.15.113:33306) 2024-03-04T07:45:21.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6a00640 1 -- 172.21.15.113:0/2801857318 learned_addr learned my addr 172.21.15.113:0/2801857318 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:45:21.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f7400640 1 --2- 172.21.15.113:0/2801857318 >> v2:172.21.15.136:3300/0 conn(0x7fb9f8152950 0x7fb9f815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412044974 0 0) 0x7fb9f80c4ce0 con 0x7fb9f806faa0 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9dc003530 con 0x7fb9f806faa0 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2904828021 0 0) 0x7fb9dc003530 con 0x7fb9f806faa0 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 >> v2:172.21.15.136:3300/0 conn(0x7fb9f8152950 msgr2=0x7fb9f815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 --2- 172.21.15.113:0/2801857318 >> v2:172.21.15.136:3300/0 conn(0x7fb9f8152950 0x7fb9f815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 >> v1:172.21.15.136:6789/0 conn(0x7fb9f80cb540 legacy=0x7fb9f8162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f80c5220 con 0x7fb9f806faa0 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fb9e4001680 con 0x7fb9f806faa0 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 >> v1:172.21.15.113:6789/0 conn(0x7fb9f806faa0 legacy=0x7fb9f806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc0039c0 0x7fb9dc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9dc0043c0 0x7fb9dc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc005090 0x7fb9dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f80c8370 con 0x7fb9dc005090 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc00bf40 con 0x7fb9dc0043c0 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc00c140 con 0x7fb9dc0039c0 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f7e00640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc005090 0x7fb9dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f7400640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc0039c0 0x7fb9dc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f6a00640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9dc0043c0 0x7fb9dc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f7e00640 1 -- 172.21.15.113:0/2801857318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc0039c0 msgr2=0x7fb9dc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f7e00640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc0039c0 0x7fb9dc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f7e00640 1 -- 172.21.15.113:0/2801857318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9dc0043c0 msgr2=0x7fb9dc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f7e00640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9dc0043c0 0x7fb9dc004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.297+0000 7fb9f7e00640 1 -- 172.21.15.113:0/2801857318 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc003530 con 0x7fb9dc005090 2024-03-04T07:45:21.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.301+0000 7fb9f7e00640 1 -- 172.21.15.113:0/2801857318 reap_dead start 2024-03-04T07:45:21.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.301+0000 7fb9f7e00640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc005090 0x7fb9dc00b9d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80c5150 tx=0x7fb9f001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b81b524933212a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:21.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.301+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0001c50 con 0x7fb9dc005090 2024-03-04T07:45:21.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.301+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f0003ba0 con 0x7fb9dc005090 2024-03-04T07:45:21.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.301+0000 7fb9f6000640 1 -- 172.21.15.113:0/2801857318 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f00050d0 con 0x7fb9dc005090 2024-03-04T07:45:21.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.305+0000 7fb9fd200640 1 -- 172.21.15.113:0/2801857318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc005090 msgr2=0x7fb9dc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:21.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.305+0000 7fb9fd200640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc005090 0x7fb9dc00b9d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80c5150 tx=0x7fb9f001c8c0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.305+0000 7fb9fd200640 1 -- 172.21.15.113:0/2801857318 shutdown_connections 2024-03-04T07:45:21.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.305+0000 7fb9fd200640 1 --2- 172.21.15.113:0/2801857318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc005090 0x7fb9dc00b9d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.305+0000 7fb9fd200640 1 -- 172.21.15.113:0/2801857318 >> 172.21.15.113:0/2801857318 conn(0x7fb9f8072f40 msgr2=0x7fb9f80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:21.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.305+0000 7fb9fd200640 1 -- 172.21.15.113:0/2801857318 shutdown_connections 2024-03-04T07:45:21.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.305+0000 7fb9fd200640 1 -- 172.21.15.113:0/2801857318 wait complete. 2024-03-04T07:45:21.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.305+0000 7fb9fd200640 1 Processor -- start 2024-03-04T07:45:21.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.305+0000 7fb9fd200640 1 -- start start 2024-03-04T07:45:21.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9fd200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc005090 0x7fb9f80ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9fd200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f8156eb0 0x7fb9f815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9fd200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f815ae50 0x7fb9f814d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9fd200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f80ca0a0 con 0x7fb9f8156eb0 2024-03-04T07:45:21.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9fd200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f80c4b40 con 0x7fb9f815ae50 2024-03-04T07:45:21.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9fd200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f80c9a50 con 0x7fb9dc005090 2024-03-04T07:45:21.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f8156eb0 0x7fb9f815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f8156eb0 0x7fb9f815b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52906/0 (socket says 172.21.15.113:52906) 2024-03-04T07:45:21.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f6a00640 1 -- 172.21.15.113:0/526317543 learned_addr learned my addr 172.21.15.113:0/526317543 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:21.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f7e00640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f815ae50 0x7fb9f814d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f7400640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc005090 0x7fb9f80ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f6a00640 1 -- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc005090 msgr2=0x7fb9f80ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:21.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f6a00640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc005090 0x7fb9f80ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f6a00640 1 -- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f815ae50 msgr2=0x7fb9f814d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:21.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f6a00640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f815ae50 0x7fb9f814d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f6a00640 1 -- 172.21.15.113:0/526317543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f80cd620 con 0x7fb9f8156eb0 2024-03-04T07:45:21.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9f6a00640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f8156eb0 0x7fb9f815b910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4004ed0 tx=0x7fb9e4007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:21.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9ef400640 1 -- 172.21.15.113:0/526317543 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e400f750 con 0x7fb9f8156eb0 2024-03-04T07:45:21.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9ef400640 1 -- 172.21.15.113:0/526317543 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e4018070 con 0x7fb9f8156eb0 2024-03-04T07:45:21.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9ef400640 1 -- 172.21.15.113:0/526317543 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e400fb50 con 0x7fb9f8156eb0 2024-03-04T07:45:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9fd200640 1 -- 172.21.15.113:0/526317543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f80cd900 con 0x7fb9f8156eb0 2024-03-04T07:45:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.309+0000 7fb9fd200640 1 -- 172.21.15.113:0/526317543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f81ede10 con 0x7fb9f8156eb0 2024-03-04T07:45:21.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.313+0000 7fb9fd200640 1 -- 172.21.15.113:0/526317543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f8075a80 con 0x7fb9f8156eb0 2024-03-04T07:45:21.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.313+0000 7fb9ef400640 1 -- 172.21.15.113:0/526317543 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9e400fcf0 con 0x7fb9f8156eb0 2024-03-04T07:45:21.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.313+0000 7fb9ef400640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9d8066f90 0x7fb9d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.313+0000 7fb9ef400640 1 -- 172.21.15.113:0/526317543 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fb9e4085100 con 0x7fb9f8156eb0 2024-03-04T07:45:21.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.313+0000 7fb9f7400640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9d8066f90 0x7fb9d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.313+0000 7fb9f7400640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9d8066f90 0x7fb9d8069450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009b60 tx=0x7fb9e8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:21.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.317+0000 7fb9ef400640 1 -- 172.21.15.113:0/526317543 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb9e4055d10 con 0x7fb9f8156eb0 2024-03-04T07:45:21.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.549+0000 7fb9fd200640 1 -- 172.21.15.113:0/526317543 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9f80c9580 con 0x7fb9f8156eb0 2024-03-04T07:45:21.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9ef400640 1 -- 172.21.15.113:0/526317543 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+5812 (secure 0 0 0) 0x7fb9e4055610 con 0x7fb9f8156eb0 2024-03-04T07:45:21.552 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:21.552 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:20.664962+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5529]},"id":1}]} 2024-03-04T07:45:21.553 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 51 2024-03-04T07:45:21.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 -- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9d8066f90 msgr2=0x7fb9d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:21.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9d8066f90 0x7fb9d8069450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8009b60 tx=0x7fb9e8009190 comp rx=0 tx=0).stop 2024-03-04T07:45:21.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 -- 172.21.15.113:0/526317543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f8156eb0 msgr2=0x7fb9f815b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:21.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f8156eb0 0x7fb9f815b910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4004ed0 tx=0x7fb9e4007430 comp rx=0 tx=0).stop 2024-03-04T07:45:21.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 -- 172.21.15.113:0/526317543 shutdown_connections 2024-03-04T07:45:21.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f815ae50 0x7fb9f814d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9d8066f90 0x7fb9d8069450 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f8156eb0 0x7fb9f815b910 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 --2- 172.21.15.113:0/526317543 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc005090 0x7fb9f80ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 -- 172.21.15.113:0/526317543 >> 172.21.15.113:0/526317543 conn(0x7fb9f8072f40 msgr2=0x7fb9f80c7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:21.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 -- 172.21.15.113:0/526317543 shutdown_connections 2024-03-04T07:45:21.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.553+0000 7fb9fd200640 1 -- 172.21.15.113:0/526317543 wait complete. 2024-03-04T07:45:21.574 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2024-03-04T07:45:21.574 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:45:21.574 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:21.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10fe400640 1 Processor -- start 2024-03-04T07:45:21.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10fe400640 1 -- start start 2024-03-04T07:45:21.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10fe400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10f8152b20 0x7f10f8070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10fe400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10f80c8b00 0x7f10f815be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10fe400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f10f815c340 0x7f10f815c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10fe400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f80c5320 con 0x7f10f80c8b00 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10fe400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f80c8f60 con 0x7f10f815c340 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10fe400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8152f00 con 0x7f10f8152b20 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10f6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10f80c8b00 0x7f10f815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10f6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10f80c8b00 0x7f10f815be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52912/0 (socket says 172.21.15.113:52912) 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10f6a00640 1 -- 172.21.15.113:0/1720806914 learned_addr learned my addr 172.21.15.113:0/1720806914 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10f7400640 1 --2- 172.21.15.113:0/1720806914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10f8152b20 0x7f10f8070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.721+0000 7f10f7e00640 1 --2- 172.21.15.113:0/1720806914 >> v2:172.21.15.136:3300/0 conn(0x7f10f815c340 0x7f10f815c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10f6a00640 1 -- 172.21.15.113:0/1720806914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10f8152b20 msgr2=0x7f10f8070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10f6a00640 1 --2- 172.21.15.113:0/1720806914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10f8152b20 0x7f10f8070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10f6a00640 1 -- 172.21.15.113:0/1720806914 >> v2:172.21.15.136:3300/0 conn(0x7f10f815c340 msgr2=0x7f10f815c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:21.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10f6a00640 1 --2- 172.21.15.113:0/1720806914 >> v2:172.21.15.136:3300/0 conn(0x7f10f815c340 0x7f10f815c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10f6a00640 1 -- 172.21.15.113:0/1720806914 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80c5040 con 0x7f10f80c8b00 2024-03-04T07:45:21.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10f6a00640 1 --2- 172.21.15.113:0/1720806914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 0x7f10f815be00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f10f80c7ca0 tx=0x7f10e401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c171a091126013 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:21.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10f6000640 1 -- 172.21.15.113:0/1720806914 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e40b0070 con 0x7f10f80c8b00 2024-03-04T07:45:21.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10f6000640 1 -- 172.21.15.113:0/1720806914 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e40ab600 con 0x7f10f80c8b00 2024-03-04T07:45:21.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10f6000640 1 -- 172.21.15.113:0/1720806914 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e40ab9d0 con 0x7f10f80c8b00 2024-03-04T07:45:21.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10fe400640 1 -- 172.21.15.113:0/1720806914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 msgr2=0x7f10f815be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:21.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10fe400640 1 --2- 172.21.15.113:0/1720806914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 0x7f10f815be00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f10f80c7ca0 tx=0x7f10e401c520 comp rx=0 tx=0).stop 2024-03-04T07:45:21.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10fe400640 1 -- 172.21.15.113:0/1720806914 shutdown_connections 2024-03-04T07:45:21.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.725+0000 7f10fe400640 1 --2- 172.21.15.113:0/1720806914 >> v2:172.21.15.136:3300/0 conn(0x7f10f815c340 0x7f10f815c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 --2- 172.21.15.113:0/1720806914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 0x7f10f815be00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 --2- 172.21.15.113:0/1720806914 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10f8152b20 0x7f10f8070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 -- 172.21.15.113:0/1720806914 >> 172.21.15.113:0/1720806914 conn(0x7f10f80709a0 msgr2=0x7f10f8070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:21.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 -- 172.21.15.113:0/1720806914 shutdown_connections 2024-03-04T07:45:21.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 -- 172.21.15.113:0/1720806914 wait complete. 2024-03-04T07:45:21.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 Processor -- start 2024-03-04T07:45:21.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 -- start start 2024-03-04T07:45:21.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 0x7f10f81f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10f8152b20 0x7f10f80ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10f815c340 0x7f10f8148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8158810 con 0x7f10f80c8b00 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80c3810 con 0x7f10f815c340 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10fe400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80c8490 con 0x7f10f8152b20 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 0x7f10f81f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 0x7f10f81f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52922/0 (socket says 172.21.15.113:52922) 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f7400640 1 -- 172.21.15.113:0/4150668244 learned_addr learned my addr 172.21.15.113:0/4150668244 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f6a00640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10f8152b20 0x7f10f80ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f7e00640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10f815c340 0x7f10f8148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f7400640 1 -- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10f8152b20 msgr2=0x7f10f80ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:21.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f7400640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10f8152b20 0x7f10f80ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f7400640 1 -- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10f815c340 msgr2=0x7f10f8148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:21.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f7400640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10f815c340 0x7f10f8148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:21.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f7400640 1 -- 172.21.15.113:0/4150668244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80cc760 con 0x7f10f80c8b00 2024-03-04T07:45:21.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.729+0000 7f10f7400640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 0x7f10f81f4140 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f10e800e8a0 tx=0x7f10e800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:21.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.733+0000 7f10ef400640 1 -- 172.21.15.113:0/4150668244 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8017070 con 0x7f10f80c8b00 2024-03-04T07:45:21.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.733+0000 7f10ef400640 1 -- 172.21.15.113:0/4150668244 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e800ee90 con 0x7f10f80c8b00 2024-03-04T07:45:21.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.733+0000 7f10fe400640 1 -- 172.21.15.113:0/4150668244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f815bd10 con 0x7f10f80c8b00 2024-03-04T07:45:21.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.733+0000 7f10ef400640 1 -- 172.21.15.113:0/4150668244 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e800cb80 con 0x7f10f80c8b00 2024-03-04T07:45:21.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.733+0000 7f10fe400640 1 -- 172.21.15.113:0/4150668244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f815ba30 con 0x7f10f80c8b00 2024-03-04T07:45:21.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.733+0000 7f10fe400640 1 -- 172.21.15.113:0/4150668244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f815bf30 con 0x7f10f80c8b00 2024-03-04T07:45:21.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.733+0000 7f10ef400640 1 -- 172.21.15.113:0/4150668244 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10e801a070 con 0x7f10f80c8b00 2024-03-04T07:45:21.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.733+0000 7f10ef400640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10d8066f90 0x7f10d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:21.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.733+0000 7f10ef400640 1 -- 172.21.15.113:0/4150668244 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f10e8085030 con 0x7f10f80c8b00 2024-03-04T07:45:21.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.733+0000 7f10f6a00640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10d8066f90 0x7f10d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:21.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.737+0000 7f10f6a00640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10d8066f90 0x7f10d8069450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f10e40ad000 tx=0x7f10e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:21.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:21.741+0000 7f10ef400640 1 -- 172.21.15.113:0/4150668244 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f10e80557b0 con 0x7f10f80c8b00 2024-03-04T07:45:22.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.017+0000 7f10fe400640 1 -- 172.21.15.113:0/4150668244 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10f806fdf0 con 0x7f10f80c8b00 2024-03-04T07:45:22.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.017+0000 7f10ef400640 1 -- 172.21.15.113:0/4150668244 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f10e8059850 con 0x7f10f80c8b00 2024-03-04T07:45:22.018 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:22.018 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:20.664962+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5529]},"id":1}]} 2024-03-04T07:45:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 51 2024-03-04T07:45:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 -- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10d8066f90 msgr2=0x7f10d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10d8066f90 0x7f10d8069450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f10e40ad000 tx=0x7f10e4008040 comp rx=0 tx=0).stop 2024-03-04T07:45:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 -- 172.21.15.113:0/4150668244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 msgr2=0x7f10f81f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 0x7f10f81f4140 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f10e800e8a0 tx=0x7f10e800ed60 comp rx=0 tx=0).stop 2024-03-04T07:45:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 -- 172.21.15.113:0/4150668244 shutdown_connections 2024-03-04T07:45:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10f815c340 0x7f10f8148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10f8152b20 0x7f10f80ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10d8066f90 0x7f10d8069450 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:22.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 --2- 172.21.15.113:0/4150668244 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10f80c8b00 0x7f10f81f4140 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:22.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 -- 172.21.15.113:0/4150668244 >> 172.21.15.113:0/4150668244 conn(0x7f10f80709a0 msgr2=0x7f10f80ca020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:22.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 -- 172.21.15.113:0/4150668244 shutdown_connections 2024-03-04T07:45:22.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:22.021+0000 7f10fe400640 1 -- 172.21.15.113:0/4150668244 wait complete. 2024-03-04T07:45:22.041 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 51, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 51, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5244": { "addr": "172.21.15.113:6837/742633123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 742633123, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 742633123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5244, "incarnation": 32, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 26 }, "gid_5529": { "addr": "172.21.15.113:6836/2299268553", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2299268553, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2299268553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5529, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 43 }, "gid_5631": { "addr": "172.21.15.113:6833/810789479", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 810789479, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 810789479, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5631, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:45:20.664962+0000", "qdb_cluster": [ 5244, 5529 ], "qdb_leader": 5244, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5244, "mds_1": 5529, "mds_2": 5631 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:45:24.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9354800640 1 Processor -- start 2024-03-04T07:45:24.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9354800640 1 -- start start 2024-03-04T07:45:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9354800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 0x7f934c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9354800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934c1591f0 con 0x7f934c152bd0 2024-03-04T07:45:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9354800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934c1594d0 con 0x7f934c0c8200 2024-03-04T07:45:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9354800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c071df0 con 0x7f934c1527f0 2024-03-04T07:45:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9352800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f934c152bd0 0x7f934c1587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33318/0 (socket says 172.21.15.113:33318) 2024-03-04T07:45:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9352800640 1 -- 172.21.15.113:0/52830347 learned_addr learned my addr 172.21.15.113:0/52830347 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:45:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9351e00640 1 --2- 172.21.15.113:0/52830347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 0x7f934c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9350a00640 1 -- 172.21.15.113:0/52830347 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889089283 0 0) 0x7f934c1591f0 con 0x7f934c152bd0 2024-03-04T07:45:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9350a00640 1 -- 172.21.15.113:0/52830347 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9334003530 con 0x7f934c152bd0 2024-03-04T07:45:24.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9350a00640 1 -- 172.21.15.113:0/52830347 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833354190 0 0) 0x7f934c1594d0 con 0x7f934c0c8200 2024-03-04T07:45:24.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9350a00640 1 -- 172.21.15.113:0/52830347 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f934c1591f0 con 0x7f934c0c8200 2024-03-04T07:45:24.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9351e00640 1 -- 172.21.15.113:0/52830347 >> v1:172.21.15.136:6789/0 conn(0x7f934c0c8200 legacy=0x7f934c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:24.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9351e00640 1 -- 172.21.15.113:0/52830347 >> v1:172.21.15.113:6789/0 conn(0x7f934c152bd0 legacy=0x7f934c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:24.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.180+0000 7f9351e00640 1 -- 172.21.15.113:0/52830347 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c0717a0 con 0x7f934c1527f0 2024-03-04T07:45:24.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9351e00640 1 --2- 172.21.15.113:0/52830347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 0x7f934c070d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93400097f0 tx=0x7f934001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b72c3a9c94cfb5c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9350a00640 1 -- 172.21.15.113:0/52830347 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93400ab5f0 con 0x7f934c1527f0 2024-03-04T07:45:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9350a00640 1 -- 172.21.15.113:0/52830347 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93400b0070 con 0x7f934c1527f0 2024-03-04T07:45:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9350a00640 1 -- 172.21.15.113:0/52830347 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93400ab990 con 0x7f934c1527f0 2024-03-04T07:45:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9354800640 1 -- 172.21.15.113:0/52830347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 msgr2=0x7f934c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9354800640 1 --2- 172.21.15.113:0/52830347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 0x7f934c070d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f93400097f0 tx=0x7f934001c4e0 comp rx=0 tx=0).stop 2024-03-04T07:45:24.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9354800640 1 -- 172.21.15.113:0/52830347 shutdown_connections 2024-03-04T07:45:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9354800640 1 --2- 172.21.15.113:0/52830347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 0x7f934c070d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9354800640 1 -- 172.21.15.113:0/52830347 >> 172.21.15.113:0/52830347 conn(0x7f934c06e820 msgr2=0x7f934c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9354800640 1 -- 172.21.15.113:0/52830347 shutdown_connections 2024-03-04T07:45:24.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.184+0000 7f9354800640 1 -- 172.21.15.113:0/52830347 wait complete. 2024-03-04T07:45:24.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 Processor -- start 2024-03-04T07:45:24.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 -- start start 2024-03-04T07:45:24.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f934c0c8200 0x7f934c0c2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:24.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 0x7f934c0c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:24.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f934c152bd0 0x7f934c1f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:24.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c166380 con 0x7f934c0c8200 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c1597b0 con 0x7f934c152bd0 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c166500 con 0x7f934c1527f0 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9351e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f934c0c8200 0x7f934c0c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9351e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f934c0c8200 0x7f934c0c2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52926/0 (socket says 172.21.15.113:52926) 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9351e00640 1 -- 172.21.15.113:0/1590144009 learned_addr learned my addr 172.21.15.113:0/1590144009 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9351400640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 0x7f934c0c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9352800640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f934c152bd0 0x7f934c1f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9351e00640 1 -- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 msgr2=0x7f934c0c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9351e00640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 0x7f934c0c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9351e00640 1 -- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f934c152bd0 msgr2=0x7f934c1f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:24.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9351e00640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f934c152bd0 0x7f934c1f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9351e00640 1 -- 172.21.15.113:0/1590144009 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c0c23d0 con 0x7f934c0c8200 2024-03-04T07:45:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9351e00640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f934c0c8200 0x7f934c0c2880 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f93400190f0 tx=0x7f9340019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9346000640 1 -- 172.21.15.113:0/1590144009 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340003820 con 0x7f934c0c8200 2024-03-04T07:45:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9346000640 1 -- 172.21.15.113:0/1590144009 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93400039c0 con 0x7f934c0c8200 2024-03-04T07:45:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9346000640 1 -- 172.21.15.113:0/1590144009 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340003d70 con 0x7f934c0c8200 2024-03-04T07:45:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 -- 172.21.15.113:0/1590144009 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934c152370 con 0x7f934c0c8200 2024-03-04T07:45:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 -- 172.21.15.113:0/1590144009 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f934c1558d0 con 0x7f934c0c8200 2024-03-04T07:45:24.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.188+0000 7f9354800640 1 -- 172.21.15.113:0/1590144009 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934c0c79d0 con 0x7f934c0c8200 2024-03-04T07:45:24.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.192+0000 7f9346000640 1 -- 172.21.15.113:0/1590144009 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93400c3060 con 0x7f934c0c8200 2024-03-04T07:45:24.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.192+0000 7f9346000640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9330066f60 0x7f9330069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:24.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.192+0000 7f9346000640 1 -- 172.21.15.113:0/1590144009 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f934011ed90 con 0x7f934c0c8200 2024-03-04T07:45:24.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.192+0000 7f9351400640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9330066f60 0x7f9330069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:24.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.196+0000 7f9351400640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9330066f60 0x7f9330069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f934c14e630 tx=0x7f933c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:24.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.200+0000 7f9346000640 1 -- 172.21.15.113:0/1590144009 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f93400160d0 con 0x7f934c0c8200 2024-03-04T07:45:24.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.396+0000 7f9354800640 1 -- 172.21.15.113:0/1590144009 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f934c075be0 con 0x7f934c0c8200 2024-03-04T07:45:24.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.396+0000 7f9346000640 1 -- 172.21.15.113:0/1590144009 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f93400efbf0 con 0x7f934c0c8200 2024-03-04T07:45:24.398 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:24.399 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:20.664962+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5529]},"id":1}]} 2024-03-04T07:45:24.399 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 51 2024-03-04T07:45:24.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 -- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9330066f60 msgr2=0x7f9330069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9330066f60 0x7f9330069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f934c14e630 tx=0x7f933c009040 comp rx=0 tx=0).stop 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 -- 172.21.15.113:0/1590144009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f934c0c8200 msgr2=0x7f934c0c2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f934c0c8200 0x7f934c0c2880 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f93400190f0 tx=0x7f9340019280 comp rx=0 tx=0).stop 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 -- 172.21.15.113:0/1590144009 shutdown_connections 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f934c152bd0 0x7f934c1f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f934c1527f0 0x7f934c0c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9330066f60 0x7f9330069420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 --2- 172.21.15.113:0/1590144009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f934c0c8200 0x7f934c0c2880 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 -- 172.21.15.113:0/1590144009 >> 172.21.15.113:0/1590144009 conn(0x7f934c06e820 msgr2=0x7f934c1586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 -- 172.21.15.113:0/1590144009 shutdown_connections 2024-03-04T07:45:24.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:24.400+0000 7f9354800640 1 -- 172.21.15.113:0/1590144009 wait complete. 2024-03-04T07:45:24.420 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:45:26.421 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:26.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b68400640 1 Processor -- start 2024-03-04T07:45:26.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b68400640 1 -- start start 2024-03-04T07:45:26.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b68400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b600c85f0 0x7f6b6006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:26.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b68400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6b6006b5a0 0x7f6b60161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:26.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b68400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b600c1ec0 con 0x7f6b600b20b0 2024-03-04T07:45:26.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b68400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b600c9b30 con 0x7f6b600c85f0 2024-03-04T07:45:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b68400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b600c12a0 con 0x7f6b6006b5a0 2024-03-04T07:45:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b65000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6b600c85f0 0x7f6b6006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b65a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6b600b20b0 0x7f6b60157710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33334/0 (socket says 172.21.15.113:33334) 2024-03-04T07:45:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b65a00640 1 -- 172.21.15.113:0/1231299947 learned_addr learned my addr 172.21.15.113:0/1231299947 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:45:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b66400640 1 --2- 172.21.15.113:0/1231299947 >> v2:172.21.15.136:3300/0 conn(0x7f6b6006b5a0 0x7f6b60161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b65000640 1 -- 172.21.15.113:0/1231299947 >> v2:172.21.15.136:3300/0 conn(0x7f6b6006b5a0 msgr2=0x7f6b60161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b65000640 1 --2- 172.21.15.113:0/1231299947 >> v2:172.21.15.136:3300/0 conn(0x7f6b6006b5a0 0x7f6b60161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b65000640 1 -- 172.21.15.113:0/1231299947 >> v1:172.21.15.113:6789/0 conn(0x7f6b600b20b0 legacy=0x7f6b60157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b65000640 1 -- 172.21.15.113:0/1231299947 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600c1580 con 0x7f6b600c85f0 2024-03-04T07:45:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.576+0000 7f6b5fe00640 1 -- 172.21.15.113:0/1231299947 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800700719 0 0) 0x7f6b600c1ec0 con 0x7f6b600b20b0 2024-03-04T07:45:26.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.580+0000 7f6b65000640 1 --2- 172.21.15.113:0/1231299947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 0x7f6b6006b060 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6b600c96d0 tx=0x7f6b4c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e939964fe2abfdd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.580+0000 7f6b5fe00640 1 -- 172.21.15.113:0/1231299947 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c0b5070 con 0x7f6b600c85f0 2024-03-04T07:45:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.580+0000 7f6b5fe00640 1 -- 172.21.15.113:0/1231299947 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c0a8970 con 0x7f6b600c85f0 2024-03-04T07:45:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.580+0000 7f6b5fe00640 1 -- 172.21.15.113:0/1231299947 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c0a8d40 con 0x7f6b600c85f0 2024-03-04T07:45:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 -- 172.21.15.113:0/1231299947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 msgr2=0x7f6b6006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 --2- 172.21.15.113:0/1231299947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 0x7f6b6006b060 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6b600c96d0 tx=0x7f6b4c01ec70 comp rx=0 tx=0).stop 2024-03-04T07:45:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 -- 172.21.15.113:0/1231299947 shutdown_connections 2024-03-04T07:45:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 --2- 172.21.15.113:0/1231299947 >> v2:172.21.15.136:3300/0 conn(0x7f6b6006b5a0 0x7f6b60161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 --2- 172.21.15.113:0/1231299947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 0x7f6b6006b060 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 -- 172.21.15.113:0/1231299947 >> 172.21.15.113:0/1231299947 conn(0x7f6b6006e820 msgr2=0x7f6b6006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 -- 172.21.15.113:0/1231299947 shutdown_connections 2024-03-04T07:45:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 -- 172.21.15.113:0/1231299947 wait complete. 2024-03-04T07:45:26.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 Processor -- start 2024-03-04T07:45:26.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 -- start start 2024-03-04T07:45:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b6006b5a0 0x7f6b60143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b600b20b0 0x7f6b600b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 0x7f6b60141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b60157e20 con 0x7f6b600c85f0 2024-03-04T07:45:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601571f0 con 0x7f6b600b20b0 2024-03-04T07:45:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b68400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600c2400 con 0x7f6b6006b5a0 2024-03-04T07:45:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b66400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 0x7f6b60141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b66400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 0x7f6b60141890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52942/0 (socket says 172.21.15.113:52942) 2024-03-04T07:45:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b66400640 1 -- 172.21.15.113:0/1322010513 learned_addr learned my addr 172.21.15.113:0/1322010513 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b65000640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b600b20b0 0x7f6b600b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b65a00640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b6006b5a0 0x7f6b60143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b66400640 1 -- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b6006b5a0 msgr2=0x7f6b60143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b66400640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b6006b5a0 0x7f6b60143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b66400640 1 -- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b600b20b0 msgr2=0x7f6b600b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b66400640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b600b20b0 0x7f6b600b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.584+0000 7f6b66400640 1 -- 172.21.15.113:0/1322010513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600afe60 con 0x7f6b600c85f0 2024-03-04T07:45:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b66400640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 0x7f6b60141890 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007e90 tx=0x7f6b5800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b5d600640 1 -- 172.21.15.113:0/1322010513 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58014420 con 0x7f6b600c85f0 2024-03-04T07:45:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b5d600640 1 -- 172.21.15.113:0/1322010513 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b58019070 con 0x7f6b600c85f0 2024-03-04T07:45:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b68400640 1 -- 172.21.15.113:0/1322010513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6014a1b0 con 0x7f6b600c85f0 2024-03-04T07:45:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b5d600640 1 -- 172.21.15.113:0/1322010513 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58014800 con 0x7f6b600c85f0 2024-03-04T07:45:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b68400640 1 -- 172.21.15.113:0/1322010513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b600b0220 con 0x7f6b600c85f0 2024-03-04T07:45:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b68400640 1 -- 172.21.15.113:0/1322010513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b60075c00 con 0x7f6b600c85f0 2024-03-04T07:45:26.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b5d600640 1 -- 172.21.15.113:0/1322010513 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b580149a0 con 0x7f6b600c85f0 2024-03-04T07:45:26.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b5d600640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b40066f90 0x7f6b40069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:26.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b5d600640 1 -- 172.21.15.113:0/1322010513 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f6b58085980 con 0x7f6b600c85f0 2024-03-04T07:45:26.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.588+0000 7f6b65a00640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b40066f90 0x7f6b40069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.592+0000 7f6b65a00640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b40066f90 0x7f6b40069450 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6b54007fd0 tx=0x7f6b54007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:26.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.592+0000 7f6b5d600640 1 -- 172.21.15.113:0/1322010513 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6b600afe60 con 0x7f6b600c85f0 2024-03-04T07:45:26.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.828+0000 7f6b68400640 1 -- 172.21.15.113:0/1322010513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b6014ab90 con 0x7f6b600c85f0 2024-03-04T07:45:26.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.828+0000 7f6b5d600640 1 -- 172.21.15.113:0/1322010513 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5814 (secure 0 0 0) 0x7f6b58056060 con 0x7f6b600c85f0 2024-03-04T07:45:26.829 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:26.830 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:25.679421+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:resolve","state_seq":11,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244]},"id":1}]} 2024-03-04T07:45:26.830 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-03-04T07:45:26.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.832+0000 7f6b68400640 1 -- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b40066f90 msgr2=0x7f6b40069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:26.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.832+0000 7f6b68400640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b40066f90 0x7f6b40069450 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6b54007fd0 tx=0x7f6b54007d60 comp rx=0 tx=0).stop 2024-03-04T07:45:26.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.832+0000 7f6b68400640 1 -- 172.21.15.113:0/1322010513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 msgr2=0x7f6b60141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:26.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.832+0000 7f6b68400640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 0x7f6b60141890 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007e90 tx=0x7f6b5800e4b0 comp rx=0 tx=0).stop 2024-03-04T07:45:26.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.832+0000 7f6b68400640 1 -- 172.21.15.113:0/1322010513 shutdown_connections 2024-03-04T07:45:26.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.832+0000 7f6b68400640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6b40066f90 0x7f6b40069450 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:26.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.832+0000 7f6b68400640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6b600c85f0 0x7f6b60141890 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:26.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.832+0000 7f6b68400640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b600b20b0 0x7f6b600b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:26.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.832+0000 7f6b68400640 1 --2- 172.21.15.113:0/1322010513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6b6006b5a0 0x7f6b60143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:26.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.832+0000 7f6b68400640 1 -- 172.21.15.113:0/1322010513 >> 172.21.15.113:0/1322010513 conn(0x7f6b6006e820 msgr2=0x7f6b60072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.836+0000 7f6b68400640 1 -- 172.21.15.113:0/1322010513 shutdown_connections 2024-03-04T07:45:26.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:26.836+0000 7f6b68400640 1 -- 172.21.15.113:0/1322010513 wait complete. 2024-03-04T07:45:26.863 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:45:27.284 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:45:27.288 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:45:28.864 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:29.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fbb600640 1 Processor -- start 2024-03-04T07:45:29.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fbb600640 1 -- start start 2024-03-04T07:45:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fbb600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9fb41527f0 0x7f9fb4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fbb600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9fb4152bd0 0x7f9fb41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fbb600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb41591f0 con 0x7f9fb40c8200 2024-03-04T07:45:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fbb600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb4071df0 con 0x7f9fb41527f0 2024-03-04T07:45:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fbb600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb41597b0 con 0x7f9fb4152bd0 2024-03-04T07:45:29.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fb8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9fb41527f0 0x7f9fb4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fb8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9fb41527f0 0x7f9fb4070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37436/0 (socket says 172.21.15.113:37436) 2024-03-04T07:45:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fb3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9fb40c8200 0x7f9fb4072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52476/0 (socket says 172.21.15.113:52476) 2024-03-04T07:45:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.004+0000 7f9fb8c00640 1 -- 172.21.15.113:0/1581482907 learned_addr learned my addr 172.21.15.113:0/1581482907 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.008+0000 7f9fb9600640 1 --2- 172.21.15.113:0/1581482907 >> v2:172.21.15.136:3300/0 conn(0x7f9fb4152bd0 0x7f9fb41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.008+0000 7f9fb8c00640 1 -- 172.21.15.113:0/1581482907 >> v2:172.21.15.136:3300/0 conn(0x7f9fb4152bd0 msgr2=0x7f9fb41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.008+0000 7f9fb8c00640 1 --2- 172.21.15.113:0/1581482907 >> v2:172.21.15.136:3300/0 conn(0x7f9fb4152bd0 0x7f9fb41587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.008+0000 7f9fb8c00640 1 -- 172.21.15.113:0/1581482907 >> v1:172.21.15.113:6789/0 conn(0x7f9fb40c8200 legacy=0x7f9fb4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:29.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.008+0000 7f9fb8c00640 1 -- 172.21.15.113:0/1581482907 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb41594d0 con 0x7f9fb41527f0 2024-03-04T07:45:29.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fb8c00640 1 --2- 172.21.15.113:0/1581482907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 0x7f9fb4070d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40712b0 tx=0x7f9fa801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4f8e58eeda86b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:29.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fb3400640 1 -- 172.21.15.113:0/1581482907 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80ab430 con 0x7f9fb41527f0 2024-03-04T07:45:29.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fb3400640 1 -- 172.21.15.113:0/1581482907 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa80b0070 con 0x7f9fb41527f0 2024-03-04T07:45:29.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fb3400640 1 -- 172.21.15.113:0/1581482907 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80ab7f0 con 0x7f9fb41527f0 2024-03-04T07:45:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fbb600640 1 -- 172.21.15.113:0/1581482907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 msgr2=0x7f9fb4070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fbb600640 1 --2- 172.21.15.113:0/1581482907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 0x7f9fb4070d40 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40712b0 tx=0x7f9fa801c520 comp rx=0 tx=0).stop 2024-03-04T07:45:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fbb600640 1 -- 172.21.15.113:0/1581482907 shutdown_connections 2024-03-04T07:45:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fbb600640 1 --2- 172.21.15.113:0/1581482907 >> v2:172.21.15.136:3300/0 conn(0x7f9fb4152bd0 0x7f9fb41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fbb600640 1 --2- 172.21.15.113:0/1581482907 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 0x7f9fb4070d40 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fbb600640 1 -- 172.21.15.113:0/1581482907 >> 172.21.15.113:0/1581482907 conn(0x7f9fb406e820 msgr2=0x7f9fb406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fbb600640 1 -- 172.21.15.113:0/1581482907 shutdown_connections 2024-03-04T07:45:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.012+0000 7f9fbb600640 1 -- 172.21.15.113:0/1581482907 wait complete. 2024-03-04T07:45:29.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 Processor -- start 2024-03-04T07:45:29.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 -- start start 2024-03-04T07:45:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fb40c8200 0x7f9fb4148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 0x7f9fb4150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fb4152bd0 0x7f9fb4150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb41654f0 con 0x7f9fb41527f0 2024-03-04T07:45:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4165370 con 0x7f9fb4152bd0 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4165670 con 0x7f9fb40c8200 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 0x7f9fb4150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 0x7f9fb4150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37450/0 (socket says 172.21.15.113:37450) 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb3e00640 1 -- 172.21.15.113:0/4248715070 learned_addr learned my addr 172.21.15.113:0/4248715070 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb8c00640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fb40c8200 0x7f9fb4148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb9600640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fb4152bd0 0x7f9fb4150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb3e00640 1 -- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fb40c8200 msgr2=0x7f9fb4148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb3e00640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fb40c8200 0x7f9fb4148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb3e00640 1 -- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fb4152bd0 msgr2=0x7f9fb4150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb3e00640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fb4152bd0 0x7f9fb4150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb3e00640 1 -- 172.21.15.113:0/4248715070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb414be20 con 0x7f9fb41527f0 2024-03-04T07:45:29.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb3e00640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 0x7f9fb4150140 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c460 tx=0x7f9fa000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:29.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb0c00640 1 -- 172.21.15.113:0/4248715070 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0018070 con 0x7f9fb41527f0 2024-03-04T07:45:29.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb0c00640 1 -- 172.21.15.113:0/4248715070 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa0007480 con 0x7f9fb41527f0 2024-03-04T07:45:29.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fb0c00640 1 -- 172.21.15.113:0/4248715070 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa00047f0 con 0x7f9fb41527f0 2024-03-04T07:45:29.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 -- 172.21.15.113:0/4248715070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb4150c60 con 0x7f9fb41527f0 2024-03-04T07:45:29.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 -- 172.21.15.113:0/4248715070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb414c090 con 0x7f9fb41527f0 2024-03-04T07:45:29.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.016+0000 7f9fbb600640 1 -- 172.21.15.113:0/4248715070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb4153100 con 0x7f9fb41527f0 2024-03-04T07:45:29.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.020+0000 7f9fb0c00640 1 -- 172.21.15.113:0/4248715070 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9fa001b070 con 0x7f9fb41527f0 2024-03-04T07:45:29.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.020+0000 7f9fb0c00640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f94066f90 0x7f9f94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:29.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.020+0000 7f9fb0c00640 1 -- 172.21.15.113:0/4248715070 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f9fa0084590 con 0x7f9fb41527f0 2024-03-04T07:45:29.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.020+0000 7f9fb8c00640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f94066f90 0x7f9f94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:29.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.024+0000 7f9fb8c00640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f94066f90 0x7f9f94069450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80af000 tx=0x7f9fa80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:29.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.024+0000 7f9fb0c00640 1 -- 172.21.15.113:0/4248715070 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9fb4153100 con 0x7f9fb41527f0 2024-03-04T07:45:29.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.312+0000 7f9fbb600640 1 -- 172.21.15.113:0/4248715070 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fb414c290 con 0x7f9fb41527f0 2024-03-04T07:45:29.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.312+0000 7f9fb0c00640 1 -- 172.21.15.113:0/4248715070 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5814 (secure 0 0 0) 0x7f9fb414c290 con 0x7f9fb41527f0 2024-03-04T07:45:29.315 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:29.316 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:25.679421+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:resolve","state_seq":11,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244]},"id":1}]} 2024-03-04T07:45:29.316 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 52 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 -- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f94066f90 msgr2=0x7f9f94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f94066f90 0x7f9f94069450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9fa80af000 tx=0x7f9fa80a7000 comp rx=0 tx=0).stop 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 -- 172.21.15.113:0/4248715070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 msgr2=0x7f9fb4150140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 0x7f9fb4150140 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000c460 tx=0x7f9fa000c920 comp rx=0 tx=0).stop 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 -- 172.21.15.113:0/4248715070 shutdown_connections 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fb4152bd0 0x7f9fb4150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f94066f90 0x7f9f94069450 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fb41527f0 0x7f9fb4150140 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 --2- 172.21.15.113:0/4248715070 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9fb40c8200 0x7f9fb4148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 -- 172.21.15.113:0/4248715070 >> 172.21.15.113:0/4248715070 conn(0x7f9fb406e820 msgr2=0x7f9fb40ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:29.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 -- 172.21.15.113:0/4248715070 shutdown_connections 2024-03-04T07:45:29.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:29.316+0000 7f9fbb600640 1 -- 172.21.15.113:0/4248715070 wait complete. 2024-03-04T07:45:29.338 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:45:31.339 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:31.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d0e400640 1 Processor -- start 2024-03-04T07:45:31.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d0e400640 1 -- start start 2024-03-04T07:45:31.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d0e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d08151b40 0x7f5d080c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:31.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d0e400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5d080c18d0 0x7f5d08071f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:31.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d0e400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0806c290 con 0x7f5d08151760 2024-03-04T07:45:31.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d0e400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0815aef0 con 0x7f5d08151b40 2024-03-04T07:45:31.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d0e400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0806ef10 con 0x7f5d080c18d0 2024-03-04T07:45:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d07e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d08151b40 0x7f5d080c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d07e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d08151b40 0x7f5d080c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37454/0 (socket says 172.21.15.113:37454) 2024-03-04T07:45:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d07e00640 1 -- 172.21.15.113:0/429870920 learned_addr learned my addr 172.21.15.113:0/429870920 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d0da00640 1 --2- 172.21.15.113:0/429870920 >> v2:172.21.15.136:3300/0 conn(0x7f5d080c18d0 0x7f5d08071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d07e00640 1 -- 172.21.15.113:0/429870920 >> v2:172.21.15.136:3300/0 conn(0x7f5d080c18d0 msgr2=0x7f5d08071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d07e00640 1 --2- 172.21.15.113:0/429870920 >> v2:172.21.15.136:3300/0 conn(0x7f5d080c18d0 0x7f5d08071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d07e00640 1 -- 172.21.15.113:0/429870920 >> v1:172.21.15.136:6789/0 conn(0x7f5d08151760 legacy=0x7f5d0815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.504+0000 7f5d07e00640 1 -- 172.21.15.113:0/429870920 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0815ba90 con 0x7f5d08151b40 2024-03-04T07:45:31.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.508+0000 7f5d07e00640 1 --2- 172.21.15.113:0/429870920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d08151b40 0x7f5d080c83d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5d08070260 tx=0x7f5cf401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e32a80486a13c34 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:31.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.508+0000 7f5d07400640 1 -- 172.21.15.113:0/429870920 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40b1070 con 0x7f5d08151b40 2024-03-04T07:45:31.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.508+0000 7f5d07400640 1 -- 172.21.15.113:0/429870920 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf40ac600 con 0x7f5d08151b40 2024-03-04T07:45:31.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.508+0000 7f5d07400640 1 -- 172.21.15.113:0/429870920 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40ac9d0 con 0x7f5d08151b40 2024-03-04T07:45:31.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.512+0000 7f5d0e400640 1 -- 172.21.15.113:0/429870920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d08151b40 msgr2=0x7f5d080c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:31.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.512+0000 7f5d0e400640 1 --2- 172.21.15.113:0/429870920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d08151b40 0x7f5d080c83d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5d08070260 tx=0x7f5cf401c490 comp rx=0 tx=0).stop 2024-03-04T07:45:31.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.512+0000 7f5d0e400640 1 -- 172.21.15.113:0/429870920 shutdown_connections 2024-03-04T07:45:31.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.512+0000 7f5d0e400640 1 --2- 172.21.15.113:0/429870920 >> v2:172.21.15.136:3300/0 conn(0x7f5d080c18d0 0x7f5d08071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:31.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.512+0000 7f5d0e400640 1 --2- 172.21.15.113:0/429870920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d08151b40 0x7f5d080c83d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:31.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.512+0000 7f5d0e400640 1 -- 172.21.15.113:0/429870920 >> 172.21.15.113:0/429870920 conn(0x7f5d0806e2a0 msgr2=0x7f5d0806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:31.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.512+0000 7f5d0e400640 1 -- 172.21.15.113:0/429870920 shutdown_connections 2024-03-04T07:45:31.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.512+0000 7f5d0e400640 1 -- 172.21.15.113:0/429870920 wait complete. 2024-03-04T07:45:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.512+0000 7f5d0e400640 1 Processor -- start 2024-03-04T07:45:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.512+0000 7f5d0e400640 1 -- start start 2024-03-04T07:45:31.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0e400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d080c18d0 0x7f5d0814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:31.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d08151760 0x7f5d0813ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:31.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d08151b40 0x7f5d0813efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:31.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080728c0 con 0x7f5d08151760 2024-03-04T07:45:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0e400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0815ab80 con 0x7f5d080c18d0 2024-03-04T07:45:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080c1e20 con 0x7f5d08151b40 2024-03-04T07:45:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d08151b40 0x7f5d0813efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d080c18d0 0x7f5d0814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d08151b40 0x7f5d0813efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:60642/0 (socket says 172.21.15.113:60642) 2024-03-04T07:45:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0da00640 1 -- 172.21.15.113:0/2595078206 learned_addr learned my addr 172.21.15.113:0/2595078206 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d07e00640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d08151760 0x7f5d0813ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:31.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0da00640 1 -- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d080c18d0 msgr2=0x7f5d0814f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:31.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0da00640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d080c18d0 0x7f5d0814f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:31.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0da00640 1 -- 172.21.15.113:0/2595078206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d08151760 msgr2=0x7f5d0813ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0da00640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d08151760 0x7f5d0813ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0da00640 1 -- 172.21.15.113:0/2595078206 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d08150270 con 0x7f5d08151b40 2024-03-04T07:45:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0d000640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d080c18d0 0x7f5d0814f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:45:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d07e00640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d08151760 0x7f5d0813ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:45:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0da00640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d08151b40 0x7f5d0813efb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e990 tx=0x7f5d0000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d04c00640 1 -- 172.21.15.113:0/2595078206 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d00017070 con 0x7f5d08151b40 2024-03-04T07:45:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0e400640 1 -- 172.21.15.113:0/2595078206 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0814f8e0 con 0x7f5d08151b40 2024-03-04T07:45:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d0e400640 1 -- 172.21.15.113:0/2595078206 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d081507f0 con 0x7f5d08151b40 2024-03-04T07:45:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d04c00640 1 -- 172.21.15.113:0/2595078206 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0000c920 con 0x7f5d08151b40 2024-03-04T07:45:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.516+0000 7f5d04c00640 1 -- 172.21.15.113:0/2595078206 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000ccf0 con 0x7f5d08151b40 2024-03-04T07:45:31.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.520+0000 7f5d0e400640 1 -- 172.21.15.113:0/2595078206 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce00052a0 con 0x7f5d08151b40 2024-03-04T07:45:31.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.520+0000 7f5d04c00640 1 -- 172.21.15.113:0/2595078206 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d0001a070 con 0x7f5d08151b40 2024-03-04T07:45:31.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.520+0000 7f5d04c00640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce8066f60 0x7f5ce8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:31.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.520+0000 7f5d04c00640 1 -- 172.21.15.113:0/2595078206 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f5d000855c0 con 0x7f5d08151b40 2024-03-04T07:45:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.520+0000 7f5d0d000640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce8066f60 0x7f5ce8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.524+0000 7f5d0d000640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce8066f60 0x7f5ce8069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800f290 tx=0x7f5cf8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:31.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.532+0000 7f5d04c00640 1 -- 172.21.15.113:0/2595078206 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5d00055c90 con 0x7f5d08151b40 2024-03-04T07:45:31.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.832+0000 7f5d0e400640 1 -- 172.21.15.113:0/2595078206 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ce0005590 con 0x7f5d08151b40 2024-03-04T07:45:31.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.832+0000 7f5d04c00640 1 -- 172.21.15.113:0/2595078206 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f5d00059d30 con 0x7f5d08151b40 2024-03-04T07:45:31.835 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:31.835 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:30.979810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5529]},"id":1}]} 2024-03-04T07:45:31.836 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 53 2024-03-04T07:45:31.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 -- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce8066f60 msgr2=0x7f5ce8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:31.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce8066f60 0x7f5ce8069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800f290 tx=0x7f5cf8009040 comp rx=0 tx=0).stop 2024-03-04T07:45:31.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 -- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d08151b40 msgr2=0x7f5d0813efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:31.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d08151b40 0x7f5d0813efb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e990 tx=0x7f5d0000ee50 comp rx=0 tx=0).stop 2024-03-04T07:45:31.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 -- 172.21.15.113:0/2595078206 shutdown_connections 2024-03-04T07:45:31.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d08151b40 0x7f5d0813efb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:31.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce8066f60 0x7f5ce8069420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:31.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d08151760 0x7f5d0813ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:31.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 --2- 172.21.15.113:0/2595078206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d080c18d0 0x7f5d0814f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:31.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 -- 172.21.15.113:0/2595078206 >> 172.21.15.113:0/2595078206 conn(0x7f5d0806e2a0 msgr2=0x7f5d08160d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:31.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 -- 172.21.15.113:0/2595078206 shutdown_connections 2024-03-04T07:45:31.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:31.836+0000 7f5d0e400640 1 -- 172.21.15.113:0/2595078206 wait complete. 2024-03-04T07:45:31.864 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:45:33.866 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:34.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.004+0000 7f240fe00640 1 Processor -- start 2024-03-04T07:45:34.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240fe00640 1 -- start start 2024-03-04T07:45:34.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408158210 0x7f24080c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:34.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24080733f0 0x7f24080cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:34.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240fe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f240806b060 0x7f24080cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:34.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2408072fb0 con 0x7f24080733f0 2024-03-04T07:45:34.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240fe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24080b2230 con 0x7f240806b060 2024-03-04T07:45:34.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24080c7bb0 con 0x7f2408158210 2024-03-04T07:45:34.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24080733f0 0x7f24080cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:34.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24080733f0 0x7f24080cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37476/0 (socket says 172.21.15.113:37476) 2024-03-04T07:45:34.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240ca00640 1 -- 172.21.15.113:0/3506269432 learned_addr learned my addr 172.21.15.113:0/3506269432 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:34.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240d400640 1 --2- 172.21.15.113:0/3506269432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408158210 0x7f24080c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:34.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240de00640 1 --2- 172.21.15.113:0/3506269432 >> v2:172.21.15.136:3300/0 conn(0x7f240806b060 0x7f24080cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:34.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240ca00640 1 -- 172.21.15.113:0/3506269432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408158210 msgr2=0x7f24080c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:34.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240ca00640 1 --2- 172.21.15.113:0/3506269432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408158210 0x7f24080c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240ca00640 1 -- 172.21.15.113:0/3506269432 >> v2:172.21.15.136:3300/0 conn(0x7f240806b060 msgr2=0x7f24080cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:34.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240ca00640 1 --2- 172.21.15.113:0/3506269432 >> v2:172.21.15.136:3300/0 conn(0x7f240806b060 0x7f24080cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.008+0000 7f240ca00640 1 -- 172.21.15.113:0/3506269432 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24080c7e50 con 0x7f24080733f0 2024-03-04T07:45:34.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240ca00640 1 --2- 172.21.15.113:0/3506269432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24080733f0 0x7f24080cd010 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f24080c3ac0 tx=0x7f23f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8466b46d540641dc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:34.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f2407e00640 1 -- 172.21.15.113:0/3506269432 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40b1070 con 0x7f24080733f0 2024-03-04T07:45:34.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f2407e00640 1 -- 172.21.15.113:0/3506269432 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f40ac630 con 0x7f24080733f0 2024-03-04T07:45:34.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f2407e00640 1 -- 172.21.15.113:0/3506269432 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40a8050 con 0x7f24080733f0 2024-03-04T07:45:34.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 -- 172.21.15.113:0/3506269432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24080733f0 msgr2=0x7f24080cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:34.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 --2- 172.21.15.113:0/3506269432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24080733f0 0x7f24080cd010 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f24080c3ac0 tx=0x7f23f401c520 comp rx=0 tx=0).stop 2024-03-04T07:45:34.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 -- 172.21.15.113:0/3506269432 shutdown_connections 2024-03-04T07:45:34.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 --2- 172.21.15.113:0/3506269432 >> v2:172.21.15.136:3300/0 conn(0x7f240806b060 0x7f24080cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 --2- 172.21.15.113:0/3506269432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24080733f0 0x7f24080cd010 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 --2- 172.21.15.113:0/3506269432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408158210 0x7f24080c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 -- 172.21.15.113:0/3506269432 >> 172.21.15.113:0/3506269432 conn(0x7f24080671b0 msgr2=0x7f240806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:34.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 -- 172.21.15.113:0/3506269432 shutdown_connections 2024-03-04T07:45:34.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 -- 172.21.15.113:0/3506269432 wait complete. 2024-03-04T07:45:34.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 Processor -- start 2024-03-04T07:45:34.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.012+0000 7f240fe00640 1 -- start start 2024-03-04T07:45:34.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f240806b060 0x7f2408151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:34.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24080733f0 0x7f2408152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:34.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2408158210 0x7f24081497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:34.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240806cff0 con 0x7f2408158210 2024-03-04T07:45:34.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240806ce70 con 0x7f240806b060 2024-03-04T07:45:34.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24080c9be0 con 0x7f24080733f0 2024-03-04T07:45:34.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2408158210 0x7f24081497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:34.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f240806b060 0x7f2408151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2408158210 0x7f24081497f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37484/0 (socket says 172.21.15.113:37484) 2024-03-04T07:45:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240de00640 1 -- 172.21.15.113:0/1769770424 learned_addr learned my addr 172.21.15.113:0/1769770424 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240ca00640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24080733f0 0x7f2408152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240de00640 1 -- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24080733f0 msgr2=0x7f2408152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240de00640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24080733f0 0x7f2408152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240de00640 1 -- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f240806b060 msgr2=0x7f2408151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:34.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240de00640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f240806b060 0x7f2408151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240de00640 1 -- 172.21.15.113:0/1769770424 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24080b1b60 con 0x7f2408158210 2024-03-04T07:45:34.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240de00640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2408158210 0x7f24081497f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2400007e90 tx=0x7f240000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:34.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f2405600640 1 -- 172.21.15.113:0/1769770424 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400014420 con 0x7f2408158210 2024-03-04T07:45:34.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f2405600640 1 -- 172.21.15.113:0/1769770424 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2400019070 con 0x7f2408158210 2024-03-04T07:45:34.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f2405600640 1 -- 172.21.15.113:0/1769770424 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400014820 con 0x7f2408158210 2024-03-04T07:45:34.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240fe00640 1 -- 172.21.15.113:0/1769770424 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24080cc190 con 0x7f2408158210 2024-03-04T07:45:34.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240fe00640 1 -- 172.21.15.113:0/1769770424 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24080cc750 con 0x7f2408158210 2024-03-04T07:45:34.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.016+0000 7f240fe00640 1 -- 172.21.15.113:0/1769770424 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2408075c00 con 0x7f2408158210 2024-03-04T07:45:34.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.020+0000 7f2405600640 1 -- 172.21.15.113:0/1769770424 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24000149c0 con 0x7f2408158210 2024-03-04T07:45:34.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.020+0000 7f2405600640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e8066f90 0x7f23e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:34.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.020+0000 7f2405600640 1 -- 172.21.15.113:0/1769770424 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f2400085b00 con 0x7f2408158210 2024-03-04T07:45:34.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.020+0000 7f240d400640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e8066f90 0x7f23e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:34.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.020+0000 7f240d400640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e8066f90 0x7f23e8069450 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009ac0 tx=0x7f23fc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:34.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.024+0000 7f2405600640 1 -- 172.21.15.113:0/1769770424 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f24000560c0 con 0x7f2408158210 2024-03-04T07:45:34.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 -- 172.21.15.113:0/1769770424 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24080cd380 con 0x7f2408158210 2024-03-04T07:45:34.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f2405600640 1 -- 172.21.15.113:0/1769770424 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f240005a160 con 0x7f2408158210 2024-03-04T07:45:34.225 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:34.225 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:30.979810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:reconnect","state_seq":13,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5529]},"id":1}]} 2024-03-04T07:45:34.225 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 53 2024-03-04T07:45:34.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 -- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e8066f90 msgr2=0x7f23e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:34.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e8066f90 0x7f23e8069450 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009ac0 tx=0x7f23fc009190 comp rx=0 tx=0).stop 2024-03-04T07:45:34.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 -- 172.21.15.113:0/1769770424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2408158210 msgr2=0x7f24081497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2408158210 0x7f24081497f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2400007e90 tx=0x7f240000e4b0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 -- 172.21.15.113:0/1769770424 shutdown_connections 2024-03-04T07:45:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e8066f90 0x7f23e8069450 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2408158210 0x7f24081497f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24080733f0 0x7f2408152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 --2- 172.21.15.113:0/1769770424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f240806b060 0x7f2408151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 -- 172.21.15.113:0/1769770424 >> 172.21.15.113:0/1769770424 conn(0x7f24080671b0 msgr2=0x7f24081563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 -- 172.21.15.113:0/1769770424 shutdown_connections 2024-03-04T07:45:34.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:34.224+0000 7f240fe00640 1 -- 172.21.15.113:0/1769770424 wait complete. 2024-03-04T07:45:34.254 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 53, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 53, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5244": { "addr": "172.21.15.113:6837/742633123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 742633123, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 742633123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5244, "incarnation": 32, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 26 }, "gid_5529": { "addr": "172.21.15.113:6836/2299268553", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2299268553, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2299268553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5529, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 43 }, "gid_5631": { "addr": "172.21.15.113:6833/810789479", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 810789479, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 810789479, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5631, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:reconnect", "state_seq": 13 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:45:30.979810+0000", "qdb_cluster": [ 5244, 5529 ], "qdb_leader": 5244, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5244, "mds_1": 5529, "mds_2": 5631 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:45:36.255 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:36.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8024000640 1 Processor -- start 2024-03-04T07:45:36.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8024000640 1 -- start start 2024-03-04T07:45:36.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8024000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 0x7f801c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:36.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8024000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f801c0c2d80 0x7f801c15b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:36.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8024000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801c0c27f0 con 0x7f801c14e4c0 2024-03-04T07:45:36.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8024000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801c15ad60 con 0x7f801c0c2d80 2024-03-04T07:45:36.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8024000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0c2a90 con 0x7f801c0c22e0 2024-03-04T07:45:36.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8022000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f801c0c2d80 0x7f801c15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:36.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8021600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f801c14e4c0 0x7f801c072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52480/0 (socket says 172.21.15.113:52480) 2024-03-04T07:45:36.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8022000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f801c0c2d80 0x7f801c15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37496/0 (socket says 172.21.15.113:37496) 2024-03-04T07:45:36.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8021600640 1 -- 172.21.15.113:0/542996592 learned_addr learned my addr 172.21.15.113:0/542996592 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:45:36.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8020c00640 1 --2- 172.21.15.113:0/542996592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 0x7f801c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:36.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8022000640 1 -- 172.21.15.113:0/542996592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 msgr2=0x7f801c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:36.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8022000640 1 --2- 172.21.15.113:0/542996592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 0x7f801c15b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:36.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8022000640 1 -- 172.21.15.113:0/542996592 >> v1:172.21.15.113:6789/0 conn(0x7f801c14e4c0 legacy=0x7f801c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:36.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.392+0000 7f8022000640 1 -- 172.21.15.113:0/542996592 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c070c60 con 0x7f801c0c2d80 2024-03-04T07:45:36.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8022000640 1 --2- 172.21.15.113:0/542996592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c0c2d80 0x7f801c15b730 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8014009d90 tx=0x7f801401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24373bc3bb8cf975 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:36.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f801be00640 1 -- 172.21.15.113:0/542996592 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140b4070 con 0x7f801c0c2d80 2024-03-04T07:45:36.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f801be00640 1 -- 172.21.15.113:0/542996592 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80140af600 con 0x7f801c0c2d80 2024-03-04T07:45:36.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f801be00640 1 -- 172.21.15.113:0/542996592 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140af9d0 con 0x7f801c0c2d80 2024-03-04T07:45:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8024000640 1 -- 172.21.15.113:0/542996592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c0c2d80 msgr2=0x7f801c15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8024000640 1 --2- 172.21.15.113:0/542996592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c0c2d80 0x7f801c15b730 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8014009d90 tx=0x7f801401f150 comp rx=0 tx=0).stop 2024-03-04T07:45:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8024000640 1 -- 172.21.15.113:0/542996592 shutdown_connections 2024-03-04T07:45:36.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8024000640 1 --2- 172.21.15.113:0/542996592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c0c2d80 0x7f801c15b730 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8024000640 1 --2- 172.21.15.113:0/542996592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 0x7f801c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8024000640 1 -- 172.21.15.113:0/542996592 >> 172.21.15.113:0/542996592 conn(0x7f801c06e820 msgr2=0x7f801c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8024000640 1 -- 172.21.15.113:0/542996592 shutdown_connections 2024-03-04T07:45:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8024000640 1 -- 172.21.15.113:0/542996592 wait complete. 2024-03-04T07:45:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8024000640 1 Processor -- start 2024-03-04T07:45:36.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.400+0000 7f8024000640 1 -- start start 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8024000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 0x7f801c0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8024000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c0c2d80 0x7f801c1e2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8024000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c14e4c0 0x7f801c1e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8024000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c071500 con 0x7f801c14e4c0 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8024000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c071220 con 0x7f801c0c2d80 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8024000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c14e8a0 con 0x7f801c0c22e0 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8022000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c14e4c0 0x7f801c1e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8022000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c14e4c0 0x7f801c1e28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37498/0 (socket says 172.21.15.113:37498) 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8022000640 1 -- 172.21.15.113:0/3534245003 learned_addr learned my addr 172.21.15.113:0/3534245003 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8021600640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 0x7f801c0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8020c00640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c0c2d80 0x7f801c1e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:36.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8022000640 1 -- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 msgr2=0x7f801c0cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:36.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8022000640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 0x7f801c0cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:36.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8022000640 1 -- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c0c2d80 msgr2=0x7f801c1e2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8022000640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c0c2d80 0x7f801c1e2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8022000640 1 -- 172.21.15.113:0/3534245003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c15ab20 con 0x7f801c14e4c0 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8021600640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 0x7f801c0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8022000640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c14e4c0 0x7f801c1e28a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f80140b2040 tx=0x7f801401bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8019600640 1 -- 172.21.15.113:0/3534245003 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140b4040 con 0x7f801c14e4c0 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8019600640 1 -- 172.21.15.113:0/3534245003 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80140c1070 con 0x7f801c14e4c0 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8024000640 1 -- 172.21.15.113:0/3534245003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c1487f0 con 0x7f801c14e4c0 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8019600640 1 -- 172.21.15.113:0/3534245003 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014009450 con 0x7f801c14e4c0 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8024000640 1 -- 172.21.15.113:0/3534245003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c148db0 con 0x7f801c14e4c0 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8024000640 1 -- 172.21.15.113:0/3534245003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801c075c00 con 0x7f801c14e4c0 2024-03-04T07:45:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.404+0000 7f8019600640 1 -- 172.21.15.113:0/3534245003 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80140ba6b0 con 0x7f801c14e4c0 2024-03-04T07:45:36.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.408+0000 7f8019600640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ffc066f60 0x7f7ffc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:36.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.408+0000 7f8019600640 1 -- 172.21.15.113:0/3534245003 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f8014123c90 con 0x7f801c14e4c0 2024-03-04T07:45:36.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.408+0000 7f8021600640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ffc066f60 0x7f7ffc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:36.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.408+0000 7f8021600640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ffc066f60 0x7f7ffc069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8010000f70 tx=0x7f8010009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:36.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.412+0000 7f8019600640 1 -- 172.21.15.113:0/3534245003 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8014018cb0 con 0x7f801c14e4c0 2024-03-04T07:45:36.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.720+0000 7f8024000640 1 -- 172.21.15.113:0/3534245003 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f801c149470 con 0x7f801c14e4c0 2024-03-04T07:45:36.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.720+0000 7f8019600640 1 -- 172.21.15.113:0/3534245003 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5813 (secure 0 0 0) 0x7f80140f4930 con 0x7f801c14e4c0 2024-03-04T07:45:36.723 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:36.723 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:36.002257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244]},"id":1}]} 2024-03-04T07:45:36.723 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 54 2024-03-04T07:45:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 -- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ffc066f60 msgr2=0x7f7ffc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ffc066f60 0x7f7ffc069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8010000f70 tx=0x7f8010009040 comp rx=0 tx=0).stop 2024-03-04T07:45:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 -- 172.21.15.113:0/3534245003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c14e4c0 msgr2=0x7f801c1e28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c14e4c0 0x7f801c1e28a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f80140b2040 tx=0x7f801401bef0 comp rx=0 tx=0).stop 2024-03-04T07:45:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 -- 172.21.15.113:0/3534245003 shutdown_connections 2024-03-04T07:45:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ffc066f60 0x7f7ffc069420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f801c14e4c0 0x7f801c1e28a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f801c0c2d80 0x7f801c1e2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 --2- 172.21.15.113:0/3534245003 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f801c0c22e0 0x7f801c0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 -- 172.21.15.113:0/3534245003 >> 172.21.15.113:0/3534245003 conn(0x7f801c06e820 msgr2=0x7f801c072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 -- 172.21.15.113:0/3534245003 shutdown_connections 2024-03-04T07:45:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:36.724+0000 7f8024000640 1 -- 172.21.15.113:0/3534245003 wait complete. 2024-03-04T07:45:36.744 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:45:38.746 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:38.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.884+0000 7f9a7d000640 1 Processor -- start 2024-03-04T07:45:38.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.884+0000 7f9a7d000640 1 -- start start 2024-03-04T07:45:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a7d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a781527f0 0x7f9a78070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a7d000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9a780c8200 0x7f9a78072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a7d000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a781591f0 con 0x7f9a78152bd0 2024-03-04T07:45:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a7d000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a78071df0 con 0x7f9a780c8200 2024-03-04T07:45:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a7d000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a781597b0 con 0x7f9a781527f0 2024-03-04T07:45:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a76800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9a78152bd0 0x7f9a781587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39156/0 (socket says 172.21.15.113:39156) 2024-03-04T07:45:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a76800640 1 -- 172.21.15.113:0/1840287430 learned_addr learned my addr 172.21.15.113:0/1840287430 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:45:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a75e00640 1 --2- 172.21.15.113:0/1840287430 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a781527f0 0x7f9a78070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a6fe00640 1 --2- 172.21.15.113:0/1840287430 >> v2:172.21.15.136:3300/0 conn(0x7f9a780c8200 0x7f9a78072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a75400640 1 -- 172.21.15.113:0/1840287430 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394599688 0 0) 0x7f9a781591f0 con 0x7f9a78152bd0 2024-03-04T07:45:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a75400640 1 -- 172.21.15.113:0/1840287430 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a5c003530 con 0x7f9a78152bd0 2024-03-04T07:45:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a6fe00640 1 -- 172.21.15.113:0/1840287430 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a781527f0 msgr2=0x7f9a78070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a6fe00640 1 --2- 172.21.15.113:0/1840287430 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a781527f0 0x7f9a78070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a6fe00640 1 -- 172.21.15.113:0/1840287430 >> v1:172.21.15.113:6789/0 conn(0x7f9a78152bd0 legacy=0x7f9a781587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.888+0000 7f9a6fe00640 1 -- 172.21.15.113:0/1840287430 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a781594d0 con 0x7f9a780c8200 2024-03-04T07:45:38.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.892+0000 7f9a6fe00640 1 --2- 172.21.15.113:0/1840287430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a780c8200 0x7f9a78072280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9a600097c0 tx=0x7f9a6001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c0e5145efbaaeaa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:38.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.892+0000 7f9a75400640 1 -- 172.21.15.113:0/1840287430 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600b1070 con 0x7f9a780c8200 2024-03-04T07:45:38.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.892+0000 7f9a75400640 1 -- 172.21.15.113:0/1840287430 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a600ac410 con 0x7f9a780c8200 2024-03-04T07:45:38.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.892+0000 7f9a75400640 1 -- 172.21.15.113:0/1840287430 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600ac7c0 con 0x7f9a780c8200 2024-03-04T07:45:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 -- 172.21.15.113:0/1840287430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a780c8200 msgr2=0x7f9a78072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 --2- 172.21.15.113:0/1840287430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a780c8200 0x7f9a78072280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9a600097c0 tx=0x7f9a6001c4a0 comp rx=0 tx=0).stop 2024-03-04T07:45:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 -- 172.21.15.113:0/1840287430 shutdown_connections 2024-03-04T07:45:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 --2- 172.21.15.113:0/1840287430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a780c8200 0x7f9a78072280 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 --2- 172.21.15.113:0/1840287430 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a781527f0 0x7f9a78070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 -- 172.21.15.113:0/1840287430 >> 172.21.15.113:0/1840287430 conn(0x7f9a7806e820 msgr2=0x7f9a7806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 -- 172.21.15.113:0/1840287430 shutdown_connections 2024-03-04T07:45:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 -- 172.21.15.113:0/1840287430 wait complete. 2024-03-04T07:45:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 Processor -- start 2024-03-04T07:45:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 -- start start 2024-03-04T07:45:38.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a780c8200 0x7f9a78153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:38.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a781527f0 0x7f9a780b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:38.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a78152bd0 0x7f9a7814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:38.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78165160 con 0x7f9a781527f0 2024-03-04T07:45:38.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780727c0 con 0x7f9a78152bd0 2024-03-04T07:45:38.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a7d000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a781652e0 con 0x7f9a780c8200 2024-03-04T07:45:38.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a6fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a781527f0 0x7f9a780b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:38.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a6fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a781527f0 0x7f9a780b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35776/0 (socket says 172.21.15.113:35776) 2024-03-04T07:45:38.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a6fe00640 1 -- 172.21.15.113:0/4220683728 learned_addr learned my addr 172.21.15.113:0/4220683728 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:38.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a75e00640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a780c8200 0x7f9a78153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:38.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a76800640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a78152bd0 0x7f9a7814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:38.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a6fe00640 1 -- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a780c8200 msgr2=0x7f9a78153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:38.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a6fe00640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a780c8200 0x7f9a78153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:38.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a6fe00640 1 -- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a78152bd0 msgr2=0x7f9a7814a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:38.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a6fe00640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a78152bd0 0x7f9a7814a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:38.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a6fe00640 1 -- 172.21.15.113:0/4220683728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780b1e10 con 0x7f9a781527f0 2024-03-04T07:45:38.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.896+0000 7f9a76800640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a78152bd0 0x7f9a7814a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:45:38.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.900+0000 7f9a6fe00640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a781527f0 0x7f9a780b0b90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9a600097c0 tx=0x7f9a60019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:38.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.900+0000 7f9a6e000640 1 -- 172.21.15.113:0/4220683728 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600b9020 con 0x7f9a781527f0 2024-03-04T07:45:38.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.900+0000 7f9a6e000640 1 -- 172.21.15.113:0/4220683728 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a600accc0 con 0x7f9a781527f0 2024-03-04T07:45:38.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.900+0000 7f9a6e000640 1 -- 172.21.15.113:0/4220683728 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600b1040 con 0x7f9a781527f0 2024-03-04T07:45:38.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.900+0000 7f9a7d000640 1 -- 172.21.15.113:0/4220683728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a78159a30 con 0x7f9a781527f0 2024-03-04T07:45:38.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.900+0000 7f9a7d000640 1 -- 172.21.15.113:0/4220683728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a780b0390 con 0x7f9a781527f0 2024-03-04T07:45:38.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.900+0000 7f9a7d000640 1 -- 172.21.15.113:0/4220683728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a78075c00 con 0x7f9a781527f0 2024-03-04T07:45:38.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.900+0000 7f9a6e000640 1 -- 172.21.15.113:0/4220683728 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a600bc020 con 0x7f9a781527f0 2024-03-04T07:45:38.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.904+0000 7f9a6e000640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 0x7f9a54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:38.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.904+0000 7f9a6e000640 1 -- 172.21.15.113:0/4220683728 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f9a6011ef50 con 0x7f9a781527f0 2024-03-04T07:45:38.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.904+0000 7f9a75e00640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 0x7f9a54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:38.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.904+0000 7f9a75e00640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 0x7f9a54069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009b60 tx=0x7f9a68009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:38.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:38.908+0000 7f9a6e000640 1 -- 172.21.15.113:0/4220683728 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9a600c3030 con 0x7f9a781527f0 2024-03-04T07:45:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.296+0000 7f9a7d000640 1 -- 172.21.15.113:0/4220683728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a780b2060 con 0x7f9a781527f0 2024-03-04T07:45:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.296+0000 7f9a6e000640 1 -- 172.21.15.113:0/4220683728 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5813 (secure 0 0 0) 0x7f9a60016090 con 0x7f9a781527f0 2024-03-04T07:45:39.297 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:36.002257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244]},"id":1}]} 2024-03-04T07:45:39.297 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 54 2024-03-04T07:45:39.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 -- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 msgr2=0x7f9a54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:39.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 0x7f9a54069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009b60 tx=0x7f9a68009190 comp rx=0 tx=0).stop 2024-03-04T07:45:39.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 -- 172.21.15.113:0/4220683728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a781527f0 msgr2=0x7f9a780b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:39.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a781527f0 0x7f9a780b0b90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9a600097c0 tx=0x7f9a60019240 comp rx=0 tx=0).stop 2024-03-04T07:45:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 -- 172.21.15.113:0/4220683728 shutdown_connections 2024-03-04T07:45:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9a78152bd0 0x7f9a7814a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a54066f60 0x7f9a54069420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a781527f0 0x7f9a780b0b90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 --2- 172.21.15.113:0/4220683728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9a780c8200 0x7f9a78153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 -- 172.21.15.113:0/4220683728 >> 172.21.15.113:0/4220683728 conn(0x7f9a7806e820 msgr2=0x7f9a780c1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 -- 172.21.15.113:0/4220683728 shutdown_connections 2024-03-04T07:45:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:39.300+0000 7f9a7d000640 1 -- 172.21.15.113:0/4220683728 wait complete. 2024-03-04T07:45:39.323 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:45:41.324 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:41.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e2a00640 1 Processor -- start 2024-03-04T07:45:41.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e2a00640 1 -- start start 2024-03-04T07:45:41.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4dc06d520 0x7ff4dc15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:41.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 0x7ff4dc0c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:41.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e2a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc0c7e10 con 0x7ff4dc154e20 2024-03-04T07:45:41.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e2a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4dc0c80f0 con 0x7ff4dc06d520 2024-03-04T07:45:41.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e2a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc0c83d0 con 0x7ff4dc0c18e0 2024-03-04T07:45:41.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e0c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4dc06d520 0x7ff4dc15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:41.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e0c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4dc06d520 0x7ff4dc15b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35780/0 (socket says 172.21.15.113:35780) 2024-03-04T07:45:41.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e0c00640 1 -- 172.21.15.113:0/2812347724 learned_addr learned my addr 172.21.15.113:0/2812347724 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:41.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e2000640 1 --2- 172.21.15.113:0/2812347724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 0x7ff4dc0c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:41.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e0c00640 1 -- 172.21.15.113:0/2812347724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 msgr2=0x7ff4dc0c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:41.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e0c00640 1 --2- 172.21.15.113:0/2812347724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 0x7ff4dc0c1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:41.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e0c00640 1 -- 172.21.15.113:0/2812347724 >> v1:172.21.15.136:6789/0 conn(0x7ff4dc154e20 legacy=0x7ff4dc06ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:41.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e0c00640 1 -- 172.21.15.113:0/2812347724 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc0c86b0 con 0x7ff4dc06d520 2024-03-04T07:45:41.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.464+0000 7ff4e2000640 1 --2- 172.21.15.113:0/2812347724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 0x7ff4dc0c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:45:41.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4e0c00640 1 --2- 172.21.15.113:0/2812347724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc06d520 0x7ff4dc15b3b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009870 tx=0x7ff4c801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a511056d6cd03dc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:41.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4dbe00640 1 -- 172.21.15.113:0/2812347724 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80b5070 con 0x7ff4dc06d520 2024-03-04T07:45:41.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4dbe00640 1 -- 172.21.15.113:0/2812347724 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c80a8a50 con 0x7ff4dc06d520 2024-03-04T07:45:41.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4dbe00640 1 -- 172.21.15.113:0/2812347724 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80a8e20 con 0x7ff4dc06d520 2024-03-04T07:45:41.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4e2a00640 1 -- 172.21.15.113:0/2812347724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc06d520 msgr2=0x7ff4dc15b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:41.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4e2a00640 1 --2- 172.21.15.113:0/2812347724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc06d520 0x7ff4dc15b3b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009870 tx=0x7ff4c801ed50 comp rx=0 tx=0).stop 2024-03-04T07:45:41.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4e2a00640 1 -- 172.21.15.113:0/2812347724 shutdown_connections 2024-03-04T07:45:41.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4e2a00640 1 --2- 172.21.15.113:0/2812347724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 0x7ff4dc0c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:41.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4e2a00640 1 --2- 172.21.15.113:0/2812347724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc06d520 0x7ff4dc15b3b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:41.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4e2a00640 1 -- 172.21.15.113:0/2812347724 >> 172.21.15.113:0/2812347724 conn(0x7ff4dc06e2a0 msgr2=0x7ff4dc06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:41.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4e2a00640 1 -- 172.21.15.113:0/2812347724 shutdown_connections 2024-03-04T07:45:41.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.468+0000 7ff4e2a00640 1 -- 172.21.15.113:0/2812347724 wait complete. 2024-03-04T07:45:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2a00640 1 Processor -- start 2024-03-04T07:45:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2a00640 1 -- start start 2024-03-04T07:45:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4dc06d520 0x7ff4dc0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 0x7ff4dc0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc154e20 0x7ff4dc1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc15ab60 con 0x7ff4dc154e20 2024-03-04T07:45:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc06c230 con 0x7ff4dc06d520 2024-03-04T07:45:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc0c68b0 con 0x7ff4dc0c18e0 2024-03-04T07:45:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc154e20 0x7ff4dc1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4dc06d520 0x7ff4dc0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 0x7ff4dc0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc154e20 0x7ff4dc1562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35790/0 (socket says 172.21.15.113:35790) 2024-03-04T07:45:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2000640 1 -- 172.21.15.113:0/3974288757 learned_addr learned my addr 172.21.15.113:0/3974288757 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4dc06d520 0x7ff4dc0cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:46222/0 (socket says 172.21.15.113:46222) 2024-03-04T07:45:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2000640 1 -- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 msgr2=0x7ff4dc0c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2000640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 0x7ff4dc0c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2000640 1 -- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4dc06d520 msgr2=0x7ff4dc0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2000640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4dc06d520 0x7ff4dc0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2000640 1 -- 172.21.15.113:0/3974288757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc1f3d90 con 0x7ff4dc154e20 2024-03-04T07:45:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2000640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc154e20 0x7ff4dc1562c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009fd0 tx=0x7ff4d400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4d9600640 1 -- 172.21.15.113:0/3974288757 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d401a070 con 0x7ff4dc154e20 2024-03-04T07:45:41.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4d9600640 1 -- 172.21.15.113:0/3974288757 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d4003bc0 con 0x7ff4dc154e20 2024-03-04T07:45:41.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4d9600640 1 -- 172.21.15.113:0/3974288757 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4004030 con 0x7ff4dc154e20 2024-03-04T07:45:41.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2a00640 1 -- 172.21.15.113:0/3974288757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc1f4070 con 0x7ff4dc154e20 2024-03-04T07:45:41.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.472+0000 7ff4e2a00640 1 -- 172.21.15.113:0/3974288757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc1f4630 con 0x7ff4dc154e20 2024-03-04T07:45:41.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.476+0000 7ff4e2a00640 1 -- 172.21.15.113:0/3974288757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4dc075a80 con 0x7ff4dc154e20 2024-03-04T07:45:41.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.476+0000 7ff4d9600640 1 -- 172.21.15.113:0/3974288757 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4d401d070 con 0x7ff4dc154e20 2024-03-04T07:45:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.476+0000 7ff4d9600640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff4bc066f60 0x7ff4bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.476+0000 7ff4d9600640 1 -- 172.21.15.113:0/3974288757 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff4d4085600 con 0x7ff4dc154e20 2024-03-04T07:45:41.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.476+0000 7ff4e1600640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff4bc066f60 0x7ff4bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:41.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.476+0000 7ff4e1600640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff4bc066f60 0x7ff4bc069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0001770 tx=0x7ff4d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:41.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.484+0000 7ff4d9600640 1 -- 172.21.15.113:0/3974288757 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff4d4010070 con 0x7ff4dc154e20 2024-03-04T07:45:41.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.684+0000 7ff4e2a00640 1 -- 172.21.15.113:0/3974288757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4dc0c1200 con 0x7ff4dc154e20 2024-03-04T07:45:41.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.684+0000 7ff4d9600640 1 -- 172.21.15.113:0/3974288757 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+5813 (secure 0 0 0) 0x7ff4d4055c50 con 0x7ff4dc154e20 2024-03-04T07:45:41.685 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:41.686 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:36.002257+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5529,5244]},"id":1}]} 2024-03-04T07:45:41.686 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 54 2024-03-04T07:45:41.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 -- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff4bc066f60 msgr2=0x7ff4bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:41.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff4bc066f60 0x7ff4bc069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0001770 tx=0x7ff4d0009040 comp rx=0 tx=0).stop 2024-03-04T07:45:41.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 -- 172.21.15.113:0/3974288757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc154e20 msgr2=0x7ff4dc1562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:41.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc154e20 0x7ff4dc1562c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009fd0 tx=0x7ff4d400d560 comp rx=0 tx=0).stop 2024-03-04T07:45:41.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 -- 172.21.15.113:0/3974288757 shutdown_connections 2024-03-04T07:45:41.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff4bc066f60 0x7ff4bc069420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:41.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4dc154e20 0x7ff4dc1562c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:41.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff4dc0c18e0 0x7ff4dc0c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:41.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 --2- 172.21.15.113:0/3974288757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4dc06d520 0x7ff4dc0cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:41.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 -- 172.21.15.113:0/3974288757 >> 172.21.15.113:0/3974288757 conn(0x7ff4dc06e2a0 msgr2=0x7ff4dc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:41.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 -- 172.21.15.113:0/3974288757 shutdown_connections 2024-03-04T07:45:41.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:41.688+0000 7ff4e2a00640 1 -- 172.21.15.113:0/3974288757 wait complete. 2024-03-04T07:45:41.710 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:45:43.711 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:43.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f043ac00640 1 Processor -- start 2024-03-04T07:45:43.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f043ac00640 1 -- start start 2024-03-04T07:45:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f043ac00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04340c8200 0x7f0434070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f043ac00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0434152bd0 0x7f04341587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f043ac00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04341591f0 con 0x7f04341527f0 2024-03-04T07:45:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f043ac00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0434071df0 con 0x7f04340c8200 2024-03-04T07:45:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f043ac00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04341597b0 con 0x7f0434152bd0 2024-03-04T07:45:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f042b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04340c8200 0x7f0434070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f042b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04340c8200 0x7f0434070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35792/0 (socket says 172.21.15.113:35792) 2024-03-04T07:45:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f042b400640 1 -- 172.21.15.113:0/527027868 learned_addr learned my addr 172.21.15.113:0/527027868 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f0433e00640 1 --1- 172.21.15.113:0/527027868 >> v1:172.21.15.113:6789/0 conn(0x7f04341527f0 0x7f0434072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39158/0 (socket says 172.21.15.113:39158) 2024-03-04T07:45:43.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f0438c00640 1 --2- 172.21.15.113:0/527027868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0434152bd0 0x7f04341587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:43.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f042b400640 1 -- 172.21.15.113:0/527027868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0434152bd0 msgr2=0x7f04341587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:43.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f042b400640 1 --2- 172.21.15.113:0/527027868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0434152bd0 0x7f04341587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:43.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f042b400640 1 -- 172.21.15.113:0/527027868 >> v1:172.21.15.113:6789/0 conn(0x7f04341527f0 legacy=0x7f0434072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:43.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f042b400640 1 -- 172.21.15.113:0/527027868 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04341594d0 con 0x7f04340c8200 2024-03-04T07:45:43.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.852+0000 7f0433400640 1 -- 172.21.15.113:0/527027868 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336943536 0 0) 0x7f04341591f0 con 0x7f04341527f0 2024-03-04T07:45:43.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f042b400640 1 --2- 172.21.15.113:0/527027868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04340c8200 0x7f0434070d40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f041c009870 tx=0x7f041c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138af054bf254d45 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:43.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f0433400640 1 -- 172.21.15.113:0/527027868 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c0a7070 con 0x7f04340c8200 2024-03-04T07:45:43.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f0433400640 1 -- 172.21.15.113:0/527027868 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041c0b0070 con 0x7f04340c8200 2024-03-04T07:45:43.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f0433400640 1 -- 172.21.15.113:0/527027868 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c0ab890 con 0x7f04340c8200 2024-03-04T07:45:43.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f043ac00640 1 -- 172.21.15.113:0/527027868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04340c8200 msgr2=0x7f0434070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:43.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f043ac00640 1 --2- 172.21.15.113:0/527027868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04340c8200 0x7f0434070d40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f041c009870 tx=0x7f041c01c560 comp rx=0 tx=0).stop 2024-03-04T07:45:43.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f043ac00640 1 -- 172.21.15.113:0/527027868 shutdown_connections 2024-03-04T07:45:43.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f043ac00640 1 --2- 172.21.15.113:0/527027868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0434152bd0 0x7f04341587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:43.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f043ac00640 1 --2- 172.21.15.113:0/527027868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04340c8200 0x7f0434070d40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:43.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f043ac00640 1 -- 172.21.15.113:0/527027868 >> 172.21.15.113:0/527027868 conn(0x7f043406e820 msgr2=0x7f043406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:43.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f043ac00640 1 -- 172.21.15.113:0/527027868 shutdown_connections 2024-03-04T07:45:43.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f043ac00640 1 -- 172.21.15.113:0/527027868 wait complete. 2024-03-04T07:45:43.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f043ac00640 1 Processor -- start 2024-03-04T07:45:43.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.860+0000 7f043ac00640 1 -- start start 2024-03-04T07:45:43.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f043ac00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04340c8200 0x7f0434141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:43.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f043ac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04341527f0 0x7f04341f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:43.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f043ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0434152bd0 0x7f0434140960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:43.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f043ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04341654f0 con 0x7f0434152bd0 2024-03-04T07:45:43.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f043ac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0434165370 con 0x7f04341527f0 2024-03-04T07:45:43.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f043ac00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0434165670 con 0x7f04340c8200 2024-03-04T07:45:43.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0438c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0434152bd0 0x7f0434140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:43.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0438c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0434152bd0 0x7f0434140960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35794/0 (socket says 172.21.15.113:35794) 2024-03-04T07:45:43.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0438c00640 1 -- 172.21.15.113:0/393058778 learned_addr learned my addr 172.21.15.113:0/393058778 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:43.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0433e00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04340c8200 0x7f0434141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:43.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f042b400640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04341527f0 0x7f04341f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:43.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0438c00640 1 -- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04340c8200 msgr2=0x7f0434141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:43.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0438c00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04340c8200 0x7f0434141920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:43.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0438c00640 1 -- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04341527f0 msgr2=0x7f04341f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:43.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0438c00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04341527f0 0x7f04341f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:43.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0438c00640 1 -- 172.21.15.113:0/393058778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0434151630 con 0x7f0434152bd0 2024-03-04T07:45:43.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0438c00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0434152bd0 0x7f0434140960 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f042c007e60 tx=0x7f042c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:43.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0430c00640 1 -- 172.21.15.113:0/393058778 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c014420 con 0x7f0434152bd0 2024-03-04T07:45:43.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0430c00640 1 -- 172.21.15.113:0/393058778 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c019070 con 0x7f0434152bd0 2024-03-04T07:45:43.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0430c00640 1 -- 172.21.15.113:0/393058778 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c014820 con 0x7f0434152bd0 2024-03-04T07:45:43.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f043ac00640 1 -- 172.21.15.113:0/393058778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04341559f0 con 0x7f0434152bd0 2024-03-04T07:45:43.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f043ac00640 1 -- 172.21.15.113:0/393058778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04341f5a50 con 0x7f0434152bd0 2024-03-04T07:45:43.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f043ac00640 1 -- 172.21.15.113:0/393058778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0434075c00 con 0x7f0434152bd0 2024-03-04T07:45:43.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.864+0000 7f0430c00640 1 -- 172.21.15.113:0/393058778 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f042c0149c0 con 0x7f0434152bd0 2024-03-04T07:45:43.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.868+0000 7f0430c00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0410066f60 0x7f0410069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:43.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.868+0000 7f0430c00640 1 -- 172.21.15.113:0/393058778 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f042c085a40 con 0x7f0434152bd0 2024-03-04T07:45:43.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.868+0000 7f0433e00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0410066f60 0x7f0410069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:43.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.868+0000 7f0433e00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0410066f60 0x7f0410069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0424007fd0 tx=0x7f0424007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:43.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:43.872+0000 7f0430c00640 1 -- 172.21.15.113:0/393058778 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f042c056130 con 0x7f0434152bd0 2024-03-04T07:45:44.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.064+0000 7f043ac00640 1 -- 172.21.15.113:0/393058778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04341572b0 con 0x7f0434152bd0 2024-03-04T07:45:44.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.064+0000 7f0430c00640 1 -- 172.21.15.113:0/393058778 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f042c05a1d0 con 0x7f0434152bd0 2024-03-04T07:45:44.066 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:44.066 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5529,5631]},"id":1}]} 2024-03-04T07:45:44.066 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 55 2024-03-04T07:45:44.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 -- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0410066f60 msgr2=0x7f0410069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:44.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0410066f60 0x7f0410069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0424007fd0 tx=0x7f0424007d60 comp rx=0 tx=0).stop 2024-03-04T07:45:44.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 -- 172.21.15.113:0/393058778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0434152bd0 msgr2=0x7f0434140960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:44.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0434152bd0 0x7f0434140960 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f042c007e60 tx=0x7f042c00e4b0 comp rx=0 tx=0).stop 2024-03-04T07:45:44.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 -- 172.21.15.113:0/393058778 shutdown_connections 2024-03-04T07:45:44.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0410066f60 0x7f0410069420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:44.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0434152bd0 0x7f0434140960 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04341527f0 0x7f04341f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 --2- 172.21.15.113:0/393058778 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04340c8200 0x7f0434141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 -- 172.21.15.113:0/393058778 >> 172.21.15.113:0/393058778 conn(0x7f043406e820 msgr2=0x7f0434070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 -- 172.21.15.113:0/393058778 shutdown_connections 2024-03-04T07:45:44.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:44.068+0000 7f043ac00640 1 -- 172.21.15.113:0/393058778 wait complete. 2024-03-04T07:45:44.087 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=2, replacing gid=5099 2024-03-04T07:45:44.087 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.b 2024-03-04T07:45:45.088 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2024-03-04T07:45:45.088 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-04T07:45:45.088 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-04T07:45:45.091 INFO:tasks.ceph.mds.b:Started 2024-03-04T07:45:45.091 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:45.167 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2024-03-04T07:45:45.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.248+0000 7f8d38200640 1 Processor -- start 2024-03-04T07:45:45.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d38200640 1 -- start start 2024-03-04T07:45:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d38200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8d300c8200 0x7f8d30070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d38200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d30152bd0 0x7f8d301587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:45.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d38200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d301591f0 con 0x7f8d301527f0 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d38200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d30071df0 con 0x7f8d300c8200 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d38200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d301597b0 con 0x7f8d30152bd0 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d35800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8d301527f0 0x7f8d30072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39174/0 (socket says 172.21.15.113:39174) 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d35800640 1 -- 172.21.15.113:0/3813319372 learned_addr learned my addr 172.21.15.113:0/3813319372 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d34e00640 1 --2- 172.21.15.113:0/3813319372 >> v2:172.21.15.136:3300/0 conn(0x7f8d300c8200 0x7f8d30070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d36200640 1 --2- 172.21.15.113:0/3813319372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d30152bd0 0x7f8d301587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d2fe00640 1 -- 172.21.15.113:0/3813319372 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99076806 0 0) 0x7f8d301591f0 con 0x7f8d301527f0 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d2fe00640 1 -- 172.21.15.113:0/3813319372 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d18003530 con 0x7f8d301527f0 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d34e00640 1 -- 172.21.15.113:0/3813319372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d30152bd0 msgr2=0x7f8d301587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d34e00640 1 --2- 172.21.15.113:0/3813319372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d30152bd0 0x7f8d301587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d34e00640 1 -- 172.21.15.113:0/3813319372 >> v1:172.21.15.113:6789/0 conn(0x7f8d301527f0 legacy=0x7f8d30072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:45.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.252+0000 7f8d34e00640 1 -- 172.21.15.113:0/3813319372 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d301594d0 con 0x7f8d300c8200 2024-03-04T07:45:45.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.256+0000 7f8d34e00640 1 --2- 172.21.15.113:0/3813319372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d300c8200 0x7f8d30070d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0097c0 tx=0x7f8d1c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c96676fe2685be7f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:45.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.256+0000 7f8d2fe00640 1 -- 172.21.15.113:0/3813319372 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0b1070 con 0x7f8d300c8200 2024-03-04T07:45:45.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.256+0000 7f8d2fe00640 1 -- 172.21.15.113:0/3813319372 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c0ac410 con 0x7f8d300c8200 2024-03-04T07:45:45.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.256+0000 7f8d2fe00640 1 -- 172.21.15.113:0/3813319372 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0ac7c0 con 0x7f8d300c8200 2024-03-04T07:45:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.260+0000 7f8d38200640 1 -- 172.21.15.113:0/3813319372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d300c8200 msgr2=0x7f8d30070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.260+0000 7f8d38200640 1 --2- 172.21.15.113:0/3813319372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d300c8200 0x7f8d30070d40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0097c0 tx=0x7f8d1c01c4a0 comp rx=0 tx=0).stop 2024-03-04T07:45:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.260+0000 7f8d38200640 1 -- 172.21.15.113:0/3813319372 shutdown_connections 2024-03-04T07:45:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.260+0000 7f8d38200640 1 --2- 172.21.15.113:0/3813319372 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d30152bd0 0x7f8d301587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.260+0000 7f8d38200640 1 --2- 172.21.15.113:0/3813319372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d300c8200 0x7f8d30070d40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.260+0000 7f8d38200640 1 -- 172.21.15.113:0/3813319372 >> 172.21.15.113:0/3813319372 conn(0x7f8d3006e820 msgr2=0x7f8d3006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.260+0000 7f8d38200640 1 -- 172.21.15.113:0/3813319372 shutdown_connections 2024-03-04T07:45:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.260+0000 7f8d38200640 1 -- 172.21.15.113:0/3813319372 wait complete. 2024-03-04T07:45:45.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.260+0000 7f8d38200640 1 Processor -- start 2024-03-04T07:45:45.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.260+0000 7f8d38200640 1 -- start start 2024-03-04T07:45:45.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d38200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300c8200 0x7f8d300c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:45.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d38200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d301527f0 0x7f8d301f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:45.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d38200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d30152bd0 0x7f8d301612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d38200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d301654f0 con 0x7f8d300c8200 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d38200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30165370 con 0x7f8d301527f0 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d38200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30165670 con 0x7f8d30152bd0 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d35800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300c8200 0x7f8d300c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d35800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300c8200 0x7f8d300c1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35804/0 (socket says 172.21.15.113:35804) 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d35800640 1 -- 172.21.15.113:0/2575347179 learned_addr learned my addr 172.21.15.113:0/2575347179 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d34e00640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d301527f0 0x7f8d301f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d36200640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d30152bd0 0x7f8d301612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d35800640 1 -- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d30152bd0 msgr2=0x7f8d301612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d35800640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d30152bd0 0x7f8d301612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d35800640 1 -- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d301527f0 msgr2=0x7f8d301f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:45.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d35800640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d301527f0 0x7f8d301f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:45.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d35800640 1 -- 172.21.15.113:0/2575347179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d301f4640 con 0x7f8d300c8200 2024-03-04T07:45:45.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d35800640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300c8200 0x7f8d300c1e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c450 tx=0x7f8d2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:45.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d2d600640 1 -- 172.21.15.113:0/2575347179 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24017070 con 0x7f8d300c8200 2024-03-04T07:45:45.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d2d600640 1 -- 172.21.15.113:0/2575347179 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d24005cf0 con 0x7f8d300c8200 2024-03-04T07:45:45.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d38200640 1 -- 172.21.15.113:0/2575347179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d30161820 con 0x7f8d300c8200 2024-03-04T07:45:45.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d2d600640 1 -- 172.21.15.113:0/2575347179 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24007290 con 0x7f8d300c8200 2024-03-04T07:45:45.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d38200640 1 -- 172.21.15.113:0/2575347179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d30162b80 con 0x7f8d300c8200 2024-03-04T07:45:45.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.264+0000 7f8d38200640 1 -- 172.21.15.113:0/2575347179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d30075c00 con 0x7f8d300c8200 2024-03-04T07:45:45.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.268+0000 7f8d2d600640 1 -- 172.21.15.113:0/2575347179 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d2401a070 con 0x7f8d300c8200 2024-03-04T07:45:45.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.268+0000 7f8d2d600640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8d10066f60 0x7f8d10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:45.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.268+0000 7f8d2d600640 1 -- 172.21.15.113:0/2575347179 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f8d24084800 con 0x7f8d300c8200 2024-03-04T07:45:45.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.268+0000 7f8d34e00640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8d10066f60 0x7f8d10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:45.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.268+0000 7f8d34e00640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8d10066f60 0x7f8d10069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0ae000 tx=0x7f8d1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:45.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.272+0000 7f8d2d600640 1 -- 172.21.15.113:0/2575347179 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8d24021080 con 0x7f8d300c8200 2024-03-04T07:45:45.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 -- 172.21.15.113:0/2575347179 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d300c79d0 con 0x7f8d300c8200 2024-03-04T07:45:45.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d2d600640 1 -- 172.21.15.113:0/2575347179 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f8d24055060 con 0x7f8d300c8200 2024-03-04T07:45:45.469 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:45.469 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244,5529,5631]},"id":1}]} 2024-03-04T07:45:45.469 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 55 2024-03-04T07:45:45.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 -- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8d10066f60 msgr2=0x7f8d10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:45.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8d10066f60 0x7f8d10069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0ae000 tx=0x7f8d1c008040 comp rx=0 tx=0).stop 2024-03-04T07:45:45.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 -- 172.21.15.113:0/2575347179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300c8200 msgr2=0x7f8d300c1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:45.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300c8200 0x7f8d300c1e10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c450 tx=0x7f8d2400c910 comp rx=0 tx=0).stop 2024-03-04T07:45:45.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 -- 172.21.15.113:0/2575347179 shutdown_connections 2024-03-04T07:45:45.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8d30152bd0 0x7f8d301612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:45.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d301527f0 0x7f8d301f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:45.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8d10066f60 0x7f8d10069420 secure :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c0ae000 tx=0x7f8d1c008040 comp rx=0 tx=0).stop 2024-03-04T07:45:45.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 --2- 172.21.15.113:0/2575347179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d300c8200 0x7f8d300c1e10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:45.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 -- 172.21.15.113:0/2575347179 >> 172.21.15.113:0/2575347179 conn(0x7f8d3006e820 msgr2=0x7f8d30151fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:45.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 -- 172.21.15.113:0/2575347179 shutdown_connections 2024-03-04T07:45:45.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:45.472+0000 7f8d38200640 1 -- 172.21.15.113:0/2575347179 wait complete. 2024-03-04T07:45:45.489 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2024-03-04T07:45:47.491 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:45:47.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.636+0000 7f8467400640 1 Processor -- start 2024-03-04T07:45:47.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.636+0000 7f8467400640 1 -- start start 2024-03-04T07:45:47.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8467400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8460152a50 0x7f84600ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:47.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8467400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f846006ff20 0x7f84600ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:47.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8467400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8460152e30 con 0x7f84600c8b90 2024-03-04T07:45:47.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8467400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84600c31b0 con 0x7f846006ff20 2024-03-04T07:45:47.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8467400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846006f840 con 0x7f8460152a50 2024-03-04T07:45:47.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8464a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8460152a50 0x7f84600ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:47.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8457e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f846006ff20 0x7f84600ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:47.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8464a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8460152a50 0x7f84600ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:41670/0 (socket says 172.21.15.113:41670) 2024-03-04T07:45:47.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8464a00640 1 -- 172.21.15.113:0/260475982 learned_addr learned my addr 172.21.15.113:0/260475982 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:47.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8464a00640 1 -- 172.21.15.113:0/260475982 >> v2:172.21.15.136:3300/0 conn(0x7f846006ff20 msgr2=0x7f84600ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:45:47.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8464a00640 1 --2- 172.21.15.113:0/260475982 >> v2:172.21.15.136:3300/0 conn(0x7f846006ff20 0x7f84600ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:47.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8464a00640 1 -- 172.21.15.113:0/260475982 >> v1:172.21.15.136:6789/0 conn(0x7f84600c8b90 legacy=0x7f846015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:47.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.640+0000 7f8464a00640 1 -- 172.21.15.113:0/260475982 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846006f280 con 0x7f8460152a50 2024-03-04T07:45:47.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f8464a00640 1 --2- 172.21.15.113:0/260475982 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8460152a50 0x7f84600ce120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8450009870 tx=0x7f845001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b871ef519e7f7f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:47.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f845fe00640 1 -- 172.21.15.113:0/260475982 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500a9620 con 0x7f8460152a50 2024-03-04T07:45:47.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f845fe00640 1 -- 172.21.15.113:0/260475982 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84500a97c0 con 0x7f8460152a50 2024-03-04T07:45:47.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f845fe00640 1 -- 172.21.15.113:0/260475982 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500a9b90 con 0x7f8460152a50 2024-03-04T07:45:47.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f8467400640 1 -- 172.21.15.113:0/260475982 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8460152a50 msgr2=0x7f84600ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:47.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f8467400640 1 --2- 172.21.15.113:0/260475982 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8460152a50 0x7f84600ce120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8450009870 tx=0x7f845001c560 comp rx=0 tx=0).stop 2024-03-04T07:45:47.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f8467400640 1 -- 172.21.15.113:0/260475982 shutdown_connections 2024-03-04T07:45:47.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f8467400640 1 --2- 172.21.15.113:0/260475982 >> v2:172.21.15.136:3300/0 conn(0x7f846006ff20 0x7f84600ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:47.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f8467400640 1 --2- 172.21.15.113:0/260475982 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8460152a50 0x7f84600ce120 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:47.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f8467400640 1 -- 172.21.15.113:0/260475982 >> 172.21.15.113:0/260475982 conn(0x7f84600734d0 msgr2=0x7f8460066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:47.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f8467400640 1 -- 172.21.15.113:0/260475982 shutdown_connections 2024-03-04T07:45:47.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.644+0000 7f8467400640 1 -- 172.21.15.113:0/260475982 wait complete. 2024-03-04T07:45:47.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8467400640 1 Processor -- start 2024-03-04T07:45:47.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8467400640 1 -- start start 2024-03-04T07:45:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8467400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006ff20 0x7f84600c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8467400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84600c8b90 0x7f846014bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8467400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8460152a50 0x7f846014c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8467400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8460165a00 con 0x7f846006ff20 2024-03-04T07:45:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8467400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8460165880 con 0x7f8460152a50 2024-03-04T07:45:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8467400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8460165b80 con 0x7f84600c8b90 2024-03-04T07:45:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8464a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006ff20 0x7f84600c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8464a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006ff20 0x7f84600c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60886/0 (socket says 172.21.15.113:60886) 2024-03-04T07:45:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8464a00640 1 -- 172.21.15.113:0/1604437553 learned_addr learned my addr 172.21.15.113:0/1604437553 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:45:47.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8465400640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8460152a50 0x7f846014c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:47.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8457e00640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84600c8b90 0x7f846014bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:47.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8464a00640 1 -- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84600c8b90 msgr2=0x7f846014bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:47.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8464a00640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84600c8b90 0x7f846014bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:47.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8464a00640 1 -- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8460152a50 msgr2=0x7f846014c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:47.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8464a00640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8460152a50 0x7f846014c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:47.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8464a00640 1 -- 172.21.15.113:0/1604437553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84601f7800 con 0x7f846006ff20 2024-03-04T07:45:47.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8457e00640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84600c8b90 0x7f846014bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:45:47.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8465400640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8460152a50 0x7f846014c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:45:47.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f8464a00640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006ff20 0x7f84600c7560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f845001a190 tx=0x7f8450019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:47.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f845d600640 1 -- 172.21.15.113:0/1604437553 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500bf070 con 0x7f846006ff20 2024-03-04T07:45:47.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.648+0000 7f845d600640 1 -- 172.21.15.113:0/1604437553 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84500a9c00 con 0x7f846006ff20 2024-03-04T07:45:47.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.652+0000 7f8467400640 1 -- 172.21.15.113:0/1604437553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84601f7a70 con 0x7f846006ff20 2024-03-04T07:45:47.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.652+0000 7f845d600640 1 -- 172.21.15.113:0/1604437553 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500ba410 con 0x7f846006ff20 2024-03-04T07:45:47.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.652+0000 7f8467400640 1 -- 172.21.15.113:0/1604437553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84600c5a10 con 0x7f846006ff20 2024-03-04T07:45:47.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.652+0000 7f8467400640 1 -- 172.21.15.113:0/1604437553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8460075c00 con 0x7f846006ff20 2024-03-04T07:45:47.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.652+0000 7f845d600640 1 -- 172.21.15.113:0/1604437553 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84500c2070 con 0x7f846006ff20 2024-03-04T07:45:47.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.652+0000 7f845d600640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f843c066f90 0x7f843c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:45:47.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.652+0000 7f845d600640 1 -- 172.21.15.113:0/1604437553 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f845011ee80 con 0x7f846006ff20 2024-03-04T07:45:47.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.652+0000 7f8457e00640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f843c066f90 0x7f843c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:45:47.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.656+0000 7f8457e00640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f843c066f90 0x7f843c069450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84600c7f40 tx=0x7f8448008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:45:47.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.656+0000 7f845d600640 1 -- 172.21.15.113:0/1604437553 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f84500ad020 con 0x7f846006ff20 2024-03-04T07:45:47.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 -- 172.21.15.113:0/1604437553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f846006ed10 con 0x7f846006ff20 2024-03-04T07:45:47.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f845d600640 1 -- 172.21.15.113:0/1604437553 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f8450016150 con 0x7f846006ff20 2024-03-04T07:45:47.899 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:45:47.900 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5417,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5631,5529,5244]},"id":1}]} 2024-03-04T07:45:47.901 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-03-04T07:45:47.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 -- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f843c066f90 msgr2=0x7f843c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:47.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f843c066f90 0x7f843c069450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84600c7f40 tx=0x7f8448008040 comp rx=0 tx=0).stop 2024-03-04T07:45:47.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 -- 172.21.15.113:0/1604437553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006ff20 msgr2=0x7f84600c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:45:47.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006ff20 0x7f84600c7560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f845001a190 tx=0x7f8450019300 comp rx=0 tx=0).stop 2024-03-04T07:45:47.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 -- 172.21.15.113:0/1604437553 shutdown_connections 2024-03-04T07:45:47.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8460152a50 0x7f846014c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:47.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84600c8b90 0x7f846014bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:47.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f843c066f90 0x7f843c069450 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:47.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 --2- 172.21.15.113:0/1604437553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006ff20 0x7f84600c7560 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:45:47.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 -- 172.21.15.113:0/1604437553 >> 172.21.15.113:0/1604437553 conn(0x7f84600734d0 msgr2=0x7f84600b2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:45:47.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 -- 172.21.15.113:0/1604437553 shutdown_connections 2024-03-04T07:45:47.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:45:47.900+0000 7f8467400640 1 -- 172.21.15.113:0/1604437553 wait complete. 2024-03-04T07:45:47.926 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2024-03-04T07:45:47.927 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 97 secs before thrashing 2024-03-04T07:45:57.711 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:45:57.714 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:46:24.887 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 1% (145/12306) Updating files: 2% (247/12306) Updating files: 2% (307/12306) Updating files: 3% (370/12306) Updating files: 4% (493/12306) Updating files: 5% (616/12306) Updating files: 5% (727/12306) Updating files: 6% (739/12306) Updating files: 6% (841/12306) Updating files: 7% (862/12306) Updating files: 7% (951/12306) Updating files: 8% (985/12306) Updating files: 8% (1093/12306) Updating files: 9% (1108/12306) Updating files: 9% (1155/12306) Updating files: 10% (1231/12306) Updating files: 10% (1352/12306) Updating files: 11% (1354/12306) Updating files: 12% (1477/12306) Updating files: 12% (1588/12306) Updating files: 13% (1600/12306) Updating files: 13% (1635/12306) Updating files: 14% (1723/12306) Updating files: 14% (1757/12306) Updating files: 15% (1846/12306) Updating files: 15% (1891/12306) Updating files: 16% (1969/12306) Updating files: 16% (2064/12306) Updating files: 17% (2093/12306) Updating files: 17% (2105/12306) Updating files: 18% (2216/12306) Updating files: 18% (2293/12306) Updating files: 19% (2339/12306) Updating files: 19% (2440/12306) Updating files: 20% (2462/12306) Updating files: 20% (2564/12306) Updating files: 21% (2585/12306) Updating files: 21% (2602/12306) Updating files: 21% (2604/12306) Updating files: 22% (2708/12306) Updating files: 22% (2750/12306) Updating files: 23% (2831/12306) Updating files: 23% (2853/12306) Updating files: 24% (2954/12306) Updating files: 24% (2980/12306) Updating files: 24% (2981/12306) Updating files: 24% (3009/12306) Updating files: 25% (3077/12306) Updating files: 26% (3200/12306) Updating files: 26% (3215/12306) Updating files: 26% (3303/12306) Updating files: 27% (3323/12306) Updating files: 27% (3327/12306) Updating files: 28% (3446/12306) Updating files: 29% (3569/12306) Updating files: 29% (3598/12306) Updating files: 29% (3662/12306) Updating files: 30% (3692/12306) Updating files: 31% (3815/12306) Updating files: 31% (3842/12306) Updating files: 31% (3933/12306) Updating files: 32% (3938/12306) Updating files: 33% (4061/12306) Updating files: 33% (4170/12306) Updating files: 34% (4185/12306) Updating files: 34% (4276/12306) Updating files: 35% (4308/12306) Updating files: 35% (4343/12306) Updating files: 36% (4431/12306) Updating files: 36% (4500/12306) Updating files: 37% (4554/12306) Updating files: 37% (4595/12306) Updating files: 38% (4677/12306) Updating files: 38% (4757/12306) Updating files: 39% (4800/12306) Updating files: 39% (4826/12306) Updating files: 40% (4923/12306) Updating files: 40% (4939/12306) Updating files: 40% (5033/12306) Updating files: 41% (5046/12306) Updating files: 42% (5169/12306) Updating files: 42% (5175/12306) Updating files: 42% (5284/12306) Updating files: 43% (5292/12306) Updating files: 44% (5415/12306) Updating files: 44% (5506/12306) Updating files: 45% (5538/12306) Updating files: 45% (5612/12306) Updating files: 46% (5661/12306) Updating files: 46% (5729/12306) Updating files: 47% (5784/12306) Updating files: 48% (5907/12306) Updating files: 48% (5959/12306) Updating files: 49% (6030/12306) Updating files: 49% (6060/12306) Updating files: 49% (6098/12306) Updating files: 50% (6153/12306) Updating files: 50% (6230/12306) Updating files: 51% (6277/12306) Updating files: 52% (6400/12306) Updating files: 52% (6465/12306) Updating files: 53% (6523/12306) Updating files: 54% (6646/12306) Updating files: 54% (6714/12306) Updating files: 55% (6769/12306) Updating files: 55% (6795/12306) Updating files: 56% (6892/12306) Updating files: 57% (7015/12306) Updating files: 57% (7124/12306) Updating files: 58% (7138/12306) Updating files: 58% (7238/12306) Updating files: 59% (7261/12306) Updating files: 60% (7384/12306) Updating files: 61% (7507/12306) Updating files: 61% (7545/12306) Updating files: 62% (7630/12306) Updating files: 62% (7674/12306) Updating files: 63% (7753/12306) Updating files: 63% (7779/12306) Updating files: 63% (7806/12306) Updating files: 64% (7876/12306) Updating files: 64% (7926/12306) Updating files: 64% (7986/12306) Updating files: 65% (7999/12306) Updating files: 66% (8122/12306) Updating files: 66% (8149/12306) Updating files: 66% (8173/12306) Updating files: 67% (8246/12306) Updating files: 67% (8321/12306) Updating files: 68% (8369/12306) Updating files: 68% (8376/12306) Updating files: 68% (8384/12306) Updating files: 69% (8492/12306) Updating files: 70% (8615/12306) Updating files: 71% (8738/12306) Updating files: 71% (8743/12306) Updating files: 71% (8751/12306) Updating files: 72% (8861/12306) Updating files: 72% (8957/12306) Updating files: 73% (8984/12306) Updating files: 74% (9107/12306) Updating files: 74% (9161/12306) Updating files: 75% (9230/12306) Updating files: 75% (9240/12306) Updating files: 75% (9288/12306) Updating files: 76% (9353/12306) Updating files: 76% (9427/12306) Updating files: 77% (9476/12306) Updating files: 77% (9589/12306) Updating files: 78% (9599/12306) Updating files: 78% (9657/12306) Updating files: 79% (9722/12306) Updating files: 79% (9804/12306) Updating files: 79% (9807/12306) Updating files: 80% (9845/12306) Updating files: 80% (9867/12306) Updating files: 81% (9968/12306) Updating files: 82% (10091/12306) Updating files: 82% (10096/12306) Updating files: 83% (10214/12306) Updating files: 83% (10251/12306) Updating files: 84% (10338/12306) Updating files: 85% (10461/12306) Updating files: 85% (10553/12306) Updating files: 86% (10584/12306) Updating files: 86% (10614/12306) Updating files: 86% (10657/12306) Updating files: 87% (10707/12306) Updating files: 88% (10830/12306) Updating files: 88% (10836/12306) Updating files: 88% (10891/12306) Updating files: 89% (10953/12306) Updating files: 90% (11076/12306) Updating files: 90% (11078/12306) Updating files: 91% (11199/12306) Updating files: 91% (11232/12306) Updating files: 92% (11322/12306) Updating files: 92% (11351/12306) Updating files: 93% (11445/12306) Updating files: 94% (11568/12306) Updating files: 94% (11598/12306) Updating files: 94% (11628/12306) Updating files: 95% (11691/12306) Updating files: 96% (11814/12306) Updating files: 96% (11832/12306) Updating files: 96% (11899/12306) Updating files: 97% (11937/12306) Updating files: 97% (11961/12306) Updating files: 98% (12060/12306) Updating files: 99% (12183/12306) Updating files: 99% (12242/12306) Updating files: 100% (12306/12306) Updating files: 100% (12306/12306), done. 2024-03-04T07:46:25.224 INFO:tasks.workunit.client.0.smithi113.stderr:+ trap - EXIT 2024-03-04T07:46:25.224 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ceph 2024-03-04T07:46:25.224 INFO:tasks.workunit.client.0.smithi113.stderr:+ seq 1 90 2024-03-04T07:46:25.225 INFO:tasks.workunit.client.0.smithi113.stderr:+ versions=1 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:2 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:3 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:4 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:5 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:6 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:7 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:8 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:9 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:10 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:11 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:12 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:13 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:14 2024-03-04T07:46:25.226 INFO:tasks.workunit.client.0.smithi113.stderr:15 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:16 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:17 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:18 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:19 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:20 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:21 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:22 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:23 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:24 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:25 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:26 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:27 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:28 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:29 2024-03-04T07:46:25.227 INFO:tasks.workunit.client.0.smithi113.stderr:30 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:31 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:32 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:33 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:34 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:35 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:36 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:37 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:38 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:39 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:40 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:41 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:42 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:43 2024-03-04T07:46:25.228 INFO:tasks.workunit.client.0.smithi113.stderr:44 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:45 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:46 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:47 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:48 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:49 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:50 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:51 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:52 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:53 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:54 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:55 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:56 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:57 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:58 2024-03-04T07:46:25.229 INFO:tasks.workunit.client.0.smithi113.stderr:59 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:60 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:61 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:62 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:63 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:64 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:65 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:66 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:67 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:68 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:69 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:70 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:71 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:72 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:73 2024-03-04T07:46:25.230 INFO:tasks.workunit.client.0.smithi113.stderr:74 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:75 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:76 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:77 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:78 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:79 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:80 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:81 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:82 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:83 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:84 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:85 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:86 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:87 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:88 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:89 2024-03-04T07:46:25.231 INFO:tasks.workunit.client.0.smithi113.stderr:90 2024-03-04T07:46:25.232 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 1 -eq 48 ] 2024-03-04T07:46:25.232 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.1 2024-03-04T07:46:25.232 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.1 2024-03-04T07:46:25.232 INFO:tasks.workunit.client.0.smithi113.stdout:v0.1 2024-03-04T07:46:25.232 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.1 2024-03-04T07:46:27.768 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:46:27.772 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:46:58.602 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:46:58.607 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:47:24.928 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:25.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.099+0000 7f38cfe00640 1 Processor -- start 2024-03-04T07:47:25.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.099+0000 7f38cfe00640 1 -- start start 2024-03-04T07:47:25.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.099+0000 7f38cfe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d00cb540 0x7f38d015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.099+0000 7f38cfe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f38d006faa0 0x7f38d00c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38cfe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d00c4ce0 con 0x7f38d0152950 2024-03-04T07:47:25.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38cfe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d00c8370 con 0x7f38d006faa0 2024-03-04T07:47:25.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38cfe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00ca0a0 con 0x7f38d00cb540 2024-03-04T07:47:25.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38cea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f38d0152950 0x7f38d006ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40054/0 (socket says 172.21.15.113:40054) 2024-03-04T07:47:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38cea00640 1 -- 172.21.15.113:0/4097578950 learned_addr learned my addr 172.21.15.113:0/4097578950 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:47:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38cf400640 1 --2- 172.21.15.113:0/4097578950 >> v2:172.21.15.113:3300/0 conn(0x7f38d006faa0 0x7f38d00c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38c6000640 1 --2- 172.21.15.113:0/4097578950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d00cb540 0x7f38d015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38ce000640 1 -- 172.21.15.113:0/4097578950 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934619710 0 0) 0x7f38d00c4ce0 con 0x7f38d0152950 2024-03-04T07:47:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38ce000640 1 -- 172.21.15.113:0/4097578950 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b4003530 con 0x7f38d0152950 2024-03-04T07:47:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38cf400640 1 -- 172.21.15.113:0/4097578950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d00cb540 msgr2=0x7f38d015bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38cf400640 1 --2- 172.21.15.113:0/4097578950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d00cb540 0x7f38d015bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38cf400640 1 -- 172.21.15.113:0/4097578950 >> v1:172.21.15.113:6789/0 conn(0x7f38d0152950 legacy=0x7f38d006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.103+0000 7f38cf400640 1 -- 172.21.15.113:0/4097578950 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00c4f80 con 0x7f38d006faa0 2024-03-04T07:47:25.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38cf400640 1 --2- 172.21.15.113:0/4097578950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d006faa0 0x7f38d00c30b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009d90 tx=0x7f38c801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6179d182836e0374 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:25.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38ce000640 1 -- 172.21.15.113:0/4097578950 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c80b4070 con 0x7f38d006faa0 2024-03-04T07:47:25.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38ce000640 1 -- 172.21.15.113:0/4097578950 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c80af600 con 0x7f38d006faa0 2024-03-04T07:47:25.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38ce000640 1 -- 172.21.15.113:0/4097578950 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c80af9d0 con 0x7f38d006faa0 2024-03-04T07:47:25.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38cfe00640 1 -- 172.21.15.113:0/4097578950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d006faa0 msgr2=0x7f38d00c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38cfe00640 1 --2- 172.21.15.113:0/4097578950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d006faa0 0x7f38d00c30b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009d90 tx=0x7f38c801f150 comp rx=0 tx=0).stop 2024-03-04T07:47:25.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38cfe00640 1 -- 172.21.15.113:0/4097578950 shutdown_connections 2024-03-04T07:47:25.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38cfe00640 1 --2- 172.21.15.113:0/4097578950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d006faa0 0x7f38d00c30b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38cfe00640 1 --2- 172.21.15.113:0/4097578950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d00cb540 0x7f38d015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38cfe00640 1 -- 172.21.15.113:0/4097578950 >> 172.21.15.113:0/4097578950 conn(0x7f38d0072f40 msgr2=0x7f38d00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:25.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38cfe00640 1 -- 172.21.15.113:0/4097578950 shutdown_connections 2024-03-04T07:47:25.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.107+0000 7f38cfe00640 1 -- 172.21.15.113:0/4097578950 wait complete. 2024-03-04T07:47:25.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cfe00640 1 Processor -- start 2024-03-04T07:47:25.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cfe00640 1 -- start start 2024-03-04T07:47:25.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38d006faa0 0x7f38d01e3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d00cb540 0x7f38d01ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cfe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d0152950 0x7f38d0149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cfe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00c9a50 con 0x7f38d00cb540 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cfe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00c5220 con 0x7f38d006faa0 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cfe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00c9d30 con 0x7f38d0152950 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d00cb540 0x7f38d01ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d00cb540 0x7f38d01ecaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45158/0 (socket says 172.21.15.113:45158) 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c6000640 1 -- 172.21.15.113:0/2163845698 learned_addr learned my addr 172.21.15.113:0/2163845698 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cf400640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d0152950 0x7f38d0149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cea00640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38d006faa0 0x7f38d01e3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c6000640 1 -- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d0152950 msgr2=0x7f38d0149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c6000640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d0152950 0x7f38d0149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c6000640 1 -- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38d006faa0 msgr2=0x7f38d01e3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c6000640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38d006faa0 0x7f38d01e3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c6000640 1 -- 172.21.15.113:0/2163845698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d01ecfe0 con 0x7f38d00cb540 2024-03-04T07:47:25.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c6000640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d00cb540 0x7f38d01ecaa0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f38b800dcc0 tx=0x7f38b8010490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:25.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c7400640 1 -- 172.21.15.113:0/2163845698 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8010da0 con 0x7f38d00cb540 2024-03-04T07:47:25.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c7400640 1 -- 172.21.15.113:0/2163845698 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b801d070 con 0x7f38d00cb540 2024-03-04T07:47:25.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38cfe00640 1 -- 172.21.15.113:0/2163845698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d00c5490 con 0x7f38d00cb540 2024-03-04T07:47:25.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.111+0000 7f38c7400640 1 -- 172.21.15.113:0/2163845698 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8018680 con 0x7f38d00cb540 2024-03-04T07:47:25.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.115+0000 7f38cfe00640 1 -- 172.21.15.113:0/2163845698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d00c5a50 con 0x7f38d00cb540 2024-03-04T07:47:25.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.115+0000 7f38cfe00640 1 -- 172.21.15.113:0/2163845698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d0075a80 con 0x7f38d00cb540 2024-03-04T07:47:25.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.115+0000 7f38c7400640 1 -- 172.21.15.113:0/2163845698 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38b8016050 con 0x7f38d00cb540 2024-03-04T07:47:25.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.119+0000 7f38c7400640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f38ac066f60 0x7f38ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.119+0000 7f38c7400640 1 -- 172.21.15.113:0/2163845698 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f38b80892e0 con 0x7f38d00cb540 2024-03-04T07:47:25.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.119+0000 7f38cea00640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f38ac066f60 0x7f38ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.119+0000 7f38cea00640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f38ac066f60 0x7f38ac069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f38c0001210 tx=0x7f38c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:25.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.123+0000 7f38c7400640 1 -- 172.21.15.113:0/2163845698 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f38b8026020 con 0x7f38d00cb540 2024-03-04T07:47:25.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.411+0000 7f38cfe00640 1 -- 172.21.15.113:0/2163845698 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38d0157010 con 0x7f38d00cb540 2024-03-04T07:47:25.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.411+0000 7f38c7400640 1 -- 172.21.15.113:0/2163845698 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f38b8014040 con 0x7f38d00cb540 2024-03-04T07:47:25.410 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:25.410 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5417,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5631,5529,5244]},"id":1}]} 2024-03-04T07:47:25.410 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-03-04T07:47:25.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 -- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f38ac066f60 msgr2=0x7f38ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f38ac066f60 0x7f38ac069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f38c0001210 tx=0x7f38c0009040 comp rx=0 tx=0).stop 2024-03-04T07:47:25.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 -- 172.21.15.113:0/2163845698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d00cb540 msgr2=0x7f38d01ecaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d00cb540 0x7f38d01ecaa0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f38b800dcc0 tx=0x7f38b8010490 comp rx=0 tx=0).stop 2024-03-04T07:47:25.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 -- 172.21.15.113:0/2163845698 shutdown_connections 2024-03-04T07:47:25.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38d0152950 0x7f38d0149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f38ac066f60 0x7f38ac069420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38d00cb540 0x7f38d01ecaa0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 --2- 172.21.15.113:0/2163845698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38d006faa0 0x7f38d01e3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 -- 172.21.15.113:0/2163845698 >> 172.21.15.113:0/2163845698 conn(0x7f38d0072f40 msgr2=0x7f38d00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:25.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 -- 172.21.15.113:0/2163845698 shutdown_connections 2024-03-04T07:47:25.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.415+0000 7f38cfe00640 1 -- 172.21.15.113:0/2163845698 wait complete. 2024-03-04T07:47:25.438 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2024-03-04T07:47:25.438 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-03-04T07:47:25.439 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:47:25.481 INFO:tasks.ceph.mds.e:Stopped 2024-03-04T07:47:25.481 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:25.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2561400640 1 Processor -- start 2024-03-04T07:47:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2561400640 1 -- start start 2024-03-04T07:47:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2561400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f255c152b20 0x7f255c070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2561400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 0x7f255c163190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2561400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255c152f00 con 0x7f255c0c8b00 2024-03-04T07:47:25.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2561400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c0c8f90 con 0x7f255c152b20 2024-03-04T07:47:25.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2561400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0c5320 con 0x7f255c15c510 2024-03-04T07:47:25.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2553e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f255c0c8b00 0x7f255c15be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40070/0 (socket says 172.21.15.113:40070) 2024-03-04T07:47:25.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2553e00640 1 -- 172.21.15.113:0/2432842455 learned_addr learned my addr 172.21.15.113:0/2432842455 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:47:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f255b200640 1 --2- 172.21.15.113:0/2432842455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 0x7f255c163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f255a800640 1 --2- 172.21.15.113:0/2432842455 >> v2:172.21.15.136:3300/0 conn(0x7f255c152b20 0x7f255c070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2559e00640 1 -- 172.21.15.113:0/2432842455 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157056841 0 0) 0x7f255c152f00 con 0x7f255c0c8b00 2024-03-04T07:47:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2559e00640 1 -- 172.21.15.113:0/2432842455 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2540003530 con 0x7f255c0c8b00 2024-03-04T07:47:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f2559e00640 1 -- 172.21.15.113:0/2432842455 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 901223747 0 0) 0x7f2540003530 con 0x7f255c0c8b00 2024-03-04T07:47:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f255b200640 1 -- 172.21.15.113:0/2432842455 >> v2:172.21.15.136:3300/0 conn(0x7f255c152b20 msgr2=0x7f255c070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f255b200640 1 --2- 172.21.15.113:0/2432842455 >> v2:172.21.15.136:3300/0 conn(0x7f255c152b20 0x7f255c070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f255b200640 1 -- 172.21.15.113:0/2432842455 >> v1:172.21.15.113:6789/0 conn(0x7f255c0c8b00 legacy=0x7f255c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.623+0000 7f255b200640 1 -- 172.21.15.113:0/2432842455 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0c5040 con 0x7f255c15c510 2024-03-04T07:47:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f255b200640 1 --2- 172.21.15.113:0/2432842455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 0x7f255c163190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2554009d90 tx=0x7f255401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c30f2e74a72c63bf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2559e00640 1 -- 172.21.15.113:0/2432842455 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25540b2070 con 0x7f255c15c510 2024-03-04T07:47:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2559e00640 1 -- 172.21.15.113:0/2432842455 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25540a5580 con 0x7f255c15c510 2024-03-04T07:47:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2559e00640 1 -- 172.21.15.113:0/2432842455 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25440016a0 con 0x7f255c15c510 2024-03-04T07:47:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2561400640 1 -- 172.21.15.113:0/2432842455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 msgr2=0x7f255c163190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2561400640 1 --2- 172.21.15.113:0/2432842455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 0x7f255c163190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2554009d90 tx=0x7f255401c890 comp rx=0 tx=0).stop 2024-03-04T07:47:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2561400640 1 -- 172.21.15.113:0/2432842455 shutdown_connections 2024-03-04T07:47:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2561400640 1 --2- 172.21.15.113:0/2432842455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 0x7f255c163190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2561400640 1 --2- 172.21.15.113:0/2432842455 >> v2:172.21.15.136:3300/0 conn(0x7f255c152b20 0x7f255c070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2561400640 1 -- 172.21.15.113:0/2432842455 >> 172.21.15.113:0/2432842455 conn(0x7f255c0709a0 msgr2=0x7f255c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2561400640 1 -- 172.21.15.113:0/2432842455 shutdown_connections 2024-03-04T07:47:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.627+0000 7f2561400640 1 -- 172.21.15.113:0/2432842455 wait complete. 2024-03-04T07:47:25.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2561400640 1 Processor -- start 2024-03-04T07:47:25.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2561400640 1 -- start start 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2561400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255c0c8b00 0x7f255c14edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2561400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f255c152b20 0x7f255c1f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2561400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 0x7f255c0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2561400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c158810 con 0x7f255c0c8b00 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2561400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0c3810 con 0x7f255c152b20 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2561400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0c8490 con 0x7f255c15c510 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f255a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255c0c8b00 0x7f255c14edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f255a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255c0c8b00 0x7f255c14edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45164/0 (socket says 172.21.15.113:45164) 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f255a800640 1 -- 172.21.15.113:0/1034133886 learned_addr learned my addr 172.21.15.113:0/1034133886 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2553e00640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f255c152b20 0x7f255c1f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f255b200640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 0x7f255c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f255a800640 1 -- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 msgr2=0x7f255c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f255a800640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 0x7f255c0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f255a800640 1 -- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f255c152b20 msgr2=0x7f255c1f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f255a800640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f255c152b20 0x7f255c1f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f255a800640 1 -- 172.21.15.113:0/1034133886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0ce430 con 0x7f255c0c8b00 2024-03-04T07:47:25.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f255a800640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255c0c8b00 0x7f255c14edf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f254c00e870 tx=0x7f254c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:25.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2552a00640 1 -- 172.21.15.113:0/1034133886 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c017070 con 0x7f255c0c8b00 2024-03-04T07:47:25.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2552a00640 1 -- 172.21.15.113:0/1034133886 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c00c940 con 0x7f255c0c8b00 2024-03-04T07:47:25.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2561400640 1 -- 172.21.15.113:0/1034133886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c14f330 con 0x7f255c0c8b00 2024-03-04T07:47:25.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2552a00640 1 -- 172.21.15.113:0/1034133886 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c00ccf0 con 0x7f255c0c8b00 2024-03-04T07:47:25.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.631+0000 7f2561400640 1 -- 172.21.15.113:0/1034133886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c0cde70 con 0x7f255c0c8b00 2024-03-04T07:47:25.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.635+0000 7f2561400640 1 -- 172.21.15.113:0/1034133886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255c075c00 con 0x7f255c0c8b00 2024-03-04T07:47:25.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.635+0000 7f2552a00640 1 -- 172.21.15.113:0/1034133886 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f254c01a070 con 0x7f255c0c8b00 2024-03-04T07:47:25.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.635+0000 7f2552a00640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2538066f60 0x7f2538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:25.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.635+0000 7f2552a00640 1 -- 172.21.15.113:0/1034133886 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f254c085230 con 0x7f255c0c8b00 2024-03-04T07:47:25.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.635+0000 7f2553e00640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2538066f60 0x7f2538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:25.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.635+0000 7f2553e00640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2538066f60 0x7f2538069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f255c1f1350 tx=0x7f2544001c70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:25.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.639+0000 7f2552a00640 1 -- 172.21.15.113:0/1034133886 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f254c055f60 con 0x7f255c0c8b00 2024-03-04T07:47:25.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.835+0000 7f2561400640 1 -- 172.21.15.113:0/1034133886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f255c0c45b0 con 0x7f255c0c8b00 2024-03-04T07:47:25.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.835+0000 7f2552a00640 1 -- 172.21.15.113:0/1034133886 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f254c055860 con 0x7f255c0c8b00 2024-03-04T07:47:25.833 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:25.833 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5417,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5631,5529,5244]},"id":1}]} 2024-03-04T07:47:25.834 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-03-04T07:47:25.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 -- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2538066f60 msgr2=0x7f2538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2538066f60 0x7f2538069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f255c1f1350 tx=0x7f2544001c70 comp rx=0 tx=0).stop 2024-03-04T07:47:25.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 -- 172.21.15.113:0/1034133886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255c0c8b00 msgr2=0x7f255c14edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:25.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255c0c8b00 0x7f255c14edf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f254c00e870 tx=0x7f254c00ec70 comp rx=0 tx=0).stop 2024-03-04T07:47:25.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 -- 172.21.15.113:0/1034133886 shutdown_connections 2024-03-04T07:47:25.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f255c15c510 0x7f255c0cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f255c152b20 0x7f255c1f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2538066f60 0x7f2538069420 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 --2- 172.21.15.113:0/1034133886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f255c0c8b00 0x7f255c14edf0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:25.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 -- 172.21.15.113:0/1034133886 >> 172.21.15.113:0/1034133886 conn(0x7f255c0709a0 msgr2=0x7f255c151cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:25.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 -- 172.21.15.113:0/1034133886 shutdown_connections 2024-03-04T07:47:25.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:25.839+0000 7f2561400640 1 -- 172.21.15.113:0/1034133886 wait complete. 2024-03-04T07:47:25.854 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:47:27.856 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:27.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbc0c00640 1 Processor -- start 2024-03-04T07:47:27.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbc0c00640 1 -- start start 2024-03-04T07:47:27.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbc0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc14e4c0 0x7fcbbc072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:27.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbc0c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcbbc0c22e0 0x7fcbbc0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:27.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbc0c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbbc0c27f0 con 0x7fcbbc0c2d80 2024-03-04T07:47:27.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbc0c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbbc15ad60 con 0x7fcbbc0c22e0 2024-03-04T07:47:27.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbc0c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc0c2a90 con 0x7fcbbc14e4c0 2024-03-04T07:47:27.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbb9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc14e4c0 0x7fcbbc072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:27.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbb3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcbbc0c22e0 0x7fcbbc0731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:27.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbb9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc14e4c0 0x7fcbbc072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:50608/0 (socket says 172.21.15.113:50608) 2024-03-04T07:47:27.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbb9e00640 1 -- 172.21.15.113:0/2246726009 learned_addr learned my addr 172.21.15.113:0/2246726009 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:27.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbb9e00640 1 -- 172.21.15.113:0/2246726009 >> v2:172.21.15.136:3300/0 conn(0x7fcbbc0c22e0 msgr2=0x7fcbbc0731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:27.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbb9e00640 1 --2- 172.21.15.113:0/2246726009 >> v2:172.21.15.136:3300/0 conn(0x7fcbbc0c22e0 0x7fcbbc0731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:27.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbb9e00640 1 -- 172.21.15.113:0/2246726009 >> v1:172.21.15.136:6789/0 conn(0x7fcbbc0c2d80 legacy=0x7fcbbc15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:27.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:27.999+0000 7fcbb9e00640 1 -- 172.21.15.113:0/2246726009 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc070c60 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbb9e00640 1 --2- 172.21.15.113:0/2246726009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc14e4c0 0x7fcbbc072c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009870 tx=0x7fcbac01ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ffb9e9e010e526 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:28.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbb9400640 1 -- 172.21.15.113:0/2246726009 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0b5070 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbb9400640 1 -- 172.21.15.113:0/2246726009 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac0a8a50 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbb9400640 1 -- 172.21.15.113:0/2246726009 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0a8e20 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbc0c00640 1 -- 172.21.15.113:0/2246726009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc14e4c0 msgr2=0x7fcbbc072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:28.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbc0c00640 1 --2- 172.21.15.113:0/2246726009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc14e4c0 0x7fcbbc072c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009870 tx=0x7fcbac01ed50 comp rx=0 tx=0).stop 2024-03-04T07:47:28.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbc0c00640 1 -- 172.21.15.113:0/2246726009 shutdown_connections 2024-03-04T07:47:28.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbc0c00640 1 --2- 172.21.15.113:0/2246726009 >> v2:172.21.15.136:3300/0 conn(0x7fcbbc0c22e0 0x7fcbbc0731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:28.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbc0c00640 1 --2- 172.21.15.113:0/2246726009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc14e4c0 0x7fcbbc072c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:28.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbc0c00640 1 -- 172.21.15.113:0/2246726009 >> 172.21.15.113:0/2246726009 conn(0x7fcbbc06e820 msgr2=0x7fcbbc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:28.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbc0c00640 1 -- 172.21.15.113:0/2246726009 shutdown_connections 2024-03-04T07:47:28.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.003+0000 7fcbc0c00640 1 -- 172.21.15.113:0/2246726009 wait complete. 2024-03-04T07:47:28.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbc0c00640 1 Processor -- start 2024-03-04T07:47:28.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbc0c00640 1 -- start start 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbc0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0c22e0 0x7fcbbc0c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbc0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbbc0c2d80 0x7fcbbc152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbc0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc14e4c0 0x7fcbbc153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbc0c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc071500 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbc0c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc071220 con 0x7fcbbc0c2d80 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbc0c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc14e8a0 con 0x7fcbbc0c22e0 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbba800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc14e4c0 0x7fcbbc153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbb9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0c22e0 0x7fcbbc0c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbb3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbbc0c2d80 0x7fcbbc152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbb9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0c22e0 0x7fcbbc0c59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:50610/0 (socket says 172.21.15.113:50610) 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbba800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc14e4c0 0x7fcbbc153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52696/0 (socket says 172.21.15.113:52696) 2024-03-04T07:47:28.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbb9e00640 1 -- 172.21.15.113:0/3393724865 learned_addr learned my addr 172.21.15.113:0/3393724865 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:28.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbba800640 1 -- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0c22e0 msgr2=0x7fcbbc0c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:28.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbba800640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0c22e0 0x7fcbbc0c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:28.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbba800640 1 -- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbbc0c2d80 msgr2=0x7fcbbc152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:28.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbba800640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbbc0c2d80 0x7fcbbc152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:28.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbba800640 1 -- 172.21.15.113:0/3393724865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc14fb90 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbba800640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc14e4c0 0x7fcbbc153490 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400c700 tx=0x7fcbb400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:28.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbb2000640 1 -- 172.21.15.113:0/3393724865 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4003b80 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbb2000640 1 -- 172.21.15.113:0/3393724865 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb4003d20 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbb2000640 1 -- 172.21.15.113:0/3393724865 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400aa30 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbc0c00640 1 -- 172.21.15.113:0/3393724865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbbc15a5b0 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.007+0000 7fcbc0c00640 1 -- 172.21.15.113:0/3393724865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc152190 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.011+0000 7fcbc0c00640 1 -- 172.21.15.113:0/3393724865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbbc075c00 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.011+0000 7fcbb2000640 1 -- 172.21.15.113:0/3393724865 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbb400abd0 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.011+0000 7fcbb2000640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb98066f60 0x7fcb98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:28.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.011+0000 7fcbb2000640 1 -- 172.21.15.113:0/3393724865 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fcbb4085000 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.011+0000 7fcbb9e00640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb98066f60 0x7fcb98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:28.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.015+0000 7fcbb9e00640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb98066f60 0x7fcb98069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcbac01ed90 tx=0x7fcbac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:28.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.019+0000 7fcbb2000640 1 -- 172.21.15.113:0/3393724865 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcbb40556f0 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.211+0000 7fcbc0c00640 1 -- 172.21.15.113:0/3393724865 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbbc155190 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbb2000640 1 -- 172.21.15.113:0/3393724865 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fcbb4059790 con 0x7fcbbc14e4c0 2024-03-04T07:47:28.211 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:28.212 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5417,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5631,5529,5244]},"id":1}]} 2024-03-04T07:47:28.212 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-03-04T07:47:28.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 -- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb98066f60 msgr2=0x7fcb98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb98066f60 0x7fcb98069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcbac01ed90 tx=0x7fcbac009040 comp rx=0 tx=0).stop 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 -- 172.21.15.113:0/3393724865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc14e4c0 msgr2=0x7fcbbc153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc14e4c0 0x7fcbbc153490 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400c700 tx=0x7fcbb400cbc0 comp rx=0 tx=0).stop 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 -- 172.21.15.113:0/3393724865 shutdown_connections 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcb98066f60 0x7fcb98069420 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbbc14e4c0 0x7fcbbc153490 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcbbc0c2d80 0x7fcbbc152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 --2- 172.21.15.113:0/3393724865 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcbbc0c22e0 0x7fcbbc0c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 -- 172.21.15.113:0/3393724865 >> 172.21.15.113:0/3393724865 conn(0x7fcbbc06e820 msgr2=0x7fcbbc14c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 -- 172.21.15.113:0/3393724865 shutdown_connections 2024-03-04T07:47:28.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:28.215+0000 7fcbc0c00640 1 -- 172.21.15.113:0/3393724865 wait complete. 2024-03-04T07:47:28.233 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:47:28.657 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:47:28.660 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:47:30.234 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:30.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a8200640 1 Processor -- start 2024-03-04T07:47:30.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a8200640 1 -- start start 2024-03-04T07:47:30.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a8200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f38a00ca580 0x7f38a015e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:30.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a8200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a0067bc0 con 0x7f38a00ca1a0 2024-03-04T07:47:30.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a8200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a015bb40 con 0x7f38a015e690 2024-03-04T07:47:30.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a8200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a00678e0 con 0x7f38a00ca580 2024-03-04T07:47:30.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a5800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f38a00ca1a0 0x7f38a00b2ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48800/0 (socket says 172.21.15.113:48800) 2024-03-04T07:47:30.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a5800640 1 -- 172.21.15.113:0/335022512 learned_addr learned my addr 172.21.15.113:0/335022512 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:47:30.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a4e00640 1 --2- 172.21.15.113:0/335022512 >> v2:172.21.15.136:3300/0 conn(0x7f38a00ca580 0x7f38a015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:30.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f389fe00640 1 -- 172.21.15.113:0/335022512 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467135585 0 0) 0x7f38a0067bc0 con 0x7f38a00ca1a0 2024-03-04T07:47:30.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f389fe00640 1 -- 172.21.15.113:0/335022512 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3888003530 con 0x7f38a00ca1a0 2024-03-04T07:47:30.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a4e00640 1 -- 172.21.15.113:0/335022512 >> v1:172.21.15.136:6789/0 conn(0x7f38a015e690 legacy=0x7f38a00cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:30.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a4e00640 1 -- 172.21.15.113:0/335022512 >> v1:172.21.15.113:6789/0 conn(0x7f38a00ca1a0 legacy=0x7f38a00b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:30.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f389fe00640 1 -- 172.21.15.113:0/335022512 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 887036471 0 0) 0x7f3888003530 con 0x7f38a00ca1a0 2024-03-04T07:47:30.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.379+0000 7f38a4e00640 1 -- 172.21.15.113:0/335022512 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a015ad80 con 0x7f38a00ca580 2024-03-04T07:47:30.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.383+0000 7f38a4e00640 1 --2- 172.21.15.113:0/335022512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38a00ca580 0x7f38a015e150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f388c0097c0 tx=0x7f388c01ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91506775a6316910 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:30.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.383+0000 7f389fe00640 1 -- 172.21.15.113:0/335022512 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0b5070 con 0x7f38a00ca580 2024-03-04T07:47:30.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.383+0000 7f389fe00640 1 -- 172.21.15.113:0/335022512 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c0a8630 con 0x7f38a00ca580 2024-03-04T07:47:30.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.383+0000 7f389fe00640 1 -- 172.21.15.113:0/335022512 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894001680 con 0x7f38a00ca580 2024-03-04T07:47:30.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.383+0000 7f38a8200640 1 -- 172.21.15.113:0/335022512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38a00ca580 msgr2=0x7f38a015e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:30.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.383+0000 7f38a8200640 1 --2- 172.21.15.113:0/335022512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38a00ca580 0x7f38a015e150 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f388c0097c0 tx=0x7f388c01ebb0 comp rx=0 tx=0).stop 2024-03-04T07:47:30.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 -- 172.21.15.113:0/335022512 shutdown_connections 2024-03-04T07:47:30.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 --2- 172.21.15.113:0/335022512 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38a00ca580 0x7f38a015e150 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:30.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 -- 172.21.15.113:0/335022512 >> 172.21.15.113:0/335022512 conn(0x7f38a006dac0 msgr2=0x7f38a0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:30.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 -- 172.21.15.113:0/335022512 shutdown_connections 2024-03-04T07:47:30.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 -- 172.21.15.113:0/335022512 wait complete. 2024-03-04T07:47:30.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 Processor -- start 2024-03-04T07:47:30.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 -- start start 2024-03-04T07:47:30.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a00ca1a0 0x7f38a00c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:30.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38a00ca580 0x7f38a00c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:30.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38a015e690 0x7f38a0154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:30.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a015b620 con 0x7f38a00ca1a0 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a015b340 con 0x7f38a015e690 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00674c0 con 0x7f38a00ca580 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a00ca1a0 0x7f38a00c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a00ca1a0 0x7f38a00c5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52706/0 (socket says 172.21.15.113:52706) 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a5800640 1 -- 172.21.15.113:0/265409307 learned_addr learned my addr 172.21.15.113:0/265409307 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a6200640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38a015e690 0x7f38a0154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a4e00640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38a00ca580 0x7f38a00c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a5800640 1 -- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38a00ca580 msgr2=0x7f38a00c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a5800640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38a00ca580 0x7f38a00c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a5800640 1 -- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38a015e690 msgr2=0x7f38a0154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a5800640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38a015e690 0x7f38a0154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:30.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a5800640 1 -- 172.21.15.113:0/265409307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0156a20 con 0x7f38a00ca1a0 2024-03-04T07:47:30.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a6200640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38a015e690 0x7f38a0154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:47:30.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f38a5800640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a00ca1a0 0x7f38a00c5380 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f389400c720 tx=0x7f389400cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:30.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f389d600640 1 -- 172.21.15.113:0/265409307 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894005ae0 con 0x7f38a00ca1a0 2024-03-04T07:47:30.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.387+0000 7f389d600640 1 -- 172.21.15.113:0/265409307 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3894005c80 con 0x7f38a00ca1a0 2024-03-04T07:47:30.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.391+0000 7f389d600640 1 -- 172.21.15.113:0/265409307 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894007270 con 0x7f38a00ca1a0 2024-03-04T07:47:30.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.391+0000 7f38a8200640 1 -- 172.21.15.113:0/265409307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a00b3d30 con 0x7f38a00ca1a0 2024-03-04T07:47:30.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.391+0000 7f38a8200640 1 -- 172.21.15.113:0/265409307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a00b4290 con 0x7f38a00ca1a0 2024-03-04T07:47:30.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.391+0000 7f38a8200640 1 -- 172.21.15.113:0/265409307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a0075c00 con 0x7f38a00ca1a0 2024-03-04T07:47:30.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.391+0000 7f389d600640 1 -- 172.21.15.113:0/265409307 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3894007410 con 0x7f38a00ca1a0 2024-03-04T07:47:30.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.391+0000 7f389d600640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3880066f60 0x7f3880069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:30.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.391+0000 7f389d600640 1 -- 172.21.15.113:0/265409307 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f3894085260 con 0x7f38a00ca1a0 2024-03-04T07:47:30.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.391+0000 7f38a4e00640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3880066f60 0x7f3880069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:30.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.395+0000 7f38a4e00640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3880066f60 0x7f3880069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f388c0b2000 tx=0x7f388c0aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:30.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.395+0000 7f389d600640 1 -- 172.21.15.113:0/265409307 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3894010300 con 0x7f38a00ca1a0 2024-03-04T07:47:30.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.583+0000 7f38a8200640 1 -- 172.21.15.113:0/265409307 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38a015eac0 con 0x7f38a00ca1a0 2024-03-04T07:47:30.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.583+0000 7f389d600640 1 -- 172.21.15.113:0/265409307 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f3894055f00 con 0x7f38a00ca1a0 2024-03-04T07:47:30.582 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:30.582 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5417,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5631,5529,5244]},"id":1}]} 2024-03-04T07:47:30.582 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-03-04T07:47:30.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 -- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3880066f60 msgr2=0x7f3880069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3880066f60 0x7f3880069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f388c0b2000 tx=0x7f388c0aa070 comp rx=0 tx=0).stop 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 -- 172.21.15.113:0/265409307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a00ca1a0 msgr2=0x7f38a00c5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a00ca1a0 0x7f38a00c5380 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f389400c720 tx=0x7f389400cb20 comp rx=0 tx=0).stop 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 -- 172.21.15.113:0/265409307 shutdown_connections 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f38a015e690 0x7f38a0154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f38a00ca580 0x7f38a00c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3880066f60 0x7f3880069420 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 --2- 172.21.15.113:0/265409307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f38a00ca1a0 0x7f38a00c5380 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 -- 172.21.15.113:0/265409307 >> 172.21.15.113:0/265409307 conn(0x7f38a006dac0 msgr2=0x7f38a015bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 -- 172.21.15.113:0/265409307 shutdown_connections 2024-03-04T07:47:30.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:30.587+0000 7f38a8200640 1 -- 172.21.15.113:0/265409307 wait complete. 2024-03-04T07:47:30.604 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:47:32.605 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:32.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f1091400640 1 Processor -- start 2024-03-04T07:47:32.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f1091400640 1 -- start start 2024-03-04T07:47:32.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f1091400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c154cf0 0x7f108c0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:32.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f1091400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c15dde0 con 0x7f108c0ce590 2024-03-04T07:47:32.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f1091400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c15e080 con 0x7f108c0c8b90 2024-03-04T07:47:32.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f1091400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c15e360 con 0x7f108c154cf0 2024-03-04T07:47:32.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f108be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f108c0ce590 0x7f108c06fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48802/0 (socket says 172.21.15.113:48802) 2024-03-04T07:47:32.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f108be00640 1 -- 172.21.15.113:0/901537531 learned_addr learned my addr 172.21.15.113:0/901537531 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:47:32.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f1090a00640 1 --2- 172.21.15.113:0/901537531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c154cf0 0x7f108c0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:32.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f108aa00640 1 -- 172.21.15.113:0/901537531 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210825666 0 0) 0x7f108c15dde0 con 0x7f108c0ce590 2024-03-04T07:47:32.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f108aa00640 1 -- 172.21.15.113:0/901537531 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1070003530 con 0x7f108c0ce590 2024-03-04T07:47:32.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f108aa00640 1 -- 172.21.15.113:0/901537531 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806588380 0 0) 0x7f108c15e080 con 0x7f108c0c8b90 2024-03-04T07:47:32.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.747+0000 7f108aa00640 1 -- 172.21.15.113:0/901537531 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f108c15dde0 con 0x7f108c0c8b90 2024-03-04T07:47:32.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.751+0000 7f108aa00640 1 -- 172.21.15.113:0/901537531 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3184643432 0 0) 0x7f1070003530 con 0x7f108c0ce590 2024-03-04T07:47:32.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.751+0000 7f1090a00640 1 -- 172.21.15.113:0/901537531 >> v1:172.21.15.136:6789/0 conn(0x7f108c0c8b90 legacy=0x7f108c1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:32.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.751+0000 7f1090a00640 1 -- 172.21.15.113:0/901537531 >> v1:172.21.15.113:6789/0 conn(0x7f108c0ce590 legacy=0x7f108c06fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:32.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.751+0000 7f1090a00640 1 -- 172.21.15.113:0/901537531 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c15e640 con 0x7f108c154cf0 2024-03-04T07:47:32.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f1090a00640 1 --2- 172.21.15.113:0/901537531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c154cf0 0x7f108c0c9e80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1084009dc0 tx=0x7f108401f0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60d938a5a6c8bbed server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:32.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f108aa00640 1 -- 172.21.15.113:0/901537531 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10840b5070 con 0x7f108c154cf0 2024-03-04T07:47:32.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f108aa00640 1 -- 172.21.15.113:0/901537531 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10840a8c30 con 0x7f108c154cf0 2024-03-04T07:47:32.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f108aa00640 1 -- 172.21.15.113:0/901537531 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c001680 con 0x7f108c154cf0 2024-03-04T07:47:32.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f1091400640 1 -- 172.21.15.113:0/901537531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c154cf0 msgr2=0x7f108c0c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:32.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f1091400640 1 --2- 172.21.15.113:0/901537531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c154cf0 0x7f108c0c9e80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1084009dc0 tx=0x7f108401f0a0 comp rx=0 tx=0).stop 2024-03-04T07:47:32.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f1091400640 1 -- 172.21.15.113:0/901537531 shutdown_connections 2024-03-04T07:47:32.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f1091400640 1 --2- 172.21.15.113:0/901537531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c154cf0 0x7f108c0c9e80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:32.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f1091400640 1 -- 172.21.15.113:0/901537531 >> 172.21.15.113:0/901537531 conn(0x7f108c066f80 msgr2=0x7f108c067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:32.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f1091400640 1 -- 172.21.15.113:0/901537531 shutdown_connections 2024-03-04T07:47:32.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.755+0000 7f1091400640 1 -- 172.21.15.113:0/901537531 wait complete. 2024-03-04T07:47:32.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1091400640 1 Processor -- start 2024-03-04T07:47:32.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1091400640 1 -- start start 2024-03-04T07:47:32.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1091400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c0c8b90 0x7f108c1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:32.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1091400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c0ce590 0x7f108c06c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1091400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f108c154cf0 0x7f108c157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1091400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0ce970 con 0x7f108c0c8b90 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1091400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c15ec00 con 0x7f108c154cf0 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1091400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0ceaf0 con 0x7f108c0ce590 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f108be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c0c8b90 0x7f108c1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f108be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c0c8b90 0x7f108c1f3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52714/0 (socket says 172.21.15.113:52714) 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f108be00640 1 -- 172.21.15.113:0/2155419128 learned_addr learned my addr 172.21.15.113:0/2155419128 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f108b400640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c0ce590 0x7f108c06c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1090a00640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f108c154cf0 0x7f108c157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f108be00640 1 -- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c0ce590 msgr2=0x7f108c06c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f108be00640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c0ce590 0x7f108c06c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f108be00640 1 -- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f108c154cf0 msgr2=0x7f108c157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:32.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f108be00640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f108c154cf0 0x7f108c157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:32.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f108be00640 1 -- 172.21.15.113:0/2155419128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c155b00 con 0x7f108c0c8b90 2024-03-04T07:47:32.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1090a00640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f108c154cf0 0x7f108c157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:47:32.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f108be00640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c0c8b90 0x7f108c1f3a00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c780 tx=0x7f107c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:32.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1083e00640 1 -- 172.21.15.113:0/2155419128 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c017070 con 0x7f108c0c8b90 2024-03-04T07:47:32.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1083e00640 1 -- 172.21.15.113:0/2155419128 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c005b00 con 0x7f108c0c8b90 2024-03-04T07:47:32.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1083e00640 1 -- 172.21.15.113:0/2155419128 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c007070 con 0x7f108c0c8b90 2024-03-04T07:47:32.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.759+0000 7f1091400640 1 -- 172.21.15.113:0/2155419128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c155540 con 0x7f108c0c8b90 2024-03-04T07:47:32.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.763+0000 7f1091400640 1 -- 172.21.15.113:0/2155419128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c14bb40 con 0x7f108c0c8b90 2024-03-04T07:47:32.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.763+0000 7f1091400640 1 -- 172.21.15.113:0/2155419128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c075a80 con 0x7f108c0c8b90 2024-03-04T07:47:32.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.763+0000 7f1083e00640 1 -- 172.21.15.113:0/2155419128 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f107c01a070 con 0x7f108c0c8b90 2024-03-04T07:47:32.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.763+0000 7f1083e00640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f106c066f60 0x7f106c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:32.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.763+0000 7f1083e00640 1 -- 172.21.15.113:0/2155419128 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f107c0848e0 con 0x7f108c0c8b90 2024-03-04T07:47:32.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.763+0000 7f108b400640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f106c066f60 0x7f106c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:32.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.767+0000 7f108b400640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f106c066f60 0x7f106c069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f108c1f3960 tx=0x7f1078009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:32.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.767+0000 7f1083e00640 1 -- 172.21.15.113:0/2155419128 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f107c054f30 con 0x7f108c0c8b90 2024-03-04T07:47:32.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.963+0000 7f1091400640 1 -- 172.21.15.113:0/2155419128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f108c0c9530 con 0x7f108c0c8b90 2024-03-04T07:47:32.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.963+0000 7f1083e00640 1 -- 172.21.15.113:0/2155419128 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f107c058fd0 con 0x7f108c0c8b90 2024-03-04T07:47:32.961 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:32.961 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5417,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5631,5529,5244]},"id":1}]} 2024-03-04T07:47:32.961 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-03-04T07:47:32.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 -- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f106c066f60 msgr2=0x7f106c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:32.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f106c066f60 0x7f106c069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f108c1f3960 tx=0x7f1078009040 comp rx=0 tx=0).stop 2024-03-04T07:47:32.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 -- 172.21.15.113:0/2155419128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c0c8b90 msgr2=0x7f108c1f3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:32.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c0c8b90 0x7f108c1f3a00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c780 tx=0x7f107c00cc40 comp rx=0 tx=0).stop 2024-03-04T07:47:32.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 -- 172.21.15.113:0/2155419128 shutdown_connections 2024-03-04T07:47:32.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f108c154cf0 0x7f108c157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:32.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f108c0ce590 0x7f108c06c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:32.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f106c066f60 0x7f106c069420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:32.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 --2- 172.21.15.113:0/2155419128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f108c0c8b90 0x7f108c1f3a00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:32.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 -- 172.21.15.113:0/2155419128 >> 172.21.15.113:0/2155419128 conn(0x7f108c066f80 msgr2=0x7f108c06f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:32.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 -- 172.21.15.113:0/2155419128 shutdown_connections 2024-03-04T07:47:32.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:32.967+0000 7f1091400640 1 -- 172.21.15.113:0/2155419128 wait complete. 2024-03-04T07:47:32.987 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:47:34.988 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:35.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.151+0000 7ff776200640 1 Processor -- start 2024-03-04T07:47:35.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff776200640 1 -- start start 2024-03-04T07:47:35.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff776200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff770152950 0x7ff77015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff776200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff77006faa0 0x7ff7700c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff776200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7700c4ce0 con 0x7ff77006ed80 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff776200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7700c8370 con 0x7ff770152950 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff776200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7700ca0a0 con 0x7ff77006faa0 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff774e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff770152950 0x7ff77015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff774e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff770152950 0x7ff77015bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52728/0 (socket says 172.21.15.113:52728) 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff774e00640 1 -- 172.21.15.113:0/283409364 learned_addr learned my addr 172.21.15.113:0/283409364 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff76fe00640 1 --2- 172.21.15.113:0/283409364 >> v2:172.21.15.136:3300/0 conn(0x7ff77006faa0 0x7ff7700c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff774e00640 1 -- 172.21.15.113:0/283409364 >> v2:172.21.15.136:3300/0 conn(0x7ff77006faa0 msgr2=0x7ff7700c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff774e00640 1 --2- 172.21.15.113:0/283409364 >> v2:172.21.15.136:3300/0 conn(0x7ff77006faa0 0x7ff7700c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff774e00640 1 -- 172.21.15.113:0/283409364 >> v1:172.21.15.136:6789/0 conn(0x7ff77006ed80 legacy=0x7ff7700ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:35.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.155+0000 7ff774e00640 1 -- 172.21.15.113:0/283409364 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7700c4f80 con 0x7ff770152950 2024-03-04T07:47:35.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff774e00640 1 --2- 172.21.15.113:0/283409364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff770152950 0x7ff77015bb40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff77006c110 tx=0x7ff76401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ddb496bd80e3a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:35.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff76f400640 1 -- 172.21.15.113:0/283409364 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640b5070 con 0x7ff770152950 2024-03-04T07:47:35.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff76f400640 1 -- 172.21.15.113:0/283409364 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7640a8a10 con 0x7ff770152950 2024-03-04T07:47:35.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff76f400640 1 -- 172.21.15.113:0/283409364 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640a8de0 con 0x7ff770152950 2024-03-04T07:47:35.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff776200640 1 -- 172.21.15.113:0/283409364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff770152950 msgr2=0x7ff77015bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:35.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff776200640 1 --2- 172.21.15.113:0/283409364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff770152950 0x7ff77015bb40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff77006c110 tx=0x7ff76401ed10 comp rx=0 tx=0).stop 2024-03-04T07:47:35.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff776200640 1 -- 172.21.15.113:0/283409364 shutdown_connections 2024-03-04T07:47:35.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff776200640 1 --2- 172.21.15.113:0/283409364 >> v2:172.21.15.136:3300/0 conn(0x7ff77006faa0 0x7ff7700c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:35.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff776200640 1 --2- 172.21.15.113:0/283409364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff770152950 0x7ff77015bb40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:35.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff776200640 1 -- 172.21.15.113:0/283409364 >> 172.21.15.113:0/283409364 conn(0x7ff770072f40 msgr2=0x7ff7700706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:35.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff776200640 1 -- 172.21.15.113:0/283409364 shutdown_connections 2024-03-04T07:47:35.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff776200640 1 -- 172.21.15.113:0/283409364 wait complete. 2024-03-04T07:47:35.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff776200640 1 Processor -- start 2024-03-04T07:47:35.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.159+0000 7ff776200640 1 -- start start 2024-03-04T07:47:35.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff776200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77006ed80 0x7ff77015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:35.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff776200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77006faa0 0x7ff77006da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:35.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff776200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff770152950 0x7ff7700caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:35.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff776200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7700c9a50 con 0x7ff77006ed80 2024-03-04T07:47:35.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff776200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77006fe80 con 0x7ff770152950 2024-03-04T07:47:35.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff776200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7700c9d30 con 0x7ff77006faa0 2024-03-04T07:47:35.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff774e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77006ed80 0x7ff77015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:35.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff774e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77006ed80 0x7ff77015ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52738/0 (socket says 172.21.15.113:52738) 2024-03-04T07:47:35.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff774e00640 1 -- 172.21.15.113:0/2191040728 learned_addr learned my addr 172.21.15.113:0/2191040728 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:35.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff76fe00640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77006faa0 0x7ff77006da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:35.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff775800640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff770152950 0x7ff7700caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:35.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff774e00640 1 -- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77006faa0 msgr2=0x7ff77006da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:35.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff774e00640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77006faa0 0x7ff77006da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:35.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff774e00640 1 -- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff770152950 msgr2=0x7ff7700caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:35.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff774e00640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff770152950 0x7ff7700caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:35.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff774e00640 1 -- 172.21.15.113:0/2191040728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77006df80 con 0x7ff77006ed80 2024-03-04T07:47:35.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff774e00640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77006ed80 0x7ff77015ba80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff76401b920 tx=0x7ff76401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:35.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff76cc00640 1 -- 172.21.15.113:0/2191040728 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640b5070 con 0x7ff77006ed80 2024-03-04T07:47:35.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff76cc00640 1 -- 172.21.15.113:0/2191040728 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7640b0b40 con 0x7ff77006ed80 2024-03-04T07:47:35.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff76cc00640 1 -- 172.21.15.113:0/2191040728 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640bb410 con 0x7ff77006ed80 2024-03-04T07:47:35.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff776200640 1 -- 172.21.15.113:0/2191040728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77006ca20 con 0x7ff77006ed80 2024-03-04T07:47:35.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff776200640 1 -- 172.21.15.113:0/2191040728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77006cfc0 con 0x7ff77006ed80 2024-03-04T07:47:35.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.163+0000 7ff776200640 1 -- 172.21.15.113:0/2191040728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff770075a80 con 0x7ff77006ed80 2024-03-04T07:47:35.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.167+0000 7ff76cc00640 1 -- 172.21.15.113:0/2191040728 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7640c2060 con 0x7ff77006ed80 2024-03-04T07:47:35.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.167+0000 7ff76cc00640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff750066f60 0x7ff750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:35.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.167+0000 7ff76cc00640 1 -- 172.21.15.113:0/2191040728 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7ff7641224e0 con 0x7ff77006ed80 2024-03-04T07:47:35.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.167+0000 7ff76fe00640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff750066f60 0x7ff750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:35.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.167+0000 7ff76fe00640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff750066f60 0x7ff750069420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff770156bc0 tx=0x7ff75c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:35.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.175+0000 7ff76cc00640 1 -- 172.21.15.113:0/2191040728 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff764018870 con 0x7ff77006ed80 2024-03-04T07:47:35.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 -- 172.21.15.113:0/2191040728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7700ca780 con 0x7ff77006ed80 2024-03-04T07:47:35.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff76cc00640 1 -- 172.21.15.113:0/2191040728 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6612 (secure 0 0 0) 0x7ff7640f3210 con 0x7ff77006ed80 2024-03-04T07:47:35.516 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:35.516 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5417,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5631,5529,5244]},"id":1}]} 2024-03-04T07:47:35.517 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-03-04T07:47:35.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 -- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff750066f60 msgr2=0x7ff750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:35.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff750066f60 0x7ff750069420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff770156bc0 tx=0x7ff75c008040 comp rx=0 tx=0).stop 2024-03-04T07:47:35.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 -- 172.21.15.113:0/2191040728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77006ed80 msgr2=0x7ff77015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:35.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77006ed80 0x7ff77015ba80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff76401b920 tx=0x7ff76401bab0 comp rx=0 tx=0).stop 2024-03-04T07:47:35.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 -- 172.21.15.113:0/2191040728 shutdown_connections 2024-03-04T07:47:35.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff770152950 0x7ff7700caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:35.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff77006faa0 0x7ff77006da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:35.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff750066f60 0x7ff750069420 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:35.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 --2- 172.21.15.113:0/2191040728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff77006ed80 0x7ff77015ba80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:35.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 -- 172.21.15.113:0/2191040728 >> 172.21.15.113:0/2191040728 conn(0x7ff770072f40 msgr2=0x7ff7700c6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:35.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 -- 172.21.15.113:0/2191040728 shutdown_connections 2024-03-04T07:47:35.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:35.519+0000 7ff776200640 1 -- 172.21.15.113:0/2191040728 wait complete. 2024-03-04T07:47:35.538 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:47:37.539 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:37.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c11c00640 1 Processor -- start 2024-03-04T07:47:37.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c11c00640 1 -- start start 2024-03-04T07:47:37.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c11c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6c0c152d60 0x7f6c0c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:37.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c11c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c0c070120 0x7f6c0c0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:37.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c11c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c0c0c9db0 0x7f6c0c158930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:37.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c11c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c5380 con 0x7f6c0c0c9db0 2024-03-04T07:47:37.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c11c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c3440 con 0x7f6c0c152d60 2024-03-04T07:47:37.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c11c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c06f620 con 0x7f6c0c070120 2024-03-04T07:47:37.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c0b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c0c0c9db0 0x7f6c0c158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:37.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c0b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6c0c0c9db0 0x7f6c0c158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34256/0 (socket says 172.21.15.113:34256) 2024-03-04T07:47:37.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c0b400640 1 -- 172.21.15.113:0/2131484374 learned_addr learned my addr 172.21.15.113:0/2131484374 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:37.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c0a000640 1 --2- 172.21.15.113:0/2131484374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c0c070120 0x7f6c0c0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:37.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c0aa00640 1 --2- 172.21.15.113:0/2131484374 >> v2:172.21.15.136:3300/0 conn(0x7f6c0c152d60 0x7f6c0c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:37.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c0b400640 1 -- 172.21.15.113:0/2131484374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c0c070120 msgr2=0x7f6c0c0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:37.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c0b400640 1 --2- 172.21.15.113:0/2131484374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c0c070120 0x7f6c0c0ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c0b400640 1 -- 172.21.15.113:0/2131484374 >> v2:172.21.15.136:3300/0 conn(0x7f6c0c152d60 msgr2=0x7f6c0c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:37.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c0b400640 1 --2- 172.21.15.113:0/2131484374 >> v2:172.21.15.136:3300/0 conn(0x7f6c0c152d60 0x7f6c0c0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.683+0000 7f6c0b400640 1 -- 172.21.15.113:0/2131484374 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0c36e0 con 0x7f6c0c0c9db0 2024-03-04T07:47:37.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c0b400640 1 --2- 172.21.15.113:0/2131484374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c0c9db0 0x7f6c0c158930 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0cbf40 tx=0x7f6c0401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df89aca791295659 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:37.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c09600640 1 -- 172.21.15.113:0/2131484374 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c040b4070 con 0x7f6c0c0c9db0 2024-03-04T07:47:37.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c09600640 1 -- 172.21.15.113:0/2131484374 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c040af400 con 0x7f6c0c0c9db0 2024-03-04T07:47:37.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c09600640 1 -- 172.21.15.113:0/2131484374 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c040af7d0 con 0x7f6c0c0c9db0 2024-03-04T07:47:37.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c11c00640 1 -- 172.21.15.113:0/2131484374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c0c9db0 msgr2=0x7f6c0c158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:37.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c11c00640 1 --2- 172.21.15.113:0/2131484374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c0c9db0 0x7f6c0c158930 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0cbf40 tx=0x7f6c0401f1e0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c11c00640 1 -- 172.21.15.113:0/2131484374 shutdown_connections 2024-03-04T07:47:37.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c11c00640 1 --2- 172.21.15.113:0/2131484374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c0c9db0 0x7f6c0c158930 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c11c00640 1 --2- 172.21.15.113:0/2131484374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c0c070120 0x7f6c0c0ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c11c00640 1 --2- 172.21.15.113:0/2131484374 >> v2:172.21.15.136:3300/0 conn(0x7f6c0c152d60 0x7f6c0c0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c11c00640 1 -- 172.21.15.113:0/2131484374 >> 172.21.15.113:0/2131484374 conn(0x7f6c0c066db0 msgr2=0x7f6c0c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:37.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c11c00640 1 -- 172.21.15.113:0/2131484374 shutdown_connections 2024-03-04T07:47:37.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.687+0000 7f6c11c00640 1 -- 172.21.15.113:0/2131484374 wait complete. 2024-03-04T07:47:37.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c11c00640 1 Processor -- start 2024-03-04T07:47:37.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c11c00640 1 -- start start 2024-03-04T07:47:37.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c11c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c070120 0x7f6c0c0b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:37.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c11c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c0c0c9db0 0x7f6c0c0c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:37.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c11c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c0c152d60 0x7f6c0c0c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:37.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c11c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c39c0 con 0x7f6c0c070120 2024-03-04T07:47:37.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c11c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c5020 con 0x7f6c0c0c9db0 2024-03-04T07:47:37.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c11c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0c3b40 con 0x7f6c0c152d60 2024-03-04T07:47:37.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c070120 0x7f6c0c0b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:37.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c070120 0x7f6c0c0b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34262/0 (socket says 172.21.15.113:34262) 2024-03-04T07:47:37.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0aa00640 1 -- 172.21.15.113:0/3465814170 learned_addr learned my addr 172.21.15.113:0/3465814170 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:37.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0a000640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c0c0c9db0 0x7f6c0c0c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:37.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0b400640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c0c152d60 0x7f6c0c0c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:37.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0aa00640 1 -- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c0c152d60 msgr2=0x7f6c0c0c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:37.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0aa00640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c0c152d60 0x7f6c0c0c83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0aa00640 1 -- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c0c0c9db0 msgr2=0x7f6c0c0c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:37.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0aa00640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c0c0c9db0 0x7f6c0c0c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0aa00640 1 -- 172.21.15.113:0/3465814170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0c9b60 con 0x7f6c0c070120 2024-03-04T07:47:37.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c0aa00640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c070120 0x7f6c0c0b3450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00d830 tx=0x7f6bfc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:37.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c02a00640 1 -- 172.21.15.113:0/3465814170 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00b6c0 con 0x7f6c0c070120 2024-03-04T07:47:37.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c02a00640 1 -- 172.21.15.113:0/3465814170 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc019070 con 0x7f6c0c070120 2024-03-04T07:47:37.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c11c00640 1 -- 172.21.15.113:0/3465814170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c15c760 con 0x7f6c0c070120 2024-03-04T07:47:37.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c02a00640 1 -- 172.21.15.113:0/3465814170 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00ba60 con 0x7f6c0c070120 2024-03-04T07:47:37.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.691+0000 7f6c11c00640 1 -- 172.21.15.113:0/3465814170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c0c60a0 con 0x7f6c0c070120 2024-03-04T07:47:37.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.695+0000 7f6c11c00640 1 -- 172.21.15.113:0/3465814170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c075c00 con 0x7f6c0c070120 2024-03-04T07:47:37.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.695+0000 7f6c02a00640 1 -- 172.21.15.113:0/3465814170 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6bfc01e460 con 0x7f6c0c070120 2024-03-04T07:47:37.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.695+0000 7f6c02a00640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bec066f60 0x7f6bec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:37.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.695+0000 7f6c02a00640 1 -- 172.21.15.113:0/3465814170 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f6bfc0859d0 con 0x7f6c0c070120 2024-03-04T07:47:37.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.695+0000 7f6c0a000640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bec066f60 0x7f6bec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:37.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.695+0000 7f6c0a000640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bec066f60 0x7f6bec069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0b16b0 tx=0x7f6bf8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:37.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.699+0000 7f6c02a00640 1 -- 172.21.15.113:0/3465814170 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6bfc055f90 con 0x7f6c0c070120 2024-03-04T07:47:37.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 -- 172.21.15.113:0/3465814170 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c0c0cece0 con 0x7f6c0c070120 2024-03-04T07:47:37.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c02a00640 1 -- 172.21.15.113:0/3465814170 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f6bfc05a030 con 0x7f6c0c070120 2024-03-04T07:47:37.892 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:37.892 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5417,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5631,5529,5244]},"id":1}]} 2024-03-04T07:47:37.892 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-03-04T07:47:37.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 -- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bec066f60 msgr2=0x7f6bec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:37.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bec066f60 0x7f6bec069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0b16b0 tx=0x7f6bf8008040 comp rx=0 tx=0).stop 2024-03-04T07:47:37.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 -- 172.21.15.113:0/3465814170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c070120 msgr2=0x7f6c0c0b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:37.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c070120 0x7f6c0c0b3450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc00d830 tx=0x7f6bfc00dcf0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 -- 172.21.15.113:0/3465814170 shutdown_connections 2024-03-04T07:47:37.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6c0c152d60 0x7f6c0c0c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c0c0c9db0 0x7f6c0c0c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bec066f60 0x7f6bec069420 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 --2- 172.21.15.113:0/3465814170 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c0c070120 0x7f6c0c0b3450 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:37.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 -- 172.21.15.113:0/3465814170 >> 172.21.15.113:0/3465814170 conn(0x7f6c0c066db0 msgr2=0x7f6c0c0b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:37.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 -- 172.21.15.113:0/3465814170 shutdown_connections 2024-03-04T07:47:37.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:37.895+0000 7f6c11c00640 1 -- 172.21.15.113:0/3465814170 wait complete. 2024-03-04T07:47:37.915 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:47:39.916 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:40.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb64c00640 1 Processor -- start 2024-03-04T07:47:40.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb64c00640 1 -- start start 2024-03-04T07:47:40.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb64c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb60151760 0x7feb600c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:40.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb64c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feb60151b40 0x7feb6015b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:40.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb64c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6006c290 con 0x7feb6015b8c0 2024-03-04T07:47:40.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb64c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6015aef0 con 0x7feb60151b40 2024-03-04T07:47:40.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb64c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6006ef10 con 0x7feb60151760 2024-03-04T07:47:40.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb5fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7feb6015b8c0 0x7feb60071f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45504/0 (socket says 172.21.15.113:45504) 2024-03-04T07:47:40.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb5fe00640 1 -- 172.21.15.113:0/962299953 learned_addr learned my addr 172.21.15.113:0/962299953 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:47:40.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb5ea00640 1 --2- 172.21.15.113:0/962299953 >> v2:172.21.15.136:3300/0 conn(0x7feb60151b40 0x7feb6015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:40.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb5f400640 1 --2- 172.21.15.113:0/962299953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb60151760 0x7feb600c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:40.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb5e000640 1 -- 172.21.15.113:0/962299953 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923940758 0 0) 0x7feb6006c290 con 0x7feb6015b8c0 2024-03-04T07:47:40.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb5e000640 1 -- 172.21.15.113:0/962299953 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb44003530 con 0x7feb6015b8c0 2024-03-04T07:47:40.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb5ea00640 1 -- 172.21.15.113:0/962299953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb60151760 msgr2=0x7feb600c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:40.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb5ea00640 1 --2- 172.21.15.113:0/962299953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb60151760 0x7feb600c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:40.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb5ea00640 1 -- 172.21.15.113:0/962299953 >> v1:172.21.15.113:6789/0 conn(0x7feb6015b8c0 legacy=0x7feb60071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:40.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.063+0000 7feb5ea00640 1 -- 172.21.15.113:0/962299953 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600c1000 con 0x7feb60151b40 2024-03-04T07:47:40.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb5ea00640 1 --2- 172.21.15.113:0/962299953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb60151b40 0x7feb6015b380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0097f0 tx=0x7feb4c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad043f262f8d71a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:40.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb5e000640 1 -- 172.21.15.113:0/962299953 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c0b0070 con 0x7feb60151b40 2024-03-04T07:47:40.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb5e000640 1 -- 172.21.15.113:0/962299953 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c0ab600 con 0x7feb60151b40 2024-03-04T07:47:40.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb5e000640 1 -- 172.21.15.113:0/962299953 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c0ab9b0 con 0x7feb60151b40 2024-03-04T07:47:40.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb64c00640 1 -- 172.21.15.113:0/962299953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb60151b40 msgr2=0x7feb6015b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:40.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb64c00640 1 --2- 172.21.15.113:0/962299953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb60151b40 0x7feb6015b380 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0097f0 tx=0x7feb4c01c410 comp rx=0 tx=0).stop 2024-03-04T07:47:40.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb64c00640 1 -- 172.21.15.113:0/962299953 shutdown_connections 2024-03-04T07:47:40.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb64c00640 1 --2- 172.21.15.113:0/962299953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb60151b40 0x7feb6015b380 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:40.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb64c00640 1 --2- 172.21.15.113:0/962299953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb60151760 0x7feb600c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:40.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb64c00640 1 -- 172.21.15.113:0/962299953 >> 172.21.15.113:0/962299953 conn(0x7feb6006e2a0 msgr2=0x7feb6006e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:40.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb64c00640 1 -- 172.21.15.113:0/962299953 shutdown_connections 2024-03-04T07:47:40.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb64c00640 1 -- 172.21.15.113:0/962299953 wait complete. 2024-03-04T07:47:40.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb64c00640 1 Processor -- start 2024-03-04T07:47:40.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.071+0000 7feb64c00640 1 -- start start 2024-03-04T07:47:40.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb64c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb60151760 0x7feb601f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:40.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb64c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb60151b40 0x7feb601575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb64c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb6014eeb0 0x7feb60141400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb64c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60164730 con 0x7feb60151b40 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb64c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb600c1e20 con 0x7feb6014eeb0 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb64c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb601648b0 con 0x7feb60151760 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb60151b40 0x7feb601575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb60151b40 0x7feb601575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34276/0 (socket says 172.21.15.113:34276) 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5ea00640 1 -- 172.21.15.113:0/599631155 learned_addr learned my addr 172.21.15.113:0/599631155 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5f400640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb60151760 0x7feb601f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5fe00640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb6014eeb0 0x7feb60141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5ea00640 1 -- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb60151760 msgr2=0x7feb601f8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5ea00640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb60151760 0x7feb601f8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5ea00640 1 -- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb6014eeb0 msgr2=0x7feb60141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:40.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5ea00640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb6014eeb0 0x7feb60141400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:40.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5ea00640 1 -- 172.21.15.113:0/599631155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb600cb800 con 0x7feb60151b40 2024-03-04T07:47:40.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb5ea00640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb60151b40 0x7feb601575c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7feb4c01a040 tx=0x7feb4c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:40.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb57400640 1 -- 172.21.15.113:0/599631155 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c0b0040 con 0x7feb60151b40 2024-03-04T07:47:40.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb57400640 1 -- 172.21.15.113:0/599631155 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c0b9070 con 0x7feb60151b40 2024-03-04T07:47:40.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb57400640 1 -- 172.21.15.113:0/599631155 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c0b4630 con 0x7feb60151b40 2024-03-04T07:47:40.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb64c00640 1 -- 172.21.15.113:0/599631155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb601fb680 con 0x7feb60151b40 2024-03-04T07:47:40.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb64c00640 1 -- 172.21.15.113:0/599631155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb601fbc40 con 0x7feb60151b40 2024-03-04T07:47:40.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb64c00640 1 -- 172.21.15.113:0/599631155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb60075a80 con 0x7feb60151b40 2024-03-04T07:47:40.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.075+0000 7feb57400640 1 -- 172.21.15.113:0/599631155 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb4c0b47d0 con 0x7feb60151b40 2024-03-04T07:47:40.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.079+0000 7feb57400640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb40066f90 0x7feb40069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:40.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.079+0000 7feb57400640 1 -- 172.21.15.113:0/599631155 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6427+0+0 (secure 0 0 0) 0x7feb4c11efc0 con 0x7feb60151b40 2024-03-04T07:47:40.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.079+0000 7feb5f400640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb40066f90 0x7feb40069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:40.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.079+0000 7feb5f400640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb40066f90 0x7feb40069450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7feb50009af0 tx=0x7feb50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:40.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.087+0000 7feb57400640 1 -- 172.21.15.113:0/599631155 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7feb4c016090 con 0x7feb60151b40 2024-03-04T07:47:40.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.343+0000 7feb64c00640 1 -- 172.21.15.113:0/599631155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb6006c070 con 0x7feb60151b40 2024-03-04T07:47:40.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.343+0000 7feb57400640 1 -- 172.21.15.113:0/599631155 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+6612 (secure 0 0 0) 0x7feb4c0efd00 con 0x7feb60151b40 2024-03-04T07:47:40.342 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:40.342 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5417,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:45:41.748534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5244,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"e","rank":0,"incarnation":32,"state":"up:active","state_seq":26,"addr":"172.21.15.113:6837/742633123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":742633123},{"type":"v1","addr":"172.21.15.113:6837","nonce":742633123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5631,5529,5244]},"id":1}]} 2024-03-04T07:47:40.342 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 56 2024-03-04T07:47:40.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 -- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb40066f90 msgr2=0x7feb40069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:40.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb40066f90 0x7feb40069450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7feb50009af0 tx=0x7feb50008040 comp rx=0 tx=0).stop 2024-03-04T07:47:40.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 -- 172.21.15.113:0/599631155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb60151b40 msgr2=0x7feb601575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:40.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb60151b40 0x7feb601575c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7feb4c01a040 tx=0x7feb4c0191b0 comp rx=0 tx=0).stop 2024-03-04T07:47:40.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 -- 172.21.15.113:0/599631155 shutdown_connections 2024-03-04T07:47:40.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb6014eeb0 0x7feb60141400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:40.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb40066f90 0x7feb40069450 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:40.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb60151b40 0x7feb601575c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:40.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 --2- 172.21.15.113:0/599631155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb60151760 0x7feb601f8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:40.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 -- 172.21.15.113:0/599631155 >> 172.21.15.113:0/599631155 conn(0x7feb6006e2a0 msgr2=0x7feb60074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:40.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 -- 172.21.15.113:0/599631155 shutdown_connections 2024-03-04T07:47:40.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:40.347+0000 7feb64c00640 1 -- 172.21.15.113:0/599631155 wait complete. 2024-03-04T07:47:40.365 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:47:42.367 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:42.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf52400640 1 Processor -- start 2024-03-04T07:47:42.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf52400640 1 -- start start 2024-03-04T07:47:42.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf52400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf4c157ef0 0x7fbf4c0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf52400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbf4c06fb70 0x7fbf4c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf52400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c0c9c50 con 0x7fbf4c070df0 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf52400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c15a9a0 con 0x7fbf4c157ef0 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf52400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c06de30 con 0x7fbf4c06fb70 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf51000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf4c157ef0 0x7fbf4c0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf4be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbf4c070df0 0x7fbf4c15b180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45514/0 (socket says 172.21.15.113:45514) 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf4be00640 1 -- 172.21.15.113:0/3864237788 learned_addr learned my addr 172.21.15.113:0/3864237788 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf51a00640 1 --2- 172.21.15.113:0/3864237788 >> v2:172.21.15.136:3300/0 conn(0x7fbf4c06fb70 0x7fbf4c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf4b400640 1 -- 172.21.15.113:0/3864237788 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895511029 0 0) 0x7fbf4c0c9c50 con 0x7fbf4c070df0 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf4b400640 1 -- 172.21.15.113:0/3864237788 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf34003530 con 0x7fbf4c070df0 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf51000640 1 -- 172.21.15.113:0/3864237788 >> v2:172.21.15.136:3300/0 conn(0x7fbf4c06fb70 msgr2=0x7fbf4c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf51000640 1 --2- 172.21.15.113:0/3864237788 >> v2:172.21.15.136:3300/0 conn(0x7fbf4c06fb70 0x7fbf4c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf51000640 1 -- 172.21.15.113:0/3864237788 >> v1:172.21.15.113:6789/0 conn(0x7fbf4c070df0 legacy=0x7fbf4c15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:42.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf51000640 1 -- 172.21.15.113:0/3864237788 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c06c210 con 0x7fbf4c157ef0 2024-03-04T07:47:42.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.511+0000 7fbf51a00640 1 --2- 172.21.15.113:0/3864237788 >> v2:172.21.15.136:3300/0 conn(0x7fbf4c06fb70 0x7fbf4c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:47:42.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf51000640 1 --2- 172.21.15.113:0/3864237788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c157ef0 0x7fbf4c0c8a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0b3350 tx=0x7fbf4001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91204bd17a4d03f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:42.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf4b400640 1 -- 172.21.15.113:0/3864237788 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf400b1070 con 0x7fbf4c157ef0 2024-03-04T07:47:42.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf4b400640 1 -- 172.21.15.113:0/3864237788 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf400ac5f0 con 0x7fbf4c157ef0 2024-03-04T07:47:42.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf4b400640 1 -- 172.21.15.113:0/3864237788 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf400ac9c0 con 0x7fbf4c157ef0 2024-03-04T07:47:42.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf52400640 1 -- 172.21.15.113:0/3864237788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c157ef0 msgr2=0x7fbf4c0c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:42.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf52400640 1 --2- 172.21.15.113:0/3864237788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c157ef0 0x7fbf4c0c8a90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0b3350 tx=0x7fbf4001c480 comp rx=0 tx=0).stop 2024-03-04T07:47:42.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf52400640 1 -- 172.21.15.113:0/3864237788 shutdown_connections 2024-03-04T07:47:42.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf52400640 1 --2- 172.21.15.113:0/3864237788 >> v2:172.21.15.136:3300/0 conn(0x7fbf4c06fb70 0x7fbf4c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf52400640 1 --2- 172.21.15.113:0/3864237788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c157ef0 0x7fbf4c0c8a90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf52400640 1 -- 172.21.15.113:0/3864237788 >> 172.21.15.113:0/3864237788 conn(0x7fbf4c072f40 msgr2=0x7fbf4c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:42.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf52400640 1 -- 172.21.15.113:0/3864237788 shutdown_connections 2024-03-04T07:47:42.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.515+0000 7fbf52400640 1 -- 172.21.15.113:0/3864237788 wait complete. 2024-03-04T07:47:42.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf52400640 1 Processor -- start 2024-03-04T07:47:42.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf52400640 1 -- start start 2024-03-04T07:47:42.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf52400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c06fb70 0x7fbf4c1f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf52400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf4c070df0 0x7fbf4c0c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf52400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf4c157ef0 0x7fbf4c14f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf52400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c1582d0 con 0x7fbf4c06fb70 2024-03-04T07:47:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf52400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c15b890 con 0x7fbf4c070df0 2024-03-04T07:47:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf52400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c158450 con 0x7fbf4c157ef0 2024-03-04T07:47:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf51000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c06fb70 0x7fbf4c1f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf51000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c06fb70 0x7fbf4c1f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34296/0 (socket says 172.21.15.113:34296) 2024-03-04T07:47:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf51000640 1 -- 172.21.15.113:0/3233858418 learned_addr learned my addr 172.21.15.113:0/3233858418 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf4be00640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf4c070df0 0x7fbf4c0c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf51a00640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf4c157ef0 0x7fbf4c14f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf51000640 1 -- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf4c157ef0 msgr2=0x7fbf4c14f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf51000640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf4c157ef0 0x7fbf4c14f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf51000640 1 -- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf4c070df0 msgr2=0x7fbf4c0c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf51000640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf4c070df0 0x7fbf4c0c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf51000640 1 -- 172.21.15.113:0/3233858418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0c7ac0 con 0x7fbf4c06fb70 2024-03-04T07:47:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf51000640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c06fb70 0x7fbf4c1f7450 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbf40000c10 tx=0x7fbf40019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf48c00640 1 -- 172.21.15.113:0/3233858418 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf400b1040 con 0x7fbf4c06fb70 2024-03-04T07:47:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf48c00640 1 -- 172.21.15.113:0/3233858418 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf400bc070 con 0x7fbf4c06fb70 2024-03-04T07:47:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf48c00640 1 -- 172.21.15.113:0/3233858418 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40006e20 con 0x7fbf4c06fb70 2024-03-04T07:47:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf52400640 1 -- 172.21.15.113:0/3233858418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c0c7da0 con 0x7fbf4c06fb70 2024-03-04T07:47:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.519+0000 7fbf52400640 1 -- 172.21.15.113:0/3233858418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c0c8360 con 0x7fbf4c06fb70 2024-03-04T07:47:42.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.523+0000 7fbf52400640 1 -- 172.21.15.113:0/3233858418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4c075a80 con 0x7fbf4c06fb70 2024-03-04T07:47:42.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.523+0000 7fbf48c00640 1 -- 172.21.15.113:0/3233858418 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf400b5650 con 0x7fbf4c06fb70 2024-03-04T07:47:42.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.523+0000 7fbf48c00640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf2c066f90 0x7fbf2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.523+0000 7fbf48c00640 1 -- 172.21.15.113:0/3233858418 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fbf4011ed10 con 0x7fbf4c06fb70 2024-03-04T07:47:42.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.523+0000 7fbf4be00640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf2c066f90 0x7fbf2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.523+0000 7fbf4be00640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf2c066f90 0x7fbf2c069450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0cb420 tx=0x7fbf38009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:42.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.527+0000 7fbf48c00640 1 -- 172.21.15.113:0/3233858418 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbf400a9070 con 0x7fbf4c06fb70 2024-03-04T07:47:42.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.719+0000 7fbf52400640 1 -- 172.21.15.113:0/3233858418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf4c156540 con 0x7fbf4c06fb70 2024-03-04T07:47:42.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.719+0000 7fbf48c00640 1 -- 172.21.15.113:0/3233858418 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5815 (secure 0 0 0) 0x7fbf40016070 con 0x7fbf4c06fb70 2024-03-04T07:47:42.717 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:42.717 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:47:40.684284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:47:42.718 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 57 2024-03-04T07:47:42.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 -- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf2c066f90 msgr2=0x7fbf2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf2c066f90 0x7fbf2c069450 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0cb420 tx=0x7fbf38009040 comp rx=0 tx=0).stop 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 -- 172.21.15.113:0/3233858418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c06fb70 msgr2=0x7fbf4c1f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c06fb70 0x7fbf4c1f7450 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbf40000c10 tx=0x7fbf40019220 comp rx=0 tx=0).stop 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 -- 172.21.15.113:0/3233858418 shutdown_connections 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbf4c157ef0 0x7fbf4c14f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbf4c070df0 0x7fbf4c0c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbf2c066f90 0x7fbf2c069450 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 --2- 172.21.15.113:0/3233858418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf4c06fb70 0x7fbf4c1f7450 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 -- 172.21.15.113:0/3233858418 >> 172.21.15.113:0/3233858418 conn(0x7fbf4c072f40 msgr2=0x7fbf4c1595d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 -- 172.21.15.113:0/3233858418 shutdown_connections 2024-03-04T07:47:42.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.723+0000 7fbf52400640 1 -- 172.21.15.113:0/3233858418 wait complete. 2024-03-04T07:47:42.739 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-03-04T07:47:42.740 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:47:42.740 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:42.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f1969800640 1 Processor -- start 2024-03-04T07:47:42.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f1969800640 1 -- start start 2024-03-04T07:47:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f1969800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f19641527f0 0x7f1964070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f1969800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19640c8200 0x7f1964072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f1969800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19641591f0 con 0x7f1964152bd0 2024-03-04T07:47:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f1969800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1964071df0 con 0x7f19640c8200 2024-03-04T07:47:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f1969800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19641597b0 con 0x7f19641527f0 2024-03-04T07:47:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f195be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19640c8200 0x7f1964072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f195be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19640c8200 0x7f1964072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34312/0 (socket says 172.21.15.113:34312) 2024-03-04T07:47:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f195be00640 1 -- 172.21.15.113:0/2473941887 learned_addr learned my addr 172.21.15.113:0/2473941887 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f1962a00640 1 --2- 172.21.15.113:0/2473941887 >> v2:172.21.15.136:3300/0 conn(0x7f19641527f0 0x7f1964070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f195be00640 1 -- 172.21.15.113:0/2473941887 >> v2:172.21.15.136:3300/0 conn(0x7f19641527f0 msgr2=0x7f1964070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f195be00640 1 --2- 172.21.15.113:0/2473941887 >> v2:172.21.15.136:3300/0 conn(0x7f19641527f0 0x7f1964070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f195be00640 1 -- 172.21.15.113:0/2473941887 >> v1:172.21.15.136:6789/0 conn(0x7f1964152bd0 legacy=0x7f19641587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:42.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.883+0000 7f195be00640 1 -- 172.21.15.113:0/2473941887 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19641594d0 con 0x7f19640c8200 2024-03-04T07:47:42.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f195be00640 1 --2- 172.21.15.113:0/2473941887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19640c8200 0x7f1964072280 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f196406f400 tx=0x7f194c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0ea9a6070e728 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:42.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1962000640 1 -- 172.21.15.113:0/2473941887 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c0ab430 con 0x7f19640c8200 2024-03-04T07:47:42.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1962000640 1 -- 172.21.15.113:0/2473941887 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c0b0070 con 0x7f19640c8200 2024-03-04T07:47:42.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1962000640 1 -- 172.21.15.113:0/2473941887 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c0ab7f0 con 0x7f19640c8200 2024-03-04T07:47:42.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1969800640 1 -- 172.21.15.113:0/2473941887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19640c8200 msgr2=0x7f1964072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:42.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1969800640 1 --2- 172.21.15.113:0/2473941887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19640c8200 0x7f1964072280 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f196406f400 tx=0x7f194c01c520 comp rx=0 tx=0).stop 2024-03-04T07:47:42.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1969800640 1 -- 172.21.15.113:0/2473941887 shutdown_connections 2024-03-04T07:47:42.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1969800640 1 --2- 172.21.15.113:0/2473941887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19640c8200 0x7f1964072280 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1969800640 1 --2- 172.21.15.113:0/2473941887 >> v2:172.21.15.136:3300/0 conn(0x7f19641527f0 0x7f1964070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1969800640 1 -- 172.21.15.113:0/2473941887 >> 172.21.15.113:0/2473941887 conn(0x7f196406e820 msgr2=0x7f196406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:42.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1969800640 1 -- 172.21.15.113:0/2473941887 shutdown_connections 2024-03-04T07:47:42.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.887+0000 7f1969800640 1 -- 172.21.15.113:0/2473941887 wait complete. 2024-03-04T07:47:42.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1969800640 1 Processor -- start 2024-03-04T07:47:42.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1969800640 1 -- start start 2024-03-04T07:47:42.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1969800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19640c8200 0x7f1964148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1969800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19641527f0 0x7f19641f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1969800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1964152bd0 0x7f196415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1969800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1964165160 con 0x7f1964152bd0 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1969800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19640727c0 con 0x7f19640c8200 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1969800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19641652e0 con 0x7f19641527f0 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1963400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1964152bd0 0x7f196415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1963400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1964152bd0 0x7f196415ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34324/0 (socket says 172.21.15.113:34324) 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1962a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19640c8200 0x7f1964148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1963400640 1 -- 172.21.15.113:0/703443806 learned_addr learned my addr 172.21.15.113:0/703443806 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f195be00640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19641527f0 0x7f19641f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1963400640 1 -- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19641527f0 msgr2=0x7f19641f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1963400640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19641527f0 0x7f19641f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1963400640 1 -- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19640c8200 msgr2=0x7f1964148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:42.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1963400640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19640c8200 0x7f1964148f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:42.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.891+0000 7f1963400640 1 -- 172.21.15.113:0/703443806 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196415a280 con 0x7f1964152bd0 2024-03-04T07:47:42.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f1963400640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1964152bd0 0x7f196415ac00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c760 tx=0x7f195c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:42.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f195aa00640 1 -- 172.21.15.113:0/703443806 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c003b80 con 0x7f1964152bd0 2024-03-04T07:47:42.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f195aa00640 1 -- 172.21.15.113:0/703443806 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c003d20 con 0x7f1964152bd0 2024-03-04T07:47:42.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f1969800640 1 -- 172.21.15.113:0/703443806 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1964159ba0 con 0x7f1964152bd0 2024-03-04T07:47:42.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f195aa00640 1 -- 172.21.15.113:0/703443806 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c00aa30 con 0x7f1964152bd0 2024-03-04T07:47:42.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f1969800640 1 -- 172.21.15.113:0/703443806 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196414b850 con 0x7f1964152bd0 2024-03-04T07:47:42.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f1969800640 1 -- 172.21.15.113:0/703443806 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19641f16b0 con 0x7f1964152bd0 2024-03-04T07:47:42.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f195aa00640 1 -- 172.21.15.113:0/703443806 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f195c00abd0 con 0x7f1964152bd0 2024-03-04T07:47:42.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f195aa00640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1940066f60 0x7f1940069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:42.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f195aa00640 1 -- 172.21.15.113:0/703443806 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f195c085140 con 0x7f1964152bd0 2024-03-04T07:47:42.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.895+0000 7f1962a00640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1940066f60 0x7f1940069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:42.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.899+0000 7f1962a00640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1940066f60 0x7f1940069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1954009b60 tx=0x7f1954009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:42.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:42.899+0000 7f195aa00640 1 -- 172.21.15.113:0/703443806 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f195c055730 con 0x7f1964152bd0 2024-03-04T07:47:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.131+0000 7f1969800640 1 -- 172.21.15.113:0/703443806 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1964159990 con 0x7f1964152bd0 2024-03-04T07:47:43.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f195aa00640 1 -- 172.21.15.113:0/703443806 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f195c0597d0 con 0x7f1964152bd0 2024-03-04T07:47:43.131 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:43.131 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:47:40.684284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:47:43.132 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 57 2024-03-04T07:47:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 -- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1940066f60 msgr2=0x7f1940069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1940066f60 0x7f1940069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1954009b60 tx=0x7f1954009190 comp rx=0 tx=0).stop 2024-03-04T07:47:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 -- 172.21.15.113:0/703443806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1964152bd0 msgr2=0x7f196415ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1964152bd0 0x7f196415ac00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c760 tx=0x7f195c00cc20 comp rx=0 tx=0).stop 2024-03-04T07:47:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 -- 172.21.15.113:0/703443806 shutdown_connections 2024-03-04T07:47:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1940066f60 0x7f1940069420 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1964152bd0 0x7f196415ac00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19641527f0 0x7f19641f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:43.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 --2- 172.21.15.113:0/703443806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19640c8200 0x7f1964148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:43.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 -- 172.21.15.113:0/703443806 >> 172.21.15.113:0/703443806 conn(0x7f196406e820 msgr2=0x7f1964158440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:43.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 -- 172.21.15.113:0/703443806 shutdown_connections 2024-03-04T07:47:43.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:43.135+0000 7f1969800640 1 -- 172.21.15.113:0/703443806 wait complete. 2024-03-04T07:47:43.153 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 57, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 57, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5417": { "addr": "172.21.15.136:6835/1691762848", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1691762848, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1691762848, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5417, "incarnation": 57, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_5529": { "addr": "172.21.15.113:6836/2299268553", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2299268553, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2299268553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5529, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 43 }, "gid_5631": { "addr": "172.21.15.113:6833/810789479", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 810789479, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 810789479, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5631, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 56, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:47:40.684284+0000", "qdb_cluster": [ 5529, 5631 ], "qdb_leader": 5529, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5417, "mds_1": 5529, "mds_2": 5631 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:47:45.154 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:45.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.295+0000 7f8171a00640 1 Processor -- start 2024-03-04T07:47:45.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.295+0000 7f8171a00640 1 -- start start 2024-03-04T07:47:45.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.295+0000 7f8171a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f816c154c00 0x7f816c0c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:45.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.295+0000 7f8171a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f816c0c7700 0x7f816c15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:45.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.295+0000 7f8171a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816c0c7d80 con 0x7f816c154fe0 2024-03-04T07:47:45.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.295+0000 7f8171a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f816c15efb0 con 0x7f816c154c00 2024-03-04T07:47:45.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.295+0000 7f8171a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f816c0c6b70 con 0x7f816c0c7700 2024-03-04T07:47:45.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f816c154c00 0x7f816c0c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:45.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f816c154c00 0x7f816c0c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34334/0 (socket says 172.21.15.113:34334) 2024-03-04T07:47:45.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816aa00640 1 -- 172.21.15.113:0/1975258655 learned_addr learned my addr 172.21.15.113:0/1975258655 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:45.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f8162000640 1 --1- 172.21.15.113:0/1975258655 >> v1:172.21.15.113:6789/0 conn(0x7f816c154fe0 0x7f816c0c1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45520/0 (socket says 172.21.15.113:45520) 2024-03-04T07:47:45.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816b400640 1 --2- 172.21.15.113:0/1975258655 >> v2:172.21.15.136:3300/0 conn(0x7f816c0c7700 0x7f816c15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:45.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816aa00640 1 -- 172.21.15.113:0/1975258655 >> v2:172.21.15.136:3300/0 conn(0x7f816c0c7700 msgr2=0x7f816c15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:45.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816aa00640 1 --2- 172.21.15.113:0/1975258655 >> v2:172.21.15.136:3300/0 conn(0x7f816c0c7700 0x7f816c15abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:45.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816aa00640 1 -- 172.21.15.113:0/1975258655 >> v1:172.21.15.113:6789/0 conn(0x7f816c154fe0 legacy=0x7f816c0c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:45.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816aa00640 1 -- 172.21.15.113:0/1975258655 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c0c8060 con 0x7f816c154c00 2024-03-04T07:47:45.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816aa00640 1 --2- 172.21.15.113:0/1975258655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c154c00 0x7f816c0c1190 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f816c06ca40 tx=0x7f815c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82292db5324d2660 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:45.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816a000640 1 -- 172.21.15.113:0/1975258655 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c0b2070 con 0x7f816c154c00 2024-03-04T07:47:45.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816a000640 1 -- 172.21.15.113:0/1975258655 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c0ad680 con 0x7f816c154c00 2024-03-04T07:47:45.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.299+0000 7f816a000640 1 -- 172.21.15.113:0/1975258655 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c0ada50 con 0x7f816c154c00 2024-03-04T07:47:45.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.303+0000 7f8171a00640 1 -- 172.21.15.113:0/1975258655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c154c00 msgr2=0x7f816c0c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:45.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.303+0000 7f8171a00640 1 --2- 172.21.15.113:0/1975258655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c154c00 0x7f816c0c1190 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f816c06ca40 tx=0x7f815c01c490 comp rx=0 tx=0).stop 2024-03-04T07:47:45.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.303+0000 7f8171a00640 1 -- 172.21.15.113:0/1975258655 shutdown_connections 2024-03-04T07:47:45.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.303+0000 7f8171a00640 1 --2- 172.21.15.113:0/1975258655 >> v2:172.21.15.136:3300/0 conn(0x7f816c0c7700 0x7f816c15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:45.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.303+0000 7f8171a00640 1 --2- 172.21.15.113:0/1975258655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c154c00 0x7f816c0c1190 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:45.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.303+0000 7f8171a00640 1 -- 172.21.15.113:0/1975258655 >> 172.21.15.113:0/1975258655 conn(0x7f816c06e820 msgr2=0x7f816c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:45.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.303+0000 7f8171a00640 1 -- 172.21.15.113:0/1975258655 shutdown_connections 2024-03-04T07:47:45.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.303+0000 7f8171a00640 1 -- 172.21.15.113:0/1975258655 wait complete. 2024-03-04T07:47:45.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.303+0000 7f8171a00640 1 Processor -- start 2024-03-04T07:47:45.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.303+0000 7f8171a00640 1 -- start start 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8171a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c0c7700 0x7f816c15c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8171a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f816c154c00 0x7f816c0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8171a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c154fe0 0x7f816c1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8171a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c0c23e0 con 0x7f816c0c7700 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8171a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c0c2260 con 0x7f816c154fe0 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8171a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816c167c10 con 0x7f816c154c00 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f816aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c0c7700 0x7f816c15c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f816aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c0c7700 0x7f816c15c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34346/0 (socket says 172.21.15.113:34346) 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f816aa00640 1 -- 172.21.15.113:0/4071996329 learned_addr learned my addr 172.21.15.113:0/4071996329 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8162000640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f816c154c00 0x7f816c0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f816b400640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c154fe0 0x7f816c1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f816aa00640 1 -- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f816c154c00 msgr2=0x7f816c0cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:45.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f816aa00640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f816c154c00 0x7f816c0cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:45.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f816aa00640 1 -- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c154fe0 msgr2=0x7f816c1570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:45.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f816aa00640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c154fe0 0x7f816c1570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:45.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f816aa00640 1 -- 172.21.15.113:0/4071996329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c15db80 con 0x7f816c0c7700 2024-03-04T07:47:45.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f816aa00640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c0c7700 0x7f816c15c640 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f815c0190a0 tx=0x7f815c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:45.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8163400640 1 -- 172.21.15.113:0/4071996329 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c0b2040 con 0x7f816c0c7700 2024-03-04T07:47:45.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8163400640 1 -- 172.21.15.113:0/4071996329 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c0adaf0 con 0x7f816c0c7700 2024-03-04T07:47:45.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8163400640 1 -- 172.21.15.113:0/4071996329 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c0b8420 con 0x7f816c0c7700 2024-03-04T07:47:45.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8171a00640 1 -- 172.21.15.113:0/4071996329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816c15de20 con 0x7f816c0c7700 2024-03-04T07:47:45.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8171a00640 1 -- 172.21.15.113:0/4071996329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c15e3e0 con 0x7f816c0c7700 2024-03-04T07:47:45.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.307+0000 7f8171a00640 1 -- 172.21.15.113:0/4071996329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816c075c00 con 0x7f816c0c7700 2024-03-04T07:47:45.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.311+0000 7f8163400640 1 -- 172.21.15.113:0/4071996329 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f815c0c0030 con 0x7f816c0c7700 2024-03-04T07:47:45.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.311+0000 7f8163400640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8148066f60 0x7f8148069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:45.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.311+0000 7f8163400640 1 -- 172.21.15.113:0/4071996329 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f815c11ee40 con 0x7f816c0c7700 2024-03-04T07:47:45.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.311+0000 7f8162000640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8148066f60 0x7f8148069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:45.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.311+0000 7f8162000640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8148066f60 0x7f8148069420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8154007fd0 tx=0x7f8154005b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:45.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.315+0000 7f8163400640 1 -- 172.21.15.113:0/4071996329 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f815c0c7020 con 0x7f816c0c7700 2024-03-04T07:47:45.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.507+0000 7f8171a00640 1 -- 172.21.15.113:0/4071996329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f816c150580 con 0x7f816c0c7700 2024-03-04T07:47:45.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.507+0000 7f8163400640 1 -- 172.21.15.113:0/4071996329 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f815c016110 con 0x7f816c0c7700 2024-03-04T07:47:45.506 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:45.506 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:47:40.684284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:47:45.506 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 57 2024-03-04T07:47:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 -- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8148066f60 msgr2=0x7f8148069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8148066f60 0x7f8148069420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8154007fd0 tx=0x7f8154005b80 comp rx=0 tx=0).stop 2024-03-04T07:47:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 -- 172.21.15.113:0/4071996329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c0c7700 msgr2=0x7f816c15c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c0c7700 0x7f816c15c640 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f815c0190a0 tx=0x7f815c01acd0 comp rx=0 tx=0).stop 2024-03-04T07:47:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 -- 172.21.15.113:0/4071996329 shutdown_connections 2024-03-04T07:47:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f816c154fe0 0x7f816c1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f816c154c00 0x7f816c0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8148066f60 0x7f8148069420 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:45.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 --2- 172.21.15.113:0/4071996329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f816c0c7700 0x7f816c15c640 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:45.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 -- 172.21.15.113:0/4071996329 >> 172.21.15.113:0/4071996329 conn(0x7f816c06e820 msgr2=0x7f816c074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:45.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 -- 172.21.15.113:0/4071996329 shutdown_connections 2024-03-04T07:47:45.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:45.511+0000 7f8171a00640 1 -- 172.21.15.113:0/4071996329 wait complete. 2024-03-04T07:47:45.536 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:47:47.537 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.699+0000 7f3d5f200640 1 Processor -- start 2024-03-04T07:47:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.699+0000 7f3d5f200640 1 -- start start 2024-03-04T07:47:47.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5f200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d60154d00 0x7f3d60071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:47.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5f200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3d600c9640 0x7f3d600c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:47.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5f200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d600c2e40 0x7f3d6015a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5f200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d600c8630 con 0x7f3d60154d00 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5f200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d601550e0 con 0x7f3d600c9640 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5f200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6015b720 con 0x7f3d600c2e40 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d60154d00 0x7f3d60071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d60154d00 0x7f3d60071a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44524/0 (socket says 172.21.15.113:44524) 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5de00640 1 -- 172.21.15.113:0/517269331 learned_addr learned my addr 172.21.15.113:0/517269331 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5d400640 1 --2- 172.21.15.113:0/517269331 >> v2:172.21.15.136:3300/0 conn(0x7f3d600c9640 0x7f3d600c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5e800640 1 --2- 172.21.15.113:0/517269331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d600c2e40 0x7f3d6015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5de00640 1 -- 172.21.15.113:0/517269331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d600c2e40 msgr2=0x7f3d6015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5de00640 1 --2- 172.21.15.113:0/517269331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d600c2e40 0x7f3d6015a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5de00640 1 -- 172.21.15.113:0/517269331 >> v2:172.21.15.136:3300/0 conn(0x7f3d600c9640 msgr2=0x7f3d600c2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5de00640 1 --2- 172.21.15.113:0/517269331 >> v2:172.21.15.136:3300/0 conn(0x7f3d600c9640 0x7f3d600c2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5de00640 1 -- 172.21.15.113:0/517269331 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600c8880 con 0x7f3d60154d00 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5d400640 1 --2- 172.21.15.113:0/517269331 >> v2:172.21.15.136:3300/0 conn(0x7f3d600c9640 0x7f3d600c2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:47:47.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.703+0000 7f3d5e800640 1 --2- 172.21.15.113:0/517269331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d600c2e40 0x7f3d6015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:47:47.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5de00640 1 --2- 172.21.15.113:0/517269331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d60154d00 0x7f3d60071a30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3d600c2850 tx=0x7f3d5001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c12040f954a1d8b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5ca00640 1 -- 172.21.15.113:0/517269331 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500b5070 con 0x7f3d60154d00 2024-03-04T07:47:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5ca00640 1 -- 172.21.15.113:0/517269331 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d500a89d0 con 0x7f3d60154d00 2024-03-04T07:47:47.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5ca00640 1 -- 172.21.15.113:0/517269331 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500a8da0 con 0x7f3d60154d00 2024-03-04T07:47:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5f200640 1 -- 172.21.15.113:0/517269331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d60154d00 msgr2=0x7f3d60071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5f200640 1 --2- 172.21.15.113:0/517269331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d60154d00 0x7f3d60071a30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3d600c2850 tx=0x7f3d5001ed10 comp rx=0 tx=0).stop 2024-03-04T07:47:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5f200640 1 -- 172.21.15.113:0/517269331 shutdown_connections 2024-03-04T07:47:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5f200640 1 --2- 172.21.15.113:0/517269331 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d600c2e40 0x7f3d6015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5f200640 1 --2- 172.21.15.113:0/517269331 >> v2:172.21.15.136:3300/0 conn(0x7f3d600c9640 0x7f3d600c2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5f200640 1 --2- 172.21.15.113:0/517269331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d60154d00 0x7f3d60071a30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5f200640 1 -- 172.21.15.113:0/517269331 >> 172.21.15.113:0/517269331 conn(0x7f3d600c9a30 msgr2=0x7f3d600c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5f200640 1 -- 172.21.15.113:0/517269331 shutdown_connections 2024-03-04T07:47:47.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.707+0000 7f3d5f200640 1 -- 172.21.15.113:0/517269331 wait complete. 2024-03-04T07:47:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5f200640 1 Processor -- start 2024-03-04T07:47:47.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5f200640 1 -- start start 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d600c2e40 0x7f3d600cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5f200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d60154d00 0x7f3d601f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d601f4970 0x7f3d601e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60071f70 con 0x7f3d600c2e40 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600c8e40 con 0x7f3d601f4970 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5f200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600c3500 con 0x7f3d60154d00 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d600c2e40 0x7f3d600cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d600c2e40 0x7f3d600cc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44528/0 (socket says 172.21.15.113:44528) 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5de00640 1 -- 172.21.15.113:0/1479605713 learned_addr learned my addr 172.21.15.113:0/1479605713 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5e800640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d601f4970 0x7f3d601e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5d400640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d60154d00 0x7f3d601f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5de00640 1 -- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d60154d00 msgr2=0x7f3d601f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:47.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5de00640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d60154d00 0x7f3d601f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5de00640 1 -- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d601f4970 msgr2=0x7f3d601e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:47.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5de00640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d601f4970 0x7f3d601e6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5de00640 1 -- 172.21.15.113:0/1479605713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6014f900 con 0x7f3d600c2e40 2024-03-04T07:47:47.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5de00640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d600c2e40 0x7f3d600cc450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3d5001d3c0 tx=0x7f3d5001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d56000640 1 -- 172.21.15.113:0/1479605713 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500b5070 con 0x7f3d600c2e40 2024-03-04T07:47:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d56000640 1 -- 172.21.15.113:0/1479605713 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d500b0a50 con 0x7f3d600c2e40 2024-03-04T07:47:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d56000640 1 -- 172.21.15.113:0/1479605713 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500b0e20 con 0x7f3d600c2e40 2024-03-04T07:47:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5f200640 1 -- 172.21.15.113:0/1479605713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6014fbe0 con 0x7f3d600c2e40 2024-03-04T07:47:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.711+0000 7f3d5f200640 1 -- 172.21.15.113:0/1479605713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d601501a0 con 0x7f3d600c2e40 2024-03-04T07:47:47.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.715+0000 7f3d5f200640 1 -- 172.21.15.113:0/1479605713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d601503e0 con 0x7f3d600c2e40 2024-03-04T07:47:47.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.715+0000 7f3d56000640 1 -- 172.21.15.113:0/1479605713 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d500c2060 con 0x7f3d600c2e40 2024-03-04T07:47:47.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.715+0000 7f3d56000640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d40066f60 0x7f3d40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:47.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.715+0000 7f3d56000640 1 -- 172.21.15.113:0/1479605713 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f3d501223f0 con 0x7f3d600c2e40 2024-03-04T07:47:47.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.715+0000 7f3d5d400640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d40066f60 0x7f3d40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:47.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.715+0000 7f3d5d400640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d40066f60 0x7f3d40069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3d60150670 tx=0x7f3d4c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:47.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.719+0000 7f3d56000640 1 -- 172.21.15.113:0/1479605713 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3d601503e0 con 0x7f3d600c2e40 2024-03-04T07:47:47.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.931+0000 7f3d5f200640 1 -- 172.21.15.113:0/1479605713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d601f3d00 con 0x7f3d600c2e40 2024-03-04T07:47:47.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.931+0000 7f3d56000640 1 -- 172.21.15.113:0/1479605713 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f3d50018870 con 0x7f3d600c2e40 2024-03-04T07:47:47.930 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:47.931 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:47:40.684284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:47:47.931 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 57 2024-03-04T07:47:47.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 -- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d40066f60 msgr2=0x7f3d40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:47.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d40066f60 0x7f3d40069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3d60150670 tx=0x7f3d4c008040 comp rx=0 tx=0).stop 2024-03-04T07:47:47.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 -- 172.21.15.113:0/1479605713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d600c2e40 msgr2=0x7f3d600cc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:47.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d600c2e40 0x7f3d600cc450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3d5001d3c0 tx=0x7f3d5001bab0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 -- 172.21.15.113:0/1479605713 shutdown_connections 2024-03-04T07:47:47.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d601f4970 0x7f3d601e6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d60154d00 0x7f3d601f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d40066f60 0x7f3d40069420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 --2- 172.21.15.113:0/1479605713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d600c2e40 0x7f3d600cc450 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:47.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 -- 172.21.15.113:0/1479605713 >> 172.21.15.113:0/1479605713 conn(0x7f3d600c9a30 msgr2=0x7f3d6006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:47.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 -- 172.21.15.113:0/1479605713 shutdown_connections 2024-03-04T07:47:47.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:47.935+0000 7f3d5f200640 1 -- 172.21.15.113:0/1479605713 wait complete. 2024-03-04T07:47:47.952 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:47:49.953 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:50.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.095+0000 7f2593200640 1 Processor -- start 2024-03-04T07:47:50.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.095+0000 7f2593200640 1 -- start start 2024-03-04T07:47:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2593200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2594157ef0 0x7f25940c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2593200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2594070df0 0x7f259415b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2593200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25940c9c50 con 0x7f259415b6c0 2024-03-04T07:47:50.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2593200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259415a9a0 con 0x7f2594157ef0 2024-03-04T07:47:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2593200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259406de30 con 0x7f2594070df0 2024-03-04T07:47:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2591e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2594157ef0 0x7f25940c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2592800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f259415b6c0 0x7f25940cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51640/0 (socket says 172.21.15.113:51640) 2024-03-04T07:47:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2592800640 1 -- 172.21.15.113:0/3162408888 learned_addr learned my addr 172.21.15.113:0/3162408888 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:47:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f258be00640 1 --2- 172.21.15.113:0/3162408888 >> v2:172.21.15.136:3300/0 conn(0x7f2594070df0 0x7f259415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2591e00640 1 -- 172.21.15.113:0/3162408888 >> v2:172.21.15.136:3300/0 conn(0x7f2594070df0 msgr2=0x7f259415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2591e00640 1 --2- 172.21.15.113:0/3162408888 >> v2:172.21.15.136:3300/0 conn(0x7f2594070df0 0x7f259415b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2591e00640 1 -- 172.21.15.113:0/3162408888 >> v1:172.21.15.113:6789/0 conn(0x7f259415b6c0 legacy=0x7f25940cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:50.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.099+0000 7f2591e00640 1 -- 172.21.15.113:0/3162408888 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259406c210 con 0x7f2594157ef0 2024-03-04T07:47:50.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2591e00640 1 --2- 172.21.15.113:0/3162408888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 0x7f25940c8a90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f25940ceb20 tx=0x7f258401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e18d8928a034b06 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:50.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2591400640 1 -- 172.21.15.113:0/3162408888 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840b1070 con 0x7f2594157ef0 2024-03-04T07:47:50.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2591400640 1 -- 172.21.15.113:0/3162408888 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25840ac5f0 con 0x7f2594157ef0 2024-03-04T07:47:50.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2591400640 1 -- 172.21.15.113:0/3162408888 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25840ac9c0 con 0x7f2594157ef0 2024-03-04T07:47:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 -- 172.21.15.113:0/3162408888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 msgr2=0x7f25940c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 --2- 172.21.15.113:0/3162408888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 0x7f25940c8a90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f25940ceb20 tx=0x7f258401c480 comp rx=0 tx=0).stop 2024-03-04T07:47:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 -- 172.21.15.113:0/3162408888 shutdown_connections 2024-03-04T07:47:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 --2- 172.21.15.113:0/3162408888 >> v2:172.21.15.136:3300/0 conn(0x7f2594070df0 0x7f259415b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 --2- 172.21.15.113:0/3162408888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 0x7f25940c8a90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 -- 172.21.15.113:0/3162408888 >> 172.21.15.113:0/3162408888 conn(0x7f2594072f40 msgr2=0x7f25940706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 -- 172.21.15.113:0/3162408888 shutdown_connections 2024-03-04T07:47:50.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 -- 172.21.15.113:0/3162408888 wait complete. 2024-03-04T07:47:50.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 Processor -- start 2024-03-04T07:47:50.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 -- start start 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2594070df0 0x7f259414a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 0x7f2594141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.103+0000 7f2593200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f259415b6c0 0x7f2594142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f2593200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594158450 con 0x7f2594157ef0 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f2593200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941582d0 con 0x7f2594070df0 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f2593200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25940ce6f0 con 0x7f259415b6c0 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 0x7f2594141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 0x7f2594141ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44540/0 (socket says 172.21.15.113:44540) 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258be00640 1 -- 172.21.15.113:0/945836608 learned_addr learned my addr 172.21.15.113:0/945836608 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f2591e00640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2594070df0 0x7f259414a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:50.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f2592800640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f259415b6c0 0x7f2594142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258be00640 1 -- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f259415b6c0 msgr2=0x7f2594142000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258be00640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f259415b6c0 0x7f2594142000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258be00640 1 -- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2594070df0 msgr2=0x7f259414a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258be00640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2594070df0 0x7f259414a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258be00640 1 -- 172.21.15.113:0/945836608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2594151c30 con 0x7f2594157ef0 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258be00640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 0x7f2594141ac0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f257c00e930 tx=0x7f257c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258a000640 1 -- 172.21.15.113:0/945836608 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c017070 con 0x7f2594157ef0 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258a000640 1 -- 172.21.15.113:0/945836608 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c00ca00 con 0x7f2594157ef0 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258a000640 1 -- 172.21.15.113:0/945836608 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00cdb0 con 0x7f2594157ef0 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f2593200640 1 -- 172.21.15.113:0/945836608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2594151f10 con 0x7f2594157ef0 2024-03-04T07:47:50.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f2593200640 1 -- 172.21.15.113:0/945836608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25940cdec0 con 0x7f2594157ef0 2024-03-04T07:47:50.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f2593200640 1 -- 172.21.15.113:0/945836608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2594075a80 con 0x7f2594157ef0 2024-03-04T07:47:50.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.107+0000 7f258a000640 1 -- 172.21.15.113:0/945836608 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f257c01a070 con 0x7f2594157ef0 2024-03-04T07:47:50.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.111+0000 7f258a000640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2570066f60 0x7f2570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:50.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.111+0000 7f258a000640 1 -- 172.21.15.113:0/945836608 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f257c0857e0 con 0x7f2594157ef0 2024-03-04T07:47:50.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.111+0000 7f2591e00640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2570066f60 0x7f2570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:50.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.111+0000 7f2591e00640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2570066f60 0x7f2570069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f25840ae000 tx=0x7f2584008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:50.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.119+0000 7f258a000640 1 -- 172.21.15.113:0/945836608 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f257c055e00 con 0x7f2594157ef0 2024-03-04T07:47:50.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.339+0000 7f2593200640 1 -- 172.21.15.113:0/945836608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2594152320 con 0x7f2594157ef0 2024-03-04T07:47:50.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.339+0000 7f258a000640 1 -- 172.21.15.113:0/945836608 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f257c059ea0 con 0x7f2594157ef0 2024-03-04T07:47:50.337 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 57 2024-03-04T07:47:50.338 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:50.338 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:47:40.684284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:47:50.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 -- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2570066f60 msgr2=0x7f2570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:50.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2570066f60 0x7f2570069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f25840ae000 tx=0x7f2584008040 comp rx=0 tx=0).stop 2024-03-04T07:47:50.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 -- 172.21.15.113:0/945836608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 msgr2=0x7f2594141ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:50.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 0x7f2594141ac0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f257c00e930 tx=0x7f257c00edf0 comp rx=0 tx=0).stop 2024-03-04T07:47:50.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 -- 172.21.15.113:0/945836608 shutdown_connections 2024-03-04T07:47:50.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f259415b6c0 0x7f2594142000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:50.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2570066f60 0x7f2570069420 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:50.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2594157ef0 0x7f2594141ac0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:50.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 --2- 172.21.15.113:0/945836608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2594070df0 0x7f259414a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:50.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 -- 172.21.15.113:0/945836608 >> 172.21.15.113:0/945836608 conn(0x7f2594072f40 msgr2=0x7f259414ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:50.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 -- 172.21.15.113:0/945836608 shutdown_connections 2024-03-04T07:47:50.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:50.343+0000 7f2593200640 1 -- 172.21.15.113:0/945836608 wait complete. 2024-03-04T07:47:50.359 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:47:52.361 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:52.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.531+0000 7f43cfe00640 1 Processor -- start 2024-03-04T07:47:52.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.531+0000 7f43cfe00640 1 -- start start 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.531+0000 7f43cfe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f43d0151840 0x7f43d00c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.531+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43d006d140 0x7f43d0157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.531+0000 7f43cfe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d0157da0 con 0x7f43d0151c20 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.531+0000 7f43cfe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d00c3440 con 0x7f43d0151840 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.531+0000 7f43cfe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0158080 con 0x7f43d006d140 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43c6000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f43d0151c20 0x7f43d006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51648/0 (socket says 172.21.15.113:51648) 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43c6000640 1 -- 172.21.15.113:0/2853954852 learned_addr learned my addr 172.21.15.113:0/2853954852 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43cea00640 1 --2- 172.21.15.113:0/2853954852 >> v2:172.21.15.136:3300/0 conn(0x7f43d0151840 0x7f43d00c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43cf400640 1 --2- 172.21.15.113:0/2853954852 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43d006d140 0x7f43d0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43ce000640 1 -- 172.21.15.113:0/2853954852 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72900145 0 0) 0x7f43d0157da0 con 0x7f43d0151c20 2024-03-04T07:47:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43ce000640 1 -- 172.21.15.113:0/2853954852 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b4003530 con 0x7f43d0151c20 2024-03-04T07:47:52.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43ce000640 1 -- 172.21.15.113:0/2853954852 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 399956824 0 0) 0x7f43b4003530 con 0x7f43d0151c20 2024-03-04T07:47:52.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43cea00640 1 -- 172.21.15.113:0/2853954852 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43d006d140 msgr2=0x7f43d0157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:52.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43cea00640 1 --2- 172.21.15.113:0/2853954852 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43d006d140 0x7f43d0157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:52.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43cea00640 1 -- 172.21.15.113:0/2853954852 >> v1:172.21.15.113:6789/0 conn(0x7f43d0151c20 legacy=0x7f43d006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:52.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43cea00640 1 -- 172.21.15.113:0/2853954852 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d0158360 con 0x7f43d0151840 2024-03-04T07:47:52.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.535+0000 7f43cea00640 1 --2- 172.21.15.113:0/2853954852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43d0151840 0x7f43d00c1f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f43c00097c0 tx=0x7f43c001ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a581a8a930dde64 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:52.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43ce000640 1 -- 172.21.15.113:0/2853954852 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c00b5070 con 0x7f43d0151840 2024-03-04T07:47:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43ce000640 1 -- 172.21.15.113:0/2853954852 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c00a85f0 con 0x7f43d0151840 2024-03-04T07:47:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43ce000640 1 -- 172.21.15.113:0/2853954852 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8001a00 con 0x7f43d0151840 2024-03-04T07:47:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 -- 172.21.15.113:0/2853954852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43d0151840 msgr2=0x7f43d00c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 --2- 172.21.15.113:0/2853954852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43d0151840 0x7f43d00c1f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f43c00097c0 tx=0x7f43c001ebb0 comp rx=0 tx=0).stop 2024-03-04T07:47:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 -- 172.21.15.113:0/2853954852 shutdown_connections 2024-03-04T07:47:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 --2- 172.21.15.113:0/2853954852 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43d006d140 0x7f43d0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 --2- 172.21.15.113:0/2853954852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43d0151840 0x7f43d00c1f50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 -- 172.21.15.113:0/2853954852 >> 172.21.15.113:0/2853954852 conn(0x7f43d00c9a30 msgr2=0x7f43d00c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 -- 172.21.15.113:0/2853954852 shutdown_connections 2024-03-04T07:47:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 -- 172.21.15.113:0/2853954852 wait complete. 2024-03-04T07:47:52.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 Processor -- start 2024-03-04T07:47:52.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 -- start start 2024-03-04T07:47:52.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43d006d140 0x7f43d00c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d0151840 0x7f43d00ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43d0151c20 0x7f43d01f0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0071db0 con 0x7f43d0151840 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d0071ad0 con 0x7f43d006d140 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.539+0000 7f43cfe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d006c880 con 0x7f43d0151c20 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d0151840 0x7f43d00ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d0151840 0x7f43d00ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44544/0 (socket says 172.21.15.113:44544) 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c6000640 1 -- 172.21.15.113:0/2593274867 learned_addr learned my addr 172.21.15.113:0/2593274867 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43cea00640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43d006d140 0x7f43d00c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43cf400640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43d0151c20 0x7f43d01f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c6000640 1 -- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43d0151c20 msgr2=0x7f43d01f0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c6000640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43d0151c20 0x7f43d01f0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c6000640 1 -- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43d006d140 msgr2=0x7f43d00c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:52.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c6000640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43d006d140 0x7f43d00c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:52.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c6000640 1 -- 172.21.15.113:0/2593274867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d01f0270 con 0x7f43d0151840 2024-03-04T07:47:52.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c6000640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d0151840 0x7f43d00ca070 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f43b8001570 tx=0x7f43b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:52.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c7400640 1 -- 172.21.15.113:0/2593274867 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8012400 con 0x7f43d0151840 2024-03-04T07:47:52.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c7400640 1 -- 172.21.15.113:0/2593274867 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b80125a0 con 0x7f43d0151840 2024-03-04T07:47:52.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43c7400640 1 -- 172.21.15.113:0/2593274867 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b80129b0 con 0x7f43d0151840 2024-03-04T07:47:52.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43cfe00640 1 -- 172.21.15.113:0/2593274867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d01f3980 con 0x7f43d0151840 2024-03-04T07:47:52.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43cfe00640 1 -- 172.21.15.113:0/2593274867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d01f3f40 con 0x7f43d0151840 2024-03-04T07:47:52.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.543+0000 7f43cfe00640 1 -- 172.21.15.113:0/2593274867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d0075a80 con 0x7f43d0151840 2024-03-04T07:47:52.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.547+0000 7f43c7400640 1 -- 172.21.15.113:0/2593274867 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43b8012b50 con 0x7f43d0151840 2024-03-04T07:47:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.547+0000 7f43c7400640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43ac066f60 0x7f43ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.547+0000 7f43c7400640 1 -- 172.21.15.113:0/2593274867 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f43b8085b70 con 0x7f43d0151840 2024-03-04T07:47:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.547+0000 7f43cea00640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43ac066f60 0x7f43ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:52.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.547+0000 7f43cea00640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43ac066f60 0x7f43ac069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f43c00b2000 tx=0x7f43c00aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:52.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.551+0000 7f43c7400640 1 -- 172.21.15.113:0/2593274867 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f43b80560d0 con 0x7f43d0151840 2024-03-04T07:47:52.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.811+0000 7f43cfe00640 1 -- 172.21.15.113:0/2593274867 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43d006cf20 con 0x7f43d0151840 2024-03-04T07:47:52.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.811+0000 7f43c7400640 1 -- 172.21.15.113:0/2593274867 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5817 (secure 0 0 0) 0x7f43b805a170 con 0x7f43d0151840 2024-03-04T07:47:52.809 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:52.809 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:47:50.483134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:resolve","state_seq":33,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5631,5529]},"id":1}]} 2024-03-04T07:47:52.810 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 58 2024-03-04T07:47:52.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 -- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43ac066f60 msgr2=0x7f43ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:52.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43ac066f60 0x7f43ac069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f43c00b2000 tx=0x7f43c00aa070 comp rx=0 tx=0).stop 2024-03-04T07:47:52.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 -- 172.21.15.113:0/2593274867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d0151840 msgr2=0x7f43d00ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:52.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d0151840 0x7f43d00ca070 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f43b8001570 tx=0x7f43b800a430 comp rx=0 tx=0).stop 2024-03-04T07:47:52.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 -- 172.21.15.113:0/2593274867 shutdown_connections 2024-03-04T07:47:52.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f43d0151c20 0x7f43d01f0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f43ac066f60 0x7f43ac069420 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43d0151840 0x7f43d00ca070 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 --2- 172.21.15.113:0/2593274867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43d006d140 0x7f43d00c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 -- 172.21.15.113:0/2593274867 >> 172.21.15.113:0/2593274867 conn(0x7f43d00c9a30 msgr2=0x7f43d0156ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 -- 172.21.15.113:0/2593274867 shutdown_connections 2024-03-04T07:47:52.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:52.815+0000 7f43cfe00640 1 -- 172.21.15.113:0/2593274867 wait complete. 2024-03-04T07:47:52.830 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:47:54.831 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:54.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.970+0000 7f6401e00640 1 Processor -- start 2024-03-04T07:47:54.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.970+0000 7f6401e00640 1 -- start start 2024-03-04T07:47:54.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.970+0000 7f6401e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63fc1527f0 0x7f63fc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:54.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.970+0000 7f6401e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63fc1591f0 con 0x7f63fc0c8200 2024-03-04T07:47:54.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.970+0000 7f6401e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63fc1594d0 con 0x7f63fc152bd0 2024-03-04T07:47:54.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.970+0000 7f6401e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63fc071df0 con 0x7f63fc1527f0 2024-03-04T07:47:54.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.974+0000 7f63faa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63fc1527f0 0x7f63fc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:54.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.974+0000 7f63fa000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f63fc0c8200 0x7f63fc072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51654/0 (socket says 172.21.15.113:51654) 2024-03-04T07:47:54.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.974+0000 7f63fa000640 1 -- 172.21.15.113:0/1022526136 learned_addr learned my addr 172.21.15.113:0/1022526136 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:47:54.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.974+0000 7f63faa00640 1 -- 172.21.15.113:0/1022526136 >> v1:172.21.15.136:6789/0 conn(0x7f63fc152bd0 legacy=0x7f63fc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:54.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.974+0000 7f63faa00640 1 -- 172.21.15.113:0/1022526136 >> v1:172.21.15.113:6789/0 conn(0x7f63fc0c8200 legacy=0x7f63fc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:54.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.974+0000 7f63faa00640 1 -- 172.21.15.113:0/1022526136 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc0717a0 con 0x7f63fc1527f0 2024-03-04T07:47:54.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.974+0000 7f63faa00640 1 --2- 172.21.15.113:0/1022526136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 0x7f63fc070d40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0097c0 tx=0x7f63ec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d2401e07994b69 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:54.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.974+0000 7f63f9600640 1 -- 172.21.15.113:0/1022526136 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec0b1070 con 0x7f63fc1527f0 2024-03-04T07:47:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.974+0000 7f63f9600640 1 -- 172.21.15.113:0/1022526136 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ec0ac410 con 0x7f63fc1527f0 2024-03-04T07:47:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.974+0000 7f63f9600640 1 -- 172.21.15.113:0/1022526136 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ec0ac7e0 con 0x7f63fc1527f0 2024-03-04T07:47:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 -- 172.21.15.113:0/1022526136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 msgr2=0x7f63fc070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 --2- 172.21.15.113:0/1022526136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 0x7f63fc070d40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0097c0 tx=0x7f63ec01c4a0 comp rx=0 tx=0).stop 2024-03-04T07:47:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 -- 172.21.15.113:0/1022526136 shutdown_connections 2024-03-04T07:47:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 --2- 172.21.15.113:0/1022526136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 0x7f63fc070d40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 -- 172.21.15.113:0/1022526136 >> 172.21.15.113:0/1022526136 conn(0x7f63fc06e820 msgr2=0x7f63fc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 -- 172.21.15.113:0/1022526136 shutdown_connections 2024-03-04T07:47:54.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 -- 172.21.15.113:0/1022526136 wait complete. 2024-03-04T07:47:54.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 Processor -- start 2024-03-04T07:47:54.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 -- start start 2024-03-04T07:47:54.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63fc0c8200 0x7f63fc0c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:54.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 0x7f63fc0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:54.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63fc152bd0 0x7f63fc1f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc166380 con 0x7f63fc1527f0 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc1597b0 con 0x7f63fc0c8200 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f6401e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc166500 con 0x7f63fc152bd0 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63fa000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 0x7f63fc0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63fa000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 0x7f63fc0c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44568/0 (socket says 172.21.15.113:44568) 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63fa000640 1 -- 172.21.15.113:0/1854470031 learned_addr learned my addr 172.21.15.113:0/1854470031 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63faa00640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63fc0c8200 0x7f63fc0c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63fb400640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63fc152bd0 0x7f63fc1f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63fa000640 1 -- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63fc152bd0 msgr2=0x7f63fc1f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63fa000640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63fc152bd0 0x7f63fc1f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63fa000640 1 -- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63fc0c8200 msgr2=0x7f63fc0c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63fa000640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63fc0c8200 0x7f63fc0c34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63fa000640 1 -- 172.21.15.113:0/1854470031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc1521d0 con 0x7f63fc1527f0 2024-03-04T07:47:54.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.978+0000 7f63fb400640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63fc152bd0 0x7f63fc1f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:47:54.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.982+0000 7f63fa000640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 0x7f63fc0c4d80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c4d0 tx=0x7f63e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:54.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.982+0000 7f63f3400640 1 -- 172.21.15.113:0/1854470031 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8017070 con 0x7f63fc1527f0 2024-03-04T07:47:54.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.982+0000 7f63f3400640 1 -- 172.21.15.113:0/1854470031 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8005cf0 con 0x7f63fc1527f0 2024-03-04T07:47:54.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.982+0000 7f63f3400640 1 -- 172.21.15.113:0/1854470031 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8007270 con 0x7f63fc1527f0 2024-03-04T07:47:54.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.982+0000 7f6401e00640 1 -- 172.21.15.113:0/1854470031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63fc152470 con 0x7f63fc1527f0 2024-03-04T07:47:54.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.982+0000 7f6401e00640 1 -- 172.21.15.113:0/1854470031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63fc0b1380 con 0x7f63fc1527f0 2024-03-04T07:47:54.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.982+0000 7f6401e00640 1 -- 172.21.15.113:0/1854470031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63fc075c00 con 0x7f63fc1527f0 2024-03-04T07:47:54.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.982+0000 7f63f3400640 1 -- 172.21.15.113:0/1854470031 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f63e801a070 con 0x7f63fc1527f0 2024-03-04T07:47:54.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.982+0000 7f63f3400640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f63dc066f60 0x7f63dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:54.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.982+0000 7f63f3400640 1 -- 172.21.15.113:0/1854470031 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f63e80846a0 con 0x7f63fc1527f0 2024-03-04T07:47:54.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.986+0000 7f63faa00640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f63dc066f60 0x7f63dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:54.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.986+0000 7f63faa00640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f63dc066f60 0x7f63dc069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0ae000 tx=0x7f63ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:54.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:54.990+0000 7f63f3400640 1 -- 172.21.15.113:0/1854470031 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f63e8054c00 con 0x7f63fc1527f0 2024-03-04T07:47:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 -- 172.21.15.113:0/1854470031 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63fc0c79d0 con 0x7f63fc1527f0 2024-03-04T07:47:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f63f3400640 1 -- 172.21.15.113:0/1854470031 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5817 (secure 0 0 0) 0x7f63e8058ca0 con 0x7f63fc1527f0 2024-03-04T07:47:55.267 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:55.268 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:47:50.483134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:resolve","state_seq":33,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5631,5529]},"id":1}]} 2024-03-04T07:47:55.268 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 58 2024-03-04T07:47:55.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 -- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f63dc066f60 msgr2=0x7f63dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:55.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f63dc066f60 0x7f63dc069420 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0ae000 tx=0x7f63ec008040 comp rx=0 tx=0).stop 2024-03-04T07:47:55.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 -- 172.21.15.113:0/1854470031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 msgr2=0x7f63fc0c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:55.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 0x7f63fc0c4d80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c4d0 tx=0x7f63e800c990 comp rx=0 tx=0).stop 2024-03-04T07:47:55.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 -- 172.21.15.113:0/1854470031 shutdown_connections 2024-03-04T07:47:55.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f63fc152bd0 0x7f63fc1f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:55.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f63dc066f60 0x7f63dc069420 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:55.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63fc1527f0 0x7f63fc0c4d80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:55.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 --2- 172.21.15.113:0/1854470031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63fc0c8200 0x7f63fc0c34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:55.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 -- 172.21.15.113:0/1854470031 >> 172.21.15.113:0/1854470031 conn(0x7f63fc06e820 msgr2=0x7f63fc158660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:55.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 -- 172.21.15.113:0/1854470031 shutdown_connections 2024-03-04T07:47:55.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:55.270+0000 7f6401e00640 1 -- 172.21.15.113:0/1854470031 wait complete. 2024-03-04T07:47:55.299 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 58, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 58, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5417": { "addr": "172.21.15.136:6835/1691762848", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1691762848, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1691762848, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5417, "incarnation": 57, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:resolve", "state_seq": 33 }, "gid_5529": { "addr": "172.21.15.113:6836/2299268553", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2299268553, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2299268553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5529, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 43 }, "gid_5631": { "addr": "172.21.15.113:6833/810789479", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 810789479, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 810789479, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5631, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 56, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:47:50.483134+0000", "qdb_cluster": [ 5631, 5529 ], "qdb_leader": 5529, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5417, "mds_1": 5529, "mds_2": 5631 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:47:57.299 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:57.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9fe00640 1 Processor -- start 2024-03-04T07:47:57.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9fe00640 1 -- start start 2024-03-04T07:47:57.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1da015bb40 0x7f1da006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:57.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9fe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00c4ce0 con 0x7f1da00cb540 2024-03-04T07:47:57.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9fe00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00c4f80 con 0x7f1da0152950 2024-03-04T07:47:57.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da00c8370 con 0x7f1da015bb40 2024-03-04T07:47:57.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1da015bb40 0x7f1da006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:57.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d96000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1da00cb540 0x7f1da0160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43270/0 (socket says 172.21.15.113:43270) 2024-03-04T07:47:57.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d96000640 1 -- 172.21.15.113:0/3050512303 learned_addr learned my addr 172.21.15.113:0/3050512303 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:47:57.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9f400640 1 -- 172.21.15.113:0/3050512303 >> v1:172.21.15.136:6789/0 conn(0x7f1da0152950 legacy=0x7f1da006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:57.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9f400640 1 -- 172.21.15.113:0/3050512303 >> v1:172.21.15.113:6789/0 conn(0x7f1da00cb540 legacy=0x7f1da0160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:57.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9f400640 1 -- 172.21.15.113:0/3050512303 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00cb920 con 0x7f1da015bb40 2024-03-04T07:47:57.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.506+0000 7f1d9e000640 1 -- 172.21.15.113:0/3050512303 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87664519 0 0) 0x7f1da00c4ce0 con 0x7f1da00cb540 2024-03-04T07:47:57.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9f400640 1 --2- 172.21.15.113:0/3050512303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da015bb40 0x7f1da006faa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009d90 tx=0x7f1d9801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de53be8d1ebdc42 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:57.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9e000640 1 -- 172.21.15.113:0/3050512303 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d980b4070 con 0x7f1da015bb40 2024-03-04T07:47:57.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9e000640 1 -- 172.21.15.113:0/3050512303 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d980af600 con 0x7f1da015bb40 2024-03-04T07:47:57.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9e000640 1 -- 172.21.15.113:0/3050512303 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d980af9d0 con 0x7f1da015bb40 2024-03-04T07:47:57.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9fe00640 1 -- 172.21.15.113:0/3050512303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da015bb40 msgr2=0x7f1da006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:57.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9fe00640 1 --2- 172.21.15.113:0/3050512303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da015bb40 0x7f1da006faa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009d90 tx=0x7f1d9801f150 comp rx=0 tx=0).stop 2024-03-04T07:47:57.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9fe00640 1 -- 172.21.15.113:0/3050512303 shutdown_connections 2024-03-04T07:47:57.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9fe00640 1 --2- 172.21.15.113:0/3050512303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da015bb40 0x7f1da006faa0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:57.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9fe00640 1 -- 172.21.15.113:0/3050512303 >> 172.21.15.113:0/3050512303 conn(0x7f1da0072f40 msgr2=0x7f1da00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:57.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9fe00640 1 -- 172.21.15.113:0/3050512303 shutdown_connections 2024-03-04T07:47:57.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.510+0000 7f1d9fe00640 1 -- 172.21.15.113:0/3050512303 wait complete. 2024-03-04T07:47:57.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9fe00640 1 Processor -- start 2024-03-04T07:47:57.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9fe00640 1 -- start start 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da00cb540 0x7f1da01516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1da0152950 0x7f1da0151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1da015bb40 0x7f1da01f0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00c9a50 con 0x7f1da00cb540 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00ca0a0 con 0x7f1da015bb40 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00c9d30 con 0x7f1da0152950 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da00cb540 0x7f1da01516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da00cb540 0x7f1da01516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32778/0 (socket says 172.21.15.113:32778) 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.514+0000 7f1d9ea00640 1 -- 172.21.15.113:0/2698461988 learned_addr learned my addr 172.21.15.113:0/2698461988 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d96000640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1da0152950 0x7f1da0151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:57.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9f400640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1da015bb40 0x7f1da01f0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9ea00640 1 -- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1da0152950 msgr2=0x7f1da0151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9ea00640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1da0152950 0x7f1da0151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9ea00640 1 -- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1da015bb40 msgr2=0x7f1da01f0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9ea00640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1da015bb40 0x7f1da01f0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9ea00640 1 -- 172.21.15.113:0/2698461988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da01f0370 con 0x7f1da00cb540 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9f400640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1da015bb40 0x7f1da01f0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d96000640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1da0152950 0x7f1da0151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9ea00640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da00cb540 0x7f1da01516e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c480 tx=0x7f1d9000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d97400640 1 -- 172.21.15.113:0/2698461988 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d900052d0 con 0x7f1da00cb540 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d97400640 1 -- 172.21.15.113:0/2698461988 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d90005490 con 0x7f1da00cb540 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9fe00640 1 -- 172.21.15.113:0/2698461988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da01e69a0 con 0x7f1da00cb540 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d97400640 1 -- 172.21.15.113:0/2698461988 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d90012850 con 0x7f1da00cb540 2024-03-04T07:47:57.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9fe00640 1 -- 172.21.15.113:0/2698461988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da006d090 con 0x7f1da00cb540 2024-03-04T07:47:57.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.518+0000 7f1d9fe00640 1 -- 172.21.15.113:0/2698461988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da0075a80 con 0x7f1da00cb540 2024-03-04T07:47:57.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.522+0000 7f1d97400640 1 -- 172.21.15.113:0/2698461988 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d900129f0 con 0x7f1da00cb540 2024-03-04T07:47:57.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.522+0000 7f1d97400640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d7c066f60 0x7f1d7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:57.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.522+0000 7f1d97400640 1 -- 172.21.15.113:0/2698461988 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f1d90084f60 con 0x7f1da00cb540 2024-03-04T07:47:57.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.522+0000 7f1d96000640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d7c066f60 0x7f1d7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:57.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.522+0000 7f1d96000640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d7c066f60 0x7f1d7c069420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1da0071100 tx=0x7f1d88007c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:57.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.530+0000 7f1d97400640 1 -- 172.21.15.113:0/2698461988 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1da01e69a0 con 0x7f1da00cb540 2024-03-04T07:47:57.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.726+0000 7f1d9fe00640 1 -- 172.21.15.113:0/2698461988 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1da01f0540 con 0x7f1da00cb540 2024-03-04T07:47:57.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.726+0000 7f1d97400640 1 -- 172.21.15.113:0/2698461988 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f1d90055cd0 con 0x7f1da00cb540 2024-03-04T07:47:57.725 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:47:57.725 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:47:56.182663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:47:57.725 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 59 2024-03-04T07:47:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 -- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d7c066f60 msgr2=0x7f1d7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d7c066f60 0x7f1d7c069420 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1da0071100 tx=0x7f1d88007c60 comp rx=0 tx=0).stop 2024-03-04T07:47:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 -- 172.21.15.113:0/2698461988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da00cb540 msgr2=0x7f1da01516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da00cb540 0x7f1da01516e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c480 tx=0x7f1d9000c940 comp rx=0 tx=0).stop 2024-03-04T07:47:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 -- 172.21.15.113:0/2698461988 shutdown_connections 2024-03-04T07:47:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1da015bb40 0x7f1da01f0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1da0152950 0x7f1da0151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:57.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d7c066f60 0x7f1d7c069420 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:57.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 --2- 172.21.15.113:0/2698461988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1da00cb540 0x7f1da01516e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:57.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 -- 172.21.15.113:0/2698461988 >> 172.21.15.113:0/2698461988 conn(0x7f1da0072f40 msgr2=0x7f1da015c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:57.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 -- 172.21.15.113:0/2698461988 shutdown_connections 2024-03-04T07:47:57.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:57.730+0000 7f1d9fe00640 1 -- 172.21.15.113:0/2698461988 wait complete. 2024-03-04T07:47:57.747 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:47:58.711 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:47:58.714 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:47:59.748 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:47:59.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.890+0000 7f0b73200640 1 Processor -- start 2024-03-04T07:47:59.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.890+0000 7f0b73200640 1 -- start start 2024-03-04T07:47:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.890+0000 7f0b73200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c22e0 0x7f0b6c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.890+0000 7f0b73200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b6c0c2d80 0x7f0b6c15b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.890+0000 7f0b73200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6c0c27f0 con 0x7f0b6c14e4c0 2024-03-04T07:47:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.890+0000 7f0b73200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6c15ad60 con 0x7f0b6c0c2d80 2024-03-04T07:47:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.890+0000 7f0b73200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c0c2a90 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b71200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b6c0c2d80 0x7f0b6c15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b71200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b6c0c2d80 0x7f0b6c15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32784/0 (socket says 172.21.15.113:32784) 2024-03-04T07:47:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b71200640 1 -- 172.21.15.113:0/1431130671 learned_addr learned my addr 172.21.15.113:0/1431130671 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:59.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b63400640 1 --2- 172.21.15.113:0/1431130671 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c22e0 0x7f0b6c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:59.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b71200640 1 -- 172.21.15.113:0/1431130671 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c22e0 msgr2=0x7f0b6c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:47:59.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b71200640 1 --2- 172.21.15.113:0/1431130671 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c22e0 0x7f0b6c15b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:59.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b71200640 1 -- 172.21.15.113:0/1431130671 >> v1:172.21.15.136:6789/0 conn(0x7f0b6c14e4c0 legacy=0x7f0b6c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:59.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b71200640 1 -- 172.21.15.113:0/1431130671 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c070c60 con 0x7f0b6c0c2d80 2024-03-04T07:47:59.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b63400640 1 --2- 172.21.15.113:0/1431130671 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c22e0 0x7f0b6c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:47:59.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b71200640 1 --2- 172.21.15.113:0/1431130671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c2d80 0x7f0b6c15b730 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009870 tx=0x7f0b6401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b35c69a4260d40 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b6b400640 1 -- 172.21.15.113:0/1431130671 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640b5070 con 0x7f0b6c0c2d80 2024-03-04T07:47:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b6b400640 1 -- 172.21.15.113:0/1431130671 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b640a8a50 con 0x7f0b6c0c2d80 2024-03-04T07:47:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.894+0000 7f0b6b400640 1 -- 172.21.15.113:0/1431130671 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b640a8e20 con 0x7f0b6c0c2d80 2024-03-04T07:47:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 -- 172.21.15.113:0/1431130671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c2d80 msgr2=0x7f0b6c15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 --2- 172.21.15.113:0/1431130671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c2d80 0x7f0b6c15b730 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009870 tx=0x7f0b6401ed50 comp rx=0 tx=0).stop 2024-03-04T07:47:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 -- 172.21.15.113:0/1431130671 shutdown_connections 2024-03-04T07:47:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 --2- 172.21.15.113:0/1431130671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c2d80 0x7f0b6c15b730 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 --2- 172.21.15.113:0/1431130671 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c22e0 0x7f0b6c15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 -- 172.21.15.113:0/1431130671 >> 172.21.15.113:0/1431130671 conn(0x7f0b6c06e820 msgr2=0x7f0b6c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:47:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 -- 172.21.15.113:0/1431130671 shutdown_connections 2024-03-04T07:47:59.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 -- 172.21.15.113:0/1431130671 wait complete. 2024-03-04T07:47:59.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 Processor -- start 2024-03-04T07:47:59.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 -- start start 2024-03-04T07:47:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c22e0 0x7f0b6c163620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c2d80 0x7f0b6c1f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6c14e4c0 0x7f0b6c14ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c071500 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c071220 con 0x7f0b6c14e4c0 2024-03-04T07:47:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b73200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c14e8a0 con 0x7f0b6c0c2d80 2024-03-04T07:47:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c22e0 0x7f0b6c163620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:59.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b6be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c22e0 0x7f0b6c163620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32796/0 (socket says 172.21.15.113:32796) 2024-03-04T07:47:59.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b6be00640 1 -- 172.21.15.113:0/2594366143 learned_addr learned my addr 172.21.15.113:0/2594366143 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:47:59.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b63400640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c2d80 0x7f0b6c1f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:59.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b71200640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6c14e4c0 0x7f0b6c14ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:59.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b6be00640 1 -- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c2d80 msgr2=0x7f0b6c1f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:59.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b6be00640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c2d80 0x7f0b6c1f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:59.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b6be00640 1 -- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6c14e4c0 msgr2=0x7f0b6c14ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:47:59.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b6be00640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6c14e4c0 0x7f0b6c14ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:47:59.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.898+0000 7f0b6be00640 1 -- 172.21.15.113:0/2594366143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c14a140 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b71200640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6c14e4c0 0x7f0b6c14ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:47:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b6be00640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c22e0 0x7f0b6c163620 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00c450 tx=0x7f0b5c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b68c00640 1 -- 172.21.15.113:0/2594366143 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c016020 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b68c00640 1 -- 172.21.15.113:0/2594366143 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c0052d0 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b73200640 1 -- 172.21.15.113:0/2594366143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6c14a3e0 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b68c00640 1 -- 172.21.15.113:0/2594366143 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c007260 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b73200640 1 -- 172.21.15.113:0/2594366143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6c14a980 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b73200640 1 -- 172.21.15.113:0/2594366143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6c075c00 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b68c00640 1 -- 172.21.15.113:0/2594366143 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b5c007400 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b68c00640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b48066f90 0x7f0b48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:47:59.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b68c00640 1 -- 172.21.15.113:0/2594366143 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f0b5c084c70 con 0x7f0b6c0c22e0 2024-03-04T07:47:59.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.902+0000 7f0b63400640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b48066f90 0x7f0b48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:47:59.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.906+0000 7f0b63400640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b48066f90 0x7f0b48069450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c14ac80 tx=0x7f0b54009870 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:47:59.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:47:59.906+0000 7f0b68c00640 1 -- 172.21.15.113:0/2594366143 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0b5c020020 con 0x7f0b6c0c22e0 2024-03-04T07:48:00.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.186+0000 7f0b73200640 1 -- 172.21.15.113:0/2594366143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b6c0c88b0 con 0x7f0b6c0c22e0 2024-03-04T07:48:00.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.186+0000 7f0b68c00640 1 -- 172.21.15.113:0/2594366143 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f0b5c005480 con 0x7f0b6c0c22e0 2024-03-04T07:48:00.185 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:48:00.185 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:47:56.182663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:48:00.185 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 59 2024-03-04T07:48:00.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 -- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b48066f90 msgr2=0x7f0b48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:00.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b48066f90 0x7f0b48069450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c14ac80 tx=0x7f0b54009870 comp rx=0 tx=0).stop 2024-03-04T07:48:00.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 -- 172.21.15.113:0/2594366143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c22e0 msgr2=0x7f0b6c163620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c22e0 0x7f0b6c163620 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00c450 tx=0x7f0b5c00c910 comp rx=0 tx=0).stop 2024-03-04T07:48:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 -- 172.21.15.113:0/2594366143 shutdown_connections 2024-03-04T07:48:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6c14e4c0 0x7f0b6c14ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6c0c2d80 0x7f0b6c1f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b48066f90 0x7f0b48069450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 --2- 172.21.15.113:0/2594366143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b6c0c22e0 0x7f0b6c163620 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 -- 172.21.15.113:0/2594366143 >> 172.21.15.113:0/2594366143 conn(0x7f0b6c06e820 msgr2=0x7f0b6c14bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 -- 172.21.15.113:0/2594366143 shutdown_connections 2024-03-04T07:48:00.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:00.190+0000 7f0b73200640 1 -- 172.21.15.113:0/2594366143 wait complete. 2024-03-04T07:48:00.211 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:48:02.212 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:48:02.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1bc000640 1 Processor -- start 2024-03-04T07:48:02.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1bc000640 1 -- start start 2024-03-04T07:48:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1bc000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1b4152b20 0x7fb1b4070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1bc000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1b415c510 0x7fb1b4163190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1bc000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b40c8f60 con 0x7fb1b40c8b00 2024-03-04T07:48:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1bc000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b40c5320 con 0x7fb1b415c510 2024-03-04T07:48:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1bc000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b4152f00 con 0x7fb1b4152b20 2024-03-04T07:48:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1ba000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1b415c510 0x7fb1b4163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:02.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1ba000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1b415c510 0x7fb1b4163190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32812/0 (socket says 172.21.15.113:32812) 2024-03-04T07:48:02.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1ba000640 1 -- 172.21.15.113:0/2966713089 learned_addr learned my addr 172.21.15.113:0/2966713089 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:48:02.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1b8c00640 1 --1- 172.21.15.113:0/2966713089 >> v1:172.21.15.113:6789/0 conn(0x7fb1b40c8b00 0x7fb1b415be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43274/0 (socket says 172.21.15.113:43274) 2024-03-04T07:48:02.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1b9600640 1 --2- 172.21.15.113:0/2966713089 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1b4152b20 0x7fb1b4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:02.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1ba000640 1 -- 172.21.15.113:0/2966713089 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1b4152b20 msgr2=0x7fb1b4070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:02.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1ba000640 1 --2- 172.21.15.113:0/2966713089 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1b4152b20 0x7fb1b4070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:02.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1ba000640 1 -- 172.21.15.113:0/2966713089 >> v1:172.21.15.113:6789/0 conn(0x7fb1b40c8b00 legacy=0x7fb1b415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:02.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.354+0000 7fb1ba000640 1 -- 172.21.15.113:0/2966713089 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b40c5040 con 0x7fb1b415c510 2024-03-04T07:48:02.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.358+0000 7fb1ba000640 1 --2- 172.21.15.113:0/2966713089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b415c510 0x7fb1b4163190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009e50 tx=0x7fb1ac01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e069e800905909f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:02.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1b3e00640 1 -- 172.21.15.113:0/2966713089 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0a57f0 con 0x7fb1b415c510 2024-03-04T07:48:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1b3e00640 1 -- 172.21.15.113:0/2966713089 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac0a5990 con 0x7fb1b415c510 2024-03-04T07:48:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1b3e00640 1 -- 172.21.15.113:0/2966713089 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0a5d60 con 0x7fb1b415c510 2024-03-04T07:48:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 -- 172.21.15.113:0/2966713089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b415c510 msgr2=0x7fb1b4163190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 --2- 172.21.15.113:0/2966713089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b415c510 0x7fb1b4163190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac009e50 tx=0x7fb1ac01ca30 comp rx=0 tx=0).stop 2024-03-04T07:48:02.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 -- 172.21.15.113:0/2966713089 shutdown_connections 2024-03-04T07:48:02.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 --2- 172.21.15.113:0/2966713089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b415c510 0x7fb1b4163190 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:02.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 --2- 172.21.15.113:0/2966713089 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1b4152b20 0x7fb1b4070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:02.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 -- 172.21.15.113:0/2966713089 >> 172.21.15.113:0/2966713089 conn(0x7fb1b40709a0 msgr2=0x7fb1b4070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:02.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 -- 172.21.15.113:0/2966713089 shutdown_connections 2024-03-04T07:48:02.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 -- 172.21.15.113:0/2966713089 wait complete. 2024-03-04T07:48:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 Processor -- start 2024-03-04T07:48:02.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 -- start start 2024-03-04T07:48:02.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1b40c8b00 0x7fb1b4154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:02.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b4152b20 0x7fb1b414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:02.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1b415c510 0x7fb1b41f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:02.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b4158810 con 0x7fb1b4152b20 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b40c3810 con 0x7fb1b40c8b00 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1bc000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b40c8490 con 0x7fb1b415c510 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1b8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b4152b20 0x7fb1b414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1b8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b4152b20 0x7fb1b414e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32828/0 (socket says 172.21.15.113:32828) 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.362+0000 7fb1b8c00640 1 -- 172.21.15.113:0/300620779 learned_addr learned my addr 172.21.15.113:0/300620779 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1ba000640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1b415c510 0x7fb1b41f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1b9600640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1b40c8b00 0x7fb1b4154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1b8c00640 1 -- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1b415c510 msgr2=0x7fb1b41f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1b8c00640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1b415c510 0x7fb1b41f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1b8c00640 1 -- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1b40c8b00 msgr2=0x7fb1b4154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1b8c00640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1b40c8b00 0x7fb1b4154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1b8c00640 1 -- 172.21.15.113:0/300620779 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b41f5300 con 0x7fb1b4152b20 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1b8c00640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b4152b20 0x7fb1b414e9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000c370 tx=0x7fb1a000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:02.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1b1600640 1 -- 172.21.15.113:0/300620779 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00074e0 con 0x7fb1b4152b20 2024-03-04T07:48:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1b1600640 1 -- 172.21.15.113:0/300620779 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a0007680 con 0x7fb1b4152b20 2024-03-04T07:48:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1bc000640 1 -- 172.21.15.113:0/300620779 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b4153c60 con 0x7fb1b4152b20 2024-03-04T07:48:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1b1600640 1 -- 172.21.15.113:0/300620779 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0004a30 con 0x7fb1b4152b20 2024-03-04T07:48:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1bc000640 1 -- 172.21.15.113:0/300620779 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b4154220 con 0x7fb1b4152b20 2024-03-04T07:48:02.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.366+0000 7fb1bc000640 1 -- 172.21.15.113:0/300620779 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b4075c00 con 0x7fb1b4152b20 2024-03-04T07:48:02.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.370+0000 7fb1b1600640 1 -- 172.21.15.113:0/300620779 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1a001a020 con 0x7fb1b4152b20 2024-03-04T07:48:02.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.370+0000 7fb1b1600640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb194066f90 0x7fb194069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:02.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.370+0000 7fb1b1600640 1 -- 172.21.15.113:0/300620779 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fb1a00849b0 con 0x7fb1b4152b20 2024-03-04T07:48:02.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.370+0000 7fb1b9600640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb194066f90 0x7fb194069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:02.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.370+0000 7fb1b9600640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb194066f90 0x7fb194069450 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009a50 tx=0x7fb1a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:02.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.374+0000 7fb1b1600640 1 -- 172.21.15.113:0/300620779 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb1a0055720 con 0x7fb1b4152b20 2024-03-04T07:48:02.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 -- 172.21.15.113:0/300620779 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1b40c87b0 con 0x7fb1b4152b20 2024-03-04T07:48:02.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1b1600640 1 -- 172.21.15.113:0/300620779 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5816 (secure 0 0 0) 0x7fb1a0055020 con 0x7fb1b4152b20 2024-03-04T07:48:02.564 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:48:02.564 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:01.342238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5631,5529]},"id":1}]} 2024-03-04T07:48:02.565 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 60 2024-03-04T07:48:02.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 -- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb194066f90 msgr2=0x7fb194069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:02.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb194066f90 0x7fb194069450 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009a50 tx=0x7fb1a8008040 comp rx=0 tx=0).stop 2024-03-04T07:48:02.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 -- 172.21.15.113:0/300620779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b4152b20 msgr2=0x7fb1b414e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:02.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b4152b20 0x7fb1b414e9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000c370 tx=0x7fb1a000c830 comp rx=0 tx=0).stop 2024-03-04T07:48:02.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 -- 172.21.15.113:0/300620779 shutdown_connections 2024-03-04T07:48:02.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1b415c510 0x7fb1b41f47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:02.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb194066f90 0x7fb194069450 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:02.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b4152b20 0x7fb1b414e9c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:02.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 --2- 172.21.15.113:0/300620779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1b40c8b00 0x7fb1b4154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:02.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 -- 172.21.15.113:0/300620779 >> 172.21.15.113:0/300620779 conn(0x7fb1b40709a0 msgr2=0x7fb1b4151c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:02.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 -- 172.21.15.113:0/300620779 shutdown_connections 2024-03-04T07:48:02.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:02.566+0000 7fb1bc000640 1 -- 172.21.15.113:0/300620779 wait complete. 2024-03-04T07:48:02.586 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:48:04.587 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:48:04.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaac200640 1 Processor -- start 2024-03-04T07:48:04.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaac200640 1 -- start start 2024-03-04T07:48:04.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaac200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa4152d60 0x7faaa40ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:04.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaac200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa40c5380 con 0x7faaa4070120 2024-03-04T07:48:04.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaac200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa40c3440 con 0x7faaa4159040 2024-03-04T07:48:04.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaac200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa406f620 con 0x7faaa4152d60 2024-03-04T07:48:04.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaa8e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7faaa4070120 0x7faaa4158930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43276/0 (socket says 172.21.15.113:43276) 2024-03-04T07:48:04.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaa8e00640 1 -- 172.21.15.113:0/3305605379 learned_addr learned my addr 172.21.15.113:0/3305605379 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:48:04.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaa9800640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa4152d60 0x7faaa40ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837657635 0 0) 0x7faaa40c5380 con 0x7faaa4070120 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.730+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa8c003530 con 0x7faaa4070120 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 469621398 0 0) 0x7faa8c003530 con 0x7faaa4070120 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa4152d60 msgr2=0x7faaa40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa4152d60 0x7faaa40ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 >> v1:172.21.15.136:6789/0 conn(0x7faaa4159040 legacy=0x7faaa41632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa40c36e0 con 0x7faaa4070120 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7faa90001680 con 0x7faaa4070120 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 >> v1:172.21.15.113:6789/0 conn(0x7faaa4070120 legacy=0x7faaa4158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c0039f0 0x7faa8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa8c004810 0x7faa8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa4152d60 0x7faa8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:04.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa406f620 con 0x7faa8c004810 2024-03-04T07:48:04.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c00ba80 con 0x7faa8c0039f0 2024-03-04T07:48:04.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c00bc30 con 0x7faaa4152d60 2024-03-04T07:48:04.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa8e00640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa8c004810 0x7faa8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:04.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa9800640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c0039f0 0x7faa8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:04.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaaa200640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa4152d60 0x7faa8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:04.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa8e00640 1 -- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa4152d60 msgr2=0x7faa8c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:04.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa8e00640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa4152d60 0x7faa8c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:04.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa8e00640 1 -- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c0039f0 msgr2=0x7faa8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:04.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa8e00640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c0039f0 0x7faa8c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:04.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.734+0000 7faaa8e00640 1 -- 172.21.15.113:0/3305605379 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c003530 con 0x7faa8c004810 2024-03-04T07:48:04.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaa8e00640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa8c004810 0x7faa8c004c90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7faa9000c160 tx=0x7faa9001f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88443921ccf6ff80 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:04.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa90007c70 con 0x7faa8c004810 2024-03-04T07:48:04.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa900ac070 con 0x7faa8c004810 2024-03-04T07:48:04.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaa3e00640 1 -- 172.21.15.113:0/3305605379 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa90005b20 con 0x7faa8c004810 2024-03-04T07:48:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaac200640 1 -- 172.21.15.113:0/3305605379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa8c004810 msgr2=0x7faa8c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaac200640 1 --2- 172.21.15.113:0/3305605379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa8c004810 0x7faa8c004c90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7faa9000c160 tx=0x7faa9001f270 comp rx=0 tx=0).stop 2024-03-04T07:48:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaaa200640 1 -- 172.21.15.113:0/3305605379 reap_dead start 2024-03-04T07:48:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaac200640 1 -- 172.21.15.113:0/3305605379 shutdown_connections 2024-03-04T07:48:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaac200640 1 -- 172.21.15.113:0/3305605379 >> 172.21.15.113:0/3305605379 conn(0x7faaa4066db0 msgr2=0x7faaa40671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:04.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaac200640 1 -- 172.21.15.113:0/3305605379 shutdown_connections 2024-03-04T07:48:04.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.738+0000 7faaac200640 1 -- 172.21.15.113:0/3305605379 wait complete. 2024-03-04T07:48:04.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaac200640 1 Processor -- start 2024-03-04T07:48:04.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaac200640 1 -- start start 2024-03-04T07:48:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaac200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c004810 0x7faaa4151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaac200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa40ae8e0 0x7faaa406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaac200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa41f03a0 0x7faaa41f0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaac200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa40c39c0 con 0x7faaa41f03a0 2024-03-04T07:48:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaac200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa40c5020 con 0x7faa8c004810 2024-03-04T07:48:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaac200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa40c3b40 con 0x7faaa40ae8e0 2024-03-04T07:48:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaaa200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa41f03a0 0x7faaa41f0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:04.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaaa200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa41f03a0 0x7faaa41f0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32844/0 (socket says 172.21.15.113:32844) 2024-03-04T07:48:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaaa200640 1 -- 172.21.15.113:0/4046282269 learned_addr learned my addr 172.21.15.113:0/4046282269 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:48:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaa8e00640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa40ae8e0 0x7faaa406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaa9800640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c004810 0x7faaa4151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaaa200640 1 -- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa40ae8e0 msgr2=0x7faaa406db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaaa200640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa40ae8e0 0x7faaa406db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaaa200640 1 -- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c004810 msgr2=0x7faaa4151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaaa200640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c004810 0x7faaa4151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaaa200640 1 -- 172.21.15.113:0/4046282269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa41f0d60 con 0x7faaa41f03a0 2024-03-04T07:48:04.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaaa200640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa41f03a0 0x7faaa41f0820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7faaa40c4f80 tx=0x7faa9c00feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:04.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaa1600640 1 -- 172.21.15.113:0/4046282269 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c019070 con 0x7faaa41f03a0 2024-03-04T07:48:04.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaa1600640 1 -- 172.21.15.113:0/4046282269 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9c006d70 con 0x7faaa41f03a0 2024-03-04T07:48:04.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaa1600640 1 -- 172.21.15.113:0/4046282269 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c003be0 con 0x7faaa41f03a0 2024-03-04T07:48:04.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaac200640 1 -- 172.21.15.113:0/4046282269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa40aee40 con 0x7faaa41f03a0 2024-03-04T07:48:04.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.742+0000 7faaac200640 1 -- 172.21.15.113:0/4046282269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa40c5c70 con 0x7faaa41f03a0 2024-03-04T07:48:04.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.746+0000 7faaac200640 1 -- 172.21.15.113:0/4046282269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa40c5700 con 0x7faaa41f03a0 2024-03-04T07:48:04.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.746+0000 7faaa1600640 1 -- 172.21.15.113:0/4046282269 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa9c01c070 con 0x7faaa41f03a0 2024-03-04T07:48:04.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.746+0000 7faaa1600640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faa84066f90 0x7faa84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:04.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.746+0000 7faaa1600640 1 -- 172.21.15.113:0/4046282269 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7faa9c084570 con 0x7faaa41f03a0 2024-03-04T07:48:04.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.750+0000 7faaa9800640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faa84066f90 0x7faa84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:04.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.750+0000 7faaa9800640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faa84066f90 0x7faa84069450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faa98009b60 tx=0x7faa98009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:04.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.750+0000 7faaa1600640 1 -- 172.21.15.113:0/4046282269 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7faaa41f0d60 con 0x7faaa41f03a0 2024-03-04T07:48:04.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.986+0000 7faaac200640 1 -- 172.21.15.113:0/4046282269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faaa4075c00 con 0x7faaa41f03a0 2024-03-04T07:48:04.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.986+0000 7faaa1600640 1 -- 172.21.15.113:0/4046282269 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5816 (secure 0 0 0) 0x7faa9c054ad0 con 0x7faaa41f03a0 2024-03-04T07:48:04.986 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:48:04.986 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:01.342238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5631,5529]},"id":1}]} 2024-03-04T07:48:04.987 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 60 2024-03-04T07:48:04.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 -- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faa84066f90 msgr2=0x7faa84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:04.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faa84066f90 0x7faa84069450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faa98009b60 tx=0x7faa98009190 comp rx=0 tx=0).stop 2024-03-04T07:48:04.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 -- 172.21.15.113:0/4046282269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa41f03a0 msgr2=0x7faaa41f0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:04.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa41f03a0 0x7faaa41f0820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7faaa40c4f80 tx=0x7faa9c00feb0 comp rx=0 tx=0).stop 2024-03-04T07:48:04.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 -- 172.21.15.113:0/4046282269 shutdown_connections 2024-03-04T07:48:04.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faa84066f90 0x7faa84069450 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:04.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faaa41f03a0 0x7faaa41f0820 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:04.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faaa40ae8e0 0x7faaa406db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:04.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 --2- 172.21.15.113:0/4046282269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faa8c004810 0x7faaa4151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:04.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 -- 172.21.15.113:0/4046282269 >> 172.21.15.113:0/4046282269 conn(0x7faaa4066db0 msgr2=0x7faaa40b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:04.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 -- 172.21.15.113:0/4046282269 shutdown_connections 2024-03-04T07:48:04.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:04.990+0000 7faaac200640 1 -- 172.21.15.113:0/4046282269 wait complete. 2024-03-04T07:48:05.015 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:48:07.017 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:48:07.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.154+0000 7f1a6ae00640 1 Processor -- start 2024-03-04T07:48:07.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.154+0000 7f1a6ae00640 1 -- start start 2024-03-04T07:48:07.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a6ae00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1a641527f0 0x7f1a64070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:07.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a6ae00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a640c8200 0x7f1a64072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:07.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a6ae00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a641591f0 con 0x7f1a64152bd0 2024-03-04T07:48:07.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a6ae00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a64071df0 con 0x7f1a641527f0 2024-03-04T07:48:07.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a6ae00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a641597b0 con 0x7f1a640c8200 2024-03-04T07:48:07.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a68e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1a64152bd0 0x7f1a641587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43278/0 (socket says 172.21.15.113:43278) 2024-03-04T07:48:07.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a68e00640 1 -- 172.21.15.113:0/105133618 learned_addr learned my addr 172.21.15.113:0/105133618 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:48:07.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a5b400640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a640c8200 0x7f1a64072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:07.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63e00640 1 --2- 172.21.15.113:0/105133618 >> v2:172.21.15.136:3300/0 conn(0x7f1a641527f0 0x7f1a64070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:07.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593417531 0 0) 0x7f1a641591f0 con 0x7f1a64152bd0 2024-03-04T07:48:07.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a48003530 con 0x7f1a64152bd0 2024-03-04T07:48:07.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2487773068 0 0) 0x7f1a48003530 con 0x7f1a64152bd0 2024-03-04T07:48:07.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a640c8200 msgr2=0x7f1a64072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:07.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a640c8200 0x7f1a64072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 >> v2:172.21.15.136:3300/0 conn(0x7f1a641527f0 msgr2=0x7f1a64070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:07.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 --2- 172.21.15.113:0/105133618 >> v2:172.21.15.136:3300/0 conn(0x7f1a641527f0 0x7f1a64070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a641594d0 con 0x7f1a64152bd0 2024-03-04T07:48:07.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f1a5c001680 con 0x7f1a64152bd0 2024-03-04T07:48:07.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 >> v1:172.21.15.113:6789/0 conn(0x7f1a64152bd0 legacy=0x7f1a641587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a480039c0 0x7f1a48003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a640c8200 0x7f1a48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a48004d50 0x7f1a480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64071df0 con 0x7f1a48004d50 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a641597b0 con 0x7f1a480039c0 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4800b9d0 con 0x7f1a640c8200 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a68e00640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a48004d50 0x7f1a480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a63e00640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a480039c0 0x7f1a48003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a5b400640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a640c8200 0x7f1a48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a68e00640 1 -- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a640c8200 msgr2=0x7f1a48004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a68e00640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a640c8200 0x7f1a48004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a68e00640 1 -- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a480039c0 msgr2=0x7f1a48003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a68e00640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a480039c0 0x7f1a48003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.158+0000 7f1a68e00640 1 -- 172.21.15.113:0/105133618 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a48003530 con 0x7f1a48004d50 2024-03-04T07:48:07.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a68e00640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a48004d50 0x7f1a480051d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00c190 tx=0x7f1a5c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff234a6ccf4d225a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:07.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c007c70 con 0x7f1a48004d50 2024-03-04T07:48:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c0a9020 con 0x7f1a48004d50 2024-03-04T07:48:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a63400640 1 -- 172.21.15.113:0/105133618 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c005ae0 con 0x7f1a48004d50 2024-03-04T07:48:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a6ae00640 1 -- 172.21.15.113:0/105133618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a48004d50 msgr2=0x7f1a480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a6ae00640 1 --2- 172.21.15.113:0/105133618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a48004d50 0x7f1a480051d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00c190 tx=0x7f1a5c01cbe0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a68e00640 1 -- 172.21.15.113:0/105133618 reap_dead start 2024-03-04T07:48:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a6ae00640 1 -- 172.21.15.113:0/105133618 shutdown_connections 2024-03-04T07:48:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a6ae00640 1 -- 172.21.15.113:0/105133618 >> 172.21.15.113:0/105133618 conn(0x7f1a6406e820 msgr2=0x7f1a6406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a6ae00640 1 -- 172.21.15.113:0/105133618 shutdown_connections 2024-03-04T07:48:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.162+0000 7f1a6ae00640 1 -- 172.21.15.113:0/105133618 wait complete. 2024-03-04T07:48:07.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 Processor -- start 2024-03-04T07:48:07.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 -- start start 2024-03-04T07:48:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a48004d50 0x7f1a641e1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a641e22a0 0x7f1a640afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64155ed0 0x7f1a640b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64158ee0 con 0x7f1a641e22a0 2024-03-04T07:48:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64071a80 con 0x7f1a64155ed0 2024-03-04T07:48:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a64072280 con 0x7f1a48004d50 2024-03-04T07:48:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a5b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a641e22a0 0x7f1a640afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a5b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a641e22a0 0x7f1a640afbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32856/0 (socket says 172.21.15.113:32856) 2024-03-04T07:48:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a5b400640 1 -- 172.21.15.113:0/1029826562 learned_addr learned my addr 172.21.15.113:0/1029826562 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:48:07.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a63e00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a48004d50 0x7f1a641e1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:07.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a68e00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64155ed0 0x7f1a640b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:07.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a5b400640 1 -- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a48004d50 msgr2=0x7f1a641e1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:07.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a5b400640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a48004d50 0x7f1a641e1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a5b400640 1 -- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64155ed0 msgr2=0x7f1a640b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:07.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a5b400640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64155ed0 0x7f1a640b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a5b400640 1 -- 172.21.15.113:0/1029826562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a64155bd0 con 0x7f1a641e22a0 2024-03-04T07:48:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a5b400640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a641e22a0 0x7f1a640afbc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c011970 tx=0x7f1a4c011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a60c00640 1 -- 172.21.15.113:0/1029826562 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c019070 con 0x7f1a641e22a0 2024-03-04T07:48:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a60c00640 1 -- 172.21.15.113:0/1029826562 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c004240 con 0x7f1a641e22a0 2024-03-04T07:48:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 -- 172.21.15.113:0/1029826562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a641e1130 con 0x7f1a641e22a0 2024-03-04T07:48:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a60c00640 1 -- 172.21.15.113:0/1029826562 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c00f8e0 con 0x7f1a641e22a0 2024-03-04T07:48:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 -- 172.21.15.113:0/1029826562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a64154330 con 0x7f1a641e22a0 2024-03-04T07:48:07.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.166+0000 7f1a6ae00640 1 -- 172.21.15.113:0/1029826562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a64075c00 con 0x7f1a641e22a0 2024-03-04T07:48:07.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.170+0000 7f1a60c00640 1 -- 172.21.15.113:0/1029826562 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a4c01c070 con 0x7f1a641e22a0 2024-03-04T07:48:07.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.170+0000 7f1a60c00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a40066f60 0x7f1a40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:07.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.170+0000 7f1a60c00640 1 -- 172.21.15.113:0/1029826562 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f1a4c0858c0 con 0x7f1a641e22a0 2024-03-04T07:48:07.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.170+0000 7f1a63e00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a40066f60 0x7f1a40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:07.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.170+0000 7f1a63e00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a40066f60 0x7f1a40069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1a54014510 tx=0x7f1a5400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:07.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.174+0000 7f1a60c00640 1 -- 172.21.15.113:0/1029826562 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1a64155bd0 con 0x7f1a641e22a0 2024-03-04T07:48:07.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.382+0000 7f1a6ae00640 1 -- 172.21.15.113:0/1029826562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a64157fb0 con 0x7f1a641e22a0 2024-03-04T07:48:07.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.382+0000 7f1a60c00640 1 -- 172.21.15.113:0/1029826562 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f1a4c055eb0 con 0x7f1a641e22a0 2024-03-04T07:48:07.381 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:48:07.381 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:01.342238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5631,5529]},"id":1}]} 2024-03-04T07:48:07.382 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 60 2024-03-04T07:48:07.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 -- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a40066f60 msgr2=0x7f1a40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:07.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a40066f60 0x7f1a40069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1a54014510 tx=0x7f1a5400b040 comp rx=0 tx=0).stop 2024-03-04T07:48:07.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 -- 172.21.15.113:0/1029826562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a641e22a0 msgr2=0x7f1a640afbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:07.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a641e22a0 0x7f1a640afbc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c011970 tx=0x7f1a4c011e30 comp rx=0 tx=0).stop 2024-03-04T07:48:07.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 -- 172.21.15.113:0/1029826562 shutdown_connections 2024-03-04T07:48:07.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a64155ed0 0x7f1a640b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a40066f60 0x7f1a40069420 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a641e22a0 0x7f1a640afbc0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 --2- 172.21.15.113:0/1029826562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a48004d50 0x7f1a641e1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:07.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 -- 172.21.15.113:0/1029826562 >> 172.21.15.113:0/1029826562 conn(0x7f1a6406e820 msgr2=0x7f1a64157630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:07.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 -- 172.21.15.113:0/1029826562 shutdown_connections 2024-03-04T07:48:07.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:07.386+0000 7f1a6ae00640 1 -- 172.21.15.113:0/1029826562 wait complete. 2024-03-04T07:48:07.404 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 60, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 60, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5417": { "addr": "172.21.15.136:6835/1691762848", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1691762848, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1691762848, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5417, "incarnation": 57, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:rejoin", "state_seq": 37 }, "gid_5529": { "addr": "172.21.15.113:6836/2299268553", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2299268553, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2299268553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5529, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 43 }, "gid_5631": { "addr": "172.21.15.113:6833/810789479", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 810789479, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 810789479, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5631, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 17 } }, "last_failure": 0, "last_failure_osd_epoch": 56, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:48:01.342238+0000", "qdb_cluster": [ 5631, 5529 ], "qdb_leader": 5529, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5417, "mds_1": 5529, "mds_2": 5631 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:48:09.405 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:48:09.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f269a800640 1 Processor -- start 2024-03-04T07:48:09.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f269a800640 1 -- start start 2024-03-04T07:48:09.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f269a800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940c1860 0x7f2694162c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:09.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f269a800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2694156160 con 0x7f2694155d80 2024-03-04T07:48:09.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f269a800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269415ef10 con 0x7f26941631b0 2024-03-04T07:48:09.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f269a800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2694162500 con 0x7f26940c1860 2024-03-04T07:48:09.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2693400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2694155d80 0x7f26940c7dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39872/0 (socket says 172.21.15.113:39872) 2024-03-04T07:48:09.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2693400640 1 -- 172.21.15.113:0/3185282145 learned_addr learned my addr 172.21.15.113:0/3185282145 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:48:09.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f268be00640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940c1860 0x7f2694162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:09.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880302107 0 0) 0x7f2694156160 con 0x7f2694155d80 2024-03-04T07:48:09.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2678003530 con 0x7f2694155d80 2024-03-04T07:48:09.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859448249 0 0) 0x7f269415ef10 con 0x7f26941631b0 2024-03-04T07:48:09.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2694156160 con 0x7f26941631b0 2024-03-04T07:48:09.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2738525021 0 0) 0x7f2678003530 con 0x7f2694155d80 2024-03-04T07:48:09.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940c1860 msgr2=0x7f2694162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:09.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940c1860 0x7f2694162c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 >> v1:172.21.15.136:6789/0 conn(0x7f26941631b0 legacy=0x7f26940c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269415f1b0 con 0x7f2694155d80 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f2684001680 con 0x7f2694155d80 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 >> v1:172.21.15.113:6789/0 conn(0x7f2694155d80 legacy=0x7f26940c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940c1860 0x7f2678003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 0x7f2678004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2678004d50 0x7f26780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2694162500 con 0x7f26780040e0 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.546+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267800bab0 con 0x7f2678004d50 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.550+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267800bcb0 con 0x7f26940c1860 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.550+0000 7f268be00640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 0x7f2678004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.550+0000 7f2693400640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940c1860 0x7f2678003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.550+0000 7f2693e00640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2678004d50 0x7f26780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:09.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.550+0000 7f268be00640 1 -- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940c1860 msgr2=0x7f2678003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:09.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.550+0000 7f268be00640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26940c1860 0x7f2678003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:09.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.550+0000 7f268be00640 1 -- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2678004d50 msgr2=0x7f26780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:09.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.550+0000 7f268be00640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2678004d50 0x7f26780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:09.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.550+0000 7f268be00640 1 -- 172.21.15.113:0/3185282145 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678003530 con 0x7f26780040e0 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f268be00640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 0x7f2678004810 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f267c009ab0 tx=0x7f267c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f639786697a692 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c0b2070 con 0x7f26780040e0 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c0a5640 con 0x7f26780040e0 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f2692a00640 1 -- 172.21.15.113:0/3185282145 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c0096b0 con 0x7f26780040e0 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f269a800640 1 -- 172.21.15.113:0/3185282145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 msgr2=0x7f2678004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f269a800640 1 --2- 172.21.15.113:0/3185282145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 0x7f2678004810 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f267c009ab0 tx=0x7f267c01ca50 comp rx=0 tx=0).stop 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f2693e00640 1 -- 172.21.15.113:0/3185282145 reap_dead start 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f269a800640 1 -- 172.21.15.113:0/3185282145 shutdown_connections 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f269a800640 1 -- 172.21.15.113:0/3185282145 >> 172.21.15.113:0/3185282145 conn(0x7f269406e820 msgr2=0x7f269406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f269a800640 1 -- 172.21.15.113:0/3185282145 shutdown_connections 2024-03-04T07:48:09.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.554+0000 7f269a800640 1 -- 172.21.15.113:0/3185282145 wait complete. 2024-03-04T07:48:09.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f269a800640 1 Processor -- start 2024-03-04T07:48:09.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f269a800640 1 -- start start 2024-03-04T07:48:09.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f269a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 0x7f26941f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:09.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f269a800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2694155090 0x7f26941f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:09.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f269a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694157800 0x7f269415d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f269a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269415f9f0 con 0x7f26780040e0 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f269a800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269415f710 con 0x7f2694157800 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f269a800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269415fcd0 con 0x7f2694155090 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f2693400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 0x7f26941f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f2693400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 0x7f26941f48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49084/0 (socket says 172.21.15.113:49084) 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f2693400640 1 -- 172.21.15.113:0/4125701810 learned_addr learned my addr 172.21.15.113:0/4125701810 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f2693e00640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694157800 0x7f269415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f268be00640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2694155090 0x7f26941f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f2693400640 1 -- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2694155090 msgr2=0x7f26941f8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f2693400640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2694155090 0x7f26941f8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f2693400640 1 -- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694157800 msgr2=0x7f269415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f2693400640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694157800 0x7f269415d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f2693400640 1 -- 172.21.15.113:0/4125701810 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2694157dd0 con 0x7f26780040e0 2024-03-04T07:48:09.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f268be00640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2694155090 0x7f26941f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:48:09.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.558+0000 7f2693400640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 0x7f26941f48b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2684007290 tx=0x7f2684007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:09.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.562+0000 7f268b400640 1 -- 172.21.15.113:0/4125701810 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400f8e0 con 0x7f26780040e0 2024-03-04T07:48:09.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.562+0000 7f268b400640 1 -- 172.21.15.113:0/4125701810 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684018070 con 0x7f26780040e0 2024-03-04T07:48:09.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.562+0000 7f269a800640 1 -- 172.21.15.113:0/4125701810 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26941580b0 con 0x7f26780040e0 2024-03-04T07:48:09.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.562+0000 7f268b400640 1 -- 172.21.15.113:0/4125701810 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268400fce0 con 0x7f26780040e0 2024-03-04T07:48:09.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.562+0000 7f269a800640 1 -- 172.21.15.113:0/4125701810 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26941f8070 con 0x7f26780040e0 2024-03-04T07:48:09.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.562+0000 7f269a800640 1 -- 172.21.15.113:0/4125701810 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2694075c00 con 0x7f26780040e0 2024-03-04T07:48:09.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.562+0000 7f268b400640 1 -- 172.21.15.113:0/4125701810 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f268401d460 con 0x7f26780040e0 2024-03-04T07:48:09.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.562+0000 7f268b400640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2670066f60 0x7f2670069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:09.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.562+0000 7f268b400640 1 -- 172.21.15.113:0/4125701810 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f26840853b0 con 0x7f26780040e0 2024-03-04T07:48:09.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.566+0000 7f268be00640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2670066f60 0x7f2670069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:09.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.566+0000 7f268be00640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2670066f60 0x7f2670069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f267c0af000 tx=0x7f267c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:09.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.566+0000 7f268b400640 1 -- 172.21.15.113:0/4125701810 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2684055a40 con 0x7f26780040e0 2024-03-04T07:48:09.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.762+0000 7f269a800640 1 -- 172.21.15.113:0/4125701810 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26940c7bb0 con 0x7f26780040e0 2024-03-04T07:48:09.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.762+0000 7f268b400640 1 -- 172.21.15.113:0/4125701810 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5822 (secure 0 0 0) 0x7f2684059ae0 con 0x7f26780040e0 2024-03-04T07:48:09.762 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:48:09.762 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:06.990218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:clientreplay","state_seq":39,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:48:09.762 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 61 2024-03-04T07:48:09.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 -- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2670066f60 msgr2=0x7f2670069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:09.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2670066f60 0x7f2670069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f267c0af000 tx=0x7f267c008040 comp rx=0 tx=0).stop 2024-03-04T07:48:09.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 -- 172.21.15.113:0/4125701810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 msgr2=0x7f26941f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:09.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 0x7f26941f48b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2684007290 tx=0x7f2684007750 comp rx=0 tx=0).stop 2024-03-04T07:48:09.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 -- 172.21.15.113:0/4125701810 shutdown_connections 2024-03-04T07:48:09.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2694157800 0x7f269415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:09.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2694155090 0x7f26941f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:09.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2670066f60 0x7f2670069420 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:09.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 --2- 172.21.15.113:0/4125701810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26780040e0 0x7f26941f48b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:09.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 -- 172.21.15.113:0/4125701810 >> 172.21.15.113:0/4125701810 conn(0x7f269406e820 msgr2=0x7f2694071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:09.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 -- 172.21.15.113:0/4125701810 shutdown_connections 2024-03-04T07:48:09.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:09.766+0000 7f269a800640 1 -- 172.21.15.113:0/4125701810 wait complete. 2024-03-04T07:48:09.782 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:48:11.783 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:48:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.926+0000 7f6be8400640 1 Processor -- start 2024-03-04T07:48:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.926+0000 7f6be8400640 1 -- start start 2024-03-04T07:48:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be8400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6be00ce6b0 0x7f6be015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be8400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be006de80 0x7f6be0154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be8400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6be006c210 con 0x7f6be00ce2d0 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be8400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be015bbf0 con 0x7f6be00ce6b0 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be8400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be006eb80 con 0x7f6be006de80 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be5000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6be00ce6b0 0x7f6be015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be5000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6be00ce6b0 0x7f6be015b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49096/0 (socket says 172.21.15.113:49096) 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be5000640 1 -- 172.21.15.113:0/3445524525 learned_addr learned my addr 172.21.15.113:0/3445524525 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be5a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6be00ce2d0 0x7f6be006f010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39876/0 (socket says 172.21.15.113:39876) 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be6400640 1 --2- 172.21.15.113:0/3445524525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be006de80 0x7f6be0154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be5000640 1 -- 172.21.15.113:0/3445524525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be006de80 msgr2=0x7f6be0154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be5000640 1 --2- 172.21.15.113:0/3445524525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be006de80 0x7f6be0154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be5000640 1 -- 172.21.15.113:0/3445524525 >> v1:172.21.15.113:6789/0 conn(0x7f6be00ce2d0 legacy=0x7f6be006f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.930+0000 7f6be5000640 1 -- 172.21.15.113:0/3445524525 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be006c4f0 con 0x7f6be00ce6b0 2024-03-04T07:48:11.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be5000640 1 --2- 172.21.15.113:0/3445524525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce6b0 0x7f6be015b6b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009870 tx=0x7f6bcc01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864981346e5f8bf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:11.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6bdfe00640 1 -- 172.21.15.113:0/3445524525 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0b5070 con 0x7f6be00ce6b0 2024-03-04T07:48:11.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6bdfe00640 1 -- 172.21.15.113:0/3445524525 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc0a8a50 con 0x7f6be00ce6b0 2024-03-04T07:48:11.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6bdfe00640 1 -- 172.21.15.113:0/3445524525 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0a8e20 con 0x7f6be00ce6b0 2024-03-04T07:48:11.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 -- 172.21.15.113:0/3445524525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce6b0 msgr2=0x7f6be015b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:11.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 --2- 172.21.15.113:0/3445524525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce6b0 0x7f6be015b6b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009870 tx=0x7f6bcc01ed50 comp rx=0 tx=0).stop 2024-03-04T07:48:11.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 -- 172.21.15.113:0/3445524525 shutdown_connections 2024-03-04T07:48:11.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 --2- 172.21.15.113:0/3445524525 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be006de80 0x7f6be0154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:11.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 --2- 172.21.15.113:0/3445524525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce6b0 0x7f6be015b6b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:11.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 -- 172.21.15.113:0/3445524525 >> 172.21.15.113:0/3445524525 conn(0x7f6be00709a0 msgr2=0x7f6be0070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:11.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 -- 172.21.15.113:0/3445524525 shutdown_connections 2024-03-04T07:48:11.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 -- 172.21.15.113:0/3445524525 wait complete. 2024-03-04T07:48:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 Processor -- start 2024-03-04T07:48:11.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 -- start start 2024-03-04T07:48:11.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be006de80 0x7f6be0145500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce2d0 0x7f6be0145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6be00ce6b0 0x7f6be0146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be006c7d0 con 0x7f6be00ce2d0 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be006e810 con 0x7f6be00ce6b0 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.934+0000 7f6be8400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be006f720 con 0x7f6be006de80 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce2d0 0x7f6be0145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce2d0 0x7f6be0145f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49100/0 (socket says 172.21.15.113:49100) 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be5000640 1 -- 172.21.15.113:0/3337783515 learned_addr learned my addr 172.21.15.113:0/3337783515 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be5a00640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be006de80 0x7f6be0145500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be6400640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6be00ce6b0 0x7f6be0146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be5000640 1 -- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be006de80 msgr2=0x7f6be0145500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:11.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be5000640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be006de80 0x7f6be0145500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be5000640 1 -- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6be00ce6b0 msgr2=0x7f6be0146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be5000640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6be00ce6b0 0x7f6be0146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be5000640 1 -- 172.21.15.113:0/3337783515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be0150b40 con 0x7f6be00ce2d0 2024-03-04T07:48:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be5000640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce2d0 0x7f6be0145f50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b3040 tx=0x7f6bcc01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6bdd600640 1 -- 172.21.15.113:0/3337783515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0c0070 con 0x7f6be00ce2d0 2024-03-04T07:48:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6bdd600640 1 -- 172.21.15.113:0/3337783515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc0ae550 con 0x7f6be00ce2d0 2024-03-04T07:48:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be8400640 1 -- 172.21.15.113:0/3337783515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be0150db0 con 0x7f6be00ce2d0 2024-03-04T07:48:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6bdd600640 1 -- 172.21.15.113:0/3337783515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0b5070 con 0x7f6be00ce2d0 2024-03-04T07:48:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be8400640 1 -- 172.21.15.113:0/3337783515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be0151370 con 0x7f6be00ce2d0 2024-03-04T07:48:11.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.938+0000 7f6be8400640 1 -- 172.21.15.113:0/3337783515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be0075c00 con 0x7f6be00ce2d0 2024-03-04T07:48:11.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.942+0000 7f6bdd600640 1 -- 172.21.15.113:0/3337783515 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6bcc0c3070 con 0x7f6be00ce2d0 2024-03-04T07:48:11.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.942+0000 7f6bdd600640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc0066f60 0x7f6bc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:11.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.942+0000 7f6bdd600640 1 -- 172.21.15.113:0/3337783515 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f6bcc122b70 con 0x7f6be00ce2d0 2024-03-04T07:48:11.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.942+0000 7f6be5a00640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc0066f60 0x7f6bc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:11.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.942+0000 7f6be5a00640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc0066f60 0x7f6bc0069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4000f70 tx=0x7f6bd4007cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:11.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:11.950+0000 7f6bdd600640 1 -- 172.21.15.113:0/3337783515 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6bcc0188b0 con 0x7f6be00ce2d0 2024-03-04T07:48:12.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.146+0000 7f6be8400640 1 -- 172.21.15.113:0/3337783515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6be006fdf0 con 0x7f6be00ce2d0 2024-03-04T07:48:12.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6bdd600640 1 -- 172.21.15.113:0/3337783515 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5822 (secure 0 0 0) 0x7f6bcc0f3850 con 0x7f6be00ce2d0 2024-03-04T07:48:12.147 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:48:12.148 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:06.990218+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:clientreplay","state_seq":39,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:48:12.148 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 61 2024-03-04T07:48:12.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 -- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc0066f60 msgr2=0x7f6bc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:12.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc0066f60 0x7f6bc0069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4000f70 tx=0x7f6bd4007cd0 comp rx=0 tx=0).stop 2024-03-04T07:48:12.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 -- 172.21.15.113:0/3337783515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce2d0 msgr2=0x7f6be0145f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:12.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce2d0 0x7f6be0145f50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b3040 tx=0x7f6bcc01baf0 comp rx=0 tx=0).stop 2024-03-04T07:48:12.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 -- 172.21.15.113:0/3337783515 shutdown_connections 2024-03-04T07:48:12.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6be00ce6b0 0x7f6be0146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6bc0066f60 0x7f6bc0069420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6be00ce2d0 0x7f6be0145f50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 --2- 172.21.15.113:0/3337783515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6be006de80 0x7f6be0145500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 -- 172.21.15.113:0/3337783515 >> 172.21.15.113:0/3337783515 conn(0x7f6be00709a0 msgr2=0x7f6be014cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 -- 172.21.15.113:0/3337783515 shutdown_connections 2024-03-04T07:48:12.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:12.150+0000 7f6be8400640 1 -- 172.21.15.113:0/3337783515 wait complete. 2024-03-04T07:48:12.174 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:48:14.175 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:48:14.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa806200640 1 Processor -- start 2024-03-04T07:48:14.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa806200640 1 -- start start 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa806200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8001527f0 0x7fa800070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa806200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa8000c8200 0x7fa800072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa806200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8001591f0 con 0x7fa800152bd0 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa806200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa800071df0 con 0x7fa8001527f0 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa806200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8001597b0 con 0x7fa8000c8200 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa7ff400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8001527f0 0x7fa800070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa7ffe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa800152bd0 0x7fa8001587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39886/0 (socket says 172.21.15.113:39886) 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa7ffe00640 1 -- 172.21.15.113:0/4231533380 learned_addr learned my addr 172.21.15.113:0/4231533380 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa7fea00640 1 --2- 172.21.15.113:0/4231533380 >> v2:172.21.15.136:3300/0 conn(0x7fa8000c8200 0x7fa800072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa7ff400640 1 -- 172.21.15.113:0/4231533380 >> v2:172.21.15.136:3300/0 conn(0x7fa8000c8200 msgr2=0x7fa800072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa7ff400640 1 --2- 172.21.15.113:0/4231533380 >> v2:172.21.15.136:3300/0 conn(0x7fa8000c8200 0x7fa800072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa7ff400640 1 -- 172.21.15.113:0/4231533380 >> v1:172.21.15.113:6789/0 conn(0x7fa800152bd0 legacy=0x7fa8001587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa7ff400640 1 -- 172.21.15.113:0/4231533380 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8001594d0 con 0x7fa8001527f0 2024-03-04T07:48:14.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.318+0000 7fa7fe000640 1 -- 172.21.15.113:0/4231533380 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822791321 0 0) 0x7fa8001591f0 con 0x7fa800152bd0 2024-03-04T07:48:14.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa7ff400640 1 --2- 172.21.15.113:0/4231533380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 0x7fa800070d40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa8000cd230 tx=0x7fa7f001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4e1c22dcfb6f5f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:14.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa7fe000640 1 -- 172.21.15.113:0/4231533380 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00b2070 con 0x7fa8001527f0 2024-03-04T07:48:14.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa7fe000640 1 -- 172.21.15.113:0/4231533380 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7f00a58e0 con 0x7fa8001527f0 2024-03-04T07:48:14.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa7fe000640 1 -- 172.21.15.113:0/4231533380 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00a5cb0 con 0x7fa8001527f0 2024-03-04T07:48:14.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa806200640 1 -- 172.21.15.113:0/4231533380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 msgr2=0x7fa800070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:14.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa806200640 1 --2- 172.21.15.113:0/4231533380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 0x7fa800070d40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa8000cd230 tx=0x7fa7f001c930 comp rx=0 tx=0).stop 2024-03-04T07:48:14.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa806200640 1 -- 172.21.15.113:0/4231533380 shutdown_connections 2024-03-04T07:48:14.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa806200640 1 --2- 172.21.15.113:0/4231533380 >> v2:172.21.15.136:3300/0 conn(0x7fa8000c8200 0x7fa800072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:14.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa806200640 1 --2- 172.21.15.113:0/4231533380 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 0x7fa800070d40 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:14.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa806200640 1 -- 172.21.15.113:0/4231533380 >> 172.21.15.113:0/4231533380 conn(0x7fa80006e820 msgr2=0x7fa80006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:14.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa806200640 1 -- 172.21.15.113:0/4231533380 shutdown_connections 2024-03-04T07:48:14.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.322+0000 7fa806200640 1 -- 172.21.15.113:0/4231533380 wait complete. 2024-03-04T07:48:14.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 Processor -- start 2024-03-04T07:48:14.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 -- start start 2024-03-04T07:48:14.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8000c8200 0x7fa80014a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:14.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 0x7fa8000c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:14.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800152bd0 0x7fa8000c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:14.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa800165160 con 0x7fa8001527f0 2024-03-04T07:48:14.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8000727c0 con 0x7fa800152bd0 2024-03-04T07:48:14.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8001652e0 con 0x7fa8000c8200 2024-03-04T07:48:14.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7fea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 0x7fa8000c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:14.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7fea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 0x7fa8000c6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49126/0 (socket says 172.21.15.113:49126) 2024-03-04T07:48:14.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7fea00640 1 -- 172.21.15.113:0/1898918469 learned_addr learned my addr 172.21.15.113:0/1898918469 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7ff400640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8000c8200 0x7fa80014a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7ffe00640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800152bd0 0x7fa8000c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7fea00640 1 -- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8000c8200 msgr2=0x7fa80014a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7fea00640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8000c8200 0x7fa80014a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7fea00640 1 -- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800152bd0 msgr2=0x7fa8000c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7fea00640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800152bd0 0x7fa8000c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7fea00640 1 -- 172.21.15.113:0/1898918469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80014acd0 con 0x7fa8001527f0 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7ff400640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8000c8200 0x7fa80014a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7fea00640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 0x7fa8000c6cb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00d800 tx=0x7fa7ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7f7400640 1 -- 172.21.15.113:0/1898918469 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00b6c0 con 0x7fa8001527f0 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7f7400640 1 -- 172.21.15.113:0/1898918469 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec019070 con 0x7fa8001527f0 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 -- 172.21.15.113:0/1898918469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80014dd10 con 0x7fa8001527f0 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa7f7400640 1 -- 172.21.15.113:0/1898918469 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00baa0 con 0x7fa8001527f0 2024-03-04T07:48:14.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 -- 172.21.15.113:0/1898918469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80014da70 con 0x7fa8001527f0 2024-03-04T07:48:14.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.326+0000 7fa806200640 1 -- 172.21.15.113:0/1898918469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa800075c00 con 0x7fa8001527f0 2024-03-04T07:48:14.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.330+0000 7fa7f7400640 1 -- 172.21.15.113:0/1898918469 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7ec00bc40 con 0x7fa8001527f0 2024-03-04T07:48:14.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.330+0000 7fa7f7400640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa7e0066f60 0x7fa7e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:14.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.330+0000 7fa7f7400640 1 -- 172.21.15.113:0/1898918469 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fa7ec0857e0 con 0x7fa8001527f0 2024-03-04T07:48:14.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.330+0000 7fa7ff400640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa7e0066f60 0x7fa7e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:14.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.330+0000 7fa7ff400640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa7e0066f60 0x7fa7e0069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00af000 tx=0x7fa7f00a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:14.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.334+0000 7fa7f7400640 1 -- 172.21.15.113:0/1898918469 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa7ec0564b0 con 0x7fa8001527f0 2024-03-04T07:48:14.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.662+0000 7fa806200640 1 -- 172.21.15.113:0/1898918469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8000c79d0 con 0x7fa8001527f0 2024-03-04T07:48:14.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.662+0000 7fa7f7400640 1 -- 172.21.15.113:0/1898918469 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+5821 (secure 0 0 0) 0x7fa7ec05ae20 con 0x7fa8001527f0 2024-03-04T07:48:14.661 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:48:14.661 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:13.835609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":41,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5417,"qdb_cluster":[5417,5529,5631]},"id":1}]} 2024-03-04T07:48:14.662 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 62 2024-03-04T07:48:14.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 -- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa7e0066f60 msgr2=0x7fa7e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:14.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa7e0066f60 0x7fa7e0069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00af000 tx=0x7fa7f00a7070 comp rx=0 tx=0).stop 2024-03-04T07:48:14.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 -- 172.21.15.113:0/1898918469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 msgr2=0x7fa8000c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:14.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 0x7fa8000c6cb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00d800 tx=0x7fa7ec00dcc0 comp rx=0 tx=0).stop 2024-03-04T07:48:14.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 -- 172.21.15.113:0/1898918469 shutdown_connections 2024-03-04T07:48:14.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa800152bd0 0x7fa8000c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:14.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa7e0066f60 0x7fa7e0069420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:14.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8001527f0 0x7fa8000c6cb0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:14.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 --2- 172.21.15.113:0/1898918469 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8000c8200 0x7fa80014a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:14.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 -- 172.21.15.113:0/1898918469 >> 172.21.15.113:0/1898918469 conn(0x7fa80006e820 msgr2=0x7fa8001520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:14.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 -- 172.21.15.113:0/1898918469 shutdown_connections 2024-03-04T07:48:14.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:14.666+0000 7fa806200640 1 -- 172.21.15.113:0/1898918469 wait complete. 2024-03-04T07:48:14.692 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=0, replacing gid=5244 2024-03-04T07:48:14.692 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.e 2024-03-04T07:48:20.784 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T07:48:20.788+0000 7f31e1bd5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T07:48:20.784 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T07:48:20.788+0000 7f31e1bd5640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T07:48:20.785 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T07:48:20.788+0000 7f31e1bd5640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-04T07:48:22.693 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-03-04T07:48:22.694 INFO:tasks.ceph.mds.e:Restarting daemon 2024-03-04T07:48:22.694 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-03-04T07:48:22.697 INFO:tasks.ceph.mds.e:Started 2024-03-04T07:48:22.698 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:48:22.783 INFO:tasks.ceph.mds.e.smithi113.stdout:starting mds.e at 2024-03-04T07:48:22.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa82d600640 1 Processor -- start 2024-03-04T07:48:22.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa82d600640 1 -- start start 2024-03-04T07:48:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa82d600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8281527f0 0x7fa828070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa82d600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa828152bd0 0x7fa8281587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa82d600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8281591f0 con 0x7fa8280c8200 2024-03-04T07:48:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa82d600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa828071df0 con 0x7fa8281527f0 2024-03-04T07:48:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa82d600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8281597b0 con 0x7fa828152bd0 2024-03-04T07:48:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa826a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8281527f0 0x7fa828070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa827400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa828152bd0 0x7fa8281587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa81e000640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fa8280c8200 0x7fa828072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:53702/0 (socket says 172.21.15.113:53702) 2024-03-04T07:48:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa81e000640 1 -- 172.21.15.113:0/323522259 learned_addr learned my addr 172.21.15.113:0/323522259 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:48:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa826a00640 1 -- 172.21.15.113:0/323522259 >> v2:172.21.15.136:3300/0 conn(0x7fa828152bd0 msgr2=0x7fa8281587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:48:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa826a00640 1 --2- 172.21.15.113:0/323522259 >> v2:172.21.15.136:3300/0 conn(0x7fa828152bd0 0x7fa8281587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa826a00640 1 -- 172.21.15.113:0/323522259 >> v1:172.21.15.136:6789/0 conn(0x7fa8280c8200 legacy=0x7fa828072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.870+0000 7fa826a00640 1 -- 172.21.15.113:0/323522259 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8281594d0 con 0x7fa8281527f0 2024-03-04T07:48:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa826a00640 1 --2- 172.21.15.113:0/323522259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8281527f0 0x7fa828070d40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa8280712b0 tx=0x7fa81801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c1d4b9f9e40a54 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:22.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa826000640 1 -- 172.21.15.113:0/323522259 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8180b1070 con 0x7fa8281527f0 2024-03-04T07:48:22.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa826000640 1 -- 172.21.15.113:0/323522259 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8180ac5f0 con 0x7fa8281527f0 2024-03-04T07:48:22.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa826000640 1 -- 172.21.15.113:0/323522259 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8180ac9c0 con 0x7fa8281527f0 2024-03-04T07:48:22.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa82d600640 1 -- 172.21.15.113:0/323522259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8281527f0 msgr2=0x7fa828070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:22.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa82d600640 1 --2- 172.21.15.113:0/323522259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8281527f0 0x7fa828070d40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa8280712b0 tx=0x7fa81801c480 comp rx=0 tx=0).stop 2024-03-04T07:48:22.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa82d600640 1 -- 172.21.15.113:0/323522259 shutdown_connections 2024-03-04T07:48:22.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa82d600640 1 --2- 172.21.15.113:0/323522259 >> v2:172.21.15.136:3300/0 conn(0x7fa828152bd0 0x7fa8281587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:22.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa82d600640 1 --2- 172.21.15.113:0/323522259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8281527f0 0x7fa828070d40 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:22.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa82d600640 1 -- 172.21.15.113:0/323522259 >> 172.21.15.113:0/323522259 conn(0x7fa82806e820 msgr2=0x7fa82806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:22.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa82d600640 1 -- 172.21.15.113:0/323522259 shutdown_connections 2024-03-04T07:48:22.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.874+0000 7fa82d600640 1 -- 172.21.15.113:0/323522259 wait complete. 2024-03-04T07:48:22.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa82d600640 1 Processor -- start 2024-03-04T07:48:22.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa82d600640 1 -- start start 2024-03-04T07:48:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa82d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8280c8200 0x7fa828153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa82d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8281527f0 0x7fa8280b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa82d600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa828152bd0 0x7fa8281f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:22.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa82d600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8281654f0 con 0x7fa8280c8200 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa82d600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa828165370 con 0x7fa8281527f0 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa82d600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa828165670 con 0x7fa828152bd0 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa826a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8280c8200 0x7fa828153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa827400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa828152bd0 0x7fa8281f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa826a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8280c8200 0x7fa828153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41810/0 (socket says 172.21.15.113:41810) 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa826a00640 1 -- 172.21.15.113:0/462413482 learned_addr learned my addr 172.21.15.113:0/462413482 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa81e000640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8281527f0 0x7fa8280b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa826a00640 1 -- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa828152bd0 msgr2=0x7fa8281f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa826a00640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa828152bd0 0x7fa8281f1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa826a00640 1 -- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8281527f0 msgr2=0x7fa8280b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa826a00640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8281527f0 0x7fa8280b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:22.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa826a00640 1 -- 172.21.15.113:0/462413482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8280b14c0 con 0x7fa8280c8200 2024-03-04T07:48:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa81e000640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8281527f0 0x7fa8280b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:48:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa826a00640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8280c8200 0x7fa828153b10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa81801ab30 tx=0x7fa818019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa81f400640 1 -- 172.21.15.113:0/462413482 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8180b1040 con 0x7fa8280c8200 2024-03-04T07:48:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa81f400640 1 -- 172.21.15.113:0/462413482 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8180bb070 con 0x7fa8280c8200 2024-03-04T07:48:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa81f400640 1 -- 172.21.15.113:0/462413482 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818006e20 con 0x7fa8280c8200 2024-03-04T07:48:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa82d600640 1 -- 172.21.15.113:0/462413482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa828151a30 con 0x7fa8280c8200 2024-03-04T07:48:22.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.878+0000 7fa82d600640 1 -- 172.21.15.113:0/462413482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa828151fd0 con 0x7fa8280c8200 2024-03-04T07:48:22.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.882+0000 7fa82d600640 1 -- 172.21.15.113:0/462413482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa828075c00 con 0x7fa8280c8200 2024-03-04T07:48:22.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.882+0000 7fa81f400640 1 -- 172.21.15.113:0/462413482 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8180b5650 con 0x7fa8280c8200 2024-03-04T07:48:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.882+0000 7fa81f400640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa804066f60 0x7fa804069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:48:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.882+0000 7fa81f400640 1 -- 172.21.15.113:0/462413482 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fa81811edd0 con 0x7fa8280c8200 2024-03-04T07:48:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.882+0000 7fa81e000640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa804066f60 0x7fa804069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:48:22.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.886+0000 7fa81e000640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa804066f60 0x7fa804069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa8280c7f10 tx=0x7fa810009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:48:22.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:22.890+0000 7fa81f400640 1 -- 172.21.15.113:0/462413482 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa8180a9070 con 0x7fa8280c8200 2024-03-04T07:48:23.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.082+0000 7fa82d600640 1 -- 172.21.15.113:0/462413482 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa828068350 con 0x7fa8280c8200 2024-03-04T07:48:23.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.082+0000 7fa81f400640 1 -- 172.21.15.113:0/462413482 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fa818120070 con 0x7fa8280c8200 2024-03-04T07:48:23.082 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:48:23.082 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5940,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:13.835609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":41,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5417,"qdb_cluster":[5631,5529,5417]},"id":1}]} 2024-03-04T07:48:23.082 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 63 2024-03-04T07:48:23.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 -- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa804066f60 msgr2=0x7fa804069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:23.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa804066f60 0x7fa804069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa8280c7f10 tx=0x7fa810009040 comp rx=0 tx=0).stop 2024-03-04T07:48:23.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 -- 172.21.15.113:0/462413482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8280c8200 msgr2=0x7fa828153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:48:23.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8280c8200 0x7fa828153b10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa81801ab30 tx=0x7fa818019220 comp rx=0 tx=0).stop 2024-03-04T07:48:23.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 -- 172.21.15.113:0/462413482 shutdown_connections 2024-03-04T07:48:23.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa828152bd0 0x7fa8281f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:23.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8281527f0 0x7fa8280b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:23.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa804066f60 0x7fa804069420 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:23.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 --2- 172.21.15.113:0/462413482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8280c8200 0x7fa828153b10 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:48:23.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 -- 172.21.15.113:0/462413482 >> 172.21.15.113:0/462413482 conn(0x7fa82806e820 msgr2=0x7fa8281581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:48:23.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 -- 172.21.15.113:0/462413482 shutdown_connections 2024-03-04T07:48:23.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:48:23.086+0000 7fa82d600640 1 -- 172.21.15.113:0/462413482 wait complete. 2024-03-04T07:48:23.107 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-03-04T07:48:23.107 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 52 secs before thrashing 2024-03-04T07:48:28.758 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:48:28.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:48:58.813 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:48:58.818 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:49:15.109 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:15.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f9715800640 1 Processor -- start 2024-03-04T07:49:15.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f9715800640 1 -- start start 2024-03-04T07:49:15.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f9715800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97100ca1a0 0x7f97100b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f9715800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97100ca580 0x7f971015e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f9715800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f971015e690 0x7f97100cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f9715800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97100b3530 con 0x7f971015e690 2024-03-04T07:49:15.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f9715800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9710067bc0 con 0x7f97100ca1a0 2024-03-04T07:49:15.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f9715800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971015bb40 con 0x7f97100ca580 2024-03-04T07:49:15.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f970f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f971015e690 0x7f97100cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f970f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f971015e690 0x7f97100cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37664/0 (socket says 172.21.15.113:37664) 2024-03-04T07:49:15.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f970f400640 1 -- 172.21.15.113:0/3116589573 learned_addr learned my addr 172.21.15.113:0/3116589573 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:15.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f970ea00640 1 --2- 172.21.15.113:0/3116589573 >> v2:172.21.15.136:3300/0 conn(0x7f97100ca1a0 0x7f97100b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f9707e00640 1 --2- 172.21.15.113:0/3116589573 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97100ca580 0x7f971015e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f970f400640 1 -- 172.21.15.113:0/3116589573 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97100ca580 msgr2=0x7f971015e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:15.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f970f400640 1 --2- 172.21.15.113:0/3116589573 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97100ca580 0x7f971015e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f970f400640 1 -- 172.21.15.113:0/3116589573 >> v2:172.21.15.136:3300/0 conn(0x7f97100ca1a0 msgr2=0x7f97100b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:15.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f970f400640 1 --2- 172.21.15.113:0/3116589573 >> v2:172.21.15.136:3300/0 conn(0x7f97100ca1a0 0x7f97100b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.282+0000 7f970f400640 1 -- 172.21.15.113:0/3116589573 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97100678e0 con 0x7f971015e690 2024-03-04T07:49:15.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f970f400640 1 --2- 172.21.15.113:0/3116589573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f971015e690 0x7f97100cca10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9708009e50 tx=0x7f970801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29faa61f657aaac server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:15.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f970e000640 1 -- 172.21.15.113:0/3116589573 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97080b4070 con 0x7f971015e690 2024-03-04T07:49:15.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f970e000640 1 -- 172.21.15.113:0/3116589573 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97080af400 con 0x7f971015e690 2024-03-04T07:49:15.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f970e000640 1 -- 172.21.15.113:0/3116589573 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97080af7d0 con 0x7f971015e690 2024-03-04T07:49:15.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f9715800640 1 -- 172.21.15.113:0/3116589573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f971015e690 msgr2=0x7f97100cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f9715800640 1 --2- 172.21.15.113:0/3116589573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f971015e690 0x7f97100cca10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9708009e50 tx=0x7f970801f1e0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f9715800640 1 -- 172.21.15.113:0/3116589573 shutdown_connections 2024-03-04T07:49:15.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f9715800640 1 --2- 172.21.15.113:0/3116589573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f971015e690 0x7f97100cca10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f9715800640 1 --2- 172.21.15.113:0/3116589573 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97100ca580 0x7f971015e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f9715800640 1 --2- 172.21.15.113:0/3116589573 >> v2:172.21.15.136:3300/0 conn(0x7f97100ca1a0 0x7f97100b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f9715800640 1 -- 172.21.15.113:0/3116589573 >> 172.21.15.113:0/3116589573 conn(0x7f971006dac0 msgr2=0x7f9710066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:15.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f9715800640 1 -- 172.21.15.113:0/3116589573 shutdown_connections 2024-03-04T07:49:15.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.286+0000 7f9715800640 1 -- 172.21.15.113:0/3116589573 wait complete. 2024-03-04T07:49:15.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9715800640 1 Processor -- start 2024-03-04T07:49:15.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9715800640 1 -- start start 2024-03-04T07:49:15.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9715800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97100ca1a0 0x7f97101e7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9715800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97100ca580 0x7f97100cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9715800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f971015e690 0x7f9710151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9715800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971015b340 con 0x7f97100ca580 2024-03-04T07:49:15.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9715800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971015b060 con 0x7f97100ca1a0 2024-03-04T07:49:15.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9715800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971015b620 con 0x7f971015e690 2024-03-04T07:49:15.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9707e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97100ca580 0x7f97100cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f970ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97100ca1a0 0x7f97101e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9707e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97100ca580 0x7f97100cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37666/0 (socket says 172.21.15.113:37666) 2024-03-04T07:49:15.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f9707e00640 1 -- 172.21.15.113:0/4128997080 learned_addr learned my addr 172.21.15.113:0/4128997080 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:15.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.290+0000 7f970f400640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f971015e690 0x7f9710151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9707e00640 1 -- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f971015e690 msgr2=0x7f9710151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9707e00640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f971015e690 0x7f9710151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9707e00640 1 -- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97100ca1a0 msgr2=0x7f97101e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9707e00640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97100ca1a0 0x7f97101e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9707e00640 1 -- 172.21.15.113:0/4128997080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97101e59a0 con 0x7f97100ca580 2024-03-04T07:49:15.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9707e00640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97100ca580 0x7f97100cf030 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f96f800d800 tx=0x7f96f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:15.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9706a00640 1 -- 172.21.15.113:0/4128997080 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f800b6c0 con 0x7f97100ca580 2024-03-04T07:49:15.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9706a00640 1 -- 172.21.15.113:0/4128997080 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8019070 con 0x7f97100ca580 2024-03-04T07:49:15.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9715800640 1 -- 172.21.15.113:0/4128997080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97101e5c40 con 0x7f97100ca580 2024-03-04T07:49:15.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9706a00640 1 -- 172.21.15.113:0/4128997080 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f800ba60 con 0x7f97100ca580 2024-03-04T07:49:15.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9715800640 1 -- 172.21.15.113:0/4128997080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97101f40b0 con 0x7f97100ca580 2024-03-04T07:49:15.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9715800640 1 -- 172.21.15.113:0/4128997080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9710075c00 con 0x7f97100ca580 2024-03-04T07:49:15.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9706a00640 1 -- 172.21.15.113:0/4128997080 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96f800bc00 con 0x7f97100ca580 2024-03-04T07:49:15.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9706a00640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f96ec066f90 0x7f96ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.294+0000 7f9706a00640 1 -- 172.21.15.113:0/4128997080 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f96f8085740 con 0x7f97100ca580 2024-03-04T07:49:15.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.298+0000 7f970ea00640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f96ec066f90 0x7f96ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.298+0000 7f970ea00640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f96ec066f90 0x7f96ec069450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9700009ae0 tx=0x7f9700009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:15.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.298+0000 7f9706a00640 1 -- 172.21.15.113:0/4128997080 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f96f8055d30 con 0x7f97100ca580 2024-03-04T07:49:15.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 -- 172.21.15.113:0/4128997080 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97100cbdd0 con 0x7f97100ca580 2024-03-04T07:49:15.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9706a00640 1 -- 172.21.15.113:0/4128997080 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f96f8059dd0 con 0x7f97100ca580 2024-03-04T07:49:15.488 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:15.488 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5940,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:13.835609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":41,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5417,"qdb_cluster":[5631,5529,5417]},"id":1}]} 2024-03-04T07:49:15.488 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 63 2024-03-04T07:49:15.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 -- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f96ec066f90 msgr2=0x7f96ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f96ec066f90 0x7f96ec069450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9700009ae0 tx=0x7f9700009190 comp rx=0 tx=0).stop 2024-03-04T07:49:15.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 -- 172.21.15.113:0/4128997080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97100ca580 msgr2=0x7f97100cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97100ca580 0x7f97100cf030 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f96f800d800 tx=0x7f96f800dcc0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 -- 172.21.15.113:0/4128997080 shutdown_connections 2024-03-04T07:49:15.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f971015e690 0x7f9710151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f96ec066f90 0x7f96ec069450 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97100ca580 0x7f97100cf030 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 --2- 172.21.15.113:0/4128997080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97100ca1a0 0x7f97101e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 -- 172.21.15.113:0/4128997080 >> 172.21.15.113:0/4128997080 conn(0x7f971006dac0 msgr2=0x7f9710159630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:15.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 -- 172.21.15.113:0/4128997080 shutdown_connections 2024-03-04T07:49:15.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.490+0000 7f9715800640 1 -- 172.21.15.113:0/4128997080 wait complete. 2024-03-04T07:49:15.509 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=0) 2024-03-04T07:49:15.509 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-04T07:49:15.509 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:49:15.523 INFO:tasks.ceph.mds.b:Stopped 2024-03-04T07:49:15.528 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:15.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.670+0000 7fb1f9800640 1 Processor -- start 2024-03-04T07:49:15.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.670+0000 7fb1f9800640 1 -- start start 2024-03-04T07:49:15.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.670+0000 7fb1f9800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb1f41527f0 0x7fb1f4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.670+0000 7fb1f9800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f41591f0 con 0x7fb1f4152bd0 2024-03-04T07:49:15.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.670+0000 7fb1f9800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f41594d0 con 0x7fb1f40c8200 2024-03-04T07:49:15.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.670+0000 7fb1f9800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f4071df0 con 0x7fb1f41527f0 2024-03-04T07:49:15.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.670+0000 7fb1f3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb1f4152bd0 0x7fb1f41587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36060/0 (socket says 172.21.15.113:36060) 2024-03-04T07:49:15.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.670+0000 7fb1f3400640 1 -- 172.21.15.113:0/3459898478 learned_addr learned my addr 172.21.15.113:0/3459898478 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:49:15.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2a00640 1 --2- 172.21.15.113:0/3459898478 >> v2:172.21.15.136:3300/0 conn(0x7fb1f41527f0 0x7fb1f4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277314417 0 0) 0x7fb1f41591f0 con 0x7fb1f4152bd0 2024-03-04T07:49:15.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d8003530 con 0x7fb1f4152bd0 2024-03-04T07:49:15.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1609289532 0 0) 0x7fb1d8003530 con 0x7fb1f4152bd0 2024-03-04T07:49:15.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 >> v2:172.21.15.136:3300/0 conn(0x7fb1f41527f0 msgr2=0x7fb1f4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:15.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 --2- 172.21.15.113:0/3459898478 >> v2:172.21.15.136:3300/0 conn(0x7fb1f41527f0 0x7fb1f4070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 >> v1:172.21.15.136:6789/0 conn(0x7fb1f40c8200 legacy=0x7fb1f4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f40717a0 con 0x7fb1f4152bd0 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fb1ec001a00 con 0x7fb1f4152bd0 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 >> v1:172.21.15.113:6789/0 conn(0x7fb1f4152bd0 legacy=0x7fb1f41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1d8003a10 0x7fb1d8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1d80051d0 0x7fb1d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f4071df0 con 0x7fb1d8004810 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d800c0d0 con 0x7fb1d80051d0 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d800c2d0 con 0x7fb1d8003a10 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1ebe00640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f3400640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1d80051d0 0x7fb1d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f2a00640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1d8003a10 0x7fb1d8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1ebe00640 1 -- 172.21.15.113:0/3459898478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1d8003a10 msgr2=0x7fb1d8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:15.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1ebe00640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1d8003a10 0x7fb1d8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1ebe00640 1 -- 172.21.15.113:0/3459898478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1d80051d0 msgr2=0x7fb1d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:15.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1ebe00640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1d80051d0 0x7fb1d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1ebe00640 1 -- 172.21.15.113:0/3459898478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d8003530 con 0x7fb1d8004810 2024-03-04T07:49:15.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.674+0000 7fb1f3400640 1 -- 172.21.15.113:0/3459898478 reap_dead start 2024-03-04T07:49:15.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1ebe00640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1d8004c90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc001750 tx=0x7fb1dc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a09603f531c1df server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:15.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0013a0 con 0x7fb1d8004810 2024-03-04T07:49:15.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc0052f0 con 0x7fb1d8004810 2024-03-04T07:49:15.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f2000640 1 -- 172.21.15.113:0/3459898478 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc003840 con 0x7fb1d8004810 2024-03-04T07:49:15.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 -- 172.21.15.113:0/3459898478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 msgr2=0x7fb1d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1d8004c90 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc001750 tx=0x7fb1dc01c3c0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 -- 172.21.15.113:0/3459898478 shutdown_connections 2024-03-04T07:49:15.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 --2- 172.21.15.113:0/3459898478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1d8004c90 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 -- 172.21.15.113:0/3459898478 >> 172.21.15.113:0/3459898478 conn(0x7fb1f406e820 msgr2=0x7fb1f406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:15.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 -- 172.21.15.113:0/3459898478 shutdown_connections 2024-03-04T07:49:15.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 -- 172.21.15.113:0/3459898478 wait complete. 2024-03-04T07:49:15.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 Processor -- start 2024-03-04T07:49:15.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 -- start start 2024-03-04T07:49:15.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1f4154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1f41e26e0 0x7fb1f4155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f414dcd0 0x7fb1f414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f4070d40 con 0x7fb1d8004810 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f41597b0 con 0x7fb1f414dcd0 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.678+0000 7fb1f9800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f4070ec0 con 0x7fb1f41e26e0 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1f4154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1f4154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37684/0 (socket says 172.21.15.113:37684) 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f2a00640 1 -- 172.21.15.113:0/1974904440 learned_addr learned my addr 172.21.15.113:0/1974904440 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f3400640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f414dcd0 0x7fb1f414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1ebe00640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1f41e26e0 0x7fb1f4155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f2a00640 1 -- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1f41e26e0 msgr2=0x7fb1f4155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f2a00640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1f41e26e0 0x7fb1f4155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f2a00640 1 -- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f414dcd0 msgr2=0x7fb1f414e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f2a00640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f414dcd0 0x7fb1f414e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f2a00640 1 -- 172.21.15.113:0/1974904440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f41ec630 con 0x7fb1d8004810 2024-03-04T07:49:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1ebe00640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1f41e26e0 0x7fb1f4155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:15.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f2a00640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1f4154a10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4011500 tx=0x7fb1e40119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:15.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1eaa00640 1 -- 172.21.15.113:0/1974904440 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4015870 con 0x7fb1d8004810 2024-03-04T07:49:15.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1eaa00640 1 -- 172.21.15.113:0/1974904440 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e4015a10 con 0x7fb1d8004810 2024-03-04T07:49:15.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1eaa00640 1 -- 172.21.15.113:0/1974904440 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4015dc0 con 0x7fb1d8004810 2024-03-04T07:49:15.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f9800640 1 -- 172.21.15.113:0/1974904440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f4072c40 con 0x7fb1d8004810 2024-03-04T07:49:15.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f9800640 1 -- 172.21.15.113:0/1974904440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f4157ab0 con 0x7fb1d8004810 2024-03-04T07:49:15.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.682+0000 7fb1f9800640 1 -- 172.21.15.113:0/1974904440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f4075c00 con 0x7fb1d8004810 2024-03-04T07:49:15.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.686+0000 7fb1eaa00640 1 -- 172.21.15.113:0/1974904440 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1e4026420 con 0x7fb1d8004810 2024-03-04T07:49:15.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.686+0000 7fb1eaa00640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1d0066f60 0x7fb1d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:15.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.686+0000 7fb1eaa00640 1 -- 172.21.15.113:0/1974904440 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fb1e4085bf0 con 0x7fb1d8004810 2024-03-04T07:49:15.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.686+0000 7fb1ebe00640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1d0066f60 0x7fb1d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:15.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.686+0000 7fb1ebe00640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1d0066f60 0x7fb1d0069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40c85e0 tx=0x7fb1dc0011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:15.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.694+0000 7fb1eaa00640 1 -- 172.21.15.113:0/1974904440 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb1e4019030 con 0x7fb1d8004810 2024-03-04T07:49:15.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 -- 172.21.15.113:0/1974904440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1f40b2180 con 0x7fb1d8004810 2024-03-04T07:49:15.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1eaa00640 1 -- 172.21.15.113:0/1974904440 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fb1e40568c0 con 0x7fb1d8004810 2024-03-04T07:49:15.904 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:15.904 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5940,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:13.835609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":41,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5417,"qdb_cluster":[5631,5529,5417]},"id":1}]} 2024-03-04T07:49:15.904 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 63 2024-03-04T07:49:15.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 -- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1d0066f60 msgr2=0x7fb1d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1d0066f60 0x7fb1d0069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40c85e0 tx=0x7fb1dc0011a0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 -- 172.21.15.113:0/1974904440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 msgr2=0x7fb1f4154a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1f4154a10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4011500 tx=0x7fb1e40119c0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 -- 172.21.15.113:0/1974904440 shutdown_connections 2024-03-04T07:49:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1f414dcd0 0x7fb1f414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb1f41e26e0 0x7fb1f4155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1d0066f60 0x7fb1d0069420 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 --2- 172.21.15.113:0/1974904440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1d8004810 0x7fb1f4154a10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 -- 172.21.15.113:0/1974904440 >> 172.21.15.113:0/1974904440 conn(0x7fb1f406e820 msgr2=0x7fb1f40c1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 -- 172.21.15.113:0/1974904440 shutdown_connections 2024-03-04T07:49:15.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:15.906+0000 7fb1f9800640 1 -- 172.21.15.113:0/1974904440 wait complete. 2024-03-04T07:49:15.934 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:49:17.935 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:18.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.073+0000 7fc1b2000640 1 Processor -- start 2024-03-04T07:49:18.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.073+0000 7fc1b2000640 1 -- start start 2024-03-04T07:49:18.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1b2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc1ac158010 0x7fc1ac06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:18.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1b2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1ac06ff20 0x7fc1ac15b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:18.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1b2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 0x7fc1ac0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:18.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1b2000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ac06f450 con 0x7fc1ac06ff20 2024-03-04T07:49:18.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1b2000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ac15ad80 con 0x7fc1ac158010 2024-03-04T07:49:18.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1b2000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac06e290 con 0x7fc1ac15b750 2024-03-04T07:49:18.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1a3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1ac06ff20 0x7fc1ac15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:18.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1a3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1ac06ff20 0x7fc1ac15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32778/0 (socket says 172.21.15.113:32778) 2024-03-04T07:49:18.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1a3e00640 1 -- 172.21.15.113:0/2030463022 learned_addr learned my addr 172.21.15.113:0/2030463022 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:18.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1ab400640 1 --2- 172.21.15.113:0/2030463022 >> v2:172.21.15.136:3300/0 conn(0x7fc1ac158010 0x7fc1ac06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:18.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1abe00640 1 --2- 172.21.15.113:0/2030463022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 0x7fc1ac0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:18.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1a3e00640 1 -- 172.21.15.113:0/2030463022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 msgr2=0x7fc1ac0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:18.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1a3e00640 1 --2- 172.21.15.113:0/2030463022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 0x7fc1ac0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1a3e00640 1 -- 172.21.15.113:0/2030463022 >> v2:172.21.15.136:3300/0 conn(0x7fc1ac158010 msgr2=0x7fc1ac06dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:18.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1a3e00640 1 --2- 172.21.15.113:0/2030463022 >> v2:172.21.15.136:3300/0 conn(0x7fc1ac158010 0x7fc1ac06dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1a3e00640 1 -- 172.21.15.113:0/2030463022 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac0ca0e0 con 0x7fc1ac06ff20 2024-03-04T07:49:18.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.077+0000 7fc1ab400640 1 --2- 172.21.15.113:0/2030463022 >> v2:172.21.15.136:3300/0 conn(0x7fc1ac158010 0x7fc1ac06dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:18.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1a3e00640 1 --2- 172.21.15.113:0/2030463022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac06ff20 0x7fc1ac15b210 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac06d140 tx=0x7fc19401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7821ef8a9764e101 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:18.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1aaa00640 1 -- 172.21.15.113:0/2030463022 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940b0070 con 0x7fc1ac06ff20 2024-03-04T07:49:18.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1aaa00640 1 -- 172.21.15.113:0/2030463022 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1940ab600 con 0x7fc1ac06ff20 2024-03-04T07:49:18.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1aaa00640 1 -- 172.21.15.113:0/2030463022 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940ab9d0 con 0x7fc1ac06ff20 2024-03-04T07:49:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 -- 172.21.15.113:0/2030463022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac06ff20 msgr2=0x7fc1ac15b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 --2- 172.21.15.113:0/2030463022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac06ff20 0x7fc1ac15b210 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac06d140 tx=0x7fc19401c520 comp rx=0 tx=0).stop 2024-03-04T07:49:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 -- 172.21.15.113:0/2030463022 shutdown_connections 2024-03-04T07:49:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 --2- 172.21.15.113:0/2030463022 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 0x7fc1ac0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 --2- 172.21.15.113:0/2030463022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac06ff20 0x7fc1ac15b210 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 --2- 172.21.15.113:0/2030463022 >> v2:172.21.15.136:3300/0 conn(0x7fc1ac158010 0x7fc1ac06dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 -- 172.21.15.113:0/2030463022 >> 172.21.15.113:0/2030463022 conn(0x7fc1ac0734d0 msgr2=0x7fc1ac066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 -- 172.21.15.113:0/2030463022 shutdown_connections 2024-03-04T07:49:18.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 -- 172.21.15.113:0/2030463022 wait complete. 2024-03-04T07:49:18.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 Processor -- start 2024-03-04T07:49:18.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.081+0000 7fc1b2000640 1 -- start start 2024-03-04T07:49:18.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1b2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1ac06ff20 0x7fc1ac1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1b2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac158010 0x7fc1ac149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1b2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 0x7fc1ac14a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1b2000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac1583f0 con 0x7fc1ac158010 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1b2000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac06f980 con 0x7fc1ac06ff20 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1b2000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac06c170 con 0x7fc1ac15b750 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac158010 0x7fc1ac149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac158010 0x7fc1ac149d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32792/0 (socket says 172.21.15.113:32792) 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3e00640 1 -- 172.21.15.113:0/1308528383 learned_addr learned my addr 172.21.15.113:0/1308528383 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1abe00640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 0x7fc1ac14a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1ab400640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1ac06ff20 0x7fc1ac1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3e00640 1 -- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 msgr2=0x7fc1ac14a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:18.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3e00640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 0x7fc1ac14a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3e00640 1 -- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1ac06ff20 msgr2=0x7fc1ac1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3e00640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1ac06ff20 0x7fc1ac1515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3e00640 1 -- 172.21.15.113:0/1308528383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac1f4000 con 0x7fc1ac158010 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1abe00640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 0x7fc1ac14a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1ab400640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1ac06ff20 0x7fc1ac1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3e00640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac158010 0x7fc1ac149d20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc1940ae040 tx=0x7fc1940192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3400640 1 -- 172.21.15.113:0/1308528383 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940b9070 con 0x7fc1ac158010 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3400640 1 -- 172.21.15.113:0/1308528383 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1940b4600 con 0x7fc1ac158010 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3400640 1 -- 172.21.15.113:0/1308528383 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940b0040 con 0x7fc1ac158010 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1b2000640 1 -- 172.21.15.113:0/1308528383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac1f42a0 con 0x7fc1ac158010 2024-03-04T07:49:18.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1b2000640 1 -- 172.21.15.113:0/1308528383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac150b10 con 0x7fc1ac158010 2024-03-04T07:49:18.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1b2000640 1 -- 172.21.15.113:0/1308528383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ac075c00 con 0x7fc1ac158010 2024-03-04T07:49:18.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.085+0000 7fc1a3400640 1 -- 172.21.15.113:0/1308528383 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1940bc070 con 0x7fc1ac158010 2024-03-04T07:49:18.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.089+0000 7fc1a3400640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc188066f60 0x7fc188069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:18.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.089+0000 7fc1a3400640 1 -- 172.21.15.113:0/1308528383 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fc19411e810 con 0x7fc1ac158010 2024-03-04T07:49:18.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.089+0000 7fc1ab400640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc188066f60 0x7fc188069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:18.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.089+0000 7fc1ab400640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc188066f60 0x7fc188069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009970 tx=0x7fc19c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:18.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.093+0000 7fc1a3400640 1 -- 172.21.15.113:0/1308528383 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc194016110 con 0x7fc1ac158010 2024-03-04T07:49:18.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.281+0000 7fc1b2000640 1 -- 172.21.15.113:0/1308528383 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1ac06fd00 con 0x7fc1ac158010 2024-03-04T07:49:18.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.281+0000 7fc1a3400640 1 -- 172.21.15.113:0/1308528383 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fc1940ef450 con 0x7fc1ac158010 2024-03-04T07:49:18.280 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:18.280 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5940,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:13.835609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":41,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5417,"qdb_cluster":[5631,5529,5417]},"id":1}]} 2024-03-04T07:49:18.281 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 63 2024-03-04T07:49:18.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 -- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc188066f60 msgr2=0x7fc188069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:18.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc188066f60 0x7fc188069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009970 tx=0x7fc19c008040 comp rx=0 tx=0).stop 2024-03-04T07:49:18.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 -- 172.21.15.113:0/1308528383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac158010 msgr2=0x7fc1ac149d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:18.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac158010 0x7fc1ac149d20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc1940ae040 tx=0x7fc1940192c0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 -- 172.21.15.113:0/1308528383 shutdown_connections 2024-03-04T07:49:18.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1ac15b750 0x7fc1ac14a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc188066f60 0x7fc188069420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1ac158010 0x7fc1ac149d20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 --2- 172.21.15.113:0/1308528383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1ac06ff20 0x7fc1ac1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:18.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 -- 172.21.15.113:0/1308528383 >> 172.21.15.113:0/1308528383 conn(0x7fc1ac0734d0 msgr2=0x7fc1ac0c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:18.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 -- 172.21.15.113:0/1308528383 shutdown_connections 2024-03-04T07:49:18.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:18.285+0000 7fc1b2000640 1 -- 172.21.15.113:0/1308528383 wait complete. 2024-03-04T07:49:18.301 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:49:20.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:20.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d57200640 1 Processor -- start 2024-03-04T07:49:20.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d57200640 1 -- start start 2024-03-04T07:49:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d57200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3d58152ac0 0x7f3d5806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d57200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d580c8fe0 0x7f3d580c9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d57200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d580c5630 con 0x7f3d580c9ab0 2024-03-04T07:49:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d57200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d580c5bb0 con 0x7f3d580c8fe0 2024-03-04T07:49:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d57200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5806fc70 con 0x7f3d58152ac0 2024-03-04T07:49:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d56800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d580c8fe0 0x7f3d580c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d4d400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3d580c9ab0 0x7f3d5815c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36452/0 (socket says 172.21.15.113:36452) 2024-03-04T07:49:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d4d400640 1 -- 172.21.15.113:0/2945830530 learned_addr learned my addr 172.21.15.113:0/2945830530 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:49:20.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d55e00640 1 --2- 172.21.15.113:0/2945830530 >> v2:172.21.15.136:3300/0 conn(0x7f3d58152ac0 0x7f3d5806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:20.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d56800640 1 -- 172.21.15.113:0/2945830530 >> v2:172.21.15.136:3300/0 conn(0x7f3d58152ac0 msgr2=0x7f3d5806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:20.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d56800640 1 --2- 172.21.15.113:0/2945830530 >> v2:172.21.15.136:3300/0 conn(0x7f3d58152ac0 0x7f3d5806ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d56800640 1 -- 172.21.15.113:0/2945830530 >> v1:172.21.15.113:6789/0 conn(0x7f3d580c9ab0 legacy=0x7f3d5815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:20.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.445+0000 7f3d56800640 1 -- 172.21.15.113:0/2945830530 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5806f490 con 0x7f3d580c8fe0 2024-03-04T07:49:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d56800640 1 --2- 172.21.15.113:0/2945830530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 0x7f3d580c9410 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3d58067620 tx=0x7f3d5001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803a62c7f16ef33d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d55400640 1 -- 172.21.15.113:0/2945830530 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500b2070 con 0x7f3d580c8fe0 2024-03-04T07:49:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d55400640 1 -- 172.21.15.113:0/2945830530 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d500a5870 con 0x7f3d580c8fe0 2024-03-04T07:49:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d55400640 1 -- 172.21.15.113:0/2945830530 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500a5c40 con 0x7f3d580c8fe0 2024-03-04T07:49:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d57200640 1 -- 172.21.15.113:0/2945830530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 msgr2=0x7f3d580c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d57200640 1 --2- 172.21.15.113:0/2945830530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 0x7f3d580c9410 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f3d58067620 tx=0x7f3d5001c8a0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d57200640 1 -- 172.21.15.113:0/2945830530 shutdown_connections 2024-03-04T07:49:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d57200640 1 --2- 172.21.15.113:0/2945830530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 0x7f3d580c9410 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d57200640 1 --2- 172.21.15.113:0/2945830530 >> v2:172.21.15.136:3300/0 conn(0x7f3d58152ac0 0x7f3d5806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d57200640 1 -- 172.21.15.113:0/2945830530 >> 172.21.15.113:0/2945830530 conn(0x7f3d58072e30 msgr2=0x7f3d58066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:20.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d57200640 1 -- 172.21.15.113:0/2945830530 shutdown_connections 2024-03-04T07:49:20.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.449+0000 7f3d57200640 1 -- 172.21.15.113:0/2945830530 wait complete. 2024-03-04T07:49:20.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d57200640 1 Processor -- start 2024-03-04T07:49:20.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d57200640 1 -- start start 2024-03-04T07:49:20.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d57200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 0x7f3d58151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d57200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d58152ac0 0x7f3d58153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d57200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d5814cc60 0x7f3d581538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d57200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d580c8a00 con 0x7f3d580c8fe0 2024-03-04T07:49:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d57200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5806ff30 con 0x7f3d58152ac0 2024-03-04T07:49:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d57200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58152ea0 con 0x7f3d5814cc60 2024-03-04T07:49:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d55e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 0x7f3d58151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d55e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 0x7f3d58151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32802/0 (socket says 172.21.15.113:32802) 2024-03-04T07:49:20.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d55e00640 1 -- 172.21.15.113:0/1056748090 learned_addr learned my addr 172.21.15.113:0/1056748090 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:20.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d56800640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d5814cc60 0x7f3d581538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:20.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d4d400640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d58152ac0 0x7f3d58153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:20.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d55e00640 1 -- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d5814cc60 msgr2=0x7f3d581538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:20.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d55e00640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d5814cc60 0x7f3d581538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d55e00640 1 -- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d58152ac0 msgr2=0x7f3d58153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:20.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d55e00640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d58152ac0 0x7f3d58153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d55e00640 1 -- 172.21.15.113:0/1056748090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5815b020 con 0x7f3d580c8fe0 2024-03-04T07:49:20.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d4d400640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d58152ac0 0x7f3d58153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:49:20.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d55e00640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 0x7f3d58151910 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800d830 tx=0x7f3d4800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:20.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d4ea00640 1 -- 172.21.15.113:0/1056748090 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800b6c0 con 0x7f3d580c8fe0 2024-03-04T07:49:20.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d4ea00640 1 -- 172.21.15.113:0/1056748090 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d48019070 con 0x7f3d580c8fe0 2024-03-04T07:49:20.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d4ea00640 1 -- 172.21.15.113:0/1056748090 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800ba60 con 0x7f3d580c8fe0 2024-03-04T07:49:20.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d57200640 1 -- 172.21.15.113:0/1056748090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5815a440 con 0x7f3d580c8fe0 2024-03-04T07:49:20.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.453+0000 7f3d57200640 1 -- 172.21.15.113:0/1056748090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5815b600 con 0x7f3d580c8fe0 2024-03-04T07:49:20.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.457+0000 7f3d57200640 1 -- 172.21.15.113:0/1056748090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d58075a80 con 0x7f3d580c8fe0 2024-03-04T07:49:20.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.457+0000 7f3d4ea00640 1 -- 172.21.15.113:0/1056748090 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d4801e460 con 0x7f3d580c8fe0 2024-03-04T07:49:20.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.457+0000 7f3d4ea00640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d34066f60 0x7f3d34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:20.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.457+0000 7f3d4ea00640 1 -- 172.21.15.113:0/1056748090 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f3d48085a80 con 0x7f3d580c8fe0 2024-03-04T07:49:20.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.457+0000 7f3d4d400640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d34066f60 0x7f3d34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:20.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.457+0000 7f3d4d400640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d34066f60 0x7f3d34069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3d40007fd0 tx=0x7f3d40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:20.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.461+0000 7f3d4ea00640 1 -- 172.21.15.113:0/1056748090 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3d48087e20 con 0x7f3d580c8fe0 2024-03-04T07:49:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.657+0000 7f3d57200640 1 -- 172.21.15.113:0/1056748090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d58068670 con 0x7f3d580c8fe0 2024-03-04T07:49:20.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.657+0000 7f3d4ea00640 1 -- 172.21.15.113:0/1056748090 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f3d480566c0 con 0x7f3d580c8fe0 2024-03-04T07:49:20.656 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:20.656 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5940,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:13.835609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":41,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5417,"qdb_cluster":[5631,5529,5417]},"id":1}]} 2024-03-04T07:49:20.657 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 63 2024-03-04T07:49:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.657+0000 7f3d57200640 1 -- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d34066f60 msgr2=0x7f3d34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.657+0000 7f3d57200640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d34066f60 0x7f3d34069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3d40007fd0 tx=0x7f3d40009040 comp rx=0 tx=0).stop 2024-03-04T07:49:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.657+0000 7f3d57200640 1 -- 172.21.15.113:0/1056748090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 msgr2=0x7f3d58151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.657+0000 7f3d57200640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 0x7f3d58151910 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800d830 tx=0x7f3d4800dcf0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.661+0000 7f3d57200640 1 -- 172.21.15.113:0/1056748090 shutdown_connections 2024-03-04T07:49:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.661+0000 7f3d57200640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d5814cc60 0x7f3d581538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.661+0000 7f3d57200640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d58152ac0 0x7f3d58153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.661+0000 7f3d57200640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d34066f60 0x7f3d34069420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.661+0000 7f3d57200640 1 --2- 172.21.15.113:0/1056748090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d580c8fe0 0x7f3d58151910 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:20.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.661+0000 7f3d57200640 1 -- 172.21.15.113:0/1056748090 >> 172.21.15.113:0/1056748090 conn(0x7f3d58072e30 msgr2=0x7f3d58066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:20.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.661+0000 7f3d57200640 1 -- 172.21.15.113:0/1056748090 shutdown_connections 2024-03-04T07:49:20.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:20.661+0000 7f3d57200640 1 -- 172.21.15.113:0/1056748090 wait complete. 2024-03-04T07:49:20.677 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:49:22.678 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:22.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d33c00640 1 Processor -- start 2024-03-04T07:49:22.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d33c00640 1 -- start start 2024-03-04T07:49:22.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d33c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2d2c152bd0 0x7f2d2c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:22.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d33c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c1591f0 con 0x7f2d2c0c8200 2024-03-04T07:49:22.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d33c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c1594d0 con 0x7f2d2c1527f0 2024-03-04T07:49:22.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d33c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c071df0 con 0x7f2d2c152bd0 2024-03-04T07:49:22.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d23e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2d2c0c8200 0x7f2d2c1587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36464/0 (socket says 172.21.15.113:36464) 2024-03-04T07:49:22.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d23e00640 1 -- 172.21.15.113:0/3658063094 learned_addr learned my addr 172.21.15.113:0/3658063094 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:49:22.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d31c00640 1 --2- 172.21.15.113:0/3658063094 >> v2:172.21.15.136:3300/0 conn(0x7f2d2c152bd0 0x7f2d2c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:22.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d2be00640 1 -- 172.21.15.113:0/3658063094 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944023207 0 0) 0x7f2d2c1591f0 con 0x7f2d2c0c8200 2024-03-04T07:49:22.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d2be00640 1 -- 172.21.15.113:0/3658063094 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d10003530 con 0x7f2d2c0c8200 2024-03-04T07:49:22.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d31c00640 1 -- 172.21.15.113:0/3658063094 >> v1:172.21.15.136:6789/0 conn(0x7f2d2c1527f0 legacy=0x7f2d2c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:22.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d31c00640 1 -- 172.21.15.113:0/3658063094 >> v1:172.21.15.113:6789/0 conn(0x7f2d2c0c8200 legacy=0x7f2d2c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:22.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d2be00640 1 -- 172.21.15.113:0/3658063094 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3498745909 0 0) 0x7f2d10003530 con 0x7f2d2c0c8200 2024-03-04T07:49:22.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.821+0000 7f2d31c00640 1 -- 172.21.15.113:0/3658063094 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c0717a0 con 0x7f2d2c152bd0 2024-03-04T07:49:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d31c00640 1 --2- 172.21.15.113:0/3658063094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2c152bd0 0x7f2d2c070d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009d90 tx=0x7f2d2401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6059ee53def549b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:22.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d2be00640 1 -- 172.21.15.113:0/3658063094 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14001680 con 0x7f2d2c152bd0 2024-03-04T07:49:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d2be00640 1 -- 172.21.15.113:0/3658063094 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d240a64e0 con 0x7f2d2c152bd0 2024-03-04T07:49:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d2be00640 1 -- 172.21.15.113:0/3658063094 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d240a6890 con 0x7f2d2c152bd0 2024-03-04T07:49:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d33c00640 1 -- 172.21.15.113:0/3658063094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2c152bd0 msgr2=0x7f2d2c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:22.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d33c00640 1 --2- 172.21.15.113:0/3658063094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2c152bd0 0x7f2d2c070d40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009d90 tx=0x7f2d2401c890 comp rx=0 tx=0).stop 2024-03-04T07:49:22.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d33c00640 1 -- 172.21.15.113:0/3658063094 shutdown_connections 2024-03-04T07:49:22.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d33c00640 1 --2- 172.21.15.113:0/3658063094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2c152bd0 0x7f2d2c070d40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:22.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d33c00640 1 -- 172.21.15.113:0/3658063094 >> 172.21.15.113:0/3658063094 conn(0x7f2d2c06e820 msgr2=0x7f2d2c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:22.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d33c00640 1 -- 172.21.15.113:0/3658063094 shutdown_connections 2024-03-04T07:49:22.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.825+0000 7f2d33c00640 1 -- 172.21.15.113:0/3658063094 wait complete. 2024-03-04T07:49:22.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d33c00640 1 Processor -- start 2024-03-04T07:49:22.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d33c00640 1 -- start start 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d33c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c0c8200 0x7f2d2c0c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d33c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d2c1527f0 0x7f2d2c155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d33c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2c152bd0 0x7f2d2c1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d33c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c166500 con 0x7f2d2c0c8200 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d33c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c166380 con 0x7f2d2c152bd0 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d33c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c166680 con 0x7f2d2c1527f0 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d31200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c0c8200 0x7f2d2c0c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d31200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c0c8200 0x7f2d2c0c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32806/0 (socket says 172.21.15.113:32806) 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d31200640 1 -- 172.21.15.113:0/1565964043 learned_addr learned my addr 172.21.15.113:0/1565964043 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d23e00640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d2c1527f0 0x7f2d2c155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d31200640 1 -- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d2c1527f0 msgr2=0x7f2d2c155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:22.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d31200640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d2c1527f0 0x7f2d2c155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d31200640 1 -- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2c152bd0 msgr2=0x7f2d2c1556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d31200640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2c152bd0 0x7f2d2c1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d31200640 1 -- 172.21.15.113:0/1565964043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c153190 con 0x7f2d2c0c8200 2024-03-04T07:49:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d31200640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c0c8200 0x7f2d2c0c4060 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00c530 tx=0x7f2d1c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:22.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d29600640 1 -- 172.21.15.113:0/1565964043 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c016020 con 0x7f2d2c0c8200 2024-03-04T07:49:22.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d29600640 1 -- 172.21.15.113:0/1565964043 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1c005220 con 0x7f2d2c0c8200 2024-03-04T07:49:22.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d33c00640 1 -- 172.21.15.113:0/1565964043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c1522c0 con 0x7f2d2c0c8200 2024-03-04T07:49:22.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d29600640 1 -- 172.21.15.113:0/1565964043 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c007030 con 0x7f2d2c0c8200 2024-03-04T07:49:22.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.829+0000 7f2d33c00640 1 -- 172.21.15.113:0/1565964043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c153710 con 0x7f2d2c0c8200 2024-03-04T07:49:22.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.833+0000 7f2d33c00640 1 -- 172.21.15.113:0/1565964043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c163f40 con 0x7f2d2c0c8200 2024-03-04T07:49:22.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.833+0000 7f2d29600640 1 -- 172.21.15.113:0/1565964043 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d1c0071d0 con 0x7f2d2c0c8200 2024-03-04T07:49:22.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.833+0000 7f2d29600640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d08066f60 0x7f2d08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:22.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.833+0000 7f2d29600640 1 -- 172.21.15.113:0/1565964043 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f2d1c084cf0 con 0x7f2d2c0c8200 2024-03-04T07:49:22.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.837+0000 7f2d23e00640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d08066f60 0x7f2d08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:22.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.837+0000 7f2d23e00640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d08066f60 0x7f2d08069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c070f10 tx=0x7f2d14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:22.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:22.845+0000 7f2d29600640 1 -- 172.21.15.113:0/1565964043 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2d1c089070 con 0x7f2d2c0c8200 2024-03-04T07:49:23.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.041+0000 7f2d33c00640 1 -- 172.21.15.113:0/1565964043 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d2c075be0 con 0x7f2d2c0c8200 2024-03-04T07:49:23.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.041+0000 7f2d29600640 1 -- 172.21.15.113:0/1565964043 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f2d1c020020 con 0x7f2d2c0c8200 2024-03-04T07:49:23.042 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:23.042 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5940,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:13.835609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":41,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5417,"qdb_cluster":[5631,5529,5417]},"id":1}]} 2024-03-04T07:49:23.042 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 63 2024-03-04T07:49:23.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 -- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d08066f60 msgr2=0x7f2d08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d08066f60 0x7f2d08069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c070f10 tx=0x7f2d14009040 comp rx=0 tx=0).stop 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 -- 172.21.15.113:0/1565964043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c0c8200 msgr2=0x7f2d2c0c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c0c8200 0x7f2d2c0c4060 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00c530 tx=0x7f2d1c00c9f0 comp rx=0 tx=0).stop 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 -- 172.21.15.113:0/1565964043 shutdown_connections 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d2c152bd0 0x7f2d2c1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d2c1527f0 0x7f2d2c155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d08066f60 0x7f2d08069420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 --2- 172.21.15.113:0/1565964043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d2c0c8200 0x7f2d2c0c4060 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 -- 172.21.15.113:0/1565964043 >> 172.21.15.113:0/1565964043 conn(0x7f2d2c06e820 msgr2=0x7f2d2c0c2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 -- 172.21.15.113:0/1565964043 shutdown_connections 2024-03-04T07:49:23.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:23.045+0000 7f2d33c00640 1 -- 172.21.15.113:0/1565964043 wait complete. 2024-03-04T07:49:23.063 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:49:25.064 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e7200640 1 Processor -- start 2024-03-04T07:49:25.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e7200640 1 -- start start 2024-03-04T07:49:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e7200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff5e8152a90 0x7ff5e80730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e7200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5e815bce0 0x7ff5e815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e7200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e806f1b0 con 0x7ff5e806cd10 2024-03-04T07:49:25.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e7200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e8152e70 con 0x7ff5e815bce0 2024-03-04T07:49:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e7200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e806c320 con 0x7ff5e8152a90 2024-03-04T07:49:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e6800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5e815bce0 0x7ff5e815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e6800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff5e815bce0 0x7ff5e815c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32820/0 (socket says 172.21.15.113:32820) 2024-03-04T07:49:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e6800640 1 -- 172.21.15.113:0/2578060326 learned_addr learned my addr 172.21.15.113:0/2578060326 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e5e00640 1 --2- 172.21.15.113:0/2578060326 >> v2:172.21.15.136:3300/0 conn(0x7ff5e8152a90 0x7ff5e80730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e6800640 1 -- 172.21.15.113:0/2578060326 >> v2:172.21.15.136:3300/0 conn(0x7ff5e8152a90 msgr2=0x7ff5e80730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:25.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e6800640 1 --2- 172.21.15.113:0/2578060326 >> v2:172.21.15.136:3300/0 conn(0x7ff5e8152a90 0x7ff5e80730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:25.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e6800640 1 -- 172.21.15.113:0/2578060326 >> v1:172.21.15.136:6789/0 conn(0x7ff5e806cd10 legacy=0x7ff5e806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:25.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.209+0000 7ff5e6800640 1 -- 172.21.15.113:0/2578060326 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8071580 con 0x7ff5e815bce0 2024-03-04T07:49:25.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.213+0000 7ff5e6800640 1 --2- 172.21.15.113:0/2578060326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 0x7ff5e815c0f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff5e806df40 tx=0x7ff5e001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b62efa6ab67c1b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:25.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.213+0000 7ff5e5400640 1 -- 172.21.15.113:0/2578060326 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00a56c0 con 0x7ff5e815bce0 2024-03-04T07:49:25.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.213+0000 7ff5e5400640 1 -- 172.21.15.113:0/2578060326 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e00a5860 con 0x7ff5e815bce0 2024-03-04T07:49:25.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.213+0000 7ff5e5400640 1 -- 172.21.15.113:0/2578060326 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00a5c30 con 0x7ff5e815bce0 2024-03-04T07:49:25.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.217+0000 7ff5e7200640 1 -- 172.21.15.113:0/2578060326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 msgr2=0x7ff5e815c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:25.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.217+0000 7ff5e7200640 1 --2- 172.21.15.113:0/2578060326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 0x7ff5e815c0f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff5e806df40 tx=0x7ff5e001c9f0 comp rx=0 tx=0).stop 2024-03-04T07:49:25.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.217+0000 7ff5e7200640 1 -- 172.21.15.113:0/2578060326 shutdown_connections 2024-03-04T07:49:25.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.217+0000 7ff5e7200640 1 --2- 172.21.15.113:0/2578060326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 0x7ff5e815c0f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:25.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.217+0000 7ff5e7200640 1 --2- 172.21.15.113:0/2578060326 >> v2:172.21.15.136:3300/0 conn(0x7ff5e8152a90 0x7ff5e80730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:25.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.217+0000 7ff5e7200640 1 -- 172.21.15.113:0/2578060326 >> 172.21.15.113:0/2578060326 conn(0x7ff5e806e2a0 msgr2=0x7ff5e806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:25.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.217+0000 7ff5e7200640 1 -- 172.21.15.113:0/2578060326 shutdown_connections 2024-03-04T07:49:25.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.217+0000 7ff5e7200640 1 -- 172.21.15.113:0/2578060326 wait complete. 2024-03-04T07:49:25.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.217+0000 7ff5e7200640 1 Processor -- start 2024-03-04T07:49:25.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.217+0000 7ff5e7200640 1 -- start start 2024-03-04T07:49:25.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e7200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5e806cd10 0x7ff5e8144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:25.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e7200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5e8152a90 0x7ff5e815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:25.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 0x7ff5e80c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:25.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8165890 con 0x7ff5e815bce0 2024-03-04T07:49:25.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e7200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8071330 con 0x7ff5e806cd10 2024-03-04T07:49:25.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e7200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8165a10 con 0x7ff5e8152a90 2024-03-04T07:49:25.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 0x7ff5e80c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:25.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 0x7ff5e80c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32832/0 (socket says 172.21.15.113:32832) 2024-03-04T07:49:25.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e6800640 1 -- 172.21.15.113:0/4243911178 learned_addr learned my addr 172.21.15.113:0/4243911178 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:25.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5dd400640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5e8152a90 0x7ff5e815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:25.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e5e00640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5e806cd10 0x7ff5e8144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:25.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e6800640 1 -- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5e8152a90 msgr2=0x7ff5e815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:25.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e6800640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5e8152a90 0x7ff5e815b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:25.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e6800640 1 -- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5e806cd10 msgr2=0x7ff5e8144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:25.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e6800640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5e806cd10 0x7ff5e8144b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:25.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e6800640 1 -- 172.21.15.113:0/4243911178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e81f7c60 con 0x7ff5e815bce0 2024-03-04T07:49:25.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e6800640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 0x7ff5e80c9cb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00af040 tx=0x7ff5e0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:25.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5dea00640 1 -- 172.21.15.113:0/4243911178 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00bc020 con 0x7ff5e815bce0 2024-03-04T07:49:25.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5dea00640 1 -- 172.21.15.113:0/4243911178 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e00b86b0 con 0x7ff5e815bce0 2024-03-04T07:49:25.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5dea00640 1 -- 172.21.15.113:0/4243911178 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0009510 con 0x7ff5e815bce0 2024-03-04T07:49:25.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e7200640 1 -- 172.21.15.113:0/4243911178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e81402e0 con 0x7ff5e815bce0 2024-03-04T07:49:25.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e7200640 1 -- 172.21.15.113:0/4243911178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e80cbb90 con 0x7ff5e815bce0 2024-03-04T07:49:25.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.221+0000 7ff5e7200640 1 -- 172.21.15.113:0/4243911178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e8140a80 con 0x7ff5e815bce0 2024-03-04T07:49:25.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.225+0000 7ff5dea00640 1 -- 172.21.15.113:0/4243911178 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5e00bf020 con 0x7ff5e815bce0 2024-03-04T07:49:25.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.225+0000 7ff5dea00640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5c4066f60 0x7ff5c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:25.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.225+0000 7ff5dea00640 1 -- 172.21.15.113:0/4243911178 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7ff5e011fce0 con 0x7ff5e815bce0 2024-03-04T07:49:25.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.225+0000 7ff5e5e00640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5c4066f60 0x7ff5c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:25.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.225+0000 7ff5e5e00640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5c4066f60 0x7ff5c4069420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009ac0 tx=0x7ff5d8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:25.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.233+0000 7ff5dea00640 1 -- 172.21.15.113:0/4243911178 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff5e8140a80 con 0x7ff5e815bce0 2024-03-04T07:49:25.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 -- 172.21.15.113:0/4243911178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5e80685f0 con 0x7ff5e815bce0 2024-03-04T07:49:25.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5dea00640 1 -- 172.21.15.113:0/4243911178 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6612 (secure 0 0 0) 0x7ff5e00c6070 con 0x7ff5e815bce0 2024-03-04T07:49:25.428 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:25.428 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5940,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:13.835609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":41,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5417,"qdb_cluster":[5631,5529,5417]},"id":1}]} 2024-03-04T07:49:25.428 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 63 2024-03-04T07:49:25.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 -- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5c4066f60 msgr2=0x7ff5c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:25.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5c4066f60 0x7ff5c4069420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009ac0 tx=0x7ff5d8009190 comp rx=0 tx=0).stop 2024-03-04T07:49:25.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 -- 172.21.15.113:0/4243911178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 msgr2=0x7ff5e80c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:25.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 0x7ff5e80c9cb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00af040 tx=0x7ff5e0019790 comp rx=0 tx=0).stop 2024-03-04T07:49:25.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 -- 172.21.15.113:0/4243911178 shutdown_connections 2024-03-04T07:49:25.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5c4066f60 0x7ff5c4069420 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:25.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5e815bce0 0x7ff5e80c9cb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:25.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff5e8152a90 0x7ff5e815b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:25.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 --2- 172.21.15.113:0/4243911178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5e806cd10 0x7ff5e8144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:25.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.429+0000 7ff5e7200640 1 -- 172.21.15.113:0/4243911178 >> 172.21.15.113:0/4243911178 conn(0x7ff5e806e2a0 msgr2=0x7ff5e806d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:25.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.433+0000 7ff5e7200640 1 -- 172.21.15.113:0/4243911178 shutdown_connections 2024-03-04T07:49:25.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:25.433+0000 7ff5e7200640 1 -- 172.21.15.113:0/4243911178 wait complete. 2024-03-04T07:49:25.449 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:49:27.450 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:27.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.589+0000 7fe986800640 1 Processor -- start 2024-03-04T07:49:27.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.589+0000 7fe986800640 1 -- start start 2024-03-04T07:49:27.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.589+0000 7fe986800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe98006de80 0x7fe98015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:27.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.589+0000 7fe986800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98006c210 con 0x7fe9800ce2d0 2024-03-04T07:49:27.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe986800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98006c4f0 con 0x7fe9800ce6b0 2024-03-04T07:49:27.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe986800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98015bbf0 con 0x7fe98006de80 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97f400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe9800ce2d0 0x7fe98006f010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57110/0 (socket says 172.21.15.113:57110) 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97f400640 1 -- 172.21.15.113:0/962066901 learned_addr learned my addr 172.21.15.113:0/962066901 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97fe00640 1 --2- 172.21.15.113:0/962066901 >> v2:172.21.15.136:3300/0 conn(0x7fe98006de80 0x7fe98015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259816223 0 0) 0x7fe98006c210 con 0x7fe9800ce2d0 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe964003530 con 0x7fe9800ce2d0 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1976887807 0 0) 0x7fe964003530 con 0x7fe9800ce2d0 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 >> v2:172.21.15.136:3300/0 conn(0x7fe98006de80 msgr2=0x7fe98015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 --2- 172.21.15.113:0/962066901 >> v2:172.21.15.136:3300/0 conn(0x7fe98006de80 0x7fe98015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 >> v1:172.21.15.136:6789/0 conn(0x7fe9800ce6b0 legacy=0x7fe980154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98006e530 con 0x7fe9800ce2d0 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fe970001680 con 0x7fe9800ce2d0 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 >> v1:172.21.15.113:6789/0 conn(0x7fe9800ce2d0 legacy=0x7fe98006f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:27.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9640039c0 0x7fe964003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe964004810 0x7fe964004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9640051d0 0x7fe96400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98015bbf0 con 0x7fe9640051d0 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96400c080 con 0x7fe9640039c0 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe96400c280 con 0x7fe964004810 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97fe00640 1 --2- 172.21.15.113:0/962066901 >> v2:172.21.15.136:3300/0 conn(0x7fe98006de80 0x7fe98015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97f400640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9640039c0 0x7fe964003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97fe00640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9640051d0 0x7fe96400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97ea00640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe964004810 0x7fe964004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97fe00640 1 -- 172.21.15.113:0/962066901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe964004810 msgr2=0x7fe964004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:27.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97fe00640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe964004810 0x7fe964004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:27.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97fe00640 1 -- 172.21.15.113:0/962066901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9640039c0 msgr2=0x7fe964003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:27.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97fe00640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9640039c0 0x7fe964003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:27.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.593+0000 7fe97fe00640 1 -- 172.21.15.113:0/962066901 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe964003530 con 0x7fe9640051d0 2024-03-04T07:49:27.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe97fe00640 1 -- 172.21.15.113:0/962066901 reap_dead start 2024-03-04T07:49:27.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe97fe00640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9640051d0 0x7fe96400b9d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe97800e350 tx=0x7fe97801ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2798f22cb650d61 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:27.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978009dc0 con 0x7fe9640051d0 2024-03-04T07:49:27.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9780039a0 con 0x7fe9640051d0 2024-03-04T07:49:27.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe97e000640 1 -- 172.21.15.113:0/962066901 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978003d90 con 0x7fe9640051d0 2024-03-04T07:49:27.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe986800640 1 -- 172.21.15.113:0/962066901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9640051d0 msgr2=0x7fe96400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:27.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe986800640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9640051d0 0x7fe96400b9d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe97800e350 tx=0x7fe97801ce60 comp rx=0 tx=0).stop 2024-03-04T07:49:27.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe986800640 1 -- 172.21.15.113:0/962066901 shutdown_connections 2024-03-04T07:49:27.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe986800640 1 --2- 172.21.15.113:0/962066901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9640051d0 0x7fe96400b9d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:27.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe986800640 1 -- 172.21.15.113:0/962066901 >> 172.21.15.113:0/962066901 conn(0x7fe9800709a0 msgr2=0x7fe980070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:27.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe986800640 1 -- 172.21.15.113:0/962066901 shutdown_connections 2024-03-04T07:49:27.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.597+0000 7fe986800640 1 -- 172.21.15.113:0/962066901 wait complete. 2024-03-04T07:49:27.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.601+0000 7fe986800640 1 Processor -- start 2024-03-04T07:49:27.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.601+0000 7fe986800640 1 -- start start 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe986800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9640051d0 0x7fe9801e7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe986800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9801f3f50 0x7fe9800c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe986800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9800c6fe0 0x7fe9801498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe986800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98006c7d0 con 0x7fe9801f3f50 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe986800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98006eb80 con 0x7fe9800c6fe0 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe986800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe980070010 con 0x7fe9640051d0 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9801f3f50 0x7fe9800c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9801f3f50 0x7fe9800c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33994/0 (socket says 172.21.15.113:33994) 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97ea00640 1 -- 172.21.15.113:0/889154614 learned_addr learned my addr 172.21.15.113:0/889154614 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97f400640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9640051d0 0x7fe9801e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:27.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97fe00640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9800c6fe0 0x7fe9801498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:27.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97ea00640 1 -- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9640051d0 msgr2=0x7fe9801e7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:27.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97ea00640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9640051d0 0x7fe9801e7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:27.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97ea00640 1 -- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9800c6fe0 msgr2=0x7fe9801498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:27.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97ea00640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9800c6fe0 0x7fe9801498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:27.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97ea00640 1 -- 172.21.15.113:0/889154614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9800c65f0 con 0x7fe9801f3f50 2024-03-04T07:49:27.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe97ea00640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9801f3f50 0x7fe9800c6aa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00f620 tx=0x7fe96c00fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:27.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe977400640 1 -- 172.21.15.113:0/889154614 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c0052e0 con 0x7fe9801f3f50 2024-03-04T07:49:27.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe977400640 1 -- 172.21.15.113:0/889154614 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c018070 con 0x7fe9801f3f50 2024-03-04T07:49:27.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe977400640 1 -- 172.21.15.113:0/889154614 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c003820 con 0x7fe9801f3f50 2024-03-04T07:49:27.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe986800640 1 -- 172.21.15.113:0/889154614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9800c7460 con 0x7fe9801f3f50 2024-03-04T07:49:27.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe986800640 1 -- 172.21.15.113:0/889154614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9801f0ff0 con 0x7fe9801f3f50 2024-03-04T07:49:27.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe986800640 1 -- 172.21.15.113:0/889154614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98006cb40 con 0x7fe9801f3f50 2024-03-04T07:49:27.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.605+0000 7fe977400640 1 -- 172.21.15.113:0/889154614 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe96c0039c0 con 0x7fe9801f3f50 2024-03-04T07:49:27.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.609+0000 7fe977400640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe960066f60 0x7fe960069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:27.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.609+0000 7fe977400640 1 -- 172.21.15.113:0/889154614 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7fe96c084f00 con 0x7fe9801f3f50 2024-03-04T07:49:27.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.609+0000 7fe97f400640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe960066f60 0x7fe960069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:27.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.609+0000 7fe97f400640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe960066f60 0x7fe960069420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe970000c10 tx=0x7fe970007900 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:27.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.613+0000 7fe977400640 1 -- 172.21.15.113:0/889154614 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe9800c7460 con 0x7fe9801f3f50 2024-03-04T07:49:27.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.897+0000 7fe986800640 1 -- 172.21.15.113:0/889154614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9800715e0 con 0x7fe9801f3f50 2024-03-04T07:49:27.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.897+0000 7fe977400640 1 -- 172.21.15.113:0/889154614 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fe96c055be0 con 0x7fe9801f3f50 2024-03-04T07:49:27.898 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:27.898 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5940,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:13.835609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":41,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5417,"qdb_cluster":[5631,5529,5417]},"id":1}]} 2024-03-04T07:49:27.898 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 63 2024-03-04T07:49:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 -- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe960066f60 msgr2=0x7fe960069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe960066f60 0x7fe960069420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe970000c10 tx=0x7fe970007900 comp rx=0 tx=0).stop 2024-03-04T07:49:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 -- 172.21.15.113:0/889154614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9801f3f50 msgr2=0x7fe9800c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9801f3f50 0x7fe9800c6aa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00f620 tx=0x7fe96c00fa20 comp rx=0 tx=0).stop 2024-03-04T07:49:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 -- 172.21.15.113:0/889154614 shutdown_connections 2024-03-04T07:49:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9800c6fe0 0x7fe9801498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe960066f60 0x7fe960069420 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9801f3f50 0x7fe9800c6aa0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:27.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 --2- 172.21.15.113:0/889154614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9640051d0 0x7fe9801e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:27.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 -- 172.21.15.113:0/889154614 >> 172.21.15.113:0/889154614 conn(0x7fe9800709a0 msgr2=0x7fe9800c4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:27.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 -- 172.21.15.113:0/889154614 shutdown_connections 2024-03-04T07:49:27.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:27.901+0000 7fe986800640 1 -- 172.21.15.113:0/889154614 wait complete. 2024-03-04T07:49:27.919 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:49:28.889 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:49:28.892 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:49:29.920 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:30.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.061+0000 7f3cbc800640 1 Processor -- start 2024-03-04T07:49:30.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.061+0000 7f3cbc800640 1 -- start start 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cbc800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb414e4c0 0x7f3cb4072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cbc800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3cb40c22e0 0x7f3cb40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cbc800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3cb40c2d80 0x7f3cb415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cbc800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb415ad60 con 0x7f3cb40c2d80 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cbc800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb40c27f0 con 0x7f3cb40c22e0 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cbc800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40c2a90 con 0x7f3cb414e4c0 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cba800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3cb40c2d80 0x7f3cb415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cba800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3cb40c2d80 0x7f3cb415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34000/0 (socket says 172.21.15.113:34000) 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cba800640 1 -- 172.21.15.113:0/1235899643 learned_addr learned my addr 172.21.15.113:0/1235899643 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cb9e00640 1 --2- 172.21.15.113:0/1235899643 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb414e4c0 0x7f3cb4072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:30.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cb9400640 1 --2- 172.21.15.113:0/1235899643 >> v2:172.21.15.136:3300/0 conn(0x7f3cb40c22e0 0x7f3cb40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:30.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cba800640 1 -- 172.21.15.113:0/1235899643 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb414e4c0 msgr2=0x7f3cb4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:30.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cba800640 1 --2- 172.21.15.113:0/1235899643 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb414e4c0 0x7f3cb4072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cba800640 1 -- 172.21.15.113:0/1235899643 >> v2:172.21.15.136:3300/0 conn(0x7f3cb40c22e0 msgr2=0x7f3cb40731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:30.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cba800640 1 --2- 172.21.15.113:0/1235899643 >> v2:172.21.15.136:3300/0 conn(0x7f3cb40c22e0 0x7f3cb40731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.065+0000 7f3cba800640 1 -- 172.21.15.113:0/1235899643 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4070c60 con 0x7f3cb40c2d80 2024-03-04T07:49:30.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cba800640 1 --2- 172.21.15.113:0/1235899643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c2d80 0x7f3cb415b1f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3cb406f370 tx=0x7f3cb00213e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98ab6225d4fbd1b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:30.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cb8a00640 1 -- 172.21.15.113:0/1235899643 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00b6070 con 0x7f3cb40c2d80 2024-03-04T07:49:30.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cb8a00640 1 -- 172.21.15.113:0/1235899643 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb000b2d0 con 0x7f3cb40c2d80 2024-03-04T07:49:30.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cb8a00640 1 -- 172.21.15.113:0/1235899643 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb000b480 con 0x7f3cb40c2d80 2024-03-04T07:49:30.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 -- 172.21.15.113:0/1235899643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c2d80 msgr2=0x7f3cb415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:30.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 --2- 172.21.15.113:0/1235899643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c2d80 0x7f3cb415b1f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3cb406f370 tx=0x7f3cb00213e0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 -- 172.21.15.113:0/1235899643 shutdown_connections 2024-03-04T07:49:30.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 --2- 172.21.15.113:0/1235899643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c2d80 0x7f3cb415b1f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 --2- 172.21.15.113:0/1235899643 >> v2:172.21.15.136:3300/0 conn(0x7f3cb40c22e0 0x7f3cb40731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 --2- 172.21.15.113:0/1235899643 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb414e4c0 0x7f3cb4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 -- 172.21.15.113:0/1235899643 >> 172.21.15.113:0/1235899643 conn(0x7f3cb406e820 msgr2=0x7f3cb406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:30.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 -- 172.21.15.113:0/1235899643 shutdown_connections 2024-03-04T07:49:30.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 -- 172.21.15.113:0/1235899643 wait complete. 2024-03-04T07:49:30.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 Processor -- start 2024-03-04T07:49:30.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.069+0000 7f3cbc800640 1 -- start start 2024-03-04T07:49:30.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cbc800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c22e0 0x7f3cb4152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:30.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cbc800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb40c2d80 0x7f3cb4158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:30.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cbc800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb414e4c0 0x7f3cb4152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:30.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cbc800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4071500 con 0x7f3cb40c22e0 2024-03-04T07:49:30.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cbc800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4071220 con 0x7f3cb414e4c0 2024-03-04T07:49:30.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cbc800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb414e8a0 con 0x7f3cb40c2d80 2024-03-04T07:49:30.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cb9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c22e0 0x7f3cb4152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:30.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cb9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c22e0 0x7f3cb4152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34004/0 (socket says 172.21.15.113:34004) 2024-03-04T07:49:30.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cb9e00640 1 -- 172.21.15.113:0/845439243 learned_addr learned my addr 172.21.15.113:0/845439243 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:30.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cb9400640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb40c2d80 0x7f3cb4158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:30.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cba800640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb414e4c0 0x7f3cb4152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:30.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cb9e00640 1 -- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb40c2d80 msgr2=0x7f3cb4158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:30.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cb9e00640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb40c2d80 0x7f3cb4158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cb9e00640 1 -- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb414e4c0 msgr2=0x7f3cb4152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:30.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cb9e00640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb414e4c0 0x7f3cb4152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cb9e00640 1 -- 172.21.15.113:0/845439243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4153660 con 0x7f3cb40c22e0 2024-03-04T07:49:30.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cba800640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb414e4c0 0x7f3cb4152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:30.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cb9e00640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c22e0 0x7f3cb4152230 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800d5f0 tx=0x7f3ca800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:30.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cae000640 1 -- 172.21.15.113:0/845439243 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80125f0 con 0x7f3cb40c22e0 2024-03-04T07:49:30.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cae000640 1 -- 172.21.15.113:0/845439243 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca8019070 con 0x7f3cb40c22e0 2024-03-04T07:49:30.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cbc800640 1 -- 172.21.15.113:0/845439243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb4152770 con 0x7f3cb40c22e0 2024-03-04T07:49:30.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cae000640 1 -- 172.21.15.113:0/845439243 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80129b0 con 0x7f3cb40c22e0 2024-03-04T07:49:30.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cbc800640 1 -- 172.21.15.113:0/845439243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb40c56c0 con 0x7f3cb40c22e0 2024-03-04T07:49:30.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.073+0000 7f3cbc800640 1 -- 172.21.15.113:0/845439243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb4075c00 con 0x7f3cb40c22e0 2024-03-04T07:49:30.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.077+0000 7f3cae000640 1 -- 172.21.15.113:0/845439243 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ca8012b50 con 0x7f3cb40c22e0 2024-03-04T07:49:30.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.077+0000 7f3cae000640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c98066f60 0x7f3c98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:30.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.077+0000 7f3cae000640 1 -- 172.21.15.113:0/845439243 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f3ca8085720 con 0x7f3cb40c22e0 2024-03-04T07:49:30.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.077+0000 7f3cb9400640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c98066f60 0x7f3c98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:30.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.077+0000 7f3cb9400640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c98066f60 0x7f3c98069420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4151f90 tx=0x7f3ca4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:30.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.081+0000 7f3cae000640 1 -- 172.21.15.113:0/845439243 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3ca80562d0 con 0x7f3cb40c22e0 2024-03-04T07:49:30.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.297+0000 7f3cbc800640 1 -- 172.21.15.113:0/845439243 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cb40c1260 con 0x7f3cb40c22e0 2024-03-04T07:49:30.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.297+0000 7f3cae000640 1 -- 172.21.15.113:0/845439243 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f3ca8055bd0 con 0x7f3cb40c22e0 2024-03-04T07:49:30.296 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:30.296 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":5940,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:48:13.835609+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5417,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5417":{"gid":5417,"name":"b","rank":0,"incarnation":57,"state":"up:active","state_seq":41,"addr":"172.21.15.136:6835/1691762848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1691762848},{"type":"v1","addr":"172.21.15.136:6835","nonce":1691762848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5417,"qdb_cluster":[5631,5529,5417]},"id":1}]} 2024-03-04T07:49:30.297 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 63 2024-03-04T07:49:30.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 -- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c98066f60 msgr2=0x7f3c98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:30.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c98066f60 0x7f3c98069420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4151f90 tx=0x7f3ca4008040 comp rx=0 tx=0).stop 2024-03-04T07:49:30.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 -- 172.21.15.113:0/845439243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c22e0 msgr2=0x7f3cb4152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:30.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c22e0 0x7f3cb4152230 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800d5f0 tx=0x7f3ca800dab0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 -- 172.21.15.113:0/845439243 shutdown_connections 2024-03-04T07:49:30.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb414e4c0 0x7f3cb4152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb40c2d80 0x7f3cb4158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c98066f60 0x7f3c98069420 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 --2- 172.21.15.113:0/845439243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb40c22e0 0x7f3cb4152230 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:30.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 -- 172.21.15.113:0/845439243 >> 172.21.15.113:0/845439243 conn(0x7f3cb406e820 msgr2=0x7f3cb40c7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:30.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 -- 172.21.15.113:0/845439243 shutdown_connections 2024-03-04T07:49:30.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:30.301+0000 7f3cbc800640 1 -- 172.21.15.113:0/845439243 wait complete. 2024-03-04T07:49:30.318 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:49:32.319 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:32.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.489+0000 7f06de400640 1 Processor -- start 2024-03-04T07:49:32.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.489+0000 7f06de400640 1 -- start start 2024-03-04T07:49:32.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06de400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f06d81527f0 0x7f06d8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06de400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06d8152bd0 0x7f06d81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06de400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d81591f0 con 0x7f06d80c8200 2024-03-04T07:49:32.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06de400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d8071df0 con 0x7f06d8152bd0 2024-03-04T07:49:32.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06de400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d81597b0 con 0x7f06d81527f0 2024-03-04T07:49:32.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06d7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06d8152bd0 0x7f06d81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06d7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06d8152bd0 0x7f06d81587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34010/0 (socket says 172.21.15.113:34010) 2024-03-04T07:49:32.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06d7e00640 1 -- 172.21.15.113:0/986490180 learned_addr learned my addr 172.21.15.113:0/986490180 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:32.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06d7400640 1 --2- 172.21.15.113:0/986490180 >> v2:172.21.15.136:3300/0 conn(0x7f06d81527f0 0x7f06d8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06d7e00640 1 -- 172.21.15.113:0/986490180 >> v2:172.21.15.136:3300/0 conn(0x7f06d81527f0 msgr2=0x7f06d8070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:32.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06d7e00640 1 --2- 172.21.15.113:0/986490180 >> v2:172.21.15.136:3300/0 conn(0x7f06d81527f0 0x7f06d8070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06d7e00640 1 -- 172.21.15.113:0/986490180 >> v1:172.21.15.113:6789/0 conn(0x7f06d80c8200 legacy=0x7f06d8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:32.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06d7e00640 1 -- 172.21.15.113:0/986490180 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d81594d0 con 0x7f06d8152bd0 2024-03-04T07:49:32.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06d6a00640 1 -- 172.21.15.113:0/986490180 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904910597 0 0) 0x7f06d81591f0 con 0x7f06d80c8200 2024-03-04T07:49:32.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.493+0000 7f06d7400640 1 --2- 172.21.15.113:0/986490180 >> v2:172.21.15.136:3300/0 conn(0x7f06d81527f0 0x7f06d8070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:32.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06d7e00640 1 --2- 172.21.15.113:0/986490180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 0x7f06d81587d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f06d806f400 tx=0x7f06d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885bf28677b459a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:32.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06d6a00640 1 -- 172.21.15.113:0/986490180 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d00a57a0 con 0x7f06d8152bd0 2024-03-04T07:49:32.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06d6a00640 1 -- 172.21.15.113:0/986490180 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d00a5940 con 0x7f06d8152bd0 2024-03-04T07:49:32.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06d6a00640 1 -- 172.21.15.113:0/986490180 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d00a5d10 con 0x7f06d8152bd0 2024-03-04T07:49:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06de400640 1 -- 172.21.15.113:0/986490180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 msgr2=0x7f06d81587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06de400640 1 --2- 172.21.15.113:0/986490180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 0x7f06d81587d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f06d806f400 tx=0x7f06d001c9f0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06de400640 1 -- 172.21.15.113:0/986490180 shutdown_connections 2024-03-04T07:49:32.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06de400640 1 --2- 172.21.15.113:0/986490180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 0x7f06d81587d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06de400640 1 --2- 172.21.15.113:0/986490180 >> v2:172.21.15.136:3300/0 conn(0x7f06d81527f0 0x7f06d8070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06de400640 1 -- 172.21.15.113:0/986490180 >> 172.21.15.113:0/986490180 conn(0x7f06d806e820 msgr2=0x7f06d806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:32.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06de400640 1 -- 172.21.15.113:0/986490180 shutdown_connections 2024-03-04T07:49:32.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.497+0000 7f06de400640 1 -- 172.21.15.113:0/986490180 wait complete. 2024-03-04T07:49:32.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.501+0000 7f06de400640 1 Processor -- start 2024-03-04T07:49:32.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.501+0000 7f06de400640 1 -- start start 2024-03-04T07:49:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.501+0000 7f06de400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d80c8200 0x7f06d8153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.501+0000 7f06de400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d81527f0 0x7f06d80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.501+0000 7f06de400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 0x7f06d8149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.501+0000 7f06de400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d81654f0 con 0x7f06d8152bd0 2024-03-04T07:49:32.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.501+0000 7f06de400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8165370 con 0x7f06d81527f0 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.501+0000 7f06de400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8165670 con 0x7f06d80c8200 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06d7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 0x7f06d8149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06d7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 0x7f06d8149fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34018/0 (socket says 172.21.15.113:34018) 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06d7e00640 1 -- 172.21.15.113:0/296199023 learned_addr learned my addr 172.21.15.113:0/296199023 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06cfe00640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d81527f0 0x7f06d80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06d7400640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d80c8200 0x7f06d8153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06d7e00640 1 -- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d80c8200 msgr2=0x7f06d8153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06d7e00640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d80c8200 0x7f06d8153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06d7e00640 1 -- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d81527f0 msgr2=0x7f06d80b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06d7e00640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d81527f0 0x7f06d80b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06d7e00640 1 -- 172.21.15.113:0/296199023 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d81503f0 con 0x7f06d8152bd0 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06cfe00640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d81527f0 0x7f06d80b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06d7e00640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 0x7f06d8149fd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f06d001b0a0 tx=0x7f06d0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:32.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06cf400640 1 -- 172.21.15.113:0/296199023 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d00bc030 con 0x7f06d8152bd0 2024-03-04T07:49:32.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06cf400640 1 -- 172.21.15.113:0/296199023 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d00b8470 con 0x7f06d8152bd0 2024-03-04T07:49:32.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06de400640 1 -- 172.21.15.113:0/296199023 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d81f7a30 con 0x7f06d8152bd0 2024-03-04T07:49:32.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06cf400640 1 -- 172.21.15.113:0/296199023 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d00b8820 con 0x7f06d8152bd0 2024-03-04T07:49:32.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06de400640 1 -- 172.21.15.113:0/296199023 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d8150b70 con 0x7f06d8152bd0 2024-03-04T07:49:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06de400640 1 -- 172.21.15.113:0/296199023 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d8075c00 con 0x7f06d8152bd0 2024-03-04T07:49:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.505+0000 7f06cf400640 1 -- 172.21.15.113:0/296199023 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06d00bf070 con 0x7f06d8152bd0 2024-03-04T07:49:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.509+0000 7f06cf400640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b4066f60 0x7f06b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.509+0000 7f06cf400640 1 -- 172.21.15.113:0/296199023 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f06d0120460 con 0x7f06d8152bd0 2024-03-04T07:49:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.509+0000 7f06d7400640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b4066f60 0x7f06b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.509+0000 7f06d7400640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b4066f60 0x7f06b4069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f06c800c900 tx=0x7f06c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:32.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.513+0000 7f06cf400640 1 -- 172.21.15.113:0/296199023 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f06d00165e0 con 0x7f06d8152bd0 2024-03-04T07:49:32.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.777+0000 7f06de400640 1 -- 172.21.15.113:0/296199023 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06d80b2330 con 0x7f06d8152bd0 2024-03-04T07:49:32.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.777+0000 7f06cf400640 1 -- 172.21.15.113:0/296199023 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f06d00f1180 con 0x7f06d8152bd0 2024-03-04T07:49:32.777 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:32.778 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:30.698677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:49:32.778 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 64 2024-03-04T07:49:32.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 -- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b4066f60 msgr2=0x7f06b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:32.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b4066f60 0x7f06b4069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f06c800c900 tx=0x7f06c8008040 comp rx=0 tx=0).stop 2024-03-04T07:49:32.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 -- 172.21.15.113:0/296199023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 msgr2=0x7f06d8149fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:32.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 0x7f06d8149fd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f06d001b0a0 tx=0x7f06d0019790 comp rx=0 tx=0).stop 2024-03-04T07:49:32.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 -- 172.21.15.113:0/296199023 shutdown_connections 2024-03-04T07:49:32.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06b4066f60 0x7f06b4069420 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06d8152bd0 0x7f06d8149fd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06d81527f0 0x7f06d80b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 --2- 172.21.15.113:0/296199023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06d80c8200 0x7f06d8153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 -- 172.21.15.113:0/296199023 >> 172.21.15.113:0/296199023 conn(0x7f06d806e820 msgr2=0x7f06d80ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:32.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 -- 172.21.15.113:0/296199023 shutdown_connections 2024-03-04T07:49:32.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.781+0000 7f06de400640 1 -- 172.21.15.113:0/296199023 wait complete. 2024-03-04T07:49:32.799 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2024-03-04T07:49:32.799 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:49:32.800 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:32.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.981+0000 7f21ea200640 1 Processor -- start 2024-03-04T07:49:32.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.981+0000 7f21ea200640 1 -- start start 2024-03-04T07:49:32.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21ea200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f21e40c22e0 0x7f21e415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21ea200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21e40c2d80 0x7f21e415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21ea200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e40c27f0 con 0x7f21e414e4c0 2024-03-04T07:49:32.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21ea200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e415ad60 con 0x7f21e40c2d80 2024-03-04T07:49:32.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21ea200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e40c2a90 con 0x7f21e40c22e0 2024-03-04T07:49:32.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21e3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21e40c2d80 0x7f21e415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21e3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21e40c2d80 0x7f21e415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34034/0 (socket says 172.21.15.113:34034) 2024-03-04T07:49:32.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21e3e00640 1 -- 172.21.15.113:0/1332462350 learned_addr learned my addr 172.21.15.113:0/1332462350 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:32.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21daa00640 1 --2- 172.21.15.113:0/1332462350 >> v2:172.21.15.136:3300/0 conn(0x7f21e40c22e0 0x7f21e415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21e3e00640 1 -- 172.21.15.113:0/1332462350 >> v2:172.21.15.136:3300/0 conn(0x7f21e40c22e0 msgr2=0x7f21e415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:32.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21e3e00640 1 --2- 172.21.15.113:0/1332462350 >> v2:172.21.15.136:3300/0 conn(0x7f21e40c22e0 0x7f21e415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21e3e00640 1 -- 172.21.15.113:0/1332462350 >> v1:172.21.15.136:6789/0 conn(0x7f21e414e4c0 legacy=0x7f21e4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:32.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.985+0000 7f21e3e00640 1 -- 172.21.15.113:0/1332462350 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4070c60 con 0x7f21e40c2d80 2024-03-04T07:49:32.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21e3e00640 1 --2- 172.21.15.113:0/1332462350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c2d80 0x7f21e415b730 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cd230 tx=0x7f21dc01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b786530fc957fc11 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:32.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21e2a00640 1 -- 172.21.15.113:0/1332462350 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc00b2d0 con 0x7f21e40c2d80 2024-03-04T07:49:32.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21e2a00640 1 -- 172.21.15.113:0/1332462350 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc00b470 con 0x7f21e40c2d80 2024-03-04T07:49:32.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21e2a00640 1 -- 172.21.15.113:0/1332462350 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0a7af0 con 0x7f21e40c2d80 2024-03-04T07:49:32.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21ea200640 1 -- 172.21.15.113:0/1332462350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c2d80 msgr2=0x7f21e415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:32.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21ea200640 1 --2- 172.21.15.113:0/1332462350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c2d80 0x7f21e415b730 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f21e40cd230 tx=0x7f21dc01ebf0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21ea200640 1 -- 172.21.15.113:0/1332462350 shutdown_connections 2024-03-04T07:49:32.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21ea200640 1 --2- 172.21.15.113:0/1332462350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c2d80 0x7f21e415b730 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21ea200640 1 --2- 172.21.15.113:0/1332462350 >> v2:172.21.15.136:3300/0 conn(0x7f21e40c22e0 0x7f21e415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21ea200640 1 -- 172.21.15.113:0/1332462350 >> 172.21.15.113:0/1332462350 conn(0x7f21e406e820 msgr2=0x7f21e406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:32.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21ea200640 1 -- 172.21.15.113:0/1332462350 shutdown_connections 2024-03-04T07:49:32.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.989+0000 7f21ea200640 1 -- 172.21.15.113:0/1332462350 wait complete. 2024-03-04T07:49:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21ea200640 1 Processor -- start 2024-03-04T07:49:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21ea200640 1 -- start start 2024-03-04T07:49:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21ea200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c22e0 0x7f21e4144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21ea200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e40c2d80 0x7f21e4153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21ea200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e414e4c0 0x7f21e4154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21ea200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40aede0 con 0x7f21e40c22e0 2024-03-04T07:49:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21ea200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e414e8a0 con 0x7f21e414e4c0 2024-03-04T07:49:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21ea200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4073390 con 0x7f21e40c2d80 2024-03-04T07:49:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21e3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c22e0 0x7f21e4144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21e3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c22e0 0x7f21e4144e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34038/0 (socket says 172.21.15.113:34038) 2024-03-04T07:49:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21e3400640 1 -- 172.21.15.113:0/2041014208 learned_addr learned my addr 172.21.15.113:0/2041014208 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21daa00640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e40c2d80 0x7f21e4153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21e3e00640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e414e4c0 0x7f21e4154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21e3400640 1 -- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e40c2d80 msgr2=0x7f21e4153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21e3400640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e40c2d80 0x7f21e4153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21e3400640 1 -- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e414e4c0 msgr2=0x7f21e4154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21e3400640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e414e4c0 0x7f21e4154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21e3400640 1 -- 172.21.15.113:0/2041014208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e41561d0 con 0x7f21e40c22e0 2024-03-04T07:49:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21e3400640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c22e0 0x7f21e4144e00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c500 tx=0x7f21d400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:32.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21dbe00640 1 -- 172.21.15.113:0/2041014208 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4016020 con 0x7f21e40c22e0 2024-03-04T07:49:32.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21dbe00640 1 -- 172.21.15.113:0/2041014208 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d4005220 con 0x7f21e40c22e0 2024-03-04T07:49:32.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21ea200640 1 -- 172.21.15.113:0/2041014208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e40c9240 con 0x7f21e40c22e0 2024-03-04T07:49:32.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21dbe00640 1 -- 172.21.15.113:0/2041014208 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4007030 con 0x7f21e40c22e0 2024-03-04T07:49:32.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.993+0000 7f21ea200640 1 -- 172.21.15.113:0/2041014208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e41e9d10 con 0x7f21e40c22e0 2024-03-04T07:49:32.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.997+0000 7f21ea200640 1 -- 172.21.15.113:0/2041014208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e4075c00 con 0x7f21e40c22e0 2024-03-04T07:49:32.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.997+0000 7f21dbe00640 1 -- 172.21.15.113:0/2041014208 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21d40071f0 con 0x7f21e40c22e0 2024-03-04T07:49:32.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.997+0000 7f21dbe00640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21c0066f60 0x7f21c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:32.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:32.997+0000 7f21dbe00640 1 -- 172.21.15.113:0/2041014208 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f21d4084cb0 con 0x7f21e40c22e0 2024-03-04T07:49:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.001+0000 7f21daa00640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21c0066f60 0x7f21c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:32.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.001+0000 7f21daa00640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21c0066f60 0x7f21c0069420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f21e4074a90 tx=0x7f21cc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:33.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.009+0000 7f21dbe00640 1 -- 172.21.15.113:0/2041014208 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f21d4020020 con 0x7f21e40c22e0 2024-03-04T07:49:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.201+0000 7f21ea200640 1 -- 172.21.15.113:0/2041014208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21e4155190 con 0x7f21e40c22e0 2024-03-04T07:49:33.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.201+0000 7f21dbe00640 1 -- 172.21.15.113:0/2041014208 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f21e41561d0 con 0x7f21e40c22e0 2024-03-04T07:49:33.202 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:33.202 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:30.698677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:49:33.202 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 64 2024-03-04T07:49:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 -- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21c0066f60 msgr2=0x7f21c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21c0066f60 0x7f21c0069420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f21e4074a90 tx=0x7f21cc009190 comp rx=0 tx=0).stop 2024-03-04T07:49:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 -- 172.21.15.113:0/2041014208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c22e0 msgr2=0x7f21e4144e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c22e0 0x7f21e4144e00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c500 tx=0x7f21d400c9c0 comp rx=0 tx=0).stop 2024-03-04T07:49:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 -- 172.21.15.113:0/2041014208 shutdown_connections 2024-03-04T07:49:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e414e4c0 0x7f21e4154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e40c2d80 0x7f21e4153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21c0066f60 0x7f21c0069420 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 --2- 172.21.15.113:0/2041014208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e40c22e0 0x7f21e4144e00 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 -- 172.21.15.113:0/2041014208 >> 172.21.15.113:0/2041014208 conn(0x7f21e406e820 msgr2=0x7f21e414c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 -- 172.21.15.113:0/2041014208 shutdown_connections 2024-03-04T07:49:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:33.205+0000 7f21ea200640 1 -- 172.21.15.113:0/2041014208 wait complete. 2024-03-04T07:49:33.224 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 64, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 64, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5529": { "addr": "172.21.15.113:6836/2299268553", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2299268553, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2299268553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5529, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 43 }, "gid_5631": { "addr": "172.21.15.113:6833/810789479", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 810789479, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 810789479, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5631, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 17 }, "gid_5940": { "addr": "172.21.15.113:6837/344022786", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 344022786, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 344022786, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5940, "incarnation": 64, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 57, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:49:30.698677+0000", "qdb_cluster": [ 5529, 5631 ], "qdb_leader": 5529, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5940, "mds_1": 5529, "mds_2": 5631 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:49:35.225 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:35.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.369+0000 7f21e0e00640 1 Processor -- start 2024-03-04T07:49:35.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.369+0000 7f21e0e00640 1 -- start start 2024-03-04T07:49:35.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21e0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc152510 0x7f21dc0c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:35.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21e0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f21dc066db0 0x7f21dc162fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:35.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21e0e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc158770 con 0x7f21dc1528f0 2024-03-04T07:49:35.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21e0e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21dc159030 con 0x7f21dc066db0 2024-03-04T07:49:35.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21e0e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0c6520 con 0x7f21dc152510 2024-03-04T07:49:35.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc152510 0x7f21dc0c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:35.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f21dc066db0 0x7f21dc162fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:35.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc152510 0x7f21dc0c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:57256/0 (socket says 172.21.15.113:57256) 2024-03-04T07:49:35.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21db400640 1 -- 172.21.15.113:0/2871831192 learned_addr learned my addr 172.21.15.113:0/2871831192 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:35.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21daa00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f21dc1528f0 0x7f21dc0c84b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:36210/0 (socket says 172.21.15.113:36210) 2024-03-04T07:49:35.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21db400640 1 -- 172.21.15.113:0/2871831192 >> v2:172.21.15.136:3300/0 conn(0x7f21dc066db0 msgr2=0x7f21dc162fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:35.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21db400640 1 --2- 172.21.15.113:0/2871831192 >> v2:172.21.15.136:3300/0 conn(0x7f21dc066db0 0x7f21dc162fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:35.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21db400640 1 -- 172.21.15.113:0/2871831192 >> v1:172.21.15.136:6789/0 conn(0x7f21dc1528f0 legacy=0x7f21dc0c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:35.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21db400640 1 -- 172.21.15.113:0/2871831192 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc0c67e0 con 0x7f21dc152510 2024-03-04T07:49:35.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.373+0000 7f21dbe00640 1 --2- 172.21.15.113:0/2871831192 >> v2:172.21.15.136:3300/0 conn(0x7f21dc066db0 0x7f21dc162fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:49:35.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21db400640 1 --2- 172.21.15.113:0/2871831192 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc152510 0x7f21dc0c3d70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009870 tx=0x7f21cc01ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31ec06d635f219c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:35.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21da000640 1 -- 172.21.15.113:0/2871831192 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0b5070 con 0x7f21dc152510 2024-03-04T07:49:35.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21da000640 1 -- 172.21.15.113:0/2871831192 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc0a8a50 con 0x7f21dc152510 2024-03-04T07:49:35.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21da000640 1 -- 172.21.15.113:0/2871831192 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0a8e20 con 0x7f21dc152510 2024-03-04T07:49:35.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21e0e00640 1 -- 172.21.15.113:0/2871831192 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc152510 msgr2=0x7f21dc0c3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:35.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21e0e00640 1 --2- 172.21.15.113:0/2871831192 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc152510 0x7f21dc0c3d70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009870 tx=0x7f21cc01ed50 comp rx=0 tx=0).stop 2024-03-04T07:49:35.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21e0e00640 1 -- 172.21.15.113:0/2871831192 shutdown_connections 2024-03-04T07:49:35.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21e0e00640 1 --2- 172.21.15.113:0/2871831192 >> v2:172.21.15.136:3300/0 conn(0x7f21dc066db0 0x7f21dc162fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:35.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21e0e00640 1 --2- 172.21.15.113:0/2871831192 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc152510 0x7f21dc0c3d70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:35.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21e0e00640 1 -- 172.21.15.113:0/2871831192 >> 172.21.15.113:0/2871831192 conn(0x7f21dc0cc660 msgr2=0x7f21dc0cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:35.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21e0e00640 1 -- 172.21.15.113:0/2871831192 shutdown_connections 2024-03-04T07:49:35.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21e0e00640 1 -- 172.21.15.113:0/2871831192 wait complete. 2024-03-04T07:49:35.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.377+0000 7f21e0e00640 1 Processor -- start 2024-03-04T07:49:35.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21e0e00640 1 -- start start 2024-03-04T07:49:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21e0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc066db0 0x7f21dc0c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21e0e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21dc152510 0x7f21dc0c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21e0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21dc1528f0 0x7f21dc1f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21e0e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc158cc0 con 0x7f21dc152510 2024-03-04T07:49:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21e0e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc1589e0 con 0x7f21dc1528f0 2024-03-04T07:49:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21e0e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0c6aa0 con 0x7f21dc066db0 2024-03-04T07:49:35.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21daa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21dc152510 0x7f21dc0c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21daa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21dc152510 0x7f21dc0c2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34054/0 (socket says 172.21.15.113:34054) 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21daa00640 1 -- 172.21.15.113:0/2310829316 learned_addr learned my addr 172.21.15.113:0/2310829316 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21db400640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc066db0 0x7f21dc0c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21dbe00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21dc1528f0 0x7f21dc1f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21daa00640 1 -- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc066db0 msgr2=0x7f21dc0c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21daa00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc066db0 0x7f21dc0c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21daa00640 1 -- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21dc1528f0 msgr2=0x7f21dc1f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21daa00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21dc1528f0 0x7f21dc1f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21daa00640 1 -- 172.21.15.113:0/2310829316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc1574b0 con 0x7f21dc152510 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21dbe00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21dc1528f0 0x7f21dc1f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21daa00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21dc152510 0x7f21dc0c2b10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f21c800c430 tx=0x7f21c800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21d3400640 1 -- 172.21.15.113:0/2310829316 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c80074e0 con 0x7f21dc152510 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21d3400640 1 -- 172.21.15.113:0/2310829316 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c8007680 con 0x7f21dc152510 2024-03-04T07:49:35.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21d3400640 1 -- 172.21.15.113:0/2310829316 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c8004a30 con 0x7f21dc152510 2024-03-04T07:49:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21e0e00640 1 -- 172.21.15.113:0/2310829316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc157fc0 con 0x7f21dc152510 2024-03-04T07:49:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21e0e00640 1 -- 172.21.15.113:0/2310829316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc157a00 con 0x7f21dc152510 2024-03-04T07:49:35.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.381+0000 7f21e0e00640 1 -- 172.21.15.113:0/2310829316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc075a80 con 0x7f21dc152510 2024-03-04T07:49:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.385+0000 7f21d3400640 1 -- 172.21.15.113:0/2310829316 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21c801a020 con 0x7f21dc152510 2024-03-04T07:49:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.385+0000 7f21d3400640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f60 0x7f21bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:35.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.385+0000 7f21d3400640 1 -- 172.21.15.113:0/2310829316 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f21c8084a90 con 0x7f21dc152510 2024-03-04T07:49:35.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.385+0000 7f21db400640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f60 0x7f21bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:35.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.385+0000 7f21db400640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f60 0x7f21bc069420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0b2000 tx=0x7f21cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:35.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.393+0000 7f21d3400640 1 -- 172.21.15.113:0/2310829316 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f21c80557b0 con 0x7f21dc152510 2024-03-04T07:49:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.597+0000 7f21e0e00640 1 -- 172.21.15.113:0/2310829316 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21dc0712d0 con 0x7f21dc152510 2024-03-04T07:49:35.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.597+0000 7f21d3400640 1 -- 172.21.15.113:0/2310829316 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f21c80550b0 con 0x7f21dc152510 2024-03-04T07:49:35.597 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:35.597 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:30.698677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:49:35.597 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 64 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 -- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f60 msgr2=0x7f21bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f60 0x7f21bc069420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0b2000 tx=0x7f21cc009040 comp rx=0 tx=0).stop 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 -- 172.21.15.113:0/2310829316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21dc152510 msgr2=0x7f21dc0c2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21dc152510 0x7f21dc0c2b10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f21c800c430 tx=0x7f21c800c8f0 comp rx=0 tx=0).stop 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 -- 172.21.15.113:0/2310829316 shutdown_connections 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21dc1528f0 0x7f21dc1f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f60 0x7f21bc069420 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21dc152510 0x7f21dc0c2b10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 --2- 172.21.15.113:0/2310829316 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21dc066db0 0x7f21dc0c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 -- 172.21.15.113:0/2310829316 >> 172.21.15.113:0/2310829316 conn(0x7f21dc0cc660 msgr2=0x7f21dc151560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 -- 172.21.15.113:0/2310829316 shutdown_connections 2024-03-04T07:49:35.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:35.601+0000 7f21e0e00640 1 -- 172.21.15.113:0/2310829316 wait complete. 2024-03-04T07:49:35.628 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:49:37.629 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:37.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0149e00640 1 Processor -- start 2024-03-04T07:49:37.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0149e00640 1 -- start start 2024-03-04T07:49:37.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0149e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0144157f80 0x7f014406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:37.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0149e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01440c9ce0 0x7f01440ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:37.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0149e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01440ca670 con 0x7f01440c8a90 2024-03-04T07:49:37.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0149e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014406e1e0 con 0x7f0144157f80 2024-03-04T07:49:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0149e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014406df00 con 0x7f01440c9ce0 2024-03-04T07:49:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0148a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0144157f80 0x7f014406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0149400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01440c9ce0 0x7f01440ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0148a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0144157f80 0x7f014406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:59174/0 (socket says 172.21.15.113:59174) 2024-03-04T07:49:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f0148a00640 1 -- 172.21.15.113:0/2353874384 learned_addr learned my addr 172.21.15.113:0/2353874384 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.781+0000 7f013be00640 1 --1- 172.21.15.113:0/2353874384 >> v1:172.21.15.136:6789/0 conn(0x7f01440c8a90 0x7f014406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:50224/0 (socket says 172.21.15.113:50224) 2024-03-04T07:49:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.785+0000 7f0148a00640 1 -- 172.21.15.113:0/2353874384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01440c9ce0 msgr2=0x7f01440ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.785+0000 7f0148a00640 1 --2- 172.21.15.113:0/2353874384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01440c9ce0 0x7f01440ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.785+0000 7f0148a00640 1 -- 172.21.15.113:0/2353874384 >> v1:172.21.15.136:6789/0 conn(0x7f01440c8a90 legacy=0x7f014406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:37.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.785+0000 7f0148a00640 1 -- 172.21.15.113:0/2353874384 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440ce1a0 con 0x7f0144157f80 2024-03-04T07:49:37.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.785+0000 7f0143e00640 1 -- 172.21.15.113:0/2353874384 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756404408 0 0) 0x7f01440ca670 con 0x7f01440c8a90 2024-03-04T07:49:37.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.785+0000 7f0148a00640 1 --2- 172.21.15.113:0/2353874384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0144157f80 0x7f014406c170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0134009870 tx=0x7f013401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1601c6a8c52c4014 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.785+0000 7f0143e00640 1 -- 172.21.15.113:0/2353874384 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01340b2070 con 0x7f0144157f80 2024-03-04T07:49:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.785+0000 7f0143e00640 1 -- 172.21.15.113:0/2353874384 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01340a9070 con 0x7f0144157f80 2024-03-04T07:49:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.785+0000 7f0143e00640 1 -- 172.21.15.113:0/2353874384 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01340ad880 con 0x7f0144157f80 2024-03-04T07:49:37.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 -- 172.21.15.113:0/2353874384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0144157f80 msgr2=0x7f014406c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:37.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 --2- 172.21.15.113:0/2353874384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0144157f80 0x7f014406c170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0134009870 tx=0x7f013401c560 comp rx=0 tx=0).stop 2024-03-04T07:49:37.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 -- 172.21.15.113:0/2353874384 shutdown_connections 2024-03-04T07:49:37.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 --2- 172.21.15.113:0/2353874384 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01440c9ce0 0x7f01440ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:37.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 --2- 172.21.15.113:0/2353874384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0144157f80 0x7f014406c170 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:37.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 -- 172.21.15.113:0/2353874384 >> 172.21.15.113:0/2353874384 conn(0x7f0144072f40 msgr2=0x7f01440706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:37.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 -- 172.21.15.113:0/2353874384 shutdown_connections 2024-03-04T07:49:37.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 -- 172.21.15.113:0/2353874384 wait complete. 2024-03-04T07:49:37.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 Processor -- start 2024-03-04T07:49:37.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 -- start start 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01440c8a90 0x7f014414c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01440c9ce0 0x7f014414d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0144157f80 0x7f01441f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014406fc30 con 0x7f01440c8a90 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01440c8e90 con 0x7f0144157f80 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0149e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014406fdb0 con 0x7f01440c9ce0 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.789+0000 7f0148a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01440c8a90 0x7f014414c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0148a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01440c8a90 0x7f014414c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39350/0 (socket says 172.21.15.113:39350) 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0148a00640 1 -- 172.21.15.113:0/133443348 learned_addr learned my addr 172.21.15.113:0/133443348 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f013be00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01440c9ce0 0x7f014414d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0149400640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0144157f80 0x7f01441f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0148a00640 1 -- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01440c9ce0 msgr2=0x7f014414d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0148a00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01440c9ce0 0x7f014414d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0148a00640 1 -- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0144157f80 msgr2=0x7f01441f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0148a00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0144157f80 0x7f01441f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0148a00640 1 -- 172.21.15.113:0/133443348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01440cd880 con 0x7f01440c8a90 2024-03-04T07:49:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0148a00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01440c8a90 0x7f014414c190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0134009870 tx=0x7f013401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0141600640 1 -- 172.21.15.113:0/133443348 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01340b2040 con 0x7f01440c8a90 2024-03-04T07:49:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0141600640 1 -- 172.21.15.113:0/133443348 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01340add00 con 0x7f01440c8a90 2024-03-04T07:49:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0149e00640 1 -- 172.21.15.113:0/133443348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014414f5d0 con 0x7f01440c8a90 2024-03-04T07:49:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0141600640 1 -- 172.21.15.113:0/133443348 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134004220 con 0x7f01440c8a90 2024-03-04T07:49:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0149e00640 1 -- 172.21.15.113:0/133443348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0144150f70 con 0x7f01440c8a90 2024-03-04T07:49:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0149e00640 1 -- 172.21.15.113:0/133443348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0144075a80 con 0x7f01440c8a90 2024-03-04T07:49:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.793+0000 7f0141600640 1 -- 172.21.15.113:0/133443348 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01340c0060 con 0x7f01440c8a90 2024-03-04T07:49:37.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.797+0000 7f0141600640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0120066f60 0x7f0120069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:37.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.797+0000 7f0141600640 1 -- 172.21.15.113:0/133443348 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f013411ea70 con 0x7f01440c8a90 2024-03-04T07:49:37.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.797+0000 7f013be00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0120066f60 0x7f0120069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:37.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.797+0000 7f013be00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0120066f60 0x7f0120069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f01440b2b20 tx=0x7f012c007df0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:37.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:37.801+0000 7f0141600640 1 -- 172.21.15.113:0/133443348 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0134016150 con 0x7f01440c8a90 2024-03-04T07:49:38.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.089+0000 7f0149e00640 1 -- 172.21.15.113:0/133443348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01441567b0 con 0x7f01440c8a90 2024-03-04T07:49:38.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.089+0000 7f0141600640 1 -- 172.21.15.113:0/133443348 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f01340eef80 con 0x7f01440c8a90 2024-03-04T07:49:38.088 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:38.088 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:30.698677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:49:38.088 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 64 2024-03-04T07:49:38.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 -- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0120066f60 msgr2=0x7f0120069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:38.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0120066f60 0x7f0120069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f01440b2b20 tx=0x7f012c007df0 comp rx=0 tx=0).stop 2024-03-04T07:49:38.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 -- 172.21.15.113:0/133443348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01440c8a90 msgr2=0x7f014414c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:38.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01440c8a90 0x7f014414c190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0134009870 tx=0x7f013401ada0 comp rx=0 tx=0).stop 2024-03-04T07:49:38.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 -- 172.21.15.113:0/133443348 shutdown_connections 2024-03-04T07:49:38.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0144157f80 0x7f01441f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:38.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01440c9ce0 0x7f014414d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:38.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0120066f60 0x7f0120069420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:38.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 --2- 172.21.15.113:0/133443348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01440c8a90 0x7f014414c190 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:38.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 -- 172.21.15.113:0/133443348 >> 172.21.15.113:0/133443348 conn(0x7f0144072f40 msgr2=0x7f0144156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:38.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 -- 172.21.15.113:0/133443348 shutdown_connections 2024-03-04T07:49:38.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:38.093+0000 7f0149e00640 1 -- 172.21.15.113:0/133443348 wait complete. 2024-03-04T07:49:38.119 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:49:40.120 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.273+0000 7fad5d200640 1 Processor -- start 2024-03-04T07:49:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.273+0000 7fad5d200640 1 -- start start 2024-03-04T07:49:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.273+0000 7fad5d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad58160820 0x7fad58162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.273+0000 7fad5d200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad580c1ec0 con 0x7fad580b20b0 2024-03-04T07:49:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.273+0000 7fad5d200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad580c9b30 con 0x7fad580c85f0 2024-03-04T07:49:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.273+0000 7fad5d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad580c12a0 con 0x7fad58160820 2024-03-04T07:49:40.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.273+0000 7fad56800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad58160820 0x7fad58162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:40.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.273+0000 7fad56800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad58160820 0x7fad58162c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39356/0 (socket says 172.21.15.113:39356) 2024-03-04T07:49:40.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.273+0000 7fad55e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fad580b20b0 0x7fad58157710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36516/0 (socket says 172.21.15.113:36516) 2024-03-04T07:49:40.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.273+0000 7fad56800640 1 -- 172.21.15.113:0/3017497627 learned_addr learned my addr 172.21.15.113:0/3017497627 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:40.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.277+0000 7fad56800640 1 -- 172.21.15.113:0/3017497627 >> v1:172.21.15.136:6789/0 conn(0x7fad580c85f0 legacy=0x7fad5806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:40.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.277+0000 7fad56800640 1 -- 172.21.15.113:0/3017497627 >> v1:172.21.15.113:6789/0 conn(0x7fad580b20b0 legacy=0x7fad58157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:40.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.277+0000 7fad56800640 1 -- 172.21.15.113:0/3017497627 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580c1580 con 0x7fad58160820 2024-03-04T07:49:40.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad56800640 1 --2- 172.21.15.113:0/3017497627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 0x7fad58162c10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fad50009e50 tx=0x7fad5001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2831b81f1245fe server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:40.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad54a00640 1 -- 172.21.15.113:0/3017497627 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad500b4070 con 0x7fad58160820 2024-03-04T07:49:40.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad54a00640 1 -- 172.21.15.113:0/3017497627 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad500af400 con 0x7fad58160820 2024-03-04T07:49:40.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad54a00640 1 -- 172.21.15.113:0/3017497627 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad500af7d0 con 0x7fad58160820 2024-03-04T07:49:40.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad5d200640 1 -- 172.21.15.113:0/3017497627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 msgr2=0x7fad58162c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:40.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad5d200640 1 --2- 172.21.15.113:0/3017497627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 0x7fad58162c10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fad50009e50 tx=0x7fad5001f210 comp rx=0 tx=0).stop 2024-03-04T07:49:40.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad5d200640 1 -- 172.21.15.113:0/3017497627 shutdown_connections 2024-03-04T07:49:40.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad5d200640 1 --2- 172.21.15.113:0/3017497627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 0x7fad58162c10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:40.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad5d200640 1 -- 172.21.15.113:0/3017497627 >> 172.21.15.113:0/3017497627 conn(0x7fad5806e820 msgr2=0x7fad5806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:40.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad5d200640 1 -- 172.21.15.113:0/3017497627 shutdown_connections 2024-03-04T07:49:40.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.281+0000 7fad5d200640 1 -- 172.21.15.113:0/3017497627 wait complete. 2024-03-04T07:49:40.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad5d200640 1 Processor -- start 2024-03-04T07:49:40.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad5d200640 1 -- start start 2024-03-04T07:49:40.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad5d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad580b20b0 0x7fad58141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:40.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad5d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad580c85f0 0x7fad580c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:40.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad5d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 0x7fad580c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:40.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad5d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad58157e20 con 0x7fad58160820 2024-03-04T07:49:40.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad5d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad581571f0 con 0x7fad580b20b0 2024-03-04T07:49:40.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad5d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580c2400 con 0x7fad580c85f0 2024-03-04T07:49:40.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad56800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 0x7fad580c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:40.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad55e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad580b20b0 0x7fad58141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:40.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad56800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 0x7fad580c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39360/0 (socket says 172.21.15.113:39360) 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad56800640 1 -- 172.21.15.113:0/2969639044 learned_addr learned my addr 172.21.15.113:0/2969639044 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad55400640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad580c85f0 0x7fad580c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad56800640 1 -- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad580c85f0 msgr2=0x7fad580c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad56800640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad580c85f0 0x7fad580c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad56800640 1 -- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad580b20b0 msgr2=0x7fad58141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad56800640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad580b20b0 0x7fad58141d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad56800640 1 -- 172.21.15.113:0/2969639044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad58071840 con 0x7fad58160820 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.285+0000 7fad56800640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 0x7fad580c6210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fad5001be20 tx=0x7fad5001bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.289+0000 7fad4e000640 1 -- 172.21.15.113:0/2969639044 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad500be030 con 0x7fad58160820 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.289+0000 7fad4e000640 1 -- 172.21.15.113:0/2969639044 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad500afdb0 con 0x7fad58160820 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.289+0000 7fad4e000640 1 -- 172.21.15.113:0/2969639044 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad500b4040 con 0x7fad58160820 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.289+0000 7fad5d200640 1 -- 172.21.15.113:0/2969639044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad581f0c00 con 0x7fad58160820 2024-03-04T07:49:40.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.289+0000 7fad5d200640 1 -- 172.21.15.113:0/2969639044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad581f11c0 con 0x7fad58160820 2024-03-04T07:49:40.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.289+0000 7fad5d200640 1 -- 172.21.15.113:0/2969639044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad58075c00 con 0x7fad58160820 2024-03-04T07:49:40.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.289+0000 7fad4e000640 1 -- 172.21.15.113:0/2969639044 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad500c1070 con 0x7fad58160820 2024-03-04T07:49:40.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.293+0000 7fad4e000640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad38066f60 0x7fad38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:40.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.293+0000 7fad4e000640 1 -- 172.21.15.113:0/2969639044 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fad50123560 con 0x7fad58160820 2024-03-04T07:49:40.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.293+0000 7fad55e00640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad38066f60 0x7fad38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:40.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.293+0000 7fad55e00640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad38066f60 0x7fad38069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fad48000f70 tx=0x7fad48007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:40.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.297+0000 7fad4e000640 1 -- 172.21.15.113:0/2969639044 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fad500c9020 con 0x7fad58160820 2024-03-04T07:49:40.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.505+0000 7fad5d200640 1 -- 172.21.15.113:0/2969639044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad58149a10 con 0x7fad58160820 2024-03-04T07:49:40.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.505+0000 7fad4e000640 1 -- 172.21.15.113:0/2969639044 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5812 (secure 0 0 0) 0x7fad50018d70 con 0x7fad58160820 2024-03-04T07:49:40.505 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:40.505 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:30.698677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:49:40.505 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 64 2024-03-04T07:49:40.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 -- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad38066f60 msgr2=0x7fad38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:40.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad38066f60 0x7fad38069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fad48000f70 tx=0x7fad48007e30 comp rx=0 tx=0).stop 2024-03-04T07:49:40.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 -- 172.21.15.113:0/2969639044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 msgr2=0x7fad580c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:40.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 0x7fad580c6210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fad5001be20 tx=0x7fad5001bfb0 comp rx=0 tx=0).stop 2024-03-04T07:49:40.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 -- 172.21.15.113:0/2969639044 shutdown_connections 2024-03-04T07:49:40.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad38066f60 0x7fad38069420 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:40.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad58160820 0x7fad580c6210 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:40.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad580c85f0 0x7fad580c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:40.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 --2- 172.21.15.113:0/2969639044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad580b20b0 0x7fad58141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:40.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 -- 172.21.15.113:0/2969639044 >> 172.21.15.113:0/2969639044 conn(0x7fad5806e820 msgr2=0x7fad580c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:40.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 -- 172.21.15.113:0/2969639044 shutdown_connections 2024-03-04T07:49:40.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:40.509+0000 7fad5d200640 1 -- 172.21.15.113:0/2969639044 wait complete. 2024-03-04T07:49:40.526 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:49:42.528 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:42.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.673+0000 7f9f4a800640 1 Processor -- start 2024-03-04T07:49:42.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.673+0000 7f9f4a800640 1 -- start start 2024-03-04T07:49:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.673+0000 7f9f4a800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4414e4c0 0x7f9f44072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f4a800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9f4415b900 0x7f9f440731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f4a800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f440c27f0 con 0x7f9f440c22e0 2024-03-04T07:49:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f4a800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f4415ad60 con 0x7f9f4415b900 2024-03-04T07:49:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f4a800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440c2a90 con 0x7f9f4414e4c0 2024-03-04T07:49:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f3be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9f440c22e0 0x7f9f4415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36532/0 (socket says 172.21.15.113:36532) 2024-03-04T07:49:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f3be00640 1 -- 172.21.15.113:0/6060842 learned_addr learned my addr 172.21.15.113:0/6060842 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:49:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f43400640 1 --2- 172.21.15.113:0/6060842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4414e4c0 0x7f9f44072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f43e00640 1 --2- 172.21.15.113:0/6060842 >> v2:172.21.15.136:3300/0 conn(0x7f9f4415b900 0x7f9f440731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f42a00640 1 -- 172.21.15.113:0/6060842 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383936780 0 0) 0x7f9f440c27f0 con 0x7f9f440c22e0 2024-03-04T07:49:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f42a00640 1 -- 172.21.15.113:0/6060842 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f28003530 con 0x7f9f440c22e0 2024-03-04T07:49:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f43400640 1 -- 172.21.15.113:0/6060842 >> v2:172.21.15.136:3300/0 conn(0x7f9f4415b900 msgr2=0x7f9f440731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f43400640 1 --2- 172.21.15.113:0/6060842 >> v2:172.21.15.136:3300/0 conn(0x7f9f4415b900 0x7f9f440731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f43400640 1 -- 172.21.15.113:0/6060842 >> v1:172.21.15.113:6789/0 conn(0x7f9f440c22e0 legacy=0x7f9f4415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f43400640 1 -- 172.21.15.113:0/6060842 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44070c60 con 0x7f9f4414e4c0 2024-03-04T07:49:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.677+0000 7f9f43e00640 1 --2- 172.21.15.113:0/6060842 >> v2:172.21.15.136:3300/0 conn(0x7f9f4415b900 0x7f9f440731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:49:42.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f43400640 1 --2- 172.21.15.113:0/6060842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4414e4c0 0x7f9f44072c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f340097c0 tx=0x7f9f3401ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc538a494fecf7d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:42.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f42a00640 1 -- 172.21.15.113:0/6060842 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340b5070 con 0x7f9f4414e4c0 2024-03-04T07:49:42.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f42a00640 1 -- 172.21.15.113:0/6060842 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f340a8990 con 0x7f9f4414e4c0 2024-03-04T07:49:42.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f42a00640 1 -- 172.21.15.113:0/6060842 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340a8d40 con 0x7f9f4414e4c0 2024-03-04T07:49:42.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f4a800640 1 -- 172.21.15.113:0/6060842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4414e4c0 msgr2=0x7f9f44072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:42.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f4a800640 1 --2- 172.21.15.113:0/6060842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4414e4c0 0x7f9f44072c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9f340097c0 tx=0x7f9f3401ec90 comp rx=0 tx=0).stop 2024-03-04T07:49:42.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f4a800640 1 -- 172.21.15.113:0/6060842 shutdown_connections 2024-03-04T07:49:42.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f4a800640 1 --2- 172.21.15.113:0/6060842 >> v2:172.21.15.136:3300/0 conn(0x7f9f4415b900 0x7f9f440731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:42.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f4a800640 1 --2- 172.21.15.113:0/6060842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4414e4c0 0x7f9f44072c80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:42.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f4a800640 1 -- 172.21.15.113:0/6060842 >> 172.21.15.113:0/6060842 conn(0x7f9f4406e820 msgr2=0x7f9f4406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:42.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f4a800640 1 -- 172.21.15.113:0/6060842 shutdown_connections 2024-03-04T07:49:42.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f4a800640 1 -- 172.21.15.113:0/6060842 wait complete. 2024-03-04T07:49:42.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f4a800640 1 Processor -- start 2024-03-04T07:49:42.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.681+0000 7f9f4a800640 1 -- start start 2024-03-04T07:49:42.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f4a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f440c22e0 0x7f9f441f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:42.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f4a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f4414e4c0 0x7f9f44159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:42.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f4a800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4415b900 0x7f9f440c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:42.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f4a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44071500 con 0x7f9f440c22e0 2024-03-04T07:49:42.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f4a800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44071220 con 0x7f9f4414e4c0 2024-03-04T07:49:42.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f4a800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4414e8a0 con 0x7f9f4415b900 2024-03-04T07:49:42.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f43400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f440c22e0 0x7f9f441f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:42.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f43400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f440c22e0 0x7f9f441f04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39372/0 (socket says 172.21.15.113:39372) 2024-03-04T07:49:42.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f43400640 1 -- 172.21.15.113:0/1967374139 learned_addr learned my addr 172.21.15.113:0/1967374139 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:42.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f43e00640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4415b900 0x7f9f440c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:42.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f3be00640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f4414e4c0 0x7f9f44159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:42.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f43400640 1 -- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4415b900 msgr2=0x7f9f440c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:42.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f43400640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4415b900 0x7f9f440c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:42.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f43400640 1 -- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f4414e4c0 msgr2=0x7f9f44159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:42.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f43400640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f4414e4c0 0x7f9f44159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:42.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f43400640 1 -- 172.21.15.113:0/1967374139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f441f0a30 con 0x7f9f440c22e0 2024-03-04T07:49:42.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f3be00640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f4414e4c0 0x7f9f44159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:42.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f43400640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f440c22e0 0x7f9f441f04f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9f3401c8c0 tx=0x7f9f3401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:42.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f3b400640 1 -- 172.21.15.113:0/1967374139 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340be070 con 0x7f9f440c22e0 2024-03-04T07:49:42.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f3b400640 1 -- 172.21.15.113:0/1967374139 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f340042f0 con 0x7f9f440c22e0 2024-03-04T07:49:42.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f3b400640 1 -- 172.21.15.113:0/1967374139 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340b5040 con 0x7f9f440c22e0 2024-03-04T07:49:42.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f4a800640 1 -- 172.21.15.113:0/1967374139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f440cb6a0 con 0x7f9f440c22e0 2024-03-04T07:49:42.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f4a800640 1 -- 172.21.15.113:0/1967374139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f44149cb0 con 0x7f9f440c22e0 2024-03-04T07:49:42.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.685+0000 7f9f4a800640 1 -- 172.21.15.113:0/1967374139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f44075c00 con 0x7f9f440c22e0 2024-03-04T07:49:42.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.689+0000 7f9f3b400640 1 -- 172.21.15.113:0/1967374139 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f340a8910 con 0x7f9f440c22e0 2024-03-04T07:49:42.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.689+0000 7f9f3b400640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f20066f60 0x7f9f20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:42.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.689+0000 7f9f3b400640 1 -- 172.21.15.113:0/1967374139 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f9f34121d20 con 0x7f9f440c22e0 2024-03-04T07:49:42.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.689+0000 7f9f3be00640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f20066f60 0x7f9f20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:42.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.689+0000 7f9f3be00640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f20066f60 0x7f9f20069420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9f441588a0 tx=0x7f9f2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:42.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.697+0000 7f9f3b400640 1 -- 172.21.15.113:0/1967374139 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9f340187f0 con 0x7f9f440c22e0 2024-03-04T07:49:42.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.921+0000 7f9f4a800640 1 -- 172.21.15.113:0/1967374139 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f44155190 con 0x7f9f440c22e0 2024-03-04T07:49:42.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.921+0000 7f9f3b400640 1 -- 172.21.15.113:0/1967374139 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f9f340f22b0 con 0x7f9f440c22e0 2024-03-04T07:49:42.922 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:42.922 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:30.698677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:49:42.922 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 64 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 -- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f20066f60 msgr2=0x7f9f20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f20066f60 0x7f9f20069420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9f441588a0 tx=0x7f9f2c009040 comp rx=0 tx=0).stop 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 -- 172.21.15.113:0/1967374139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f440c22e0 msgr2=0x7f9f441f04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f440c22e0 0x7f9f441f04f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9f3401c8c0 tx=0x7f9f3401ba30 comp rx=0 tx=0).stop 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 -- 172.21.15.113:0/1967374139 shutdown_connections 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f4415b900 0x7f9f440c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f4414e4c0 0x7f9f44159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f20066f60 0x7f9f20069420 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 --2- 172.21.15.113:0/1967374139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f440c22e0 0x7f9f441f04f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 -- 172.21.15.113:0/1967374139 >> 172.21.15.113:0/1967374139 conn(0x7f9f4406e820 msgr2=0x7f9f44072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:42.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 -- 172.21.15.113:0/1967374139 shutdown_connections 2024-03-04T07:49:42.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:42.925+0000 7f9f4a800640 1 -- 172.21.15.113:0/1967374139 wait complete. 2024-03-04T07:49:42.948 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:49:44.949 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:45.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f0087000640 1 Processor -- start 2024-03-04T07:49:45.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f0087000640 1 -- start start 2024-03-04T07:49:45.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f0087000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00800ce2d0 0x7f008015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:45.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f0087000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00800ce6b0 0x7f008006f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:45.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f0087000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f008006c210 con 0x7f008006de80 2024-03-04T07:49:45.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f0087000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008015bbf0 con 0x7f00800ce2d0 2024-03-04T07:49:45.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f0087000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008006eb80 con 0x7f00800ce6b0 2024-03-04T07:49:45.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f007fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00800ce2d0 0x7f008015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:45.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f0085000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f008006de80 0x7f0080154b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36544/0 (socket says 172.21.15.113:36544) 2024-03-04T07:49:45.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f0085000640 1 -- 172.21.15.113:0/3596654195 learned_addr learned my addr 172.21.15.113:0/3596654195 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:49:45.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.093+0000 7f0077e00640 1 --2- 172.21.15.113:0/3596654195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00800ce6b0 0x7f008006f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:45.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.097+0000 7f007fe00640 1 -- 172.21.15.113:0/3596654195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00800ce6b0 msgr2=0x7f008006f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:45.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.097+0000 7f007fe00640 1 --2- 172.21.15.113:0/3596654195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00800ce6b0 0x7f008006f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:45.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.097+0000 7f007fe00640 1 -- 172.21.15.113:0/3596654195 >> v1:172.21.15.113:6789/0 conn(0x7f008006de80 legacy=0x7f0080154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:45.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.097+0000 7f007fe00640 1 -- 172.21.15.113:0/3596654195 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008006c4f0 con 0x7f00800ce2d0 2024-03-04T07:49:45.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.097+0000 7f007f400640 1 -- 172.21.15.113:0/3596654195 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846638594 0 0) 0x7f008006c210 con 0x7f008006de80 2024-03-04T07:49:45.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f007fe00640 1 --2- 172.21.15.113:0/3596654195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 0x7f008015b6b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f00700097c0 tx=0x7f007001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9724a2716fe2faf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:45.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f007f400640 1 -- 172.21.15.113:0/3596654195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00700b5070 con 0x7f00800ce2d0 2024-03-04T07:49:45.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f007f400640 1 -- 172.21.15.113:0/3596654195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00700a8990 con 0x7f00800ce2d0 2024-03-04T07:49:45.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f007f400640 1 -- 172.21.15.113:0/3596654195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00700a8d60 con 0x7f00800ce2d0 2024-03-04T07:49:45.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f0087000640 1 -- 172.21.15.113:0/3596654195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 msgr2=0x7f008015b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:45.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f0087000640 1 --2- 172.21.15.113:0/3596654195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 0x7f008015b6b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f00700097c0 tx=0x7f007001ec90 comp rx=0 tx=0).stop 2024-03-04T07:49:45.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f0087000640 1 -- 172.21.15.113:0/3596654195 shutdown_connections 2024-03-04T07:49:45.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f0087000640 1 --2- 172.21.15.113:0/3596654195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00800ce6b0 0x7f008006f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:45.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f0087000640 1 --2- 172.21.15.113:0/3596654195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 0x7f008015b6b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:45.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f0087000640 1 -- 172.21.15.113:0/3596654195 >> 172.21.15.113:0/3596654195 conn(0x7f00800709a0 msgr2=0x7f0080070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:45.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f0087000640 1 -- 172.21.15.113:0/3596654195 shutdown_connections 2024-03-04T07:49:45.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.101+0000 7f0087000640 1 -- 172.21.15.113:0/3596654195 wait complete. 2024-03-04T07:49:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0087000640 1 Processor -- start 2024-03-04T07:49:45.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0087000640 1 -- start start 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0087000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f008006de80 0x7f00800c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0087000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 0x7f0080150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0087000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00800ce6b0 0x7f008014d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0087000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008006c7d0 con 0x7f00800ce2d0 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0087000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008006e810 con 0x7f00800ce6b0 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0087000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0080155220 con 0x7f008006de80 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0077e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 0x7f0080150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0077e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 0x7f0080150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39390/0 (socket says 172.21.15.113:39390) 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0077e00640 1 -- 172.21.15.113:0/1653195196 learned_addr learned my addr 172.21.15.113:0/1653195196 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f007fe00640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f008006de80 0x7f00800c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0085000640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00800ce6b0 0x7f008014d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:45.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0077e00640 1 -- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f008006de80 msgr2=0x7f00800c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0077e00640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f008006de80 0x7f00800c5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0077e00640 1 -- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00800ce6b0 msgr2=0x7f008014d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0077e00640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00800ce6b0 0x7f008014d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0077e00640 1 -- 172.21.15.113:0/1653195196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0080153340 con 0x7f00800ce2d0 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0077e00640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 0x7f0080150b40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f006800e900 tx=0x7f006800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f007cc00640 1 -- 172.21.15.113:0/1653195196 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068017070 con 0x7f00800ce2d0 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f007cc00640 1 -- 172.21.15.113:0/1653195196 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006800ca00 con 0x7f00800ce2d0 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f007cc00640 1 -- 172.21.15.113:0/1653195196 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006800cdf0 con 0x7f00800ce2d0 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0087000640 1 -- 172.21.15.113:0/1653195196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00801f3ef0 con 0x7f00800ce2d0 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.105+0000 7f0087000640 1 -- 172.21.15.113:0/1653195196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00801f44b0 con 0x7f00800ce2d0 2024-03-04T07:49:45.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.109+0000 7f0087000640 1 -- 172.21.15.113:0/1653195196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0080075c00 con 0x7f00800ce2d0 2024-03-04T07:49:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.109+0000 7f007cc00640 1 -- 172.21.15.113:0/1653195196 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f006801a070 con 0x7f00800ce2d0 2024-03-04T07:49:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.109+0000 7f007cc00640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f005c066f60 0x7f005c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.109+0000 7f007cc00640 1 -- 172.21.15.113:0/1653195196 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f00680856a0 con 0x7f00800ce2d0 2024-03-04T07:49:45.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.109+0000 7f007fe00640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f005c066f60 0x7f005c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:45.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.109+0000 7f007fe00640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f005c066f60 0x7f005c069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f00700b2000 tx=0x7f0070008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:45.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.113+0000 7f007cc00640 1 -- 172.21.15.113:0/1653195196 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0068055ce0 con 0x7f00800ce2d0 2024-03-04T07:49:45.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 -- 172.21.15.113:0/1653195196 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f008006cb40 con 0x7f00800ce2d0 2024-03-04T07:49:45.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f007cc00640 1 -- 172.21.15.113:0/1653195196 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5814 (secure 0 0 0) 0x7f0068059d80 con 0x7f00800ce2d0 2024-03-04T07:49:45.330 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:45.331 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:43.154898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:resolve","state_seq":22,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5631,5529]},"id":1}]} 2024-03-04T07:49:45.331 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 65 2024-03-04T07:49:45.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 -- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f005c066f60 msgr2=0x7f005c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f005c066f60 0x7f005c069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f00700b2000 tx=0x7f0070008040 comp rx=0 tx=0).stop 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 -- 172.21.15.113:0/1653195196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 msgr2=0x7f0080150b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 0x7f0080150b40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f006800e900 tx=0x7f006800edc0 comp rx=0 tx=0).stop 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 -- 172.21.15.113:0/1653195196 shutdown_connections 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00800ce6b0 0x7f008014d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f005c066f60 0x7f005c069420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00800ce2d0 0x7f0080150b40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 --2- 172.21.15.113:0/1653195196 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f008006de80 0x7f00800c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 -- 172.21.15.113:0/1653195196 >> 172.21.15.113:0/1653195196 conn(0x7f00800709a0 msgr2=0x7f00800c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 -- 172.21.15.113:0/1653195196 shutdown_connections 2024-03-04T07:49:45.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:45.333+0000 7f0087000640 1 -- 172.21.15.113:0/1653195196 wait complete. 2024-03-04T07:49:45.352 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 65, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 65, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5529": { "addr": "172.21.15.113:6836/2299268553", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2299268553, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2299268553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5529, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 43 }, "gid_5631": { "addr": "172.21.15.113:6833/810789479", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 810789479, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 810789479, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5631, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 17 }, "gid_5940": { "addr": "172.21.15.113:6837/344022786", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 344022786, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 344022786, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5940, "incarnation": 64, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:resolve", "state_seq": 22 } }, "last_failure": 0, "last_failure_osd_epoch": 57, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:49:43.154898+0000", "qdb_cluster": [ 5631, 5529 ], "qdb_leader": 5529, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5940, "mds_1": 5529, "mds_2": 5631 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:49:47.353 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:47.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.493+0000 7fe3d0600640 1 Processor -- start 2024-03-04T07:49:47.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.493+0000 7fe3d0600640 1 -- start start 2024-03-04T07:49:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3d0600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe3c81527f0 0x7fe3c8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3d0600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c81591f0 con 0x7fe3c80c8200 2024-03-04T07:49:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3d0600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c81594d0 con 0x7fe3c8152bd0 2024-03-04T07:49:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3d0600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c8071df0 con 0x7fe3c81527f0 2024-03-04T07:49:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cd200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe3c80c8200 0x7fe3c8072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45554/0 (socket says 172.21.15.113:45554) 2024-03-04T07:49:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cd200640 1 -- 172.21.15.113:0/3729973653 learned_addr learned my addr 172.21.15.113:0/3729973653 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:49:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cdc00640 1 --2- 172.21.15.113:0/3729973653 >> v2:172.21.15.136:3300/0 conn(0x7fe3c81527f0 0x7fe3c8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977751186 0 0) 0x7fe3c81591f0 con 0x7fe3c80c8200 2024-03-04T07:49:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3b0003530 con 0x7fe3c80c8200 2024-03-04T07:49:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554641432 0 0) 0x7fe3c81594d0 con 0x7fe3c8152bd0 2024-03-04T07:49:47.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3c81591f0 con 0x7fe3c8152bd0 2024-03-04T07:49:47.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3352002908 0 0) 0x7fe3b0003530 con 0x7fe3c80c8200 2024-03-04T07:49:47.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 >> v2:172.21.15.136:3300/0 conn(0x7fe3c81527f0 msgr2=0x7fe3c8070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:47.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 --2- 172.21.15.113:0/3729973653 >> v2:172.21.15.136:3300/0 conn(0x7fe3c81527f0 0x7fe3c8070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 >> v1:172.21.15.136:6789/0 conn(0x7fe3c8152bd0 legacy=0x7fe3c81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:47.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c80717a0 con 0x7fe3c80c8200 2024-03-04T07:49:47.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fe3b4001680 con 0x7fe3c80c8200 2024-03-04T07:49:47.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 >> v1:172.21.15.113:6789/0 conn(0x7fe3c80c8200 legacy=0x7fe3c8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:47.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3b0003ba0 0x7fe3b0003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:47.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:47.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b00051d0 0x7fe3b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:47.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8071df0 con 0x7fe3b0004810 2024-03-04T07:49:47.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b000c0a0 con 0x7fe3b00051d0 2024-03-04T07:49:47.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b000c2a0 con 0x7fe3b0003ba0 2024-03-04T07:49:47.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cd200640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:47.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cdc00640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3b0003ba0 0x7fe3b0003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:47.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3ce600640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b00051d0 0x7fe3b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:47.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cd200640 1 -- 172.21.15.113:0/3729973653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3b0003ba0 msgr2=0x7fe3b0003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:47.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cd200640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3b0003ba0 0x7fe3b0003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cd200640 1 -- 172.21.15.113:0/3729973653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b00051d0 msgr2=0x7fe3b000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:47.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cd200640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b00051d0 0x7fe3b000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cd200640 1 -- 172.21.15.113:0/3729973653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b0003530 con 0x7fe3b0004810 2024-03-04T07:49:47.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3cdc00640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3b0003ba0 0x7fe3b0003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:47.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3ce600640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3b00051d0 0x7fe3b000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:47.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.497+0000 7fe3ce600640 1 -- 172.21.15.113:0/3729973653 reap_dead start 2024-03-04T07:49:47.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3cd200640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3b0004c90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4007fa0 tx=0x7fe3b401cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a9f99795daced2d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:47.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4007bf0 con 0x7fe3b0004810 2024-03-04T07:49:47.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b40a9020 con 0x7fe3b0004810 2024-03-04T07:49:47.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3c7e00640 1 -- 172.21.15.113:0/3729973653 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4005b00 con 0x7fe3b0004810 2024-03-04T07:49:47.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3d0600640 1 -- 172.21.15.113:0/3729973653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 msgr2=0x7fe3b0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:47.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3d0600640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3b0004c90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4007fa0 tx=0x7fe3b401cae0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3d0600640 1 -- 172.21.15.113:0/3729973653 shutdown_connections 2024-03-04T07:49:47.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3d0600640 1 --2- 172.21.15.113:0/3729973653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3b0004c90 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3d0600640 1 -- 172.21.15.113:0/3729973653 >> 172.21.15.113:0/3729973653 conn(0x7fe3c806e820 msgr2=0x7fe3c806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:47.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3d0600640 1 -- 172.21.15.113:0/3729973653 shutdown_connections 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3d0600640 1 -- 172.21.15.113:0/3729973653 wait complete. 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3d0600640 1 Processor -- start 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.501+0000 7fe3d0600640 1 -- start start 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3d0600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3c81e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3d0600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c8155e30 0x7fe3c81ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3d0600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3c81ec630 0x7fe3c81ef660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3d0600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8070d40 con 0x7fe3b0004810 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3d0600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c81597b0 con 0x7fe3c8155e30 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3d0600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8070ec0 con 0x7fe3c81ec630 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3cdc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3c81e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3cdc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3c81e1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36282/0 (socket says 172.21.15.113:36282) 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3cdc00640 1 -- 172.21.15.113:0/3264111225 learned_addr learned my addr 172.21.15.113:0/3264111225 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3ce600640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3c81ec630 0x7fe3c81ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:47.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3cd200640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c8155e30 0x7fe3c81ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3cdc00640 1 -- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3c81ec630 msgr2=0x7fe3c81ef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3cdc00640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3c81ec630 0x7fe3c81ef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3cdc00640 1 -- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c8155e30 msgr2=0x7fe3c81ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3cdc00640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c8155e30 0x7fe3c81ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3cdc00640 1 -- 172.21.15.113:0/3264111225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8157eb0 con 0x7fe3b0004810 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3cdc00640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3c81e1c80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc011540 tx=0x7fe3bc011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3c5600640 1 -- 172.21.15.113:0/3264111225 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc004220 con 0x7fe3b0004810 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3c5600640 1 -- 172.21.15.113:0/3264111225 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc0043c0 con 0x7fe3b0004810 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3d0600640 1 -- 172.21.15.113:0/3264111225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c8153d80 con 0x7fe3b0004810 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3c5600640 1 -- 172.21.15.113:0/3264111225 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc015890 con 0x7fe3b0004810 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3d0600640 1 -- 172.21.15.113:0/3264111225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c81532a0 con 0x7fe3b0004810 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.505+0000 7fe3d0600640 1 -- 172.21.15.113:0/3264111225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c8075c00 con 0x7fe3b0004810 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.509+0000 7fe3c5600640 1 -- 172.21.15.113:0/3264111225 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3bc015a30 con 0x7fe3b0004810 2024-03-04T07:49:47.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.509+0000 7fe3c5600640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe3a8066f60 0x7fe3a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:47.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.509+0000 7fe3c5600640 1 -- 172.21.15.113:0/3264111225 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fe3bc085860 con 0x7fe3b0004810 2024-03-04T07:49:47.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.509+0000 7fe3cd200640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe3a8066f60 0x7fe3a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:47.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.509+0000 7fe3cd200640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe3a8066f60 0x7fe3a8069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40aa000 tx=0x7fe3c8071d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:47.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.517+0000 7fe3c5600640 1 -- 172.21.15.113:0/3264111225 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe3bc019030 con 0x7fe3b0004810 2024-03-04T07:49:47.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 -- 172.21.15.113:0/3264111225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3c80c79d0 con 0x7fe3b0004810 2024-03-04T07:49:47.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3c5600640 1 -- 172.21.15.113:0/3264111225 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5814 (secure 0 0 0) 0x7fe3bc0564f0 con 0x7fe3b0004810 2024-03-04T07:49:47.747 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:47.748 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:43.154898+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:resolve","state_seq":22,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5631,5529]},"id":1}]} 2024-03-04T07:49:47.748 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 65 2024-03-04T07:49:47.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 -- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe3a8066f60 msgr2=0x7fe3a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:47.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe3a8066f60 0x7fe3a8069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40aa000 tx=0x7fe3c8071d60 comp rx=0 tx=0).stop 2024-03-04T07:49:47.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 -- 172.21.15.113:0/3264111225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 msgr2=0x7fe3c81e1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:47.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3c81e1c80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc011540 tx=0x7fe3bc011a00 comp rx=0 tx=0).stop 2024-03-04T07:49:47.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 -- 172.21.15.113:0/3264111225 shutdown_connections 2024-03-04T07:49:47.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3c81ec630 0x7fe3c81ef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c8155e30 0x7fe3c81ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe3a8066f60 0x7fe3a8069420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 --2- 172.21.15.113:0/3264111225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3b0004810 0x7fe3c81e1c80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:47.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 -- 172.21.15.113:0/3264111225 >> 172.21.15.113:0/3264111225 conn(0x7fe3c806e820 msgr2=0x7fe3c80c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:47.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 -- 172.21.15.113:0/3264111225 shutdown_connections 2024-03-04T07:49:47.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:47.749+0000 7fe3d0600640 1 -- 172.21.15.113:0/3264111225 wait complete. 2024-03-04T07:49:47.768 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:49:49.770 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:49.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.909+0000 7fe285200640 1 Processor -- start 2024-03-04T07:49:49.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.909+0000 7fe285200640 1 -- start start 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe285200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe28006d420 0x7fe2800730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe285200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe28015bce0 0x7fe28015c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe285200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28006f1b0 con 0x7fe280152a90 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe285200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe280152e70 con 0x7fe28006d420 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe285200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28006c320 con 0x7fe28015bce0 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe27ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe28006d420 0x7fe2800730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe27ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe28006d420 0x7fe2800730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36294/0 (socket says 172.21.15.113:36294) 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe27ea00640 1 -- 172.21.15.113:0/135400433 learned_addr learned my addr 172.21.15.113:0/135400433 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe27fe00640 1 --2- 172.21.15.113:0/135400433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe28015bce0 0x7fe28015c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe27ea00640 1 -- 172.21.15.113:0/135400433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe28015bce0 msgr2=0x7fe28015c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:49.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe27ea00640 1 --2- 172.21.15.113:0/135400433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe28015bce0 0x7fe28015c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:49.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe27ea00640 1 -- 172.21.15.113:0/135400433 >> v1:172.21.15.136:6789/0 conn(0x7fe280152a90 legacy=0x7fe28006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:49.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.913+0000 7fe27ea00640 1 -- 172.21.15.113:0/135400433 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe280071580 con 0x7fe28006d420 2024-03-04T07:49:49.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe27ea00640 1 --2- 172.21.15.113:0/135400433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 0x7fe2800730d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009870 tx=0x7fe26c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d115e8c2a6954b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:49.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe27e000640 1 -- 172.21.15.113:0/135400433 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0b2070 con 0x7fe28006d420 2024-03-04T07:49:49.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe27e000640 1 -- 172.21.15.113:0/135400433 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c0a9070 con 0x7fe28006d420 2024-03-04T07:49:49.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe27e000640 1 -- 172.21.15.113:0/135400433 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0ad880 con 0x7fe28006d420 2024-03-04T07:49:49.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe285200640 1 -- 172.21.15.113:0/135400433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 msgr2=0x7fe2800730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:49.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe285200640 1 --2- 172.21.15.113:0/135400433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 0x7fe2800730d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009870 tx=0x7fe26c01c560 comp rx=0 tx=0).stop 2024-03-04T07:49:49.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe285200640 1 -- 172.21.15.113:0/135400433 shutdown_connections 2024-03-04T07:49:49.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe285200640 1 --2- 172.21.15.113:0/135400433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe28015bce0 0x7fe28015c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:49.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe285200640 1 --2- 172.21.15.113:0/135400433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 0x7fe2800730d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:49.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe285200640 1 -- 172.21.15.113:0/135400433 >> 172.21.15.113:0/135400433 conn(0x7fe28006e2a0 msgr2=0x7fe28006e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:49.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe285200640 1 -- 172.21.15.113:0/135400433 shutdown_connections 2024-03-04T07:49:49.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe285200640 1 -- 172.21.15.113:0/135400433 wait complete. 2024-03-04T07:49:49.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe285200640 1 Processor -- start 2024-03-04T07:49:49.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.917+0000 7fe285200640 1 -- start start 2024-03-04T07:49:49.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe285200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 0x7fe280143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe285200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe280152a90 0x7fe2800cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe285200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe28015bce0 0x7fe28013b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe285200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28015ca80 con 0x7fe28006d420 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe285200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe280071330 con 0x7fe28015bce0 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe285200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe280165aa0 con 0x7fe280152a90 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 0x7fe280143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 0x7fe280143f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36302/0 (socket says 172.21.15.113:36302) 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27f400640 1 -- 172.21.15.113:0/528836733 learned_addr learned my addr 172.21.15.113:0/528836733 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27ea00640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe280152a90 0x7fe2800cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27fe00640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe28015bce0 0x7fe28013b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27f400640 1 -- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe280152a90 msgr2=0x7fe2800cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:49.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27f400640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe280152a90 0x7fe2800cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:49.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27f400640 1 -- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe28015bce0 msgr2=0x7fe28013b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:49.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27f400640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe28015bce0 0x7fe28013b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:49.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27f400640 1 -- 172.21.15.113:0/528836733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28013cdb0 con 0x7fe28006d420 2024-03-04T07:49:49.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27fe00640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe28015bce0 0x7fe28013b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:49.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe27f400640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 0x7fe280143f70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe27000c370 tx=0x7fe27000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:49.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe277400640 1 -- 172.21.15.113:0/528836733 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2700052d0 con 0x7fe28006d420 2024-03-04T07:49:49.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe277400640 1 -- 172.21.15.113:0/528836733 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe270005470 con 0x7fe28006d420 2024-03-04T07:49:49.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe277400640 1 -- 172.21.15.113:0/528836733 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270007260 con 0x7fe28006d420 2024-03-04T07:49:49.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe285200640 1 -- 172.21.15.113:0/528836733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28013d050 con 0x7fe28006d420 2024-03-04T07:49:49.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.921+0000 7fe285200640 1 -- 172.21.15.113:0/528836733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2801f4c50 con 0x7fe28006d420 2024-03-04T07:49:49.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.925+0000 7fe285200640 1 -- 172.21.15.113:0/528836733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe280075a80 con 0x7fe28006d420 2024-03-04T07:49:49.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.925+0000 7fe277400640 1 -- 172.21.15.113:0/528836733 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe270007400 con 0x7fe28006d420 2024-03-04T07:49:49.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.925+0000 7fe277400640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe260066f60 0x7fe260069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:49.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.925+0000 7fe277400640 1 -- 172.21.15.113:0/528836733 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fe270084820 con 0x7fe28006d420 2024-03-04T07:49:49.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.925+0000 7fe27ea00640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe260066f60 0x7fe260069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:49.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.925+0000 7fe27ea00640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe260066f60 0x7fe260069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0af000 tx=0x7fe26c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:49.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:49.929+0000 7fe277400640 1 -- 172.21.15.113:0/528836733 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe270054dd0 con 0x7fe28006d420 2024-03-04T07:49:50.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 -- 172.21.15.113:0/528836733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2800c5f30 con 0x7fe28006d420 2024-03-04T07:49:50.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe277400640 1 -- 172.21.15.113:0/528836733 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5816 (secure 0 0 0) 0x7fe270058e70 con 0x7fe28006d420 2024-03-04T07:49:50.151 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:50.151 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:49.141946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:49:50.151 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 66 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 -- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe260066f60 msgr2=0x7fe260069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe260066f60 0x7fe260069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0af000 tx=0x7fe26c008040 comp rx=0 tx=0).stop 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 -- 172.21.15.113:0/528836733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 msgr2=0x7fe280143f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 0x7fe280143f70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe27000c370 tx=0x7fe27000c830 comp rx=0 tx=0).stop 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 -- 172.21.15.113:0/528836733 shutdown_connections 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe28015bce0 0x7fe28013b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe280152a90 0x7fe2800cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe260066f60 0x7fe260069420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 --2- 172.21.15.113:0/528836733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe28006d420 0x7fe280143f70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 -- 172.21.15.113:0/528836733 >> 172.21.15.113:0/528836733 conn(0x7fe28006e2a0 msgr2=0x7fe280146010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 -- 172.21.15.113:0/528836733 shutdown_connections 2024-03-04T07:49:50.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:50.153+0000 7fe285200640 1 -- 172.21.15.113:0/528836733 wait complete. 2024-03-04T07:49:50.181 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:49:52.182 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b40200640 1 Processor -- start 2024-03-04T07:49:52.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b40200640 1 -- start start 2024-03-04T07:49:52.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b40200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b381527f0 0x7f9b38070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:52.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b40200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b381591f0 con 0x7f9b380c8200 2024-03-04T07:49:52.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b40200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b381594d0 con 0x7f9b38152bd0 2024-03-04T07:49:52.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b40200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b38071df0 con 0x7f9b381527f0 2024-03-04T07:49:52.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b3d800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b381527f0 0x7f9b38070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:52.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b3ce00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9b380c8200 0x7f9b38072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45558/0 (socket says 172.21.15.113:45558) 2024-03-04T07:49:52.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b3ce00640 1 -- 172.21.15.113:0/1226943442 learned_addr learned my addr 172.21.15.113:0/1226943442 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:49:52.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b3d800640 1 -- 172.21.15.113:0/1226943442 >> v1:172.21.15.136:6789/0 conn(0x7f9b38152bd0 legacy=0x7f9b381587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:52.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b3d800640 1 -- 172.21.15.113:0/1226943442 >> v1:172.21.15.113:6789/0 conn(0x7f9b380c8200 legacy=0x7f9b38072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:52.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b3d800640 1 -- 172.21.15.113:0/1226943442 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b380717a0 con 0x7f9b381527f0 2024-03-04T07:49:52.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.325+0000 7f9b37e00640 1 -- 172.21.15.113:0/1226943442 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2095525256 0 0) 0x7f9b381591f0 con 0x7f9b380c8200 2024-03-04T07:49:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b3d800640 1 --2- 172.21.15.113:0/1226943442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b381527f0 0x7f9b38070d40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0097c0 tx=0x7f9b2c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574caa29773dcca4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b37e00640 1 -- 172.21.15.113:0/1226943442 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0b1070 con 0x7f9b381527f0 2024-03-04T07:49:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b37e00640 1 -- 172.21.15.113:0/1226943442 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c0ac410 con 0x7f9b381527f0 2024-03-04T07:49:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b37e00640 1 -- 172.21.15.113:0/1226943442 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0ac7e0 con 0x7f9b381527f0 2024-03-04T07:49:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b40200640 1 -- 172.21.15.113:0/1226943442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b381527f0 msgr2=0x7f9b38070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b40200640 1 --2- 172.21.15.113:0/1226943442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b381527f0 0x7f9b38070d40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0097c0 tx=0x7f9b2c01c4a0 comp rx=0 tx=0).stop 2024-03-04T07:49:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b40200640 1 -- 172.21.15.113:0/1226943442 shutdown_connections 2024-03-04T07:49:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b40200640 1 --2- 172.21.15.113:0/1226943442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b381527f0 0x7f9b38070d40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b40200640 1 -- 172.21.15.113:0/1226943442 >> 172.21.15.113:0/1226943442 conn(0x7f9b3806e820 msgr2=0x7f9b3806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:52.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b40200640 1 -- 172.21.15.113:0/1226943442 shutdown_connections 2024-03-04T07:49:52.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.329+0000 7f9b40200640 1 -- 172.21.15.113:0/1226943442 wait complete. 2024-03-04T07:49:52.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 Processor -- start 2024-03-04T07:49:52.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 -- start start 2024-03-04T07:49:52.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b380c8200 0x7f9b380c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:52.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b381527f0 0x7f9b38149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:52.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b38152bd0 0x7f9b3814b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b38166380 con 0x7f9b38152bd0 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b381597b0 con 0x7f9b380c8200 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b38166500 con 0x7f9b381527f0 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b38152bd0 0x7f9b3814b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b380c8200 0x7f9b380c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b38152bd0 0x7f9b3814b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36314/0 (socket says 172.21.15.113:36314) 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3e200640 1 -- 172.21.15.113:0/3308578873 learned_addr learned my addr 172.21.15.113:0/3308578873 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3ce00640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b381527f0 0x7f9b38149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3e200640 1 -- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b381527f0 msgr2=0x7f9b38149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3e200640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b381527f0 0x7f9b38149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3e200640 1 -- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b380c8200 msgr2=0x7f9b380c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3e200640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b380c8200 0x7f9b380c1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3e200640 1 -- 172.21.15.113:0/3308578873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b38155db0 con 0x7f9b38152bd0 2024-03-04T07:49:52.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3ce00640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b381527f0 0x7f9b38149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:49:52.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b3e200640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b38152bd0 0x7f9b3814b210 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000c730 tx=0x7f9b3000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:52.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b35600640 1 -- 172.21.15.113:0/3308578873 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30003b80 con 0x7f9b38152bd0 2024-03-04T07:49:52.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b35600640 1 -- 172.21.15.113:0/3308578873 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b30003d20 con 0x7f9b38152bd0 2024-03-04T07:49:52.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b35600640 1 -- 172.21.15.113:0/3308578873 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3000aa30 con 0x7f9b38152bd0 2024-03-04T07:49:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 -- 172.21.15.113:0/3308578873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b380c2310 con 0x7f9b38152bd0 2024-03-04T07:49:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 -- 172.21.15.113:0/3308578873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b381f71e0 con 0x7f9b38152bd0 2024-03-04T07:49:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.333+0000 7f9b40200640 1 -- 172.21.15.113:0/3308578873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b38075c00 con 0x7f9b38152bd0 2024-03-04T07:49:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.337+0000 7f9b35600640 1 -- 172.21.15.113:0/3308578873 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b3000abd0 con 0x7f9b38152bd0 2024-03-04T07:49:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.337+0000 7f9b35600640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b18066f60 0x7f9b18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:52.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.337+0000 7f9b35600640 1 -- 172.21.15.113:0/3308578873 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f9b30085280 con 0x7f9b38152bd0 2024-03-04T07:49:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.337+0000 7f9b3d800640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b18066f60 0x7f9b18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.337+0000 7f9b3d800640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b18066f60 0x7f9b18069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0ae000 tx=0x7f9b2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:52.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.345+0000 7f9b35600640 1 -- 172.21.15.113:0/3308578873 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9b38155db0 con 0x7f9b38152bd0 2024-03-04T07:49:52.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.613+0000 7f9b40200640 1 -- 172.21.15.113:0/3308578873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b380c79d0 con 0x7f9b38152bd0 2024-03-04T07:49:52.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.613+0000 7f9b35600640 1 -- 172.21.15.113:0/3308578873 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f9b30055e70 con 0x7f9b38152bd0 2024-03-04T07:49:52.614 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:52.614 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:49.141946+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:49:52.614 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 66 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 -- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b18066f60 msgr2=0x7f9b18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b18066f60 0x7f9b18069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0ae000 tx=0x7f9b2c008040 comp rx=0 tx=0).stop 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 -- 172.21.15.113:0/3308578873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b38152bd0 msgr2=0x7f9b3814b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b38152bd0 0x7f9b3814b210 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000c730 tx=0x7f9b3000cbf0 comp rx=0 tx=0).stop 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 -- 172.21.15.113:0/3308578873 shutdown_connections 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b18066f60 0x7f9b18069420 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b38152bd0 0x7f9b3814b210 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b381527f0 0x7f9b38149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 --2- 172.21.15.113:0/3308578873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b380c8200 0x7f9b380c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 -- 172.21.15.113:0/3308578873 >> 172.21.15.113:0/3308578873 conn(0x7f9b3806e820 msgr2=0x7f9b3806ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 -- 172.21.15.113:0/3308578873 shutdown_connections 2024-03-04T07:49:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:52.617+0000 7f9b40200640 1 -- 172.21.15.113:0/3308578873 wait complete. 2024-03-04T07:49:52.640 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:49:54.641 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:54.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.781+0000 7fa0a3e00640 1 Processor -- start 2024-03-04T07:49:54.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa0a3e00640 1 -- start start 2024-03-04T07:49:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa0a3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a4152a90 0x7fa0a40730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa0a3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0a406cd10 0x7fa0a406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa0a3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a406d690 con 0x7fa0a415bce0 2024-03-04T07:49:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa0a3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a4152e70 con 0x7fa0a406cd10 2024-03-04T07:49:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa0a3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a406c320 con 0x7fa0a4152a90 2024-03-04T07:49:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa09a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0a406cd10 0x7fa0a406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa0a3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa0a415bce0 0x7fa0a415c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45568/0 (socket says 172.21.15.113:45568) 2024-03-04T07:49:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa0a3400640 1 -- 172.21.15.113:0/2037133595 learned_addr learned my addr 172.21.15.113:0/2037133595 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:49:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa09a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0a406cd10 0x7fa0a406d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36324/0 (socket says 172.21.15.113:36324) 2024-03-04T07:49:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa0a2a00640 1 --2- 172.21.15.113:0/2037133595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a4152a90 0x7fa0a40730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa09a000640 1 -- 172.21.15.113:0/2037133595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a4152a90 msgr2=0x7fa0a40730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa09a000640 1 --2- 172.21.15.113:0/2037133595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a4152a90 0x7fa0a40730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa09a000640 1 -- 172.21.15.113:0/2037133595 >> v1:172.21.15.113:6789/0 conn(0x7fa0a415bce0 legacy=0x7fa0a415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:54.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.785+0000 7fa09a000640 1 -- 172.21.15.113:0/2037133595 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a406f1b0 con 0x7fa0a406cd10 2024-03-04T07:49:54.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa09a000640 1 --2- 172.21.15.113:0/2037133595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a406cd10 0x7fa0a406d150 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0097c0 tx=0x7fa08c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b357216aed4c2f14 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:54.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a2000640 1 -- 172.21.15.113:0/2037133595 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0b0070 con 0x7fa0a406cd10 2024-03-04T07:49:54.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a2000640 1 -- 172.21.15.113:0/2037133595 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c0ab600 con 0x7fa0a406cd10 2024-03-04T07:49:54.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a2000640 1 -- 172.21.15.113:0/2037133595 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0ab9d0 con 0x7fa0a406cd10 2024-03-04T07:49:54.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a3e00640 1 -- 172.21.15.113:0/2037133595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a406cd10 msgr2=0x7fa0a406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:54.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/2037133595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a406cd10 0x7fa0a406d150 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0097c0 tx=0x7fa08c01c4a0 comp rx=0 tx=0).stop 2024-03-04T07:49:54.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a3e00640 1 -- 172.21.15.113:0/2037133595 shutdown_connections 2024-03-04T07:49:54.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/2037133595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a406cd10 0x7fa0a406d150 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:54.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/2037133595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a4152a90 0x7fa0a40730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:54.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a3e00640 1 -- 172.21.15.113:0/2037133595 >> 172.21.15.113:0/2037133595 conn(0x7fa0a406e2a0 msgr2=0x7fa0a406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:54.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a3e00640 1 -- 172.21.15.113:0/2037133595 shutdown_connections 2024-03-04T07:49:54.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a3e00640 1 -- 172.21.15.113:0/2037133595 wait complete. 2024-03-04T07:49:54.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a3e00640 1 Processor -- start 2024-03-04T07:49:54.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.789+0000 7fa0a3e00640 1 -- start start 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0a406cd10 0x7fa0a4148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a4152a90 0x7fa0a41f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a415bce0 0x7fa0a41514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a415c860 con 0x7fa0a4152a90 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a3e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a4071330 con 0x7fa0a406cd10 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a415c9e0 con 0x7fa0a415bce0 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a4152a90 0x7fa0a41f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a4152a90 0x7fa0a41f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36334/0 (socket says 172.21.15.113:36334) 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09a000640 1 -- 172.21.15.113:0/4065381290 learned_addr learned my addr 172.21.15.113:0/4065381290 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a2a00640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0a406cd10 0x7fa0a4148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a3400640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a415bce0 0x7fa0a41514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:54.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09a000640 1 -- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a415bce0 msgr2=0x7fa0a41514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:54.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09a000640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a415bce0 0x7fa0a41514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:54.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09a000640 1 -- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0a406cd10 msgr2=0x7fa0a4148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:54.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09a000640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0a406cd10 0x7fa0a4148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:54.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09a000640 1 -- 172.21.15.113:0/4065381290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a4153250 con 0x7fa0a4152a90 2024-03-04T07:49:54.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09a000640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a4152a90 0x7fa0a41f37a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0ae040 tx=0x7fa08c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:54.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09b400640 1 -- 172.21.15.113:0/4065381290 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0b0040 con 0x7fa0a4152a90 2024-03-04T07:49:54.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09b400640 1 -- 172.21.15.113:0/4065381290 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c0ab400 con 0x7fa0a4152a90 2024-03-04T07:49:54.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa09b400640 1 -- 172.21.15.113:0/4065381290 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0b4650 con 0x7fa0a4152a90 2024-03-04T07:49:54.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a3e00640 1 -- 172.21.15.113:0/4065381290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a4159b60 con 0x7fa0a4152a90 2024-03-04T07:49:54.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a3e00640 1 -- 172.21.15.113:0/4065381290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a41f3ce0 con 0x7fa0a4152a90 2024-03-04T07:49:54.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.793+0000 7fa0a3e00640 1 -- 172.21.15.113:0/4065381290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a415a170 con 0x7fa0a4152a90 2024-03-04T07:49:54.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.797+0000 7fa09b400640 1 -- 172.21.15.113:0/4065381290 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa08c0bc020 con 0x7fa0a4152a90 2024-03-04T07:49:54.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.797+0000 7fa09b400640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa080066f90 0x7fa080069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:54.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.797+0000 7fa09b400640 1 -- 172.21.15.113:0/4065381290 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fa08c11e7b0 con 0x7fa0a4152a90 2024-03-04T07:49:54.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.797+0000 7fa0a2a00640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa080066f90 0x7fa080069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:54.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.797+0000 7fa0a2a00640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa080066f90 0x7fa080069450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa09400b930 tx=0x7fa094008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:54.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:54.805+0000 7fa09b400640 1 -- 172.21.15.113:0/4065381290 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa08c0c3020 con 0x7fa0a4152a90 2024-03-04T07:49:55.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 -- 172.21.15.113:0/4065381290 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0a4075a80 con 0x7fa0a4152a90 2024-03-04T07:49:55.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa09b400640 1 -- 172.21.15.113:0/4065381290 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5813 (secure 0 0 0) 0x7fa08c016090 con 0x7fa0a4152a90 2024-03-04T07:49:55.187 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:55.187 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:55.067314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":26,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5631,5529]},"id":1}]} 2024-03-04T07:49:55.188 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 67 2024-03-04T07:49:55.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 -- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa080066f90 msgr2=0x7fa080069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:55.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa080066f90 0x7fa080069450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa09400b930 tx=0x7fa094008040 comp rx=0 tx=0).stop 2024-03-04T07:49:55.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 -- 172.21.15.113:0/4065381290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a4152a90 msgr2=0x7fa0a41f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:55.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a4152a90 0x7fa0a41f37a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0ae040 tx=0x7fa08c019240 comp rx=0 tx=0).stop 2024-03-04T07:49:55.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 -- 172.21.15.113:0/4065381290 shutdown_connections 2024-03-04T07:49:55.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0a415bce0 0x7fa0a41514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:55.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa080066f90 0x7fa080069450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:55.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0a4152a90 0x7fa0a41f37a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:55.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 --2- 172.21.15.113:0/4065381290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0a406cd10 0x7fa0a4148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:55.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 -- 172.21.15.113:0/4065381290 >> 172.21.15.113:0/4065381290 conn(0x7fa0a406e2a0 msgr2=0x7fa0a415c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:55.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 -- 172.21.15.113:0/4065381290 shutdown_connections 2024-03-04T07:49:55.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:55.189+0000 7fa0a3e00640 1 -- 172.21.15.113:0/4065381290 wait complete. 2024-03-04T07:49:55.209 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:49:57.210 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:57.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.349+0000 7f4192400640 1 Processor -- start 2024-03-04T07:49:57.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.349+0000 7f4192400640 1 -- start start 2024-03-04T07:49:57.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.349+0000 7f4192400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f418c1527f0 0x7f418c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:57.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.349+0000 7f4192400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f418c0c8200 0x7f418c072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:57.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f4192400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f418c1591f0 con 0x7f418c152bd0 2024-03-04T07:49:57.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f4192400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418c071df0 con 0x7f418c1527f0 2024-03-04T07:49:57.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f4192400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418c1597b0 con 0x7f418c0c8200 2024-03-04T07:49:57.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f418c1527f0 0x7f418c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:57.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f418c152bd0 0x7f418c1587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41108/0 (socket says 172.21.15.113:41108) 2024-03-04T07:49:57.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418be00640 1 -- 172.21.15.113:0/1876349962 learned_addr learned my addr 172.21.15.113:0/1876349962 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:49:57.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418aa00640 1 --2- 172.21.15.113:0/1876349962 >> v2:172.21.15.136:3300/0 conn(0x7f418c0c8200 0x7f418c072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:57.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418b400640 1 -- 172.21.15.113:0/1876349962 >> v2:172.21.15.136:3300/0 conn(0x7f418c0c8200 msgr2=0x7f418c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:57.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418a000640 1 -- 172.21.15.113:0/1876349962 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079685617 0 0) 0x7f418c1591f0 con 0x7f418c152bd0 2024-03-04T07:49:57.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418b400640 1 --2- 172.21.15.113:0/1876349962 >> v2:172.21.15.136:3300/0 conn(0x7f418c0c8200 0x7f418c072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:57.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418b400640 1 -- 172.21.15.113:0/1876349962 >> v1:172.21.15.113:6789/0 conn(0x7f418c152bd0 legacy=0x7f418c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:57.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418b400640 1 -- 172.21.15.113:0/1876349962 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c1594d0 con 0x7f418c1527f0 2024-03-04T07:49:57.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418b400640 1 --2- 172.21.15.113:0/1876349962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c1527f0 0x7f418c070d40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f418c0cd230 tx=0x7f417c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9392a7fd345daab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:57.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.353+0000 7f418a000640 1 -- 172.21.15.113:0/1876349962 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c0b0070 con 0x7f418c1527f0 2024-03-04T07:49:57.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f418a000640 1 -- 172.21.15.113:0/1876349962 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c0ab600 con 0x7f418c1527f0 2024-03-04T07:49:57.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f418a000640 1 -- 172.21.15.113:0/1876349962 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c0ab9d0 con 0x7f418c1527f0 2024-03-04T07:49:57.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 -- 172.21.15.113:0/1876349962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c1527f0 msgr2=0x7f418c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:57.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 --2- 172.21.15.113:0/1876349962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c1527f0 0x7f418c070d40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f418c0cd230 tx=0x7f417c01c480 comp rx=0 tx=0).stop 2024-03-04T07:49:57.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 -- 172.21.15.113:0/1876349962 shutdown_connections 2024-03-04T07:49:57.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 --2- 172.21.15.113:0/1876349962 >> v2:172.21.15.136:3300/0 conn(0x7f418c0c8200 0x7f418c072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:57.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 --2- 172.21.15.113:0/1876349962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c1527f0 0x7f418c070d40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:57.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 -- 172.21.15.113:0/1876349962 >> 172.21.15.113:0/1876349962 conn(0x7f418c06e820 msgr2=0x7f418c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:57.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 -- 172.21.15.113:0/1876349962 shutdown_connections 2024-03-04T07:49:57.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 -- 172.21.15.113:0/1876349962 wait complete. 2024-03-04T07:49:57.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 Processor -- start 2024-03-04T07:49:57.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 -- start start 2024-03-04T07:49:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c0c8200 0x7f418c14a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f418c1527f0 0x7f418c0c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:57.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f418c152bd0 0x7f418c0c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c165160 con 0x7f418c0c8200 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c0727c0 con 0x7f418c1527f0 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f4192400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c1652e0 con 0x7f418c152bd0 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f418b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c0c8200 0x7f418c14a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f418b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c0c8200 0x7f418c14a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45182/0 (socket says 172.21.15.113:45182) 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f418b400640 1 -- 172.21.15.113:0/640832961 learned_addr learned my addr 172.21.15.113:0/640832961 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f418aa00640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f418c1527f0 0x7f418c0c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f418be00640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f418c152bd0 0x7f418c0c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f418b400640 1 -- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f418c152bd0 msgr2=0x7f418c0c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.357+0000 7f418b400640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f418c152bd0 0x7f418c0c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f418b400640 1 -- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f418c1527f0 msgr2=0x7f418c0c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f418b400640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f418c1527f0 0x7f418c0c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:57.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f418b400640 1 -- 172.21.15.113:0/640832961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c14acd0 con 0x7f418c0c8200 2024-03-04T07:49:57.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f418b400640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c0c8200 0x7f418c14a2d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f417c0ae040 tx=0x7f417c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:57.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f4183400640 1 -- 172.21.15.113:0/640832961 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c0b0040 con 0x7f418c0c8200 2024-03-04T07:49:57.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f4183400640 1 -- 172.21.15.113:0/640832961 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c0b9070 con 0x7f418c0c8200 2024-03-04T07:49:57.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f4183400640 1 -- 172.21.15.113:0/640832961 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c0b4650 con 0x7f418c0c8200 2024-03-04T07:49:57.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f4192400640 1 -- 172.21.15.113:0/640832961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418c14dd10 con 0x7f418c0c8200 2024-03-04T07:49:57.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f4192400640 1 -- 172.21.15.113:0/640832961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418c14da70 con 0x7f418c0c8200 2024-03-04T07:49:57.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f4192400640 1 -- 172.21.15.113:0/640832961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418c075c00 con 0x7f418c0c8200 2024-03-04T07:49:57.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.361+0000 7f4183400640 1 -- 172.21.15.113:0/640832961 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f417c0b47f0 con 0x7f418c0c8200 2024-03-04T07:49:57.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.365+0000 7f4183400640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f416c066f60 0x7f416c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:57.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.365+0000 7f4183400640 1 -- 172.21.15.113:0/640832961 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f417c11fc00 con 0x7f418c0c8200 2024-03-04T07:49:57.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.365+0000 7f418aa00640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f416c066f60 0x7f416c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:57.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.365+0000 7f418aa00640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f416c066f60 0x7f416c069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f418c0caf20 tx=0x7f4178008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:57.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.365+0000 7f4183400640 1 -- 172.21.15.113:0/640832961 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f417c016070 con 0x7f418c0c8200 2024-03-04T07:49:57.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 -- 172.21.15.113:0/640832961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f418c0c79d0 con 0x7f418c0c8200 2024-03-04T07:49:57.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4183400640 1 -- 172.21.15.113:0/640832961 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5813 (secure 0 0 0) 0x7f417c0f0900 con 0x7f418c0c8200 2024-03-04T07:49:57.667 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:49:57.667 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:55.067314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":26,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5631,5529]},"id":1}]} 2024-03-04T07:49:57.667 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 67 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 -- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f416c066f60 msgr2=0x7f416c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f416c066f60 0x7f416c069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f418c0caf20 tx=0x7f4178008040 comp rx=0 tx=0).stop 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 -- 172.21.15.113:0/640832961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c0c8200 msgr2=0x7f418c14a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c0c8200 0x7f418c14a2d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f417c0ae040 tx=0x7f417c019220 comp rx=0 tx=0).stop 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 -- 172.21.15.113:0/640832961 shutdown_connections 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f418c152bd0 0x7f418c0c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f418c1527f0 0x7f418c0c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f416c066f60 0x7f416c069420 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 --2- 172.21.15.113:0/640832961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f418c0c8200 0x7f418c14a2d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 -- 172.21.15.113:0/640832961 >> 172.21.15.113:0/640832961 conn(0x7f418c06e820 msgr2=0x7f418c1520a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 -- 172.21.15.113:0/640832961 shutdown_connections 2024-03-04T07:49:57.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:57.669+0000 7f4192400640 1 -- 172.21.15.113:0/640832961 wait complete. 2024-03-04T07:49:57.694 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 67, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 67, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5529": { "addr": "172.21.15.113:6836/2299268553", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 2299268553, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 2299268553, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5529, "incarnation": 45, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:active", "state_seq": 43 }, "gid_5631": { "addr": "172.21.15.113:6833/810789479", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 810789479, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 810789479, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5631, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 17 }, "gid_5940": { "addr": "172.21.15.113:6837/344022786", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 344022786, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 344022786, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5940, "incarnation": 64, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:rejoin", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 57, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:49:55.067314+0000", "qdb_cluster": [ 5631, 5529 ], "qdb_leader": 5529, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5940, "mds_1": 5529, "mds_2": 5631 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:49:59.000 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:49:59.004 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:49:59.696 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:49:59.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdac1000640 1 Processor -- start 2024-03-04T07:49:59.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdac1000640 1 -- start start 2024-03-04T07:49:59.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdac1000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdabc151760 0x7fdabc0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdac1000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdabc151b40 0x7fdabc15b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdac1000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc06c290 con 0x7fdabc15b8c0 2024-03-04T07:49:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdac1000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc15aef0 con 0x7fdabc151760 2024-03-04T07:49:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdac1000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc06ef10 con 0x7fdabc151b40 2024-03-04T07:49:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdabb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdabc151760 0x7fdabc0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdabb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdabc151760 0x7fdabc0c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45186/0 (socket says 172.21.15.113:45186) 2024-03-04T07:49:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdabb400640 1 -- 172.21.15.113:0/186474763 learned_addr learned my addr 172.21.15.113:0/186474763 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdab3e00640 1 --2- 172.21.15.113:0/186474763 >> v2:172.21.15.136:3300/0 conn(0x7fdabc151b40 0x7fdabc15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:59.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdabb400640 1 -- 172.21.15.113:0/186474763 >> v2:172.21.15.136:3300/0 conn(0x7fdabc151b40 msgr2=0x7fdabc15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:49:59.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdabb400640 1 --2- 172.21.15.113:0/186474763 >> v2:172.21.15.136:3300/0 conn(0x7fdabc151b40 0x7fdabc15b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:59.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdabb400640 1 -- 172.21.15.113:0/186474763 >> v1:172.21.15.136:6789/0 conn(0x7fdabc15b8c0 legacy=0x7fdabc071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:59.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.837+0000 7fdabb400640 1 -- 172.21.15.113:0/186474763 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0c1000 con 0x7fdabc151760 2024-03-04T07:49:59.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdabb400640 1 --2- 172.21.15.113:0/186474763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 0x7fdabc0c83d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdabc06c4b0 tx=0x7fdaac01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edb76871919b61a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:59.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdabaa00640 1 -- 172.21.15.113:0/186474763 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0b2070 con 0x7fdabc151760 2024-03-04T07:49:59.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdabaa00640 1 -- 172.21.15.113:0/186474763 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac0ad680 con 0x7fdabc151760 2024-03-04T07:49:59.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdabaa00640 1 -- 172.21.15.113:0/186474763 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0ada50 con 0x7fdabc151760 2024-03-04T07:49:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdac1000640 1 -- 172.21.15.113:0/186474763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 msgr2=0x7fdabc0c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdac1000640 1 --2- 172.21.15.113:0/186474763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 0x7fdabc0c83d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdabc06c4b0 tx=0x7fdaac01c490 comp rx=0 tx=0).stop 2024-03-04T07:49:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdac1000640 1 -- 172.21.15.113:0/186474763 shutdown_connections 2024-03-04T07:49:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdac1000640 1 --2- 172.21.15.113:0/186474763 >> v2:172.21.15.136:3300/0 conn(0x7fdabc151b40 0x7fdabc15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdac1000640 1 --2- 172.21.15.113:0/186474763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 0x7fdabc0c83d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdac1000640 1 -- 172.21.15.113:0/186474763 >> 172.21.15.113:0/186474763 conn(0x7fdabc06e2a0 msgr2=0x7fdabc06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:49:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdac1000640 1 -- 172.21.15.113:0/186474763 shutdown_connections 2024-03-04T07:49:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdac1000640 1 -- 172.21.15.113:0/186474763 wait complete. 2024-03-04T07:49:59.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdac1000640 1 Processor -- start 2024-03-04T07:49:59.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.845+0000 7fdac1000640 1 -- start start 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdac1000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 0x7fdabc0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdac1000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdabc151b40 0x7fdabc14fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdac1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdabc13ab80 0x7fdabc0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdac1000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc164730 con 0x7fdabc151760 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdac1000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0c1e20 con 0x7fdabc13ab80 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdac1000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc1648b0 con 0x7fdabc151b40 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdabb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 0x7fdabc0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdabb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 0x7fdabc0cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45194/0 (socket says 172.21.15.113:45194) 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdabb400640 1 -- 172.21.15.113:0/3339311894 learned_addr learned my addr 172.21.15.113:0/3339311894 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdab3e00640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdabc151b40 0x7fdabc14fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdabbe00640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdabc13ab80 0x7fdabc0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdabb400640 1 -- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdabc151b40 msgr2=0x7fdabc14fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdabb400640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdabc151b40 0x7fdabc14fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdabb400640 1 -- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdabc13ab80 msgr2=0x7fdabc0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:49:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdabb400640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdabc13ab80 0x7fdabc0c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:49:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdabb400640 1 -- 172.21.15.113:0/3339311894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc150f60 con 0x7fdabc151760 2024-03-04T07:49:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdabb400640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 0x7fdabc0cb800 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdaac01a0c0 tx=0x7fdaac01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdab3400640 1 -- 172.21.15.113:0/3339311894 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0b2040 con 0x7fdabc151760 2024-03-04T07:49:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdab3400640 1 -- 172.21.15.113:0/3339311894 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac0bd070 con 0x7fdabc151760 2024-03-04T07:49:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdab3400640 1 -- 172.21.15.113:0/3339311894 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0ade30 con 0x7fdabc151760 2024-03-04T07:49:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdac1000640 1 -- 172.21.15.113:0/3339311894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc150c80 con 0x7fdabc151760 2024-03-04T07:49:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdac1000640 1 -- 172.21.15.113:0/3339311894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc14f3a0 con 0x7fdabc151760 2024-03-04T07:49:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdac1000640 1 -- 172.21.15.113:0/3339311894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc075a80 con 0x7fdabc151760 2024-03-04T07:49:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.849+0000 7fdab3400640 1 -- 172.21.15.113:0/3339311894 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdaac0c2460 con 0x7fdabc151760 2024-03-04T07:49:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.853+0000 7fdab3400640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda98066f90 0x7fda98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:49:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.853+0000 7fdab3400640 1 -- 172.21.15.113:0/3339311894 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fdaac11f5a0 con 0x7fdabc151760 2024-03-04T07:49:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.853+0000 7fdab3e00640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda98066f90 0x7fda98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:49:59.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.853+0000 7fdab3e00640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda98066f90 0x7fda98069450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdabc13b470 tx=0x7fdaa4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:49:59.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:49:59.857+0000 7fdab3400640 1 -- 172.21.15.113:0/3339311894 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdabc150f60 con 0x7fdabc151760 2024-03-04T07:50:00.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 -- 172.21.15.113:0/3339311894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdabc06c070 con 0x7fdabc151760 2024-03-04T07:50:00.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdab3400640 1 -- 172.21.15.113:0/3339311894 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5813 (secure 0 0 0) 0x7fdaac016110 con 0x7fdabc151760 2024-03-04T07:50:00.099 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:50:00.099 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:49:55.067314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":26,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5631,5529]},"id":1}]} 2024-03-04T07:50:00.100 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 67 2024-03-04T07:50:00.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 -- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda98066f90 msgr2=0x7fda98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:00.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda98066f90 0x7fda98069450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdabc13b470 tx=0x7fdaa4008040 comp rx=0 tx=0).stop 2024-03-04T07:50:00.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 -- 172.21.15.113:0/3339311894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 msgr2=0x7fdabc0cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:00.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 0x7fdabc0cb800 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdaac01a0c0 tx=0x7fdaac01acd0 comp rx=0 tx=0).stop 2024-03-04T07:50:00.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 -- 172.21.15.113:0/3339311894 shutdown_connections 2024-03-04T07:50:00.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdabc13ab80 0x7fdabc0c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:00.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdabc151b40 0x7fdabc14fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:00.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda98066f90 0x7fda98069450 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:00.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 --2- 172.21.15.113:0/3339311894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdabc151760 0x7fdabc0cb800 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:00.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 -- 172.21.15.113:0/3339311894 >> 172.21.15.113:0/3339311894 conn(0x7fdabc06e2a0 msgr2=0x7fdabc06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:00.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.101+0000 7fdac1000640 1 -- 172.21.15.113:0/3339311894 shutdown_connections 2024-03-04T07:50:00.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:00.105+0000 7fdac1000640 1 -- 172.21.15.113:0/3339311894 wait complete. 2024-03-04T07:50:00.127 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:50:02.128 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:50:02.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.269+0000 7fc4cbe00640 1 Processor -- start 2024-03-04T07:50:02.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.269+0000 7fc4cbe00640 1 -- start start 2024-03-04T07:50:02.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.269+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 0x7fc4cc0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:02.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.269+0000 7fc4cbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc4cc06c170 0x7fc4cc0c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:02.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.269+0000 7fc4cbe00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4cc06de30 con 0x7fc4cc1581d0 2024-03-04T07:50:02.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.269+0000 7fc4cbe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4cc06dc00 con 0x7fc4cc06c170 2024-03-04T07:50:02.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.269+0000 7fc4cbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc06c5f0 con 0x7fc4cc157df0 2024-03-04T07:50:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.269+0000 7fc4caa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 0x7fc4cc0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.273+0000 7fc4cb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc4cc06c170 0x7fc4cc0c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.273+0000 7fc4caa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 0x7fc4cc0c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:39438/0 (socket says 172.21.15.113:39438) 2024-03-04T07:50:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.273+0000 7fc4caa00640 1 -- 172.21.15.113:0/2714891018 learned_addr learned my addr 172.21.15.113:0/2714891018 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:50:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.273+0000 7fc4cb400640 1 -- 172.21.15.113:0/2714891018 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 msgr2=0x7fc4cc0c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:50:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.273+0000 7fc4cb400640 1 --2- 172.21.15.113:0/2714891018 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 0x7fc4cc0c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.273+0000 7fc4cb400640 1 -- 172.21.15.113:0/2714891018 >> v1:172.21.15.136:6789/0 conn(0x7fc4cc1581d0 legacy=0x7fc4cc06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:02.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.273+0000 7fc4cb400640 1 -- 172.21.15.113:0/2714891018 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc06e110 con 0x7fc4cc06c170 2024-03-04T07:50:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.273+0000 7fc4cb400640 1 --2- 172.21.15.113:0/2714891018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4cc06c170 0x7fc4cc0c9be0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009d90 tx=0x7fc4c401f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fc26e2e0c97c24d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4ca000640 1 -- 172.21.15.113:0/2714891018 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40b4070 con 0x7fc4cc06c170 2024-03-04T07:50:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4ca000640 1 -- 172.21.15.113:0/2714891018 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40af600 con 0x7fc4cc06c170 2024-03-04T07:50:02.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4ca000640 1 -- 172.21.15.113:0/2714891018 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40af9d0 con 0x7fc4cc06c170 2024-03-04T07:50:02.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 -- 172.21.15.113:0/2714891018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4cc06c170 msgr2=0x7fc4cc0c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:02.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 --2- 172.21.15.113:0/2714891018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4cc06c170 0x7fc4cc0c9be0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009d90 tx=0x7fc4c401f150 comp rx=0 tx=0).stop 2024-03-04T07:50:02.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 -- 172.21.15.113:0/2714891018 shutdown_connections 2024-03-04T07:50:02.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 --2- 172.21.15.113:0/2714891018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4cc06c170 0x7fc4cc0c9be0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:02.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 --2- 172.21.15.113:0/2714891018 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 0x7fc4cc0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:02.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 -- 172.21.15.113:0/2714891018 >> 172.21.15.113:0/2714891018 conn(0x7fc4cc072f40 msgr2=0x7fc4cc0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:02.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 -- 172.21.15.113:0/2714891018 shutdown_connections 2024-03-04T07:50:02.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 -- 172.21.15.113:0/2714891018 wait complete. 2024-03-04T07:50:02.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 Processor -- start 2024-03-04T07:50:02.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 -- start start 2024-03-04T07:50:02.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4cc06c170 0x7fc4cc152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:02.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 0x7fc4cc150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:02.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4cc1581d0 0x7fc4cc14f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:02.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc06f490 con 0x7fc4cc1581d0 2024-03-04T07:50:02.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc0ce3a0 con 0x7fc4cc06c170 2024-03-04T07:50:02.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc06f610 con 0x7fc4cc157df0 2024-03-04T07:50:02.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4cc1581d0 0x7fc4cc14f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:02.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4cc1581d0 0x7fc4cc14f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45202/0 (socket says 172.21.15.113:45202) 2024-03-04T07:50:02.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4cb400640 1 -- 172.21.15.113:0/195538057 learned_addr learned my addr 172.21.15.113:0/195538057 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:50:02.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4c3e00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 0x7fc4cc150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:02.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.277+0000 7fc4caa00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4cc06c170 0x7fc4cc152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4cb400640 1 -- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 msgr2=0x7fc4cc150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4cb400640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 0x7fc4cc150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4cb400640 1 -- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4cc06c170 msgr2=0x7fc4cc152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4cb400640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4cc06c170 0x7fc4cc152570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4cb400640 1 -- 172.21.15.113:0/195538057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc1f3e20 con 0x7fc4cc1581d0 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4cb400640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4cc1581d0 0x7fc4cc14f8e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009d90 tx=0x7fc4c401bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4c2a00640 1 -- 172.21.15.113:0/195538057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40b4040 con 0x7fc4cc1581d0 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4c2a00640 1 -- 172.21.15.113:0/195538057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40c1070 con 0x7fc4cc1581d0 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4cbe00640 1 -- 172.21.15.113:0/195538057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4cc1f14b0 con 0x7fc4cc1581d0 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4c2a00640 1 -- 172.21.15.113:0/195538057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4009450 con 0x7fc4cc1581d0 2024-03-04T07:50:02.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4cbe00640 1 -- 172.21.15.113:0/195538057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc1f3b80 con 0x7fc4cc1581d0 2024-03-04T07:50:02.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4cbe00640 1 -- 172.21.15.113:0/195538057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4cc075a80 con 0x7fc4cc1581d0 2024-03-04T07:50:02.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.281+0000 7fc4c2a00640 1 -- 172.21.15.113:0/195538057 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4c40ba6b0 con 0x7fc4cc1581d0 2024-03-04T07:50:02.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.285+0000 7fc4c2a00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4a8066f60 0x7fc4a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:02.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.285+0000 7fc4c2a00640 1 -- 172.21.15.113:0/195538057 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fc4c4123be0 con 0x7fc4cc1581d0 2024-03-04T07:50:02.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.285+0000 7fc4caa00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4a8066f60 0x7fc4a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:02.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.285+0000 7fc4caa00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4a8066f60 0x7fc4a8069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009b50 tx=0x7fc4bc009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:02.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.289+0000 7fc4c2a00640 1 -- 172.21.15.113:0/195538057 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc4c4018cb0 con 0x7fc4cc1581d0 2024-03-04T07:50:02.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.505+0000 7fc4cbe00640 1 -- 172.21.15.113:0/195538057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4cc0c9f70 con 0x7fc4cc1581d0 2024-03-04T07:50:02.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.505+0000 7fc4c2a00640 1 -- 172.21.15.113:0/195538057 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5819 (secure 0 0 0) 0x7fc4c4123020 con 0x7fc4cc1581d0 2024-03-04T07:50:02.504 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:50:02.504 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:00.214185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:clientreplay","state_seq":28,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:50:02.505 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-03-04T07:50:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 -- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4a8066f60 msgr2=0x7fc4a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4a8066f60 0x7fc4a8069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009b50 tx=0x7fc4bc009240 comp rx=0 tx=0).stop 2024-03-04T07:50:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 -- 172.21.15.113:0/195538057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4cc1581d0 msgr2=0x7fc4cc14f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4cc1581d0 0x7fc4cc14f8e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009d90 tx=0x7fc4c401bef0 comp rx=0 tx=0).stop 2024-03-04T07:50:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 -- 172.21.15.113:0/195538057 shutdown_connections 2024-03-04T07:50:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc4a8066f60 0x7fc4a8069420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4cc1581d0 0x7fc4cc14f8e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc4cc157df0 0x7fc4cc150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:02.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 --2- 172.21.15.113:0/195538057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4cc06c170 0x7fc4cc152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:02.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 -- 172.21.15.113:0/195538057 >> 172.21.15.113:0/195538057 conn(0x7fc4cc072f40 msgr2=0x7fc4cc0b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:02.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 -- 172.21.15.113:0/195538057 shutdown_connections 2024-03-04T07:50:02.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:02.509+0000 7fc4cbe00640 1 -- 172.21.15.113:0/195538057 wait complete. 2024-03-04T07:50:02.525 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:50:04.527 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:50:04.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.673+0000 7fef4da00640 1 Processor -- start 2024-03-04T07:50:04.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.673+0000 7fef4da00640 1 -- start start 2024-03-04T07:50:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef4da00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fef480c22e0 0x7fef4815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef4da00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef480c27f0 con 0x7fef480c2d80 2024-03-04T07:50:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef4da00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef480c2a90 con 0x7fef4814e4c0 2024-03-04T07:50:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef4da00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4815ad60 con 0x7fef480c22e0 2024-03-04T07:50:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef46000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fef480c22e0 0x7fef4815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef46000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fef480c22e0 0x7fef4815b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45218/0 (socket says 172.21.15.113:45218) 2024-03-04T07:50:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef46000640 1 -- 172.21.15.113:0/559088481 learned_addr learned my addr 172.21.15.113:0/559088481 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:50:04.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef47400640 1 --1- 172.21.15.113:0/559088481 >> v1:172.21.15.113:6789/0 conn(0x7fef480c2d80 0x7fef481620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41116/0 (socket says 172.21.15.113:41116) 2024-03-04T07:50:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef46000640 1 -- 172.21.15.113:0/559088481 >> v1:172.21.15.136:6789/0 conn(0x7fef4814e4c0 legacy=0x7fef48072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef46000640 1 -- 172.21.15.113:0/559088481 >> v1:172.21.15.113:6789/0 conn(0x7fef480c2d80 legacy=0x7fef481620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:04.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.677+0000 7fef46000640 1 -- 172.21.15.113:0/559088481 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef48070c60 con 0x7fef480c22e0 2024-03-04T07:50:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef46000640 1 --2- 172.21.15.113:0/559088481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef480c22e0 0x7fef4815b1f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fef34009870 tx=0x7fef3401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937e140226b13169 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:04.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef45600640 1 -- 172.21.15.113:0/559088481 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef340ab650 con 0x7fef480c22e0 2024-03-04T07:50:04.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef45600640 1 -- 172.21.15.113:0/559088481 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef340b0070 con 0x7fef480c22e0 2024-03-04T07:50:04.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef45600640 1 -- 172.21.15.113:0/559088481 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef340aba10 con 0x7fef480c22e0 2024-03-04T07:50:04.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef4da00640 1 -- 172.21.15.113:0/559088481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef480c22e0 msgr2=0x7fef4815b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:04.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef4da00640 1 --2- 172.21.15.113:0/559088481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef480c22e0 0x7fef4815b1f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fef34009870 tx=0x7fef3401c560 comp rx=0 tx=0).stop 2024-03-04T07:50:04.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef4da00640 1 -- 172.21.15.113:0/559088481 shutdown_connections 2024-03-04T07:50:04.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef4da00640 1 --2- 172.21.15.113:0/559088481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef480c22e0 0x7fef4815b1f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:04.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef4da00640 1 -- 172.21.15.113:0/559088481 >> 172.21.15.113:0/559088481 conn(0x7fef4806e820 msgr2=0x7fef4806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:04.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef4da00640 1 -- 172.21.15.113:0/559088481 shutdown_connections 2024-03-04T07:50:04.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef4da00640 1 -- 172.21.15.113:0/559088481 wait complete. 2024-03-04T07:50:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef4da00640 1 Processor -- start 2024-03-04T07:50:04.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.681+0000 7fef4da00640 1 -- start start 2024-03-04T07:50:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef4da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef480c22e0 0x7fef48149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef4da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef480c2d80 0x7fef48150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:04.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef4da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef4814e4c0 0x7fef48143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef4da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4814e8a0 con 0x7fef4814e4c0 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef4da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4815bb40 con 0x7fef480c22e0 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef4da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef480aede0 con 0x7fef480c2d80 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef47400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef4814e4c0 0x7fef48143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef47400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef4814e4c0 0x7fef48143f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45230/0 (socket says 172.21.15.113:45230) 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef47400640 1 -- 172.21.15.113:0/2813903818 learned_addr learned my addr 172.21.15.113:0/2813903818 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef46000640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef480c2d80 0x7fef48150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef46a00640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef480c22e0 0x7fef48149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef47400640 1 -- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef480c2d80 msgr2=0x7fef48150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef47400640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef480c2d80 0x7fef48150140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef47400640 1 -- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef480c22e0 msgr2=0x7fef48149620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef47400640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef480c22e0 0x7fef48149620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:04.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef47400640 1 -- 172.21.15.113:0/2813903818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef480c7960 con 0x7fef4814e4c0 2024-03-04T07:50:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef47400640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef4814e4c0 0x7fef48143f60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c730 tx=0x7fef4000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef3ea00640 1 -- 172.21.15.113:0/2813903818 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40017070 con 0x7fef4814e4c0 2024-03-04T07:50:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef3ea00640 1 -- 172.21.15.113:0/2813903818 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4000ce40 con 0x7fef4814e4c0 2024-03-04T07:50:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef3ea00640 1 -- 172.21.15.113:0/2813903818 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4000a840 con 0x7fef4814e4c0 2024-03-04T07:50:04.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef4da00640 1 -- 172.21.15.113:0/2813903818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4814f660 con 0x7fef4814e4c0 2024-03-04T07:50:04.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef4da00640 1 -- 172.21.15.113:0/2813903818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4814fc20 con 0x7fef4814e4c0 2024-03-04T07:50:04.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.685+0000 7fef4da00640 1 -- 172.21.15.113:0/2813903818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef48075c00 con 0x7fef4814e4c0 2024-03-04T07:50:04.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.689+0000 7fef3ea00640 1 -- 172.21.15.113:0/2813903818 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef4001a070 con 0x7fef4814e4c0 2024-03-04T07:50:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.689+0000 7fef3ea00640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef28066f90 0x7fef28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:04.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.689+0000 7fef3ea00640 1 -- 172.21.15.113:0/2813903818 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fef40084860 con 0x7fef4814e4c0 2024-03-04T07:50:04.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.689+0000 7fef46a00640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef28066f90 0x7fef28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:04.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.689+0000 7fef46a00640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef28066f90 0x7fef28069450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fef38000f70 tx=0x7fef38009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:04.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.697+0000 7fef3ea00640 1 -- 172.21.15.113:0/2813903818 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fef40054e90 con 0x7fef4814e4c0 2024-03-04T07:50:04.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.929+0000 7fef4da00640 1 -- 172.21.15.113:0/2813903818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef48155190 con 0x7fef4814e4c0 2024-03-04T07:50:04.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef3ea00640 1 -- 172.21.15.113:0/2813903818 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5819 (secure 0 0 0) 0x7fef40058f30 con 0x7fef4814e4c0 2024-03-04T07:50:04.930 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:50:04.930 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:00.214185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:clientreplay","state_seq":28,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5529,"qdb_cluster":[5529,5631]},"id":1}]} 2024-03-04T07:50:04.930 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 68 2024-03-04T07:50:04.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 -- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef28066f90 msgr2=0x7fef28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:04.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef28066f90 0x7fef28069450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fef38000f70 tx=0x7fef38009040 comp rx=0 tx=0).stop 2024-03-04T07:50:04.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 -- 172.21.15.113:0/2813903818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef4814e4c0 msgr2=0x7fef48143f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:04.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef4814e4c0 0x7fef48143f60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fef4000c730 tx=0x7fef4000cbf0 comp rx=0 tx=0).stop 2024-03-04T07:50:04.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 -- 172.21.15.113:0/2813903818 shutdown_connections 2024-03-04T07:50:04.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef28066f90 0x7fef28069450 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:04.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef4814e4c0 0x7fef48143f60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:04.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef480c2d80 0x7fef48150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:04.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 --2- 172.21.15.113:0/2813903818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef480c22e0 0x7fef48149620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:04.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 -- 172.21.15.113:0/2813903818 >> 172.21.15.113:0/2813903818 conn(0x7fef4806e820 msgr2=0x7fef480c8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:04.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 -- 172.21.15.113:0/2813903818 shutdown_connections 2024-03-04T07:50:04.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:04.933+0000 7fef4da00640 1 -- 172.21.15.113:0/2813903818 wait complete. 2024-03-04T07:50:04.952 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:50:06.954 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:50:07.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.101+0000 7f1ed6800640 1 Processor -- start 2024-03-04T07:50:07.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.101+0000 7f1ed6800640 1 -- start start 2024-03-04T07:50:07.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ed6800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ed00cd650 0x7f1ed00c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:07.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ed00c7a80 0x7f1ed015b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:07.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ed6800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed0157830 con 0x7f1ed00cd270 2024-03-04T07:50:07.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ed6800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed0157af0 con 0x7f1ed00cd650 2024-03-04T07:50:07.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ed6800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed0157dd0 con 0x7f1ed00c7a80 2024-03-04T07:50:07.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ecea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ed00cd650 0x7f1ed00c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:07.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ecea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ed00cd650 0x7f1ed00c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45232/0 (socket says 172.21.15.113:45232) 2024-03-04T07:50:07.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ecea00640 1 -- 172.21.15.113:0/3479093564 learned_addr learned my addr 172.21.15.113:0/3479093564 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:50:07.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ecfe00640 1 --2- 172.21.15.113:0/3479093564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ed00c7a80 0x7f1ed015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:07.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ecea00640 1 -- 172.21.15.113:0/3479093564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ed00c7a80 msgr2=0x7f1ed015b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:50:07.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ecea00640 1 --2- 172.21.15.113:0/3479093564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ed00c7a80 0x7f1ed015b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:07.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ecea00640 1 -- 172.21.15.113:0/3479093564 >> v1:172.21.15.136:6789/0 conn(0x7f1ed00cd270 legacy=0x7f1ed0151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:07.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.105+0000 7f1ecea00640 1 -- 172.21.15.113:0/3479093564 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed01580b0 con 0x7f1ed00cd650 2024-03-04T07:50:07.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ecea00640 1 --2- 172.21.15.113:0/3479093564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd650 0x7f1ed00c8df0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009870 tx=0x7f1ebc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a9ad5742c2a746 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:07.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ece000640 1 -- 172.21.15.113:0/3479093564 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0a7070 con 0x7f1ed00cd650 2024-03-04T07:50:07.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ece000640 1 -- 172.21.15.113:0/3479093564 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc0b0070 con 0x7f1ed00cd650 2024-03-04T07:50:07.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ece000640 1 -- 172.21.15.113:0/3479093564 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0ab890 con 0x7f1ed00cd650 2024-03-04T07:50:07.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ed6800640 1 -- 172.21.15.113:0/3479093564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd650 msgr2=0x7f1ed00c8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:07.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ed6800640 1 --2- 172.21.15.113:0/3479093564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd650 0x7f1ed00c8df0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009870 tx=0x7f1ebc01c560 comp rx=0 tx=0).stop 2024-03-04T07:50:07.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ed6800640 1 -- 172.21.15.113:0/3479093564 shutdown_connections 2024-03-04T07:50:07.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ed6800640 1 --2- 172.21.15.113:0/3479093564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ed00c7a80 0x7f1ed015b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:07.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ed6800640 1 --2- 172.21.15.113:0/3479093564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd650 0x7f1ed00c8df0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ed6800640 1 -- 172.21.15.113:0/3479093564 >> 172.21.15.113:0/3479093564 conn(0x7f1ed00671b0 msgr2=0x7f1ed006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ed6800640 1 -- 172.21.15.113:0/3479093564 shutdown_connections 2024-03-04T07:50:07.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.109+0000 7f1ed6800640 1 -- 172.21.15.113:0/3479093564 wait complete. 2024-03-04T07:50:07.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ed6800640 1 Processor -- start 2024-03-04T07:50:07.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ed6800640 1 -- start start 2024-03-04T07:50:07.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ed00c7a80 0x7f1ed0158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:07.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd270 0x7f1ed0144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ed00cd650 0x7f1ed015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ed6800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed0164850 con 0x7f1ed00cd270 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ed6800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed0151bb0 con 0x7f1ed00cd650 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ed6800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed01649d0 con 0x7f1ed00c7a80 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd270 0x7f1ed0144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd270 0x7f1ed0144c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45234/0 (socket says 172.21.15.113:45234) 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecea00640 1 -- 172.21.15.113:0/3863815707 learned_addr learned my addr 172.21.15.113:0/3863815707 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecfe00640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ed00cd650 0x7f1ed015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecf400640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ed00c7a80 0x7f1ed0158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecea00640 1 -- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ed00c7a80 msgr2=0x7f1ed0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecea00640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ed00c7a80 0x7f1ed0158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecea00640 1 -- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ed00cd650 msgr2=0x7f1ed015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecea00640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ed00cd650 0x7f1ed015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecea00640 1 -- 172.21.15.113:0/3863815707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed00cc000 con 0x7f1ed00cd270 2024-03-04T07:50:07.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ecea00640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd270 0x7f1ed0144c50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0ad040 tx=0x7f1ebc019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ec7400640 1 -- 172.21.15.113:0/3863815707 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0bf610 con 0x7f1ed00cd270 2024-03-04T07:50:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ec7400640 1 -- 172.21.15.113:0/3863815707 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc0bf7d0 con 0x7f1ed00cd270 2024-03-04T07:50:07.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ed6800640 1 -- 172.21.15.113:0/3863815707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed00cc2e0 con 0x7f1ed00cd270 2024-03-04T07:50:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ec7400640 1 -- 172.21.15.113:0/3863815707 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0bfbc0 con 0x7f1ed00cd270 2024-03-04T07:50:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.113+0000 7f1ed6800640 1 -- 172.21.15.113:0/3863815707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed0145190 con 0x7f1ed00cd270 2024-03-04T07:50:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.117+0000 7f1ed6800640 1 -- 172.21.15.113:0/3863815707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed00cc800 con 0x7f1ed00cd270 2024-03-04T07:50:07.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.117+0000 7f1ec7400640 1 -- 172.21.15.113:0/3863815707 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ebc0bd020 con 0x7f1ed00cd270 2024-03-04T07:50:07.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.117+0000 7f1ec7400640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1eb0066f90 0x7f1eb0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:07.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.117+0000 7f1ec7400640 1 -- 172.21.15.113:0/3863815707 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f1ebc11fa50 con 0x7f1ed00cd270 2024-03-04T07:50:07.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.117+0000 7f1ecf400640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1eb0066f90 0x7f1eb0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:07.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.121+0000 7f1ecf400640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1eb0066f90 0x7f1eb0069450 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0001810 tx=0x7f1ec0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:07.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.125+0000 7f1ec7400640 1 -- 172.21.15.113:0/3863815707 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1ed00cc800 con 0x7f1ed00cd270 2024-03-04T07:50:07.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.321+0000 7f1ed6800640 1 -- 172.21.15.113:0/3863815707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ed0075c00 con 0x7f1ed00cd270 2024-03-04T07:50:07.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.321+0000 7f1ec7400640 1 -- 172.21.15.113:0/3863815707 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f1ebc016150 con 0x7f1ed00cd270 2024-03-04T07:50:07.319 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:50:07.319 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:06.860683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5529,5631,5940]},"id":1}]} 2024-03-04T07:50:07.319 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 69 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 -- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1eb0066f90 msgr2=0x7f1eb0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1eb0066f90 0x7f1eb0069450 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0001810 tx=0x7f1ec0009040 comp rx=0 tx=0).stop 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 -- 172.21.15.113:0/3863815707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd270 msgr2=0x7f1ed0144c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd270 0x7f1ed0144c50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0ad040 tx=0x7f1ebc019300 comp rx=0 tx=0).stop 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 -- 172.21.15.113:0/3863815707 shutdown_connections 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ed00cd650 0x7f1ed015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1eb0066f90 0x7f1eb0069450 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ed00cd270 0x7f1ed0144c50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 --2- 172.21.15.113:0/3863815707 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ed00c7a80 0x7f1ed0158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 -- 172.21.15.113:0/3863815707 >> 172.21.15.113:0/3863815707 conn(0x7f1ed00671b0 msgr2=0x7f1ed006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 -- 172.21.15.113:0/3863815707 shutdown_connections 2024-03-04T07:50:07.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:07.325+0000 7f1ed6800640 1 -- 172.21.15.113:0/3863815707 wait complete. 2024-03-04T07:50:07.342 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=0, replacing gid=5417 2024-03-04T07:50:07.342 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.b 2024-03-04T07:50:09.343 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2024-03-04T07:50:09.343 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-04T07:50:09.344 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-04T07:50:09.347 INFO:tasks.ceph.mds.b:Started 2024-03-04T07:50:09.347 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:50:09.424 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2024-03-04T07:50:09.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6ce4a00640 1 Processor -- start 2024-03-04T07:50:09.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6ce4a00640 1 -- start start 2024-03-04T07:50:09.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6ce4a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6ce015b900 0x7f6ce0163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:09.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6ce4a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce00c27f0 con 0x7f6ce00c22e0 2024-03-04T07:50:09.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6ce4a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce00c2a90 con 0x7f6ce014e4c0 2024-03-04T07:50:09.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6ce4a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce015ad60 con 0x7f6ce015b900 2024-03-04T07:50:09.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6cd7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6ce00c22e0 0x7f6ce015b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39472/0 (socket says 172.21.15.113:39472) 2024-03-04T07:50:09.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6cd7e00640 1 -- 172.21.15.113:0/3582145539 learned_addr learned my addr 172.21.15.113:0/3582145539 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:50:09.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6cde800640 1 --2- 172.21.15.113:0/3582145539 >> v2:172.21.15.113:3300/0 conn(0x7f6ce015b900 0x7f6ce0163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:09.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6cdd400640 1 -- 172.21.15.113:0/3582145539 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626021800 0 0) 0x7f6ce00c27f0 con 0x7f6ce00c22e0 2024-03-04T07:50:09.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6cdd400640 1 -- 172.21.15.113:0/3582145539 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc4003530 con 0x7f6ce00c22e0 2024-03-04T07:50:09.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6cde800640 1 -- 172.21.15.113:0/3582145539 >> v1:172.21.15.136:6789/0 conn(0x7f6ce014e4c0 legacy=0x7f6ce0072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:09.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6cde800640 1 -- 172.21.15.113:0/3582145539 >> v1:172.21.15.113:6789/0 conn(0x7f6ce00c22e0 legacy=0x7f6ce015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:09.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.489+0000 7f6cde800640 1 -- 172.21.15.113:0/3582145539 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce0070c60 con 0x7f6ce015b900 2024-03-04T07:50:09.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6cde800640 1 --2- 172.21.15.113:0/3582145539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce015b900 0x7f6ce0163100 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009d90 tx=0x7f6cd801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6d25327b9649fe server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:09.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6cdd400640 1 -- 172.21.15.113:0/3582145539 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80b4070 con 0x7f6ce015b900 2024-03-04T07:50:09.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6cdd400640 1 -- 172.21.15.113:0/3582145539 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd80af600 con 0x7f6ce015b900 2024-03-04T07:50:09.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6cdd400640 1 -- 172.21.15.113:0/3582145539 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80af9d0 con 0x7f6ce015b900 2024-03-04T07:50:09.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3582145539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce015b900 msgr2=0x7f6ce0163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:09.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6ce4a00640 1 --2- 172.21.15.113:0/3582145539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce015b900 0x7f6ce0163100 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8009d90 tx=0x7f6cd801f150 comp rx=0 tx=0).stop 2024-03-04T07:50:09.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3582145539 shutdown_connections 2024-03-04T07:50:09.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6ce4a00640 1 --2- 172.21.15.113:0/3582145539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce015b900 0x7f6ce0163100 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:09.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3582145539 >> 172.21.15.113:0/3582145539 conn(0x7f6ce006e820 msgr2=0x7f6ce006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:09.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3582145539 shutdown_connections 2024-03-04T07:50:09.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.493+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3582145539 wait complete. 2024-03-04T07:50:09.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6ce4a00640 1 Processor -- start 2024-03-04T07:50:09.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6ce4a00640 1 -- start start 2024-03-04T07:50:09.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6ce4a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ce00c22e0 0x7f6ce01513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:09.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6ce4a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce014e4c0 0x7f6ce014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:09.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6ce4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce015b900 0x7f6ce0150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:09.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6ce4a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce0071500 con 0x7f6ce014e4c0 2024-03-04T07:50:09.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6ce4a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce0071220 con 0x7f6ce015b900 2024-03-04T07:50:09.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6ce4a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce014e8a0 con 0x7f6ce00c22e0 2024-03-04T07:50:09.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce014e4c0 0x7f6ce014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:09.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce014e4c0 0x7f6ce014fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56296/0 (socket says 172.21.15.113:56296) 2024-03-04T07:50:09.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd7e00640 1 -- 172.21.15.113:0/3365367706 learned_addr learned my addr 172.21.15.113:0/3365367706 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:50:09.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cdde00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ce00c22e0 0x7f6ce01513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:09.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cde800640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce015b900 0x7f6ce0150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:09.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd7e00640 1 -- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ce00c22e0 msgr2=0x7f6ce01513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:09.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd7e00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ce00c22e0 0x7f6ce01513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:09.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd7e00640 1 -- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce015b900 msgr2=0x7f6ce0150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:09.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd7e00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce015b900 0x7f6ce0150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:09.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd7e00640 1 -- 172.21.15.113:0/3365367706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce00c5b00 con 0x7f6ce014e4c0 2024-03-04T07:50:09.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cde800640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce015b900 0x7f6ce0150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:50:09.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cdde00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ce00c22e0 0x7f6ce01513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:50:09.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd7e00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce014e4c0 0x7f6ce014fc10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800c420 tx=0x7f6cc800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:09.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd6000640 1 -- 172.21.15.113:0/3365367706 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8017070 con 0x7f6ce014e4c0 2024-03-04T07:50:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd6000640 1 -- 172.21.15.113:0/3365367706 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc8005cf0 con 0x7f6ce014e4c0 2024-03-04T07:50:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6cd6000640 1 -- 172.21.15.113:0/3365367706 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc80072d0 con 0x7f6ce014e4c0 2024-03-04T07:50:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3365367706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce0154920 con 0x7f6ce014e4c0 2024-03-04T07:50:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.497+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3365367706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce0154640 con 0x7f6ce014e4c0 2024-03-04T07:50:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.501+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3365367706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce0075c00 con 0x7f6ce014e4c0 2024-03-04T07:50:09.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.501+0000 7f6cd6000640 1 -- 172.21.15.113:0/3365367706 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6cc801a070 con 0x7f6ce014e4c0 2024-03-04T07:50:09.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.501+0000 7f6cd6000640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6cbc066f90 0x7f6cbc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:09.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.501+0000 7f6cd6000640 1 -- 172.21.15.113:0/3365367706 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f6cc8084640 con 0x7f6ce014e4c0 2024-03-04T07:50:09.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.501+0000 7f6cdde00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6cbc066f90 0x7f6cbc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:09.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.505+0000 7f6cdde00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6cbc066f90 0x7f6cbc069450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0000f70 tx=0x7f6cd0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.513+0000 7f6cd6000640 1 -- 172.21.15.113:0/3365367706 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6ce00c5b00 con 0x7f6ce014e4c0 2024-03-04T07:50:09.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.701+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3365367706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ce00c1260 con 0x7f6ce014e4c0 2024-03-04T07:50:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6cd6000640 1 -- 172.21.15.113:0/3365367706 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f6cc8054c70 con 0x7f6ce014e4c0 2024-03-04T07:50:09.702 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:50:09.702 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:06.860683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5529,5631,5940]},"id":1}]} 2024-03-04T07:50:09.703 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 69 2024-03-04T07:50:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6cbc066f90 msgr2=0x7f6cbc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6cbc066f90 0x7f6cbc069450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0000f70 tx=0x7f6cd0009040 comp rx=0 tx=0).stop 2024-03-04T07:50:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3365367706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce014e4c0 msgr2=0x7f6ce014fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:09.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce014e4c0 0x7f6ce014fc10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800c420 tx=0x7f6cc800c8e0 comp rx=0 tx=0).stop 2024-03-04T07:50:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3365367706 shutdown_connections 2024-03-04T07:50:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ce015b900 0x7f6ce0150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6cbc066f90 0x7f6cbc069450 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ce014e4c0 0x7f6ce014fc10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 --2- 172.21.15.113:0/3365367706 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6ce00c22e0 0x7f6ce01513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3365367706 >> 172.21.15.113:0/3365367706 conn(0x7f6ce006e820 msgr2=0x7f6ce00c8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3365367706 shutdown_connections 2024-03-04T07:50:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:09.705+0000 7f6ce4a00640 1 -- 172.21.15.113:0/3365367706 wait complete. 2024-03-04T07:50:09.724 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2024-03-04T07:50:11.725 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:50:11.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.913+0000 7f0542a00640 1 Processor -- start 2024-03-04T07:50:11.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.913+0000 7f0542a00640 1 -- start start 2024-03-04T07:50:11.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.913+0000 7f0542a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c06cde0 0x7f053c1663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:11.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.913+0000 7f0542a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053c072710 con 0x7f053c154d50 2024-03-04T07:50:11.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.913+0000 7f0542a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053c15b710 con 0x7f053c06b6b0 2024-03-04T07:50:11.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.913+0000 7f0542a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053c072c90 con 0x7f053c06cde0 2024-03-04T07:50:11.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.913+0000 7f053be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f053c154d50 0x7f053c0c20a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39480/0 (socket says 172.21.15.113:39480) 2024-03-04T07:50:11.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.913+0000 7f053be00640 1 -- 172.21.15.113:0/1565989300 learned_addr learned my addr 172.21.15.113:0/1565989300 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:50:11.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.913+0000 7f0540a00640 1 --2- 172.21.15.113:0/1565989300 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c06cde0 0x7f053c1663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:11.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.917+0000 7f053aa00640 1 -- 172.21.15.113:0/1565989300 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390919676 0 0) 0x7f053c072710 con 0x7f053c154d50 2024-03-04T07:50:11.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.917+0000 7f053aa00640 1 -- 172.21.15.113:0/1565989300 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0520003530 con 0x7f053c154d50 2024-03-04T07:50:11.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.917+0000 7f0540a00640 1 -- 172.21.15.113:0/1565989300 >> v1:172.21.15.136:6789/0 conn(0x7f053c06b6b0 legacy=0x7f053c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:11.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.917+0000 7f0540a00640 1 -- 172.21.15.113:0/1565989300 >> v1:172.21.15.113:6789/0 conn(0x7f053c154d50 legacy=0x7f053c0c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:11.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.917+0000 7f0540a00640 1 -- 172.21.15.113:0/1565989300 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c0729b0 con 0x7f053c06cde0 2024-03-04T07:50:11.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f0540a00640 1 --2- 172.21.15.113:0/1565989300 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c06cde0 0x7f053c1663c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0534009dc0 tx=0x7f053401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=560a20138d589efd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:11.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f053aa00640 1 -- 172.21.15.113:0/1565989300 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340b2070 con 0x7f053c06cde0 2024-03-04T07:50:11.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f053aa00640 1 -- 172.21.15.113:0/1565989300 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05340a5850 con 0x7f053c06cde0 2024-03-04T07:50:11.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f053aa00640 1 -- 172.21.15.113:0/1565989300 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340a5c00 con 0x7f053c06cde0 2024-03-04T07:50:11.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f0542a00640 1 -- 172.21.15.113:0/1565989300 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c06cde0 msgr2=0x7f053c1663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:11.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f0542a00640 1 --2- 172.21.15.113:0/1565989300 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c06cde0 0x7f053c1663c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0534009dc0 tx=0x7f053401c990 comp rx=0 tx=0).stop 2024-03-04T07:50:11.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f0542a00640 1 -- 172.21.15.113:0/1565989300 shutdown_connections 2024-03-04T07:50:11.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f0542a00640 1 --2- 172.21.15.113:0/1565989300 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c06cde0 0x7f053c1663c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:11.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f0542a00640 1 -- 172.21.15.113:0/1565989300 >> 172.21.15.113:0/1565989300 conn(0x7f053c0671b0 msgr2=0x7f053c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:11.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f0542a00640 1 -- 172.21.15.113:0/1565989300 shutdown_connections 2024-03-04T07:50:11.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.921+0000 7f0542a00640 1 -- 172.21.15.113:0/1565989300 wait complete. 2024-03-04T07:50:11.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0542a00640 1 Processor -- start 2024-03-04T07:50:11.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0542a00640 1 -- start start 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0542a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053c06cde0 0x7f053c166130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0542a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c154d50 0x7f053c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0542a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053c0aea30 0x7f053c156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0542a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c169120 con 0x7f053c0aea30 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0542a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c155340 con 0x7f053c06cde0 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0542a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053c1692a0 con 0x7f053c154d50 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0540a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053c0aea30 0x7f053c156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0540a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053c0aea30 0x7f053c156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56302/0 (socket says 172.21.15.113:56302) 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0540a00640 1 -- 172.21.15.113:0/1723243830 learned_addr learned my addr 172.21.15.113:0/1723243830 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f053b400640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c154d50 0x7f053c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f053be00640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053c06cde0 0x7f053c166130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0540a00640 1 -- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c154d50 msgr2=0x7f053c0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0540a00640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c154d50 0x7f053c0cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0540a00640 1 -- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053c06cde0 msgr2=0x7f053c166130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0540a00640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053c06cde0 0x7f053c166130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0540a00640 1 -- 172.21.15.113:0/1723243830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c0c8960 con 0x7f053c0aea30 2024-03-04T07:50:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.925+0000 7f0540a00640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053c0aea30 0x7f053c156ee0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f053401ce70 tx=0x7f0534019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.929+0000 7f0533e00640 1 -- 172.21.15.113:0/1723243830 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340b2050 con 0x7f053c0aea30 2024-03-04T07:50:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.929+0000 7f0533e00640 1 -- 172.21.15.113:0/1723243830 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05340be070 con 0x7f053c0aea30 2024-03-04T07:50:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.929+0000 7f0533e00640 1 -- 172.21.15.113:0/1723243830 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534003db0 con 0x7f053c0aea30 2024-03-04T07:50:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.929+0000 7f0542a00640 1 -- 172.21.15.113:0/1723243830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c0c6960 con 0x7f053c0aea30 2024-03-04T07:50:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.929+0000 7f0542a00640 1 -- 172.21.15.113:0/1723243830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c0c6680 con 0x7f053c0aea30 2024-03-04T07:50:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.929+0000 7f0531600640 1 -- 172.21.15.113:0/1723243830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053c075c00 con 0x7f053c0aea30 2024-03-04T07:50:11.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.929+0000 7f0533e00640 1 -- 172.21.15.113:0/1723243830 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0534004040 con 0x7f053c0aea30 2024-03-04T07:50:11.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.929+0000 7f0533e00640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f051c066f60 0x7f051c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:50:11.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.929+0000 7f0533e00640 1 -- 172.21.15.113:0/1723243830 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f0534120360 con 0x7f053c0aea30 2024-03-04T07:50:11.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.929+0000 7f053be00640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f051c066f60 0x7f051c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:50:11.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.933+0000 7f053be00640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f051c066f60 0x7f051c069420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f052c0011f0 tx=0x7f052c007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:50:11.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:11.941+0000 7f0533e00640 1 -- 172.21.15.113:0/1723243830 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0534016580 con 0x7f053c0aea30 2024-03-04T07:50:12.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.169+0000 7f0531600640 1 -- 172.21.15.113:0/1723243830 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f053c0c3eb0 con 0x7f053c0aea30 2024-03-04T07:50:12.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.169+0000 7f0533e00640 1 -- 172.21.15.113:0/1723243830 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f05340f0900 con 0x7f053c0aea30 2024-03-04T07:50:12.168 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:50:12.169 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:06.860683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631,5529]},"id":1}]} 2024-03-04T07:50:12.169 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 70 2024-03-04T07:50:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 -- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f051c066f60 msgr2=0x7f051c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f051c066f60 0x7f051c069420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f052c0011f0 tx=0x7f052c007d40 comp rx=0 tx=0).stop 2024-03-04T07:50:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 -- 172.21.15.113:0/1723243830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053c0aea30 msgr2=0x7f053c156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:50:12.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053c0aea30 0x7f053c156ee0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f053401ce70 tx=0x7f0534019730 comp rx=0 tx=0).stop 2024-03-04T07:50:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 -- 172.21.15.113:0/1723243830 shutdown_connections 2024-03-04T07:50:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f051c066f60 0x7f051c069420 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053c0aea30 0x7f053c156ee0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f053c154d50 0x7f053c0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 --2- 172.21.15.113:0/1723243830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053c06cde0 0x7f053c166130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:50:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 -- 172.21.15.113:0/1723243830 >> 172.21.15.113:0/1723243830 conn(0x7f053c0671b0 msgr2=0x7f053c153e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:50:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 -- 172.21.15.113:0/1723243830 shutdown_connections 2024-03-04T07:50:12.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:50:12.173+0000 7f0531600640 1 -- 172.21.15.113:0/1723243830 wait complete. 2024-03-04T07:50:12.197 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2024-03-04T07:50:12.197 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 103 secs before thrashing 2024-03-04T07:50:12.865 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T07:50:12.865+0000 7f4a0d600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T07:50:12.865 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T07:50:12.865+0000 7f4a0d600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T07:50:12.865 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T07:50:12.865+0000 7f4a0d600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-04T07:50:29.046 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:50:29.051 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:50:59.100 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:50:59.104 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:51:29.221 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:51:29.226 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:51:55.199 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:51:55.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.431+0000 7f2519e00640 1 Processor -- start 2024-03-04T07:51:55.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.431+0000 7f2519e00640 1 -- start start 2024-03-04T07:51:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.431+0000 7f2519e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f25140c8200 0x7f2514070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.431+0000 7f2519e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25141591f0 con 0x7f25141527f0 2024-03-04T07:51:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.431+0000 7f2519e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25141594d0 con 0x7f2514152bd0 2024-03-04T07:51:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.431+0000 7f2519e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2514071df0 con 0x7f25140c8200 2024-03-04T07:51:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.431+0000 7f2512a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f25141527f0 0x7f2514072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60178/0 (socket says 172.21.15.113:60178) 2024-03-04T07:51:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.431+0000 7f2512a00640 1 -- 172.21.15.113:0/2431363814 learned_addr learned my addr 172.21.15.113:0/2431363814 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:51:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.431+0000 7f2512000640 1 --2- 172.21.15.113:0/2431363814 >> v2:172.21.15.136:3300/0 conn(0x7f25140c8200 0x7f2514070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.435+0000 7f2511600640 1 -- 172.21.15.113:0/2431363814 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260306166 0 0) 0x7f25141591f0 con 0x7f25141527f0 2024-03-04T07:51:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.435+0000 7f2511600640 1 -- 172.21.15.113:0/2431363814 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f8003530 con 0x7f25141527f0 2024-03-04T07:51:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.435+0000 7f2512000640 1 -- 172.21.15.113:0/2431363814 >> v1:172.21.15.136:6789/0 conn(0x7f2514152bd0 legacy=0x7f25141587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.435+0000 7f2512000640 1 -- 172.21.15.113:0/2431363814 >> v1:172.21.15.113:6789/0 conn(0x7f25141527f0 legacy=0x7f2514072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.435+0000 7f2512000640 1 -- 172.21.15.113:0/2431363814 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140717a0 con 0x7f25140c8200 2024-03-04T07:51:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2512000640 1 --2- 172.21.15.113:0/2431363814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25140c8200 0x7f2514070d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f25000097c0 tx=0x7f250001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=582fd416f75c74b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:51:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2511600640 1 -- 172.21.15.113:0/2431363814 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000b1070 con 0x7f25140c8200 2024-03-04T07:51:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2511600640 1 -- 172.21.15.113:0/2431363814 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25000ac410 con 0x7f25140c8200 2024-03-04T07:51:55.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2511600640 1 -- 172.21.15.113:0/2431363814 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000ac7c0 con 0x7f25140c8200 2024-03-04T07:51:55.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2519e00640 1 -- 172.21.15.113:0/2431363814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25140c8200 msgr2=0x7f2514070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2519e00640 1 --2- 172.21.15.113:0/2431363814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25140c8200 0x7f2514070d40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f25000097c0 tx=0x7f250001c4a0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2519e00640 1 -- 172.21.15.113:0/2431363814 shutdown_connections 2024-03-04T07:51:55.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2519e00640 1 --2- 172.21.15.113:0/2431363814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25140c8200 0x7f2514070d40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2519e00640 1 -- 172.21.15.113:0/2431363814 >> 172.21.15.113:0/2431363814 conn(0x7f251406e820 msgr2=0x7f251406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:51:55.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2519e00640 1 -- 172.21.15.113:0/2431363814 shutdown_connections 2024-03-04T07:51:55.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.439+0000 7f2519e00640 1 -- 172.21.15.113:0/2431363814 wait complete. 2024-03-04T07:51:55.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2519e00640 1 Processor -- start 2024-03-04T07:51:55.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2519e00640 1 -- start start 2024-03-04T07:51:55.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2519e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25140c8200 0x7f2514158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2519e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25141527f0 0x7f25140ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2519e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2514152bd0 0x7f25140ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2519e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2514166380 con 0x7f2514152bd0 2024-03-04T07:51:55.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2519e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141597b0 con 0x7f25140c8200 2024-03-04T07:51:55.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2519e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2514166500 con 0x7f25141527f0 2024-03-04T07:51:55.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2513400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2514152bd0 0x7f25140ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2513400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2514152bd0 0x7f25140ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48182/0 (socket says 172.21.15.113:48182) 2024-03-04T07:51:55.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2513400640 1 -- 172.21.15.113:0/1110641840 learned_addr learned my addr 172.21.15.113:0/1110641840 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:51:55.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2512a00640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25140c8200 0x7f2514158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2512000640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25141527f0 0x7f25140ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2513400640 1 -- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25141527f0 msgr2=0x7f25140ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2513400640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25141527f0 0x7f25140ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2513400640 1 -- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25140c8200 msgr2=0x7f2514158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2513400640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25140c8200 0x7f2514158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2513400640 1 -- 172.21.15.113:0/1110641840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140cbd20 con 0x7f2514152bd0 2024-03-04T07:51:55.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2513400640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2514152bd0 0x7f25140ca8e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f250c00c760 tx=0x7f250c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:51:55.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f250b400640 1 -- 172.21.15.113:0/1110641840 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c003b80 con 0x7f2514152bd0 2024-03-04T07:51:55.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f250b400640 1 -- 172.21.15.113:0/1110641840 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c003d20 con 0x7f2514152bd0 2024-03-04T07:51:55.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2519e00640 1 -- 172.21.15.113:0/1110641840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25140caec0 con 0x7f2514152bd0 2024-03-04T07:51:55.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f250b400640 1 -- 172.21.15.113:0/1110641840 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c00aa30 con 0x7f2514152bd0 2024-03-04T07:51:55.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.443+0000 7f2519e00640 1 -- 172.21.15.113:0/1110641840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25140cb480 con 0x7f2514152bd0 2024-03-04T07:51:55.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.447+0000 7f2519e00640 1 -- 172.21.15.113:0/1110641840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2514075c00 con 0x7f2514152bd0 2024-03-04T07:51:55.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.447+0000 7f250b400640 1 -- 172.21.15.113:0/1110641840 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f250c00abd0 con 0x7f2514152bd0 2024-03-04T07:51:55.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.447+0000 7f250b400640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24f4066f60 0x7f24f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.447+0000 7f250b400640 1 -- 172.21.15.113:0/1110641840 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f250c085240 con 0x7f2514152bd0 2024-03-04T07:51:55.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.447+0000 7f2512a00640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24f4066f60 0x7f24f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.447+0000 7f2512a00640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24f4066f60 0x7f24f4069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f25040011f0 tx=0x7f2504007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:51:55.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.451+0000 7f250b400640 1 -- 172.21.15.113:0/1110641840 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f250c055750 con 0x7f2514152bd0 2024-03-04T07:51:55.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.699+0000 7f2519e00640 1 -- 172.21.15.113:0/1110641840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25140af850 con 0x7f2514152bd0 2024-03-04T07:51:55.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f250b400640 1 -- 172.21.15.113:0/1110641840 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f250c0597f0 con 0x7f2514152bd0 2024-03-04T07:51:55.699 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:51:55.700 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:06.860683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631,5529]},"id":1}]} 2024-03-04T07:51:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 70 2024-03-04T07:51:55.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 -- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24f4066f60 msgr2=0x7f24f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24f4066f60 0x7f24f4069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f25040011f0 tx=0x7f2504007d40 comp rx=0 tx=0).stop 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 -- 172.21.15.113:0/1110641840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2514152bd0 msgr2=0x7f25140ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2514152bd0 0x7f25140ca8e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f250c00c760 tx=0x7f250c00cc20 comp rx=0 tx=0).stop 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 -- 172.21.15.113:0/1110641840 shutdown_connections 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24f4066f60 0x7f24f4069420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2514152bd0 0x7f25140ca8e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25141527f0 0x7f25140ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 --2- 172.21.15.113:0/1110641840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25140c8200 0x7f2514158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 -- 172.21.15.113:0/1110641840 >> 172.21.15.113:0/1110641840 conn(0x7f251406e820 msgr2=0x7f25140c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 -- 172.21.15.113:0/1110641840 shutdown_connections 2024-03-04T07:51:55.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.703+0000 7f2519e00640 1 -- 172.21.15.113:0/1110641840 wait complete. 2024-03-04T07:51:55.719 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=1) 2024-03-04T07:51:55.719 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-04T07:51:55.719 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:51:55.739 INFO:tasks.ceph.mds.a:Stopped 2024-03-04T07:51:55.739 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:51:55.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.919+0000 7f9524000640 1 Processor -- start 2024-03-04T07:51:55.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.919+0000 7f9524000640 1 -- start start 2024-03-04T07:51:55.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.919+0000 7f9524000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f951c152a50 0x7f951c0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.919+0000 7f9524000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f951c06ff20 0x7f951c0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.919+0000 7f9524000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c152e30 con 0x7f951c0c8b90 2024-03-04T07:51:55.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.919+0000 7f9524000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c0c31b0 con 0x7f951c06ff20 2024-03-04T07:51:55.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.919+0000 7f9524000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c06f840 con 0x7f951c152a50 2024-03-04T07:51:55.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f9520c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f951c06ff20 0x7f951c0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f9520c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f951c06ff20 0x7f951c0ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48184/0 (socket says 172.21.15.113:48184) 2024-03-04T07:51:55.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f9520c00640 1 -- 172.21.15.113:0/764106586 learned_addr learned my addr 172.21.15.113:0/764106586 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:51:55.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f9521600640 1 --2- 172.21.15.113:0/764106586 >> v2:172.21.15.136:3300/0 conn(0x7f951c152a50 0x7f951c0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f9520c00640 1 -- 172.21.15.113:0/764106586 >> v2:172.21.15.136:3300/0 conn(0x7f951c152a50 msgr2=0x7f951c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:51:55.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f9520c00640 1 --2- 172.21.15.113:0/764106586 >> v2:172.21.15.136:3300/0 conn(0x7f951c152a50 0x7f951c0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f9520c00640 1 -- 172.21.15.113:0/764106586 >> v1:172.21.15.136:6789/0 conn(0x7f951c0c8b90 legacy=0x7f951c15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f9520c00640 1 -- 172.21.15.113:0/764106586 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c06f280 con 0x7f951c06ff20 2024-03-04T07:51:55.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f9520c00640 1 --2- 172.21.15.113:0/764106586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c06ff20 0x7f951c0ce660 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f951c067cd0 tx=0x7f950801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c3c0ecc25015f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:51:55.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f951be00640 1 -- 172.21.15.113:0/764106586 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080aa680 con 0x7f951c06ff20 2024-03-04T07:51:55.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f951be00640 1 -- 172.21.15.113:0/764106586 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95080af070 con 0x7f951c06ff20 2024-03-04T07:51:55.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.923+0000 7f951be00640 1 -- 172.21.15.113:0/764106586 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080aaa40 con 0x7f951c06ff20 2024-03-04T07:51:55.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.927+0000 7f9524000640 1 -- 172.21.15.113:0/764106586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c06ff20 msgr2=0x7f951c0ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.927+0000 7f9524000640 1 --2- 172.21.15.113:0/764106586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c06ff20 0x7f951c0ce660 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f951c067cd0 tx=0x7f950801c520 comp rx=0 tx=0).stop 2024-03-04T07:51:55.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.927+0000 7f9524000640 1 -- 172.21.15.113:0/764106586 shutdown_connections 2024-03-04T07:51:55.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.927+0000 7f9524000640 1 --2- 172.21.15.113:0/764106586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c06ff20 0x7f951c0ce660 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.927+0000 7f9524000640 1 --2- 172.21.15.113:0/764106586 >> v2:172.21.15.136:3300/0 conn(0x7f951c152a50 0x7f951c0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.927+0000 7f9524000640 1 -- 172.21.15.113:0/764106586 >> 172.21.15.113:0/764106586 conn(0x7f951c0734d0 msgr2=0x7f951c066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:51:55.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.927+0000 7f9524000640 1 -- 172.21.15.113:0/764106586 shutdown_connections 2024-03-04T07:51:55.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.927+0000 7f9524000640 1 -- 172.21.15.113:0/764106586 wait complete. 2024-03-04T07:51:55.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.927+0000 7f9524000640 1 Processor -- start 2024-03-04T07:51:55.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.927+0000 7f9524000640 1 -- start start 2024-03-04T07:51:55.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9524000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f951c06ff20 0x7f951c1554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9524000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c0c8b90 0x7f951c1f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9524000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f951c152a50 0x7f951c1f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9524000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c165a00 con 0x7f951c0c8b90 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9524000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c165880 con 0x7f951c06ff20 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9524000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c165b80 con 0x7f951c152a50 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9520c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c0c8b90 0x7f951c1f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9520c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c0c8b90 0x7f951c1f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48192/0 (socket says 172.21.15.113:48192) 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9520c00640 1 -- 172.21.15.113:0/1180737294 learned_addr learned my addr 172.21.15.113:0/1180737294 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9522000640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f951c152a50 0x7f951c1f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9521600640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f951c06ff20 0x7f951c1554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9520c00640 1 -- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f951c152a50 msgr2=0x7f951c1f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9520c00640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f951c152a50 0x7f951c1f3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9520c00640 1 -- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f951c06ff20 msgr2=0x7f951c1554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9520c00640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f951c06ff20 0x7f951c1554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9520c00640 1 -- 172.21.15.113:0/1180737294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c14d6f0 con 0x7f951c0c8b90 2024-03-04T07:51:55.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9520c00640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c0c8b90 0x7f951c1f3890 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f95080ac040 tx=0x7f950801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:51:55.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9519600640 1 -- 172.21.15.113:0/1180737294 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080bd5f0 con 0x7f951c0c8b90 2024-03-04T07:51:55.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9519600640 1 -- 172.21.15.113:0/1180737294 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95080bd7b0 con 0x7f951c0c8b90 2024-03-04T07:51:55.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9524000640 1 -- 172.21.15.113:0/1180737294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c162a60 con 0x7f951c0c8b90 2024-03-04T07:51:55.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9519600640 1 -- 172.21.15.113:0/1180737294 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95080bdba0 con 0x7f951c0c8b90 2024-03-04T07:51:55.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9524000640 1 -- 172.21.15.113:0/1180737294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c14cc10 con 0x7f951c0c8b90 2024-03-04T07:51:55.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.931+0000 7f9524000640 1 -- 172.21.15.113:0/1180737294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c075c00 con 0x7f951c0c8b90 2024-03-04T07:51:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.935+0000 7f9519600640 1 -- 172.21.15.113:0/1180737294 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95080a8050 con 0x7f951c0c8b90 2024-03-04T07:51:55.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.935+0000 7f9519600640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94fc066f60 0x7f94fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:55.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.935+0000 7f9519600640 1 -- 172.21.15.113:0/1180737294 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f950811fe40 con 0x7f951c0c8b90 2024-03-04T07:51:55.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.935+0000 7f9521600640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94fc066f60 0x7f94fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:55.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.935+0000 7f9521600640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94fc066f60 0x7f94fc069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f951000b9d0 tx=0x7f9510008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:51:55.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:55.939+0000 7f9519600640 1 -- 172.21.15.113:0/1180737294 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f95080c3060 con 0x7f951c0c8b90 2024-03-04T07:51:56.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.135+0000 7f9524000640 1 -- 172.21.15.113:0/1180737294 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f951c15b840 con 0x7f951c0c8b90 2024-03-04T07:51:56.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.135+0000 7f9519600640 1 -- 172.21.15.113:0/1180737294 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f95080a6040 con 0x7f951c0c8b90 2024-03-04T07:51:56.134 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:51:56.134 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:06.860683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631,5529]},"id":1}]} 2024-03-04T07:51:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 70 2024-03-04T07:51:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 -- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94fc066f60 msgr2=0x7f94fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94fc066f60 0x7f94fc069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f951000b9d0 tx=0x7f9510008040 comp rx=0 tx=0).stop 2024-03-04T07:51:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 -- 172.21.15.113:0/1180737294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c0c8b90 msgr2=0x7f951c1f3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c0c8b90 0x7f951c1f3890 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f95080ac040 tx=0x7f950801ad60 comp rx=0 tx=0).stop 2024-03-04T07:51:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 -- 172.21.15.113:0/1180737294 shutdown_connections 2024-03-04T07:51:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f951c152a50 0x7f951c1f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94fc066f60 0x7f94fc069420 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:56.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f951c0c8b90 0x7f951c1f3890 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 --2- 172.21.15.113:0/1180737294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f951c06ff20 0x7f951c1554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 -- 172.21.15.113:0/1180737294 >> 172.21.15.113:0/1180737294 conn(0x7f951c0734d0 msgr2=0x7f951c0ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:51:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 -- 172.21.15.113:0/1180737294 shutdown_connections 2024-03-04T07:51:56.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:56.139+0000 7f9524000640 1 -- 172.21.15.113:0/1180737294 wait complete. 2024-03-04T07:51:56.154 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:51:58.155 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:51:58.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.347+0000 7fd17b200640 1 Processor -- start 2024-03-04T07:51:58.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.347+0000 7fd17b200640 1 -- start start 2024-03-04T07:51:58.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd17b200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd17c152950 0x7fd17c15bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:58.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd17b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd17c06faa0 0x7fd17c0c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:58.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd17b200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c0c4ce0 con 0x7fd17c06ed80 2024-03-04T07:51:58.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd17b200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17c0c8370 con 0x7fd17c152950 2024-03-04T07:51:58.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd17b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0ca0a0 con 0x7fd17c06faa0 2024-03-04T07:51:58.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd179e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd17c152950 0x7fd17c15bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:58.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd171400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd17c06faa0 0x7fd17c0c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:58.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd179e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd17c152950 0x7fd17c15bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:42060/0 (socket says 172.21.15.113:42060) 2024-03-04T07:51:58.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd179e00640 1 -- 172.21.15.113:0/585972267 learned_addr learned my addr 172.21.15.113:0/585972267 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:51:58.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd179e00640 1 -- 172.21.15.113:0/585972267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd17c06faa0 msgr2=0x7fd17c0c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:51:58.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd179e00640 1 --2- 172.21.15.113:0/585972267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd17c06faa0 0x7fd17c0c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:58.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd179e00640 1 -- 172.21.15.113:0/585972267 >> v1:172.21.15.136:6789/0 conn(0x7fd17c06ed80 legacy=0x7fd17c0ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:58.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.351+0000 7fd179e00640 1 -- 172.21.15.113:0/585972267 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0c4f80 con 0x7fd17c152950 2024-03-04T07:51:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd179e00640 1 --2- 172.21.15.113:0/585972267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd17c152950 0x7fd17c15bb40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009870 tx=0x7fd16c01ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ef6b56112573c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:51:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd179400640 1 -- 172.21.15.113:0/585972267 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0b5070 con 0x7fd17c152950 2024-03-04T07:51:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd179400640 1 -- 172.21.15.113:0/585972267 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c0a8a50 con 0x7fd17c152950 2024-03-04T07:51:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd179400640 1 -- 172.21.15.113:0/585972267 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0a8e20 con 0x7fd17c152950 2024-03-04T07:51:58.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd17b200640 1 -- 172.21.15.113:0/585972267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd17c152950 msgr2=0x7fd17c15bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:58.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd17b200640 1 --2- 172.21.15.113:0/585972267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd17c152950 0x7fd17c15bb40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009870 tx=0x7fd16c01ed50 comp rx=0 tx=0).stop 2024-03-04T07:51:58.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd17b200640 1 -- 172.21.15.113:0/585972267 shutdown_connections 2024-03-04T07:51:58.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd17b200640 1 --2- 172.21.15.113:0/585972267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd17c06faa0 0x7fd17c0c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd17b200640 1 --2- 172.21.15.113:0/585972267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd17c152950 0x7fd17c15bb40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd17b200640 1 -- 172.21.15.113:0/585972267 >> 172.21.15.113:0/585972267 conn(0x7fd17c072f40 msgr2=0x7fd17c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:51:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd17b200640 1 -- 172.21.15.113:0/585972267 shutdown_connections 2024-03-04T07:51:58.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.355+0000 7fd17b200640 1 -- 172.21.15.113:0/585972267 wait complete. 2024-03-04T07:51:58.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd17b200640 1 Processor -- start 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd17b200640 1 -- start start 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd17b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd17c06ed80 0x7fd17c0b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd17b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd17c06faa0 0x7fd17c15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd17b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd17c152950 0x7fd17c158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd17b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0c9a50 con 0x7fd17c06faa0 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd17b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c06fe80 con 0x7fd17c06ed80 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd17b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0c9d30 con 0x7fd17c152950 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd171400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd17c06faa0 0x7fd17c15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd171400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd17c06faa0 0x7fd17c15bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47540/0 (socket says 172.21.15.113:47540) 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd171400640 1 -- 172.21.15.113:0/2775043858 learned_addr learned my addr 172.21.15.113:0/2775043858 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd179e00640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd17c06ed80 0x7fd17c0b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:58.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd17a800640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd17c152950 0x7fd17c158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd171400640 1 -- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd17c152950 msgr2=0x7fd17c158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd171400640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd17c152950 0x7fd17c158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd171400640 1 -- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd17c06ed80 msgr2=0x7fd17c0b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd171400640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd17c06ed80 0x7fd17c0b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.359+0000 7fd171400640 1 -- 172.21.15.113:0/2775043858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c15a480 con 0x7fd17c06faa0 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.363+0000 7fd171400640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd17c06faa0 0x7fd17c15bb20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd16400e960 tx=0x7fd16400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.363+0000 7fd172a00640 1 -- 172.21.15.113:0/2775043858 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164017070 con 0x7fd17c06faa0 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.363+0000 7fd172a00640 1 -- 172.21.15.113:0/2775043858 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16400ca00 con 0x7fd17c06faa0 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.363+0000 7fd17b200640 1 -- 172.21.15.113:0/2775043858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c15a760 con 0x7fd17c06faa0 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.363+0000 7fd172a00640 1 -- 172.21.15.113:0/2775043858 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16400cdb0 con 0x7fd17c06faa0 2024-03-04T07:51:58.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.363+0000 7fd17b200640 1 -- 172.21.15.113:0/2775043858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c15ad20 con 0x7fd17c06faa0 2024-03-04T07:51:58.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.363+0000 7fd17b200640 1 -- 172.21.15.113:0/2775043858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c075a80 con 0x7fd17c06faa0 2024-03-04T07:51:58.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.363+0000 7fd172a00640 1 -- 172.21.15.113:0/2775043858 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd16401a070 con 0x7fd17c06faa0 2024-03-04T07:51:58.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.367+0000 7fd172a00640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd158066f90 0x7fd158069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:51:58.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.367+0000 7fd172a00640 1 -- 172.21.15.113:0/2775043858 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fd164085650 con 0x7fd17c06faa0 2024-03-04T07:51:58.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.367+0000 7fd179e00640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd158066f90 0x7fd158069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:51:58.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.367+0000 7fd179e00640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd158066f90 0x7fd158069450 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0b2000 tx=0x7fd16c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:51:58.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.371+0000 7fd172a00640 1 -- 172.21.15.113:0/2775043858 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd164055d20 con 0x7fd17c06faa0 2024-03-04T07:51:58.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.587+0000 7fd17b200640 1 -- 172.21.15.113:0/2775043858 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd17c0c9580 con 0x7fd17c06faa0 2024-03-04T07:51:58.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.587+0000 7fd172a00640 1 -- 172.21.15.113:0/2775043858 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fd164059dc0 con 0x7fd17c06faa0 2024-03-04T07:51:58.585 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:51:58.585 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:06.860683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631,5529]},"id":1}]} 2024-03-04T07:51:58.585 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 70 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 -- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd158066f90 msgr2=0x7fd158069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd158066f90 0x7fd158069450 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0b2000 tx=0x7fd16c009040 comp rx=0 tx=0).stop 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 -- 172.21.15.113:0/2775043858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd17c06faa0 msgr2=0x7fd17c15bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd17c06faa0 0x7fd17c15bb20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd16400e960 tx=0x7fd16400ee20 comp rx=0 tx=0).stop 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 -- 172.21.15.113:0/2775043858 shutdown_connections 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd17c152950 0x7fd17c158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd158066f90 0x7fd158069450 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd17c06faa0 0x7fd17c15bb20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 --2- 172.21.15.113:0/2775043858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd17c06ed80 0x7fd17c0b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 -- 172.21.15.113:0/2775043858 >> 172.21.15.113:0/2775043858 conn(0x7fd17c072f40 msgr2=0x7fd17c0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 -- 172.21.15.113:0/2775043858 shutdown_connections 2024-03-04T07:51:58.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:51:58.591+0000 7fd17b200640 1 -- 172.21.15.113:0/2775043858 wait complete. 2024-03-04T07:51:58.615 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:51:59.324 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:51:59.327 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:52:00.616 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:00.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.759+0000 7fde43200640 1 Processor -- start 2024-03-04T07:52:00.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.759+0000 7fde43200640 1 -- start start 2024-03-04T07:52:00.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde43200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde440c8a90 0x7fde4415e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:00.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde43200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fde4415ea60 0x7fde440ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:00.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde43200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde44070fd0 con 0x7fde44157df0 2024-03-04T07:52:00.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde43200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4415e090 con 0x7fde440c8a90 2024-03-04T07:52:00.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde43200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4406dfd0 con 0x7fde4415ea60 2024-03-04T07:52:00.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde41400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde440c8a90 0x7fde4415e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:00.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde41400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde440c8a90 0x7fde4415e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47552/0 (socket says 172.21.15.113:47552) 2024-03-04T07:52:00.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde41e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fde44157df0 0x7fde4406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51616/0 (socket says 172.21.15.113:51616) 2024-03-04T07:52:00.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde41400640 1 -- 172.21.15.113:0/787792780 learned_addr learned my addr 172.21.15.113:0/787792780 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:00.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde42800640 1 --2- 172.21.15.113:0/787792780 >> v2:172.21.15.136:3300/0 conn(0x7fde4415ea60 0x7fde440ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:00.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde41400640 1 -- 172.21.15.113:0/787792780 >> v2:172.21.15.136:3300/0 conn(0x7fde4415ea60 msgr2=0x7fde440ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:00.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde41400640 1 --2- 172.21.15.113:0/787792780 >> v2:172.21.15.136:3300/0 conn(0x7fde4415ea60 0x7fde440ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:00.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde41400640 1 -- 172.21.15.113:0/787792780 >> v1:172.21.15.113:6789/0 conn(0x7fde44157df0 legacy=0x7fde4406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:00.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde41400640 1 -- 172.21.15.113:0/787792780 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4406fcd0 con 0x7fde440c8a90 2024-03-04T07:52:00.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.763+0000 7fde40a00640 1 -- 172.21.15.113:0/787792780 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102319577 0 0) 0x7fde34001120 con 0x7fde44157df0 2024-03-04T07:52:00.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde41400640 1 --2- 172.21.15.113:0/787792780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 0x7fde4415e520 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fde440b37e0 tx=0x7fde3001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6320ce8c7333c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:00.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde40a00640 1 -- 172.21.15.113:0/787792780 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde300b0070 con 0x7fde440c8a90 2024-03-04T07:52:00.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde40a00640 1 -- 172.21.15.113:0/787792780 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde300ab600 con 0x7fde440c8a90 2024-03-04T07:52:00.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde40a00640 1 -- 172.21.15.113:0/787792780 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde300ab9d0 con 0x7fde440c8a90 2024-03-04T07:52:00.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde43200640 1 -- 172.21.15.113:0/787792780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 msgr2=0x7fde4415e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:00.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde43200640 1 --2- 172.21.15.113:0/787792780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 0x7fde4415e520 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fde440b37e0 tx=0x7fde3001c520 comp rx=0 tx=0).stop 2024-03-04T07:52:00.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde43200640 1 -- 172.21.15.113:0/787792780 shutdown_connections 2024-03-04T07:52:00.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde43200640 1 --2- 172.21.15.113:0/787792780 >> v2:172.21.15.136:3300/0 conn(0x7fde4415ea60 0x7fde440ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:00.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde43200640 1 --2- 172.21.15.113:0/787792780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 0x7fde4415e520 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:00.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde43200640 1 -- 172.21.15.113:0/787792780 >> 172.21.15.113:0/787792780 conn(0x7fde44072f40 msgr2=0x7fde440706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:00.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde43200640 1 -- 172.21.15.113:0/787792780 shutdown_connections 2024-03-04T07:52:00.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.767+0000 7fde43200640 1 -- 172.21.15.113:0/787792780 wait complete. 2024-03-04T07:52:00.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde43200640 1 Processor -- start 2024-03-04T07:52:00.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde43200640 1 -- start start 2024-03-04T07:52:00.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde43200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 0x7fde4414a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:00.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde43200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde44157df0 0x7fde4414d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:00.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde43200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde4415ea60 0x7fde44146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:00.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde43200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4406dd80 con 0x7fde440c8a90 2024-03-04T07:52:00.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde43200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4406dc00 con 0x7fde4415ea60 2024-03-04T07:52:00.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde43200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde441581d0 con 0x7fde44157df0 2024-03-04T07:52:00.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde41e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 0x7fde4414a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:00.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde41e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 0x7fde4414a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47554/0 (socket says 172.21.15.113:47554) 2024-03-04T07:52:00.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde41e00640 1 -- 172.21.15.113:0/92690515 learned_addr learned my addr 172.21.15.113:0/92690515 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:00.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde42800640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde4415ea60 0x7fde44146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:00.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde41400640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde44157df0 0x7fde4414d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:00.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde41e00640 1 -- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde44157df0 msgr2=0x7fde4414d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:00.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde41e00640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde44157df0 0x7fde4414d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:00.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde41e00640 1 -- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde4415ea60 msgr2=0x7fde44146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:00.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde41e00640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde4415ea60 0x7fde44146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:00.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde41e00640 1 -- 172.21.15.113:0/92690515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4414a5b0 con 0x7fde440c8a90 2024-03-04T07:52:00.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde41e00640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 0x7fde4414a070 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fde3400c7b0 tx=0x7fde3400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:00.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde3a000640 1 -- 172.21.15.113:0/92690515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34018070 con 0x7fde440c8a90 2024-03-04T07:52:00.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde3a000640 1 -- 172.21.15.113:0/92690515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde340072f0 con 0x7fde440c8a90 2024-03-04T07:52:00.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde3a000640 1 -- 172.21.15.113:0/92690515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde340047f0 con 0x7fde440c8a90 2024-03-04T07:52:00.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde43200640 1 -- 172.21.15.113:0/92690515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde44149320 con 0x7fde440c8a90 2024-03-04T07:52:00.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.771+0000 7fde43200640 1 -- 172.21.15.113:0/92690515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde441495c0 con 0x7fde440c8a90 2024-03-04T07:52:00.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.775+0000 7fde43200640 1 -- 172.21.15.113:0/92690515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde44075a80 con 0x7fde440c8a90 2024-03-04T07:52:00.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.775+0000 7fde3a000640 1 -- 172.21.15.113:0/92690515 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde34004990 con 0x7fde440c8a90 2024-03-04T07:52:00.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.779+0000 7fde3a000640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde24066f60 0x7fde24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:00.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.779+0000 7fde3a000640 1 -- 172.21.15.113:0/92690515 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fde340847c0 con 0x7fde440c8a90 2024-03-04T07:52:00.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.779+0000 7fde41400640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde24066f60 0x7fde24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:00.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.779+0000 7fde41400640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde24066f60 0x7fde24069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fde300ad000 tx=0x7fde300a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:00.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:00.783+0000 7fde3a000640 1 -- 172.21.15.113:0/92690515 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fde44070fd0 con 0x7fde440c8a90 2024-03-04T07:52:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.007+0000 7fde43200640 1 -- 172.21.15.113:0/92690515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde4406f8a0 con 0x7fde440c8a90 2024-03-04T07:52:01.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.007+0000 7fde3a000640 1 -- 172.21.15.113:0/92690515 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fde340553c0 con 0x7fde440c8a90 2024-03-04T07:52:01.004 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:01.004 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:06.860683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631,5529]},"id":1}]} 2024-03-04T07:52:01.004 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 70 2024-03-04T07:52:01.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 -- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde24066f60 msgr2=0x7fde24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde24066f60 0x7fde24069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fde300ad000 tx=0x7fde300a6000 comp rx=0 tx=0).stop 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 -- 172.21.15.113:0/92690515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 msgr2=0x7fde4414a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 0x7fde4414a070 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fde3400c7b0 tx=0x7fde3400cc70 comp rx=0 tx=0).stop 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 -- 172.21.15.113:0/92690515 shutdown_connections 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde4415ea60 0x7fde44146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde44157df0 0x7fde4414d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde24066f60 0x7fde24069420 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 --2- 172.21.15.113:0/92690515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde440c8a90 0x7fde4414a070 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 -- 172.21.15.113:0/92690515 >> 172.21.15.113:0/92690515 conn(0x7fde44072f40 msgr2=0x7fde440c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 -- 172.21.15.113:0/92690515 shutdown_connections 2024-03-04T07:52:01.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:01.011+0000 7fde43200640 1 -- 172.21.15.113:0/92690515 wait complete. 2024-03-04T07:52:01.034 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:52:03.035 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.199+0000 7f4664800640 1 Processor -- start 2024-03-04T07:52:03.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.199+0000 7f4664800640 1 -- start start 2024-03-04T07:52:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.199+0000 7f4664800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f465c0c22e0 0x7f465c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.199+0000 7f4664800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c0c2d80 0x7f465c15b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.199+0000 7f4664800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f465c0c27f0 con 0x7f465c14e4c0 2024-03-04T07:52:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.199+0000 7f4664800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c15ad60 con 0x7f465c0c22e0 2024-03-04T07:52:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.199+0000 7f4664800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c0c2a90 con 0x7f465c0c2d80 2024-03-04T07:52:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.203+0000 7f4661400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f465c0c22e0 0x7f465c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.203+0000 7f4661400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f465c0c22e0 0x7f465c15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47562/0 (socket says 172.21.15.113:47562) 2024-03-04T07:52:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.203+0000 7f4661400640 1 -- 172.21.15.113:0/2444950985 learned_addr learned my addr 172.21.15.113:0/2444950985 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:03.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.203+0000 7f4662800640 1 --2- 172.21.15.113:0/2444950985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c0c2d80 0x7f465c15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:03.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.203+0000 7f4661400640 1 -- 172.21.15.113:0/2444950985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c0c2d80 msgr2=0x7f465c15b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:03.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.203+0000 7f4661400640 1 --2- 172.21.15.113:0/2444950985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c0c2d80 0x7f465c15b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.203+0000 7f4661400640 1 -- 172.21.15.113:0/2444950985 >> v1:172.21.15.136:6789/0 conn(0x7f465c14e4c0 legacy=0x7f465c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:03.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.203+0000 7f4661400640 1 -- 172.21.15.113:0/2444950985 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c070c60 con 0x7f465c0c22e0 2024-03-04T07:52:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4661400640 1 --2- 172.21.15.113:0/2444950985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c22e0 0x7f465c15b1f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f464c009870 tx=0x7f464c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6694b95a41d8537b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:03.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4660a00640 1 -- 172.21.15.113:0/2444950985 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0b5070 con 0x7f465c0c22e0 2024-03-04T07:52:03.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4660a00640 1 -- 172.21.15.113:0/2444950985 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c0a89d0 con 0x7f465c0c22e0 2024-03-04T07:52:03.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4660a00640 1 -- 172.21.15.113:0/2444950985 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0a8da0 con 0x7f465c0c22e0 2024-03-04T07:52:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4664800640 1 -- 172.21.15.113:0/2444950985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c22e0 msgr2=0x7f465c15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4664800640 1 --2- 172.21.15.113:0/2444950985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c22e0 0x7f465c15b1f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f464c009870 tx=0x7f464c01ed10 comp rx=0 tx=0).stop 2024-03-04T07:52:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4664800640 1 -- 172.21.15.113:0/2444950985 shutdown_connections 2024-03-04T07:52:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4664800640 1 --2- 172.21.15.113:0/2444950985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c0c2d80 0x7f465c15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4664800640 1 --2- 172.21.15.113:0/2444950985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c22e0 0x7f465c15b1f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4664800640 1 -- 172.21.15.113:0/2444950985 >> 172.21.15.113:0/2444950985 conn(0x7f465c06e820 msgr2=0x7f465c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4664800640 1 -- 172.21.15.113:0/2444950985 shutdown_connections 2024-03-04T07:52:03.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.207+0000 7f4664800640 1 -- 172.21.15.113:0/2444950985 wait complete. 2024-03-04T07:52:03.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4664800640 1 Processor -- start 2024-03-04T07:52:03.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4664800640 1 -- start start 2024-03-04T07:52:03.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4664800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f465c0c22e0 0x7f465c0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:03.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4664800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c2d80 0x7f465c0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4664800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c14e4c0 0x7f465c153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4664800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c071500 con 0x7f465c0c2d80 2024-03-04T07:52:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4664800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c071220 con 0x7f465c0c22e0 2024-03-04T07:52:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4664800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465c14e8a0 con 0x7f465c14e4c0 2024-03-04T07:52:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4661400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c2d80 0x7f465c0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4661400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c2d80 0x7f465c0b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47566/0 (socket says 172.21.15.113:47566) 2024-03-04T07:52:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4661400640 1 -- 172.21.15.113:0/1084850513 learned_addr learned my addr 172.21.15.113:0/1084850513 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4661e00640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f465c0c22e0 0x7f465c0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4662800640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c14e4c0 0x7f465c153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4661400640 1 -- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c14e4c0 msgr2=0x7f465c153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4661400640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c14e4c0 0x7f465c153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4661400640 1 -- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f465c0c22e0 msgr2=0x7f465c0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4661400640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f465c0c22e0 0x7f465c0b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4661400640 1 -- 172.21.15.113:0/1084850513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0ae930 con 0x7f465c0c2d80 2024-03-04T07:52:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4662800640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c14e4c0 0x7f465c153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:52:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.211+0000 7f4661400640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c2d80 0x7f465c0b07a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f464c01c940 tx=0x7f464c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.215+0000 7f4656000640 1 -- 172.21.15.113:0/1084850513 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0be070 con 0x7f465c0c2d80 2024-03-04T07:52:03.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.215+0000 7f4656000640 1 -- 172.21.15.113:0/1084850513 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c0b0640 con 0x7f465c0c2d80 2024-03-04T07:52:03.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.215+0000 7f4656000640 1 -- 172.21.15.113:0/1084850513 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0b5040 con 0x7f465c0c2d80 2024-03-04T07:52:03.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.215+0000 7f4664800640 1 -- 172.21.15.113:0/1084850513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c0aec10 con 0x7f465c0c2d80 2024-03-04T07:52:03.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.215+0000 7f4664800640 1 -- 172.21.15.113:0/1084850513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c153570 con 0x7f465c0c2d80 2024-03-04T07:52:03.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.215+0000 7f4664800640 1 -- 172.21.15.113:0/1084850513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c1530b0 con 0x7f465c0c2d80 2024-03-04T07:52:03.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.219+0000 7f4656000640 1 -- 172.21.15.113:0/1084850513 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f464c0b0a10 con 0x7f465c0c2d80 2024-03-04T07:52:03.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.219+0000 7f4656000640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640067060 0x7f4640069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:03.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.219+0000 7f4656000640 1 -- 172.21.15.113:0/1084850513 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f464c121d30 con 0x7f465c0c2d80 2024-03-04T07:52:03.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.219+0000 7f4661e00640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640067060 0x7f4640069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:03.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.223+0000 7f4661e00640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640067060 0x7f4640069520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4650004e30 tx=0x7f4650009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:03.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.227+0000 7f4656000640 1 -- 172.21.15.113:0/1084850513 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f465c1530b0 con 0x7f465c0c2d80 2024-03-04T07:52:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 -- 172.21.15.113:0/1084850513 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f465c075c00 con 0x7f465c0c2d80 2024-03-04T07:52:03.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4656000640 1 -- 172.21.15.113:0/1084850513 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f464c018870 con 0x7f465c0c2d80 2024-03-04T07:52:03.435 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:03.435 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:06.860683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631,5529]},"id":1}]} 2024-03-04T07:52:03.435 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 70 2024-03-04T07:52:03.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 -- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640067060 msgr2=0x7f4640069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640067060 0x7f4640069520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4650004e30 tx=0x7f4650009040 comp rx=0 tx=0).stop 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 -- 172.21.15.113:0/1084850513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c2d80 msgr2=0x7f465c0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c2d80 0x7f465c0b07a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f464c01c940 tx=0x7f464c01bab0 comp rx=0 tx=0).stop 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 -- 172.21.15.113:0/1084850513 shutdown_connections 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f465c14e4c0 0x7f465c153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640067060 0x7f4640069520 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f465c0c2d80 0x7f465c0b07a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 --2- 172.21.15.113:0/1084850513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f465c0c22e0 0x7f465c0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 -- 172.21.15.113:0/1084850513 >> 172.21.15.113:0/1084850513 conn(0x7f465c06e820 msgr2=0x7f465c0c15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 -- 172.21.15.113:0/1084850513 shutdown_connections 2024-03-04T07:52:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:03.439+0000 7f4664800640 1 -- 172.21.15.113:0/1084850513 wait complete. 2024-03-04T07:52:03.461 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:52:05.462 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:05.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd21ca00640 1 Processor -- start 2024-03-04T07:52:05.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd21ca00640 1 -- start start 2024-03-04T07:52:05.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd21ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2181527f0 0x7fd218070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:05.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd21ca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd218152bd0 0x7fd2181587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:05.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd21ca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2181591f0 con 0x7fd2180c8200 2024-03-04T07:52:05.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd21ca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd218071df0 con 0x7fd218152bd0 2024-03-04T07:52:05.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd21ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2181597b0 con 0x7fd2181527f0 2024-03-04T07:52:05.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd215e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2181527f0 0x7fd218070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:05.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd216800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd218152bd0 0x7fd2181587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:05.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd215e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2181527f0 0x7fd218070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:56986/0 (socket says 172.21.15.113:56986) 2024-03-04T07:52:05.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.607+0000 7fd215e00640 1 -- 172.21.15.113:0/3484764615 learned_addr learned my addr 172.21.15.113:0/3484764615 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:05.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.611+0000 7fd215e00640 1 -- 172.21.15.113:0/3484764615 >> v2:172.21.15.136:3300/0 conn(0x7fd218152bd0 msgr2=0x7fd2181587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:05.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.611+0000 7fd215e00640 1 --2- 172.21.15.113:0/3484764615 >> v2:172.21.15.136:3300/0 conn(0x7fd218152bd0 0x7fd2181587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:05.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.611+0000 7fd215e00640 1 -- 172.21.15.113:0/3484764615 >> v1:172.21.15.136:6789/0 conn(0x7fd2180c8200 legacy=0x7fd218072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:05.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.611+0000 7fd215e00640 1 -- 172.21.15.113:0/3484764615 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2181594d0 con 0x7fd2181527f0 2024-03-04T07:52:05.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.611+0000 7fd215e00640 1 --2- 172.21.15.113:0/3484764615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2181527f0 0x7fd218070d40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd208009e50 tx=0x7fd20801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2e85376f9e0b9b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:05.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.611+0000 7fd215400640 1 -- 172.21.15.113:0/3484764615 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080a5810 con 0x7fd2181527f0 2024-03-04T07:52:05.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.611+0000 7fd215400640 1 -- 172.21.15.113:0/3484764615 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2080a59b0 con 0x7fd2181527f0 2024-03-04T07:52:05.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.611+0000 7fd215400640 1 -- 172.21.15.113:0/3484764615 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080a5d80 con 0x7fd2181527f0 2024-03-04T07:52:05.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.611+0000 7fd21ca00640 1 -- 172.21.15.113:0/3484764615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2181527f0 msgr2=0x7fd218070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:05.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.611+0000 7fd21ca00640 1 --2- 172.21.15.113:0/3484764615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2181527f0 0x7fd218070d40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd208009e50 tx=0x7fd20801ca30 comp rx=0 tx=0).stop 2024-03-04T07:52:05.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 -- 172.21.15.113:0/3484764615 shutdown_connections 2024-03-04T07:52:05.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 --2- 172.21.15.113:0/3484764615 >> v2:172.21.15.136:3300/0 conn(0x7fd218152bd0 0x7fd2181587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:05.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 --2- 172.21.15.113:0/3484764615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2181527f0 0x7fd218070d40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:05.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 -- 172.21.15.113:0/3484764615 >> 172.21.15.113:0/3484764615 conn(0x7fd21806e820 msgr2=0x7fd21806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:05.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 -- 172.21.15.113:0/3484764615 shutdown_connections 2024-03-04T07:52:05.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 -- 172.21.15.113:0/3484764615 wait complete. 2024-03-04T07:52:05.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 Processor -- start 2024-03-04T07:52:05.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 -- start start 2024-03-04T07:52:05.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2180c8200 0x7fd2181f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:05.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2181527f0 0x7fd218147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:05.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd218152bd0 0x7fd2181534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:05.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2181654f0 con 0x7fd2181527f0 2024-03-04T07:52:05.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd218165370 con 0x7fd218152bd0 2024-03-04T07:52:05.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd21ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd218165670 con 0x7fd2180c8200 2024-03-04T07:52:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd20d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2181527f0 0x7fd218147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd215e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2180c8200 0x7fd2181f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd20d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2181527f0 0x7fd218147d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47580/0 (socket says 172.21.15.113:47580) 2024-03-04T07:52:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd20d400640 1 -- 172.21.15.113:0/1387453423 learned_addr learned my addr 172.21.15.113:0/1387453423 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd216800640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd218152bd0 0x7fd2181534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd20d400640 1 -- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2180c8200 msgr2=0x7fd2181f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd20d400640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2180c8200 0x7fd2181f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd20d400640 1 -- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd218152bd0 msgr2=0x7fd2181534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd20d400640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd218152bd0 0x7fd2181534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd20d400640 1 -- 172.21.15.113:0/1387453423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd218151110 con 0x7fd2181527f0 2024-03-04T07:52:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.615+0000 7fd20d400640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2181527f0 0x7fd218147d80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd20000c490 tx=0x7fd20000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.619+0000 7fd20ea00640 1 -- 172.21.15.113:0/1387453423 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2000052d0 con 0x7fd2181527f0 2024-03-04T07:52:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.619+0000 7fd20ea00640 1 -- 172.21.15.113:0/1387453423 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd200005470 con 0x7fd2181527f0 2024-03-04T07:52:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.619+0000 7fd21ca00640 1 -- 172.21.15.113:0/1387453423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd218152050 con 0x7fd2181527f0 2024-03-04T07:52:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.619+0000 7fd20ea00640 1 -- 172.21.15.113:0/1387453423 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200007260 con 0x7fd2181527f0 2024-03-04T07:52:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.619+0000 7fd21ca00640 1 -- 172.21.15.113:0/1387453423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd218150630 con 0x7fd2181527f0 2024-03-04T07:52:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.619+0000 7fd21ca00640 1 -- 172.21.15.113:0/1387453423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2180c9b50 con 0x7fd2181527f0 2024-03-04T07:52:05.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.619+0000 7fd20ea00640 1 -- 172.21.15.113:0/1387453423 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd200007400 con 0x7fd2181527f0 2024-03-04T07:52:05.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.619+0000 7fd20ea00640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1f4066f90 0x7fd1f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:05.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.623+0000 7fd20ea00640 1 -- 172.21.15.113:0/1387453423 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fd2000847d0 con 0x7fd2181527f0 2024-03-04T07:52:05.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.623+0000 7fd215e00640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1f4066f90 0x7fd1f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:05.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.623+0000 7fd215e00640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1f4066f90 0x7fd1f4069450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd2080ae000 tx=0x7fd2080a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:05.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.627+0000 7fd20ea00640 1 -- 172.21.15.113:0/1387453423 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd200054e00 con 0x7fd2181527f0 2024-03-04T07:52:05.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.851+0000 7fd21ca00640 1 -- 172.21.15.113:0/1387453423 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd218075c00 con 0x7fd2181527f0 2024-03-04T07:52:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.851+0000 7fd20ea00640 1 -- 172.21.15.113:0/1387453423 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fd200058ea0 con 0x7fd2181527f0 2024-03-04T07:52:05.849 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:05.849 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:06.860683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631,5529]},"id":1}]} 2024-03-04T07:52:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 70 2024-03-04T07:52:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 -- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1f4066f90 msgr2=0x7fd1f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1f4066f90 0x7fd1f4069450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd2080ae000 tx=0x7fd2080a7070 comp rx=0 tx=0).stop 2024-03-04T07:52:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 -- 172.21.15.113:0/1387453423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2181527f0 msgr2=0x7fd218147d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2181527f0 0x7fd218147d80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd20000c490 tx=0x7fd20000c950 comp rx=0 tx=0).stop 2024-03-04T07:52:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 -- 172.21.15.113:0/1387453423 shutdown_connections 2024-03-04T07:52:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd218152bd0 0x7fd2181534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1f4066f90 0x7fd1f4069450 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2181527f0 0x7fd218147d80 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 --2- 172.21.15.113:0/1387453423 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2180c8200 0x7fd2181f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:05.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 -- 172.21.15.113:0/1387453423 >> 172.21.15.113:0/1387453423 conn(0x7fd21806e820 msgr2=0x7fd218072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:05.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 -- 172.21.15.113:0/1387453423 shutdown_connections 2024-03-04T07:52:05.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:05.855+0000 7fd21ca00640 1 -- 172.21.15.113:0/1387453423 wait complete. 2024-03-04T07:52:05.877 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:52:07.878 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:08.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e99200640 1 Processor -- start 2024-03-04T07:52:08.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e99200640 1 -- start start 2024-03-04T07:52:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e99200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9e9414e4c0 0x7f9e94072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e99200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c22e0 0x7f9e940731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e99200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e940c27f0 con 0x7f9e940c2d80 2024-03-04T07:52:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e99200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9415ad60 con 0x7f9e9414e4c0 2024-03-04T07:52:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e99200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940c2a90 con 0x7f9e940c22e0 2024-03-04T07:52:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e92800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9e940c2d80 0x7f9e9415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59942/0 (socket says 172.21.15.113:59942) 2024-03-04T07:52:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e92800640 1 -- 172.21.15.113:0/428145686 learned_addr learned my addr 172.21.15.113:0/428145686 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:52:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e91400640 1 --2- 172.21.15.113:0/428145686 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c22e0 0x7f9e940731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:08.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e91e00640 1 --2- 172.21.15.113:0/428145686 >> v2:172.21.15.136:3300/0 conn(0x7f9e9414e4c0 0x7f9e94072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:08.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e90a00640 1 -- 172.21.15.113:0/428145686 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277449875 0 0) 0x7f9e940c27f0 con 0x7f9e940c2d80 2024-03-04T07:52:08.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e90a00640 1 -- 172.21.15.113:0/428145686 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e78005740 con 0x7f9e940c2d80 2024-03-04T07:52:08.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e91400640 1 -- 172.21.15.113:0/428145686 >> v2:172.21.15.136:3300/0 conn(0x7f9e9414e4c0 msgr2=0x7f9e94072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:08.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e91400640 1 --2- 172.21.15.113:0/428145686 >> v2:172.21.15.136:3300/0 conn(0x7f9e9414e4c0 0x7f9e94072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:08.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e91400640 1 -- 172.21.15.113:0/428145686 >> v1:172.21.15.113:6789/0 conn(0x7f9e940c2d80 legacy=0x7f9e9415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:08.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.079+0000 7f9e91400640 1 -- 172.21.15.113:0/428145686 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e94070c60 con 0x7f9e940c22e0 2024-03-04T07:52:08.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e91400640 1 --2- 172.21.15.113:0/428145686 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c22e0 0x7f9e940731c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9e800097c0 tx=0x7f9e8001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198ee0df005f7954 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e90a00640 1 -- 172.21.15.113:0/428145686 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e800b5070 con 0x7f9e940c22e0 2024-03-04T07:52:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e90a00640 1 -- 172.21.15.113:0/428145686 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e800a8990 con 0x7f9e940c22e0 2024-03-04T07:52:08.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e90a00640 1 -- 172.21.15.113:0/428145686 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e800a8d40 con 0x7f9e940c22e0 2024-03-04T07:52:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e99200640 1 -- 172.21.15.113:0/428145686 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c22e0 msgr2=0x7f9e940731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e99200640 1 --2- 172.21.15.113:0/428145686 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c22e0 0x7f9e940731c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9e800097c0 tx=0x7f9e8001ec90 comp rx=0 tx=0).stop 2024-03-04T07:52:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e99200640 1 -- 172.21.15.113:0/428145686 shutdown_connections 2024-03-04T07:52:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e99200640 1 --2- 172.21.15.113:0/428145686 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c22e0 0x7f9e940731c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e99200640 1 --2- 172.21.15.113:0/428145686 >> v2:172.21.15.136:3300/0 conn(0x7f9e9414e4c0 0x7f9e94072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e99200640 1 -- 172.21.15.113:0/428145686 >> 172.21.15.113:0/428145686 conn(0x7f9e9406e820 msgr2=0x7f9e9406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e99200640 1 -- 172.21.15.113:0/428145686 shutdown_connections 2024-03-04T07:52:08.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e99200640 1 -- 172.21.15.113:0/428145686 wait complete. 2024-03-04T07:52:08.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e99200640 1 Processor -- start 2024-03-04T07:52:08.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.087+0000 7f9e99200640 1 -- start start 2024-03-04T07:52:08.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e99200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e940c22e0 0x7f9e941f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:08.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e99200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e940c2d80 0x7f9e940cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:08.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e99200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e9414e4c0 0x7f9e9414a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:08.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e99200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e94071500 con 0x7f9e940c22e0 2024-03-04T07:52:08.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e99200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e94071220 con 0x7f9e940c2d80 2024-03-04T07:52:08.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e99200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9414e8a0 con 0x7f9e9414e4c0 2024-03-04T07:52:08.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e91e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e940c22e0 0x7f9e941f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:08.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e91e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e940c22e0 0x7f9e941f16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53578/0 (socket says 172.21.15.113:53578) 2024-03-04T07:52:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e91e00640 1 -- 172.21.15.113:0/463847082 learned_addr learned my addr 172.21.15.113:0/463847082 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e91400640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e940c2d80 0x7f9e940cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e92800640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e9414e4c0 0x7f9e9414a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e91e00640 1 -- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e9414e4c0 msgr2=0x7f9e9414a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e91e00640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e9414e4c0 0x7f9e9414a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e91e00640 1 -- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e940c2d80 msgr2=0x7f9e940cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e91e00640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e940c2d80 0x7f9e940cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e91e00640 1 -- 172.21.15.113:0/463847082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e941f0c70 con 0x7f9e940c22e0 2024-03-04T07:52:08.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e91e00640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e940c22e0 0x7f9e941f16f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400e930 tx=0x7f9e8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e8a000640 1 -- 172.21.15.113:0/463847082 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e84017070 con 0x7f9e940c22e0 2024-03-04T07:52:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e8a000640 1 -- 172.21.15.113:0/463847082 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8400ca00 con 0x7f9e940c22e0 2024-03-04T07:52:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e8a000640 1 -- 172.21.15.113:0/463847082 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400cdb0 con 0x7f9e940c22e0 2024-03-04T07:52:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e99200640 1 -- 172.21.15.113:0/463847082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e941e0400 con 0x7f9e940c22e0 2024-03-04T07:52:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e99200640 1 -- 172.21.15.113:0/463847082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e941e09c0 con 0x7f9e940c22e0 2024-03-04T07:52:08.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e99200640 1 -- 172.21.15.113:0/463847082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e94075c00 con 0x7f9e940c22e0 2024-03-04T07:52:08.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.091+0000 7f9e8a000640 1 -- 172.21.15.113:0/463847082 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e8401a070 con 0x7f9e940c22e0 2024-03-04T07:52:08.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.095+0000 7f9e8a000640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e74066f60 0x7f9e74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:08.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.095+0000 7f9e8a000640 1 -- 172.21.15.113:0/463847082 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7f9e840853d0 con 0x7f9e940c22e0 2024-03-04T07:52:08.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.095+0000 7f9e91400640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e74066f60 0x7f9e74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:08.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.095+0000 7f9e91400640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e74066f60 0x7f9e74069420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9e800b2000 tx=0x7f9e80008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:08.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.099+0000 7f9e8a000640 1 -- 172.21.15.113:0/463847082 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9e941f0c70 con 0x7f9e940c22e0 2024-03-04T07:52:08.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.295+0000 7f9e99200640 1 -- 172.21.15.113:0/463847082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e94155190 con 0x7f9e940c22e0 2024-03-04T07:52:08.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e8a000640 1 -- 172.21.15.113:0/463847082 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f9e84055a30 con 0x7f9e940c22e0 2024-03-04T07:52:08.294 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:08.294 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:50:06.860683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":5529,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5529":{"gid":5529,"name":"a","rank":1,"incarnation":45,"state":"up:active","state_seq":43,"addr":"172.21.15.113:6836/2299268553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2299268553},{"type":"v1","addr":"172.21.15.113:6836","nonce":2299268553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631,5529]},"id":1}]} 2024-03-04T07:52:08.294 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 70 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 -- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e74066f60 msgr2=0x7f9e74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e74066f60 0x7f9e74069420 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9e800b2000 tx=0x7f9e80008040 comp rx=0 tx=0).stop 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 -- 172.21.15.113:0/463847082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e940c22e0 msgr2=0x7f9e941f16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e940c22e0 0x7f9e941f16f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400e930 tx=0x7f9e8400edf0 comp rx=0 tx=0).stop 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 -- 172.21.15.113:0/463847082 shutdown_connections 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e9414e4c0 0x7f9e9414a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e940c2d80 0x7f9e940cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e74066f60 0x7f9e74069420 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 --2- 172.21.15.113:0/463847082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e940c22e0 0x7f9e941f16f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 -- 172.21.15.113:0/463847082 >> 172.21.15.113:0/463847082 conn(0x7f9e9406e820 msgr2=0x7f9e940c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 -- 172.21.15.113:0/463847082 shutdown_connections 2024-03-04T07:52:08.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:08.299+0000 7f9e99200640 1 -- 172.21.15.113:0/463847082 wait complete. 2024-03-04T07:52:08.315 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T07:52:10.316 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:10.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc55000640 1 Processor -- start 2024-03-04T07:52:10.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc55000640 1 -- start start 2024-03-04T07:52:10.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc55000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdc50152bd0 0x7fdc50070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc55000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc501591f0 con 0x7fdc501527f0 2024-03-04T07:52:10.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc55000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc501594d0 con 0x7fdc500c8200 2024-03-04T07:52:10.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc55000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc50071df0 con 0x7fdc50152bd0 2024-03-04T07:52:10.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc4de00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdc501527f0 0x7fdc50072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59948/0 (socket says 172.21.15.113:59948) 2024-03-04T07:52:10.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc4de00640 1 -- 172.21.15.113:0/3092028189 learned_addr learned my addr 172.21.15.113:0/3092028189 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:52:10.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc4e800640 1 --2- 172.21.15.113:0/3092028189 >> v2:172.21.15.113:3300/0 conn(0x7fdc50152bd0 0x7fdc50070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc4d400640 1 -- 172.21.15.113:0/3092028189 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362075110 0 0) 0x7fdc501591f0 con 0x7fdc501527f0 2024-03-04T07:52:10.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.507+0000 7fdc4d400640 1 -- 172.21.15.113:0/3092028189 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc34003530 con 0x7fdc501527f0 2024-03-04T07:52:10.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.511+0000 7fdc4e800640 1 -- 172.21.15.113:0/3092028189 >> v1:172.21.15.136:6789/0 conn(0x7fdc500c8200 legacy=0x7fdc501587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.511+0000 7fdc4e800640 1 -- 172.21.15.113:0/3092028189 >> v1:172.21.15.113:6789/0 conn(0x7fdc501527f0 legacy=0x7fdc50072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.511+0000 7fdc4e800640 1 -- 172.21.15.113:0/3092028189 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc500717a0 con 0x7fdc50152bd0 2024-03-04T07:52:10.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc4e800640 1 --2- 172.21.15.113:0/3092028189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 0x7fdc50070d40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdc480097c0 tx=0x7fdc4801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ace3215c33d201a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:10.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc4d400640 1 -- 172.21.15.113:0/3092028189 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480b0070 con 0x7fdc50152bd0 2024-03-04T07:52:10.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc4d400640 1 -- 172.21.15.113:0/3092028189 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc480ab600 con 0x7fdc50152bd0 2024-03-04T07:52:10.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc4d400640 1 -- 172.21.15.113:0/3092028189 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480ab9d0 con 0x7fdc50152bd0 2024-03-04T07:52:10.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc55000640 1 -- 172.21.15.113:0/3092028189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 msgr2=0x7fdc50070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc55000640 1 --2- 172.21.15.113:0/3092028189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 0x7fdc50070d40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdc480097c0 tx=0x7fdc4801c4a0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc55000640 1 -- 172.21.15.113:0/3092028189 shutdown_connections 2024-03-04T07:52:10.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc55000640 1 --2- 172.21.15.113:0/3092028189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 0x7fdc50070d40 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc55000640 1 -- 172.21.15.113:0/3092028189 >> 172.21.15.113:0/3092028189 conn(0x7fdc5006e820 msgr2=0x7fdc5006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:10.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc55000640 1 -- 172.21.15.113:0/3092028189 shutdown_connections 2024-03-04T07:52:10.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc55000640 1 -- 172.21.15.113:0/3092028189 wait complete. 2024-03-04T07:52:10.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc55000640 1 Processor -- start 2024-03-04T07:52:10.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.515+0000 7fdc55000640 1 -- start start 2024-03-04T07:52:10.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc55000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc500c8200 0x7fdc50161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc55000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc501527f0 0x7fdc500c1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc55000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 0x7fdc500c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc55000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50166500 con 0x7fdc50152bd0 2024-03-04T07:52:10.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc55000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50166380 con 0x7fdc500c8200 2024-03-04T07:52:10.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc55000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50166680 con 0x7fdc501527f0 2024-03-04T07:52:10.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 0x7fdc500c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 0x7fdc500c16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53586/0 (socket says 172.21.15.113:53586) 2024-03-04T07:52:10.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4e800640 1 -- 172.21.15.113:0/3555762472 learned_addr learned my addr 172.21.15.113:0/3555762472 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:10.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4de00640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc500c8200 0x7fdc50161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc45400640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc501527f0 0x7fdc500c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4e800640 1 -- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc501527f0 msgr2=0x7fdc500c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4e800640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc501527f0 0x7fdc500c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4e800640 1 -- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc500c8200 msgr2=0x7fdc50161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4e800640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc500c8200 0x7fdc50161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4e800640 1 -- 172.21.15.113:0/3555762472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc50152580 con 0x7fdc50152bd0 2024-03-04T07:52:10.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4de00640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc500c8200 0x7fdc50161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:52:10.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc4e800640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 0x7fdc500c16d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdc480190b0 tx=0x7fdc48019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:10.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc46a00640 1 -- 172.21.15.113:0/3555762472 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480b0040 con 0x7fdc50152bd0 2024-03-04T07:52:10.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc46a00640 1 -- 172.21.15.113:0/3555762472 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc480b9070 con 0x7fdc50152bd0 2024-03-04T07:52:10.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc55000640 1 -- 172.21.15.113:0/3555762472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc501f38a0 con 0x7fdc50152bd0 2024-03-04T07:52:10.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc46a00640 1 -- 172.21.15.113:0/3555762472 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480b4650 con 0x7fdc50152bd0 2024-03-04T07:52:10.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc55000640 1 -- 172.21.15.113:0/3555762472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc501f3b60 con 0x7fdc50152bd0 2024-03-04T07:52:10.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.519+0000 7fdc55000640 1 -- 172.21.15.113:0/3555762472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc50075c00 con 0x7fdc50152bd0 2024-03-04T07:52:10.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.523+0000 7fdc46a00640 1 -- 172.21.15.113:0/3555762472 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc480b47f0 con 0x7fdc50152bd0 2024-03-04T07:52:10.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.523+0000 7fdc46a00640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc2c066f60 0x7fdc2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.523+0000 7fdc46a00640 1 -- 172.21.15.113:0/3555762472 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6599+0+0 (secure 0 0 0) 0x7fdc4811efa0 con 0x7fdc50152bd0 2024-03-04T07:52:10.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.523+0000 7fdc4de00640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc2c066f60 0x7fdc2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.523+0000 7fdc4de00640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc2c066f60 0x7fdc2c069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdc40001570 tx=0x7fdc40009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:10.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.527+0000 7fdc46a00640 1 -- 172.21.15.113:0/3555762472 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdc48016090 con 0x7fdc50152bd0 2024-03-04T07:52:10.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 -- 172.21.15.113:0/3555762472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc5006ee50 con 0x7fdc50152bd0 2024-03-04T07:52:10.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc46a00640 1 -- 172.21.15.113:0/3555762472 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+5812 (secure 0 0 0) 0x7fdc480efc20 con 0x7fdc50152bd0 2024-03-04T07:52:10.734 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:10.734 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:10.722548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5631,5940]},"id":1}]} 2024-03-04T07:52:10.735 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 71 2024-03-04T07:52:10.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 -- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc2c066f60 msgr2=0x7fdc2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc2c066f60 0x7fdc2c069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdc40001570 tx=0x7fdc40009040 comp rx=0 tx=0).stop 2024-03-04T07:52:10.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 -- 172.21.15.113:0/3555762472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 msgr2=0x7fdc500c16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 0x7fdc500c16d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdc480190b0 tx=0x7fdc48019240 comp rx=0 tx=0).stop 2024-03-04T07:52:10.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 -- 172.21.15.113:0/3555762472 shutdown_connections 2024-03-04T07:52:10.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc2c066f60 0x7fdc2c069420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc50152bd0 0x7fdc500c16d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc501527f0 0x7fdc500c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 --2- 172.21.15.113:0/3555762472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc500c8200 0x7fdc50161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.739+0000 7fdc55000640 1 -- 172.21.15.113:0/3555762472 >> 172.21.15.113:0/3555762472 conn(0x7fdc5006e820 msgr2=0x7fdc50072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:10.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.743+0000 7fdc55000640 1 -- 172.21.15.113:0/3555762472 shutdown_connections 2024-03-04T07:52:10.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.743+0000 7fdc55000640 1 -- 172.21.15.113:0/3555762472 wait complete. 2024-03-04T07:52:10.764 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-03-04T07:52:10.764 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:52:10.764 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:10.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.907+0000 7f0c0de00640 1 Processor -- start 2024-03-04T07:52:10.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.907+0000 7f0c0de00640 1 -- start start 2024-03-04T07:52:10.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c0de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0c08152d60 0x7f0c080ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c0de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c08070120 0x7f0c080ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c0de00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c080c5380 con 0x7f0c080c9db0 2024-03-04T07:52:10.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c0de00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c080c3440 con 0x7f0c08152d60 2024-03-04T07:52:10.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c0de00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0806f620 con 0x7f0c08070120 2024-03-04T07:52:10.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c06a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0c08152d60 0x7f0c080ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c06a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0c08152d60 0x7f0c080ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53600/0 (socket says 172.21.15.113:53600) 2024-03-04T07:52:10.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c06a00640 1 -- 172.21.15.113:0/3289214495 learned_addr learned my addr 172.21.15.113:0/3289214495 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:10.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0bfe000640 1 --2- 172.21.15.113:0/3289214495 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c08070120 0x7f0c080ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c06a00640 1 -- 172.21.15.113:0/3289214495 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c08070120 msgr2=0x7f0c080ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:10.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c06a00640 1 --2- 172.21.15.113:0/3289214495 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c08070120 0x7f0c080ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c06a00640 1 -- 172.21.15.113:0/3289214495 >> v1:172.21.15.136:6789/0 conn(0x7f0c080c9db0 legacy=0x7f0c08158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0c06a00640 1 -- 172.21.15.113:0/3289214495 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c080c36e0 con 0x7f0c08152d60 2024-03-04T07:52:10.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.911+0000 7f0bfe000640 1 --2- 172.21.15.113:0/3289214495 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c08070120 0x7f0c080ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:52:10.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c06a00640 1 --2- 172.21.15.113:0/3289214495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08152d60 0x7f0c080ce310 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009870 tx=0x7f0bf801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b170b231f75a02 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:10.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c06000640 1 -- 172.21.15.113:0/3289214495 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80b5070 con 0x7f0c08152d60 2024-03-04T07:52:10.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c06000640 1 -- 172.21.15.113:0/3289214495 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf80a8a50 con 0x7f0c08152d60 2024-03-04T07:52:10.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c06000640 1 -- 172.21.15.113:0/3289214495 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80a8e20 con 0x7f0c08152d60 2024-03-04T07:52:10.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c0de00640 1 -- 172.21.15.113:0/3289214495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08152d60 msgr2=0x7f0c080ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c0de00640 1 --2- 172.21.15.113:0/3289214495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08152d60 0x7f0c080ce310 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009870 tx=0x7f0bf801ed50 comp rx=0 tx=0).stop 2024-03-04T07:52:10.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c0de00640 1 -- 172.21.15.113:0/3289214495 shutdown_connections 2024-03-04T07:52:10.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c0de00640 1 --2- 172.21.15.113:0/3289214495 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c08070120 0x7f0c080ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c0de00640 1 --2- 172.21.15.113:0/3289214495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08152d60 0x7f0c080ce310 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c0de00640 1 -- 172.21.15.113:0/3289214495 >> 172.21.15.113:0/3289214495 conn(0x7f0c08066db0 msgr2=0x7f0c080671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:10.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c0de00640 1 -- 172.21.15.113:0/3289214495 shutdown_connections 2024-03-04T07:52:10.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.915+0000 7f0c0de00640 1 -- 172.21.15.113:0/3289214495 wait complete. 2024-03-04T07:52:10.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c0de00640 1 Processor -- start 2024-03-04T07:52:10.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c0de00640 1 -- start start 2024-03-04T07:52:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c0de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08070120 0x7f0c0815a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c0de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c080c9db0 0x7f0c081ee930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c0de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08152d60 0x7f0c08148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c0de00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080c39c0 con 0x7f0c08070120 2024-03-04T07:52:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c0de00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080c5020 con 0x7f0c08152d60 2024-03-04T07:52:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c0de00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080c3b40 con 0x7f0c080c9db0 2024-03-04T07:52:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c06a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08070120 0x7f0c0815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c06a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08070120 0x7f0c0815a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53606/0 (socket says 172.21.15.113:53606) 2024-03-04T07:52:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c06a00640 1 -- 172.21.15.113:0/1994945045 learned_addr learned my addr 172.21.15.113:0/1994945045 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0bfe000640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c080c9db0 0x7f0c081ee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c07400640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08152d60 0x7f0c08148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c06a00640 1 -- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c080c9db0 msgr2=0x7f0c081ee930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c06a00640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c080c9db0 0x7f0c081ee930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c06a00640 1 -- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08152d60 msgr2=0x7f0c08148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c06a00640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08152d60 0x7f0c08148170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c06a00640 1 -- 172.21.15.113:0/1994945045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0815bcf0 con 0x7f0c08070120 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c06a00640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08070120 0x7f0c0815a370 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80b3040 tx=0x7f0bf801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0bffe00640 1 -- 172.21.15.113:0/1994945045 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80c0070 con 0x7f0c08070120 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0bffe00640 1 -- 172.21.15.113:0/1994945045 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf80ae550 con 0x7f0c08070120 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c0de00640 1 -- 172.21.15.113:0/1994945045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0815bfb0 con 0x7f0c08070120 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0bffe00640 1 -- 172.21.15.113:0/1994945045 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80b5070 con 0x7f0c08070120 2024-03-04T07:52:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.919+0000 7f0c0de00640 1 -- 172.21.15.113:0/1994945045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0815c570 con 0x7f0c08070120 2024-03-04T07:52:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.923+0000 7f0c0de00640 1 -- 172.21.15.113:0/1994945045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c08075c00 con 0x7f0c08070120 2024-03-04T07:52:10.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.923+0000 7f0bffe00640 1 -- 172.21.15.113:0/1994945045 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bf80c3070 con 0x7f0c08070120 2024-03-04T07:52:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.923+0000 7f0bffe00640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0be4066f90 0x7f0be4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.923+0000 7f0bffe00640 1 -- 172.21.15.113:0/1994945045 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f0bf8122b60 con 0x7f0c08070120 2024-03-04T07:52:10.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.923+0000 7f0bfe000640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0be4066f90 0x7f0be4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:10.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.923+0000 7f0bfe000640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0be4066f90 0x7f0be4069450 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0c081efa60 tx=0x7f0bf00097f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:10.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:10.931+0000 7f0bffe00640 1 -- 172.21.15.113:0/1994945045 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0bf80188b0 con 0x7f0c08070120 2024-03-04T07:52:11.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.215+0000 7f0c0de00640 1 -- 172.21.15.113:0/1994945045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c0806ff00 con 0x7f0c08070120 2024-03-04T07:52:11.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.215+0000 7f0bffe00640 1 -- 172.21.15.113:0/1994945045 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f0bf80f3130 con 0x7f0c08070120 2024-03-04T07:52:11.212 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:11.213 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:10.722548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5631,5940]},"id":1}]} 2024-03-04T07:52:11.213 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 71 2024-03-04T07:52:11.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 -- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0be4066f90 msgr2=0x7f0be4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:11.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0be4066f90 0x7f0be4069450 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0c081efa60 tx=0x7f0bf00097f0 comp rx=0 tx=0).stop 2024-03-04T07:52:11.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 -- 172.21.15.113:0/1994945045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08070120 msgr2=0x7f0c0815a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:11.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08070120 0x7f0c0815a370 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80b3040 tx=0x7f0bf801baf0 comp rx=0 tx=0).stop 2024-03-04T07:52:11.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 -- 172.21.15.113:0/1994945045 shutdown_connections 2024-03-04T07:52:11.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c08152d60 0x7f0c08148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:11.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c080c9db0 0x7f0c081ee930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:11.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0be4066f90 0x7f0be4069450 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:11.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 --2- 172.21.15.113:0/1994945045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c08070120 0x7f0c0815a370 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:11.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 -- 172.21.15.113:0/1994945045 >> 172.21.15.113:0/1994945045 conn(0x7f0c08066db0 msgr2=0x7f0c080b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:11.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 -- 172.21.15.113:0/1994945045 shutdown_connections 2024-03-04T07:52:11.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:11.219+0000 7f0c0de00640 1 -- 172.21.15.113:0/1994945045 wait complete. 2024-03-04T07:52:11.234 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 71, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 71, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5631": { "addr": "172.21.15.113:6833/810789479", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 810789479, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 810789479, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5631, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 17 }, "gid_5940": { "addr": "172.21.15.113:6837/344022786", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 344022786, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 344022786, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5940, "incarnation": 64, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 30 }, "gid_6145": { "addr": "172.21.15.136:6835/2368734308", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2368734308, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2368734308, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6145, "incarnation": 71, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 58, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:52:10.722548+0000", "qdb_cluster": [ 5631, 5940 ], "qdb_leader": 5940, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5940, "mds_1": 6145, "mds_2": 5631 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:52:13.235 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:13.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b86200640 1 Processor -- start 2024-03-04T07:52:13.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b86200640 1 -- start start 2024-03-04T07:52:13.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b86200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b80150da0 0x7f4b80151250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:13.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b86200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b80071b90 con 0x7f4b800ccce0 2024-03-04T07:52:13.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b86200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8015b400 con 0x7f4b800cc900 2024-03-04T07:52:13.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b86200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b80071e30 con 0x7f4b80150da0 2024-03-04T07:52:13.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b77e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4b800ccce0 0x7f4b800722c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59964/0 (socket says 172.21.15.113:59964) 2024-03-04T07:52:13.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b77e00640 1 -- 172.21.15.113:0/288519315 learned_addr learned my addr 172.21.15.113:0/288519315 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:52:13.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b7fe00640 1 --2- 172.21.15.113:0/288519315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b80150da0 0x7f4b80151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:13.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b7ea00640 1 -- 172.21.15.113:0/288519315 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425496887 0 0) 0x7f4b80071b90 con 0x7f4b800ccce0 2024-03-04T07:52:13.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b7ea00640 1 -- 172.21.15.113:0/288519315 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b64003530 con 0x7f4b800ccce0 2024-03-04T07:52:13.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b7ea00640 1 -- 172.21.15.113:0/288519315 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552484630 0 0) 0x7f4b8015b400 con 0x7f4b800cc900 2024-03-04T07:52:13.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b7ea00640 1 -- 172.21.15.113:0/288519315 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b80071b90 con 0x7f4b800cc900 2024-03-04T07:52:13.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b7fe00640 1 -- 172.21.15.113:0/288519315 >> v1:172.21.15.136:6789/0 conn(0x7f4b800cc900 legacy=0x7f4b801579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:13.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b7fe00640 1 -- 172.21.15.113:0/288519315 >> v1:172.21.15.113:6789/0 conn(0x7f4b800ccce0 legacy=0x7f4b800722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:13.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.375+0000 7f4b7fe00640 1 -- 172.21.15.113:0/288519315 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8015a660 con 0x7f4b80150da0 2024-03-04T07:52:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b7fe00640 1 --2- 172.21.15.113:0/288519315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b80150da0 0x7f4b80151250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800aed0 tx=0x7f4b7801f190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19f116bdde54ee3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:13.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b7ea00640 1 -- 172.21.15.113:0/288519315 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780b4070 con 0x7f4b80150da0 2024-03-04T07:52:13.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b7ea00640 1 -- 172.21.15.113:0/288519315 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b78008440 con 0x7f4b80150da0 2024-03-04T07:52:13.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b7ea00640 1 -- 172.21.15.113:0/288519315 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780087f0 con 0x7f4b80150da0 2024-03-04T07:52:13.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b86200640 1 -- 172.21.15.113:0/288519315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b80150da0 msgr2=0x7f4b80151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:13.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b86200640 1 --2- 172.21.15.113:0/288519315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b80150da0 0x7f4b80151250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800aed0 tx=0x7f4b7801f190 comp rx=0 tx=0).stop 2024-03-04T07:52:13.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b86200640 1 -- 172.21.15.113:0/288519315 shutdown_connections 2024-03-04T07:52:13.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b86200640 1 --2- 172.21.15.113:0/288519315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b80150da0 0x7f4b80151250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:13.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b86200640 1 -- 172.21.15.113:0/288519315 >> 172.21.15.113:0/288519315 conn(0x7f4b8006b060 msgr2=0x7f4b8006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:13.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b86200640 1 -- 172.21.15.113:0/288519315 shutdown_connections 2024-03-04T07:52:13.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.379+0000 7f4b86200640 1 -- 172.21.15.113:0/288519315 wait complete. 2024-03-04T07:52:13.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.383+0000 7f4b86200640 1 Processor -- start 2024-03-04T07:52:13.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.383+0000 7f4b86200640 1 -- start start 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b86200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b800cc900 0x7f4b800cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b86200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b800ccce0 0x7f4b8006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b86200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b80150da0 0x7f4b8014e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b86200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8015af00 con 0x7f4b800ccce0 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b86200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8015ac20 con 0x7f4b80150da0 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b86200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b800cd150 con 0x7f4b800cc900 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b800ccce0 0x7f4b8006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b800ccce0 0x7f4b8006f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53618/0 (socket says 172.21.15.113:53618) 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77e00640 1 -- 172.21.15.113:0/1002074609 learned_addr learned my addr 172.21.15.113:0/1002074609 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b7f400640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b800cc900 0x7f4b800cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b7fe00640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b80150da0 0x7f4b8014e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:13.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77e00640 1 -- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b800cc900 msgr2=0x7f4b800cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:13.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77e00640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b800cc900 0x7f4b800cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:13.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77e00640 1 -- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b80150da0 msgr2=0x7f4b8014e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:13.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77e00640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b80150da0 0x7f4b8014e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:13.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77e00640 1 -- 172.21.15.113:0/1002074609 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80156d30 con 0x7f4b800ccce0 2024-03-04T07:52:13.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77e00640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b800ccce0 0x7f4b8006f1d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c450 tx=0x7f4b6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:13.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77400640 1 -- 172.21.15.113:0/1002074609 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68017070 con 0x7f4b800ccce0 2024-03-04T07:52:13.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77400640 1 -- 172.21.15.113:0/1002074609 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68005cf0 con 0x7f4b800ccce0 2024-03-04T07:52:13.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77400640 1 -- 172.21.15.113:0/1002074609 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68007270 con 0x7f4b800ccce0 2024-03-04T07:52:13.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b86200640 1 -- 172.21.15.113:0/1002074609 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b801548d0 con 0x7f4b800ccce0 2024-03-04T07:52:13.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b86200640 1 -- 172.21.15.113:0/1002074609 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8014e350 con 0x7f4b800ccce0 2024-03-04T07:52:13.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b86200640 1 -- 172.21.15.113:0/1002074609 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b80075c00 con 0x7f4b800ccce0 2024-03-04T07:52:13.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.387+0000 7f4b77400640 1 -- 172.21.15.113:0/1002074609 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b6801a070 con 0x7f4b800ccce0 2024-03-04T07:52:13.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.391+0000 7f4b77400640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b5c066f60 0x7f4b5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:13.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.391+0000 7f4b77400640 1 -- 172.21.15.113:0/1002074609 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f4b68084600 con 0x7f4b800ccce0 2024-03-04T07:52:13.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.391+0000 7f4b7f400640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b5c066f60 0x7f4b5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:13.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.391+0000 7f4b7f400640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b5c066f60 0x7f4b5c069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b700011f0 tx=0x7f4b70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:13.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.395+0000 7f4b77400640 1 -- 172.21.15.113:0/1002074609 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4b68054be0 con 0x7f4b800ccce0 2024-03-04T07:52:13.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.667+0000 7f4b86200640 1 -- 172.21.15.113:0/1002074609 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b800ca770 con 0x7f4b800ccce0 2024-03-04T07:52:13.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.667+0000 7f4b77400640 1 -- 172.21.15.113:0/1002074609 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f4b68058c80 con 0x7f4b800ccce0 2024-03-04T07:52:13.664 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:13.664 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:10.722548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5631,5940]},"id":1}]} 2024-03-04T07:52:13.664 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 71 2024-03-04T07:52:13.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 -- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b5c066f60 msgr2=0x7f4b5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:13.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b5c066f60 0x7f4b5c069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b700011f0 tx=0x7f4b70009040 comp rx=0 tx=0).stop 2024-03-04T07:52:13.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 -- 172.21.15.113:0/1002074609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b800ccce0 msgr2=0x7f4b8006f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:13.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b800ccce0 0x7f4b8006f1d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c450 tx=0x7f4b6800c910 comp rx=0 tx=0).stop 2024-03-04T07:52:13.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 -- 172.21.15.113:0/1002074609 shutdown_connections 2024-03-04T07:52:13.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b80150da0 0x7f4b8014e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:13.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4b5c066f60 0x7f4b5c069420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:13.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b800ccce0 0x7f4b8006f1d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:13.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 --2- 172.21.15.113:0/1002074609 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b800cc900 0x7f4b800cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:13.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 -- 172.21.15.113:0/1002074609 >> 172.21.15.113:0/1002074609 conn(0x7f4b8006b060 msgr2=0x7f4b80066eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:13.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 -- 172.21.15.113:0/1002074609 shutdown_connections 2024-03-04T07:52:13.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:13.671+0000 7f4b86200640 1 -- 172.21.15.113:0/1002074609 wait complete. 2024-03-04T07:52:13.692 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:52:15.693 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:15.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1113200640 1 Processor -- start 2024-03-04T07:52:15.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1113200640 1 -- start start 2024-03-04T07:52:15.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1113200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1114151760 0x7f11140c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:15.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1113200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111406c290 con 0x7f11140c18d0 2024-03-04T07:52:15.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1113200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111406ef10 con 0x7f1114151b40 2024-03-04T07:52:15.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1113200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111415aef0 con 0x7f1114151760 2024-03-04T07:52:15.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1111e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1114151760 0x7f11140c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:15.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1111e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1114151760 0x7f11140c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53622/0 (socket says 172.21.15.113:53622) 2024-03-04T07:52:15.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1111e00640 1 -- 172.21.15.113:0/3161698856 learned_addr learned my addr 172.21.15.113:0/3161698856 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:15.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1111e00640 1 -- 172.21.15.113:0/3161698856 >> v1:172.21.15.136:6789/0 conn(0x7f1114151b40 legacy=0x7f111415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:15.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1111e00640 1 -- 172.21.15.113:0/3161698856 >> v1:172.21.15.113:6789/0 conn(0x7f11140c18d0 legacy=0x7f1114071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:15.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.863+0000 7f1111e00640 1 -- 172.21.15.113:0/3161698856 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111415ba90 con 0x7f1114151760 2024-03-04T07:52:15.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1111e00640 1 --2- 172.21.15.113:0/3161698856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1114151760 0x7f11140c83d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1104009870 tx=0x7f110401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f998b9614d27a33 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:15.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1111400640 1 -- 172.21.15.113:0/3161698856 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11040b1070 con 0x7f1114151760 2024-03-04T07:52:15.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1111400640 1 -- 172.21.15.113:0/3161698856 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11040a8050 con 0x7f1114151760 2024-03-04T07:52:15.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1111400640 1 -- 172.21.15.113:0/3161698856 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11040ac860 con 0x7f1114151760 2024-03-04T07:52:15.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1113200640 1 -- 172.21.15.113:0/3161698856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1114151760 msgr2=0x7f11140c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:15.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1113200640 1 --2- 172.21.15.113:0/3161698856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1114151760 0x7f11140c83d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1104009870 tx=0x7f110401c4d0 comp rx=0 tx=0).stop 2024-03-04T07:52:15.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1113200640 1 -- 172.21.15.113:0/3161698856 shutdown_connections 2024-03-04T07:52:15.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1113200640 1 --2- 172.21.15.113:0/3161698856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1114151760 0x7f11140c83d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:15.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1113200640 1 -- 172.21.15.113:0/3161698856 >> 172.21.15.113:0/3161698856 conn(0x7f111406e2a0 msgr2=0x7f111406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:15.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1113200640 1 -- 172.21.15.113:0/3161698856 shutdown_connections 2024-03-04T07:52:15.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1113200640 1 -- 172.21.15.113:0/3161698856 wait complete. 2024-03-04T07:52:15.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1113200640 1 Processor -- start 2024-03-04T07:52:15.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.871+0000 7f1113200640 1 -- start start 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1113200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11140c18d0 0x7f11141f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1113200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1114151760 0x7f111414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1113200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1114151b40 0x7f11141407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1113200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11140c1e20 con 0x7f11140c18d0 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1113200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111415ab80 con 0x7f1114151760 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1113200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1114165740 con 0x7f1114151b40 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1111e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11140c18d0 0x7f11141f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1111e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11140c18d0 0x7f11141f80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53630/0 (socket says 172.21.15.113:53630) 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1111e00640 1 -- 172.21.15.113:0/1035754360 learned_addr learned my addr 172.21.15.113:0/1035754360 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f110be00640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1114151760 0x7f111414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1112800640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1114151b40 0x7f11141407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:15.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1111e00640 1 -- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1114151b40 msgr2=0x7f11141407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1111e00640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1114151b40 0x7f11141407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1111e00640 1 -- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1114151760 msgr2=0x7f111414ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1111e00640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1114151760 0x7f111414ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1111e00640 1 -- 172.21.15.113:0/1035754360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11141535f0 con 0x7f11140c18d0 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f110be00640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1114151760 0x7f111414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1111e00640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11140c18d0 0x7f11141f80a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f11040af040 tx=0x7f110401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f110a000640 1 -- 172.21.15.113:0/1035754360 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11040bb020 con 0x7f11140c18d0 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f110a000640 1 -- 172.21.15.113:0/1035754360 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11040b7420 con 0x7f11140c18d0 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1113200640 1 -- 172.21.15.113:0/1035754360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11141f85e0 con 0x7f11140c18d0 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f110a000640 1 -- 172.21.15.113:0/1035754360 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11040b1040 con 0x7f11140c18d0 2024-03-04T07:52:15.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1113200640 1 -- 172.21.15.113:0/1035754360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111415a6f0 con 0x7f11140c18d0 2024-03-04T07:52:15.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.875+0000 7f1113200640 1 -- 172.21.15.113:0/1035754360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1114075a80 con 0x7f11140c18d0 2024-03-04T07:52:15.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.879+0000 7f110a000640 1 -- 172.21.15.113:0/1035754360 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11040be020 con 0x7f11140c18d0 2024-03-04T07:52:15.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.879+0000 7f110a000640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f0066f90 0x7f10f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:15.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.879+0000 7f110a000640 1 -- 172.21.15.113:0/1035754360 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f110411f470 con 0x7f11140c18d0 2024-03-04T07:52:15.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.879+0000 7f110be00640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f0066f90 0x7f10f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:15.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.879+0000 7f110be00640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f0066f90 0x7f10f0069450 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1114151180 tx=0x7f10fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:15.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:15.883+0000 7f110a000640 1 -- 172.21.15.113:0/1035754360 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f11040c5070 con 0x7f11140c18d0 2024-03-04T07:52:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 -- 172.21.15.113:0/1035754360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1114070f20 con 0x7f11140c18d0 2024-03-04T07:52:16.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f110a000640 1 -- 172.21.15.113:0/1035754360 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+5814 (secure 0 0 0) 0x7f1104016150 con 0x7f11140c18d0 2024-03-04T07:52:16.175 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:16.175 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:15.736016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:resolve","state_seq":33,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631]},"id":1}]} 2024-03-04T07:52:16.175 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 72 2024-03-04T07:52:16.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 -- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f0066f90 msgr2=0x7f10f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:16.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f0066f90 0x7f10f0069450 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1114151180 tx=0x7f10fc009040 comp rx=0 tx=0).stop 2024-03-04T07:52:16.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 -- 172.21.15.113:0/1035754360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11140c18d0 msgr2=0x7f11141f80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:16.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11140c18d0 0x7f11141f80a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f11040af040 tx=0x7f110401ad10 comp rx=0 tx=0).stop 2024-03-04T07:52:16.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 -- 172.21.15.113:0/1035754360 shutdown_connections 2024-03-04T07:52:16.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1114151b40 0x7f11141407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:16.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1114151760 0x7f111414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:16.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f0066f90 0x7f10f0069450 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:16.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 --2- 172.21.15.113:0/1035754360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11140c18d0 0x7f11141f80a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:16.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 -- 172.21.15.113:0/1035754360 >> 172.21.15.113:0/1035754360 conn(0x7f111406e2a0 msgr2=0x7f11140c6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:16.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 -- 172.21.15.113:0/1035754360 shutdown_connections 2024-03-04T07:52:16.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:16.179+0000 7f1113200640 1 -- 172.21.15.113:0/1035754360 wait complete. 2024-03-04T07:52:16.196 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:52:18.197 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:18.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.375+0000 7fc257000640 1 Processor -- start 2024-03-04T07:52:18.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.375+0000 7fc257000640 1 -- start start 2024-03-04T07:52:18.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.375+0000 7fc257000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc250159040 0x7fc2501642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:18.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.375+0000 7fc257000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2500c5380 con 0x7fc250070120 2024-03-04T07:52:18.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.375+0000 7fc257000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2500c3440 con 0x7fc250152d60 2024-03-04T07:52:18.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.375+0000 7fc257000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25006f620 con 0x7fc250159040 2024-03-04T07:52:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.375+0000 7fc247400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc250070120 0x7fc250158930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47632/0 (socket says 172.21.15.113:47632) 2024-03-04T07:52:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.375+0000 7fc247400640 1 -- 172.21.15.113:0/2729749501 learned_addr learned my addr 172.21.15.113:0/2729749501 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:52:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.375+0000 7fc255000640 1 --2- 172.21.15.113:0/2729749501 >> v2:172.21.15.136:3300/0 conn(0x7fc250159040 0x7fc2501642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564245679 0 0) 0x7fc2500c5380 con 0x7fc250070120 2024-03-04T07:52:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc234003530 con 0x7fc250070120 2024-03-04T07:52:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 106656065 0 0) 0x7fc234003530 con 0x7fc250070120 2024-03-04T07:52:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 >> v2:172.21.15.136:3300/0 conn(0x7fc250159040 msgr2=0x7fc2501642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 --2- 172.21.15.113:0/2729749501 >> v2:172.21.15.136:3300/0 conn(0x7fc250159040 0x7fc2501642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:18.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 >> v1:172.21.15.136:6789/0 conn(0x7fc250152d60 legacy=0x7fc2500ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500c36e0 con 0x7fc250070120 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fc238001680 con 0x7fc250070120 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 >> v1:172.21.15.113:6789/0 conn(0x7fc250070120 legacy=0x7fc250158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2340039c0 0x7fc234003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc234004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2340051d0 0x7fc23400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25006f620 con 0x7fc234004810 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400c070 con 0x7fc2340051d0 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400c270 con 0x7fc2340039c0 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc247400640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc234004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc24fe00640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2340039c0 0x7fc234003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:18.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc255000640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2340051d0 0x7fc23400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc247400640 1 -- 172.21.15.113:0/2729749501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2340039c0 msgr2=0x7fc234003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc247400640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2340039c0 0x7fc234003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc247400640 1 -- 172.21.15.113:0/2729749501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2340051d0 msgr2=0x7fc23400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc247400640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2340051d0 0x7fc23400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc247400640 1 -- 172.21.15.113:0/2729749501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc234003530 con 0x7fc234004810 2024-03-04T07:52:18.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.379+0000 7fc255000640 1 -- 172.21.15.113:0/2729749501 reap_dead start 2024-03-04T07:52:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc247400640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc234004c90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c240 tx=0x7fc23801f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1301adfbda8493fe server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238007d20 con 0x7fc234004810 2024-03-04T07:52:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc238005ae0 con 0x7fc234004810 2024-03-04T07:52:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc24f400640 1 -- 172.21.15.113:0/2729749501 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238007070 con 0x7fc234004810 2024-03-04T07:52:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc257000640 1 -- 172.21.15.113:0/2729749501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 msgr2=0x7fc234004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc257000640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc234004c90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c240 tx=0x7fc23801f370 comp rx=0 tx=0).stop 2024-03-04T07:52:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc257000640 1 -- 172.21.15.113:0/2729749501 shutdown_connections 2024-03-04T07:52:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc257000640 1 --2- 172.21.15.113:0/2729749501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc234004c90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc257000640 1 -- 172.21.15.113:0/2729749501 >> 172.21.15.113:0/2729749501 conn(0x7fc250066db0 msgr2=0x7fc2500671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc257000640 1 -- 172.21.15.113:0/2729749501 shutdown_connections 2024-03-04T07:52:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.383+0000 7fc257000640 1 -- 172.21.15.113:0/2729749501 wait complete. 2024-03-04T07:52:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc257000640 1 Processor -- start 2024-03-04T07:52:18.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc257000640 1 -- start start 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc257000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc2501f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc257000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc250158370 0x7fc2501578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc257000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc25006d2c0 0x7fc25006d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc257000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500c39c0 con 0x7fc234004810 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc257000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500c5020 con 0x7fc250158370 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc257000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500c3b40 con 0x7fc25006d2c0 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc2501f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc2501f11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56260/0 (socket says 172.21.15.113:56260) 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24fe00640 1 -- 172.21.15.113:0/4038893498 learned_addr learned my addr 172.21.15.113:0/4038893498 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc247400640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc250158370 0x7fc2501578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc255000640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc25006d2c0 0x7fc25006d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24fe00640 1 -- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc25006d2c0 msgr2=0x7fc25006d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:18.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24fe00640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc25006d2c0 0x7fc25006d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24fe00640 1 -- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc250158370 msgr2=0x7fc2501578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24fe00640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc250158370 0x7fc2501578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24fe00640 1 -- 172.21.15.113:0/4038893498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc250158840 con 0x7fc234004810 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc255000640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc25006d2c0 0x7fc25006d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24fe00640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc2501f11f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc24000fb10 tx=0x7fc240002070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24cc00640 1 -- 172.21.15.113:0/4038893498 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240003b90 con 0x7fc234004810 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24cc00640 1 -- 172.21.15.113:0/4038893498 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc240018070 con 0x7fc234004810 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc24cc00640 1 -- 172.21.15.113:0/4038893498 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24000d7c0 con 0x7fc234004810 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc257000640 1 -- 172.21.15.113:0/4038893498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc250158b20 con 0x7fc234004810 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.387+0000 7fc257000640 1 -- 172.21.15.113:0/4038893498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25006c7e0 con 0x7fc234004810 2024-03-04T07:52:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.391+0000 7fc257000640 1 -- 172.21.15.113:0/4038893498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc250075c00 con 0x7fc234004810 2024-03-04T07:52:18.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.391+0000 7fc24cc00640 1 -- 172.21.15.113:0/4038893498 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc24000d960 con 0x7fc234004810 2024-03-04T07:52:18.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.391+0000 7fc24cc00640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc22c066f90 0x7fc22c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:18.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.391+0000 7fc24cc00640 1 -- 172.21.15.113:0/4038893498 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fc240085660 con 0x7fc234004810 2024-03-04T07:52:18.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.391+0000 7fc247400640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc22c066f90 0x7fc22c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:18.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.395+0000 7fc247400640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc22c066f90 0x7fc22c069450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc2380af000 tx=0x7fc2380aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:18.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.395+0000 7fc24cc00640 1 -- 172.21.15.113:0/4038893498 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc24001a310 con 0x7fc234004810 2024-03-04T07:52:18.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.615+0000 7fc257000640 1 -- 172.21.15.113:0/4038893498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc25015cd30 con 0x7fc234004810 2024-03-04T07:52:18.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.615+0000 7fc24cc00640 1 -- 172.21.15.113:0/4038893498 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+5814 (secure 0 0 0) 0x7fc240056170 con 0x7fc234004810 2024-03-04T07:52:18.613 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:18.613 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:15.736016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:resolve","state_seq":33,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631]},"id":1}]} 2024-03-04T07:52:18.613 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 72 2024-03-04T07:52:18.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 -- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc22c066f90 msgr2=0x7fc22c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:18.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc22c066f90 0x7fc22c069450 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc2380af000 tx=0x7fc2380aa000 comp rx=0 tx=0).stop 2024-03-04T07:52:18.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 -- 172.21.15.113:0/4038893498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 msgr2=0x7fc2501f11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:18.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc2501f11f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc24000fb10 tx=0x7fc240002070 comp rx=0 tx=0).stop 2024-03-04T07:52:18.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 -- 172.21.15.113:0/4038893498 shutdown_connections 2024-03-04T07:52:18.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc25006d2c0 0x7fc25006d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:18.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc250158370 0x7fc2501578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:18.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc22c066f90 0x7fc22c069450 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:18.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 --2- 172.21.15.113:0/4038893498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc234004810 0x7fc2501f11f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:18.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 -- 172.21.15.113:0/4038893498 >> 172.21.15.113:0/4038893498 conn(0x7fc250066db0 msgr2=0x7fc2500c93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:18.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 -- 172.21.15.113:0/4038893498 shutdown_connections 2024-03-04T07:52:18.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:18.619+0000 7fc257000640 1 -- 172.21.15.113:0/4038893498 wait complete. 2024-03-04T07:52:18.634 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:52:20.635 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:20.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.775+0000 7fa71fc00640 1 Processor -- start 2024-03-04T07:52:20.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.775+0000 7fa71fc00640 1 -- start start 2024-03-04T07:52:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.775+0000 7fa71fc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7180ce2d0 0x7fa71815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.775+0000 7fa71fc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71806c210 con 0x7fa71806de80 2024-03-04T07:52:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.775+0000 7fa71fc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71806c4f0 con 0x7fa7180ce6b0 2024-03-04T07:52:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.775+0000 7fa71fc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71815bbf0 con 0x7fa7180ce2d0 2024-03-04T07:52:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa71dc00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa71806de80 0x7fa718154b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47644/0 (socket says 172.21.15.113:47644) 2024-03-04T07:52:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa71dc00640 1 -- 172.21.15.113:0/3341557119 learned_addr learned my addr 172.21.15.113:0/3341557119 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:52:20.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa71d200640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7180ce2d0 0x7fa71815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:20.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88778549 0 0) 0x7fa71806c210 con 0x7fa71806de80 2024-03-04T07:52:20.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa700003530 con 0x7fa71806de80 2024-03-04T07:52:20.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828642647 0 0) 0x7fa71806c4f0 con 0x7fa7180ce6b0 2024-03-04T07:52:20.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa71806c210 con 0x7fa7180ce6b0 2024-03-04T07:52:20.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1822307565 0 0) 0x7fa700003530 con 0x7fa71806de80 2024-03-04T07:52:20.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7180ce2d0 msgr2=0x7fa71815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:20.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7180ce2d0 0x7fa71815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:20.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 >> v1:172.21.15.136:6789/0 conn(0x7fa7180ce6b0 legacy=0x7fa71806f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:20.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71806e530 con 0x7fa71806de80 2024-03-04T07:52:20.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fa710001a00 con 0x7fa71806de80 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 >> v1:172.21.15.113:6789/0 conn(0x7fa71806de80 legacy=0x7fa718154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7180ce2d0 0x7fa700003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa700004130 0x7fa700004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa700004d50 0x7fa7000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71815bbf0 con 0x7fa700004130 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70000bb10 con 0x7fa700004d50 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70000bd10 con 0x7fa7180ce2d0 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717e00640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa700004130 0x7fa700004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa71d200640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7180ce2d0 0x7fa700003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa71dc00640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa700004d50 0x7fa7000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717e00640 1 -- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7180ce2d0 msgr2=0x7fa700003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717e00640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa7180ce2d0 0x7fa700003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717e00640 1 -- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa700004d50 msgr2=0x7fa7000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717e00640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa700004d50 0x7fa7000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:20.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.779+0000 7fa717e00640 1 -- 172.21.15.113:0/3341557119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa700003530 con 0x7fa700004130 2024-03-04T07:52:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa717e00640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa700004130 0x7fa700004810 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa7180cead0 tx=0x7fa70401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32af5f5a64b749e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704001520 con 0x7fa700004130 2024-03-04T07:52:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7040a8050 con 0x7fa700004130 2024-03-04T07:52:20.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa717400640 1 -- 172.21.15.113:0/3341557119 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704003600 con 0x7fa700004130 2024-03-04T07:52:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa71fc00640 1 -- 172.21.15.113:0/3341557119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa700004130 msgr2=0x7fa700004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa71fc00640 1 --2- 172.21.15.113:0/3341557119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa700004130 0x7fa700004810 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa7180cead0 tx=0x7fa70401c4a0 comp rx=0 tx=0).stop 2024-03-04T07:52:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa71dc00640 1 -- 172.21.15.113:0/3341557119 reap_dead start 2024-03-04T07:52:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa71fc00640 1 -- 172.21.15.113:0/3341557119 shutdown_connections 2024-03-04T07:52:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa71fc00640 1 -- 172.21.15.113:0/3341557119 >> 172.21.15.113:0/3341557119 conn(0x7fa7180709a0 msgr2=0x7fa718070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa71fc00640 1 -- 172.21.15.113:0/3341557119 shutdown_connections 2024-03-04T07:52:20.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.783+0000 7fa71fc00640 1 -- 172.21.15.113:0/3341557119 wait complete. 2024-03-04T07:52:20.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa71fc00640 1 Processor -- start 2024-03-04T07:52:20.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa71fc00640 1 -- start start 2024-03-04T07:52:20.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa71fc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa700004130 0x7fa7180c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:20.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa71fc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa718156580 0x7fa7180c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:20.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa71fc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa718157e40 0x7fa71814f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:20.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa71fc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71806c7d0 con 0x7fa718156580 2024-03-04T07:52:20.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa71fc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71806eb80 con 0x7fa700004130 2024-03-04T07:52:20.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa71fc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71806f720 con 0x7fa718157e40 2024-03-04T07:52:20.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa717e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa718156580 0x7fa7180c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:20.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa717e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa718156580 0x7fa7180c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56286/0 (socket says 172.21.15.113:56286) 2024-03-04T07:52:20.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa717e00640 1 -- 172.21.15.113:0/1004674719 learned_addr learned my addr 172.21.15.113:0/1004674719 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:20.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa71d200640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa700004130 0x7fa7180c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:20.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa71dc00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa718157e40 0x7fa71814f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:20.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa717e00640 1 -- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa718157e40 msgr2=0x7fa71814f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:20.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa717e00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa718157e40 0x7fa71814f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:20.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa717e00640 1 -- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa700004130 msgr2=0x7fa7180c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:20.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa717e00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa700004130 0x7fa7180c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:20.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa717e00640 1 -- 172.21.15.113:0/1004674719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180c7cc0 con 0x7fa718156580 2024-03-04T07:52:20.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa717e00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa718156580 0x7fa7180c72c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa7040ae040 tx=0x7fa700005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:20.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.787+0000 7fa714c00640 1 -- 172.21.15.113:0/1004674719 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040bc020 con 0x7fa718156580 2024-03-04T07:52:20.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.791+0000 7fa714c00640 1 -- 172.21.15.113:0/1004674719 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa704003d40 con 0x7fa718156580 2024-03-04T07:52:20.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.791+0000 7fa714c00640 1 -- 172.21.15.113:0/1004674719 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040ac670 con 0x7fa718156580 2024-03-04T07:52:20.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.791+0000 7fa71fc00640 1 -- 172.21.15.113:0/1004674719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7180c84c0 con 0x7fa718156580 2024-03-04T07:52:20.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.791+0000 7fa71fc00640 1 -- 172.21.15.113:0/1004674719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7180c81e0 con 0x7fa718156580 2024-03-04T07:52:20.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.791+0000 7fa71fc00640 1 -- 172.21.15.113:0/1004674719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa718075c00 con 0x7fa718156580 2024-03-04T07:52:20.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.791+0000 7fa714c00640 1 -- 172.21.15.113:0/1004674719 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7040bf020 con 0x7fa718156580 2024-03-04T07:52:20.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.791+0000 7fa714c00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6f8066f60 0x7fa6f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:20.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.791+0000 7fa714c00640 1 -- 172.21.15.113:0/1004674719 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fa70411f890 con 0x7fa718156580 2024-03-04T07:52:20.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.791+0000 7fa71d200640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6f8066f60 0x7fa6f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:20.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.795+0000 7fa71d200640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6f8066f60 0x7fa6f8069420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009b00 tx=0x7fa70c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:20.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:20.799+0000 7fa714c00640 1 -- 172.21.15.113:0/1004674719 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa7040c6030 con 0x7fa718156580 2024-03-04T07:52:21.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.043+0000 7fa71fc00640 1 -- 172.21.15.113:0/1004674719 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa718158640 con 0x7fa718156580 2024-03-04T07:52:21.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.043+0000 7fa714c00640 1 -- 172.21.15.113:0/1004674719 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5816 (secure 0 0 0) 0x7fa7040161b0 con 0x7fa718156580 2024-03-04T07:52:21.039 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:21.039 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:20.792498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5631,5940]},"id":1}]} 2024-03-04T07:52:21.039 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 73 2024-03-04T07:52:21.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 -- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6f8066f60 msgr2=0x7fa6f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:21.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6f8066f60 0x7fa6f8069420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009b00 tx=0x7fa70c009190 comp rx=0 tx=0).stop 2024-03-04T07:52:21.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 -- 172.21.15.113:0/1004674719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa718156580 msgr2=0x7fa7180c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa718156580 0x7fa7180c72c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa7040ae040 tx=0x7fa700005840 comp rx=0 tx=0).stop 2024-03-04T07:52:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 -- 172.21.15.113:0/1004674719 shutdown_connections 2024-03-04T07:52:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa718157e40 0x7fa71814f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6f8066f60 0x7fa6f8069420 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa718156580 0x7fa7180c72c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 --2- 172.21.15.113:0/1004674719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa700004130 0x7fa7180c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 -- 172.21.15.113:0/1004674719 >> 172.21.15.113:0/1004674719 conn(0x7fa7180709a0 msgr2=0x7fa7180c5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 -- 172.21.15.113:0/1004674719 shutdown_connections 2024-03-04T07:52:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:21.047+0000 7fa71fc00640 1 -- 172.21.15.113:0/1004674719 wait complete. 2024-03-04T07:52:21.060 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:52:23.062 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:23.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.235+0000 7f5c56a00640 1 Processor -- start 2024-03-04T07:52:23.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.235+0000 7f5c56a00640 1 -- start start 2024-03-04T07:52:23.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.235+0000 7f5c56a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5c50152920 0x7f5c50071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:23.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.235+0000 7f5c56a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5006c580 0x7f5c500c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:23.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.235+0000 7f5c56a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5006bb10 con 0x7f5c5015bf20 2024-03-04T07:52:23.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.235+0000 7f5c56a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c500c7a40 con 0x7f5c50152920 2024-03-04T07:52:23.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.235+0000 7f5c56a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500c7d20 con 0x7f5c5006c580 2024-03-04T07:52:23.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.239+0000 7f5c56000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5c5015bf20 0x7f5c5015c3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47652/0 (socket says 172.21.15.113:47652) 2024-03-04T07:52:23.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.239+0000 7f5c56000640 1 -- 172.21.15.113:0/1687981054 learned_addr learned my addr 172.21.15.113:0/1687981054 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:52:23.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.239+0000 7f5c54c00640 1 --2- 172.21.15.113:0/1687981054 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5006c580 0x7f5c500c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:23.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.239+0000 7f5c55600640 1 --2- 172.21.15.113:0/1687981054 >> v2:172.21.15.136:3300/0 conn(0x7f5c50152920 0x7f5c50071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:23.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.239+0000 7f5c4fe00640 1 -- 172.21.15.113:0/1687981054 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049874822 0 0) 0x7f5c5006bb10 con 0x7f5c5015bf20 2024-03-04T07:52:23.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.239+0000 7f5c4fe00640 1 -- 172.21.15.113:0/1687981054 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c38003530 con 0x7f5c5015bf20 2024-03-04T07:52:23.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.239+0000 7f5c55600640 1 -- 172.21.15.113:0/1687981054 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5006c580 msgr2=0x7f5c500c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:23.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.239+0000 7f5c55600640 1 --2- 172.21.15.113:0/1687981054 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5006c580 0x7f5c500c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.239+0000 7f5c55600640 1 -- 172.21.15.113:0/1687981054 >> v1:172.21.15.113:6789/0 conn(0x7f5c5015bf20 legacy=0x7f5c5015c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:23.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.239+0000 7f5c55600640 1 -- 172.21.15.113:0/1687981054 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5006bd80 con 0x7f5c50152920 2024-03-04T07:52:23.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c55600640 1 --2- 172.21.15.113:0/1687981054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c50152920 0x7f5c50071a30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097c0 tx=0x7f5c4401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71c27393635ba012 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:23.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c4fe00640 1 -- 172.21.15.113:0/1687981054 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440ab600 con 0x7f5c50152920 2024-03-04T07:52:23.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c4fe00640 1 -- 172.21.15.113:0/1687981054 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c440b0070 con 0x7f5c50152920 2024-03-04T07:52:23.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c4fe00640 1 -- 172.21.15.113:0/1687981054 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440ab9a0 con 0x7f5c50152920 2024-03-04T07:52:23.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c56a00640 1 -- 172.21.15.113:0/1687981054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c50152920 msgr2=0x7f5c50071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:23.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c56a00640 1 --2- 172.21.15.113:0/1687981054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c50152920 0x7f5c50071a30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5c440097c0 tx=0x7f5c4401c3e0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c56a00640 1 -- 172.21.15.113:0/1687981054 shutdown_connections 2024-03-04T07:52:23.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c56a00640 1 --2- 172.21.15.113:0/1687981054 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5006c580 0x7f5c500c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c56a00640 1 --2- 172.21.15.113:0/1687981054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c50152920 0x7f5c50071a30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c56a00640 1 -- 172.21.15.113:0/1687981054 >> 172.21.15.113:0/1687981054 conn(0x7f5c500c9a30 msgr2=0x7f5c500c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:23.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c56a00640 1 -- 172.21.15.113:0/1687981054 shutdown_connections 2024-03-04T07:52:23.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c56a00640 1 -- 172.21.15.113:0/1687981054 wait complete. 2024-03-04T07:52:23.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c56a00640 1 Processor -- start 2024-03-04T07:52:23.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.243+0000 7f5c56a00640 1 -- start start 2024-03-04T07:52:23.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c5006c580 0x7f5c5014fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:23.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c50152920 0x7f5c50157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:23.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5015bf20 0x7f5c500c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:23.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500c95a0 con 0x7f5c50152920 2024-03-04T07:52:23.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50152d00 con 0x7f5c5006c580 2024-03-04T07:52:23.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500c9720 con 0x7f5c5015bf20 2024-03-04T07:52:23.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c54c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c50152920 0x7f5c50157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:23.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c55600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c5006c580 0x7f5c5014fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:23.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c54c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c50152920 0x7f5c50157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56300/0 (socket says 172.21.15.113:56300) 2024-03-04T07:52:23.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c54c00640 1 -- 172.21.15.113:0/3507162606 learned_addr learned my addr 172.21.15.113:0/3507162606 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:23.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56000640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5015bf20 0x7f5c500c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:23.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c54c00640 1 -- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5015bf20 msgr2=0x7f5c500c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:23.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c54c00640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5015bf20 0x7f5c500c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c54c00640 1 -- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c5006c580 msgr2=0x7f5c5014fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:23.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c54c00640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c5006c580 0x7f5c5014fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c54c00640 1 -- 172.21.15.113:0/3507162606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c50158760 con 0x7f5c50152920 2024-03-04T07:52:23.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56000640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5015bf20 0x7f5c500c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:52:23.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c54c00640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c50152920 0x7f5c50157d70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00e930 tx=0x7f5c3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:23.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c4d600640 1 -- 172.21.15.113:0/3507162606 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c017070 con 0x7f5c50152920 2024-03-04T07:52:23.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c4d600640 1 -- 172.21.15.113:0/3507162606 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00ca00 con 0x7f5c50152920 2024-03-04T07:52:23.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56a00640 1 -- 172.21.15.113:0/3507162606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5014f9c0 con 0x7f5c50152920 2024-03-04T07:52:23.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c4d600640 1 -- 172.21.15.113:0/3507162606 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00cdd0 con 0x7f5c50152920 2024-03-04T07:52:23.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56a00640 1 -- 172.21.15.113:0/3507162606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c50158c50 con 0x7f5c50152920 2024-03-04T07:52:23.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.247+0000 7f5c56a00640 1 -- 172.21.15.113:0/3507162606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c50075a80 con 0x7f5c50152920 2024-03-04T07:52:23.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.251+0000 7f5c4d600640 1 -- 172.21.15.113:0/3507162606 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c3c01a070 con 0x7f5c50152920 2024-03-04T07:52:23.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.251+0000 7f5c4d600640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5c30066f60 0x7f5c30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:23.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.251+0000 7f5c4d600640 1 -- 172.21.15.113:0/3507162606 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f5c3c085320 con 0x7f5c50152920 2024-03-04T07:52:23.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.251+0000 7f5c55600640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5c30066f60 0x7f5c30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:23.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.251+0000 7f5c55600640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5c30066f60 0x7f5c30069420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5c440af000 tx=0x7f5c440a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:23.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.255+0000 7f5c4d600640 1 -- 172.21.15.113:0/3507162606 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5c5014f9c0 con 0x7f5c50152920 2024-03-04T07:52:23.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 -- 172.21.15.113:0/3507162606 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c5006b8f0 con 0x7f5c50152920 2024-03-04T07:52:23.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c4d600640 1 -- 172.21.15.113:0/3507162606 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f5c3c0559d0 con 0x7f5c50152920 2024-03-04T07:52:23.450 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:23.450 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:20.792498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5631,5940]},"id":1}]} 2024-03-04T07:52:23.450 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 73 2024-03-04T07:52:23.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 -- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5c30066f60 msgr2=0x7f5c30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:23.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5c30066f60 0x7f5c30069420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5c440af000 tx=0x7f5c440a7000 comp rx=0 tx=0).stop 2024-03-04T07:52:23.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 -- 172.21.15.113:0/3507162606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c50152920 msgr2=0x7f5c50157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:23.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c50152920 0x7f5c50157d70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00e930 tx=0x7f5c3c00edf0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 -- 172.21.15.113:0/3507162606 shutdown_connections 2024-03-04T07:52:23.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5c5015bf20 0x7f5c500c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5c30066f60 0x7f5c30069420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c50152920 0x7f5c50157d70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 --2- 172.21.15.113:0/3507162606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c5006c580 0x7f5c5014fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:23.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 -- 172.21.15.113:0/3507162606 >> 172.21.15.113:0/3507162606 conn(0x7f5c500c9a30 msgr2=0x7f5c501508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:23.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 -- 172.21.15.113:0/3507162606 shutdown_connections 2024-03-04T07:52:23.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:23.455+0000 7f5c56a00640 1 -- 172.21.15.113:0/3507162606 wait complete. 2024-03-04T07:52:23.472 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 73, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 73, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5631": { "addr": "172.21.15.113:6833/810789479", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 810789479, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 810789479, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5631, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 17 }, "gid_5940": { "addr": "172.21.15.113:6837/344022786", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 344022786, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 344022786, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5940, "incarnation": 64, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 30 }, "gid_6145": { "addr": "172.21.15.136:6835/2368734308", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2368734308, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2368734308, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6145, "incarnation": 71, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:reconnect", "state_seq": 35 } }, "last_failure": 0, "last_failure_osd_epoch": 58, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:52:20.792498+0000", "qdb_cluster": [ 5631, 5940 ], "qdb_leader": 5940, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5940, "mds_1": 6145, "mds_2": 5631 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:52:25.473 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.615+0000 7fb21be00640 1 Processor -- start 2024-03-04T07:52:25.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.615+0000 7fb21be00640 1 -- start start 2024-03-04T07:52:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.615+0000 7fb21be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2140c8200 0x7fb214070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.615+0000 7fb21be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb214152bd0 0x7fb2141587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.615+0000 7fb21be00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2141591f0 con 0x7fb2141527f0 2024-03-04T07:52:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.615+0000 7fb21be00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb214071df0 con 0x7fb214152bd0 2024-03-04T07:52:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.615+0000 7fb21be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2141597b0 con 0x7fb2140c8200 2024-03-04T07:52:25.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.619+0000 7fb219e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb214152bd0 0x7fb2141587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:25.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.619+0000 7fb219e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb214152bd0 0x7fb2141587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56312/0 (socket says 172.21.15.113:56312) 2024-03-04T07:52:25.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.619+0000 7fb219e00640 1 -- 172.21.15.113:0/1842529568 learned_addr learned my addr 172.21.15.113:0/1842529568 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:25.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.619+0000 7fb218a00640 1 --2- 172.21.15.113:0/1842529568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2140c8200 0x7fb214070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:25.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.619+0000 7fb219e00640 1 -- 172.21.15.113:0/1842529568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2140c8200 msgr2=0x7fb214070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:25.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.619+0000 7fb219e00640 1 --2- 172.21.15.113:0/1842529568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2140c8200 0x7fb214070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:25.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.619+0000 7fb219e00640 1 -- 172.21.15.113:0/1842529568 >> v1:172.21.15.136:6789/0 conn(0x7fb2141527f0 legacy=0x7fb214072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:25.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.619+0000 7fb219e00640 1 -- 172.21.15.113:0/1842529568 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2141594d0 con 0x7fb214152bd0 2024-03-04T07:52:25.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.619+0000 7fb218a00640 1 --2- 172.21.15.113:0/1842529568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2140c8200 0x7fb214070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:52:25.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb219e00640 1 --2- 172.21.15.113:0/1842529568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb214152bd0 0x7fb2141587d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009e50 tx=0x7fb20c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82047c4960e79902 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:25.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb213e00640 1 -- 172.21.15.113:0/1842529568 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c0a57f0 con 0x7fb214152bd0 2024-03-04T07:52:25.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb213e00640 1 -- 172.21.15.113:0/1842529568 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c0a5990 con 0x7fb214152bd0 2024-03-04T07:52:25.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb213e00640 1 -- 172.21.15.113:0/1842529568 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c0a5d60 con 0x7fb214152bd0 2024-03-04T07:52:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb21be00640 1 -- 172.21.15.113:0/1842529568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb214152bd0 msgr2=0x7fb2141587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb21be00640 1 --2- 172.21.15.113:0/1842529568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb214152bd0 0x7fb2141587d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb20c009e50 tx=0x7fb20c01ca30 comp rx=0 tx=0).stop 2024-03-04T07:52:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb21be00640 1 -- 172.21.15.113:0/1842529568 shutdown_connections 2024-03-04T07:52:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb21be00640 1 --2- 172.21.15.113:0/1842529568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb214152bd0 0x7fb2141587d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb21be00640 1 --2- 172.21.15.113:0/1842529568 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2140c8200 0x7fb214070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb21be00640 1 -- 172.21.15.113:0/1842529568 >> 172.21.15.113:0/1842529568 conn(0x7fb21406e820 msgr2=0x7fb21406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb21be00640 1 -- 172.21.15.113:0/1842529568 shutdown_connections 2024-03-04T07:52:25.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.623+0000 7fb21be00640 1 -- 172.21.15.113:0/1842529568 wait complete. 2024-03-04T07:52:25.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 Processor -- start 2024-03-04T07:52:25.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 -- start start 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140c8200 0x7fb2140b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2141527f0 0x7fb214151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb214152bd0 0x7fb214156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2141654f0 con 0x7fb2140c8200 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb214165370 con 0x7fb214152bd0 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb214165670 con 0x7fb2141527f0 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140c8200 0x7fb2140b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140c8200 0x7fb2140b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56318/0 (socket says 172.21.15.113:56318) 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219400640 1 -- 172.21.15.113:0/3869956731 learned_addr learned my addr 172.21.15.113:0/3869956731 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb218a00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2141527f0 0x7fb214151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219e00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb214152bd0 0x7fb214156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:25.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219400640 1 -- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2141527f0 msgr2=0x7fb214151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219400640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2141527f0 0x7fb214151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219400640 1 -- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb214152bd0 msgr2=0x7fb214156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219400640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb214152bd0 0x7fb214156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219400640 1 -- 172.21.15.113:0/3869956731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2141579e0 con 0x7fb2140c8200 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219e00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb214152bd0 0x7fb214156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb219400640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140c8200 0x7fb2140b0c30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb20800c480 tx=0x7fb20800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb211600640 1 -- 172.21.15.113:0/3869956731 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208016020 con 0x7fb2140c8200 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb211600640 1 -- 172.21.15.113:0/3869956731 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2080052d0 con 0x7fb2140c8200 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb211600640 1 -- 172.21.15.113:0/3869956731 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb208007260 con 0x7fb2140c8200 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 -- 172.21.15.113:0/3869956731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21414ad50 con 0x7fb2140c8200 2024-03-04T07:52:25.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 -- 172.21.15.113:0/3869956731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb214157c60 con 0x7fb2140c8200 2024-03-04T07:52:25.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.627+0000 7fb21be00640 1 -- 172.21.15.113:0/3869956731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb214075c00 con 0x7fb2140c8200 2024-03-04T07:52:25.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.631+0000 7fb211600640 1 -- 172.21.15.113:0/3869956731 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb208007400 con 0x7fb2140c8200 2024-03-04T07:52:25.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.631+0000 7fb211600640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f4066f90 0x7fb1f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:25.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.631+0000 7fb211600640 1 -- 172.21.15.113:0/3869956731 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fb208084ca0 con 0x7fb2140c8200 2024-03-04T07:52:25.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.631+0000 7fb218a00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f4066f90 0x7fb1f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:25.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.631+0000 7fb218a00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f4066f90 0x7fb1f4069450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2140c2d30 tx=0x7fb200008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:25.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.639+0000 7fb211600640 1 -- 172.21.15.113:0/3869956731 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb208020020 con 0x7fb2140c8200 2024-03-04T07:52:25.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.867+0000 7fb21be00640 1 -- 172.21.15.113:0/3869956731 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb214162f30 con 0x7fb2140c8200 2024-03-04T07:52:25.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.867+0000 7fb211600640 1 -- 172.21.15.113:0/3869956731 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5816 (secure 0 0 0) 0x7fb2141579e0 con 0x7fb2140c8200 2024-03-04T07:52:25.865 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:25.865 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:20.792498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:reconnect","state_seq":35,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5631,5940]},"id":1}]} 2024-03-04T07:52:25.865 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 73 2024-03-04T07:52:25.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 -- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f4066f90 msgr2=0x7fb1f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:25.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f4066f90 0x7fb1f4069450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2140c2d30 tx=0x7fb200008040 comp rx=0 tx=0).stop 2024-03-04T07:52:25.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 -- 172.21.15.113:0/3869956731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140c8200 msgr2=0x7fb2140b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:25.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140c8200 0x7fb2140b0c30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb20800c480 tx=0x7fb20800c940 comp rx=0 tx=0).stop 2024-03-04T07:52:25.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 -- 172.21.15.113:0/3869956731 shutdown_connections 2024-03-04T07:52:25.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb214152bd0 0x7fb214156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:25.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2141527f0 0x7fb214151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:25.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f4066f90 0x7fb1f4069450 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:25.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 --2- 172.21.15.113:0/3869956731 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb2140c8200 0x7fb2140b0c30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:25.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 -- 172.21.15.113:0/3869956731 >> 172.21.15.113:0/3869956731 conn(0x7fb21406e820 msgr2=0x7fb2140c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:25.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 -- 172.21.15.113:0/3869956731 shutdown_connections 2024-03-04T07:52:25.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:25.871+0000 7fb21be00640 1 -- 172.21.15.113:0/3869956731 wait complete. 2024-03-04T07:52:25.886 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:52:27.888 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbea00640 1 Processor -- start 2024-03-04T07:52:28.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbea00640 1 -- start start 2024-03-04T07:52:28.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2bb80c6f10 0x7f2bb8072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:28.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6240 0x7f2bb815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:28.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb80c6a30 con 0x7f2bb8158e60 2024-03-04T07:52:28.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb81620e0 con 0x7f2bb80c6f10 2024-03-04T07:52:28.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8162c60 con 0x7f2bb80c6240 2024-03-04T07:52:28.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbcc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2bb80c6f10 0x7f2bb8072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:28.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbcc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2bb80c6f10 0x7f2bb8072590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36956/0 (socket says 172.21.15.113:36956) 2024-03-04T07:52:28.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbcc00640 1 -- 172.21.15.113:0/2356250114 learned_addr learned my addr 172.21.15.113:0/2356250114 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:28.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.066+0000 7f2bbe000640 1 --2- 172.21.15.113:0/2356250114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6240 0x7f2bb815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:28.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.070+0000 7f2bbcc00640 1 -- 172.21.15.113:0/2356250114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6240 msgr2=0x7f2bb815eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.070+0000 7f2bbcc00640 1 --2- 172.21.15.113:0/2356250114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6240 0x7f2bb815eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.070+0000 7f2bbcc00640 1 -- 172.21.15.113:0/2356250114 >> v1:172.21.15.113:6789/0 conn(0x7f2bb8158e60 legacy=0x7f2bb80c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.070+0000 7f2bbcc00640 1 -- 172.21.15.113:0/2356250114 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8159240 con 0x7f2bb80c6f10 2024-03-04T07:52:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.070+0000 7f2bbe000640 1 --2- 172.21.15.113:0/2356250114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6240 0x7f2bb815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:52:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bbcc00640 1 --2- 172.21.15.113:0/2356250114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6f10 0x7f2bb8072590 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009870 tx=0x7f2ba401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535b413e4ac533ab server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:28.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bb7e00640 1 -- 172.21.15.113:0/2356250114 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40b2070 con 0x7f2bb80c6f10 2024-03-04T07:52:28.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bb7e00640 1 -- 172.21.15.113:0/2356250114 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba40a9070 con 0x7f2bb80c6f10 2024-03-04T07:52:28.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bb7e00640 1 -- 172.21.15.113:0/2356250114 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40ad880 con 0x7f2bb80c6f10 2024-03-04T07:52:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bbea00640 1 -- 172.21.15.113:0/2356250114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6f10 msgr2=0x7f2bb8072590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bbea00640 1 --2- 172.21.15.113:0/2356250114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6f10 0x7f2bb8072590 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009870 tx=0x7f2ba401c560 comp rx=0 tx=0).stop 2024-03-04T07:52:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bbea00640 1 -- 172.21.15.113:0/2356250114 shutdown_connections 2024-03-04T07:52:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bbea00640 1 --2- 172.21.15.113:0/2356250114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6240 0x7f2bb815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bbea00640 1 --2- 172.21.15.113:0/2356250114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6f10 0x7f2bb8072590 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bbea00640 1 -- 172.21.15.113:0/2356250114 >> 172.21.15.113:0/2356250114 conn(0x7f2bb806b230 msgr2=0x7f2bb806b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bbea00640 1 -- 172.21.15.113:0/2356250114 shutdown_connections 2024-03-04T07:52:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.074+0000 7f2bbea00640 1 -- 172.21.15.113:0/2356250114 wait complete. 2024-03-04T07:52:28.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbea00640 1 Processor -- start 2024-03-04T07:52:28.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbea00640 1 -- start start 2024-03-04T07:52:28.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6240 0x7f2bb81f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:28.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6f10 0x7f2bb8155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:28.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb8158e60 0x7f2bb81fadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:28.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80c9950 con 0x7f2bb80c6240 2024-03-04T07:52:28.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb81628f0 con 0x7f2bb8158e60 2024-03-04T07:52:28.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80c9ad0 con 0x7f2bb80c6f10 2024-03-04T07:52:28.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6240 0x7f2bb81f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:28.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6240 0x7f2bb81f96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36958/0 (socket says 172.21.15.113:36958) 2024-03-04T07:52:28.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbd600640 1 -- 172.21.15.113:0/4257556550 learned_addr learned my addr 172.21.15.113:0/4257556550 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:28.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbe000640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb8158e60 0x7f2bb81fadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:28.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbcc00640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6f10 0x7f2bb8155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:28.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbd600640 1 -- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6f10 msgr2=0x7f2bb8155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:28.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbd600640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6f10 0x7f2bb8155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:28.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbd600640 1 -- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb8158e60 msgr2=0x7f2bb81fadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:28.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbd600640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb8158e60 0x7f2bb81fadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:28.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbd600640 1 -- 172.21.15.113:0/4257556550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb81fbc50 con 0x7f2bb80c6240 2024-03-04T07:52:28.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbe000640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb8158e60 0x7f2bb81fadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:52:28.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbd600640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6240 0x7f2bb81f96a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00c450 tx=0x7f2bac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:28.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bb5600640 1 -- 172.21.15.113:0/4257556550 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac016020 con 0x7f2bb80c6240 2024-03-04T07:52:28.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bb5600640 1 -- 172.21.15.113:0/4257556550 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac0052d0 con 0x7f2bb80c6240 2024-03-04T07:52:28.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bb5600640 1 -- 172.21.15.113:0/4257556550 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac007260 con 0x7f2bb80c6240 2024-03-04T07:52:28.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbea00640 1 -- 172.21.15.113:0/4257556550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb8150ae0 con 0x7f2bb80c6240 2024-03-04T07:52:28.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.078+0000 7f2bbea00640 1 -- 172.21.15.113:0/4257556550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb8150360 con 0x7f2bb80c6240 2024-03-04T07:52:28.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.082+0000 7f2bbea00640 1 -- 172.21.15.113:0/4257556550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb8075a80 con 0x7f2bb80c6240 2024-03-04T07:52:28.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.082+0000 7f2bb5600640 1 -- 172.21.15.113:0/4257556550 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bac007400 con 0x7f2bb80c6240 2024-03-04T07:52:28.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.082+0000 7f2bb5600640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2b98066f60 0x7f2b98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:28.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.082+0000 7f2bb5600640 1 -- 172.21.15.113:0/4257556550 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f2bac084d40 con 0x7f2bb80c6240 2024-03-04T07:52:28.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.086+0000 7f2bbcc00640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2b98066f60 0x7f2b98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:28.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.086+0000 7f2bbcc00640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2b98066f60 0x7f2b98069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40af000 tx=0x7f2ba4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:28.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.094+0000 7f2bb5600640 1 -- 172.21.15.113:0/4257556550 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2bac020020 con 0x7f2bb80c6240 2024-03-04T07:52:28.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.286+0000 7f2bbea00640 1 -- 172.21.15.113:0/4257556550 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bb80c7970 con 0x7f2bb80c6240 2024-03-04T07:52:28.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.286+0000 7f2bb5600640 1 -- 172.21.15.113:0/4257556550 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5813 (secure 0 0 0) 0x7f2bac005480 con 0x7f2bb80c6240 2024-03-04T07:52:28.285 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:28.286 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:25.898357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631]},"id":1}]} 2024-03-04T07:52:28.286 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 74 2024-03-04T07:52:28.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 -- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2b98066f60 msgr2=0x7f2b98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:28.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2b98066f60 0x7f2b98069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40af000 tx=0x7f2ba4008040 comp rx=0 tx=0).stop 2024-03-04T07:52:28.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 -- 172.21.15.113:0/4257556550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6240 msgr2=0x7f2bb81f96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:28.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6240 0x7f2bb81f96a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00c450 tx=0x7f2bac00c910 comp rx=0 tx=0).stop 2024-03-04T07:52:28.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 -- 172.21.15.113:0/4257556550 shutdown_connections 2024-03-04T07:52:28.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bb8158e60 0x7f2bb81fadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:28.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bb80c6f10 0x7f2bb8155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:28.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2b98066f60 0x7f2b98069420 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:28.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 --2- 172.21.15.113:0/4257556550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bb80c6240 0x7f2bb81f96a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:28.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 -- 172.21.15.113:0/4257556550 >> 172.21.15.113:0/4257556550 conn(0x7f2bb806b230 msgr2=0x7f2bb80cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:28.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 -- 172.21.15.113:0/4257556550 shutdown_connections 2024-03-04T07:52:28.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:28.290+0000 7f2bbea00640 1 -- 172.21.15.113:0/4257556550 wait complete. 2024-03-04T07:52:28.307 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:52:29.428 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:52:29.434 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:52:30.308 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:30.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46ffe00640 1 Processor -- start 2024-03-04T07:52:30.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46ffe00640 1 -- start start 2024-03-04T07:52:30.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46ffe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 0x7f46f80ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:30.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46ffe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f8152e30 con 0x7f46f815c4e0 2024-03-04T07:52:30.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46ffe00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f806f280 con 0x7f46f806ff20 2024-03-04T07:52:30.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46ffe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80c31b0 con 0x7f46f8152a50 2024-03-04T07:52:30.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46fde00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f46f815c4e0 0x7f46f8163060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53082/0 (socket says 172.21.15.113:53082) 2024-03-04T07:52:30.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46fde00640 1 -- 172.21.15.113:0/372811822 learned_addr learned my addr 172.21.15.113:0/372811822 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:52:30.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46fd400640 1 --2- 172.21.15.113:0/372811822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 0x7f46f80ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:30.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46f7e00640 1 -- 172.21.15.113:0/372811822 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946827917 0 0) 0x7f46f8152e30 con 0x7f46f815c4e0 2024-03-04T07:52:30.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46f7e00640 1 -- 172.21.15.113:0/372811822 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46e0003530 con 0x7f46f815c4e0 2024-03-04T07:52:30.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46fd400640 1 -- 172.21.15.113:0/372811822 >> v1:172.21.15.136:6789/0 conn(0x7f46f806ff20 legacy=0x7f46f815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:30.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46fd400640 1 -- 172.21.15.113:0/372811822 >> v1:172.21.15.113:6789/0 conn(0x7f46f815c4e0 legacy=0x7f46f8163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:30.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.482+0000 7f46fd400640 1 -- 172.21.15.113:0/372811822 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f806f560 con 0x7f46f8152a50 2024-03-04T07:52:30.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.486+0000 7f46fd400640 1 --2- 172.21.15.113:0/372811822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 0x7f46f80ce120 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0097f0 tx=0x7f46ec01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3607b3108edbeefd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:30.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.486+0000 7f46f7e00640 1 -- 172.21.15.113:0/372811822 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0ab5f0 con 0x7f46f8152a50 2024-03-04T07:52:30.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.486+0000 7f46f7e00640 1 -- 172.21.15.113:0/372811822 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec0b0070 con 0x7f46f8152a50 2024-03-04T07:52:30.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.486+0000 7f46f7e00640 1 -- 172.21.15.113:0/372811822 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0ab990 con 0x7f46f8152a50 2024-03-04T07:52:30.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.490+0000 7f46ffe00640 1 -- 172.21.15.113:0/372811822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 msgr2=0x7f46f80ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:30.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.490+0000 7f46ffe00640 1 --2- 172.21.15.113:0/372811822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 0x7f46f80ce120 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0097f0 tx=0x7f46ec01c4e0 comp rx=0 tx=0).stop 2024-03-04T07:52:30.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.490+0000 7f46ffe00640 1 -- 172.21.15.113:0/372811822 shutdown_connections 2024-03-04T07:52:30.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.490+0000 7f46ffe00640 1 --2- 172.21.15.113:0/372811822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 0x7f46f80ce120 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:30.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.490+0000 7f46ffe00640 1 -- 172.21.15.113:0/372811822 >> 172.21.15.113:0/372811822 conn(0x7f46f80734d0 msgr2=0x7f46f8066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:30.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.490+0000 7f46ffe00640 1 -- 172.21.15.113:0/372811822 shutdown_connections 2024-03-04T07:52:30.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.490+0000 7f46ffe00640 1 -- 172.21.15.113:0/372811822 wait complete. 2024-03-04T07:52:30.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.490+0000 7f46ffe00640 1 Processor -- start 2024-03-04T07:52:30.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.490+0000 7f46ffe00640 1 -- start start 2024-03-04T07:52:30.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46ffe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f806ff20 0x7f46f80c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:30.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46ffe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 0x7f46f8158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:30.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46ffe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f815c4e0 0x7f46f80c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:30.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46ffe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8166fa0 con 0x7f46f815c4e0 2024-03-04T07:52:30.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46ffe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80ce930 con 0x7f46f806ff20 2024-03-04T07:52:30.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46ffe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8167120 con 0x7f46f8152a50 2024-03-04T07:52:30.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f815c4e0 0x7f46f80c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:30.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f806ff20 0x7f46f80c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:30.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f815c4e0 0x7f46f80c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36966/0 (socket says 172.21.15.113:36966) 2024-03-04T07:52:30.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fde00640 1 -- 172.21.15.113:0/1948080230 learned_addr learned my addr 172.21.15.113:0/1948080230 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:30.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 0x7f46f8158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:30.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fde00640 1 -- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 msgr2=0x7f46f8158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:30.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fde00640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 0x7f46f8158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:30.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fde00640 1 -- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f806ff20 msgr2=0x7f46f80c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:30.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fde00640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f806ff20 0x7f46f80c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:30.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fde00640 1 -- 172.21.15.113:0/1948080230 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f8154b60 con 0x7f46f815c4e0 2024-03-04T07:52:30.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46fde00640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f815c4e0 0x7f46f80c5590 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f46f000cb40 tx=0x7f46f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:30.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46f5600640 1 -- 172.21.15.113:0/1948080230 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0017070 con 0x7f46f815c4e0 2024-03-04T07:52:30.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46f5600640 1 -- 172.21.15.113:0/1948080230 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f000ac40 con 0x7f46f815c4e0 2024-03-04T07:52:30.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46ffe00640 1 -- 172.21.15.113:0/1948080230 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f8154e20 con 0x7f46f815c4e0 2024-03-04T07:52:30.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46f5600640 1 -- 172.21.15.113:0/1948080230 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0012620 con 0x7f46f815c4e0 2024-03-04T07:52:30.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46ffe00640 1 -- 172.21.15.113:0/1948080230 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f81557d0 con 0x7f46f815c4e0 2024-03-04T07:52:30.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.494+0000 7f46ffe00640 1 -- 172.21.15.113:0/1948080230 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f8075c00 con 0x7f46f815c4e0 2024-03-04T07:52:30.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.498+0000 7f46f5600640 1 -- 172.21.15.113:0/1948080230 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46f001a070 con 0x7f46f815c4e0 2024-03-04T07:52:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.498+0000 7f46f5600640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 0x7f46d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.498+0000 7f46f5600640 1 -- 172.21.15.113:0/1948080230 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f46f0085010 con 0x7f46f815c4e0 2024-03-04T07:52:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.498+0000 7f46fd400640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 0x7f46d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:30.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.498+0000 7f46fd400640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 0x7f46d8069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0af000 tx=0x7f46ec0b8040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:30.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.502+0000 7f46f5600640 1 -- 172.21.15.113:0/1948080230 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f46f00554e0 con 0x7f46f815c4e0 2024-03-04T07:52:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 -- 172.21.15.113:0/1948080230 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46f80cac60 con 0x7f46f815c4e0 2024-03-04T07:52:30.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46f5600640 1 -- 172.21.15.113:0/1948080230 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5813 (secure 0 0 0) 0x7f46f0059580 con 0x7f46f815c4e0 2024-03-04T07:52:30.730 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:30.731 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:25.898357+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:rejoin","state_seq":37,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,5631]},"id":1}]} 2024-03-04T07:52:30.731 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 74 2024-03-04T07:52:30.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 -- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 msgr2=0x7f46d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 0x7f46d8069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0af000 tx=0x7f46ec0b8040 comp rx=0 tx=0).stop 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 -- 172.21.15.113:0/1948080230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f815c4e0 msgr2=0x7f46f80c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f815c4e0 0x7f46f80c5590 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f46f000cb40 tx=0x7f46f000a060 comp rx=0 tx=0).stop 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 -- 172.21.15.113:0/1948080230 shutdown_connections 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 0x7f46d8069420 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46f815c4e0 0x7f46f80c5590 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46f8152a50 0x7f46f8158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 --2- 172.21.15.113:0/1948080230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46f806ff20 0x7f46f80c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 -- 172.21.15.113:0/1948080230 >> 172.21.15.113:0/1948080230 conn(0x7f46f80734d0 msgr2=0x7f46f8159e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 -- 172.21.15.113:0/1948080230 shutdown_connections 2024-03-04T07:52:30.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:30.734+0000 7f46ffe00640 1 -- 172.21.15.113:0/1948080230 wait complete. 2024-03-04T07:52:30.760 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:52:32.761 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:32.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.902+0000 7f4451400640 1 Processor -- start 2024-03-04T07:52:32.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.902+0000 7f4451400640 1 -- start start 2024-03-04T07:52:32.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4451400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f444c0cb540 0x7f444c15bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:32.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4451400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f444c06faa0 0x7f444c0c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:32.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4451400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c0c4ce0 con 0x7f444c152950 2024-03-04T07:52:32.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4451400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c0c8370 con 0x7f444c06faa0 2024-03-04T07:52:32.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4451400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0ca0a0 con 0x7f444c0cb540 2024-03-04T07:52:32.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4450a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f444c06faa0 0x7f444c0c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:32.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4450a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f444c06faa0 0x7f444c0c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36970/0 (socket says 172.21.15.113:36970) 2024-03-04T07:52:32.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4450a00640 1 -- 172.21.15.113:0/63487162 learned_addr learned my addr 172.21.15.113:0/63487162 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:32.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4443e00640 1 --2- 172.21.15.113:0/63487162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f444c0cb540 0x7f444c15bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:32.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4450a00640 1 -- 172.21.15.113:0/63487162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f444c0cb540 msgr2=0x7f444c15bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:32.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4450a00640 1 --2- 172.21.15.113:0/63487162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f444c0cb540 0x7f444c15bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:32.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4450a00640 1 -- 172.21.15.113:0/63487162 >> v1:172.21.15.136:6789/0 conn(0x7f444c152950 legacy=0x7f444c06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:32.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.906+0000 7f4450a00640 1 -- 172.21.15.113:0/63487162 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c0c4f80 con 0x7f444c06faa0 2024-03-04T07:52:32.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f4450a00640 1 --2- 172.21.15.113:0/63487162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c06faa0 0x7f444c0c30b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4444009e50 tx=0x7f444401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad3931f6572f8f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f444b400640 1 -- 172.21.15.113:0/63487162 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44440b4070 con 0x7f444c06faa0 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f444b400640 1 -- 172.21.15.113:0/63487162 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44440af400 con 0x7f444c06faa0 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f444b400640 1 -- 172.21.15.113:0/63487162 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44440af7d0 con 0x7f444c06faa0 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f4451400640 1 -- 172.21.15.113:0/63487162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c06faa0 msgr2=0x7f444c0c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f4451400640 1 --2- 172.21.15.113:0/63487162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c06faa0 0x7f444c0c30b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4444009e50 tx=0x7f444401f210 comp rx=0 tx=0).stop 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f4451400640 1 -- 172.21.15.113:0/63487162 shutdown_connections 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f4451400640 1 --2- 172.21.15.113:0/63487162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c06faa0 0x7f444c0c30b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f4451400640 1 --2- 172.21.15.113:0/63487162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f444c0cb540 0x7f444c15bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f4451400640 1 -- 172.21.15.113:0/63487162 >> 172.21.15.113:0/63487162 conn(0x7f444c072f40 msgr2=0x7f444c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f4451400640 1 -- 172.21.15.113:0/63487162 shutdown_connections 2024-03-04T07:52:32.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.910+0000 7f4451400640 1 -- 172.21.15.113:0/63487162 wait complete. 2024-03-04T07:52:32.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4451400640 1 Processor -- start 2024-03-04T07:52:32.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4451400640 1 -- start start 2024-03-04T07:52:32.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4451400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f444c06faa0 0x7f444c15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:32.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4451400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c0cb540 0x7f444c06c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:32.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4451400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f444c152950 0x7f444c06c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:32.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4451400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0c9a50 con 0x7f444c0cb540 2024-03-04T07:52:32.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4451400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0c5220 con 0x7f444c06faa0 2024-03-04T07:52:32.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4451400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c0c9d30 con 0x7f444c152950 2024-03-04T07:52:32.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4443e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c0cb540 0x7f444c06c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:32.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4443e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c0cb540 0x7f444c06c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36974/0 (socket says 172.21.15.113:36974) 2024-03-04T07:52:32.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4443e00640 1 -- 172.21.15.113:0/4219002942 learned_addr learned my addr 172.21.15.113:0/4219002942 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:32.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4450a00640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f444c152950 0x7f444c06c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:32.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f444be00640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f444c06faa0 0x7f444c15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:32.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4443e00640 1 -- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f444c152950 msgr2=0x7f444c06c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:32.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4443e00640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f444c152950 0x7f444c06c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:32.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4443e00640 1 -- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f444c06faa0 msgr2=0x7f444c15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:32.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4443e00640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f444c06faa0 0x7f444c15a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:32.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4443e00640 1 -- 172.21.15.113:0/4219002942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c075c90 con 0x7f444c0cb540 2024-03-04T07:52:32.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f444be00640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f444c06faa0 0x7f444c15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:52:32.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4443e00640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c0cb540 0x7f444c06c200 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f443400e930 tx=0x7f443400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:32.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4448c00640 1 -- 172.21.15.113:0/4219002942 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434017070 con 0x7f444c0cb540 2024-03-04T07:52:32.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4448c00640 1 -- 172.21.15.113:0/4219002942 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443400ca00 con 0x7f444c0cb540 2024-03-04T07:52:32.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4451400640 1 -- 172.21.15.113:0/4219002942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c150150 con 0x7f444c0cb540 2024-03-04T07:52:32.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4448c00640 1 -- 172.21.15.113:0/4219002942 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443400cdb0 con 0x7f444c0cb540 2024-03-04T07:52:32.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.914+0000 7f4451400640 1 -- 172.21.15.113:0/4219002942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c157740 con 0x7f444c0cb540 2024-03-04T07:52:32.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.918+0000 7f4451400640 1 -- 172.21.15.113:0/4219002942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c075a80 con 0x7f444c0cb540 2024-03-04T07:52:32.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.918+0000 7f4448c00640 1 -- 172.21.15.113:0/4219002942 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f443401a070 con 0x7f444c0cb540 2024-03-04T07:52:32.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.918+0000 7f4448c00640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4428066f60 0x7f4428069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:32.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.918+0000 7f4448c00640 1 -- 172.21.15.113:0/4219002942 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f4434085410 con 0x7f444c0cb540 2024-03-04T07:52:32.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.918+0000 7f444be00640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4428066f60 0x7f4428069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:32.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.922+0000 7f444be00640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4428066f60 0x7f4428069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f443c000f70 tx=0x7f443c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:32.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:32.922+0000 7f4448c00640 1 -- 172.21.15.113:0/4219002942 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4434055a20 con 0x7f444c0cb540 2024-03-04T07:52:33.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 -- 172.21.15.113:0/4219002942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f444c157910 con 0x7f444c0cb540 2024-03-04T07:52:33.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4448c00640 1 -- 172.21.15.113:0/4219002942 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f4434059ac0 con 0x7f444c0cb540 2024-03-04T07:52:33.131 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:33.131 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:30.928216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5631,5940,6145]},"id":1}]} 2024-03-04T07:52:33.131 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 75 2024-03-04T07:52:33.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 -- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4428066f60 msgr2=0x7f4428069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:33.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4428066f60 0x7f4428069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f443c000f70 tx=0x7f443c009040 comp rx=0 tx=0).stop 2024-03-04T07:52:33.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 -- 172.21.15.113:0/4219002942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c0cb540 msgr2=0x7f444c06c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:33.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c0cb540 0x7f444c06c200 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f443400e930 tx=0x7f443400edf0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 -- 172.21.15.113:0/4219002942 shutdown_connections 2024-03-04T07:52:33.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f444c152950 0x7f444c06c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4428066f60 0x7f4428069420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f444c0cb540 0x7f444c06c200 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 --2- 172.21.15.113:0/4219002942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f444c06faa0 0x7f444c15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 -- 172.21.15.113:0/4219002942 >> 172.21.15.113:0/4219002942 conn(0x7f444c072f40 msgr2=0x7f444c158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:33.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 -- 172.21.15.113:0/4219002942 shutdown_connections 2024-03-04T07:52:33.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.134+0000 7f4451400640 1 -- 172.21.15.113:0/4219002942 wait complete. 2024-03-04T07:52:33.152 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=1, replacing gid=5529 2024-03-04T07:52:33.152 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.a 2024-03-04T07:52:33.152 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-03-04T07:52:33.152 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-04T07:52:33.152 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-04T07:52:33.156 INFO:tasks.ceph.mds.a:Started 2024-03-04T07:52:33.156 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:33.243 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-03-04T07:52:33.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46c00640 1 Processor -- start 2024-03-04T07:52:33.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46c00640 1 -- start start 2024-03-04T07:52:33.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbc4006c170 0x7fbc400c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:33.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbc400c9ce0 0x7fbc400ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:33.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4006c5b0 con 0x7fbc40157f80 2024-03-04T07:52:33.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc400c8fd0 con 0x7fbc400c9ce0 2024-03-04T07:52:33.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4006e1e0 con 0x7fbc4006c170 2024-03-04T07:52:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbc400c9ce0 0x7fbc400ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbc400c9ce0 0x7fbc400ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37004/0 (socket says 172.21.15.113:37004) 2024-03-04T07:52:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46200640 1 -- 172.21.15.113:0/1755279248 learned_addr learned my addr 172.21.15.113:0/1755279248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:33.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc44e00640 1 --2- 172.21.15.113:0/1755279248 >> v2:172.21.15.136:3300/0 conn(0x7fbc4006c170 0x7fbc400c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46200640 1 -- 172.21.15.113:0/1755279248 >> v2:172.21.15.136:3300/0 conn(0x7fbc4006c170 msgr2=0x7fbc400c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46200640 1 --2- 172.21.15.113:0/1755279248 >> v2:172.21.15.136:3300/0 conn(0x7fbc4006c170 0x7fbc400c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46200640 1 -- 172.21.15.113:0/1755279248 >> v1:172.21.15.136:6789/0 conn(0x7fbc40157f80 legacy=0x7fbc4006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:33.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.302+0000 7fbc46200640 1 -- 172.21.15.113:0/1755279248 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc400ca630 con 0x7fbc400c9ce0 2024-03-04T07:52:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc46200640 1 --2- 172.21.15.113:0/1755279248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc400c9ce0 0x7fbc400ca0f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbc40070260 tx=0x7fbc3801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c038bb40183b3629 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc3fe00640 1 -- 172.21.15.113:0/1755279248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc380a5670 con 0x7fbc400c9ce0 2024-03-04T07:52:33.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc3fe00640 1 -- 172.21.15.113:0/1755279248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc380a5810 con 0x7fbc400c9ce0 2024-03-04T07:52:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc3fe00640 1 -- 172.21.15.113:0/1755279248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc380a5be0 con 0x7fbc400c9ce0 2024-03-04T07:52:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc46c00640 1 -- 172.21.15.113:0/1755279248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc400c9ce0 msgr2=0x7fbc400ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc46c00640 1 --2- 172.21.15.113:0/1755279248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc400c9ce0 0x7fbc400ca0f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbc40070260 tx=0x7fbc3801c9b0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc46c00640 1 -- 172.21.15.113:0/1755279248 shutdown_connections 2024-03-04T07:52:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc46c00640 1 --2- 172.21.15.113:0/1755279248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc400c9ce0 0x7fbc400ca0f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc46c00640 1 --2- 172.21.15.113:0/1755279248 >> v2:172.21.15.136:3300/0 conn(0x7fbc4006c170 0x7fbc400c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc46c00640 1 -- 172.21.15.113:0/1755279248 >> 172.21.15.113:0/1755279248 conn(0x7fbc40072f40 msgr2=0x7fbc400706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc46c00640 1 -- 172.21.15.113:0/1755279248 shutdown_connections 2024-03-04T07:52:33.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.306+0000 7fbc46c00640 1 -- 172.21.15.113:0/1755279248 wait complete. 2024-03-04T07:52:33.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46c00640 1 Processor -- start 2024-03-04T07:52:33.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46c00640 1 -- start start 2024-03-04T07:52:33.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc4006c170 0x7fbc4014bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:33.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc400c9ce0 0x7fbc400caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:33.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc40157f80 0x7fbc400cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:33.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc40158360 con 0x7fbc40157f80 2024-03-04T07:52:33.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4006dc00 con 0x7fbc4006c170 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4006fc30 con 0x7fbc400c9ce0 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc40157f80 0x7fbc400cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc40157f80 0x7fbc400cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37016/0 (socket says 172.21.15.113:37016) 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46200640 1 -- 172.21.15.113:0/319590938 learned_addr learned my addr 172.21.15.113:0/319590938 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc45800640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc4006c170 0x7fbc4014bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc44e00640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc400c9ce0 0x7fbc400caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46200640 1 -- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc400c9ce0 msgr2=0x7fbc400caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46200640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc400c9ce0 0x7fbc400caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46200640 1 -- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc4006c170 msgr2=0x7fbc4014bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46200640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc4006c170 0x7fbc4014bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46200640 1 -- 172.21.15.113:0/319590938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4014c1d0 con 0x7fbc40157f80 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46200640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc40157f80 0x7fbc400cc8d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbc3801a5e0 tx=0x7fbc38019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:33.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc3d600640 1 -- 172.21.15.113:0/319590938 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38009510 con 0x7fbc40157f80 2024-03-04T07:52:33.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc3d600640 1 -- 172.21.15.113:0/319590938 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc380be070 con 0x7fbc40157f80 2024-03-04T07:52:33.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46c00640 1 -- 172.21.15.113:0/319590938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc400cb410 con 0x7fbc40157f80 2024-03-04T07:52:33.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc3d600640 1 -- 172.21.15.113:0/319590938 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc380b88b0 con 0x7fbc40157f80 2024-03-04T07:52:33.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.310+0000 7fbc46c00640 1 -- 172.21.15.113:0/319590938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4014c770 con 0x7fbc40157f80 2024-03-04T07:52:33.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.314+0000 7fbc46c00640 1 -- 172.21.15.113:0/319590938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc40075a80 con 0x7fbc40157f80 2024-03-04T07:52:33.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.314+0000 7fbc3d600640 1 -- 172.21.15.113:0/319590938 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc380b8a50 con 0x7fbc40157f80 2024-03-04T07:52:33.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.314+0000 7fbc3d600640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbc20066f90 0x7fbc20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:33.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.314+0000 7fbc3d600640 1 -- 172.21.15.113:0/319590938 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fbc38120690 con 0x7fbc40157f80 2024-03-04T07:52:33.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.314+0000 7fbc45800640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbc20066f90 0x7fbc20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:33.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.314+0000 7fbc45800640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbc20066f90 0x7fbc20069450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbc340098e0 tx=0x7fbc34009060 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:33.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.322+0000 7fbc3d600640 1 -- 172.21.15.113:0/319590938 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbc380165a0 con 0x7fbc40157f80 2024-03-04T07:52:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.550+0000 7fbc46c00640 1 -- 172.21.15.113:0/319590938 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc400c97b0 con 0x7fbc40157f80 2024-03-04T07:52:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc3d600640 1 -- 172.21.15.113:0/319590938 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5818 (secure 0 0 0) 0x7fbc380f0d00 con 0x7fbc40157f80 2024-03-04T07:52:33.550 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:33.550 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:30.928216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5631,5940,6145]},"id":1}]} 2024-03-04T07:52:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 75 2024-03-04T07:52:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 -- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbc20066f90 msgr2=0x7fbc20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbc20066f90 0x7fbc20069450 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbc340098e0 tx=0x7fbc34009060 comp rx=0 tx=0).stop 2024-03-04T07:52:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 -- 172.21.15.113:0/319590938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc40157f80 msgr2=0x7fbc400cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:33.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc40157f80 0x7fbc400cc8d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbc3801a5e0 tx=0x7fbc38019750 comp rx=0 tx=0).stop 2024-03-04T07:52:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 -- 172.21.15.113:0/319590938 shutdown_connections 2024-03-04T07:52:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbc20066f90 0x7fbc20069450 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc40157f80 0x7fbc400cc8d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbc400c9ce0 0x7fbc400caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 --2- 172.21.15.113:0/319590938 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbc4006c170 0x7fbc4014bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 -- 172.21.15.113:0/319590938 >> 172.21.15.113:0/319590938 conn(0x7fbc40072f40 msgr2=0x7fbc401445f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 -- 172.21.15.113:0/319590938 shutdown_connections 2024-03-04T07:52:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:33.554+0000 7fbc46c00640 1 -- 172.21.15.113:0/319590938 wait complete. 2024-03-04T07:52:33.574 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-03-04T07:52:35.575 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:52:35.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad81c00640 1 Processor -- start 2024-03-04T07:52:35.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad81c00640 1 -- start start 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad81c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0c8200 0x7fad7c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad81c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad7c152bd0 0x7fad7c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad81c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7c1591f0 con 0x7fad7c1527f0 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad81c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7c071df0 con 0x7fad7c152bd0 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad81c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c1597b0 con 0x7fad7c0c8200 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad7b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad7c152bd0 0x7fad7c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad7b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad7c152bd0 0x7fad7c1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37032/0 (socket says 172.21.15.113:37032) 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad7b400640 1 -- 172.21.15.113:0/2471634360 learned_addr learned my addr 172.21.15.113:0/2471634360 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad7a000640 1 --2- 172.21.15.113:0/2471634360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0c8200 0x7fad7c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad7b400640 1 -- 172.21.15.113:0/2471634360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0c8200 msgr2=0x7fad7c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad7b400640 1 --2- 172.21.15.113:0/2471634360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0c8200 0x7fad7c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:35.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad7b400640 1 -- 172.21.15.113:0/2471634360 >> v1:172.21.15.136:6789/0 conn(0x7fad7c1527f0 legacy=0x7fad7c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:35.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.754+0000 7fad7b400640 1 -- 172.21.15.113:0/2471634360 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c1594d0 con 0x7fad7c152bd0 2024-03-04T07:52:35.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad7b400640 1 --2- 172.21.15.113:0/2471634360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c152bd0 0x7fad7c1587d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fad74009e50 tx=0x7fad7401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6167ee66d70b0949 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad79600640 1 -- 172.21.15.113:0/2471634360 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740a57f0 con 0x7fad7c152bd0 2024-03-04T07:52:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad79600640 1 -- 172.21.15.113:0/2471634360 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad740a5990 con 0x7fad7c152bd0 2024-03-04T07:52:35.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad79600640 1 -- 172.21.15.113:0/2471634360 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740a5d60 con 0x7fad7c152bd0 2024-03-04T07:52:35.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad81c00640 1 -- 172.21.15.113:0/2471634360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c152bd0 msgr2=0x7fad7c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:35.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad81c00640 1 --2- 172.21.15.113:0/2471634360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c152bd0 0x7fad7c1587d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fad74009e50 tx=0x7fad7401ca30 comp rx=0 tx=0).stop 2024-03-04T07:52:35.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad81c00640 1 -- 172.21.15.113:0/2471634360 shutdown_connections 2024-03-04T07:52:35.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad81c00640 1 --2- 172.21.15.113:0/2471634360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c152bd0 0x7fad7c1587d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:35.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad81c00640 1 --2- 172.21.15.113:0/2471634360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c0c8200 0x7fad7c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:35.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad81c00640 1 -- 172.21.15.113:0/2471634360 >> 172.21.15.113:0/2471634360 conn(0x7fad7c06e820 msgr2=0x7fad7c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:35.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad81c00640 1 -- 172.21.15.113:0/2471634360 shutdown_connections 2024-03-04T07:52:35.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad81c00640 1 -- 172.21.15.113:0/2471634360 wait complete. 2024-03-04T07:52:35.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad81c00640 1 Processor -- start 2024-03-04T07:52:35.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.762+0000 7fad81c00640 1 -- start start 2024-03-04T07:52:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad81c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c0c8200 0x7fad7c151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad81c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1527f0 0x7fad7c1618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad81c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c152bd0 0x7fad7c161de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad81c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c1654f0 con 0x7fad7c0c8200 2024-03-04T07:52:35.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad81c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c165370 con 0x7fad7c1527f0 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad81c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c165670 con 0x7fad7c152bd0 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c0c8200 0x7fad7c151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c0c8200 0x7fad7c151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37038/0 (socket says 172.21.15.113:37038) 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7aa00640 1 -- 172.21.15.113:0/1292860136 learned_addr learned my addr 172.21.15.113:0/1292860136 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7b400640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c152bd0 0x7fad7c161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7a000640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1527f0 0x7fad7c1618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7aa00640 1 -- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c152bd0 msgr2=0x7fad7c161de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7aa00640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c152bd0 0x7fad7c161de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7aa00640 1 -- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1527f0 msgr2=0x7fad7c1618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7aa00640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1527f0 0x7fad7c1618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7aa00640 1 -- 172.21.15.113:0/1292860136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c1f4c20 con 0x7fad7c0c8200 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7a000640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1527f0 0x7fad7c1618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:52:35.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad7aa00640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c0c8200 0x7fad7c151490 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c450 tx=0x7fad6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:35.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad72a00640 1 -- 172.21.15.113:0/1292860136 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c016020 con 0x7fad7c0c8200 2024-03-04T07:52:35.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad72a00640 1 -- 172.21.15.113:0/1292860136 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c0052d0 con 0x7fad7c0c8200 2024-03-04T07:52:35.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad81c00640 1 -- 172.21.15.113:0/1292860136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c162e90 con 0x7fad7c0c8200 2024-03-04T07:52:35.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad72a00640 1 -- 172.21.15.113:0/1292860136 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c007260 con 0x7fad7c0c8200 2024-03-04T07:52:35.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad81c00640 1 -- 172.21.15.113:0/1292860136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c1f4380 con 0x7fad7c0c8200 2024-03-04T07:52:35.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad81c00640 1 -- 172.21.15.113:0/1292860136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c075c00 con 0x7fad7c0c8200 2024-03-04T07:52:35.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.766+0000 7fad72a00640 1 -- 172.21.15.113:0/1292860136 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad6c007400 con 0x7fad7c0c8200 2024-03-04T07:52:35.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.770+0000 7fad72a00640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad58066f60 0x7fad58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:52:35.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.770+0000 7fad72a00640 1 -- 172.21.15.113:0/1292860136 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fad6c084d40 con 0x7fad7c0c8200 2024-03-04T07:52:35.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.770+0000 7fad7a000640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad58066f60 0x7fad58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:52:35.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.770+0000 7fad7a000640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad58066f60 0x7fad58069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0b1970 tx=0x7fad64008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:52:35.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.774+0000 7fad72a00640 1 -- 172.21.15.113:0/1292860136 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fad6c020020 con 0x7fad7c0c8200 2024-03-04T07:52:35.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.970+0000 7fad81c00640 1 -- 172.21.15.113:0/1292860136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad7c0c79d0 con 0x7fad7c0c8200 2024-03-04T07:52:35.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad72a00640 1 -- 172.21.15.113:0/1292860136 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+6609 (secure 0 0 0) 0x7fad6c005480 con 0x7fad7c0c8200 2024-03-04T07:52:35.972 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:52:35.972 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6240,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:30.928216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940,5631]},"id":1}]} 2024-03-04T07:52:35.973 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 76 2024-03-04T07:52:35.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 -- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad58066f60 msgr2=0x7fad58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:35.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad58066f60 0x7fad58069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0b1970 tx=0x7fad64008040 comp rx=0 tx=0).stop 2024-03-04T07:52:35.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 -- 172.21.15.113:0/1292860136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c0c8200 msgr2=0x7fad7c151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:52:35.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c0c8200 0x7fad7c151490 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c450 tx=0x7fad6c00c910 comp rx=0 tx=0).stop 2024-03-04T07:52:35.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 -- 172.21.15.113:0/1292860136 shutdown_connections 2024-03-04T07:52:35.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad7c152bd0 0x7fad7c161de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:35.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad7c1527f0 0x7fad7c1618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:35.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad58066f60 0x7fad58069420 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:35.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 --2- 172.21.15.113:0/1292860136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad7c0c8200 0x7fad7c151490 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:52:35.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 -- 172.21.15.113:0/1292860136 >> 172.21.15.113:0/1292860136 conn(0x7fad7c06e820 msgr2=0x7fad7c1581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:52:35.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 -- 172.21.15.113:0/1292860136 shutdown_connections 2024-03-04T07:52:35.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:52:35.974+0000 7fad81c00640 1 -- 172.21.15.113:0/1292860136 wait complete. 2024-03-04T07:52:35.990 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-03-04T07:52:35.991 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 113 secs before thrashing 2024-03-04T07:52:54.523 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 1% (172/13096) Updating files: 1% (185/13096) Updating files: 1% (211/13096) Updating files: 2% (262/13096) Updating files: 2% (263/13096) Updating files: 2% (361/13096) Updating files: 2% (388/13096) Updating files: 3% (393/13096) Updating files: 4% (524/13096) Updating files: 4% (566/13096) Updating files: 4% (632/13096) Updating files: 5% (655/13096) Updating files: 5% (679/13096) Updating files: 5% (753/13096) Updating files: 5% (761/13096) Updating files: 6% (786/13096) Updating files: 7% (917/13096) Updating files: 7% (926/13096) Updating files: 7% (974/13096) Updating files: 7% (1026/13096) Updating files: 8% (1048/13096) Updating files: 8% (1167/13096) Updating files: 9% (1179/13096) Updating files: 9% (1219/13096) Updating files: 9% (1246/13096) Updating files: 9% (1301/13096) Updating files: 10% (1310/13096) Updating files: 10% (1402/13096) Updating files: 11% (1441/13096) Updating files: 11% (1524/13096) Updating files: 11% (1548/13096) Updating files: 12% (1572/13096) Updating files: 12% (1653/13096) Updating files: 12% (1659/13096) Updating files: 13% (1703/13096) Updating files: 13% (1710/13096) Updating files: 13% (1801/13096) Updating files: 14% (1834/13096) Updating files: 14% (1885/13096) Updating files: 14% (1933/13096) Updating files: 14% (1943/13096) Updating files: 15% (1965/13096) Updating files: 15% (1997/13096) Updating files: 15% (2071/13096) Updating files: 16% (2096/13096) Updating files: 16% (2120/13096) Updating files: 16% (2160/13096) Updating files: 16% (2211/13096) Updating files: 17% (2227/13096) Updating files: 17% (2276/13096) Updating files: 18% (2358/13096) Updating files: 18% (2398/13096) Updating files: 18% (2423/13096) Updating files: 18% (2461/13096) Updating files: 19% (2489/13096) Updating files: 19% (2491/13096) Updating files: 19% (2496/13096) Updating files: 19% (2558/13096) Updating files: 19% (2599/13096) Updating files: 20% (2620/13096) Updating files: 20% (2636/13096) Updating files: 20% (2675/13096) Updating files: 20% (2707/13096) Updating files: 20% (2750/13096) Updating files: 21% (2751/13096) Updating files: 21% (2782/13096) Updating files: 21% (2833/13096) Updating files: 21% (2880/13096) Updating files: 22% (2882/13096) Updating files: 22% (2924/13096) Updating files: 22% (2929/13096) Updating files: 22% (2932/13096) Updating files: 23% (3013/13096) Updating files: 23% (3047/13096) Updating files: 23% (3084/13096) Updating files: 23% (3112/13096) Updating files: 23% (3126/13096) Updating files: 24% (3144/13096) Updating files: 24% (3154/13096) Updating files: 24% (3214/13096) Updating files: 25% (3274/13096) Updating files: 25% (3311/13096) Updating files: 25% (3352/13096) Updating files: 25% (3358/13096) Updating files: 25% (3367/13096) Updating files: 26% (3405/13096) Updating files: 26% (3445/13096) Updating files: 26% (3471/13096) Updating files: 26% (3501/13096) Updating files: 26% (3510/13096) Updating files: 27% (3536/13096) Updating files: 27% (3577/13096) Updating files: 27% (3589/13096) Updating files: 27% (3597/13096) Updating files: 27% (3617/13096) Updating files: 27% (3643/13096) Updating files: 28% (3667/13096) Updating files: 28% (3707/13096) Updating files: 28% (3736/13096) Updating files: 28% (3794/13096) Updating files: 29% (3798/13096) Updating files: 29% (3874/13096) Updating files: 30% (3929/13096) Updating files: 30% (3964/13096) Updating files: 30% (4056/13096) Updating files: 31% (4060/13096) Updating files: 31% (4130/13096) Updating files: 31% (4135/13096) Updating files: 31% (4154/13096) Updating files: 32% (4191/13096) Updating files: 32% (4202/13096) Updating files: 32% (4238/13096) Updating files: 32% (4292/13096) Updating files: 32% (4316/13096) Updating files: 33% (4322/13096) Updating files: 33% (4406/13096) Updating files: 34% (4453/13096) Updating files: 34% (4470/13096) Updating files: 34% (4556/13096) Updating files: 35% (4584/13096) Updating files: 35% (4586/13096) Updating files: 35% (4673/13096) Updating files: 36% (4715/13096) Updating files: 36% (4722/13096) Updating files: 36% (4772/13096) Updating files: 37% (4846/13096) Updating files: 37% (4850/13096) Updating files: 38% (4977/13096) Updating files: 38% (4984/13096) Updating files: 38% (4988/13096) Updating files: 38% (5019/13096) Updating files: 38% (5064/13096) Updating files: 39% (5108/13096) Updating files: 39% (5172/13096) Updating files: 39% (5199/13096) Updating files: 40% (5239/13096) Updating files: 40% (5270/13096) Updating files: 40% (5336/13096) Updating files: 40% (5346/13096) Updating files: 40% (5362/13096) Updating files: 40% (5363/13096) Updating files: 41% (5370/13096) Updating files: 41% (5435/13096) Updating files: 42% (5501/13096) Updating files: 42% (5520/13096) Updating files: 42% (5579/13096) Updating files: 43% (5632/13096) Updating files: 43% (5639/13096) Updating files: 43% (5695/13096) Updating files: 44% (5763/13096) Updating files: 44% (5779/13096) Updating files: 44% (5849/13096) Updating files: 45% (5894/13096) Updating files: 45% (5936/13096) Updating files: 46% (6025/13096) Updating files: 46% (6073/13096) Updating files: 46% (6098/13096) Updating files: 47% (6156/13096) Updating files: 47% (6183/13096) Updating files: 47% (6238/13096) Updating files: 48% (6287/13096) Updating files: 48% (6382/13096) Updating files: 49% (6418/13096) Updating files: 49% (6469/13096) Updating files: 50% (6548/13096) Updating files: 50% (6577/13096) Updating files: 50% (6601/13096) Updating files: 50% (6623/13096) Updating files: 51% (6679/13096) Updating files: 51% (6758/13096) Updating files: 51% (6780/13096) Updating files: 52% (6810/13096) Updating files: 52% (6852/13096) Updating files: 52% (6902/13096) Updating files: 52% (6927/13096) Updating files: 53% (6941/13096) Updating files: 53% (7042/13096) Updating files: 54% (7072/13096) Updating files: 54% (7085/13096) Updating files: 54% (7154/13096) Updating files: 54% (7161/13096) Updating files: 55% (7203/13096) Updating files: 55% (7252/13096) Updating files: 55% (7285/13096) Updating files: 56% (7334/13096) Updating files: 56% (7453/13096) Updating files: 56% (7458/13096) Updating files: 57% (7465/13096) Updating files: 57% (7497/13096) Updating files: 57% (7498/13096) Updating files: 57% (7527/13096) Updating files: 58% (7596/13096) Updating files: 58% (7669/13096) Updating files: 58% (7673/13096) Updating files: 58% (7689/13096) Updating files: 59% (7727/13096) Updating files: 59% (7755/13096) Updating files: 60% (7858/13096) Updating files: 60% (7885/13096) Updating files: 61% (7989/13096) Updating files: 61% (7998/13096) Updating files: 61% (8019/13096) Updating files: 61% (8052/13096) Updating files: 61% (8062/13096) Updating files: 61% (8098/13096) Updating files: 61% (8102/13096) Updating files: 61% (8118/13096) Updating files: 62% (8120/13096) Updating files: 62% (8193/13096) Updating files: 62% (8236/13096) Updating files: 63% (8251/13096) Updating files: 63% (8270/13096) Updating files: 63% (8288/13096) Updating files: 63% (8308/13096) Updating files: 64% (8382/13096) Updating files: 64% (8386/13096) Updating files: 64% (8449/13096) Updating files: 65% (8513/13096) Updating files: 65% (8564/13096) Updating files: 65% (8615/13096) Updating files: 66% (8644/13096) Updating files: 66% (8664/13096) Updating files: 66% (8728/13096) Updating files: 66% (8747/13096) Updating files: 67% (8775/13096) Updating files: 67% (8780/13096) Updating files: 67% (8796/13096) Updating files: 67% (8816/13096) Updating files: 67% (8862/13096) Updating files: 67% (8877/13096) Updating files: 68% (8906/13096) Updating files: 68% (8941/13096) Updating files: 68% (8958/13096) Updating files: 68% (8980/13096) Updating files: 68% (9015/13096) Updating files: 68% (9018/13096) Updating files: 69% (9037/13096) Updating files: 70% (9168/13096) Updating files: 70% (9170/13096) Updating files: 70% (9236/13096) Updating files: 71% (9299/13096) Updating files: 71% (9306/13096) Updating files: 71% (9308/13096) Updating files: 71% (9351/13096) Updating files: 71% (9425/13096) Updating files: 72% (9430/13096) Updating files: 72% (9454/13096) Updating files: 72% (9492/13096) Updating files: 73% (9561/13096) Updating files: 73% (9689/13096) Updating files: 74% (9692/13096) Updating files: 74% (9739/13096) Updating files: 74% (9749/13096) Updating files: 75% (9822/13096) Updating files: 75% (9841/13096) Updating files: 75% (9907/13096) Updating files: 75% (9932/13096) Updating files: 76% (9953/13096) Updating files: 76% (10037/13096) Updating files: 76% (10060/13096) Updating files: 77% (10084/13096) Updating files: 77% (10103/13096) Updating files: 78% (10215/13096) Updating files: 78% (10252/13096) Updating files: 78% (10306/13096) Updating files: 78% (10316/13096) Updating files: 79% (10346/13096) Updating files: 80% (10477/13096) Updating files: 80% (10501/13096) Updating files: 80% (10519/13096) Updating files: 80% (10569/13096) Updating files: 81% (10608/13096) Updating files: 81% (10638/13096) Updating files: 81% (10705/13096) Updating files: 82% (10739/13096) Updating files: 82% (10767/13096) Updating files: 82% (10832/13096) Updating files: 83% (10870/13096) Updating files: 83% (10885/13096) Updating files: 83% (10963/13096) Updating files: 84% (11001/13096) Updating files: 84% (11071/13096) Updating files: 84% (11127/13096) Updating files: 85% (11132/13096) Updating files: 86% (11263/13096) Updating files: 86% (11292/13096) Updating files: 86% (11316/13096) Updating files: 86% (11334/13096) Updating files: 87% (11394/13096) Updating files: 87% (11442/13096) Updating files: 87% (11502/13096) Updating files: 88% (11525/13096) Updating files: 88% (11645/13096) Updating files: 89% (11656/13096) Updating files: 89% (11662/13096) Updating files: 89% (11692/13096) Updating files: 89% (11693/13096) Updating files: 89% (11771/13096) Updating files: 90% (11787/13096) Updating files: 90% (11803/13096) Updating files: 91% (11918/13096) Updating files: 91% (11961/13096) Updating files: 91% (11999/13096) Updating files: 91% (12025/13096) Updating files: 91% (12038/13096) Updating files: 92% (12049/13096) Updating files: 92% (12116/13096) Updating files: 92% (12164/13096) Updating files: 93% (12180/13096) Updating files: 94% (12311/13096) Updating files: 95% (12442/13096) Updating files: 96% (12573/13096) Updating files: 96% (12588/13096) Updating files: 96% (12646/13096) Updating files: 97% (12704/13096) Updating files: 97% (12713/13096) Updating files: 97% (12766/13096) Updating files: 98% (12835/13096) Updating files: 98% (12863/13096) Updating files: 98% (12906/13096) Updating files: 98% (12962/13096) Updating files: 99% (12966/13096) Updating files: 99% (13025/13096) Updating files: 99% (13027/13096) Updating files: 100% (13096/13096) Updating files: 100% (13096/13096), done. 2024-03-04T07:52:55.139 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 204c175eea3 msg: blobhash on entity_name_t causes strange badness... should look into that, but avoid for now 2024-03-04T07:52:55.147 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.1 2024-03-04T07:52:55.184 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 2 -eq 48 ] 2024-03-04T07:52:55.184 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.2 2024-03-04T07:52:55.184 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.2 2024-03-04T07:52:55.184 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.2 2024-03-04T07:52:55.184 INFO:tasks.workunit.client.0.smithi113.stdout:v0.2 2024-03-04T07:52:58.296 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 46% (104/222) Updating files: 47% (105/222) Updating files: 48% (107/222) Updating files: 49% (109/222) Updating files: 50% (111/222) Updating files: 51% (114/222) Updating files: 52% (116/222) Updating files: 53% (118/222) Updating files: 54% (120/222) Updating files: 55% (123/222) Updating files: 56% (125/222) Updating files: 57% (127/222) Updating files: 58% (129/222) Updating files: 59% (131/222) Updating files: 60% (134/222) Updating files: 61% (136/222) Updating files: 62% (138/222) Updating files: 63% (140/222) Updating files: 64% (143/222) Updating files: 65% (145/222) Updating files: 66% (147/222) Updating files: 67% (149/222) Updating files: 68% (151/222) Updating files: 69% (154/222) Updating files: 70% (156/222) Updating files: 71% (158/222) Updating files: 72% (160/222) Updating files: 73% (163/222) Updating files: 74% (165/222) Updating files: 75% (167/222) Updating files: 76% (169/222) Updating files: 77% (171/222) Updating files: 78% (174/222) Updating files: 79% (176/222) Updating files: 80% (178/222) Updating files: 81% (180/222) Updating files: 82% (183/222) Updating files: 83% (185/222) Updating files: 84% (187/222) Updating files: 85% (189/222) Updating files: 86% (191/222) Updating files: 87% (194/222) Updating files: 88% (196/222) Updating files: 89% (198/222) Updating files: 90% (200/222) Updating files: 91% (203/222) Updating files: 92% (205/222) Updating files: 93% (207/222) Updating files: 94% (209/222) Updating files: 95% (211/222) Updating files: 96% (214/222) Updating files: 97% (216/222) Updating files: 98% (218/222) Updating files: 99% (220/222) Updating files: 100% (222/222) Updating files: 100% (222/222), done. 2024-03-04T07:52:58.329 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 2b5f76f48e9 readme 2024-03-04T07:52:58.341 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.2 2024-03-04T07:52:58.611 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 3 -eq 48 ] 2024-03-04T07:52:58.612 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.3 2024-03-04T07:52:58.612 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.3 2024-03-04T07:52:58.612 INFO:tasks.workunit.client.0.smithi113.stdout:v0.3 2024-03-04T07:52:58.612 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.3 2024-03-04T07:52:59.556 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:52:59.563 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:53:01.675 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 46% (138/295) Updating files: 47% (139/295) Updating files: 48% (142/295) Updating files: 49% (145/295) Updating files: 50% (148/295) Updating files: 51% (151/295) Updating files: 52% (154/295) Updating files: 53% (157/295) Updating files: 54% (160/295) Updating files: 55% (163/295) Updating files: 56% (166/295) Updating files: 57% (169/295) Updating files: 58% (172/295) Updating files: 59% (175/295) Updating files: 60% (177/295) Updating files: 61% (180/295) Updating files: 62% (183/295) Updating files: 63% (186/295) Updating files: 64% (189/295) Updating files: 65% (192/295) Updating files: 66% (195/295) Updating files: 67% (198/295) Updating files: 67% (200/295) Updating files: 68% (201/295) Updating files: 69% (204/295) Updating files: 70% (207/295) Updating files: 71% (210/295) Updating files: 72% (213/295) Updating files: 73% (216/295) Updating files: 74% (219/295) Updating files: 75% (222/295) Updating files: 76% (225/295) Updating files: 77% (228/295) Updating files: 78% (231/295) Updating files: 79% (234/295) Updating files: 80% (236/295) Updating files: 81% (239/295) Updating files: 82% (242/295) Updating files: 83% (245/295) Updating files: 84% (248/295) Updating files: 85% (251/295) Updating files: 86% (254/295) Updating files: 87% (257/295) Updating files: 88% (260/295) Updating files: 89% (263/295) Updating files: 90% (266/295) Updating files: 91% (269/295) Updating files: 92% (272/295) Updating files: 93% (275/295) Updating files: 94% (278/295) Updating files: 95% (281/295) Updating files: 96% (284/295) Updating files: 97% (287/295) Updating files: 98% (290/295) Updating files: 99% (293/295) Updating files: 100% (295/295) Updating files: 100% (295/295), done. 2024-03-04T07:53:01.696 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 2e986fe9ddf include ceph.spec in configure.ac 2024-03-04T07:53:01.714 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.3 2024-03-04T07:53:02.118 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 4 -eq 48 ] 2024-03-04T07:53:02.119 INFO:tasks.workunit.client.0.smithi113.stdout:v0.4 2024-03-04T07:53:02.119 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.4 2024-03-04T07:53:02.119 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.4 2024-03-04T07:53:02.119 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.4 2024-03-04T07:53:04.097 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 48% (119/247) Updating files: 49% (122/247) Updating files: 50% (124/247) Updating files: 51% (126/247) Updating files: 52% (129/247) Updating files: 53% (131/247) Updating files: 54% (134/247) Updating files: 55% (136/247) Updating files: 56% (139/247) Updating files: 57% (141/247) Updating files: 58% (144/247) Updating files: 59% (146/247) Updating files: 60% (149/247) Updating files: 61% (151/247) Updating files: 62% (154/247) Updating files: 63% (156/247) Updating files: 64% (159/247) Updating files: 65% (161/247) Updating files: 66% (164/247) Updating files: 67% (166/247) Updating files: 68% (168/247) Updating files: 69% (171/247) Updating files: 70% (173/247) Updating files: 71% (176/247) Updating files: 72% (178/247) Updating files: 73% (181/247) Updating files: 74% (183/247) Updating files: 75% (186/247) Updating files: 76% (188/247) Updating files: 77% (191/247) Updating files: 78% (193/247) Updating files: 79% (196/247) Updating files: 80% (198/247) Updating files: 81% (201/247) Updating files: 82% (203/247) Updating files: 83% (206/247) Updating files: 84% (208/247) Updating files: 85% (210/247) Updating files: 86% (213/247) Updating files: 87% (215/247) Updating files: 88% (218/247) Updating files: 89% (220/247) Updating files: 90% (223/247) Updating files: 91% (225/247) Updating files: 92% (228/247) Updating files: 93% (230/247) Updating files: 94% (233/247) Updating files: 95% (235/247) Updating files: 96% (238/247) Updating files: 97% (240/247) Updating files: 98% (243/247) Updating files: 99% (245/247) Updating files: 100% (247/247) Updating files: 100% (247/247), done. 2024-03-04T07:53:04.151 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 13b1bf7c492 kclient: initializing kaddr (merge fix) 2024-03-04T07:53:04.165 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.4 2024-03-04T07:53:04.952 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 5 -eq 48 ] 2024-03-04T07:53:04.952 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.5 2024-03-04T07:53:04.953 INFO:tasks.workunit.client.0.smithi113.stdout:v0.5 2024-03-04T07:53:04.953 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.5 2024-03-04T07:53:04.953 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.5 2024-03-04T07:53:09.457 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 13% (32/245) Updating files: 14% (35/245) Updating files: 15% (37/245) Updating files: 16% (40/245) Updating files: 17% (42/245) Updating files: 18% (45/245) Updating files: 18% (46/245) Updating files: 19% (47/245) Updating files: 20% (49/245) Updating files: 21% (52/245) Updating files: 22% (54/245) Updating files: 23% (57/245) Updating files: 24% (59/245) Updating files: 25% (62/245) Updating files: 26% (64/245) Updating files: 27% (67/245) Updating files: 28% (69/245) Updating files: 29% (72/245) Updating files: 30% (74/245) Updating files: 31% (76/245) Updating files: 32% (79/245) Updating files: 33% (81/245) Updating files: 34% (84/245) Updating files: 35% (86/245) Updating files: 36% (89/245) Updating files: 37% (91/245) Updating files: 38% (94/245) Updating files: 39% (96/245) Updating files: 39% (97/245) Updating files: 40% (98/245) Updating files: 41% (101/245) Updating files: 42% (103/245) Updating files: 43% (106/245) Updating files: 44% (108/245) Updating files: 45% (111/245) Updating files: 46% (113/245) Updating files: 47% (116/245) Updating files: 48% (118/245) Updating files: 49% (121/245) Updating files: 50% (123/245) Updating files: 51% (125/245) Updating files: 52% (128/245) Updating files: 53% (130/245) Updating files: 54% (133/245) Updating files: 55% (135/245) Updating files: 56% (138/245) Updating files: 57% (140/245) Updating files: 58% (143/245) Updating files: 59% (145/245) Updating files: 60% (147/245) Updating files: 61% (150/245) Updating files: 62% (152/245) Updating files: 63% (155/245) Updating files: 64% (157/245) Updating files: 65% (160/245) Updating files: 66% (162/245) Updating files: 67% (165/245) Updating files: 68% (167/245) Updating files: 69% (170/245) Updating files: 70% (172/245) Updating files: 71% (174/245) Updating files: 72% (177/245) Updating files: 73% (179/245) Updating files: 74% (182/245) Updating files: 75% (184/245) Updating files: 76% (187/245) Updating files: 77% (189/245) Updating files: 78% (192/245) Updating files: 79% (194/245) Updating files: 80% (196/245) Updating files: 81% (199/245) Updating files: 82% (201/245) Updating files: 83% (204/245) Updating files: 84% (206/245) Updating files: 85% (209/245) Updating files: 86% (211/245) Updating files: 87% (214/245) Updating files: 88% (216/245) Updating files: 89% (219/245) Updating files: 90% (221/245) Updating files: 91% (223/245) Updating files: 92% (226/245) Updating files: 93% (228/245) Updating files: 94% (231/245) Updating files: 95% (233/245) Updating files: 96% (236/245) Updating files: 97% (238/245) Updating files: 97% (239/245) Updating files: 98% (241/245) Updating files: 99% (243/245) Updating files: 100% (245/245) Updating files: 100% (245/245), done. 2024-03-04T07:53:09.497 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 1dd420957f9 journal: detect size of raw block devices properly 2024-03-04T07:53:09.503 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.5 2024-03-04T07:53:10.111 INFO:tasks.workunit.client.0.smithi113.stdout:v0.6 2024-03-04T07:53:10.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 6 -eq 48 ] 2024-03-04T07:53:10.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.6 2024-03-04T07:53:10.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.6 2024-03-04T07:53:10.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.6 2024-03-04T07:53:12.983 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 37% (86/227) Updating files: 38% (87/227) Updating files: 39% (89/227) Updating files: 40% (91/227) Updating files: 41% (94/227) Updating files: 42% (96/227) Updating files: 43% (98/227) Updating files: 44% (100/227) Updating files: 45% (103/227) Updating files: 46% (105/227) Updating files: 47% (107/227) Updating files: 48% (109/227) Updating files: 49% (112/227) Updating files: 50% (114/227) Updating files: 51% (116/227) Updating files: 52% (119/227) Updating files: 53% (121/227) Updating files: 54% (123/227) Updating files: 55% (125/227) Updating files: 56% (128/227) Updating files: 57% (130/227) Updating files: 58% (132/227) Updating files: 59% (134/227) Updating files: 60% (137/227) Updating files: 61% (139/227) Updating files: 62% (141/227) Updating files: 63% (144/227) Updating files: 64% (146/227) Updating files: 65% (148/227) Updating files: 66% (150/227) Updating files: 67% (153/227) Updating files: 68% (155/227) Updating files: 69% (157/227) Updating files: 70% (159/227) Updating files: 71% (162/227) Updating files: 72% (164/227) Updating files: 73% (166/227) Updating files: 74% (168/227) Updating files: 75% (171/227) Updating files: 76% (173/227) Updating files: 77% (175/227) Updating files: 78% (178/227) Updating files: 79% (180/227) Updating files: 80% (182/227) Updating files: 81% (184/227) Updating files: 82% (187/227) Updating files: 83% (189/227) Updating files: 84% (191/227) Updating files: 85% (193/227) Updating files: 86% (196/227) Updating files: 87% (198/227) Updating files: 88% (200/227) Updating files: 89% (203/227) Updating files: 90% (205/227) Updating files: 91% (207/227) Updating files: 92% (209/227) Updating files: 93% (212/227) Updating files: 94% (214/227) Updating files: 95% (216/227) Updating files: 96% (218/227) Updating files: 97% (221/227) Updating files: 98% (223/227) Updating files: 99% (225/227) Updating files: 100% (227/227) Updating files: 100% (227/227), done. 2024-03-04T07:53:13.014 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at a4c752df713 v0.6 2024-03-04T07:53:13.020 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.6 2024-03-04T07:53:13.081 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 7 -eq 48 ] 2024-03-04T07:53:13.082 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.7 2024-03-04T07:53:13.082 INFO:tasks.workunit.client.0.smithi113.stdout:v0.7 2024-03-04T07:53:13.082 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.7 2024-03-04T07:53:13.082 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.7 2024-03-04T07:53:28.679 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 8% (18/201) Updating files: 9% (19/201) Updating files: 10% (21/201) Updating files: 11% (23/201) Updating files: 12% (25/201) Updating files: 13% (27/201) Updating files: 14% (29/201) Updating files: 15% (31/201) Updating files: 16% (33/201) Updating files: 17% (35/201) Updating files: 18% (37/201) Updating files: 19% (39/201) Updating files: 20% (41/201) Updating files: 21% (43/201) Updating files: 22% (45/201) Updating files: 23% (47/201) Updating files: 24% (49/201) Updating files: 25% (51/201) Updating files: 26% (53/201) Updating files: 27% (55/201) Updating files: 28% (57/201) Updating files: 29% (59/201) Updating files: 30% (61/201) Updating files: 31% (63/201) Updating files: 32% (65/201) Updating files: 33% (67/201) Updating files: 34% (69/201) Updating files: 35% (71/201) Updating files: 36% (73/201) Updating files: 37% (75/201) Updating files: 38% (77/201) Updating files: 39% (79/201) Updating files: 40% (81/201) Updating files: 41% (83/201) Updating files: 42% (85/201) Updating files: 43% (87/201) Updating files: 43% (88/201) Updating files: 44% (89/201) Updating files: 45% (91/201) Updating files: 46% (93/201) Updating files: 47% (95/201) Updating files: 48% (97/201) Updating files: 49% (99/201) Updating files: 50% (101/201) Updating files: 51% (103/201) Updating files: 52% (105/201) Updating files: 53% (107/201) Updating files: 54% (109/201) Updating files: 55% (111/201) Updating files: 56% (113/201) Updating files: 57% (115/201) Updating files: 58% (117/201) Updating files: 59% (119/201) Updating files: 60% (121/201) Updating files: 61% (123/201) Updating files: 62% (125/201) Updating files: 63% (127/201) Updating files: 64% (129/201) Updating files: 65% (131/201) Updating files: 66% (133/201) Updating files: 67% (135/201) Updating files: 68% (137/201) Updating files: 69% (139/201) Updating files: 70% (141/201) Updating files: 71% (143/201) Updating files: 72% (145/201) Updating files: 73% (147/201) Updating files: 74% (149/201) Updating files: 75% (151/201) Updating files: 76% (153/201) Updating files: 77% (155/201) Updating files: 78% (157/201) Updating files: 79% (159/201) Updating files: 80% (161/201) Updating files: 81% (163/201) Updating files: 82% (165/201) Updating files: 83% (167/201) Updating files: 84% (169/201) Updating files: 85% (171/201) Updating files: 86% (173/201) Updating files: 87% (175/201) Updating files: 88% (177/201) Updating files: 89% (179/201) Updating files: 90% (181/201) Updating files: 91% (183/201) Updating files: 92% (185/201) Updating files: 93% (187/201) Updating files: 94% (189/201) Updating files: 95% (191/201) Updating files: 96% (193/201) Updating files: 97% (195/201) Updating files: 98% (197/201) Updating files: 99% (199/201) Updating files: 100% (201/201) Updating files: 100% (201/201), done. 2024-03-04T07:53:28.722 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at c46e1b320c6 mkcephfs: btrfsctl -a between mkfs and mount 2024-03-04T07:53:28.731 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.7 2024-03-04T07:53:29.264 INFO:tasks.workunit.client.0.smithi113.stdout:v0.8 2024-03-04T07:53:29.264 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 8 -eq 48 ] 2024-03-04T07:53:29.264 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.8 2024-03-04T07:53:29.264 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.8 2024-03-04T07:53:29.264 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.8 2024-03-04T07:53:29.639 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:53:29.642 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:53:41.188 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 21% (48/225) Updating files: 22% (50/225) Updating files: 23% (52/225) Updating files: 24% (54/225) Updating files: 25% (57/225) Updating files: 26% (59/225) Updating files: 27% (61/225) Updating files: 28% (63/225) Updating files: 29% (66/225) Updating files: 30% (68/225) Updating files: 31% (70/225) Updating files: 32% (72/225) Updating files: 33% (75/225) Updating files: 34% (77/225) Updating files: 35% (79/225) Updating files: 36% (81/225) Updating files: 37% (84/225) Updating files: 38% (86/225) Updating files: 39% (88/225) Updating files: 40% (90/225) Updating files: 41% (93/225) Updating files: 42% (95/225) Updating files: 43% (97/225) Updating files: 44% (99/225) Updating files: 45% (102/225) Updating files: 46% (104/225) Updating files: 47% (106/225) Updating files: 48% (108/225) Updating files: 49% (111/225) Updating files: 50% (113/225) Updating files: 51% (115/225) Updating files: 52% (117/225) Updating files: 53% (120/225) Updating files: 54% (122/225) Updating files: 55% (124/225) Updating files: 56% (126/225) Updating files: 57% (129/225) Updating files: 58% (131/225) Updating files: 59% (133/225) Updating files: 60% (135/225) Updating files: 61% (138/225) Updating files: 62% (140/225) Updating files: 63% (142/225) Updating files: 64% (144/225) Updating files: 65% (147/225) Updating files: 66% (149/225) Updating files: 67% (151/225) Updating files: 68% (153/225) Updating files: 69% (156/225) Updating files: 70% (158/225) Updating files: 71% (160/225) Updating files: 72% (162/225) Updating files: 73% (165/225) Updating files: 74% (167/225) Updating files: 75% (169/225) Updating files: 76% (171/225) Updating files: 77% (174/225) Updating files: 78% (176/225) Updating files: 79% (178/225) Updating files: 80% (180/225) Updating files: 80% (182/225) Updating files: 81% (183/225) Updating files: 81% (184/225) Updating files: 82% (185/225) Updating files: 83% (187/225) Updating files: 83% (188/225) Updating files: 84% (189/225) Updating files: 85% (192/225) Updating files: 86% (194/225) Updating files: 87% (196/225) Updating files: 88% (198/225) Updating files: 88% (200/225) Updating files: 89% (201/225) Updating files: 90% (203/225) Updating files: 91% (205/225) Updating files: 92% (207/225) Updating files: 93% (210/225) Updating files: 94% (212/225) Updating files: 95% (214/225) Updating files: 96% (216/225) Updating files: 97% (219/225) Updating files: 98% (221/225) Updating files: 99% (223/225) Updating files: 100% (225/225) Updating files: 100% (225/225), done. 2024-03-04T07:53:41.271 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 284161c00ab kclient: kill dput/d_drop debug cruft 2024-03-04T07:53:41.277 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.8 2024-03-04T07:53:41.299 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 9 -eq 48 ] 2024-03-04T07:53:41.299 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.9 2024-03-04T07:53:41.299 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.9 2024-03-04T07:53:41.299 INFO:tasks.workunit.client.0.smithi113.stdout:v0.9 2024-03-04T07:53:41.299 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.9 2024-03-04T07:53:44.123 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 60% (116/193) Updating files: 61% (118/193) Updating files: 62% (120/193) Updating files: 63% (122/193) Updating files: 64% (124/193) Updating files: 65% (126/193) Updating files: 66% (128/193) Updating files: 67% (130/193) Updating files: 68% (132/193) Updating files: 69% (134/193) Updating files: 70% (136/193) Updating files: 71% (138/193) Updating files: 72% (139/193) Updating files: 73% (141/193) Updating files: 74% (143/193) Updating files: 75% (145/193) Updating files: 76% (147/193) Updating files: 77% (149/193) Updating files: 78% (151/193) Updating files: 79% (153/193) Updating files: 80% (155/193) Updating files: 81% (157/193) Updating files: 82% (159/193) Updating files: 83% (161/193) Updating files: 84% (163/193) Updating files: 85% (165/193) Updating files: 86% (166/193) Updating files: 87% (168/193) Updating files: 88% (170/193) Updating files: 89% (172/193) Updating files: 90% (174/193) Updating files: 91% (176/193) Updating files: 92% (178/193) Updating files: 93% (180/193) Updating files: 94% (182/193) Updating files: 95% (184/193) Updating files: 96% (186/193) Updating files: 97% (188/193) Updating files: 98% (190/193) Updating files: 99% (192/193) Updating files: 100% (193/193) Updating files: 100% (193/193), done. 2024-03-04T07:53:44.401 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 41481853924 kclient: strip out kernel version compatibility cruft 2024-03-04T07:53:44.407 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.9 2024-03-04T07:53:45.122 INFO:tasks.workunit.client.0.smithi113.stdout:v0.10 2024-03-04T07:53:45.122 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 10 -eq 48 ] 2024-03-04T07:53:45.122 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.10 2024-03-04T07:53:45.122 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.10 2024-03-04T07:53:45.122 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.10 2024-03-04T07:53:48.571 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 33% (55/164) Updating files: 34% (56/164) Updating files: 35% (58/164) Updating files: 36% (60/164) Updating files: 37% (61/164) Updating files: 38% (63/164) Updating files: 39% (64/164) Updating files: 40% (66/164) Updating files: 41% (68/164) Updating files: 42% (69/164) Updating files: 43% (71/164) Updating files: 44% (73/164) Updating files: 45% (74/164) Updating files: 46% (76/164) Updating files: 47% (78/164) Updating files: 48% (79/164) Updating files: 49% (81/164) Updating files: 50% (82/164) Updating files: 51% (84/164) Updating files: 52% (86/164) Updating files: 53% (87/164) Updating files: 54% (89/164) Updating files: 55% (91/164) Updating files: 56% (92/164) Updating files: 57% (94/164) Updating files: 58% (96/164) Updating files: 59% (97/164) Updating files: 60% (99/164) Updating files: 61% (101/164) Updating files: 62% (102/164) Updating files: 63% (104/164) Updating files: 64% (105/164) Updating files: 65% (107/164) Updating files: 66% (109/164) Updating files: 67% (110/164) Updating files: 68% (112/164) Updating files: 69% (114/164) Updating files: 70% (115/164) Updating files: 71% (117/164) Updating files: 71% (118/164) Updating files: 72% (119/164) Updating files: 73% (120/164) Updating files: 74% (122/164) Updating files: 75% (123/164) Updating files: 76% (125/164) Updating files: 77% (127/164) Updating files: 78% (128/164) Updating files: 79% (130/164) Updating files: 80% (132/164) Updating files: 81% (133/164) Updating files: 82% (135/164) Updating files: 83% (137/164) Updating files: 84% (138/164) Updating files: 85% (140/164) Updating files: 86% (142/164) Updating files: 87% (143/164) Updating files: 88% (145/164) Updating files: 89% (146/164) Updating files: 90% (148/164) Updating files: 91% (150/164) Updating files: 92% (151/164) Updating files: 93% (153/164) Updating files: 94% (155/164) Updating files: 95% (156/164) Updating files: 96% (158/164) Updating files: 97% (160/164) Updating files: 98% (161/164) Updating files: 99% (163/164) Updating files: 100% (164/164) Updating files: 100% (164/164), done. 2024-03-04T07:53:48.602 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 007313fe844 v0.10 2024-03-04T07:53:48.608 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.10 2024-03-04T07:53:48.683 INFO:tasks.workunit.client.0.smithi113.stdout:v0.11 2024-03-04T07:53:48.684 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 11 -eq 48 ] 2024-03-04T07:53:48.684 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.11 2024-03-04T07:53:48.684 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.11 2024-03-04T07:53:48.684 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.11 2024-03-04T07:53:49.721 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 9fa6f5e7d09 v0.11 2024-03-04T07:53:49.724 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.11 2024-03-04T07:53:49.986 INFO:tasks.workunit.client.0.smithi113.stdout:v0.12 2024-03-04T07:53:49.987 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 12 -eq 48 ] 2024-03-04T07:53:49.987 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.12 2024-03-04T07:53:49.987 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.12 2024-03-04T07:53:49.987 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.12 2024-03-04T07:53:51.047 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at f524150822a v0.12 2024-03-04T07:53:51.056 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.12 2024-03-04T07:53:52.131 INFO:tasks.workunit.client.0.smithi113.stdout:v0.13 2024-03-04T07:53:52.131 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 13 -eq 48 ] 2024-03-04T07:53:52.132 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.13 2024-03-04T07:53:52.132 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.13 2024-03-04T07:53:52.132 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.13 2024-03-04T07:53:53.567 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 17% (23/133) Updating files: 18% (24/133) Updating files: 19% (26/133) Updating files: 20% (27/133) Updating files: 21% (28/133) Updating files: 22% (30/133) Updating files: 23% (31/133) Updating files: 24% (32/133) Updating files: 25% (34/133) Updating files: 26% (35/133) Updating files: 27% (36/133) Updating files: 28% (38/133) Updating files: 29% (39/133) Updating files: 30% (40/133) Updating files: 31% (42/133) Updating files: 32% (43/133) Updating files: 33% (44/133) Updating files: 34% (46/133) Updating files: 35% (47/133) Updating files: 36% (48/133) Updating files: 37% (50/133) Updating files: 38% (51/133) Updating files: 39% (52/133) Updating files: 40% (54/133) Updating files: 41% (55/133) Updating files: 42% (56/133) Updating files: 43% (58/133) Updating files: 44% (59/133) Updating files: 45% (60/133) Updating files: 46% (62/133) Updating files: 47% (63/133) Updating files: 48% (64/133) Updating files: 49% (66/133) Updating files: 50% (67/133) Updating files: 51% (68/133) Updating files: 52% (70/133) Updating files: 53% (71/133) Updating files: 54% (72/133) Updating files: 55% (74/133) Updating files: 56% (75/133) Updating files: 57% (76/133) Updating files: 58% (78/133) Updating files: 59% (79/133) Updating files: 60% (80/133) Updating files: 61% (82/133) Updating files: 62% (83/133) Updating files: 63% (84/133) Updating files: 64% (86/133) Updating files: 65% (87/133) Updating files: 66% (88/133) Updating files: 67% (90/133) Updating files: 68% (91/133) Updating files: 69% (92/133) Updating files: 70% (94/133) Updating files: 71% (95/133) Updating files: 72% (96/133) Updating files: 73% (98/133) Updating files: 74% (99/133) Updating files: 75% (100/133) Updating files: 76% (102/133) Updating files: 77% (103/133) Updating files: 78% (104/133) Updating files: 79% (106/133) Updating files: 80% (107/133) Updating files: 81% (108/133) Updating files: 82% (110/133) Updating files: 83% (111/133) Updating files: 84% (112/133) Updating files: 85% (114/133) Updating files: 86% (115/133) Updating files: 87% (116/133) Updating files: 88% (118/133) Updating files: 89% (119/133) Updating files: 90% (120/133) Updating files: 91% (122/133) Updating files: 92% (123/133) Updating files: 93% (124/133) Updating files: 94% (126/133) Updating files: 95% (127/133) Updating files: 96% (128/133) Updating files: 97% (130/133) Updating files: 98% (131/133) Updating files: 99% (132/133) Updating files: 100% (133/133) Updating files: 100% (133/133), done. 2024-03-04T07:53:53.595 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 2b32297267d v0.13 2024-03-04T07:53:53.600 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.13 2024-03-04T07:53:54.114 INFO:tasks.workunit.client.0.smithi113.stdout:v0.14 2024-03-04T07:53:54.114 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 14 -eq 48 ] 2024-03-04T07:53:54.114 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.14 2024-03-04T07:53:54.114 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.14 2024-03-04T07:53:54.114 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.14 2024-03-04T07:54:00.874 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:54:00.878 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:54:08.194 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 69% (105/151) Updating files: 70% (106/151) Updating files: 71% (108/151) Updating files: 72% (109/151) Updating files: 73% (111/151) Updating files: 74% (112/151) Updating files: 75% (114/151) Updating files: 76% (115/151) Updating files: 77% (117/151) Updating files: 78% (118/151) Updating files: 79% (120/151) Updating files: 80% (121/151) Updating files: 81% (123/151) Updating files: 82% (124/151) Updating files: 83% (126/151) Updating files: 84% (127/151) Updating files: 85% (129/151) Updating files: 86% (130/151) Updating files: 87% (132/151) Updating files: 88% (133/151) Updating files: 89% (135/151) Updating files: 90% (136/151) Updating files: 91% (138/151) Updating files: 92% (139/151) Updating files: 93% (141/151) Updating files: 94% (142/151) Updating files: 95% (144/151) Updating files: 96% (145/151) Updating files: 97% (147/151) Updating files: 98% (148/151) Updating files: 99% (150/151) Updating files: 100% (151/151) Updating files: 100% (151/151), done. 2024-03-04T07:54:08.231 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 1ec1e8d6fb6 v0.14 2024-03-04T07:54:08.235 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.14 2024-03-04T07:54:08.341 INFO:tasks.workunit.client.0.smithi113.stdout:v0.15 2024-03-04T07:54:08.341 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 15 -eq 48 ] 2024-03-04T07:54:08.341 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.15 2024-03-04T07:54:08.341 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.15 2024-03-04T07:54:08.341 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.15 2024-03-04T07:54:09.760 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at ab3c421eb95 v0.15 2024-03-04T07:54:09.764 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.15 2024-03-04T07:54:10.115 INFO:tasks.workunit.client.0.smithi113.stdout:v0.16 2024-03-04T07:54:10.115 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 16 -eq 48 ] 2024-03-04T07:54:10.116 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.16 2024-03-04T07:54:10.116 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.16 2024-03-04T07:54:10.116 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.16 2024-03-04T07:54:27.124 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 94% (101/107) Updating files: 95% (102/107) Updating files: 96% (103/107) Updating files: 97% (104/107) Updating files: 98% (105/107) Updating files: 99% (106/107) Updating files: 100% (107/107) Updating files: 100% (107/107), done. 2024-03-04T07:54:27.160 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 38dbaa5175c v0.16 2024-03-04T07:54:27.164 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.16 2024-03-04T07:54:27.199 INFO:tasks.workunit.client.0.smithi113.stdout:v0.17 2024-03-04T07:54:27.199 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 17 -eq 48 ] 2024-03-04T07:54:27.200 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.17 2024-03-04T07:54:27.200 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.17 2024-03-04T07:54:27.200 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.17 2024-03-04T07:54:28.992 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:29.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.140+0000 7f479b200640 1 Processor -- start 2024-03-04T07:54:29.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.140+0000 7f479b200640 1 -- start start 2024-03-04T07:54:29.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f479b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f479c152950 0x7f479c15bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f479b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f479c06faa0 0x7f479c0c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f479b200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f479c06ed80 0x7f479c06f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f479b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479c0c8370 con 0x7f479c152950 2024-03-04T07:54:29.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f479b200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479c0ca0a0 con 0x7f479c06ed80 2024-03-04T07:54:29.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f479b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0c4ce0 con 0x7f479c06faa0 2024-03-04T07:54:29.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f4799e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f479c152950 0x7f479c15bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f4799e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f479c152950 0x7f479c15bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38362/0 (socket says 172.21.15.113:38362) 2024-03-04T07:54:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f4793e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f479c06faa0 0x7f479c0c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f4799e00640 1 -- 172.21.15.113:0/1459433463 learned_addr learned my addr 172.21.15.113:0/1459433463 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f479a800640 1 --2- 172.21.15.113:0/1459433463 >> v2:172.21.15.136:3300/0 conn(0x7f479c06ed80 0x7f479c06f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f4799e00640 1 -- 172.21.15.113:0/1459433463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f479c06faa0 msgr2=0x7f479c0c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f4799e00640 1 --2- 172.21.15.113:0/1459433463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f479c06faa0 0x7f479c0c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f4799e00640 1 -- 172.21.15.113:0/1459433463 >> v2:172.21.15.136:3300/0 conn(0x7f479c06ed80 msgr2=0x7f479c06f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f4799e00640 1 --2- 172.21.15.113:0/1459433463 >> v2:172.21.15.136:3300/0 conn(0x7f479c06ed80 0x7f479c06f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.144+0000 7f4799e00640 1 -- 172.21.15.113:0/1459433463 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0c4f80 con 0x7f479c152950 2024-03-04T07:54:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f4799e00640 1 --2- 172.21.15.113:0/1459433463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 0x7f479c15bb40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f479c0c9510 tx=0x7f478c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3cb2f623236297 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f4799400640 1 -- 172.21.15.113:0/1459433463 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c0b5070 con 0x7f479c152950 2024-03-04T07:54:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f4799400640 1 -- 172.21.15.113:0/1459433463 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478c0a89d0 con 0x7f479c152950 2024-03-04T07:54:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f4799400640 1 -- 172.21.15.113:0/1459433463 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478c0a8da0 con 0x7f479c152950 2024-03-04T07:54:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f479b200640 1 -- 172.21.15.113:0/1459433463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 msgr2=0x7f479c15bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f479b200640 1 --2- 172.21.15.113:0/1459433463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 0x7f479c15bb40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f479c0c9510 tx=0x7f478c01ed10 comp rx=0 tx=0).stop 2024-03-04T07:54:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f479b200640 1 -- 172.21.15.113:0/1459433463 shutdown_connections 2024-03-04T07:54:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f479b200640 1 --2- 172.21.15.113:0/1459433463 >> v2:172.21.15.136:3300/0 conn(0x7f479c06ed80 0x7f479c06f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f479b200640 1 --2- 172.21.15.113:0/1459433463 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f479c06faa0 0x7f479c0c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f479b200640 1 --2- 172.21.15.113:0/1459433463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 0x7f479c15bb40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f479b200640 1 -- 172.21.15.113:0/1459433463 >> 172.21.15.113:0/1459433463 conn(0x7f479c072f40 msgr2=0x7f479c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f479b200640 1 -- 172.21.15.113:0/1459433463 shutdown_connections 2024-03-04T07:54:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.148+0000 7f479b200640 1 -- 172.21.15.113:0/1459433463 wait complete. 2024-03-04T07:54:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479b200640 1 Processor -- start 2024-03-04T07:54:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479b200640 1 -- start start 2024-03-04T07:54:29.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c06ed80 0x7f479c1f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f479c06faa0 0x7f479c0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 0x7f479c1f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0c9a50 con 0x7f479c152950 2024-03-04T07:54:29.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c06fe80 con 0x7f479c06ed80 2024-03-04T07:54:29.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0c9d30 con 0x7f479c06faa0 2024-03-04T07:54:29.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 0x7f479c1f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 0x7f479c1f0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38372/0 (socket says 172.21.15.113:38372) 2024-03-04T07:54:29.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479a800640 1 -- 172.21.15.113:0/1643924804 learned_addr learned my addr 172.21.15.113:0/1643924804 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:29.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f4793e00640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f479c06faa0 0x7f479c0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f4799e00640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c06ed80 0x7f479c1f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479a800640 1 -- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f479c06faa0 msgr2=0x7f479c0c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479a800640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f479c06faa0 0x7f479c0c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479a800640 1 -- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c06ed80 msgr2=0x7f479c1f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479a800640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c06ed80 0x7f479c1f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479a800640 1 -- 172.21.15.113:0/1643924804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c1ee4c0 con 0x7f479c152950 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479a800640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 0x7f479c1f0cc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f479400eed0 tx=0x7f479400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f4792000640 1 -- 172.21.15.113:0/1643924804 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794018020 con 0x7f479c152950 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f4792000640 1 -- 172.21.15.113:0/1643924804 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479400bd20 con 0x7f479c152950 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479b200640 1 -- 172.21.15.113:0/1643924804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479c1f04e0 con 0x7f479c152950 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f4792000640 1 -- 172.21.15.113:0/1643924804 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794014640 con 0x7f479c152950 2024-03-04T07:54:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.152+0000 7f479b200640 1 -- 172.21.15.113:0/1643924804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479c0cd660 con 0x7f479c152950 2024-03-04T07:54:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.156+0000 7f479b200640 1 -- 172.21.15.113:0/1643924804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47740052a0 con 0x7f479c152950 2024-03-04T07:54:29.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.156+0000 7f4792000640 1 -- 172.21.15.113:0/1643924804 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f479401b020 con 0x7f479c152950 2024-03-04T07:54:29.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.156+0000 7f4792000640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4778066f60 0x7f4778069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.160+0000 7f4792000640 1 -- 172.21.15.113:0/1643924804 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f4794085b20 con 0x7f479c152950 2024-03-04T07:54:29.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.160+0000 7f4799e00640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4778066f60 0x7f4778069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.160+0000 7f4799e00640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4778066f60 0x7f4778069420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f478c0b2000 tx=0x7f478c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:29.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.164+0000 7f4792000640 1 -- 172.21.15.113:0/1643924804 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4794022030 con 0x7f479c152950 2024-03-04T07:54:29.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.396+0000 7f479b200640 1 -- 172.21.15.113:0/1643924804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4774005860 con 0x7f479c152950 2024-03-04T07:54:29.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.396+0000 7f4792000640 1 -- 172.21.15.113:0/1643924804 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+6609 (secure 0 0 0) 0x7f4794056110 con 0x7f479c152950 2024-03-04T07:54:29.394 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:29.394 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6240,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:30.928216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940,5631]},"id":1}]} 2024-03-04T07:54:29.394 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 76 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 -- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4778066f60 msgr2=0x7f4778069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4778066f60 0x7f4778069420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f478c0b2000 tx=0x7f478c009040 comp rx=0 tx=0).stop 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 -- 172.21.15.113:0/1643924804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 msgr2=0x7f479c1f0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 0x7f479c1f0cc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f479400eed0 tx=0x7f479400b480 comp rx=0 tx=0).stop 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 -- 172.21.15.113:0/1643924804 shutdown_connections 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4778066f60 0x7f4778069420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f479c152950 0x7f479c1f0cc0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f479c06faa0 0x7f479c0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 --2- 172.21.15.113:0/1643924804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f479c06ed80 0x7f479c1f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 -- 172.21.15.113:0/1643924804 >> 172.21.15.113:0/1643924804 conn(0x7f479c072f40 msgr2=0x7f479c159920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 -- 172.21.15.113:0/1643924804 shutdown_connections 2024-03-04T07:54:29.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.400+0000 7f479b200640 1 -- 172.21.15.113:0/1643924804 wait complete. 2024-03-04T07:54:29.416 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=2) 2024-03-04T07:54:29.416 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-04T07:54:29.416 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:54:29.431 INFO:tasks.ceph.mds.c:Stopped 2024-03-04T07:54:29.431 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd82a00640 1 Processor -- start 2024-03-04T07:54:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd82a00640 1 -- start start 2024-03-04T07:54:29.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd82a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdd7c1527f0 0x7fdd7c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd82a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdd7c0c8200 0x7fdd7c072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd82a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7c1591f0 con 0x7fdd7c152bd0 2024-03-04T07:54:29.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd82a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7c071df0 con 0x7fdd7c0c8200 2024-03-04T07:54:29.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd82a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7c1597b0 con 0x7fdd7c1527f0 2024-03-04T07:54:29.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd73e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdd7c0c8200 0x7fdd7c072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd73e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdd7c0c8200 0x7fdd7c072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38376/0 (socket says 172.21.15.113:38376) 2024-03-04T07:54:29.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd73e00640 1 -- 172.21.15.113:0/1586609913 learned_addr learned my addr 172.21.15.113:0/1586609913 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:29.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd7be00640 1 --2- 172.21.15.113:0/1586609913 >> v2:172.21.15.136:3300/0 conn(0x7fdd7c1527f0 0x7fdd7c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd73e00640 1 -- 172.21.15.113:0/1586609913 >> v2:172.21.15.136:3300/0 conn(0x7fdd7c1527f0 msgr2=0x7fdd7c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:29.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd73e00640 1 --2- 172.21.15.113:0/1586609913 >> v2:172.21.15.136:3300/0 conn(0x7fdd7c1527f0 0x7fdd7c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd73e00640 1 -- 172.21.15.113:0/1586609913 >> v1:172.21.15.136:6789/0 conn(0x7fdd7c152bd0 legacy=0x7fdd7c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd73e00640 1 -- 172.21.15.113:0/1586609913 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c1594d0 con 0x7fdd7c0c8200 2024-03-04T07:54:29.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.616+0000 7fdd7be00640 1 --2- 172.21.15.113:0/1586609913 >> v2:172.21.15.136:3300/0 conn(0x7fdd7c1527f0 0x7fdd7c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:54:29.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.620+0000 7fdd73e00640 1 --2- 172.21.15.113:0/1586609913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c0c8200 0x7fdd7c072280 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c06f400 tx=0x7fdd6401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e5f0abd4a31100 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:29.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd7b400640 1 -- 172.21.15.113:0/1586609913 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640ab430 con 0x7fdd7c0c8200 2024-03-04T07:54:29.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd7b400640 1 -- 172.21.15.113:0/1586609913 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd640b0070 con 0x7fdd7c0c8200 2024-03-04T07:54:29.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd7b400640 1 -- 172.21.15.113:0/1586609913 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640ab7f0 con 0x7fdd7c0c8200 2024-03-04T07:54:29.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 -- 172.21.15.113:0/1586609913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c0c8200 msgr2=0x7fdd7c072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 --2- 172.21.15.113:0/1586609913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c0c8200 0x7fdd7c072280 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c06f400 tx=0x7fdd6401c520 comp rx=0 tx=0).stop 2024-03-04T07:54:29.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 -- 172.21.15.113:0/1586609913 shutdown_connections 2024-03-04T07:54:29.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 --2- 172.21.15.113:0/1586609913 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c0c8200 0x7fdd7c072280 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 --2- 172.21.15.113:0/1586609913 >> v2:172.21.15.136:3300/0 conn(0x7fdd7c1527f0 0x7fdd7c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 -- 172.21.15.113:0/1586609913 >> 172.21.15.113:0/1586609913 conn(0x7fdd7c06e820 msgr2=0x7fdd7c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:29.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 -- 172.21.15.113:0/1586609913 shutdown_connections 2024-03-04T07:54:29.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 -- 172.21.15.113:0/1586609913 wait complete. 2024-03-04T07:54:29.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 Processor -- start 2024-03-04T07:54:29.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 -- start start 2024-03-04T07:54:29.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd7c0c8200 0x7fdd7c1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c1527f0 0x7fdd7c1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd7c152bd0 0x7fdd7c14a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c165160 con 0x7fdd7c1527f0 2024-03-04T07:54:29.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c0727c0 con 0x7fdd7c152bd0 2024-03-04T07:54:29.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd82a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c1652e0 con 0x7fdd7c0c8200 2024-03-04T07:54:29.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd73e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c1527f0 0x7fdd7c1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd73e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c1527f0 0x7fdd7c1565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38380/0 (socket says 172.21.15.113:38380) 2024-03-04T07:54:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd73e00640 1 -- 172.21.15.113:0/522660315 learned_addr learned my addr 172.21.15.113:0/522660315 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd7be00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd7c0c8200 0x7fdd7c1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.624+0000 7fdd80a00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd7c152bd0 0x7fdd7c14a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd73e00640 1 -- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd7c0c8200 msgr2=0x7fdd7c1500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd73e00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd7c0c8200 0x7fdd7c1500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd73e00640 1 -- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd7c152bd0 msgr2=0x7fdd7c14a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd73e00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd7c152bd0 0x7fdd7c14a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd73e00640 1 -- 172.21.15.113:0/522660315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c14bac0 con 0x7fdd7c1527f0 2024-03-04T07:54:29.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd73e00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c1527f0 0x7fdd7c1565d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdd640ad040 tx=0x7fdd6401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:29.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd78c00640 1 -- 172.21.15.113:0/522660315 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640be5f0 con 0x7fdd7c1527f0 2024-03-04T07:54:29.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd78c00640 1 -- 172.21.15.113:0/522660315 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd640be790 con 0x7fdd7c1527f0 2024-03-04T07:54:29.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd78c00640 1 -- 172.21.15.113:0/522660315 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640beb60 con 0x7fdd7c1527f0 2024-03-04T07:54:29.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd82a00640 1 -- 172.21.15.113:0/522660315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7c1f4aa0 con 0x7fdd7c1527f0 2024-03-04T07:54:29.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd82a00640 1 -- 172.21.15.113:0/522660315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c14c060 con 0x7fdd7c1527f0 2024-03-04T07:54:29.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd82a00640 1 -- 172.21.15.113:0/522660315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7c075c00 con 0x7fdd7c1527f0 2024-03-04T07:54:29.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd78c00640 1 -- 172.21.15.113:0/522660315 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd640b43f0 con 0x7fdd7c1527f0 2024-03-04T07:54:29.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd78c00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd58066f90 0x7fdd58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:29.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.628+0000 7fdd78c00640 1 -- 172.21.15.113:0/522660315 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fdd640a8030 con 0x7fdd7c1527f0 2024-03-04T07:54:29.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.632+0000 7fdd7be00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd58066f90 0x7fdd58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:29.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.632+0000 7fdd7be00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd58066f90 0x7fdd58069450 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009950 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:29.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.636+0000 7fdd78c00640 1 -- 172.21.15.113:0/522660315 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdd64016110 con 0x7fdd7c1527f0 2024-03-04T07:54:29.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 -- 172.21.15.113:0/522660315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd7c075c00 con 0x7fdd7c1527f0 2024-03-04T07:54:29.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd78c00640 1 -- 172.21.15.113:0/522660315 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+6609 (secure 0 0 0) 0x7fdd640f0150 con 0x7fdd7c1527f0 2024-03-04T07:54:29.819 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:29.820 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6240,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:30.928216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940,5631]},"id":1}]} 2024-03-04T07:54:29.820 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 76 2024-03-04T07:54:29.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 -- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd58066f90 msgr2=0x7fdd58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd58066f90 0x7fdd58069450 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009950 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 -- 172.21.15.113:0/522660315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c1527f0 msgr2=0x7fdd7c1565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c1527f0 0x7fdd7c1565d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdd640ad040 tx=0x7fdd6401ad60 comp rx=0 tx=0).stop 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 -- 172.21.15.113:0/522660315 shutdown_connections 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd7c152bd0 0x7fdd7c14a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdd58066f90 0x7fdd58069450 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd7c1527f0 0x7fdd7c1565d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 --2- 172.21.15.113:0/522660315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd7c0c8200 0x7fdd7c1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 -- 172.21.15.113:0/522660315 >> 172.21.15.113:0/522660315 conn(0x7fdd7c06e820 msgr2=0x7fdd7c0c1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 -- 172.21.15.113:0/522660315 shutdown_connections 2024-03-04T07:54:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:29.824+0000 7fdd82a00640 1 -- 172.21.15.113:0/522660315 wait complete. 2024-03-04T07:54:29.841 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:54:31.009 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:54:31.012 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:54:31.283 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 38% (84/217) Updating files: 39% (85/217) Updating files: 40% (87/217) Updating files: 41% (89/217) Updating files: 42% (92/217) Updating files: 43% (94/217) Updating files: 44% (96/217) Updating files: 45% (98/217) Updating files: 46% (100/217) Updating files: 47% (102/217) Updating files: 48% (105/217) Updating files: 48% (106/217) Updating files: 49% (107/217) Updating files: 50% (109/217) Updating files: 51% (111/217) Updating files: 52% (113/217) Updating files: 53% (116/217) Updating files: 54% (118/217) Updating files: 55% (120/217) Updating files: 56% (122/217) Updating files: 57% (124/217) Updating files: 58% (126/217) Updating files: 59% (129/217) Updating files: 60% (131/217) Updating files: 61% (133/217) Updating files: 62% (135/217) Updating files: 63% (137/217) Updating files: 64% (139/217) Updating files: 65% (142/217) Updating files: 66% (144/217) Updating files: 67% (146/217) Updating files: 68% (148/217) Updating files: 69% (150/217) Updating files: 70% (152/217) Updating files: 71% (155/217) Updating files: 71% (156/217) Updating files: 72% (157/217) Updating files: 73% (159/217) Updating files: 74% (161/217) Updating files: 75% (163/217) Updating files: 76% (165/217) Updating files: 77% (168/217) Updating files: 78% (170/217) Updating files: 79% (172/217) Updating files: 80% (174/217) Updating files: 81% (176/217) Updating files: 82% (178/217) Updating files: 83% (181/217) Updating files: 84% (183/217) Updating files: 85% (185/217) Updating files: 86% (187/217) Updating files: 87% (189/217) Updating files: 88% (191/217) Updating files: 89% (194/217) Updating files: 90% (196/217) Updating files: 91% (198/217) Updating files: 92% (200/217) Updating files: 93% (202/217) Updating files: 94% (204/217) Updating files: 95% (207/217) Updating files: 96% (209/217) Updating files: 97% (211/217) Updating files: 98% (213/217) Updating files: 99% (215/217) Updating files: 100% (217/217) Updating files: 100% (217/217), done. 2024-03-04T07:54:31.311 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at dac3a8c35fb v0.17 2024-03-04T07:54:31.315 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.17 2024-03-04T07:54:31.842 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:31.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd98dc00640 1 Processor -- start 2024-03-04T07:54:31.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd98dc00640 1 -- start start 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd98dc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd98806df10 0x7fd98815e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd98dc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd9880703e0 0x7fd9880ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd98dc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98806c310 con 0x7fd988158230 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd98dc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9880c95f0 con 0x7fd98806df10 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd98dc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98806c5b0 con 0x7fd9880703e0 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd97fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd98806df10 0x7fd98815e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd97fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd98806df10 0x7fd98815e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38384/0 (socket says 172.21.15.113:38384) 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd97fe00640 1 -- 172.21.15.113:0/3425168152 learned_addr learned my addr 172.21.15.113:0/3425168152 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd986a00640 1 --1- 172.21.15.113:0/3425168152 >> v1:172.21.15.113:6789/0 conn(0x7fd988158230 0x7fd98806f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39366/0 (socket says 172.21.15.113:39366) 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd987400640 1 --2- 172.21.15.113:0/3425168152 >> v2:172.21.15.136:3300/0 conn(0x7fd9880703e0 0x7fd9880ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd97fe00640 1 -- 172.21.15.113:0/3425168152 >> v2:172.21.15.136:3300/0 conn(0x7fd9880703e0 msgr2=0x7fd9880ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd97fe00640 1 --2- 172.21.15.113:0/3425168152 >> v2:172.21.15.136:3300/0 conn(0x7fd9880703e0 0x7fd9880ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd97fe00640 1 -- 172.21.15.113:0/3425168152 >> v1:172.21.15.113:6789/0 conn(0x7fd988158230 legacy=0x7fd98806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.988+0000 7fd97fe00640 1 -- 172.21.15.113:0/3425168152 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880c9890 con 0x7fd98806df10 2024-03-04T07:54:31.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd97fe00640 1 --2- 172.21.15.113:0/3425168152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd98806df10 0x7fd98815e190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd9880c9010 tx=0x7fd97001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca45b2d2a2aac851 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd986000640 1 -- 172.21.15.113:0/3425168152 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700b5070 con 0x7fd98806df10 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd986000640 1 -- 172.21.15.113:0/3425168152 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9700a8a10 con 0x7fd98806df10 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd986000640 1 -- 172.21.15.113:0/3425168152 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9700a8de0 con 0x7fd98806df10 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd98dc00640 1 -- 172.21.15.113:0/3425168152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd98806df10 msgr2=0x7fd98815e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd98dc00640 1 --2- 172.21.15.113:0/3425168152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd98806df10 0x7fd98815e190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd9880c9010 tx=0x7fd97001ed10 comp rx=0 tx=0).stop 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd98dc00640 1 -- 172.21.15.113:0/3425168152 shutdown_connections 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd98dc00640 1 --2- 172.21.15.113:0/3425168152 >> v2:172.21.15.136:3300/0 conn(0x7fd9880703e0 0x7fd9880ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd98dc00640 1 --2- 172.21.15.113:0/3425168152 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd98806df10 0x7fd98815e190 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd98dc00640 1 -- 172.21.15.113:0/3425168152 >> 172.21.15.113:0/3425168152 conn(0x7fd988066db0 msgr2=0x7fd9880671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd98dc00640 1 -- 172.21.15.113:0/3425168152 shutdown_connections 2024-03-04T07:54:31.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.992+0000 7fd98dc00640 1 -- 172.21.15.113:0/3425168152 wait complete. 2024-03-04T07:54:31.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd98dc00640 1 Processor -- start 2024-03-04T07:54:31.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd98dc00640 1 -- start start 2024-03-04T07:54:31.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd98dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd98806df10 0x7fd988151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:31.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd98dc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9880703e0 0x7fd9880c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd98dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd988158230 0x7fd9880c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd98dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98806ca10 con 0x7fd988158230 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd98dc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98806c890 con 0x7fd98806df10 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd98dc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98815e6d0 con 0x7fd9880703e0 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd987400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd988158230 0x7fd9880c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd987400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd988158230 0x7fd9880c7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38394/0 (socket says 172.21.15.113:38394) 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd987400640 1 -- 172.21.15.113:0/3859695086 learned_addr learned my addr 172.21.15.113:0/3859695086 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd986a00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd98806df10 0x7fd988151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd97fe00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9880703e0 0x7fd9880c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd987400640 1 -- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9880703e0 msgr2=0x7fd9880c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd987400640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9880703e0 0x7fd9880c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd987400640 1 -- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd98806df10 msgr2=0x7fd988151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:31.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd987400640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd98806df10 0x7fd988151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:31.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd987400640 1 -- 172.21.15.113:0/3859695086 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98815b8c0 con 0x7fd988158230 2024-03-04T07:54:31.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd97fe00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9880703e0 0x7fd9880c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:54:31.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd987400640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd988158230 0x7fd9880c7330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd980007d90 tx=0x7fd98000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:31.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd97ea00640 1 -- 172.21.15.113:0/3859695086 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980014420 con 0x7fd988158230 2024-03-04T07:54:31.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd97ea00640 1 -- 172.21.15.113:0/3859695086 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd980019070 con 0x7fd988158230 2024-03-04T07:54:31.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd98dc00640 1 -- 172.21.15.113:0/3859695086 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9881f4210 con 0x7fd988158230 2024-03-04T07:54:31.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd97ea00640 1 -- 172.21.15.113:0/3859695086 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd980014820 con 0x7fd988158230 2024-03-04T07:54:31.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:31.996+0000 7fd98dc00640 1 -- 172.21.15.113:0/3859695086 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd988158e80 con 0x7fd988158230 2024-03-04T07:54:31.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.000+0000 7fd98dc00640 1 -- 172.21.15.113:0/3859695086 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd988075c00 con 0x7fd988158230 2024-03-04T07:54:31.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.000+0000 7fd97ea00640 1 -- 172.21.15.113:0/3859695086 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9800149c0 con 0x7fd988158230 2024-03-04T07:54:31.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.000+0000 7fd97ea00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd964066f90 0x7fd964069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:31.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.000+0000 7fd97ea00640 1 -- 172.21.15.113:0/3859695086 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7fd980085ac0 con 0x7fd988158230 2024-03-04T07:54:31.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.000+0000 7fd986a00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd964066f90 0x7fd964069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:31.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.000+0000 7fd986a00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd964066f90 0x7fd964069450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd978000f70 tx=0x7fd978007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:32.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.012+0000 7fd97ea00640 1 -- 172.21.15.113:0/3859695086 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd980056790 con 0x7fd988158230 2024-03-04T07:54:32.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 -- 172.21.15.113:0/3859695086 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9880701c0 con 0x7fd988158230 2024-03-04T07:54:32.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd97ea00640 1 -- 172.21.15.113:0/3859695086 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+6609 (secure 0 0 0) 0x7fd980056090 con 0x7fd988158230 2024-03-04T07:54:32.200 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:32.200 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6240,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:30.928216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940,5631]},"id":1}]} 2024-03-04T07:54:32.200 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 76 2024-03-04T07:54:32.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 -- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd964066f90 msgr2=0x7fd964069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd964066f90 0x7fd964069450 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd978000f70 tx=0x7fd978007e30 comp rx=0 tx=0).stop 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 -- 172.21.15.113:0/3859695086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd988158230 msgr2=0x7fd9880c7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd988158230 0x7fd9880c7330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd980007d90 tx=0x7fd98000e4b0 comp rx=0 tx=0).stop 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 -- 172.21.15.113:0/3859695086 shutdown_connections 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd964066f90 0x7fd964069450 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd988158230 0x7fd9880c7330 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd9880703e0 0x7fd9880c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 --2- 172.21.15.113:0/3859695086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd98806df10 0x7fd988151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 -- 172.21.15.113:0/3859695086 >> 172.21.15.113:0/3859695086 conn(0x7fd988066db0 msgr2=0x7fd9880cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 -- 172.21.15.113:0/3859695086 shutdown_connections 2024-03-04T07:54:32.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:32.204+0000 7fd98dc00640 1 -- 172.21.15.113:0/3859695086 wait complete. 2024-03-04T07:54:32.221 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:54:34.221 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:34.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffacbe00640 1 Processor -- start 2024-03-04T07:54:34.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffacbe00640 1 -- start start 2024-03-04T07:54:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffacbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffac4152b20 0x7ffac4070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffacbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffac40c8b00 0x7ffac415be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffacbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac415c340 0x7ffac415c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffacbe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac40c8f90 con 0x7ffac40c8b00 2024-03-04T07:54:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffacbe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac40c5320 con 0x7ffac4152b20 2024-03-04T07:54:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffacbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac4152f00 con 0x7ffac415c340 2024-03-04T07:54:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac8a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffac40c8b00 0x7ffac415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac8a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffac40c8b00 0x7ffac415be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38400/0 (socket says 172.21.15.113:38400) 2024-03-04T07:54:34.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac8a00640 1 -- 172.21.15.113:0/3403631558 learned_addr learned my addr 172.21.15.113:0/3403631558 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:34.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac9400640 1 --2- 172.21.15.113:0/3403631558 >> v2:172.21.15.136:3300/0 conn(0x7ffac4152b20 0x7ffac4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:34.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac9e00640 1 --2- 172.21.15.113:0/3403631558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac415c340 0x7ffac415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:34.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac8a00640 1 -- 172.21.15.113:0/3403631558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac415c340 msgr2=0x7ffac415c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:34.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac8a00640 1 --2- 172.21.15.113:0/3403631558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac415c340 0x7ffac415c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac8a00640 1 -- 172.21.15.113:0/3403631558 >> v2:172.21.15.136:3300/0 conn(0x7ffac4152b20 msgr2=0x7ffac4070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:34.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac8a00640 1 --2- 172.21.15.113:0/3403631558 >> v2:172.21.15.136:3300/0 conn(0x7ffac4152b20 0x7ffac4070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac8a00640 1 -- 172.21.15.113:0/3403631558 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac40c5040 con 0x7ffac40c8b00 2024-03-04T07:54:34.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.364+0000 7ffac9e00640 1 --2- 172.21.15.113:0/3403631558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac415c340 0x7ffac415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:54:34.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffac8a00640 1 --2- 172.21.15.113:0/3403631558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac40c8b00 0x7ffac415be00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ffac4072a50 tx=0x7ffab001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e8e6534b324bd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:34.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffac3e00640 1 -- 172.21.15.113:0/3403631558 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00b0070 con 0x7ffac40c8b00 2024-03-04T07:54:34.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffac3e00640 1 -- 172.21.15.113:0/3403631558 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab00ab600 con 0x7ffac40c8b00 2024-03-04T07:54:34.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffac3e00640 1 -- 172.21.15.113:0/3403631558 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00ab9d0 con 0x7ffac40c8b00 2024-03-04T07:54:34.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffacbe00640 1 -- 172.21.15.113:0/3403631558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac40c8b00 msgr2=0x7ffac415be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:34.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffacbe00640 1 --2- 172.21.15.113:0/3403631558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac40c8b00 0x7ffac415be00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ffac4072a50 tx=0x7ffab001c520 comp rx=0 tx=0).stop 2024-03-04T07:54:34.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffacbe00640 1 -- 172.21.15.113:0/3403631558 shutdown_connections 2024-03-04T07:54:34.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffacbe00640 1 --2- 172.21.15.113:0/3403631558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac415c340 0x7ffac415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffacbe00640 1 --2- 172.21.15.113:0/3403631558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac40c8b00 0x7ffac415be00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffacbe00640 1 --2- 172.21.15.113:0/3403631558 >> v2:172.21.15.136:3300/0 conn(0x7ffac4152b20 0x7ffac4070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffacbe00640 1 -- 172.21.15.113:0/3403631558 >> 172.21.15.113:0/3403631558 conn(0x7ffac40709a0 msgr2=0x7ffac4070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:34.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffacbe00640 1 -- 172.21.15.113:0/3403631558 shutdown_connections 2024-03-04T07:54:34.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.368+0000 7ffacbe00640 1 -- 172.21.15.113:0/3403631558 wait complete. 2024-03-04T07:54:34.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffacbe00640 1 Processor -- start 2024-03-04T07:54:34.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffacbe00640 1 -- start start 2024-03-04T07:54:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffacbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac40c8b00 0x7ffac415a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffacbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac4152b20 0x7ffac414b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffacbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffac415c340 0x7ffac4149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffacbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac4158810 con 0x7ffac4152b20 2024-03-04T07:54:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffacbe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40c3810 con 0x7ffac415c340 2024-03-04T07:54:34.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffacbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac40c8490 con 0x7ffac40c8b00 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac4152b20 0x7ffac414b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac9400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac40c8b00 0x7ffac415a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac4152b20 0x7ffac414b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38412/0 (socket says 172.21.15.113:38412) 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac8a00640 1 -- 172.21.15.113:0/1608372826 learned_addr learned my addr 172.21.15.113:0/1608372826 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac9e00640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffac415c340 0x7ffac4149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac8a00640 1 -- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac40c8b00 msgr2=0x7ffac415a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac8a00640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac40c8b00 0x7ffac415a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac8a00640 1 -- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffac415c340 msgr2=0x7ffac4149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac8a00640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffac415c340 0x7ffac4149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac8a00640 1 -- 172.21.15.113:0/1608372826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac414c9b0 con 0x7ffac4152b20 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac8a00640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac4152b20 0x7ffac414b950 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ffab001abd0 tx=0x7ffab00192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac1600640 1 -- 172.21.15.113:0/1608372826 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00ba070 con 0x7ffac4152b20 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac1600640 1 -- 172.21.15.113:0/1608372826 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab00b4620 con 0x7ffac4152b20 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffac1600640 1 -- 172.21.15.113:0/1608372826 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00b0040 con 0x7ffac4152b20 2024-03-04T07:54:34.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffacbe00640 1 -- 172.21.15.113:0/1608372826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac4159b70 con 0x7ffac4152b20 2024-03-04T07:54:34.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.372+0000 7ffacbe00640 1 -- 172.21.15.113:0/1608372826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac414cc50 con 0x7ffac4152b20 2024-03-04T07:54:34.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.376+0000 7ffacbe00640 1 -- 172.21.15.113:0/1608372826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac4075c00 con 0x7ffac4152b20 2024-03-04T07:54:34.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.376+0000 7ffac1600640 1 -- 172.21.15.113:0/1608372826 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffab00bd070 con 0x7ffac4152b20 2024-03-04T07:54:34.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.376+0000 7ffac1600640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffaa4066f60 0x7ffaa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:34.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.376+0000 7ffac1600640 1 -- 172.21.15.113:0/1608372826 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7ffab011e4f0 con 0x7ffac4152b20 2024-03-04T07:54:34.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.376+0000 7ffac9400640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffaa4066f60 0x7ffaa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:34.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.376+0000 7ffac9400640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffaa4066f60 0x7ffaa4069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffab8010c60 tx=0x7ffab8012040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:34.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.384+0000 7ffac1600640 1 -- 172.21.15.113:0/1608372826 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ffab0016110 con 0x7ffac4152b20 2024-03-04T07:54:34.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.644+0000 7ffacbe00640 1 -- 172.21.15.113:0/1608372826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffac4162f70 con 0x7ffac4152b20 2024-03-04T07:54:34.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.644+0000 7ffac1600640 1 -- 172.21.15.113:0/1608372826 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+6609 (secure 0 0 0) 0x7ffab00eea40 con 0x7ffac4152b20 2024-03-04T07:54:34.639 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:34.639 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6240,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:30.928216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940,5631]},"id":1}]} 2024-03-04T07:54:34.640 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 76 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 -- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffaa4066f60 msgr2=0x7ffaa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffaa4066f60 0x7ffaa4069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffab8010c60 tx=0x7ffab8012040 comp rx=0 tx=0).stop 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 -- 172.21.15.113:0/1608372826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac4152b20 msgr2=0x7ffac414b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac4152b20 0x7ffac414b950 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ffab001abd0 tx=0x7ffab00192c0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 -- 172.21.15.113:0/1608372826 shutdown_connections 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffac415c340 0x7ffac4149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffaa4066f60 0x7ffaa4069420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffac4152b20 0x7ffac414b950 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 --2- 172.21.15.113:0/1608372826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffac40c8b00 0x7ffac415a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 -- 172.21.15.113:0/1608372826 >> 172.21.15.113:0/1608372826 conn(0x7ffac40709a0 msgr2=0x7ffac40cbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:34.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 -- 172.21.15.113:0/1608372826 shutdown_connections 2024-03-04T07:54:34.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:34.648+0000 7ffacbe00640 1 -- 172.21.15.113:0/1608372826 wait complete. 2024-03-04T07:54:34.664 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:54:36.665 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:36.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.803+0000 7f2379000640 1 Processor -- start 2024-03-04T07:54:36.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.803+0000 7f2379000640 1 -- start start 2024-03-04T07:54:36.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f2379000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f237406de10 0x7f2374070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:36.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f2379000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237415b150 0x7f237415b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:36.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f2379000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740c8da0 con 0x7f2374158390 2024-03-04T07:54:36.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f2379000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2374158770 con 0x7f237406de10 2024-03-04T07:54:36.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f2379000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237406f770 con 0x7f237415b150 2024-03-04T07:54:36.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f236be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f237406de10 0x7f2374070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:36.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f2371e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2374158390 0x7f23740ce270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39378/0 (socket says 172.21.15.113:39378) 2024-03-04T07:54:36.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f2371e00640 1 -- 172.21.15.113:0/3621360073 learned_addr learned my addr 172.21.15.113:0/3621360073 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:54:36.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f236be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f237406de10 0x7f2374070130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38416/0 (socket says 172.21.15.113:38416) 2024-03-04T07:54:36.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f2372800640 1 --2- 172.21.15.113:0/3621360073 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237415b150 0x7f237415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:36.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f236be00640 1 -- 172.21.15.113:0/3621360073 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237415b150 msgr2=0x7f237415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:36.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f236be00640 1 --2- 172.21.15.113:0/3621360073 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237415b150 0x7f237415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:36.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f236be00640 1 -- 172.21.15.113:0/3621360073 >> v1:172.21.15.113:6789/0 conn(0x7f2374158390 legacy=0x7f23740ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:36.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f2371400640 1 -- 172.21.15.113:0/3621360073 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495931353 0 0) 0x7f2364001120 con 0x7f2374158390 2024-03-04T07:54:36.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.807+0000 7f236be00640 1 -- 172.21.15.113:0/3621360073 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237406fa50 con 0x7f237406de10 2024-03-04T07:54:36.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f236be00640 1 --2- 172.21.15.113:0/3621360073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 0x7f2374070130 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f235c0097c0 tx=0x7f235c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1789b94049d4b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:36.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2371400640 1 -- 172.21.15.113:0/3621360073 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c0b0070 con 0x7f237406de10 2024-03-04T07:54:36.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2371400640 1 -- 172.21.15.113:0/3621360073 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c0ab600 con 0x7f237406de10 2024-03-04T07:54:36.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2371400640 1 -- 172.21.15.113:0/3621360073 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c0ab9d0 con 0x7f237406de10 2024-03-04T07:54:36.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 -- 172.21.15.113:0/3621360073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 msgr2=0x7f2374070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:36.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 --2- 172.21.15.113:0/3621360073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 0x7f2374070130 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f235c0097c0 tx=0x7f235c01c410 comp rx=0 tx=0).stop 2024-03-04T07:54:36.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 -- 172.21.15.113:0/3621360073 shutdown_connections 2024-03-04T07:54:36.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 --2- 172.21.15.113:0/3621360073 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237415b150 0x7f237415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:36.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 --2- 172.21.15.113:0/3621360073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 0x7f2374070130 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:36.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 -- 172.21.15.113:0/3621360073 >> 172.21.15.113:0/3621360073 conn(0x7f2374066db0 msgr2=0x7f23740671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:36.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 -- 172.21.15.113:0/3621360073 shutdown_connections 2024-03-04T07:54:36.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 -- 172.21.15.113:0/3621360073 wait complete. 2024-03-04T07:54:36.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 Processor -- start 2024-03-04T07:54:36.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 -- start start 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 0x7f23740aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2374158390 0x7f23740afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237415b150 0x7f23740b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740b1070 con 0x7f237406de10 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740b0ef0 con 0x7f2374158390 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2379000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740ce980 con 0x7f237415b150 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2371e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 0x7f23740aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2371e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 0x7f23740aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38432/0 (socket says 172.21.15.113:38432) 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.811+0000 7f2371e00640 1 -- 172.21.15.113:0/3599462161 learned_addr learned my addr 172.21.15.113:0/3599462161 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f2372800640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237415b150 0x7f23740b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f236be00640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2374158390 0x7f23740afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:36.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f2371e00640 1 -- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237415b150 msgr2=0x7f23740b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:36.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f2371e00640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237415b150 0x7f23740b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:36.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f2371e00640 1 -- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2374158390 msgr2=0x7f23740afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:36.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f2371e00640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2374158390 0x7f23740afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:36.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f2371e00640 1 -- 172.21.15.113:0/3599462161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740cc770 con 0x7f237406de10 2024-03-04T07:54:36.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f2371e00640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 0x7f23740aee80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f236400c780 tx=0x7f236400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:36.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f236a000640 1 -- 172.21.15.113:0/3599462161 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364017070 con 0x7f237406de10 2024-03-04T07:54:36.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f236a000640 1 -- 172.21.15.113:0/3599462161 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364005ac0 con 0x7f237406de10 2024-03-04T07:54:36.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f236a000640 1 -- 172.21.15.113:0/3599462161 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364007070 con 0x7f237406de10 2024-03-04T07:54:36.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f2379000640 1 -- 172.21.15.113:0/3599462161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23740cd720 con 0x7f237406de10 2024-03-04T07:54:36.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f2379000640 1 -- 172.21.15.113:0/3599462161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23740cdce0 con 0x7f237406de10 2024-03-04T07:54:36.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f2379000640 1 -- 172.21.15.113:0/3599462161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2374075c00 con 0x7f237406de10 2024-03-04T07:54:36.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.815+0000 7f236a000640 1 -- 172.21.15.113:0/3599462161 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f236401a070 con 0x7f237406de10 2024-03-04T07:54:36.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.819+0000 7f236a000640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 0x7f2350069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:36.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.819+0000 7f236a000640 1 -- 172.21.15.113:0/3599462161 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f23640848a0 con 0x7f237406de10 2024-03-04T07:54:36.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.819+0000 7f236be00640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 0x7f2350069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:36.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.819+0000 7f236be00640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 0x7f2350069450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f235c0ad000 tx=0x7f235c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:36.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:36.823+0000 7f236a000640 1 -- 172.21.15.113:0/3599462161 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f23740c8da0 con 0x7f237406de10 2024-03-04T07:54:37.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.011+0000 7f2379000640 1 -- 172.21.15.113:0/3599462161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f237406ff10 con 0x7f237406de10 2024-03-04T07:54:37.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.011+0000 7f236a000640 1 -- 172.21.15.113:0/3599462161 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+6609 (secure 0 0 0) 0x7f2364054df0 con 0x7f237406de10 2024-03-04T07:54:37.008 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:37.008 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6240,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:30.928216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940,5631]},"id":1}]} 2024-03-04T07:54:37.008 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 76 2024-03-04T07:54:37.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 -- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 msgr2=0x7f2350069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:37.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 0x7f2350069450 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f235c0ad000 tx=0x7f235c008040 comp rx=0 tx=0).stop 2024-03-04T07:54:37.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 -- 172.21.15.113:0/3599462161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 msgr2=0x7f23740aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:37.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 0x7f23740aee80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f236400c780 tx=0x7f236400cc40 comp rx=0 tx=0).stop 2024-03-04T07:54:37.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 -- 172.21.15.113:0/3599462161 shutdown_connections 2024-03-04T07:54:37.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237415b150 0x7f23740b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:37.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2374158390 0x7f23740afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:37.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 0x7f2350069450 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:37.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 --2- 172.21.15.113:0/3599462161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f237406de10 0x7f23740aee80 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:37.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 -- 172.21.15.113:0/3599462161 >> 172.21.15.113:0/3599462161 conn(0x7f2374066db0 msgr2=0x7f237415a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:37.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 -- 172.21.15.113:0/3599462161 shutdown_connections 2024-03-04T07:54:37.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:37.015+0000 7f2379000640 1 -- 172.21.15.113:0/3599462161 wait complete. 2024-03-04T07:54:37.029 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:54:39.030 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.167+0000 7f56fd600640 1 Processor -- start 2024-03-04T07:54:39.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.167+0000 7f56fd600640 1 -- start start 2024-03-04T07:54:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56fd600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f56f8154fe0 0x7f56f80c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56fd600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56f80c7700 0x7f56f815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56fd600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f80c7d80 con 0x7f56f8154c00 2024-03-04T07:54:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56fd600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f815efb0 con 0x7f56f80c7700 2024-03-04T07:54:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56fd600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f80c6b70 con 0x7f56f8154fe0 2024-03-04T07:54:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56f7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56f80c7700 0x7f56f815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56f7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f56f80c7700 0x7f56f815abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48368/0 (socket says 172.21.15.113:48368) 2024-03-04T07:54:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56efe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f56f8154fe0 0x7f56f80c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:39.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56f7400640 1 -- 172.21.15.113:0/4203684764 learned_addr learned my addr 172.21.15.113:0/4203684764 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56f7400640 1 -- 172.21.15.113:0/4203684764 >> v2:172.21.15.136:3300/0 conn(0x7f56f8154fe0 msgr2=0x7f56f80c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56f7400640 1 --2- 172.21.15.113:0/4203684764 >> v2:172.21.15.136:3300/0 conn(0x7f56f8154fe0 0x7f56f80c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56f7400640 1 -- 172.21.15.113:0/4203684764 >> v1:172.21.15.136:6789/0 conn(0x7f56f8154c00 legacy=0x7f56f80c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:39.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.171+0000 7f56f7400640 1 -- 172.21.15.113:0/4203684764 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f80c8060 con 0x7f56f80c7700 2024-03-04T07:54:39.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56f7400640 1 --2- 172.21.15.113:0/4203684764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 0x7f56f815abe0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f56f815bc00 tx=0x7f56f001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5c4183fc31dcf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:39.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56f6000640 1 -- 172.21.15.113:0/4203684764 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f00a5710 con 0x7f56f80c7700 2024-03-04T07:54:39.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56f6000640 1 -- 172.21.15.113:0/4203684764 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f00a58b0 con 0x7f56f80c7700 2024-03-04T07:54:39.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56f6000640 1 -- 172.21.15.113:0/4203684764 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f00a5c80 con 0x7f56f80c7700 2024-03-04T07:54:39.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56fd600640 1 -- 172.21.15.113:0/4203684764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 msgr2=0x7f56f815abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:39.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56fd600640 1 --2- 172.21.15.113:0/4203684764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 0x7f56f815abe0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f56f815bc00 tx=0x7f56f001c960 comp rx=0 tx=0).stop 2024-03-04T07:54:39.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56fd600640 1 -- 172.21.15.113:0/4203684764 shutdown_connections 2024-03-04T07:54:39.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56fd600640 1 --2- 172.21.15.113:0/4203684764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 0x7f56f815abe0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:39.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56fd600640 1 --2- 172.21.15.113:0/4203684764 >> v2:172.21.15.136:3300/0 conn(0x7f56f8154fe0 0x7f56f80c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:39.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56fd600640 1 -- 172.21.15.113:0/4203684764 >> 172.21.15.113:0/4203684764 conn(0x7f56f806e820 msgr2=0x7f56f806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:39.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56fd600640 1 -- 172.21.15.113:0/4203684764 shutdown_connections 2024-03-04T07:54:39.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.175+0000 7f56fd600640 1 -- 172.21.15.113:0/4203684764 wait complete. 2024-03-04T07:54:39.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56fd600640 1 Processor -- start 2024-03-04T07:54:39.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56fd600640 1 -- start start 2024-03-04T07:54:39.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56fd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 0x7f56f8156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:39.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56fd600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56f8154c00 0x7f56f814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:39.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56fd600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f8154fe0 0x7f56f814cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:39.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56fd600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80c23e0 con 0x7f56f80c7700 2024-03-04T07:54:39.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56fd600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f80c2260 con 0x7f56f8154c00 2024-03-04T07:54:39.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56fd600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f8167c10 con 0x7f56f8154fe0 2024-03-04T07:54:39.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56f6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 0x7f56f8156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56f6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 0x7f56f8156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48372/0 (socket says 172.21.15.113:48372) 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56f6a00640 1 -- 172.21.15.113:0/2666651245 learned_addr learned my addr 172.21.15.113:0/2666651245 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56efe00640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56f8154c00 0x7f56f814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56f7400640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f8154fe0 0x7f56f814cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56f6a00640 1 -- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f8154fe0 msgr2=0x7f56f814cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56f6a00640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f8154fe0 0x7f56f814cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56f6a00640 1 -- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56f8154c00 msgr2=0x7f56f814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56f6a00640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56f8154c00 0x7f56f814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56f6a00640 1 -- 172.21.15.113:0/2666651245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8156940 con 0x7f56f80c7700 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56efe00640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56f8154c00 0x7f56f814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56f6a00640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 0x7f56f8156eb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c3a0 tx=0x7f56e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56eea00640 1 -- 172.21.15.113:0/2666651245 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e80052d0 con 0x7f56f80c7700 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56eea00640 1 -- 172.21.15.113:0/2666651245 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8005470 con 0x7f56f80c7700 2024-03-04T07:54:39.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56eea00640 1 -- 172.21.15.113:0/2666651245 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e8007260 con 0x7f56f80c7700 2024-03-04T07:54:39.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56fd600640 1 -- 172.21.15.113:0/2666651245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f80c3ad0 con 0x7f56f80c7700 2024-03-04T07:54:39.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.179+0000 7f56fd600640 1 -- 172.21.15.113:0/2666651245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f80c3510 con 0x7f56f80c7700 2024-03-04T07:54:39.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.183+0000 7f56fd600640 1 -- 172.21.15.113:0/2666651245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f8075c00 con 0x7f56f80c7700 2024-03-04T07:54:39.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.183+0000 7f56eea00640 1 -- 172.21.15.113:0/2666651245 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56e8007400 con 0x7f56f80c7700 2024-03-04T07:54:39.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.183+0000 7f56eea00640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56d4066f60 0x7f56d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:39.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.183+0000 7f56eea00640 1 -- 172.21.15.113:0/2666651245 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6685+0+0 (secure 0 0 0) 0x7f56e8084840 con 0x7f56f80c7700 2024-03-04T07:54:39.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.183+0000 7f56efe00640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56d4066f60 0x7f56d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:39.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.187+0000 7f56efe00640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56d4066f60 0x7f56d4069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f56f80cc070 tx=0x7f56e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:39.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.187+0000 7f56eea00640 1 -- 172.21.15.113:0/2666651245 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f56e8055480 con 0x7f56f80c7700 2024-03-04T07:54:39.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.387+0000 7f56fd600640 1 -- 172.21.15.113:0/2666651245 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56f81587c0 con 0x7f56f80c7700 2024-03-04T07:54:39.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.387+0000 7f56eea00640 1 -- 172.21.15.113:0/2666651245 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+6609 (secure 0 0 0) 0x7f56e8054d80 con 0x7f56f80c7700 2024-03-04T07:54:39.384 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:39.384 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6240,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:52:30.928216+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"c","rank":2,"incarnation":51,"state":"up:active","state_seq":17,"addr":"172.21.15.113:6833/810789479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":810789479},{"type":"v1","addr":"172.21.15.113:6833","nonce":810789479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940,5631]},"id":1}]} 2024-03-04T07:54:39.384 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 76 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.387+0000 7f56fd600640 1 -- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56d4066f60 msgr2=0x7f56d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.387+0000 7f56fd600640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56d4066f60 0x7f56d4069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f56f80cc070 tx=0x7f56e0008040 comp rx=0 tx=0).stop 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.387+0000 7f56fd600640 1 -- 172.21.15.113:0/2666651245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 msgr2=0x7f56f8156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.387+0000 7f56fd600640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 0x7f56f8156eb0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c3a0 tx=0x7f56e800c860 comp rx=0 tx=0).stop 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.387+0000 7f56fd600640 1 -- 172.21.15.113:0/2666651245 shutdown_connections 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.387+0000 7f56fd600640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f56f8154fe0 0x7f56f814cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.387+0000 7f56fd600640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56f8154c00 0x7f56f814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.391+0000 7f56fd600640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56d4066f60 0x7f56d4069420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.391+0000 7f56fd600640 1 --2- 172.21.15.113:0/2666651245 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56f80c7700 0x7f56f8156eb0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.391+0000 7f56fd600640 1 -- 172.21.15.113:0/2666651245 >> 172.21.15.113:0/2666651245 conn(0x7f56f806e820 msgr2=0x7f56f80c9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.391+0000 7f56fd600640 1 -- 172.21.15.113:0/2666651245 shutdown_connections 2024-03-04T07:54:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:39.391+0000 7f56fd600640 1 -- 172.21.15.113:0/2666651245 wait complete. 2024-03-04T07:54:39.405 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T07:54:41.119 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 18 -eq 48 ] 2024-03-04T07:54:41.120 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.18 2024-03-04T07:54:41.120 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.18 2024-03-04T07:54:41.120 INFO:tasks.workunit.client.0.smithi113.stdout:v0.18 2024-03-04T07:54:41.120 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.18 2024-03-04T07:54:41.406 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee42c00640 1 Processor -- start 2024-03-04T07:54:41.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee42c00640 1 -- start start 2024-03-04T07:54:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee42c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee3c152d60 0x7fee3c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee42c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fee3c070120 0x7fee3c0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee42c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee3c0c9db0 0x7fee3c158930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee42c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3c0c5380 con 0x7fee3c152d60 2024-03-04T07:54:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee42c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3c0c3440 con 0x7fee3c070120 2024-03-04T07:54:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee42c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c06f620 con 0x7fee3c0c9db0 2024-03-04T07:54:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee3be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee3c152d60 0x7fee3c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee3be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee3c152d60 0x7fee3c0ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48380/0 (socket says 172.21.15.113:48380) 2024-03-04T07:54:41.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee3be00640 1 -- 172.21.15.113:0/4079011809 learned_addr learned my addr 172.21.15.113:0/4079011809 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee40c00640 1 --2- 172.21.15.113:0/4079011809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee3c0c9db0 0x7fee3c158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee33400640 1 --2- 172.21.15.113:0/4079011809 >> v2:172.21.15.136:3300/0 conn(0x7fee3c070120 0x7fee3c0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee3be00640 1 -- 172.21.15.113:0/4079011809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee3c0c9db0 msgr2=0x7fee3c158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee3be00640 1 --2- 172.21.15.113:0/4079011809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee3c0c9db0 0x7fee3c158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee3be00640 1 -- 172.21.15.113:0/4079011809 >> v2:172.21.15.136:3300/0 conn(0x7fee3c070120 msgr2=0x7fee3c0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee3be00640 1 --2- 172.21.15.113:0/4079011809 >> v2:172.21.15.136:3300/0 conn(0x7fee3c070120 0x7fee3c0ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.571+0000 7fee3be00640 1 -- 172.21.15.113:0/4079011809 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c0c36e0 con 0x7fee3c152d60 2024-03-04T07:54:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee3be00640 1 --2- 172.21.15.113:0/4079011809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 0x7fee3c0ce310 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0cbef0 tx=0x7fee2c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf54a3b09e3db59d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:41.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee3b400640 1 -- 172.21.15.113:0/4079011809 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0b5070 con 0x7fee3c152d60 2024-03-04T07:54:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee3b400640 1 -- 172.21.15.113:0/4079011809 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c0a89d0 con 0x7fee3c152d60 2024-03-04T07:54:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee3b400640 1 -- 172.21.15.113:0/4079011809 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0a8da0 con 0x7fee3c152d60 2024-03-04T07:54:41.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee42c00640 1 -- 172.21.15.113:0/4079011809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 msgr2=0x7fee3c0ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee42c00640 1 --2- 172.21.15.113:0/4079011809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 0x7fee3c0ce310 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0cbef0 tx=0x7fee2c01ed10 comp rx=0 tx=0).stop 2024-03-04T07:54:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee42c00640 1 -- 172.21.15.113:0/4079011809 shutdown_connections 2024-03-04T07:54:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee42c00640 1 --2- 172.21.15.113:0/4079011809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee3c0c9db0 0x7fee3c158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee42c00640 1 --2- 172.21.15.113:0/4079011809 >> v2:172.21.15.136:3300/0 conn(0x7fee3c070120 0x7fee3c0ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee42c00640 1 --2- 172.21.15.113:0/4079011809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 0x7fee3c0ce310 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.575+0000 7fee42c00640 1 -- 172.21.15.113:0/4079011809 >> 172.21.15.113:0/4079011809 conn(0x7fee3c066db0 msgr2=0x7fee3c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee42c00640 1 -- 172.21.15.113:0/4079011809 shutdown_connections 2024-03-04T07:54:41.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee42c00640 1 -- 172.21.15.113:0/4079011809 wait complete. 2024-03-04T07:54:41.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee42c00640 1 Processor -- start 2024-03-04T07:54:41.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee42c00640 1 -- start start 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee42c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee3c070120 0x7fee3c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee42c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee3c0c9db0 0x7fee3c0cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee42c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 0x7fee3c0aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee42c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0c39c0 con 0x7fee3c152d60 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee42c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0c5020 con 0x7fee3c0c9db0 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee42c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0c3b40 con 0x7fee3c070120 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee40c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 0x7fee3c0aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee40c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 0x7fee3c0aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48384/0 (socket says 172.21.15.113:48384) 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee40c00640 1 -- 172.21.15.113:0/235206302 learned_addr learned my addr 172.21.15.113:0/235206302 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee3be00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee3c070120 0x7fee3c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee33400640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee3c0c9db0 0x7fee3c0cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee40c00640 1 -- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee3c070120 msgr2=0x7fee3c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee40c00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee3c070120 0x7fee3c0ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee40c00640 1 -- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee3c0c9db0 msgr2=0x7fee3c0cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee40c00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee3c0c9db0 0x7fee3c0cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee40c00640 1 -- 172.21.15.113:0/235206302 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c1f0ce0 con 0x7fee3c152d60 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.579+0000 7fee33400640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee3c0c9db0 0x7fee3c0cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.583+0000 7fee40c00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 0x7fee3c0aeb40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fee34007d90 tx=0x7fee3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.583+0000 7fee38c00640 1 -- 172.21.15.113:0/235206302 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34014420 con 0x7fee3c152d60 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.583+0000 7fee38c00640 1 -- 172.21.15.113:0/235206302 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee34019070 con 0x7fee3c152d60 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.583+0000 7fee42c00640 1 -- 172.21.15.113:0/235206302 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c1f0f80 con 0x7fee3c152d60 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.583+0000 7fee38c00640 1 -- 172.21.15.113:0/235206302 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34014820 con 0x7fee3c152d60 2024-03-04T07:54:41.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.583+0000 7fee42c00640 1 -- 172.21.15.113:0/235206302 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c15cb00 con 0x7fee3c152d60 2024-03-04T07:54:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.583+0000 7fee42c00640 1 -- 172.21.15.113:0/235206302 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3c075c00 con 0x7fee3c152d60 2024-03-04T07:54:41.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.583+0000 7fee38c00640 1 -- 172.21.15.113:0/235206302 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee340149c0 con 0x7fee3c152d60 2024-03-04T07:54:41.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.583+0000 7fee38c00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee18066f60 0x7fee18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.587+0000 7fee38c00640 1 -- 172.21.15.113:0/235206302 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7fee34085b90 con 0x7fee3c152d60 2024-03-04T07:54:41.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.587+0000 7fee3be00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee18066f60 0x7fee18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.587+0000 7fee3be00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee18066f60 0x7fee18069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0b2000 tx=0x7fee2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:41.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.587+0000 7fee38c00640 1 -- 172.21.15.113:0/235206302 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fee34056800 con 0x7fee3c152d60 2024-03-04T07:54:41.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 -- 172.21.15.113:0/235206302 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee3c06ff00 con 0x7fee3c152d60 2024-03-04T07:54:41.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee38c00640 1 -- 172.21.15.113:0/235206302 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5812 (secure 0 0 0) 0x7fee34056100 con 0x7fee3c152d60 2024-03-04T07:54:41.798 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:41.799 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:54:40.746610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,6145]},"id":1}]} 2024-03-04T07:54:41.799 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 77 2024-03-04T07:54:41.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 -- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee18066f60 msgr2=0x7fee18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee18066f60 0x7fee18069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0b2000 tx=0x7fee2c009040 comp rx=0 tx=0).stop 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 -- 172.21.15.113:0/235206302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 msgr2=0x7fee3c0aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 0x7fee3c0aeb40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fee34007d90 tx=0x7fee3400e4b0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 -- 172.21.15.113:0/235206302 shutdown_connections 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee18066f60 0x7fee18069420 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee3c152d60 0x7fee3c0aeb40 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee3c0c9db0 0x7fee3c0cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 --2- 172.21.15.113:0/235206302 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee3c070120 0x7fee3c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 -- 172.21.15.113:0/235206302 >> 172.21.15.113:0/235206302 conn(0x7fee3c066db0 msgr2=0x7fee3c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 -- 172.21.15.113:0/235206302 shutdown_connections 2024-03-04T07:54:41.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.803+0000 7fee42c00640 1 -- 172.21.15.113:0/235206302 wait complete. 2024-03-04T07:54:41.819 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-03-04T07:54:41.819 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:54:41.820 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:41.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.979+0000 7f49e7200640 1 Processor -- start 2024-03-04T07:54:41.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.979+0000 7f49e7200640 1 -- start start 2024-03-04T07:54:41.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.979+0000 7f49e7200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f49e01527f0 0x7f49e0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.979+0000 7f49e7200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49e00c8200 0x7f49e0072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.979+0000 7f49e7200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e01591f0 con 0x7f49e0152bd0 2024-03-04T07:54:41.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.979+0000 7f49e7200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e0071df0 con 0x7f49e00c8200 2024-03-04T07:54:41.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.979+0000 7f49e7200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e01597b0 con 0x7f49e01527f0 2024-03-04T07:54:41.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49d7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49e00c8200 0x7f49e0072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49dfe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f49e01527f0 0x7f49e0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49d7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49e00c8200 0x7f49e0072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48400/0 (socket says 172.21.15.113:48400) 2024-03-04T07:54:41.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49d7400640 1 -- 172.21.15.113:0/522779509 learned_addr learned my addr 172.21.15.113:0/522779509 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:41.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49e5200640 1 --1- 172.21.15.113:0/522779509 >> v1:172.21.15.113:6789/0 conn(0x7f49e0152bd0 0x7f49e01587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43920/0 (socket says 172.21.15.113:43920) 2024-03-04T07:54:41.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49d7400640 1 -- 172.21.15.113:0/522779509 >> v2:172.21.15.136:3300/0 conn(0x7f49e01527f0 msgr2=0x7f49e0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:41.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49d7400640 1 --2- 172.21.15.113:0/522779509 >> v2:172.21.15.136:3300/0 conn(0x7f49e01527f0 0x7f49e0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49d7400640 1 -- 172.21.15.113:0/522779509 >> v1:172.21.15.113:6789/0 conn(0x7f49e0152bd0 legacy=0x7f49e01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:41.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49df400640 1 -- 172.21.15.113:0/522779509 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530120536 0 0) 0x7f49e01591f0 con 0x7f49e0152bd0 2024-03-04T07:54:41.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49d7400640 1 -- 172.21.15.113:0/522779509 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e01594d0 con 0x7f49e00c8200 2024-03-04T07:54:41.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49d7400640 1 --2- 172.21.15.113:0/522779509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00c8200 0x7f49e0072280 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f49e006f400 tx=0x7f49c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e974d452446f590 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:41.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49df400640 1 -- 172.21.15.113:0/522779509 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c80ab610 con 0x7f49e00c8200 2024-03-04T07:54:41.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49df400640 1 -- 172.21.15.113:0/522779509 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c80b0070 con 0x7f49e00c8200 2024-03-04T07:54:41.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.983+0000 7f49df400640 1 -- 172.21.15.113:0/522779509 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c80ab9d0 con 0x7f49e00c8200 2024-03-04T07:54:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.987+0000 7f49e7200640 1 -- 172.21.15.113:0/522779509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00c8200 msgr2=0x7f49e0072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.987+0000 7f49e7200640 1 --2- 172.21.15.113:0/522779509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00c8200 0x7f49e0072280 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f49e006f400 tx=0x7f49c801c520 comp rx=0 tx=0).stop 2024-03-04T07:54:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.987+0000 7f49e7200640 1 -- 172.21.15.113:0/522779509 shutdown_connections 2024-03-04T07:54:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.987+0000 7f49e7200640 1 --2- 172.21.15.113:0/522779509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e00c8200 0x7f49e0072280 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.987+0000 7f49e7200640 1 --2- 172.21.15.113:0/522779509 >> v2:172.21.15.136:3300/0 conn(0x7f49e01527f0 0x7f49e0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.987+0000 7f49e7200640 1 -- 172.21.15.113:0/522779509 >> 172.21.15.113:0/522779509 conn(0x7f49e006e820 msgr2=0x7f49e006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.987+0000 7f49e7200640 1 -- 172.21.15.113:0/522779509 shutdown_connections 2024-03-04T07:54:41.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.987+0000 7f49e7200640 1 -- 172.21.15.113:0/522779509 wait complete. 2024-03-04T07:54:41.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.987+0000 7f49e7200640 1 Processor -- start 2024-03-04T07:54:41.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.987+0000 7f49e7200640 1 -- start start 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e7200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e00c8200 0x7f49e014a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e7200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49e01527f0 0x7f49e01f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e0152bd0 0x7f49e01f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e0165160 con 0x7f49e0152bd0 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e7200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00727c0 con 0x7f49e00c8200 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e7200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e01652e0 con 0x7f49e01527f0 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e0152bd0 0x7f49e01f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49dfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e00c8200 0x7f49e014a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e0152bd0 0x7f49e01f4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48412/0 (socket says 172.21.15.113:48412) 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e5200640 1 -- 172.21.15.113:0/1546372448 learned_addr learned my addr 172.21.15.113:0/1546372448 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49d7400640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49e01527f0 0x7f49e01f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e5200640 1 -- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49e01527f0 msgr2=0x7f49e01f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:41.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e5200640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49e01527f0 0x7f49e01f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e5200640 1 -- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e00c8200 msgr2=0x7f49e014a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:41.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e5200640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e00c8200 0x7f49e014a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:41.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e5200640 1 -- 172.21.15.113:0/1546372448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0155410 con 0x7f49e0152bd0 2024-03-04T07:54:41.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e5200640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e0152bd0 0x7f49e01f4fa0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c3e0 tx=0x7f49d800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:41.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49dcc00640 1 -- 172.21.15.113:0/1546372448 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80074e0 con 0x7f49e0152bd0 2024-03-04T07:54:41.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49dcc00640 1 -- 172.21.15.113:0/1546372448 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d80047f0 con 0x7f49e0152bd0 2024-03-04T07:54:41.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e7200640 1 -- 172.21.15.113:0/1546372448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e01f5700 con 0x7f49e0152bd0 2024-03-04T07:54:41.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49dcc00640 1 -- 172.21.15.113:0/1546372448 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8004be0 con 0x7f49e0152bd0 2024-03-04T07:54:41.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e7200640 1 -- 172.21.15.113:0/1546372448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e0159990 con 0x7f49e0152bd0 2024-03-04T07:54:41.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.991+0000 7f49e7200640 1 -- 172.21.15.113:0/1546372448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e0075c00 con 0x7f49e0152bd0 2024-03-04T07:54:41.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.995+0000 7f49dcc00640 1 -- 172.21.15.113:0/1546372448 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49d8011070 con 0x7f49e0152bd0 2024-03-04T07:54:41.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.995+0000 7f49dcc00640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49bc066f60 0x7f49bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:41.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.995+0000 7f49dcc00640 1 -- 172.21.15.113:0/1546372448 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f49d8084f10 con 0x7f49e0152bd0 2024-03-04T07:54:41.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.995+0000 7f49dfe00640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49bc066f60 0x7f49bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:41.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.995+0000 7f49dfe00640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49bc066f60 0x7f49bc069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f49d00113e0 tx=0x7f49d000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:41.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:41.999+0000 7f49dcc00640 1 -- 172.21.15.113:0/1546372448 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f49d8055400 con 0x7f49e0152bd0 2024-03-04T07:54:42.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.207+0000 7f49e7200640 1 -- 172.21.15.113:0/1546372448 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49e00cb1d0 con 0x7f49e0152bd0 2024-03-04T07:54:42.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.207+0000 7f49dcc00640 1 -- 172.21.15.113:0/1546372448 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f49d80594a0 con 0x7f49e0152bd0 2024-03-04T07:54:42.204 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:42.204 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:54:40.746610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,6145]},"id":1}]} 2024-03-04T07:54:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 77 2024-03-04T07:54:42.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 -- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49bc066f60 msgr2=0x7f49bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:42.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49bc066f60 0x7f49bc069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f49d00113e0 tx=0x7f49d000b040 comp rx=0 tx=0).stop 2024-03-04T07:54:42.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 -- 172.21.15.113:0/1546372448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e0152bd0 msgr2=0x7f49e01f4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:42.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e0152bd0 0x7f49e01f4fa0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c3e0 tx=0x7f49d800c8a0 comp rx=0 tx=0).stop 2024-03-04T07:54:42.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 -- 172.21.15.113:0/1546372448 shutdown_connections 2024-03-04T07:54:42.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49bc066f60 0x7f49bc069420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:42.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49e0152bd0 0x7f49e01f4fa0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:42.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49e01527f0 0x7f49e01f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:42.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 --2- 172.21.15.113:0/1546372448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49e00c8200 0x7f49e014a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:42.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 -- 172.21.15.113:0/1546372448 >> 172.21.15.113:0/1546372448 conn(0x7f49e006e820 msgr2=0x7f49e0151f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:42.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 -- 172.21.15.113:0/1546372448 shutdown_connections 2024-03-04T07:54:42.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:42.211+0000 7f49e7200640 1 -- 172.21.15.113:0/1546372448 wait complete. 2024-03-04T07:54:42.225 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 77, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 77, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5940": { "addr": "172.21.15.113:6837/344022786", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 344022786, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 344022786, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5940, "incarnation": 64, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 30 }, "gid_6145": { "addr": "172.21.15.136:6835/2368734308", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2368734308, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2368734308, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6145, "incarnation": 71, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_6240": { "addr": "172.21.15.113:6836/986515123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 986515123, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 986515123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6240, "incarnation": 77, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:54:40.746610+0000", "qdb_cluster": [ 5940, 6145 ], "qdb_leader": 5940, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5940, "mds_1": 6145, "mds_2": 6240 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:54:44.227 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:44.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bfd000640 1 Processor -- start 2024-03-04T07:54:44.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bfd000640 1 -- start start 2024-03-04T07:54:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bfd000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 0x7f2bf80ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bfd000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2bf806ff20 0x7f2bf80ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bfd000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2bf80c8b90 0x7f2bf815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bfd000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf80c31b0 con 0x7f2bf80c8b90 2024-03-04T07:54:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bfd000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf806f840 con 0x7f2bf806ff20 2024-03-04T07:54:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bfd000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf8152e30 con 0x7f2bf8152a50 2024-03-04T07:54:44.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf6800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2bf80c8b90 0x7f2bf815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf6800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2bf80c8b90 0x7f2bf815bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48424/0 (socket says 172.21.15.113:48424) 2024-03-04T07:54:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf6800640 1 -- 172.21.15.113:0/786796573 learned_addr learned my addr 172.21.15.113:0/786796573 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf5400640 1 --2- 172.21.15.113:0/786796573 >> v2:172.21.15.136:3300/0 conn(0x7f2bf806ff20 0x7f2bf80ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf5e00640 1 --2- 172.21.15.113:0/786796573 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 0x7f2bf80ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf6800640 1 -- 172.21.15.113:0/786796573 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 msgr2=0x7f2bf80ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf6800640 1 --2- 172.21.15.113:0/786796573 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 0x7f2bf80ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf6800640 1 -- 172.21.15.113:0/786796573 >> v2:172.21.15.136:3300/0 conn(0x7f2bf806ff20 msgr2=0x7f2bf80ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf6800640 1 --2- 172.21.15.113:0/786796573 >> v2:172.21.15.136:3300/0 conn(0x7f2bf806ff20 0x7f2bf80ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf6800640 1 -- 172.21.15.113:0/786796573 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf806f280 con 0x7f2bf80c8b90 2024-03-04T07:54:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.367+0000 7f2bf5400640 1 --2- 172.21.15.113:0/786796573 >> v2:172.21.15.136:3300/0 conn(0x7f2bf806ff20 0x7f2bf80ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:54:44.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bf6800640 1 --2- 172.21.15.113:0/786796573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf80c8b90 0x7f2bf815bdd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8067be0 tx=0x7f2bf001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9538a95c210d6d33 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:44.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bf4a00640 1 -- 172.21.15.113:0/786796573 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00a5870 con 0x7f2bf80c8b90 2024-03-04T07:54:44.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bf4a00640 1 -- 172.21.15.113:0/786796573 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf00a5a10 con 0x7f2bf80c8b90 2024-03-04T07:54:44.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bf4a00640 1 -- 172.21.15.113:0/786796573 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00a5de0 con 0x7f2bf80c8b90 2024-03-04T07:54:44.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bfd000640 1 -- 172.21.15.113:0/786796573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf80c8b90 msgr2=0x7f2bf815bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:44.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bfd000640 1 --2- 172.21.15.113:0/786796573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf80c8b90 0x7f2bf815bdd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8067be0 tx=0x7f2bf001c9f0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bfd000640 1 -- 172.21.15.113:0/786796573 shutdown_connections 2024-03-04T07:54:44.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bfd000640 1 --2- 172.21.15.113:0/786796573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf80c8b90 0x7f2bf815bdd0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bfd000640 1 --2- 172.21.15.113:0/786796573 >> v2:172.21.15.136:3300/0 conn(0x7f2bf806ff20 0x7f2bf80ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bfd000640 1 --2- 172.21.15.113:0/786796573 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 0x7f2bf80ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bfd000640 1 -- 172.21.15.113:0/786796573 >> 172.21.15.113:0/786796573 conn(0x7f2bf80734d0 msgr2=0x7f2bf8066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bfd000640 1 -- 172.21.15.113:0/786796573 shutdown_connections 2024-03-04T07:54:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.371+0000 7f2bfd000640 1 -- 172.21.15.113:0/786796573 wait complete. 2024-03-04T07:54:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bfd000640 1 Processor -- start 2024-03-04T07:54:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bfd000640 1 -- start start 2024-03-04T07:54:44.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bfd000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf806ff20 0x7f2bf81557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:44.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bfd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bf80c8b90 0x7f2bf80cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bfd000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 0x7f2bf81f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bfd000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf8158f10 con 0x7f2bf806ff20 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bfd000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80c3400 con 0x7f2bf80c8b90 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bfd000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf8159090 con 0x7f2bf8152a50 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf806ff20 0x7f2bf81557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf806ff20 0x7f2bf81557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48430/0 (socket says 172.21.15.113:48430) 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5e00640 1 -- 172.21.15.113:0/1620009536 learned_addr learned my addr 172.21.15.113:0/1620009536 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5400640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bf80c8b90 0x7f2bf80cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf6800640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 0x7f2bf81f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5e00640 1 -- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 msgr2=0x7f2bf81f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5e00640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 0x7f2bf81f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5e00640 1 -- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bf80c8b90 msgr2=0x7f2bf80cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:44.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5e00640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bf80c8b90 0x7f2bf80cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5e00640 1 -- 172.21.15.113:0/1620009536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf80cdde0 con 0x7f2bf806ff20 2024-03-04T07:54:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf6800640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 0x7f2bf81f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:54:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5400640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bf80c8b90 0x7f2bf80cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:54:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bf5e00640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf806ff20 0x7f2bf81557b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2be800d5f0 tx=0x7f2be800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bee000640 1 -- 172.21.15.113:0/1620009536 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be80125f0 con 0x7f2bf806ff20 2024-03-04T07:54:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bee000640 1 -- 172.21.15.113:0/1620009536 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be8019070 con 0x7f2bf806ff20 2024-03-04T07:54:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.375+0000 7f2bee000640 1 -- 172.21.15.113:0/1620009536 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be80129b0 con 0x7f2bf806ff20 2024-03-04T07:54:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.379+0000 7f2bfd000640 1 -- 172.21.15.113:0/1620009536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf80ce0c0 con 0x7f2bf806ff20 2024-03-04T07:54:44.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.379+0000 7f2bfd000640 1 -- 172.21.15.113:0/1620009536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf8154fb0 con 0x7f2bf806ff20 2024-03-04T07:54:44.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.379+0000 7f2bfd000640 1 -- 172.21.15.113:0/1620009536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf8154ac0 con 0x7f2bf806ff20 2024-03-04T07:54:44.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.379+0000 7f2bee000640 1 -- 172.21.15.113:0/1620009536 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2be8012b50 con 0x7f2bf806ff20 2024-03-04T07:54:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.379+0000 7f2bee000640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2bd8066f60 0x7f2bd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.379+0000 7f2bee000640 1 -- 172.21.15.113:0/1620009536 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f2be8085890 con 0x7f2bf806ff20 2024-03-04T07:54:44.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.379+0000 7f2bf5400640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2bd8066f60 0x7f2bd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:44.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.383+0000 7f2bf5400640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2bd8066f60 0x7f2bd8069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80c8a60 tx=0x7f2be4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:44.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.383+0000 7f2bee000640 1 -- 172.21.15.113:0/1620009536 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2be8056460 con 0x7f2bf806ff20 2024-03-04T07:54:44.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.675+0000 7f2bfd000640 1 -- 172.21.15.113:0/1620009536 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bf8075c00 con 0x7f2bf806ff20 2024-03-04T07:54:44.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.675+0000 7f2bee000640 1 -- 172.21.15.113:0/1620009536 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5812 (secure 0 0 0) 0x7f2be8055d60 con 0x7f2bf806ff20 2024-03-04T07:54:44.673 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:44.673 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:54:40.746610+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,6145]},"id":1}]} 2024-03-04T07:54:44.673 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 77 2024-03-04T07:54:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.679+0000 7f2be3400640 1 -- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2bd8066f60 msgr2=0x7f2bd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.679+0000 7f2be3400640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2bd8066f60 0x7f2bd8069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80c8a60 tx=0x7f2be4008040 comp rx=0 tx=0).stop 2024-03-04T07:54:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.679+0000 7f2be3400640 1 -- 172.21.15.113:0/1620009536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf806ff20 msgr2=0x7f2bf81557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:44.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.679+0000 7f2be3400640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf806ff20 0x7f2bf81557b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2be800d5f0 tx=0x7f2be800dab0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.679+0000 7f2be3400640 1 -- 172.21.15.113:0/1620009536 shutdown_connections 2024-03-04T07:54:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.679+0000 7f2be3400640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2bf8152a50 0x7f2bf81f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.679+0000 7f2be3400640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2bf80c8b90 0x7f2bf80cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.679+0000 7f2be3400640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2bd8066f60 0x7f2bd8069420 secure :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2bf80c8a60 tx=0x7f2be4008040 comp rx=0 tx=0).stop 2024-03-04T07:54:44.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.679+0000 7f2be3400640 1 --2- 172.21.15.113:0/1620009536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2bf806ff20 0x7f2bf81557b0 secure :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2be800d5f0 tx=0x7f2be800dab0 comp rx=0 tx=0).stop 2024-03-04T07:54:44.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.679+0000 7f2be3400640 1 -- 172.21.15.113:0/1620009536 >> 172.21.15.113:0/1620009536 conn(0x7f2bf80734d0 msgr2=0x7f2bf8066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:44.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.683+0000 7f2be3400640 1 -- 172.21.15.113:0/1620009536 shutdown_connections 2024-03-04T07:54:44.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:44.683+0000 7f2be3400640 1 -- 172.21.15.113:0/1620009536 wait complete. 2024-03-04T07:54:44.708 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:54:46.709 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:46.800 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 11% (92/779) Updating files: 12% (94/779) Updating files: 13% (102/779) Updating files: 14% (110/779) Updating files: 15% (117/779) Updating files: 16% (125/779) Updating files: 17% (133/779) Updating files: 18% (141/779) Updating files: 19% (149/779) Updating files: 20% (156/779) Updating files: 21% (164/779) Updating files: 21% (170/779) Updating files: 22% (172/779) Updating files: 23% (180/779) Updating files: 24% (187/779) Updating files: 25% (195/779) Updating files: 26% (203/779) Updating files: 27% (211/779) Updating files: 28% (219/779) Updating files: 29% (226/779) Updating files: 30% (234/779) Updating files: 31% (242/779) Updating files: 32% (250/779) Updating files: 33% (258/779) Updating files: 34% (265/779) Updating files: 35% (273/779) Updating files: 36% (281/779) Updating files: 37% (289/779) Updating files: 38% (297/779) Updating files: 39% (304/779) Updating files: 40% (312/779) Updating files: 41% (320/779) Updating files: 42% (328/779) Updating files: 43% (335/779) Updating files: 44% (343/779) Updating files: 45% (351/779) Updating files: 46% (359/779) Updating files: 47% (367/779) Updating files: 48% (374/779) Updating files: 49% (382/779) Updating files: 50% (390/779) Updating files: 51% (398/779) Updating files: 52% (406/779) Updating files: 53% (413/779) Updating files: 54% (421/779) Updating files: 55% (429/779) Updating files: 56% (437/779) Updating files: 57% (445/779) Updating files: 58% (452/779) Updating files: 59% (460/779) Updating files: 60% (468/779) Updating files: 61% (476/779) Updating files: 62% (483/779) Updating files: 63% (491/779) Updating files: 64% (499/779) Updating files: 65% (507/779) Updating files: 66% (515/779) Updating files: 67% (522/779) Updating files: 68% (530/779) Updating files: 68% (537/779) Updating files: 69% (538/779) Updating files: 70% (546/779) Updating files: 71% (554/779) Updating files: 72% (561/779) Updating files: 73% (569/779) Updating files: 74% (577/779) Updating files: 75% (585/779) Updating files: 76% (593/779) Updating files: 77% (600/779) Updating files: 78% (608/779) Updating files: 79% (616/779) Updating files: 80% (624/779) Updating files: 81% (631/779) Updating files: 82% (639/779) Updating files: 83% (647/779) Updating files: 84% (655/779) Updating files: 85% (663/779) Updating files: 85% (668/779) Updating files: 86% (670/779) Updating files: 87% (678/779) Updating files: 88% (686/779) Updating files: 89% (694/779) Updating files: 90% (702/779) Updating files: 91% (709/779) Updating files: 92% (717/779) Updating files: 93% (725/779) Updating files: 94% (733/779) Updating files: 95% (741/779) Updating files: 96% (748/779) Updating files: 97% (756/779) Updating files: 98% (764/779) Updating files: 99% (772/779) Updating files: 100% (779/779) Updating files: 100% (779/779), done. 2024-03-04T07:54:46.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7897200640 1 Processor -- start 2024-03-04T07:54:46.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7897200640 1 -- start start 2024-03-04T07:54:46.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7897200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7898152950 0x7f789815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:46.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7897200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78980c4ce0 con 0x7f789806faa0 2024-03-04T07:54:46.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7897200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78980c4f80 con 0x7f78980cb540 2024-03-04T07:54:46.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7897200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78980c8370 con 0x7f7898152950 2024-03-04T07:54:46.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7895e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7898152950 0x7f789815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:46.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7895e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7898152950 0x7f789815bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48438/0 (socket says 172.21.15.113:48438) 2024-03-04T07:54:46.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7895e00640 1 -- 172.21.15.113:0/3292764226 learned_addr learned my addr 172.21.15.113:0/3292764226 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:46.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7895e00640 1 -- 172.21.15.113:0/3292764226 >> v1:172.21.15.136:6789/0 conn(0x7f78980cb540 legacy=0x7f7898162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:46.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7895e00640 1 -- 172.21.15.113:0/3292764226 >> v1:172.21.15.113:6789/0 conn(0x7f789806faa0 legacy=0x7f789806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:46.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.867+0000 7f7895e00640 1 -- 172.21.15.113:0/3292764226 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78980c5220 con 0x7f7898152950 2024-03-04T07:54:46.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.871+0000 7f7895e00640 1 --2- 172.21.15.113:0/3292764226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7898152950 0x7f789815bb40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7888009e50 tx=0x7f788801f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3d162089fd134d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:46.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.871+0000 7f7895400640 1 -- 172.21.15.113:0/3292764226 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880b4070 con 0x7f7898152950 2024-03-04T07:54:46.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.871+0000 7f7895400640 1 -- 172.21.15.113:0/3292764226 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78880af400 con 0x7f7898152950 2024-03-04T07:54:46.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.871+0000 7f7895400640 1 -- 172.21.15.113:0/3292764226 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880af7d0 con 0x7f7898152950 2024-03-04T07:54:46.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.875+0000 7f7897200640 1 -- 172.21.15.113:0/3292764226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7898152950 msgr2=0x7f789815bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:46.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.875+0000 7f7897200640 1 --2- 172.21.15.113:0/3292764226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7898152950 0x7f789815bb40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7888009e50 tx=0x7f788801f210 comp rx=0 tx=0).stop 2024-03-04T07:54:46.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.875+0000 7f7897200640 1 -- 172.21.15.113:0/3292764226 shutdown_connections 2024-03-04T07:54:46.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.875+0000 7f7897200640 1 --2- 172.21.15.113:0/3292764226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7898152950 0x7f789815bb40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:46.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.875+0000 7f7897200640 1 -- 172.21.15.113:0/3292764226 >> 172.21.15.113:0/3292764226 conn(0x7f7898072f40 msgr2=0x7f78980706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:46.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.875+0000 7f7897200640 1 -- 172.21.15.113:0/3292764226 shutdown_connections 2024-03-04T07:54:46.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.875+0000 7f7897200640 1 -- 172.21.15.113:0/3292764226 wait complete. 2024-03-04T07:54:46.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.875+0000 7f7897200640 1 Processor -- start 2024-03-04T07:54:46.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.875+0000 7f7897200640 1 -- start start 2024-03-04T07:54:46.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7897200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f789806faa0 0x7f78980c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:46.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7897200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78980cb540 0x7f78981f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7897200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7898152950 0x7f789806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7897200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78980c9a50 con 0x7f789806faa0 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7897200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78980ca0a0 con 0x7f78980cb540 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7897200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78980c9d30 con 0x7f7898152950 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7895e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f789806faa0 0x7f78980c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7895e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f789806faa0 0x7f78980c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48444/0 (socket says 172.21.15.113:48444) 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7895e00640 1 -- 172.21.15.113:0/2980274722 learned_addr learned my addr 172.21.15.113:0/2980274722 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7896800640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7898152950 0x7f789806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f788d400640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78980cb540 0x7f78981f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7895e00640 1 -- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7898152950 msgr2=0x7f789806ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7895e00640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7898152950 0x7f789806ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7895e00640 1 -- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78980cb540 msgr2=0x7f78981f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:46.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7895e00640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78980cb540 0x7f78981f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:46.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7895e00640 1 -- 172.21.15.113:0/2980274722 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78981f46d0 con 0x7f789806faa0 2024-03-04T07:54:46.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7895e00640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f789806faa0 0x7f78980c5ca0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f788801ce40 tx=0x7f788801bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:46.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f788ea00640 1 -- 172.21.15.113:0/2980274722 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880c0070 con 0x7f789806faa0 2024-03-04T07:54:46.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f788ea00640 1 -- 172.21.15.113:0/2980274722 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78880afe10 con 0x7f789806faa0 2024-03-04T07:54:46.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7897200640 1 -- 172.21.15.113:0/2980274722 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78980c6300 con 0x7f789806faa0 2024-03-04T07:54:46.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f788ea00640 1 -- 172.21.15.113:0/2980274722 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880b4040 con 0x7f789806faa0 2024-03-04T07:54:46.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.879+0000 7f7897200640 1 -- 172.21.15.113:0/2980274722 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78980c68a0 con 0x7f789806faa0 2024-03-04T07:54:46.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.883+0000 7f7887400640 1 -- 172.21.15.113:0/2980274722 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7898075a80 con 0x7f789806faa0 2024-03-04T07:54:46.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.883+0000 7f788ea00640 1 -- 172.21.15.113:0/2980274722 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78880c3070 con 0x7f789806faa0 2024-03-04T07:54:46.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.883+0000 7f788ea00640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7874066f60 0x7f7874069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:46.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.883+0000 7f788ea00640 1 -- 172.21.15.113:0/2980274722 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f7888122b60 con 0x7f789806faa0 2024-03-04T07:54:46.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.887+0000 7f788d400640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7874066f60 0x7f7874069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:46.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.887+0000 7f788d400640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7874066f60 0x7f7874069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f78980c7680 tx=0x7f7880009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:46.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:46.891+0000 7f788ea00640 1 -- 172.21.15.113:0/2980274722 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7888018d70 con 0x7f789806faa0 2024-03-04T07:54:47.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 -- 172.21.15.113:0/2980274722 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78980c9580 con 0x7f789806faa0 2024-03-04T07:54:47.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f788ea00640 1 -- 172.21.15.113:0/2980274722 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+5814 (secure 0 0 0) 0x7f78880f3080 con 0x7f789806faa0 2024-03-04T07:54:47.112 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:47.112 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:54:45.757571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:resolve","state_seq":35,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940]},"id":1}]} 2024-03-04T07:54:47.112 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 78 2024-03-04T07:54:47.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 -- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7874066f60 msgr2=0x7f7874069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7874066f60 0x7f7874069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f78980c7680 tx=0x7f7880009040 comp rx=0 tx=0).stop 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 -- 172.21.15.113:0/2980274722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f789806faa0 msgr2=0x7f78980c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f789806faa0 0x7f78980c5ca0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f788801ce40 tx=0x7f788801bfb0 comp rx=0 tx=0).stop 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 -- 172.21.15.113:0/2980274722 shutdown_connections 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7898152950 0x7f789806ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78980cb540 0x7f78981f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7874066f60 0x7f7874069420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 --2- 172.21.15.113:0/2980274722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f789806faa0 0x7f78980c5ca0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 -- 172.21.15.113:0/2980274722 >> 172.21.15.113:0/2980274722 conn(0x7f7898072f40 msgr2=0x7f7898156390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 -- 172.21.15.113:0/2980274722 shutdown_connections 2024-03-04T07:54:47.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:47.115+0000 7f7887400640 1 -- 172.21.15.113:0/2980274722 wait complete. 2024-03-04T07:54:47.132 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:54:47.509 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 2f10137986a v0.18 2024-03-04T07:54:47.513 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.18 2024-03-04T07:54:49.133 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:49.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.271+0000 7fbdebc00640 1 Processor -- start 2024-03-04T07:54:49.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.271+0000 7fbdebc00640 1 -- start start 2024-03-04T07:54:49.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbdebc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbde40c8200 0x7fbde4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:49.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbdebc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbde4152bd0 0x7fbde41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:49.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbdebc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde41591f0 con 0x7fbde41527f0 2024-03-04T07:54:49.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbdebc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde4071df0 con 0x7fbde4152bd0 2024-03-04T07:54:49.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbdebc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde41597b0 con 0x7fbde40c8200 2024-03-04T07:54:49.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbde9200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbde41527f0 0x7fbde4072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37830/0 (socket says 172.21.15.113:37830) 2024-03-04T07:54:49.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbde9200640 1 -- 172.21.15.113:0/1314461719 learned_addr learned my addr 172.21.15.113:0/1314461719 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:54:49.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbde9c00640 1 --2- 172.21.15.113:0/1314461719 >> v2:172.21.15.113:3300/0 conn(0x7fbde4152bd0 0x7fbde41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:49.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbde3e00640 1 --2- 172.21.15.113:0/1314461719 >> v2:172.21.15.136:3300/0 conn(0x7fbde40c8200 0x7fbde4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:49.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbde3400640 1 -- 172.21.15.113:0/1314461719 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557586405 0 0) 0x7fbde41591f0 con 0x7fbde41527f0 2024-03-04T07:54:49.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbde9c00640 1 -- 172.21.15.113:0/1314461719 >> v2:172.21.15.136:3300/0 conn(0x7fbde40c8200 msgr2=0x7fbde4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:49.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbde9c00640 1 --2- 172.21.15.113:0/1314461719 >> v2:172.21.15.136:3300/0 conn(0x7fbde40c8200 0x7fbde4070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:49.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbde9c00640 1 -- 172.21.15.113:0/1314461719 >> v1:172.21.15.113:6789/0 conn(0x7fbde41527f0 legacy=0x7fbde4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:49.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbde9c00640 1 -- 172.21.15.113:0/1314461719 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde41594d0 con 0x7fbde4152bd0 2024-03-04T07:54:49.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.275+0000 7fbde3e00640 1 --2- 172.21.15.113:0/1314461719 >> v2:172.21.15.136:3300/0 conn(0x7fbde40c8200 0x7fbde4070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:54:49.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbde9c00640 1 --2- 172.21.15.113:0/1314461719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbde4152bd0 0x7fbde41587d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbde40cd230 tx=0x7fbddc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7b6e207b0ad183 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:49.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbde3400640 1 -- 172.21.15.113:0/1314461719 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0b2070 con 0x7fbde4152bd0 2024-03-04T07:54:49.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbde3400640 1 -- 172.21.15.113:0/1314461719 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc0a58e0 con 0x7fbde4152bd0 2024-03-04T07:54:49.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbde3400640 1 -- 172.21.15.113:0/1314461719 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0a5cb0 con 0x7fbde4152bd0 2024-03-04T07:54:49.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbdebc00640 1 -- 172.21.15.113:0/1314461719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbde4152bd0 msgr2=0x7fbde41587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:49.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbdebc00640 1 --2- 172.21.15.113:0/1314461719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbde4152bd0 0x7fbde41587d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbde40cd230 tx=0x7fbddc01c930 comp rx=0 tx=0).stop 2024-03-04T07:54:49.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbdebc00640 1 -- 172.21.15.113:0/1314461719 shutdown_connections 2024-03-04T07:54:49.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbdebc00640 1 --2- 172.21.15.113:0/1314461719 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbde4152bd0 0x7fbde41587d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:49.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbdebc00640 1 --2- 172.21.15.113:0/1314461719 >> v2:172.21.15.136:3300/0 conn(0x7fbde40c8200 0x7fbde4070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:49.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbdebc00640 1 -- 172.21.15.113:0/1314461719 >> 172.21.15.113:0/1314461719 conn(0x7fbde406e820 msgr2=0x7fbde406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:49.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbdebc00640 1 -- 172.21.15.113:0/1314461719 shutdown_connections 2024-03-04T07:54:49.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbdebc00640 1 -- 172.21.15.113:0/1314461719 wait complete. 2024-03-04T07:54:49.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbdebc00640 1 Processor -- start 2024-03-04T07:54:49.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.279+0000 7fbdebc00640 1 -- start start 2024-03-04T07:54:49.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbdebc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbde40c8200 0x7fbde41f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:49.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbdebc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde41527f0 0x7fbde415ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:49.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbdebc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbde4152bd0 0x7fbde415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:49.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbdebc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde41654f0 con 0x7fbde4152bd0 2024-03-04T07:54:49.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbdebc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde4165370 con 0x7fbde41527f0 2024-03-04T07:54:49.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbdebc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde4165670 con 0x7fbde40c8200 2024-03-04T07:54:49.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde9c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbde4152bd0 0x7fbde415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:49.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde9200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbde40c8200 0x7fbde41f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:49.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde9200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbde40c8200 0x7fbde41f4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:51602/0 (socket says 172.21.15.113:51602) 2024-03-04T07:54:49.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde9200640 1 -- 172.21.15.113:0/833242328 learned_addr learned my addr 172.21.15.113:0/833242328 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:49.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde3e00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde41527f0 0x7fbde415ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde9c00640 1 -- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbde40c8200 msgr2=0x7fbde41f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde9c00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbde40c8200 0x7fbde41f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde9c00640 1 -- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde41527f0 msgr2=0x7fbde415ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde9c00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde41527f0 0x7fbde415ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde9c00640 1 -- 172.21.15.113:0/833242328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde4155bd0 con 0x7fbde4152bd0 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde9c00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbde4152bd0 0x7fbde415b2a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0ac000 tx=0x7fbddc0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde0c00640 1 -- 172.21.15.113:0/833242328 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0b2050 con 0x7fbde4152bd0 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde0c00640 1 -- 172.21.15.113:0/833242328 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc0be070 con 0x7fbde4152bd0 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbde0c00640 1 -- 172.21.15.113:0/833242328 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0ada50 con 0x7fbde4152bd0 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbdebc00640 1 -- 172.21.15.113:0/833242328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde4155e90 con 0x7fbde4152bd0 2024-03-04T07:54:49.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbdebc00640 1 -- 172.21.15.113:0/833242328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde41f3970 con 0x7fbde4152bd0 2024-03-04T07:54:49.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.283+0000 7fbdebc00640 1 -- 172.21.15.113:0/833242328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde4075c00 con 0x7fbde4152bd0 2024-03-04T07:54:49.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.287+0000 7fbde0c00640 1 -- 172.21.15.113:0/833242328 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbddc0a5760 con 0x7fbde4152bd0 2024-03-04T07:54:49.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.287+0000 7fbde0c00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdc4066f60 0x7fbdc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:49.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.287+0000 7fbde0c00640 1 -- 172.21.15.113:0/833242328 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7fbddc120500 con 0x7fbde4152bd0 2024-03-04T07:54:49.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.287+0000 7fbde9200640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdc4066f60 0x7fbdc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:49.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.287+0000 7fbde9200640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdc4066f60 0x7fbdc4069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8000f70 tx=0x7fbdd8007e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:49.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.291+0000 7fbde0c00640 1 -- 172.21.15.113:0/833242328 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbddc016520 con 0x7fbde4152bd0 2024-03-04T07:54:49.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 -- 172.21.15.113:0/833242328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbde40c79d0 con 0x7fbde4152bd0 2024-03-04T07:54:49.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbde0c00640 1 -- 172.21.15.113:0/833242328 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+5814 (secure 0 0 0) 0x7fbddc0f09f0 con 0x7fbde4152bd0 2024-03-04T07:54:49.475 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:49.475 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:54:45.757571+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:resolve","state_seq":35,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940]},"id":1}]} 2024-03-04T07:54:49.475 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 78 2024-03-04T07:54:49.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 -- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdc4066f60 msgr2=0x7fbdc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdc4066f60 0x7fbdc4069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8000f70 tx=0x7fbdd8007e20 comp rx=0 tx=0).stop 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 -- 172.21.15.113:0/833242328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbde4152bd0 msgr2=0x7fbde415b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbde4152bd0 0x7fbde415b2a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0ac000 tx=0x7fbddc0196d0 comp rx=0 tx=0).stop 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 -- 172.21.15.113:0/833242328 shutdown_connections 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdc4066f60 0x7fbdc4069420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbde4152bd0 0x7fbde415b2a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbde41527f0 0x7fbde415ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 --2- 172.21.15.113:0/833242328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbde40c8200 0x7fbde41f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 -- 172.21.15.113:0/833242328 >> 172.21.15.113:0/833242328 conn(0x7fbde406e820 msgr2=0x7fbde4072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 -- 172.21.15.113:0/833242328 shutdown_connections 2024-03-04T07:54:49.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:49.479+0000 7fbdebc00640 1 -- 172.21.15.113:0/833242328 wait complete. 2024-03-04T07:54:49.496 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:54:51.059 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 19 -eq 48 ] 2024-03-04T07:54:51.060 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.19 2024-03-04T07:54:51.060 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.19 2024-03-04T07:54:51.060 INFO:tasks.workunit.client.0.smithi113.stdout:v0.19 2024-03-04T07:54:51.060 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.19 2024-03-04T07:54:51.497 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:51.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.659+0000 7f231e400640 1 Processor -- start 2024-03-04T07:54:51.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.659+0000 7f231e400640 1 -- start start 2024-03-04T07:54:51.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f231806c170 0x7f231815b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:51.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231e400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f23180c8f70 0x7f23180c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:51.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231e400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23180c9920 con 0x7f23180c4a30 2024-03-04T07:54:51.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231e400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231815b980 con 0x7f23180c8f70 2024-03-04T07:54:51.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231806f280 con 0x7f231806c170 2024-03-04T07:54:51.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231d000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f23180c4a30 0x7f23180ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37832/0 (socket says 172.21.15.113:37832) 2024-03-04T07:54:51.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231d000640 1 -- 172.21.15.113:0/3379612184 learned_addr learned my addr 172.21.15.113:0/3379612184 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:54:51.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f2317e00640 1 --2- 172.21.15.113:0/3379612184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f231806c170 0x7f231815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:51.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231da00640 1 --2- 172.21.15.113:0/3379612184 >> v2:172.21.15.136:3300/0 conn(0x7f23180c8f70 0x7f23180c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f2317400640 1 -- 172.21.15.113:0/3379612184 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037406405 0 0) 0x7f23180c9920 con 0x7f23180c4a30 2024-03-04T07:54:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f2317400640 1 -- 172.21.15.113:0/3379612184 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2300003530 con 0x7f23180c4a30 2024-03-04T07:54:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231da00640 1 -- 172.21.15.113:0/3379612184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f231806c170 msgr2=0x7f231815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231da00640 1 --2- 172.21.15.113:0/3379612184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f231806c170 0x7f231815b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231da00640 1 -- 172.21.15.113:0/3379612184 >> v1:172.21.15.113:6789/0 conn(0x7f23180c4a30 legacy=0x7f23180ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f231da00640 1 -- 172.21.15.113:0/3379612184 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231806dd30 con 0x7f23180c8f70 2024-03-04T07:54:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.663+0000 7f2317e00640 1 --2- 172.21.15.113:0/3379612184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f231806c170 0x7f231815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:54:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f231da00640 1 --2- 172.21.15.113:0/3379612184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23180c8f70 0x7f23180c93e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2310009d90 tx=0x7f231001f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79872e4c4dc64c84 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:51.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f2317400640 1 -- 172.21.15.113:0/3379612184 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23100b4070 con 0x7f23180c8f70 2024-03-04T07:54:51.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f2317400640 1 -- 172.21.15.113:0/3379612184 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23100af600 con 0x7f23180c8f70 2024-03-04T07:54:51.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f2317400640 1 -- 172.21.15.113:0/3379612184 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23100af9b0 con 0x7f23180c8f70 2024-03-04T07:54:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f231e400640 1 -- 172.21.15.113:0/3379612184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23180c8f70 msgr2=0x7f23180c93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f231e400640 1 --2- 172.21.15.113:0/3379612184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23180c8f70 0x7f23180c93e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2310009d90 tx=0x7f231001f150 comp rx=0 tx=0).stop 2024-03-04T07:54:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f231e400640 1 -- 172.21.15.113:0/3379612184 shutdown_connections 2024-03-04T07:54:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f231e400640 1 --2- 172.21.15.113:0/3379612184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23180c8f70 0x7f23180c93e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f231e400640 1 --2- 172.21.15.113:0/3379612184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f231806c170 0x7f231815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f231e400640 1 -- 172.21.15.113:0/3379612184 >> 172.21.15.113:0/3379612184 conn(0x7f2318072e70 msgr2=0x7f2318066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f231e400640 1 -- 172.21.15.113:0/3379612184 shutdown_connections 2024-03-04T07:54:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.667+0000 7f231e400640 1 -- 172.21.15.113:0/3379612184 wait complete. 2024-03-04T07:54:51.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f231e400640 1 Processor -- start 2024-03-04T07:54:51.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f231e400640 1 -- start start 2024-03-04T07:54:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f231e400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f231806c170 0x7f23181e0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f231e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23180c4a30 0x7f23180c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f231e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23180c8f70 0x7f23180cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f231e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231806dfd0 con 0x7f23180c4a30 2024-03-04T07:54:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f231e400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231806fe40 con 0x7f231806c170 2024-03-04T07:54:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f231e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231806f4d0 con 0x7f23180c8f70 2024-03-04T07:54:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f2317e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23180c4a30 0x7f23180c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f231d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f231806c170 0x7f23181e0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f2317e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23180c4a30 0x7f23180c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56644/0 (socket says 172.21.15.113:56644) 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.671+0000 7f2317e00640 1 -- 172.21.15.113:0/1973030565 learned_addr learned my addr 172.21.15.113:0/1973030565 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f231da00640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23180c8f70 0x7f23180cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f2317e00640 1 -- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23180c8f70 msgr2=0x7f23180cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f2317e00640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23180c8f70 0x7f23180cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f2317e00640 1 -- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f231806c170 msgr2=0x7f23181e0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f2317e00640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f231806c170 0x7f23181e0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f2317e00640 1 -- 172.21.15.113:0/1973030565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2318159280 con 0x7f23180c4a30 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f2317e00640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23180c4a30 0x7f23180c7c10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f230400ec10 tx=0x7f230400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f2314c00640 1 -- 172.21.15.113:0/1973030565 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304017070 con 0x7f23180c4a30 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f2314c00640 1 -- 172.21.15.113:0/1973030565 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230400cc30 con 0x7f23180c4a30 2024-03-04T07:54:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f2314c00640 1 -- 172.21.15.113:0/1973030565 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304013470 con 0x7f23180c4a30 2024-03-04T07:54:51.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f231e400640 1 -- 172.21.15.113:0/1973030565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23181e64c0 con 0x7f23180c4a30 2024-03-04T07:54:51.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f231e400640 1 -- 172.21.15.113:0/1973030565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23181e6760 con 0x7f23180c4a30 2024-03-04T07:54:51.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.675+0000 7f231e400640 1 -- 172.21.15.113:0/1973030565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2318075a80 con 0x7f23180c4a30 2024-03-04T07:54:51.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.679+0000 7f2314c00640 1 -- 172.21.15.113:0/1973030565 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f230401a070 con 0x7f23180c4a30 2024-03-04T07:54:51.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.679+0000 7f2314c00640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22f8066f90 0x7f22f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:51.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.679+0000 7f2314c00640 1 -- 172.21.15.113:0/1973030565 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f2304085440 con 0x7f23180c4a30 2024-03-04T07:54:51.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.679+0000 7f231d000640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22f8066f90 0x7f22f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:51.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.679+0000 7f231d000640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22f8066f90 0x7f22f8069450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f230c007fd0 tx=0x7f230c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:51.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.683+0000 7f2314c00640 1 -- 172.21.15.113:0/1973030565 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f23040558a0 con 0x7f23180c4a30 2024-03-04T07:54:51.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.871+0000 7f231e400640 1 -- 172.21.15.113:0/1973030565 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f231806fb90 con 0x7f23180c4a30 2024-03-04T07:54:51.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f2314c00640 1 -- 172.21.15.113:0/1973030565 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f2304059940 con 0x7f23180c4a30 2024-03-04T07:54:51.870 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:51.870 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:54:51.055199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:reconnect","state_seq":37,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,6145]},"id":1}]} 2024-03-04T07:54:51.870 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 79 2024-03-04T07:54:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 -- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22f8066f90 msgr2=0x7f22f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22f8066f90 0x7f22f8069450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f230c007fd0 tx=0x7f230c009040 comp rx=0 tx=0).stop 2024-03-04T07:54:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 -- 172.21.15.113:0/1973030565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23180c4a30 msgr2=0x7f23180c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23180c4a30 0x7f23180c7c10 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f230400ec10 tx=0x7f230400c520 comp rx=0 tx=0).stop 2024-03-04T07:54:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 -- 172.21.15.113:0/1973030565 shutdown_connections 2024-03-04T07:54:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23180c8f70 0x7f23180cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22f8066f90 0x7f22f8069450 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23180c4a30 0x7f23180c7c10 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 --2- 172.21.15.113:0/1973030565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f231806c170 0x7f23181e0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 -- 172.21.15.113:0/1973030565 >> 172.21.15.113:0/1973030565 conn(0x7f2318072e70 msgr2=0x7f231815b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 -- 172.21.15.113:0/1973030565 shutdown_connections 2024-03-04T07:54:51.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:51.875+0000 7f231e400640 1 -- 172.21.15.113:0/1973030565 wait complete. 2024-03-04T07:54:51.892 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:54:53.892 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:54.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.031+0000 7ff8ce000640 1 Processor -- start 2024-03-04T07:54:54.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.031+0000 7ff8ce000640 1 -- start start 2024-03-04T07:54:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.031+0000 7ff8ce000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8c8152a50 0x7ff8c80ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.031+0000 7ff8ce000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff8c815c4e0 0x7ff8c80ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.031+0000 7ff8ce000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c8152e30 con 0x7ff8c806ff20 2024-03-04T07:54:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.031+0000 7ff8ce000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c80c31b0 con 0x7ff8c8152a50 2024-03-04T07:54:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.031+0000 7ff8ce000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c806f840 con 0x7ff8c815c4e0 2024-03-04T07:54:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8c8152a50 0x7ff8c80ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff8c8152a50 0x7ff8c80ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56660/0 (socket says 172.21.15.113:56660) 2024-03-04T07:54:54.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c7400640 1 -- 172.21.15.113:0/4163436980 learned_addr learned my addr 172.21.15.113:0/4163436980 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:54.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c7e00640 1 --2- 172.21.15.113:0/4163436980 >> v2:172.21.15.136:3300/0 conn(0x7ff8c815c4e0 0x7ff8c80ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:54.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c7400640 1 -- 172.21.15.113:0/4163436980 >> v2:172.21.15.136:3300/0 conn(0x7ff8c815c4e0 msgr2=0x7ff8c80ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:54.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c7400640 1 --2- 172.21.15.113:0/4163436980 >> v2:172.21.15.136:3300/0 conn(0x7ff8c815c4e0 0x7ff8c80ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:54.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c7400640 1 -- 172.21.15.113:0/4163436980 >> v1:172.21.15.113:6789/0 conn(0x7ff8c806ff20 legacy=0x7ff8c815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:54.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c7400640 1 -- 172.21.15.113:0/4163436980 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c806f280 con 0x7ff8c8152a50 2024-03-04T07:54:54.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c6000640 1 -- 172.21.15.113:0/4163436980 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510702807 0 0) 0x7ff8c8152e30 con 0x7ff8c806ff20 2024-03-04T07:54:54.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c7e00640 1 --2- 172.21.15.113:0/4163436980 >> v2:172.21.15.136:3300/0 conn(0x7ff8c815c4e0 0x7ff8c80ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:54:54.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c7400640 1 --2- 172.21.15.113:0/4163436980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 0x7ff8c80ce120 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff8c806ef00 tx=0x7ff8b801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a645e375d3bfc0b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:54.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c6000640 1 -- 172.21.15.113:0/4163436980 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80aa680 con 0x7ff8c8152a50 2024-03-04T07:54:54.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c6000640 1 -- 172.21.15.113:0/4163436980 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b80af070 con 0x7ff8c8152a50 2024-03-04T07:54:54.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.035+0000 7ff8c6000640 1 -- 172.21.15.113:0/4163436980 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80aaa40 con 0x7ff8c8152a50 2024-03-04T07:54:54.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.039+0000 7ff8ce000640 1 -- 172.21.15.113:0/4163436980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 msgr2=0x7ff8c80ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:54.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.039+0000 7ff8ce000640 1 --2- 172.21.15.113:0/4163436980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 0x7ff8c80ce120 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff8c806ef00 tx=0x7ff8b801c520 comp rx=0 tx=0).stop 2024-03-04T07:54:54.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.039+0000 7ff8ce000640 1 -- 172.21.15.113:0/4163436980 shutdown_connections 2024-03-04T07:54:54.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.039+0000 7ff8ce000640 1 --2- 172.21.15.113:0/4163436980 >> v2:172.21.15.136:3300/0 conn(0x7ff8c815c4e0 0x7ff8c80ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:54.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.039+0000 7ff8ce000640 1 --2- 172.21.15.113:0/4163436980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 0x7ff8c80ce120 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:54.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.039+0000 7ff8ce000640 1 -- 172.21.15.113:0/4163436980 >> 172.21.15.113:0/4163436980 conn(0x7ff8c80734d0 msgr2=0x7ff8c8066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:54.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.039+0000 7ff8ce000640 1 -- 172.21.15.113:0/4163436980 shutdown_connections 2024-03-04T07:54:54.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.039+0000 7ff8ce000640 1 -- 172.21.15.113:0/4163436980 wait complete. 2024-03-04T07:54:54.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.039+0000 7ff8ce000640 1 Processor -- start 2024-03-04T07:54:54.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.039+0000 7ff8ce000640 1 -- start start 2024-03-04T07:54:54.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8ce000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8c806ff20 0x7ff8c815af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:54.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8ce000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 0x7ff8c81587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:54.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8ce000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8c815c4e0 0x7ff8c8158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8ce000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8165a00 con 0x7ff8c8152a50 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8ce000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8165880 con 0x7ff8c815c4e0 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8ce000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8165b80 con 0x7ff8c806ff20 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 0x7ff8c81587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8c806ff20 0x7ff8c815af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 0x7ff8c81587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56676/0 (socket says 172.21.15.113:56676) 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c6a00640 1 -- 172.21.15.113:0/3054535334 learned_addr learned my addr 172.21.15.113:0/3054535334 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c7e00640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8c815c4e0 0x7ff8c8158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c6a00640 1 -- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8c806ff20 msgr2=0x7ff8c815af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c6a00640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8c806ff20 0x7ff8c815af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c6a00640 1 -- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8c815c4e0 msgr2=0x7ff8c8158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:54.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c6a00640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8c815c4e0 0x7ff8c8158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:54.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c6a00640 1 -- 172.21.15.113:0/3054535334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c814de40 con 0x7ff8c8152a50 2024-03-04T07:54:54.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8c6a00640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 0x7ff8c81587c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400c4d0 tx=0x7ff8b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:54.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8bf400640 1 -- 172.21.15.113:0/3054535334 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4017070 con 0x7ff8c8152a50 2024-03-04T07:54:54.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8bf400640 1 -- 172.21.15.113:0/3054535334 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b4005480 con 0x7ff8c8152a50 2024-03-04T07:54:54.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8ce000640 1 -- 172.21.15.113:0/3054535334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c81f47c0 con 0x7ff8c8152a50 2024-03-04T07:54:54.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8bf400640 1 -- 172.21.15.113:0/3054535334 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4003840 con 0x7ff8c8152a50 2024-03-04T07:54:54.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8ce000640 1 -- 172.21.15.113:0/3054535334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c814e3c0 con 0x7ff8c8152a50 2024-03-04T07:54:54.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.043+0000 7ff8ce000640 1 -- 172.21.15.113:0/3054535334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c8075c00 con 0x7ff8c8152a50 2024-03-04T07:54:54.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.047+0000 7ff8bf400640 1 -- 172.21.15.113:0/3054535334 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8b401a070 con 0x7ff8c8152a50 2024-03-04T07:54:54.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.047+0000 7ff8bf400640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8a8066f60 0x7ff8a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:54.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.047+0000 7ff8bf400640 1 -- 172.21.15.113:0/3054535334 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7ff8b40848d0 con 0x7ff8c8152a50 2024-03-04T07:54:54.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.047+0000 7ff8c7400640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8a8066f60 0x7ff8a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:54.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.047+0000 7ff8c7400640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8a8066f60 0x7ff8a8069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80ae000 tx=0x7ff8b80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:54.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.051+0000 7ff8bf400640 1 -- 172.21.15.113:0/3054535334 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff8b4054df0 con 0x7ff8c8152a50 2024-03-04T07:54:54.376 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 64% (140/217) Updating files: 65% (142/217) Updating files: 66% (144/217) Updating files: 67% (146/217) Updating files: 68% (148/217) Updating files: 69% (150/217) Updating files: 70% (152/217) Updating files: 71% (155/217) Updating files: 72% (157/217) Updating files: 73% (159/217) Updating files: 74% (161/217) Updating files: 75% (163/217) Updating files: 76% (165/217) Updating files: 77% (168/217) Updating files: 78% (170/217) Updating files: 79% (172/217) Updating files: 80% (174/217) Updating files: 81% (176/217) Updating files: 82% (178/217) Updating files: 83% (181/217) Updating files: 84% (183/217) Updating files: 85% (185/217) Updating files: 86% (187/217) Updating files: 87% (189/217) Updating files: 88% (191/217) Updating files: 89% (194/217) Updating files: 90% (196/217) Updating files: 91% (198/217) Updating files: 92% (200/217) Updating files: 93% (202/217) Updating files: 94% (204/217) Updating files: 95% (207/217) Updating files: 96% (209/217) Updating files: 97% (211/217) Updating files: 98% (213/217) Updating files: 99% (215/217) Updating files: 100% (217/217) Updating files: 100% (217/217), done. 2024-03-04T07:54:54.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.423+0000 7ff8ce000640 1 -- 172.21.15.113:0/3054535334 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8c815c1e0 con 0x7ff8c8152a50 2024-03-04T07:54:54.442 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:54.442 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:54:51.055199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:reconnect","state_seq":37,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,6145]},"id":1}]} 2024-03-04T07:54:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.439+0000 7ff8bf400640 1 -- 172.21.15.113:0/3054535334 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+5816 (secure 0 0 0) 0x7ff8b4058e90 con 0x7ff8c8152a50 2024-03-04T07:54:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 79 2024-03-04T07:54:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.447+0000 7ff8ce000640 1 -- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8a8066f60 msgr2=0x7ff8a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.447+0000 7ff8ce000640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8a8066f60 0x7ff8a8069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80ae000 tx=0x7ff8b80a7000 comp rx=0 tx=0).stop 2024-03-04T07:54:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.447+0000 7ff8ce000640 1 -- 172.21.15.113:0/3054535334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 msgr2=0x7ff8c81587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.447+0000 7ff8ce000640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 0x7ff8c81587c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400c4d0 tx=0x7ff8b400c990 comp rx=0 tx=0).stop 2024-03-04T07:54:54.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.451+0000 7ff8ce000640 1 -- 172.21.15.113:0/3054535334 shutdown_connections 2024-03-04T07:54:54.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.451+0000 7ff8ce000640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8c815c4e0 0x7ff8c8158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:54.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.451+0000 7ff8ce000640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff8a8066f60 0x7ff8a8069420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:54.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.451+0000 7ff8ce000640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8c8152a50 0x7ff8c81587c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:54.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.451+0000 7ff8ce000640 1 --2- 172.21.15.113:0/3054535334 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff8c806ff20 0x7ff8c815af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:54.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.451+0000 7ff8ce000640 1 -- 172.21.15.113:0/3054535334 >> 172.21.15.113:0/3054535334 conn(0x7ff8c80734d0 msgr2=0x7ff8c80ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:54.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.455+0000 7ff8ce000640 1 -- 172.21.15.113:0/3054535334 shutdown_connections 2024-03-04T07:54:54.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:54.455+0000 7ff8ce000640 1 -- 172.21.15.113:0/3054535334 wait complete. 2024-03-04T07:54:54.522 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 79, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 79, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5940": { "addr": "172.21.15.113:6837/344022786", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 344022786, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 344022786, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 5940, "incarnation": 64, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 30 }, "gid_6145": { "addr": "172.21.15.136:6835/2368734308", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2368734308, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2368734308, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6145, "incarnation": 71, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_6240": { "addr": "172.21.15.113:6836/986515123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 986515123, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 986515123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6240, "incarnation": 77, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:reconnect", "state_seq": 37 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:54:51.055199+0000", "qdb_cluster": [ 5940, 6145 ], "qdb_leader": 5940, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5940, "mds_1": 6145, "mds_2": 6240 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:54:56.107 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 98f5be53f85 v0.19 2024-03-04T07:54:56.110 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.19 2024-03-04T07:54:56.523 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:56.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf81400640 1 Processor -- start 2024-03-04T07:54:56.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf81400640 1 -- start start 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf81400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c0c8200 0x7faf7c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf81400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7c1591f0 con 0x7faf7c1527f0 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf81400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7c1594d0 con 0x7faf7c152bd0 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf81400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c071df0 con 0x7faf7c0c8200 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf7a800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7faf7c1527f0 0x7faf7c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37858/0 (socket says 172.21.15.113:37858) 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf7a800640 1 -- 172.21.15.113:0/1569387768 learned_addr learned my addr 172.21.15.113:0/1569387768 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf73e00640 1 --2- 172.21.15.113:0/1569387768 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c0c8200 0x7faf7c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf79e00640 1 -- 172.21.15.113:0/1569387768 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780153672 0 0) 0x7faf7c1591f0 con 0x7faf7c1527f0 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.663+0000 7faf79e00640 1 -- 172.21.15.113:0/1569387768 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf60003530 con 0x7faf7c1527f0 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf73e00640 1 -- 172.21.15.113:0/1569387768 >> v1:172.21.15.136:6789/0 conn(0x7faf7c152bd0 legacy=0x7faf7c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:56.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf73e00640 1 -- 172.21.15.113:0/1569387768 >> v1:172.21.15.113:6789/0 conn(0x7faf7c1527f0 legacy=0x7faf7c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:56.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf73e00640 1 -- 172.21.15.113:0/1569387768 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c0717a0 con 0x7faf7c0c8200 2024-03-04T07:54:56.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf73e00640 1 --2- 172.21.15.113:0/1569387768 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c0c8200 0x7faf7c070d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf640097f0 tx=0x7faf6401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ce98851fc4515aa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:56.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf79e00640 1 -- 172.21.15.113:0/1569387768 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf640ab5f0 con 0x7faf7c0c8200 2024-03-04T07:54:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf79e00640 1 -- 172.21.15.113:0/1569387768 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf640b0070 con 0x7faf7c0c8200 2024-03-04T07:54:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf79e00640 1 -- 172.21.15.113:0/1569387768 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf640ab990 con 0x7faf7c0c8200 2024-03-04T07:54:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf81400640 1 -- 172.21.15.113:0/1569387768 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c0c8200 msgr2=0x7faf7c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf81400640 1 --2- 172.21.15.113:0/1569387768 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c0c8200 0x7faf7c070d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf640097f0 tx=0x7faf6401c4e0 comp rx=0 tx=0).stop 2024-03-04T07:54:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf81400640 1 -- 172.21.15.113:0/1569387768 shutdown_connections 2024-03-04T07:54:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf81400640 1 --2- 172.21.15.113:0/1569387768 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c0c8200 0x7faf7c070d40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf81400640 1 -- 172.21.15.113:0/1569387768 >> 172.21.15.113:0/1569387768 conn(0x7faf7c06e820 msgr2=0x7faf7c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:56.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf81400640 1 -- 172.21.15.113:0/1569387768 shutdown_connections 2024-03-04T07:54:56.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.667+0000 7faf81400640 1 -- 172.21.15.113:0/1569387768 wait complete. 2024-03-04T07:54:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf81400640 1 Processor -- start 2024-03-04T07:54:56.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf81400640 1 -- start start 2024-03-04T07:54:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf81400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf7c0c8200 0x7faf7c1f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf81400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7c1527f0 0x7faf7c149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf81400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c152bd0 0x7faf7c0c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf81400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c166380 con 0x7faf7c1527f0 2024-03-04T07:54:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf81400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c1597b0 con 0x7faf7c0c8200 2024-03-04T07:54:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf81400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c166500 con 0x7faf7c152bd0 2024-03-04T07:54:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf73e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7c1527f0 0x7faf7c149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:56.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf73e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7c1527f0 0x7faf7c149530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56688/0 (socket says 172.21.15.113:56688) 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf73e00640 1 -- 172.21.15.113:0/3242935571 learned_addr learned my addr 172.21.15.113:0/3242935571 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf7b200640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c152bd0 0x7faf7c0c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf7a800640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf7c0c8200 0x7faf7c1f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf73e00640 1 -- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c152bd0 msgr2=0x7faf7c0c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf73e00640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c152bd0 0x7faf7c0c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf73e00640 1 -- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf7c0c8200 msgr2=0x7faf7c1f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf73e00640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf7c0c8200 0x7faf7c1f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf73e00640 1 -- 172.21.15.113:0/3242935571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c06baf0 con 0x7faf7c1527f0 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf73e00640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7c1527f0 0x7faf7c149530 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7faf640097f0 tx=0x7faf64019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf72a00640 1 -- 172.21.15.113:0/3242935571 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf640c1070 con 0x7faf7c1527f0 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf72a00640 1 -- 172.21.15.113:0/3242935571 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf64003820 con 0x7faf7c1527f0 2024-03-04T07:54:56.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf81400640 1 -- 172.21.15.113:0/3242935571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c1f38c0 con 0x7faf7c1527f0 2024-03-04T07:54:56.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.671+0000 7faf72a00640 1 -- 172.21.15.113:0/3242935571 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64003bd0 con 0x7faf7c1527f0 2024-03-04T07:54:56.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.675+0000 7faf81400640 1 -- 172.21.15.113:0/3242935571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c1f78d0 con 0x7faf7c1527f0 2024-03-04T07:54:56.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.675+0000 7faf81400640 1 -- 172.21.15.113:0/3242935571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7c075c00 con 0x7faf7c1527f0 2024-03-04T07:54:56.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.675+0000 7faf72a00640 1 -- 172.21.15.113:0/3242935571 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf640c4070 con 0x7faf7c1527f0 2024-03-04T07:54:56.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.675+0000 7faf72a00640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faf58066f60 0x7faf58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:56.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.675+0000 7faf72a00640 1 -- 172.21.15.113:0/3242935571 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7faf6411f200 con 0x7faf7c1527f0 2024-03-04T07:54:56.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.675+0000 7faf7a800640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faf58066f60 0x7faf58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:56.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.679+0000 7faf7a800640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faf58066f60 0x7faf58069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0011f0 tx=0x7faf6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:56.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.683+0000 7faf72a00640 1 -- 172.21.15.113:0/3242935571 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7faf640160d0 con 0x7faf7c1527f0 2024-03-04T07:54:56.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.871+0000 7faf81400640 1 -- 172.21.15.113:0/3242935571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf7c0c79d0 con 0x7faf7c1527f0 2024-03-04T07:54:56.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.871+0000 7faf72a00640 1 -- 172.21.15.113:0/3242935571 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5813 (secure 0 0 0) 0x7faf640ef780 con 0x7faf7c1527f0 2024-03-04T07:54:56.869 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:56.869 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:54:56.079247+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:rejoin","state_seq":39,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940]},"id":1}]} 2024-03-04T07:54:56.870 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 80 2024-03-04T07:54:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 -- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faf58066f60 msgr2=0x7faf58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faf58066f60 0x7faf58069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0011f0 tx=0x7faf6c009040 comp rx=0 tx=0).stop 2024-03-04T07:54:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 -- 172.21.15.113:0/3242935571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7c1527f0 msgr2=0x7faf7c149530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7c1527f0 0x7faf7c149530 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7faf640097f0 tx=0x7faf64019280 comp rx=0 tx=0).stop 2024-03-04T07:54:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 -- 172.21.15.113:0/3242935571 shutdown_connections 2024-03-04T07:54:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7faf7c152bd0 0x7faf7c0c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faf58066f60 0x7faf58069420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faf7c1527f0 0x7faf7c149530 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 --2- 172.21.15.113:0/3242935571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf7c0c8200 0x7faf7c1f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:56.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 -- 172.21.15.113:0/3242935571 >> 172.21.15.113:0/3242935571 conn(0x7faf7c06e820 msgr2=0x7faf7c0cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:56.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 -- 172.21.15.113:0/3242935571 shutdown_connections 2024-03-04T07:54:56.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:56.875+0000 7faf81400640 1 -- 172.21.15.113:0/3242935571 wait complete. 2024-03-04T07:54:56.890 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:54:58.891 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:54:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e9400640 1 Processor -- start 2024-03-04T07:54:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e9400640 1 -- start start 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e9400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10e4152b20 0x7f10e4070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e9400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f10e40c8b00 0x7f10e415be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e9400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e4152f00 con 0x7f10e415c340 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e9400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e40c8f90 con 0x7f10e4152b20 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e9400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e40c5320 con 0x7f10e40c8b00 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10e4152b20 0x7f10e4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f10e4152b20 0x7f10e4070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50960/0 (socket says 172.21.15.113:50960) 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e2800640 1 -- 172.21.15.113:0/2125458461 learned_addr learned my addr 172.21.15.113:0/2125458461 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10dbe00640 1 --2- 172.21.15.113:0/2125458461 >> v2:172.21.15.136:3300/0 conn(0x7f10e40c8b00 0x7f10e415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e2800640 1 -- 172.21.15.113:0/2125458461 >> v2:172.21.15.136:3300/0 conn(0x7f10e40c8b00 msgr2=0x7f10e415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e2800640 1 --2- 172.21.15.113:0/2125458461 >> v2:172.21.15.136:3300/0 conn(0x7f10e40c8b00 0x7f10e415be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e2800640 1 -- 172.21.15.113:0/2125458461 >> v1:172.21.15.136:6789/0 conn(0x7f10e415c340 legacy=0x7f10e40c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:54:59.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.031+0000 7f10e2800640 1 -- 172.21.15.113:0/2125458461 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40c5040 con 0x7f10e4152b20 2024-03-04T07:54:59.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e2800640 1 --2- 172.21.15.113:0/2125458461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 0x7f10e4070010 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f10e4050420 tx=0x7f10d401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec3bff94fd3f4e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:59.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e1e00640 1 -- 172.21.15.113:0/2125458461 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40b1070 con 0x7f10e4152b20 2024-03-04T07:54:59.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e1e00640 1 -- 172.21.15.113:0/2125458461 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d40ac630 con 0x7f10e4152b20 2024-03-04T07:54:59.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e1e00640 1 -- 172.21.15.113:0/2125458461 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40a8050 con 0x7f10e4152b20 2024-03-04T07:54:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e9400640 1 -- 172.21.15.113:0/2125458461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 msgr2=0x7f10e4070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e9400640 1 --2- 172.21.15.113:0/2125458461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 0x7f10e4070010 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f10e4050420 tx=0x7f10d401c4e0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e9400640 1 -- 172.21.15.113:0/2125458461 shutdown_connections 2024-03-04T07:54:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e9400640 1 --2- 172.21.15.113:0/2125458461 >> v2:172.21.15.136:3300/0 conn(0x7f10e40c8b00 0x7f10e415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e9400640 1 --2- 172.21.15.113:0/2125458461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 0x7f10e4070010 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e9400640 1 -- 172.21.15.113:0/2125458461 >> 172.21.15.113:0/2125458461 conn(0x7f10e40709a0 msgr2=0x7f10e4070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e9400640 1 -- 172.21.15.113:0/2125458461 shutdown_connections 2024-03-04T07:54:59.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.035+0000 7f10e9400640 1 -- 172.21.15.113:0/2125458461 wait complete. 2024-03-04T07:54:59.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e9400640 1 Processor -- start 2024-03-04T07:54:59.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e9400640 1 -- start start 2024-03-04T07:54:59.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10e40c8b00 0x7f10e41f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:59.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 0x7f10e40cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:59.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e9400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10e415c340 0x7f10e41f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:59.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e9400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40c3810 con 0x7f10e4152b20 2024-03-04T07:54:59.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e9400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e415cb50 con 0x7f10e40c8b00 2024-03-04T07:54:59.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e9400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e4158810 con 0x7f10e415c340 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 0x7f10e40cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10e40c8b00 0x7f10e41f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 0x7f10e40cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50976/0 (socket says 172.21.15.113:50976) 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10dbe00640 1 -- 172.21.15.113:0/4228928955 learned_addr learned my addr 172.21.15.113:0/4228928955 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e3200640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10e415c340 0x7f10e41f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10dbe00640 1 -- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10e415c340 msgr2=0x7f10e41f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10dbe00640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10e415c340 0x7f10e41f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10dbe00640 1 -- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10e40c8b00 msgr2=0x7f10e41f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10dbe00640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10e40c8b00 0x7f10e41f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10dbe00640 1 -- 172.21.15.113:0/4228928955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40cd720 con 0x7f10e4152b20 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e2800640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10e40c8b00 0x7f10e41f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:54:59.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10dbe00640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 0x7f10e40cd130 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00d620 tx=0x7f10cc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:59.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10daa00640 1 -- 172.21.15.113:0/4228928955 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc00b620 con 0x7f10e4152b20 2024-03-04T07:54:59.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10daa00640 1 -- 172.21.15.113:0/4228928955 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc019070 con 0x7f10e4152b20 2024-03-04T07:54:59.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e9400640 1 -- 172.21.15.113:0/4228928955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e40cda00 con 0x7f10e4152b20 2024-03-04T07:54:59.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10daa00640 1 -- 172.21.15.113:0/4228928955 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc00b9e0 con 0x7f10e4152b20 2024-03-04T07:54:59.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.039+0000 7f10e9400640 1 -- 172.21.15.113:0/4228928955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e41f6ca0 con 0x7f10e4152b20 2024-03-04T07:54:59.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.043+0000 7f10e9400640 1 -- 172.21.15.113:0/4228928955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e4075c00 con 0x7f10e4152b20 2024-03-04T07:54:59.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.043+0000 7f10daa00640 1 -- 172.21.15.113:0/4228928955 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10cc00bb80 con 0x7f10e4152b20 2024-03-04T07:54:59.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.043+0000 7f10daa00640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10c0066f60 0x7f10c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:54:59.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.043+0000 7f10daa00640 1 -- 172.21.15.113:0/4228928955 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f10cc085850 con 0x7f10e4152b20 2024-03-04T07:54:59.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.043+0000 7f10e2800640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10c0066f60 0x7f10c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:54:59.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.043+0000 7f10e2800640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10c0066f60 0x7f10c0069420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f10d40ae000 tx=0x7f10d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:54:59.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.047+0000 7f10daa00640 1 -- 172.21.15.113:0/4228928955 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f10cc055cb0 con 0x7f10e4152b20 2024-03-04T07:54:59.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.235+0000 7f10e9400640 1 -- 172.21.15.113:0/4228928955 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10e4159310 con 0x7f10e4152b20 2024-03-04T07:54:59.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.235+0000 7f10daa00640 1 -- 172.21.15.113:0/4228928955 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5813 (secure 0 0 0) 0x7f10cc059d50 con 0x7f10e4152b20 2024-03-04T07:54:59.234 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:54:59.234 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:54:56.079247+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:rejoin","state_seq":39,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6145,5940]},"id":1}]} 2024-03-04T07:54:59.234 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 80 2024-03-04T07:54:59.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 -- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10c0066f60 msgr2=0x7f10c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:59.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10c0066f60 0x7f10c0069420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f10d40ae000 tx=0x7f10d4008040 comp rx=0 tx=0).stop 2024-03-04T07:54:59.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 -- 172.21.15.113:0/4228928955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 msgr2=0x7f10e40cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:54:59.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 0x7f10e40cd130 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00d620 tx=0x7f10cc00dae0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 -- 172.21.15.113:0/4228928955 shutdown_connections 2024-03-04T07:54:59.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10e415c340 0x7f10e41f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10c0066f60 0x7f10c0069420 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10e4152b20 0x7f10e40cd130 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 --2- 172.21.15.113:0/4228928955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10e40c8b00 0x7f10e41f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:54:59.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 -- 172.21.15.113:0/4228928955 >> 172.21.15.113:0/4228928955 conn(0x7f10e40709a0 msgr2=0x7f10e40c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:54:59.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 -- 172.21.15.113:0/4228928955 shutdown_connections 2024-03-04T07:54:59.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:54:59.239+0000 7f10e9400640 1 -- 172.21.15.113:0/4228928955 wait complete. 2024-03-04T07:54:59.255 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:55:01.099 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 20 -eq 48 ] 2024-03-04T07:55:01.100 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.20 2024-03-04T07:55:01.100 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.20 2024-03-04T07:55:01.100 INFO:tasks.workunit.client.0.smithi113.stdout:v0.20 2024-03-04T07:55:01.100 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.20 2024-03-04T07:55:01.129 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:55:01.132 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:55:01.256 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:55:01.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.395+0000 7f3ea7a00640 1 Processor -- start 2024-03-04T07:55:01.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.395+0000 7f3ea7a00640 1 -- start start 2024-03-04T07:55:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.395+0000 7f3ea7a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ea00ce2d0 0x7f3ea015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.395+0000 7f3ea7a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3ea00ce6b0 0x7f3ea006f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea7a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea006c210 con 0x7f3ea006de80 2024-03-04T07:55:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea7a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea015bbf0 con 0x7f3ea00ce2d0 2024-03-04T07:55:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea7a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea006eb80 con 0x7f3ea00ce6b0 2024-03-04T07:55:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea5000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ea00ce2d0 0x7f3ea015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:01.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3e9fe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3ea00ce6b0 0x7f3ea006f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:01.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea5000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3ea00ce2d0 0x7f3ea015b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50990/0 (socket says 172.21.15.113:50990) 2024-03-04T07:55:01.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea5000640 1 -- 172.21.15.113:0/2325580837 learned_addr learned my addr 172.21.15.113:0/2325580837 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:55:01.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea5000640 1 -- 172.21.15.113:0/2325580837 >> v2:172.21.15.136:3300/0 conn(0x7f3ea00ce6b0 msgr2=0x7f3ea006f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:55:01.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea5000640 1 --2- 172.21.15.113:0/2325580837 >> v2:172.21.15.136:3300/0 conn(0x7f3ea00ce6b0 0x7f3ea006f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:01.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea5000640 1 -- 172.21.15.113:0/2325580837 >> v1:172.21.15.136:6789/0 conn(0x7f3ea006de80 legacy=0x7f3ea0154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:01.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea5000640 1 -- 172.21.15.113:0/2325580837 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea006c4f0 con 0x7f3ea00ce2d0 2024-03-04T07:55:01.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3ea5000640 1 --2- 172.21.15.113:0/2325580837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea00ce2d0 0x7f3ea015b6b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0158860 tx=0x7f3e9401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb3cc37cd7c1c78 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:55:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.399+0000 7f3e9f400640 1 -- 172.21.15.113:0/2325580837 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940b5070 con 0x7f3ea00ce2d0 2024-03-04T07:55:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3e9f400640 1 -- 172.21.15.113:0/2325580837 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e940a8a10 con 0x7f3ea00ce2d0 2024-03-04T07:55:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3e9f400640 1 -- 172.21.15.113:0/2325580837 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940a8de0 con 0x7f3ea00ce2d0 2024-03-04T07:55:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3ea7a00640 1 -- 172.21.15.113:0/2325580837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea00ce2d0 msgr2=0x7f3ea015b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3ea7a00640 1 --2- 172.21.15.113:0/2325580837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea00ce2d0 0x7f3ea015b6b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0158860 tx=0x7f3e9401ed10 comp rx=0 tx=0).stop 2024-03-04T07:55:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3ea7a00640 1 -- 172.21.15.113:0/2325580837 shutdown_connections 2024-03-04T07:55:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3ea7a00640 1 --2- 172.21.15.113:0/2325580837 >> v2:172.21.15.136:3300/0 conn(0x7f3ea00ce6b0 0x7f3ea006f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3ea7a00640 1 --2- 172.21.15.113:0/2325580837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea00ce2d0 0x7f3ea015b6b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3ea7a00640 1 -- 172.21.15.113:0/2325580837 >> 172.21.15.113:0/2325580837 conn(0x7f3ea00709a0 msgr2=0x7f3ea0070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:55:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3ea7a00640 1 -- 172.21.15.113:0/2325580837 shutdown_connections 2024-03-04T07:55:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3ea7a00640 1 -- 172.21.15.113:0/2325580837 wait complete. 2024-03-04T07:55:01.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3ea7a00640 1 Processor -- start 2024-03-04T07:55:01.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.403+0000 7f3ea7a00640 1 -- start start 2024-03-04T07:55:01.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea7a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea006de80 0x7f3ea01f47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:01.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea7a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea00ce2d0 0x7f3ea0143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:01.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea7a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea00ce6b0 0x7f3ea01f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:01.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea7a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea006c7d0 con 0x7f3ea006de80 2024-03-04T07:55:01.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea7a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea006e810 con 0x7f3ea00ce2d0 2024-03-04T07:55:01.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea7a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea0155220 con 0x7f3ea00ce6b0 2024-03-04T07:55:01.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea006de80 0x7f3ea01f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:01.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea006de80 0x7f3ea01f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50998/0 (socket says 172.21.15.113:50998) 2024-03-04T07:55:01.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea5000640 1 -- 172.21.15.113:0/1966368431 learned_addr learned my addr 172.21.15.113:0/1966368431 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:55:01.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3e9fe00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea00ce2d0 0x7f3ea0143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea5a00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea00ce6b0 0x7f3ea01f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea5000640 1 -- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea00ce6b0 msgr2=0x7f3ea01f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea5000640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea00ce6b0 0x7f3ea01f0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea5000640 1 -- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea00ce2d0 msgr2=0x7f3ea0143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea5000640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea00ce2d0 0x7f3ea0143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea5000640 1 -- 172.21.15.113:0/1966368431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea0156910 con 0x7f3ea006de80 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea5000640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea006de80 0x7f3ea01f47d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3e9401b920 tx=0x7f3e9401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3e9cc00640 1 -- 172.21.15.113:0/1966368431 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940b5070 con 0x7f3ea006de80 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3e9cc00640 1 -- 172.21.15.113:0/1966368431 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e940b0a50 con 0x7f3ea006de80 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea7a00640 1 -- 172.21.15.113:0/1966368431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea01531e0 con 0x7f3ea006de80 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3e9cc00640 1 -- 172.21.15.113:0/1966368431 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940b0e40 con 0x7f3ea006de80 2024-03-04T07:55:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea7a00640 1 -- 172.21.15.113:0/1966368431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea0153760 con 0x7f3ea006de80 2024-03-04T07:55:01.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.407+0000 7f3ea7a00640 1 -- 172.21.15.113:0/1966368431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea0075c00 con 0x7f3ea006de80 2024-03-04T07:55:01.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.411+0000 7f3e9cc00640 1 -- 172.21.15.113:0/1966368431 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e940c2060 con 0x7f3ea006de80 2024-03-04T07:55:01.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.411+0000 7f3e9cc00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e80066f60 0x7f3e80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:01.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.411+0000 7f3e9cc00640 1 -- 172.21.15.113:0/1966368431 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f3e941224c0 con 0x7f3ea006de80 2024-03-04T07:55:01.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.411+0000 7f3e9fe00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e80066f60 0x7f3e80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:01.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.411+0000 7f3e9fe00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e80066f60 0x7f3e80069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3ea015b830 tx=0x7f3e8c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:55:01.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.415+0000 7f3e9cc00640 1 -- 172.21.15.113:0/1966368431 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3e94018870 con 0x7f3ea006de80 2024-03-04T07:55:01.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 -- 172.21.15.113:0/1966368431 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ea006cb40 con 0x7f3ea006de80 2024-03-04T07:55:01.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3e9cc00640 1 -- 172.21.15.113:0/1966368431 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f3e940f30a0 con 0x7f3ea006de80 2024-03-04T07:55:01.603 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:55:01.603 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:55:01.109616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,6145,6240]},"id":1}]} 2024-03-04T07:55:01.603 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 81 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 -- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e80066f60 msgr2=0x7f3e80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e80066f60 0x7f3e80069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3ea015b830 tx=0x7f3e8c009190 comp rx=0 tx=0).stop 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 -- 172.21.15.113:0/1966368431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea006de80 msgr2=0x7f3ea01f47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea006de80 0x7f3ea01f47d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3e9401b920 tx=0x7f3e9401bab0 comp rx=0 tx=0).stop 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 -- 172.21.15.113:0/1966368431 shutdown_connections 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3ea00ce6b0 0x7f3ea01f0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ea00ce2d0 0x7f3ea0143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e80066f60 0x7f3e80069420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 --2- 172.21.15.113:0/1966368431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ea006de80 0x7f3ea01f47d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 -- 172.21.15.113:0/1966368431 >> 172.21.15.113:0/1966368431 conn(0x7f3ea00709a0 msgr2=0x7f3ea015a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 -- 172.21.15.113:0/1966368431 shutdown_connections 2024-03-04T07:55:01.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:01.607+0000 7f3ea7a00640 1 -- 172.21.15.113:0/1966368431 wait complete. 2024-03-04T07:55:01.623 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=2, replacing gid=5631 2024-03-04T07:55:01.624 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.c 2024-03-04T07:55:05.592 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 48% (120/245) Updating files: 49% (121/245) Updating files: 50% (123/245) Updating files: 51% (125/245) Updating files: 52% (128/245) Updating files: 53% (130/245) Updating files: 54% (133/245) Updating files: 55% (135/245) Updating files: 56% (138/245) Updating files: 57% (140/245) Updating files: 58% (143/245) Updating files: 59% (145/245) Updating files: 60% (147/245) Updating files: 61% (150/245) Updating files: 62% (152/245) Updating files: 63% (155/245) Updating files: 64% (157/245) Updating files: 65% (160/245) Updating files: 66% (162/245) Updating files: 67% (165/245) Updating files: 68% (167/245) Updating files: 69% (170/245) Updating files: 70% (172/245) Updating files: 71% (174/245) Updating files: 72% (177/245) Updating files: 73% (179/245) Updating files: 74% (182/245) Updating files: 75% (184/245) Updating files: 76% (187/245) Updating files: 77% (189/245) Updating files: 78% (192/245) Updating files: 79% (194/245) Updating files: 80% (196/245) Updating files: 81% (199/245) Updating files: 82% (201/245) Updating files: 83% (204/245) Updating files: 84% (206/245) Updating files: 85% (209/245) Updating files: 86% (211/245) Updating files: 87% (214/245) Updating files: 88% (216/245) Updating files: 89% (219/245) Updating files: 90% (221/245) Updating files: 91% (223/245) Updating files: 92% (226/245) Updating files: 93% (228/245) Updating files: 94% (231/245) Updating files: 95% (233/245) Updating files: 96% (236/245) Updating files: 97% (238/245) Updating files: 98% (241/245) Updating files: 99% (243/245) Updating files: 100% (245/245) Updating files: 100% (245/245), done. 2024-03-04T07:55:05.620 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 266d2076b6c debian: fix up debian scripts 2024-03-04T07:55:05.623 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.20 2024-03-04T07:55:06.728 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 21 -eq 48 ] 2024-03-04T07:55:06.728 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.21 2024-03-04T07:55:06.729 INFO:tasks.workunit.client.0.smithi113.stdout:v0.21 2024-03-04T07:55:06.729 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.21 2024-03-04T07:55:06.729 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.21 2024-03-04T07:55:08.624 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-03-04T07:55:08.624 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-04T07:55:08.624 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-04T07:55:08.628 INFO:tasks.ceph.mds.c:Started 2024-03-04T07:55:08.628 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:55:08.714 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-03-04T07:55:08.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b80600640 1 Processor -- start 2024-03-04T07:55:08.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b80600640 1 -- start start 2024-03-04T07:55:08.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b80600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b7815b6a0 0x7f5b7806ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:08.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b80600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b780b0e30 0x7f5b780ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:08.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b80600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7806c3c0 con 0x7f5b780ce2d0 2024-03-04T07:55:08.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b80600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7806c170 con 0x7f5b780b0e30 2024-03-04T07:55:08.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b80600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780c9460 con 0x7f5b7815b6a0 2024-03-04T07:55:08.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b7e600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b780b0e30 0x7f5b780ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:08.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b7dc00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5b780ce2d0 0x7f5b7815af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55434/0 (socket says 172.21.15.113:55434) 2024-03-04T07:55:08.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b7dc00640 1 -- 172.21.15.113:0/3622193527 learned_addr learned my addr 172.21.15.113:0/3622193527 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:55:08.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b7d200640 1 --2- 172.21.15.113:0/3622193527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b7815b6a0 0x7f5b7806ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:08.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b7e600640 1 -- 172.21.15.113:0/3622193527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b7815b6a0 msgr2=0x7f5b7806ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:55:08.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b7e600640 1 --2- 172.21.15.113:0/3622193527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b7815b6a0 0x7f5b7806ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:08.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b7e600640 1 -- 172.21.15.113:0/3622193527 >> v1:172.21.15.113:6789/0 conn(0x7f5b780ce2d0 legacy=0x7f5b7815af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:08.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.775+0000 7f5b7e600640 1 -- 172.21.15.113:0/3622193527 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7806c6a0 con 0x7f5b780b0e30 2024-03-04T07:55:08.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.779+0000 7f5b7e600640 1 --2- 172.21.15.113:0/3622193527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b780b0e30 0x7f5b780ce6b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009d90 tx=0x7f5b7001f170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8436d96ec920b89a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:55:08.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.779+0000 7f5b77e00640 1 -- 172.21.15.113:0/3622193527 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b700b4070 con 0x7f5b780b0e30 2024-03-04T07:55:08.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.779+0000 7f5b77e00640 1 -- 172.21.15.113:0/3622193527 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b700af600 con 0x7f5b780b0e30 2024-03-04T07:55:08.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.779+0000 7f5b77e00640 1 -- 172.21.15.113:0/3622193527 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b700af9d0 con 0x7f5b780b0e30 2024-03-04T07:55:08.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 -- 172.21.15.113:0/3622193527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b780b0e30 msgr2=0x7f5b780ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:08.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 --2- 172.21.15.113:0/3622193527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b780b0e30 0x7f5b780ce6b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009d90 tx=0x7f5b7001f170 comp rx=0 tx=0).stop 2024-03-04T07:55:08.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 -- 172.21.15.113:0/3622193527 shutdown_connections 2024-03-04T07:55:08.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 --2- 172.21.15.113:0/3622193527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b780b0e30 0x7f5b780ce6b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:08.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 --2- 172.21.15.113:0/3622193527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b7815b6a0 0x7f5b7806ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:08.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 -- 172.21.15.113:0/3622193527 >> 172.21.15.113:0/3622193527 conn(0x7f5b78066db0 msgr2=0x7f5b780671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:55:08.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 -- 172.21.15.113:0/3622193527 shutdown_connections 2024-03-04T07:55:08.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 -- 172.21.15.113:0/3622193527 wait complete. 2024-03-04T07:55:08.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 Processor -- start 2024-03-04T07:55:08.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 -- start start 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b780b0e30 0x7f5b7806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b780ce2d0 0x7f5b781f0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b7815b6a0 0x7f5b78151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b781585a0 con 0x7f5b7815b6a0 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780c9990 con 0x7f5b780ce2d0 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b80600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b78070230 con 0x7f5b780b0e30 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b7815b6a0 0x7f5b78151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b7815b6a0 0x7f5b78151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37524/0 (socket says 172.21.15.113:37524) 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7e600640 1 -- 172.21.15.113:0/1047900840 learned_addr learned my addr 172.21.15.113:0/1047900840 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7dc00640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b780b0e30 0x7f5b7806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7d200640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b780ce2d0 0x7f5b781f0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7e600640 1 -- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b780b0e30 msgr2=0x7f5b7806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:08.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7e600640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b780b0e30 0x7f5b7806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:08.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7e600640 1 -- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b780ce2d0 msgr2=0x7f5b781f0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:08.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7e600640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b780ce2d0 0x7f5b781f0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:08.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7e600640 1 -- 172.21.15.113:0/1047900840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b781f4040 con 0x7f5b7815b6a0 2024-03-04T07:55:08.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.783+0000 7f5b7d200640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b780ce2d0 0x7f5b781f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:55:08.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.787+0000 7f5b7e600640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b7815b6a0 0x7f5b78151d20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5b700b2040 tx=0x7f5b7001bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:55:08.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.787+0000 7f5b75600640 1 -- 172.21.15.113:0/1047900840 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b700bf020 con 0x7f5b7815b6a0 2024-03-04T07:55:08.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.787+0000 7f5b75600640 1 -- 172.21.15.113:0/1047900840 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b700afe30 con 0x7f5b7815b6a0 2024-03-04T07:55:08.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.787+0000 7f5b75600640 1 -- 172.21.15.113:0/1047900840 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b700b4040 con 0x7f5b7815b6a0 2024-03-04T07:55:08.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.787+0000 7f5b80600640 1 -- 172.21.15.113:0/1047900840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7815a000 con 0x7f5b7815b6a0 2024-03-04T07:55:08.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.787+0000 7f5b80600640 1 -- 172.21.15.113:0/1047900840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b781f42e0 con 0x7f5b7815b6a0 2024-03-04T07:55:08.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.787+0000 7f5b80600640 1 -- 172.21.15.113:0/1047900840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b78075c00 con 0x7f5b7815b6a0 2024-03-04T07:55:08.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.787+0000 7f5b75600640 1 -- 172.21.15.113:0/1047900840 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b700c2020 con 0x7f5b7815b6a0 2024-03-04T07:55:08.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.787+0000 7f5b75600640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b58066f90 0x7f5b58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:08.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.787+0000 7f5b75600640 1 -- 172.21.15.113:0/1047900840 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f5b70123a80 con 0x7f5b7815b6a0 2024-03-04T07:55:08.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.791+0000 7f5b7dc00640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b58066f90 0x7f5b58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:08.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.791+0000 7f5b7dc00640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b58066f90 0x7f5b58069450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c000f70 tx=0x7f5b6c007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:55:08.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.791+0000 7f5b75600640 1 -- 172.21.15.113:0/1047900840 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5b70125e20 con 0x7f5b7815b6a0 2024-03-04T07:55:08.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 -- 172.21.15.113:0/1047900840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b780b0a00 con 0x7f5b7815b6a0 2024-03-04T07:55:08.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b75600640 1 -- 172.21.15.113:0/1047900840 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f5b700c9030 con 0x7f5b7815b6a0 2024-03-04T07:55:08.983 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:55:08.983 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:55:01.109616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[5940,6145,6240]},"id":1}]} 2024-03-04T07:55:08.984 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 81 2024-03-04T07:55:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 -- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b58066f90 msgr2=0x7f5b58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b58066f90 0x7f5b58069450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c000f70 tx=0x7f5b6c007e90 comp rx=0 tx=0).stop 2024-03-04T07:55:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 -- 172.21.15.113:0/1047900840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b7815b6a0 msgr2=0x7f5b78151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b7815b6a0 0x7f5b78151d20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5b700b2040 tx=0x7f5b7001bf10 comp rx=0 tx=0).stop 2024-03-04T07:55:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 -- 172.21.15.113:0/1047900840 shutdown_connections 2024-03-04T07:55:08.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b58066f90 0x7f5b58069450 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:08.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b7815b6a0 0x7f5b78151d20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:08.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b780ce2d0 0x7f5b781f0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:08.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 --2- 172.21.15.113:0/1047900840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b780b0e30 0x7f5b7806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:08.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 -- 172.21.15.113:0/1047900840 >> 172.21.15.113:0/1047900840 conn(0x7f5b78066db0 msgr2=0x7f5b78070990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:55:08.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 -- 172.21.15.113:0/1047900840 shutdown_connections 2024-03-04T07:55:08.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:08.987+0000 7f5b80600640 1 -- 172.21.15.113:0/1047900840 wait complete. 2024-03-04T07:55:09.004 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-03-04T07:55:11.005 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:55:11.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4e2400640 1 Processor -- start 2024-03-04T07:55:11.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4e2400640 1 -- start start 2024-03-04T07:55:11.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4e2400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4dc0c22e0 0x7fe4dc15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:11.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4e2400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe4dc0c2d80 0x7fe4dc15b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:11.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4e2400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4dc0c27f0 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4e2400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4dc15ad60 con 0x7fe4dc0c22e0 2024-03-04T07:55:11.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4e2400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4dc0c2a90 con 0x7fe4dc0c2d80 2024-03-04T07:55:11.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4dc0c22e0 0x7fe4dc15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:11.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4dc0c22e0 0x7fe4dc15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37530/0 (socket says 172.21.15.113:37530) 2024-03-04T07:55:11.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4daa00640 1 -- 172.21.15.113:0/3144134436 learned_addr learned my addr 172.21.15.113:0/3144134436 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:55:11.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4dbe00640 1 --2- 172.21.15.113:0/3144134436 >> v2:172.21.15.136:3300/0 conn(0x7fe4dc0c2d80 0x7fe4dc15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:11.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4daa00640 1 -- 172.21.15.113:0/3144134436 >> v2:172.21.15.136:3300/0 conn(0x7fe4dc0c2d80 msgr2=0x7fe4dc15b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:55:11.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4daa00640 1 --2- 172.21.15.113:0/3144134436 >> v2:172.21.15.136:3300/0 conn(0x7fe4dc0c2d80 0x7fe4dc15b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:11.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4daa00640 1 -- 172.21.15.113:0/3144134436 >> v1:172.21.15.136:6789/0 conn(0x7fe4dc14e4c0 legacy=0x7fe4dc072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:11.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.178+0000 7fe4daa00640 1 -- 172.21.15.113:0/3144134436 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc070c60 con 0x7fe4dc0c22e0 2024-03-04T07:55:11.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4daa00640 1 --2- 172.21.15.113:0/3144134436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc0c22e0 0x7fe4dc15b1f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0c2700 tx=0x7fe4c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823b0ef0ddb9b425 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:55:11.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4da000640 1 -- 172.21.15.113:0/3144134436 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80ab430 con 0x7fe4dc0c22e0 2024-03-04T07:55:11.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4da000640 1 -- 172.21.15.113:0/3144134436 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c80b0070 con 0x7fe4dc0c22e0 2024-03-04T07:55:11.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4da000640 1 -- 172.21.15.113:0/3144134436 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80ab7f0 con 0x7fe4dc0c22e0 2024-03-04T07:55:11.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4e2400640 1 -- 172.21.15.113:0/3144134436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc0c22e0 msgr2=0x7fe4dc15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:11.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4e2400640 1 --2- 172.21.15.113:0/3144134436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc0c22e0 0x7fe4dc15b1f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0c2700 tx=0x7fe4c801c520 comp rx=0 tx=0).stop 2024-03-04T07:55:11.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4e2400640 1 -- 172.21.15.113:0/3144134436 shutdown_connections 2024-03-04T07:55:11.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4e2400640 1 --2- 172.21.15.113:0/3144134436 >> v2:172.21.15.136:3300/0 conn(0x7fe4dc0c2d80 0x7fe4dc15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:11.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4e2400640 1 --2- 172.21.15.113:0/3144134436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc0c22e0 0x7fe4dc15b1f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:11.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4e2400640 1 -- 172.21.15.113:0/3144134436 >> 172.21.15.113:0/3144134436 conn(0x7fe4dc06e820 msgr2=0x7fe4dc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:55:11.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4e2400640 1 -- 172.21.15.113:0/3144134436 shutdown_connections 2024-03-04T07:55:11.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4e2400640 1 -- 172.21.15.113:0/3144134436 wait complete. 2024-03-04T07:55:11.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4e2400640 1 Processor -- start 2024-03-04T07:55:11.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.186+0000 7fe4e2400640 1 -- start start 2024-03-04T07:55:11.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4dc0c22e0 0x7fe4dc1f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:11.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4dc0c2d80 0x7fe4dc13eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:11.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4e2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc14e4c0 0x7fe4dc1f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:11.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4e2400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc0aede0 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4e2400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc14e8a0 con 0x7fe4dc0c22e0 2024-03-04T07:55:11.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4e2400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc073390 con 0x7fe4dc0c2d80 2024-03-04T07:55:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc14e4c0 0x7fe4dc1f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc14e4c0 0x7fe4dc1f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37532/0 (socket says 172.21.15.113:37532) 2024-03-04T07:55:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4dbe00640 1 -- 172.21.15.113:0/4208648275 learned_addr learned my addr 172.21.15.113:0/4208648275 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:55:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4db400640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4dc0c22e0 0x7fe4dc1f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4daa00640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4dc0c2d80 0x7fe4dc13eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4dbe00640 1 -- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4dc0c2d80 msgr2=0x7fe4dc13eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4dbe00640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4dc0c2d80 0x7fe4dc13eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4dbe00640 1 -- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4dc0c22e0 msgr2=0x7fe4dc1f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4dbe00640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4dc0c22e0 0x7fe4dc1f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:11.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4dbe00640 1 -- 172.21.15.113:0/4208648275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc1604e0 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4dbe00640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc14e4c0 0x7fe4dc1f4010 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009d30 tx=0x7fe4d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:55:11.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4d3400640 1 -- 172.21.15.113:0/4208648275 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4018020 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4d3400640 1 -- 172.21.15.113:0/4208648275 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d4014480 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4e2400640 1 -- 172.21.15.113:0/4208648275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc160760 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4e2400640 1 -- 172.21.15.113:0/4208648275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc1f3830 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.190+0000 7fe4d3400640 1 -- 172.21.15.113:0/4208648275 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4014870 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.194+0000 7fe4e2400640 1 -- 172.21.15.113:0/4208648275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b40052a0 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.194+0000 7fe4d3400640 1 -- 172.21.15.113:0/4208648275 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4d401b020 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.198+0000 7fe4d3400640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4bc066f60 0x7fe4bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:55:11.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.198+0000 7fe4d3400640 1 -- 172.21.15.113:0/4208648275 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7fe4d4086530 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.198+0000 7fe4db400640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4bc066f60 0x7fe4bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:55:11.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.198+0000 7fe4db400640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4bc066f60 0x7fe4bc069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc001810 tx=0x7fe4cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:55:11.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.202+0000 7fe4d3400640 1 -- 172.21.15.113:0/4208648275 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe4d4022070 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.558+0000 7fe4e2400640 1 -- 172.21.15.113:0/4208648275 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4b4005590 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4d3400640 1 -- 172.21.15.113:0/4208648275 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fe4dc1604e0 con 0x7fe4dc14e4c0 2024-03-04T07:55:11.558 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:55:11.558 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:55:01.109616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6240,6145,5940]},"id":1}]} 2024-03-04T07:55:11.559 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 82 2024-03-04T07:55:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 -- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4bc066f60 msgr2=0x7fe4bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:11.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4bc066f60 0x7fe4bc069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc001810 tx=0x7fe4cc009040 comp rx=0 tx=0).stop 2024-03-04T07:55:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 -- 172.21.15.113:0/4208648275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc14e4c0 msgr2=0x7fe4dc1f4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:55:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc14e4c0 0x7fe4dc1f4010 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009d30 tx=0x7fe4d400c490 comp rx=0 tx=0).stop 2024-03-04T07:55:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 -- 172.21.15.113:0/4208648275 shutdown_connections 2024-03-04T07:55:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4bc066f60 0x7fe4bc069420 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4dc14e4c0 0x7fe4dc1f4010 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4dc0c2d80 0x7fe4dc13eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 --2- 172.21.15.113:0/4208648275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4dc0c22e0 0x7fe4dc1f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:55:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 -- 172.21.15.113:0/4208648275 >> 172.21.15.113:0/4208648275 conn(0x7fe4dc06e820 msgr2=0x7fe4dc0c8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:55:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 -- 172.21.15.113:0/4208648275 shutdown_connections 2024-03-04T07:55:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:55:11.562+0000 7fe4e2400640 1 -- 172.21.15.113:0/4208648275 wait complete. 2024-03-04T07:55:11.583 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-03-04T07:55:11.583 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 102 secs before thrashing 2024-03-04T07:55:14.581 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 31% (170/537) Updating files: 32% (172/537) Updating files: 33% (178/537) Updating files: 34% (183/537) Updating files: 35% (188/537) Updating files: 36% (194/537) Updating files: 37% (199/537) Updating files: 38% (205/537) Updating files: 39% (210/537) Updating files: 40% (215/537) Updating files: 41% (221/537) Updating files: 42% (226/537) Updating files: 43% (231/537) Updating files: 44% (237/537) Updating files: 45% (242/537) Updating files: 46% (248/537) Updating files: 47% (253/537) Updating files: 48% (258/537) Updating files: 49% (264/537) Updating files: 49% (266/537) Updating files: 49% (267/537) Updating files: 50% (269/537) Updating files: 51% (274/537) Updating files: 52% (280/537) Updating files: 53% (285/537) Updating files: 54% (290/537) Updating files: 55% (296/537) Updating files: 56% (301/537) Updating files: 57% (307/537) Updating files: 58% (312/537) Updating files: 59% (317/537) Updating files: 60% (323/537) Updating files: 61% (328/537) Updating files: 62% (333/537) Updating files: 63% (339/537) Updating files: 64% (344/537) Updating files: 65% (350/537) Updating files: 66% (355/537) Updating files: 67% (360/537) Updating files: 67% (363/537) Updating files: 68% (366/537) Updating files: 69% (371/537) Updating files: 70% (376/537) Updating files: 71% (382/537) Updating files: 72% (387/537) Updating files: 73% (393/537) Updating files: 74% (398/537) Updating files: 75% (403/537) Updating files: 76% (409/537) Updating files: 77% (414/537) Updating files: 78% (419/537) Updating files: 79% (425/537) Updating files: 80% (430/537) Updating files: 81% (435/537) Updating files: 82% (441/537) Updating files: 83% (446/537) Updating files: 83% (451/537) Updating files: 84% (452/537) Updating files: 85% (457/537) Updating files: 86% (462/537) Updating files: 87% (468/537) Updating files: 88% (473/537) Updating files: 89% (478/537) Updating files: 90% (484/537) Updating files: 91% (489/537) Updating files: 92% (495/537) Updating files: 93% (500/537) Updating files: 94% (505/537) Updating files: 95% (511/537) Updating files: 96% (516/537) Updating files: 97% (521/537) Updating files: 98% (527/537) Updating files: 99% (532/537) Updating files: 100% (537/537) Updating files: 100% (537/537), done. 2024-03-04T07:55:15.113 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 090436f56ac v0.21 2024-03-04T07:55:15.119 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.21 2024-03-04T07:55:15.449 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 22 -eq 48 ] 2024-03-04T07:55:15.450 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.22 2024-03-04T07:55:15.450 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.22 2024-03-04T07:55:15.450 INFO:tasks.workunit.client.0.smithi113.stdout:v0.22 2024-03-04T07:55:15.450 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.22 2024-03-04T07:55:20.165 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 53% (109/204) Updating files: 54% (111/204) Updating files: 55% (113/204) Updating files: 56% (115/204) Updating files: 57% (117/204) Updating files: 58% (119/204) Updating files: 59% (121/204) Updating files: 60% (123/204) Updating files: 61% (125/204) Updating files: 62% (127/204) Updating files: 63% (129/204) Updating files: 64% (131/204) Updating files: 65% (133/204) Updating files: 66% (135/204) Updating files: 67% (137/204) Updating files: 68% (139/204) Updating files: 69% (141/204) Updating files: 70% (143/204) Updating files: 71% (145/204) Updating files: 72% (147/204) Updating files: 73% (149/204) Updating files: 74% (151/204) Updating files: 74% (152/204) Updating files: 75% (153/204) Updating files: 76% (156/204) Updating files: 77% (158/204) Updating files: 78% (160/204) Updating files: 79% (162/204) Updating files: 79% (163/204) Updating files: 80% (164/204) Updating files: 81% (166/204) Updating files: 82% (168/204) Updating files: 83% (170/204) Updating files: 84% (172/204) Updating files: 85% (174/204) Updating files: 86% (176/204) Updating files: 87% (178/204) Updating files: 88% (180/204) Updating files: 89% (182/204) Updating files: 90% (184/204) Updating files: 91% (186/204) Updating files: 91% (187/204) Updating files: 92% (188/204) Updating files: 93% (190/204) Updating files: 94% (192/204) Updating files: 95% (194/204) Updating files: 96% (196/204) Updating files: 97% (198/204) Updating files: 98% (200/204) Updating files: 99% (202/204) Updating files: 100% (204/204) Updating files: 100% (204/204), done. 2024-03-04T07:55:20.447 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 8a7c95f60ad v0.22 2024-03-04T07:55:20.453 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.22 2024-03-04T07:55:20.549 INFO:tasks.workunit.client.0.smithi113.stdout:v0.23 2024-03-04T07:55:20.549 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 23 -eq 48 ] 2024-03-04T07:55:20.549 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.23 2024-03-04T07:55:20.549 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.23 2024-03-04T07:55:20.549 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.23 2024-03-04T07:55:31.466 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:55:31.470 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:55:38.499 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 9% (15/163) Updating files: 10% (17/163) Updating files: 11% (18/163) Updating files: 12% (20/163) Updating files: 12% (21/163) Updating files: 13% (22/163) Updating files: 14% (23/163) Updating files: 15% (25/163) Updating files: 16% (27/163) Updating files: 17% (28/163) Updating files: 18% (30/163) Updating files: 19% (31/163) Updating files: 20% (33/163) Updating files: 21% (35/163) Updating files: 22% (36/163) Updating files: 23% (38/163) Updating files: 24% (40/163) Updating files: 25% (41/163) Updating files: 26% (43/163) Updating files: 27% (45/163) Updating files: 28% (46/163) Updating files: 29% (48/163) Updating files: 30% (49/163) Updating files: 31% (51/163) Updating files: 32% (53/163) Updating files: 33% (54/163) Updating files: 34% (56/163) Updating files: 35% (58/163) Updating files: 36% (59/163) Updating files: 37% (61/163) Updating files: 38% (62/163) Updating files: 39% (64/163) Updating files: 40% (66/163) Updating files: 41% (67/163) Updating files: 42% (69/163) Updating files: 43% (71/163) Updating files: 44% (72/163) Updating files: 45% (74/163) Updating files: 46% (75/163) Updating files: 47% (77/163) Updating files: 48% (79/163) Updating files: 49% (80/163) Updating files: 50% (82/163) Updating files: 51% (84/163) Updating files: 52% (85/163) Updating files: 53% (87/163) Updating files: 54% (89/163) Updating files: 55% (90/163) Updating files: 56% (92/163) Updating files: 57% (93/163) Updating files: 58% (95/163) Updating files: 59% (97/163) Updating files: 60% (98/163) Updating files: 61% (100/163) Updating files: 62% (102/163) Updating files: 63% (103/163) Updating files: 64% (105/163) Updating files: 65% (106/163) Updating files: 66% (108/163) Updating files: 67% (110/163) Updating files: 68% (111/163) Updating files: 69% (113/163) Updating files: 70% (115/163) Updating files: 71% (116/163) Updating files: 72% (118/163) Updating files: 73% (119/163) Updating files: 74% (121/163) Updating files: 75% (123/163) Updating files: 76% (124/163) Updating files: 77% (126/163) Updating files: 78% (128/163) Updating files: 79% (129/163) Updating files: 80% (131/163) Updating files: 81% (133/163) Updating files: 82% (134/163) Updating files: 83% (136/163) Updating files: 84% (137/163) Updating files: 85% (139/163) Updating files: 86% (141/163) Updating files: 87% (142/163) Updating files: 88% (144/163) Updating files: 89% (146/163) Updating files: 90% (147/163) Updating files: 91% (149/163) Updating files: 92% (150/163) Updating files: 93% (152/163) Updating files: 94% (154/163) Updating files: 95% (155/163) Updating files: 96% (157/163) Updating files: 97% (159/163) Updating files: 98% (160/163) Updating files: 99% (162/163) Updating files: 100% (163/163) Updating files: 100% (163/163), done. 2024-03-04T07:55:38.528 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 5d1d8d0c460 v0.23 2024-03-04T07:55:38.533 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.23 2024-03-04T07:55:38.660 INFO:tasks.workunit.client.0.smithi113.stdout:v0.24 2024-03-04T07:55:38.660 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 24 -eq 48 ] 2024-03-04T07:55:38.661 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.24 2024-03-04T07:55:38.661 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.24 2024-03-04T07:55:38.661 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.24 2024-03-04T07:55:40.805 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 45% (84/185) Updating files: 46% (86/185) Updating files: 47% (87/185) Updating files: 48% (89/185) Updating files: 49% (91/185) Updating files: 50% (93/185) Updating files: 51% (95/185) Updating files: 52% (97/185) Updating files: 53% (99/185) Updating files: 54% (100/185) Updating files: 55% (102/185) Updating files: 56% (104/185) Updating files: 57% (106/185) Updating files: 58% (108/185) Updating files: 59% (110/185) Updating files: 60% (111/185) Updating files: 61% (113/185) Updating files: 62% (115/185) Updating files: 63% (117/185) Updating files: 64% (119/185) Updating files: 65% (121/185) Updating files: 66% (123/185) Updating files: 67% (124/185) Updating files: 68% (126/185) Updating files: 69% (128/185) Updating files: 70% (130/185) Updating files: 71% (132/185) Updating files: 72% (134/185) Updating files: 73% (136/185) Updating files: 74% (137/185) Updating files: 75% (139/185) Updating files: 76% (141/185) Updating files: 77% (143/185) Updating files: 78% (145/185) Updating files: 79% (147/185) Updating files: 80% (148/185) Updating files: 81% (150/185) Updating files: 82% (152/185) Updating files: 83% (154/185) Updating files: 84% (156/185) Updating files: 85% (158/185) Updating files: 86% (160/185) Updating files: 87% (161/185) Updating files: 88% (163/185) Updating files: 89% (165/185) Updating files: 90% (167/185) Updating files: 91% (169/185) Updating files: 92% (171/185) Updating files: 93% (173/185) Updating files: 94% (174/185) Updating files: 95% (176/185) Updating files: 96% (178/185) Updating files: 97% (180/185) Updating files: 98% (182/185) Updating files: 99% (184/185) Updating files: 100% (185/185) Updating files: 100% (185/185), done. 2024-03-04T07:55:41.126 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 180a4176035 v0.24 2024-03-04T07:55:41.135 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.24 2024-03-04T07:55:41.163 INFO:tasks.workunit.client.0.smithi113.stdout:v0.25 2024-03-04T07:55:41.163 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 25 -eq 48 ] 2024-03-04T07:55:41.163 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.25 2024-03-04T07:55:41.163 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.25 2024-03-04T07:55:41.163 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.25 2024-03-04T07:55:50.754 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 11% (69/622) Updating files: 12% (75/622) Updating files: 13% (81/622) Updating files: 14% (88/622) Updating files: 15% (94/622) Updating files: 16% (100/622) Updating files: 17% (106/622) Updating files: 18% (112/622) Updating files: 18% (114/622) Updating files: 19% (119/622) Updating files: 20% (125/622) Updating files: 21% (131/622) Updating files: 22% (137/622) Updating files: 23% (144/622) Updating files: 23% (147/622) Updating files: 24% (150/622) Updating files: 25% (156/622) Updating files: 26% (162/622) Updating files: 27% (168/622) Updating files: 28% (175/622) Updating files: 29% (181/622) Updating files: 30% (187/622) Updating files: 31% (193/622) Updating files: 32% (200/622) Updating files: 33% (206/622) Updating files: 34% (212/622) Updating files: 35% (218/622) Updating files: 36% (224/622) Updating files: 37% (231/622) Updating files: 38% (237/622) Updating files: 39% (243/622) Updating files: 40% (249/622) Updating files: 41% (256/622) Updating files: 42% (262/622) Updating files: 43% (268/622) Updating files: 44% (274/622) Updating files: 45% (280/622) Updating files: 46% (287/622) Updating files: 47% (293/622) Updating files: 48% (299/622) Updating files: 49% (305/622) Updating files: 50% (311/622) Updating files: 51% (318/622) Updating files: 52% (324/622) Updating files: 53% (330/622) Updating files: 54% (336/622) Updating files: 55% (343/622) Updating files: 56% (349/622) Updating files: 57% (355/622) Updating files: 58% (361/622) Updating files: 58% (365/622) Updating files: 59% (367/622) Updating files: 60% (374/622) Updating files: 61% (380/622) Updating files: 62% (386/622) Updating files: 63% (392/622) Updating files: 64% (399/622) Updating files: 65% (405/622) Updating files: 66% (411/622) Updating files: 67% (417/622) Updating files: 68% (423/622) Updating files: 69% (430/622) Updating files: 70% (436/622) Updating files: 71% (442/622) Updating files: 72% (448/622) Updating files: 73% (455/622) Updating files: 74% (461/622) Updating files: 75% (467/622) Updating files: 75% (469/622) Updating files: 76% (473/622) Updating files: 77% (479/622) Updating files: 78% (486/622) Updating files: 79% (492/622) Updating files: 80% (498/622) Updating files: 81% (504/622) Updating files: 82% (511/622) Updating files: 83% (517/622) Updating files: 84% (523/622) Updating files: 85% (529/622) Updating files: 86% (535/622) Updating files: 87% (542/622) Updating files: 88% (548/622) Updating files: 89% (554/622) Updating files: 90% (560/622) Updating files: 91% (567/622) Updating files: 92% (573/622) Updating files: 93% (579/622) Updating files: 94% (585/622) Updating files: 95% (591/622) Updating files: 96% (598/622) Updating files: 97% (604/622) Updating files: 98% (610/622) Updating files: 99% (616/622) Updating files: 100% (622/622) Updating files: 100% (622/622), done. 2024-03-04T07:55:50.953 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 3fa121bf0a8 v0.25 2024-03-04T07:55:50.957 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.25 2024-03-04T07:55:51.144 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 26 -eq 48 ] 2024-03-04T07:55:51.144 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.26 2024-03-04T07:55:51.144 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.26 2024-03-04T07:55:51.145 INFO:tasks.workunit.client.0.smithi113.stdout:v0.26 2024-03-04T07:55:51.145 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.26 2024-03-04T07:55:53.825 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 31% (75/239) Updating files: 32% (77/239) Updating files: 33% (79/239) Updating files: 34% (82/239) Updating files: 35% (84/239) Updating files: 36% (87/239) Updating files: 37% (89/239) Updating files: 38% (91/239) Updating files: 39% (94/239) Updating files: 40% (96/239) Updating files: 41% (98/239) Updating files: 42% (101/239) Updating files: 43% (103/239) Updating files: 44% (106/239) Updating files: 45% (108/239) Updating files: 46% (110/239) Updating files: 47% (113/239) Updating files: 48% (115/239) Updating files: 49% (118/239) Updating files: 50% (120/239) Updating files: 51% (122/239) Updating files: 52% (125/239) Updating files: 53% (127/239) Updating files: 54% (130/239) Updating files: 55% (132/239) Updating files: 56% (134/239) Updating files: 57% (137/239) Updating files: 58% (139/239) Updating files: 59% (142/239) Updating files: 60% (144/239) Updating files: 61% (146/239) Updating files: 62% (149/239) Updating files: 63% (151/239) Updating files: 64% (153/239) Updating files: 65% (156/239) Updating files: 66% (158/239) Updating files: 66% (159/239) Updating files: 67% (161/239) Updating files: 68% (163/239) Updating files: 69% (165/239) Updating files: 70% (168/239) Updating files: 71% (170/239) Updating files: 72% (173/239) Updating files: 73% (175/239) Updating files: 74% (177/239) Updating files: 75% (180/239) Updating files: 76% (182/239) Updating files: 77% (185/239) Updating files: 78% (187/239) Updating files: 79% (189/239) Updating files: 80% (192/239) Updating files: 81% (194/239) Updating files: 82% (196/239) Updating files: 83% (199/239) Updating files: 84% (201/239) Updating files: 85% (204/239) Updating files: 86% (206/239) Updating files: 87% (208/239) Updating files: 88% (211/239) Updating files: 89% (213/239) Updating files: 90% (216/239) Updating files: 91% (218/239) Updating files: 92% (220/239) Updating files: 93% (223/239) Updating files: 94% (225/239) Updating files: 95% (228/239) Updating files: 96% (230/239) Updating files: 97% (232/239) Updating files: 98% (235/239) Updating files: 99% (237/239) Updating files: 100% (239/239) Updating files: 100% (239/239), done. 2024-03-04T07:55:53.871 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 9981ff90968 v0.26 2024-03-04T07:55:53.879 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.26 2024-03-04T07:55:53.966 INFO:tasks.workunit.client.0.smithi113.stdout:v0.27 2024-03-04T07:55:53.966 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 27 -eq 48 ] 2024-03-04T07:55:53.966 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.27 2024-03-04T07:55:53.966 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.27 2024-03-04T07:55:53.967 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.27 2024-03-04T07:56:01.521 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:56:01.525 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:56:12.938 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 44% (112/253) Updating files: 45% (114/253) Updating files: 46% (117/253) Updating files: 47% (119/253) Updating files: 48% (122/253) Updating files: 49% (124/253) Updating files: 50% (127/253) Updating files: 51% (130/253) Updating files: 52% (132/253) Updating files: 53% (135/253) Updating files: 54% (137/253) Updating files: 55% (140/253) Updating files: 56% (142/253) Updating files: 57% (145/253) Updating files: 58% (147/253) Updating files: 59% (150/253) Updating files: 60% (152/253) Updating files: 61% (155/253) Updating files: 62% (157/253) Updating files: 63% (160/253) Updating files: 64% (162/253) Updating files: 65% (165/253) Updating files: 66% (167/253) Updating files: 67% (170/253) Updating files: 68% (173/253) Updating files: 69% (175/253) Updating files: 70% (178/253) Updating files: 71% (180/253) Updating files: 72% (183/253) Updating files: 73% (185/253) Updating files: 73% (186/253) Updating files: 74% (188/253) Updating files: 75% (190/253) Updating files: 76% (193/253) Updating files: 77% (195/253) Updating files: 78% (198/253) Updating files: 79% (200/253) Updating files: 80% (203/253) Updating files: 81% (205/253) Updating files: 82% (208/253) Updating files: 83% (210/253) Updating files: 84% (213/253) Updating files: 85% (216/253) Updating files: 86% (218/253) Updating files: 87% (221/253) Updating files: 88% (223/253) Updating files: 89% (226/253) Updating files: 90% (228/253) Updating files: 91% (231/253) Updating files: 92% (233/253) Updating files: 93% (236/253) Updating files: 94% (238/253) Updating files: 95% (241/253) Updating files: 96% (243/253) Updating files: 97% (246/253) Updating files: 98% (248/253) Updating files: 99% (251/253) Updating files: 100% (253/253) Updating files: 100% (253/253), done. 2024-03-04T07:56:12.988 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 793034c62c8 v0.27 2024-03-04T07:56:12.997 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.27 2024-03-04T07:56:13.077 INFO:tasks.workunit.client.0.smithi113.stdout:v0.28 2024-03-04T07:56:13.078 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 28 -eq 48 ] 2024-03-04T07:56:13.078 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.28 2024-03-04T07:56:13.078 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.28 2024-03-04T07:56:13.078 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.28 2024-03-04T07:56:17.594 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 48% (110/227) Updating files: 48% (111/227) Updating files: 49% (112/227) Updating files: 50% (114/227) Updating files: 51% (116/227) Updating files: 52% (119/227) Updating files: 52% (120/227) Updating files: 53% (121/227) Updating files: 54% (123/227) Updating files: 55% (125/227) Updating files: 56% (128/227) Updating files: 57% (130/227) Updating files: 58% (132/227) Updating files: 59% (134/227) Updating files: 60% (137/227) Updating files: 61% (139/227) Updating files: 62% (141/227) Updating files: 63% (144/227) Updating files: 64% (146/227) Updating files: 65% (148/227) Updating files: 66% (150/227) Updating files: 67% (153/227) Updating files: 68% (155/227) Updating files: 69% (157/227) Updating files: 70% (159/227) Updating files: 71% (162/227) Updating files: 72% (164/227) Updating files: 73% (166/227) Updating files: 74% (168/227) Updating files: 75% (171/227) Updating files: 76% (173/227) Updating files: 77% (175/227) Updating files: 78% (178/227) Updating files: 79% (180/227) Updating files: 80% (182/227) Updating files: 81% (184/227) Updating files: 82% (187/227) Updating files: 83% (189/227) Updating files: 84% (191/227) Updating files: 85% (193/227) Updating files: 86% (196/227) Updating files: 87% (198/227) Updating files: 88% (200/227) Updating files: 89% (203/227) Updating files: 90% (205/227) Updating files: 91% (207/227) Updating files: 92% (209/227) Updating files: 93% (212/227) Updating files: 94% (214/227) Updating files: 95% (216/227) Updating files: 96% (218/227) Updating files: 97% (221/227) Updating files: 98% (223/227) Updating files: 99% (225/227) Updating files: 100% (227/227) Updating files: 100% (227/227), done. 2024-03-04T07:56:17.882 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 071881d7e55 v0.28 2024-03-04T07:56:17.891 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.28 2024-03-04T07:56:18.113 INFO:tasks.workunit.client.0.smithi113.stdout:v0.29 2024-03-04T07:56:18.114 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 29 -eq 48 ] 2024-03-04T07:56:18.114 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.29 2024-03-04T07:56:18.114 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.29 2024-03-04T07:56:18.114 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.29 2024-03-04T07:56:29.402 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 11% (18/159) Updating files: 12% (20/159) Updating files: 13% (21/159) Updating files: 14% (23/159) Updating files: 15% (24/159) Updating files: 16% (26/159) Updating files: 17% (28/159) Updating files: 18% (29/159) Updating files: 19% (31/159) Updating files: 20% (32/159) Updating files: 21% (34/159) Updating files: 22% (35/159) Updating files: 23% (37/159) Updating files: 24% (39/159) Updating files: 25% (40/159) Updating files: 26% (42/159) Updating files: 27% (43/159) Updating files: 28% (45/159) Updating files: 29% (47/159) Updating files: 30% (48/159) Updating files: 31% (50/159) Updating files: 32% (51/159) Updating files: 33% (53/159) Updating files: 34% (55/159) Updating files: 35% (56/159) Updating files: 36% (58/159) Updating files: 37% (59/159) Updating files: 38% (61/159) Updating files: 39% (63/159) Updating files: 40% (64/159) Updating files: 41% (66/159) Updating files: 42% (67/159) Updating files: 43% (69/159) Updating files: 44% (70/159) Updating files: 45% (72/159) Updating files: 46% (74/159) Updating files: 47% (75/159) Updating files: 48% (77/159) Updating files: 49% (78/159) Updating files: 50% (80/159) Updating files: 51% (82/159) Updating files: 52% (83/159) Updating files: 53% (85/159) Updating files: 54% (86/159) Updating files: 55% (88/159) Updating files: 56% (90/159) Updating files: 57% (91/159) Updating files: 58% (93/159) Updating files: 59% (94/159) Updating files: 60% (96/159) Updating files: 61% (97/159) Updating files: 62% (99/159) Updating files: 63% (101/159) Updating files: 64% (102/159) Updating files: 65% (104/159) Updating files: 66% (105/159) Updating files: 67% (107/159) Updating files: 68% (109/159) Updating files: 69% (110/159) Updating files: 70% (112/159) Updating files: 71% (113/159) Updating files: 72% (115/159) Updating files: 73% (117/159) Updating files: 74% (118/159) Updating files: 75% (120/159) Updating files: 76% (121/159) Updating files: 77% (123/159) Updating files: 78% (125/159) Updating files: 79% (126/159) Updating files: 80% (128/159) Updating files: 81% (129/159) Updating files: 82% (131/159) Updating files: 83% (132/159) Updating files: 84% (134/159) Updating files: 85% (136/159) Updating files: 86% (137/159) Updating files: 87% (139/159) Updating files: 88% (140/159) Updating files: 89% (142/159) Updating files: 90% (144/159) Updating files: 91% (145/159) Updating files: 92% (147/159) Updating files: 93% (148/159) Updating files: 94% (150/159) Updating files: 95% (152/159) Updating files: 96% (153/159) Updating files: 97% (155/159) Updating files: 98% (156/159) Updating files: 99% (158/159) Updating files: 100% (159/159) Updating files: 100% (159/159), done. 2024-03-04T07:56:30.175 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 8e69c39f699 v0.29 2024-03-04T07:56:30.181 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.29 2024-03-04T07:56:30.210 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 30 -eq 48 ] 2024-03-04T07:56:30.210 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.30 2024-03-04T07:56:30.210 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.30 2024-03-04T07:56:30.211 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.30 2024-03-04T07:56:30.211 INFO:tasks.workunit.client.0.smithi113.stdout:v0.30 2024-03-04T07:56:31.574 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:56:31.578 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:56:44.125 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 56% (162/285) Updating files: 57% (163/285) Updating files: 58% (166/285) Updating files: 59% (169/285) Updating files: 60% (171/285) Updating files: 61% (174/285) Updating files: 62% (177/285) Updating files: 63% (180/285) Updating files: 64% (183/285) Updating files: 65% (186/285) Updating files: 66% (189/285) Updating files: 66% (190/285) Updating files: 67% (191/285) Updating files: 68% (194/285) Updating files: 69% (197/285) Updating files: 70% (200/285) Updating files: 71% (203/285) Updating files: 72% (206/285) Updating files: 73% (209/285) Updating files: 74% (211/285) Updating files: 75% (214/285) Updating files: 76% (217/285) Updating files: 77% (220/285) Updating files: 78% (223/285) Updating files: 79% (226/285) Updating files: 80% (228/285) Updating files: 81% (231/285) Updating files: 82% (234/285) Updating files: 83% (237/285) Updating files: 84% (240/285) Updating files: 84% (241/285) Updating files: 85% (243/285) Updating files: 86% (246/285) Updating files: 87% (248/285) Updating files: 88% (251/285) Updating files: 89% (254/285) Updating files: 90% (257/285) Updating files: 91% (260/285) Updating files: 92% (263/285) Updating files: 93% (266/285) Updating files: 94% (268/285) Updating files: 95% (271/285) Updating files: 96% (274/285) Updating files: 97% (277/285) Updating files: 98% (280/285) Updating files: 99% (283/285) Updating files: 100% (285/285) Updating files: 100% (285/285), done. 2024-03-04T07:56:44.381 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 64b1b2c70f0 v0.30 2024-03-04T07:56:44.385 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.30 2024-03-04T07:56:45.123 INFO:tasks.workunit.client.0.smithi113.stdout:v0.31 2024-03-04T07:56:45.123 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 31 -eq 48 ] 2024-03-04T07:56:45.124 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.31 2024-03-04T07:56:45.124 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.31 2024-03-04T07:56:45.124 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.31 2024-03-04T07:56:53.584 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:56:53.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd636000640 1 Processor -- start 2024-03-04T07:56:53.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd636000640 1 -- start start 2024-03-04T07:56:53.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd636000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6301527f0 0x7fd630070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:53.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd636000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd630152bd0 0x7fd6301587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:53.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd636000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6301591f0 con 0x7fd6300c8200 2024-03-04T07:56:53.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd636000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd630071df0 con 0x7fd630152bd0 2024-03-04T07:56:53.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd636000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6301597b0 con 0x7fd6301527f0 2024-03-04T07:56:53.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd62fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd630152bd0 0x7fd6301587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:53.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd62fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd630152bd0 0x7fd6301587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57164/0 (socket says 172.21.15.113:57164) 2024-03-04T07:56:53.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd62fe00640 1 -- 172.21.15.113:0/1623038412 learned_addr learned my addr 172.21.15.113:0/1623038412 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:56:53.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.727+0000 7fd62f400640 1 --2- 172.21.15.113:0/1623038412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6301527f0 0x7fd630070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:53.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd62fe00640 1 -- 172.21.15.113:0/1623038412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6301527f0 msgr2=0x7fd630070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:56:53.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd62fe00640 1 --2- 172.21.15.113:0/1623038412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6301527f0 0x7fd630070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:53.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd62fe00640 1 -- 172.21.15.113:0/1623038412 >> v1:172.21.15.136:6789/0 conn(0x7fd6300c8200 legacy=0x7fd630072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:53.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd62fe00640 1 -- 172.21.15.113:0/1623038412 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6301594d0 con 0x7fd630152bd0 2024-03-04T07:56:53.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd62fe00640 1 --2- 172.21.15.113:0/1623038412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd630152bd0 0x7fd6301587d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd628009e50 tx=0x7fd62801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cacce3db7aa143 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:53.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd62ea00640 1 -- 172.21.15.113:0/1623038412 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280a57f0 con 0x7fd630152bd0 2024-03-04T07:56:53.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd62ea00640 1 -- 172.21.15.113:0/1623038412 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6280a5990 con 0x7fd630152bd0 2024-03-04T07:56:53.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd62ea00640 1 -- 172.21.15.113:0/1623038412 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6280a5d60 con 0x7fd630152bd0 2024-03-04T07:56:53.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd636000640 1 -- 172.21.15.113:0/1623038412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd630152bd0 msgr2=0x7fd6301587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd636000640 1 --2- 172.21.15.113:0/1623038412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd630152bd0 0x7fd6301587d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd628009e50 tx=0x7fd62801ca30 comp rx=0 tx=0).stop 2024-03-04T07:56:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd636000640 1 -- 172.21.15.113:0/1623038412 shutdown_connections 2024-03-04T07:56:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd636000640 1 --2- 172.21.15.113:0/1623038412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd630152bd0 0x7fd6301587d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd636000640 1 --2- 172.21.15.113:0/1623038412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6301527f0 0x7fd630070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.731+0000 7fd636000640 1 -- 172.21.15.113:0/1623038412 >> 172.21.15.113:0/1623038412 conn(0x7fd63006e820 msgr2=0x7fd63006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:56:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 -- 172.21.15.113:0/1623038412 shutdown_connections 2024-03-04T07:56:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 -- 172.21.15.113:0/1623038412 wait complete. 2024-03-04T07:56:53.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 Processor -- start 2024-03-04T07:56:53.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 -- start start 2024-03-04T07:56:53.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6300c8200 0x7fd630148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301527f0 0x7fd6301f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd630152bd0 0x7fd630151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6301654f0 con 0x7fd6301527f0 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd630165370 con 0x7fd6300c8200 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd630165670 con 0x7fd630152bd0 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd626a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301527f0 0x7fd6301f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd626a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301527f0 0x7fd6301f39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57170/0 (socket says 172.21.15.113:57170) 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd626a00640 1 -- 172.21.15.113:0/2176256709 learned_addr learned my addr 172.21.15.113:0/2176256709 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd62f400640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6300c8200 0x7fd630148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd62fe00640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd630152bd0 0x7fd630151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd626a00640 1 -- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd630152bd0 msgr2=0x7fd630151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd626a00640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd630152bd0 0x7fd630151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd626a00640 1 -- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6300c8200 msgr2=0x7fd630148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:53.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd626a00640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6300c8200 0x7fd630148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:53.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd626a00640 1 -- 172.21.15.113:0/2176256709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630150210 con 0x7fd6301527f0 2024-03-04T07:56:53.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd626a00640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301527f0 0x7fd6301f39f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c430 tx=0x7fd61800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:53.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd627e00640 1 -- 172.21.15.113:0/2176256709 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180052d0 con 0x7fd6301527f0 2024-03-04T07:56:53.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd627e00640 1 -- 172.21.15.113:0/2176256709 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd618005470 con 0x7fd6301527f0 2024-03-04T07:56:53.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.735+0000 7fd636000640 1 -- 172.21.15.113:0/2176256709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63015b980 con 0x7fd6301527f0 2024-03-04T07:56:53.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.739+0000 7fd627e00640 1 -- 172.21.15.113:0/2176256709 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd618007260 con 0x7fd6301527f0 2024-03-04T07:56:53.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.739+0000 7fd636000640 1 -- 172.21.15.113:0/2176256709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63015bf40 con 0x7fd6301527f0 2024-03-04T07:56:53.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.739+0000 7fd636000640 1 -- 172.21.15.113:0/2176256709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd630075c00 con 0x7fd6301527f0 2024-03-04T07:56:53.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.739+0000 7fd627e00640 1 -- 172.21.15.113:0/2176256709 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd618007400 con 0x7fd6301527f0 2024-03-04T07:56:53.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.739+0000 7fd627e00640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd60c066f60 0x7fd60c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:53.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.739+0000 7fd627e00640 1 -- 172.21.15.113:0/2176256709 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7fd618084e90 con 0x7fd6301527f0 2024-03-04T07:56:53.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.743+0000 7fd62f400640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd60c066f60 0x7fd60c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:53.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.743+0000 7fd62f400640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd60c066f60 0x7fd60c069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd620009b60 tx=0x7fd620009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:53.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:53.743+0000 7fd627e00640 1 -- 172.21.15.113:0/2176256709 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd618055370 con 0x7fd6301527f0 2024-03-04T07:56:54.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 -- 172.21.15.113:0/2176256709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6300c79d0 con 0x7fd6301527f0 2024-03-04T07:56:54.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd627e00640 1 -- 172.21.15.113:0/2176256709 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fd618059410 con 0x7fd6301527f0 2024-03-04T07:56:54.001 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:56:54.001 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:55:01.109616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6240,6145,5940]},"id":1}]} 2024-03-04T07:56:54.001 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 82 2024-03-04T07:56:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 -- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd60c066f60 msgr2=0x7fd60c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd60c066f60 0x7fd60c069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd620009b60 tx=0x7fd620009190 comp rx=0 tx=0).stop 2024-03-04T07:56:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 -- 172.21.15.113:0/2176256709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301527f0 msgr2=0x7fd6301f39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301527f0 0x7fd6301f39f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c430 tx=0x7fd61800c8f0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 -- 172.21.15.113:0/2176256709 shutdown_connections 2024-03-04T07:56:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd630152bd0 0x7fd630151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd60c066f60 0x7fd60c069420 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6301527f0 0x7fd6301f39f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 --2- 172.21.15.113:0/2176256709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6300c8200 0x7fd630148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 -- 172.21.15.113:0/2176256709 >> 172.21.15.113:0/2176256709 conn(0x7fd63006e820 msgr2=0x7fd6300cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:56:54.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 -- 172.21.15.113:0/2176256709 shutdown_connections 2024-03-04T07:56:54.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.003+0000 7fd636000640 1 -- 172.21.15.113:0/2176256709 wait complete. 2024-03-04T07:56:54.024 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2024-03-04T07:56:54.025 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-03-04T07:56:54.025 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:56:54.052 INFO:tasks.ceph.mds.e:Stopped 2024-03-04T07:56:54.052 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:56:54.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad19000640 1 Processor -- start 2024-03-04T07:56:54.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad19000640 1 -- start start 2024-03-04T07:56:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad19000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad140ce6b0 0x7fad1415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad19000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1406de80 0x7fad14154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad19000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1406c210 con 0x7fad140ce2d0 2024-03-04T07:56:54.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad19000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1415bbf0 con 0x7fad140ce6b0 2024-03-04T07:56:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad19000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1406eb80 con 0x7fad1406de80 2024-03-04T07:56:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad11400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad140ce6b0 0x7fad1415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad11400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad140ce6b0 0x7fad1415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57186/0 (socket says 172.21.15.113:57186) 2024-03-04T07:56:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad11400640 1 -- 172.21.15.113:0/1320606273 learned_addr learned my addr 172.21.15.113:0/1320606273 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:56:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad12800640 1 --2- 172.21.15.113:0/1320606273 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1406de80 0x7fad14154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad11400640 1 -- 172.21.15.113:0/1320606273 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1406de80 msgr2=0x7fad14154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:56:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad11400640 1 --2- 172.21.15.113:0/1320606273 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1406de80 0x7fad14154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad11400640 1 -- 172.21.15.113:0/1320606273 >> v1:172.21.15.136:6789/0 conn(0x7fad140ce2d0 legacy=0x7fad1406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:54.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.239+0000 7fad11400640 1 -- 172.21.15.113:0/1320606273 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1406c4f0 con 0x7fad140ce6b0 2024-03-04T07:56:54.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad11400640 1 --2- 172.21.15.113:0/1320606273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 0x7fad1415b6b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fad00009870 tx=0x7fad0001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c1224620600be6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:54.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad10a00640 1 -- 172.21.15.113:0/1320606273 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000b5070 con 0x7fad140ce6b0 2024-03-04T07:56:54.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad10a00640 1 -- 172.21.15.113:0/1320606273 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad000a8a50 con 0x7fad140ce6b0 2024-03-04T07:56:54.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad10a00640 1 -- 172.21.15.113:0/1320606273 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000a8e20 con 0x7fad140ce6b0 2024-03-04T07:56:54.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad19000640 1 -- 172.21.15.113:0/1320606273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 msgr2=0x7fad1415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:54.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad19000640 1 --2- 172.21.15.113:0/1320606273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 0x7fad1415b6b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fad00009870 tx=0x7fad0001ed50 comp rx=0 tx=0).stop 2024-03-04T07:56:54.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad19000640 1 -- 172.21.15.113:0/1320606273 shutdown_connections 2024-03-04T07:56:54.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad19000640 1 --2- 172.21.15.113:0/1320606273 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1406de80 0x7fad14154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad19000640 1 --2- 172.21.15.113:0/1320606273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 0x7fad1415b6b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad19000640 1 -- 172.21.15.113:0/1320606273 >> 172.21.15.113:0/1320606273 conn(0x7fad140709a0 msgr2=0x7fad14070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:56:54.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad19000640 1 -- 172.21.15.113:0/1320606273 shutdown_connections 2024-03-04T07:56:54.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.243+0000 7fad19000640 1 -- 172.21.15.113:0/1320606273 wait complete. 2024-03-04T07:56:54.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad19000640 1 Processor -- start 2024-03-04T07:56:54.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad19000640 1 -- start start 2024-03-04T07:56:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad19000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1406de80 0x7fad14152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad19000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad140ce2d0 0x7fad1414fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad19000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 0x7fad14150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad19000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1406c7d0 con 0x7fad140ce6b0 2024-03-04T07:56:54.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad19000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1406e810 con 0x7fad140ce2d0 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad19000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1406f720 con 0x7fad1406de80 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad12800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 0x7fad14150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad12800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 0x7fad14150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57196/0 (socket says 172.21.15.113:57196) 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad12800640 1 -- 172.21.15.113:0/1866839915 learned_addr learned my addr 172.21.15.113:0/1866839915 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad11400640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad140ce2d0 0x7fad1414fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad11e00640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1406de80 0x7fad14152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad12800640 1 -- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1406de80 msgr2=0x7fad14152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad12800640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1406de80 0x7fad14152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad12800640 1 -- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad140ce2d0 msgr2=0x7fad1414fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad12800640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad140ce2d0 0x7fad1414fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad12800640 1 -- 172.21.15.113:0/1866839915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad14150c20 con 0x7fad140ce6b0 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad11400640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad140ce2d0 0x7fad1414fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad12800640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 0x7fad14150180 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fad0c007e40 tx=0x7fad0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:54.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad0a000640 1 -- 172.21.15.113:0/1866839915 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c014620 con 0x7fad140ce6b0 2024-03-04T07:56:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad0a000640 1 -- 172.21.15.113:0/1866839915 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c019070 con 0x7fad140ce6b0 2024-03-04T07:56:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad19000640 1 -- 172.21.15.113:0/1866839915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad14150ec0 con 0x7fad140ce6b0 2024-03-04T07:56:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.247+0000 7fad0a000640 1 -- 172.21.15.113:0/1866839915 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c014a20 con 0x7fad140ce6b0 2024-03-04T07:56:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.251+0000 7fad19000640 1 -- 172.21.15.113:0/1866839915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad14151480 con 0x7fad140ce6b0 2024-03-04T07:56:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.251+0000 7fad19000640 1 -- 172.21.15.113:0/1866839915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad14075c00 con 0x7fad140ce6b0 2024-03-04T07:56:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.251+0000 7fad0a000640 1 -- 172.21.15.113:0/1866839915 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad0c014bc0 con 0x7fad140ce6b0 2024-03-04T07:56:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.251+0000 7fad0a000640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.251+0000 7fad0a000640 1 -- 172.21.15.113:0/1866839915 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7fad0c085e50 con 0x7fad140ce6b0 2024-03-04T07:56:54.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.251+0000 7fad11e00640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:54.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.251+0000 7fad11e00640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facf4066f60 0x7facf4069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fad04001650 tx=0x7fad04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:54.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.255+0000 7fad0a000640 1 -- 172.21.15.113:0/1866839915 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fad0c056900 con 0x7fad140ce6b0 2024-03-04T07:56:54.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.471+0000 7fad19000640 1 -- 172.21.15.113:0/1866839915 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad1406cb40 con 0x7fad140ce6b0 2024-03-04T07:56:54.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.471+0000 7fad0a000640 1 -- 172.21.15.113:0/1866839915 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fad0c056200 con 0x7fad140ce6b0 2024-03-04T07:56:54.471 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:56:54.471 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:55:01.109616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6240,6145,5940]},"id":1}]} 2024-03-04T07:56:54.471 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 82 2024-03-04T07:56:54.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 -- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facf4066f60 msgr2=0x7facf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:54.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facf4066f60 0x7facf4069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fad04001650 tx=0x7fad04009040 comp rx=0 tx=0).stop 2024-03-04T07:56:54.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 -- 172.21.15.113:0/1866839915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 msgr2=0x7fad14150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:54.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 0x7fad14150180 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fad0c007e40 tx=0x7fad0c00e4b0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 -- 172.21.15.113:0/1866839915 shutdown_connections 2024-03-04T07:56:54.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad140ce6b0 0x7fad14150180 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad140ce2d0 0x7fad1414fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 --2- 172.21.15.113:0/1866839915 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad1406de80 0x7fad14152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:54.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 -- 172.21.15.113:0/1866839915 >> 172.21.15.113:0/1866839915 conn(0x7fad140709a0 msgr2=0x7fad1414cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:56:54.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 -- 172.21.15.113:0/1866839915 shutdown_connections 2024-03-04T07:56:54.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:54.475+0000 7fad19000640 1 -- 172.21.15.113:0/1866839915 wait complete. 2024-03-04T07:56:54.499 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:56:56.501 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:56:56.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.639+0000 7fa010400640 1 Processor -- start 2024-03-04T07:56:56.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.639+0000 7fa010400640 1 -- start start 2024-03-04T07:56:56.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.639+0000 7fa010400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa008158010 0x7fa00806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:56.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa010400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa00806ff20 0x7fa00815b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:56.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa010400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa00815b750 0x7fa0080ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:56.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa010400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00806f450 con 0x7fa00806ff20 2024-03-04T07:56:56.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa010400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa00815ad80 con 0x7fa00815b750 2024-03-04T07:56:56.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa010400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00806e290 con 0x7fa008158010 2024-03-04T07:56:56.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00d000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa00806ff20 0x7fa00815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:56.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00d000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa00806ff20 0x7fa00815b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57206/0 (socket says 172.21.15.113:57206) 2024-03-04T07:56:56.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00d000640 1 -- 172.21.15.113:0/1611988651 learned_addr learned my addr 172.21.15.113:0/1611988651 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:56:56.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00da00640 1 --2- 172.21.15.113:0/1611988651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa008158010 0x7fa00806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:56.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00e400640 1 --2- 172.21.15.113:0/1611988651 >> v2:172.21.15.136:3300/0 conn(0x7fa00815b750 0x7fa0080ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:56.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00d000640 1 -- 172.21.15.113:0/1611988651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa008158010 msgr2=0x7fa00806dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:56:56.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00d000640 1 --2- 172.21.15.113:0/1611988651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa008158010 0x7fa00806dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00d000640 1 -- 172.21.15.113:0/1611988651 >> v2:172.21.15.136:3300/0 conn(0x7fa00815b750 msgr2=0x7fa0080ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:56:56.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00d000640 1 --2- 172.21.15.113:0/1611988651 >> v2:172.21.15.136:3300/0 conn(0x7fa00815b750 0x7fa0080ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00d000640 1 -- 172.21.15.113:0/1611988651 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0080ca0e0 con 0x7fa00806ff20 2024-03-04T07:56:56.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa00d000640 1 --2- 172.21.15.113:0/1611988651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa00806ff20 0x7fa00815b210 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa0080cec40 tx=0x7f9ff401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e39ea74dfefece server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa007e00640 1 -- 172.21.15.113:0/1611988651 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40b0070 con 0x7fa00806ff20 2024-03-04T07:56:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa007e00640 1 -- 172.21.15.113:0/1611988651 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff40ab600 con 0x7fa00806ff20 2024-03-04T07:56:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.643+0000 7fa007e00640 1 -- 172.21.15.113:0/1611988651 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40ab9d0 con 0x7fa00806ff20 2024-03-04T07:56:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 -- 172.21.15.113:0/1611988651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa00806ff20 msgr2=0x7fa00815b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:56.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 --2- 172.21.15.113:0/1611988651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa00806ff20 0x7fa00815b210 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa0080cec40 tx=0x7f9ff401c520 comp rx=0 tx=0).stop 2024-03-04T07:56:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 -- 172.21.15.113:0/1611988651 shutdown_connections 2024-03-04T07:56:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 --2- 172.21.15.113:0/1611988651 >> v2:172.21.15.136:3300/0 conn(0x7fa00815b750 0x7fa0080ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 --2- 172.21.15.113:0/1611988651 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa00806ff20 0x7fa00815b210 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 --2- 172.21.15.113:0/1611988651 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa008158010 0x7fa00806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 -- 172.21.15.113:0/1611988651 >> 172.21.15.113:0/1611988651 conn(0x7fa0080734d0 msgr2=0x7fa008066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:56:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 -- 172.21.15.113:0/1611988651 shutdown_connections 2024-03-04T07:56:56.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 -- 172.21.15.113:0/1611988651 wait complete. 2024-03-04T07:56:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 Processor -- start 2024-03-04T07:56:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 -- start start 2024-03-04T07:56:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa00806ff20 0x7fa0080b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa008158010 0x7fa0080b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa00815b750 0x7fa0080c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0081583f0 con 0x7fa008158010 2024-03-04T07:56:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00806f980 con 0x7fa00815b750 2024-03-04T07:56:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa010400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00806c170 con 0x7fa00806ff20 2024-03-04T07:56:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa008158010 0x7fa0080b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:56.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa008158010 0x7fa0080b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57212/0 (socket says 172.21.15.113:57212) 2024-03-04T07:56:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00d000640 1 -- 172.21.15.113:0/2440477672 learned_addr learned my addr 172.21.15.113:0/2440477672 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:56:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00da00640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa00806ff20 0x7fa0080b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00e400640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa00815b750 0x7fa0080c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00d000640 1 -- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa00806ff20 msgr2=0x7fa0080b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00d000640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa00806ff20 0x7fa0080b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00d000640 1 -- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa00815b750 msgr2=0x7fa0080c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00d000640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa00815b750 0x7fa0080c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00d000640 1 -- 172.21.15.113:0/2440477672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00806ddc0 con 0x7fa008158010 2024-03-04T07:56:56.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00e400640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa00815b750 0x7fa0080c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:56:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.647+0000 7fa00da00640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa00806ff20 0x7fa0080b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:56:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa00d000640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa008158010 0x7fa0080b1790 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40ae040 tx=0x7f9ff40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa005600640 1 -- 172.21.15.113:0/2440477672 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40b9070 con 0x7fa008158010 2024-03-04T07:56:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa005600640 1 -- 172.21.15.113:0/2440477672 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff40b4400 con 0x7fa008158010 2024-03-04T07:56:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa005600640 1 -- 172.21.15.113:0/2440477672 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40b0040 con 0x7fa008158010 2024-03-04T07:56:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa010400640 1 -- 172.21.15.113:0/2440477672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0080c3350 con 0x7fa008158010 2024-03-04T07:56:56.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa010400640 1 -- 172.21.15.113:0/2440477672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0081f76c0 con 0x7fa008158010 2024-03-04T07:56:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa010400640 1 -- 172.21.15.113:0/2440477672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa008075c00 con 0x7fa008158010 2024-03-04T07:56:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa005600640 1 -- 172.21.15.113:0/2440477672 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ff40bc070 con 0x7fa008158010 2024-03-04T07:56:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa005600640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9fe8066f60 0x7f9fe8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa005600640 1 -- 172.21.15.113:0/2440477672 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f9ff411e520 con 0x7fa008158010 2024-03-04T07:56:56.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.651+0000 7fa00da00640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9fe8066f60 0x7f9fe8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:56.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.655+0000 7fa00da00640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9fe8066f60 0x7f9fe8069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009bb0 tx=0x7f9ffc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:56.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.655+0000 7fa005600640 1 -- 172.21.15.113:0/2440477672 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9ff4016110 con 0x7fa008158010 2024-03-04T07:56:56.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 -- 172.21.15.113:0/2440477672 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0080c96b0 con 0x7fa008158010 2024-03-04T07:56:56.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa005600640 1 -- 172.21.15.113:0/2440477672 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f9ff40ef0f0 con 0x7fa008158010 2024-03-04T07:56:56.849 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:56:56.849 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:55:01.109616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6240,6145,5940]},"id":1}]} 2024-03-04T07:56:56.849 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 82 2024-03-04T07:56:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 -- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9fe8066f60 msgr2=0x7f9fe8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9fe8066f60 0x7f9fe8069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009bb0 tx=0x7f9ffc008040 comp rx=0 tx=0).stop 2024-03-04T07:56:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 -- 172.21.15.113:0/2440477672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa008158010 msgr2=0x7fa0080b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa008158010 0x7fa0080b1790 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9ff40ae040 tx=0x7f9ff40192c0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 -- 172.21.15.113:0/2440477672 shutdown_connections 2024-03-04T07:56:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa00815b750 0x7fa0080c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9fe8066f60 0x7f9fe8069420 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa008158010 0x7fa0080b1790 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 --2- 172.21.15.113:0/2440477672 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa00806ff20 0x7fa0080b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 -- 172.21.15.113:0/2440477672 >> 172.21.15.113:0/2440477672 conn(0x7fa0080734d0 msgr2=0x7fa0080cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:56:56.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 -- 172.21.15.113:0/2440477672 shutdown_connections 2024-03-04T07:56:56.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:56.851+0000 7fa010400640 1 -- 172.21.15.113:0/2440477672 wait complete. 2024-03-04T07:56:56.870 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:56:58.871 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:56:59.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.007+0000 7f82f1c00640 1 Processor -- start 2024-03-04T07:56:59.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.007+0000 7f82f1c00640 1 -- start start 2024-03-04T07:56:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82f1c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82ec157ef0 0x7f82ec0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82f1c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82ec06fb70 0x7f82ec0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82f1c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ec0c9c50 con 0x7f82ec070df0 2024-03-04T07:56:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82f1c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ec15a9a0 con 0x7f82ec06fb70 2024-03-04T07:56:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82f1c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec06de30 con 0x7f82ec157ef0 2024-03-04T07:56:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82e3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f82ec070df0 0x7f82ec15b180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50168/0 (socket says 172.21.15.113:50168) 2024-03-04T07:56:59.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82e3e00640 1 -- 172.21.15.113:0/3463044772 learned_addr learned my addr 172.21.15.113:0/3463044772 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:56:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82f1200640 1 --2- 172.21.15.113:0/3463044772 >> v2:172.21.15.113:3300/0 conn(0x7f82ec06fb70 0x7f82ec0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82ebe00640 1 --2- 172.21.15.113:0/3463044772 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82ec157ef0 0x7f82ec0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82eb400640 1 -- 172.21.15.113:0/3463044772 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643603175 0 0) 0x7f82ec0c9c50 con 0x7f82ec070df0 2024-03-04T07:56:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82f1200640 1 -- 172.21.15.113:0/3463044772 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82ec157ef0 msgr2=0x7f82ec0c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:56:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82f1200640 1 --2- 172.21.15.113:0/3463044772 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82ec157ef0 0x7f82ec0c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82f1200640 1 -- 172.21.15.113:0/3463044772 >> v1:172.21.15.113:6789/0 conn(0x7f82ec070df0 legacy=0x7f82ec15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:59.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.011+0000 7f82f1200640 1 -- 172.21.15.113:0/3463044772 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec06c210 con 0x7f82ec06fb70 2024-03-04T07:56:59.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82f1200640 1 --2- 172.21.15.113:0/3463044772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 0x7f82ec0cdfe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009d90 tx=0x7f82e401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890a7deffb7ede6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82eb400640 1 -- 172.21.15.113:0/3463044772 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e40b2070 con 0x7f82ec06fb70 2024-03-04T07:56:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82eb400640 1 -- 172.21.15.113:0/3463044772 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e40a5940 con 0x7f82ec06fb70 2024-03-04T07:56:59.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82eb400640 1 -- 172.21.15.113:0/3463044772 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e40a5d10 con 0x7f82ec06fb70 2024-03-04T07:56:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82f1c00640 1 -- 172.21.15.113:0/3463044772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 msgr2=0x7f82ec0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82f1c00640 1 --2- 172.21.15.113:0/3463044772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 0x7f82ec0cdfe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009d90 tx=0x7f82e401c970 comp rx=0 tx=0).stop 2024-03-04T07:56:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82f1c00640 1 -- 172.21.15.113:0/3463044772 shutdown_connections 2024-03-04T07:56:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82f1c00640 1 --2- 172.21.15.113:0/3463044772 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 0x7f82ec0cdfe0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82f1c00640 1 --2- 172.21.15.113:0/3463044772 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82ec157ef0 0x7f82ec0c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82f1c00640 1 -- 172.21.15.113:0/3463044772 >> 172.21.15.113:0/3463044772 conn(0x7f82ec072f40 msgr2=0x7f82ec0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:56:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82f1c00640 1 -- 172.21.15.113:0/3463044772 shutdown_connections 2024-03-04T07:56:59.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.015+0000 7f82f1c00640 1 -- 172.21.15.113:0/3463044772 wait complete. 2024-03-04T07:56:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82f1c00640 1 Processor -- start 2024-03-04T07:56:59.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82f1c00640 1 -- start start 2024-03-04T07:56:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82f1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 0x7f82ec159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82f1c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82ec070df0 0x7f82ec156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82f1c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82ec157ef0 0x7f82ec1f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82f1c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec1582d0 con 0x7f82ec06fb70 2024-03-04T07:56:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82f1c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec15b890 con 0x7f82ec157ef0 2024-03-04T07:56:59.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82f1c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ec158450 con 0x7f82ec070df0 2024-03-04T07:56:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82ebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 0x7f82ec159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82ebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 0x7f82ec159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42804/0 (socket says 172.21.15.113:42804) 2024-03-04T07:56:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82ebe00640 1 -- 172.21.15.113:0/3078735910 learned_addr learned my addr 172.21.15.113:0/3078735910 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:56:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82e3e00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82ec070df0 0x7f82ec156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.019+0000 7f82f1200640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82ec157ef0 0x7f82ec1f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82ebe00640 1 -- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82ec070df0 msgr2=0x7f82ec156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82ebe00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82ec070df0 0x7f82ec156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82ebe00640 1 -- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82ec157ef0 msgr2=0x7f82ec1f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82ebe00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82ec157ef0 0x7f82ec1f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:59.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82ebe00640 1 -- 172.21.15.113:0/3078735910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec150cd0 con 0x7f82ec06fb70 2024-03-04T07:56:59.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82f1200640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82ec157ef0 0x7f82ec1f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:56:59.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82ebe00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 0x7f82ec159630 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00e8a0 tx=0x7f82dc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:59.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82e8c00640 1 -- 172.21.15.113:0/3078735910 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc017070 con 0x7f82ec06fb70 2024-03-04T07:56:59.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82e8c00640 1 -- 172.21.15.113:0/3078735910 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc00c8e0 con 0x7f82ec06fb70 2024-03-04T07:56:59.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82f1c00640 1 -- 172.21.15.113:0/3078735910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ec150fb0 con 0x7f82ec06fb70 2024-03-04T07:56:59.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82e8c00640 1 -- 172.21.15.113:0/3078735910 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00cc90 con 0x7f82ec06fb70 2024-03-04T07:56:59.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82f1c00640 1 -- 172.21.15.113:0/3078735910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ec151570 con 0x7f82ec06fb70 2024-03-04T07:56:59.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82f1c00640 1 -- 172.21.15.113:0/3078735910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ec075a80 con 0x7f82ec06fb70 2024-03-04T07:56:59.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.023+0000 7f82e8c00640 1 -- 172.21.15.113:0/3078735910 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82dc01a070 con 0x7f82ec06fb70 2024-03-04T07:56:59.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.027+0000 7f82e8c00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82c8066f60 0x7f82c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:56:59.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.027+0000 7f82e8c00640 1 -- 172.21.15.113:0/3078735910 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7f82dc0857d0 con 0x7f82ec06fb70 2024-03-04T07:56:59.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.027+0000 7f82e3e00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82c8066f60 0x7f82c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:56:59.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.027+0000 7f82e3e00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82c8066f60 0x7f82c8069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0b3290 tx=0x7f82d4007b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:56:59.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.027+0000 7f82e8c00640 1 -- 172.21.15.113:0/3078735910 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f82dc056280 con 0x7f82ec06fb70 2024-03-04T07:56:59.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.219+0000 7f82f1c00640 1 -- 172.21.15.113:0/3078735910 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82ec06f950 con 0x7f82ec06fb70 2024-03-04T07:56:59.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.219+0000 7f82e8c00640 1 -- 172.21.15.113:0/3078735910 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+6612 (secure 0 0 0) 0x7f82dc055b80 con 0x7f82ec06fb70 2024-03-04T07:56:59.219 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:56:59.219 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:55:01.109616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6240,6145,5940]},"id":1}]} 2024-03-04T07:56:59.219 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 82 2024-03-04T07:56:59.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 -- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82c8066f60 msgr2=0x7f82c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82c8066f60 0x7f82c8069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f82ec0b3290 tx=0x7f82d4007b60 comp rx=0 tx=0).stop 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 -- 172.21.15.113:0/3078735910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 msgr2=0x7f82ec159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 0x7f82ec159630 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00e8a0 tx=0x7f82dc00eca0 comp rx=0 tx=0).stop 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 -- 172.21.15.113:0/3078735910 shutdown_connections 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82ec157ef0 0x7f82ec1f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82ec070df0 0x7f82ec156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82c8066f60 0x7f82c8069420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 --2- 172.21.15.113:0/3078735910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82ec06fb70 0x7f82ec159630 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 -- 172.21.15.113:0/3078735910 >> 172.21.15.113:0/3078735910 conn(0x7f82ec072f40 msgr2=0x7f82ec0c8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 -- 172.21.15.113:0/3078735910 shutdown_connections 2024-03-04T07:56:59.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:56:59.223+0000 7f82f1c00640 1 -- 172.21.15.113:0/3078735910 wait complete. 2024-03-04T07:56:59.248 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:57:01.249 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:01.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.387+0000 7eff7d000640 1 Processor -- start 2024-03-04T07:57:01.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.387+0000 7eff7d000640 1 -- start start 2024-03-04T07:57:01.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.387+0000 7eff7d000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7eff78152a90 0x7eff780730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:01.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.387+0000 7eff7d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 0x7eff7815c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:01.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.387+0000 7eff7d000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7806f1b0 con 0x7eff7806cd10 2024-03-04T07:57:01.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.387+0000 7eff7d000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff78152e70 con 0x7eff78152a90 2024-03-04T07:57:01.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.387+0000 7eff7d000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7806c320 con 0x7eff7815bce0 2024-03-04T07:57:01.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.387+0000 7eff77400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7eff78152a90 0x7eff780730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:01.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff76a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7eff7806cd10 0x7eff7806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50170/0 (socket says 172.21.15.113:50170) 2024-03-04T07:57:01.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff77400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7eff78152a90 0x7eff780730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42808/0 (socket says 172.21.15.113:42808) 2024-03-04T07:57:01.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff76a00640 1 -- 172.21.15.113:0/1625762288 learned_addr learned my addr 172.21.15.113:0/1625762288 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:57:01.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff77e00640 1 --2- 172.21.15.113:0/1625762288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 0x7eff7815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:01.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff76000640 1 -- 172.21.15.113:0/1625762288 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125872981 0 0) 0x7eff7806f1b0 con 0x7eff7806cd10 2024-03-04T07:57:01.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff76000640 1 -- 172.21.15.113:0/1625762288 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff5c003530 con 0x7eff7806cd10 2024-03-04T07:57:01.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff77e00640 1 -- 172.21.15.113:0/1625762288 >> v2:172.21.15.113:3300/0 conn(0x7eff78152a90 msgr2=0x7eff780730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:01.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff77e00640 1 --2- 172.21.15.113:0/1625762288 >> v2:172.21.15.113:3300/0 conn(0x7eff78152a90 0x7eff780730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:01.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff77e00640 1 -- 172.21.15.113:0/1625762288 >> v1:172.21.15.113:6789/0 conn(0x7eff7806cd10 legacy=0x7eff7806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:01.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff77e00640 1 -- 172.21.15.113:0/1625762288 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78071580 con 0x7eff7815bce0 2024-03-04T07:57:01.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.391+0000 7eff77400640 1 --2- 172.21.15.113:0/1625762288 >> v2:172.21.15.113:3300/0 conn(0x7eff78152a90 0x7eff780730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:57:01.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff77e00640 1 --2- 172.21.15.113:0/1625762288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 0x7eff7815c130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7eff70009d90 tx=0x7eff7001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6252c7b8c9bda3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff76000640 1 -- 172.21.15.113:0/1625762288 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700b2070 con 0x7eff7815bce0 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff76000640 1 -- 172.21.15.113:0/1625762288 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff700a5830 con 0x7eff7815bce0 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff76000640 1 -- 172.21.15.113:0/1625762288 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700a5c00 con 0x7eff7815bce0 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff7d000640 1 -- 172.21.15.113:0/1625762288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 msgr2=0x7eff7815c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff7d000640 1 --2- 172.21.15.113:0/1625762288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 0x7eff7815c130 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7eff70009d90 tx=0x7eff7001c970 comp rx=0 tx=0).stop 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff7d000640 1 -- 172.21.15.113:0/1625762288 shutdown_connections 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff7d000640 1 --2- 172.21.15.113:0/1625762288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 0x7eff7815c130 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff7d000640 1 --2- 172.21.15.113:0/1625762288 >> v2:172.21.15.113:3300/0 conn(0x7eff78152a90 0x7eff780730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff7d000640 1 -- 172.21.15.113:0/1625762288 >> 172.21.15.113:0/1625762288 conn(0x7eff7806e2a0 msgr2=0x7eff7806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff7d000640 1 -- 172.21.15.113:0/1625762288 shutdown_connections 2024-03-04T07:57:01.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.395+0000 7eff7d000640 1 -- 172.21.15.113:0/1625762288 wait complete. 2024-03-04T07:57:01.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 Processor -- start 2024-03-04T07:57:01.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 -- start start 2024-03-04T07:57:01.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff7806cd10 0x7eff78144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff78152a90 0x7eff781f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 0x7eff781f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815ca80 con 0x7eff7806cd10 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff78071330 con 0x7eff78152a90 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff78165aa0 con 0x7eff7815bce0 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff7806cd10 0x7eff78144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff7806cd10 0x7eff78144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42824/0 (socket says 172.21.15.113:42824) 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77400640 1 -- 172.21.15.113:0/3990668643 learned_addr learned my addr 172.21.15.113:0/3990668643 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff76a00640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff78152a90 0x7eff781f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77e00640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 0x7eff781f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77400640 1 -- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 msgr2=0x7eff781f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77400640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 0x7eff781f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:01.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77400640 1 -- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff78152a90 msgr2=0x7eff781f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77400640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff78152a90 0x7eff781f6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77400640 1 -- 172.21.15.113:0/3990668643 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7814e460 con 0x7eff7806cd10 2024-03-04T07:57:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77e00640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 0x7eff781f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:57:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff77400640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff7806cd10 0x7eff78144820 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7eff6800e980 tx=0x7eff6800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff6f400640 1 -- 172.21.15.113:0/3990668643 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68017070 con 0x7eff7806cd10 2024-03-04T07:57:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff6f400640 1 -- 172.21.15.113:0/3990668643 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6800c9a0 con 0x7eff7806cd10 2024-03-04T07:57:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 -- 172.21.15.113:0/3990668643 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7814cd40 con 0x7eff7806cd10 2024-03-04T07:57:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff6f400640 1 -- 172.21.15.113:0/3990668643 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6800cd70 con 0x7eff7806cd10 2024-03-04T07:57:01.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 -- 172.21.15.113:0/3990668643 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff78162050 con 0x7eff7806cd10 2024-03-04T07:57:01.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.399+0000 7eff7d000640 1 -- 172.21.15.113:0/3990668643 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff78075a80 con 0x7eff7806cd10 2024-03-04T07:57:01.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.403+0000 7eff6f400640 1 -- 172.21.15.113:0/3990668643 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff6801a070 con 0x7eff7806cd10 2024-03-04T07:57:01.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.403+0000 7eff6f400640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff58066f60 0x7eff58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:01.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.403+0000 7eff6f400640 1 -- 172.21.15.113:0/3990668643 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7eff68085320 con 0x7eff7806cd10 2024-03-04T07:57:01.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.403+0000 7eff76a00640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff58066f60 0x7eff58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:01.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.403+0000 7eff76a00640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff58066f60 0x7eff58069420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7eff7815aef0 tx=0x7eff64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:01.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.411+0000 7eff6f400640 1 -- 172.21.15.113:0/3990668643 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7eff7814e460 con 0x7eff7806cd10 2024-03-04T07:57:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 -- 172.21.15.113:0/3990668643 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff780c6380 con 0x7eff7806cd10 2024-03-04T07:57:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff6f400640 1 -- 172.21.15.113:0/3990668643 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+6612 (secure 0 0 0) 0x7eff68055ef0 con 0x7eff7806cd10 2024-03-04T07:57:01.597 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:01.597 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:55:01.109616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6240,6145,5940]},"id":1}]} 2024-03-04T07:57:01.597 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 82 2024-03-04T07:57:01.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 -- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff58066f60 msgr2=0x7eff58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff58066f60 0x7eff58069420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7eff7815aef0 tx=0x7eff64009040 comp rx=0 tx=0).stop 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 -- 172.21.15.113:0/3990668643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff7806cd10 msgr2=0x7eff78144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff7806cd10 0x7eff78144820 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7eff6800e980 tx=0x7eff6800ee40 comp rx=0 tx=0).stop 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 -- 172.21.15.113:0/3990668643 shutdown_connections 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff7815bce0 0x7eff781f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff78152a90 0x7eff781f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff58066f60 0x7eff58069420 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 --2- 172.21.15.113:0/3990668643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff7806cd10 0x7eff78144820 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 -- 172.21.15.113:0/3990668643 >> 172.21.15.113:0/3990668643 conn(0x7eff7806e2a0 msgr2=0x7eff78148d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 -- 172.21.15.113:0/3990668643 shutdown_connections 2024-03-04T07:57:01.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:01.599+0000 7eff7d000640 1 -- 172.21.15.113:0/3990668643 wait complete. 2024-03-04T07:57:01.617 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:57:01.629 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:57:01.632 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:57:03.619 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:03.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.755+0000 7fd5e9a00640 1 Processor -- start 2024-03-04T07:57:03.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.755+0000 7fd5e9a00640 1 -- start start 2024-03-04T07:57:03.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e9a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd5e4152950 0x7fd5e415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:03.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e9a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5e40cb540 0x7fd5e40c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:03.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e9a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e40c4ce0 con 0x7fd5e406faa0 2024-03-04T07:57:03.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e9a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e40c8370 con 0x7fd5e40cb540 2024-03-04T07:57:03.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e9a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e40ca0a0 con 0x7fd5e4152950 2024-03-04T07:57:03.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e9000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5e40cb540 0x7fd5e40c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:03.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5dbe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd5e406faa0 0x7fd5e406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50174/0 (socket says 172.21.15.113:50174) 2024-03-04T07:57:03.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5dbe00640 1 -- 172.21.15.113:0/3739472084 learned_addr learned my addr 172.21.15.113:0/3739472084 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:57:03.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e3e00640 1 --2- 172.21.15.113:0/3739472084 >> v2:172.21.15.136:3300/0 conn(0x7fd5e4152950 0x7fd5e415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:03.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e9000640 1 -- 172.21.15.113:0/3739472084 >> v2:172.21.15.136:3300/0 conn(0x7fd5e4152950 msgr2=0x7fd5e415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:03.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e9000640 1 --2- 172.21.15.113:0/3739472084 >> v2:172.21.15.136:3300/0 conn(0x7fd5e4152950 0x7fd5e415bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:03.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e9000640 1 -- 172.21.15.113:0/3739472084 >> v1:172.21.15.113:6789/0 conn(0x7fd5e406faa0 legacy=0x7fd5e406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:03.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e9000640 1 -- 172.21.15.113:0/3739472084 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e40c4f80 con 0x7fd5e40cb540 2024-03-04T07:57:03.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.759+0000 7fd5e3400640 1 -- 172.21.15.113:0/3739472084 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698572279 0 0) 0x7fd5e40c4ce0 con 0x7fd5e406faa0 2024-03-04T07:57:03.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9000640 1 --2- 172.21.15.113:0/3739472084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 0x7fd5e40c30b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40ceb90 tx=0x7fd5dc01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929bb15c8cc5fc2a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:03.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e3400640 1 -- 172.21.15.113:0/3739472084 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0b4070 con 0x7fd5e40cb540 2024-03-04T07:57:03.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e3400640 1 -- 172.21.15.113:0/3739472084 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc0af600 con 0x7fd5e40cb540 2024-03-04T07:57:03.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e3400640 1 -- 172.21.15.113:0/3739472084 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc0af9d0 con 0x7fd5e40cb540 2024-03-04T07:57:03.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9a00640 1 -- 172.21.15.113:0/3739472084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 msgr2=0x7fd5e40c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:03.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9a00640 1 --2- 172.21.15.113:0/3739472084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 0x7fd5e40c30b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40ceb90 tx=0x7fd5dc01f110 comp rx=0 tx=0).stop 2024-03-04T07:57:03.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9a00640 1 -- 172.21.15.113:0/3739472084 shutdown_connections 2024-03-04T07:57:03.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9a00640 1 --2- 172.21.15.113:0/3739472084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 0x7fd5e40c30b0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:03.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9a00640 1 --2- 172.21.15.113:0/3739472084 >> v2:172.21.15.136:3300/0 conn(0x7fd5e4152950 0x7fd5e415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:03.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9a00640 1 -- 172.21.15.113:0/3739472084 >> 172.21.15.113:0/3739472084 conn(0x7fd5e4072f40 msgr2=0x7fd5e40706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:03.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9a00640 1 -- 172.21.15.113:0/3739472084 shutdown_connections 2024-03-04T07:57:03.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9a00640 1 -- 172.21.15.113:0/3739472084 wait complete. 2024-03-04T07:57:03.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9a00640 1 Processor -- start 2024-03-04T07:57:03.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.763+0000 7fd5e9a00640 1 -- start start 2024-03-04T07:57:03.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e9a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5e406faa0 0x7fd5e4067010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:03.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 0x7fd5e41546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:03.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e9a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4152950 0x7fd5e4154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:03.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e9a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40c9a50 con 0x7fd5e40cb540 2024-03-04T07:57:03.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e9a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e406fe80 con 0x7fd5e4152950 2024-03-04T07:57:03.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e9a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40c9d30 con 0x7fd5e406faa0 2024-03-04T07:57:03.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 0x7fd5e41546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:03.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5e406faa0 0x7fd5e4067010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 0x7fd5e41546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42836/0 (socket says 172.21.15.113:42836) 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5dbe00640 1 -- 172.21.15.113:0/1676040566 learned_addr learned my addr 172.21.15.113:0/1676040566 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e9000640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4152950 0x7fd5e4154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5dbe00640 1 -- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5e406faa0 msgr2=0x7fd5e4067010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5dbe00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5e406faa0 0x7fd5e4067010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5dbe00640 1 -- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4152950 msgr2=0x7fd5e4154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5dbe00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4152950 0x7fd5e4154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5dbe00640 1 -- 172.21.15.113:0/1676040566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e40b2df0 con 0x7fd5e40cb540 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5dbe00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 0x7fd5e41546e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00c480 tx=0x7fd5cc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e0c00640 1 -- 172.21.15.113:0/1676040566 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc018070 con 0x7fd5e40cb540 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e0c00640 1 -- 172.21.15.113:0/1676040566 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5cc007500 con 0x7fd5e40cb540 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e0c00640 1 -- 172.21.15.113:0/1676040566 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5cc004a30 con 0x7fd5e40cb540 2024-03-04T07:57:03.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e9a00640 1 -- 172.21.15.113:0/1676040566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e4153f50 con 0x7fd5e40cb540 2024-03-04T07:57:03.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e9a00640 1 -- 172.21.15.113:0/1676040566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e415b970 con 0x7fd5e40cb540 2024-03-04T07:57:03.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e9a00640 1 -- 172.21.15.113:0/1676040566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e4075a80 con 0x7fd5e40cb540 2024-03-04T07:57:03.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.767+0000 7fd5e0c00640 1 -- 172.21.15.113:0/1676040566 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5cc004bd0 con 0x7fd5e40cb540 2024-03-04T07:57:03.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.771+0000 7fd5e0c00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5c0066f60 0x7fd5c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:03.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.771+0000 7fd5e0c00640 1 -- 172.21.15.113:0/1676040566 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6771+0+0 (secure 0 0 0) 0x7fd5cc084670 con 0x7fd5e40cb540 2024-03-04T07:57:03.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.771+0000 7fd5e3e00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5c0066f60 0x7fd5c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:03.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.771+0000 7fd5e3e00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5c0066f60 0x7fd5c0069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009b60 tx=0x7fd5d4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:03.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.775+0000 7fd5e0c00640 1 -- 172.21.15.113:0/1676040566 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd5cc054c90 con 0x7fd5e40cb540 2024-03-04T07:57:03.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 -- 172.21.15.113:0/1676040566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5e40c9580 con 0x7fd5e40cb540 2024-03-04T07:57:03.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e0c00640 1 -- 172.21.15.113:0/1676040566 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+6612 (secure 0 0 0) 0x7fd5cc058d30 con 0x7fd5e40cb540 2024-03-04T07:57:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:03.960 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6372,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:55:01.109616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5940,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5940":{"gid":5940,"name":"e","rank":0,"incarnation":64,"state":"up:active","state_seq":30,"addr":"172.21.15.113:6837/344022786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":344022786},{"type":"v1","addr":"172.21.15.113:6837","nonce":344022786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5940,"qdb_cluster":[6240,6145,5940]},"id":1}]} 2024-03-04T07:57:03.961 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 82 2024-03-04T07:57:03.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 -- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5c0066f60 msgr2=0x7fd5c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:03.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5c0066f60 0x7fd5c0069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009b60 tx=0x7fd5d4009190 comp rx=0 tx=0).stop 2024-03-04T07:57:03.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 -- 172.21.15.113:0/1676040566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 msgr2=0x7fd5e41546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:03.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 0x7fd5e41546e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd5cc00c480 tx=0x7fd5cc00c940 comp rx=0 tx=0).stop 2024-03-04T07:57:03.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 -- 172.21.15.113:0/1676040566 shutdown_connections 2024-03-04T07:57:03.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5e4152950 0x7fd5e4154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:03.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5c0066f60 0x7fd5c0069420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:03.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5e40cb540 0x7fd5e41546e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:03.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 --2- 172.21.15.113:0/1676040566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5e406faa0 0x7fd5e4067010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:03.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 -- 172.21.15.113:0/1676040566 >> 172.21.15.113:0/1676040566 conn(0x7fd5e4072f40 msgr2=0x7fd5e4158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:03.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 -- 172.21.15.113:0/1676040566 shutdown_connections 2024-03-04T07:57:03.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:03.963+0000 7fd5e9a00640 1 -- 172.21.15.113:0/1676040566 wait complete. 2024-03-04T07:57:03.983 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T07:57:05.984 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:06.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a18a00640 1 Processor -- start 2024-03-04T07:57:06.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a18a00640 1 -- start start 2024-03-04T07:57:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a18a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 0x7f1a140c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a18a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1a140cd650 0x7f1a14151290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a18a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a14157830 con 0x7f1a140c7a80 2024-03-04T07:57:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a18a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a14157af0 con 0x7f1a140cd650 2024-03-04T07:57:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a18a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a14157dd0 con 0x7f1a140cd270 2024-03-04T07:57:06.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a11e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 0x7f1a140c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a0be00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1a140cd650 0x7f1a14151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a11e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 0x7f1a140c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:60324/0 (socket says 172.21.15.113:60324) 2024-03-04T07:57:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.123+0000 7f1a11e00640 1 -- 172.21.15.113:0/327324107 learned_addr learned my addr 172.21.15.113:0/327324107 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.127+0000 7f1a0be00640 1 -- 172.21.15.113:0/327324107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 msgr2=0x7f1a140c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.127+0000 7f1a0be00640 1 --2- 172.21.15.113:0/327324107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 0x7f1a140c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.127+0000 7f1a0be00640 1 -- 172.21.15.113:0/327324107 >> v1:172.21.15.136:6789/0 conn(0x7f1a140c7a80 legacy=0x7f1a1415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.127+0000 7f1a0be00640 1 -- 172.21.15.113:0/327324107 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a141580b0 con 0x7f1a140cd650 2024-03-04T07:57:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.127+0000 7f1a0be00640 1 --2- 172.21.15.113:0/327324107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a140cd650 0x7f1a14151290 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0097c0 tx=0x7f19fc01e670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e9477cdd0503f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:06.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a11400640 1 -- 172.21.15.113:0/327324107 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0b2070 con 0x7f1a140cd650 2024-03-04T07:57:06.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a11400640 1 -- 172.21.15.113:0/327324107 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc004230 con 0x7f1a140cd650 2024-03-04T07:57:06.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a11400640 1 -- 172.21.15.113:0/327324107 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc003620 con 0x7f1a140cd650 2024-03-04T07:57:06.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a18a00640 1 -- 172.21.15.113:0/327324107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a140cd650 msgr2=0x7f1a14151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a18a00640 1 --2- 172.21.15.113:0/327324107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a140cd650 0x7f1a14151290 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f19fc0097c0 tx=0x7f19fc01e670 comp rx=0 tx=0).stop 2024-03-04T07:57:06.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a18a00640 1 -- 172.21.15.113:0/327324107 shutdown_connections 2024-03-04T07:57:06.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a18a00640 1 --2- 172.21.15.113:0/327324107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a140cd650 0x7f1a14151290 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a18a00640 1 --2- 172.21.15.113:0/327324107 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 0x7f1a140c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a18a00640 1 -- 172.21.15.113:0/327324107 >> 172.21.15.113:0/327324107 conn(0x7f1a140671b0 msgr2=0x7f1a1406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:06.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a18a00640 1 -- 172.21.15.113:0/327324107 shutdown_connections 2024-03-04T07:57:06.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.131+0000 7f1a18a00640 1 -- 172.21.15.113:0/327324107 wait complete. 2024-03-04T07:57:06.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a18a00640 1 Processor -- start 2024-03-04T07:57:06.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a18a00640 1 -- start start 2024-03-04T07:57:06.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a18a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a140c7a80 0x7f1a140cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a18a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 0x7f1a14146d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a18a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a140cd650 0x7f1a14142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a18a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a14151ba0 con 0x7f1a140cd650 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a18a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a14151a20 con 0x7f1a140c7a80 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a18a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1406b870 con 0x7f1a140cd270 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a12800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a140cd650 0x7f1a14142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a0be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 0x7f1a14146d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a12800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a140cd650 0x7f1a14142d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42838/0 (socket says 172.21.15.113:42838) 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a12800640 1 -- 172.21.15.113:0/3909979079 learned_addr learned my addr 172.21.15.113:0/3909979079 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a11e00640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a140c7a80 0x7f1a140cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a12800640 1 -- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 msgr2=0x7f1a14146d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a12800640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 0x7f1a14146d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a12800640 1 -- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a140c7a80 msgr2=0x7f1a140cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a12800640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a140c7a80 0x7f1a140cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a12800640 1 -- 172.21.15.113:0/3909979079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a141f7a50 con 0x7f1a140cd650 2024-03-04T07:57:06.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a12800640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a140cd650 0x7f1a14142d90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00c7e0 tx=0x7f1a0c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:06.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a0a000640 1 -- 172.21.15.113:0/3909979079 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c016020 con 0x7f1a140cd650 2024-03-04T07:57:06.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a0a000640 1 -- 172.21.15.113:0/3909979079 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c00a9f0 con 0x7f1a140cd650 2024-03-04T07:57:06.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a0a000640 1 -- 172.21.15.113:0/3909979079 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c00ade0 con 0x7f1a140cd650 2024-03-04T07:57:06.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a18a00640 1 -- 172.21.15.113:0/3909979079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a141f7d30 con 0x7f1a140cd650 2024-03-04T07:57:06.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.135+0000 7f1a18a00640 1 -- 172.21.15.113:0/3909979079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a140cb160 con 0x7f1a140cd650 2024-03-04T07:57:06.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.139+0000 7f1a18a00640 1 -- 172.21.15.113:0/3909979079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a14075c00 con 0x7f1a140cd650 2024-03-04T07:57:06.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.139+0000 7f1a0a000640 1 -- 172.21.15.113:0/3909979079 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a0c019020 con 0x7f1a140cd650 2024-03-04T07:57:06.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.139+0000 7f1a0a000640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19f0066f90 0x7f19f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.139+0000 7f1a0a000640 1 -- 172.21.15.113:0/3909979079 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f1a0c085400 con 0x7f1a140cd650 2024-03-04T07:57:06.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.139+0000 7f1a11e00640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19f0066f90 0x7f19f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.139+0000 7f1a11e00640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19f0066f90 0x7f19f0069450 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009b50 tx=0x7f1a04009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:06.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.143+0000 7f1a0a000640 1 -- 172.21.15.113:0/3909979079 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1a0c020030 con 0x7f1a140cd650 2024-03-04T07:57:06.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.331+0000 7f1a18a00640 1 -- 172.21.15.113:0/3909979079 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a14157540 con 0x7f1a140cd650 2024-03-04T07:57:06.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a0a000640 1 -- 172.21.15.113:0/3909979079 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f1a0c055810 con 0x7f1a140cd650 2024-03-04T07:57:06.332 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:06.333 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:05.766686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6145,6240]},"id":1}]} 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 83 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 -- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19f0066f90 msgr2=0x7f19f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19f0066f90 0x7f19f0069450 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009b50 tx=0x7f1a04009240 comp rx=0 tx=0).stop 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 -- 172.21.15.113:0/3909979079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a140cd650 msgr2=0x7f1a14142d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a140cd650 0x7f1a14142d90 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00c7e0 tx=0x7f1a0c00cca0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 -- 172.21.15.113:0/3909979079 shutdown_connections 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19f0066f90 0x7f19f0069450 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a140cd650 0x7f1a14142d90 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a140cd270 0x7f1a14146d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 --2- 172.21.15.113:0/3909979079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a140c7a80 0x7f1a140cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 -- 172.21.15.113:0/3909979079 >> 172.21.15.113:0/3909979079 conn(0x7f1a140671b0 msgr2=0x7f1a141443e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:06.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 -- 172.21.15.113:0/3909979079 shutdown_connections 2024-03-04T07:57:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.335+0000 7f1a18a00640 1 -- 172.21.15.113:0/3909979079 wait complete. 2024-03-04T07:57:06.353 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-03-04T07:57:06.353 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:57:06.353 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:06.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dfa00640 1 Processor -- start 2024-03-04T07:57:06.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dfa00640 1 -- start start 2024-03-04T07:57:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dfa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01d8159470 0x7f01d80c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dfa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d8159980 con 0x7f01d8152d60 2024-03-04T07:57:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dfa00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d8153140 con 0x7f01d80c38c0 2024-03-04T07:57:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dfa00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d815a5b0 con 0x7f01d8159470 2024-03-04T07:57:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dda00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01d8159470 0x7f01d80c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dda00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01d8159470 0x7f01d80c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42854/0 (socket says 172.21.15.113:42854) 2024-03-04T07:57:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dda00640 1 -- 172.21.15.113:0/1205085235 learned_addr learned my addr 172.21.15.113:0/1205085235 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dd000640 1 --1- 172.21.15.113:0/1205085235 >> v1:172.21.15.113:6789/0 conn(0x7f01d8152d60 0x7f01d8159ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50186/0 (socket says 172.21.15.113:50186) 2024-03-04T07:57:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dda00640 1 -- 172.21.15.113:0/1205085235 >> v1:172.21.15.136:6789/0 conn(0x7f01d80c38c0 legacy=0x7f01d80ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dda00640 1 -- 172.21.15.113:0/1205085235 >> v1:172.21.15.113:6789/0 conn(0x7f01d8152d60 legacy=0x7f01d8159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.491+0000 7f01dda00640 1 -- 172.21.15.113:0/1205085235 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80c5390 con 0x7f01d8159470 2024-03-04T07:57:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01dda00640 1 --2- 172.21.15.113:0/1205085235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 0x7f01d80c8d80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009e50 tx=0x7f01d001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632fa9080f47a24c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01d7400640 1 -- 172.21.15.113:0/1205085235 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00a5700 con 0x7f01d8159470 2024-03-04T07:57:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01d7400640 1 -- 172.21.15.113:0/1205085235 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d00a58a0 con 0x7f01d8159470 2024-03-04T07:57:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01d7400640 1 -- 172.21.15.113:0/1205085235 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00a5c70 con 0x7f01d8159470 2024-03-04T07:57:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01dfa00640 1 -- 172.21.15.113:0/1205085235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 msgr2=0x7f01d80c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01dfa00640 1 --2- 172.21.15.113:0/1205085235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 0x7f01d80c8d80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009e50 tx=0x7f01d001ca30 comp rx=0 tx=0).stop 2024-03-04T07:57:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01dfa00640 1 -- 172.21.15.113:0/1205085235 shutdown_connections 2024-03-04T07:57:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01dfa00640 1 --2- 172.21.15.113:0/1205085235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 0x7f01d80c8d80 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01dfa00640 1 -- 172.21.15.113:0/1205085235 >> 172.21.15.113:0/1205085235 conn(0x7f01d8066db0 msgr2=0x7f01d80671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01dfa00640 1 -- 172.21.15.113:0/1205085235 shutdown_connections 2024-03-04T07:57:06.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.495+0000 7f01dfa00640 1 -- 172.21.15.113:0/1205085235 wait complete. 2024-03-04T07:57:06.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dfa00640 1 Processor -- start 2024-03-04T07:57:06.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dfa00640 1 -- start start 2024-03-04T07:57:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dfa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01d80c38c0 0x7f01d80cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dfa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01d8152d60 0x7f01d814ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dfa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 0x7f01d814bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dfa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8070230 con 0x7f01d8159470 2024-03-04T07:57:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dfa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80c5130 con 0x7f01d80c38c0 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dfa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80703b0 con 0x7f01d8152d60 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 0x7f01d814bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 0x7f01d814bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42860/0 (socket says 172.21.15.113:42860) 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dda00640 1 -- 172.21.15.113:0/1473555661 learned_addr learned my addr 172.21.15.113:0/1473555661 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dd000640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01d80c38c0 0x7f01d80cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01d7e00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01d8152d60 0x7f01d814ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dda00640 1 -- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01d8152d60 msgr2=0x7f01d814ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dda00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01d8152d60 0x7f01d814ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dda00640 1 -- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01d80c38c0 msgr2=0x7f01d80cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dda00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01d80c38c0 0x7f01d80cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dda00640 1 -- 172.21.15.113:0/1473555661 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80c9e70 con 0x7f01d8159470 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dd000640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01d80c38c0 0x7f01d80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dda00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 0x7f01d814bf50 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f01d0019640 tx=0x7f01d00197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01d4c00640 1 -- 172.21.15.113:0/1473555661 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00bc020 con 0x7f01d8159470 2024-03-04T07:57:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01d4c00640 1 -- 172.21.15.113:0/1473555661 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d00b8670 con 0x7f01d8159470 2024-03-04T07:57:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dfa00640 1 -- 172.21.15.113:0/1473555661 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d80ca110 con 0x7f01d8159470 2024-03-04T07:57:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01d4c00640 1 -- 172.21.15.113:0/1473555661 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00b8a60 con 0x7f01d8159470 2024-03-04T07:57:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.499+0000 7f01dfa00640 1 -- 172.21.15.113:0/1473555661 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d81505a0 con 0x7f01d8159470 2024-03-04T07:57:06.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.503+0000 7f01dfa00640 1 -- 172.21.15.113:0/1473555661 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d8068eb0 con 0x7f01d8159470 2024-03-04T07:57:06.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.503+0000 7f01d4c00640 1 -- 172.21.15.113:0/1473555661 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01d00bf020 con 0x7f01d8159470 2024-03-04T07:57:06.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.503+0000 7f01d4c00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01b8066f60 0x7f01b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:06.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.503+0000 7f01d4c00640 1 -- 172.21.15.113:0/1473555661 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f01d01206d0 con 0x7f01d8159470 2024-03-04T07:57:06.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.503+0000 7f01dd000640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01b8066f60 0x7f01b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:06.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.503+0000 7f01dd000640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01b8066f60 0x7f01b8069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f01cc000f70 tx=0x7f01cc007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:06.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.507+0000 7f01d4c00640 1 -- 172.21.15.113:0/1473555661 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f01d00c6030 con 0x7f01d8159470 2024-03-04T07:57:06.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.695+0000 7f01dfa00640 1 -- 172.21.15.113:0/1473555661 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01d8150c60 con 0x7f01d8159470 2024-03-04T07:57:06.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.695+0000 7f01d4c00640 1 -- 172.21.15.113:0/1473555661 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f01d8150c60 con 0x7f01d8159470 2024-03-04T07:57:06.693 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:06.693 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:05.766686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6145,6240]},"id":1}]} 2024-03-04T07:57:06.693 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 83 2024-03-04T07:57:06.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 -- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01b8066f60 msgr2=0x7f01b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01b8066f60 0x7f01b8069420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f01cc000f70 tx=0x7f01cc007d50 comp rx=0 tx=0).stop 2024-03-04T07:57:06.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 -- 172.21.15.113:0/1473555661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 msgr2=0x7f01d814bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:06.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 0x7f01d814bf50 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f01d0019640 tx=0x7f01d00197d0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 -- 172.21.15.113:0/1473555661 shutdown_connections 2024-03-04T07:57:06.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01b8066f60 0x7f01b8069420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01d8159470 0x7f01d814bf50 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01d8152d60 0x7f01d814ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 --2- 172.21.15.113:0/1473555661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01d80c38c0 0x7f01d80cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:06.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 -- 172.21.15.113:0/1473555661 >> 172.21.15.113:0/1473555661 conn(0x7f01d8066db0 msgr2=0x7f01d815a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:06.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 -- 172.21.15.113:0/1473555661 shutdown_connections 2024-03-04T07:57:06.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:06.699+0000 7f01dfa00640 1 -- 172.21.15.113:0/1473555661 wait complete. 2024-03-04T07:57:06.718 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 83, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 83, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6145": { "addr": "172.21.15.136:6835/2368734308", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2368734308, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2368734308, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6145, "incarnation": 71, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_6240": { "addr": "172.21.15.113:6836/986515123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 986515123, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 986515123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6240, "incarnation": 77, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6372": { "addr": "172.21.15.113:6833/2732565639", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2732565639, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2732565639, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6372, "incarnation": 83, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 60, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:57:05.766686+0000", "qdb_cluster": [ 6145, 6240 ], "qdb_leader": 6145, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6372, "mds_1": 6145, "mds_2": 6240 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:57:08.720 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:08.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.859+0000 7f7e47200640 1 Processor -- start 2024-03-04T07:57:08.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.859+0000 7f7e47200640 1 -- start start 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.859+0000 7f7e47200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7e4815af20 0x7f7e48156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.859+0000 7f7e47200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7e48157230 0x7f7e4806c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.859+0000 7f7e47200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e480c9c30 con 0x7f7e48151030 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.859+0000 7f7e47200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4815aa90 con 0x7f7e4815af20 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.859+0000 7f7e47200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e481577a0 con 0x7f7e48157230 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e45400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7e4815af20 0x7f7e48156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e45400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7e4815af20 0x7f7e48156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52730/0 (socket says 172.21.15.113:52730) 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e45400640 1 -- 172.21.15.113:0/2671674950 learned_addr learned my addr 172.21.15.113:0/2671674950 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e46800640 1 --2- 172.21.15.113:0/2671674950 >> v2:172.21.15.136:3300/0 conn(0x7f7e48157230 0x7f7e4806c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e45400640 1 -- 172.21.15.113:0/2671674950 >> v2:172.21.15.136:3300/0 conn(0x7f7e48157230 msgr2=0x7f7e4806c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e45400640 1 --2- 172.21.15.113:0/2671674950 >> v2:172.21.15.136:3300/0 conn(0x7f7e48157230 0x7f7e4806c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e45400640 1 -- 172.21.15.113:0/2671674950 >> v1:172.21.15.136:6789/0 conn(0x7f7e48151030 legacy=0x7f7e480c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e45400640 1 -- 172.21.15.113:0/2671674950 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48157a80 con 0x7f7e4815af20 2024-03-04T07:57:08.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e45400640 1 --2- 172.21.15.113:0/2671674950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e4815af20 0x7f7e48156cf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7e48072a50 tx=0x7f7e3401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ad45eb1ce028ef server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:08.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e44a00640 1 -- 172.21.15.113:0/2671674950 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e340aa650 con 0x7f7e4815af20 2024-03-04T07:57:08.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e44a00640 1 -- 172.21.15.113:0/2671674950 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e340af070 con 0x7f7e4815af20 2024-03-04T07:57:08.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.863+0000 7f7e44a00640 1 -- 172.21.15.113:0/2671674950 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e340aaa10 con 0x7f7e4815af20 2024-03-04T07:57:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.867+0000 7f7e47200640 1 -- 172.21.15.113:0/2671674950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e4815af20 msgr2=0x7f7e48156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.867+0000 7f7e47200640 1 --2- 172.21.15.113:0/2671674950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e4815af20 0x7f7e48156cf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7e48072a50 tx=0x7f7e3401c4f0 comp rx=0 tx=0).stop 2024-03-04T07:57:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.867+0000 7f7e47200640 1 -- 172.21.15.113:0/2671674950 shutdown_connections 2024-03-04T07:57:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.867+0000 7f7e47200640 1 --2- 172.21.15.113:0/2671674950 >> v2:172.21.15.136:3300/0 conn(0x7f7e48157230 0x7f7e4806c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.867+0000 7f7e47200640 1 --2- 172.21.15.113:0/2671674950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e4815af20 0x7f7e48156cf0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.867+0000 7f7e47200640 1 -- 172.21.15.113:0/2671674950 >> 172.21.15.113:0/2671674950 conn(0x7f7e4806e9a0 msgr2=0x7f7e4806edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.867+0000 7f7e47200640 1 -- 172.21.15.113:0/2671674950 shutdown_connections 2024-03-04T07:57:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.867+0000 7f7e47200640 1 -- 172.21.15.113:0/2671674950 wait complete. 2024-03-04T07:57:08.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.867+0000 7f7e47200640 1 Processor -- start 2024-03-04T07:57:08.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.867+0000 7f7e47200640 1 -- start start 2024-03-04T07:57:08.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e47200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e48151030 0x7f7e481463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:08.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e47200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e48157230 0x7f7e48143b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e47200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e4815af20 0x7f7e48159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e47200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e481617d0 con 0x7f7e48151030 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e47200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48161650 con 0x7f7e48157230 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e47200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48161950 con 0x7f7e4815af20 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e48151030 0x7f7e481463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e48151030 0x7f7e481463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52740/0 (socket says 172.21.15.113:52740) 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45e00640 1 -- 172.21.15.113:0/241929996 learned_addr learned my addr 172.21.15.113:0/241929996 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45400640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e48157230 0x7f7e48143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e46800640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e4815af20 0x7f7e48159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45e00640 1 -- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e4815af20 msgr2=0x7f7e48159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45e00640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e4815af20 0x7f7e48159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45e00640 1 -- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e48157230 msgr2=0x7f7e48143b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45e00640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e48157230 0x7f7e48143b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45e00640 1 -- 172.21.15.113:0/241929996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48158d90 con 0x7f7e48151030 2024-03-04T07:57:08.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45400640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e48157230 0x7f7e48143b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:57:08.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e45e00640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e48151030 0x7f7e481463b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800c450 tx=0x7f7e3800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:08.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e3e000640 1 -- 172.21.15.113:0/241929996 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380052d0 con 0x7f7e48151030 2024-03-04T07:57:08.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e3e000640 1 -- 172.21.15.113:0/241929996 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e38005470 con 0x7f7e48151030 2024-03-04T07:57:08.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e3e000640 1 -- 172.21.15.113:0/241929996 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38007260 con 0x7f7e48151030 2024-03-04T07:57:08.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e47200640 1 -- 172.21.15.113:0/241929996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e481f4e90 con 0x7f7e48151030 2024-03-04T07:57:08.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e47200640 1 -- 172.21.15.113:0/241929996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e481f40d0 con 0x7f7e48151030 2024-03-04T07:57:08.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.871+0000 7f7e47200640 1 -- 172.21.15.113:0/241929996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e481f49a0 con 0x7f7e48151030 2024-03-04T07:57:08.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.875+0000 7f7e3e000640 1 -- 172.21.15.113:0/241929996 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e38007400 con 0x7f7e48151030 2024-03-04T07:57:08.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.875+0000 7f7e3e000640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7e28066fe0 0x7f7e280694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:08.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.875+0000 7f7e3e000640 1 -- 172.21.15.113:0/241929996 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f7e38087360 con 0x7f7e48151030 2024-03-04T07:57:08.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.875+0000 7f7e45400640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7e28066fe0 0x7f7e280694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:08.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.875+0000 7f7e45400640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7e28066fe0 0x7f7e280694a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e340ae000 tx=0x7f7e340a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:08.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:08.879+0000 7f7e3e000640 1 -- 172.21.15.113:0/241929996 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7e3801f020 con 0x7f7e48151030 2024-03-04T07:57:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 -- 172.21.15.113:0/241929996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e480c8620 con 0x7f7e48151030 2024-03-04T07:57:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e3e000640 1 -- 172.21.15.113:0/241929996 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f7e48158d90 con 0x7f7e48151030 2024-03-04T07:57:09.064 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:09.065 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:05.766686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6145,6240]},"id":1}]} 2024-03-04T07:57:09.065 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 83 2024-03-04T07:57:09.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 -- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7e28066fe0 msgr2=0x7f7e280694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:09.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7e28066fe0 0x7f7e280694a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e340ae000 tx=0x7f7e340a7000 comp rx=0 tx=0).stop 2024-03-04T07:57:09.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 -- 172.21.15.113:0/241929996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e48151030 msgr2=0x7f7e481463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:09.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e48151030 0x7f7e481463b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800c450 tx=0x7f7e3800c910 comp rx=0 tx=0).stop 2024-03-04T07:57:09.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 -- 172.21.15.113:0/241929996 shutdown_connections 2024-03-04T07:57:09.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7e4815af20 0x7f7e48159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:09.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7e48157230 0x7f7e48143b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:09.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7e28066fe0 0x7f7e280694a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:09.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 --2- 172.21.15.113:0/241929996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e48151030 0x7f7e481463b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:09.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 -- 172.21.15.113:0/241929996 >> 172.21.15.113:0/241929996 conn(0x7f7e4806e9a0 msgr2=0x7f7e480c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:09.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 -- 172.21.15.113:0/241929996 shutdown_connections 2024-03-04T07:57:09.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:09.067+0000 7f7e47200640 1 -- 172.21.15.113:0/241929996 wait complete. 2024-03-04T07:57:09.086 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:57:11.087 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:11.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.235+0000 7fc9b7e00640 1 Processor -- start 2024-03-04T07:57:11.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.235+0000 7fc9b7e00640 1 -- start start 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.235+0000 7fc9b7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9b01527f0 0x7fc9b0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.235+0000 7fc9b7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 0x7fc9b01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.235+0000 7fc9b7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b01591f0 con 0x7fc9b00c8200 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.235+0000 7fc9b7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b0071df0 con 0x7fc9b01527f0 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.235+0000 7fc9b7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01597b0 con 0x7fc9b0152bd0 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.239+0000 7fc9b5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9b01527f0 0x7fc9b0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.239+0000 7fc9b4a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc9b00c8200 0x7fc9b0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53922/0 (socket says 172.21.15.113:53922) 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.239+0000 7fc9b4a00640 1 -- 172.21.15.113:0/2844632378 learned_addr learned my addr 172.21.15.113:0/2844632378 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.239+0000 7fc9b5e00640 1 --2- 172.21.15.113:0/2844632378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 0x7fc9b01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.239+0000 7fc9b5400640 1 -- 172.21.15.113:0/2844632378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 msgr2=0x7fc9b01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.239+0000 7fc9b5400640 1 --2- 172.21.15.113:0/2844632378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 0x7fc9b01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.239+0000 7fc9b5400640 1 -- 172.21.15.113:0/2844632378 >> v1:172.21.15.113:6789/0 conn(0x7fc9b00c8200 legacy=0x7fc9b0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.239+0000 7fc9b5400640 1 -- 172.21.15.113:0/2844632378 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b01594d0 con 0x7fc9b01527f0 2024-03-04T07:57:11.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9b5400640 1 --2- 172.21.15.113:0/2844632378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b01527f0 0x7fc9b0070d40 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40097c0 tx=0x7fc9a401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b392a4a672a79104 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:11.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9afe00640 1 -- 172.21.15.113:0/2844632378 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40b0070 con 0x7fc9b01527f0 2024-03-04T07:57:11.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9afe00640 1 -- 172.21.15.113:0/2844632378 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a40ab600 con 0x7fc9b01527f0 2024-03-04T07:57:11.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9afe00640 1 -- 172.21.15.113:0/2844632378 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40ab9d0 con 0x7fc9b01527f0 2024-03-04T07:57:11.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9b7e00640 1 -- 172.21.15.113:0/2844632378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b01527f0 msgr2=0x7fc9b0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:11.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9b7e00640 1 --2- 172.21.15.113:0/2844632378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b01527f0 0x7fc9b0070d40 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40097c0 tx=0x7fc9a401c330 comp rx=0 tx=0).stop 2024-03-04T07:57:11.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9b7e00640 1 -- 172.21.15.113:0/2844632378 shutdown_connections 2024-03-04T07:57:11.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9b7e00640 1 --2- 172.21.15.113:0/2844632378 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 0x7fc9b01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:11.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9b7e00640 1 --2- 172.21.15.113:0/2844632378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b01527f0 0x7fc9b0070d40 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:11.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9b7e00640 1 -- 172.21.15.113:0/2844632378 >> 172.21.15.113:0/2844632378 conn(0x7fc9b006e820 msgr2=0x7fc9b006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:11.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9b7e00640 1 -- 172.21.15.113:0/2844632378 shutdown_connections 2024-03-04T07:57:11.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.243+0000 7fc9b7e00640 1 -- 172.21.15.113:0/2844632378 wait complete. 2024-03-04T07:57:11.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 Processor -- start 2024-03-04T07:57:11.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 -- start start 2024-03-04T07:57:11.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c8200 0x7fc9b00b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:11.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9b01527f0 0x7fc9b01f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:11.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 0x7fc9b01f1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0158d10 con 0x7fc9b00c8200 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0071a80 con 0x7fc9b01527f0 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0158e90 con 0x7fc9b0152bd0 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c8200 0x7fc9b00b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c8200 0x7fc9b00b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52780/0 (socket says 172.21.15.113:52780) 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5400640 1 -- 172.21.15.113:0/3039609339 learned_addr learned my addr 172.21.15.113:0/3039609339 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b4a00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9b01527f0 0x7fc9b01f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5e00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 0x7fc9b01f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5400640 1 -- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 msgr2=0x7fc9b01f1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5400640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 0x7fc9b01f1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5400640 1 -- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9b01527f0 msgr2=0x7fc9b01f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5400640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9b01527f0 0x7fc9b01f0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5400640 1 -- 172.21.15.113:0/3039609339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b014e2e0 con 0x7fc9b00c8200 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5e00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 0x7fc9b01f1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:57:11.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b5400640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c8200 0x7fc9b00b0e30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc9a401a9e0 tx=0x7fc9a40190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9ad600640 1 -- 172.21.15.113:0/3039609339 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40b0040 con 0x7fc9b00c8200 2024-03-04T07:57:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9ad600640 1 -- 172.21.15.113:0/3039609339 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a40b9070 con 0x7fc9b00c8200 2024-03-04T07:57:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9ad600640 1 -- 172.21.15.113:0/3039609339 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40aa9f0 con 0x7fc9b00c8200 2024-03-04T07:57:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 -- 172.21.15.113:0/3039609339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b014e040 con 0x7fc9b00c8200 2024-03-04T07:57:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 -- 172.21.15.113:0/3039609339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b014eaf0 con 0x7fc9b00c8200 2024-03-04T07:57:11.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.247+0000 7fc9b7e00640 1 -- 172.21.15.113:0/3039609339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b0075c00 con 0x7fc9b00c8200 2024-03-04T07:57:11.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.251+0000 7fc9ad600640 1 -- 172.21.15.113:0/3039609339 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9a40aab90 con 0x7fc9b00c8200 2024-03-04T07:57:11.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.251+0000 7fc9ad600640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f60 0x7fc990069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:11.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.251+0000 7fc9ad600640 1 -- 172.21.15.113:0/3039609339 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fc9a411f680 con 0x7fc9b00c8200 2024-03-04T07:57:11.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.251+0000 7fc9b4a00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f60 0x7fc990069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:11.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.255+0000 7fc9b4a00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f60 0x7fc990069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009fd0 tx=0x7fc99c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:11.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.255+0000 7fc9ad600640 1 -- 172.21.15.113:0/3039609339 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc9a4015fb0 con 0x7fc9b00c8200 2024-03-04T07:57:11.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.451+0000 7fc9b7e00640 1 -- 172.21.15.113:0/3039609339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9b00732f0 con 0x7fc9b00c8200 2024-03-04T07:57:11.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.451+0000 7fc9ad600640 1 -- 172.21.15.113:0/3039609339 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5815 (secure 0 0 0) 0x7fc9a40f0170 con 0x7fc9b00c8200 2024-03-04T07:57:11.451 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:11.451 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:05.766686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6145,6240]},"id":1}]} 2024-03-04T07:57:11.451 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 83 2024-03-04T07:57:11.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 -- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f60 msgr2=0x7fc990069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:11.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f60 0x7fc990069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009fd0 tx=0x7fc99c00b040 comp rx=0 tx=0).stop 2024-03-04T07:57:11.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 -- 172.21.15.113:0/3039609339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c8200 msgr2=0x7fc9b00b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:11.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c8200 0x7fc9b00b0e30 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc9a401a9e0 tx=0x7fc9a40190d0 comp rx=0 tx=0).stop 2024-03-04T07:57:11.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 -- 172.21.15.113:0/3039609339 shutdown_connections 2024-03-04T07:57:11.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b0152bd0 0x7fc9b01f1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:11.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9b01527f0 0x7fc9b01f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:11.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f60 0x7fc990069420 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:11.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 --2- 172.21.15.113:0/3039609339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c8200 0x7fc9b00b0e30 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:11.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 -- 172.21.15.113:0/3039609339 >> 172.21.15.113:0/3039609339 conn(0x7fc9b006e820 msgr2=0x7fc9b01564e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:11.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 -- 172.21.15.113:0/3039609339 shutdown_connections 2024-03-04T07:57:11.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:11.455+0000 7fc9b7e00640 1 -- 172.21.15.113:0/3039609339 wait complete. 2024-03-04T07:57:11.472 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:57:13.474 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:13.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc8a00640 1 Processor -- start 2024-03-04T07:57:13.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc8a00640 1 -- start start 2024-03-04T07:57:13.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc8a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc4070df0 0x7ffbc40c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:13.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc8a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc40c9c50 con 0x7ffbc4157ef0 2024-03-04T07:57:13.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc8a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc415a9a0 con 0x7ffbc406fb70 2024-03-04T07:57:13.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc8a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc406de30 con 0x7ffbc4070df0 2024-03-04T07:57:13.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ffbc4157ef0 0x7ffbc415b180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53938/0 (socket says 172.21.15.113:53938) 2024-03-04T07:57:13.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc3400640 1 -- 172.21.15.113:0/3662614025 learned_addr learned my addr 172.21.15.113:0/3662614025 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:57:13.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbbaa00640 1 --2- 172.21.15.113:0/3662614025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc4070df0 0x7ffbc40c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:13.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc2a00640 1 -- 172.21.15.113:0/3662614025 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291045339 0 0) 0x7ffbc40c9c50 con 0x7ffbc4157ef0 2024-03-04T07:57:13.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc2a00640 1 -- 172.21.15.113:0/3662614025 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba8003530 con 0x7ffbc4157ef0 2024-03-04T07:57:13.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbc2a00640 1 -- 172.21.15.113:0/3662614025 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540474198 0 0) 0x7ffbc415a9a0 con 0x7ffbc406fb70 2024-03-04T07:57:13.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbbaa00640 1 -- 172.21.15.113:0/3662614025 >> v1:172.21.15.136:6789/0 conn(0x7ffbc406fb70 legacy=0x7ffbc40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:13.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbbaa00640 1 -- 172.21.15.113:0/3662614025 >> v1:172.21.15.113:6789/0 conn(0x7ffbc4157ef0 legacy=0x7ffbc415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:13.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.643+0000 7ffbbaa00640 1 -- 172.21.15.113:0/3662614025 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc406c210 con 0x7ffbc4070df0 2024-03-04T07:57:13.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbbaa00640 1 --2- 172.21.15.113:0/3662614025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc4070df0 0x7ffbc40c8a90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0097f0 tx=0x7ffbac01c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61c113338ea493e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbc2a00640 1 -- 172.21.15.113:0/3662614025 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0b0070 con 0x7ffbc4070df0 2024-03-04T07:57:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbc2a00640 1 -- 172.21.15.113:0/3662614025 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac0ab400 con 0x7ffbc4070df0 2024-03-04T07:57:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbc2a00640 1 -- 172.21.15.113:0/3662614025 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4001a00 con 0x7ffbc4070df0 2024-03-04T07:57:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbc8a00640 1 -- 172.21.15.113:0/3662614025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc4070df0 msgr2=0x7ffbc40c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbc8a00640 1 --2- 172.21.15.113:0/3662614025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc4070df0 0x7ffbc40c8a90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0097f0 tx=0x7ffbac01c320 comp rx=0 tx=0).stop 2024-03-04T07:57:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbc8a00640 1 -- 172.21.15.113:0/3662614025 shutdown_connections 2024-03-04T07:57:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbc8a00640 1 --2- 172.21.15.113:0/3662614025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc4070df0 0x7ffbc40c8a90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbc8a00640 1 -- 172.21.15.113:0/3662614025 >> 172.21.15.113:0/3662614025 conn(0x7ffbc4072f40 msgr2=0x7ffbc40706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbc8a00640 1 -- 172.21.15.113:0/3662614025 shutdown_connections 2024-03-04T07:57:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.647+0000 7ffbc8a00640 1 -- 172.21.15.113:0/3662614025 wait complete. 2024-03-04T07:57:13.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc8a00640 1 Processor -- start 2024-03-04T07:57:13.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc8a00640 1 -- start start 2024-03-04T07:57:13.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc8a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc406fb70 0x7ffbc41f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbc4070df0 0x7ffbc4155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbc4157ef0 0x7ffbc41565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc41582d0 con 0x7ffbc4070df0 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc8a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc415b890 con 0x7ffbc4157ef0 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc8a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4158450 con 0x7ffbc406fb70 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc406fb70 0x7ffbc41f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbbaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbc4070df0 0x7ffbc4155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc406fb70 0x7ffbc41f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:34904/0 (socket says 172.21.15.113:34904) 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc3400640 1 -- 172.21.15.113:0/165238201 learned_addr learned my addr 172.21.15.113:0/165238201 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc3e00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbc4157ef0 0x7ffbc41565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbbaa00640 1 -- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc406fb70 msgr2=0x7ffbc41f36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbbaa00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc406fb70 0x7ffbc41f36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbbaa00640 1 -- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbc4157ef0 msgr2=0x7ffbc41565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbbaa00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbc4157ef0 0x7ffbc41565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbbaa00640 1 -- 172.21.15.113:0/165238201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40ca740 con 0x7ffbc4070df0 2024-03-04T07:57:13.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbbaa00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbc4070df0 0x7ffbc4155410 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0ab000 tx=0x7ffbac0190c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:13.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbbbe00640 1 -- 172.21.15.113:0/165238201 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0b0040 con 0x7ffbc4070df0 2024-03-04T07:57:13.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbbbe00640 1 -- 172.21.15.113:0/165238201 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac0037e0 con 0x7ffbc4070df0 2024-03-04T07:57:13.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbbbe00640 1 -- 172.21.15.113:0/165238201 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac003bb0 con 0x7ffbc4070df0 2024-03-04T07:57:13.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.651+0000 7ffbc8a00640 1 -- 172.21.15.113:0/165238201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc40caa20 con 0x7ffbc4070df0 2024-03-04T07:57:13.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.655+0000 7ffbc8a00640 1 -- 172.21.15.113:0/165238201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc4155950 con 0x7ffbc4070df0 2024-03-04T07:57:13.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.655+0000 7ffbc8a00640 1 -- 172.21.15.113:0/165238201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc4075a80 con 0x7ffbc4070df0 2024-03-04T07:57:13.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.655+0000 7ffbbbe00640 1 -- 172.21.15.113:0/165238201 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffbac0bd020 con 0x7ffbc4070df0 2024-03-04T07:57:13.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.659+0000 7ffbbbe00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffba0066f60 0x7ffba0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:13.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.659+0000 7ffbbbe00640 1 -- 172.21.15.113:0/165238201 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7ffbac11eb30 con 0x7ffbc4070df0 2024-03-04T07:57:13.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.659+0000 7ffbc3400640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffba0066f60 0x7ffba0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:13.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.659+0000 7ffbc3400640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffba0066f60 0x7ffba0069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4007fd0 tx=0x7ffbb4004d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:13.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.663+0000 7ffbbbe00640 1 -- 172.21.15.113:0/165238201 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ffbac0c4020 con 0x7ffbc4070df0 2024-03-04T07:57:13.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.875+0000 7ffbc8a00640 1 -- 172.21.15.113:0/165238201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbc406f950 con 0x7ffbc4070df0 2024-03-04T07:57:13.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.875+0000 7ffbbbe00640 1 -- 172.21.15.113:0/165238201 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5815 (secure 0 0 0) 0x7ffbac015fd0 con 0x7ffbc4070df0 2024-03-04T07:57:13.874 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:13.874 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:05.766686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6145,6240]},"id":1}]} 2024-03-04T07:57:13.874 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 83 2024-03-04T07:57:13.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 -- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffba0066f60 msgr2=0x7ffba0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:13.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffba0066f60 0x7ffba0069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4007fd0 tx=0x7ffbb4004d30 comp rx=0 tx=0).stop 2024-03-04T07:57:13.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 -- 172.21.15.113:0/165238201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbc4070df0 msgr2=0x7ffbc4155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:13.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbc4070df0 0x7ffbc4155410 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0ab000 tx=0x7ffbac0190c0 comp rx=0 tx=0).stop 2024-03-04T07:57:13.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 -- 172.21.15.113:0/165238201 shutdown_connections 2024-03-04T07:57:13.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffbc4157ef0 0x7ffbc41565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:13.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffba0066f60 0x7ffba0069420 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:13.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbc4070df0 0x7ffbc4155410 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:13.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 --2- 172.21.15.113:0/165238201 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffbc406fb70 0x7ffbc41f36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:13.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 -- 172.21.15.113:0/165238201 >> 172.21.15.113:0/165238201 conn(0x7ffbc4072f40 msgr2=0x7ffbc41571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:13.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 -- 172.21.15.113:0/165238201 shutdown_connections 2024-03-04T07:57:13.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:13.879+0000 7ffbc8a00640 1 -- 172.21.15.113:0/165238201 wait complete. 2024-03-04T07:57:13.902 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:57:15.903 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:16.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d3f600640 1 Processor -- start 2024-03-04T07:57:16.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d3f600640 1 -- start start 2024-03-04T07:57:16.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d3f600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 0x7f6d38162c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:16.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d3f600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d38156160 con 0x7f6d38155d80 2024-03-04T07:57:16.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d3f600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3815ef10 con 0x7f6d381631b0 2024-03-04T07:57:16.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d3f600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d38162500 con 0x7f6d380c1860 2024-03-04T07:57:16.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d3cc00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6d38155d80 0x7f6d380c7dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53942/0 (socket says 172.21.15.113:53942) 2024-03-04T07:57:16.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d3cc00640 1 -- 172.21.15.113:0/3139004488 learned_addr learned my addr 172.21.15.113:0/3139004488 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:57:16.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d2fe00640 1 --2- 172.21.15.113:0/3139004488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 0x7f6d38162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:16.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d37e00640 1 -- 172.21.15.113:0/3139004488 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787573033 0 0) 0x7f6d3815ef10 con 0x7f6d381631b0 2024-03-04T07:57:16.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d2fe00640 1 -- 172.21.15.113:0/3139004488 >> v1:172.21.15.136:6789/0 conn(0x7f6d381631b0 legacy=0x7f6d380c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:16.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d2fe00640 1 -- 172.21.15.113:0/3139004488 >> v1:172.21.15.113:6789/0 conn(0x7f6d38155d80 legacy=0x7f6d380c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:16.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.055+0000 7f6d2fe00640 1 -- 172.21.15.113:0/3139004488 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3815f1b0 con 0x7f6d380c1860 2024-03-04T07:57:16.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.059+0000 7f6d2fe00640 1 --2- 172.21.15.113:0/3139004488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 0x7f6d38162c70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6d200097f0 tx=0x7f6d2001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=680d6e3cf34cc79c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.059+0000 7f6d37e00640 1 -- 172.21.15.113:0/3139004488 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d200b1070 con 0x7f6d380c1860 2024-03-04T07:57:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.059+0000 7f6d37e00640 1 -- 172.21.15.113:0/3139004488 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d200ac630 con 0x7f6d380c1860 2024-03-04T07:57:16.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.059+0000 7f6d37e00640 1 -- 172.21.15.113:0/3139004488 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d200a8050 con 0x7f6d380c1860 2024-03-04T07:57:16.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.063+0000 7f6d3f600640 1 -- 172.21.15.113:0/3139004488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 msgr2=0x7f6d38162c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:16.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.063+0000 7f6d3f600640 1 --2- 172.21.15.113:0/3139004488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 0x7f6d38162c70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6d200097f0 tx=0x7f6d2001c4e0 comp rx=0 tx=0).stop 2024-03-04T07:57:16.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.063+0000 7f6d3f600640 1 -- 172.21.15.113:0/3139004488 shutdown_connections 2024-03-04T07:57:16.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.063+0000 7f6d3f600640 1 --2- 172.21.15.113:0/3139004488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 0x7f6d38162c70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:16.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.063+0000 7f6d3f600640 1 -- 172.21.15.113:0/3139004488 >> 172.21.15.113:0/3139004488 conn(0x7f6d3806e820 msgr2=0x7f6d3806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:16.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.063+0000 7f6d3f600640 1 -- 172.21.15.113:0/3139004488 shutdown_connections 2024-03-04T07:57:16.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.063+0000 7f6d3f600640 1 -- 172.21.15.113:0/3139004488 wait complete. 2024-03-04T07:57:16.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.063+0000 7f6d3f600640 1 Processor -- start 2024-03-04T07:57:16.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.063+0000 7f6d3f600640 1 -- start start 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3f600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 0x7f6d380b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3f600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d38155d80 0x7f6d381fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3f600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d381631b0 0x7f6d3814f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3f600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380c1e10 con 0x7f6d38155d80 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3f600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3815fcd0 con 0x7f6d381631b0 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3f600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380c7760 con 0x7f6d380c1860 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d2fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d38155d80 0x7f6d381fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d2fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d38155d80 0x7f6d381fb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52792/0 (socket says 172.21.15.113:52792) 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d2fe00640 1 -- 172.21.15.113:0/4215646470 learned_addr learned my addr 172.21.15.113:0/4215646470 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3d600640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d381631b0 0x7f6d3814f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:16.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3cc00640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 0x7f6d380b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d2fe00640 1 -- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 msgr2=0x7f6d380b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:16.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d2fe00640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 0x7f6d380b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d2fe00640 1 -- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d381631b0 msgr2=0x7f6d3814f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d2fe00640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d381631b0 0x7f6d3814f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d2fe00640 1 -- 172.21.15.113:0/4215646470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d381fb600 con 0x7f6d38155d80 2024-03-04T07:57:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3d600640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d381631b0 0x7f6d3814f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:57:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d2fe00640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d38155d80 0x7f6d381fb0c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6d200097f0 tx=0x7f6d2001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d35600640 1 -- 172.21.15.113:0/4215646470 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d200bd070 con 0x7f6d38155d80 2024-03-04T07:57:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d35600640 1 -- 172.21.15.113:0/4215646470 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d200b2040 con 0x7f6d38155d80 2024-03-04T07:57:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3f600640 1 -- 172.21.15.113:0/4215646470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3815cf50 con 0x7f6d38155d80 2024-03-04T07:57:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d35600640 1 -- 172.21.15.113:0/4215646470 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d200b1050 con 0x7f6d38155d80 2024-03-04T07:57:16.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.067+0000 7f6d3f600640 1 -- 172.21.15.113:0/4215646470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d380c26e0 con 0x7f6d38155d80 2024-03-04T07:57:16.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.071+0000 7f6d3f600640 1 -- 172.21.15.113:0/4215646470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d38075c00 con 0x7f6d38155d80 2024-03-04T07:57:16.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.071+0000 7f6d35600640 1 -- 172.21.15.113:0/4215646470 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d20003620 con 0x7f6d38155d80 2024-03-04T07:57:16.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.071+0000 7f6d35600640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d14066f60 0x7f6d14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:16.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.071+0000 7f6d35600640 1 -- 172.21.15.113:0/4215646470 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f6d2011f060 con 0x7f6d38155d80 2024-03-04T07:57:16.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.075+0000 7f6d3cc00640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d14066f60 0x7f6d14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:16.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.075+0000 7f6d3cc00640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d14066f60 0x7f6d14069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6d28005890 tx=0x7f6d28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:16.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.075+0000 7f6d35600640 1 -- 172.21.15.113:0/4215646470 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6d200160d0 con 0x7f6d38155d80 2024-03-04T07:57:16.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.315+0000 7f6d3f600640 1 -- 172.21.15.113:0/4215646470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d3814f6b0 con 0x7f6d38155d80 2024-03-04T07:57:16.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.315+0000 7f6d35600640 1 -- 172.21.15.113:0/4215646470 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f6d200ef500 con 0x7f6d38155d80 2024-03-04T07:57:16.313 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:16.313 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:05.766686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6145,6240]},"id":1}]} 2024-03-04T07:57:16.313 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 83 2024-03-04T07:57:16.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 -- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d14066f60 msgr2=0x7f6d14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:16.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d14066f60 0x7f6d14069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6d28005890 tx=0x7f6d28009040 comp rx=0 tx=0).stop 2024-03-04T07:57:16.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 -- 172.21.15.113:0/4215646470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d38155d80 msgr2=0x7f6d381fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:16.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d38155d80 0x7f6d381fb0c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6d200097f0 tx=0x7f6d2001ad20 comp rx=0 tx=0).stop 2024-03-04T07:57:16.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 -- 172.21.15.113:0/4215646470 shutdown_connections 2024-03-04T07:57:16.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d381631b0 0x7f6d3814f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:16.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d14066f60 0x7f6d14069420 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:16.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d38155d80 0x7f6d381fb0c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:16.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 --2- 172.21.15.113:0/4215646470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d380c1860 0x7f6d380b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:16.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 -- 172.21.15.113:0/4215646470 >> 172.21.15.113:0/4215646470 conn(0x7f6d3806e820 msgr2=0x7f6d380cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:16.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 -- 172.21.15.113:0/4215646470 shutdown_connections 2024-03-04T07:57:16.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:16.319+0000 7f6d3f600640 1 -- 172.21.15.113:0/4215646470 wait complete. 2024-03-04T07:57:16.345 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:57:18.346 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:18.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.483+0000 7fe350a00640 1 Processor -- start 2024-03-04T07:57:18.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe350a00640 1 -- start start 2024-03-04T07:57:18.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe350a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe34c0c7760 0x7fe34c159150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:18.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe350a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34c0c62c0 con 0x7fe34c0c88e0 2024-03-04T07:57:18.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe350a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34c06f790 con 0x7fe34c152940 2024-03-04T07:57:18.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe350a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34c0c68a0 con 0x7fe34c0c7760 2024-03-04T07:57:18.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe34c0c88e0 0x7fe34c0c7050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47614/0 (socket says 172.21.15.113:47614) 2024-03-04T07:57:18.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349e00640 1 -- 172.21.15.113:0/35988524 learned_addr learned my addr 172.21.15.113:0/35988524 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:57:18.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe34b200640 1 --2- 172.21.15.113:0/35988524 >> v2:172.21.15.136:3300/0 conn(0x7fe34c0c7760 0x7fe34c159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963729848 0 0) 0x7fe34c0c62c0 con 0x7fe34c0c88e0 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe330003530 con 0x7fe34c0c88e0 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1631839864 0 0) 0x7fe330003530 con 0x7fe34c0c88e0 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 >> v2:172.21.15.136:3300/0 conn(0x7fe34c0c7760 msgr2=0x7fe34c159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 --2- 172.21.15.113:0/35988524 >> v2:172.21.15.136:3300/0 conn(0x7fe34c0c7760 0x7fe34c159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 >> v1:172.21.15.136:6789/0 conn(0x7fe34c152940 legacy=0x7fe34c06e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c066fc0 con 0x7fe34c0c88e0 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fe338001680 con 0x7fe34c0c88e0 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 >> v1:172.21.15.113:6789/0 conn(0x7fe34c0c88e0 legacy=0x7fe34c0c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3300039c0 0x7fe330003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:18.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe330004810 0x7fe330004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3300051d0 0x7fe33000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0c68a0 con 0x7fe3300039c0 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33000c070 con 0x7fe330004810 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33000c270 con 0x7fe3300051d0 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe34a800640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3300039c0 0x7fe330003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe34b200640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3300051d0 0x7fe33000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349e00640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe330004810 0x7fe330004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe34a800640 1 -- 172.21.15.113:0/35988524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3300051d0 msgr2=0x7fe33000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe34a800640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3300051d0 0x7fe33000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe34a800640 1 -- 172.21.15.113:0/35988524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe330004810 msgr2=0x7fe330004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe34a800640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe330004810 0x7fe330004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe34a800640 1 -- 172.21.15.113:0/35988524 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe330003530 con 0x7fe3300039c0 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe349e00640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe330004810 0x7fe330004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:57:18.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.487+0000 7fe34b200640 1 -- 172.21.15.113:0/35988524 reap_dead start 2024-03-04T07:57:18.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe34a800640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3300039c0 0x7fe330003de0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe33c001780 tx=0x7fe33c01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a1e1cca3599fa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:18.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0013a0 con 0x7fe3300039c0 2024-03-04T07:57:18.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c0052f0 con 0x7fe3300039c0 2024-03-04T07:57:18.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe349400640 1 -- 172.21.15.113:0/35988524 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c003840 con 0x7fe3300039c0 2024-03-04T07:57:18.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe350a00640 1 -- 172.21.15.113:0/35988524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3300039c0 msgr2=0x7fe330003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:18.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe350a00640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3300039c0 0x7fe330003de0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe33c001780 tx=0x7fe33c01c240 comp rx=0 tx=0).stop 2024-03-04T07:57:18.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe350a00640 1 -- 172.21.15.113:0/35988524 shutdown_connections 2024-03-04T07:57:18.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe350a00640 1 --2- 172.21.15.113:0/35988524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3300039c0 0x7fe330003de0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe350a00640 1 -- 172.21.15.113:0/35988524 >> 172.21.15.113:0/35988524 conn(0x7fe34c0c8fa0 msgr2=0x7fe34c0c93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:18.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe350a00640 1 -- 172.21.15.113:0/35988524 shutdown_connections 2024-03-04T07:57:18.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.491+0000 7fe350a00640 1 -- 172.21.15.113:0/35988524 wait complete. 2024-03-04T07:57:18.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe350a00640 1 Processor -- start 2024-03-04T07:57:18.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe350a00640 1 -- start start 2024-03-04T07:57:18.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe350a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3300039c0 0x7fe34c150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe350a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe34c155c60 0x7fe34c150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe350a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe34c13fde0 0x7fe34c140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe350a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c152d20 con 0x7fe34c13fde0 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe350a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c159560 con 0x7fe34c155c60 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe350a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0c7440 con 0x7fe3300039c0 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe34b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe34c13fde0 0x7fe34c140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe34a800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3300039c0 0x7fe34c150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe34b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe34c13fde0 0x7fe34c140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52732/0 (socket says 172.21.15.113:52732) 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe34b200640 1 -- 172.21.15.113:0/3401716027 learned_addr learned my addr 172.21.15.113:0/3401716027 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe349e00640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe34c155c60 0x7fe34c150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe34b200640 1 -- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3300039c0 msgr2=0x7fe34c150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe34b200640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3300039c0 0x7fe34c150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe34b200640 1 -- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe34c155c60 msgr2=0x7fe34c150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:18.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe34b200640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe34c155c60 0x7fe34c150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe34b200640 1 -- 172.21.15.113:0/3401716027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c158440 con 0x7fe34c13fde0 2024-03-04T07:57:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe34b200640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe34c13fde0 0x7fe34c140290 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0c9c50 tx=0x7fe34400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe342a00640 1 -- 172.21.15.113:0/3401716027 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34400fc80 con 0x7fe34c13fde0 2024-03-04T07:57:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe342a00640 1 -- 172.21.15.113:0/3401716027 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34400fe20 con 0x7fe34c13fde0 2024-03-04T07:57:18.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe342a00640 1 -- 172.21.15.113:0/3401716027 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344015670 con 0x7fe34c13fde0 2024-03-04T07:57:18.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe350a00640 1 -- 172.21.15.113:0/3401716027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34c1407d0 con 0x7fe34c13fde0 2024-03-04T07:57:18.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.495+0000 7fe350a00640 1 -- 172.21.15.113:0/3401716027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34c154130 con 0x7fe34c13fde0 2024-03-04T07:57:18.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.499+0000 7fe350a00640 1 -- 172.21.15.113:0/3401716027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34c075a80 con 0x7fe34c13fde0 2024-03-04T07:57:18.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.499+0000 7fe342a00640 1 -- 172.21.15.113:0/3401716027 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe344015810 con 0x7fe34c13fde0 2024-03-04T07:57:18.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.499+0000 7fe342a00640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe32c066f90 0x7fe32c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:18.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.499+0000 7fe342a00640 1 -- 172.21.15.113:0/3401716027 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fe344086100 con 0x7fe34c13fde0 2024-03-04T07:57:18.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.499+0000 7fe34a800640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe32c066f90 0x7fe32c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:18.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.499+0000 7fe34a800640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe32c066f90 0x7fe32c069450 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe33c001750 tx=0x7fe33c0011a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:18.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.503+0000 7fe342a00640 1 -- 172.21.15.113:0/3401716027 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe344019030 con 0x7fe34c13fde0 2024-03-04T07:57:18.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.731+0000 7fe350a00640 1 -- 172.21.15.113:0/3401716027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe34c0c8150 con 0x7fe34c13fde0 2024-03-04T07:57:18.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.731+0000 7fe342a00640 1 -- 172.21.15.113:0/3401716027 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5817 (secure 0 0 0) 0x7fe344010040 con 0x7fe34c13fde0 2024-03-04T07:57:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:18.729 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:17.701487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:resolve","state_seq":34,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6240,6145]},"id":1}]} 2024-03-04T07:57:18.729 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 84 2024-03-04T07:57:18.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 -- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe32c066f90 msgr2=0x7fe32c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:18.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe32c066f90 0x7fe32c069450 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe33c001750 tx=0x7fe33c0011a0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 -- 172.21.15.113:0/3401716027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe34c13fde0 msgr2=0x7fe34c140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:18.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe34c13fde0 0x7fe34c140290 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0c9c50 tx=0x7fe34400f1b0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 -- 172.21.15.113:0/3401716027 shutdown_connections 2024-03-04T07:57:18.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe32c066f90 0x7fe32c069450 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe34c13fde0 0x7fe34c140290 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe34c155c60 0x7fe34c150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 --2- 172.21.15.113:0/3401716027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3300039c0 0x7fe34c150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 -- 172.21.15.113:0/3401716027 >> 172.21.15.113:0/3401716027 conn(0x7fe34c0c8fa0 msgr2=0x7fe34c14c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 -- 172.21.15.113:0/3401716027 shutdown_connections 2024-03-04T07:57:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:18.735+0000 7fe350a00640 1 -- 172.21.15.113:0/3401716027 wait complete. 2024-03-04T07:57:18.750 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 84, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 84, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6145": { "addr": "172.21.15.136:6835/2368734308", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2368734308, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2368734308, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6145, "incarnation": 71, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_6240": { "addr": "172.21.15.113:6836/986515123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 986515123, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 986515123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6240, "incarnation": 77, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6372": { "addr": "172.21.15.113:6833/2732565639", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2732565639, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2732565639, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6372, "incarnation": 83, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:resolve", "state_seq": 34 } }, "last_failure": 0, "last_failure_osd_epoch": 60, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:57:17.701487+0000", "qdb_cluster": [ 6240, 6145 ], "qdb_leader": 6145, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6372, "mds_1": 6145, "mds_2": 6240 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:57:20.752 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:20.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff092c00640 1 Processor -- start 2024-03-04T07:57:20.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff092c00640 1 -- start start 2024-03-04T07:57:20.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff092c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c06cde0 0x7ff08c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:20.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff092c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff08c0733f0 0x7ff08c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:20.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff092c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08c15b6a0 con 0x7ff08c155040 2024-03-04T07:57:20.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff092c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08c0c7e00 con 0x7ff08c0733f0 2024-03-04T07:57:20.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff092c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c072fe0 con 0x7ff08c06cde0 2024-03-04T07:57:20.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff090c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff08c0733f0 0x7ff08c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:20.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff083e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c06cde0 0x7ff08c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:20.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff083e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c06cde0 0x7ff08c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:34550/0 (socket says 172.21.15.113:34550) 2024-03-04T07:57:20.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff083e00640 1 -- 172.21.15.113:0/2465701024 learned_addr learned my addr 172.21.15.113:0/2465701024 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:20.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff090c00640 1 -- 172.21.15.113:0/2465701024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c06cde0 msgr2=0x7ff08c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:20.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff090c00640 1 --2- 172.21.15.113:0/2465701024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c06cde0 0x7ff08c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:20.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff090c00640 1 -- 172.21.15.113:0/2465701024 >> v1:172.21.15.136:6789/0 conn(0x7ff08c155040 legacy=0x7ff08c0c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:20.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.890+0000 7ff090c00640 1 -- 172.21.15.113:0/2465701024 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c15b980 con 0x7ff08c0733f0 2024-03-04T07:57:20.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff090c00640 1 --2- 172.21.15.113:0/2465701024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff08c0733f0 0x7ff08c0cd530 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff084009dc0 tx=0x7ff08401f140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab3edc76ab28e75 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:20.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff08b400640 1 -- 172.21.15.113:0/2465701024 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840b4070 con 0x7ff08c0733f0 2024-03-04T07:57:20.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff08b400640 1 -- 172.21.15.113:0/2465701024 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0840af600 con 0x7ff08c0733f0 2024-03-04T07:57:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff08b400640 1 -- 172.21.15.113:0/2465701024 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840af9d0 con 0x7ff08c0733f0 2024-03-04T07:57:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff092c00640 1 -- 172.21.15.113:0/2465701024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff08c0733f0 msgr2=0x7ff08c0cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff092c00640 1 --2- 172.21.15.113:0/2465701024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff08c0733f0 0x7ff08c0cd530 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff084009dc0 tx=0x7ff08401f140 comp rx=0 tx=0).stop 2024-03-04T07:57:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff092c00640 1 -- 172.21.15.113:0/2465701024 shutdown_connections 2024-03-04T07:57:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff092c00640 1 --2- 172.21.15.113:0/2465701024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff08c0733f0 0x7ff08c0cd530 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff092c00640 1 --2- 172.21.15.113:0/2465701024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c06cde0 0x7ff08c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff092c00640 1 -- 172.21.15.113:0/2465701024 >> 172.21.15.113:0/2465701024 conn(0x7ff08c0671b0 msgr2=0x7ff08c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff092c00640 1 -- 172.21.15.113:0/2465701024 shutdown_connections 2024-03-04T07:57:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.894+0000 7ff092c00640 1 -- 172.21.15.113:0/2465701024 wait complete. 2024-03-04T07:57:20.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff092c00640 1 Processor -- start 2024-03-04T07:57:20.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff092c00640 1 -- start start 2024-03-04T07:57:20.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff092c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff08c06cde0 0x7ff08c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:20.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff092c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c0733f0 0x7ff08c0ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:20.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff092c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c155040 0x7ff08c0afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:20.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff092c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c072c30 con 0x7ff08c155040 2024-03-04T07:57:20.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff092c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c072950 con 0x7ff08c06cde0 2024-03-04T07:57:20.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff092c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c06b100 con 0x7ff08c0733f0 2024-03-04T07:57:20.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff090c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c155040 0x7ff08c0afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:20.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff090c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c155040 0x7ff08c0afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52736/0 (socket says 172.21.15.113:52736) 2024-03-04T07:57:20.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff090c00640 1 -- 172.21.15.113:0/734575828 learned_addr learned my addr 172.21.15.113:0/734575828 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:20.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff083e00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c0733f0 0x7ff08c0ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:20.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff08be00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff08c06cde0 0x7ff08c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:20.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff090c00640 1 -- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c0733f0 msgr2=0x7ff08c0ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:20.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff090c00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c0733f0 0x7ff08c0ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:20.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff090c00640 1 -- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff08c06cde0 msgr2=0x7ff08c0b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:20.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff090c00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff08c06cde0 0x7ff08c0b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:20.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff090c00640 1 -- 172.21.15.113:0/734575828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c1f0680 con 0x7ff08c155040 2024-03-04T07:57:20.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff090c00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c155040 0x7ff08c0afaa0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff08401cd70 tx=0x7ff08401bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:20.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff088c00640 1 -- 172.21.15.113:0/734575828 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0840b4040 con 0x7ff08c155040 2024-03-04T07:57:20.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff088c00640 1 -- 172.21.15.113:0/734575828 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0840c2070 con 0x7ff08c155040 2024-03-04T07:57:20.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff088c00640 1 -- 172.21.15.113:0/734575828 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084009450 con 0x7ff08c155040 2024-03-04T07:57:20.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff092c00640 1 -- 172.21.15.113:0/734575828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c159d10 con 0x7ff08c155040 2024-03-04T07:57:20.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.898+0000 7ff092c00640 1 -- 172.21.15.113:0/734575828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08c1f0c00 con 0x7ff08c155040 2024-03-04T07:57:20.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.902+0000 7ff092c00640 1 -- 172.21.15.113:0/734575828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08c075c00 con 0x7ff08c155040 2024-03-04T07:57:20.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.902+0000 7ff088c00640 1 -- 172.21.15.113:0/734575828 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0840ba660 con 0x7ff08c155040 2024-03-04T07:57:20.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.902+0000 7ff088c00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff068066f60 0x7ff068069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:20.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.902+0000 7ff088c00640 1 -- 172.21.15.113:0/734575828 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7ff0841239c0 con 0x7ff08c155040 2024-03-04T07:57:20.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.902+0000 7ff08be00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff068066f60 0x7ff068069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:20.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.906+0000 7ff08be00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff068066f60 0x7ff068069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff07c001770 tx=0x7ff07c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:20.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:20.906+0000 7ff088c00640 1 -- 172.21.15.113:0/734575828 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff084018ca0 con 0x7ff08c155040 2024-03-04T07:57:21.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 -- 172.21.15.113:0/734575828 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff08c06c180 con 0x7ff08c155040 2024-03-04T07:57:21.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff088c00640 1 -- 172.21.15.113:0/734575828 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+5817 (secure 0 0 0) 0x7ff0840f3f80 con 0x7ff08c155040 2024-03-04T07:57:21.154 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:21.154 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:17.701487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:resolve","state_seq":34,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6240,6145]},"id":1}]} 2024-03-04T07:57:21.154 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 84 2024-03-04T07:57:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 -- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff068066f60 msgr2=0x7ff068069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff068066f60 0x7ff068069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff07c001770 tx=0x7ff07c009040 comp rx=0 tx=0).stop 2024-03-04T07:57:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 -- 172.21.15.113:0/734575828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c155040 msgr2=0x7ff08c0afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c155040 0x7ff08c0afaa0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff08401cd70 tx=0x7ff08401bee0 comp rx=0 tx=0).stop 2024-03-04T07:57:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 -- 172.21.15.113:0/734575828 shutdown_connections 2024-03-04T07:57:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff068066f60 0x7ff068069420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff08c155040 0x7ff08c0afaa0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff08c0733f0 0x7ff08c0ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 --2- 172.21.15.113:0/734575828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff08c06cde0 0x7ff08c0b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.154+0000 7ff092c00640 1 -- 172.21.15.113:0/734575828 >> 172.21.15.113:0/734575828 conn(0x7ff08c0671b0 msgr2=0x7ff08c06fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.158+0000 7ff092c00640 1 -- 172.21.15.113:0/734575828 shutdown_connections 2024-03-04T07:57:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:21.158+0000 7ff092c00640 1 -- 172.21.15.113:0/734575828 wait complete. 2024-03-04T07:57:21.174 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:57:23.175 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:23.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c08400640 1 Processor -- start 2024-03-04T07:57:23.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c08400640 1 -- start start 2024-03-04T07:57:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c08400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c00158010 0x7f7c0006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c08400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7c0006ff20 0x7f7c0015b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c08400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0015b750 0x7f7c000ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c08400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0006f450 con 0x7f7c00158010 2024-03-04T07:57:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c08400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0015ad80 con 0x7f7c0006ff20 2024-03-04T07:57:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c08400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0006e290 con 0x7f7c0015b750 2024-03-04T07:57:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c05a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c00158010 0x7f7c0006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c06400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0015b750 0x7f7c000ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c05a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7c00158010 0x7f7c0006dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52752/0 (socket says 172.21.15.113:52752) 2024-03-04T07:57:23.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c05a00640 1 -- 172.21.15.113:0/403548843 learned_addr learned my addr 172.21.15.113:0/403548843 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:23.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.318+0000 7f7c05000640 1 --2- 172.21.15.113:0/403548843 >> v2:172.21.15.136:3300/0 conn(0x7f7c0006ff20 0x7f7c0015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:23.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.322+0000 7f7c05a00640 1 -- 172.21.15.113:0/403548843 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0015b750 msgr2=0x7f7c000ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:23.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.322+0000 7f7c05a00640 1 --2- 172.21.15.113:0/403548843 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0015b750 0x7f7c000ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.322+0000 7f7c05a00640 1 -- 172.21.15.113:0/403548843 >> v2:172.21.15.136:3300/0 conn(0x7f7c0006ff20 msgr2=0x7f7c0015b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:23.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.322+0000 7f7c05a00640 1 --2- 172.21.15.113:0/403548843 >> v2:172.21.15.136:3300/0 conn(0x7f7c0006ff20 0x7f7c0015b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.322+0000 7f7c05a00640 1 -- 172.21.15.113:0/403548843 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c000ca0e0 con 0x7f7c00158010 2024-03-04T07:57:23.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.322+0000 7f7c06400640 1 --2- 172.21.15.113:0/403548843 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0015b750 0x7f7c000ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:57:23.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.322+0000 7f7c05a00640 1 --2- 172.21.15.113:0/403548843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 0x7f7c0006dd50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7c000cc040 tx=0x7f7bf401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21b6605612698b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:23.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.322+0000 7f7bffe00640 1 -- 172.21.15.113:0/403548843 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40b0070 con 0x7f7c00158010 2024-03-04T07:57:23.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.322+0000 7f7bffe00640 1 -- 172.21.15.113:0/403548843 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf40ab600 con 0x7f7c00158010 2024-03-04T07:57:23.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.322+0000 7f7bffe00640 1 -- 172.21.15.113:0/403548843 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40ab9d0 con 0x7f7c00158010 2024-03-04T07:57:23.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 -- 172.21.15.113:0/403548843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 msgr2=0x7f7c0006dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:23.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 --2- 172.21.15.113:0/403548843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 0x7f7c0006dd50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7c000cc040 tx=0x7f7bf401c520 comp rx=0 tx=0).stop 2024-03-04T07:57:23.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 -- 172.21.15.113:0/403548843 shutdown_connections 2024-03-04T07:57:23.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 --2- 172.21.15.113:0/403548843 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0015b750 0x7f7c000ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 --2- 172.21.15.113:0/403548843 >> v2:172.21.15.136:3300/0 conn(0x7f7c0006ff20 0x7f7c0015b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 --2- 172.21.15.113:0/403548843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 0x7f7c0006dd50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 -- 172.21.15.113:0/403548843 >> 172.21.15.113:0/403548843 conn(0x7f7c000734d0 msgr2=0x7f7c00066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:23.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 -- 172.21.15.113:0/403548843 shutdown_connections 2024-03-04T07:57:23.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 -- 172.21.15.113:0/403548843 wait complete. 2024-03-04T07:57:23.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 Processor -- start 2024-03-04T07:57:23.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 -- start start 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0006ff20 0x7f7c00159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 0x7f7c000aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0015b750 0x7f7c000af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c001583f0 con 0x7f7c00158010 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0006f980 con 0x7f7c0015b750 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c08400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0006c170 con 0x7f7c0006ff20 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c05000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 0x7f7c000aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c05a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0006ff20 0x7f7c00159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c05000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 0x7f7c000aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52758/0 (socket says 172.21.15.113:52758) 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c05000640 1 -- 172.21.15.113:0/1532398574 learned_addr learned my addr 172.21.15.113:0/1532398574 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c06400640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0015b750 0x7f7c000af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c05000640 1 -- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0006ff20 msgr2=0x7f7c00159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:23.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c05000640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0006ff20 0x7f7c00159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c05000640 1 -- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0015b750 msgr2=0x7f7c000af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c05000640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0015b750 0x7f7c000af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c05000640 1 -- 172.21.15.113:0/1532398574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0015a710 con 0x7f7c00158010 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.326+0000 7f7c05a00640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0006ff20 0x7f7c00159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7c05000640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 0x7f7c000aeb50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00e930 tx=0x7f7bec00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7bfd600640 1 -- 172.21.15.113:0/1532398574 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec017070 con 0x7f7c00158010 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7bfd600640 1 -- 172.21.15.113:0/1532398574 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec00c940 con 0x7f7c00158010 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7bfd600640 1 -- 172.21.15.113:0/1532398574 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec00cd10 con 0x7f7c00158010 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7c08400640 1 -- 172.21.15.113:0/1532398574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c001f3780 con 0x7f7c00158010 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7c08400640 1 -- 172.21.15.113:0/1532398574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c001f3d40 con 0x7f7c00158010 2024-03-04T07:57:23.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7c08400640 1 -- 172.21.15.113:0/1532398574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c00075c00 con 0x7f7c00158010 2024-03-04T07:57:23.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7bfd600640 1 -- 172.21.15.113:0/1532398574 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7bec01a070 con 0x7f7c00158010 2024-03-04T07:57:23.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7bfd600640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be0066f60 0x7f7be0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:23.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7bfd600640 1 -- 172.21.15.113:0/1532398574 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f7bec085750 con 0x7f7c00158010 2024-03-04T07:57:23.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.330+0000 7f7c05a00640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be0066f60 0x7f7be0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:23.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.334+0000 7f7c05a00640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be0066f60 0x7f7be0069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40ad000 tx=0x7f7bf4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:23.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.342+0000 7f7bfd600640 1 -- 172.21.15.113:0/1532398574 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7bec055cb0 con 0x7f7c00158010 2024-03-04T07:57:23.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 -- 172.21.15.113:0/1532398574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c0006fd00 con 0x7f7c00158010 2024-03-04T07:57:23.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7bfd600640 1 -- 172.21.15.113:0/1532398574 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f7bec059d50 con 0x7f7c00158010 2024-03-04T07:57:23.532 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:23.532 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:23.286600+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:reconnect","state_seq":36,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6145,6240]},"id":1}]} 2024-03-04T07:57:23.533 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-03-04T07:57:23.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 -- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be0066f60 msgr2=0x7f7be0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:23.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be0066f60 0x7f7be0069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40ad000 tx=0x7f7bf4008040 comp rx=0 tx=0).stop 2024-03-04T07:57:23.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 -- 172.21.15.113:0/1532398574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 msgr2=0x7f7c000aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:23.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 0x7f7c000aeb50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00e930 tx=0x7f7bec00ed30 comp rx=0 tx=0).stop 2024-03-04T07:57:23.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 -- 172.21.15.113:0/1532398574 shutdown_connections 2024-03-04T07:57:23.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0015b750 0x7f7c000af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be0066f60 0x7f7be0069420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c00158010 0x7f7c000aeb50 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 --2- 172.21.15.113:0/1532398574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0006ff20 0x7f7c00159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:23.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 -- 172.21.15.113:0/1532398574 >> 172.21.15.113:0/1532398574 conn(0x7f7c000734d0 msgr2=0x7f7c00070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:23.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 -- 172.21.15.113:0/1532398574 shutdown_connections 2024-03-04T07:57:23.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:23.534+0000 7f7c08400640 1 -- 172.21.15.113:0/1532398574 wait complete. 2024-03-04T07:57:23.553 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:57:25.554 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:25.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.690+0000 7fca29e00640 1 Processor -- start 2024-03-04T07:57:25.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.690+0000 7fca29e00640 1 -- start start 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca29e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca241527f0 0x7fca24070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca29e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fca240c8200 0x7fca24072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca29e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca24152bd0 0x7fca241587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca29e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca24071df0 con 0x7fca241527f0 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca29e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca241597b0 con 0x7fca240c8200 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca29e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca241591f0 con 0x7fca24152bd0 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca22a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca241527f0 0x7fca24070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca22a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca241527f0 0x7fca24070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52762/0 (socket says 172.21.15.113:52762) 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca22a00640 1 -- 172.21.15.113:0/158517277 learned_addr learned my addr 172.21.15.113:0/158517277 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca1be00640 1 --2- 172.21.15.113:0/158517277 >> v2:172.21.15.136:3300/0 conn(0x7fca240c8200 0x7fca24072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:25.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca23400640 1 --2- 172.21.15.113:0/158517277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca24152bd0 0x7fca241587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:25.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca22a00640 1 -- 172.21.15.113:0/158517277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca24152bd0 msgr2=0x7fca241587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:25.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca22a00640 1 --2- 172.21.15.113:0/158517277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca24152bd0 0x7fca241587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca22a00640 1 -- 172.21.15.113:0/158517277 >> v2:172.21.15.136:3300/0 conn(0x7fca240c8200 msgr2=0x7fca24072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:25.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca22a00640 1 --2- 172.21.15.113:0/158517277 >> v2:172.21.15.136:3300/0 conn(0x7fca240c8200 0x7fca24072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.694+0000 7fca22a00640 1 -- 172.21.15.113:0/158517277 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca241594d0 con 0x7fca241527f0 2024-03-04T07:57:25.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca22a00640 1 --2- 172.21.15.113:0/158517277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca241527f0 0x7fca24070d40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fca240cd230 tx=0x7fca1401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d8115cab58fb90 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:25.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca22000640 1 -- 172.21.15.113:0/158517277 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca140ab5f0 con 0x7fca241527f0 2024-03-04T07:57:25.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca22000640 1 -- 172.21.15.113:0/158517277 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca140b0070 con 0x7fca241527f0 2024-03-04T07:57:25.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca22000640 1 -- 172.21.15.113:0/158517277 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca140ab9b0 con 0x7fca241527f0 2024-03-04T07:57:25.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 -- 172.21.15.113:0/158517277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca241527f0 msgr2=0x7fca24070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:25.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 --2- 172.21.15.113:0/158517277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca241527f0 0x7fca24070d40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fca240cd230 tx=0x7fca1401c520 comp rx=0 tx=0).stop 2024-03-04T07:57:25.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 -- 172.21.15.113:0/158517277 shutdown_connections 2024-03-04T07:57:25.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 --2- 172.21.15.113:0/158517277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca24152bd0 0x7fca241587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 --2- 172.21.15.113:0/158517277 >> v2:172.21.15.136:3300/0 conn(0x7fca240c8200 0x7fca24072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 --2- 172.21.15.113:0/158517277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca241527f0 0x7fca24070d40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 -- 172.21.15.113:0/158517277 >> 172.21.15.113:0/158517277 conn(0x7fca2406e820 msgr2=0x7fca2406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:25.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 -- 172.21.15.113:0/158517277 shutdown_connections 2024-03-04T07:57:25.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 -- 172.21.15.113:0/158517277 wait complete. 2024-03-04T07:57:25.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 Processor -- start 2024-03-04T07:57:25.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.698+0000 7fca29e00640 1 -- start start 2024-03-04T07:57:25.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca29e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca240c8200 0x7fca240b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:25.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca29e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca241527f0 0x7fca241f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:25.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca29e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca24152bd0 0x7fca24144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:25.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca29e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2415bf40 con 0x7fca240c8200 2024-03-04T07:57:25.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca29e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2415bdc0 con 0x7fca24152bd0 2024-03-04T07:57:25.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca29e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2415c0c0 con 0x7fca241527f0 2024-03-04T07:57:25.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca22a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca240c8200 0x7fca240b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:25.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca22a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca240c8200 0x7fca240b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52768/0 (socket says 172.21.15.113:52768) 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca22a00640 1 -- 172.21.15.113:0/1395550178 learned_addr learned my addr 172.21.15.113:0/1395550178 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca1be00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca241527f0 0x7fca241f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca23400640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca24152bd0 0x7fca24144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca22a00640 1 -- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca241527f0 msgr2=0x7fca241f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca22a00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca241527f0 0x7fca241f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca22a00640 1 -- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca24152bd0 msgr2=0x7fca24144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca22a00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca24152bd0 0x7fca24144cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca22a00640 1 -- 172.21.15.113:0/1395550178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca240b14b0 con 0x7fca240c8200 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca22a00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca240c8200 0x7fca240b0960 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fca1401abd0 tx=0x7fca1401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca1b400640 1 -- 172.21.15.113:0/1395550178 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca140be3f0 con 0x7fca240c8200 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca1b400640 1 -- 172.21.15.113:0/1395550178 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca140be590 con 0x7fca240c8200 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca1b400640 1 -- 172.21.15.113:0/1395550178 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca140be940 con 0x7fca240c8200 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca29e00640 1 -- 172.21.15.113:0/1395550178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca24149390 con 0x7fca240c8200 2024-03-04T07:57:25.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca29e00640 1 -- 172.21.15.113:0/1395550178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca24149910 con 0x7fca240c8200 2024-03-04T07:57:25.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca29e00640 1 -- 172.21.15.113:0/1395550178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca24075c00 con 0x7fca240c8200 2024-03-04T07:57:25.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.702+0000 7fca1b400640 1 -- 172.21.15.113:0/1395550178 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca140b43f0 con 0x7fca240c8200 2024-03-04T07:57:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.706+0000 7fca1b400640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca00066f90 0x7fca00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.706+0000 7fca1b400640 1 -- 172.21.15.113:0/1395550178 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fca1411f410 con 0x7fca240c8200 2024-03-04T07:57:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.706+0000 7fca1be00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca00066f90 0x7fca00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:25.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.706+0000 7fca1be00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca00066f90 0x7fca00069450 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fca24162900 tx=0x7fca0c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:25.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.710+0000 7fca1b400640 1 -- 172.21.15.113:0/1395550178 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fca14016110 con 0x7fca240c8200 2024-03-04T07:57:25.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.898+0000 7fca29e00640 1 -- 172.21.15.113:0/1395550178 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca2415a380 con 0x7fca240c8200 2024-03-04T07:57:25.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.898+0000 7fca1b400640 1 -- 172.21.15.113:0/1395550178 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5819 (secure 0 0 0) 0x7fca140ef940 con 0x7fca240c8200 2024-03-04T07:57:25.898 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:25.898 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:23.286600+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:reconnect","state_seq":36,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6145,6240]},"id":1}]} 2024-03-04T07:57:25.898 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 -- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca00066f90 msgr2=0x7fca00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca00066f90 0x7fca00069450 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fca24162900 tx=0x7fca0c009190 comp rx=0 tx=0).stop 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 -- 172.21.15.113:0/1395550178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca240c8200 msgr2=0x7fca240b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca240c8200 0x7fca240b0960 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fca1401abd0 tx=0x7fca1401ad60 comp rx=0 tx=0).stop 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 -- 172.21.15.113:0/1395550178 shutdown_connections 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca24152bd0 0x7fca24144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca241527f0 0x7fca241f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca00066f90 0x7fca00069450 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 --2- 172.21.15.113:0/1395550178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca240c8200 0x7fca240b0960 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 -- 172.21.15.113:0/1395550178 >> 172.21.15.113:0/1395550178 conn(0x7fca2406e820 msgr2=0x7fca24070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:25.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 -- 172.21.15.113:0/1395550178 shutdown_connections 2024-03-04T07:57:25.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:25.902+0000 7fca29e00640 1 -- 172.21.15.113:0/1395550178 wait complete. 2024-03-04T07:57:25.918 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:57:27.919 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:28.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf8600640 1 Processor -- start 2024-03-04T07:57:28.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf8600640 1 -- start start 2024-03-04T07:57:28.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf8600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fedf00c8200 0x7fedf0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:28.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf8600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf01591f0 con 0x7fedf0152bd0 2024-03-04T07:57:28.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf8600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf01594d0 con 0x7fedf01527f0 2024-03-04T07:57:28.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf8600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf0071df0 con 0x7fedf00c8200 2024-03-04T07:57:28.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf5200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fedf00c8200 0x7fedf0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:28.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf5200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fedf00c8200 0x7fedf0070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42084/0 (socket says 172.21.15.113:42084) 2024-03-04T07:57:28.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf5200640 1 -- 172.21.15.113:0/1711453319 learned_addr learned my addr 172.21.15.113:0/1711453319 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:28.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf6600640 1 --1- 172.21.15.113:0/1711453319 >> v1:172.21.15.113:6789/0 conn(0x7fedf0152bd0 0x7fedf01587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55362/0 (socket says 172.21.15.113:55362) 2024-03-04T07:57:28.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf5200640 1 -- 172.21.15.113:0/1711453319 >> v1:172.21.15.136:6789/0 conn(0x7fedf01527f0 legacy=0x7fedf0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:28.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf5200640 1 -- 172.21.15.113:0/1711453319 >> v1:172.21.15.113:6789/0 conn(0x7fedf0152bd0 legacy=0x7fedf01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:28.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedf5200640 1 -- 172.21.15.113:0/1711453319 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf00717a0 con 0x7fedf00c8200 2024-03-04T07:57:28.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.058+0000 7fedefe00640 1 -- 172.21.15.113:0/1711453319 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858874552 0 0) 0x7fedf01591f0 con 0x7fedf0152bd0 2024-03-04T07:57:28.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedf5200640 1 --2- 172.21.15.113:0/1711453319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf00c8200 0x7fedf0070d40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7feddc009870 tx=0x7feddc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1aae3af2e15a24 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:28.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedefe00640 1 -- 172.21.15.113:0/1711453319 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc0b2070 con 0x7fedf00c8200 2024-03-04T07:57:28.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedefe00640 1 -- 172.21.15.113:0/1711453319 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc0a9070 con 0x7fedf00c8200 2024-03-04T07:57:28.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedefe00640 1 -- 172.21.15.113:0/1711453319 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc0ad880 con 0x7fedf00c8200 2024-03-04T07:57:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedf8600640 1 -- 172.21.15.113:0/1711453319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf00c8200 msgr2=0x7fedf0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedf8600640 1 --2- 172.21.15.113:0/1711453319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf00c8200 0x7fedf0070d40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7feddc009870 tx=0x7feddc01c560 comp rx=0 tx=0).stop 2024-03-04T07:57:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedf8600640 1 -- 172.21.15.113:0/1711453319 shutdown_connections 2024-03-04T07:57:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedf8600640 1 --2- 172.21.15.113:0/1711453319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf00c8200 0x7fedf0070d40 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedf8600640 1 -- 172.21.15.113:0/1711453319 >> 172.21.15.113:0/1711453319 conn(0x7fedf006e820 msgr2=0x7fedf006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedf8600640 1 -- 172.21.15.113:0/1711453319 shutdown_connections 2024-03-04T07:57:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedf8600640 1 -- 172.21.15.113:0/1711453319 wait complete. 2024-03-04T07:57:28.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.062+0000 7fedf8600640 1 Processor -- start 2024-03-04T07:57:28.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf8600640 1 -- start start 2024-03-04T07:57:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf8600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedf00c8200 0x7fedf01f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf8600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedf01527f0 0x7fedf0157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf8600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf0152bd0 0x7fedf0154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf8600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0166500 con 0x7fedf0152bd0 2024-03-04T07:57:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf8600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0166380 con 0x7fedf01527f0 2024-03-04T07:57:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf8600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0166680 con 0x7fedf00c8200 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf6600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf0152bd0 0x7fedf0154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf5200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedf01527f0 0x7fedf0157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf6600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf0152bd0 0x7fedf0154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42092/0 (socket says 172.21.15.113:42092) 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf6600640 1 -- 172.21.15.113:0/553917683 learned_addr learned my addr 172.21.15.113:0/553917683 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf5c00640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedf00c8200 0x7fedf01f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf6600640 1 -- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedf00c8200 msgr2=0x7fedf01f4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf6600640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedf00c8200 0x7fedf01f4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf6600640 1 -- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedf01527f0 msgr2=0x7fedf0157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf6600640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedf01527f0 0x7fedf0157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf6600640 1 -- 172.21.15.113:0/553917683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0153100 con 0x7fedf0152bd0 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf6600640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf0152bd0 0x7fedf0154220 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fede800c700 tx=0x7fede800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7feded600640 1 -- 172.21.15.113:0/553917683 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8018070 con 0x7fedf0152bd0 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7feded600640 1 -- 172.21.15.113:0/553917683 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede800ce10 con 0x7fedf0152bd0 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7feded600640 1 -- 172.21.15.113:0/553917683 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede800a880 con 0x7fedf0152bd0 2024-03-04T07:57:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf8600640 1 -- 172.21.15.113:0/553917683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf0156f10 con 0x7fedf0152bd0 2024-03-04T07:57:28.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf8600640 1 -- 172.21.15.113:0/553917683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf0153680 con 0x7fedf0152bd0 2024-03-04T07:57:28.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.066+0000 7fedf8600640 1 -- 172.21.15.113:0/553917683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf0075c00 con 0x7fedf0152bd0 2024-03-04T07:57:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.070+0000 7feded600640 1 -- 172.21.15.113:0/553917683 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fede800aa20 con 0x7fedf0152bd0 2024-03-04T07:57:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.070+0000 7feded600640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedd0066f60 0x7fedd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.070+0000 7feded600640 1 -- 172.21.15.113:0/553917683 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fede8084aa0 con 0x7fedf0152bd0 2024-03-04T07:57:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.070+0000 7fedf5c00640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedd0066f60 0x7fedd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.070+0000 7fedf5c00640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedd0066f60 0x7fedd0069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fede4007fd0 tx=0x7fede4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.074+0000 7feded600640 1 -- 172.21.15.113:0/553917683 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fede8054eb0 con 0x7fedf0152bd0 2024-03-04T07:57:28.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 -- 172.21.15.113:0/553917683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedf0163f40 con 0x7fedf0152bd0 2024-03-04T07:57:28.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7feded600640 1 -- 172.21.15.113:0/553917683 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5819 (secure 0 0 0) 0x7fede8058f50 con 0x7fedf0152bd0 2024-03-04T07:57:28.261 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:28.261 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:23.286600+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:reconnect","state_seq":36,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6145,6240]},"id":1}]} 2024-03-04T07:57:28.262 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 85 2024-03-04T07:57:28.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 -- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedd0066f60 msgr2=0x7fedd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:28.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedd0066f60 0x7fedd0069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fede4007fd0 tx=0x7fede4009040 comp rx=0 tx=0).stop 2024-03-04T07:57:28.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 -- 172.21.15.113:0/553917683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf0152bd0 msgr2=0x7fedf0154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:28.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf0152bd0 0x7fedf0154220 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fede800c700 tx=0x7fede800cbc0 comp rx=0 tx=0).stop 2024-03-04T07:57:28.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 -- 172.21.15.113:0/553917683 shutdown_connections 2024-03-04T07:57:28.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fedd0066f60 0x7fedd0069420 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:28.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedf0152bd0 0x7fedf0154220 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:28.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fedf01527f0 0x7fedf0157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:28.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 --2- 172.21.15.113:0/553917683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fedf00c8200 0x7fedf01f4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:28.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 -- 172.21.15.113:0/553917683 >> 172.21.15.113:0/553917683 conn(0x7fedf006e820 msgr2=0x7fedf01586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:28.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.262+0000 7fedf8600640 1 -- 172.21.15.113:0/553917683 shutdown_connections 2024-03-04T07:57:28.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:28.266+0000 7fedf8600640 1 -- 172.21.15.113:0/553917683 wait complete. 2024-03-04T07:57:28.283 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:57:30.284 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:30.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.418+0000 7f0e15e00640 1 Processor -- start 2024-03-04T07:57:30.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.418+0000 7f0e15e00640 1 -- start start 2024-03-04T07:57:30.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e15e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e10158210 0x7f0e100c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:30.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e15e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e100733f0 0x7f0e100cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:30.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e15e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e100b2230 con 0x7f0e1006b060 2024-03-04T07:57:30.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e15e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e10072fb0 con 0x7f0e10158210 2024-03-04T07:57:30.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e15e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100c7bb0 con 0x7f0e100733f0 2024-03-04T07:57:30.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e0ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e10158210 0x7f0e100c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:30.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e0ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e10158210 0x7f0e100c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42102/0 (socket says 172.21.15.113:42102) 2024-03-04T07:57:30.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e0ea00640 1 -- 172.21.15.113:0/2334242975 learned_addr learned my addr 172.21.15.113:0/2334242975 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:30.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e0f400640 1 --1- 172.21.15.113:0/2334242975 >> v1:172.21.15.113:6789/0 conn(0x7f0e1006b060 0x7f0e100c93f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55370/0 (socket says 172.21.15.113:55370) 2024-03-04T07:57:30.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e07e00640 1 --2- 172.21.15.113:0/2334242975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e100733f0 0x7f0e100cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e0ea00640 1 -- 172.21.15.113:0/2334242975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e100733f0 msgr2=0x7f0e100cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e0ea00640 1 --2- 172.21.15.113:0/2334242975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e100733f0 0x7f0e100cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e0ea00640 1 -- 172.21.15.113:0/2334242975 >> v1:172.21.15.113:6789/0 conn(0x7f0e1006b060 legacy=0x7f0e100c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:30.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.422+0000 7f0e0ea00640 1 -- 172.21.15.113:0/2334242975 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e100c7e50 con 0x7f0e10158210 2024-03-04T07:57:30.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e0ea00640 1 --2- 172.21.15.113:0/2334242975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 0x7f0e100c8bb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009870 tx=0x7f0e0001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1e1e7a7650ef22 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:30.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e0e000640 1 -- 172.21.15.113:0/2334242975 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000b2070 con 0x7f0e10158210 2024-03-04T07:57:30.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e0e000640 1 -- 172.21.15.113:0/2334242975 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e000a9070 con 0x7f0e10158210 2024-03-04T07:57:30.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e0e000640 1 -- 172.21.15.113:0/2334242975 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000ad880 con 0x7f0e10158210 2024-03-04T07:57:30.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e15e00640 1 -- 172.21.15.113:0/2334242975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 msgr2=0x7f0e100c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:30.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e15e00640 1 --2- 172.21.15.113:0/2334242975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 0x7f0e100c8bb0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009870 tx=0x7f0e0001c560 comp rx=0 tx=0).stop 2024-03-04T07:57:30.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e15e00640 1 -- 172.21.15.113:0/2334242975 shutdown_connections 2024-03-04T07:57:30.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e15e00640 1 --2- 172.21.15.113:0/2334242975 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e100733f0 0x7f0e100cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:30.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e15e00640 1 --2- 172.21.15.113:0/2334242975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 0x7f0e100c8bb0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:30.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e15e00640 1 -- 172.21.15.113:0/2334242975 >> 172.21.15.113:0/2334242975 conn(0x7f0e100671b0 msgr2=0x7f0e1006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:30.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e15e00640 1 -- 172.21.15.113:0/2334242975 shutdown_connections 2024-03-04T07:57:30.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e15e00640 1 -- 172.21.15.113:0/2334242975 wait complete. 2024-03-04T07:57:30.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e15e00640 1 Processor -- start 2024-03-04T07:57:30.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.426+0000 7f0e15e00640 1 -- start start 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e15e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e1006b060 0x7f0e101f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e15e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e100733f0 0x7f0e100c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e15e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 0x7f0e10142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e15e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1006cff0 con 0x7f0e10158210 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e15e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1006ce70 con 0x7f0e100733f0 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e15e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e10167ad0 con 0x7f0e1006b060 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e0f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 0x7f0e10142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e0f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 0x7f0e10142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42110/0 (socket says 172.21.15.113:42110) 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e0f400640 1 -- 172.21.15.113:0/1444689469 learned_addr learned my addr 172.21.15.113:0/1444689469 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e0ea00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e1006b060 0x7f0e101f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e07e00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e100733f0 0x7f0e100c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:30.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e0f400640 1 -- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e1006b060 msgr2=0x7f0e101f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e0f400640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e1006b060 0x7f0e101f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e0f400640 1 -- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e100733f0 msgr2=0x7f0e100c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e0f400640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e100733f0 0x7f0e100c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e0f400640 1 -- 172.21.15.113:0/1444689469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1014fdc0 con 0x7f0e10158210 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e07e00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e100733f0 0x7f0e100c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e0f400640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 0x7f0e10142d60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800c730 tx=0x7f0e0800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e07400640 1 -- 172.21.15.113:0/1444689469 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08017070 con 0x7f0e10158210 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e07400640 1 -- 172.21.15.113:0/1444689469 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0800ce40 con 0x7f0e10158210 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e15e00640 1 -- 172.21.15.113:0/1444689469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1014faf0 con 0x7f0e10158210 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e07400640 1 -- 172.21.15.113:0/1444689469 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0800a840 con 0x7f0e10158210 2024-03-04T07:57:30.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e15e00640 1 -- 172.21.15.113:0/1444689469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e10150b00 con 0x7f0e10158210 2024-03-04T07:57:30.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e15e00640 1 -- 172.21.15.113:0/1444689469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e10075c00 con 0x7f0e10158210 2024-03-04T07:57:30.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.430+0000 7f0e07400640 1 -- 172.21.15.113:0/1444689469 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e0801a070 con 0x7f0e10158210 2024-03-04T07:57:30.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.434+0000 7f0e07400640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0dec066f60 0x7f0dec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:30.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.434+0000 7f0e07400640 1 -- 172.21.15.113:0/1444689469 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f0e080849f0 con 0x7f0e10158210 2024-03-04T07:57:30.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.434+0000 7f0e0ea00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0dec066f60 0x7f0dec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:30.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.434+0000 7f0e0ea00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0dec066f60 0x7f0dec069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0e000af000 tx=0x7f0e00008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:30.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.438+0000 7f0e07400640 1 -- 172.21.15.113:0/1444689469 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0e08054e90 con 0x7f0e10158210 2024-03-04T07:57:30.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 -- 172.21.15.113:0/1444689469 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e1014f6a0 con 0x7f0e10158210 2024-03-04T07:57:30.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e07400640 1 -- 172.21.15.113:0/1444689469 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f0e08082e20 con 0x7f0e10158210 2024-03-04T07:57:30.649 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:30.649 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:28.368230+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:rejoin","state_seq":38,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6240,6145]},"id":1}]} 2024-03-04T07:57:30.649 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 86 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 -- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0dec066f60 msgr2=0x7f0dec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0dec066f60 0x7f0dec069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0e000af000 tx=0x7f0e00008040 comp rx=0 tx=0).stop 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 -- 172.21.15.113:0/1444689469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 msgr2=0x7f0e10142d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 0x7f0e10142d60 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800c730 tx=0x7f0e0800cbf0 comp rx=0 tx=0).stop 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 -- 172.21.15.113:0/1444689469 shutdown_connections 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0dec066f60 0x7f0dec069420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e10158210 0x7f0e10142d60 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e100733f0 0x7f0e100c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 --2- 172.21.15.113:0/1444689469 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e1006b060 0x7f0e101f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 -- 172.21.15.113:0/1444689469 >> 172.21.15.113:0/1444689469 conn(0x7f0e100671b0 msgr2=0x7f0e100c93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 -- 172.21.15.113:0/1444689469 shutdown_connections 2024-03-04T07:57:30.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:30.650+0000 7f0e15e00640 1 -- 172.21.15.113:0/1444689469 wait complete. 2024-03-04T07:57:30.671 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 86, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 86, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6145": { "addr": "172.21.15.136:6835/2368734308", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2368734308, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2368734308, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6145, "incarnation": 71, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 39 }, "gid_6240": { "addr": "172.21.15.113:6836/986515123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 986515123, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 986515123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6240, "incarnation": 77, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6372": { "addr": "172.21.15.113:6833/2732565639", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2732565639, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2732565639, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6372, "incarnation": 83, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:rejoin", "state_seq": 38 } }, "last_failure": 0, "last_failure_osd_epoch": 60, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:57:28.368230+0000", "qdb_cluster": [ 6240, 6145 ], "qdb_leader": 6145, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6372, "mds_1": 6145, "mds_2": 6240 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:57:31.756 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:57:31.759 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:57:32.672 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:32.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.806+0000 7fe2ed400640 1 Processor -- start 2024-03-04T07:57:32.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.806+0000 7fe2ed400640 1 -- start start 2024-03-04T07:57:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2ed400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe2e8152950 0x7fe2e815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2ed400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80cb540 0x7fe2e80c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2ed400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e80c4ce0 con 0x7fe2e806faa0 2024-03-04T07:57:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2ed400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e80c8370 con 0x7fe2e8152950 2024-03-04T07:57:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2ed400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80ca0a0 con 0x7fe2e80cb540 2024-03-04T07:57:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2df400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe2e806faa0 0x7fe2e806ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55376/0 (socket says 172.21.15.113:55376) 2024-03-04T07:57:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2df400640 1 -- 172.21.15.113:0/241420797 learned_addr learned my addr 172.21.15.113:0/241420797 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:57:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2eca00640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80cb540 0x7fe2e80c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7e00640 1 --2- 172.21.15.113:0/241420797 >> v2:172.21.15.136:3300/0 conn(0x7fe2e8152950 0x7fe2e815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977364682 0 0) 0x7fe2e80c4ce0 con 0x7fe2e806faa0 2024-03-04T07:57:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2cc003530 con 0x7fe2e806faa0 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 873076486 0 0) 0x7fe2cc003530 con 0x7fe2e806faa0 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80cb540 msgr2=0x7fe2e80c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80cb540 0x7fe2e80c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 >> v2:172.21.15.136:3300/0 conn(0x7fe2e8152950 msgr2=0x7fe2e815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 --2- 172.21.15.113:0/241420797 >> v2:172.21.15.136:3300/0 conn(0x7fe2e8152950 0x7fe2e815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e80c4f80 con 0x7fe2e806faa0 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fe2d0001680 con 0x7fe2e806faa0 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 >> v1:172.21.15.113:6789/0 conn(0x7fe2e806faa0 legacy=0x7fe2e806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2cc0039c0 0x7fe2cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2cc004810 0x7fe2cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80cb540 0x7fe2cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:32.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80c8370 con 0x7fe2cc004810 2024-03-04T07:57:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80ca0a0 con 0x7fe2cc0039c0 2024-03-04T07:57:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc00b9d0 con 0x7fe2e80cb540 2024-03-04T07:57:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2df400640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2cc004810 0x7fe2cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2eca00640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80cb540 0x7fe2cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2e7e00640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2cc0039c0 0x7fe2cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2df400640 1 -- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80cb540 msgr2=0x7fe2cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2df400640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2e80cb540 0x7fe2cc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2df400640 1 -- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2cc0039c0 msgr2=0x7fe2cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2df400640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2cc0039c0 0x7fe2cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.810+0000 7fe2df400640 1 -- 172.21.15.113:0/241420797 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc003530 con 0x7fe2cc004810 2024-03-04T07:57:32.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2df400640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2cc004810 0x7fe2cc004c90 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000c240 tx=0x7fe2d001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adc6d15cf8a8a09 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:32.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0007d40 con 0x7fe2cc004810 2024-03-04T07:57:32.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d00a9020 con 0x7fe2cc004810 2024-03-04T07:57:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2e7400640 1 -- 172.21.15.113:0/241420797 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0005ae0 con 0x7fe2cc004810 2024-03-04T07:57:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2ed400640 1 -- 172.21.15.113:0/241420797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2cc004810 msgr2=0x7fe2cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2ed400640 1 --2- 172.21.15.113:0/241420797 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2cc004810 0x7fe2cc004c90 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000c240 tx=0x7fe2d001cca0 comp rx=0 tx=0).stop 2024-03-04T07:57:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2eca00640 1 -- 172.21.15.113:0/241420797 reap_dead start 2024-03-04T07:57:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2ed400640 1 -- 172.21.15.113:0/241420797 shutdown_connections 2024-03-04T07:57:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2ed400640 1 -- 172.21.15.113:0/241420797 >> 172.21.15.113:0/241420797 conn(0x7fe2e8072f40 msgr2=0x7fe2e80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2ed400640 1 -- 172.21.15.113:0/241420797 shutdown_connections 2024-03-04T07:57:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.814+0000 7fe2ed400640 1 -- 172.21.15.113:0/241420797 wait complete. 2024-03-04T07:57:32.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2ed400640 1 Processor -- start 2024-03-04T07:57:32.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2ed400640 1 -- start start 2024-03-04T07:57:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2ed400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2cc004810 0x7fe2e8153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2ed400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e81557c0 0x7fe2e80c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2ed400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2e81ef000 0x7fe2e81ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2ed400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80c9a50 con 0x7fe2e81ef000 2024-03-04T07:57:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2ed400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e806fe80 con 0x7fe2e81557c0 2024-03-04T07:57:32.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2ed400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e80c9d30 con 0x7fe2cc004810 2024-03-04T07:57:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2eca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2e81ef000 0x7fe2e81ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2df400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e81557c0 0x7fe2e80c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2eca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2e81ef000 0x7fe2e81ef480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42122/0 (socket says 172.21.15.113:42122) 2024-03-04T07:57:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2eca00640 1 -- 172.21.15.113:0/3940526733 learned_addr learned my addr 172.21.15.113:0/3940526733 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2e7e00640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2cc004810 0x7fe2e8153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2eca00640 1 -- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2cc004810 msgr2=0x7fe2e8153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2eca00640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2cc004810 0x7fe2e8153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2eca00640 1 -- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e81557c0 msgr2=0x7fe2e80c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2eca00640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e81557c0 0x7fe2e80c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2eca00640 1 -- 172.21.15.113:0/3940526733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e8155d20 con 0x7fe2e81ef000 2024-03-04T07:57:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2eca00640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2e81ef000 0x7fe2e81ef480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe2e815cc60 tx=0x7fe2e0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2e4c00640 1 -- 172.21.15.113:0/3940526733 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0004240 con 0x7fe2e81ef000 2024-03-04T07:57:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2e4c00640 1 -- 172.21.15.113:0/3940526733 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e001b070 con 0x7fe2e81ef000 2024-03-04T07:57:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2e4c00640 1 -- 172.21.15.113:0/3940526733 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e000f770 con 0x7fe2e81ef000 2024-03-04T07:57:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2ed400640 1 -- 172.21.15.113:0/3940526733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e81f1350 con 0x7fe2e81ef000 2024-03-04T07:57:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.818+0000 7fe2ed400640 1 -- 172.21.15.113:0/3940526733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e81f1910 con 0x7fe2e81ef000 2024-03-04T07:57:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.822+0000 7fe2ed400640 1 -- 172.21.15.113:0/3940526733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e8075a80 con 0x7fe2e81ef000 2024-03-04T07:57:32.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.822+0000 7fe2e4c00640 1 -- 172.21.15.113:0/3940526733 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2e0012040 con 0x7fe2e81ef000 2024-03-04T07:57:32.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.822+0000 7fe2e4c00640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2c4066f90 0x7fe2c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:32.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.822+0000 7fe2e4c00640 1 -- 172.21.15.113:0/3940526733 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fe2e0085090 con 0x7fe2e81ef000 2024-03-04T07:57:32.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.822+0000 7fe2e7e00640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2c4066f90 0x7fe2c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:32.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.826+0000 7fe2e7e00640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2c4066f90 0x7fe2c4069450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8003b90 tx=0x7fe2d800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:32.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:32.830+0000 7fe2e4c00640 1 -- 172.21.15.113:0/3940526733 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe2e00555d0 con 0x7fe2e81ef000 2024-03-04T07:57:33.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.018+0000 7fe2ed400640 1 -- 172.21.15.113:0/3940526733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2e80c9580 con 0x7fe2e81ef000 2024-03-04T07:57:33.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.018+0000 7fe2e4c00640 1 -- 172.21.15.113:0/3940526733 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5816 (secure 0 0 0) 0x7fe2e0059670 con 0x7fe2e81ef000 2024-03-04T07:57:33.019 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:33.019 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:28.368230+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:rejoin","state_seq":38,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6145,"qdb_cluster":[6240,6145]},"id":1}]} 2024-03-04T07:57:33.020 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 86 2024-03-04T07:57:33.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 -- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2c4066f90 msgr2=0x7fe2c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:33.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2c4066f90 0x7fe2c4069450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8003b90 tx=0x7fe2d800b040 comp rx=0 tx=0).stop 2024-03-04T07:57:33.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 -- 172.21.15.113:0/3940526733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2e81ef000 msgr2=0x7fe2e81ef480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:33.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2e81ef000 0x7fe2e81ef480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe2e815cc60 tx=0x7fe2e0011cf0 comp rx=0 tx=0).stop 2024-03-04T07:57:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 -- 172.21.15.113:0/3940526733 shutdown_connections 2024-03-04T07:57:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe2c4066f90 0x7fe2c4069450 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe2e81ef000 0x7fe2e81ef480 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe2e81557c0 0x7fe2e80c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 --2- 172.21.15.113:0/3940526733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe2cc004810 0x7fe2e8153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 -- 172.21.15.113:0/3940526733 >> 172.21.15.113:0/3940526733 conn(0x7fe2e8072f40 msgr2=0x7fe2e8158fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 -- 172.21.15.113:0/3940526733 shutdown_connections 2024-03-04T07:57:33.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:33.022+0000 7fe2ed400640 1 -- 172.21.15.113:0/3940526733 wait complete. 2024-03-04T07:57:33.047 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:57:35.049 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:35.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.230+0000 7efd58400640 1 Processor -- start 2024-03-04T07:57:35.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.230+0000 7efd58400640 1 -- start start 2024-03-04T07:57:35.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.230+0000 7efd58400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd50152bd0 0x7efd50070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:35.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.230+0000 7efd58400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd501591f0 con 0x7efd500c8200 2024-03-04T07:57:35.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.230+0000 7efd58400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd501594d0 con 0x7efd501527f0 2024-03-04T07:57:35.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.230+0000 7efd58400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd50071df0 con 0x7efd50152bd0 2024-03-04T07:57:35.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.230+0000 7efd55000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efd500c8200 0x7efd501587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55380/0 (socket says 172.21.15.113:55380) 2024-03-04T07:57:35.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.230+0000 7efd55000640 1 -- 172.21.15.113:0/2221847456 learned_addr learned my addr 172.21.15.113:0/2221847456 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:57:35.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.230+0000 7efd56400640 1 --2- 172.21.15.113:0/2221847456 >> v2:172.21.15.113:3300/0 conn(0x7efd50152bd0 0x7efd50070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.230+0000 7efd4fe00640 1 -- 172.21.15.113:0/2221847456 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591694549 0 0) 0x7efd501591f0 con 0x7efd500c8200 2024-03-04T07:57:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd4fe00640 1 -- 172.21.15.113:0/2221847456 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd38003530 con 0x7efd500c8200 2024-03-04T07:57:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd56400640 1 -- 172.21.15.113:0/2221847456 >> v1:172.21.15.136:6789/0 conn(0x7efd501527f0 legacy=0x7efd50072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd56400640 1 -- 172.21.15.113:0/2221847456 >> v1:172.21.15.113:6789/0 conn(0x7efd500c8200 legacy=0x7efd501587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:35.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd56400640 1 -- 172.21.15.113:0/2221847456 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd500717a0 con 0x7efd50152bd0 2024-03-04T07:57:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd56400640 1 --2- 172.21.15.113:0/2221847456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd50152bd0 0x7efd50070d40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7efd48009d90 tx=0x7efd4801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965d199b1b061d95 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd4fe00640 1 -- 172.21.15.113:0/2221847456 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd480b2070 con 0x7efd50152bd0 2024-03-04T07:57:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd4fe00640 1 -- 172.21.15.113:0/2221847456 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd480a5830 con 0x7efd50152bd0 2024-03-04T07:57:35.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd4fe00640 1 -- 172.21.15.113:0/2221847456 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd480a5c00 con 0x7efd50152bd0 2024-03-04T07:57:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd58400640 1 -- 172.21.15.113:0/2221847456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd50152bd0 msgr2=0x7efd50070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd58400640 1 --2- 172.21.15.113:0/2221847456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd50152bd0 0x7efd50070d40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7efd48009d90 tx=0x7efd4801c970 comp rx=0 tx=0).stop 2024-03-04T07:57:35.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd58400640 1 -- 172.21.15.113:0/2221847456 shutdown_connections 2024-03-04T07:57:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd58400640 1 --2- 172.21.15.113:0/2221847456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd50152bd0 0x7efd50070d40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd58400640 1 -- 172.21.15.113:0/2221847456 >> 172.21.15.113:0/2221847456 conn(0x7efd5006e820 msgr2=0x7efd5006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.234+0000 7efd58400640 1 -- 172.21.15.113:0/2221847456 shutdown_connections 2024-03-04T07:57:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 -- 172.21.15.113:0/2221847456 wait complete. 2024-03-04T07:57:35.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 Processor -- start 2024-03-04T07:57:35.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 -- start start 2024-03-04T07:57:35.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd500c8200 0x7efd500b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:35.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd501527f0 0x7efd500b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd50152bd0 0x7efd50153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd50166500 con 0x7efd501527f0 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd50166380 con 0x7efd50152bd0 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd50166680 con 0x7efd500c8200 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd501527f0 0x7efd500b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd501527f0 0x7efd500b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42126/0 (socket says 172.21.15.113:42126) 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55000640 1 -- 172.21.15.113:0/650777078 learned_addr learned my addr 172.21.15.113:0/650777078 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55a00640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd500c8200 0x7efd500b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd56400640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd50152bd0 0x7efd50153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55000640 1 -- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd500c8200 msgr2=0x7efd500b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55000640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd500c8200 0x7efd500b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55000640 1 -- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd50152bd0 msgr2=0x7efd50153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:35.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55000640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd50152bd0 0x7efd50153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55000640 1 -- 172.21.15.113:0/650777078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5014ab40 con 0x7efd501527f0 2024-03-04T07:57:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55a00640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd500c8200 0x7efd500b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:57:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd55000640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd501527f0 0x7efd500b0e70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7efd3c001ec0 tx=0x7efd3c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd4d600640 1 -- 172.21.15.113:0/650777078 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c017070 con 0x7efd501527f0 2024-03-04T07:57:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd4d600640 1 -- 172.21.15.113:0/650777078 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c005d50 con 0x7efd501527f0 2024-03-04T07:57:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 -- 172.21.15.113:0/650777078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd501531a0 con 0x7efd501527f0 2024-03-04T07:57:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd4d600640 1 -- 172.21.15.113:0/650777078 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c007270 con 0x7efd501527f0 2024-03-04T07:57:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.238+0000 7efd58400640 1 -- 172.21.15.113:0/650777078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5014b0a0 con 0x7efd501527f0 2024-03-04T07:57:35.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.242+0000 7efd58400640 1 -- 172.21.15.113:0/650777078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd50075c00 con 0x7efd501527f0 2024-03-04T07:57:35.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.242+0000 7efd4d600640 1 -- 172.21.15.113:0/650777078 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd3c01a070 con 0x7efd501527f0 2024-03-04T07:57:35.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.242+0000 7efd4d600640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd30066f60 0x7efd30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:35.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.242+0000 7efd4d600640 1 -- 172.21.15.113:0/650777078 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7efd3c084770 con 0x7efd501527f0 2024-03-04T07:57:35.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.242+0000 7efd55a00640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd30066f60 0x7efd30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:35.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.246+0000 7efd55a00640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd30066f60 0x7efd30069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efd44001710 tx=0x7efd44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:35.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.246+0000 7efd4d600640 1 -- 172.21.15.113:0/650777078 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efd3c054d60 con 0x7efd501527f0 2024-03-04T07:57:35.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.434+0000 7efd58400640 1 -- 172.21.15.113:0/650777078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd500c79d0 con 0x7efd501527f0 2024-03-04T07:57:35.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.434+0000 7efd4d600640 1 -- 172.21.15.113:0/650777078 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+5821 (secure 0 0 0) 0x7efd3c058e00 con 0x7efd501527f0 2024-03-04T07:57:35.435 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:35.435 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6145,6240,6372]},"id":1}]} 2024-03-04T07:57:35.435 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 87 2024-03-04T07:57:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 -- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd30066f60 msgr2=0x7efd30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd30066f60 0x7efd30069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efd44001710 tx=0x7efd44009040 comp rx=0 tx=0).stop 2024-03-04T07:57:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 -- 172.21.15.113:0/650777078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd501527f0 msgr2=0x7efd500b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd501527f0 0x7efd500b0e70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7efd3c001ec0 tx=0x7efd3c00c960 comp rx=0 tx=0).stop 2024-03-04T07:57:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 -- 172.21.15.113:0/650777078 shutdown_connections 2024-03-04T07:57:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd50152bd0 0x7efd50153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:35.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd30066f60 0x7efd30069420 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:35.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd501527f0 0x7efd500b0e70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:35.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 --2- 172.21.15.113:0/650777078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd500c8200 0x7efd500b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:35.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 -- 172.21.15.113:0/650777078 >> 172.21.15.113:0/650777078 conn(0x7efd5006e820 msgr2=0x7efd501563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:35.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 -- 172.21.15.113:0/650777078 shutdown_connections 2024-03-04T07:57:35.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:35.438+0000 7efd58400640 1 -- 172.21.15.113:0/650777078 wait complete. 2024-03-04T07:57:35.457 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=0, replacing gid=5940 2024-03-04T07:57:35.457 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 9 secs before reviving mds.e 2024-03-04T07:57:44.457 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-03-04T07:57:44.458 INFO:tasks.ceph.mds.e:Restarting daemon 2024-03-04T07:57:44.458 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-03-04T07:57:44.461 INFO:tasks.ceph.mds.e:Started 2024-03-04T07:57:44.461 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:44.549 INFO:tasks.ceph.mds.e.smithi113.stdout:starting mds.e at 2024-03-04T07:57:44.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.630+0000 7f371ac00640 1 Processor -- start 2024-03-04T07:57:44.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.630+0000 7f371ac00640 1 -- start start 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f371ac00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3714157ef0 0x7f37140c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f371ac00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 0x7f371415b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f371ac00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f371415b6c0 0x7f37140cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f371ac00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37140c9c50 con 0x7f3714157ef0 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f371ac00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371415a9a0 con 0x7f371415b6c0 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f371ac00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371406de30 con 0x7f3714070df0 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f3719800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3714157ef0 0x7f37140c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f3718e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 0x7f371415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f371a200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f371415b6c0 0x7f37140cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f3719800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3714157ef0 0x7f37140c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44202/0 (socket says 172.21.15.113:44202) 2024-03-04T07:57:44.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f3719800640 1 -- 172.21.15.113:0/1620674507 learned_addr learned my addr 172.21.15.113:0/1620674507 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:44.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f3719800640 1 -- 172.21.15.113:0/1620674507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 msgr2=0x7f371415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:44.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f3719800640 1 --2- 172.21.15.113:0/1620674507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 0x7f371415b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f3719800640 1 -- 172.21.15.113:0/1620674507 >> v2:172.21.15.136:3300/0 conn(0x7f371415b6c0 msgr2=0x7f37140cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:44.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f3719800640 1 --2- 172.21.15.113:0/1620674507 >> v2:172.21.15.136:3300/0 conn(0x7f371415b6c0 0x7f37140cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f3719800640 1 -- 172.21.15.113:0/1620674507 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371406c210 con 0x7f3714157ef0 2024-03-04T07:57:44.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.634+0000 7f3718e00640 1 --2- 172.21.15.113:0/1620674507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 0x7f371415b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:57:44.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f3719800640 1 --2- 172.21.15.113:0/1620674507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3714157ef0 0x7f37140c8a90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3714073820 tx=0x7f370801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3e45a3a5a3d194 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:44.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f3713e00640 1 -- 172.21.15.113:0/1620674507 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37080ab5f0 con 0x7f3714157ef0 2024-03-04T07:57:44.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f3713e00640 1 -- 172.21.15.113:0/1620674507 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37080b0070 con 0x7f3714157ef0 2024-03-04T07:57:44.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f3713e00640 1 -- 172.21.15.113:0/1620674507 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37080ab9b0 con 0x7f3714157ef0 2024-03-04T07:57:44.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f371ac00640 1 -- 172.21.15.113:0/1620674507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3714157ef0 msgr2=0x7f37140c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:44.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f371ac00640 1 --2- 172.21.15.113:0/1620674507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3714157ef0 0x7f37140c8a90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3714073820 tx=0x7f370801c520 comp rx=0 tx=0).stop 2024-03-04T07:57:44.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f371ac00640 1 -- 172.21.15.113:0/1620674507 shutdown_connections 2024-03-04T07:57:44.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f371ac00640 1 --2- 172.21.15.113:0/1620674507 >> v2:172.21.15.136:3300/0 conn(0x7f371415b6c0 0x7f37140cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f371ac00640 1 --2- 172.21.15.113:0/1620674507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 0x7f371415b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f371ac00640 1 --2- 172.21.15.113:0/1620674507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3714157ef0 0x7f37140c8a90 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f371ac00640 1 -- 172.21.15.113:0/1620674507 >> 172.21.15.113:0/1620674507 conn(0x7f3714072f40 msgr2=0x7f37140706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:44.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f371ac00640 1 -- 172.21.15.113:0/1620674507 shutdown_connections 2024-03-04T07:57:44.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f371ac00640 1 -- 172.21.15.113:0/1620674507 wait complete. 2024-03-04T07:57:44.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.638+0000 7f371ac00640 1 Processor -- start 2024-03-04T07:57:44.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371ac00640 1 -- start start 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371ac00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 0x7f37140b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371ac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3714157ef0 0x7f37140aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371ac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371415b6c0 0x7f37141f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371ac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3714158450 con 0x7f371415b6c0 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371ac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37141582d0 con 0x7f3714157ef0 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371ac00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371415e720 con 0x7f3714070df0 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f3719800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 0x7f37140b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371415b6c0 0x7f37141f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f3719800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 0x7f37140b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:34564/0 (socket says 172.21.15.113:34564) 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f3719800640 1 -- 172.21.15.113:0/815996515 learned_addr learned my addr 172.21.15.113:0/815996515 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f3718e00640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3714157ef0 0x7f37140aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:44.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371a200640 1 -- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 msgr2=0x7f37140b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:44.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371a200640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 0x7f37140b2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371a200640 1 -- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3714157ef0 msgr2=0x7f37140aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:44.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371a200640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3714157ef0 0x7f37140aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371a200640 1 -- 172.21.15.113:0/815996515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37141f5300 con 0x7f371415b6c0 2024-03-04T07:57:44.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371a200640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371415b6c0 0x7f37141f5ac0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f370c00ebd0 tx=0x7f370c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:44.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f3711600640 1 -- 172.21.15.113:0/815996515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c018020 con 0x7f371415b6c0 2024-03-04T07:57:44.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f3711600640 1 -- 172.21.15.113:0/815996515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c00cd20 con 0x7f371415b6c0 2024-03-04T07:57:44.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f3711600640 1 -- 172.21.15.113:0/815996515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c0146c0 con 0x7f371415b6c0 2024-03-04T07:57:44.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371ac00640 1 -- 172.21.15.113:0/815996515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37141546a0 con 0x7f371415b6c0 2024-03-04T07:57:44.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371ac00640 1 -- 172.21.15.113:0/815996515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3714154c60 con 0x7f371415b6c0 2024-03-04T07:57:44.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.642+0000 7f371ac00640 1 -- 172.21.15.113:0/815996515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37141f55d0 con 0x7f371415b6c0 2024-03-04T07:57:44.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.646+0000 7f3711600640 1 -- 172.21.15.113:0/815996515 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f370c01b020 con 0x7f371415b6c0 2024-03-04T07:57:44.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.646+0000 7f3711600640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f36f4066f60 0x7f36f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:44.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.646+0000 7f3711600640 1 -- 172.21.15.113:0/815996515 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f370c085ef0 con 0x7f371415b6c0 2024-03-04T07:57:44.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.646+0000 7f3719800640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f36f4066f60 0x7f36f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:44.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.650+0000 7f3719800640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f36f4066f60 0x7f36f4069420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f370801c560 tx=0x7f37080a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:44.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.650+0000 7f3711600640 1 -- 172.21.15.113:0/815996515 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f370c022030 con 0x7f371415b6c0 2024-03-04T07:57:44.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 -- 172.21.15.113:0/815996515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3714075a80 con 0x7f371415b6c0 2024-03-04T07:57:44.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f3711600640 1 -- 172.21.15.113:0/815996515 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+5821 (secure 0 0 0) 0x7f370c010040 con 0x7f371415b6c0 2024-03-04T07:57:44.845 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:44.846 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6145,6240,6372]},"id":1}]} 2024-03-04T07:57:44.846 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 87 2024-03-04T07:57:44.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 -- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f36f4066f60 msgr2=0x7f36f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:44.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f36f4066f60 0x7f36f4069420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f370801c560 tx=0x7f37080a7000 comp rx=0 tx=0).stop 2024-03-04T07:57:44.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 -- 172.21.15.113:0/815996515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371415b6c0 msgr2=0x7f37141f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:44.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371415b6c0 0x7f37141f5ac0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f370c00ebd0 tx=0x7f370c00c490 comp rx=0 tx=0).stop 2024-03-04T07:57:44.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 -- 172.21.15.113:0/815996515 shutdown_connections 2024-03-04T07:57:44.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f36f4066f60 0x7f36f4069420 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f371415b6c0 0x7f37141f5ac0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3714157ef0 0x7f37140aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 --2- 172.21.15.113:0/815996515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3714070df0 0x7f37140b2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:44.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.846+0000 7f371ac00640 1 -- 172.21.15.113:0/815996515 >> 172.21.15.113:0/815996515 conn(0x7f3714072f40 msgr2=0x7f37141528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:44.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.850+0000 7f371ac00640 1 -- 172.21.15.113:0/815996515 shutdown_connections 2024-03-04T07:57:44.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:44.850+0000 7f371ac00640 1 -- 172.21.15.113:0/815996515 wait complete. 2024-03-04T07:57:44.867 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2024-03-04T07:57:46.868 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:57:47.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2e800640 1 Processor -- start 2024-03-04T07:57:47.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2e800640 1 -- start start 2024-03-04T07:57:47.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2e800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea28152920 0x7fea28071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:47.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2e800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fea2815bf20 0x7fea2815c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:47.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2e800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2806bb10 con 0x7fea2806c580 2024-03-04T07:57:47.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2e800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea280c7a40 con 0x7fea2815bf20 2024-03-04T07:57:47.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2e800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea280c7d20 con 0x7fea28152920 2024-03-04T07:57:47.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2ca00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fea2806c580 0x7fea280c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48698/0 (socket says 172.21.15.113:48698) 2024-03-04T07:57:47.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2ca00640 1 -- 172.21.15.113:0/3730471156 learned_addr learned my addr 172.21.15.113:0/3730471156 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:57:47.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2d400640 1 --2- 172.21.15.113:0/3730471156 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea28152920 0x7fea28071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:47.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2de00640 1 --2- 172.21.15.113:0/3730471156 >> v2:172.21.15.136:3300/0 conn(0x7fea2815bf20 0x7fea2815c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:47.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea27e00640 1 -- 172.21.15.113:0/3730471156 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966235291 0 0) 0x7fea2806bb10 con 0x7fea2806c580 2024-03-04T07:57:47.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea27e00640 1 -- 172.21.15.113:0/3730471156 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea10003530 con 0x7fea2806c580 2024-03-04T07:57:47.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2d400640 1 -- 172.21.15.113:0/3730471156 >> v2:172.21.15.136:3300/0 conn(0x7fea2815bf20 msgr2=0x7fea2815c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:57:47.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2d400640 1 --2- 172.21.15.113:0/3730471156 >> v2:172.21.15.136:3300/0 conn(0x7fea2815bf20 0x7fea2815c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:47.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2d400640 1 -- 172.21.15.113:0/3730471156 >> v1:172.21.15.113:6789/0 conn(0x7fea2806c580 legacy=0x7fea280c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:47.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2d400640 1 -- 172.21.15.113:0/3730471156 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2806bd80 con 0x7fea28152920 2024-03-04T07:57:47.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.010+0000 7fea2de00640 1 --2- 172.21.15.113:0/3730471156 >> v2:172.21.15.136:3300/0 conn(0x7fea2815bf20 0x7fea2815c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:57:47.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea2d400640 1 --2- 172.21.15.113:0/3730471156 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea28152920 0x7fea28071a30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0097c0 tx=0x7fea1c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27f5c8395281b7c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:47.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea27e00640 1 -- 172.21.15.113:0/3730471156 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0b0070 con 0x7fea28152920 2024-03-04T07:57:47.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea27e00640 1 -- 172.21.15.113:0/3730471156 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c0ab600 con 0x7fea28152920 2024-03-04T07:57:47.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea27e00640 1 -- 172.21.15.113:0/3730471156 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0ab9b0 con 0x7fea28152920 2024-03-04T07:57:47.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea2e800640 1 -- 172.21.15.113:0/3730471156 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea28152920 msgr2=0x7fea28071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:47.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea2e800640 1 --2- 172.21.15.113:0/3730471156 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea28152920 0x7fea28071a30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0097c0 tx=0x7fea1c01c410 comp rx=0 tx=0).stop 2024-03-04T07:57:47.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea2e800640 1 -- 172.21.15.113:0/3730471156 shutdown_connections 2024-03-04T07:57:47.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea2e800640 1 --2- 172.21.15.113:0/3730471156 >> v2:172.21.15.136:3300/0 conn(0x7fea2815bf20 0x7fea2815c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:47.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea2e800640 1 --2- 172.21.15.113:0/3730471156 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea28152920 0x7fea28071a30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:47.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea2e800640 1 -- 172.21.15.113:0/3730471156 >> 172.21.15.113:0/3730471156 conn(0x7fea280c9a30 msgr2=0x7fea280c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:47.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea2e800640 1 -- 172.21.15.113:0/3730471156 shutdown_connections 2024-03-04T07:57:47.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.014+0000 7fea2e800640 1 -- 172.21.15.113:0/3730471156 wait complete. 2024-03-04T07:57:47.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 Processor -- start 2024-03-04T07:57:47.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 -- start start 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea2806c580 0x7fea281f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea28152920 0x7fea28148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea2815bf20 0x7fea2814a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea280c4430 con 0x7fea2806c580 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea28152d00 con 0x7fea28152920 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea280c45b0 con 0x7fea2815bf20 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea2806c580 0x7fea281f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea2806c580 0x7fea281f1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44212/0 (socket says 172.21.15.113:44212) 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2d400640 1 -- 172.21.15.113:0/3225181344 learned_addr learned my addr 172.21.15.113:0/3225181344 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2ca00640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea28152920 0x7fea28148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:47.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2de00640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea2815bf20 0x7fea2814a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2d400640 1 -- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea2815bf20 msgr2=0x7fea2814a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2d400640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea2815bf20 0x7fea2814a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2d400640 1 -- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea28152920 msgr2=0x7fea28148bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2d400640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea28152920 0x7fea28148bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2d400640 1 -- 172.21.15.113:0/3225181344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea281f1130 con 0x7fea2806c580 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2ca00640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea28152920 0x7fea28148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2d400640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea2806c580 0x7fea281f1ba0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fea1c01a040 tx=0x7fea1c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea25600640 1 -- 172.21.15.113:0/3225181344 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0b0050 con 0x7fea2806c580 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea25600640 1 -- 172.21.15.113:0/3225181344 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c0bc070 con 0x7fea2806c580 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea25600640 1 -- 172.21.15.113:0/3225181344 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c006e20 con 0x7fea2806c580 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 -- 172.21.15.113:0/3225181344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea280b2a40 con 0x7fea2806c580 2024-03-04T07:57:47.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 -- 172.21.15.113:0/3225181344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea281f13a0 con 0x7fea2806c580 2024-03-04T07:57:47.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.018+0000 7fea2e800640 1 -- 172.21.15.113:0/3225181344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea28075a80 con 0x7fea2806c580 2024-03-04T07:57:47.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.022+0000 7fea25600640 1 -- 172.21.15.113:0/3225181344 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea1c0b5810 con 0x7fea2806c580 2024-03-04T07:57:47.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.022+0000 7fea25600640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea08066f90 0x7fea08069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:57:47.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.022+0000 7fea25600640 1 -- 172.21.15.113:0/3225181344 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fea1c11fb70 con 0x7fea2806c580 2024-03-04T07:57:47.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.022+0000 7fea2ca00640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea08066f90 0x7fea08069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:57:47.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.022+0000 7fea2ca00640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea08066f90 0x7fea08069450 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fea28159980 tx=0x7fea14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:57:47.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.026+0000 7fea25600640 1 -- 172.21.15.113:0/3225181344 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fea1c016090 con 0x7fea2806c580 2024-03-04T07:57:47.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.222+0000 7fea2e800640 1 -- 172.21.15.113:0/3225181344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea280c8300 con 0x7fea2806c580 2024-03-04T07:57:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea25600640 1 -- 172.21.15.113:0/3225181344 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6615 (secure 0 0 0) 0x7fea1c0f05d0 con 0x7fea2806c580 2024-03-04T07:57:47.224 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:57:47.224 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6534,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240,6145]},"id":1}]} 2024-03-04T07:57:47.224 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 88 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 -- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea08066f90 msgr2=0x7fea08069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea08066f90 0x7fea08069450 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fea28159980 tx=0x7fea14009040 comp rx=0 tx=0).stop 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 -- 172.21.15.113:0/3225181344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea2806c580 msgr2=0x7fea281f1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea2806c580 0x7fea281f1ba0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fea1c01a040 tx=0x7fea1c01ac50 comp rx=0 tx=0).stop 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 -- 172.21.15.113:0/3225181344 shutdown_connections 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea2815bf20 0x7fea2814a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea28152920 0x7fea28148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea08066f90 0x7fea08069450 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 --2- 172.21.15.113:0/3225181344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea2806c580 0x7fea281f1ba0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 -- 172.21.15.113:0/3225181344 >> 172.21.15.113:0/3225181344 conn(0x7fea280c9a30 msgr2=0x7fea2815ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 -- 172.21.15.113:0/3225181344 shutdown_connections 2024-03-04T07:57:47.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:57:47.226+0000 7fea2e800640 1 -- 172.21.15.113:0/3225181344 wait complete. 2024-03-04T07:57:47.245 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-03-04T07:57:47.245 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 40 secs before thrashing 2024-03-04T07:57:49.247 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 53% (211/398) Updating files: 54% (215/398) Updating files: 54% (216/398) Updating files: 55% (219/398) Updating files: 56% (223/398) Updating files: 57% (227/398) Updating files: 58% (231/398) Updating files: 59% (235/398) Updating files: 60% (239/398) Updating files: 61% (243/398) Updating files: 62% (247/398) Updating files: 63% (251/398) Updating files: 64% (255/398) Updating files: 65% (259/398) Updating files: 66% (263/398) Updating files: 67% (267/398) Updating files: 68% (271/398) Updating files: 69% (275/398) Updating files: 70% (279/398) Updating files: 71% (283/398) Updating files: 72% (287/398) Updating files: 73% (291/398) Updating files: 74% (295/398) Updating files: 75% (299/398) Updating files: 76% (303/398) Updating files: 77% (307/398) Updating files: 78% (311/398) Updating files: 79% (315/398) Updating files: 80% (319/398) Updating files: 81% (323/398) Updating files: 82% (327/398) Updating files: 82% (330/398) Updating files: 83% (331/398) Updating files: 84% (335/398) Updating files: 85% (339/398) Updating files: 86% (343/398) Updating files: 87% (347/398) Updating files: 87% (349/398) Updating files: 88% (351/398) Updating files: 89% (355/398) Updating files: 90% (359/398) Updating files: 91% (363/398) Updating files: 92% (367/398) Updating files: 93% (371/398) Updating files: 94% (375/398) Updating files: 95% (379/398) Updating files: 96% (383/398) Updating files: 97% (387/398) Updating files: 98% (391/398) Updating files: 99% (395/398) Updating files: 100% (398/398) Updating files: 100% (398/398), done. 2024-03-04T07:57:49.614 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 9019c6ce640 v0.31 2024-03-04T07:57:49.616 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.31 2024-03-04T07:57:49.638 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 32 -eq 48 ] 2024-03-04T07:57:49.639 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.32 2024-03-04T07:57:49.639 INFO:tasks.workunit.client.0.smithi113.stdout:v0.32 2024-03-04T07:57:49.639 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.32 2024-03-04T07:57:49.639 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.32 2024-03-04T07:57:52.616 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 74% (146/195) Updating files: 75% (147/195) Updating files: 76% (149/195) Updating files: 77% (151/195) Updating files: 78% (153/195) Updating files: 79% (155/195) Updating files: 80% (156/195) Updating files: 81% (158/195) Updating files: 82% (160/195) Updating files: 83% (162/195) Updating files: 84% (164/195) Updating files: 85% (166/195) Updating files: 86% (168/195) Updating files: 87% (170/195) Updating files: 88% (172/195) Updating files: 89% (174/195) Updating files: 90% (176/195) Updating files: 91% (178/195) Updating files: 92% (180/195) Updating files: 93% (182/195) Updating files: 94% (184/195) Updating files: 95% (186/195) Updating files: 96% (188/195) Updating files: 97% (190/195) Updating files: 98% (192/195) Updating files: 99% (194/195) Updating files: 100% (195/195) Updating files: 100% (195/195), done. 2024-03-04T07:57:52.647 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at c08d08baa6a v0.32 2024-03-04T07:57:52.651 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.32 2024-03-04T07:57:52.890 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 33 -eq 48 ] 2024-03-04T07:57:52.890 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.33 2024-03-04T07:57:52.891 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.33 2024-03-04T07:57:52.891 INFO:tasks.workunit.client.0.smithi113.stdout:v0.33 2024-03-04T07:57:52.891 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.33 2024-03-04T07:57:57.735 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 46% (74/160) Updating files: 47% (76/160) Updating files: 48% (77/160) Updating files: 49% (79/160) Updating files: 50% (80/160) Updating files: 51% (82/160) Updating files: 52% (84/160) Updating files: 53% (85/160) Updating files: 54% (87/160) Updating files: 55% (88/160) Updating files: 56% (90/160) Updating files: 57% (92/160) Updating files: 58% (93/160) Updating files: 59% (95/160) Updating files: 60% (96/160) Updating files: 61% (98/160) Updating files: 62% (100/160) Updating files: 63% (101/160) Updating files: 64% (103/160) Updating files: 65% (104/160) Updating files: 66% (106/160) Updating files: 67% (108/160) Updating files: 68% (109/160) Updating files: 69% (111/160) Updating files: 70% (112/160) Updating files: 71% (114/160) Updating files: 72% (116/160) Updating files: 73% (117/160) Updating files: 74% (119/160) Updating files: 75% (120/160) Updating files: 76% (122/160) Updating files: 76% (123/160) Updating files: 77% (124/160) Updating files: 78% (125/160) Updating files: 79% (127/160) Updating files: 80% (128/160) Updating files: 81% (130/160) Updating files: 82% (132/160) Updating files: 83% (133/160) Updating files: 84% (135/160) Updating files: 85% (136/160) Updating files: 86% (138/160) Updating files: 87% (140/160) Updating files: 88% (141/160) Updating files: 89% (143/160) Updating files: 90% (144/160) Updating files: 91% (146/160) Updating files: 92% (148/160) Updating files: 93% (149/160) Updating files: 94% (151/160) Updating files: 95% (152/160) Updating files: 96% (154/160) Updating files: 97% (156/160) Updating files: 98% (157/160) Updating files: 99% (159/160) Updating files: 100% (160/160) Updating files: 100% (160/160), done. 2024-03-04T07:57:57.781 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 340b4348bf6 v0.33 2024-03-04T07:57:57.789 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.33 2024-03-04T07:57:57.901 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 34 -eq 48 ] 2024-03-04T07:57:57.901 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.34 2024-03-04T07:57:57.901 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.34 2024-03-04T07:57:57.901 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.34 2024-03-04T07:57:57.901 INFO:tasks.workunit.client.0.smithi113.stdout:v0.34 2024-03-04T07:57:59.009 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 90% (117/129) Updating files: 91% (118/129) Updating files: 92% (119/129) Updating files: 93% (120/129) Updating files: 94% (122/129) Updating files: 95% (123/129) Updating files: 96% (124/129) Updating files: 97% (126/129) Updating files: 98% (127/129) Updating files: 99% (128/129) Updating files: 100% (129/129) Updating files: 100% (129/129), done. 2024-03-04T07:57:59.040 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 2f039eeeb74 v0.34 2024-03-04T07:57:59.043 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.34 2024-03-04T07:58:01.876 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:58:01.879 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:58:03.330 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 35 -eq 48 ] 2024-03-04T07:58:03.331 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.35 2024-03-04T07:58:03.331 INFO:tasks.workunit.client.0.smithi113.stdout:v0.35 2024-03-04T07:58:03.331 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.35 2024-03-04T07:58:03.331 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.35 2024-03-04T07:58:07.549 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 11% (40/334) Updating files: 12% (41/334) Updating files: 13% (44/334) Updating files: 14% (47/334) Updating files: 15% (51/334) Updating files: 16% (54/334) Updating files: 17% (57/334) Updating files: 18% (61/334) Updating files: 19% (64/334) Updating files: 20% (67/334) Updating files: 21% (71/334) Updating files: 22% (74/334) Updating files: 23% (77/334) Updating files: 24% (81/334) Updating files: 25% (84/334) Updating files: 26% (87/334) Updating files: 27% (91/334) Updating files: 28% (94/334) Updating files: 29% (97/334) Updating files: 30% (101/334) Updating files: 31% (104/334) Updating files: 32% (107/334) Updating files: 33% (111/334) Updating files: 34% (114/334) Updating files: 35% (117/334) Updating files: 36% (121/334) Updating files: 37% (124/334) Updating files: 38% (127/334) Updating files: 39% (131/334) Updating files: 40% (134/334) Updating files: 41% (137/334) Updating files: 42% (141/334) Updating files: 43% (144/334) Updating files: 44% (147/334) Updating files: 45% (151/334) Updating files: 45% (153/334) Updating files: 46% (154/334) Updating files: 47% (157/334) Updating files: 48% (161/334) Updating files: 49% (164/334) Updating files: 50% (167/334) Updating files: 51% (171/334) Updating files: 52% (174/334) Updating files: 53% (178/334) Updating files: 54% (181/334) Updating files: 55% (184/334) Updating files: 56% (188/334) Updating files: 57% (191/334) Updating files: 58% (194/334) Updating files: 59% (198/334) Updating files: 60% (201/334) Updating files: 61% (204/334) Updating files: 62% (208/334) Updating files: 63% (211/334) Updating files: 64% (214/334) Updating files: 65% (218/334) Updating files: 66% (221/334) Updating files: 67% (224/334) Updating files: 68% (228/334) Updating files: 69% (231/334) Updating files: 70% (234/334) Updating files: 71% (238/334) Updating files: 72% (241/334) Updating files: 73% (244/334) Updating files: 74% (248/334) Updating files: 75% (251/334) Updating files: 76% (254/334) Updating files: 77% (258/334) Updating files: 78% (261/334) Updating files: 79% (264/334) Updating files: 80% (268/334) Updating files: 81% (271/334) Updating files: 82% (274/334) Updating files: 82% (277/334) Updating files: 83% (278/334) Updating files: 84% (281/334) Updating files: 85% (284/334) Updating files: 86% (288/334) Updating files: 87% (291/334) Updating files: 88% (294/334) Updating files: 89% (298/334) Updating files: 90% (301/334) Updating files: 91% (304/334) Updating files: 92% (308/334) Updating files: 93% (311/334) Updating files: 94% (314/334) Updating files: 95% (318/334) Updating files: 96% (321/334) Updating files: 97% (324/334) Updating files: 98% (328/334) Updating files: 99% (331/334) Updating files: 100% (334/334) Updating files: 100% (334/334), done. 2024-03-04T07:58:08.732 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 0afda378124 v0.35 2024-03-04T07:58:08.736 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.35 2024-03-04T07:58:08.835 INFO:tasks.workunit.client.0.smithi113.stdout:v0.36 2024-03-04T07:58:08.835 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 36 -eq 48 ] 2024-03-04T07:58:08.835 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.36 2024-03-04T07:58:08.835 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.36 2024-03-04T07:58:08.835 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.36 2024-03-04T07:58:19.071 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 51% (125/244) Updating files: 52% (127/244) Updating files: 53% (130/244) Updating files: 54% (132/244) Updating files: 55% (135/244) Updating files: 56% (137/244) Updating files: 57% (140/244) Updating files: 58% (142/244) Updating files: 58% (143/244) Updating files: 59% (144/244) Updating files: 60% (147/244) Updating files: 61% (149/244) Updating files: 62% (152/244) Updating files: 63% (154/244) Updating files: 64% (157/244) Updating files: 65% (159/244) Updating files: 66% (162/244) Updating files: 67% (164/244) Updating files: 68% (166/244) Updating files: 69% (169/244) Updating files: 70% (171/244) Updating files: 71% (174/244) Updating files: 72% (176/244) Updating files: 73% (179/244) Updating files: 74% (181/244) Updating files: 75% (183/244) Updating files: 76% (186/244) Updating files: 77% (188/244) Updating files: 78% (191/244) Updating files: 79% (193/244) Updating files: 80% (196/244) Updating files: 81% (198/244) Updating files: 82% (201/244) Updating files: 83% (203/244) Updating files: 84% (205/244) Updating files: 85% (208/244) Updating files: 86% (210/244) Updating files: 86% (211/244) Updating files: 87% (213/244) Updating files: 88% (215/244) Updating files: 89% (218/244) Updating files: 90% (220/244) Updating files: 91% (223/244) Updating files: 92% (225/244) Updating files: 93% (227/244) Updating files: 94% (230/244) Updating files: 95% (232/244) Updating files: 96% (235/244) Updating files: 97% (237/244) Updating files: 98% (240/244) Updating files: 99% (242/244) Updating files: 100% (244/244) Updating files: 100% (244/244), done. 2024-03-04T07:58:19.817 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 877cacbbdbb v0.36 2024-03-04T07:58:19.822 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.36 2024-03-04T07:58:20.082 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 37 -eq 48 ] 2024-03-04T07:58:20.082 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.37 2024-03-04T07:58:20.083 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.37 2024-03-04T07:58:20.083 INFO:tasks.workunit.client.0.smithi113.stdout:v0.37 2024-03-04T07:58:20.083 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.37 2024-03-04T07:58:22.954 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 12% (23/181) Updating files: 13% (24/181) Updating files: 14% (26/181) Updating files: 15% (28/181) Updating files: 16% (29/181) Updating files: 17% (31/181) Updating files: 18% (33/181) Updating files: 19% (35/181) Updating files: 20% (37/181) Updating files: 21% (39/181) Updating files: 22% (40/181) Updating files: 23% (42/181) Updating files: 24% (44/181) Updating files: 25% (46/181) Updating files: 26% (48/181) Updating files: 27% (49/181) Updating files: 28% (51/181) Updating files: 29% (53/181) Updating files: 30% (55/181) Updating files: 31% (57/181) Updating files: 32% (58/181) Updating files: 33% (60/181) Updating files: 34% (62/181) Updating files: 35% (64/181) Updating files: 36% (66/181) Updating files: 37% (67/181) Updating files: 38% (69/181) Updating files: 39% (71/181) Updating files: 40% (73/181) Updating files: 41% (75/181) Updating files: 42% (77/181) Updating files: 43% (78/181) Updating files: 44% (80/181) Updating files: 45% (82/181) Updating files: 46% (84/181) Updating files: 47% (86/181) Updating files: 48% (87/181) Updating files: 49% (89/181) Updating files: 50% (91/181) Updating files: 51% (93/181) Updating files: 52% (95/181) Updating files: 53% (96/181) Updating files: 54% (98/181) Updating files: 55% (100/181) Updating files: 56% (102/181) Updating files: 57% (104/181) Updating files: 58% (105/181) Updating files: 59% (107/181) Updating files: 60% (109/181) Updating files: 61% (111/181) Updating files: 62% (113/181) Updating files: 63% (115/181) Updating files: 64% (116/181) Updating files: 65% (118/181) Updating files: 66% (120/181) Updating files: 67% (122/181) Updating files: 68% (124/181) Updating files: 69% (125/181) Updating files: 70% (127/181) Updating files: 71% (129/181) Updating files: 72% (131/181) Updating files: 73% (133/181) Updating files: 74% (134/181) Updating files: 75% (136/181) Updating files: 76% (138/181) Updating files: 77% (140/181) Updating files: 78% (142/181) Updating files: 79% (143/181) Updating files: 80% (145/181) Updating files: 81% (147/181) Updating files: 82% (149/181) Updating files: 83% (151/181) Updating files: 84% (153/181) Updating files: 85% (154/181) Updating files: 86% (156/181) Updating files: 87% (158/181) Updating files: 88% (160/181) Updating files: 89% (162/181) Updating files: 90% (163/181) Updating files: 91% (165/181) Updating files: 92% (167/181) Updating files: 93% (169/181) Updating files: 94% (171/181) Updating files: 95% (172/181) Updating files: 96% (174/181) Updating files: 97% (176/181) Updating files: 98% (178/181) Updating files: 99% (180/181) Updating files: 100% (181/181) Updating files: 100% (181/181), done. 2024-03-04T07:58:22.987 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at a6f3bbb744a v0.37 2024-03-04T07:58:22.994 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.37 2024-03-04T07:58:24.326 INFO:tasks.workunit.client.0.smithi113.stdout:v0.38 2024-03-04T07:58:24.326 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 38 -eq 48 ] 2024-03-04T07:58:24.327 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.38 2024-03-04T07:58:24.327 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.38 2024-03-04T07:58:24.327 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.38 2024-03-04T07:58:27.246 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:27.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe17200640 1 Processor -- start 2024-03-04T07:58:27.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe17200640 1 -- start start 2024-03-04T07:58:27.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe17200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbe1014e4c0 0x7fbe10072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe17200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe100c27f0 con 0x7fbe100c22e0 2024-03-04T07:58:27.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe17200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe100c2a90 con 0x7fbe1015b900 2024-03-04T07:58:27.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe17200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1015ad60 con 0x7fbe1014e4c0 2024-03-04T07:58:27.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe07400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbe100c22e0 0x7fbe1015b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38804/0 (socket says 172.21.15.113:38804) 2024-03-04T07:58:27.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe07400640 1 -- 172.21.15.113:0/1759607430 learned_addr learned my addr 172.21.15.113:0/1759607430 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:58:27.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe0fe00640 1 --2- 172.21.15.113:0/1759607430 >> v2:172.21.15.136:3300/0 conn(0x7fbe1014e4c0 0x7fbe10072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe0f400640 1 -- 172.21.15.113:0/1759607430 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293025839 0 0) 0x7fbe100c27f0 con 0x7fbe100c22e0 2024-03-04T07:58:27.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe0f400640 1 -- 172.21.15.113:0/1759607430 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf4003530 con 0x7fbe100c22e0 2024-03-04T07:58:27.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe0f400640 1 -- 172.21.15.113:0/1759607430 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2406378814 0 0) 0x7fbdf4003530 con 0x7fbe100c22e0 2024-03-04T07:58:27.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe0fe00640 1 -- 172.21.15.113:0/1759607430 >> v1:172.21.15.136:6789/0 conn(0x7fbe1015b900 legacy=0x7fbe101620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe0fe00640 1 -- 172.21.15.113:0/1759607430 >> v1:172.21.15.113:6789/0 conn(0x7fbe100c22e0 legacy=0x7fbe1015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.405+0000 7fbe0fe00640 1 -- 172.21.15.113:0/1759607430 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe10070c60 con 0x7fbe1014e4c0 2024-03-04T07:58:27.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe0fe00640 1 --2- 172.21.15.113:0/1759607430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe1014e4c0 0x7fbe10072c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbe000097c0 tx=0x7fbe0001ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd0947e6e4a178a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe0f400640 1 -- 172.21.15.113:0/1759607430 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000b5070 con 0x7fbe1014e4c0 2024-03-04T07:58:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe0f400640 1 -- 172.21.15.113:0/1759607430 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe000a8630 con 0x7fbe1014e4c0 2024-03-04T07:58:27.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe0f400640 1 -- 172.21.15.113:0/1759607430 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8001680 con 0x7fbe1014e4c0 2024-03-04T07:58:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe17200640 1 -- 172.21.15.113:0/1759607430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe1014e4c0 msgr2=0x7fbe10072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe17200640 1 --2- 172.21.15.113:0/1759607430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe1014e4c0 0x7fbe10072c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbe000097c0 tx=0x7fbe0001ebb0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe17200640 1 -- 172.21.15.113:0/1759607430 shutdown_connections 2024-03-04T07:58:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe17200640 1 --2- 172.21.15.113:0/1759607430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe1014e4c0 0x7fbe10072c80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe17200640 1 -- 172.21.15.113:0/1759607430 >> 172.21.15.113:0/1759607430 conn(0x7fbe1006e820 msgr2=0x7fbe1006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe17200640 1 -- 172.21.15.113:0/1759607430 shutdown_connections 2024-03-04T07:58:27.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.409+0000 7fbe17200640 1 -- 172.21.15.113:0/1759607430 wait complete. 2024-03-04T07:58:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.413+0000 7fbe17200640 1 Processor -- start 2024-03-04T07:58:27.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.413+0000 7fbe17200640 1 -- start start 2024-03-04T07:58:27.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.413+0000 7fbe17200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe100c22e0 0x7fbe10153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.413+0000 7fbe17200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1014e4c0 0x7fbe10152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.413+0000 7fbe17200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe1015b900 0x7fbe101531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.413+0000 7fbe17200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe10071500 con 0x7fbe1014e4c0 2024-03-04T07:58:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.413+0000 7fbe17200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe10071220 con 0x7fbe100c22e0 2024-03-04T07:58:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.413+0000 7fbe17200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1014e8a0 con 0x7fbe1015b900 2024-03-04T07:58:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.413+0000 7fbe07400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1014e4c0 0x7fbe10152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe07400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1014e4c0 0x7fbe10152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57832/0 (socket says 172.21.15.113:57832) 2024-03-04T07:58:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe07400640 1 -- 172.21.15.113:0/3753467580 learned_addr learned my addr 172.21.15.113:0/3753467580 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe0fe00640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe100c22e0 0x7fbe10153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe15200640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe1015b900 0x7fbe101531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe07400640 1 -- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe1015b900 msgr2=0x7fbe101531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe07400640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe1015b900 0x7fbe101531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe07400640 1 -- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe100c22e0 msgr2=0x7fbe10153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe07400640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe100c22e0 0x7fbe10153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe07400640 1 -- 172.21.15.113:0/3753467580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe100b2290 con 0x7fbe1014e4c0 2024-03-04T07:58:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe07400640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1014e4c0 0x7fbe10152c90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800c700 tx=0x7fbdf800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe0cc00640 1 -- 172.21.15.113:0/3753467580 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8005ae0 con 0x7fbe1014e4c0 2024-03-04T07:58:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe0cc00640 1 -- 172.21.15.113:0/3753467580 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf8005c80 con 0x7fbe1014e4c0 2024-03-04T07:58:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe0cc00640 1 -- 172.21.15.113:0/3753467580 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8007290 con 0x7fbe1014e4c0 2024-03-04T07:58:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe17200640 1 -- 172.21.15.113:0/3753467580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe10154590 con 0x7fbe1014e4c0 2024-03-04T07:58:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe17200640 1 -- 172.21.15.113:0/3753467580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe10154b50 con 0x7fbe1014e4c0 2024-03-04T07:58:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.417+0000 7fbe17200640 1 -- 172.21.15.113:0/3753467580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe10075c00 con 0x7fbe1014e4c0 2024-03-04T07:58:27.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.421+0000 7fbe0cc00640 1 -- 172.21.15.113:0/3753467580 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbdf8007430 con 0x7fbe1014e4c0 2024-03-04T07:58:27.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.421+0000 7fbe0cc00640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdec066f60 0x7fbdec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.421+0000 7fbe0cc00640 1 -- 172.21.15.113:0/3753467580 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fbdf80851c0 con 0x7fbe1014e4c0 2024-03-04T07:58:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.421+0000 7fbe0fe00640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdec066f60 0x7fbdec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.421+0000 7fbe0fe00640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdec066f60 0x7fbdec069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbe000b2000 tx=0x7fbe000aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.425+0000 7fbe0cc00640 1 -- 172.21.15.113:0/3753467580 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbdf8007720 con 0x7fbe1014e4c0 2024-03-04T07:58:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.661+0000 7fbe17200640 1 -- 172.21.15.113:0/3753467580 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe10155190 con 0x7fbe1014e4c0 2024-03-04T07:58:27.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.661+0000 7fbe0cc00640 1 -- 172.21.15.113:0/3753467580 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6615 (secure 0 0 0) 0x7fbdf8055670 con 0x7fbe1014e4c0 2024-03-04T07:58:27.661 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:27.661 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6534,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240,6145]},"id":1}]} 2024-03-04T07:58:27.662 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 88 2024-03-04T07:58:27.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.661+0000 7fbe17200640 1 -- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdec066f60 msgr2=0x7fbdec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.661+0000 7fbe17200640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdec066f60 0x7fbdec069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbe000b2000 tx=0x7fbe000aa070 comp rx=0 tx=0).stop 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.661+0000 7fbe17200640 1 -- 172.21.15.113:0/3753467580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1014e4c0 msgr2=0x7fbe10152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.661+0000 7fbe17200640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1014e4c0 0x7fbe10152c90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800c700 tx=0x7fbdf800cbc0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.665+0000 7fbe17200640 1 -- 172.21.15.113:0/3753467580 shutdown_connections 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.665+0000 7fbe17200640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe1015b900 0x7fbe101531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.665+0000 7fbe17200640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbdec066f60 0x7fbdec069420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.665+0000 7fbe17200640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe1014e4c0 0x7fbe10152c90 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.665+0000 7fbe17200640 1 --2- 172.21.15.113:0/3753467580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe100c22e0 0x7fbe10153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.665+0000 7fbe17200640 1 -- 172.21.15.113:0/3753467580 >> 172.21.15.113:0/3753467580 conn(0x7fbe1006e820 msgr2=0x7fbe100c70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.665+0000 7fbe17200640 1 -- 172.21.15.113:0/3753467580 shutdown_connections 2024-03-04T07:58:27.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.665+0000 7fbe17200640 1 -- 172.21.15.113:0/3753467580 wait complete. 2024-03-04T07:58:27.682 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=1) 2024-03-04T07:58:27.683 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-04T07:58:27.683 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T07:58:27.695 INFO:tasks.ceph.mds.b:Stopped 2024-03-04T07:58:27.695 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:27.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20b3600640 1 Processor -- start 2024-03-04T07:58:27.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20b3600640 1 -- start start 2024-03-04T07:58:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20b3600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac06ff20 0x7f20ac15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20b3600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20ac0c8b90 0x7f20ac15c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20b3600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ac152e30 con 0x7f20ac152a50 2024-03-04T07:58:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20b3600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ac0c31b0 con 0x7f20ac0c8b90 2024-03-04T07:58:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20b3600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac06f840 con 0x7f20ac06ff20 2024-03-04T07:58:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20b1600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f20ac0c8b90 0x7f20ac15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20b0c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f20ac152a50 0x7f20ac0ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38818/0 (socket says 172.21.15.113:38818) 2024-03-04T07:58:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20b0c00640 1 -- 172.21.15.113:0/3545621390 learned_addr learned my addr 172.21.15.113:0/3545621390 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:58:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.857+0000 7f20a3e00640 1 --2- 172.21.15.113:0/3545621390 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac06ff20 0x7f20ac15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.861+0000 7f20b1600640 1 -- 172.21.15.113:0/3545621390 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac06ff20 msgr2=0x7f20ac15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.861+0000 7f20b1600640 1 --2- 172.21.15.113:0/3545621390 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac06ff20 0x7f20ac15bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.861+0000 7f20b1600640 1 -- 172.21.15.113:0/3545621390 >> v1:172.21.15.113:6789/0 conn(0x7f20ac152a50 legacy=0x7f20ac0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.861+0000 7f20abe00640 1 -- 172.21.15.113:0/3545621390 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32447805 0 0) 0x7f20ac152e30 con 0x7f20ac152a50 2024-03-04T07:58:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.861+0000 7f20b1600640 1 -- 172.21.15.113:0/3545621390 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac06f280 con 0x7f20ac0c8b90 2024-03-04T07:58:27.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20b1600640 1 --2- 172.21.15.113:0/3545621390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 0x7f20ac15c310 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009d90 tx=0x7f20a401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95be45b726af310a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:27.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20abe00640 1 -- 172.21.15.113:0/3545621390 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40b2070 con 0x7f20ac0c8b90 2024-03-04T07:58:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20abe00640 1 -- 172.21.15.113:0/3545621390 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a40a59c0 con 0x7f20ac0c8b90 2024-03-04T07:58:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20abe00640 1 -- 172.21.15.113:0/3545621390 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40a5d90 con 0x7f20ac0c8b90 2024-03-04T07:58:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20b3600640 1 -- 172.21.15.113:0/3545621390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 msgr2=0x7f20ac15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20b3600640 1 --2- 172.21.15.113:0/3545621390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 0x7f20ac15c310 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009d90 tx=0x7f20a401c8e0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20b3600640 1 -- 172.21.15.113:0/3545621390 shutdown_connections 2024-03-04T07:58:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20b3600640 1 --2- 172.21.15.113:0/3545621390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 0x7f20ac15c310 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20b3600640 1 --2- 172.21.15.113:0/3545621390 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac06ff20 0x7f20ac15bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20b3600640 1 -- 172.21.15.113:0/3545621390 >> 172.21.15.113:0/3545621390 conn(0x7f20ac0734d0 msgr2=0x7f20ac066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20b3600640 1 -- 172.21.15.113:0/3545621390 shutdown_connections 2024-03-04T07:58:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.865+0000 7f20b3600640 1 -- 172.21.15.113:0/3545621390 wait complete. 2024-03-04T07:58:27.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20b3600640 1 Processor -- start 2024-03-04T07:58:27.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20b3600640 1 -- start start 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20b3600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac06ff20 0x7f20ac0c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20b3600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 0x7f20ac1575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20b3600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20ac152a50 0x7f20ac162200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20b3600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0ce9b0 con 0x7f20ac0c8b90 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20b3600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0ce830 con 0x7f20ac152a50 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20b3600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac165880 con 0x7f20ac06ff20 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 0x7f20ac1575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 0x7f20ac1575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57856/0 (socket says 172.21.15.113:57856) 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20a3e00640 1 -- 172.21.15.113:0/3185393233 learned_addr learned my addr 172.21.15.113:0/3185393233 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20b0c00640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac06ff20 0x7f20ac0c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20b1600640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20ac152a50 0x7f20ac162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20a3e00640 1 -- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac06ff20 msgr2=0x7f20ac0c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20a3e00640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac06ff20 0x7f20ac0c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20a3e00640 1 -- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20ac152a50 msgr2=0x7f20ac162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:27.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20a3e00640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20ac152a50 0x7f20ac162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:27.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20a3e00640 1 -- 172.21.15.113:0/3185393233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac158990 con 0x7f20ac0c8b90 2024-03-04T07:58:27.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.869+0000 7f20a3e00640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 0x7f20ac1575c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f209400e960 tx=0x7f209400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:27.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.873+0000 7f20a9600640 1 -- 172.21.15.113:0/3185393233 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094017070 con 0x7f20ac0c8b90 2024-03-04T07:58:27.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.873+0000 7f20a9600640 1 -- 172.21.15.113:0/3185393233 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209400ca00 con 0x7f20ac0c8b90 2024-03-04T07:58:27.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.873+0000 7f20a9600640 1 -- 172.21.15.113:0/3185393233 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209400cdb0 con 0x7f20ac0c8b90 2024-03-04T07:58:27.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.873+0000 7f20b3600640 1 -- 172.21.15.113:0/3185393233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac1f37a0 con 0x7f20ac0c8b90 2024-03-04T07:58:27.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.873+0000 7f20b3600640 1 -- 172.21.15.113:0/3185393233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac158c30 con 0x7f20ac0c8b90 2024-03-04T07:58:27.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.873+0000 7f20b3600640 1 -- 172.21.15.113:0/3185393233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ac075c00 con 0x7f20ac0c8b90 2024-03-04T07:58:27.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.873+0000 7f20a9600640 1 -- 172.21.15.113:0/3185393233 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f209401a070 con 0x7f20ac0c8b90 2024-03-04T07:58:27.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.873+0000 7f20a9600640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2088066f60 0x7f2088069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:27.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.873+0000 7f20a9600640 1 -- 172.21.15.113:0/3185393233 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f2094085420 con 0x7f20ac0c8b90 2024-03-04T07:58:27.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.873+0000 7f20b0c00640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2088066f60 0x7f2088069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:27.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.877+0000 7f20b0c00640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2088066f60 0x7f2088069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f209c000f70 tx=0x7f209c007df0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:27.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:27.877+0000 7f20a9600640 1 -- 172.21.15.113:0/3185393233 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2094055960 con 0x7f20ac0c8b90 2024-03-04T07:58:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.069+0000 7f20b3600640 1 -- 172.21.15.113:0/3185393233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20ac0caa40 con 0x7f20ac0c8b90 2024-03-04T07:58:28.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.069+0000 7f20a9600640 1 -- 172.21.15.113:0/3185393233 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f2094059a00 con 0x7f20ac0c8b90 2024-03-04T07:58:28.070 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:28.070 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6534,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240,6145]},"id":1}]} 2024-03-04T07:58:28.070 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 88 2024-03-04T07:58:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 -- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2088066f60 msgr2=0x7f2088069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2088066f60 0x7f2088069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f209c000f70 tx=0x7f209c007df0 comp rx=0 tx=0).stop 2024-03-04T07:58:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 -- 172.21.15.113:0/3185393233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 msgr2=0x7f20ac1575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 0x7f20ac1575c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f209400e960 tx=0x7f209400ee20 comp rx=0 tx=0).stop 2024-03-04T07:58:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 -- 172.21.15.113:0/3185393233 shutdown_connections 2024-03-04T07:58:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20ac152a50 0x7f20ac162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2088066f60 0x7f2088069420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f20ac0c8b90 0x7f20ac1575c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 --2- 172.21.15.113:0/3185393233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f20ac06ff20 0x7f20ac0c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 -- 172.21.15.113:0/3185393233 >> 172.21.15.113:0/3185393233 conn(0x7f20ac0734d0 msgr2=0x7f20ac0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 -- 172.21.15.113:0/3185393233 shutdown_connections 2024-03-04T07:58:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:28.073+0000 7f20b3600640 1 -- 172.21.15.113:0/3185393233 wait complete. 2024-03-04T07:58:28.091 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:58:30.092 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:30.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.249+0000 7f061be00640 1 Processor -- start 2024-03-04T07:58:30.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.249+0000 7f061be00640 1 -- start start 2024-03-04T07:58:30.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.249+0000 7f061be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f061406dd50 0x7f06141662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:30.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.249+0000 7f061be00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06140ca0e0 con 0x7f0614158010 2024-03-04T07:58:30.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.249+0000 7f061be00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061415aa30 con 0x7f061406ff20 2024-03-04T07:58:30.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.249+0000 7f061be00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061406f450 con 0x7f061406dd50 2024-03-04T07:58:30.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.249+0000 7f0619e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f061406dd50 0x7f06141662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:30.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.249+0000 7f0619400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0614158010 0x7f061415b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38832/0 (socket says 172.21.15.113:38832) 2024-03-04T07:58:30.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.249+0000 7f0619400640 1 -- 172.21.15.113:0/867469594 learned_addr learned my addr 172.21.15.113:0/867469594 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:58:30.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f0619e00640 1 -- 172.21.15.113:0/867469594 >> v1:172.21.15.136:6789/0 conn(0x7f061406ff20 legacy=0x7f06140ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:30.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f0619e00640 1 -- 172.21.15.113:0/867469594 >> v1:172.21.15.113:6789/0 conn(0x7f0614158010 legacy=0x7f061415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:30.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f0619e00640 1 -- 172.21.15.113:0/867469594 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061406c590 con 0x7f061406dd50 2024-03-04T07:58:30.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f0619e00640 1 --2- 172.21.15.113:0/867469594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 0x7f06141662e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f060c009d90 tx=0x7f060c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e876cebe005a24 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:30.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f0613e00640 1 -- 172.21.15.113:0/867469594 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c0b2070 con 0x7f061406dd50 2024-03-04T07:58:30.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f0613e00640 1 -- 172.21.15.113:0/867469594 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060c0a5830 con 0x7f061406dd50 2024-03-04T07:58:30.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f0613e00640 1 -- 172.21.15.113:0/867469594 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c0a5c00 con 0x7f061406dd50 2024-03-04T07:58:30.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f061be00640 1 -- 172.21.15.113:0/867469594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 msgr2=0x7f06141662e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:30.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f061be00640 1 --2- 172.21.15.113:0/867469594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 0x7f06141662e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f060c009d90 tx=0x7f060c01c970 comp rx=0 tx=0).stop 2024-03-04T07:58:30.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f061be00640 1 -- 172.21.15.113:0/867469594 shutdown_connections 2024-03-04T07:58:30.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f061be00640 1 --2- 172.21.15.113:0/867469594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 0x7f06141662e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:30.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f061be00640 1 -- 172.21.15.113:0/867469594 >> 172.21.15.113:0/867469594 conn(0x7f06140734d0 msgr2=0x7f0614066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:30.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f061be00640 1 -- 172.21.15.113:0/867469594 shutdown_connections 2024-03-04T07:58:30.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.253+0000 7f061be00640 1 -- 172.21.15.113:0/867469594 wait complete. 2024-03-04T07:58:30.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.257+0000 7f061be00640 1 Processor -- start 2024-03-04T07:58:30.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.257+0000 7f061be00640 1 -- start start 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.257+0000 7f061be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 0x7f06140cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.257+0000 7f061be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f061406ff20 0x7f061414e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f061be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0614158010 0x7f061414ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f061be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061415b920 con 0x7f061406dd50 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f061be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061415ad80 con 0x7f0614158010 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f061be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06141583f0 con 0x7f061406ff20 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 0x7f06140cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 0x7f06140cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57870/0 (socket says 172.21.15.113:57870) 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619400640 1 -- 172.21.15.113:0/1577723668 learned_addr learned my addr 172.21.15.113:0/1577723668 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619e00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0614158010 0x7f061414ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:30.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0618a00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f061406ff20 0x7f061414e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619400640 1 -- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f061406ff20 msgr2=0x7f061414e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619400640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f061406ff20 0x7f061414e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619400640 1 -- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0614158010 msgr2=0x7f061414ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619400640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0614158010 0x7f061414ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619400640 1 -- 172.21.15.113:0/1577723668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061414fd10 con 0x7f061406dd50 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619e00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0614158010 0x7f061414ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0619400640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 0x7f06140cd7e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f060800c430 tx=0x7f060800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0611600640 1 -- 172.21.15.113:0/1577723668 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608017070 con 0x7f061406dd50 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0611600640 1 -- 172.21.15.113:0/1577723668 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0608005cf0 con 0x7f061406dd50 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f0611600640 1 -- 172.21.15.113:0/1577723668 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06080072d0 con 0x7f061406dd50 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f061be00640 1 -- 172.21.15.113:0/1577723668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061414fff0 con 0x7f061406dd50 2024-03-04T07:58:30.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f061be00640 1 -- 172.21.15.113:0/1577723668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06141505b0 con 0x7f061406dd50 2024-03-04T07:58:30.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.261+0000 7f061be00640 1 -- 172.21.15.113:0/1577723668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0614075c00 con 0x7f061406dd50 2024-03-04T07:58:30.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.265+0000 7f0611600640 1 -- 172.21.15.113:0/1577723668 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f060801a070 con 0x7f061406dd50 2024-03-04T07:58:30.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.265+0000 7f0611600640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f05f4066f60 0x7f05f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:30.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.265+0000 7f0611600640 1 -- 172.21.15.113:0/1577723668 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f0608084790 con 0x7f061406dd50 2024-03-04T07:58:30.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.265+0000 7f0618a00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f05f4066f60 0x7f05f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:30.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.265+0000 7f0618a00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f05f4066f60 0x7f05f4069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0614146550 tx=0x7f0600009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:30.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.273+0000 7f0611600640 1 -- 172.21.15.113:0/1577723668 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0608054c30 con 0x7f061406dd50 2024-03-04T07:58:30.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.521+0000 7f061be00640 1 -- 172.21.15.113:0/1577723668 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f061414db30 con 0x7f061406dd50 2024-03-04T07:58:30.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.521+0000 7f0611600640 1 -- 172.21.15.113:0/1577723668 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f0608058cd0 con 0x7f061406dd50 2024-03-04T07:58:30.521 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:30.522 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6534,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240,6145]},"id":1}]} 2024-03-04T07:58:30.522 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 88 2024-03-04T07:58:30.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 -- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f05f4066f60 msgr2=0x7f05f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:30.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f05f4066f60 0x7f05f4069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0614146550 tx=0x7f0600009040 comp rx=0 tx=0).stop 2024-03-04T07:58:30.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 -- 172.21.15.113:0/1577723668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 msgr2=0x7f06140cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:30.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 0x7f06140cd7e0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f060800c430 tx=0x7f060800c8f0 comp rx=0 tx=0).stop 2024-03-04T07:58:30.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 -- 172.21.15.113:0/1577723668 shutdown_connections 2024-03-04T07:58:30.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0614158010 0x7f061414ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:30.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f061406ff20 0x7f061414e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:30.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f05f4066f60 0x7f05f4069420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:30.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 --2- 172.21.15.113:0/1577723668 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f061406dd50 0x7f06140cd7e0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:30.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 -- 172.21.15.113:0/1577723668 >> 172.21.15.113:0/1577723668 conn(0x7f06140734d0 msgr2=0x7f06140c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:30.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 -- 172.21.15.113:0/1577723668 shutdown_connections 2024-03-04T07:58:30.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:30.525+0000 7f061be00640 1 -- 172.21.15.113:0/1577723668 wait complete. 2024-03-04T07:58:30.551 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:58:31.989 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:58:31.992 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:58:32.552 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:32.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f013be00640 1 Processor -- start 2024-03-04T07:58:32.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f013be00640 1 -- start start 2024-03-04T07:58:32.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f013be00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f01340703e0 0x7f01341641b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:32.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f013be00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01340c5470 con 0x7f0134152d00 2024-03-04T07:58:32.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f013be00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01340cba40 con 0x7f01340ca0d0 2024-03-04T07:58:32.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f013be00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01340c5220 con 0x7f01340703e0 2024-03-04T07:58:32.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0139400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0134152d00 0x7f01340ca5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38842/0 (socket says 172.21.15.113:38842) 2024-03-04T07:58:32.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0139400640 1 -- 172.21.15.113:0/3065179013 learned_addr learned my addr 172.21.15.113:0/3065179013 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:58:32.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0139e00640 1 --2- 172.21.15.113:0/3065179013 >> v2:172.21.15.136:3300/0 conn(0x7f01340703e0 0x7f01341641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:32.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800313152 0 0) 0x7f01340c5470 con 0x7f0134152d00 2024-03-04T07:58:32.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f011c003530 con 0x7f0134152d00 2024-03-04T07:58:32.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2365400267 0 0) 0x7f011c003530 con 0x7f0134152d00 2024-03-04T07:58:32.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 >> v2:172.21.15.136:3300/0 conn(0x7f01340703e0 msgr2=0x7f01341641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:32.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 --2- 172.21.15.113:0/3065179013 >> v2:172.21.15.136:3300/0 conn(0x7f01340703e0 0x7f01341641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 >> v1:172.21.15.136:6789/0 conn(0x7f01340ca0d0 legacy=0x7f01340ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:32.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013406f690 con 0x7f0134152d00 2024-03-04T07:58:32.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f0128001680 con 0x7f0134152d00 2024-03-04T07:58:32.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 >> v1:172.21.15.113:6789/0 conn(0x7f0134152d00 legacy=0x7f01340ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:32.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f011c003a10 0x7f011c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:32.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f011c004810 0x7f011c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:32.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f011c005220 0x7f011c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:32.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01340c5220 con 0x7f011c004810 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c00c170 con 0x7f011c005220 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011c00c370 con 0x7f011c003a10 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0138a00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f011c004810 0x7f011c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0139400640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f011c003a10 0x7f011c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0139e00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f011c005220 0x7f011c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0138a00640 1 -- 172.21.15.113:0/3065179013 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f011c003a10 msgr2=0x7f011c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0138a00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f011c003a10 0x7f011c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0138a00640 1 -- 172.21.15.113:0/3065179013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f011c005220 msgr2=0x7f011c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0138a00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f011c005220 0x7f011c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0138a00640 1 -- 172.21.15.113:0/3065179013 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c003530 con 0x7f011c004810 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0139e00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f011c005220 0x7f011c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:58:32.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.693+0000 7f0139e00640 1 -- 172.21.15.113:0/3065179013 reap_dead start 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f0138a00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f011c004810 0x7f011c004c90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0120001750 tx=0x7f012001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53131c96cb53762 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01200013a0 con 0x7f011c004810 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01200052f0 con 0x7f011c004810 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f0133e00640 1 -- 172.21.15.113:0/3065179013 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120003840 con 0x7f011c004810 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f013be00640 1 -- 172.21.15.113:0/3065179013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f011c004810 msgr2=0x7f011c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f013be00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f011c004810 0x7f011c004c90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f0120001750 tx=0x7f012001c3c0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f013be00640 1 -- 172.21.15.113:0/3065179013 shutdown_connections 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f013be00640 1 --2- 172.21.15.113:0/3065179013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f011c004810 0x7f011c004c90 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f013be00640 1 -- 172.21.15.113:0/3065179013 >> 172.21.15.113:0/3065179013 conn(0x7f0134066db0 msgr2=0x7f01340671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f013be00640 1 -- 172.21.15.113:0/3065179013 shutdown_connections 2024-03-04T07:58:32.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.697+0000 7f013be00640 1 -- 172.21.15.113:0/3065179013 wait complete. 2024-03-04T07:58:32.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.701+0000 7f013be00640 1 Processor -- start 2024-03-04T07:58:32.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.701+0000 7f013be00640 1 -- start start 2024-03-04T07:58:32.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.701+0000 7f013be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f011c004810 0x7f0134146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:32.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f013be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01341f0fe0 0x7f01341f1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:32.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f013be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01341f25d0 0x7f01341f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:32.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f013be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01340ce3c0 con 0x7f01341f25d0 2024-03-04T07:58:32.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f013be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013406e180 con 0x7f01341f0fe0 2024-03-04T07:58:32.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f013be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01340ce540 con 0x7f011c004810 2024-03-04T07:58:32.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0139e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01341f25d0 0x7f01341f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:32.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0139400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f011c004810 0x7f0134146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:32.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0139e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01341f25d0 0x7f01341f2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57886/0 (socket says 172.21.15.113:57886) 2024-03-04T07:58:32.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0139e00640 1 -- 172.21.15.113:0/2940418599 learned_addr learned my addr 172.21.15.113:0/2940418599 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:32.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0138a00640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01341f0fe0 0x7f01341f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:32.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0139e00640 1 -- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f011c004810 msgr2=0x7f0134146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:32.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0139e00640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f011c004810 0x7f0134146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0139e00640 1 -- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01341f0fe0 msgr2=0x7f01341f1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:32.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0139e00640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01341f0fe0 0x7f01341f1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0139e00640 1 -- 172.21.15.113:0/2940418599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0134149ed0 con 0x7f01341f25d0 2024-03-04T07:58:32.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0139e00640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01341f25d0 0x7f01341f2a50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f013406e5a0 tx=0x7f012c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:32.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0131600640 1 -- 172.21.15.113:0/2940418599 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c010b90 con 0x7f01341f25d0 2024-03-04T07:58:32.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0131600640 1 -- 172.21.15.113:0/2940418599 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c010d50 con 0x7f01341f25d0 2024-03-04T07:58:32.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f0131600640 1 -- 172.21.15.113:0/2940418599 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c003be0 con 0x7f01341f25d0 2024-03-04T07:58:32.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f013be00640 1 -- 172.21.15.113:0/2940418599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01340747a0 con 0x7f01341f25d0 2024-03-04T07:58:32.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f013be00640 1 -- 172.21.15.113:0/2940418599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013414a190 con 0x7f01341f25d0 2024-03-04T07:58:32.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.705+0000 7f013be00640 1 -- 172.21.15.113:0/2940418599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0134075c00 con 0x7f01341f25d0 2024-03-04T07:58:32.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.709+0000 7f0131600640 1 -- 172.21.15.113:0/2940418599 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f012c01e020 con 0x7f01341f25d0 2024-03-04T07:58:32.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.709+0000 7f0131600640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0114066f90 0x7f0114069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:32.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.709+0000 7f0131600640 1 -- 172.21.15.113:0/2940418599 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f012c085950 con 0x7f01341f25d0 2024-03-04T07:58:32.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.709+0000 7f0139400640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0114066f90 0x7f0114069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:32.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.709+0000 7f0139400640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0114066f90 0x7f0114069450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0128000c10 tx=0x7f0128001dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:32.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.713+0000 7f0131600640 1 -- 172.21.15.113:0/2940418599 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f012c055e90 con 0x7f01341f25d0 2024-03-04T07:58:32.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.929+0000 7f013be00640 1 -- 172.21.15.113:0/2940418599 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0134151720 con 0x7f01341f25d0 2024-03-04T07:58:32.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f0131600640 1 -- 172.21.15.113:0/2940418599 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f012c083e20 con 0x7f01341f25d0 2024-03-04T07:58:32.932 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:32.932 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6534,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240,6145]},"id":1}]} 2024-03-04T07:58:32.932 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 88 2024-03-04T07:58:32.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 -- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0114066f90 msgr2=0x7f0114069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:32.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0114066f90 0x7f0114069450 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0128000c10 tx=0x7f0128001dc0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 -- 172.21.15.113:0/2940418599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01341f25d0 msgr2=0x7f01341f2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:32.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01341f25d0 0x7f01341f2a50 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f013406e5a0 tx=0x7f012c010590 comp rx=0 tx=0).stop 2024-03-04T07:58:32.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 -- 172.21.15.113:0/2940418599 shutdown_connections 2024-03-04T07:58:32.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0114066f90 0x7f0114069450 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01341f25d0 0x7f01341f2a50 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01341f0fe0 0x7f01341f1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 --2- 172.21.15.113:0/2940418599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f011c004810 0x7f0134146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:32.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 -- 172.21.15.113:0/2940418599 >> 172.21.15.113:0/2940418599 conn(0x7f0134066db0 msgr2=0x7f01340ccd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:32.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 -- 172.21.15.113:0/2940418599 shutdown_connections 2024-03-04T07:58:32.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:32.933+0000 7f013be00640 1 -- 172.21.15.113:0/2940418599 wait complete. 2024-03-04T07:58:32.953 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:58:34.954 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:35.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.105+0000 7f0c18c00640 1 Processor -- start 2024-03-04T07:58:35.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.105+0000 7f0c18c00640 1 -- start start 2024-03-04T07:58:35.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.105+0000 7f0c18c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1406ff20 0x7f0c1406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:35.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.105+0000 7f0c18c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0c140ce120 0x7f0c140ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:35.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.105+0000 7f0c18c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c140ca0e0 con 0x7f0c14158010 2024-03-04T07:58:35.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.105+0000 7f0c18c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1406f450 con 0x7f0c140ce120 2024-03-04T07:58:35.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.105+0000 7f0c18c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1415ad80 con 0x7f0c1406ff20 2024-03-04T07:58:35.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.105+0000 7f0c11e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0c14158010 0x7f0c1415b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38850/0 (socket says 172.21.15.113:38850) 2024-03-04T07:58:35.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c11e00640 1 -- 172.21.15.113:0/1386025024 learned_addr learned my addr 172.21.15.113:0/1386025024 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:58:35.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c11400640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1406ff20 0x7f0c1406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:35.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c12800640 1 --2- 172.21.15.113:0/1386025024 >> v2:172.21.15.136:3300/0 conn(0x7f0c140ce120 0x7f0c140ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:35.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239375000 0 0) 0x7f0c140ca0e0 con 0x7f0c14158010 2024-03-04T07:58:35.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bf8003530 con 0x7f0c14158010 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4012281415 0 0) 0x7f0bf8003530 con 0x7f0c14158010 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1406ff20 msgr2=0x7f0c1406dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1406ff20 0x7f0c1406dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 >> v2:172.21.15.136:3300/0 conn(0x7f0c140ce120 msgr2=0x7f0c140ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 --2- 172.21.15.113:0/1386025024 >> v2:172.21.15.136:3300/0 conn(0x7f0c140ce120 0x7f0c140ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1415aa30 con 0x7f0c14158010 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f0c04001680 con 0x7f0c14158010 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 >> v1:172.21.15.113:6789/0 conn(0x7f0c14158010 legacy=0x7f0c1415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf80039c0 0x7f0bf8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1406ff20 0x7f0bf8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bf8004d50 0x7f0bf80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:35.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1406f450 con 0x7f0bf80039c0 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1415ad80 con 0x7f0bf8004d50 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf800b9d0 con 0x7f0c1406ff20 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c11e00640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf80039c0 0x7f0bf8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c11400640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1406ff20 0x7f0bf8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c12800640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bf8004d50 0x7f0bf80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c11e00640 1 -- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1406ff20 msgr2=0x7f0bf8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c11e00640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c1406ff20 0x7f0bf8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c11e00640 1 -- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bf8004d50 msgr2=0x7f0bf80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c11e00640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bf8004d50 0x7f0bf80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c11e00640 1 -- 172.21.15.113:0/1386025024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf8003530 con 0x7f0bf80039c0 2024-03-04T07:58:35.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.109+0000 7f0c12800640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bf8004d50 0x7f0bf80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:58:35.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c11e00640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf80039c0 0x7f0bf8003de0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c160 tx=0x7f0c0401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906074e797b6a10e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:35.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04007c90 con 0x7f0bf80039c0 2024-03-04T07:58:35.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c040a9020 con 0x7f0bf80039c0 2024-03-04T07:58:35.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c10a00640 1 -- 172.21.15.113:0/1386025024 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04005b00 con 0x7f0bf80039c0 2024-03-04T07:58:35.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c18c00640 1 -- 172.21.15.113:0/1386025024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf80039c0 msgr2=0x7f0bf8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:35.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c18c00640 1 --2- 172.21.15.113:0/1386025024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf80039c0 0x7f0bf8003de0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c160 tx=0x7f0c0401cbc0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c12800640 1 -- 172.21.15.113:0/1386025024 reap_dead start 2024-03-04T07:58:35.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c18c00640 1 -- 172.21.15.113:0/1386025024 shutdown_connections 2024-03-04T07:58:35.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c18c00640 1 -- 172.21.15.113:0/1386025024 >> 172.21.15.113:0/1386025024 conn(0x7f0c140734d0 msgr2=0x7f0c14066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:35.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c18c00640 1 -- 172.21.15.113:0/1386025024 shutdown_connections 2024-03-04T07:58:35.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c18c00640 1 -- 172.21.15.113:0/1386025024 wait complete. 2024-03-04T07:58:35.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c18c00640 1 Processor -- start 2024-03-04T07:58:35.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.113+0000 7f0c18c00640 1 -- start start 2024-03-04T07:58:35.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c18c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf80039c0 0x7f0c1414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:35.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c18c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140c31b0 0x7f0c141646a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:35.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c18c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c140cc470 0x7f0c140cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:35.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c18c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c141583f0 con 0x7f0c140c31b0 2024-03-04T07:58:35.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c18c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1415b920 con 0x7f0c140cc470 2024-03-04T07:58:35.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c18c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1406e160 con 0x7f0bf80039c0 2024-03-04T07:58:35.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140c31b0 0x7f0c141646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:35.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140c31b0 0x7f0c141646a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57894/0 (socket says 172.21.15.113:57894) 2024-03-04T07:58:35.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11400640 1 -- 172.21.15.113:0/111394386 learned_addr learned my addr 172.21.15.113:0/111394386 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:35.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11e00640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf80039c0 0x7f0c1414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c12800640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c140cc470 0x7f0c140cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11400640 1 -- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf80039c0 msgr2=0x7f0c1414a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11400640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf80039c0 0x7f0c1414a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11400640 1 -- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c140cc470 msgr2=0x7f0c140cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11400640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c140cc470 0x7f0c140cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11400640 1 -- 172.21.15.113:0/111394386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14156bf0 con 0x7f0c140c31b0 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11e00640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf80039c0 0x7f0c1414a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c12800640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c140cc470 0x7f0c140cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c11400640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140c31b0 0x7f0c141646a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0c00011990 tx=0x7f0c00011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c0a000640 1 -- 172.21.15.113:0/111394386 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0000fa70 con 0x7f0c140c31b0 2024-03-04T07:58:35.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c0a000640 1 -- 172.21.15.113:0/111394386 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0000fc10 con 0x7f0c140c31b0 2024-03-04T07:58:35.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c0a000640 1 -- 172.21.15.113:0/111394386 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00014440 con 0x7f0c140c31b0 2024-03-04T07:58:35.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c18c00640 1 -- 172.21.15.113:0/111394386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c140cce30 con 0x7f0c140c31b0 2024-03-04T07:58:35.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c18c00640 1 -- 172.21.15.113:0/111394386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c14153f10 con 0x7f0c140c31b0 2024-03-04T07:58:35.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.117+0000 7f0c18c00640 1 -- 172.21.15.113:0/111394386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c14075c00 con 0x7f0c140c31b0 2024-03-04T07:58:35.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.121+0000 7f0c0a000640 1 -- 172.21.15.113:0/111394386 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c00012040 con 0x7f0c140c31b0 2024-03-04T07:58:35.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.121+0000 7f0c0a000640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bf4066f60 0x7f0bf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:35.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.121+0000 7f0c0a000640 1 -- 172.21.15.113:0/111394386 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f0c00085ae0 con 0x7f0c140c31b0 2024-03-04T07:58:35.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.121+0000 7f0c11e00640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bf4066f60 0x7f0bf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:35.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.121+0000 7f0c11e00640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bf4066f60 0x7f0bf4069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0c040aa000 tx=0x7f0c04000ff0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:35.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.129+0000 7f0c0a000640 1 -- 172.21.15.113:0/111394386 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0c140cce30 con 0x7f0c140c31b0 2024-03-04T07:58:35.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.325+0000 7f0c18c00640 1 -- 172.21.15.113:0/111394386 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c1406fd00 con 0x7f0c140c31b0 2024-03-04T07:58:35.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.325+0000 7f0c0a000640 1 -- 172.21.15.113:0/111394386 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f0c00019030 con 0x7f0c140c31b0 2024-03-04T07:58:35.327 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:35.327 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6534,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240,6145]},"id":1}]} 2024-03-04T07:58:35.328 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 88 2024-03-04T07:58:35.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.329+0000 7f0c18c00640 1 -- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bf4066f60 msgr2=0x7f0bf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:35.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.329+0000 7f0c18c00640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bf4066f60 0x7f0bf4069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0c040aa000 tx=0x7f0c04000ff0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.329+0000 7f0c18c00640 1 -- 172.21.15.113:0/111394386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140c31b0 msgr2=0x7f0c141646a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:35.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.329+0000 7f0c18c00640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140c31b0 0x7f0c141646a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0c00011990 tx=0x7f0c00011e50 comp rx=0 tx=0).stop 2024-03-04T07:58:35.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.329+0000 7f0c18c00640 1 -- 172.21.15.113:0/111394386 shutdown_connections 2024-03-04T07:58:35.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.329+0000 7f0c18c00640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c140cc470 0x7f0c140cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.329+0000 7f0c18c00640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bf4066f60 0x7f0bf4069420 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.329+0000 7f0c18c00640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c140c31b0 0x7f0c141646a0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.329+0000 7f0c18c00640 1 --2- 172.21.15.113:0/111394386 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf80039c0 0x7f0c1414a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:35.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.329+0000 7f0c18c00640 1 -- 172.21.15.113:0/111394386 >> 172.21.15.113:0/111394386 conn(0x7f0c140734d0 msgr2=0x7f0c1414d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:35.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.337+0000 7f0c18c00640 1 -- 172.21.15.113:0/111394386 shutdown_connections 2024-03-04T07:58:35.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:35.337+0000 7f0c18c00640 1 -- 172.21.15.113:0/111394386 wait complete. 2024-03-04T07:58:35.373 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:58:37.374 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:37.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.513+0000 7f0d8f200640 1 Processor -- start 2024-03-04T07:58:37.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8f200640 1 -- start start 2024-03-04T07:58:37.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8f200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d90152950 0x7f0d9015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:37.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8f200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0d9006faa0 0x7f0d900c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:37.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8f200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d900c4ce0 con 0x7f0d9006ed80 2024-03-04T07:58:37.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8f200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d900c8370 con 0x7f0d90152950 2024-03-04T07:58:37.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8f200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d900ca0a0 con 0x7f0d9006faa0 2024-03-04T07:58:37.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d90152950 0x7f0d9015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:37.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d90152950 0x7f0d9015bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42484/0 (socket says 172.21.15.113:42484) 2024-03-04T07:58:37.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8de00640 1 -- 172.21.15.113:0/2509501588 learned_addr learned my addr 172.21.15.113:0/2509501588 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:37.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d87e00640 1 --2- 172.21.15.113:0/2509501588 >> v2:172.21.15.136:3300/0 conn(0x7f0d9006faa0 0x7f0d900c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:37.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8de00640 1 -- 172.21.15.113:0/2509501588 >> v2:172.21.15.136:3300/0 conn(0x7f0d9006faa0 msgr2=0x7f0d900c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:37.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8de00640 1 --2- 172.21.15.113:0/2509501588 >> v2:172.21.15.136:3300/0 conn(0x7f0d9006faa0 0x7f0d900c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8de00640 1 -- 172.21.15.113:0/2509501588 >> v1:172.21.15.136:6789/0 conn(0x7f0d9006ed80 legacy=0x7f0d900ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:37.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d8de00640 1 -- 172.21.15.113:0/2509501588 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900c4f80 con 0x7f0d90152950 2024-03-04T07:58:37.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.517+0000 7f0d87e00640 1 --2- 172.21.15.113:0/2509501588 >> v2:172.21.15.136:3300/0 conn(0x7f0d9006faa0 0x7f0d900c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T07:58:37.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8de00640 1 --2- 172.21.15.113:0/2509501588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d90152950 0x7f0d9015bb40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0d9006c110 tx=0x7f0d8001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1b8f65065b81ff server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:37.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8d400640 1 -- 172.21.15.113:0/2509501588 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800b5070 con 0x7f0d90152950 2024-03-04T07:58:37.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8d400640 1 -- 172.21.15.113:0/2509501588 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d800a89d0 con 0x7f0d90152950 2024-03-04T07:58:37.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8d400640 1 -- 172.21.15.113:0/2509501588 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800a8da0 con 0x7f0d90152950 2024-03-04T07:58:37.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8f200640 1 -- 172.21.15.113:0/2509501588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d90152950 msgr2=0x7f0d9015bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:37.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8f200640 1 --2- 172.21.15.113:0/2509501588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d90152950 0x7f0d9015bb40 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0d9006c110 tx=0x7f0d8001ecd0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8f200640 1 -- 172.21.15.113:0/2509501588 shutdown_connections 2024-03-04T07:58:37.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8f200640 1 --2- 172.21.15.113:0/2509501588 >> v2:172.21.15.136:3300/0 conn(0x7f0d9006faa0 0x7f0d900c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8f200640 1 --2- 172.21.15.113:0/2509501588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d90152950 0x7f0d9015bb40 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8f200640 1 -- 172.21.15.113:0/2509501588 >> 172.21.15.113:0/2509501588 conn(0x7f0d90072f40 msgr2=0x7f0d900706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:37.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8f200640 1 -- 172.21.15.113:0/2509501588 shutdown_connections 2024-03-04T07:58:37.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.521+0000 7f0d8f200640 1 -- 172.21.15.113:0/2509501588 wait complete. 2024-03-04T07:58:37.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8f200640 1 Processor -- start 2024-03-04T07:58:37.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8f200640 1 -- start start 2024-03-04T07:58:37.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d9006ed80 0x7f0d90154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:37.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9006faa0 0x7f0d901530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:37.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8f200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d90152950 0x7f0d90153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900c9a50 con 0x7f0d9006faa0 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9006fe80 con 0x7f0d9006ed80 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8f200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900c9d30 con 0x7f0d90152950 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9006faa0 0x7f0d901530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9006faa0 0x7f0d901530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42494/0 (socket says 172.21.15.113:42494) 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d87e00640 1 -- 172.21.15.113:0/1370156818 learned_addr learned my addr 172.21.15.113:0/1370156818 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8de00640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d9006ed80 0x7f0d90154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8e800640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d90152950 0x7f0d90153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d87e00640 1 -- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d90152950 msgr2=0x7f0d90153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d87e00640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d90152950 0x7f0d90153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d87e00640 1 -- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d9006ed80 msgr2=0x7f0d90154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:37.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d87e00640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d9006ed80 0x7f0d90154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d87e00640 1 -- 172.21.15.113:0/1370156818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d901e3840 con 0x7f0d9006faa0 2024-03-04T07:58:37.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d87e00640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9006faa0 0x7f0d901530c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c7f0 tx=0x7f0d7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:37.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d86000640 1 -- 172.21.15.113:0/1370156818 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78007a30 con 0x7f0d9006faa0 2024-03-04T07:58:37.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d86000640 1 -- 172.21.15.113:0/1370156818 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d78007bd0 con 0x7f0d9006faa0 2024-03-04T07:58:37.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d86000640 1 -- 172.21.15.113:0/1370156818 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780113f0 con 0x7f0d9006faa0 2024-03-04T07:58:37.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8f200640 1 -- 172.21.15.113:0/1370156818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d90154c10 con 0x7f0d9006faa0 2024-03-04T07:58:37.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.525+0000 7f0d8f200640 1 -- 172.21.15.113:0/1370156818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d901e35a0 con 0x7f0d9006faa0 2024-03-04T07:58:37.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.529+0000 7f0d8f200640 1 -- 172.21.15.113:0/1370156818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d90075a80 con 0x7f0d9006faa0 2024-03-04T07:58:37.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.529+0000 7f0d86000640 1 -- 172.21.15.113:0/1370156818 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d78011590 con 0x7f0d9006faa0 2024-03-04T07:58:37.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.529+0000 7f0d86000640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d6c066f90 0x7f0d6c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:37.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.529+0000 7f0d86000640 1 -- 172.21.15.113:0/1370156818 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f0d780855d0 con 0x7f0d9006faa0 2024-03-04T07:58:37.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.529+0000 7f0d8de00640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d6c066f90 0x7f0d6c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:37.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.533+0000 7f0d8de00640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d6c066f90 0x7f0d6c069450 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0d800b2000 tx=0x7f0d80009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:37.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.533+0000 7f0d86000640 1 -- 172.21.15.113:0/1370156818 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0d78015020 con 0x7f0d9006faa0 2024-03-04T07:58:37.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 -- 172.21.15.113:0/1370156818 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d9006f880 con 0x7f0d9006faa0 2024-03-04T07:58:37.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d86000640 1 -- 172.21.15.113:0/1370156818 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f0d78055b00 con 0x7f0d9006faa0 2024-03-04T07:58:37.876 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:37.876 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6534,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240,6145]},"id":1}]} 2024-03-04T07:58:37.877 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 88 2024-03-04T07:58:37.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 -- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d6c066f90 msgr2=0x7f0d6c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:37.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d6c066f90 0x7f0d6c069450 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0d800b2000 tx=0x7f0d80009040 comp rx=0 tx=0).stop 2024-03-04T07:58:37.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 -- 172.21.15.113:0/1370156818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9006faa0 msgr2=0x7f0d901530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:37.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9006faa0 0x7f0d901530c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c7f0 tx=0x7f0d7800ccb0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 -- 172.21.15.113:0/1370156818 shutdown_connections 2024-03-04T07:58:37.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d90152950 0x7f0d90153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d6c066f90 0x7f0d6c069450 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d9006faa0 0x7f0d901530c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 --2- 172.21.15.113:0/1370156818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d9006ed80 0x7f0d90154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:37.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 -- 172.21.15.113:0/1370156818 >> 172.21.15.113:0/1370156818 conn(0x7f0d90072f40 msgr2=0x7f0d900705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:37.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 -- 172.21.15.113:0/1370156818 shutdown_connections 2024-03-04T07:58:37.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:37.877+0000 7f0d8f200640 1 -- 172.21.15.113:0/1370156818 wait complete. 2024-03-04T07:58:37.898 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:58:39.899 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:40.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.036+0000 7fab8b400640 1 Processor -- start 2024-03-04T07:58:40.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.036+0000 7fab8b400640 1 -- start start 2024-03-04T07:58:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab8b400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fab841527f0 0x7fab84070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab8b400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab841591f0 con 0x7fab84152bd0 2024-03-04T07:58:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab8b400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab841594d0 con 0x7fab840c8200 2024-03-04T07:58:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab8b400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab84071df0 con 0x7fab841527f0 2024-03-04T07:58:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab89400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fab84152bd0 0x7fab841587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41694/0 (socket says 172.21.15.113:41694) 2024-03-04T07:58:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab89400640 1 -- 172.21.15.113:0/4135029660 learned_addr learned my addr 172.21.15.113:0/4135029660 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:58:40.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab88a00640 1 --2- 172.21.15.113:0/4135029660 >> v2:172.21.15.136:3300/0 conn(0x7fab841527f0 0x7fab84070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:40.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab83400640 1 -- 172.21.15.113:0/4135029660 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883698990 0 0) 0x7fab841591f0 con 0x7fab84152bd0 2024-03-04T07:58:40.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab83400640 1 -- 172.21.15.113:0/4135029660 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab6c003530 con 0x7fab84152bd0 2024-03-04T07:58:40.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab88a00640 1 -- 172.21.15.113:0/4135029660 >> v1:172.21.15.136:6789/0 conn(0x7fab840c8200 legacy=0x7fab84072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:40.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab88a00640 1 -- 172.21.15.113:0/4135029660 >> v1:172.21.15.113:6789/0 conn(0x7fab84152bd0 legacy=0x7fab841587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:40.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.040+0000 7fab88a00640 1 -- 172.21.15.113:0/4135029660 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab840717a0 con 0x7fab841527f0 2024-03-04T07:58:40.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab88a00640 1 --2- 172.21.15.113:0/4135029660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab841527f0 0x7fab84070d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fab780097c0 tx=0x7fab7801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c72986d5d22842c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:40.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab83400640 1 -- 172.21.15.113:0/4135029660 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab780b1070 con 0x7fab841527f0 2024-03-04T07:58:40.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab83400640 1 -- 172.21.15.113:0/4135029660 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab780ac410 con 0x7fab841527f0 2024-03-04T07:58:40.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab83400640 1 -- 172.21.15.113:0/4135029660 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab780ac7c0 con 0x7fab841527f0 2024-03-04T07:58:40.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab8b400640 1 -- 172.21.15.113:0/4135029660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab841527f0 msgr2=0x7fab84070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:40.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab8b400640 1 --2- 172.21.15.113:0/4135029660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab841527f0 0x7fab84070d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fab780097c0 tx=0x7fab7801c4a0 comp rx=0 tx=0).stop 2024-03-04T07:58:40.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab8b400640 1 -- 172.21.15.113:0/4135029660 shutdown_connections 2024-03-04T07:58:40.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab8b400640 1 --2- 172.21.15.113:0/4135029660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab841527f0 0x7fab84070d40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:40.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab8b400640 1 -- 172.21.15.113:0/4135029660 >> 172.21.15.113:0/4135029660 conn(0x7fab8406e820 msgr2=0x7fab8406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:40.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab8b400640 1 -- 172.21.15.113:0/4135029660 shutdown_connections 2024-03-04T07:58:40.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.044+0000 7fab8b400640 1 -- 172.21.15.113:0/4135029660 wait complete. 2024-03-04T07:58:40.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab8b400640 1 Processor -- start 2024-03-04T07:58:40.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab8b400640 1 -- start start 2024-03-04T07:58:40.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab8b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab840c8200 0x7fab840c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:40.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab8b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab841527f0 0x7fab840cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:40.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab8b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab84152bd0 0x7fab841f1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab8b400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab84166380 con 0x7fab84152bd0 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab8b400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab841597b0 con 0x7fab840c8200 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab8b400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab84166500 con 0x7fab841527f0 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab89400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab84152bd0 0x7fab841f1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab89400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab84152bd0 0x7fab841f1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42504/0 (socket says 172.21.15.113:42504) 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab89400640 1 -- 172.21.15.113:0/994402687 learned_addr learned my addr 172.21.15.113:0/994402687 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab88a00640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab840c8200 0x7fab840c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab83e00640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab841527f0 0x7fab840cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab89400640 1 -- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab841527f0 msgr2=0x7fab840cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab89400640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab841527f0 0x7fab840cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab89400640 1 -- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab840c8200 msgr2=0x7fab840c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab89400640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab840c8200 0x7fab840c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab89400640 1 -- 172.21.15.113:0/994402687 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab840c3f70 con 0x7fab84152bd0 2024-03-04T07:58:40.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab89400640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab84152bd0 0x7fab841f1db0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00caa0 tx=0x7fab7c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:40.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab80c00640 1 -- 172.21.15.113:0/994402687 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c017070 con 0x7fab84152bd0 2024-03-04T07:58:40.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab80c00640 1 -- 172.21.15.113:0/994402687 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7c00aba0 con 0x7fab84152bd0 2024-03-04T07:58:40.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab8b400640 1 -- 172.21.15.113:0/994402687 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab840c41e0 con 0x7fab84152bd0 2024-03-04T07:58:40.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.048+0000 7fab80c00640 1 -- 172.21.15.113:0/994402687 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7c012400 con 0x7fab84152bd0 2024-03-04T07:58:40.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.052+0000 7fab8b400640 1 -- 172.21.15.113:0/994402687 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab840c47a0 con 0x7fab84152bd0 2024-03-04T07:58:40.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.052+0000 7fab8b400640 1 -- 172.21.15.113:0/994402687 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab84075c00 con 0x7fab84152bd0 2024-03-04T07:58:40.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.052+0000 7fab80c00640 1 -- 172.21.15.113:0/994402687 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab7c01a070 con 0x7fab84152bd0 2024-03-04T07:58:40.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.052+0000 7fab80c00640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab64066f60 0x7fab64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:40.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.052+0000 7fab80c00640 1 -- 172.21.15.113:0/994402687 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fab7c085100 con 0x7fab84152bd0 2024-03-04T07:58:40.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.052+0000 7fab88a00640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab64066f60 0x7fab64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:40.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.056+0000 7fab88a00640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab64066f60 0x7fab64069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fab780ae000 tx=0x7fab780a6040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:40.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.056+0000 7fab80c00640 1 -- 172.21.15.113:0/994402687 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fab7c0556d0 con 0x7fab84152bd0 2024-03-04T07:58:40.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 -- 172.21.15.113:0/994402687 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab840c9a50 con 0x7fab84152bd0 2024-03-04T07:58:40.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab80c00640 1 -- 172.21.15.113:0/994402687 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6615 (secure 0 0 0) 0x7fab7c059770 con 0x7fab84152bd0 2024-03-04T07:58:40.368 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:40.369 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6534,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240,6145]},"id":1}]} 2024-03-04T07:58:40.369 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 88 2024-03-04T07:58:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 -- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab64066f60 msgr2=0x7fab64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab64066f60 0x7fab64069420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fab780ae000 tx=0x7fab780a6040 comp rx=0 tx=0).stop 2024-03-04T07:58:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 -- 172.21.15.113:0/994402687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab84152bd0 msgr2=0x7fab841f1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab84152bd0 0x7fab841f1db0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00caa0 tx=0x7fab7c00cf60 comp rx=0 tx=0).stop 2024-03-04T07:58:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 -- 172.21.15.113:0/994402687 shutdown_connections 2024-03-04T07:58:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab64066f60 0x7fab64069420 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab84152bd0 0x7fab841f1db0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab841527f0 0x7fab840cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:40.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 --2- 172.21.15.113:0/994402687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab840c8200 0x7fab840c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:40.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.368+0000 7fab8b400640 1 -- 172.21.15.113:0/994402687 >> 172.21.15.113:0/994402687 conn(0x7fab8406e820 msgr2=0x7fab840b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:40.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.372+0000 7fab8b400640 1 -- 172.21.15.113:0/994402687 shutdown_connections 2024-03-04T07:58:40.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:40.372+0000 7fab8b400640 1 -- 172.21.15.113:0/994402687 wait complete. 2024-03-04T07:58:40.397 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:58:41.962 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 12% (22/171) Updating files: 13% (23/171) Updating files: 14% (24/171) Updating files: 15% (26/171) Updating files: 15% (27/171) Updating files: 16% (28/171) Updating files: 17% (30/171) Updating files: 18% (31/171) Updating files: 19% (33/171) Updating files: 20% (35/171) Updating files: 21% (36/171) Updating files: 22% (38/171) Updating files: 23% (40/171) Updating files: 24% (42/171) Updating files: 25% (43/171) Updating files: 26% (45/171) Updating files: 27% (47/171) Updating files: 28% (48/171) Updating files: 29% (50/171) Updating files: 30% (52/171) Updating files: 31% (54/171) Updating files: 32% (55/171) Updating files: 33% (57/171) Updating files: 34% (59/171) Updating files: 35% (60/171) Updating files: 36% (62/171) Updating files: 37% (64/171) Updating files: 38% (65/171) Updating files: 39% (67/171) Updating files: 40% (69/171) Updating files: 41% (71/171) Updating files: 42% (72/171) Updating files: 43% (74/171) Updating files: 44% (76/171) Updating files: 45% (77/171) Updating files: 46% (79/171) Updating files: 47% (81/171) Updating files: 48% (83/171) Updating files: 49% (84/171) Updating files: 50% (86/171) Updating files: 51% (88/171) Updating files: 52% (89/171) Updating files: 53% (91/171) Updating files: 54% (93/171) Updating files: 55% (95/171) Updating files: 56% (96/171) Updating files: 57% (98/171) Updating files: 58% (100/171) Updating files: 59% (101/171) Updating files: 60% (103/171) Updating files: 61% (105/171) Updating files: 62% (107/171) Updating files: 63% (108/171) Updating files: 64% (110/171) Updating files: 65% (112/171) Updating files: 66% (113/171) Updating files: 67% (115/171) Updating files: 68% (117/171) Updating files: 69% (118/171) Updating files: 70% (120/171) Updating files: 71% (122/171) Updating files: 72% (124/171) Updating files: 73% (125/171) Updating files: 74% (127/171) Updating files: 75% (129/171) Updating files: 76% (130/171) Updating files: 77% (132/171) Updating files: 78% (134/171) Updating files: 79% (136/171) Updating files: 80% (137/171) Updating files: 81% (139/171) Updating files: 82% (141/171) Updating files: 83% (142/171) Updating files: 84% (144/171) Updating files: 85% (146/171) Updating files: 86% (148/171) Updating files: 87% (149/171) Updating files: 88% (151/171) Updating files: 89% (153/171) Updating files: 90% (154/171) Updating files: 91% (156/171) Updating files: 92% (158/171) Updating files: 93% (160/171) Updating files: 94% (161/171) Updating files: 95% (163/171) Updating files: 96% (165/171) Updating files: 97% (166/171) Updating files: 98% (168/171) Updating files: 99% (170/171) Updating files: 100% (171/171) Updating files: 100% (171/171), done. 2024-03-04T07:58:42.005 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at b600ec2ac7c v0.38 2024-03-04T07:58:42.009 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.38 2024-03-04T07:58:42.398 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.536+0000 7fa6f6400640 1 Processor -- start 2024-03-04T07:58:42.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.536+0000 7fa6f6400640 1 -- start start 2024-03-04T07:58:42.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6f6400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6f01582e0 0x7fa6f00701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:42.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6f6400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f01586c0 con 0x7fa6f00b0ed0 2024-03-04T07:58:42.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6f6400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f006c210 con 0x7fa6f006ddf0 2024-03-04T07:58:42.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6f6400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f006c4f0 con 0x7fa6f01582e0 2024-03-04T07:58:42.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6e7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa6f00b0ed0 0x7fa6f00ce2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41704/0 (socket says 172.21.15.113:41704) 2024-03-04T07:58:42.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6e7e00640 1 -- 172.21.15.113:0/3314014919 learned_addr learned my addr 172.21.15.113:0/3314014919 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:58:42.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6ef400640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6f01582e0 0x7fa6f00701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007752567 0 0) 0x7fa6f01586c0 con 0x7fa6f00b0ed0 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6d4003530 con 0x7fa6f00b0ed0 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639392708 0 0) 0x7fa6f006c210 con 0x7fa6f006ddf0 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f01586c0 con 0x7fa6f006ddf0 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2301641670 0 0) 0x7fa6d4003530 con 0x7fa6f00b0ed0 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6f01582e0 msgr2=0x7fa6f00701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6f01582e0 0x7fa6f00701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 >> v1:172.21.15.136:6789/0 conn(0x7fa6f006ddf0 legacy=0x7fa6f015af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f006f6a0 con 0x7fa6f00b0ed0 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fa6d8001680 con 0x7fa6f00b0ed0 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 >> v1:172.21.15.113:6789/0 conn(0x7fa6f00b0ed0 legacy=0x7fa6f00ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6f01582e0 0x7fa6d4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:42.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6d40040e0 0x7fa6d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d4004d50 0x7fa6d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f006c4f0 con 0x7fa6d4004d50 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d400bab0 con 0x7fa6d40040e0 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d400bcb0 con 0x7fa6f01582e0 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6efe00640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d4004d50 0x7fa6d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6ef400640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6f01582e0 0x7fa6d4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6e7e00640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6d40040e0 0x7fa6d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6efe00640 1 -- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6f01582e0 msgr2=0x7fa6d4003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6efe00640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6f01582e0 0x7fa6d4003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6efe00640 1 -- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6d40040e0 msgr2=0x7fa6d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6efe00640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6d40040e0 0x7fa6d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:42.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.540+0000 7fa6efe00640 1 -- 172.21.15.113:0/3314014919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d4003530 con 0x7fa6d4004d50 2024-03-04T07:58:42.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.544+0000 7fa6efe00640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d4004d50 0x7fa6d40051d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8001fd0 tx=0x7fa6e801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2fcec2fca2ef4d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:42.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.544+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8001c20 con 0x7fa6d4004d50 2024-03-04T07:58:42.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.544+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8003ba0 con 0x7fa6d4004d50 2024-03-04T07:58:42.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.544+0000 7fa6eea00640 1 -- 172.21.15.113:0/3314014919 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80050d0 con 0x7fa6d4004d50 2024-03-04T07:58:42.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.548+0000 7fa6f6400640 1 -- 172.21.15.113:0/3314014919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d4004d50 msgr2=0x7fa6d40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:42.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.548+0000 7fa6f6400640 1 --2- 172.21.15.113:0/3314014919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6d4004d50 0x7fa6d40051d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8001fd0 tx=0x7fa6e801c900 comp rx=0 tx=0).stop 2024-03-04T07:58:42.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.548+0000 7fa6efe00640 1 -- 172.21.15.113:0/3314014919 reap_dead start 2024-03-04T07:58:42.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.548+0000 7fa6f6400640 1 -- 172.21.15.113:0/3314014919 shutdown_connections 2024-03-04T07:58:42.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.548+0000 7fa6f6400640 1 -- 172.21.15.113:0/3314014919 >> 172.21.15.113:0/3314014919 conn(0x7fa6f0066db0 msgr2=0x7fa6f00671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:42.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.548+0000 7fa6f6400640 1 -- 172.21.15.113:0/3314014919 shutdown_connections 2024-03-04T07:58:42.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.548+0000 7fa6f6400640 1 -- 172.21.15.113:0/3314014919 wait complete. 2024-03-04T07:58:42.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.548+0000 7fa6f6400640 1 Processor -- start 2024-03-04T07:58:42.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.548+0000 7fa6f6400640 1 -- start start 2024-03-04T07:58:42.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6f6400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6d4004d50 0x7fa6f00c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:42.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6f6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6f00c6ec0 0x7fa6f00ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:42.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6f6400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f0154620 0x7fa6f0154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:42.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6f6400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f006c7d0 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6f6400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f006fc60 con 0x7fa6f0154620 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6f6400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f00ce9e0 con 0x7fa6d4004d50 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6f00c6ec0 0x7fa6f00ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6f00c6ec0 0x7fa6f00ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42518/0 (socket says 172.21.15.113:42518) 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7e00640 1 -- 172.21.15.113:0/780236598 learned_addr learned my addr 172.21.15.113:0/780236598 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6ef400640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6d4004d50 0x7fa6f00c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6efe00640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f0154620 0x7fa6f0154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7e00640 1 -- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6d4004d50 msgr2=0x7fa6f00c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7e00640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6d4004d50 0x7fa6f00c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7e00640 1 -- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f0154620 msgr2=0x7fa6f0154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7e00640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f0154620 0x7fa6f0154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:42.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7e00640 1 -- 172.21.15.113:0/780236598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f01f1120 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7e00640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6f00c6ec0 0x7fa6f00ccba0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8012540 tx=0x7fa6d8012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:42.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7400640 1 -- 172.21.15.113:0/780236598 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d801c070 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7400640 1 -- 172.21.15.113:0/780236598 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d8014830 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6f6400640 1 -- 172.21.15.113:0/780236598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f01f05a0 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6e7400640 1 -- 172.21.15.113:0/780236598 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8014c20 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6f6400640 1 -- 172.21.15.113:0/780236598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f01f0b60 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.552+0000 7fa6f6400640 1 -- 172.21.15.113:0/780236598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f0075c00 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.556+0000 7fa6e7400640 1 -- 172.21.15.113:0/780236598 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6d801f070 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.556+0000 7fa6e7400640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6cc066f60 0x7fa6cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:42.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.556+0000 7fa6e7400640 1 -- 172.21.15.113:0/780236598 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fa6d8087ea0 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.556+0000 7fa6ef400640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6cc066f60 0x7fa6cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:42.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.556+0000 7fa6ef400640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6cc066f60 0x7fa6cc069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009b20 tx=0x7fa6e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:42.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.560+0000 7fa6e7400640 1 -- 172.21.15.113:0/780236598 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa6d8058a20 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.752+0000 7fa6f6400640 1 -- 172.21.15.113:0/780236598 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6f00ce470 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.752+0000 7fa6e7400640 1 -- 172.21.15.113:0/780236598 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6615 (secure 0 0 0) 0x7fa6d8058320 con 0x7fa6f00c6ec0 2024-03-04T07:58:42.755 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:42.755 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6534,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240,6145]},"id":1}]} 2024-03-04T07:58:42.755 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 88 2024-03-04T07:58:42.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 -- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6cc066f60 msgr2=0x7fa6cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6cc066f60 0x7fa6cc069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0009b20 tx=0x7fa6e0008040 comp rx=0 tx=0).stop 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 -- 172.21.15.113:0/780236598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6f00c6ec0 msgr2=0x7fa6f00ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6f00c6ec0 0x7fa6f00ccba0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8012540 tx=0x7fa6d8012a00 comp rx=0 tx=0).stop 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 -- 172.21.15.113:0/780236598 shutdown_connections 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6f0154620 0x7fa6f0154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa6cc066f60 0x7fa6cc069420 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6f00c6ec0 0x7fa6f00ccba0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 --2- 172.21.15.113:0/780236598 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa6d4004d50 0x7fa6f00c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 -- 172.21.15.113:0/780236598 >> 172.21.15.113:0/780236598 conn(0x7fa6f0066db0 msgr2=0x7fa6f00cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 -- 172.21.15.113:0/780236598 shutdown_connections 2024-03-04T07:58:42.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:42.756+0000 7fa6f6400640 1 -- 172.21.15.113:0/780236598 wait complete. 2024-03-04T07:58:42.776 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:58:44.777 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:44.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.916+0000 7f0713000640 1 Processor -- start 2024-03-04T07:58:44.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.916+0000 7f0713000640 1 -- start start 2024-03-04T07:58:44.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.916+0000 7f0713000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f070c1527f0 0x7f070c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:44.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.916+0000 7f0713000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f070c152bd0 0x7f070c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:44.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.916+0000 7f0713000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070c1591f0 con 0x7f070c0c8200 2024-03-04T07:58:44.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.916+0000 7f0713000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c071df0 con 0x7f070c1527f0 2024-03-04T07:58:44.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.916+0000 7f0713000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c1597b0 con 0x7f070c152bd0 2024-03-04T07:58:44.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.916+0000 7f070be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f070c1527f0 0x7f070c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:44.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.916+0000 7f070be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f070c1527f0 0x7f070c070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42532/0 (socket says 172.21.15.113:42532) 2024-03-04T07:58:44.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.916+0000 7f070be00640 1 -- 172.21.15.113:0/4130013024 learned_addr learned my addr 172.21.15.113:0/4130013024 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:44.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.920+0000 7f0711000640 1 --2- 172.21.15.113:0/4130013024 >> v2:172.21.15.136:3300/0 conn(0x7f070c152bd0 0x7f070c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:44.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.920+0000 7f070be00640 1 -- 172.21.15.113:0/4130013024 >> v2:172.21.15.136:3300/0 conn(0x7f070c152bd0 msgr2=0x7f070c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:44.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.920+0000 7f070be00640 1 --2- 172.21.15.113:0/4130013024 >> v2:172.21.15.136:3300/0 conn(0x7f070c152bd0 0x7f070c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:44.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.920+0000 7f070be00640 1 -- 172.21.15.113:0/4130013024 >> v1:172.21.15.136:6789/0 conn(0x7f070c0c8200 legacy=0x7f070c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:44.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.920+0000 7f070be00640 1 -- 172.21.15.113:0/4130013024 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c1594d0 con 0x7f070c1527f0 2024-03-04T07:58:44.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.920+0000 7f0711000640 1 --2- 172.21.15.113:0/4130013024 >> v2:172.21.15.136:3300/0 conn(0x7f070c152bd0 0x7f070c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:58:44.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f070be00640 1 --2- 172.21.15.113:0/4130013024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 0x7f070c070d40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f070c0712b0 tx=0x7f06fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a771313d531090 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:44.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f070b400640 1 -- 172.21.15.113:0/4130013024 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0ab430 con 0x7f070c1527f0 2024-03-04T07:58:44.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f070b400640 1 -- 172.21.15.113:0/4130013024 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06fc0b0070 con 0x7f070c1527f0 2024-03-04T07:58:44.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f070b400640 1 -- 172.21.15.113:0/4130013024 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0ab7f0 con 0x7f070c1527f0 2024-03-04T07:58:44.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f0713000640 1 -- 172.21.15.113:0/4130013024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 msgr2=0x7f070c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:44.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f0713000640 1 --2- 172.21.15.113:0/4130013024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 0x7f070c070d40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f070c0712b0 tx=0x7f06fc01c520 comp rx=0 tx=0).stop 2024-03-04T07:58:44.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f0713000640 1 -- 172.21.15.113:0/4130013024 shutdown_connections 2024-03-04T07:58:44.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f0713000640 1 --2- 172.21.15.113:0/4130013024 >> v2:172.21.15.136:3300/0 conn(0x7f070c152bd0 0x7f070c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:44.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f0713000640 1 --2- 172.21.15.113:0/4130013024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 0x7f070c070d40 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:44.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f0713000640 1 -- 172.21.15.113:0/4130013024 >> 172.21.15.113:0/4130013024 conn(0x7f070c06e820 msgr2=0x7f070c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:44.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f0713000640 1 -- 172.21.15.113:0/4130013024 shutdown_connections 2024-03-04T07:58:44.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.924+0000 7f0713000640 1 -- 172.21.15.113:0/4130013024 wait complete. 2024-03-04T07:58:44.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0713000640 1 Processor -- start 2024-03-04T07:58:44.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0713000640 1 -- start start 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0713000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f070c0c8200 0x7f070c154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0713000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 0x7f070c148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0713000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070c152bd0 0x7f070c148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0713000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c1654f0 con 0x7f070c1527f0 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0713000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c165370 con 0x7f070c152bd0 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0713000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c165670 con 0x7f070c0c8200 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0703e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 0x7f070c148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0703e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 0x7f070c148300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42546/0 (socket says 172.21.15.113:42546) 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0703e00640 1 -- 172.21.15.113:0/815971298 learned_addr learned my addr 172.21.15.113:0/815971298 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0711000640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070c152bd0 0x7f070c148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:44.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f070be00640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f070c0c8200 0x7f070c154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0703e00640 1 -- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f070c0c8200 msgr2=0x7f070c154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0703e00640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f070c0c8200 0x7f070c154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0703e00640 1 -- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070c152bd0 msgr2=0x7f070c148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0703e00640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070c152bd0 0x7f070c148840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0703e00640 1 -- 172.21.15.113:0/815971298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c154b40 con 0x7f070c1527f0 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f070be00640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f070c0c8200 0x7f070c154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0703e00640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 0x7f070c148300 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c460 tx=0x7f06f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0708c00640 1 -- 172.21.15.113:0/815971298 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4016020 con 0x7f070c1527f0 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0708c00640 1 -- 172.21.15.113:0/815971298 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f40052d0 con 0x7f070c1527f0 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0713000640 1 -- 172.21.15.113:0/815971298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c0b1a60 con 0x7f070c1527f0 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0708c00640 1 -- 172.21.15.113:0/815971298 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4007260 con 0x7f070c1527f0 2024-03-04T07:58:44.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.928+0000 7f0713000640 1 -- 172.21.15.113:0/815971298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c0b1fe0 con 0x7f070c1527f0 2024-03-04T07:58:44.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.932+0000 7f0713000640 1 -- 172.21.15.113:0/815971298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c075c00 con 0x7f070c1527f0 2024-03-04T07:58:44.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.932+0000 7f0708c00640 1 -- 172.21.15.113:0/815971298 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06f4007400 con 0x7f070c1527f0 2024-03-04T07:58:44.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.932+0000 7f0708c00640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06e8066f60 0x7f06e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:44.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.932+0000 7f0708c00640 1 -- 172.21.15.113:0/815971298 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f06f4084e00 con 0x7f070c1527f0 2024-03-04T07:58:44.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.932+0000 7f070be00640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06e8066f60 0x7f06e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:44.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.936+0000 7f070be00640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06e8066f60 0x7f06e8069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0af000 tx=0x7f06fc0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:44.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:44.936+0000 7f0708c00640 1 -- 172.21.15.113:0/815971298 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f06f4020020 con 0x7f070c1527f0 2024-03-04T07:58:45.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 -- 172.21.15.113:0/815971298 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f070c0c79d0 con 0x7f070c1527f0 2024-03-04T07:58:45.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0708c00640 1 -- 172.21.15.113:0/815971298 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f06f4005480 con 0x7f070c1527f0 2024-03-04T07:58:45.132 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:45.133 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6534,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:57:35.018968+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6145,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6145":{"gid":6145,"name":"b","rank":1,"incarnation":71,"state":"up:active","state_seq":39,"addr":"172.21.15.136:6835/2368734308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2368734308},{"type":"v1","addr":"172.21.15.136:6835","nonce":2368734308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240,6145]},"id":1}]} 2024-03-04T07:58:45.133 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 88 2024-03-04T07:58:45.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 -- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06e8066f60 msgr2=0x7f06e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:45.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06e8066f60 0x7f06e8069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f06fc0af000 tx=0x7f06fc0a7000 comp rx=0 tx=0).stop 2024-03-04T07:58:45.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 -- 172.21.15.113:0/815971298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 msgr2=0x7f070c148300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:45.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 0x7f070c148300 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c460 tx=0x7f06f400c920 comp rx=0 tx=0).stop 2024-03-04T07:58:45.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 -- 172.21.15.113:0/815971298 shutdown_connections 2024-03-04T07:58:45.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f070c152bd0 0x7f070c148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:45.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f06e8066f60 0x7f06e8069420 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:45.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f070c1527f0 0x7f070c148300 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:45.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 --2- 172.21.15.113:0/815971298 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f070c0c8200 0x7f070c154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:45.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 -- 172.21.15.113:0/815971298 >> 172.21.15.113:0/815971298 conn(0x7f070c06e820 msgr2=0x7f070c152020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:45.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 -- 172.21.15.113:0/815971298 shutdown_connections 2024-03-04T07:58:45.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:45.132+0000 7f0713000640 1 -- 172.21.15.113:0/815971298 wait complete. 2024-03-04T07:58:45.153 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T07:58:45.802 INFO:tasks.workunit.client.0.smithi113.stdout:v0.39 2024-03-04T07:58:45.803 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 39 -eq 48 ] 2024-03-04T07:58:45.803 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.39 2024-03-04T07:58:45.803 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.39 2024-03-04T07:58:45.805 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.39 2024-03-04T07:58:47.154 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:47.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.348+0000 7fa240600640 1 Processor -- start 2024-03-04T07:58:47.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.348+0000 7fa240600640 1 -- start start 2024-03-04T07:58:47.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.348+0000 7fa240600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa238152d60 0x7fa2380ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.348+0000 7fa240600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa238070120 0x7fa2380ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.348+0000 7fa240600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2380c5380 con 0x7fa2380c9db0 2024-03-04T07:58:47.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.348+0000 7fa240600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2380c3440 con 0x7fa238152d60 2024-03-04T07:58:47.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.348+0000 7fa240600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23806f620 con 0x7fa238070120 2024-03-04T07:58:47.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.348+0000 7fa23dc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa238152d60 0x7fa2380ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.348+0000 7fa23e600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa2380c9db0 0x7fa238158930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46746/0 (socket says 172.21.15.113:46746) 2024-03-04T07:58:47.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.348+0000 7fa23e600640 1 -- 172.21.15.113:0/2318398101 learned_addr learned my addr 172.21.15.113:0/2318398101 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:58:47.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa23d200640 1 --2- 172.21.15.113:0/2318398101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa238070120 0x7fa2380ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa237e00640 1 -- 172.21.15.113:0/2318398101 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712677788 0 0) 0x7fa2380c5380 con 0x7fa2380c9db0 2024-03-04T07:58:47.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa237e00640 1 -- 172.21.15.113:0/2318398101 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa220003530 con 0x7fa2380c9db0 2024-03-04T07:58:47.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa23dc00640 1 -- 172.21.15.113:0/2318398101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa238070120 msgr2=0x7fa2380ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:47.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa23dc00640 1 --2- 172.21.15.113:0/2318398101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa238070120 0x7fa2380ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa23dc00640 1 -- 172.21.15.113:0/2318398101 >> v1:172.21.15.113:6789/0 conn(0x7fa2380c9db0 legacy=0x7fa238158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa23dc00640 1 -- 172.21.15.113:0/2318398101 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2380c36e0 con 0x7fa238152d60 2024-03-04T07:58:47.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa23dc00640 1 --2- 172.21.15.113:0/2318398101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa238152d60 0x7fa2380ce310 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0097c0 tx=0x7fa22c01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c331b1ab5a4b3523 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:47.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa237e00640 1 -- 172.21.15.113:0/2318398101 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0b5070 con 0x7fa238152d60 2024-03-04T07:58:47.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa237e00640 1 -- 172.21.15.113:0/2318398101 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c0a8990 con 0x7fa238152d60 2024-03-04T07:58:47.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.352+0000 7fa237e00640 1 -- 172.21.15.113:0/2318398101 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0a8d60 con 0x7fa238152d60 2024-03-04T07:58:47.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.356+0000 7fa240600640 1 -- 172.21.15.113:0/2318398101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa238152d60 msgr2=0x7fa2380ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.356+0000 7fa240600640 1 --2- 172.21.15.113:0/2318398101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa238152d60 0x7fa2380ce310 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0097c0 tx=0x7fa22c01ec90 comp rx=0 tx=0).stop 2024-03-04T07:58:47.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.356+0000 7fa240600640 1 -- 172.21.15.113:0/2318398101 shutdown_connections 2024-03-04T07:58:47.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.356+0000 7fa240600640 1 --2- 172.21.15.113:0/2318398101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa238070120 0x7fa2380ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.356+0000 7fa240600640 1 --2- 172.21.15.113:0/2318398101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa238152d60 0x7fa2380ce310 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.356+0000 7fa240600640 1 -- 172.21.15.113:0/2318398101 >> 172.21.15.113:0/2318398101 conn(0x7fa238066db0 msgr2=0x7fa2380671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:47.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.356+0000 7fa240600640 1 -- 172.21.15.113:0/2318398101 shutdown_connections 2024-03-04T07:58:47.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.356+0000 7fa240600640 1 -- 172.21.15.113:0/2318398101 wait complete. 2024-03-04T07:58:47.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.356+0000 7fa240600640 1 Processor -- start 2024-03-04T07:58:47.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.356+0000 7fa240600640 1 -- start start 2024-03-04T07:58:47.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa240600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa238070120 0x7fa2380c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa240600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2380c9db0 0x7fa2380cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa240600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa238152d60 0x7fa2381eea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa240600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380c39c0 con 0x7fa2380c9db0 2024-03-04T07:58:47.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa240600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380c5020 con 0x7fa238070120 2024-03-04T07:58:47.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa240600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380c3b40 con 0x7fa238152d60 2024-03-04T07:58:47.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2380c9db0 0x7fa2380cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2380c9db0 0x7fa2380cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35050/0 (socket says 172.21.15.113:35050) 2024-03-04T07:58:47.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23d200640 1 -- 172.21.15.113:0/1751578333 learned_addr learned my addr 172.21.15.113:0/1751578333 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:47.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23e600640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa238152d60 0x7fa2381eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23dc00640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa238070120 0x7fa2380c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23d200640 1 -- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa238152d60 msgr2=0x7fa2381eea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23d200640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa238152d60 0x7fa2381eea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23d200640 1 -- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa238070120 msgr2=0x7fa2380c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23d200640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa238070120 0x7fa2380c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23d200640 1 -- 172.21.15.113:0/1751578333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2380c81a0 con 0x7fa2380c9db0 2024-03-04T07:58:47.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23dc00640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa238070120 0x7fa2380c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:58:47.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa23d200640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2380c9db0 0x7fa2380cc3e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa22400e8a0 tx=0x7fa22400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:47.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa235600640 1 -- 172.21.15.113:0/1751578333 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224017070 con 0x7fa2380c9db0 2024-03-04T07:58:47.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa235600640 1 -- 172.21.15.113:0/1751578333 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22400c940 con 0x7fa2380c9db0 2024-03-04T07:58:47.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa240600640 1 -- 172.21.15.113:0/1751578333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2380c7cb0 con 0x7fa2380c9db0 2024-03-04T07:58:47.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa235600640 1 -- 172.21.15.113:0/1751578333 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22400ccf0 con 0x7fa2380c9db0 2024-03-04T07:58:47.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa240600640 1 -- 172.21.15.113:0/1751578333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2380c8440 con 0x7fa2380c9db0 2024-03-04T07:58:47.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa240600640 1 -- 172.21.15.113:0/1751578333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2380c8610 con 0x7fa2380c9db0 2024-03-04T07:58:47.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.360+0000 7fa235600640 1 -- 172.21.15.113:0/1751578333 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa22401a070 con 0x7fa2380c9db0 2024-03-04T07:58:47.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.364+0000 7fa235600640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa218066f90 0x7fa218069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.364+0000 7fa235600640 1 -- 172.21.15.113:0/1751578333 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7fa2240853d0 con 0x7fa2380c9db0 2024-03-04T07:58:47.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.364+0000 7fa23dc00640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa218066f90 0x7fa218069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.364+0000 7fa23dc00640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa218066f90 0x7fa218069450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0b2000 tx=0x7fa22c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:47.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.368+0000 7fa235600640 1 -- 172.21.15.113:0/1751578333 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa224055820 con 0x7fa2380c9db0 2024-03-04T07:58:47.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.604+0000 7fa240600640 1 -- 172.21.15.113:0/1751578333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa238075c00 con 0x7fa2380c9db0 2024-03-04T07:58:47.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.604+0000 7fa235600640 1 -- 172.21.15.113:0/1751578333 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5815 (secure 0 0 0) 0x7fa2240598c0 con 0x7fa2380c9db0 2024-03-04T07:58:47.605 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:47.605 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:58:45.780625+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6240,6372]},"id":1}]} 2024-03-04T07:58:47.605 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 89 2024-03-04T07:58:47.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 -- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa218066f90 msgr2=0x7fa218069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa218066f90 0x7fa218069450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0b2000 tx=0x7fa22c008040 comp rx=0 tx=0).stop 2024-03-04T07:58:47.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 -- 172.21.15.113:0/1751578333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2380c9db0 msgr2=0x7fa2380cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2380c9db0 0x7fa2380cc3e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa22400e8a0 tx=0x7fa22400eca0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 -- 172.21.15.113:0/1751578333 shutdown_connections 2024-03-04T07:58:47.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa238152d60 0x7fa2381eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa218066f90 0x7fa218069450 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa2380c9db0 0x7fa2380cc3e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 --2- 172.21.15.113:0/1751578333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa238070120 0x7fa2380c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 -- 172.21.15.113:0/1751578333 >> 172.21.15.113:0/1751578333 conn(0x7fa238066db0 msgr2=0x7fa238067190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:47.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 -- 172.21.15.113:0/1751578333 shutdown_connections 2024-03-04T07:58:47.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.608+0000 7fa240600640 1 -- 172.21.15.113:0/1751578333 wait complete. 2024-03-04T07:58:47.636 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2024-03-04T07:58:47.636 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T07:58:47.636 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:47.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.828+0000 7f1fc7e00640 1 Processor -- start 2024-03-04T07:58:47.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.828+0000 7f1fc7e00640 1 -- start start 2024-03-04T07:58:47.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1fc815bb40 0x7f1fc806faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc80c4ce0 con 0x7f1fc8152950 2024-03-04T07:58:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc80c4f80 con 0x7f1fc80cb540 2024-03-04T07:58:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc80c8370 con 0x7f1fc815bb40 2024-03-04T07:58:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc6a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1fc8152950 0x7f1fc806ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46752/0 (socket says 172.21.15.113:46752) 2024-03-04T07:58:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc6a00640 1 -- 172.21.15.113:0/2158366829 learned_addr learned my addr 172.21.15.113:0/2158366829 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:58:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc7400640 1 --2- 172.21.15.113:0/2158366829 >> v2:172.21.15.113:3300/0 conn(0x7f1fc815bb40 0x7f1fc806faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc6000640 1 -- 172.21.15.113:0/2158366829 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181743772 0 0) 0x7f1fc80c4ce0 con 0x7f1fc8152950 2024-03-04T07:58:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc6000640 1 -- 172.21.15.113:0/2158366829 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fac003530 con 0x7f1fc8152950 2024-03-04T07:58:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc7400640 1 -- 172.21.15.113:0/2158366829 >> v1:172.21.15.136:6789/0 conn(0x7f1fc80cb540 legacy=0x7f1fc8160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc7400640 1 -- 172.21.15.113:0/2158366829 >> v1:172.21.15.113:6789/0 conn(0x7f1fc8152950 legacy=0x7f1fc806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.832+0000 7f1fc7400640 1 -- 172.21.15.113:0/2158366829 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc80cb920 con 0x7f1fc815bb40 2024-03-04T07:58:47.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc7400640 1 --2- 172.21.15.113:0/2158366829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc815bb40 0x7f1fc806faa0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00097c0 tx=0x7f1fc001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148823399960f1d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:47.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc6000640 1 -- 172.21.15.113:0/2158366829 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00b5070 con 0x7f1fc815bb40 2024-03-04T07:58:47.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc6000640 1 -- 172.21.15.113:0/2158366829 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc00a8990 con 0x7f1fc815bb40 2024-03-04T07:58:47.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc6000640 1 -- 172.21.15.113:0/2158366829 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00a8d60 con 0x7f1fc815bb40 2024-03-04T07:58:47.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc7e00640 1 -- 172.21.15.113:0/2158366829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc815bb40 msgr2=0x7f1fc806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/2158366829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc815bb40 0x7f1fc806faa0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00097c0 tx=0x7f1fc001ec90 comp rx=0 tx=0).stop 2024-03-04T07:58:47.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc7e00640 1 -- 172.21.15.113:0/2158366829 shutdown_connections 2024-03-04T07:58:47.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/2158366829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc815bb40 0x7f1fc806faa0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc7e00640 1 -- 172.21.15.113:0/2158366829 >> 172.21.15.113:0/2158366829 conn(0x7f1fc8072f40 msgr2=0x7f1fc80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:47.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc7e00640 1 -- 172.21.15.113:0/2158366829 shutdown_connections 2024-03-04T07:58:47.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.836+0000 7f1fc7e00640 1 -- 172.21.15.113:0/2158366829 wait complete. 2024-03-04T07:58:47.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7e00640 1 Processor -- start 2024-03-04T07:58:47.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7e00640 1 -- start start 2024-03-04T07:58:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc80cb540 0x7f1fc81f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc8152950 0x7f1fc81f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fc815bb40 0x7f1fc806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80c9a50 con 0x7f1fc80cb540 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80ca0a0 con 0x7f1fc815bb40 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80c9d30 con 0x7f1fc8152950 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc80cb540 0x7f1fc81f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc80cb540 0x7f1fc81f3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35064/0 (socket says 172.21.15.113:35064) 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc6a00640 1 -- 172.21.15.113:0/3742762002 learned_addr learned my addr 172.21.15.113:0/3742762002 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fbe000640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc8152950 0x7f1fc81f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7400640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fc815bb40 0x7f1fc806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc6a00640 1 -- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc8152950 msgr2=0x7f1fc81f0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc6a00640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc8152950 0x7f1fc81f0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc6a00640 1 -- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fc815bb40 msgr2=0x7f1fc806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc6a00640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fc815bb40 0x7f1fc806d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc6a00640 1 -- 172.21.15.113:0/3742762002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc80ce870 con 0x7f1fc80cb540 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc6a00640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc80cb540 0x7f1fc81f3680 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800c500 tx=0x7f1fb800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fbf400640 1 -- 172.21.15.113:0/3742762002 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8017070 con 0x7f1fc80cb540 2024-03-04T07:58:47.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fbf400640 1 -- 172.21.15.113:0/3742762002 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb8005cf0 con 0x7f1fc80cb540 2024-03-04T07:58:47.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fbf400640 1 -- 172.21.15.113:0/3742762002 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80072f0 con 0x7f1fc80cb540 2024-03-04T07:58:47.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7e00640 1 -- 172.21.15.113:0/3742762002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc81e1bb0 con 0x7f1fc80cb540 2024-03-04T07:58:47.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.840+0000 7f1fc7e00640 1 -- 172.21.15.113:0/3742762002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc81f0420 con 0x7f1fc80cb540 2024-03-04T07:58:47.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.844+0000 7f1fc7e00640 1 -- 172.21.15.113:0/3742762002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc8075a80 con 0x7f1fc80cb540 2024-03-04T07:58:47.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.844+0000 7f1fbf400640 1 -- 172.21.15.113:0/3742762002 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fb801a070 con 0x7f1fc80cb540 2024-03-04T07:58:47.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.844+0000 7f1fbf400640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 0x7f1fa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:47.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.844+0000 7f1fbf400640 1 -- 172.21.15.113:0/3742762002 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f1fb8084840 con 0x7f1fc80cb540 2024-03-04T07:58:47.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.844+0000 7f1fbe000640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 0x7f1fa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:47.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.844+0000 7f1fbe000640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 0x7f1fa4069420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1fc81f47a0 tx=0x7f1fb0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:47.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:47.848+0000 7f1fbf400640 1 -- 172.21.15.113:0/3742762002 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1fb8054d30 con 0x7f1fc80cb540 2024-03-04T07:58:48.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 -- 172.21.15.113:0/3742762002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fc80c9580 con 0x7f1fc80cb540 2024-03-04T07:58:48.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fbf400640 1 -- 172.21.15.113:0/3742762002 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5815 (secure 0 0 0) 0x7f1fb8058dd0 con 0x7f1fc80cb540 2024-03-04T07:58:48.093 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:48.094 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:58:45.780625+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6240,6372]},"id":1}]} 2024-03-04T07:58:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 89 2024-03-04T07:58:48.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 -- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 msgr2=0x7f1fa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:48.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 0x7f1fa4069420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1fc81f47a0 tx=0x7f1fb0009040 comp rx=0 tx=0).stop 2024-03-04T07:58:48.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 -- 172.21.15.113:0/3742762002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc80cb540 msgr2=0x7f1fc81f3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:48.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc80cb540 0x7f1fc81f3680 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800c500 tx=0x7f1fb800c9c0 comp rx=0 tx=0).stop 2024-03-04T07:58:48.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 -- 172.21.15.113:0/3742762002 shutdown_connections 2024-03-04T07:58:48.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fc815bb40 0x7f1fc806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:48.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc8152950 0x7f1fc81f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:48.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 0x7f1fa4069420 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:48.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/3742762002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc80cb540 0x7f1fc81f3680 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:48.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.092+0000 7f1fc7e00640 1 -- 172.21.15.113:0/3742762002 >> 172.21.15.113:0/3742762002 conn(0x7f1fc8072f40 msgr2=0x7f1fc80c5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:48.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.096+0000 7f1fc7e00640 1 -- 172.21.15.113:0/3742762002 shutdown_connections 2024-03-04T07:58:48.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:48.096+0000 7f1fc7e00640 1 -- 172.21.15.113:0/3742762002 wait complete. 2024-03-04T07:58:48.121 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 89, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 89, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6240": { "addr": "172.21.15.113:6836/986515123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 986515123, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 986515123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6240, "incarnation": 77, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6372": { "addr": "172.21.15.113:6833/2732565639", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2732565639, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2732565639, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6372, "incarnation": 83, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 40 }, "gid_6534": { "addr": "172.21.15.113:6837/4197507712", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 4197507712, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 4197507712, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6534, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 61, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:58:45.780625+0000", "qdb_cluster": [ 6240, 6372 ], "qdb_leader": 6372, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6372, "mds_1": 6534, "mds_2": 6240 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:58:50.122 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:50.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10f200640 1 Processor -- start 2024-03-04T07:58:50.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10f200640 1 -- start start 2024-03-04T07:58:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10f200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe1101516d0 0x7fe11006e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:50.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10f200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe1100c34c0 0x7fe1100c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10f200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1100c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10f200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1100736a0 con 0x7fe1101516d0 2024-03-04T07:58:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10f200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1100c3be0 con 0x7fe1100c34c0 2024-03-04T07:58:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10f200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100c3ec0 con 0x7fe1100c2a90 2024-03-04T07:58:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10d400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe1100c34c0 0x7fe1100c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10d400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe1100c34c0 0x7fe1100c2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:37036/0 (socket says 172.21.15.113:37036) 2024-03-04T07:58:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10d400640 1 -- 172.21.15.113:0/2299200516 learned_addr learned my addr 172.21.15.113:0/2299200516 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:50.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10e800640 1 --2- 172.21.15.113:0/2299200516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1100c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10de00640 1 --2- 172.21.15.113:0/2299200516 >> v2:172.21.15.113:3300/0 conn(0x7fe1101516d0 0x7fe11006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.296+0000 7fe10de00640 1 -- 172.21.15.113:0/2299200516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 msgr2=0x7fe1100c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.300+0000 7fe10de00640 1 --2- 172.21.15.113:0/2299200516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1100c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.300+0000 7fe10de00640 1 -- 172.21.15.113:0/2299200516 >> v2:172.21.15.136:3300/0 conn(0x7fe1100c34c0 msgr2=0x7fe1100c2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:50.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.300+0000 7fe10de00640 1 --2- 172.21.15.113:0/2299200516 >> v2:172.21.15.136:3300/0 conn(0x7fe1100c34c0 0x7fe1100c2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.300+0000 7fe10de00640 1 -- 172.21.15.113:0/2299200516 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe110072bc0 con 0x7fe1101516d0 2024-03-04T07:58:50.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.300+0000 7fe10d400640 1 --2- 172.21.15.113:0/2299200516 >> v2:172.21.15.136:3300/0 conn(0x7fe1100c34c0 0x7fe1100c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:58:50.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.300+0000 7fe10e800640 1 --2- 172.21.15.113:0/2299200516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1100c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:58:50.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10de00640 1 --2- 172.21.15.113:0/2299200516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101516d0 0x7fe11006e950 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe110067b60 tx=0x7fe10001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716852d7aa8df55d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:50.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10ca00640 1 -- 172.21.15.113:0/2299200516 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000b0070 con 0x7fe1101516d0 2024-03-04T07:58:50.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10ca00640 1 -- 172.21.15.113:0/2299200516 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1000ab600 con 0x7fe1101516d0 2024-03-04T07:58:50.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10ca00640 1 -- 172.21.15.113:0/2299200516 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000ab9d0 con 0x7fe1101516d0 2024-03-04T07:58:50.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10f200640 1 -- 172.21.15.113:0/2299200516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101516d0 msgr2=0x7fe11006e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:50.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10f200640 1 --2- 172.21.15.113:0/2299200516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101516d0 0x7fe11006e950 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe110067b60 tx=0x7fe10001c480 comp rx=0 tx=0).stop 2024-03-04T07:58:50.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10f200640 1 -- 172.21.15.113:0/2299200516 shutdown_connections 2024-03-04T07:58:50.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10f200640 1 --2- 172.21.15.113:0/2299200516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1100c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10f200640 1 --2- 172.21.15.113:0/2299200516 >> v2:172.21.15.136:3300/0 conn(0x7fe1100c34c0 0x7fe1100c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10f200640 1 --2- 172.21.15.113:0/2299200516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101516d0 0x7fe11006e950 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10f200640 1 -- 172.21.15.113:0/2299200516 >> 172.21.15.113:0/2299200516 conn(0x7fe11006f420 msgr2=0x7fe11006f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:50.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10f200640 1 -- 172.21.15.113:0/2299200516 shutdown_connections 2024-03-04T07:58:50.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.304+0000 7fe10f200640 1 -- 172.21.15.113:0/2299200516 wait complete. 2024-03-04T07:58:50.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.308+0000 7fe10f200640 1 Processor -- start 2024-03-04T07:58:50.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.308+0000 7fe10f200640 1 -- start start 2024-03-04T07:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.308+0000 7fe10f200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1101fb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:50.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.308+0000 7fe10f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1100c34c0 0x7fe1101568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.308+0000 7fe10f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101516d0 0x7fe11014ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110151ab0 con 0x7fe1101516d0 2024-03-04T07:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11006c2c0 con 0x7fe1100c34c0 2024-03-04T07:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10f200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11014c1b0 con 0x7fe1100c2a90 2024-03-04T07:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101516d0 0x7fe11014ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1101fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1101fb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:57780/0 (socket says 172.21.15.113:57780) 2024-03-04T07:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10de00640 1 -- 172.21.15.113:0/2468607006 learned_addr learned my addr 172.21.15.113:0/2468607006 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:50.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10d400640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1100c34c0 0x7fe1101568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:50.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10de00640 1 -- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1100c34c0 msgr2=0x7fe1101568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:50.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10de00640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1100c34c0 0x7fe1101568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10de00640 1 -- 172.21.15.113:0/2468607006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101516d0 msgr2=0x7fe11014ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:50.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10de00640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101516d0 0x7fe11014ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10de00640 1 -- 172.21.15.113:0/2468607006 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe110149fa0 con 0x7fe1100c2a90 2024-03-04T07:58:50.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10e800640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101516d0 0x7fe11014ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:58:50.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10de00640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1101fb510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1000ae040 tx=0x7fe100019220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:50.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe106000640 1 -- 172.21.15.113:0/2468607006 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000b0040 con 0x7fe1100c2a90 2024-03-04T07:58:50.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe106000640 1 -- 172.21.15.113:0/2468607006 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1000ba070 con 0x7fe1100c2a90 2024-03-04T07:58:50.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe106000640 1 -- 172.21.15.113:0/2468607006 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100006e20 con 0x7fe1100c2a90 2024-03-04T07:58:50.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10f200640 1 -- 172.21.15.113:0/2468607006 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11014a280 con 0x7fe1100c2a90 2024-03-04T07:58:50.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10f200640 1 -- 172.21.15.113:0/2468607006 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11014a840 con 0x7fe1100c2a90 2024-03-04T07:58:50.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.312+0000 7fe10f200640 1 -- 172.21.15.113:0/2468607006 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe110075a80 con 0x7fe1100c2a90 2024-03-04T07:58:50.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.316+0000 7fe106000640 1 -- 172.21.15.113:0/2468607006 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1000b4650 con 0x7fe1100c2a90 2024-03-04T07:58:50.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.316+0000 7fe106000640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe0f0066f60 0x7fe0f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:50.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.316+0000 7fe106000640 1 -- 172.21.15.113:0/2468607006 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7fe10011fc10 con 0x7fe1100c2a90 2024-03-04T07:58:50.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.320+0000 7fe10d400640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe0f0066f60 0x7fe0f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:50.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.320+0000 7fe10d400640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe0f0066f60 0x7fe0f0069420 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe110150350 tx=0x7fe0fc009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:50.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.328+0000 7fe106000640 1 -- 172.21.15.113:0/2468607006 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe100016070 con 0x7fe1100c2a90 2024-03-04T07:58:50.535 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 19% (51/258) Updating files: 20% (52/258) Updating files: 21% (55/258) Updating files: 22% (57/258) Updating files: 23% (60/258) Updating files: 24% (62/258) Updating files: 25% (65/258) Updating files: 26% (68/258) Updating files: 27% (70/258) Updating files: 28% (73/258) Updating files: 29% (75/258) Updating files: 30% (78/258) Updating files: 31% (80/258) Updating files: 32% (83/258) Updating files: 33% (86/258) Updating files: 34% (88/258) Updating files: 35% (91/258) Updating files: 36% (93/258) Updating files: 37% (96/258) Updating files: 38% (99/258) Updating files: 39% (101/258) Updating files: 39% (103/258) Updating files: 40% (104/258) Updating files: 41% (106/258) Updating files: 42% (109/258) Updating files: 43% (111/258) Updating files: 44% (114/258) Updating files: 45% (117/258) Updating files: 46% (119/258) Updating files: 47% (122/258) Updating files: 48% (124/258) Updating files: 49% (127/258) Updating files: 50% (129/258) Updating files: 51% (132/258) Updating files: 52% (135/258) Updating files: 53% (137/258) Updating files: 54% (140/258) Updating files: 55% (142/258) Updating files: 56% (145/258) Updating files: 57% (148/258) Updating files: 58% (150/258) Updating files: 59% (153/258) Updating files: 60% (155/258) Updating files: 61% (158/258) Updating files: 62% (160/258) Updating files: 63% (163/258) Updating files: 64% (166/258) Updating files: 65% (168/258) Updating files: 66% (171/258) Updating files: 67% (173/258) Updating files: 68% (176/258) Updating files: 69% (179/258) Updating files: 70% (181/258) Updating files: 71% (184/258) Updating files: 72% (186/258) Updating files: 73% (189/258) Updating files: 74% (191/258) Updating files: 75% (194/258) Updating files: 76% (197/258) Updating files: 77% (199/258) Updating files: 78% (202/258) Updating files: 79% (204/258) Updating files: 80% (207/258) Updating files: 81% (209/258) Updating files: 82% (212/258) Updating files: 83% (215/258) Updating files: 84% (217/258) Updating files: 85% (220/258) Updating files: 86% (222/258) Updating files: 87% (225/258) Updating files: 88% (228/258) Updating files: 89% (230/258) Updating files: 90% (233/258) Updating files: 91% (235/258) Updating files: 92% (238/258) Updating files: 93% (240/258) Updating files: 94% (243/258) Updating files: 95% (246/258) Updating files: 96% (248/258) Updating files: 97% (251/258) Updating files: 98% (253/258) Updating files: 99% (256/258) Updating files: 100% (258/258) Updating files: 100% (258/258), done. 2024-03-04T07:58:50.580 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 321ecdaba2c v0.39 2024-03-04T07:58:50.586 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.39 2024-03-04T07:58:50.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.660+0000 7fe10f200640 1 -- 172.21.15.113:0/2468607006 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1101fba50 con 0x7fe1100c2a90 2024-03-04T07:58:50.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.660+0000 7fe106000640 1 -- 172.21.15.113:0/2468607006 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5815 (secure 0 0 0) 0x7fe1000f07c0 con 0x7fe1100c2a90 2024-03-04T07:58:50.661 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:50.661 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:58:45.780625+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6240,6372]},"id":1}]} 2024-03-04T07:58:50.661 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 89 2024-03-04T07:58:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 -- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe0f0066f60 msgr2=0x7fe0f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe0f0066f60 0x7fe0f0069420 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe110150350 tx=0x7fe0fc009240 comp rx=0 tx=0).stop 2024-03-04T07:58:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 -- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 msgr2=0x7fe1101fb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1101fb510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe1000ae040 tx=0x7fe100019220 comp rx=0 tx=0).stop 2024-03-04T07:58:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 -- 172.21.15.113:0/2468607006 shutdown_connections 2024-03-04T07:58:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe0f0066f60 0x7fe0f0069420 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe1101516d0 0x7fe11014ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1100c34c0 0x7fe1101568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 --2- 172.21.15.113:0/2468607006 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe1100c2a90 0x7fe1101fb510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:50.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 -- 172.21.15.113:0/2468607006 >> 172.21.15.113:0/2468607006 conn(0x7fe11006f420 msgr2=0x7fe11006f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:50.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 -- 172.21.15.113:0/2468607006 shutdown_connections 2024-03-04T07:58:50.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:50.664+0000 7fe10f200640 1 -- 172.21.15.113:0/2468607006 wait complete. 2024-03-04T07:58:50.671 INFO:tasks.workunit.client.0.smithi113.stdout:v0.40 2024-03-04T07:58:50.672 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 40 -eq 48 ] 2024-03-04T07:58:50.672 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.40 2024-03-04T07:58:50.672 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.40 2024-03-04T07:58:50.672 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.40 2024-03-04T07:58:50.712 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:58:52.713 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:52.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.868+0000 7ff003200640 1 Processor -- start 2024-03-04T07:58:52.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.868+0000 7ff003200640 1 -- start start 2024-03-04T07:58:52.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7ff003200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feffc15e900 0x7feffc06f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:52.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7ff003200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feffc0c92f0 0x7feffc06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:52.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7ff003200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feffc15f8b0 0x7feffc0ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:52.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7ff003200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feffc15fe60 con 0x7feffc0c92f0 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7ff003200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feffc06df10 con 0x7feffc15e900 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7ff003200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc15ece0 con 0x7feffc15f8b0 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7feff3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feffc0c92f0 0x7feffc06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7feff3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feffc0c92f0 0x7feffc06ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35088/0 (socket says 172.21.15.113:35088) 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7feff3e00640 1 -- 172.21.15.113:0/2001413324 learned_addr learned my addr 172.21.15.113:0/2001413324 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7feffbe00640 1 --2- 172.21.15.113:0/2001413324 >> v2:172.21.15.136:3300/0 conn(0x7feffc15e900 0x7feffc06f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7ff001200640 1 --2- 172.21.15.113:0/2001413324 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feffc15f8b0 0x7feffc0ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7feff3e00640 1 -- 172.21.15.113:0/2001413324 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feffc15f8b0 msgr2=0x7feffc0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7feff3e00640 1 --2- 172.21.15.113:0/2001413324 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feffc15f8b0 0x7feffc0ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7feff3e00640 1 -- 172.21.15.113:0/2001413324 >> v2:172.21.15.136:3300/0 conn(0x7feffc15e900 msgr2=0x7feffc06f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7feff3e00640 1 --2- 172.21.15.113:0/2001413324 >> v2:172.21.15.136:3300/0 conn(0x7feffc15e900 0x7feffc06f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:52.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.872+0000 7feff3e00640 1 -- 172.21.15.113:0/2001413324 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc06c210 con 0x7feffc0c92f0 2024-03-04T07:58:52.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7feff3e00640 1 --2- 172.21.15.113:0/2001413324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 0x7feffc06ff20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7feffc067cd0 tx=0x7fefe401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52555d517b71580 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:52.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7feffb400640 1 -- 172.21.15.113:0/2001413324 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe40b5070 con 0x7feffc0c92f0 2024-03-04T07:58:52.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7feffb400640 1 -- 172.21.15.113:0/2001413324 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe40a89d0 con 0x7feffc0c92f0 2024-03-04T07:58:52.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7feffb400640 1 -- 172.21.15.113:0/2001413324 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe40a8da0 con 0x7feffc0c92f0 2024-03-04T07:58:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7ff003200640 1 -- 172.21.15.113:0/2001413324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 msgr2=0x7feffc06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7ff003200640 1 --2- 172.21.15.113:0/2001413324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 0x7feffc06ff20 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7feffc067cd0 tx=0x7fefe401ed10 comp rx=0 tx=0).stop 2024-03-04T07:58:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7ff003200640 1 -- 172.21.15.113:0/2001413324 shutdown_connections 2024-03-04T07:58:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7ff003200640 1 --2- 172.21.15.113:0/2001413324 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feffc15f8b0 0x7feffc0ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7ff003200640 1 --2- 172.21.15.113:0/2001413324 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 0x7feffc06ff20 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7ff003200640 1 --2- 172.21.15.113:0/2001413324 >> v2:172.21.15.136:3300/0 conn(0x7feffc15e900 0x7feffc06f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7ff003200640 1 -- 172.21.15.113:0/2001413324 >> 172.21.15.113:0/2001413324 conn(0x7feffc0734d0 msgr2=0x7feffc066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7ff003200640 1 -- 172.21.15.113:0/2001413324 shutdown_connections 2024-03-04T07:58:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.876+0000 7ff003200640 1 -- 172.21.15.113:0/2001413324 wait complete. 2024-03-04T07:58:52.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff003200640 1 Processor -- start 2024-03-04T07:58:52.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff003200640 1 -- start start 2024-03-04T07:58:52.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff003200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 0x7feffc1f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:52.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff003200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feffc15e900 0x7feffc0c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:52.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff003200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feffc15f8b0 0x7feffc0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:52.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff003200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc0ce120 con 0x7feffc0c92f0 2024-03-04T07:58:52.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff003200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc06e1b0 con 0x7feffc15f8b0 2024-03-04T07:58:52.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff003200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc0ce2a0 con 0x7feffc15e900 2024-03-04T07:58:52.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feffbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 0x7feffc1f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:52.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feffbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 0x7feffc1f4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35096/0 (socket says 172.21.15.113:35096) 2024-03-04T07:58:52.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feffbe00640 1 -- 172.21.15.113:0/2180215130 learned_addr learned my addr 172.21.15.113:0/2180215130 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:52.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feff3e00640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feffc15e900 0x7feffc0c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:52.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff001200640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feffc15f8b0 0x7feffc0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:52.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feffbe00640 1 -- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feffc15e900 msgr2=0x7feffc0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:52.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feffbe00640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feffc15e900 0x7feffc0c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:52.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feffbe00640 1 -- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feffc15f8b0 msgr2=0x7feffc0cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:52.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feffbe00640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feffc15f8b0 0x7feffc0cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:52.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feffbe00640 1 -- 172.21.15.113:0/2180215130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc1f83b0 con 0x7feffc0c92f0 2024-03-04T07:58:52.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feffbe00640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 0x7feffc1f4ca0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fefec00e8b0 tx=0x7fefec00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:52.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feff8c00640 1 -- 172.21.15.113:0/2180215130 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec017070 con 0x7feffc0c92f0 2024-03-04T07:58:52.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feff8c00640 1 -- 172.21.15.113:0/2180215130 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec00c940 con 0x7feffc0c92f0 2024-03-04T07:58:52.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7feff8c00640 1 -- 172.21.15.113:0/2180215130 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec00cd10 con 0x7feffc0c92f0 2024-03-04T07:58:52.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff003200640 1 -- 172.21.15.113:0/2180215130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feffc1f51e0 con 0x7feffc0c92f0 2024-03-04T07:58:52.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.880+0000 7ff003200640 1 -- 172.21.15.113:0/2180215130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc0b0b70 con 0x7feffc0c92f0 2024-03-04T07:58:52.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.884+0000 7ff003200640 1 -- 172.21.15.113:0/2180215130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feffc075c00 con 0x7feffc0c92f0 2024-03-04T07:58:52.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.884+0000 7feff8c00640 1 -- 172.21.15.113:0/2180215130 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fefec01a070 con 0x7feffc0c92f0 2024-03-04T07:58:52.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.884+0000 7feff8c00640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd8066f90 0x7fefd8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:52.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.884+0000 7feff8c00640 1 -- 172.21.15.113:0/2180215130 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7fefec0853a0 con 0x7feffc0c92f0 2024-03-04T07:58:52.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.888+0000 7feff3e00640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd8066f90 0x7fefd8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:52.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.888+0000 7feff3e00640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd8066f90 0x7fefd8069450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fefe40b2000 tx=0x7fefe4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:52.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:52.892+0000 7feff8c00640 1 -- 172.21.15.113:0/2180215130 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fefec055920 con 0x7feffc0c92f0 2024-03-04T07:58:53.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.088+0000 7ff003200640 1 -- 172.21.15.113:0/2180215130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feffc0c9d40 con 0x7feffc0c92f0 2024-03-04T07:58:53.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7feff8c00640 1 -- 172.21.15.113:0/2180215130 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5817 (secure 0 0 0) 0x7fefec0599c0 con 0x7feffc0c92f0 2024-03-04T07:58:53.092 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:53.092 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:58:50.802501+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:resolve","state_seq":18,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240]},"id":1}]} 2024-03-04T07:58:53.092 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 90 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 -- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd8066f90 msgr2=0x7fefd8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd8066f90 0x7fefd8069450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fefe40b2000 tx=0x7fefe4009040 comp rx=0 tx=0).stop 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 -- 172.21.15.113:0/2180215130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 msgr2=0x7feffc1f4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 0x7feffc1f4ca0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fefec00e8b0 tx=0x7fefec00ed70 comp rx=0 tx=0).stop 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 -- 172.21.15.113:0/2180215130 shutdown_connections 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feffc15f8b0 0x7feffc0cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feffc15e900 0x7feffc0c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd8066f90 0x7fefd8069450 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 --2- 172.21.15.113:0/2180215130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feffc0c92f0 0x7feffc1f4ca0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 -- 172.21.15.113:0/2180215130 >> 172.21.15.113:0/2180215130 conn(0x7feffc0734d0 msgr2=0x7feffc15d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 -- 172.21.15.113:0/2180215130 shutdown_connections 2024-03-04T07:58:53.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:53.092+0000 7ff003200640 1 -- 172.21.15.113:0/2180215130 wait complete. 2024-03-04T07:58:53.113 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:58:55.114 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:55.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e99c00640 1 Processor -- start 2024-03-04T07:58:55.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e99c00640 1 -- start start 2024-03-04T07:58:55.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e99c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 0x7f9e940c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:55.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e99c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e940c5630 con 0x7f9e940c9ab0 2024-03-04T07:58:55.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e99c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9406fc70 con 0x7f9e94152ac0 2024-03-04T07:58:55.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e99c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940c5bb0 con 0x7f9e940c8fe0 2024-03-04T07:58:55.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e8be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9e940c9ab0 0x7f9e9415c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46758/0 (socket says 172.21.15.113:46758) 2024-03-04T07:58:55.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e8be00640 1 -- 172.21.15.113:0/2598528908 learned_addr learned my addr 172.21.15.113:0/2598528908 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:58:55.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e99200640 1 --2- 172.21.15.113:0/2598528908 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 0x7f9e940c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e93400640 1 -- 172.21.15.113:0/2598528908 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3939553836 0 0) 0x7f9e940c5630 con 0x7f9e940c9ab0 2024-03-04T07:58:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e93400640 1 -- 172.21.15.113:0/2598528908 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e78003530 con 0x7f9e940c9ab0 2024-03-04T07:58:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e99200640 1 -- 172.21.15.113:0/2598528908 >> v1:172.21.15.136:6789/0 conn(0x7f9e94152ac0 legacy=0x7f9e9406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e99200640 1 -- 172.21.15.113:0/2598528908 >> v1:172.21.15.113:6789/0 conn(0x7f9e940c9ab0 legacy=0x7f9e9415c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.260+0000 7f9e99200640 1 -- 172.21.15.113:0/2598528908 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940c58d0 con 0x7f9e940c8fe0 2024-03-04T07:58:55.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e99200640 1 --2- 172.21.15.113:0/2598528908 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 0x7f9e940c9430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009dc0 tx=0x7f9e8c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e203337278443091 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:55.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e93400640 1 -- 172.21.15.113:0/2598528908 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0b2070 con 0x7f9e940c8fe0 2024-03-04T07:58:55.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e93400640 1 -- 172.21.15.113:0/2598528908 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c0a58b0 con 0x7f9e940c8fe0 2024-03-04T07:58:55.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e93400640 1 -- 172.21.15.113:0/2598528908 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0a5c80 con 0x7f9e940c8fe0 2024-03-04T07:58:55.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e99c00640 1 -- 172.21.15.113:0/2598528908 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 msgr2=0x7f9e940c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:55.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e99c00640 1 --2- 172.21.15.113:0/2598528908 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 0x7f9e940c9430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009dc0 tx=0x7f9e8c01c900 comp rx=0 tx=0).stop 2024-03-04T07:58:55.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e99c00640 1 -- 172.21.15.113:0/2598528908 shutdown_connections 2024-03-04T07:58:55.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e99c00640 1 --2- 172.21.15.113:0/2598528908 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 0x7f9e940c9430 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:55.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e99c00640 1 -- 172.21.15.113:0/2598528908 >> 172.21.15.113:0/2598528908 conn(0x7f9e94072e30 msgr2=0x7f9e94066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:55.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e99c00640 1 -- 172.21.15.113:0/2598528908 shutdown_connections 2024-03-04T07:58:55.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.264+0000 7f9e99c00640 1 -- 172.21.15.113:0/2598528908 wait complete. 2024-03-04T07:58:55.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 Processor -- start 2024-03-04T07:58:55.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 -- start start 2024-03-04T07:58:55.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 0x7f9e9414b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:55.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e94152ac0 0x7f9e94146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e941f4b70 0x7f9e941f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e94152ea0 con 0x7f9e941f4b70 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940c8a00 con 0x7f9e94152ac0 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e940c70d0 con 0x7f9e940c8fe0 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e941f4b70 0x7f9e941f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e941f4b70 0x7f9e941f5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35104/0 (socket says 172.21.15.113:35104) 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99200640 1 -- 172.21.15.113:0/3404376541 learned_addr learned my addr 172.21.15.113:0/3404376541 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e8be00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e94152ac0 0x7f9e94146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e93e00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 0x7f9e9414b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99200640 1 -- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 msgr2=0x7f9e9414b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99200640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 0x7f9e9414b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99200640 1 -- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e94152ac0 msgr2=0x7f9e94146f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99200640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e94152ac0 0x7f9e94146f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99200640 1 -- 172.21.15.113:0/3404376541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9415b710 con 0x7f9e941f4b70 2024-03-04T07:58:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99200640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e941f4b70 0x7f9e941f5020 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c01cde0 tx=0x7f9e8c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e90c00640 1 -- 172.21.15.113:0/3404376541 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0b2050 con 0x7f9e941f4b70 2024-03-04T07:58:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e90c00640 1 -- 172.21.15.113:0/3404376541 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c0be070 con 0x7f9e941f4b70 2024-03-04T07:58:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 -- 172.21.15.113:0/3404376541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9415b150 con 0x7f9e941f4b70 2024-03-04T07:58:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e90c00640 1 -- 172.21.15.113:0/3404376541 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0ad890 con 0x7f9e941f4b70 2024-03-04T07:58:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 -- 172.21.15.113:0/3404376541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9414aec0 con 0x7f9e941f4b70 2024-03-04T07:58:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.268+0000 7f9e99c00640 1 -- 172.21.15.113:0/3404376541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e94075a80 con 0x7f9e941f4b70 2024-03-04T07:58:55.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.272+0000 7f9e90c00640 1 -- 172.21.15.113:0/3404376541 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e8c0ada30 con 0x7f9e941f4b70 2024-03-04T07:58:55.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.272+0000 7f9e90c00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e70066f60 0x7f9e70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:55.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.272+0000 7f9e90c00640 1 -- 172.21.15.113:0/3404376541 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f9e8c120630 con 0x7f9e941f4b70 2024-03-04T07:58:55.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.272+0000 7f9e93e00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e70066f60 0x7f9e70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:55.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.272+0000 7f9e93e00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e70066f60 0x7f9e70069420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9e840015b0 tx=0x7f9e84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:55.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.280+0000 7f9e90c00640 1 -- 172.21.15.113:0/3404376541 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9e8c016580 con 0x7f9e941f4b70 2024-03-04T07:58:55.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.604+0000 7f9e99c00640 1 -- 172.21.15.113:0/3404376541 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e940c4c00 con 0x7f9e941f4b70 2024-03-04T07:58:55.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.604+0000 7f9e90c00640 1 -- 172.21.15.113:0/3404376541 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5817 (secure 0 0 0) 0x7f9e8c0f1170 con 0x7f9e941f4b70 2024-03-04T07:58:55.603 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:55.603 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:58:50.802501+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:resolve","state_seq":18,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240]},"id":1}]} 2024-03-04T07:58:55.603 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 90 2024-03-04T07:58:55.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 -- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e70066f60 msgr2=0x7f9e70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:55.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e70066f60 0x7f9e70069420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9e840015b0 tx=0x7f9e84009040 comp rx=0 tx=0).stop 2024-03-04T07:58:55.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 -- 172.21.15.113:0/3404376541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e941f4b70 msgr2=0x7f9e941f5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:55.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e941f4b70 0x7f9e941f5020 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c01cde0 tx=0x7f9e8c0196a0 comp rx=0 tx=0).stop 2024-03-04T07:58:55.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 -- 172.21.15.113:0/3404376541 shutdown_connections 2024-03-04T07:58:55.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e70066f60 0x7f9e70069420 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:55.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e941f4b70 0x7f9e941f5020 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:55.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e94152ac0 0x7f9e94146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:55.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 --2- 172.21.15.113:0/3404376541 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e940c8fe0 0x7f9e9414b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:55.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 -- 172.21.15.113:0/3404376541 >> 172.21.15.113:0/3404376541 conn(0x7f9e94072e30 msgr2=0x7f9e9414db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:55.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 -- 172.21.15.113:0/3404376541 shutdown_connections 2024-03-04T07:58:55.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:55.608+0000 7f9e99c00640 1 -- 172.21.15.113:0/3404376541 wait complete. 2024-03-04T07:58:55.630 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:58:56.396 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 15% (35/226) Updating files: 16% (37/226) Updating files: 17% (39/226) Updating files: 18% (41/226) Updating files: 19% (43/226) Updating files: 20% (46/226) Updating files: 21% (48/226) Updating files: 22% (50/226) Updating files: 23% (52/226) Updating files: 24% (55/226) Updating files: 25% (57/226) Updating files: 26% (59/226) Updating files: 27% (62/226) Updating files: 28% (64/226) Updating files: 29% (66/226) Updating files: 30% (68/226) Updating files: 31% (71/226) Updating files: 32% (73/226) Updating files: 33% (75/226) Updating files: 34% (77/226) Updating files: 35% (80/226) Updating files: 36% (82/226) Updating files: 37% (84/226) Updating files: 38% (86/226) Updating files: 39% (89/226) Updating files: 40% (91/226) Updating files: 41% (93/226) Updating files: 42% (95/226) Updating files: 43% (98/226) Updating files: 44% (100/226) Updating files: 45% (102/226) Updating files: 46% (104/226) Updating files: 47% (107/226) Updating files: 48% (109/226) Updating files: 49% (111/226) Updating files: 50% (113/226) Updating files: 51% (116/226) Updating files: 52% (118/226) Updating files: 53% (120/226) Updating files: 54% (123/226) Updating files: 55% (125/226) Updating files: 56% (127/226) Updating files: 57% (129/226) Updating files: 58% (132/226) Updating files: 59% (134/226) Updating files: 60% (136/226) Updating files: 61% (138/226) Updating files: 62% (141/226) Updating files: 63% (143/226) Updating files: 64% (145/226) Updating files: 65% (147/226) Updating files: 66% (150/226) Updating files: 67% (152/226) Updating files: 68% (154/226) Updating files: 69% (156/226) Updating files: 70% (159/226) Updating files: 71% (161/226) Updating files: 72% (163/226) Updating files: 73% (165/226) Updating files: 74% (168/226) Updating files: 75% (170/226) Updating files: 76% (172/226) Updating files: 77% (175/226) Updating files: 78% (177/226) Updating files: 79% (179/226) Updating files: 80% (181/226) Updating files: 81% (184/226) Updating files: 82% (186/226) Updating files: 83% (188/226) Updating files: 84% (190/226) Updating files: 85% (193/226) Updating files: 86% (195/226) Updating files: 87% (197/226) Updating files: 88% (199/226) Updating files: 89% (202/226) Updating files: 90% (204/226) Updating files: 91% (206/226) Updating files: 92% (208/226) Updating files: 93% (211/226) Updating files: 94% (213/226) Updating files: 95% (215/226) Updating files: 96% (217/226) Updating files: 97% (220/226) Updating files: 98% (222/226) Updating files: 99% (224/226) Updating files: 100% (226/226) Updating files: 100% (226/226), done. 2024-03-04T07:58:56.422 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 7eea40ea37f v0.40 2024-03-04T07:58:56.427 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.40 2024-03-04T07:58:56.840 INFO:tasks.workunit.client.0.smithi113.stdout:v0.41 2024-03-04T07:58:56.841 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 41 -eq 48 ] 2024-03-04T07:58:56.841 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.41 2024-03-04T07:58:56.841 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.41 2024-03-04T07:58:56.841 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.41 2024-03-04T07:58:57.631 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:58:57.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f6400640 1 Processor -- start 2024-03-04T07:58:57.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f6400640 1 -- start start 2024-03-04T07:58:57.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f6400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb3f0070df0 0x7fb3f00c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:57.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f6400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f006fb70 0x7fb3f00cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:57.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f6400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3f00c9c50 con 0x7fb3f0157ef0 2024-03-04T07:58:57.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f6400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3f015a9a0 con 0x7fb3f0070df0 2024-03-04T07:58:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f6400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f006de30 con 0x7fb3f006fb70 2024-03-04T07:58:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f5a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f006fb70 0x7fb3f00cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3efe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb3f0070df0 0x7fb3f00c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f5a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f006fb70 0x7fb3f00cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:37418/0 (socket says 172.21.15.113:37418) 2024-03-04T07:58:57.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f5a00640 1 -- 172.21.15.113:0/1246645270 learned_addr learned my addr 172.21.15.113:0/1246645270 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f5a00640 1 -- 172.21.15.113:0/1246645270 >> v2:172.21.15.136:3300/0 conn(0x7fb3f0070df0 msgr2=0x7fb3f00c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:58:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f5a00640 1 --2- 172.21.15.113:0/1246645270 >> v2:172.21.15.136:3300/0 conn(0x7fb3f0070df0 0x7fb3f00c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f5a00640 1 -- 172.21.15.113:0/1246645270 >> v1:172.21.15.136:6789/0 conn(0x7fb3f0157ef0 legacy=0x7fb3f015b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:57.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.788+0000 7fb3f5a00640 1 -- 172.21.15.113:0/1246645270 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f006c210 con 0x7fb3f006fb70 2024-03-04T07:58:57.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3f5a00640 1 --2- 172.21.15.113:0/1246645270 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f006fb70 0x7fb3f00cdfe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009e50 tx=0x7fb3e801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e68db8932a89c28 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:57.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3ef400640 1 -- 172.21.15.113:0/1246645270 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80a5810 con 0x7fb3f006fb70 2024-03-04T07:58:57.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3ef400640 1 -- 172.21.15.113:0/1246645270 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e80a59b0 con 0x7fb3f006fb70 2024-03-04T07:58:57.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3ef400640 1 -- 172.21.15.113:0/1246645270 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80a5d80 con 0x7fb3f006fb70 2024-03-04T07:58:57.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3f6400640 1 -- 172.21.15.113:0/1246645270 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f006fb70 msgr2=0x7fb3f00cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:57.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3f6400640 1 --2- 172.21.15.113:0/1246645270 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f006fb70 0x7fb3f00cdfe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009e50 tx=0x7fb3e801ca30 comp rx=0 tx=0).stop 2024-03-04T07:58:57.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3f6400640 1 -- 172.21.15.113:0/1246645270 shutdown_connections 2024-03-04T07:58:57.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3f6400640 1 --2- 172.21.15.113:0/1246645270 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f006fb70 0x7fb3f00cdfe0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:57.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3f6400640 1 --2- 172.21.15.113:0/1246645270 >> v2:172.21.15.136:3300/0 conn(0x7fb3f0070df0 0x7fb3f00c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:57.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3f6400640 1 -- 172.21.15.113:0/1246645270 >> 172.21.15.113:0/1246645270 conn(0x7fb3f0072f40 msgr2=0x7fb3f00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:57.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3f6400640 1 -- 172.21.15.113:0/1246645270 shutdown_connections 2024-03-04T07:58:57.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.792+0000 7fb3f6400640 1 -- 172.21.15.113:0/1246645270 wait complete. 2024-03-04T07:58:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f6400640 1 Processor -- start 2024-03-04T07:58:57.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f6400640 1 -- start start 2024-03-04T07:58:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f6400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f006fb70 0x7fb3f01f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f6400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f0070df0 0x7fb3f0150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f0157ef0 0x7fb3f014e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:57.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f6400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f01582d0 con 0x7fb3f0157ef0 2024-03-04T07:58:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f6400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f015b890 con 0x7fb3f006fb70 2024-03-04T07:58:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f6400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f0158450 con 0x7fb3f0070df0 2024-03-04T07:58:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f0157ef0 0x7fb3f014e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f0157ef0 0x7fb3f014e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41776/0 (socket says 172.21.15.113:41776) 2024-03-04T07:58:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f5a00640 1 -- 172.21.15.113:0/2460283699 learned_addr learned my addr 172.21.15.113:0/2460283699 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:58:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f5000640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f006fb70 0x7fb3f01f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3efe00640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f0070df0 0x7fb3f0150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f5a00640 1 -- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f0070df0 msgr2=0x7fb3f0150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f5a00640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f0070df0 0x7fb3f0150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f5a00640 1 -- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f006fb70 msgr2=0x7fb3f01f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f5a00640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f006fb70 0x7fb3f01f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f5a00640 1 -- 172.21.15.113:0/2460283699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0153a10 con 0x7fb3f0157ef0 2024-03-04T07:58:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f5a00640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f0157ef0 0x7fb3f014e040 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80af040 tx=0x7fb3e80197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3ecc00640 1 -- 172.21.15.113:0/2460283699 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80b8670 con 0x7fb3f0157ef0 2024-03-04T07:58:57.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3ecc00640 1 -- 172.21.15.113:0/2460283699 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3e80be070 con 0x7fb3f0157ef0 2024-03-04T07:58:57.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f6400640 1 -- 172.21.15.113:0/2460283699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f0153cb0 con 0x7fb3f0157ef0 2024-03-04T07:58:57.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3ecc00640 1 -- 172.21.15.113:0/2460283699 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80b8a70 con 0x7fb3f0157ef0 2024-03-04T07:58:57.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.796+0000 7fb3f6400640 1 -- 172.21.15.113:0/2460283699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f0154210 con 0x7fb3f0157ef0 2024-03-04T07:58:57.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.800+0000 7fb3f6400640 1 -- 172.21.15.113:0/2460283699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3f0154450 con 0x7fb3f0157ef0 2024-03-04T07:58:57.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.800+0000 7fb3ecc00640 1 -- 172.21.15.113:0/2460283699 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3e80b8c10 con 0x7fb3f0157ef0 2024-03-04T07:58:57.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.800+0000 7fb3ecc00640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3d0066f60 0x7fb3d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:58:57.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.800+0000 7fb3ecc00640 1 -- 172.21.15.113:0/2460283699 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7fb3e81204d0 con 0x7fb3f0157ef0 2024-03-04T07:58:57.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.800+0000 7fb3f5000640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3d0066f60 0x7fb3d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:58:57.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.804+0000 7fb3f5000640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3d0066f60 0x7fb3d0069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4001770 tx=0x7fb3e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:58:57.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:57.804+0000 7fb3ecc00640 1 -- 172.21.15.113:0/2460283699 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb3e8016620 con 0x7fb3f0157ef0 2024-03-04T07:58:58.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.024+0000 7fb3f6400640 1 -- 172.21.15.113:0/2460283699 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3f0075a80 con 0x7fb3f0157ef0 2024-03-04T07:58:58.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.024+0000 7fb3ecc00640 1 -- 172.21.15.113:0/2460283699 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+5819 (secure 0 0 0) 0x7fb3e80f0890 con 0x7fb3f0157ef0 2024-03-04T07:58:58.020 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:58:58.020 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:58:56.790379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6240,6372]},"id":1}]} 2024-03-04T07:58:58.021 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 91 2024-03-04T07:58:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 -- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3d0066f60 msgr2=0x7fb3d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3d0066f60 0x7fb3d0069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4001770 tx=0x7fb3e4009040 comp rx=0 tx=0).stop 2024-03-04T07:58:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 -- 172.21.15.113:0/2460283699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f0157ef0 msgr2=0x7fb3f014e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:58:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f0157ef0 0x7fb3f014e040 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80af040 tx=0x7fb3e80197d0 comp rx=0 tx=0).stop 2024-03-04T07:58:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 -- 172.21.15.113:0/2460283699 shutdown_connections 2024-03-04T07:58:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3d0066f60 0x7fb3d0069420 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:58.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3f0157ef0 0x7fb3f014e040 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3f0070df0 0x7fb3f0150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 --2- 172.21.15.113:0/2460283699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f006fb70 0x7fb3f01f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:58:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 -- 172.21.15.113:0/2460283699 >> 172.21.15.113:0/2460283699 conn(0x7fb3f0072f40 msgr2=0x7fb3f0159700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:58:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 -- 172.21.15.113:0/2460283699 shutdown_connections 2024-03-04T07:58:58.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:58:58.028+0000 7fb3f6400640 1 -- 172.21.15.113:0/2460283699 wait complete. 2024-03-04T07:58:58.056 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:59:00.058 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:59:00.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.268+0000 7fd33e600640 1 Processor -- start 2024-03-04T07:59:00.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd33e600640 1 -- start start 2024-03-04T07:59:00.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd33e600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd338152a50 0x7fd3380ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:00.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd33e600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd33815c4e0 0x7fd3380ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:00.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd33e600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd338152e30 con 0x7fd33806ff20 2024-03-04T07:59:00.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd33e600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3380c31b0 con 0x7fd33815c4e0 2024-03-04T07:59:00.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd33e600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33806f840 con 0x7fd338152a50 2024-03-04T07:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd336a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd33806ff20 0x7fd33815bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49600/0 (socket says 172.21.15.113:49600) 2024-03-04T07:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd336a00640 1 -- 172.21.15.113:0/4051018211 learned_addr learned my addr 172.21.15.113:0/4051018211 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd337400640 1 --2- 172.21.15.113:0/4051018211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd338152a50 0x7fd3380ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd337e00640 1 --2- 172.21.15.113:0/4051018211 >> v2:172.21.15.136:3300/0 conn(0x7fd33815c4e0 0x7fd3380ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd336000640 1 -- 172.21.15.113:0/4051018211 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212943458 0 0) 0x7fd338152e30 con 0x7fd33806ff20 2024-03-04T07:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd336000640 1 -- 172.21.15.113:0/4051018211 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd31c003530 con 0x7fd33806ff20 2024-03-04T07:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd337400640 1 -- 172.21.15.113:0/4051018211 >> v2:172.21.15.136:3300/0 conn(0x7fd33815c4e0 msgr2=0x7fd3380ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd337400640 1 --2- 172.21.15.113:0/4051018211 >> v2:172.21.15.136:3300/0 conn(0x7fd33815c4e0 0x7fd3380ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd337400640 1 -- 172.21.15.113:0/4051018211 >> v1:172.21.15.113:6789/0 conn(0x7fd33806ff20 legacy=0x7fd33815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:00.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.272+0000 7fd337400640 1 -- 172.21.15.113:0/4051018211 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33806f280 con 0x7fd338152a50 2024-03-04T07:59:00.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd337400640 1 --2- 172.21.15.113:0/4051018211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd338152a50 0x7fd3380ce120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3280097c0 tx=0x7fd32801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f3d064a41dc52bb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd336000640 1 -- 172.21.15.113:0/4051018211 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3280aa600 con 0x7fd338152a50 2024-03-04T07:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd336000640 1 -- 172.21.15.113:0/4051018211 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3280af070 con 0x7fd338152a50 2024-03-04T07:59:00.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd336000640 1 -- 172.21.15.113:0/4051018211 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3280aa9a0 con 0x7fd338152a50 2024-03-04T07:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd33e600640 1 -- 172.21.15.113:0/4051018211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd338152a50 msgr2=0x7fd3380ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd33e600640 1 --2- 172.21.15.113:0/4051018211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd338152a50 0x7fd3380ce120 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3280097c0 tx=0x7fd32801c4a0 comp rx=0 tx=0).stop 2024-03-04T07:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd33e600640 1 -- 172.21.15.113:0/4051018211 shutdown_connections 2024-03-04T07:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd33e600640 1 --2- 172.21.15.113:0/4051018211 >> v2:172.21.15.136:3300/0 conn(0x7fd33815c4e0 0x7fd3380ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd33e600640 1 --2- 172.21.15.113:0/4051018211 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd338152a50 0x7fd3380ce120 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd33e600640 1 -- 172.21.15.113:0/4051018211 >> 172.21.15.113:0/4051018211 conn(0x7fd3380734d0 msgr2=0x7fd338066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd33e600640 1 -- 172.21.15.113:0/4051018211 shutdown_connections 2024-03-04T07:59:00.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.276+0000 7fd33e600640 1 -- 172.21.15.113:0/4051018211 wait complete. 2024-03-04T07:59:00.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd33e600640 1 Processor -- start 2024-03-04T07:59:00.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd33e600640 1 -- start start 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd33e600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd33806ff20 0x7fd33814db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd33e600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd338152a50 0x7fd3381598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd33e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33815c4e0 0x7fd33815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd33e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd338165a00 con 0x7fd33815c4e0 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd33e600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd338165880 con 0x7fd338152a50 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd33e600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd338165b80 con 0x7fd33806ff20 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd337e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33815c4e0 0x7fd33815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd337e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33815c4e0 0x7fd33815b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41782/0 (socket says 172.21.15.113:41782) 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd337e00640 1 -- 172.21.15.113:0/1112784815 learned_addr learned my addr 172.21.15.113:0/1112784815 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd336a00640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd338152a50 0x7fd3381598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:00.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd337400640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd33806ff20 0x7fd33814db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:00.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd337e00640 1 -- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd33806ff20 msgr2=0x7fd33814db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:00.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd337e00640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd33806ff20 0x7fd33814db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:00.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd337e00640 1 -- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd338152a50 msgr2=0x7fd3381598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:00.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd337e00640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd338152a50 0x7fd3381598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:00.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.280+0000 7fd337e00640 1 -- 172.21.15.113:0/1112784815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380cd470 con 0x7fd33815c4e0 2024-03-04T07:59:00.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.284+0000 7fd337e00640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33815c4e0 0x7fd33815b470 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd33000efc0 tx=0x7fd33000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:00.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.284+0000 7fd32f400640 1 -- 172.21.15.113:0/1112784815 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330018020 con 0x7fd33815c4e0 2024-03-04T07:59:00.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.284+0000 7fd32f400640 1 -- 172.21.15.113:0/1112784815 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd330014480 con 0x7fd33815c4e0 2024-03-04T07:59:00.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.284+0000 7fd33e600640 1 -- 172.21.15.113:0/1112784815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3380cdf10 con 0x7fd33815c4e0 2024-03-04T07:59:00.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.284+0000 7fd32f400640 1 -- 172.21.15.113:0/1112784815 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd330014850 con 0x7fd33815c4e0 2024-03-04T07:59:00.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.284+0000 7fd33e600640 1 -- 172.21.15.113:0/1112784815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3381534f0 con 0x7fd33815c4e0 2024-03-04T07:59:00.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.284+0000 7fd33e600640 1 -- 172.21.15.113:0/1112784815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd338075c00 con 0x7fd33815c4e0 2024-03-04T07:59:00.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.288+0000 7fd32f400640 1 -- 172.21.15.113:0/1112784815 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd33001b020 con 0x7fd33815c4e0 2024-03-04T07:59:00.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.288+0000 7fd32f400640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd318066f60 0x7fd318069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:00.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.288+0000 7fd32f400640 1 -- 172.21.15.113:0/1112784815 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7fd330086700 con 0x7fd33815c4e0 2024-03-04T07:59:00.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.288+0000 7fd337400640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd318066f60 0x7fd318069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:00.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.288+0000 7fd337400640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd318066f60 0x7fd318069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd3280ae000 tx=0x7fd3280b5040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:00.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.292+0000 7fd32f400640 1 -- 172.21.15.113:0/1112784815 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd330022070 con 0x7fd33815c4e0 2024-03-04T07:59:00.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 -- 172.21.15.113:0/1112784815 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd338076a60 con 0x7fd33815c4e0 2024-03-04T07:59:00.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd32f400640 1 -- 172.21.15.113:0/1112784815 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+5819 (secure 0 0 0) 0x7fd330010040 con 0x7fd33815c4e0 2024-03-04T07:59:00.597 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:59:00.597 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:58:56.790379+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6240,6372]},"id":1}]} 2024-03-04T07:59:00.597 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 91 2024-03-04T07:59:00.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 -- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd318066f60 msgr2=0x7fd318069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:00.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd318066f60 0x7fd318069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd3280ae000 tx=0x7fd3280b5040 comp rx=0 tx=0).stop 2024-03-04T07:59:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 -- 172.21.15.113:0/1112784815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33815c4e0 msgr2=0x7fd33815b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33815c4e0 0x7fd33815b470 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd33000efc0 tx=0x7fd33000c490 comp rx=0 tx=0).stop 2024-03-04T07:59:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 -- 172.21.15.113:0/1112784815 shutdown_connections 2024-03-04T07:59:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd318066f60 0x7fd318069420 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd33815c4e0 0x7fd33815b470 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd338152a50 0x7fd3381598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 --2- 172.21.15.113:0/1112784815 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd33806ff20 0x7fd33814db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 -- 172.21.15.113:0/1112784815 >> 172.21.15.113:0/1112784815 conn(0x7fd3380734d0 msgr2=0x7fd3381628d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:59:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.600+0000 7fd33e600640 1 -- 172.21.15.113:0/1112784815 shutdown_connections 2024-03-04T07:59:00.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:00.604+0000 7fd33e600640 1 -- 172.21.15.113:0/1112784815 wait complete. 2024-03-04T07:59:00.627 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 91, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 91, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6240": { "addr": "172.21.15.113:6836/986515123", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 986515123, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 986515123, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6240, "incarnation": 77, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6372": { "addr": "172.21.15.113:6833/2732565639", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2732565639, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2732565639, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6372, "incarnation": 83, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 40 }, "gid_6534": { "addr": "172.21.15.113:6837/4197507712", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 4197507712, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 4197507712, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6534, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:reconnect", "state_seq": 20 } }, "last_failure": 0, "last_failure_osd_epoch": 61, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T07:58:56.790379+0000", "qdb_cluster": [ 6240, 6372 ], "qdb_leader": 6372, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6372, "mds_1": 6534, "mds_2": 6240 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T07:59:02.127 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:59:02.131 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:59:02.628 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:59:02.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f2920800640 1 Processor -- start 2024-03-04T07:59:02.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f2920800640 1 -- start start 2024-03-04T07:59:02.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f2920800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2918070120 0x7f2918158930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:02.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f2920800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29180c9db0 0x7f2918158e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:02.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f2920800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29180c5380 con 0x7f2918152d60 2024-03-04T07:59:02.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f2920800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29180c3440 con 0x7f29180c9db0 2024-03-04T07:59:02.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f2920800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291806f620 con 0x7f2918070120 2024-03-04T07:59:02.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f291e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29180c9db0 0x7f2918158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:02.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f291e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29180c9db0 0x7f2918158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41796/0 (socket says 172.21.15.113:41796) 2024-03-04T07:59:02.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f291e800640 1 -- 172.21.15.113:0/533450922 learned_addr learned my addr 172.21.15.113:0/533450922 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:59:02.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f291d400640 1 --2- 172.21.15.113:0/533450922 >> v2:172.21.15.136:3300/0 conn(0x7f2918070120 0x7f2918158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:02.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f291e800640 1 -- 172.21.15.113:0/533450922 >> v2:172.21.15.136:3300/0 conn(0x7f2918070120 msgr2=0x7f2918158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:59:02.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f291e800640 1 --2- 172.21.15.113:0/533450922 >> v2:172.21.15.136:3300/0 conn(0x7f2918070120 0x7f2918158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:02.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f291e800640 1 -- 172.21.15.113:0/533450922 >> v1:172.21.15.136:6789/0 conn(0x7f2918152d60 legacy=0x7f29180ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:02.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.792+0000 7f291e800640 1 -- 172.21.15.113:0/533450922 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180c36e0 con 0x7f29180c9db0 2024-03-04T07:59:02.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f291e800640 1 --2- 172.21.15.113:0/533450922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 0x7f2918158e70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f29180cbef0 tx=0x7f291401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c0432739f0b11b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f291ca00640 1 -- 172.21.15.113:0/533450922 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29140b4070 con 0x7f29180c9db0 2024-03-04T07:59:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f291ca00640 1 -- 172.21.15.113:0/533450922 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29140af400 con 0x7f29180c9db0 2024-03-04T07:59:02.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f291ca00640 1 -- 172.21.15.113:0/533450922 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29140af7d0 con 0x7f29180c9db0 2024-03-04T07:59:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f2920800640 1 -- 172.21.15.113:0/533450922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 msgr2=0x7f2918158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f2920800640 1 --2- 172.21.15.113:0/533450922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 0x7f2918158e70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f29180cbef0 tx=0x7f291401f1e0 comp rx=0 tx=0).stop 2024-03-04T07:59:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f2920800640 1 -- 172.21.15.113:0/533450922 shutdown_connections 2024-03-04T07:59:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f2920800640 1 --2- 172.21.15.113:0/533450922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 0x7f2918158e70 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f2920800640 1 --2- 172.21.15.113:0/533450922 >> v2:172.21.15.136:3300/0 conn(0x7f2918070120 0x7f2918158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f2920800640 1 -- 172.21.15.113:0/533450922 >> 172.21.15.113:0/533450922 conn(0x7f2918066db0 msgr2=0x7f29180671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:59:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f2920800640 1 -- 172.21.15.113:0/533450922 shutdown_connections 2024-03-04T07:59:02.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.800+0000 7f2920800640 1 -- 172.21.15.113:0/533450922 wait complete. 2024-03-04T07:59:02.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2920800640 1 Processor -- start 2024-03-04T07:59:02.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2920800640 1 -- start start 2024-03-04T07:59:02.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2920800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2918070120 0x7f29181f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:02.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2920800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 0x7f29181f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:02.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2920800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2918152d60 0x7f29181f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:02.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2920800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29180c39c0 con 0x7f29180c9db0 2024-03-04T07:59:02.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2920800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29180c5020 con 0x7f2918070120 2024-03-04T07:59:02.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2920800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29180c3b40 con 0x7f2918152d60 2024-03-04T07:59:02.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 0x7f29181f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:02.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 0x7f29181f4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41798/0 (socket says 172.21.15.113:41798) 2024-03-04T07:59:02.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291d400640 1 -- 172.21.15.113:0/2131591508 learned_addr learned my addr 172.21.15.113:0/2131591508 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:59:02.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291de00640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2918070120 0x7f29181f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:02.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291e800640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2918152d60 0x7f29181f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:02.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291d400640 1 -- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2918152d60 msgr2=0x7f29181f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:02.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291d400640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2918152d60 0x7f29181f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:02.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291d400640 1 -- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2918070120 msgr2=0x7f29181f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:02.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291d400640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2918070120 0x7f29181f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:02.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291d400640 1 -- 172.21.15.113:0/2131591508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180c85c0 con 0x7f29180c9db0 2024-03-04T07:59:02.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f291d400640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 0x7f29181f4780 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f290800d5f0 tx=0x7f290800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:02.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2912000640 1 -- 172.21.15.113:0/2131591508 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080125f0 con 0x7f29180c9db0 2024-03-04T07:59:02.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2912000640 1 -- 172.21.15.113:0/2131591508 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908019070 con 0x7f29180c9db0 2024-03-04T07:59:02.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2912000640 1 -- 172.21.15.113:0/2131591508 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080129b0 con 0x7f29180c9db0 2024-03-04T07:59:02.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.804+0000 7f2920800640 1 -- 172.21.15.113:0/2131591508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29180c8860 con 0x7f29180c9db0 2024-03-04T07:59:02.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.808+0000 7f2920800640 1 -- 172.21.15.113:0/2131591508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29181f4e40 con 0x7f29180c9db0 2024-03-04T07:59:02.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.808+0000 7f2920800640 1 -- 172.21.15.113:0/2131591508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29181f28a0 con 0x7f29180c9db0 2024-03-04T07:59:02.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.808+0000 7f2912000640 1 -- 172.21.15.113:0/2131591508 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2908012b50 con 0x7f29180c9db0 2024-03-04T07:59:02.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.808+0000 7f2912000640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28fc066f60 0x7f28fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:02.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.812+0000 7f2912000640 1 -- 172.21.15.113:0/2131591508 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f2908085770 con 0x7f29180c9db0 2024-03-04T07:59:02.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.812+0000 7f291de00640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28fc066f60 0x7f28fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:02.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.812+0000 7f291de00640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28fc066f60 0x7f28fc069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f290c001810 tx=0x7f290c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:02.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:02.816+0000 7f2912000640 1 -- 172.21.15.113:0/2131591508 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2908055c50 con 0x7f29180c9db0 2024-03-04T07:59:03.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 -- 172.21.15.113:0/2131591508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2918075c00 con 0x7f29180c9db0 2024-03-04T07:59:03.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2912000640 1 -- 172.21.15.113:0/2131591508 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f2908059cf0 con 0x7f29180c9db0 2024-03-04T07:59:03.012 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:59:03.013 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:59:01.820308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240]},"id":1}]} 2024-03-04T07:59:03.013 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 92 2024-03-04T07:59:03.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 -- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28fc066f60 msgr2=0x7f28fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28fc066f60 0x7f28fc069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f290c001810 tx=0x7f290c009040 comp rx=0 tx=0).stop 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 -- 172.21.15.113:0/2131591508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 msgr2=0x7f29181f4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 0x7f29181f4780 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f290800d5f0 tx=0x7f290800dab0 comp rx=0 tx=0).stop 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 -- 172.21.15.113:0/2131591508 shutdown_connections 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2918152d60 0x7f29181f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28fc066f60 0x7f28fc069420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29180c9db0 0x7f29181f4780 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 --2- 172.21.15.113:0/2131591508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2918070120 0x7f29181f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 -- 172.21.15.113:0/2131591508 >> 172.21.15.113:0/2131591508 conn(0x7f2918066db0 msgr2=0x7f29180ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 -- 172.21.15.113:0/2131591508 shutdown_connections 2024-03-04T07:59:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:03.016+0000 7f2920800640 1 -- 172.21.15.113:0/2131591508 wait complete. 2024-03-04T07:59:03.035 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:59:05.036 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:59:05.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f33c00640 1 Processor -- start 2024-03-04T07:59:05.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f33c00640 1 -- start start 2024-03-04T07:59:05.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f33c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f2c0c8200 0x7f0f2c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:05.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f33c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0f2c152bd0 0x7f0f2c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:05.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f33c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2c1591f0 con 0x7f0f2c1527f0 2024-03-04T07:59:05.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f33c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c071df0 con 0x7f0f2c0c8200 2024-03-04T07:59:05.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f33c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c1597b0 con 0x7f0f2c152bd0 2024-03-04T07:59:05.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f2be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f2c0c8200 0x7f0f2c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:05.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f2be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f2c0c8200 0x7f0f2c070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41814/0 (socket says 172.21.15.113:41814) 2024-03-04T07:59:05.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f2be00640 1 -- 172.21.15.113:0/3543797292 learned_addr learned my addr 172.21.15.113:0/3543797292 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:59:05.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f31c00640 1 --2- 172.21.15.113:0/3543797292 >> v2:172.21.15.136:3300/0 conn(0x7f0f2c152bd0 0x7f0f2c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:05.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f2be00640 1 -- 172.21.15.113:0/3543797292 >> v2:172.21.15.136:3300/0 conn(0x7f0f2c152bd0 msgr2=0x7f0f2c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:59:05.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f2be00640 1 --2- 172.21.15.113:0/3543797292 >> v2:172.21.15.136:3300/0 conn(0x7f0f2c152bd0 0x7f0f2c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:05.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f2be00640 1 -- 172.21.15.113:0/3543797292 >> v1:172.21.15.136:6789/0 conn(0x7f0f2c1527f0 legacy=0x7f0f2c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:05.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.196+0000 7f0f2be00640 1 -- 172.21.15.113:0/3543797292 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c1594d0 con 0x7f0f2c0c8200 2024-03-04T07:59:05.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f2be00640 1 --2- 172.21.15.113:0/3543797292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 0x7f0f2c070d40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0712b0 tx=0x7f0f1801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a773a002124fff88 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:05.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f2b400640 1 -- 172.21.15.113:0/3543797292 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f180ab430 con 0x7f0f2c0c8200 2024-03-04T07:59:05.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f2b400640 1 -- 172.21.15.113:0/3543797292 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f180b0070 con 0x7f0f2c0c8200 2024-03-04T07:59:05.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f2b400640 1 -- 172.21.15.113:0/3543797292 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f180ab7f0 con 0x7f0f2c0c8200 2024-03-04T07:59:05.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f33c00640 1 -- 172.21.15.113:0/3543797292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 msgr2=0x7f0f2c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:05.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f33c00640 1 --2- 172.21.15.113:0/3543797292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 0x7f0f2c070d40 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0712b0 tx=0x7f0f1801c520 comp rx=0 tx=0).stop 2024-03-04T07:59:05.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f33c00640 1 -- 172.21.15.113:0/3543797292 shutdown_connections 2024-03-04T07:59:05.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f33c00640 1 --2- 172.21.15.113:0/3543797292 >> v2:172.21.15.136:3300/0 conn(0x7f0f2c152bd0 0x7f0f2c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:05.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f33c00640 1 --2- 172.21.15.113:0/3543797292 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 0x7f0f2c070d40 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:05.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f33c00640 1 -- 172.21.15.113:0/3543797292 >> 172.21.15.113:0/3543797292 conn(0x7f0f2c06e820 msgr2=0x7f0f2c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:59:05.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f33c00640 1 -- 172.21.15.113:0/3543797292 shutdown_connections 2024-03-04T07:59:05.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.200+0000 7f0f33c00640 1 -- 172.21.15.113:0/3543797292 wait complete. 2024-03-04T07:59:05.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f33c00640 1 Processor -- start 2024-03-04T07:59:05.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f33c00640 1 -- start start 2024-03-04T07:59:05.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f33c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 0x7f0f2c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:05.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f33c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f2c1527f0 0x7f0f2c1f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:05.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f33c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f2c152bd0 0x7f0f2c0b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:05.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f33c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c1654f0 con 0x7f0f2c0c8200 2024-03-04T07:59:05.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f33c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c165370 con 0x7f0f2c152bd0 2024-03-04T07:59:05.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f33c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c165670 con 0x7f0f2c1527f0 2024-03-04T07:59:05.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f31200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 0x7f0f2c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:05.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f31200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 0x7f0f2c15c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41822/0 (socket says 172.21.15.113:41822) 2024-03-04T07:59:05.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f31200640 1 -- 172.21.15.113:0/79610956 learned_addr learned my addr 172.21.15.113:0/79610956 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:59:05.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f31c00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f2c152bd0 0x7f0f2c0b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:05.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f2be00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f2c1527f0 0x7f0f2c1f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:05.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f31200640 1 -- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f2c1527f0 msgr2=0x7f0f2c1f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:05.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f31200640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f2c1527f0 0x7f0f2c1f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:05.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f31200640 1 -- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f2c152bd0 msgr2=0x7f0f2c0b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:05.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f31200640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f2c152bd0 0x7f0f2c0b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:05.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f31200640 1 -- 172.21.15.113:0/79610956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c1f1690 con 0x7f0f2c0c8200 2024-03-04T07:59:05.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.204+0000 7f0f31c00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f2c152bd0 0x7f0f2c0b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T07:59:05.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f31200640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 0x7f0f2c15c880 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000c530 tx=0x7f0f2000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:05.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f28c00640 1 -- 172.21.15.113:0/79610956 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20016020 con 0x7f0f2c0c8200 2024-03-04T07:59:05.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f28c00640 1 -- 172.21.15.113:0/79610956 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f20005220 con 0x7f0f2c0c8200 2024-03-04T07:59:05.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f33c00640 1 -- 172.21.15.113:0/79610956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2c1f1950 con 0x7f0f2c0c8200 2024-03-04T07:59:05.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f28c00640 1 -- 172.21.15.113:0/79610956 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20007030 con 0x7f0f2c0c8200 2024-03-04T07:59:05.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f33c00640 1 -- 172.21.15.113:0/79610956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c1f1f10 con 0x7f0f2c0c8200 2024-03-04T07:59:05.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f33c00640 1 -- 172.21.15.113:0/79610956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2c1f25d0 con 0x7f0f2c0c8200 2024-03-04T07:59:05.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f28c00640 1 -- 172.21.15.113:0/79610956 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f200071d0 con 0x7f0f2c0c8200 2024-03-04T07:59:05.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f28c00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f0c066f90 0x7f0f0c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:05.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f28c00640 1 -- 172.21.15.113:0/79610956 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f0f20084f50 con 0x7f0f2c0c8200 2024-03-04T07:59:05.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.208+0000 7f0f2be00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f0c066f90 0x7f0f0c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:05.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.212+0000 7f0f2be00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f0c066f90 0x7f0f0c069450 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0f180af000 tx=0x7f0f180a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:05.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.220+0000 7f0f28c00640 1 -- 172.21.15.113:0/79610956 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0f2c1f25d0 con 0x7f0f2c0c8200 2024-03-04T07:59:05.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.480+0000 7f0f33c00640 1 -- 172.21.15.113:0/79610956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f2c075c00 con 0x7f0f2c0c8200 2024-03-04T07:59:05.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f28c00640 1 -- 172.21.15.113:0/79610956 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+5816 (secure 0 0 0) 0x7f0f20020020 con 0x7f0f2c0c8200 2024-03-04T07:59:05.480 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:59:05.480 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":92,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:59:01.820308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6240]},"id":1}]} 2024-03-04T07:59:05.480 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 92 2024-03-04T07:59:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 -- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f0c066f90 msgr2=0x7f0f0c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f0c066f90 0x7f0f0c069450 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0f180af000 tx=0x7f0f180a7000 comp rx=0 tx=0).stop 2024-03-04T07:59:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 -- 172.21.15.113:0/79610956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 msgr2=0x7f0f2c15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:05.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 0x7f0f2c15c880 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000c530 tx=0x7f0f2000c9f0 comp rx=0 tx=0).stop 2024-03-04T07:59:05.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 -- 172.21.15.113:0/79610956 shutdown_connections 2024-03-04T07:59:05.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0f2c152bd0 0x7f0f2c0b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:05.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0f2c1527f0 0x7f0f2c1f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:05.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0f0c066f90 0x7f0f0c069450 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:05.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 --2- 172.21.15.113:0/79610956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2c0c8200 0x7f0f2c15c880 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:05.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 -- 172.21.15.113:0/79610956 >> 172.21.15.113:0/79610956 conn(0x7f0f2c06e820 msgr2=0x7f0f2c157500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:59:05.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 -- 172.21.15.113:0/79610956 shutdown_connections 2024-03-04T07:59:05.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:05.484+0000 7f0f33c00640 1 -- 172.21.15.113:0/79610956 wait complete. 2024-03-04T07:59:05.514 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T07:59:07.515 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:59:07.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.684+0000 7f5376400640 1 Processor -- start 2024-03-04T07:59:07.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.684+0000 7f5376400640 1 -- start start 2024-03-04T07:59:07.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.684+0000 7f5376400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f53701582e0 0x7f53700701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:07.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.684+0000 7f5376400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 0x7f537015af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:07.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.684+0000 7f5376400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53701586c0 con 0x7f53700b0ed0 2024-03-04T07:59:07.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.688+0000 7f5376400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537006c210 con 0x7f53701582e0 2024-03-04T07:59:07.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.688+0000 7f5376400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537006c4f0 con 0x7f537006ddf0 2024-03-04T07:59:07.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.688+0000 7f536fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 0x7f537015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:07.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.688+0000 7f536f400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f53701582e0 0x7f53700701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:07.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.688+0000 7f536fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 0x7f537015af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:44012/0 (socket says 172.21.15.113:44012) 2024-03-04T07:59:07.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.688+0000 7f536fe00640 1 -- 172.21.15.113:0/1363653734 learned_addr learned my addr 172.21.15.113:0/1363653734 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:59:07.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.688+0000 7f536fe00640 1 -- 172.21.15.113:0/1363653734 >> v2:172.21.15.136:3300/0 conn(0x7f53701582e0 msgr2=0x7f53700701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:59:07.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.688+0000 7f536fe00640 1 --2- 172.21.15.113:0/1363653734 >> v2:172.21.15.136:3300/0 conn(0x7f53701582e0 0x7f53700701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:07.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.688+0000 7f536fe00640 1 -- 172.21.15.113:0/1363653734 >> v1:172.21.15.136:6789/0 conn(0x7f53700b0ed0 legacy=0x7f53700ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:07.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.688+0000 7f536fe00640 1 -- 172.21.15.113:0/1363653734 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537006f6a0 con 0x7f537006ddf0 2024-03-04T07:59:07.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f536fe00640 1 --2- 172.21.15.113:0/1363653734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 0x7f537015af40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5368009e50 tx=0x7f536801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f492e1bddc2485 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:07.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f536e000640 1 -- 172.21.15.113:0/1363653734 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53680a5700 con 0x7f537006ddf0 2024-03-04T07:59:07.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f536e000640 1 -- 172.21.15.113:0/1363653734 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53680a58a0 con 0x7f537006ddf0 2024-03-04T07:59:07.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f536e000640 1 -- 172.21.15.113:0/1363653734 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53680a5c70 con 0x7f537006ddf0 2024-03-04T07:59:07.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 -- 172.21.15.113:0/1363653734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 msgr2=0x7f537015af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:07.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 --2- 172.21.15.113:0/1363653734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 0x7f537015af40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5368009e50 tx=0x7f536801ca30 comp rx=0 tx=0).stop 2024-03-04T07:59:07.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 -- 172.21.15.113:0/1363653734 shutdown_connections 2024-03-04T07:59:07.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 --2- 172.21.15.113:0/1363653734 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 0x7f537015af40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:07.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 --2- 172.21.15.113:0/1363653734 >> v2:172.21.15.136:3300/0 conn(0x7f53701582e0 0x7f53700701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:07.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 -- 172.21.15.113:0/1363653734 >> 172.21.15.113:0/1363653734 conn(0x7f5370066db0 msgr2=0x7f53700671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:59:07.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 -- 172.21.15.113:0/1363653734 shutdown_connections 2024-03-04T07:59:07.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 -- 172.21.15.113:0/1363653734 wait complete. 2024-03-04T07:59:07.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 Processor -- start 2024-03-04T07:59:07.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 -- start start 2024-03-04T07:59:07.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 0x7f5370154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:07.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53700b0ed0 0x7f53700b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:07.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53701582e0 0x7f53700b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:07.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53700ce9e0 con 0x7f53700b0ed0 2024-03-04T07:59:07.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537006c7d0 con 0x7f53701582e0 2024-03-04T07:59:07.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f5376400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53700ceb60 con 0x7f537006ddf0 2024-03-04T07:59:07.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f536ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53700b0ed0 0x7f53700b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:07.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f536ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53700b0ed0 0x7f53700b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60502/0 (socket says 172.21.15.113:60502) 2024-03-04T07:59:07.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f536ea00640 1 -- 172.21.15.113:0/1615536669 learned_addr learned my addr 172.21.15.113:0/1615536669 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.692+0000 7f536f400640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 0x7f5370154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f536fe00640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53701582e0 0x7f53700b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f536ea00640 1 -- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 msgr2=0x7f5370154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f536ea00640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 0x7f5370154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f536ea00640 1 -- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53701582e0 msgr2=0x7f53700b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f536ea00640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53701582e0 0x7f53700b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f536ea00640 1 -- 172.21.15.113:0/1615536669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53701f3ef0 con 0x7f53700b0ed0 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f536ea00640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53700b0ed0 0x7f53700b14b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c420 tx=0x7f535c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f5367400640 1 -- 172.21.15.113:0/1615536669 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c0052d0 con 0x7f53700b0ed0 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f5367400640 1 -- 172.21.15.113:0/1615536669 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c005470 con 0x7f53700b0ed0 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f5367400640 1 -- 172.21.15.113:0/1615536669 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c007260 con 0x7f53700b0ed0 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f5376400640 1 -- 172.21.15.113:0/1615536669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53701f41d0 con 0x7f53700b0ed0 2024-03-04T07:59:07.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f5376400640 1 -- 172.21.15.113:0/1615536669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53701f4790 con 0x7f53700b0ed0 2024-03-04T07:59:07.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.696+0000 7f5376400640 1 -- 172.21.15.113:0/1615536669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5370068eb0 con 0x7f53700b0ed0 2024-03-04T07:59:07.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.700+0000 7f5367400640 1 -- 172.21.15.113:0/1615536669 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f535c007400 con 0x7f53700b0ed0 2024-03-04T07:59:07.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.700+0000 7f5367400640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5350066f90 0x7f5350069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:07.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.700+0000 7f5367400640 1 -- 172.21.15.113:0/1615536669 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f535c084960 con 0x7f53700b0ed0 2024-03-04T07:59:07.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.700+0000 7f536f400640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5350066f90 0x7f5350069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:07.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.700+0000 7f536f400640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5350066f90 0x7f5350069450 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5360009ac0 tx=0x7f5360009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:07.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.704+0000 7f5367400640 1 -- 172.21.15.113:0/1615536669 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f535c054e50 con 0x7f53700b0ed0 2024-03-04T07:59:07.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.904+0000 7f5376400640 1 -- 172.21.15.113:0/1615536669 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5370075c00 con 0x7f53700b0ed0 2024-03-04T07:59:07.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.904+0000 7f5367400640 1 -- 172.21.15.113:0/1615536669 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+5821 (secure 0 0 0) 0x7f535c058ef0 con 0x7f53700b0ed0 2024-03-04T07:59:07.903 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:59:07.903 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:59:07.046470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6240,6372,6534]},"id":1}]} 2024-03-04T07:59:07.903 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 93 2024-03-04T07:59:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 -- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5350066f90 msgr2=0x7f5350069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5350066f90 0x7f5350069450 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5360009ac0 tx=0x7f5360009190 comp rx=0 tx=0).stop 2024-03-04T07:59:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 -- 172.21.15.113:0/1615536669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53700b0ed0 msgr2=0x7f53700b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53700b0ed0 0x7f53700b14b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c420 tx=0x7f535c00c8e0 comp rx=0 tx=0).stop 2024-03-04T07:59:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 -- 172.21.15.113:0/1615536669 shutdown_connections 2024-03-04T07:59:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53701582e0 0x7f53700b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5350066f90 0x7f5350069450 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53700b0ed0 0x7f53700b14b0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:07.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 --2- 172.21.15.113:0/1615536669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f537006ddf0 0x7f5370154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:07.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 -- 172.21.15.113:0/1615536669 >> 172.21.15.113:0/1615536669 conn(0x7f5370066db0 msgr2=0x7f53701571e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:59:07.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 -- 172.21.15.113:0/1615536669 shutdown_connections 2024-03-04T07:59:07.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:07.908+0000 7f5376400640 1 -- 172.21.15.113:0/1615536669 wait complete. 2024-03-04T07:59:07.924 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=1, replacing gid=6145 2024-03-04T07:59:07.924 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.b 2024-03-04T07:59:08.399 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 19% (17/87) Updating files: 20% (18/87) Updating files: 21% (19/87) Updating files: 22% (20/87) Updating files: 24% (21/87) Updating files: 25% (22/87) Updating files: 26% (23/87) Updating files: 27% (24/87) Updating files: 28% (25/87) Updating files: 29% (26/87) Updating files: 31% (27/87) Updating files: 32% (28/87) Updating files: 33% (29/87) Updating files: 34% (30/87) Updating files: 35% (31/87) Updating files: 36% (32/87) Updating files: 37% (33/87) Updating files: 39% (34/87) Updating files: 40% (35/87) Updating files: 41% (36/87) Updating files: 42% (37/87) Updating files: 43% (38/87) Updating files: 44% (39/87) Updating files: 45% (40/87) Updating files: 47% (41/87) Updating files: 48% (42/87) Updating files: 49% (43/87) Updating files: 50% (44/87) Updating files: 51% (45/87) Updating files: 52% (46/87) Updating files: 54% (47/87) Updating files: 55% (48/87) Updating files: 56% (49/87) Updating files: 57% (50/87) Updating files: 58% (51/87) Updating files: 59% (52/87) Updating files: 60% (53/87) Updating files: 62% (54/87) Updating files: 63% (55/87) Updating files: 64% (56/87) Updating files: 65% (57/87) Updating files: 66% (58/87) Updating files: 67% (59/87) Updating files: 68% (60/87) Updating files: 70% (61/87) Updating files: 71% (62/87) Updating files: 72% (63/87) Updating files: 73% (64/87) Updating files: 74% (65/87) Updating files: 75% (66/87) Updating files: 77% (67/87) Updating files: 78% (68/87) Updating files: 79% (69/87) Updating files: 80% (70/87) Updating files: 81% (71/87) Updating files: 82% (72/87) Updating files: 83% (73/87) Updating files: 85% (74/87) Updating files: 86% (75/87) Updating files: 87% (76/87) Updating files: 88% (77/87) Updating files: 89% (78/87) Updating files: 90% (79/87) Updating files: 91% (80/87) Updating files: 93% (81/87) Updating files: 94% (82/87) Updating files: 95% (83/87) Updating files: 96% (84/87) Updating files: 97% (85/87) Updating files: 98% (86/87) Updating files: 100% (87/87) Updating files: 100% (87/87), done. 2024-03-04T07:59:09.084 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at c1345f7136a v0.41 2024-03-04T07:59:09.087 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.41 2024-03-04T07:59:12.065 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 42 -eq 48 ] 2024-03-04T07:59:12.066 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.42 2024-03-04T07:59:12.066 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.42 2024-03-04T07:59:12.066 INFO:tasks.workunit.client.0.smithi113.stdout:v0.42 2024-03-04T07:59:12.066 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.42 2024-03-04T07:59:13.925 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2024-03-04T07:59:13.925 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-04T07:59:13.925 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-04T07:59:13.928 INFO:tasks.ceph.mds.b:Started 2024-03-04T07:59:13.928 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T07:59:14.008 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2024-03-04T07:59:14.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.112+0000 7fa0ef200640 1 Processor -- start 2024-03-04T07:59:14.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.112+0000 7fa0ef200640 1 -- start start 2024-03-04T07:59:14.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ef200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 0x7fa0f00c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:14.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ef200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0f0070df0 0x7fa0f015b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:14.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ef200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f00c9c50 con 0x7fa0f015b6c0 2024-03-04T07:59:14.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ef200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f015a9a0 con 0x7fa0f0070df0 2024-03-04T07:59:14.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ef200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f006de30 con 0x7fa0f0157ef0 2024-03-04T07:59:14.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ed400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa0f0070df0 0x7fa0f015b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:14.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ee800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa0f015b6c0 0x7fa0f00cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59720/0 (socket says 172.21.15.113:59720) 2024-03-04T07:59:14.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ee800640 1 -- 172.21.15.113:0/3332036009 learned_addr learned my addr 172.21.15.113:0/3332036009 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T07:59:14.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ede00640 1 --2- 172.21.15.113:0/3332036009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 0x7fa0f00c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:14.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ed400640 1 -- 172.21.15.113:0/3332036009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 msgr2=0x7fa0f00c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T07:59:14.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ed400640 1 --2- 172.21.15.113:0/3332036009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 0x7fa0f00c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ed400640 1 -- 172.21.15.113:0/3332036009 >> v1:172.21.15.113:6789/0 conn(0x7fa0f015b6c0 legacy=0x7fa0f00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:14.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0ed400640 1 -- 172.21.15.113:0/3332036009 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f006c210 con 0x7fa0f0070df0 2024-03-04T07:59:14.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.116+0000 7fa0eca00640 1 -- 172.21.15.113:0/3332036009 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194670042 0 0) 0x7fa0f00c9c50 con 0x7fa0f015b6c0 2024-03-04T07:59:14.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.120+0000 7fa0ed400640 1 --2- 172.21.15.113:0/3332036009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0070df0 0x7fa0f015b180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0097c0 tx=0x7fa0dc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a34aaa3e6ae2a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:14.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.120+0000 7fa0eca00640 1 -- 172.21.15.113:0/3332036009 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc0b1070 con 0x7fa0f0070df0 2024-03-04T07:59:14.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.120+0000 7fa0eca00640 1 -- 172.21.15.113:0/3332036009 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0dc0ac410 con 0x7fa0f0070df0 2024-03-04T07:59:14.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.120+0000 7fa0eca00640 1 -- 172.21.15.113:0/3332036009 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc0ac7e0 con 0x7fa0f0070df0 2024-03-04T07:59:14.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.120+0000 7fa0ef200640 1 -- 172.21.15.113:0/3332036009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0070df0 msgr2=0x7fa0f015b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:14.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.120+0000 7fa0ef200640 1 --2- 172.21.15.113:0/3332036009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0070df0 0x7fa0f015b180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0097c0 tx=0x7fa0dc01c4a0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.120+0000 7fa0ef200640 1 -- 172.21.15.113:0/3332036009 shutdown_connections 2024-03-04T07:59:14.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.120+0000 7fa0ef200640 1 --2- 172.21.15.113:0/3332036009 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f0070df0 0x7fa0f015b180 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.120+0000 7fa0ef200640 1 --2- 172.21.15.113:0/3332036009 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 0x7fa0f00c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.120+0000 7fa0ef200640 1 -- 172.21.15.113:0/3332036009 >> 172.21.15.113:0/3332036009 conn(0x7fa0f0072f40 msgr2=0x7fa0f00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:59:14.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ef200640 1 -- 172.21.15.113:0/3332036009 shutdown_connections 2024-03-04T07:59:14.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ef200640 1 -- 172.21.15.113:0/3332036009 wait complete. 2024-03-04T07:59:14.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ef200640 1 Processor -- start 2024-03-04T07:59:14.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ef200640 1 -- start start 2024-03-04T07:59:14.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ef200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0f0070df0 0x7fa0f0154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:14.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ef200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 0x7fa0f0151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:14.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ef200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f015b6c0 0x7fa0f0159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:14.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ef200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0158450 con 0x7fa0f015b6c0 2024-03-04T07:59:14.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ef200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f01582d0 con 0x7fa0f0070df0 2024-03-04T07:59:14.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ef200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00ce6f0 con 0x7fa0f0157ef0 2024-03-04T07:59:14.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ee800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f015b6c0 0x7fa0f0159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:14.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ee800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f015b6c0 0x7fa0f0159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60528/0 (socket says 172.21.15.113:60528) 2024-03-04T07:59:14.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ee800640 1 -- 172.21.15.113:0/1791514810 learned_addr learned my addr 172.21.15.113:0/1791514810 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T07:59:14.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.124+0000 7fa0ed400640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 0x7fa0f0151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:14.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ede00640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0f0070df0 0x7fa0f0154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ee800640 1 -- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 msgr2=0x7fa0f0151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ee800640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 0x7fa0f0151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ee800640 1 -- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0f0070df0 msgr2=0x7fa0f0154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ee800640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0f0070df0 0x7fa0f0154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ee800640 1 -- 172.21.15.113:0/1791514810 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00caff0 con 0x7fa0f015b6c0 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ed400640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 0x7fa0f0151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ee800640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f015b6c0 0x7fa0f0159540 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800c700 tx=0x7fa0e800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0e6000640 1 -- 172.21.15.113:0/1791514810 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8018070 con 0x7fa0f015b6c0 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0e6000640 1 -- 172.21.15.113:0/1791514810 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e800ce10 con 0x7fa0f015b6c0 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0e6000640 1 -- 172.21.15.113:0/1791514810 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e800a860 con 0x7fa0f015b6c0 2024-03-04T07:59:14.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ef200640 1 -- 172.21.15.113:0/1791514810 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f0153950 con 0x7fa0f015b6c0 2024-03-04T07:59:14.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ef200640 1 -- 172.21.15.113:0/1791514810 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f01510e0 con 0x7fa0f015b6c0 2024-03-04T07:59:14.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0ef200640 1 -- 172.21.15.113:0/1791514810 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f0075a80 con 0x7fa0f015b6c0 2024-03-04T07:59:14.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.128+0000 7fa0e6000640 1 -- 172.21.15.113:0/1791514810 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0e800aa00 con 0x7fa0f015b6c0 2024-03-04T07:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.132+0000 7fa0e6000640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0d0066f60 0x7fa0d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T07:59:14.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.132+0000 7fa0e6000640 1 -- 172.21.15.113:0/1791514810 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7fa0e8084ac0 con 0x7fa0f015b6c0 2024-03-04T07:59:14.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.132+0000 7fa0ede00640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0d0066f60 0x7fa0d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T07:59:14.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.132+0000 7fa0ede00640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0d0066f60 0x7fa0d0069420 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009b60 tx=0x7fa0e0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T07:59:14.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.140+0000 7fa0e6000640 1 -- 172.21.15.113:0/1791514810 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa0e8054f10 con 0x7fa0f015b6c0 2024-03-04T07:59:14.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.416+0000 7fa0ef200640 1 -- 172.21.15.113:0/1791514810 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0f006cfb0 con 0x7fa0f015b6c0 2024-03-04T07:59:14.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.416+0000 7fa0e6000640 1 -- 172.21.15.113:0/1791514810 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+6615 (secure 0 0 0) 0x7fa0e8058fb0 con 0x7fa0f015b6c0 2024-03-04T07:59:14.417 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T07:59:14.417 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:59:07.046470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372,6240]},"id":1}]} 2024-03-04T07:59:14.418 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 94 2024-03-04T07:59:14.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 -- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0d0066f60 msgr2=0x7fa0d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:14.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0d0066f60 0x7fa0d0069420 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009b60 tx=0x7fa0e0009190 comp rx=0 tx=0).stop 2024-03-04T07:59:14.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 -- 172.21.15.113:0/1791514810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f015b6c0 msgr2=0x7fa0f0159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T07:59:14.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f015b6c0 0x7fa0f0159540 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800c700 tx=0x7fa0e800cbc0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 -- 172.21.15.113:0/1791514810 shutdown_connections 2024-03-04T07:59:14.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0d0066f60 0x7fa0d0069420 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa0f015b6c0 0x7fa0f0159540 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa0f0157ef0 0x7fa0f0151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 --2- 172.21.15.113:0/1791514810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa0f0070df0 0x7fa0f0154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T07:59:14.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 -- 172.21.15.113:0/1791514810 >> 172.21.15.113:0/1791514810 conn(0x7fa0f0072f40 msgr2=0x7fa0f014f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T07:59:14.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 -- 172.21.15.113:0/1791514810 shutdown_connections 2024-03-04T07:59:14.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T07:59:14.420+0000 7fa0ef200640 1 -- 172.21.15.113:0/1791514810 wait complete. 2024-03-04T07:59:14.441 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2024-03-04T07:59:14.442 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 74 secs before thrashing 2024-03-04T07:59:28.986 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 42% (105/250) Updating files: 43% (108/250) Updating files: 44% (110/250) Updating files: 45% (113/250) Updating files: 46% (115/250) Updating files: 47% (118/250) Updating files: 48% (120/250) Updating files: 49% (123/250) Updating files: 50% (125/250) Updating files: 51% (128/250) Updating files: 52% (130/250) Updating files: 53% (133/250) Updating files: 54% (135/250) Updating files: 55% (138/250) Updating files: 56% (140/250) Updating files: 57% (143/250) Updating files: 58% (145/250) Updating files: 59% (148/250) Updating files: 60% (150/250) Updating files: 61% (153/250) Updating files: 62% (155/250) Updating files: 63% (158/250) Updating files: 64% (160/250) Updating files: 65% (163/250) Updating files: 66% (165/250) Updating files: 67% (168/250) Updating files: 68% (170/250) Updating files: 69% (173/250) Updating files: 70% (175/250) Updating files: 71% (178/250) Updating files: 72% (180/250) Updating files: 73% (183/250) Updating files: 74% (185/250) Updating files: 75% (188/250) Updating files: 76% (190/250) Updating files: 77% (193/250) Updating files: 78% (195/250) Updating files: 79% (198/250) Updating files: 80% (200/250) Updating files: 81% (203/250) Updating files: 82% (205/250) Updating files: 83% (208/250) Updating files: 84% (210/250) Updating files: 85% (213/250) Updating files: 86% (215/250) Updating files: 87% (218/250) Updating files: 88% (220/250) Updating files: 89% (223/250) Updating files: 90% (225/250) Updating files: 91% (228/250) Updating files: 92% (230/250) Updating files: 93% (233/250) Updating files: 94% (235/250) Updating files: 95% (238/250) Updating files: 96% (240/250) Updating files: 97% (243/250) Updating files: 98% (245/250) Updating files: 99% (248/250) Updating files: 100% (250/250) Updating files: 100% (250/250), done. 2024-03-04T07:59:29.044 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at b205c64c687 v0.42 2024-03-04T07:59:29.050 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.42 2024-03-04T07:59:29.348 INFO:tasks.workunit.client.0.smithi113.stdout:v0.43 2024-03-04T07:59:29.348 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 43 -eq 48 ] 2024-03-04T07:59:29.351 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.43 2024-03-04T07:59:29.351 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.43 2024-03-04T07:59:29.351 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.43 2024-03-04T07:59:31.505 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 57% (64/111) Updating files: 58% (65/111) Updating files: 59% (66/111) Updating files: 60% (67/111) Updating files: 61% (68/111) Updating files: 62% (69/111) Updating files: 63% (70/111) Updating files: 64% (72/111) Updating files: 65% (73/111) Updating files: 66% (74/111) Updating files: 67% (75/111) Updating files: 68% (76/111) Updating files: 69% (77/111) Updating files: 70% (78/111) Updating files: 71% (79/111) Updating files: 72% (80/111) Updating files: 73% (82/111) Updating files: 74% (83/111) Updating files: 75% (84/111) Updating files: 76% (85/111) Updating files: 77% (86/111) Updating files: 78% (87/111) Updating files: 79% (88/111) Updating files: 80% (89/111) Updating files: 81% (90/111) Updating files: 82% (92/111) Updating files: 83% (93/111) Updating files: 84% (94/111) Updating files: 85% (95/111) Updating files: 86% (96/111) Updating files: 87% (97/111) Updating files: 88% (98/111) Updating files: 89% (99/111) Updating files: 90% (100/111) Updating files: 91% (102/111) Updating files: 92% (103/111) Updating files: 93% (104/111) Updating files: 94% (105/111) Updating files: 95% (106/111) Updating files: 96% (107/111) Updating files: 97% (108/111) Updating files: 98% (109/111) Updating files: 99% (110/111) Updating files: 100% (111/111) Updating files: 100% (111/111), done. 2024-03-04T07:59:31.780 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 9fa8781c014 v0.43 2024-03-04T07:59:31.782 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.43 2024-03-04T07:59:31.909 INFO:tasks.workunit.client.0.smithi113.stdout:v0.44 2024-03-04T07:59:31.909 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 44 -eq 48 ] 2024-03-04T07:59:31.909 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.44 2024-03-04T07:59:31.909 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.44 2024-03-04T07:59:31.910 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.44 2024-03-04T07:59:32.195 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:59:32.199 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T07:59:35.123 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 71% (133/186) Updating files: 72% (134/186) Updating files: 73% (136/186) Updating files: 74% (138/186) Updating files: 75% (140/186) Updating files: 76% (142/186) Updating files: 77% (144/186) Updating files: 78% (146/186) Updating files: 79% (147/186) Updating files: 80% (149/186) Updating files: 81% (151/186) Updating files: 82% (153/186) Updating files: 83% (155/186) Updating files: 84% (157/186) Updating files: 85% (159/186) Updating files: 86% (160/186) Updating files: 87% (162/186) Updating files: 88% (164/186) Updating files: 89% (166/186) Updating files: 90% (168/186) Updating files: 91% (170/186) Updating files: 92% (172/186) Updating files: 93% (173/186) Updating files: 94% (175/186) Updating files: 95% (177/186) Updating files: 96% (179/186) Updating files: 97% (181/186) Updating files: 98% (183/186) Updating files: 99% (185/186) Updating files: 100% (186/186) Updating files: 100% (186/186), done. 2024-03-04T07:59:35.164 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at cdd5298d276 v0.44 2024-03-04T07:59:35.167 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.44 2024-03-04T07:59:35.259 INFO:tasks.workunit.client.0.smithi113.stdout:v0.45 2024-03-04T07:59:35.260 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 45 -eq 48 ] 2024-03-04T07:59:35.260 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.45 2024-03-04T07:59:35.260 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.45 2024-03-04T07:59:35.260 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.45 2024-03-04T07:59:37.087 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 0aea1cb1df5 v0.45 2024-03-04T07:59:37.091 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.45 2024-03-04T07:59:37.246 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 46 -eq 48 ] 2024-03-04T07:59:37.246 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.46 2024-03-04T07:59:37.246 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.46 2024-03-04T07:59:37.247 INFO:tasks.workunit.client.0.smithi113.stdout:v0.46 2024-03-04T07:59:37.247 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.46 2024-03-04T07:59:56.451 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 28% (88/309) Updating files: 29% (90/309) Updating files: 30% (93/309) Updating files: 31% (96/309) Updating files: 32% (99/309) Updating files: 33% (102/309) Updating files: 34% (106/309) Updating files: 35% (109/309) Updating files: 36% (112/309) Updating files: 37% (115/309) Updating files: 38% (118/309) Updating files: 39% (121/309) Updating files: 40% (124/309) Updating files: 41% (127/309) Updating files: 42% (130/309) Updating files: 43% (133/309) Updating files: 44% (136/309) Updating files: 45% (140/309) Updating files: 46% (143/309) Updating files: 47% (146/309) Updating files: 48% (149/309) Updating files: 49% (152/309) Updating files: 50% (155/309) Updating files: 51% (158/309) Updating files: 52% (161/309) Updating files: 53% (164/309) Updating files: 54% (167/309) Updating files: 55% (170/309) Updating files: 56% (174/309) Updating files: 57% (177/309) Updating files: 58% (180/309) Updating files: 59% (183/309) Updating files: 60% (186/309) Updating files: 61% (189/309) Updating files: 62% (192/309) Updating files: 62% (193/309) Updating files: 63% (195/309) Updating files: 64% (198/309) Updating files: 65% (201/309) Updating files: 66% (204/309) Updating files: 67% (208/309) Updating files: 68% (211/309) Updating files: 69% (214/309) Updating files: 70% (217/309) Updating files: 71% (220/309) Updating files: 72% (223/309) Updating files: 73% (226/309) Updating files: 74% (229/309) Updating files: 75% (232/309) Updating files: 76% (235/309) Updating files: 76% (237/309) Updating files: 77% (238/309) Updating files: 78% (242/309) Updating files: 79% (245/309) Updating files: 80% (248/309) Updating files: 81% (251/309) Updating files: 82% (254/309) Updating files: 83% (257/309) Updating files: 84% (260/309) Updating files: 85% (263/309) Updating files: 86% (266/309) Updating files: 87% (269/309) Updating files: 88% (272/309) Updating files: 89% (276/309) Updating files: 90% (279/309) Updating files: 91% (282/309) Updating files: 92% (285/309) Updating files: 93% (288/309) Updating files: 94% (291/309) Updating files: 95% (294/309) Updating files: 96% (297/309) Updating files: 97% (300/309) Updating files: 98% (303/309) Updating files: 99% (306/309) Updating files: 100% (309/309) Updating files: 100% (309/309), done. 2024-03-04T07:59:56.498 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at cb7f1c9c752 v0.46 2024-03-04T07:59:56.502 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.46 2024-03-04T07:59:56.981 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 47 -eq 48 ] 2024-03-04T07:59:56.982 INFO:tasks.workunit.client.0.smithi113.stdout:v0.47 2024-03-04T07:59:56.982 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.47 2024-03-04T07:59:56.982 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.47 2024-03-04T07:59:56.982 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.47 2024-03-04T08:00:03.459 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:00:03.464 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:00:07.455 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 37% (82/219) Updating files: 38% (84/219) Updating files: 39% (86/219) Updating files: 40% (88/219) Updating files: 41% (90/219) Updating files: 42% (92/219) Updating files: 43% (95/219) Updating files: 44% (97/219) Updating files: 45% (99/219) Updating files: 46% (101/219) Updating files: 47% (103/219) Updating files: 48% (106/219) Updating files: 49% (108/219) Updating files: 50% (110/219) Updating files: 51% (112/219) Updating files: 52% (114/219) Updating files: 53% (117/219) Updating files: 54% (119/219) Updating files: 55% (121/219) Updating files: 56% (123/219) Updating files: 57% (125/219) Updating files: 58% (128/219) Updating files: 59% (130/219) Updating files: 60% (132/219) Updating files: 61% (134/219) Updating files: 62% (136/219) Updating files: 63% (138/219) Updating files: 64% (141/219) Updating files: 65% (143/219) Updating files: 66% (145/219) Updating files: 67% (147/219) Updating files: 68% (149/219) Updating files: 69% (152/219) Updating files: 70% (154/219) Updating files: 71% (156/219) Updating files: 72% (158/219) Updating files: 73% (160/219) Updating files: 74% (163/219) Updating files: 75% (165/219) Updating files: 76% (167/219) Updating files: 77% (169/219) Updating files: 78% (171/219) Updating files: 79% (174/219) Updating files: 79% (175/219) Updating files: 80% (176/219) Updating files: 81% (178/219) Updating files: 82% (180/219) Updating files: 83% (182/219) Updating files: 84% (184/219) Updating files: 85% (187/219) Updating files: 86% (189/219) Updating files: 87% (191/219) Updating files: 88% (193/219) Updating files: 89% (195/219) Updating files: 90% (198/219) Updating files: 91% (200/219) Updating files: 92% (202/219) Updating files: 93% (204/219) Updating files: 94% (206/219) Updating files: 95% (209/219) Updating files: 96% (211/219) Updating files: 97% (213/219) Updating files: 98% (215/219) Updating files: 99% (217/219) Updating files: 100% (219/219) Updating files: 100% (219/219), done. 2024-03-04T08:00:07.638 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at ad663d58c89 v0.47 2024-03-04T08:00:07.645 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.47 2024-03-04T08:00:07.709 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 48 -eq 48 ] 2024-03-04T08:00:07.710 INFO:tasks.workunit.client.0.smithi113.stderr:+ continue 2024-03-04T08:00:07.710 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 49 -eq 48 ] 2024-03-04T08:00:07.710 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.49 2024-03-04T08:00:07.710 INFO:tasks.workunit.client.0.smithi113.stdout:v0.49 2024-03-04T08:00:07.710 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.49 2024-03-04T08:00:07.711 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.49 2024-03-04T08:00:14.386 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 28% (138/481) Updating files: 29% (140/481) Updating files: 30% (145/481) Updating files: 31% (150/481) Updating files: 32% (154/481) Updating files: 33% (159/481) Updating files: 34% (164/481) Updating files: 34% (168/481) Updating files: 35% (169/481) Updating files: 36% (174/481) Updating files: 37% (178/481) Updating files: 38% (183/481) Updating files: 39% (188/481) Updating files: 40% (193/481) Updating files: 41% (198/481) Updating files: 42% (203/481) Updating files: 42% (205/481) Updating files: 43% (207/481) Updating files: 44% (212/481) Updating files: 45% (217/481) Updating files: 46% (222/481) Updating files: 47% (227/481) Updating files: 48% (231/481) Updating files: 49% (236/481) Updating files: 50% (241/481) Updating files: 51% (246/481) Updating files: 52% (251/481) Updating files: 53% (255/481) Updating files: 54% (260/481) Updating files: 55% (265/481) Updating files: 56% (270/481) Updating files: 57% (275/481) Updating files: 58% (279/481) Updating files: 59% (284/481) Updating files: 60% (289/481) Updating files: 61% (294/481) Updating files: 62% (299/481) Updating files: 63% (304/481) Updating files: 64% (308/481) Updating files: 65% (313/481) Updating files: 66% (318/481) Updating files: 67% (323/481) Updating files: 68% (328/481) Updating files: 69% (332/481) Updating files: 70% (337/481) Updating files: 71% (342/481) Updating files: 72% (347/481) Updating files: 73% (352/481) Updating files: 74% (356/481) Updating files: 75% (361/481) Updating files: 76% (366/481) Updating files: 77% (371/481) Updating files: 78% (376/481) Updating files: 79% (380/481) Updating files: 80% (385/481) Updating files: 81% (390/481) Updating files: 82% (395/481) Updating files: 82% (397/481) Updating files: 83% (400/481) Updating files: 84% (405/481) Updating files: 85% (409/481) Updating files: 86% (414/481) Updating files: 86% (416/481) Updating files: 87% (419/481) Updating files: 88% (424/481) Updating files: 89% (429/481) Updating files: 90% (433/481) Updating files: 91% (438/481) Updating files: 92% (443/481) Updating files: 93% (448/481) Updating files: 94% (453/481) Updating files: 95% (457/481) Updating files: 96% (462/481) Updating files: 97% (467/481) Updating files: 98% (472/481) Updating files: 99% (477/481) Updating files: 100% (481/481) Updating files: 100% (481/481), done. 2024-03-04T08:00:14.420 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at ca6265d0f4d v0.49 2024-03-04T08:00:14.426 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.49 2024-03-04T08:00:14.723 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 50 -eq 48 ] 2024-03-04T08:00:14.723 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.50 2024-03-04T08:00:14.723 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.50 2024-03-04T08:00:14.723 INFO:tasks.workunit.client.0.smithi113.stdout:v0.50 2024-03-04T08:00:14.724 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.50 2024-03-04T08:00:17.295 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 83% (114/137) Updating files: 84% (116/137) Updating files: 85% (117/137) Updating files: 86% (118/137) Updating files: 87% (120/137) Updating files: 88% (121/137) Updating files: 89% (122/137) Updating files: 90% (124/137) Updating files: 91% (125/137) Updating files: 92% (127/137) Updating files: 93% (128/137) Updating files: 94% (129/137) Updating files: 95% (131/137) Updating files: 96% (132/137) Updating files: 97% (133/137) Updating files: 98% (135/137) Updating files: 99% (136/137) Updating files: 100% (137/137) Updating files: 100% (137/137), done. 2024-03-04T08:00:17.352 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at a4428bd0e19 v0.50 2024-03-04T08:00:17.355 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.50 2024-03-04T08:00:17.642 INFO:tasks.workunit.client.0.smithi113.stdout:v0.51 2024-03-04T08:00:17.642 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 51 -eq 48 ] 2024-03-04T08:00:17.642 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.51 2024-03-04T08:00:17.642 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.51 2024-03-04T08:00:17.642 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.51 2024-03-04T08:00:28.442 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.598+0000 7fda6b200640 1 Processor -- start 2024-03-04T08:00:28.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.598+0000 7fda6b200640 1 -- start start 2024-03-04T08:00:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.598+0000 7fda6b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fda6c0730d0 0x7fda6c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.598+0000 7fda6b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c06f1b0 con 0x7fda6c152a90 2024-03-04T08:00:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.598+0000 7fda6b200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6c071580 con 0x7fda6c06d420 2024-03-04T08:00:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.598+0000 7fda6b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6c152e70 con 0x7fda6c0730d0 2024-03-04T08:00:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.602+0000 7fda69e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fda6c152a90 0x7fda6c06cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39498/0 (socket says 172.21.15.113:39498) 2024-03-04T08:00:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.602+0000 7fda69e00640 1 -- 172.21.15.113:0/1194880586 learned_addr learned my addr 172.21.15.113:0/1194880586 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:00:28.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.602+0000 7fda6a800640 1 --2- 172.21.15.113:0/1194880586 >> v2:172.21.15.113:3300/0 conn(0x7fda6c0730d0 0x7fda6c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:28.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.602+0000 7fda68a00640 1 -- 172.21.15.113:0/1194880586 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207864225 0 0) 0x7fda6c06f1b0 con 0x7fda6c152a90 2024-03-04T08:00:28.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.602+0000 7fda68a00640 1 -- 172.21.15.113:0/1194880586 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda50003530 con 0x7fda6c152a90 2024-03-04T08:00:28.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.602+0000 7fda68a00640 1 -- 172.21.15.113:0/1194880586 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585217314 0 0) 0x7fda6c071580 con 0x7fda6c06d420 2024-03-04T08:00:28.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.602+0000 7fda6a800640 1 -- 172.21.15.113:0/1194880586 >> v1:172.21.15.136:6789/0 conn(0x7fda6c06d420 legacy=0x7fda6c15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.602+0000 7fda6a800640 1 -- 172.21.15.113:0/1194880586 >> v1:172.21.15.113:6789/0 conn(0x7fda6c152a90 legacy=0x7fda6c06cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:28.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.602+0000 7fda6a800640 1 -- 172.21.15.113:0/1194880586 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c071860 con 0x7fda6c0730d0 2024-03-04T08:00:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda6a800640 1 --2- 172.21.15.113:0/1194880586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c0730d0 0x7fda6c15c3f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fda64009d90 tx=0x7fda6401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0673f271335d7e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda68a00640 1 -- 172.21.15.113:0/1194880586 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640b2070 con 0x7fda6c0730d0 2024-03-04T08:00:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda68a00640 1 -- 172.21.15.113:0/1194880586 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda640a5830 con 0x7fda6c0730d0 2024-03-04T08:00:28.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda68a00640 1 -- 172.21.15.113:0/1194880586 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640a5c00 con 0x7fda6c0730d0 2024-03-04T08:00:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda6b200640 1 -- 172.21.15.113:0/1194880586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c0730d0 msgr2=0x7fda6c15c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda6b200640 1 --2- 172.21.15.113:0/1194880586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c0730d0 0x7fda6c15c3f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fda64009d90 tx=0x7fda6401c970 comp rx=0 tx=0).stop 2024-03-04T08:00:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda6b200640 1 -- 172.21.15.113:0/1194880586 shutdown_connections 2024-03-04T08:00:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda6b200640 1 --2- 172.21.15.113:0/1194880586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c0730d0 0x7fda6c15c3f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:28.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda6b200640 1 -- 172.21.15.113:0/1194880586 >> 172.21.15.113:0/1194880586 conn(0x7fda6c06e2a0 msgr2=0x7fda6c06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:28.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda6b200640 1 -- 172.21.15.113:0/1194880586 shutdown_connections 2024-03-04T08:00:28.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.606+0000 7fda6b200640 1 -- 172.21.15.113:0/1194880586 wait complete. 2024-03-04T08:00:28.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6b200640 1 Processor -- start 2024-03-04T08:00:28.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6b200640 1 -- start start 2024-03-04T08:00:28.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda6c06d420 0x7fda6c0c33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:28.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda6c0730d0 0x7fda6c142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:28.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c152a90 0x7fda6c142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:28.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c071330 con 0x7fda6c152a90 2024-03-04T08:00:28.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c1668a0 con 0x7fda6c0730d0 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6c166a20 con 0x7fda6c06d420 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c152a90 0x7fda6c142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c152a90 0x7fda6c142aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46584/0 (socket says 172.21.15.113:46584) 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda69e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda6c06d420 0x7fda6c0c33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6a800640 1 -- 172.21.15.113:0/3512184126 learned_addr learned my addr 172.21.15.113:0/3512184126 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda69400640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda6c0730d0 0x7fda6c142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6a800640 1 -- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda6c06d420 msgr2=0x7fda6c0c33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6a800640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda6c06d420 0x7fda6c0c33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6a800640 1 -- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda6c0730d0 msgr2=0x7fda6c142560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6a800640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda6c0730d0 0x7fda6c142560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:28.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.610+0000 7fda6a800640 1 -- 172.21.15.113:0/3512184126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0c5820 con 0x7fda6c152a90 2024-03-04T08:00:28.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.614+0000 7fda6a800640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c152a90 0x7fda6c142aa0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fda6401ce50 tx=0x7fda64019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:28.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.614+0000 7fda62000640 1 -- 172.21.15.113:0/3512184126 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640bf070 con 0x7fda6c152a90 2024-03-04T08:00:28.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.614+0000 7fda62000640 1 -- 172.21.15.113:0/3512184126 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda640b3040 con 0x7fda6c152a90 2024-03-04T08:00:28.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.614+0000 7fda6b200640 1 -- 172.21.15.113:0/3512184126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda6c148970 con 0x7fda6c152a90 2024-03-04T08:00:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.614+0000 7fda62000640 1 -- 172.21.15.113:0/3512184126 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640b2050 con 0x7fda6c152a90 2024-03-04T08:00:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.614+0000 7fda6b200640 1 -- 172.21.15.113:0/3512184126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda6c148c30 con 0x7fda6c152a90 2024-03-04T08:00:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.614+0000 7fda6b200640 1 -- 172.21.15.113:0/3512184126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6c148e30 con 0x7fda6c152a90 2024-03-04T08:00:28.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.614+0000 7fda62000640 1 -- 172.21.15.113:0/3512184126 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda640c2070 con 0x7fda6c152a90 2024-03-04T08:00:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.618+0000 7fda62000640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda4c066f60 0x7fda4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.618+0000 7fda62000640 1 -- 172.21.15.113:0/3512184126 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7fda6411f6d0 con 0x7fda6c152a90 2024-03-04T08:00:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.618+0000 7fda69e00640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda4c066f60 0x7fda4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:28.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.618+0000 7fda69e00640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda4c066f60 0x7fda4c069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007fd0 tx=0x7fda5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.622+0000 7fda62000640 1 -- 172.21.15.113:0/3512184126 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fda6c148e30 con 0x7fda6c152a90 2024-03-04T08:00:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.882+0000 7fda6b200640 1 -- 172.21.15.113:0/3512184126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda6c075a80 con 0x7fda6c152a90 2024-03-04T08:00:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.882+0000 7fda62000640 1 -- 172.21.15.113:0/3512184126 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+6615 (secure 0 0 0) 0x7fda64016560 con 0x7fda6c152a90 2024-03-04T08:00:28.881 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:28.882 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:59:07.046470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372,6240]},"id":1}]} 2024-03-04T08:00:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 94 2024-03-04T08:00:28.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 -- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda4c066f60 msgr2=0x7fda4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:28.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda4c066f60 0x7fda4c069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007fd0 tx=0x7fda5c009040 comp rx=0 tx=0).stop 2024-03-04T08:00:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 -- 172.21.15.113:0/3512184126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c152a90 msgr2=0x7fda6c142aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c152a90 0x7fda6c142aa0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fda6401ce50 tx=0x7fda64019710 comp rx=0 tx=0).stop 2024-03-04T08:00:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 -- 172.21.15.113:0/3512184126 shutdown_connections 2024-03-04T08:00:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda4c066f60 0x7fda4c069420 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda6c152a90 0x7fda6c142aa0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda6c0730d0 0x7fda6c142560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 --2- 172.21.15.113:0/3512184126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda6c06d420 0x7fda6c0c33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 -- 172.21.15.113:0/3512184126 >> 172.21.15.113:0/3512184126 conn(0x7fda6c06e2a0 msgr2=0x7fda6c15b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 -- 172.21.15.113:0/3512184126 shutdown_connections 2024-03-04T08:00:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:28.886+0000 7fda6b200640 1 -- 172.21.15.113:0/3512184126 wait complete. 2024-03-04T08:00:28.913 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=2) 2024-03-04T08:00:28.913 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-04T08:00:28.913 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:00:28.931 INFO:tasks.ceph.mds.a:Stopped 2024-03-04T08:00:28.931 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:29.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff344200640 1 Processor -- start 2024-03-04T08:00:29.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff344200640 1 -- start start 2024-03-04T08:00:29.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff344200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c1582e0 0x7ff33c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:29.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff344200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff33c0b0ed0 0x7ff33c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:29.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff344200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c1586c0 con 0x7ff33c06ddf0 2024-03-04T08:00:29.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff344200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c06c210 con 0x7ff33c0b0ed0 2024-03-04T08:00:29.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff344200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c06c4f0 con 0x7ff33c1582e0 2024-03-04T08:00:29.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff341800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c1582e0 0x7ff33c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:29.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff340e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff33c0b0ed0 0x7ff33c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:29.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff341800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c1582e0 0x7ff33c0701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:54014/0 (socket says 172.21.15.113:54014) 2024-03-04T08:00:29.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff341800640 1 -- 172.21.15.113:0/2724793988 learned_addr learned my addr 172.21.15.113:0/2724793988 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:29.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff340e00640 1 -- 172.21.15.113:0/2724793988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c1582e0 msgr2=0x7ff33c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:29.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff340e00640 1 --2- 172.21.15.113:0/2724793988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c1582e0 0x7ff33c0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff340e00640 1 -- 172.21.15.113:0/2724793988 >> v1:172.21.15.136:6789/0 conn(0x7ff33c06ddf0 legacy=0x7ff33c15af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:29.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.122+0000 7ff340e00640 1 -- 172.21.15.113:0/2724793988 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c06f6a0 con 0x7ff33c0b0ed0 2024-03-04T08:00:29.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff340e00640 1 --2- 172.21.15.113:0/2724793988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c0b0ed0 0x7ff33c0ce2d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff3280097c0 tx=0x7ff32801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4d79b0b5aad1ea3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:29.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff33be00640 1 -- 172.21.15.113:0/2724793988 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3280b0070 con 0x7ff33c0b0ed0 2024-03-04T08:00:29.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff33be00640 1 -- 172.21.15.113:0/2724793988 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3280ab600 con 0x7ff33c0b0ed0 2024-03-04T08:00:29.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff33be00640 1 -- 172.21.15.113:0/2724793988 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3280ab9d0 con 0x7ff33c0b0ed0 2024-03-04T08:00:29.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff344200640 1 -- 172.21.15.113:0/2724793988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c0b0ed0 msgr2=0x7ff33c0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:29.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff344200640 1 --2- 172.21.15.113:0/2724793988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c0b0ed0 0x7ff33c0ce2d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff3280097c0 tx=0x7ff32801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff344200640 1 -- 172.21.15.113:0/2724793988 shutdown_connections 2024-03-04T08:00:29.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff344200640 1 --2- 172.21.15.113:0/2724793988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c0b0ed0 0x7ff33c0ce2d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff344200640 1 --2- 172.21.15.113:0/2724793988 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c1582e0 0x7ff33c0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff344200640 1 -- 172.21.15.113:0/2724793988 >> 172.21.15.113:0/2724793988 conn(0x7ff33c066db0 msgr2=0x7ff33c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:29.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff344200640 1 -- 172.21.15.113:0/2724793988 shutdown_connections 2024-03-04T08:00:29.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.126+0000 7ff344200640 1 -- 172.21.15.113:0/2724793988 wait complete. 2024-03-04T08:00:29.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.130+0000 7ff344200640 1 Processor -- start 2024-03-04T08:00:29.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.130+0000 7ff344200640 1 -- start start 2024-03-04T08:00:29.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff344200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c06ddf0 0x7ff33c1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff344200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c0b0ed0 0x7ff33c1f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff344200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33c1582e0 0x7ff33c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff344200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c167ed0 con 0x7ff33c1582e0 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff344200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c167d50 con 0x7ff33c06ddf0 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff344200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c168050 con 0x7ff33c0b0ed0 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff342200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33c1582e0 0x7ff33c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff342200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33c1582e0 0x7ff33c159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46600/0 (socket says 172.21.15.113:46600) 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff342200640 1 -- 172.21.15.113:0/1663605564 learned_addr learned my addr 172.21.15.113:0/1663605564 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff340e00640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c0b0ed0 0x7ff33c1f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff341800640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c06ddf0 0x7ff33c1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff342200640 1 -- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c0b0ed0 msgr2=0x7ff33c1f6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:29.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff342200640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c0b0ed0 0x7ff33c1f6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff342200640 1 -- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c06ddf0 msgr2=0x7ff33c1f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:29.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff342200640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c06ddf0 0x7ff33c1f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff342200640 1 -- 172.21.15.113:0/1663605564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c1ecee0 con 0x7ff33c1582e0 2024-03-04T08:00:29.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff342200640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33c1582e0 0x7ff33c159ee0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c6f0 tx=0x7ff33400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:29.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff339600640 1 -- 172.21.15.113:0/1663605564 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334003980 con 0x7ff33c1582e0 2024-03-04T08:00:29.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff339600640 1 -- 172.21.15.113:0/1663605564 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff334003b20 con 0x7ff33c1582e0 2024-03-04T08:00:29.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff344200640 1 -- 172.21.15.113:0/1663605564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c1f72d0 con 0x7ff33c1582e0 2024-03-04T08:00:29.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff339600640 1 -- 172.21.15.113:0/1663605564 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400a850 con 0x7ff33c1582e0 2024-03-04T08:00:29.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff344200640 1 -- 172.21.15.113:0/1663605564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c1f7890 con 0x7ff33c1582e0 2024-03-04T08:00:29.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff344200640 1 -- 172.21.15.113:0/1663605564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c068eb0 con 0x7ff33c1582e0 2024-03-04T08:00:29.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.134+0000 7ff339600640 1 -- 172.21.15.113:0/1663605564 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff33400a9f0 con 0x7ff33c1582e0 2024-03-04T08:00:29.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.138+0000 7ff339600640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 0x7ff31c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:29.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.138+0000 7ff339600640 1 -- 172.21.15.113:0/1663605564 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7ff334085290 con 0x7ff33c1582e0 2024-03-04T08:00:29.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.138+0000 7ff341800640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 0x7ff31c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:29.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.138+0000 7ff341800640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 0x7ff31c069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff330009ab0 tx=0x7ff330009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:29.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.142+0000 7ff339600640 1 -- 172.21.15.113:0/1663605564 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff3340556e0 con 0x7ff33c1582e0 2024-03-04T08:00:29.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.334+0000 7ff344200640 1 -- 172.21.15.113:0/1663605564 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff33c06ffa0 con 0x7ff33c1582e0 2024-03-04T08:00:29.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.334+0000 7ff339600640 1 -- 172.21.15.113:0/1663605564 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+6615 (secure 0 0 0) 0x7ff334059780 con 0x7ff33c1582e0 2024-03-04T08:00:29.335 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:29.335 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:59:07.046470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372,6240]},"id":1}]} 2024-03-04T08:00:29.335 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 94 2024-03-04T08:00:29.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 -- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 msgr2=0x7ff31c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:29.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 0x7ff31c069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff330009ab0 tx=0x7ff330009240 comp rx=0 tx=0).stop 2024-03-04T08:00:29.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 -- 172.21.15.113:0/1663605564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33c1582e0 msgr2=0x7ff33c159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:29.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33c1582e0 0x7ff33c159ee0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff33400c6f0 tx=0x7ff33400cbb0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 -- 172.21.15.113:0/1663605564 shutdown_connections 2024-03-04T08:00:29.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 0x7ff31c069420 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff33c1582e0 0x7ff33c159ee0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff33c0b0ed0 0x7ff33c1f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 --2- 172.21.15.113:0/1663605564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c06ddf0 0x7ff33c1f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:29.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 -- 172.21.15.113:0/1663605564 >> 172.21.15.113:0/1663605564 conn(0x7ff33c066db0 msgr2=0x7ff33c156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:29.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 -- 172.21.15.113:0/1663605564 shutdown_connections 2024-03-04T08:00:29.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:29.338+0000 7ff344200640 1 -- 172.21.15.113:0/1663605564 wait complete. 2024-03-04T08:00:29.356 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:00:31.217 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 1% (2/105) Updating files: 2% (3/105) Updating files: 3% (4/105) Updating files: 4% (5/105) Updating files: 5% (6/105) Updating files: 6% (7/105) Updating files: 7% (8/105) Updating files: 8% (9/105) Updating files: 9% (10/105) Updating files: 10% (11/105) Updating files: 11% (12/105) Updating files: 12% (13/105) Updating files: 13% (14/105) Updating files: 14% (15/105) Updating files: 15% (16/105) Updating files: 16% (17/105) Updating files: 17% (18/105) Updating files: 18% (19/105) Updating files: 19% (20/105) Updating files: 20% (21/105) Updating files: 21% (23/105) Updating files: 22% (24/105) Updating files: 23% (25/105) Updating files: 24% (26/105) Updating files: 25% (27/105) Updating files: 26% (28/105) Updating files: 27% (29/105) Updating files: 28% (30/105) Updating files: 29% (31/105) Updating files: 30% (32/105) Updating files: 31% (33/105) Updating files: 32% (34/105) Updating files: 33% (35/105) Updating files: 34% (36/105) Updating files: 35% (37/105) Updating files: 36% (38/105) Updating files: 37% (39/105) Updating files: 38% (40/105) Updating files: 39% (41/105) Updating files: 40% (42/105) Updating files: 41% (44/105) Updating files: 42% (45/105) Updating files: 43% (46/105) Updating files: 44% (47/105) Updating files: 45% (48/105) Updating files: 46% (49/105) Updating files: 47% (50/105) Updating files: 48% (51/105) Updating files: 49% (52/105) Updating files: 50% (53/105) Updating files: 51% (54/105) Updating files: 52% (55/105) Updating files: 53% (56/105) Updating files: 54% (57/105) Updating files: 55% (58/105) Updating files: 56% (59/105) Updating files: 57% (60/105) Updating files: 58% (61/105) Updating files: 59% (62/105) Updating files: 60% (63/105) Updating files: 61% (65/105) Updating files: 62% (66/105) Updating files: 63% (67/105) Updating files: 64% (68/105) Updating files: 65% (69/105) Updating files: 66% (70/105) Updating files: 67% (71/105) Updating files: 68% (72/105) Updating files: 69% (73/105) Updating files: 70% (74/105) Updating files: 71% (75/105) Updating files: 72% (76/105) Updating files: 73% (77/105) Updating files: 74% (78/105) Updating files: 75% (79/105) Updating files: 76% (80/105) Updating files: 77% (81/105) Updating files: 78% (82/105) Updating files: 79% (83/105) Updating files: 80% (84/105) Updating files: 81% (86/105) Updating files: 82% (87/105) Updating files: 83% (88/105) Updating files: 84% (89/105) Updating files: 85% (90/105) Updating files: 86% (91/105) Updating files: 87% (92/105) Updating files: 88% (93/105) Updating files: 89% (94/105) Updating files: 90% (95/105) Updating files: 91% (96/105) Updating files: 92% (97/105) Updating files: 93% (98/105) Updating files: 94% (99/105) Updating files: 95% (100/105) Updating files: 96% (101/105) Updating files: 97% (102/105) Updating files: 98% (103/105) Updating files: 99% (104/105) Updating files: 100% (105/105) Updating files: 100% (105/105), done. 2024-03-04T08:00:31.357 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:31.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.498+0000 7f0b6a400640 1 Processor -- start 2024-03-04T08:00:31.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.498+0000 7f0b6a400640 1 -- start start 2024-03-04T08:00:31.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b6a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b64152a50 0x7f0b640ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:31.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b6a400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b64152e30 con 0x7f0b6415c4e0 2024-03-04T08:00:31.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b6a400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6406f280 con 0x7f0b6406ff20 2024-03-04T08:00:31.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b6a400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b640c31b0 con 0x7f0b64152a50 2024-03-04T08:00:31.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b63400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b64152a50 0x7f0b640ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:31.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b63e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0b6415c4e0 0x7f0b64163060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39514/0 (socket says 172.21.15.113:39514) 2024-03-04T08:00:31.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b63e00640 1 -- 172.21.15.113:0/2991591114 learned_addr learned my addr 172.21.15.113:0/2991591114 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:00:31.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b63400640 1 -- 172.21.15.113:0/2991591114 >> v1:172.21.15.136:6789/0 conn(0x7f0b6406ff20 legacy=0x7f0b6415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:31.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b63400640 1 -- 172.21.15.113:0/2991591114 >> v1:172.21.15.113:6789/0 conn(0x7f0b6415c4e0 legacy=0x7f0b64163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:31.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b63400640 1 -- 172.21.15.113:0/2991591114 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6406f560 con 0x7f0b64152a50 2024-03-04T08:00:31.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.502+0000 7f0b62a00640 1 -- 172.21.15.113:0/2991591114 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956696830 0 0) 0x7f0b64152e30 con 0x7f0b6415c4e0 2024-03-04T08:00:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.506+0000 7f0b63400640 1 --2- 172.21.15.113:0/2991591114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 0x7f0b640ce120 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0b540097c0 tx=0x7f0b5401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d084430df7bcf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.506+0000 7f0b62a00640 1 -- 172.21.15.113:0/2991591114 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b540b1070 con 0x7f0b64152a50 2024-03-04T08:00:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.506+0000 7f0b62a00640 1 -- 172.21.15.113:0/2991591114 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b540ac410 con 0x7f0b64152a50 2024-03-04T08:00:31.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.506+0000 7f0b62a00640 1 -- 172.21.15.113:0/2991591114 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b540ac7e0 con 0x7f0b64152a50 2024-03-04T08:00:31.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.510+0000 7f0b6a400640 1 -- 172.21.15.113:0/2991591114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 msgr2=0x7f0b640ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:31.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.510+0000 7f0b6a400640 1 --2- 172.21.15.113:0/2991591114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 0x7f0b640ce120 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0b540097c0 tx=0x7f0b5401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:00:31.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.510+0000 7f0b6a400640 1 -- 172.21.15.113:0/2991591114 shutdown_connections 2024-03-04T08:00:31.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.510+0000 7f0b6a400640 1 --2- 172.21.15.113:0/2991591114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 0x7f0b640ce120 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:31.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.510+0000 7f0b6a400640 1 -- 172.21.15.113:0/2991591114 >> 172.21.15.113:0/2991591114 conn(0x7f0b640734d0 msgr2=0x7f0b64066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:31.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.510+0000 7f0b6a400640 1 -- 172.21.15.113:0/2991591114 shutdown_connections 2024-03-04T08:00:31.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.510+0000 7f0b6a400640 1 -- 172.21.15.113:0/2991591114 wait complete. 2024-03-04T08:00:31.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.510+0000 7f0b6a400640 1 Processor -- start 2024-03-04T08:00:31.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.510+0000 7f0b6a400640 1 -- start start 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b6a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6406ff20 0x7f0b641f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b6a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 0x7f0b64158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b6a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6415c4e0 0x7f0b6414ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b6a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b640ce9f0 con 0x7f0b64152a50 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b6a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b640ce870 con 0x7f0b6415c4e0 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b6a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b64166fa0 con 0x7f0b6406ff20 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 0x7f0b64158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 0x7f0b64158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46612/0 (socket says 172.21.15.113:46612) 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5be00640 1 -- 172.21.15.113:0/4276271019 learned_addr learned my addr 172.21.15.113:0/4276271019 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b63400640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6406ff20 0x7f0b641f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:31.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b63e00640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6415c4e0 0x7f0b6414ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5be00640 1 -- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6406ff20 msgr2=0x7f0b641f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5be00640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6406ff20 0x7f0b641f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5be00640 1 -- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6415c4e0 msgr2=0x7f0b6414ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5be00640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6415c4e0 0x7f0b6414ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5be00640 1 -- 172.21.15.113:0/4276271019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b64156610 con 0x7f0b64152a50 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b63e00640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6415c4e0 0x7f0b6414ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5be00640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 0x7f0b64158f80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00c3c0 tx=0x7f0b4c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5b400640 1 -- 172.21.15.113:0/4276271019 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c017070 con 0x7f0b64152a50 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5b400640 1 -- 172.21.15.113:0/4276271019 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c0052d0 con 0x7f0b64152a50 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b5b400640 1 -- 172.21.15.113:0/4276271019 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c003840 con 0x7f0b64152a50 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b6a400640 1 -- 172.21.15.113:0/4276271019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b64155e70 con 0x7f0b64152a50 2024-03-04T08:00:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b6a400640 1 -- 172.21.15.113:0/4276271019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b641573e0 con 0x7f0b64152a50 2024-03-04T08:00:31.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.514+0000 7f0b6a400640 1 -- 172.21.15.113:0/4276271019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b64075c00 con 0x7f0b64152a50 2024-03-04T08:00:31.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.518+0000 7f0b5b400640 1 -- 172.21.15.113:0/4276271019 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b4c01a070 con 0x7f0b64152a50 2024-03-04T08:00:31.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.518+0000 7f0b5b400640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b40066f60 0x7f0b40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:31.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.518+0000 7f0b5b400640 1 -- 172.21.15.113:0/4276271019 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f0b4c084800 con 0x7f0b64152a50 2024-03-04T08:00:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.518+0000 7f0b63400640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b40066f60 0x7f0b40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.518+0000 7f0b63400640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b40066f60 0x7f0b40069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b540ae000 tx=0x7f0b540a6040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.526+0000 7f0b5b400640 1 -- 172.21.15.113:0/4276271019 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0b4c005480 con 0x7f0b64152a50 2024-03-04T08:00:31.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 -- 172.21.15.113:0/4276271019 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b641567e0 con 0x7f0b64152a50 2024-03-04T08:00:31.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b5b400640 1 -- 172.21.15.113:0/4276271019 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f0b4c054c60 con 0x7f0b64152a50 2024-03-04T08:00:31.784 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:31.784 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:59:07.046470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372,6240]},"id":1}]} 2024-03-04T08:00:31.784 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 94 2024-03-04T08:00:31.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 -- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b40066f60 msgr2=0x7f0b40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:31.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b40066f60 0x7f0b40069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0b540ae000 tx=0x7f0b540a6040 comp rx=0 tx=0).stop 2024-03-04T08:00:31.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 -- 172.21.15.113:0/4276271019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 msgr2=0x7f0b64158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:31.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 0x7f0b64158f80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00c3c0 tx=0x7f0b4c00c880 comp rx=0 tx=0).stop 2024-03-04T08:00:31.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 -- 172.21.15.113:0/4276271019 shutdown_connections 2024-03-04T08:00:31.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b6415c4e0 0x7f0b6414ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:31.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b40066f60 0x7f0b40069420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:31.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b64152a50 0x7f0b64158f80 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:31.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 --2- 172.21.15.113:0/4276271019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b6406ff20 0x7f0b641f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:31.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 -- 172.21.15.113:0/4276271019 >> 172.21.15.113:0/4276271019 conn(0x7f0b640734d0 msgr2=0x7f0b641520b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:31.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 -- 172.21.15.113:0/4276271019 shutdown_connections 2024-03-04T08:00:31.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:31.786+0000 7f0b6a400640 1 -- 172.21.15.113:0/4276271019 wait complete. 2024-03-04T08:00:31.804 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:00:33.520 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:00:33.525 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:00:33.805 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:33.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0ce4e00640 1 Processor -- start 2024-03-04T08:00:33.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0ce4e00640 1 -- start start 2024-03-04T08:00:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0ce4e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ce01527f0 0x7f0ce0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0ce4e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0ce0152bd0 0x7f0ce01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0ce4e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce01591f0 con 0x7f0ce00c8200 2024-03-04T08:00:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0ce4e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce0071df0 con 0x7f0ce01527f0 2024-03-04T08:00:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0ce4e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce01597b0 con 0x7f0ce0152bd0 2024-03-04T08:00:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0cdde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ce01527f0 0x7f0ce0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:33.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0cdde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ce01527f0 0x7f0ce0070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46626/0 (socket says 172.21.15.113:46626) 2024-03-04T08:00:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0cdde00640 1 -- 172.21.15.113:0/3119791693 learned_addr learned my addr 172.21.15.113:0/3119791693 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0cd5400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0ce00c8200 0x7f0ce0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39524/0 (socket says 172.21.15.113:39524) 2024-03-04T08:00:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0cde800640 1 --2- 172.21.15.113:0/3119791693 >> v2:172.21.15.136:3300/0 conn(0x7f0ce0152bd0 0x7f0ce01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0cdde00640 1 -- 172.21.15.113:0/3119791693 >> v2:172.21.15.136:3300/0 conn(0x7f0ce0152bd0 msgr2=0x7f0ce01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0cdde00640 1 --2- 172.21.15.113:0/3119791693 >> v2:172.21.15.136:3300/0 conn(0x7f0ce0152bd0 0x7f0ce01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0cdde00640 1 -- 172.21.15.113:0/3119791693 >> v1:172.21.15.113:6789/0 conn(0x7f0ce00c8200 legacy=0x7f0ce0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0cdde00640 1 -- 172.21.15.113:0/3119791693 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce01594d0 con 0x7f0ce01527f0 2024-03-04T08:00:33.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.946+0000 7f0cdd400640 1 -- 172.21.15.113:0/3119791693 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928729484 0 0) 0x7f0ce01591f0 con 0x7f0ce00c8200 2024-03-04T08:00:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0cdde00640 1 --2- 172.21.15.113:0/3119791693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce01527f0 0x7f0ce0070d40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00712b0 tx=0x7f0cd001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c0d08d8a780bb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:33.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0cdd400640 1 -- 172.21.15.113:0/3119791693 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00ab430 con 0x7f0ce01527f0 2024-03-04T08:00:33.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0cdd400640 1 -- 172.21.15.113:0/3119791693 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd00b0070 con 0x7f0ce01527f0 2024-03-04T08:00:33.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0cdd400640 1 -- 172.21.15.113:0/3119791693 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00ab7f0 con 0x7f0ce01527f0 2024-03-04T08:00:33.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0ce4e00640 1 -- 172.21.15.113:0/3119791693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce01527f0 msgr2=0x7f0ce0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:33.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0ce4e00640 1 --2- 172.21.15.113:0/3119791693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce01527f0 0x7f0ce0070d40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00712b0 tx=0x7f0cd001c520 comp rx=0 tx=0).stop 2024-03-04T08:00:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0ce4e00640 1 -- 172.21.15.113:0/3119791693 shutdown_connections 2024-03-04T08:00:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0ce4e00640 1 --2- 172.21.15.113:0/3119791693 >> v2:172.21.15.136:3300/0 conn(0x7f0ce0152bd0 0x7f0ce01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0ce4e00640 1 --2- 172.21.15.113:0/3119791693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce01527f0 0x7f0ce0070d40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0ce4e00640 1 -- 172.21.15.113:0/3119791693 >> 172.21.15.113:0/3119791693 conn(0x7f0ce006e820 msgr2=0x7f0ce006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0ce4e00640 1 -- 172.21.15.113:0/3119791693 shutdown_connections 2024-03-04T08:00:33.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.950+0000 7f0ce4e00640 1 -- 172.21.15.113:0/3119791693 wait complete. 2024-03-04T08:00:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0ce4e00640 1 Processor -- start 2024-03-04T08:00:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0ce4e00640 1 -- start start 2024-03-04T08:00:33.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0ce4e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce00c8200 0x7f0ce00c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0ce4e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ce01527f0 0x7f0ce0153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0ce4e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ce0152bd0 0x7f0ce00af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0ce4e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01654f0 con 0x7f0ce00c8200 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0ce4e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0165370 con 0x7f0ce01527f0 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0ce4e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0165670 con 0x7f0ce0152bd0 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cdde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce00c8200 0x7f0ce00c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cdde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce00c8200 0x7f0ce00c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46642/0 (socket says 172.21.15.113:46642) 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cdde00640 1 -- 172.21.15.113:0/174839387 learned_addr learned my addr 172.21.15.113:0/174839387 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cd5400640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ce01527f0 0x7f0ce0153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cde800640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ce0152bd0 0x7f0ce00af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cdde00640 1 -- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ce0152bd0 msgr2=0x7f0ce00af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cdde00640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ce0152bd0 0x7f0ce00af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cdde00640 1 -- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ce01527f0 msgr2=0x7f0ce0153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:33.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cdde00640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ce01527f0 0x7f0ce0153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:33.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cdde00640 1 -- 172.21.15.113:0/174839387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce00c1960 con 0x7f0ce00c8200 2024-03-04T08:00:33.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cdde00640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce00c8200 0x7f0ce00c6030 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00ad040 tx=0x7f0cd001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:33.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cd6a00640 1 -- 172.21.15.113:0/174839387 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00be5f0 con 0x7f0ce00c8200 2024-03-04T08:00:33.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cd6a00640 1 -- 172.21.15.113:0/174839387 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd00be7b0 con 0x7f0ce00c8200 2024-03-04T08:00:33.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0ce4e00640 1 -- 172.21.15.113:0/174839387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce01f7ed0 con 0x7f0ce00c8200 2024-03-04T08:00:33.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0cd6a00640 1 -- 172.21.15.113:0/174839387 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00beba0 con 0x7f0ce00c8200 2024-03-04T08:00:33.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.954+0000 7f0ce4e00640 1 -- 172.21.15.113:0/174839387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce00c5830 con 0x7f0ce00c8200 2024-03-04T08:00:33.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.958+0000 7f0ce4e00640 1 -- 172.21.15.113:0/174839387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce0075c00 con 0x7f0ce00c8200 2024-03-04T08:00:33.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.958+0000 7f0cd6a00640 1 -- 172.21.15.113:0/174839387 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0cd00b43f0 con 0x7f0ce00c8200 2024-03-04T08:00:33.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.958+0000 7f0cd6a00640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0cbc066f60 0x7f0cbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:33.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.958+0000 7f0cd6a00640 1 -- 172.21.15.113:0/174839387 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f0cd00a8070 con 0x7f0ce00c8200 2024-03-04T08:00:33.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.958+0000 7f0cd5400640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0cbc066f60 0x7f0cbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:33.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.962+0000 7f0cd5400640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0cbc066f60 0x7f0cbc069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00c2040 tx=0x7f0cc8007b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:33.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:33.962+0000 7f0cd6a00640 1 -- 172.21.15.113:0/174839387 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0cd0016110 con 0x7f0ce00c8200 2024-03-04T08:00:34.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.170+0000 7f0ce4e00640 1 -- 172.21.15.113:0/174839387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ce014e700 con 0x7f0ce00c8200 2024-03-04T08:00:34.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0cd6a00640 1 -- 172.21.15.113:0/174839387 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f0cd00f0810 con 0x7f0ce00c8200 2024-03-04T08:00:34.171 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:34.171 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:59:07.046470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372,6240]},"id":1}]} 2024-03-04T08:00:34.171 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 94 2024-03-04T08:00:34.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 -- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0cbc066f60 msgr2=0x7f0cbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:34.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0cbc066f60 0x7f0cbc069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00c2040 tx=0x7f0cc8007b80 comp rx=0 tx=0).stop 2024-03-04T08:00:34.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 -- 172.21.15.113:0/174839387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce00c8200 msgr2=0x7f0ce00c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:34.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce00c8200 0x7f0ce00c6030 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00ad040 tx=0x7f0cd001ad60 comp rx=0 tx=0).stop 2024-03-04T08:00:34.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 -- 172.21.15.113:0/174839387 shutdown_connections 2024-03-04T08:00:34.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ce0152bd0 0x7f0ce00af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:34.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ce01527f0 0x7f0ce0153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:34.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0cbc066f60 0x7f0cbc069420 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 --2- 172.21.15.113:0/174839387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ce00c8200 0x7f0ce00c6030 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 -- 172.21.15.113:0/174839387 >> 172.21.15.113:0/174839387 conn(0x7f0ce006e820 msgr2=0x7f0ce01520d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 -- 172.21.15.113:0/174839387 shutdown_connections 2024-03-04T08:00:34.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:34.174+0000 7f0ce4e00640 1 -- 172.21.15.113:0/174839387 wait complete. 2024-03-04T08:00:34.200 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:00:36.201 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:36.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f455f000640 1 Processor -- start 2024-03-04T08:00:36.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f455f000640 1 -- start start 2024-03-04T08:00:36.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f455f000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45580ca0d0 0x7f45580ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:36.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f455f000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f45580703e0 0x7f45580ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:36.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f455f000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45580c5470 con 0x7f4558152d00 2024-03-04T08:00:36.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f455f000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45580c5220 con 0x7f45580ca0d0 2024-03-04T08:00:36.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f455f000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45580cba40 con 0x7f45580703e0 2024-03-04T08:00:36.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f4557400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45580ca0d0 0x7f45580ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:36.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f4557e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4558152d00 0x7f45580ca5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39526/0 (socket says 172.21.15.113:39526) 2024-03-04T08:00:36.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f4557e00640 1 -- 172.21.15.113:0/2082556386 learned_addr learned my addr 172.21.15.113:0/2082556386 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:00:36.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f455d000640 1 --2- 172.21.15.113:0/2082556386 >> v2:172.21.15.136:3300/0 conn(0x7f45580703e0 0x7f45580ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:36.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f4557400640 1 -- 172.21.15.113:0/2082556386 >> v2:172.21.15.136:3300/0 conn(0x7f45580703e0 msgr2=0x7f45580ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:36.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f4557400640 1 --2- 172.21.15.113:0/2082556386 >> v2:172.21.15.136:3300/0 conn(0x7f45580703e0 0x7f45580ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f4557400640 1 -- 172.21.15.113:0/2082556386 >> v1:172.21.15.113:6789/0 conn(0x7f4558152d00 legacy=0x7f45580ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:36.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f4556a00640 1 -- 172.21.15.113:0/2082556386 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804577125 0 0) 0x7f45580c5470 con 0x7f4558152d00 2024-03-04T08:00:36.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f4557400640 1 -- 172.21.15.113:0/2082556386 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455806f690 con 0x7f45580ca0d0 2024-03-04T08:00:36.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.346+0000 7f455d000640 1 --2- 172.21.15.113:0/2082556386 >> v2:172.21.15.136:3300/0 conn(0x7f45580703e0 0x7f45580ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:00:36.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f4557400640 1 --2- 172.21.15.113:0/2082556386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45580ca0d0 0x7f45580ce3c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4558072e00 tx=0x7f454401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2f3cfba7e01693 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:36.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f4556a00640 1 -- 172.21.15.113:0/2082556386 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440b0070 con 0x7f45580ca0d0 2024-03-04T08:00:36.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f4556a00640 1 -- 172.21.15.113:0/2082556386 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45440ab600 con 0x7f45580ca0d0 2024-03-04T08:00:36.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f4556a00640 1 -- 172.21.15.113:0/2082556386 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440ab9d0 con 0x7f45580ca0d0 2024-03-04T08:00:36.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f455f000640 1 -- 172.21.15.113:0/2082556386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45580ca0d0 msgr2=0x7f45580ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:36.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f455f000640 1 --2- 172.21.15.113:0/2082556386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45580ca0d0 0x7f45580ce3c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f4558072e00 tx=0x7f454401c3f0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f455f000640 1 -- 172.21.15.113:0/2082556386 shutdown_connections 2024-03-04T08:00:36.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f455f000640 1 --2- 172.21.15.113:0/2082556386 >> v2:172.21.15.136:3300/0 conn(0x7f45580703e0 0x7f45580ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f455f000640 1 --2- 172.21.15.113:0/2082556386 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45580ca0d0 0x7f45580ce3c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f455f000640 1 -- 172.21.15.113:0/2082556386 >> 172.21.15.113:0/2082556386 conn(0x7f4558066db0 msgr2=0x7f45580671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:36.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f455f000640 1 -- 172.21.15.113:0/2082556386 shutdown_connections 2024-03-04T08:00:36.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.350+0000 7f455f000640 1 -- 172.21.15.113:0/2082556386 wait complete. 2024-03-04T08:00:36.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.354+0000 7f455f000640 1 Processor -- start 2024-03-04T08:00:36.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.354+0000 7f455f000640 1 -- start start 2024-03-04T08:00:36.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.354+0000 7f455f000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45580703e0 0x7f45580c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:36.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.354+0000 7f455f000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45580ca0d0 0x7f45581f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:36.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.354+0000 7f455f000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4558152d00 0x7f45580ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:36.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.354+0000 7f455f000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4558165b40 con 0x7f4558152d00 2024-03-04T08:00:36.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.354+0000 7f455f000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45581659c0 con 0x7f45580ca0d0 2024-03-04T08:00:36.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.354+0000 7f455f000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4558165cc0 con 0x7f45580703e0 2024-03-04T08:00:36.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4558152d00 0x7f45580ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:36.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4558152d00 0x7f45580ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46648/0 (socket says 172.21.15.113:46648) 2024-03-04T08:00:36.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455d000640 1 -- 172.21.15.113:0/975772501 learned_addr learned my addr 172.21.15.113:0/975772501 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:36.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f4557e00640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45580703e0 0x7f45580c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:36.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f4557400640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45580ca0d0 0x7f45581f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:36.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455d000640 1 -- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45580703e0 msgr2=0x7f45580c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:36.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455d000640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45580703e0 0x7f45580c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455d000640 1 -- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45580ca0d0 msgr2=0x7f45581f7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:36.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455d000640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45580ca0d0 0x7f45581f7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455d000640 1 -- 172.21.15.113:0/975772501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45580c8870 con 0x7f4558152d00 2024-03-04T08:00:36.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455d000640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4558152d00 0x7f45580ca8c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4550007de0 tx=0x7f455000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:36.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f454fe00640 1 -- 172.21.15.113:0/975772501 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455001a070 con 0x7f4558152d00 2024-03-04T08:00:36.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f454fe00640 1 -- 172.21.15.113:0/975772501 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45500039a0 con 0x7f4558152d00 2024-03-04T08:00:36.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455f000640 1 -- 172.21.15.113:0/975772501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45581534e0 con 0x7f4558152d00 2024-03-04T08:00:36.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f454fe00640 1 -- 172.21.15.113:0/975772501 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550003d90 con 0x7f4558152d00 2024-03-04T08:00:36.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455f000640 1 -- 172.21.15.113:0/975772501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4558153aa0 con 0x7f4558152d00 2024-03-04T08:00:36.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.358+0000 7f455f000640 1 -- 172.21.15.113:0/975772501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4558075c00 con 0x7f4558152d00 2024-03-04T08:00:36.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.362+0000 7f454fe00640 1 -- 172.21.15.113:0/975772501 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f455001d070 con 0x7f4558152d00 2024-03-04T08:00:36.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.362+0000 7f454fe00640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4538066f60 0x7f4538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:36.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.362+0000 7f454fe00640 1 -- 172.21.15.113:0/975772501 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f4550085670 con 0x7f4558152d00 2024-03-04T08:00:36.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.362+0000 7f4557e00640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4538066f60 0x7f4538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:36.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.362+0000 7f4557e00640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4538066f60 0x7f4538069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4548000f70 tx=0x7f4548007df0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:36.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.366+0000 7f454fe00640 1 -- 172.21.15.113:0/975772501 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4550010070 con 0x7f4558152d00 2024-03-04T08:00:36.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.634+0000 7f455f000640 1 -- 172.21.15.113:0/975772501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45580cb370 con 0x7f4558152d00 2024-03-04T08:00:36.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.634+0000 7f454fe00640 1 -- 172.21.15.113:0/975772501 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f4550055bf0 con 0x7f4558152d00 2024-03-04T08:00:36.635 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:36.635 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:59:07.046470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372,6240]},"id":1}]} 2024-03-04T08:00:36.635 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 94 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 -- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4538066f60 msgr2=0x7f4538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4538066f60 0x7f4538069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4548000f70 tx=0x7f4548007df0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 -- 172.21.15.113:0/975772501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4558152d00 msgr2=0x7f45580ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4558152d00 0x7f45580ca8c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4550007de0 tx=0x7f455000d4a0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 -- 172.21.15.113:0/975772501 shutdown_connections 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4538066f60 0x7f4538069420 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4558152d00 0x7f45580ca8c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f45580ca0d0 0x7f45581f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 --2- 172.21.15.113:0/975772501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f45580703e0 0x7f45580c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 -- 172.21.15.113:0/975772501 >> 172.21.15.113:0/975772501 conn(0x7f4558066db0 msgr2=0x7f4558162870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 -- 172.21.15.113:0/975772501 shutdown_connections 2024-03-04T08:00:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:36.638+0000 7f455f000640 1 -- 172.21.15.113:0/975772501 wait complete. 2024-03-04T08:00:36.662 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:00:38.664 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:38.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.814+0000 7f4adbe00640 1 Processor -- start 2024-03-04T08:00:38.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4adbe00640 1 -- start start 2024-03-04T08:00:38.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4adbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ad40ce6b0 0x7f4ad415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:38.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4adbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ad406de80 0x7f4ad4154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4adbe00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad406c210 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4adbe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad415bbf0 con 0x7f4ad406de80 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4adbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad406eb80 con 0x7f4ad40ce6b0 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4ad9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ad406de80 0x7f4ad4154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4ad9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4ad406de80 0x7f4ad4154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35176/0 (socket says 172.21.15.113:35176) 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4ad9e00640 1 -- 172.21.15.113:0/581727716 learned_addr learned my addr 172.21.15.113:0/581727716 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4ad8a00640 1 --2- 172.21.15.113:0/581727716 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ad40ce6b0 0x7f4ad415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4ad9e00640 1 -- 172.21.15.113:0/581727716 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ad40ce6b0 msgr2=0x7f4ad415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4ad9e00640 1 --2- 172.21.15.113:0/581727716 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ad40ce6b0 0x7f4ad415b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4ad9e00640 1 -- 172.21.15.113:0/581727716 >> v1:172.21.15.136:6789/0 conn(0x7f4ad40ce2d0 legacy=0x7f4ad406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.818+0000 7f4ad9e00640 1 -- 172.21.15.113:0/581727716 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad406c4f0 con 0x7f4ad406de80 2024-03-04T08:00:38.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4ad9e00640 1 --2- 172.21.15.113:0/581727716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad406de80 0x7f4ad4154b10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009870 tx=0x7f4acc01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2fa503fd377f2a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:38.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4ad3e00640 1 -- 172.21.15.113:0/581727716 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0b5070 con 0x7f4ad406de80 2024-03-04T08:00:38.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4ad3e00640 1 -- 172.21.15.113:0/581727716 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc0a8a50 con 0x7f4ad406de80 2024-03-04T08:00:38.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4ad3e00640 1 -- 172.21.15.113:0/581727716 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0a8e20 con 0x7f4ad406de80 2024-03-04T08:00:38.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4adbe00640 1 -- 172.21.15.113:0/581727716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad406de80 msgr2=0x7f4ad4154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:38.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4adbe00640 1 --2- 172.21.15.113:0/581727716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad406de80 0x7f4ad4154b10 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4acc009870 tx=0x7f4acc01ed50 comp rx=0 tx=0).stop 2024-03-04T08:00:38.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4adbe00640 1 -- 172.21.15.113:0/581727716 shutdown_connections 2024-03-04T08:00:38.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4adbe00640 1 --2- 172.21.15.113:0/581727716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad406de80 0x7f4ad4154b10 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:38.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4adbe00640 1 --2- 172.21.15.113:0/581727716 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ad40ce6b0 0x7f4ad415b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:38.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4adbe00640 1 -- 172.21.15.113:0/581727716 >> 172.21.15.113:0/581727716 conn(0x7f4ad40709a0 msgr2=0x7f4ad4070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:38.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4adbe00640 1 -- 172.21.15.113:0/581727716 shutdown_connections 2024-03-04T08:00:38.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4adbe00640 1 -- 172.21.15.113:0/581727716 wait complete. 2024-03-04T08:00:38.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4adbe00640 1 Processor -- start 2024-03-04T08:00:38.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.822+0000 7f4adbe00640 1 -- start start 2024-03-04T08:00:38.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4adbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ad406de80 0x7f4ad4150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:38.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4adbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad40ce2d0 0x7f4ad40c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:38.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4adbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ad40ce6b0 0x7f4ad4158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:38.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4adbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad406c7d0 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4adbe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad406e810 con 0x7f4ad406de80 2024-03-04T08:00:38.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4adbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad406f720 con 0x7f4ad40ce6b0 2024-03-04T08:00:38.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad40ce2d0 0x7f4ad40c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:38.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad40ce2d0 0x7f4ad40c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35188/0 (socket says 172.21.15.113:35188) 2024-03-04T08:00:38.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad8a00640 1 -- 172.21.15.113:0/3794067041 learned_addr learned my addr 172.21.15.113:0/3794067041 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:38.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad9e00640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ad40ce6b0 0x7f4ad4158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:38.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad9400640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ad406de80 0x7f4ad4150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:38.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad8a00640 1 -- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ad40ce6b0 msgr2=0x7f4ad4158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:38.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad8a00640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ad40ce6b0 0x7f4ad4158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:38.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad8a00640 1 -- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ad406de80 msgr2=0x7f4ad4150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:38.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad8a00640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ad406de80 0x7f4ad4150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:38.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad8a00640 1 -- 172.21.15.113:0/3794067041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad4151fd0 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad8a00640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad40ce2d0 0x7f4ad40c7cc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000d800 tx=0x7f4ac000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:38.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad1600640 1 -- 172.21.15.113:0/3794067041 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000b6c0 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad1600640 1 -- 172.21.15.113:0/3794067041 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac0019070 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4ad1600640 1 -- 172.21.15.113:0/3794067041 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac000ba80 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4adbe00640 1 -- 172.21.15.113:0/3794067041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad4152270 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4adbe00640 1 -- 172.21.15.113:0/3794067041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad4152830 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.826+0000 7f4adbe00640 1 -- 172.21.15.113:0/3794067041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad4159b70 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.830+0000 7f4ad1600640 1 -- 172.21.15.113:0/3794067041 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ac000bc20 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.830+0000 7f4ad1600640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ab4066f60 0x7f4ab4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.830+0000 7f4ad1600640 1 -- 172.21.15.113:0/3794067041 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6943+0+0 (secure 0 0 0) 0x7f4ac0085a00 con 0x7f4ad40ce2d0 2024-03-04T08:00:38.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.830+0000 7f4ad9400640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ab4066f60 0x7f4ab4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:38.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.830+0000 7f4ad9400640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ab4066f60 0x7f4ab4069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8001d50 tx=0x7f4ac8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:38.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:38.834+0000 7f4ad1600640 1 -- 172.21.15.113:0/3794067041 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4ac00564b0 con 0x7f4ad40ce2d0 2024-03-04T08:00:39.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.090+0000 7f4adbe00640 1 -- 172.21.15.113:0/3794067041 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ad406cb40 con 0x7f4ad40ce2d0 2024-03-04T08:00:39.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4ad1600640 1 -- 172.21.15.113:0/3794067041 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+6615 (secure 0 0 0) 0x7f4ac0055db0 con 0x7f4ad40ce2d0 2024-03-04T08:00:39.091 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:39.091 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6053,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T07:59:07.046470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6240},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6240":{"gid":6240,"name":"a","rank":2,"incarnation":77,"state":"up:active","state_seq":41,"addr":"172.21.15.113:6836/986515123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":986515123},{"type":"v1","addr":"172.21.15.113:6836","nonce":986515123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372,6240]},"id":1}]} 2024-03-04T08:00:39.092 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 94 2024-03-04T08:00:39.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 -- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ab4066f60 msgr2=0x7f4ab4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:39.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ab4066f60 0x7f4ab4069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8001d50 tx=0x7f4ac8009040 comp rx=0 tx=0).stop 2024-03-04T08:00:39.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 -- 172.21.15.113:0/3794067041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad40ce2d0 msgr2=0x7f4ad40c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:39.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad40ce2d0 0x7f4ad40c7cc0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000d800 tx=0x7f4ac000dcc0 comp rx=0 tx=0).stop 2024-03-04T08:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 -- 172.21.15.113:0/3794067041 shutdown_connections 2024-03-04T08:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4ad40ce6b0 0x7f4ad4158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ab4066f60 0x7f4ab4069420 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4ad40ce2d0 0x7f4ad40c7cc0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 --2- 172.21.15.113:0/3794067041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ad406de80 0x7f4ad4150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 -- 172.21.15.113:0/3794067041 >> 172.21.15.113:0/3794067041 conn(0x7f4ad40709a0 msgr2=0x7f4ad414c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 -- 172.21.15.113:0/3794067041 shutdown_connections 2024-03-04T08:00:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:39.094+0000 7f4adbe00640 1 -- 172.21.15.113:0/3794067041 wait complete. 2024-03-04T08:00:39.112 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:00:41.113 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:41.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff87da00640 1 Processor -- start 2024-03-04T08:00:41.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff87da00640 1 -- start start 2024-03-04T08:00:41.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff87da00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff87806d420 0x7ff8780730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff87da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff87815bce0 0x7ff87815c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff87da00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87806f1b0 con 0x7ff878152a90 2024-03-04T08:00:41.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff87da00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff878152e70 con 0x7ff87806d420 2024-03-04T08:00:41.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff87da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87806c320 con 0x7ff87815bce0 2024-03-04T08:00:41.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff877400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff87806d420 0x7ff8780730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff877400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff87806d420 0x7ff8780730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35194/0 (socket says 172.21.15.113:35194) 2024-03-04T08:00:41.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff877400640 1 -- 172.21.15.113:0/1825507178 learned_addr learned my addr 172.21.15.113:0/1825507178 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:41.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff87d000640 1 --2- 172.21.15.113:0/1825507178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff87815bce0 0x7ff87815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff877400640 1 -- 172.21.15.113:0/1825507178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff87815bce0 msgr2=0x7ff87815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:41.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff877400640 1 --2- 172.21.15.113:0/1825507178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff87815bce0 0x7ff87815c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff877400640 1 -- 172.21.15.113:0/1825507178 >> v1:172.21.15.136:6789/0 conn(0x7ff878152a90 legacy=0x7ff87806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.254+0000 7ff877400640 1 -- 172.21.15.113:0/1825507178 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff878071580 con 0x7ff87806d420 2024-03-04T08:00:41.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff877400640 1 --2- 172.21.15.113:0/1825507178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff87806d420 0x7ff8780730d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff864009870 tx=0x7ff86401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d30d76d6e5d37b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:41.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff876a00640 1 -- 172.21.15.113:0/1825507178 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640b2070 con 0x7ff87806d420 2024-03-04T08:00:41.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff876a00640 1 -- 172.21.15.113:0/1825507178 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8640a9070 con 0x7ff87806d420 2024-03-04T08:00:41.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff876a00640 1 -- 172.21.15.113:0/1825507178 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640ad880 con 0x7ff87806d420 2024-03-04T08:00:41.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff87da00640 1 -- 172.21.15.113:0/1825507178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff87806d420 msgr2=0x7ff8780730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff87da00640 1 --2- 172.21.15.113:0/1825507178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff87806d420 0x7ff8780730d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff864009870 tx=0x7ff86401c560 comp rx=0 tx=0).stop 2024-03-04T08:00:41.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff87da00640 1 -- 172.21.15.113:0/1825507178 shutdown_connections 2024-03-04T08:00:41.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff87da00640 1 --2- 172.21.15.113:0/1825507178 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff87815bce0 0x7ff87815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff87da00640 1 --2- 172.21.15.113:0/1825507178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff87806d420 0x7ff8780730d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff87da00640 1 -- 172.21.15.113:0/1825507178 >> 172.21.15.113:0/1825507178 conn(0x7ff87806e2a0 msgr2=0x7ff87806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:41.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff87da00640 1 -- 172.21.15.113:0/1825507178 shutdown_connections 2024-03-04T08:00:41.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.258+0000 7ff87da00640 1 -- 172.21.15.113:0/1825507178 wait complete. 2024-03-04T08:00:41.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 Processor -- start 2024-03-04T08:00:41.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 -- start start 2024-03-04T08:00:41.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff87806d420 0x7ff8780cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff878152a90 0x7ff8780cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff87815bce0 0x7ff878159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87815ca80 con 0x7ff878152a90 2024-03-04T08:00:41.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff878071330 con 0x7ff87815bce0 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff878165aa0 con 0x7ff87806d420 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff877400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff878152a90 0x7ff8780cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff877400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff878152a90 0x7ff8780cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35202/0 (socket says 172.21.15.113:35202) 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff877400640 1 -- 172.21.15.113:0/3916068207 learned_addr learned my addr 172.21.15.113:0/3916068207 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87d000640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff87815bce0 0x7ff878159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff877e00640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff87806d420 0x7ff8780cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff877400640 1 -- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff87806d420 msgr2=0x7ff8780cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff877400640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff87806d420 0x7ff8780cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff877400640 1 -- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff87815bce0 msgr2=0x7ff878159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff877400640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff87815bce0 0x7ff878159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff877400640 1 -- 172.21.15.113:0/3916068207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87813b890 con 0x7ff878152a90 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87d000640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff87815bce0 0x7ff878159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:00:41.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff877400640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff878152a90 0x7ff8780cb5b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff8640b0040 tx=0x7ff864019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:41.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff86fe00640 1 -- 172.21.15.113:0/3916068207 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640bb070 con 0x7ff878152a90 2024-03-04T08:00:41.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff86fe00640 1 -- 172.21.15.113:0/3916068207 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8640adce0 con 0x7ff878152a90 2024-03-04T08:00:41.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 -- 172.21.15.113:0/3916068207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff878162160 con 0x7ff878152a90 2024-03-04T08:00:41.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff86fe00640 1 -- 172.21.15.113:0/3916068207 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640b2040 con 0x7ff878152a90 2024-03-04T08:00:41.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 -- 172.21.15.113:0/3916068207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87813be10 con 0x7ff878152a90 2024-03-04T08:00:41.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.262+0000 7ff87da00640 1 -- 172.21.15.113:0/3916068207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff878075a80 con 0x7ff878152a90 2024-03-04T08:00:41.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.266+0000 7ff86fe00640 1 -- 172.21.15.113:0/3916068207 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8640be070 con 0x7ff878152a90 2024-03-04T08:00:41.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.266+0000 7ff86fe00640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff858066f60 0x7ff858069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.266+0000 7ff86fe00640 1 -- 172.21.15.113:0/3916068207 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7ff86411e440 con 0x7ff878152a90 2024-03-04T08:00:41.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.266+0000 7ff877e00640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff858066f60 0x7ff858069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.266+0000 7ff877e00640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff858066f60 0x7ff858069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff8680015b0 tx=0x7ff868009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:41.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.274+0000 7ff86fe00640 1 -- 172.21.15.113:0/3916068207 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff864016150 con 0x7ff878152a90 2024-03-04T08:00:41.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.470+0000 7ff87da00640 1 -- 172.21.15.113:0/3916068207 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8780c5f30 con 0x7ff878152a90 2024-03-04T08:00:41.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.470+0000 7ff86fe00640 1 -- 172.21.15.113:0/3916068207 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+5818 (secure 0 0 0) 0x7ff8640eee90 con 0x7ff878152a90 2024-03-04T08:00:41.470 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:41.470 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":95,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:00:40.798422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:00:41.470 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 95 2024-03-04T08:00:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 -- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff858066f60 msgr2=0x7ff858069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff858066f60 0x7ff858069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff8680015b0 tx=0x7ff868009040 comp rx=0 tx=0).stop 2024-03-04T08:00:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 -- 172.21.15.113:0/3916068207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff878152a90 msgr2=0x7ff8780cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff878152a90 0x7ff8780cb5b0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff8640b0040 tx=0x7ff864019300 comp rx=0 tx=0).stop 2024-03-04T08:00:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 -- 172.21.15.113:0/3916068207 shutdown_connections 2024-03-04T08:00:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff87815bce0 0x7ff878159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff858066f60 0x7ff858069420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff878152a90 0x7ff8780cb5b0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 --2- 172.21.15.113:0/3916068207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff87806d420 0x7ff8780cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 -- 172.21.15.113:0/3916068207 >> 172.21.15.113:0/3916068207 conn(0x7ff87806e2a0 msgr2=0x7ff878146010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 -- 172.21.15.113:0/3916068207 shutdown_connections 2024-03-04T08:00:41.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.474+0000 7ff87da00640 1 -- 172.21.15.113:0/3916068207 wait complete. 2024-03-04T08:00:41.500 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-03-04T08:00:41.500 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:00:41.501 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:41.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28bb200640 1 Processor -- start 2024-03-04T08:00:41.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28bb200640 1 -- start start 2024-03-04T08:00:41.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28bb200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f28bc157df0 0x7f28bc06c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28bb200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28bc070fd0 con 0x7f28bc0ce020 2024-03-04T08:00:41.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28bb200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28bc06c6b0 con 0x7f28bc0c8a90 2024-03-04T08:00:41.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28bb200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc15e090 con 0x7f28bc157df0 2024-03-04T08:00:41.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28ba800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f28bc0ce020 0x7f28bc1650b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49072/0 (socket says 172.21.15.113:49072) 2024-03-04T08:00:41.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28ba800640 1 -- 172.21.15.113:0/122076819 learned_addr learned my addr 172.21.15.113:0/122076819 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:00:41.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28b9e00640 1 --2- 172.21.15.113:0/122076819 >> v2:172.21.15.136:3300/0 conn(0x7f28bc157df0 0x7f28bc06c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28b9400640 1 -- 172.21.15.113:0/122076819 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501451967 0 0) 0x7f28b4001120 con 0x7f28bc0ce020 2024-03-04T08:00:41.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28b9400640 1 -- 172.21.15.113:0/122076819 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28a0003530 con 0x7f28bc0ce020 2024-03-04T08:00:41.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28b9400640 1 -- 172.21.15.113:0/122076819 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447241350 0 0) 0x7f28bc06c6b0 con 0x7f28bc0c8a90 2024-03-04T08:00:41.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28b9400640 1 -- 172.21.15.113:0/122076819 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28b4001120 con 0x7f28bc0c8a90 2024-03-04T08:00:41.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28b9e00640 1 -- 172.21.15.113:0/122076819 >> v1:172.21.15.136:6789/0 conn(0x7f28bc0c8a90 legacy=0x7f28bc15e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28b9e00640 1 -- 172.21.15.113:0/122076819 >> v1:172.21.15.113:6789/0 conn(0x7f28bc0ce020 legacy=0x7f28bc1650b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28b9400640 1 -- 172.21.15.113:0/122076819 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2318514498 0 0) 0x7f28a0003530 con 0x7f28bc0ce020 2024-03-04T08:00:41.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.646+0000 7f28b9e00640 1 -- 172.21.15.113:0/122076819 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc06dfd0 con 0x7f28bc157df0 2024-03-04T08:00:41.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.650+0000 7f28b9e00640 1 --2- 172.21.15.113:0/122076819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc157df0 0x7f28bc06c170 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009d90 tx=0x7f28ac01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c9a30ed8b9e08eb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:41.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.650+0000 7f28b9400640 1 -- 172.21.15.113:0/122076819 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4001850 con 0x7f28bc157df0 2024-03-04T08:00:41.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28b9400640 1 -- 172.21.15.113:0/122076819 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac0a64e0 con 0x7f28bc157df0 2024-03-04T08:00:41.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28b9400640 1 -- 172.21.15.113:0/122076819 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ac0a6890 con 0x7f28bc157df0 2024-03-04T08:00:41.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28bb200640 1 -- 172.21.15.113:0/122076819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc157df0 msgr2=0x7f28bc06c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28bb200640 1 --2- 172.21.15.113:0/122076819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc157df0 0x7f28bc06c170 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009d90 tx=0x7f28ac01c890 comp rx=0 tx=0).stop 2024-03-04T08:00:41.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28bb200640 1 -- 172.21.15.113:0/122076819 shutdown_connections 2024-03-04T08:00:41.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28bb200640 1 --2- 172.21.15.113:0/122076819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc157df0 0x7f28bc06c170 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28bb200640 1 -- 172.21.15.113:0/122076819 >> 172.21.15.113:0/122076819 conn(0x7f28bc072f40 msgr2=0x7f28bc0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:41.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28bb200640 1 -- 172.21.15.113:0/122076819 shutdown_connections 2024-03-04T08:00:41.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28bb200640 1 -- 172.21.15.113:0/122076819 wait complete. 2024-03-04T08:00:41.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28bb200640 1 Processor -- start 2024-03-04T08:00:41.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.654+0000 7f28bb200640 1 -- start start 2024-03-04T08:00:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28bb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc0c8a90 0x7f28bc14de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28bb200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc0ce020 0x7f28bc158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28bb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28bc157df0 0x7f28bc1592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28bb200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc15ec30 con 0x7f28bc157df0 2024-03-04T08:00:41.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28bb200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc15dd20 con 0x7f28bc0c8a90 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28bb200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0c30b0 con 0x7f28bc0ce020 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28ba800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28bc157df0 0x7f28bc1592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28ba800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28bc157df0 0x7f28bc1592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35210/0 (socket says 172.21.15.113:35210) 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28ba800640 1 -- 172.21.15.113:0/1233869470 learned_addr learned my addr 172.21.15.113:0/1233869470 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28b9e00640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc0c8a90 0x7f28bc14de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28b1400640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc0ce020 0x7f28bc158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28ba800640 1 -- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc0ce020 msgr2=0x7f28bc158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28ba800640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc0ce020 0x7f28bc158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28ba800640 1 -- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc0c8a90 msgr2=0x7f28bc14de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28ba800640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc0c8a90 0x7f28bc14de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28ba800640 1 -- 172.21.15.113:0/1233869470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc1f7990 con 0x7f28bc157df0 2024-03-04T08:00:41.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28ba800640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28bc157df0 0x7f28bc1592e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f28b400c7c0 tx=0x7f28b400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:41.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28b2a00640 1 -- 172.21.15.113:0/1233869470 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4005cf0 con 0x7f28bc157df0 2024-03-04T08:00:41.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28b2a00640 1 -- 172.21.15.113:0/1233869470 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b4007070 con 0x7f28bc157df0 2024-03-04T08:00:41.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28bb200640 1 -- 172.21.15.113:0/1233869470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc15b010 con 0x7f28bc157df0 2024-03-04T08:00:41.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28b2a00640 1 -- 172.21.15.113:0/1233869470 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b4007460 con 0x7f28bc157df0 2024-03-04T08:00:41.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28bb200640 1 -- 172.21.15.113:0/1233869470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc15bb70 con 0x7f28bc157df0 2024-03-04T08:00:41.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.658+0000 7f28bb200640 1 -- 172.21.15.113:0/1233869470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc158700 con 0x7f28bc157df0 2024-03-04T08:00:41.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.662+0000 7f28b2a00640 1 -- 172.21.15.113:0/1233869470 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28b4007600 con 0x7f28bc157df0 2024-03-04T08:00:41.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.662+0000 7f28b2a00640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2898066f60 0x7f2898069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:41.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.662+0000 7f28b2a00640 1 -- 172.21.15.113:0/1233869470 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f28b4085dc0 con 0x7f28bc157df0 2024-03-04T08:00:41.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.662+0000 7f28b9e00640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2898066f60 0x7f2898069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:41.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.662+0000 7f28b9e00640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2898066f60 0x7f2898069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0af000 tx=0x7f28ac0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:41.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.670+0000 7f28b2a00640 1 -- 172.21.15.113:0/1233869470 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f28bc070fd0 con 0x7f28bc157df0 2024-03-04T08:00:41.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 -- 172.21.15.113:0/1233869470 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28bc075a80 con 0x7f28bc157df0 2024-03-04T08:00:41.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28b2a00640 1 -- 172.21.15.113:0/1233869470 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f28b4056480 con 0x7f28bc157df0 2024-03-04T08:00:41.857 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:41.857 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":95,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:00:40.798422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:00:41.858 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 95 2024-03-04T08:00:41.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 -- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2898066f60 msgr2=0x7f2898069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2898066f60 0x7f2898069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f28ac0af000 tx=0x7f28ac0a7000 comp rx=0 tx=0).stop 2024-03-04T08:00:41.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 -- 172.21.15.113:0/1233869470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28bc157df0 msgr2=0x7f28bc1592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:41.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28bc157df0 0x7f28bc1592e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f28b400c7c0 tx=0x7f28b400cc80 comp rx=0 tx=0).stop 2024-03-04T08:00:41.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 -- 172.21.15.113:0/1233869470 shutdown_connections 2024-03-04T08:00:41.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2898066f60 0x7f2898069420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28bc157df0 0x7f28bc1592e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28bc0ce020 0x7f28bc158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 --2- 172.21.15.113:0/1233869470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28bc0c8a90 0x7f28bc14de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:41.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.858+0000 7f28bb200640 1 -- 172.21.15.113:0/1233869470 >> 172.21.15.113:0/1233869470 conn(0x7f28bc072f40 msgr2=0x7f28bc15d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:41.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.862+0000 7f28bb200640 1 -- 172.21.15.113:0/1233869470 shutdown_connections 2024-03-04T08:00:41.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:41.862+0000 7f28bb200640 1 -- 172.21.15.113:0/1233869470 wait complete. 2024-03-04T08:00:41.881 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 95, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 95, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6053": { "addr": "172.21.15.136:6835/3388011401", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3388011401, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3388011401, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6053, "incarnation": 95, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:replay", "state_seq": 1 }, "gid_6372": { "addr": "172.21.15.113:6833/2732565639", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2732565639, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2732565639, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6372, "incarnation": 83, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 40 }, "gid_6534": { "addr": "172.21.15.113:6837/4197507712", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 4197507712, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 4197507712, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6534, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 62, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:00:40.798422+0000", "qdb_cluster": [ 6372, 6534 ], "qdb_leader": 6372, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6372, "mds_1": 6534, "mds_2": 6053 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:00:43.882 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:44.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.022+0000 7f28c9800640 1 Processor -- start 2024-03-04T08:00:44.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.022+0000 7f28c9800640 1 -- start start 2024-03-04T08:00:44.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c9800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f28c4152bd0 0x7f28c4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:44.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c9800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c41591f0 con 0x7f28c41527f0 2024-03-04T08:00:44.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c9800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c41594d0 con 0x7f28c40c8200 2024-03-04T08:00:44.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c9800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c4071df0 con 0x7f28c4152bd0 2024-03-04T08:00:44.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f28c41527f0 0x7f28c4072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49074/0 (socket says 172.21.15.113:49074) 2024-03-04T08:00:44.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2a00640 1 -- 172.21.15.113:0/409986193 learned_addr learned my addr 172.21.15.113:0/409986193 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:00:44.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c3400640 1 --2- 172.21.15.113:0/409986193 >> v2:172.21.15.136:3300/0 conn(0x7f28c4152bd0 0x7f28c4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:44.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931016004 0 0) 0x7f28c41591f0 con 0x7f28c41527f0 2024-03-04T08:00:44.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28a8003530 con 0x7f28c41527f0 2024-03-04T08:00:44.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2846514658 0 0) 0x7f28a8003530 con 0x7f28c41527f0 2024-03-04T08:00:44.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 >> v2:172.21.15.136:3300/0 conn(0x7f28c4152bd0 msgr2=0x7f28c4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:44.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 --2- 172.21.15.113:0/409986193 >> v2:172.21.15.136:3300/0 conn(0x7f28c4152bd0 0x7f28c4070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 >> v1:172.21.15.136:6789/0 conn(0x7f28c40c8200 legacy=0x7f28c41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:44.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c40717a0 con 0x7f28c41527f0 2024-03-04T08:00:44.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f28b4001680 con 0x7f28c41527f0 2024-03-04T08:00:44.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 >> v1:172.21.15.113:6789/0 conn(0x7f28c41527f0 legacy=0x7f28c4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:44.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28a8003a10 0x7f28a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:44.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8004810 0x7f28a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:44.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:44.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4071df0 con 0x7f28a80051d0 2024-03-04T08:00:44.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a800c0d0 con 0x7f28a8003a10 2024-03-04T08:00:44.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a800c2d0 con 0x7f28a8004810 2024-03-04T08:00:44.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c3400640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:44.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c2a00640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28a8003a10 0x7f28a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:44.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28bbe00640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8004810 0x7f28a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:44.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c3400640 1 -- 172.21.15.113:0/409986193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8004810 msgr2=0x7f28a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:44.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c3400640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28a8004810 0x7f28a8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c3400640 1 -- 172.21.15.113:0/409986193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28a8003a10 msgr2=0x7f28a8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:44.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c3400640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28a8003a10 0x7f28a8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.026+0000 7f28c3400640 1 -- 172.21.15.113:0/409986193 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a8003530 con 0x7f28a80051d0 2024-03-04T08:00:44.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c3400640 1 -- 172.21.15.113:0/409986193 reap_dead start 2024-03-04T08:00:44.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c3400640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28a800b9d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009fc0 tx=0x7f28bc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b1c4d4731f0d96 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:44.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc009c10 con 0x7f28a80051d0 2024-03-04T08:00:44.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc0a5cd0 con 0x7f28a80051d0 2024-03-04T08:00:44.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c2000640 1 -- 172.21.15.113:0/409986193 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc003be0 con 0x7f28a80051d0 2024-03-04T08:00:44.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c9800640 1 -- 172.21.15.113:0/409986193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 msgr2=0x7f28a800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:44.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c9800640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28a800b9d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009fc0 tx=0x7f28bc01ccc0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c9800640 1 -- 172.21.15.113:0/409986193 shutdown_connections 2024-03-04T08:00:44.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c9800640 1 --2- 172.21.15.113:0/409986193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28a800b9d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c9800640 1 -- 172.21.15.113:0/409986193 >> 172.21.15.113:0/409986193 conn(0x7f28c406e820 msgr2=0x7f28c406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:44.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c9800640 1 -- 172.21.15.113:0/409986193 shutdown_connections 2024-03-04T08:00:44.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c9800640 1 -- 172.21.15.113:0/409986193 wait complete. 2024-03-04T08:00:44.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c9800640 1 Processor -- start 2024-03-04T08:00:44.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.030+0000 7f28c9800640 1 -- start start 2024-03-04T08:00:44.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28c414ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:44.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c9800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28c4153e00 0x7f28c4155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:44.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c9800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28c4155dd0 0x7f28c41f0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:44.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c9800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4070d40 con 0x7f28a80051d0 2024-03-04T08:00:44.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c9800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c41597b0 con 0x7f28c4153e00 2024-03-04T08:00:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c9800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c4070ec0 con 0x7f28c4155dd0 2024-03-04T08:00:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28c414ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28c414ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35230/0 (socket says 172.21.15.113:35230) 2024-03-04T08:00:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c2a00640 1 -- 172.21.15.113:0/1410536361 learned_addr learned my addr 172.21.15.113:0/1410536361 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c3400640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28c4155dd0 0x7f28c41f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28bbe00640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28c4153e00 0x7f28c4155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c2a00640 1 -- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28c4155dd0 msgr2=0x7f28c41f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c2a00640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28c4155dd0 0x7f28c41f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c2a00640 1 -- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28c4153e00 msgr2=0x7f28c4155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c2a00640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28c4153e00 0x7f28c4155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c2a00640 1 -- 172.21.15.113:0/1410536361 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c4159a30 con 0x7f28a80051d0 2024-03-04T08:00:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28bbe00640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28c4153e00 0x7f28c4155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:00:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c2a00640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28c414ef30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f28b4004f30 tx=0x7f28b40074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28baa00640 1 -- 172.21.15.113:0/1410536361 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b401a070 con 0x7f28a80051d0 2024-03-04T08:00:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28baa00640 1 -- 172.21.15.113:0/1410536361 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b400f7b0 con 0x7f28a80051d0 2024-03-04T08:00:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28baa00640 1 -- 172.21.15.113:0/1410536361 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b400fb80 con 0x7f28a80051d0 2024-03-04T08:00:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c9800640 1 -- 172.21.15.113:0/1410536361 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c4159d10 con 0x7f28a80051d0 2024-03-04T08:00:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c9800640 1 -- 172.21.15.113:0/1410536361 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c40b14a0 con 0x7f28a80051d0 2024-03-04T08:00:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28c9800640 1 -- 172.21.15.113:0/1410536361 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c4075c00 con 0x7f28a80051d0 2024-03-04T08:00:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.034+0000 7f28baa00640 1 -- 172.21.15.113:0/1410536361 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28b401d070 con 0x7f28a80051d0 2024-03-04T08:00:44.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.038+0000 7f28baa00640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28a0066f60 0x7f28a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:44.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.038+0000 7f28baa00640 1 -- 172.21.15.113:0/1410536361 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f28b4085100 con 0x7f28a80051d0 2024-03-04T08:00:44.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.038+0000 7f28bbe00640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28a0066f60 0x7f28a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:44.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.038+0000 7f28bbe00640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28a0066f60 0x7f28a0069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f28c40c3370 tx=0x7f28ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:44.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.046+0000 7f28baa00640 1 -- 172.21.15.113:0/1410536361 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f28b4055b40 con 0x7f28a80051d0 2024-03-04T08:00:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.366+0000 7f28c9800640 1 -- 172.21.15.113:0/1410536361 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28c40c79d0 con 0x7f28a80051d0 2024-03-04T08:00:44.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.366+0000 7f28baa00640 1 -- 172.21.15.113:0/1410536361 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+5818 (secure 0 0 0) 0x7f28b4055440 con 0x7f28a80051d0 2024-03-04T08:00:44.366 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:44.366 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":95,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:00:40.798422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:00:44.366 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 95 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 -- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28a0066f60 msgr2=0x7f28a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28a0066f60 0x7f28a0069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f28c40c3370 tx=0x7f28ac009040 comp rx=0 tx=0).stop 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 -- 172.21.15.113:0/1410536361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 msgr2=0x7f28c414ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28c414ef30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f28b4004f30 tx=0x7f28b40074f0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 -- 172.21.15.113:0/1410536361 shutdown_connections 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28c4155dd0 0x7f28c41f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28c4153e00 0x7f28c4155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28a0066f60 0x7f28a0069420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 --2- 172.21.15.113:0/1410536361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28a80051d0 0x7f28c414ef30 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 -- 172.21.15.113:0/1410536361 >> 172.21.15.113:0/1410536361 conn(0x7f28c406e820 msgr2=0x7f28c40b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:44.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 -- 172.21.15.113:0/1410536361 shutdown_connections 2024-03-04T08:00:44.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:44.370+0000 7f28c9800640 1 -- 172.21.15.113:0/1410536361 wait complete. 2024-03-04T08:00:44.398 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:00:46.399 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:46.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7119a00640 1 Processor -- start 2024-03-04T08:00:46.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7119a00640 1 -- start start 2024-03-04T08:00:46.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7119a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7114152950 0x7f711415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7119a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f711406faa0 0x7f71140c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7119a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f711406ed80 0x7f711406f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7119a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71140c8370 con 0x7f711406faa0 2024-03-04T08:00:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7119a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71140ca0a0 con 0x7f711406ed80 2024-03-04T08:00:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7119a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71140c4ce0 con 0x7f7114152950 2024-03-04T08:00:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7113400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f711406faa0 0x7f71140c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7113e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7114152950 0x7f711415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7113400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f711406faa0 0x7f71140c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35242/0 (socket says 172.21.15.113:35242) 2024-03-04T08:00:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7113400640 1 -- 172.21.15.113:0/1186443497 learned_addr learned my addr 172.21.15.113:0/1186443497 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:46.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7119000640 1 --2- 172.21.15.113:0/1186443497 >> v2:172.21.15.136:3300/0 conn(0x7f711406ed80 0x7f711406f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:46.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7113400640 1 -- 172.21.15.113:0/1186443497 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7114152950 msgr2=0x7f711415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:46.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7113400640 1 --2- 172.21.15.113:0/1186443497 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7114152950 0x7f711415bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7113400640 1 -- 172.21.15.113:0/1186443497 >> v2:172.21.15.136:3300/0 conn(0x7f711406ed80 msgr2=0x7f711406f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:46.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7113400640 1 --2- 172.21.15.113:0/1186443497 >> v2:172.21.15.136:3300/0 conn(0x7f711406ed80 0x7f711406f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.546+0000 7f7113400640 1 -- 172.21.15.113:0/1186443497 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71140c4f80 con 0x7f711406faa0 2024-03-04T08:00:46.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7113400640 1 --2- 172.21.15.113:0/1186443497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 0x7f71140c30b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f71140c9510 tx=0x7f710001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ee843050c0a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:46.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7112a00640 1 -- 172.21.15.113:0/1186443497 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71000b5070 con 0x7f711406faa0 2024-03-04T08:00:46.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7112a00640 1 -- 172.21.15.113:0/1186443497 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71000a89d0 con 0x7f711406faa0 2024-03-04T08:00:46.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7112a00640 1 -- 172.21.15.113:0/1186443497 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71000a8da0 con 0x7f711406faa0 2024-03-04T08:00:46.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7119a00640 1 -- 172.21.15.113:0/1186443497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 msgr2=0x7f71140c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:46.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7119a00640 1 --2- 172.21.15.113:0/1186443497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 0x7f71140c30b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f71140c9510 tx=0x7f710001ed10 comp rx=0 tx=0).stop 2024-03-04T08:00:46.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7119a00640 1 -- 172.21.15.113:0/1186443497 shutdown_connections 2024-03-04T08:00:46.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7119a00640 1 --2- 172.21.15.113:0/1186443497 >> v2:172.21.15.136:3300/0 conn(0x7f711406ed80 0x7f711406f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7119a00640 1 --2- 172.21.15.113:0/1186443497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 0x7f71140c30b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7119a00640 1 --2- 172.21.15.113:0/1186443497 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7114152950 0x7f711415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7119a00640 1 -- 172.21.15.113:0/1186443497 >> 172.21.15.113:0/1186443497 conn(0x7f7114072f40 msgr2=0x7f71140706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:46.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7119a00640 1 -- 172.21.15.113:0/1186443497 shutdown_connections 2024-03-04T08:00:46.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.550+0000 7f7119a00640 1 -- 172.21.15.113:0/1186443497 wait complete. 2024-03-04T08:00:46.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 Processor -- start 2024-03-04T08:00:46.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 -- start start 2024-03-04T08:00:46.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711406ed80 0x7f71141f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:46.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 0x7f7114153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:46.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7114152950 0x7f71140c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:46.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71140c9a50 con 0x7f711406faa0 2024-03-04T08:00:46.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711406fe80 con 0x7f711406ed80 2024-03-04T08:00:46.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71140c9d30 con 0x7f7114152950 2024-03-04T08:00:46.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 0x7f7114153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:46.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 0x7f7114153470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35254/0 (socket says 172.21.15.113:35254) 2024-03-04T08:00:46.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113400640 1 -- 172.21.15.113:0/2134636868 learned_addr learned my addr 172.21.15.113:0/2134636868 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:46.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113e00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711406ed80 0x7f71141f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119000640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7114152950 0x7f71140c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113400640 1 -- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7114152950 msgr2=0x7f71140c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113400640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7114152950 0x7f71140c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113400640 1 -- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711406ed80 msgr2=0x7f71141f0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113400640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711406ed80 0x7f71141f0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113400640 1 -- 172.21.15.113:0/2134636868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71140cd2d0 con 0x7f711406faa0 2024-03-04T08:00:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113e00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711406ed80 0x7f71141f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:00:46.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7113400640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 0x7f7114153470 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f710001d3c0 tx=0x7f710001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:46.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f710be00640 1 -- 172.21.15.113:0/2134636868 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71000b5070 con 0x7f711406faa0 2024-03-04T08:00:46.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f710be00640 1 -- 172.21.15.113:0/2134636868 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71000b0950 con 0x7f711406faa0 2024-03-04T08:00:46.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 -- 172.21.15.113:0/2134636868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71140cd5b0 con 0x7f711406faa0 2024-03-04T08:00:46.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f710be00640 1 -- 172.21.15.113:0/2134636868 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71000b0d40 con 0x7f711406faa0 2024-03-04T08:00:46.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 -- 172.21.15.113:0/2134636868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71140c5610 con 0x7f711406faa0 2024-03-04T08:00:46.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.554+0000 7f7119a00640 1 -- 172.21.15.113:0/2134636868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7114075a80 con 0x7f711406faa0 2024-03-04T08:00:46.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.558+0000 7f710be00640 1 -- 172.21.15.113:0/2134636868 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71000c2060 con 0x7f711406faa0 2024-03-04T08:00:46.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.558+0000 7f710be00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70f4066f60 0x7f70f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:46.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.558+0000 7f710be00640 1 -- 172.21.15.113:0/2134636868 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f7100122630 con 0x7f711406faa0 2024-03-04T08:00:46.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.558+0000 7f7113e00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70f4066f60 0x7f70f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:46.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.558+0000 7f7113e00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70f4066f60 0x7f70f4069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7104009bb0 tx=0x7f7104008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:46.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.566+0000 7f710be00640 1 -- 172.21.15.113:0/2134636868 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7100018870 con 0x7f711406faa0 2024-03-04T08:00:46.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 -- 172.21.15.113:0/2134636868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71140c9580 con 0x7f711406faa0 2024-03-04T08:00:46.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f710be00640 1 -- 172.21.15.113:0/2134636868 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5820 (secure 0 0 0) 0x7f71000f3070 con 0x7f711406faa0 2024-03-04T08:00:46.764 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:46.764 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:00:45.811821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:resolve","state_seq":24,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372]},"id":1}]} 2024-03-04T08:00:46.764 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 96 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 -- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70f4066f60 msgr2=0x7f70f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70f4066f60 0x7f70f4069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7104009bb0 tx=0x7f7104008040 comp rx=0 tx=0).stop 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 -- 172.21.15.113:0/2134636868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 msgr2=0x7f7114153470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 0x7f7114153470 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f710001d3c0 tx=0x7f710001bab0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 -- 172.21.15.113:0/2134636868 shutdown_connections 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7114152950 0x7f71140c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70f4066f60 0x7f70f4069420 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f711406faa0 0x7f7114153470 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 --2- 172.21.15.113:0/2134636868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711406ed80 0x7f71141f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 -- 172.21.15.113:0/2134636868 >> 172.21.15.113:0/2134636868 conn(0x7f7114072f40 msgr2=0x7f71140ca380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:46.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 -- 172.21.15.113:0/2134636868 shutdown_connections 2024-03-04T08:00:46.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:46.766+0000 7f7119a00640 1 -- 172.21.15.113:0/2134636868 wait complete. 2024-03-04T08:00:46.786 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:00:48.787 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:48.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe89800640 1 Processor -- start 2024-03-04T08:00:48.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe89800640 1 -- start start 2024-03-04T08:00:48.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe89800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbe8414e4c0 0x7fbe84072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:48.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe89800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe840c22e0 0x7fbe840731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:48.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe89800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe840c27f0 con 0x7fbe840c2d80 2024-03-04T08:00:48.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe89800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8415ad60 con 0x7fbe840c22e0 2024-03-04T08:00:48.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe89800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840c2a90 con 0x7fbe8414e4c0 2024-03-04T08:00:48.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe7a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe840c22e0 0x7fbe840731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:48.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe7a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe840c22e0 0x7fbe840731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55064/0 (socket says 172.21.15.113:55064) 2024-03-04T08:00:48.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe7a000640 1 -- 172.21.15.113:0/1152383557 learned_addr learned my addr 172.21.15.113:0/1152383557 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:48.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe82a00640 1 --2- 172.21.15.113:0/1152383557 >> v2:172.21.15.136:3300/0 conn(0x7fbe8414e4c0 0x7fbe84072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:48.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe7a000640 1 -- 172.21.15.113:0/1152383557 >> v2:172.21.15.136:3300/0 conn(0x7fbe8414e4c0 msgr2=0x7fbe84072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:48.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe7a000640 1 --2- 172.21.15.113:0/1152383557 >> v2:172.21.15.136:3300/0 conn(0x7fbe8414e4c0 0x7fbe84072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:48.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe7a000640 1 -- 172.21.15.113:0/1152383557 >> v1:172.21.15.136:6789/0 conn(0x7fbe840c2d80 legacy=0x7fbe8415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:48.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe7a000640 1 -- 172.21.15.113:0/1152383557 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe84070c60 con 0x7fbe840c22e0 2024-03-04T08:00:48.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.938+0000 7fbe82a00640 1 --2- 172.21.15.113:0/1152383557 >> v2:172.21.15.136:3300/0 conn(0x7fbe8414e4c0 0x7fbe84072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:00:48.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.942+0000 7fbe7a000640 1 --2- 172.21.15.113:0/1152383557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe840c22e0 0x7fbe840731c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbe8406f690 tx=0x7fbe6c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5419dfe20d0cba31 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:48.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.942+0000 7fbe82000640 1 -- 172.21.15.113:0/1152383557 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c0b5070 con 0x7fbe840c22e0 2024-03-04T08:00:48.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.942+0000 7fbe82000640 1 -- 172.21.15.113:0/1152383557 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c0a8a10 con 0x7fbe840c22e0 2024-03-04T08:00:48.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.942+0000 7fbe82000640 1 -- 172.21.15.113:0/1152383557 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c0a8de0 con 0x7fbe840c22e0 2024-03-04T08:00:48.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.946+0000 7fbe89800640 1 -- 172.21.15.113:0/1152383557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe840c22e0 msgr2=0x7fbe840731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:48.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.946+0000 7fbe89800640 1 --2- 172.21.15.113:0/1152383557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe840c22e0 0x7fbe840731c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbe8406f690 tx=0x7fbe6c01ed10 comp rx=0 tx=0).stop 2024-03-04T08:00:48.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.946+0000 7fbe89800640 1 -- 172.21.15.113:0/1152383557 shutdown_connections 2024-03-04T08:00:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.946+0000 7fbe89800640 1 --2- 172.21.15.113:0/1152383557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe840c22e0 0x7fbe840731c0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.946+0000 7fbe89800640 1 --2- 172.21.15.113:0/1152383557 >> v2:172.21.15.136:3300/0 conn(0x7fbe8414e4c0 0x7fbe84072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.946+0000 7fbe89800640 1 -- 172.21.15.113:0/1152383557 >> 172.21.15.113:0/1152383557 conn(0x7fbe8406e820 msgr2=0x7fbe8406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.946+0000 7fbe89800640 1 -- 172.21.15.113:0/1152383557 shutdown_connections 2024-03-04T08:00:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.946+0000 7fbe89800640 1 -- 172.21.15.113:0/1152383557 wait complete. 2024-03-04T08:00:48.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.946+0000 7fbe89800640 1 Processor -- start 2024-03-04T08:00:48.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.946+0000 7fbe89800640 1 -- start start 2024-03-04T08:00:48.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe89800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe840c22e0 0x7fbe84152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:48.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe89800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe840c2d80 0x7fbe840c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe89800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8414e4c0 0x7fbe840c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe89800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84071500 con 0x7fbe8414e4c0 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe89800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84071220 con 0x7fbe840c2d80 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe89800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8414e8a0 con 0x7fbe840c22e0 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe83400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8414e4c0 0x7fbe840c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe83400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8414e4c0 0x7fbe840c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55070/0 (socket says 172.21.15.113:55070) 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe83400640 1 -- 172.21.15.113:0/1381600184 learned_addr learned my addr 172.21.15.113:0/1381600184 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe82a00640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe840c22e0 0x7fbe84152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe7a000640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe840c2d80 0x7fbe840c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe83400640 1 -- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe840c22e0 msgr2=0x7fbe84152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe83400640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe840c22e0 0x7fbe84152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe83400640 1 -- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe840c2d80 msgr2=0x7fbe840c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe83400640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe840c2d80 0x7fbe840c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:48.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe83400640 1 -- 172.21.15.113:0/1381600184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840cbb00 con 0x7fbe8414e4c0 2024-03-04T08:00:48.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe83400640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8414e4c0 0x7fbe840c4a30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00c370 tx=0x7fbe7c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:48.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe7b400640 1 -- 172.21.15.113:0/1381600184 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c005220 con 0x7fbe8414e4c0 2024-03-04T08:00:48.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe7b400640 1 -- 172.21.15.113:0/1381600184 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c0053c0 con 0x7fbe8414e4c0 2024-03-04T08:00:48.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe89800640 1 -- 172.21.15.113:0/1381600184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe840cbde0 con 0x7fbe8414e4c0 2024-03-04T08:00:48.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe7b400640 1 -- 172.21.15.113:0/1381600184 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c007270 con 0x7fbe8414e4c0 2024-03-04T08:00:48.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe89800640 1 -- 172.21.15.113:0/1381600184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe84154190 con 0x7fbe8414e4c0 2024-03-04T08:00:48.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.950+0000 7fbe89800640 1 -- 172.21.15.113:0/1381600184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe84075c00 con 0x7fbe8414e4c0 2024-03-04T08:00:48.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.954+0000 7fbe7b400640 1 -- 172.21.15.113:0/1381600184 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe7c019020 con 0x7fbe8414e4c0 2024-03-04T08:00:48.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.954+0000 7fbe7b400640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe60066f60 0x7fbe60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:48.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.954+0000 7fbe7b400640 1 -- 172.21.15.113:0/1381600184 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7fbe7c084730 con 0x7fbe8414e4c0 2024-03-04T08:00:48.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.954+0000 7fbe82a00640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe60066f60 0x7fbe60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:48.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.954+0000 7fbe82a00640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe60066f60 0x7fbe60069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009fd0 tx=0x7fbe74009870 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:48.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:48.958+0000 7fbe7b400640 1 -- 172.21.15.113:0/1381600184 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbe7c054bb0 con 0x7fbe8414e4c0 2024-03-04T08:00:49.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.154+0000 7fbe89800640 1 -- 172.21.15.113:0/1381600184 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe84155190 con 0x7fbe8414e4c0 2024-03-04T08:00:49.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.154+0000 7fbe7b400640 1 -- 172.21.15.113:0/1381600184 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5820 (secure 0 0 0) 0x7fbe7c058c50 con 0x7fbe8414e4c0 2024-03-04T08:00:49.154 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:49.154 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:00:45.811821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:resolve","state_seq":24,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372]},"id":1}]} 2024-03-04T08:00:49.154 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 96 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 -- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe60066f60 msgr2=0x7fbe60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe60066f60 0x7fbe60069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009fd0 tx=0x7fbe74009870 comp rx=0 tx=0).stop 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 -- 172.21.15.113:0/1381600184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8414e4c0 msgr2=0x7fbe840c4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8414e4c0 0x7fbe840c4a30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00c370 tx=0x7fbe7c00c830 comp rx=0 tx=0).stop 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 -- 172.21.15.113:0/1381600184 shutdown_connections 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe60066f60 0x7fbe60069420 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe8414e4c0 0x7fbe840c4a30 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe840c2d80 0x7fbe840c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 --2- 172.21.15.113:0/1381600184 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe840c22e0 0x7fbe84152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 -- 172.21.15.113:0/1381600184 >> 172.21.15.113:0/1381600184 conn(0x7fbe8406e820 msgr2=0x7fbe8414ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 -- 172.21.15.113:0/1381600184 shutdown_connections 2024-03-04T08:00:49.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:49.158+0000 7fbe89800640 1 -- 172.21.15.113:0/1381600184 wait complete. 2024-03-04T08:00:49.176 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:00:51.177 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:51.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f1152400640 1 Processor -- start 2024-03-04T08:00:51.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f1152400640 1 -- start start 2024-03-04T08:00:51.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f1152400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114c154c00 0x7f114c0c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:51.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f1152400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f114c154fe0 0x7f114c0c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:51.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f1152400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114c0c7d80 con 0x7f114c0c7700 2024-03-04T08:00:51.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f1152400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c15efb0 con 0x7f114c154fe0 2024-03-04T08:00:51.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f1152400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0c6b70 con 0x7f114c154c00 2024-03-04T08:00:51.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f1143e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f114c154fe0 0x7f114c0c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:51.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f1143e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f114c154fe0 0x7f114c0c1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:41606/0 (socket says 172.21.15.113:41606) 2024-03-04T08:00:51.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f1143e00640 1 -- 172.21.15.113:0/2291705269 learned_addr learned my addr 172.21.15.113:0/2291705269 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:51.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f114be00640 1 --1- 172.21.15.113:0/2291705269 >> v1:172.21.15.136:6789/0 conn(0x7f114c0c7700 0x7f114c15abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:50210/0 (socket says 172.21.15.113:50210) 2024-03-04T08:00:51.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.330+0000 7f114b400640 1 --2- 172.21.15.113:0/2291705269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114c154c00 0x7f114c0c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:51.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.334+0000 7f1143e00640 1 -- 172.21.15.113:0/2291705269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114c154c00 msgr2=0x7f114c0c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:51.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.334+0000 7f1143e00640 1 --2- 172.21.15.113:0/2291705269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114c154c00 0x7f114c0c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.334+0000 7f1143e00640 1 -- 172.21.15.113:0/2291705269 >> v1:172.21.15.136:6789/0 conn(0x7f114c0c7700 legacy=0x7f114c15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:51.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.334+0000 7f1143e00640 1 -- 172.21.15.113:0/2291705269 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c0c8060 con 0x7f114c154fe0 2024-03-04T08:00:51.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.334+0000 7f1143e00640 1 --2- 172.21.15.113:0/2291705269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f114c154fe0 0x7f114c0c1b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1134009870 tx=0x7f113401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d354269e26196cee server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:51.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.334+0000 7f114aa00640 1 -- 172.21.15.113:0/2291705269 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11340b1070 con 0x7f114c154fe0 2024-03-04T08:00:51.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.334+0000 7f114aa00640 1 -- 172.21.15.113:0/2291705269 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11340ac620 con 0x7f114c154fe0 2024-03-04T08:00:51.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.334+0000 7f114aa00640 1 -- 172.21.15.113:0/2291705269 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11340ac9f0 con 0x7f114c154fe0 2024-03-04T08:00:51.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.338+0000 7f1152400640 1 -- 172.21.15.113:0/2291705269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f114c154fe0 msgr2=0x7f114c0c1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:51.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.338+0000 7f1152400640 1 --2- 172.21.15.113:0/2291705269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f114c154fe0 0x7f114c0c1b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1134009870 tx=0x7f113401c4d0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.338+0000 7f1152400640 1 -- 172.21.15.113:0/2291705269 shutdown_connections 2024-03-04T08:00:51.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.338+0000 7f1152400640 1 --2- 172.21.15.113:0/2291705269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f114c154fe0 0x7f114c0c1b50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.338+0000 7f1152400640 1 --2- 172.21.15.113:0/2291705269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114c154c00 0x7f114c0c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.338+0000 7f1152400640 1 -- 172.21.15.113:0/2291705269 >> 172.21.15.113:0/2291705269 conn(0x7f114c06e820 msgr2=0x7f114c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:51.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.338+0000 7f1152400640 1 -- 172.21.15.113:0/2291705269 shutdown_connections 2024-03-04T08:00:51.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.338+0000 7f1152400640 1 -- 172.21.15.113:0/2291705269 wait complete. 2024-03-04T08:00:51.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.338+0000 7f1152400640 1 Processor -- start 2024-03-04T08:00:51.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.338+0000 7f1152400640 1 -- start start 2024-03-04T08:00:51.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1152400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f114c0c7700 0x7f114c1f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:51.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1152400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114c154c00 0x7f114c149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:51.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1152400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f114c154fe0 0x7f114c15cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1152400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0c2460 con 0x7f114c154fe0 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1152400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0c22e0 con 0x7f114c0c7700 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1152400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c167c10 con 0x7f114c154c00 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f114be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f114c154fe0 0x7f114c15cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f114be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f114c154fe0 0x7f114c15cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55074/0 (socket says 172.21.15.113:55074) 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f114be00640 1 -- 172.21.15.113:0/1867372409 learned_addr learned my addr 172.21.15.113:0/1867372409 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f114b400640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f114c0c7700 0x7f114c1f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1143e00640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114c154c00 0x7f114c149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f114be00640 1 -- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114c154c00 msgr2=0x7f114c149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f114be00640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114c154c00 0x7f114c149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f114be00640 1 -- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f114c0c7700 msgr2=0x7f114c1f7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f114be00640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f114c0c7700 0x7f114c1f7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f114be00640 1 -- 172.21.15.113:0/1867372409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c148750 con 0x7f114c154fe0 2024-03-04T08:00:51.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f114be00640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f114c154fe0 0x7f114c15cfc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f114400c700 tx=0x7f114400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:51.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1143400640 1 -- 172.21.15.113:0/1867372409 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144017070 con 0x7f114c154fe0 2024-03-04T08:00:51.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1143400640 1 -- 172.21.15.113:0/1867372409 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114400ce10 con 0x7f114c154fe0 2024-03-04T08:00:51.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1152400640 1 -- 172.21.15.113:0/1867372409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c148a30 con 0x7f114c154fe0 2024-03-04T08:00:51.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1143400640 1 -- 172.21.15.113:0/1867372409 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114400a840 con 0x7f114c154fe0 2024-03-04T08:00:51.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.342+0000 7f1152400640 1 -- 172.21.15.113:0/1867372409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c148ff0 con 0x7f114c154fe0 2024-03-04T08:00:51.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.346+0000 7f1152400640 1 -- 172.21.15.113:0/1867372409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c075c00 con 0x7f114c154fe0 2024-03-04T08:00:51.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.346+0000 7f1143400640 1 -- 172.21.15.113:0/1867372409 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f114401a070 con 0x7f114c154fe0 2024-03-04T08:00:51.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.346+0000 7f1143400640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1128066f60 0x7f1128069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:51.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.346+0000 7f1143400640 1 -- 172.21.15.113:0/1867372409 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f1144084a40 con 0x7f114c154fe0 2024-03-04T08:00:51.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.346+0000 7f114b400640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1128066f60 0x7f1128069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:51.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.346+0000 7f114b400640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1128066f60 0x7f1128069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f113c009ac0 tx=0x7f113c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:51.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.354+0000 7f1143400640 1 -- 172.21.15.113:0/1867372409 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1144054ec0 con 0x7f114c154fe0 2024-03-04T08:00:51.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.666+0000 7f1152400640 1 -- 172.21.15.113:0/1867372409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f114c0c74c0 con 0x7f114c154fe0 2024-03-04T08:00:51.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1143400640 1 -- 172.21.15.113:0/1867372409 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+5822 (secure 0 0 0) 0x7f1144058f60 con 0x7f114c154fe0 2024-03-04T08:00:51.667 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:51.667 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:00:51.100885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:00:51.667 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 97 2024-03-04T08:00:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 -- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1128066f60 msgr2=0x7f1128069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1128066f60 0x7f1128069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f113c009ac0 tx=0x7f113c009190 comp rx=0 tx=0).stop 2024-03-04T08:00:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 -- 172.21.15.113:0/1867372409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f114c154fe0 msgr2=0x7f114c15cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f114c154fe0 0x7f114c15cfc0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f114400c700 tx=0x7f114400cbc0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 -- 172.21.15.113:0/1867372409 shutdown_connections 2024-03-04T08:00:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1128066f60 0x7f1128069420 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f114c154fe0 0x7f114c15cfc0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114c154c00 0x7f114c149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 --2- 172.21.15.113:0/1867372409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f114c0c7700 0x7f114c1f7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 -- 172.21.15.113:0/1867372409 >> 172.21.15.113:0/1867372409 conn(0x7f114c06e820 msgr2=0x7f114c150d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 -- 172.21.15.113:0/1867372409 shutdown_connections 2024-03-04T08:00:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:51.670+0000 7f1152400640 1 -- 172.21.15.113:0/1867372409 wait complete. 2024-03-04T08:00:51.693 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:00:53.695 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:53.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.838+0000 7f938de00640 1 Processor -- start 2024-03-04T08:00:53.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.838+0000 7f938de00640 1 -- start start 2024-03-04T08:00:53.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f938de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93881527f0 0x7f9388070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:53.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f938de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93880c8200 0x7f9388072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:53.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f938de00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93881591f0 con 0x7f9388152bd0 2024-03-04T08:00:53.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f938de00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9388071df0 con 0x7f93880c8200 2024-03-04T08:00:53.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f938de00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93881597b0 con 0x7f93881527f0 2024-03-04T08:00:53.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f937e000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93880c8200 0x7f9388072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:53.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f937e000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93880c8200 0x7f9388072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55090/0 (socket says 172.21.15.113:55090) 2024-03-04T08:00:53.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f937e000640 1 -- 172.21.15.113:0/3724432588 learned_addr learned my addr 172.21.15.113:0/3724432588 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:53.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f9387400640 1 --1- 172.21.15.113:0/3724432588 >> v1:172.21.15.113:6789/0 conn(0x7f9388152bd0 0x7f93881587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37814/0 (socket says 172.21.15.113:37814) 2024-03-04T08:00:53.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f9386a00640 1 --2- 172.21.15.113:0/3724432588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93881527f0 0x7f9388070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:53.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f937e000640 1 -- 172.21.15.113:0/3724432588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93881527f0 msgr2=0x7f9388070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:53.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f937e000640 1 --2- 172.21.15.113:0/3724432588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93881527f0 0x7f9388070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:53.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f937e000640 1 -- 172.21.15.113:0/3724432588 >> v1:172.21.15.113:6789/0 conn(0x7f9388152bd0 legacy=0x7f93881587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:53.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.842+0000 7f937e000640 1 -- 172.21.15.113:0/3724432588 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93881594d0 con 0x7f93880c8200 2024-03-04T08:00:53.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f937e000640 1 --2- 172.21.15.113:0/3724432588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93880c8200 0x7f9388072280 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9370009870 tx=0x7f937001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bbc5d87c85b793 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:53.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f9386000640 1 -- 172.21.15.113:0/3724432588 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93700a7070 con 0x7f93880c8200 2024-03-04T08:00:53.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f9386000640 1 -- 172.21.15.113:0/3724432588 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93700b0070 con 0x7f93880c8200 2024-03-04T08:00:53.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f9386000640 1 -- 172.21.15.113:0/3724432588 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93700ab890 con 0x7f93880c8200 2024-03-04T08:00:53.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f938de00640 1 -- 172.21.15.113:0/3724432588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93880c8200 msgr2=0x7f9388072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:53.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f938de00640 1 --2- 172.21.15.113:0/3724432588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93880c8200 0x7f9388072280 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9370009870 tx=0x7f937001c560 comp rx=0 tx=0).stop 2024-03-04T08:00:53.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f938de00640 1 -- 172.21.15.113:0/3724432588 shutdown_connections 2024-03-04T08:00:53.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f938de00640 1 --2- 172.21.15.113:0/3724432588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93880c8200 0x7f9388072280 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:53.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f938de00640 1 --2- 172.21.15.113:0/3724432588 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93881527f0 0x7f9388070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:53.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f938de00640 1 -- 172.21.15.113:0/3724432588 >> 172.21.15.113:0/3724432588 conn(0x7f938806e820 msgr2=0x7f938806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:53.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f938de00640 1 -- 172.21.15.113:0/3724432588 shutdown_connections 2024-03-04T08:00:53.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.846+0000 7f938de00640 1 -- 172.21.15.113:0/3724432588 wait complete. 2024-03-04T08:00:53.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f938de00640 1 Processor -- start 2024-03-04T08:00:53.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f938de00640 1 -- start start 2024-03-04T08:00:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f938de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93880c8200 0x7f93881f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f938de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93881527f0 0x7f93880b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f938de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9388152bd0 0x7f93881474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f938de00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9388165160 con 0x7f9388152bd0 2024-03-04T08:00:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f938de00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93880727c0 con 0x7f93880c8200 2024-03-04T08:00:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f938de00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93881652e0 con 0x7f93881527f0 2024-03-04T08:00:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f9387400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9388152bd0 0x7f93881474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f9387400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9388152bd0 0x7f93881474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55092/0 (socket says 172.21.15.113:55092) 2024-03-04T08:00:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f9387400640 1 -- 172.21.15.113:0/152357382 learned_addr learned my addr 172.21.15.113:0/152357382 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f937e000640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93881527f0 0x7f93880b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f9386a00640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93880c8200 0x7f93881f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f9387400640 1 -- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93881527f0 msgr2=0x7f93880b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f9387400640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93881527f0 0x7f93880b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f9387400640 1 -- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93880c8200 msgr2=0x7f93881f1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f9387400640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93880c8200 0x7f93881f1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f9387400640 1 -- 172.21.15.113:0/152357382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9388151ae0 con 0x7f9388152bd0 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f9387400640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9388152bd0 0x7f93881474d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f938000c730 tx=0x7f938000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f937fe00640 1 -- 172.21.15.113:0/152357382 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380017070 con 0x7f9388152bd0 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f937fe00640 1 -- 172.21.15.113:0/152357382 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938000ce40 con 0x7f9388152bd0 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f938de00640 1 -- 172.21.15.113:0/152357382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93881469f0 con 0x7f9388152bd0 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f937fe00640 1 -- 172.21.15.113:0/152357382 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938000a840 con 0x7f9388152bd0 2024-03-04T08:00:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.850+0000 7f938de00640 1 -- 172.21.15.113:0/152357382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9388146fb0 con 0x7f9388152bd0 2024-03-04T08:00:53.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.854+0000 7f938de00640 1 -- 172.21.15.113:0/152357382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9388075c00 con 0x7f9388152bd0 2024-03-04T08:00:53.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.854+0000 7f937fe00640 1 -- 172.21.15.113:0/152357382 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f938001a070 con 0x7f9388152bd0 2024-03-04T08:00:53.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.854+0000 7f937fe00640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9364066f90 0x7f9364069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:53.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.854+0000 7f937fe00640 1 -- 172.21.15.113:0/152357382 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f9380084a70 con 0x7f9388152bd0 2024-03-04T08:00:53.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.854+0000 7f9386a00640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9364066f90 0x7f9364069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:53.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.858+0000 7f9386a00640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9364066f90 0x7f9364069450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9378009b60 tx=0x7f9378009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:53.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:53.862+0000 7f937fe00640 1 -- 172.21.15.113:0/152357382 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9380054ef0 con 0x7f9388152bd0 2024-03-04T08:00:54.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.074+0000 7f938de00640 1 -- 172.21.15.113:0/152357382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93880c9760 con 0x7f9388152bd0 2024-03-04T08:00:54.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.074+0000 7f937fe00640 1 -- 172.21.15.113:0/152357382 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+5822 (secure 0 0 0) 0x7f9380058f90 con 0x7f9388152bd0 2024-03-04T08:00:54.073 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:54.073 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:00:51.100885+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:00:54.073 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 97 2024-03-04T08:00:54.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 -- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9364066f90 msgr2=0x7f9364069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:54.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9364066f90 0x7f9364069450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9378009b60 tx=0x7f9378009190 comp rx=0 tx=0).stop 2024-03-04T08:00:54.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 -- 172.21.15.113:0/152357382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9388152bd0 msgr2=0x7f93881474d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:54.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9388152bd0 0x7f93881474d0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f938000c730 tx=0x7f938000cbf0 comp rx=0 tx=0).stop 2024-03-04T08:00:54.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 -- 172.21.15.113:0/152357382 shutdown_connections 2024-03-04T08:00:54.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9364066f90 0x7f9364069450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:54.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9388152bd0 0x7f93881474d0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:54.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93881527f0 0x7f93880b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:54.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 --2- 172.21.15.113:0/152357382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93880c8200 0x7f93881f1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:54.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 -- 172.21.15.113:0/152357382 >> 172.21.15.113:0/152357382 conn(0x7f938806e820 msgr2=0x7f93881583d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:54.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 -- 172.21.15.113:0/152357382 shutdown_connections 2024-03-04T08:00:54.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:54.078+0000 7f938de00640 1 -- 172.21.15.113:0/152357382 wait complete. 2024-03-04T08:00:54.099 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 97, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 97, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6053": { "addr": "172.21.15.136:6835/3388011401", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 3388011401, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 3388011401, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6053, "incarnation": 95, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:reconnect", "state_seq": 26 }, "gid_6372": { "addr": "172.21.15.113:6833/2732565639", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2732565639, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2732565639, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6372, "incarnation": 83, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 40 }, "gid_6534": { "addr": "172.21.15.113:6837/4197507712", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 4197507712, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 4197507712, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6534, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 62, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:00:51.100885+0000", "qdb_cluster": [ 6372, 6534 ], "qdb_leader": 6372, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6372, "mds_1": 6534, "mds_2": 6053 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:00:56.100 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:56.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.242+0000 7f69cce00640 1 Processor -- start 2024-03-04T08:00:56.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.242+0000 7f69cce00640 1 -- start start 2024-03-04T08:00:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.242+0000 7f69cce00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69c80c8a90 0x7f69c815e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.242+0000 7f69cce00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f69c815ea60 0x7f69c80ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.242+0000 7f69cce00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c8070fd0 con 0x7f69c8157df0 2024-03-04T08:00:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.242+0000 7f69cce00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c815e090 con 0x7f69c80c8a90 2024-03-04T08:00:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.242+0000 7f69cce00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c806dfd0 con 0x7f69c815ea60 2024-03-04T08:00:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69c80c8a90 0x7f69c815e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69c80c8a90 0x7f69c815e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55094/0 (socket says 172.21.15.113:55094) 2024-03-04T08:00:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6a00640 1 -- 172.21.15.113:0/4082557836 learned_addr learned my addr 172.21.15.113:0/4082557836 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c7e00640 1 --2- 172.21.15.113:0/4082557836 >> v2:172.21.15.136:3300/0 conn(0x7f69c815ea60 0x7f69c80ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:56.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6a00640 1 -- 172.21.15.113:0/4082557836 >> v2:172.21.15.136:3300/0 conn(0x7f69c815ea60 msgr2=0x7f69c80ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:56.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6a00640 1 --2- 172.21.15.113:0/4082557836 >> v2:172.21.15.136:3300/0 conn(0x7f69c815ea60 0x7f69c80ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:56.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6a00640 1 -- 172.21.15.113:0/4082557836 >> v1:172.21.15.136:6789/0 conn(0x7f69c8157df0 legacy=0x7f69c806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:00:56.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6a00640 1 -- 172.21.15.113:0/4082557836 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c806fcd0 con 0x7f69c80c8a90 2024-03-04T08:00:56.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6a00640 1 --2- 172.21.15.113:0/4082557836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c80c8a90 0x7f69c815e520 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f69c80b37e0 tx=0x7f69b401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d03878b116a564 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:56.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6000640 1 -- 172.21.15.113:0/4082557836 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b40b0070 con 0x7f69c80c8a90 2024-03-04T08:00:56.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6000640 1 -- 172.21.15.113:0/4082557836 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b40ab600 con 0x7f69c80c8a90 2024-03-04T08:00:56.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.246+0000 7f69c6000640 1 -- 172.21.15.113:0/4082557836 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b40ab9d0 con 0x7f69c80c8a90 2024-03-04T08:00:56.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 -- 172.21.15.113:0/4082557836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c80c8a90 msgr2=0x7f69c815e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:56.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 --2- 172.21.15.113:0/4082557836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c80c8a90 0x7f69c815e520 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f69c80b37e0 tx=0x7f69b401c300 comp rx=0 tx=0).stop 2024-03-04T08:00:56.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 -- 172.21.15.113:0/4082557836 shutdown_connections 2024-03-04T08:00:56.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 --2- 172.21.15.113:0/4082557836 >> v2:172.21.15.136:3300/0 conn(0x7f69c815ea60 0x7f69c80ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:56.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 --2- 172.21.15.113:0/4082557836 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c80c8a90 0x7f69c815e520 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:56.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 -- 172.21.15.113:0/4082557836 >> 172.21.15.113:0/4082557836 conn(0x7f69c8072f40 msgr2=0x7f69c80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:56.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 -- 172.21.15.113:0/4082557836 shutdown_connections 2024-03-04T08:00:56.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 -- 172.21.15.113:0/4082557836 wait complete. 2024-03-04T08:00:56.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 Processor -- start 2024-03-04T08:00:56.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 -- start start 2024-03-04T08:00:56.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69c80c8a90 0x7f69c81f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:56.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c8157df0 0x7f69c81f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:56.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69c815ea60 0x7f69c814c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:56.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c806dd80 con 0x7f69c8157df0 2024-03-04T08:00:56.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c806dc00 con 0x7f69c815ea60 2024-03-04T08:00:56.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69cce00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c81581d0 con 0x7f69c80c8a90 2024-03-04T08:00:56.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69c6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c8157df0 0x7f69c81f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:56.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69c6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c8157df0 0x7f69c81f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55104/0 (socket says 172.21.15.113:55104) 2024-03-04T08:00:56.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69c6a00640 1 -- 172.21.15.113:0/564406166 learned_addr learned my addr 172.21.15.113:0/564406166 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:56.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69c7400640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69c80c8a90 0x7f69c81f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:56.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69c7e00640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69c815ea60 0x7f69c814c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:56.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69c6a00640 1 -- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69c80c8a90 msgr2=0x7f69c81f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:56.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69c6a00640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69c80c8a90 0x7f69c81f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:56.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69c6a00640 1 -- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69c815ea60 msgr2=0x7f69c814c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:56.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69c6a00640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69c815ea60 0x7f69c814c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:56.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.250+0000 7f69c6a00640 1 -- 172.21.15.113:0/564406166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c814cc00 con 0x7f69c8157df0 2024-03-04T08:00:56.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.254+0000 7f69c6a00640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c8157df0 0x7f69c81f4d40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f69b401a9b0 tx=0x7f69b401ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:56.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.254+0000 7f69bf400640 1 -- 172.21.15.113:0/564406166 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b40b0040 con 0x7f69c8157df0 2024-03-04T08:00:56.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.254+0000 7f69bf400640 1 -- 172.21.15.113:0/564406166 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b40bc070 con 0x7f69c8157df0 2024-03-04T08:00:56.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.254+0000 7f69bf400640 1 -- 172.21.15.113:0/564406166 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b40b6880 con 0x7f69c8157df0 2024-03-04T08:00:56.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.254+0000 7f69cce00640 1 -- 172.21.15.113:0/564406166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c81574d0 con 0x7f69c8157df0 2024-03-04T08:00:56.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.254+0000 7f69cce00640 1 -- 172.21.15.113:0/564406166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c81f3cc0 con 0x7f69c8157df0 2024-03-04T08:00:56.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.254+0000 7f69cce00640 1 -- 172.21.15.113:0/564406166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c8075a80 con 0x7f69c8157df0 2024-03-04T08:00:56.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.254+0000 7f69bf400640 1 -- 172.21.15.113:0/564406166 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69b40ab4c0 con 0x7f69c8157df0 2024-03-04T08:00:56.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.258+0000 7f69bf400640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69a8066f60 0x7f69a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:56.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.258+0000 7f69bf400640 1 -- 172.21.15.113:0/564406166 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f69b411f8f0 con 0x7f69c8157df0 2024-03-04T08:00:56.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.258+0000 7f69c7400640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69a8066f60 0x7f69a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:56.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.258+0000 7f69c7400640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69a8066f60 0x7f69a8069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f69b8007fd0 tx=0x7f69b8007e40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:56.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.262+0000 7f69bf400640 1 -- 172.21.15.113:0/564406166 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f69c8070fd0 con 0x7f69c8157df0 2024-03-04T08:00:56.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.470+0000 7f69cce00640 1 -- 172.21.15.113:0/564406166 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69c80c9610 con 0x7f69c8157df0 2024-03-04T08:00:56.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.470+0000 7f69bf400640 1 -- 172.21.15.113:0/564406166 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f69b40f03c0 con 0x7f69c8157df0 2024-03-04T08:00:56.470 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:56.471 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:00:56.119910+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372]},"id":1}]} 2024-03-04T08:00:56.471 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 98 2024-03-04T08:00:56.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 -- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69a8066f60 msgr2=0x7f69a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69a8066f60 0x7f69a8069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f69b8007fd0 tx=0x7f69b8007e40 comp rx=0 tx=0).stop 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 -- 172.21.15.113:0/564406166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c8157df0 msgr2=0x7f69c81f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c8157df0 0x7f69c81f4d40 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f69b401a9b0 tx=0x7f69b401ab40 comp rx=0 tx=0).stop 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 -- 172.21.15.113:0/564406166 shutdown_connections 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69c815ea60 0x7f69c814c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69a8066f60 0x7f69a8069420 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69c8157df0 0x7f69c81f4d40 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 --2- 172.21.15.113:0/564406166 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69c80c8a90 0x7f69c81f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 -- 172.21.15.113:0/564406166 >> 172.21.15.113:0/564406166 conn(0x7f69c8072f40 msgr2=0x7f69c80ca8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 -- 172.21.15.113:0/564406166 shutdown_connections 2024-03-04T08:00:56.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:56.474+0000 7f69cce00640 1 -- 172.21.15.113:0/564406166 wait complete. 2024-03-04T08:00:56.491 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:00:58.492 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:00:58.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.630+0000 7f0009a00640 1 Processor -- start 2024-03-04T08:00:58.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.630+0000 7f0009a00640 1 -- start start 2024-03-04T08:00:58.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0009a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f000406f340 0x7f0004154140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:58.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0009a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00040c1230 con 0x7f000415a2f0 2024-03-04T08:00:58.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0009a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00040c14d0 con 0x7f00040c8370 2024-03-04T08:00:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0009a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00040c17b0 con 0x7f000406f340 2024-03-04T08:00:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0002a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f000406f340 0x7f0004154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0003400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f000415a2f0 0x7f000415a740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44300/0 (socket says 172.21.15.113:44300) 2024-03-04T08:00:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0003400640 1 -- 172.21.15.113:0/2957998950 learned_addr learned my addr 172.21.15.113:0/2957998950 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:00:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0002a00640 1 -- 172.21.15.113:0/2957998950 >> v1:172.21.15.136:6789/0 conn(0x7f00040c8370 legacy=0x7f00040c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0002a00640 1 -- 172.21.15.113:0/2957998950 >> v1:172.21.15.113:6789/0 conn(0x7f000415a2f0 legacy=0x7f000415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0002a00640 1 -- 172.21.15.113:0/2957998950 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00040c1a90 con 0x7f000406f340 2024-03-04T08:00:58.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.634+0000 7f0002000640 1 -- 172.21.15.113:0/2957998950 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150535537 0 0) 0x7f00040c1230 con 0x7f000415a2f0 2024-03-04T08:00:58.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0002a00640 1 --2- 172.21.15.113:0/2957998950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 0x7f0004154140 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7efff400ba50 tx=0x7efff4021140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455b1ae5ab1a8494 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:58.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0002000640 1 -- 172.21.15.113:0/2957998950 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40b8070 con 0x7f000406f340 2024-03-04T08:00:58.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0002000640 1 -- 172.21.15.113:0/2957998950 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff40b3410 con 0x7f000406f340 2024-03-04T08:00:58.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0002000640 1 -- 172.21.15.113:0/2957998950 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40b37e0 con 0x7f000406f340 2024-03-04T08:00:58.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0009a00640 1 -- 172.21.15.113:0/2957998950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 msgr2=0x7f0004154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:58.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0009a00640 1 --2- 172.21.15.113:0/2957998950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 0x7f0004154140 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7efff400ba50 tx=0x7efff4021140 comp rx=0 tx=0).stop 2024-03-04T08:00:58.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0009a00640 1 -- 172.21.15.113:0/2957998950 shutdown_connections 2024-03-04T08:00:58.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0009a00640 1 --2- 172.21.15.113:0/2957998950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 0x7f0004154140 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:58.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0009a00640 1 -- 172.21.15.113:0/2957998950 >> 172.21.15.113:0/2957998950 conn(0x7f000406e820 msgr2=0x7f000406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:58.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0009a00640 1 -- 172.21.15.113:0/2957998950 shutdown_connections 2024-03-04T08:00:58.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.638+0000 7f0009a00640 1 -- 172.21.15.113:0/2957998950 wait complete. 2024-03-04T08:00:58.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0009a00640 1 Processor -- start 2024-03-04T08:00:58.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0009a00640 1 -- start start 2024-03-04T08:00:58.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0009a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 0x7f000414ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:58.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0009a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00040c8370 0x7f00040cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:58.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0009a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000415a2f0 0x7f00041ed780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:58.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0009a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00040c87e0 con 0x7f000406f340 2024-03-04T08:00:58.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0009a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00040729a0 con 0x7f000415a2f0 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0009a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00040c8960 con 0x7f00040c8370 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0002a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 0x7f000414ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0002a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 0x7f000414ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52038/0 (socket says 172.21.15.113:52038) 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0002a00640 1 -- 172.21.15.113:0/1336760047 learned_addr learned my addr 172.21.15.113:0/1336760047 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7efffbe00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00040c8370 0x7f00040cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0003400640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000415a2f0 0x7f00041ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0002a00640 1 -- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00040c8370 msgr2=0x7f00040cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0002a00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00040c8370 0x7f00040cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0002a00640 1 -- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000415a2f0 msgr2=0x7f00041ed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0002a00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000415a2f0 0x7f00041ed780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0002a00640 1 -- 172.21.15.113:0/1336760047 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00041510a0 con 0x7f000406f340 2024-03-04T08:00:58.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0002a00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 0x7f000414ba30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7efff400ba50 tx=0x7efff401dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:58.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7efffaa00640 1 -- 172.21.15.113:0/1336760047 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40b8040 con 0x7f000406f340 2024-03-04T08:00:58.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7efffaa00640 1 -- 172.21.15.113:0/1336760047 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff40be440 con 0x7f000406f340 2024-03-04T08:00:58.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7efffaa00640 1 -- 172.21.15.113:0/1336760047 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff40be810 con 0x7f000406f340 2024-03-04T08:00:58.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0009a00640 1 -- 172.21.15.113:0/1336760047 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00041f0460 con 0x7f000406f340 2024-03-04T08:00:58.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.642+0000 7f0009a00640 1 -- 172.21.15.113:0/1336760047 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000414e920 con 0x7f000406f340 2024-03-04T08:00:58.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.646+0000 7f0009a00640 1 -- 172.21.15.113:0/1336760047 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0004075c00 con 0x7f000406f340 2024-03-04T08:00:58.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.646+0000 7efffaa00640 1 -- 172.21.15.113:0/1336760047 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efff40c5020 con 0x7f000406f340 2024-03-04T08:00:58.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.646+0000 7efffaa00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7effe0066f90 0x7effe0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:00:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.646+0000 7efffaa00640 1 -- 172.21.15.113:0/1336760047 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7efff41257d0 con 0x7f000406f340 2024-03-04T08:00:58.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.646+0000 7efffbe00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7effe0066f90 0x7effe0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:00:58.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.646+0000 7efffbe00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7effe0066f90 0x7effe0069450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f00041554d0 tx=0x7effec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:00:58.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.654+0000 7efffaa00640 1 -- 172.21.15.113:0/1336760047 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efff40cc020 con 0x7f000406f340 2024-03-04T08:00:58.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.858+0000 7f0009a00640 1 -- 172.21.15.113:0/1336760047 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00040c65f0 con 0x7f000406f340 2024-03-04T08:00:58.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7efffaa00640 1 -- 172.21.15.113:0/1336760047 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+5819 (secure 0 0 0) 0x7efff401aca0 con 0x7f000406f340 2024-03-04T08:00:58.859 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:00:58.859 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:00:56.119910+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372]},"id":1}]} 2024-03-04T08:00:58.859 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 98 2024-03-04T08:00:58.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 -- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7effe0066f90 msgr2=0x7effe0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:58.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7effe0066f90 0x7effe0069450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f00041554d0 tx=0x7effec009040 comp rx=0 tx=0).stop 2024-03-04T08:00:58.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 -- 172.21.15.113:0/1336760047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 msgr2=0x7f000414ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:00:58.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 0x7f000414ba30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7efff400ba50 tx=0x7efff401dee0 comp rx=0 tx=0).stop 2024-03-04T08:00:58.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 -- 172.21.15.113:0/1336760047 shutdown_connections 2024-03-04T08:00:58.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f000415a2f0 0x7f00041ed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:58.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00040c8370 0x7f00040cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:58.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7effe0066f90 0x7effe0069450 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:58.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 --2- 172.21.15.113:0/1336760047 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f000406f340 0x7f000414ba30 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:00:58.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 -- 172.21.15.113:0/1336760047 >> 172.21.15.113:0/1336760047 conn(0x7f000406e820 msgr2=0x7f000406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:00:58.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 -- 172.21.15.113:0/1336760047 shutdown_connections 2024-03-04T08:00:58.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:00:58.862+0000 7f0009a00640 1 -- 172.21.15.113:0/1336760047 wait complete. 2024-03-04T08:00:58.889 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:01:00.890 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:01.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.030+0000 7f540a200640 1 Processor -- start 2024-03-04T08:01:01.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.030+0000 7f540a200640 1 -- start start 2024-03-04T08:01:01.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f540a200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5404160820 0x7f5404162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:01.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f540a200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54040c1ec0 con 0x7f54040c85f0 2024-03-04T08:01:01.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f540a200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54040c9b30 con 0x7f54040b20b0 2024-03-04T08:01:01.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f540a200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54040c12a0 con 0x7f5404160820 2024-03-04T08:01:01.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f53faa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f54040c85f0 0x7f540406b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44306/0 (socket says 172.21.15.113:44306) 2024-03-04T08:01:01.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f53faa00640 1 -- 172.21.15.113:0/2310660492 learned_addr learned my addr 172.21.15.113:0/2310660492 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:01.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f5403e00640 1 --2- 172.21.15.113:0/2310660492 >> v2:172.21.15.136:3300/0 conn(0x7f5404160820 0x7f5404162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:01.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f5402a00640 1 -- 172.21.15.113:0/2310660492 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097220913 0 0) 0x7f54040c1ec0 con 0x7f54040c85f0 2024-03-04T08:01:01.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f5402a00640 1 -- 172.21.15.113:0/2310660492 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53e8003450 con 0x7f54040c85f0 2024-03-04T08:01:01.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f5403e00640 1 -- 172.21.15.113:0/2310660492 >> v1:172.21.15.136:6789/0 conn(0x7f54040b20b0 legacy=0x7f5404157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:01.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f5402a00640 1 -- 172.21.15.113:0/2310660492 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2858132782 0 0) 0x7f53e8003450 con 0x7f54040c85f0 2024-03-04T08:01:01.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f5403e00640 1 -- 172.21.15.113:0/2310660492 >> v1:172.21.15.113:6789/0 conn(0x7f54040c85f0 legacy=0x7f540406b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:01.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.034+0000 7f5403e00640 1 -- 172.21.15.113:0/2310660492 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54040c1580 con 0x7f5404160820 2024-03-04T08:01:01.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.038+0000 7f5403e00640 1 --2- 172.21.15.113:0/2310660492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5404160820 0x7f5404162c10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0097c0 tx=0x7f53fc01ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=579fa620862a075f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:01.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.038+0000 7f5402a00640 1 -- 172.21.15.113:0/2310660492 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0b5070 con 0x7f5404160820 2024-03-04T08:01:01.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.038+0000 7f5402a00640 1 -- 172.21.15.113:0/2310660492 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc0a8630 con 0x7f5404160820 2024-03-04T08:01:01.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.038+0000 7f5402a00640 1 -- 172.21.15.113:0/2310660492 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec001a20 con 0x7f5404160820 2024-03-04T08:01:01.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.042+0000 7f540a200640 1 -- 172.21.15.113:0/2310660492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5404160820 msgr2=0x7f5404162c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:01.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.042+0000 7f540a200640 1 --2- 172.21.15.113:0/2310660492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5404160820 0x7f5404162c10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0097c0 tx=0x7f53fc01ebb0 comp rx=0 tx=0).stop 2024-03-04T08:01:01.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.042+0000 7f540a200640 1 -- 172.21.15.113:0/2310660492 shutdown_connections 2024-03-04T08:01:01.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.042+0000 7f540a200640 1 --2- 172.21.15.113:0/2310660492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5404160820 0x7f5404162c10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:01.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.042+0000 7f540a200640 1 -- 172.21.15.113:0/2310660492 >> 172.21.15.113:0/2310660492 conn(0x7f540406e820 msgr2=0x7f540406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:01.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.042+0000 7f540a200640 1 -- 172.21.15.113:0/2310660492 shutdown_connections 2024-03-04T08:01:01.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.042+0000 7f540a200640 1 -- 172.21.15.113:0/2310660492 wait complete. 2024-03-04T08:01:01.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.042+0000 7f540a200640 1 Processor -- start 2024-03-04T08:01:01.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.042+0000 7f540a200640 1 -- start start 2024-03-04T08:01:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f540a200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54040b20b0 0x7f54040cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f540a200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54040c85f0 0x7f54041f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f540a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5404160820 0x7f54041e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f540a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5404157e20 con 0x7f5404160820 2024-03-04T08:01:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f540a200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54041571f0 con 0x7f54040c85f0 2024-03-04T08:01:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f540a200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54040c2400 con 0x7f54040b20b0 2024-03-04T08:01:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f5403e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5404160820 0x7f54041e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f5403e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5404160820 0x7f54041e5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52046/0 (socket says 172.21.15.113:52046) 2024-03-04T08:01:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f5403e00640 1 -- 172.21.15.113:0/3212960713 learned_addr learned my addr 172.21.15.113:0/3212960713 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f53faa00640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54040c85f0 0x7f54041f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f5403400640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54040b20b0 0x7f54040cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f5403e00640 1 -- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54040b20b0 msgr2=0x7f54040cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f5403e00640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54040b20b0 0x7f54040cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f5403e00640 1 -- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54040c85f0 msgr2=0x7f54041f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f5403e00640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54040c85f0 0x7f54041f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f5403e00640 1 -- 172.21.15.113:0/3212960713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5404146650 con 0x7f5404160820 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f5403e00640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5404160820 0x7f54041e5e50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f53fc01c7e0 tx=0x7f53fc01b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f53fbe00640 1 -- 172.21.15.113:0/3212960713 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0b5040 con 0x7f5404160820 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f53fbe00640 1 -- 172.21.15.113:0/3212960713 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc0be070 con 0x7f5404160820 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f53fbe00640 1 -- 172.21.15.113:0/3212960713 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0a8be0 con 0x7f5404160820 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f540a200640 1 -- 172.21.15.113:0/3212960713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540414f8d0 con 0x7f5404160820 2024-03-04T08:01:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f540a200640 1 -- 172.21.15.113:0/3212960713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54040ca180 con 0x7f5404160820 2024-03-04T08:01:01.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.046+0000 7f540a200640 1 -- 172.21.15.113:0/3212960713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5404075c00 con 0x7f5404160820 2024-03-04T08:01:01.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.050+0000 7f53fbe00640 1 -- 172.21.15.113:0/3212960713 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53fc003620 con 0x7f5404160820 2024-03-04T08:01:01.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.050+0000 7f53fbe00640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53e0066f60 0x7f53e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:01.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.050+0000 7f53fbe00640 1 -- 172.21.15.113:0/3212960713 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f53fc1226e0 con 0x7f5404160820 2024-03-04T08:01:01.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.050+0000 7f5403400640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53e0066f60 0x7f53e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:01.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.050+0000 7f5403400640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53e0066f60 0x7f53e0069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f53f4007fd0 tx=0x7f53f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:01.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.054+0000 7f53fbe00640 1 -- 172.21.15.113:0/3212960713 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f53fc018710 con 0x7f5404160820 2024-03-04T08:01:01.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.274+0000 7f540a200640 1 -- 172.21.15.113:0/3212960713 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5404072f10 con 0x7f5404160820 2024-03-04T08:01:01.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.274+0000 7f53fbe00640 1 -- 172.21.15.113:0/3212960713 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+5819 (secure 0 0 0) 0x7f53fc0f2d00 con 0x7f5404160820 2024-03-04T08:01:01.274 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:01.274 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:00:56.119910+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372]},"id":1}]} 2024-03-04T08:01:01.274 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 98 2024-03-04T08:01:01.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 -- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53e0066f60 msgr2=0x7f53e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:01.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53e0066f60 0x7f53e0069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f53f4007fd0 tx=0x7f53f4009040 comp rx=0 tx=0).stop 2024-03-04T08:01:01.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 -- 172.21.15.113:0/3212960713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5404160820 msgr2=0x7f54041e5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:01.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5404160820 0x7f54041e5e50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f53fc01c7e0 tx=0x7f53fc01b950 comp rx=0 tx=0).stop 2024-03-04T08:01:01.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 -- 172.21.15.113:0/3212960713 shutdown_connections 2024-03-04T08:01:01.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53e0066f60 0x7f53e0069420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:01.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5404160820 0x7f54041e5e50 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:01.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54040c85f0 0x7f54041f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:01.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 --2- 172.21.15.113:0/3212960713 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54040b20b0 0x7f54040cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:01.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 -- 172.21.15.113:0/3212960713 >> 172.21.15.113:0/3212960713 conn(0x7f540406e820 msgr2=0x7f5404148030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:01.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 -- 172.21.15.113:0/3212960713 shutdown_connections 2024-03-04T08:01:01.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:01.278+0000 7f540a200640 1 -- 172.21.15.113:0/3212960713 wait complete. 2024-03-04T08:01:01.296 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:01:03.297 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.438+0000 7fd7c5600640 1 Processor -- start 2024-03-04T08:01:03.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.438+0000 7fd7c5600640 1 -- start start 2024-03-04T08:01:03.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.438+0000 7fd7c5600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7c015b2f0 0x7fd7c00c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.438+0000 7fd7c5600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c00c7d80 con 0x7fd7c0154fe0 2024-03-04T08:01:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.438+0000 7fd7c5600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c00c8060 con 0x7fd7c0154c00 2024-03-04T08:01:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.438+0000 7fd7c5600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c015efb0 con 0x7fd7c015b2f0 2024-03-04T08:01:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7bf400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7c015b2f0 0x7fd7c00c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7be000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd7c0154fe0 0x7fd7c015abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44310/0 (socket says 172.21.15.113:44310) 2024-03-04T08:01:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7bf400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7c015b2f0 0x7fd7c00c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52060/0 (socket says 172.21.15.113:52060) 2024-03-04T08:01:03.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7be000640 1 -- 172.21.15.113:0/2571586079 learned_addr learned my addr 172.21.15.113:0/2571586079 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:03.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7bf400640 1 -- 172.21.15.113:0/2571586079 >> v1:172.21.15.136:6789/0 conn(0x7fd7c0154c00 legacy=0x7fd7c00c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:03.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7bd600640 1 -- 172.21.15.113:0/2571586079 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946060917 0 0) 0x7fd7c00c7d80 con 0x7fd7c0154fe0 2024-03-04T08:01:03.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7bf400640 1 -- 172.21.15.113:0/2571586079 >> v1:172.21.15.113:6789/0 conn(0x7fd7c0154fe0 legacy=0x7fd7c015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:03.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7bf400640 1 -- 172.21.15.113:0/2571586079 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00c6dc0 con 0x7fd7c015b2f0 2024-03-04T08:01:03.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7bf400640 1 --2- 172.21.15.113:0/2571586079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c015b2f0 0x7fd7c00c1190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009d90 tx=0x7fd7b801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22cc55f69f3b00f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:03.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7bd600640 1 -- 172.21.15.113:0/2571586079 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80b2070 con 0x7fd7c015b2f0 2024-03-04T08:01:03.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7bd600640 1 -- 172.21.15.113:0/2571586079 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b80a57a0 con 0x7fd7c015b2f0 2024-03-04T08:01:03.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.442+0000 7fd7bd600640 1 -- 172.21.15.113:0/2571586079 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b80a5b70 con 0x7fd7c015b2f0 2024-03-04T08:01:03.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 -- 172.21.15.113:0/2571586079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c015b2f0 msgr2=0x7fd7c00c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:03.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 --2- 172.21.15.113:0/2571586079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c015b2f0 0x7fd7c00c1190 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009d90 tx=0x7fd7b801c8e0 comp rx=0 tx=0).stop 2024-03-04T08:01:03.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 -- 172.21.15.113:0/2571586079 shutdown_connections 2024-03-04T08:01:03.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 --2- 172.21.15.113:0/2571586079 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c015b2f0 0x7fd7c00c1190 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:03.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 -- 172.21.15.113:0/2571586079 >> 172.21.15.113:0/2571586079 conn(0x7fd7c006e820 msgr2=0x7fd7c006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:03.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 -- 172.21.15.113:0/2571586079 shutdown_connections 2024-03-04T08:01:03.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 -- 172.21.15.113:0/2571586079 wait complete. 2024-03-04T08:01:03.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 Processor -- start 2024-03-04T08:01:03.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 -- start start 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0154c00 0x7fd7c00cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7c0154fe0 0x7fd7c00b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7c015b2f0 0x7fd7c00b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c0168fb0 con 0x7fd7c0154c00 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c0168e30 con 0x7fd7c015b2f0 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7c5600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c0169130 con 0x7fd7c0154fe0 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7bea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0154c00 0x7fd7c00cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7bea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0154c00 0x7fd7c00cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52066/0 (socket says 172.21.15.113:52066) 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7bea00640 1 -- 172.21.15.113:0/946991631 learned_addr learned my addr 172.21.15.113:0/946991631 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7bf400640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7c015b2f0 0x7fd7c00b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7be000640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7c0154fe0 0x7fd7c00b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7bea00640 1 -- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7c0154fe0 msgr2=0x7fd7c00b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7bea00640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7c0154fe0 0x7fd7c00b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7bea00640 1 -- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7c015b2f0 msgr2=0x7fd7c00b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7bea00640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7c015b2f0 0x7fd7c00b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7bea00640 1 -- 172.21.15.113:0/946991631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00b1890 con 0x7fd7c0154c00 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.446+0000 7fd7bf400640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7c015b2f0 0x7fd7c00b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7bea00640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0154c00 0x7fd7c00cb350 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000c3a0 tx=0x7fd7b000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7b6a00640 1 -- 172.21.15.113:0/946991631 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b00052d0 con 0x7fd7c0154c00 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7b6a00640 1 -- 172.21.15.113:0/946991631 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b0005470 con 0x7fd7c0154c00 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7b6a00640 1 -- 172.21.15.113:0/946991631 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0007260 con 0x7fd7c0154c00 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7c5600640 1 -- 172.21.15.113:0/946991631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c00b15f0 con 0x7fd7c0154c00 2024-03-04T08:01:03.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7c5600640 1 -- 172.21.15.113:0/946991631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c01f4270 con 0x7fd7c0154c00 2024-03-04T08:01:03.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7c5600640 1 -- 172.21.15.113:0/946991631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c0075c00 con 0x7fd7c0154c00 2024-03-04T08:01:03.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7b6a00640 1 -- 172.21.15.113:0/946991631 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7b0007400 con 0x7fd7c0154c00 2024-03-04T08:01:03.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7b6a00640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7a0066f60 0x7fd7a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:03.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7b6a00640 1 -- 172.21.15.113:0/946991631 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7fd7b00849f0 con 0x7fd7c0154c00 2024-03-04T08:01:03.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.450+0000 7fd7be000640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7a0066f60 0x7fd7a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:03.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.454+0000 7fd7be000640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7a0066f60 0x7fd7a0069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0154140 tx=0x7fd7ac007bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:03.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.458+0000 7fd7b6a00640 1 -- 172.21.15.113:0/946991631 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd7b00554d0 con 0x7fd7c0154c00 2024-03-04T08:01:03.648 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:01:03.652 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:01:03.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.657+0000 7fd7c5600640 1 -- 172.21.15.113:0/946991631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7c0159df0 con 0x7fd7c0154c00 2024-03-04T08:01:03.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.657+0000 7fd7b6a00640 1 -- 172.21.15.113:0/946991631 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+5824 (secure 0 0 0) 0x7fd7b0054dd0 con 0x7fd7c0154c00 2024-03-04T08:01:03.658 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:03.658 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6053,6534,6372]},"id":1}]} 2024-03-04T08:01:03.658 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 99 2024-03-04T08:01:03.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 -- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7a0066f60 msgr2=0x7fd7a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:03.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7a0066f60 0x7fd7a0069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0154140 tx=0x7fd7ac007bc0 comp rx=0 tx=0).stop 2024-03-04T08:01:03.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 -- 172.21.15.113:0/946991631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0154c00 msgr2=0x7fd7c00cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:03.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0154c00 0x7fd7c00cb350 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000c3a0 tx=0x7fd7b000c860 comp rx=0 tx=0).stop 2024-03-04T08:01:03.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 -- 172.21.15.113:0/946991631 shutdown_connections 2024-03-04T08:01:03.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7c015b2f0 0x7fd7c00b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:03.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7c0154fe0 0x7fd7c00b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:03.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7a0066f60 0x7fd7a0069420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:03.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 --2- 172.21.15.113:0/946991631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7c0154c00 0x7fd7c00cb350 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:03.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 -- 172.21.15.113:0/946991631 >> 172.21.15.113:0/946991631 conn(0x7fd7c006e820 msgr2=0x7fd7c00c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:03.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 -- 172.21.15.113:0/946991631 shutdown_connections 2024-03-04T08:01:03.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:03.661+0000 7fd7c5600640 1 -- 172.21.15.113:0/946991631 wait complete. 2024-03-04T08:01:03.684 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=2, replacing gid=6240 2024-03-04T08:01:03.684 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.a 2024-03-04T08:01:07.175 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at c03ca95d235 v0.51 2024-03-04T08:01:07.177 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.51 2024-03-04T08:01:07.190 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 52 -eq 48 ] 2024-03-04T08:01:07.190 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.52 2024-03-04T08:01:07.190 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.52 2024-03-04T08:01:07.190 INFO:tasks.workunit.client.0.smithi113.stdout:v0.52 2024-03-04T08:01:07.190 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.52 2024-03-04T08:01:10.685 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-03-04T08:01:10.685 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-04T08:01:10.685 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-04T08:01:10.689 INFO:tasks.ceph.mds.a:Started 2024-03-04T08:01:10.689 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:10.775 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-03-04T08:01:10.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.837+0000 7fc886400640 1 Processor -- start 2024-03-04T08:01:10.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.837+0000 7fc886400640 1 -- start start 2024-03-04T08:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.837+0000 7fc886400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc880152bd0 0x7fc880070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.837+0000 7fc886400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8801591f0 con 0x7fc8800c8200 2024-03-04T08:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.837+0000 7fc886400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8801594d0 con 0x7fc8801527f0 2024-03-04T08:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.837+0000 7fc886400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc880071df0 con 0x7fc880152bd0 2024-03-04T08:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.837+0000 7fc876a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc8800c8200 0x7fc8801587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57100/0 (socket says 172.21.15.113:57100) 2024-03-04T08:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.837+0000 7fc876a00640 1 -- 172.21.15.113:0/658501064 learned_addr learned my addr 172.21.15.113:0/658501064 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:10.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.837+0000 7fc87fe00640 1 --2- 172.21.15.113:0/658501064 >> v2:172.21.15.113:3300/0 conn(0x7fc880152bd0 0x7fc880070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:10.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc87fe00640 1 -- 172.21.15.113:0/658501064 >> v1:172.21.15.136:6789/0 conn(0x7fc8801527f0 legacy=0x7fc880072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:10.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc87ea00640 1 -- 172.21.15.113:0/658501064 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149822181 0 0) 0x7fc8801594d0 con 0x7fc8801527f0 2024-03-04T08:01:10.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc87fe00640 1 -- 172.21.15.113:0/658501064 >> v1:172.21.15.113:6789/0 conn(0x7fc8800c8200 legacy=0x7fc8801587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:10.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc87fe00640 1 -- 172.21.15.113:0/658501064 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8800717a0 con 0x7fc880152bd0 2024-03-04T08:01:10.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc87fe00640 1 --2- 172.21.15.113:0/658501064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 0x7fc880070d40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc878009d90 tx=0x7fc87801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc54532f9724a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc87ea00640 1 -- 172.21.15.113:0/658501064 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8780b2070 con 0x7fc880152bd0 2024-03-04T08:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc87ea00640 1 -- 172.21.15.113:0/658501064 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8780a5830 con 0x7fc880152bd0 2024-03-04T08:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc87ea00640 1 -- 172.21.15.113:0/658501064 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8780a5c00 con 0x7fc880152bd0 2024-03-04T08:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc886400640 1 -- 172.21.15.113:0/658501064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 msgr2=0x7fc880070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:10.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc886400640 1 --2- 172.21.15.113:0/658501064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 0x7fc880070d40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc878009d90 tx=0x7fc87801c970 comp rx=0 tx=0).stop 2024-03-04T08:01:10.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc886400640 1 -- 172.21.15.113:0/658501064 shutdown_connections 2024-03-04T08:01:10.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc886400640 1 --2- 172.21.15.113:0/658501064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 0x7fc880070d40 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:10.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.841+0000 7fc886400640 1 -- 172.21.15.113:0/658501064 >> 172.21.15.113:0/658501064 conn(0x7fc88006e820 msgr2=0x7fc88006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:10.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.845+0000 7fc886400640 1 -- 172.21.15.113:0/658501064 shutdown_connections 2024-03-04T08:01:10.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.845+0000 7fc886400640 1 -- 172.21.15.113:0/658501064 wait complete. 2024-03-04T08:01:10.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.845+0000 7fc886400640 1 Processor -- start 2024-03-04T08:01:10.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.845+0000 7fc886400640 1 -- start start 2024-03-04T08:01:10.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc886400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8800c8200 0x7fc880149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:10.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc886400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8801527f0 0x7fc88014b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:10.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc886400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 0x7fc88014bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc886400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc880166500 con 0x7fc880152bd0 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc886400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc880166380 con 0x7fc8800c8200 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc886400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc880166680 con 0x7fc8801527f0 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 0x7fc88014bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc876a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8801527f0 0x7fc88014b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc87fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 0x7fc88014bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59782/0 (socket says 172.21.15.113:59782) 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc87fe00640 1 -- 172.21.15.113:0/157708997 learned_addr learned my addr 172.21.15.113:0/157708997 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc876a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8801527f0 0x7fc88014b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:46844/0 (socket says 172.21.15.113:46844) 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc87f400640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8800c8200 0x7fc880149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc87fe00640 1 -- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8801527f0 msgr2=0x7fc88014b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc87fe00640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8801527f0 0x7fc88014b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc87fe00640 1 -- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8800c8200 msgr2=0x7fc880149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc87fe00640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8800c8200 0x7fc880149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:10.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc87fe00640 1 -- 172.21.15.113:0/157708997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc880156930 con 0x7fc880152bd0 2024-03-04T08:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc87fe00640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 0x7fc88014bbd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc8780b0040 tx=0x7fc878019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc877e00640 1 -- 172.21.15.113:0/157708997 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8780b2050 con 0x7fc880152bd0 2024-03-04T08:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc877e00640 1 -- 172.21.15.113:0/157708997 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8780be070 con 0x7fc880152bd0 2024-03-04T08:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc877e00640 1 -- 172.21.15.113:0/157708997 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8780ada50 con 0x7fc880152bd0 2024-03-04T08:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc886400640 1 -- 172.21.15.113:0/157708997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc880156c10 con 0x7fc880152bd0 2024-03-04T08:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc886400640 1 -- 172.21.15.113:0/157708997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8801571d0 con 0x7fc880152bd0 2024-03-04T08:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc886400640 1 -- 172.21.15.113:0/157708997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc880075c00 con 0x7fc880152bd0 2024-03-04T08:01:10.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.849+0000 7fc877e00640 1 -- 172.21.15.113:0/157708997 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8780adbf0 con 0x7fc880152bd0 2024-03-04T08:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.853+0000 7fc877e00640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc85c066f90 0x7fc85c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.853+0000 7fc877e00640 1 -- 172.21.15.113:0/157708997 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7fc878120a60 con 0x7fc880152bd0 2024-03-04T08:01:10.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.853+0000 7fc87f400640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc85c066f90 0x7fc85c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:10.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.853+0000 7fc87f400640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc85c066f90 0x7fc85c069450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc870000f70 tx=0x7fc870009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:10.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:10.857+0000 7fc877e00640 1 -- 172.21.15.113:0/157708997 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc8780c7020 con 0x7fc880152bd0 2024-03-04T08:01:11.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 -- 172.21.15.113:0/157708997 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8800c79d0 con 0x7fc880152bd0 2024-03-04T08:01:11.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc877e00640 1 -- 172.21.15.113:0/157708997 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+5824 (secure 0 0 0) 0x7fc878016560 con 0x7fc880152bd0 2024-03-04T08:01:11.059 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:11.059 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6053,6534,6372]},"id":1}]} 2024-03-04T08:01:11.060 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 99 2024-03-04T08:01:11.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 -- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc85c066f90 msgr2=0x7fc85c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:11.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc85c066f90 0x7fc85c069450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc870000f70 tx=0x7fc870009040 comp rx=0 tx=0).stop 2024-03-04T08:01:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 -- 172.21.15.113:0/157708997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 msgr2=0x7fc88014bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 0x7fc88014bbd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc8780b0040 tx=0x7fc878019710 comp rx=0 tx=0).stop 2024-03-04T08:01:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 -- 172.21.15.113:0/157708997 shutdown_connections 2024-03-04T08:01:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc85c066f90 0x7fc85c069450 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc880152bd0 0x7fc88014bbd0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8801527f0 0x7fc88014b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 --2- 172.21.15.113:0/157708997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8800c8200 0x7fc880149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 -- 172.21.15.113:0/157708997 >> 172.21.15.113:0/157708997 conn(0x7fc88006e820 msgr2=0x7fc8800c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 -- 172.21.15.113:0/157708997 shutdown_connections 2024-03-04T08:01:11.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:11.061+0000 7fc886400640 1 -- 172.21.15.113:0/157708997 wait complete. 2024-03-04T08:01:11.090 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-03-04T08:01:13.090 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:13.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14eba00640 1 Processor -- start 2024-03-04T08:01:13.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14eba00640 1 -- start start 2024-03-04T08:01:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14eba00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f14e40ce2d0 0x7f14e415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14eba00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f14e40ce6b0 0x7f14e406f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14eba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e406de80 0x7f14e4154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14eba00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e415bbf0 con 0x7f14e40ce2d0 2024-03-04T08:01:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14eba00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e406eb80 con 0x7f14e40ce6b0 2024-03-04T08:01:13.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14eba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e406c210 con 0x7f14e406de80 2024-03-04T08:01:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e9000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f14e40ce2d0 0x7f14e415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e9000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f14e40ce2d0 0x7f14e415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59794/0 (socket says 172.21.15.113:59794) 2024-03-04T08:01:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e9000640 1 -- 172.21.15.113:0/2752894823 learned_addr learned my addr 172.21.15.113:0/2752894823 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e3e00640 1 --2- 172.21.15.113:0/2752894823 >> v2:172.21.15.136:3300/0 conn(0x7f14e40ce6b0 0x7f14e406f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e9a00640 1 --2- 172.21.15.113:0/2752894823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e406de80 0x7f14e4154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e9000640 1 -- 172.21.15.113:0/2752894823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e406de80 msgr2=0x7f14e4154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:13.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e9000640 1 --2- 172.21.15.113:0/2752894823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e406de80 0x7f14e4154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e9000640 1 -- 172.21.15.113:0/2752894823 >> v2:172.21.15.136:3300/0 conn(0x7f14e40ce6b0 msgr2=0x7f14e406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e9000640 1 --2- 172.21.15.113:0/2752894823 >> v2:172.21.15.136:3300/0 conn(0x7f14e40ce6b0 0x7f14e406f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e9000640 1 -- 172.21.15.113:0/2752894823 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e406c4f0 con 0x7f14e40ce2d0 2024-03-04T08:01:13.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.237+0000 7f14e9a00640 1 --2- 172.21.15.113:0/2752894823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e406de80 0x7f14e4154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14e9000640 1 --2- 172.21.15.113:0/2752894823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e40ce2d0 0x7f14e415b6b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f14e4158860 tx=0x7f14d801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebf140aca802c9e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14e3400640 1 -- 172.21.15.113:0/2752894823 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d80b5070 con 0x7f14e40ce2d0 2024-03-04T08:01:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14e3400640 1 -- 172.21.15.113:0/2752894823 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d80a89d0 con 0x7f14e40ce2d0 2024-03-04T08:01:13.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14e3400640 1 -- 172.21.15.113:0/2752894823 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d80a8da0 con 0x7f14e40ce2d0 2024-03-04T08:01:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 -- 172.21.15.113:0/2752894823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e40ce2d0 msgr2=0x7f14e415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 --2- 172.21.15.113:0/2752894823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e40ce2d0 0x7f14e415b6b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f14e4158860 tx=0x7f14d801ed10 comp rx=0 tx=0).stop 2024-03-04T08:01:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 -- 172.21.15.113:0/2752894823 shutdown_connections 2024-03-04T08:01:13.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 --2- 172.21.15.113:0/2752894823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e406de80 0x7f14e4154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 --2- 172.21.15.113:0/2752894823 >> v2:172.21.15.136:3300/0 conn(0x7f14e40ce6b0 0x7f14e406f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 --2- 172.21.15.113:0/2752894823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e40ce2d0 0x7f14e415b6b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 -- 172.21.15.113:0/2752894823 >> 172.21.15.113:0/2752894823 conn(0x7f14e40709a0 msgr2=0x7f14e4070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 -- 172.21.15.113:0/2752894823 shutdown_connections 2024-03-04T08:01:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 -- 172.21.15.113:0/2752894823 wait complete. 2024-03-04T08:01:13.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 Processor -- start 2024-03-04T08:01:13.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.245+0000 7f14eba00640 1 -- start start 2024-03-04T08:01:13.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14eba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e406de80 0x7f14e414aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14eba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14e40ce2d0 0x7f14e415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14eba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e40ce6b0 0x7f14e4143d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14eba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e406c7d0 con 0x7f14e406de80 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14eba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e406e810 con 0x7f14e40ce2d0 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14eba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4070010 con 0x7f14e40ce6b0 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e406de80 0x7f14e414aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e406de80 0x7f14e414aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59798/0 (socket says 172.21.15.113:59798) 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e9000640 1 -- 172.21.15.113:0/3746207923 learned_addr learned my addr 172.21.15.113:0/3746207923 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e9a00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e40ce6b0 0x7f14e4143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e3e00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14e40ce2d0 0x7f14e415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e9000640 1 -- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e40ce6b0 msgr2=0x7f14e4143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e9000640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e40ce6b0 0x7f14e4143d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e9000640 1 -- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14e40ce2d0 msgr2=0x7f14e415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e9000640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14e40ce2d0 0x7f14e415abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e9000640 1 -- 172.21.15.113:0/3746207923 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e41530b0 con 0x7f14e406de80 2024-03-04T08:01:13.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e9000640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e406de80 0x7f14e414aa30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f14d801d3c0 tx=0x7f14d801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:13.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e0c00640 1 -- 172.21.15.113:0/3746207923 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d80b5070 con 0x7f14e406de80 2024-03-04T08:01:13.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e0c00640 1 -- 172.21.15.113:0/3746207923 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d80b09d0 con 0x7f14e406de80 2024-03-04T08:01:13.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14eba00640 1 -- 172.21.15.113:0/3746207923 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e4153390 con 0x7f14e406de80 2024-03-04T08:01:13.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14e0c00640 1 -- 172.21.15.113:0/3746207923 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d80b0dc0 con 0x7f14e406de80 2024-03-04T08:01:13.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14eba00640 1 -- 172.21.15.113:0/3746207923 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e4155ee0 con 0x7f14e406de80 2024-03-04T08:01:13.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.249+0000 7f14eba00640 1 -- 172.21.15.113:0/3746207923 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e4075c00 con 0x7f14e406de80 2024-03-04T08:01:13.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.253+0000 7f14e0c00640 1 -- 172.21.15.113:0/3746207923 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14d80c2060 con 0x7f14e406de80 2024-03-04T08:01:13.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.253+0000 7f14e0c00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14c4066f60 0x7f14c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.253+0000 7f14e0c00640 1 -- 172.21.15.113:0/3746207923 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f14d81224e0 con 0x7f14e406de80 2024-03-04T08:01:13.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.253+0000 7f14e3e00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14c4066f60 0x7f14c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.253+0000 7f14e3e00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14c4066f60 0x7f14c4069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f14e4152dc0 tx=0x7f14d0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:13.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.257+0000 7f14e0c00640 1 -- 172.21.15.113:0/3746207923 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f14d8018870 con 0x7f14e406de80 2024-03-04T08:01:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.469+0000 7f14eba00640 1 -- 172.21.15.113:0/3746207923 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14e4153b40 con 0x7f14e406de80 2024-03-04T08:01:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14e0c00640 1 -- 172.21.15.113:0/3746207923 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+6620 (secure 0 0 0) 0x7f14d80f2970 con 0x7f14e406de80 2024-03-04T08:01:13.471 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:13.471 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534,6053]},"id":1}]} 2024-03-04T08:01:13.471 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 100 2024-03-04T08:01:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 -- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14c4066f60 msgr2=0x7f14c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14c4066f60 0x7f14c4069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f14e4152dc0 tx=0x7f14d0009190 comp rx=0 tx=0).stop 2024-03-04T08:01:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 -- 172.21.15.113:0/3746207923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e406de80 msgr2=0x7f14e414aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e406de80 0x7f14e414aa30 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f14d801d3c0 tx=0x7f14d801bab0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 -- 172.21.15.113:0/3746207923 shutdown_connections 2024-03-04T08:01:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f14e40ce6b0 0x7f14e4143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14e40ce2d0 0x7f14e415abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f14c4066f60 0x7f14c4069420 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 --2- 172.21.15.113:0/3746207923 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14e406de80 0x7f14e414aa30 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 -- 172.21.15.113:0/3746207923 >> 172.21.15.113:0/3746207923 conn(0x7f14e40709a0 msgr2=0x7f14e4150040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:13.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 -- 172.21.15.113:0/3746207923 shutdown_connections 2024-03-04T08:01:13.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.473+0000 7f14eba00640 1 -- 172.21.15.113:0/3746207923 wait complete. 2024-03-04T08:01:13.492 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-03-04T08:01:13.492 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:13.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.633+0000 7fafe8e00640 1 Processor -- start 2024-03-04T08:01:13.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.633+0000 7fafe8e00640 1 -- start start 2024-03-04T08:01:13.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.633+0000 7fafe8e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fafe4158010 0x7fafe406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.633+0000 7fafe8e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafe406ff20 0x7fafe415b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafe8e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe40ca0e0 con 0x7fafe415b750 2024-03-04T08:01:13.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafe8e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe406f450 con 0x7fafe4158010 2024-03-04T08:01:13.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafe8e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe415ad80 con 0x7fafe406ff20 2024-03-04T08:01:13.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafe1e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fafe4158010 0x7fafe406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafe1e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fafe4158010 0x7fafe406dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59814/0 (socket says 172.21.15.113:59814) 2024-03-04T08:01:13.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafe1e00640 1 -- 172.21.15.113:0/3011774408 learned_addr learned my addr 172.21.15.113:0/3011774408 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:13.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafd9400640 1 --2- 172.21.15.113:0/3011774408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafe406ff20 0x7fafe415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafe1e00640 1 -- 172.21.15.113:0/3011774408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafe406ff20 msgr2=0x7fafe415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:13.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafe1e00640 1 --2- 172.21.15.113:0/3011774408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafe406ff20 0x7fafe415b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafe1e00640 1 -- 172.21.15.113:0/3011774408 >> v1:172.21.15.136:6789/0 conn(0x7fafe415b750 legacy=0x7fafe40ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.637+0000 7fafe1e00640 1 -- 172.21.15.113:0/3011774408 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe415aa30 con 0x7fafe4158010 2024-03-04T08:01:13.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe1e00640 1 --2- 172.21.15.113:0/3011774408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 0x7fafe406dd50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009870 tx=0x7fafd401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591730981e937791 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:13.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe1400640 1 -- 172.21.15.113:0/3011774408 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40b2070 con 0x7fafe4158010 2024-03-04T08:01:13.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe1400640 1 -- 172.21.15.113:0/3011774408 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd40a9070 con 0x7fafe4158010 2024-03-04T08:01:13.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe1400640 1 -- 172.21.15.113:0/3011774408 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40ad880 con 0x7fafe4158010 2024-03-04T08:01:13.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe8e00640 1 -- 172.21.15.113:0/3011774408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 msgr2=0x7fafe406dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe8e00640 1 --2- 172.21.15.113:0/3011774408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 0x7fafe406dd50 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009870 tx=0x7fafd401c560 comp rx=0 tx=0).stop 2024-03-04T08:01:13.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe8e00640 1 -- 172.21.15.113:0/3011774408 shutdown_connections 2024-03-04T08:01:13.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe8e00640 1 --2- 172.21.15.113:0/3011774408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafe406ff20 0x7fafe415b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe8e00640 1 --2- 172.21.15.113:0/3011774408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 0x7fafe406dd50 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe8e00640 1 -- 172.21.15.113:0/3011774408 >> 172.21.15.113:0/3011774408 conn(0x7fafe40734d0 msgr2=0x7fafe4066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:13.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe8e00640 1 -- 172.21.15.113:0/3011774408 shutdown_connections 2024-03-04T08:01:13.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.641+0000 7fafe8e00640 1 -- 172.21.15.113:0/3011774408 wait complete. 2024-03-04T08:01:13.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafe8e00640 1 Processor -- start 2024-03-04T08:01:13.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafe8e00640 1 -- start start 2024-03-04T08:01:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafe8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafe406ff20 0x7fafe414f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafe8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 0x7fafe40b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafe8e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafe415b750 0x7fafe40ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafe8e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe41583f0 con 0x7fafe4158010 2024-03-04T08:01:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafe8e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe406e290 con 0x7fafe406ff20 2024-03-04T08:01:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafe8e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe406c170 con 0x7fafe415b750 2024-03-04T08:01:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafd9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 0x7fafe40b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafe1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafe406ff20 0x7fafe414f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafd9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 0x7fafe40b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59818/0 (socket says 172.21.15.113:59818) 2024-03-04T08:01:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafd9400640 1 -- 172.21.15.113:0/863210104 learned_addr learned my addr 172.21.15.113:0/863210104 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafe2800640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafe415b750 0x7fafe40ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafd9400640 1 -- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafe415b750 msgr2=0x7fafe40ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafd9400640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafe415b750 0x7fafe40ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafd9400640 1 -- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafe406ff20 msgr2=0x7fafe414f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafd9400640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafe406ff20 0x7fafe414f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafd9400640 1 -- 172.21.15.113:0/863210104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe40b0c20 con 0x7fafe4158010 2024-03-04T08:01:13.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.645+0000 7fafd9400640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 0x7fafe40b0060 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00e930 tx=0x7fafcc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:13.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.649+0000 7fafdaa00640 1 -- 172.21.15.113:0/863210104 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc017070 con 0x7fafe4158010 2024-03-04T08:01:13.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.649+0000 7fafdaa00640 1 -- 172.21.15.113:0/863210104 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc00ca00 con 0x7fafe4158010 2024-03-04T08:01:13.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.649+0000 7fafe8e00640 1 -- 172.21.15.113:0/863210104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe4150040 con 0x7fafe4158010 2024-03-04T08:01:13.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.649+0000 7fafdaa00640 1 -- 172.21.15.113:0/863210104 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc00cdd0 con 0x7fafe4158010 2024-03-04T08:01:13.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.649+0000 7fafe8e00640 1 -- 172.21.15.113:0/863210104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe41f3250 con 0x7fafe4158010 2024-03-04T08:01:13.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.649+0000 7fafe8e00640 1 -- 172.21.15.113:0/863210104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe4075c00 con 0x7fafe4158010 2024-03-04T08:01:13.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.649+0000 7fafdaa00640 1 -- 172.21.15.113:0/863210104 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafcc01a070 con 0x7fafe4158010 2024-03-04T08:01:13.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.649+0000 7fafdaa00640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafc0066f60 0x7fafc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:13.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.649+0000 7fafdaa00640 1 -- 172.21.15.113:0/863210104 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7fafcc085520 con 0x7fafe4158010 2024-03-04T08:01:13.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.649+0000 7fafe1e00640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafc0066f60 0x7fafc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:13.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.653+0000 7fafe1e00640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafc0066f60 0x7fafc0069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fafd40af000 tx=0x7fafd40a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:13.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.653+0000 7fafdaa00640 1 -- 172.21.15.113:0/863210104 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fafe40b0c20 con 0x7fafe4158010 2024-03-04T08:01:13.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.849+0000 7fafe8e00640 1 -- 172.21.15.113:0/863210104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafe40cba20 con 0x7fafe4158010 2024-03-04T08:01:13.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafdaa00640 1 -- 172.21.15.113:0/863210104 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+6620 (secure 0 0 0) 0x7fafcc0559d0 con 0x7fafe4158010 2024-03-04T08:01:13.851 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:13.851 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534,6053]},"id":1}]} 2024-03-04T08:01:13.851 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 100 2024-03-04T08:01:13.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 -- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafc0066f60 msgr2=0x7fafc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafc0066f60 0x7fafc0069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fafd40af000 tx=0x7fafd40a7000 comp rx=0 tx=0).stop 2024-03-04T08:01:13.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 -- 172.21.15.113:0/863210104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 msgr2=0x7fafe40b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:13.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 0x7fafe40b0060 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00e930 tx=0x7fafcc00edf0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 -- 172.21.15.113:0/863210104 shutdown_connections 2024-03-04T08:01:13.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafe415b750 0x7fafe40ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafc0066f60 0x7fafc0069420 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafe4158010 0x7fafe40b0060 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 --2- 172.21.15.113:0/863210104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafe406ff20 0x7fafe414f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:13.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 -- 172.21.15.113:0/863210104 >> 172.21.15.113:0/863210104 conn(0x7fafe40734d0 msgr2=0x7fafe4071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:13.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 -- 172.21.15.113:0/863210104 shutdown_connections 2024-03-04T08:01:13.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:13.853+0000 7fafe8e00640 1 -- 172.21.15.113:0/863210104 wait complete. 2024-03-04T08:01:13.874 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=2) 2024-03-04T08:01:13.875 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-04T08:01:13.875 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:01:13.888 INFO:tasks.ceph.mds.b:Stopped 2024-03-04T08:01:13.888 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:14.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feffb200640 1 Processor -- start 2024-03-04T08:01:14.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feffb200640 1 -- start start 2024-03-04T08:01:14.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feffb200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feff40c22e0 0x7feff415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:14.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feffb200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feff40c2d80 0x7feff415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:14.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feffb200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff40c27f0 con 0x7feff414e4c0 2024-03-04T08:01:14.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feffb200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff415ad60 con 0x7feff40c2d80 2024-03-04T08:01:14.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feffb200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff40c2a90 con 0x7feff40c22e0 2024-03-04T08:01:14.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feff9200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feff40c2d80 0x7feff415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:14.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7fefebe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feff40c22e0 0x7feff415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:14.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feff9200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feff40c2d80 0x7feff415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:58202/0 (socket says 172.21.15.113:58202) 2024-03-04T08:01:14.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feff9200640 1 -- 172.21.15.113:0/4099990542 learned_addr learned my addr 172.21.15.113:0/4099990542 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:14.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feff9200640 1 -- 172.21.15.113:0/4099990542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feff40c22e0 msgr2=0x7feff415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:14.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feff9200640 1 --2- 172.21.15.113:0/4099990542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feff40c22e0 0x7feff415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:14.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feff9200640 1 -- 172.21.15.113:0/4099990542 >> v1:172.21.15.136:6789/0 conn(0x7feff414e4c0 legacy=0x7feff4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:14.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.029+0000 7feff9200640 1 -- 172.21.15.113:0/4099990542 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4070c60 con 0x7feff40c2d80 2024-03-04T08:01:14.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feff9200640 1 --2- 172.21.15.113:0/4099990542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feff40c2d80 0x7feff415b730 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fefec009e50 tx=0x7fefec01f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6d72aaad50b53b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:14.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feff3400640 1 -- 172.21.15.113:0/4099990542 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec0b4070 con 0x7feff40c2d80 2024-03-04T08:01:14.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feff3400640 1 -- 172.21.15.113:0/4099990542 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec0af400 con 0x7feff40c2d80 2024-03-04T08:01:14.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feff3400640 1 -- 172.21.15.113:0/4099990542 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec0af7d0 con 0x7feff40c2d80 2024-03-04T08:01:14.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feffb200640 1 -- 172.21.15.113:0/4099990542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feff40c2d80 msgr2=0x7feff415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:14.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feffb200640 1 --2- 172.21.15.113:0/4099990542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feff40c2d80 0x7feff415b730 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fefec009e50 tx=0x7fefec01f210 comp rx=0 tx=0).stop 2024-03-04T08:01:14.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feffb200640 1 -- 172.21.15.113:0/4099990542 shutdown_connections 2024-03-04T08:01:14.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feffb200640 1 --2- 172.21.15.113:0/4099990542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feff40c2d80 0x7feff415b730 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:14.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feffb200640 1 --2- 172.21.15.113:0/4099990542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feff40c22e0 0x7feff415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:14.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feffb200640 1 -- 172.21.15.113:0/4099990542 >> 172.21.15.113:0/4099990542 conn(0x7feff406e820 msgr2=0x7feff406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:14.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feffb200640 1 -- 172.21.15.113:0/4099990542 shutdown_connections 2024-03-04T08:01:14.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.033+0000 7feffb200640 1 -- 172.21.15.113:0/4099990542 wait complete. 2024-03-04T08:01:14.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feffb200640 1 Processor -- start 2024-03-04T08:01:14.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feffb200640 1 -- start start 2024-03-04T08:01:14.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feffb200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feff40c22e0 0x7feff41493b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:14.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feffb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feff40c2d80 0x7feff414b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:14.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feffb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feff414e4c0 0x7feff40cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:14.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feffb200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff4071500 con 0x7feff40c2d80 2024-03-04T08:01:14.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feffb200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff4071220 con 0x7feff414e4c0 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feffb200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff414e8a0 con 0x7feff40c22e0 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7fefebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feff40c2d80 0x7feff414b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7fefebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feff40c2d80 0x7feff414b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59832/0 (socket says 172.21.15.113:59832) 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7fefebe00640 1 -- 172.21.15.113:0/847856085 learned_addr learned my addr 172.21.15.113:0/847856085 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feff3e00640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feff40c22e0 0x7feff41493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feff9200640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feff414e4c0 0x7feff40cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7fefebe00640 1 -- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feff40c22e0 msgr2=0x7feff41493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7fefebe00640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feff40c22e0 0x7feff41493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7fefebe00640 1 -- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feff414e4c0 msgr2=0x7feff40cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7fefebe00640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feff414e4c0 0x7feff40cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:14.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7fefebe00640 1 -- 172.21.15.113:0/847856085 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff415a500 con 0x7feff40c2d80 2024-03-04T08:01:14.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7fefebe00640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feff40c2d80 0x7feff414b410 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00e870 tx=0x7fefdc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:14.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feff0c00640 1 -- 172.21.15.113:0/847856085 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc017070 con 0x7feff40c2d80 2024-03-04T08:01:14.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feff0c00640 1 -- 172.21.15.113:0/847856085 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc00c940 con 0x7feff40c2d80 2024-03-04T08:01:14.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feffb200640 1 -- 172.21.15.113:0/847856085 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff415a7e0 con 0x7feff40c2d80 2024-03-04T08:01:14.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feff0c00640 1 -- 172.21.15.113:0/847856085 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc00ccf0 con 0x7feff40c2d80 2024-03-04T08:01:14.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.037+0000 7feffb200640 1 -- 172.21.15.113:0/847856085 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff4148bb0 con 0x7feff40c2d80 2024-03-04T08:01:14.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.041+0000 7feffb200640 1 -- 172.21.15.113:0/847856085 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff4155190 con 0x7feff40c2d80 2024-03-04T08:01:14.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.041+0000 7feff0c00640 1 -- 172.21.15.113:0/847856085 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fefdc01a070 con 0x7feff40c2d80 2024-03-04T08:01:14.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.041+0000 7feff0c00640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd0066f60 0x7fefd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:14.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.041+0000 7feff0c00640 1 -- 172.21.15.113:0/847856085 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7fefdc085800 con 0x7feff40c2d80 2024-03-04T08:01:14.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.041+0000 7feff3e00640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd0066f60 0x7fefd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:14.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.041+0000 7feff3e00640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd0066f60 0x7fefd0069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fefe4001810 tx=0x7fefe4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:14.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.049+0000 7feff0c00640 1 -- 172.21.15.113:0/847856085 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fefdc055b60 con 0x7feff40c2d80 2024-03-04T08:01:14.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.269+0000 7feffb200640 1 -- 172.21.15.113:0/847856085 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feff40c1260 con 0x7feff40c2d80 2024-03-04T08:01:14.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.269+0000 7feff0c00640 1 -- 172.21.15.113:0/847856085 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+6620 (secure 0 0 0) 0x7fefdc059c00 con 0x7feff40c2d80 2024-03-04T08:01:14.270 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:14.270 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534,6053]},"id":1}]} 2024-03-04T08:01:14.270 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 100 2024-03-04T08:01:14.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 -- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd0066f60 msgr2=0x7fefd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:14.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd0066f60 0x7fefd0069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fefe4001810 tx=0x7fefe4009040 comp rx=0 tx=0).stop 2024-03-04T08:01:14.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 -- 172.21.15.113:0/847856085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feff40c2d80 msgr2=0x7feff414b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:14.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feff40c2d80 0x7feff414b410 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00e870 tx=0x7fefdc00ec70 comp rx=0 tx=0).stop 2024-03-04T08:01:14.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 -- 172.21.15.113:0/847856085 shutdown_connections 2024-03-04T08:01:14.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feff414e4c0 0x7feff40cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:14.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fefd0066f60 0x7fefd0069420 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:14.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feff40c2d80 0x7feff414b410 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:14.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 --2- 172.21.15.113:0/847856085 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feff40c22e0 0x7feff41493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:14.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 -- 172.21.15.113:0/847856085 >> 172.21.15.113:0/847856085 conn(0x7feff406e820 msgr2=0x7feff414c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:14.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 -- 172.21.15.113:0/847856085 shutdown_connections 2024-03-04T08:01:14.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:14.273+0000 7feffb200640 1 -- 172.21.15.113:0/847856085 wait complete. 2024-03-04T08:01:14.292 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:01:15.357 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 38% (70/183) Updating files: 39% (72/183) Updating files: 40% (74/183) Updating files: 41% (76/183) Updating files: 42% (77/183) Updating files: 43% (79/183) Updating files: 44% (81/183) Updating files: 45% (83/183) Updating files: 46% (85/183) Updating files: 47% (87/183) Updating files: 48% (88/183) Updating files: 49% (90/183) Updating files: 50% (92/183) Updating files: 51% (94/183) Updating files: 51% (95/183) Updating files: 52% (96/183) Updating files: 53% (97/183) Updating files: 54% (99/183) Updating files: 55% (101/183) Updating files: 56% (103/183) Updating files: 57% (105/183) Updating files: 58% (107/183) Updating files: 59% (108/183) Updating files: 60% (110/183) Updating files: 61% (112/183) Updating files: 62% (114/183) Updating files: 63% (116/183) Updating files: 64% (118/183) Updating files: 65% (119/183) Updating files: 66% (121/183) Updating files: 67% (123/183) Updating files: 68% (125/183) Updating files: 69% (127/183) Updating files: 70% (129/183) Updating files: 71% (130/183) Updating files: 72% (132/183) Updating files: 73% (134/183) Updating files: 74% (136/183) Updating files: 75% (138/183) Updating files: 76% (140/183) Updating files: 77% (141/183) Updating files: 78% (143/183) Updating files: 79% (145/183) Updating files: 80% (147/183) Updating files: 81% (149/183) Updating files: 82% (151/183) Updating files: 83% (152/183) Updating files: 84% (154/183) Updating files: 85% (156/183) Updating files: 86% (158/183) Updating files: 87% (160/183) Updating files: 88% (162/183) Updating files: 89% (163/183) Updating files: 90% (165/183) Updating files: 91% (167/183) Updating files: 92% (169/183) Updating files: 93% (171/183) Updating files: 94% (173/183) Updating files: 95% (174/183) Updating files: 96% (176/183) Updating files: 97% (178/183) Updating files: 98% (180/183) Updating files: 99% (182/183) Updating files: 100% (183/183) Updating files: 100% (183/183), done. 2024-03-04T08:01:16.293 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:16.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fd0000640 1 Processor -- start 2024-03-04T08:01:16.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fd0000640 1 -- start start 2024-03-04T08:01:16.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fd0000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8070010 0x7f7fc80c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:16.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fd0000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc8152f00 con 0x7f7fc815c510 2024-03-04T08:01:16.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fd0000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc80c5040 con 0x7f7fc8152b20 2024-03-04T08:01:16.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fd0000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80c5320 con 0x7f7fc8070010 2024-03-04T08:01:16.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fce000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7fc815c510 0x7f7fc8163190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57104/0 (socket says 172.21.15.113:57104) 2024-03-04T08:01:16.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fce000640 1 -- 172.21.15.113:0/633117566 learned_addr learned my addr 172.21.15.113:0/633117566 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:16.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fccc00640 1 --2- 172.21.15.113:0/633117566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8070010 0x7f7fc80c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:16.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fc7e00640 1 -- 172.21.15.113:0/633117566 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691439833 0 0) 0x7f7fc8152f00 con 0x7f7fc815c510 2024-03-04T08:01:16.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fc7e00640 1 -- 172.21.15.113:0/633117566 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fb0003530 con 0x7f7fc815c510 2024-03-04T08:01:16.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fc7e00640 1 -- 172.21.15.113:0/633117566 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735970235 0 0) 0x7f7fc80c5040 con 0x7f7fc8152b20 2024-03-04T08:01:16.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fc7e00640 1 -- 172.21.15.113:0/633117566 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fc8152f00 con 0x7f7fc8152b20 2024-03-04T08:01:16.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fc7e00640 1 -- 172.21.15.113:0/633117566 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 439891849 0 0) 0x7f7fb0003530 con 0x7f7fc815c510 2024-03-04T08:01:16.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fccc00640 1 -- 172.21.15.113:0/633117566 >> v1:172.21.15.136:6789/0 conn(0x7f7fc8152b20 legacy=0x7f7fc815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:16.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fccc00640 1 -- 172.21.15.113:0/633117566 >> v1:172.21.15.113:6789/0 conn(0x7f7fc815c510 legacy=0x7f7fc8163190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:16.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.497+0000 7f7fccc00640 1 -- 172.21.15.113:0/633117566 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc80c3250 con 0x7f7fc8070010 2024-03-04T08:01:16.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fccc00640 1 --2- 172.21.15.113:0/633117566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8070010 0x7f7fc80c8b00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40097f0 tx=0x7f7fb401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e56fa49af384a349 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:16.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fc7e00640 1 -- 172.21.15.113:0/633117566 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40b0070 con 0x7f7fc8070010 2024-03-04T08:01:16.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fc7e00640 1 -- 172.21.15.113:0/633117566 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb40ab400 con 0x7f7fc8070010 2024-03-04T08:01:16.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fc7e00640 1 -- 172.21.15.113:0/633117566 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0001a00 con 0x7f7fc8070010 2024-03-04T08:01:16.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fd0000640 1 -- 172.21.15.113:0/633117566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8070010 msgr2=0x7f7fc80c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:16.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fd0000640 1 --2- 172.21.15.113:0/633117566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8070010 0x7f7fc80c8b00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40097f0 tx=0x7f7fb401c400 comp rx=0 tx=0).stop 2024-03-04T08:01:16.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fd0000640 1 -- 172.21.15.113:0/633117566 shutdown_connections 2024-03-04T08:01:16.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fd0000640 1 --2- 172.21.15.113:0/633117566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8070010 0x7f7fc80c8b00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:16.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fd0000640 1 -- 172.21.15.113:0/633117566 >> 172.21.15.113:0/633117566 conn(0x7f7fc80709a0 msgr2=0x7f7fc8070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:16.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fd0000640 1 -- 172.21.15.113:0/633117566 shutdown_connections 2024-03-04T08:01:16.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.501+0000 7f7fd0000640 1 -- 172.21.15.113:0/633117566 wait complete. 2024-03-04T08:01:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fd0000640 1 Processor -- start 2024-03-04T08:01:16.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fd0000640 1 -- start start 2024-03-04T08:01:16.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fd0000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc8070010 0x7f7fc81f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:16.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fd0000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8152b20 0x7f7fc8151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:16.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fd0000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc815c510 0x7f7fc8154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:16.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fd0000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80c8490 con 0x7f7fc8070010 2024-03-04T08:01:16.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fd0000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc8158810 con 0x7f7fc815c510 2024-03-04T08:01:16.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fd0000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc80c70a0 con 0x7f7fc8152b20 2024-03-04T08:01:16.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fcd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc8070010 0x7f7fc81f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:16.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fcd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc8070010 0x7f7fc81f70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59838/0 (socket says 172.21.15.113:59838) 2024-03-04T08:01:16.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fcd600640 1 -- 172.21.15.113:0/4152276827 learned_addr learned my addr 172.21.15.113:0/4152276827 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fccc00640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8152b20 0x7f7fc8151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fce000640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc815c510 0x7f7fc8154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fcd600640 1 -- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8152b20 msgr2=0x7f7fc8151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fcd600640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8152b20 0x7f7fc8151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fcd600640 1 -- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc815c510 msgr2=0x7f7fc8154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.505+0000 7f7fcd600640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc815c510 0x7f7fc8154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.509+0000 7f7fcd600640 1 -- 172.21.15.113:0/4152276827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc815ba40 con 0x7f7fc8070010 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.509+0000 7f7fccc00640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8152b20 0x7f7fc8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.509+0000 7f7fcd600640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc8070010 0x7f7fc81f70b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00c430 tx=0x7f7fbc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.509+0000 7f7fc5600640 1 -- 172.21.15.113:0/4152276827 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc017070 con 0x7f7fc8070010 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.509+0000 7f7fc5600640 1 -- 172.21.15.113:0/4152276827 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc0052f0 con 0x7f7fc8070010 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.509+0000 7f7fc5600640 1 -- 172.21.15.113:0/4152276827 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc003840 con 0x7f7fc8070010 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.509+0000 7f7fd0000640 1 -- 172.21.15.113:0/4152276827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc815bd20 con 0x7f7fc8070010 2024-03-04T08:01:16.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.509+0000 7f7fd0000640 1 -- 172.21.15.113:0/4152276827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc81f4010 con 0x7f7fc8070010 2024-03-04T08:01:16.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.509+0000 7f7fd0000640 1 -- 172.21.15.113:0/4152276827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc81f3b20 con 0x7f7fc8070010 2024-03-04T08:01:16.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.509+0000 7f7fc5600640 1 -- 172.21.15.113:0/4152276827 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fbc01a070 con 0x7f7fc8070010 2024-03-04T08:01:16.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.513+0000 7f7fc5600640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa8066f60 0x7f7fa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:16.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.513+0000 7f7fc5600640 1 -- 172.21.15.113:0/4152276827 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f7fbc084990 con 0x7f7fc8070010 2024-03-04T08:01:16.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.513+0000 7f7fccc00640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa8066f60 0x7f7fa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:16.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.513+0000 7f7fccc00640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa8066f60 0x7f7fa8069420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40ad000 tx=0x7f7fb4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:16.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.517+0000 7f7fc5600640 1 -- 172.21.15.113:0/4152276827 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7fbc055470 con 0x7f7fc8070010 2024-03-04T08:01:16.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.769+0000 7f7fd0000640 1 -- 172.21.15.113:0/4152276827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fc8075c00 con 0x7f7fc8070010 2024-03-04T08:01:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.769+0000 7f7fc5600640 1 -- 172.21.15.113:0/4152276827 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+6620 (secure 0 0 0) 0x7f7fbc054d70 con 0x7f7fc8070010 2024-03-04T08:01:16.769 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:16.769 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534,6053]},"id":1}]} 2024-03-04T08:01:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 100 2024-03-04T08:01:16.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 -- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa8066f60 msgr2=0x7f7fa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:16.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa8066f60 0x7f7fa8069420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40ad000 tx=0x7f7fb4008040 comp rx=0 tx=0).stop 2024-03-04T08:01:16.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 -- 172.21.15.113:0/4152276827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc8070010 msgr2=0x7f7fc81f70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:16.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc8070010 0x7f7fc81f70b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00c430 tx=0x7f7fbc00c8f0 comp rx=0 tx=0).stop 2024-03-04T08:01:16.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 -- 172.21.15.113:0/4152276827 shutdown_connections 2024-03-04T08:01:16.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc815c510 0x7f7fc8154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:16.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc8152b20 0x7f7fc8151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:16.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa8066f60 0x7f7fa8069420 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:16.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 --2- 172.21.15.113:0/4152276827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc8070010 0x7f7fc81f70b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:16.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 -- 172.21.15.113:0/4152276827 >> 172.21.15.113:0/4152276827 conn(0x7f7fc80709a0 msgr2=0x7f7fc806f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:16.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 -- 172.21.15.113:0/4152276827 shutdown_connections 2024-03-04T08:01:16.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:16.773+0000 7f7fd0000640 1 -- 172.21.15.113:0/4152276827 wait complete. 2024-03-04T08:01:16.790 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:01:18.791 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:18.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267e00640 1 Processor -- start 2024-03-04T08:01:18.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267e00640 1 -- start start 2024-03-04T08:01:18.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268152a90 0x7f92680730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:18.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f926815bce0 0x7f926815c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:18.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926806f1b0 con 0x7f926806cd10 2024-03-04T08:01:18.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9268152e70 con 0x7f926815bce0 2024-03-04T08:01:18.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926806c320 con 0x7f9268152a90 2024-03-04T08:01:18.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f925e000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f926806cd10 0x7f926806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:32798/0 (socket says 172.21.15.113:32798) 2024-03-04T08:01:18.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f925e000640 1 -- 172.21.15.113:0/1593809826 learned_addr learned my addr 172.21.15.113:0/1593809826 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:18.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267400640 1 --2- 172.21.15.113:0/1593809826 >> v2:172.21.15.113:3300/0 conn(0x7f926815bce0 0x7f926815c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:18.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9266a00640 1 --2- 172.21.15.113:0/1593809826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268152a90 0x7f92680730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:18.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9266000640 1 -- 172.21.15.113:0/1593809826 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934001905 0 0) 0x7f926806f1b0 con 0x7f926806cd10 2024-03-04T08:01:18.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9266000640 1 -- 172.21.15.113:0/1593809826 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f924c003530 con 0x7f926806cd10 2024-03-04T08:01:18.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267400640 1 -- 172.21.15.113:0/1593809826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268152a90 msgr2=0x7f92680730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:18.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267400640 1 --2- 172.21.15.113:0/1593809826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268152a90 0x7f92680730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:18.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267400640 1 -- 172.21.15.113:0/1593809826 >> v1:172.21.15.113:6789/0 conn(0x7f926806cd10 legacy=0x7f926806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:18.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.977+0000 7f9267400640 1 -- 172.21.15.113:0/1593809826 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9268071580 con 0x7f926815bce0 2024-03-04T08:01:18.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267400640 1 --2- 172.21.15.113:0/1593809826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f926815bce0 0x7f926815c150 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9260009d90 tx=0x7f926001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a47365509a1e0c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:18.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9266000640 1 -- 172.21.15.113:0/1593809826 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92600b2070 con 0x7f926815bce0 2024-03-04T08:01:18.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9266000640 1 -- 172.21.15.113:0/1593809826 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92600a5830 con 0x7f926815bce0 2024-03-04T08:01:18.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9266000640 1 -- 172.21.15.113:0/1593809826 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92600a5c00 con 0x7f926815bce0 2024-03-04T08:01:18.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 -- 172.21.15.113:0/1593809826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f926815bce0 msgr2=0x7f926815c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:18.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 --2- 172.21.15.113:0/1593809826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f926815bce0 0x7f926815c150 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9260009d90 tx=0x7f926001c970 comp rx=0 tx=0).stop 2024-03-04T08:01:18.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 -- 172.21.15.113:0/1593809826 shutdown_connections 2024-03-04T08:01:18.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 --2- 172.21.15.113:0/1593809826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f926815bce0 0x7f926815c150 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:18.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 --2- 172.21.15.113:0/1593809826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268152a90 0x7f92680730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:18.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 -- 172.21.15.113:0/1593809826 >> 172.21.15.113:0/1593809826 conn(0x7f926806e2a0 msgr2=0x7f926806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:18.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 -- 172.21.15.113:0/1593809826 shutdown_connections 2024-03-04T08:01:18.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 -- 172.21.15.113:0/1593809826 wait complete. 2024-03-04T08:01:18.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 Processor -- start 2024-03-04T08:01:18.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 -- start start 2024-03-04T08:01:18.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926806cd10 0x7f926815b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:18.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.985+0000 7f9267e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9268152a90 0x7f9268140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:18.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f9267e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f926815bce0 0x7f92681450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:18.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f9267e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926815caa0 con 0x7f9268152a90 2024-03-04T08:01:18.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f9267e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9268071330 con 0x7f926806cd10 2024-03-04T08:01:18.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f9267e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9268165aa0 con 0x7f926815bce0 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9268152a90 0x7f9268140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9268152a90 0x7f9268140770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53128/0 (socket says 172.21.15.113:53128) 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925e000640 1 -- 172.21.15.113:0/753738887 learned_addr learned my addr 172.21.15.113:0/753738887 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f9267400640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f926815bce0 0x7f92681450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f9266a00640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926806cd10 0x7f926815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925e000640 1 -- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f926815bce0 msgr2=0x7f92681450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925e000640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f926815bce0 0x7f92681450b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925e000640 1 -- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926806cd10 msgr2=0x7f926815b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925e000640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926806cd10 0x7f926815b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925e000640 1 -- 172.21.15.113:0/753738887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92680cb180 con 0x7f9268152a90 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f9266a00640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926806cd10 0x7f926815b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925e000640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9268152a90 0x7f9268140770 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f925000c3c0 tx=0x7f925000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925f400640 1 -- 172.21.15.113:0/753738887 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9250017070 con 0x7f9268152a90 2024-03-04T08:01:18.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925f400640 1 -- 172.21.15.113:0/753738887 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9250005db0 con 0x7f9268152a90 2024-03-04T08:01:18.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925f400640 1 -- 172.21.15.113:0/753738887 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92500072f0 con 0x7f9268152a90 2024-03-04T08:01:18.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f9267e00640 1 -- 172.21.15.113:0/753738887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9268140cb0 con 0x7f9268152a90 2024-03-04T08:01:18.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f9267e00640 1 -- 172.21.15.113:0/753738887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92680cb860 con 0x7f9268152a90 2024-03-04T08:01:18.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f9267e00640 1 -- 172.21.15.113:0/753738887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9268075a80 con 0x7f9268152a90 2024-03-04T08:01:18.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.989+0000 7f925f400640 1 -- 172.21.15.113:0/753738887 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f925001a070 con 0x7f9268152a90 2024-03-04T08:01:18.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.993+0000 7f925f400640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9244066f60 0x7f9244069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:18.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.993+0000 7f925f400640 1 -- 172.21.15.113:0/753738887 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f9250084960 con 0x7f9268152a90 2024-03-04T08:01:18.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.993+0000 7f9266a00640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9244066f60 0x7f9244069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:18.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.993+0000 7f9266a00640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9244066f60 0x7f9244069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9258009b10 tx=0x7f9258008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:18.997+0000 7f925f400640 1 -- 172.21.15.113:0/753738887 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9250054d50 con 0x7f9268152a90 2024-03-04T08:01:19.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.185+0000 7f9267e00640 1 -- 172.21.15.113:0/753738887 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9268074ae0 con 0x7f9268152a90 2024-03-04T08:01:19.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f925f400640 1 -- 172.21.15.113:0/753738887 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+6620 (secure 0 0 0) 0x7f9250058df0 con 0x7f9268152a90 2024-03-04T08:01:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:19.187 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534,6053]},"id":1}]} 2024-03-04T08:01:19.187 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 100 2024-03-04T08:01:19.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 -- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9244066f60 msgr2=0x7f9244069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:19.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9244066f60 0x7f9244069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9258009b10 tx=0x7f9258008040 comp rx=0 tx=0).stop 2024-03-04T08:01:19.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 -- 172.21.15.113:0/753738887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9268152a90 msgr2=0x7f9268140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:19.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9268152a90 0x7f9268140770 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f925000c3c0 tx=0x7f925000c880 comp rx=0 tx=0).stop 2024-03-04T08:01:19.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 -- 172.21.15.113:0/753738887 shutdown_connections 2024-03-04T08:01:19.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f926815bce0 0x7f92681450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:19.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9244066f60 0x7f9244069420 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:19.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9268152a90 0x7f9268140770 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:19.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 --2- 172.21.15.113:0/753738887 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f926806cd10 0x7f926815b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:19.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 -- 172.21.15.113:0/753738887 >> 172.21.15.113:0/753738887 conn(0x7f926806e2a0 msgr2=0x7f9268150c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:19.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 -- 172.21.15.113:0/753738887 shutdown_connections 2024-03-04T08:01:19.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:19.189+0000 7f9267e00640 1 -- 172.21.15.113:0/753738887 wait complete. 2024-03-04T08:01:19.210 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:01:21.211 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:21.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.349+0000 7f88d7a00640 1 Processor -- start 2024-03-04T08:01:21.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.349+0000 7f88d7a00640 1 -- start start 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88d7a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88d00c8200 0x7f88d0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88d7a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f88d0152bd0 0x7f88d01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88d7a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d01591f0 con 0x7f88d01527f0 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88d7a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d0071df0 con 0x7f88d00c8200 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88d7a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d01597b0 con 0x7f88d0152bd0 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88c7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88d00c8200 0x7f88d0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88c7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f88d00c8200 0x7f88d0070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53142/0 (socket says 172.21.15.113:53142) 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88c7e00640 1 -- 172.21.15.113:0/1720659100 learned_addr learned my addr 172.21.15.113:0/1720659100 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88d5a00640 1 --2- 172.21.15.113:0/1720659100 >> v2:172.21.15.136:3300/0 conn(0x7f88d0152bd0 0x7f88d01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88c7e00640 1 -- 172.21.15.113:0/1720659100 >> v2:172.21.15.136:3300/0 conn(0x7f88d0152bd0 msgr2=0x7f88d01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88c7e00640 1 --2- 172.21.15.113:0/1720659100 >> v2:172.21.15.136:3300/0 conn(0x7f88d0152bd0 0x7f88d01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88c7e00640 1 -- 172.21.15.113:0/1720659100 >> v1:172.21.15.136:6789/0 conn(0x7f88d01527f0 legacy=0x7f88d0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:21.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.353+0000 7f88c7e00640 1 -- 172.21.15.113:0/1720659100 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d01594d0 con 0x7f88d00c8200 2024-03-04T08:01:21.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88c7e00640 1 --2- 172.21.15.113:0/1720659100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d00c8200 0x7f88d0070d40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f88d00712b0 tx=0x7f88b801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90152ba07b741d3b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88cfe00640 1 -- 172.21.15.113:0/1720659100 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80ab430 con 0x7f88d00c8200 2024-03-04T08:01:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88cfe00640 1 -- 172.21.15.113:0/1720659100 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b80b0070 con 0x7f88d00c8200 2024-03-04T08:01:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88cfe00640 1 -- 172.21.15.113:0/1720659100 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80ab7f0 con 0x7f88d00c8200 2024-03-04T08:01:21.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 -- 172.21.15.113:0/1720659100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d00c8200 msgr2=0x7f88d0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:21.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 --2- 172.21.15.113:0/1720659100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d00c8200 0x7f88d0070d40 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f88d00712b0 tx=0x7f88b801c520 comp rx=0 tx=0).stop 2024-03-04T08:01:21.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 -- 172.21.15.113:0/1720659100 shutdown_connections 2024-03-04T08:01:21.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 --2- 172.21.15.113:0/1720659100 >> v2:172.21.15.136:3300/0 conn(0x7f88d0152bd0 0x7f88d01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:21.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 --2- 172.21.15.113:0/1720659100 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d00c8200 0x7f88d0070d40 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:21.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 -- 172.21.15.113:0/1720659100 >> 172.21.15.113:0/1720659100 conn(0x7f88d006e820 msgr2=0x7f88d006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:21.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 -- 172.21.15.113:0/1720659100 shutdown_connections 2024-03-04T08:01:21.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 -- 172.21.15.113:0/1720659100 wait complete. 2024-03-04T08:01:21.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 Processor -- start 2024-03-04T08:01:21.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 -- start start 2024-03-04T08:01:21.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.357+0000 7f88d7a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88d00c8200 0x7f88d00c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:21.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d7a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d01527f0 0x7f88d0148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:21.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d7a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88d0152bd0 0x7f88d015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:21.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d7a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01654f0 con 0x7f88d01527f0 2024-03-04T08:01:21.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d7a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0165370 con 0x7f88d00c8200 2024-03-04T08:01:21.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d7a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0165670 con 0x7f88d0152bd0 2024-03-04T08:01:21.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d01527f0 0x7f88d0148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:21.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d5000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88d00c8200 0x7f88d00c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:21.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d5000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88d00c8200 0x7f88d00c25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:51782/0 (socket says 172.21.15.113:51782) 2024-03-04T08:01:21.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d01527f0 0x7f88d0148990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53144/0 (socket says 172.21.15.113:53144) 2024-03-04T08:01:21.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d5000640 1 -- 172.21.15.113:0/2511643084 learned_addr learned my addr 172.21.15.113:0/2511643084 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:21.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d5a00640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88d0152bd0 0x7f88d015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:21.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88c7e00640 1 -- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88d0152bd0 msgr2=0x7f88d015ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:21.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88c7e00640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88d0152bd0 0x7f88d015ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:21.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88c7e00640 1 -- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88d00c8200 msgr2=0x7f88d00c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:21.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88c7e00640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88d00c8200 0x7f88d00c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:21.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88c7e00640 1 -- 172.21.15.113:0/2511643084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d01f4b50 con 0x7f88d01527f0 2024-03-04T08:01:21.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88c7e00640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d01527f0 0x7f88d0148990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f88d006c2a0 tx=0x7f88b801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:21.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88cd600640 1 -- 172.21.15.113:0/2511643084 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80be5f0 con 0x7f88d01527f0 2024-03-04T08:01:21.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88cd600640 1 -- 172.21.15.113:0/2511643084 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b80be7b0 con 0x7f88d01527f0 2024-03-04T08:01:21.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88cd600640 1 -- 172.21.15.113:0/2511643084 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80beba0 con 0x7f88d01527f0 2024-03-04T08:01:21.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d7a00640 1 -- 172.21.15.113:0/2511643084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d01f7dc0 con 0x7f88d01527f0 2024-03-04T08:01:21.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d7a00640 1 -- 172.21.15.113:0/2511643084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d01f4070 con 0x7f88d01527f0 2024-03-04T08:01:21.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88d7a00640 1 -- 172.21.15.113:0/2511643084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d0075c00 con 0x7f88d01527f0 2024-03-04T08:01:21.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.361+0000 7f88cd600640 1 -- 172.21.15.113:0/2511643084 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88b80b4600 con 0x7f88d01527f0 2024-03-04T08:01:21.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.365+0000 7f88cd600640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f88ac066f60 0x7f88ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:21.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.365+0000 7f88cd600640 1 -- 172.21.15.113:0/2511643084 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f88b80a8030 con 0x7f88d01527f0 2024-03-04T08:01:21.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.365+0000 7f88d5000640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f88ac066f60 0x7f88ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:21.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.365+0000 7f88d5000640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f88ac066f60 0x7f88ac069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f88c0007fd0 tx=0x7f88c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:21.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.369+0000 7f88cd600640 1 -- 172.21.15.113:0/2511643084 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f88b80c6020 con 0x7f88d01527f0 2024-03-04T08:01:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.681+0000 7f88d7a00640 1 -- 172.21.15.113:0/2511643084 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88d00c79d0 con 0x7f88d01527f0 2024-03-04T08:01:21.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.681+0000 7f88cd600640 1 -- 172.21.15.113:0/2511643084 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+6620 (secure 0 0 0) 0x7f88b8016110 con 0x7f88d01527f0 2024-03-04T08:01:21.682 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:21.682 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534,6053]},"id":1}]} 2024-03-04T08:01:21.682 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 100 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 -- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f88ac066f60 msgr2=0x7f88ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f88ac066f60 0x7f88ac069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f88c0007fd0 tx=0x7f88c0009040 comp rx=0 tx=0).stop 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 -- 172.21.15.113:0/2511643084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d01527f0 msgr2=0x7f88d0148990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d01527f0 0x7f88d0148990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f88d006c2a0 tx=0x7f88b801ad60 comp rx=0 tx=0).stop 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 -- 172.21.15.113:0/2511643084 shutdown_connections 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f88d0152bd0 0x7f88d015ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f88ac066f60 0x7f88ac069420 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88d01527f0 0x7f88d0148990 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 --2- 172.21.15.113:0/2511643084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f88d00c8200 0x7f88d00c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 -- 172.21.15.113:0/2511643084 >> 172.21.15.113:0/2511643084 conn(0x7f88d006e820 msgr2=0x7f88d0072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 -- 172.21.15.113:0/2511643084 shutdown_connections 2024-03-04T08:01:21.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:21.685+0000 7f88d7a00640 1 -- 172.21.15.113:0/2511643084 wait complete. 2024-03-04T08:01:21.703 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:01:23.704 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:23.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f084ca00640 1 Processor -- start 2024-03-04T08:01:23.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f084ca00640 1 -- start start 2024-03-04T08:01:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f084ca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0848152950 0x7f084815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f084ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084806faa0 0x7f08480c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f084ca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08480c4ce0 con 0x7f084806ed80 2024-03-04T08:01:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f084ca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08480c8370 con 0x7f0848152950 2024-03-04T08:01:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f084ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08480ca0a0 con 0x7f084806faa0 2024-03-04T08:01:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0847200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f084806ed80 0x7f08480ca530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:32812/0 (socket says 172.21.15.113:32812) 2024-03-04T08:01:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0847200640 1 -- 172.21.15.113:0/346525032 learned_addr learned my addr 172.21.15.113:0/346525032 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f083fe00640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084806faa0 0x7f08480c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0846800640 1 --2- 172.21.15.113:0/346525032 >> v2:172.21.15.136:3300/0 conn(0x7f0848152950 0x7f084815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668057743 0 0) 0x7f08480c4ce0 con 0x7f084806ed80 2024-03-04T08:01:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f082c003530 con 0x7f084806ed80 2024-03-04T08:01:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3764620187 0 0) 0x7f082c003530 con 0x7f084806ed80 2024-03-04T08:01:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084806faa0 msgr2=0x7f08480c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084806faa0 0x7f08480c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 >> v2:172.21.15.136:3300/0 conn(0x7f0848152950 msgr2=0x7f084815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 --2- 172.21.15.113:0/346525032 >> v2:172.21.15.136:3300/0 conn(0x7f0848152950 0x7f084815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08480c4f80 con 0x7f084806ed80 2024-03-04T08:01:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f0840001a20 con 0x7f084806ed80 2024-03-04T08:01:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 >> v1:172.21.15.113:6789/0 conn(0x7f084806ed80 legacy=0x7f08480ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f082c0039c0 0x7f082c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 0x7f082c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084806faa0 0x7f082c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480c8370 con 0x7f082c004810 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480ca0a0 con 0x7f082c0039c0 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.845+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c00b9d0 con 0x7f084806faa0 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f0846800640 1 --2- 172.21.15.113:0/346525032 >> v2:172.21.15.136:3300/0 conn(0x7f0848152950 0x7f084815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f083fe00640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 0x7f082c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f0847200640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084806faa0 0x7f082c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f0846800640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f082c0039c0 0x7f082c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f083fe00640 1 -- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084806faa0 msgr2=0x7f082c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f083fe00640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084806faa0 0x7f082c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f083fe00640 1 -- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f082c0039c0 msgr2=0x7f082c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f083fe00640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f082c0039c0 0x7f082c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:23.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f083fe00640 1 -- 172.21.15.113:0/346525032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c003530 con 0x7f082c004810 2024-03-04T08:01:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f0847200640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084806faa0 0x7f082c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.849+0000 7f0846800640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f082c0039c0 0x7f082c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f083fe00640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 0x7f082c004c90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0830009ae0 tx=0x7f083001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19cea3fe33f67f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08300b2070 con 0x7f082c004810 2024-03-04T08:01:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08300a5580 con 0x7f082c004810 2024-03-04T08:01:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f0845e00640 1 -- 172.21.15.113:0/346525032 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08300096b0 con 0x7f082c004810 2024-03-04T08:01:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f084ca00640 1 -- 172.21.15.113:0/346525032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 msgr2=0x7f082c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f084ca00640 1 --2- 172.21.15.113:0/346525032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 0x7f082c004c90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f0830009ae0 tx=0x7f083001ca70 comp rx=0 tx=0).stop 2024-03-04T08:01:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f0847200640 1 -- 172.21.15.113:0/346525032 reap_dead start 2024-03-04T08:01:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f084ca00640 1 -- 172.21.15.113:0/346525032 shutdown_connections 2024-03-04T08:01:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f084ca00640 1 -- 172.21.15.113:0/346525032 >> 172.21.15.113:0/346525032 conn(0x7f0848072f40 msgr2=0x7f08480706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f084ca00640 1 -- 172.21.15.113:0/346525032 shutdown_connections 2024-03-04T08:01:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.853+0000 7f084ca00640 1 -- 172.21.15.113:0/346525032 wait complete. 2024-03-04T08:01:23.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 Processor -- start 2024-03-04T08:01:23.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 -- start start 2024-03-04T08:01:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 0x7f084814e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08480cd720 0x7f084806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08481f3c80 0x7f084806dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08480c9a50 con 0x7f082c004810 2024-03-04T08:01:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084806fe80 con 0x7f08481f3c80 2024-03-04T08:01:23.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08480c9d30 con 0x7f08480cd720 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0846800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 0x7f084814e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0846800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 0x7f084814e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53174/0 (socket says 172.21.15.113:53174) 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0846800640 1 -- 172.21.15.113:0/399836186 learned_addr learned my addr 172.21.15.113:0/399836186 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f083fe00640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08480cd720 0x7f084806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0847200640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08481f3c80 0x7f084806dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0846800640 1 -- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08480cd720 msgr2=0x7f084806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0846800640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08480cd720 0x7f084806d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0846800640 1 -- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08481f3c80 msgr2=0x7f084806dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0846800640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08481f3c80 0x7f084806dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0846800640 1 -- 172.21.15.113:0/399836186 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08481f0b50 con 0x7f082c004810 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0847200640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08481f3c80 0x7f084806dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:01:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f0846800640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 0x7f084814e480 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0838011880 tx=0x7f0838011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:23.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f083ea00640 1 -- 172.21.15.113:0/399836186 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083800fb70 con 0x7f082c004810 2024-03-04T08:01:23.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f083ea00640 1 -- 172.21.15.113:0/399836186 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083800fd10 con 0x7f082c004810 2024-03-04T08:01:23.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f083ea00640 1 -- 172.21.15.113:0/399836186 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0838015660 con 0x7f082c004810 2024-03-04T08:01:23.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 -- 172.21.15.113:0/399836186 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08481f0e30 con 0x7f082c004810 2024-03-04T08:01:23.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 -- 172.21.15.113:0/399836186 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08481f13f0 con 0x7f082c004810 2024-03-04T08:01:23.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.857+0000 7f084ca00640 1 -- 172.21.15.113:0/399836186 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0848075a80 con 0x7f082c004810 2024-03-04T08:01:23.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.861+0000 7f083ea00640 1 -- 172.21.15.113:0/399836186 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0838015800 con 0x7f082c004810 2024-03-04T08:01:23.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.861+0000 7f083ea00640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066f90 0x7f0824069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:23.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.861+0000 7f083ea00640 1 -- 172.21.15.113:0/399836186 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7f0838086130 con 0x7f082c004810 2024-03-04T08:01:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.861+0000 7f083fe00640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066f90 0x7f0824069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:23.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.861+0000 7f083fe00640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066f90 0x7f0824069450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f08300af000 tx=0x7f08300046d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:23.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:23.865+0000 7f083ea00640 1 -- 172.21.15.113:0/399836186 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0838019030 con 0x7f082c004810 2024-03-04T08:01:24.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.065+0000 7f084ca00640 1 -- 172.21.15.113:0/399836186 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0848159710 con 0x7f082c004810 2024-03-04T08:01:24.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.065+0000 7f083ea00640 1 -- 172.21.15.113:0/399836186 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+6620 (secure 0 0 0) 0x7f0838056c10 con 0x7f082c004810 2024-03-04T08:01:24.066 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:24.066 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534,6053]},"id":1}]} 2024-03-04T08:01:24.067 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 100 2024-03-04T08:01:24.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 -- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066f90 msgr2=0x7f0824069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066f90 0x7f0824069450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f08300af000 tx=0x7f08300046d0 comp rx=0 tx=0).stop 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 -- 172.21.15.113:0/399836186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 msgr2=0x7f084814e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 0x7f084814e480 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f0838011880 tx=0x7f0838011d40 comp rx=0 tx=0).stop 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 -- 172.21.15.113:0/399836186 shutdown_connections 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08481f3c80 0x7f084806dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f08480cd720 0x7f084806d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066f90 0x7f0824069450 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 --2- 172.21.15.113:0/399836186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f082c004810 0x7f084814e480 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 -- 172.21.15.113:0/399836186 >> 172.21.15.113:0/399836186 conn(0x7f0848072f40 msgr2=0x7f08480705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 -- 172.21.15.113:0/399836186 shutdown_connections 2024-03-04T08:01:24.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:24.069+0000 7f084ca00640 1 -- 172.21.15.113:0/399836186 wait complete. 2024-03-04T08:01:24.096 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:01:26.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.237+0000 7fe91a600640 1 Processor -- start 2024-03-04T08:01:26.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.237+0000 7fe91a600640 1 -- start start 2024-03-04T08:01:26.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe91a600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe91414e4c0 0x7fe914072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe91a600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe91415b900 0x7fe9140731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe91a600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9140c27f0 con 0x7fe9140c22e0 2024-03-04T08:01:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe91a600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91415ad60 con 0x7fe91415b900 2024-03-04T08:01:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe91a600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140c2a90 con 0x7fe91414e4c0 2024-03-04T08:01:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe913e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe91415b900 0x7fe9140731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe913400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe91414e4c0 0x7fe914072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe913e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe91415b900 0x7fe9140731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:51824/0 (socket says 172.21.15.113:51824) 2024-03-04T08:01:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe90aa00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fe9140c22e0 0x7fe91415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:52664/0 (socket says 172.21.15.113:52664) 2024-03-04T08:01:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe913e00640 1 -- 172.21.15.113:0/3878550285 learned_addr learned my addr 172.21.15.113:0/3878550285 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe912a00640 1 -- 172.21.15.113:0/3878550285 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891228713 0 0) 0x7fe9140c27f0 con 0x7fe9140c22e0 2024-03-04T08:01:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe912a00640 1 -- 172.21.15.113:0/3878550285 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f8003530 con 0x7fe9140c22e0 2024-03-04T08:01:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe913e00640 1 -- 172.21.15.113:0/3878550285 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe91414e4c0 msgr2=0x7fe914072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe913e00640 1 --2- 172.21.15.113:0/3878550285 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe91414e4c0 0x7fe914072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe913e00640 1 -- 172.21.15.113:0/3878550285 >> v1:172.21.15.136:6789/0 conn(0x7fe9140c22e0 legacy=0x7fe91415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe913e00640 1 -- 172.21.15.113:0/3878550285 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe914070c60 con 0x7fe91415b900 2024-03-04T08:01:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.241+0000 7fe913400640 1 --2- 172.21.15.113:0/3878550285 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe91414e4c0 0x7fe914072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:01:26.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe913e00640 1 --2- 172.21.15.113:0/3878550285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe91415b900 0x7fe9140731c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009e50 tx=0x7fe90c01f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=538451a85a0a3079 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:26.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe912a00640 1 -- 172.21.15.113:0/3878550285 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0b4070 con 0x7fe91415b900 2024-03-04T08:01:26.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe912a00640 1 -- 172.21.15.113:0/3878550285 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c0af400 con 0x7fe91415b900 2024-03-04T08:01:26.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe912a00640 1 -- 172.21.15.113:0/3878550285 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0af7b0 con 0x7fe91415b900 2024-03-04T08:01:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe91a600640 1 -- 172.21.15.113:0/3878550285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe91415b900 msgr2=0x7fe9140731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe91a600640 1 --2- 172.21.15.113:0/3878550285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe91415b900 0x7fe9140731c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009e50 tx=0x7fe90c01f210 comp rx=0 tx=0).stop 2024-03-04T08:01:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe91a600640 1 -- 172.21.15.113:0/3878550285 shutdown_connections 2024-03-04T08:01:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe91a600640 1 --2- 172.21.15.113:0/3878550285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe91415b900 0x7fe9140731c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe91a600640 1 --2- 172.21.15.113:0/3878550285 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe91414e4c0 0x7fe914072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe91a600640 1 -- 172.21.15.113:0/3878550285 >> 172.21.15.113:0/3878550285 conn(0x7fe91406e820 msgr2=0x7fe91406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe91a600640 1 -- 172.21.15.113:0/3878550285 shutdown_connections 2024-03-04T08:01:26.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe91a600640 1 -- 172.21.15.113:0/3878550285 wait complete. 2024-03-04T08:01:26.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe91a600640 1 Processor -- start 2024-03-04T08:01:26.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.245+0000 7fe91a600640 1 -- start start 2024-03-04T08:01:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe91a600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9140c22e0 0x7fe914145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe91a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe91414e4c0 0x7fe9141f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe91a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe91415b900 0x7fe914148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe91a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe914071500 con 0x7fe91415b900 2024-03-04T08:01:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe91a600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe914071220 con 0x7fe91414e4c0 2024-03-04T08:01:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe91a600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91414e8a0 con 0x7fe9140c22e0 2024-03-04T08:01:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe913400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9140c22e0 0x7fe914145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe913e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe91415b900 0x7fe914148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe913400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9140c22e0 0x7fe914145530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:37118/0 (socket says 172.21.15.113:37118) 2024-03-04T08:01:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe913400640 1 -- 172.21.15.113:0/1508006774 learned_addr learned my addr 172.21.15.113:0/1508006774 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe90aa00640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe91414e4c0 0x7fe9141f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe913e00640 1 -- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9140c22e0 msgr2=0x7fe914145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe913e00640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9140c22e0 0x7fe914145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe913e00640 1 -- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe91414e4c0 msgr2=0x7fe9141f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe913e00640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe91414e4c0 0x7fe9141f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe913e00640 1 -- 172.21.15.113:0/1508006774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9141f22b0 con 0x7fe91415b900 2024-03-04T08:01:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe913e00640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe91415b900 0x7fe914148280 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0ae000 tx=0x7fe90c01bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe90be00640 1 -- 172.21.15.113:0/1508006774 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0c0070 con 0x7fe91415b900 2024-03-04T08:01:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe90be00640 1 -- 172.21.15.113:0/1508006774 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c0090d0 con 0x7fe91415b900 2024-03-04T08:01:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe90be00640 1 -- 172.21.15.113:0/1508006774 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0b4040 con 0x7fe91415b900 2024-03-04T08:01:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe91a600640 1 -- 172.21.15.113:0/1508006774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91414ca10 con 0x7fe91415b900 2024-03-04T08:01:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe91a600640 1 -- 172.21.15.113:0/1508006774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9141f2550 con 0x7fe91415b900 2024-03-04T08:01:26.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe91a600640 1 -- 172.21.15.113:0/1508006774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe914075c00 con 0x7fe91415b900 2024-03-04T08:01:26.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.249+0000 7fe90be00640 1 -- 172.21.15.113:0/1508006774 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe90c0c3070 con 0x7fe91415b900 2024-03-04T08:01:26.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.253+0000 7fe90be00640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe8f0066f60 0x7fe8f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:26.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.253+0000 7fe90be00640 1 -- 172.21.15.113:0/1508006774 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7fe90c122930 con 0x7fe91415b900 2024-03-04T08:01:26.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.253+0000 7fe913400640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe8f0066f60 0x7fe8f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:26.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.253+0000 7fe913400640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe8f0066f60 0x7fe8f0069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe904009600 tx=0x7fe904009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:26.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.257+0000 7fe90be00640 1 -- 172.21.15.113:0/1508006774 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe90c018d70 con 0x7fe91415b900 2024-03-04T08:01:26.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.453+0000 7fe91a600640 1 -- 172.21.15.113:0/1508006774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe914151d90 con 0x7fe91415b900 2024-03-04T08:01:26.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.453+0000 7fe90be00640 1 -- 172.21.15.113:0/1508006774 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+6620 (secure 0 0 0) 0x7fe90c0f2db0 con 0x7fe91415b900 2024-03-04T08:01:26.453 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:26.453 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534,6053]},"id":1}]} 2024-03-04T08:01:26.453 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 100 2024-03-04T08:01:26.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 -- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe8f0066f60 msgr2=0x7fe8f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:26.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe8f0066f60 0x7fe8f0069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe904009600 tx=0x7fe904009190 comp rx=0 tx=0).stop 2024-03-04T08:01:26.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 -- 172.21.15.113:0/1508006774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe91415b900 msgr2=0x7fe914148280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:26.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe91415b900 0x7fe914148280 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fe90c0ae000 tx=0x7fe90c01bfb0 comp rx=0 tx=0).stop 2024-03-04T08:01:26.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 -- 172.21.15.113:0/1508006774 shutdown_connections 2024-03-04T08:01:26.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe8f0066f60 0x7fe8f0069420 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:26.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe91415b900 0x7fe914148280 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:26.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe91414e4c0 0x7fe9141f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:26.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 --2- 172.21.15.113:0/1508006774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9140c22e0 0x7fe914145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:26.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 -- 172.21.15.113:0/1508006774 >> 172.21.15.113:0/1508006774 conn(0x7fe91406e820 msgr2=0x7fe9140c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:26.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 -- 172.21.15.113:0/1508006774 shutdown_connections 2024-03-04T08:01:26.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:26.457+0000 7fe91a600640 1 -- 172.21.15.113:0/1508006774 wait complete. 2024-03-04T08:01:26.475 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:01:28.476 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa25000640 1 Processor -- start 2024-03-04T08:01:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa25000640 1 -- start start 2024-03-04T08:01:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa25000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffa20152bd0 0x7ffa20070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa25000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa201591f0 con 0x7ffa201527f0 2024-03-04T08:01:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa25000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa201594d0 con 0x7ffa200c8200 2024-03-04T08:01:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa25000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa20071df0 con 0x7ffa20152bd0 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1de00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ffa201527f0 0x7ffa20072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50332/0 (socket says 172.21.15.113:50332) 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1de00640 1 -- 172.21.15.113:0/2523552236 learned_addr learned my addr 172.21.15.113:0/2523552236 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1e800640 1 --2- 172.21.15.113:0/2523552236 >> v2:172.21.15.136:3300/0 conn(0x7ffa20152bd0 0x7ffa20070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157683827 0 0) 0x7ffa201591f0 con 0x7ffa201527f0 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa04003530 con 0x7ffa201527f0 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302227538 0 0) 0x7ffa201594d0 con 0x7ffa200c8200 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa201591f0 con 0x7ffa200c8200 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2948107515 0 0) 0x7ffa04003530 con 0x7ffa201527f0 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 >> v2:172.21.15.136:3300/0 conn(0x7ffa20152bd0 msgr2=0x7ffa20070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 --2- 172.21.15.113:0/2523552236 >> v2:172.21.15.136:3300/0 conn(0x7ffa20152bd0 0x7ffa20070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 >> v1:172.21.15.136:6789/0 conn(0x7ffa200c8200 legacy=0x7ffa201587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa200717a0 con 0x7ffa201527f0 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7ffa10001680 con 0x7ffa201527f0 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 >> v1:172.21.15.113:6789/0 conn(0x7ffa201527f0 legacy=0x7ffa20072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa04003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa04004810 0x7ffa04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa040051d0 0x7ffa0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20071df0 con 0x7ffa04003ba0 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0400c0a0 con 0x7ffa04004810 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0400c2a0 con 0x7ffa040051d0 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1de00640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa04003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa15400640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa04004810 0x7ffa04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1e800640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa040051d0 0x7ffa0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1de00640 1 -- 172.21.15.113:0/2523552236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa040051d0 msgr2=0x7ffa0400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1de00640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa040051d0 0x7ffa0400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1de00640 1 -- 172.21.15.113:0/2523552236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa04004810 msgr2=0x7ffa04004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1de00640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa04004810 0x7ffa04004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1de00640 1 -- 172.21.15.113:0/2523552236 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04003530 con 0x7ffa04003ba0 2024-03-04T08:01:28.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.617+0000 7ffa1e800640 1 -- 172.21.15.113:0/2523552236 reap_dead start 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa1de00640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa04003fa0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ffa10007fb0 tx=0x7ffa1001cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecc366026def0e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10007bd0 con 0x7ffa04003ba0 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa100a9020 con 0x7ffa04003ba0 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa1d400640 1 -- 172.21.15.113:0/2523552236 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10005b00 con 0x7ffa04003ba0 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa25000640 1 -- 172.21.15.113:0/2523552236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 msgr2=0x7ffa04003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa25000640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa04003fa0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ffa10007fb0 tx=0x7ffa1001cb40 comp rx=0 tx=0).stop 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa25000640 1 -- 172.21.15.113:0/2523552236 shutdown_connections 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa25000640 1 --2- 172.21.15.113:0/2523552236 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa04003fa0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa25000640 1 -- 172.21.15.113:0/2523552236 >> 172.21.15.113:0/2523552236 conn(0x7ffa2006e820 msgr2=0x7ffa2006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa25000640 1 -- 172.21.15.113:0/2523552236 shutdown_connections 2024-03-04T08:01:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.621+0000 7ffa25000640 1 -- 172.21.15.113:0/2523552236 wait complete. 2024-03-04T08:01:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa25000640 1 Processor -- start 2024-03-04T08:01:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa25000640 1 -- start start 2024-03-04T08:01:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa25000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa200c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa25000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa20154170 0x7ffa201f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa25000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa200c1630 0x7ffa201e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa25000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20158b60 con 0x7ffa04003ba0 2024-03-04T08:01:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa25000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa201589e0 con 0x7ffa200c1630 2024-03-04T08:01:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa25000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa20158ce0 con 0x7ffa20154170 2024-03-04T08:01:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa1de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa200c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa1de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa200c2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36394/0 (socket says 172.21.15.113:36394) 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa1de00640 1 -- 172.21.15.113:0/3919468176 learned_addr learned my addr 172.21.15.113:0/3919468176 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa15400640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa20154170 0x7ffa201f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa1e800640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa200c1630 0x7ffa201e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa1de00640 1 -- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa20154170 msgr2=0x7ffa201f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa1de00640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa20154170 0x7ffa201f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa1de00640 1 -- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa200c1630 msgr2=0x7ffa201e84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa1de00640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa200c1630 0x7ffa201e84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa1de00640 1 -- 172.21.15.113:0/3919468176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa20155960 con 0x7ffa04003ba0 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa1de00640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa200c2bd0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ffa1001c970 tx=0x7ffa04005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa16a00640 1 -- 172.21.15.113:0/3919468176 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10007bd0 con 0x7ffa04003ba0 2024-03-04T08:01:28.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa16a00640 1 -- 172.21.15.113:0/3919468176 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa100b2070 con 0x7ffa04003ba0 2024-03-04T08:01:28.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa16a00640 1 -- 172.21.15.113:0/3919468176 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10001b30 con 0x7ffa04003ba0 2024-03-04T08:01:28.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa25000640 1 -- 172.21.15.113:0/3919468176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa200c9210 con 0x7ffa04003ba0 2024-03-04T08:01:28.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.625+0000 7ffa25000640 1 -- 172.21.15.113:0/3919468176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa20155bd0 con 0x7ffa04003ba0 2024-03-04T08:01:28.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.629+0000 7ffa25000640 1 -- 172.21.15.113:0/3919468176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa20075c00 con 0x7ffa04003ba0 2024-03-04T08:01:28.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.629+0000 7ffa16a00640 1 -- 172.21.15.113:0/3919468176 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa10007510 con 0x7ffa04003ba0 2024-03-04T08:01:28.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.629+0000 7ffa16a00640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff9fc066f60 0x7ff9fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:28.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.629+0000 7ffa16a00640 1 -- 172.21.15.113:0/3919468176 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7029+0+0 (secure 0 0 0) 0x7ffa1011ec20 con 0x7ffa04003ba0 2024-03-04T08:01:28.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.629+0000 7ffa15400640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff9fc066f60 0x7ff9fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:28.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.633+0000 7ffa15400640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff9fc066f60 0x7ff9fc069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ffa201f4740 tx=0x7ffa08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:28.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.633+0000 7ffa16a00640 1 -- 172.21.15.113:0/3919468176 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ffa10016850 con 0x7ffa04003ba0 2024-03-04T08:01:28.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 -- 172.21.15.113:0/3919468176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa200732f0 con 0x7ffa04003ba0 2024-03-04T08:01:28.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa16a00640 1 -- 172.21.15.113:0/3919468176 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+6620 (secure 0 0 0) 0x7ffa100ef790 con 0x7ffa04003ba0 2024-03-04T08:01:28.828 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:28.828 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6807,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:02.149280+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6053":{"gid":6053,"name":"b","rank":2,"incarnation":95,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/3388011401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":3388011401},{"type":"v1","addr":"172.21.15.136:6835","nonce":3388011401}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534,6053]},"id":1}]} 2024-03-04T08:01:28.829 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 100 2024-03-04T08:01:28.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 -- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff9fc066f60 msgr2=0x7ff9fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:28.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff9fc066f60 0x7ff9fc069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ffa201f4740 tx=0x7ffa08009040 comp rx=0 tx=0).stop 2024-03-04T08:01:28.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 -- 172.21.15.113:0/3919468176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 msgr2=0x7ffa200c2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa200c2bd0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ffa1001c970 tx=0x7ffa04005840 comp rx=0 tx=0).stop 2024-03-04T08:01:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 -- 172.21.15.113:0/3919468176 shutdown_connections 2024-03-04T08:01:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa200c1630 0x7ffa201e84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa20154170 0x7ffa201f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff9fc066f60 0x7ff9fc069420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 --2- 172.21.15.113:0/3919468176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa04003ba0 0x7ffa200c2bd0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 -- 172.21.15.113:0/3919468176 >> 172.21.15.113:0/3919468176 conn(0x7ffa2006e820 msgr2=0x7ffa200cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 -- 172.21.15.113:0/3919468176 shutdown_connections 2024-03-04T08:01:28.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:28.829+0000 7ffa25000640 1 -- 172.21.15.113:0/3919468176 wait complete. 2024-03-04T08:01:28.850 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:01:30.850 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:30.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.993+0000 7fbe55a00640 1 Processor -- start 2024-03-04T08:01:30.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.993+0000 7fbe55a00640 1 -- start start 2024-03-04T08:01:30.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.993+0000 7fbe55a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbe50152b20 0x7fbe50070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:30.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.993+0000 7fbe55a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe5015c510 0x7fbe50163190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:30.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.993+0000 7fbe55a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe50152f00 con 0x7fbe500c8b00 2024-03-04T08:01:30.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe55a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe500c8f90 con 0x7fbe5015c510 2024-03-04T08:01:30.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe55a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe500c5320 con 0x7fbe50152b20 2024-03-04T08:01:30.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe4f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe5015c510 0x7fbe50163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:30.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe46000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbe500c8b00 0x7fbe5015be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50340/0 (socket says 172.21.15.113:50340) 2024-03-04T08:01:30.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe46000640 1 -- 172.21.15.113:0/1746854369 learned_addr learned my addr 172.21.15.113:0/1746854369 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:30.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe4ea00640 1 --2- 172.21.15.113:0/1746854369 >> v2:172.21.15.136:3300/0 conn(0x7fbe50152b20 0x7fbe50070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:30.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe4f400640 1 -- 172.21.15.113:0/1746854369 >> v2:172.21.15.136:3300/0 conn(0x7fbe50152b20 msgr2=0x7fbe50070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:30.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe4f400640 1 --2- 172.21.15.113:0/1746854369 >> v2:172.21.15.136:3300/0 conn(0x7fbe50152b20 0x7fbe50070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:30.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe4e000640 1 -- 172.21.15.113:0/1746854369 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929981444 0 0) 0x7fbe50152f00 con 0x7fbe500c8b00 2024-03-04T08:01:30.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe4f400640 1 -- 172.21.15.113:0/1746854369 >> v1:172.21.15.113:6789/0 conn(0x7fbe500c8b00 legacy=0x7fbe5015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:30.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:30.997+0000 7fbe4f400640 1 -- 172.21.15.113:0/1746854369 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500c5040 con 0x7fbe5015c510 2024-03-04T08:01:31.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe4f400640 1 --2- 172.21.15.113:0/1746854369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 0x7fbe50163190 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbe50072a50 tx=0x7fbe4801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f3b9387c3dd723 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:31.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe4e000640 1 -- 172.21.15.113:0/1746854369 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe480b0070 con 0x7fbe5015c510 2024-03-04T08:01:31.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe4e000640 1 -- 172.21.15.113:0/1746854369 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe480ab600 con 0x7fbe5015c510 2024-03-04T08:01:31.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe4e000640 1 -- 172.21.15.113:0/1746854369 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe480ab9d0 con 0x7fbe5015c510 2024-03-04T08:01:31.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe55a00640 1 -- 172.21.15.113:0/1746854369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 msgr2=0x7fbe50163190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe55a00640 1 --2- 172.21.15.113:0/1746854369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 0x7fbe50163190 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbe50072a50 tx=0x7fbe4801c480 comp rx=0 tx=0).stop 2024-03-04T08:01:31.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe55a00640 1 -- 172.21.15.113:0/1746854369 shutdown_connections 2024-03-04T08:01:31.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe55a00640 1 --2- 172.21.15.113:0/1746854369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 0x7fbe50163190 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe55a00640 1 --2- 172.21.15.113:0/1746854369 >> v2:172.21.15.136:3300/0 conn(0x7fbe50152b20 0x7fbe50070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe55a00640 1 -- 172.21.15.113:0/1746854369 >> 172.21.15.113:0/1746854369 conn(0x7fbe500709a0 msgr2=0x7fbe50070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:31.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe55a00640 1 -- 172.21.15.113:0/1746854369 shutdown_connections 2024-03-04T08:01:31.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.001+0000 7fbe55a00640 1 -- 172.21.15.113:0/1746854369 wait complete. 2024-03-04T08:01:31.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 Processor -- start 2024-03-04T08:01:31.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 -- start start 2024-03-04T08:01:31.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe500c8b00 0x7fbe500c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe50152b20 0x7fbe5014de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 0x7fbe500ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe50158810 con 0x7fbe5015c510 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500c3810 con 0x7fbe50152b20 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe500c8490 con 0x7fbe500c8b00 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe4f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 0x7fbe500ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe4f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 0x7fbe500ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36418/0 (socket says 172.21.15.113:36418) 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe4f400640 1 -- 172.21.15.113:0/1583923717 learned_addr learned my addr 172.21.15.113:0/1583923717 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe4ea00640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe500c8b00 0x7fbe500c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe46000640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe50152b20 0x7fbe5014de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe4f400640 1 -- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe500c8b00 msgr2=0x7fbe500c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe4f400640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe500c8b00 0x7fbe500c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe4f400640 1 -- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe50152b20 msgr2=0x7fbe5014de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe4f400640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe50152b20 0x7fbe5014de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe4f400640 1 -- 172.21.15.113:0/1583923717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe501f7a20 con 0x7fbe5015c510 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe46000640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe50152b20 0x7fbe5014de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe4f400640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 0x7fbe500ccf50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbe48019090 tx=0x7fbe48019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:31.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe47400640 1 -- 172.21.15.113:0/1583923717 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe480b0040 con 0x7fbe5015c510 2024-03-04T08:01:31.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe47400640 1 -- 172.21.15.113:0/1583923717 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe480b9070 con 0x7fbe5015c510 2024-03-04T08:01:31.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe47400640 1 -- 172.21.15.113:0/1583923717 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe480b4630 con 0x7fbe5015c510 2024-03-04T08:01:31.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 -- 172.21.15.113:0/1583923717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe501f6bc0 con 0x7fbe5015c510 2024-03-04T08:01:31.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 -- 172.21.15.113:0/1583923717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe501f7180 con 0x7fbe5015c510 2024-03-04T08:01:31.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.005+0000 7fbe55a00640 1 -- 172.21.15.113:0/1583923717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5006fdf0 con 0x7fbe5015c510 2024-03-04T08:01:31.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.009+0000 7fbe47400640 1 -- 172.21.15.113:0/1583923717 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe480b47d0 con 0x7fbe5015c510 2024-03-04T08:01:31.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.009+0000 7fbe47400640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe2c066f60 0x7fbe2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.009+0000 7fbe47400640 1 -- 172.21.15.113:0/1583923717 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7fbe4811f160 con 0x7fbe5015c510 2024-03-04T08:01:31.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.009+0000 7fbe4ea00640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe2c066f60 0x7fbe2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.009+0000 7fbe4ea00640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe2c066f60 0x7fbe2c069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbe40003ab0 tx=0x7fbe40009830 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:31.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.013+0000 7fbe47400640 1 -- 172.21.15.113:0/1583923717 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbe48016070 con 0x7fbe5015c510 2024-03-04T08:01:31.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.213+0000 7fbe55a00640 1 -- 172.21.15.113:0/1583923717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe50075be0 con 0x7fbe5015c510 2024-03-04T08:01:31.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.213+0000 7fbe47400640 1 -- 172.21.15.113:0/1583923717 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5821 (secure 0 0 0) 0x7fbe480efb60 con 0x7fbe5015c510 2024-03-04T08:01:31.212 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:31.212 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:30.805651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:01:31.212 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 101 2024-03-04T08:01:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 -- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe2c066f60 msgr2=0x7fbe2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe2c066f60 0x7fbe2c069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbe40003ab0 tx=0x7fbe40009830 comp rx=0 tx=0).stop 2024-03-04T08:01:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 -- 172.21.15.113:0/1583923717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 msgr2=0x7fbe500ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 0x7fbe500ccf50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbe48019090 tx=0x7fbe48019220 comp rx=0 tx=0).stop 2024-03-04T08:01:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 -- 172.21.15.113:0/1583923717 shutdown_connections 2024-03-04T08:01:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe2c066f60 0x7fbe2c069420 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe5015c510 0x7fbe500ccf50 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe50152b20 0x7fbe5014de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 --2- 172.21.15.113:0/1583923717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe500c8b00 0x7fbe500c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 -- 172.21.15.113:0/1583923717 >> 172.21.15.113:0/1583923717 conn(0x7fbe500709a0 msgr2=0x7fbe50150dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:31.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 -- 172.21.15.113:0/1583923717 shutdown_connections 2024-03-04T08:01:31.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.217+0000 7fbe55a00640 1 -- 172.21.15.113:0/1583923717 wait complete. 2024-03-04T08:01:31.234 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2024-03-04T08:01:31.235 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:01:31.235 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:31.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0216600640 1 Processor -- start 2024-03-04T08:01:31.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0216600640 1 -- start start 2024-03-04T08:01:31.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0216600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f021006d420 0x7f02100730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0216600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f021015bce0 0x7f021015c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0216600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021006f1b0 con 0x7f0210152a90 2024-03-04T08:01:31.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0216600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0210152e70 con 0x7f021015bce0 2024-03-04T08:01:31.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0216600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021006c320 con 0x7f021006d420 2024-03-04T08:01:31.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0215200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0210152a90 0x7f021006cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50350/0 (socket says 172.21.15.113:50350) 2024-03-04T08:01:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0215200640 1 -- 172.21.15.113:0/3130090090 learned_addr learned my addr 172.21.15.113:0/3130090090 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0215c00640 1 --2- 172.21.15.113:0/3130090090 >> v2:172.21.15.136:3300/0 conn(0x7f021015bce0 0x7f021015c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0207e00640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f021006d420 0x7f02100730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511607623 0 0) 0x7f021006f1b0 con 0x7f0210152a90 2024-03-04T08:01:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f4003530 con 0x7f0210152a90 2024-03-04T08:01:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3344931743 0 0) 0x7f01f4003530 con 0x7f0210152a90 2024-03-04T08:01:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f021006d420 msgr2=0x7f02100730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f021006d420 0x7f02100730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 >> v2:172.21.15.136:3300/0 conn(0x7f021015bce0 msgr2=0x7f021015c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 --2- 172.21.15.113:0/3130090090 >> v2:172.21.15.136:3300/0 conn(0x7f021015bce0 0x7f021015c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0210071580 con 0x7f0210152a90 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f0200001680 con 0x7f0210152a90 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 >> v1:172.21.15.113:6789/0 conn(0x7f0210152a90 legacy=0x7f021006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01f40039c0 0x7f01f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f021006d420 0x7f01f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f4004d50 0x7f01f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f400b9d0 con 0x7f01f40039c0 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f400bb50 con 0x7f01f4004d50 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f400bcd0 con 0x7f021006d420 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0215200640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01f40039c0 0x7f01f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0207e00640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f021006d420 0x7f01f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0215c00640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f4004d50 0x7f01f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0215200640 1 -- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f021006d420 msgr2=0x7f01f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:31.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0215200640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f021006d420 0x7f01f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0215200640 1 -- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f4004d50 msgr2=0x7f01f40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:31.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0215200640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f4004d50 0x7f01f40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.373+0000 7f0215200640 1 -- 172.21.15.113:0/3130090090 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4003530 con 0x7f01f40039c0 2024-03-04T08:01:31.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f0215200640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01f40039c0 0x7f01f4003de0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f020000c240 tx=0x7f020001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376a998e91650ad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:31.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200007d60 con 0x7f01f40039c0 2024-03-04T08:01:31.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02000a9020 con 0x7f01f40039c0 2024-03-04T08:01:31.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f020fe00640 1 -- 172.21.15.113:0/3130090090 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200005ae0 con 0x7f01f40039c0 2024-03-04T08:01:31.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f0216600640 1 -- 172.21.15.113:0/3130090090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01f40039c0 msgr2=0x7f01f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f0216600640 1 --2- 172.21.15.113:0/3130090090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01f40039c0 0x7f01f4003de0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f020000c240 tx=0x7f020001ccc0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f0215c00640 1 -- 172.21.15.113:0/3130090090 reap_dead start 2024-03-04T08:01:31.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f0216600640 1 -- 172.21.15.113:0/3130090090 shutdown_connections 2024-03-04T08:01:31.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f0216600640 1 -- 172.21.15.113:0/3130090090 >> 172.21.15.113:0/3130090090 conn(0x7f021006e2a0 msgr2=0x7f021006e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:31.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f0216600640 1 -- 172.21.15.113:0/3130090090 shutdown_connections 2024-03-04T08:01:31.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.377+0000 7f0216600640 1 -- 172.21.15.113:0/3130090090 wait complete. 2024-03-04T08:01:31.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0216600640 1 Processor -- start 2024-03-04T08:01:31.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0216600640 1 -- start start 2024-03-04T08:01:31.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0216600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40039c0 0x7f02100c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0216600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02100cab80 0x7f02100cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0216600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0210145930 0x7f0210145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0216600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021015caa0 con 0x7f02100cab80 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0216600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0210071330 con 0x7f01f40039c0 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0216600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0210163880 con 0x7f0210145930 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0207e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02100cab80 0x7f02100cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0207e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02100cab80 0x7f02100cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36440/0 (socket says 172.21.15.113:36440) 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0207e00640 1 -- 172.21.15.113:0/1427417994 learned_addr learned my addr 172.21.15.113:0/1427417994 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0215200640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40039c0 0x7f02100c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0215c00640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0210145930 0x7f0210145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0207e00640 1 -- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0210145930 msgr2=0x7f0210145db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0207e00640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0210145930 0x7f0210145db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0207e00640 1 -- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40039c0 msgr2=0x7f02100c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0207e00640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40039c0 0x7f02100c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0207e00640 1 -- 172.21.15.113:0/1427417994 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02101462f0 con 0x7f02100cab80 2024-03-04T08:01:31.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0207e00640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02100cab80 0x7f02100cb000 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f01f80115c0 tx=0x7f01f8011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:31.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f020d600640 1 -- 172.21.15.113:0/1427417994 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8015930 con 0x7f02100cab80 2024-03-04T08:01:31.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f020d600640 1 -- 172.21.15.113:0/1427417994 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f8015ad0 con 0x7f02100cab80 2024-03-04T08:01:31.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f020d600640 1 -- 172.21.15.113:0/1427417994 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f800f480 con 0x7f02100cab80 2024-03-04T08:01:31.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0216600640 1 -- 172.21.15.113:0/1427417994 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02101f3d50 con 0x7f02100cab80 2024-03-04T08:01:31.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.381+0000 7f0216600640 1 -- 172.21.15.113:0/1427417994 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02101f4310 con 0x7f02100cab80 2024-03-04T08:01:31.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.385+0000 7f0216600640 1 -- 172.21.15.113:0/1427417994 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0210075a80 con 0x7f02100cab80 2024-03-04T08:01:31.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.385+0000 7f020d600640 1 -- 172.21.15.113:0/1427417994 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01f800f620 con 0x7f02100cab80 2024-03-04T08:01:31.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.385+0000 7f020d600640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:31.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.385+0000 7f020d600640 1 -- 172.21.15.113:0/1427417994 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f01f8085d90 con 0x7f02100cab80 2024-03-04T08:01:31.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.385+0000 7f0215200640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:31.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.389+0000 7f0215200640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01ec066f60 0x7f01ec069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f02000aa000 tx=0x7f02000b1040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:31.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.389+0000 7f020d600640 1 -- 172.21.15.113:0/1427417994 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f02101462f0 con 0x7f02100cab80 2024-03-04T08:01:31.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 -- 172.21.15.113:0/1427417994 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0210151670 con 0x7f02100cab80 2024-03-04T08:01:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f020d600640 1 -- 172.21.15.113:0/1427417994 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5821 (secure 0 0 0) 0x7f01f8019030 con 0x7f02100cab80 2024-03-04T08:01:31.636 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:31.636 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:30.805651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:01:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 101 2024-03-04T08:01:31.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 -- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01ec066f60 msgr2=0x7f01ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01ec066f60 0x7f01ec069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f02000aa000 tx=0x7f02000b1040 comp rx=0 tx=0).stop 2024-03-04T08:01:31.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 -- 172.21.15.113:0/1427417994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02100cab80 msgr2=0x7f02100cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:31.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02100cab80 0x7f02100cb000 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f01f80115c0 tx=0x7f01f8011a80 comp rx=0 tx=0).stop 2024-03-04T08:01:31.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 -- 172.21.15.113:0/1427417994 shutdown_connections 2024-03-04T08:01:31.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0210145930 0x7f0210145db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01ec066f60 0x7f01ec069420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f02100cab80 0x7f02100cb000 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 --2- 172.21.15.113:0/1427417994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01f40039c0 0x7f02100c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 -- 172.21.15.113:0/1427417994 >> 172.21.15.113:0/1427417994 conn(0x7f021006e2a0 msgr2=0x7f02101478a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 -- 172.21.15.113:0/1427417994 shutdown_connections 2024-03-04T08:01:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:31.637+0000 7f0216600640 1 -- 172.21.15.113:0/1427417994 wait complete. 2024-03-04T08:01:31.659 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 101, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 101, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6372": { "addr": "172.21.15.113:6833/2732565639", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2732565639, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2732565639, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6372, "incarnation": 83, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 40 }, "gid_6534": { "addr": "172.21.15.113:6837/4197507712", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 4197507712, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 4197507712, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6534, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 24 }, "gid_6807": { "addr": "172.21.15.113:6836/1035982016", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1035982016, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1035982016, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6807, "incarnation": 101, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 63, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:01:30.805651+0000", "qdb_cluster": [ 6372, 6534 ], "qdb_leader": 6372, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6372, "mds_1": 6534, "mds_2": 6807 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:01:33.660 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:33.776 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:01:33.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:01:33.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c8200640 1 Processor -- start 2024-03-04T08:01:33.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c8200640 1 -- start start 2024-03-04T08:01:33.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c8200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa8c00c22e0 0x7fa8c015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:33.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c2d80 0x7fa8c015b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:33.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c8200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c00c27f0 con 0x7fa8c014e4c0 2024-03-04T08:01:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c8200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c015ad60 con 0x7fa8c00c22e0 2024-03-04T08:01:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00c2a90 con 0x7fa8c00c2d80 2024-03-04T08:01:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c6200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c2d80 0x7fa8c015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c4e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa8c00c22e0 0x7fa8c015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c6200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c2d80 0x7fa8c015b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:55830/0 (socket says 172.21.15.113:55830) 2024-03-04T08:01:33.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c6200640 1 -- 172.21.15.113:0/831835171 learned_addr learned my addr 172.21.15.113:0/831835171 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c6200640 1 -- 172.21.15.113:0/831835171 >> v2:172.21.15.136:3300/0 conn(0x7fa8c00c22e0 msgr2=0x7fa8c015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c6200640 1 --2- 172.21.15.113:0/831835171 >> v2:172.21.15.136:3300/0 conn(0x7fa8c00c22e0 0x7fa8c015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8bfe00640 1 -- 172.21.15.113:0/831835171 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946248754 0 0) 0x7fa8c00c27f0 con 0x7fa8c014e4c0 2024-03-04T08:01:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c6200640 1 -- 172.21.15.113:0/831835171 >> v1:172.21.15.136:6789/0 conn(0x7fa8c014e4c0 legacy=0x7fa8c0072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:33.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.801+0000 7fa8c6200640 1 -- 172.21.15.113:0/831835171 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c0070c60 con 0x7fa8c00c2d80 2024-03-04T08:01:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.805+0000 7fa8c6200640 1 --2- 172.21.15.113:0/831835171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c2d80 0x7fa8c015b730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009e50 tx=0x7fa8b801f210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=382a7e037ac4f46c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.805+0000 7fa8bfe00640 1 -- 172.21.15.113:0/831835171 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80b4070 con 0x7fa8c00c2d80 2024-03-04T08:01:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.805+0000 7fa8bfe00640 1 -- 172.21.15.113:0/831835171 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b80af400 con 0x7fa8c00c2d80 2024-03-04T08:01:33.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.805+0000 7fa8bfe00640 1 -- 172.21.15.113:0/831835171 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80af7d0 con 0x7fa8c00c2d80 2024-03-04T08:01:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.805+0000 7fa8c8200640 1 -- 172.21.15.113:0/831835171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c2d80 msgr2=0x7fa8c015b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.805+0000 7fa8c8200640 1 --2- 172.21.15.113:0/831835171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c2d80 0x7fa8c015b730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009e50 tx=0x7fa8b801f210 comp rx=0 tx=0).stop 2024-03-04T08:01:33.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.809+0000 7fa8c8200640 1 -- 172.21.15.113:0/831835171 shutdown_connections 2024-03-04T08:01:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.809+0000 7fa8c8200640 1 --2- 172.21.15.113:0/831835171 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c2d80 0x7fa8c015b730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.809+0000 7fa8c8200640 1 --2- 172.21.15.113:0/831835171 >> v2:172.21.15.136:3300/0 conn(0x7fa8c00c22e0 0x7fa8c015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.809+0000 7fa8c8200640 1 -- 172.21.15.113:0/831835171 >> 172.21.15.113:0/831835171 conn(0x7fa8c006e820 msgr2=0x7fa8c006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.809+0000 7fa8c8200640 1 -- 172.21.15.113:0/831835171 shutdown_connections 2024-03-04T08:01:33.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.809+0000 7fa8c8200640 1 -- 172.21.15.113:0/831835171 wait complete. 2024-03-04T08:01:33.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.809+0000 7fa8c8200640 1 Processor -- start 2024-03-04T08:01:33.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.809+0000 7fa8c8200640 1 -- start start 2024-03-04T08:01:33.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c22e0 0x7fa8c0152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:33.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c00c2d80 0x7fa8c0153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:33.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c8200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8c014e4c0 0x7fa8c01540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:33.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0071500 con 0x7fa8c00c2d80 2024-03-04T08:01:33.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c8200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0071220 con 0x7fa8c014e4c0 2024-03-04T08:01:33.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c014e8a0 con 0x7fa8c00c22e0 2024-03-04T08:01:33.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c4e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c00c2d80 0x7fa8c0153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:33.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c5800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c22e0 0x7fa8c0152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:33.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c6200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8c014e4c0 0x7fa8c01540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:33.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c4e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c00c2d80 0x7fa8c0153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36448/0 (socket says 172.21.15.113:36448) 2024-03-04T08:01:33.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c4e00640 1 -- 172.21.15.113:0/1328409658 learned_addr learned my addr 172.21.15.113:0/1328409658 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c4e00640 1 -- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c22e0 msgr2=0x7fa8c0152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c4e00640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c22e0 0x7fa8c0152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c4e00640 1 -- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8c014e4c0 msgr2=0x7fa8c01540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c4e00640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8c014e4c0 0x7fa8c01540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c4e00640 1 -- 172.21.15.113:0/1328409658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c01548a0 con 0x7fa8c00c2d80 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c4e00640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c00c2d80 0x7fa8c0153b70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00e8a0 tx=0x7fa8ac00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8bd600640 1 -- 172.21.15.113:0/1328409658 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac017070 con 0x7fa8c00c2d80 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8bd600640 1 -- 172.21.15.113:0/1328409658 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac00c940 con 0x7fa8c00c2d80 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8bd600640 1 -- 172.21.15.113:0/1328409658 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac00cd10 con 0x7fa8c00c2d80 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c8200640 1 -- 172.21.15.113:0/1328409658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c0154b40 con 0x7fa8c00c2d80 2024-03-04T08:01:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c8200640 1 -- 172.21.15.113:0/1328409658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c0152d40 con 0x7fa8c00c2d80 2024-03-04T08:01:33.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.813+0000 7fa8c8200640 1 -- 172.21.15.113:0/1328409658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c0075c00 con 0x7fa8c00c2d80 2024-03-04T08:01:33.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.817+0000 7fa8bd600640 1 -- 172.21.15.113:0/1328409658 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8ac01a070 con 0x7fa8c00c2d80 2024-03-04T08:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.817+0000 7fa8bd600640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8a0066f60 0x7fa8a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.817+0000 7fa8bd600640 1 -- 172.21.15.113:0/1328409658 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7fa8ac085700 con 0x7fa8c00c2d80 2024-03-04T08:01:33.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.817+0000 7fa8c5800640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8a0066f60 0x7fa8a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:33.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.817+0000 7fa8c5800640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8a0066f60 0x7fa8a0069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4000f70 tx=0x7fa8b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:33.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:33.821+0000 7fa8bd600640 1 -- 172.21.15.113:0/1328409658 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa8ac055bd0 con 0x7fa8c00c2d80 2024-03-04T08:01:34.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 -- 172.21.15.113:0/1328409658 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8c0155190 con 0x7fa8c00c2d80 2024-03-04T08:01:34.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8bd600640 1 -- 172.21.15.113:0/1328409658 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5821 (secure 0 0 0) 0x7fa8ac059c70 con 0x7fa8c00c2d80 2024-03-04T08:01:34.020 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:34.020 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:30.805651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:01:34.020 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 101 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 -- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8a0066f60 msgr2=0x7fa8a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8a0066f60 0x7fa8a0069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4000f70 tx=0x7fa8b4009040 comp rx=0 tx=0).stop 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 -- 172.21.15.113:0/1328409658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c00c2d80 msgr2=0x7fa8c0153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c00c2d80 0x7fa8c0153b70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00e8a0 tx=0x7fa8ac00ed60 comp rx=0 tx=0).stop 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 -- 172.21.15.113:0/1328409658 shutdown_connections 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8c014e4c0 0x7fa8c01540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8a0066f60 0x7fa8a0069420 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c00c2d80 0x7fa8c0153b70 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 --2- 172.21.15.113:0/1328409658 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa8c00c22e0 0x7fa8c0152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 -- 172.21.15.113:0/1328409658 >> 172.21.15.113:0/1328409658 conn(0x7fa8c006e820 msgr2=0x7fa8c014b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:34.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 -- 172.21.15.113:0/1328409658 shutdown_connections 2024-03-04T08:01:34.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:34.021+0000 7fa8c8200640 1 -- 172.21.15.113:0/1328409658 wait complete. 2024-03-04T08:01:34.042 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:01:36.043 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff97200640 1 Processor -- start 2024-03-04T08:01:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff97200640 1 -- start start 2024-03-04T08:01:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff97200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7eff981547a0 0x7eff980c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff97200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff980c9160 con 0x7eff980c2180 2024-03-04T08:01:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff97200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff980c9400 con 0x7eff9815b240 2024-03-04T08:01:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff97200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff980c96e0 con 0x7eff981547a0 2024-03-04T08:01:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff95400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7eff980c2180 0x7eff9815ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50360/0 (socket says 172.21.15.113:50360) 2024-03-04T08:01:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff95400640 1 -- 172.21.15.113:0/3499188435 learned_addr learned my addr 172.21.15.113:0/3499188435 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff95e00640 1 --2- 172.21.15.113:0/3499188435 >> v2:172.21.15.136:3300/0 conn(0x7eff981547a0 0x7eff980c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff94a00640 1 -- 172.21.15.113:0/3499188435 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055397142 0 0) 0x7eff980c9160 con 0x7eff980c2180 2024-03-04T08:01:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff94a00640 1 -- 172.21.15.113:0/3499188435 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff7c003530 con 0x7eff980c2180 2024-03-04T08:01:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff94a00640 1 -- 172.21.15.113:0/3499188435 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2779081635 0 0) 0x7eff7c003530 con 0x7eff980c2180 2024-03-04T08:01:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff95e00640 1 -- 172.21.15.113:0/3499188435 >> v1:172.21.15.136:6789/0 conn(0x7eff9815b240 legacy=0x7eff98165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff95e00640 1 -- 172.21.15.113:0/3499188435 >> v1:172.21.15.113:6789/0 conn(0x7eff980c2180 legacy=0x7eff9815ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:36.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.185+0000 7eff95e00640 1 -- 172.21.15.113:0/3499188435 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980c7c00 con 0x7eff981547a0 2024-03-04T08:01:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff95e00640 1 --2- 172.21.15.113:0/3499188435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff981547a0 0x7eff980c8620 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7eff880097c0 tx=0x7eff8801ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f67793ce6f39416 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff94a00640 1 -- 172.21.15.113:0/3499188435 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff880b5070 con 0x7eff981547a0 2024-03-04T08:01:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff94a00640 1 -- 172.21.15.113:0/3499188435 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff880a8630 con 0x7eff981547a0 2024-03-04T08:01:36.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff94a00640 1 -- 172.21.15.113:0/3499188435 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff840016a0 con 0x7eff981547a0 2024-03-04T08:01:36.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff97200640 1 -- 172.21.15.113:0/3499188435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff981547a0 msgr2=0x7eff980c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:36.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff97200640 1 --2- 172.21.15.113:0/3499188435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff981547a0 0x7eff980c8620 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7eff880097c0 tx=0x7eff8801ebb0 comp rx=0 tx=0).stop 2024-03-04T08:01:36.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff97200640 1 -- 172.21.15.113:0/3499188435 shutdown_connections 2024-03-04T08:01:36.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff97200640 1 --2- 172.21.15.113:0/3499188435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff981547a0 0x7eff980c8620 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:36.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff97200640 1 -- 172.21.15.113:0/3499188435 >> 172.21.15.113:0/3499188435 conn(0x7eff980cd080 msgr2=0x7eff980cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:36.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff97200640 1 -- 172.21.15.113:0/3499188435 shutdown_connections 2024-03-04T08:01:36.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.189+0000 7eff97200640 1 -- 172.21.15.113:0/3499188435 wait complete. 2024-03-04T08:01:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff97200640 1 Processor -- start 2024-03-04T08:01:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff97200640 1 -- start start 2024-03-04T08:01:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff97200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff980c2180 0x7eff98152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff97200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff981547a0 0x7eff9814b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff97200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff9815b240 0x7eff981439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff97200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff98154b80 con 0x7eff9815b240 2024-03-04T08:01:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff97200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff980c81c0 con 0x7eff980c2180 2024-03-04T08:01:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff97200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff98154d00 con 0x7eff981547a0 2024-03-04T08:01:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff96800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff9815b240 0x7eff981439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:36.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff96800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff9815b240 0x7eff981439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36452/0 (socket says 172.21.15.113:36452) 2024-03-04T08:01:36.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff96800640 1 -- 172.21.15.113:0/3923211675 learned_addr learned my addr 172.21.15.113:0/3923211675 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:36.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff95400640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff981547a0 0x7eff9814b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:36.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff95e00640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff980c2180 0x7eff98152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff96800640 1 -- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff981547a0 msgr2=0x7eff9814b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff96800640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff981547a0 0x7eff9814b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff96800640 1 -- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff980c2180 msgr2=0x7eff98152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff96800640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff980c2180 0x7eff98152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff96800640 1 -- 172.21.15.113:0/3923211675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff98151990 con 0x7eff9815b240 2024-03-04T08:01:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff96800640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff9815b240 0x7eff981439c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7eff9000c760 tx=0x7eff9000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff8e000640 1 -- 172.21.15.113:0/3923211675 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90003b80 con 0x7eff9815b240 2024-03-04T08:01:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff8e000640 1 -- 172.21.15.113:0/3923211675 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff90003d20 con 0x7eff9815b240 2024-03-04T08:01:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff8e000640 1 -- 172.21.15.113:0/3923211675 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9000aa30 con 0x7eff9815b240 2024-03-04T08:01:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff97200640 1 -- 172.21.15.113:0/3923211675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff981516b0 con 0x7eff9815b240 2024-03-04T08:01:36.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.193+0000 7eff97200640 1 -- 172.21.15.113:0/3923211675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9814cef0 con 0x7eff9815b240 2024-03-04T08:01:36.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.197+0000 7eff97200640 1 -- 172.21.15.113:0/3923211675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff98075a80 con 0x7eff9815b240 2024-03-04T08:01:36.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.197+0000 7eff8e000640 1 -- 172.21.15.113:0/3923211675 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff9000abd0 con 0x7eff9815b240 2024-03-04T08:01:36.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.197+0000 7eff8e000640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff78066f60 0x7eff78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:36.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.197+0000 7eff8e000640 1 -- 172.21.15.113:0/3923211675 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7eff900853d0 con 0x7eff9815b240 2024-03-04T08:01:36.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.197+0000 7eff95e00640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff78066f60 0x7eff78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:36.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.201+0000 7eff95e00640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff78066f60 0x7eff78069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7eff880b2000 tx=0x7eff880aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.201+0000 7eff8e000640 1 -- 172.21.15.113:0/3923211675 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7eff90055770 con 0x7eff9815b240 2024-03-04T08:01:36.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.477+0000 7eff97200640 1 -- 172.21.15.113:0/3923211675 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff981527b0 con 0x7eff9815b240 2024-03-04T08:01:36.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.477+0000 7eff8e000640 1 -- 172.21.15.113:0/3923211675 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+5821 (secure 0 0 0) 0x7eff90059810 con 0x7eff9815b240 2024-03-04T08:01:36.477 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:36.477 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":101,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:30.805651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:01:36.478 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 101 2024-03-04T08:01:36.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 -- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff78066f60 msgr2=0x7eff78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:36.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff78066f60 0x7eff78069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7eff880b2000 tx=0x7eff880aa070 comp rx=0 tx=0).stop 2024-03-04T08:01:36.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 -- 172.21.15.113:0/3923211675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff9815b240 msgr2=0x7eff981439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:36.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff9815b240 0x7eff981439c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7eff9000c760 tx=0x7eff9000cc20 comp rx=0 tx=0).stop 2024-03-04T08:01:36.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 -- 172.21.15.113:0/3923211675 shutdown_connections 2024-03-04T08:01:36.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff78066f60 0x7eff78069420 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:36.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff9815b240 0x7eff981439c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:36.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff981547a0 0x7eff9814b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:36.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 --2- 172.21.15.113:0/3923211675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff980c2180 0x7eff98152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:36.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 -- 172.21.15.113:0/3923211675 >> 172.21.15.113:0/3923211675 conn(0x7eff980cd080 msgr2=0x7eff980ca2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:36.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 -- 172.21.15.113:0/3923211675 shutdown_connections 2024-03-04T08:01:36.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:36.481+0000 7eff97200640 1 -- 172.21.15.113:0/3923211675 wait complete. 2024-03-04T08:01:36.500 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:01:38.500 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.641+0000 7f8ebca00640 1 Processor -- start 2024-03-04T08:01:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.641+0000 7f8ebca00640 1 -- start start 2024-03-04T08:01:38.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8ebca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8eb8152950 0x7f8eb815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:38.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8ebca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8eb806faa0 0x7f8eb80c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:38.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8ebca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb80c4ce0 con 0x7f8eb806ed80 2024-03-04T08:01:38.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8ebca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb80c8370 con 0x7f8eb806faa0 2024-03-04T08:01:38.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8ebca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb80ca0a0 con 0x7f8eb8152950 2024-03-04T08:01:38.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8eafe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8eb806faa0 0x7f8eb80c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:38.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8eafe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8eb806faa0 0x7f8eb80c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56100/0 (socket says 172.21.15.113:56100) 2024-03-04T08:01:38.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8eafe00640 1 -- 172.21.15.113:0/4082692665 learned_addr learned my addr 172.21.15.113:0/4082692665 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:38.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8eb7400640 1 --2- 172.21.15.113:0/4082692665 >> v2:172.21.15.136:3300/0 conn(0x7f8eb8152950 0x7f8eb815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:38.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8eafe00640 1 -- 172.21.15.113:0/4082692665 >> v2:172.21.15.136:3300/0 conn(0x7f8eb8152950 msgr2=0x7f8eb815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:38.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8eafe00640 1 --2- 172.21.15.113:0/4082692665 >> v2:172.21.15.136:3300/0 conn(0x7f8eb8152950 0x7f8eb815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:38.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8eafe00640 1 -- 172.21.15.113:0/4082692665 >> v1:172.21.15.136:6789/0 conn(0x7f8eb806ed80 legacy=0x7f8eb80ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:38.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8eafe00640 1 -- 172.21.15.113:0/4082692665 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb80c4f80 con 0x7f8eb806faa0 2024-03-04T08:01:38.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.645+0000 7f8eafe00640 1 --2- 172.21.15.113:0/4082692665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 0x7f8eb80c30b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80ceb90 tx=0x7f8ea001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0ecefd4433ee43 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:38.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8eb6a00640 1 -- 172.21.15.113:0/4082692665 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00b5070 con 0x7f8eb806faa0 2024-03-04T08:01:38.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8eb6a00640 1 -- 172.21.15.113:0/4082692665 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea00a8a10 con 0x7f8eb806faa0 2024-03-04T08:01:38.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8eb6a00640 1 -- 172.21.15.113:0/4082692665 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00a8de0 con 0x7f8eb806faa0 2024-03-04T08:01:38.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8ebca00640 1 -- 172.21.15.113:0/4082692665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 msgr2=0x7f8eb80c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:38.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8ebca00640 1 --2- 172.21.15.113:0/4082692665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 0x7f8eb80c30b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8eb80ceb90 tx=0x7f8ea001ed10 comp rx=0 tx=0).stop 2024-03-04T08:01:38.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8ebca00640 1 -- 172.21.15.113:0/4082692665 shutdown_connections 2024-03-04T08:01:38.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8ebca00640 1 --2- 172.21.15.113:0/4082692665 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 0x7f8eb80c30b0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:38.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8ebca00640 1 --2- 172.21.15.113:0/4082692665 >> v2:172.21.15.136:3300/0 conn(0x7f8eb8152950 0x7f8eb815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:38.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8ebca00640 1 -- 172.21.15.113:0/4082692665 >> 172.21.15.113:0/4082692665 conn(0x7f8eb8072f40 msgr2=0x7f8eb80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:38.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8ebca00640 1 -- 172.21.15.113:0/4082692665 shutdown_connections 2024-03-04T08:01:38.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8ebca00640 1 -- 172.21.15.113:0/4082692665 wait complete. 2024-03-04T08:01:38.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8ebca00640 1 Processor -- start 2024-03-04T08:01:38.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.649+0000 7f8ebca00640 1 -- start start 2024-03-04T08:01:38.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8ebca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb806ed80 0x7f8eb814b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:38.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8ebca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 0x7f8eb81f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:38.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8ebca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8eb8152950 0x7f8eb815a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:38.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8ebca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80c9a50 con 0x7f8eb806faa0 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8ebca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb806fe80 con 0x7f8eb806ed80 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8ebca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80c9d30 con 0x7f8eb8152950 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eafe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 0x7f8eb81f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eafe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 0x7f8eb81f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56112/0 (socket says 172.21.15.113:56112) 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eafe00640 1 -- 172.21.15.113:0/1107631872 learned_addr learned my addr 172.21.15.113:0/1107631872 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eb7400640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb806ed80 0x7f8eb814b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eb7e00640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8eb8152950 0x7f8eb815a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eafe00640 1 -- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8eb8152950 msgr2=0x7f8eb815a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eafe00640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8eb8152950 0x7f8eb815a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eafe00640 1 -- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb806ed80 msgr2=0x7f8eb814b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eafe00640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb806ed80 0x7f8eb814b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eafe00640 1 -- 172.21.15.113:0/1107631872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb815b650 con 0x7f8eb806faa0 2024-03-04T08:01:38.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eb7400640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb806ed80 0x7f8eb814b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:38.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eafe00640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 0x7f8eb81f22f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8ea001b920 tx=0x7f8ea001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:38.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eaf400640 1 -- 172.21.15.113:0/1107631872 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00b5070 con 0x7f8eb806faa0 2024-03-04T08:01:38.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eaf400640 1 -- 172.21.15.113:0/1107631872 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea00b0b40 con 0x7f8eb806faa0 2024-03-04T08:01:38.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8ebca00640 1 -- 172.21.15.113:0/1107631872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb815b090 con 0x7f8eb806faa0 2024-03-04T08:01:38.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8eaf400640 1 -- 172.21.15.113:0/1107631872 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00bb410 con 0x7f8eb806faa0 2024-03-04T08:01:38.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8ebca00640 1 -- 172.21.15.113:0/1107631872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb81f1810 con 0x7f8eb806faa0 2024-03-04T08:01:38.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.653+0000 7f8ebca00640 1 -- 172.21.15.113:0/1107631872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb80c3b10 con 0x7f8eb806faa0 2024-03-04T08:01:38.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.657+0000 7f8eaf400640 1 -- 172.21.15.113:0/1107631872 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ea00c2060 con 0x7f8eb806faa0 2024-03-04T08:01:38.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.657+0000 7f8eaf400640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e94066f60 0x7f8e94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:38.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.657+0000 7f8eaf400640 1 -- 172.21.15.113:0/1107631872 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f8ea0122740 con 0x7f8eb806faa0 2024-03-04T08:01:38.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.657+0000 7f8eb7400640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e94066f60 0x7f8e94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:38.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.657+0000 7f8eb7400640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e94066f60 0x7f8e94069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8007a70 tx=0x7f8ea8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:38.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.661+0000 7f8eaf400640 1 -- 172.21.15.113:0/1107631872 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8ea0018870 con 0x7f8eb806faa0 2024-03-04T08:01:38.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.857+0000 7f8ebca00640 1 -- 172.21.15.113:0/1107631872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8eb8075a80 con 0x7f8eb806faa0 2024-03-04T08:01:38.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.857+0000 7f8eaf400640 1 -- 172.21.15.113:0/1107631872 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5822 (secure 0 0 0) 0x7f8ea00f31c0 con 0x7f8eb806faa0 2024-03-04T08:01:38.856 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:38.856 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":102,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:35.818961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:resolve","state_seq":8,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372]},"id":1}]} 2024-03-04T08:01:38.857 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 102 2024-03-04T08:01:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 -- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e94066f60 msgr2=0x7f8e94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e94066f60 0x7f8e94069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8007a70 tx=0x7f8ea8008040 comp rx=0 tx=0).stop 2024-03-04T08:01:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 -- 172.21.15.113:0/1107631872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 msgr2=0x7f8eb81f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 0x7f8eb81f22f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8ea001b920 tx=0x7f8ea001bab0 comp rx=0 tx=0).stop 2024-03-04T08:01:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 -- 172.21.15.113:0/1107631872 shutdown_connections 2024-03-04T08:01:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8eb8152950 0x7f8eb815a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:38.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8e94066f60 0x7f8e94069420 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:38.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8eb806faa0 0x7f8eb81f22f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:38.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 --2- 172.21.15.113:0/1107631872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8eb806ed80 0x7f8eb814b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:38.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 -- 172.21.15.113:0/1107631872 >> 172.21.15.113:0/1107631872 conn(0x7f8eb8072f40 msgr2=0x7f8eb8157900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:38.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 -- 172.21.15.113:0/1107631872 shutdown_connections 2024-03-04T08:01:38.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:38.861+0000 7f8ebca00640 1 -- 172.21.15.113:0/1107631872 wait complete. 2024-03-04T08:01:38.879 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:01:40.880 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.021+0000 7f8f16800640 1 Processor -- start 2024-03-04T08:01:41.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.021+0000 7f8f16800640 1 -- start start 2024-03-04T08:01:41.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.021+0000 7f8f16800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100ce570 0x7f8f10154c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:41.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.021+0000 7f8f16800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8f10070120 0x7f8f101551d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:41.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.021+0000 7f8f16800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1006c860 con 0x7f8f100c8d80 2024-03-04T08:01:41.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.021+0000 7f8f16800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1006f8d0 con 0x7f8f10070120 2024-03-04T08:01:41.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.021+0000 7f8f16800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1006f310 con 0x7f8f100ce570 2024-03-04T08:01:41.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0ea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8f100c8d80 0x7f8f1015b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44528/0 (socket says 172.21.15.113:44528) 2024-03-04T08:01:41.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0ea00640 1 -- 172.21.15.113:0/755656764 learned_addr learned my addr 172.21.15.113:0/755656764 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:41.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0f400640 1 --2- 172.21.15.113:0/755656764 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100ce570 0x7f8f10154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:41.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0fe00640 1 --2- 172.21.15.113:0/755656764 >> v2:172.21.15.136:3300/0 conn(0x7f8f10070120 0x7f8f101551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:41.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0e000640 1 -- 172.21.15.113:0/755656764 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804204914 0 0) 0x7f8f1006c860 con 0x7f8f100c8d80 2024-03-04T08:01:41.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0e000640 1 -- 172.21.15.113:0/755656764 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ef4003530 con 0x7f8f100c8d80 2024-03-04T08:01:41.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0f400640 1 -- 172.21.15.113:0/755656764 >> v2:172.21.15.136:3300/0 conn(0x7f8f10070120 msgr2=0x7f8f101551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:41.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0f400640 1 --2- 172.21.15.113:0/755656764 >> v2:172.21.15.136:3300/0 conn(0x7f8f10070120 0x7f8f101551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:41.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0f400640 1 -- 172.21.15.113:0/755656764 >> v1:172.21.15.113:6789/0 conn(0x7f8f100c8d80 legacy=0x7f8f1015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:41.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0f400640 1 -- 172.21.15.113:0/755656764 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1006f5f0 con 0x7f8f100ce570 2024-03-04T08:01:41.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0fe00640 1 --2- 172.21.15.113:0/755656764 >> v2:172.21.15.136:3300/0 conn(0x7f8f10070120 0x7f8f101551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.025+0000 7f8f0f400640 1 --2- 172.21.15.113:0/755656764 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100ce570 0x7f8f10154c90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8f000097f0 tx=0x7f8f0001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34097950e3d23d17 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f0e000640 1 -- 172.21.15.113:0/755656764 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000b5070 con 0x7f8f100ce570 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f0e000640 1 -- 172.21.15.113:0/755656764 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f000a8990 con 0x7f8f100ce570 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f0e000640 1 -- 172.21.15.113:0/755656764 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000a8d40 con 0x7f8f100ce570 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 -- 172.21.15.113:0/755656764 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100ce570 msgr2=0x7f8f10154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 --2- 172.21.15.113:0/755656764 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100ce570 0x7f8f10154c90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8f000097f0 tx=0x7f8f0001ec90 comp rx=0 tx=0).stop 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 -- 172.21.15.113:0/755656764 shutdown_connections 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 --2- 172.21.15.113:0/755656764 >> v2:172.21.15.136:3300/0 conn(0x7f8f10070120 0x7f8f101551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 --2- 172.21.15.113:0/755656764 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100ce570 0x7f8f10154c90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 -- 172.21.15.113:0/755656764 >> 172.21.15.113:0/755656764 conn(0x7f8f10066db0 msgr2=0x7f8f100671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 -- 172.21.15.113:0/755656764 shutdown_connections 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 -- 172.21.15.113:0/755656764 wait complete. 2024-03-04T08:01:41.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 Processor -- start 2024-03-04T08:01:41.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 -- start start 2024-03-04T08:01:41.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f10070120 0x7f8f1014fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:41.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100c8d80 0x7f8f1015ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f100ce570 0x7f8f1015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1006fb80 con 0x7f8f10070120 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1006c4f0 con 0x7f8f100ce570 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f16800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1006fd00 con 0x7f8f100c8d80 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f0f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f10070120 0x7f8f1014fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f0f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f10070120 0x7f8f1014fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56116/0 (socket says 172.21.15.113:56116) 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f0f400640 1 -- 172.21.15.113:0/3979393673 learned_addr learned my addr 172.21.15.113:0/3979393673 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f0ea00640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100c8d80 0x7f8f1015ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.029+0000 7f8f0fe00640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f100ce570 0x7f8f1015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f0f400640 1 -- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100c8d80 msgr2=0x7f8f1015ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f0f400640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100c8d80 0x7f8f1015ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f0f400640 1 -- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f100ce570 msgr2=0x7f8f1015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:41.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f0f400640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f100ce570 0x7f8f1015d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:41.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f0f400640 1 -- 172.21.15.113:0/3979393673 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100cacb0 con 0x7f8f10070120 2024-03-04T08:01:41.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f0ea00640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100c8d80 0x7f8f1015ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:41.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f0f400640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f10070120 0x7f8f1014fc50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8f0001d340 tx=0x7f8f0001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:41.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f07400640 1 -- 172.21.15.113:0/3979393673 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000bf070 con 0x7f8f10070120 2024-03-04T08:01:41.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f07400640 1 -- 172.21.15.113:0/3979393673 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f00004240 con 0x7f8f10070120 2024-03-04T08:01:41.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f07400640 1 -- 172.21.15.113:0/3979393673 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000b5040 con 0x7f8f10070120 2024-03-04T08:01:41.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f16800640 1 -- 172.21.15.113:0/3979393673 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f100ca410 con 0x7f8f10070120 2024-03-04T08:01:41.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f16800640 1 -- 172.21.15.113:0/3979393673 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f100ca9d0 con 0x7f8f10070120 2024-03-04T08:01:41.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f16800640 1 -- 172.21.15.113:0/3979393673 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f10075c00 con 0x7f8f10070120 2024-03-04T08:01:41.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f07400640 1 -- 172.21.15.113:0/3979393673 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f00003620 con 0x7f8f10070120 2024-03-04T08:01:41.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f07400640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:41.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.033+0000 7f8f07400640 1 -- 172.21.15.113:0/3979393673 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f8f00121db0 con 0x7f8f10070120 2024-03-04T08:01:41.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.037+0000 7f8f0ea00640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:41.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.037+0000 7f8f0ea00640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ef0066f60 0x7f8ef0069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f10151210 tx=0x7f8efc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:41.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.045+0000 7f8f07400640 1 -- 172.21.15.113:0/3979393673 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8f00126070 con 0x7f8f10070120 2024-03-04T08:01:41.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 -- 172.21.15.113:0/3979393673 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f1006ff00 con 0x7f8f10070120 2024-03-04T08:01:41.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f07400640 1 -- 172.21.15.113:0/3979393673 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5822 (secure 0 0 0) 0x7f8f000187f0 con 0x7f8f10070120 2024-03-04T08:01:41.241 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":102,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:35.818961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:resolve","state_seq":8,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372]},"id":1}]} 2024-03-04T08:01:41.242 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 102 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 -- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ef0066f60 msgr2=0x7f8ef0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ef0066f60 0x7f8ef0069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8f10151210 tx=0x7f8efc009040 comp rx=0 tx=0).stop 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 -- 172.21.15.113:0/3979393673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f10070120 msgr2=0x7f8f1014fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f10070120 0x7f8f1014fc50 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8f0001d340 tx=0x7f8f0001ba30 comp rx=0 tx=0).stop 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 -- 172.21.15.113:0/3979393673 shutdown_connections 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f100ce570 0x7f8f1015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f100c8d80 0x7f8f1015ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 --2- 172.21.15.113:0/3979393673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f10070120 0x7f8f1014fc50 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 -- 172.21.15.113:0/3979393673 >> 172.21.15.113:0/3979393673 conn(0x7f8f10066db0 msgr2=0x7f8f101521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 -- 172.21.15.113:0/3979393673 shutdown_connections 2024-03-04T08:01:41.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:41.241+0000 7f8f16800640 1 -- 172.21.15.113:0/3979393673 wait complete. 2024-03-04T08:01:41.262 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:01:43.263 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:43.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c9800640 1 Processor -- start 2024-03-04T08:01:43.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c9800640 1 -- start start 2024-03-04T08:01:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c9800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f53c40c8200 0x7f53c4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c9800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f53c4152bd0 0x7f53c41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c9800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c41591f0 con 0x7f53c41527f0 2024-03-04T08:01:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c9800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c4071df0 con 0x7f53c4152bd0 2024-03-04T08:01:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c9800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c41597b0 con 0x7f53c40c8200 2024-03-04T08:01:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f53c4152bd0 0x7f53c41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c2a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f53c41527f0 0x7f53c4072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44532/0 (socket says 172.21.15.113:44532) 2024-03-04T08:01:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c2a00640 1 -- 172.21.15.113:0/3757207190 learned_addr learned my addr 172.21.15.113:0/3757207190 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53ba000640 1 --2- 172.21.15.113:0/3757207190 >> v2:172.21.15.136:3300/0 conn(0x7f53c40c8200 0x7f53c4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:43.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c3400640 1 -- 172.21.15.113:0/3757207190 >> v2:172.21.15.136:3300/0 conn(0x7f53c40c8200 msgr2=0x7f53c4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:43.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c3400640 1 --2- 172.21.15.113:0/3757207190 >> v2:172.21.15.136:3300/0 conn(0x7f53c40c8200 0x7f53c4070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:43.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c3400640 1 -- 172.21.15.113:0/3757207190 >> v1:172.21.15.113:6789/0 conn(0x7f53c41527f0 legacy=0x7f53c4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:43.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c3400640 1 -- 172.21.15.113:0/3757207190 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c41594d0 con 0x7f53c4152bd0 2024-03-04T08:01:43.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53ba000640 1 --2- 172.21.15.113:0/3757207190 >> v2:172.21.15.136:3300/0 conn(0x7f53c40c8200 0x7f53c4070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:43.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.405+0000 7f53c2000640 1 -- 172.21.15.113:0/3757207190 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916261847 0 0) 0x7f53c41591f0 con 0x7f53c41527f0 2024-03-04T08:01:43.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c3400640 1 --2- 172.21.15.113:0/3757207190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c4152bd0 0x7f53c41587d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f53c40cd230 tx=0x7f53bc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b502c2dfbc87bb9d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c2000640 1 -- 172.21.15.113:0/3757207190 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0b0070 con 0x7f53c4152bd0 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c2000640 1 -- 172.21.15.113:0/3757207190 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc0ab600 con 0x7f53c4152bd0 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c2000640 1 -- 172.21.15.113:0/3757207190 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0ab9d0 con 0x7f53c4152bd0 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c9800640 1 -- 172.21.15.113:0/3757207190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c4152bd0 msgr2=0x7f53c41587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c9800640 1 --2- 172.21.15.113:0/3757207190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c4152bd0 0x7f53c41587d0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f53c40cd230 tx=0x7f53bc01c480 comp rx=0 tx=0).stop 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c9800640 1 -- 172.21.15.113:0/3757207190 shutdown_connections 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c9800640 1 --2- 172.21.15.113:0/3757207190 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c4152bd0 0x7f53c41587d0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c9800640 1 --2- 172.21.15.113:0/3757207190 >> v2:172.21.15.136:3300/0 conn(0x7f53c40c8200 0x7f53c4070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c9800640 1 -- 172.21.15.113:0/3757207190 >> 172.21.15.113:0/3757207190 conn(0x7f53c406e820 msgr2=0x7f53c406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c9800640 1 -- 172.21.15.113:0/3757207190 shutdown_connections 2024-03-04T08:01:43.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c9800640 1 -- 172.21.15.113:0/3757207190 wait complete. 2024-03-04T08:01:43.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c9800640 1 Processor -- start 2024-03-04T08:01:43.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.413+0000 7f53c9800640 1 -- start start 2024-03-04T08:01:43.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c40c8200 0x7f53c41473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:43.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c9800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53c41527f0 0x7f53c40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:43.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c9800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53c4152bd0 0x7f53c40afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:43.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c9800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41654f0 con 0x7f53c40c8200 2024-03-04T08:01:43.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c9800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4165370 con 0x7f53c41527f0 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c9800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4165670 con 0x7f53c4152bd0 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c40c8200 0x7f53c41473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c40c8200 0x7f53c41473c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56124/0 (socket says 172.21.15.113:56124) 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c2a00640 1 -- 172.21.15.113:0/2189070951 learned_addr learned my addr 172.21.15.113:0/2189070951 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53ba000640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53c41527f0 0x7f53c40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c3400640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53c4152bd0 0x7f53c40afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c2a00640 1 -- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53c4152bd0 msgr2=0x7f53c40afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c2a00640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53c4152bd0 0x7f53c40afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c2a00640 1 -- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53c41527f0 msgr2=0x7f53c40af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c2a00640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53c41527f0 0x7f53c40af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c2a00640 1 -- 172.21.15.113:0/2189070951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c41464a0 con 0x7f53c40c8200 2024-03-04T08:01:43.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c2a00640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c40c8200 0x7f53c41473c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f53b4001570 tx=0x7f53b400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:43.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53bb400640 1 -- 172.21.15.113:0/2189070951 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4018070 con 0x7f53c40c8200 2024-03-04T08:01:43.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53bb400640 1 -- 172.21.15.113:0/2189070951 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b400a5e0 con 0x7f53c40c8200 2024-03-04T08:01:43.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c9800640 1 -- 172.21.15.113:0/2189070951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c41461c0 con 0x7f53c40c8200 2024-03-04T08:01:43.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53bb400640 1 -- 172.21.15.113:0/2189070951 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b400a9d0 con 0x7f53c40c8200 2024-03-04T08:01:43.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c9800640 1 -- 172.21.15.113:0/2189070951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c41575d0 con 0x7f53c40c8200 2024-03-04T08:01:43.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53c9800640 1 -- 172.21.15.113:0/2189070951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c4075c00 con 0x7f53c40c8200 2024-03-04T08:01:43.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.417+0000 7f53bb400640 1 -- 172.21.15.113:0/2189070951 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53b401b070 con 0x7f53c40c8200 2024-03-04T08:01:43.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.421+0000 7f53bb400640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53a0066f90 0x7f53a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:43.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.421+0000 7f53bb400640 1 -- 172.21.15.113:0/2189070951 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f53b4084ba0 con 0x7f53c40c8200 2024-03-04T08:01:43.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.421+0000 7f53ba000640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53a0066f90 0x7f53a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:43.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.421+0000 7f53ba000640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53a0066f90 0x7f53a0069450 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f53c4145fb0 tx=0x7f53ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:43.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.429+0000 7f53bb400640 1 -- 172.21.15.113:0/2189070951 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f53b4054f40 con 0x7f53c40c8200 2024-03-04T08:01:43.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 -- 172.21.15.113:0/2189070951 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53c4152340 con 0x7f53c40c8200 2024-03-04T08:01:43.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53bb400640 1 -- 172.21.15.113:0/2189070951 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+5825 (secure 0 0 0) 0x7f53b4058fe0 con 0x7f53c40c8200 2024-03-04T08:01:43.620 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:43.620 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:41.817994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:reconnect","state_seq":10,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:01:43.620 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 103 2024-03-04T08:01:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 -- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53a0066f90 msgr2=0x7f53a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53a0066f90 0x7f53a0069450 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f53c4145fb0 tx=0x7f53ac008040 comp rx=0 tx=0).stop 2024-03-04T08:01:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 -- 172.21.15.113:0/2189070951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c40c8200 msgr2=0x7f53c41473c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c40c8200 0x7f53c41473c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f53b4001570 tx=0x7f53b400cd20 comp rx=0 tx=0).stop 2024-03-04T08:01:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 -- 172.21.15.113:0/2189070951 shutdown_connections 2024-03-04T08:01:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53c4152bd0 0x7f53c40afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53c41527f0 0x7f53c40af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f53a0066f90 0x7f53a0069450 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:43.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 --2- 172.21.15.113:0/2189070951 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53c40c8200 0x7f53c41473c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:43.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 -- 172.21.15.113:0/2189070951 >> 172.21.15.113:0/2189070951 conn(0x7f53c406e820 msgr2=0x7f53c406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:43.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 -- 172.21.15.113:0/2189070951 shutdown_connections 2024-03-04T08:01:43.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:43.621+0000 7f53c9800640 1 -- 172.21.15.113:0/2189070951 wait complete. 2024-03-04T08:01:43.642 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 103, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 103, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6372": { "addr": "172.21.15.113:6833/2732565639", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2732565639, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2732565639, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6372, "incarnation": 83, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 40 }, "gid_6534": { "addr": "172.21.15.113:6837/4197507712", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 4197507712, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 4197507712, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6534, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 24 }, "gid_6807": { "addr": "172.21.15.113:6836/1035982016", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1035982016, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1035982016, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6807, "incarnation": 101, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:reconnect", "state_seq": 10 } }, "last_failure": 0, "last_failure_osd_epoch": 63, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:01:41.817994+0000", "qdb_cluster": [ 6372, 6534 ], "qdb_leader": 6372, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6372, "mds_1": 6534, "mds_2": 6807 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:01:45.643 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:45.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.789+0000 7f19f9200640 1 Processor -- start 2024-03-04T08:01:45.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f9200640 1 -- start start 2024-03-04T08:01:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f9200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 0x7f19f415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f9200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f19f406faa0 0x7f19f40c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f9200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f40c4ce0 con 0x7f19f406ed80 2024-03-04T08:01:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f9200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40c8370 con 0x7f19f406faa0 2024-03-04T08:01:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f9200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40ca0a0 con 0x7f19f4152950 2024-03-04T08:01:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f19f406ed80 0x7f19f40ca530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44540/0 (socket says 172.21.15.113:44540) 2024-03-04T08:01:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f3e00640 1 -- 172.21.15.113:0/1406735990 learned_addr learned my addr 172.21.15.113:0/1406735990 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f3400640 1 --2- 172.21.15.113:0/1406735990 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 0x7f19f415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:45.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19eaa00640 1 --2- 172.21.15.113:0/1406735990 >> v2:172.21.15.136:3300/0 conn(0x7f19f406faa0 0x7f19f40c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f2a00640 1 -- 172.21.15.113:0/1406735990 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850243950 0 0) 0x7f19f40c4ce0 con 0x7f19f406ed80 2024-03-04T08:01:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f2a00640 1 -- 172.21.15.113:0/1406735990 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19d8003530 con 0x7f19f406ed80 2024-03-04T08:01:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f2a00640 1 -- 172.21.15.113:0/1406735990 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4175672032 0 0) 0x7f19d8003530 con 0x7f19f406ed80 2024-03-04T08:01:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f3400640 1 -- 172.21.15.113:0/1406735990 >> v2:172.21.15.136:3300/0 conn(0x7f19f406faa0 msgr2=0x7f19f40c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f3400640 1 --2- 172.21.15.113:0/1406735990 >> v2:172.21.15.136:3300/0 conn(0x7f19f406faa0 0x7f19f40c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f3400640 1 -- 172.21.15.113:0/1406735990 >> v1:172.21.15.113:6789/0 conn(0x7f19f406ed80 legacy=0x7f19f40ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:45.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.793+0000 7f19f3400640 1 -- 172.21.15.113:0/1406735990 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40c4f80 con 0x7f19f4152950 2024-03-04T08:01:45.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.797+0000 7f19f3400640 1 --2- 172.21.15.113:0/1406735990 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 0x7f19f415bb40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f19e40097c0 tx=0x7f19e401ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ffe736e749f4be9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:45.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.797+0000 7f19f2a00640 1 -- 172.21.15.113:0/1406735990 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e40b5070 con 0x7f19f4152950 2024-03-04T08:01:45.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.797+0000 7f19f2a00640 1 -- 172.21.15.113:0/1406735990 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e40a85f0 con 0x7f19f4152950 2024-03-04T08:01:45.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.797+0000 7f19f2a00640 1 -- 172.21.15.113:0/1406735990 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec001a20 con 0x7f19f4152950 2024-03-04T08:01:45.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 -- 172.21.15.113:0/1406735990 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 msgr2=0x7f19f415bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:45.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 --2- 172.21.15.113:0/1406735990 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 0x7f19f415bb40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f19e40097c0 tx=0x7f19e401ebb0 comp rx=0 tx=0).stop 2024-03-04T08:01:45.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 -- 172.21.15.113:0/1406735990 shutdown_connections 2024-03-04T08:01:45.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 --2- 172.21.15.113:0/1406735990 >> v2:172.21.15.136:3300/0 conn(0x7f19f406faa0 0x7f19f40c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:45.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 --2- 172.21.15.113:0/1406735990 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 0x7f19f415bb40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:45.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 -- 172.21.15.113:0/1406735990 >> 172.21.15.113:0/1406735990 conn(0x7f19f4072f40 msgr2=0x7f19f40706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:45.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 -- 172.21.15.113:0/1406735990 shutdown_connections 2024-03-04T08:01:45.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 -- 172.21.15.113:0/1406735990 wait complete. 2024-03-04T08:01:45.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 Processor -- start 2024-03-04T08:01:45.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 -- start start 2024-03-04T08:01:45.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f406ed80 0x7f19f4158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:45.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f406faa0 0x7f19f41e5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:45.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 0x7f19f41f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:45.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40c9a50 con 0x7f19f406faa0 2024-03-04T08:01:45.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f406fe80 con 0x7f19f406ed80 2024-03-04T08:01:45.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f9200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40c9d30 con 0x7f19f4152950 2024-03-04T08:01:45.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19eaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f406faa0 0x7f19f41e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:45.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19eaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f406faa0 0x7f19f41e5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56138/0 (socket says 172.21.15.113:56138) 2024-03-04T08:01:45.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19eaa00640 1 -- 172.21.15.113:0/2505793757 learned_addr learned my addr 172.21.15.113:0/2505793757 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:45.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f3400640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f406ed80 0x7f19f4158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:45.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f3e00640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 0x7f19f41f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:45.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19eaa00640 1 -- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 msgr2=0x7f19f41f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:45.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19eaa00640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 0x7f19f41f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:45.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19eaa00640 1 -- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f406ed80 msgr2=0x7f19f4158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:45.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19eaa00640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f406ed80 0x7f19f4158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:45.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19eaa00640 1 -- 172.21.15.113:0/2505793757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40ce610 con 0x7f19f406faa0 2024-03-04T08:01:45.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.801+0000 7f19f3400640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f406ed80 0x7f19f4158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:45.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.805+0000 7f19eaa00640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f406faa0 0x7f19f41e5490 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00e960 tx=0x7f19dc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:45.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.805+0000 7f19ebe00640 1 -- 172.21.15.113:0/2505793757 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc017070 con 0x7f19f406faa0 2024-03-04T08:01:45.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.805+0000 7f19ebe00640 1 -- 172.21.15.113:0/2505793757 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc00ca00 con 0x7f19f406faa0 2024-03-04T08:01:45.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.805+0000 7f19f9200640 1 -- 172.21.15.113:0/2505793757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f406c170 con 0x7f19f406faa0 2024-03-04T08:01:45.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.805+0000 7f19ebe00640 1 -- 172.21.15.113:0/2505793757 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00cdb0 con 0x7f19f406faa0 2024-03-04T08:01:45.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.805+0000 7f19f9200640 1 -- 172.21.15.113:0/2505793757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f406c6d0 con 0x7f19f406faa0 2024-03-04T08:01:45.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.805+0000 7f19f9200640 1 -- 172.21.15.113:0/2505793757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f4075a80 con 0x7f19f406faa0 2024-03-04T08:01:45.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.805+0000 7f19ebe00640 1 -- 172.21.15.113:0/2505793757 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19dc01a070 con 0x7f19f406faa0 2024-03-04T08:01:45.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.809+0000 7f19ebe00640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d0066f60 0x7f19d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:45.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.809+0000 7f19ebe00640 1 -- 172.21.15.113:0/2505793757 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f19dc085670 con 0x7f19f406faa0 2024-03-04T08:01:45.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.809+0000 7f19f3400640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d0066f60 0x7f19d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:45.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.809+0000 7f19f3400640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d0066f60 0x7f19d0069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f19e40b2000 tx=0x7f19e40aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:45.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:45.813+0000 7f19ebe00640 1 -- 172.21.15.113:0/2505793757 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f19f406c170 con 0x7f19f406faa0 2024-03-04T08:01:46.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 -- 172.21.15.113:0/2505793757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19f406f880 con 0x7f19f406faa0 2024-03-04T08:01:46.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19ebe00640 1 -- 172.21.15.113:0/2505793757 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+5825 (secure 0 0 0) 0x7f19dc055980 con 0x7f19f406faa0 2024-03-04T08:01:46.019 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:46.019 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":103,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:41.817994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:reconnect","state_seq":10,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534]},"id":1}]} 2024-03-04T08:01:46.019 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 103 2024-03-04T08:01:46.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 -- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d0066f60 msgr2=0x7f19d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:46.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d0066f60 0x7f19d0069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f19e40b2000 tx=0x7f19e40aa070 comp rx=0 tx=0).stop 2024-03-04T08:01:46.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 -- 172.21.15.113:0/2505793757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f406faa0 msgr2=0x7f19f41e5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:46.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f406faa0 0x7f19f41e5490 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00e960 tx=0x7f19dc00ee20 comp rx=0 tx=0).stop 2024-03-04T08:01:46.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 -- 172.21.15.113:0/2505793757 shutdown_connections 2024-03-04T08:01:46.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f4152950 0x7f19f41f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:46.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d0066f60 0x7f19d0069420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:46.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f406faa0 0x7f19f41e5490 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:46.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 --2- 172.21.15.113:0/2505793757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f406ed80 0x7f19f4158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:46.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 -- 172.21.15.113:0/2505793757 >> 172.21.15.113:0/2505793757 conn(0x7f19f4072f40 msgr2=0x7f19f4157970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:46.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 -- 172.21.15.113:0/2505793757 shutdown_connections 2024-03-04T08:01:46.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:46.021+0000 7f19f9200640 1 -- 172.21.15.113:0/2505793757 wait complete. 2024-03-04T08:01:46.042 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:01:48.043 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:48.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdff200640 1 Processor -- start 2024-03-04T08:01:48.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdff200640 1 -- start start 2024-03-04T08:01:48.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdff200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe00151b40 0x7efe000c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:48.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdff200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0006c290 con 0x7efe000c18d0 2024-03-04T08:01:48.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdff200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0006ef10 con 0x7efe00151760 2024-03-04T08:01:48.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdff200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0015aef0 con 0x7efe00151b40 2024-03-04T08:01:48.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfe800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efe000c18d0 0x7efe00071f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49658/0 (socket says 172.21.15.113:49658) 2024-03-04T08:01:48.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfe800640 1 -- 172.21.15.113:0/1473062160 learned_addr learned my addr 172.21.15.113:0/1473062160 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:48.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdf5400640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe00151b40 0x7efe000c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:48.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213249179 0 0) 0x7efe0006c290 con 0x7efe000c18d0 2024-03-04T08:01:48.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde4003530 con 0x7efe000c18d0 2024-03-04T08:01:48.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131118568 0 0) 0x7efde4003530 con 0x7efe000c18d0 2024-03-04T08:01:48.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe00151b40 msgr2=0x7efe000c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:48.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe00151b40 0x7efe000c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 >> v1:172.21.15.136:6789/0 conn(0x7efe00151760 legacy=0x7efe0015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0015ba90 con 0x7efe000c18d0 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7efdf8001a00 con 0x7efe000c18d0 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 >> v1:172.21.15.113:6789/0 conn(0x7efe000c18d0 legacy=0x7efe00071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 0x7efde4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe00151b40 0x7efde4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efde4004d50 0x7efde40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0015aef0 con 0x7efde40039f0 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde400bb60 con 0x7efde4004d50 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde400bd60 con 0x7efe00151b40 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdfde00640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 0x7efde4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.201+0000 7efdf5400640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe00151b40 0x7efde4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:48.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.205+0000 7efdfe800640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efde4004d50 0x7efde40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:48.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.205+0000 7efdfde00640 1 -- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe00151b40 msgr2=0x7efde4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:48.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.205+0000 7efdfde00640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe00151b40 0x7efde4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:48.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.205+0000 7efdfde00640 1 -- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efde4004d50 msgr2=0x7efde40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:48.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.205+0000 7efdfde00640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efde4004d50 0x7efde40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:48.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.205+0000 7efdfde00640 1 -- 172.21.15.113:0/1473062160 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde4003530 con 0x7efde40039f0 2024-03-04T08:01:48.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.205+0000 7efdf5400640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe00151b40 0x7efde4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:01:48.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdfde00640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 0x7efde4003e10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7efdf0001780 tx=0x7efdf001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d431106df19a73f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:48.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00013a0 con 0x7efde40039f0 2024-03-04T08:01:48.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf00052f0 con 0x7efde40039f0 2024-03-04T08:01:48.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdfd400640 1 -- 172.21.15.113:0/1473062160 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0003840 con 0x7efde40039f0 2024-03-04T08:01:48.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdff200640 1 -- 172.21.15.113:0/1473062160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 msgr2=0x7efde4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:48.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdff200640 1 --2- 172.21.15.113:0/1473062160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 0x7efde4003e10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7efdf0001780 tx=0x7efdf001c420 comp rx=0 tx=0).stop 2024-03-04T08:01:48.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdfe800640 1 -- 172.21.15.113:0/1473062160 reap_dead start 2024-03-04T08:01:48.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdff200640 1 -- 172.21.15.113:0/1473062160 shutdown_connections 2024-03-04T08:01:48.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdff200640 1 -- 172.21.15.113:0/1473062160 >> 172.21.15.113:0/1473062160 conn(0x7efe0006e2a0 msgr2=0x7efe0006e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:48.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdff200640 1 -- 172.21.15.113:0/1473062160 shutdown_connections 2024-03-04T08:01:48.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.209+0000 7efdff200640 1 -- 172.21.15.113:0/1473062160 wait complete. 2024-03-04T08:01:48.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdff200640 1 Processor -- start 2024-03-04T08:01:48.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdff200640 1 -- start start 2024-03-04T08:01:48.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdff200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 0x7efe001f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:48.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdff200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe0014e060 0x7efe000c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:48.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdff200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe0014c800 0x7efe0014cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:48.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdff200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0015ab80 con 0x7efde40039f0 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdff200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe000c87e0 con 0x7efe0014e060 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdff200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe000c1e20 con 0x7efe0014c800 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdfde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 0x7efe001f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdfde00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 0x7efe001f45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48854/0 (socket says 172.21.15.113:48854) 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdfde00640 1 -- 172.21.15.113:0/3717024333 learned_addr learned my addr 172.21.15.113:0/3717024333 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdfe800640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe0014c800 0x7efe0014cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdf5400640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe0014e060 0x7efe000c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdfde00640 1 -- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe0014c800 msgr2=0x7efe0014cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdfde00640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe0014c800 0x7efe0014cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdfde00640 1 -- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe0014e060 msgr2=0x7efe000c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdfde00640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe0014e060 0x7efe000c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdfde00640 1 -- 172.21.15.113:0/3717024333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0014d1d0 con 0x7efde40039f0 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdf5400640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe0014e060 0x7efe000c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:01:48.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.213+0000 7efdfde00640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 0x7efe001f45e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efdf0000c10 tx=0x7efde4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:48.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.217+0000 7efdf6a00640 1 -- 172.21.15.113:0/3717024333 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0007a40 con 0x7efde40039f0 2024-03-04T08:01:48.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.217+0000 7efdf6a00640 1 -- 172.21.15.113:0/3717024333 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf00054a0 con 0x7efde40039f0 2024-03-04T08:01:48.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.217+0000 7efdf6a00640 1 -- 172.21.15.113:0/3717024333 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0003d00 con 0x7efde40039f0 2024-03-04T08:01:48.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.217+0000 7efdff200640 1 -- 172.21.15.113:0/3717024333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0015d3b0 con 0x7efde40039f0 2024-03-04T08:01:48.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.217+0000 7efdff200640 1 -- 172.21.15.113:0/3717024333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0015d930 con 0x7efde40039f0 2024-03-04T08:01:48.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.217+0000 7efdff200640 1 -- 172.21.15.113:0/3717024333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe00075a80 con 0x7efde40039f0 2024-03-04T08:01:48.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.217+0000 7efdf6a00640 1 -- 172.21.15.113:0/3717024333 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efdf00bd020 con 0x7efde40039f0 2024-03-04T08:01:48.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.217+0000 7efdf6a00640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efddc066f90 0x7efddc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:48.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.217+0000 7efdf6a00640 1 -- 172.21.15.113:0/3717024333 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7efdf011ec00 con 0x7efde40039f0 2024-03-04T08:01:48.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.221+0000 7efdf5400640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efddc066f90 0x7efddc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:48.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.221+0000 7efdf5400640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efddc066f90 0x7efddc069450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efe0014e460 tx=0x7efde8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:48.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.221+0000 7efdf6a00640 1 -- 172.21.15.113:0/3717024333 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efdf0125080 con 0x7efde40039f0 2024-03-04T08:01:48.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.469+0000 7efdff200640 1 -- 172.21.15.113:0/3717024333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe0014ec80 con 0x7efde40039f0 2024-03-04T08:01:48.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.469+0000 7efdf6a00640 1 -- 172.21.15.113:0/3717024333 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+5822 (secure 0 0 0) 0x7efdf00c4020 con 0x7efde40039f0 2024-03-04T08:01:48.471 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:48.471 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:46.843157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372]},"id":1}]} 2024-03-04T08:01:48.473 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 104 2024-03-04T08:01:48.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 -- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efddc066f90 msgr2=0x7efddc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:48.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efddc066f90 0x7efddc069450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7efe0014e460 tx=0x7efde8008040 comp rx=0 tx=0).stop 2024-03-04T08:01:48.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 -- 172.21.15.113:0/3717024333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 msgr2=0x7efe001f45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:48.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 0x7efe001f45e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7efdf0000c10 tx=0x7efde4005840 comp rx=0 tx=0).stop 2024-03-04T08:01:48.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 -- 172.21.15.113:0/3717024333 shutdown_connections 2024-03-04T08:01:48.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efe0014c800 0x7efe0014cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:48.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe0014e060 0x7efe000c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:48.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efddc066f90 0x7efddc069450 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:48.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 --2- 172.21.15.113:0/3717024333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde40039f0 0x7efe001f45e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:48.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 -- 172.21.15.113:0/3717024333 >> 172.21.15.113:0/3717024333 conn(0x7efe0006e2a0 msgr2=0x7efe0014e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:48.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 -- 172.21.15.113:0/3717024333 shutdown_connections 2024-03-04T08:01:48.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:48.473+0000 7efdff200640 1 -- 172.21.15.113:0/3717024333 wait complete. 2024-03-04T08:01:48.497 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:01:50.498 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:50.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea87a00640 1 Processor -- start 2024-03-04T08:01:50.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea87a00640 1 -- start start 2024-03-04T08:01:50.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea87a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea80152d60 0x7fea800c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:50.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea87a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea80159470 0x7fea800ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:50.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea87a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea80159980 con 0x7fea800c38c0 2024-03-04T08:01:50.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea87a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8015a5b0 con 0x7fea80152d60 2024-03-04T08:01:50.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea87a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8006f3b0 con 0x7fea80159470 2024-03-04T08:01:50.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea85000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea80152d60 0x7fea800c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:50.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea85000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea80152d60 0x7fea800c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48864/0 (socket says 172.21.15.113:48864) 2024-03-04T08:01:50.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea85000640 1 -- 172.21.15.113:0/345893491 learned_addr learned my addr 172.21.15.113:0/345893491 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:50.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.637+0000 7fea85a00640 1 --2- 172.21.15.113:0/345893491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea80159470 0x7fea800ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:50.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.641+0000 7fea85000640 1 -- 172.21.15.113:0/345893491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea80159470 msgr2=0x7fea800ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:50.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.641+0000 7fea85000640 1 --2- 172.21.15.113:0/345893491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea80159470 0x7fea800ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:50.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.641+0000 7fea85000640 1 -- 172.21.15.113:0/345893491 >> v1:172.21.15.136:6789/0 conn(0x7fea800c38c0 legacy=0x7fea80159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:50.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.641+0000 7fea85000640 1 -- 172.21.15.113:0/345893491 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80153140 con 0x7fea80152d60 2024-03-04T08:01:50.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.641+0000 7fea7f400640 1 -- 172.21.15.113:0/345893491 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48672192 0 0) 0x7fea80159980 con 0x7fea800c38c0 2024-03-04T08:01:50.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea85000640 1 --2- 172.21.15.113:0/345893491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea80152d60 0x7fea800c8d80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fea74009870 tx=0x7fea7401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d204ade91bbfd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:50.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea7f400640 1 -- 172.21.15.113:0/345893491 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740b2070 con 0x7fea80152d60 2024-03-04T08:01:50.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea7f400640 1 -- 172.21.15.113:0/345893491 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea740a9070 con 0x7fea80152d60 2024-03-04T08:01:50.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea7f400640 1 -- 172.21.15.113:0/345893491 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740ad880 con 0x7fea80152d60 2024-03-04T08:01:50.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea87a00640 1 -- 172.21.15.113:0/345893491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea80152d60 msgr2=0x7fea800c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:50.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea87a00640 1 --2- 172.21.15.113:0/345893491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea80152d60 0x7fea800c8d80 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fea74009870 tx=0x7fea7401c560 comp rx=0 tx=0).stop 2024-03-04T08:01:50.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea87a00640 1 -- 172.21.15.113:0/345893491 shutdown_connections 2024-03-04T08:01:50.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea87a00640 1 --2- 172.21.15.113:0/345893491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea80159470 0x7fea800ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:50.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea87a00640 1 --2- 172.21.15.113:0/345893491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea80152d60 0x7fea800c8d80 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:50.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea87a00640 1 -- 172.21.15.113:0/345893491 >> 172.21.15.113:0/345893491 conn(0x7fea80066db0 msgr2=0x7fea800671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:50.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea87a00640 1 -- 172.21.15.113:0/345893491 shutdown_connections 2024-03-04T08:01:50.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.645+0000 7fea87a00640 1 -- 172.21.15.113:0/345893491 wait complete. 2024-03-04T08:01:50.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 Processor -- start 2024-03-04T08:01:50.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 -- start start 2024-03-04T08:01:50.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea800c38c0 0x7fea800b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:50.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea80152d60 0x7fea800b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:50.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea80159470 0x7fea800b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea80070230 con 0x7fea800c38c0 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800c5130 con 0x7fea80159470 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea800703b0 con 0x7fea80152d60 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea85000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea800c38c0 0x7fea800b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea85000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea800c38c0 0x7fea800b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48876/0 (socket says 172.21.15.113:48876) 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea85000640 1 -- 172.21.15.113:0/2579147587 learned_addr learned my addr 172.21.15.113:0/2579147587 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea7fe00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea80152d60 0x7fea800b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea85a00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea80159470 0x7fea800b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea85000640 1 -- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea80152d60 msgr2=0x7fea800b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea85000640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea80152d60 0x7fea800b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:50.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea85000640 1 -- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea80159470 msgr2=0x7fea800b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:50.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea85000640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea80159470 0x7fea800b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:50.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea85000640 1 -- 172.21.15.113:0/2579147587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea800cc4d0 con 0x7fea800c38c0 2024-03-04T08:01:50.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea85000640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea800c38c0 0x7fea800b15f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fea7401a190 tx=0x7fea7401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:50.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea7cc00640 1 -- 172.21.15.113:0/2579147587 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740bc020 con 0x7fea800c38c0 2024-03-04T08:01:50.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea7cc00640 1 -- 172.21.15.113:0/2579147587 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea740addc0 con 0x7fea800c38c0 2024-03-04T08:01:50.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 -- 172.21.15.113:0/2579147587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea800cc180 con 0x7fea800c38c0 2024-03-04T08:01:50.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea7cc00640 1 -- 172.21.15.113:0/2579147587 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea740b2040 con 0x7fea800c38c0 2024-03-04T08:01:50.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 -- 172.21.15.113:0/2579147587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea800ce3a0 con 0x7fea800c38c0 2024-03-04T08:01:50.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.649+0000 7fea87a00640 1 -- 172.21.15.113:0/2579147587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea80068eb0 con 0x7fea800c38c0 2024-03-04T08:01:50.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.653+0000 7fea7cc00640 1 -- 172.21.15.113:0/2579147587 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea740bf020 con 0x7fea800c38c0 2024-03-04T08:01:50.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.653+0000 7fea7cc00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea60066f60 0x7fea60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:50.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.653+0000 7fea7cc00640 1 -- 172.21.15.113:0/2579147587 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7fea7411fba0 con 0x7fea800c38c0 2024-03-04T08:01:50.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.653+0000 7fea7fe00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea60066f60 0x7fea60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:50.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.657+0000 7fea7fe00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea60066f60 0x7fea60069420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fea800ce7e0 tx=0x7fea6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:50.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.657+0000 7fea7cc00640 1 -- 172.21.15.113:0/2579147587 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fea7411fe20 con 0x7fea800c38c0 2024-03-04T08:01:50.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 -- 172.21.15.113:0/2579147587 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea80075c00 con 0x7fea800c38c0 2024-03-04T08:01:50.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea7cc00640 1 -- 172.21.15.113:0/2579147587 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+5822 (secure 0 0 0) 0x7fea740c6030 con 0x7fea800c38c0 2024-03-04T08:01:50.904 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:50.904 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:46.843157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372]},"id":1}]} 2024-03-04T08:01:50.905 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 104 2024-03-04T08:01:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 -- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea60066f60 msgr2=0x7fea60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea60066f60 0x7fea60069420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fea800ce7e0 tx=0x7fea6c009040 comp rx=0 tx=0).stop 2024-03-04T08:01:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 -- 172.21.15.113:0/2579147587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea800c38c0 msgr2=0x7fea800b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea800c38c0 0x7fea800b15f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fea7401a190 tx=0x7fea7401ada0 comp rx=0 tx=0).stop 2024-03-04T08:01:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 -- 172.21.15.113:0/2579147587 shutdown_connections 2024-03-04T08:01:50.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea80159470 0x7fea800b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fea80152d60 0x7fea800b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fea60066f60 0x7fea60069420 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 --2- 172.21.15.113:0/2579147587 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea800c38c0 0x7fea800b15f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 -- 172.21.15.113:0/2579147587 >> 172.21.15.113:0/2579147587 conn(0x7fea80066db0 msgr2=0x7fea8006d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 -- 172.21.15.113:0/2579147587 shutdown_connections 2024-03-04T08:01:50.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:50.905+0000 7fea87a00640 1 -- 172.21.15.113:0/2579147587 wait complete. 2024-03-04T08:01:50.930 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:01:52.931 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:53.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2bbe00640 1 Processor -- start 2024-03-04T08:01:53.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2bbe00640 1 -- start start 2024-03-04T08:01:53.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2bbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff2b406ff20 0x7ff2b415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:53.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2bbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff2b40c8b90 0x7ff2b415c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:53.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2bbe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b4152e30 con 0x7ff2b4152a50 2024-03-04T08:01:53.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2bbe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b40c31b0 con 0x7ff2b406ff20 2024-03-04T08:01:53.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2bbe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b406f840 con 0x7ff2b40c8b90 2024-03-04T08:01:53.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2b8a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff2b406ff20 0x7ff2b415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:53.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2b9400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff2b4152a50 0x7ff2b40ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49672/0 (socket says 172.21.15.113:49672) 2024-03-04T08:01:53.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2b9400640 1 -- 172.21.15.113:0/567678438 learned_addr learned my addr 172.21.15.113:0/567678438 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:53.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2b9e00640 1 --2- 172.21.15.113:0/567678438 >> v2:172.21.15.136:3300/0 conn(0x7ff2b40c8b90 0x7ff2b415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:53.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2b8a00640 1 -- 172.21.15.113:0/567678438 >> v2:172.21.15.136:3300/0 conn(0x7ff2b40c8b90 msgr2=0x7ff2b415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:01:53.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2b8a00640 1 --2- 172.21.15.113:0/567678438 >> v2:172.21.15.136:3300/0 conn(0x7ff2b40c8b90 0x7ff2b415c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:53.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2b8a00640 1 -- 172.21.15.113:0/567678438 >> v1:172.21.15.113:6789/0 conn(0x7ff2b4152a50 legacy=0x7ff2b40ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:53.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.097+0000 7ff2b8a00640 1 -- 172.21.15.113:0/567678438 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b406f280 con 0x7ff2b406ff20 2024-03-04T08:01:53.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2b8a00640 1 --2- 172.21.15.113:0/567678438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b406ff20 0x7ff2b415bdd0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff2b406ef00 tx=0x7ff2a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a5222fe23ce754 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:53.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2b3e00640 1 -- 172.21.15.113:0/567678438 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00aa600 con 0x7ff2b406ff20 2024-03-04T08:01:53.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2b3e00640 1 -- 172.21.15.113:0/567678438 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00af070 con 0x7ff2b406ff20 2024-03-04T08:01:53.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2b3e00640 1 -- 172.21.15.113:0/567678438 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00aa9c0 con 0x7ff2b406ff20 2024-03-04T08:01:53.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2bbe00640 1 -- 172.21.15.113:0/567678438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b406ff20 msgr2=0x7ff2b415bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:53.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2bbe00640 1 --2- 172.21.15.113:0/567678438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b406ff20 0x7ff2b415bdd0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ff2b406ef00 tx=0x7ff2a001c480 comp rx=0 tx=0).stop 2024-03-04T08:01:53.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2bbe00640 1 -- 172.21.15.113:0/567678438 shutdown_connections 2024-03-04T08:01:53.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2bbe00640 1 --2- 172.21.15.113:0/567678438 >> v2:172.21.15.136:3300/0 conn(0x7ff2b40c8b90 0x7ff2b415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:53.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2bbe00640 1 --2- 172.21.15.113:0/567678438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b406ff20 0x7ff2b415bdd0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:53.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2bbe00640 1 -- 172.21.15.113:0/567678438 >> 172.21.15.113:0/567678438 conn(0x7ff2b40734d0 msgr2=0x7ff2b4066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:53.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2bbe00640 1 -- 172.21.15.113:0/567678438 shutdown_connections 2024-03-04T08:01:53.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.101+0000 7ff2bbe00640 1 -- 172.21.15.113:0/567678438 wait complete. 2024-03-04T08:01:53.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2bbe00640 1 Processor -- start 2024-03-04T08:01:53.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2bbe00640 1 -- start start 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2bbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b406ff20 0x7ff2b41f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2bbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b40c8b90 0x7ff2b4153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2bbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b4152a50 0x7ff2b414bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2bbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40ce9b0 con 0x7ff2b40c8b90 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2bbe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40ce830 con 0x7ff2b406ff20 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2bbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4165880 con 0x7ff2b4152a50 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2b8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b40c8b90 0x7ff2b4153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2b8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b40c8b90 0x7ff2b4153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48888/0 (socket says 172.21.15.113:48888) 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2b8a00640 1 -- 172.21.15.113:0/1008648775 learned_addr learned my addr 172.21.15.113:0/1008648775 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2b9400640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b406ff20 0x7ff2b41f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2b9e00640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b4152a50 0x7ff2b414bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:53.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2b8a00640 1 -- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b4152a50 msgr2=0x7ff2b414bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:53.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2b8a00640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b4152a50 0x7ff2b414bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:53.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2b8a00640 1 -- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b406ff20 msgr2=0x7ff2b41f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:53.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2b8a00640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b406ff20 0x7ff2b41f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:53.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.105+0000 7ff2b8a00640 1 -- 172.21.15.113:0/1008648775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b406c3c0 con 0x7ff2b40c8b90 2024-03-04T08:01:53.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.109+0000 7ff2b8a00640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b40c8b90 0x7ff2b4153a80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00ac040 tx=0x7ff2a001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:53.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.109+0000 7ff2b1600640 1 -- 172.21.15.113:0/1008648775 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00be610 con 0x7ff2b40c8b90 2024-03-04T08:01:53.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.109+0000 7ff2b1600640 1 -- 172.21.15.113:0/1008648775 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00be7b0 con 0x7ff2b40c8b90 2024-03-04T08:01:53.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.109+0000 7ff2bbe00640 1 -- 172.21.15.113:0/1008648775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b406c6a0 con 0x7ff2b40c8b90 2024-03-04T08:01:53.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.109+0000 7ff2b1600640 1 -- 172.21.15.113:0/1008648775 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00beba0 con 0x7ff2b40c8b90 2024-03-04T08:01:53.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.109+0000 7ff2bbe00640 1 -- 172.21.15.113:0/1008648775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b4153560 con 0x7ff2b40c8b90 2024-03-04T08:01:53.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.109+0000 7ff2bbe00640 1 -- 172.21.15.113:0/1008648775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b4153070 con 0x7ff2b40c8b90 2024-03-04T08:01:53.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.113+0000 7ff2b1600640 1 -- 172.21.15.113:0/1008648775 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2a00bc020 con 0x7ff2b40c8b90 2024-03-04T08:01:53.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.113+0000 7ff2b1600640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f90 0x7ff294069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:53.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.113+0000 7ff2b1600640 1 -- 172.21.15.113:0/1008648775 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7ff2a011f7e0 con 0x7ff2b40c8b90 2024-03-04T08:01:53.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.113+0000 7ff2b9400640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f90 0x7ff294069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:53.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.113+0000 7ff2b9400640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f90 0x7ff294069450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8000f70 tx=0x7ff2a8007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:53.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.121+0000 7ff2b1600640 1 -- 172.21.15.113:0/1008648775 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff2a0016070 con 0x7ff2b40c8b90 2024-03-04T08:01:53.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 -- 172.21.15.113:0/1008648775 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2b4075c00 con 0x7ff2b40c8b90 2024-03-04T08:01:53.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2b1600640 1 -- 172.21.15.113:0/1008648775 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+5822 (secure 0 0 0) 0x7ff2a00efd40 con 0x7ff2b40c8b90 2024-03-04T08:01:53.315 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:53.316 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":104,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:46.843157+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6534,6372]},"id":1}]} 2024-03-04T08:01:53.316 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 104 2024-03-04T08:01:53.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 -- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f90 msgr2=0x7ff294069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:53.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f90 0x7ff294069450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8000f70 tx=0x7ff2a8007e90 comp rx=0 tx=0).stop 2024-03-04T08:01:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 -- 172.21.15.113:0/1008648775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b40c8b90 msgr2=0x7ff2b4153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b40c8b90 0x7ff2b4153a80 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00ac040 tx=0x7ff2a001acc0 comp rx=0 tx=0).stop 2024-03-04T08:01:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 -- 172.21.15.113:0/1008648775 shutdown_connections 2024-03-04T08:01:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b4152a50 0x7ff2b414bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f90 0x7ff294069450 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b40c8b90 0x7ff2b4153a80 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 --2- 172.21.15.113:0/1008648775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b406ff20 0x7ff2b41f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 -- 172.21.15.113:0/1008648775 >> 172.21.15.113:0/1008648775 conn(0x7ff2b40734d0 msgr2=0x7ff2b40ccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 -- 172.21.15.113:0/1008648775 shutdown_connections 2024-03-04T08:01:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:53.317+0000 7ff2bbe00640 1 -- 172.21.15.113:0/1008648775 wait complete. 2024-03-04T08:01:53.336 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:01:55.337 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:01:55.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.477+0000 7f701b200640 1 Processor -- start 2024-03-04T08:01:55.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.477+0000 7f701b200640 1 -- start start 2024-03-04T08:01:55.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f701b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f701c152920 0x7f701c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:55.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f701b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c06bb10 con 0x7f701c06c580 2024-03-04T08:01:55.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f701b200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c0c7a40 con 0x7f701c15bf20 2024-03-04T08:01:55.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f701b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701c0c7d20 con 0x7f701c152920 2024-03-04T08:01:55.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f7019e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f701c152920 0x7f701c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:55.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f7011400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f701c06c580 0x7f701c0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49680/0 (socket says 172.21.15.113:49680) 2024-03-04T08:01:55.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f7011400640 1 -- 172.21.15.113:0/1790814383 learned_addr learned my addr 172.21.15.113:0/1790814383 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:01:55.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f7019e00640 1 -- 172.21.15.113:0/1790814383 >> v1:172.21.15.136:6789/0 conn(0x7f701c15bf20 legacy=0x7f701c15c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:55.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f7019e00640 1 -- 172.21.15.113:0/1790814383 >> v1:172.21.15.113:6789/0 conn(0x7f701c06c580 legacy=0x7f701c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:55.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f7019400640 1 -- 172.21.15.113:0/1790814383 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597230486 0 0) 0x7f701c06bb10 con 0x7f701c06c580 2024-03-04T08:01:55.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.481+0000 7f7019e00640 1 -- 172.21.15.113:0/1790814383 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c06bd80 con 0x7f701c152920 2024-03-04T08:01:55.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f7019e00640 1 --2- 172.21.15.113:0/1790814383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c152920 0x7f701c071a30 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f700c0097c0 tx=0x7f700c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738cfb830b767ba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:55.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f7019400640 1 -- 172.21.15.113:0/1790814383 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c0b0070 con 0x7f701c152920 2024-03-04T08:01:55.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f7019400640 1 -- 172.21.15.113:0/1790814383 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700c0ab600 con 0x7f701c152920 2024-03-04T08:01:55.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f7019400640 1 -- 172.21.15.113:0/1790814383 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c0ab9d0 con 0x7f701c152920 2024-03-04T08:01:55.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 -- 172.21.15.113:0/1790814383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c152920 msgr2=0x7f701c071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:55.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 --2- 172.21.15.113:0/1790814383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c152920 0x7f701c071a30 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f700c0097c0 tx=0x7f700c01c410 comp rx=0 tx=0).stop 2024-03-04T08:01:55.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 -- 172.21.15.113:0/1790814383 shutdown_connections 2024-03-04T08:01:55.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 --2- 172.21.15.113:0/1790814383 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c152920 0x7f701c071a30 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:55.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 -- 172.21.15.113:0/1790814383 >> 172.21.15.113:0/1790814383 conn(0x7f701c0c9a30 msgr2=0x7f701c0c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:55.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 -- 172.21.15.113:0/1790814383 shutdown_connections 2024-03-04T08:01:55.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 -- 172.21.15.113:0/1790814383 wait complete. 2024-03-04T08:01:55.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 Processor -- start 2024-03-04T08:01:55.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 -- start start 2024-03-04T08:01:55.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f701c06c580 0x7f701c163420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:55.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f701c152920 0x7f701c0c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c15bf20 0x7f701c1626d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0c4430 con 0x7f701c15bf20 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c152d00 con 0x7f701c06c580 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.485+0000 7f701b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c0c45b0 con 0x7f701c152920 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c15bf20 0x7f701c1626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c15bf20 0x7f701c1626d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48894/0 (socket says 172.21.15.113:48894) 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701a800640 1 -- 172.21.15.113:0/3417502619 learned_addr learned my addr 172.21.15.113:0/3417502619 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f7019e00640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f701c06c580 0x7f701c163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f7011400640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f701c152920 0x7f701c0c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701a800640 1 -- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f701c152920 msgr2=0x7f701c0c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701a800640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f701c152920 0x7f701c0c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701a800640 1 -- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f701c06c580 msgr2=0x7f701c163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:55.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701a800640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f701c06c580 0x7f701c163420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:55.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701a800640 1 -- 172.21.15.113:0/3417502619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c0c6c40 con 0x7f701c15bf20 2024-03-04T08:01:55.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701a800640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c15bf20 0x7f701c1626d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f701400c460 tx=0x7f701400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:55.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f7012a00640 1 -- 172.21.15.113:0/3417502619 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014016020 con 0x7f701c15bf20 2024-03-04T08:01:55.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f7012a00640 1 -- 172.21.15.113:0/3417502619 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7014005220 con 0x7f701c15bf20 2024-03-04T08:01:55.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f7012a00640 1 -- 172.21.15.113:0/3417502619 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014007030 con 0x7f701c15bf20 2024-03-04T08:01:55.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701b200640 1 -- 172.21.15.113:0/3417502619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c1630a0 con 0x7f701c15bf20 2024-03-04T08:01:55.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701b200640 1 -- 172.21.15.113:0/3417502619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c157400 con 0x7f701c15bf20 2024-03-04T08:01:55.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f701b200640 1 -- 172.21.15.113:0/3417502619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c075a80 con 0x7f701c15bf20 2024-03-04T08:01:55.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.489+0000 7f7012a00640 1 -- 172.21.15.113:0/3417502619 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70140071d0 con 0x7f701c15bf20 2024-03-04T08:01:55.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.493+0000 7f7012a00640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ff8066f60 0x7f6ff8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:01:55.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.493+0000 7f7012a00640 1 -- 172.21.15.113:0/3417502619 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f7014084dd0 con 0x7f701c15bf20 2024-03-04T08:01:55.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.493+0000 7f7019e00640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ff8066f60 0x7f6ff8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:01:55.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.493+0000 7f7019e00640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ff8066f60 0x7f6ff8069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f700c0ad000 tx=0x7f700c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:01:55.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.497+0000 7f7012a00640 1 -- 172.21.15.113:0/3417502619 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7014020020 con 0x7f701c15bf20 2024-03-04T08:01:55.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 -- 172.21.15.113:0/3417502619 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f701c0c8300 con 0x7f701c15bf20 2024-03-04T08:01:55.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f7012a00640 1 -- 172.21.15.113:0/3417502619 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+5827 (secure 0 0 0) 0x7f701c0c6c40 con 0x7f701c15bf20 2024-03-04T08:01:55.695 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:01:55.695 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":105,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:54.112369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6372,6534,6807]},"id":1}]} 2024-03-04T08:01:55.696 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 105 2024-03-04T08:01:55.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 -- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ff8066f60 msgr2=0x7f6ff8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ff8066f60 0x7f6ff8069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f700c0ad000 tx=0x7f700c0a6000 comp rx=0 tx=0).stop 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 -- 172.21.15.113:0/3417502619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c15bf20 msgr2=0x7f701c1626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c15bf20 0x7f701c1626d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f701400c460 tx=0x7f701400c920 comp rx=0 tx=0).stop 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 -- 172.21.15.113:0/3417502619 shutdown_connections 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6ff8066f60 0x7f6ff8069420 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f701c15bf20 0x7f701c1626d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f701c152920 0x7f701c0c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 --2- 172.21.15.113:0/3417502619 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f701c06c580 0x7f701c163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 -- 172.21.15.113:0/3417502619 >> 172.21.15.113:0/3417502619 conn(0x7f701c0c9a30 msgr2=0x7f701c151b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 -- 172.21.15.113:0/3417502619 shutdown_connections 2024-03-04T08:01:55.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:01:55.697+0000 7f701b200640 1 -- 172.21.15.113:0/3417502619 wait complete. 2024-03-04T08:01:55.717 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=2, replacing gid=6053 2024-03-04T08:01:55.717 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.b 2024-03-04T08:02:00.123 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at e48859474c4 v0.52 2024-03-04T08:02:00.127 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.52 2024-03-04T08:02:00.718 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2024-03-04T08:02:00.718 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-04T08:02:00.718 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-04T08:02:00.721 INFO:tasks.ceph.mds.b:Started 2024-03-04T08:02:00.721 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:00.798 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 53 -eq 48 ] 2024-03-04T08:02:00.798 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.53 2024-03-04T08:02:00.798 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.53 2024-03-04T08:02:00.798 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.53 2024-03-04T08:02:00.798 INFO:tasks.workunit.client.0.smithi113.stdout:v0.53 2024-03-04T08:02:00.799 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2024-03-04T08:02:00.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.892+0000 7fee53800640 1 Processor -- start 2024-03-04T08:02:00.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.892+0000 7fee53800640 1 -- start start 2024-03-04T08:02:00.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee53800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fee4c0c85f0 0x7fee4c06b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee53800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c0c1ec0 con 0x7fee4c06b5a0 2024-03-04T08:02:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee53800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c0c9b30 con 0x7fee4c0b20b0 2024-03-04T08:02:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee53800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c0c12a0 con 0x7fee4c0c85f0 2024-03-04T08:02:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee51800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fee4c06b5a0 0x7fee4c161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56420/0 (socket says 172.21.15.113:56420) 2024-03-04T08:02:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee51800640 1 -- 172.21.15.113:0/122567253 learned_addr learned my addr 172.21.15.113:0/122567253 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:02:00.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee43e00640 1 --2- 172.21.15.113:0/122567253 >> v2:172.21.15.136:3300/0 conn(0x7fee4c0c85f0 0x7fee4c06b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:00.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee4be00640 1 -- 172.21.15.113:0/122567253 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163044695 0 0) 0x7fee4c0c1ec0 con 0x7fee4c06b5a0 2024-03-04T08:02:00.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee4be00640 1 -- 172.21.15.113:0/122567253 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee30003530 con 0x7fee4c06b5a0 2024-03-04T08:02:00.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee43e00640 1 -- 172.21.15.113:0/122567253 >> v1:172.21.15.136:6789/0 conn(0x7fee4c0b20b0 legacy=0x7fee4c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:00.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee43e00640 1 -- 172.21.15.113:0/122567253 >> v1:172.21.15.113:6789/0 conn(0x7fee4c06b5a0 legacy=0x7fee4c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:00.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.896+0000 7fee43e00640 1 -- 172.21.15.113:0/122567253 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0c1580 con 0x7fee4c0c85f0 2024-03-04T08:02:00.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee43e00640 1 --2- 172.21.15.113:0/122567253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee4c0c85f0 0x7fee4c06b060 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fee340097c0 tx=0x7fee3401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9552bd9180eea545 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:00.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee4be00640 1 -- 172.21.15.113:0/122567253 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340b5070 con 0x7fee4c0c85f0 2024-03-04T08:02:00.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee4be00640 1 -- 172.21.15.113:0/122567253 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee340a8990 con 0x7fee4c0c85f0 2024-03-04T08:02:00.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee4be00640 1 -- 172.21.15.113:0/122567253 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340a8d40 con 0x7fee4c0c85f0 2024-03-04T08:02:00.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee53800640 1 -- 172.21.15.113:0/122567253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee4c0c85f0 msgr2=0x7fee4c06b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:00.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee53800640 1 --2- 172.21.15.113:0/122567253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee4c0c85f0 0x7fee4c06b060 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fee340097c0 tx=0x7fee3401ec90 comp rx=0 tx=0).stop 2024-03-04T08:02:00.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee53800640 1 -- 172.21.15.113:0/122567253 shutdown_connections 2024-03-04T08:02:00.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee53800640 1 --2- 172.21.15.113:0/122567253 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee4c0c85f0 0x7fee4c06b060 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:00.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee53800640 1 -- 172.21.15.113:0/122567253 >> 172.21.15.113:0/122567253 conn(0x7fee4c06e820 msgr2=0x7fee4c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:00.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee53800640 1 -- 172.21.15.113:0/122567253 shutdown_connections 2024-03-04T08:02:00.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee53800640 1 -- 172.21.15.113:0/122567253 wait complete. 2024-03-04T08:02:00.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee53800640 1 Processor -- start 2024-03-04T08:02:00.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.900+0000 7fee53800640 1 -- start start 2024-03-04T08:02:00.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee53800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee4c06b5a0 0x7fee4c1df220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:00.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee53800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee4c0b20b0 0x7fee4c156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:00.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee53800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee4c0c85f0 0x7fee4c14eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:00.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee53800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c157e20 con 0x7fee4c0b20b0 2024-03-04T08:02:00.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee53800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c1571f0 con 0x7fee4c06b5a0 2024-03-04T08:02:00.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee53800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0c2400 con 0x7fee4c0c85f0 2024-03-04T08:02:00.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee43e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee4c0b20b0 0x7fee4c156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:00.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee43e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee4c0b20b0 0x7fee4c156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52140/0 (socket says 172.21.15.113:52140) 2024-03-04T08:02:00.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee43e00640 1 -- 172.21.15.113:0/2442917126 learned_addr learned my addr 172.21.15.113:0/2442917126 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:00.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee50e00640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee4c06b5a0 0x7fee4c1df220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:00.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee51800640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee4c0c85f0 0x7fee4c14eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:00.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee43e00640 1 -- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee4c0c85f0 msgr2=0x7fee4c14eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:00.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee43e00640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee4c0c85f0 0x7fee4c14eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:00.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee43e00640 1 -- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee4c06b5a0 msgr2=0x7fee4c1df220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:00.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee43e00640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee4c06b5a0 0x7fee4c1df220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:00.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee43e00640 1 -- 172.21.15.113:0/2442917126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c1ec480 con 0x7fee4c0b20b0 2024-03-04T08:02:00.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee50e00640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee4c06b5a0 0x7fee4c1df220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:02:00.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee43e00640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee4c0b20b0 0x7fee4c156870 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fee340097c0 tx=0x7fee3401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:00.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee49600640 1 -- 172.21.15.113:0/2442917126 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340ca070 con 0x7fee4c0b20b0 2024-03-04T08:02:00.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee49600640 1 -- 172.21.15.113:0/2442917126 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee34004040 con 0x7fee4c0b20b0 2024-03-04T08:02:00.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee53800640 1 -- 172.21.15.113:0/2442917126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c1ec760 con 0x7fee4c0b20b0 2024-03-04T08:02:00.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee49600640 1 -- 172.21.15.113:0/2442917126 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340b5040 con 0x7fee4c0b20b0 2024-03-04T08:02:00.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee53800640 1 -- 172.21.15.113:0/2442917126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c1ecd20 con 0x7fee4c0b20b0 2024-03-04T08:02:00.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.904+0000 7fee53800640 1 -- 172.21.15.113:0/2442917126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c075c00 con 0x7fee4c0b20b0 2024-03-04T08:02:00.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.908+0000 7fee49600640 1 -- 172.21.15.113:0/2442917126 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee340cd070 con 0x7fee4c0b20b0 2024-03-04T08:02:00.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.908+0000 7fee49600640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee28066f60 0x7fee28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:00.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.908+0000 7fee49600640 1 -- 172.21.15.113:0/2442917126 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7fee3412ad30 con 0x7fee4c0b20b0 2024-03-04T08:02:00.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.908+0000 7fee50e00640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee28066f60 0x7fee28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:00.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.908+0000 7fee50e00640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee28066f60 0x7fee28069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fee3c000f70 tx=0x7fee3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:00.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:00.912+0000 7fee49600640 1 -- 172.21.15.113:0/2442917126 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fee340187f0 con 0x7fee4c0b20b0 2024-03-04T08:02:01.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.164+0000 7fee53800640 1 -- 172.21.15.113:0/2442917126 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee4c0c83d0 con 0x7fee4c0b20b0 2024-03-04T08:02:01.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.164+0000 7fee49600640 1 -- 172.21.15.113:0/2442917126 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6622 (secure 0 0 0) 0x7fee340fb7c0 con 0x7fee4c0b20b0 2024-03-04T08:02:01.164 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:01.164 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:54.112369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6807,6534,6372]},"id":1}]} 2024-03-04T08:02:01.165 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 106 2024-03-04T08:02:01.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 -- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee28066f60 msgr2=0x7fee28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee28066f60 0x7fee28069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fee3c000f70 tx=0x7fee3c009040 comp rx=0 tx=0).stop 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 -- 172.21.15.113:0/2442917126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee4c0b20b0 msgr2=0x7fee4c156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee4c0b20b0 0x7fee4c156870 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fee340097c0 tx=0x7fee3401ba30 comp rx=0 tx=0).stop 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 -- 172.21.15.113:0/2442917126 shutdown_connections 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee4c0c85f0 0x7fee4c14eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee28066f60 0x7fee28069420 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee4c0b20b0 0x7fee4c156870 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 --2- 172.21.15.113:0/2442917126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee4c06b5a0 0x7fee4c1df220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 -- 172.21.15.113:0/2442917126 >> 172.21.15.113:0/2442917126 conn(0x7fee4c06e820 msgr2=0x7fee4c06ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 -- 172.21.15.113:0/2442917126 shutdown_connections 2024-03-04T08:02:01.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:01.168+0000 7fee53800640 1 -- 172.21.15.113:0/2442917126 wait complete. 2024-03-04T08:02:01.187 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2024-03-04T08:02:01.187 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 27 secs before thrashing 2024-03-04T08:02:03.889 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:02:03.893 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:02:17.552 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 14% (55/386) Updating files: 15% (58/386) Updating files: 16% (62/386) Updating files: 17% (66/386) Updating files: 18% (70/386) Updating files: 19% (74/386) Updating files: 20% (78/386) Updating files: 21% (82/386) Updating files: 22% (85/386) Updating files: 23% (89/386) Updating files: 24% (93/386) Updating files: 25% (97/386) Updating files: 26% (101/386) Updating files: 27% (105/386) Updating files: 28% (109/386) Updating files: 29% (112/386) Updating files: 30% (116/386) Updating files: 30% (118/386) Updating files: 31% (120/386) Updating files: 32% (124/386) Updating files: 33% (128/386) Updating files: 34% (132/386) Updating files: 35% (136/386) Updating files: 36% (139/386) Updating files: 37% (143/386) Updating files: 38% (147/386) Updating files: 38% (149/386) Updating files: 39% (151/386) Updating files: 40% (155/386) Updating files: 41% (159/386) Updating files: 42% (163/386) Updating files: 43% (166/386) Updating files: 43% (167/386) Updating files: 44% (170/386) Updating files: 45% (174/386) Updating files: 46% (178/386) Updating files: 47% (182/386) Updating files: 48% (186/386) Updating files: 49% (190/386) Updating files: 50% (193/386) Updating files: 51% (197/386) Updating files: 52% (201/386) Updating files: 53% (205/386) Updating files: 54% (209/386) Updating files: 55% (213/386) Updating files: 55% (215/386) Updating files: 56% (217/386) Updating files: 57% (221/386) Updating files: 58% (224/386) Updating files: 59% (228/386) Updating files: 60% (232/386) Updating files: 61% (236/386) Updating files: 62% (240/386) Updating files: 63% (244/386) Updating files: 64% (248/386) Updating files: 65% (251/386) Updating files: 66% (255/386) Updating files: 67% (259/386) Updating files: 68% (263/386) Updating files: 69% (267/386) Updating files: 70% (271/386) Updating files: 71% (275/386) Updating files: 72% (278/386) Updating files: 73% (282/386) Updating files: 73% (284/386) Updating files: 74% (286/386) Updating files: 74% (288/386) Updating files: 74% (289/386) Updating files: 75% (290/386) Updating files: 76% (294/386) Updating files: 77% (298/386) Updating files: 78% (302/386) Updating files: 79% (305/386) Updating files: 80% (309/386) Updating files: 81% (313/386) Updating files: 82% (317/386) Updating files: 83% (321/386) Updating files: 84% (325/386) Updating files: 85% (329/386) Updating files: 86% (332/386) Updating files: 87% (336/386) Updating files: 88% (340/386) Updating files: 89% (344/386) Updating files: 90% (348/386) Updating files: 91% (352/386) Updating files: 92% (356/386) Updating files: 93% (359/386) Updating files: 94% (363/386) Updating files: 95% (367/386) Updating files: 96% (371/386) Updating files: 97% (375/386) Updating files: 98% (379/386) Updating files: 99% (383/386) Updating files: 100% (386/386) Updating files: 100% (386/386), done. 2024-03-04T08:02:17.609 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 2528b5ee105 v0.43 2024-03-04T08:02:17.617 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.53 2024-03-04T08:02:18.113 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 54 -eq 48 ] 2024-03-04T08:02:18.113 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.54 2024-03-04T08:02:18.113 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.54 2024-03-04T08:02:18.113 INFO:tasks.workunit.client.0.smithi113.stdout:v0.54 2024-03-04T08:02:18.113 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.54 2024-03-04T08:02:22.463 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 48% (102/212) Updating files: 49% (104/212) Updating files: 50% (106/212) Updating files: 51% (109/212) Updating files: 52% (111/212) Updating files: 53% (113/212) Updating files: 54% (115/212) Updating files: 55% (117/212) Updating files: 56% (119/212) Updating files: 57% (121/212) Updating files: 58% (123/212) Updating files: 59% (126/212) Updating files: 60% (128/212) Updating files: 61% (130/212) Updating files: 62% (132/212) Updating files: 63% (134/212) Updating files: 64% (136/212) Updating files: 65% (138/212) Updating files: 66% (140/212) Updating files: 67% (143/212) Updating files: 68% (145/212) Updating files: 69% (147/212) Updating files: 70% (149/212) Updating files: 71% (151/212) Updating files: 72% (153/212) Updating files: 73% (155/212) Updating files: 74% (157/212) Updating files: 75% (159/212) Updating files: 76% (162/212) Updating files: 77% (164/212) Updating files: 78% (166/212) Updating files: 79% (168/212) Updating files: 80% (170/212) Updating files: 81% (172/212) Updating files: 82% (174/212) Updating files: 83% (176/212) Updating files: 84% (179/212) Updating files: 85% (181/212) Updating files: 86% (183/212) Updating files: 87% (185/212) Updating files: 88% (187/212) Updating files: 89% (189/212) Updating files: 90% (191/212) Updating files: 91% (193/212) Updating files: 92% (196/212) Updating files: 93% (198/212) Updating files: 94% (200/212) Updating files: 95% (202/212) Updating files: 96% (204/212) Updating files: 97% (206/212) Updating files: 98% (208/212) Updating files: 99% (210/212) Updating files: 100% (212/212) Updating files: 100% (212/212), done. 2024-03-04T08:02:22.510 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 60b84b095b1 v0.54 2024-03-04T08:02:22.522 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.54 2024-03-04T08:02:23.117 INFO:tasks.workunit.client.0.smithi113.stdout:v0.55 2024-03-04T08:02:23.117 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 55 -eq 48 ] 2024-03-04T08:02:23.117 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.55 2024-03-04T08:02:23.117 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.55 2024-03-04T08:02:23.117 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.55 2024-03-04T08:02:28.188 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:28.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dcf000640 1 Processor -- start 2024-03-04T08:02:28.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dcf000640 1 -- start start 2024-03-04T08:02:28.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dcf000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc8154a90 0x7f6dc80c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dcf000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6dc80b2f80 0x7f6dc815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dcf000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc80c22d0 con 0x7f6dc80c8940 2024-03-04T08:02:28.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dcf000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc80c20a0 con 0x7f6dc80b2f80 2024-03-04T08:02:28.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dcf000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc815b510 con 0x7f6dc8154a90 2024-03-04T08:02:28.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dce600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6dc80c8940 0x7f6dc815e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42242/0 (socket says 172.21.15.113:42242) 2024-03-04T08:02:28.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dce600640 1 -- 172.21.15.113:0/449191040 learned_addr learned my addr 172.21.15.113:0/449191040 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:02:28.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dcdc00640 1 --2- 172.21.15.113:0/449191040 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc8154a90 0x7f6dc80c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.348+0000 7f6dcd200640 1 --2- 172.21.15.113:0/449191040 >> v2:172.21.15.136:3300/0 conn(0x7f6dc80b2f80 0x7f6dc815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dc7e00640 1 -- 172.21.15.113:0/449191040 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155757801 0 0) 0x7f6dc80c22d0 con 0x7f6dc80c8940 2024-03-04T08:02:28.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dc7e00640 1 -- 172.21.15.113:0/449191040 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6db0003530 con 0x7f6dc80c8940 2024-03-04T08:02:28.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dcdc00640 1 -- 172.21.15.113:0/449191040 >> v2:172.21.15.136:3300/0 conn(0x7f6dc80b2f80 msgr2=0x7f6dc815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:28.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dc7e00640 1 -- 172.21.15.113:0/449191040 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3382400345 0 0) 0x7f6db0003530 con 0x7f6dc80c8940 2024-03-04T08:02:28.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dcdc00640 1 --2- 172.21.15.113:0/449191040 >> v2:172.21.15.136:3300/0 conn(0x7f6dc80b2f80 0x7f6dc815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dcdc00640 1 -- 172.21.15.113:0/449191040 >> v1:172.21.15.113:6789/0 conn(0x7f6dc80c8940 legacy=0x7f6dc815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dcdc00640 1 -- 172.21.15.113:0/449191040 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc80c7b20 con 0x7f6dc8154a90 2024-03-04T08:02:28.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dcdc00640 1 --2- 172.21.15.113:0/449191040 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc8154a90 0x7f6dc80c9240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0097c0 tx=0x7f6dbc01ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=461346429e87a533 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:28.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dc7e00640 1 -- 172.21.15.113:0/449191040 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc0b5070 con 0x7f6dc8154a90 2024-03-04T08:02:28.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dc7e00640 1 -- 172.21.15.113:0/449191040 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc0a85f0 con 0x7f6dc8154a90 2024-03-04T08:02:28.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.352+0000 7f6dc7e00640 1 -- 172.21.15.113:0/449191040 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0001a00 con 0x7f6dc8154a90 2024-03-04T08:02:28.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.356+0000 7f6dcf000640 1 -- 172.21.15.113:0/449191040 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc8154a90 msgr2=0x7f6dc80c9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.356+0000 7f6dcf000640 1 --2- 172.21.15.113:0/449191040 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc8154a90 0x7f6dc80c9240 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0097c0 tx=0x7f6dbc01ebb0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.356+0000 7f6dcf000640 1 -- 172.21.15.113:0/449191040 shutdown_connections 2024-03-04T08:02:28.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.356+0000 7f6dcf000640 1 --2- 172.21.15.113:0/449191040 >> v2:172.21.15.136:3300/0 conn(0x7f6dc80b2f80 0x7f6dc815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.356+0000 7f6dcf000640 1 --2- 172.21.15.113:0/449191040 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc8154a90 0x7f6dc80c9240 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.356+0000 7f6dcf000640 1 -- 172.21.15.113:0/449191040 >> 172.21.15.113:0/449191040 conn(0x7f6dc80cd080 msgr2=0x7f6dc80cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:28.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.356+0000 7f6dcf000640 1 -- 172.21.15.113:0/449191040 shutdown_connections 2024-03-04T08:02:28.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.356+0000 7f6dcf000640 1 -- 172.21.15.113:0/449191040 wait complete. 2024-03-04T08:02:28.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.356+0000 7f6dcf000640 1 Processor -- start 2024-03-04T08:02:28.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.356+0000 7f6dcf000640 1 -- start start 2024-03-04T08:02:28.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcf000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dc80b2f80 0x7f6dc8159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcf000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc80c8940 0x7f6dc80c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcf000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc8154a90 0x7f6dc80c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcf000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8154e70 con 0x7f6dc8154a90 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcf000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80c9780 con 0x7f6dc80b2f80 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcf000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8154ff0 con 0x7f6dc80c8940 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dce600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc8154a90 0x7f6dc80c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dce600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc8154a90 0x7f6dc80c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60394/0 (socket says 172.21.15.113:60394) 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dce600640 1 -- 172.21.15.113:0/1690634458 learned_addr learned my addr 172.21.15.113:0/1690634458 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcdc00640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dc80b2f80 0x7f6dc8159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcd200640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc80c8940 0x7f6dc80c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dce600640 1 -- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc80c8940 msgr2=0x7f6dc80c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dce600640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc80c8940 0x7f6dc80c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dce600640 1 -- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dc80b2f80 msgr2=0x7f6dc8159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dce600640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dc80b2f80 0x7f6dc8159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dce600640 1 -- 172.21.15.113:0/1690634458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc80c48d0 con 0x7f6dc8154a90 2024-03-04T08:02:28.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dce600640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc8154a90 0x7f6dc80c70c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0001f60 tx=0x7f6dc000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:28.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dc5600640 1 -- 172.21.15.113:0/1690634458 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0012400 con 0x7f6dc8154a90 2024-03-04T08:02:28.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dc5600640 1 -- 172.21.15.113:0/1690634458 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc00125a0 con 0x7f6dc8154a90 2024-03-04T08:02:28.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dc5600640 1 -- 172.21.15.113:0/1690634458 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0012990 con 0x7f6dc8154a90 2024-03-04T08:02:28.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcf000640 1 -- 172.21.15.113:0/1690634458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc80c4bb0 con 0x7f6dc8154a90 2024-03-04T08:02:28.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcf000640 1 -- 172.21.15.113:0/1690634458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc80c5170 con 0x7f6dc8154a90 2024-03-04T08:02:28.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.360+0000 7f6dcf000640 1 -- 172.21.15.113:0/1690634458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc8075a80 con 0x7f6dc8154a90 2024-03-04T08:02:28.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.364+0000 7f6dc5600640 1 -- 172.21.15.113:0/1690634458 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6dc0012b30 con 0x7f6dc8154a90 2024-03-04T08:02:28.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.364+0000 7f6dc5600640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6da8066fb0 0x7f6da8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.364+0000 7f6dc5600640 1 -- 172.21.15.113:0/1690634458 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f6dc0085d10 con 0x7f6dc8154a90 2024-03-04T08:02:28.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.364+0000 7f6dcdc00640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6da8066fb0 0x7f6da8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.364+0000 7f6dcdc00640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6da8066fb0 0x7f6da8069470 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0b2000 tx=0x7f6dbc0aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:28.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.368+0000 7f6dc5600640 1 -- 172.21.15.113:0/1690634458 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6dc008b020 con 0x7f6dc8154a90 2024-03-04T08:02:28.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.672+0000 7f6dcf000640 1 -- 172.21.15.113:0/1690634458 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dc80ccc10 con 0x7f6dc8154a90 2024-03-04T08:02:28.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.672+0000 7f6dc5600640 1 -- 172.21.15.113:0/1690634458 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6622 (secure 0 0 0) 0x7f6dc0012e20 con 0x7f6dc8154a90 2024-03-04T08:02:28.669 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:28.670 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:54.112369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6807,6534,6372]},"id":1}]} 2024-03-04T08:02:28.670 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 106 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 -- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6da8066fb0 msgr2=0x7f6da8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6da8066fb0 0x7f6da8069470 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc0b2000 tx=0x7f6dbc0aa070 comp rx=0 tx=0).stop 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 -- 172.21.15.113:0/1690634458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc8154a90 msgr2=0x7f6dc80c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc8154a90 0x7f6dc80c70c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0001f60 tx=0x7f6dc000a430 comp rx=0 tx=0).stop 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 -- 172.21.15.113:0/1690634458 shutdown_connections 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6da8066fb0 0x7f6da8069470 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dc8154a90 0x7f6dc80c70c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6dc80c8940 0x7f6dc80c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 --2- 172.21.15.113:0/1690634458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dc80b2f80 0x7f6dc8159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 -- 172.21.15.113:0/1690634458 >> 172.21.15.113:0/1690634458 conn(0x7f6dc80cd080 msgr2=0x7f6dc80cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 -- 172.21.15.113:0/1690634458 shutdown_connections 2024-03-04T08:02:28.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.676+0000 7f6dcf000640 1 -- 172.21.15.113:0/1690634458 wait complete. 2024-03-04T08:02:28.691 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=0) 2024-03-04T08:02:28.691 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-04T08:02:28.691 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:02:28.728 INFO:tasks.ceph.mds.c:Stopped 2024-03-04T08:02:28.729 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:28.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.916+0000 7fa325200640 1 Processor -- start 2024-03-04T08:02:28.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.916+0000 7fa325200640 1 -- start start 2024-03-04T08:02:28.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.916+0000 7fa325200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa3200c8200 0x7fa320070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.916+0000 7fa325200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3201591f0 con 0x7fa320152bd0 2024-03-04T08:02:28.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.916+0000 7fa325200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3201594d0 con 0x7fa3201527f0 2024-03-04T08:02:28.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.916+0000 7fa325200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa320071df0 con 0x7fa3200c8200 2024-03-04T08:02:28.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.920+0000 7fa31e800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa320152bd0 0x7fa3201587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42252/0 (socket says 172.21.15.113:42252) 2024-03-04T08:02:28.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.920+0000 7fa31e800640 1 -- 172.21.15.113:0/155214789 learned_addr learned my addr 172.21.15.113:0/155214789 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:02:28.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.920+0000 7fa31d400640 1 --2- 172.21.15.113:0/155214789 >> v2:172.21.15.136:3300/0 conn(0x7fa3200c8200 0x7fa320070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.920+0000 7fa31ca00640 1 -- 172.21.15.113:0/155214789 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 468914451 0 0) 0x7fa3201591f0 con 0x7fa320152bd0 2024-03-04T08:02:28.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.920+0000 7fa31ca00640 1 -- 172.21.15.113:0/155214789 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa304003530 con 0x7fa320152bd0 2024-03-04T08:02:28.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.920+0000 7fa31d400640 1 -- 172.21.15.113:0/155214789 >> v1:172.21.15.136:6789/0 conn(0x7fa3201527f0 legacy=0x7fa320072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.920+0000 7fa31d400640 1 -- 172.21.15.113:0/155214789 >> v1:172.21.15.113:6789/0 conn(0x7fa320152bd0 legacy=0x7fa3201587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.920+0000 7fa31ca00640 1 -- 172.21.15.113:0/155214789 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 903200822 0 0) 0x7fa304003530 con 0x7fa320152bd0 2024-03-04T08:02:28.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.920+0000 7fa31d400640 1 -- 172.21.15.113:0/155214789 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3200717a0 con 0x7fa3200c8200 2024-03-04T08:02:28.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa31d400640 1 --2- 172.21.15.113:0/155214789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3200c8200 0x7fa320070d40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0097c0 tx=0x7fa30c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d72f26f85495da8c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:28.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa31ca00640 1 -- 172.21.15.113:0/155214789 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0b0070 con 0x7fa3200c8200 2024-03-04T08:02:28.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa31ca00640 1 -- 172.21.15.113:0/155214789 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c0ab400 con 0x7fa3200c8200 2024-03-04T08:02:28.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa31ca00640 1 -- 172.21.15.113:0/155214789 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318001a00 con 0x7fa3200c8200 2024-03-04T08:02:28.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa325200640 1 -- 172.21.15.113:0/155214789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3200c8200 msgr2=0x7fa320070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa325200640 1 --2- 172.21.15.113:0/155214789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3200c8200 0x7fa320070d40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0097c0 tx=0x7fa30c01c3c0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa325200640 1 -- 172.21.15.113:0/155214789 shutdown_connections 2024-03-04T08:02:28.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa325200640 1 --2- 172.21.15.113:0/155214789 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3200c8200 0x7fa320070d40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa325200640 1 -- 172.21.15.113:0/155214789 >> 172.21.15.113:0/155214789 conn(0x7fa32006e820 msgr2=0x7fa32006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:28.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa325200640 1 -- 172.21.15.113:0/155214789 shutdown_connections 2024-03-04T08:02:28.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.924+0000 7fa325200640 1 -- 172.21.15.113:0/155214789 wait complete. 2024-03-04T08:02:28.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa325200640 1 Processor -- start 2024-03-04T08:02:28.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa325200640 1 -- start start 2024-03-04T08:02:28.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa325200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3200c8200 0x7fa320140d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa325200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3201527f0 0x7fa320158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa325200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320152bd0 0x7fa320158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa325200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa320166500 con 0x7fa3200c8200 2024-03-04T08:02:28.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa325200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa320166380 con 0x7fa320152bd0 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa325200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa320166680 con 0x7fa3201527f0 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3200c8200 0x7fa320140d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3200c8200 0x7fa320140d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60408/0 (socket says 172.21.15.113:60408) 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31de00640 1 -- 172.21.15.113:0/3927554136 learned_addr learned my addr 172.21.15.113:0/3927554136 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31e800640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320152bd0 0x7fa320158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31d400640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3201527f0 0x7fa320158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31de00640 1 -- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3201527f0 msgr2=0x7fa320158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31de00640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3201527f0 0x7fa320158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31de00640 1 -- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320152bd0 msgr2=0x7fa320158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31de00640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320152bd0 0x7fa320158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:28.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31de00640 1 -- 172.21.15.113:0/3927554136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa320154090 con 0x7fa3200c8200 2024-03-04T08:02:28.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa31de00640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3200c8200 0x7fa320140d10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c530 tx=0x7fa31000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:28.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa316000640 1 -- 172.21.15.113:0/3927554136 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310016020 con 0x7fa3200c8200 2024-03-04T08:02:28.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa316000640 1 -- 172.21.15.113:0/3927554136 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa310005220 con 0x7fa3200c8200 2024-03-04T08:02:28.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa325200640 1 -- 172.21.15.113:0/3927554136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa320154370 con 0x7fa3200c8200 2024-03-04T08:02:28.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa316000640 1 -- 172.21.15.113:0/3927554136 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310007030 con 0x7fa3200c8200 2024-03-04T08:02:28.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.928+0000 7fa325200640 1 -- 172.21.15.113:0/3927554136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3201496b0 con 0x7fa3200c8200 2024-03-04T08:02:28.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.932+0000 7fa325200640 1 -- 172.21.15.113:0/3927554136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa320075c00 con 0x7fa3200c8200 2024-03-04T08:02:28.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.932+0000 7fa316000640 1 -- 172.21.15.113:0/3927554136 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3100071d0 con 0x7fa3200c8200 2024-03-04T08:02:28.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.932+0000 7fa316000640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa300066f60 0x7fa300069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:28.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.932+0000 7fa316000640 1 -- 172.21.15.113:0/3927554136 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7fa310084e90 con 0x7fa3200c8200 2024-03-04T08:02:28.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.936+0000 7fa31d400640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa300066f60 0x7fa300069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:28.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.936+0000 7fa31d400640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa300066f60 0x7fa300069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0ad000 tx=0x7fa30c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:28.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:28.936+0000 7fa316000640 1 -- 172.21.15.113:0/3927554136 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa310020020 con 0x7fa3200c8200 2024-03-04T08:02:29.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.312+0000 7fa325200640 1 -- 172.21.15.113:0/3927554136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3200c79d0 con 0x7fa3200c8200 2024-03-04T08:02:29.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.312+0000 7fa316000640 1 -- 172.21.15.113:0/3927554136 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6622 (secure 0 0 0) 0x7fa310086090 con 0x7fa3200c8200 2024-03-04T08:02:29.309 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:29.309 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:54.112369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6807,6534,6372]},"id":1}]} 2024-03-04T08:02:29.309 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 106 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 -- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa300066f60 msgr2=0x7fa300069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa300066f60 0x7fa300069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0ad000 tx=0x7fa30c0a6000 comp rx=0 tx=0).stop 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 -- 172.21.15.113:0/3927554136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3200c8200 msgr2=0x7fa320140d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3200c8200 0x7fa320140d10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c530 tx=0x7fa31000c9f0 comp rx=0 tx=0).stop 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 -- 172.21.15.113:0/3927554136 shutdown_connections 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa320152bd0 0x7fa320158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3201527f0 0x7fa320158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa300066f60 0x7fa300069420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 --2- 172.21.15.113:0/3927554136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3200c8200 0x7fa320140d10 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 -- 172.21.15.113:0/3927554136 >> 172.21.15.113:0/3927554136 conn(0x7fa32006e820 msgr2=0x7fa3200c8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:29.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 -- 172.21.15.113:0/3927554136 shutdown_connections 2024-03-04T08:02:29.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:29.316+0000 7fa325200640 1 -- 172.21.15.113:0/3927554136 wait complete. 2024-03-04T08:02:29.339 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:02:31.340 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:31.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21d1c00640 1 Processor -- start 2024-03-04T08:02:31.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21d1c00640 1 -- start start 2024-03-04T08:02:31.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21d1c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f21cc070010 0x7f21cc0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:31.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21d1c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21cc15c510 0x7f21cc163190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:31.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21d1c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc152f00 con 0x7f21cc152b20 2024-03-04T08:02:31.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21d1c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21cc0c5320 con 0x7f21cc15c510 2024-03-04T08:02:31.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21d1c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21cc0c5040 con 0x7f21cc070010 2024-03-04T08:02:31.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21cb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21cc15c510 0x7f21cc163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:31.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21caa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f21cc152b20 0x7f21cc15be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42264/0 (socket says 172.21.15.113:42264) 2024-03-04T08:02:31.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21caa00640 1 -- 172.21.15.113:0/1084791407 learned_addr learned my addr 172.21.15.113:0/1084791407 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:02:31.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21c3e00640 1 --2- 172.21.15.113:0/1084791407 >> v2:172.21.15.136:3300/0 conn(0x7f21cc070010 0x7f21cc0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:31.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21cb400640 1 -- 172.21.15.113:0/1084791407 >> v2:172.21.15.136:3300/0 conn(0x7f21cc070010 msgr2=0x7f21cc0c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:31.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21cb400640 1 --2- 172.21.15.113:0/1084791407 >> v2:172.21.15.136:3300/0 conn(0x7f21cc070010 0x7f21cc0c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:31.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.484+0000 7f21cb400640 1 -- 172.21.15.113:0/1084791407 >> v1:172.21.15.113:6789/0 conn(0x7f21cc152b20 legacy=0x7f21cc15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:31.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21cb400640 1 -- 172.21.15.113:0/1084791407 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc0c3250 con 0x7f21cc15c510 2024-03-04T08:02:31.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21cb400640 1 --2- 172.21.15.113:0/1084791407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc15c510 0x7f21cc163190 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f21cc050420 tx=0x7f21c401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17026e1ce53f897 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:31.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21ca000640 1 -- 172.21.15.113:0/1084791407 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40b2070 con 0x7f21cc15c510 2024-03-04T08:02:31.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21ca000640 1 -- 172.21.15.113:0/1084791407 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c40a57f0 con 0x7f21cc15c510 2024-03-04T08:02:31.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21ca000640 1 -- 172.21.15.113:0/1084791407 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40a5bc0 con 0x7f21cc15c510 2024-03-04T08:02:31.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21d1c00640 1 -- 172.21.15.113:0/1084791407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc15c510 msgr2=0x7f21cc163190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:31.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21d1c00640 1 --2- 172.21.15.113:0/1084791407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc15c510 0x7f21cc163190 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f21cc050420 tx=0x7f21c401c930 comp rx=0 tx=0).stop 2024-03-04T08:02:31.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21d1c00640 1 -- 172.21.15.113:0/1084791407 shutdown_connections 2024-03-04T08:02:31.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21d1c00640 1 --2- 172.21.15.113:0/1084791407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc15c510 0x7f21cc163190 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:31.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21d1c00640 1 --2- 172.21.15.113:0/1084791407 >> v2:172.21.15.136:3300/0 conn(0x7f21cc070010 0x7f21cc0c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:31.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21d1c00640 1 -- 172.21.15.113:0/1084791407 >> 172.21.15.113:0/1084791407 conn(0x7f21cc0709a0 msgr2=0x7f21cc070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:31.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21d1c00640 1 -- 172.21.15.113:0/1084791407 shutdown_connections 2024-03-04T08:02:31.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.488+0000 7f21d1c00640 1 -- 172.21.15.113:0/1084791407 wait complete. 2024-03-04T08:02:31.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21d1c00640 1 Processor -- start 2024-03-04T08:02:31.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21d1c00640 1 -- start start 2024-03-04T08:02:31.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21d1c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21cc070010 0x7f21cc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:31.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21d1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc152b20 0x7f21cc161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:31.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21d1c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21cc15c510 0x7f21cc15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:31.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21d1c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc0c8490 con 0x7f21cc152b20 2024-03-04T08:02:31.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21d1c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc158810 con 0x7f21cc070010 2024-03-04T08:02:31.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21d1c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc06d3b0 con 0x7f21cc15c510 2024-03-04T08:02:31.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc152b20 0x7f21cc161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:31.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc152b20 0x7f21cc161320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60416/0 (socket says 172.21.15.113:60416) 2024-03-04T08:02:31.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c3e00640 1 -- 172.21.15.113:0/716878276 learned_addr learned my addr 172.21.15.113:0/716878276 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:31.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21caa00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21cc070010 0x7f21cc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:31.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21cb400640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21cc15c510 0x7f21cc15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:31.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c3e00640 1 -- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21cc15c510 msgr2=0x7f21cc15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:31.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c3e00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21cc15c510 0x7f21cc15b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:31.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c3e00640 1 -- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21cc070010 msgr2=0x7f21cc06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:31.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c3e00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21cc070010 0x7f21cc06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:31.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c3e00640 1 -- 172.21.15.113:0/716878276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc1f78a0 con 0x7f21cc152b20 2024-03-04T08:02:31.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c3e00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc152b20 0x7f21cc161320 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f21b400d620 tx=0x7f21b400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:31.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c2a00640 1 -- 172.21.15.113:0/716878276 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400b620 con 0x7f21cc152b20 2024-03-04T08:02:31.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c2a00640 1 -- 172.21.15.113:0/716878276 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b4019070 con 0x7f21cc152b20 2024-03-04T08:02:31.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21d1c00640 1 -- 172.21.15.113:0/716878276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc06dd70 con 0x7f21cc152b20 2024-03-04T08:02:31.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.492+0000 7f21c2a00640 1 -- 172.21.15.113:0/716878276 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400b9e0 con 0x7f21cc152b20 2024-03-04T08:02:31.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.496+0000 7f21d1c00640 1 -- 172.21.15.113:0/716878276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc06cbb0 con 0x7f21cc152b20 2024-03-04T08:02:31.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.496+0000 7f21d1c00640 1 -- 172.21.15.113:0/716878276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21cc075c00 con 0x7f21cc152b20 2024-03-04T08:02:31.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.496+0000 7f21c2a00640 1 -- 172.21.15.113:0/716878276 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21b400bb80 con 0x7f21cc152b20 2024-03-04T08:02:31.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.496+0000 7f21c2a00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21a8066f60 0x7f21a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:31.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.496+0000 7f21c2a00640 1 -- 172.21.15.113:0/716878276 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f21b4085910 con 0x7f21cc152b20 2024-03-04T08:02:31.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.496+0000 7f21caa00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21a8066f60 0x7f21a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:31.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.500+0000 7f21caa00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21a8066f60 0x7f21a8069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f21bc000f70 tx=0x7f21bc007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:31.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.500+0000 7f21c2a00640 1 -- 172.21.15.113:0/716878276 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f21b4055cb0 con 0x7f21cc152b20 2024-03-04T08:02:31.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 -- 172.21.15.113:0/716878276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21cc06fdf0 con 0x7f21cc152b20 2024-03-04T08:02:31.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21c2a00640 1 -- 172.21.15.113:0/716878276 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6622 (secure 0 0 0) 0x7f21b4059d50 con 0x7f21cc152b20 2024-03-04T08:02:31.716 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:31.716 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:54.112369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6807,6534,6372]},"id":1}]} 2024-03-04T08:02:31.716 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 106 2024-03-04T08:02:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 -- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21a8066f60 msgr2=0x7f21a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21a8066f60 0x7f21a8069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f21bc000f70 tx=0x7f21bc007e90 comp rx=0 tx=0).stop 2024-03-04T08:02:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 -- 172.21.15.113:0/716878276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc152b20 msgr2=0x7f21cc161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc152b20 0x7f21cc161320 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f21b400d620 tx=0x7f21b400dae0 comp rx=0 tx=0).stop 2024-03-04T08:02:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 -- 172.21.15.113:0/716878276 shutdown_connections 2024-03-04T08:02:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21cc15c510 0x7f21cc15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21a8066f60 0x7f21a8069420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:31.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21cc152b20 0x7f21cc161320 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:31.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 --2- 172.21.15.113:0/716878276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21cc070010 0x7f21cc06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:31.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.720+0000 7f21d1c00640 1 -- 172.21.15.113:0/716878276 >> 172.21.15.113:0/716878276 conn(0x7f21cc0709a0 msgr2=0x7f21cc06f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:31.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.724+0000 7f21d1c00640 1 -- 172.21.15.113:0/716878276 shutdown_connections 2024-03-04T08:02:31.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:31.724+0000 7f21d1c00640 1 -- 172.21.15.113:0/716878276 wait complete. 2024-03-04T08:02:31.737 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:02:33.737 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:33.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f87a9800640 1 Processor -- start 2024-03-04T08:02:33.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f87a9800640 1 -- start start 2024-03-04T08:02:33.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f87a9800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87a40b20b0 0x7f87a4157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:33.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f87a9800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87a4157c50 0x7f87a406b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:33.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f87a9800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a40c1ec0 con 0x7f87a40c85f0 2024-03-04T08:02:33.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f87a9800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a40c9b30 con 0x7f87a4157c50 2024-03-04T08:02:33.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f87a9800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40c12a0 con 0x7f87a40b20b0 2024-03-04T08:02:33.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f879be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87a4157c50 0x7f87a406b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:33.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f879be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87a4157c50 0x7f87a406b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60424/0 (socket says 172.21.15.113:60424) 2024-03-04T08:02:33.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f879be00640 1 -- 172.21.15.113:0/3578318623 learned_addr learned my addr 172.21.15.113:0/3578318623 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:33.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.884+0000 7f87a2a00640 1 --2- 172.21.15.113:0/3578318623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87a40b20b0 0x7f87a4157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:33.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.888+0000 7f879be00640 1 -- 172.21.15.113:0/3578318623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87a40b20b0 msgr2=0x7f87a4157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:33.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.888+0000 7f879be00640 1 --2- 172.21.15.113:0/3578318623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87a40b20b0 0x7f87a4157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:33.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.888+0000 7f879be00640 1 -- 172.21.15.113:0/3578318623 >> v1:172.21.15.136:6789/0 conn(0x7f87a40c85f0 legacy=0x7f87a415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:33.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.888+0000 7f879be00640 1 -- 172.21.15.113:0/3578318623 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a40c1580 con 0x7f87a4157c50 2024-03-04T08:02:33.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f879be00640 1 --2- 172.21.15.113:0/3578318623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a4157c50 0x7f87a406b060 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f878c009870 tx=0x7f878c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145ee180f0d288d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:33.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a2000640 1 -- 172.21.15.113:0/3578318623 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0b1070 con 0x7f87a4157c50 2024-03-04T08:02:33.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a2000640 1 -- 172.21.15.113:0/3578318623 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c0ac620 con 0x7f87a4157c50 2024-03-04T08:02:33.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a2000640 1 -- 172.21.15.113:0/3578318623 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0ac9f0 con 0x7f87a4157c50 2024-03-04T08:02:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a9800640 1 -- 172.21.15.113:0/3578318623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a4157c50 msgr2=0x7f87a406b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a9800640 1 --2- 172.21.15.113:0/3578318623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a4157c50 0x7f87a406b060 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f878c009870 tx=0x7f878c01c530 comp rx=0 tx=0).stop 2024-03-04T08:02:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a9800640 1 -- 172.21.15.113:0/3578318623 shutdown_connections 2024-03-04T08:02:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a9800640 1 --2- 172.21.15.113:0/3578318623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a4157c50 0x7f87a406b060 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a9800640 1 --2- 172.21.15.113:0/3578318623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87a40b20b0 0x7f87a4157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a9800640 1 -- 172.21.15.113:0/3578318623 >> 172.21.15.113:0/3578318623 conn(0x7f87a406e820 msgr2=0x7f87a406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a9800640 1 -- 172.21.15.113:0/3578318623 shutdown_connections 2024-03-04T08:02:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a9800640 1 -- 172.21.15.113:0/3578318623 wait complete. 2024-03-04T08:02:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a9800640 1 Processor -- start 2024-03-04T08:02:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.892+0000 7f87a9800640 1 -- start start 2024-03-04T08:02:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a9800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87a40b20b0 0x7f87a40c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a40c85f0 0x7f87a414c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a9800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87a4157c50 0x7f87a41f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a9800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40c1b80 con 0x7f87a40c85f0 2024-03-04T08:02:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a9800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a4156ec0 con 0x7f87a40b20b0 2024-03-04T08:02:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a9800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40c80c0 con 0x7f87a4157c50 2024-03-04T08:02:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a40c85f0 0x7f87a414c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a40c85f0 0x7f87a414c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60434/0 (socket says 172.21.15.113:60434) 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879be00640 1 -- 172.21.15.113:0/3681480187 learned_addr learned my addr 172.21.15.113:0/3681480187 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a2a00640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87a40b20b0 0x7f87a40c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a3400640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87a4157c50 0x7f87a41f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879be00640 1 -- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87a4157c50 msgr2=0x7f87a41f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879be00640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87a4157c50 0x7f87a41f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879be00640 1 -- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87a40b20b0 msgr2=0x7f87a40c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879be00640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87a40b20b0 0x7f87a40c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879be00640 1 -- 172.21.15.113:0/3681480187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a40c3ec0 con 0x7f87a40c85f0 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879be00640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a40c85f0 0x7f87a414c2e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f878c01abe0 tx=0x7f878c0192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879aa00640 1 -- 172.21.15.113:0/3681480187 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0b1040 con 0x7f87a40c85f0 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879aa00640 1 -- 172.21.15.113:0/3681480187 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c0ac420 con 0x7f87a40c85f0 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f879aa00640 1 -- 172.21.15.113:0/3681480187 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0b5630 con 0x7f87a40c85f0 2024-03-04T08:02:33.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a9800640 1 -- 172.21.15.113:0/3681480187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a40c41a0 con 0x7f87a40c85f0 2024-03-04T08:02:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a9800640 1 -- 172.21.15.113:0/3681480187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a4145790 con 0x7f87a40c85f0 2024-03-04T08:02:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.896+0000 7f87a9800640 1 -- 172.21.15.113:0/3681480187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a4075c00 con 0x7f87a40c85f0 2024-03-04T08:02:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.900+0000 7f879aa00640 1 -- 172.21.15.113:0/3681480187 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f878c0bd020 con 0x7f87a40c85f0 2024-03-04T08:02:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.900+0000 7f879aa00640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8780066f60 0x7f8780069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.900+0000 7f879aa00640 1 -- 172.21.15.113:0/3681480187 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f878c11eac0 con 0x7f87a40c85f0 2024-03-04T08:02:33.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.900+0000 7f87a2a00640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8780066f60 0x7f8780069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:33.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.900+0000 7f87a2a00640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8780066f60 0x7f8780069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f87940079c0 tx=0x7f8794008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:33.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:33.904+0000 7f879aa00640 1 -- 172.21.15.113:0/3681480187 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f878c0c4020 con 0x7f87a40c85f0 2024-03-04T08:02:34.008 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:02:34.013 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:02:34.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.100+0000 7f87a9800640 1 -- 172.21.15.113:0/3681480187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87a40c83d0 con 0x7f87a40c85f0 2024-03-04T08:02:34.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.100+0000 7f879aa00640 1 -- 172.21.15.113:0/3681480187 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6622 (secure 0 0 0) 0x7f878c016150 con 0x7f87a40c85f0 2024-03-04T08:02:34.099 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:34.099 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:54.112369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6807,6534,6372]},"id":1}]} 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 106 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 -- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8780066f60 msgr2=0x7f8780069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8780066f60 0x7f8780069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f87940079c0 tx=0x7f8794008040 comp rx=0 tx=0).stop 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 -- 172.21.15.113:0/3681480187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a40c85f0 msgr2=0x7f87a414c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a40c85f0 0x7f87a414c2e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f878c01abe0 tx=0x7f878c0192d0 comp rx=0 tx=0).stop 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 -- 172.21.15.113:0/3681480187 shutdown_connections 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87a4157c50 0x7f87a41f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8780066f60 0x7f8780069420 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87a40c85f0 0x7f87a414c2e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 --2- 172.21.15.113:0/3681480187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87a40b20b0 0x7f87a40c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 -- 172.21.15.113:0/3681480187 >> 172.21.15.113:0/3681480187 conn(0x7f87a406e820 msgr2=0x7f87a4149730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:34.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 -- 172.21.15.113:0/3681480187 shutdown_connections 2024-03-04T08:02:34.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:34.104+0000 7f87a9800640 1 -- 172.21.15.113:0/3681480187 wait complete. 2024-03-04T08:02:34.118 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:02:36.119 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:36.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44c5800640 1 Processor -- start 2024-03-04T08:02:36.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44c5800640 1 -- start start 2024-03-04T08:02:36.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44c5800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44c0158390 0x7f44c0070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:36.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44c5800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c015b150 0x7f44c015b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:36.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44c5800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c00c8da0 con 0x7f44c006de10 2024-03-04T08:02:36.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44c5800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c0158770 con 0x7f44c0158390 2024-03-04T08:02:36.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44c5800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c006f770 con 0x7f44c015b150 2024-03-04T08:02:36.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44bea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44c0158390 0x7f44c0070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:36.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44be000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f44c006de10 0x7f44c00ce270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42278/0 (socket says 172.21.15.113:42278) 2024-03-04T08:02:36.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44be000640 1 -- 172.21.15.113:0/4278582145 learned_addr learned my addr 172.21.15.113:0/4278582145 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:02:36.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44bea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44c0158390 0x7f44c0070130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60442/0 (socket says 172.21.15.113:60442) 2024-03-04T08:02:36.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44bf400640 1 --2- 172.21.15.113:0/4278582145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c015b150 0x7f44c015b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44bea00640 1 -- 172.21.15.113:0/4278582145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c015b150 msgr2=0x7f44c015b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44bea00640 1 --2- 172.21.15.113:0/4278582145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c015b150 0x7f44c015b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44bea00640 1 -- 172.21.15.113:0/4278582145 >> v1:172.21.15.113:6789/0 conn(0x7f44c006de10 legacy=0x7f44c00ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44bea00640 1 -- 172.21.15.113:0/4278582145 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c006fa50 con 0x7f44c0158390 2024-03-04T08:02:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.264+0000 7f44bd600640 1 -- 172.21.15.113:0/4278582145 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595688913 0 0) 0x7f44ac001120 con 0x7f44c006de10 2024-03-04T08:02:36.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44bea00640 1 --2- 172.21.15.113:0/4278582145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c0158390 0x7f44c0070130 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f44b00097c0 tx=0x7f44b001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509b33b1b8521f16 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44bd600640 1 -- 172.21.15.113:0/4278582145 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b00b0070 con 0x7f44c0158390 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44bd600640 1 -- 172.21.15.113:0/4278582145 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b00ab600 con 0x7f44c0158390 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44bd600640 1 -- 172.21.15.113:0/4278582145 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b00ab9d0 con 0x7f44c0158390 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44c5800640 1 -- 172.21.15.113:0/4278582145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c0158390 msgr2=0x7f44c0070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44c5800640 1 --2- 172.21.15.113:0/4278582145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c0158390 0x7f44c0070130 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f44b00097c0 tx=0x7f44b001c410 comp rx=0 tx=0).stop 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44c5800640 1 -- 172.21.15.113:0/4278582145 shutdown_connections 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44c5800640 1 --2- 172.21.15.113:0/4278582145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c015b150 0x7f44c015b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44c5800640 1 --2- 172.21.15.113:0/4278582145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c0158390 0x7f44c0070130 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44c5800640 1 -- 172.21.15.113:0/4278582145 >> 172.21.15.113:0/4278582145 conn(0x7f44c0066db0 msgr2=0x7f44c00671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44c5800640 1 -- 172.21.15.113:0/4278582145 shutdown_connections 2024-03-04T08:02:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.268+0000 7f44c5800640 1 -- 172.21.15.113:0/4278582145 wait complete. 2024-03-04T08:02:36.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44c5800640 1 Processor -- start 2024-03-04T08:02:36.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44c5800640 1 -- start start 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44c5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c006de10 0x7f44c014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44c5800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c0158390 0x7f44c014ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44c5800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44c015b150 0x7f44c0148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44c5800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00b1070 con 0x7f44c006de10 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44c5800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00b0ef0 con 0x7f44c015b150 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44c5800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c00ce980 con 0x7f44c0158390 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44bea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c006de10 0x7f44c014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44bea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c006de10 0x7f44c014cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60456/0 (socket says 172.21.15.113:60456) 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44bea00640 1 -- 172.21.15.113:0/366243325 learned_addr learned my addr 172.21.15.113:0/366243325 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44be000640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c0158390 0x7f44c014ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44bf400640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44c015b150 0x7f44c0148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:36.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44bea00640 1 -- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c0158390 msgr2=0x7f44c014ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:36.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44bea00640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c0158390 0x7f44c014ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:36.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44bea00640 1 -- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44c015b150 msgr2=0x7f44c0148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:36.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44bea00640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44c015b150 0x7f44c0148ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:36.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44bea00640 1 -- 172.21.15.113:0/366243325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c01f7170 con 0x7f44c006de10 2024-03-04T08:02:36.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.272+0000 7f44be000640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c0158390 0x7f44c014ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:02:36.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44bea00640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c006de10 0x7f44c014cf50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f44b00097c0 tx=0x7f44b00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:36.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44b6a00640 1 -- 172.21.15.113:0/366243325 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b00b0040 con 0x7f44c006de10 2024-03-04T08:02:36.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44b6a00640 1 -- 172.21.15.113:0/366243325 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b00b43f0 con 0x7f44c006de10 2024-03-04T08:02:36.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44c5800640 1 -- 172.21.15.113:0/366243325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c0068350 con 0x7f44c006de10 2024-03-04T08:02:36.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44b6a00640 1 -- 172.21.15.113:0/366243325 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b00b47c0 con 0x7f44c006de10 2024-03-04T08:02:36.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44c5800640 1 -- 172.21.15.113:0/366243325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c01f7410 con 0x7f44c006de10 2024-03-04T08:02:36.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44c5800640 1 -- 172.21.15.113:0/366243325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c0075c00 con 0x7f44c006de10 2024-03-04T08:02:36.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44b6a00640 1 -- 172.21.15.113:0/366243325 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44b00b4960 con 0x7f44c006de10 2024-03-04T08:02:36.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44b6a00640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f44a0066f90 0x7f44a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:36.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44b6a00640 1 -- 172.21.15.113:0/366243325 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f44b011f0c0 con 0x7f44c006de10 2024-03-04T08:02:36.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.276+0000 7f44be000640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f44a0066f90 0x7f44a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:36.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.280+0000 7f44be000640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f44a0066f90 0x7f44a0069450 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f44ac007fd0 tx=0x7f44ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:36.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.284+0000 7f44b6a00640 1 -- 172.21.15.113:0/366243325 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f44b0016090 con 0x7f44c006de10 2024-03-04T08:02:36.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 -- 172.21.15.113:0/366243325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44c006ff10 con 0x7f44c006de10 2024-03-04T08:02:36.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44b6a00640 1 -- 172.21.15.113:0/366243325 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6622 (secure 0 0 0) 0x7f44b00efb40 con 0x7f44c006de10 2024-03-04T08:02:36.503 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:36.503 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:54.112369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6807,6534,6372]},"id":1}]} 2024-03-04T08:02:36.503 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 106 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 -- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f44a0066f90 msgr2=0x7f44a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f44a0066f90 0x7f44a0069450 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f44ac007fd0 tx=0x7f44ac009040 comp rx=0 tx=0).stop 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 -- 172.21.15.113:0/366243325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c006de10 msgr2=0x7f44c014cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c006de10 0x7f44c014cf50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f44b00097c0 tx=0x7f44b00191b0 comp rx=0 tx=0).stop 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 -- 172.21.15.113:0/366243325 shutdown_connections 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44c015b150 0x7f44c0148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44c0158390 0x7f44c014ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f44a0066f90 0x7f44a0069450 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 --2- 172.21.15.113:0/366243325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44c006de10 0x7f44c014cf50 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 -- 172.21.15.113:0/366243325 >> 172.21.15.113:0/366243325 conn(0x7f44c0066db0 msgr2=0x7f44c006c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 -- 172.21.15.113:0/366243325 shutdown_connections 2024-03-04T08:02:36.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:36.508+0000 7f44c5800640 1 -- 172.21.15.113:0/366243325 wait complete. 2024-03-04T08:02:36.526 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:02:38.527 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:38.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f8600640 1 Processor -- start 2024-03-04T08:02:38.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f8600640 1 -- start start 2024-03-04T08:02:38.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f8600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59f01527f0 0x7f59f0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:38.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f8600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f00c8200 0x7f59f0072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:38.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f8600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f01591f0 con 0x7f59f0152bd0 2024-03-04T08:02:38.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f8600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f0071df0 con 0x7f59f01527f0 2024-03-04T08:02:38.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f8600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f01597b0 con 0x7f59f00c8200 2024-03-04T08:02:38.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f5c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59f01527f0 0x7f59f0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:38.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f5c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59f01527f0 0x7f59f0070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51048/0 (socket says 172.21.15.113:51048) 2024-03-04T08:02:38.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f5c00640 1 -- 172.21.15.113:0/1969353393 learned_addr learned my addr 172.21.15.113:0/1969353393 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:38.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f5200640 1 --2- 172.21.15.113:0/1969353393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f00c8200 0x7f59f0072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:38.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f5c00640 1 -- 172.21.15.113:0/1969353393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f00c8200 msgr2=0x7f59f0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:38.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f5c00640 1 --2- 172.21.15.113:0/1969353393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f00c8200 0x7f59f0072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:38.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f5c00640 1 -- 172.21.15.113:0/1969353393 >> v1:172.21.15.136:6789/0 conn(0x7f59f0152bd0 legacy=0x7f59f01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:38.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.672+0000 7f59f5c00640 1 -- 172.21.15.113:0/1969353393 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f01594d0 con 0x7f59f01527f0 2024-03-04T08:02:38.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59f5c00640 1 --2- 172.21.15.113:0/1969353393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01527f0 0x7f59f0070d40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009870 tx=0x7f59e401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a79e73c08c3f04 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59efe00640 1 -- 172.21.15.113:0/1969353393 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e40a7070 con 0x7f59f01527f0 2024-03-04T08:02:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59efe00640 1 -- 172.21.15.113:0/1969353393 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e40b0070 con 0x7f59f01527f0 2024-03-04T08:02:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59efe00640 1 -- 172.21.15.113:0/1969353393 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e40ab890 con 0x7f59f01527f0 2024-03-04T08:02:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59f8600640 1 -- 172.21.15.113:0/1969353393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01527f0 msgr2=0x7f59f0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59f8600640 1 --2- 172.21.15.113:0/1969353393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01527f0 0x7f59f0070d40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009870 tx=0x7f59e401c560 comp rx=0 tx=0).stop 2024-03-04T08:02:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59f8600640 1 -- 172.21.15.113:0/1969353393 shutdown_connections 2024-03-04T08:02:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59f8600640 1 --2- 172.21.15.113:0/1969353393 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f00c8200 0x7f59f0072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59f8600640 1 --2- 172.21.15.113:0/1969353393 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f01527f0 0x7f59f0070d40 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:38.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59f8600640 1 -- 172.21.15.113:0/1969353393 >> 172.21.15.113:0/1969353393 conn(0x7f59f006e820 msgr2=0x7f59f006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59f8600640 1 -- 172.21.15.113:0/1969353393 shutdown_connections 2024-03-04T08:02:38.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.676+0000 7f59f8600640 1 -- 172.21.15.113:0/1969353393 wait complete. 2024-03-04T08:02:38.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f8600640 1 Processor -- start 2024-03-04T08:02:38.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f8600640 1 -- start start 2024-03-04T08:02:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f8600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c8200 0x7f59f01558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f8600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f01527f0 0x7f59f0148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f8600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f0152bd0 0x7f59f0148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f8600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f0165160 con 0x7f59f00c8200 2024-03-04T08:02:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f8600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f00727c0 con 0x7f59f0152bd0 2024-03-04T08:02:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f8600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f01652e0 con 0x7f59f01527f0 2024-03-04T08:02:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f5c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c8200 0x7f59f01558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:38.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f5c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c8200 0x7f59f01558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51060/0 (socket says 172.21.15.113:51060) 2024-03-04T08:02:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f5c00640 1 -- 172.21.15.113:0/3649792486 learned_addr learned my addr 172.21.15.113:0/3649792486 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f5200640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f01527f0 0x7f59f0148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.680+0000 7f59f6600640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f0152bd0 0x7f59f0148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59f5c00640 1 -- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f01527f0 msgr2=0x7f59f0148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59f5c00640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f01527f0 0x7f59f0148900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59f5c00640 1 -- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f0152bd0 msgr2=0x7f59f0148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59f5c00640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f0152bd0 0x7f59f0148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:38.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59f5c00640 1 -- 172.21.15.113:0/3649792486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f01552b0 con 0x7f59f00c8200 2024-03-04T08:02:38.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59f5c00640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c8200 0x7f59f01558a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f59e401a190 tx=0x7f59e401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:38.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59ed600640 1 -- 172.21.15.113:0/3649792486 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e40be5f0 con 0x7f59f00c8200 2024-03-04T08:02:38.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59ed600640 1 -- 172.21.15.113:0/3649792486 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e40be7b0 con 0x7f59f00c8200 2024-03-04T08:02:38.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59f8600640 1 -- 172.21.15.113:0/3649792486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f01f4db0 con 0x7f59f00c8200 2024-03-04T08:02:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59ed600640 1 -- 172.21.15.113:0/3649792486 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e40beba0 con 0x7f59f00c8200 2024-03-04T08:02:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59f8600640 1 -- 172.21.15.113:0/3649792486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f0152010 con 0x7f59f00c8200 2024-03-04T08:02:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59f8600640 1 -- 172.21.15.113:0/3649792486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f0075c00 con 0x7f59f00c8200 2024-03-04T08:02:38.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.684+0000 7f59ed600640 1 -- 172.21.15.113:0/3649792486 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59e40b43f0 con 0x7f59f00c8200 2024-03-04T08:02:38.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.688+0000 7f59ed600640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59d0066f60 0x7f59d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:38.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.688+0000 7f59ed600640 1 -- 172.21.15.113:0/3649792486 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f59e411ff60 con 0x7f59f00c8200 2024-03-04T08:02:38.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.688+0000 7f59f5200640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59d0066f60 0x7f59d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:38.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.688+0000 7f59f5200640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59d0066f60 0x7f59d0069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f59f00ae710 tx=0x7f59dc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:38.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.692+0000 7f59ed600640 1 -- 172.21.15.113:0/3649792486 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f59e40c4060 con 0x7f59f00c8200 2024-03-04T08:02:38.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.940+0000 7f59f8600640 1 -- 172.21.15.113:0/3649792486 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59f00c8d20 con 0x7f59f00c8200 2024-03-04T08:02:38.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.940+0000 7f59ed600640 1 -- 172.21.15.113:0/3649792486 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6622 (secure 0 0 0) 0x7f59e4016150 con 0x7f59f00c8200 2024-03-04T08:02:38.938 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:38.938 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:54.112369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6807,6534,6372]},"id":1}]} 2024-03-04T08:02:38.938 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 106 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 -- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59d0066f60 msgr2=0x7f59d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59d0066f60 0x7f59d0069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f59f00ae710 tx=0x7f59dc009190 comp rx=0 tx=0).stop 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 -- 172.21.15.113:0/3649792486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c8200 msgr2=0x7f59f01558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c8200 0x7f59f01558a0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f59e401a190 tx=0x7f59e401ada0 comp rx=0 tx=0).stop 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 -- 172.21.15.113:0/3649792486 shutdown_connections 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59f0152bd0 0x7f59f0148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59f01527f0 0x7f59f0148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59d0066f60 0x7f59d0069420 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 --2- 172.21.15.113:0/3649792486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59f00c8200 0x7f59f01558a0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 -- 172.21.15.113:0/3649792486 >> 172.21.15.113:0/3649792486 conn(0x7f59f006e820 msgr2=0x7f59f00c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 -- 172.21.15.113:0/3649792486 shutdown_connections 2024-03-04T08:02:38.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:38.944+0000 7f59f8600640 1 -- 172.21.15.113:0/3649792486 wait complete. 2024-03-04T08:02:38.961 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:02:40.961 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:41.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2567200640 1 Processor -- start 2024-03-04T08:02:41.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2567200640 1 -- start start 2024-03-04T08:02:41.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2567200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2568152950 0x7f256815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:41.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2567200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f256806faa0 0x7f25680c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:41.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2567200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25680c4ce0 con 0x7f256806ed80 2024-03-04T08:02:41.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2567200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25680c8370 con 0x7f2568152950 2024-03-04T08:02:41.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2567200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25680ca0a0 con 0x7f256806faa0 2024-03-04T08:02:41.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2565e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2568152950 0x7f256815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:41.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2565e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2568152950 0x7f256815bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51074/0 (socket says 172.21.15.113:51074) 2024-03-04T08:02:41.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2565e00640 1 -- 172.21.15.113:0/1917872274 learned_addr learned my addr 172.21.15.113:0/1917872274 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:41.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2565400640 1 --2- 172.21.15.113:0/1917872274 >> v2:172.21.15.136:3300/0 conn(0x7f256806faa0 0x7f25680c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:41.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2565e00640 1 -- 172.21.15.113:0/1917872274 >> v2:172.21.15.136:3300/0 conn(0x7f256806faa0 msgr2=0x7f25680c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:41.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2565e00640 1 --2- 172.21.15.113:0/1917872274 >> v2:172.21.15.136:3300/0 conn(0x7f256806faa0 0x7f25680c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:41.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2565e00640 1 -- 172.21.15.113:0/1917872274 >> v1:172.21.15.136:6789/0 conn(0x7f256806ed80 legacy=0x7f25680ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:41.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.112+0000 7f2565e00640 1 -- 172.21.15.113:0/1917872274 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25680c4f80 con 0x7f2568152950 2024-03-04T08:02:41.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.116+0000 7f2565e00640 1 --2- 172.21.15.113:0/1917872274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2568152950 0x7f256815bb40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f256806c110 tx=0x7f255801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac37f5e6eb05079e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:41.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.116+0000 7f2564a00640 1 -- 172.21.15.113:0/1917872274 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580b5070 con 0x7f2568152950 2024-03-04T08:02:41.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.116+0000 7f2564a00640 1 -- 172.21.15.113:0/1917872274 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25580a8a10 con 0x7f2568152950 2024-03-04T08:02:41.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.116+0000 7f2564a00640 1 -- 172.21.15.113:0/1917872274 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580a8de0 con 0x7f2568152950 2024-03-04T08:02:41.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.120+0000 7f2567200640 1 -- 172.21.15.113:0/1917872274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2568152950 msgr2=0x7f256815bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:41.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.120+0000 7f2567200640 1 --2- 172.21.15.113:0/1917872274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2568152950 0x7f256815bb40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f256806c110 tx=0x7f255801ed10 comp rx=0 tx=0).stop 2024-03-04T08:02:41.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.120+0000 7f2567200640 1 -- 172.21.15.113:0/1917872274 shutdown_connections 2024-03-04T08:02:41.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.120+0000 7f2567200640 1 --2- 172.21.15.113:0/1917872274 >> v2:172.21.15.136:3300/0 conn(0x7f256806faa0 0x7f25680c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:41.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.120+0000 7f2567200640 1 --2- 172.21.15.113:0/1917872274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2568152950 0x7f256815bb40 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:41.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.120+0000 7f2567200640 1 -- 172.21.15.113:0/1917872274 >> 172.21.15.113:0/1917872274 conn(0x7f2568072f40 msgr2=0x7f25680706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:41.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.120+0000 7f2567200640 1 -- 172.21.15.113:0/1917872274 shutdown_connections 2024-03-04T08:02:41.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.120+0000 7f2567200640 1 -- 172.21.15.113:0/1917872274 wait complete. 2024-03-04T08:02:41.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.120+0000 7f2567200640 1 Processor -- start 2024-03-04T08:02:41.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.120+0000 7f2567200640 1 -- start start 2024-03-04T08:02:41.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2567200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256806ed80 0x7f256815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:41.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2567200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f256806faa0 0x7f256806da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2567200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2568152950 0x7f25680caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2567200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25680c9a50 con 0x7f256806ed80 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2567200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256806fe80 con 0x7f2568152950 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2567200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25680c9d30 con 0x7f256806faa0 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2565e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256806ed80 0x7f256815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2565e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256806ed80 0x7f256815ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51078/0 (socket says 172.21.15.113:51078) 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2565e00640 1 -- 172.21.15.113:0/2031984549 learned_addr learned my addr 172.21.15.113:0/2031984549 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2565400640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f256806faa0 0x7f256806da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2566800640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2568152950 0x7f25680caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2565e00640 1 -- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f256806faa0 msgr2=0x7f256806da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2565e00640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f256806faa0 0x7f256806da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2565e00640 1 -- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2568152950 msgr2=0x7f25680caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:41.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2565e00640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2568152950 0x7f25680caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:41.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2565e00640 1 -- 172.21.15.113:0/2031984549 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256806df80 con 0x7f256806ed80 2024-03-04T08:02:41.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2565e00640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256806ed80 0x7f256815ba80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f255801b920 tx=0x7f255801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:41.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f255e000640 1 -- 172.21.15.113:0/2031984549 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580b5070 con 0x7f256806ed80 2024-03-04T08:02:41.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f255e000640 1 -- 172.21.15.113:0/2031984549 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25580b0a50 con 0x7f256806ed80 2024-03-04T08:02:41.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2567200640 1 -- 172.21.15.113:0/2031984549 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256806ca20 con 0x7f256806ed80 2024-03-04T08:02:41.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f255e000640 1 -- 172.21.15.113:0/2031984549 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580b0e40 con 0x7f256806ed80 2024-03-04T08:02:41.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2567200640 1 -- 172.21.15.113:0/2031984549 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256806cfc0 con 0x7f256806ed80 2024-03-04T08:02:41.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.124+0000 7f2567200640 1 -- 172.21.15.113:0/2031984549 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2568075a80 con 0x7f256806ed80 2024-03-04T08:02:41.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.128+0000 7f255e000640 1 -- 172.21.15.113:0/2031984549 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25580c2060 con 0x7f256806ed80 2024-03-04T08:02:41.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.128+0000 7f255e000640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2548066f60 0x7f2548069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:41.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.128+0000 7f255e000640 1 -- 172.21.15.113:0/2031984549 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f25581226b0 con 0x7f256806ed80 2024-03-04T08:02:41.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.128+0000 7f2565400640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2548066f60 0x7f2548069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:41.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.128+0000 7f2565400640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2548066f60 0x7f2548069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2568156bc0 tx=0x7f2554008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:41.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.132+0000 7f255e000640 1 -- 172.21.15.113:0/2031984549 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2558018870 con 0x7f256806ed80 2024-03-04T08:02:41.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.332+0000 7f2567200640 1 -- 172.21.15.113:0/2031984549 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25680ca780 con 0x7f256806ed80 2024-03-04T08:02:41.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.332+0000 7f255e000640 1 -- 172.21.15.113:0/2031984549 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6622 (secure 0 0 0) 0x7f25580f3130 con 0x7f256806ed80 2024-03-04T08:02:41.329 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:41.329 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:54.112369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6807,6534,6372]},"id":1}]} 2024-03-04T08:02:41.329 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 106 2024-03-04T08:02:41.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 -- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2548066f60 msgr2=0x7f2548069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:41.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2548066f60 0x7f2548069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2568156bc0 tx=0x7f2554008040 comp rx=0 tx=0).stop 2024-03-04T08:02:41.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 -- 172.21.15.113:0/2031984549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256806ed80 msgr2=0x7f256815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:41.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256806ed80 0x7f256815ba80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f255801b920 tx=0x7f255801bab0 comp rx=0 tx=0).stop 2024-03-04T08:02:41.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 -- 172.21.15.113:0/2031984549 shutdown_connections 2024-03-04T08:02:41.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2568152950 0x7f25680caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:41.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f256806faa0 0x7f256806da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:41.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2548066f60 0x7f2548069420 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:41.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 --2- 172.21.15.113:0/2031984549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f256806ed80 0x7f256815ba80 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:41.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 -- 172.21.15.113:0/2031984549 >> 172.21.15.113:0/2031984549 conn(0x7f2568072f40 msgr2=0x7f25680c6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:41.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 -- 172.21.15.113:0/2031984549 shutdown_connections 2024-03-04T08:02:41.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:41.336+0000 7f2567200640 1 -- 172.21.15.113:0/2031984549 wait complete. 2024-03-04T08:02:41.350 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:02:43.352 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:43.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f6946600640 1 Processor -- start 2024-03-04T08:02:43.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f6946600640 1 -- start start 2024-03-04T08:02:43.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f6946600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6940158230 0x7f694006f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f6946600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694006df10 0x7f694015e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f6946600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f69400703e0 0x7f69400ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f6946600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f694006c310 con 0x7f6940158230 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f6946600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400c95f0 con 0x7f69400703e0 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f6946600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694006c5b0 con 0x7f694006df10 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f693f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6940158230 0x7f694006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f693f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6940158230 0x7f694006f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51094/0 (socket says 172.21.15.113:51094) 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f693f400640 1 -- 172.21.15.113:0/2527292077 learned_addr learned my addr 172.21.15.113:0/2527292077 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f6936a00640 1 --2- 172.21.15.113:0/2527292077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694006df10 0x7f694015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f693fe00640 1 --2- 172.21.15.113:0/2527292077 >> v2:172.21.15.136:3300/0 conn(0x7f69400703e0 0x7f69400ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f693f400640 1 -- 172.21.15.113:0/2527292077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694006df10 msgr2=0x7f694015e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f693f400640 1 --2- 172.21.15.113:0/2527292077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694006df10 0x7f694015e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f693f400640 1 -- 172.21.15.113:0/2527292077 >> v2:172.21.15.136:3300/0 conn(0x7f69400703e0 msgr2=0x7f69400ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:43.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f693f400640 1 --2- 172.21.15.113:0/2527292077 >> v2:172.21.15.136:3300/0 conn(0x7f69400703e0 0x7f69400ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.496+0000 7f693f400640 1 -- 172.21.15.113:0/2527292077 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69400c9890 con 0x7f6940158230 2024-03-04T08:02:43.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f693f400640 1 --2- 172.21.15.113:0/2527292077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6940158230 0x7f694006f840 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f69400723a0 tx=0x7f693001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4367426452d3a8a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:43.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f693ea00640 1 -- 172.21.15.113:0/2527292077 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69300b0070 con 0x7f6940158230 2024-03-04T08:02:43.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f693ea00640 1 -- 172.21.15.113:0/2527292077 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69300ab600 con 0x7f6940158230 2024-03-04T08:02:43.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f693ea00640 1 -- 172.21.15.113:0/2527292077 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69300ab9d0 con 0x7f6940158230 2024-03-04T08:02:43.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f6946600640 1 -- 172.21.15.113:0/2527292077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6940158230 msgr2=0x7f694006f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:43.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f6946600640 1 --2- 172.21.15.113:0/2527292077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6940158230 0x7f694006f840 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f69400723a0 tx=0x7f693001c4f0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f6946600640 1 -- 172.21.15.113:0/2527292077 shutdown_connections 2024-03-04T08:02:43.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f6946600640 1 --2- 172.21.15.113:0/2527292077 >> v2:172.21.15.136:3300/0 conn(0x7f69400703e0 0x7f69400ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f6946600640 1 --2- 172.21.15.113:0/2527292077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694006df10 0x7f694015e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f6946600640 1 --2- 172.21.15.113:0/2527292077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6940158230 0x7f694006f840 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.500+0000 7f6946600640 1 -- 172.21.15.113:0/2527292077 >> 172.21.15.113:0/2527292077 conn(0x7f6940066db0 msgr2=0x7f69400671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:43.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 -- 172.21.15.113:0/2527292077 shutdown_connections 2024-03-04T08:02:43.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 -- 172.21.15.113:0/2527292077 wait complete. 2024-03-04T08:02:43.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 Processor -- start 2024-03-04T08:02:43.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 -- start start 2024-03-04T08:02:43.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694006df10 0x7f69401f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:43.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69400703e0 0x7f69400cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:43.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6940158230 0x7f69400ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:43.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69400cb810 con 0x7f69400703e0 2024-03-04T08:02:43.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69400cb690 con 0x7f6940158230 2024-03-04T08:02:43.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69400cb990 con 0x7f694006df10 2024-03-04T08:02:43.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6936a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69400703e0 0x7f69400cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:43.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6936a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69400703e0 0x7f69400cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51110/0 (socket says 172.21.15.113:51110) 2024-03-04T08:02:43.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6936a00640 1 -- 172.21.15.113:0/2440644466 learned_addr learned my addr 172.21.15.113:0/2440644466 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:43.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f693f400640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694006df10 0x7f69401f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:43.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f693fe00640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6940158230 0x7f69400ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:43.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6936a00640 1 -- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694006df10 msgr2=0x7f69401f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:43.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6936a00640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694006df10 0x7f69401f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6936a00640 1 -- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6940158230 msgr2=0x7f69400ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:43.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6936a00640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6940158230 0x7f69400ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6936a00640 1 -- 172.21.15.113:0/2440644466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69400ce430 con 0x7f69400703e0 2024-03-04T08:02:43.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6936a00640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69400703e0 0x7f69400cc9b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6928007e90 tx=0x7f692800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:43.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6937e00640 1 -- 172.21.15.113:0/2440644466 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928014620 con 0x7f69400703e0 2024-03-04T08:02:43.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6937e00640 1 -- 172.21.15.113:0/2440644466 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6928019070 con 0x7f69400703e0 2024-03-04T08:02:43.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6946600640 1 -- 172.21.15.113:0/2440644466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69401f7bf0 con 0x7f69400703e0 2024-03-04T08:02:43.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.504+0000 7f6937e00640 1 -- 172.21.15.113:0/2440644466 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928014a00 con 0x7f69400703e0 2024-03-04T08:02:43.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.508+0000 7f6946600640 1 -- 172.21.15.113:0/2440644466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694015d900 con 0x7f69400703e0 2024-03-04T08:02:43.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.508+0000 7f6946600640 1 -- 172.21.15.113:0/2440644466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6940075c00 con 0x7f69400703e0 2024-03-04T08:02:43.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.508+0000 7f6937e00640 1 -- 172.21.15.113:0/2440644466 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6928014ba0 con 0x7f69400703e0 2024-03-04T08:02:43.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.508+0000 7f6937e00640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f691c066f60 0x7f691c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:43.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.508+0000 7f6937e00640 1 -- 172.21.15.113:0/2440644466 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7115+0+0 (secure 0 0 0) 0x7f6928085cd0 con 0x7f69400703e0 2024-03-04T08:02:43.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.508+0000 7f693f400640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f691c066f60 0x7f691c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:43.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.512+0000 7f693f400640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f691c066f60 0x7f691c069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f69300ad000 tx=0x7f6930008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:43.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.516+0000 7f6937e00640 1 -- 172.21.15.113:0/2440644466 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6928056110 con 0x7f69400703e0 2024-03-04T08:02:43.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.752+0000 7f6946600640 1 -- 172.21.15.113:0/2440644466 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69400701c0 con 0x7f69400703e0 2024-03-04T08:02:43.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.752+0000 7f6937e00640 1 -- 172.21.15.113:0/2440644466 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6622 (secure 0 0 0) 0x7f692805a1b0 con 0x7f69400703e0 2024-03-04T08:02:43.749 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:43.750 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":6925,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:01:54.112369+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6372,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6372":{"gid":6372,"name":"c","rank":0,"incarnation":83,"state":"up:active","state_seq":40,"addr":"172.21.15.113:6833/2732565639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2732565639},{"type":"v1","addr":"172.21.15.113:6833","nonce":2732565639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6372,"qdb_cluster":[6807,6534,6372]},"id":1}]} 2024-03-04T08:02:43.750 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 106 2024-03-04T08:02:43.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 -- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f691c066f60 msgr2=0x7f691c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:43.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f691c066f60 0x7f691c069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f69300ad000 tx=0x7f6930008040 comp rx=0 tx=0).stop 2024-03-04T08:02:43.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 -- 172.21.15.113:0/2440644466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69400703e0 msgr2=0x7f69400cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:43.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69400703e0 0x7f69400cc9b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6928007e90 tx=0x7f692800e4b0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 -- 172.21.15.113:0/2440644466 shutdown_connections 2024-03-04T08:02:43.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6940158230 0x7f69400ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f691c066f60 0x7f691c069420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69400703e0 0x7f69400cc9b0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 --2- 172.21.15.113:0/2440644466 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f694006df10 0x7f69401f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:43.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 -- 172.21.15.113:0/2440644466 >> 172.21.15.113:0/2440644466 conn(0x7f6940066db0 msgr2=0x7f6940067190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:43.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 -- 172.21.15.113:0/2440644466 shutdown_connections 2024-03-04T08:02:43.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:43.756+0000 7f6946600640 1 -- 172.21.15.113:0/2440644466 wait complete. 2024-03-04T08:02:43.771 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:02:45.772 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:45.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.920+0000 7fd5a7a00640 1 Processor -- start 2024-03-04T08:02:45.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.920+0000 7fd5a7a00640 1 -- start start 2024-03-04T08:02:45.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.920+0000 7fd5a7a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd5a015b440 0x7fd5a006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:45.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.920+0000 7fd5a7a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a015efd0 con 0x7fd5a0155100 2024-03-04T08:02:45.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd5a7a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a00c7ef0 con 0x7fd5a00c08d0 2024-03-04T08:02:45.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd5a7a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a00c8890 con 0x7fd5a015b440 2024-03-04T08:02:45.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd5a5000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd5a0155100 0x7fd5a00cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49672/0 (socket says 172.21.15.113:49672) 2024-03-04T08:02:45.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd5a5000640 1 -- 172.21.15.113:0/1745656908 learned_addr learned my addr 172.21.15.113:0/1745656908 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:02:45.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd5a5a00640 1 --2- 172.21.15.113:0/1745656908 >> v2:172.21.15.136:3300/0 conn(0x7fd5a015b440 0x7fd5a006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:45.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd59f400640 1 -- 172.21.15.113:0/1745656908 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882142206 0 0) 0x7fd5a015efd0 con 0x7fd5a0155100 2024-03-04T08:02:45.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd59f400640 1 -- 172.21.15.113:0/1745656908 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd588003530 con 0x7fd5a0155100 2024-03-04T08:02:45.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd5a5a00640 1 -- 172.21.15.113:0/1745656908 >> v1:172.21.15.136:6789/0 conn(0x7fd5a00c08d0 legacy=0x7fd5a015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:45.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd5a5a00640 1 -- 172.21.15.113:0/1745656908 >> v1:172.21.15.113:6789/0 conn(0x7fd5a0155100 legacy=0x7fd5a00cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd59f400640 1 -- 172.21.15.113:0/1745656908 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642573661 0 0) 0x7fd588003530 con 0x7fd5a0155100 2024-03-04T08:02:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.924+0000 7fd5a5a00640 1 -- 172.21.15.113:0/1745656908 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a00c81d0 con 0x7fd5a015b440 2024-03-04T08:02:45.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd5a5a00640 1 --2- 172.21.15.113:0/1745656908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a015b440 0x7fd5a006b060 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd598009d90 tx=0x7fd59801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd1fde4a7a202ad server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:45.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd59f400640 1 -- 172.21.15.113:0/1745656908 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594001680 con 0x7fd5a015b440 2024-03-04T08:02:45.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd59f400640 1 -- 172.21.15.113:0/1745656908 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5980a64e0 con 0x7fd5a015b440 2024-03-04T08:02:45.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd59f400640 1 -- 172.21.15.113:0/1745656908 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5980a6890 con 0x7fd5a015b440 2024-03-04T08:02:45.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd5a7a00640 1 -- 172.21.15.113:0/1745656908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a015b440 msgr2=0x7fd5a006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:45.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd5a7a00640 1 --2- 172.21.15.113:0/1745656908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a015b440 0x7fd5a006b060 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd598009d90 tx=0x7fd59801c890 comp rx=0 tx=0).stop 2024-03-04T08:02:45.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd5a7a00640 1 -- 172.21.15.113:0/1745656908 shutdown_connections 2024-03-04T08:02:45.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd5a7a00640 1 --2- 172.21.15.113:0/1745656908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a015b440 0x7fd5a006b060 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:45.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd5a7a00640 1 -- 172.21.15.113:0/1745656908 >> 172.21.15.113:0/1745656908 conn(0x7fd5a006deb0 msgr2=0x7fd5a006e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:45.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd5a7a00640 1 -- 172.21.15.113:0/1745656908 shutdown_connections 2024-03-04T08:02:45.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.928+0000 7fd5a7a00640 1 -- 172.21.15.113:0/1745656908 wait complete. 2024-03-04T08:02:45.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a7a00640 1 Processor -- start 2024-03-04T08:02:45.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a7a00640 1 -- start start 2024-03-04T08:02:45.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a7a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a00c08d0 0x7fd5a00c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:45.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a7a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0155100 0x7fd5a00ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:45.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a7a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5a015b440 0x7fd5a01f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:45.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a7a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00cd950 con 0x7fd5a00c08d0 2024-03-04T08:02:45.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a7a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a00cd7d0 con 0x7fd5a0155100 2024-03-04T08:02:45.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a7a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a0168d40 con 0x7fd5a015b440 2024-03-04T08:02:45.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a00c08d0 0x7fd5a00c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a00c08d0 0x7fd5a00c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51122/0 (socket says 172.21.15.113:51122) 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5000640 1 -- 172.21.15.113:0/599351457 learned_addr learned my addr 172.21.15.113:0/599351457 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5a00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5a015b440 0x7fd5a01f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd59fe00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0155100 0x7fd5a00ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5000640 1 -- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5a015b440 msgr2=0x7fd5a01f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5000640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5a015b440 0x7fd5a01f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5000640 1 -- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0155100 msgr2=0x7fd5a00ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5000640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0155100 0x7fd5a00ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5000640 1 -- 172.21.15.113:0/599351457 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a00c60e0 con 0x7fd5a00c08d0 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd59fe00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0155100 0x7fd5a00ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5a00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5a015b440 0x7fd5a01f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:02:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a5000640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a00c08d0 0x7fd5a00c6590 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c730 tx=0x7fd59400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd59cc00640 1 -- 172.21.15.113:0/599351457 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594017070 con 0x7fd5a00c08d0 2024-03-04T08:02:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd59cc00640 1 -- 172.21.15.113:0/599351457 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd594005b00 con 0x7fd5a00c08d0 2024-03-04T08:02:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a7a00640 1 -- 172.21.15.113:0/599351457 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a00c4080 con 0x7fd5a00c08d0 2024-03-04T08:02:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd59cc00640 1 -- 172.21.15.113:0/599351457 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594007070 con 0x7fd5a00c08d0 2024-03-04T08:02:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.932+0000 7fd5a7a00640 1 -- 172.21.15.113:0/599351457 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a015d800 con 0x7fd5a00c08d0 2024-03-04T08:02:45.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.936+0000 7fd5a7a00640 1 -- 172.21.15.113:0/599351457 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a0075c00 con 0x7fd5a00c08d0 2024-03-04T08:02:45.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.936+0000 7fd59cc00640 1 -- 172.21.15.113:0/599351457 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd59401a070 con 0x7fd5a00c08d0 2024-03-04T08:02:45.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.936+0000 7fd59cc00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd580066f60 0x7fd580069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:45.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.936+0000 7fd59cc00640 1 -- 172.21.15.113:0/599351457 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7fd594084c40 con 0x7fd5a00c08d0 2024-03-04T08:02:45.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.936+0000 7fd59fe00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd580066f60 0x7fd580069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:45.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.936+0000 7fd59fe00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd580066f60 0x7fd580069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd5a014ce60 tx=0x7fd58c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:45.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:45.940+0000 7fd59cc00640 1 -- 172.21.15.113:0/599351457 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd5940555d0 con 0x7fd5a00c08d0 2024-03-04T08:02:46.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 -- 172.21.15.113:0/599351457 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5a015efd0 con 0x7fd5a00c08d0 2024-03-04T08:02:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd59cc00640 1 -- 172.21.15.113:0/599351457 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5822 (secure 0 0 0) 0x7fd594054ed0 con 0x7fd5a00c08d0 2024-03-04T08:02:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:46.131 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:02:45.817405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6534,6807]},"id":1}]} 2024-03-04T08:02:46.132 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 107 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 -- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd580066f60 msgr2=0x7fd580069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd580066f60 0x7fd580069420 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd5a014ce60 tx=0x7fd58c009040 comp rx=0 tx=0).stop 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 -- 172.21.15.113:0/599351457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a00c08d0 msgr2=0x7fd5a00c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a00c08d0 0x7fd5a00c6590 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c730 tx=0x7fd59400cbf0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 -- 172.21.15.113:0/599351457 shutdown_connections 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5a015b440 0x7fd5a01f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5a0155100 0x7fd5a00ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd580066f60 0x7fd580069420 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 --2- 172.21.15.113:0/599351457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5a00c08d0 0x7fd5a00c6590 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 -- 172.21.15.113:0/599351457 >> 172.21.15.113:0/599351457 conn(0x7fd5a006deb0 msgr2=0x7fd5a006f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 -- 172.21.15.113:0/599351457 shutdown_connections 2024-03-04T08:02:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.136+0000 7fd5a7a00640 1 -- 172.21.15.113:0/599351457 wait complete. 2024-03-04T08:02:46.154 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-03-04T08:02:46.154 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:02:46.154 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:46.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.308+0000 7f6049400640 1 Processor -- start 2024-03-04T08:02:46.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.308+0000 7f6049400640 1 -- start start 2024-03-04T08:02:46.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6049400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6044154fe0 0x7f60440c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:46.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6049400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f604415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:46.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6049400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60440c7d80 con 0x7f6044154c00 2024-03-04T08:02:46.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6049400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f604415efb0 con 0x7f6044154fe0 2024-03-04T08:02:46.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6049400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60440c6b70 con 0x7f60440c7700 2024-03-04T08:02:46.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6043200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f604415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:46.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f603be00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6044154fe0 0x7f60440c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:46.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6043200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f604415abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:34692/0 (socket says 172.21.15.113:34692) 2024-03-04T08:02:46.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6043200640 1 -- 172.21.15.113:0/1636938840 learned_addr learned my addr 172.21.15.113:0/1636938840 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:46.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6043200640 1 -- 172.21.15.113:0/1636938840 >> v2:172.21.15.136:3300/0 conn(0x7f6044154fe0 msgr2=0x7f60440c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:46.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6043200640 1 --2- 172.21.15.113:0/1636938840 >> v2:172.21.15.136:3300/0 conn(0x7f6044154fe0 0x7f60440c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6043200640 1 -- 172.21.15.113:0/1636938840 >> v1:172.21.15.136:6789/0 conn(0x7f6044154c00 legacy=0x7f60440c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:46.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.312+0000 7f6043200640 1 -- 172.21.15.113:0/1636938840 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440c8060 con 0x7f60440c7700 2024-03-04T08:02:46.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6043200640 1 --2- 172.21.15.113:0/1636938840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f604415abe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f603c009e50 tx=0x7f603c01c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42fb301d663c0aaf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:46.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6041e00640 1 -- 172.21.15.113:0/1636938840 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0a5750 con 0x7f60440c7700 2024-03-04T08:02:46.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6041e00640 1 -- 172.21.15.113:0/1636938840 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603c0a58f0 con 0x7f60440c7700 2024-03-04T08:02:46.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6041e00640 1 -- 172.21.15.113:0/1636938840 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603c0a5cc0 con 0x7f60440c7700 2024-03-04T08:02:46.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6049400640 1 -- 172.21.15.113:0/1636938840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 msgr2=0x7f604415abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:46.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6049400640 1 --2- 172.21.15.113:0/1636938840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f604415abe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f603c009e50 tx=0x7f603c01c9a0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6049400640 1 -- 172.21.15.113:0/1636938840 shutdown_connections 2024-03-04T08:02:46.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6049400640 1 --2- 172.21.15.113:0/1636938840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f604415abe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6049400640 1 --2- 172.21.15.113:0/1636938840 >> v2:172.21.15.136:3300/0 conn(0x7f6044154fe0 0x7f60440c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6049400640 1 -- 172.21.15.113:0/1636938840 >> 172.21.15.113:0/1636938840 conn(0x7f604406e820 msgr2=0x7f604406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:46.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6049400640 1 -- 172.21.15.113:0/1636938840 shutdown_connections 2024-03-04T08:02:46.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6049400640 1 -- 172.21.15.113:0/1636938840 wait complete. 2024-03-04T08:02:46.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6049400640 1 Processor -- start 2024-03-04T08:02:46.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.316+0000 7f6049400640 1 -- start start 2024-03-04T08:02:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6049400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f6044152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6049400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6044154c00 0x7f604415bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6049400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044154fe0 0x7f604414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6049400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60440c23e0 con 0x7f6044154c00 2024-03-04T08:02:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6049400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60440c2260 con 0x7f6044154fe0 2024-03-04T08:02:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6049400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6044167c10 con 0x7f60440c7700 2024-03-04T08:02:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6044154c00 0x7f604415bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6044154c00 0x7f604415bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51130/0 (socket says 172.21.15.113:51130) 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603be00640 1 -- 172.21.15.113:0/215936574 learned_addr learned my addr 172.21.15.113:0/215936574 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6042800640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f6044152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6043200640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044154fe0 0x7f604414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603be00640 1 -- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 msgr2=0x7f6044152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603be00640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f6044152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603be00640 1 -- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044154fe0 msgr2=0x7f604414c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603be00640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044154fe0 0x7f604414c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603be00640 1 -- 172.21.15.113:0/215936574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6044151c40 con 0x7f6044154c00 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6042800640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f6044152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:02:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603be00640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6044154c00 0x7f604415bef0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f602c00e8a0 tx=0x7f602c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:46.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603aa00640 1 -- 172.21.15.113:0/215936574 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c017070 con 0x7f6044154c00 2024-03-04T08:02:46.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603aa00640 1 -- 172.21.15.113:0/215936574 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c00c940 con 0x7f6044154c00 2024-03-04T08:02:46.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f603aa00640 1 -- 172.21.15.113:0/215936574 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c00ccf0 con 0x7f6044154c00 2024-03-04T08:02:46.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6049400640 1 -- 172.21.15.113:0/215936574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6044151f20 con 0x7f6044154c00 2024-03-04T08:02:46.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6049400640 1 -- 172.21.15.113:0/215936574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60441524e0 con 0x7f6044154c00 2024-03-04T08:02:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.320+0000 7f6049400640 1 -- 172.21.15.113:0/215936574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6044075c00 con 0x7f6044154c00 2024-03-04T08:02:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.324+0000 7f603aa00640 1 -- 172.21.15.113:0/215936574 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f602c01a070 con 0x7f6044154c00 2024-03-04T08:02:46.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.324+0000 7f603aa00640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6020066f60 0x7f6020069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:46.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.324+0000 7f603aa00640 1 -- 172.21.15.113:0/215936574 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f602c085840 con 0x7f6044154c00 2024-03-04T08:02:46.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.324+0000 7f6042800640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6020066f60 0x7f6020069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:46.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.324+0000 7f6042800640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6020066f60 0x7f6020069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6034001670 tx=0x7f6034009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:46.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.328+0000 7f603aa00640 1 -- 172.21.15.113:0/215936574 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f602c056270 con 0x7f6044154c00 2024-03-04T08:02:46.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.540+0000 7f6049400640 1 -- 172.21.15.113:0/215936574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60440b22b0 con 0x7f6044154c00 2024-03-04T08:02:46.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.540+0000 7f603aa00640 1 -- 172.21.15.113:0/215936574 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5822 (secure 0 0 0) 0x7f602c055b70 con 0x7f6044154c00 2024-03-04T08:02:46.538 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:46.538 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:02:45.817405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6534,6807]},"id":1}]} 2024-03-04T08:02:46.538 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 107 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 -- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6020066f60 msgr2=0x7f6020069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6020066f60 0x7f6020069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6034001670 tx=0x7f6034009040 comp rx=0 tx=0).stop 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 -- 172.21.15.113:0/215936574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6044154c00 msgr2=0x7f604415bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6044154c00 0x7f604415bef0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f602c00e8a0 tx=0x7f602c00eca0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 -- 172.21.15.113:0/215936574 shutdown_connections 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6044154fe0 0x7f604414c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6020066f60 0x7f6020069420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6044154c00 0x7f604415bef0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 --2- 172.21.15.113:0/215936574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60440c7700 0x7f6044152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 -- 172.21.15.113:0/215936574 >> 172.21.15.113:0/215936574 conn(0x7f604406e820 msgr2=0x7f6044159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:46.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 -- 172.21.15.113:0/215936574 shutdown_connections 2024-03-04T08:02:46.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:46.544+0000 7f6049400640 1 -- 172.21.15.113:0/215936574 wait complete. 2024-03-04T08:02:46.568 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 107, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 107, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6534": { "addr": "172.21.15.113:6837/4197507712", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 4197507712, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 4197507712, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6534, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 24 }, "gid_6807": { "addr": "172.21.15.113:6836/1035982016", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1035982016, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1035982016, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6807, "incarnation": 101, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 14 }, "gid_6925": { "addr": "172.21.15.136:6835/2522634751", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2522634751, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2522634751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6925, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 64, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:02:45.817405+0000", "qdb_cluster": [ 6534, 6807 ], "qdb_leader": 6534, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6925, "mds_1": 6534, "mds_2": 6807 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:02:48.570 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:48.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0933200640 1 Processor -- start 2024-03-04T08:02:48.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0933200640 1 -- start start 2024-03-04T08:02:48.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0933200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0934157ef0 0x7f09340c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:48.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0933200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f093406fb70 0x7f09340cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:48.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0933200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340c9c50 con 0x7f0934070df0 2024-03-04T08:02:48.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0933200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093415a9a0 con 0x7f093406fb70 2024-03-04T08:02:48.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0933200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093406de30 con 0x7f0934157ef0 2024-03-04T08:02:48.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0932800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f093406fb70 0x7f09340cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:48.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0932800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f093406fb70 0x7f09340cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53436/0 (socket says 172.21.15.113:53436) 2024-03-04T08:02:48.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0932800640 1 -- 172.21.15.113:0/1886938588 learned_addr learned my addr 172.21.15.113:0/1886938588 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:48.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0931e00640 1 --2- 172.21.15.113:0/1886938588 >> v2:172.21.15.136:3300/0 conn(0x7f0934157ef0 0x7f09340c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:48.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0932800640 1 -- 172.21.15.113:0/1886938588 >> v2:172.21.15.136:3300/0 conn(0x7f0934157ef0 msgr2=0x7f09340c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:48.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0932800640 1 --2- 172.21.15.113:0/1886938588 >> v2:172.21.15.136:3300/0 conn(0x7f0934157ef0 0x7f09340c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0932800640 1 -- 172.21.15.113:0/1886938588 >> v1:172.21.15.136:6789/0 conn(0x7f0934070df0 legacy=0x7f093415b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:48.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.716+0000 7f0932800640 1 -- 172.21.15.113:0/1886938588 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093406c210 con 0x7f093406fb70 2024-03-04T08:02:48.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0932800640 1 --2- 172.21.15.113:0/1886938588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 0x7f09340cdfe0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0934071490 tx=0x7f092c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306529cfc1a43b11 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:48.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0931400640 1 -- 172.21.15.113:0/1886938588 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0a57d0 con 0x7f093406fb70 2024-03-04T08:02:48.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0931400640 1 -- 172.21.15.113:0/1886938588 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c0a5970 con 0x7f093406fb70 2024-03-04T08:02:48.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0931400640 1 -- 172.21.15.113:0/1886938588 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0a5d40 con 0x7f093406fb70 2024-03-04T08:02:48.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 -- 172.21.15.113:0/1886938588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 msgr2=0x7f09340cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:48.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 --2- 172.21.15.113:0/1886938588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 0x7f09340cdfe0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0934071490 tx=0x7f092c01c9f0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 -- 172.21.15.113:0/1886938588 shutdown_connections 2024-03-04T08:02:48.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 --2- 172.21.15.113:0/1886938588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 0x7f09340cdfe0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 --2- 172.21.15.113:0/1886938588 >> v2:172.21.15.136:3300/0 conn(0x7f0934157ef0 0x7f09340c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 -- 172.21.15.113:0/1886938588 >> 172.21.15.113:0/1886938588 conn(0x7f0934072f40 msgr2=0x7f09340706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:48.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 -- 172.21.15.113:0/1886938588 shutdown_connections 2024-03-04T08:02:48.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 -- 172.21.15.113:0/1886938588 wait complete. 2024-03-04T08:02:48.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 Processor -- start 2024-03-04T08:02:48.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 -- start start 2024-03-04T08:02:48.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 0x7f0934148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:48.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0934070df0 0x7f0934146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0934157ef0 0x7f09341f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09341582d0 con 0x7f093406fb70 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093415b890 con 0x7f0934157ef0 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.724+0000 7f0933200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0934158450 con 0x7f0934070df0 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0931e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 0x7f0934148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0931e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 0x7f0934148900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53452/0 (socket says 172.21.15.113:53452) 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0931e00640 1 -- 172.21.15.113:0/2082961328 learned_addr learned my addr 172.21.15.113:0/2082961328 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0932800640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0934157ef0 0x7f09341f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0929400640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0934070df0 0x7f0934146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0931e00640 1 -- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0934070df0 msgr2=0x7f0934146400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0931e00640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0934070df0 0x7f0934146400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0931e00640 1 -- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0934157ef0 msgr2=0x7f09341f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:48.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0931e00640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0934157ef0 0x7f09341f7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0931e00640 1 -- 172.21.15.113:0/2082961328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0934147860 con 0x7f093406fb70 2024-03-04T08:02:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0929400640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0934070df0 0x7f0934146400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:02:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0931e00640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 0x7f0934148900 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f092400d830 tx=0x7f092400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f092aa00640 1 -- 172.21.15.113:0/2082961328 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400b6c0 con 0x7f093406fb70 2024-03-04T08:02:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f092aa00640 1 -- 172.21.15.113:0/2082961328 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0924019070 con 0x7f093406fb70 2024-03-04T08:02:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f092aa00640 1 -- 172.21.15.113:0/2082961328 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092400ba60 con 0x7f093406fb70 2024-03-04T08:02:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0933200640 1 -- 172.21.15.113:0/2082961328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0934147b40 con 0x7f093406fb70 2024-03-04T08:02:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0933200640 1 -- 172.21.15.113:0/2082961328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0934148100 con 0x7f093406fb70 2024-03-04T08:02:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f0933200640 1 -- 172.21.15.113:0/2082961328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0934075a80 con 0x7f093406fb70 2024-03-04T08:02:48.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.728+0000 7f092aa00640 1 -- 172.21.15.113:0/2082961328 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f092401e460 con 0x7f093406fb70 2024-03-04T08:02:48.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.732+0000 7f092aa00640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0910066f60 0x7f0910069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:48.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.732+0000 7f092aa00640 1 -- 172.21.15.113:0/2082961328 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f0924085c00 con 0x7f093406fb70 2024-03-04T08:02:48.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.732+0000 7f0929400640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0910066f60 0x7f0910069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:48.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.732+0000 7f0929400640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0910066f60 0x7f0910069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f09340cc400 tx=0x7f091c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:48.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.740+0000 7f092aa00640 1 -- 172.21.15.113:0/2082961328 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f092408c080 con 0x7f093406fb70 2024-03-04T08:02:48.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.944+0000 7f0933200640 1 -- 172.21.15.113:0/2082961328 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0934070bd0 con 0x7f093406fb70 2024-03-04T08:02:48.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.944+0000 7f092aa00640 1 -- 172.21.15.113:0/2082961328 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5822 (secure 0 0 0) 0x7f0924056630 con 0x7f093406fb70 2024-03-04T08:02:48.940 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:48.940 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:02:45.817405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6534,6807]},"id":1}]} 2024-03-04T08:02:48.940 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 107 2024-03-04T08:02:48.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 -- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0910066f60 msgr2=0x7f0910069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:48.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0910066f60 0x7f0910069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f09340cc400 tx=0x7f091c009040 comp rx=0 tx=0).stop 2024-03-04T08:02:48.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 -- 172.21.15.113:0/2082961328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 msgr2=0x7f0934148900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:48.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 0x7f0934148900 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f092400d830 tx=0x7f092400dcf0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 -- 172.21.15.113:0/2082961328 shutdown_connections 2024-03-04T08:02:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0934157ef0 0x7f09341f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0934070df0 0x7f0934146400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0910066f60 0x7f0910069420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 --2- 172.21.15.113:0/2082961328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093406fb70 0x7f0934148900 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 -- 172.21.15.113:0/2082961328 >> 172.21.15.113:0/2082961328 conn(0x7f0934072f40 msgr2=0x7f093406ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 -- 172.21.15.113:0/2082961328 shutdown_connections 2024-03-04T08:02:48.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:48.948+0000 7f0933200640 1 -- 172.21.15.113:0/2082961328 wait complete. 2024-03-04T08:02:48.963 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:02:50.964 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:51.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.112+0000 7ff613c00640 1 Processor -- start 2024-03-04T08:02:51.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.112+0000 7ff613c00640 1 -- start start 2024-03-04T08:02:51.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff613c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff60c0b20b0 0x7ff60c0c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:51.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff613c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c06b060 0x7ff60c06b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:51.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff613c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60c0c1ec0 con 0x7ff60c0c85f0 2024-03-04T08:02:51.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff613c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c0c9b30 con 0x7ff60c0b20b0 2024-03-04T08:02:51.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff613c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0c12a0 con 0x7ff60c06b060 2024-03-04T08:02:51.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff60c0c85f0 0x7ff60c157710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36424/0 (socket says 172.21.15.113:36424) 2024-03-04T08:02:51.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60be00640 1 -- 172.21.15.113:0/2616015368 learned_addr learned my addr 172.21.15.113:0/2616015368 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:02:51.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff611c00640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c06b060 0x7ff60c06b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:51.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035179234 0 0) 0x7ff60c0c1ec0 con 0x7ff60c0c85f0 2024-03-04T08:02:51.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f4005740 con 0x7ff60c0c85f0 2024-03-04T08:02:51.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff611200640 1 --2- 172.21.15.113:0/2616015368 >> v2:172.21.15.136:3300/0 conn(0x7ff60c0b20b0 0x7ff60c0c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:51.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2465300312 0 0) 0x7ff5f4005740 con 0x7ff60c0c85f0 2024-03-04T08:02:51.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c06b060 msgr2=0x7ff60c06b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:51.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c06b060 0x7ff60c06b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 >> v2:172.21.15.136:3300/0 conn(0x7ff60c0b20b0 msgr2=0x7ff60c0c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 --2- 172.21.15.113:0/2616015368 >> v2:172.21.15.136:3300/0 conn(0x7ff60c0b20b0 0x7ff60c0c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0c1580 con 0x7ff60c0c85f0 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7ff5f8001680 con 0x7ff60c0c85f0 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 >> v1:172.21.15.113:6789/0 conn(0x7ff60c0c85f0 legacy=0x7ff60c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c06b060 0x7ff5f4005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f4006150 0x7ff5f4006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f4006f60 0x7ff5f40073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0c9b30 con 0x7ff5f4006f60 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0c12a0 con 0x7ff5f4006150 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f400dcc0 con 0x7ff60c06b060 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff611c00640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f4006f60 0x7ff5f40073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff60be00640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f4006150 0x7ff5f4006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff611200640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c06b060 0x7ff5f4005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:51.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff611c00640 1 -- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c06b060 msgr2=0x7ff5f4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:51.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff611c00640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c06b060 0x7ff5f4005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:51.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff611c00640 1 -- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f4006150 msgr2=0x7ff5f4006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:51.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff611c00640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f4006150 0x7ff5f4006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:51.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.116+0000 7ff611c00640 1 -- 172.21.15.113:0/2616015368 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4005740 con 0x7ff5f4006f60 2024-03-04T08:02:51.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff611c00640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f4006f60 0x7ff5f40073e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff604009fc0 tx=0x7ff60401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af923e2a963a111 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:51.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040b2070 con 0x7ff5f4006f60 2024-03-04T08:02:51.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6040a5a90 con 0x7ff5f4006f60 2024-03-04T08:02:51.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff60b400640 1 -- 172.21.15.113:0/2616015368 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604009c10 con 0x7ff5f4006f60 2024-03-04T08:02:51.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff613c00640 1 -- 172.21.15.113:0/2616015368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f4006f60 msgr2=0x7ff5f40073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:51.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff613c00640 1 --2- 172.21.15.113:0/2616015368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5f4006f60 0x7ff5f40073e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7ff604009fc0 tx=0x7ff60401cc80 comp rx=0 tx=0).stop 2024-03-04T08:02:51.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff611c00640 1 -- 172.21.15.113:0/2616015368 reap_dead start 2024-03-04T08:02:51.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff613c00640 1 -- 172.21.15.113:0/2616015368 shutdown_connections 2024-03-04T08:02:51.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff613c00640 1 -- 172.21.15.113:0/2616015368 >> 172.21.15.113:0/2616015368 conn(0x7ff60c06e820 msgr2=0x7ff60c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:51.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff613c00640 1 -- 172.21.15.113:0/2616015368 shutdown_connections 2024-03-04T08:02:51.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.120+0000 7ff613c00640 1 -- 172.21.15.113:0/2616015368 wait complete. 2024-03-04T08:02:51.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.124+0000 7ff613c00640 1 Processor -- start 2024-03-04T08:02:51.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.124+0000 7ff613c00640 1 -- start start 2024-03-04T08:02:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff613c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f4006f60 0x7ff60c0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff613c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c13fc10 0x7ff60c140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff613c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff60c14d150 0x7ff60c14d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff613c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c157e20 con 0x7ff60c14d150 2024-03-04T08:02:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff613c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c1571f0 con 0x7ff5f4006f60 2024-03-04T08:02:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff613c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0c2400 con 0x7ff60c13fc10 2024-03-04T08:02:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff60c14d150 0x7ff60c14d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f4006f60 0x7ff60c0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff60be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c13fc10 0x7ff60c140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff60c14d150 0x7ff60c14d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53468/0 (socket says 172.21.15.113:53468) 2024-03-04T08:02:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611c00640 1 -- 172.21.15.113:0/1488506553 learned_addr learned my addr 172.21.15.113:0/1488506553 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f4006f60 0x7ff60c0cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:45196/0 (socket says 172.21.15.113:45196) 2024-03-04T08:02:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611c00640 1 -- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c13fc10 msgr2=0x7ff60c140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611c00640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c13fc10 0x7ff60c140090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611c00640 1 -- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f4006f60 msgr2=0x7ff60c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611c00640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f4006f60 0x7ff60c0cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611c00640 1 -- 172.21.15.113:0/1488506553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c144340 con 0x7ff60c14d150 2024-03-04T08:02:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff611c00640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff60c14d150 0x7ff60c14d5d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff6040b0040 tx=0x7ff604009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:51.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff608c00640 1 -- 172.21.15.113:0/1488506553 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040bd070 con 0x7ff60c14d150 2024-03-04T08:02:51.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff608c00640 1 -- 172.21.15.113:0/1488506553 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6040aeb60 con 0x7ff60c14d150 2024-03-04T08:02:51.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff613c00640 1 -- 172.21.15.113:0/1488506553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c1445e0 con 0x7ff60c14d150 2024-03-04T08:02:51.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff608c00640 1 -- 172.21.15.113:0/1488506553 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040b2040 con 0x7ff60c14d150 2024-03-04T08:02:51.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.128+0000 7ff613c00640 1 -- 172.21.15.113:0/1488506553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c144ba0 con 0x7ff60c14d150 2024-03-04T08:02:51.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.132+0000 7ff613c00640 1 -- 172.21.15.113:0/1488506553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60c075c00 con 0x7ff60c14d150 2024-03-04T08:02:51.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.132+0000 7ff608c00640 1 -- 172.21.15.113:0/1488506553 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6040c0070 con 0x7ff60c14d150 2024-03-04T08:02:51.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.132+0000 7ff608c00640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5ec066f60 0x7ff5ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:51.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.132+0000 7ff608c00640 1 -- 172.21.15.113:0/1488506553 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7ff60411f6a0 con 0x7ff60c14d150 2024-03-04T08:02:51.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.132+0000 7ff611200640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5ec066f60 0x7ff5ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:51.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.136+0000 7ff611200640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5ec066f60 0x7ff5ec069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff60000bb20 tx=0x7ff600008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:51.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.136+0000 7ff608c00640 1 -- 172.21.15.113:0/1488506553 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff604016990 con 0x7ff60c14d150 2024-03-04T08:02:51.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.472+0000 7ff613c00640 1 -- 172.21.15.113:0/1488506553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff60c0c83d0 con 0x7ff60c14d150 2024-03-04T08:02:51.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.472+0000 7ff608c00640 1 -- 172.21.15.113:0/1488506553 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5822 (secure 0 0 0) 0x7ff6040efa70 con 0x7ff60c14d150 2024-03-04T08:02:51.468 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:51.468 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:02:45.817405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6534,6807]},"id":1}]} 2024-03-04T08:02:51.468 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 107 2024-03-04T08:02:51.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 -- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5ec066f60 msgr2=0x7ff5ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5ec066f60 0x7ff5ec069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff60000bb20 tx=0x7ff600008040 comp rx=0 tx=0).stop 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 -- 172.21.15.113:0/1488506553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff60c14d150 msgr2=0x7ff60c14d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff60c14d150 0x7ff60c14d5d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff6040b0040 tx=0x7ff604009750 comp rx=0 tx=0).stop 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 -- 172.21.15.113:0/1488506553 shutdown_connections 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff5ec066f60 0x7ff5ec069420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff60c14d150 0x7ff60c14d5d0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff60c13fc10 0x7ff60c140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 --2- 172.21.15.113:0/1488506553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5f4006f60 0x7ff60c0cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 -- 172.21.15.113:0/1488506553 >> 172.21.15.113:0/1488506553 conn(0x7ff60c06e820 msgr2=0x7ff60c157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 -- 172.21.15.113:0/1488506553 shutdown_connections 2024-03-04T08:02:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:51.476+0000 7ff613c00640 1 -- 172.21.15.113:0/1488506553 wait complete. 2024-03-04T08:02:51.492 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:02:53.493 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:53.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58b1600640 1 Processor -- start 2024-03-04T08:02:53.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58b1600640 1 -- start start 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58b1600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac151660 0x7f58ac157e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58b1600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f58ac0c8290 0x7f58ac0c7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58b1600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac0c8710 con 0x7f58ac0c6820 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58b1600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac0c89b0 con 0x7f58ac0c8290 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58b1600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0c8c50 con 0x7f58ac151660 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58abe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac151660 0x7f58ac157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58abe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac151660 0x7f58ac157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:43238/0 (socket says 172.21.15.113:43238) 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58abe00640 1 -- 172.21.15.113:0/4130729866 learned_addr learned my addr 172.21.15.113:0/4130729866 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58b0c00640 1 --2- 172.21.15.113:0/4130729866 >> v2:172.21.15.136:3300/0 conn(0x7f58ac0c8290 0x7f58ac0c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58abe00640 1 -- 172.21.15.113:0/4130729866 >> v2:172.21.15.136:3300/0 conn(0x7f58ac0c8290 msgr2=0x7f58ac0c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58abe00640 1 --2- 172.21.15.113:0/4130729866 >> v2:172.21.15.136:3300/0 conn(0x7f58ac0c8290 0x7f58ac0c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.640+0000 7f58abe00640 1 -- 172.21.15.113:0/4130729866 >> v1:172.21.15.136:6789/0 conn(0x7f58ac0c6820 legacy=0x7f58ac0c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:53.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.644+0000 7f58abe00640 1 -- 172.21.15.113:0/4130729866 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac0713e0 con 0x7f58ac151660 2024-03-04T08:02:53.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.644+0000 7f58abe00640 1 --2- 172.21.15.113:0/4130729866 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac151660 0x7f58ac157e40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f589c00bb10 tx=0x7f589c01ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0ce12c08f65df5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:53.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.644+0000 7f58aaa00640 1 -- 172.21.15.113:0/4130729866 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0b4070 con 0x7f58ac151660 2024-03-04T08:02:53.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.644+0000 7f58aaa00640 1 -- 172.21.15.113:0/4130729866 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c0a7c20 con 0x7f58ac151660 2024-03-04T08:02:53.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.644+0000 7f58aaa00640 1 -- 172.21.15.113:0/4130729866 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0af440 con 0x7f58ac151660 2024-03-04T08:02:53.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.644+0000 7f58b1600640 1 -- 172.21.15.113:0/4130729866 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac151660 msgr2=0x7f58ac157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:53.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.644+0000 7f58b1600640 1 --2- 172.21.15.113:0/4130729866 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac151660 0x7f58ac157e40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f589c00bb10 tx=0x7f589c01ec30 comp rx=0 tx=0).stop 2024-03-04T08:02:53.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.644+0000 7f58b1600640 1 -- 172.21.15.113:0/4130729866 shutdown_connections 2024-03-04T08:02:53.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 --2- 172.21.15.113:0/4130729866 >> v2:172.21.15.136:3300/0 conn(0x7f58ac0c8290 0x7f58ac0c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:53.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 --2- 172.21.15.113:0/4130729866 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac151660 0x7f58ac157e40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:53.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 -- 172.21.15.113:0/4130729866 >> 172.21.15.113:0/4130729866 conn(0x7f58ac06b230 msgr2=0x7f58ac06b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:53.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 -- 172.21.15.113:0/4130729866 shutdown_connections 2024-03-04T08:02:53.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 -- 172.21.15.113:0/4130729866 wait complete. 2024-03-04T08:02:53.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 Processor -- start 2024-03-04T08:02:53.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 -- start start 2024-03-04T08:02:53.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58ac0c6820 0x7f58ac13cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:53.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac0c8290 0x7f58ac145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:53.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58ac151660 0x7f58ac14cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:53.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0c9ad0 con 0x7f58ac151660 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac0c9950 con 0x7f58ac0c6820 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b1600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac06eff0 con 0x7f58ac0c8290 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58abe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58ac0c6820 0x7f58ac13cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58ab400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac0c8290 0x7f58ac145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58abe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58ac0c6820 0x7f58ac13cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:45212/0 (socket says 172.21.15.113:45212) 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58ab400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac0c8290 0x7f58ac145db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:43250/0 (socket says 172.21.15.113:43250) 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58abe00640 1 -- 172.21.15.113:0/2121537507 learned_addr learned my addr 172.21.15.113:0/2121537507 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58b0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58ac151660 0x7f58ac14cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58abe00640 1 -- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac0c8290 msgr2=0x7f58ac145db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58abe00640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac0c8290 0x7f58ac145db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58abe00640 1 -- 172.21.15.113:0/2121537507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58ac151660 msgr2=0x7f58ac14cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:53.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58abe00640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58ac151660 0x7f58ac14cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:53.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58abe00640 1 -- 172.21.15.113:0/2121537507 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac072db0 con 0x7f58ac0c6820 2024-03-04T08:02:53.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.648+0000 7f58abe00640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58ac0c6820 0x7f58ac13cac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f589c01d2e0 tx=0x7f589c01b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:53.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.652+0000 7f58a3e00640 1 -- 172.21.15.113:0/2121537507 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0b4070 con 0x7f58ac0c6820 2024-03-04T08:02:53.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.652+0000 7f58a3e00640 1 -- 172.21.15.113:0/2121537507 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c0b6070 con 0x7f58ac0c6820 2024-03-04T08:02:53.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.652+0000 7f58b1600640 1 -- 172.21.15.113:0/2121537507 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac072ad0 con 0x7f58ac0c6820 2024-03-04T08:02:53.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.652+0000 7f58a3e00640 1 -- 172.21.15.113:0/2121537507 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0bb6b0 con 0x7f58ac0c6820 2024-03-04T08:02:53.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.652+0000 7f58b1600640 1 -- 172.21.15.113:0/2121537507 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac1f4b20 con 0x7f58ac0c6820 2024-03-04T08:02:53.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.652+0000 7f58b1600640 1 -- 172.21.15.113:0/2121537507 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac075a80 con 0x7f58ac0c6820 2024-03-04T08:02:53.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.652+0000 7f58a3e00640 1 -- 172.21.15.113:0/2121537507 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f589c0a7ae0 con 0x7f58ac0c6820 2024-03-04T08:02:53.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.656+0000 7f58a3e00640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f588c066f60 0x7f588c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:53.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.656+0000 7f58a3e00640 1 -- 172.21.15.113:0/2121537507 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f589c11ff60 con 0x7f58ac0c6820 2024-03-04T08:02:53.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.656+0000 7f58ab400640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f588c066f60 0x7f588c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:53.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.656+0000 7f58ab400640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f588c066f60 0x7f588c069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f58ac147e80 tx=0x7f5898009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:53.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.664+0000 7f58a3e00640 1 -- 172.21.15.113:0/2121537507 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f589c0ca080 con 0x7f58ac0c6820 2024-03-04T08:02:53.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 -- 172.21.15.113:0/2121537507 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58ac06b990 con 0x7f58ac0c6820 2024-03-04T08:02:53.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58a3e00640 1 -- 172.21.15.113:0/2121537507 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5822 (secure 0 0 0) 0x7f589c018820 con 0x7f58ac0c6820 2024-03-04T08:02:53.855 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:53.856 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:02:45.817405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6534,6807]},"id":1}]} 2024-03-04T08:02:53.856 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 107 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 -- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f588c066f60 msgr2=0x7f588c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f588c066f60 0x7f588c069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f58ac147e80 tx=0x7f5898009040 comp rx=0 tx=0).stop 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 -- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58ac0c6820 msgr2=0x7f58ac13cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58ac0c6820 0x7f58ac13cac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f589c01d2e0 tx=0x7f589c01b9d0 comp rx=0 tx=0).stop 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 -- 172.21.15.113:0/2121537507 shutdown_connections 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f588c066f60 0x7f588c069420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58ac151660 0x7f58ac14cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f58ac0c8290 0x7f58ac145db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 --2- 172.21.15.113:0/2121537507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58ac0c6820 0x7f58ac13cac0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 -- 172.21.15.113:0/2121537507 >> 172.21.15.113:0/2121537507 conn(0x7f58ac06b230 msgr2=0x7f58ac14eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.860+0000 7f58b1600640 1 -- 172.21.15.113:0/2121537507 shutdown_connections 2024-03-04T08:02:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:53.864+0000 7f58b1600640 1 -- 172.21.15.113:0/2121537507 wait complete. 2024-03-04T08:02:53.877 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:02:55.878 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:56.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86ef600640 1 Processor -- start 2024-03-04T08:02:56.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86ef600640 1 -- start start 2024-03-04T08:02:56.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86ef600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e8070120 0x7f86e8158930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:56.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86ef600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f86e80c9db0 0x7f86e8158e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:56.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86ef600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e80c5380 con 0x7f86e8152d60 2024-03-04T08:02:56.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86ef600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e80c3440 con 0x7f86e80c9db0 2024-03-04T08:02:56.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86ef600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e806f620 con 0x7f86e8070120 2024-03-04T08:02:56.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86ecc00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f86e8152d60 0x7f86e80ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36436/0 (socket says 172.21.15.113:36436) 2024-03-04T08:02:56.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86ecc00640 1 -- 172.21.15.113:0/1473406895 learned_addr learned my addr 172.21.15.113:0/1473406895 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:02:56.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86ed600640 1 --2- 172.21.15.113:0/1473406895 >> v2:172.21.15.136:3300/0 conn(0x7f86e80c9db0 0x7f86e8158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:56.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7e00640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e8070120 0x7f86e8158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:56.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451384541 0 0) 0x7f86e80c5380 con 0x7f86e8152d60 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d0003530 con 0x7f86e8152d60 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3249100574 0 0) 0x7f86d0003530 con 0x7f86e8152d60 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e8070120 msgr2=0x7f86e8158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e8070120 0x7f86e8158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 >> v2:172.21.15.136:3300/0 conn(0x7f86e80c9db0 msgr2=0x7f86e8158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 --2- 172.21.15.113:0/1473406895 >> v2:172.21.15.136:3300/0 conn(0x7f86e80c9db0 0x7f86e8158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80c36e0 con 0x7f86e8152d60 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f86dc001680 con 0x7f86e8152d60 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 >> v1:172.21.15.113:6789/0 conn(0x7f86e8152d60 legacy=0x7f86e80ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86d00039c0 0x7f86d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e8070120 0x7f86d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.023+0000 7f86e7400640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 0x7f86d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80c3440 con 0x7f86d0004d50 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e806f620 con 0x7f86d00039c0 2024-03-04T08:02:56.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d000b9d0 con 0x7f86e8070120 2024-03-04T08:02:56.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86ed600640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 0x7f86d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:56.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86e7e00640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e8070120 0x7f86d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:56.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86ecc00640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86d00039c0 0x7f86d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:56.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86ed600640 1 -- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e8070120 msgr2=0x7f86d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:56.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86ed600640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e8070120 0x7f86d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86ed600640 1 -- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86d00039c0 msgr2=0x7f86d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:56.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86ed600640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86d00039c0 0x7f86d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.027+0000 7f86ed600640 1 -- 172.21.15.113:0/1473406895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d0003530 con 0x7f86d0004d50 2024-03-04T08:02:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86ed600640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 0x7f86d00051d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009f90 tx=0x7f86e001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be547c5f4013a287 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00b2070 con 0x7f86d0004d50 2024-03-04T08:02:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e00a5c90 con 0x7f86d0004d50 2024-03-04T08:02:56.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86e7400640 1 -- 172.21.15.113:0/1473406895 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0009c10 con 0x7f86d0004d50 2024-03-04T08:02:56.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86ef600640 1 -- 172.21.15.113:0/1473406895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 msgr2=0x7f86d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:56.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86ef600640 1 --2- 172.21.15.113:0/1473406895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 0x7f86d00051d0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009f90 tx=0x7f86e001cc80 comp rx=0 tx=0).stop 2024-03-04T08:02:56.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86ed600640 1 -- 172.21.15.113:0/1473406895 reap_dead start 2024-03-04T08:02:56.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86ef600640 1 -- 172.21.15.113:0/1473406895 shutdown_connections 2024-03-04T08:02:56.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86ef600640 1 -- 172.21.15.113:0/1473406895 >> 172.21.15.113:0/1473406895 conn(0x7f86e8066db0 msgr2=0x7f86e80671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:56.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86ef600640 1 -- 172.21.15.113:0/1473406895 shutdown_connections 2024-03-04T08:02:56.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.031+0000 7f86ef600640 1 -- 172.21.15.113:0/1473406895 wait complete. 2024-03-04T08:02:56.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 Processor -- start 2024-03-04T08:02:56.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 -- start start 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 0x7f86e80afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e81500e0 0x7f86e814d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e80b09f0 0x7f86e814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80c39c0 con 0x7f86d0004d50 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80c5020 con 0x7f86e81500e0 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80c3b40 con 0x7f86e80b09f0 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ecc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 0x7f86e80afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ecc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 0x7f86e80afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53498/0 (socket says 172.21.15.113:53498) 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ecc00640 1 -- 172.21.15.113:0/235466961 learned_addr learned my addr 172.21.15.113:0/235466961 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86e7e00640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e81500e0 0x7f86e814d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ed600640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e80b09f0 0x7f86e814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:56.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ecc00640 1 -- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e80b09f0 msgr2=0x7f86e814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:56.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ecc00640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e80b09f0 0x7f86e814dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ecc00640 1 -- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e81500e0 msgr2=0x7f86e814d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:56.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ecc00640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e81500e0 0x7f86e814d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ecc00640 1 -- 172.21.15.113:0/235466961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80b02d0 con 0x7f86d0004d50 2024-03-04T08:02:56.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ecc00640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 0x7f86e80afd90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00e7f0 tx=0x7f86dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:56.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86e4c00640 1 -- 172.21.15.113:0/235466961 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc00c870 con 0x7f86d0004d50 2024-03-04T08:02:56.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86e4c00640 1 -- 172.21.15.113:0/235466961 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc018070 con 0x7f86d0004d50 2024-03-04T08:02:56.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86e4c00640 1 -- 172.21.15.113:0/235466961 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc00cc10 con 0x7f86d0004d50 2024-03-04T08:02:56.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 -- 172.21.15.113:0/235466961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e80aea10 con 0x7f86d0004d50 2024-03-04T08:02:56.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 -- 172.21.15.113:0/235466961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e80aefd0 con 0x7f86d0004d50 2024-03-04T08:02:56.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.035+0000 7f86ef600640 1 -- 172.21.15.113:0/235466961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e8075c00 con 0x7f86d0004d50 2024-03-04T08:02:56.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.039+0000 7f86e4c00640 1 -- 172.21.15.113:0/235466961 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86dc01d460 con 0x7f86d0004d50 2024-03-04T08:02:56.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.039+0000 7f86e4c00640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c8066fb0 0x7f86c8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:56.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.039+0000 7f86e4c00640 1 -- 172.21.15.113:0/235466961 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f86dc0854e0 con 0x7f86d0004d50 2024-03-04T08:02:56.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.039+0000 7f86e7e00640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c8066fb0 0x7f86c8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:56.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.039+0000 7f86e7e00640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c8066fb0 0x7f86c8069470 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f86e80c8930 tx=0x7f86d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:56.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.043+0000 7f86e4c00640 1 -- 172.21.15.113:0/235466961 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f86dc055f80 con 0x7f86d0004d50 2024-03-04T08:02:56.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.243+0000 7f86ef600640 1 -- 172.21.15.113:0/235466961 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86e8156b40 con 0x7f86d0004d50 2024-03-04T08:02:56.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.243+0000 7f86e4c00640 1 -- 172.21.15.113:0/235466961 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f86dc055880 con 0x7f86d0004d50 2024-03-04T08:02:56.241 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:56.241 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:02:55.293810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:resolve","state_seq":15,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6807,6534]},"id":1}]} 2024-03-04T08:02:56.241 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 108 2024-03-04T08:02:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 -- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c8066fb0 msgr2=0x7f86c8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c8066fb0 0x7f86c8069470 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f86e80c8930 tx=0x7f86d4008040 comp rx=0 tx=0).stop 2024-03-04T08:02:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 -- 172.21.15.113:0/235466961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 msgr2=0x7f86e80afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 0x7f86e80afd90 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00e7f0 tx=0x7f86dc00ecb0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 -- 172.21.15.113:0/235466961 shutdown_connections 2024-03-04T08:02:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e80b09f0 0x7f86e814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e81500e0 0x7f86e814d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c8066fb0 0x7f86c8069470 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 --2- 172.21.15.113:0/235466961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86d0004d50 0x7f86e80afd90 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:56.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 -- 172.21.15.113:0/235466961 >> 172.21.15.113:0/235466961 conn(0x7f86e8066db0 msgr2=0x7f86e8067190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:56.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 -- 172.21.15.113:0/235466961 shutdown_connections 2024-03-04T08:02:56.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:56.247+0000 7f86ef600640 1 -- 172.21.15.113:0/235466961 wait complete. 2024-03-04T08:02:56.263 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:02:58.264 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:02:58.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.407+0000 7f5123e00640 1 Processor -- start 2024-03-04T08:02:58.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.407+0000 7f5123e00640 1 -- start start 2024-03-04T08:02:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f5123e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51240c8a90 0x7f512415e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f5123e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f512415ea60 0x7f51240ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f5123e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5124070fd0 con 0x7f5124157df0 2024-03-04T08:02:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f5123e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512415e090 con 0x7f51240c8a90 2024-03-04T08:02:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f5123e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512406dfd0 con 0x7f512415ea60 2024-03-04T08:02:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f511be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51240c8a90 0x7f512415e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:58.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f511be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f51240c8a90 0x7f512415e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45902/0 (socket says 172.21.15.113:45902) 2024-03-04T08:02:58.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f511be00640 1 -- 172.21.15.113:0/378323757 learned_addr learned my addr 172.21.15.113:0/378323757 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:58.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f5123400640 1 --2- 172.21.15.113:0/378323757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f512415ea60 0x7f51240ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:58.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f511be00640 1 -- 172.21.15.113:0/378323757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f512415ea60 msgr2=0x7f51240ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:02:58.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f511be00640 1 --2- 172.21.15.113:0/378323757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f512415ea60 0x7f51240ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:58.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f511be00640 1 -- 172.21.15.113:0/378323757 >> v1:172.21.15.136:6789/0 conn(0x7f5124157df0 legacy=0x7f512406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:58.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f511be00640 1 -- 172.21.15.113:0/378323757 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512406fcd0 con 0x7f51240c8a90 2024-03-04T08:02:58.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.411+0000 7f511be00640 1 --2- 172.21.15.113:0/378323757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 0x7f512415e520 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f510c009870 tx=0x7f510c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f001c06fc5feb8b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:58.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5122000640 1 -- 172.21.15.113:0/378323757 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c0b2070 con 0x7f51240c8a90 2024-03-04T08:02:58.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5122000640 1 -- 172.21.15.113:0/378323757 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c0a9070 con 0x7f51240c8a90 2024-03-04T08:02:58.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5122000640 1 -- 172.21.15.113:0/378323757 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c0ad880 con 0x7f51240c8a90 2024-03-04T08:02:58.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 -- 172.21.15.113:0/378323757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 msgr2=0x7f512415e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:58.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 --2- 172.21.15.113:0/378323757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 0x7f512415e520 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f510c009870 tx=0x7f510c01c560 comp rx=0 tx=0).stop 2024-03-04T08:02:58.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 -- 172.21.15.113:0/378323757 shutdown_connections 2024-03-04T08:02:58.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 --2- 172.21.15.113:0/378323757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f512415ea60 0x7f51240ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:58.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 --2- 172.21.15.113:0/378323757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 0x7f512415e520 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:58.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 -- 172.21.15.113:0/378323757 >> 172.21.15.113:0/378323757 conn(0x7f5124072f40 msgr2=0x7f51240706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:58.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 -- 172.21.15.113:0/378323757 shutdown_connections 2024-03-04T08:02:58.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 -- 172.21.15.113:0/378323757 wait complete. 2024-03-04T08:02:58.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 Processor -- start 2024-03-04T08:02:58.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 -- start start 2024-03-04T08:02:58.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 0x7f512415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:58.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5124157df0 0x7f512414d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:58.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f512415ea60 0x7f512414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:58.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512406dd80 con 0x7f51240c8a90 2024-03-04T08:02:58.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512406dc00 con 0x7f512415ea60 2024-03-04T08:02:58.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51241581d0 con 0x7f5124157df0 2024-03-04T08:02:58.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5122a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 0x7f512415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:58.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5122a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 0x7f512415b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45904/0 (socket says 172.21.15.113:45904) 2024-03-04T08:02:58.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5122a00640 1 -- 172.21.15.113:0/1698251142 learned_addr learned my addr 172.21.15.113:0/1698251142 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:02:58.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f511be00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5124157df0 0x7f512414d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:58.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.415+0000 7f5123400640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f512415ea60 0x7f512414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:58.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f5122a00640 1 -- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5124157df0 msgr2=0x7f512414d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:58.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f5122a00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5124157df0 0x7f512414d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:58.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f5122a00640 1 -- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f512415ea60 msgr2=0x7f512414dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:58.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f5122a00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f512415ea60 0x7f512414dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f5122a00640 1 -- 172.21.15.113:0/1698251142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240ce050 con 0x7f51240c8a90 2024-03-04T08:02:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f5123400640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f512415ea60 0x7f512414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:02:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f5122a00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 0x7f512415b090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f511400c420 tx=0x7f511400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f511aa00640 1 -- 172.21.15.113:0/1698251142 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51140052d0 con 0x7f51240c8a90 2024-03-04T08:02:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f511aa00640 1 -- 172.21.15.113:0/1698251142 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5114005470 con 0x7f51240c8a90 2024-03-04T08:02:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f511aa00640 1 -- 172.21.15.113:0/1698251142 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114007260 con 0x7f51240c8a90 2024-03-04T08:02:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f5123e00640 1 -- 172.21.15.113:0/1698251142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51241f4680 con 0x7f51240c8a90 2024-03-04T08:02:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f5123e00640 1 -- 172.21.15.113:0/1698251142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51241f3de0 con 0x7f51240c8a90 2024-03-04T08:02:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f5123e00640 1 -- 172.21.15.113:0/1698251142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5124075a80 con 0x7f51240c8a90 2024-03-04T08:02:58.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f511aa00640 1 -- 172.21.15.113:0/1698251142 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5114007400 con 0x7f51240c8a90 2024-03-04T08:02:58.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f511aa00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5100066f60 0x7f5100069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:02:58.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.419+0000 7f511aa00640 1 -- 172.21.15.113:0/1698251142 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f5114084b90 con 0x7f51240c8a90 2024-03-04T08:02:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.423+0000 7f511be00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5100066f60 0x7f5100069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:02:58.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.423+0000 7f511be00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5100066f60 0x7f5100069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f510c0af000 tx=0x7f510c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:02:58.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.423+0000 7f511aa00640 1 -- 172.21.15.113:0/1698251142 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5124070fd0 con 0x7f51240c8a90 2024-03-04T08:02:58.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 -- 172.21.15.113:0/1698251142 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51240c9610 con 0x7f51240c8a90 2024-03-04T08:02:58.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f511aa00640 1 -- 172.21.15.113:0/1698251142 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f5114054e40 con 0x7f51240c8a90 2024-03-04T08:02:58.615 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:02:58.616 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:02:55.293810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:resolve","state_seq":15,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6807,6534]},"id":1}]} 2024-03-04T08:02:58.616 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 108 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 -- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5100066f60 msgr2=0x7f5100069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5100066f60 0x7f5100069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f510c0af000 tx=0x7f510c008040 comp rx=0 tx=0).stop 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 -- 172.21.15.113:0/1698251142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 msgr2=0x7f512415b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 0x7f512415b090 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f511400c420 tx=0x7f511400c8e0 comp rx=0 tx=0).stop 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 -- 172.21.15.113:0/1698251142 shutdown_connections 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f512415ea60 0x7f512414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5124157df0 0x7f512414d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5100066f60 0x7f5100069420 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 --2- 172.21.15.113:0/1698251142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51240c8a90 0x7f512415b090 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 -- 172.21.15.113:0/1698251142 >> 172.21.15.113:0/1698251142 conn(0x7f5124072f40 msgr2=0x7f51240ca8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 -- 172.21.15.113:0/1698251142 shutdown_connections 2024-03-04T08:02:58.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:02:58.619+0000 7f5123e00640 1 -- 172.21.15.113:0/1698251142 wait complete. 2024-03-04T08:02:58.646 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 108, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 108, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6534": { "addr": "172.21.15.113:6837/4197507712", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 4197507712, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 4197507712, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6534, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 24 }, "gid_6807": { "addr": "172.21.15.113:6836/1035982016", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1035982016, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1035982016, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6807, "incarnation": 101, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 14 }, "gid_6925": { "addr": "172.21.15.136:6835/2522634751", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2522634751, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2522634751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6925, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:resolve", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 64, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:02:55.293810+0000", "qdb_cluster": [ 6807, 6534 ], "qdb_leader": 6534, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6925, "mds_1": 6534, "mds_2": 6807 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:03:00.647 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:00.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.787+0000 7fa552e00640 1 Processor -- start 2024-03-04T08:03:00.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.787+0000 7fa552e00640 1 -- start start 2024-03-04T08:03:00.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa552e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa54c155100 0x7fa54c0c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:00.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa552e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa54c15ad30 0x7fa54c15b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:00.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa552e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa54c15efd0 con 0x7fa54c06b060 2024-03-04T08:03:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa552e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa54c0c8890 con 0x7fa54c15ad30 2024-03-04T08:03:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa552e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c06b4a0 con 0x7fa54c155100 2024-03-04T08:03:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa550e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa54c15ad30 0x7fa54c15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa550e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa54c15ad30 0x7fa54c15b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45916/0 (socket says 172.21.15.113:45916) 2024-03-04T08:03:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa550e00640 1 -- 172.21.15.113:0/2418038577 learned_addr learned my addr 172.21.15.113:0/2418038577 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa54b400640 1 --1- 172.21.15.113:0/2418038577 >> v1:172.21.15.113:6789/0 conn(0x7fa54c06b060 0x7fa54c0cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48228/0 (socket says 172.21.15.113:48228) 2024-03-04T08:03:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa54be00640 1 --2- 172.21.15.113:0/2418038577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa54c155100 0x7fa54c0c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa550e00640 1 -- 172.21.15.113:0/2418038577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa54c155100 msgr2=0x7fa54c0c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:03:00.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa550e00640 1 --2- 172.21.15.113:0/2418038577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa54c155100 0x7fa54c0c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:00.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa550e00640 1 -- 172.21.15.113:0/2418038577 >> v1:172.21.15.113:6789/0 conn(0x7fa54c06b060 legacy=0x7fa54c0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:00.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.791+0000 7fa550e00640 1 -- 172.21.15.113:0/2418038577 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0c7ef0 con 0x7fa54c15ad30 2024-03-04T08:03:00.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa550e00640 1 --2- 172.21.15.113:0/2418038577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 0x7fa54c15b140 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa544009e50 tx=0x7fa54401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fd28bb670924b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:00.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa54aa00640 1 -- 172.21.15.113:0/2418038577 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440a57f0 con 0x7fa54c15ad30 2024-03-04T08:03:00.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa54aa00640 1 -- 172.21.15.113:0/2418038577 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5440a5990 con 0x7fa54c15ad30 2024-03-04T08:03:00.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa54aa00640 1 -- 172.21.15.113:0/2418038577 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440a5d60 con 0x7fa54c15ad30 2024-03-04T08:03:00.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa552e00640 1 -- 172.21.15.113:0/2418038577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 msgr2=0x7fa54c15b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:00.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa552e00640 1 --2- 172.21.15.113:0/2418038577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 0x7fa54c15b140 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fa544009e50 tx=0x7fa54401ca30 comp rx=0 tx=0).stop 2024-03-04T08:03:00.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa552e00640 1 -- 172.21.15.113:0/2418038577 shutdown_connections 2024-03-04T08:03:00.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa552e00640 1 --2- 172.21.15.113:0/2418038577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 0x7fa54c15b140 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:00.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa552e00640 1 --2- 172.21.15.113:0/2418038577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa54c155100 0x7fa54c0c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:00.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa552e00640 1 -- 172.21.15.113:0/2418038577 >> 172.21.15.113:0/2418038577 conn(0x7fa54c06deb0 msgr2=0x7fa54c06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:00.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa552e00640 1 -- 172.21.15.113:0/2418038577 shutdown_connections 2024-03-04T08:03:00.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.795+0000 7fa552e00640 1 -- 172.21.15.113:0/2418038577 wait complete. 2024-03-04T08:03:00.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa552e00640 1 Processor -- start 2024-03-04T08:03:00.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa552e00640 1 -- start start 2024-03-04T08:03:00.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa552e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa54c06b060 0x7fa54c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa552e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa54c155100 0x7fa54c1f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa552e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 0x7fa54c14c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa552e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c15ebb0 con 0x7fa54c15ad30 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa552e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c15b680 con 0x7fa54c155100 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa552e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa54c15ed30 con 0x7fa54c06b060 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa550e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 0x7fa54c14c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa550e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 0x7fa54c14c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45926/0 (socket says 172.21.15.113:45926) 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa550e00640 1 -- 172.21.15.113:0/19890183 learned_addr learned my addr 172.21.15.113:0/19890183 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa54be00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa54c06b060 0x7fa54c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa54b400640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa54c155100 0x7fa54c1f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa550e00640 1 -- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa54c06b060 msgr2=0x7fa54c15c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa550e00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa54c06b060 0x7fa54c15c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa550e00640 1 -- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa54c155100 msgr2=0x7fa54c1f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:00.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa550e00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa54c155100 0x7fa54c1f4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:00.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa550e00640 1 -- 172.21.15.113:0/19890183 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0c7c80 con 0x7fa54c15ad30 2024-03-04T08:03:00.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa550e00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 0x7fa54c14c430 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa54401a660 tx=0x7fa5440197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:00.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa543e00640 1 -- 172.21.15.113:0/19890183 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440b8470 con 0x7fa54c15ad30 2024-03-04T08:03:00.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa543e00640 1 -- 172.21.15.113:0/19890183 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5440b8630 con 0x7fa54c15ad30 2024-03-04T08:03:00.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa543e00640 1 -- 172.21.15.113:0/19890183 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5440b8a20 con 0x7fa54c15ad30 2024-03-04T08:03:00.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa552e00640 1 -- 172.21.15.113:0/19890183 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54c0caa70 con 0x7fa54c15ad30 2024-03-04T08:03:00.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.799+0000 7fa552e00640 1 -- 172.21.15.113:0/19890183 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54c06b640 con 0x7fa54c15ad30 2024-03-04T08:03:00.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.803+0000 7fa552e00640 1 -- 172.21.15.113:0/19890183 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa54c075c00 con 0x7fa54c15ad30 2024-03-04T08:03:00.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.803+0000 7fa543e00640 1 -- 172.21.15.113:0/19890183 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5440c0020 con 0x7fa54c15ad30 2024-03-04T08:03:00.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.803+0000 7fa543e00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 0x7fa52c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:00.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.803+0000 7fa543e00640 1 -- 172.21.15.113:0/19890183 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7fa54411fff0 con 0x7fa54c15ad30 2024-03-04T08:03:00.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.803+0000 7fa54be00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 0x7fa52c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:00.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.807+0000 7fa54be00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 0x7fa52c069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009b60 tx=0x7fa53c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:00.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.807+0000 7fa543e00640 1 -- 172.21.15.113:0/19890183 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa544016620 con 0x7fa54c15ad30 2024-03-04T08:03:00.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.999+0000 7fa552e00640 1 -- 172.21.15.113:0/19890183 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa54c068a60 con 0x7fa54c15ad30 2024-03-04T08:03:00.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:00.999+0000 7fa543e00640 1 -- 172.21.15.113:0/19890183 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+5824 (secure 0 0 0) 0x7fa5440f09e0 con 0x7fa54c15ad30 2024-03-04T08:03:00.997 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:00.997 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:02:55.293810+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:resolve","state_seq":15,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6807,6534]},"id":1}]} 2024-03-04T08:03:00.997 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 108 2024-03-04T08:03:00.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 -- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 msgr2=0x7fa52c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:00.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 0x7fa52c069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa53c009b60 tx=0x7fa53c009190 comp rx=0 tx=0).stop 2024-03-04T08:03:00.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 -- 172.21.15.113:0/19890183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 msgr2=0x7fa54c14c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:00.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 0x7fa54c14c430 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa54401a660 tx=0x7fa5440197d0 comp rx=0 tx=0).stop 2024-03-04T08:03:00.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 -- 172.21.15.113:0/19890183 shutdown_connections 2024-03-04T08:03:00.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa52c066f60 0x7fa52c069420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:00.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa54c15ad30 0x7fa54c14c430 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:00.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa54c155100 0x7fa54c1f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:00.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 --2- 172.21.15.113:0/19890183 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa54c06b060 0x7fa54c15c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:01.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 -- 172.21.15.113:0/19890183 >> 172.21.15.113:0/19890183 conn(0x7fa54c06deb0 msgr2=0x7fa54c151f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:01.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 -- 172.21.15.113:0/19890183 shutdown_connections 2024-03-04T08:03:01.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:01.003+0000 7fa552e00640 1 -- 172.21.15.113:0/19890183 wait complete. 2024-03-04T08:03:01.026 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:03:03.027 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:03.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd99a000640 1 Processor -- start 2024-03-04T08:03:03.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd99a000640 1 -- start start 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd99a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd994158010 0x7fd99406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd99a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd99406ff20 0x7fd99415b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd99a000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9940ca0e0 con 0x7fd99415b750 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd99a000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99406f450 con 0x7fd99406ff20 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd99a000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99415ad80 con 0x7fd994158010 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd98be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd99406ff20 0x7fd99415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd993e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd99415b750 0x7fd9940ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48236/0 (socket says 172.21.15.113:48236) 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd993e00640 1 -- 172.21.15.113:0/175833410 learned_addr learned my addr 172.21.15.113:0/175833410 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd993400640 1 --2- 172.21.15.113:0/175833410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd994158010 0x7fd99406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd992a00640 1 -- 172.21.15.113:0/175833410 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22623530 0 0) 0x7fd9940ca0e0 con 0x7fd99415b750 2024-03-04T08:03:03.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd992a00640 1 -- 172.21.15.113:0/175833410 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd978003530 con 0x7fd99415b750 2024-03-04T08:03:03.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd98be00640 1 -- 172.21.15.113:0/175833410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd994158010 msgr2=0x7fd99406dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:03:03.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd98be00640 1 --2- 172.21.15.113:0/175833410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd994158010 0x7fd99406dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:03.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd98be00640 1 -- 172.21.15.113:0/175833410 >> v1:172.21.15.113:6789/0 conn(0x7fd99415b750 legacy=0x7fd9940ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:03.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.171+0000 7fd98be00640 1 -- 172.21.15.113:0/175833410 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99415aa30 con 0x7fd99406ff20 2024-03-04T08:03:03.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.175+0000 7fd98be00640 1 --2- 172.21.15.113:0/175833410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99406ff20 0x7fd99415b210 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0097c0 tx=0x7fd97c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4486b0e03b730da3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd992a00640 1 -- 172.21.15.113:0/175833410 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b0070 con 0x7fd99406ff20 2024-03-04T08:03:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd992a00640 1 -- 172.21.15.113:0/175833410 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c0ab600 con 0x7fd99406ff20 2024-03-04T08:03:03.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd992a00640 1 -- 172.21.15.113:0/175833410 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0ab9b0 con 0x7fd99406ff20 2024-03-04T08:03:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 -- 172.21.15.113:0/175833410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99406ff20 msgr2=0x7fd99415b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 --2- 172.21.15.113:0/175833410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99406ff20 0x7fd99415b210 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0097c0 tx=0x7fd97c01c4a0 comp rx=0 tx=0).stop 2024-03-04T08:03:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 -- 172.21.15.113:0/175833410 shutdown_connections 2024-03-04T08:03:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 --2- 172.21.15.113:0/175833410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99406ff20 0x7fd99415b210 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 --2- 172.21.15.113:0/175833410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd994158010 0x7fd99406dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 -- 172.21.15.113:0/175833410 >> 172.21.15.113:0/175833410 conn(0x7fd9940734d0 msgr2=0x7fd994066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 -- 172.21.15.113:0/175833410 shutdown_connections 2024-03-04T08:03:03.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 -- 172.21.15.113:0/175833410 wait complete. 2024-03-04T08:03:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 Processor -- start 2024-03-04T08:03:03.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 -- start start 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd99406ff20 0x7fd9940b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd994158010 0x7fd9940cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99415b750 0x7fd9940ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99406c170 con 0x7fd99415b750 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9941583f0 con 0x7fd99406ff20 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.179+0000 7fd99a000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99406c2f0 con 0x7fd994158010 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd993e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99415b750 0x7fd9940ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd993400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd99406ff20 0x7fd9940b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd993e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99415b750 0x7fd9940ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45948/0 (socket says 172.21.15.113:45948) 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd993e00640 1 -- 172.21.15.113:0/1797931007 learned_addr learned my addr 172.21.15.113:0/1797931007 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd98be00640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd994158010 0x7fd9940cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd993e00640 1 -- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd994158010 msgr2=0x7fd9940cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd993e00640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd994158010 0x7fd9940cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd993e00640 1 -- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd99406ff20 msgr2=0x7fd9940b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd993e00640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd99406ff20 0x7fd9940b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd993e00640 1 -- 172.21.15.113:0/1797931007 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd994153060 con 0x7fd99415b750 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd993e00640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99415b750 0x7fd9940ccc00 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00caa0 tx=0x7fd98c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd98b400640 1 -- 172.21.15.113:0/1797931007 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c017070 con 0x7fd99415b750 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd98b400640 1 -- 172.21.15.113:0/1797931007 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c00ac00 con 0x7fd99415b750 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd99a000640 1 -- 172.21.15.113:0/1797931007 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd994158940 con 0x7fd99415b750 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd98b400640 1 -- 172.21.15.113:0/1797931007 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c012400 con 0x7fd99415b750 2024-03-04T08:03:03.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd99a000640 1 -- 172.21.15.113:0/1797931007 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd994153340 con 0x7fd99415b750 2024-03-04T08:03:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd99a000640 1 -- 172.21.15.113:0/1797931007 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd994075c00 con 0x7fd99415b750 2024-03-04T08:03:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd98b400640 1 -- 172.21.15.113:0/1797931007 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd98c01a070 con 0x7fd99415b750 2024-03-04T08:03:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.183+0000 7fd98b400640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd970066f90 0x7fd970069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:03.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.187+0000 7fd98b400640 1 -- 172.21.15.113:0/1797931007 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7fd98c0852d0 con 0x7fd99415b750 2024-03-04T08:03:03.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.187+0000 7fd993400640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd970066f90 0x7fd970069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:03.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.187+0000 7fd993400640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd970066f90 0x7fd970069450 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd984009b60 tx=0x7fd984009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.191+0000 7fd98b400640 1 -- 172.21.15.113:0/1797931007 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd98c055610 con 0x7fd99415b750 2024-03-04T08:03:03.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 -- 172.21.15.113:0/1797931007 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd99406fd00 con 0x7fd99415b750 2024-03-04T08:03:03.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd98b400640 1 -- 172.21.15.113:0/1797931007 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+5826 (secure 0 0 0) 0x7fd98c0596b0 con 0x7fd99415b750 2024-03-04T08:03:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:03.398 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":109,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:01.301985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6534,6807]},"id":1}]} 2024-03-04T08:03:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 109 2024-03-04T08:03:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 -- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd970066f90 msgr2=0x7fd970069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd970066f90 0x7fd970069450 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd984009b60 tx=0x7fd984009190 comp rx=0 tx=0).stop 2024-03-04T08:03:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 -- 172.21.15.113:0/1797931007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99415b750 msgr2=0x7fd9940ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:03.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99415b750 0x7fd9940ccc00 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00caa0 tx=0x7fd98c00cf60 comp rx=0 tx=0).stop 2024-03-04T08:03:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 -- 172.21.15.113:0/1797931007 shutdown_connections 2024-03-04T08:03:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd970066f90 0x7fd970069450 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd99415b750 0x7fd9940ccc00 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd994158010 0x7fd9940cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 --2- 172.21.15.113:0/1797931007 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd99406ff20 0x7fd9940b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 -- 172.21.15.113:0/1797931007 >> 172.21.15.113:0/1797931007 conn(0x7fd9940734d0 msgr2=0x7fd994156480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 -- 172.21.15.113:0/1797931007 shutdown_connections 2024-03-04T08:03:03.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:03.399+0000 7fd99a000640 1 -- 172.21.15.113:0/1797931007 wait complete. 2024-03-04T08:03:03.418 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:03:04.145 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:03:04.150 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:03:05.419 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.563+0000 7fee29a00640 1 Processor -- start 2024-03-04T08:03:05.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.563+0000 7fee29a00640 1 -- start start 2024-03-04T08:03:05.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee29a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee2414e4c0 0x7fee24072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:05.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee29a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee240c22e0 0x7fee240731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:05.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee29a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fee240c2d80 0x7fee2415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:05.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee29a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee2415ad60 con 0x7fee2414e4c0 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee29a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee240c27f0 con 0x7fee240c2d80 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee29a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee240c2a90 con 0x7fee240c22e0 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee22a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee2414e4c0 0x7fee24072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee22a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fee2414e4c0 0x7fee24072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45964/0 (socket says 172.21.15.113:45964) 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee22a00640 1 -- 172.21.15.113:0/2765773488 learned_addr learned my addr 172.21.15.113:0/2765773488 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee22000640 1 --2- 172.21.15.113:0/2765773488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee240c22e0 0x7fee240731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee23400640 1 --2- 172.21.15.113:0/2765773488 >> v2:172.21.15.136:3300/0 conn(0x7fee240c2d80 0x7fee2415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee22a00640 1 -- 172.21.15.113:0/2765773488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee240c22e0 msgr2=0x7fee240731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee22a00640 1 --2- 172.21.15.113:0/2765773488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee240c22e0 0x7fee240731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee22a00640 1 -- 172.21.15.113:0/2765773488 >> v2:172.21.15.136:3300/0 conn(0x7fee240c2d80 msgr2=0x7fee2415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee22a00640 1 --2- 172.21.15.113:0/2765773488 >> v2:172.21.15.136:3300/0 conn(0x7fee240c2d80 0x7fee2415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee22a00640 1 -- 172.21.15.113:0/2765773488 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee24070c60 con 0x7fee2414e4c0 2024-03-04T08:03:05.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee22a00640 1 --2- 172.21.15.113:0/2765773488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2414e4c0 0x7fee24072c80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fee240c2700 tx=0x7fee1401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bb209575c75563 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:05.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.567+0000 7fee21600640 1 -- 172.21.15.113:0/2765773488 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee140b5070 con 0x7fee2414e4c0 2024-03-04T08:03:05.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee21600640 1 -- 172.21.15.113:0/2765773488 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee140a89d0 con 0x7fee2414e4c0 2024-03-04T08:03:05.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee21600640 1 -- 172.21.15.113:0/2765773488 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee140a8da0 con 0x7fee2414e4c0 2024-03-04T08:03:05.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 -- 172.21.15.113:0/2765773488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2414e4c0 msgr2=0x7fee24072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:05.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 --2- 172.21.15.113:0/2765773488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2414e4c0 0x7fee24072c80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fee240c2700 tx=0x7fee1401ed10 comp rx=0 tx=0).stop 2024-03-04T08:03:05.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 -- 172.21.15.113:0/2765773488 shutdown_connections 2024-03-04T08:03:05.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 --2- 172.21.15.113:0/2765773488 >> v2:172.21.15.136:3300/0 conn(0x7fee240c2d80 0x7fee2415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 --2- 172.21.15.113:0/2765773488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee240c22e0 0x7fee240731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 --2- 172.21.15.113:0/2765773488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee2414e4c0 0x7fee24072c80 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 -- 172.21.15.113:0/2765773488 >> 172.21.15.113:0/2765773488 conn(0x7fee2406e820 msgr2=0x7fee2406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:05.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 -- 172.21.15.113:0/2765773488 shutdown_connections 2024-03-04T08:03:05.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 -- 172.21.15.113:0/2765773488 wait complete. 2024-03-04T08:03:05.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 Processor -- start 2024-03-04T08:03:05.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 -- start start 2024-03-04T08:03:05.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee240c22e0 0x7fee24156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:05.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee240c2d80 0x7fee241553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:05.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee2414e4c0 0x7fee24159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:05.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee24071500 con 0x7fee240c2d80 2024-03-04T08:03:05.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee24071220 con 0x7fee240c22e0 2024-03-04T08:03:05.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee29a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2414e8a0 con 0x7fee2414e4c0 2024-03-04T08:03:05.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee22000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee240c2d80 0x7fee241553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:05.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee22000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee240c2d80 0x7fee241553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45968/0 (socket says 172.21.15.113:45968) 2024-03-04T08:03:05.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee22000640 1 -- 172.21.15.113:0/2592146602 learned_addr learned my addr 172.21.15.113:0/2592146602 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:05.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee23400640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee2414e4c0 0x7fee24159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:05.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.571+0000 7fee22a00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee240c22e0 0x7fee24156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:05.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee22000640 1 -- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee2414e4c0 msgr2=0x7fee24159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:05.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee22000640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee2414e4c0 0x7fee24159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee22000640 1 -- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee240c22e0 msgr2=0x7fee24156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:05.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee22000640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee240c22e0 0x7fee24156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee22000640 1 -- 172.21.15.113:0/2592146602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2414f770 con 0x7fee240c2d80 2024-03-04T08:03:05.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee22a00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee240c22e0 0x7fee24156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:03:05.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee22000640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee240c2d80 0x7fee241553b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fee1000d7d0 tx=0x7fee1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:05.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee1aa00640 1 -- 172.21.15.113:0/2592146602 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee10004020 con 0x7fee240c2d80 2024-03-04T08:03:05.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee1aa00640 1 -- 172.21.15.113:0/2592146602 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee100041c0 con 0x7fee240c2d80 2024-03-04T08:03:05.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee29a00640 1 -- 172.21.15.113:0/2592146602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee241517e0 con 0x7fee240c2d80 2024-03-04T08:03:05.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee1aa00640 1 -- 172.21.15.113:0/2592146602 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee100043b0 con 0x7fee240c2d80 2024-03-04T08:03:05.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee29a00640 1 -- 172.21.15.113:0/2592146602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2414fa10 con 0x7fee240c2d80 2024-03-04T08:03:05.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee0fe00640 1 -- 172.21.15.113:0/2592146602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee24075c00 con 0x7fee240c2d80 2024-03-04T08:03:05.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee1aa00640 1 -- 172.21.15.113:0/2592146602 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee1001b020 con 0x7fee240c2d80 2024-03-04T08:03:05.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.575+0000 7fee1aa00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee04066f60 0x7fee04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:05.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.579+0000 7fee1aa00640 1 -- 172.21.15.113:0/2592146602 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7fee100852b0 con 0x7fee240c2d80 2024-03-04T08:03:05.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.579+0000 7fee22a00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee04066f60 0x7fee04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:05.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.579+0000 7fee22a00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee04066f60 0x7fee04069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fee140b2000 tx=0x7fee14008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:05.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.583+0000 7fee1aa00640 1 -- 172.21.15.113:0/2592146602 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fee10055d70 con 0x7fee240c2d80 2024-03-04T08:03:05.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.871+0000 7fee0fe00640 1 -- 172.21.15.113:0/2592146602 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee240c68c0 con 0x7fee240c2d80 2024-03-04T08:03:05.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee1aa00640 1 -- 172.21.15.113:0/2592146602 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+5826 (secure 0 0 0) 0x7fee10055670 con 0x7fee240c2d80 2024-03-04T08:03:05.870 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:05.870 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":109,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:01.301985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:reconnect","state_seq":17,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6534,6807]},"id":1}]} 2024-03-04T08:03:05.871 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 109 2024-03-04T08:03:05.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 -- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee04066f60 msgr2=0x7fee04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:05.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee04066f60 0x7fee04069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fee140b2000 tx=0x7fee14008040 comp rx=0 tx=0).stop 2024-03-04T08:03:05.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 -- 172.21.15.113:0/2592146602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee240c2d80 msgr2=0x7fee241553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:05.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee240c2d80 0x7fee241553b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fee1000d7d0 tx=0x7fee1000dc90 comp rx=0 tx=0).stop 2024-03-04T08:03:05.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 -- 172.21.15.113:0/2592146602 shutdown_connections 2024-03-04T08:03:05.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fee2414e4c0 0x7fee24159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fee04066f60 0x7fee04069420 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fee240c2d80 0x7fee241553b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 --2- 172.21.15.113:0/2592146602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee240c22e0 0x7fee24156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:05.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 -- 172.21.15.113:0/2592146602 >> 172.21.15.113:0/2592146602 conn(0x7fee2406e820 msgr2=0x7fee2406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:05.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 -- 172.21.15.113:0/2592146602 shutdown_connections 2024-03-04T08:03:05.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:05.875+0000 7fee0fe00640 1 -- 172.21.15.113:0/2592146602 wait complete. 2024-03-04T08:03:05.892 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:03:07.893 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:08.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f980c800640 1 Processor -- start 2024-03-04T08:03:08.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f980c800640 1 -- start start 2024-03-04T08:03:08.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f980c800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9804152bd0 0x7f9804070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:08.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f980c800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98041591f0 con 0x7f98041527f0 2024-03-04T08:03:08.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f980c800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98041594d0 con 0x7f98040c8200 2024-03-04T08:03:08.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f980c800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9804071df0 con 0x7f9804152bd0 2024-03-04T08:03:08.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f9809e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f98041527f0 0x7f9804072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44074/0 (socket says 172.21.15.113:44074) 2024-03-04T08:03:08.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f9809e00640 1 -- 172.21.15.113:0/2672484896 learned_addr learned my addr 172.21.15.113:0/2672484896 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:03:08.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f980a800640 1 --2- 172.21.15.113:0/2672484896 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9804152bd0 0x7f9804070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:08.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f9808a00640 1 -- 172.21.15.113:0/2672484896 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115591207 0 0) 0x7f98041591f0 con 0x7f98041527f0 2024-03-04T08:03:08.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f9808a00640 1 -- 172.21.15.113:0/2672484896 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97ec003530 con 0x7f98041527f0 2024-03-04T08:03:08.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f9808a00640 1 -- 172.21.15.113:0/2672484896 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2528650562 0 0) 0x7f97ec003530 con 0x7f98041527f0 2024-03-04T08:03:08.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f980a800640 1 -- 172.21.15.113:0/2672484896 >> v1:172.21.15.136:6789/0 conn(0x7f98040c8200 legacy=0x7f98041587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:08.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f980a800640 1 -- 172.21.15.113:0/2672484896 >> v1:172.21.15.113:6789/0 conn(0x7f98041527f0 legacy=0x7f9804072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:08.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.035+0000 7f980a800640 1 -- 172.21.15.113:0/2672484896 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98040717a0 con 0x7f9804152bd0 2024-03-04T08:03:08.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.039+0000 7f980a800640 1 --2- 172.21.15.113:0/2672484896 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9804152bd0 0x7f9804070d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9800009dc0 tx=0x7f980001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=439cf80cde502a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.039+0000 7f9808a00640 1 -- 172.21.15.113:0/2672484896 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8001680 con 0x7f9804152bd0 2024-03-04T08:03:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.039+0000 7f9808a00640 1 -- 172.21.15.113:0/2672484896 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98000a6500 con 0x7f9804152bd0 2024-03-04T08:03:08.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.039+0000 7f9808a00640 1 -- 172.21.15.113:0/2672484896 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98000a68b0 con 0x7f9804152bd0 2024-03-04T08:03:08.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.039+0000 7f980c800640 1 -- 172.21.15.113:0/2672484896 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9804152bd0 msgr2=0x7f9804070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:08.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.039+0000 7f980c800640 1 --2- 172.21.15.113:0/2672484896 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9804152bd0 0x7f9804070d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9800009dc0 tx=0x7f980001c8b0 comp rx=0 tx=0).stop 2024-03-04T08:03:08.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.039+0000 7f980c800640 1 -- 172.21.15.113:0/2672484896 shutdown_connections 2024-03-04T08:03:08.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.039+0000 7f980c800640 1 --2- 172.21.15.113:0/2672484896 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9804152bd0 0x7f9804070d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:08.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 -- 172.21.15.113:0/2672484896 >> 172.21.15.113:0/2672484896 conn(0x7f980406e820 msgr2=0x7f980406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:08.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 -- 172.21.15.113:0/2672484896 shutdown_connections 2024-03-04T08:03:08.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 -- 172.21.15.113:0/2672484896 wait complete. 2024-03-04T08:03:08.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 Processor -- start 2024-03-04T08:03:08.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 -- start start 2024-03-04T08:03:08.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98040c8200 0x7f980414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:08.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98041527f0 0x7f98040c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:08.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9804152bd0 0x7f98040c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9804166500 con 0x7f9804152bd0 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9804166380 con 0x7f98041527f0 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980c800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9804166680 con 0x7f98040c8200 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9804152bd0 0x7f98040c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9804152bd0 0x7f98040c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40884/0 (socket says 172.21.15.113:40884) 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980a800640 1 -- 172.21.15.113:0/1222691374 learned_addr learned my addr 172.21.15.113:0/1222691374 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f9809e00640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98040c8200 0x7f980414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f9809400640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98041527f0 0x7f98040c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980a800640 1 -- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98040c8200 msgr2=0x7f980414fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980a800640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98040c8200 0x7f980414fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:08.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980a800640 1 -- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98041527f0 msgr2=0x7f98040c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:08.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980a800640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98041527f0 0x7f98040c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:08.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980a800640 1 -- 172.21.15.113:0/1222691374 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980414dce0 con 0x7f9804152bd0 2024-03-04T08:03:08.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.043+0000 7f980a800640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9804152bd0 0x7f98040c9030 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f980001af60 tx=0x7f9800019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:08.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.047+0000 7f97fe000640 1 -- 172.21.15.113:0/1222691374 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98000a6b40 con 0x7f9804152bd0 2024-03-04T08:03:08.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.047+0000 7f97fe000640 1 -- 172.21.15.113:0/1222691374 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98000bf070 con 0x7f9804152bd0 2024-03-04T08:03:08.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.047+0000 7f97fe000640 1 -- 172.21.15.113:0/1222691374 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800004040 con 0x7f9804152bd0 2024-03-04T08:03:08.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.047+0000 7f980c800640 1 -- 172.21.15.113:0/1222691374 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980414ef60 con 0x7f9804152bd0 2024-03-04T08:03:08.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.047+0000 7f980c800640 1 -- 172.21.15.113:0/1222691374 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98040c43a0 con 0x7f9804152bd0 2024-03-04T08:03:08.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.047+0000 7f980c800640 1 -- 172.21.15.113:0/1222691374 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9804075c00 con 0x7f9804152bd0 2024-03-04T08:03:08.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.047+0000 7f97fe000640 1 -- 172.21.15.113:0/1222691374 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98000041e0 con 0x7f9804152bd0 2024-03-04T08:03:08.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.047+0000 7f97fe000640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97e8066fb0 0x7f97e8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:08.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.047+0000 7f97fe000640 1 -- 172.21.15.113:0/1222691374 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f9800120580 con 0x7f9804152bd0 2024-03-04T08:03:08.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.047+0000 7f9809e00640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97e8066fb0 0x7f97e8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:08.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.051+0000 7f9809e00640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97e8066fb0 0x7f97e8069470 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005890 tx=0x7f97f8005800 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:08.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.051+0000 7f97fe000640 1 -- 172.21.15.113:0/1222691374 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f98000164a0 con 0x7f9804152bd0 2024-03-04T08:03:08.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 -- 172.21.15.113:0/1222691374 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f980414e1b0 con 0x7f9804152bd0 2024-03-04T08:03:08.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f97fe000640 1 -- 172.21.15.113:0/1222691374 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f98000f0ad0 con 0x7f9804152bd0 2024-03-04T08:03:08.248 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:08.248 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:07.241830+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6807,6534]},"id":1}]} 2024-03-04T08:03:08.248 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 110 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 -- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97e8066fb0 msgr2=0x7f97e8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97e8066fb0 0x7f97e8069470 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f97f8005890 tx=0x7f97f8005800 comp rx=0 tx=0).stop 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 -- 172.21.15.113:0/1222691374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9804152bd0 msgr2=0x7f98040c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9804152bd0 0x7f98040c9030 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f980001af60 tx=0x7f9800019650 comp rx=0 tx=0).stop 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 -- 172.21.15.113:0/1222691374 shutdown_connections 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97e8066fb0 0x7f97e8069470 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9804152bd0 0x7f98040c9030 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98041527f0 0x7f98040c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 --2- 172.21.15.113:0/1222691374 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98040c8200 0x7f980414fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.251+0000 7f980c800640 1 -- 172.21.15.113:0/1222691374 >> 172.21.15.113:0/1222691374 conn(0x7f980406e820 msgr2=0x7f980406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.255+0000 7f980c800640 1 -- 172.21.15.113:0/1222691374 shutdown_connections 2024-03-04T08:03:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:08.255+0000 7f980c800640 1 -- 172.21.15.113:0/1222691374 wait complete. 2024-03-04T08:03:08.269 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:03:10.271 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:10.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.411+0000 7eff0ba00640 1 Processor -- start 2024-03-04T08:03:10.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.411+0000 7eff0ba00640 1 -- start start 2024-03-04T08:03:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff0ba00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7eff040b20b0 0x7eff040c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff0ba00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7eff0406b060 0x7eff0406b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff0ba00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff040c1ec0 con 0x7eff040c85f0 2024-03-04T08:03:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff0ba00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff040c9b30 con 0x7eff040b20b0 2024-03-04T08:03:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff0ba00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff040c12a0 con 0x7eff0406b060 2024-03-04T08:03:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff09000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7eff040b20b0 0x7eff040c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:10.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff03e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7eff040c85f0 0x7eff04157710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44076/0 (socket says 172.21.15.113:44076) 2024-03-04T08:03:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff03e00640 1 -- 172.21.15.113:0/2488637093 learned_addr learned my addr 172.21.15.113:0/2488637093 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:03:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff09a00640 1 --2- 172.21.15.113:0/2488637093 >> v2:172.21.15.136:3300/0 conn(0x7eff0406b060 0x7eff0406b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff09000640 1 -- 172.21.15.113:0/2488637093 >> v2:172.21.15.136:3300/0 conn(0x7eff0406b060 msgr2=0x7eff0406b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:03:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff09000640 1 --2- 172.21.15.113:0/2488637093 >> v2:172.21.15.136:3300/0 conn(0x7eff0406b060 0x7eff0406b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff09000640 1 -- 172.21.15.113:0/2488637093 >> v1:172.21.15.113:6789/0 conn(0x7eff040c85f0 legacy=0x7eff04157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff09000640 1 -- 172.21.15.113:0/2488637093 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff040c1580 con 0x7eff040b20b0 2024-03-04T08:03:10.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.415+0000 7eff03400640 1 -- 172.21.15.113:0/2488637093 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904548417 0 0) 0x7eff040c1ec0 con 0x7eff040c85f0 2024-03-04T08:03:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff09000640 1 --2- 172.21.15.113:0/2488637093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 0x7eff040c8c70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7eff040c96d0 tx=0x7efef801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ae8e350b3c910a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff03400640 1 -- 172.21.15.113:0/2488637093 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef80b5070 con 0x7eff040b20b0 2024-03-04T08:03:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff03400640 1 -- 172.21.15.113:0/2488637093 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef80a8970 con 0x7eff040b20b0 2024-03-04T08:03:10.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff03400640 1 -- 172.21.15.113:0/2488637093 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef80a8d40 con 0x7eff040b20b0 2024-03-04T08:03:10.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff0ba00640 1 -- 172.21.15.113:0/2488637093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 msgr2=0x7eff040c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:10.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff0ba00640 1 --2- 172.21.15.113:0/2488637093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 0x7eff040c8c70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7eff040c96d0 tx=0x7efef801ec70 comp rx=0 tx=0).stop 2024-03-04T08:03:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff0ba00640 1 -- 172.21.15.113:0/2488637093 shutdown_connections 2024-03-04T08:03:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff0ba00640 1 --2- 172.21.15.113:0/2488637093 >> v2:172.21.15.136:3300/0 conn(0x7eff0406b060 0x7eff0406b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff0ba00640 1 --2- 172.21.15.113:0/2488637093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 0x7eff040c8c70 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff0ba00640 1 -- 172.21.15.113:0/2488637093 >> 172.21.15.113:0/2488637093 conn(0x7eff0406e820 msgr2=0x7eff0406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff0ba00640 1 -- 172.21.15.113:0/2488637093 shutdown_connections 2024-03-04T08:03:10.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff0ba00640 1 -- 172.21.15.113:0/2488637093 wait complete. 2024-03-04T08:03:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff0ba00640 1 Processor -- start 2024-03-04T08:03:10.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.419+0000 7eff0ba00640 1 -- start start 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff0ba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0406b060 0x7eff041f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff0ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 0x7eff041f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff0ba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff040c85f0 0x7eff041499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff0ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff04157e20 con 0x7eff040b20b0 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff0ba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff041571f0 con 0x7eff0406b060 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff0ba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff040c2400 con 0x7eff040c85f0 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff03e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 0x7eff041f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff03e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 0x7eff041f1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40914/0 (socket says 172.21.15.113:40914) 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff03e00640 1 -- 172.21.15.113:0/4066318659 learned_addr learned my addr 172.21.15.113:0/4066318659 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff09000640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0406b060 0x7eff041f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:10.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff09a00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff040c85f0 0x7eff041499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:10.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff03e00640 1 -- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff040c85f0 msgr2=0x7eff041499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:10.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff03e00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff040c85f0 0x7eff041499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:10.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff03e00640 1 -- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0406b060 msgr2=0x7eff041f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:10.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff03e00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0406b060 0x7eff041f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:10.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff03e00640 1 -- 172.21.15.113:0/4066318659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff041e2630 con 0x7eff040b20b0 2024-03-04T08:03:10.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff03e00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 0x7eff041f1ec0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efef00011f0 tx=0x7efef000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:10.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff00c00640 1 -- 172.21.15.113:0/4066318659 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0018070 con 0x7eff040b20b0 2024-03-04T08:03:10.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff00c00640 1 -- 172.21.15.113:0/4066318659 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef00074e0 con 0x7eff040b20b0 2024-03-04T08:03:10.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff0ba00640 1 -- 172.21.15.113:0/4066318659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff041e3380 con 0x7eff040b20b0 2024-03-04T08:03:10.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff00c00640 1 -- 172.21.15.113:0/4066318659 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0004a10 con 0x7eff040b20b0 2024-03-04T08:03:10.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff0ba00640 1 -- 172.21.15.113:0/4066318659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff041e3620 con 0x7eff040b20b0 2024-03-04T08:03:10.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.423+0000 7eff0ba00640 1 -- 172.21.15.113:0/4066318659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff04075c00 con 0x7eff040b20b0 2024-03-04T08:03:10.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.427+0000 7eff00c00640 1 -- 172.21.15.113:0/4066318659 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efef001b070 con 0x7eff040b20b0 2024-03-04T08:03:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.427+0000 7eff00c00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efee4066f90 0x7efee4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.427+0000 7eff00c00640 1 -- 172.21.15.113:0/4066318659 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7efef00847f0 con 0x7eff040b20b0 2024-03-04T08:03:10.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.427+0000 7eff09000640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efee4066f90 0x7efee4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:10.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.427+0000 7eff09000640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efee4066f90 0x7efee4069450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efef80b2000 tx=0x7efef80aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:10.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.435+0000 7eff00c00640 1 -- 172.21.15.113:0/4066318659 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efef0054bd0 con 0x7eff040b20b0 2024-03-04T08:03:10.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.631+0000 7eff0ba00640 1 -- 172.21.15.113:0/4066318659 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff040c83d0 con 0x7eff040b20b0 2024-03-04T08:03:10.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.631+0000 7eff00c00640 1 -- 172.21.15.113:0/4066318659 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+5823 (secure 0 0 0) 0x7efef0058c70 con 0x7eff040b20b0 2024-03-04T08:03:10.629 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:10.629 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:07.241830+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6807,6534]},"id":1}]} 2024-03-04T08:03:10.629 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 110 2024-03-04T08:03:10.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 -- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efee4066f90 msgr2=0x7efee4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:10.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efee4066f90 0x7efee4069450 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efef80b2000 tx=0x7efef80aa070 comp rx=0 tx=0).stop 2024-03-04T08:03:10.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 -- 172.21.15.113:0/4066318659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 msgr2=0x7eff041f1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:10.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 0x7eff041f1ec0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7efef00011f0 tx=0x7efef000c9b0 comp rx=0 tx=0).stop 2024-03-04T08:03:10.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 -- 172.21.15.113:0/4066318659 shutdown_connections 2024-03-04T08:03:10.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7eff040c85f0 0x7eff041499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:10.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efee4066f90 0x7efee4069450 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:10.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff040b20b0 0x7eff041f1ec0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:10.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 --2- 172.21.15.113:0/4066318659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff0406b060 0x7eff041f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:10.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 -- 172.21.15.113:0/4066318659 >> 172.21.15.113:0/4066318659 conn(0x7eff0406e820 msgr2=0x7eff04157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:10.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 -- 172.21.15.113:0/4066318659 shutdown_connections 2024-03-04T08:03:10.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:10.635+0000 7eff0ba00640 1 -- 172.21.15.113:0/4066318659 wait complete. 2024-03-04T08:03:10.651 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 110, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 110, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6534": { "addr": "172.21.15.113:6837/4197507712", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 4197507712, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 4197507712, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6534, "incarnation": 89, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 24 }, "gid_6807": { "addr": "172.21.15.113:6836/1035982016", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1035982016, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1035982016, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6807, "incarnation": 101, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 14 }, "gid_6925": { "addr": "172.21.15.136:6835/2522634751", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2522634751, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2522634751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6925, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:rejoin", "state_seq": 19 } }, "last_failure": 0, "last_failure_osd_epoch": 64, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:03:07.241830+0000", "qdb_cluster": [ 6807, 6534 ], "qdb_leader": 6534, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6925, "mds_1": 6534, "mds_2": 6807 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:03:12.652 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:12.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb7000640 1 Processor -- start 2024-03-04T08:03:12.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb7000640 1 -- start start 2024-03-04T08:03:12.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb7000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb0152920 0x7f5eb0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:12.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb7000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5eb006c580 0x7f5eb00c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:12.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb7000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb006bb10 con 0x7f5eb015bf20 2024-03-04T08:03:12.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb7000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb00c7a40 con 0x7f5eb006c580 2024-03-04T08:03:12.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb7000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00c7d20 con 0x7f5eb0152920 2024-03-04T08:03:12.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb6600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5eb015bf20 0x7f5eb015c3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44084/0 (socket says 172.21.15.113:44084) 2024-03-04T08:03:12.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb6600640 1 -- 172.21.15.113:0/2795396822 learned_addr learned my addr 172.21.15.113:0/2795396822 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:03:12.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb5c00640 1 --2- 172.21.15.113:0/2795396822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb0152920 0x7f5eb0071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:12.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb5200640 1 --2- 172.21.15.113:0/2795396822 >> v2:172.21.15.136:3300/0 conn(0x7f5eb006c580 0x7f5eb00c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:12.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eafe00640 1 -- 172.21.15.113:0/2795396822 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686363367 0 0) 0x7f5eb006bb10 con 0x7f5eb015bf20 2024-03-04T08:03:12.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eafe00640 1 -- 172.21.15.113:0/2795396822 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e98003530 con 0x7f5eb015bf20 2024-03-04T08:03:12.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb5c00640 1 -- 172.21.15.113:0/2795396822 >> v2:172.21.15.136:3300/0 conn(0x7f5eb006c580 msgr2=0x7f5eb00c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:03:12.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eafe00640 1 -- 172.21.15.113:0/2795396822 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3295224156 0 0) 0x7f5e98003530 con 0x7f5eb015bf20 2024-03-04T08:03:12.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb5c00640 1 --2- 172.21.15.113:0/2795396822 >> v2:172.21.15.136:3300/0 conn(0x7f5eb006c580 0x7f5eb00c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:12.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb5c00640 1 -- 172.21.15.113:0/2795396822 >> v1:172.21.15.113:6789/0 conn(0x7f5eb015bf20 legacy=0x7f5eb015c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:12.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb5c00640 1 -- 172.21.15.113:0/2795396822 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb006bd80 con 0x7f5eb0152920 2024-03-04T08:03:12.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.795+0000 7f5eb5200640 1 --2- 172.21.15.113:0/2795396822 >> v2:172.21.15.136:3300/0 conn(0x7f5eb006c580 0x7f5eb00c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:03:12.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.799+0000 7f5eb5c00640 1 --2- 172.21.15.113:0/2795396822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb0152920 0x7f5eb0071a30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40097c0 tx=0x7f5ea401c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec99ef432a0886a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:12.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eafe00640 1 -- 172.21.15.113:0/2795396822 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40b0070 con 0x7f5eb0152920 2024-03-04T08:03:12.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eafe00640 1 -- 172.21.15.113:0/2795396822 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea40ab400 con 0x7f5eb0152920 2024-03-04T08:03:12.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eafe00640 1 -- 172.21.15.113:0/2795396822 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8001a00 con 0x7f5eb0152920 2024-03-04T08:03:12.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eb7000640 1 -- 172.21.15.113:0/2795396822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb0152920 msgr2=0x7f5eb0071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:12.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eb7000640 1 --2- 172.21.15.113:0/2795396822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb0152920 0x7f5eb0071a30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40097c0 tx=0x7f5ea401c300 comp rx=0 tx=0).stop 2024-03-04T08:03:12.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eb7000640 1 -- 172.21.15.113:0/2795396822 shutdown_connections 2024-03-04T08:03:12.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eb7000640 1 --2- 172.21.15.113:0/2795396822 >> v2:172.21.15.136:3300/0 conn(0x7f5eb006c580 0x7f5eb00c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:12.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eb7000640 1 --2- 172.21.15.113:0/2795396822 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb0152920 0x7f5eb0071a30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:12.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eb7000640 1 -- 172.21.15.113:0/2795396822 >> 172.21.15.113:0/2795396822 conn(0x7f5eb00c9a30 msgr2=0x7f5eb00c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:12.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eb7000640 1 -- 172.21.15.113:0/2795396822 shutdown_connections 2024-03-04T08:03:12.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eb7000640 1 -- 172.21.15.113:0/2795396822 wait complete. 2024-03-04T08:03:12.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eb7000640 1 Processor -- start 2024-03-04T08:03:12.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.803+0000 7f5eb7000640 1 -- start start 2024-03-04T08:03:12.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb7000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5eb006c580 0x7f5eb00c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:12.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb7000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb0152920 0x7f5eb01eaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:12.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb7000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb015bf20 0x7f5eb01f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:12.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb7000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00c95a0 con 0x7f5eb0152920 2024-03-04T08:03:12.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb7000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb0152d00 con 0x7f5eb006c580 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb7000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00c9720 con 0x7f5eb015bf20 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb0152920 0x7f5eb01eaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb5200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb0152920 0x7f5eb01eaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40928/0 (socket says 172.21.15.113:40928) 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb5200640 1 -- 172.21.15.113:0/179241842 learned_addr learned my addr 172.21.15.113:0/179241842 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb6600640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb015bf20 0x7f5eb01f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb5c00640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5eb006c580 0x7f5eb00c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb5200640 1 -- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb015bf20 msgr2=0x7f5eb01f4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb5200640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb015bf20 0x7f5eb01f4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb5200640 1 -- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5eb006c580 msgr2=0x7f5eb00c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb5200640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5eb006c580 0x7f5eb00c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb5200640 1 -- 172.21.15.113:0/179241842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb01eb310 con 0x7f5eb0152920 2024-03-04T08:03:12.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb5200640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb0152920 0x7f5eb01eaa40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00ebb0 tx=0x7f5e9c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5ead600640 1 -- 172.21.15.113:0/179241842 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c017070 con 0x7f5eb0152920 2024-03-04T08:03:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5ead600640 1 -- 172.21.15.113:0/179241842 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c00cc30 con 0x7f5eb0152920 2024-03-04T08:03:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb7000640 1 -- 172.21.15.113:0/179241842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb00c6750 con 0x7f5eb0152920 2024-03-04T08:03:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5ead600640 1 -- 172.21.15.113:0/179241842 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c013470 con 0x7f5eb0152920 2024-03-04T08:03:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb7000640 1 -- 172.21.15.113:0/179241842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb01eb5b0 con 0x7f5eb0152920 2024-03-04T08:03:12.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.807+0000 7f5eb7000640 1 -- 172.21.15.113:0/179241842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb0075a80 con 0x7f5eb0152920 2024-03-04T08:03:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.811+0000 7f5ead600640 1 -- 172.21.15.113:0/179241842 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e9c01a070 con 0x7f5eb0152920 2024-03-04T08:03:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.811+0000 7f5ead600640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e90066f60 0x7f5e90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.811+0000 7f5ead600640 1 -- 172.21.15.113:0/179241842 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f5e9c0856f0 con 0x7f5eb0152920 2024-03-04T08:03:12.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.811+0000 7f5eb5c00640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e90066f60 0x7f5e90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:12.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.811+0000 7f5eb5c00640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e90066f60 0x7f5e90069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40ad000 tx=0x7f5ea40a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:12.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:12.815+0000 7f5ead600640 1 -- 172.21.15.113:0/179241842 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5e9c055f70 con 0x7f5eb0152920 2024-03-04T08:03:13.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.011+0000 7f5eb7000640 1 -- 172.21.15.113:0/179241842 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5eb00c8300 con 0x7f5eb0152920 2024-03-04T08:03:13.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5ead600640 1 -- 172.21.15.113:0/179241842 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f5e9c055870 con 0x7f5eb0152920 2024-03-04T08:03:13.010 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:13.010 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:07.241830+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6807,6534]},"id":1}]} 2024-03-04T08:03:13.011 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 110 2024-03-04T08:03:13.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 -- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e90066f60 msgr2=0x7f5e90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:13.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e90066f60 0x7f5e90069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5ea40ad000 tx=0x7f5ea40a6000 comp rx=0 tx=0).stop 2024-03-04T08:03:13.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 -- 172.21.15.113:0/179241842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb0152920 msgr2=0x7f5eb01eaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:13.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb0152920 0x7f5eb01eaa40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00ebb0 tx=0x7f5e9c00c520 comp rx=0 tx=0).stop 2024-03-04T08:03:13.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 -- 172.21.15.113:0/179241842 shutdown_connections 2024-03-04T08:03:13.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5eb015bf20 0x7f5eb01f4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:13.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5e90066f60 0x7f5e90069420 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:13.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5eb0152920 0x7f5eb01eaa40 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:13.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 --2- 172.21.15.113:0/179241842 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5eb006c580 0x7f5eb00c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:13.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 -- 172.21.15.113:0/179241842 >> 172.21.15.113:0/179241842 conn(0x7f5eb00c9a30 msgr2=0x7f5eb00cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:13.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 -- 172.21.15.113:0/179241842 shutdown_connections 2024-03-04T08:03:13.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:13.015+0000 7f5eb7000640 1 -- 172.21.15.113:0/179241842 wait complete. 2024-03-04T08:03:13.040 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:03:15.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:15.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.183+0000 7f30b5000640 1 Processor -- start 2024-03-04T08:03:15.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.183+0000 7f30b5000640 1 -- start start 2024-03-04T08:03:15.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30b5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b00c8200 0x7f30b0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:15.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30b5000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f30b0152bd0 0x7f30b01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:15.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30b5000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b01591f0 con 0x7f30b01527f0 2024-03-04T08:03:15.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30b5000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b0071df0 con 0x7f30b0152bd0 2024-03-04T08:03:15.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30b5000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b01597b0 con 0x7f30b00c8200 2024-03-04T08:03:15.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30ade00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f30b01527f0 0x7f30b0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44094/0 (socket says 172.21.15.113:44094) 2024-03-04T08:03:15.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30ade00640 1 -- 172.21.15.113:0/3281725696 learned_addr learned my addr 172.21.15.113:0/3281725696 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:03:15.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30ae800640 1 --2- 172.21.15.113:0/3281725696 >> v2:172.21.15.113:3300/0 conn(0x7f30b0152bd0 0x7f30b01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:15.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30a7e00640 1 --2- 172.21.15.113:0/3281725696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b00c8200 0x7f30b0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:15.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30ad400640 1 -- 172.21.15.113:0/3281725696 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171197544 0 0) 0x7f30b01591f0 con 0x7f30b01527f0 2024-03-04T08:03:15.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30ad400640 1 -- 172.21.15.113:0/3281725696 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3094003530 con 0x7f30b01527f0 2024-03-04T08:03:15.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30ae800640 1 -- 172.21.15.113:0/3281725696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b00c8200 msgr2=0x7f30b0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:03:15.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30ae800640 1 --2- 172.21.15.113:0/3281725696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b00c8200 0x7f30b0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:15.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30ae800640 1 -- 172.21.15.113:0/3281725696 >> v1:172.21.15.113:6789/0 conn(0x7f30b01527f0 legacy=0x7f30b0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:15.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.187+0000 7f30ae800640 1 -- 172.21.15.113:0/3281725696 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b01594d0 con 0x7f30b0152bd0 2024-03-04T08:03:15.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30ae800640 1 --2- 172.21.15.113:0/3281725696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b0152bd0 0x7f30b01587d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009d90 tx=0x7f30a801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9872bf50cbf986 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:15.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30ad400640 1 -- 172.21.15.113:0/3281725696 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80b2070 con 0x7f30b0152bd0 2024-03-04T08:03:15.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30ad400640 1 -- 172.21.15.113:0/3281725696 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a80a5720 con 0x7f30b0152bd0 2024-03-04T08:03:15.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30ad400640 1 -- 172.21.15.113:0/3281725696 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a80a5af0 con 0x7f30b0152bd0 2024-03-04T08:03:15.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30b5000640 1 -- 172.21.15.113:0/3281725696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b0152bd0 msgr2=0x7f30b01587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:15.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30b5000640 1 --2- 172.21.15.113:0/3281725696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b0152bd0 0x7f30b01587d0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f30a8009d90 tx=0x7f30a801c970 comp rx=0 tx=0).stop 2024-03-04T08:03:15.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30b5000640 1 -- 172.21.15.113:0/3281725696 shutdown_connections 2024-03-04T08:03:15.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30b5000640 1 --2- 172.21.15.113:0/3281725696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b0152bd0 0x7f30b01587d0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:15.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30b5000640 1 --2- 172.21.15.113:0/3281725696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b00c8200 0x7f30b0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:15.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30b5000640 1 -- 172.21.15.113:0/3281725696 >> 172.21.15.113:0/3281725696 conn(0x7f30b006e820 msgr2=0x7f30b006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:15.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30b5000640 1 -- 172.21.15.113:0/3281725696 shutdown_connections 2024-03-04T08:03:15.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.191+0000 7f30b5000640 1 -- 172.21.15.113:0/3281725696 wait complete. 2024-03-04T08:03:15.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30b5000640 1 Processor -- start 2024-03-04T08:03:15.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30b5000640 1 -- start start 2024-03-04T08:03:15.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30b5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b00c8200 0x7f30b015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:15.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30b5000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b01527f0 0x7f30b014b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:15.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30b5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b0152bd0 0x7f30b0150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30b5000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b01634e0 con 0x7f30b00c8200 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30b5000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b0163360 con 0x7f30b01527f0 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30b5000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b0163660 con 0x7f30b0152bd0 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30ade00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b00c8200 0x7f30b015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30ade00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b00c8200 0x7f30b015a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40952/0 (socket says 172.21.15.113:40952) 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30ade00640 1 -- 172.21.15.113:0/747684516 learned_addr learned my addr 172.21.15.113:0/747684516 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30ae800640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b0152bd0 0x7f30b0150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30a7e00640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b01527f0 0x7f30b014b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30ade00640 1 -- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b0152bd0 msgr2=0x7f30b0150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30ade00640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b0152bd0 0x7f30b0150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30ade00640 1 -- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b01527f0 msgr2=0x7f30b014b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30ade00640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b01527f0 0x7f30b014b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30ade00640 1 -- 172.21.15.113:0/747684516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b014e5a0 con 0x7f30b00c8200 2024-03-04T08:03:15.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30a7e00640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b01527f0 0x7f30b014b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:03:15.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30ade00640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b00c8200 0x7f30b015a550 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f30a000c450 tx=0x7f30a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:15.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30a6000640 1 -- 172.21.15.113:0/747684516 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0017070 con 0x7f30b00c8200 2024-03-04T08:03:15.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30a6000640 1 -- 172.21.15.113:0/747684516 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a0005cf0 con 0x7f30b00c8200 2024-03-04T08:03:15.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30a6000640 1 -- 172.21.15.113:0/747684516 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0007290 con 0x7f30b00c8200 2024-03-04T08:03:15.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30b5000640 1 -- 172.21.15.113:0/747684516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b015b160 con 0x7f30b00c8200 2024-03-04T08:03:15.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.195+0000 7f30b5000640 1 -- 172.21.15.113:0/747684516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b015b720 con 0x7f30b00c8200 2024-03-04T08:03:15.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.199+0000 7f30b5000640 1 -- 172.21.15.113:0/747684516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b0075c00 con 0x7f30b00c8200 2024-03-04T08:03:15.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.199+0000 7f30a6000640 1 -- 172.21.15.113:0/747684516 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30a001a070 con 0x7f30b00c8200 2024-03-04T08:03:15.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.199+0000 7f30a6000640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f308c066f60 0x7f308c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:15.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.199+0000 7f30a6000640 1 -- 172.21.15.113:0/747684516 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f30a0084a30 con 0x7f30b00c8200 2024-03-04T08:03:15.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.199+0000 7f30a7e00640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f308c066f60 0x7f308c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:15.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.199+0000 7f30a7e00640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f308c066f60 0x7f308c069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f30b015cd70 tx=0x7f309800f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:15.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.207+0000 7f30a6000640 1 -- 172.21.15.113:0/747684516 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f30a0055450 con 0x7f30b00c8200 2024-03-04T08:03:15.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.459+0000 7f30b5000640 1 -- 172.21.15.113:0/747684516 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30b00ae7d0 con 0x7f30b00c8200 2024-03-04T08:03:15.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.459+0000 7f30a6000640 1 -- 172.21.15.113:0/747684516 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5829 (secure 0 0 0) 0x7f30a0054d50 con 0x7f30b00c8200 2024-03-04T08:03:15.457 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:15.457 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:13.362835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:clientreplay","state_seq":21,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6534,6807]},"id":1}]} 2024-03-04T08:03:15.457 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 111 2024-03-04T08:03:15.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 -- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f308c066f60 msgr2=0x7f308c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:15.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f308c066f60 0x7f308c069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f30b015cd70 tx=0x7f309800f040 comp rx=0 tx=0).stop 2024-03-04T08:03:15.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 -- 172.21.15.113:0/747684516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b00c8200 msgr2=0x7f30b015a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:15.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b00c8200 0x7f30b015a550 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f30a000c450 tx=0x7f30a000c910 comp rx=0 tx=0).stop 2024-03-04T08:03:15.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 -- 172.21.15.113:0/747684516 shutdown_connections 2024-03-04T08:03:15.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30b0152bd0 0x7f30b0150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:15.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30b01527f0 0x7f30b014b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:15.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f308c066f60 0x7f308c069420 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:15.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 --2- 172.21.15.113:0/747684516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30b00c8200 0x7f30b015a550 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:15.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 -- 172.21.15.113:0/747684516 >> 172.21.15.113:0/747684516 conn(0x7f30b006e820 msgr2=0x7f30b01520b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:15.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 -- 172.21.15.113:0/747684516 shutdown_connections 2024-03-04T08:03:15.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:15.463+0000 7f30b5000640 1 -- 172.21.15.113:0/747684516 wait complete. 2024-03-04T08:03:15.478 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:03:17.479 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:17.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facb1e00640 1 Processor -- start 2024-03-04T08:03:17.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facb1e00640 1 -- start start 2024-03-04T08:03:17.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facb1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 0x7facac15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facb1e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7facac06de80 0x7facac154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facb1e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facac06c210 con 0x7facac0ce6b0 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facb1e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facac15bbf0 con 0x7facac06de80 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facb1e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac06eb80 con 0x7facac0ce2d0 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facaaa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 0x7facac15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facab400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7facac06de80 0x7facac154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facaaa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 0x7facac15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:44724/0 (socket says 172.21.15.113:44724) 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facaaa00640 1 -- 172.21.15.113:0/3466739633 learned_addr learned my addr 172.21.15.113:0/3466739633 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facaaa00640 1 -- 172.21.15.113:0/3466739633 >> v2:172.21.15.136:3300/0 conn(0x7facac06de80 msgr2=0x7facac154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facaaa00640 1 --2- 172.21.15.113:0/3466739633 >> v2:172.21.15.136:3300/0 conn(0x7facac06de80 0x7facac154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facaaa00640 1 -- 172.21.15.113:0/3466739633 >> v1:172.21.15.136:6789/0 conn(0x7facac0ce6b0 legacy=0x7facac06f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:17.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.623+0000 7facaaa00640 1 -- 172.21.15.113:0/3466739633 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac06c4f0 con 0x7facac0ce2d0 2024-03-04T08:03:17.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facaaa00640 1 --2- 172.21.15.113:0/3466739633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 0x7facac15b6b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009870 tx=0x7fac9c01ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f320fc3a2c36bf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7faca9600640 1 -- 172.21.15.113:0/3466739633 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0b5070 con 0x7facac0ce2d0 2024-03-04T08:03:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7faca9600640 1 -- 172.21.15.113:0/3466739633 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c0a8a50 con 0x7facac0ce2d0 2024-03-04T08:03:17.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7faca9600640 1 -- 172.21.15.113:0/3466739633 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0a8e20 con 0x7facac0ce2d0 2024-03-04T08:03:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facb1e00640 1 -- 172.21.15.113:0/3466739633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 msgr2=0x7facac15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facb1e00640 1 --2- 172.21.15.113:0/3466739633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 0x7facac15b6b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009870 tx=0x7fac9c01ed50 comp rx=0 tx=0).stop 2024-03-04T08:03:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facb1e00640 1 -- 172.21.15.113:0/3466739633 shutdown_connections 2024-03-04T08:03:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facb1e00640 1 --2- 172.21.15.113:0/3466739633 >> v2:172.21.15.136:3300/0 conn(0x7facac06de80 0x7facac154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facb1e00640 1 --2- 172.21.15.113:0/3466739633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 0x7facac15b6b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facb1e00640 1 -- 172.21.15.113:0/3466739633 >> 172.21.15.113:0/3466739633 conn(0x7facac0709a0 msgr2=0x7facac070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facb1e00640 1 -- 172.21.15.113:0/3466739633 shutdown_connections 2024-03-04T08:03:17.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facb1e00640 1 -- 172.21.15.113:0/3466739633 wait complete. 2024-03-04T08:03:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facb1e00640 1 Processor -- start 2024-03-04T08:03:17.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.631+0000 7facb1e00640 1 -- start start 2024-03-04T08:03:17.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facb1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facac06de80 0x7facac1f1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:17.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facb1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 0x7facac0c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:17.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facb1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0ce6b0 0x7facac154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:17.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facb1e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac06c7d0 con 0x7facac06de80 2024-03-04T08:03:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facb1e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac06e810 con 0x7facac0ce6b0 2024-03-04T08:03:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facb1e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac06f720 con 0x7facac0ce2d0 2024-03-04T08:03:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facaaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facac06de80 0x7facac1f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facaaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facac06de80 0x7facac1f1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34252/0 (socket says 172.21.15.113:34252) 2024-03-04T08:03:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facaaa00640 1 -- 172.21.15.113:0/675198222 learned_addr learned my addr 172.21.15.113:0/675198222 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facaa000640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 0x7facac0c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:17.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facab400640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0ce6b0 0x7facac154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facaaa00640 1 -- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 msgr2=0x7facac0c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facaaa00640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 0x7facac0c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facaaa00640 1 -- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0ce6b0 msgr2=0x7facac154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facaaa00640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0ce6b0 0x7facac154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facaaa00640 1 -- 172.21.15.113:0/675198222 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac1f4e30 con 0x7facac06de80 2024-03-04T08:03:17.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facaaa00640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facac06de80 0x7facac1f1480 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0b3040 tx=0x7fac9c01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:17.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7faca3400640 1 -- 172.21.15.113:0/675198222 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0b5070 con 0x7facac06de80 2024-03-04T08:03:17.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7faca3400640 1 -- 172.21.15.113:0/675198222 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c0b0de0 con 0x7facac06de80 2024-03-04T08:03:17.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facb1e00640 1 -- 172.21.15.113:0/675198222 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac0c6c20 con 0x7facac06de80 2024-03-04T08:03:17.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7faca3400640 1 -- 172.21.15.113:0/675198222 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c0bb670 con 0x7facac06de80 2024-03-04T08:03:17.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facb1e00640 1 -- 172.21.15.113:0/675198222 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facac0cd880 con 0x7facac06de80 2024-03-04T08:03:17.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7facb1e00640 1 -- 172.21.15.113:0/675198222 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facac075c00 con 0x7facac06de80 2024-03-04T08:03:17.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.635+0000 7faca3400640 1 -- 172.21.15.113:0/675198222 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac9c0c2060 con 0x7facac06de80 2024-03-04T08:03:17.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.639+0000 7faca3400640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac8c066f60 0x7fac8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:17.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.639+0000 7faca3400640 1 -- 172.21.15.113:0/675198222 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7fac9c122bb0 con 0x7facac06de80 2024-03-04T08:03:17.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.639+0000 7facaa000640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac8c066f60 0x7fac8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:17.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.639+0000 7facaa000640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac8c066f60 0x7fac8c069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7facac0cead0 tx=0x7fac98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:17.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.643+0000 7faca3400640 1 -- 172.21.15.113:0/675198222 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fac9c0188b0 con 0x7facac06de80 2024-03-04T08:03:17.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.831+0000 7facb1e00640 1 -- 172.21.15.113:0/675198222 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facac06cb40 con 0x7facac06de80 2024-03-04T08:03:17.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.831+0000 7faca3400640 1 -- 172.21.15.113:0/675198222 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5829 (secure 0 0 0) 0x7fac9c0f3560 con 0x7facac06de80 2024-03-04T08:03:17.830 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:17.830 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:13.362835+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:clientreplay","state_seq":21,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6534,"qdb_cluster":[6534,6807]},"id":1}]} 2024-03-04T08:03:17.830 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 111 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 -- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac8c066f60 msgr2=0x7fac8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac8c066f60 0x7fac8c069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7facac0cead0 tx=0x7fac98009040 comp rx=0 tx=0).stop 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 -- 172.21.15.113:0/675198222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facac06de80 msgr2=0x7facac1f1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facac06de80 0x7facac1f1480 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fac9c0b3040 tx=0x7fac9c01baf0 comp rx=0 tx=0).stop 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 -- 172.21.15.113:0/675198222 shutdown_connections 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7facac0ce6b0 0x7facac154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7facac0ce2d0 0x7facac0c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac8c066f60 0x7fac8c069420 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 --2- 172.21.15.113:0/675198222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7facac06de80 0x7facac1f1480 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 -- 172.21.15.113:0/675198222 >> 172.21.15.113:0/675198222 conn(0x7facac0709a0 msgr2=0x7facac06f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 -- 172.21.15.113:0/675198222 shutdown_connections 2024-03-04T08:03:17.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:17.835+0000 7facb1e00640 1 -- 172.21.15.113:0/675198222 wait complete. 2024-03-04T08:03:17.850 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:03:19.851 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:20.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.019+0000 7fe59dc00640 1 Processor -- start 2024-03-04T08:03:20.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.019+0000 7fe59dc00640 1 -- start start 2024-03-04T08:03:20.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe59dc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5981527f0 0x7fe598070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:20.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe59dc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5981591f0 con 0x7fe598152bd0 2024-03-04T08:03:20.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe59dc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5981594d0 con 0x7fe5980c8200 2024-03-04T08:03:20.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe59dc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe598071df0 con 0x7fe5981527f0 2024-03-04T08:03:20.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe596a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5981527f0 0x7fe598070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:20.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe597400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe598152bd0 0x7fe5981587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35834/0 (socket says 172.21.15.113:35834) 2024-03-04T08:03:20.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe596a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5981527f0 0x7fe598070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34256/0 (socket says 172.21.15.113:34256) 2024-03-04T08:03:20.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe597400640 1 -- 172.21.15.113:0/2736601173 learned_addr learned my addr 172.21.15.113:0/2736601173 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:03:20.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe596a00640 1 -- 172.21.15.113:0/2736601173 >> v1:172.21.15.136:6789/0 conn(0x7fe5980c8200 legacy=0x7fe598072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:20.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe596a00640 1 -- 172.21.15.113:0/2736601173 >> v1:172.21.15.113:6789/0 conn(0x7fe598152bd0 legacy=0x7fe5981587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:20.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.023+0000 7fe596a00640 1 -- 172.21.15.113:0/2736601173 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5980717a0 con 0x7fe5981527f0 2024-03-04T08:03:20.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe596a00640 1 --2- 172.21.15.113:0/2736601173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5981527f0 0x7fe598070d40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe5880097c0 tx=0x7fe58801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f75e842ce23cc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:20.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe596000640 1 -- 172.21.15.113:0/2736601173 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5880b0070 con 0x7fe5981527f0 2024-03-04T08:03:20.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe596000640 1 -- 172.21.15.113:0/2736601173 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5880ab600 con 0x7fe5981527f0 2024-03-04T08:03:20.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe596000640 1 -- 172.21.15.113:0/2736601173 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5880ab9d0 con 0x7fe5981527f0 2024-03-04T08:03:20.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe59dc00640 1 -- 172.21.15.113:0/2736601173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5981527f0 msgr2=0x7fe598070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:20.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe59dc00640 1 --2- 172.21.15.113:0/2736601173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5981527f0 0x7fe598070d40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe5880097c0 tx=0x7fe58801c330 comp rx=0 tx=0).stop 2024-03-04T08:03:20.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe59dc00640 1 -- 172.21.15.113:0/2736601173 shutdown_connections 2024-03-04T08:03:20.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe59dc00640 1 --2- 172.21.15.113:0/2736601173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5981527f0 0x7fe598070d40 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:20.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe59dc00640 1 -- 172.21.15.113:0/2736601173 >> 172.21.15.113:0/2736601173 conn(0x7fe59806e820 msgr2=0x7fe59806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:20.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe59dc00640 1 -- 172.21.15.113:0/2736601173 shutdown_connections 2024-03-04T08:03:20.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe59dc00640 1 -- 172.21.15.113:0/2736601173 wait complete. 2024-03-04T08:03:20.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe59dc00640 1 Processor -- start 2024-03-04T08:03:20.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.027+0000 7fe59dc00640 1 -- start start 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe59dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5980c8200 0x7fe5981567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe59dc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5981527f0 0x7fe598157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe59dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe598152bd0 0x7fe59814c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe59dc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe598158ee0 con 0x7fe598152bd0 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe59dc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5981597b0 con 0x7fe5980c8200 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe59dc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59806b060 con 0x7fe5981527f0 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe597400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe598152bd0 0x7fe59814c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe597400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe598152bd0 0x7fe59814c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34264/0 (socket says 172.21.15.113:34264) 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe597400640 1 -- 172.21.15.113:0/1668927435 learned_addr learned my addr 172.21.15.113:0/1668927435 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe596a00640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5980c8200 0x7fe5981567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:20.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe58e000640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5981527f0 0x7fe598157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe597400640 1 -- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5981527f0 msgr2=0x7fe598157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe597400640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5981527f0 0x7fe598157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe597400640 1 -- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5980c8200 msgr2=0x7fe5981567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe597400640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5980c8200 0x7fe5981567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe597400640 1 -- 172.21.15.113:0/1668927435 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe598155ec0 con 0x7fe598152bd0 2024-03-04T08:03:20.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe597400640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe598152bd0 0x7fe59814c9b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe590009e00 tx=0x7fe59000c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe58f400640 1 -- 172.21.15.113:0/1668927435 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590019070 con 0x7fe598152bd0 2024-03-04T08:03:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe58f400640 1 -- 172.21.15.113:0/1668927435 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5900078f0 con 0x7fe598152bd0 2024-03-04T08:03:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe59dc00640 1 -- 172.21.15.113:0/1668927435 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe598153980 con 0x7fe598152bd0 2024-03-04T08:03:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe58f400640 1 -- 172.21.15.113:0/1668927435 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590007ce0 con 0x7fe598152bd0 2024-03-04T08:03:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe59dc00640 1 -- 172.21.15.113:0/1668927435 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe598153f40 con 0x7fe598152bd0 2024-03-04T08:03:20.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.031+0000 7fe59dc00640 1 -- 172.21.15.113:0/1668927435 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe598075c00 con 0x7fe598152bd0 2024-03-04T08:03:20.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.035+0000 7fe58f400640 1 -- 172.21.15.113:0/1668927435 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe59001c070 con 0x7fe598152bd0 2024-03-04T08:03:20.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.035+0000 7fe58f400640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe574066f90 0x7fe574069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:20.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.035+0000 7fe58f400640 1 -- 172.21.15.113:0/1668927435 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7fe590087290 con 0x7fe598152bd0 2024-03-04T08:03:20.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.035+0000 7fe596a00640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe574066f90 0x7fe574069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:20.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.039+0000 7fe596a00640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe574066f90 0x7fe574069450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe5880ad000 tx=0x7fe588008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:20.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.039+0000 7fe58f400640 1 -- 172.21.15.113:0/1668927435 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe5900575d0 con 0x7fe598152bd0 2024-03-04T08:03:20.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 -- 172.21.15.113:0/1668927435 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5980c79d0 con 0x7fe598152bd0 2024-03-04T08:03:20.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe58f400640 1 -- 172.21.15.113:0/1668927435 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+5828 (secure 0 0 0) 0x7fe59005b670 con 0x7fe598152bd0 2024-03-04T08:03:20.247 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:20.247 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:19.898691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6534,6807,6925]},"id":1}]} 2024-03-04T08:03:20.247 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 112 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 -- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe574066f90 msgr2=0x7fe574069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe574066f90 0x7fe574069450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe5880ad000 tx=0x7fe588008040 comp rx=0 tx=0).stop 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 -- 172.21.15.113:0/1668927435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe598152bd0 msgr2=0x7fe59814c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe598152bd0 0x7fe59814c9b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fe590009e00 tx=0x7fe59000c430 comp rx=0 tx=0).stop 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 -- 172.21.15.113:0/1668927435 shutdown_connections 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe574066f90 0x7fe574069450 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe598152bd0 0x7fe59814c9b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5981527f0 0x7fe598157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 --2- 172.21.15.113:0/1668927435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5980c8200 0x7fe5981567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.251+0000 7fe59dc00640 1 -- 172.21.15.113:0/1668927435 >> 172.21.15.113:0/1668927435 conn(0x7fe59806e820 msgr2=0x7fe598151fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.255+0000 7fe59dc00640 1 -- 172.21.15.113:0/1668927435 shutdown_connections 2024-03-04T08:03:20.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:20.255+0000 7fe59dc00640 1 -- 172.21.15.113:0/1668927435 wait complete. 2024-03-04T08:03:20.278 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=0, replacing gid=6372 2024-03-04T08:03:20.278 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.c 2024-03-04T08:03:27.145 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T08:03:27.148+0000 7fd207b4d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T08:03:27.146 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T08:03:27.148+0000 7fd207b4d640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T08:03:27.146 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T08:03:27.148+0000 7fd207b4d640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-04T08:03:27.279 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-03-04T08:03:27.279 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-04T08:03:27.279 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-04T08:03:27.283 INFO:tasks.ceph.mds.c:Started 2024-03-04T08:03:27.283 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:27.371 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-03-04T08:03:27.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.447+0000 7f8f03400640 1 Processor -- start 2024-03-04T08:03:27.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.447+0000 7f8f03400640 1 -- start start 2024-03-04T08:03:27.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8f03400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc152d60 0x7f8efc0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:27.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8f03400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8efc0c5380 con 0x7f8efc070120 2024-03-04T08:03:27.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8f03400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8efc0c3440 con 0x7f8efc159040 2024-03-04T08:03:27.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8f03400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc06f620 con 0x7f8efc152d60 2024-03-04T08:03:27.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8efbe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8efc070120 0x7f8efc158930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43630/0 (socket says 172.21.15.113:43630) 2024-03-04T08:03:27.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8efbe00640 1 -- 172.21.15.113:0/2117910472 learned_addr learned my addr 172.21.15.113:0/2117910472 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:03:27.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8f00a00640 1 --2- 172.21.15.113:0/2117910472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc152d60 0x7f8efc0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:27.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8efb400640 1 -- 172.21.15.113:0/2117910472 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720061267 0 0) 0x7f8efc0c5380 con 0x7f8efc070120 2024-03-04T08:03:27.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8efb400640 1 -- 172.21.15.113:0/2117910472 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee4003530 con 0x7f8efc070120 2024-03-04T08:03:27.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8efb400640 1 -- 172.21.15.113:0/2117910472 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4105038833 0 0) 0x7f8efc0c3440 con 0x7f8efc159040 2024-03-04T08:03:27.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8efb400640 1 -- 172.21.15.113:0/2117910472 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8efc0c5380 con 0x7f8efc159040 2024-03-04T08:03:27.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8f00a00640 1 -- 172.21.15.113:0/2117910472 >> v1:172.21.15.136:6789/0 conn(0x7f8efc159040 legacy=0x7f8efc1632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:27.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8f00a00640 1 -- 172.21.15.113:0/2117910472 >> v1:172.21.15.113:6789/0 conn(0x7f8efc070120 legacy=0x7f8efc158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:27.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.451+0000 7f8f00a00640 1 -- 172.21.15.113:0/2117910472 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc0c36e0 con 0x7f8efc152d60 2024-03-04T08:03:27.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8f00a00640 1 --2- 172.21.15.113:0/2117910472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc152d60 0x7f8efc0ce310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00097f0 tx=0x7f8ef001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75535a199134d876 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:27.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8efb400640 1 -- 172.21.15.113:0/2117910472 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00b5070 con 0x7f8efc152d60 2024-03-04T08:03:27.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8efb400640 1 -- 172.21.15.113:0/2117910472 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef00a89d0 con 0x7f8efc152d60 2024-03-04T08:03:27.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8efb400640 1 -- 172.21.15.113:0/2117910472 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef00a8da0 con 0x7f8efc152d60 2024-03-04T08:03:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8f03400640 1 -- 172.21.15.113:0/2117910472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc152d60 msgr2=0x7f8efc0ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8f03400640 1 --2- 172.21.15.113:0/2117910472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc152d60 0x7f8efc0ce310 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00097f0 tx=0x7f8ef001ecd0 comp rx=0 tx=0).stop 2024-03-04T08:03:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8f03400640 1 -- 172.21.15.113:0/2117910472 shutdown_connections 2024-03-04T08:03:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8f03400640 1 --2- 172.21.15.113:0/2117910472 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc152d60 0x7f8efc0ce310 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8f03400640 1 -- 172.21.15.113:0/2117910472 >> 172.21.15.113:0/2117910472 conn(0x7f8efc066db0 msgr2=0x7f8efc0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:27.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8f03400640 1 -- 172.21.15.113:0/2117910472 shutdown_connections 2024-03-04T08:03:27.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8f03400640 1 -- 172.21.15.113:0/2117910472 wait complete. 2024-03-04T08:03:27.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8f03400640 1 Processor -- start 2024-03-04T08:03:27.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.455+0000 7f8f03400640 1 -- start start 2024-03-04T08:03:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f03400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc070120 0x7f8efc0b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f03400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8efc152d60 0x7f8efc1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f03400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1e4500 0x7f8efc1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f03400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0c39c0 con 0x7f8efc152d60 2024-03-04T08:03:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f03400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0c5020 con 0x7f8efc1e4500 2024-03-04T08:03:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f03400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc0c3b40 con 0x7f8efc070120 2024-03-04T08:03:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8efbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8efc152d60 0x7f8efc1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:27.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8efbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8efc152d60 0x7f8efc1552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42864/0 (socket says 172.21.15.113:42864) 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8efbe00640 1 -- 172.21.15.113:0/138693631 learned_addr learned my addr 172.21.15.113:0/138693631 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f00a00640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc070120 0x7f8efc0b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f01400640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1e4500 0x7f8efc1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8efbe00640 1 -- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc070120 msgr2=0x7f8efc0b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8efbe00640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc070120 0x7f8efc0b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8efbe00640 1 -- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1e4500 msgr2=0x7f8efc1557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8efbe00640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1e4500 0x7f8efc1557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8efbe00640 1 -- 172.21.15.113:0/138693631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc1ef020 con 0x7f8efc152d60 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8efbe00640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8efc152d60 0x7f8efc1552b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c500 tx=0x7f8ee800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8ef8c00640 1 -- 172.21.15.113:0/138693631 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8017070 con 0x7f8efc152d60 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8ef8c00640 1 -- 172.21.15.113:0/138693631 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee8005cf0 con 0x7f8efc152d60 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8ef8c00640 1 -- 172.21.15.113:0/138693631 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8007330 con 0x7f8efc152d60 2024-03-04T08:03:27.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f03400640 1 -- 172.21.15.113:0/138693631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc1ef300 con 0x7f8efc152d60 2024-03-04T08:03:27.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f03400640 1 -- 172.21.15.113:0/138693631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc1536c0 con 0x7f8efc152d60 2024-03-04T08:03:27.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.459+0000 7f8f03400640 1 -- 172.21.15.113:0/138693631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc075c00 con 0x7f8efc152d60 2024-03-04T08:03:27.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.463+0000 7f8ef8c00640 1 -- 172.21.15.113:0/138693631 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ee801a070 con 0x7f8efc152d60 2024-03-04T08:03:27.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.463+0000 7f8ef8c00640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8edc066f90 0x7f8edc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:27.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.463+0000 7f8ef8c00640 1 -- 172.21.15.113:0/138693631 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f8ee8084980 con 0x7f8efc152d60 2024-03-04T08:03:27.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.463+0000 7f8f00a00640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8edc066f90 0x7f8edc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:27.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.463+0000 7f8f00a00640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8edc066f90 0x7f8edc069450 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00b2000 tx=0x7f8ef0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:27.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.467+0000 7f8ef8c00640 1 -- 172.21.15.113:0/138693631 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8ee8054dd0 con 0x7f8efc152d60 2024-03-04T08:03:27.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 -- 172.21.15.113:0/138693631 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8efc06ff00 con 0x7f8efc152d60 2024-03-04T08:03:27.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8ef8c00640 1 -- 172.21.15.113:0/138693631 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+5828 (secure 0 0 0) 0x7f8ee8058e70 con 0x7f8efc152d60 2024-03-04T08:03:27.658 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:27.658 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:19.898691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6534,6807,6925]},"id":1}]} 2024-03-04T08:03:27.659 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 112 2024-03-04T08:03:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 -- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8edc066f90 msgr2=0x7f8edc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8edc066f90 0x7f8edc069450 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00b2000 tx=0x7f8ef0008040 comp rx=0 tx=0).stop 2024-03-04T08:03:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 -- 172.21.15.113:0/138693631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8efc152d60 msgr2=0x7f8efc1552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8efc152d60 0x7f8efc1552b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c500 tx=0x7f8ee800c9c0 comp rx=0 tx=0).stop 2024-03-04T08:03:27.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 -- 172.21.15.113:0/138693631 shutdown_connections 2024-03-04T08:03:27.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8efc1e4500 0x7f8efc1557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:27.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8edc066f90 0x7f8edc069450 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:27.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8efc152d60 0x7f8efc1552b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:27.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 --2- 172.21.15.113:0/138693631 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8efc070120 0x7f8efc0b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:27.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 -- 172.21.15.113:0/138693631 >> 172.21.15.113:0/138693631 conn(0x7f8efc066db0 msgr2=0x7f8efc0c8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:27.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 -- 172.21.15.113:0/138693631 shutdown_connections 2024-03-04T08:03:27.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:27.663+0000 7f8f03400640 1 -- 172.21.15.113:0/138693631 wait complete. 2024-03-04T08:03:27.680 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-03-04T08:03:29.681 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:03:29.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.823+0000 7f9c66c00640 1 Processor -- start 2024-03-04T08:03:29.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.823+0000 7f9c66c00640 1 -- start start 2024-03-04T08:03:29.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.823+0000 7f9c66c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c600c08d0 0x7f9c6006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:29.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.823+0000 7f9c66c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6015efd0 con 0x7f9c60155100 2024-03-04T08:03:29.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.823+0000 7f9c66c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c600c7ef0 con 0x7f9c6015ad30 2024-03-04T08:03:29.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.823+0000 7f9c66c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c600c8890 con 0x7f9c600c08d0 2024-03-04T08:03:29.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.823+0000 7f9c5fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9c60155100 0x7f9c600cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43636/0 (socket says 172.21.15.113:43636) 2024-03-04T08:03:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.823+0000 7f9c5fe00640 1 -- 172.21.15.113:0/625450972 learned_addr learned my addr 172.21.15.113:0/625450972 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:03:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.827+0000 7f9c5f400640 1 --2- 172.21.15.113:0/625450972 >> v2:172.21.15.113:3300/0 conn(0x7f9c600c08d0 0x7f9c6006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.827+0000 7f9c5ea00640 1 -- 172.21.15.113:0/625450972 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639143554 0 0) 0x7f9c6015efd0 con 0x7f9c60155100 2024-03-04T08:03:29.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.827+0000 7f9c5ea00640 1 -- 172.21.15.113:0/625450972 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c44003530 con 0x7f9c60155100 2024-03-04T08:03:29.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.827+0000 7f9c5ea00640 1 -- 172.21.15.113:0/625450972 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331124915 0 0) 0x7f9c600c7ef0 con 0x7f9c6015ad30 2024-03-04T08:03:29.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.827+0000 7f9c5ea00640 1 -- 172.21.15.113:0/625450972 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c6015efd0 con 0x7f9c6015ad30 2024-03-04T08:03:29.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.827+0000 7f9c5f400640 1 -- 172.21.15.113:0/625450972 >> v1:172.21.15.136:6789/0 conn(0x7f9c6015ad30 legacy=0x7f9c6015b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:29.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.827+0000 7f9c5f400640 1 -- 172.21.15.113:0/625450972 >> v1:172.21.15.113:6789/0 conn(0x7f9c60155100 legacy=0x7f9c600cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:29.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.827+0000 7f9c5f400640 1 -- 172.21.15.113:0/625450972 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600c81d0 con 0x7f9c600c08d0 2024-03-04T08:03:29.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c5f400640 1 --2- 172.21.15.113:0/625450972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c600c08d0 0x7f9c6006b060 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0097c0 tx=0x7f9c4c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95300689333ca4d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:29.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c5ea00640 1 -- 172.21.15.113:0/625450972 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0b1070 con 0x7f9c600c08d0 2024-03-04T08:03:29.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c5ea00640 1 -- 172.21.15.113:0/625450972 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c0ac5f0 con 0x7f9c600c08d0 2024-03-04T08:03:29.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c5ea00640 1 -- 172.21.15.113:0/625450972 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0ac9a0 con 0x7f9c600c08d0 2024-03-04T08:03:29.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c66c00640 1 -- 172.21.15.113:0/625450972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c600c08d0 msgr2=0x7f9c6006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:29.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c66c00640 1 --2- 172.21.15.113:0/625450972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c600c08d0 0x7f9c6006b060 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0097c0 tx=0x7f9c4c01c4a0 comp rx=0 tx=0).stop 2024-03-04T08:03:29.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c66c00640 1 -- 172.21.15.113:0/625450972 shutdown_connections 2024-03-04T08:03:29.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c66c00640 1 --2- 172.21.15.113:0/625450972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c600c08d0 0x7f9c6006b060 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:29.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c66c00640 1 -- 172.21.15.113:0/625450972 >> 172.21.15.113:0/625450972 conn(0x7f9c6006deb0 msgr2=0x7f9c6006e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:29.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c66c00640 1 -- 172.21.15.113:0/625450972 shutdown_connections 2024-03-04T08:03:29.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.831+0000 7f9c66c00640 1 -- 172.21.15.113:0/625450972 wait complete. 2024-03-04T08:03:29.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 Processor -- start 2024-03-04T08:03:29.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 -- start start 2024-03-04T08:03:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c600c08d0 0x7f9c601f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c60155100 0x7f9c60150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6015ad30 0x7f9c601f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c600cd950 con 0x7f9c6015ad30 2024-03-04T08:03:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c600cd7d0 con 0x7f9c60155100 2024-03-04T08:03:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c60168d40 con 0x7f9c600c08d0 2024-03-04T08:03:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c64c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6015ad30 0x7f9c601f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c64c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6015ad30 0x7f9c601f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42876/0 (socket says 172.21.15.113:42876) 2024-03-04T08:03:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c64c00640 1 -- 172.21.15.113:0/2990129437 learned_addr learned my addr 172.21.15.113:0/2990129437 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:03:29.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c5fe00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c600c08d0 0x7f9c601f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:29.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c5f400640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c60155100 0x7f9c60150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c64c00640 1 -- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c600c08d0 msgr2=0x7f9c601f31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c64c00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c600c08d0 0x7f9c601f31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c64c00640 1 -- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c60155100 msgr2=0x7f9c60150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c64c00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c60155100 0x7f9c60150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c64c00640 1 -- 172.21.15.113:0/2990129437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c601f4700 con 0x7f9c6015ad30 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c64c00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6015ad30 0x7f9c601f5a80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800c730 tx=0x7f9c5800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c57e00640 1 -- 172.21.15.113:0/2990129437 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58003b80 con 0x7f9c6015ad30 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c57e00640 1 -- 172.21.15.113:0/2990129437 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c58003d20 con 0x7f9c6015ad30 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c57e00640 1 -- 172.21.15.113:0/2990129437 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5800aa30 con 0x7f9c6015ad30 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 -- 172.21.15.113:0/2990129437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c601f49e0 con 0x7f9c6015ad30 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 -- 172.21.15.113:0/2990129437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c601f4fa0 con 0x7f9c6015ad30 2024-03-04T08:03:29.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.835+0000 7f9c66c00640 1 -- 172.21.15.113:0/2990129437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c60075c00 con 0x7f9c6015ad30 2024-03-04T08:03:29.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.839+0000 7f9c57e00640 1 -- 172.21.15.113:0/2990129437 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c5800abd0 con 0x7f9c6015ad30 2024-03-04T08:03:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.839+0000 7f9c57e00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9c40066f60 0x7f9c40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:03:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.839+0000 7f9c57e00640 1 -- 172.21.15.113:0/2990129437 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f9c58085480 con 0x7f9c6015ad30 2024-03-04T08:03:29.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.839+0000 7f9c5fe00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9c40066f60 0x7f9c40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:03:29.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.839+0000 7f9c5fe00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9c40066f60 0x7f9c40069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000f70 tx=0x7f9c50007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:03:29.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:29.847+0000 7f9c57e00640 1 -- 172.21.15.113:0/2990129437 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9c58055730 con 0x7f9c6015ad30 2024-03-04T08:03:30.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.043+0000 7f9c66c00640 1 -- 172.21.15.113:0/2990129437 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c600c8a20 con 0x7f9c6015ad30 2024-03-04T08:03:30.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.043+0000 7f9c57e00640 1 -- 172.21.15.113:0/2990129437 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+6623 (secure 0 0 0) 0x7f9c580597d0 con 0x7f9c6015ad30 2024-03-04T08:03:30.040 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:03:30.040 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":113,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7119,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:19.898691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807,6534]},"id":1}]} 2024-03-04T08:03:30.040 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 113 2024-03-04T08:03:30.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 -- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9c40066f60 msgr2=0x7f9c40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:30.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9c40066f60 0x7f9c40069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9c50000f70 tx=0x7f9c50007d40 comp rx=0 tx=0).stop 2024-03-04T08:03:30.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 -- 172.21.15.113:0/2990129437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6015ad30 msgr2=0x7f9c601f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:03:30.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6015ad30 0x7f9c601f5a80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9c5800c730 tx=0x7f9c5800cbf0 comp rx=0 tx=0).stop 2024-03-04T08:03:30.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 -- 172.21.15.113:0/2990129437 shutdown_connections 2024-03-04T08:03:30.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9c40066f60 0x7f9c40069420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:30.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c6015ad30 0x7f9c601f5a80 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:30.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9c60155100 0x7f9c60150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:30.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 --2- 172.21.15.113:0/2990129437 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9c600c08d0 0x7f9c601f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:03:30.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 -- 172.21.15.113:0/2990129437 >> 172.21.15.113:0/2990129437 conn(0x7f9c6006deb0 msgr2=0x7f9c6015b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:03:30.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 -- 172.21.15.113:0/2990129437 shutdown_connections 2024-03-04T08:03:30.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:03:30.047+0000 7f9c66c00640 1 -- 172.21.15.113:0/2990129437 wait complete. 2024-03-04T08:03:30.063 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-03-04T08:03:30.064 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 46 secs before thrashing 2024-03-04T08:03:34.234 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:03:34.237 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:03:34.445 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 7% (42/540) Updating files: 8% (44/540) Updating files: 9% (49/540) Updating files: 10% (54/540) Updating files: 11% (60/540) Updating files: 12% (65/540) Updating files: 12% (68/540) Updating files: 13% (71/540) Updating files: 14% (76/540) Updating files: 15% (81/540) Updating files: 16% (87/540) Updating files: 17% (92/540) Updating files: 18% (98/540) Updating files: 19% (103/540) Updating files: 20% (108/540) Updating files: 21% (114/540) Updating files: 22% (119/540) Updating files: 23% (125/540) Updating files: 24% (130/540) Updating files: 25% (135/540) Updating files: 26% (141/540) Updating files: 27% (146/540) Updating files: 28% (152/540) Updating files: 29% (157/540) Updating files: 30% (162/540) Updating files: 31% (168/540) Updating files: 32% (173/540) Updating files: 33% (179/540) Updating files: 34% (184/540) Updating files: 35% (189/540) Updating files: 36% (195/540) Updating files: 37% (200/540) Updating files: 38% (206/540) Updating files: 39% (211/540) Updating files: 40% (216/540) Updating files: 41% (222/540) Updating files: 42% (227/540) Updating files: 43% (233/540) Updating files: 44% (238/540) Updating files: 45% (243/540) Updating files: 46% (249/540) Updating files: 47% (254/540) Updating files: 48% (260/540) Updating files: 49% (265/540) Updating files: 50% (270/540) Updating files: 51% (276/540) Updating files: 52% (281/540) Updating files: 53% (287/540) Updating files: 54% (292/540) Updating files: 55% (297/540) Updating files: 56% (303/540) Updating files: 56% (307/540) Updating files: 57% (308/540) Updating files: 58% (314/540) Updating files: 59% (319/540) Updating files: 60% (324/540) Updating files: 61% (330/540) Updating files: 62% (335/540) Updating files: 63% (341/540) Updating files: 64% (346/540) Updating files: 65% (351/540) Updating files: 66% (357/540) Updating files: 67% (362/540) Updating files: 68% (368/540) Updating files: 69% (373/540) Updating files: 70% (378/540) Updating files: 71% (384/540) Updating files: 72% (389/540) Updating files: 73% (395/540) Updating files: 74% (400/540) Updating files: 74% (401/540) Updating files: 75% (405/540) Updating files: 76% (411/540) Updating files: 77% (416/540) Updating files: 77% (419/540) Updating files: 77% (420/540) Updating files: 78% (422/540) Updating files: 79% (427/540) Updating files: 79% (431/540) Updating files: 80% (432/540) Updating files: 81% (438/540) Updating files: 82% (443/540) Updating files: 83% (449/540) Updating files: 84% (454/540) Updating files: 85% (459/540) Updating files: 86% (465/540) Updating files: 87% (470/540) Updating files: 88% (476/540) Updating files: 89% (481/540) Updating files: 90% (486/540) Updating files: 91% (492/540) Updating files: 92% (497/540) Updating files: 93% (503/540) Updating files: 93% (507/540) Updating files: 94% (508/540) Updating files: 95% (513/540) Updating files: 96% (519/540) Updating files: 97% (524/540) Updating files: 97% (528/540) Updating files: 98% (530/540) Updating files: 99% (535/540) Updating files: 100% (540/540) Updating files: 100% (540/540), done. 2024-03-04T08:03:34.737 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 690f8175606 v0.55 2024-03-04T08:03:34.745 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.55 2024-03-04T08:03:34.791 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 56 -eq 48 ] 2024-03-04T08:03:34.791 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.56 2024-03-04T08:03:34.791 INFO:tasks.workunit.client.0.smithi113.stdout:v0.56 2024-03-04T08:03:34.792 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.56 2024-03-04T08:03:34.792 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.56 2024-03-04T08:03:49.760 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 14% (36/246) Updating files: 15% (37/246) Updating files: 16% (40/246) Updating files: 17% (42/246) Updating files: 18% (45/246) Updating files: 19% (47/246) Updating files: 20% (50/246) Updating files: 21% (52/246) Updating files: 22% (55/246) Updating files: 23% (57/246) Updating files: 24% (60/246) Updating files: 25% (62/246) Updating files: 26% (64/246) Updating files: 27% (67/246) Updating files: 28% (69/246) Updating files: 29% (72/246) Updating files: 30% (74/246) Updating files: 31% (77/246) Updating files: 32% (79/246) Updating files: 32% (80/246) Updating files: 33% (82/246) Updating files: 34% (84/246) Updating files: 35% (87/246) Updating files: 36% (89/246) Updating files: 36% (90/246) Updating files: 37% (92/246) Updating files: 38% (94/246) Updating files: 39% (96/246) Updating files: 40% (99/246) Updating files: 41% (101/246) Updating files: 42% (104/246) Updating files: 43% (106/246) Updating files: 44% (109/246) Updating files: 45% (111/246) Updating files: 46% (114/246) Updating files: 47% (116/246) Updating files: 48% (119/246) Updating files: 49% (121/246) Updating files: 50% (123/246) Updating files: 51% (126/246) Updating files: 52% (128/246) Updating files: 53% (131/246) Updating files: 54% (133/246) Updating files: 55% (136/246) Updating files: 56% (138/246) Updating files: 57% (141/246) Updating files: 58% (143/246) Updating files: 59% (146/246) Updating files: 60% (148/246) Updating files: 61% (151/246) Updating files: 62% (153/246) Updating files: 63% (155/246) Updating files: 64% (158/246) Updating files: 65% (160/246) Updating files: 66% (163/246) Updating files: 67% (165/246) Updating files: 68% (168/246) Updating files: 69% (170/246) Updating files: 70% (173/246) Updating files: 71% (175/246) Updating files: 72% (178/246) Updating files: 73% (180/246) Updating files: 74% (183/246) Updating files: 75% (185/246) Updating files: 76% (187/246) Updating files: 77% (190/246) Updating files: 78% (192/246) Updating files: 79% (195/246) Updating files: 80% (197/246) Updating files: 81% (200/246) Updating files: 82% (202/246) Updating files: 83% (205/246) Updating files: 84% (207/246) Updating files: 85% (210/246) Updating files: 86% (212/246) Updating files: 87% (215/246) Updating files: 88% (217/246) Updating files: 89% (219/246) Updating files: 90% (222/246) Updating files: 91% (224/246) Updating files: 92% (227/246) Updating files: 93% (229/246) Updating files: 94% (232/246) Updating files: 95% (234/246) Updating files: 96% (237/246) Updating files: 97% (239/246) Updating files: 98% (242/246) Updating files: 99% (244/246) Updating files: 100% (246/246) Updating files: 100% (246/246), done. 2024-03-04T08:03:49.795 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 1a32f0a0b42 v0.56 2024-03-04T08:03:49.803 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.56 2024-03-04T08:03:50.623 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 57 -eq 48 ] 2024-03-04T08:03:50.623 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.57 2024-03-04T08:03:50.623 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.57 2024-03-04T08:03:50.624 INFO:tasks.workunit.client.0.smithi113.stdout:v0.57 2024-03-04T08:03:50.624 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.57 2024-03-04T08:04:02.980 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 18% (55/304) Updating files: 19% (58/304) Updating files: 20% (61/304) Updating files: 21% (64/304) Updating files: 22% (67/304) Updating files: 23% (70/304) Updating files: 24% (73/304) Updating files: 25% (76/304) Updating files: 26% (80/304) Updating files: 27% (83/304) Updating files: 28% (86/304) Updating files: 29% (89/304) Updating files: 30% (92/304) Updating files: 31% (95/304) Updating files: 32% (98/304) Updating files: 33% (101/304) Updating files: 34% (104/304) Updating files: 35% (107/304) Updating files: 36% (110/304) Updating files: 37% (113/304) Updating files: 38% (116/304) Updating files: 39% (119/304) Updating files: 40% (122/304) Updating files: 41% (125/304) Updating files: 42% (128/304) Updating files: 42% (129/304) Updating files: 43% (131/304) Updating files: 44% (134/304) Updating files: 45% (137/304) Updating files: 46% (140/304) Updating files: 47% (143/304) Updating files: 48% (146/304) Updating files: 49% (149/304) Updating files: 50% (152/304) Updating files: 50% (155/304) Updating files: 51% (156/304) Updating files: 52% (159/304) Updating files: 53% (162/304) Updating files: 54% (165/304) Updating files: 54% (166/304) Updating files: 55% (168/304) Updating files: 56% (171/304) Updating files: 57% (174/304) Updating files: 58% (177/304) Updating files: 59% (180/304) Updating files: 60% (183/304) Updating files: 61% (186/304) Updating files: 62% (189/304) Updating files: 63% (192/304) Updating files: 64% (195/304) Updating files: 65% (198/304) Updating files: 66% (201/304) Updating files: 67% (204/304) Updating files: 68% (207/304) Updating files: 69% (210/304) Updating files: 70% (213/304) Updating files: 71% (216/304) Updating files: 72% (219/304) Updating files: 73% (222/304) Updating files: 74% (225/304) Updating files: 75% (228/304) Updating files: 76% (232/304) Updating files: 77% (235/304) Updating files: 78% (238/304) Updating files: 79% (241/304) Updating files: 80% (244/304) Updating files: 81% (247/304) Updating files: 82% (250/304) Updating files: 83% (253/304) Updating files: 84% (256/304) Updating files: 85% (259/304) Updating files: 86% (262/304) Updating files: 87% (265/304) Updating files: 88% (268/304) Updating files: 89% (271/304) Updating files: 90% (274/304) Updating files: 91% (277/304) Updating files: 92% (280/304) Updating files: 93% (283/304) Updating files: 94% (286/304) Updating files: 95% (289/304) Updating files: 96% (292/304) Updating files: 97% (295/304) Updating files: 97% (296/304) Updating files: 98% (298/304) Updating files: 99% (301/304) Updating files: 100% (304/304) Updating files: 100% (304/304), done. 2024-03-04T08:04:03.027 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 9a7a9d06c06 v0.57 2024-03-04T08:04:03.034 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.57 2024-03-04T08:04:03.851 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 58 -eq 48 ] 2024-03-04T08:04:03.851 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.58 2024-03-04T08:04:03.851 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.58 2024-03-04T08:04:03.852 INFO:tasks.workunit.client.0.smithi113.stdout:v0.58 2024-03-04T08:04:03.852 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.58 2024-03-04T08:04:04.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:04:04.305 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:04:16.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:16.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.206+0000 7fdace200640 1 Processor -- start 2024-03-04T08:04:16.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdace200640 1 -- start start 2024-03-04T08:04:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdace200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac8160820 0x7fdac8162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:16.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdace200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80c1ec0 con 0x7fdac80c85f0 2024-03-04T08:04:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdace200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80c9b30 con 0x7fdac80b20b0 2024-03-04T08:04:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdace200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac80c12a0 con 0x7fdac8160820 2024-03-04T08:04:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdac7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac8160820 0x7fdac8162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdac6a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdac80c85f0 0x7fdac806b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42828/0 (socket says 172.21.15.113:42828) 2024-03-04T08:04:16.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdac6a00640 1 -- 172.21.15.113:0/4133039023 learned_addr learned my addr 172.21.15.113:0/4133039023 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:16.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdac7e00640 1 -- 172.21.15.113:0/4133039023 >> v1:172.21.15.136:6789/0 conn(0x7fdac80b20b0 legacy=0x7fdac8157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdac7e00640 1 -- 172.21.15.113:0/4133039023 >> v1:172.21.15.113:6789/0 conn(0x7fdac80c85f0 legacy=0x7fdac806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdac7e00640 1 -- 172.21.15.113:0/4133039023 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80c1580 con 0x7fdac8160820 2024-03-04T08:04:16.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.210+0000 7fdac6000640 1 -- 172.21.15.113:0/4133039023 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151953918 0 0) 0x7fdac80c1ec0 con 0x7fdac80c85f0 2024-03-04T08:04:16.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.214+0000 7fdac7e00640 1 --2- 172.21.15.113:0/4133039023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac8160820 0x7fdac8162c10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdac000aed0 tx=0x7fdac0020290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c4f370b1811df0d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:16.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdac6000640 1 -- 172.21.15.113:0/4133039023 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac00b5070 con 0x7fdac8160820 2024-03-04T08:04:16.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdac6000640 1 -- 172.21.15.113:0/4133039023 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac0008440 con 0x7fdac8160820 2024-03-04T08:04:16.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdac6000640 1 -- 172.21.15.113:0/4133039023 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0008810 con 0x7fdac8160820 2024-03-04T08:04:16.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdace200640 1 -- 172.21.15.113:0/4133039023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac8160820 msgr2=0x7fdac8162c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdace200640 1 --2- 172.21.15.113:0/4133039023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac8160820 0x7fdac8162c10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdac000aed0 tx=0x7fdac0020290 comp rx=0 tx=0).stop 2024-03-04T08:04:16.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdace200640 1 -- 172.21.15.113:0/4133039023 shutdown_connections 2024-03-04T08:04:16.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdace200640 1 --2- 172.21.15.113:0/4133039023 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac8160820 0x7fdac8162c10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdace200640 1 -- 172.21.15.113:0/4133039023 >> 172.21.15.113:0/4133039023 conn(0x7fdac806e820 msgr2=0x7fdac806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:16.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdace200640 1 -- 172.21.15.113:0/4133039023 shutdown_connections 2024-03-04T08:04:16.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdace200640 1 -- 172.21.15.113:0/4133039023 wait complete. 2024-03-04T08:04:16.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdace200640 1 Processor -- start 2024-03-04T08:04:16.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.218+0000 7fdace200640 1 -- start start 2024-03-04T08:04:16.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdace200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac80b20b0 0x7fdac81ef550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:16.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdace200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac80c85f0 0x7fdac814df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:16.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdace200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac8160820 0x7fdac814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:16.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdace200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8157e20 con 0x7fdac80c85f0 2024-03-04T08:04:16.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdace200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac81571f0 con 0x7fdac8160820 2024-03-04T08:04:16.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdace200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac80c2400 con 0x7fdac80b20b0 2024-03-04T08:04:16.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac80c85f0 0x7fdac814df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:16.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac80c85f0 0x7fdac814df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52654/0 (socket says 172.21.15.113:52654) 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac80b20b0 0x7fdac81ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac6a00640 1 -- 172.21.15.113:0/2806550247 learned_addr learned my addr 172.21.15.113:0/2806550247 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac7e00640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac8160820 0x7fdac814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac7400640 1 -- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac8160820 msgr2=0x7fdac814e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac7400640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac8160820 0x7fdac814e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac7400640 1 -- 172.21.15.113:0/2806550247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac80c85f0 msgr2=0x7fdac814df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac7400640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac80c85f0 0x7fdac814df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac7400640 1 -- 172.21.15.113:0/2806550247 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80cb910 con 0x7fdac80b20b0 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac6a00640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac80c85f0 0x7fdac814df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdac7400640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac80b20b0 0x7fdac81ef550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdab800c390 tx=0x7fdab800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:16.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdabf400640 1 -- 172.21.15.113:0/2806550247 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8017070 con 0x7fdac80b20b0 2024-03-04T08:04:16.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdabf400640 1 -- 172.21.15.113:0/2806550247 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab80052d0 con 0x7fdac80b20b0 2024-03-04T08:04:16.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdace200640 1 -- 172.21.15.113:0/2806550247 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac80cbe90 con 0x7fdac80b20b0 2024-03-04T08:04:16.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdabf400640 1 -- 172.21.15.113:0/2806550247 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80054c0 con 0x7fdac80b20b0 2024-03-04T08:04:16.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdace200640 1 -- 172.21.15.113:0/2806550247 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac80cbbb0 con 0x7fdac80b20b0 2024-03-04T08:04:16.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.222+0000 7fdace200640 1 -- 172.21.15.113:0/2806550247 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac8075c00 con 0x7fdac80b20b0 2024-03-04T08:04:16.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.230+0000 7fdabf400640 1 -- 172.21.15.113:0/2806550247 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdab801a070 con 0x7fdac80b20b0 2024-03-04T08:04:16.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.230+0000 7fdabf400640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdaa8066f90 0x7fdaa8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:16.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.230+0000 7fdabf400640 1 -- 172.21.15.113:0/2806550247 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7fdab8084d80 con 0x7fdac80b20b0 2024-03-04T08:04:16.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.230+0000 7fdac6a00640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdaa8066f90 0x7fdaa8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:16.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.234+0000 7fdac6a00640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdaa8066f90 0x7fdaa8069450 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdac814f190 tx=0x7fdab4007e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:16.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.242+0000 7fdabf400640 1 -- 172.21.15.113:0/2806550247 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdab8021020 con 0x7fdac80b20b0 2024-03-04T08:04:16.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.442+0000 7fdace200640 1 -- 172.21.15.113:0/2806550247 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdac814cbb0 con 0x7fdac80b20b0 2024-03-04T08:04:16.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.442+0000 7fdabf400640 1 -- 172.21.15.113:0/2806550247 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+6623 (secure 0 0 0) 0x7fdab8055150 con 0x7fdac80b20b0 2024-03-04T08:04:16.441 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:16.441 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":113,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7119,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:19.898691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807,6534]},"id":1}]} 2024-03-04T08:04:16.441 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 113 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 -- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdaa8066f90 msgr2=0x7fdaa8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdaa8066f90 0x7fdaa8069450 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdac814f190 tx=0x7fdab4007e20 comp rx=0 tx=0).stop 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 -- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac80b20b0 msgr2=0x7fdac81ef550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac80b20b0 0x7fdac81ef550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdab800c390 tx=0x7fdab800c850 comp rx=0 tx=0).stop 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 -- 172.21.15.113:0/2806550247 shutdown_connections 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdac8160820 0x7fdac814e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdaa8066f90 0x7fdaa8069450 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdac80c85f0 0x7fdac814df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 --2- 172.21.15.113:0/2806550247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdac80b20b0 0x7fdac81ef550 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 -- 172.21.15.113:0/2806550247 >> 172.21.15.113:0/2806550247 conn(0x7fdac806e820 msgr2=0x7fdac80c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:16.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 -- 172.21.15.113:0/2806550247 shutdown_connections 2024-03-04T08:04:16.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.446+0000 7fdace200640 1 -- 172.21.15.113:0/2806550247 wait complete. 2024-03-04T08:04:16.462 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=1) 2024-03-04T08:04:16.462 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-03-04T08:04:16.463 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:04:16.477 INFO:tasks.ceph.mds.e:Stopped 2024-03-04T08:04:16.477 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:16.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be8200640 1 Processor -- start 2024-03-04T08:04:16.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be8200640 1 -- start start 2024-03-04T08:04:16.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be8200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4be0152d60 0x7f4be00ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:16.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be8200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be00c5380 con 0x7f4be0159040 2024-03-04T08:04:16.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be8200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be00c3440 con 0x7f4be0070120 2024-03-04T08:04:16.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be8200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be006f620 con 0x7f4be0152d60 2024-03-04T08:04:16.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be6200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4be0159040 0x7f4be01632b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42836/0 (socket says 172.21.15.113:42836) 2024-03-04T08:04:16.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be6200640 1 -- 172.21.15.113:0/111965210 learned_addr learned my addr 172.21.15.113:0/111965210 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:16.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be5800640 1 --2- 172.21.15.113:0/111965210 >> v2:172.21.15.136:3300/0 conn(0x7f4be0152d60 0x7f4be00ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:16.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4bdfe00640 1 -- 172.21.15.113:0/111965210 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458326214 0 0) 0x7f4be00c5380 con 0x7f4be0159040 2024-03-04T08:04:16.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4bdfe00640 1 -- 172.21.15.113:0/111965210 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc8003530 con 0x7f4be0159040 2024-03-04T08:04:16.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be5800640 1 -- 172.21.15.113:0/111965210 >> v1:172.21.15.136:6789/0 conn(0x7f4be0070120 legacy=0x7f4be0158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be5800640 1 -- 172.21.15.113:0/111965210 >> v1:172.21.15.113:6789/0 conn(0x7f4be0159040 legacy=0x7f4be01632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.658+0000 7f4be5800640 1 -- 172.21.15.113:0/111965210 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00c36e0 con 0x7f4be0152d60 2024-03-04T08:04:16.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4be5800640 1 --2- 172.21.15.113:0/111965210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be0152d60 0x7f4be00ce310 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40097c0 tx=0x7f4bd401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f15d97b2a9b71bd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:16.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4bdfe00640 1 -- 172.21.15.113:0/111965210 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40b1070 con 0x7f4be0152d60 2024-03-04T08:04:16.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4bdfe00640 1 -- 172.21.15.113:0/111965210 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd40ac410 con 0x7f4be0152d60 2024-03-04T08:04:16.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4bdfe00640 1 -- 172.21.15.113:0/111965210 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40ac7c0 con 0x7f4be0152d60 2024-03-04T08:04:16.660 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 47% (127/268) Updating files: 48% (129/268) Updating files: 49% (132/268) Updating files: 50% (134/268) Updating files: 51% (137/268) Updating files: 52% (140/268) Updating files: 53% (143/268) Updating files: 54% (145/268) Updating files: 55% (148/268) Updating files: 55% (150/268) Updating files: 56% (151/268) Updating files: 57% (153/268) Updating files: 58% (156/268) Updating files: 59% (159/268) Updating files: 60% (161/268) Updating files: 61% (164/268) Updating files: 62% (167/268) Updating files: 63% (169/268) Updating files: 64% (172/268) Updating files: 65% (175/268) Updating files: 66% (177/268) Updating files: 67% (180/268) Updating files: 67% (181/268) Updating files: 68% (183/268) Updating files: 69% (185/268) Updating files: 70% (188/268) Updating files: 71% (191/268) Updating files: 72% (193/268) Updating files: 73% (196/268) Updating files: 74% (199/268) Updating files: 75% (201/268) Updating files: 76% (204/268) Updating files: 77% (207/268) Updating files: 78% (210/268) Updating files: 79% (212/268) Updating files: 80% (215/268) Updating files: 80% (217/268) Updating files: 81% (218/268) Updating files: 82% (220/268) Updating files: 83% (223/268) Updating files: 84% (226/268) Updating files: 85% (228/268) Updating files: 86% (231/268) Updating files: 87% (234/268) Updating files: 88% (236/268) Updating files: 89% (239/268) Updating files: 90% (242/268) Updating files: 91% (244/268) Updating files: 92% (247/268) Updating files: 93% (250/268) Updating files: 94% (252/268) Updating files: 95% (255/268) Updating files: 96% (258/268) Updating files: 97% (260/268) Updating files: 98% (263/268) Updating files: 99% (266/268) Updating files: 99% (267/268) Updating files: 100% (268/268) Updating files: 100% (268/268), done. 2024-03-04T08:04:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4be8200640 1 -- 172.21.15.113:0/111965210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be0152d60 msgr2=0x7f4be00ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4be8200640 1 --2- 172.21.15.113:0/111965210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be0152d60 0x7f4be00ce310 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40097c0 tx=0x7f4bd401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4be8200640 1 -- 172.21.15.113:0/111965210 shutdown_connections 2024-03-04T08:04:16.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4be8200640 1 --2- 172.21.15.113:0/111965210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be0152d60 0x7f4be00ce310 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4be8200640 1 -- 172.21.15.113:0/111965210 >> 172.21.15.113:0/111965210 conn(0x7f4be0066db0 msgr2=0x7f4be00671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:16.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4be8200640 1 -- 172.21.15.113:0/111965210 shutdown_connections 2024-03-04T08:04:16.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.662+0000 7f4be8200640 1 -- 172.21.15.113:0/111965210 wait complete. 2024-03-04T08:04:16.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 Processor -- start 2024-03-04T08:04:16.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 -- start start 2024-03-04T08:04:16.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be0070120 0x7f4be01f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4be0152d60 0x7f4be00c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be014b7d0 0x7f4be01578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00ce9d0 con 0x7f4be0070120 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00ce850 con 0x7f4be014b7d0 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00ceb50 con 0x7f4be0152d60 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be0070120 0x7f4be01f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be0070120 0x7f4be01f3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52660/0 (socket says 172.21.15.113:52660) 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be5800640 1 -- 172.21.15.113:0/2330462454 learned_addr learned my addr 172.21.15.113:0/2330462454 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be6200640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be014b7d0 0x7f4be01578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be4e00640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4be0152d60 0x7f4be00c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be5800640 1 -- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4be0152d60 msgr2=0x7f4be00c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be5800640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4be0152d60 0x7f4be00c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be5800640 1 -- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be014b7d0 msgr2=0x7f4be01578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be5800640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be014b7d0 0x7f4be01578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be5800640 1 -- 172.21.15.113:0/2330462454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be0067700 con 0x7f4be0070120 2024-03-04T08:04:16.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be5800640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be0070120 0x7f4be01f3980 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40af040 tx=0x7f4bd401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:16.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4bdd600640 1 -- 172.21.15.113:0/2330462454 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40bd070 con 0x7f4be0070120 2024-03-04T08:04:16.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4bdd600640 1 -- 172.21.15.113:0/2330462454 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd40b2040 con 0x7f4be0070120 2024-03-04T08:04:16.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 -- 172.21.15.113:0/2330462454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be01f3ec0 con 0x7f4be0070120 2024-03-04T08:04:16.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4bdd600640 1 -- 172.21.15.113:0/2330462454 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40b1050 con 0x7f4be0070120 2024-03-04T08:04:16.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 -- 172.21.15.113:0/2330462454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be01f4130 con 0x7f4be0070120 2024-03-04T08:04:16.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.666+0000 7f4be8200640 1 -- 172.21.15.113:0/2330462454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be0075c00 con 0x7f4be0070120 2024-03-04T08:04:16.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.670+0000 7f4bdd600640 1 -- 172.21.15.113:0/2330462454 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4bd40c0070 con 0x7f4be0070120 2024-03-04T08:04:16.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.670+0000 7f4bdd600640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bc0066f60 0x7f4bc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:16.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.670+0000 7f4bdd600640 1 -- 172.21.15.113:0/2330462454 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f4bd411eff0 con 0x7f4be0070120 2024-03-04T08:04:16.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.670+0000 7f4be4e00640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bc0066f60 0x7f4bc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:16.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.674+0000 7f4be4e00640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bc0066f60 0x7f4bc0069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4be01ee2c0 tx=0x7f4bcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:16.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.678+0000 7f4bdd600640 1 -- 172.21.15.113:0/2330462454 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4bd4016090 con 0x7f4be0070120 2024-03-04T08:04:16.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.874+0000 7f4be8200640 1 -- 172.21.15.113:0/2330462454 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4be006ff00 con 0x7f4be0070120 2024-03-04T08:04:16.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.874+0000 7f4bdd600640 1 -- 172.21.15.113:0/2330462454 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+6623 (secure 0 0 0) 0x7f4bd40ef2e0 con 0x7f4be0070120 2024-03-04T08:04:16.873 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:16.873 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":113,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7119,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:19.898691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807,6534]},"id":1}]} 2024-03-04T08:04:16.873 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 113 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 -- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bc0066f60 msgr2=0x7f4bc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bc0066f60 0x7f4bc0069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4be01ee2c0 tx=0x7f4bcc009040 comp rx=0 tx=0).stop 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 -- 172.21.15.113:0/2330462454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be0070120 msgr2=0x7f4be01f3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be0070120 0x7f4be01f3980 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40af040 tx=0x7f4bd401ace0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 -- 172.21.15.113:0/2330462454 shutdown_connections 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4be014b7d0 0x7f4be01578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4be0152d60 0x7f4be00c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bc0066f60 0x7f4bc0069420 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 --2- 172.21.15.113:0/2330462454 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be0070120 0x7f4be01f3980 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 -- 172.21.15.113:0/2330462454 >> 172.21.15.113:0/2330462454 conn(0x7f4be0066db0 msgr2=0x7f4be00c93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:16.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 -- 172.21.15.113:0/2330462454 shutdown_connections 2024-03-04T08:04:16.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:16.878+0000 7f4be8200640 1 -- 172.21.15.113:0/2330462454 wait complete. 2024-03-04T08:04:16.902 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:04:17.097 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at ba3f91e7504 v0.58 2024-03-04T08:04:17.104 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.58 2024-03-04T08:04:18.903 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:19.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0ee600640 1 Processor -- start 2024-03-04T08:04:19.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0ee600640 1 -- start start 2024-03-04T08:04:19.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0ee600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd0e8158010 0x7fd0e806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:19.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0ee600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd0e806ff20 0x7fd0e815b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:19.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0ee600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e80ca0e0 con 0x7fd0e815b750 2024-03-04T08:04:19.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0ee600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e806f450 con 0x7fd0e8158010 2024-03-04T08:04:19.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0ee600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e815ad80 con 0x7fd0e806ff20 2024-03-04T08:04:19.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0e7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd0e8158010 0x7fd0e806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:19.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0e7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd0e8158010 0x7fd0e806dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46650/0 (socket says 172.21.15.113:46650) 2024-03-04T08:04:19.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0e7400640 1 -- 172.21.15.113:0/1917926688 learned_addr learned my addr 172.21.15.113:0/1917926688 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:19.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0dfe00640 1 --2- 172.21.15.113:0/1917926688 >> v2:172.21.15.136:3300/0 conn(0x7fd0e806ff20 0x7fd0e815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:19.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0e7400640 1 -- 172.21.15.113:0/1917926688 >> v2:172.21.15.136:3300/0 conn(0x7fd0e806ff20 msgr2=0x7fd0e815b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:19.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0e7400640 1 --2- 172.21.15.113:0/1917926688 >> v2:172.21.15.136:3300/0 conn(0x7fd0e806ff20 0x7fd0e815b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:19.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0e7400640 1 -- 172.21.15.113:0/1917926688 >> v1:172.21.15.136:6789/0 conn(0x7fd0e815b750 legacy=0x7fd0e80ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:19.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.041+0000 7fd0e7400640 1 -- 172.21.15.113:0/1917926688 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e815aa30 con 0x7fd0e8158010 2024-03-04T08:04:19.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0e7400640 1 --2- 172.21.15.113:0/1917926688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 0x7fd0e806dd50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80cc040 tx=0x7fd0d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28140b25d780dac server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:19.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0e6a00640 1 -- 172.21.15.113:0/1917926688 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80b0070 con 0x7fd0e8158010 2024-03-04T08:04:19.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0e6a00640 1 -- 172.21.15.113:0/1917926688 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d80ab600 con 0x7fd0e8158010 2024-03-04T08:04:19.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0e6a00640 1 -- 172.21.15.113:0/1917926688 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80ab9d0 con 0x7fd0e8158010 2024-03-04T08:04:19.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0ee600640 1 -- 172.21.15.113:0/1917926688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 msgr2=0x7fd0e806dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:19.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0ee600640 1 --2- 172.21.15.113:0/1917926688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 0x7fd0e806dd50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80cc040 tx=0x7fd0d801c520 comp rx=0 tx=0).stop 2024-03-04T08:04:19.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0ee600640 1 -- 172.21.15.113:0/1917926688 shutdown_connections 2024-03-04T08:04:19.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0ee600640 1 --2- 172.21.15.113:0/1917926688 >> v2:172.21.15.136:3300/0 conn(0x7fd0e806ff20 0x7fd0e815b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:19.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0ee600640 1 --2- 172.21.15.113:0/1917926688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 0x7fd0e806dd50 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:19.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0ee600640 1 -- 172.21.15.113:0/1917926688 >> 172.21.15.113:0/1917926688 conn(0x7fd0e80734d0 msgr2=0x7fd0e8066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:19.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0ee600640 1 -- 172.21.15.113:0/1917926688 shutdown_connections 2024-03-04T08:04:19.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0ee600640 1 -- 172.21.15.113:0/1917926688 wait complete. 2024-03-04T08:04:19.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0ee600640 1 Processor -- start 2024-03-04T08:04:19.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.049+0000 7fd0ee600640 1 -- start start 2024-03-04T08:04:19.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0ee600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0e806ff20 0x7fd0e80cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:19.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0ee600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 0x7fd0e8151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:19.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0ee600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e815b750 0x7fd0e81f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:19.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0ee600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e81583f0 con 0x7fd0e8158010 2024-03-04T08:04:19.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0ee600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e806e290 con 0x7fd0e815b750 2024-03-04T08:04:19.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0ee600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e806c170 con 0x7fd0e806ff20 2024-03-04T08:04:19.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 0x7fd0e8151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:19.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0dfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 0x7fd0e8151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46652/0 (socket says 172.21.15.113:46652) 2024-03-04T08:04:19.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0dfe00640 1 -- 172.21.15.113:0/975426710 learned_addr learned my addr 172.21.15.113:0/975426710 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:19.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0e7400640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0e806ff20 0x7fd0e80cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:19.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0e7e00640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e815b750 0x7fd0e81f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:19.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0dfe00640 1 -- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0e806ff20 msgr2=0x7fd0e80cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:19.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0dfe00640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0e806ff20 0x7fd0e80cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0dfe00640 1 -- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e815b750 msgr2=0x7fd0e81f31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0dfe00640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e815b750 0x7fd0e81f31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0dfe00640 1 -- 172.21.15.113:0/975426710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e81f2d30 con 0x7fd0e8158010 2024-03-04T08:04:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0e7e00640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e815b750 0x7fd0e81f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:04:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0dfe00640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 0x7fd0e8151f70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000e900 tx=0x7fd0d000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0df400640 1 -- 172.21.15.113:0/975426710 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0017070 con 0x7fd0e8158010 2024-03-04T08:04:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0df400640 1 -- 172.21.15.113:0/975426710 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d000ca00 con 0x7fd0e8158010 2024-03-04T08:04:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0df400640 1 -- 172.21.15.113:0/975426710 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000cdd0 con 0x7fd0e8158010 2024-03-04T08:04:19.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0ee600640 1 -- 172.21.15.113:0/975426710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e80b2740 con 0x7fd0e8158010 2024-03-04T08:04:19.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0ee600640 1 -- 172.21.15.113:0/975426710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e80b2d00 con 0x7fd0e8158010 2024-03-04T08:04:19.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0ee600640 1 -- 172.21.15.113:0/975426710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e8075c00 con 0x7fd0e8158010 2024-03-04T08:04:19.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.053+0000 7fd0df400640 1 -- 172.21.15.113:0/975426710 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0d001a070 con 0x7fd0e8158010 2024-03-04T08:04:19.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.057+0000 7fd0df400640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd0c4066f60 0x7fd0c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:19.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.057+0000 7fd0df400640 1 -- 172.21.15.113:0/975426710 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7fd0d00859c0 con 0x7fd0e8158010 2024-03-04T08:04:19.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.057+0000 7fd0e7400640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd0c4066f60 0x7fd0c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:19.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.057+0000 7fd0e7400640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd0c4066f60 0x7fd0c4069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80ad000 tx=0x7fd0d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:19.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.061+0000 7fd0df400640 1 -- 172.21.15.113:0/975426710 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd0e81f2d30 con 0x7fd0e8158010 2024-03-04T08:04:19.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.249+0000 7fd0ee600640 1 -- 172.21.15.113:0/975426710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0e8076a60 con 0x7fd0e8158010 2024-03-04T08:04:19.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.249+0000 7fd0df400640 1 -- 172.21.15.113:0/975426710 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+6623 (secure 0 0 0) 0x7fd0d0055d00 con 0x7fd0e8158010 2024-03-04T08:04:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:19.249 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":113,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7119,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:19.898691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807,6534]},"id":1}]} 2024-03-04T08:04:19.250 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 113 2024-03-04T08:04:19.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 -- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd0c4066f60 msgr2=0x7fd0c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:19.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd0c4066f60 0x7fd0c4069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80ad000 tx=0x7fd0d8008040 comp rx=0 tx=0).stop 2024-03-04T08:04:19.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 -- 172.21.15.113:0/975426710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 msgr2=0x7fd0e8151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:19.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 0x7fd0e8151f70 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000e900 tx=0x7fd0d000edc0 comp rx=0 tx=0).stop 2024-03-04T08:04:19.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 -- 172.21.15.113:0/975426710 shutdown_connections 2024-03-04T08:04:19.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0e815b750 0x7fd0e81f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:19.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd0c4066f60 0x7fd0c4069420 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:19.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0e8158010 0x7fd0e8151f70 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:19.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 --2- 172.21.15.113:0/975426710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0e806ff20 0x7fd0e80cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:19.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 -- 172.21.15.113:0/975426710 >> 172.21.15.113:0/975426710 conn(0x7fd0e80734d0 msgr2=0x7fd0e81552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:19.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 -- 172.21.15.113:0/975426710 shutdown_connections 2024-03-04T08:04:19.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:19.253+0000 7fd0ee600640 1 -- 172.21.15.113:0/975426710 wait complete. 2024-03-04T08:04:19.271 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:04:21.272 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:21.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.409+0000 7fb544400640 1 Processor -- start 2024-03-04T08:04:21.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.409+0000 7fb544400640 1 -- start start 2024-03-04T08:04:21.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.409+0000 7fb544400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb53c152d60 0x7fb53c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:21.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.409+0000 7fb544400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb53c070120 0x7fb53c0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:21.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.409+0000 7fb544400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb53c0c9db0 0x7fb53c158930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:21.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.409+0000 7fb544400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53c0c5380 con 0x7fb53c0c9db0 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.409+0000 7fb544400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53c0c3440 con 0x7fb53c070120 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.409+0000 7fb544400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c06f620 con 0x7fb53c152d60 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb542400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb53c0c9db0 0x7fb53c158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb542400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb53c0c9db0 0x7fb53c158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46662/0 (socket says 172.21.15.113:46662) 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb542400640 1 -- 172.21.15.113:0/1688327015 learned_addr learned my addr 172.21.15.113:0/1688327015 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb541000640 1 --2- 172.21.15.113:0/1688327015 >> v2:172.21.15.136:3300/0 conn(0x7fb53c070120 0x7fb53c0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb541a00640 1 --2- 172.21.15.113:0/1688327015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb53c152d60 0x7fb53c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb542400640 1 -- 172.21.15.113:0/1688327015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb53c152d60 msgr2=0x7fb53c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb542400640 1 --2- 172.21.15.113:0/1688327015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb53c152d60 0x7fb53c0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb542400640 1 -- 172.21.15.113:0/1688327015 >> v2:172.21.15.136:3300/0 conn(0x7fb53c070120 msgr2=0x7fb53c0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb542400640 1 --2- 172.21.15.113:0/1688327015 >> v2:172.21.15.136:3300/0 conn(0x7fb53c070120 0x7fb53c0ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb542400640 1 -- 172.21.15.113:0/1688327015 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c0c36e0 con 0x7fb53c0c9db0 2024-03-04T08:04:21.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb542400640 1 --2- 172.21.15.113:0/1688327015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 0x7fb53c158930 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb53c070580 tx=0x7fb53401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7a243a7bd929be server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:21.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb53be00640 1 -- 172.21.15.113:0/1688327015 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340b4070 con 0x7fb53c0c9db0 2024-03-04T08:04:21.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb53be00640 1 -- 172.21.15.113:0/1688327015 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5340af400 con 0x7fb53c0c9db0 2024-03-04T08:04:21.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.413+0000 7fb53be00640 1 -- 172.21.15.113:0/1688327015 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340af7d0 con 0x7fb53c0c9db0 2024-03-04T08:04:21.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 -- 172.21.15.113:0/1688327015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 msgr2=0x7fb53c158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:21.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 --2- 172.21.15.113:0/1688327015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 0x7fb53c158930 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb53c070580 tx=0x7fb53401f1e0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 -- 172.21.15.113:0/1688327015 shutdown_connections 2024-03-04T08:04:21.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 --2- 172.21.15.113:0/1688327015 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 0x7fb53c158930 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 --2- 172.21.15.113:0/1688327015 >> v2:172.21.15.136:3300/0 conn(0x7fb53c070120 0x7fb53c0ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 --2- 172.21.15.113:0/1688327015 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb53c152d60 0x7fb53c0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 -- 172.21.15.113:0/1688327015 >> 172.21.15.113:0/1688327015 conn(0x7fb53c066db0 msgr2=0x7fb53c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:21.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 -- 172.21.15.113:0/1688327015 shutdown_connections 2024-03-04T08:04:21.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 -- 172.21.15.113:0/1688327015 wait complete. 2024-03-04T08:04:21.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 Processor -- start 2024-03-04T08:04:21.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 -- start start 2024-03-04T08:04:21.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb53c070120 0x7fb53c15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:21.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 0x7fb53c1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb53c152d60 0x7fb53c151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0c39c0 con 0x7fb53c0c9db0 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0c5020 con 0x7fb53c070120 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb544400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0c3b40 con 0x7fb53c152d60 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb541000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 0x7fb53c1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb541000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 0x7fb53c1511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46676/0 (socket says 172.21.15.113:46676) 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb541000640 1 -- 172.21.15.113:0/3497054826 learned_addr learned my addr 172.21.15.113:0/3497054826 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb542400640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb53c152d60 0x7fb53c151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb541a00640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb53c070120 0x7fb53c15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb541000640 1 -- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb53c152d60 msgr2=0x7fb53c151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb541000640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb53c152d60 0x7fb53c151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb541000640 1 -- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb53c070120 msgr2=0x7fb53c15c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb541000640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb53c070120 0x7fb53c15c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb541000640 1 -- 172.21.15.113:0/3497054826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c0b37e0 con 0x7fb53c0c9db0 2024-03-04T08:04:21.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.417+0000 7fb541a00640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb53c070120 0x7fb53c15c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:04:21.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb541000640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 0x7fb53c1511c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb52800e870 tx=0x7fb52800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:21.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb539600640 1 -- 172.21.15.113:0/3497054826 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528017070 con 0x7fb53c0c9db0 2024-03-04T08:04:21.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb539600640 1 -- 172.21.15.113:0/3497054826 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52800ee60 con 0x7fb53c0c9db0 2024-03-04T08:04:21.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb539600640 1 -- 172.21.15.113:0/3497054826 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800cb80 con 0x7fb53c0c9db0 2024-03-04T08:04:21.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb544400640 1 -- 172.21.15.113:0/3497054826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c1f09f0 con 0x7fb53c0c9db0 2024-03-04T08:04:21.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb544400640 1 -- 172.21.15.113:0/3497054826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c1f0fb0 con 0x7fb53c0c9db0 2024-03-04T08:04:21.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb544400640 1 -- 172.21.15.113:0/3497054826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53c075c00 con 0x7fb53c0c9db0 2024-03-04T08:04:21.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb539600640 1 -- 172.21.15.113:0/3497054826 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb52801a070 con 0x7fb53c0c9db0 2024-03-04T08:04:21.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb539600640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb51c066f60 0x7fb51c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:21.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb539600640 1 -- 172.21.15.113:0/3497054826 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7fb5280857f0 con 0x7fb53c0c9db0 2024-03-04T08:04:21.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.421+0000 7fb541a00640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb51c066f60 0x7fb51c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:21.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.425+0000 7fb541a00640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb51c066f60 0x7fb51c069420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb530009bb0 tx=0x7fb530008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:21.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.425+0000 7fb539600640 1 -- 172.21.15.113:0/3497054826 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb528055b30 con 0x7fb53c0c9db0 2024-03-04T08:04:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.613+0000 7fb544400640 1 -- 172.21.15.113:0/3497054826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb53c06ff00 con 0x7fb53c0c9db0 2024-03-04T08:04:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.613+0000 7fb539600640 1 -- 172.21.15.113:0/3497054826 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+6623 (secure 0 0 0) 0x7fb528059bd0 con 0x7fb53c0c9db0 2024-03-04T08:04:21.613 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:21.613 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":113,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7119,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:19.898691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807,6534]},"id":1}]} 2024-03-04T08:04:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 113 2024-03-04T08:04:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 -- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb51c066f60 msgr2=0x7fb51c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb51c066f60 0x7fb51c069420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb530009bb0 tx=0x7fb530008040 comp rx=0 tx=0).stop 2024-03-04T08:04:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 -- 172.21.15.113:0/3497054826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 msgr2=0x7fb53c1511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 0x7fb53c1511c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb52800e870 tx=0x7fb52800ed30 comp rx=0 tx=0).stop 2024-03-04T08:04:21.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 -- 172.21.15.113:0/3497054826 shutdown_connections 2024-03-04T08:04:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb53c152d60 0x7fb53c151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb51c066f60 0x7fb51c069420 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb53c0c9db0 0x7fb53c1511c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 --2- 172.21.15.113:0/3497054826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb53c070120 0x7fb53c15c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 -- 172.21.15.113:0/3497054826 >> 172.21.15.113:0/3497054826 conn(0x7fb53c066db0 msgr2=0x7fb53c0cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 -- 172.21.15.113:0/3497054826 shutdown_connections 2024-03-04T08:04:21.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:21.617+0000 7fb544400640 1 -- 172.21.15.113:0/3497054826 wait complete. 2024-03-04T08:04:21.635 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:04:23.635 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:23.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.773+0000 7f244a000640 1 Processor -- start 2024-03-04T08:04:23.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.773+0000 7f244a000640 1 -- start start 2024-03-04T08:04:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.773+0000 7f244a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24440c8200 0x7f2444070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.773+0000 7f244a000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24441591f0 con 0x7f24441527f0 2024-03-04T08:04:23.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.773+0000 7f244a000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24441594d0 con 0x7f2444152bd0 2024-03-04T08:04:23.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.773+0000 7f244a000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2444071df0 con 0x7f24440c8200 2024-03-04T08:04:23.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.773+0000 7f2443400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f24441527f0 0x7f2444072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53212/0 (socket says 172.21.15.113:53212) 2024-03-04T08:04:23.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.773+0000 7f2443400640 1 -- 172.21.15.113:0/2106423650 learned_addr learned my addr 172.21.15.113:0/2106423650 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:23.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.777+0000 7f2442a00640 1 --2- 172.21.15.113:0/2106423650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24440c8200 0x7f2444070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:23.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.777+0000 7f2442000640 1 -- 172.21.15.113:0/2106423650 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620819916 0 0) 0x7f24441591f0 con 0x7f24441527f0 2024-03-04T08:04:23.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.777+0000 7f2442000640 1 -- 172.21.15.113:0/2106423650 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2428003530 con 0x7f24441527f0 2024-03-04T08:04:23.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.777+0000 7f2442000640 1 -- 172.21.15.113:0/2106423650 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124211117 0 0) 0x7f24441594d0 con 0x7f2444152bd0 2024-03-04T08:04:23.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.777+0000 7f2442000640 1 -- 172.21.15.113:0/2106423650 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24441591f0 con 0x7f2444152bd0 2024-03-04T08:04:23.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.777+0000 7f2442a00640 1 -- 172.21.15.113:0/2106423650 >> v1:172.21.15.136:6789/0 conn(0x7f2444152bd0 legacy=0x7f24441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:23.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.777+0000 7f2442a00640 1 -- 172.21.15.113:0/2106423650 >> v1:172.21.15.113:6789/0 conn(0x7f24441527f0 legacy=0x7f2444072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:23.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.777+0000 7f2442000640 1 -- 172.21.15.113:0/2106423650 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 299613304 0 0) 0x7f2428003530 con 0x7f24441527f0 2024-03-04T08:04:23.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.777+0000 7f2442a00640 1 -- 172.21.15.113:0/2106423650 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440717a0 con 0x7f24440c8200 2024-03-04T08:04:23.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.777+0000 7f2442a00640 1 --2- 172.21.15.113:0/2106423650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24440c8200 0x7f2444070d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24300097f0 tx=0x7f243001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5da7f58b8534c37d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:23.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2442000640 1 -- 172.21.15.113:0/2106423650 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300b0070 con 0x7f24440c8200 2024-03-04T08:04:23.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2442000640 1 -- 172.21.15.113:0/2106423650 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24300ab400 con 0x7f24440c8200 2024-03-04T08:04:23.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2442000640 1 -- 172.21.15.113:0/2106423650 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434001680 con 0x7f24440c8200 2024-03-04T08:04:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 -- 172.21.15.113:0/2106423650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24440c8200 msgr2=0x7f2444070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 --2- 172.21.15.113:0/2106423650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24440c8200 0x7f2444070d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24300097f0 tx=0x7f243001c400 comp rx=0 tx=0).stop 2024-03-04T08:04:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 -- 172.21.15.113:0/2106423650 shutdown_connections 2024-03-04T08:04:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 --2- 172.21.15.113:0/2106423650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24440c8200 0x7f2444070d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 -- 172.21.15.113:0/2106423650 >> 172.21.15.113:0/2106423650 conn(0x7f244406e820 msgr2=0x7f244406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 -- 172.21.15.113:0/2106423650 shutdown_connections 2024-03-04T08:04:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 -- 172.21.15.113:0/2106423650 wait complete. 2024-03-04T08:04:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 Processor -- start 2024-03-04T08:04:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 -- start start 2024-03-04T08:04:23.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24440c8200 0x7f244414a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24441527f0 0x7f244414d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2444152bd0 0x7f24440b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2444166380 con 0x7f24440c8200 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24441597b0 con 0x7f24441527f0 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f244a000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2444166500 con 0x7f2444152bd0 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2443400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24440c8200 0x7f244414a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2443400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24440c8200 0x7f244414a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46692/0 (socket says 172.21.15.113:46692) 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2443400640 1 -- 172.21.15.113:0/2488962320 learned_addr learned my addr 172.21.15.113:0/2488962320 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2443e00640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2444152bd0 0x7f24440b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2442a00640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24441527f0 0x7f244414d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2443400640 1 -- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2444152bd0 msgr2=0x7f24440b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2443400640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2444152bd0 0x7f24440b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2443400640 1 -- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24441527f0 msgr2=0x7f244414d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:23.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2443400640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24441527f0 0x7f244414d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:23.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.781+0000 7f2443400640 1 -- 172.21.15.113:0/2488962320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244414d630 con 0x7f24440c8200 2024-03-04T08:04:23.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f2443400640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24440c8200 0x7f244414a6e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f243400c780 tx=0x7f243400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:23.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f243b400640 1 -- 172.21.15.113:0/2488962320 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434005cf0 con 0x7f24440c8200 2024-03-04T08:04:23.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f243b400640 1 -- 172.21.15.113:0/2488962320 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243400ce30 con 0x7f24440c8200 2024-03-04T08:04:23.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f243b400640 1 -- 172.21.15.113:0/2488962320 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434007290 con 0x7f24440c8200 2024-03-04T08:04:23.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f244a000640 1 -- 172.21.15.113:0/2488962320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24441f5960 con 0x7f24440c8200 2024-03-04T08:04:23.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f244a000640 1 -- 172.21.15.113:0/2488962320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24441f1c10 con 0x7f24440c8200 2024-03-04T08:04:23.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f244a000640 1 -- 172.21.15.113:0/2488962320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2444075c00 con 0x7f24440c8200 2024-03-04T08:04:23.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f243b400640 1 -- 172.21.15.113:0/2488962320 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2434007430 con 0x7f24440c8200 2024-03-04T08:04:23.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f243b400640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2424066f60 0x7f2424069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:23.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f243b400640 1 -- 172.21.15.113:0/2488962320 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7f2434085400 con 0x7f24440c8200 2024-03-04T08:04:23.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.785+0000 7f2442a00640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2424066f60 0x7f2424069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:23.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.789+0000 7f2442a00640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2424066f60 0x7f2424069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f24300ad000 tx=0x7f24300a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:23.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.793+0000 7f243b400640 1 -- 172.21.15.113:0/2488962320 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2434007720 con 0x7f24440c8200 2024-03-04T08:04:23.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.985+0000 7f244a000640 1 -- 172.21.15.113:0/2488962320 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2444145700 con 0x7f24440c8200 2024-03-04T08:04:23.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.985+0000 7f243b400640 1 -- 172.21.15.113:0/2488962320 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+6623 (secure 0 0 0) 0x7f2434055800 con 0x7f24440c8200 2024-03-04T08:04:23.986 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:23.986 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":113,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7119,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:19.898691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807,6534]},"id":1}]} 2024-03-04T08:04:23.986 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 113 2024-03-04T08:04:23.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 -- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2424066f60 msgr2=0x7f2424069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:23.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2424066f60 0x7f2424069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f24300ad000 tx=0x7f24300a6000 comp rx=0 tx=0).stop 2024-03-04T08:04:23.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 -- 172.21.15.113:0/2488962320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24440c8200 msgr2=0x7f244414a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:23.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24440c8200 0x7f244414a6e0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f243400c780 tx=0x7f243400cc40 comp rx=0 tx=0).stop 2024-03-04T08:04:23.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 -- 172.21.15.113:0/2488962320 shutdown_connections 2024-03-04T08:04:23.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2444152bd0 0x7f24440b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:23.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24441527f0 0x7f244414d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:23.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2424066f60 0x7f2424069420 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:23.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 --2- 172.21.15.113:0/2488962320 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24440c8200 0x7f244414a6e0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:23.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 -- 172.21.15.113:0/2488962320 >> 172.21.15.113:0/2488962320 conn(0x7f244406e820 msgr2=0x7f2444072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:23.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 -- 172.21.15.113:0/2488962320 shutdown_connections 2024-03-04T08:04:23.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:23.989+0000 7f244a000640 1 -- 172.21.15.113:0/2488962320 wait complete. 2024-03-04T08:04:24.006 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:04:26.007 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:26.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.145+0000 7fc73d600640 1 Processor -- start 2024-03-04T08:04:26.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.145+0000 7fc73d600640 1 -- start start 2024-03-04T08:04:26.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.145+0000 7fc73d600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc738157f80 0x7fc73806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:26.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.145+0000 7fc73d600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73806df00 con 0x7fc7380c8a90 2024-03-04T08:04:26.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.145+0000 7fc73d600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7380ce1a0 con 0x7fc7380c9ce0 2024-03-04T08:04:26.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.145+0000 7fc73d600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73806e1e0 con 0x7fc738157f80 2024-03-04T08:04:26.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.145+0000 7fc737e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc738157f80 0x7fc73806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:26.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.145+0000 7fc72fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc7380c8a90 0x7fc73806ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53214/0 (socket says 172.21.15.113:53214) 2024-03-04T08:04:26.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.145+0000 7fc72fe00640 1 -- 172.21.15.113:0/4013889130 learned_addr learned my addr 172.21.15.113:0/4013889130 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.149+0000 7fc737400640 1 -- 172.21.15.113:0/4013889130 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920724203 0 0) 0x7fc73806df00 con 0x7fc7380c8a90 2024-03-04T08:04:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.149+0000 7fc737400640 1 -- 172.21.15.113:0/4013889130 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc71c003530 con 0x7fc7380c8a90 2024-03-04T08:04:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.149+0000 7fc737e00640 1 -- 172.21.15.113:0/4013889130 >> v1:172.21.15.136:6789/0 conn(0x7fc7380c9ce0 legacy=0x7fc7380ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.149+0000 7fc737e00640 1 -- 172.21.15.113:0/4013889130 >> v1:172.21.15.113:6789/0 conn(0x7fc7380c8a90 legacy=0x7fc73806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.149+0000 7fc737e00640 1 -- 172.21.15.113:0/4013889130 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7380ce480 con 0x7fc738157f80 2024-03-04T08:04:26.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.149+0000 7fc737e00640 1 --2- 172.21.15.113:0/4013889130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc738157f80 0x7fc73806c170 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc7280097c0 tx=0x7fc72801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01152fda27d6dc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:26.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.149+0000 7fc737400640 1 -- 172.21.15.113:0/4013889130 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7280b0070 con 0x7fc738157f80 2024-03-04T08:04:26.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.149+0000 7fc737400640 1 -- 172.21.15.113:0/4013889130 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7280ab600 con 0x7fc738157f80 2024-03-04T08:04:26.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.149+0000 7fc737400640 1 -- 172.21.15.113:0/4013889130 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7280ab9d0 con 0x7fc738157f80 2024-03-04T08:04:26.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 -- 172.21.15.113:0/4013889130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc738157f80 msgr2=0x7fc73806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:26.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 --2- 172.21.15.113:0/4013889130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc738157f80 0x7fc73806c170 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc7280097c0 tx=0x7fc72801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:04:26.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 -- 172.21.15.113:0/4013889130 shutdown_connections 2024-03-04T08:04:26.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 --2- 172.21.15.113:0/4013889130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc738157f80 0x7fc73806c170 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:26.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 -- 172.21.15.113:0/4013889130 >> 172.21.15.113:0/4013889130 conn(0x7fc738072f40 msgr2=0x7fc7380706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:26.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 -- 172.21.15.113:0/4013889130 shutdown_connections 2024-03-04T08:04:26.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 -- 172.21.15.113:0/4013889130 wait complete. 2024-03-04T08:04:26.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 Processor -- start 2024-03-04T08:04:26.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 -- start start 2024-03-04T08:04:26.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7380c8a90 0x7fc7381f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:26.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7380c9ce0 0x7fc738158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:26.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc738157f80 0x7fc738147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:26.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73806fdb0 con 0x7fc7380c9ce0 2024-03-04T08:04:26.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73806fc30 con 0x7fc7380c8a90 2024-03-04T08:04:26.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73d600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73806f490 con 0x7fc738157f80 2024-03-04T08:04:26.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc72fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7380c9ce0 0x7fc738158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:26.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc72fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7380c9ce0 0x7fc738158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46712/0 (socket says 172.21.15.113:46712) 2024-03-04T08:04:26.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc72fe00640 1 -- 172.21.15.113:0/2757259562 learned_addr learned my addr 172.21.15.113:0/2757259562 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc737e00640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7380c8a90 0x7fc7381f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc73cc00640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc738157f80 0x7fc738147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc72fe00640 1 -- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc738157f80 msgr2=0x7fc738147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc72fe00640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc738157f80 0x7fc738147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc72fe00640 1 -- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7380c8a90 msgr2=0x7fc7381f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc72fe00640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7380c8a90 0x7fc7381f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc72fe00640 1 -- 172.21.15.113:0/2757259562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7380c81c0 con 0x7fc7380c9ce0 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc737e00640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7380c8a90 0x7fc7381f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.153+0000 7fc72fe00640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7380c9ce0 0x7fc738158810 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc72000c450 tx=0x7fc72000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.157+0000 7fc734c00640 1 -- 172.21.15.113:0/2757259562 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720017070 con 0x7fc7380c9ce0 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.157+0000 7fc734c00640 1 -- 172.21.15.113:0/2757259562 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc720005cf0 con 0x7fc7380c9ce0 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.157+0000 7fc734c00640 1 -- 172.21.15.113:0/2757259562 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200072d0 con 0x7fc7380c9ce0 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.157+0000 7fc73d600640 1 -- 172.21.15.113:0/2757259562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73814b2f0 con 0x7fc7380c9ce0 2024-03-04T08:04:26.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.157+0000 7fc73d600640 1 -- 172.21.15.113:0/2757259562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7380c8480 con 0x7fc7380c9ce0 2024-03-04T08:04:26.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.157+0000 7fc73d600640 1 -- 172.21.15.113:0/2757259562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc738075a80 con 0x7fc7380c9ce0 2024-03-04T08:04:26.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.157+0000 7fc734c00640 1 -- 172.21.15.113:0/2757259562 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc72001a070 con 0x7fc7380c9ce0 2024-03-04T08:04:26.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.157+0000 7fc734c00640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc714066f60 0x7fc714069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:26.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.157+0000 7fc734c00640 1 -- 172.21.15.113:0/2757259562 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7fc7200848d0 con 0x7fc7380c9ce0 2024-03-04T08:04:26.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.157+0000 7fc737e00640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc714066f60 0x7fc714069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:26.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.161+0000 7fc737e00640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc714066f60 0x7fc714069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc7280ad000 tx=0x7fc728008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.161+0000 7fc734c00640 1 -- 172.21.15.113:0/2757259562 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc720054ca0 con 0x7fc7380c9ce0 2024-03-04T08:04:26.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.409+0000 7fc73d600640 1 -- 172.21.15.113:0/2757259562 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7380c97b0 con 0x7fc7380c9ce0 2024-03-04T08:04:26.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.409+0000 7fc734c00640 1 -- 172.21.15.113:0/2757259562 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+6623 (secure 0 0 0) 0x7fc720058d40 con 0x7fc7380c9ce0 2024-03-04T08:04:26.410 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:26.410 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":113,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7119,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:19.898691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807,6534]},"id":1}]} 2024-03-04T08:04:26.410 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 113 2024-03-04T08:04:26.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 -- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc714066f60 msgr2=0x7fc714069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:26.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc714066f60 0x7fc714069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc7280ad000 tx=0x7fc728008040 comp rx=0 tx=0).stop 2024-03-04T08:04:26.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 -- 172.21.15.113:0/2757259562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7380c9ce0 msgr2=0x7fc738158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:26.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7380c9ce0 0x7fc738158810 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc72000c450 tx=0x7fc72000c910 comp rx=0 tx=0).stop 2024-03-04T08:04:26.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 -- 172.21.15.113:0/2757259562 shutdown_connections 2024-03-04T08:04:26.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc738157f80 0x7fc738147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:26.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc714066f60 0x7fc714069420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:26.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7380c9ce0 0x7fc738158810 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:26.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 --2- 172.21.15.113:0/2757259562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7380c8a90 0x7fc7381f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:26.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 -- 172.21.15.113:0/2757259562 >> 172.21.15.113:0/2757259562 conn(0x7fc738072f40 msgr2=0x7fc7381562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:26.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 -- 172.21.15.113:0/2757259562 shutdown_connections 2024-03-04T08:04:26.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:26.413+0000 7fc73d600640 1 -- 172.21.15.113:0/2757259562 wait complete. 2024-03-04T08:04:26.433 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:04:28.434 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.569+0000 7ffba2600640 1 Processor -- start 2024-03-04T08:04:28.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.569+0000 7ffba2600640 1 -- start start 2024-03-04T08:04:28.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffba2600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffb9c06ff20 0x7ffb9c15bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:28.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffba2600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb9c0c8b90 0x7ffb9c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:28.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffba2600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9c152e30 con 0x7ffb9c152a50 2024-03-04T08:04:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffba2600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9c0c31b0 con 0x7ffb9c0c8b90 2024-03-04T08:04:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffba2600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9c06f840 con 0x7ffb9c06ff20 2024-03-04T08:04:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffb9be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb9c0c8b90 0x7ffb9c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffb9b400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ffb9c152a50 0x7ffb9c0ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35310/0 (socket says 172.21.15.113:35310) 2024-03-04T08:04:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffb9b400640 1 -- 172.21.15.113:0/1604548406 learned_addr learned my addr 172.21.15.113:0/1604548406 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffb93e00640 1 --2- 172.21.15.113:0/1604548406 >> v2:172.21.15.136:3300/0 conn(0x7ffb9c06ff20 0x7ffb9c15bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffb9be00640 1 -- 172.21.15.113:0/1604548406 >> v2:172.21.15.136:3300/0 conn(0x7ffb9c06ff20 msgr2=0x7ffb9c15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffb9be00640 1 --2- 172.21.15.113:0/1604548406 >> v2:172.21.15.136:3300/0 conn(0x7ffb9c06ff20 0x7ffb9c15bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffb9be00640 1 -- 172.21.15.113:0/1604548406 >> v1:172.21.15.113:6789/0 conn(0x7ffb9c152a50 legacy=0x7ffb9c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:28.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.573+0000 7ffb9be00640 1 -- 172.21.15.113:0/1604548406 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c06f280 con 0x7ffb9c0c8b90 2024-03-04T08:04:28.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffb9be00640 1 --2- 172.21.15.113:0/1604548406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c0c8b90 0x7ffb9c15c310 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c06e430 tx=0x7ffb9401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a515aec9d0b85945 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:28.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffb9aa00640 1 -- 172.21.15.113:0/1604548406 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb940b2070 con 0x7ffb9c0c8b90 2024-03-04T08:04:28.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffb9aa00640 1 -- 172.21.15.113:0/1604548406 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb940a5a10 con 0x7ffb9c0c8b90 2024-03-04T08:04:28.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffb9aa00640 1 -- 172.21.15.113:0/1604548406 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb940a5de0 con 0x7ffb9c0c8b90 2024-03-04T08:04:28.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 -- 172.21.15.113:0/1604548406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c0c8b90 msgr2=0x7ffb9c15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:28.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 --2- 172.21.15.113:0/1604548406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c0c8b90 0x7ffb9c15c310 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c06e430 tx=0x7ffb9401c930 comp rx=0 tx=0).stop 2024-03-04T08:04:28.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 -- 172.21.15.113:0/1604548406 shutdown_connections 2024-03-04T08:04:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 --2- 172.21.15.113:0/1604548406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c0c8b90 0x7ffb9c15c310 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 --2- 172.21.15.113:0/1604548406 >> v2:172.21.15.136:3300/0 conn(0x7ffb9c06ff20 0x7ffb9c15bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 -- 172.21.15.113:0/1604548406 >> 172.21.15.113:0/1604548406 conn(0x7ffb9c0734d0 msgr2=0x7ffb9c066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 -- 172.21.15.113:0/1604548406 shutdown_connections 2024-03-04T08:04:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 -- 172.21.15.113:0/1604548406 wait complete. 2024-03-04T08:04:28.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 Processor -- start 2024-03-04T08:04:28.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 -- start start 2024-03-04T08:04:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c06ff20 0x7ffb9c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9c0c8b90 0x7ffb9c14c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb9c152a50 0x7ffb9c158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c0ce9b0 con 0x7ffb9c06ff20 2024-03-04T08:04:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c0ce830 con 0x7ffb9c0c8b90 2024-03-04T08:04:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffba2600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c165880 con 0x7ffb9c152a50 2024-03-04T08:04:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.577+0000 7ffb9b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c06ff20 0x7ffb9c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:28.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb9b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c06ff20 0x7ffb9c154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51326/0 (socket says 172.21.15.113:51326) 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb9b400640 1 -- 172.21.15.113:0/3730401933 learned_addr learned my addr 172.21.15.113:0/3730401933 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb9be00640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb9c152a50 0x7ffb9c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb93e00640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9c0c8b90 0x7ffb9c14c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb9b400640 1 -- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb9c152a50 msgr2=0x7ffb9c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb9b400640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb9c152a50 0x7ffb9c158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb9b400640 1 -- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9c0c8b90 msgr2=0x7ffb9c14c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb9b400640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9c0c8b90 0x7ffb9c14c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb9b400640 1 -- 172.21.15.113:0/3730401933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c14f870 con 0x7ffb9c06ff20 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb9b400640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c06ff20 0x7ffb9c154e50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00c450 tx=0x7ffb8c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb93400640 1 -- 172.21.15.113:0/3730401933 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c017070 con 0x7ffb9c06ff20 2024-03-04T08:04:28.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb93400640 1 -- 172.21.15.113:0/3730401933 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c005cf0 con 0x7ffb9c06ff20 2024-03-04T08:04:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffba2600640 1 -- 172.21.15.113:0/3730401933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c1579a0 con 0x7ffb9c06ff20 2024-03-04T08:04:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb93400640 1 -- 172.21.15.113:0/3730401933 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0072f0 con 0x7ffb9c06ff20 2024-03-04T08:04:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffba2600640 1 -- 172.21.15.113:0/3730401933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c14ed90 con 0x7ffb9c06ff20 2024-03-04T08:04:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffba2600640 1 -- 172.21.15.113:0/3730401933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c075c00 con 0x7ffb9c06ff20 2024-03-04T08:04:28.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.581+0000 7ffb93400640 1 -- 172.21.15.113:0/3730401933 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb8c01a070 con 0x7ffb9c06ff20 2024-03-04T08:04:28.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.585+0000 7ffb93400640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb78066f60 0x7ffb78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:28.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.585+0000 7ffb93400640 1 -- 172.21.15.113:0/3730401933 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7201+0+0 (secure 0 0 0) 0x7ffb8c0849f0 con 0x7ffb9c06ff20 2024-03-04T08:04:28.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.585+0000 7ffb93e00640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb78066f60 0x7ffb78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:28.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.585+0000 7ffb93e00640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb78066f60 0x7ffb78069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c15a6a0 tx=0x7ffb84008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:28.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.589+0000 7ffb93400640 1 -- 172.21.15.113:0/3730401933 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ffb8c055410 con 0x7ffb9c06ff20 2024-03-04T08:04:28.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.781+0000 7ffba2600640 1 -- 172.21.15.113:0/3730401933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb9c162e40 con 0x7ffb9c06ff20 2024-03-04T08:04:28.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.781+0000 7ffb93400640 1 -- 172.21.15.113:0/3730401933 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+6623 (secure 0 0 0) 0x7ffb8c054d10 con 0x7ffb9c06ff20 2024-03-04T08:04:28.780 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:28.780 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":113,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7119,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":113}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:03:19.898691+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":6534,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6534":{"gid":6534,"name":"e","rank":1,"incarnation":89,"state":"up:active","state_seq":24,"addr":"172.21.15.113:6837/4197507712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":4197507712},{"type":"v1","addr":"172.21.15.113:6837","nonce":4197507712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807,6534]},"id":1}]} 2024-03-04T08:04:28.781 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 113 2024-03-04T08:04:28.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 -- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb78066f60 msgr2=0x7ffb78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:28.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb78066f60 0x7ffb78069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c15a6a0 tx=0x7ffb84008040 comp rx=0 tx=0).stop 2024-03-04T08:04:28.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 -- 172.21.15.113:0/3730401933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c06ff20 msgr2=0x7ffb9c154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:28.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c06ff20 0x7ffb9c154e50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00c450 tx=0x7ffb8c00c910 comp rx=0 tx=0).stop 2024-03-04T08:04:28.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 -- 172.21.15.113:0/3730401933 shutdown_connections 2024-03-04T08:04:28.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb9c152a50 0x7ffb9c158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:28.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9c0c8b90 0x7ffb9c14c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:28.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb78066f60 0x7ffb78069420 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:28.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 --2- 172.21.15.113:0/3730401933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb9c06ff20 0x7ffb9c154e50 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:28.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 -- 172.21.15.113:0/3730401933 >> 172.21.15.113:0/3730401933 conn(0x7ffb9c0734d0 msgr2=0x7ffb9c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:28.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 -- 172.21.15.113:0/3730401933 shutdown_connections 2024-03-04T08:04:28.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:28.785+0000 7ffba2600640 1 -- 172.21.15.113:0/3730401933 wait complete. 2024-03-04T08:04:28.802 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:04:30.803 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:30.851 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 59 -eq 48 ] 2024-03-04T08:04:30.851 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.59 2024-03-04T08:04:30.852 INFO:tasks.workunit.client.0.smithi113.stdout:v0.59 2024-03-04T08:04:30.852 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.59 2024-03-04T08:04:30.852 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.59 2024-03-04T08:04:30.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.945+0000 7f28ddc00640 1 Processor -- start 2024-03-04T08:04:30.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.945+0000 7f28ddc00640 1 -- start start 2024-03-04T08:04:30.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28ddc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f28d80c8a90 0x7f28d815e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:30.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28ddc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d815ea60 0x7f28d80ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:30.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28ddc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d8070fd0 con 0x7f28d8157df0 2024-03-04T08:04:30.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28ddc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d815e090 con 0x7f28d80c8a90 2024-03-04T08:04:30.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28ddc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d806dfd0 con 0x7f28d815ea60 2024-03-04T08:04:30.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28d7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f28d8157df0 0x7f28d806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35318/0 (socket says 172.21.15.113:35318) 2024-03-04T08:04:30.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28d7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f28d80c8a90 0x7f28d815e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:30.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28d7e00640 1 -- 172.21.15.113:0/840671994 learned_addr learned my addr 172.21.15.113:0/840671994 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:30.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28dd200640 1 --2- 172.21.15.113:0/840671994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d815ea60 0x7f28d80ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:30.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28d7400640 1 -- 172.21.15.113:0/840671994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d815ea60 msgr2=0x7f28d80ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:30.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28d7400640 1 --2- 172.21.15.113:0/840671994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d815ea60 0x7f28d80ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:30.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28d7400640 1 -- 172.21.15.113:0/840671994 >> v1:172.21.15.113:6789/0 conn(0x7f28d8157df0 legacy=0x7f28d806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:30.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28d7400640 1 -- 172.21.15.113:0/840671994 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d806fcd0 con 0x7f28d80c8a90 2024-03-04T08:04:30.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28d6a00640 1 -- 172.21.15.113:0/840671994 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180043685 0 0) 0x7f28c8001120 con 0x7f28d8157df0 2024-03-04T08:04:30.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.949+0000 7f28d7400640 1 --2- 172.21.15.113:0/840671994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d80c8a90 0x7f28d815e520 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f28c40097c0 tx=0x7f28c401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1e2cfbdbbf5af5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:30.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28d6a00640 1 -- 172.21.15.113:0/840671994 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c40b0070 con 0x7f28d80c8a90 2024-03-04T08:04:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28d6a00640 1 -- 172.21.15.113:0/840671994 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c40ab600 con 0x7f28d80c8a90 2024-03-04T08:04:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28d6a00640 1 -- 172.21.15.113:0/840671994 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c40ab9d0 con 0x7f28d80c8a90 2024-03-04T08:04:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 -- 172.21.15.113:0/840671994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d80c8a90 msgr2=0x7f28d815e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 --2- 172.21.15.113:0/840671994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d80c8a90 0x7f28d815e520 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f28c40097c0 tx=0x7f28c401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:04:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 -- 172.21.15.113:0/840671994 shutdown_connections 2024-03-04T08:04:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 --2- 172.21.15.113:0/840671994 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d815ea60 0x7f28d80ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 --2- 172.21.15.113:0/840671994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d80c8a90 0x7f28d815e520 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 -- 172.21.15.113:0/840671994 >> 172.21.15.113:0/840671994 conn(0x7f28d8072f40 msgr2=0x7f28d80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 -- 172.21.15.113:0/840671994 shutdown_connections 2024-03-04T08:04:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 -- 172.21.15.113:0/840671994 wait complete. 2024-03-04T08:04:30.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 Processor -- start 2024-03-04T08:04:30.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 -- start start 2024-03-04T08:04:30.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28d80c8a90 0x7f28d8159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:30.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d8157df0 0x7f28d8154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:30.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d815ea60 0x7f28d8155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d806dd80 con 0x7f28d815ea60 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d806dc00 con 0x7f28d80c8a90 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28ddc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d81581d0 con 0x7f28d8157df0 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28dd200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d815ea60 0x7f28d8155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28dd200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d815ea60 0x7f28d8155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51344/0 (socket says 172.21.15.113:51344) 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28d7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28d80c8a90 0x7f28d8159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28dd200640 1 -- 172.21.15.113:0/2953864617 learned_addr learned my addr 172.21.15.113:0/2953864617 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.953+0000 7f28d7400640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d8157df0 0x7f28d8154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28dd200640 1 -- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d8157df0 msgr2=0x7f28d8154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28dd200640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d8157df0 0x7f28d8154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28dd200640 1 -- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28d80c8a90 msgr2=0x7f28d8159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28dd200640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28d80c8a90 0x7f28d8159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28dd200640 1 -- 172.21.15.113:0/2953864617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d814d230 con 0x7f28d815ea60 2024-03-04T08:04:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28d7400640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d8157df0 0x7f28d8154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:04:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28dd200640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d815ea60 0x7f28d8155410 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009fd0 tx=0x7f28d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28cfe00640 1 -- 172.21.15.113:0/2953864617 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0014420 con 0x7f28d815ea60 2024-03-04T08:04:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28cfe00640 1 -- 172.21.15.113:0/2953864617 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d0019070 con 0x7f28d815ea60 2024-03-04T08:04:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28cfe00640 1 -- 172.21.15.113:0/2953864617 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0014800 con 0x7f28d815ea60 2024-03-04T08:04:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28ddc00640 1 -- 172.21.15.113:0/2953864617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d81572a0 con 0x7f28d815ea60 2024-03-04T08:04:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28ddc00640 1 -- 172.21.15.113:0/2953864617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d815cf70 con 0x7f28d815ea60 2024-03-04T08:04:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28ddc00640 1 -- 172.21.15.113:0/2953864617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d8075a80 con 0x7f28d815ea60 2024-03-04T08:04:30.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28cfe00640 1 -- 172.21.15.113:0/2953864617 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28d00149a0 con 0x7f28d815ea60 2024-03-04T08:04:30.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28cfe00640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28b8066f60 0x7f28b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:30.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28cfe00640 1 -- 172.21.15.113:0/2953864617 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f28d0085f00 con 0x7f28d815ea60 2024-03-04T08:04:30.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.957+0000 7f28d7e00640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28b8066f60 0x7f28b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:30.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.961+0000 7f28d7e00640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28b8066f60 0x7f28b8069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f28c8000f70 tx=0x7f28c8007b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:30.961+0000 7f28cfe00640 1 -- 172.21.15.113:0/2953864617 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f28d00567b0 con 0x7f28d815ea60 2024-03-04T08:04:31.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.161+0000 7f28ddc00640 1 -- 172.21.15.113:0/2953864617 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28d80c9610 con 0x7f28d815ea60 2024-03-04T08:04:31.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.161+0000 7f28cfe00640 1 -- 172.21.15.113:0/2953864617 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f28d00560b0 con 0x7f28d815ea60 2024-03-04T08:04:31.161 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:31.161 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":114,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:30.830470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6807,6925]},"id":1}]} 2024-03-04T08:04:31.161 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 114 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 -- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28b8066f60 msgr2=0x7f28b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28b8066f60 0x7f28b8069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f28c8000f70 tx=0x7f28c8007b60 comp rx=0 tx=0).stop 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 -- 172.21.15.113:0/2953864617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d815ea60 msgr2=0x7f28d8155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d815ea60 0x7f28d8155410 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009fd0 tx=0x7f28d000e4b0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 -- 172.21.15.113:0/2953864617 shutdown_connections 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f28b8066f60 0x7f28b8069420 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28d815ea60 0x7f28d8155410 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28d8157df0 0x7f28d8154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 --2- 172.21.15.113:0/2953864617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28d80c8a90 0x7f28d8159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 -- 172.21.15.113:0/2953864617 >> 172.21.15.113:0/2953864617 conn(0x7f28d8072f40 msgr2=0x7f28d80ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 -- 172.21.15.113:0/2953864617 shutdown_connections 2024-03-04T08:04:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.165+0000 7f28ddc00640 1 -- 172.21.15.113:0/2953864617 wait complete. 2024-03-04T08:04:31.182 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-03-04T08:04:31.182 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:04:31.182 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:31.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.361+0000 7fe33ec00640 1 Processor -- start 2024-03-04T08:04:31.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.361+0000 7fe33ec00640 1 -- start start 2024-03-04T08:04:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe33ec00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe338152d00 0x7fe3380ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe33ec00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3380ca0d0 0x7fe3380ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe33ec00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3380c5470 con 0x7fe3380703e0 2024-03-04T08:04:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe33ec00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3380c5220 con 0x7fe3380ca0d0 2024-03-04T08:04:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe33ec00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3380cba40 con 0x7fe338152d00 2024-03-04T08:04:31.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe32fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3380ca0d0 0x7fe3380ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:31.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe32fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe3380ca0d0 0x7fe3380ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51354/0 (socket says 172.21.15.113:51354) 2024-03-04T08:04:31.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe33cc00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe3380703e0 0x7fe33815be30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35328/0 (socket says 172.21.15.113:35328) 2024-03-04T08:04:31.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe32fe00640 1 -- 172.21.15.113:0/1290932459 learned_addr learned my addr 172.21.15.113:0/1290932459 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:31.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe337e00640 1 --2- 172.21.15.113:0/1290932459 >> v2:172.21.15.136:3300/0 conn(0x7fe338152d00 0x7fe3380ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:31.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe32fe00640 1 -- 172.21.15.113:0/1290932459 >> v2:172.21.15.136:3300/0 conn(0x7fe338152d00 msgr2=0x7fe3380ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:31.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe32fe00640 1 --2- 172.21.15.113:0/1290932459 >> v2:172.21.15.136:3300/0 conn(0x7fe338152d00 0x7fe3380ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe32fe00640 1 -- 172.21.15.113:0/1290932459 >> v1:172.21.15.113:6789/0 conn(0x7fe3380703e0 legacy=0x7fe33815be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:31.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.365+0000 7fe32fe00640 1 -- 172.21.15.113:0/1290932459 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33806f690 con 0x7fe3380ca0d0 2024-03-04T08:04:31.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe32fe00640 1 --2- 172.21.15.113:0/1290932459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 0x7fe3380ce3c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe3380cf050 tx=0x7fe32001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269bbf575bef7f0c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:31.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe337400640 1 -- 172.21.15.113:0/1290932459 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3200b0070 con 0x7fe3380ca0d0 2024-03-04T08:04:31.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe337400640 1 -- 172.21.15.113:0/1290932459 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3200ab600 con 0x7fe3380ca0d0 2024-03-04T08:04:31.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe337400640 1 -- 172.21.15.113:0/1290932459 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3200ab9d0 con 0x7fe3380ca0d0 2024-03-04T08:04:31.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe33ec00640 1 -- 172.21.15.113:0/1290932459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 msgr2=0x7fe3380ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:31.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe33ec00640 1 --2- 172.21.15.113:0/1290932459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 0x7fe3380ce3c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe3380cf050 tx=0x7fe32001c490 comp rx=0 tx=0).stop 2024-03-04T08:04:31.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe33ec00640 1 -- 172.21.15.113:0/1290932459 shutdown_connections 2024-03-04T08:04:31.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe33ec00640 1 --2- 172.21.15.113:0/1290932459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 0x7fe3380ce3c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe33ec00640 1 --2- 172.21.15.113:0/1290932459 >> v2:172.21.15.136:3300/0 conn(0x7fe338152d00 0x7fe3380ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe33ec00640 1 -- 172.21.15.113:0/1290932459 >> 172.21.15.113:0/1290932459 conn(0x7fe338066db0 msgr2=0x7fe3380671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:31.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe33ec00640 1 -- 172.21.15.113:0/1290932459 shutdown_connections 2024-03-04T08:04:31.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.369+0000 7fe33ec00640 1 -- 172.21.15.113:0/1290932459 wait complete. 2024-03-04T08:04:31.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33ec00640 1 Processor -- start 2024-03-04T08:04:31.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33ec00640 1 -- start start 2024-03-04T08:04:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33ec00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3380703e0 0x7fe3380c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33ec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 0x7fe3380cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33ec00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe338152d00 0x7fe3381539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33ec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380cacb0 con 0x7fe3380ca0d0 2024-03-04T08:04:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33ec00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380cab30 con 0x7fe338152d00 2024-03-04T08:04:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33ec00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3381661c0 con 0x7fe3380703e0 2024-03-04T08:04:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe32fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 0x7fe3380cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:31.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe32fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 0x7fe3380cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51362/0 (socket says 172.21.15.113:51362) 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe32fe00640 1 -- 172.21.15.113:0/2021113531 learned_addr learned my addr 172.21.15.113:0/2021113531 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe337e00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3380703e0 0x7fe3380c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33cc00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe338152d00 0x7fe3381539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe32fe00640 1 -- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3380703e0 msgr2=0x7fe3380c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe32fe00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3380703e0 0x7fe3380c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe32fe00640 1 -- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe338152d00 msgr2=0x7fe3381539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe32fe00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe338152d00 0x7fe3381539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe32fe00640 1 -- 172.21.15.113:0/2021113531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33814ca80 con 0x7fe3380ca0d0 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe337e00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3380703e0 0x7fe3380c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33cc00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe338152d00 0x7fe3381539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe32fe00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 0x7fe3380cd510 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe3200ae040 tx=0x7fe320019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe334c00640 1 -- 172.21.15.113:0/2021113531 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3200b0040 con 0x7fe3380ca0d0 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe334c00640 1 -- 172.21.15.113:0/2021113531 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3200b9070 con 0x7fe3380ca0d0 2024-03-04T08:04:31.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33ec00640 1 -- 172.21.15.113:0/2021113531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe338159220 con 0x7fe3380ca0d0 2024-03-04T08:04:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe334c00640 1 -- 172.21.15.113:0/2021113531 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3200b4680 con 0x7fe3380ca0d0 2024-03-04T08:04:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.373+0000 7fe33ec00640 1 -- 172.21.15.113:0/2021113531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3381f2440 con 0x7fe3380ca0d0 2024-03-04T08:04:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.377+0000 7fe33ec00640 1 -- 172.21.15.113:0/2021113531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3381f2230 con 0x7fe3380ca0d0 2024-03-04T08:04:31.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.377+0000 7fe334c00640 1 -- 172.21.15.113:0/2021113531 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3200b4820 con 0x7fe3380ca0d0 2024-03-04T08:04:31.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.377+0000 7fe334c00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 0x7fe314069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:31.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.377+0000 7fe334c00640 1 -- 172.21.15.113:0/2021113531 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7fe32011f360 con 0x7fe3380ca0d0 2024-03-04T08:04:31.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.377+0000 7fe337e00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 0x7fe314069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:31.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.377+0000 7fe337e00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 0x7fe314069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe32800b9f0 tx=0x7fe328008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:31.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.381+0000 7fe334c00640 1 -- 172.21.15.113:0/2021113531 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe320016110 con 0x7fe3380ca0d0 2024-03-04T08:04:31.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.573+0000 7fe33ec00640 1 -- 172.21.15.113:0/2021113531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe338075c00 con 0x7fe3380ca0d0 2024-03-04T08:04:31.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.573+0000 7fe334c00640 1 -- 172.21.15.113:0/2021113531 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+5823 (secure 0 0 0) 0x7fe3200efd30 con 0x7fe3380ca0d0 2024-03-04T08:04:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:31.573 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":114,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:30.830470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6807,6925]},"id":1}]} 2024-03-04T08:04:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 114 2024-03-04T08:04:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 -- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 msgr2=0x7fe314069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 0x7fe314069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe32800b9f0 tx=0x7fe328008040 comp rx=0 tx=0).stop 2024-03-04T08:04:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 -- 172.21.15.113:0/2021113531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 msgr2=0x7fe3380cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 0x7fe3380cd510 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fe3200ae040 tx=0x7fe320019230 comp rx=0 tx=0).stop 2024-03-04T08:04:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 -- 172.21.15.113:0/2021113531 shutdown_connections 2024-03-04T08:04:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe338152d00 0x7fe3381539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe314066f60 0x7fe314069420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3380ca0d0 0x7fe3380cd510 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 --2- 172.21.15.113:0/2021113531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe3380703e0 0x7fe3380c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:31.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 -- 172.21.15.113:0/2021113531 >> 172.21.15.113:0/2021113531 conn(0x7fe338066db0 msgr2=0x7fe33814d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:31.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 -- 172.21.15.113:0/2021113531 shutdown_connections 2024-03-04T08:04:31.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:31.577+0000 7fe33ec00640 1 -- 172.21.15.113:0/2021113531 wait complete. 2024-03-04T08:04:31.595 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 114, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 114, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6807": { "addr": "172.21.15.113:6836/1035982016", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1035982016, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1035982016, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6807, "incarnation": 101, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 14 }, "gid_6925": { "addr": "172.21.15.136:6835/2522634751", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2522634751, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2522634751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6925, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 23 }, "gid_7119": { "addr": "172.21.15.113:6833/3825257303", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 3825257303, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 3825257303, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7119, "incarnation": 114, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 65, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:04:30.830470+0000", "qdb_cluster": [ 6807, 6925 ], "qdb_leader": 6925, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6925, "mds_1": 7119, "mds_2": 6807 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:04:33.596 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:33.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40de600640 1 Processor -- start 2024-03-04T08:04:33.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40de600640 1 -- start start 2024-03-04T08:04:33.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40de600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f40d81591a0 0x7f40d815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:33.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40de600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40d806cde0 0x7f40d80cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:33.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40de600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40d815f430 0x7f40d80cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40de600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d80c8c20 con 0x7f40d806cde0 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40de600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d806b3d0 con 0x7f40d81591a0 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40de600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80c8e50 con 0x7f40d815f430 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40d806cde0 0x7f40d80cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f40d806cde0 0x7f40d80cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51372/0 (socket says 172.21.15.113:51372) 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d6a00640 1 -- 172.21.15.113:0/1014198788 learned_addr learned my addr 172.21.15.113:0/1014198788 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d7e00640 1 --2- 172.21.15.113:0/1014198788 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40d815f430 0x7f40d80cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d7400640 1 --2- 172.21.15.113:0/1014198788 >> v2:172.21.15.136:3300/0 conn(0x7f40d81591a0 0x7f40d815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d6a00640 1 -- 172.21.15.113:0/1014198788 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40d815f430 msgr2=0x7f40d80cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d6a00640 1 --2- 172.21.15.113:0/1014198788 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40d815f430 0x7f40d80cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d6a00640 1 -- 172.21.15.113:0/1014198788 >> v2:172.21.15.136:3300/0 conn(0x7f40d81591a0 msgr2=0x7f40d815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d6a00640 1 --2- 172.21.15.113:0/1014198788 >> v2:172.21.15.136:3300/0 conn(0x7f40d81591a0 0x7f40d815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d6a00640 1 -- 172.21.15.113:0/1014198788 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d8072bc0 con 0x7f40d806cde0 2024-03-04T08:04:33.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.741+0000 7f40d7400640 1 --2- 172.21.15.113:0/1014198788 >> v2:172.21.15.136:3300/0 conn(0x7f40d81591a0 0x7f40d815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:04:33.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40d6a00640 1 --2- 172.21.15.113:0/1014198788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 0x7f40d80cd010 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f40d80cac10 tx=0x7f40c401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80804b834bbc2c38 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:33.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40d6000640 1 -- 172.21.15.113:0/1014198788 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c40b5070 con 0x7f40d806cde0 2024-03-04T08:04:33.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40d6000640 1 -- 172.21.15.113:0/1014198788 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c40a89d0 con 0x7f40d806cde0 2024-03-04T08:04:33.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40d6000640 1 -- 172.21.15.113:0/1014198788 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c40a8da0 con 0x7f40d806cde0 2024-03-04T08:04:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 -- 172.21.15.113:0/1014198788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 msgr2=0x7f40d80cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 --2- 172.21.15.113:0/1014198788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 0x7f40d80cd010 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f40d80cac10 tx=0x7f40c401ed10 comp rx=0 tx=0).stop 2024-03-04T08:04:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 -- 172.21.15.113:0/1014198788 shutdown_connections 2024-03-04T08:04:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 --2- 172.21.15.113:0/1014198788 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40d815f430 0x7f40d80cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 --2- 172.21.15.113:0/1014198788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 0x7f40d80cd010 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 --2- 172.21.15.113:0/1014198788 >> v2:172.21.15.136:3300/0 conn(0x7f40d81591a0 0x7f40d815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 -- 172.21.15.113:0/1014198788 >> 172.21.15.113:0/1014198788 conn(0x7f40d80671b0 msgr2=0x7f40d806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:33.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 -- 172.21.15.113:0/1014198788 shutdown_connections 2024-03-04T08:04:33.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 -- 172.21.15.113:0/1014198788 wait complete. 2024-03-04T08:04:33.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 Processor -- start 2024-03-04T08:04:33.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.745+0000 7f40de600640 1 -- start start 2024-03-04T08:04:33.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40de600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 0x7f40d80c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:33.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40de600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40d81591a0 0x7f40d8151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:33.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40de600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40d815f430 0x7f40d81516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:33.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40de600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80c7b20 con 0x7f40d806cde0 2024-03-04T08:04:33.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40de600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80728e0 con 0x7f40d815f430 2024-03-04T08:04:33.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40de600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80c7e00 con 0x7f40d81591a0 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 0x7f40d80c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 0x7f40d80c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51376/0 (socket says 172.21.15.113:51376) 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7400640 1 -- 172.21.15.113:0/3098647688 learned_addr learned my addr 172.21.15.113:0/3098647688 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d6a00640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40d81591a0 0x7f40d8151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7e00640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40d815f430 0x7f40d81516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7400640 1 -- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40d81591a0 msgr2=0x7f40d8151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7400640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40d81591a0 0x7f40d8151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7400640 1 -- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40d815f430 msgr2=0x7f40d81516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7400640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40d815f430 0x7f40d81516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7400640 1 -- 172.21.15.113:0/3098647688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d80c47f0 con 0x7f40d806cde0 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7e00640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40d815f430 0x7f40d81516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40d7400640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 0x7f40d80c9380 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e810 tx=0x7f40c800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40cf400640 1 -- 172.21.15.113:0/3098647688 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c800f040 con 0x7f40d806cde0 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40cf400640 1 -- 172.21.15.113:0/3098647688 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c8004030 con 0x7f40d806cde0 2024-03-04T08:04:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40de600640 1 -- 172.21.15.113:0/3098647688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d81f8890 con 0x7f40d806cde0 2024-03-04T08:04:33.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40cf400640 1 -- 172.21.15.113:0/3098647688 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8004400 con 0x7f40d806cde0 2024-03-04T08:04:33.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40de600640 1 -- 172.21.15.113:0/3098647688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d81f8e50 con 0x7f40d806cde0 2024-03-04T08:04:33.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.749+0000 7f40de600640 1 -- 172.21.15.113:0/3098647688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d80c6e60 con 0x7f40d806cde0 2024-03-04T08:04:33.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.753+0000 7f40cf400640 1 -- 172.21.15.113:0/3098647688 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40c800c6d0 con 0x7f40d806cde0 2024-03-04T08:04:33.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.753+0000 7f40cf400640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f40b8066f60 0x7f40b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:33.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.753+0000 7f40cf400640 1 -- 172.21.15.113:0/3098647688 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f40c8085220 con 0x7f40d806cde0 2024-03-04T08:04:33.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.753+0000 7f40d6a00640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f40b8066f60 0x7f40b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.753+0000 7f40d6a00640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f40b8066f60 0x7f40b8069420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f40c40b2000 tx=0x7f40c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:33.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.757+0000 7f40cf400640 1 -- 172.21.15.113:0/3098647688 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f40c8017020 con 0x7f40d806cde0 2024-03-04T08:04:33.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.949+0000 7f40de600640 1 -- 172.21.15.113:0/3098647688 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40d8075c00 con 0x7f40d806cde0 2024-03-04T08:04:33.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40cf400640 1 -- 172.21.15.113:0/3098647688 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f40c80555a0 con 0x7f40d806cde0 2024-03-04T08:04:33.950 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:33.950 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":114,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:30.830470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6807,6925]},"id":1}]} 2024-03-04T08:04:33.950 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 114 2024-03-04T08:04:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 -- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f40b8066f60 msgr2=0x7f40b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f40b8066f60 0x7f40b8069420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f40c40b2000 tx=0x7f40c4009040 comp rx=0 tx=0).stop 2024-03-04T08:04:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 -- 172.21.15.113:0/3098647688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 msgr2=0x7f40d80c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 0x7f40d80c9380 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f40c800e810 tx=0x7f40c800ecd0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 -- 172.21.15.113:0/3098647688 shutdown_connections 2024-03-04T08:04:33.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f40d815f430 0x7f40d81516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f40d81591a0 0x7f40d8151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f40b8066f60 0x7f40b8069420 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 --2- 172.21.15.113:0/3098647688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f40d806cde0 0x7f40d80c9380 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:33.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 -- 172.21.15.113:0/3098647688 >> 172.21.15.113:0/3098647688 conn(0x7f40d80671b0 msgr2=0x7f40d806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:33.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 -- 172.21.15.113:0/3098647688 shutdown_connections 2024-03-04T08:04:33.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:33.953+0000 7f40de600640 1 -- 172.21.15.113:0/3098647688 wait complete. 2024-03-04T08:04:33.973 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:04:34.364 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:04:34.367 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:04:34.510 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 39% (83/211) Updating files: 40% (85/211) Updating files: 41% (87/211) Updating files: 42% (89/211) Updating files: 43% (91/211) Updating files: 44% (93/211) Updating files: 45% (95/211) Updating files: 46% (98/211) Updating files: 47% (100/211) Updating files: 48% (102/211) Updating files: 49% (104/211) Updating files: 50% (106/211) Updating files: 51% (108/211) Updating files: 52% (110/211) Updating files: 53% (112/211) Updating files: 54% (114/211) Updating files: 55% (117/211) Updating files: 56% (119/211) Updating files: 57% (121/211) Updating files: 58% (123/211) Updating files: 59% (125/211) Updating files: 60% (127/211) Updating files: 61% (129/211) Updating files: 62% (131/211) Updating files: 63% (133/211) Updating files: 64% (136/211) Updating files: 65% (138/211) Updating files: 66% (140/211) Updating files: 67% (142/211) Updating files: 68% (144/211) Updating files: 69% (146/211) Updating files: 70% (148/211) Updating files: 71% (150/211) Updating files: 72% (152/211) Updating files: 73% (155/211) Updating files: 74% (157/211) Updating files: 75% (159/211) Updating files: 76% (161/211) Updating files: 77% (163/211) Updating files: 78% (165/211) Updating files: 79% (167/211) Updating files: 80% (169/211) Updating files: 81% (171/211) Updating files: 82% (174/211) Updating files: 83% (176/211) Updating files: 84% (178/211) Updating files: 85% (180/211) Updating files: 86% (182/211) Updating files: 87% (184/211) Updating files: 88% (186/211) Updating files: 89% (188/211) Updating files: 90% (190/211) Updating files: 91% (193/211) Updating files: 92% (195/211) Updating files: 93% (197/211) Updating files: 94% (199/211) Updating files: 95% (201/211) Updating files: 96% (203/211) Updating files: 97% (205/211) Updating files: 98% (207/211) Updating files: 99% (209/211) Updating files: 100% (211/211) Updating files: 100% (211/211), done. 2024-03-04T08:04:34.687 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at cbae6a435c6 v0.59 2024-03-04T08:04:34.694 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.59 2024-03-04T08:04:35.018 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 60 -eq 48 ] 2024-03-04T08:04:35.018 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.60 2024-03-04T08:04:35.018 INFO:tasks.workunit.client.0.smithi113.stdout:v0.60 2024-03-04T08:04:35.019 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.60 2024-03-04T08:04:35.019 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.60 2024-03-04T08:04:35.974 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:36.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.109+0000 7fb404a00640 1 Processor -- start 2024-03-04T08:04:36.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.109+0000 7fb404a00640 1 -- start start 2024-03-04T08:04:36.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb404a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb400158010 0x7fb40006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:36.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb404a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4000ca0e0 con 0x7fb4000ce120 2024-03-04T08:04:36.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb404a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40015aa30 con 0x7fb40006ff20 2024-03-04T08:04:36.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb404a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40006f450 con 0x7fb400158010 2024-03-04T08:04:36.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb3fe800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb4000ce120 0x7fb4001652d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35334/0 (socket says 172.21.15.113:35334) 2024-03-04T08:04:36.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb3fe800640 1 -- 172.21.15.113:0/2073680087 learned_addr learned my addr 172.21.15.113:0/2073680087 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:36.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb3fde00640 1 --2- 172.21.15.113:0/2073680087 >> v2:172.21.15.136:3300/0 conn(0x7fb400158010 0x7fb40006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:36.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb3fd400640 1 -- 172.21.15.113:0/2073680087 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193112456 0 0) 0x7fb4000ca0e0 con 0x7fb4000ce120 2024-03-04T08:04:36.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb3fd400640 1 -- 172.21.15.113:0/2073680087 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e4003530 con 0x7fb4000ce120 2024-03-04T08:04:36.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb3fde00640 1 -- 172.21.15.113:0/2073680087 >> v1:172.21.15.136:6789/0 conn(0x7fb40006ff20 legacy=0x7fb40015b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:36.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb3fde00640 1 -- 172.21.15.113:0/2073680087 >> v1:172.21.15.113:6789/0 conn(0x7fb4000ce120 legacy=0x7fb4001652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:36.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.113+0000 7fb3fde00640 1 -- 172.21.15.113:0/2073680087 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40006c590 con 0x7fb400158010 2024-03-04T08:04:36.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb3fde00640 1 --2- 172.21.15.113:0/2073680087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb400158010 0x7fb40006dd50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00097c0 tx=0x7fb3f001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe37e7c417c66334 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:36.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb3fd400640 1 -- 172.21.15.113:0/2073680087 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00b0070 con 0x7fb400158010 2024-03-04T08:04:36.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb3fd400640 1 -- 172.21.15.113:0/2073680087 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f00ab600 con 0x7fb400158010 2024-03-04T08:04:36.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb3fd400640 1 -- 172.21.15.113:0/2073680087 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00ab9b0 con 0x7fb400158010 2024-03-04T08:04:36.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb404a00640 1 -- 172.21.15.113:0/2073680087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb400158010 msgr2=0x7fb40006dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:36.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb404a00640 1 --2- 172.21.15.113:0/2073680087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb400158010 0x7fb40006dd50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00097c0 tx=0x7fb3f001c4a0 comp rx=0 tx=0).stop 2024-03-04T08:04:36.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb404a00640 1 -- 172.21.15.113:0/2073680087 shutdown_connections 2024-03-04T08:04:36.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb404a00640 1 --2- 172.21.15.113:0/2073680087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb400158010 0x7fb40006dd50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:36.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb404a00640 1 -- 172.21.15.113:0/2073680087 >> 172.21.15.113:0/2073680087 conn(0x7fb4000734d0 msgr2=0x7fb400066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:36.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb404a00640 1 -- 172.21.15.113:0/2073680087 shutdown_connections 2024-03-04T08:04:36.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb404a00640 1 -- 172.21.15.113:0/2073680087 wait complete. 2024-03-04T08:04:36.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb404a00640 1 Processor -- start 2024-03-04T08:04:36.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.117+0000 7fb404a00640 1 -- start start 2024-03-04T08:04:36.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb404a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb40006ff20 0x7fb4000cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:36.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb404a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4000ce120 0x7fb4000cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:36.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb404a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb400158010 0x7fb40013fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb404a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40006e290 con 0x7fb400158010 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb404a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40015ad80 con 0x7fb4000ce120 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb404a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40015b920 con 0x7fb40006ff20 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3fe800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb400158010 0x7fb40013fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3fe800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb400158010 0x7fb40013fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51378/0 (socket says 172.21.15.113:51378) 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3fe800640 1 -- 172.21.15.113:0/4247341002 learned_addr learned my addr 172.21.15.113:0/4247341002 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3f7e00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4000ce120 0x7fb4000cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3fde00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb40006ff20 0x7fb4000cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3fe800640 1 -- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb40006ff20 msgr2=0x7fb4000cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3fe800640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb40006ff20 0x7fb4000cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3fe800640 1 -- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4000ce120 msgr2=0x7fb4000cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3fe800640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4000ce120 0x7fb4000cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3fe800640 1 -- 172.21.15.113:0/4247341002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400159420 con 0x7fb400158010 2024-03-04T08:04:36.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3f7e00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4000ce120 0x7fb4000cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:04:36.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3fe800640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb400158010 0x7fb40013fa10 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800cb40 tx=0x7fb3f800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:36.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3f6000640 1 -- 172.21.15.113:0/4247341002 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8017070 con 0x7fb400158010 2024-03-04T08:04:36.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3f6000640 1 -- 172.21.15.113:0/4247341002 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f800aca0 con 0x7fb400158010 2024-03-04T08:04:36.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb404a00640 1 -- 172.21.15.113:0/4247341002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40015a630 con 0x7fb400158010 2024-03-04T08:04:36.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3f6000640 1 -- 172.21.15.113:0/4247341002 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8012620 con 0x7fb400158010 2024-03-04T08:04:36.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb404a00640 1 -- 172.21.15.113:0/4247341002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb400158690 con 0x7fb400158010 2024-03-04T08:04:36.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb404a00640 1 -- 172.21.15.113:0/4247341002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40006fd00 con 0x7fb400158010 2024-03-04T08:04:36.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.121+0000 7fb3f6000640 1 -- 172.21.15.113:0/4247341002 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3f801a070 con 0x7fb400158010 2024-03-04T08:04:36.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.125+0000 7fb3f6000640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3dc066f90 0x7fb3dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:36.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.125+0000 7fb3f6000640 1 -- 172.21.15.113:0/4247341002 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7fb3f8085460 con 0x7fb400158010 2024-03-04T08:04:36.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.125+0000 7fb3fde00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3dc066f90 0x7fb3dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:36.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.125+0000 7fb3fde00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3dc066f90 0x7fb3dc069450 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00ad000 tx=0x7fb3f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:36.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.129+0000 7fb3f6000640 1 -- 172.21.15.113:0/4247341002 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb3f8055750 con 0x7fb400158010 2024-03-04T08:04:36.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.325+0000 7fb404a00640 1 -- 172.21.15.113:0/4247341002 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4000ca8a0 con 0x7fb400158010 2024-03-04T08:04:36.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.325+0000 7fb3f6000640 1 -- 172.21.15.113:0/4247341002 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+5825 (secure 0 0 0) 0x7fb3f80597f0 con 0x7fb400158010 2024-03-04T08:04:36.325 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:36.325 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:35.841527+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:resolve","state_seq":19,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807]},"id":1}]} 2024-03-04T08:04:36.325 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 115 2024-03-04T08:04:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 -- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3dc066f90 msgr2=0x7fb3dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3dc066f90 0x7fb3dc069450 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00ad000 tx=0x7fb3f0008040 comp rx=0 tx=0).stop 2024-03-04T08:04:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 -- 172.21.15.113:0/4247341002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb400158010 msgr2=0x7fb40013fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb400158010 0x7fb40013fa10 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800cb40 tx=0x7fb3f800a060 comp rx=0 tx=0).stop 2024-03-04T08:04:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 -- 172.21.15.113:0/4247341002 shutdown_connections 2024-03-04T08:04:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3dc066f90 0x7fb3dc069450 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb400158010 0x7fb40013fa10 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4000ce120 0x7fb4000cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 --2- 172.21.15.113:0/4247341002 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb40006ff20 0x7fb4000cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:36.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 -- 172.21.15.113:0/4247341002 >> 172.21.15.113:0/4247341002 conn(0x7fb4000734d0 msgr2=0x7fb400155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:36.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 -- 172.21.15.113:0/4247341002 shutdown_connections 2024-03-04T08:04:36.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:36.329+0000 7fb404a00640 1 -- 172.21.15.113:0/4247341002 wait complete. 2024-03-04T08:04:36.346 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:04:38.347 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:38.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabf2a00640 1 Processor -- start 2024-03-04T08:04:38.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabf2a00640 1 -- start start 2024-03-04T08:04:38.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabf2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec15c4e0 0x7fabec164070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:38.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabf2a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec152e30 con 0x7fabec06ff20 2024-03-04T08:04:38.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabf2a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec06f280 con 0x7fabec152a50 2024-03-04T08:04:38.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabf2a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0c31b0 con 0x7fabec15c4e0 2024-03-04T08:04:38.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabe3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fabec06ff20 0x7fabec15bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55768/0 (socket says 172.21.15.113:55768) 2024-03-04T08:04:38.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabe3e00640 1 -- 172.21.15.113:0/1456675673 learned_addr learned my addr 172.21.15.113:0/1456675673 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:38.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabf0a00640 1 --2- 172.21.15.113:0/1456675673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec15c4e0 0x7fabec164070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:38.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabeb400640 1 -- 172.21.15.113:0/1456675673 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437247640 0 0) 0x7fabec152e30 con 0x7fabec06ff20 2024-03-04T08:04:38.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.485+0000 7fabeb400640 1 -- 172.21.15.113:0/1456675673 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd0003530 con 0x7fabec06ff20 2024-03-04T08:04:38.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.489+0000 7fabeb400640 1 -- 172.21.15.113:0/1456675673 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704257815 0 0) 0x7fabec06f280 con 0x7fabec152a50 2024-03-04T08:04:38.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.489+0000 7fabeb400640 1 -- 172.21.15.113:0/1456675673 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabec152e30 con 0x7fabec152a50 2024-03-04T08:04:38.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.489+0000 7fabf0a00640 1 -- 172.21.15.113:0/1456675673 >> v1:172.21.15.136:6789/0 conn(0x7fabec152a50 legacy=0x7fabec0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:38.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.489+0000 7fabf0a00640 1 -- 172.21.15.113:0/1456675673 >> v1:172.21.15.113:6789/0 conn(0x7fabec06ff20 legacy=0x7fabec15bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:38.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.489+0000 7fabf0a00640 1 -- 172.21.15.113:0/1456675673 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec06f560 con 0x7fabec15c4e0 2024-03-04T08:04:38.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.489+0000 7fabf0a00640 1 --2- 172.21.15.113:0/1456675673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec15c4e0 0x7fabec164070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabec0c3330 tx=0x7fabe401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d358ca5e873c4418 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:38.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.489+0000 7fabeb400640 1 -- 172.21.15.113:0/1456675673 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40b2070 con 0x7fabec15c4e0 2024-03-04T08:04:38.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.489+0000 7fabeb400640 1 -- 172.21.15.113:0/1456675673 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40a5960 con 0x7fabec15c4e0 2024-03-04T08:04:38.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.489+0000 7fabeb400640 1 -- 172.21.15.113:0/1456675673 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40a5d10 con 0x7fabec15c4e0 2024-03-04T08:04:38.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.493+0000 7fabf2a00640 1 -- 172.21.15.113:0/1456675673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec15c4e0 msgr2=0x7fabec164070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:38.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.493+0000 7fabf2a00640 1 --2- 172.21.15.113:0/1456675673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec15c4e0 0x7fabec164070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fabec0c3330 tx=0x7fabe401c990 comp rx=0 tx=0).stop 2024-03-04T08:04:38.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.493+0000 7fabf2a00640 1 -- 172.21.15.113:0/1456675673 shutdown_connections 2024-03-04T08:04:38.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.493+0000 7fabf2a00640 1 --2- 172.21.15.113:0/1456675673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec15c4e0 0x7fabec164070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:38.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.493+0000 7fabf2a00640 1 -- 172.21.15.113:0/1456675673 >> 172.21.15.113:0/1456675673 conn(0x7fabec0734d0 msgr2=0x7fabec066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:38.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.493+0000 7fabf2a00640 1 -- 172.21.15.113:0/1456675673 shutdown_connections 2024-03-04T08:04:38.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.493+0000 7fabf2a00640 1 -- 172.21.15.113:0/1456675673 wait complete. 2024-03-04T08:04:38.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.493+0000 7fabf2a00640 1 Processor -- start 2024-03-04T08:04:38.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.493+0000 7fabf2a00640 1 -- start start 2024-03-04T08:04:38.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabec06ff20 0x7fabec1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:38.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec152a50 0x7fabec06d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:38.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabec15c4e0 0x7fabec162b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:38.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf2a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec166f50 con 0x7fabec15c4e0 2024-03-04T08:04:38.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf2a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec166dd0 con 0x7fabec06ff20 2024-03-04T08:04:38.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf2a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec1670d0 con 0x7fabec152a50 2024-03-04T08:04:38.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabec15c4e0 0x7fabec162b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:38.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabec15c4e0 0x7fabec162b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41194/0 (socket says 172.21.15.113:41194) 2024-03-04T08:04:38.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf0a00640 1 -- 172.21.15.113:0/3548410404 learned_addr learned my addr 172.21.15.113:0/3548410404 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:38.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabe3e00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec152a50 0x7fabec06d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:38.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabebe00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabec06ff20 0x7fabec1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:38.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf0a00640 1 -- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec152a50 msgr2=0x7fabec06d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:38.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf0a00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec152a50 0x7fabec06d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:38.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf0a00640 1 -- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabec06ff20 msgr2=0x7fabec1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:38.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf0a00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabec06ff20 0x7fabec1615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:38.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf0a00640 1 -- 172.21.15.113:0/3548410404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0cdd50 con 0x7fabec15c4e0 2024-03-04T08:04:38.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf0a00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabec15c4e0 0x7fabec162b60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fabe40b0040 tx=0x7fabe4019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:38.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabe8c00640 1 -- 172.21.15.113:0/3548410404 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40b2050 con 0x7fabec15c4e0 2024-03-04T08:04:38.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabe8c00640 1 -- 172.21.15.113:0/3548410404 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40be070 con 0x7fabec15c4e0 2024-03-04T08:04:38.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf2a00640 1 -- 172.21.15.113:0/3548410404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec162380 con 0x7fabec15c4e0 2024-03-04T08:04:38.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabe8c00640 1 -- 172.21.15.113:0/3548410404 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4004040 con 0x7fabec15c4e0 2024-03-04T08:04:38.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf2a00640 1 -- 172.21.15.113:0/3548410404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec161e70 con 0x7fabec15c4e0 2024-03-04T08:04:38.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabf2a00640 1 -- 172.21.15.113:0/3548410404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec075c00 con 0x7fabec15c4e0 2024-03-04T08:04:38.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.497+0000 7fabe8c00640 1 -- 172.21.15.113:0/3548410404 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabe40041e0 con 0x7fabec15c4e0 2024-03-04T08:04:38.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.501+0000 7fabe8c00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabc8066f90 0x7fabc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:38.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.501+0000 7fabe8c00640 1 -- 172.21.15.113:0/3548410404 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7fabe4120620 con 0x7fabec15c4e0 2024-03-04T08:04:38.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.501+0000 7fabebe00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabc8066f90 0x7fabc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:38.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.501+0000 7fabebe00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabc8066f90 0x7fabc8069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fabec0c87c0 tx=0x7fabdc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:38.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.505+0000 7fabe8c00640 1 -- 172.21.15.113:0/3548410404 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fabe4016580 con 0x7fabec15c4e0 2024-03-04T08:04:38.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.693+0000 7fabf2a00640 1 -- 172.21.15.113:0/3548410404 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabec067210 con 0x7fabec15c4e0 2024-03-04T08:04:38.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.693+0000 7fabe8c00640 1 -- 172.21.15.113:0/3548410404 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+5825 (secure 0 0 0) 0x7fabe40f0910 con 0x7fabec15c4e0 2024-03-04T08:04:38.692 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:38.692 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:35.841527+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:resolve","state_seq":19,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807]},"id":1}]} 2024-03-04T08:04:38.693 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 115 2024-03-04T08:04:38.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 -- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabc8066f90 msgr2=0x7fabc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabc8066f90 0x7fabc8069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fabec0c87c0 tx=0x7fabdc009040 comp rx=0 tx=0).stop 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 -- 172.21.15.113:0/3548410404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabec15c4e0 msgr2=0x7fabec162b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabec15c4e0 0x7fabec162b60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fabe40b0040 tx=0x7fabe4019730 comp rx=0 tx=0).stop 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 -- 172.21.15.113:0/3548410404 shutdown_connections 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabc8066f90 0x7fabc8069450 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabec15c4e0 0x7fabec162b60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabec152a50 0x7fabec06d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 --2- 172.21.15.113:0/3548410404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabec06ff20 0x7fabec1615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 -- 172.21.15.113:0/3548410404 >> 172.21.15.113:0/3548410404 conn(0x7fabec0734d0 msgr2=0x7fabec159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 -- 172.21.15.113:0/3548410404 shutdown_connections 2024-03-04T08:04:38.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:38.697+0000 7fabf2a00640 1 -- 172.21.15.113:0/3548410404 wait complete. 2024-03-04T08:04:38.715 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:04:39.942 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 51% (133/256) Updating files: 52% (134/256) Updating files: 53% (136/256) Updating files: 54% (139/256) Updating files: 55% (141/256) Updating files: 56% (144/256) Updating files: 57% (146/256) Updating files: 58% (149/256) Updating files: 59% (152/256) Updating files: 60% (154/256) Updating files: 61% (157/256) Updating files: 62% (159/256) Updating files: 63% (162/256) Updating files: 64% (164/256) Updating files: 65% (167/256) Updating files: 66% (169/256) Updating files: 67% (172/256) Updating files: 68% (175/256) Updating files: 68% (176/256) Updating files: 69% (177/256) Updating files: 70% (180/256) Updating files: 71% (182/256) Updating files: 72% (185/256) Updating files: 73% (187/256) Updating files: 74% (190/256) Updating files: 75% (192/256) Updating files: 76% (195/256) Updating files: 77% (198/256) Updating files: 78% (200/256) Updating files: 79% (203/256) Updating files: 80% (205/256) Updating files: 81% (208/256) Updating files: 82% (210/256) Updating files: 83% (213/256) Updating files: 84% (216/256) Updating files: 85% (218/256) Updating files: 86% (221/256) Updating files: 87% (223/256) Updating files: 88% (226/256) Updating files: 89% (228/256) Updating files: 90% (231/256) Updating files: 91% (233/256) Updating files: 92% (236/256) Updating files: 93% (239/256) Updating files: 94% (241/256) Updating files: 95% (244/256) Updating files: 96% (246/256) Updating files: 97% (249/256) Updating files: 98% (251/256) Updating files: 99% (254/256) Updating files: 100% (256/256) Updating files: 100% (256/256), done. 2024-03-04T08:04:40.672 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at f26f7a39021 v0.60 2024-03-04T08:04:40.681 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.60 2024-03-04T08:04:40.716 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:40.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f845c600640 1 Processor -- start 2024-03-04T08:04:40.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f845c600640 1 -- start start 2024-03-04T08:04:40.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f845c600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f845414e4c0 0x7f8454072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:40.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f845c600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f845415b900 0x7f84540731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:40.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f845c600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84540c27f0 con 0x7f84540c22e0 2024-03-04T08:04:40.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f845c600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845415ad60 con 0x7f845414e4c0 2024-03-04T08:04:40.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f845c600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84540c2a90 con 0x7f845415b900 2024-03-04T08:04:40.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f8459c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f845414e4c0 0x7f8454072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:40.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f8459200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f84540c22e0 0x7f845415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55782/0 (socket says 172.21.15.113:55782) 2024-03-04T08:04:40.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f8459200640 1 -- 172.21.15.113:0/1207151145 learned_addr learned my addr 172.21.15.113:0/1207151145 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:40.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f845a600640 1 --2- 172.21.15.113:0/1207151145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f845415b900 0x7f84540731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:40.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f8453e00640 1 -- 172.21.15.113:0/1207151145 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640707728 0 0) 0x7f84540c27f0 con 0x7f84540c22e0 2024-03-04T08:04:40.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f8459c00640 1 -- 172.21.15.113:0/1207151145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f845415b900 msgr2=0x7f84540731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:40.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f8459c00640 1 --2- 172.21.15.113:0/1207151145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f845415b900 0x7f84540731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:40.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f8459c00640 1 -- 172.21.15.113:0/1207151145 >> v1:172.21.15.113:6789/0 conn(0x7f84540c22e0 legacy=0x7f845415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:40.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.853+0000 7f8459c00640 1 -- 172.21.15.113:0/1207151145 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454070c60 con 0x7f845414e4c0 2024-03-04T08:04:40.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f8459c00640 1 --2- 172.21.15.113:0/1207151145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 0x7f8454072c80 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f84480097c0 tx=0x7f844801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85c16fd4a64d604 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:40.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f8453e00640 1 -- 172.21.15.113:0/1207151145 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84480b5070 con 0x7f845414e4c0 2024-03-04T08:04:40.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f8453e00640 1 -- 172.21.15.113:0/1207151145 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84480a8990 con 0x7f845414e4c0 2024-03-04T08:04:40.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f8453e00640 1 -- 172.21.15.113:0/1207151145 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84480a8d60 con 0x7f845414e4c0 2024-03-04T08:04:40.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f845c600640 1 -- 172.21.15.113:0/1207151145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 msgr2=0x7f8454072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:40.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f845c600640 1 --2- 172.21.15.113:0/1207151145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 0x7f8454072c80 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f84480097c0 tx=0x7f844801ec90 comp rx=0 tx=0).stop 2024-03-04T08:04:40.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f845c600640 1 -- 172.21.15.113:0/1207151145 shutdown_connections 2024-03-04T08:04:40.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f845c600640 1 --2- 172.21.15.113:0/1207151145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f845415b900 0x7f84540731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:40.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f845c600640 1 --2- 172.21.15.113:0/1207151145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 0x7f8454072c80 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:40.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f845c600640 1 -- 172.21.15.113:0/1207151145 >> 172.21.15.113:0/1207151145 conn(0x7f845406e820 msgr2=0x7f845406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:40.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f845c600640 1 -- 172.21.15.113:0/1207151145 shutdown_connections 2024-03-04T08:04:40.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.857+0000 7f845c600640 1 -- 172.21.15.113:0/1207151145 wait complete. 2024-03-04T08:04:40.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845c600640 1 Processor -- start 2024-03-04T08:04:40.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845c600640 1 -- start start 2024-03-04T08:04:40.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845c600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84540c22e0 0x7f84540ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:40.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845c600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 0x7f84540c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:40.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845c600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f845415b900 0x7f8454146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:40.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845c600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8454071500 con 0x7f845414e4c0 2024-03-04T08:04:40.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845c600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8454071220 con 0x7f845415b900 2024-03-04T08:04:40.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845c600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845414e8a0 con 0x7f84540c22e0 2024-03-04T08:04:40.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8459200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 0x7f84540c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8459200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 0x7f84540c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41208/0 (socket says 172.21.15.113:41208) 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8459200640 1 -- 172.21.15.113:0/133756456 learned_addr learned my addr 172.21.15.113:0/133756456 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8459c00640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84540c22e0 0x7f84540ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845a600640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f845415b900 0x7f8454146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8459200640 1 -- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84540c22e0 msgr2=0x7f84540ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8459200640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84540c22e0 0x7f84540ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8459200640 1 -- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f845415b900 msgr2=0x7f8454146c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8459200640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f845415b900 0x7f8454146c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8459200640 1 -- 172.21.15.113:0/133756456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454149560 con 0x7f845414e4c0 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845a600640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f845415b900 0x7f8454146c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8459200640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 0x7f84540c4790 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f844000c480 tx=0x7f844000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8451600640 1 -- 172.21.15.113:0/133756456 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440018070 con 0x7f845414e4c0 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8451600640 1 -- 172.21.15.113:0/133756456 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8440007500 con 0x7f845414e4c0 2024-03-04T08:04:40.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f8451600640 1 -- 172.21.15.113:0/133756456 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440004a30 con 0x7f845414e4c0 2024-03-04T08:04:40.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845c600640 1 -- 172.21.15.113:0/133756456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84541f0ad0 con 0x7f845414e4c0 2024-03-04T08:04:40.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.861+0000 7f845c600640 1 -- 172.21.15.113:0/133756456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8454148a80 con 0x7f845414e4c0 2024-03-04T08:04:40.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.865+0000 7f845c600640 1 -- 172.21.15.113:0/133756456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8454075c00 con 0x7f845414e4c0 2024-03-04T08:04:40.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.865+0000 7f8451600640 1 -- 172.21.15.113:0/133756456 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8440004bd0 con 0x7f845414e4c0 2024-03-04T08:04:40.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.865+0000 7f8451600640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8434066f60 0x7f8434069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:40.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.865+0000 7f8451600640 1 -- 172.21.15.113:0/133756456 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f84400849a0 con 0x7f845414e4c0 2024-03-04T08:04:40.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.865+0000 7f8459c00640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8434066f60 0x7f8434069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:40.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.869+0000 7f8459c00640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8434066f60 0x7f8434069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f84480b2000 tx=0x7f8448008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:40.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:40.873+0000 7f8451600640 1 -- 172.21.15.113:0/133756456 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f84400552e0 con 0x7f845414e4c0 2024-03-04T08:04:41.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 -- 172.21.15.113:0/133756456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8454155190 con 0x7f845414e4c0 2024-03-04T08:04:41.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f8451600640 1 -- 172.21.15.113:0/133756456 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+5825 (secure 0 0 0) 0x7f8440054be0 con 0x7f845414e4c0 2024-03-04T08:04:41.060 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:41.060 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:35.841527+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:resolve","state_seq":19,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807]},"id":1}]} 2024-03-04T08:04:41.060 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 115 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 -- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8434066f60 msgr2=0x7f8434069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8434066f60 0x7f8434069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f84480b2000 tx=0x7f8448008040 comp rx=0 tx=0).stop 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 -- 172.21.15.113:0/133756456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 msgr2=0x7f84540c4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 0x7f84540c4790 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f844000c480 tx=0x7f844000c940 comp rx=0 tx=0).stop 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 -- 172.21.15.113:0/133756456 shutdown_connections 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f845415b900 0x7f8454146c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8434066f60 0x7f8434069420 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f845414e4c0 0x7f84540c4790 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 --2- 172.21.15.113:0/133756456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f84540c22e0 0x7f84540ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 -- 172.21.15.113:0/133756456 >> 172.21.15.113:0/133756456 conn(0x7f845406e820 msgr2=0x7f8454147bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 -- 172.21.15.113:0/133756456 shutdown_connections 2024-03-04T08:04:41.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:41.061+0000 7f845c600640 1 -- 172.21.15.113:0/133756456 wait complete. 2024-03-04T08:04:41.081 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:04:41.183 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 61 -eq 48 ] 2024-03-04T08:04:41.184 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.61 2024-03-04T08:04:41.184 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.61 2024-03-04T08:04:41.184 INFO:tasks.workunit.client.0.smithi113.stdout:v0.61 2024-03-04T08:04:41.185 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.61 2024-03-04T08:04:43.082 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d74e00640 1 Processor -- start 2024-03-04T08:04:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d74e00640 1 -- start start 2024-03-04T08:04:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d74e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3d7006ff20 0x7f3d7006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d74e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d700ce120 0x7f3d700ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d74e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d700ca0e0 con 0x7f3d70158010 2024-03-04T08:04:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d74e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7006f450 con 0x7f3d700ce120 2024-03-04T08:04:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d74e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7015ad80 con 0x7f3d7006ff20 2024-03-04T08:04:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d6e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d700ce120 0x7f3d700ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d6e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d700ce120 0x7f3d700ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41222/0 (socket says 172.21.15.113:41222) 2024-03-04T08:04:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d6e800640 1 -- 172.21.15.113:0/3357510206 learned_addr learned my addr 172.21.15.113:0/3357510206 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.225+0000 7f3d6d400640 1 --2- 172.21.15.113:0/3357510206 >> v2:172.21.15.136:3300/0 conn(0x7f3d7006ff20 0x7f3d7006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d6e800640 1 -- 172.21.15.113:0/3357510206 >> v2:172.21.15.136:3300/0 conn(0x7f3d7006ff20 msgr2=0x7f3d7006dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d6e800640 1 --2- 172.21.15.113:0/3357510206 >> v2:172.21.15.136:3300/0 conn(0x7f3d7006ff20 0x7f3d7006dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d6e800640 1 -- 172.21.15.113:0/3357510206 >> v1:172.21.15.113:6789/0 conn(0x7f3d70158010 legacy=0x7f3d7015b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d6e800640 1 -- 172.21.15.113:0/3357510206 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7015aa30 con 0x7f3d700ce120 2024-03-04T08:04:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d6e800640 1 --2- 172.21.15.113:0/3357510206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d700ce120 0x7f3d700ce5d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3d700cc040 tx=0x7f3d6801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3892ae796b1a36 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:43.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d6ca00640 1 -- 172.21.15.113:0/3357510206 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680a56c0 con 0x7f3d700ce120 2024-03-04T08:04:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d6ca00640 1 -- 172.21.15.113:0/3357510206 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d680a5860 con 0x7f3d700ce120 2024-03-04T08:04:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d6ca00640 1 -- 172.21.15.113:0/3357510206 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680a5c30 con 0x7f3d700ce120 2024-03-04T08:04:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d74e00640 1 -- 172.21.15.113:0/3357510206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d700ce120 msgr2=0x7f3d700ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d74e00640 1 --2- 172.21.15.113:0/3357510206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d700ce120 0x7f3d700ce5d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3d700cc040 tx=0x7f3d6801c9f0 comp rx=0 tx=0).stop 2024-03-04T08:04:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d74e00640 1 -- 172.21.15.113:0/3357510206 shutdown_connections 2024-03-04T08:04:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d74e00640 1 --2- 172.21.15.113:0/3357510206 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d700ce120 0x7f3d700ce5d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d74e00640 1 --2- 172.21.15.113:0/3357510206 >> v2:172.21.15.136:3300/0 conn(0x7f3d7006ff20 0x7f3d7006dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d74e00640 1 -- 172.21.15.113:0/3357510206 >> 172.21.15.113:0/3357510206 conn(0x7f3d700734d0 msgr2=0x7f3d70066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d74e00640 1 -- 172.21.15.113:0/3357510206 shutdown_connections 2024-03-04T08:04:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.229+0000 7f3d74e00640 1 -- 172.21.15.113:0/3357510206 wait complete. 2024-03-04T08:04:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d74e00640 1 Processor -- start 2024-03-04T08:04:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d74e00640 1 -- start start 2024-03-04T08:04:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d74e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d7006ff20 0x7f3d700b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d74e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d700ce120 0x7f3d700b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d74e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d70158010 0x7f3d700cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d74e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7015b920 con 0x7f3d70158010 2024-03-04T08:04:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d74e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7006e290 con 0x7f3d7006ff20 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d74e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d701583f0 con 0x7f3d700ce120 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d70158010 0x7f3d700cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d70158010 0x7f3d700cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41236/0 (socket says 172.21.15.113:41236) 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6e800640 1 -- 172.21.15.113:0/1001869288 learned_addr learned my addr 172.21.15.113:0/1001869288 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6d400640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d700ce120 0x7f3d700b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6de00640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d7006ff20 0x7f3d700b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6e800640 1 -- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d700ce120 msgr2=0x7f3d700b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6e800640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d700ce120 0x7f3d700b2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6e800640 1 -- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d7006ff20 msgr2=0x7f3d700b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6e800640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d7006ff20 0x7f3d700b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6e800640 1 -- 172.21.15.113:0/1001869288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7014c370 con 0x7f3d70158010 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d6e800640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d70158010 0x7f3d700cbf90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3d680af040 tx=0x7f3d68019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d66000640 1 -- 172.21.15.113:0/1001869288 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680bc020 con 0x7f3d70158010 2024-03-04T08:04:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d66000640 1 -- 172.21.15.113:0/1001869288 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d680b8690 con 0x7f3d70158010 2024-03-04T08:04:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d66000640 1 -- 172.21.15.113:0/1001869288 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680b8a80 con 0x7f3d70158010 2024-03-04T08:04:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.233+0000 7f3d74e00640 1 -- 172.21.15.113:0/1001869288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7014aac0 con 0x7f3d70158010 2024-03-04T08:04:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.237+0000 7f3d74e00640 1 -- 172.21.15.113:0/1001869288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7014b080 con 0x7f3d70158010 2024-03-04T08:04:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.237+0000 7f3d74e00640 1 -- 172.21.15.113:0/1001869288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d70075c00 con 0x7f3d70158010 2024-03-04T08:04:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.237+0000 7f3d66000640 1 -- 172.21.15.113:0/1001869288 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d680bf020 con 0x7f3d70158010 2024-03-04T08:04:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.237+0000 7f3d66000640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d50066f60 0x7f3d50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.237+0000 7f3d66000640 1 -- 172.21.15.113:0/1001869288 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f3d68120650 con 0x7f3d70158010 2024-03-04T08:04:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.237+0000 7f3d6de00640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d50066f60 0x7f3d50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.241+0000 7f3d6de00640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d50066f60 0x7f3d50069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3d60001810 tx=0x7f3d60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:43.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.241+0000 7f3d66000640 1 -- 172.21.15.113:0/1001869288 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3d680c6030 con 0x7f3d70158010 2024-03-04T08:04:43.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 -- 172.21.15.113:0/1001869288 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d7006fd00 con 0x7f3d70158010 2024-03-04T08:04:43.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d66000640 1 -- 172.21.15.113:0/1001869288 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5827 (secure 0 0 0) 0x7f3d680165e0 con 0x7f3d70158010 2024-03-04T08:04:43.432 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:43.432 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:41.180330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6807,6925]},"id":1}]} 2024-03-04T08:04:43.432 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 116 2024-03-04T08:04:43.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 -- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d50066f60 msgr2=0x7f3d50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:43.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d50066f60 0x7f3d50069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3d60001810 tx=0x7f3d60009040 comp rx=0 tx=0).stop 2024-03-04T08:04:43.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 -- 172.21.15.113:0/1001869288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d70158010 msgr2=0x7f3d700cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:43.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d70158010 0x7f3d700cbf90 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f3d680af040 tx=0x7f3d68019790 comp rx=0 tx=0).stop 2024-03-04T08:04:43.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 -- 172.21.15.113:0/1001869288 shutdown_connections 2024-03-04T08:04:43.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d50066f60 0x7f3d50069420 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:43.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d70158010 0x7f3d700cbf90 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:43.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d700ce120 0x7f3d700b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:43.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 --2- 172.21.15.113:0/1001869288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d7006ff20 0x7f3d700b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:43.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 -- 172.21.15.113:0/1001869288 >> 172.21.15.113:0/1001869288 conn(0x7f3d700734d0 msgr2=0x7f3d700c8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:43.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 -- 172.21.15.113:0/1001869288 shutdown_connections 2024-03-04T08:04:43.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:43.433+0000 7f3d74e00640 1 -- 172.21.15.113:0/1001869288 wait complete. 2024-03-04T08:04:43.455 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 116, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 116, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6807": { "addr": "172.21.15.113:6836/1035982016", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1035982016, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1035982016, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6807, "incarnation": 101, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:active", "state_seq": 14 }, "gid_6925": { "addr": "172.21.15.136:6835/2522634751", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2522634751, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2522634751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6925, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 23 }, "gid_7119": { "addr": "172.21.15.113:6833/3825257303", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 3825257303, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 3825257303, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7119, "incarnation": 114, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:reconnect", "state_seq": 21 } }, "last_failure": 0, "last_failure_osd_epoch": 65, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:04:41.180330+0000", "qdb_cluster": [ 6807, 6925 ], "qdb_leader": 6925, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6925, "mds_1": 7119, "mds_2": 6807 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:04:45.457 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:45.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5854200640 1 Processor -- start 2024-03-04T08:04:45.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5854200640 1 -- start start 2024-03-04T08:04:45.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5854200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f584c06df10 0x7f584c15e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:45.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5854200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f584c0703e0 0x7f584c0ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:45.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5854200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584c06c310 con 0x7f584c158230 2024-03-04T08:04:45.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5854200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584c0c95f0 con 0x7f584c0703e0 2024-03-04T08:04:45.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5854200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c06c5b0 con 0x7f584c06df10 2024-03-04T08:04:45.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5852200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f584c0703e0 0x7f584c0ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:45.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5852200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f584c0703e0 0x7f584c0ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41250/0 (socket says 172.21.15.113:41250) 2024-03-04T08:04:45.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5852200640 1 -- 172.21.15.113:0/3610389180 learned_addr learned my addr 172.21.15.113:0/3610389180 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:45.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5851800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f584c158230 0x7f584c06f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55810/0 (socket says 172.21.15.113:55810) 2024-03-04T08:04:45.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.593+0000 7f5850e00640 1 --2- 172.21.15.113:0/3610389180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f584c06df10 0x7f584c15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:45.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5852200640 1 -- 172.21.15.113:0/3610389180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f584c06df10 msgr2=0x7f584c15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:45.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5852200640 1 --2- 172.21.15.113:0/3610389180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f584c06df10 0x7f584c15e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:45.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5852200640 1 -- 172.21.15.113:0/3610389180 >> v1:172.21.15.113:6789/0 conn(0x7f584c158230 legacy=0x7f584c06f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:45.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5852200640 1 -- 172.21.15.113:0/3610389180 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c0c9890 con 0x7f584c0703e0 2024-03-04T08:04:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5852200640 1 --2- 172.21.15.113:0/3610389180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c0703e0 0x7f584c0ce3c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f5844009e50 tx=0x7f584401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726a7ed491b12488 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f584be00640 1 -- 172.21.15.113:0/3610389180 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58440a5700 con 0x7f584c0703e0 2024-03-04T08:04:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f584be00640 1 -- 172.21.15.113:0/3610389180 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58440a58a0 con 0x7f584c0703e0 2024-03-04T08:04:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f584be00640 1 -- 172.21.15.113:0/3610389180 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58440a5c70 con 0x7f584c0703e0 2024-03-04T08:04:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5854200640 1 -- 172.21.15.113:0/3610389180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c0703e0 msgr2=0x7f584c0ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5854200640 1 --2- 172.21.15.113:0/3610389180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c0703e0 0x7f584c0ce3c0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f5844009e50 tx=0x7f584401ca30 comp rx=0 tx=0).stop 2024-03-04T08:04:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5854200640 1 -- 172.21.15.113:0/3610389180 shutdown_connections 2024-03-04T08:04:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5854200640 1 --2- 172.21.15.113:0/3610389180 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c0703e0 0x7f584c0ce3c0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5854200640 1 --2- 172.21.15.113:0/3610389180 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f584c06df10 0x7f584c15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.597+0000 7f5854200640 1 -- 172.21.15.113:0/3610389180 >> 172.21.15.113:0/3610389180 conn(0x7f584c066db0 msgr2=0x7f584c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 -- 172.21.15.113:0/3610389180 shutdown_connections 2024-03-04T08:04:45.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 -- 172.21.15.113:0/3610389180 wait complete. 2024-03-04T08:04:45.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 Processor -- start 2024-03-04T08:04:45.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 -- start start 2024-03-04T08:04:45.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f584c06df10 0x7f584c15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:45.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f584c0703e0 0x7f584c1f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:45.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c158230 0x7f584c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:45.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c168070 con 0x7f584c158230 2024-03-04T08:04:45.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c167ef0 con 0x7f584c0703e0 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c1681f0 con 0x7f584c06df10 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5852200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c158230 0x7f584c15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5852200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c158230 0x7f584c15bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41260/0 (socket says 172.21.15.113:41260) 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5852200640 1 -- 172.21.15.113:0/3298745227 learned_addr learned my addr 172.21.15.113:0/3298745227 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5851800640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f584c06df10 0x7f584c15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5850e00640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f584c0703e0 0x7f584c1f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5852200640 1 -- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f584c06df10 msgr2=0x7f584c15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5852200640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f584c06df10 0x7f584c15dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5852200640 1 -- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f584c0703e0 msgr2=0x7f584c1f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5852200640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f584c0703e0 0x7f584c1f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5852200640 1 -- 172.21.15.113:0/3298745227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c0ce6e0 con 0x7f584c158230 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5850e00640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f584c0703e0 0x7f584c1f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:04:45.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5852200640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c158230 0x7f584c15bd80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f58440af040 tx=0x7f58440197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:45.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5849600640 1 -- 172.21.15.113:0/3298745227 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844009510 con 0x7f584c158230 2024-03-04T08:04:45.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5849600640 1 -- 172.21.15.113:0/3298745227 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58440be070 con 0x7f584c158230 2024-03-04T08:04:45.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5849600640 1 -- 172.21.15.113:0/3298745227 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58440b88b0 con 0x7f584c158230 2024-03-04T08:04:45.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.601+0000 7f5854200640 1 -- 172.21.15.113:0/3298745227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584c1f38d0 con 0x7f584c158230 2024-03-04T08:04:45.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.605+0000 7f5854200640 1 -- 172.21.15.113:0/3298745227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c1f3b70 con 0x7f584c158230 2024-03-04T08:04:45.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.605+0000 7f5854200640 1 -- 172.21.15.113:0/3298745227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584c075c00 con 0x7f584c158230 2024-03-04T08:04:45.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.605+0000 7f5849600640 1 -- 172.21.15.113:0/3298745227 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58440b8a50 con 0x7f584c158230 2024-03-04T08:04:45.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.605+0000 7f5849600640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f582c066f60 0x7f582c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:45.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.605+0000 7f5849600640 1 -- 172.21.15.113:0/3298745227 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f5844120880 con 0x7f584c158230 2024-03-04T08:04:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.605+0000 7f5851800640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f582c066f60 0x7f582c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.605+0000 7f5851800640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f582c066f60 0x7f582c069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5840000f70 tx=0x7f5840007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:45.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.613+0000 7f5849600640 1 -- 172.21.15.113:0/3298745227 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f584c0ce6e0 con 0x7f584c158230 2024-03-04T08:04:45.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.813+0000 7f5854200640 1 -- 172.21.15.113:0/3298745227 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f584c0701c0 con 0x7f584c158230 2024-03-04T08:04:45.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.813+0000 7f5849600640 1 -- 172.21.15.113:0/3298745227 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5827 (secure 0 0 0) 0x7f5844016620 con 0x7f584c158230 2024-03-04T08:04:45.813 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:45.813 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:41.180330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:reconnect","state_seq":21,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6807,6925]},"id":1}]} 2024-03-04T08:04:45.813 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 116 2024-03-04T08:04:45.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 -- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f582c066f60 msgr2=0x7f582c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:45.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f582c066f60 0x7f582c069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5840000f70 tx=0x7f5840007e90 comp rx=0 tx=0).stop 2024-03-04T08:04:45.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 -- 172.21.15.113:0/3298745227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c158230 msgr2=0x7f584c15bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:45.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c158230 0x7f584c15bd80 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f58440af040 tx=0x7f58440197d0 comp rx=0 tx=0).stop 2024-03-04T08:04:45.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 -- 172.21.15.113:0/3298745227 shutdown_connections 2024-03-04T08:04:45.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f582c066f60 0x7f582c069420 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:45.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f584c158230 0x7f584c15bd80 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:45.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f584c0703e0 0x7f584c1f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:45.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 --2- 172.21.15.113:0/3298745227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f584c06df10 0x7f584c15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:45.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 -- 172.21.15.113:0/3298745227 >> 172.21.15.113:0/3298745227 conn(0x7f584c066db0 msgr2=0x7f584c1568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:45.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 -- 172.21.15.113:0/3298745227 shutdown_connections 2024-03-04T08:04:45.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:45.817+0000 7f5854200640 1 -- 172.21.15.113:0/3298745227 wait complete. 2024-03-04T08:04:45.837 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:04:47.838 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:47.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b7e00640 1 Processor -- start 2024-03-04T08:04:47.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b7e00640 1 -- start start 2024-03-04T08:04:47.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f37b806d420 0x7f37b80730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:47.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f37b815bce0 0x7f37b815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:47.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b806f1b0 con 0x7f37b8152a90 2024-03-04T08:04:47.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b8152e70 con 0x7f37b806d420 2024-03-04T08:04:47.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b806c320 con 0x7f37b815bce0 2024-03-04T08:04:47.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b6000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f37b806d420 0x7f37b80730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:47.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b7400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f37b815bce0 0x7f37b815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:47.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b6000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f37b806d420 0x7f37b80730d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34630/0 (socket says 172.21.15.113:34630) 2024-03-04T08:04:47.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b6000640 1 -- 172.21.15.113:0/520330282 learned_addr learned my addr 172.21.15.113:0/520330282 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b6000640 1 -- 172.21.15.113:0/520330282 >> v2:172.21.15.136:3300/0 conn(0x7f37b815bce0 msgr2=0x7f37b815c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b6000640 1 --2- 172.21.15.113:0/520330282 >> v2:172.21.15.136:3300/0 conn(0x7f37b815bce0 0x7f37b815c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b6000640 1 -- 172.21.15.113:0/520330282 >> v1:172.21.15.136:6789/0 conn(0x7f37b8152a90 legacy=0x7f37b806cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.977+0000 7f37b6000640 1 -- 172.21.15.113:0/520330282 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b8071580 con 0x7f37b806d420 2024-03-04T08:04:47.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b6000640 1 --2- 172.21.15.113:0/520330282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 0x7f37b80730d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f37b815cde0 tx=0x7f37a401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d042d60de6de7029 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:47.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b5600640 1 -- 172.21.15.113:0/520330282 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a40b0070 con 0x7f37b806d420 2024-03-04T08:04:47.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b5600640 1 -- 172.21.15.113:0/520330282 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a40ab600 con 0x7f37b806d420 2024-03-04T08:04:47.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b5600640 1 -- 172.21.15.113:0/520330282 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a40ab9d0 con 0x7f37b806d420 2024-03-04T08:04:47.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b7e00640 1 -- 172.21.15.113:0/520330282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 msgr2=0x7f37b80730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:47.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b7e00640 1 --2- 172.21.15.113:0/520330282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 0x7f37b80730d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f37b815cde0 tx=0x7f37a401c520 comp rx=0 tx=0).stop 2024-03-04T08:04:47.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b7e00640 1 -- 172.21.15.113:0/520330282 shutdown_connections 2024-03-04T08:04:47.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b7e00640 1 --2- 172.21.15.113:0/520330282 >> v2:172.21.15.136:3300/0 conn(0x7f37b815bce0 0x7f37b815c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:47.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b7e00640 1 --2- 172.21.15.113:0/520330282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 0x7f37b80730d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:47.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b7e00640 1 -- 172.21.15.113:0/520330282 >> 172.21.15.113:0/520330282 conn(0x7f37b806e2a0 msgr2=0x7f37b806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:47.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b7e00640 1 -- 172.21.15.113:0/520330282 shutdown_connections 2024-03-04T08:04:47.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.981+0000 7f37b7e00640 1 -- 172.21.15.113:0/520330282 wait complete. 2024-03-04T08:04:47.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b7e00640 1 Processor -- start 2024-03-04T08:04:47.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b7e00640 1 -- start start 2024-03-04T08:04:47.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 0x7f37b8146680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37b8152a90 0x7f37b81408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37b815bce0 0x7f37b8140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b8165890 con 0x7f37b806d420 2024-03-04T08:04:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b8071330 con 0x7f37b8152a90 2024-03-04T08:04:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b8165a10 con 0x7f37b815bce0 2024-03-04T08:04:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 0x7f37b8146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 0x7f37b8146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34636/0 (socket says 172.21.15.113:34636) 2024-03-04T08:04:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6a00640 1 -- 172.21.15.113:0/1862366226 learned_addr learned my addr 172.21.15.113:0/1862366226 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6000640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37b8152a90 0x7f37b81408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b7400640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37b815bce0 0x7f37b8140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6a00640 1 -- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37b815bce0 msgr2=0x7f37b8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6a00640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37b815bce0 0x7f37b8140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6a00640 1 -- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37b8152a90 msgr2=0x7f37b81408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6a00640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37b8152a90 0x7f37b81408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6a00640 1 -- 172.21.15.113:0/1862366226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b81419a0 con 0x7f37b806d420 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6000640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37b8152a90 0x7f37b81408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b6a00640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 0x7f37b8146680 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f37a800c3a0 tx=0x7f37a800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37aea00640 1 -- 172.21.15.113:0/1862366226 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a80052d0 con 0x7f37b806d420 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37aea00640 1 -- 172.21.15.113:0/1862366226 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a8005470 con 0x7f37b806d420 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37b7e00640 1 -- 172.21.15.113:0/1862366226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b8158ef0 con 0x7f37b806d420 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.985+0000 7f37aea00640 1 -- 172.21.15.113:0/1862366226 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8007260 con 0x7f37b806d420 2024-03-04T08:04:47.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.989+0000 7f37b7e00640 1 -- 172.21.15.113:0/1862366226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b81569e0 con 0x7f37b806d420 2024-03-04T08:04:47.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.989+0000 7f37b7e00640 1 -- 172.21.15.113:0/1862366226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b8141b70 con 0x7f37b806d420 2024-03-04T08:04:47.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.989+0000 7f37aea00640 1 -- 172.21.15.113:0/1862366226 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37a8007400 con 0x7f37b806d420 2024-03-04T08:04:47.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.989+0000 7f37aea00640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3798066f60 0x7f3798069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:47.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.993+0000 7f37aea00640 1 -- 172.21.15.113:0/1862366226 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f37a8084a40 con 0x7f37b806d420 2024-03-04T08:04:47.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.993+0000 7f37b6000640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3798066f60 0x7f3798069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:47.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.993+0000 7f37b6000640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3798066f60 0x7f3798069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f37a40ad000 tx=0x7f37a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:47.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:47.997+0000 7f37aea00640 1 -- 172.21.15.113:0/1862366226 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f37a8054dd0 con 0x7f37b806d420 2024-03-04T08:04:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.185+0000 7f37b7e00640 1 -- 172.21.15.113:0/1862366226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37b8075a80 con 0x7f37b806d420 2024-03-04T08:04:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.185+0000 7f37aea00640 1 -- 172.21.15.113:0/1862366226 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f37a8058e70 con 0x7f37b806d420 2024-03-04T08:04:48.186 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:48.186 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":117,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:46.193185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807]},"id":1}]} 2024-03-04T08:04:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 117 2024-03-04T08:04:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 -- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3798066f60 msgr2=0x7f3798069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3798066f60 0x7f3798069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f37a40ad000 tx=0x7f37a4008040 comp rx=0 tx=0).stop 2024-03-04T08:04:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 -- 172.21.15.113:0/1862366226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 msgr2=0x7f37b8146680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 0x7f37b8146680 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f37a800c3a0 tx=0x7f37a800c860 comp rx=0 tx=0).stop 2024-03-04T08:04:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 -- 172.21.15.113:0/1862366226 shutdown_connections 2024-03-04T08:04:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f37b815bce0 0x7f37b8140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:48.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37b8152a90 0x7f37b81408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3798066f60 0x7f3798069420 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 --2- 172.21.15.113:0/1862366226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37b806d420 0x7f37b8146680 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 -- 172.21.15.113:0/1862366226 >> 172.21.15.113:0/1862366226 conn(0x7f37b806e2a0 msgr2=0x7f37b80c7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 -- 172.21.15.113:0/1862366226 shutdown_connections 2024-03-04T08:04:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:48.189+0000 7f37b7e00640 1 -- 172.21.15.113:0/1862366226 wait complete. 2024-03-04T08:04:48.208 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:04:50.208 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:50.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.397+0000 7f5f3f000640 1 Processor -- start 2024-03-04T08:04:50.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.397+0000 7f5f3f000640 1 -- start start 2024-03-04T08:04:50.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3f000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f3815b8a0 0x7f5f3806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:50.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3f000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f3806dcb0 0x7f5f3806f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:50.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3f000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3806c310 con 0x7f5f380c8990 2024-03-04T08:04:50.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3f000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3815c6c0 con 0x7f5f3806dcb0 2024-03-04T08:04:50.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3f000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3806c5f0 con 0x7f5f3815b8a0 2024-03-04T08:04:50.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f3806dcb0 0x7f5f3806f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:50.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f3806dcb0 0x7f5f3806f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34644/0 (socket says 172.21.15.113:34644) 2024-03-04T08:04:50.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3d200640 1 -- 172.21.15.113:0/2545080071 learned_addr learned my addr 172.21.15.113:0/2545080071 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:50.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3dc00640 1 --2- 172.21.15.113:0/2545080071 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f3815b8a0 0x7f5f3806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:50.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3d200640 1 -- 172.21.15.113:0/2545080071 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f3815b8a0 msgr2=0x7f5f3806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:50.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3d200640 1 --2- 172.21.15.113:0/2545080071 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f3815b8a0 0x7f5f3806ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:50.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3d200640 1 -- 172.21.15.113:0/2545080071 >> v1:172.21.15.136:6789/0 conn(0x7f5f380c8990 legacy=0x7f5f3815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:50.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3d200640 1 -- 172.21.15.113:0/2545080071 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3815c070 con 0x7f5f3806dcb0 2024-03-04T08:04:50.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.401+0000 7f5f3d200640 1 --2- 172.21.15.113:0/2545080071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3806dcb0 0x7f5f3806f490 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009870 tx=0x7f5f2401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e820380f961bfe3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:50.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f37e00640 1 -- 172.21.15.113:0/2545080071 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240b5070 con 0x7f5f3806dcb0 2024-03-04T08:04:50.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f37e00640 1 -- 172.21.15.113:0/2545080071 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f240a8a50 con 0x7f5f3806dcb0 2024-03-04T08:04:50.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f37e00640 1 -- 172.21.15.113:0/2545080071 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240a8e20 con 0x7f5f3806dcb0 2024-03-04T08:04:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 -- 172.21.15.113:0/2545080071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3806dcb0 msgr2=0x7f5f3806f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 --2- 172.21.15.113:0/2545080071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3806dcb0 0x7f5f3806f490 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009870 tx=0x7f5f2401ed50 comp rx=0 tx=0).stop 2024-03-04T08:04:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 -- 172.21.15.113:0/2545080071 shutdown_connections 2024-03-04T08:04:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 --2- 172.21.15.113:0/2545080071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f3806dcb0 0x7f5f3806f490 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 --2- 172.21.15.113:0/2545080071 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f3815b8a0 0x7f5f3806ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 -- 172.21.15.113:0/2545080071 >> 172.21.15.113:0/2545080071 conn(0x7f5f38072e30 msgr2=0x7f5f38066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 -- 172.21.15.113:0/2545080071 shutdown_connections 2024-03-04T08:04:50.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 -- 172.21.15.113:0/2545080071 wait complete. 2024-03-04T08:04:50.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 Processor -- start 2024-03-04T08:04:50.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 -- start start 2024-03-04T08:04:50.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f3806dcb0 0x7f5f380cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:50.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380c8990 0x7f5f381e4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:50.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f3815b8a0 0x7f5f381e5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:50.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3806d840 con 0x7f5f380c8990 2024-03-04T08:04:50.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3815bc80 con 0x7f5f3815b8a0 2024-03-04T08:04:50.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3f000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3815d260 con 0x7f5f3806dcb0 2024-03-04T08:04:50.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380c8990 0x7f5f381e4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:50.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380c8990 0x7f5f381e4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34660/0 (socket says 172.21.15.113:34660) 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3d200640 1 -- 172.21.15.113:0/3229943874 learned_addr learned my addr 172.21.15.113:0/3229943874 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3dc00640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f3806dcb0 0x7f5f380cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.405+0000 7f5f3e600640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f3815b8a0 0x7f5f381e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f3d200640 1 -- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f3806dcb0 msgr2=0x7f5f380cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f3d200640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f3806dcb0 0x7f5f380cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f3d200640 1 -- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f3815b8a0 msgr2=0x7f5f381e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f3d200640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f3815b8a0 0x7f5f381e5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f3d200640 1 -- 172.21.15.113:0/3229943874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f381f0910 con 0x7f5f380c8990 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f3d200640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380c8990 0x7f5f381e4f30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5f240b3040 tx=0x7f5f2401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f35600640 1 -- 172.21.15.113:0/3229943874 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240c0070 con 0x7f5f380c8990 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f35600640 1 -- 172.21.15.113:0/3229943874 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f240b0b70 con 0x7f5f380c8990 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f3f000640 1 -- 172.21.15.113:0/3229943874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f381e6cf0 con 0x7f5f380c8990 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f35600640 1 -- 172.21.15.113:0/3229943874 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f240b5040 con 0x7f5f380c8990 2024-03-04T08:04:50.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f3f000640 1 -- 172.21.15.113:0/3229943874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f381f0eb0 con 0x7f5f380c8990 2024-03-04T08:04:50.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f3f000640 1 -- 172.21.15.113:0/3229943874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f38075a80 con 0x7f5f380c8990 2024-03-04T08:04:50.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.409+0000 7f5f35600640 1 -- 172.21.15.113:0/3229943874 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f240c3070 con 0x7f5f380c8990 2024-03-04T08:04:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.413+0000 7f5f35600640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f18066f90 0x7f5f18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.413+0000 7f5f35600640 1 -- 172.21.15.113:0/3229943874 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f5f24122690 con 0x7f5f380c8990 2024-03-04T08:04:50.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.413+0000 7f5f3dc00640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f18066f90 0x7f5f18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:50.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.413+0000 7f5f3dc00640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f18066f90 0x7f5f18069450 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009ac0 tx=0x7f5f2c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:50.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.417+0000 7f5f35600640 1 -- 172.21.15.113:0/3229943874 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5f240188b0 con 0x7f5f380c8990 2024-03-04T08:04:50.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.605+0000 7f5f3f000640 1 -- 172.21.15.113:0/3229943874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f380caf10 con 0x7f5f380c8990 2024-03-04T08:04:50.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.605+0000 7f5f35600640 1 -- 172.21.15.113:0/3229943874 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f5f240f2ab0 con 0x7f5f380c8990 2024-03-04T08:04:50.605 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:50.605 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":117,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:46.193185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,6807]},"id":1}]} 2024-03-04T08:04:50.605 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 117 2024-03-04T08:04:50.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 -- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f18066f90 msgr2=0x7f5f18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:50.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f18066f90 0x7f5f18069450 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009ac0 tx=0x7f5f2c009190 comp rx=0 tx=0).stop 2024-03-04T08:04:50.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 -- 172.21.15.113:0/3229943874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380c8990 msgr2=0x7f5f381e4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:50.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380c8990 0x7f5f381e4f30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f5f240b3040 tx=0x7f5f2401baf0 comp rx=0 tx=0).stop 2024-03-04T08:04:50.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 -- 172.21.15.113:0/3229943874 shutdown_connections 2024-03-04T08:04:50.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f3815b8a0 0x7f5f381e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:50.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f18066f90 0x7f5f18069450 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:50.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f380c8990 0x7f5f381e4f30 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:50.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 --2- 172.21.15.113:0/3229943874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f3806dcb0 0x7f5f380cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:50.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 -- 172.21.15.113:0/3229943874 >> 172.21.15.113:0/3229943874 conn(0x7f5f38072e30 msgr2=0x7f5f3815ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:50.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 -- 172.21.15.113:0/3229943874 shutdown_connections 2024-03-04T08:04:50.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:50.609+0000 7f5f3f000640 1 -- 172.21.15.113:0/3229943874 wait complete. 2024-03-04T08:04:50.627 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:04:52.152 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T08:04:52.154+0000 7fd207b4d640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T08:04:52.627 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:52.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc785200640 1 Processor -- start 2024-03-04T08:04:52.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc785200640 1 -- start start 2024-03-04T08:04:52.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc785200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc780070010 0x7fc7800c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:52.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc785200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc78015c510 0x7fc780163190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:52.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc785200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc780152f00 con 0x7fc780152b20 2024-03-04T08:04:52.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc785200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7800c5320 con 0x7fc78015c510 2024-03-04T08:04:52.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc785200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800c5040 con 0x7fc780070010 2024-03-04T08:04:52.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc77de00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc780152b20 0x7fc78015be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50226/0 (socket says 172.21.15.113:50226) 2024-03-04T08:04:52.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc77de00640 1 -- 172.21.15.113:0/794300599 learned_addr learned my addr 172.21.15.113:0/794300599 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:52.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc777e00640 1 --2- 172.21.15.113:0/794300599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc780070010 0x7fc7800c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:52.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc77e800640 1 --2- 172.21.15.113:0/794300599 >> v2:172.21.15.136:3300/0 conn(0x7fc78015c510 0x7fc780163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:52.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc77d400640 1 -- 172.21.15.113:0/794300599 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725013601 0 0) 0x7fc780152f00 con 0x7fc780152b20 2024-03-04T08:04:52.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc77d400640 1 -- 172.21.15.113:0/794300599 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc764003530 con 0x7fc780152b20 2024-03-04T08:04:52.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc77e800640 1 -- 172.21.15.113:0/794300599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc780070010 msgr2=0x7fc7800c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:52.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc77e800640 1 --2- 172.21.15.113:0/794300599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc780070010 0x7fc7800c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:52.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc77e800640 1 -- 172.21.15.113:0/794300599 >> v1:172.21.15.113:6789/0 conn(0x7fc780152b20 legacy=0x7fc78015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:52.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc77d400640 1 -- 172.21.15.113:0/794300599 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 431381209 0 0) 0x7fc764003530 con 0x7fc780152b20 2024-03-04T08:04:52.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.765+0000 7fc77e800640 1 -- 172.21.15.113:0/794300599 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800c3250 con 0x7fc78015c510 2024-03-04T08:04:52.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc77e800640 1 --2- 172.21.15.113:0/794300599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78015c510 0x7fc780163190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc7800c54a0 tx=0x7fc77801c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e56ae7f805e9d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:52.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc77d400640 1 -- 172.21.15.113:0/794300599 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7780ac070 con 0x7fc78015c510 2024-03-04T08:04:52.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc77d400640 1 -- 172.21.15.113:0/794300599 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7780a6480 con 0x7fc78015c510 2024-03-04T08:04:52.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc77d400640 1 -- 172.21.15.113:0/794300599 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770001680 con 0x7fc78015c510 2024-03-04T08:04:52.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc785200640 1 -- 172.21.15.113:0/794300599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78015c510 msgr2=0x7fc780163190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:52.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc785200640 1 --2- 172.21.15.113:0/794300599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78015c510 0x7fc780163190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc7800c54a0 tx=0x7fc77801c870 comp rx=0 tx=0).stop 2024-03-04T08:04:52.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc785200640 1 -- 172.21.15.113:0/794300599 shutdown_connections 2024-03-04T08:04:52.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc785200640 1 --2- 172.21.15.113:0/794300599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78015c510 0x7fc780163190 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:52.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc785200640 1 --2- 172.21.15.113:0/794300599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc780070010 0x7fc7800c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:52.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc785200640 1 -- 172.21.15.113:0/794300599 >> 172.21.15.113:0/794300599 conn(0x7fc7800709a0 msgr2=0x7fc780070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:52.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc785200640 1 -- 172.21.15.113:0/794300599 shutdown_connections 2024-03-04T08:04:52.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.769+0000 7fc785200640 1 -- 172.21.15.113:0/794300599 wait complete. 2024-03-04T08:04:52.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc785200640 1 Processor -- start 2024-03-04T08:04:52.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc785200640 1 -- start start 2024-03-04T08:04:52.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc785200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780070010 0x7fc780161990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:52.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc785200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc780152b20 0x7fc7800cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:52.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc785200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78015c510 0x7fc78015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:52.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc785200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800c8490 con 0x7fc780070010 2024-03-04T08:04:52.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc785200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc780158810 con 0x7fc78015c510 2024-03-04T08:04:52.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc785200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7801f39f0 con 0x7fc780152b20 2024-03-04T08:04:52.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc77de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780070010 0x7fc780161990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:52.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc77de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780070010 0x7fc780161990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34662/0 (socket says 172.21.15.113:34662) 2024-03-04T08:04:52.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc77de00640 1 -- 172.21.15.113:0/2634221530 learned_addr learned my addr 172.21.15.113:0/2634221530 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:52.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc777e00640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc780152b20 0x7fc7800cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc77e800640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78015c510 0x7fc78015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc77de00640 1 -- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc780152b20 msgr2=0x7fc7800cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc77de00640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc780152b20 0x7fc7800cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc77de00640 1 -- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78015c510 msgr2=0x7fc78015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc77de00640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78015c510 0x7fc78015a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc77de00640 1 -- 172.21.15.113:0/2634221530 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7801f30e0 con 0x7fc780070010 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc77de00640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780070010 0x7fc780161990 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc7700017f0 tx=0x7fc77000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc776000640 1 -- 172.21.15.113:0/2634221530 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770005ae0 con 0x7fc780070010 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc776000640 1 -- 172.21.15.113:0/2634221530 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc770005ca0 con 0x7fc780070010 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc785200640 1 -- 172.21.15.113:0/2634221530 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7800ce2c0 con 0x7fc780070010 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc776000640 1 -- 172.21.15.113:0/2634221530 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700072b0 con 0x7fc780070010 2024-03-04T08:04:52.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.773+0000 7fc785200640 1 -- 172.21.15.113:0/2634221530 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7801f3680 con 0x7fc780070010 2024-03-04T08:04:52.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.777+0000 7fc785200640 1 -- 172.21.15.113:0/2634221530 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc780075c00 con 0x7fc780070010 2024-03-04T08:04:52.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.777+0000 7fc776000640 1 -- 172.21.15.113:0/2634221530 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc770007450 con 0x7fc780070010 2024-03-04T08:04:52.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.777+0000 7fc776000640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc75c066f60 0x7fc75c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:52.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.777+0000 7fc776000640 1 -- 172.21.15.113:0/2634221530 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7fc770085340 con 0x7fc780070010 2024-03-04T08:04:52.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.777+0000 7fc777e00640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc75c066f60 0x7fc75c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:52.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.781+0000 7fc777e00640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc75c066f60 0x7fc75c069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc7801f7710 tx=0x7fc768009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:52.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.781+0000 7fc776000640 1 -- 172.21.15.113:0/2634221530 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc7700556d0 con 0x7fc780070010 2024-03-04T08:04:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.969+0000 7fc785200640 1 -- 172.21.15.113:0/2634221530 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc78006fdf0 con 0x7fc780070010 2024-03-04T08:04:52.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc776000640 1 -- 172.21.15.113:0/2634221530 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5829 (secure 0 0 0) 0x7fc770059770 con 0x7fc780070010 2024-03-04T08:04:52.971 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:52.971 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":118,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:51.232251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6807,7119,6925]},"id":1}]} 2024-03-04T08:04:52.971 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 118 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 -- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc75c066f60 msgr2=0x7fc75c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc75c066f60 0x7fc75c069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc7801f7710 tx=0x7fc768009190 comp rx=0 tx=0).stop 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 -- 172.21.15.113:0/2634221530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780070010 msgr2=0x7fc780161990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780070010 0x7fc780161990 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc7700017f0 tx=0x7fc77000cbe0 comp rx=0 tx=0).stop 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 -- 172.21.15.113:0/2634221530 shutdown_connections 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc78015c510 0x7fc78015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc780152b20 0x7fc7800cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc75c066f60 0x7fc75c069420 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 --2- 172.21.15.113:0/2634221530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc780070010 0x7fc780161990 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 -- 172.21.15.113:0/2634221530 >> 172.21.15.113:0/2634221530 conn(0x7fc7800709a0 msgr2=0x7fc780151b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 -- 172.21.15.113:0/2634221530 shutdown_connections 2024-03-04T08:04:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:52.973+0000 7fc785200640 1 -- 172.21.15.113:0/2634221530 wait complete. 2024-03-04T08:04:52.992 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=1, replacing gid=6534 2024-03-04T08:04:52.992 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.e 2024-03-04T08:04:52.992 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-03-04T08:04:52.992 INFO:tasks.ceph.mds.e:Restarting daemon 2024-03-04T08:04:52.992 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-03-04T08:04:52.996 INFO:tasks.ceph.mds.e:Started 2024-03-04T08:04:52.997 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:04:53.082 INFO:tasks.ceph.mds.e.smithi113.stdout:starting mds.e at 2024-03-04T08:04:53.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f8686800640 1 Processor -- start 2024-03-04T08:04:53.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f8686800640 1 -- start start 2024-03-04T08:04:53.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f8686800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8680155100 0x7f86800c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:53.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f8686800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f868015ad30 0x7f868015b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:53.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f8686800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868015efd0 con 0x7f868006b060 2024-03-04T08:04:53.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f8686800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86800c8890 con 0x7f868015ad30 2024-03-04T08:04:53.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f8686800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868006b4a0 con 0x7f8680155100 2024-03-04T08:04:53.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f8676a00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f868006b060 0x7f86800cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:53176/0 (socket says 172.21.15.113:53176) 2024-03-04T08:04:53.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f867f400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8680155100 0x7f86800c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:53.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f8676a00640 1 -- 172.21.15.113:0/4066564574 learned_addr learned my addr 172.21.15.113:0/4066564574 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:04:53.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f867fe00640 1 --2- 172.21.15.113:0/4066564574 >> v2:172.21.15.136:3300/0 conn(0x7f868015ad30 0x7f868015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:53.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f867fe00640 1 -- 172.21.15.113:0/4066564574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8680155100 msgr2=0x7f86800c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:04:53.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f867fe00640 1 --2- 172.21.15.113:0/4066564574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8680155100 0x7f86800c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:53.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f867ea00640 1 -- 172.21.15.113:0/4066564574 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722059299 0 0) 0x7f868015efd0 con 0x7f868006b060 2024-03-04T08:04:53.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f867fe00640 1 -- 172.21.15.113:0/4066564574 >> v1:172.21.15.136:6789/0 conn(0x7f868006b060 legacy=0x7f86800cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:53.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f867fe00640 1 -- 172.21.15.113:0/4066564574 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800c7ef0 con 0x7f868015ad30 2024-03-04T08:04:53.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.141+0000 7f867f400640 1 --2- 172.21.15.113:0/4066564574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8680155100 0x7f86800c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:04:53.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.145+0000 7f867fe00640 1 --2- 172.21.15.113:0/4066564574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f868015ad30 0x7f868015b140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f86780097c0 tx=0x7f867801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=484ce2033de22ea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:53.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.145+0000 7f867ea00640 1 -- 172.21.15.113:0/4066564574 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780b2070 con 0x7f868015ad30 2024-03-04T08:04:53.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f867ea00640 1 -- 172.21.15.113:0/4066564574 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86780ad640 con 0x7f868015ad30 2024-03-04T08:04:53.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f867ea00640 1 -- 172.21.15.113:0/4066564574 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780ada10 con 0x7f868015ad30 2024-03-04T08:04:53.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 -- 172.21.15.113:0/4066564574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f868015ad30 msgr2=0x7f868015b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:53.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 --2- 172.21.15.113:0/4066564574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f868015ad30 0x7f868015b140 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f86780097c0 tx=0x7f867801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:04:53.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 -- 172.21.15.113:0/4066564574 shutdown_connections 2024-03-04T08:04:53.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 --2- 172.21.15.113:0/4066564574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f868015ad30 0x7f868015b140 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:53.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 --2- 172.21.15.113:0/4066564574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8680155100 0x7f86800c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:53.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 -- 172.21.15.113:0/4066564574 >> 172.21.15.113:0/4066564574 conn(0x7f868006deb0 msgr2=0x7f868006e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:53.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 -- 172.21.15.113:0/4066564574 shutdown_connections 2024-03-04T08:04:53.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 -- 172.21.15.113:0/4066564574 wait complete. 2024-03-04T08:04:53.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 Processor -- start 2024-03-04T08:04:53.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 -- start start 2024-03-04T08:04:53.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f868006b060 0x7f86801f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8680155100 0x7f868014bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 0x7f86800cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868015ebb0 con 0x7f868015ad30 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868015b680 con 0x7f8680155100 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f8686800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868015ed30 con 0x7f868006b060 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.149+0000 7f867fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 0x7f86800cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f867f400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f868006b060 0x7f86801f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f867fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 0x7f86800cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34694/0 (socket says 172.21.15.113:34694) 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f867fe00640 1 -- 172.21.15.113:0/536598357 learned_addr learned my addr 172.21.15.113:0/536598357 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f8676a00640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8680155100 0x7f868014bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f867fe00640 1 -- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f868006b060 msgr2=0x7f86801f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f867fe00640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f868006b060 0x7f86801f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:53.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f867fe00640 1 -- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8680155100 msgr2=0x7f868014bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:53.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f867fe00640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8680155100 0x7f868014bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:53.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f867fe00640 1 -- 172.21.15.113:0/536598357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868006ca70 con 0x7f868015ad30 2024-03-04T08:04:53.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f867fe00640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 0x7f86800cb510 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f86780097c0 tx=0x7f8678019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:53.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f8677e00640 1 -- 172.21.15.113:0/536598357 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780bd070 con 0x7f868015ad30 2024-03-04T08:04:53.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f8677e00640 1 -- 172.21.15.113:0/536598357 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8678004030 con 0x7f868015ad30 2024-03-04T08:04:53.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f8686800640 1 -- 172.21.15.113:0/536598357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86800cc840 con 0x7f868015ad30 2024-03-04T08:04:53.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f8677e00640 1 -- 172.21.15.113:0/536598357 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780b2040 con 0x7f868015ad30 2024-03-04T08:04:53.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f8686800640 1 -- 172.21.15.113:0/536598357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86800ccae0 con 0x7f868015ad30 2024-03-04T08:04:53.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f8686800640 1 -- 172.21.15.113:0/536598357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8680075c00 con 0x7f868015ad30 2024-03-04T08:04:53.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.153+0000 7f8677e00640 1 -- 172.21.15.113:0/536598357 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86780c0070 con 0x7f868015ad30 2024-03-04T08:04:53.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.157+0000 7f8677e00640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f865c066f90 0x7f865c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:04:53.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.157+0000 7f8677e00640 1 -- 172.21.15.113:0/536598357 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f867811e740 con 0x7f868015ad30 2024-03-04T08:04:53.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.157+0000 7f867f400640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f865c066f90 0x7f865c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:04:53.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.157+0000 7f867f400640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f865c066f90 0x7f865c069450 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8670009bd0 tx=0x7f867000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:04:53.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.161+0000 7f8677e00640 1 -- 172.21.15.113:0/536598357 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8678016090 con 0x7f868015ad30 2024-03-04T08:04:53.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.348+0000 7f8686800640 1 -- 172.21.15.113:0/536598357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86800c73d0 con 0x7f868015ad30 2024-03-04T08:04:53.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8677e00640 1 -- 172.21.15.113:0/536598357 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f86780eead0 con 0x7f868015ad30 2024-03-04T08:04:53.350 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:04:53.351 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":119,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:51.232251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119,6807]},"id":1}]} 2024-03-04T08:04:53.351 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 119 2024-03-04T08:04:53.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 -- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f865c066f90 msgr2=0x7f865c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f865c066f90 0x7f865c069450 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8670009bd0 tx=0x7f867000b040 comp rx=0 tx=0).stop 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 -- 172.21.15.113:0/536598357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 msgr2=0x7f86800cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 0x7f86800cb510 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f86780097c0 tx=0x7f8678019240 comp rx=0 tx=0).stop 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 -- 172.21.15.113:0/536598357 shutdown_connections 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f865c066f90 0x7f865c069450 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f868015ad30 0x7f86800cb510 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8680155100 0x7f868014bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 --2- 172.21.15.113:0/536598357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f868006b060 0x7f86801f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 -- 172.21.15.113:0/536598357 >> 172.21.15.113:0/536598357 conn(0x7f868006deb0 msgr2=0x7f8680159300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 -- 172.21.15.113:0/536598357 shutdown_connections 2024-03-04T08:04:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:04:53.352+0000 7f8686800640 1 -- 172.21.15.113:0/536598357 wait complete. 2024-03-04T08:04:53.372 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-03-04T08:04:53.372 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 39 secs before thrashing 2024-03-04T08:05:02.566 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 12% (47/376) Updating files: 13% (49/376) Updating files: 14% (53/376) Updating files: 14% (54/376) Updating files: 15% (57/376) Updating files: 16% (61/376) Updating files: 17% (64/376) Updating files: 18% (68/376) Updating files: 19% (72/376) Updating files: 20% (76/376) Updating files: 21% (79/376) Updating files: 22% (83/376) Updating files: 23% (87/376) Updating files: 24% (91/376) Updating files: 25% (94/376) Updating files: 26% (98/376) Updating files: 27% (102/376) Updating files: 28% (106/376) Updating files: 29% (110/376) Updating files: 30% (113/376) Updating files: 31% (117/376) Updating files: 32% (121/376) Updating files: 33% (125/376) Updating files: 34% (128/376) Updating files: 35% (132/376) Updating files: 36% (136/376) Updating files: 37% (140/376) Updating files: 38% (143/376) Updating files: 39% (147/376) Updating files: 40% (151/376) Updating files: 40% (154/376) Updating files: 41% (155/376) Updating files: 42% (158/376) Updating files: 43% (162/376) Updating files: 44% (166/376) Updating files: 45% (170/376) Updating files: 46% (173/376) Updating files: 47% (177/376) Updating files: 47% (179/376) Updating files: 48% (181/376) Updating files: 49% (185/376) Updating files: 50% (188/376) Updating files: 51% (192/376) Updating files: 52% (196/376) Updating files: 53% (200/376) Updating files: 54% (204/376) Updating files: 55% (207/376) Updating files: 56% (211/376) Updating files: 57% (215/376) Updating files: 57% (217/376) Updating files: 58% (219/376) Updating files: 59% (222/376) Updating files: 60% (226/376) Updating files: 61% (230/376) Updating files: 62% (234/376) Updating files: 63% (237/376) Updating files: 64% (241/376) Updating files: 65% (245/376) Updating files: 66% (249/376) Updating files: 67% (252/376) Updating files: 68% (256/376) Updating files: 69% (260/376) Updating files: 70% (264/376) Updating files: 71% (267/376) Updating files: 72% (271/376) Updating files: 73% (275/376) Updating files: 74% (279/376) Updating files: 75% (282/376) Updating files: 76% (286/376) Updating files: 77% (290/376) Updating files: 77% (292/376) Updating files: 78% (294/376) Updating files: 79% (298/376) Updating files: 80% (301/376) Updating files: 80% (304/376) Updating files: 81% (305/376) Updating files: 82% (309/376) Updating files: 83% (313/376) Updating files: 84% (316/376) Updating files: 85% (320/376) Updating files: 86% (324/376) Updating files: 87% (328/376) Updating files: 88% (331/376) Updating files: 89% (335/376) Updating files: 90% (339/376) Updating files: 91% (343/376) Updating files: 92% (346/376) Updating files: 93% (350/376) Updating files: 94% (354/376) Updating files: 94% (356/376) Updating files: 95% (358/376) Updating files: 96% (361/376) Updating files: 97% (365/376) Updating files: 98% (369/376) Updating files: 99% (373/376) Updating files: 100% (376/376) Updating files: 100% (376/376), done. 2024-03-04T08:05:03.191 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 237f3f1e8d8 v0.61 2024-03-04T08:05:03.200 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.61 2024-03-04T08:05:03.852 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 62 -eq 48 ] 2024-03-04T08:05:03.852 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.62 2024-03-04T08:05:03.852 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.62 2024-03-04T08:05:03.853 INFO:tasks.workunit.client.0.smithi113.stdout:v0.62 2024-03-04T08:05:03.853 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.62 2024-03-04T08:05:04.626 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:05:04.631 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:05:08.191 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 18% (11/59) Updating files: 20% (12/59) Updating files: 22% (13/59) Updating files: 23% (14/59) Updating files: 25% (15/59) Updating files: 27% (16/59) Updating files: 28% (17/59) Updating files: 30% (18/59) Updating files: 32% (19/59) Updating files: 33% (20/59) Updating files: 35% (21/59) Updating files: 37% (22/59) Updating files: 38% (23/59) Updating files: 40% (24/59) Updating files: 42% (25/59) Updating files: 44% (26/59) Updating files: 45% (27/59) Updating files: 47% (28/59) Updating files: 49% (29/59) Updating files: 50% (30/59) Updating files: 52% (31/59) Updating files: 54% (32/59) Updating files: 55% (33/59) Updating files: 57% (34/59) Updating files: 59% (35/59) Updating files: 61% (36/59) Updating files: 62% (37/59) Updating files: 64% (38/59) Updating files: 66% (39/59) Updating files: 67% (40/59) Updating files: 69% (41/59) Updating files: 71% (42/59) Updating files: 72% (43/59) Updating files: 74% (44/59) Updating files: 76% (45/59) Updating files: 77% (46/59) Updating files: 79% (47/59) Updating files: 81% (48/59) Updating files: 83% (49/59) Updating files: 84% (50/59) Updating files: 86% (51/59) Updating files: 88% (52/59) Updating files: 89% (53/59) Updating files: 91% (54/59) Updating files: 93% (55/59) Updating files: 94% (56/59) Updating files: 96% (57/59) Updating files: 98% (58/59) Updating files: 100% (59/59) Updating files: 100% (59/59), done. 2024-03-04T08:05:08.251 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 9382379cc6b v0.62 2024-03-04T08:05:08.257 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.62 2024-03-04T08:05:08.463 INFO:tasks.workunit.client.0.smithi113.stdout:v0.63 2024-03-04T08:05:08.463 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 63 -eq 48 ] 2024-03-04T08:05:08.463 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.63 2024-03-04T08:05:08.463 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.63 2024-03-04T08:05:08.463 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.63 2024-03-04T08:05:12.227 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 21% (32/146) Updating files: 22% (33/146) Updating files: 23% (34/146) Updating files: 24% (36/146) Updating files: 25% (37/146) Updating files: 26% (38/146) Updating files: 27% (40/146) Updating files: 28% (41/146) Updating files: 29% (43/146) Updating files: 30% (44/146) Updating files: 31% (46/146) Updating files: 32% (47/146) Updating files: 33% (49/146) Updating files: 34% (50/146) Updating files: 35% (52/146) Updating files: 36% (53/146) Updating files: 37% (55/146) Updating files: 38% (56/146) Updating files: 39% (57/146) Updating files: 40% (59/146) Updating files: 41% (60/146) Updating files: 42% (62/146) Updating files: 43% (63/146) Updating files: 44% (65/146) Updating files: 45% (66/146) Updating files: 46% (68/146) Updating files: 47% (69/146) Updating files: 48% (71/146) Updating files: 49% (72/146) Updating files: 50% (73/146) Updating files: 51% (75/146) Updating files: 52% (76/146) Updating files: 53% (78/146) Updating files: 54% (79/146) Updating files: 55% (81/146) Updating files: 56% (82/146) Updating files: 57% (84/146) Updating files: 58% (85/146) Updating files: 59% (87/146) Updating files: 60% (88/146) Updating files: 61% (90/146) Updating files: 62% (91/146) Updating files: 63% (92/146) Updating files: 64% (94/146) Updating files: 65% (95/146) Updating files: 66% (97/146) Updating files: 67% (98/146) Updating files: 68% (100/146) Updating files: 69% (101/146) Updating files: 70% (103/146) Updating files: 71% (104/146) Updating files: 72% (106/146) Updating files: 73% (107/146) Updating files: 74% (109/146) Updating files: 75% (110/146) Updating files: 76% (111/146) Updating files: 77% (113/146) Updating files: 78% (114/146) Updating files: 79% (116/146) Updating files: 80% (117/146) Updating files: 81% (119/146) Updating files: 82% (120/146) Updating files: 83% (122/146) Updating files: 84% (123/146) Updating files: 85% (125/146) Updating files: 86% (126/146) Updating files: 87% (128/146) Updating files: 88% (129/146) Updating files: 89% (130/146) Updating files: 90% (132/146) Updating files: 91% (133/146) Updating files: 92% (135/146) Updating files: 93% (136/146) Updating files: 94% (138/146) Updating files: 95% (139/146) Updating files: 96% (141/146) Updating files: 97% (142/146) Updating files: 98% (144/146) Updating files: 99% (145/146) Updating files: 100% (146/146) Updating files: 100% (146/146), done. 2024-03-04T08:05:12.304 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 054e96cf79e v0.63 2024-03-04T08:05:12.312 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.63 2024-03-04T08:05:12.907 INFO:tasks.workunit.client.0.smithi113.stdout:v0.64 2024-03-04T08:05:12.907 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 64 -eq 48 ] 2024-03-04T08:05:12.907 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.64 2024-03-04T08:05:12.908 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.64 2024-03-04T08:05:12.908 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.64 2024-03-04T08:05:14.063 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 42e06c12db6 v0.64 2024-03-04T08:05:14.076 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.64 2024-03-04T08:05:14.282 INFO:tasks.workunit.client.0.smithi113.stdout:v0.65 2024-03-04T08:05:14.283 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 65 -eq 48 ] 2024-03-04T08:05:14.283 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.65 2024-03-04T08:05:14.283 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.65 2024-03-04T08:05:14.283 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.65 2024-03-04T08:05:19.174 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 18% (37/203) Updating files: 19% (39/203) Updating files: 20% (41/203) Updating files: 21% (43/203) Updating files: 22% (45/203) Updating files: 23% (47/203) Updating files: 24% (49/203) Updating files: 25% (51/203) Updating files: 26% (53/203) Updating files: 27% (55/203) Updating files: 28% (57/203) Updating files: 29% (59/203) Updating files: 30% (61/203) Updating files: 31% (63/203) Updating files: 32% (65/203) Updating files: 33% (67/203) Updating files: 34% (70/203) Updating files: 35% (72/203) Updating files: 36% (74/203) Updating files: 37% (76/203) Updating files: 38% (78/203) Updating files: 39% (80/203) Updating files: 40% (82/203) Updating files: 41% (84/203) Updating files: 42% (86/203) Updating files: 43% (88/203) Updating files: 44% (90/203) Updating files: 45% (92/203) Updating files: 46% (94/203) Updating files: 47% (96/203) Updating files: 48% (98/203) Updating files: 49% (100/203) Updating files: 50% (102/203) Updating files: 51% (104/203) Updating files: 52% (106/203) Updating files: 53% (108/203) Updating files: 54% (110/203) Updating files: 55% (112/203) Updating files: 56% (114/203) Updating files: 57% (116/203) Updating files: 58% (118/203) Updating files: 59% (120/203) Updating files: 60% (122/203) Updating files: 61% (124/203) Updating files: 62% (126/203) Updating files: 63% (128/203) Updating files: 64% (130/203) Updating files: 65% (132/203) Updating files: 66% (134/203) Updating files: 67% (137/203) Updating files: 68% (139/203) Updating files: 69% (141/203) Updating files: 70% (143/203) Updating files: 71% (145/203) Updating files: 72% (147/203) Updating files: 73% (149/203) Updating files: 74% (151/203) Updating files: 75% (153/203) Updating files: 76% (155/203) Updating files: 77% (157/203) Updating files: 78% (159/203) Updating files: 79% (161/203) Updating files: 80% (163/203) Updating files: 81% (165/203) Updating files: 82% (167/203) Updating files: 83% (169/203) Updating files: 84% (171/203) Updating files: 85% (173/203) Updating files: 86% (175/203) Updating files: 87% (177/203) Updating files: 88% (179/203) Updating files: 89% (181/203) Updating files: 90% (183/203) Updating files: 91% (185/203) Updating files: 92% (187/203) Updating files: 93% (189/203) Updating files: 94% (191/203) Updating files: 95% (193/203) Updating files: 96% (195/203) Updating files: 97% (197/203) Updating files: 98% (199/203) Updating files: 99% (201/203) Updating files: 100% (203/203) Updating files: 100% (203/203), done. 2024-03-04T08:05:19.440 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at c2d517ef966 v0.65 2024-03-04T08:05:19.446 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.65 2024-03-04T08:05:19.773 INFO:tasks.workunit.client.0.smithi113.stdout:v0.66 2024-03-04T08:05:19.774 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 66 -eq 48 ] 2024-03-04T08:05:19.774 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.66 2024-03-04T08:05:19.774 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.66 2024-03-04T08:05:19.774 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.66 2024-03-04T08:05:28.281 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 4% (7/152) Updating files: 5% (8/152) Updating files: 6% (10/152) Updating files: 7% (11/152) Updating files: 8% (13/152) Updating files: 9% (14/152) Updating files: 10% (16/152) Updating files: 11% (17/152) Updating files: 12% (19/152) Updating files: 13% (20/152) Updating files: 14% (22/152) Updating files: 15% (23/152) Updating files: 16% (25/152) Updating files: 17% (26/152) Updating files: 18% (28/152) Updating files: 19% (29/152) Updating files: 20% (31/152) Updating files: 21% (32/152) Updating files: 22% (34/152) Updating files: 23% (35/152) Updating files: 24% (37/152) Updating files: 25% (38/152) Updating files: 26% (40/152) Updating files: 27% (42/152) Updating files: 28% (43/152) Updating files: 29% (45/152) Updating files: 30% (46/152) Updating files: 31% (48/152) Updating files: 32% (49/152) Updating files: 33% (51/152) Updating files: 34% (52/152) Updating files: 35% (54/152) Updating files: 36% (55/152) Updating files: 37% (57/152) Updating files: 38% (58/152) Updating files: 39% (60/152) Updating files: 40% (61/152) Updating files: 40% (62/152) Updating files: 41% (63/152) Updating files: 42% (64/152) Updating files: 43% (66/152) Updating files: 44% (67/152) Updating files: 45% (69/152) Updating files: 46% (70/152) Updating files: 47% (72/152) Updating files: 48% (73/152) Updating files: 49% (75/152) Updating files: 50% (76/152) Updating files: 51% (78/152) Updating files: 52% (80/152) Updating files: 53% (81/152) Updating files: 54% (83/152) Updating files: 55% (84/152) Updating files: 56% (86/152) Updating files: 57% (87/152) Updating files: 58% (89/152) Updating files: 59% (90/152) Updating files: 60% (92/152) Updating files: 61% (93/152) Updating files: 62% (95/152) Updating files: 63% (96/152) Updating files: 64% (98/152) Updating files: 65% (99/152) Updating files: 66% (101/152) Updating files: 67% (102/152) Updating files: 68% (104/152) Updating files: 69% (105/152) Updating files: 70% (107/152) Updating files: 71% (108/152) Updating files: 72% (110/152) Updating files: 73% (111/152) Updating files: 74% (113/152) Updating files: 75% (114/152) Updating files: 76% (116/152) Updating files: 77% (118/152) Updating files: 78% (119/152) Updating files: 79% (121/152) Updating files: 80% (122/152) Updating files: 81% (124/152) Updating files: 82% (125/152) Updating files: 83% (127/152) Updating files: 84% (128/152) Updating files: 84% (129/152) Updating files: 85% (130/152) Updating files: 86% (131/152) Updating files: 87% (133/152) Updating files: 88% (134/152) Updating files: 89% (136/152) Updating files: 90% (137/152) Updating files: 91% (139/152) Updating files: 92% (140/152) Updating files: 93% (142/152) Updating files: 94% (143/152) Updating files: 95% (145/152) Updating files: 96% (146/152) Updating files: 97% (148/152) Updating files: 98% (149/152) Updating files: 99% (151/152) Updating files: 100% (152/152) Updating files: 100% (152/152), done. 2024-03-04T08:05:28.338 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at b6b48dbefad v0.66 2024-03-04T08:05:28.344 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.66 2024-03-04T08:05:28.922 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 67 -eq 48 ] 2024-03-04T08:05:28.922 INFO:tasks.workunit.client.0.smithi113.stdout:v0.67 2024-03-04T08:05:28.923 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.67 2024-03-04T08:05:28.923 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.67 2024-03-04T08:05:28.923 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.67 2024-03-04T08:05:32.374 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.515+0000 7f87bc600640 1 Processor -- start 2024-03-04T08:05:32.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.515+0000 7f87bc600640 1 -- start start 2024-03-04T08:05:32.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87bc600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87b4070120 0x7f87b4158930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:32.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87bc600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f87b40c9db0 0x7f87b4158e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:32.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87bc600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40c5380 con 0x7f87b4152d60 2024-03-04T08:05:32.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87bc600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b40c3440 con 0x7f87b4070120 2024-03-04T08:05:32.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87bc600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b406f620 con 0x7f87b40c9db0 2024-03-04T08:05:32.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87b9200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87b4070120 0x7f87b4158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:32.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87b9200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87b4070120 0x7f87b4158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57098/0 (socket says 172.21.15.113:57098) 2024-03-04T08:05:32.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87b9200640 1 -- 172.21.15.113:0/2447219519 learned_addr learned my addr 172.21.15.113:0/2447219519 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:32.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87ba600640 1 --2- 172.21.15.113:0/2447219519 >> v2:172.21.15.136:3300/0 conn(0x7f87b40c9db0 0x7f87b4158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:32.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87b9200640 1 -- 172.21.15.113:0/2447219519 >> v2:172.21.15.136:3300/0 conn(0x7f87b40c9db0 msgr2=0x7f87b4158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:32.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87b9200640 1 --2- 172.21.15.113:0/2447219519 >> v2:172.21.15.136:3300/0 conn(0x7f87b40c9db0 0x7f87b4158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:32.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87b9200640 1 -- 172.21.15.113:0/2447219519 >> v1:172.21.15.136:6789/0 conn(0x7f87b4152d60 legacy=0x7f87b40ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:32.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.519+0000 7f87b9200640 1 -- 172.21.15.113:0/2447219519 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40c36e0 con 0x7f87b4070120 2024-03-04T08:05:32.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87b9200640 1 --2- 172.21.15.113:0/2447219519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4070120 0x7f87b4158930 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f87b41593b0 tx=0x7f87a001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0014484069c791 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:32.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87b3e00640 1 -- 172.21.15.113:0/2447219519 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a00b5070 con 0x7f87b4070120 2024-03-04T08:05:32.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87b3e00640 1 -- 172.21.15.113:0/2447219519 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a00a8a10 con 0x7f87b4070120 2024-03-04T08:05:32.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87b3e00640 1 -- 172.21.15.113:0/2447219519 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a00a8de0 con 0x7f87b4070120 2024-03-04T08:05:32.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87bc600640 1 -- 172.21.15.113:0/2447219519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4070120 msgr2=0x7f87b4158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:32.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87bc600640 1 --2- 172.21.15.113:0/2447219519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4070120 0x7f87b4158930 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f87b41593b0 tx=0x7f87a001ed10 comp rx=0 tx=0).stop 2024-03-04T08:05:32.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87bc600640 1 -- 172.21.15.113:0/2447219519 shutdown_connections 2024-03-04T08:05:32.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87bc600640 1 --2- 172.21.15.113:0/2447219519 >> v2:172.21.15.136:3300/0 conn(0x7f87b40c9db0 0x7f87b4158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:32.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87bc600640 1 --2- 172.21.15.113:0/2447219519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4070120 0x7f87b4158930 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:32.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87bc600640 1 -- 172.21.15.113:0/2447219519 >> 172.21.15.113:0/2447219519 conn(0x7f87b4066db0 msgr2=0x7f87b40671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:32.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87bc600640 1 -- 172.21.15.113:0/2447219519 shutdown_connections 2024-03-04T08:05:32.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87bc600640 1 -- 172.21.15.113:0/2447219519 wait complete. 2024-03-04T08:05:32.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87bc600640 1 Processor -- start 2024-03-04T08:05:32.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.523+0000 7f87bc600640 1 -- start start 2024-03-04T08:05:32.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87bc600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87b4070120 0x7f87b41517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:32.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87bc600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87b40c9db0 0x7f87b415c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:32.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87bc600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4152d60 0x7f87b415c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:32.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87bc600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40c39c0 con 0x7f87b4152d60 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87bc600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40c5020 con 0x7f87b4070120 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87bc600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40c3b40 con 0x7f87b40c9db0 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87ba600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4152d60 0x7f87b415c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87b9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87b4070120 0x7f87b41517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87b9200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87b40c9db0 0x7f87b415c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87b9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87b4070120 0x7f87b41517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:35248/0 (socket says 172.21.15.113:35248) 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87b9200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87b40c9db0 0x7f87b415c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:32960/0 (socket says 172.21.15.113:32960) 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87ba600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4152d60 0x7f87b415c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57106/0 (socket says 172.21.15.113:57106) 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87b9c00640 1 -- 172.21.15.113:0/3489142514 learned_addr learned my addr 172.21.15.113:0/3489142514 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87ba600640 1 -- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87b40c9db0 msgr2=0x7f87b415c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87ba600640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87b40c9db0 0x7f87b415c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87ba600640 1 -- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87b4070120 msgr2=0x7f87b41517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87ba600640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87b4070120 0x7f87b41517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:32.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87ba600640 1 -- 172.21.15.113:0/3489142514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b41582c0 con 0x7f87b4152d60 2024-03-04T08:05:32.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87ba600640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4152d60 0x7f87b415c8f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f87ac007d70 tx=0x7f87ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:32.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87b1600640 1 -- 172.21.15.113:0/3489142514 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00ed70 con 0x7f87b4152d60 2024-03-04T08:05:32.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87b1600640 1 -- 172.21.15.113:0/3489142514 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac0039a0 con 0x7f87b4152d60 2024-03-04T08:05:32.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87bc600640 1 -- 172.21.15.113:0/3489142514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b41585a0 con 0x7f87b4152d60 2024-03-04T08:05:32.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87b1600640 1 -- 172.21.15.113:0/3489142514 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac003d90 con 0x7f87b4152d60 2024-03-04T08:05:32.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.527+0000 7f87bc600640 1 -- 172.21.15.113:0/3489142514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b4158b60 con 0x7f87b4152d60 2024-03-04T08:05:32.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.531+0000 7f87bc600640 1 -- 172.21.15.113:0/3489142514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b4075c00 con 0x7f87b4152d60 2024-03-04T08:05:32.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.531+0000 7f87b1600640 1 -- 172.21.15.113:0/3489142514 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87ac01b020 con 0x7f87b4152d60 2024-03-04T08:05:32.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.531+0000 7f87b1600640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8794066f90 0x7f8794069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:32.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.531+0000 7f87b1600640 1 -- 172.21.15.113:0/3489142514 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f87ac085730 con 0x7f87b4152d60 2024-03-04T08:05:32.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.531+0000 7f87b9c00640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8794066f90 0x7f8794069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:32.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.535+0000 7f87b9c00640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8794066f90 0x7f8794069450 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f87a80018b0 tx=0x7f87a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:32.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.543+0000 7f87b1600640 1 -- 172.21.15.113:0/3489142514 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f87ac055990 con 0x7f87b4152d60 2024-03-04T08:05:32.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.827+0000 7f87bc600640 1 -- 172.21.15.113:0/3489142514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87b4152600 con 0x7f87b4152d60 2024-03-04T08:05:32.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87b1600640 1 -- 172.21.15.113:0/3489142514 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f87ac059a30 con 0x7f87b4152d60 2024-03-04T08:05:32.827 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:32.827 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":119,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:51.232251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119,6807]},"id":1}]} 2024-03-04T08:05:32.827 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 119 2024-03-04T08:05:32.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 -- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8794066f90 msgr2=0x7f8794069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:32.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8794066f90 0x7f8794069450 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f87a80018b0 tx=0x7f87a8009040 comp rx=0 tx=0).stop 2024-03-04T08:05:32.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 -- 172.21.15.113:0/3489142514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4152d60 msgr2=0x7f87b415c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:32.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4152d60 0x7f87b415c8f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f87ac007d70 tx=0x7f87ac00e4b0 comp rx=0 tx=0).stop 2024-03-04T08:05:32.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 -- 172.21.15.113:0/3489142514 shutdown_connections 2024-03-04T08:05:32.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8794066f90 0x7f8794069450 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:32.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87b4152d60 0x7f87b415c8f0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:32.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f87b40c9db0 0x7f87b415c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:32.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 --2- 172.21.15.113:0/3489142514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87b4070120 0x7f87b41517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:32.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 -- 172.21.15.113:0/3489142514 >> 172.21.15.113:0/3489142514 conn(0x7f87b4066db0 msgr2=0x7f87b4067190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:32.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 -- 172.21.15.113:0/3489142514 shutdown_connections 2024-03-04T08:05:32.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:32.831+0000 7f87bc600640 1 -- 172.21.15.113:0/3489142514 wait complete. 2024-03-04T08:05:32.850 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=2) 2024-03-04T08:05:32.851 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-04T08:05:32.851 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:05:32.864 INFO:tasks.ceph.mds.a:Stopped 2024-03-04T08:05:32.864 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:33.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.043+0000 7f0d53000640 1 Processor -- start 2024-03-04T08:05:33.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.043+0000 7f0d53000640 1 -- start start 2024-03-04T08:05:33.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.043+0000 7f0d53000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0d4c06ff20 0x7f0d4c06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:33.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.043+0000 7f0d53000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c0ca0e0 con 0x7f0d4c158010 2024-03-04T08:05:33.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.043+0000 7f0d53000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c15aa30 con 0x7f0d4c0ce120 2024-03-04T08:05:33.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.043+0000 7f0d53000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4c06f450 con 0x7f0d4c06ff20 2024-03-04T08:05:33.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0d4c158010 0x7f0d4c15b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44940/0 (socket says 172.21.15.113:44940) 2024-03-04T08:05:33.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4be00640 1 -- 172.21.15.113:0/1743205140 learned_addr learned my addr 172.21.15.113:0/1743205140 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:05:33.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d43e00640 1 --2- 172.21.15.113:0/1743205140 >> v2:172.21.15.136:3300/0 conn(0x7f0d4c06ff20 0x7f0d4c06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:33.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423953584 0 0) 0x7f0d4c0ca0e0 con 0x7f0d4c158010 2024-03-04T08:05:33.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d30003530 con 0x7f0d4c158010 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 247184676 0 0) 0x7f0d30003530 con 0x7f0d4c158010 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 >> v2:172.21.15.136:3300/0 conn(0x7f0d4c06ff20 msgr2=0x7f0d4c06dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 --2- 172.21.15.113:0/1743205140 >> v2:172.21.15.136:3300/0 conn(0x7f0d4c06ff20 0x7f0d4c06dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 >> v1:172.21.15.136:6789/0 conn(0x7f0d4c0ce120 legacy=0x7f0d4c1652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c06c590 con 0x7f0d4c158010 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f0d3c001680 con 0x7f0d4c158010 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 >> v1:172.21.15.113:6789/0 conn(0x7f0d4c158010 legacy=0x7f0d4c15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d300039c0 0x7f0d30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d30004810 0x7f0d30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d300051d0 0x7f0d3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c06f450 con 0x7f0d30004810 2024-03-04T08:05:33.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3000c070 con 0x7f0d300039c0 2024-03-04T08:05:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3000c270 con 0x7f0d300051d0 2024-03-04T08:05:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d43e00640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d30004810 0x7f0d30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d4be00640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d300039c0 0x7f0d30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d51000640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d300051d0 0x7f0d3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d43e00640 1 -- 172.21.15.113:0/1743205140 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d300051d0 msgr2=0x7f0d3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d43e00640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d300051d0 0x7f0d3000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d43e00640 1 -- 172.21.15.113:0/1743205140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d300039c0 msgr2=0x7f0d30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d43e00640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d300039c0 0x7f0d30003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d43e00640 1 -- 172.21.15.113:0/1743205140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d30003530 con 0x7f0d30004810 2024-03-04T08:05:33.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.047+0000 7f0d51000640 1 -- 172.21.15.113:0/1743205140 reap_dead start 2024-03-04T08:05:33.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d43e00640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d30004810 0x7f0d30004c90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009ab0 tx=0x7f0d3401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce11d5e0b1c03a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:33.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340096b0 con 0x7f0d30004810 2024-03-04T08:05:33.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d340a6460 con 0x7f0d30004810 2024-03-04T08:05:33.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d4b400640 1 -- 172.21.15.113:0/1743205140 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340a6850 con 0x7f0d30004810 2024-03-04T08:05:33.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d53000640 1 -- 172.21.15.113:0/1743205140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d30004810 msgr2=0x7f0d30004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:33.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d53000640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d30004810 0x7f0d30004c90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009ab0 tx=0x7f0d3401ca50 comp rx=0 tx=0).stop 2024-03-04T08:05:33.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d53000640 1 -- 172.21.15.113:0/1743205140 shutdown_connections 2024-03-04T08:05:33.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d53000640 1 --2- 172.21.15.113:0/1743205140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d30004810 0x7f0d30004c90 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:33.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d53000640 1 -- 172.21.15.113:0/1743205140 >> 172.21.15.113:0/1743205140 conn(0x7f0d4c0734d0 msgr2=0x7f0d4c066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:33.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d53000640 1 -- 172.21.15.113:0/1743205140 shutdown_connections 2024-03-04T08:05:33.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d53000640 1 -- 172.21.15.113:0/1743205140 wait complete. 2024-03-04T08:05:33.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d53000640 1 Processor -- start 2024-03-04T08:05:33.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.051+0000 7f0d53000640 1 -- start start 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d53000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d30004810 0x7f0d4c152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d53000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d4c0c31b0 0x7f0d4c0afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d53000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4c0aeca0 0x7f0d4c0b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d53000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c06f980 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d53000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c15ad80 con 0x7f0d4c0c31b0 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d53000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c1583f0 con 0x7f0d30004810 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d51000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4c0aeca0 0x7f0d4c0b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d51000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4c0aeca0 0x7f0d4c0b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57120/0 (socket says 172.21.15.113:57120) 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d51000640 1 -- 172.21.15.113:0/4092426916 learned_addr learned my addr 172.21.15.113:0/4092426916 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d43e00640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d4c0c31b0 0x7f0d4c0afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:33.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d4be00640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d30004810 0x7f0d4c152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d51000640 1 -- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d30004810 msgr2=0x7f0d4c152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d51000640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d30004810 0x7f0d4c152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d51000640 1 -- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d4c0c31b0 msgr2=0x7f0d4c0afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d51000640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d4c0c31b0 0x7f0d4c0afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d51000640 1 -- 172.21.15.113:0/4092426916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c0c3c60 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d4be00640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d30004810 0x7f0d4c152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d51000640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4c0aeca0 0x7f0d4c0b0240 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c06ec10 tx=0x7f0d4400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d48c00640 1 -- 172.21.15.113:0/4092426916 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44003b90 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d48c00640 1 -- 172.21.15.113:0/4092426916 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d44018070 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d48c00640 1 -- 172.21.15.113:0/4092426916 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d440050d0 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d53000640 1 -- 172.21.15.113:0/4092426916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c0c36a0 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d53000640 1 -- 172.21.15.113:0/4092426916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c151aa0 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d53000640 1 -- 172.21.15.113:0/4092426916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c075c00 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.055+0000 7f0d48c00640 1 -- 172.21.15.113:0/4092426916 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d44005270 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.059+0000 7f0d48c00640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d28066f90 0x7f0d28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:33.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.059+0000 7f0d48c00640 1 -- 172.21.15.113:0/4092426916 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f0d440852c0 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.059+0000 7f0d4be00640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d28066f90 0x7f0d28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:33.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.059+0000 7f0d4be00640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d28066f90 0x7f0d28069450 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c001060 tx=0x7f0d3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:33.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.059+0000 7f0d48c00640 1 -- 172.21.15.113:0/4092426916 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0d4c0c3c60 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 -- 172.21.15.113:0/4092426916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d4c0cc210 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d48c00640 1 -- 172.21.15.113:0/4092426916 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f0d44055b70 con 0x7f0d4c0aeca0 2024-03-04T08:05:33.252 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:33.253 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":119,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:51.232251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119,6807]},"id":1}]} 2024-03-04T08:05:33.253 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 119 2024-03-04T08:05:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 -- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d28066f90 msgr2=0x7f0d28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d28066f90 0x7f0d28069450 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c001060 tx=0x7f0d3c009040 comp rx=0 tx=0).stop 2024-03-04T08:05:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 -- 172.21.15.113:0/4092426916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4c0aeca0 msgr2=0x7f0d4c0b0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4c0aeca0 0x7f0d4c0b0240 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c06ec10 tx=0x7f0d4400e970 comp rx=0 tx=0).stop 2024-03-04T08:05:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 -- 172.21.15.113:0/4092426916 shutdown_connections 2024-03-04T08:05:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d28066f90 0x7f0d28069450 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4c0aeca0 0x7f0d4c0b0240 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d4c0c31b0 0x7f0d4c0afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 --2- 172.21.15.113:0/4092426916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d30004810 0x7f0d4c152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 -- 172.21.15.113:0/4092426916 >> 172.21.15.113:0/4092426916 conn(0x7f0d4c0734d0 msgr2=0x7f0d4c154eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.255+0000 7f0d53000640 1 -- 172.21.15.113:0/4092426916 shutdown_connections 2024-03-04T08:05:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:33.259+0000 7f0d53000640 1 -- 172.21.15.113:0/4092426916 wait complete. 2024-03-04T08:05:33.273 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:05:34.747 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:05:34.750 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:05:35.274 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:35.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f23a6c00640 1 Processor -- start 2024-03-04T08:05:35.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f23a6c00640 1 -- start start 2024-03-04T08:05:35.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f23a6c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f23a015b2f0 0x7f23a00c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:35.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f23a6c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a00c7d80 con 0x7f23a0154fe0 2024-03-04T08:05:35.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f23a6c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a00c8060 con 0x7f23a0154c00 2024-03-04T08:05:35.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f23a6c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a015efb0 con 0x7f23a015b2f0 2024-03-04T08:05:35.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f2397e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f23a0154fe0 0x7f23a015abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44942/0 (socket says 172.21.15.113:44942) 2024-03-04T08:05:35.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f2397e00640 1 -- 172.21.15.113:0/279698677 learned_addr learned my addr 172.21.15.113:0/279698677 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:05:35.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f23a4c00640 1 --2- 172.21.15.113:0/279698677 >> v2:172.21.15.136:3300/0 conn(0x7f23a015b2f0 0x7f23a00c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:35.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431695910 0 0) 0x7f23a00c7d80 con 0x7f23a0154fe0 2024-03-04T08:05:35.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2384003530 con 0x7f23a0154fe0 2024-03-04T08:05:35.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075487682 0 0) 0x7f23a00c8060 con 0x7f23a0154c00 2024-03-04T08:05:35.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a00c7d80 con 0x7f23a0154c00 2024-03-04T08:05:35.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4180723569 0 0) 0x7f2384003530 con 0x7f23a0154fe0 2024-03-04T08:05:35.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 >> v2:172.21.15.136:3300/0 conn(0x7f23a015b2f0 msgr2=0x7f23a00c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:35.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 --2- 172.21.15.113:0/279698677 >> v2:172.21.15.136:3300/0 conn(0x7f23a015b2f0 0x7f23a00c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 >> v1:172.21.15.136:6789/0 conn(0x7f23a0154c00 legacy=0x7f23a00c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00c6dc0 con 0x7f23a0154fe0 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f2388001680 con 0x7f23a0154fe0 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 >> v1:172.21.15.113:6789/0 conn(0x7f23a0154fe0 legacy=0x7f23a015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2384003b50 0x7f2384003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2384004810 0x7f2384004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23840051d0 0x7f238400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a015efb0 con 0x7f2384004810 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238400c100 con 0x7f2384003b50 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238400c300 con 0x7f23840051d0 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f2397e00640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2384004810 0x7f2384004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:35.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f23a4c00640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23840051d0 0x7f238400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:35.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.415+0000 7f239fe00640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2384003b50 0x7f2384003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:35.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f2397e00640 1 -- 172.21.15.113:0/279698677 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23840051d0 msgr2=0x7f238400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:35.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f2397e00640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23840051d0 0x7f238400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f2397e00640 1 -- 172.21.15.113:0/279698677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2384003b50 msgr2=0x7f2384003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:35.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f2397e00640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2384003b50 0x7f2384003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f2397e00640 1 -- 172.21.15.113:0/279698677 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2384003530 con 0x7f2384004810 2024-03-04T08:05:35.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f23a4c00640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23840051d0 0x7f238400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:05:35.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f23a4c00640 1 -- 172.21.15.113:0/279698677 reap_dead start 2024-03-04T08:05:35.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f2397e00640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2384004810 0x7f2384004c90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f238800c1c0 tx=0x7f238801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6b48f9dd5cce22 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:35.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388007cd0 con 0x7f2384004810 2024-03-04T08:05:35.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23880a9020 con 0x7f2384004810 2024-03-04T08:05:35.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f239f400640 1 -- 172.21.15.113:0/279698677 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388005b00 con 0x7f2384004810 2024-03-04T08:05:35.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f23a6c00640 1 -- 172.21.15.113:0/279698677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2384004810 msgr2=0x7f2384004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:35.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f23a6c00640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2384004810 0x7f2384004c90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f238800c1c0 tx=0x7f238801cbc0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f23a6c00640 1 -- 172.21.15.113:0/279698677 shutdown_connections 2024-03-04T08:05:35.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f23a6c00640 1 --2- 172.21.15.113:0/279698677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2384004810 0x7f2384004c90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f23a6c00640 1 -- 172.21.15.113:0/279698677 >> 172.21.15.113:0/279698677 conn(0x7f23a006e820 msgr2=0x7f23a006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:35.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f23a6c00640 1 -- 172.21.15.113:0/279698677 shutdown_connections 2024-03-04T08:05:35.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.419+0000 7f23a6c00640 1 -- 172.21.15.113:0/279698677 wait complete. 2024-03-04T08:05:35.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a6c00640 1 Processor -- start 2024-03-04T08:05:35.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a6c00640 1 -- start start 2024-03-04T08:05:35.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a6c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2384004810 0x7f23a01506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:35.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a6c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a0157fb0 0x7f23a0151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:35.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a6c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0151960 0x7f23a014ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:35.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a6c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00c1310 con 0x7f23a0157fb0 2024-03-04T08:05:35.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a6c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00c1190 con 0x7f23a0151960 2024-03-04T08:05:35.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a6c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a00c1490 con 0x7f2384004810 2024-03-04T08:05:35.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f2397e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a0157fb0 0x7f23a0151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:35.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f2397e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a0157fb0 0x7f23a0151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57134/0 (socket says 172.21.15.113:57134) 2024-03-04T08:05:35.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f2397e00640 1 -- 172.21.15.113:0/325690876 learned_addr learned my addr 172.21.15.113:0/325690876 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:35.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f239fe00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2384004810 0x7f23a01506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:35.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a4c00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0151960 0x7f23a014ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:35.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f2397e00640 1 -- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2384004810 msgr2=0x7f23a01506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:35.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f2397e00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2384004810 0x7f23a01506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f2397e00640 1 -- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0151960 msgr2=0x7f23a014ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:35.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f2397e00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0151960 0x7f23a014ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f2397e00640 1 -- 172.21.15.113:0/325690876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00b21b0 con 0x7f23a0157fb0 2024-03-04T08:05:35.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f239fe00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2384004810 0x7f23a01506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:05:35.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f2397e00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a0157fb0 0x7f23a0151420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f23880ab040 tx=0x7f2384005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:35.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f239cc00640 1 -- 172.21.15.113:0/325690876 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388007cd0 con 0x7f23a0157fb0 2024-03-04T08:05:35.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f239cc00640 1 -- 172.21.15.113:0/325690876 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23880b2070 con 0x7f23a0157fb0 2024-03-04T08:05:35.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f239cc00640 1 -- 172.21.15.113:0/325690876 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388001680 con 0x7f23a0157fb0 2024-03-04T08:05:35.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a6c00640 1 -- 172.21.15.113:0/325690876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a01f44e0 con 0x7f23a0157fb0 2024-03-04T08:05:35.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.423+0000 7f23a6c00640 1 -- 172.21.15.113:0/325690876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a01f3720 con 0x7f23a0157fb0 2024-03-04T08:05:35.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.427+0000 7f23a6c00640 1 -- 172.21.15.113:0/325690876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a0075c00 con 0x7f23a0157fb0 2024-03-04T08:05:35.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.427+0000 7f239cc00640 1 -- 172.21.15.113:0/325690876 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2388007310 con 0x7f23a0157fb0 2024-03-04T08:05:35.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.427+0000 7f239cc00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f237c066f60 0x7f237c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:35.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.427+0000 7f239cc00640 1 -- 172.21.15.113:0/325690876 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f238811ebc0 con 0x7f23a0157fb0 2024-03-04T08:05:35.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.427+0000 7f239fe00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f237c066f60 0x7f237c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:35.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.427+0000 7f239fe00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f237c066f60 0x7f237c069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2390000c10 tx=0x7f2390009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:35.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.431+0000 7f239cc00640 1 -- 172.21.15.113:0/325690876 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f23880168d0 con 0x7f23a0157fb0 2024-03-04T08:05:35.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.619+0000 7f23a6c00640 1 -- 172.21.15.113:0/325690876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23a00cbe80 con 0x7f23a0157fb0 2024-03-04T08:05:35.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f239cc00640 1 -- 172.21.15.113:0/325690876 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f23880ef500 con 0x7f23a0157fb0 2024-03-04T08:05:35.619 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":119,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:51.232251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119,6807]},"id":1}]} 2024-03-04T08:05:35.619 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 119 2024-03-04T08:05:35.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 -- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f237c066f60 msgr2=0x7f237c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:35.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f237c066f60 0x7f237c069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2390000c10 tx=0x7f2390009040 comp rx=0 tx=0).stop 2024-03-04T08:05:35.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 -- 172.21.15.113:0/325690876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a0157fb0 msgr2=0x7f23a0151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:35.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a0157fb0 0x7f23a0151420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f23880ab040 tx=0x7f2384005840 comp rx=0 tx=0).stop 2024-03-04T08:05:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 -- 172.21.15.113:0/325690876 shutdown_connections 2024-03-04T08:05:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a0151960 0x7f23a014ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f237c066f60 0x7f237c069420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a0157fb0 0x7f23a0151420 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 --2- 172.21.15.113:0/325690876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2384004810 0x7f23a01506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 -- 172.21.15.113:0/325690876 >> 172.21.15.113:0/325690876 conn(0x7f23a006e820 msgr2=0x7f23a01588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 -- 172.21.15.113:0/325690876 shutdown_connections 2024-03-04T08:05:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:35.623+0000 7f23a6c00640 1 -- 172.21.15.113:0/325690876 wait complete. 2024-03-04T08:05:35.640 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:05:37.641 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:37.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.779+0000 7fe58fe00640 1 Processor -- start 2024-03-04T08:05:37.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.779+0000 7fe58fe00640 1 -- start start 2024-03-04T08:05:37.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.779+0000 7fe58fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe59006c580 0x7fe5900c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:37.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.779+0000 7fe58fe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59006bb10 con 0x7fe59015bf20 2024-03-04T08:05:37.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.779+0000 7fe58fe00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5900c7a40 con 0x7fe590152920 2024-03-04T08:05:37.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.779+0000 7fe58fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900c7d20 con 0x7fe59006c580 2024-03-04T08:05:37.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58f400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe59015bf20 0x7fe59015c3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47600/0 (socket says 172.21.15.113:47600) 2024-03-04T08:05:37.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58f400640 1 -- 172.21.15.113:0/3705373535 learned_addr learned my addr 172.21.15.113:0/3705373535 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:05:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe587e00640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe59006c580 0x7fe5900c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648445288 0 0) 0x7fe59006bb10 con 0x7fe59015bf20 2024-03-04T08:05:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe574003530 con 0x7fe59015bf20 2024-03-04T08:05:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459363884 0 0) 0x7fe5900c7a40 con 0x7fe590152920 2024-03-04T08:05:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe59006bb10 con 0x7fe590152920 2024-03-04T08:05:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3330926319 0 0) 0x7fe574003530 con 0x7fe59015bf20 2024-03-04T08:05:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe59006c580 msgr2=0x7fe5900c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:37.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe59006c580 0x7fe5900c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 >> v1:172.21.15.136:6789/0 conn(0x7fe590152920 legacy=0x7fe590071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59006bd80 con 0x7fe59015bf20 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fe588001a00 con 0x7fe59015bf20 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 >> v1:172.21.15.113:6789/0 conn(0x7fe59015bf20 legacy=0x7fe59015c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe574003b80 0x7fe574003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574004810 0x7fe574004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5740051d0 0x7fe57400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900c7d20 con 0x7fe574003b80 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57400c070 con 0x7fe574004810 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57400c270 con 0x7fe5740051d0 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe587e00640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe59006c580 0x7fe5900c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:05:37.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58ea00640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe574003b80 0x7fe574003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:37.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58f400640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5740051d0 0x7fe57400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:37.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe587e00640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574004810 0x7fe574004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:37.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58ea00640 1 -- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5740051d0 msgr2=0x7fe57400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:37.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58ea00640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5740051d0 0x7fe57400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:37.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58ea00640 1 -- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574004810 msgr2=0x7fe574004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:37.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58ea00640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574004810 0x7fe574004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:37.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.783+0000 7fe58ea00640 1 -- 172.21.15.113:0/3705373535 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574003530 con 0x7fe574003b80 2024-03-04T08:05:37.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58ea00640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe574003b80 0x7fe574003f80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe5800018d0 tx=0x7fe58001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e246cb3ba289d23 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580001520 con 0x7fe574003b80 2024-03-04T08:05:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5800052f0 con 0x7fe574003b80 2024-03-04T08:05:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58e000640 1 -- 172.21.15.113:0/3705373535 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe580003840 con 0x7fe574003b80 2024-03-04T08:05:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58fe00640 1 -- 172.21.15.113:0/3705373535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe574003b80 msgr2=0x7fe574003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58fe00640 1 --2- 172.21.15.113:0/3705373535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe574003b80 0x7fe574003f80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe5800018d0 tx=0x7fe58001c520 comp rx=0 tx=0).stop 2024-03-04T08:05:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58f400640 1 -- 172.21.15.113:0/3705373535 reap_dead start 2024-03-04T08:05:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58fe00640 1 -- 172.21.15.113:0/3705373535 shutdown_connections 2024-03-04T08:05:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58fe00640 1 -- 172.21.15.113:0/3705373535 >> 172.21.15.113:0/3705373535 conn(0x7fe5900c9a30 msgr2=0x7fe5900c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58fe00640 1 -- 172.21.15.113:0/3705373535 shutdown_connections 2024-03-04T08:05:37.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.787+0000 7fe58fe00640 1 -- 172.21.15.113:0/3705373535 wait complete. 2024-03-04T08:05:37.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58fe00640 1 Processor -- start 2024-03-04T08:05:37.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58fe00640 1 -- start start 2024-03-04T08:05:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574003b80 0x7fe5900cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5901f6dc0 0x7fe5900c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5901f7220 0x7fe59014f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900c45b0 con 0x7fe5901f6dc0 2024-03-04T08:05:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5900c4430 con 0x7fe574003b80 2024-03-04T08:05:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59015cae0 con 0x7fe5901f7220 2024-03-04T08:05:37.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe587e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5901f6dc0 0x7fe5900c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe587e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5901f6dc0 0x7fe5900c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38192/0 (socket says 172.21.15.113:38192) 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe587e00640 1 -- 172.21.15.113:0/3220813727 learned_addr learned my addr 172.21.15.113:0/3220813727 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58f400640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5901f7220 0x7fe59014f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58ea00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574003b80 0x7fe5900cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe587e00640 1 -- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5901f7220 msgr2=0x7fe59014f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe587e00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5901f7220 0x7fe59014f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe587e00640 1 -- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574003b80 msgr2=0x7fe5900cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe587e00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574003b80 0x7fe5900cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe587e00640 1 -- 172.21.15.113:0/3220813727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5900c2030 con 0x7fe5901f6dc0 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe587e00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5901f6dc0 0x7fe5900c61a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe578011860 tx=0x7fe578011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe586a00640 1 -- 172.21.15.113:0/3220813727 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57800fb70 con 0x7fe5901f6dc0 2024-03-04T08:05:37.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe586a00640 1 -- 172.21.15.113:0/3220813727 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57800fd10 con 0x7fe5901f6dc0 2024-03-04T08:05:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58fe00640 1 -- 172.21.15.113:0/3220813727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5900c51e0 con 0x7fe5901f6dc0 2024-03-04T08:05:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe586a00640 1 -- 172.21.15.113:0/3220813727 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578015660 con 0x7fe5901f6dc0 2024-03-04T08:05:37.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.791+0000 7fe58fe00640 1 -- 172.21.15.113:0/3220813727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5900c5780 con 0x7fe5901f6dc0 2024-03-04T08:05:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.795+0000 7fe58fe00640 1 -- 172.21.15.113:0/3220813727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe590075a80 con 0x7fe5901f6dc0 2024-03-04T08:05:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.795+0000 7fe586a00640 1 -- 172.21.15.113:0/3220813727 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe578015800 con 0x7fe5901f6dc0 2024-03-04T08:05:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.795+0000 7fe586a00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe56c066f60 0x7fe56c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.795+0000 7fe586a00640 1 -- 172.21.15.113:0/3220813727 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7fe5780860f0 con 0x7fe5901f6dc0 2024-03-04T08:05:37.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.795+0000 7fe58ea00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe56c066f60 0x7fe56c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:37.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.795+0000 7fe58ea00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe56c066f60 0x7fe56c069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe5800ac000 tx=0x7fe5800a6070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:37.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:37.799+0000 7fe586a00640 1 -- 172.21.15.113:0/3220813727 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe578019030 con 0x7fe5901f6dc0 2024-03-04T08:05:38.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.079+0000 7fe58fe00640 1 -- 172.21.15.113:0/3220813727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5900c8300 con 0x7fe5901f6dc0 2024-03-04T08:05:38.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.079+0000 7fe586a00640 1 -- 172.21.15.113:0/3220813727 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+6624 (secure 0 0 0) 0x7fe578056350 con 0x7fe5901f6dc0 2024-03-04T08:05:38.076 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:38.076 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":119,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:51.232251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119,6807]},"id":1}]} 2024-03-04T08:05:38.077 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 119 2024-03-04T08:05:38.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 -- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe56c066f60 msgr2=0x7fe56c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:38.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe56c066f60 0x7fe56c069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe5800ac000 tx=0x7fe5800a6070 comp rx=0 tx=0).stop 2024-03-04T08:05:38.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 -- 172.21.15.113:0/3220813727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5901f6dc0 msgr2=0x7fe5900c61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:38.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5901f6dc0 0x7fe5900c61a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe578011860 tx=0x7fe578011d20 comp rx=0 tx=0).stop 2024-03-04T08:05:38.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 -- 172.21.15.113:0/3220813727 shutdown_connections 2024-03-04T08:05:38.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5901f7220 0x7fe59014f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:38.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe56c066f60 0x7fe56c069420 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:38.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5901f6dc0 0x7fe5900c61a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:38.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 --2- 172.21.15.113:0/3220813727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574003b80 0x7fe5900cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:38.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 -- 172.21.15.113:0/3220813727 >> 172.21.15.113:0/3220813727 conn(0x7fe5900c9a30 msgr2=0x7fe5900721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:38.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 -- 172.21.15.113:0/3220813727 shutdown_connections 2024-03-04T08:05:38.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:38.083+0000 7fe58fe00640 1 -- 172.21.15.113:0/3220813727 wait complete. 2024-03-04T08:05:38.105 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:05:40.106 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:40.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eeb200640 1 Processor -- start 2024-03-04T08:05:40.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eeb200640 1 -- start start 2024-03-04T08:05:40.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eeb200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec0c8a90 0x7f3eec15e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:40.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eeb200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3eec15ea60 0x7f3eec0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:40.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eeb200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec070fd0 con 0x7f3eec157df0 2024-03-04T08:05:40.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eeb200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eec15e090 con 0x7f3eec15ea60 2024-03-04T08:05:40.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eeb200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec06dfd0 con 0x7f3eec0c8a90 2024-03-04T08:05:40.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3ee9400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec0c8a90 0x7f3eec15e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:40.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eea800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3eec15ea60 0x7f3eec0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:40.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eea800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3eec15ea60 0x7f3eec0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:35452/0 (socket says 172.21.15.113:35452) 2024-03-04T08:05:40.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3ee9400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec0c8a90 0x7f3eec15e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:32960/0 (socket says 172.21.15.113:32960) 2024-03-04T08:05:40.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eea800640 1 -- 172.21.15.113:0/2672280343 learned_addr learned my addr 172.21.15.113:0/2672280343 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:40.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eea800640 1 -- 172.21.15.113:0/2672280343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec0c8a90 msgr2=0x7f3eec15e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:40.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eea800640 1 --2- 172.21.15.113:0/2672280343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec0c8a90 0x7f3eec15e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:40.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eea800640 1 -- 172.21.15.113:0/2672280343 >> v1:172.21.15.136:6789/0 conn(0x7f3eec157df0 legacy=0x7f3eec06c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:40.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.259+0000 7f3eea800640 1 -- 172.21.15.113:0/2672280343 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec06fcd0 con 0x7f3eec15ea60 2024-03-04T08:05:40.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3eea800640 1 --2- 172.21.15.113:0/2672280343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eec15ea60 0x7f3eec0ce020 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009e50 tx=0x7f3ee401f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e9637f0d63260d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:40.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3ee8a00640 1 -- 172.21.15.113:0/2672280343 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b4070 con 0x7f3eec15ea60 2024-03-04T08:05:40.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3ee8a00640 1 -- 172.21.15.113:0/2672280343 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee40af400 con 0x7f3eec15ea60 2024-03-04T08:05:40.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3ee8a00640 1 -- 172.21.15.113:0/2672280343 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40af7d0 con 0x7f3eec15ea60 2024-03-04T08:05:40.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3eeb200640 1 -- 172.21.15.113:0/2672280343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eec15ea60 msgr2=0x7f3eec0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:40.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3eeb200640 1 --2- 172.21.15.113:0/2672280343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eec15ea60 0x7f3eec0ce020 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009e50 tx=0x7f3ee401f210 comp rx=0 tx=0).stop 2024-03-04T08:05:40.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3eeb200640 1 -- 172.21.15.113:0/2672280343 shutdown_connections 2024-03-04T08:05:40.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3eeb200640 1 --2- 172.21.15.113:0/2672280343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eec15ea60 0x7f3eec0ce020 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:40.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3eeb200640 1 --2- 172.21.15.113:0/2672280343 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec0c8a90 0x7f3eec15e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:40.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3eeb200640 1 -- 172.21.15.113:0/2672280343 >> 172.21.15.113:0/2672280343 conn(0x7f3eec072f40 msgr2=0x7f3eec0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:40.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3eeb200640 1 -- 172.21.15.113:0/2672280343 shutdown_connections 2024-03-04T08:05:40.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.263+0000 7f3eeb200640 1 -- 172.21.15.113:0/2672280343 wait complete. 2024-03-04T08:05:40.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3eeb200640 1 Processor -- start 2024-03-04T08:05:40.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3eeb200640 1 -- start start 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3eeb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eec0c8a90 0x7f3eec1f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3eeb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3eec157df0 0x7f3eec1542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3eeb200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec15ea60 0x7f3eec14db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3eeb200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0c30b0 con 0x7f3eec157df0 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3eeb200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec15dd20 con 0x7f3eec0c8a90 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3eeb200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec06dc00 con 0x7f3eec15ea60 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3ee9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3eec157df0 0x7f3eec1542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3ee9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3eec157df0 0x7f3eec1542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38196/0 (socket says 172.21.15.113:38196) 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3ee9400640 1 -- 172.21.15.113:0/646721025 learned_addr learned my addr 172.21.15.113:0/646721025 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3ee9e00640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eec0c8a90 0x7f3eec1f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.267+0000 7f3eea800640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec15ea60 0x7f3eec14db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee9400640 1 -- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec15ea60 msgr2=0x7f3eec14db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee9400640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec15ea60 0x7f3eec14db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee9400640 1 -- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eec0c8a90 msgr2=0x7f3eec1f11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee9400640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eec0c8a90 0x7f3eec1f11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee9400640 1 -- 172.21.15.113:0/646721025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec0cdbd0 con 0x7f3eec157df0 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee9400640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3eec157df0 0x7f3eec1542f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800e930 tx=0x7f3ed800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee2000640 1 -- 172.21.15.113:0/646721025 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8017070 con 0x7f3eec157df0 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee2000640 1 -- 172.21.15.113:0/646721025 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed800ca00 con 0x7f3eec157df0 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3eeb200640 1 -- 172.21.15.113:0/646721025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec0cdeb0 con 0x7f3eec157df0 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee2000640 1 -- 172.21.15.113:0/646721025 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed800cdb0 con 0x7f3eec157df0 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3eeb200640 1 -- 172.21.15.113:0/646721025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec1e3e30 con 0x7f3eec157df0 2024-03-04T08:05:40.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3eeb200640 1 -- 172.21.15.113:0/646721025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec075a80 con 0x7f3eec157df0 2024-03-04T08:05:40.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee2000640 1 -- 172.21.15.113:0/646721025 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ed801a070 con 0x7f3eec157df0 2024-03-04T08:05:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee2000640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ecc066f60 0x7f3ecc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee2000640 1 -- 172.21.15.113:0/646721025 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7f3ed8085ae0 con 0x7f3eec157df0 2024-03-04T08:05:40.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.271+0000 7f3ee9e00640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ecc066f60 0x7f3ecc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.275+0000 7f3ee9e00640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ecc066f60 0x7f3ecc069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3edc001810 tx=0x7f3edc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:40.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.283+0000 7f3ee2000640 1 -- 172.21.15.113:0/646721025 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3ed8055e00 con 0x7f3eec157df0 2024-03-04T08:05:40.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 -- 172.21.15.113:0/646721025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eec0c9610 con 0x7f3eec157df0 2024-03-04T08:05:40.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3ee2000640 1 -- 172.21.15.113:0/646721025 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f3ed8059ea0 con 0x7f3eec157df0 2024-03-04T08:05:40.479 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:40.479 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":119,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:51.232251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119,6807]},"id":1}]} 2024-03-04T08:05:40.479 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 119 2024-03-04T08:05:40.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 -- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ecc066f60 msgr2=0x7f3ecc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:40.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ecc066f60 0x7f3ecc069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3edc001810 tx=0x7f3edc009040 comp rx=0 tx=0).stop 2024-03-04T08:05:40.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 -- 172.21.15.113:0/646721025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3eec157df0 msgr2=0x7f3eec1542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:40.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3eec157df0 0x7f3eec1542f0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800e930 tx=0x7f3ed800edf0 comp rx=0 tx=0).stop 2024-03-04T08:05:40.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 -- 172.21.15.113:0/646721025 shutdown_connections 2024-03-04T08:05:40.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3eec15ea60 0x7f3eec14db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:40.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3ecc066f60 0x7f3ecc069420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:40.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3eec157df0 0x7f3eec1542f0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:40.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 --2- 172.21.15.113:0/646721025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3eec0c8a90 0x7f3eec1f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:40.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 -- 172.21.15.113:0/646721025 >> 172.21.15.113:0/646721025 conn(0x7f3eec072f40 msgr2=0x7f3eec0c32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:40.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 -- 172.21.15.113:0/646721025 shutdown_connections 2024-03-04T08:05:40.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:40.483+0000 7f3eeb200640 1 -- 172.21.15.113:0/646721025 wait complete. 2024-03-04T08:05:40.509 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:05:42.510 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:42.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.667+0000 7fa3abc00640 1 Processor -- start 2024-03-04T08:05:42.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.667+0000 7fa3abc00640 1 -- start start 2024-03-04T08:05:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3abc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3a40cd270 0x7fa3a40c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3abc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a4157830 con 0x7fa3a40c7a80 2024-03-04T08:05:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3abc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a4157af0 con 0x7fa3a40cd650 2024-03-04T08:05:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3abc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a4157dd0 con 0x7fa3a40cd270 2024-03-04T08:05:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3a9200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3a40cd270 0x7fa3a40c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3a9c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa3a40c7a80 0x7fa3a415b010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47610/0 (socket says 172.21.15.113:47610) 2024-03-04T08:05:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3a9c00640 1 -- 172.21.15.113:0/149588745 learned_addr learned my addr 172.21.15.113:0/149588745 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:05:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3a9200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3a40cd270 0x7fa3a40c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38200/0 (socket says 172.21.15.113:38200) 2024-03-04T08:05:42.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3a3400640 1 -- 172.21.15.113:0/149588745 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849442935 0 0) 0x7fa3a4157830 con 0x7fa3a40c7a80 2024-03-04T08:05:42.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3a9200640 1 -- 172.21.15.113:0/149588745 >> v1:172.21.15.136:6789/0 conn(0x7fa3a40cd650 legacy=0x7fa3a4151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:42.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3a9200640 1 -- 172.21.15.113:0/149588745 >> v1:172.21.15.113:6789/0 conn(0x7fa3a40c7a80 legacy=0x7fa3a415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:42.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.671+0000 7fa3a9200640 1 -- 172.21.15.113:0/149588745 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a41580b0 con 0x7fa3a40cd270 2024-03-04T08:05:42.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3a9200640 1 --2- 172.21.15.113:0/149588745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 0x7fa3a40c8df0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa3980097c0 tx=0x7fa39801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc38728b4b081fd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:42.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3a3400640 1 -- 172.21.15.113:0/149588745 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3980b1070 con 0x7fa3a40cd270 2024-03-04T08:05:42.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3a3400640 1 -- 172.21.15.113:0/149588745 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3980ac410 con 0x7fa3a40cd270 2024-03-04T08:05:42.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3a3400640 1 -- 172.21.15.113:0/149588745 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3980ac7e0 con 0x7fa3a40cd270 2024-03-04T08:05:42.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3abc00640 1 -- 172.21.15.113:0/149588745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 msgr2=0x7fa3a40c8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:42.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3abc00640 1 --2- 172.21.15.113:0/149588745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 0x7fa3a40c8df0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fa3980097c0 tx=0x7fa39801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:05:42.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3abc00640 1 -- 172.21.15.113:0/149588745 shutdown_connections 2024-03-04T08:05:42.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3abc00640 1 --2- 172.21.15.113:0/149588745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 0x7fa3a40c8df0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:42.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3abc00640 1 -- 172.21.15.113:0/149588745 >> 172.21.15.113:0/149588745 conn(0x7fa3a40671b0 msgr2=0x7fa3a406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:42.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3abc00640 1 -- 172.21.15.113:0/149588745 shutdown_connections 2024-03-04T08:05:42.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3abc00640 1 -- 172.21.15.113:0/149588745 wait complete. 2024-03-04T08:05:42.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3abc00640 1 Processor -- start 2024-03-04T08:05:42.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.675+0000 7fa3abc00640 1 -- start start 2024-03-04T08:05:42.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3abc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3a40c7a80 0x7fa3a413c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:42.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3abc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 0x7fa3a41f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:42.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3abc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3a40cd650 0x7fa3a4136850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:42.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3abc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a4165a70 con 0x7fa3a40cd270 2024-03-04T08:05:42.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3abc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a4151bb0 con 0x7fa3a40c7a80 2024-03-04T08:05:42.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3abc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a4165bf0 con 0x7fa3a40cd650 2024-03-04T08:05:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 0x7fa3a41f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 0x7fa3a41f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38210/0 (socket says 172.21.15.113:38210) 2024-03-04T08:05:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a3e00640 1 -- 172.21.15.113:0/2805520849 learned_addr learned my addr 172.21.15.113:0/2805520849 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a9c00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3a40cd650 0x7fa3a4136850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a9200640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3a40c7a80 0x7fa3a413c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a3e00640 1 -- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3a40cd650 msgr2=0x7fa3a4136850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a3e00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3a40cd650 0x7fa3a4136850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a3e00640 1 -- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3a40c7a80 msgr2=0x7fa3a413c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a3e00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3a40c7a80 0x7fa3a413c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:42.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a3e00640 1 -- 172.21.15.113:0/2805520849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4138f90 con 0x7fa3a40cd270 2024-03-04T08:05:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a3e00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 0x7fa3a41f6ec0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa39000c3f0 tx=0x7fa39000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a0c00640 1 -- 172.21.15.113:0/2805520849 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390017070 con 0x7fa3a40cd270 2024-03-04T08:05:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a0c00640 1 -- 172.21.15.113:0/2805520849 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3900052d0 con 0x7fa3a40cd270 2024-03-04T08:05:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3a0c00640 1 -- 172.21.15.113:0/2805520849 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390003840 con 0x7fa3a40cd270 2024-03-04T08:05:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3abc00640 1 -- 172.21.15.113:0/2805520849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a4138210 con 0x7fa3a40cd270 2024-03-04T08:05:42.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3abc00640 1 -- 172.21.15.113:0/2805520849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a41384b0 con 0x7fa3a40cd270 2024-03-04T08:05:42.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.679+0000 7fa3abc00640 1 -- 172.21.15.113:0/2805520849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a4075c00 con 0x7fa3a40cd270 2024-03-04T08:05:42.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.683+0000 7fa3a0c00640 1 -- 172.21.15.113:0/2805520849 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa39001a070 con 0x7fa3a40cd270 2024-03-04T08:05:42.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.683+0000 7fa3a0c00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa384066f60 0x7fa384069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:42.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.683+0000 7fa3a0c00640 1 -- 172.21.15.113:0/2805520849 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7fa390084a80 con 0x7fa3a40cd270 2024-03-04T08:05:42.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.683+0000 7fa3a9200640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa384066f60 0x7fa384069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:42.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.683+0000 7fa3a9200640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa384066f60 0x7fa384069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa3980ae000 tx=0x7fa398008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:42.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.687+0000 7fa3a0c00640 1 -- 172.21.15.113:0/2805520849 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa390086e20 con 0x7fa3a40cd270 2024-03-04T08:05:42.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.899+0000 7fa3abc00640 1 -- 172.21.15.113:0/2805520849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3a415b1b0 con 0x7fa3a40cd270 2024-03-04T08:05:42.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.899+0000 7fa3a0c00640 1 -- 172.21.15.113:0/2805520849 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+6624 (secure 0 0 0) 0x7fa390005480 con 0x7fa3a40cd270 2024-03-04T08:05:42.897 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:42.897 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":119,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:51.232251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119,6807]},"id":1}]} 2024-03-04T08:05:42.897 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 119 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 -- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa384066f60 msgr2=0x7fa384069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa384066f60 0x7fa384069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa3980ae000 tx=0x7fa398008040 comp rx=0 tx=0).stop 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 -- 172.21.15.113:0/2805520849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 msgr2=0x7fa3a41f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 0x7fa3a41f6ec0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa39000c3f0 tx=0x7fa39000c8b0 comp rx=0 tx=0).stop 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 -- 172.21.15.113:0/2805520849 shutdown_connections 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa3a40cd650 0x7fa3a4136850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa384066f60 0x7fa384069420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3a40cd270 0x7fa3a41f6ec0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 --2- 172.21.15.113:0/2805520849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3a40c7a80 0x7fa3a413c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 -- 172.21.15.113:0/2805520849 >> 172.21.15.113:0/2805520849 conn(0x7fa3a40671b0 msgr2=0x7fa3a4072430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 -- 172.21.15.113:0/2805520849 shutdown_connections 2024-03-04T08:05:42.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:42.903+0000 7fa3abc00640 1 -- 172.21.15.113:0/2805520849 wait complete. 2024-03-04T08:05:42.916 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:05:44.917 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:45.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.403+0000 7fd4a9400640 1 Processor -- start 2024-03-04T08:05:45.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.403+0000 7fd4a9400640 1 -- start start 2024-03-04T08:05:45.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a9400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd4a40c22e0 0x7fd4a415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:45.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a9400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c2d80 0x7fd4a415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:45.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a9400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a40c27f0 con 0x7fd4a414e4c0 2024-03-04T08:05:45.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a9400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a415ad60 con 0x7fd4a40c22e0 2024-03-04T08:05:45.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a9400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40c2a90 con 0x7fd4a40c2d80 2024-03-04T08:05:45.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a3200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c2d80 0x7fd4a415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:45.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd499e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd4a40c22e0 0x7fd4a415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:45.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a2800640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fd4a414e4c0 0x7fd4a4072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:37510/0 (socket says 172.21.15.113:37510) 2024-03-04T08:05:45.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a2800640 1 -- 172.21.15.113:0/3794788960 learned_addr learned my addr 172.21.15.113:0/3794788960 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:05:45.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a3200640 1 -- 172.21.15.113:0/3794788960 >> v2:172.21.15.136:3300/0 conn(0x7fd4a40c22e0 msgr2=0x7fd4a415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:45.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a1e00640 1 -- 172.21.15.113:0/3794788960 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504243403 0 0) 0x7fd4a40c27f0 con 0x7fd4a414e4c0 2024-03-04T08:05:45.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a3200640 1 --2- 172.21.15.113:0/3794788960 >> v2:172.21.15.136:3300/0 conn(0x7fd4a40c22e0 0x7fd4a415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:45.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a3200640 1 -- 172.21.15.113:0/3794788960 >> v1:172.21.15.136:6789/0 conn(0x7fd4a414e4c0 legacy=0x7fd4a4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:45.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.407+0000 7fd4a3200640 1 -- 172.21.15.113:0/3794788960 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a4070c60 con 0x7fd4a40c2d80 2024-03-04T08:05:45.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a3200640 1 --2- 172.21.15.113:0/3794788960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c2d80 0x7fd4a415b730 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0097c0 tx=0x7fd49c01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df7c43c730b15f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:45.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a1e00640 1 -- 172.21.15.113:0/3794788960 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0b5070 con 0x7fd4a40c2d80 2024-03-04T08:05:45.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a1e00640 1 -- 172.21.15.113:0/3794788960 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c0a8990 con 0x7fd4a40c2d80 2024-03-04T08:05:45.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a1e00640 1 -- 172.21.15.113:0/3794788960 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c0a8d60 con 0x7fd4a40c2d80 2024-03-04T08:05:45.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a9400640 1 -- 172.21.15.113:0/3794788960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c2d80 msgr2=0x7fd4a415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:45.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a9400640 1 --2- 172.21.15.113:0/3794788960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c2d80 0x7fd4a415b730 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0097c0 tx=0x7fd49c01ec90 comp rx=0 tx=0).stop 2024-03-04T08:05:45.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a9400640 1 -- 172.21.15.113:0/3794788960 shutdown_connections 2024-03-04T08:05:45.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a9400640 1 --2- 172.21.15.113:0/3794788960 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c2d80 0x7fd4a415b730 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:45.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a9400640 1 --2- 172.21.15.113:0/3794788960 >> v2:172.21.15.136:3300/0 conn(0x7fd4a40c22e0 0x7fd4a415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:45.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a9400640 1 -- 172.21.15.113:0/3794788960 >> 172.21.15.113:0/3794788960 conn(0x7fd4a406e820 msgr2=0x7fd4a406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:45.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a9400640 1 -- 172.21.15.113:0/3794788960 shutdown_connections 2024-03-04T08:05:45.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.411+0000 7fd4a9400640 1 -- 172.21.15.113:0/3794788960 wait complete. 2024-03-04T08:05:45.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a9400640 1 Processor -- start 2024-03-04T08:05:45.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a9400640 1 -- start start 2024-03-04T08:05:45.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a9400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c22e0 0x7fd4a414a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:45.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a40c2d80 0x7fd4a41ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:45.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a414e4c0 0x7fd4a4143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:45.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a9400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a4071500 con 0x7fd4a40c2d80 2024-03-04T08:05:45.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a9400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a4071220 con 0x7fd4a414e4c0 2024-03-04T08:05:45.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a9400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a414e8a0 con 0x7fd4a40c22e0 2024-03-04T08:05:45.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd499e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a40c2d80 0x7fd4a41ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:45.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a2800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c22e0 0x7fd4a414a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:45.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd499e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a40c2d80 0x7fd4a41ef1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38222/0 (socket says 172.21.15.113:38222) 2024-03-04T08:05:45.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd499e00640 1 -- 172.21.15.113:0/3636269440 learned_addr learned my addr 172.21.15.113:0/3636269440 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:45.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a3200640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a414e4c0 0x7fd4a4143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd499e00640 1 -- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c22e0 msgr2=0x7fd4a414a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd499e00640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c22e0 0x7fd4a414a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd499e00640 1 -- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a414e4c0 msgr2=0x7fd4a4143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd499e00640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a414e4c0 0x7fd4a4143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd499e00640 1 -- 172.21.15.113:0/3636269440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a41ef6e0 con 0x7fd4a40c2d80 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a3200640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a414e4c0 0x7fd4a4143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd499e00640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a40c2d80 0x7fd4a41ef1a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e8a0 tx=0x7fd48c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd49b400640 1 -- 172.21.15.113:0/3636269440 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c017070 con 0x7fd4a40c2d80 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd49b400640 1 -- 172.21.15.113:0/3636269440 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c00c940 con 0x7fd4a40c2d80 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd49b400640 1 -- 172.21.15.113:0/3636269440 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00ccf0 con 0x7fd4a40c2d80 2024-03-04T08:05:45.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.415+0000 7fd4a9400640 1 -- 172.21.15.113:0/3636269440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a415a500 con 0x7fd4a40c2d80 2024-03-04T08:05:45.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.419+0000 7fd4a9400640 1 -- 172.21.15.113:0/3636269440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a4147f50 con 0x7fd4a40c2d80 2024-03-04T08:05:45.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.419+0000 7fd4a9400640 1 -- 172.21.15.113:0/3636269440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a4075c00 con 0x7fd4a40c2d80 2024-03-04T08:05:45.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.419+0000 7fd49b400640 1 -- 172.21.15.113:0/3636269440 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd48c01a070 con 0x7fd4a40c2d80 2024-03-04T08:05:45.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.419+0000 7fd49b400640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd480066f60 0x7fd480069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:45.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.419+0000 7fd49b400640 1 -- 172.21.15.113:0/3636269440 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7287+0+0 (secure 0 0 0) 0x7fd48c0857f0 con 0x7fd4a40c2d80 2024-03-04T08:05:45.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.419+0000 7fd4a2800640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd480066f60 0x7fd480069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:45.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.423+0000 7fd4a2800640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd480066f60 0x7fd480069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd4940012f0 tx=0x7fd494001770 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:45.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.427+0000 7fd49b400640 1 -- 172.21.15.113:0/3636269440 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd48c055b70 con 0x7fd4a40c2d80 2024-03-04T08:05:45.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.619+0000 7fd4a9400640 1 -- 172.21.15.113:0/3636269440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4a4155190 con 0x7fd4a40c2d80 2024-03-04T08:05:45.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.619+0000 7fd49b400640 1 -- 172.21.15.113:0/3636269440 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+6624 (secure 0 0 0) 0x7fd48c059c10 con 0x7fd4a40c2d80 2024-03-04T08:05:45.617 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:45.617 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":119,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7269,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:04:51.232251+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":6807},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6807":{"gid":6807,"name":"a","rank":2,"incarnation":101,"state":"up:active","state_seq":14,"addr":"172.21.15.113:6836/1035982016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1035982016},{"type":"v1","addr":"172.21.15.113:6836","nonce":1035982016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119,6807]},"id":1}]} 2024-03-04T08:05:45.617 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 119 2024-03-04T08:05:45.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 -- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd480066f60 msgr2=0x7fd480069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:45.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd480066f60 0x7fd480069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd4940012f0 tx=0x7fd494001770 comp rx=0 tx=0).stop 2024-03-04T08:05:45.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 -- 172.21.15.113:0/3636269440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a40c2d80 msgr2=0x7fd4a41ef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:45.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a40c2d80 0x7fd4a41ef1a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00e8a0 tx=0x7fd48c00eca0 comp rx=0 tx=0).stop 2024-03-04T08:05:45.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 -- 172.21.15.113:0/3636269440 shutdown_connections 2024-03-04T08:05:45.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4a414e4c0 0x7fd4a4143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:45.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd480066f60 0x7fd480069420 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:45.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd4a40c2d80 0x7fd4a41ef1a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:45.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 --2- 172.21.15.113:0/3636269440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4a40c22e0 0x7fd4a414a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:45.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 -- 172.21.15.113:0/3636269440 >> 172.21.15.113:0/3636269440 conn(0x7fd4a406e820 msgr2=0x7fd4a414ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:45.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 -- 172.21.15.113:0/3636269440 shutdown_connections 2024-03-04T08:05:45.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:45.623+0000 7fd4a9400640 1 -- 172.21.15.113:0/3636269440 wait complete. 2024-03-04T08:05:45.641 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:05:46.226 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 16% (60/355) Updating files: 17% (61/355) Updating files: 18% (64/355) Updating files: 19% (68/355) Updating files: 20% (71/355) Updating files: 21% (75/355) Updating files: 22% (79/355) Updating files: 23% (82/355) Updating files: 24% (86/355) Updating files: 25% (89/355) Updating files: 26% (93/355) Updating files: 27% (96/355) Updating files: 28% (100/355) Updating files: 29% (103/355) Updating files: 30% (107/355) Updating files: 31% (111/355) Updating files: 32% (114/355) Updating files: 33% (118/355) Updating files: 34% (121/355) Updating files: 35% (125/355) Updating files: 36% (128/355) Updating files: 37% (132/355) Updating files: 38% (135/355) Updating files: 39% (139/355) Updating files: 40% (142/355) Updating files: 41% (146/355) Updating files: 42% (150/355) Updating files: 43% (153/355) Updating files: 44% (157/355) Updating files: 45% (160/355) Updating files: 46% (164/355) Updating files: 47% (167/355) Updating files: 48% (171/355) Updating files: 49% (174/355) Updating files: 49% (176/355) Updating files: 50% (178/355) Updating files: 51% (182/355) Updating files: 52% (185/355) Updating files: 53% (189/355) Updating files: 54% (192/355) Updating files: 55% (196/355) Updating files: 56% (199/355) Updating files: 57% (203/355) Updating files: 58% (206/355) Updating files: 59% (210/355) Updating files: 60% (213/355) Updating files: 61% (217/355) Updating files: 62% (221/355) Updating files: 63% (224/355) Updating files: 63% (226/355) Updating files: 64% (228/355) Updating files: 65% (231/355) Updating files: 66% (235/355) Updating files: 67% (238/355) Updating files: 68% (242/355) Updating files: 69% (245/355) Updating files: 70% (249/355) Updating files: 71% (253/355) Updating files: 72% (256/355) Updating files: 73% (260/355) Updating files: 74% (263/355) Updating files: 75% (267/355) Updating files: 76% (270/355) Updating files: 77% (274/355) Updating files: 78% (277/355) Updating files: 79% (281/355) Updating files: 80% (284/355) Updating files: 81% (288/355) Updating files: 82% (292/355) Updating files: 83% (295/355) Updating files: 84% (299/355) Updating files: 85% (302/355) Updating files: 86% (306/355) Updating files: 87% (309/355) Updating files: 88% (313/355) Updating files: 88% (314/355) Updating files: 89% (316/355) Updating files: 90% (320/355) Updating files: 91% (324/355) Updating files: 92% (327/355) Updating files: 93% (331/355) Updating files: 94% (334/355) Updating files: 95% (338/355) Updating files: 96% (341/355) Updating files: 97% (345/355) Updating files: 98% (348/355) Updating files: 99% (352/355) Updating files: 100% (355/355) Updating files: 100% (355/355), done. 2024-03-04T08:05:47.641 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:47.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.779+0000 7f4e90600640 1 Processor -- start 2024-03-04T08:05:47.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.779+0000 7f4e90600640 1 -- start start 2024-03-04T08:05:47.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e90600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e881527f0 0x7f4e88070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:47.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e90600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e88152bd0 0x7f4e881587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:47.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e90600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e881591f0 con 0x7f4e880c8200 2024-03-04T08:05:47.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e90600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e88071df0 con 0x7f4e881527f0 2024-03-04T08:05:47.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e90600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e881597b0 con 0x7f4e88152bd0 2024-03-04T08:05:47.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e8dc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e881527f0 0x7f4e88070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:47.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e8dc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4e881527f0 0x7f4e88070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57102/0 (socket says 172.21.15.113:57102) 2024-03-04T08:05:47.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e8dc00640 1 -- 172.21.15.113:0/3559938014 learned_addr learned my addr 172.21.15.113:0/3559938014 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:47.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e8e600640 1 --2- 172.21.15.113:0/3559938014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e88152bd0 0x7f4e881587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:47.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e8dc00640 1 -- 172.21.15.113:0/3559938014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e88152bd0 msgr2=0x7f4e881587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:47.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e8dc00640 1 --2- 172.21.15.113:0/3559938014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e88152bd0 0x7f4e881587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:47.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e8dc00640 1 -- 172.21.15.113:0/3559938014 >> v1:172.21.15.136:6789/0 conn(0x7f4e880c8200 legacy=0x7f4e88072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:47.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.783+0000 7f4e8dc00640 1 -- 172.21.15.113:0/3559938014 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e881594d0 con 0x7f4e881527f0 2024-03-04T08:05:47.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e8dc00640 1 --2- 172.21.15.113:0/3559938014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e881527f0 0x7f4e88070d40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009870 tx=0x7f4e7c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d6e51233580b72 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:47.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e87e00640 1 -- 172.21.15.113:0/3559938014 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0a7070 con 0x7f4e881527f0 2024-03-04T08:05:47.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e87e00640 1 -- 172.21.15.113:0/3559938014 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0b0070 con 0x7f4e881527f0 2024-03-04T08:05:47.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e87e00640 1 -- 172.21.15.113:0/3559938014 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0ab890 con 0x7f4e881527f0 2024-03-04T08:05:47.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 -- 172.21.15.113:0/3559938014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e881527f0 msgr2=0x7f4e88070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:47.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 --2- 172.21.15.113:0/3559938014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e881527f0 0x7f4e88070d40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009870 tx=0x7f4e7c01c560 comp rx=0 tx=0).stop 2024-03-04T08:05:47.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 -- 172.21.15.113:0/3559938014 shutdown_connections 2024-03-04T08:05:47.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 --2- 172.21.15.113:0/3559938014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e88152bd0 0x7f4e881587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:47.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 --2- 172.21.15.113:0/3559938014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e881527f0 0x7f4e88070d40 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:47.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 -- 172.21.15.113:0/3559938014 >> 172.21.15.113:0/3559938014 conn(0x7f4e8806e820 msgr2=0x7f4e8806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:47.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 -- 172.21.15.113:0/3559938014 shutdown_connections 2024-03-04T08:05:47.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 -- 172.21.15.113:0/3559938014 wait complete. 2024-03-04T08:05:47.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 Processor -- start 2024-03-04T08:05:47.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 -- start start 2024-03-04T08:05:47.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e880c8200 0x7f4e880c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:47.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e881527f0 0x7f4e880c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:47.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e88152bd0 0x7f4e88147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e881654f0 con 0x7f4e880c8200 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e88165370 con 0x7f4e881527f0 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e90600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e88165670 con 0x7f4e88152bd0 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.787+0000 7f4e8dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e880c8200 0x7f4e880c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8dc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e880c8200 0x7f4e880c1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57110/0 (socket says 172.21.15.113:57110) 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8dc00640 1 -- 172.21.15.113:0/4237267746 learned_addr learned my addr 172.21.15.113:0/4237267746 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8e600640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e88152bd0 0x7f4e88147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8d200640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e881527f0 0x7f4e880c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8dc00640 1 -- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e88152bd0 msgr2=0x7f4e88147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8dc00640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e88152bd0 0x7f4e88147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8dc00640 1 -- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e881527f0 msgr2=0x7f4e880c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8dc00640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e881527f0 0x7f4e880c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8dc00640 1 -- 172.21.15.113:0/4237267746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e880c2690 con 0x7f4e880c8200 2024-03-04T08:05:47.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8d200640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e881527f0 0x7f4e880c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:05:47.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e8dc00640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e880c8200 0x7f4e880c1e20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c01a190 tx=0x7f4e7c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:47.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e85600640 1 -- 172.21.15.113:0/4237267746 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0be5f0 con 0x7f4e880c8200 2024-03-04T08:05:47.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e85600640 1 -- 172.21.15.113:0/4237267746 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0be7b0 con 0x7f4e880c8200 2024-03-04T08:05:47.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e90600640 1 -- 172.21.15.113:0/4237267746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e880c2360 con 0x7f4e880c8200 2024-03-04T08:05:47.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e85600640 1 -- 172.21.15.113:0/4237267746 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0beba0 con 0x7f4e880c8200 2024-03-04T08:05:47.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e90600640 1 -- 172.21.15.113:0/4237267746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e880c2c10 con 0x7f4e880c8200 2024-03-04T08:05:47.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e90600640 1 -- 172.21.15.113:0/4237267746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e88075c00 con 0x7f4e880c8200 2024-03-04T08:05:47.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.791+0000 7f4e85600640 1 -- 172.21.15.113:0/4237267746 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e7c0b43f0 con 0x7f4e880c8200 2024-03-04T08:05:47.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.795+0000 7f4e85600640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e68066f90 0x7f4e68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:47.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.795+0000 7f4e85600640 1 -- 172.21.15.113:0/4237267746 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f4e7c11fb90 con 0x7f4e880c8200 2024-03-04T08:05:47.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.795+0000 7f4e8d200640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e68066f90 0x7f4e68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:47.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.795+0000 7f4e8d200640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e68066f90 0x7f4e68069450 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e8806ee70 tx=0x7f4e74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:47.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.799+0000 7f4e85600640 1 -- 172.21.15.113:0/4237267746 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4e7c0c4060 con 0x7f4e880c8200 2024-03-04T08:05:47.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.983+0000 7f4e90600640 1 -- 172.21.15.113:0/4237267746 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e88156f20 con 0x7f4e880c8200 2024-03-04T08:05:47.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.983+0000 7f4e85600640 1 -- 172.21.15.113:0/4237267746 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f4e7c016150 con 0x7f4e880c8200 2024-03-04T08:05:47.982 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:47.982 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":120,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:05:45.841121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119]},"id":1}]} 2024-03-04T08:05:47.982 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 120 2024-03-04T08:05:47.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 -- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e68066f90 msgr2=0x7f4e68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:47.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e68066f90 0x7f4e68069450 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e8806ee70 tx=0x7f4e74009040 comp rx=0 tx=0).stop 2024-03-04T08:05:47.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 -- 172.21.15.113:0/4237267746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e880c8200 msgr2=0x7f4e880c1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:47.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e880c8200 0x7f4e880c1e20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c01a190 tx=0x7f4e7c01ada0 comp rx=0 tx=0).stop 2024-03-04T08:05:47.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 -- 172.21.15.113:0/4237267746 shutdown_connections 2024-03-04T08:05:47.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4e88152bd0 0x7f4e88147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:47.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4e881527f0 0x7f4e880c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:47.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4e68066f90 0x7f4e68069450 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:47.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 --2- 172.21.15.113:0/4237267746 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4e880c8200 0x7f4e880c1e20 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:47.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 -- 172.21.15.113:0/4237267746 >> 172.21.15.113:0/4237267746 conn(0x7f4e8806e820 msgr2=0x7f4e88070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:47.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 -- 172.21.15.113:0/4237267746 shutdown_connections 2024-03-04T08:05:47.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:47.987+0000 7f4e90600640 1 -- 172.21.15.113:0/4237267746 wait complete. 2024-03-04T08:05:48.004 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-03-04T08:05:48.004 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:05:48.004 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:48.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.143+0000 7f7048800640 1 Processor -- start 2024-03-04T08:05:48.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.143+0000 7f7048800640 1 -- start start 2024-03-04T08:05:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7048800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7040152d60 0x7f70400c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7048800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70400c38c0 0x7f7040159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7048800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7040159470 0x7f70400ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7048800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704006f3b0 con 0x7f7040152d60 2024-03-04T08:05:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7048800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70400c3430 con 0x7f7040159470 2024-03-04T08:05:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7048800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70400c5130 con 0x7f70400c38c0 2024-03-04T08:05:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7045e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7040152d60 0x7f70400c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:48.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7045400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70400c38c0 0x7f7040159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7045400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70400c38c0 0x7f7040159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47852/0 (socket says 172.21.15.113:47852) 2024-03-04T08:05:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7045400640 1 -- 172.21.15.113:0/1171959409 learned_addr learned my addr 172.21.15.113:0/1171959409 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7046800640 1 --2- 172.21.15.113:0/1171959409 >> v2:172.21.15.136:3300/0 conn(0x7f7040159470 0x7f70400ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7045400640 1 -- 172.21.15.113:0/1171959409 >> v2:172.21.15.136:3300/0 conn(0x7f7040159470 msgr2=0x7f70400ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:48.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7045400640 1 --2- 172.21.15.113:0/1171959409 >> v2:172.21.15.136:3300/0 conn(0x7f7040159470 0x7f70400ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7045400640 1 -- 172.21.15.113:0/1171959409 >> v2:172.21.15.113:3300/0 conn(0x7f7040152d60 msgr2=0x7f70400c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7045400640 1 --2- 172.21.15.113:0/1171959409 >> v2:172.21.15.113:3300/0 conn(0x7f7040152d60 0x7f70400c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.147+0000 7f7045400640 1 -- 172.21.15.113:0/1171959409 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040159980 con 0x7f70400c38c0 2024-03-04T08:05:48.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7045400640 1 --2- 172.21.15.113:0/1171959409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70400c38c0 0x7f7040159ea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7040072e00 tx=0x7f703001c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de5ff1e0e08582a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:48.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7044a00640 1 -- 172.21.15.113:0/1171959409 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70300ab5f0 con 0x7f70400c38c0 2024-03-04T08:05:48.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7044a00640 1 -- 172.21.15.113:0/1171959409 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70300b0070 con 0x7f70400c38c0 2024-03-04T08:05:48.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7044a00640 1 -- 172.21.15.113:0/1171959409 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70300ab9b0 con 0x7f70400c38c0 2024-03-04T08:05:48.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 -- 172.21.15.113:0/1171959409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70400c38c0 msgr2=0x7f7040159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:48.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 --2- 172.21.15.113:0/1171959409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70400c38c0 0x7f7040159ea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7040072e00 tx=0x7f703001c520 comp rx=0 tx=0).stop 2024-03-04T08:05:48.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 -- 172.21.15.113:0/1171959409 shutdown_connections 2024-03-04T08:05:48.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 --2- 172.21.15.113:0/1171959409 >> v2:172.21.15.136:3300/0 conn(0x7f7040159470 0x7f70400ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 --2- 172.21.15.113:0/1171959409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70400c38c0 0x7f7040159ea0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 --2- 172.21.15.113:0/1171959409 >> v2:172.21.15.113:3300/0 conn(0x7f7040152d60 0x7f70400c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 -- 172.21.15.113:0/1171959409 >> 172.21.15.113:0/1171959409 conn(0x7f7040066db0 msgr2=0x7f70400671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:48.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 -- 172.21.15.113:0/1171959409 shutdown_connections 2024-03-04T08:05:48.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 -- 172.21.15.113:0/1171959409 wait complete. 2024-03-04T08:05:48.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 Processor -- start 2024-03-04T08:05:48.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 -- start start 2024-03-04T08:05:48.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70400c38c0 0x7f704006dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.151+0000 7f7048800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7040152d60 0x7f70400cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7048800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7040159470 0x7f704006c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7048800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704015a3e0 con 0x7f70400c38c0 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7048800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704006f8e0 con 0x7f7040159470 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7048800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704015a560 con 0x7f7040152d60 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7045e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70400c38c0 0x7f704006dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7045e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70400c38c0 0x7f704006dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57122/0 (socket says 172.21.15.113:57122) 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7045e00640 1 -- 172.21.15.113:0/30229130 learned_addr learned my addr 172.21.15.113:0/30229130 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7045400640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7040152d60 0x7f70400cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7046800640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7040159470 0x7f704006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7045e00640 1 -- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7040152d60 msgr2=0x7f70400cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7045e00640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7040152d60 0x7f70400cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7045e00640 1 -- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7040159470 msgr2=0x7f704006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7045e00640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7040159470 0x7f704006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7045e00640 1 -- 172.21.15.113:0/30229130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70401f44f0 con 0x7f70400c38c0 2024-03-04T08:05:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7045e00640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70400c38c0 0x7f704006dbd0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f703400e8e0 tx=0x7f703400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f703a000640 1 -- 172.21.15.113:0/30229130 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034017070 con 0x7f70400c38c0 2024-03-04T08:05:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f703a000640 1 -- 172.21.15.113:0/30229130 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703400ca00 con 0x7f70400c38c0 2024-03-04T08:05:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7048800640 1 -- 172.21.15.113:0/30229130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70401f6bc0 con 0x7f70400c38c0 2024-03-04T08:05:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f703a000640 1 -- 172.21.15.113:0/30229130 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703400cdd0 con 0x7f70400c38c0 2024-03-04T08:05:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7048800640 1 -- 172.21.15.113:0/30229130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70401f3a10 con 0x7f70400c38c0 2024-03-04T08:05:48.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f7048800640 1 -- 172.21.15.113:0/30229130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7040068eb0 con 0x7f70400c38c0 2024-03-04T08:05:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.155+0000 7f703a000640 1 -- 172.21.15.113:0/30229130 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f703401a070 con 0x7f70400c38c0 2024-03-04T08:05:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.159+0000 7f703a000640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7024066f90 0x7f7024069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:48.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.159+0000 7f703a000640 1 -- 172.21.15.113:0/30229130 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f70340856c0 con 0x7f70400c38c0 2024-03-04T08:05:48.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.159+0000 7f7045400640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7024066f90 0x7f7024069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:48.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.159+0000 7f7045400640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7024066f90 0x7f7024069450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f70300af000 tx=0x7f70300a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:48.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.167+0000 7f703a000640 1 -- 172.21.15.113:0/30229130 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7034055970 con 0x7f70400c38c0 2024-03-04T08:05:48.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.415+0000 7f7048800640 1 -- 172.21.15.113:0/30229130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f704014ac10 con 0x7f70400c38c0 2024-03-04T08:05:48.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f703a000640 1 -- 172.21.15.113:0/30229130 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f7034059a10 con 0x7f70400c38c0 2024-03-04T08:05:48.414 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:48.415 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":120,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:05:45.841121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119]},"id":1}]} 2024-03-04T08:05:48.415 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 120 2024-03-04T08:05:48.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 -- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7024066f90 msgr2=0x7f7024069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:48.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7024066f90 0x7f7024069450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f70300af000 tx=0x7f70300a7000 comp rx=0 tx=0).stop 2024-03-04T08:05:48.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 -- 172.21.15.113:0/30229130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70400c38c0 msgr2=0x7f704006dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:48.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70400c38c0 0x7f704006dbd0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f703400e8e0 tx=0x7f703400eda0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 -- 172.21.15.113:0/30229130 shutdown_connections 2024-03-04T08:05:48.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7040159470 0x7f704006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7040152d60 0x7f70400cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7024066f90 0x7f7024069450 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 --2- 172.21.15.113:0/30229130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70400c38c0 0x7f704006dbd0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:48.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 -- 172.21.15.113:0/30229130 >> 172.21.15.113:0/30229130 conn(0x7f7040066db0 msgr2=0x7f7040157fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:48.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 -- 172.21.15.113:0/30229130 shutdown_connections 2024-03-04T08:05:48.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:48.419+0000 7f7048800640 1 -- 172.21.15.113:0/30229130 wait complete. 2024-03-04T08:05:48.438 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 120, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 120, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6925": { "addr": "172.21.15.136:6835/2522634751", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2522634751, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2522634751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6925, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 23 }, "gid_7119": { "addr": "172.21.15.113:6833/3825257303", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 3825257303, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 3825257303, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7119, "incarnation": 114, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 25 }, "gid_7269": { "addr": "172.21.15.113:6837/1401330284", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1401330284, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1401330284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7269, "incarnation": 120, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 66, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:05:45.841121+0000", "qdb_cluster": [ 6925, 7119 ], "qdb_leader": 6925, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6925, "mds_1": 7119, "mds_2": 7269 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:05:50.438 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:50.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb8ba00640 1 Processor -- start 2024-03-04T08:05:50.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb8ba00640 1 -- start start 2024-03-04T08:05:50.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb8ba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb840c8200 0x7fbb84070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:50.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb8ba00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb841591f0 con 0x7fbb841527f0 2024-03-04T08:05:50.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb8ba00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb841594d0 con 0x7fbb84152bd0 2024-03-04T08:05:50.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb8ba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84071df0 con 0x7fbb840c8200 2024-03-04T08:05:50.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb89000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbb841527f0 0x7fbb84072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49604/0 (socket says 172.21.15.113:49604) 2024-03-04T08:05:50.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb89000640 1 -- 172.21.15.113:0/2348760834 learned_addr learned my addr 172.21.15.113:0/2348760834 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:05:50.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb83e00640 1 --2- 172.21.15.113:0/2348760834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb840c8200 0x7fbb84070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:50.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb83400640 1 -- 172.21.15.113:0/2348760834 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679296267 0 0) 0x7fbb841591f0 con 0x7fbb841527f0 2024-03-04T08:05:50.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb83400640 1 -- 172.21.15.113:0/2348760834 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb6c003530 con 0x7fbb841527f0 2024-03-04T08:05:50.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb83400640 1 -- 172.21.15.113:0/2348760834 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986551801 0 0) 0x7fbb841594d0 con 0x7fbb84152bd0 2024-03-04T08:05:50.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.579+0000 7fbb83400640 1 -- 172.21.15.113:0/2348760834 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb841591f0 con 0x7fbb84152bd0 2024-03-04T08:05:50.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb83e00640 1 -- 172.21.15.113:0/2348760834 >> v1:172.21.15.136:6789/0 conn(0x7fbb84152bd0 legacy=0x7fbb841587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb83e00640 1 -- 172.21.15.113:0/2348760834 >> v1:172.21.15.113:6789/0 conn(0x7fbb841527f0 legacy=0x7fbb84072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb83400640 1 -- 172.21.15.113:0/2348760834 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3619400165 0 0) 0x7fbb6c003530 con 0x7fbb841527f0 2024-03-04T08:05:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb83e00640 1 -- 172.21.15.113:0/2348760834 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840717a0 con 0x7fbb840c8200 2024-03-04T08:05:50.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb83e00640 1 --2- 172.21.15.113:0/2348760834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb840c8200 0x7fbb84070d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbb700097f0 tx=0x7fbb7001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb15a6b70895992c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb83400640 1 -- 172.21.15.113:0/2348760834 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700ab5f0 con 0x7fbb840c8200 2024-03-04T08:05:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb83400640 1 -- 172.21.15.113:0/2348760834 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb700b0070 con 0x7fbb840c8200 2024-03-04T08:05:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb83400640 1 -- 172.21.15.113:0/2348760834 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700ab990 con 0x7fbb840c8200 2024-03-04T08:05:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb8ba00640 1 -- 172.21.15.113:0/2348760834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb840c8200 msgr2=0x7fbb84070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb8ba00640 1 --2- 172.21.15.113:0/2348760834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb840c8200 0x7fbb84070d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbb700097f0 tx=0x7fbb7001c4e0 comp rx=0 tx=0).stop 2024-03-04T08:05:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb8ba00640 1 -- 172.21.15.113:0/2348760834 shutdown_connections 2024-03-04T08:05:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb8ba00640 1 --2- 172.21.15.113:0/2348760834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb840c8200 0x7fbb84070d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.583+0000 7fbb8ba00640 1 -- 172.21.15.113:0/2348760834 >> 172.21.15.113:0/2348760834 conn(0x7fbb8406e820 msgr2=0x7fbb8406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb8ba00640 1 -- 172.21.15.113:0/2348760834 shutdown_connections 2024-03-04T08:05:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb8ba00640 1 -- 172.21.15.113:0/2348760834 wait complete. 2024-03-04T08:05:50.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb8ba00640 1 Processor -- start 2024-03-04T08:05:50.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb8ba00640 1 -- start start 2024-03-04T08:05:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb8ba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb840c8200 0x7fbb841f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb8ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb841527f0 0x7fbb840c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb8ba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb84152bd0 0x7fbb841f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb8ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84166380 con 0x7fbb841527f0 2024-03-04T08:05:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb8ba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb841597b0 con 0x7fbb840c8200 2024-03-04T08:05:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb8ba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb84166500 con 0x7fbb84152bd0 2024-03-04T08:05:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb83e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb841527f0 0x7fbb840c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb83e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb841527f0 0x7fbb840c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57130/0 (socket says 172.21.15.113:57130) 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb83e00640 1 -- 172.21.15.113:0/1431042494 learned_addr learned my addr 172.21.15.113:0/1431042494 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb89000640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb840c8200 0x7fbb841f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb89a00640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb84152bd0 0x7fbb841f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb83e00640 1 -- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb84152bd0 msgr2=0x7fbb841f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb83e00640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb84152bd0 0x7fbb841f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb83e00640 1 -- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb840c8200 msgr2=0x7fbb841f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb83e00640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb840c8200 0x7fbb841f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb83e00640 1 -- 172.21.15.113:0/1431042494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840c3900 con 0x7fbb841527f0 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.587+0000 7fbb83e00640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb841527f0 0x7fbb840c4c40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbb700097f0 tx=0x7fbb70019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.591+0000 7fbb80c00640 1 -- 172.21.15.113:0/1431042494 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70003820 con 0x7fbb841527f0 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.591+0000 7fbb80c00640 1 -- 172.21.15.113:0/1431042494 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb700039c0 con 0x7fbb841527f0 2024-03-04T08:05:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.591+0000 7fbb80c00640 1 -- 172.21.15.113:0/1431042494 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70003d70 con 0x7fbb841527f0 2024-03-04T08:05:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.591+0000 7fbb8ba00640 1 -- 172.21.15.113:0/1431042494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb840c3ba0 con 0x7fbb841527f0 2024-03-04T08:05:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.591+0000 7fbb8ba00640 1 -- 172.21.15.113:0/1431042494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb840c4160 con 0x7fbb841527f0 2024-03-04T08:05:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.591+0000 7fbb76000640 1 -- 172.21.15.113:0/1431042494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb84075c00 con 0x7fbb841527f0 2024-03-04T08:05:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.591+0000 7fbb80c00640 1 -- 172.21.15.113:0/1431042494 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb700c3060 con 0x7fbb841527f0 2024-03-04T08:05:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.591+0000 7fbb80c00640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb64066f60 0x7fbb64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.591+0000 7fbb80c00640 1 -- 172.21.15.113:0/1431042494 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7fbb7011f070 con 0x7fbb841527f0 2024-03-04T08:05:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.591+0000 7fbb89000640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb64066f60 0x7fbb64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.595+0000 7fbb89000640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb64066f60 0x7fbb64069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbb78005890 tx=0x7fbb78009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.595+0000 7fbb80c00640 1 -- 172.21.15.113:0/1431042494 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbb700160d0 con 0x7fbb841527f0 2024-03-04T08:05:50.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.815+0000 7fbb76000640 1 -- 172.21.15.113:0/1431042494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb840c79d0 con 0x7fbb841527f0 2024-03-04T08:05:50.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb80c00640 1 -- 172.21.15.113:0/1431042494 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+5823 (secure 0 0 0) 0x7fbb700ef430 con 0x7fbb841527f0 2024-03-04T08:05:50.815 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:50.816 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":120,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:05:45.841121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119]},"id":1}]} 2024-03-04T08:05:50.816 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 120 2024-03-04T08:05:50.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 -- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb64066f60 msgr2=0x7fbb64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:50.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb64066f60 0x7fbb64069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbb78005890 tx=0x7fbb78009040 comp rx=0 tx=0).stop 2024-03-04T08:05:50.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 -- 172.21.15.113:0/1431042494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb841527f0 msgr2=0x7fbb840c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:50.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb841527f0 0x7fbb840c4c40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbb700097f0 tx=0x7fbb70019280 comp rx=0 tx=0).stop 2024-03-04T08:05:50.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 -- 172.21.15.113:0/1431042494 shutdown_connections 2024-03-04T08:05:50.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb84152bd0 0x7fbb841f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:50.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb64066f60 0x7fbb64069420 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:50.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb841527f0 0x7fbb840c4c40 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 --2- 172.21.15.113:0/1431042494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb840c8200 0x7fbb841f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 -- 172.21.15.113:0/1431042494 >> 172.21.15.113:0/1431042494 conn(0x7fbb8406e820 msgr2=0x7fbb840c2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 -- 172.21.15.113:0/1431042494 shutdown_connections 2024-03-04T08:05:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:50.819+0000 7fbb76000640 1 -- 172.21.15.113:0/1431042494 wait complete. 2024-03-04T08:05:50.836 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:05:52.837 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.975+0000 7fe427a00640 1 Processor -- start 2024-03-04T08:05:52.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.975+0000 7fe427a00640 1 -- start start 2024-03-04T08:05:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe427a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200c8200 0x7fe420070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe427a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4201591f0 con 0x7fe4201527f0 2024-03-04T08:05:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe427a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4201594d0 con 0x7fe420152bd0 2024-03-04T08:05:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe427a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe420071df0 con 0x7fe4200c8200 2024-03-04T08:05:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe425000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe4201527f0 0x7fe420072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49606/0 (socket says 172.21.15.113:49606) 2024-03-04T08:05:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe425000640 1 -- 172.21.15.113:0/2451683226 learned_addr learned my addr 172.21.15.113:0/2451683226 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:05:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41fe00640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200c8200 0x7fe420070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890179878 0 0) 0x7fe4201591f0 con 0x7fe4201527f0 2024-03-04T08:05:52.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe408003530 con 0x7fe4201527f0 2024-03-04T08:05:52.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4077483453 0 0) 0x7fe408003530 con 0x7fe4201527f0 2024-03-04T08:05:52.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200c8200 msgr2=0x7fe420070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:52.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200c8200 0x7fe420070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:52.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 >> v1:172.21.15.136:6789/0 conn(0x7fe420152bd0 legacy=0x7fe4201587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:52.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4200717a0 con 0x7fe4201527f0 2024-03-04T08:05:52.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fe414001680 con 0x7fe4201527f0 2024-03-04T08:05:52.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 >> v1:172.21.15.113:6789/0 conn(0x7fe4201527f0 legacy=0x7fe420072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200c8200 0x7fe4080039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe408003f50 0x7fe408004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe408004d50 0x7fe4080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420071df0 con 0x7fe408004d50 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40800bab0 con 0x7fe408003f50 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40800bcb0 con 0x7fe4200c8200 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe425a00640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe408004d50 0x7fe4080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe425000640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200c8200 0x7fe4080039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe425a00640 1 -- 172.21.15.113:0/2451683226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200c8200 msgr2=0x7fe4080039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe425a00640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200c8200 0x7fe4080039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe425a00640 1 -- 172.21.15.113:0/2451683226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe408003f50 msgr2=0x7fe408004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe425a00640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe408003f50 0x7fe408004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:52.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.979+0000 7fe425a00640 1 -- 172.21.15.113:0/2451683226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408003530 con 0x7fe408004d50 2024-03-04T08:05:52.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe425a00640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe408004d50 0x7fe4080051d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe418001d50 tx=0x7fe41801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9184b29cf7c45931 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:52.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180ad070 con 0x7fe408004d50 2024-03-04T08:05:52.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe418003ba0 con 0x7fe408004d50 2024-03-04T08:05:52.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe41f400640 1 -- 172.21.15.113:0/2451683226 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418001950 con 0x7fe408004d50 2024-03-04T08:05:52.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe427a00640 1 -- 172.21.15.113:0/2451683226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe408004d50 msgr2=0x7fe4080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:52.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe427a00640 1 --2- 172.21.15.113:0/2451683226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe408004d50 0x7fe4080051d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe418001d50 tx=0x7fe41801c680 comp rx=0 tx=0).stop 2024-03-04T08:05:52.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe425a00640 1 -- 172.21.15.113:0/2451683226 reap_dead start 2024-03-04T08:05:52.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe427a00640 1 -- 172.21.15.113:0/2451683226 shutdown_connections 2024-03-04T08:05:52.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe427a00640 1 -- 172.21.15.113:0/2451683226 >> 172.21.15.113:0/2451683226 conn(0x7fe42006e820 msgr2=0x7fe42006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:52.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe427a00640 1 -- 172.21.15.113:0/2451683226 shutdown_connections 2024-03-04T08:05:52.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe427a00640 1 -- 172.21.15.113:0/2451683226 wait complete. 2024-03-04T08:05:52.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe427a00640 1 Processor -- start 2024-03-04T08:05:52.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.983+0000 7fe427a00640 1 -- start start 2024-03-04T08:05:52.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe427a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe408004d50 0x7fe4200cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:52.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe427a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200af850 0x7fe4201530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:52.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe427a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42014dc80 0x7fe42014e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:52.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe427a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420158ee0 con 0x7fe42014dc80 2024-03-04T08:05:52.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe427a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4201597b0 con 0x7fe408004d50 2024-03-04T08:05:52.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe427a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe420070d40 con 0x7fe4200af850 2024-03-04T08:05:52.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe425a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42014dc80 0x7fe42014e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:52.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe425a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42014dc80 0x7fe42014e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57154/0 (socket says 172.21.15.113:57154) 2024-03-04T08:05:52.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe425a00640 1 -- 172.21.15.113:0/694526189 learned_addr learned my addr 172.21.15.113:0/694526189 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:52.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe41fe00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200af850 0x7fe4201530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:52.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe425a00640 1 -- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200af850 msgr2=0x7fe4201530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:52.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe425a00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200af850 0x7fe4201530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:52.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe425a00640 1 -- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe408004d50 msgr2=0x7fe4200cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:52.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe425a00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe408004d50 0x7fe4200cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:52.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe425a00640 1 -- 172.21.15.113:0/694526189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42014f500 con 0x7fe42014dc80 2024-03-04T08:05:52.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe425a00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42014dc80 0x7fe42014e100 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe41801cb60 tx=0x7fe408005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:52.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe41cc00640 1 -- 172.21.15.113:0/694526189 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180ad070 con 0x7fe42014dc80 2024-03-04T08:05:52.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe41cc00640 1 -- 172.21.15.113:0/694526189 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4180ae040 con 0x7fe42014dc80 2024-03-04T08:05:52.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe41cc00640 1 -- 172.21.15.113:0/694526189 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180054b0 con 0x7fe42014dc80 2024-03-04T08:05:52.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe427a00640 1 -- 172.21.15.113:0/694526189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42014f7a0 con 0x7fe42014dc80 2024-03-04T08:05:52.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe427a00640 1 -- 172.21.15.113:0/694526189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4201ef320 con 0x7fe42014dc80 2024-03-04T08:05:52.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.987+0000 7fe427a00640 1 -- 172.21.15.113:0/694526189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe420075c00 con 0x7fe42014dc80 2024-03-04T08:05:52.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.991+0000 7fe41cc00640 1 -- 172.21.15.113:0/694526189 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4180bb060 con 0x7fe42014dc80 2024-03-04T08:05:52.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.991+0000 7fe41cc00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe400066f60 0x7fe400069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:52.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.991+0000 7fe41cc00640 1 -- 172.21.15.113:0/694526189 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7fe41811e3a0 con 0x7fe42014dc80 2024-03-04T08:05:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.991+0000 7fe425000640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe400066f60 0x7fe400069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.991+0000 7fe425000640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe400066f60 0x7fe400069420 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe4140017f0 tx=0x7fe4140014a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:52.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:52.999+0000 7fe41cc00640 1 -- 172.21.15.113:0/694526189 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe418016390 con 0x7fe42014dc80 2024-03-04T08:05:53.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 -- 172.21.15.113:0/694526189 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4200c6850 con 0x7fe42014dc80 2024-03-04T08:05:53.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe41cc00640 1 -- 172.21.15.113:0/694526189 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5825 (secure 0 0 0) 0x7fe4180ee640 con 0x7fe42014dc80 2024-03-04T08:05:53.192 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:53.192 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:05:50.853109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7119,6925]},"id":1}]} 2024-03-04T08:05:53.192 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 121 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 -- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe400066f60 msgr2=0x7fe400069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe400066f60 0x7fe400069420 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe4140017f0 tx=0x7fe4140014a0 comp rx=0 tx=0).stop 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 -- 172.21.15.113:0/694526189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42014dc80 msgr2=0x7fe42014e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42014dc80 0x7fe42014e100 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe41801cb60 tx=0x7fe408005840 comp rx=0 tx=0).stop 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 -- 172.21.15.113:0/694526189 shutdown_connections 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe400066f60 0x7fe400069420 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe42014dc80 0x7fe42014e100 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4200af850 0x7fe4201530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 --2- 172.21.15.113:0/694526189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe408004d50 0x7fe4200cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 -- 172.21.15.113:0/694526189 >> 172.21.15.113:0/694526189 conn(0x7fe42006e820 msgr2=0x7fe4200ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 -- 172.21.15.113:0/694526189 shutdown_connections 2024-03-04T08:05:53.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:53.195+0000 7fe427a00640 1 -- 172.21.15.113:0/694526189 wait complete. 2024-03-04T08:05:53.218 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:05:55.218 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:55.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2ab5800640 1 Processor -- start 2024-03-04T08:05:55.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2ab5800640 1 -- start start 2024-03-04T08:05:55.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2ab5800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2ab015bb40 0x7f2ab006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:55.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2ab5800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00c4ce0 con 0x7f2ab00cb540 2024-03-04T08:05:55.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2ab5800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab00c4f80 con 0x7f2ab0152950 2024-03-04T08:05:55.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2ab5800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00c8370 con 0x7f2ab015bb40 2024-03-04T08:05:55.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2aaf400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2ab00cb540 0x7f2ab0160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49612/0 (socket says 172.21.15.113:49612) 2024-03-04T08:05:55.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2aaf400640 1 -- 172.21.15.113:0/1662291046 learned_addr learned my addr 172.21.15.113:0/1662291046 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:05:55.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2ab4e00640 1 --2- 172.21.15.113:0/1662291046 >> v2:172.21.15.136:3300/0 conn(0x7f2ab015bb40 0x7f2ab006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:55.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2aaea00640 1 -- 172.21.15.113:0/1662291046 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661977114 0 0) 0x7f2ab00c4ce0 con 0x7f2ab00cb540 2024-03-04T08:05:55.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2aaea00640 1 -- 172.21.15.113:0/1662291046 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a94003530 con 0x7f2ab00cb540 2024-03-04T08:05:55.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2aaea00640 1 -- 172.21.15.113:0/1662291046 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180734093 0 0) 0x7f2ab00c4f80 con 0x7f2ab0152950 2024-03-04T08:05:55.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2aaea00640 1 -- 172.21.15.113:0/1662291046 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ab00c4ce0 con 0x7f2ab0152950 2024-03-04T08:05:55.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2ab4e00640 1 -- 172.21.15.113:0/1662291046 >> v1:172.21.15.136:6789/0 conn(0x7f2ab0152950 legacy=0x7f2ab006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:55.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2aaea00640 1 -- 172.21.15.113:0/1662291046 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2746708230 0 0) 0x7f2a94003530 con 0x7f2ab00cb540 2024-03-04T08:05:55.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2ab4e00640 1 -- 172.21.15.113:0/1662291046 >> v1:172.21.15.113:6789/0 conn(0x7f2ab00cb540 legacy=0x7f2ab0160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:55.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.359+0000 7f2ab4e00640 1 -- 172.21.15.113:0/1662291046 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00cb920 con 0x7f2ab015bb40 2024-03-04T08:05:55.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2ab4e00640 1 --2- 172.21.15.113:0/1662291046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ab015bb40 0x7f2ab006faa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009d90 tx=0x7f2aa801f080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebb7058a4ac0fbcf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:55.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2aaea00640 1 -- 172.21.15.113:0/1662291046 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80b5070 con 0x7f2ab015bb40 2024-03-04T08:05:55.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2aaea00640 1 -- 172.21.15.113:0/1662291046 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa80a8c10 con 0x7f2ab015bb40 2024-03-04T08:05:55.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2aaea00640 1 -- 172.21.15.113:0/1662291046 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c001680 con 0x7f2ab015bb40 2024-03-04T08:05:55.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2ab5800640 1 -- 172.21.15.113:0/1662291046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ab015bb40 msgr2=0x7f2ab006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:55.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2ab5800640 1 --2- 172.21.15.113:0/1662291046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ab015bb40 0x7f2ab006faa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009d90 tx=0x7f2aa801f080 comp rx=0 tx=0).stop 2024-03-04T08:05:55.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2ab5800640 1 -- 172.21.15.113:0/1662291046 shutdown_connections 2024-03-04T08:05:55.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2ab5800640 1 --2- 172.21.15.113:0/1662291046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ab015bb40 0x7f2ab006faa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:55.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2ab5800640 1 -- 172.21.15.113:0/1662291046 >> 172.21.15.113:0/1662291046 conn(0x7f2ab0072f40 msgr2=0x7f2ab00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:55.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2ab5800640 1 -- 172.21.15.113:0/1662291046 shutdown_connections 2024-03-04T08:05:55.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.363+0000 7f2ab5800640 1 -- 172.21.15.113:0/1662291046 wait complete. 2024-03-04T08:05:55.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 Processor -- start 2024-03-04T08:05:55.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 -- start start 2024-03-04T08:05:55.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ab00cb540 0x7f2ab0156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:55.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ab0152950 0x7f2ab01f0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:55.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ab015bb40 0x7f2ab01f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:55.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00c9a50 con 0x7f2ab00cb540 2024-03-04T08:05:55.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00ca0a0 con 0x7f2ab015bb40 2024-03-04T08:05:55.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00c9d30 con 0x7f2ab0152950 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aafe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ab00cb540 0x7f2ab0156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aafe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ab00cb540 0x7f2ab0156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57170/0 (socket says 172.21.15.113:57170) 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aafe00640 1 -- 172.21.15.113:0/3375585770 learned_addr learned my addr 172.21.15.113:0/3375585770 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aaf400640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ab0152950 0x7f2ab01f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab4e00640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ab015bb40 0x7f2ab01f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aafe00640 1 -- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ab0152950 msgr2=0x7f2ab01f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aafe00640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ab0152950 0x7f2ab01f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aafe00640 1 -- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ab015bb40 msgr2=0x7f2ab01f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aafe00640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ab015bb40 0x7f2ab01f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aafe00640 1 -- 172.21.15.113:0/3375585770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab01f3a30 con 0x7f2ab00cb540 2024-03-04T08:05:55.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aafe00640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ab00cb540 0x7f2ab0156620 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000c4c0 tx=0x7f2aa000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:55.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aa7e00640 1 -- 172.21.15.113:0/3375585770 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00052d0 con 0x7f2ab00cb540 2024-03-04T08:05:55.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aa7e00640 1 -- 172.21.15.113:0/3375585770 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa0005490 con 0x7f2ab00cb540 2024-03-04T08:05:55.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 -- 172.21.15.113:0/3375585770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab01f3ca0 con 0x7f2ab00cb540 2024-03-04T08:05:55.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2aa7e00640 1 -- 172.21.15.113:0/3375585770 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0012850 con 0x7f2ab00cb540 2024-03-04T08:05:55.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 -- 172.21.15.113:0/3375585770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab01f4260 con 0x7f2ab00cb540 2024-03-04T08:05:55.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.367+0000 7f2ab5800640 1 -- 172.21.15.113:0/3375585770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab0075a80 con 0x7f2ab00cb540 2024-03-04T08:05:55.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.371+0000 7f2aa7e00640 1 -- 172.21.15.113:0/3375585770 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2aa00129f0 con 0x7f2ab00cb540 2024-03-04T08:05:55.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.371+0000 7f2aa7e00640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a90066f60 0x7f2a90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:55.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.371+0000 7f2aa7e00640 1 -- 172.21.15.113:0/3375585770 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f2aa0085220 con 0x7f2ab00cb540 2024-03-04T08:05:55.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.371+0000 7f2aaf400640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a90066f60 0x7f2a90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:55.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.371+0000 7f2aaf400640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a90066f60 0x7f2a90069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0152f10 tx=0x7f2a9c0016a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:55.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.375+0000 7f2aa7e00640 1 -- 172.21.15.113:0/3375585770 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2aa00554c0 con 0x7f2ab00cb540 2024-03-04T08:05:55.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.575+0000 7f2ab5800640 1 -- 172.21.15.113:0/3375585770 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ab01572e0 con 0x7f2ab00cb540 2024-03-04T08:05:55.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.575+0000 7f2aa7e00640 1 -- 172.21.15.113:0/3375585770 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+5825 (secure 0 0 0) 0x7f2aa0059560 con 0x7f2ab00cb540 2024-03-04T08:05:55.573 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:55.574 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:05:50.853109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7119,6925]},"id":1}]} 2024-03-04T08:05:55.574 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 121 2024-03-04T08:05:55.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 -- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a90066f60 msgr2=0x7f2a90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:55.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a90066f60 0x7f2a90069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0152f10 tx=0x7f2a9c0016a0 comp rx=0 tx=0).stop 2024-03-04T08:05:55.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 -- 172.21.15.113:0/3375585770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ab00cb540 msgr2=0x7f2ab0156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:55.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ab00cb540 0x7f2ab0156620 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000c4c0 tx=0x7f2aa000c980 comp rx=0 tx=0).stop 2024-03-04T08:05:55.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 -- 172.21.15.113:0/3375585770 shutdown_connections 2024-03-04T08:05:55.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ab015bb40 0x7f2ab01f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:55.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ab0152950 0x7f2ab01f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:55.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a90066f60 0x7f2a90069420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:55.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 --2- 172.21.15.113:0/3375585770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ab00cb540 0x7f2ab0156620 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:55.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 -- 172.21.15.113:0/3375585770 >> 172.21.15.113:0/3375585770 conn(0x7f2ab0072f40 msgr2=0x7f2ab00c7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:55.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 -- 172.21.15.113:0/3375585770 shutdown_connections 2024-03-04T08:05:55.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:55.579+0000 7f2ab5800640 1 -- 172.21.15.113:0/3375585770 wait complete. 2024-03-04T08:05:55.595 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:05:57.596 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:05:57.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.735+0000 7f0b42c00640 1 Processor -- start 2024-03-04T08:05:57.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.735+0000 7f0b42c00640 1 -- start start 2024-03-04T08:05:57.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b42c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0b3c0c7a80 0x7f0b3c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:57.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b42c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b3c06cb60 0x7f0b3c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:57.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b42c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c06b620 con 0x7f0b3c152c00 2024-03-04T08:05:57.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b42c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c072e10 con 0x7f0b3c06cb60 2024-03-04T08:05:57.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b42c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c072980 con 0x7f0b3c0c7a80 2024-03-04T08:05:57.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b40c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b3c06cb60 0x7f0b3c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:57.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b3b400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0b3c0c7a80 0x7f0b3c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:57.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b40c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0b3c06cb60 0x7f0b3c0cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60998/0 (socket says 172.21.15.113:60998) 2024-03-04T08:05:57.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b40c00640 1 -- 172.21.15.113:0/886918677 learned_addr learned my addr 172.21.15.113:0/886918677 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:57.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b40c00640 1 -- 172.21.15.113:0/886918677 >> v2:172.21.15.136:3300/0 conn(0x7f0b3c0c7a80 msgr2=0x7f0b3c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:05:57.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b40c00640 1 --2- 172.21.15.113:0/886918677 >> v2:172.21.15.136:3300/0 conn(0x7f0b3c0c7a80 0x7f0b3c0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:57.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b40c00640 1 -- 172.21.15.113:0/886918677 >> v1:172.21.15.136:6789/0 conn(0x7f0b3c152c00 legacy=0x7f0b3c15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:57.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.739+0000 7f0b40c00640 1 -- 172.21.15.113:0/886918677 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c073580 con 0x7f0b3c06cb60 2024-03-04T08:05:57.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.743+0000 7f0b40c00640 1 --2- 172.21.15.113:0/886918677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 0x7f0b3c0cd550 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0ce140 tx=0x7f0b3401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f788ea859d99eb22 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.743+0000 7f0b3aa00640 1 -- 172.21.15.113:0/886918677 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340a5880 con 0x7f0b3c06cb60 2024-03-04T08:05:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.743+0000 7f0b3aa00640 1 -- 172.21.15.113:0/886918677 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b340a5a20 con 0x7f0b3c06cb60 2024-03-04T08:05:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.743+0000 7f0b3aa00640 1 -- 172.21.15.113:0/886918677 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340a5df0 con 0x7f0b3c06cb60 2024-03-04T08:05:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.743+0000 7f0b42c00640 1 -- 172.21.15.113:0/886918677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 msgr2=0x7f0b3c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.743+0000 7f0b42c00640 1 --2- 172.21.15.113:0/886918677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 0x7f0b3c0cd550 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0ce140 tx=0x7f0b3401c9c0 comp rx=0 tx=0).stop 2024-03-04T08:05:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.743+0000 7f0b42c00640 1 -- 172.21.15.113:0/886918677 shutdown_connections 2024-03-04T08:05:57.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.743+0000 7f0b42c00640 1 --2- 172.21.15.113:0/886918677 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 0x7f0b3c0cd550 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:57.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.743+0000 7f0b42c00640 1 --2- 172.21.15.113:0/886918677 >> v2:172.21.15.136:3300/0 conn(0x7f0b3c0c7a80 0x7f0b3c0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:57.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.743+0000 7f0b42c00640 1 -- 172.21.15.113:0/886918677 >> 172.21.15.113:0/886918677 conn(0x7f0b3c0671b0 msgr2=0x7f0b3c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:57.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 -- 172.21.15.113:0/886918677 shutdown_connections 2024-03-04T08:05:57.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 -- 172.21.15.113:0/886918677 wait complete. 2024-03-04T08:05:57.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 Processor -- start 2024-03-04T08:05:57.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 -- start start 2024-03-04T08:05:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 0x7f0b3c154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b3c0c7a80 0x7f0b3c0c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b3c152c00 0x7f0b3c153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0731d0 con 0x7f0b3c06cb60 2024-03-04T08:05:57.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c15c450 con 0x7f0b3c152c00 2024-03-04T08:05:57.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0c6640 con 0x7f0b3c0c7a80 2024-03-04T08:05:57.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b3be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 0x7f0b3c154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:57.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b3be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 0x7f0b3c154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32770/0 (socket says 172.21.15.113:32770) 2024-03-04T08:05:57.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b3be00640 1 -- 172.21.15.113:0/2494829840 learned_addr learned my addr 172.21.15.113:0/2494829840 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:05:57.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b3b400640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b3c0c7a80 0x7f0b3c0c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:57.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b40c00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b3c152c00 0x7f0b3c153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:57.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b3be00640 1 -- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b3c0c7a80 msgr2=0x7f0b3c0c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:57.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b3be00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b3c0c7a80 0x7f0b3c0c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b3be00640 1 -- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b3c152c00 msgr2=0x7f0b3c153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b3be00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b3c152c00 0x7f0b3c153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b3be00640 1 -- 172.21.15.113:0/2494829840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0c63a0 con 0x7f0b3c06cb60 2024-03-04T08:05:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b40c00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b3c152c00 0x7f0b3c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:05:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b3be00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 0x7f0b3c154fb0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00c3c0 tx=0x7f0b2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b33e00640 1 -- 172.21.15.113:0/2494829840 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c017070 con 0x7f0b3c06cb60 2024-03-04T08:05:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b33e00640 1 -- 172.21.15.113:0/2494829840 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c0052d0 con 0x7f0b3c06cb60 2024-03-04T08:05:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 -- 172.21.15.113:0/2494829840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c155c80 con 0x7f0b3c06cb60 2024-03-04T08:05:57.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b33e00640 1 -- 172.21.15.113:0/2494829840 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c003840 con 0x7f0b3c06cb60 2024-03-04T08:05:57.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.747+0000 7f0b42c00640 1 -- 172.21.15.113:0/2494829840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c0c6060 con 0x7f0b3c06cb60 2024-03-04T08:05:57.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.751+0000 7f0b42c00640 1 -- 172.21.15.113:0/2494829840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c1561a0 con 0x7f0b3c06cb60 2024-03-04T08:05:57.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.751+0000 7f0b33e00640 1 -- 172.21.15.113:0/2494829840 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b2c01a070 con 0x7f0b3c06cb60 2024-03-04T08:05:57.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.751+0000 7f0b33e00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b1c066f90 0x7f0b1c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:05:57.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.751+0000 7f0b33e00640 1 -- 172.21.15.113:0/2494829840 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f0b2c0849a0 con 0x7f0b3c06cb60 2024-03-04T08:05:57.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.755+0000 7f0b3b400640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b1c066f90 0x7f0b1c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:05:57.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.755+0000 7f0b3b400640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b1c066f90 0x7f0b1c069450 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0cd1a0 tx=0x7f0b28008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:05:57.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.759+0000 7f0b33e00640 1 -- 172.21.15.113:0/2494829840 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0b3c1561a0 con 0x7f0b3c06cb60 2024-03-04T08:05:57.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.947+0000 7f0b42c00640 1 -- 172.21.15.113:0/2494829840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b3c075c00 con 0x7f0b3c06cb60 2024-03-04T08:05:57.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.947+0000 7f0b33e00640 1 -- 172.21.15.113:0/2494829840 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+5827 (secure 0 0 0) 0x7f0b3c06b620 con 0x7f0b3c06cb60 2024-03-04T08:05:57.945 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:05:57.945 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":122,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:05:56.272511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119]},"id":1}]} 2024-03-04T08:05:57.946 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 122 2024-03-04T08:05:57.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 -- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b1c066f90 msgr2=0x7f0b1c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:57.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b1c066f90 0x7f0b1c069450 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0cd1a0 tx=0x7f0b28008040 comp rx=0 tx=0).stop 2024-03-04T08:05:57.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 -- 172.21.15.113:0/2494829840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 msgr2=0x7f0b3c154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:05:57.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 0x7f0b3c154fb0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00c3c0 tx=0x7f0b2c00c880 comp rx=0 tx=0).stop 2024-03-04T08:05:57.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 -- 172.21.15.113:0/2494829840 shutdown_connections 2024-03-04T08:05:57.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b3c152c00 0x7f0b3c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:57.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0b3c0c7a80 0x7f0b3c0c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:57.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0b1c066f90 0x7f0b1c069450 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:57.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 --2- 172.21.15.113:0/2494829840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0b3c06cb60 0x7f0b3c154fb0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:05:57.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 -- 172.21.15.113:0/2494829840 >> 172.21.15.113:0/2494829840 conn(0x7f0b3c0671b0 msgr2=0x7f0b3c151c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:05:57.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 -- 172.21.15.113:0/2494829840 shutdown_connections 2024-03-04T08:05:57.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:05:57.951+0000 7f0b42c00640 1 -- 172.21.15.113:0/2494829840 wait complete. 2024-03-04T08:05:57.966 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:05:59.968 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:06:00.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93bbe00640 1 Processor -- start 2024-03-04T08:06:00.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93bbe00640 1 -- start start 2024-03-04T08:06:00.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93bbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f93b406dd50 0x7f93b41662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:00.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93bbe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b40ca0e0 con 0x7f93b406ff20 2024-03-04T08:06:00.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93bbe00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b415aa30 con 0x7f93b4158010 2024-03-04T08:06:00.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93bbe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b406f450 con 0x7f93b406dd50 2024-03-04T08:06:00.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93b8a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f93b406ff20 0x7f93b40ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60282/0 (socket says 172.21.15.113:60282) 2024-03-04T08:06:00.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93b8a00640 1 -- 172.21.15.113:0/522605841 learned_addr learned my addr 172.21.15.113:0/522605841 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:06:00.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93b9e00640 1 --2- 172.21.15.113:0/522605841 >> v2:172.21.15.113:3300/0 conn(0x7f93b406dd50 0x7f93b41662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:00.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93b3e00640 1 -- 172.21.15.113:0/522605841 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135439778 0 0) 0x7f93b40ca0e0 con 0x7f93b406ff20 2024-03-04T08:06:00.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93b3e00640 1 -- 172.21.15.113:0/522605841 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f939c003530 con 0x7f93b406ff20 2024-03-04T08:06:00.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93b9e00640 1 -- 172.21.15.113:0/522605841 >> v1:172.21.15.136:6789/0 conn(0x7f93b4158010 legacy=0x7f93b415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:00.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93b9e00640 1 -- 172.21.15.113:0/522605841 >> v1:172.21.15.113:6789/0 conn(0x7f93b406ff20 legacy=0x7f93b40ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:00.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.115+0000 7f93b9e00640 1 -- 172.21.15.113:0/522605841 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b406c590 con 0x7f93b406dd50 2024-03-04T08:06:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.119+0000 7f93b9e00640 1 --2- 172.21.15.113:0/522605841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b406dd50 0x7f93b41662e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0097c0 tx=0x7f93ac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1337f98b32e0988b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93b3e00640 1 -- 172.21.15.113:0/522605841 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0b0070 con 0x7f93b406dd50 2024-03-04T08:06:00.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93b3e00640 1 -- 172.21.15.113:0/522605841 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac0ab600 con 0x7f93b406dd50 2024-03-04T08:06:00.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93b3e00640 1 -- 172.21.15.113:0/522605841 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0ab9d0 con 0x7f93b406dd50 2024-03-04T08:06:00.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 -- 172.21.15.113:0/522605841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b406dd50 msgr2=0x7f93b41662e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:00.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 --2- 172.21.15.113:0/522605841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b406dd50 0x7f93b41662e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0097c0 tx=0x7f93ac01c4a0 comp rx=0 tx=0).stop 2024-03-04T08:06:00.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 -- 172.21.15.113:0/522605841 shutdown_connections 2024-03-04T08:06:00.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 --2- 172.21.15.113:0/522605841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b406dd50 0x7f93b41662e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:00.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 -- 172.21.15.113:0/522605841 >> 172.21.15.113:0/522605841 conn(0x7f93b40734d0 msgr2=0x7f93b4066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:00.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 -- 172.21.15.113:0/522605841 shutdown_connections 2024-03-04T08:06:00.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 -- 172.21.15.113:0/522605841 wait complete. 2024-03-04T08:06:00.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 Processor -- start 2024-03-04T08:06:00.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 -- start start 2024-03-04T08:06:00.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b406dd50 0x7f93b41fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:00.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b406ff20 0x7f93b41f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:00.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b4158010 0x7f93b40c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:00.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.123+0000 7f93bbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b415b920 con 0x7f93b4158010 2024-03-04T08:06:00.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93bbe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b415ad80 con 0x7f93b406dd50 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93bbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b41583f0 con 0x7f93b406ff20 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b4158010 0x7f93b40c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b8a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b406ff20 0x7f93b41f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b4158010 0x7f93b40c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32790/0 (socket says 172.21.15.113:32790) 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9e00640 1 -- 172.21.15.113:0/513375882 learned_addr learned my addr 172.21.15.113:0/513375882 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9400640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b406dd50 0x7f93b41fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9e00640 1 -- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b406ff20 msgr2=0x7f93b41f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9e00640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b406ff20 0x7f93b41f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9e00640 1 -- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b406dd50 msgr2=0x7f93b41fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9e00640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b406dd50 0x7f93b41fb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9e00640 1 -- 172.21.15.113:0/513375882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b40ca6d0 con 0x7f93b4158010 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9400640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b406dd50 0x7f93b41fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:06:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b9e00640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b4158010 0x7f93b40c3410 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f93ac01a0d0 tx=0x7f93ac019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b1600640 1 -- 172.21.15.113:0/513375882 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0b0040 con 0x7f93b4158010 2024-03-04T08:06:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b1600640 1 -- 172.21.15.113:0/513375882 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac0b9070 con 0x7f93b4158010 2024-03-04T08:06:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93bbe00640 1 -- 172.21.15.113:0/513375882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b40ca9b0 con 0x7f93b4158010 2024-03-04T08:06:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93b1600640 1 -- 172.21.15.113:0/513375882 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0b4650 con 0x7f93b4158010 2024-03-04T08:06:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93bbe00640 1 -- 172.21.15.113:0/513375882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b41f7f30 con 0x7f93b4158010 2024-03-04T08:06:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.127+0000 7f93bbe00640 1 -- 172.21.15.113:0/513375882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b41f7a40 con 0x7f93b4158010 2024-03-04T08:06:00.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.131+0000 7f93b1600640 1 -- 172.21.15.113:0/513375882 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93ac0b47f0 con 0x7f93b4158010 2024-03-04T08:06:00.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.131+0000 7f93b1600640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9394066f60 0x7f9394069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.131+0000 7f93b1600640 1 -- 172.21.15.113:0/513375882 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f93ac11f3c0 con 0x7f93b4158010 2024-03-04T08:06:00.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.131+0000 7f93b9400640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9394066f60 0x7f9394069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:00.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.131+0000 7f93b9400640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9394066f60 0x7f9394069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f93a80012f0 tx=0x7f93a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.135+0000 7f93b1600640 1 -- 172.21.15.113:0/513375882 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f93ac016090 con 0x7f93b4158010 2024-03-04T08:06:00.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 -- 172.21.15.113:0/513375882 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93b4075c00 con 0x7f93b4158010 2024-03-04T08:06:00.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93b1600640 1 -- 172.21.15.113:0/513375882 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+5827 (secure 0 0 0) 0x7f93ac0efc20 con 0x7f93b4158010 2024-03-04T08:06:00.355 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:06:00.356 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":122,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:05:56.272511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119]},"id":1}]} 2024-03-04T08:06:00.356 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 122 2024-03-04T08:06:00.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 -- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9394066f60 msgr2=0x7f9394069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:00.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9394066f60 0x7f9394069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f93a80012f0 tx=0x7f93a8009040 comp rx=0 tx=0).stop 2024-03-04T08:06:00.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 -- 172.21.15.113:0/513375882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b4158010 msgr2=0x7f93b40c3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:00.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b4158010 0x7f93b40c3410 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f93ac01a0d0 tx=0x7f93ac019240 comp rx=0 tx=0).stop 2024-03-04T08:06:00.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 -- 172.21.15.113:0/513375882 shutdown_connections 2024-03-04T08:06:00.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9394066f60 0x7f9394069420 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:00.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93b4158010 0x7f93b40c3410 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:00.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93b406ff20 0x7f93b41f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:00.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 --2- 172.21.15.113:0/513375882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93b406dd50 0x7f93b41fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:00.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.359+0000 7f93bbe00640 1 -- 172.21.15.113:0/513375882 >> 172.21.15.113:0/513375882 conn(0x7f93b40734d0 msgr2=0x7f93b415b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:00.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.363+0000 7f93bbe00640 1 -- 172.21.15.113:0/513375882 shutdown_connections 2024-03-04T08:06:00.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:00.363+0000 7f93bbe00640 1 -- 172.21.15.113:0/513375882 wait complete. 2024-03-04T08:06:00.386 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 122, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 122, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6925": { "addr": "172.21.15.136:6835/2522634751", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 2522634751, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 2522634751, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 6925, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 23 }, "gid_7119": { "addr": "172.21.15.113:6833/3825257303", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 3825257303, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 3825257303, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7119, "incarnation": 114, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 25 }, "gid_7269": { "addr": "172.21.15.113:6837/1401330284", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1401330284, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1401330284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7269, "incarnation": 120, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:reconnect", "state_seq": 18 } }, "last_failure": 0, "last_failure_osd_epoch": 66, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:05:56.272511+0000", "qdb_cluster": [ 6925, 7119 ], "qdb_leader": 6925, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6925, "mds_1": 7119, "mds_2": 7269 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:06:02.387 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:06:02.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.523+0000 7f7d0da00640 1 Processor -- start 2024-03-04T08:06:02.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.523+0000 7f7d0da00640 1 -- start start 2024-03-04T08:06:02.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d0da00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7d080c8b00 0x7f7d081641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:02.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d0da00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d0815cc20 con 0x7f7d08070010 2024-03-04T08:06:02.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d0da00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d08152f00 con 0x7f7d08152b20 2024-03-04T08:06:02.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d0da00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d080c5320 con 0x7f7d080c8b00 2024-03-04T08:06:02.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d06000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7d08070010 0x7f7d0815c510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60288/0 (socket says 172.21.15.113:60288) 2024-03-04T08:06:02.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d06000640 1 -- 172.21.15.113:0/1908070029 learned_addr learned my addr 172.21.15.113:0/1908070029 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d07400640 1 --2- 172.21.15.113:0/1908070029 >> v2:172.21.15.136:3300/0 conn(0x7f7d080c8b00 0x7f7d081641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060642914 0 0) 0x7f7d0815cc20 con 0x7f7d08070010 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cec003530 con 0x7f7d08070010 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789484971 0 0) 0x7f7d08152f00 con 0x7f7d08152b20 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d0815cc20 con 0x7f7d08152b20 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1769701781 0 0) 0x7f7cec003530 con 0x7f7d08070010 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 >> v2:172.21.15.136:3300/0 conn(0x7f7d080c8b00 msgr2=0x7f7d081641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 --2- 172.21.15.113:0/1908070029 >> v2:172.21.15.136:3300/0 conn(0x7f7d080c8b00 0x7f7d081641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 >> v1:172.21.15.136:6789/0 conn(0x7f7d08152b20 legacy=0x7f7d0815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d080c8f00 con 0x7f7d08070010 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f7cf4001680 con 0x7f7d08070010 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 >> v1:172.21.15.113:6789/0 conn(0x7f7d08070010 legacy=0x7f7d0815c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7cec003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:02.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cec004810 0x7f7cec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7cec0051d0 0x7f7cec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d080c5320 con 0x7f7cec003b50 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec00c050 con 0x7f7cec004810 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec00c250 con 0x7f7cec0051d0 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d06a00640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7cec003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d06000640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cec004810 0x7f7cec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d07400640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7cec0051d0 0x7f7cec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d06a00640 1 -- 172.21.15.113:0/1908070029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7cec0051d0 msgr2=0x7f7cec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d06a00640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7cec0051d0 0x7f7cec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d06a00640 1 -- 172.21.15.113:0/1908070029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cec004810 msgr2=0x7f7cec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d06a00640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cec004810 0x7f7cec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d06a00640 1 -- 172.21.15.113:0/1908070029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec003530 con 0x7f7cec003b50 2024-03-04T08:06:02.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d06000640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7cec004810 0x7f7cec004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:06:02.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.527+0000 7f7d07400640 1 -- 172.21.15.113:0/1908070029 reap_dead start 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d06a00640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7cec003f50 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80018d0 tx=0x7f7cf8020930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f83e59d9aa09ba7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8001520 con 0x7f7cec003b50 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf80052f0 con 0x7f7cec003b50 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d05600640 1 -- 172.21.15.113:0/1908070029 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8003840 con 0x7f7cec003b50 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d0da00640 1 -- 172.21.15.113:0/1908070029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 msgr2=0x7f7cec003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d0da00640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7cec003f50 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80018d0 tx=0x7f7cf8020930 comp rx=0 tx=0).stop 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d0da00640 1 -- 172.21.15.113:0/1908070029 shutdown_connections 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d0da00640 1 --2- 172.21.15.113:0/1908070029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7cec003f50 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d0da00640 1 -- 172.21.15.113:0/1908070029 >> 172.21.15.113:0/1908070029 conn(0x7f7d080709a0 msgr2=0x7f7d08070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d0da00640 1 -- 172.21.15.113:0/1908070029 shutdown_connections 2024-03-04T08:06:02.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.531+0000 7f7d0da00640 1 -- 172.21.15.113:0/1908070029 wait complete. 2024-03-04T08:06:02.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d0da00640 1 Processor -- start 2024-03-04T08:06:02.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d0da00640 1 -- start start 2024-03-04T08:06:02.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d0da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7d080c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:02.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d0da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d080cabf0 0x7f7d0814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:02.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d0da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d08147dd0 0x7f7d08148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:02.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d0da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d080c92f0 con 0x7f7cec003b50 2024-03-04T08:06:02.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d0da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0806e180 con 0x7f7d080cabf0 2024-03-04T08:06:02.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d0da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d080c3810 con 0x7f7d08147dd0 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d06a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7d080c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d06a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7d080c5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32808/0 (socket says 172.21.15.113:32808) 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d06a00640 1 -- 172.21.15.113:0/3793348250 learned_addr learned my addr 172.21.15.113:0/3793348250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d06000640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d080cabf0 0x7f7d0814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d07400640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d08147dd0 0x7f7d08148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d06a00640 1 -- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d08147dd0 msgr2=0x7f7d08148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d06a00640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d08147dd0 0x7f7d08148250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d06a00640 1 -- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d080cabf0 msgr2=0x7f7d0814e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d06a00640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d080cabf0 0x7f7d0814e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d06a00640 1 -- 172.21.15.113:0/3793348250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08148790 con 0x7f7cec003b50 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7d06a00640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7d080c5ba0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8020760 tx=0x7f7cec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.535+0000 7f7cfea00640 1 -- 172.21.15.113:0/3793348250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf80054a0 con 0x7f7cec003b50 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.539+0000 7f7cfea00640 1 -- 172.21.15.113:0/3793348250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf80b9030 con 0x7f7cec003b50 2024-03-04T08:06:02.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.539+0000 7f7cfea00640 1 -- 172.21.15.113:0/3793348250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8001500 con 0x7f7cec003b50 2024-03-04T08:06:02.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.539+0000 7f7d0da00640 1 -- 172.21.15.113:0/3793348250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d08158be0 con 0x7f7cec003b50 2024-03-04T08:06:02.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.539+0000 7f7d0da00640 1 -- 172.21.15.113:0/3793348250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d080cb9f0 con 0x7f7cec003b50 2024-03-04T08:06:02.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.539+0000 7f7d0da00640 1 -- 172.21.15.113:0/3793348250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d08075c00 con 0x7f7cec003b50 2024-03-04T08:06:02.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.539+0000 7f7cfea00640 1 -- 172.21.15.113:0/3793348250 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7cf80b5430 con 0x7f7cec003b50 2024-03-04T08:06:02.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.539+0000 7f7cfea00640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ce8066f60 0x7f7ce8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:02.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.539+0000 7f7cfea00640 1 -- 172.21.15.113:0/3793348250 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f7cf81232f0 con 0x7f7cec003b50 2024-03-04T08:06:02.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.539+0000 7f7d06000640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ce8066f60 0x7f7ce8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:02.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.539+0000 7f7d06000640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ce8066f60 0x7f7ce8069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7d081efa30 tx=0x7f7cf4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:02.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.543+0000 7f7cfea00640 1 -- 172.21.15.113:0/3793348250 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7d08148790 con 0x7f7cec003b50 2024-03-04T08:06:02.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.735+0000 7f7d0da00640 1 -- 172.21.15.113:0/3793348250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d0806fdf0 con 0x7f7cec003b50 2024-03-04T08:06:02.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.735+0000 7f7cfea00640 1 -- 172.21.15.113:0/3793348250 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f7cf801a640 con 0x7f7cec003b50 2024-03-04T08:06:02.733 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:06:02.733 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":123,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:01.639489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7119,6925]},"id":1}]} 2024-03-04T08:06:02.733 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 123 2024-03-04T08:06:02.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 -- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ce8066f60 msgr2=0x7f7ce8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:02.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ce8066f60 0x7f7ce8069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7d081efa30 tx=0x7f7cf4009040 comp rx=0 tx=0).stop 2024-03-04T08:06:02.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 -- 172.21.15.113:0/3793348250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 msgr2=0x7f7d080c5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:02.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7d080c5ba0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8020760 tx=0x7f7cec005840 comp rx=0 tx=0).stop 2024-03-04T08:06:02.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 -- 172.21.15.113:0/3793348250 shutdown_connections 2024-03-04T08:06:02.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d08147dd0 0x7f7d08148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:02.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d080cabf0 0x7f7d0814e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:02.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7ce8066f60 0x7f7ce8069420 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:02.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 --2- 172.21.15.113:0/3793348250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cec003b50 0x7f7d080c5ba0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:02.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 -- 172.21.15.113:0/3793348250 >> 172.21.15.113:0/3793348250 conn(0x7f7d080709a0 msgr2=0x7f7d081e49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:02.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 -- 172.21.15.113:0/3793348250 shutdown_connections 2024-03-04T08:06:02.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:02.739+0000 7f7d0da00640 1 -- 172.21.15.113:0/3793348250 wait complete. 2024-03-04T08:06:02.763 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:06:04.764 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:06:04.885 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:06:04.888 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:06:04.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.903+0000 7fb654a00640 1 Processor -- start 2024-03-04T08:06:04.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.903+0000 7fb654a00640 1 -- start start 2024-03-04T08:06:04.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.903+0000 7fb654a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb65006de10 0x7fb650070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:04.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb654a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb65015b150 0x7fb65015b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:04.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb654a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6500c8da0 con 0x7fb650158390 2024-03-04T08:06:04.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb654a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb650158770 con 0x7fb65015b150 2024-03-04T08:06:04.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb654a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65006f770 con 0x7fb65006de10 2024-03-04T08:06:04.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb65015b150 0x7fb65015b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:04.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb65015b150 0x7fb65015b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32810/0 (socket says 172.21.15.113:32810) 2024-03-04T08:06:04.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64e800640 1 -- 172.21.15.113:0/1429743334 learned_addr learned my addr 172.21.15.113:0/1429743334 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:06:04.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb647e00640 1 --2- 172.21.15.113:0/1429743334 >> v2:172.21.15.136:3300/0 conn(0x7fb65006de10 0x7fb650070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:04.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64e800640 1 -- 172.21.15.113:0/1429743334 >> v2:172.21.15.136:3300/0 conn(0x7fb65006de10 msgr2=0x7fb650070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:06:04.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64e800640 1 --2- 172.21.15.113:0/1429743334 >> v2:172.21.15.136:3300/0 conn(0x7fb65006de10 0x7fb650070130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:04.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64e800640 1 -- 172.21.15.113:0/1429743334 >> v1:172.21.15.136:6789/0 conn(0x7fb650158390 legacy=0x7fb6500ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:04.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64e800640 1 -- 172.21.15.113:0/1429743334 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65006fa50 con 0x7fb65015b150 2024-03-04T08:06:04.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64e800640 1 --2- 172.21.15.113:0/1429743334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65015b150 0x7fb65015b530 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb6500cc1c0 tx=0x7fb64801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff697cdacf91da5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:04.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64d400640 1 -- 172.21.15.113:0/1429743334 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6480a5620 con 0x7fb65015b150 2024-03-04T08:06:04.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64d400640 1 -- 172.21.15.113:0/1429743334 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6480a57c0 con 0x7fb65015b150 2024-03-04T08:06:04.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.907+0000 7fb64d400640 1 -- 172.21.15.113:0/1429743334 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6480a5b90 con 0x7fb65015b150 2024-03-04T08:06:04.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 -- 172.21.15.113:0/1429743334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65015b150 msgr2=0x7fb65015b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:04.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 --2- 172.21.15.113:0/1429743334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65015b150 0x7fb65015b530 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fb6500cc1c0 tx=0x7fb64801c960 comp rx=0 tx=0).stop 2024-03-04T08:06:04.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 -- 172.21.15.113:0/1429743334 shutdown_connections 2024-03-04T08:06:04.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 --2- 172.21.15.113:0/1429743334 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65015b150 0x7fb65015b530 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:04.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 --2- 172.21.15.113:0/1429743334 >> v2:172.21.15.136:3300/0 conn(0x7fb65006de10 0x7fb650070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:04.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 -- 172.21.15.113:0/1429743334 >> 172.21.15.113:0/1429743334 conn(0x7fb650066db0 msgr2=0x7fb6500671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:04.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 -- 172.21.15.113:0/1429743334 shutdown_connections 2024-03-04T08:06:04.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 -- 172.21.15.113:0/1429743334 wait complete. 2024-03-04T08:06:04.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 Processor -- start 2024-03-04T08:06:04.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 -- start start 2024-03-04T08:06:04.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65006de10 0x7fb65014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:04.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb650158390 0x7fb6500cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:04.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65015b150 0x7fb6501f71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:04.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500b1070 con 0x7fb650158390 2024-03-04T08:06:04.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500b0ef0 con 0x7fb65006de10 2024-03-04T08:06:04.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb654a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500ce980 con 0x7fb65015b150 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb647e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb650158390 0x7fb6500cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb647e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb650158390 0x7fb6500cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32820/0 (socket says 172.21.15.113:32820) 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb647e00640 1 -- 172.21.15.113:0/20699753 learned_addr learned my addr 172.21.15.113:0/20699753 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb64e800640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65015b150 0x7fb6501f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb64de00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65006de10 0x7fb65014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb647e00640 1 -- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65015b150 msgr2=0x7fb6501f71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb647e00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65015b150 0x7fb6501f71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb647e00640 1 -- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65006de10 msgr2=0x7fb65014e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb647e00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65006de10 0x7fb65014e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.911+0000 7fb647e00640 1 -- 172.21.15.113:0/20699753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb650149790 con 0x7fb650158390 2024-03-04T08:06:04.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.915+0000 7fb647e00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb650158390 0x7fb6500cc4d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e930 tx=0x7fb63800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:04.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.915+0000 7fb646000640 1 -- 172.21.15.113:0/20699753 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638017070 con 0x7fb650158390 2024-03-04T08:06:04.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.915+0000 7fb646000640 1 -- 172.21.15.113:0/20699753 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63800c9a0 con 0x7fb650158390 2024-03-04T08:06:04.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.915+0000 7fb646000640 1 -- 172.21.15.113:0/20699753 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63800cd50 con 0x7fb650158390 2024-03-04T08:06:04.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.915+0000 7fb654a00640 1 -- 172.21.15.113:0/20699753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6500c33d0 con 0x7fb650158390 2024-03-04T08:06:04.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.915+0000 7fb654a00640 1 -- 172.21.15.113:0/20699753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6500c5050 con 0x7fb650158390 2024-03-04T08:06:04.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.915+0000 7fb654a00640 1 -- 172.21.15.113:0/20699753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb650075c00 con 0x7fb650158390 2024-03-04T08:06:04.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.915+0000 7fb646000640 1 -- 172.21.15.113:0/20699753 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb63801a070 con 0x7fb650158390 2024-03-04T08:06:04.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.915+0000 7fb646000640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb62c066f60 0x7fb62c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:04.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.915+0000 7fb646000640 1 -- 172.21.15.113:0/20699753 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7fb638085730 con 0x7fb650158390 2024-03-04T08:06:04.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.919+0000 7fb64de00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb62c066f60 0x7fb62c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:04.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.919+0000 7fb64de00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb62c066f60 0x7fb62c069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb640007fd0 tx=0x7fb640009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:04.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:04.923+0000 7fb646000640 1 -- 172.21.15.113:0/20699753 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb650149790 con 0x7fb650158390 2024-03-04T08:06:05.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.119+0000 7fb654a00640 1 -- 172.21.15.113:0/20699753 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6500b35d0 con 0x7fb650158390 2024-03-04T08:06:05.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.119+0000 7fb646000640 1 -- 172.21.15.113:0/20699753 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+5824 (secure 0 0 0) 0x7fb638055a00 con 0x7fb650158390 2024-03-04T08:06:05.117 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:06:05.117 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":123,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:01.639489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7119,6925]},"id":1}]} 2024-03-04T08:06:05.117 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 123 2024-03-04T08:06:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 -- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb62c066f60 msgr2=0x7fb62c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb62c066f60 0x7fb62c069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb640007fd0 tx=0x7fb640009040 comp rx=0 tx=0).stop 2024-03-04T08:06:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 -- 172.21.15.113:0/20699753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb650158390 msgr2=0x7fb6500cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb650158390 0x7fb6500cc4d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fb63800e930 tx=0x7fb63800edf0 comp rx=0 tx=0).stop 2024-03-04T08:06:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 -- 172.21.15.113:0/20699753 shutdown_connections 2024-03-04T08:06:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65015b150 0x7fb6501f71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb62c066f60 0x7fb62c069420 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb650158390 0x7fb6500cc4d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:05.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 --2- 172.21.15.113:0/20699753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65006de10 0x7fb65014e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:05.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 -- 172.21.15.113:0/20699753 >> 172.21.15.113:0/20699753 conn(0x7fb650066db0 msgr2=0x7fb650152690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:05.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 -- 172.21.15.113:0/20699753 shutdown_connections 2024-03-04T08:06:05.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:05.123+0000 7fb654a00640 1 -- 172.21.15.113:0/20699753 wait complete. 2024-03-04T08:06:05.138 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:06:07.139 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:06:07.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.295+0000 7fb19d400640 1 Processor -- start 2024-03-04T08:06:07.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.295+0000 7fb19d400640 1 -- start start 2024-03-04T08:06:07.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb19d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1980cd110 0x7fb1981664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:07.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb19d400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb198155330 con 0x7fb198154f50 2024-03-04T08:06:07.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb19d400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1980c8180 con 0x7fb1980671b0 2024-03-04T08:06:07.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb19d400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19815e150 con 0x7fb1980cd110 2024-03-04T08:06:07.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb196800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb198154f50 0x7fb1980c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53760/0 (socket says 172.21.15.113:53760) 2024-03-04T08:06:07.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb196800640 1 -- 172.21.15.113:0/2276511207 learned_addr learned my addr 172.21.15.113:0/2276511207 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:06:07.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb197200640 1 --2- 172.21.15.113:0/2276511207 >> v2:172.21.15.113:3300/0 conn(0x7fb1980cd110 0x7fb1981664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:07.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb197200640 1 -- 172.21.15.113:0/2276511207 >> v1:172.21.15.136:6789/0 conn(0x7fb1980671b0 legacy=0x7fb198071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:07.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb197200640 1 -- 172.21.15.113:0/2276511207 >> v1:172.21.15.113:6789/0 conn(0x7fb198154f50 legacy=0x7fb1980c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:07.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb197200640 1 -- 172.21.15.113:0/2276511207 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19815e3a0 con 0x7fb1980cd110 2024-03-04T08:06:07.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.299+0000 7fb195400640 1 -- 172.21.15.113:0/2276511207 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725866896 0 0) 0x7fb188001120 con 0x7fb198154f50 2024-03-04T08:06:07.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb197200640 1 --2- 172.21.15.113:0/2276511207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980cd110 0x7fb1981664d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb190009d90 tx=0x7fb19001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d477873498afea server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:07.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb195400640 1 -- 172.21.15.113:0/2276511207 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1900b4070 con 0x7fb1980cd110 2024-03-04T08:06:07.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb195400640 1 -- 172.21.15.113:0/2276511207 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1900af600 con 0x7fb1980cd110 2024-03-04T08:06:07.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb195400640 1 -- 172.21.15.113:0/2276511207 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1900af9d0 con 0x7fb1980cd110 2024-03-04T08:06:07.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 -- 172.21.15.113:0/2276511207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980cd110 msgr2=0x7fb1981664d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:07.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 --2- 172.21.15.113:0/2276511207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980cd110 0x7fb1981664d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb190009d90 tx=0x7fb19001f150 comp rx=0 tx=0).stop 2024-03-04T08:06:07.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 -- 172.21.15.113:0/2276511207 shutdown_connections 2024-03-04T08:06:07.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 --2- 172.21.15.113:0/2276511207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980cd110 0x7fb1981664d0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:07.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 -- 172.21.15.113:0/2276511207 >> 172.21.15.113:0/2276511207 conn(0x7fb198067a40 msgr2=0x7fb19806ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:07.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 -- 172.21.15.113:0/2276511207 shutdown_connections 2024-03-04T08:06:07.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 -- 172.21.15.113:0/2276511207 wait complete. 2024-03-04T08:06:07.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 Processor -- start 2024-03-04T08:06:07.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 -- start start 2024-03-04T08:06:07.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980671b0 0x7fb198071c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:07.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1980cd110 0x7fb1981e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb198154f50 0x7fb1981f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19815ec40 con 0x7fb1980671b0 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19815e960 con 0x7fb1980cd110 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.303+0000 7fb19d400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1980c7e20 con 0x7fb198154f50 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb196800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980671b0 0x7fb198071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb196800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980671b0 0x7fb198071c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40716/0 (socket says 172.21.15.113:40716) 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb196800640 1 -- 172.21.15.113:0/3675974786 learned_addr learned my addr 172.21.15.113:0/3675974786 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb195e00640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1980cd110 0x7fb1981e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb197200640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb198154f50 0x7fb1981f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb196800640 1 -- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb198154f50 msgr2=0x7fb1981f4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb196800640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb198154f50 0x7fb1981f4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb196800640 1 -- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1980cd110 msgr2=0x7fb1981e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb196800640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1980cd110 0x7fb1981e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:07.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb196800640 1 -- 172.21.15.113:0/3675974786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1980c3630 con 0x7fb1980671b0 2024-03-04T08:06:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb195e00640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1980cd110 0x7fb1981e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:06:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb196800640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980671b0 0x7fb198071c10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb18800c440 tx=0x7fb18800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb18ea00640 1 -- 172.21.15.113:0/3675974786 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188018070 con 0x7fb1980671b0 2024-03-04T08:06:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb18ea00640 1 -- 172.21.15.113:0/3675974786 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1880074e0 con 0x7fb1980671b0 2024-03-04T08:06:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb18ea00640 1 -- 172.21.15.113:0/3675974786 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb188004a10 con 0x7fb1980671b0 2024-03-04T08:06:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb19d400640 1 -- 172.21.15.113:0/3675974786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1981577a0 con 0x7fb1980671b0 2024-03-04T08:06:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb19d400640 1 -- 172.21.15.113:0/3675974786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb198157a10 con 0x7fb1980671b0 2024-03-04T08:06:07.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb19d400640 1 -- 172.21.15.113:0/3675974786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb198075c00 con 0x7fb1980671b0 2024-03-04T08:06:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb18ea00640 1 -- 172.21.15.113:0/3675974786 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb188004bb0 con 0x7fb1980671b0 2024-03-04T08:06:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb18ea00640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb178066f90 0x7fb178069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.307+0000 7fb18ea00640 1 -- 172.21.15.113:0/3675974786 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7fb1880848b0 con 0x7fb1980671b0 2024-03-04T08:06:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.311+0000 7fb195e00640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb178066f90 0x7fb178069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:07.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.311+0000 7fb195e00640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb178066f90 0x7fb178069450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb19814e880 tx=0x7fb184009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:07.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.315+0000 7fb18ea00640 1 -- 172.21.15.113:0/3675974786 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb198155330 con 0x7fb1980671b0 2024-03-04T08:06:07.337 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at e3b7bc5bce8 v0.67 2024-03-04T08:06:07.353 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.67 2024-03-04T08:06:07.374 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 68 -eq 48 ] 2024-03-04T08:06:07.374 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.68 2024-03-04T08:06:07.375 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.68 2024-03-04T08:06:07.375 INFO:tasks.workunit.client.0.smithi113.stdout:v0.68 2024-03-04T08:06:07.375 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.68 2024-03-04T08:06:07.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.531+0000 7fb19d400640 1 -- 172.21.15.113:0/3675974786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1980727b0 con 0x7fb1980671b0 2024-03-04T08:06:07.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.531+0000 7fb18ea00640 1 -- 172.21.15.113:0/3675974786 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+5824 (secure 0 0 0) 0x7fb1880551a0 con 0x7fb1980671b0 2024-03-04T08:06:07.530 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:06:07.530 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":123,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":123,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:01.639489+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7119,6925]},"id":1}]} 2024-03-04T08:06:07.530 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 123 2024-03-04T08:06:07.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 -- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb178066f90 msgr2=0x7fb178069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:07.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb178066f90 0x7fb178069450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb19814e880 tx=0x7fb184009040 comp rx=0 tx=0).stop 2024-03-04T08:06:07.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 -- 172.21.15.113:0/3675974786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980671b0 msgr2=0x7fb198071c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:07.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980671b0 0x7fb198071c10 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb18800c440 tx=0x7fb18800c900 comp rx=0 tx=0).stop 2024-03-04T08:06:07.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 -- 172.21.15.113:0/3675974786 shutdown_connections 2024-03-04T08:06:07.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb198154f50 0x7fb1981f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:07.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1980cd110 0x7fb1981e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:07.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb178066f90 0x7fb178069450 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:07.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 --2- 172.21.15.113:0/3675974786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1980671b0 0x7fb198071c10 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:07.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 -- 172.21.15.113:0/3675974786 >> 172.21.15.113:0/3675974786 conn(0x7fb198067a40 msgr2=0x7fb19814fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:07.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 -- 172.21.15.113:0/3675974786 shutdown_connections 2024-03-04T08:06:07.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:07.535+0000 7fb19d400640 1 -- 172.21.15.113:0/3675974786 wait complete. 2024-03-04T08:06:07.551 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:06:09.553 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:06:09.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff495200640 1 Processor -- start 2024-03-04T08:06:09.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff495200640 1 -- start start 2024-03-04T08:06:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff495200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff49006ff20 0x7ff49006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff495200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff4900ce120 0x7ff4900ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff495200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4900ca0e0 con 0x7ff490158010 2024-03-04T08:06:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff495200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49006f450 con 0x7ff49006ff20 2024-03-04T08:06:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff495200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49015ad80 con 0x7ff4900ce120 2024-03-04T08:06:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff48d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff49006ff20 0x7ff49006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff48d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff49006ff20 0x7ff49006dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40720/0 (socket says 172.21.15.113:40720) 2024-03-04T08:06:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff48d400640 1 -- 172.21.15.113:0/1994983164 learned_addr learned my addr 172.21.15.113:0/1994983164 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:06:09.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.702+0000 7ff48e800640 1 --2- 172.21.15.113:0/1994983164 >> v2:172.21.15.136:3300/0 conn(0x7ff4900ce120 0x7ff4900ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:09.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.706+0000 7ff48d400640 1 -- 172.21.15.113:0/1994983164 >> v2:172.21.15.136:3300/0 conn(0x7ff4900ce120 msgr2=0x7ff4900ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:06:09.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.706+0000 7ff48d400640 1 --2- 172.21.15.113:0/1994983164 >> v2:172.21.15.136:3300/0 conn(0x7ff4900ce120 0x7ff4900ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:09.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.706+0000 7ff48d400640 1 -- 172.21.15.113:0/1994983164 >> v1:172.21.15.136:6789/0 conn(0x7ff490158010 legacy=0x7ff49015b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:09.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.706+0000 7ff48d400640 1 -- 172.21.15.113:0/1994983164 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49015aa30 con 0x7ff49006ff20 2024-03-04T08:06:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.706+0000 7ff48d400640 1 --2- 172.21.15.113:0/1994983164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 0x7ff49006dd50 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff4900cbc40 tx=0x7ff47c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5076fabe24eb6b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.706+0000 7ff48ca00640 1 -- 172.21.15.113:0/1994983164 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0b0070 con 0x7ff49006ff20 2024-03-04T08:06:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.706+0000 7ff48ca00640 1 -- 172.21.15.113:0/1994983164 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47c0ab600 con 0x7ff49006ff20 2024-03-04T08:06:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.706+0000 7ff48ca00640 1 -- 172.21.15.113:0/1994983164 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0ab9d0 con 0x7ff49006ff20 2024-03-04T08:06:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.706+0000 7ff495200640 1 -- 172.21.15.113:0/1994983164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 msgr2=0x7ff49006dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.706+0000 7ff495200640 1 --2- 172.21.15.113:0/1994983164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 0x7ff49006dd50 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff4900cbc40 tx=0x7ff47c01c520 comp rx=0 tx=0).stop 2024-03-04T08:06:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.710+0000 7ff495200640 1 -- 172.21.15.113:0/1994983164 shutdown_connections 2024-03-04T08:06:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.710+0000 7ff495200640 1 --2- 172.21.15.113:0/1994983164 >> v2:172.21.15.136:3300/0 conn(0x7ff4900ce120 0x7ff4900ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.710+0000 7ff495200640 1 --2- 172.21.15.113:0/1994983164 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 0x7ff49006dd50 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.710+0000 7ff495200640 1 -- 172.21.15.113:0/1994983164 >> 172.21.15.113:0/1994983164 conn(0x7ff4900734d0 msgr2=0x7ff490066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.710+0000 7ff495200640 1 -- 172.21.15.113:0/1994983164 shutdown_connections 2024-03-04T08:06:09.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.710+0000 7ff495200640 1 -- 172.21.15.113:0/1994983164 wait complete. 2024-03-04T08:06:09.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.710+0000 7ff495200640 1 Processor -- start 2024-03-04T08:06:09.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.710+0000 7ff495200640 1 -- start start 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff495200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 0x7ff4901444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff495200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4900ce120 0x7ff4901f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff495200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff490158010 0x7ff490159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff495200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901583f0 con 0x7ff49006ff20 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff495200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49015b920 con 0x7ff4900ce120 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff495200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49006c170 con 0x7ff490158010 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 0x7ff4901444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 0x7ff4901444e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40724/0 (socket says 172.21.15.113:40724) 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48de00640 1 -- 172.21.15.113:0/1925597422 learned_addr learned my addr 172.21.15.113:0/1925597422 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48d400640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4900ce120 0x7ff4901f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48e800640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff490158010 0x7ff490159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:09.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48de00640 1 -- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff490158010 msgr2=0x7ff490159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48de00640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff490158010 0x7ff490159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48de00640 1 -- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4900ce120 msgr2=0x7ff4901f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48de00640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4900ce120 0x7ff4901f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48de00640 1 -- 172.21.15.113:0/1925597422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff490158850 con 0x7ff49006ff20 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48d400640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4900ce120 0x7ff4901f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff48de00640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 0x7ff4901444e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c450 tx=0x7ff48000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff486000640 1 -- 172.21.15.113:0/1925597422 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480016020 con 0x7ff49006ff20 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff486000640 1 -- 172.21.15.113:0/1925597422 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4800052d0 con 0x7ff49006ff20 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff495200640 1 -- 172.21.15.113:0/1925597422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4901f33f0 con 0x7ff49006ff20 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff486000640 1 -- 172.21.15.113:0/1925597422 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480007260 con 0x7ff49006ff20 2024-03-04T08:06:09.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff495200640 1 -- 172.21.15.113:0/1925597422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4901f3990 con 0x7ff49006ff20 2024-03-04T08:06:09.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff495200640 1 -- 172.21.15.113:0/1925597422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff490075c00 con 0x7ff49006ff20 2024-03-04T08:06:09.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.714+0000 7ff486000640 1 -- 172.21.15.113:0/1925597422 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff480007400 con 0x7ff49006ff20 2024-03-04T08:06:09.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.718+0000 7ff486000640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff470066f60 0x7ff470069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:09.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.718+0000 7ff486000640 1 -- 172.21.15.113:0/1925597422 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7ff480084ff0 con 0x7ff49006ff20 2024-03-04T08:06:09.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.718+0000 7ff48d400640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff470066f60 0x7ff470069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:09.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.718+0000 7ff48d400640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff470066f60 0x7ff470069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0ad000 tx=0x7ff47c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:09.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.722+0000 7ff486000640 1 -- 172.21.15.113:0/1925597422 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff480020020 con 0x7ff49006ff20 2024-03-04T08:06:09.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.906+0000 7ff495200640 1 -- 172.21.15.113:0/1925597422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff490154ec0 con 0x7ff49006ff20 2024-03-04T08:06:09.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.906+0000 7ff486000640 1 -- 172.21.15.113:0/1925597422 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+5829 (secure 0 0 0) 0x7ff480005480 con 0x7ff49006ff20 2024-03-04T08:06:09.906 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:06:09.906 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":124,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:08.199463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[6925,7119,7269]},"id":1}]} 2024-03-04T08:06:09.906 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 124 2024-03-04T08:06:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 -- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff470066f60 msgr2=0x7ff470069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff470066f60 0x7ff470069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0ad000 tx=0x7ff47c008040 comp rx=0 tx=0).stop 2024-03-04T08:06:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 -- 172.21.15.113:0/1925597422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 msgr2=0x7ff4901444e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 0x7ff4901444e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff48000c450 tx=0x7ff48000c910 comp rx=0 tx=0).stop 2024-03-04T08:06:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 -- 172.21.15.113:0/1925597422 shutdown_connections 2024-03-04T08:06:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff490158010 0x7ff490159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:09.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4900ce120 0x7ff4901f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff470066f60 0x7ff470069420 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 --2- 172.21.15.113:0/1925597422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff49006ff20 0x7ff4901444e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 -- 172.21.15.113:0/1925597422 >> 172.21.15.113:0/1925597422 conn(0x7ff4900734d0 msgr2=0x7ff490156080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 -- 172.21.15.113:0/1925597422 shutdown_connections 2024-03-04T08:06:09.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:09.910+0000 7ff495200640 1 -- 172.21.15.113:0/1925597422 wait complete. 2024-03-04T08:06:09.927 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=2, replacing gid=6807 2024-03-04T08:06:09.927 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.a 2024-03-04T08:06:10.928 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-03-04T08:06:10.929 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-04T08:06:10.929 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-04T08:06:10.933 INFO:tasks.ceph.mds.a:Started 2024-03-04T08:06:10.933 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:06:11.020 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-03-04T08:06:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.078+0000 7fb9b0000640 1 Processor -- start 2024-03-04T08:06:11.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.078+0000 7fb9b0000640 1 -- start start 2024-03-04T08:06:11.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.078+0000 7fb9b0000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9a80ce2d0 0x7fb9a815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:11.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.078+0000 7fb9b0000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb9a806de80 0x7fb9a8154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:11.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.078+0000 7fb9b0000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a806c210 con 0x7fb9a80ce6b0 2024-03-04T08:06:11.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.078+0000 7fb9b0000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a815bbf0 con 0x7fb9a80ce2d0 2024-03-04T08:06:11.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.078+0000 7fb9b0000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a806eb80 con 0x7fb9a806de80 2024-03-04T08:06:11.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9ad600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9a80ce2d0 0x7fb9a815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:11.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9ad600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9a80ce2d0 0x7fb9a815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40742/0 (socket says 172.21.15.113:40742) 2024-03-04T08:06:11.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9ad600640 1 -- 172.21.15.113:0/1169995749 learned_addr learned my addr 172.21.15.113:0/1169995749 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:06:11.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9ae000640 1 --2- 172.21.15.113:0/1169995749 >> v2:172.21.15.136:3300/0 conn(0x7fb9a806de80 0x7fb9a8154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:11.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9ad600640 1 -- 172.21.15.113:0/1169995749 >> v2:172.21.15.136:3300/0 conn(0x7fb9a806de80 msgr2=0x7fb9a8154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:06:11.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9ad600640 1 --2- 172.21.15.113:0/1169995749 >> v2:172.21.15.136:3300/0 conn(0x7fb9a806de80 0x7fb9a8154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9ad600640 1 -- 172.21.15.113:0/1169995749 >> v1:172.21.15.136:6789/0 conn(0x7fb9a80ce6b0 legacy=0x7fb9a806f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:11.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9ad600640 1 -- 172.21.15.113:0/1169995749 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a806c4f0 con 0x7fb9a80ce2d0 2024-03-04T08:06:11.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9ad600640 1 --2- 172.21.15.113:0/1169995749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a80ce2d0 0x7fb9a815b6b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8072830 tx=0x7fb99c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab53f47a7c6217d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9a7e00640 1 -- 172.21.15.113:0/1169995749 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0b5070 con 0x7fb9a80ce2d0 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9a7e00640 1 -- 172.21.15.113:0/1169995749 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c0a8a10 con 0x7fb9a80ce2d0 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.082+0000 7fb9a7e00640 1 -- 172.21.15.113:0/1169995749 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0a8de0 con 0x7fb9a80ce2d0 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 -- 172.21.15.113:0/1169995749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a80ce2d0 msgr2=0x7fb9a815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 --2- 172.21.15.113:0/1169995749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a80ce2d0 0x7fb9a815b6b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8072830 tx=0x7fb99c01ed10 comp rx=0 tx=0).stop 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 -- 172.21.15.113:0/1169995749 shutdown_connections 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 --2- 172.21.15.113:0/1169995749 >> v2:172.21.15.136:3300/0 conn(0x7fb9a806de80 0x7fb9a8154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 --2- 172.21.15.113:0/1169995749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a80ce2d0 0x7fb9a815b6b0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 -- 172.21.15.113:0/1169995749 >> 172.21.15.113:0/1169995749 conn(0x7fb9a80709a0 msgr2=0x7fb9a8070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 -- 172.21.15.113:0/1169995749 shutdown_connections 2024-03-04T08:06:11.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 -- 172.21.15.113:0/1169995749 wait complete. 2024-03-04T08:06:11.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 Processor -- start 2024-03-04T08:06:11.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 -- start start 2024-03-04T08:06:11.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a806de80 0x7fb9a8143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:11.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9a80ce2d0 0x7fb9a81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9a80ce6b0 0x7fb9a815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a806c7d0 con 0x7fb9a806de80 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a806e810 con 0x7fb9a80ce6b0 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9b0000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a806f720 con 0x7fb9a80ce2d0 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9ad600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a806de80 0x7fb9a8143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9ad600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a806de80 0x7fb9a8143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40748/0 (socket says 172.21.15.113:40748) 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9ad600640 1 -- 172.21.15.113:0/3389385642 learned_addr learned my addr 172.21.15.113:0/3389385642 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9acc00640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9a80ce2d0 0x7fb9a81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9ae000640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9a80ce6b0 0x7fb9a815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9ad600640 1 -- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9a80ce2d0 msgr2=0x7fb9a81555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9ad600640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9a80ce2d0 0x7fb9a81555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9ad600640 1 -- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9a80ce6b0 msgr2=0x7fb9a815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9ad600640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9a80ce6b0 0x7fb9a815ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.086+0000 7fb9ad600640 1 -- 172.21.15.113:0/3389385642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a8144430 con 0x7fb9a806de80 2024-03-04T08:06:11.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9ad600640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a806de80 0x7fb9a8143ef0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb99c01b920 tx=0x7fb99c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:11.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9a5600640 1 -- 172.21.15.113:0/3389385642 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0b5070 con 0x7fb9a806de80 2024-03-04T08:06:11.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9a5600640 1 -- 172.21.15.113:0/3389385642 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c0b0a50 con 0x7fb9a806de80 2024-03-04T08:06:11.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9a5600640 1 -- 172.21.15.113:0/3389385642 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0b0e40 con 0x7fb9a806de80 2024-03-04T08:06:11.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9b0000640 1 -- 172.21.15.113:0/3389385642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a8155e50 con 0x7fb9a806de80 2024-03-04T08:06:11.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9b0000640 1 -- 172.21.15.113:0/3389385642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a8156410 con 0x7fb9a806de80 2024-03-04T08:06:11.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9b0000640 1 -- 172.21.15.113:0/3389385642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a806cb40 con 0x7fb9a806de80 2024-03-04T08:06:11.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9a5600640 1 -- 172.21.15.113:0/3389385642 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb99c0c2060 con 0x7fb9a806de80 2024-03-04T08:06:11.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9a5600640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb988066f60 0x7fb988069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:06:11.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9a5600640 1 -- 172.21.15.113:0/3389385642 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7fb99c122830 con 0x7fb9a806de80 2024-03-04T08:06:11.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.090+0000 7fb9acc00640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb988066f60 0x7fb988069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:06:11.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.094+0000 7fb9acc00640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb988066f60 0x7fb988069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb9a815b8e0 tx=0x7fb994009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:06:11.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.094+0000 7fb9a5600640 1 -- 172.21.15.113:0/3389385642 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb99c018870 con 0x7fb9a806de80 2024-03-04T08:06:11.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.286+0000 7fb9b0000640 1 -- 172.21.15.113:0/3389385642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9a806fdf0 con 0x7fb9a806de80 2024-03-04T08:06:11.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.286+0000 7fb9a5600640 1 -- 172.21.15.113:0/3389385642 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+6624 (secure 0 0 0) 0x7fb99c0f3120 con 0x7fb9a806de80 2024-03-04T08:06:11.286 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:06:11.286 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:08.199463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7269,7119,6925]},"id":1}]} 2024-03-04T08:06:11.287 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 125 2024-03-04T08:06:11.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 -- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb988066f60 msgr2=0x7fb988069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:11.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb988066f60 0x7fb988069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb9a815b8e0 tx=0x7fb994009040 comp rx=0 tx=0).stop 2024-03-04T08:06:11.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 -- 172.21.15.113:0/3389385642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a806de80 msgr2=0x7fb9a8143ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:06:11.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a806de80 0x7fb9a8143ef0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb99c01b920 tx=0x7fb99c01bab0 comp rx=0 tx=0).stop 2024-03-04T08:06:11.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 -- 172.21.15.113:0/3389385642 shutdown_connections 2024-03-04T08:06:11.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9a80ce6b0 0x7fb9a815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:11.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9a80ce2d0 0x7fb9a81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:11.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb988066f60 0x7fb988069420 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:11.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 --2- 172.21.15.113:0/3389385642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9a806de80 0x7fb9a8143ef0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:06:11.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 -- 172.21.15.113:0/3389385642 >> 172.21.15.113:0/3389385642 conn(0x7fb9a80709a0 msgr2=0x7fb9a806f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:06:11.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 -- 172.21.15.113:0/3389385642 shutdown_connections 2024-03-04T08:06:11.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:06:11.290+0000 7fb9b0000640 1 -- 172.21.15.113:0/3389385642 wait complete. 2024-03-04T08:06:11.307 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-03-04T08:06:11.308 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 62 secs before thrashing 2024-03-04T08:06:22.954 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 7% (17/241) Updating files: 8% (20/241) Updating files: 9% (22/241) Updating files: 10% (25/241) Updating files: 11% (27/241) Updating files: 12% (29/241) Updating files: 13% (32/241) Updating files: 14% (34/241) Updating files: 15% (37/241) Updating files: 16% (39/241) Updating files: 17% (41/241) Updating files: 18% (44/241) Updating files: 19% (46/241) Updating files: 20% (49/241) Updating files: 21% (51/241) Updating files: 22% (54/241) Updating files: 23% (56/241) Updating files: 24% (58/241) Updating files: 25% (61/241) Updating files: 26% (63/241) Updating files: 27% (66/241) Updating files: 28% (68/241) Updating files: 29% (70/241) Updating files: 30% (73/241) Updating files: 31% (75/241) Updating files: 31% (77/241) Updating files: 32% (78/241) Updating files: 33% (80/241) Updating files: 34% (82/241) Updating files: 35% (85/241) Updating files: 36% (87/241) Updating files: 37% (90/241) Updating files: 38% (92/241) Updating files: 39% (94/241) Updating files: 40% (97/241) Updating files: 41% (99/241) Updating files: 42% (102/241) Updating files: 43% (104/241) Updating files: 44% (107/241) Updating files: 45% (109/241) Updating files: 46% (111/241) Updating files: 47% (114/241) Updating files: 48% (116/241) Updating files: 49% (119/241) Updating files: 50% (121/241) Updating files: 51% (123/241) Updating files: 52% (126/241) Updating files: 53% (128/241) Updating files: 54% (131/241) Updating files: 55% (133/241) Updating files: 56% (135/241) Updating files: 56% (136/241) Updating files: 57% (138/241) Updating files: 58% (140/241) Updating files: 59% (143/241) Updating files: 60% (145/241) Updating files: 61% (148/241) Updating files: 62% (150/241) Updating files: 63% (152/241) Updating files: 64% (155/241) Updating files: 65% (157/241) Updating files: 66% (160/241) Updating files: 67% (162/241) Updating files: 68% (164/241) Updating files: 69% (167/241) Updating files: 70% (169/241) Updating files: 71% (172/241) Updating files: 72% (174/241) Updating files: 73% (176/241) Updating files: 74% (179/241) Updating files: 75% (181/241) Updating files: 76% (184/241) Updating files: 77% (186/241) Updating files: 78% (188/241) Updating files: 79% (191/241) Updating files: 80% (193/241) Updating files: 81% (196/241) Updating files: 82% (198/241) Updating files: 83% (201/241) Updating files: 84% (203/241) Updating files: 85% (205/241) Updating files: 86% (208/241) Updating files: 87% (210/241) Updating files: 88% (213/241) Updating files: 89% (215/241) Updating files: 90% (217/241) Updating files: 91% (220/241) Updating files: 92% (222/241) Updating files: 93% (225/241) Updating files: 94% (227/241) Updating files: 95% (229/241) Updating files: 96% (232/241) Updating files: 97% (234/241) Updating files: 98% (237/241) Updating files: 99% (239/241) Updating files: 100% (241/241) Updating files: 100% (241/241), done. 2024-03-04T08:06:23.059 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at b4cf0f2574b v0.68 2024-03-04T08:06:23.067 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.68 2024-03-04T08:06:23.361 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 69 -eq 48 ] 2024-03-04T08:06:23.362 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.69 2024-03-04T08:06:23.362 INFO:tasks.workunit.client.0.smithi113.stdout:v0.69 2024-03-04T08:06:23.363 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.69 2024-03-04T08:06:23.363 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.69 2024-03-04T08:06:29.948 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 49% (76/155) Updating files: 50% (78/155) Updating files: 51% (80/155) Updating files: 52% (81/155) Updating files: 53% (83/155) Updating files: 54% (84/155) Updating files: 55% (86/155) Updating files: 56% (87/155) Updating files: 57% (89/155) Updating files: 58% (90/155) Updating files: 59% (92/155) Updating files: 60% (93/155) Updating files: 61% (95/155) Updating files: 62% (97/155) Updating files: 63% (98/155) Updating files: 64% (100/155) Updating files: 65% (101/155) Updating files: 66% (103/155) Updating files: 67% (104/155) Updating files: 68% (106/155) Updating files: 69% (107/155) Updating files: 70% (109/155) Updating files: 71% (111/155) Updating files: 72% (112/155) Updating files: 73% (114/155) Updating files: 74% (115/155) Updating files: 75% (117/155) Updating files: 76% (118/155) Updating files: 77% (120/155) Updating files: 78% (121/155) Updating files: 79% (123/155) Updating files: 80% (124/155) Updating files: 81% (126/155) Updating files: 82% (128/155) Updating files: 83% (129/155) Updating files: 84% (131/155) Updating files: 85% (132/155) Updating files: 86% (134/155) Updating files: 87% (135/155) Updating files: 88% (137/155) Updating files: 89% (138/155) Updating files: 90% (140/155) Updating files: 91% (142/155) Updating files: 92% (143/155) Updating files: 93% (145/155) Updating files: 94% (146/155) Updating files: 95% (148/155) Updating files: 96% (149/155) Updating files: 97% (151/155) Updating files: 98% (152/155) Updating files: 99% (154/155) Updating files: 100% (155/155) Updating files: 100% (155/155), done. 2024-03-04T08:06:30.009 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 6ca6f2f9f75 v0.69 2024-03-04T08:06:30.023 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.69 2024-03-04T08:06:30.203 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 70 -eq 48 ] 2024-03-04T08:06:30.203 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.70 2024-03-04T08:06:30.203 INFO:tasks.workunit.client.0.smithi113.stdout:v0.70 2024-03-04T08:06:30.203 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.70 2024-03-04T08:06:30.204 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.70 2024-03-04T08:06:32.778 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 54% (24/44) Updating files: 56% (25/44) Updating files: 59% (26/44) Updating files: 61% (27/44) Updating files: 63% (28/44) Updating files: 65% (29/44) Updating files: 68% (30/44) Updating files: 70% (31/44) Updating files: 72% (32/44) Updating files: 75% (33/44) Updating files: 77% (34/44) Updating files: 79% (35/44) Updating files: 81% (36/44) Updating files: 84% (37/44) Updating files: 86% (38/44) Updating files: 88% (39/44) Updating files: 90% (40/44) Updating files: 93% (41/44) Updating files: 95% (42/44) Updating files: 97% (43/44) Updating files: 100% (44/44) Updating files: 100% (44/44), done. 2024-03-04T08:06:32.853 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at e3bb0656d92 v0.70 2024-03-04T08:06:32.855 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.70 2024-03-04T08:06:32.992 INFO:tasks.workunit.client.0.smithi113.stdout:v0.71 2024-03-04T08:06:32.992 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 71 -eq 48 ] 2024-03-04T08:06:32.993 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.71 2024-03-04T08:06:32.993 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.71 2024-03-04T08:06:32.993 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.71 2024-03-04T08:06:35.013 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:06:35.017 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:06:41.907 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 7% (31/405) Updating files: 8% (33/405) Updating files: 9% (37/405) Updating files: 10% (41/405) Updating files: 11% (45/405) Updating files: 12% (49/405) Updating files: 13% (53/405) Updating files: 14% (57/405) Updating files: 15% (61/405) Updating files: 16% (65/405) Updating files: 16% (68/405) Updating files: 17% (69/405) Updating files: 18% (73/405) Updating files: 19% (77/405) Updating files: 20% (81/405) Updating files: 21% (86/405) Updating files: 22% (90/405) Updating files: 23% (94/405) Updating files: 24% (98/405) Updating files: 25% (102/405) Updating files: 26% (106/405) Updating files: 27% (110/405) Updating files: 28% (114/405) Updating files: 29% (118/405) Updating files: 30% (122/405) Updating files: 31% (126/405) Updating files: 32% (130/405) Updating files: 33% (134/405) Updating files: 34% (138/405) Updating files: 35% (142/405) Updating files: 35% (144/405) Updating files: 36% (146/405) Updating files: 37% (150/405) Updating files: 38% (154/405) Updating files: 39% (158/405) Updating files: 40% (162/405) Updating files: 41% (167/405) Updating files: 42% (171/405) Updating files: 43% (175/405) Updating files: 44% (179/405) Updating files: 45% (183/405) Updating files: 46% (187/405) Updating files: 47% (191/405) Updating files: 48% (195/405) Updating files: 49% (199/405) Updating files: 50% (203/405) Updating files: 51% (207/405) Updating files: 52% (211/405) Updating files: 53% (215/405) Updating files: 54% (219/405) Updating files: 55% (223/405) Updating files: 56% (227/405) Updating files: 57% (231/405) Updating files: 58% (235/405) Updating files: 59% (239/405) Updating files: 59% (242/405) Updating files: 60% (243/405) Updating files: 61% (248/405) Updating files: 62% (252/405) Updating files: 63% (256/405) Updating files: 64% (260/405) Updating files: 65% (264/405) Updating files: 66% (268/405) Updating files: 67% (272/405) Updating files: 68% (276/405) Updating files: 69% (280/405) Updating files: 70% (284/405) Updating files: 71% (288/405) Updating files: 72% (292/405) Updating files: 73% (296/405) Updating files: 74% (300/405) Updating files: 75% (304/405) Updating files: 76% (308/405) Updating files: 77% (312/405) Updating files: 77% (315/405) Updating files: 78% (316/405) Updating files: 79% (320/405) Updating files: 80% (324/405) Updating files: 81% (329/405) Updating files: 82% (333/405) Updating files: 83% (337/405) Updating files: 84% (341/405) Updating files: 85% (345/405) Updating files: 86% (349/405) Updating files: 87% (353/405) Updating files: 88% (357/405) Updating files: 88% (359/405) Updating files: 89% (361/405) Updating files: 90% (365/405) Updating files: 91% (369/405) Updating files: 92% (373/405) Updating files: 93% (377/405) Updating files: 94% (381/405) Updating files: 95% (385/405) Updating files: 95% (387/405) Updating files: 96% (389/405) Updating files: 97% (393/405) Updating files: 98% (397/405) Updating files: 99% (401/405) Updating files: 100% (405/405) Updating files: 100% (405/405), done. 2024-03-04T08:06:41.962 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at e509cb1e69c v0.71 2024-03-04T08:06:41.971 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.71 2024-03-04T08:06:42.129 INFO:tasks.workunit.client.0.smithi113.stdout:v0.72 2024-03-04T08:06:42.129 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 72 -eq 48 ] 2024-03-04T08:06:42.129 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.72 2024-03-04T08:06:42.129 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.72 2024-03-04T08:06:42.130 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.72 2024-03-04T08:06:46.745 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 18% (36/191) Updating files: 19% (37/191) Updating files: 20% (39/191) Updating files: 21% (41/191) Updating files: 22% (43/191) Updating files: 23% (44/191) Updating files: 24% (46/191) Updating files: 25% (48/191) Updating files: 26% (50/191) Updating files: 27% (52/191) Updating files: 28% (54/191) Updating files: 29% (56/191) Updating files: 30% (58/191) Updating files: 31% (60/191) Updating files: 32% (62/191) Updating files: 33% (64/191) Updating files: 34% (65/191) Updating files: 35% (67/191) Updating files: 36% (69/191) Updating files: 37% (71/191) Updating files: 38% (73/191) Updating files: 39% (75/191) Updating files: 40% (77/191) Updating files: 41% (79/191) Updating files: 42% (81/191) Updating files: 43% (83/191) Updating files: 44% (85/191) Updating files: 45% (86/191) Updating files: 46% (88/191) Updating files: 47% (90/191) Updating files: 48% (92/191) Updating files: 49% (94/191) Updating files: 50% (96/191) Updating files: 51% (98/191) Updating files: 52% (100/191) Updating files: 53% (102/191) Updating files: 54% (104/191) Updating files: 54% (105/191) Updating files: 55% (106/191) Updating files: 56% (107/191) Updating files: 57% (109/191) Updating files: 58% (111/191) Updating files: 59% (113/191) Updating files: 60% (115/191) Updating files: 61% (117/191) Updating files: 62% (119/191) Updating files: 63% (121/191) Updating files: 64% (123/191) Updating files: 65% (125/191) Updating files: 66% (127/191) Updating files: 67% (128/191) Updating files: 68% (130/191) Updating files: 69% (132/191) Updating files: 70% (134/191) Updating files: 71% (136/191) Updating files: 72% (138/191) Updating files: 73% (140/191) Updating files: 74% (142/191) Updating files: 75% (144/191) Updating files: 76% (146/191) Updating files: 77% (148/191) Updating files: 78% (149/191) Updating files: 79% (151/191) Updating files: 80% (153/191) Updating files: 81% (155/191) Updating files: 82% (157/191) Updating files: 83% (159/191) Updating files: 84% (161/191) Updating files: 85% (163/191) Updating files: 86% (165/191) Updating files: 87% (167/191) Updating files: 88% (169/191) Updating files: 89% (170/191) Updating files: 90% (172/191) Updating files: 91% (174/191) Updating files: 92% (176/191) Updating files: 93% (178/191) Updating files: 94% (180/191) Updating files: 95% (182/191) Updating files: 96% (184/191) Updating files: 97% (186/191) Updating files: 98% (188/191) Updating files: 99% (190/191) Updating files: 100% (191/191) Updating files: 100% (191/191), done. 2024-03-04T08:06:46.821 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 5832e2603c7 v0.72 2024-03-04T08:06:46.830 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.72 2024-03-04T08:06:47.096 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 73 -eq 48 ] 2024-03-04T08:06:47.096 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.73 2024-03-04T08:06:47.096 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.73 2024-03-04T08:06:47.096 INFO:tasks.workunit.client.0.smithi113.stdout:v0.73 2024-03-04T08:06:47.097 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.73 2024-03-04T08:06:49.032 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 77% (114/148) Updating files: 78% (116/148) Updating files: 79% (117/148) Updating files: 80% (119/148) Updating files: 81% (120/148) Updating files: 82% (122/148) Updating files: 83% (123/148) Updating files: 84% (125/148) Updating files: 85% (126/148) Updating files: 86% (128/148) Updating files: 87% (129/148) Updating files: 88% (131/148) Updating files: 89% (132/148) Updating files: 90% (134/148) Updating files: 91% (135/148) Updating files: 92% (137/148) Updating files: 93% (138/148) Updating files: 94% (140/148) Updating files: 95% (141/148) Updating files: 96% (143/148) Updating files: 97% (144/148) Updating files: 98% (146/148) Updating files: 99% (147/148) Updating files: 100% (148/148) Updating files: 100% (148/148), done. 2024-03-04T08:06:49.089 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at d8ad51ee8ab v0.73 2024-03-04T08:06:49.094 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.73 2024-03-04T08:06:49.795 INFO:tasks.workunit.client.0.smithi113.stdout:v0.74 2024-03-04T08:06:49.796 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 74 -eq 48 ] 2024-03-04T08:06:49.796 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.74 2024-03-04T08:06:49.796 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.74 2024-03-04T08:06:49.796 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.74 2024-03-04T08:06:57.121 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 22% (44/192) Updating files: 23% (45/192) Updating files: 24% (47/192) Updating files: 25% (48/192) Updating files: 26% (50/192) Updating files: 27% (52/192) Updating files: 28% (54/192) Updating files: 29% (56/192) Updating files: 30% (58/192) Updating files: 31% (60/192) Updating files: 32% (62/192) Updating files: 33% (64/192) Updating files: 34% (66/192) Updating files: 35% (68/192) Updating files: 36% (70/192) Updating files: 37% (72/192) Updating files: 38% (73/192) Updating files: 39% (75/192) Updating files: 40% (77/192) Updating files: 41% (79/192) Updating files: 42% (81/192) Updating files: 43% (83/192) Updating files: 44% (85/192) Updating files: 45% (87/192) Updating files: 46% (89/192) Updating files: 47% (91/192) Updating files: 48% (93/192) Updating files: 49% (95/192) Updating files: 50% (96/192) Updating files: 51% (98/192) Updating files: 52% (100/192) Updating files: 53% (102/192) Updating files: 54% (104/192) Updating files: 55% (106/192) Updating files: 56% (108/192) Updating files: 56% (109/192) Updating files: 57% (110/192) Updating files: 58% (112/192) Updating files: 59% (114/192) Updating files: 60% (116/192) Updating files: 61% (118/192) Updating files: 62% (120/192) Updating files: 63% (121/192) Updating files: 64% (123/192) Updating files: 65% (125/192) Updating files: 66% (127/192) Updating files: 67% (129/192) Updating files: 68% (131/192) Updating files: 69% (133/192) Updating files: 70% (135/192) Updating files: 71% (137/192) Updating files: 72% (139/192) Updating files: 73% (141/192) Updating files: 74% (143/192) Updating files: 75% (144/192) Updating files: 76% (146/192) Updating files: 77% (148/192) Updating files: 78% (150/192) Updating files: 79% (152/192) Updating files: 80% (154/192) Updating files: 81% (156/192) Updating files: 82% (158/192) Updating files: 83% (160/192) Updating files: 84% (162/192) Updating files: 85% (164/192) Updating files: 86% (166/192) Updating files: 87% (168/192) Updating files: 88% (169/192) Updating files: 89% (171/192) Updating files: 90% (173/192) Updating files: 91% (175/192) Updating files: 92% (177/192) Updating files: 93% (179/192) Updating files: 94% (181/192) Updating files: 95% (183/192) Updating files: 96% (185/192) Updating files: 97% (187/192) Updating files: 98% (189/192) Updating files: 99% (191/192) Updating files: 100% (192/192) Updating files: 100% (192/192), done. 2024-03-04T08:06:57.155 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at c165483bc72 v0.74 2024-03-04T08:06:57.162 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.74 2024-03-04T08:06:58.115 INFO:tasks.workunit.client.0.smithi113.stdout:v0.75 2024-03-04T08:06:58.116 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 75 -eq 48 ] 2024-03-04T08:06:58.116 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.75 2024-03-04T08:06:58.116 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.75 2024-03-04T08:06:58.116 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.75 2024-03-04T08:07:05.058 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:07:05.062 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:07:13.308 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.477+0000 7f0eaae00640 1 Processor -- start 2024-03-04T08:07:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.477+0000 7f0eaae00640 1 -- start start 2024-03-04T08:07:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.477+0000 7f0eaae00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0ea4152b20 0x7f0ea4070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.477+0000 7f0eaae00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea40c8b00 0x7f0ea415be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.477+0000 7f0eaae00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea40c8f60 con 0x7f0ea415c340 2024-03-04T08:07:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.477+0000 7f0eaae00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea40c5320 con 0x7f0ea4152b20 2024-03-04T08:07:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.477+0000 7f0eaae00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4152f00 con 0x7f0ea40c8b00 2024-03-04T08:07:13.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.481+0000 7f0e9b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea40c8b00 0x7f0ea415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.481+0000 7f0ea3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0ea4152b20 0x7f0ea4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.481+0000 7f0ea3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0ea4152b20 0x7f0ea4070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:38334/0 (socket says 172.21.15.113:38334) 2024-03-04T08:07:13.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.481+0000 7f0ea3e00640 1 -- 172.21.15.113:0/3277709207 learned_addr learned my addr 172.21.15.113:0/3277709207 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:13.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.481+0000 7f0e9b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea40c8b00 0x7f0ea415be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:36398/0 (socket says 172.21.15.113:36398) 2024-03-04T08:07:13.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.481+0000 7f0ea3e00640 1 -- 172.21.15.113:0/3277709207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea40c8b00 msgr2=0x7f0ea415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:13.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.481+0000 7f0ea3e00640 1 --2- 172.21.15.113:0/3277709207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea40c8b00 0x7f0ea415be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.481+0000 7f0ea3e00640 1 -- 172.21.15.113:0/3277709207 >> v1:172.21.15.136:6789/0 conn(0x7f0ea415c340 legacy=0x7f0ea40c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.481+0000 7f0ea3e00640 1 -- 172.21.15.113:0/3277709207 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea40c5040 con 0x7f0ea4152b20 2024-03-04T08:07:13.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0ea3e00640 1 --2- 172.21.15.113:0/3277709207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ea4152b20 0x7f0ea4070010 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009870 tx=0x7f0e9401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f41c2a0079a6322d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:13.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0ea3400640 1 -- 172.21.15.113:0/3277709207 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e940ab650 con 0x7f0ea4152b20 2024-03-04T08:07:13.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0ea3400640 1 -- 172.21.15.113:0/3277709207 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e940b0070 con 0x7f0ea4152b20 2024-03-04T08:07:13.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0ea3400640 1 -- 172.21.15.113:0/3277709207 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e940aba10 con 0x7f0ea4152b20 2024-03-04T08:07:13.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0eaae00640 1 -- 172.21.15.113:0/3277709207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ea4152b20 msgr2=0x7f0ea4070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0eaae00640 1 --2- 172.21.15.113:0/3277709207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ea4152b20 0x7f0ea4070010 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009870 tx=0x7f0e9401c560 comp rx=0 tx=0).stop 2024-03-04T08:07:13.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0eaae00640 1 -- 172.21.15.113:0/3277709207 shutdown_connections 2024-03-04T08:07:13.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0eaae00640 1 --2- 172.21.15.113:0/3277709207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea40c8b00 0x7f0ea415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0eaae00640 1 --2- 172.21.15.113:0/3277709207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ea4152b20 0x7f0ea4070010 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0eaae00640 1 -- 172.21.15.113:0/3277709207 >> 172.21.15.113:0/3277709207 conn(0x7f0ea40709a0 msgr2=0x7f0ea4070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:13.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0eaae00640 1 -- 172.21.15.113:0/3277709207 shutdown_connections 2024-03-04T08:07:13.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.485+0000 7f0eaae00640 1 -- 172.21.15.113:0/3277709207 wait complete. 2024-03-04T08:07:13.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0eaae00640 1 Processor -- start 2024-03-04T08:07:13.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0eaae00640 1 -- start start 2024-03-04T08:07:13.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0eaae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea40c8b00 0x7f0ea41f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0eaae00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ea4152b20 0x7f0ea406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0eaae00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea415c340 0x7f0ea40c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0eaae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea40c3810 con 0x7f0ea40c8b00 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0eaae00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea415cb60 con 0x7f0ea4152b20 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0eaae00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4158810 con 0x7f0ea415c340 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea40c8b00 0x7f0ea41f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea40c8b00 0x7f0ea41f3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35244/0 (socket says 172.21.15.113:35244) 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea3e00640 1 -- 172.21.15.113:0/225922410 learned_addr learned my addr 172.21.15.113:0/225922410 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0e9b400640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ea4152b20 0x7f0ea406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea8e00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea415c340 0x7f0ea40c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea3e00640 1 -- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea415c340 msgr2=0x7f0ea40c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea3e00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea415c340 0x7f0ea40c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea3e00640 1 -- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ea4152b20 msgr2=0x7f0ea406e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea3e00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ea4152b20 0x7f0ea406e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea3e00640 1 -- 172.21.15.113:0/225922410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea4163270 con 0x7f0ea40c8b00 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0e9b400640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ea4152b20 0x7f0ea406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:07:13.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea3e00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea40c8b00 0x7f0ea41f3e90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f0e940ad040 tx=0x7f0e9401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea0c00640 1 -- 172.21.15.113:0/225922410 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e940be5f0 con 0x7f0ea40c8b00 2024-03-04T08:07:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea0c00640 1 -- 172.21.15.113:0/225922410 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e940be790 con 0x7f0ea40c8b00 2024-03-04T08:07:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0ea0c00640 1 -- 172.21.15.113:0/225922410 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e940beb80 con 0x7f0ea40c8b00 2024-03-04T08:07:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0eaae00640 1 -- 172.21.15.113:0/225922410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea40cdde0 con 0x7f0ea40c8b00 2024-03-04T08:07:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.489+0000 7f0eaae00640 1 -- 172.21.15.113:0/225922410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea40cd020 con 0x7f0ea40c8b00 2024-03-04T08:07:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.493+0000 7f0eaae00640 1 -- 172.21.15.113:0/225922410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea4075c00 con 0x7f0ea40c8b00 2024-03-04T08:07:13.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.493+0000 7f0ea0c00640 1 -- 172.21.15.113:0/225922410 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e940b43f0 con 0x7f0ea40c8b00 2024-03-04T08:07:13.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.493+0000 7f0ea0c00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e80066f60 0x7f0e80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.493+0000 7f0ea0c00640 1 -- 172.21.15.113:0/225922410 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f0e94120090 con 0x7f0ea40c8b00 2024-03-04T08:07:13.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.493+0000 7f0e9b400640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e80066f60 0x7f0e80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.493+0000 7f0e9b400640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e80066f60 0x7f0e80069420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009fd0 tx=0x7f0e8c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:13.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.497+0000 7f0ea0c00640 1 -- 172.21.15.113:0/225922410 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0e94016150 con 0x7f0ea40c8b00 2024-03-04T08:07:13.579 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 7% (22/291) Updating files: 8% (24/291) Updating files: 8% (25/291) Updating files: 9% (27/291) Updating files: 10% (30/291) Updating files: 11% (33/291) Updating files: 12% (35/291) Updating files: 13% (38/291) Updating files: 14% (41/291) Updating files: 15% (44/291) Updating files: 16% (47/291) Updating files: 17% (50/291) Updating files: 18% (53/291) Updating files: 19% (56/291) Updating files: 20% (59/291) Updating files: 21% (62/291) Updating files: 21% (64/291) Updating files: 22% (65/291) Updating files: 23% (67/291) Updating files: 24% (70/291) Updating files: 25% (73/291) Updating files: 26% (76/291) Updating files: 27% (79/291) Updating files: 28% (82/291) Updating files: 29% (85/291) Updating files: 30% (88/291) Updating files: 31% (91/291) Updating files: 32% (94/291) Updating files: 33% (97/291) Updating files: 34% (99/291) Updating files: 35% (102/291) Updating files: 36% (105/291) Updating files: 37% (108/291) Updating files: 38% (111/291) Updating files: 39% (114/291) Updating files: 40% (117/291) Updating files: 41% (120/291) Updating files: 42% (123/291) Updating files: 43% (126/291) Updating files: 44% (129/291) Updating files: 45% (131/291) Updating files: 46% (134/291) Updating files: 47% (137/291) Updating files: 48% (140/291) Updating files: 49% (143/291) Updating files: 50% (146/291) Updating files: 51% (149/291) Updating files: 52% (152/291) Updating files: 53% (155/291) Updating files: 54% (158/291) Updating files: 55% (161/291) Updating files: 56% (163/291) Updating files: 57% (166/291) Updating files: 58% (169/291) Updating files: 59% (172/291) Updating files: 60% (175/291) Updating files: 61% (178/291) Updating files: 62% (181/291) Updating files: 63% (184/291) Updating files: 64% (187/291) Updating files: 65% (190/291) Updating files: 66% (193/291) Updating files: 67% (195/291) Updating files: 68% (198/291) Updating files: 69% (201/291) Updating files: 70% (204/291) Updating files: 71% (207/291) Updating files: 72% (210/291) Updating files: 73% (213/291) Updating files: 74% (216/291) Updating files: 75% (219/291) Updating files: 76% (222/291) Updating files: 77% (225/291) Updating files: 78% (227/291) Updating files: 79% (230/291) Updating files: 80% (233/291) Updating files: 81% (236/291) Updating files: 82% (239/291) Updating files: 83% (242/291) Updating files: 84% (245/291) Updating files: 85% (248/291) Updating files: 86% (251/291) Updating files: 87% (254/291) Updating files: 88% (257/291) Updating files: 89% (259/291) Updating files: 90% (262/291) Updating files: 91% (265/291) Updating files: 92% (268/291) Updating files: 93% (271/291) Updating files: 94% (274/291) Updating files: 95% (277/291) Updating files: 96% (280/291) Updating files: 97% (283/291) Updating files: 98% (286/291) Updating files: 99% (289/291) Updating files: 100% (291/291) Updating files: 100% (291/291), done. 2024-03-04T08:07:13.643 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 946d6036958 v0.75 2024-03-04T08:07:13.655 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.75 2024-03-04T08:07:13.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.701+0000 7f0eaae00640 1 -- 172.21.15.113:0/225922410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ea40cb5c0 con 0x7f0ea40c8b00 2024-03-04T08:07:13.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.705+0000 7f0ea0c00640 1 -- 172.21.15.113:0/225922410 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f0e940f02a0 con 0x7f0ea40c8b00 2024-03-04T08:07:13.703 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:13.703 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:08.199463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7269,7119,6925]},"id":1}]} 2024-03-04T08:07:13.704 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 125 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 -- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e80066f60 msgr2=0x7f0e80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e80066f60 0x7f0e80069420 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009fd0 tx=0x7f0e8c00b040 comp rx=0 tx=0).stop 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 -- 172.21.15.113:0/225922410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea40c8b00 msgr2=0x7f0ea41f3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea40c8b00 0x7f0ea41f3e90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f0e940ad040 tx=0x7f0e9401ada0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 -- 172.21.15.113:0/225922410 shutdown_connections 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ea415c340 0x7f0ea40c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ea4152b20 0x7f0ea406e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e80066f60 0x7f0e80069420 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 --2- 172.21.15.113:0/225922410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea40c8b00 0x7f0ea41f3e90 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 -- 172.21.15.113:0/225922410 >> 172.21.15.113:0/225922410 conn(0x7f0ea40709a0 msgr2=0x7f0ea4158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 -- 172.21.15.113:0/225922410 shutdown_connections 2024-03-04T08:07:13.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.709+0000 7f0eaae00640 1 -- 172.21.15.113:0/225922410 wait complete. 2024-03-04T08:07:13.734 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=0) 2024-03-04T08:07:13.734 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-04T08:07:13.735 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:07:13.755 INFO:tasks.ceph.mds.b:Stopped 2024-03-04T08:07:13.755 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:13.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd7600640 1 Processor -- start 2024-03-04T08:07:13.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd7600640 1 -- start start 2024-03-04T08:07:13.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd7600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd0154fe0 0x7fafd00c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd7600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd00c7d80 con 0x7fafd0154c00 2024-03-04T08:07:13.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd7600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd00c8060 con 0x7fafd00c7700 2024-03-04T08:07:13.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd7600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd015efb0 con 0x7fafd0154fe0 2024-03-04T08:07:13.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd4c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fafd0154c00 0x7fafd00c1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48264/0 (socket says 172.21.15.113:48264) 2024-03-04T08:07:13.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd4c00640 1 -- 172.21.15.113:0/1899490081 learned_addr learned my addr 172.21.15.113:0/1899490081 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:13.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcfe00640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd0154fe0 0x7fafd00c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883764937 0 0) 0x7fafd00c7d80 con 0x7fafd0154c00 2024-03-04T08:07:13.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb8003530 con 0x7fafd0154c00 2024-03-04T08:07:13.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505617617 0 0) 0x7fafb8003530 con 0x7fafd0154c00 2024-03-04T08:07:13.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd0154fe0 msgr2=0x7fafd00c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:13.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd0154fe0 0x7fafd00c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 >> v1:172.21.15.136:6789/0 conn(0x7fafd00c7700 legacy=0x7fafd015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd00c6dc0 con 0x7fafd0154c00 2024-03-04T08:07:13.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fafc4001680 con 0x7fafd0154c00 2024-03-04T08:07:13.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 >> v1:172.21.15.113:6789/0 conn(0x7fafd0154c00 legacy=0x7fafd00c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 0x7fafb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafb8004810 0x7fafb8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd0154fe0 0x7fafb80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd015efb0 con 0x7fafb80039f0 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb800bad0 con 0x7fafb8004810 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb800bcd0 con 0x7fafd0154fe0 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd4c00640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 0x7fafb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcfe00640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafb8004810 0x7fafb8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd5600640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd0154fe0 0x7fafb80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd4c00640 1 -- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd0154fe0 msgr2=0x7fafb80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd4c00640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd0154fe0 0x7fafb80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd4c00640 1 -- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafb8004810 msgr2=0x7fafb8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd4c00640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafb8004810 0x7fafb8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafd4c00640 1 -- 172.21.15.113:0/1899490081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb8003530 con 0x7fafb80039f0 2024-03-04T08:07:13.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.901+0000 7fafcfe00640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafb8004810 0x7fafb8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:13.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd4c00640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 0x7fafb8003dd0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c160 tx=0x7fafc401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfea011678364a2c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4007c90 con 0x7fafb80039f0 2024-03-04T08:07:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc40a9070 con 0x7fafb80039f0 2024-03-04T08:07:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafcf400640 1 -- 172.21.15.113:0/1899490081 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4005d40 con 0x7fafb80039f0 2024-03-04T08:07:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 -- 172.21.15.113:0/1899490081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 msgr2=0x7fafb8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 --2- 172.21.15.113:0/1899490081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 0x7fafb8003dd0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fafc400c160 tx=0x7fafc401cbe0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd5600640 1 -- 172.21.15.113:0/1899490081 reap_dead start 2024-03-04T08:07:13.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 -- 172.21.15.113:0/1899490081 shutdown_connections 2024-03-04T08:07:13.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 -- 172.21.15.113:0/1899490081 >> 172.21.15.113:0/1899490081 conn(0x7fafd006e820 msgr2=0x7fafd006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:13.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 -- 172.21.15.113:0/1899490081 shutdown_connections 2024-03-04T08:07:13.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 -- 172.21.15.113:0/1899490081 wait complete. 2024-03-04T08:07:13.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 Processor -- start 2024-03-04T08:07:13.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 -- start start 2024-03-04T08:07:13.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 0x7fafd015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafd01607d0 0x7fafd014fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd01f68d0 0x7fafd00cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00c1520 con 0x7fafb80039f0 2024-03-04T08:07:13.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00c13a0 con 0x7fafd01607d0 2024-03-04T08:07:13.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.909+0000 7fafd7600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd0166c10 con 0x7fafd01f68d0 2024-03-04T08:07:13.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 0x7fafd015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 0x7fafd015bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35276/0 (socket says 172.21.15.113:35276) 2024-03-04T08:07:13.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd4c00640 1 -- 172.21.15.113:0/2992429983 learned_addr learned my addr 172.21.15.113:0/2992429983 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:13.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafcfe00640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafd01607d0 0x7fafd014fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd5600640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd01f68d0 0x7fafd00cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd4c00640 1 -- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd01f68d0 msgr2=0x7fafd00cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd4c00640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd01f68d0 0x7fafd00cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd4c00640 1 -- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafd01607d0 msgr2=0x7fafd014fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:13.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd4c00640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafd01607d0 0x7fafd014fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:13.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd4c00640 1 -- 172.21.15.113:0/2992429983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd015cbe0 con 0x7fafb80039f0 2024-03-04T08:07:13.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd5600640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd01f68d0 0x7fafd00cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:13.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd4c00640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 0x7fafd015bc30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fafc4098300 tx=0x7fafb8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:13.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafccc00640 1 -- 172.21.15.113:0/2992429983 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4000fa0 con 0x7fafb80039f0 2024-03-04T08:07:13.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafccc00640 1 -- 172.21.15.113:0/2992429983 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc4005d20 con 0x7fafb80039f0 2024-03-04T08:07:13.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafccc00640 1 -- 172.21.15.113:0/2992429983 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc40049f0 con 0x7fafb80039f0 2024-03-04T08:07:13.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd7600640 1 -- 172.21.15.113:0/2992429983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd01503b0 con 0x7fafb80039f0 2024-03-04T08:07:13.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd7600640 1 -- 172.21.15.113:0/2992429983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd015e380 con 0x7fafb80039f0 2024-03-04T08:07:13.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafd7600640 1 -- 172.21.15.113:0/2992429983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd0075c00 con 0x7fafb80039f0 2024-03-04T08:07:13.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.913+0000 7fafccc00640 1 -- 172.21.15.113:0/2992429983 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafc40b7030 con 0x7fafb80039f0 2024-03-04T08:07:13.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.917+0000 7fafccc00640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafb0066f60 0x7fafb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:13.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.917+0000 7fafccc00640 1 -- 172.21.15.113:0/2992429983 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7fafc411e8f0 con 0x7fafb80039f0 2024-03-04T08:07:13.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.917+0000 7fafcfe00640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafb0066f60 0x7fafb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:13.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.917+0000 7fafcfe00640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafb0066f60 0x7fafb0069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fafd01f0f90 tx=0x7fafbc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:13.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:13.925+0000 7fafccc00640 1 -- 172.21.15.113:0/2992429983 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fafc40be020 con 0x7fafb80039f0 2024-03-04T08:07:14.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 -- 172.21.15.113:0/2992429983 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafd0165010 con 0x7fafb80039f0 2024-03-04T08:07:14.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafccc00640 1 -- 172.21.15.113:0/2992429983 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+6624 (secure 0 0 0) 0x7fafc40168f0 con 0x7fafb80039f0 2024-03-04T08:07:14.156 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:14.156 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:08.199463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7269,7119,6925]},"id":1}]} 2024-03-04T08:07:14.156 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 125 2024-03-04T08:07:14.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 -- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafb0066f60 msgr2=0x7fafb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:14.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafb0066f60 0x7fafb0069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fafd01f0f90 tx=0x7fafbc008040 comp rx=0 tx=0).stop 2024-03-04T08:07:14.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 -- 172.21.15.113:0/2992429983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 msgr2=0x7fafd015bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:14.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 0x7fafd015bc30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fafc4098300 tx=0x7fafb8005840 comp rx=0 tx=0).stop 2024-03-04T08:07:14.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 -- 172.21.15.113:0/2992429983 shutdown_connections 2024-03-04T08:07:14.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafd01f68d0 0x7fafd00cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:14.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafd01607d0 0x7fafd014fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:14.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafb0066f60 0x7fafb0069420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:14.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 --2- 172.21.15.113:0/2992429983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafb80039f0 0x7fafd015bc30 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:14.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 -- 172.21.15.113:0/2992429983 >> 172.21.15.113:0/2992429983 conn(0x7fafd006e820 msgr2=0x7fafd015a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:14.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 -- 172.21.15.113:0/2992429983 shutdown_connections 2024-03-04T08:07:14.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:14.157+0000 7fafd7600640 1 -- 172.21.15.113:0/2992429983 wait complete. 2024-03-04T08:07:14.178 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:07:16.179 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:16.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f7090e00640 1 Processor -- start 2024-03-04T08:07:16.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f7090e00640 1 -- start start 2024-03-04T08:07:16.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f7090e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f708c0b20b0 0x7f708c157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:16.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f7090e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f708c0c85f0 0x7f708c161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:16.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f7090e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708c0c1ec0 con 0x7f708c157c50 2024-03-04T08:07:16.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f7090e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708c0c9b30 con 0x7f708c0b20b0 2024-03-04T08:07:16.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f7090e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708c0c12a0 con 0x7f708c0c85f0 2024-03-04T08:07:16.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f7089e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f708c0b20b0 0x7f708c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:16.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f7089e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f708c0b20b0 0x7f708c157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35290/0 (socket says 172.21.15.113:35290) 2024-03-04T08:07:16.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f7089e00640 1 -- 172.21.15.113:0/1181599941 learned_addr learned my addr 172.21.15.113:0/1181599941 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:16.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.325+0000 7f708a800640 1 --2- 172.21.15.113:0/1181599941 >> v2:172.21.15.136:3300/0 conn(0x7f708c0c85f0 0x7f708c161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:16.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7089e00640 1 -- 172.21.15.113:0/1181599941 >> v2:172.21.15.136:3300/0 conn(0x7f708c0c85f0 msgr2=0x7f708c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:16.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7089e00640 1 --2- 172.21.15.113:0/1181599941 >> v2:172.21.15.136:3300/0 conn(0x7f708c0c85f0 0x7f708c161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7089e00640 1 -- 172.21.15.113:0/1181599941 >> v1:172.21.15.136:6789/0 conn(0x7f708c157c50 legacy=0x7f708c06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:16.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7089e00640 1 -- 172.21.15.113:0/1181599941 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0c1580 con 0x7f708c0b20b0 2024-03-04T08:07:16.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7089e00640 1 --2- 172.21.15.113:0/1181599941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 0x7f708c157710 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f708c0c96d0 tx=0x7f707c0200a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f886fe726b3df5a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:16.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7088a00640 1 -- 172.21.15.113:0/1181599941 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0b6070 con 0x7f708c0b20b0 2024-03-04T08:07:16.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7088a00640 1 -- 172.21.15.113:0/1181599941 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c0b1440 con 0x7f708c0b20b0 2024-03-04T08:07:16.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7088a00640 1 -- 172.21.15.113:0/1181599941 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c00b040 con 0x7f708c0b20b0 2024-03-04T08:07:16.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7090e00640 1 -- 172.21.15.113:0/1181599941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 msgr2=0x7f708c157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:16.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7090e00640 1 --2- 172.21.15.113:0/1181599941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 0x7f708c157710 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f708c0c96d0 tx=0x7f707c0200a0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7090e00640 1 -- 172.21.15.113:0/1181599941 shutdown_connections 2024-03-04T08:07:16.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7090e00640 1 --2- 172.21.15.113:0/1181599941 >> v2:172.21.15.136:3300/0 conn(0x7f708c0c85f0 0x7f708c161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7090e00640 1 --2- 172.21.15.113:0/1181599941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 0x7f708c157710 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.329+0000 7f7090e00640 1 -- 172.21.15.113:0/1181599941 >> 172.21.15.113:0/1181599941 conn(0x7f708c06e820 msgr2=0x7f708c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:16.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 -- 172.21.15.113:0/1181599941 shutdown_connections 2024-03-04T08:07:16.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 -- 172.21.15.113:0/1181599941 wait complete. 2024-03-04T08:07:16.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 Processor -- start 2024-03-04T08:07:16.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 -- start start 2024-03-04T08:07:16.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 0x7f708c1e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:16.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f708c0c85f0 0x7f708c1e2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:16.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f708c157c50 0x7f708c0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:16.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c0c2400 con 0x7f708c0b20b0 2024-03-04T08:07:16.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c1571f0 con 0x7f708c0c85f0 2024-03-04T08:07:16.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c156ec0 con 0x7f708c157c50 2024-03-04T08:07:16.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 0x7f708c1e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 0x7f708c1e17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35294/0 (socket says 172.21.15.113:35294) 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089e00640 1 -- 172.21.15.113:0/1794652089 learned_addr learned my addr 172.21.15.113:0/1794652089 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089400640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f708c0c85f0 0x7f708c1e2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f708a800640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f708c157c50 0x7f708c0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089e00640 1 -- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f708c157c50 msgr2=0x7f708c0c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089e00640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f708c157c50 0x7f708c0c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089e00640 1 -- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f708c0c85f0 msgr2=0x7f708c1e2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089e00640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f708c0c85f0 0x7f708c1e2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089e00640 1 -- 172.21.15.113:0/1794652089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0aed00 con 0x7f708c0b20b0 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089400640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f708c0c85f0 0x7f708c1e2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7089e00640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 0x7f708c1e17a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f707c00a1c0 tx=0x7f707c01e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7082000640 1 -- 172.21.15.113:0/1794652089 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0bf070 con 0x7f708c0b20b0 2024-03-04T08:07:16.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7082000640 1 -- 172.21.15.113:0/1794652089 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c0af550 con 0x7f708c0b20b0 2024-03-04T08:07:16.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7090e00640 1 -- 172.21.15.113:0/1794652089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c1f36c0 con 0x7f708c0b20b0 2024-03-04T08:07:16.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.333+0000 7f7082000640 1 -- 172.21.15.113:0/1794652089 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0b6030 con 0x7f708c0b20b0 2024-03-04T08:07:16.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.337+0000 7f7090e00640 1 -- 172.21.15.113:0/1794652089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c1f3c80 con 0x7f708c0b20b0 2024-03-04T08:07:16.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.337+0000 7f7090e00640 1 -- 172.21.15.113:0/1794652089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c075c00 con 0x7f708c0b20b0 2024-03-04T08:07:16.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.337+0000 7f7082000640 1 -- 172.21.15.113:0/1794652089 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f707c0c2070 con 0x7f708c0b20b0 2024-03-04T08:07:16.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.341+0000 7f7082000640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f706c066f60 0x7f706c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:16.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.341+0000 7f7082000640 1 -- 172.21.15.113:0/1794652089 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f707c1225d0 con 0x7f708c0b20b0 2024-03-04T08:07:16.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.341+0000 7f7089400640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f706c066f60 0x7f706c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:16.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.341+0000 7f7089400640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f706c066f60 0x7f706c069420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f708c0c7960 tx=0x7f7078009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:16.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.349+0000 7f7082000640 1 -- 172.21.15.113:0/1794652089 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f707c019c00 con 0x7f708c0b20b0 2024-03-04T08:07:16.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.557+0000 7f7090e00640 1 -- 172.21.15.113:0/1794652089 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f708c0c83d0 con 0x7f708c0b20b0 2024-03-04T08:07:16.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.557+0000 7f7082000640 1 -- 172.21.15.113:0/1794652089 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f707c0f2ff0 con 0x7f708c0b20b0 2024-03-04T08:07:16.558 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:16.558 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:08.199463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7269,7119,6925]},"id":1}]} 2024-03-04T08:07:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 125 2024-03-04T08:07:16.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 -- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f706c066f60 msgr2=0x7f706c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:16.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f706c066f60 0x7f706c069420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f708c0c7960 tx=0x7f7078009040 comp rx=0 tx=0).stop 2024-03-04T08:07:16.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 -- 172.21.15.113:0/1794652089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 msgr2=0x7f708c1e17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:16.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 0x7f708c1e17a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f707c00a1c0 tx=0x7f707c01e8e0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 -- 172.21.15.113:0/1794652089 shutdown_connections 2024-03-04T08:07:16.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f708c157c50 0x7f708c0c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f708c0c85f0 0x7f708c1e2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f706c066f60 0x7f706c069420 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 --2- 172.21.15.113:0/1794652089 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f708c0b20b0 0x7f708c1e17a0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:16.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 -- 172.21.15.113:0/1794652089 >> 172.21.15.113:0/1794652089 conn(0x7f708c06e820 msgr2=0x7f708c0cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:16.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 -- 172.21.15.113:0/1794652089 shutdown_connections 2024-03-04T08:07:16.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:16.561+0000 7f7090e00640 1 -- 172.21.15.113:0/1794652089 wait complete. 2024-03-04T08:07:16.581 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:07:18.583 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:18.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.721+0000 7f9426000640 1 Processor -- start 2024-03-04T08:07:18.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.721+0000 7f9426000640 1 -- start start 2024-03-04T08:07:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f9426000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f94200c7a80 0x7f94200cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f9426000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9420072d80 con 0x7f9420152c60 2024-03-04T08:07:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f9426000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9420073510 con 0x7f942006cb60 2024-03-04T08:07:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f9426000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942006b060 con 0x7f94200c7a80 2024-03-04T08:07:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941f400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9420152c60 0x7f942015bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:32794/0 (socket says 172.21.15.113:32794) 2024-03-04T08:07:18.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941f400640 1 -- 172.21.15.113:0/913978203 learned_addr learned my addr 172.21.15.113:0/913978203 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:18.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941ea00640 1 --2- 172.21.15.113:0/913978203 >> v2:172.21.15.136:3300/0 conn(0x7f94200c7a80 0x7f94200cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:18.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844950694 0 0) 0x7f9420072d80 con 0x7f9420152c60 2024-03-04T08:07:18.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9404003530 con 0x7f9420152c60 2024-03-04T08:07:18.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 144047152 0 0) 0x7f9404003530 con 0x7f9420152c60 2024-03-04T08:07:18.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 >> v2:172.21.15.136:3300/0 conn(0x7f94200c7a80 msgr2=0x7f94200cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:18.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 --2- 172.21.15.113:0/913978203 >> v2:172.21.15.136:3300/0 conn(0x7f94200c7a80 0x7f94200cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 >> v1:172.21.15.136:6789/0 conn(0x7f942006cb60 legacy=0x7f94201630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942006b2b0 con 0x7f9420152c60 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f9410001680 con 0x7f9420152c60 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 >> v1:172.21.15.113:6789/0 conn(0x7f9420152c60 legacy=0x7f942015bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9404003a10 0x7f9404003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9404004810 0x7f9404004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404005220 0x7f940400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942006b060 con 0x7f9404003a10 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940400c170 con 0x7f9404005220 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940400c370 con 0x7f9404004810 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941f400640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9404003a10 0x7f9404003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941ea00640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9404004810 0x7f9404004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941fe00640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404005220 0x7f940400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941f400640 1 -- 172.21.15.113:0/913978203 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9404004810 msgr2=0x7f9404004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:18.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941f400640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9404004810 0x7f9404004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941f400640 1 -- 172.21.15.113:0/913978203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404005220 msgr2=0x7f940400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:18.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941f400640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404005220 0x7f940400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941f400640 1 -- 172.21.15.113:0/913978203 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9404003530 con 0x7f9404003a10 2024-03-04T08:07:18.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.725+0000 7f941fe00640 1 -- 172.21.15.113:0/913978203 reap_dead start 2024-03-04T08:07:18.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f941f400640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9404003a10 0x7f9404003e30 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f941000c160 tx=0x7f941001cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57961d1b227d61b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:18.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410007c70 con 0x7f9404003a10 2024-03-04T08:07:18.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94100a9070 con 0x7f9404003a10 2024-03-04T08:07:18.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f941e000640 1 -- 172.21.15.113:0/913978203 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410005da0 con 0x7f9404003a10 2024-03-04T08:07:18.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f9426000640 1 -- 172.21.15.113:0/913978203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9404003a10 msgr2=0x7f9404003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:18.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f9426000640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9404003a10 0x7f9404003e30 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f941000c160 tx=0x7f941001cb50 comp rx=0 tx=0).stop 2024-03-04T08:07:18.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f9426000640 1 -- 172.21.15.113:0/913978203 shutdown_connections 2024-03-04T08:07:18.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f9426000640 1 --2- 172.21.15.113:0/913978203 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9404003a10 0x7f9404003e30 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f9426000640 1 -- 172.21.15.113:0/913978203 >> 172.21.15.113:0/913978203 conn(0x7f94200671b0 msgr2=0x7f942006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:18.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f9426000640 1 -- 172.21.15.113:0/913978203 shutdown_connections 2024-03-04T08:07:18.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.729+0000 7f9426000640 1 -- 172.21.15.113:0/913978203 wait complete. 2024-03-04T08:07:18.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 Processor -- start 2024-03-04T08:07:18.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 -- start start 2024-03-04T08:07:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404003a10 0x7f94201f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9420157a40 0x7f94201f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94201f6fe0 0x7f9420158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94200c8ac0 con 0x7f9420157a40 2024-03-04T08:07:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94200731d0 con 0x7f9404003a10 2024-03-04T08:07:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94200c8c40 con 0x7f94201f6fe0 2024-03-04T08:07:18.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9420157a40 0x7f94201f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9420157a40 0x7f94201f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49366/0 (socket says 172.21.15.113:49366) 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941ea00640 1 -- 172.21.15.113:0/3868578325 learned_addr learned my addr 172.21.15.113:0/3868578325 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941f400640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404003a10 0x7f94201f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941fe00640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94201f6fe0 0x7f9420158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941ea00640 1 -- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94201f6fe0 msgr2=0x7f9420158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941ea00640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94201f6fe0 0x7f9420158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941ea00640 1 -- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404003a10 msgr2=0x7f94201f12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941ea00640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404003a10 0x7f94201f12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941ea00640 1 -- 172.21.15.113:0/3868578325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94201f7490 con 0x7f9420157a40 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f941ea00640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9420157a40 0x7f94201f6aa0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f940c011530 tx=0x7f940c0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9417400640 1 -- 172.21.15.113:0/3868578325 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c015870 con 0x7f9420157a40 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9417400640 1 -- 172.21.15.113:0/3868578325 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c015a10 con 0x7f9420157a40 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 -- 172.21.15.113:0/3868578325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9420157280 con 0x7f9420157a40 2024-03-04T08:07:18.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9417400640 1 -- 172.21.15.113:0/3868578325 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c015dc0 con 0x7f9420157a40 2024-03-04T08:07:18.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 -- 172.21.15.113:0/3868578325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94201f17f0 con 0x7f9420157a40 2024-03-04T08:07:18.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.733+0000 7f9426000640 1 -- 172.21.15.113:0/3868578325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9420075c00 con 0x7f9420157a40 2024-03-04T08:07:18.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.737+0000 7f9417400640 1 -- 172.21.15.113:0/3868578325 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f940c013400 con 0x7f9420157a40 2024-03-04T08:07:18.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.737+0000 7f9417400640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9400066f60 0x7f9400069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:18.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.737+0000 7f9417400640 1 -- 172.21.15.113:0/3868578325 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f940c085db0 con 0x7f9420157a40 2024-03-04T08:07:18.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.737+0000 7f941f400640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9400066f60 0x7f9400069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:18.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.737+0000 7f941f400640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9400066f60 0x7f9400069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9404004790 tx=0x7f94100a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:18.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.741+0000 7f9417400640 1 -- 172.21.15.113:0/3868578325 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f940c08a070 con 0x7f9420157a40 2024-03-04T08:07:18.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.937+0000 7f9426000640 1 -- 172.21.15.113:0/3868578325 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9420072fb0 con 0x7f9420157a40 2024-03-04T08:07:18.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9417400640 1 -- 172.21.15.113:0/3868578325 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f940c019030 con 0x7f9420157a40 2024-03-04T08:07:18.939 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:18.939 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:08.199463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7269,7119,6925]},"id":1}]} 2024-03-04T08:07:18.939 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 125 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 -- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9400066f60 msgr2=0x7f9400069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9400066f60 0x7f9400069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9404004790 tx=0x7f94100a7070 comp rx=0 tx=0).stop 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 -- 172.21.15.113:0/3868578325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9420157a40 msgr2=0x7f94201f6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9420157a40 0x7f94201f6aa0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f940c011530 tx=0x7f940c0119f0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 -- 172.21.15.113:0/3868578325 shutdown_connections 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94201f6fe0 0x7f9420158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9400066f60 0x7f9400069420 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9420157a40 0x7f94201f6aa0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 --2- 172.21.15.113:0/3868578325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9404003a10 0x7f94201f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 -- 172.21.15.113:0/3868578325 >> 172.21.15.113:0/3868578325 conn(0x7f94200671b0 msgr2=0x7f94200c20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:18.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 -- 172.21.15.113:0/3868578325 shutdown_connections 2024-03-04T08:07:18.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:18.941+0000 7f9426000640 1 -- 172.21.15.113:0/3868578325 wait complete. 2024-03-04T08:07:18.960 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:07:20.961 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:21.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.109+0000 7f9f14c00640 1 Processor -- start 2024-03-04T08:07:21.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.109+0000 7f9f14c00640 1 -- start start 2024-03-04T08:07:21.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f14c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9f1015b900 0x7f9f10163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:21.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f14c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f100c27f0 con 0x7f9f1014e4c0 2024-03-04T08:07:21.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f14c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f100c2a90 con 0x7f9f100c22e0 2024-03-04T08:07:21.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f14c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1015ad60 con 0x7f9f1015b900 2024-03-04T08:07:21.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f0de00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9f1014e4c0 0x7f9f10072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:32806/0 (socket says 172.21.15.113:32806) 2024-03-04T08:07:21.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f0de00640 1 -- 172.21.15.113:0/4095178674 learned_addr learned my addr 172.21.15.113:0/4095178674 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:21.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f0e800640 1 --2- 172.21.15.113:0/4095178674 >> v2:172.21.15.113:3300/0 conn(0x7f9f1015b900 0x7f9f10163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:21.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f0d400640 1 -- 172.21.15.113:0/4095178674 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605768578 0 0) 0x7f9f100c27f0 con 0x7f9f1014e4c0 2024-03-04T08:07:21.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f0d400640 1 -- 172.21.15.113:0/4095178674 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef4005740 con 0x7f9f1014e4c0 2024-03-04T08:07:21.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f0e800640 1 -- 172.21.15.113:0/4095178674 >> v1:172.21.15.136:6789/0 conn(0x7f9f100c22e0 legacy=0x7f9f1015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:21.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f0e800640 1 -- 172.21.15.113:0/4095178674 >> v1:172.21.15.113:6789/0 conn(0x7f9f1014e4c0 legacy=0x7f9f10072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:21.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.113+0000 7f9f0e800640 1 -- 172.21.15.113:0/4095178674 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f10070c60 con 0x7f9f1015b900 2024-03-04T08:07:21.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f0e800640 1 --2- 172.21.15.113:0/4095178674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1015b900 0x7f9f10163100 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009d90 tx=0x7f9f0801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8000a0bb540a82 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:21.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f0d400640 1 -- 172.21.15.113:0/4095178674 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f080b4070 con 0x7f9f1015b900 2024-03-04T08:07:21.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f0d400640 1 -- 172.21.15.113:0/4095178674 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f080af600 con 0x7f9f1015b900 2024-03-04T08:07:21.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f0d400640 1 -- 172.21.15.113:0/4095178674 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f080af9d0 con 0x7f9f1015b900 2024-03-04T08:07:21.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f14c00640 1 -- 172.21.15.113:0/4095178674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1015b900 msgr2=0x7f9f10163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:21.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f14c00640 1 --2- 172.21.15.113:0/4095178674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1015b900 0x7f9f10163100 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009d90 tx=0x7f9f0801f150 comp rx=0 tx=0).stop 2024-03-04T08:07:21.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f14c00640 1 -- 172.21.15.113:0/4095178674 shutdown_connections 2024-03-04T08:07:21.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f14c00640 1 --2- 172.21.15.113:0/4095178674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1015b900 0x7f9f10163100 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:21.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f14c00640 1 -- 172.21.15.113:0/4095178674 >> 172.21.15.113:0/4095178674 conn(0x7f9f1006e820 msgr2=0x7f9f1006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:21.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f14c00640 1 -- 172.21.15.113:0/4095178674 shutdown_connections 2024-03-04T08:07:21.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.117+0000 7f9f14c00640 1 -- 172.21.15.113:0/4095178674 wait complete. 2024-03-04T08:07:21.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 Processor -- start 2024-03-04T08:07:21.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 -- start start 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f100c22e0 0x7f9f10145200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1014e4c0 0x7f9f100cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1015b900 0x7f9f101ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f10071500 con 0x7f9f1014e4c0 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f10071220 con 0x7f9f100c22e0 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1014e8a0 con 0x7f9f1015b900 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f05400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1014e4c0 0x7f9f100cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f05400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1014e4c0 0x7f9f100cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49378/0 (socket says 172.21.15.113:49378) 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f05400640 1 -- 172.21.15.113:0/840906613 learned_addr learned my addr 172.21.15.113:0/840906613 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f0e800640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1015b900 0x7f9f101ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f0de00640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f100c22e0 0x7f9f10145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f05400640 1 -- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1015b900 msgr2=0x7f9f101ecb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f05400640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1015b900 0x7f9f101ecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f05400640 1 -- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f100c22e0 msgr2=0x7f9f10145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f05400640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f100c22e0 0x7f9f10145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f05400640 1 -- 172.21.15.113:0/840906613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f10159010 con 0x7f9f1014e4c0 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f0e800640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1015b900 0x7f9f101ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f05400640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1014e4c0 0x7f9f100cb350 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800c3c0 tx=0x7f9ef800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f06a00640 1 -- 172.21.15.113:0/840906613 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8017070 con 0x7f9f1014e4c0 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f06a00640 1 -- 172.21.15.113:0/840906613 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef80052d0 con 0x7f9f1014e4c0 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 -- 172.21.15.113:0/840906613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f101592b0 con 0x7f9f1014e4c0 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f06a00640 1 -- 172.21.15.113:0/840906613 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8003840 con 0x7f9f1014e4c0 2024-03-04T08:07:21.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 -- 172.21.15.113:0/840906613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f10140b10 con 0x7f9f1014e4c0 2024-03-04T08:07:21.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.121+0000 7f9f14c00640 1 -- 172.21.15.113:0/840906613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f10075c00 con 0x7f9f1014e4c0 2024-03-04T08:07:21.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.125+0000 7f9f06a00640 1 -- 172.21.15.113:0/840906613 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ef801a070 con 0x7f9f1014e4c0 2024-03-04T08:07:21.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.125+0000 7f9f06a00640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9eec066f60 0x7f9eec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:21.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.125+0000 7f9f06a00640 1 -- 172.21.15.113:0/840906613 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f9ef8084a70 con 0x7f9f1014e4c0 2024-03-04T08:07:21.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.125+0000 7f9f0de00640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9eec066f60 0x7f9eec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:21.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.125+0000 7f9f0de00640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9eec066f60 0x7f9eec069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9f000011f0 tx=0x7f9f00009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:21.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.129+0000 7f9f06a00640 1 -- 172.21.15.113:0/840906613 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9ef8005480 con 0x7f9f1014e4c0 2024-03-04T08:07:21.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 -- 172.21.15.113:0/840906613 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f10155190 con 0x7f9f1014e4c0 2024-03-04T08:07:21.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f06a00640 1 -- 172.21.15.113:0/840906613 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f9ef8055360 con 0x7f9f1014e4c0 2024-03-04T08:07:21.383 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:21.384 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:08.199463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7269,7119,6925]},"id":1}]} 2024-03-04T08:07:21.384 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 125 2024-03-04T08:07:21.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 -- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9eec066f60 msgr2=0x7f9eec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:21.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9eec066f60 0x7f9eec069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9f000011f0 tx=0x7f9f00009040 comp rx=0 tx=0).stop 2024-03-04T08:07:21.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 -- 172.21.15.113:0/840906613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1014e4c0 msgr2=0x7f9f100cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:21.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1014e4c0 0x7f9f100cb350 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800c3c0 tx=0x7f9ef800c880 comp rx=0 tx=0).stop 2024-03-04T08:07:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 -- 172.21.15.113:0/840906613 shutdown_connections 2024-03-04T08:07:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f1015b900 0x7f9f101ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9eec066f60 0x7f9eec069420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f1014e4c0 0x7f9f100cb350 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 --2- 172.21.15.113:0/840906613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f100c22e0 0x7f9f10145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 -- 172.21.15.113:0/840906613 >> 172.21.15.113:0/840906613 conn(0x7f9f1006e820 msgr2=0x7f9f10147c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 -- 172.21.15.113:0/840906613 shutdown_connections 2024-03-04T08:07:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:21.385+0000 7f9f14c00640 1 -- 172.21.15.113:0/840906613 wait complete. 2024-03-04T08:07:21.407 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:07:23.408 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:23.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.557+0000 7fe57cc00640 1 Processor -- start 2024-03-04T08:07:23.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.557+0000 7fe57cc00640 1 -- start start 2024-03-04T08:07:23.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe57cc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe57806ceb0 0x7fe578157520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:23.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe57cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5780c8e50 0x7fe5780c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:23.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe57cc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5780c9820 con 0x7fe578151940 2024-03-04T08:07:23.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe57cc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe578157b40 con 0x7fe57806ceb0 2024-03-04T08:07:23.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe57cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780c8670 con 0x7fe5780c8e50 2024-03-04T08:07:23.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe577400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe578151940 0x7fe57806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:32820/0 (socket says 172.21.15.113:32820) 2024-03-04T08:07:23.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe577400640 1 -- 172.21.15.113:0/2488996553 learned_addr learned my addr 172.21.15.113:0/2488996553 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:23.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe576a00640 1 --2- 172.21.15.113:0/2488996553 >> v2:172.21.15.136:3300/0 conn(0x7fe57806ceb0 0x7fe578157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:23.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe577e00640 1 --2- 172.21.15.113:0/2488996553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5780c8e50 0x7fe5780c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:23.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe576000640 1 -- 172.21.15.113:0/2488996553 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593385921 0 0) 0x7fe5780c9820 con 0x7fe578151940 2024-03-04T08:07:23.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe576000640 1 -- 172.21.15.113:0/2488996553 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe55c003530 con 0x7fe578151940 2024-03-04T08:07:23.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe577e00640 1 -- 172.21.15.113:0/2488996553 >> v2:172.21.15.136:3300/0 conn(0x7fe57806ceb0 msgr2=0x7fe578157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:23.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe577e00640 1 --2- 172.21.15.113:0/2488996553 >> v2:172.21.15.136:3300/0 conn(0x7fe57806ceb0 0x7fe578157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:23.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe577e00640 1 -- 172.21.15.113:0/2488996553 >> v1:172.21.15.113:6789/0 conn(0x7fe578151940 legacy=0x7fe57806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:23.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.561+0000 7fe577e00640 1 -- 172.21.15.113:0/2488996553 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe578157e00 con 0x7fe5780c8e50 2024-03-04T08:07:23.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe577e00640 1 --2- 172.21.15.113:0/2488996553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5780c8e50 0x7fe5780c92e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe570009d90 tx=0x7fe57001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=695ba52ada043219 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:23.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe576000640 1 -- 172.21.15.113:0/2488996553 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700b2070 con 0x7fe5780c8e50 2024-03-04T08:07:23.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe576000640 1 -- 172.21.15.113:0/2488996553 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5700a5830 con 0x7fe5780c8e50 2024-03-04T08:07:23.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe576000640 1 -- 172.21.15.113:0/2488996553 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700a5be0 con 0x7fe5780c8e50 2024-03-04T08:07:23.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe57cc00640 1 -- 172.21.15.113:0/2488996553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5780c8e50 msgr2=0x7fe5780c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:23.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe57cc00640 1 --2- 172.21.15.113:0/2488996553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5780c8e50 0x7fe5780c92e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe570009d90 tx=0x7fe57001c970 comp rx=0 tx=0).stop 2024-03-04T08:07:23.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe57cc00640 1 -- 172.21.15.113:0/2488996553 shutdown_connections 2024-03-04T08:07:23.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe57cc00640 1 --2- 172.21.15.113:0/2488996553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5780c8e50 0x7fe5780c92e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:23.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe57cc00640 1 --2- 172.21.15.113:0/2488996553 >> v2:172.21.15.136:3300/0 conn(0x7fe57806ceb0 0x7fe578157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:23.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe57cc00640 1 -- 172.21.15.113:0/2488996553 >> 172.21.15.113:0/2488996553 conn(0x7fe5780cd080 msgr2=0x7fe5780cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:23.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe57cc00640 1 -- 172.21.15.113:0/2488996553 shutdown_connections 2024-03-04T08:07:23.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.565+0000 7fe57cc00640 1 -- 172.21.15.113:0/2488996553 wait complete. 2024-03-04T08:07:23.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe57cc00640 1 Processor -- start 2024-03-04T08:07:23.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe57cc00640 1 -- start start 2024-03-04T08:07:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe57cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57806ceb0 0x7fe578143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe57cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5780c8e50 0x7fe578156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe57cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe578151940 0x7fe57806b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe57cc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780cdba0 con 0x7fe57806ceb0 2024-03-04T08:07:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe57cc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780c20a0 con 0x7fe5780c8e50 2024-03-04T08:07:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe57cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5781f42b0 con 0x7fe578151940 2024-03-04T08:07:23.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe577400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57806ceb0 0x7fe578143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:23.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe577400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57806ceb0 0x7fe578143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49390/0 (socket says 172.21.15.113:49390) 2024-03-04T08:07:23.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe577400640 1 -- 172.21.15.113:0/1848299349 learned_addr learned my addr 172.21.15.113:0/1848299349 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:23.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe576a00640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5780c8e50 0x7fe578156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:23.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe577e00640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe578151940 0x7fe57806b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:23.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe577400640 1 -- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe578151940 msgr2=0x7fe57806b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:23.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe577400640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe578151940 0x7fe57806b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:23.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe577400640 1 -- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5780c8e50 msgr2=0x7fe578156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:23.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe577400640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5780c8e50 0x7fe578156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe577400640 1 -- 172.21.15.113:0/1848299349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5780cd5f0 con 0x7fe57806ceb0 2024-03-04T08:07:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe577400640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57806ceb0 0x7fe578143460 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fe56800c480 tx=0x7fe56800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe56f400640 1 -- 172.21.15.113:0/1848299349 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568017070 con 0x7fe57806ceb0 2024-03-04T08:07:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe56f400640 1 -- 172.21.15.113:0/1848299349 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe568005cf0 con 0x7fe57806ceb0 2024-03-04T08:07:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe57cc00640 1 -- 172.21.15.113:0/1848299349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe578143df0 con 0x7fe57806ceb0 2024-03-04T08:07:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe56f400640 1 -- 172.21.15.113:0/1848299349 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568007290 con 0x7fe57806ceb0 2024-03-04T08:07:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.569+0000 7fe57cc00640 1 -- 172.21.15.113:0/1848299349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5781f3fd0 con 0x7fe57806ceb0 2024-03-04T08:07:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.573+0000 7fe57cc00640 1 -- 172.21.15.113:0/1848299349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe578075a80 con 0x7fe57806ceb0 2024-03-04T08:07:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.573+0000 7fe56f400640 1 -- 172.21.15.113:0/1848299349 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe56801a070 con 0x7fe57806ceb0 2024-03-04T08:07:23.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.573+0000 7fe56f400640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe558066f60 0x7fe558069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:23.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.573+0000 7fe56f400640 1 -- 172.21.15.113:0/1848299349 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7fe568084b00 con 0x7fe57806ceb0 2024-03-04T08:07:23.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.573+0000 7fe576a00640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe558066f60 0x7fe558069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:23.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.573+0000 7fe576a00640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe558066f60 0x7fe558069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe57814cb40 tx=0x7fe564009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:23.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.581+0000 7fe56f400640 1 -- 172.21.15.113:0/1848299349 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe568054da0 con 0x7fe57806ceb0 2024-03-04T08:07:23.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.777+0000 7fe57cc00640 1 -- 172.21.15.113:0/1848299349 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5780c8240 con 0x7fe57806ceb0 2024-03-04T08:07:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.777+0000 7fe56f400640 1 -- 172.21.15.113:0/1848299349 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+6624 (secure 0 0 0) 0x7fe568084e20 con 0x7fe57806ceb0 2024-03-04T08:07:23.777 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:23.777 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:08.199463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7269,7119,6925]},"id":1}]} 2024-03-04T08:07:23.777 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 125 2024-03-04T08:07:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 -- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe558066f60 msgr2=0x7fe558069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe558066f60 0x7fe558069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe57814cb40 tx=0x7fe564009190 comp rx=0 tx=0).stop 2024-03-04T08:07:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 -- 172.21.15.113:0/1848299349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57806ceb0 msgr2=0x7fe578143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:23.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57806ceb0 0x7fe578143460 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fe56800c480 tx=0x7fe56800c940 comp rx=0 tx=0).stop 2024-03-04T08:07:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 -- 172.21.15.113:0/1848299349 shutdown_connections 2024-03-04T08:07:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe578151940 0x7fe57806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5780c8e50 0x7fe578156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe558066f60 0x7fe558069420 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 --2- 172.21.15.113:0/1848299349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe57806ceb0 0x7fe578143460 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 -- 172.21.15.113:0/1848299349 >> 172.21.15.113:0/1848299349 conn(0x7fe5780cd080 msgr2=0x7fe5781509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 -- 172.21.15.113:0/1848299349 shutdown_connections 2024-03-04T08:07:23.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:23.781+0000 7fe57cc00640 1 -- 172.21.15.113:0/1848299349 wait complete. 2024-03-04T08:07:23.799 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:07:25.800 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:25.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe55cc00640 1 Processor -- start 2024-03-04T08:07:25.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe55cc00640 1 -- start start 2024-03-04T08:07:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe55cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 0x7fe558070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe55cc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe558152bd0 0x7fe5581587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe55cc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5581591f0 con 0x7fe5580c8200 2024-03-04T08:07:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe55cc00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe558071df0 con 0x7fe558152bd0 2024-03-04T08:07:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe55cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5581597b0 con 0x7fe5581527f0 2024-03-04T08:07:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe556800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe558152bd0 0x7fe5581587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe556800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe558152bd0 0x7fe5581587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49392/0 (socket says 172.21.15.113:49392) 2024-03-04T08:07:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe556800640 1 -- 172.21.15.113:0/2574255794 learned_addr learned my addr 172.21.15.113:0/2574255794 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:25.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe555e00640 1 --2- 172.21.15.113:0/2574255794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 0x7fe558070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:25.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe556800640 1 -- 172.21.15.113:0/2574255794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 msgr2=0x7fe558070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:25.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe556800640 1 --2- 172.21.15.113:0/2574255794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 0x7fe558070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:25.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe556800640 1 -- 172.21.15.113:0/2574255794 >> v1:172.21.15.136:6789/0 conn(0x7fe5580c8200 legacy=0x7fe558072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:25.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe556800640 1 -- 172.21.15.113:0/2574255794 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5581594d0 con 0x7fe558152bd0 2024-03-04T08:07:25.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.941+0000 7fe555e00640 1 --2- 172.21.15.113:0/2574255794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 0x7fe558070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:25.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.945+0000 7fe556800640 1 --2- 172.21.15.113:0/2574255794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe558152bd0 0x7fe5581587d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe550009870 tx=0x7fe55001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7628d51b0cdc2c40 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:25.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe555400640 1 -- 172.21.15.113:0/2574255794 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5500ab650 con 0x7fe558152bd0 2024-03-04T08:07:25.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe555400640 1 -- 172.21.15.113:0/2574255794 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5500b0070 con 0x7fe558152bd0 2024-03-04T08:07:25.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe555400640 1 -- 172.21.15.113:0/2574255794 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5500aba10 con 0x7fe558152bd0 2024-03-04T08:07:25.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe55cc00640 1 -- 172.21.15.113:0/2574255794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe558152bd0 msgr2=0x7fe5581587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe55cc00640 1 --2- 172.21.15.113:0/2574255794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe558152bd0 0x7fe5581587d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe550009870 tx=0x7fe55001c560 comp rx=0 tx=0).stop 2024-03-04T08:07:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe55cc00640 1 -- 172.21.15.113:0/2574255794 shutdown_connections 2024-03-04T08:07:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe55cc00640 1 --2- 172.21.15.113:0/2574255794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe558152bd0 0x7fe5581587d0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe55cc00640 1 --2- 172.21.15.113:0/2574255794 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 0x7fe558070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe55cc00640 1 -- 172.21.15.113:0/2574255794 >> 172.21.15.113:0/2574255794 conn(0x7fe55806e820 msgr2=0x7fe55806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe55cc00640 1 -- 172.21.15.113:0/2574255794 shutdown_connections 2024-03-04T08:07:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe55cc00640 1 -- 172.21.15.113:0/2574255794 wait complete. 2024-03-04T08:07:25.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.949+0000 7fe55cc00640 1 Processor -- start 2024-03-04T08:07:25.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe55cc00640 1 -- start start 2024-03-04T08:07:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe55cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5580c8200 0x7fe558148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe55cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 0x7fe558148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe55cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe558152bd0 0x7fe5581616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe55cc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5581654f0 con 0x7fe5580c8200 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe55cc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe558165370 con 0x7fe558152bd0 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe55cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe558165670 con 0x7fe5581527f0 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe555e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5580c8200 0x7fe558148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe555e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5580c8200 0x7fe558148a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49396/0 (socket says 172.21.15.113:49396) 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe555e00640 1 -- 172.21.15.113:0/652236427 learned_addr learned my addr 172.21.15.113:0/652236427 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe556800640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe558152bd0 0x7fe5581616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe54d400640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 0x7fe558148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe555e00640 1 -- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 msgr2=0x7fe558148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe555e00640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 0x7fe558148f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe555e00640 1 -- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe558152bd0 msgr2=0x7fe5581616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe555e00640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe558152bd0 0x7fe5581616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe555e00640 1 -- 172.21.15.113:0/652236427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5581f4630 con 0x7fe5580c8200 2024-03-04T08:07:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe555e00640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5580c8200 0x7fe558148a00 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe548007fa0 tx=0x7fe54800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe54ea00640 1 -- 172.21.15.113:0/652236427 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54801a070 con 0x7fe5580c8200 2024-03-04T08:07:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe54ea00640 1 -- 172.21.15.113:0/652236427 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe548003ba0 con 0x7fe5580c8200 2024-03-04T08:07:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe55cc00640 1 -- 172.21.15.113:0/652236427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5581f48d0 con 0x7fe5580c8200 2024-03-04T08:07:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe54ea00640 1 -- 172.21.15.113:0/652236427 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548004030 con 0x7fe5580c8200 2024-03-04T08:07:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe55cc00640 1 -- 172.21.15.113:0/652236427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe558162cd0 con 0x7fe5580c8200 2024-03-04T08:07:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.953+0000 7fe55cc00640 1 -- 172.21.15.113:0/652236427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe558075c00 con 0x7fe5580c8200 2024-03-04T08:07:25.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.957+0000 7fe54ea00640 1 -- 172.21.15.113:0/652236427 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe54801d070 con 0x7fe5580c8200 2024-03-04T08:07:25.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.957+0000 7fe54ea00640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe534066f90 0x7fe534069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:25.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.957+0000 7fe54ea00640 1 -- 172.21.15.113:0/652236427 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7fe5480859b0 con 0x7fe5580c8200 2024-03-04T08:07:25.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.957+0000 7fe54d400640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe534066f90 0x7fe534069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:25.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.957+0000 7fe54d400640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe534066f90 0x7fe534069450 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe5581558c0 tx=0x7fe540009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:25.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:25.965+0000 7fe54ea00640 1 -- 172.21.15.113:0/652236427 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe548010070 con 0x7fe5580c8200 2024-03-04T08:07:26.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.157+0000 7fe55cc00640 1 -- 172.21.15.113:0/652236427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5581612e0 con 0x7fe5580c8200 2024-03-04T08:07:26.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe54ea00640 1 -- 172.21.15.113:0/652236427 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+6624 (secure 0 0 0) 0x7fe548055c50 con 0x7fe5580c8200 2024-03-04T08:07:26.159 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:26.159 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:08.199463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7269,7119,6925]},"id":1}]} 2024-03-04T08:07:26.160 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 125 2024-03-04T08:07:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 -- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe534066f90 msgr2=0x7fe534069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe534066f90 0x7fe534069450 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe5581558c0 tx=0x7fe540009040 comp rx=0 tx=0).stop 2024-03-04T08:07:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 -- 172.21.15.113:0/652236427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5580c8200 msgr2=0x7fe558148a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5580c8200 0x7fe558148a00 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe548007fa0 tx=0x7fe54800d560 comp rx=0 tx=0).stop 2024-03-04T08:07:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 -- 172.21.15.113:0/652236427 shutdown_connections 2024-03-04T08:07:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe558152bd0 0x7fe5581616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5581527f0 0x7fe558148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe534066f90 0x7fe534069450 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:26.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 --2- 172.21.15.113:0/652236427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5580c8200 0x7fe558148a00 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:26.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 -- 172.21.15.113:0/652236427 >> 172.21.15.113:0/652236427 conn(0x7fe55806e820 msgr2=0x7fe558158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:26.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 -- 172.21.15.113:0/652236427 shutdown_connections 2024-03-04T08:07:26.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:26.161+0000 7fe55cc00640 1 -- 172.21.15.113:0/652236427 wait complete. 2024-03-04T08:07:26.180 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:07:28.181 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:28.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19c0a00640 1 Processor -- start 2024-03-04T08:07:28.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19c0a00640 1 -- start start 2024-03-04T08:07:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19c0a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f19bc152d60 0x7f19bc0c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19c0a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 0x7f19bc159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19c0a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19bc159980 con 0x7f19bc159470 2024-03-04T08:07:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19c0a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19bc06f3b0 con 0x7f19bc152d60 2024-03-04T08:07:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19c0a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc0c3430 con 0x7f19bc0c38c0 2024-03-04T08:07:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19b9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f19bc152d60 0x7f19bc0c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19b3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 0x7f19bc159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19b9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f19bc152d60 0x7f19bc0c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:49056/0 (socket says 172.21.15.113:49056) 2024-03-04T08:07:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19b9e00640 1 -- 172.21.15.113:0/3708991663 learned_addr learned my addr 172.21.15.113:0/3708991663 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19b3e00640 1 -- 172.21.15.113:0/3708991663 >> v2:172.21.15.136:3300/0 conn(0x7f19bc152d60 msgr2=0x7f19bc0c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19b3e00640 1 --2- 172.21.15.113:0/3708991663 >> v2:172.21.15.136:3300/0 conn(0x7f19bc152d60 0x7f19bc0c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19b3e00640 1 -- 172.21.15.113:0/3708991663 >> v1:172.21.15.136:6789/0 conn(0x7f19bc159470 legacy=0x7f19bc0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19b3e00640 1 -- 172.21.15.113:0/3708991663 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc153140 con 0x7f19bc0c38c0 2024-03-04T08:07:28.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.325+0000 7f19b9e00640 1 --2- 172.21.15.113:0/3708991663 >> v2:172.21.15.136:3300/0 conn(0x7f19bc152d60 0x7f19bc0c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:07:28.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19b3e00640 1 --2- 172.21.15.113:0/3708991663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 0x7f19bc159ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097c0 tx=0x7f19a401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d2546387fc05aa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:28.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19b9400640 1 -- 172.21.15.113:0/3708991663 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40b2070 con 0x7f19bc0c38c0 2024-03-04T08:07:28.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19b9400640 1 -- 172.21.15.113:0/3708991663 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a40ad640 con 0x7f19bc0c38c0 2024-03-04T08:07:28.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19b9400640 1 -- 172.21.15.113:0/3708991663 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40ada10 con 0x7f19bc0c38c0 2024-03-04T08:07:28.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19c0a00640 1 -- 172.21.15.113:0/3708991663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 msgr2=0x7f19bc159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:28.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19c0a00640 1 --2- 172.21.15.113:0/3708991663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 0x7f19bc159ea0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097c0 tx=0x7f19a401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:07:28.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19c0a00640 1 -- 172.21.15.113:0/3708991663 shutdown_connections 2024-03-04T08:07:28.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19c0a00640 1 --2- 172.21.15.113:0/3708991663 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 0x7f19bc159ea0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:28.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19c0a00640 1 --2- 172.21.15.113:0/3708991663 >> v2:172.21.15.136:3300/0 conn(0x7f19bc152d60 0x7f19bc0c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:28.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19c0a00640 1 -- 172.21.15.113:0/3708991663 >> 172.21.15.113:0/3708991663 conn(0x7f19bc066db0 msgr2=0x7f19bc0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:28.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19c0a00640 1 -- 172.21.15.113:0/3708991663 shutdown_connections 2024-03-04T08:07:28.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19c0a00640 1 -- 172.21.15.113:0/3708991663 wait complete. 2024-03-04T08:07:28.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19c0a00640 1 Processor -- start 2024-03-04T08:07:28.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.333+0000 7f19c0a00640 1 -- start start 2024-03-04T08:07:28.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19c0a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 0x7f19bc151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:28.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19c0a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19bc152d60 0x7f19bc1f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:28.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19c0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19bc159470 0x7f19bc1f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19c0a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc15a560 con 0x7f19bc159470 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19c0a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc15a3e0 con 0x7f19bc152d60 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19c0a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19bc0ce9e0 con 0x7f19bc0c38c0 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19ba800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19bc159470 0x7f19bc1f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19ba800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19bc159470 0x7f19bc1f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46826/0 (socket says 172.21.15.113:46826) 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19ba800640 1 -- 172.21.15.113:0/607685529 learned_addr learned my addr 172.21.15.113:0/607685529 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19b3e00640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19bc152d60 0x7f19bc1f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19b9e00640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 0x7f19bc151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19ba800640 1 -- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 msgr2=0x7f19bc151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19ba800640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 0x7f19bc151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19ba800640 1 -- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19bc152d60 msgr2=0x7f19bc1f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19ba800640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19bc152d60 0x7f19bc1f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19ba800640 1 -- 172.21.15.113:0/607685529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc0b2d60 con 0x7f19bc159470 2024-03-04T08:07:28.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19ba800640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19bc159470 0x7f19bc1f42f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f19b400c760 tx=0x7f19b400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:28.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19b2000640 1 -- 172.21.15.113:0/607685529 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b4003b80 con 0x7f19bc159470 2024-03-04T08:07:28.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19b2000640 1 -- 172.21.15.113:0/607685529 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b4003d20 con 0x7f19bc159470 2024-03-04T08:07:28.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19b2000640 1 -- 172.21.15.113:0/607685529 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b400aa90 con 0x7f19bc159470 2024-03-04T08:07:28.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19c0a00640 1 -- 172.21.15.113:0/607685529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc0b3040 con 0x7f19bc159470 2024-03-04T08:07:28.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19c0a00640 1 -- 172.21.15.113:0/607685529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19bc0b3600 con 0x7f19bc159470 2024-03-04T08:07:28.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.337+0000 7f19c0a00640 1 -- 172.21.15.113:0/607685529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19bc068eb0 con 0x7f19bc159470 2024-03-04T08:07:28.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.341+0000 7f19b2000640 1 -- 172.21.15.113:0/607685529 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19b400ac30 con 0x7f19bc159470 2024-03-04T08:07:28.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.341+0000 7f19b2000640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1998066f60 0x7f1998069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:28.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.341+0000 7f19b2000640 1 -- 172.21.15.113:0/607685529 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f19b40853a0 con 0x7f19bc159470 2024-03-04T08:07:28.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.341+0000 7f19b9e00640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1998066f60 0x7f1998069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:28.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.341+0000 7f19b9e00640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1998066f60 0x7f1998069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f19ac006cb0 tx=0x7f19ac009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:28.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.349+0000 7f19b2000640 1 -- 172.21.15.113:0/607685529 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f19b40556e0 con 0x7f19bc159470 2024-03-04T08:07:28.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.541+0000 7f19c0a00640 1 -- 172.21.15.113:0/607685529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19bc075c00 con 0x7f19bc159470 2024-03-04T08:07:28.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.541+0000 7f19b2000640 1 -- 172.21.15.113:0/607685529 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f19b4059780 con 0x7f19bc159470 2024-03-04T08:07:28.541 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:28.542 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7389,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:06:08.199463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6925,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6925":{"gid":6925,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":23,"addr":"172.21.15.136:6835/2522634751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":2522634751},{"type":"v1","addr":"172.21.15.136:6835","nonce":2522634751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6925,"qdb_cluster":[7269,7119,6925]},"id":1}]} 2024-03-04T08:07:28.542 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 125 2024-03-04T08:07:28.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 -- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1998066f60 msgr2=0x7f1998069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:28.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1998066f60 0x7f1998069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f19ac006cb0 tx=0x7f19ac009240 comp rx=0 tx=0).stop 2024-03-04T08:07:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 -- 172.21.15.113:0/607685529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19bc159470 msgr2=0x7f19bc1f42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19bc159470 0x7f19bc1f42f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f19b400c760 tx=0x7f19b400cc20 comp rx=0 tx=0).stop 2024-03-04T08:07:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 -- 172.21.15.113:0/607685529 shutdown_connections 2024-03-04T08:07:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1998066f60 0x7f1998069420 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19bc159470 0x7f19bc1f42f0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19bc152d60 0x7f19bc1f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 --2- 172.21.15.113:0/607685529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19bc0c38c0 0x7f19bc151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 -- 172.21.15.113:0/607685529 >> 172.21.15.113:0/607685529 conn(0x7f19bc066db0 msgr2=0x7f19bc06c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 -- 172.21.15.113:0/607685529 shutdown_connections 2024-03-04T08:07:28.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:28.545+0000 7f19c0a00640 1 -- 172.21.15.113:0/607685529 wait complete. 2024-03-04T08:07:28.563 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:07:30.564 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:30.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.701+0000 7f2125c00640 1 Processor -- start 2024-03-04T08:07:30.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.701+0000 7f2125c00640 1 -- start start 2024-03-04T08:07:30.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.701+0000 7f2125c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21200c4fd0 0x7f21200c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:30.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.701+0000 7f2125c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f21200c9df0 0x7f21200ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:30.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f2125c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21200c9490 con 0x7f2120152bd0 2024-03-04T08:07:30.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f2125c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21200c9c50 con 0x7f21200c9df0 2024-03-04T08:07:30.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f2125c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21200c97c0 con 0x7f21200c4fd0 2024-03-04T08:07:30.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f211ea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2120152bd0 0x7f2120158980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52838/0 (socket says 172.21.15.113:52838) 2024-03-04T08:07:30.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f211ea00640 1 -- 172.21.15.113:0/1389856703 learned_addr learned my addr 172.21.15.113:0/1389856703 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:30.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f211f400640 1 --2- 172.21.15.113:0/1389856703 >> v2:172.21.15.113:3300/0 conn(0x7f21200c9df0 0x7f21200ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:30.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f2116000640 1 --2- 172.21.15.113:0/1389856703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21200c4fd0 0x7f21200c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:30.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f211f400640 1 -- 172.21.15.113:0/1389856703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21200c4fd0 msgr2=0x7f21200c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:30.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f211f400640 1 --2- 172.21.15.113:0/1389856703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21200c4fd0 0x7f21200c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:30.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f211f400640 1 -- 172.21.15.113:0/1389856703 >> v1:172.21.15.113:6789/0 conn(0x7f2120152bd0 legacy=0x7f2120158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:30.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f211f400640 1 -- 172.21.15.113:0/1389856703 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21200c54a0 con 0x7f21200c9df0 2024-03-04T08:07:30.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f211e000640 1 -- 172.21.15.113:0/1389856703 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991853191 0 0) 0x7f21200c9490 con 0x7f2120152bd0 2024-03-04T08:07:30.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.705+0000 7f211f400640 1 --2- 172.21.15.113:0/1389856703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 0x7f21200ce270 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2118009d90 tx=0x7f211801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7f44a2faec11bc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:30.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f211e000640 1 -- 172.21.15.113:0/1389856703 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21180b2070 con 0x7f21200c9df0 2024-03-04T08:07:30.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f211e000640 1 -- 172.21.15.113:0/1389856703 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21180a5910 con 0x7f21200c9df0 2024-03-04T08:07:30.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f211e000640 1 -- 172.21.15.113:0/1389856703 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21180a5ce0 con 0x7f21200c9df0 2024-03-04T08:07:30.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f2125c00640 1 -- 172.21.15.113:0/1389856703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 msgr2=0x7f21200ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:30.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f2125c00640 1 --2- 172.21.15.113:0/1389856703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 0x7f21200ce270 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2118009d90 tx=0x7f211801c940 comp rx=0 tx=0).stop 2024-03-04T08:07:30.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f2125c00640 1 -- 172.21.15.113:0/1389856703 shutdown_connections 2024-03-04T08:07:30.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f2125c00640 1 --2- 172.21.15.113:0/1389856703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 0x7f21200ce270 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:30.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f2125c00640 1 --2- 172.21.15.113:0/1389856703 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21200c4fd0 0x7f21200c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:30.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f2125c00640 1 -- 172.21.15.113:0/1389856703 >> 172.21.15.113:0/1389856703 conn(0x7f2120066db0 msgr2=0x7f21200671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:30.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f2125c00640 1 -- 172.21.15.113:0/1389856703 shutdown_connections 2024-03-04T08:07:30.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f2125c00640 1 -- 172.21.15.113:0/1389856703 wait complete. 2024-03-04T08:07:30.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f2125c00640 1 Processor -- start 2024-03-04T08:07:30.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.709+0000 7f2125c00640 1 -- start start 2024-03-04T08:07:30.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2125c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21200c4fd0 0x7f212015b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:30.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2125c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 0x7f21201f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:30.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2125c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2120152bd0 0x7f21201f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:30.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2125c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21200c3b10 con 0x7f21200c9df0 2024-03-04T08:07:30.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2125c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21200c3990 con 0x7f2120152bd0 2024-03-04T08:07:30.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2125c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2120152fb0 con 0x7f21200c4fd0 2024-03-04T08:07:30.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2116000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 0x7f21201f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:30.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2116000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 0x7f21201f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46852/0 (socket says 172.21.15.113:46852) 2024-03-04T08:07:30.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2116000640 1 -- 172.21.15.113:0/3711760979 learned_addr learned my addr 172.21.15.113:0/3711760979 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:30.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f211ea00640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21200c4fd0 0x7f212015b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:30.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f211f400640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2120152bd0 0x7f21201f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:30.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2116000640 1 -- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21200c4fd0 msgr2=0x7f212015b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:30.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2116000640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21200c4fd0 0x7f212015b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:30.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2116000640 1 -- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2120152bd0 msgr2=0x7f21201f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:30.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2116000640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2120152bd0 0x7f21201f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:30.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2116000640 1 -- 172.21.15.113:0/3711760979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21200c8700 con 0x7f21200c9df0 2024-03-04T08:07:30.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2116000640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 0x7f21201f3640 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f210800e870 tx=0x7f210800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:30.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2117400640 1 -- 172.21.15.113:0/3711760979 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108017070 con 0x7f21200c9df0 2024-03-04T08:07:30.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2117400640 1 -- 172.21.15.113:0/3711760979 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210800c940 con 0x7f21200c9df0 2024-03-04T08:07:30.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2125c00640 1 -- 172.21.15.113:0/3711760979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21200c8140 con 0x7f21200c9df0 2024-03-04T08:07:30.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2117400640 1 -- 172.21.15.113:0/3711760979 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210800cd30 con 0x7f21200c9df0 2024-03-04T08:07:30.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2125c00640 1 -- 172.21.15.113:0/3711760979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21201f2b60 con 0x7f21200c9df0 2024-03-04T08:07:30.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.713+0000 7f2125c00640 1 -- 172.21.15.113:0/3711760979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2120075c00 con 0x7f21200c9df0 2024-03-04T08:07:30.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.717+0000 7f2117400640 1 -- 172.21.15.113:0/3711760979 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f210801a070 con 0x7f21200c9df0 2024-03-04T08:07:30.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.717+0000 7f2117400640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20fc066f60 0x7f20fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:30.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.717+0000 7f2117400640 1 -- 172.21.15.113:0/3711760979 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7373+0+0 (secure 0 0 0) 0x7f21080859b0 con 0x7f21200c9df0 2024-03-04T08:07:30.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.717+0000 7f211ea00640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20fc066f60 0x7f20fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:30.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.717+0000 7f211ea00640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20fc066f60 0x7f20fc069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2110000f70 tx=0x7f2110009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:30.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.725+0000 7f2117400640 1 -- 172.21.15.113:0/3711760979 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f21080562a0 con 0x7f21200c9df0 2024-03-04T08:07:30.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.993+0000 7f2125c00640 1 -- 172.21.15.113:0/3711760979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f212015cd80 con 0x7f21200c9df0 2024-03-04T08:07:30.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.993+0000 7f2117400640 1 -- 172.21.15.113:0/3711760979 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f2108055ba0 con 0x7f21200c9df0 2024-03-04T08:07:30.994 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:30.994 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":126,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:30.861349+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7119,7269]},"id":1}]} 2024-03-04T08:07:30.994 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 126 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 -- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20fc066f60 msgr2=0x7f20fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20fc066f60 0x7f20fc069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2110000f70 tx=0x7f2110009040 comp rx=0 tx=0).stop 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 -- 172.21.15.113:0/3711760979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 msgr2=0x7f21201f3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 0x7f21201f3640 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f210800e870 tx=0x7f210800ec70 comp rx=0 tx=0).stop 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 -- 172.21.15.113:0/3711760979 shutdown_connections 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2120152bd0 0x7f21201f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f20fc066f60 0x7f20fc069420 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21200c9df0 0x7f21201f3640 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 --2- 172.21.15.113:0/3711760979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21200c4fd0 0x7f212015b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 -- 172.21.15.113:0/3711760979 >> 172.21.15.113:0/3711760979 conn(0x7f2120066db0 msgr2=0x7f2120152130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 -- 172.21.15.113:0/3711760979 shutdown_connections 2024-03-04T08:07:30.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:30.997+0000 7f2125c00640 1 -- 172.21.15.113:0/3711760979 wait complete. 2024-03-04T08:07:31.025 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2024-03-04T08:07:31.025 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:07:31.025 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:31.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47e5e00640 1 Processor -- start 2024-03-04T08:07:31.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47e5e00640 1 -- start start 2024-03-04T08:07:31.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47e5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e00c8200 0x7f47e0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:31.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47e5e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e01591f0 con 0x7f47e01527f0 2024-03-04T08:07:31.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47e5e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e01594d0 con 0x7f47e0152bd0 2024-03-04T08:07:31.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47e5e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0071df0 con 0x7f47e00c8200 2024-03-04T08:07:31.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47dea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f47e01527f0 0x7f47e0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52848/0 (socket says 172.21.15.113:52848) 2024-03-04T08:07:31.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47dea00640 1 -- 172.21.15.113:0/301384101 learned_addr learned my addr 172.21.15.113:0/301384101 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:31.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47d6000640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e00c8200 0x7f47e0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:31.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898209838 0 0) 0x7f47e01591f0 con 0x7f47e01527f0 2024-03-04T08:07:31.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c4003530 con 0x7f47e01527f0 2024-03-04T08:07:31.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 9330073 0 0) 0x7f47c4003530 con 0x7f47e01527f0 2024-03-04T08:07:31.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e00c8200 msgr2=0x7f47e0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:31.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e00c8200 0x7f47e0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:31.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 >> v1:172.21.15.136:6789/0 conn(0x7f47e0152bd0 legacy=0x7f47e01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:31.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e00717a0 con 0x7f47e01527f0 2024-03-04T08:07:31.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f47d0001a00 con 0x7f47e01527f0 2024-03-04T08:07:31.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 >> v1:172.21.15.113:6789/0 conn(0x7f47e01527f0 legacy=0x7f47e0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e00c8200 0x7f47c40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47c4003f30 0x7f47c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 0x7f47c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0071df0 con 0x7f47c4004d50 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c400ba80 con 0x7f47c4003f30 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c400bc80 con 0x7f47e00c8200 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.165+0000 7f47df400640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 0x7f47c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47dea00640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e00c8200 0x7f47c40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47d6000640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47c4003f30 0x7f47c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47df400640 1 -- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e00c8200 msgr2=0x7f47c40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47df400640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e00c8200 0x7f47c40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47df400640 1 -- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47c4003f30 msgr2=0x7f47c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47df400640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47c4003f30 0x7f47c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:31.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47df400640 1 -- 172.21.15.113:0/301384101 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4003530 con 0x7f47c4004d50 2024-03-04T08:07:31.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47df400640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 0x7f47c40051d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f47d8001860 tx=0x7f47d801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdcba122906a9c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:31.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8001460 con 0x7f47c4004d50 2024-03-04T08:07:31.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d80b0070 con 0x7f47c4004d50 2024-03-04T08:07:31.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47de000640 1 -- 172.21.15.113:0/301384101 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d80ab880 con 0x7f47c4004d50 2024-03-04T08:07:31.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47e5e00640 1 -- 172.21.15.113:0/301384101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 msgr2=0x7f47c40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:31.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47e5e00640 1 --2- 172.21.15.113:0/301384101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 0x7f47c40051d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f47d8001860 tx=0x7f47d801c500 comp rx=0 tx=0).stop 2024-03-04T08:07:31.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.169+0000 7f47df400640 1 -- 172.21.15.113:0/301384101 reap_dead start 2024-03-04T08:07:31.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 -- 172.21.15.113:0/301384101 shutdown_connections 2024-03-04T08:07:31.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 -- 172.21.15.113:0/301384101 >> 172.21.15.113:0/301384101 conn(0x7f47e006e820 msgr2=0x7f47e006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:31.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 -- 172.21.15.113:0/301384101 shutdown_connections 2024-03-04T08:07:31.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 -- 172.21.15.113:0/301384101 wait complete. 2024-03-04T08:07:31.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 Processor -- start 2024-03-04T08:07:31.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 -- start start 2024-03-04T08:07:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 0x7f47e015bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47e014f5c0 0x7f47e0155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e01556a0 0x7f47e014d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0158ee0 con 0x7f47c4004d50 2024-03-04T08:07:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01597b0 con 0x7f47e014f5c0 2024-03-04T08:07:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47e5e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0070d40 con 0x7f47e01556a0 2024-03-04T08:07:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47dea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 0x7f47e015bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47dea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 0x7f47e015bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46864/0 (socket says 172.21.15.113:46864) 2024-03-04T08:07:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47dea00640 1 -- 172.21.15.113:0/2018887049 learned_addr learned my addr 172.21.15.113:0/2018887049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47d6000640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47e014f5c0 0x7f47e0155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47df400640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e01556a0 0x7f47e014d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47dea00640 1 -- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e01556a0 msgr2=0x7f47e014d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47dea00640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e01556a0 0x7f47e014d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47dea00640 1 -- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47e014f5c0 msgr2=0x7f47e0155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47dea00640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47e014f5c0 0x7f47e0155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47dea00640 1 -- 172.21.15.113:0/2018887049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e01543f0 con 0x7f47c4004d50 2024-03-04T08:07:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.173+0000 7f47dea00640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 0x7f47e015bed0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f47d0011940 tx=0x7f47d0011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:31.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.177+0000 7f47d7e00640 1 -- 172.21.15.113:0/2018887049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0019070 con 0x7f47c4004d50 2024-03-04T08:07:31.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.177+0000 7f47d7e00640 1 -- 172.21.15.113:0/2018887049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d000f960 con 0x7f47c4004d50 2024-03-04T08:07:31.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.177+0000 7f47d7e00640 1 -- 172.21.15.113:0/2018887049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d000fd30 con 0x7f47c4004d50 2024-03-04T08:07:31.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.177+0000 7f47e5e00640 1 -- 172.21.15.113:0/2018887049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e0153350 con 0x7f47c4004d50 2024-03-04T08:07:31.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.177+0000 7f47e5e00640 1 -- 172.21.15.113:0/2018887049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e0153910 con 0x7f47c4004d50 2024-03-04T08:07:31.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.177+0000 7f47e5e00640 1 -- 172.21.15.113:0/2018887049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e0075c00 con 0x7f47c4004d50 2024-03-04T08:07:31.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.177+0000 7f47d7e00640 1 -- 172.21.15.113:0/2018887049 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47d001c070 con 0x7f47c4004d50 2024-03-04T08:07:31.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.177+0000 7f47d7e00640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f47bc066f60 0x7f47bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:31.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.177+0000 7f47d7e00640 1 -- 172.21.15.113:0/2018887049 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f47d0088970 con 0x7f47c4004d50 2024-03-04T08:07:31.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.181+0000 7f47d6000640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f47bc066f60 0x7f47bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:31.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.181+0000 7f47d6000640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f47bc066f60 0x7f47bc069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f47e01f28d0 tx=0x7f47c8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:31.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.181+0000 7f47d7e00640 1 -- 172.21.15.113:0/2018887049 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f47d0058bb0 con 0x7f47c4004d50 2024-03-04T08:07:31.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.397+0000 7f47e5e00640 1 -- 172.21.15.113:0/2018887049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47e0073610 con 0x7f47c4004d50 2024-03-04T08:07:31.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.397+0000 7f47d7e00640 1 -- 172.21.15.113:0/2018887049 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f47d005cc50 con 0x7f47c4004d50 2024-03-04T08:07:31.397 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:31.397 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":126,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:30.861349+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7119,7269]},"id":1}]} 2024-03-04T08:07:31.397 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 126 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 -- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f47bc066f60 msgr2=0x7f47bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f47bc066f60 0x7f47bc069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f47e01f28d0 tx=0x7f47c8009190 comp rx=0 tx=0).stop 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 -- 172.21.15.113:0/2018887049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 msgr2=0x7f47e015bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 0x7f47e015bed0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f47d0011940 tx=0x7f47d0011d40 comp rx=0 tx=0).stop 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 -- 172.21.15.113:0/2018887049 shutdown_connections 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f47e01556a0 0x7f47e014d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47e014f5c0 0x7f47e0155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f47bc066f60 0x7f47bc069420 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 --2- 172.21.15.113:0/2018887049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c4004d50 0x7f47e015bed0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 -- 172.21.15.113:0/2018887049 >> 172.21.15.113:0/2018887049 conn(0x7f47e006e820 msgr2=0x7f47e01563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 -- 172.21.15.113:0/2018887049 shutdown_connections 2024-03-04T08:07:31.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:31.401+0000 7f47e5e00640 1 -- 172.21.15.113:0/2018887049 wait complete. 2024-03-04T08:07:31.419 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 126, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 126, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7119": { "addr": "172.21.15.113:6833/3825257303", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 3825257303, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 3825257303, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7119, "incarnation": 114, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 25 }, "gid_7269": { "addr": "172.21.15.113:6837/1401330284", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1401330284, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1401330284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7269, "incarnation": 120, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7389": { "addr": "172.21.15.113:6836/1310057122", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1310057122, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1310057122, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7389, "incarnation": 126, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 67, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:07:30.861349+0000", "qdb_cluster": [ 7119, 7269 ], "qdb_leader": 7119, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7389, "mds_1": 7119, "mds_2": 7269 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:07:33.420 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:33.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b4b800640 1 Processor -- start 2024-03-04T08:07:33.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b4b800640 1 -- start start 2024-03-04T08:07:33.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b4b800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b441527f0 0x7f1b44070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:33.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b4b800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1b44152bd0 0x7f1b441587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:33.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b4b800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b441591f0 con 0x7f1b440c8200 2024-03-04T08:07:33.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b4b800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b44071df0 con 0x7f1b441527f0 2024-03-04T08:07:33.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b4b800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b441597b0 con 0x7f1b44152bd0 2024-03-04T08:07:33.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b43e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1b440c8200 0x7f1b44072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52850/0 (socket says 172.21.15.113:52850) 2024-03-04T08:07:33.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b43e00640 1 -- 172.21.15.113:0/3789141328 learned_addr learned my addr 172.21.15.113:0/3789141328 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:33.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b48e00640 1 --2- 172.21.15.113:0/3789141328 >> v2:172.21.15.113:3300/0 conn(0x7f1b441527f0 0x7f1b44070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:33.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b49800640 1 --2- 172.21.15.113:0/3789141328 >> v2:172.21.15.136:3300/0 conn(0x7f1b44152bd0 0x7f1b441587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:33.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b48e00640 1 -- 172.21.15.113:0/3789141328 >> v2:172.21.15.136:3300/0 conn(0x7f1b44152bd0 msgr2=0x7f1b441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:33.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b48e00640 1 --2- 172.21.15.113:0/3789141328 >> v2:172.21.15.136:3300/0 conn(0x7f1b44152bd0 0x7f1b441587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:33.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b48e00640 1 -- 172.21.15.113:0/3789141328 >> v1:172.21.15.113:6789/0 conn(0x7f1b440c8200 legacy=0x7f1b44072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:33.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.561+0000 7f1b48e00640 1 -- 172.21.15.113:0/3789141328 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b441594d0 con 0x7f1b441527f0 2024-03-04T08:07:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b48e00640 1 --2- 172.21.15.113:0/3789141328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 0x7f1b44070d40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1b440712b0 tx=0x7f1b3801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df7a94fe53c481c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b43400640 1 -- 172.21.15.113:0/3789141328 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b380b1070 con 0x7f1b441527f0 2024-03-04T08:07:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b43400640 1 -- 172.21.15.113:0/3789141328 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b380ac5f0 con 0x7f1b441527f0 2024-03-04T08:07:33.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b43400640 1 -- 172.21.15.113:0/3789141328 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b380ac9c0 con 0x7f1b441527f0 2024-03-04T08:07:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b4b800640 1 -- 172.21.15.113:0/3789141328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 msgr2=0x7f1b44070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b4b800640 1 --2- 172.21.15.113:0/3789141328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 0x7f1b44070d40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1b440712b0 tx=0x7f1b3801c480 comp rx=0 tx=0).stop 2024-03-04T08:07:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b4b800640 1 -- 172.21.15.113:0/3789141328 shutdown_connections 2024-03-04T08:07:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b4b800640 1 --2- 172.21.15.113:0/3789141328 >> v2:172.21.15.136:3300/0 conn(0x7f1b44152bd0 0x7f1b441587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b4b800640 1 --2- 172.21.15.113:0/3789141328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 0x7f1b44070d40 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:33.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b4b800640 1 -- 172.21.15.113:0/3789141328 >> 172.21.15.113:0/3789141328 conn(0x7f1b4406e820 msgr2=0x7f1b4406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:33.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b4b800640 1 -- 172.21.15.113:0/3789141328 shutdown_connections 2024-03-04T08:07:33.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.565+0000 7f1b4b800640 1 -- 172.21.15.113:0/3789141328 wait complete. 2024-03-04T08:07:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b4b800640 1 Processor -- start 2024-03-04T08:07:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b4b800640 1 -- start start 2024-03-04T08:07:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b4b800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b440c8200 0x7f1b440c2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b4b800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 0x7f1b44154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:33.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b4b800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b44152bd0 0x7f1b44155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b4b800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b441654f0 con 0x7f1b441527f0 2024-03-04T08:07:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b4b800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b44165370 con 0x7f1b44152bd0 2024-03-04T08:07:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b4b800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b44165670 con 0x7f1b440c8200 2024-03-04T08:07:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b43e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 0x7f1b44154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b43e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 0x7f1b44154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46896/0 (socket says 172.21.15.113:46896) 2024-03-04T08:07:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b43e00640 1 -- 172.21.15.113:0/2762168072 learned_addr learned my addr 172.21.15.113:0/2762168072 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b48e00640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b440c8200 0x7f1b440c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b49800640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b44152bd0 0x7f1b44155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b43e00640 1 -- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b440c8200 msgr2=0x7f1b440c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b43e00640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b440c8200 0x7f1b440c2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b43e00640 1 -- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b44152bd0 msgr2=0x7f1b44155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b43e00640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b44152bd0 0x7f1b44155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b43e00640 1 -- 172.21.15.113:0/2762168072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b440b1970 con 0x7f1b441527f0 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b49800640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b44152bd0 0x7f1b44155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b43e00640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 0x7f1b44154dc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000c430 tx=0x7f1b3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b40c00640 1 -- 172.21.15.113:0/2762168072 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b30017070 con 0x7f1b441527f0 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b40c00640 1 -- 172.21.15.113:0/2762168072 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b30005cf0 con 0x7f1b441527f0 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b40c00640 1 -- 172.21.15.113:0/2762168072 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b300072f0 con 0x7f1b441527f0 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b4b800640 1 -- 172.21.15.113:0/2762168072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b440afbc0 con 0x7f1b441527f0 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.569+0000 7f1b4b800640 1 -- 172.21.15.113:0/2762168072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b441542e0 con 0x7f1b441527f0 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.573+0000 7f1b4b800640 1 -- 172.21.15.113:0/2762168072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b44075c00 con 0x7f1b441527f0 2024-03-04T08:07:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.573+0000 7f1b40c00640 1 -- 172.21.15.113:0/2762168072 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b3001a070 con 0x7f1b441527f0 2024-03-04T08:07:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.573+0000 7f1b40c00640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b24066f60 0x7f1b24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.573+0000 7f1b40c00640 1 -- 172.21.15.113:0/2762168072 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f1b30084b70 con 0x7f1b441527f0 2024-03-04T08:07:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.573+0000 7f1b48e00640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b24066f60 0x7f1b24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:33.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.577+0000 7f1b48e00640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b24066f60 0x7f1b24069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1b380ae000 tx=0x7f1b38008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:33.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.577+0000 7f1b40c00640 1 -- 172.21.15.113:0/2762168072 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1b30055400 con 0x7f1b441527f0 2024-03-04T08:07:33.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.773+0000 7f1b4b800640 1 -- 172.21.15.113:0/2762168072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b440c79d0 con 0x7f1b441527f0 2024-03-04T08:07:33.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.773+0000 7f1b40c00640 1 -- 172.21.15.113:0/2762168072 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f1b30054d00 con 0x7f1b441527f0 2024-03-04T08:07:33.774 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:33.774 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":126,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:30.861349+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7119,7269]},"id":1}]} 2024-03-04T08:07:33.775 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 126 2024-03-04T08:07:33.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 -- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b24066f60 msgr2=0x7f1b24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b24066f60 0x7f1b24069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1b380ae000 tx=0x7f1b38008040 comp rx=0 tx=0).stop 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 -- 172.21.15.113:0/2762168072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 msgr2=0x7f1b44154dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 0x7f1b44154dc0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000c430 tx=0x7f1b3000c8f0 comp rx=0 tx=0).stop 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 -- 172.21.15.113:0/2762168072 shutdown_connections 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b44152bd0 0x7f1b44155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b24066f60 0x7f1b24069420 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b441527f0 0x7f1b44154dc0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 --2- 172.21.15.113:0/2762168072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b440c8200 0x7f1b440c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 -- 172.21.15.113:0/2762168072 >> 172.21.15.113:0/2762168072 conn(0x7f1b4406e820 msgr2=0x7f1b440c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 -- 172.21.15.113:0/2762168072 shutdown_connections 2024-03-04T08:07:33.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:33.777+0000 7f1b4b800640 1 -- 172.21.15.113:0/2762168072 wait complete. 2024-03-04T08:07:33.796 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:07:35.179 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:07:35.184 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:07:35.797 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.992+0000 7fb4d0e00640 1 Processor -- start 2024-03-04T08:07:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.992+0000 7fb4d0e00640 1 -- start start 2024-03-04T08:07:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4d0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4cc06cc20 0x7fb4cc159de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4d0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb4cc0cd010 0x7fb4cc0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4d0e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc06b460 con 0x7fb4cc152a90 2024-03-04T08:07:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4d0e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc06b740 con 0x7fb4cc0cd010 2024-03-04T08:07:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4d0e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc072660 con 0x7fb4cc06cc20 2024-03-04T08:07:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4ca800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb4cc0cd010 0x7fb4cc0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4ca800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb4cc0cd010 0x7fb4cc0cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46906/0 (socket says 172.21.15.113:46906) 2024-03-04T08:07:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4ca800640 1 -- 172.21.15.113:0/1761338777 learned_addr learned my addr 172.21.15.113:0/1761338777 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:35.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4c9400640 1 --2- 172.21.15.113:0/1761338777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4cc06cc20 0x7fb4cc159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4ca800640 1 -- 172.21.15.113:0/1761338777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4cc06cc20 msgr2=0x7fb4cc159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4ca800640 1 --2- 172.21.15.113:0/1761338777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4cc06cc20 0x7fb4cc159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4ca800640 1 -- 172.21.15.113:0/1761338777 >> v1:172.21.15.136:6789/0 conn(0x7fb4cc152a90 legacy=0x7fb4cc159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:35.996+0000 7fb4ca800640 1 -- 172.21.15.113:0/1761338777 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc072940 con 0x7fb4cc0cd010 2024-03-04T08:07:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4ca800640 1 --2- 172.21.15.113:0/1761338777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc0cd010 0x7fb4cc0cd4a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009e50 tx=0x7fb4c401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41d5708b407db9c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4c8a00640 1 -- 172.21.15.113:0/1761338777 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40b4070 con 0x7fb4cc0cd010 2024-03-04T08:07:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4c8a00640 1 -- 172.21.15.113:0/1761338777 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c40af400 con 0x7fb4cc0cd010 2024-03-04T08:07:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4c8a00640 1 -- 172.21.15.113:0/1761338777 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40af7d0 con 0x7fb4cc0cd010 2024-03-04T08:07:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1761338777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc0cd010 msgr2=0x7fb4cc0cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4d0e00640 1 --2- 172.21.15.113:0/1761338777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc0cd010 0x7fb4cc0cd4a0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009e50 tx=0x7fb4c401f210 comp rx=0 tx=0).stop 2024-03-04T08:07:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1761338777 shutdown_connections 2024-03-04T08:07:36.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4d0e00640 1 --2- 172.21.15.113:0/1761338777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc0cd010 0x7fb4cc0cd4a0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:36.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4d0e00640 1 --2- 172.21.15.113:0/1761338777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4cc06cc20 0x7fb4cc159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:36.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1761338777 >> 172.21.15.113:0/1761338777 conn(0x7fb4cc0671e0 msgr2=0x7fb4cc06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:36.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1761338777 shutdown_connections 2024-03-04T08:07:36.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1761338777 wait complete. 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.000+0000 7fb4d0e00640 1 Processor -- start 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4d0e00640 1 -- start start 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4d0e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc06cc20 0x7fb4cc14d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4d0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4cc0cd010 0x7fb4cc157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4d0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4cc152a90 0x7fb4cc1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4d0e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc073420 con 0x7fb4cc06cc20 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4d0e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc072f00 con 0x7fb4cc0cd010 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4d0e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc0735a0 con 0x7fb4cc152a90 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc06cc20 0x7fb4cc14d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc06cc20 0x7fb4cc14d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46920/0 (socket says 172.21.15.113:46920) 2024-03-04T08:07:36.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9e00640 1 -- 172.21.15.113:0/1242276641 learned_addr learned my addr 172.21.15.113:0/1242276641 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4ca800640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4cc152a90 0x7fb4cc1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9400640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4cc0cd010 0x7fb4cc157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9e00640 1 -- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4cc152a90 msgr2=0x7fb4cc1583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9e00640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4cc152a90 0x7fb4cc1583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9e00640 1 -- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4cc0cd010 msgr2=0x7fb4cc157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9e00640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4cc0cd010 0x7fb4cc157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9e00640 1 -- 172.21.15.113:0/1242276641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0c3b20 con 0x7fb4cc06cc20 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9400640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4cc0cd010 0x7fb4cc157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c9e00640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc06cc20 0x7fb4cc14d790 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00c370 tx=0x7fb4bc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c2000640 1 -- 172.21.15.113:0/1242276641 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0052d0 con 0x7fb4cc06cc20 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c2000640 1 -- 172.21.15.113:0/1242276641 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc005470 con 0x7fb4cc06cc20 2024-03-04T08:07:36.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1242276641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc0c3dc0 con 0x7fb4cc06cc20 2024-03-04T08:07:36.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4c2000640 1 -- 172.21.15.113:0/1242276641 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc007260 con 0x7fb4cc06cc20 2024-03-04T08:07:36.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.004+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1242276641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc0b0b90 con 0x7fb4cc06cc20 2024-03-04T08:07:36.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.008+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1242276641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc075c00 con 0x7fb4cc06cc20 2024-03-04T08:07:36.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.008+0000 7fb4c2000640 1 -- 172.21.15.113:0/1242276641 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4bc007400 con 0x7fb4cc06cc20 2024-03-04T08:07:36.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.008+0000 7fb4c2000640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4ac066f60 0x7fb4ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:36.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.008+0000 7fb4c2000640 1 -- 172.21.15.113:0/1242276641 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7fb4bc084b70 con 0x7fb4cc06cc20 2024-03-04T08:07:36.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.008+0000 7fb4c9400640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4ac066f60 0x7fb4ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:36.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.008+0000 7fb4c9400640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4ac066f60 0x7fb4ac069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0b18e0 tx=0x7fb4b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:36.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.012+0000 7fb4c2000640 1 -- 172.21.15.113:0/1242276641 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb4bc054db0 con 0x7fb4cc06cc20 2024-03-04T08:07:36.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1242276641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4cc0b2520 con 0x7fb4cc06cc20 2024-03-04T08:07:36.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4c2000640 1 -- 172.21.15.113:0/1242276641 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+5823 (secure 0 0 0) 0x7fb4bc058e50 con 0x7fb4cc06cc20 2024-03-04T08:07:36.252 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:36.252 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":126,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:30.861349+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7119,7269]},"id":1}]} 2024-03-04T08:07:36.252 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 126 2024-03-04T08:07:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4ac066f60 msgr2=0x7fb4ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4ac066f60 0x7fb4ac069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0b18e0 tx=0x7fb4b8008040 comp rx=0 tx=0).stop 2024-03-04T08:07:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1242276641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc06cc20 msgr2=0x7fb4cc14d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc06cc20 0x7fb4cc14d790 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00c370 tx=0x7fb4bc00c830 comp rx=0 tx=0).stop 2024-03-04T08:07:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1242276641 shutdown_connections 2024-03-04T08:07:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4cc152a90 0x7fb4cc1583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4cc0cd010 0x7fb4cc157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4ac066f60 0x7fb4ac069420 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:36.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 --2- 172.21.15.113:0/1242276641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4cc06cc20 0x7fb4cc14d790 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:36.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1242276641 >> 172.21.15.113:0/1242276641 conn(0x7fb4cc0671e0 msgr2=0x7fb4cc074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:36.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1242276641 shutdown_connections 2024-03-04T08:07:36.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:36.252+0000 7fb4d0e00640 1 -- 172.21.15.113:0/1242276641 wait complete. 2024-03-04T08:07:36.283 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:07:38.284 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:38.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a4d000640 1 Processor -- start 2024-03-04T08:07:38.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a4d000640 1 -- start start 2024-03-04T08:07:38.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a4d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a480ca1a0 0x7f2a480b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a4d000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a480ca580 0x7f2a4815e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a4d000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a480b3530 con 0x7f2a4815e690 2024-03-04T08:07:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a4d000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a48067bc0 con 0x7f2a480ca580 2024-03-04T08:07:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a4d000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4815bb40 con 0x7f2a480ca1a0 2024-03-04T08:07:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a45400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a480ca580 0x7f2a4815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a45e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a480ca1a0 0x7f2a480b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a45400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a480ca580 0x7f2a4815e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46106/0 (socket says 172.21.15.113:46106) 2024-03-04T08:07:38.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a45400640 1 -- 172.21.15.113:0/732855650 learned_addr learned my addr 172.21.15.113:0/732855650 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a45400640 1 -- 172.21.15.113:0/732855650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a480ca1a0 msgr2=0x7f2a480b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a45400640 1 --2- 172.21.15.113:0/732855650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a480ca1a0 0x7f2a480b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a45400640 1 -- 172.21.15.113:0/732855650 >> v1:172.21.15.136:6789/0 conn(0x7f2a4815e690 legacy=0x7f2a480cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:38.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.456+0000 7f2a45400640 1 -- 172.21.15.113:0/732855650 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480678e0 con 0x7f2a480ca580 2024-03-04T08:07:38.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.460+0000 7f2a45400640 1 --2- 172.21.15.113:0/732855650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 0x7f2a4815e150 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2a340098a0 tx=0x7f2a3401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea22d17c7f4705ca server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:38.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.460+0000 7f2a44a00640 1 -- 172.21.15.113:0/732855650 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340b5070 con 0x7f2a480ca580 2024-03-04T08:07:38.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.460+0000 7f2a44a00640 1 -- 172.21.15.113:0/732855650 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a340a8a30 con 0x7f2a480ca580 2024-03-04T08:07:38.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.460+0000 7f2a44a00640 1 -- 172.21.15.113:0/732855650 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340a8e00 con 0x7f2a480ca580 2024-03-04T08:07:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 -- 172.21.15.113:0/732855650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 msgr2=0x7f2a4815e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 --2- 172.21.15.113:0/732855650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 0x7f2a4815e150 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f2a340098a0 tx=0x7f2a3401ed30 comp rx=0 tx=0).stop 2024-03-04T08:07:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 -- 172.21.15.113:0/732855650 shutdown_connections 2024-03-04T08:07:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 --2- 172.21.15.113:0/732855650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 0x7f2a4815e150 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 --2- 172.21.15.113:0/732855650 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a480ca1a0 0x7f2a480b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 -- 172.21.15.113:0/732855650 >> 172.21.15.113:0/732855650 conn(0x7f2a4806dac0 msgr2=0x7f2a48066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 -- 172.21.15.113:0/732855650 shutdown_connections 2024-03-04T08:07:38.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 -- 172.21.15.113:0/732855650 wait complete. 2024-03-04T08:07:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 Processor -- start 2024-03-04T08:07:38.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 -- start start 2024-03-04T08:07:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a480ca1a0 0x7f2a4815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 0x7f2a4815dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:38.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a4815e690 0x7f2a480ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4815b340 con 0x7f2a480ca580 2024-03-04T08:07:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4815b060 con 0x7f2a480ca1a0 2024-03-04T08:07:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a4d000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4815b620 con 0x7f2a4815e690 2024-03-04T08:07:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a45400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 0x7f2a4815dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a45400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 0x7f2a4815dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46114/0 (socket says 172.21.15.113:46114) 2024-03-04T08:07:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a45400640 1 -- 172.21.15.113:0/4261997595 learned_addr learned my addr 172.21.15.113:0/4261997595 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a45e00640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a480ca1a0 0x7f2a4815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.464+0000 7f2a46800640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a4815e690 0x7f2a480ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a45400640 1 -- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a4815e690 msgr2=0x7f2a480ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:38.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a45400640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a4815e690 0x7f2a480ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:38.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a45400640 1 -- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a480ca1a0 msgr2=0x7f2a4815d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:38.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a45400640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a480ca1a0 0x7f2a4815d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:38.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a45400640 1 -- 172.21.15.113:0/4261997595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a481f4720 con 0x7f2a480ca580 2024-03-04T08:07:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a45400640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 0x7f2a4815dc50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2a3401b940 tx=0x7f2a3401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a3e000640 1 -- 172.21.15.113:0/4261997595 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340b5040 con 0x7f2a480ca580 2024-03-04T08:07:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a3e000640 1 -- 172.21.15.113:0/4261997595 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a340be070 con 0x7f2a480ca580 2024-03-04T08:07:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a4d000640 1 -- 172.21.15.113:0/4261997595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a481f49c0 con 0x7f2a480ca580 2024-03-04T08:07:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a3e000640 1 -- 172.21.15.113:0/4261997595 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340b1a70 con 0x7f2a480ca580 2024-03-04T08:07:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a4d000640 1 -- 172.21.15.113:0/4261997595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a480d0080 con 0x7f2a480ca580 2024-03-04T08:07:38.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.468+0000 7f2a4d000640 1 -- 172.21.15.113:0/4261997595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a48075c00 con 0x7f2a480ca580 2024-03-04T08:07:38.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.472+0000 7f2a3e000640 1 -- 172.21.15.113:0/4261997595 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a340b1c10 con 0x7f2a480ca580 2024-03-04T08:07:38.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.472+0000 7f2a3e000640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a28066f60 0x7f2a28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:38.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.472+0000 7f2a3e000640 1 -- 172.21.15.113:0/4261997595 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f2a34122a10 con 0x7f2a480ca580 2024-03-04T08:07:38.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.472+0000 7f2a45e00640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a28066f60 0x7f2a28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:38.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.472+0000 7f2a45e00640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a28066f60 0x7f2a28069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009af0 tx=0x7f2a38008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:38.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.480+0000 7f2a3e000640 1 -- 172.21.15.113:0/4261997595 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2a34018890 con 0x7f2a480ca580 2024-03-04T08:07:38.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.736+0000 7f2a4d000640 1 -- 172.21.15.113:0/4261997595 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a480cbdd0 con 0x7f2a480ca580 2024-03-04T08:07:38.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.736+0000 7f2a3e000640 1 -- 172.21.15.113:0/4261997595 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f2a340f3330 con 0x7f2a480ca580 2024-03-04T08:07:38.737 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:38.737 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":126,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:30.861349+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7119,7269]},"id":1}]} 2024-03-04T08:07:38.737 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 126 2024-03-04T08:07:38.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 -- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a28066f60 msgr2=0x7f2a28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:38.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a28066f60 0x7f2a28069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009af0 tx=0x7f2a38008040 comp rx=0 tx=0).stop 2024-03-04T08:07:38.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 -- 172.21.15.113:0/4261997595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 msgr2=0x7f2a4815dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:38.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 0x7f2a4815dc50 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f2a3401b940 tx=0x7f2a3401bad0 comp rx=0 tx=0).stop 2024-03-04T08:07:38.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 -- 172.21.15.113:0/4261997595 shutdown_connections 2024-03-04T08:07:38.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a4815e690 0x7f2a480ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:38.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a28066f60 0x7f2a28069420 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:38.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a480ca580 0x7f2a4815dc50 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:38.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 --2- 172.21.15.113:0/4261997595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a480ca1a0 0x7f2a4815d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:38.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 -- 172.21.15.113:0/4261997595 >> 172.21.15.113:0/4261997595 conn(0x7f2a4806dac0 msgr2=0x7f2a480b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:38.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 -- 172.21.15.113:0/4261997595 shutdown_connections 2024-03-04T08:07:38.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:38.740+0000 7f2a4d000640 1 -- 172.21.15.113:0/4261997595 wait complete. 2024-03-04T08:07:38.763 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:07:40.764 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:40.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95e4a00640 1 Processor -- start 2024-03-04T08:07:40.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95e4a00640 1 -- start start 2024-03-04T08:07:40.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95e4a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f95e014e4c0 0x7f95e0072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:40.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95e4a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e00c27f0 con 0x7f95e015b900 2024-03-04T08:07:40.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95e4a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e00c2a90 con 0x7f95e00c22e0 2024-03-04T08:07:40.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95e4a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e015ad60 con 0x7f95e014e4c0 2024-03-04T08:07:40.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95de800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f95e015b900 0x7f95e01620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52334/0 (socket says 172.21.15.113:52334) 2024-03-04T08:07:40.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95de800640 1 -- 172.21.15.113:0/927934776 learned_addr learned my addr 172.21.15.113:0/927934776 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:40.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dde00640 1 --2- 172.21.15.113:0/927934776 >> v2:172.21.15.136:3300/0 conn(0x7f95e014e4c0 0x7f95e0072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:40.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405356379 0 0) 0x7f95e00c27f0 con 0x7f95e015b900 2024-03-04T08:07:40.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c4003530 con 0x7f95e015b900 2024-03-04T08:07:40.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 17679432 0 0) 0x7f95c4003530 con 0x7f95e015b900 2024-03-04T08:07:40.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 >> v2:172.21.15.136:3300/0 conn(0x7f95e014e4c0 msgr2=0x7f95e0072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:40.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 --2- 172.21.15.113:0/927934776 >> v2:172.21.15.136:3300/0 conn(0x7f95e014e4c0 0x7f95e0072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:40.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 >> v1:172.21.15.136:6789/0 conn(0x7f95e00c22e0 legacy=0x7f95e015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:40.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e0070c60 con 0x7f95e015b900 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f95d8001a00 con 0x7f95e015b900 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 >> v1:172.21.15.113:6789/0 conn(0x7f95e015b900 legacy=0x7f95e01620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95c40039c0 0x7f95c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95c4004810 0x7f95c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c40051d0 0x7f95c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c400c050 con 0x7f95c40051d0 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c400c200 con 0x7f95c4004810 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c400c400 con 0x7f95c40039c0 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95de800640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c40051d0 0x7f95c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.912+0000 7f95dde00640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95c40039c0 0x7f95c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.916+0000 7f95dd400640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95c4004810 0x7f95c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.916+0000 7f95de800640 1 -- 172.21.15.113:0/927934776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95c40039c0 msgr2=0x7f95c4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.916+0000 7f95de800640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95c40039c0 0x7f95c4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:40.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.916+0000 7f95de800640 1 -- 172.21.15.113:0/927934776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95c4004810 msgr2=0x7f95c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:40.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.916+0000 7f95de800640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95c4004810 0x7f95c4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:40.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.916+0000 7f95de800640 1 -- 172.21.15.113:0/927934776 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4003530 con 0x7f95c40051d0 2024-03-04T08:07:40.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95de800640 1 -- 172.21.15.113:0/927934776 reap_dead start 2024-03-04T08:07:40.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95de800640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c40051d0 0x7f95c400b9d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f95d800a840 tx=0x7f95d801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9e060800c43865 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:40.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d800a460 con 0x7f95c40051d0 2024-03-04T08:07:40.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d80ac640 con 0x7f95c40051d0 2024-03-04T08:07:40.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95dca00640 1 -- 172.21.15.113:0/927934776 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d80aca30 con 0x7f95c40051d0 2024-03-04T08:07:40.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95e4a00640 1 -- 172.21.15.113:0/927934776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c40051d0 msgr2=0x7f95c400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:40.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95e4a00640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c40051d0 0x7f95c400b9d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f95d800a840 tx=0x7f95d801d360 comp rx=0 tx=0).stop 2024-03-04T08:07:40.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95e4a00640 1 -- 172.21.15.113:0/927934776 shutdown_connections 2024-03-04T08:07:40.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95e4a00640 1 --2- 172.21.15.113:0/927934776 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95c40051d0 0x7f95c400b9d0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:40.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95e4a00640 1 -- 172.21.15.113:0/927934776 >> 172.21.15.113:0/927934776 conn(0x7f95e006e820 msgr2=0x7f95e006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:40.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95e4a00640 1 -- 172.21.15.113:0/927934776 shutdown_connections 2024-03-04T08:07:40.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.920+0000 7f95e4a00640 1 -- 172.21.15.113:0/927934776 wait complete. 2024-03-04T08:07:40.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95e4a00640 1 Processor -- start 2024-03-04T08:07:40.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95e4a00640 1 -- start start 2024-03-04T08:07:40.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95e4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95c40051d0 0x7f95e015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:40.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95e4a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e013da70 0x7f95e013deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:40.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95e4a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95e01553b0 0x7f95e0155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:40.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95e4a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0071500 con 0x7f95e013da70 2024-03-04T08:07:40.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95e4a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e0071220 con 0x7f95c40051d0 2024-03-04T08:07:40.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95e4a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e014e8a0 con 0x7f95e01553b0 2024-03-04T08:07:40.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95dd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e013da70 0x7f95e013deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:40.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95dd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e013da70 0x7f95e013deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46138/0 (socket says 172.21.15.113:46138) 2024-03-04T08:07:40.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95dd400640 1 -- 172.21.15.113:0/1754454360 learned_addr learned my addr 172.21.15.113:0/1754454360 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:40.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95dde00640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95c40051d0 0x7f95e015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:40.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95de800640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95e01553b0 0x7f95e0155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:40.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95dd400640 1 -- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95e01553b0 msgr2=0x7f95e0155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:40.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95dd400640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95e01553b0 0x7f95e0155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:40.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95dd400640 1 -- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95c40051d0 msgr2=0x7f95e015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:40.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95dd400640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95c40051d0 0x7f95e015a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:40.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95dd400640 1 -- 172.21.15.113:0/1754454360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e01f48d0 con 0x7f95e013da70 2024-03-04T08:07:40.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.924+0000 7f95dd400640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e013da70 0x7f95e013deb0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00f5f0 tx=0x7f95cc00f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:40.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.928+0000 7f95d6000640 1 -- 172.21.15.113:0/1754454360 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc0052e0 con 0x7f95e013da70 2024-03-04T08:07:40.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.928+0000 7f95d6000640 1 -- 172.21.15.113:0/1754454360 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95cc018070 con 0x7f95e013da70 2024-03-04T08:07:40.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.928+0000 7f95e4a00640 1 -- 172.21.15.113:0/1754454360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e01f4b70 con 0x7f95e013da70 2024-03-04T08:07:40.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.928+0000 7f95d6000640 1 -- 172.21.15.113:0/1754454360 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc005480 con 0x7f95e013da70 2024-03-04T08:07:40.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.928+0000 7f95e4a00640 1 -- 172.21.15.113:0/1754454360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e0148910 con 0x7f95e013da70 2024-03-04T08:07:40.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.928+0000 7f95e4a00640 1 -- 172.21.15.113:0/1754454360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e0075c00 con 0x7f95e013da70 2024-03-04T08:07:40.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.928+0000 7f95d6000640 1 -- 172.21.15.113:0/1754454360 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95cc003820 con 0x7f95e013da70 2024-03-04T08:07:40.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.932+0000 7f95d6000640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95c0066f60 0x7f95c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:40.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.932+0000 7f95d6000640 1 -- 172.21.15.113:0/1754454360 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f95cc085260 con 0x7f95e013da70 2024-03-04T08:07:40.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.932+0000 7f95dde00640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95c0066f60 0x7f95c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:40.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.932+0000 7f95dde00640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95c0066f60 0x7f95c0069420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009b60 tx=0x7f95d0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:40.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:40.936+0000 7f95d6000640 1 -- 172.21.15.113:0/1754454360 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f95cc055c20 con 0x7f95e013da70 2024-03-04T08:07:41.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.136+0000 7f95e4a00640 1 -- 172.21.15.113:0/1754454360 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95e0155190 con 0x7f95e013da70 2024-03-04T08:07:41.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.136+0000 7f95d6000640 1 -- 172.21.15.113:0/1754454360 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f95cc055520 con 0x7f95e013da70 2024-03-04T08:07:41.137 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:41.138 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":126,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:30.861349+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7119,7269]},"id":1}]} 2024-03-04T08:07:41.138 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 126 2024-03-04T08:07:41.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 -- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95c0066f60 msgr2=0x7f95c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:41.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95c0066f60 0x7f95c0069420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009b60 tx=0x7f95d0009190 comp rx=0 tx=0).stop 2024-03-04T08:07:41.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 -- 172.21.15.113:0/1754454360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e013da70 msgr2=0x7f95e013deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:41.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e013da70 0x7f95e013deb0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00f5f0 tx=0x7f95cc00f9f0 comp rx=0 tx=0).stop 2024-03-04T08:07:41.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 -- 172.21.15.113:0/1754454360 shutdown_connections 2024-03-04T08:07:41.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95e01553b0 0x7f95e0155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:41.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95c0066f60 0x7f95c0069420 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:41.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95e013da70 0x7f95e013deb0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:41.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 --2- 172.21.15.113:0/1754454360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95c40051d0 0x7f95e015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:41.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 -- 172.21.15.113:0/1754454360 >> 172.21.15.113:0/1754454360 conn(0x7f95e006e820 msgr2=0x7f95e00c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:41.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 -- 172.21.15.113:0/1754454360 shutdown_connections 2024-03-04T08:07:41.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:41.140+0000 7f95e4a00640 1 -- 172.21.15.113:0/1754454360 wait complete. 2024-03-04T08:07:41.158 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:07:43.158 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:43.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.316+0000 7f6266800640 1 Processor -- start 2024-03-04T08:07:43.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.316+0000 7f6266800640 1 -- start start 2024-03-04T08:07:43.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f6266800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62600c8b90 0x7f62601545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:43.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f6266800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626015dde0 con 0x7f62600ce590 2024-03-04T08:07:43.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f6266800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626015e080 con 0x7f62600ce970 2024-03-04T08:07:43.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f6266800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626015e360 con 0x7f62600c8b90 2024-03-04T08:07:43.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f6265400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f62600ce590 0x7f626006fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52336/0 (socket says 172.21.15.113:52336) 2024-03-04T08:07:43.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f6265400640 1 -- 172.21.15.113:0/658726711 learned_addr learned my addr 172.21.15.113:0/658726711 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:43.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f6264a00640 1 --2- 172.21.15.113:0/658726711 >> v2:172.21.15.113:3300/0 conn(0x7f62600c8b90 0x7f62601545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:43.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f6264a00640 1 -- 172.21.15.113:0/658726711 >> v1:172.21.15.136:6789/0 conn(0x7f62600ce970 legacy=0x7f62600c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:43.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f625fe00640 1 -- 172.21.15.113:0/658726711 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155549702 0 0) 0x7f626015dde0 con 0x7f62600ce590 2024-03-04T08:07:43.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f6264a00640 1 -- 172.21.15.113:0/658726711 >> v1:172.21.15.113:6789/0 conn(0x7f62600ce590 legacy=0x7f626006fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:43.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.320+0000 7f6264a00640 1 -- 172.21.15.113:0/658726711 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626015e640 con 0x7f62600c8b90 2024-03-04T08:07:43.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f6264a00640 1 --2- 172.21.15.113:0/658726711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 0x7f62601545e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f624c0097c0 tx=0x7f624c01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5484071ffe051ce server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:43.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f625fe00640 1 -- 172.21.15.113:0/658726711 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c0b5070 con 0x7f62600c8b90 2024-03-04T08:07:43.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f625fe00640 1 -- 172.21.15.113:0/658726711 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c0a8990 con 0x7f62600c8b90 2024-03-04T08:07:43.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f625fe00640 1 -- 172.21.15.113:0/658726711 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c0a8d60 con 0x7f62600c8b90 2024-03-04T08:07:43.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f6266800640 1 -- 172.21.15.113:0/658726711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 msgr2=0x7f62601545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:43.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f6266800640 1 --2- 172.21.15.113:0/658726711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 0x7f62601545e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f624c0097c0 tx=0x7f624c01ec90 comp rx=0 tx=0).stop 2024-03-04T08:07:43.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f6266800640 1 -- 172.21.15.113:0/658726711 shutdown_connections 2024-03-04T08:07:43.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f6266800640 1 --2- 172.21.15.113:0/658726711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 0x7f62601545e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:43.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f6266800640 1 -- 172.21.15.113:0/658726711 >> 172.21.15.113:0/658726711 conn(0x7f6260066f80 msgr2=0x7f6260067390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:43.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f6266800640 1 -- 172.21.15.113:0/658726711 shutdown_connections 2024-03-04T08:07:43.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.324+0000 7f6266800640 1 -- 172.21.15.113:0/658726711 wait complete. 2024-03-04T08:07:43.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 Processor -- start 2024-03-04T08:07:43.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 -- start start 2024-03-04T08:07:43.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 0x7f6260150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:43.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62600ce590 0x7f6260150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:43.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62600ce970 0x7f62600af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62600ca590 con 0x7f62600c8b90 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626015ec00 con 0x7f62600ce590 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62600ca710 con 0x7f62600ce970 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6265400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 0x7f6260150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6265400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 0x7f6260150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46154/0 (socket says 172.21.15.113:46154) 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6265400640 1 -- 172.21.15.113:0/365067082 learned_addr learned my addr 172.21.15.113:0/365067082 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6264a00640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62600ce590 0x7f6260150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6265e00640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62600ce970 0x7f62600af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6265400640 1 -- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62600ce970 msgr2=0x7f62600af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6265400640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62600ce970 0x7f62600af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6265400640 1 -- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62600ce590 msgr2=0x7f6260150570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6265400640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62600ce590 0x7f6260150570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6265400640 1 -- 172.21.15.113:0/365067082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62600b0b00 con 0x7f62600c8b90 2024-03-04T08:07:43.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6265400640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 0x7f6260150030 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f625400c480 tx=0x7f625400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:43.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f625d600640 1 -- 172.21.15.113:0/365067082 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254005cf0 con 0x7f62600c8b90 2024-03-04T08:07:43.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f625d600640 1 -- 172.21.15.113:0/365067082 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6254007070 con 0x7f62600c8b90 2024-03-04T08:07:43.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 -- 172.21.15.113:0/365067082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6260150fd0 con 0x7f62600c8b90 2024-03-04T08:07:43.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f625d600640 1 -- 172.21.15.113:0/365067082 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62540074e0 con 0x7f62600c8b90 2024-03-04T08:07:43.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 -- 172.21.15.113:0/365067082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626014f270 con 0x7f62600c8b90 2024-03-04T08:07:43.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.328+0000 7f6266800640 1 -- 172.21.15.113:0/365067082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6260075a80 con 0x7f62600c8b90 2024-03-04T08:07:43.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.332+0000 7f625d600640 1 -- 172.21.15.113:0/365067082 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f625401b400 con 0x7f62600c8b90 2024-03-04T08:07:43.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.332+0000 7f625d600640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6240066f60 0x7f6240069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:43.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.332+0000 7f625d600640 1 -- 172.21.15.113:0/365067082 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f62540853e0 con 0x7f62600c8b90 2024-03-04T08:07:43.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.332+0000 7f6264a00640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6240066f60 0x7f6240069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:43.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.336+0000 7f6264a00640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6240066f60 0x7f6240069420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f624c0b2000 tx=0x7f624c0aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:43.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.336+0000 7f625d600640 1 -- 172.21.15.113:0/365067082 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6254055da0 con 0x7f62600c8b90 2024-03-04T08:07:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.532+0000 7f6266800640 1 -- 172.21.15.113:0/365067082 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f626006d480 con 0x7f62600c8b90 2024-03-04T08:07:43.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.532+0000 7f625d600640 1 -- 172.21.15.113:0/365067082 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) v1 ==== 95+0+5825 (secure 0 0 0) 0x7f62540556a0 con 0x7f62600c8b90 2024-03-04T08:07:43.533 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:43.533 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":127,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:40.716053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:resolve","state_seq":24,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7269,7119]},"id":1}]} 2024-03-04T08:07:43.533 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 127 2024-03-04T08:07:43.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 -- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6240066f60 msgr2=0x7f6240069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:43.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6240066f60 0x7f6240069420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f624c0b2000 tx=0x7f624c0aa070 comp rx=0 tx=0).stop 2024-03-04T08:07:43.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 -- 172.21.15.113:0/365067082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 msgr2=0x7f6260150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:43.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 0x7f6260150030 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f625400c480 tx=0x7f625400c940 comp rx=0 tx=0).stop 2024-03-04T08:07:43.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 -- 172.21.15.113:0/365067082 shutdown_connections 2024-03-04T08:07:43.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62600ce970 0x7f62600af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:43.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62600ce590 0x7f6260150570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:43.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6240066f60 0x7f6240069420 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:43.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 --2- 172.21.15.113:0/365067082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62600c8b90 0x7f6260150030 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:43.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 -- 172.21.15.113:0/365067082 >> 172.21.15.113:0/365067082 conn(0x7f6260066f80 msgr2=0x7f62600cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:43.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 -- 172.21.15.113:0/365067082 shutdown_connections 2024-03-04T08:07:43.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:43.536+0000 7f6266800640 1 -- 172.21.15.113:0/365067082 wait complete. 2024-03-04T08:07:43.555 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 127, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 127, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7119": { "addr": "172.21.15.113:6833/3825257303", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 3825257303, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 3825257303, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7119, "incarnation": 114, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 25 }, "gid_7269": { "addr": "172.21.15.113:6837/1401330284", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1401330284, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1401330284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7269, "incarnation": 120, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7389": { "addr": "172.21.15.113:6836/1310057122", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1310057122, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1310057122, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7389, "incarnation": 126, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:resolve", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 67, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:07:40.716053+0000", "qdb_cluster": [ 7269, 7119 ], "qdb_leader": 7119, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7389, "mds_1": 7119, "mds_2": 7269 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:07:45.556 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:45.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.696+0000 7f6a22800640 1 Processor -- start 2024-03-04T08:07:45.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.696+0000 7f6a22800640 1 -- start start 2024-03-04T08:07:45.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a22800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a1c154c00 0x7f6a1c0c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:45.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a22800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a1c0c7700 0x7f6a1c15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:45.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a22800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1c0c7d80 con 0x7f6a1c154fe0 2024-03-04T08:07:45.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a22800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1c15efb0 con 0x7f6a1c0c7700 2024-03-04T08:07:45.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a22800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c0c6b70 con 0x7f6a1c154c00 2024-03-04T08:07:45.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a1be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a1c0c7700 0x7f6a1c15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:45.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a1be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a1c0c7700 0x7f6a1c15abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46160/0 (socket says 172.21.15.113:46160) 2024-03-04T08:07:45.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a1be00640 1 -- 172.21.15.113:0/3162966828 learned_addr learned my addr 172.21.15.113:0/3162966828 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:45.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a1b400640 1 --2- 172.21.15.113:0/3162966828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a1c154c00 0x7f6a1c0c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:45.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a1be00640 1 -- 172.21.15.113:0/3162966828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a1c154c00 msgr2=0x7f6a1c0c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:45.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a1be00640 1 --2- 172.21.15.113:0/3162966828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a1c154c00 0x7f6a1c0c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a1be00640 1 -- 172.21.15.113:0/3162966828 >> v1:172.21.15.136:6789/0 conn(0x7f6a1c154fe0 legacy=0x7f6a1c0c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:45.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.700+0000 7f6a1be00640 1 -- 172.21.15.113:0/3162966828 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c0c8060 con 0x7f6a1c0c7700 2024-03-04T08:07:45.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a1be00640 1 --2- 172.21.15.113:0/3162966828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 0x7f6a1c15abe0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009e50 tx=0x7f6a1401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5dbadc89b7130e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:45.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a1aa00640 1 -- 172.21.15.113:0/3162966828 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a140a5750 con 0x7f6a1c0c7700 2024-03-04T08:07:45.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a1aa00640 1 -- 172.21.15.113:0/3162966828 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a140a58f0 con 0x7f6a1c0c7700 2024-03-04T08:07:45.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a1aa00640 1 -- 172.21.15.113:0/3162966828 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a140a5cc0 con 0x7f6a1c0c7700 2024-03-04T08:07:45.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 -- 172.21.15.113:0/3162966828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 msgr2=0x7f6a1c15abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:45.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 --2- 172.21.15.113:0/3162966828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 0x7f6a1c15abe0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009e50 tx=0x7f6a1401c9a0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 -- 172.21.15.113:0/3162966828 shutdown_connections 2024-03-04T08:07:45.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 --2- 172.21.15.113:0/3162966828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 0x7f6a1c15abe0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 --2- 172.21.15.113:0/3162966828 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a1c154c00 0x7f6a1c0c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 -- 172.21.15.113:0/3162966828 >> 172.21.15.113:0/3162966828 conn(0x7f6a1c06e820 msgr2=0x7f6a1c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:45.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 -- 172.21.15.113:0/3162966828 shutdown_connections 2024-03-04T08:07:45.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 -- 172.21.15.113:0/3162966828 wait complete. 2024-03-04T08:07:45.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 Processor -- start 2024-03-04T08:07:45.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 -- start start 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 0x7f6a1c0c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a1c154c00 0x7f6a1c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a1c154fe0 0x7f6a1c14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c0c23e0 con 0x7f6a1c0c7700 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c0c2260 con 0x7f6a1c154c00 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.704+0000 7f6a22800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c167c10 con 0x7f6a1c154fe0 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a1b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 0x7f6a1c0c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a1b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 0x7f6a1c0c5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46176/0 (socket says 172.21.15.113:46176) 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a1b400640 1 -- 172.21.15.113:0/1119162148 learned_addr learned my addr 172.21.15.113:0/1119162148 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a12a00640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a1c154c00 0x7f6a1c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a1be00640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a1c154fe0 0x7f6a1c14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a1b400640 1 -- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a1c154fe0 msgr2=0x7f6a1c14e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a1b400640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a1c154fe0 0x7f6a1c14e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a1b400640 1 -- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a1c154c00 msgr2=0x7f6a1c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:45.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a1b400640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a1c154c00 0x7f6a1c15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a1b400640 1 -- 172.21.15.113:0/1119162148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c158c20 con 0x7f6a1c0c7700 2024-03-04T08:07:45.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a1b400640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 0x7f6a1c0c5240 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00e8a0 tx=0x7f6a0c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:45.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a13e00640 1 -- 172.21.15.113:0/1119162148 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c017070 con 0x7f6a1c0c7700 2024-03-04T08:07:45.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a13e00640 1 -- 172.21.15.113:0/1119162148 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c00c940 con 0x7f6a1c0c7700 2024-03-04T08:07:45.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a13e00640 1 -- 172.21.15.113:0/1119162148 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00ccf0 con 0x7f6a1c0c7700 2024-03-04T08:07:45.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a22800640 1 -- 172.21.15.113:0/1119162148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c0c5cc0 con 0x7f6a1c0c7700 2024-03-04T08:07:45.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a22800640 1 -- 172.21.15.113:0/1119162148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c1f4710 con 0x7f6a1c0c7700 2024-03-04T08:07:45.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a22800640 1 -- 172.21.15.113:0/1119162148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1c075c00 con 0x7f6a1c0c7700 2024-03-04T08:07:45.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.708+0000 7f6a13e00640 1 -- 172.21.15.113:0/1119162148 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a0c01a070 con 0x7f6a1c0c7700 2024-03-04T08:07:45.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.712+0000 7f6a13e00640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69f8066f90 0x7f69f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:45.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.712+0000 7f6a13e00640 1 -- 172.21.15.113:0/1119162148 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f6a0c085600 con 0x7f6a1c0c7700 2024-03-04T08:07:45.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.712+0000 7f6a12a00640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69f8066f90 0x7f69f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:45.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.712+0000 7f6a12a00640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69f8066f90 0x7f69f8069450 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0685e0 tx=0x7f6a04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:45.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.712+0000 7f6a13e00640 1 -- 172.21.15.113:0/1119162148 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6a0c055f20 con 0x7f6a1c0c7700 2024-03-04T08:07:45.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.912+0000 7f6a22800640 1 -- 172.21.15.113:0/1119162148 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a1c154430 con 0x7f6a1c0c7700 2024-03-04T08:07:45.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a13e00640 1 -- 172.21.15.113:0/1119162148 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) v1 ==== 95+0+5825 (secure 0 0 0) 0x7f6a0c055820 con 0x7f6a1c0c7700 2024-03-04T08:07:45.914 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:45.915 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":127,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:40.716053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:resolve","state_seq":24,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7269,7119]},"id":1}]} 2024-03-04T08:07:45.915 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 127 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 -- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69f8066f90 msgr2=0x7f69f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69f8066f90 0x7f69f8069450 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0685e0 tx=0x7f6a04009040 comp rx=0 tx=0).stop 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 -- 172.21.15.113:0/1119162148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 msgr2=0x7f6a1c0c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 0x7f6a1c0c5240 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00e8a0 tx=0x7f6a0c00eca0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 -- 172.21.15.113:0/1119162148 shutdown_connections 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a1c154fe0 0x7f6a1c14e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a1c154c00 0x7f6a1c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f69f8066f90 0x7f69f8069450 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 --2- 172.21.15.113:0/1119162148 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a1c0c7700 0x7f6a1c0c5240 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 -- 172.21.15.113:0/1119162148 >> 172.21.15.113:0/1119162148 conn(0x7f6a1c06e820 msgr2=0x7f6a1c06ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 -- 172.21.15.113:0/1119162148 shutdown_connections 2024-03-04T08:07:45.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:45.916+0000 7f6a22800640 1 -- 172.21.15.113:0/1119162148 wait complete. 2024-03-04T08:07:45.937 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:07:47.938 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:48.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.080+0000 7fb408c00640 1 Processor -- start 2024-03-04T08:07:48.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb408c00640 1 -- start start 2024-03-04T08:07:48.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb408c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb404157df0 0x7fb40406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:48.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb408c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb4040ce020 0x7fb4040ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:48.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb408c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb404070fd0 con 0x7fb4040c8a90 2024-03-04T08:07:48.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb408c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40415e090 con 0x7fb4040ce020 2024-03-04T08:07:48.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb408c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40406c6b0 con 0x7fb404157df0 2024-03-04T08:07:48.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb3fbe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb4040c8a90 0x7fb40415e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42068/0 (socket says 172.21.15.113:42068) 2024-03-04T08:07:48.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb3fbe00640 1 -- 172.21.15.113:0/3206730770 learned_addr learned my addr 172.21.15.113:0/3206730770 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:48.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb403e00640 1 --2- 172.21.15.113:0/3206730770 >> v2:172.21.15.113:3300/0 conn(0x7fb4040ce020 0x7fb4040ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:48.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb403400640 1 --2- 172.21.15.113:0/3206730770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb404157df0 0x7fb40406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:48.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb403e00640 1 -- 172.21.15.113:0/3206730770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb404157df0 msgr2=0x7fb40406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:48.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb403e00640 1 --2- 172.21.15.113:0/3206730770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb404157df0 0x7fb40406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:48.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb403e00640 1 -- 172.21.15.113:0/3206730770 >> v1:172.21.15.113:6789/0 conn(0x7fb4040c8a90 legacy=0x7fb40415e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:48.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.084+0000 7fb403e00640 1 -- 172.21.15.113:0/3206730770 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40406dfd0 con 0x7fb4040ce020 2024-03-04T08:07:48.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb403e00640 1 --2- 172.21.15.113:0/3206730770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 0x7fb4040ce460 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009d90 tx=0x7fb3fc01c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fea97b20583f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:48.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb402a00640 1 -- 172.21.15.113:0/3206730770 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc0b2070 con 0x7fb4040ce020 2024-03-04T08:07:48.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb402a00640 1 -- 172.21.15.113:0/3206730770 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3fc0a5820 con 0x7fb4040ce020 2024-03-04T08:07:48.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb402a00640 1 -- 172.21.15.113:0/3206730770 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc0a5bf0 con 0x7fb4040ce020 2024-03-04T08:07:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb408c00640 1 -- 172.21.15.113:0/3206730770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 msgr2=0x7fb4040ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb408c00640 1 --2- 172.21.15.113:0/3206730770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 0x7fb4040ce460 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009d90 tx=0x7fb3fc01c800 comp rx=0 tx=0).stop 2024-03-04T08:07:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb408c00640 1 -- 172.21.15.113:0/3206730770 shutdown_connections 2024-03-04T08:07:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb408c00640 1 --2- 172.21.15.113:0/3206730770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 0x7fb4040ce460 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb408c00640 1 --2- 172.21.15.113:0/3206730770 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb404157df0 0x7fb40406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb408c00640 1 -- 172.21.15.113:0/3206730770 >> 172.21.15.113:0/3206730770 conn(0x7fb404072f40 msgr2=0x7fb4040706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb408c00640 1 -- 172.21.15.113:0/3206730770 shutdown_connections 2024-03-04T08:07:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.088+0000 7fb408c00640 1 -- 172.21.15.113:0/3206730770 wait complete. 2024-03-04T08:07:48.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb408c00640 1 Processor -- start 2024-03-04T08:07:48.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb408c00640 1 -- start start 2024-03-04T08:07:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb408c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4040c8a90 0x7fb4041f1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb408c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 0x7fb4041f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb408c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb404157df0 0x7fb404156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb408c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40415dd20 con 0x7fb4040ce020 2024-03-04T08:07:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb408c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40406ffb0 con 0x7fb404157df0 2024-03-04T08:07:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb408c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40415ec30 con 0x7fb4040c8a90 2024-03-04T08:07:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb3fbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 0x7fb4041f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb3fbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 0x7fb4041f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45762/0 (socket says 172.21.15.113:45762) 2024-03-04T08:07:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb3fbe00640 1 -- 172.21.15.113:0/3369169572 learned_addr learned my addr 172.21.15.113:0/3369169572 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb403400640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4040c8a90 0x7fb4041f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:48.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb403e00640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb404157df0 0x7fb404156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:48.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb3fbe00640 1 -- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4040c8a90 msgr2=0x7fb4041f1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:48.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb3fbe00640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4040c8a90 0x7fb4041f1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:48.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb3fbe00640 1 -- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb404157df0 msgr2=0x7fb404156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:48.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb3fbe00640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb404157df0 0x7fb404156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:48.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb3fbe00640 1 -- 172.21.15.113:0/3369169572 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4040caec0 con 0x7fb4040ce020 2024-03-04T08:07:48.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.092+0000 7fb3fbe00640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 0x7fb4041f4580 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00fb20 tx=0x7fb3ec00d060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:48.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.096+0000 7fb3fb400640 1 -- 172.21.15.113:0/3369169572 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec01b070 con 0x7fb4040ce020 2024-03-04T08:07:48.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.096+0000 7fb3fb400640 1 -- 172.21.15.113:0/3369169572 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ec007d00 con 0x7fb4040ce020 2024-03-04T08:07:48.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.096+0000 7fb408c00640 1 -- 172.21.15.113:0/3369169572 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4040cb1a0 con 0x7fb4040ce020 2024-03-04T08:07:48.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.096+0000 7fb3fb400640 1 -- 172.21.15.113:0/3369169572 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec00da70 con 0x7fb4040ce020 2024-03-04T08:07:48.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.096+0000 7fb408c00640 1 -- 172.21.15.113:0/3369169572 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4041f0850 con 0x7fb4040ce020 2024-03-04T08:07:48.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.096+0000 7fb408c00640 1 -- 172.21.15.113:0/3369169572 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb404075a80 con 0x7fb4040ce020 2024-03-04T08:07:48.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.096+0000 7fb3fb400640 1 -- 172.21.15.113:0/3369169572 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3ec01e070 con 0x7fb4040ce020 2024-03-04T08:07:48.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.096+0000 7fb3fb400640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3e0066f60 0x7fb3e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:48.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.096+0000 7fb3fb400640 1 -- 172.21.15.113:0/3369169572 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7fb3ec087fd0 con 0x7fb4040ce020 2024-03-04T08:07:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.096+0000 7fb403400640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3e0066f60 0x7fb3e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.100+0000 7fb403400640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3e0066f60 0x7fb3e0069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009ac0 tx=0x7fb3f4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:48.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.100+0000 7fb3fb400640 1 -- 172.21.15.113:0/3369169572 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb404070fd0 con 0x7fb4040ce020 2024-03-04T08:07:48.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.296+0000 7fb408c00640 1 -- 172.21.15.113:0/3369169572 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb40406f8a0 con 0x7fb4040ce020 2024-03-04T08:07:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.296+0000 7fb3fb400640 1 -- 172.21.15.113:0/3369169572 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+5827 (secure 0 0 0) 0x7fb3ec0582a0 con 0x7fb4040ce020 2024-03-04T08:07:48.294 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:48.294 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":128,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:46.338586+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7119,7269]},"id":1}]} 2024-03-04T08:07:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 128 2024-03-04T08:07:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 -- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3e0066f60 msgr2=0x7fb3e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3e0066f60 0x7fb3e0069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009ac0 tx=0x7fb3f4009190 comp rx=0 tx=0).stop 2024-03-04T08:07:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 -- 172.21.15.113:0/3369169572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 msgr2=0x7fb4041f4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 0x7fb4041f4580 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00fb20 tx=0x7fb3ec00d060 comp rx=0 tx=0).stop 2024-03-04T08:07:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 -- 172.21.15.113:0/3369169572 shutdown_connections 2024-03-04T08:07:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb404157df0 0x7fb404156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb3e0066f60 0x7fb3e0069420 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4040ce020 0x7fb4041f4580 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:48.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 --2- 172.21.15.113:0/3369169572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4040c8a90 0x7fb4041f1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 -- 172.21.15.113:0/3369169572 >> 172.21.15.113:0/3369169572 conn(0x7fb404072f40 msgr2=0x7fb4040cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 -- 172.21.15.113:0/3369169572 shutdown_connections 2024-03-04T08:07:48.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:48.300+0000 7fb408c00640 1 -- 172.21.15.113:0/3369169572 wait complete. 2024-03-04T08:07:48.324 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:07:50.326 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:50.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.468+0000 7f2a8e800640 1 Processor -- start 2024-03-04T08:07:50.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.468+0000 7f2a8e800640 1 -- start start 2024-03-04T08:07:50.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.468+0000 7f2a8e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a881527f0 0x7f2a88070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:50.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.468+0000 7f2a8e800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a88152bd0 0x7f2a881587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:50.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.468+0000 7f2a8e800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a881591f0 con 0x7f2a880c8200 2024-03-04T08:07:50.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.468+0000 7f2a8e800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a88071df0 con 0x7f2a881527f0 2024-03-04T08:07:50.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.468+0000 7f2a8e800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a881597b0 con 0x7f2a88152bd0 2024-03-04T08:07:50.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.468+0000 7f2a87400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a881527f0 0x7f2a88070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:50.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.468+0000 7f2a7fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2a880c8200 0x7f2a88072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42074/0 (socket says 172.21.15.113:42074) 2024-03-04T08:07:50.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a7fe00640 1 -- 172.21.15.113:0/3986675273 learned_addr learned my addr 172.21.15.113:0/3986675273 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:50.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a87e00640 1 --2- 172.21.15.113:0/3986675273 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a88152bd0 0x7f2a881587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:50.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a87400640 1 -- 172.21.15.113:0/3986675273 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a88152bd0 msgr2=0x7f2a881587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:50.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a87400640 1 --2- 172.21.15.113:0/3986675273 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a88152bd0 0x7f2a881587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:50.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a87400640 1 -- 172.21.15.113:0/3986675273 >> v1:172.21.15.113:6789/0 conn(0x7f2a880c8200 legacy=0x7f2a88072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:50.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a87400640 1 -- 172.21.15.113:0/3986675273 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a881594d0 con 0x7f2a881527f0 2024-03-04T08:07:50.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a86a00640 1 -- 172.21.15.113:0/3986675273 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359897611 0 0) 0x7f2a881591f0 con 0x7f2a880c8200 2024-03-04T08:07:50.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a87400640 1 --2- 172.21.15.113:0/3986675273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 0x7f2a88070d40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f2a780097c0 tx=0x7f2a7801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25e25d5bc963603 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:50.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a86a00640 1 -- 172.21.15.113:0/3986675273 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a780b1070 con 0x7f2a881527f0 2024-03-04T08:07:50.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a86a00640 1 -- 172.21.15.113:0/3986675273 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a780ac410 con 0x7f2a881527f0 2024-03-04T08:07:50.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.472+0000 7f2a86a00640 1 -- 172.21.15.113:0/3986675273 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a780ac7e0 con 0x7f2a881527f0 2024-03-04T08:07:50.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.476+0000 7f2a8e800640 1 -- 172.21.15.113:0/3986675273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 msgr2=0x7f2a88070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:50.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.476+0000 7f2a8e800640 1 --2- 172.21.15.113:0/3986675273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 0x7f2a88070d40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f2a780097c0 tx=0x7f2a7801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:07:50.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.476+0000 7f2a8e800640 1 -- 172.21.15.113:0/3986675273 shutdown_connections 2024-03-04T08:07:50.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.476+0000 7f2a8e800640 1 --2- 172.21.15.113:0/3986675273 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a88152bd0 0x7f2a881587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:50.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.476+0000 7f2a8e800640 1 --2- 172.21.15.113:0/3986675273 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 0x7f2a88070d40 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:50.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.476+0000 7f2a8e800640 1 -- 172.21.15.113:0/3986675273 >> 172.21.15.113:0/3986675273 conn(0x7f2a8806e820 msgr2=0x7f2a8806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:50.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.476+0000 7f2a8e800640 1 -- 172.21.15.113:0/3986675273 shutdown_connections 2024-03-04T08:07:50.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.476+0000 7f2a8e800640 1 -- 172.21.15.113:0/3986675273 wait complete. 2024-03-04T08:07:50.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.476+0000 7f2a8e800640 1 Processor -- start 2024-03-04T08:07:50.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.476+0000 7f2a8e800640 1 -- start start 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a8e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a880c8200 0x7f2a881f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a8e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 0x7f2a880b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a8e800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a88152bd0 0x7f2a88153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a8e800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a881654f0 con 0x7f2a881527f0 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a8e800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a88165370 con 0x7f2a880c8200 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a8e800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a88165670 con 0x7f2a88152bd0 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 0x7f2a880b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 0x7f2a880b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45780/0 (socket says 172.21.15.113:45780) 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7fe00640 1 -- 172.21.15.113:0/2356421942 learned_addr learned my addr 172.21.15.113:0/2356421942 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a87e00640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a88152bd0 0x7f2a88153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a87400640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a880c8200 0x7f2a881f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:50.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7fe00640 1 -- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a88152bd0 msgr2=0x7f2a88153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:50.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7fe00640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a88152bd0 0x7f2a88153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:50.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7fe00640 1 -- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a880c8200 msgr2=0x7f2a881f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:50.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7fe00640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a880c8200 0x7f2a881f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:50.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7fe00640 1 -- 172.21.15.113:0/2356421942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a88152030 con 0x7f2a881527f0 2024-03-04T08:07:50.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a87400640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a880c8200 0x7f2a881f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:50.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7fe00640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 0x7f2a880b0cc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000c4b0 tx=0x7f2a7000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:50.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7f400640 1 -- 172.21.15.113:0/2356421942 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70018070 con 0x7f2a881527f0 2024-03-04T08:07:50.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7f400640 1 -- 172.21.15.113:0/2356421942 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a70007500 con 0x7f2a881527f0 2024-03-04T08:07:50.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a7f400640 1 -- 172.21.15.113:0/2356421942 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70004a30 con 0x7f2a881527f0 2024-03-04T08:07:50.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a8e800640 1 -- 172.21.15.113:0/2356421942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a88152310 con 0x7f2a881527f0 2024-03-04T08:07:50.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a8e800640 1 -- 172.21.15.113:0/2356421942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a881f1fd0 con 0x7f2a881527f0 2024-03-04T08:07:50.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.480+0000 7f2a8e800640 1 -- 172.21.15.113:0/2356421942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a88075c00 con 0x7f2a881527f0 2024-03-04T08:07:50.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.484+0000 7f2a7f400640 1 -- 172.21.15.113:0/2356421942 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a70004bd0 con 0x7f2a881527f0 2024-03-04T08:07:50.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.484+0000 7f2a7f400640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a64066f60 0x7f2a64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:50.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.484+0000 7f2a7f400640 1 -- 172.21.15.113:0/2356421942 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f2a70084b70 con 0x7f2a881527f0 2024-03-04T08:07:50.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.484+0000 7f2a87400640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a64066f60 0x7f2a64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:50.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.484+0000 7f2a87400640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a64066f60 0x7f2a64069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2a780ae000 tx=0x7f2a78008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:50.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.492+0000 7f2a7f400640 1 -- 172.21.15.113:0/2356421942 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2a70054de0 con 0x7f2a881527f0 2024-03-04T08:07:50.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.768+0000 7f2a8e800640 1 -- 172.21.15.113:0/2356421942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a880c79d0 con 0x7f2a881527f0 2024-03-04T08:07:50.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a7f400640 1 -- 172.21.15.113:0/2356421942 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+5827 (secure 0 0 0) 0x7f2a70058e80 con 0x7f2a881527f0 2024-03-04T08:07:50.767 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:50.767 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":128,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:46.338586+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7119,7269]},"id":1}]} 2024-03-04T08:07:50.767 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 128 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 -- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a64066f60 msgr2=0x7f2a64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a64066f60 0x7f2a64069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2a780ae000 tx=0x7f2a78008040 comp rx=0 tx=0).stop 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 -- 172.21.15.113:0/2356421942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 msgr2=0x7f2a880b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 0x7f2a880b0cc0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000c4b0 tx=0x7f2a7000c970 comp rx=0 tx=0).stop 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 -- 172.21.15.113:0/2356421942 shutdown_connections 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a88152bd0 0x7f2a88153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a64066f60 0x7f2a64069420 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a881527f0 0x7f2a880b0cc0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 --2- 172.21.15.113:0/2356421942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a880c8200 0x7f2a881f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 -- 172.21.15.113:0/2356421942 >> 172.21.15.113:0/2356421942 conn(0x7f2a8806e820 msgr2=0x7f2a88070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:50.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 -- 172.21.15.113:0/2356421942 shutdown_connections 2024-03-04T08:07:50.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:50.772+0000 7f2a8e800640 1 -- 172.21.15.113:0/2356421942 wait complete. 2024-03-04T08:07:50.789 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:07:52.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:52.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7593000640 1 Processor -- start 2024-03-04T08:07:52.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7593000640 1 -- start start 2024-03-04T08:07:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7593000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 0x7f758c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7593000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f758c0c8200 0x7f758c072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7593000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f758c152bd0 0x7f758c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7593000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c071df0 con 0x7f758c152bd0 2024-03-04T08:07:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7593000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c1597b0 con 0x7f758c0c8200 2024-03-04T08:07:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7593000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c1591f0 con 0x7f758c1527f0 2024-03-04T08:07:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7591000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f758c152bd0 0x7f758c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:52.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7591000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f758c152bd0 0x7f758c1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45792/0 (socket says 172.21.15.113:45792) 2024-03-04T08:07:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7591000640 1 -- 172.21.15.113:0/614325049 learned_addr learned my addr 172.21.15.113:0/614325049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f758be00640 1 --2- 172.21.15.113:0/614325049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 0x7f758c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7583e00640 1 --2- 172.21.15.113:0/614325049 >> v2:172.21.15.136:3300/0 conn(0x7f758c0c8200 0x7f758c072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7591000640 1 -- 172.21.15.113:0/614325049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 msgr2=0x7f758c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7591000640 1 --2- 172.21.15.113:0/614325049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 0x7f758c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7591000640 1 -- 172.21.15.113:0/614325049 >> v2:172.21.15.136:3300/0 conn(0x7f758c0c8200 msgr2=0x7f758c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7591000640 1 --2- 172.21.15.113:0/614325049 >> v2:172.21.15.136:3300/0 conn(0x7f758c0c8200 0x7f758c072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:52.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.932+0000 7f7591000640 1 -- 172.21.15.113:0/614325049 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c1594d0 con 0x7f758c152bd0 2024-03-04T08:07:52.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.936+0000 7f7591000640 1 --2- 172.21.15.113:0/614325049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 0x7f758c1587d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f758c0c3520 tx=0x7f758401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3685b7ba2fe5f99 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:52.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f758b400640 1 -- 172.21.15.113:0/614325049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840a5760 con 0x7f758c152bd0 2024-03-04T08:07:52.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f758b400640 1 -- 172.21.15.113:0/614325049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75840a5900 con 0x7f758c152bd0 2024-03-04T08:07:52.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f758b400640 1 -- 172.21.15.113:0/614325049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840a5cd0 con 0x7f758c152bd0 2024-03-04T08:07:52.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 -- 172.21.15.113:0/614325049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 msgr2=0x7f758c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:52.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 --2- 172.21.15.113:0/614325049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 0x7f758c1587d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f758c0c3520 tx=0x7f758401c9f0 comp rx=0 tx=0).stop 2024-03-04T08:07:52.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 -- 172.21.15.113:0/614325049 shutdown_connections 2024-03-04T08:07:52.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 --2- 172.21.15.113:0/614325049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 0x7f758c1587d0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:52.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 --2- 172.21.15.113:0/614325049 >> v2:172.21.15.136:3300/0 conn(0x7f758c0c8200 0x7f758c072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:52.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 --2- 172.21.15.113:0/614325049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 0x7f758c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:52.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 -- 172.21.15.113:0/614325049 >> 172.21.15.113:0/614325049 conn(0x7f758c06e820 msgr2=0x7f758c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:52.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 -- 172.21.15.113:0/614325049 shutdown_connections 2024-03-04T08:07:52.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 -- 172.21.15.113:0/614325049 wait complete. 2024-03-04T08:07:52.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 Processor -- start 2024-03-04T08:07:52.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 -- start start 2024-03-04T08:07:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f758c0c8200 0x7f758c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 0x7f758c150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 0x7f758c155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15bf40 con 0x7f758c152bd0 2024-03-04T08:07:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15bdc0 con 0x7f758c0c8200 2024-03-04T08:07:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7593000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15c0c0 con 0x7f758c1527f0 2024-03-04T08:07:52.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7591000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 0x7f758c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7591000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 0x7f758c155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45804/0 (socket says 172.21.15.113:45804) 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7591000640 1 -- 172.21.15.113:0/104137161 learned_addr learned my addr 172.21.15.113:0/104137161 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f758be00640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f758c0c8200 0x7f758c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.940+0000 7f7583e00640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 0x7f758c150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7591000640 1 -- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 msgr2=0x7f758c150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7591000640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 0x7f758c150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7591000640 1 -- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f758c0c8200 msgr2=0x7f758c0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7591000640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f758c0c8200 0x7f758c0b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7591000640 1 -- 172.21.15.113:0/104137161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0736e0 con 0x7f758c152bd0 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7583e00640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 0x7f758c150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f758be00640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f758c0c8200 0x7f758c0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7591000640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 0x7f758c155830 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f75840af040 tx=0x7f7584019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7588c00640 1 -- 172.21.15.113:0/104137161 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840bc020 con 0x7f758c152bd0 2024-03-04T08:07:52.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7588c00640 1 -- 172.21.15.113:0/104137161 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75840b86d0 con 0x7f758c152bd0 2024-03-04T08:07:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7588c00640 1 -- 172.21.15.113:0/104137161 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840b8ac0 con 0x7f758c152bd0 2024-03-04T08:07:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7593000640 1 -- 172.21.15.113:0/104137161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c0720d0 con 0x7f758c152bd0 2024-03-04T08:07:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7593000640 1 -- 172.21.15.113:0/104137161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c1f3bb0 con 0x7f758c152bd0 2024-03-04T08:07:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7593000640 1 -- 172.21.15.113:0/104137161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c1f8040 con 0x7f758c152bd0 2024-03-04T08:07:52.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7588c00640 1 -- 172.21.15.113:0/104137161 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75840bf020 con 0x7f758c152bd0 2024-03-04T08:07:52.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7588c00640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7568066f60 0x7f7568069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:52.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.944+0000 7f7588c00640 1 -- 172.21.15.113:0/104137161 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f7584120670 con 0x7f758c152bd0 2024-03-04T08:07:52.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.948+0000 7f758be00640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7568066f60 0x7f7568069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:52.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.948+0000 7f758be00640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7568066f60 0x7f7568069420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f757c007a70 tx=0x7f757c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:52.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:52.956+0000 7f7588c00640 1 -- 172.21.15.113:0/104137161 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f75840165e0 con 0x7f758c152bd0 2024-03-04T08:07:53.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 -- 172.21.15.113:0/104137161 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f758c1f3e50 con 0x7f758c152bd0 2024-03-04T08:07:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7588c00640 1 -- 172.21.15.113:0/104137161 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f75840f1050 con 0x7f758c152bd0 2024-03-04T08:07:53.235 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:53.235 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":129,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:52.036281+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7269,7119]},"id":1}]} 2024-03-04T08:07:53.236 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 129 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 -- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7568066f60 msgr2=0x7f7568069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7568066f60 0x7f7568069420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f757c007a70 tx=0x7f757c008040 comp rx=0 tx=0).stop 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 -- 172.21.15.113:0/104137161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 msgr2=0x7f758c155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 0x7f758c155830 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f75840af040 tx=0x7f7584019790 comp rx=0 tx=0).stop 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 -- 172.21.15.113:0/104137161 shutdown_connections 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7568066f60 0x7f7568069420 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f758c152bd0 0x7f758c155830 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f758c1527f0 0x7f758c150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 --2- 172.21.15.113:0/104137161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f758c0c8200 0x7f758c0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 -- 172.21.15.113:0/104137161 >> 172.21.15.113:0/104137161 conn(0x7f758c06e820 msgr2=0x7f758c0c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 -- 172.21.15.113:0/104137161 shutdown_connections 2024-03-04T08:07:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:53.240+0000 7f7593000640 1 -- 172.21.15.113:0/104137161 wait complete. 2024-03-04T08:07:53.263 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:07:55.264 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e4400640 1 Processor -- start 2024-03-04T08:07:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e4400640 1 -- start start 2024-03-04T08:07:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e4400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f24dc1527f0 0x7f24dc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e4400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 0x7f24dc072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e4400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24dc152bd0 0x7f24dc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e4400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24dc071df0 con 0x7f24dc152bd0 2024-03-04T08:07:55.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e4400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24dc1597b0 con 0x7f24dc1527f0 2024-03-04T08:07:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e4400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc1591f0 con 0x7f24dc0c8200 2024-03-04T08:07:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e2400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24dc152bd0 0x7f24dc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e2400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f24dc152bd0 0x7f24dc1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45806/0 (socket says 172.21.15.113:45806) 2024-03-04T08:07:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e2400640 1 -- 172.21.15.113:0/3292099028 learned_addr learned my addr 172.21.15.113:0/3292099028 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e1000640 1 --2- 172.21.15.113:0/3292099028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 0x7f24dc072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e1a00640 1 --2- 172.21.15.113:0/3292099028 >> v2:172.21.15.136:3300/0 conn(0x7f24dc1527f0 0x7f24dc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e2400640 1 -- 172.21.15.113:0/3292099028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 msgr2=0x7f24dc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:55.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e2400640 1 --2- 172.21.15.113:0/3292099028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 0x7f24dc072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e2400640 1 -- 172.21.15.113:0/3292099028 >> v2:172.21.15.136:3300/0 conn(0x7f24dc1527f0 msgr2=0x7f24dc070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:55.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e2400640 1 --2- 172.21.15.113:0/3292099028 >> v2:172.21.15.136:3300/0 conn(0x7f24dc1527f0 0x7f24dc070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e2400640 1 -- 172.21.15.113:0/3292099028 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc1594d0 con 0x7f24dc152bd0 2024-03-04T08:07:55.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e1000640 1 --2- 172.21.15.113:0/3292099028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 0x7f24dc072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:55.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.408+0000 7f24e1a00640 1 --2- 172.21.15.113:0/3292099028 >> v2:172.21.15.136:3300/0 conn(0x7f24dc1527f0 0x7f24dc070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:55.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24e2400640 1 --2- 172.21.15.113:0/3292099028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 0x7f24dc1587d0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f24dc06f400 tx=0x7f24d401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663906a33ae30537 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24dbe00640 1 -- 172.21.15.113:0/3292099028 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40a5760 con 0x7f24dc152bd0 2024-03-04T08:07:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24dbe00640 1 -- 172.21.15.113:0/3292099028 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d40a5900 con 0x7f24dc152bd0 2024-03-04T08:07:55.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24dbe00640 1 -- 172.21.15.113:0/3292099028 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40a5cd0 con 0x7f24dc152bd0 2024-03-04T08:07:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24e4400640 1 -- 172.21.15.113:0/3292099028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 msgr2=0x7f24dc1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24e4400640 1 --2- 172.21.15.113:0/3292099028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 0x7f24dc1587d0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f24dc06f400 tx=0x7f24d401c9f0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24e4400640 1 -- 172.21.15.113:0/3292099028 shutdown_connections 2024-03-04T08:07:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24e4400640 1 --2- 172.21.15.113:0/3292099028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 0x7f24dc1587d0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24e4400640 1 --2- 172.21.15.113:0/3292099028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 0x7f24dc072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24e4400640 1 --2- 172.21.15.113:0/3292099028 >> v2:172.21.15.136:3300/0 conn(0x7f24dc1527f0 0x7f24dc070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24e4400640 1 -- 172.21.15.113:0/3292099028 >> 172.21.15.113:0/3292099028 conn(0x7f24dc06e820 msgr2=0x7f24dc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24e4400640 1 -- 172.21.15.113:0/3292099028 shutdown_connections 2024-03-04T08:07:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.412+0000 7f24e4400640 1 -- 172.21.15.113:0/3292099028 wait complete. 2024-03-04T08:07:55.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 Processor -- start 2024-03-04T08:07:55.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 -- start start 2024-03-04T08:07:55.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 0x7f24dc158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24dc1527f0 0x7f24dc0b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 0x7f24dc1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc15bf40 con 0x7f24dc152bd0 2024-03-04T08:07:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc15bdc0 con 0x7f24dc1527f0 2024-03-04T08:07:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc15c0c0 con 0x7f24dc0c8200 2024-03-04T08:07:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 0x7f24dc1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 0x7f24dc1575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45814/0 (socket says 172.21.15.113:45814) 2024-03-04T08:07:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e2400640 1 -- 172.21.15.113:0/1562239433 learned_addr learned my addr 172.21.15.113:0/1562239433 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e1a00640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 0x7f24dc158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e1000640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24dc1527f0 0x7f24dc0b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:55.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e2400640 1 -- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 msgr2=0x7f24dc158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:55.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e2400640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 0x7f24dc158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e2400640 1 -- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24dc1527f0 msgr2=0x7f24dc0b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:55.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e2400640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24dc1527f0 0x7f24dc0b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e2400640 1 -- 172.21.15.113:0/1562239433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc158ac0 con 0x7f24dc152bd0 2024-03-04T08:07:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e2400640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 0x7f24dc1575a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f24d40af040 tx=0x7f24d4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24d9600640 1 -- 172.21.15.113:0/1562239433 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40bc020 con 0x7f24dc152bd0 2024-03-04T08:07:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24d9600640 1 -- 172.21.15.113:0/1562239433 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d40b8690 con 0x7f24dc152bd0 2024-03-04T08:07:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 -- 172.21.15.113:0/1562239433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24dc1f4920 con 0x7f24dc152bd0 2024-03-04T08:07:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24d9600640 1 -- 172.21.15.113:0/1562239433 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d40b8a80 con 0x7f24dc152bd0 2024-03-04T08:07:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 -- 172.21.15.113:0/1562239433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24dc1f4640 con 0x7f24dc152bd0 2024-03-04T08:07:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.416+0000 7f24e4400640 1 -- 172.21.15.113:0/1562239433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24dc075c00 con 0x7f24dc152bd0 2024-03-04T08:07:55.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.420+0000 7f24d9600640 1 -- 172.21.15.113:0/1562239433 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24d40bf020 con 0x7f24dc152bd0 2024-03-04T08:07:55.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.420+0000 7f24d9600640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24bc066f60 0x7f24bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:55.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.420+0000 7f24d9600640 1 -- 172.21.15.113:0/1562239433 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f24d4120720 con 0x7f24dc152bd0 2024-03-04T08:07:55.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.420+0000 7f24e1a00640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24bc066f60 0x7f24bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:55.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.420+0000 7f24e1a00640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24bc066f60 0x7f24bc069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0734d0 tx=0x7f24d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:55.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.428+0000 7f24d9600640 1 -- 172.21.15.113:0/1562239433 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f24d40c6030 con 0x7f24dc152bd0 2024-03-04T08:07:55.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.648+0000 7f24e4400640 1 -- 172.21.15.113:0/1562239433 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24dc0720d0 con 0x7f24dc152bd0 2024-03-04T08:07:55.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.648+0000 7f24d9600640 1 -- 172.21.15.113:0/1562239433 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f24d40165e0 con 0x7f24dc152bd0 2024-03-04T08:07:55.644 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:55.645 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":129,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:52.036281+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7269,7119]},"id":1}]} 2024-03-04T08:07:55.645 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 129 2024-03-04T08:07:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 -- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24bc066f60 msgr2=0x7f24bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24bc066f60 0x7f24bc069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0734d0 tx=0x7f24d0008040 comp rx=0 tx=0).stop 2024-03-04T08:07:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 -- 172.21.15.113:0/1562239433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 msgr2=0x7f24dc1575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 0x7f24dc1575a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f24d40af040 tx=0x7f24d4019790 comp rx=0 tx=0).stop 2024-03-04T08:07:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 -- 172.21.15.113:0/1562239433 shutdown_connections 2024-03-04T08:07:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f24bc066f60 0x7f24bc069420 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f24dc152bd0 0x7f24dc1575a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f24dc1527f0 0x7f24dc0b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 --2- 172.21.15.113:0/1562239433 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24dc0c8200 0x7f24dc158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:55.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 -- 172.21.15.113:0/1562239433 >> 172.21.15.113:0/1562239433 conn(0x7f24dc06e820 msgr2=0x7f24dc15b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:55.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 -- 172.21.15.113:0/1562239433 shutdown_connections 2024-03-04T08:07:55.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:55.652+0000 7f24e4400640 1 -- 172.21.15.113:0/1562239433 wait complete. 2024-03-04T08:07:55.675 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 129, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 129, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7119": { "addr": "172.21.15.113:6833/3825257303", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 3825257303, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 3825257303, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7119, "incarnation": 114, "join_fscid": -1, "name": "c", "rank": 1, "state": "up:active", "state_seq": 25 }, "gid_7269": { "addr": "172.21.15.113:6837/1401330284", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1401330284, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1401330284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7269, "incarnation": 120, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7389": { "addr": "172.21.15.113:6836/1310057122", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1310057122, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1310057122, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7389, "incarnation": 126, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:rejoin", "state_seq": 28 } }, "last_failure": 0, "last_failure_osd_epoch": 67, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:07:52.036281+0000", "qdb_cluster": [ 7269, 7119 ], "qdb_leader": 7119, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7389, "mds_1": 7119, "mds_2": 7269 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:07:57.676 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:07:57.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.820+0000 7f2a4d200640 1 Processor -- start 2024-03-04T08:07:57.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.820+0000 7f2a4d200640 1 -- start start 2024-03-04T08:07:57.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a4d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a481527f0 0x7f2a48070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:57.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a4d200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2a480c8200 0x7f2a48072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:57.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a4d200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a481591f0 con 0x7f2a48152bd0 2024-03-04T08:07:57.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a4d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a48071df0 con 0x7f2a481527f0 2024-03-04T08:07:57.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a4d200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a481597b0 con 0x7f2a480c8200 2024-03-04T08:07:57.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a46800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2a48152bd0 0x7f2a481587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60364/0 (socket says 172.21.15.113:60364) 2024-03-04T08:07:57.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a45e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a481527f0 0x7f2a48070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:57.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a46800640 1 -- 172.21.15.113:0/2919724622 learned_addr learned my addr 172.21.15.113:0/2919724622 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:07:57.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a3d400640 1 --2- 172.21.15.113:0/2919724622 >> v2:172.21.15.136:3300/0 conn(0x7f2a480c8200 0x7f2a48072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:57.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a45e00640 1 -- 172.21.15.113:0/2919724622 >> v2:172.21.15.136:3300/0 conn(0x7f2a480c8200 msgr2=0x7f2a48072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:07:57.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a45e00640 1 --2- 172.21.15.113:0/2919724622 >> v2:172.21.15.136:3300/0 conn(0x7f2a480c8200 0x7f2a48072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:57.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a45e00640 1 -- 172.21.15.113:0/2919724622 >> v1:172.21.15.113:6789/0 conn(0x7f2a48152bd0 legacy=0x7f2a481587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:57.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.824+0000 7f2a45e00640 1 -- 172.21.15.113:0/2919724622 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a481594d0 con 0x7f2a481527f0 2024-03-04T08:07:57.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a45e00640 1 --2- 172.21.15.113:0/2919724622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a481527f0 0x7f2a48070d40 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f2a480cd230 tx=0x7f2a3801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352cc2cbc69f6cd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:57.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a45400640 1 -- 172.21.15.113:0/2919724622 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a380b2070 con 0x7f2a481527f0 2024-03-04T08:07:57.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a45400640 1 -- 172.21.15.113:0/2919724622 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a380a5900 con 0x7f2a481527f0 2024-03-04T08:07:57.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a45400640 1 -- 172.21.15.113:0/2919724622 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a380a5cd0 con 0x7f2a481527f0 2024-03-04T08:07:57.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a4d200640 1 -- 172.21.15.113:0/2919724622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a481527f0 msgr2=0x7f2a48070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:57.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a4d200640 1 --2- 172.21.15.113:0/2919724622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a481527f0 0x7f2a48070d40 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f2a480cd230 tx=0x7f2a3801c930 comp rx=0 tx=0).stop 2024-03-04T08:07:57.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a4d200640 1 -- 172.21.15.113:0/2919724622 shutdown_connections 2024-03-04T08:07:57.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a4d200640 1 --2- 172.21.15.113:0/2919724622 >> v2:172.21.15.136:3300/0 conn(0x7f2a480c8200 0x7f2a48072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:57.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a4d200640 1 --2- 172.21.15.113:0/2919724622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a481527f0 0x7f2a48070d40 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:57.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a4d200640 1 -- 172.21.15.113:0/2919724622 >> 172.21.15.113:0/2919724622 conn(0x7f2a4806e820 msgr2=0x7f2a4806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:57.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a4d200640 1 -- 172.21.15.113:0/2919724622 shutdown_connections 2024-03-04T08:07:57.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a4d200640 1 -- 172.21.15.113:0/2919724622 wait complete. 2024-03-04T08:07:57.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a4d200640 1 Processor -- start 2024-03-04T08:07:57.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.828+0000 7f2a4d200640 1 -- start start 2024-03-04T08:07:57.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a4d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a480c8200 0x7f2a481f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:57.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a4d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a481527f0 0x7f2a481566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:57.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a4d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a48152bd0 0x7f2a4814dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:57.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a4d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48165160 con 0x7f2a48152bd0 2024-03-04T08:07:57.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a4d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a480727c0 con 0x7f2a480c8200 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a4d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a481652e0 con 0x7f2a481527f0 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a46800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a48152bd0 0x7f2a4814dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a46800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a48152bd0 0x7f2a4814dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34384/0 (socket says 172.21.15.113:34384) 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a46800640 1 -- 172.21.15.113:0/2611063004 learned_addr learned my addr 172.21.15.113:0/2611063004 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a3d400640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a481527f0 0x7f2a481566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a45e00640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a480c8200 0x7f2a481f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a46800640 1 -- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a481527f0 msgr2=0x7f2a481566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a46800640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a481527f0 0x7f2a481566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a46800640 1 -- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a480c8200 msgr2=0x7f2a481f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a46800640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a480c8200 0x7f2a481f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a46800640 1 -- 172.21.15.113:0/2611063004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a481f76d0 con 0x7f2a48152bd0 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a3d400640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a481527f0 0x7f2a481566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a46800640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a48152bd0 0x7f2a4814dc90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000c3f0 tx=0x7f2a4000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a3ea00640 1 -- 172.21.15.113:0/2611063004 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40017070 con 0x7f2a48152bd0 2024-03-04T08:07:57.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a3ea00640 1 -- 172.21.15.113:0/2611063004 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a40005cf0 con 0x7f2a48152bd0 2024-03-04T08:07:57.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a3ea00640 1 -- 172.21.15.113:0/2611063004 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a400072f0 con 0x7f2a48152bd0 2024-03-04T08:07:57.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a4d200640 1 -- 172.21.15.113:0/2611063004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a481f3730 con 0x7f2a48152bd0 2024-03-04T08:07:57.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a4d200640 1 -- 172.21.15.113:0/2611063004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4814e650 con 0x7f2a48152bd0 2024-03-04T08:07:57.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.832+0000 7f2a4d200640 1 -- 172.21.15.113:0/2611063004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a48075c00 con 0x7f2a48152bd0 2024-03-04T08:07:57.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.836+0000 7f2a3ea00640 1 -- 172.21.15.113:0/2611063004 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a4001a070 con 0x7f2a48152bd0 2024-03-04T08:07:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.836+0000 7f2a3ea00640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a24066f60 0x7f2a24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:07:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.836+0000 7f2a3ea00640 1 -- 172.21.15.113:0/2611063004 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f2a40084a40 con 0x7f2a48152bd0 2024-03-04T08:07:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.836+0000 7f2a45e00640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a24066f60 0x7f2a24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:07:57.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.836+0000 7f2a45e00640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a24066f60 0x7f2a24069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a380af000 tx=0x7f2a380a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:07:57.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:57.840+0000 7f2a3ea00640 1 -- 172.21.15.113:0/2611063004 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2a40055360 con 0x7f2a48152bd0 2024-03-04T08:07:58.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.032+0000 7f2a4d200640 1 -- 172.21.15.113:0/2611063004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a480c79d0 con 0x7f2a48152bd0 2024-03-04T08:07:58.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.032+0000 7f2a3ea00640 1 -- 172.21.15.113:0/2611063004 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f2a40054c60 con 0x7f2a48152bd0 2024-03-04T08:07:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:07:58.031 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":129,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:52.036281+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7269,7119]},"id":1}]} 2024-03-04T08:07:58.031 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 129 2024-03-04T08:07:58.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 -- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a24066f60 msgr2=0x7f2a24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:58.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a24066f60 0x7f2a24069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a380af000 tx=0x7f2a380a7070 comp rx=0 tx=0).stop 2024-03-04T08:07:58.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 -- 172.21.15.113:0/2611063004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a48152bd0 msgr2=0x7f2a4814dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:07:58.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a48152bd0 0x7f2a4814dc90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000c3f0 tx=0x7f2a4000c8b0 comp rx=0 tx=0).stop 2024-03-04T08:07:58.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 -- 172.21.15.113:0/2611063004 shutdown_connections 2024-03-04T08:07:58.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2a24066f60 0x7f2a24069420 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:58.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a48152bd0 0x7f2a4814dc90 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:58.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a481527f0 0x7f2a481566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:58.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 --2- 172.21.15.113:0/2611063004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a480c8200 0x7f2a481f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:07:58.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 -- 172.21.15.113:0/2611063004 >> 172.21.15.113:0/2611063004 conn(0x7f2a4806e820 msgr2=0x7f2a480cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:07:58.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 -- 172.21.15.113:0/2611063004 shutdown_connections 2024-03-04T08:07:58.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:07:58.036+0000 7f2a4d200640 1 -- 172.21.15.113:0/2611063004 wait complete. 2024-03-04T08:07:58.052 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:08:00.053 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:08:00.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.196+0000 7ff03b200640 1 Processor -- start 2024-03-04T08:08:00.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.196+0000 7ff03b200640 1 -- start start 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff03b200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff034152950 0x7ff03406b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff03b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff034159200 0x7ff0340c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff03b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0340c7fc0 0x7ff0340cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff03b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0340731b0 con 0x7ff034159200 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff03b200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03406b660 con 0x7ff034152950 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff03b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340725a0 con 0x7ff0340c7fc0 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff02be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff034159200 0x7ff0340c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff02be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff034159200 0x7ff0340c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34392/0 (socket says 172.21.15.113:34392) 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff02be00640 1 -- 172.21.15.113:0/3644567530 learned_addr learned my addr 172.21.15.113:0/3644567530 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff033e00640 1 --2- 172.21.15.113:0/3644567530 >> v2:172.21.15.136:3300/0 conn(0x7ff034152950 0x7ff03406b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:00.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff039200640 1 --2- 172.21.15.113:0/3644567530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0340c7fc0 0x7ff0340cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:00.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff02be00640 1 -- 172.21.15.113:0/3644567530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0340c7fc0 msgr2=0x7ff0340cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:08:00.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff02be00640 1 --2- 172.21.15.113:0/3644567530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0340c7fc0 0x7ff0340cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff02be00640 1 -- 172.21.15.113:0/3644567530 >> v2:172.21.15.136:3300/0 conn(0x7ff034152950 msgr2=0x7ff03406b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:08:00.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff02be00640 1 --2- 172.21.15.113:0/3644567530 >> v2:172.21.15.136:3300/0 conn(0x7ff034152950 0x7ff03406b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.200+0000 7ff02be00640 1 -- 172.21.15.113:0/3644567530 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff034072880 con 0x7ff034159200 2024-03-04T08:08:00.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff02be00640 1 --2- 172.21.15.113:0/3644567530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 0x7ff0340c7a80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ff03406d190 tx=0x7ff01c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9b325d79a5db54 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:00.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff033400640 1 -- 172.21.15.113:0/3644567530 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0b5070 con 0x7ff034159200 2024-03-04T08:08:00.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff033400640 1 -- 172.21.15.113:0/3644567530 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c0a89d0 con 0x7ff034159200 2024-03-04T08:08:00.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff033400640 1 -- 172.21.15.113:0/3644567530 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c0a8da0 con 0x7ff034159200 2024-03-04T08:08:00.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 -- 172.21.15.113:0/3644567530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 msgr2=0x7ff0340c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:00.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 --2- 172.21.15.113:0/3644567530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 0x7ff0340c7a80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7ff03406d190 tx=0x7ff01c01ed10 comp rx=0 tx=0).stop 2024-03-04T08:08:00.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 -- 172.21.15.113:0/3644567530 shutdown_connections 2024-03-04T08:08:00.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 --2- 172.21.15.113:0/3644567530 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff0340c7fc0 0x7ff0340cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 --2- 172.21.15.113:0/3644567530 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 0x7ff0340c7a80 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 --2- 172.21.15.113:0/3644567530 >> v2:172.21.15.136:3300/0 conn(0x7ff034152950 0x7ff03406b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 -- 172.21.15.113:0/3644567530 >> 172.21.15.113:0/3644567530 conn(0x7ff0340671b0 msgr2=0x7ff03406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:08:00.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 -- 172.21.15.113:0/3644567530 shutdown_connections 2024-03-04T08:08:00.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 -- 172.21.15.113:0/3644567530 wait complete. 2024-03-04T08:08:00.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 Processor -- start 2024-03-04T08:08:00.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.204+0000 7ff03b200640 1 -- start start 2024-03-04T08:08:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff03b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0340c7fc0 0x7ff034157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff03b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff034152950 0x7ff0340ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff03b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 0x7ff034150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff03b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034159620 con 0x7ff034159200 2024-03-04T08:08:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff03b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034072e40 con 0x7ff0340c7fc0 2024-03-04T08:08:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff03b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0341597a0 con 0x7ff034152950 2024-03-04T08:08:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff039200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 0x7ff034150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff039200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 0x7ff034150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34408/0 (socket says 172.21.15.113:34408) 2024-03-04T08:08:00.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff039200640 1 -- 172.21.15.113:0/1026430993 learned_addr learned my addr 172.21.15.113:0/1026430993 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:08:00.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff02be00640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff034152950 0x7ff0340ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:00.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff033e00640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0340c7fc0 0x7ff034157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:00.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff039200640 1 -- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff034152950 msgr2=0x7ff0340ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:00.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff039200640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff034152950 0x7ff0340ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff039200640 1 -- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0340c7fc0 msgr2=0x7ff034157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:00.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff039200640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0340c7fc0 0x7ff034157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff039200640 1 -- 172.21.15.113:0/1026430993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03415b200 con 0x7ff034159200 2024-03-04T08:08:00.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff039200640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 0x7ff034150e20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00ef00 tx=0x7ff02c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:00.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff030c00640 1 -- 172.21.15.113:0/1026430993 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c018020 con 0x7ff034159200 2024-03-04T08:08:00.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff030c00640 1 -- 172.21.15.113:0/1026430993 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c00bd20 con 0x7ff034159200 2024-03-04T08:08:00.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff03b200640 1 -- 172.21.15.113:0/1026430993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03415af60 con 0x7ff034159200 2024-03-04T08:08:00.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff030c00640 1 -- 172.21.15.113:0/1026430993 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c014640 con 0x7ff034159200 2024-03-04T08:08:00.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff03b200640 1 -- 172.21.15.113:0/1026430993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0340b0b20 con 0x7ff034159200 2024-03-04T08:08:00.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff03b200640 1 -- 172.21.15.113:0/1026430993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff034075c00 con 0x7ff034159200 2024-03-04T08:08:00.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.208+0000 7ff030c00640 1 -- 172.21.15.113:0/1026430993 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff02c01b020 con 0x7ff034159200 2024-03-04T08:08:00.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.212+0000 7ff030c00640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff010066f60 0x7ff010069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:00.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.212+0000 7ff030c00640 1 -- 172.21.15.113:0/1026430993 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7ff02c085ad0 con 0x7ff034159200 2024-03-04T08:08:00.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.212+0000 7ff033e00640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff010066f60 0x7ff010069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:00.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.212+0000 7ff033e00640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff010066f60 0x7ff010069420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff024009b60 tx=0x7ff024009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:00.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.216+0000 7ff030c00640 1 -- 172.21.15.113:0/1026430993 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff02c022070 con 0x7ff034159200 2024-03-04T08:08:00.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.408+0000 7ff03b200640 1 -- 172.21.15.113:0/1026430993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff034158110 con 0x7ff034159200 2024-03-04T08:08:00.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff030c00640 1 -- 172.21.15.113:0/1026430993 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+5830 (secure 0 0 0) 0x7ff02c055e40 con 0x7ff034159200 2024-03-04T08:08:00.407 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:08:00.407 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":130,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:57.331078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:clientreplay","state_seq":30,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7119,7269]},"id":1}]} 2024-03-04T08:08:00.407 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 130 2024-03-04T08:08:00.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 -- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff010066f60 msgr2=0x7ff010069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff010066f60 0x7ff010069420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff024009b60 tx=0x7ff024009190 comp rx=0 tx=0).stop 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 -- 172.21.15.113:0/1026430993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 msgr2=0x7ff034150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 0x7ff034150e20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00ef00 tx=0x7ff02c00b480 comp rx=0 tx=0).stop 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 -- 172.21.15.113:0/1026430993 shutdown_connections 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff010066f60 0x7ff010069420 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff034159200 0x7ff034150e20 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff034152950 0x7ff0340ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 --2- 172.21.15.113:0/1026430993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0340c7fc0 0x7ff034157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 -- 172.21.15.113:0/1026430993 >> 172.21.15.113:0/1026430993 conn(0x7ff0340671b0 msgr2=0x7ff0340c20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 -- 172.21.15.113:0/1026430993 shutdown_connections 2024-03-04T08:08:00.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:00.412+0000 7ff03b200640 1 -- 172.21.15.113:0/1026430993 wait complete. 2024-03-04T08:08:00.438 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:08:02.439 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.584+0000 7f0a42e00640 1 Processor -- start 2024-03-04T08:08:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.584+0000 7f0a42e00640 1 -- start start 2024-03-04T08:08:02.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a42e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0a3c0ccb40 0x7f0a3c0c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:02.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a42e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0a3c15aa40 0x7f0a3c154800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:02.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a42e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c15aed0 con 0x7f0a3c0ccf20 2024-03-04T08:08:02.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a42e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c15b170 con 0x7f0a3c15aa40 2024-03-04T08:08:02.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a42e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c15b410 con 0x7f0a3c0ccb40 2024-03-04T08:08:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a40e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0a3c0ccf20 0x7f0a3c15e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60380/0 (socket says 172.21.15.113:60380) 2024-03-04T08:08:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a40e00640 1 -- 172.21.15.113:0/2053402476 learned_addr learned my addr 172.21.15.113:0/2053402476 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:08:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a33e00640 1 --2- 172.21.15.113:0/2053402476 >> v2:172.21.15.113:3300/0 conn(0x7f0a3c15aa40 0x7f0a3c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a3be00640 1 --2- 172.21.15.113:0/2053402476 >> v2:172.21.15.136:3300/0 conn(0x7f0a3c0ccb40 0x7f0a3c0c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a3b400640 1 -- 172.21.15.113:0/2053402476 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863498745 0 0) 0x7f0a3c15aed0 con 0x7f0a3c0ccf20 2024-03-04T08:08:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a3b400640 1 -- 172.21.15.113:0/2053402476 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a20003530 con 0x7f0a3c0ccf20 2024-03-04T08:08:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a33e00640 1 -- 172.21.15.113:0/2053402476 >> v2:172.21.15.136:3300/0 conn(0x7f0a3c0ccb40 msgr2=0x7f0a3c0c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:08:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a33e00640 1 --2- 172.21.15.113:0/2053402476 >> v2:172.21.15.136:3300/0 conn(0x7f0a3c0ccb40 0x7f0a3c0c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a33e00640 1 -- 172.21.15.113:0/2053402476 >> v1:172.21.15.113:6789/0 conn(0x7f0a3c0ccf20 legacy=0x7f0a3c15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:02.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.588+0000 7f0a33e00640 1 -- 172.21.15.113:0/2053402476 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c15b9b0 con 0x7f0a3c15aa40 2024-03-04T08:08:02.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a33e00640 1 --2- 172.21.15.113:0/2053402476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c15aa40 0x7f0a3c154800 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f0a240097c0 tx=0x7f0a2401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1f43f04cce73f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:02.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a3b400640 1 -- 172.21.15.113:0/2053402476 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a240b5070 con 0x7f0a3c15aa40 2024-03-04T08:08:02.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a3b400640 1 -- 172.21.15.113:0/2053402476 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a240a8950 con 0x7f0a3c15aa40 2024-03-04T08:08:02.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a3b400640 1 -- 172.21.15.113:0/2053402476 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a240a8d20 con 0x7f0a3c15aa40 2024-03-04T08:08:02.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a42e00640 1 -- 172.21.15.113:0/2053402476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c15aa40 msgr2=0x7f0a3c154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:02.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a42e00640 1 --2- 172.21.15.113:0/2053402476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c15aa40 0x7f0a3c154800 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f0a240097c0 tx=0x7f0a2401ec50 comp rx=0 tx=0).stop 2024-03-04T08:08:02.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a42e00640 1 -- 172.21.15.113:0/2053402476 shutdown_connections 2024-03-04T08:08:02.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a42e00640 1 --2- 172.21.15.113:0/2053402476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c15aa40 0x7f0a3c154800 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:02.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a42e00640 1 --2- 172.21.15.113:0/2053402476 >> v2:172.21.15.136:3300/0 conn(0x7f0a3c0ccb40 0x7f0a3c0c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:02.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a42e00640 1 -- 172.21.15.113:0/2053402476 >> 172.21.15.113:0/2053402476 conn(0x7f0a3c06b060 msgr2=0x7f0a3c06b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:08:02.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a42e00640 1 -- 172.21.15.113:0/2053402476 shutdown_connections 2024-03-04T08:08:02.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a42e00640 1 -- 172.21.15.113:0/2053402476 wait complete. 2024-03-04T08:08:02.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a42e00640 1 Processor -- start 2024-03-04T08:08:02.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.592+0000 7f0a42e00640 1 -- start start 2024-03-04T08:08:02.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a42e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a3c0ccb40 0x7f0a3c1f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:02.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a42e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a3c0ccf20 0x7f0a3c153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:02.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a42e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c159c90 0x7f0a3c0c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:02.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a42e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0c78b0 con 0x7f0a3c159c90 2024-03-04T08:08:02.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a42e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c15e0d0 con 0x7f0a3c0ccb40 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a42e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c15ed00 con 0x7f0a3c0ccf20 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a40e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c159c90 0x7f0a3c0c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a40e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c159c90 0x7f0a3c0c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34438/0 (socket says 172.21.15.113:34438) 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a40e00640 1 -- 172.21.15.113:0/2448086591 learned_addr learned my addr 172.21.15.113:0/2448086591 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a33e00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a3c0ccf20 0x7f0a3c153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a3be00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a3c0ccb40 0x7f0a3c1f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a40e00640 1 -- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a3c0ccf20 msgr2=0x7f0a3c153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a40e00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a3c0ccf20 0x7f0a3c153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a40e00640 1 -- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a3c0ccb40 msgr2=0x7f0a3c1f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a40e00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a3c0ccb40 0x7f0a3c1f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a40e00640 1 -- 172.21.15.113:0/2448086591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c1f4160 con 0x7f0a3c159c90 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a40e00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c159c90 0x7f0a3c0c4b40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400cb40 tx=0x7f0a3400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a38c00640 1 -- 172.21.15.113:0/2448086591 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34017070 con 0x7f0a3c159c90 2024-03-04T08:08:02.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a38c00640 1 -- 172.21.15.113:0/2448086591 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3400aca0 con 0x7f0a3c159c90 2024-03-04T08:08:02.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a42e00640 1 -- 172.21.15.113:0/2448086591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c1efee0 con 0x7f0a3c159c90 2024-03-04T08:08:02.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a38c00640 1 -- 172.21.15.113:0/2448086591 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34012620 con 0x7f0a3c159c90 2024-03-04T08:08:02.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a42e00640 1 -- 172.21.15.113:0/2448086591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c1f04a0 con 0x7f0a3c159c90 2024-03-04T08:08:02.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.596+0000 7f0a42e00640 1 -- 172.21.15.113:0/2448086591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c075c00 con 0x7f0a3c159c90 2024-03-04T08:08:02.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.600+0000 7f0a38c00640 1 -- 172.21.15.113:0/2448086591 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a3401a070 con 0x7f0a3c159c90 2024-03-04T08:08:02.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.600+0000 7f0a38c00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a18066f90 0x7f0a18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:02.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.600+0000 7f0a38c00640 1 -- 172.21.15.113:0/2448086591 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f0a34085550 con 0x7f0a3c159c90 2024-03-04T08:08:02.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.600+0000 7f0a3be00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a18066f90 0x7f0a18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:02.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.600+0000 7f0a3be00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a18066f90 0x7f0a18069450 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009b60 tx=0x7f0a2c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:02.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.604+0000 7f0a38c00640 1 -- 172.21.15.113:0/2448086591 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0a34055700 con 0x7f0a3c159c90 2024-03-04T08:08:02.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.924+0000 7f0a42e00640 1 -- 172.21.15.113:0/2448086591 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a3c0c9f10 con 0x7f0a3c159c90 2024-03-04T08:08:02.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.924+0000 7f0a38c00640 1 -- 172.21.15.113:0/2448086591 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+5830 (secure 0 0 0) 0x7f0a340597a0 con 0x7f0a3c159c90 2024-03-04T08:08:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:08:02.922 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":130,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:07:57.331078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:clientreplay","state_seq":30,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7119,"qdb_cluster":[7119,7269]},"id":1}]} 2024-03-04T08:08:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 130 2024-03-04T08:08:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 -- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a18066f90 msgr2=0x7f0a18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a18066f90 0x7f0a18069450 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009b60 tx=0x7f0a2c009190 comp rx=0 tx=0).stop 2024-03-04T08:08:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 -- 172.21.15.113:0/2448086591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c159c90 msgr2=0x7f0a3c0c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c159c90 0x7f0a3c0c4b40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400cb40 tx=0x7f0a3400a060 comp rx=0 tx=0).stop 2024-03-04T08:08:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 -- 172.21.15.113:0/2448086591 shutdown_connections 2024-03-04T08:08:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a18066f90 0x7f0a18069450 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a3c159c90 0x7f0a3c0c4b40 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a3c0ccf20 0x7f0a3c153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 --2- 172.21.15.113:0/2448086591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a3c0ccb40 0x7f0a3c1f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 -- 172.21.15.113:0/2448086591 >> 172.21.15.113:0/2448086591 conn(0x7f0a3c06b060 msgr2=0x7f0a3c06b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:08:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 -- 172.21.15.113:0/2448086591 shutdown_connections 2024-03-04T08:08:02.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:02.928+0000 7f0a42e00640 1 -- 172.21.15.113:0/2448086591 wait complete. 2024-03-04T08:08:02.945 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:08:04.946 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:08:05.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.088+0000 7f8367e00640 1 Processor -- start 2024-03-04T08:08:05.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.088+0000 7f8367e00640 1 -- start start 2024-03-04T08:08:05.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8367e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8360152d00 0x7f83600ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:05.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8367e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 0x7f83601631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:05.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8367e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83600c5470 con 0x7f83600ca0d0 2024-03-04T08:08:05.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8367e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83600c5220 con 0x7f8360152d00 2024-03-04T08:08:05.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8367e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83600cba40 con 0x7f83600703e0 2024-03-04T08:08:05.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8365400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8360152d00 0x7f83600ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:05.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8364a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f83600ca0d0 0x7f83600ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60394/0 (socket says 172.21.15.113:60394) 2024-03-04T08:08:05.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8364a00640 1 -- 172.21.15.113:0/253111349 learned_addr learned my addr 172.21.15.113:0/253111349 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:08:05.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8365e00640 1 --2- 172.21.15.113:0/253111349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 0x7f83601631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:05.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f835fe00640 1 -- 172.21.15.113:0/253111349 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678121668 0 0) 0x7f83600c5470 con 0x7f83600ca0d0 2024-03-04T08:08:05.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f835fe00640 1 -- 172.21.15.113:0/253111349 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8348003530 con 0x7f83600ca0d0 2024-03-04T08:08:05.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8365400640 1 -- 172.21.15.113:0/253111349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 msgr2=0x7f83601631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:08:05.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8365400640 1 --2- 172.21.15.113:0/253111349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 0x7f83601631a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:05.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8365400640 1 -- 172.21.15.113:0/253111349 >> v1:172.21.15.113:6789/0 conn(0x7f83600ca0d0 legacy=0x7f83600ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:05.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8365400640 1 -- 172.21.15.113:0/253111349 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836006f690 con 0x7f8360152d00 2024-03-04T08:08:05.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.092+0000 7f8365e00640 1 --2- 172.21.15.113:0/253111349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 0x7f83601631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:08:05.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8365400640 1 --2- 172.21.15.113:0/253111349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8360152d00 0x7f83600ca5f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f83540097c0 tx=0x7f835401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979394de55d94ad7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:05.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f835fe00640 1 -- 172.21.15.113:0/253111349 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83540b0070 con 0x7f8360152d00 2024-03-04T08:08:05.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f835fe00640 1 -- 172.21.15.113:0/253111349 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83540ab600 con 0x7f8360152d00 2024-03-04T08:08:05.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f835fe00640 1 -- 172.21.15.113:0/253111349 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83540ab9d0 con 0x7f8360152d00 2024-03-04T08:08:05.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8367e00640 1 -- 172.21.15.113:0/253111349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8360152d00 msgr2=0x7f83600ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:05.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8367e00640 1 --2- 172.21.15.113:0/253111349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8360152d00 0x7f83600ca5f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f83540097c0 tx=0x7f835401c410 comp rx=0 tx=0).stop 2024-03-04T08:08:05.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8367e00640 1 -- 172.21.15.113:0/253111349 shutdown_connections 2024-03-04T08:08:05.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8367e00640 1 --2- 172.21.15.113:0/253111349 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 0x7f83601631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:05.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8367e00640 1 --2- 172.21.15.113:0/253111349 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8360152d00 0x7f83600ca5f0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:05.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8367e00640 1 -- 172.21.15.113:0/253111349 >> 172.21.15.113:0/253111349 conn(0x7f8360066db0 msgr2=0x7f83600671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:08:05.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8367e00640 1 -- 172.21.15.113:0/253111349 shutdown_connections 2024-03-04T08:08:05.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8367e00640 1 -- 172.21.15.113:0/253111349 wait complete. 2024-03-04T08:08:05.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8367e00640 1 Processor -- start 2024-03-04T08:08:05.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.096+0000 7f8367e00640 1 -- start start 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8367e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 0x7f83600cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8367e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83600ca0d0 0x7f8360162af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8367e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8360152d00 0x7f836006c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8367e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83600cacb0 con 0x7f83600ca0d0 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8367e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83600cab30 con 0x7f8360152d00 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8367e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83600cead0 con 0x7f83600703e0 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8364a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83600ca0d0 0x7f8360162af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8364a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83600ca0d0 0x7f8360162af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34468/0 (socket says 172.21.15.113:34468) 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8364a00640 1 -- 172.21.15.113:0/1838914484 learned_addr learned my addr 172.21.15.113:0/1838914484 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8365400640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 0x7f83600cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8365e00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8360152d00 0x7f836006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:05.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8364a00640 1 -- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 msgr2=0x7f83600cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:05.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8364a00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 0x7f83600cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:05.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8364a00640 1 -- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8360152d00 msgr2=0x7f836006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:05.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8364a00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8360152d00 0x7f836006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:05.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8364a00640 1 -- 172.21.15.113:0/1838914484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836006d740 con 0x7f83600ca0d0 2024-03-04T08:08:05.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8365400640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 0x7f83600cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:08:05.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8365e00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8360152d00 0x7f836006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:08:05.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8364a00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83600ca0d0 0x7f8360162af0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c480 tx=0x7f834c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:05.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f835d600640 1 -- 172.21.15.113:0/1838914484 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c017070 con 0x7f83600ca0d0 2024-03-04T08:08:05.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f835d600640 1 -- 172.21.15.113:0/1838914484 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c005cf0 con 0x7f83600ca0d0 2024-03-04T08:08:05.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f835d600640 1 -- 172.21.15.113:0/1838914484 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c007290 con 0x7f83600ca0d0 2024-03-04T08:08:05.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8367e00640 1 -- 172.21.15.113:0/1838914484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836006d9b0 con 0x7f83600ca0d0 2024-03-04T08:08:05.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8367e00640 1 -- 172.21.15.113:0/1838914484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8360153720 con 0x7f83600ca0d0 2024-03-04T08:08:05.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.100+0000 7f8367e00640 1 -- 172.21.15.113:0/1838914484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8360153230 con 0x7f83600ca0d0 2024-03-04T08:08:05.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.104+0000 7f835d600640 1 -- 172.21.15.113:0/1838914484 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f834c01a070 con 0x7f83600ca0d0 2024-03-04T08:08:05.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.104+0000 7f835d600640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8340066f60 0x7f8340069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:05.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.104+0000 7f835d600640 1 -- 172.21.15.113:0/1838914484 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f834c084b70 con 0x7f83600ca0d0 2024-03-04T08:08:05.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.104+0000 7f8365400640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8340066f60 0x7f8340069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:05.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.104+0000 7f8365400640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8340066f60 0x7f8340069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f83540ad000 tx=0x7f83540a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:05.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.108+0000 7f835d600640 1 -- 172.21.15.113:0/1838914484 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f834c0554a0 con 0x7f83600ca0d0 2024-03-04T08:08:05.317 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:08:05.320 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:08:05.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.396+0000 7f8367e00640 1 -- 172.21.15.113:0/1838914484 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8360075c00 con 0x7f83600ca0d0 2024-03-04T08:08:05.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.396+0000 7f835d600640 1 -- 172.21.15.113:0/1838914484 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5829 (secure 0 0 0) 0x7f834c054da0 con 0x7f83600ca0d0 2024-03-04T08:08:05.392 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:08:05.392 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:08:04.266596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7119,7269,7389]},"id":1}]} 2024-03-04T08:08:05.392 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 131 2024-03-04T08:08:05.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 -- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8340066f60 msgr2=0x7f8340069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:05.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8340066f60 0x7f8340069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f83540ad000 tx=0x7f83540a6000 comp rx=0 tx=0).stop 2024-03-04T08:08:05.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 -- 172.21.15.113:0/1838914484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83600ca0d0 msgr2=0x7f8360162af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:05.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83600ca0d0 0x7f8360162af0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c480 tx=0x7f834c00c940 comp rx=0 tx=0).stop 2024-03-04T08:08:05.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 -- 172.21.15.113:0/1838914484 shutdown_connections 2024-03-04T08:08:05.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8360152d00 0x7f836006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:05.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8340066f60 0x7f8340069420 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:05.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83600ca0d0 0x7f8360162af0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:05.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 --2- 172.21.15.113:0/1838914484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83600703e0 0x7f83600cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:05.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 -- 172.21.15.113:0/1838914484 >> 172.21.15.113:0/1838914484 conn(0x7f8360066db0 msgr2=0x7f83600c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:08:05.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 -- 172.21.15.113:0/1838914484 shutdown_connections 2024-03-04T08:08:05.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:05.400+0000 7f8367e00640 1 -- 172.21.15.113:0/1838914484 wait complete. 2024-03-04T08:08:05.415 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=0, replacing gid=6925 2024-03-04T08:08:05.416 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.b 2024-03-04T08:08:07.417 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2024-03-04T08:08:07.417 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-04T08:08:07.418 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-04T08:08:07.421 INFO:tasks.ceph.mds.b:Started 2024-03-04T08:08:07.421 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:08:07.498 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2024-03-04T08:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d20200640 1 Processor -- start 2024-03-04T08:08:07.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d20200640 1 -- start start 2024-03-04T08:08:07.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d20200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d181527f0 0x7f5d18070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:07.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d20200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5d180c8200 0x7f5d18072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:07.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d20200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d181591f0 con 0x7f5d18152bd0 2024-03-04T08:08:07.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d20200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d18071df0 con 0x7f5d181527f0 2024-03-04T08:08:07.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d20200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d181597b0 con 0x7f5d180c8200 2024-03-04T08:08:07.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d1d800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d181527f0 0x7f5d18070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:07.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d1d800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d181527f0 0x7f5d18070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34442/0 (socket says 172.21.15.113:34442) 2024-03-04T08:08:07.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d1d800640 1 -- 172.21.15.113:0/1457045183 learned_addr learned my addr 172.21.15.113:0/1457045183 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:08:07.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d1ce00640 1 --2- 172.21.15.113:0/1457045183 >> v2:172.21.15.136:3300/0 conn(0x7f5d180c8200 0x7f5d18072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:07.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d1d800640 1 -- 172.21.15.113:0/1457045183 >> v2:172.21.15.136:3300/0 conn(0x7f5d180c8200 msgr2=0x7f5d18072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:08:07.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d1d800640 1 --2- 172.21.15.113:0/1457045183 >> v2:172.21.15.136:3300/0 conn(0x7f5d180c8200 0x7f5d18072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:07.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d1d800640 1 -- 172.21.15.113:0/1457045183 >> v1:172.21.15.136:6789/0 conn(0x7f5d18152bd0 legacy=0x7f5d181587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:07.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.564+0000 7f5d1d800640 1 -- 172.21.15.113:0/1457045183 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d181594d0 con 0x7f5d181527f0 2024-03-04T08:08:07.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d1d800640 1 --2- 172.21.15.113:0/1457045183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 0x7f5d18070d40 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f5d180cd230 tx=0x7f5d0c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985a86cf5d298e4d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:07.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d17e00640 1 -- 172.21.15.113:0/1457045183 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c0ab430 con 0x7f5d181527f0 2024-03-04T08:08:07.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d17e00640 1 -- 172.21.15.113:0/1457045183 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c0b0070 con 0x7f5d181527f0 2024-03-04T08:08:07.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d17e00640 1 -- 172.21.15.113:0/1457045183 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c0ab7f0 con 0x7f5d181527f0 2024-03-04T08:08:07.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d20200640 1 -- 172.21.15.113:0/1457045183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 msgr2=0x7f5d18070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:07.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d20200640 1 --2- 172.21.15.113:0/1457045183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 0x7f5d18070d40 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f5d180cd230 tx=0x7f5d0c01c520 comp rx=0 tx=0).stop 2024-03-04T08:08:07.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d20200640 1 -- 172.21.15.113:0/1457045183 shutdown_connections 2024-03-04T08:08:07.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d20200640 1 --2- 172.21.15.113:0/1457045183 >> v2:172.21.15.136:3300/0 conn(0x7f5d180c8200 0x7f5d18072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:07.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d20200640 1 --2- 172.21.15.113:0/1457045183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 0x7f5d18070d40 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:07.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d20200640 1 -- 172.21.15.113:0/1457045183 >> 172.21.15.113:0/1457045183 conn(0x7f5d1806e820 msgr2=0x7f5d1806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:08:07.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d20200640 1 -- 172.21.15.113:0/1457045183 shutdown_connections 2024-03-04T08:08:07.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.568+0000 7f5d20200640 1 -- 172.21.15.113:0/1457045183 wait complete. 2024-03-04T08:08:07.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d20200640 1 Processor -- start 2024-03-04T08:08:07.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d20200640 1 -- start start 2024-03-04T08:08:07.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d20200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d180c8200 0x7f5d181428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:07.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d20200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 0x7f5d1813db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:07.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d20200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d18152bd0 0x7f5d1813e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:07.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d20200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d18165160 con 0x7f5d181527f0 2024-03-04T08:08:07.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d20200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d180727c0 con 0x7f5d180c8200 2024-03-04T08:08:07.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d20200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d181652e0 con 0x7f5d18152bd0 2024-03-04T08:08:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d1ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 0x7f5d1813db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d1ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 0x7f5d1813db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34448/0 (socket says 172.21.15.113:34448) 2024-03-04T08:08:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d1ce00640 1 -- 172.21.15.113:0/805223999 learned_addr learned my addr 172.21.15.113:0/805223999 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:08:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d1d800640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d180c8200 0x7f5d181428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.572+0000 7f5d1e200640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d18152bd0 0x7f5d1813e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d1ce00640 1 -- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d18152bd0 msgr2=0x7f5d1813e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d1ce00640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d18152bd0 0x7f5d1813e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d1ce00640 1 -- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d180c8200 msgr2=0x7f5d181428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d1ce00640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d180c8200 0x7f5d181428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d1ce00640 1 -- 172.21.15.113:0/805223999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d180afaf0 con 0x7f5d181527f0 2024-03-04T08:08:07.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d1ce00640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 0x7f5d1813db60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400e900 tx=0x7f5d0400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:07.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d15600640 1 -- 172.21.15.113:0/805223999 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04017070 con 0x7f5d181527f0 2024-03-04T08:08:07.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d15600640 1 -- 172.21.15.113:0/805223999 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0400ca00 con 0x7f5d181527f0 2024-03-04T08:08:07.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d15600640 1 -- 172.21.15.113:0/805223999 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400cdf0 con 0x7f5d181527f0 2024-03-04T08:08:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d20200640 1 -- 172.21.15.113:0/805223999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d180afdd0 con 0x7f5d181527f0 2024-03-04T08:08:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d20200640 1 -- 172.21.15.113:0/805223999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d180b1500 con 0x7f5d181527f0 2024-03-04T08:08:07.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d20200640 1 -- 172.21.15.113:0/805223999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d18075c00 con 0x7f5d181527f0 2024-03-04T08:08:07.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.576+0000 7f5d15600640 1 -- 172.21.15.113:0/805223999 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d0401a070 con 0x7f5d181527f0 2024-03-04T08:08:07.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.580+0000 7f5d15600640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cf8066f60 0x7f5cf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:07.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.580+0000 7f5d15600640 1 -- 172.21.15.113:0/805223999 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f5d04085a00 con 0x7f5d181527f0 2024-03-04T08:08:07.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.580+0000 7f5d1d800640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cf8066f60 0x7f5cf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:07.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.580+0000 7f5d1d800640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cf8066f60 0x7f5cf8069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0af000 tx=0x7f5d0c0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:07.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.588+0000 7f5d15600640 1 -- 172.21.15.113:0/805223999 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5d04055ce0 con 0x7f5d181527f0 2024-03-04T08:08:07.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 -- 172.21.15.113:0/805223999 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d18074c10 con 0x7f5d181527f0 2024-03-04T08:08:07.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d15600640 1 -- 172.21.15.113:0/805223999 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+5829 (secure 0 0 0) 0x7f5d04059d80 con 0x7f5d181527f0 2024-03-04T08:08:07.781 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:08:07.781 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:08:04.266596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7119,7269,7389]},"id":1}]} 2024-03-04T08:08:07.781 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 131 2024-03-04T08:08:07.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 -- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cf8066f60 msgr2=0x7f5cf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:07.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cf8066f60 0x7f5cf8069420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0af000 tx=0x7f5d0c0a7000 comp rx=0 tx=0).stop 2024-03-04T08:08:07.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 -- 172.21.15.113:0/805223999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 msgr2=0x7f5d1813db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:07.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 0x7f5d1813db60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400e900 tx=0x7f5d0400edc0 comp rx=0 tx=0).stop 2024-03-04T08:08:07.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 -- 172.21.15.113:0/805223999 shutdown_connections 2024-03-04T08:08:07.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d18152bd0 0x7f5d1813e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5cf8066f60 0x7f5cf8069420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d181527f0 0x7f5d1813db60 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 --2- 172.21.15.113:0/805223999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d180c8200 0x7f5d181428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 -- 172.21.15.113:0/805223999 >> 172.21.15.113:0/805223999 conn(0x7f5d1806e820 msgr2=0x7f5d180ca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:08:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 -- 172.21.15.113:0/805223999 shutdown_connections 2024-03-04T08:08:07.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:07.784+0000 7f5d20200640 1 -- 172.21.15.113:0/805223999 wait complete. 2024-03-04T08:08:07.802 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2024-03-04T08:08:09.804 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:08:10.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8be00640 1 Processor -- start 2024-03-04T08:08:10.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8be00640 1 -- start start 2024-03-04T08:08:10.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8be00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fca8c154c40 0x7fca8c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:10.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca8c0cc2d0 0x7fca8c0c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:10.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c0d0030 0x7fca8c0d0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:10.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8be00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c0cb4c0 con 0x7fca8c0cc2d0 2024-03-04T08:08:10.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8be00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c155020 con 0x7fca8c154c40 2024-03-04T08:08:10.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c15df00 con 0x7fca8c0d0030 2024-03-04T08:08:10.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca8c0cc2d0 0x7fca8c0c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:10.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca8c0cc2d0 0x7fca8c0c4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34456/0 (socket says 172.21.15.113:34456) 2024-03-04T08:08:10.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8a000640 1 -- 172.21.15.113:0/1313065775 learned_addr learned my addr 172.21.15.113:0/1313065775 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:08:10.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8b400640 1 --2- 172.21.15.113:0/1313065775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c0d0030 0x7fca8c0d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8aa00640 1 --2- 172.21.15.113:0/1313065775 >> v2:172.21.15.136:3300/0 conn(0x7fca8c154c40 0x7fca8c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8a000640 1 -- 172.21.15.113:0/1313065775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c0d0030 msgr2=0x7fca8c0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:08:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8a000640 1 --2- 172.21.15.113:0/1313065775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c0d0030 0x7fca8c0d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8a000640 1 -- 172.21.15.113:0/1313065775 >> v2:172.21.15.136:3300/0 conn(0x7fca8c154c40 msgr2=0x7fca8c15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:08:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8a000640 1 --2- 172.21.15.113:0/1313065775 >> v2:172.21.15.136:3300/0 conn(0x7fca8c154c40 0x7fca8c15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.020+0000 7fca8a000640 1 -- 172.21.15.113:0/1313065775 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c15e1e0 con 0x7fca8c0cc2d0 2024-03-04T08:08:10.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.024+0000 7fca8a000640 1 --2- 172.21.15.113:0/1313065775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8c0cc2d0 0x7fca8c0c4fc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0c71a0 tx=0x7fca7801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c8ad3abaa95a42 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:10.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.024+0000 7fca89600640 1 -- 172.21.15.113:0/1313065775 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780b5070 con 0x7fca8c0cc2d0 2024-03-04T08:08:10.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.024+0000 7fca89600640 1 -- 172.21.15.113:0/1313065775 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca780a89d0 con 0x7fca8c0cc2d0 2024-03-04T08:08:10.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.024+0000 7fca89600640 1 -- 172.21.15.113:0/1313065775 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780a8da0 con 0x7fca8c0cc2d0 2024-03-04T08:08:10.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 -- 172.21.15.113:0/1313065775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8c0cc2d0 msgr2=0x7fca8c0c4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:10.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 --2- 172.21.15.113:0/1313065775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8c0cc2d0 0x7fca8c0c4fc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0c71a0 tx=0x7fca7801ed10 comp rx=0 tx=0).stop 2024-03-04T08:08:10.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 -- 172.21.15.113:0/1313065775 shutdown_connections 2024-03-04T08:08:10.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 --2- 172.21.15.113:0/1313065775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c0d0030 0x7fca8c0d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 --2- 172.21.15.113:0/1313065775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8c0cc2d0 0x7fca8c0c4fc0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 --2- 172.21.15.113:0/1313065775 >> v2:172.21.15.136:3300/0 conn(0x7fca8c154c40 0x7fca8c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 -- 172.21.15.113:0/1313065775 >> 172.21.15.113:0/1313065775 conn(0x7fca8c06d3e0 msgr2=0x7fca8c06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:08:10.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 -- 172.21.15.113:0/1313065775 shutdown_connections 2024-03-04T08:08:10.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 -- 172.21.15.113:0/1313065775 wait complete. 2024-03-04T08:08:10.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 Processor -- start 2024-03-04T08:08:10.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.028+0000 7fca8be00640 1 -- start start 2024-03-04T08:08:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8c0cc2d0 0x7fca8c156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca8c0d0030 0x7fca8c1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c154c40 0x7fca8c14a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c15ecc0 con 0x7fca8c0cc2d0 2024-03-04T08:08:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c15e7a0 con 0x7fca8c0d0030 2024-03-04T08:08:10.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0cb180 con 0x7fca8c154c40 2024-03-04T08:08:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c154c40 0x7fca8c14a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c154c40 0x7fca8c14a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:60426/0 (socket says 172.21.15.113:60426) 2024-03-04T08:08:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8b400640 1 -- 172.21.15.113:0/3110660298 learned_addr learned my addr 172.21.15.113:0/3110660298 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:08:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8a000640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca8c0d0030 0x7fca8c1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:10.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8aa00640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8c0cc2d0 0x7fca8c156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8a000640 1 -- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c154c40 msgr2=0x7fca8c14a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8a000640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c154c40 0x7fca8c14a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8a000640 1 -- 172.21.15.113:0/3110660298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8c0cc2d0 msgr2=0x7fca8c156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8a000640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8c0cc2d0 0x7fca8c156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8a000640 1 -- 172.21.15.113:0/3110660298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c1f8b20 con 0x7fca8c0d0030 2024-03-04T08:08:10.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8a000640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca8c0d0030 0x7fca8c1565b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca780af000 tx=0x7fca7801bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca82a00640 1 -- 172.21.15.113:0/3110660298 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780b5070 con 0x7fca8c0d0030 2024-03-04T08:08:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca82a00640 1 -- 172.21.15.113:0/3110660298 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca780b0b50 con 0x7fca8c0d0030 2024-03-04T08:08:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca82a00640 1 -- 172.21.15.113:0/3110660298 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780bb410 con 0x7fca8c0d0030 2024-03-04T08:08:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.032+0000 7fca8be00640 1 -- 172.21.15.113:0/3110660298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c154470 con 0x7fca8c0d0030 2024-03-04T08:08:10.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.036+0000 7fca8be00640 1 -- 172.21.15.113:0/3110660298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c15a8a0 con 0x7fca8c0d0030 2024-03-04T08:08:10.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.036+0000 7fca8be00640 1 -- 172.21.15.113:0/3110660298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8c075a80 con 0x7fca8c0d0030 2024-03-04T08:08:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.040+0000 7fca82a00640 1 -- 172.21.15.113:0/3110660298 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca780c2060 con 0x7fca8c0d0030 2024-03-04T08:08:10.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.040+0000 7fca82a00640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca6c066fe0 0x7fca6c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:08:10.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.040+0000 7fca82a00640 1 -- 172.21.15.113:0/3110660298 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7fca781229e0 con 0x7fca8c0d0030 2024-03-04T08:08:10.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.040+0000 7fca8aa00640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca6c066fe0 0x7fca6c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:08:10.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.040+0000 7fca8aa00640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca6c066fe0 0x7fca6c0694a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0079a0 tx=0x7fca7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:08:10.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.056+0000 7fca82a00640 1 -- 172.21.15.113:0/3110660298 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fca78018870 con 0x7fca8c0d0030 2024-03-04T08:08:10.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.316+0000 7fca8be00640 1 -- 172.21.15.113:0/3110660298 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca8c15adb0 con 0x7fca8c0d0030 2024-03-04T08:08:10.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.316+0000 7fca82a00640 1 -- 172.21.15.113:0/3110660298 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+6621 (secure 0 0 0) 0x7fca780f2e30 con 0x7fca8c0d0030 2024-03-04T08:08:10.314 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:08:10.315 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:08:04.266596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269,7119]},"id":1}]} 2024-03-04T08:08:10.315 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 132 2024-03-04T08:08:10.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 -- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca6c066fe0 msgr2=0x7fca6c0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca6c066fe0 0x7fca6c0694a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0079a0 tx=0x7fca7c008040 comp rx=0 tx=0).stop 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 -- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca8c0d0030 msgr2=0x7fca8c1565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca8c0d0030 0x7fca8c1565b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fca780af000 tx=0x7fca7801bab0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 -- 172.21.15.113:0/3110660298 shutdown_connections 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca8c154c40 0x7fca8c14a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca8c0d0030 0x7fca8c1565b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca6c066fe0 0x7fca6c0694a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 --2- 172.21.15.113:0/3110660298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca8c0cc2d0 0x7fca8c156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 -- 172.21.15.113:0/3110660298 >> 172.21.15.113:0/3110660298 conn(0x7fca8c06d3e0 msgr2=0x7fca8c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 -- 172.21.15.113:0/3110660298 shutdown_connections 2024-03-04T08:08:10.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:08:10.320+0000 7fca8be00640 1 -- 172.21.15.113:0/3110660298 wait complete. 2024-03-04T08:08:10.337 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2024-03-04T08:08:10.337 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 92 secs before thrashing 2024-03-04T08:08:11.220 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-03-04T08:08:11.224+0000 7f9920e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T08:08:11.220 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-03-04T08:08:11.224+0000 7f9920e00640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T08:08:11.220 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-03-04T08:08:11.224+0000 7f9920e00640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-04T08:08:11.330 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 76 -eq 48 ] 2024-03-04T08:08:11.330 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.76 2024-03-04T08:08:11.330 INFO:tasks.workunit.client.0.smithi113.stdout:v0.76 2024-03-04T08:08:11.331 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.76 2024-03-04T08:08:11.331 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.76 2024-03-04T08:08:29.618 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 32% (54/168) Updating files: 33% (56/168) Updating files: 34% (58/168) Updating files: 35% (59/168) Updating files: 36% (61/168) Updating files: 37% (63/168) Updating files: 38% (64/168) Updating files: 39% (66/168) Updating files: 40% (68/168) Updating files: 41% (69/168) Updating files: 42% (71/168) Updating files: 43% (73/168) Updating files: 44% (74/168) Updating files: 45% (76/168) Updating files: 46% (78/168) Updating files: 47% (79/168) Updating files: 48% (81/168) Updating files: 49% (83/168) Updating files: 50% (84/168) Updating files: 51% (86/168) Updating files: 52% (88/168) Updating files: 53% (90/168) Updating files: 54% (91/168) Updating files: 55% (93/168) Updating files: 56% (95/168) Updating files: 57% (96/168) Updating files: 58% (98/168) Updating files: 59% (100/168) Updating files: 60% (101/168) Updating files: 61% (103/168) Updating files: 62% (105/168) Updating files: 63% (106/168) Updating files: 64% (108/168) Updating files: 65% (110/168) Updating files: 66% (111/168) Updating files: 67% (113/168) Updating files: 68% (115/168) Updating files: 69% (116/168) Updating files: 70% (118/168) Updating files: 71% (120/168) Updating files: 72% (121/168) Updating files: 73% (123/168) Updating files: 73% (124/168) Updating files: 74% (125/168) Updating files: 75% (126/168) Updating files: 76% (128/168) Updating files: 77% (130/168) Updating files: 78% (132/168) Updating files: 79% (133/168) Updating files: 80% (135/168) Updating files: 81% (137/168) Updating files: 82% (138/168) Updating files: 83% (140/168) Updating files: 84% (142/168) Updating files: 85% (143/168) Updating files: 86% (145/168) Updating files: 87% (147/168) Updating files: 88% (148/168) Updating files: 89% (150/168) Updating files: 90% (152/168) Updating files: 91% (153/168) Updating files: 92% (155/168) Updating files: 93% (157/168) Updating files: 94% (158/168) Updating files: 95% (160/168) Updating files: 96% (162/168) Updating files: 97% (163/168) Updating files: 98% (165/168) Updating files: 99% (167/168) Updating files: 100% (168/168) Updating files: 100% (168/168), done. 2024-03-04T08:08:29.782 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 3b990136bfa v0.76 2024-03-04T08:08:29.787 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.76 2024-03-04T08:08:30.503 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 77 -eq 48 ] 2024-03-04T08:08:30.503 INFO:tasks.workunit.client.0.smithi113.stdout:v0.77 2024-03-04T08:08:30.504 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.77 2024-03-04T08:08:30.504 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.77 2024-03-04T08:08:30.504 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.77 2024-03-04T08:08:35.369 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:08:35.373 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:08:46.761 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 26% (80/298) Updating files: 27% (81/298) Updating files: 28% (84/298) Updating files: 29% (87/298) Updating files: 30% (90/298) Updating files: 31% (93/298) Updating files: 32% (96/298) Updating files: 33% (99/298) Updating files: 34% (102/298) Updating files: 35% (105/298) Updating files: 36% (108/298) Updating files: 37% (111/298) Updating files: 38% (114/298) Updating files: 39% (117/298) Updating files: 40% (120/298) Updating files: 41% (123/298) Updating files: 42% (126/298) Updating files: 43% (129/298) Updating files: 44% (132/298) Updating files: 45% (135/298) Updating files: 46% (138/298) Updating files: 47% (141/298) Updating files: 48% (144/298) Updating files: 49% (147/298) Updating files: 50% (149/298) Updating files: 51% (152/298) Updating files: 52% (155/298) Updating files: 53% (158/298) Updating files: 54% (161/298) Updating files: 55% (164/298) Updating files: 56% (167/298) Updating files: 57% (170/298) Updating files: 58% (173/298) Updating files: 59% (176/298) Updating files: 60% (179/298) Updating files: 61% (182/298) Updating files: 62% (185/298) Updating files: 63% (188/298) Updating files: 64% (191/298) Updating files: 65% (194/298) Updating files: 66% (197/298) Updating files: 67% (200/298) Updating files: 68% (203/298) Updating files: 69% (206/298) Updating files: 70% (209/298) Updating files: 70% (211/298) Updating files: 71% (212/298) Updating files: 72% (215/298) Updating files: 73% (218/298) Updating files: 74% (221/298) Updating files: 75% (224/298) Updating files: 76% (227/298) Updating files: 77% (230/298) Updating files: 78% (233/298) Updating files: 79% (236/298) Updating files: 80% (239/298) Updating files: 81% (242/298) Updating files: 82% (245/298) Updating files: 83% (248/298) Updating files: 83% (250/298) Updating files: 84% (251/298) Updating files: 85% (254/298) Updating files: 86% (257/298) Updating files: 87% (260/298) Updating files: 88% (263/298) Updating files: 89% (266/298) Updating files: 90% (269/298) Updating files: 91% (272/298) Updating files: 92% (275/298) Updating files: 93% (278/298) Updating files: 94% (281/298) Updating files: 95% (284/298) Updating files: 96% (287/298) Updating files: 97% (290/298) Updating files: 98% (293/298) Updating files: 99% (296/298) Updating files: 100% (298/298) Updating files: 100% (298/298), done. 2024-03-04T08:08:46.819 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 1bca9c5c412 v0.77 2024-03-04T08:08:46.831 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.77 2024-03-04T08:08:47.105 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 78 -eq 48 ] 2024-03-04T08:08:47.105 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.78 2024-03-04T08:08:47.105 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.78 2024-03-04T08:08:47.106 INFO:tasks.workunit.client.0.smithi113.stdout:v0.78 2024-03-04T08:08:47.106 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.78 2024-03-04T08:09:04.351 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 20% (104/496) Updating files: 21% (105/496) Updating files: 22% (110/496) Updating files: 23% (115/496) Updating files: 24% (120/496) Updating files: 25% (124/496) Updating files: 26% (129/496) Updating files: 27% (134/496) Updating files: 28% (139/496) Updating files: 29% (144/496) Updating files: 30% (149/496) Updating files: 31% (154/496) Updating files: 32% (159/496) Updating files: 33% (164/496) Updating files: 34% (169/496) Updating files: 35% (174/496) Updating files: 35% (175/496) Updating files: 36% (179/496) Updating files: 37% (184/496) Updating files: 38% (189/496) Updating files: 39% (194/496) Updating files: 40% (199/496) Updating files: 41% (204/496) Updating files: 42% (209/496) Updating files: 43% (214/496) Updating files: 44% (219/496) Updating files: 45% (224/496) Updating files: 46% (229/496) Updating files: 47% (234/496) Updating files: 48% (239/496) Updating files: 49% (244/496) Updating files: 50% (248/496) Updating files: 51% (253/496) Updating files: 52% (258/496) Updating files: 53% (263/496) Updating files: 54% (268/496) Updating files: 55% (273/496) Updating files: 56% (278/496) Updating files: 57% (283/496) Updating files: 58% (288/496) Updating files: 59% (293/496) Updating files: 60% (298/496) Updating files: 61% (303/496) Updating files: 62% (308/496) Updating files: 63% (313/496) Updating files: 64% (318/496) Updating files: 65% (323/496) Updating files: 66% (328/496) Updating files: 67% (333/496) Updating files: 68% (338/496) Updating files: 69% (343/496) Updating files: 69% (345/496) Updating files: 69% (346/496) Updating files: 70% (348/496) Updating files: 71% (353/496) Updating files: 72% (358/496) Updating files: 73% (363/496) Updating files: 74% (368/496) Updating files: 75% (372/496) Updating files: 76% (377/496) Updating files: 77% (382/496) Updating files: 78% (387/496) Updating files: 78% (389/496) Updating files: 79% (392/496) Updating files: 80% (397/496) Updating files: 81% (402/496) Updating files: 82% (407/496) Updating files: 83% (412/496) Updating files: 84% (417/496) Updating files: 85% (422/496) Updating files: 86% (427/496) Updating files: 87% (432/496) Updating files: 88% (437/496) Updating files: 89% (442/496) Updating files: 90% (447/496) Updating files: 91% (452/496) Updating files: 92% (457/496) Updating files: 93% (462/496) Updating files: 94% (467/496) Updating files: 94% (469/496) Updating files: 95% (472/496) Updating files: 96% (477/496) Updating files: 97% (482/496) Updating files: 98% (487/496) Updating files: 99% (492/496) Updating files: 99% (494/496) Updating files: 100% (496/496) Updating files: 100% (496/496), done. 2024-03-04T08:09:04.453 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at f6c746c314d 0.78 2024-03-04T08:09:04.463 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.78 2024-03-04T08:09:05.300 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 79 -eq 48 ] 2024-03-04T08:09:05.300 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.79 2024-03-04T08:09:05.300 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.79 2024-03-04T08:09:05.300 INFO:tasks.workunit.client.0.smithi113.stdout:v0.79 2024-03-04T08:09:05.301 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.79 2024-03-04T08:09:05.540 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:09:05.544 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:09:16.373 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 19% (63/329) Updating files: 20% (66/329) Updating files: 21% (70/329) Updating files: 22% (73/329) Updating files: 23% (76/329) Updating files: 24% (79/329) Updating files: 25% (83/329) Updating files: 26% (86/329) Updating files: 27% (89/329) Updating files: 28% (93/329) Updating files: 29% (96/329) Updating files: 30% (99/329) Updating files: 31% (102/329) Updating files: 32% (106/329) Updating files: 33% (109/329) Updating files: 34% (112/329) Updating files: 35% (116/329) Updating files: 36% (119/329) Updating files: 37% (122/329) Updating files: 38% (126/329) Updating files: 39% (129/329) Updating files: 40% (132/329) Updating files: 41% (135/329) Updating files: 42% (139/329) Updating files: 43% (142/329) Updating files: 44% (145/329) Updating files: 45% (149/329) Updating files: 45% (151/329) Updating files: 46% (152/329) Updating files: 46% (153/329) Updating files: 47% (155/329) Updating files: 48% (158/329) Updating files: 49% (162/329) Updating files: 50% (165/329) Updating files: 51% (168/329) Updating files: 52% (172/329) Updating files: 53% (175/329) Updating files: 54% (178/329) Updating files: 55% (181/329) Updating files: 56% (185/329) Updating files: 57% (188/329) Updating files: 58% (191/329) Updating files: 59% (195/329) Updating files: 60% (198/329) Updating files: 61% (201/329) Updating files: 62% (204/329) Updating files: 63% (208/329) Updating files: 64% (211/329) Updating files: 65% (214/329) Updating files: 66% (218/329) Updating files: 67% (221/329) Updating files: 68% (224/329) Updating files: 68% (227/329) Updating files: 69% (228/329) Updating files: 70% (231/329) Updating files: 71% (234/329) Updating files: 72% (237/329) Updating files: 73% (241/329) Updating files: 74% (244/329) Updating files: 75% (247/329) Updating files: 76% (251/329) Updating files: 77% (254/329) Updating files: 78% (257/329) Updating files: 79% (260/329) Updating files: 80% (264/329) Updating files: 81% (267/329) Updating files: 82% (270/329) Updating files: 83% (274/329) Updating files: 84% (277/329) Updating files: 85% (280/329) Updating files: 86% (283/329) Updating files: 87% (287/329) Updating files: 88% (290/329) Updating files: 89% (293/329) Updating files: 90% (297/329) Updating files: 91% (300/329) Updating files: 92% (303/329) Updating files: 93% (306/329) Updating files: 93% (309/329) Updating files: 94% (310/329) Updating files: 95% (313/329) Updating files: 96% (316/329) Updating files: 97% (320/329) Updating files: 98% (323/329) Updating files: 99% (326/329) Updating files: 99% (327/329) Updating files: 100% (329/329) Updating files: 100% (329/329), done. 2024-03-04T08:09:16.464 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 4c2d73a5095 0.79 2024-03-04T08:09:16.472 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.79 2024-03-04T08:09:16.539 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 80 -eq 48 ] 2024-03-04T08:09:16.539 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.80 2024-03-04T08:09:16.539 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.80 2024-03-04T08:09:16.539 INFO:tasks.workunit.client.0.smithi113.stdout:v0.80 2024-03-04T08:09:16.540 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.80 2024-03-04T08:09:18.178 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 65% (115/176) Updating files: 66% (117/176) Updating files: 67% (118/176) Updating files: 68% (120/176) Updating files: 69% (122/176) Updating files: 70% (124/176) Updating files: 71% (125/176) Updating files: 72% (127/176) Updating files: 73% (129/176) Updating files: 74% (131/176) Updating files: 75% (132/176) Updating files: 76% (134/176) Updating files: 77% (136/176) Updating files: 78% (138/176) Updating files: 79% (140/176) Updating files: 80% (141/176) Updating files: 81% (143/176) Updating files: 82% (145/176) Updating files: 83% (147/176) Updating files: 84% (148/176) Updating files: 85% (150/176) Updating files: 86% (152/176) Updating files: 87% (154/176) Updating files: 88% (155/176) Updating files: 89% (157/176) Updating files: 90% (159/176) Updating files: 91% (161/176) Updating files: 92% (162/176) Updating files: 93% (164/176) Updating files: 94% (166/176) Updating files: 95% (168/176) Updating files: 96% (169/176) Updating files: 97% (171/176) Updating files: 98% (173/176) Updating files: 99% (175/176) Updating files: 100% (176/176) Updating files: 100% (176/176), done. 2024-03-04T08:09:18.488 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at b78644e7dee 0.80 2024-03-04T08:09:18.495 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.80 2024-03-04T08:09:18.553 INFO:tasks.workunit.client.0.smithi113.stdout:v0.81 2024-03-04T08:09:18.553 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 81 -eq 48 ] 2024-03-04T08:09:18.553 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.81 2024-03-04T08:09:18.553 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.81 2024-03-04T08:09:18.553 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.81 2024-03-04T08:09:22.786 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 20% (63/309) Updating files: 21% (65/309) Updating files: 22% (68/309) Updating files: 23% (72/309) Updating files: 24% (75/309) Updating files: 25% (78/309) Updating files: 26% (81/309) Updating files: 27% (84/309) Updating files: 28% (87/309) Updating files: 29% (90/309) Updating files: 30% (93/309) Updating files: 31% (96/309) Updating files: 32% (99/309) Updating files: 33% (102/309) Updating files: 34% (106/309) Updating files: 35% (109/309) Updating files: 36% (112/309) Updating files: 37% (115/309) Updating files: 38% (118/309) Updating files: 39% (121/309) Updating files: 40% (124/309) Updating files: 41% (127/309) Updating files: 42% (130/309) Updating files: 43% (133/309) Updating files: 44% (136/309) Updating files: 45% (140/309) Updating files: 46% (143/309) Updating files: 47% (146/309) Updating files: 48% (149/309) Updating files: 49% (152/309) Updating files: 50% (155/309) Updating files: 51% (158/309) Updating files: 52% (161/309) Updating files: 53% (164/309) Updating files: 54% (167/309) Updating files: 55% (170/309) Updating files: 56% (174/309) Updating files: 57% (177/309) Updating files: 58% (180/309) Updating files: 59% (183/309) Updating files: 60% (186/309) Updating files: 61% (189/309) Updating files: 62% (192/309) Updating files: 63% (195/309) Updating files: 64% (198/309) Updating files: 65% (201/309) Updating files: 66% (204/309) Updating files: 67% (208/309) Updating files: 68% (211/309) Updating files: 69% (214/309) Updating files: 70% (217/309) Updating files: 71% (220/309) Updating files: 72% (223/309) Updating files: 73% (226/309) Updating files: 74% (229/309) Updating files: 75% (232/309) Updating files: 76% (235/309) Updating files: 77% (238/309) Updating files: 78% (242/309) Updating files: 79% (245/309) Updating files: 80% (248/309) Updating files: 81% (251/309) Updating files: 82% (254/309) Updating files: 83% (257/309) Updating files: 84% (260/309) Updating files: 85% (263/309) Updating files: 86% (266/309) Updating files: 87% (269/309) Updating files: 88% (272/309) Updating files: 89% (276/309) Updating files: 90% (279/309) Updating files: 91% (282/309) Updating files: 92% (285/309) Updating files: 93% (288/309) Updating files: 94% (291/309) Updating files: 95% (294/309) Updating files: 96% (297/309) Updating files: 97% (300/309) Updating files: 98% (303/309) Updating files: 99% (306/309) Updating files: 100% (309/309) Updating files: 100% (309/309), done. 2024-03-04T08:09:22.842 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 8de9501df27 0.81 2024-03-04T08:09:22.857 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.81 2024-03-04T08:09:23.163 INFO:tasks.workunit.client.0.smithi113.stdout:v0.82 2024-03-04T08:09:23.163 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 82 -eq 48 ] 2024-03-04T08:09:23.163 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.82 2024-03-04T08:09:23.163 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.82 2024-03-04T08:09:23.163 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.82 2024-03-04T08:09:25.582 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 34% (70/204) Updating files: 35% (72/204) Updating files: 36% (74/204) Updating files: 37% (76/204) Updating files: 38% (78/204) Updating files: 39% (80/204) Updating files: 40% (82/204) Updating files: 41% (84/204) Updating files: 42% (86/204) Updating files: 43% (88/204) Updating files: 44% (90/204) Updating files: 45% (92/204) Updating files: 46% (94/204) Updating files: 47% (96/204) Updating files: 48% (98/204) Updating files: 49% (100/204) Updating files: 50% (102/204) Updating files: 51% (105/204) Updating files: 52% (107/204) Updating files: 53% (109/204) Updating files: 54% (111/204) Updating files: 55% (113/204) Updating files: 56% (115/204) Updating files: 57% (117/204) Updating files: 58% (119/204) Updating files: 59% (121/204) Updating files: 60% (123/204) Updating files: 61% (125/204) Updating files: 62% (127/204) Updating files: 63% (129/204) Updating files: 64% (131/204) Updating files: 65% (133/204) Updating files: 66% (135/204) Updating files: 67% (137/204) Updating files: 68% (139/204) Updating files: 69% (141/204) Updating files: 70% (143/204) Updating files: 71% (145/204) Updating files: 72% (147/204) Updating files: 73% (149/204) Updating files: 74% (151/204) Updating files: 75% (153/204) Updating files: 76% (156/204) Updating files: 77% (158/204) Updating files: 78% (160/204) Updating files: 79% (162/204) Updating files: 80% (164/204) Updating files: 81% (166/204) Updating files: 82% (168/204) Updating files: 83% (170/204) Updating files: 84% (172/204) Updating files: 85% (174/204) Updating files: 86% (176/204) Updating files: 87% (178/204) Updating files: 88% (180/204) Updating files: 89% (182/204) Updating files: 90% (184/204) Updating files: 91% (186/204) Updating files: 92% (188/204) Updating files: 93% (190/204) Updating files: 94% (192/204) Updating files: 95% (194/204) Updating files: 96% (196/204) Updating files: 97% (198/204) Updating files: 98% (200/204) Updating files: 99% (202/204) Updating files: 100% (204/204) Updating files: 100% (204/204), done. 2024-03-04T08:09:25.653 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 14085f42ddd 0.82 2024-03-04T08:09:25.664 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.82 2024-03-04T08:09:25.867 INFO:tasks.workunit.client.0.smithi113.stdout:v0.83 2024-03-04T08:09:25.867 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 83 -eq 48 ] 2024-03-04T08:09:25.867 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.83 2024-03-04T08:09:25.867 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.83 2024-03-04T08:09:25.867 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.83 2024-03-04T08:09:30.764 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 54% (119/218) Updating files: 55% (120/218) Updating files: 56% (123/218) Updating files: 57% (125/218) Updating files: 58% (127/218) Updating files: 59% (129/218) Updating files: 60% (131/218) Updating files: 61% (133/218) Updating files: 62% (136/218) Updating files: 62% (137/218) Updating files: 63% (138/218) Updating files: 64% (140/218) Updating files: 65% (142/218) Updating files: 66% (144/218) Updating files: 67% (147/218) Updating files: 68% (149/218) Updating files: 69% (151/218) Updating files: 70% (153/218) Updating files: 71% (155/218) Updating files: 72% (157/218) Updating files: 73% (160/218) Updating files: 74% (162/218) Updating files: 75% (164/218) Updating files: 76% (166/218) Updating files: 77% (168/218) Updating files: 78% (171/218) Updating files: 79% (173/218) Updating files: 80% (175/218) Updating files: 81% (177/218) Updating files: 82% (179/218) Updating files: 83% (181/218) Updating files: 84% (184/218) Updating files: 85% (186/218) Updating files: 86% (188/218) Updating files: 87% (190/218) Updating files: 88% (192/218) Updating files: 89% (195/218) Updating files: 90% (197/218) Updating files: 91% (199/218) Updating files: 92% (201/218) Updating files: 93% (203/218) Updating files: 94% (205/218) Updating files: 95% (208/218) Updating files: 96% (210/218) Updating files: 97% (212/218) Updating files: 98% (214/218) Updating files: 99% (216/218) Updating files: 100% (218/218) Updating files: 100% (218/218), done. 2024-03-04T08:09:30.829 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 78ff1f0a5df 0.83 2024-03-04T08:09:30.838 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.83 2024-03-04T08:09:30.967 INFO:tasks.workunit.client.0.smithi113.stdout:v0.84 2024-03-04T08:09:30.967 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 84 -eq 48 ] 2024-03-04T08:09:30.967 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.84 2024-03-04T08:09:30.967 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.84 2024-03-04T08:09:30.967 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.84 2024-03-04T08:09:35.610 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:09:35.614 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:09:42.339 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:09:42.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.498+0000 7efc0fe00640 1 Processor -- start 2024-03-04T08:09:42.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.498+0000 7efc0fe00640 1 -- start start 2024-03-04T08:09:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.498+0000 7efc0fe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efc10158f10 0x7efc1015f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.498+0000 7efc0fe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1006dc00 con 0x7efc1006c170 2024-03-04T08:09:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.498+0000 7efc0fe00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc101592f0 con 0x7efc1006faa0 2024-03-04T08:09:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.498+0000 7efc0fe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc1006e150 con 0x7efc10158f10 2024-03-04T08:09:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.498+0000 7efc0f400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efc1006c170 0x7efc101694d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37962/0 (socket says 172.21.15.113:37962) 2024-03-04T08:09:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.498+0000 7efc0f400640 1 -- 172.21.15.113:0/202454038 learned_addr learned my addr 172.21.15.113:0/202454038 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:09:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.498+0000 7efc0ea00640 1 --2- 172.21.15.113:0/202454038 >> v2:172.21.15.136:3300/0 conn(0x7efc10158f10 0x7efc1015f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:42.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756806994 0 0) 0x7efc1006dc00 con 0x7efc1006c170 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbf4003530 con 0x7efc1006c170 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540041370 0 0) 0x7efc101592f0 con 0x7efc1006faa0 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc1006dc00 con 0x7efc1006faa0 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 13166895 0 0) 0x7efbf4003530 con 0x7efc1006c170 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 >> v2:172.21.15.136:3300/0 conn(0x7efc10158f10 msgr2=0x7efc1015f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 --2- 172.21.15.113:0/202454038 >> v2:172.21.15.136:3300/0 conn(0x7efc10158f10 0x7efc1015f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 >> v1:172.21.15.136:6789/0 conn(0x7efc1006faa0 legacy=0x7efc1006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1006de70 con 0x7efc1006c170 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7efc08001a00 con 0x7efc1006c170 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 >> v1:172.21.15.113:6789/0 conn(0x7efc1006c170 legacy=0x7efc101694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efbf4003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbf4004810 0x7efbf4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbf40051d0 0x7efbf400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1006e150 con 0x7efbf4003ba0 2024-03-04T08:09:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf400bff0 con 0x7efbf40051d0 2024-03-04T08:09:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf400c1f0 con 0x7efbf4004810 2024-03-04T08:09:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0ea00640 1 --2- 172.21.15.113:0/202454038 >> v2:172.21.15.136:3300/0 conn(0x7efc10158f10 0x7efc1015f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:09:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0ea00640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efbf4003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0f400640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbf40051d0 0x7efbf400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:42.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc06000640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbf4004810 0x7efbf4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:42.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0ea00640 1 -- 172.21.15.113:0/202454038 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbf4004810 msgr2=0x7efbf4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:42.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0ea00640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efbf4004810 0x7efbf4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0ea00640 1 -- 172.21.15.113:0/202454038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbf40051d0 msgr2=0x7efbf400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:42.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0ea00640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efbf40051d0 0x7efbf400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0ea00640 1 -- 172.21.15.113:0/202454038 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf4003530 con 0x7efbf4003ba0 2024-03-04T08:09:42.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.502+0000 7efc0f400640 1 -- 172.21.15.113:0/202454038 reap_dead start 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0ea00640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efbf4003fa0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7efc00009c90 tx=0x7efc0001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934b51611e6c8bab server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc00009860 con 0x7efbf4003ba0 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc00004020 con 0x7efbf4003ba0 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0e000640 1 -- 172.21.15.113:0/202454038 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc00004450 con 0x7efbf4003ba0 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0fe00640 1 -- 172.21.15.113:0/202454038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 msgr2=0x7efbf4003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0fe00640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efbf4003fa0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7efc00009c90 tx=0x7efc0001cbf0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0fe00640 1 -- 172.21.15.113:0/202454038 shutdown_connections 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0fe00640 1 --2- 172.21.15.113:0/202454038 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efbf4003fa0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0fe00640 1 -- 172.21.15.113:0/202454038 >> 172.21.15.113:0/202454038 conn(0x7efc10072f40 msgr2=0x7efc100706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0fe00640 1 -- 172.21.15.113:0/202454038 shutdown_connections 2024-03-04T08:09:42.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0fe00640 1 -- 172.21.15.113:0/202454038 wait complete. 2024-03-04T08:09:42.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0fe00640 1 Processor -- start 2024-03-04T08:09:42.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.506+0000 7efc0fe00640 1 -- start start 2024-03-04T08:09:42.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efc10155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc100c5a20 0x7efc101600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc101f54b0 0x7efc101605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1015f240 con 0x7efbf4003ba0 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1015ef60 con 0x7efc101f54b0 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1015f760 con 0x7efc100c5a20 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efc10155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efc10155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57954/0 (socket says 172.21.15.113:57954) 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0ea00640 1 -- 172.21.15.113:0/1696096149 learned_addr learned my addr 172.21.15.113:0/1696096149 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc06000640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc100c5a20 0x7efc101600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0f400640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc101f54b0 0x7efc101605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0ea00640 1 -- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc100c5a20 msgr2=0x7efc101600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0ea00640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc100c5a20 0x7efc101600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0ea00640 1 -- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc101f54b0 msgr2=0x7efc101605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:42.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0ea00640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc101f54b0 0x7efc101605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0ea00640 1 -- 172.21.15.113:0/1696096149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1015d2a0 con 0x7efbf4003ba0 2024-03-04T08:09:42.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc06000640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc100c5a20 0x7efc101600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:09:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0ea00640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efc10155ae0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7efc0001c660 tx=0x7efbf4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc07400640 1 -- 172.21.15.113:0/1696096149 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc000a58b0 con 0x7efbf4003ba0 2024-03-04T08:09:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc07400640 1 -- 172.21.15.113:0/1696096149 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc000bc070 con 0x7efbf4003ba0 2024-03-04T08:09:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc07400640 1 -- 172.21.15.113:0/1696096149 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc000a5c90 con 0x7efbf4003ba0 2024-03-04T08:09:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0fe00640 1 -- 172.21.15.113:0/1696096149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc10161f00 con 0x7efbf4003ba0 2024-03-04T08:09:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0fe00640 1 -- 172.21.15.113:0/1696096149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc10162480 con 0x7efbf4003ba0 2024-03-04T08:09:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.510+0000 7efc0fe00640 1 -- 172.21.15.113:0/1696096149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc10075a80 con 0x7efbf4003ba0 2024-03-04T08:09:42.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.514+0000 7efc07400640 1 -- 172.21.15.113:0/1696096149 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc00006040 con 0x7efbf4003ba0 2024-03-04T08:09:42.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.514+0000 7efc07400640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efbec066f90 0x7efbec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:42.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.514+0000 7efc07400640 1 -- 172.21.15.113:0/1696096149 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7efc00120180 con 0x7efbf4003ba0 2024-03-04T08:09:42.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.514+0000 7efc06000640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efbec066f90 0x7efbec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:42.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.514+0000 7efc06000640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efbec066f90 0x7efbec069450 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7efbf8000f70 tx=0x7efbf8000ee0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:42.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.518+0000 7efc07400640 1 -- 172.21.15.113:0/1696096149 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efc00016900 con 0x7efbf4003ba0 2024-03-04T08:09:42.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.814+0000 7efc0fe00640 1 -- 172.21.15.113:0/1696096149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc1015fd10 con 0x7efbf4003ba0 2024-03-04T08:09:42.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.814+0000 7efc07400640 1 -- 172.21.15.113:0/1696096149 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+6621 (secure 0 0 0) 0x7efc000f0a10 con 0x7efbf4003ba0 2024-03-04T08:09:42.811 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:09:42.811 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:08:04.266596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269,7119]},"id":1}]} 2024-03-04T08:09:42.812 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 132 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.814+0000 7efc0fe00640 1 -- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efbec066f90 msgr2=0x7efbec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.814+0000 7efc0fe00640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efbec066f90 0x7efbec069450 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7efbf8000f70 tx=0x7efbf8000ee0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.814+0000 7efc0fe00640 1 -- 172.21.15.113:0/1696096149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 msgr2=0x7efc10155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.814+0000 7efc0fe00640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efc10155ae0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7efc0001c660 tx=0x7efbf4005840 comp rx=0 tx=0).stop 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.818+0000 7efc0fe00640 1 -- 172.21.15.113:0/1696096149 shutdown_connections 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.818+0000 7efc0fe00640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc101f54b0 0x7efc101605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.818+0000 7efc0fe00640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc100c5a20 0x7efc101600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.818+0000 7efc0fe00640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efbec066f90 0x7efbec069450 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.818+0000 7efc0fe00640 1 --2- 172.21.15.113:0/1696096149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf4003ba0 0x7efc10155ae0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.818+0000 7efc0fe00640 1 -- 172.21.15.113:0/1696096149 >> 172.21.15.113:0/1696096149 conn(0x7efc10072f40 msgr2=0x7efc1015d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.818+0000 7efc0fe00640 1 -- 172.21.15.113:0/1696096149 shutdown_connections 2024-03-04T08:09:42.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:42.818+0000 7efc0fe00640 1 -- 172.21.15.113:0/1696096149 wait complete. 2024-03-04T08:09:42.841 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=1) 2024-03-04T08:09:42.841 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-04T08:09:42.841 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:09:42.855 INFO:tasks.ceph.mds.c:Stopped 2024-03-04T08:09:42.855 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:09:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.074+0000 7f2f40a00640 1 Processor -- start 2024-03-04T08:09:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f40a00640 1 -- start start 2024-03-04T08:09:43.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f40a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2f3c15af20 0x7f2f3c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:43.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f40a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f3c157230 0x7f2f3c06c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:43.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f40a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c0c9c30 con 0x7f2f3c151030 2024-03-04T08:09:43.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f40a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3c15aa90 con 0x7f2f3c15af20 2024-03-04T08:09:43.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f40a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1577a0 con 0x7f2f3c157230 2024-03-04T08:09:43.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f3aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2f3c15af20 0x7f2f3c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:43.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f3b400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2f3c151030 0x7f2f3c0c6b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37972/0 (socket says 172.21.15.113:37972) 2024-03-04T08:09:43.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f3b400640 1 -- 172.21.15.113:0/1377828802 learned_addr learned my addr 172.21.15.113:0/1377828802 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:09:43.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f3be00640 1 --2- 172.21.15.113:0/1377828802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f3c157230 0x7f2f3c06c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:43.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f3aa00640 1 -- 172.21.15.113:0/1377828802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f3c157230 msgr2=0x7f2f3c06c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:43.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f3aa00640 1 --2- 172.21.15.113:0/1377828802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f3c157230 0x7f2f3c06c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:43.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f3aa00640 1 -- 172.21.15.113:0/1377828802 >> v1:172.21.15.113:6789/0 conn(0x7f2f3c151030 legacy=0x7f2f3c0c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:43.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.078+0000 7f2f3aa00640 1 -- 172.21.15.113:0/1377828802 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c157a80 con 0x7f2f3c15af20 2024-03-04T08:09:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f3aa00640 1 --2- 172.21.15.113:0/1377828802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c15af20 0x7f2f3c156cf0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2f280097c0 tx=0x7f2f2801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85b9eab4cb0bc6f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f3a000640 1 -- 172.21.15.113:0/1377828802 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f280a7070 con 0x7f2f3c15af20 2024-03-04T08:09:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f3a000640 1 -- 172.21.15.113:0/1377828802 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f280b0070 con 0x7f2f3c15af20 2024-03-04T08:09:43.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f3a000640 1 -- 172.21.15.113:0/1377828802 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f280ab890 con 0x7f2f3c15af20 2024-03-04T08:09:43.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f40a00640 1 -- 172.21.15.113:0/1377828802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c15af20 msgr2=0x7f2f3c156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:43.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f40a00640 1 --2- 172.21.15.113:0/1377828802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c15af20 0x7f2f3c156cf0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f2f280097c0 tx=0x7f2f2801c470 comp rx=0 tx=0).stop 2024-03-04T08:09:43.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f40a00640 1 -- 172.21.15.113:0/1377828802 shutdown_connections 2024-03-04T08:09:43.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f40a00640 1 --2- 172.21.15.113:0/1377828802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f3c157230 0x7f2f3c06c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:43.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f40a00640 1 --2- 172.21.15.113:0/1377828802 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c15af20 0x7f2f3c156cf0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:43.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f40a00640 1 -- 172.21.15.113:0/1377828802 >> 172.21.15.113:0/1377828802 conn(0x7f2f3c06e9a0 msgr2=0x7f2f3c06edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:43.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f40a00640 1 -- 172.21.15.113:0/1377828802 shutdown_connections 2024-03-04T08:09:43.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.082+0000 7f2f40a00640 1 -- 172.21.15.113:0/1377828802 wait complete. 2024-03-04T08:09:43.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f40a00640 1 Processor -- start 2024-03-04T08:09:43.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f40a00640 1 -- start start 2024-03-04T08:09:43.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f40a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f3c151030 0x7f2f3c148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:43.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f40a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c157230 0x7f2f3c14e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f40a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f3c15af20 0x7f2f3c14ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f40a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1637e0 con 0x7f2f3c157230 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f40a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c163660 con 0x7f2f3c151030 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f40a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c163960 con 0x7f2f3c15af20 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c157230 0x7f2f3c14e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c157230 0x7f2f3c14e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57962/0 (socket says 172.21.15.113:57962) 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3aa00640 1 -- 172.21.15.113:0/614893570 learned_addr learned my addr 172.21.15.113:0/614893570 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3be00640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f3c15af20 0x7f2f3c14ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3b400640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f3c151030 0x7f2f3c148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3aa00640 1 -- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f3c15af20 msgr2=0x7f2f3c14ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3aa00640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f3c15af20 0x7f2f3c14ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3aa00640 1 -- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f3c151030 msgr2=0x7f2f3c148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:43.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3aa00640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f3c151030 0x7f2f3c148360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:43.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3aa00640 1 -- 172.21.15.113:0/614893570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c1488a0 con 0x7f2f3c157230 2024-03-04T08:09:43.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f3aa00640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c157230 0x7f2f3c14e530 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2f280ad040 tx=0x7f2f2801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:43.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f33400640 1 -- 172.21.15.113:0/614893570 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f280bf610 con 0x7f2f3c157230 2024-03-04T08:09:43.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f33400640 1 -- 172.21.15.113:0/614893570 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f280bf7d0 con 0x7f2f3c157230 2024-03-04T08:09:43.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f40a00640 1 -- 172.21.15.113:0/614893570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c155c80 con 0x7f2f3c157230 2024-03-04T08:09:43.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f33400640 1 -- 172.21.15.113:0/614893570 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f280bfbc0 con 0x7f2f3c157230 2024-03-04T08:09:43.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.086+0000 7f2f40a00640 1 -- 172.21.15.113:0/614893570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c156200 con 0x7f2f3c157230 2024-03-04T08:09:43.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.090+0000 7f2f40a00640 1 -- 172.21.15.113:0/614893570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c075a80 con 0x7f2f3c157230 2024-03-04T08:09:43.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.090+0000 7f2f33400640 1 -- 172.21.15.113:0/614893570 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f280bd020 con 0x7f2f3c157230 2024-03-04T08:09:43.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.090+0000 7f2f33400640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2f1c066f90 0x7f2f1c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:43.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.090+0000 7f2f33400640 1 -- 172.21.15.113:0/614893570 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f2f281201c0 con 0x7f2f3c157230 2024-03-04T08:09:43.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.090+0000 7f2f3b400640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2f1c066f90 0x7f2f1c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:43.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.094+0000 7f2f3b400640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2f1c066f90 0x7f2f1c069450 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c000f70 tx=0x7f2f2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:43.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.098+0000 7f2f33400640 1 -- 172.21.15.113:0/614893570 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2f28016090 con 0x7f2f3c157230 2024-03-04T08:09:43.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.350+0000 7f2f40a00640 1 -- 172.21.15.113:0/614893570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f3c15a220 con 0x7f2f3c157230 2024-03-04T08:09:43.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.350+0000 7f2f33400640 1 -- 172.21.15.113:0/614893570 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f2f280f09c0 con 0x7f2f3c157230 2024-03-04T08:09:43.350 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:09:43.350 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:08:04.266596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269,7119]},"id":1}]} 2024-03-04T08:09:43.351 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 132 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 -- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2f1c066f90 msgr2=0x7f2f1c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2f1c066f90 0x7f2f1c069450 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c000f70 tx=0x7f2f2c009040 comp rx=0 tx=0).stop 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 -- 172.21.15.113:0/614893570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c157230 msgr2=0x7f2f3c14e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c157230 0x7f2f3c14e530 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f2f280ad040 tx=0x7f2f2801acb0 comp rx=0 tx=0).stop 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 -- 172.21.15.113:0/614893570 shutdown_connections 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2f3c15af20 0x7f2f3c14ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2f1c066f90 0x7f2f1c069450 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2f3c157230 0x7f2f3c14e530 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 --2- 172.21.15.113:0/614893570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f3c151030 0x7f2f3c148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 -- 172.21.15.113:0/614893570 >> 172.21.15.113:0/614893570 conn(0x7f2f3c06e9a0 msgr2=0x7f2f3c15f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 -- 172.21.15.113:0/614893570 shutdown_connections 2024-03-04T08:09:43.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:43.354+0000 7f2f40a00640 1 -- 172.21.15.113:0/614893570 wait complete. 2024-03-04T08:09:43.378 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:09:45.369 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 14% (53/356) Updating files: 15% (54/356) Updating files: 16% (57/356) Updating files: 17% (61/356) Updating files: 18% (65/356) Updating files: 19% (68/356) Updating files: 20% (72/356) Updating files: 21% (75/356) Updating files: 22% (79/356) Updating files: 23% (82/356) Updating files: 24% (86/356) Updating files: 25% (89/356) Updating files: 26% (93/356) Updating files: 27% (97/356) Updating files: 28% (100/356) Updating files: 29% (104/356) Updating files: 30% (107/356) Updating files: 31% (111/356) Updating files: 32% (114/356) Updating files: 33% (118/356) Updating files: 34% (122/356) Updating files: 35% (125/356) Updating files: 36% (129/356) Updating files: 37% (132/356) Updating files: 37% (134/356) Updating files: 38% (136/356) Updating files: 39% (139/356) Updating files: 40% (143/356) Updating files: 41% (146/356) Updating files: 42% (150/356) Updating files: 43% (154/356) Updating files: 44% (157/356) Updating files: 45% (161/356) Updating files: 46% (164/356) Updating files: 47% (168/356) Updating files: 48% (171/356) Updating files: 49% (175/356) Updating files: 50% (178/356) Updating files: 50% (181/356) Updating files: 51% (182/356) Updating files: 52% (186/356) Updating files: 53% (189/356) Updating files: 54% (193/356) Updating files: 55% (196/356) Updating files: 56% (200/356) Updating files: 57% (203/356) Updating files: 58% (207/356) Updating files: 59% (211/356) Updating files: 60% (214/356) Updating files: 61% (218/356) Updating files: 62% (221/356) Updating files: 63% (225/356) Updating files: 64% (228/356) Updating files: 65% (232/356) Updating files: 66% (235/356) Updating files: 67% (239/356) Updating files: 68% (243/356) Updating files: 69% (246/356) Updating files: 70% (250/356) Updating files: 71% (253/356) Updating files: 72% (257/356) Updating files: 73% (260/356) Updating files: 74% (264/356) Updating files: 75% (267/356) Updating files: 76% (271/356) Updating files: 77% (275/356) Updating files: 78% (278/356) Updating files: 79% (282/356) Updating files: 80% (285/356) Updating files: 81% (289/356) Updating files: 82% (292/356) Updating files: 83% (296/356) Updating files: 84% (300/356) Updating files: 85% (303/356) Updating files: 86% (307/356) Updating files: 87% (310/356) Updating files: 88% (314/356) Updating files: 89% (317/356) Updating files: 90% (321/356) Updating files: 91% (324/356) Updating files: 92% (328/356) Updating files: 93% (332/356) Updating files: 94% (335/356) Updating files: 95% (339/356) Updating files: 96% (342/356) Updating files: 97% (346/356) Updating files: 98% (349/356) Updating files: 98% (352/356) Updating files: 99% (353/356) Updating files: 100% (356/356) Updating files: 100% (356/356), done. 2024-03-04T08:09:45.379 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:09:45.424 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 8336f81c5c7 0.84 2024-03-04T08:09:45.442 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.84 2024-03-04T08:09:45.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3323e00640 1 Processor -- start 2024-03-04T08:09:45.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3323e00640 1 -- start start 2024-03-04T08:09:45.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3323e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3324152a90 0x7f33240730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:45.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3323e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 0x7f332406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:45.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3323e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332406d690 con 0x7f332415bce0 2024-03-04T08:09:45.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3323e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3324152e70 con 0x7f3324152a90 2024-03-04T08:09:45.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3323e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332406c320 con 0x7f332406cd10 2024-03-04T08:09:45.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3322a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3324152a90 0x7f33240730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:45.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3323400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f332415bce0 0x7f332415c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37980/0 (socket says 172.21.15.113:37980) 2024-03-04T08:09:45.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3323400640 1 -- 172.21.15.113:0/3734020678 learned_addr learned my addr 172.21.15.113:0/3734020678 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:09:45.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3322000640 1 --2- 172.21.15.113:0/3734020678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 0x7f332406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:45.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3322a00640 1 -- 172.21.15.113:0/3734020678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 msgr2=0x7f332406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:45.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3322a00640 1 --2- 172.21.15.113:0/3734020678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 0x7f332406d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3322a00640 1 -- 172.21.15.113:0/3734020678 >> v1:172.21.15.113:6789/0 conn(0x7f332415bce0 legacy=0x7f332415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:45.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3322a00640 1 -- 172.21.15.113:0/3734020678 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332406f1b0 con 0x7f3324152a90 2024-03-04T08:09:45.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.590+0000 7f3321600640 1 -- 172.21.15.113:0/3734020678 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867261793 0 0) 0x7f332406d690 con 0x7f332415bce0 2024-03-04T08:09:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3322a00640 1 --2- 172.21.15.113:0/3734020678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3324152a90 0x7f33240730d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f33100097c0 tx=0x7f331001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee53263770d52db server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3321600640 1 -- 172.21.15.113:0/3734020678 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33100b0070 con 0x7f3324152a90 2024-03-04T08:09:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3321600640 1 -- 172.21.15.113:0/3734020678 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33100ab600 con 0x7f3324152a90 2024-03-04T08:09:45.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3321600640 1 -- 172.21.15.113:0/3734020678 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33100ab9d0 con 0x7f3324152a90 2024-03-04T08:09:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3323e00640 1 -- 172.21.15.113:0/3734020678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3324152a90 msgr2=0x7f33240730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3323e00640 1 --2- 172.21.15.113:0/3734020678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3324152a90 0x7f33240730d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f33100097c0 tx=0x7f331001c4a0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3323e00640 1 -- 172.21.15.113:0/3734020678 shutdown_connections 2024-03-04T08:09:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3323e00640 1 --2- 172.21.15.113:0/3734020678 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 0x7f332406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3323e00640 1 --2- 172.21.15.113:0/3734020678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3324152a90 0x7f33240730d0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3323e00640 1 -- 172.21.15.113:0/3734020678 >> 172.21.15.113:0/3734020678 conn(0x7f332406e2a0 msgr2=0x7f332406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:45.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3323e00640 1 -- 172.21.15.113:0/3734020678 shutdown_connections 2024-03-04T08:09:45.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.594+0000 7f3323e00640 1 -- 172.21.15.113:0/3734020678 wait complete. 2024-03-04T08:09:45.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323e00640 1 Processor -- start 2024-03-04T08:09:45.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323e00640 1 -- start start 2024-03-04T08:09:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 0x7f3324153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3324152a90 0x7f3324150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332415bce0 0x7f33241f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332415c860 con 0x7f332415bce0 2024-03-04T08:09:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3324071330 con 0x7f3324152a90 2024-03-04T08:09:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332415c9e0 con 0x7f332406cd10 2024-03-04T08:09:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332415bce0 0x7f33241f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332415bce0 0x7f33241f69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57984/0 (socket says 172.21.15.113:57984) 2024-03-04T08:09:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3323400640 1 -- 172.21.15.113:0/464480217 learned_addr learned my addr 172.21.15.113:0/464480217 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3322a00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 0x7f3324153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:45.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.598+0000 7f3322000640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3324152a90 0x7f3324150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:45.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3323400640 1 -- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 msgr2=0x7f3324153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:45.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3323400640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 0x7f3324153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3323400640 1 -- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3324152a90 msgr2=0x7f3324150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:45.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3323400640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3324152a90 0x7f3324150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3323400640 1 -- 172.21.15.113:0/464480217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3324147940 con 0x7f332415bce0 2024-03-04T08:09:45.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3322000640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3324152a90 0x7f3324150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:09:45.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3322a00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 0x7f3324153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:09:45.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3323400640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332415bce0 0x7f33241f69e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f331400c410 tx=0x7f331400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:45.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f331aa00640 1 -- 172.21.15.113:0/464480217 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33140074e0 con 0x7f332415bce0 2024-03-04T08:09:45.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f331aa00640 1 -- 172.21.15.113:0/464480217 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33140047f0 con 0x7f332415bce0 2024-03-04T08:09:45.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3323e00640 1 -- 172.21.15.113:0/464480217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3324147be0 con 0x7f332415bce0 2024-03-04T08:09:45.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f331aa00640 1 -- 172.21.15.113:0/464480217 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314007680 con 0x7f332415bce0 2024-03-04T08:09:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3323e00640 1 -- 172.21.15.113:0/464480217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33240ca320 con 0x7f332415bce0 2024-03-04T08:09:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f3323e00640 1 -- 172.21.15.113:0/464480217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3324075a80 con 0x7f332415bce0 2024-03-04T08:09:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.602+0000 7f331aa00640 1 -- 172.21.15.113:0/464480217 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3314011070 con 0x7f332415bce0 2024-03-04T08:09:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.606+0000 7f331aa00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3304066f60 0x7f3304069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.606+0000 7f331aa00640 1 -- 172.21.15.113:0/464480217 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f33140852a0 con 0x7f332415bce0 2024-03-04T08:09:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.606+0000 7f3322a00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3304066f60 0x7f3304069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:45.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.606+0000 7f3322a00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3304066f60 0x7f3304069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f33100ad000 tx=0x7f33100a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:45.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.610+0000 7f331aa00640 1 -- 172.21.15.113:0/464480217 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3314055bc0 con 0x7f332415bce0 2024-03-04T08:09:45.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.878+0000 7f3323e00640 1 -- 172.21.15.113:0/464480217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33241560a0 con 0x7f332415bce0 2024-03-04T08:09:45.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.878+0000 7f331aa00640 1 -- 172.21.15.113:0/464480217 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f33140554c0 con 0x7f332415bce0 2024-03-04T08:09:45.877 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:09:45.877 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:08:04.266596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269,7119]},"id":1}]} 2024-03-04T08:09:45.877 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 132 2024-03-04T08:09:45.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 -- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3304066f60 msgr2=0x7f3304069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:45.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3304066f60 0x7f3304069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f33100ad000 tx=0x7f33100a6000 comp rx=0 tx=0).stop 2024-03-04T08:09:45.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 -- 172.21.15.113:0/464480217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332415bce0 msgr2=0x7f33241f69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:45.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332415bce0 0x7f33241f69e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f331400c410 tx=0x7f331400c8d0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 -- 172.21.15.113:0/464480217 shutdown_connections 2024-03-04T08:09:45.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3304066f60 0x7f3304069420 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f332415bce0 0x7f33241f69e0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3324152a90 0x7f3324150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 --2- 172.21.15.113:0/464480217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f332406cd10 0x7f3324153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:45.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 -- 172.21.15.113:0/464480217 >> 172.21.15.113:0/464480217 conn(0x7f332406e2a0 msgr2=0x7f332415c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:45.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 -- 172.21.15.113:0/464480217 shutdown_connections 2024-03-04T08:09:45.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:45.882+0000 7f3323e00640 1 -- 172.21.15.113:0/464480217 wait complete. 2024-03-04T08:09:45.908 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:09:47.909 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:09:48.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.050+0000 7efcb7600640 1 Processor -- start 2024-03-04T08:09:48.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.050+0000 7efcb7600640 1 -- start start 2024-03-04T08:09:48.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb7600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efcb00b20b0 0x7efcb0157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:48.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb7600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb0157c50 0x7efcb006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:48.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb7600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb00c1ec0 con 0x7efcb00c85f0 2024-03-04T08:09:48.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb7600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb00c9b30 con 0x7efcb00b20b0 2024-03-04T08:09:48.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb7600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb00c12a0 con 0x7efcb0157c50 2024-03-04T08:09:48.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb4c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efcb00b20b0 0x7efcb0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:48.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efca7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb0157c50 0x7efcb006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:48.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb4c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efcb00b20b0 0x7efcb0157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:46380/0 (socket says 172.21.15.113:46380) 2024-03-04T08:09:48.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb4c00640 1 -- 172.21.15.113:0/3878655047 learned_addr learned my addr 172.21.15.113:0/3878655047 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:48.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb5600640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7efcb00c85f0 0x7efcb015a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:34264/0 (socket says 172.21.15.113:34264) 2024-03-04T08:09:48.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb4c00640 1 -- 172.21.15.113:0/3878655047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb0157c50 msgr2=0x7efcb006b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb4c00640 1 --2- 172.21.15.113:0/3878655047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb0157c50 0x7efcb006b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb4c00640 1 -- 172.21.15.113:0/3878655047 >> v1:172.21.15.136:6789/0 conn(0x7efcb00c85f0 legacy=0x7efcb015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efcb4c00640 1 -- 172.21.15.113:0/3878655047 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb00c1580 con 0x7efcb00b20b0 2024-03-04T08:09:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.054+0000 7efca7e00640 1 --2- 172.21.15.113:0/3878655047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb0157c50 0x7efcb006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:09:48.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcb4c00640 1 --2- 172.21.15.113:0/3878655047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcb00b20b0 0x7efcb0157710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efca0009870 tx=0x7efca001c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ec37c94ad6a420 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:48.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcafe00640 1 -- 172.21.15.113:0/3878655047 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca00b1070 con 0x7efcb00b20b0 2024-03-04T08:09:48.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcafe00640 1 -- 172.21.15.113:0/3878655047 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca00ac620 con 0x7efcb00b20b0 2024-03-04T08:09:48.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcafe00640 1 -- 172.21.15.113:0/3878655047 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca00ac9f0 con 0x7efcb00b20b0 2024-03-04T08:09:48.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcb7600640 1 -- 172.21.15.113:0/3878655047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcb00b20b0 msgr2=0x7efcb0157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcb7600640 1 --2- 172.21.15.113:0/3878655047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcb00b20b0 0x7efcb0157710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efca0009870 tx=0x7efca001c530 comp rx=0 tx=0).stop 2024-03-04T08:09:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcb7600640 1 -- 172.21.15.113:0/3878655047 shutdown_connections 2024-03-04T08:09:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcb7600640 1 --2- 172.21.15.113:0/3878655047 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb0157c50 0x7efcb006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcb7600640 1 --2- 172.21.15.113:0/3878655047 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcb00b20b0 0x7efcb0157710 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcb7600640 1 -- 172.21.15.113:0/3878655047 >> 172.21.15.113:0/3878655047 conn(0x7efcb006e820 msgr2=0x7efcb006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcb7600640 1 -- 172.21.15.113:0/3878655047 shutdown_connections 2024-03-04T08:09:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.058+0000 7efcb7600640 1 -- 172.21.15.113:0/3878655047 wait complete. 2024-03-04T08:09:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efcb7600640 1 Processor -- start 2024-03-04T08:09:48.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efcb7600640 1 -- start start 2024-03-04T08:09:48.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efcb7600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb00b20b0 0x7efcb014a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efcb7600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb00c85f0 0x7efcb01508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efcb7600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcb0157c50 0x7efcb014aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efcb7600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb00c1b80 con 0x7efcb00c85f0 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efcb7600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb0156ec0 con 0x7efcb0157c50 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efcb7600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb00c80c0 con 0x7efcb00b20b0 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efca7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb00c85f0 0x7efcb01508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efca7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb00c85f0 0x7efcb01508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59446/0 (socket says 172.21.15.113:59446) 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efca7e00640 1 -- 172.21.15.113:0/2587740947 learned_addr learned my addr 172.21.15.113:0/2587740947 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efcb4c00640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb00b20b0 0x7efcb014a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.062+0000 7efcb5600640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcb0157c50 0x7efcb014aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efca7e00640 1 -- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb00b20b0 msgr2=0x7efcb014a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efca7e00640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb00b20b0 0x7efcb014a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:48.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efca7e00640 1 -- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcb0157c50 msgr2=0x7efcb014aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efca7e00640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcb0157c50 0x7efcb014aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efca7e00640 1 -- 172.21.15.113:0/2587740947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb014a720 con 0x7efcb00c85f0 2024-03-04T08:09:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efcb5600640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcb0157c50 0x7efcb014aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:09:48.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efca7e00640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb00c85f0 0x7efcb01508b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7efc9800e8f0 tx=0x7efc9800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efcad600640 1 -- 172.21.15.113:0/2587740947 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98016070 con 0x7efcb00c85f0 2024-03-04T08:09:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efcad600640 1 -- 172.21.15.113:0/2587740947 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9800cc40 con 0x7efcb00c85f0 2024-03-04T08:09:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efcb7600640 1 -- 172.21.15.113:0/2587740947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb014b450 con 0x7efcb00c85f0 2024-03-04T08:09:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efcad600640 1 -- 172.21.15.113:0/2587740947 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9800b040 con 0x7efcb00c85f0 2024-03-04T08:09:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efcb7600640 1 -- 172.21.15.113:0/2587740947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb00c4d00 con 0x7efcb00c85f0 2024-03-04T08:09:48.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efcb7600640 1 -- 172.21.15.113:0/2587740947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb0075c00 con 0x7efcb00c85f0 2024-03-04T08:09:48.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.066+0000 7efcad600640 1 -- 172.21.15.113:0/2587740947 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc98019070 con 0x7efcb00c85f0 2024-03-04T08:09:48.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.070+0000 7efcad600640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc8c066f90 0x7efc8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:48.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.070+0000 7efcad600640 1 -- 172.21.15.113:0/2587740947 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7efc980855e0 con 0x7efcb00c85f0 2024-03-04T08:09:48.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.070+0000 7efcb4c00640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc8c066f90 0x7efc8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:48.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.070+0000 7efcb4c00640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc8c066f90 0x7efc8c069450 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efca00ae000 tx=0x7efca0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:48.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.074+0000 7efcad600640 1 -- 172.21.15.113:0/2587740947 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efc98055790 con 0x7efcb00c85f0 2024-03-04T08:09:48.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.410+0000 7efcb7600640 1 -- 172.21.15.113:0/2587740947 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcb00c83d0 con 0x7efcb00c85f0 2024-03-04T08:09:48.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.410+0000 7efcad600640 1 -- 172.21.15.113:0/2587740947 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+6621 (secure 0 0 0) 0x7efc98059830 con 0x7efcb00c85f0 2024-03-04T08:09:48.409 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:09:48.409 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:08:04.266596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269,7119]},"id":1}]} 2024-03-04T08:09:48.409 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 132 2024-03-04T08:09:48.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 -- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc8c066f90 msgr2=0x7efc8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc8c066f90 0x7efc8c069450 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7efca00ae000 tx=0x7efca0008040 comp rx=0 tx=0).stop 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 -- 172.21.15.113:0/2587740947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb00c85f0 msgr2=0x7efcb01508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb00c85f0 0x7efcb01508b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7efc9800e8f0 tx=0x7efc9800edb0 comp rx=0 tx=0).stop 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 -- 172.21.15.113:0/2587740947 shutdown_connections 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efcb0157c50 0x7efcb014aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc8c066f90 0x7efc8c069450 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcb00c85f0 0x7efcb01508b0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 --2- 172.21.15.113:0/2587740947 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efcb00b20b0 0x7efcb014a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 -- 172.21.15.113:0/2587740947 >> 172.21.15.113:0/2587740947 conn(0x7efcb006e820 msgr2=0x7efcb014b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 -- 172.21.15.113:0/2587740947 shutdown_connections 2024-03-04T08:09:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:48.414+0000 7efcb7600640 1 -- 172.21.15.113:0/2587740947 wait complete. 2024-03-04T08:09:48.430 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:09:50.432 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:09:50.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e8200640 1 Processor -- start 2024-03-04T08:09:50.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e8200640 1 -- start start 2024-03-04T08:09:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e8200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7e0152b20 0x7fc7e0070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e8200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc7e00c8b00 0x7fc7e015be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7e015c340 0x7fc7e015c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e8200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e00c8f90 con 0x7fc7e0152b20 2024-03-04T08:09:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e8200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e00c5320 con 0x7fc7e00c8b00 2024-03-04T08:09:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0152f00 con 0x7fc7e015c340 2024-03-04T08:09:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e5800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7e0152b20 0x7fc7e0070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e5800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7e0152b20 0x7fc7e0070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59456/0 (socket says 172.21.15.113:59456) 2024-03-04T08:09:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e5800640 1 -- 172.21.15.113:0/1653157219 learned_addr learned my addr 172.21.15.113:0/1653157219 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e4e00640 1 --2- 172.21.15.113:0/1653157219 >> v2:172.21.15.136:3300/0 conn(0x7fc7e00c8b00 0x7fc7e015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:50.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e6200640 1 --2- 172.21.15.113:0/1653157219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7e015c340 0x7fc7e015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:50.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e5800640 1 -- 172.21.15.113:0/1653157219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7e015c340 msgr2=0x7fc7e015c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:50.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e5800640 1 --2- 172.21.15.113:0/1653157219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7e015c340 0x7fc7e015c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e5800640 1 -- 172.21.15.113:0/1653157219 >> v2:172.21.15.136:3300/0 conn(0x7fc7e00c8b00 msgr2=0x7fc7e015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:50.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e5800640 1 --2- 172.21.15.113:0/1653157219 >> v2:172.21.15.136:3300/0 conn(0x7fc7e00c8b00 0x7fc7e015be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.574+0000 7fc7e5800640 1 -- 172.21.15.113:0/1653157219 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e00c5040 con 0x7fc7e0152b20 2024-03-04T08:09:50.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7e5800640 1 --2- 172.21.15.113:0/1653157219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e0152b20 0x7fc7e0070010 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0050420 tx=0x7fc7d401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a4f98ad563a5bb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7dfe00640 1 -- 172.21.15.113:0/1653157219 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40b0070 con 0x7fc7e0152b20 2024-03-04T08:09:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7dfe00640 1 -- 172.21.15.113:0/1653157219 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d40ab600 con 0x7fc7e0152b20 2024-03-04T08:09:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7dfe00640 1 -- 172.21.15.113:0/1653157219 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40ab9d0 con 0x7fc7e0152b20 2024-03-04T08:09:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7e8200640 1 -- 172.21.15.113:0/1653157219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e0152b20 msgr2=0x7fc7e0070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7e8200640 1 --2- 172.21.15.113:0/1653157219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e0152b20 0x7fc7e0070010 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0050420 tx=0x7fc7d401c520 comp rx=0 tx=0).stop 2024-03-04T08:09:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7e8200640 1 -- 172.21.15.113:0/1653157219 shutdown_connections 2024-03-04T08:09:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7e8200640 1 --2- 172.21.15.113:0/1653157219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7e015c340 0x7fc7e015c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7e8200640 1 --2- 172.21.15.113:0/1653157219 >> v2:172.21.15.136:3300/0 conn(0x7fc7e00c8b00 0x7fc7e015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7e8200640 1 --2- 172.21.15.113:0/1653157219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e0152b20 0x7fc7e0070010 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7e8200640 1 -- 172.21.15.113:0/1653157219 >> 172.21.15.113:0/1653157219 conn(0x7fc7e00709a0 msgr2=0x7fc7e0070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:50.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7e8200640 1 -- 172.21.15.113:0/1653157219 shutdown_connections 2024-03-04T08:09:50.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.578+0000 7fc7e8200640 1 -- 172.21.15.113:0/1653157219 wait complete. 2024-03-04T08:09:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e8200640 1 Processor -- start 2024-03-04T08:09:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e8200640 1 -- start start 2024-03-04T08:09:50.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e00c8b00 0x7fc7e00c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7e0152b20 0x7fc7e0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e8200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7e015c340 0x7fc7e014d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e0158810 con 0x7fc7e00c8b00 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e8200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e00c3810 con 0x7fc7e015c340 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e00c8490 con 0x7fc7e0152b20 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e00c8b00 0x7fc7e00c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e00c8b00 0x7fc7e00c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59458/0 (socket says 172.21.15.113:59458) 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e5800640 1 -- 172.21.15.113:0/1648108387 learned_addr learned my addr 172.21.15.113:0/1648108387 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e4e00640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7e0152b20 0x7fc7e0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e6200640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7e015c340 0x7fc7e014d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e5800640 1 -- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7e0152b20 msgr2=0x7fc7e0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e5800640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7e0152b20 0x7fc7e0156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e5800640 1 -- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7e015c340 msgr2=0x7fc7e014d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e5800640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7e015c340 0x7fc7e014d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e5800640 1 -- 172.21.15.113:0/1648108387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e014a260 con 0x7fc7e00c8b00 2024-03-04T08:09:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e5800640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e00c8b00 0x7fc7e00c7180 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40ae040 tx=0x7fc7d40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7dd600640 1 -- 172.21.15.113:0/1648108387 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40b0040 con 0x7fc7e00c8b00 2024-03-04T08:09:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7dd600640 1 -- 172.21.15.113:0/1648108387 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d40b9070 con 0x7fc7e00c8b00 2024-03-04T08:09:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e8200640 1 -- 172.21.15.113:0/1648108387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e014a540 con 0x7fc7e00c8b00 2024-03-04T08:09:50.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7dd600640 1 -- 172.21.15.113:0/1648108387 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40b46a0 con 0x7fc7e00c8b00 2024-03-04T08:09:50.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.582+0000 7fc7e8200640 1 -- 172.21.15.113:0/1648108387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e014bf80 con 0x7fc7e00c8b00 2024-03-04T08:09:50.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.586+0000 7fc7e8200640 1 -- 172.21.15.113:0/1648108387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e0075c00 con 0x7fc7e00c8b00 2024-03-04T08:09:50.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.586+0000 7fc7dd600640 1 -- 172.21.15.113:0/1648108387 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7d40b4840 con 0x7fc7e00c8b00 2024-03-04T08:09:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.586+0000 7fc7dd600640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7c0066f60 0x7fc7c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.586+0000 7fc7dd600640 1 -- 172.21.15.113:0/1648108387 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7fc7d411f280 con 0x7fc7e00c8b00 2024-03-04T08:09:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.586+0000 7fc7e4e00640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7c0066f60 0x7fc7c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.586+0000 7fc7e4e00640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7c0066f60 0x7fc7c0069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc7e006f830 tx=0x7fc7cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.594+0000 7fc7dd600640 1 -- 172.21.15.113:0/1648108387 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc7d4016110 con 0x7fc7e00c8b00 2024-03-04T08:09:50.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.818+0000 7fc7e8200640 1 -- 172.21.15.113:0/1648108387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7e006fdf0 con 0x7fc7e00c8b00 2024-03-04T08:09:50.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.818+0000 7fc7dd600640 1 -- 172.21.15.113:0/1648108387 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+6621 (secure 0 0 0) 0x7fc7d40ef550 con 0x7fc7e00c8b00 2024-03-04T08:09:50.817 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:09:50.817 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:08:04.266596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269,7119]},"id":1}]} 2024-03-04T08:09:50.817 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 132 2024-03-04T08:09:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 -- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7c0066f60 msgr2=0x7fc7c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7c0066f60 0x7fc7c0069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc7e006f830 tx=0x7fc7cc008040 comp rx=0 tx=0).stop 2024-03-04T08:09:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 -- 172.21.15.113:0/1648108387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e00c8b00 msgr2=0x7fc7e00c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e00c8b00 0x7fc7e00c7180 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40ae040 tx=0x7fc7d40192c0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 -- 172.21.15.113:0/1648108387 shutdown_connections 2024-03-04T08:09:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7e015c340 0x7fc7e014d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7e0152b20 0x7fc7e0156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7c0066f60 0x7fc7c0069420 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 --2- 172.21.15.113:0/1648108387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7e00c8b00 0x7fc7e00c7180 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:50.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 -- 172.21.15.113:0/1648108387 >> 172.21.15.113:0/1648108387 conn(0x7fc7e00709a0 msgr2=0x7fc7e006ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:50.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 -- 172.21.15.113:0/1648108387 shutdown_connections 2024-03-04T08:09:50.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:50.822+0000 7fc7e8200640 1 -- 172.21.15.113:0/1648108387 wait complete. 2024-03-04T08:09:50.845 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:09:52.846 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:09:52.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.985+0000 7f5efd400640 1 Processor -- start 2024-03-04T08:09:52.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.985+0000 7f5efd400640 1 -- start start 2024-03-04T08:09:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5efd400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5ef8152920 0x7f5ef81593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5efd400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ef80c7370 0x7f5ef80cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5efd400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef81598e0 con 0x7f5ef80c6bf0 2024-03-04T08:09:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5efd400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef815ce70 con 0x7f5ef8152920 2024-03-04T08:09:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5efd400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80c78e0 con 0x7f5ef80c7370 2024-03-04T08:09:52.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5eede00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5ef80c6bf0 0x7f5ef8159e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47138/0 (socket says 172.21.15.113:47138) 2024-03-04T08:09:52.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5eede00640 1 -- 172.21.15.113:0/1340899897 learned_addr learned my addr 172.21.15.113:0/1340899897 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:09:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef6800640 1 --2- 172.21.15.113:0/1340899897 >> v2:172.21.15.136:3300/0 conn(0x7f5ef8152920 0x7f5ef81593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef7200640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ef80c7370 0x7f5ef80cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115313987 0 0) 0x7f5ee0001120 con 0x7f5ef80c6bf0 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5edc003530 con 0x7f5ef80c6bf0 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1822336291 0 0) 0x7f5edc003530 con 0x7f5ef80c6bf0 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ef80c7370 msgr2=0x7f5ef80cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ef80c7370 0x7f5ef80cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 >> v2:172.21.15.136:3300/0 conn(0x7f5ef8152920 msgr2=0x7f5ef81593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 --2- 172.21.15.113:0/1340899897 >> v2:172.21.15.136:3300/0 conn(0x7f5ef8152920 0x7f5ef81593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef815c0b0 con 0x7f5ef80c6bf0 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f5ee0001850 con 0x7f5ef80c6bf0 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 >> v1:172.21.15.113:6789/0 conn(0x7f5ef80c6bf0 legacy=0x7f5ef8159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5edc0039c0 0x7f5edc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ef80c7370 0x7f5edc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5edc004d50 0x7f5edc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef815ce70 con 0x7f5edc0039c0 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80c78e0 con 0x7f5edc004d50 2024-03-04T08:09:52.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5edc00b9d0 con 0x7f5ef80c7370 2024-03-04T08:09:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef6800640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5edc0039c0 0x7f5edc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5eede00640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ef80c7370 0x7f5edc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef7200640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5edc004d50 0x7f5edc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef6800640 1 -- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ef80c7370 msgr2=0x7f5edc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef6800640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ef80c7370 0x7f5edc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef6800640 1 -- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5edc004d50 msgr2=0x7f5edc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef6800640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5edc004d50 0x7f5edc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:52.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.989+0000 7f5ef6800640 1 -- 172.21.15.113:0/1340899897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc003530 con 0x7f5edc0039c0 2024-03-04T08:09:52.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5ef6800640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5edc0039c0 0x7f5edc003de0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009fc0 tx=0x7f5ee801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26148d4f8676472 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80b2070 con 0x7f5edc0039c0 2024-03-04T08:09:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee80a5ba0 con 0x7f5edc0039c0 2024-03-04T08:09:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5ef5e00640 1 -- 172.21.15.113:0/1340899897 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8009c10 con 0x7f5edc0039c0 2024-03-04T08:09:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5efd400640 1 -- 172.21.15.113:0/1340899897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5edc0039c0 msgr2=0x7f5edc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5efd400640 1 --2- 172.21.15.113:0/1340899897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5edc0039c0 0x7f5edc003de0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009fc0 tx=0x7f5ee801ccc0 comp rx=0 tx=0).stop 2024-03-04T08:09:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5ef7200640 1 -- 172.21.15.113:0/1340899897 reap_dead start 2024-03-04T08:09:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5efd400640 1 -- 172.21.15.113:0/1340899897 shutdown_connections 2024-03-04T08:09:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5efd400640 1 -- 172.21.15.113:0/1340899897 >> 172.21.15.113:0/1340899897 conn(0x7f5ef806b060 msgr2=0x7f5ef806b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5efd400640 1 -- 172.21.15.113:0/1340899897 shutdown_connections 2024-03-04T08:09:52.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.993+0000 7f5efd400640 1 -- 172.21.15.113:0/1340899897 wait complete. 2024-03-04T08:09:52.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5efd400640 1 Processor -- start 2024-03-04T08:09:52.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5efd400640 1 -- start start 2024-03-04T08:09:52.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5efd400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5edc0039c0 0x7f5ef815d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:52.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5efd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef8073110 0x7f5ef8157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:52.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5efd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef814e350 0x7f5ef8071a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:52.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5efd400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8152d00 con 0x7f5ef8073110 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5efd400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef815a520 con 0x7f5ef814e350 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5efd400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef81632a0 con 0x7f5edc0039c0 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eede00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef8073110 0x7f5ef8157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eede00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef8073110 0x7f5ef8157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59484/0 (socket says 172.21.15.113:59484) 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eede00640 1 -- 172.21.15.113:0/1940302953 learned_addr learned my addr 172.21.15.113:0/1940302953 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5ef6800640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5edc0039c0 0x7f5ef815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5ef7200640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef814e350 0x7f5ef8071a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eede00640 1 -- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5edc0039c0 msgr2=0x7f5ef815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eede00640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5edc0039c0 0x7f5ef815d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eede00640 1 -- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef814e350 msgr2=0x7f5ef8071a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eede00640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef814e350 0x7f5ef8071a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eede00640 1 -- 172.21.15.113:0/1940302953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef81f4540 con 0x7f5ef8073110 2024-03-04T08:09:52.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eede00640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef8073110 0x7f5ef8157c40 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000e820 tx=0x7f5ee000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:52.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eef400640 1 -- 172.21.15.113:0/1940302953 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000c8d0 con 0x7f5ef8073110 2024-03-04T08:09:52.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eef400640 1 -- 172.21.15.113:0/1940302953 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee0018070 con 0x7f5ef8073110 2024-03-04T08:09:52.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5eef400640 1 -- 172.21.15.113:0/1940302953 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000ccd0 con 0x7f5ef8073110 2024-03-04T08:09:52.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5efd400640 1 -- 172.21.15.113:0/1940302953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef80715b0 con 0x7f5ef8073110 2024-03-04T08:09:52.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:52.997+0000 7f5efd400640 1 -- 172.21.15.113:0/1940302953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef8153140 con 0x7f5ef8073110 2024-03-04T08:09:53.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.001+0000 7f5efd400640 1 -- 172.21.15.113:0/1940302953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef8075c00 con 0x7f5ef8073110 2024-03-04T08:09:53.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.001+0000 7f5eef400640 1 -- 172.21.15.113:0/1940302953 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ee0013420 con 0x7f5ef8073110 2024-03-04T08:09:53.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.001+0000 7f5eef400640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ed4066f90 0x7f5ed4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:53.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.001+0000 7f5eef400640 1 -- 172.21.15.113:0/1940302953 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7f5ee00854a0 con 0x7f5ef8073110 2024-03-04T08:09:53.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.001+0000 7f5ef6800640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ed4066f90 0x7f5ed4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:53.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.001+0000 7f5ef6800640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ed4066f90 0x7f5ed4069450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80af000 tx=0x7f5ee80a7040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:53.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.005+0000 7f5eef400640 1 -- 172.21.15.113:0/1940302953 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5ef81598e0 con 0x7f5ef8073110 2024-03-04T08:09:53.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.221+0000 7f5efd400640 1 -- 172.21.15.113:0/1940302953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ef80c7ce0 con 0x7f5ef8073110 2024-03-04T08:09:53.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.221+0000 7f5eef400640 1 -- 172.21.15.113:0/1940302953 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f5ee0055cb0 con 0x7f5ef8073110 2024-03-04T08:09:53.222 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:09:53.222 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:08:04.266596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269,7119]},"id":1}]} 2024-03-04T08:09:53.223 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 132 2024-03-04T08:09:53.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 -- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ed4066f90 msgr2=0x7f5ed4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:53.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ed4066f90 0x7f5ed4069450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80af000 tx=0x7f5ee80a7040 comp rx=0 tx=0).stop 2024-03-04T08:09:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 -- 172.21.15.113:0/1940302953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef8073110 msgr2=0x7f5ef8157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef8073110 0x7f5ef8157c40 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000e820 tx=0x7f5ee000ece0 comp rx=0 tx=0).stop 2024-03-04T08:09:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 -- 172.21.15.113:0/1940302953 shutdown_connections 2024-03-04T08:09:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef814e350 0x7f5ef8071a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ed4066f90 0x7f5ed4069450 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ef8073110 0x7f5ef8157c40 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 --2- 172.21.15.113:0/1940302953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5edc0039c0 0x7f5ef815d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 -- 172.21.15.113:0/1940302953 >> 172.21.15.113:0/1940302953 conn(0x7f5ef806b060 msgr2=0x7f5ef80c8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 -- 172.21.15.113:0/1940302953 shutdown_connections 2024-03-04T08:09:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:53.225+0000 7f5efd400640 1 -- 172.21.15.113:0/1940302953 wait complete. 2024-03-04T08:09:53.251 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:09:55.253 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:09:55.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff328a00640 1 Processor -- start 2024-03-04T08:09:55.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff328a00640 1 -- start start 2024-03-04T08:09:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff328a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff32415dfc0 0x7ff32415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff328a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32415e930 con 0x7ff324154c00 2024-03-04T08:09:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff328a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32415ab20 con 0x7ff3240c8ae0 2024-03-04T08:09:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff328a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32415ae00 con 0x7ff32415dfc0 2024-03-04T08:09:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff322800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff324154c00 0x7ff3240c7090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47144/0 (socket says 172.21.15.113:47144) 2024-03-04T08:09:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff322800640 1 -- 172.21.15.113:0/3683318562 learned_addr learned my addr 172.21.15.113:0/3683318562 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:09:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff323200640 1 --2- 172.21.15.113:0/3683318562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff32415dfc0 0x7ff32415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff321e00640 1 -- 172.21.15.113:0/3683318562 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354155150 0 0) 0x7ff32415e930 con 0x7ff324154c00 2024-03-04T08:09:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff321e00640 1 -- 172.21.15.113:0/3683318562 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff308003530 con 0x7ff324154c00 2024-03-04T08:09:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff321e00640 1 -- 172.21.15.113:0/3683318562 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951024577 0 0) 0x7ff32415ab20 con 0x7ff3240c8ae0 2024-03-04T08:09:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff321e00640 1 -- 172.21.15.113:0/3683318562 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff32415e930 con 0x7ff3240c8ae0 2024-03-04T08:09:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff323200640 1 -- 172.21.15.113:0/3683318562 >> v1:172.21.15.136:6789/0 conn(0x7ff3240c8ae0 legacy=0x7ff3240cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff323200640 1 -- 172.21.15.113:0/3683318562 >> v1:172.21.15.113:6789/0 conn(0x7ff324154c00 legacy=0x7ff3240c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:55.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.401+0000 7ff323200640 1 -- 172.21.15.113:0/3683318562 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32415b0e0 con 0x7ff32415dfc0 2024-03-04T08:09:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.405+0000 7ff323200640 1 --2- 172.21.15.113:0/3683318562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff32415dfc0 0x7ff32415e3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009dc0 tx=0x7ff31c01c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471cd13ccd67c087 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.405+0000 7ff321e00640 1 -- 172.21.15.113:0/3683318562 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0b2070 con 0x7ff32415dfc0 2024-03-04T08:09:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.405+0000 7ff321e00640 1 -- 172.21.15.113:0/3683318562 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c0a57c0 con 0x7ff32415dfc0 2024-03-04T08:09:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.405+0000 7ff321e00640 1 -- 172.21.15.113:0/3683318562 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0a5b70 con 0x7ff32415dfc0 2024-03-04T08:09:55.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.405+0000 7ff328a00640 1 -- 172.21.15.113:0/3683318562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff32415dfc0 msgr2=0x7ff32415e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:55.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.405+0000 7ff328a00640 1 --2- 172.21.15.113:0/3683318562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff32415dfc0 0x7ff32415e3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009dc0 tx=0x7ff31c01c900 comp rx=0 tx=0).stop 2024-03-04T08:09:55.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.409+0000 7ff328a00640 1 -- 172.21.15.113:0/3683318562 shutdown_connections 2024-03-04T08:09:55.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.409+0000 7ff328a00640 1 --2- 172.21.15.113:0/3683318562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff32415dfc0 0x7ff32415e3f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:55.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.409+0000 7ff328a00640 1 -- 172.21.15.113:0/3683318562 >> 172.21.15.113:0/3683318562 conn(0x7ff32406b060 msgr2=0x7ff32406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:55.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.409+0000 7ff328a00640 1 -- 172.21.15.113:0/3683318562 shutdown_connections 2024-03-04T08:09:55.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.409+0000 7ff328a00640 1 -- 172.21.15.113:0/3683318562 wait complete. 2024-03-04T08:09:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.409+0000 7ff328a00640 1 Processor -- start 2024-03-04T08:09:55.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.409+0000 7ff328a00640 1 -- start start 2024-03-04T08:09:55.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff328a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3240c8ae0 0x7ff324153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:55.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff328a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff324154c00 0x7ff324150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:55.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff328a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff32415dfc0 0x7ff3241505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:55.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff328a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240ccd00 con 0x7ff324154c00 2024-03-04T08:09:55.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff328a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240c8ef0 con 0x7ff32415dfc0 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff328a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240cce80 con 0x7ff3240c8ae0 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff319e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff324154c00 0x7ff324150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff319e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff324154c00 0x7ff324150080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59496/0 (socket says 172.21.15.113:59496) 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff319e00640 1 -- 172.21.15.113:0/1020694532 learned_addr learned my addr 172.21.15.113:0/1020694532 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff322800640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3240c8ae0 0x7ff324153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff323200640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff32415dfc0 0x7ff3241505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff319e00640 1 -- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3240c8ae0 msgr2=0x7ff324153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff319e00640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3240c8ae0 0x7ff324153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff319e00640 1 -- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff32415dfc0 msgr2=0x7ff3241505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff319e00640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff32415dfc0 0x7ff3241505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:55.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff319e00640 1 -- 172.21.15.113:0/1020694532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3240c43c0 con 0x7ff324154c00 2024-03-04T08:09:55.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff319e00640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff324154c00 0x7ff324150080 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c460 tx=0x7ff30c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:55.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff31b400640 1 -- 172.21.15.113:0/1020694532 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c017070 con 0x7ff324154c00 2024-03-04T08:09:55.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff31b400640 1 -- 172.21.15.113:0/1020694532 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c0052f0 con 0x7ff324154c00 2024-03-04T08:09:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff328a00640 1 -- 172.21.15.113:0/1020694532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3240c4630 con 0x7ff324154c00 2024-03-04T08:09:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff31b400640 1 -- 172.21.15.113:0/1020694532 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c003840 con 0x7ff324154c00 2024-03-04T08:09:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff328a00640 1 -- 172.21.15.113:0/1020694532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3240c4bf0 con 0x7ff324154c00 2024-03-04T08:09:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff328a00640 1 -- 172.21.15.113:0/1020694532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff324075a80 con 0x7ff324154c00 2024-03-04T08:09:55.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.413+0000 7ff31b400640 1 -- 172.21.15.113:0/1020694532 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff30c01a070 con 0x7ff324154c00 2024-03-04T08:09:55.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.417+0000 7ff31b400640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff300066f90 0x7ff300069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:55.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.417+0000 7ff31b400640 1 -- 172.21.15.113:0/1020694532 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7459+0+0 (secure 0 0 0) 0x7ff30c084a90 con 0x7ff324154c00 2024-03-04T08:09:55.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.417+0000 7ff322800640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff300066f90 0x7ff300069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:55.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.417+0000 7ff322800640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff300066f90 0x7ff300069450 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff3140011f0 tx=0x7ff314007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:55.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.425+0000 7ff31b400640 1 -- 172.21.15.113:0/1020694532 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff30c054cd0 con 0x7ff324154c00 2024-03-04T08:09:55.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.685+0000 7ff328a00640 1 -- 172.21.15.113:0/1020694532 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff32414fd80 con 0x7ff324154c00 2024-03-04T08:09:55.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff31b400640 1 -- 172.21.15.113:0/1020694532 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+6621 (secure 0 0 0) 0x7ff32414fd80 con 0x7ff324154c00 2024-03-04T08:09:55.688 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:09:55.688 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:08:04.266596+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7119,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7119":{"gid":7119,"name":"c","rank":1,"incarnation":114,"state":"up:active","state_seq":25,"addr":"172.21.15.113:6833/3825257303","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3825257303},{"type":"v1","addr":"172.21.15.113:6833","nonce":3825257303}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269,7119]},"id":1}]} 2024-03-04T08:09:55.688 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 132 2024-03-04T08:09:55.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 -- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff300066f90 msgr2=0x7ff300069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:55.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff300066f90 0x7ff300069450 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff3140011f0 tx=0x7ff314007d40 comp rx=0 tx=0).stop 2024-03-04T08:09:55.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 -- 172.21.15.113:0/1020694532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff324154c00 msgr2=0x7ff324150080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:55.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff324154c00 0x7ff324150080 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c460 tx=0x7ff30c00c920 comp rx=0 tx=0).stop 2024-03-04T08:09:55.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 -- 172.21.15.113:0/1020694532 shutdown_connections 2024-03-04T08:09:55.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff32415dfc0 0x7ff3241505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:55.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff300066f90 0x7ff300069450 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:55.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff324154c00 0x7ff324150080 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:55.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 --2- 172.21.15.113:0/1020694532 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3240c8ae0 0x7ff324153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:55.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 -- 172.21.15.113:0/1020694532 >> 172.21.15.113:0/1020694532 conn(0x7ff32406b060 msgr2=0x7ff32414e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:55.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 -- 172.21.15.113:0/1020694532 shutdown_connections 2024-03-04T08:09:55.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:55.689+0000 7ff328a00640 1 -- 172.21.15.113:0/1020694532 wait complete. 2024-03-04T08:09:55.718 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:09:57.719 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:09:57.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.873+0000 7f0ed7e00640 1 Processor -- start 2024-03-04T08:09:57.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed7e00640 1 -- start start 2024-03-04T08:09:57.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0ed8157df0 0x7f0ed80c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:57.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed81581d0 0x7f0ed806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:57.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed806de30 con 0x7f0ed806ed80 2024-03-04T08:09:57.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed806dc00 con 0x7f0ed8157df0 2024-03-04T08:09:57.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed806e110 con 0x7f0ed81581d0 2024-03-04T08:09:57.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ece000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed81581d0 0x7f0ed806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:57.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed6a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0ed8157df0 0x7f0ed80c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:57.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ece000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed81581d0 0x7f0ed806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:36492/0 (socket says 172.21.15.113:36492) 2024-03-04T08:09:57.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed7400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f0ed806ed80 0x7f0ed80c9be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:55928/0 (socket says 172.21.15.113:55928) 2024-03-04T08:09:57.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed7400640 1 -- 172.21.15.113:0/1699957062 learned_addr learned my addr 172.21.15.113:0/1699957062 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:09:57.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed6000640 1 -- 172.21.15.113:0/1699957062 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945574210 0 0) 0x7f0ed806de30 con 0x7f0ed806ed80 2024-03-04T08:09:57.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ed6000640 1 -- 172.21.15.113:0/1699957062 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ebc003530 con 0x7f0ed806ed80 2024-03-04T08:09:57.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ece000640 1 -- 172.21.15.113:0/1699957062 >> v2:172.21.15.136:3300/0 conn(0x7f0ed8157df0 msgr2=0x7f0ed80c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:57.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ece000640 1 --2- 172.21.15.113:0/1699957062 >> v2:172.21.15.136:3300/0 conn(0x7f0ed8157df0 0x7f0ed80c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:57.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ece000640 1 -- 172.21.15.113:0/1699957062 >> v1:172.21.15.136:6789/0 conn(0x7f0ed806ed80 legacy=0x7f0ed80c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:57.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.877+0000 7f0ece000640 1 -- 172.21.15.113:0/1699957062 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80ce0c0 con 0x7f0ed81581d0 2024-03-04T08:09:57.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.881+0000 7f0ece000640 1 --2- 172.21.15.113:0/1699957062 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed81581d0 0x7f0ed806c170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00097c0 tx=0x7f0ec001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e429c07551dad7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:57.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.881+0000 7f0ed6000640 1 -- 172.21.15.113:0/1699957062 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00b5070 con 0x7f0ed81581d0 2024-03-04T08:09:57.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.881+0000 7f0ed6000640 1 -- 172.21.15.113:0/1699957062 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec00a8990 con 0x7f0ed81581d0 2024-03-04T08:09:57.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.881+0000 7f0ed6000640 1 -- 172.21.15.113:0/1699957062 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00a8d40 con 0x7f0ed81581d0 2024-03-04T08:09:57.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.881+0000 7f0ed7e00640 1 -- 172.21.15.113:0/1699957062 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed81581d0 msgr2=0x7f0ed806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:57.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.881+0000 7f0ed7e00640 1 --2- 172.21.15.113:0/1699957062 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed81581d0 0x7f0ed806c170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00097c0 tx=0x7f0ec001ec90 comp rx=0 tx=0).stop 2024-03-04T08:09:57.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.881+0000 7f0ed7e00640 1 -- 172.21.15.113:0/1699957062 shutdown_connections 2024-03-04T08:09:57.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.881+0000 7f0ed7e00640 1 --2- 172.21.15.113:0/1699957062 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed81581d0 0x7f0ed806c170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:57.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.881+0000 7f0ed7e00640 1 --2- 172.21.15.113:0/1699957062 >> v2:172.21.15.136:3300/0 conn(0x7f0ed8157df0 0x7f0ed80c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:57.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.881+0000 7f0ed7e00640 1 -- 172.21.15.113:0/1699957062 >> 172.21.15.113:0/1699957062 conn(0x7f0ed8072f40 msgr2=0x7f0ed80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:57.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.885+0000 7f0ed7e00640 1 -- 172.21.15.113:0/1699957062 shutdown_connections 2024-03-04T08:09:57.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.885+0000 7f0ed7e00640 1 -- 172.21.15.113:0/1699957062 wait complete. 2024-03-04T08:09:57.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.885+0000 7f0ed7e00640 1 Processor -- start 2024-03-04T08:09:57.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.885+0000 7f0ed7e00640 1 -- start start 2024-03-04T08:09:57.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed806ed80 0x7f0ed81f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:57.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed8157df0 0x7f0ed814f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:57.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ed81581d0 0x7f0ed80c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:57.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed806fd90 con 0x7f0ed8157df0 2024-03-04T08:09:57.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed806fc10 con 0x7f0ed81581d0 2024-03-04T08:09:57.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80ca2f0 con 0x7f0ed806ed80 2024-03-04T08:09:57.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ece000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed8157df0 0x7f0ed814f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:57.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ece000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed8157df0 0x7f0ed814f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43536/0 (socket says 172.21.15.113:43536) 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ece000640 1 -- 172.21.15.113:0/2756115118 learned_addr learned my addr 172.21.15.113:0/2756115118 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed6a00640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed806ed80 0x7f0ed81f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed7400640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ed81581d0 0x7f0ed80c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ece000640 1 -- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed806ed80 msgr2=0x7f0ed81f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ece000640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed806ed80 0x7f0ed81f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ece000640 1 -- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ed81581d0 msgr2=0x7f0ed80c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ece000640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ed81581d0 0x7f0ed80c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ece000640 1 -- 172.21.15.113:0/2756115118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed81544d0 con 0x7f0ed8157df0 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ece000640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed8157df0 0x7f0ed814f320 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0ec001d340 tx=0x7f0ec001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ecf400640 1 -- 172.21.15.113:0/2756115118 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00c1070 con 0x7f0ed8157df0 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ecf400640 1 -- 172.21.15.113:0/2756115118 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec00b0640 con 0x7f0ed8157df0 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ecf400640 1 -- 172.21.15.113:0/2756115118 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00b5040 con 0x7f0ed8157df0 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed7e00640 1 -- 172.21.15.113:0/2756115118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed81f43f0 con 0x7f0ed8157df0 2024-03-04T08:09:57.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed7e00640 1 -- 172.21.15.113:0/2756115118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed8154a50 con 0x7f0ed8157df0 2024-03-04T08:09:57.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.889+0000 7f0ed7e00640 1 -- 172.21.15.113:0/2756115118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed8154c90 con 0x7f0ed8157df0 2024-03-04T08:09:57.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.893+0000 7f0ecf400640 1 -- 172.21.15.113:0/2756115118 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ec00b09f0 con 0x7f0ed8157df0 2024-03-04T08:09:57.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.893+0000 7f0ecf400640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0eb4066f60 0x7f0eb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:57.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.893+0000 7f0ecf400640 1 -- 172.21.15.113:0/2756115118 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f0ec01225e0 con 0x7f0ed8157df0 2024-03-04T08:09:57.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.893+0000 7f0ed6a00640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0eb4066f60 0x7f0eb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:57.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.893+0000 7f0ed6a00640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0eb4066f60 0x7f0eb4069420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009b60 tx=0x7f0ec8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:57.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:57.901+0000 7f0ecf400640 1 -- 172.21.15.113:0/2756115118 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0ed8154c90 con 0x7f0ed8157df0 2024-03-04T08:09:58.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 -- 172.21.15.113:0/2756115118 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ed80c97b0 con 0x7f0ed8157df0 2024-03-04T08:09:58.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ecf400640 1 -- 172.21.15.113:0/2756115118 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+5820 (secure 0 0 0) 0x7f0ec00187f0 con 0x7f0ed8157df0 2024-03-04T08:09:58.164 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:09:58.165 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":133,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:09:55.881108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7269,7389]},"id":1}]} 2024-03-04T08:09:58.165 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 133 2024-03-04T08:09:58.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 -- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0eb4066f60 msgr2=0x7f0eb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:58.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0eb4066f60 0x7f0eb4069420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009b60 tx=0x7f0ec8009190 comp rx=0 tx=0).stop 2024-03-04T08:09:58.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 -- 172.21.15.113:0/2756115118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed8157df0 msgr2=0x7f0ed814f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:58.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed8157df0 0x7f0ed814f320 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0ec001d340 tx=0x7f0ec001ba30 comp rx=0 tx=0).stop 2024-03-04T08:09:58.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 -- 172.21.15.113:0/2756115118 shutdown_connections 2024-03-04T08:09:58.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0ed81581d0 0x7f0ed80c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0eb4066f60 0x7f0eb4069420 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ed8157df0 0x7f0ed814f320 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 --2- 172.21.15.113:0/2756115118 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0ed806ed80 0x7f0ed81f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 -- 172.21.15.113:0/2756115118 >> 172.21.15.113:0/2756115118 conn(0x7f0ed8072f40 msgr2=0x7f0ed80c9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 -- 172.21.15.113:0/2756115118 shutdown_connections 2024-03-04T08:09:58.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.165+0000 7f0ed7e00640 1 -- 172.21.15.113:0/2756115118 wait complete. 2024-03-04T08:09:58.187 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-03-04T08:09:58.187 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:09:58.187 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:09:58.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.329+0000 7efd5f000640 1 Processor -- start 2024-03-04T08:09:58.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.329+0000 7efd5f000640 1 -- start start 2024-03-04T08:09:58.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.329+0000 7efd5f000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd58070df0 0x7efd580c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:58.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5f000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd580c9c50 con 0x7efd58157ef0 2024-03-04T08:09:58.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5f000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5815a9a0 con 0x7efd5806fb70 2024-03-04T08:09:58.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5f000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5806de30 con 0x7efd58070df0 2024-03-04T08:09:58.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd58070df0 0x7efd580c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:58.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd58070df0 0x7efd580c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43550/0 (socket says 172.21.15.113:43550) 2024-03-04T08:09:58.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5dc00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efd58157ef0 0x7efd5815b180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38556/0 (socket says 172.21.15.113:38556) 2024-03-04T08:09:58.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5d200640 1 -- 172.21.15.113:0/2098096680 learned_addr learned my addr 172.21.15.113:0/2098096680 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:58.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5d200640 1 -- 172.21.15.113:0/2098096680 >> v1:172.21.15.136:6789/0 conn(0x7efd5806fb70 legacy=0x7efd580cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:09:58.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5d200640 1 -- 172.21.15.113:0/2098096680 >> v1:172.21.15.113:6789/0 conn(0x7efd58157ef0 legacy=0x7efd5815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:58.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5d200640 1 -- 172.21.15.113:0/2098096680 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5806c210 con 0x7efd58070df0 2024-03-04T08:09:58.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd5d200640 1 --2- 172.21.15.113:0/2098096680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 0x7efd580c8a90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7efd44009870 tx=0x7efd4401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb43895f10d515eb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:58.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.333+0000 7efd57e00640 1 -- 172.21.15.113:0/2098096680 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd440b2070 con 0x7efd58070df0 2024-03-04T08:09:58.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd57e00640 1 -- 172.21.15.113:0/2098096680 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd440ad640 con 0x7efd58070df0 2024-03-04T08:09:58.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd57e00640 1 -- 172.21.15.113:0/2098096680 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd440ada10 con 0x7efd58070df0 2024-03-04T08:09:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd5f000640 1 -- 172.21.15.113:0/2098096680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 msgr2=0x7efd580c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd5f000640 1 --2- 172.21.15.113:0/2098096680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 0x7efd580c8a90 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7efd44009870 tx=0x7efd4401c3b0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd5f000640 1 -- 172.21.15.113:0/2098096680 shutdown_connections 2024-03-04T08:09:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd5f000640 1 --2- 172.21.15.113:0/2098096680 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 0x7efd580c8a90 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd5f000640 1 -- 172.21.15.113:0/2098096680 >> 172.21.15.113:0/2098096680 conn(0x7efd58072f40 msgr2=0x7efd580706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd5f000640 1 -- 172.21.15.113:0/2098096680 shutdown_connections 2024-03-04T08:09:58.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd5f000640 1 -- 172.21.15.113:0/2098096680 wait complete. 2024-03-04T08:09:58.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd5f000640 1 Processor -- start 2024-03-04T08:09:58.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.337+0000 7efd5f000640 1 -- start start 2024-03-04T08:09:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5f000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd5806fb70 0x7efd580cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5f000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 0x7efd581e28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5f000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd58157ef0 0x7efd581e2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5f000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5806e110 con 0x7efd58070df0 2024-03-04T08:09:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5f000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5815ac60 con 0x7efd5806fb70 2024-03-04T08:09:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5f000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5815b890 con 0x7efd58157ef0 2024-03-04T08:09:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 0x7efd581e28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 0x7efd581e28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43566/0 (socket says 172.21.15.113:43566) 2024-03-04T08:09:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5d200640 1 -- 172.21.15.113:0/4289380151 learned_addr learned my addr 172.21.15.113:0/4289380151 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:09:58.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5e600640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd58157ef0 0x7efd581e2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5dc00640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd5806fb70 0x7efd580cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5d200640 1 -- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd58157ef0 msgr2=0x7efd581e2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5d200640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd58157ef0 0x7efd581e2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5d200640 1 -- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd5806fb70 msgr2=0x7efd580cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5d200640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd5806fb70 0x7efd580cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5d200640 1 -- 172.21.15.113:0/4289380151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd581e0c50 con 0x7efd58070df0 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5e600640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd58157ef0 0x7efd581e2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5d200640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 0x7efd581e28f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7efd440b0040 tx=0x7efd44019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd55600640 1 -- 172.21.15.113:0/4289380151 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd440bd070 con 0x7efd58070df0 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd55600640 1 -- 172.21.15.113:0/4289380151 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd440ab8f0 con 0x7efd58070df0 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5f000640 1 -- 172.21.15.113:0/4289380151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd581537e0 con 0x7efd58070df0 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd55600640 1 -- 172.21.15.113:0/4289380151 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd440b2040 con 0x7efd58070df0 2024-03-04T08:09:58.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5f000640 1 -- 172.21.15.113:0/4289380151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd58153aa0 con 0x7efd58070df0 2024-03-04T08:09:58.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.341+0000 7efd5f000640 1 -- 172.21.15.113:0/4289380151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd58075a80 con 0x7efd58070df0 2024-03-04T08:09:58.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.345+0000 7efd55600640 1 -- 172.21.15.113:0/4289380151 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd440c0070 con 0x7efd58070df0 2024-03-04T08:09:58.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.345+0000 7efd55600640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd38066f60 0x7efd38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:09:58.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.345+0000 7efd55600640 1 -- 172.21.15.113:0/4289380151 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7efd4411f630 con 0x7efd58070df0 2024-03-04T08:09:58.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.345+0000 7efd5dc00640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd38066f60 0x7efd38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:09:58.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.345+0000 7efd5dc00640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd38066f60 0x7efd38069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009fd0 tx=0x7efd4c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:09:58.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.349+0000 7efd55600640 1 -- 172.21.15.113:0/4289380151 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efd44016070 con 0x7efd58070df0 2024-03-04T08:09:58.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.541+0000 7efd5f000640 1 -- 172.21.15.113:0/4289380151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd5806f950 con 0x7efd58070df0 2024-03-04T08:09:58.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.541+0000 7efd55600640 1 -- 172.21.15.113:0/4289380151 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+5820 (secure 0 0 0) 0x7efd440f0080 con 0x7efd58070df0 2024-03-04T08:09:58.542 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:09:58.542 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":133,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:09:55.881108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7269,7389]},"id":1}]} 2024-03-04T08:09:58.542 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 133 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 -- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd38066f60 msgr2=0x7efd38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd38066f60 0x7efd38069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009fd0 tx=0x7efd4c00b040 comp rx=0 tx=0).stop 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 -- 172.21.15.113:0/4289380151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 msgr2=0x7efd581e28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 0x7efd581e28f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7efd440b0040 tx=0x7efd44019150 comp rx=0 tx=0).stop 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 -- 172.21.15.113:0/4289380151 shutdown_connections 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd58157ef0 0x7efd581e2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd38066f60 0x7efd38069420 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd58070df0 0x7efd581e28f0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 --2- 172.21.15.113:0/4289380151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd5806fb70 0x7efd580cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 -- 172.21.15.113:0/4289380151 >> 172.21.15.113:0/4289380151 conn(0x7efd58072f40 msgr2=0x7efd58155680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 -- 172.21.15.113:0/4289380151 shutdown_connections 2024-03-04T08:09:58.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:09:58.545+0000 7efd5f000640 1 -- 172.21.15.113:0/4289380151 wait complete. 2024-03-04T08:09:58.575 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 133, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 133, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7269": { "addr": "172.21.15.113:6837/1401330284", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1401330284, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1401330284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7269, "incarnation": 120, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7389": { "addr": "172.21.15.113:6836/1310057122", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1310057122, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1310057122, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7389, "incarnation": 126, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7519": { "addr": "172.21.15.136:6835/247269911", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 247269911, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 247269911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7519, "incarnation": 133, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 68, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:09:55.881108+0000", "qdb_cluster": [ 7269, 7389 ], "qdb_leader": 7389, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7389, "mds_1": 7519, "mds_2": 7269 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:10:00.575 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:00.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.741+0000 7fd3a3200640 1 Processor -- start 2024-03-04T08:10:00.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.741+0000 7fd3a3200640 1 -- start start 2024-03-04T08:10:00.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.745+0000 7fd3a3200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3a406c070 0x7fd3a4172f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:00.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.745+0000 7fd3a3200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a406f310 con 0x7fd3a408c640 2024-03-04T08:10:00.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.745+0000 7fd3a3200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a406ed90 con 0x7fd3a4164bf0 2024-03-04T08:10:00.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.745+0000 7fd3a3200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a40c83d0 con 0x7fd3a406c070 2024-03-04T08:10:00.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.745+0000 7fd3a2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3a406c070 0x7fd3a4172f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:00.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.745+0000 7fd3a2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3a406c070 0x7fd3a4172f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43574/0 (socket says 172.21.15.113:43574) 2024-03-04T08:10:00.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.745+0000 7fd3a2800640 1 -- 172.21.15.113:0/215453648 learned_addr learned my addr 172.21.15.113:0/215453648 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:00.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.745+0000 7fd399400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd3a408c640 0x7fd3a41676a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38570/0 (socket says 172.21.15.113:38570) 2024-03-04T08:10:00.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.757+0000 7fd3a2800640 1 -- 172.21.15.113:0/215453648 >> v1:172.21.15.136:6789/0 conn(0x7fd3a4164bf0 legacy=0x7fd3a40c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:00.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.745+0000 7fd3a1400640 1 -- 172.21.15.113:0/215453648 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147132976 0 0) 0x7fd3a406f310 con 0x7fd3a408c640 2024-03-04T08:10:00.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.757+0000 7fd3a2800640 1 -- 172.21.15.113:0/215453648 >> v1:172.21.15.113:6789/0 conn(0x7fd3a408c640 legacy=0x7fd3a41676a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:00.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.757+0000 7fd3a2800640 1 -- 172.21.15.113:0/215453648 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a40c7dc0 con 0x7fd3a406c070 2024-03-04T08:10:00.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.761+0000 7fd3a2800640 1 --2- 172.21.15.113:0/215453648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a406c070 0x7fd3a4172f50 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40c8550 tx=0x7fd39c01caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe49dd504d9e01be server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:00.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.761+0000 7fd3a1400640 1 -- 172.21.15.113:0/215453648 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c004230 con 0x7fd3a406c070 2024-03-04T08:10:00.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.761+0000 7fd3a1400640 1 -- 172.21.15.113:0/215453648 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c0a9070 con 0x7fd3a406c070 2024-03-04T08:10:00.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.761+0000 7fd3a1400640 1 -- 172.21.15.113:0/215453648 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c004230 con 0x7fd3a406c070 2024-03-04T08:10:00.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.765+0000 7fd3a3200640 1 -- 172.21.15.113:0/215453648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a406c070 msgr2=0x7fd3a4172f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:00.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.765+0000 7fd3a3200640 1 --2- 172.21.15.113:0/215453648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a406c070 0x7fd3a4172f50 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40c8550 tx=0x7fd39c01caf0 comp rx=0 tx=0).stop 2024-03-04T08:10:00.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.769+0000 7fd3a3200640 1 -- 172.21.15.113:0/215453648 shutdown_connections 2024-03-04T08:10:00.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.769+0000 7fd3a3200640 1 --2- 172.21.15.113:0/215453648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a406c070 0x7fd3a4172f50 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:00.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.769+0000 7fd3a3200640 1 -- 172.21.15.113:0/215453648 >> 172.21.15.113:0/215453648 conn(0x7fd3a406e2a0 msgr2=0x7fd3a406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:00.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.769+0000 7fd3a3200640 1 -- 172.21.15.113:0/215453648 shutdown_connections 2024-03-04T08:10:00.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.769+0000 7fd3a3200640 1 -- 172.21.15.113:0/215453648 wait complete. 2024-03-04T08:10:00.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a3200640 1 Processor -- start 2024-03-04T08:10:00.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a3200640 1 -- start start 2024-03-04T08:10:00.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a3200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a406c070 0x7fd3a40c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:00.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a3200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a408c640 0x7fd3a40af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:00.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a3200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd3a4164bf0 0x7fd3a40af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:00.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a3200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a4165150 con 0x7fd3a408c640 2024-03-04T08:10:00.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a3200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a4164fd0 con 0x7fd3a406c070 2024-03-04T08:10:00.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a3200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a40c1fd0 con 0x7fd3a4164bf0 2024-03-04T08:10:00.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd399400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a408c640 0x7fd3a40af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:00.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a406c070 0x7fd3a40c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:00.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a2800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd3a4164bf0 0x7fd3a40af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:00.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a406c070 0x7fd3a40c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:49874/0 (socket says 172.21.15.113:49874) 2024-03-04T08:10:00.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a1e00640 1 -- 172.21.15.113:0/649757052 learned_addr learned my addr 172.21.15.113:0/649757052 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:00.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd399400640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a408c640 0x7fd3a40af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43580/0 (socket says 172.21.15.113:43580) 2024-03-04T08:10:00.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a1e00640 1 -- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd3a4164bf0 msgr2=0x7fd3a40af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a1e00640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd3a4164bf0 0x7fd3a40af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a1e00640 1 -- 172.21.15.113:0/649757052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a408c640 msgr2=0x7fd3a40af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a1e00640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a408c640 0x7fd3a40af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a1e00640 1 -- 172.21.15.113:0/649757052 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a4205340 con 0x7fd3a406c070 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd399400640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a408c640 0x7fd3a40af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.773+0000 7fd3a1e00640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a406c070 0x7fd3a40c42e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd39400caf0 tx=0x7fd394002180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.777+0000 7fd39aa00640 1 -- 172.21.15.113:0/649757052 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd394017070 con 0x7fd3a406c070 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.777+0000 7fd39aa00640 1 -- 172.21.15.113:0/649757052 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd394003db0 con 0x7fd3a406c070 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.777+0000 7fd3a3200640 1 -- 172.21.15.113:0/649757052 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a40c5220 con 0x7fd3a406c070 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.777+0000 7fd39aa00640 1 -- 172.21.15.113:0/649757052 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39400a9a0 con 0x7fd3a406c070 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.777+0000 7fd3a3200640 1 -- 172.21.15.113:0/649757052 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a40aea80 con 0x7fd3a406c070 2024-03-04T08:10:00.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.777+0000 7fd3a3200640 1 -- 172.21.15.113:0/649757052 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a40c5740 con 0x7fd3a406c070 2024-03-04T08:10:00.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.777+0000 7fd39aa00640 1 -- 172.21.15.113:0/649757052 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd39401a070 con 0x7fd3a406c070 2024-03-04T08:10:00.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.781+0000 7fd39aa00640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd380066f60 0x7fd380069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:00.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.781+0000 7fd39aa00640 1 -- 172.21.15.113:0/649757052 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7fd394085410 con 0x7fd3a406c070 2024-03-04T08:10:00.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.781+0000 7fd399400640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd380066f60 0x7fd380069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:00.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.781+0000 7fd399400640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd380066f60 0x7fd380069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd38c007bc0 tx=0x7fd38c007b10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:00.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:00.789+0000 7fd39aa00640 1 -- 172.21.15.113:0/649757052 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd3a40c5740 con 0x7fd3a406c070 2024-03-04T08:10:01.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.041+0000 7fd3a3200640 1 -- 172.21.15.113:0/649757052 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3a4075a80 con 0x7fd3a406c070 2024-03-04T08:10:01.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd39aa00640 1 -- 172.21.15.113:0/649757052 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+5820 (secure 0 0 0) 0x7fd394055690 con 0x7fd3a406c070 2024-03-04T08:10:01.043 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:01.043 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":133,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:09:55.881108+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7269,7389]},"id":1}]} 2024-03-04T08:10:01.043 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 133 2024-03-04T08:10:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 -- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd380066f60 msgr2=0x7fd380069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd380066f60 0x7fd380069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd38c007bc0 tx=0x7fd38c007b10 comp rx=0 tx=0).stop 2024-03-04T08:10:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 -- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a406c070 msgr2=0x7fd3a40c42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a406c070 0x7fd3a40c42e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd39400caf0 tx=0x7fd394002180 comp rx=0 tx=0).stop 2024-03-04T08:10:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 -- 172.21.15.113:0/649757052 shutdown_connections 2024-03-04T08:10:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd3a4164bf0 0x7fd3a40af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:01.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd380066f60 0x7fd380069420 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3a408c640 0x7fd3a40af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 --2- 172.21.15.113:0/649757052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3a406c070 0x7fd3a40c42e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 -- 172.21.15.113:0/649757052 >> 172.21.15.113:0/649757052 conn(0x7fd3a406e2a0 msgr2=0x7fd3a406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 -- 172.21.15.113:0/649757052 shutdown_connections 2024-03-04T08:10:01.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:01.045+0000 7fd3a3200640 1 -- 172.21.15.113:0/649757052 wait complete. 2024-03-04T08:10:01.065 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:10:03.066 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:03.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9297200640 1 Processor -- start 2024-03-04T08:10:03.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9297200640 1 -- start start 2024-03-04T08:10:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9297200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9298152950 0x7f929815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9297200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 0x7f92980c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9297200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92980c4ce0 con 0x7f929806ed80 2024-03-04T08:10:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9297200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92980c8370 con 0x7f9298152950 2024-03-04T08:10:03.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9297200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92980ca0a0 con 0x7f929806faa0 2024-03-04T08:10:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9295400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 0x7f92980c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9295e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9298152950 0x7f929815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9295400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 0x7f92980c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:36530/0 (socket says 172.21.15.113:36530) 2024-03-04T08:10:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9295400640 1 -- 172.21.15.113:0/3889858170 learned_addr learned my addr 172.21.15.113:0/3889858170 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9295400640 1 -- 172.21.15.113:0/3889858170 >> v2:172.21.15.136:3300/0 conn(0x7f9298152950 msgr2=0x7f929815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9295400640 1 --2- 172.21.15.113:0/3889858170 >> v2:172.21.15.136:3300/0 conn(0x7f9298152950 0x7f929815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9295400640 1 -- 172.21.15.113:0/3889858170 >> v1:172.21.15.136:6789/0 conn(0x7f929806ed80 legacy=0x7f92980ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:03.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.209+0000 7f9295400640 1 -- 172.21.15.113:0/3889858170 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92980c4f80 con 0x7f929806faa0 2024-03-04T08:10:03.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9295400640 1 --2- 172.21.15.113:0/3889858170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 0x7f92980c30b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9284009870 tx=0x7f928401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a17eb29b8407e64b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:03.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9294a00640 1 -- 172.21.15.113:0/3889858170 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92840b5070 con 0x7f929806faa0 2024-03-04T08:10:03.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9294a00640 1 -- 172.21.15.113:0/3889858170 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92840a8a50 con 0x7f929806faa0 2024-03-04T08:10:03.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9294a00640 1 -- 172.21.15.113:0/3889858170 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92840a8e20 con 0x7f929806faa0 2024-03-04T08:10:03.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9297200640 1 -- 172.21.15.113:0/3889858170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 msgr2=0x7f92980c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:03.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9297200640 1 --2- 172.21.15.113:0/3889858170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 0x7f92980c30b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9284009870 tx=0x7f928401ed50 comp rx=0 tx=0).stop 2024-03-04T08:10:03.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9297200640 1 -- 172.21.15.113:0/3889858170 shutdown_connections 2024-03-04T08:10:03.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9297200640 1 --2- 172.21.15.113:0/3889858170 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 0x7f92980c30b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:03.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9297200640 1 --2- 172.21.15.113:0/3889858170 >> v2:172.21.15.136:3300/0 conn(0x7f9298152950 0x7f929815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:03.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9297200640 1 -- 172.21.15.113:0/3889858170 >> 172.21.15.113:0/3889858170 conn(0x7f9298072f40 msgr2=0x7f92980706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:03.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9297200640 1 -- 172.21.15.113:0/3889858170 shutdown_connections 2024-03-04T08:10:03.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.213+0000 7f9297200640 1 -- 172.21.15.113:0/3889858170 wait complete. 2024-03-04T08:10:03.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9297200640 1 Processor -- start 2024-03-04T08:10:03.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9297200640 1 -- start start 2024-03-04T08:10:03.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9297200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f929806ed80 0x7f929806c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:03.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9297200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 0x7f92980b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:03.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9297200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9298152950 0x7f92980b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:03.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9297200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92980c9a50 con 0x7f9298152950 2024-03-04T08:10:03.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9297200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929806fe80 con 0x7f929806ed80 2024-03-04T08:10:03.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9297200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92980c9d30 con 0x7f929806faa0 2024-03-04T08:10:03.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9296800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9298152950 0x7f92980b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:03.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9296800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9298152950 0x7f92980b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43582/0 (socket says 172.21.15.113:43582) 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9296800640 1 -- 172.21.15.113:0/3749822520 learned_addr learned my addr 172.21.15.113:0/3749822520 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9295400640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 0x7f92980b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.217+0000 7f9295e00640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f929806ed80 0x7f929806c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f9296800640 1 -- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 msgr2=0x7f92980b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f9296800640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 0x7f92980b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f9296800640 1 -- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f929806ed80 msgr2=0x7f929806c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f9296800640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f929806ed80 0x7f929806c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f9296800640 1 -- 172.21.15.113:0/3749822520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9298152e60 con 0x7f9298152950 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f9296800640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9298152950 0x7f92980b31b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f929000c760 tx=0x7f929000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f928e000640 1 -- 172.21.15.113:0/3749822520 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290003ba0 con 0x7f9298152950 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f928e000640 1 -- 172.21.15.113:0/3749822520 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9290003d40 con 0x7f9298152950 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f928e000640 1 -- 172.21.15.113:0/3749822520 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929000aa90 con 0x7f9298152950 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f9297200640 1 -- 172.21.15.113:0/3749822520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929806cea0 con 0x7f9298152950 2024-03-04T08:10:03.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f9297200640 1 -- 172.21.15.113:0/3749822520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9298155a90 con 0x7f9298152950 2024-03-04T08:10:03.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f9297200640 1 -- 172.21.15.113:0/3749822520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9298075a80 con 0x7f9298152950 2024-03-04T08:10:03.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f928e000640 1 -- 172.21.15.113:0/3749822520 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f929000ac30 con 0x7f9298152950 2024-03-04T08:10:03.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f928e000640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9278066f60 0x7f9278069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:03.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f928e000640 1 -- 172.21.15.113:0/3749822520 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f9290085510 con 0x7f9298152950 2024-03-04T08:10:03.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.221+0000 7f9295e00640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9278066f60 0x7f9278069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:03.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.225+0000 7f9295e00640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9278066f60 0x7f9278069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9288009b60 tx=0x7f9288009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:03.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.225+0000 7f928e000640 1 -- 172.21.15.113:0/3749822520 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9290055700 con 0x7f9298152950 2024-03-04T08:10:03.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.421+0000 7f9297200640 1 -- 172.21.15.113:0/3749822520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92980c9580 con 0x7f9298152950 2024-03-04T08:10:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.421+0000 7f928e000640 1 -- 172.21.15.113:0/3749822520 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+5822 (secure 0 0 0) 0x7f92900597a0 con 0x7f9298152950 2024-03-04T08:10:03.422 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:03.422 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":134,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:01.097078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:resolve","state_seq":30,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269]},"id":1}]} 2024-03-04T08:10:03.422 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 134 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 -- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9278066f60 msgr2=0x7f9278069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9278066f60 0x7f9278069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9288009b60 tx=0x7f9288009190 comp rx=0 tx=0).stop 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 -- 172.21.15.113:0/3749822520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9298152950 msgr2=0x7f92980b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9298152950 0x7f92980b31b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f929000c760 tx=0x7f929000cc20 comp rx=0 tx=0).stop 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 -- 172.21.15.113:0/3749822520 shutdown_connections 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9278066f60 0x7f9278069420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9298152950 0x7f92980b31b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f929806faa0 0x7f92980b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 --2- 172.21.15.113:0/3749822520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f929806ed80 0x7f929806c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 -- 172.21.15.113:0/3749822520 >> 172.21.15.113:0/3749822520 conn(0x7f9298072f40 msgr2=0x7f9298156150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 -- 172.21.15.113:0/3749822520 shutdown_connections 2024-03-04T08:10:03.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:03.425+0000 7f9297200640 1 -- 172.21.15.113:0/3749822520 wait complete. 2024-03-04T08:10:03.452 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:10:05.454 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:05.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.609+0000 7f44be600640 1 Processor -- start 2024-03-04T08:10:05.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.609+0000 7f44be600640 1 -- start start 2024-03-04T08:10:05.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44be600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44b8152950 0x7f44b815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:05.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44be600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44b806faa0 0x7f44b80c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:05.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44be600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b80c4ce0 con 0x7f44b806ed80 2024-03-04T08:10:05.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44be600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b80c8370 con 0x7f44b806faa0 2024-03-04T08:10:05.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44be600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b80ca0a0 con 0x7f44b8152950 2024-03-04T08:10:05.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44afe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44b806faa0 0x7f44b80c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:05.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44afe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f44b806faa0 0x7f44b80c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43590/0 (socket says 172.21.15.113:43590) 2024-03-04T08:10:05.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44afe00640 1 -- 172.21.15.113:0/2355101989 learned_addr learned my addr 172.21.15.113:0/2355101989 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:05.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44bd200640 1 --2- 172.21.15.113:0/2355101989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44b8152950 0x7f44b815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:05.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44afe00640 1 -- 172.21.15.113:0/2355101989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44b8152950 msgr2=0x7f44b815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:05.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44afe00640 1 --2- 172.21.15.113:0/2355101989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44b8152950 0x7f44b815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:05.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44afe00640 1 -- 172.21.15.113:0/2355101989 >> v1:172.21.15.136:6789/0 conn(0x7f44b806ed80 legacy=0x7f44b80ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:05.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.613+0000 7f44afe00640 1 -- 172.21.15.113:0/2355101989 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b80c4f80 con 0x7f44b806faa0 2024-03-04T08:10:05.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44afe00640 1 --2- 172.21.15.113:0/2355101989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 0x7f44b80c30b0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009870 tx=0x7f44a001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a1989a49ce867e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:05.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44b7e00640 1 -- 172.21.15.113:0/2355101989 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a00b5070 con 0x7f44b806faa0 2024-03-04T08:10:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44b7e00640 1 -- 172.21.15.113:0/2355101989 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a00a8a50 con 0x7f44b806faa0 2024-03-04T08:10:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44b7e00640 1 -- 172.21.15.113:0/2355101989 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a00a8e20 con 0x7f44b806faa0 2024-03-04T08:10:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 -- 172.21.15.113:0/2355101989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 msgr2=0x7f44b80c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 --2- 172.21.15.113:0/2355101989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 0x7f44b80c30b0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009870 tx=0x7f44a001ed50 comp rx=0 tx=0).stop 2024-03-04T08:10:05.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 -- 172.21.15.113:0/2355101989 shutdown_connections 2024-03-04T08:10:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 --2- 172.21.15.113:0/2355101989 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 0x7f44b80c30b0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 --2- 172.21.15.113:0/2355101989 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44b8152950 0x7f44b815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 -- 172.21.15.113:0/2355101989 >> 172.21.15.113:0/2355101989 conn(0x7f44b8072f40 msgr2=0x7f44b80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 -- 172.21.15.113:0/2355101989 shutdown_connections 2024-03-04T08:10:05.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 -- 172.21.15.113:0/2355101989 wait complete. 2024-03-04T08:10:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 Processor -- start 2024-03-04T08:10:05.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 -- start start 2024-03-04T08:10:05.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44b806ed80 0x7f44b81f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:05.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 0x7f44b81f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44b8152950 0x7f44b8154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b80c9a50 con 0x7f44b806faa0 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b806fe80 con 0x7f44b806ed80 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44be600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b80c9d30 con 0x7f44b8152950 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44afe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 0x7f44b81f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44afe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 0x7f44b81f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43602/0 (socket says 172.21.15.113:43602) 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.617+0000 7f44afe00640 1 -- 172.21.15.113:0/3627739736 learned_addr learned my addr 172.21.15.113:0/3627739736 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44bd200640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44b806ed80 0x7f44b81f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44bdc00640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44b8152950 0x7f44b8154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44afe00640 1 -- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44b8152950 msgr2=0x7f44b8154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44afe00640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44b8152950 0x7f44b8154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44afe00640 1 -- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44b806ed80 msgr2=0x7f44b81f3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44afe00640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44b806ed80 0x7f44b81f3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:05.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44afe00640 1 -- 172.21.15.113:0/3627739736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b8071040 con 0x7f44b806faa0 2024-03-04T08:10:05.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44afe00640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 0x7f44b81f4210 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f44a001d400 tx=0x7f44a001baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:05.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44b5600640 1 -- 172.21.15.113:0/3627739736 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a00b5070 con 0x7f44b806faa0 2024-03-04T08:10:05.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44b5600640 1 -- 172.21.15.113:0/3627739736 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a00b0de0 con 0x7f44b806faa0 2024-03-04T08:10:05.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44be600640 1 -- 172.21.15.113:0/3627739736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b81f4840 con 0x7f44b806faa0 2024-03-04T08:10:05.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44b5600640 1 -- 172.21.15.113:0/3627739736 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a00bb670 con 0x7f44b806faa0 2024-03-04T08:10:05.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44be600640 1 -- 172.21.15.113:0/3627739736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b8153690 con 0x7f44b806faa0 2024-03-04T08:10:05.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44be600640 1 -- 172.21.15.113:0/3627739736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b8075a80 con 0x7f44b806faa0 2024-03-04T08:10:05.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.621+0000 7f44b5600640 1 -- 172.21.15.113:0/3627739736 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44a00c2060 con 0x7f44b806faa0 2024-03-04T08:10:05.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.625+0000 7f44b5600640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4494066f90 0x7f4494069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:05.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.625+0000 7f44b5600640 1 -- 172.21.15.113:0/3627739736 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f44a0122c50 con 0x7f44b806faa0 2024-03-04T08:10:05.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.625+0000 7f44bd200640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4494066f90 0x7f4494069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:05.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.625+0000 7f44bd200640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4494066f90 0x7f4494069450 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009b60 tx=0x7f44a8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:05.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.629+0000 7f44b5600640 1 -- 172.21.15.113:0/3627739736 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f44a00188b0 con 0x7f44b806faa0 2024-03-04T08:10:05.660 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:10:05.663 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:10:05.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 -- 172.21.15.113:0/3627739736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44b80c9580 con 0x7f44b806faa0 2024-03-04T08:10:05.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44b5600640 1 -- 172.21.15.113:0/3627739736 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+5822 (secure 0 0 0) 0x7f44a00f3400 con 0x7f44b806faa0 2024-03-04T08:10:05.892 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:05.892 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":134,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:01.097078+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:resolve","state_seq":30,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269]},"id":1}]} 2024-03-04T08:10:05.892 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 134 2024-03-04T08:10:05.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 -- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4494066f90 msgr2=0x7f4494069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4494066f90 0x7f4494069450 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009b60 tx=0x7f44a8009190 comp rx=0 tx=0).stop 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 -- 172.21.15.113:0/3627739736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 msgr2=0x7f44b81f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 0x7f44b81f4210 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f44a001d400 tx=0x7f44a001baf0 comp rx=0 tx=0).stop 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 -- 172.21.15.113:0/3627739736 shutdown_connections 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f44b8152950 0x7f44b8154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4494066f90 0x7f4494069450 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f44b806faa0 0x7f44b81f4210 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 --2- 172.21.15.113:0/3627739736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44b806ed80 0x7f44b81f3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 -- 172.21.15.113:0/3627739736 >> 172.21.15.113:0/3627739736 conn(0x7f44b8072f40 msgr2=0x7f44b80c5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 -- 172.21.15.113:0/3627739736 shutdown_connections 2024-03-04T08:10:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:05.893+0000 7f44be600640 1 -- 172.21.15.113:0/3627739736 wait complete. 2024-03-04T08:10:05.912 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:10:07.913 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:08.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.093+0000 7f1e81a00640 1 Processor -- start 2024-03-04T08:10:08.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.093+0000 7f1e81a00640 1 -- start start 2024-03-04T08:10:08.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e81a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7c15bb40 0x7f1e7c06faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:08.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e81a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7c0c4ce0 con 0x7f1e7c152950 2024-03-04T08:10:08.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e81a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7c0c4f80 con 0x7f1e7c0cb540 2024-03-04T08:10:08.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e81a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7c0c8370 con 0x7f1e7c15bb40 2024-03-04T08:10:08.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e7be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1e7c152950 0x7f1e7c06ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47460/0 (socket says 172.21.15.113:47460) 2024-03-04T08:10:08.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e7be00640 1 -- 172.21.15.113:0/4114799988 learned_addr learned my addr 172.21.15.113:0/4114799988 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:10:08.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e81000640 1 --2- 172.21.15.113:0/4114799988 >> v2:172.21.15.113:3300/0 conn(0x7f1e7c15bb40 0x7f1e7c06faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:08.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e7b400640 1 -- 172.21.15.113:0/4114799988 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193682232 0 0) 0x7f1e7c0c4ce0 con 0x7f1e7c152950 2024-03-04T08:10:08.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e7b400640 1 -- 172.21.15.113:0/4114799988 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e60003530 con 0x7f1e7c152950 2024-03-04T08:10:08.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e81000640 1 -- 172.21.15.113:0/4114799988 >> v1:172.21.15.136:6789/0 conn(0x7f1e7c0cb540 legacy=0x7f1e7c160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:08.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e81000640 1 -- 172.21.15.113:0/4114799988 >> v1:172.21.15.113:6789/0 conn(0x7f1e7c152950 legacy=0x7f1e7c06ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:08.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.097+0000 7f1e81000640 1 -- 172.21.15.113:0/4114799988 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c0cb920 con 0x7f1e7c15bb40 2024-03-04T08:10:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e81000640 1 --2- 172.21.15.113:0/4114799988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c15bb40 0x7f1e7c06faa0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009d90 tx=0x7f1e7401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398d7cd889c99e6a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e7b400640 1 -- 172.21.15.113:0/4114799988 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e740b4070 con 0x7f1e7c15bb40 2024-03-04T08:10:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e7b400640 1 -- 172.21.15.113:0/4114799988 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e740af600 con 0x7f1e7c15bb40 2024-03-04T08:10:08.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e7b400640 1 -- 172.21.15.113:0/4114799988 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e740af9d0 con 0x7f1e7c15bb40 2024-03-04T08:10:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e81a00640 1 -- 172.21.15.113:0/4114799988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c15bb40 msgr2=0x7f1e7c06faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e81a00640 1 --2- 172.21.15.113:0/4114799988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c15bb40 0x7f1e7c06faa0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009d90 tx=0x7f1e7401f150 comp rx=0 tx=0).stop 2024-03-04T08:10:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e81a00640 1 -- 172.21.15.113:0/4114799988 shutdown_connections 2024-03-04T08:10:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e81a00640 1 --2- 172.21.15.113:0/4114799988 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c15bb40 0x7f1e7c06faa0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:08.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e81a00640 1 -- 172.21.15.113:0/4114799988 >> 172.21.15.113:0/4114799988 conn(0x7f1e7c072f40 msgr2=0x7f1e7c0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:08.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e81a00640 1 -- 172.21.15.113:0/4114799988 shutdown_connections 2024-03-04T08:10:08.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.101+0000 7f1e81a00640 1 -- 172.21.15.113:0/4114799988 wait complete. 2024-03-04T08:10:08.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81a00640 1 Processor -- start 2024-03-04T08:10:08.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81a00640 1 -- start start 2024-03-04T08:10:08.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c0cb540 0x7f1e7c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:08.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e7c152950 0x7f1e7c15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:08.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e7c15bb40 0x7f1e7c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:08.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c0c9a50 con 0x7f1e7c0cb540 2024-03-04T08:10:08.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c0ca0a0 con 0x7f1e7c15bb40 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c0c9d30 con 0x7f1e7c152950 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e7be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c0cb540 0x7f1e7c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e7be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c0cb540 0x7f1e7c15b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36470/0 (socket says 172.21.15.113:36470) 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e7be00640 1 -- 172.21.15.113:0/2935135615 learned_addr learned my addr 172.21.15.113:0/2935135615 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81000640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e7c15bb40 0x7f1e7c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e73400640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e7c152950 0x7f1e7c15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e7be00640 1 -- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e7c152950 msgr2=0x7f1e7c15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e7be00640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e7c152950 0x7f1e7c15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e7be00640 1 -- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e7c15bb40 msgr2=0x7f1e7c1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e7be00640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e7c15bb40 0x7f1e7c1599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e7be00640 1 -- 172.21.15.113:0/2935135615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c154e00 con 0x7f1e7c0cb540 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e7be00640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c0cb540 0x7f1e7c15b280 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c4d0 tx=0x7f1e6c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:08.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e78c00640 1 -- 172.21.15.113:0/2935135615 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c017070 con 0x7f1e7c0cb540 2024-03-04T08:10:08.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e78c00640 1 -- 172.21.15.113:0/2935135615 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c005cf0 con 0x7f1e7c0cb540 2024-03-04T08:10:08.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e78c00640 1 -- 172.21.15.113:0/2935135615 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c007270 con 0x7f1e7c0cb540 2024-03-04T08:10:08.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81a00640 1 -- 172.21.15.113:0/2935135615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c1550e0 con 0x7f1e7c0cb540 2024-03-04T08:10:08.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.105+0000 7f1e81a00640 1 -- 172.21.15.113:0/2935135615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c1556a0 con 0x7f1e7c0cb540 2024-03-04T08:10:08.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.109+0000 7f1e81a00640 1 -- 172.21.15.113:0/2935135615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7c075a80 con 0x7f1e7c0cb540 2024-03-04T08:10:08.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.109+0000 7f1e78c00640 1 -- 172.21.15.113:0/2935135615 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e6c01a070 con 0x7f1e7c0cb540 2024-03-04T08:10:08.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.109+0000 7f1e78c00640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e58066f60 0x7f1e58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:08.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.109+0000 7f1e78c00640 1 -- 172.21.15.113:0/2935135615 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f1e6c084ac0 con 0x7f1e7c0cb540 2024-03-04T08:10:08.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.109+0000 7f1e73400640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e58066f60 0x7f1e58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:08.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.109+0000 7f1e73400640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e58066f60 0x7f1e58069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c156070 tx=0x7f1e64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:08.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.113+0000 7f1e78c00640 1 -- 172.21.15.113:0/2935135615 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1e6c054d40 con 0x7f1e7c0cb540 2024-03-04T08:10:08.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.377+0000 7f1e81a00640 1 -- 172.21.15.113:0/2935135615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e7c0c9580 con 0x7f1e7c0cb540 2024-03-04T08:10:08.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.377+0000 7f1e78c00640 1 -- 172.21.15.113:0/2935135615 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f1e6c058de0 con 0x7f1e7c0cb540 2024-03-04T08:10:08.377 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:08.377 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":135,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:06.290097+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:reconnect","state_seq":32,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7269,7389]},"id":1}]} 2024-03-04T08:10:08.377 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 135 2024-03-04T08:10:08.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 -- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e58066f60 msgr2=0x7f1e58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:08.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e58066f60 0x7f1e58069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c156070 tx=0x7f1e64009040 comp rx=0 tx=0).stop 2024-03-04T08:10:08.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 -- 172.21.15.113:0/2935135615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c0cb540 msgr2=0x7f1e7c15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:08.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c0cb540 0x7f1e7c15b280 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c00c4d0 tx=0x7f1e6c00c990 comp rx=0 tx=0).stop 2024-03-04T08:10:08.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 -- 172.21.15.113:0/2935135615 shutdown_connections 2024-03-04T08:10:08.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e7c15bb40 0x7f1e7c1599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:08.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e7c152950 0x7f1e7c15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:08.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e58066f60 0x7f1e58069420 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:08.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 --2- 172.21.15.113:0/2935135615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e7c0cb540 0x7f1e7c15b280 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:08.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 -- 172.21.15.113:0/2935135615 >> 172.21.15.113:0/2935135615 conn(0x7f1e7c072f40 msgr2=0x7f1e7c158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:08.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 -- 172.21.15.113:0/2935135615 shutdown_connections 2024-03-04T08:10:08.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:08.381+0000 7f1e81a00640 1 -- 172.21.15.113:0/2935135615 wait complete. 2024-03-04T08:10:08.399 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:10:10.400 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:10.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.537+0000 7ff1a5600640 1 Processor -- start 2024-03-04T08:10:10.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.537+0000 7ff1a5600640 1 -- start start 2024-03-04T08:10:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.537+0000 7ff1a5600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0152bd0 0x7ff1a0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.537+0000 7ff1a5600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a01591f0 con 0x7ff1a01527f0 2024-03-04T08:10:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.537+0000 7ff1a5600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a01594d0 con 0x7ff1a00c8200 2024-03-04T08:10:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.537+0000 7ff1a5600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a0071df0 con 0x7ff1a0152bd0 2024-03-04T08:10:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19ea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff1a01527f0 0x7ff1a0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47474/0 (socket says 172.21.15.113:47474) 2024-03-04T08:10:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19ea00640 1 -- 172.21.15.113:0/3159341696 learned_addr learned my addr 172.21.15.113:0/3159341696 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:10:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19f400640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0152bd0 0x7ff1a0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356048674 0 0) 0x7ff1a01591f0 con 0x7ff1a01527f0 2024-03-04T08:10:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff184003530 con 0x7ff1a01527f0 2024-03-04T08:10:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2213353243 0 0) 0x7ff184003530 con 0x7ff1a01527f0 2024-03-04T08:10:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0152bd0 msgr2=0x7ff1a0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0152bd0 0x7ff1a0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 >> v1:172.21.15.136:6789/0 conn(0x7ff1a00c8200 legacy=0x7ff1a01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a00717a0 con 0x7ff1a01527f0 2024-03-04T08:10:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7ff190001680 con 0x7ff1a01527f0 2024-03-04T08:10:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 >> v1:172.21.15.113:6789/0 conn(0x7ff1a01527f0 legacy=0x7ff1a0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1840039f0 0x7ff184003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:10.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0152bd0 0x7ff184004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff184004d50 0x7ff1840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a0071df0 con 0x7ff184004d50 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18400ba80 con 0x7ff1840039f0 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18400bc50 con 0x7ff1a0152bd0 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19f400640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff184004d50 0x7ff1840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19ea00640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1840039f0 0x7ff184003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff197e00640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0152bd0 0x7ff184004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19f400640 1 -- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0152bd0 msgr2=0x7ff184004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19f400640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0152bd0 0x7ff184004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19f400640 1 -- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1840039f0 msgr2=0x7ff184003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19f400640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1840039f0 0x7ff184003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:10.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.541+0000 7ff19f400640 1 -- 172.21.15.113:0/3159341696 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff184003530 con 0x7ff184004d50 2024-03-04T08:10:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff19f400640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff184004d50 0x7ff1840051d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff19800e170 tx=0x7ff19801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4eb58e41add3f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980b2070 con 0x7ff184004d50 2024-03-04T08:10:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1980a5c70 con 0x7ff184004d50 2024-03-04T08:10:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff19e000640 1 -- 172.21.15.113:0/3159341696 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198009c10 con 0x7ff184004d50 2024-03-04T08:10:10.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff1a5600640 1 -- 172.21.15.113:0/3159341696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff184004d50 msgr2=0x7ff1840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:10.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff1a5600640 1 --2- 172.21.15.113:0/3159341696 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff184004d50 0x7ff1840051d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff19800e170 tx=0x7ff19801cc60 comp rx=0 tx=0).stop 2024-03-04T08:10:10.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff19f400640 1 -- 172.21.15.113:0/3159341696 reap_dead start 2024-03-04T08:10:10.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff1a5600640 1 -- 172.21.15.113:0/3159341696 shutdown_connections 2024-03-04T08:10:10.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff1a5600640 1 -- 172.21.15.113:0/3159341696 >> 172.21.15.113:0/3159341696 conn(0x7ff1a006e820 msgr2=0x7ff1a006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:10.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff1a5600640 1 -- 172.21.15.113:0/3159341696 shutdown_connections 2024-03-04T08:10:10.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.545+0000 7ff1a5600640 1 -- 172.21.15.113:0/3159341696 wait complete. 2024-03-04T08:10:10.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff1a5600640 1 Processor -- start 2024-03-04T08:10:10.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff1a5600640 1 -- start start 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff1a5600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184004d50 0x7ff1a00cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff1a5600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0159d70 0x7ff1a015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff1a5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a014bab0 0x7ff1a014bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff1a5600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a0158ee0 con 0x7ff1a014bab0 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff1a5600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a01597b0 con 0x7ff184004d50 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff1a5600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a0070d40 con 0x7ff1a0159d70 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff19f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a014bab0 0x7ff1a014bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff19f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a014bab0 0x7ff1a014bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36480/0 (socket says 172.21.15.113:36480) 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff19f400640 1 -- 172.21.15.113:0/3909914130 learned_addr learned my addr 172.21.15.113:0/3909914130 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff197e00640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0159d70 0x7ff1a015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff19ea00640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184004d50 0x7ff1a00cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:10.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff19f400640 1 -- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0159d70 msgr2=0x7ff1a015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff19f400640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0159d70 0x7ff1a015a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff19f400640 1 -- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184004d50 msgr2=0x7ff1a00cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff19f400640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184004d50 0x7ff1a00cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff19f400640 1 -- 172.21.15.113:0/3909914130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a015c8f0 con 0x7ff1a014bab0 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff197e00640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0159d70 0x7ff1a015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff19f400640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a014bab0 0x7ff1a014bf30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff19800e170 tx=0x7ff198009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff196a00640 1 -- 172.21.15.113:0/3909914130 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980b2040 con 0x7ff1a014bab0 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff196a00640 1 -- 172.21.15.113:0/3909914130 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1980a5a70 con 0x7ff1a014bab0 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff196a00640 1 -- 172.21.15.113:0/3909914130 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190007e60 con 0x7ff1a014bab0 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff1a5600640 1 -- 172.21.15.113:0/3909914130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a0155e90 con 0x7ff1a014bab0 2024-03-04T08:10:10.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.549+0000 7ff1a5600640 1 -- 172.21.15.113:0/3909914130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a015b6c0 con 0x7ff1a014bab0 2024-03-04T08:10:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.553+0000 7ff1a5600640 1 -- 172.21.15.113:0/3909914130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a0075c00 con 0x7ff1a014bab0 2024-03-04T08:10:10.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.553+0000 7ff196a00640 1 -- 172.21.15.113:0/3909914130 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1980be020 con 0x7ff1a014bab0 2024-03-04T08:10:10.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.553+0000 7ff196a00640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff17c066f60 0x7ff17c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:10.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.553+0000 7ff196a00640 1 -- 172.21.15.113:0/3909914130 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7ff19811f360 con 0x7ff1a014bab0 2024-03-04T08:10:10.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.553+0000 7ff19ea00640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff17c066f60 0x7ff17c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:10.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.557+0000 7ff19ea00640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff17c066f60 0x7ff17c069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff1900078f0 tx=0x7ff190001de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:10.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.557+0000 7ff196a00640 1 -- 172.21.15.113:0/3909914130 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff1980c5020 con 0x7ff1a014bab0 2024-03-04T08:10:10.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.749+0000 7ff1a5600640 1 -- 172.21.15.113:0/3909914130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1a00c79d0 con 0x7ff1a014bab0 2024-03-04T08:10:10.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.749+0000 7ff196a00640 1 -- 172.21.15.113:0/3909914130 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+5824 (secure 0 0 0) 0x7ff1980a5e20 con 0x7ff1a014bab0 2024-03-04T08:10:10.750 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:10.750 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":135,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:06.290097+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:reconnect","state_seq":32,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7269,7389]},"id":1}]} 2024-03-04T08:10:10.750 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 135 2024-03-04T08:10:10.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 -- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff17c066f60 msgr2=0x7ff17c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:10.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff17c066f60 0x7ff17c069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff1900078f0 tx=0x7ff190001de0 comp rx=0 tx=0).stop 2024-03-04T08:10:10.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 -- 172.21.15.113:0/3909914130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a014bab0 msgr2=0x7ff1a014bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:10.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a014bab0 0x7ff1a014bf30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7ff19800e170 tx=0x7ff198009750 comp rx=0 tx=0).stop 2024-03-04T08:10:10.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 -- 172.21.15.113:0/3909914130 shutdown_connections 2024-03-04T08:10:10.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff17c066f60 0x7ff17c069420 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:10.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1a014bab0 0x7ff1a014bf30 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:10.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff1a0159d70 0x7ff1a015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:10.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 --2- 172.21.15.113:0/3909914130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff184004d50 0x7ff1a00cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:10.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 -- 172.21.15.113:0/3909914130 >> 172.21.15.113:0/3909914130 conn(0x7ff1a006e820 msgr2=0x7ff1a0156640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:10.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 -- 172.21.15.113:0/3909914130 shutdown_connections 2024-03-04T08:10:10.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:10.753+0000 7ff1a5600640 1 -- 172.21.15.113:0/3909914130 wait complete. 2024-03-04T08:10:10.772 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 135, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 135, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7269": { "addr": "172.21.15.113:6837/1401330284", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 1401330284, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 1401330284, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7269, "incarnation": 120, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7389": { "addr": "172.21.15.113:6836/1310057122", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1310057122, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1310057122, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7389, "incarnation": 126, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7519": { "addr": "172.21.15.136:6835/247269911", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 247269911, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 247269911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7519, "incarnation": 133, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:reconnect", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 68, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:10:06.290097+0000", "qdb_cluster": [ 7269, 7389 ], "qdb_leader": 7389, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7389, "mds_1": 7519, "mds_2": 7269 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:10:12.772 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:12.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.921+0000 7f1029200640 1 Processor -- start 2024-03-04T08:10:12.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.921+0000 7f1029200640 1 -- start start 2024-03-04T08:10:12.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.921+0000 7f1029200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f102406c170 0x7f10240c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:12.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.921+0000 7f1029200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10240c9ce0 0x7f10240ca130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:12.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.921+0000 7f1029200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10240ca670 con 0x7f1024157f80 2024-03-04T08:10:12.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.921+0000 7f1029200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10240c8fd0 con 0x7f102406c170 2024-03-04T08:10:12.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.921+0000 7f1029200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102406e1e0 con 0x7f10240c9ce0 2024-03-04T08:10:12.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.925+0000 7f101be00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f102406c170 0x7f10240c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:12.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.925+0000 7f1023e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10240c9ce0 0x7f10240ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:12.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.925+0000 7f101be00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f102406c170 0x7f10240c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:57458/0 (socket says 172.21.15.113:57458) 2024-03-04T08:10:12.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.925+0000 7f101be00640 1 -- 172.21.15.113:0/3698917477 learned_addr learned my addr 172.21.15.113:0/3698917477 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:12.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.925+0000 7f101be00640 1 -- 172.21.15.113:0/3698917477 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10240c9ce0 msgr2=0x7f10240ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:12.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.925+0000 7f101be00640 1 --2- 172.21.15.113:0/3698917477 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10240c9ce0 0x7f10240ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:12.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.925+0000 7f101be00640 1 -- 172.21.15.113:0/3698917477 >> v1:172.21.15.136:6789/0 conn(0x7f1024157f80 legacy=0x7f102406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:12.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.925+0000 7f101be00640 1 -- 172.21.15.113:0/3698917477 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102406c5b0 con 0x7f102406c170 2024-03-04T08:10:12.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.925+0000 7f1023e00640 1 --2- 172.21.15.113:0/3698917477 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10240c9ce0 0x7f10240ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:10:12.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f101be00640 1 --2- 172.21.15.113:0/3698917477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f102406c170 0x7f10240c8a90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f100c0098a0 tx=0x7f100c01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef167eb69f5bcba2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:12.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1022a00640 1 -- 172.21.15.113:0/3698917477 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c0b0070 con 0x7f102406c170 2024-03-04T08:10:12.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1022a00640 1 -- 172.21.15.113:0/3698917477 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c0ab600 con 0x7f102406c170 2024-03-04T08:10:12.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1022a00640 1 -- 172.21.15.113:0/3698917477 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c0ab9d0 con 0x7f102406c170 2024-03-04T08:10:12.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 -- 172.21.15.113:0/3698917477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f102406c170 msgr2=0x7f10240c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:12.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 --2- 172.21.15.113:0/3698917477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f102406c170 0x7f10240c8a90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f100c0098a0 tx=0x7f100c01c540 comp rx=0 tx=0).stop 2024-03-04T08:10:12.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 -- 172.21.15.113:0/3698917477 shutdown_connections 2024-03-04T08:10:12.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 --2- 172.21.15.113:0/3698917477 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f10240c9ce0 0x7f10240ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:12.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 --2- 172.21.15.113:0/3698917477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f102406c170 0x7f10240c8a90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:12.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 -- 172.21.15.113:0/3698917477 >> 172.21.15.113:0/3698917477 conn(0x7f1024072f40 msgr2=0x7f10240706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:12.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 -- 172.21.15.113:0/3698917477 shutdown_connections 2024-03-04T08:10:12.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 -- 172.21.15.113:0/3698917477 wait complete. 2024-03-04T08:10:12.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 Processor -- start 2024-03-04T08:10:12.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 -- start start 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f102406c170 0x7f10240c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10240c9ce0 0x7f10240c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1024157f80 0x7f102414b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1024158360 con 0x7f1024157f80 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102406dc00 con 0x7f10240c9ce0 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.929+0000 7f1029200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102406fc30 con 0x7f102406c170 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1023e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1024157f80 0x7f102414b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1023e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1024157f80 0x7f102414b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36484/0 (socket says 172.21.15.113:36484) 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1023e00640 1 -- 172.21.15.113:0/3157260027 learned_addr learned my addr 172.21.15.113:0/3157260027 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1023400640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f102406c170 0x7f10240c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:12.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f101be00640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10240c9ce0 0x7f10240c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:12.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1023e00640 1 -- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f102406c170 msgr2=0x7f10240c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:12.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1023e00640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f102406c170 0x7f10240c56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:12.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1023e00640 1 -- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10240c9ce0 msgr2=0x7f10240c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:12.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1023e00640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10240c9ce0 0x7f10240c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:12.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1023e00640 1 -- 172.21.15.113:0/3157260027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102414a380 con 0x7f1024157f80 2024-03-04T08:10:12.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1023e00640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1024157f80 0x7f102414b5b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f101c00efc0 tx=0x7f101c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:12.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f101b400640 1 -- 172.21.15.113:0/3157260027 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c018030 con 0x7f1024157f80 2024-03-04T08:10:12.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f101b400640 1 -- 172.21.15.113:0/3157260027 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c00cde0 con 0x7f1024157f80 2024-03-04T08:10:12.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f101b400640 1 -- 172.21.15.113:0/3157260027 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c003bc0 con 0x7f1024157f80 2024-03-04T08:10:12.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1029200640 1 -- 172.21.15.113:0/3157260027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1024149f60 con 0x7f1024157f80 2024-03-04T08:10:12.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1029200640 1 -- 172.21.15.113:0/3157260027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102414aad0 con 0x7f1024157f80 2024-03-04T08:10:12.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f1029200640 1 -- 172.21.15.113:0/3157260027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1024149a00 con 0x7f1024157f80 2024-03-04T08:10:12.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.933+0000 7f101b400640 1 -- 172.21.15.113:0/3157260027 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f101c01b070 con 0x7f1024157f80 2024-03-04T08:10:12.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.937+0000 7f101b400640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1000066f90 0x7f1000069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:12.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.937+0000 7f101b400640 1 -- 172.21.15.113:0/3157260027 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f101c0860b0 con 0x7f1024157f80 2024-03-04T08:10:12.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.937+0000 7f1023400640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1000066f90 0x7f1000069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:12.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.937+0000 7f1023400640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1000066f90 0x7f1000069450 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1014001770 tx=0x7f1014009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:12.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:12.941+0000 7f101b400640 1 -- 172.21.15.113:0/3157260027 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1024149a00 con 0x7f1024157f80 2024-03-04T08:10:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 -- 172.21.15.113:0/3157260027 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10240c97b0 con 0x7f1024157f80 2024-03-04T08:10:13.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f101b400640 1 -- 172.21.15.113:0/3157260027 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+5821 (secure 0 0 0) 0x7f101c022070 con 0x7f1024157f80 2024-03-04T08:10:13.148 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:13.148 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":136,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:11.314881+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:rejoin","state_seq":34,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269]},"id":1}]} 2024-03-04T08:10:13.148 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 136 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 -- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1000066f90 msgr2=0x7f1000069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1000066f90 0x7f1000069450 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1014001770 tx=0x7f1014009040 comp rx=0 tx=0).stop 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 -- 172.21.15.113:0/3157260027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1024157f80 msgr2=0x7f102414b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1024157f80 0x7f102414b5b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f101c00efc0 tx=0x7f101c009f90 comp rx=0 tx=0).stop 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 -- 172.21.15.113:0/3157260027 shutdown_connections 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1000066f90 0x7f1000069450 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1024157f80 0x7f102414b5b0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10240c9ce0 0x7f10240c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 --2- 172.21.15.113:0/3157260027 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f102406c170 0x7f10240c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 -- 172.21.15.113:0/3157260027 >> 172.21.15.113:0/3157260027 conn(0x7f1024072f40 msgr2=0x7f10240c30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 -- 172.21.15.113:0/3157260027 shutdown_connections 2024-03-04T08:10:13.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:13.149+0000 7f1029200640 1 -- 172.21.15.113:0/3157260027 wait complete. 2024-03-04T08:10:13.170 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:10:15.171 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:15.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.309+0000 7f0657c00640 1 Processor -- start 2024-03-04T08:10:15.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.309+0000 7f0657c00640 1 -- start start 2024-03-04T08:10:15.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0657c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0650154c00 0x7f06500c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:15.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0657c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0650154fe0 0x7f06500c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:15.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0657c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06500c7d80 con 0x7f06500c7700 2024-03-04T08:10:15.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0657c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065015efb0 con 0x7f0650154c00 2024-03-04T08:10:15.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0657c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06500c6b70 con 0x7f0650154fe0 2024-03-04T08:10:15.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0655200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0650154c00 0x7f06500c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:15.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0655200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0650154c00 0x7f06500c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36498/0 (socket says 172.21.15.113:36498) 2024-03-04T08:10:15.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0655200640 1 -- 172.21.15.113:0/164106050 learned_addr learned my addr 172.21.15.113:0/164106050 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:15.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f064fe00640 1 --2- 172.21.15.113:0/164106050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0650154fe0 0x7f06500c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0655200640 1 -- 172.21.15.113:0/164106050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0650154fe0 msgr2=0x7f06500c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0655200640 1 --2- 172.21.15.113:0/164106050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0650154fe0 0x7f06500c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0655200640 1 -- 172.21.15.113:0/164106050 >> v1:172.21.15.136:6789/0 conn(0x7f06500c7700 legacy=0x7f065015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:15.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.313+0000 7f0655200640 1 -- 172.21.15.113:0/164106050 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06500c8060 con 0x7f0650154c00 2024-03-04T08:10:15.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f0655200640 1 --2- 172.21.15.113:0/164106050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0650154c00 0x7f06500c1190 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f0644009870 tx=0x7f064401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616bcf24542af894 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:15.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f064f400640 1 -- 172.21.15.113:0/164106050 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06440b0070 con 0x7f0650154c00 2024-03-04T08:10:15.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f064f400640 1 -- 172.21.15.113:0/164106050 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06440ab600 con 0x7f0650154c00 2024-03-04T08:10:15.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f064f400640 1 -- 172.21.15.113:0/164106050 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06440ab9d0 con 0x7f0650154c00 2024-03-04T08:10:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f0657c00640 1 -- 172.21.15.113:0/164106050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0650154c00 msgr2=0x7f06500c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f0657c00640 1 --2- 172.21.15.113:0/164106050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0650154c00 0x7f06500c1190 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f0644009870 tx=0x7f064401c490 comp rx=0 tx=0).stop 2024-03-04T08:10:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f0657c00640 1 -- 172.21.15.113:0/164106050 shutdown_connections 2024-03-04T08:10:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f0657c00640 1 --2- 172.21.15.113:0/164106050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0650154fe0 0x7f06500c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f0657c00640 1 --2- 172.21.15.113:0/164106050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0650154c00 0x7f06500c1190 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f0657c00640 1 -- 172.21.15.113:0/164106050 >> 172.21.15.113:0/164106050 conn(0x7f065006e820 msgr2=0x7f065006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f0657c00640 1 -- 172.21.15.113:0/164106050 shutdown_connections 2024-03-04T08:10:15.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f0657c00640 1 -- 172.21.15.113:0/164106050 wait complete. 2024-03-04T08:10:15.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.317+0000 7f0657c00640 1 Processor -- start 2024-03-04T08:10:15.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0657c00640 1 -- start start 2024-03-04T08:10:15.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0657c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06500c7700 0x7f06500c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:15.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0657c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0650154c00 0x7f065015d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:15.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0657c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0650154fe0 0x7f06500c3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:15.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0657c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0650167c10 con 0x7f06500c7700 2024-03-04T08:10:15.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0657c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06500c2320 con 0x7f0650154fe0 2024-03-04T08:10:15.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0657c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0650167d90 con 0x7f0650154c00 2024-03-04T08:10:15.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0655200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06500c7700 0x7f06500c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:15.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0655200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06500c7700 0x7f06500c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36500/0 (socket says 172.21.15.113:36500) 2024-03-04T08:10:15.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0655200640 1 -- 172.21.15.113:0/2161179495 learned_addr learned my addr 172.21.15.113:0/2161179495 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f064fe00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0650154c00 0x7f065015d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0655c00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0650154fe0 0x7f06500c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0655200640 1 -- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0650154c00 msgr2=0x7f065015d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0655200640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0650154c00 0x7f065015d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0655200640 1 -- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0650154fe0 msgr2=0x7f06500c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0655200640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0650154fe0 0x7f06500c3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0655200640 1 -- 172.21.15.113:0/2161179495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06500cba70 con 0x7f06500c7700 2024-03-04T08:10:15.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0655200640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06500c7700 0x7f06500c5ea0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f064401cd30 tx=0x7f0644019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f064cc00640 1 -- 172.21.15.113:0/2161179495 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06440b9070 con 0x7f06500c7700 2024-03-04T08:10:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f064cc00640 1 -- 172.21.15.113:0/2161179495 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06440b4420 con 0x7f06500c7700 2024-03-04T08:10:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0657c00640 1 -- 172.21.15.113:0/2161179495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06501f4da0 con 0x7f06500c7700 2024-03-04T08:10:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f064cc00640 1 -- 172.21.15.113:0/2161179495 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06440b0040 con 0x7f06500c7700 2024-03-04T08:10:15.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0657c00640 1 -- 172.21.15.113:0/2161179495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06500c56a0 con 0x7f06500c7700 2024-03-04T08:10:15.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.321+0000 7f0657c00640 1 -- 172.21.15.113:0/2161179495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0650075c00 con 0x7f06500c7700 2024-03-04T08:10:15.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.325+0000 7f064cc00640 1 -- 172.21.15.113:0/2161179495 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06440bc070 con 0x7f06500c7700 2024-03-04T08:10:15.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.325+0000 7f064cc00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0630066f60 0x7f0630069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:15.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.325+0000 7f064cc00640 1 -- 172.21.15.113:0/2161179495 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f064411e880 con 0x7f06500c7700 2024-03-04T08:10:15.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.325+0000 7f064fe00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0630066f60 0x7f0630069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:15.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.325+0000 7f064fe00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0630066f60 0x7f0630069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0650158960 tx=0x7f063c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:15.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.329+0000 7f064cc00640 1 -- 172.21.15.113:0/2161179495 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0644016110 con 0x7f06500c7700 2024-03-04T08:10:15.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.525+0000 7f0657c00640 1 -- 172.21.15.113:0/2161179495 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06500683e0 con 0x7f06500c7700 2024-03-04T08:10:15.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.525+0000 7f064cc00640 1 -- 172.21.15.113:0/2161179495 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+5821 (secure 0 0 0) 0x7f06440eeba0 con 0x7f06500c7700 2024-03-04T08:10:15.526 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:15.526 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":136,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:11.314881+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:rejoin","state_seq":34,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7269]},"id":1}]} 2024-03-04T08:10:15.526 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 136 2024-03-04T08:10:15.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 -- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0630066f60 msgr2=0x7f0630069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:15.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0630066f60 0x7f0630069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0650158960 tx=0x7f063c009190 comp rx=0 tx=0).stop 2024-03-04T08:10:15.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 -- 172.21.15.113:0/2161179495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06500c7700 msgr2=0x7f06500c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:15.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06500c7700 0x7f06500c5ea0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f064401cd30 tx=0x7f0644019230 comp rx=0 tx=0).stop 2024-03-04T08:10:15.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 -- 172.21.15.113:0/2161179495 shutdown_connections 2024-03-04T08:10:15.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0650154fe0 0x7f06500c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:15.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0650154c00 0x7f065015d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:15.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0630066f60 0x7f0630069420 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:15.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 --2- 172.21.15.113:0/2161179495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06500c7700 0x7f06500c5ea0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:15.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 -- 172.21.15.113:0/2161179495 >> 172.21.15.113:0/2161179495 conn(0x7f065006e820 msgr2=0x7f065006b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:15.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 -- 172.21.15.113:0/2161179495 shutdown_connections 2024-03-04T08:10:15.528 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:15.529+0000 7f0657c00640 1 -- 172.21.15.113:0/2161179495 wait complete. 2024-03-04T08:10:15.548 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:10:16.441 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 85 -eq 48 ] 2024-03-04T08:10:16.441 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.85 2024-03-04T08:10:16.441 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.85 2024-03-04T08:10:16.441 INFO:tasks.workunit.client.0.smithi113.stdout:v0.85 2024-03-04T08:10:16.442 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.85 2024-03-04T08:10:17.549 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:17.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.693+0000 7fddd1200640 1 Processor -- start 2024-03-04T08:10:17.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.693+0000 7fddd1200640 1 -- start start 2024-03-04T08:10:17.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.693+0000 7fddd1200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fddcc0c7f50 0x7fddcc071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:17.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.693+0000 7fddd1200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc0c4200 con 0x7fddcc154b20 2024-03-04T08:10:17.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.693+0000 7fddd1200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc15deb0 con 0x7fddcc071fc0 2024-03-04T08:10:17.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.693+0000 7fddd1200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddcc15eac0 con 0x7fddcc0c7f50 2024-03-04T08:10:17.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.693+0000 7fddcb400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fddcc154b20 0x7fddcc072c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59332/0 (socket says 172.21.15.113:59332) 2024-03-04T08:10:17.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.693+0000 7fddcb400640 1 -- 172.21.15.113:0/3460096266 learned_addr learned my addr 172.21.15.113:0/3460096266 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:10:17.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.693+0000 7fddcaa00640 1 --2- 172.21.15.113:0/3460096266 >> v2:172.21.15.136:3300/0 conn(0x7fddcc0c7f50 0x7fddcc071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:17.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.693+0000 7fddca000640 1 -- 172.21.15.113:0/3460096266 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887444813 0 0) 0x7fddcc0c4200 con 0x7fddcc154b20 2024-03-04T08:10:17.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.697+0000 7fddca000640 1 -- 172.21.15.113:0/3460096266 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb0003530 con 0x7fddcc154b20 2024-03-04T08:10:17.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.697+0000 7fddcaa00640 1 -- 172.21.15.113:0/3460096266 >> v1:172.21.15.136:6789/0 conn(0x7fddcc071fc0 legacy=0x7fddcc0c34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:17.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.697+0000 7fddca000640 1 -- 172.21.15.113:0/3460096266 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069796384 0 0) 0x7fddcc15deb0 con 0x7fddcc071fc0 2024-03-04T08:10:17.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.697+0000 7fddcaa00640 1 -- 172.21.15.113:0/3460096266 >> v1:172.21.15.113:6789/0 conn(0x7fddcc154b20 legacy=0x7fddcc072c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:17.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.697+0000 7fddcaa00640 1 -- 172.21.15.113:0/3460096266 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc15e190 con 0x7fddcc0c7f50 2024-03-04T08:10:17.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddcaa00640 1 --2- 172.21.15.113:0/3460096266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddcc0c7f50 0x7fddcc071a80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fddb80097c0 tx=0x7fddb801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ceab9e578fa8c27 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:17.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddca000640 1 -- 172.21.15.113:0/3460096266 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80b5070 con 0x7fddcc0c7f50 2024-03-04T08:10:17.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddca000640 1 -- 172.21.15.113:0/3460096266 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb80a8990 con 0x7fddcc0c7f50 2024-03-04T08:10:17.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddca000640 1 -- 172.21.15.113:0/3460096266 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80a8d40 con 0x7fddcc0c7f50 2024-03-04T08:10:17.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddd1200640 1 -- 172.21.15.113:0/3460096266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddcc0c7f50 msgr2=0x7fddcc071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:17.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddd1200640 1 --2- 172.21.15.113:0/3460096266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddcc0c7f50 0x7fddcc071a80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fddb80097c0 tx=0x7fddb801ec90 comp rx=0 tx=0).stop 2024-03-04T08:10:17.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddd1200640 1 -- 172.21.15.113:0/3460096266 shutdown_connections 2024-03-04T08:10:17.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddd1200640 1 --2- 172.21.15.113:0/3460096266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddcc0c7f50 0x7fddcc071a80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:17.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddd1200640 1 -- 172.21.15.113:0/3460096266 >> 172.21.15.113:0/3460096266 conn(0x7fddcc0cd110 msgr2=0x7fddcc0cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:17.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddd1200640 1 -- 172.21.15.113:0/3460096266 shutdown_connections 2024-03-04T08:10:17.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddd1200640 1 -- 172.21.15.113:0/3460096266 wait complete. 2024-03-04T08:10:17.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.701+0000 7fddd1200640 1 Processor -- start 2024-03-04T08:10:17.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddd1200640 1 -- start start 2024-03-04T08:10:17.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddd1200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddcc071fc0 0x7fddcc0c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:17.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddd1200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddcc0c7f50 0x7fddcc14de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:17.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddd1200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddcc154b20 0x7fddcc15c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:17.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddd1200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc154f00 con 0x7fddcc071fc0 2024-03-04T08:10:17.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddd1200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc15e750 con 0x7fddcc0c7f50 2024-03-04T08:10:17.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddd1200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc155080 con 0x7fddcc154b20 2024-03-04T08:10:17.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddcc071fc0 0x7fddcc0c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:17.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddcc071fc0 0x7fddcc0c6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33166/0 (socket says 172.21.15.113:33166) 2024-03-04T08:10:17.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcb400640 1 -- 172.21.15.113:0/910050693 learned_addr learned my addr 172.21.15.113:0/910050693 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcaa00640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddcc0c7f50 0x7fddcc14de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcb400640 1 -- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddcc154b20 msgr2=0x7fddcc15c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcbe00640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddcc154b20 0x7fddcc15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcb400640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddcc154b20 0x7fddcc15c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcb400640 1 -- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddcc0c7f50 msgr2=0x7fddcc14de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcb400640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddcc0c7f50 0x7fddcc14de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcb400640 1 -- 172.21.15.113:0/910050693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc1f36a0 con 0x7fddcc071fc0 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcaa00640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddcc0c7f50 0x7fddcc14de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcbe00640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddcc154b20 0x7fddcc15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddcb400640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddcc071fc0 0x7fddcc0c6ce0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00c3c0 tx=0x7fddbc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddc3400640 1 -- 172.21.15.113:0/910050693 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc017070 con 0x7fddcc071fc0 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddc3400640 1 -- 172.21.15.113:0/910050693 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc005cf0 con 0x7fddcc071fc0 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddc3400640 1 -- 172.21.15.113:0/910050693 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc0072f0 con 0x7fddcc071fc0 2024-03-04T08:10:17.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddd1200640 1 -- 172.21.15.113:0/910050693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddcc0c7400 con 0x7fddcc071fc0 2024-03-04T08:10:17.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.705+0000 7fddd1200640 1 -- 172.21.15.113:0/910050693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddcc1f3c20 con 0x7fddcc071fc0 2024-03-04T08:10:17.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.709+0000 7fddd1200640 1 -- 172.21.15.113:0/910050693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddcc075a80 con 0x7fddcc071fc0 2024-03-04T08:10:17.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.709+0000 7fddc3400640 1 -- 172.21.15.113:0/910050693 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fddbc01a070 con 0x7fddcc071fc0 2024-03-04T08:10:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.709+0000 7fddc3400640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fddac066f60 0x7fddac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.709+0000 7fddc3400640 1 -- 172.21.15.113:0/910050693 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7fddbc084b70 con 0x7fddcc071fc0 2024-03-04T08:10:17.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.709+0000 7fddcaa00640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fddac066f60 0x7fddac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:17.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.713+0000 7fddcaa00640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fddac066f60 0x7fddac069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fddb80b2000 tx=0x7fddb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:17.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.713+0000 7fddc3400640 1 -- 172.21.15.113:0/910050693 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fddbc055440 con 0x7fddcc071fc0 2024-03-04T08:10:17.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.941+0000 7fddd1200640 1 -- 172.21.15.113:0/910050693 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddcc0b3340 con 0x7fddcc071fc0 2024-03-04T08:10:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.941+0000 7fddc3400640 1 -- 172.21.15.113:0/910050693 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+5826 (secure 0 0 0) 0x7fddbc054d40 con 0x7fddcc071fc0 2024-03-04T08:10:17.941 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:17.941 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":137,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:16.583518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7269,7389,7519]},"id":1}]} 2024-03-04T08:10:17.942 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 137 2024-03-04T08:10:17.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 -- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fddac066f60 msgr2=0x7fddac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:17.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fddac066f60 0x7fddac069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fddb80b2000 tx=0x7fddb8008040 comp rx=0 tx=0).stop 2024-03-04T08:10:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 -- 172.21.15.113:0/910050693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddcc071fc0 msgr2=0x7fddcc0c6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddcc071fc0 0x7fddcc0c6ce0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00c3c0 tx=0x7fddbc00c880 comp rx=0 tx=0).stop 2024-03-04T08:10:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 -- 172.21.15.113:0/910050693 shutdown_connections 2024-03-04T08:10:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fddcc154b20 0x7fddcc15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fddcc0c7f50 0x7fddcc14de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fddac066f60 0x7fddac069420 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 --2- 172.21.15.113:0/910050693 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fddcc071fc0 0x7fddcc0c6ce0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 -- 172.21.15.113:0/910050693 >> 172.21.15.113:0/910050693 conn(0x7fddcc0cd110 msgr2=0x7fddcc0c8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 -- 172.21.15.113:0/910050693 shutdown_connections 2024-03-04T08:10:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:17.945+0000 7fddd1200640 1 -- 172.21.15.113:0/910050693 wait complete. 2024-03-04T08:10:17.971 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=1, replacing gid=7119 2024-03-04T08:10:17.972 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.c 2024-03-04T08:10:18.973 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-03-04T08:10:18.973 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-04T08:10:18.974 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-04T08:10:18.979 INFO:tasks.ceph.mds.c:Started 2024-03-04T08:10:18.979 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:19.082 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-03-04T08:10:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.145+0000 7f9eb9800640 1 Processor -- start 2024-03-04T08:10:19.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.145+0000 7f9eb9800640 1 -- start start 2024-03-04T08:10:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb9800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9eb41527f0 0x7f9eb4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb9800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb41591f0 con 0x7f9eb40c8200 2024-03-04T08:10:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb9800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb41594d0 con 0x7f9eb4152bd0 2024-03-04T08:10:19.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb9800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb4071df0 con 0x7f9eb41527f0 2024-03-04T08:10:19.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb2000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9eb40c8200 0x7f9eb4072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59342/0 (socket says 172.21.15.113:59342) 2024-03-04T08:10:19.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb2000640 1 -- 172.21.15.113:0/2411937070 learned_addr learned my addr 172.21.15.113:0/2411937070 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:10:19.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb2a00640 1 --2- 172.21.15.113:0/2411937070 >> v2:172.21.15.136:3300/0 conn(0x7f9eb41527f0 0x7f9eb4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:19.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb1600640 1 -- 172.21.15.113:0/2411937070 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864844603 0 0) 0x7f9eb41591f0 con 0x7f9eb40c8200 2024-03-04T08:10:19.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb1600640 1 -- 172.21.15.113:0/2411937070 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e98003530 con 0x7f9eb40c8200 2024-03-04T08:10:19.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb2a00640 1 -- 172.21.15.113:0/2411937070 >> v1:172.21.15.136:6789/0 conn(0x7f9eb4152bd0 legacy=0x7f9eb41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:19.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb2a00640 1 -- 172.21.15.113:0/2411937070 >> v1:172.21.15.113:6789/0 conn(0x7f9eb40c8200 legacy=0x7f9eb4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:19.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.149+0000 7f9eb2a00640 1 -- 172.21.15.113:0/2411937070 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb40717a0 con 0x7f9eb41527f0 2024-03-04T08:10:19.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb2a00640 1 --2- 172.21.15.113:0/2411937070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9eb41527f0 0x7f9eb4070d40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40097c0 tx=0x7f9ea401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d7ea24ad3a8b12e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:19.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb1600640 1 -- 172.21.15.113:0/2411937070 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40b1070 con 0x7f9eb41527f0 2024-03-04T08:10:19.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb1600640 1 -- 172.21.15.113:0/2411937070 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea40ac410 con 0x7f9eb41527f0 2024-03-04T08:10:19.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb1600640 1 -- 172.21.15.113:0/2411937070 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40ac7c0 con 0x7f9eb41527f0 2024-03-04T08:10:19.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb9800640 1 -- 172.21.15.113:0/2411937070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9eb41527f0 msgr2=0x7f9eb4070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:19.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb9800640 1 --2- 172.21.15.113:0/2411937070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9eb41527f0 0x7f9eb4070d40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40097c0 tx=0x7f9ea401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:10:19.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb9800640 1 -- 172.21.15.113:0/2411937070 shutdown_connections 2024-03-04T08:10:19.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb9800640 1 --2- 172.21.15.113:0/2411937070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9eb41527f0 0x7f9eb4070d40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:19.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb9800640 1 -- 172.21.15.113:0/2411937070 >> 172.21.15.113:0/2411937070 conn(0x7f9eb406e820 msgr2=0x7f9eb406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:19.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb9800640 1 -- 172.21.15.113:0/2411937070 shutdown_connections 2024-03-04T08:10:19.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb9800640 1 -- 172.21.15.113:0/2411937070 wait complete. 2024-03-04T08:10:19.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb9800640 1 Processor -- start 2024-03-04T08:10:19.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.153+0000 7f9eb9800640 1 -- start start 2024-03-04T08:10:19.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb9800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9eb40c8200 0x7f9eb40b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:19.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb41527f0 0x7f9eb414ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb9800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9eb4152bd0 0x7f9eb40b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb9800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb4166380 con 0x7f9eb41527f0 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb9800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb41597b0 con 0x7f9eb4152bd0 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb9800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb4166500 con 0x7f9eb40c8200 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb41527f0 0x7f9eb414ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9eb40c8200 0x7f9eb40b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb41527f0 0x7f9eb414ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33204/0 (socket says 172.21.15.113:33204) 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb2000640 1 -- 172.21.15.113:0/1579393051 learned_addr learned my addr 172.21.15.113:0/1579393051 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb3400640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9eb4152bd0 0x7f9eb40b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb2000640 1 -- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9eb40c8200 msgr2=0x7f9eb40b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb2000640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9eb40c8200 0x7f9eb40b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb2000640 1 -- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9eb4152bd0 msgr2=0x7f9eb40b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb2000640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9eb4152bd0 0x7f9eb40b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:19.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb2000640 1 -- 172.21.15.113:0/1579393051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb41580e0 con 0x7f9eb41527f0 2024-03-04T08:10:19.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb2000640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb41527f0 0x7f9eb414ce50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0001ec0 tx=0x7f9ea000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:19.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eaaa00640 1 -- 172.21.15.113:0/1579393051 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0017070 con 0x7f9eb41527f0 2024-03-04T08:10:19.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eaaa00640 1 -- 172.21.15.113:0/1579393051 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0005d50 con 0x7f9eb41527f0 2024-03-04T08:10:19.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eaaa00640 1 -- 172.21.15.113:0/1579393051 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0007270 con 0x7f9eb41527f0 2024-03-04T08:10:19.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb9800640 1 -- 172.21.15.113:0/1579393051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb41583c0 con 0x7f9eb41527f0 2024-03-04T08:10:19.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb9800640 1 -- 172.21.15.113:0/1579393051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb41f45b0 con 0x7f9eb41527f0 2024-03-04T08:10:19.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.157+0000 7f9eb9800640 1 -- 172.21.15.113:0/1579393051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb4075c00 con 0x7f9eb41527f0 2024-03-04T08:10:19.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.161+0000 7f9eaaa00640 1 -- 172.21.15.113:0/1579393051 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ea0007410 con 0x7f9eb41527f0 2024-03-04T08:10:19.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.161+0000 7f9eaaa00640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e94066f60 0x7f9e94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:19.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.161+0000 7f9eaaa00640 1 -- 172.21.15.113:0/1579393051 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f9ea00849a0 con 0x7f9eb41527f0 2024-03-04T08:10:19.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.161+0000 7f9eb2a00640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e94066f60 0x7f9e94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:19.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.161+0000 7f9eb2a00640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e94066f60 0x7f9e94069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ae000 tx=0x7f9ea40a6040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:19.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.165+0000 7f9eaaa00640 1 -- 172.21.15.113:0/1579393051 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9ea0054cf0 con 0x7f9eb41527f0 2024-03-04T08:10:19.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.413+0000 7f9eb9800640 1 -- 172.21.15.113:0/1579393051 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9eb406f5a0 con 0x7f9eb41527f0 2024-03-04T08:10:19.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eaaa00640 1 -- 172.21.15.113:0/1579393051 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+5826 (secure 0 0 0) 0x7f9ea0058d90 con 0x7f9eb41527f0 2024-03-04T08:10:19.415 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:19.415 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":137,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:16.583518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7269,7389,7519]},"id":1}]} 2024-03-04T08:10:19.415 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 137 2024-03-04T08:10:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 -- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e94066f60 msgr2=0x7f9e94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e94066f60 0x7f9e94069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ae000 tx=0x7f9ea40a6040 comp rx=0 tx=0).stop 2024-03-04T08:10:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 -- 172.21.15.113:0/1579393051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb41527f0 msgr2=0x7f9eb414ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb41527f0 0x7f9eb414ce50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0001ec0 tx=0x7f9ea000c960 comp rx=0 tx=0).stop 2024-03-04T08:10:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 -- 172.21.15.113:0/1579393051 shutdown_connections 2024-03-04T08:10:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9eb4152bd0 0x7f9eb40b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e94066f60 0x7f9e94069420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9eb41527f0 0x7f9eb414ce50 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 --2- 172.21.15.113:0/1579393051 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9eb40c8200 0x7f9eb40b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:19.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 -- 172.21.15.113:0/1579393051 >> 172.21.15.113:0/1579393051 conn(0x7f9eb406e820 msgr2=0x7f9eb40c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 -- 172.21.15.113:0/1579393051 shutdown_connections 2024-03-04T08:10:19.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:19.417+0000 7f9eb9800640 1 -- 172.21.15.113:0/1579393051 wait complete. 2024-03-04T08:10:19.443 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is in active, standby or standby-replay 2024-03-04T08:10:21.444 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:21.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.585+0000 7f4701c00640 1 Processor -- start 2024-03-04T08:10:21.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.585+0000 7f4701c00640 1 -- start start 2024-03-04T08:10:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f4701c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f46fc1527f0 0x7f46fc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f4701c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46fc0c8200 0x7f46fc072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f4701c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 0x7f46fc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f4701c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc071df0 con 0x7f46fc0c8200 2024-03-04T08:10:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f4701c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc1597b0 con 0x7f46fc1527f0 2024-03-04T08:10:21.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f4701c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc1591f0 con 0x7f46fc152bd0 2024-03-04T08:10:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46faa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f46fc1527f0 0x7f46fc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46fb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 0x7f46fc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46faa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f46fc1527f0 0x7f46fc070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:53058/0 (socket says 172.21.15.113:53058) 2024-03-04T08:10:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46faa00640 1 -- 172.21.15.113:0/1010225645 learned_addr learned my addr 172.21.15.113:0/1010225645 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46f2000640 1 --2- 172.21.15.113:0/1010225645 >> v2:172.21.15.113:3300/0 conn(0x7f46fc0c8200 0x7f46fc072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46faa00640 1 -- 172.21.15.113:0/1010225645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 msgr2=0x7f46fc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46faa00640 1 --2- 172.21.15.113:0/1010225645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 0x7f46fc1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46faa00640 1 -- 172.21.15.113:0/1010225645 >> v2:172.21.15.113:3300/0 conn(0x7f46fc0c8200 msgr2=0x7f46fc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:21.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46faa00640 1 --2- 172.21.15.113:0/1010225645 >> v2:172.21.15.113:3300/0 conn(0x7f46fc0c8200 0x7f46fc072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46faa00640 1 -- 172.21.15.113:0/1010225645 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc1594d0 con 0x7f46fc1527f0 2024-03-04T08:10:21.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46fb400640 1 --2- 172.21.15.113:0/1010225645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 0x7f46fc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:10:21.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.589+0000 7f46f2000640 1 --2- 172.21.15.113:0/1010225645 >> v2:172.21.15.113:3300/0 conn(0x7f46fc0c8200 0x7f46fc072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:10:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f46faa00640 1 --2- 172.21.15.113:0/1010225645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc1527f0 0x7f46fc070d40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f46fc06f400 tx=0x7f46ec01c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e68fa62aa791866b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f46fa000640 1 -- 172.21.15.113:0/1010225645 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0ab5f0 con 0x7f46fc1527f0 2024-03-04T08:10:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f46fa000640 1 -- 172.21.15.113:0/1010225645 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec0b0070 con 0x7f46fc1527f0 2024-03-04T08:10:21.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f46fa000640 1 -- 172.21.15.113:0/1010225645 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0ab9b0 con 0x7f46fc1527f0 2024-03-04T08:10:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f4701c00640 1 -- 172.21.15.113:0/1010225645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc1527f0 msgr2=0x7f46fc070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f4701c00640 1 --2- 172.21.15.113:0/1010225645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc1527f0 0x7f46fc070d40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f46fc06f400 tx=0x7f46ec01c520 comp rx=0 tx=0).stop 2024-03-04T08:10:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f4701c00640 1 -- 172.21.15.113:0/1010225645 shutdown_connections 2024-03-04T08:10:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f4701c00640 1 --2- 172.21.15.113:0/1010225645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 0x7f46fc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f4701c00640 1 --2- 172.21.15.113:0/1010225645 >> v2:172.21.15.113:3300/0 conn(0x7f46fc0c8200 0x7f46fc072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f4701c00640 1 --2- 172.21.15.113:0/1010225645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc1527f0 0x7f46fc070d40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f4701c00640 1 -- 172.21.15.113:0/1010225645 >> 172.21.15.113:0/1010225645 conn(0x7f46fc06e820 msgr2=0x7f46fc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:21.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f4701c00640 1 -- 172.21.15.113:0/1010225645 shutdown_connections 2024-03-04T08:10:21.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.593+0000 7f4701c00640 1 -- 172.21.15.113:0/1010225645 wait complete. 2024-03-04T08:10:21.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f4701c00640 1 Processor -- start 2024-03-04T08:10:21.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f4701c00640 1 -- start start 2024-03-04T08:10:21.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f4701c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46fc0c8200 0x7f46fc14b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:21.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f4701c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc1527f0 0x7f46fc0cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:21.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f4701c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 0x7f46fc14a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:21.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f4701c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15bf40 con 0x7f46fc0c8200 2024-03-04T08:10:21.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f4701c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15bdc0 con 0x7f46fc1527f0 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f4701c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15c0c0 con 0x7f46fc152bd0 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46faa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46fc0c8200 0x7f46fc14b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46faa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46fc0c8200 0x7f46fc14b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33236/0 (socket says 172.21.15.113:33236) 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46faa00640 1 -- 172.21.15.113:0/1531358344 learned_addr learned my addr 172.21.15.113:0/1531358344 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46fb400640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 0x7f46fc14a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46f2000640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc1527f0 0x7f46fc0cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46faa00640 1 -- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 msgr2=0x7f46fc14a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46faa00640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 0x7f46fc14a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46faa00640 1 -- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc1527f0 msgr2=0x7f46fc0cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46faa00640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc1527f0 0x7f46fc0cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46faa00640 1 -- 172.21.15.113:0/1531358344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc1f4d80 con 0x7f46fc0c8200 2024-03-04T08:10:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46faa00640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46fc0c8200 0x7f46fc14b2e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f46ec01abd0 tx=0x7f46ec01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46f3400640 1 -- 172.21.15.113:0/1531358344 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0be3f0 con 0x7f46fc0c8200 2024-03-04T08:10:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46f3400640 1 -- 172.21.15.113:0/1531358344 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec0be590 con 0x7f46fc0c8200 2024-03-04T08:10:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f46f3400640 1 -- 172.21.15.113:0/1531358344 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0be960 con 0x7f46fc0c8200 2024-03-04T08:10:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f4701c00640 1 -- 172.21.15.113:0/1531358344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc14b940 con 0x7f46fc0c8200 2024-03-04T08:10:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.597+0000 7f4701c00640 1 -- 172.21.15.113:0/1531358344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc1586a0 con 0x7f46fc0c8200 2024-03-04T08:10:21.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.601+0000 7f4701c00640 1 -- 172.21.15.113:0/1531358344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc075c00 con 0x7f46fc0c8200 2024-03-04T08:10:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.601+0000 7f46f3400640 1 -- 172.21.15.113:0/1531358344 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46ec0b43f0 con 0x7f46fc0c8200 2024-03-04T08:10:21.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.601+0000 7f46f3400640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 0x7f46d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.601+0000 7f46f3400640 1 -- 172.21.15.113:0/1531358344 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f46ec11f640 con 0x7f46fc0c8200 2024-03-04T08:10:21.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.601+0000 7f46f2000640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 0x7f46d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:21.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.605+0000 7f46f2000640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 0x7f46d8069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009bd0 tx=0x7f46e400d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:21.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.605+0000 7f46f3400640 1 -- 172.21.15.113:0/1531358344 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f46ec016110 con 0x7f46fc0c8200 2024-03-04T08:10:21.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.801+0000 7f4701c00640 1 -- 172.21.15.113:0/1531358344 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46fc0c9a50 con 0x7f46fc0c8200 2024-03-04T08:10:21.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.801+0000 7f46f3400640 1 -- 172.21.15.113:0/1531358344 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f46ec0ef8c0 con 0x7f46fc0c8200 2024-03-04T08:10:21.801 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:21.801 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":138,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:16.583518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389,7269]},"id":1}]} 2024-03-04T08:10:21.801 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 138 2024-03-04T08:10:21.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 -- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 msgr2=0x7f46d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:21.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 0x7f46d8069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009bd0 tx=0x7f46e400d040 comp rx=0 tx=0).stop 2024-03-04T08:10:21.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 -- 172.21.15.113:0/1531358344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46fc0c8200 msgr2=0x7f46fc14b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:21.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46fc0c8200 0x7f46fc14b2e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f46ec01abd0 tx=0x7f46ec01ad60 comp rx=0 tx=0).stop 2024-03-04T08:10:21.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 -- 172.21.15.113:0/1531358344 shutdown_connections 2024-03-04T08:10:21.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46fc152bd0 0x7f46fc14a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc1527f0 0x7f46fc0cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f46d8066f60 0x7f46d8069420 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 --2- 172.21.15.113:0/1531358344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46fc0c8200 0x7f46fc14b2e0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:21.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 -- 172.21.15.113:0/1531358344 >> 172.21.15.113:0/1531358344 conn(0x7f46fc06e820 msgr2=0x7f46fc06ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:21.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 -- 172.21.15.113:0/1531358344 shutdown_connections 2024-03-04T08:10:21.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:21.805+0000 7f4701c00640 1 -- 172.21.15.113:0/1531358344 wait complete. 2024-03-04T08:10:21.823 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-03-04T08:10:21.823 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 34 secs before thrashing 2024-03-04T08:10:30.632 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 28% (69/243) Updating files: 29% (71/243) Updating files: 30% (73/243) Updating files: 31% (76/243) Updating files: 32% (78/243) Updating files: 33% (81/243) Updating files: 34% (83/243) Updating files: 35% (86/243) Updating files: 36% (88/243) Updating files: 37% (90/243) Updating files: 38% (93/243) Updating files: 39% (95/243) Updating files: 40% (98/243) Updating files: 41% (100/243) Updating files: 42% (103/243) Updating files: 43% (105/243) Updating files: 44% (107/243) Updating files: 45% (110/243) Updating files: 46% (112/243) Updating files: 47% (115/243) Updating files: 48% (117/243) Updating files: 49% (120/243) Updating files: 50% (122/243) Updating files: 51% (124/243) Updating files: 52% (127/243) Updating files: 53% (129/243) Updating files: 54% (132/243) Updating files: 55% (134/243) Updating files: 56% (137/243) Updating files: 57% (139/243) Updating files: 58% (141/243) Updating files: 59% (144/243) Updating files: 60% (146/243) Updating files: 61% (149/243) Updating files: 62% (151/243) Updating files: 63% (154/243) Updating files: 64% (156/243) Updating files: 65% (158/243) Updating files: 66% (161/243) Updating files: 67% (163/243) Updating files: 68% (166/243) Updating files: 69% (168/243) Updating files: 69% (170/243) Updating files: 70% (171/243) Updating files: 71% (173/243) Updating files: 72% (175/243) Updating files: 73% (178/243) Updating files: 74% (180/243) Updating files: 74% (182/243) Updating files: 75% (183/243) Updating files: 76% (185/243) Updating files: 77% (188/243) Updating files: 78% (190/243) Updating files: 79% (192/243) Updating files: 80% (195/243) Updating files: 81% (197/243) Updating files: 82% (200/243) Updating files: 83% (202/243) Updating files: 84% (205/243) Updating files: 85% (207/243) Updating files: 86% (209/243) Updating files: 86% (210/243) Updating files: 87% (212/243) Updating files: 88% (214/243) Updating files: 89% (217/243) Updating files: 90% (219/243) Updating files: 91% (222/243) Updating files: 92% (224/243) Updating files: 93% (226/243) Updating files: 94% (229/243) Updating files: 95% (231/243) Updating files: 96% (234/243) Updating files: 97% (236/243) Updating files: 98% (239/243) Updating files: 99% (241/243) Updating files: 100% (243/243) Updating files: 100% (243/243), done. 2024-03-04T08:10:30.832 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at a0c22842db9 0.85 2024-03-04T08:10:30.839 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.85 2024-03-04T08:10:31.834 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 86 -eq 48 ] 2024-03-04T08:10:31.835 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.86 2024-03-04T08:10:31.835 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.86 2024-03-04T08:10:31.835 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.86 2024-03-04T08:10:31.835 INFO:tasks.workunit.client.0.smithi113.stdout:v0.86 2024-03-04T08:10:35.718 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:10:35.721 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:10:42.563 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 7% (32/447) Updating files: 8% (36/447) Updating files: 9% (41/447) Updating files: 10% (45/447) Updating files: 11% (50/447) Updating files: 12% (54/447) Updating files: 13% (59/447) Updating files: 14% (63/447) Updating files: 15% (68/447) Updating files: 16% (72/447) Updating files: 17% (76/447) Updating files: 18% (81/447) Updating files: 19% (85/447) Updating files: 20% (90/447) Updating files: 21% (94/447) Updating files: 22% (99/447) Updating files: 22% (102/447) Updating files: 23% (103/447) Updating files: 24% (108/447) Updating files: 25% (112/447) Updating files: 26% (117/447) Updating files: 27% (121/447) Updating files: 28% (126/447) Updating files: 29% (130/447) Updating files: 30% (135/447) Updating files: 31% (139/447) Updating files: 32% (144/447) Updating files: 33% (148/447) Updating files: 34% (152/447) Updating files: 35% (157/447) Updating files: 36% (161/447) Updating files: 37% (166/447) Updating files: 38% (170/447) Updating files: 39% (175/447) Updating files: 40% (179/447) Updating files: 41% (184/447) Updating files: 42% (188/447) Updating files: 43% (193/447) Updating files: 44% (197/447) Updating files: 45% (202/447) Updating files: 46% (206/447) Updating files: 46% (208/447) Updating files: 47% (211/447) Updating files: 48% (215/447) Updating files: 49% (220/447) Updating files: 50% (224/447) Updating files: 51% (228/447) Updating files: 52% (233/447) Updating files: 53% (237/447) Updating files: 54% (242/447) Updating files: 55% (246/447) Updating files: 56% (251/447) Updating files: 57% (255/447) Updating files: 58% (260/447) Updating files: 59% (264/447) Updating files: 60% (269/447) Updating files: 61% (273/447) Updating files: 62% (278/447) Updating files: 63% (282/447) Updating files: 64% (287/447) Updating files: 65% (291/447) Updating files: 66% (296/447) Updating files: 67% (300/447) Updating files: 68% (304/447) Updating files: 69% (309/447) Updating files: 70% (313/447) Updating files: 71% (318/447) Updating files: 72% (322/447) Updating files: 72% (324/447) Updating files: 73% (327/447) Updating files: 74% (331/447) Updating files: 75% (336/447) Updating files: 76% (340/447) Updating files: 77% (345/447) Updating files: 78% (349/447) Updating files: 79% (354/447) Updating files: 80% (358/447) Updating files: 81% (363/447) Updating files: 82% (367/447) Updating files: 83% (372/447) Updating files: 84% (376/447) Updating files: 85% (380/447) Updating files: 86% (385/447) Updating files: 87% (389/447) Updating files: 88% (394/447) Updating files: 89% (398/447) Updating files: 90% (403/447) Updating files: 91% (407/447) Updating files: 92% (412/447) Updating files: 93% (416/447) Updating files: 94% (421/447) Updating files: 95% (425/447) Updating files: 96% (430/447) Updating files: 97% (434/447) Updating files: 98% (439/447) Updating files: 99% (443/447) Updating files: 99% (444/447) Updating files: 100% (447/447) Updating files: 100% (447/447), done. 2024-03-04T08:10:42.647 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 97dcc0539df 0.86 2024-03-04T08:10:42.661 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.86 2024-03-04T08:10:42.982 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 87 -eq 48 ] 2024-03-04T08:10:42.982 INFO:tasks.workunit.client.0.smithi113.stdout:v0.87 2024-03-04T08:10:42.983 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.87 2024-03-04T08:10:42.983 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.87 2024-03-04T08:10:42.983 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.87 2024-03-04T08:10:46.676 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at c51c8f9d80f 0.87 2024-03-04T08:10:46.679 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.87 2024-03-04T08:10:46.899 INFO:tasks.workunit.client.0.smithi113.stdout:v0.88 2024-03-04T08:10:46.899 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 88 -eq 48 ] 2024-03-04T08:10:46.899 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.88 2024-03-04T08:10:46.899 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.88 2024-03-04T08:10:46.899 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.88 2024-03-04T08:10:55.824 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:55.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fc1e00640 1 Processor -- start 2024-03-04T08:10:55.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fc1e00640 1 -- start start 2024-03-04T08:10:55.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fc1e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8fbc0c8200 0x7f8fbc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:55.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fc1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8fbc152bd0 0x7f8fbc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:55.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fc1e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fbc1591f0 con 0x7f8fbc1527f0 2024-03-04T08:10:55.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fc1e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fbc071df0 con 0x7f8fbc0c8200 2024-03-04T08:10:55.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fc1e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc1597b0 con 0x7f8fbc152bd0 2024-03-04T08:10:55.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fb2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8fbc0c8200 0x7f8fbc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:55.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fb2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8fbc0c8200 0x7f8fbc070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57500/0 (socket says 172.21.15.113:57500) 2024-03-04T08:10:55.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fb2000640 1 -- 172.21.15.113:0/3605555287 learned_addr learned my addr 172.21.15.113:0/3605555287 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:55.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fbb400640 1 --2- 172.21.15.113:0/3605555287 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8fbc152bd0 0x7f8fbc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:55.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fb2000640 1 -- 172.21.15.113:0/3605555287 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8fbc152bd0 msgr2=0x7f8fbc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:55.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fb2000640 1 --2- 172.21.15.113:0/3605555287 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8fbc152bd0 0x7f8fbc1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:55.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fb2000640 1 -- 172.21.15.113:0/3605555287 >> v1:172.21.15.136:6789/0 conn(0x7f8fbc1527f0 legacy=0x7f8fbc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:55.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.968+0000 7f8fb2000640 1 -- 172.21.15.113:0/3605555287 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc1594d0 con 0x7f8fbc0c8200 2024-03-04T08:10:55.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.972+0000 7f8fb2000640 1 --2- 172.21.15.113:0/3605555287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc0c8200 0x7f8fbc070d40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009e50 tx=0x7f8fa401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e095b9a4094a4de9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:55.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fba000640 1 -- 172.21.15.113:0/3605555287 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa40a5810 con 0x7f8fbc0c8200 2024-03-04T08:10:55.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fba000640 1 -- 172.21.15.113:0/3605555287 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa40a59b0 con 0x7f8fbc0c8200 2024-03-04T08:10:55.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fba000640 1 -- 172.21.15.113:0/3605555287 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa40a5d80 con 0x7f8fbc0c8200 2024-03-04T08:10:55.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fc1e00640 1 -- 172.21.15.113:0/3605555287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc0c8200 msgr2=0x7f8fbc070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:55.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fc1e00640 1 --2- 172.21.15.113:0/3605555287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc0c8200 0x7f8fbc070d40 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009e50 tx=0x7f8fa401ca30 comp rx=0 tx=0).stop 2024-03-04T08:10:55.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fc1e00640 1 -- 172.21.15.113:0/3605555287 shutdown_connections 2024-03-04T08:10:55.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fc1e00640 1 --2- 172.21.15.113:0/3605555287 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8fbc152bd0 0x7f8fbc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:55.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fc1e00640 1 --2- 172.21.15.113:0/3605555287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc0c8200 0x7f8fbc070d40 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:55.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fc1e00640 1 -- 172.21.15.113:0/3605555287 >> 172.21.15.113:0/3605555287 conn(0x7f8fbc06e820 msgr2=0x7f8fbc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:55.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fc1e00640 1 -- 172.21.15.113:0/3605555287 shutdown_connections 2024-03-04T08:10:55.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fc1e00640 1 -- 172.21.15.113:0/3605555287 wait complete. 2024-03-04T08:10:55.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fc1e00640 1 Processor -- start 2024-03-04T08:10:55.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.976+0000 7f8fc1e00640 1 -- start start 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fc1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8fbc0c8200 0x7f8fbc161db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fc1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fbc1527f0 0x7f8fbc0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fc1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc152bd0 0x7f8fbc162600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fc1e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc1654f0 con 0x7f8fbc152bd0 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fc1e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc165370 con 0x7f8fbc1527f0 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fc1e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc165670 con 0x7f8fbc0c8200 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc152bd0 0x7f8fbc162600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8fbc0c8200 0x7f8fbc161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fbb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc152bd0 0x7f8fbc162600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57512/0 (socket says 172.21.15.113:57512) 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fbb400640 1 -- 172.21.15.113:0/2129360315 learned_addr learned my addr 172.21.15.113:0/2129360315 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fb2000640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fbc1527f0 0x7f8fbc0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fbb400640 1 -- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8fbc0c8200 msgr2=0x7f8fbc161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fbb400640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8fbc0c8200 0x7f8fbc161db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fbb400640 1 -- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fbc1527f0 msgr2=0x7f8fbc0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fbb400640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fbc1527f0 0x7f8fbc0b1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fbb400640 1 -- 172.21.15.113:0/2129360315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc161940 con 0x7f8fbc152bd0 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fbb400640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc152bd0 0x7f8fbc162600 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400d590 tx=0x7f8fb400da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fb3e00640 1 -- 172.21.15.113:0/2129360315 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40125f0 con 0x7f8fbc152bd0 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fb3e00640 1 -- 172.21.15.113:0/2129360315 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb4019070 con 0x7f8fbc152bd0 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fb3e00640 1 -- 172.21.15.113:0/2129360315 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40129f0 con 0x7f8fbc152bd0 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fc1e00640 1 -- 172.21.15.113:0/2129360315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fbc1f4f30 con 0x7f8fbc152bd0 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fc1e00640 1 -- 172.21.15.113:0/2129360315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc161600 con 0x7f8fbc152bd0 2024-03-04T08:10:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.980+0000 7f8fc1e00640 1 -- 172.21.15.113:0/2129360315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc075c00 con 0x7f8fbc152bd0 2024-03-04T08:10:55.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.984+0000 7f8fb3e00640 1 -- 172.21.15.113:0/2129360315 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fb4012b90 con 0x7f8fbc152bd0 2024-03-04T08:10:55.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.984+0000 7f8fb3e00640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f98066f60 0x7f8f98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:55.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.984+0000 7f8fb3e00640 1 -- 172.21.15.113:0/2129360315 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f8fb4085a90 con 0x7f8fbc152bd0 2024-03-04T08:10:55.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.984+0000 7f8fbaa00640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f98066f60 0x7f8f98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:55.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.984+0000 7f8fbaa00640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f98066f60 0x7f8f98069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8fac001810 tx=0x7f8fac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:55.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:55.992+0000 7f8fb3e00640 1 -- 172.21.15.113:0/2129360315 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8fb4055c80 con 0x7f8fbc152bd0 2024-03-04T08:10:56.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 -- 172.21.15.113:0/2129360315 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fbc156f20 con 0x7f8fbc152bd0 2024-03-04T08:10:56.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fb3e00640 1 -- 172.21.15.113:0/2129360315 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f8fb4059d20 con 0x7f8fbc152bd0 2024-03-04T08:10:56.300 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:56.300 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":138,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:16.583518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389,7269]},"id":1}]} 2024-03-04T08:10:56.301 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 138 2024-03-04T08:10:56.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 -- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f98066f60 msgr2=0x7f8f98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:56.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f98066f60 0x7f8f98069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8fac001810 tx=0x7f8fac009040 comp rx=0 tx=0).stop 2024-03-04T08:10:56.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 -- 172.21.15.113:0/2129360315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc152bd0 msgr2=0x7f8fbc162600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:56.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc152bd0 0x7f8fbc162600 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400d590 tx=0x7f8fb400da50 comp rx=0 tx=0).stop 2024-03-04T08:10:56.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 -- 172.21.15.113:0/2129360315 shutdown_connections 2024-03-04T08:10:56.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f98066f60 0x7f8f98069420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8fbc152bd0 0x7f8fbc162600 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8fbc1527f0 0x7f8fbc0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 --2- 172.21.15.113:0/2129360315 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8fbc0c8200 0x7f8fbc161db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 -- 172.21.15.113:0/2129360315 >> 172.21.15.113:0/2129360315 conn(0x7f8fbc06e820 msgr2=0x7f8fbc070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:56.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 -- 172.21.15.113:0/2129360315 shutdown_connections 2024-03-04T08:10:56.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.304+0000 7f8fc1e00640 1 -- 172.21.15.113:0/2129360315 wait complete. 2024-03-04T08:10:56.320 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=2) 2024-03-04T08:10:56.321 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-03-04T08:10:56.321 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:10:56.334 INFO:tasks.ceph.mds.e:Stopped 2024-03-04T08:10:56.334 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:56.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97e0c00640 1 Processor -- start 2024-03-04T08:10:56.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97e0c00640 1 -- start start 2024-03-04T08:10:56.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97e0c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97dc0ce2d0 0x7f97dc15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:56.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97e0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce6b0 0x7f97dc06f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:56.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97e0c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97dc06de80 0x7f97dc154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:56.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97e0c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97dc15bbf0 con 0x7f97dc0ce2d0 2024-03-04T08:10:56.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97e0c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97dc06eb80 con 0x7f97dc06de80 2024-03-04T08:10:56.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97e0c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc06c210 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97d9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97dc0ce2d0 0x7f97dc15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97d9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97dc0ce2d0 0x7f97dc15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57516/0 (socket says 172.21.15.113:57516) 2024-03-04T08:10:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97d9e00640 1 -- 172.21.15.113:0/2601746537 learned_addr learned my addr 172.21.15.113:0/2601746537 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97d3e00640 1 --2- 172.21.15.113:0/2601746537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce6b0 0x7f97dc06f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.516+0000 7f97da800640 1 --2- 172.21.15.113:0/2601746537 >> v2:172.21.15.136:3300/0 conn(0x7f97dc06de80 0x7f97dc154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97d9e00640 1 -- 172.21.15.113:0/2601746537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce6b0 msgr2=0x7f97dc06f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:56.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97d9e00640 1 --2- 172.21.15.113:0/2601746537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce6b0 0x7f97dc06f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97d9e00640 1 -- 172.21.15.113:0/2601746537 >> v2:172.21.15.136:3300/0 conn(0x7f97dc06de80 msgr2=0x7f97dc154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97d9e00640 1 --2- 172.21.15.113:0/2601746537 >> v2:172.21.15.136:3300/0 conn(0x7f97dc06de80 0x7f97dc154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97d9e00640 1 -- 172.21.15.113:0/2601746537 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc06c4f0 con 0x7f97dc0ce2d0 2024-03-04T08:10:56.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97da800640 1 --2- 172.21.15.113:0/2601746537 >> v2:172.21.15.136:3300/0 conn(0x7f97dc06de80 0x7f97dc154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:10:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97d9e00640 1 --2- 172.21.15.113:0/2601746537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce2d0 0x7f97dc15b6b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0ceed0 tx=0x7f97cc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148016ecf8c23e30 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:56.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97d9400640 1 -- 172.21.15.113:0/2601746537 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0b5070 con 0x7f97dc0ce2d0 2024-03-04T08:10:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97d9400640 1 -- 172.21.15.113:0/2601746537 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc0a89d0 con 0x7f97dc0ce2d0 2024-03-04T08:10:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97d9400640 1 -- 172.21.15.113:0/2601746537 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0a8da0 con 0x7f97dc0ce2d0 2024-03-04T08:10:56.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97e0c00640 1 -- 172.21.15.113:0/2601746537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce2d0 msgr2=0x7f97dc15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.520+0000 7f97e0c00640 1 --2- 172.21.15.113:0/2601746537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce2d0 0x7f97dc15b6b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0ceed0 tx=0x7f97cc01ed10 comp rx=0 tx=0).stop 2024-03-04T08:10:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 -- 172.21.15.113:0/2601746537 shutdown_connections 2024-03-04T08:10:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 --2- 172.21.15.113:0/2601746537 >> v2:172.21.15.136:3300/0 conn(0x7f97dc06de80 0x7f97dc154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 --2- 172.21.15.113:0/2601746537 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce6b0 0x7f97dc06f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 --2- 172.21.15.113:0/2601746537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce2d0 0x7f97dc15b6b0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 -- 172.21.15.113:0/2601746537 >> 172.21.15.113:0/2601746537 conn(0x7f97dc0709a0 msgr2=0x7f97dc070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 -- 172.21.15.113:0/2601746537 shutdown_connections 2024-03-04T08:10:56.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 -- 172.21.15.113:0/2601746537 wait complete. 2024-03-04T08:10:56.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 Processor -- start 2024-03-04T08:10:56.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 -- start start 2024-03-04T08:10:56.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97dc06de80 0x7f97dc1f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:56.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce2d0 0x7f97dc1f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:56.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce6b0 0x7f97dc1474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:56.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc06c7d0 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc06e810 con 0x7f97dc06de80 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc070010 con 0x7f97dc0ce2d0 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97da800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce6b0 0x7f97dc1474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97da800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce6b0 0x7f97dc1474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57526/0 (socket says 172.21.15.113:57526) 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97da800640 1 -- 172.21.15.113:0/1822110445 learned_addr learned my addr 172.21.15.113:0/1822110445 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97d3e00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce2d0 0x7f97dc1f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97d9e00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97dc06de80 0x7f97dc1f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97da800640 1 -- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce2d0 msgr2=0x7f97dc1f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97da800640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce2d0 0x7f97dc1f1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97da800640 1 -- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97dc06de80 msgr2=0x7f97dc1f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97da800640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97dc06de80 0x7f97dc1f4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97da800640 1 -- 172.21.15.113:0/1822110445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc1f19c0 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97d3e00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce2d0 0x7f97dc1f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97da800640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce6b0 0x7f97dc1474c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f97d4007e40 tx=0x7f97d400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:56.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97d2000640 1 -- 172.21.15.113:0/1822110445 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d401a070 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97d2000640 1 -- 172.21.15.113:0/1822110445 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d4003ba0 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 -- 172.21.15.113:0/1822110445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc1f40f0 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97d2000640 1 -- 172.21.15.113:0/1822110445 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d400de80 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.524+0000 7f97e0c00640 1 -- 172.21.15.113:0/1822110445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc1f16e0 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.528+0000 7f97e0c00640 1 -- 172.21.15.113:0/1822110445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc075c00 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.528+0000 7f97d2000640 1 -- 172.21.15.113:0/1822110445 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97d401d070 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.528+0000 7f97d2000640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97b8066f90 0x7f97b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:56.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.528+0000 7f97d2000640 1 -- 172.21.15.113:0/1822110445 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f97d4085980 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.528+0000 7f97d9e00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97b8066f90 0x7f97b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:56.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.528+0000 7f97d9e00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97b8066f90 0x7f97b8069450 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0b2000 tx=0x7f97cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:56.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.532+0000 7f97d2000640 1 -- 172.21.15.113:0/1822110445 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f97d4010070 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 -- 172.21.15.113:0/1822110445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97dc06cb40 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97d2000640 1 -- 172.21.15.113:0/1822110445 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f97d40562f0 con 0x7f97dc0ce6b0 2024-03-04T08:10:56.792 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:56.792 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":138,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:16.583518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389,7269]},"id":1}]} 2024-03-04T08:10:56.792 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 138 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 -- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97b8066f90 msgr2=0x7f97b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97b8066f90 0x7f97b8069450 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0b2000 tx=0x7f97cc008040 comp rx=0 tx=0).stop 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 -- 172.21.15.113:0/1822110445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce6b0 msgr2=0x7f97dc1474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce6b0 0x7f97dc1474c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f97d4007e40 tx=0x7f97d400d4a0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 -- 172.21.15.113:0/1822110445 shutdown_connections 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97b8066f90 0x7f97b8069450 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97dc0ce6b0 0x7f97dc1474c0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97dc0ce2d0 0x7f97dc1f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 --2- 172.21.15.113:0/1822110445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97dc06de80 0x7f97dc1f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 -- 172.21.15.113:0/1822110445 >> 172.21.15.113:0/1822110445 conn(0x7f97dc0709a0 msgr2=0x7f97dc0b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 -- 172.21.15.113:0/1822110445 shutdown_connections 2024-03-04T08:10:56.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:56.796+0000 7f97e0c00640 1 -- 172.21.15.113:0/1822110445 wait complete. 2024-03-04T08:10:56.821 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:10:58.822 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:10:58.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.960+0000 7f0d2e400640 1 Processor -- start 2024-03-04T08:10:58.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.960+0000 7f0d2e400640 1 -- start start 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d2e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce2d0 0x7f0d2815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d2e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d2806de80 0x7f0d28154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d2e400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2806c210 con 0x7f0d280ce6b0 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d2e400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2815bbf0 con 0x7f0d2806de80 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d2e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2806eb80 con 0x7f0d280ce2d0 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d27e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d2806de80 0x7f0d28154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d27e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d2806de80 0x7f0d28154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58936/0 (socket says 172.21.15.113:58936) 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d27e00640 1 -- 172.21.15.113:0/986896520 learned_addr learned my addr 172.21.15.113:0/986896520 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d27400640 1 --2- 172.21.15.113:0/986896520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce2d0 0x7f0d2815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d27e00640 1 -- 172.21.15.113:0/986896520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce2d0 msgr2=0x7f0d2815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d27e00640 1 --2- 172.21.15.113:0/986896520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce2d0 0x7f0d2815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:58.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d27e00640 1 -- 172.21.15.113:0/986896520 >> v1:172.21.15.136:6789/0 conn(0x7f0d280ce6b0 legacy=0x7f0d2806f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d27e00640 1 -- 172.21.15.113:0/986896520 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2806c4f0 con 0x7f0d2806de80 2024-03-04T08:10:58.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.964+0000 7f0d27400640 1 --2- 172.21.15.113:0/986896520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce2d0 0x7f0d2815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:10:58.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d27e00640 1 --2- 172.21.15.113:0/986896520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d2806de80 0x7f0d28154b10 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009e50 tx=0x7f0d2001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccd3050d30bd6cc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d26000640 1 -- 172.21.15.113:0/986896520 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d200b4070 con 0x7f0d2806de80 2024-03-04T08:10:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d26000640 1 -- 172.21.15.113:0/986896520 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d200af400 con 0x7f0d2806de80 2024-03-04T08:10:58.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d26000640 1 -- 172.21.15.113:0/986896520 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d200af7d0 con 0x7f0d2806de80 2024-03-04T08:10:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d2e400640 1 -- 172.21.15.113:0/986896520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d2806de80 msgr2=0x7f0d28154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d2e400640 1 --2- 172.21.15.113:0/986896520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d2806de80 0x7f0d28154b10 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f0d20009e50 tx=0x7f0d2001f210 comp rx=0 tx=0).stop 2024-03-04T08:10:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d2e400640 1 -- 172.21.15.113:0/986896520 shutdown_connections 2024-03-04T08:10:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d2e400640 1 --2- 172.21.15.113:0/986896520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d2806de80 0x7f0d28154b10 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d2e400640 1 --2- 172.21.15.113:0/986896520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce2d0 0x7f0d2815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d2e400640 1 -- 172.21.15.113:0/986896520 >> 172.21.15.113:0/986896520 conn(0x7f0d280709a0 msgr2=0x7f0d28070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d2e400640 1 -- 172.21.15.113:0/986896520 shutdown_connections 2024-03-04T08:10:58.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d2e400640 1 -- 172.21.15.113:0/986896520 wait complete. 2024-03-04T08:10:58.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d2e400640 1 Processor -- start 2024-03-04T08:10:58.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.968+0000 7f0d2e400640 1 -- start start 2024-03-04T08:10:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d2e400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d2806de80 0x7f0d28146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:58.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d2e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d280ce2d0 0x7f0d28143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d2e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce6b0 0x7f0d280c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d2e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2806c7d0 con 0x7f0d280ce2d0 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d2e400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2806e810 con 0x7f0d2806de80 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d2e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2806f720 con 0x7f0d280ce6b0 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d26a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d280ce2d0 0x7f0d28143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d26a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d280ce2d0 0x7f0d28143cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58950/0 (socket says 172.21.15.113:58950) 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d26a00640 1 -- 172.21.15.113:0/3508295119 learned_addr learned my addr 172.21.15.113:0/3508295119 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d27400640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d2806de80 0x7f0d28146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d27e00640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce6b0 0x7f0d280c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d26a00640 1 -- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce6b0 msgr2=0x7f0d280c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d26a00640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce6b0 0x7f0d280c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d26a00640 1 -- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d2806de80 msgr2=0x7f0d28146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:58.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d26a00640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d2806de80 0x7f0d28146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d26a00640 1 -- 172.21.15.113:0/3508295119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d280c4df0 con 0x7f0d280ce2d0 2024-03-04T08:10:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d26a00640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d280ce2d0 0x7f0d28143cd0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400c370 tx=0x7f0d1400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d1f400640 1 -- 172.21.15.113:0/3508295119 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d140052d0 con 0x7f0d280ce2d0 2024-03-04T08:10:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d1f400640 1 -- 172.21.15.113:0/3508295119 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d14005470 con 0x7f0d280ce2d0 2024-03-04T08:10:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d1f400640 1 -- 172.21.15.113:0/3508295119 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14007260 con 0x7f0d280ce2d0 2024-03-04T08:10:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d2e400640 1 -- 172.21.15.113:0/3508295119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d281505d0 con 0x7f0d280ce2d0 2024-03-04T08:10:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d2e400640 1 -- 172.21.15.113:0/3508295119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d28146eb0 con 0x7f0d280ce2d0 2024-03-04T08:10:58.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.972+0000 7f0d2e400640 1 -- 172.21.15.113:0/3508295119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d281f0850 con 0x7f0d280ce2d0 2024-03-04T08:10:58.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.976+0000 7f0d1f400640 1 -- 172.21.15.113:0/3508295119 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d14007400 con 0x7f0d280ce2d0 2024-03-04T08:10:58.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.976+0000 7f0d1f400640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d08066f90 0x7f0d08069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:10:58.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.976+0000 7f0d1f400640 1 -- 172.21.15.113:0/3508295119 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f0d14084b40 con 0x7f0d280ce2d0 2024-03-04T08:10:58.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.976+0000 7f0d27400640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d08066f90 0x7f0d08069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:10:58.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.976+0000 7f0d27400640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d08066f90 0x7f0d08069450 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009950 tx=0x7f0d18008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:10:58.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:58.980+0000 7f0d1f400640 1 -- 172.21.15.113:0/3508295119 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0d281f0850 con 0x7f0d280ce2d0 2024-03-04T08:10:59.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 -- 172.21.15.113:0/3508295119 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d2806cb40 con 0x7f0d280ce2d0 2024-03-04T08:10:59.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d1f400640 1 -- 172.21.15.113:0/3508295119 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f0d14054dd0 con 0x7f0d280ce2d0 2024-03-04T08:10:59.164 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:10:59.164 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":138,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:16.583518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389,7269]},"id":1}]} 2024-03-04T08:10:59.164 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 138 2024-03-04T08:10:59.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 -- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d08066f90 msgr2=0x7f0d08069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:59.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d08066f90 0x7f0d08069450 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009950 tx=0x7f0d18008040 comp rx=0 tx=0).stop 2024-03-04T08:10:59.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 -- 172.21.15.113:0/3508295119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d280ce2d0 msgr2=0x7f0d28143cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:10:59.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d280ce2d0 0x7f0d28143cd0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400c370 tx=0x7f0d1400c830 comp rx=0 tx=0).stop 2024-03-04T08:10:59.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 -- 172.21.15.113:0/3508295119 shutdown_connections 2024-03-04T08:10:59.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0d280ce6b0 0x7f0d280c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:59.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0d08066f90 0x7f0d08069450 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:59.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d280ce2d0 0x7f0d28143cd0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:59.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 --2- 172.21.15.113:0/3508295119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0d2806de80 0x7f0d28146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:10:59.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 -- 172.21.15.113:0/3508295119 >> 172.21.15.113:0/3508295119 conn(0x7f0d280709a0 msgr2=0x7f0d2815abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:10:59.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 -- 172.21.15.113:0/3508295119 shutdown_connections 2024-03-04T08:10:59.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:10:59.168+0000 7f0d2e400640 1 -- 172.21.15.113:0/3508295119 wait complete. 2024-03-04T08:10:59.184 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:11:01.187 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:01.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.364+0000 7f22be200640 1 Processor -- start 2024-03-04T08:11:01.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.364+0000 7f22be200640 1 -- start start 2024-03-04T08:11:01.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22be200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f22b80cb540 0x7f22b815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:01.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22be200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22b806faa0 0x7f22b80c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:01.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22be200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b80c4ce0 con 0x7f22b8152950 2024-03-04T08:11:01.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22be200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b80c8370 con 0x7f22b806faa0 2024-03-04T08:11:01.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22be200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b80ca0a0 con 0x7f22b80cb540 2024-03-04T08:11:01.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22bd800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22b806faa0 0x7f22b80c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:01.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22bce00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f22b8152950 0x7f22b806ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56164/0 (socket says 172.21.15.113:56164) 2024-03-04T08:11:01.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22bce00640 1 -- 172.21.15.113:0/3498258486 learned_addr learned my addr 172.21.15.113:0/3498258486 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:11:01.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22b7e00640 1 --2- 172.21.15.113:0/3498258486 >> v2:172.21.15.136:3300/0 conn(0x7f22b80cb540 0x7f22b815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:01.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22bd800640 1 -- 172.21.15.113:0/3498258486 >> v2:172.21.15.136:3300/0 conn(0x7f22b80cb540 msgr2=0x7f22b815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:01.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22bd800640 1 --2- 172.21.15.113:0/3498258486 >> v2:172.21.15.136:3300/0 conn(0x7f22b80cb540 0x7f22b815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:01.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22bd800640 1 -- 172.21.15.113:0/3498258486 >> v1:172.21.15.113:6789/0 conn(0x7f22b8152950 legacy=0x7f22b806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:01.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22bd800640 1 -- 172.21.15.113:0/3498258486 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b80c4f80 con 0x7f22b806faa0 2024-03-04T08:11:01.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.368+0000 7f22b7400640 1 -- 172.21.15.113:0/3498258486 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552325661 0 0) 0x7f22b80c4ce0 con 0x7f22b8152950 2024-03-04T08:11:01.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22bd800640 1 --2- 172.21.15.113:0/3498258486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b806faa0 0x7f22b80c30b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f22b806c110 tx=0x7f22b001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57fd6bae1d88a6b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:01.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22b7400640 1 -- 172.21.15.113:0/3498258486 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b00b4070 con 0x7f22b806faa0 2024-03-04T08:11:01.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22b7400640 1 -- 172.21.15.113:0/3498258486 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b00af600 con 0x7f22b806faa0 2024-03-04T08:11:01.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22b7400640 1 -- 172.21.15.113:0/3498258486 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b00af9d0 con 0x7f22b806faa0 2024-03-04T08:11:01.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 -- 172.21.15.113:0/3498258486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b806faa0 msgr2=0x7f22b80c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:01.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 --2- 172.21.15.113:0/3498258486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b806faa0 0x7f22b80c30b0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f22b806c110 tx=0x7f22b001f110 comp rx=0 tx=0).stop 2024-03-04T08:11:01.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 -- 172.21.15.113:0/3498258486 shutdown_connections 2024-03-04T08:11:01.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 --2- 172.21.15.113:0/3498258486 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b806faa0 0x7f22b80c30b0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 --2- 172.21.15.113:0/3498258486 >> v2:172.21.15.136:3300/0 conn(0x7f22b80cb540 0x7f22b815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 -- 172.21.15.113:0/3498258486 >> 172.21.15.113:0/3498258486 conn(0x7f22b8072f40 msgr2=0x7f22b80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 -- 172.21.15.113:0/3498258486 shutdown_connections 2024-03-04T08:11:01.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 -- 172.21.15.113:0/3498258486 wait complete. 2024-03-04T08:11:01.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 Processor -- start 2024-03-04T08:11:01.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 -- start start 2024-03-04T08:11:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22b806faa0 0x7f22b80c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b80cb540 0x7f22b81f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b8152950 0x7f22b81f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:01.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80c9a50 con 0x7f22b8152950 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80c5220 con 0x7f22b80cb540 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22be200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80c9d30 con 0x7f22b806faa0 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22bd800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b8152950 0x7f22b81f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22bd800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b8152950 0x7f22b81f4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58970/0 (socket says 172.21.15.113:58970) 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22bd800640 1 -- 172.21.15.113:0/887163831 learned_addr learned my addr 172.21.15.113:0/887163831 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.372+0000 7f22bce00640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22b806faa0 0x7f22b80c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22b7e00640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b80cb540 0x7f22b81f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22bd800640 1 -- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22b806faa0 msgr2=0x7f22b80c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22bd800640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22b806faa0 0x7f22b80c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22bd800640 1 -- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b80cb540 msgr2=0x7f22b81f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22bd800640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b80cb540 0x7f22b81f3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22bd800640 1 -- 172.21.15.113:0/887163831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b806e190 con 0x7f22b8152950 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22b7e00640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b80cb540 0x7f22b81f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22bd800640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b8152950 0x7f22b81f4430 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f22b00b2040 tx=0x7f22b001beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:01.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22b4c00640 1 -- 172.21.15.113:0/887163831 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b00b4040 con 0x7f22b8152950 2024-03-04T08:11:01.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22b4c00640 1 -- 172.21.15.113:0/887163831 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b00039a0 con 0x7f22b8152950 2024-03-04T08:11:01.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22be200640 1 -- 172.21.15.113:0/887163831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b80cd570 con 0x7f22b8152950 2024-03-04T08:11:01.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22b4c00640 1 -- 172.21.15.113:0/887163831 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0009450 con 0x7f22b8152950 2024-03-04T08:11:01.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22be200640 1 -- 172.21.15.113:0/887163831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b80cd810 con 0x7f22b8152950 2024-03-04T08:11:01.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22be200640 1 -- 172.21.15.113:0/887163831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b81f3be0 con 0x7f22b8152950 2024-03-04T08:11:01.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.376+0000 7f22b4c00640 1 -- 172.21.15.113:0/887163831 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22b00c4060 con 0x7f22b8152950 2024-03-04T08:11:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.380+0000 7f22b4c00640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 0x7f2298069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.380+0000 7f22b4c00640 1 -- 172.21.15.113:0/887163831 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f22b01233d0 con 0x7f22b8152950 2024-03-04T08:11:01.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.380+0000 7f22bce00640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 0x7f2298069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:01.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.380+0000 7f22bce00640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 0x7f2298069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f22ac007fd0 tx=0x7f22ac007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:01.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.384+0000 7f22b4c00640 1 -- 172.21.15.113:0/887163831 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f22b0018c70 con 0x7f22b8152950 2024-03-04T08:11:01.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.672+0000 7f22be200640 1 -- 172.21.15.113:0/887163831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22b8075a80 con 0x7f22b8152950 2024-03-04T08:11:01.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22b4c00640 1 -- 172.21.15.113:0/887163831 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f22b00f3c10 con 0x7f22b8152950 2024-03-04T08:11:01.671 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:01.671 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":138,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:16.583518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389,7269]},"id":1}]} 2024-03-04T08:11:01.671 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 138 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 -- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 msgr2=0x7f2298069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 0x7f2298069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f22ac007fd0 tx=0x7f22ac007d60 comp rx=0 tx=0).stop 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 -- 172.21.15.113:0/887163831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b8152950 msgr2=0x7f22b81f4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b8152950 0x7f22b81f4430 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f22b00b2040 tx=0x7f22b001beb0 comp rx=0 tx=0).stop 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 -- 172.21.15.113:0/887163831 shutdown_connections 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 0x7f2298069420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22b8152950 0x7f22b81f4430 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22b80cb540 0x7f22b81f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 --2- 172.21.15.113:0/887163831 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22b806faa0 0x7f22b80c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 -- 172.21.15.113:0/887163831 >> 172.21.15.113:0/887163831 conn(0x7f22b8072f40 msgr2=0x7f22b8157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 -- 172.21.15.113:0/887163831 shutdown_connections 2024-03-04T08:11:01.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:01.676+0000 7f22be200640 1 -- 172.21.15.113:0/887163831 wait complete. 2024-03-04T08:11:01.692 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:11:03.693 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:03.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.832+0000 7f9862c00640 1 Processor -- start 2024-03-04T08:11:03.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.832+0000 7f9862c00640 1 -- start start 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f9862c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f985c15b900 0x7f985c163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f9862c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c0c27f0 con 0x7f985c0c22e0 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f9862c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c0c2a90 con 0x7f985c14e4c0 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f9862c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985c15ad60 con 0x7f985c15b900 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f985b400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f985c0c22e0 0x7f985c15b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56180/0 (socket says 172.21.15.113:56180) 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f985b400640 1 -- 172.21.15.113:0/1690129174 learned_addr learned my addr 172.21.15.113:0/1690129174 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f9860c00640 1 --2- 172.21.15.113:0/1690129174 >> v2:172.21.15.113:3300/0 conn(0x7f985c15b900 0x7f985c163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f985aa00640 1 -- 172.21.15.113:0/1690129174 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753006335 0 0) 0x7f985c0c27f0 con 0x7f985c0c22e0 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f985aa00640 1 -- 172.21.15.113:0/1690129174 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9840005740 con 0x7f985c0c22e0 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f9860c00640 1 -- 172.21.15.113:0/1690129174 >> v1:172.21.15.136:6789/0 conn(0x7f985c14e4c0 legacy=0x7f985c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:03.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f9860c00640 1 -- 172.21.15.113:0/1690129174 >> v1:172.21.15.113:6789/0 conn(0x7f985c0c22e0 legacy=0x7f985c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:03.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.836+0000 7f9860c00640 1 -- 172.21.15.113:0/1690129174 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c070c60 con 0x7f985c15b900 2024-03-04T08:11:03.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.840+0000 7f9860c00640 1 --2- 172.21.15.113:0/1690129174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c15b900 0x7f985c163100 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9854009d90 tx=0x7f985401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5ca3abce8c1a5b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:03.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.840+0000 7f985aa00640 1 -- 172.21.15.113:0/1690129174 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540b4070 con 0x7f985c15b900 2024-03-04T08:11:03.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.840+0000 7f985aa00640 1 -- 172.21.15.113:0/1690129174 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98540af600 con 0x7f985c15b900 2024-03-04T08:11:03.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.840+0000 7f985aa00640 1 -- 172.21.15.113:0/1690129174 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540af9d0 con 0x7f985c15b900 2024-03-04T08:11:03.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 -- 172.21.15.113:0/1690129174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c15b900 msgr2=0x7f985c163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:03.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 --2- 172.21.15.113:0/1690129174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c15b900 0x7f985c163100 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9854009d90 tx=0x7f985401f150 comp rx=0 tx=0).stop 2024-03-04T08:11:03.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 -- 172.21.15.113:0/1690129174 shutdown_connections 2024-03-04T08:11:03.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 --2- 172.21.15.113:0/1690129174 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c15b900 0x7f985c163100 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:03.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 -- 172.21.15.113:0/1690129174 >> 172.21.15.113:0/1690129174 conn(0x7f985c06e820 msgr2=0x7f985c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:03.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 -- 172.21.15.113:0/1690129174 shutdown_connections 2024-03-04T08:11:03.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 -- 172.21.15.113:0/1690129174 wait complete. 2024-03-04T08:11:03.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 Processor -- start 2024-03-04T08:11:03.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 -- start start 2024-03-04T08:11:03.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c0c22e0 0x7f985c159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:03.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f985c14e4c0 0x7f985c1e45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:03.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f985c15b900 0x7f985c1e31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:03.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c071500 con 0x7f985c0c22e0 2024-03-04T08:11:03.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c071220 con 0x7f985c15b900 2024-03-04T08:11:03.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9862c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c14e8a0 con 0x7f985c14e4c0 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f985be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c0c22e0 0x7f985c159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f985be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c0c22e0 0x7f985c159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58998/0 (socket says 172.21.15.113:58998) 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f985be00640 1 -- 172.21.15.113:0/3779220823 learned_addr learned my addr 172.21.15.113:0/3779220823 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f985b400640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f985c14e4c0 0x7f985c1e45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9860c00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f985c15b900 0x7f985c1e31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f985be00640 1 -- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f985c14e4c0 msgr2=0x7f985c1e45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f985be00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f985c14e4c0 0x7f985c1e45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f985be00640 1 -- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f985c15b900 msgr2=0x7f985c1e31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f985be00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f985c15b900 0x7f985c1e31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f985be00640 1 -- 172.21.15.113:0/3779220823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c1f0d20 con 0x7f985c0c22e0 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f985be00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c0c22e0 0x7f985c159340 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c530 tx=0x7f984c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.844+0000 7f9853e00640 1 -- 172.21.15.113:0/3779220823 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c016020 con 0x7f985c0c22e0 2024-03-04T08:11:03.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.848+0000 7f9853e00640 1 -- 172.21.15.113:0/3779220823 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c005220 con 0x7f985c0c22e0 2024-03-04T08:11:03.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.848+0000 7f9862c00640 1 -- 172.21.15.113:0/3779220823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c1f1000 con 0x7f985c0c22e0 2024-03-04T08:11:03.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.848+0000 7f9853e00640 1 -- 172.21.15.113:0/3779220823 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c007030 con 0x7f985c0c22e0 2024-03-04T08:11:03.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.848+0000 7f9862c00640 1 -- 172.21.15.113:0/3779220823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c1f15c0 con 0x7f985c0c22e0 2024-03-04T08:11:03.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.848+0000 7f9862c00640 1 -- 172.21.15.113:0/3779220823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c155190 con 0x7f985c0c22e0 2024-03-04T08:11:03.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.848+0000 7f9853e00640 1 -- 172.21.15.113:0/3779220823 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f984c0071d0 con 0x7f985c0c22e0 2024-03-04T08:11:03.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.848+0000 7f9853e00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:03.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.848+0000 7f9853e00640 1 -- 172.21.15.113:0/3779220823 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7f984c085190 con 0x7f985c0c22e0 2024-03-04T08:11:03.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.848+0000 7f985b400640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:03.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.852+0000 7f985b400640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 0x7f983c069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f985c1e4e80 tx=0x7f9848009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:03.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:03.852+0000 7f9853e00640 1 -- 172.21.15.113:0/3779220823 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f984c020020 con 0x7f985c0c22e0 2024-03-04T08:11:04.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.048+0000 7f9862c00640 1 -- 172.21.15.113:0/3779220823 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f985c075be0 con 0x7f985c0c22e0 2024-03-04T08:11:04.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.048+0000 7f9853e00640 1 -- 172.21.15.113:0/3779220823 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f984c0074c0 con 0x7f985c0c22e0 2024-03-04T08:11:04.045 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:04.045 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":138,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:16.583518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389,7269]},"id":1}]} 2024-03-04T08:11:04.046 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 138 2024-03-04T08:11:04.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 -- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 msgr2=0x7f983c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:04.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 0x7f983c069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f985c1e4e80 tx=0x7f9848009040 comp rx=0 tx=0).stop 2024-03-04T08:11:04.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 -- 172.21.15.113:0/3779220823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c0c22e0 msgr2=0x7f985c159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:04.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c0c22e0 0x7f985c159340 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c530 tx=0x7f984c00c9f0 comp rx=0 tx=0).stop 2024-03-04T08:11:04.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 -- 172.21.15.113:0/3779220823 shutdown_connections 2024-03-04T08:11:04.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f985c15b900 0x7f985c1e31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:04.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f985c14e4c0 0x7f985c1e45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:04.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:04.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 --2- 172.21.15.113:0/3779220823 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f985c0c22e0 0x7f985c159340 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:04.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 -- 172.21.15.113:0/3779220823 >> 172.21.15.113:0/3779220823 conn(0x7f985c06e820 msgr2=0x7f985c06ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:04.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 -- 172.21.15.113:0/3779220823 shutdown_connections 2024-03-04T08:11:04.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:04.052+0000 7f9862c00640 1 -- 172.21.15.113:0/3779220823 wait complete. 2024-03-04T08:11:04.066 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:11:04.990 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 21% (71/335) Updating files: 22% (74/335) Updating files: 23% (78/335) Updating files: 24% (81/335) Updating files: 24% (82/335) Updating files: 25% (84/335) Updating files: 26% (88/335) Updating files: 27% (91/335) Updating files: 28% (94/335) Updating files: 29% (98/335) Updating files: 30% (101/335) Updating files: 31% (104/335) Updating files: 32% (108/335) Updating files: 33% (111/335) Updating files: 34% (114/335) Updating files: 35% (118/335) Updating files: 36% (121/335) Updating files: 37% (124/335) Updating files: 38% (128/335) Updating files: 38% (129/335) Updating files: 39% (131/335) Updating files: 40% (134/335) Updating files: 41% (138/335) Updating files: 42% (141/335) Updating files: 43% (145/335) Updating files: 44% (148/335) Updating files: 45% (151/335) Updating files: 46% (155/335) Updating files: 47% (158/335) Updating files: 48% (161/335) Updating files: 49% (165/335) Updating files: 50% (168/335) Updating files: 51% (171/335) Updating files: 52% (175/335) Updating files: 53% (178/335) Updating files: 54% (181/335) Updating files: 55% (185/335) Updating files: 56% (188/335) Updating files: 57% (191/335) Updating files: 58% (195/335) Updating files: 59% (198/335) Updating files: 60% (201/335) Updating files: 61% (205/335) Updating files: 62% (208/335) Updating files: 63% (212/335) Updating files: 64% (215/335) Updating files: 65% (218/335) Updating files: 66% (222/335) Updating files: 67% (225/335) Updating files: 68% (228/335) Updating files: 69% (232/335) Updating files: 70% (235/335) Updating files: 71% (238/335) Updating files: 72% (242/335) Updating files: 73% (245/335) Updating files: 73% (246/335) Updating files: 74% (248/335) Updating files: 75% (252/335) Updating files: 76% (255/335) Updating files: 77% (258/335) Updating files: 78% (262/335) Updating files: 79% (265/335) Updating files: 80% (268/335) Updating files: 81% (272/335) Updating files: 81% (273/335) Updating files: 82% (275/335) Updating files: 83% (279/335) Updating files: 84% (282/335) Updating files: 85% (285/335) Updating files: 86% (289/335) Updating files: 87% (292/335) Updating files: 87% (293/335) Updating files: 88% (295/335) Updating files: 89% (299/335) Updating files: 90% (302/335) Updating files: 91% (305/335) Updating files: 92% (309/335) Updating files: 93% (312/335) Updating files: 94% (315/335) Updating files: 95% (319/335) Updating files: 96% (322/335) Updating files: 97% (325/335) Updating files: 98% (329/335) Updating files: 99% (332/335) Updating files: 100% (335/335) Updating files: 100% (335/335), done. 2024-03-04T08:11:05.813 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:11:05.816 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:11:06.067 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:06.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.208+0000 7fe8ba200640 1 Processor -- start 2024-03-04T08:11:06.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.208+0000 7fe8ba200640 1 -- start start 2024-03-04T08:11:06.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8ba200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b40ca3f0 0x7fe8b40c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:06.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8ba200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b406c5b0 con 0x7fe8b4157f80 2024-03-04T08:11:06.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8ba200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b406df00 con 0x7fe8b406c170 2024-03-04T08:11:06.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8ba200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40c8fd0 con 0x7fe8b40ca3f0 2024-03-04T08:11:06.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b8e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe8b4157f80 0x7fe8b406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56186/0 (socket says 172.21.15.113:56186) 2024-03-04T08:11:06.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b8e00640 1 -- 172.21.15.113:0/193102029 learned_addr learned my addr 172.21.15.113:0/193102029 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:11:06.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b9800640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b40ca3f0 0x7fe8b40c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:06.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597823567 0 0) 0x7fe8b406c5b0 con 0x7fe8b4157f80 2024-03-04T08:11:06.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe898003530 con 0x7fe8b4157f80 2024-03-04T08:11:06.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683024710 0 0) 0x7fe8b406df00 con 0x7fe8b406c170 2024-03-04T08:11:06.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b406c5b0 con 0x7fe8b406c170 2024-03-04T08:11:06.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1151662538 0 0) 0x7fe898003530 con 0x7fe8b4157f80 2024-03-04T08:11:06.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b40ca3f0 msgr2=0x7fe8b40c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:06.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b40ca3f0 0x7fe8b40c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 >> v1:172.21.15.136:6789/0 conn(0x7fe8b406c170 legacy=0x7fe8b40c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:06.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b40ce1a0 con 0x7fe8b4157f80 2024-03-04T08:11:06.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fe8a4001680 con 0x7fe8b4157f80 2024-03-04T08:11:06.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b9800640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b40ca3f0 0x7fe8b40c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:11:06.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 >> v1:172.21.15.113:6789/0 conn(0x7fe8b4157f80 legacy=0x7fe8b406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:06.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b40ca3f0 0x7fe898003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:06.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898004110 0x7fe898004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:06.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe898004d50 0x7fe8980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:06.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b40c8fd0 con 0x7fe898004d50 2024-03-04T08:11:06.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89800bae0 con 0x7fe898004110 2024-03-04T08:11:06.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89800bce0 con 0x7fe8b40ca3f0 2024-03-04T08:11:06.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b9800640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe898004d50 0x7fe8980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:06.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b8e00640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b40ca3f0 0x7fe898003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:06.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8abe00640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898004110 0x7fe898004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:06.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b9800640 1 -- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b40ca3f0 msgr2=0x7fe898003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:06.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b9800640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b40ca3f0 0x7fe898003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b9800640 1 -- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898004110 msgr2=0x7fe898004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:06.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b9800640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898004110 0x7fe898004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8b9800640 1 -- 172.21.15.113:0/193102029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe898003530 con 0x7fe898004d50 2024-03-04T08:11:06.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.212+0000 7fe8abe00640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe898004110 0x7fe898004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:11:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8b9800640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe898004d50 0x7fe8980051d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e320 tx=0x7fe8ac01cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330dee0462d3d278 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac009dc0 con 0x7fe898004d50 2024-03-04T08:11:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ac0a5dd0 con 0x7fe898004d50 2024-03-04T08:11:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8b3e00640 1 -- 172.21.15.113:0/193102029 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac003ba0 con 0x7fe898004d50 2024-03-04T08:11:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8ba200640 1 -- 172.21.15.113:0/193102029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe898004d50 msgr2=0x7fe8980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8ba200640 1 --2- 172.21.15.113:0/193102029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe898004d50 0x7fe8980051d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00e320 tx=0x7fe8ac01cdc0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8b9800640 1 -- 172.21.15.113:0/193102029 reap_dead start 2024-03-04T08:11:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8ba200640 1 -- 172.21.15.113:0/193102029 shutdown_connections 2024-03-04T08:11:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8ba200640 1 -- 172.21.15.113:0/193102029 >> 172.21.15.113:0/193102029 conn(0x7fe8b4072f40 msgr2=0x7fe8b40706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:06.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8ba200640 1 -- 172.21.15.113:0/193102029 shutdown_connections 2024-03-04T08:11:06.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.220+0000 7fe8ba200640 1 -- 172.21.15.113:0/193102029 wait complete. 2024-03-04T08:11:06.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8ba200640 1 Processor -- start 2024-03-04T08:11:06.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8ba200640 1 -- start start 2024-03-04T08:11:06.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8ba200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b40cb260 0x7fe8b41f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:06.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8ba200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b4156950 0x7fe8b41f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:06.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8ba200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b40cb640 0x7fe8b4153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:06.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8ba200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b406dc00 con 0x7fe8b40cb640 2024-03-04T08:11:06.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8ba200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b406e1e0 con 0x7fe8b40cb260 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8ba200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b4158360 con 0x7fe8b4156950 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b40cb260 0x7fe8b41f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b40cb640 0x7fe8b4153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b40cb640 0x7fe8b4153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59012/0 (socket says 172.21.15.113:59012) 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b40cb260 0x7fe8b41f0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:55566/0 (socket says 172.21.15.113:55566) 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8abe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b4156950 0x7fe8b41f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b9800640 1 -- 172.21.15.113:0/2182252413 learned_addr learned my addr 172.21.15.113:0/2182252413 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b9800640 1 -- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b4156950 msgr2=0x7fe8b41f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b9800640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b4156950 0x7fe8b41f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b9800640 1 -- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b40cb260 msgr2=0x7fe8b41f0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b9800640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b40cb260 0x7fe8b41f0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b9800640 1 -- 172.21.15.113:0/2182252413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b40c8060 con 0x7fe8b40cb640 2024-03-04T08:11:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b9800640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b40cb640 0x7fe8b4153ad0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac01cbf0 tx=0x7fe8ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:06.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b1600640 1 -- 172.21.15.113:0/2182252413 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac009dc0 con 0x7fe8b40cb640 2024-03-04T08:11:06.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b1600640 1 -- 172.21.15.113:0/2182252413 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ac009a50 con 0x7fe8b40cb640 2024-03-04T08:11:06.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8b1600640 1 -- 172.21.15.113:0/2182252413 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac004460 con 0x7fe8b40cb640 2024-03-04T08:11:06.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8ba200640 1 -- 172.21.15.113:0/2182252413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b41f4b10 con 0x7fe8b40cb640 2024-03-04T08:11:06.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.224+0000 7fe8ba200640 1 -- 172.21.15.113:0/2182252413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b41f4270 con 0x7fe8b40cb640 2024-03-04T08:11:06.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.228+0000 7fe8ba200640 1 -- 172.21.15.113:0/2182252413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b4075a80 con 0x7fe8b40cb640 2024-03-04T08:11:06.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.228+0000 7fe8b1600640 1 -- 172.21.15.113:0/2182252413 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8ac0c0060 con 0x7fe8b40cb640 2024-03-04T08:11:06.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.228+0000 7fe8b1600640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe890066f60 0x7fe890069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:06.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.228+0000 7fe8b1600640 1 -- 172.21.15.113:0/2182252413 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7fe8ac11fc70 con 0x7fe8b40cb640 2024-03-04T08:11:06.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.228+0000 7fe8b8e00640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe890066f60 0x7fe890069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:06.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.228+0000 7fe8b8e00640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe890066f60 0x7fe890069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4000c10 tx=0x7fe8a40015e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:06.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.232+0000 7fe8b1600640 1 -- 172.21.15.113:0/2182252413 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe8ac016ad0 con 0x7fe8b40cb640 2024-03-04T08:11:06.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.424+0000 7fe8ba200640 1 -- 172.21.15.113:0/2182252413 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8b40c97b0 con 0x7fe8b40cb640 2024-03-04T08:11:06.421 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:06.421 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":138,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:16.583518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389,7269]},"id":1}]} 2024-03-04T08:11:06.421 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 138 2024-03-04T08:11:06.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.424+0000 7fe8b1600640 1 -- 172.21.15.113:0/2182252413 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+6621 (secure 0 0 0) 0x7fe8ac0efe50 con 0x7fe8b40cb640 2024-03-04T08:11:06.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 -- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe890066f60 msgr2=0x7fe890069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:06.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe890066f60 0x7fe890069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4000c10 tx=0x7fe8a40015e0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 -- 172.21.15.113:0/2182252413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b40cb640 msgr2=0x7fe8b4153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:06.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b40cb640 0x7fe8b4153ad0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac01cbf0 tx=0x7fe8ac009750 comp rx=0 tx=0).stop 2024-03-04T08:11:06.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 -- 172.21.15.113:0/2182252413 shutdown_connections 2024-03-04T08:11:06.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe890066f60 0x7fe890069420 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe8b40cb640 0x7fe8b4153ad0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe8b4156950 0x7fe8b41f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 --2- 172.21.15.113:0/2182252413 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8b40cb260 0x7fe8b41f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:06.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 -- 172.21.15.113:0/2182252413 >> 172.21.15.113:0/2182252413 conn(0x7fe8b4072f40 msgr2=0x7fe8b4066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:06.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 -- 172.21.15.113:0/2182252413 shutdown_connections 2024-03-04T08:11:06.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:06.428+0000 7fe8ba200640 1 -- 172.21.15.113:0/2182252413 wait complete. 2024-03-04T08:11:06.676 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:11:08.677 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:08.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.816+0000 7fc9b7a00640 1 Processor -- start 2024-03-04T08:11:08.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.816+0000 7fc9b7a00640 1 -- start start 2024-03-04T08:11:08.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.816+0000 7fc9b7a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9b00c22e0 0x7fc9b015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:08.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.816+0000 7fc9b7a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b00c27f0 con 0x7fc9b014e4c0 2024-03-04T08:11:08.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.816+0000 7fc9b7a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b00c2a90 con 0x7fc9b00c2d80 2024-03-04T08:11:08.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.816+0000 7fc9b7a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b015ad60 con 0x7fc9b00c22e0 2024-03-04T08:11:08.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9afe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9b00c22e0 0x7fc9b015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:08.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9afe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9b00c22e0 0x7fc9b015b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56640/0 (socket says 172.21.15.113:56640) 2024-03-04T08:11:08.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9b5000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc9b014e4c0 0x7fc9b0072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55126/0 (socket says 172.21.15.113:55126) 2024-03-04T08:11:08.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9afe00640 1 -- 172.21.15.113:0/173286400 learned_addr learned my addr 172.21.15.113:0/173286400 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:08.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9afe00640 1 -- 172.21.15.113:0/173286400 >> v1:172.21.15.136:6789/0 conn(0x7fc9b00c2d80 legacy=0x7fc9b01620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:08.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9afe00640 1 -- 172.21.15.113:0/173286400 >> v1:172.21.15.113:6789/0 conn(0x7fc9b014e4c0 legacy=0x7fc9b0072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:08.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9afe00640 1 -- 172.21.15.113:0/173286400 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b0070c60 con 0x7fc9b00c22e0 2024-03-04T08:11:08.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9af400640 1 -- 172.21.15.113:0/173286400 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325253457 0 0) 0x7fc9b00c27f0 con 0x7fc9b014e4c0 2024-03-04T08:11:08.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9afe00640 1 --2- 172.21.15.113:0/173286400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c22e0 0x7fc9b015b1f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009870 tx=0x7fc99c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cb8321feefa74d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:08.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9af400640 1 -- 172.21.15.113:0/173286400 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0ab650 con 0x7fc9b00c22e0 2024-03-04T08:11:08.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9af400640 1 -- 172.21.15.113:0/173286400 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c0b0070 con 0x7fc9b00c22e0 2024-03-04T08:11:08.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.820+0000 7fc9af400640 1 -- 172.21.15.113:0/173286400 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0aba10 con 0x7fc9b00c22e0 2024-03-04T08:11:08.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 -- 172.21.15.113:0/173286400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c22e0 msgr2=0x7fc9b015b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:08.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 --2- 172.21.15.113:0/173286400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c22e0 0x7fc9b015b1f0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009870 tx=0x7fc99c01c560 comp rx=0 tx=0).stop 2024-03-04T08:11:08.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 -- 172.21.15.113:0/173286400 shutdown_connections 2024-03-04T08:11:08.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 --2- 172.21.15.113:0/173286400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b00c22e0 0x7fc9b015b1f0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:08.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 -- 172.21.15.113:0/173286400 >> 172.21.15.113:0/173286400 conn(0x7fc9b006e820 msgr2=0x7fc9b006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:08.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 -- 172.21.15.113:0/173286400 shutdown_connections 2024-03-04T08:11:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 -- 172.21.15.113:0/173286400 wait complete. 2024-03-04T08:11:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 Processor -- start 2024-03-04T08:11:08.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 -- start start 2024-03-04T08:11:08.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9b00c22e0 0x7fc9b0145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:08.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b00c2d80 0x7fc9b01f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:08.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b014e4c0 0x7fc9b01f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b014e8a0 con 0x7fc9b014e4c0 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b015bb40 con 0x7fc9b00c22e0 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b7a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b00aede0 con 0x7fc9b00c2d80 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b014e4c0 0x7fc9b01f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b014e4c0 0x7fc9b01f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56650/0 (socket says 172.21.15.113:56650) 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b5a00640 1 -- 172.21.15.113:0/748209267 learned_addr learned my addr 172.21.15.113:0/748209267 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b5000640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9b00c22e0 0x7fc9b0145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9afe00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b00c2d80 0x7fc9b01f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b5a00640 1 -- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b00c2d80 msgr2=0x7fc9b01f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b5a00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b00c2d80 0x7fc9b01f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b5a00640 1 -- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9b00c22e0 msgr2=0x7fc9b0145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:08.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b5a00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9b00c22e0 0x7fc9b0145f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:08.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b5a00640 1 -- 172.21.15.113:0/748209267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b01492d0 con 0x7fc9b014e4c0 2024-03-04T08:11:08.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.824+0000 7fc9b5a00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b014e4c0 0x7fc9b01f4a40 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c730 tx=0x7fc9a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:08.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.828+0000 7fc9acc00640 1 -- 172.21.15.113:0/748209267 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8003b80 con 0x7fc9b014e4c0 2024-03-04T08:11:08.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.828+0000 7fc9acc00640 1 -- 172.21.15.113:0/748209267 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a8003d20 con 0x7fc9b014e4c0 2024-03-04T08:11:08.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.828+0000 7fc9acc00640 1 -- 172.21.15.113:0/748209267 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a800aa30 con 0x7fc9b014e4c0 2024-03-04T08:11:08.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.828+0000 7fc9b7a00640 1 -- 172.21.15.113:0/748209267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b01548e0 con 0x7fc9b014e4c0 2024-03-04T08:11:08.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.828+0000 7fc9b7a00640 1 -- 172.21.15.113:0/748209267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b014a5c0 con 0x7fc9b014e4c0 2024-03-04T08:11:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.828+0000 7fc9b7a00640 1 -- 172.21.15.113:0/748209267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b014a790 con 0x7fc9b014e4c0 2024-03-04T08:11:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.828+0000 7fc9acc00640 1 -- 172.21.15.113:0/748209267 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9a800abd0 con 0x7fc9b014e4c0 2024-03-04T08:11:08.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.828+0000 7fc9acc00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f90 0x7fc990069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.828+0000 7fc9acc00640 1 -- 172.21.15.113:0/748209267 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7545+0+0 (secure 0 0 0) 0x7fc9a80855a0 con 0x7fc9b014e4c0 2024-03-04T08:11:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.828+0000 7fc9b5000640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f90 0x7fc990069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:08.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.832+0000 7fc9b5000640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f90 0x7fc990069450 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4000f70 tx=0x7fc9a4007240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:08.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:08.832+0000 7fc9acc00640 1 -- 172.21.15.113:0/748209267 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc9a8055e70 con 0x7fc9b014e4c0 2024-03-04T08:11:09.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.044+0000 7fc9b7a00640 1 -- 172.21.15.113:0/748209267 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9b0075c00 con 0x7fc9b014e4c0 2024-03-04T08:11:09.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.044+0000 7fc9acc00640 1 -- 172.21.15.113:0/748209267 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+6621 (secure 0 0 0) 0x7fc9a8055770 con 0x7fc9b014e4c0 2024-03-04T08:11:09.292 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:09.293 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":138,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7692,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":138}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:10:16.583518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7269},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7269":{"gid":7269,"name":"e","rank":2,"incarnation":120,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6837/1401330284","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":1401330284},{"type":"v1","addr":"172.21.15.113:6837","nonce":1401330284}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389,7269]},"id":1}]} 2024-03-04T08:11:09.294 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 138 2024-03-04T08:11:09.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 -- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f90 msgr2=0x7fc990069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:09.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f90 0x7fc990069450 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4000f70 tx=0x7fc9a4007240 comp rx=0 tx=0).stop 2024-03-04T08:11:09.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 -- 172.21.15.113:0/748209267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b014e4c0 msgr2=0x7fc9b01f4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:09.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b014e4c0 0x7fc9b01f4a40 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c730 tx=0x7fc9a800cbf0 comp rx=0 tx=0).stop 2024-03-04T08:11:09.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 -- 172.21.15.113:0/748209267 shutdown_connections 2024-03-04T08:11:09.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc990066f90 0x7fc990069450 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:09.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9b014e4c0 0x7fc9b01f4a40 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:09.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9b00c2d80 0x7fc9b01f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:09.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 --2- 172.21.15.113:0/748209267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9b00c22e0 0x7fc9b0145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:09.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 -- 172.21.15.113:0/748209267 >> 172.21.15.113:0/748209267 conn(0x7fc9b006e820 msgr2=0x7fc9b0072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:09.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 -- 172.21.15.113:0/748209267 shutdown_connections 2024-03-04T08:11:09.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:09.048+0000 7fc9b7a00640 1 -- 172.21.15.113:0/748209267 wait complete. 2024-03-04T08:11:09.295 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:11:11.296 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:11.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f272e600640 1 Processor -- start 2024-03-04T08:11:11.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f272e600640 1 -- start start 2024-03-04T08:11:11.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f272e600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 0x7f2728070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f272e600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2728152bd0 0x7f27281587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f272e600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27281591f0 con 0x7f27281527f0 2024-03-04T08:11:11.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f272e600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2728071df0 con 0x7f2728152bd0 2024-03-04T08:11:11.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f272e600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27281597b0 con 0x7f27280c8200 2024-03-04T08:11:11.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f2727e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2728152bd0 0x7f27281587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f271fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 0x7f2728070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f2727400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f27281527f0 0x7f2728072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:45652/0 (socket says 172.21.15.113:45652) 2024-03-04T08:11:11.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f2727400640 1 -- 172.21.15.113:0/3239502345 learned_addr learned my addr 172.21.15.113:0/3239502345 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:11:11.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f2727e00640 1 -- 172.21.15.113:0/3239502345 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 msgr2=0x7f2728070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:11.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f2727e00640 1 --2- 172.21.15.113:0/3239502345 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 0x7f2728070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f2727e00640 1 -- 172.21.15.113:0/3239502345 >> v1:172.21.15.136:6789/0 conn(0x7f27281527f0 legacy=0x7f2728072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:11.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.440+0000 7f2727e00640 1 -- 172.21.15.113:0/3239502345 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27281594d0 con 0x7f2728152bd0 2024-03-04T08:11:11.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.436+0000 7f2726a00640 1 -- 172.21.15.113:0/3239502345 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194471823 0 0) 0x7f27281591f0 con 0x7f27281527f0 2024-03-04T08:11:11.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f2727e00640 1 --2- 172.21.15.113:0/3239502345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2728152bd0 0x7f27281587d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2720009d90 tx=0x7f272001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f707f0dff8921047 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:11.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f2726a00640 1 -- 172.21.15.113:0/3239502345 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27200b2070 con 0x7f2728152bd0 2024-03-04T08:11:11.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f2726a00640 1 -- 172.21.15.113:0/3239502345 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27200a5920 con 0x7f2728152bd0 2024-03-04T08:11:11.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f2726a00640 1 -- 172.21.15.113:0/3239502345 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27200a5cf0 con 0x7f2728152bd0 2024-03-04T08:11:11.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f272e600640 1 -- 172.21.15.113:0/3239502345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2728152bd0 msgr2=0x7f27281587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:11.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f272e600640 1 --2- 172.21.15.113:0/3239502345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2728152bd0 0x7f27281587d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f2720009d90 tx=0x7f272001c970 comp rx=0 tx=0).stop 2024-03-04T08:11:11.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f272e600640 1 -- 172.21.15.113:0/3239502345 shutdown_connections 2024-03-04T08:11:11.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f272e600640 1 --2- 172.21.15.113:0/3239502345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2728152bd0 0x7f27281587d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f272e600640 1 --2- 172.21.15.113:0/3239502345 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 0x7f2728070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f272e600640 1 -- 172.21.15.113:0/3239502345 >> 172.21.15.113:0/3239502345 conn(0x7f272806e820 msgr2=0x7f272806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f272e600640 1 -- 172.21.15.113:0/3239502345 shutdown_connections 2024-03-04T08:11:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f272e600640 1 -- 172.21.15.113:0/3239502345 wait complete. 2024-03-04T08:11:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f272e600640 1 Processor -- start 2024-03-04T08:11:11.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.444+0000 7f272e600640 1 -- start start 2024-03-04T08:11:11.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f272e600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 0x7f2728146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f272e600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27281527f0 0x7f2728146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f272e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728152bd0 0x7f27281450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f272e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27281654f0 con 0x7f2728152bd0 2024-03-04T08:11:11.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f272e600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2728165370 con 0x7f27281527f0 2024-03-04T08:11:11.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f272e600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2728165670 con 0x7f27280c8200 2024-03-04T08:11:11.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728152bd0 0x7f27281450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 0x7f2728146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728152bd0 0x7f27281450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56660/0 (socket says 172.21.15.113:56660) 2024-03-04T08:11:11.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 0x7f2728146120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:52826/0 (socket says 172.21.15.113:52826) 2024-03-04T08:11:11.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727e00640 1 -- 172.21.15.113:0/1313527181 learned_addr learned my addr 172.21.15.113:0/1313527181 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:11.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f271fe00640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27281527f0 0x7f2728146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727e00640 1 -- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 msgr2=0x7f2728146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:11.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727e00640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 0x7f2728146120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727e00640 1 -- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27281527f0 msgr2=0x7f2728146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:11.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727e00640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27281527f0 0x7f2728146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727e00640 1 -- 172.21.15.113:0/1313527181 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2728148140 con 0x7f2728152bd0 2024-03-04T08:11:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f2727e00640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728152bd0 0x7f27281450b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f27200b0040 tx=0x7f2720019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f271f400640 1 -- 172.21.15.113:0/1313527181 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27200b2050 con 0x7f2728152bd0 2024-03-04T08:11:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f271f400640 1 -- 172.21.15.113:0/1313527181 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27200b3040 con 0x7f2728152bd0 2024-03-04T08:11:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f272e600640 1 -- 172.21.15.113:0/1313527181 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2728150c70 con 0x7f2728152bd0 2024-03-04T08:11:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f271f400640 1 -- 172.21.15.113:0/1313527181 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27200ad850 con 0x7f2728152bd0 2024-03-04T08:11:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f272e600640 1 -- 172.21.15.113:0/1313527181 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2728148e80 con 0x7f2728152bd0 2024-03-04T08:11:11.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.448+0000 7f272e600640 1 -- 172.21.15.113:0/1313527181 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2728075c00 con 0x7f2728152bd0 2024-03-04T08:11:11.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.452+0000 7f271f400640 1 -- 172.21.15.113:0/1313527181 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27200c0020 con 0x7f2728152bd0 2024-03-04T08:11:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.452+0000 7f271f400640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2704066f60 0x7f2704069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.452+0000 7f271f400640 1 -- 172.21.15.113:0/1313527181 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f272011ff60 con 0x7f2728152bd0 2024-03-04T08:11:11.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.452+0000 7f2727400640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2704066f60 0x7f2704069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.452+0000 7f2727400640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2704066f60 0x7f2704069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2718000f70 tx=0x7f2718007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:11.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.456+0000 7f271f400640 1 -- 172.21.15.113:0/1313527181 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2728148140 con 0x7f2728152bd0 2024-03-04T08:11:11.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 -- 172.21.15.113:0/1313527181 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27280c79d0 con 0x7f2728152bd0 2024-03-04T08:11:11.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f271f400640 1 -- 172.21.15.113:0/1313527181 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+5820 (secure 0 0 0) 0x7f2720016560 con 0x7f2728152bd0 2024-03-04T08:11:11.644 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:11.644 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":139,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:10.891544+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7519]},"id":1}]} 2024-03-04T08:11:11.644 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 139 2024-03-04T08:11:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 -- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2704066f60 msgr2=0x7f2704069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2704066f60 0x7f2704069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2718000f70 tx=0x7f2718007d60 comp rx=0 tx=0).stop 2024-03-04T08:11:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 -- 172.21.15.113:0/1313527181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728152bd0 msgr2=0x7f27281450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:11.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728152bd0 0x7f27281450b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f27200b0040 tx=0x7f2720019710 comp rx=0 tx=0).stop 2024-03-04T08:11:11.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 -- 172.21.15.113:0/1313527181 shutdown_connections 2024-03-04T08:11:11.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2704066f60 0x7f2704069420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2728152bd0 0x7f27281450b0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27281527f0 0x7f2728146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 --2- 172.21.15.113:0/1313527181 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27280c8200 0x7f2728146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 -- 172.21.15.113:0/1313527181 >> 172.21.15.113:0/1313527181 conn(0x7f272806e820 msgr2=0x7f27280cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:11.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 -- 172.21.15.113:0/1313527181 shutdown_connections 2024-03-04T08:11:11.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.648+0000 7f272e600640 1 -- 172.21.15.113:0/1313527181 wait complete. 2024-03-04T08:11:11.665 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-03-04T08:11:11.666 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:11:11.666 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:11.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.804+0000 7fd858c00640 1 Processor -- start 2024-03-04T08:11:11.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.804+0000 7fd858c00640 1 -- start start 2024-03-04T08:11:11.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd858c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd85406ff20 0x7fd85415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd858c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd8540c8b90 0x7fd85415c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd858c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd854152e30 con 0x7fd854152a50 2024-03-04T08:11:11.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd858c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8540c31b0 con 0x7fd85406ff20 2024-03-04T08:11:11.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd858c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85406f840 con 0x7fd8540c8b90 2024-03-04T08:11:11.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd851400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd85406ff20 0x7fd85415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd851400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd85406ff20 0x7fd85415bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56666/0 (socket says 172.21.15.113:56666) 2024-03-04T08:11:11.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd851400640 1 -- 172.21.15.113:0/141771925 learned_addr learned my addr 172.21.15.113:0/141771925 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:11.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd852800640 1 --2- 172.21.15.113:0/141771925 >> v2:172.21.15.136:3300/0 conn(0x7fd8540c8b90 0x7fd85415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd851400640 1 -- 172.21.15.113:0/141771925 >> v2:172.21.15.136:3300/0 conn(0x7fd8540c8b90 msgr2=0x7fd85415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:11.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd851400640 1 --2- 172.21.15.113:0/141771925 >> v2:172.21.15.136:3300/0 conn(0x7fd8540c8b90 0x7fd85415c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd851400640 1 -- 172.21.15.113:0/141771925 >> v1:172.21.15.113:6789/0 conn(0x7fd854152a50 legacy=0x7fd8540ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:11.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd851400640 1 -- 172.21.15.113:0/141771925 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85406f280 con 0x7fd85406ff20 2024-03-04T08:11:11.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.808+0000 7fd851400640 1 --2- 172.21.15.113:0/141771925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85406ff20 0x7fd85415bdd0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd85406ef00 tx=0x7fd84001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b10af3dbc817a77 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:11.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd850a00640 1 -- 172.21.15.113:0/141771925 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400a9400 con 0x7fd85406ff20 2024-03-04T08:11:11.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd850a00640 1 -- 172.21.15.113:0/141771925 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8400a95a0 con 0x7fd85406ff20 2024-03-04T08:11:11.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd850a00640 1 -- 172.21.15.113:0/141771925 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400a9970 con 0x7fd85406ff20 2024-03-04T08:11:11.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 -- 172.21.15.113:0/141771925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85406ff20 msgr2=0x7fd85415bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:11.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 --2- 172.21.15.113:0/141771925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85406ff20 0x7fd85415bdd0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fd85406ef00 tx=0x7fd84001c520 comp rx=0 tx=0).stop 2024-03-04T08:11:11.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 -- 172.21.15.113:0/141771925 shutdown_connections 2024-03-04T08:11:11.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 --2- 172.21.15.113:0/141771925 >> v2:172.21.15.136:3300/0 conn(0x7fd8540c8b90 0x7fd85415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 --2- 172.21.15.113:0/141771925 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd85406ff20 0x7fd85415bdd0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 -- 172.21.15.113:0/141771925 >> 172.21.15.113:0/141771925 conn(0x7fd8540734d0 msgr2=0x7fd854066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:11.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 -- 172.21.15.113:0/141771925 shutdown_connections 2024-03-04T08:11:11.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 -- 172.21.15.113:0/141771925 wait complete. 2024-03-04T08:11:11.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 Processor -- start 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 -- start start 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd85406ff20 0x7fd854154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8540c8b90 0x7fd85414de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd854152a50 0x7fd85414e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540ce9b0 con 0x7fd8540c8b90 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8540ce830 con 0x7fd854152a50 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd858c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd854165880 con 0x7fd85406ff20 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd851400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8540c8b90 0x7fd85414de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd851400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8540c8b90 0x7fd85414de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56674/0 (socket says 172.21.15.113:56674) 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd851400640 1 -- 172.21.15.113:0/670508319 learned_addr learned my addr 172.21.15.113:0/670508319 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:11.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd852800640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd854152a50 0x7fd85414e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd851e00640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd85406ff20 0x7fd854154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd851400640 1 -- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd85406ff20 msgr2=0x7fd854154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd851400640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd85406ff20 0x7fd854154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd851400640 1 -- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd854152a50 msgr2=0x7fd85414e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.812+0000 7fd851400640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd854152a50 0x7fd85414e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd851400640 1 -- 172.21.15.113:0/670508319 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85414fe60 con 0x7fd8540c8b90 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd851e00640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd85406ff20 0x7fd854154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd851400640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8540c8b90 0x7fd85414de40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd84001ca00 tx=0x7fd8400192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd84a000640 1 -- 172.21.15.113:0/670508319 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400a9400 con 0x7fd8540c8b90 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd84a000640 1 -- 172.21.15.113:0/670508319 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8400b1800 con 0x7fd8540c8b90 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd84a000640 1 -- 172.21.15.113:0/670508319 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8400b1bb0 con 0x7fd8540c8b90 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd858c00640 1 -- 172.21.15.113:0/670508319 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd854150140 con 0x7fd8540c8b90 2024-03-04T08:11:11.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd858c00640 1 -- 172.21.15.113:0/670508319 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd854150700 con 0x7fd8540c8b90 2024-03-04T08:11:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd858c00640 1 -- 172.21.15.113:0/670508319 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd854075c00 con 0x7fd8540c8b90 2024-03-04T08:11:11.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd84a000640 1 -- 172.21.15.113:0/670508319 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8400c0060 con 0x7fd8540c8b90 2024-03-04T08:11:11.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd84a000640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd834066f60 0x7fd834069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:11.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd84a000640 1 -- 172.21.15.113:0/670508319 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7fd84011f790 con 0x7fd8540c8b90 2024-03-04T08:11:11.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.816+0000 7fd851e00640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd834066f60 0x7fd834069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:11.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.820+0000 7fd851e00640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd834066f60 0x7fd834069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd844001710 tx=0x7fd844009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:11.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:11.820+0000 7fd84a000640 1 -- 172.21.15.113:0/670508319 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd8400ad020 con 0x7fd8540c8b90 2024-03-04T08:11:12.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.012+0000 7fd858c00640 1 -- 172.21.15.113:0/670508319 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd854076a60 con 0x7fd8540c8b90 2024-03-04T08:11:12.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.012+0000 7fd84a000640 1 -- 172.21.15.113:0/670508319 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+5820 (secure 0 0 0) 0x7fd840016110 con 0x7fd8540c8b90 2024-03-04T08:11:12.013 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:12.013 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":139,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:10.891544+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7519]},"id":1}]} 2024-03-04T08:11:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 139 2024-03-04T08:11:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 -- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd834066f60 msgr2=0x7fd834069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd834066f60 0x7fd834069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd844001710 tx=0x7fd844009040 comp rx=0 tx=0).stop 2024-03-04T08:11:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 -- 172.21.15.113:0/670508319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8540c8b90 msgr2=0x7fd85414de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8540c8b90 0x7fd85414de40 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd84001ca00 tx=0x7fd8400192c0 comp rx=0 tx=0).stop 2024-03-04T08:11:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 -- 172.21.15.113:0/670508319 shutdown_connections 2024-03-04T08:11:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd854152a50 0x7fd85414e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd834066f60 0x7fd834069420 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8540c8b90 0x7fd85414de40 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:12.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 --2- 172.21.15.113:0/670508319 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd85406ff20 0x7fd854154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 -- 172.21.15.113:0/670508319 >> 172.21.15.113:0/670508319 conn(0x7fd8540734d0 msgr2=0x7fd854066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 -- 172.21.15.113:0/670508319 shutdown_connections 2024-03-04T08:11:12.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:12.016+0000 7fd858c00640 1 -- 172.21.15.113:0/670508319 wait complete. 2024-03-04T08:11:12.041 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 139, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 139, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7389": { "addr": "172.21.15.113:6836/1310057122", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1310057122, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1310057122, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7389, "incarnation": 126, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7519": { "addr": "172.21.15.136:6835/247269911", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 247269911, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 247269911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7519, "incarnation": 133, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 36 }, "gid_7692": { "addr": "172.21.15.113:6833/1928887637", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 1928887637, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 1928887637, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7692, "incarnation": 139, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 69, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:11:10.891544+0000", "qdb_cluster": [ 7389, 7519 ], "qdb_leader": 7389, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7389, "mds_1": 7519, "mds_2": 7692 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:11:14.042 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:14.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.288+0000 7fbe05a00640 1 Processor -- start 2024-03-04T08:11:14.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.288+0000 7fbe05a00640 1 -- start start 2024-03-04T08:11:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.288+0000 7fbe05a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe001592d0 0x7fbe00071380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.288+0000 7fbe05a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe000c7120 0x7fbe00071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.288+0000 7fbe05a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbe000722d0 0x7fbe0015f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.288+0000 7fbe05a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe001626e0 con 0x7fbe000c7120 2024-03-04T08:11:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.288+0000 7fbe05a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe001629c0 con 0x7fbe000722d0 2024-03-04T08:11:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.288+0000 7fbe05a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00162ca0 con 0x7fbe001592d0 2024-03-04T08:11:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdfe000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe000c7120 0x7fbe00071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdff400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbe000722d0 0x7fbe0015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdfea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe001592d0 0x7fbe00071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdfe000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe000c7120 0x7fbe00071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56678/0 (socket says 172.21.15.113:56678) 2024-03-04T08:11:14.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdfe000640 1 -- 172.21.15.113:0/3415815592 learned_addr learned my addr 172.21.15.113:0/3415815592 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:14.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdff400640 1 -- 172.21.15.113:0/3415815592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe001592d0 msgr2=0x7fbe00071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:14.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdff400640 1 --2- 172.21.15.113:0/3415815592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe001592d0 0x7fbe00071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdff400640 1 -- 172.21.15.113:0/3415815592 >> v2:172.21.15.113:3300/0 conn(0x7fbe000c7120 msgr2=0x7fbe00071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:14.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdff400640 1 --2- 172.21.15.113:0/3415815592 >> v2:172.21.15.113:3300/0 conn(0x7fbe000c7120 0x7fbe00071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdff400640 1 -- 172.21.15.113:0/3415815592 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe00162f80 con 0x7fbe000722d0 2024-03-04T08:11:14.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdfe000640 1 --2- 172.21.15.113:0/3415815592 >> v2:172.21.15.113:3300/0 conn(0x7fbe000c7120 0x7fbe00071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:11:14.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdfea00640 1 --2- 172.21.15.113:0/3415815592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe001592d0 0x7fbe00071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:11:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.292+0000 7fbdff400640 1 --2- 172.21.15.113:0/3415815592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe000722d0 0x7fbe0015f490 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbe00160150 tx=0x7fbdf801c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ab329bd263838f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbdfd600640 1 -- 172.21.15.113:0/3415815592 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf80b2070 con 0x7fbe000722d0 2024-03-04T08:11:14.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbdfd600640 1 -- 172.21.15.113:0/3415815592 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf80a5880 con 0x7fbe000722d0 2024-03-04T08:11:14.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbdfd600640 1 -- 172.21.15.113:0/3415815592 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf80a5c50 con 0x7fbe000722d0 2024-03-04T08:11:14.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 -- 172.21.15.113:0/3415815592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe000722d0 msgr2=0x7fbe0015f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:14.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 --2- 172.21.15.113:0/3415815592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe000722d0 0x7fbe0015f490 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbe00160150 tx=0x7fbdf801c930 comp rx=0 tx=0).stop 2024-03-04T08:11:14.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 -- 172.21.15.113:0/3415815592 shutdown_connections 2024-03-04T08:11:14.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 --2- 172.21.15.113:0/3415815592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe000722d0 0x7fbe0015f490 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 --2- 172.21.15.113:0/3415815592 >> v2:172.21.15.113:3300/0 conn(0x7fbe000c7120 0x7fbe00071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 --2- 172.21.15.113:0/3415815592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe001592d0 0x7fbe00071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 -- 172.21.15.113:0/3415815592 >> 172.21.15.113:0/3415815592 conn(0x7fbe00067a40 msgr2=0x7fbe0006ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:14.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 -- 172.21.15.113:0/3415815592 shutdown_connections 2024-03-04T08:11:14.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 -- 172.21.15.113:0/3415815592 wait complete. 2024-03-04T08:11:14.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 Processor -- start 2024-03-04T08:11:14.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 -- start start 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe000722d0 0x7fbe000c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.296+0000 7fbe05a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe000c7120 0x7fbe0014cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbe05a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe001592d0 0x7fbe001f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbe05a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe001596b0 con 0x7fbe000722d0 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbe05a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000672d0 con 0x7fbe001592d0 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbe05a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe00159830 con 0x7fbe000c7120 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdfea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe000722d0 0x7fbe000c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdfe000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe000c7120 0x7fbe0014cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdfe000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe000c7120 0x7fbe0014cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:52852/0 (socket says 172.21.15.113:52852) 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdfe000640 1 -- 172.21.15.113:0/282568017 learned_addr learned my addr 172.21.15.113:0/282568017 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:14.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdff400640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe001592d0 0x7fbe001f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:14.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdfe000640 1 -- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe001592d0 msgr2=0x7fbe001f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:14.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdfe000640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe001592d0 0x7fbe001f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdfe000640 1 -- 172.21.15.113:0/282568017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe000722d0 msgr2=0x7fbe000c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:14.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdfe000640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe000722d0 0x7fbe000c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdfe000640 1 -- 172.21.15.113:0/282568017 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0014f400 con 0x7fbe000c7120 2024-03-04T08:11:14.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdff400640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe001592d0 0x7fbe001f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:11:14.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdfe000640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe000c7120 0x7fbe0014cce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00e8e0 tx=0x7fbdec00eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:14.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdf6a00640 1 -- 172.21.15.113:0/282568017 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec004030 con 0x7fbe000c7120 2024-03-04T08:11:14.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdf6a00640 1 -- 172.21.15.113:0/282568017 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdec0041d0 con 0x7fbe000c7120 2024-03-04T08:11:14.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbdf6a00640 1 -- 172.21.15.113:0/282568017 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00c940 con 0x7fbe000c7120 2024-03-04T08:11:14.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbe05a00640 1 -- 172.21.15.113:0/282568017 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe001601a0 con 0x7fbe000c7120 2024-03-04T08:11:14.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbe05a00640 1 -- 172.21.15.113:0/282568017 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe00160760 con 0x7fbe000c7120 2024-03-04T08:11:14.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.300+0000 7fbe05a00640 1 -- 172.21.15.113:0/282568017 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe00075c00 con 0x7fbe000c7120 2024-03-04T08:11:14.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.304+0000 7fbdf6a00640 1 -- 172.21.15.113:0/282568017 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbdec00cae0 con 0x7fbe000c7120 2024-03-04T08:11:14.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.304+0000 7fbdf6a00640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbde0066f60 0x7fbde0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:14.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.304+0000 7fbdf6a00640 1 -- 172.21.15.113:0/282568017 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7fbdec085ac0 con 0x7fbe000c7120 2024-03-04T08:11:14.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.304+0000 7fbdfea00640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbde0066f60 0x7fbde0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:14.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.304+0000 7fbdfea00640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbde0066f60 0x7fbde0069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009580 tx=0x7fbdf0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:14.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.312+0000 7fbdf6a00640 1 -- 172.21.15.113:0/282568017 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbdec017020 con 0x7fbe000c7120 2024-03-04T08:11:14.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 -- 172.21.15.113:0/282568017 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe001696c0 con 0x7fbe000c7120 2024-03-04T08:11:14.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbdf6a00640 1 -- 172.21.15.113:0/282568017 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+5820 (secure 0 0 0) 0x7fbdec055c70 con 0x7fbe000c7120 2024-03-04T08:11:14.588 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:14.588 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":139,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:10.891544+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7519]},"id":1}]} 2024-03-04T08:11:14.588 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 139 2024-03-04T08:11:14.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 -- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbde0066f60 msgr2=0x7fbde0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:14.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbde0066f60 0x7fbde0069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0009580 tx=0x7fbdf0009190 comp rx=0 tx=0).stop 2024-03-04T08:11:14.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 -- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe000c7120 msgr2=0x7fbe0014cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:14.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe000c7120 0x7fbe0014cce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00e8e0 tx=0x7fbdec00eda0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 -- 172.21.15.113:0/282568017 shutdown_connections 2024-03-04T08:11:14.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe001592d0 0x7fbe001f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe000c7120 0x7fbe0014cce0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbde0066f60 0x7fbde0069420 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 --2- 172.21.15.113:0/282568017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe000722d0 0x7fbe000c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:14.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 -- 172.21.15.113:0/282568017 >> 172.21.15.113:0/282568017 conn(0x7fbe00067a40 msgr2=0x7fbe0006ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:14.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 -- 172.21.15.113:0/282568017 shutdown_connections 2024-03-04T08:11:14.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:14.592+0000 7fbe05a00640 1 -- 172.21.15.113:0/282568017 wait complete. 2024-03-04T08:11:14.610 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:11:16.611 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:16.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.752+0000 7f22c0c00640 1 Processor -- start 2024-03-04T08:11:16.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.752+0000 7f22c0c00640 1 -- start start 2024-03-04T08:11:16.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22c0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22bc1527f0 0x7f22bc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:16.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22c0c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22bc152bd0 0x7f22bc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:16.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22c0c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc1591f0 con 0x7f22bc0c8200 2024-03-04T08:11:16.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22c0c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22bc071df0 con 0x7f22bc152bd0 2024-03-04T08:11:16.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22c0c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc1597b0 con 0x7f22bc1527f0 2024-03-04T08:11:16.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22b1400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f22bc0c8200 0x7f22bc072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55144/0 (socket says 172.21.15.113:55144) 2024-03-04T08:11:16.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22b1400640 1 -- 172.21.15.113:0/614145561 learned_addr learned my addr 172.21.15.113:0/614145561 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:11:16.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22ba800640 1 --2- 172.21.15.113:0/614145561 >> v2:172.21.15.113:3300/0 conn(0x7f22bc152bd0 0x7f22bc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:16.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22b9e00640 1 --2- 172.21.15.113:0/614145561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22bc1527f0 0x7f22bc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:16.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22b9400640 1 -- 172.21.15.113:0/614145561 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533259731 0 0) 0x7f22bc1591f0 con 0x7f22bc0c8200 2024-03-04T08:11:16.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22b9400640 1 -- 172.21.15.113:0/614145561 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a0003530 con 0x7f22bc0c8200 2024-03-04T08:11:16.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22ba800640 1 -- 172.21.15.113:0/614145561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22bc1527f0 msgr2=0x7f22bc070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:16.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22ba800640 1 --2- 172.21.15.113:0/614145561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22bc1527f0 0x7f22bc070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:16.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22ba800640 1 -- 172.21.15.113:0/614145561 >> v1:172.21.15.113:6789/0 conn(0x7f22bc0c8200 legacy=0x7f22bc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:16.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.756+0000 7f22ba800640 1 -- 172.21.15.113:0/614145561 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc1594d0 con 0x7f22bc152bd0 2024-03-04T08:11:16.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22ba800640 1 --2- 172.21.15.113:0/614145561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 0x7f22bc1587d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f22b40097c0 tx=0x7f22b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b7392a9538f450 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22b9400640 1 -- 172.21.15.113:0/614145561 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40b2070 con 0x7f22bc152bd0 2024-03-04T08:11:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22b9400640 1 -- 172.21.15.113:0/614145561 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b40ad640 con 0x7f22bc152bd0 2024-03-04T08:11:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22b9400640 1 -- 172.21.15.113:0/614145561 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40ada10 con 0x7f22bc152bd0 2024-03-04T08:11:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22c0c00640 1 -- 172.21.15.113:0/614145561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 msgr2=0x7f22bc1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22c0c00640 1 --2- 172.21.15.113:0/614145561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 0x7f22bc1587d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f22b40097c0 tx=0x7f22b401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:11:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22c0c00640 1 -- 172.21.15.113:0/614145561 shutdown_connections 2024-03-04T08:11:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22c0c00640 1 --2- 172.21.15.113:0/614145561 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 0x7f22bc1587d0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22c0c00640 1 --2- 172.21.15.113:0/614145561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22bc1527f0 0x7f22bc070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:16.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22c0c00640 1 -- 172.21.15.113:0/614145561 >> 172.21.15.113:0/614145561 conn(0x7f22bc06e820 msgr2=0x7f22bc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:16.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22c0c00640 1 -- 172.21.15.113:0/614145561 shutdown_connections 2024-03-04T08:11:16.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.760+0000 7f22c0c00640 1 -- 172.21.15.113:0/614145561 wait complete. 2024-03-04T08:11:16.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 Processor -- start 2024-03-04T08:11:16.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 -- start start 2024-03-04T08:11:16.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22bc0c8200 0x7f22bc1f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:16.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22bc1527f0 0x7f22bc15a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:16.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 0x7f22bc1f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:16.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc1654f0 con 0x7f22bc152bd0 2024-03-04T08:11:16.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc165370 con 0x7f22bc1527f0 2024-03-04T08:11:16.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc165670 con 0x7f22bc0c8200 2024-03-04T08:11:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22ba800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 0x7f22bc1f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22ba800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 0x7f22bc1f4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56706/0 (socket says 172.21.15.113:56706) 2024-03-04T08:11:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22ba800640 1 -- 172.21.15.113:0/794890255 learned_addr learned my addr 172.21.15.113:0/794890255 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22b9e00640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22bc0c8200 0x7f22bc1f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22b1400640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22bc1527f0 0x7f22bc15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:16.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22ba800640 1 -- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22bc0c8200 msgr2=0x7f22bc1f2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22ba800640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22bc0c8200 0x7f22bc1f2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22ba800640 1 -- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22bc1527f0 msgr2=0x7f22bc15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22ba800640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22bc1527f0 0x7f22bc15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22ba800640 1 -- 172.21.15.113:0/794890255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc15a890 con 0x7f22bc152bd0 2024-03-04T08:11:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22b1400640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22bc1527f0 0x7f22bc15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:11:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22ba800640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 0x7f22bc1f4ed0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f22b401a0d0 tx=0x7f22b4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22b2a00640 1 -- 172.21.15.113:0/794890255 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40b2040 con 0x7f22bc152bd0 2024-03-04T08:11:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22b2a00640 1 -- 172.21.15.113:0/794890255 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b40bc070 con 0x7f22bc152bd0 2024-03-04T08:11:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 -- 172.21.15.113:0/794890255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22bc14cbf0 con 0x7f22bc152bd0 2024-03-04T08:11:16.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22b2a00640 1 -- 172.21.15.113:0/794890255 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4006e20 con 0x7f22bc152bd0 2024-03-04T08:11:16.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 -- 172.21.15.113:0/794890255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc15bf00 con 0x7f22bc152bd0 2024-03-04T08:11:16.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.764+0000 7f22c0c00640 1 -- 172.21.15.113:0/794890255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22bc075c00 con 0x7f22bc152bd0 2024-03-04T08:11:16.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.768+0000 7f22b2a00640 1 -- 172.21.15.113:0/794890255 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22b40b6650 con 0x7f22bc152bd0 2024-03-04T08:11:16.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.768+0000 7f22b2a00640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 0x7f2298069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:16.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.768+0000 7f22b2a00640 1 -- 172.21.15.113:0/794890255 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f22b411f670 con 0x7f22bc152bd0 2024-03-04T08:11:16.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.768+0000 7f22b9e00640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 0x7f2298069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:16.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.768+0000 7f22b9e00640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 0x7f2298069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0113e0 tx=0x7f22ac00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:16.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.772+0000 7f22b2a00640 1 -- 172.21.15.113:0/794890255 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f22b4016090 con 0x7f22bc152bd0 2024-03-04T08:11:16.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.964+0000 7f22c0c00640 1 -- 172.21.15.113:0/794890255 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22bc0c79d0 con 0x7f22bc152bd0 2024-03-04T08:11:16.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.964+0000 7f22b2a00640 1 -- 172.21.15.113:0/794890255 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+5822 (secure 0 0 0) 0x7f22b40efdd0 con 0x7f22bc152bd0 2024-03-04T08:11:16.962 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:16.962 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:15.903460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389]},"id":1}]} 2024-03-04T08:11:16.963 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 140 2024-03-04T08:11:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 -- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 msgr2=0x7f2298069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 0x7f2298069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0113e0 tx=0x7f22ac00b040 comp rx=0 tx=0).stop 2024-03-04T08:11:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 -- 172.21.15.113:0/794890255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 msgr2=0x7f22bc1f4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 0x7f22bc1f4ed0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f22b401a0d0 tx=0x7f22b4019240 comp rx=0 tx=0).stop 2024-03-04T08:11:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 -- 172.21.15.113:0/794890255 shutdown_connections 2024-03-04T08:11:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2298066f60 0x7f2298069420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22bc152bd0 0x7f22bc1f4ed0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22bc1527f0 0x7f22bc15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:16.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 --2- 172.21.15.113:0/794890255 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22bc0c8200 0x7f22bc1f2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:16.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 -- 172.21.15.113:0/794890255 >> 172.21.15.113:0/794890255 conn(0x7f22bc06e820 msgr2=0x7f22bc0c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:16.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 -- 172.21.15.113:0/794890255 shutdown_connections 2024-03-04T08:11:16.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:16.968+0000 7f22c0c00640 1 -- 172.21.15.113:0/794890255 wait complete. 2024-03-04T08:11:16.983 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:11:18.984 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:19.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c7e00640 1 Processor -- start 2024-03-04T08:11:19.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c7e00640 1 -- start start 2024-03-04T08:11:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f90c0158010 0x7f90c006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c00ca0e0 con 0x7f90c00ce120 2024-03-04T08:11:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c015aa30 con 0x7f90c006ff20 2024-03-04T08:11:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c006f450 con 0x7f90c0158010 2024-03-04T08:11:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f90c0158010 0x7f90c006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c5e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f90c00ce120 0x7f90c01652d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45624/0 (socket says 172.21.15.113:45624) 2024-03-04T08:11:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c5e00640 1 -- 172.21.15.113:0/1138879443 learned_addr learned my addr 172.21.15.113:0/1138879443 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:11:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c5400640 1 -- 172.21.15.113:0/1138879443 >> v1:172.21.15.136:6789/0 conn(0x7f90c006ff20 legacy=0x7f90c015b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c5400640 1 -- 172.21.15.113:0/1138879443 >> v1:172.21.15.113:6789/0 conn(0x7f90c00ce120 legacy=0x7f90c01652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.124+0000 7f90c5400640 1 -- 172.21.15.113:0/1138879443 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c006c590 con 0x7f90c0158010 2024-03-04T08:11:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.128+0000 7f90c5400640 1 --2- 172.21.15.113:0/1138879443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 0x7f90c006dd50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f90b40097c0 tx=0x7f90b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a64f457aa4fd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.128+0000 7f90bfe00640 1 -- 172.21.15.113:0/1138879443 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40b0070 con 0x7f90c0158010 2024-03-04T08:11:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.128+0000 7f90bfe00640 1 -- 172.21.15.113:0/1138879443 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b40ab600 con 0x7f90c0158010 2024-03-04T08:11:19.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.128+0000 7f90bfe00640 1 -- 172.21.15.113:0/1138879443 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40ab9d0 con 0x7f90c0158010 2024-03-04T08:11:19.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.128+0000 7f90c7e00640 1 -- 172.21.15.113:0/1138879443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 msgr2=0x7f90c006dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:19.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.128+0000 7f90c7e00640 1 --2- 172.21.15.113:0/1138879443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 0x7f90c006dd50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f90b40097c0 tx=0x7f90b401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:11:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.132+0000 7f90c7e00640 1 -- 172.21.15.113:0/1138879443 shutdown_connections 2024-03-04T08:11:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.132+0000 7f90c7e00640 1 --2- 172.21.15.113:0/1138879443 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 0x7f90c006dd50 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.132+0000 7f90c7e00640 1 -- 172.21.15.113:0/1138879443 >> 172.21.15.113:0/1138879443 conn(0x7f90c00734d0 msgr2=0x7f90c0066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.132+0000 7f90c7e00640 1 -- 172.21.15.113:0/1138879443 shutdown_connections 2024-03-04T08:11:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.132+0000 7f90c7e00640 1 -- 172.21.15.113:0/1138879443 wait complete. 2024-03-04T08:11:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.132+0000 7f90c7e00640 1 Processor -- start 2024-03-04T08:11:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.132+0000 7f90c7e00640 1 -- start start 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90c006ff20 0x7f90c014dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90c00ce120 0x7f90c0154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 0x7f90c0155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c006e290 con 0x7f90c0158010 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c015ad80 con 0x7f90c00ce120 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c015b920 con 0x7f90c006ff20 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 0x7f90c0155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90c006ff20 0x7f90c014dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 0x7f90c0155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59664/0 (socket says 172.21.15.113:59664) 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c5e00640 1 -- 172.21.15.113:0/3535715418 learned_addr learned my addr 172.21.15.113:0/3535715418 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c4a00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90c00ce120 0x7f90c0154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c5e00640 1 -- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90c006ff20 msgr2=0x7f90c014dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c5e00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90c006ff20 0x7f90c014dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c5e00640 1 -- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90c00ce120 msgr2=0x7f90c0154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c5e00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90c00ce120 0x7f90c0154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c5e00640 1 -- 172.21.15.113:0/3535715418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c00aee20 con 0x7f90c0158010 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c4a00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90c00ce120 0x7f90c0154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c5e00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 0x7f90c0155020 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f90b800cbf0 tx=0x7f90b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90bd600640 1 -- 172.21.15.113:0/3535715418 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8017070 con 0x7f90c0158010 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90bd600640 1 -- 172.21.15.113:0/3535715418 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b800ad40 con 0x7f90c0158010 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90bd600640 1 -- 172.21.15.113:0/3535715418 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8012600 con 0x7f90c0158010 2024-03-04T08:11:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c7e00640 1 -- 172.21.15.113:0/3535715418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c0153d20 con 0x7f90c0158010 2024-03-04T08:11:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c7e00640 1 -- 172.21.15.113:0/3535715418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c01542e0 con 0x7f90c0158010 2024-03-04T08:11:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90c7e00640 1 -- 172.21.15.113:0/3535715418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c0075c00 con 0x7f90c0158010 2024-03-04T08:11:19.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.136+0000 7f90bd600640 1 -- 172.21.15.113:0/3535715418 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90b801a070 con 0x7f90c0158010 2024-03-04T08:11:19.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.140+0000 7f90bd600640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f90a0066f60 0x7f90a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:19.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.140+0000 7f90bd600640 1 -- 172.21.15.113:0/3535715418 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f90b80854e0 con 0x7f90c0158010 2024-03-04T08:11:19.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.140+0000 7f90c5400640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f90a0066f60 0x7f90a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:19.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.140+0000 7f90c5400640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f90a0066f60 0x7f90a0069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f90b40ad000 tx=0x7f90b40a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:19.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.144+0000 7f90bd600640 1 -- 172.21.15.113:0/3535715418 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f90b8055680 con 0x7f90c0158010 2024-03-04T08:11:19.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.428+0000 7f90c7e00640 1 -- 172.21.15.113:0/3535715418 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90c00c84c0 con 0x7f90c0158010 2024-03-04T08:11:19.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.428+0000 7f90bd600640 1 -- 172.21.15.113:0/3535715418 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+5822 (secure 0 0 0) 0x7f90b8059720 con 0x7f90c0158010 2024-03-04T08:11:19.424 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:19.424 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:15.903460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389]},"id":1}]} 2024-03-04T08:11:19.424 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 140 2024-03-04T08:11:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 -- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f90a0066f60 msgr2=0x7f90a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f90a0066f60 0x7f90a0069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f90b40ad000 tx=0x7f90b40a6000 comp rx=0 tx=0).stop 2024-03-04T08:11:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 -- 172.21.15.113:0/3535715418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 msgr2=0x7f90c0155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 0x7f90c0155020 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f90b800cbf0 tx=0x7f90b800a430 comp rx=0 tx=0).stop 2024-03-04T08:11:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 -- 172.21.15.113:0/3535715418 shutdown_connections 2024-03-04T08:11:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f90a0066f60 0x7f90a0069420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:19.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90c0158010 0x7f90c0155020 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90c00ce120 0x7f90c0154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 --2- 172.21.15.113:0/3535715418 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90c006ff20 0x7f90c014dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 -- 172.21.15.113:0/3535715418 >> 172.21.15.113:0/3535715418 conn(0x7f90c00734d0 msgr2=0x7f90c00711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 -- 172.21.15.113:0/3535715418 shutdown_connections 2024-03-04T08:11:19.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:19.432+0000 7f90c7e00640 1 -- 172.21.15.113:0/3535715418 wait complete. 2024-03-04T08:11:19.450 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:11:21.451 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.599+0000 7efd88e00640 1 Processor -- start 2024-03-04T08:11:21.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.599+0000 7efd88e00640 1 -- start start 2024-03-04T08:11:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd88e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd840cb540 0x7efd8415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd88e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd8406faa0 0x7efd840c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd88e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd840c4ce0 con 0x7efd84152950 2024-03-04T08:11:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd88e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd840c8370 con 0x7efd8406faa0 2024-03-04T08:11:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd88e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd840ca0a0 con 0x7efd840cb540 2024-03-04T08:11:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd83400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efd84152950 0x7efd8406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45626/0 (socket says 172.21.15.113:45626) 2024-03-04T08:11:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd83400640 1 -- 172.21.15.113:0/930393186 learned_addr learned my addr 172.21.15.113:0/930393186 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:11:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd83e00640 1 --2- 172.21.15.113:0/930393186 >> v2:172.21.15.113:3300/0 conn(0x7efd8406faa0 0x7efd840c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:21.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd7aa00640 1 --2- 172.21.15.113:0/930393186 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd840cb540 0x7efd8415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:21.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd82a00640 1 -- 172.21.15.113:0/930393186 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5824573 0 0) 0x7efd840c4ce0 con 0x7efd84152950 2024-03-04T08:11:21.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd82a00640 1 -- 172.21.15.113:0/930393186 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd68003530 con 0x7efd84152950 2024-03-04T08:11:21.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd83e00640 1 -- 172.21.15.113:0/930393186 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd840cb540 msgr2=0x7efd8415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:21.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd83e00640 1 --2- 172.21.15.113:0/930393186 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd840cb540 0x7efd8415bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:21.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd83e00640 1 -- 172.21.15.113:0/930393186 >> v1:172.21.15.113:6789/0 conn(0x7efd84152950 legacy=0x7efd8406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:21.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.603+0000 7efd83e00640 1 -- 172.21.15.113:0/930393186 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd840c4f80 con 0x7efd8406faa0 2024-03-04T08:11:21.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd83e00640 1 --2- 172.21.15.113:0/930393186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8406faa0 0x7efd840c30b0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009d90 tx=0x7efd7c01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e697d51c5da4a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:21.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd82a00640 1 -- 172.21.15.113:0/930393186 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0b4070 con 0x7efd8406faa0 2024-03-04T08:11:21.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd82a00640 1 -- 172.21.15.113:0/930393186 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c0af600 con 0x7efd8406faa0 2024-03-04T08:11:21.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd82a00640 1 -- 172.21.15.113:0/930393186 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0af9d0 con 0x7efd8406faa0 2024-03-04T08:11:21.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd88e00640 1 -- 172.21.15.113:0/930393186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8406faa0 msgr2=0x7efd840c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:21.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd88e00640 1 --2- 172.21.15.113:0/930393186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8406faa0 0x7efd840c30b0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009d90 tx=0x7efd7c01f150 comp rx=0 tx=0).stop 2024-03-04T08:11:21.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd88e00640 1 -- 172.21.15.113:0/930393186 shutdown_connections 2024-03-04T08:11:21.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd88e00640 1 --2- 172.21.15.113:0/930393186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd8406faa0 0x7efd840c30b0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:21.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd88e00640 1 --2- 172.21.15.113:0/930393186 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd840cb540 0x7efd8415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:21.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd88e00640 1 -- 172.21.15.113:0/930393186 >> 172.21.15.113:0/930393186 conn(0x7efd84072f40 msgr2=0x7efd840706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:21.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd88e00640 1 -- 172.21.15.113:0/930393186 shutdown_connections 2024-03-04T08:11:21.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.607+0000 7efd88e00640 1 -- 172.21.15.113:0/930393186 wait complete. 2024-03-04T08:11:21.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd88e00640 1 Processor -- start 2024-03-04T08:11:21.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd88e00640 1 -- start start 2024-03-04T08:11:21.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd88e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd8406faa0 0x7efd84159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd88e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840cb540 0x7efd8415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd88e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd84152950 0x7efd8415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd88e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd840c9a50 con 0x7efd840cb540 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd88e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd840c5220 con 0x7efd8406faa0 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd88e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd840c9d30 con 0x7efd84152950 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840cb540 0x7efd8415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840cb540 0x7efd8415ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59682/0 (socket says 172.21.15.113:59682) 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7aa00640 1 -- 172.21.15.113:0/3311337208 learned_addr learned my addr 172.21.15.113:0/3311337208 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd83400640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd8406faa0 0x7efd84159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd83e00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd84152950 0x7efd8415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7aa00640 1 -- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd84152950 msgr2=0x7efd8415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7aa00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd84152950 0x7efd8415b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:21.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7aa00640 1 -- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd8406faa0 msgr2=0x7efd84159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:21.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7aa00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd8406faa0 0x7efd84159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:21.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7aa00640 1 -- 172.21.15.113:0/3311337208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd84157110 con 0x7efd840cb540 2024-03-04T08:11:21.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd83400640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd8406faa0 0x7efd84159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:11:21.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7aa00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840cb540 0x7efd8415ae50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00e960 tx=0x7efd6c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:21.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7be00640 1 -- 172.21.15.113:0/3311337208 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c017070 con 0x7efd840cb540 2024-03-04T08:11:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7be00640 1 -- 172.21.15.113:0/3311337208 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c00ca00 con 0x7efd840cb540 2024-03-04T08:11:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd7be00640 1 -- 172.21.15.113:0/3311337208 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00cdb0 con 0x7efd840cb540 2024-03-04T08:11:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.611+0000 7efd88e00640 1 -- 172.21.15.113:0/3311337208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd841e1b20 con 0x7efd840cb540 2024-03-04T08:11:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.615+0000 7efd88e00640 1 -- 172.21.15.113:0/3311337208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd841e20e0 con 0x7efd840cb540 2024-03-04T08:11:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.615+0000 7efd88e00640 1 -- 172.21.15.113:0/3311337208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd84075a80 con 0x7efd840cb540 2024-03-04T08:11:21.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.615+0000 7efd7be00640 1 -- 172.21.15.113:0/3311337208 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd6c01a070 con 0x7efd840cb540 2024-03-04T08:11:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.615+0000 7efd7be00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd60066f60 0x7efd60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.615+0000 7efd7be00640 1 -- 172.21.15.113:0/3311337208 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7efd6c085870 con 0x7efd840cb540 2024-03-04T08:11:21.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.615+0000 7efd83400640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd60066f60 0x7efd60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:21.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.619+0000 7efd83400640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd60066f60 0x7efd60069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efd74001210 tx=0x7efd74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:21.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.623+0000 7efd7be00640 1 -- 172.21.15.113:0/3311337208 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efd6c056060 con 0x7efd840cb540 2024-03-04T08:11:21.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 -- 172.21.15.113:0/3311337208 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd840c9580 con 0x7efd840cb540 2024-03-04T08:11:21.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd7be00640 1 -- 172.21.15.113:0/3311337208 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5824 (secure 0 0 0) 0x7efd6c055960 con 0x7efd840cb540 2024-03-04T08:11:21.815 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:21.815 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:21.324967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7519]},"id":1}]} 2024-03-04T08:11:21.816 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 141 2024-03-04T08:11:21.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 -- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd60066f60 msgr2=0x7efd60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd60066f60 0x7efd60069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efd74001210 tx=0x7efd74009040 comp rx=0 tx=0).stop 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 -- 172.21.15.113:0/3311337208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840cb540 msgr2=0x7efd8415ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840cb540 0x7efd8415ae50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00e960 tx=0x7efd6c00ee20 comp rx=0 tx=0).stop 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 -- 172.21.15.113:0/3311337208 shutdown_connections 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd84152950 0x7efd8415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd60066f60 0x7efd60069420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd840cb540 0x7efd8415ae50 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 --2- 172.21.15.113:0/3311337208 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd8406faa0 0x7efd84159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 -- 172.21.15.113:0/3311337208 >> 172.21.15.113:0/3311337208 conn(0x7efd84072f40 msgr2=0x7efd841567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 -- 172.21.15.113:0/3311337208 shutdown_connections 2024-03-04T08:11:21.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:21.819+0000 7efd88e00640 1 -- 172.21.15.113:0/3311337208 wait complete. 2024-03-04T08:11:21.836 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:11:23.837 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:23.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05ab800640 1 Processor -- start 2024-03-04T08:11:23.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05ab800640 1 -- start start 2024-03-04T08:11:23.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05ab800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05a4155100 0x7f05a40c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05ab800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f05a415ad30 0x7f05a415b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05ab800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a415efd0 con 0x7f05a406b060 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05ab800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a40c8890 con 0x7f05a415ad30 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05ab800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a406b4a0 con 0x7f05a4155100 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05a3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f05a406b060 0x7f05a40cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45628/0 (socket says 172.21.15.113:45628) 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05a3e00640 1 -- 172.21.15.113:0/770544554 learned_addr learned my addr 172.21.15.113:0/770544554 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05a9800640 1 --2- 172.21.15.113:0/770544554 >> v2:172.21.15.113:3300/0 conn(0x7f05a415ad30 0x7f05a415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.975+0000 7f05a8e00640 1 --2- 172.21.15.113:0/770544554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05a4155100 0x7f05a40c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.979+0000 7f05a3400640 1 -- 172.21.15.113:0/770544554 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627553728 0 0) 0x7f05a415efd0 con 0x7f05a406b060 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.979+0000 7f05a3400640 1 -- 172.21.15.113:0/770544554 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f058c003530 con 0x7f05a406b060 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.979+0000 7f05a9800640 1 -- 172.21.15.113:0/770544554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05a4155100 msgr2=0x7f05a40c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.979+0000 7f05a9800640 1 --2- 172.21.15.113:0/770544554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05a4155100 0x7f05a40c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:23.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.979+0000 7f05a9800640 1 -- 172.21.15.113:0/770544554 >> v1:172.21.15.113:6789/0 conn(0x7f05a406b060 legacy=0x7f05a40cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:23.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.979+0000 7f05a9800640 1 -- 172.21.15.113:0/770544554 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a40c7ef0 con 0x7f05a415ad30 2024-03-04T08:11:23.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.979+0000 7f05a9800640 1 --2- 172.21.15.113:0/770544554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a415ad30 0x7f05a415b140 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f059c009d90 tx=0x7f059c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386484e1bd0ecc79 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:23.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.979+0000 7f05a3400640 1 -- 172.21.15.113:0/770544554 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c0b2070 con 0x7f05a415ad30 2024-03-04T08:11:23.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.979+0000 7f05a3400640 1 -- 172.21.15.113:0/770544554 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c0a5920 con 0x7f05a415ad30 2024-03-04T08:11:23.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.979+0000 7f05a3400640 1 -- 172.21.15.113:0/770544554 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c0a5cf0 con 0x7f05a415ad30 2024-03-04T08:11:23.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 -- 172.21.15.113:0/770544554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a415ad30 msgr2=0x7f05a415b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:23.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 --2- 172.21.15.113:0/770544554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a415ad30 0x7f05a415b140 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f059c009d90 tx=0x7f059c01c970 comp rx=0 tx=0).stop 2024-03-04T08:11:23.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 -- 172.21.15.113:0/770544554 shutdown_connections 2024-03-04T08:11:23.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 --2- 172.21.15.113:0/770544554 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a415ad30 0x7f05a415b140 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:23.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 --2- 172.21.15.113:0/770544554 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05a4155100 0x7f05a40c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:23.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 -- 172.21.15.113:0/770544554 >> 172.21.15.113:0/770544554 conn(0x7f05a406deb0 msgr2=0x7f05a406e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:23.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 -- 172.21.15.113:0/770544554 shutdown_connections 2024-03-04T08:11:23.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 -- 172.21.15.113:0/770544554 wait complete. 2024-03-04T08:11:23.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 Processor -- start 2024-03-04T08:11:23.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 -- start start 2024-03-04T08:11:23.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a406b060 0x7f05a4154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:23.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05a4155100 0x7f05a415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:23.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a415ad30 0x7f05a415cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:23.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a415ebb0 con 0x7f05a406b060 2024-03-04T08:11:23.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a415b680 con 0x7f05a415ad30 2024-03-04T08:11:23.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05ab800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a415ed30 con 0x7f05a4155100 2024-03-04T08:11:23.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a406b060 0x7f05a4154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:23.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a8e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a406b060 0x7f05a4154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59708/0 (socket says 172.21.15.113:59708) 2024-03-04T08:11:23.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a8e00640 1 -- 172.21.15.113:0/3557775572 learned_addr learned my addr 172.21.15.113:0/3557775572 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:23.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a3e00640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05a4155100 0x7f05a415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:23.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a9800640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a415ad30 0x7f05a415cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:23.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a8e00640 1 -- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05a4155100 msgr2=0x7f05a415c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:23.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a8e00640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05a4155100 0x7f05a415c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:23.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a8e00640 1 -- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a415ad30 msgr2=0x7f05a415cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:23.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a8e00640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a415ad30 0x7f05a415cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:23.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a8e00640 1 -- 172.21.15.113:0/3557775572 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a41f39f0 con 0x7f05a406b060 2024-03-04T08:11:23.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.983+0000 7f05a9800640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a415ad30 0x7f05a415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:11:23.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05a8e00640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a406b060 0x7f05a4154130 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f059800e930 tx=0x7f059800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:23.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05a0c00640 1 -- 172.21.15.113:0/3557775572 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598017070 con 0x7f05a406b060 2024-03-04T08:11:23.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05a0c00640 1 -- 172.21.15.113:0/3557775572 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059800ca00 con 0x7f05a406b060 2024-03-04T08:11:23.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05a0c00640 1 -- 172.21.15.113:0/3557775572 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800cdb0 con 0x7f05a406b060 2024-03-04T08:11:23.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05ab800640 1 -- 172.21.15.113:0/3557775572 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a41f3cb0 con 0x7f05a406b060 2024-03-04T08:11:23.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05ab800640 1 -- 172.21.15.113:0/3557775572 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a41f4270 con 0x7f05a406b060 2024-03-04T08:11:23.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05ab800640 1 -- 172.21.15.113:0/3557775572 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a4075c00 con 0x7f05a406b060 2024-03-04T08:11:23.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05a0c00640 1 -- 172.21.15.113:0/3557775572 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f059801a070 con 0x7f05a406b060 2024-03-04T08:11:23.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05a0c00640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0584066f60 0x7f0584069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:23.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05a0c00640 1 -- 172.21.15.113:0/3557775572 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f05980857d0 con 0x7f05a406b060 2024-03-04T08:11:23.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.987+0000 7f05a3e00640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0584066f60 0x7f0584069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:23.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.991+0000 7f05a3e00640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0584066f60 0x7f0584069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0590007fd0 tx=0x7f0590009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:23.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:23.991+0000 7f05a0c00640 1 -- 172.21.15.113:0/3557775572 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0598055a30 con 0x7f05a406b060 2024-03-04T08:11:24.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 -- 172.21.15.113:0/3557775572 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05a40c8a20 con 0x7f05a406b060 2024-03-04T08:11:24.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05a0c00640 1 -- 172.21.15.113:0/3557775572 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f0598059ad0 con 0x7f05a406b060 2024-03-04T08:11:24.215 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:24.215 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:21.324967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7519]},"id":1}]} 2024-03-04T08:11:24.216 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 141 2024-03-04T08:11:24.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 -- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0584066f60 msgr2=0x7f0584069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:24.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0584066f60 0x7f0584069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0590007fd0 tx=0x7f0590009040 comp rx=0 tx=0).stop 2024-03-04T08:11:24.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 -- 172.21.15.113:0/3557775572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a406b060 msgr2=0x7f05a4154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:24.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a406b060 0x7f05a4154130 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f059800e930 tx=0x7f059800edf0 comp rx=0 tx=0).stop 2024-03-04T08:11:24.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 -- 172.21.15.113:0/3557775572 shutdown_connections 2024-03-04T08:11:24.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05a415ad30 0x7f05a415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:24.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05a4155100 0x7f05a415c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:24.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0584066f60 0x7f0584069420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:24.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 --2- 172.21.15.113:0/3557775572 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a406b060 0x7f05a4154130 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:24.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 -- 172.21.15.113:0/3557775572 >> 172.21.15.113:0/3557775572 conn(0x7f05a406deb0 msgr2=0x7f05a4151d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:24.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 -- 172.21.15.113:0/3557775572 shutdown_connections 2024-03-04T08:11:24.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:24.219+0000 7f05ab800640 1 -- 172.21.15.113:0/3557775572 wait complete. 2024-03-04T08:11:24.237 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 141, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 141, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7389": { "addr": "172.21.15.113:6836/1310057122", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 1310057122, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 1310057122, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7389, "incarnation": 126, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7519": { "addr": "172.21.15.136:6835/247269911", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 247269911, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 247269911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7519, "incarnation": 133, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 36 }, "gid_7692": { "addr": "172.21.15.113:6833/1928887637", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 1928887637, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 1928887637, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7692, "incarnation": 139, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:reconnect", "state_seq": 18 } }, "last_failure": 0, "last_failure_osd_epoch": 69, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:11:21.324967+0000", "qdb_cluster": [ 7389, 7519 ], "qdb_leader": 7389, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7389, "mds_1": 7519, "mds_2": 7692 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:11:26.238 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:26.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb28fe00640 1 Processor -- start 2024-03-04T08:11:26.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb28fe00640 1 -- start start 2024-03-04T08:11:26.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb28fe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb29015b510 0x7fb290071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:26.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb28fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2900c3190 0x7fb2900c8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:26.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb28fe00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2900c2410 con 0x7fb290154940 2024-03-04T08:11:26.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb28fe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb290154d20 con 0x7fb29015b510 2024-03-04T08:11:26.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb28fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900c8040 con 0x7fb2900c3190 2024-03-04T08:11:26.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb287e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb29015b510 0x7fb290071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:26.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb28f400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2900c3190 0x7fb2900c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:26.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb287e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb29015b510 0x7fb290071a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:32996/0 (socket says 172.21.15.113:32996) 2024-03-04T08:11:26.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb287e00640 1 -- 172.21.15.113:0/1156135479 learned_addr learned my addr 172.21.15.113:0/1156135479 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:26.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb287e00640 1 -- 172.21.15.113:0/1156135479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2900c3190 msgr2=0x7fb2900c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:26.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb287e00640 1 --2- 172.21.15.113:0/1156135479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2900c3190 0x7fb2900c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:26.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb287e00640 1 -- 172.21.15.113:0/1156135479 >> v1:172.21.15.136:6789/0 conn(0x7fb290154940 legacy=0x7fb2900c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:26.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.379+0000 7fb287e00640 1 -- 172.21.15.113:0/1156135479 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2900c3600 con 0x7fb29015b510 2024-03-04T08:11:26.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb287e00640 1 --2- 172.21.15.113:0/1156135479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb29015b510 0x7fb290071a30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb278009870 tx=0x7fb27801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c19ae18a3be25d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:26.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28e000640 1 -- 172.21.15.113:0/1156135479 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2780b2070 con 0x7fb29015b510 2024-03-04T08:11:26.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28e000640 1 -- 172.21.15.113:0/1156135479 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2780a9070 con 0x7fb29015b510 2024-03-04T08:11:26.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28e000640 1 -- 172.21.15.113:0/1156135479 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2780ad880 con 0x7fb29015b510 2024-03-04T08:11:26.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28fe00640 1 -- 172.21.15.113:0/1156135479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb29015b510 msgr2=0x7fb290071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:26.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28fe00640 1 --2- 172.21.15.113:0/1156135479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb29015b510 0x7fb290071a30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb278009870 tx=0x7fb27801c560 comp rx=0 tx=0).stop 2024-03-04T08:11:26.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28fe00640 1 -- 172.21.15.113:0/1156135479 shutdown_connections 2024-03-04T08:11:26.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28fe00640 1 --2- 172.21.15.113:0/1156135479 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb2900c3190 0x7fb2900c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:26.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28fe00640 1 --2- 172.21.15.113:0/1156135479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb29015b510 0x7fb290071a30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:26.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28fe00640 1 -- 172.21.15.113:0/1156135479 >> 172.21.15.113:0/1156135479 conn(0x7fb2900c9a30 msgr2=0x7fb2900c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:26.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28fe00640 1 -- 172.21.15.113:0/1156135479 shutdown_connections 2024-03-04T08:11:26.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.383+0000 7fb28fe00640 1 -- 172.21.15.113:0/1156135479 wait complete. 2024-03-04T08:11:26.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28fe00640 1 Processor -- start 2024-03-04T08:11:26.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28fe00640 1 -- start start 2024-03-04T08:11:26.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2900c3190 0x7fb2900c4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:26.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb290154940 0x7fb29014f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:26.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29015b510 0x7fb2901f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:26.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900c8fb0 con 0x7fb29015b510 2024-03-04T08:11:26.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2900c2990 con 0x7fb2900c3190 2024-03-04T08:11:26.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb290167980 con 0x7fb290154940 2024-03-04T08:11:26.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29015b510 0x7fb2901f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:26.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2900c3190 0x7fb2900c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:26.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb287e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb290154940 0x7fb29014f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:26.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2900c3190 0x7fb2900c4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:33006/0 (socket says 172.21.15.113:33006) 2024-03-04T08:11:26.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28ea00640 1 -- 172.21.15.113:0/1812665412 learned_addr learned my addr 172.21.15.113:0/1812665412 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:26.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28f400640 1 -- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb290154940 msgr2=0x7fb29014f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:26.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28f400640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb290154940 0x7fb29014f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:26.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28f400640 1 -- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2900c3190 msgr2=0x7fb2900c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:26.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28f400640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2900c3190 0x7fb2900c4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:26.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28f400640 1 -- 172.21.15.113:0/1812665412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2900c4a40 con 0x7fb29015b510 2024-03-04T08:11:26.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28ea00640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2900c3190 0x7fb2900c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:11:26.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28f400640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29015b510 0x7fb2901f6cc0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fb28800efc0 tx=0x7fb28800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:26.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb286a00640 1 -- 172.21.15.113:0/1812665412 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288018020 con 0x7fb29015b510 2024-03-04T08:11:26.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb286a00640 1 -- 172.21.15.113:0/1812665412 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb288014480 con 0x7fb29015b510 2024-03-04T08:11:26.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28fe00640 1 -- 172.21.15.113:0/1812665412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2901557e0 con 0x7fb29015b510 2024-03-04T08:11:26.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb286a00640 1 -- 172.21.15.113:0/1812665412 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288014870 con 0x7fb29015b510 2024-03-04T08:11:26.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.387+0000 7fb28fe00640 1 -- 172.21.15.113:0/1812665412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb290155da0 con 0x7fb29015b510 2024-03-04T08:11:26.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.391+0000 7fb28fe00640 1 -- 172.21.15.113:0/1812665412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb290075a80 con 0x7fb29015b510 2024-03-04T08:11:26.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.391+0000 7fb286a00640 1 -- 172.21.15.113:0/1812665412 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb28801b020 con 0x7fb29015b510 2024-03-04T08:11:26.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.391+0000 7fb286a00640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 0x7fb26c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:26.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.391+0000 7fb286a00640 1 -- 172.21.15.113:0/1812665412 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7fb288086650 con 0x7fb29015b510 2024-03-04T08:11:26.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.391+0000 7fb28ea00640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 0x7fb26c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:26.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.391+0000 7fb28ea00640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 0x7fb26c069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb280001cf0 tx=0x7fb280009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:26.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.395+0000 7fb286a00640 1 -- 172.21.15.113:0/1812665412 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb288022020 con 0x7fb29015b510 2024-03-04T08:11:26.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.639+0000 7fb28fe00640 1 -- 172.21.15.113:0/1812665412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2900ca190 con 0x7fb29015b510 2024-03-04T08:11:26.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.639+0000 7fb286a00640 1 -- 172.21.15.113:0/1812665412 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5824 (secure 0 0 0) 0x7fb288010040 con 0x7fb29015b510 2024-03-04T08:11:26.637 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:26.637 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:21.324967+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7519]},"id":1}]} 2024-03-04T08:11:26.638 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 141 2024-03-04T08:11:26.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 -- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 msgr2=0x7fb26c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:26.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 0x7fb26c069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb280001cf0 tx=0x7fb280009040 comp rx=0 tx=0).stop 2024-03-04T08:11:26.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 -- 172.21.15.113:0/1812665412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29015b510 msgr2=0x7fb2901f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:26.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29015b510 0x7fb2901f6cc0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fb28800efc0 tx=0x7fb28800c490 comp rx=0 tx=0).stop 2024-03-04T08:11:26.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 -- 172.21.15.113:0/1812665412 shutdown_connections 2024-03-04T08:11:26.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb26c066f60 0x7fb26c069420 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:26.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb29015b510 0x7fb2901f6cc0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:26.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb290154940 0x7fb29014f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:26.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 --2- 172.21.15.113:0/1812665412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2900c3190 0x7fb2900c4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:26.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 -- 172.21.15.113:0/1812665412 >> 172.21.15.113:0/1812665412 conn(0x7fb2900c9a30 msgr2=0x7fb290072690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:26.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 -- 172.21.15.113:0/1812665412 shutdown_connections 2024-03-04T08:11:26.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:26.643+0000 7fb28fe00640 1 -- 172.21.15.113:0/1812665412 wait complete. 2024-03-04T08:11:26.658 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:11:28.659 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:28.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bf600640 1 Processor -- start 2024-03-04T08:11:28.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bf600640 1 -- start start 2024-03-04T08:11:28.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bf600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 0x7fb4b81640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:28.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bf600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b8072d80 con 0x7fb4b8152c60 2024-03-04T08:11:28.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bf600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b8073510 con 0x7fb4b80c7a80 2024-03-04T08:11:28.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bf600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b806b060 con 0x7fb4b806cb60 2024-03-04T08:11:28.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bcc00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb4b8152c60 0x7fb4b815bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53130/0 (socket says 172.21.15.113:53130) 2024-03-04T08:11:28.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bcc00640 1 -- 172.21.15.113:0/28528858 learned_addr learned my addr 172.21.15.113:0/28528858 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:11:28.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bd600640 1 --2- 172.21.15.113:0/28528858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 0x7fb4b81640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:28.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4b7400640 1 -- 172.21.15.113:0/28528858 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065089504 0 0) 0x7fb4b8072d80 con 0x7fb4b8152c60 2024-03-04T08:11:28.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4b7400640 1 -- 172.21.15.113:0/28528858 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a0003530 con 0x7fb4b8152c60 2024-03-04T08:11:28.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4b7400640 1 -- 172.21.15.113:0/28528858 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491087413 0 0) 0x7fb4b8073510 con 0x7fb4b80c7a80 2024-03-04T08:11:28.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4b7400640 1 -- 172.21.15.113:0/28528858 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b8072d80 con 0x7fb4b80c7a80 2024-03-04T08:11:28.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bd600640 1 -- 172.21.15.113:0/28528858 >> v1:172.21.15.136:6789/0 conn(0x7fb4b80c7a80 legacy=0x7fb4b80cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:28.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bd600640 1 -- 172.21.15.113:0/28528858 >> v1:172.21.15.113:6789/0 conn(0x7fb4b8152c60 legacy=0x7fb4b815bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:28.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4b7400640 1 -- 172.21.15.113:0/28528858 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2146246731 0 0) 0x7fb4a0003530 con 0x7fb4b8152c60 2024-03-04T08:11:28.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.799+0000 7fb4bd600640 1 -- 172.21.15.113:0/28528858 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b806b2b0 con 0x7fb4b806cb60 2024-03-04T08:11:28.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4bd600640 1 --2- 172.21.15.113:0/28528858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 0x7fb4b81640e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009dc0 tx=0x7fb4b001c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab8367a15206564e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:28.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4b7400640 1 -- 172.21.15.113:0/28528858 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b00b2070 con 0x7fb4b806cb60 2024-03-04T08:11:28.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4b7400640 1 -- 172.21.15.113:0/28528858 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b00a56c0 con 0x7fb4b806cb60 2024-03-04T08:11:28.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4b7400640 1 -- 172.21.15.113:0/28528858 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac001680 con 0x7fb4b806cb60 2024-03-04T08:11:28.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4bf600640 1 -- 172.21.15.113:0/28528858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 msgr2=0x7fb4b81640e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:28.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4bf600640 1 --2- 172.21.15.113:0/28528858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 0x7fb4b81640e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009dc0 tx=0x7fb4b001c880 comp rx=0 tx=0).stop 2024-03-04T08:11:28.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4bf600640 1 -- 172.21.15.113:0/28528858 shutdown_connections 2024-03-04T08:11:28.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4bf600640 1 --2- 172.21.15.113:0/28528858 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 0x7fb4b81640e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:28.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4bf600640 1 -- 172.21.15.113:0/28528858 >> 172.21.15.113:0/28528858 conn(0x7fb4b80671b0 msgr2=0x7fb4b806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:28.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4bf600640 1 -- 172.21.15.113:0/28528858 shutdown_connections 2024-03-04T08:11:28.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.803+0000 7fb4bf600640 1 -- 172.21.15.113:0/28528858 wait complete. 2024-03-04T08:11:28.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bf600640 1 Processor -- start 2024-03-04T08:11:28.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bf600640 1 -- start start 2024-03-04T08:11:28.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bf600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 0x7fb4b8161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:28.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bf600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b80c7a80 0x7fb4b80c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:28.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bf600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b8152c60 0x7fb4b8158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:28.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bf600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80731d0 con 0x7fb4b80c7a80 2024-03-04T08:11:28.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bf600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b806b590 con 0x7fb4b8152c60 2024-03-04T08:11:28.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bf600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80cd720 con 0x7fb4b806cb60 2024-03-04T08:11:28.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b80c7a80 0x7fb4b80c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:28.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b80c7a80 0x7fb4b80c20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46432/0 (socket says 172.21.15.113:46432) 2024-03-04T08:11:28.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b7e00640 1 -- 172.21.15.113:0/2497351128 learned_addr learned my addr 172.21.15.113:0/2497351128 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:28.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bd600640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b8152c60 0x7fb4b8158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:28.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bcc00640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 0x7fb4b8161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:28.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b7e00640 1 -- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 msgr2=0x7fb4b8161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:28.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b7e00640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 0x7fb4b8161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:28.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b7e00640 1 -- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b8152c60 msgr2=0x7fb4b8158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:28.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b7e00640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b8152c60 0x7fb4b8158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:28.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b7e00640 1 -- 172.21.15.113:0/2497351128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80c2ed0 con 0x7fb4b80c7a80 2024-03-04T08:11:28.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bcc00640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 0x7fb4b8161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:11:28.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b7e00640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b80c7a80 0x7fb4b80c20a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400c490 tx=0x7fb4a400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:28.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b4c00640 1 -- 172.21.15.113:0/2497351128 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40052d0 con 0x7fb4b80c7a80 2024-03-04T08:11:28.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b4c00640 1 -- 172.21.15.113:0/2497351128 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a4005470 con 0x7fb4b80c7a80 2024-03-04T08:11:28.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bf600640 1 -- 172.21.15.113:0/2497351128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b8163440 con 0x7fb4b80c7a80 2024-03-04T08:11:28.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4b4c00640 1 -- 172.21.15.113:0/2497351128 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4012870 con 0x7fb4b80c7a80 2024-03-04T08:11:28.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.807+0000 7fb4bf600640 1 -- 172.21.15.113:0/2497351128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b80cc720 con 0x7fb4b80c7a80 2024-03-04T08:11:28.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.811+0000 7fb4bf600640 1 -- 172.21.15.113:0/2497351128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b8075c00 con 0x7fb4b80c7a80 2024-03-04T08:11:28.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.811+0000 7fb4b4c00640 1 -- 172.21.15.113:0/2497351128 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4a4012a10 con 0x7fb4b80c7a80 2024-03-04T08:11:28.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.811+0000 7fb4b4c00640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb498066f90 0x7fb498069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:28.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.811+0000 7fb4b4c00640 1 -- 172.21.15.113:0/2497351128 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7fb4a4085330 con 0x7fb4b80c7a80 2024-03-04T08:11:28.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.811+0000 7fb4bcc00640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb498066f90 0x7fb498069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:28.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.811+0000 7fb4bcc00640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb498066f90 0x7fb498069450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0011f0 tx=0x7fb4ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:28.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:28.815+0000 7fb4b4c00640 1 -- 172.21.15.113:0/2497351128 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb4a4055bb0 con 0x7fb4b80c7a80 2024-03-04T08:11:29.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.015+0000 7fb4bf600640 1 -- 172.21.15.113:0/2497351128 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4b80c30a0 con 0x7fb4b80c7a80 2024-03-04T08:11:29.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.015+0000 7fb4b4c00640 1 -- 172.21.15.113:0/2497351128 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) v1 ==== 95+0+5821 (secure 0 0 0) 0x7fb4a40554b0 con 0x7fb4b80c7a80 2024-03-04T08:11:29.014 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:29.014 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":142,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:26.494962+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389]},"id":1}]} 2024-03-04T08:11:29.015 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 142 2024-03-04T08:11:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 -- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb498066f90 msgr2=0x7fb498069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb498066f90 0x7fb498069450 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0011f0 tx=0x7fb4ac009040 comp rx=0 tx=0).stop 2024-03-04T08:11:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 -- 172.21.15.113:0/2497351128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b80c7a80 msgr2=0x7fb4b80c20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b80c7a80 0x7fb4b80c20a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400c490 tx=0x7fb4a400c950 comp rx=0 tx=0).stop 2024-03-04T08:11:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 -- 172.21.15.113:0/2497351128 shutdown_connections 2024-03-04T08:11:29.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b8152c60 0x7fb4b8158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb498066f90 0x7fb498069450 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b80c7a80 0x7fb4b80c20a0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 --2- 172.21.15.113:0/2497351128 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b806cb60 0x7fb4b8161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 -- 172.21.15.113:0/2497351128 >> 172.21.15.113:0/2497351128 conn(0x7fb4b80671b0 msgr2=0x7fb4b815bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 -- 172.21.15.113:0/2497351128 shutdown_connections 2024-03-04T08:11:29.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:29.019+0000 7fb4bf600640 1 -- 172.21.15.113:0/2497351128 wait complete. 2024-03-04T08:11:29.035 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:11:31.037 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:31.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a4d800640 1 Processor -- start 2024-03-04T08:11:31.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a4d800640 1 -- start start 2024-03-04T08:11:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a4d800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a48152bd0 0x7f6a480c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a4d800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a480c9df0 0x7f6a480ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a4d800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a480c9490 con 0x7f6a480c4fd0 2024-03-04T08:11:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a4d800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a480c9c50 con 0x7f6a480c9df0 2024-03-04T08:11:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a4d800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c97c0 con 0x7f6a48152bd0 2024-03-04T08:11:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a46a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a48152bd0 0x7f6a480c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a47400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6a480c9df0 0x7f6a480ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a46a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a48152bd0 0x7f6a480c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:55180/0 (socket says 172.21.15.113:55180) 2024-03-04T08:11:31.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.175+0000 7f6a46a00640 1 -- 172.21.15.113:0/4220693927 learned_addr learned my addr 172.21.15.113:0/4220693927 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.179+0000 7f6a47400640 1 -- 172.21.15.113:0/4220693927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a48152bd0 msgr2=0x7f6a480c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.179+0000 7f6a47400640 1 --2- 172.21.15.113:0/4220693927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a48152bd0 0x7f6a480c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.179+0000 7f6a47400640 1 -- 172.21.15.113:0/4220693927 >> v1:172.21.15.136:6789/0 conn(0x7f6a480c4fd0 legacy=0x7f6a48158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:31.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.179+0000 7f6a47400640 1 -- 172.21.15.113:0/4220693927 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a480c54a0 con 0x7f6a480c9df0 2024-03-04T08:11:31.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a47400640 1 --2- 172.21.15.113:0/4220693927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a480c9df0 0x7f6a480ce270 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009d90 tx=0x7f6a4001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ec81352a7e3afd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:31.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a46000640 1 -- 172.21.15.113:0/4220693927 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a400b2070 con 0x7f6a480c9df0 2024-03-04T08:11:31.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a46000640 1 -- 172.21.15.113:0/4220693927 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a400a5910 con 0x7f6a480c9df0 2024-03-04T08:11:31.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a46000640 1 -- 172.21.15.113:0/4220693927 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a400a5ce0 con 0x7f6a480c9df0 2024-03-04T08:11:31.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a4d800640 1 -- 172.21.15.113:0/4220693927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a480c9df0 msgr2=0x7f6a480ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:31.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a4d800640 1 --2- 172.21.15.113:0/4220693927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a480c9df0 0x7f6a480ce270 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009d90 tx=0x7f6a4001c940 comp rx=0 tx=0).stop 2024-03-04T08:11:31.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a4d800640 1 -- 172.21.15.113:0/4220693927 shutdown_connections 2024-03-04T08:11:31.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a4d800640 1 --2- 172.21.15.113:0/4220693927 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a480c9df0 0x7f6a480ce270 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:31.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a4d800640 1 --2- 172.21.15.113:0/4220693927 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a48152bd0 0x7f6a480c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:31.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a4d800640 1 -- 172.21.15.113:0/4220693927 >> 172.21.15.113:0/4220693927 conn(0x7f6a48066db0 msgr2=0x7f6a480671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:31.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a4d800640 1 -- 172.21.15.113:0/4220693927 shutdown_connections 2024-03-04T08:11:31.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a4d800640 1 -- 172.21.15.113:0/4220693927 wait complete. 2024-03-04T08:11:31.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a4d800640 1 Processor -- start 2024-03-04T08:11:31.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.183+0000 7f6a4d800640 1 -- start start 2024-03-04T08:11:31.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a4d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a480c4fd0 0x7f6a481457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a4d800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a480c9df0 0x7f6a48161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a4d800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a48152bd0 0x7f6a4806d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a4d800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c3b10 con 0x7f6a48152bd0 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a4d800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480c3990 con 0x7f6a480c4fd0 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a4d800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a48152fb0 con 0x7f6a480c9df0 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a47400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a48152bd0 0x7f6a4806d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a47400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a48152bd0 0x7f6a4806d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46458/0 (socket says 172.21.15.113:46458) 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a47400640 1 -- 172.21.15.113:0/403666639 learned_addr learned my addr 172.21.15.113:0/403666639 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a46a00640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a480c4fd0 0x7f6a481457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a3fe00640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a480c9df0 0x7f6a48161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a47400640 1 -- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a480c9df0 msgr2=0x7f6a48161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:31.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a47400640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a480c9df0 0x7f6a48161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:31.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a47400640 1 -- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a480c4fd0 msgr2=0x7f6a481457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:31.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a47400640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a480c4fd0 0x7f6a481457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:31.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a47400640 1 -- 172.21.15.113:0/403666639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a481525e0 con 0x7f6a48152bd0 2024-03-04T08:11:31.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a47400640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a48152bd0 0x7f6a4806d010 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f6a400b0040 tx=0x7f6a400196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:31.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a3ea00640 1 -- 172.21.15.113:0/403666639 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a400b2050 con 0x7f6a48152bd0 2024-03-04T08:11:31.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a3ea00640 1 -- 172.21.15.113:0/403666639 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a400be070 con 0x7f6a48152bd0 2024-03-04T08:11:31.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a4d800640 1 -- 172.21.15.113:0/403666639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4806da50 con 0x7f6a48152bd0 2024-03-04T08:11:31.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a3ea00640 1 -- 172.21.15.113:0/403666639 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a400ada90 con 0x7f6a48152bd0 2024-03-04T08:11:31.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a4d800640 1 -- 172.21.15.113:0/403666639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4814f200 con 0x7f6a48152bd0 2024-03-04T08:11:31.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.187+0000 7f6a4d800640 1 -- 172.21.15.113:0/403666639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4806df70 con 0x7f6a48152bd0 2024-03-04T08:11:31.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.191+0000 7f6a3ea00640 1 -- 172.21.15.113:0/403666639 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a400adc30 con 0x7f6a48152bd0 2024-03-04T08:11:31.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.191+0000 7f6a3ea00640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6a24066f60 0x7f6a24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:31.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.191+0000 7f6a3ea00640 1 -- 172.21.15.113:0/403666639 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f6a401208b0 con 0x7f6a48152bd0 2024-03-04T08:11:31.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.191+0000 7f6a46a00640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6a24066f60 0x7f6a24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:31.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.191+0000 7f6a46a00640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6a24066f60 0x7f6a24069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009b50 tx=0x7f6a38009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:31.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.199+0000 7f6a3ea00640 1 -- 172.21.15.113:0/403666639 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6a4806df70 con 0x7f6a48152bd0 2024-03-04T08:11:31.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.387+0000 7f6a4d800640 1 -- 172.21.15.113:0/403666639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a48075c00 con 0x7f6a48152bd0 2024-03-04T08:11:31.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.387+0000 7f6a3ea00640 1 -- 172.21.15.113:0/403666639 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) v1 ==== 95+0+5821 (secure 0 0 0) 0x7f6a40016560 con 0x7f6a48152bd0 2024-03-04T08:11:31.385 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:31.385 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":142,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:26.494962+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7519,7389]},"id":1}]} 2024-03-04T08:11:31.385 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 142 2024-03-04T08:11:31.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 -- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6a24066f60 msgr2=0x7f6a24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:31.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6a24066f60 0x7f6a24069420 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009b50 tx=0x7f6a38009240 comp rx=0 tx=0).stop 2024-03-04T08:11:31.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 -- 172.21.15.113:0/403666639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a48152bd0 msgr2=0x7f6a4806d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:31.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a48152bd0 0x7f6a4806d010 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f6a400b0040 tx=0x7f6a400196e0 comp rx=0 tx=0).stop 2024-03-04T08:11:31.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 -- 172.21.15.113:0/403666639 shutdown_connections 2024-03-04T08:11:31.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6a24066f60 0x7f6a24069420 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:31.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6a48152bd0 0x7f6a4806d010 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:31.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6a480c9df0 0x7f6a48161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:31.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 --2- 172.21.15.113:0/403666639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a480c4fd0 0x7f6a481457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:31.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 -- 172.21.15.113:0/403666639 >> 172.21.15.113:0/403666639 conn(0x7f6a48066db0 msgr2=0x7f6a480b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:31.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 -- 172.21.15.113:0/403666639 shutdown_connections 2024-03-04T08:11:31.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:31.391+0000 7f6a4d800640 1 -- 172.21.15.113:0/403666639 wait complete. 2024-03-04T08:11:31.406 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:11:33.407 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a10600640 1 Processor -- start 2024-03-04T08:11:33.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a10600640 1 -- start start 2024-03-04T08:11:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a10600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a08070120 0x7f2a081666b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a10600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0806c860 con 0x7f2a080c8d80 2024-03-04T08:11:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a10600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0806f8d0 con 0x7f2a080ce570 2024-03-04T08:11:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a10600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0806f310 con 0x7f2a08070120 2024-03-04T08:11:33.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a0e600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a08070120 0x7f2a081666b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a0e600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a08070120 0x7f2a081666b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46470/0 (socket says 172.21.15.113:46470) 2024-03-04T08:11:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a0e600640 1 -- 172.21.15.113:0/107483101 learned_addr learned my addr 172.21.15.113:0/107483101 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a0e600640 1 -- 172.21.15.113:0/107483101 >> v1:172.21.15.136:6789/0 conn(0x7f2a080ce570 legacy=0x7f2a08154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a0e600640 1 -- 172.21.15.113:0/107483101 >> v1:172.21.15.113:6789/0 conn(0x7f2a080c8d80 legacy=0x7f2a0815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:33.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.547+0000 7f2a0e600640 1 -- 172.21.15.113:0/107483101 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0806f5f0 con 0x7f2a08070120 2024-03-04T08:11:33.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a0e600640 1 --2- 172.21.15.113:0/107483101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a08070120 0x7f2a081666b0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009e50 tx=0x7f2a0001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e69bc89464836 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a07e00640 1 -- 172.21.15.113:0/107483101 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a000b4070 con 0x7f2a08070120 2024-03-04T08:11:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a07e00640 1 -- 172.21.15.113:0/107483101 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a000af400 con 0x7f2a08070120 2024-03-04T08:11:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a07e00640 1 -- 172.21.15.113:0/107483101 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a000af7d0 con 0x7f2a08070120 2024-03-04T08:11:33.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a10600640 1 -- 172.21.15.113:0/107483101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a08070120 msgr2=0x7f2a081666b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a10600640 1 --2- 172.21.15.113:0/107483101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a08070120 0x7f2a081666b0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009e50 tx=0x7f2a0001f210 comp rx=0 tx=0).stop 2024-03-04T08:11:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a10600640 1 -- 172.21.15.113:0/107483101 shutdown_connections 2024-03-04T08:11:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a10600640 1 --2- 172.21.15.113:0/107483101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a08070120 0x7f2a081666b0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a10600640 1 -- 172.21.15.113:0/107483101 >> 172.21.15.113:0/107483101 conn(0x7f2a08066db0 msgr2=0x7f2a080671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a10600640 1 -- 172.21.15.113:0/107483101 shutdown_connections 2024-03-04T08:11:33.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.551+0000 7f2a10600640 1 -- 172.21.15.113:0/107483101 wait complete. 2024-03-04T08:11:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 Processor -- start 2024-03-04T08:11:33.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 -- start start 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a08070120 0x7f2a08157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a080c8d80 0x7f2a080c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a080ce570 0x7f2a080c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0806fb80 con 0x7f2a080c8d80 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0806c4f0 con 0x7f2a080ce570 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0806fd00 con 0x7f2a08070120 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a080c8d80 0x7f2a080c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a080c8d80 0x7f2a080c4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46484/0 (socket says 172.21.15.113:46484) 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0d200640 1 -- 172.21.15.113:0/1348607639 learned_addr learned my addr 172.21.15.113:0/1348607639 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0e600640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a080ce570 0x7f2a080c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0dc00640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a08070120 0x7f2a08157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0d200640 1 -- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a08070120 msgr2=0x7f2a08157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0d200640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a08070120 0x7f2a08157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:33.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0d200640 1 -- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a080ce570 msgr2=0x7f2a080c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0d200640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a080ce570 0x7f2a080c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0d200640 1 -- 172.21.15.113:0/1348607639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a080c9660 con 0x7f2a080c8d80 2024-03-04T08:11:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a0d200640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a080c8d80 0x7f2a080c4eb0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c530 tx=0x7f29f400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a05600640 1 -- 172.21.15.113:0/1348607639 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4016020 con 0x7f2a080c8d80 2024-03-04T08:11:33.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a05600640 1 -- 172.21.15.113:0/1348607639 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f4005220 con 0x7f2a080c8d80 2024-03-04T08:11:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a05600640 1 -- 172.21.15.113:0/1348607639 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f4007030 con 0x7f2a080c8d80 2024-03-04T08:11:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 -- 172.21.15.113:0/1348607639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a080c4410 con 0x7f2a080c8d80 2024-03-04T08:11:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 -- 172.21.15.113:0/1348607639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a080c46b0 con 0x7f2a080c8d80 2024-03-04T08:11:33.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.555+0000 7f2a10600640 1 -- 172.21.15.113:0/1348607639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a08075c00 con 0x7f2a080c8d80 2024-03-04T08:11:33.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.559+0000 7f2a05600640 1 -- 172.21.15.113:0/1348607639 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29f40071d0 con 0x7f2a080c8d80 2024-03-04T08:11:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.559+0000 7f2a05600640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f29e8066f60 0x7f29e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.559+0000 7f2a05600640 1 -- 172.21.15.113:0/1348607639 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f29f4085100 con 0x7f2a080c8d80 2024-03-04T08:11:33.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.559+0000 7f2a0dc00640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f29e8066f60 0x7f29e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:33.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.563+0000 7f2a0dc00640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f29e8066f60 0x7f29e8069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f29fc000f70 tx=0x7f29fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:33.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.563+0000 7f2a05600640 1 -- 172.21.15.113:0/1348607639 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f29f4020020 con 0x7f2a080c8d80 2024-03-04T08:11:33.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.751+0000 7f2a10600640 1 -- 172.21.15.113:0/1348607639 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a0806ff00 con 0x7f2a080c8d80 2024-03-04T08:11:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.751+0000 7f2a05600640 1 -- 172.21.15.113:0/1348607639 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+5826 (secure 0 0 0) 0x7f29f40074c0 con 0x7f2a080c8d80 2024-03-04T08:11:33.749 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:33.749 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":143,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7519,7692]},"id":1}]} 2024-03-04T08:11:33.749 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 143 2024-03-04T08:11:33.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 -- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f29e8066f60 msgr2=0x7f29e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:33.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f29e8066f60 0x7f29e8069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f29fc000f70 tx=0x7f29fc009040 comp rx=0 tx=0).stop 2024-03-04T08:11:33.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 -- 172.21.15.113:0/1348607639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a080c8d80 msgr2=0x7f2a080c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a080c8d80 0x7f2a080c4eb0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f29f400c530 tx=0x7f29f400c9f0 comp rx=0 tx=0).stop 2024-03-04T08:11:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 -- 172.21.15.113:0/1348607639 shutdown_connections 2024-03-04T08:11:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a080ce570 0x7f2a080c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f29e8066f60 0x7f29e8069420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a080c8d80 0x7f2a080c4eb0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 --2- 172.21.15.113:0/1348607639 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2a08070120 0x7f2a08157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 -- 172.21.15.113:0/1348607639 >> 172.21.15.113:0/1348607639 conn(0x7f2a08066db0 msgr2=0x7f2a080cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 -- 172.21.15.113:0/1348607639 shutdown_connections 2024-03-04T08:11:33.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:33.755+0000 7f2a10600640 1 -- 172.21.15.113:0/1348607639 wait complete. 2024-03-04T08:11:33.771 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c has gained rank=2, replacing gid=7269 2024-03-04T08:11:33.771 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.e 2024-03-04T08:11:36.969 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:11:36.972 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:11:37.467 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 4be687bf448 0.88 2024-03-04T08:11:37.470 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.88 2024-03-04T08:11:37.486 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 89 -eq 48 ] 2024-03-04T08:11:37.486 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.89 2024-03-04T08:11:37.486 INFO:tasks.workunit.client.0.smithi113.stdout:v0.89 2024-03-04T08:11:37.486 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.89 2024-03-04T08:11:37.486 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.89 2024-03-04T08:11:37.772 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-03-04T08:11:37.772 INFO:tasks.ceph.mds.e:Restarting daemon 2024-03-04T08:11:37.772 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-03-04T08:11:37.775 INFO:tasks.ceph.mds.e:Started 2024-03-04T08:11:37.776 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:37.889 INFO:tasks.ceph.mds.e.smithi113.stdout:starting mds.e at 2024-03-04T08:11:37.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97fcc00640 1 Processor -- start 2024-03-04T08:11:37.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97fcc00640 1 -- start start 2024-03-04T08:11:37.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97fcc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f8154c40 0x7f97f815b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:37.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97fcc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97f80cc6d0 0x7f97f80d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:37.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97fcc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80cb4c0 con 0x7f97f80cc2d0 2024-03-04T08:11:37.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97fcc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f8155020 con 0x7f97f80cc6d0 2024-03-04T08:11:37.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97fcc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f815df00 con 0x7f97f8154c40 2024-03-04T08:11:37.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97f7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f8154c40 0x7f97f815b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:37.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97f7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f97f80cc6d0 0x7f97f80d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:37.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97f7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f8154c40 0x7f97f815b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:38894/0 (socket says 172.21.15.113:38894) 2024-03-04T08:11:37.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.927+0000 7f97f7400640 1 -- 172.21.15.113:0/1481109596 learned_addr learned my addr 172.21.15.113:0/1481109596 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:37.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.931+0000 7f97f7400640 1 -- 172.21.15.113:0/1481109596 >> v2:172.21.15.136:3300/0 conn(0x7f97f80cc6d0 msgr2=0x7f97f80d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:11:37.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.931+0000 7f97f7400640 1 --2- 172.21.15.113:0/1481109596 >> v2:172.21.15.136:3300/0 conn(0x7f97f80cc6d0 0x7f97f80d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:37.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.931+0000 7f97f7400640 1 -- 172.21.15.113:0/1481109596 >> v1:172.21.15.136:6789/0 conn(0x7f97f80cc2d0 legacy=0x7f97f80c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:37.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.931+0000 7f97f7400640 1 -- 172.21.15.113:0/1481109596 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f815e1e0 con 0x7f97f8154c40 2024-03-04T08:11:37.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.931+0000 7f97f7e00640 1 --2- 172.21.15.113:0/1481109596 >> v2:172.21.15.136:3300/0 conn(0x7f97f80cc6d0 0x7f97f80d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:11:37.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.931+0000 7f97f7400640 1 --2- 172.21.15.113:0/1481109596 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f8154c40 0x7f97f815b450 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009870 tx=0x7f97e801ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41e2831df4934934 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:37.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.931+0000 7f97f6a00640 1 -- 172.21.15.113:0/1481109596 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80b5070 con 0x7f97f8154c40 2024-03-04T08:11:37.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.931+0000 7f97f6a00640 1 -- 172.21.15.113:0/1481109596 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e80a8a50 con 0x7f97f8154c40 2024-03-04T08:11:37.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.931+0000 7f97f6a00640 1 -- 172.21.15.113:0/1481109596 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80a8e20 con 0x7f97f8154c40 2024-03-04T08:11:37.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.935+0000 7f97fcc00640 1 -- 172.21.15.113:0/1481109596 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f8154c40 msgr2=0x7f97f815b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:37.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.935+0000 7f97fcc00640 1 --2- 172.21.15.113:0/1481109596 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f8154c40 0x7f97f815b450 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009870 tx=0x7f97e801ed50 comp rx=0 tx=0).stop 2024-03-04T08:11:37.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.935+0000 7f97fcc00640 1 -- 172.21.15.113:0/1481109596 shutdown_connections 2024-03-04T08:11:37.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.935+0000 7f97fcc00640 1 --2- 172.21.15.113:0/1481109596 >> v2:172.21.15.136:3300/0 conn(0x7f97f80cc6d0 0x7f97f80d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:37.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.935+0000 7f97fcc00640 1 --2- 172.21.15.113:0/1481109596 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f8154c40 0x7f97f815b450 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:37.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.935+0000 7f97fcc00640 1 -- 172.21.15.113:0/1481109596 >> 172.21.15.113:0/1481109596 conn(0x7f97f806d3e0 msgr2=0x7f97f806d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:37.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.935+0000 7f97fcc00640 1 -- 172.21.15.113:0/1481109596 shutdown_connections 2024-03-04T08:11:37.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.935+0000 7f97fcc00640 1 -- 172.21.15.113:0/1481109596 wait complete. 2024-03-04T08:11:37.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.935+0000 7f97fcc00640 1 Processor -- start 2024-03-04T08:11:37.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.935+0000 7f97fcc00640 1 -- start start 2024-03-04T08:11:37.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97fcc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f80cc2d0 0x7f97f8158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:37.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97fcc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f80cc6d0 0x7f97f81489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:37.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97fcc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f8154c40 0x7f97f8148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:37.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97fcc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f815ecc0 con 0x7f97f80cc6d0 2024-03-04T08:11:37.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97fcc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f815e7a0 con 0x7f97f8154c40 2024-03-04T08:11:37.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97fcc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80cb180 con 0x7f97f80cc2d0 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97eea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f80cc6d0 0x7f97f81489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97f7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f80cc2d0 0x7f97f8158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97eea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f80cc6d0 0x7f97f81489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54080/0 (socket says 172.21.15.113:54080) 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97eea00640 1 -- 172.21.15.113:0/2225229824 learned_addr learned my addr 172.21.15.113:0/2225229824 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97f7e00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f8154c40 0x7f97f8148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97eea00640 1 -- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f80cc2d0 msgr2=0x7f97f8158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97eea00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f80cc2d0 0x7f97f8158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97eea00640 1 -- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f8154c40 msgr2=0x7f97f8148f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97eea00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f8154c40 0x7f97f8148f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97eea00640 1 -- 172.21.15.113:0/2225229824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f81e6520 con 0x7f97f80cc6d0 2024-03-04T08:11:37.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97eea00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f80cc6d0 0x7f97f81489d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f97e000c420 tx=0x7f97e000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:37.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97efe00640 1 -- 172.21.15.113:0/2225229824 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e00052d0 con 0x7f97f80cc6d0 2024-03-04T08:11:37.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97efe00640 1 -- 172.21.15.113:0/2225229824 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e0005470 con 0x7f97f80cc6d0 2024-03-04T08:11:37.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97efe00640 1 -- 172.21.15.113:0/2225229824 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0007260 con 0x7f97f80cc6d0 2024-03-04T08:11:37.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97fcc00640 1 -- 172.21.15.113:0/2225229824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f81e51e0 con 0x7f97f80cc6d0 2024-03-04T08:11:37.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97fcc00640 1 -- 172.21.15.113:0/2225229824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f81e4700 con 0x7f97f80cc6d0 2024-03-04T08:11:37.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.939+0000 7f97fcc00640 1 -- 172.21.15.113:0/2225229824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f8075a80 con 0x7f97f80cc6d0 2024-03-04T08:11:37.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.943+0000 7f97efe00640 1 -- 172.21.15.113:0/2225229824 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97e0007400 con 0x7f97f80cc6d0 2024-03-04T08:11:37.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.943+0000 7f97efe00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 0x7f97d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:37.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.943+0000 7f97efe00640 1 -- 172.21.15.113:0/2225229824 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f97e0084cb0 con 0x7f97f80cc6d0 2024-03-04T08:11:37.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.943+0000 7f97f7400640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 0x7f97d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:37.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.943+0000 7f97f7400640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 0x7f97d4069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f97e80b2000 tx=0x7f97e80aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:37.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:37.951+0000 7f97efe00640 1 -- 172.21.15.113:0/2225229824 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f97f81e6520 con 0x7f97f80cc6d0 2024-03-04T08:11:38.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.187+0000 7f97fcc00640 1 -- 172.21.15.113:0/2225229824 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97f8153ba0 con 0x7f97f80cc6d0 2024-03-04T08:11:38.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.187+0000 7f97efe00640 1 -- 172.21.15.113:0/2225229824 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+5826 (secure 0 0 0) 0x7f97e0054e40 con 0x7f97f80cc6d0 2024-03-04T08:11:38.185 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:38.185 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":143,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7389,7519,7692]},"id":1}]} 2024-03-04T08:11:38.186 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 143 2024-03-04T08:11:38.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 -- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 msgr2=0x7f97d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:38.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 0x7f97d4069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f97e80b2000 tx=0x7f97e80aa070 comp rx=0 tx=0).stop 2024-03-04T08:11:38.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 -- 172.21.15.113:0/2225229824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f80cc6d0 msgr2=0x7f97f81489d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:38.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f80cc6d0 0x7f97f81489d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f97e000c420 tx=0x7f97e000c8e0 comp rx=0 tx=0).stop 2024-03-04T08:11:38.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 -- 172.21.15.113:0/2225229824 shutdown_connections 2024-03-04T08:11:38.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97f8154c40 0x7f97f8148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:38.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f97d4066f60 0x7f97d4069420 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:38.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97f80cc6d0 0x7f97f81489d0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:38.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 --2- 172.21.15.113:0/2225229824 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f97f80cc2d0 0x7f97f8158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:38.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 -- 172.21.15.113:0/2225229824 >> 172.21.15.113:0/2225229824 conn(0x7f97f806d3e0 msgr2=0x7f97f81518c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:38.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 -- 172.21.15.113:0/2225229824 shutdown_connections 2024-03-04T08:11:38.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:38.191+0000 7f97fcc00640 1 -- 172.21.15.113:0/2225229824 wait complete. 2024-03-04T08:11:38.217 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2024-03-04T08:11:40.219 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:11:40.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcad2800640 1 Processor -- start 2024-03-04T08:11:40.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcad2800640 1 -- start start 2024-03-04T08:11:40.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcad2800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 0x7fcacc15b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:40.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcad2800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc0cb4c0 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcad2800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc155020 con 0x7fcacc0cc6d0 2024-03-04T08:11:40.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcad2800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc15df00 con 0x7fcacc154c40 2024-03-04T08:11:40.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcad0a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcacc0cc2d0 0x7fcacc0c4fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49988/0 (socket says 172.21.15.113:49988) 2024-03-04T08:11:40.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcad0a00640 1 -- 172.21.15.113:0/3133246599 learned_addr learned my addr 172.21.15.113:0/3133246599 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:11:40.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcad1400640 1 --2- 172.21.15.113:0/3133246599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 0x7fcacc15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:40.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcacbe00640 1 -- 172.21.15.113:0/3133246599 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247501626 0 0) 0x7fcacc0cb4c0 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcacbe00640 1 -- 172.21.15.113:0/3133246599 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcab4003530 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.387+0000 7fcacbe00640 1 -- 172.21.15.113:0/3133246599 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678865537 0 0) 0x7fcacc155020 con 0x7fcacc0cc6d0 2024-03-04T08:11:40.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.391+0000 7fcad1400640 1 -- 172.21.15.113:0/3133246599 >> v1:172.21.15.136:6789/0 conn(0x7fcacc0cc6d0 legacy=0x7fcacc0d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:40.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.391+0000 7fcad1400640 1 -- 172.21.15.113:0/3133246599 >> v1:172.21.15.113:6789/0 conn(0x7fcacc0cc2d0 legacy=0x7fcacc0c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:40.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.391+0000 7fcad1400640 1 -- 172.21.15.113:0/3133246599 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc15e1e0 con 0x7fcacc154c40 2024-03-04T08:11:40.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcad1400640 1 --2- 172.21.15.113:0/3133246599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 0x7fcacc15b450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009dc0 tx=0x7fcac001f170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a49e85ee345e36d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:40.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcacbe00640 1 -- 172.21.15.113:0/3133246599 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac00b4070 con 0x7fcacc154c40 2024-03-04T08:11:40.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcacbe00640 1 -- 172.21.15.113:0/3133246599 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac00af600 con 0x7fcacc154c40 2024-03-04T08:11:40.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcacbe00640 1 -- 172.21.15.113:0/3133246599 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac00af9b0 con 0x7fcacc154c40 2024-03-04T08:11:40.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcad2800640 1 -- 172.21.15.113:0/3133246599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 msgr2=0x7fcacc15b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:40.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcad2800640 1 --2- 172.21.15.113:0/3133246599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 0x7fcacc15b450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009dc0 tx=0x7fcac001f170 comp rx=0 tx=0).stop 2024-03-04T08:11:40.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcad2800640 1 -- 172.21.15.113:0/3133246599 shutdown_connections 2024-03-04T08:11:40.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcad2800640 1 --2- 172.21.15.113:0/3133246599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 0x7fcacc15b450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:40.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcad2800640 1 -- 172.21.15.113:0/3133246599 >> 172.21.15.113:0/3133246599 conn(0x7fcacc06d3e0 msgr2=0x7fcacc06d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:40.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcad2800640 1 -- 172.21.15.113:0/3133246599 shutdown_connections 2024-03-04T08:11:40.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.395+0000 7fcad2800640 1 -- 172.21.15.113:0/3133246599 wait complete. 2024-03-04T08:11:40.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad2800640 1 Processor -- start 2024-03-04T08:11:40.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad2800640 1 -- start start 2024-03-04T08:11:40.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcacc0cc2d0 0x7fcacc1f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:40.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc0cc6d0 0x7fcacc0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:40.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad2800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 0x7fcacc14ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:40.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad2800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc15ecc0 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad2800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc15e7a0 con 0x7fcacc0cc6d0 2024-03-04T08:11:40.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad2800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0cb180 con 0x7fcacc154c40 2024-03-04T08:11:40.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcacc0cc2d0 0x7fcacc1f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:40.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 0x7fcacc14ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:40.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad0a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc0cc6d0 0x7fcacc0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:40.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 0x7fcacc14ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:38924/0 (socket says 172.21.15.113:38924) 2024-03-04T08:11:40.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad1e00640 1 -- 172.21.15.113:0/4099641853 learned_addr learned my addr 172.21.15.113:0/4099641853 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:11:40.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad1400640 1 -- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 msgr2=0x7fcacc14ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad1400640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 0x7fcacc14ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad1400640 1 -- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc0cc6d0 msgr2=0x7fcacc0ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad1400640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc0cc6d0 0x7fcacc0ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad1400640 1 -- 172.21.15.113:0/4099641853 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc1f1500 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad1400640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcacc0cc2d0 0x7fcacc1f0fc0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fcac00af000 tx=0x7fcac001bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcac9600640 1 -- 172.21.15.113:0/4099641853 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac00b4040 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcac9600640 1 -- 172.21.15.113:0/4099641853 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac00af400 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcac9600640 1 -- 172.21.15.113:0/4099641853 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac00af9b0 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad2800640 1 -- 172.21.15.113:0/4099641853 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc0cf1b0 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.399+0000 7fcad2800640 1 -- 172.21.15.113:0/4099641853 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc0b5700 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.403+0000 7fcad2800640 1 -- 172.21.15.113:0/4099641853 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc075a80 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.403+0000 7fcac9600640 1 -- 172.21.15.113:0/4099641853 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcac00c1020 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.403+0000 7fcac9600640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcaac066f60 0x7fcaac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:11:40.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.403+0000 7fcac9600640 1 -- 172.21.15.113:0/4099641853 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7fcac0123230 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.403+0000 7fcad0a00640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcaac066f60 0x7fcaac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:11:40.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.403+0000 7fcad0a00640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcaac066f60 0x7fcaac069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcab8007fd0 tx=0x7fcab8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:11:40.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.407+0000 7fcac9600640 1 -- 172.21.15.113:0/4099641853 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcac0018cd0 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.639+0000 7fcad2800640 1 -- 172.21.15.113:0/4099641853 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcacc0b2a70 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcac9600640 1 -- 172.21.15.113:0/4099641853 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+6621 (secure 0 0 0) 0x7fcac00f3340 con 0x7fcacc0cc2d0 2024-03-04T08:11:40.639 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:11:40.639 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7839,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7692,7519,7389]},"id":1}]} 2024-03-04T08:11:40.640 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 144 2024-03-04T08:11:40.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 -- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcaac066f60 msgr2=0x7fcaac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:40.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcaac066f60 0x7fcaac069420 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fcab8007fd0 tx=0x7fcab8009040 comp rx=0 tx=0).stop 2024-03-04T08:11:40.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 -- 172.21.15.113:0/4099641853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcacc0cc2d0 msgr2=0x7fcacc1f0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:11:40.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcacc0cc2d0 0x7fcacc1f0fc0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fcac00af000 tx=0x7fcac001bf10 comp rx=0 tx=0).stop 2024-03-04T08:11:40.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 -- 172.21.15.113:0/4099641853 shutdown_connections 2024-03-04T08:11:40.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcacc154c40 0x7fcacc14ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:40.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc0cc6d0 0x7fcacc0ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:40.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcaac066f60 0x7fcaac069420 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:40.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 --2- 172.21.15.113:0/4099641853 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcacc0cc2d0 0x7fcacc1f0fc0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:11:40.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 -- 172.21.15.113:0/4099641853 >> 172.21.15.113:0/4099641853 conn(0x7fcacc06d3e0 msgr2=0x7fcacc150680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:11:40.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 -- 172.21.15.113:0/4099641853 shutdown_connections 2024-03-04T08:11:40.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:11:40.643+0000 7fcad2800640 1 -- 172.21.15.113:0/4099641853 wait complete. 2024-03-04T08:11:40.660 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-03-04T08:11:40.660 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 110 secs before thrashing 2024-03-04T08:11:50.997 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 42% (84/197) Updating files: 43% (85/197) Updating files: 44% (87/197) Updating files: 45% (89/197) Updating files: 46% (91/197) Updating files: 47% (93/197) Updating files: 48% (95/197) Updating files: 49% (97/197) Updating files: 50% (99/197) Updating files: 51% (101/197) Updating files: 52% (103/197) Updating files: 53% (105/197) Updating files: 54% (107/197) Updating files: 55% (109/197) Updating files: 56% (111/197) Updating files: 57% (113/197) Updating files: 58% (115/197) Updating files: 59% (117/197) Updating files: 60% (119/197) Updating files: 61% (121/197) Updating files: 62% (123/197) Updating files: 63% (125/197) Updating files: 64% (127/197) Updating files: 65% (129/197) Updating files: 66% (131/197) Updating files: 67% (132/197) Updating files: 68% (134/197) Updating files: 69% (136/197) Updating files: 70% (138/197) Updating files: 71% (140/197) Updating files: 72% (142/197) Updating files: 73% (144/197) Updating files: 73% (145/197) Updating files: 74% (146/197) Updating files: 75% (148/197) Updating files: 76% (150/197) Updating files: 77% (152/197) Updating files: 77% (153/197) Updating files: 78% (154/197) Updating files: 79% (156/197) Updating files: 80% (158/197) Updating files: 81% (160/197) Updating files: 82% (162/197) Updating files: 83% (164/197) Updating files: 84% (166/197) Updating files: 85% (168/197) Updating files: 86% (170/197) Updating files: 87% (172/197) Updating files: 88% (174/197) Updating files: 89% (176/197) Updating files: 90% (178/197) Updating files: 91% (180/197) Updating files: 92% (182/197) Updating files: 93% (184/197) Updating files: 94% (186/197) Updating files: 95% (188/197) Updating files: 96% (190/197) Updating files: 97% (192/197) Updating files: 98% (194/197) Updating files: 99% (196/197) Updating files: 100% (197/197) Updating files: 100% (197/197), done. 2024-03-04T08:11:51.096 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 68fdc0f68e6 0.89 2024-03-04T08:11:51.103 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.89 2024-03-04T08:11:51.430 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 90 -eq 48 ] 2024-03-04T08:11:51.430 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.90 2024-03-04T08:11:51.430 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo v0.90 2024-03-04T08:11:51.430 INFO:tasks.workunit.client.0.smithi113.stdout:v0.90 2024-03-04T08:11:51.430 INFO:tasks.workunit.client.0.smithi113.stderr:+ git reset --hard v0.90 2024-03-04T08:12:05.359 INFO:tasks.workunit.client.0.smithi113.stderr:Updating files: 22% (60/265) Updating files: 23% (61/265) Updating files: 24% (64/265) Updating files: 25% (67/265) Updating files: 26% (69/265) Updating files: 27% (72/265) Updating files: 28% (75/265) Updating files: 29% (77/265) Updating files: 30% (80/265) Updating files: 31% (83/265) Updating files: 32% (85/265) Updating files: 33% (88/265) Updating files: 33% (89/265) Updating files: 34% (91/265) Updating files: 35% (93/265) Updating files: 36% (96/265) Updating files: 37% (99/265) Updating files: 38% (101/265) Updating files: 39% (104/265) Updating files: 40% (106/265) Updating files: 41% (109/265) Updating files: 42% (112/265) Updating files: 43% (114/265) Updating files: 43% (115/265) Updating files: 44% (117/265) Updating files: 45% (120/265) Updating files: 46% (122/265) Updating files: 47% (125/265) Updating files: 48% (128/265) Updating files: 49% (130/265) Updating files: 50% (133/265) Updating files: 51% (136/265) Updating files: 52% (138/265) Updating files: 53% (141/265) Updating files: 54% (144/265) Updating files: 55% (146/265) Updating files: 56% (149/265) Updating files: 57% (152/265) Updating files: 58% (154/265) Updating files: 59% (157/265) Updating files: 60% (159/265) Updating files: 61% (162/265) Updating files: 62% (165/265) Updating files: 63% (167/265) Updating files: 64% (170/265) Updating files: 65% (173/265) Updating files: 66% (175/265) Updating files: 67% (178/265) Updating files: 68% (181/265) Updating files: 69% (183/265) Updating files: 70% (186/265) Updating files: 71% (189/265) Updating files: 72% (191/265) Updating files: 73% (194/265) Updating files: 74% (197/265) Updating files: 75% (199/265) Updating files: 76% (202/265) Updating files: 77% (205/265) Updating files: 78% (207/265) Updating files: 79% (210/265) Updating files: 80% (212/265) Updating files: 81% (215/265) Updating files: 82% (218/265) Updating files: 83% (220/265) Updating files: 84% (223/265) Updating files: 85% (226/265) Updating files: 86% (228/265) Updating files: 87% (231/265) Updating files: 88% (234/265) Updating files: 89% (236/265) Updating files: 90% (239/265) Updating files: 91% (242/265) Updating files: 92% (244/265) Updating files: 93% (247/265) Updating files: 94% (250/265) Updating files: 95% (252/265) Updating files: 96% (255/265) Updating files: 97% (258/265) Updating files: 97% (259/265) Updating files: 98% (260/265) Updating files: 99% (263/265) Updating files: 100% (265/265) Updating files: 100% (265/265), done. 2024-03-04T08:12:05.406 INFO:tasks.workunit.client.0.smithi113.stdout:HEAD is now at 08bd1e1eee3 0.90 2024-03-04T08:12:05.411 INFO:tasks.workunit.client.0.smithi113.stderr:+ mkdir .snap/v0.90 2024-03-04T08:12:05.496 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.1 2024-03-04T08:12:05.497 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 1 -eq 48 ] 2024-03-04T08:12:05.497 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.1 2024-03-04T08:12:05.497 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.1 2024-03-04T08:12:05.497 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.1 2024-03-04T08:12:05.497 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:07.056 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:12:07.061 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:12:11.723 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:11.724 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.2 2024-03-04T08:12:11.724 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 2 -eq 48 ] 2024-03-04T08:12:11.724 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.2 2024-03-04T08:12:11.725 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.2 2024-03-04T08:12:11.725 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.2 2024-03-04T08:12:11.725 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:15.650 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:15.651 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.3 2024-03-04T08:12:15.651 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 3 -eq 48 ] 2024-03-04T08:12:15.651 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.3 2024-03-04T08:12:15.652 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.3 2024-03-04T08:12:15.652 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.3 2024-03-04T08:12:15.652 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:17.799 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:17.799 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.4 2024-03-04T08:12:17.800 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 4 -eq 48 ] 2024-03-04T08:12:17.800 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.4 2024-03-04T08:12:17.800 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.4 2024-03-04T08:12:17.800 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.4 2024-03-04T08:12:17.801 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:21.229 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:21.229 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 5 -eq 48 ] 2024-03-04T08:12:21.229 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.5 2024-03-04T08:12:21.230 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.5 2024-03-04T08:12:21.230 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.5 2024-03-04T08:12:21.230 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.5 2024-03-04T08:12:21.230 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:23.974 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:23.974 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.6 2024-03-04T08:12:23.975 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 6 -eq 48 ] 2024-03-04T08:12:23.975 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.6 2024-03-04T08:12:23.975 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.6 2024-03-04T08:12:23.975 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.6 2024-03-04T08:12:23.975 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:30.063 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:30.063 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 7 -eq 48 ] 2024-03-04T08:12:30.063 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.7 2024-03-04T08:12:30.063 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.7 2024-03-04T08:12:30.064 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.7 2024-03-04T08:12:30.064 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.7 2024-03-04T08:12:30.064 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:33.397 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:33.397 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.8 2024-03-04T08:12:33.398 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 8 -eq 48 ] 2024-03-04T08:12:33.398 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.8 2024-03-04T08:12:33.398 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.8 2024-03-04T08:12:33.398 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.8 2024-03-04T08:12:33.398 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:36.079 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:36.079 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 9 -eq 48 ] 2024-03-04T08:12:36.079 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.9 2024-03-04T08:12:36.079 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.9 2024-03-04T08:12:36.080 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.9 2024-03-04T08:12:36.080 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.9 2024-03-04T08:12:36.080 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:37.120 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:12:37.124 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:12:37.308 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:37.308 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 10 -eq 48 ] 2024-03-04T08:12:37.308 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.10 2024-03-04T08:12:37.308 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.10 2024-03-04T08:12:37.309 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.10 2024-03-04T08:12:37.309 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.10 2024-03-04T08:12:37.309 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:39.763 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:39.764 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.11 2024-03-04T08:12:39.764 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 11 -eq 48 ] 2024-03-04T08:12:39.764 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.11 2024-03-04T08:12:39.765 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.11 2024-03-04T08:12:39.765 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.11 2024-03-04T08:12:39.765 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:42.342 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:42.342 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 12 -eq 48 ] 2024-03-04T08:12:42.343 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.12 2024-03-04T08:12:42.343 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.12 2024-03-04T08:12:42.343 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.12 2024-03-04T08:12:42.343 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.12 2024-03-04T08:12:42.343 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:45.703 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:45.703 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 13 -eq 48 ] 2024-03-04T08:12:45.703 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.13 2024-03-04T08:12:45.703 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.13 2024-03-04T08:12:45.703 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.13 2024-03-04T08:12:45.704 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.13 2024-03-04T08:12:45.704 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:48.937 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:48.938 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 14 -eq 48 ] 2024-03-04T08:12:48.938 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.14 2024-03-04T08:12:48.938 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.14 2024-03-04T08:12:48.938 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.14 2024-03-04T08:12:48.939 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.14 2024-03-04T08:12:48.939 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:52.439 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:52.439 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.15 2024-03-04T08:12:52.440 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 15 -eq 48 ] 2024-03-04T08:12:52.440 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.15 2024-03-04T08:12:52.440 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.15 2024-03-04T08:12:52.440 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.15 2024-03-04T08:12:52.440 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:56.560 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:56.561 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.16 2024-03-04T08:12:56.561 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 16 -eq 48 ] 2024-03-04T08:12:56.561 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.16 2024-03-04T08:12:56.561 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.16 2024-03-04T08:12:56.561 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.16 2024-03-04T08:12:56.561 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:12:59.963 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:12:59.963 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 17 -eq 48 ] 2024-03-04T08:12:59.963 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.17 2024-03-04T08:12:59.964 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.17 2024-03-04T08:12:59.964 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.17 2024-03-04T08:12:59.964 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.17 2024-03-04T08:12:59.964 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:13:02.164 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:13:02.165 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.18 2024-03-04T08:13:02.165 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 18 -eq 48 ] 2024-03-04T08:13:02.165 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.18 2024-03-04T08:13:02.165 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.18 2024-03-04T08:13:02.165 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.18 2024-03-04T08:13:02.165 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:13:06.967 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:13:06.967 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.19 2024-03-04T08:13:06.967 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 19 -eq 48 ] 2024-03-04T08:13:06.968 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.19 2024-03-04T08:13:06.968 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.19 2024-03-04T08:13:06.968 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.19 2024-03-04T08:13:06.968 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:13:07.236 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:13:07.239 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:13:12.800 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:13:12.800 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.20 2024-03-04T08:13:12.800 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 20 -eq 48 ] 2024-03-04T08:13:12.801 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.20 2024-03-04T08:13:12.801 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.20 2024-03-04T08:13:12.801 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.20 2024-03-04T08:13:12.801 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:13:14.920 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:13:14.920 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 21 -eq 48 ] 2024-03-04T08:13:14.920 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.21 2024-03-04T08:13:14.920 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.21 2024-03-04T08:13:14.921 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.21 2024-03-04T08:13:14.921 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.21 2024-03-04T08:13:14.921 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:13:17.377 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:13:17.377 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 22 -eq 48 ] 2024-03-04T08:13:17.377 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.22 2024-03-04T08:13:17.377 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.22 2024-03-04T08:13:17.377 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.22 2024-03-04T08:13:17.378 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.22 2024-03-04T08:13:17.378 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:13:20.505 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:13:20.505 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 23 -eq 48 ] 2024-03-04T08:13:20.505 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.23 2024-03-04T08:13:20.505 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.23 2024-03-04T08:13:20.506 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.23 2024-03-04T08:13:20.506 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.23 2024-03-04T08:13:20.506 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:13:27.175 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:13:27.175 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 24 -eq 48 ] 2024-03-04T08:13:27.175 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.24 2024-03-04T08:13:27.176 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.24 2024-03-04T08:13:27.176 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.24 2024-03-04T08:13:27.176 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.24 2024-03-04T08:13:27.176 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:13:30.662 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.801+0000 7f6af6c00640 1 Processor -- start 2024-03-04T08:13:30.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.801+0000 7f6af6c00640 1 -- start start 2024-03-04T08:13:30.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6af6c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6af00ccb40 0x7f6af00c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:30.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6af6c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af0154f10 0x7f6af015e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:30.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6af6c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af015aed0 con 0x7f6af015aa40 2024-03-04T08:13:30.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6af6c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af015b170 con 0x7f6af00ccb40 2024-03-04T08:13:30.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6af6c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af015b410 con 0x7f6af0154f10 2024-03-04T08:13:30.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6ae7400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6af015aa40 0x7f6af0154800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39816/0 (socket says 172.21.15.113:39816) 2024-03-04T08:13:30.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6ae7400640 1 -- 172.21.15.113:0/4251185307 learned_addr learned my addr 172.21.15.113:0/4251185307 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:13:30.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6aefe00640 1 --2- 172.21.15.113:0/4251185307 >> v2:172.21.15.113:3300/0 conn(0x7f6af00ccb40 0x7f6af00c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:30.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6af4c00640 1 --2- 172.21.15.113:0/4251185307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af0154f10 0x7f6af015e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:30.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6aef400640 1 -- 172.21.15.113:0/4251185307 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863038691 0 0) 0x7f6af015aed0 con 0x7f6af015aa40 2024-03-04T08:13:30.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6aef400640 1 -- 172.21.15.113:0/4251185307 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ad4003530 con 0x7f6af015aa40 2024-03-04T08:13:30.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6aefe00640 1 -- 172.21.15.113:0/4251185307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af0154f10 msgr2=0x7f6af015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:30.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6aefe00640 1 --2- 172.21.15.113:0/4251185307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af0154f10 0x7f6af015e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:30.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6aefe00640 1 -- 172.21.15.113:0/4251185307 >> v1:172.21.15.113:6789/0 conn(0x7f6af015aa40 legacy=0x7f6af0154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:30.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.805+0000 7f6aefe00640 1 -- 172.21.15.113:0/4251185307 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af015b9b0 con 0x7f6af00ccb40 2024-03-04T08:13:30.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6aefe00640 1 --2- 172.21.15.113:0/4251185307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6af00ccb40 0x7f6af00c7370 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00097f0 tx=0x7f6ae001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9183e9d4910319d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:30.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6aef400640 1 -- 172.21.15.113:0/4251185307 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00b5070 con 0x7f6af00ccb40 2024-03-04T08:13:30.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6aef400640 1 -- 172.21.15.113:0/4251185307 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae00a8990 con 0x7f6af00ccb40 2024-03-04T08:13:30.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6aef400640 1 -- 172.21.15.113:0/4251185307 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00a8d60 con 0x7f6af00ccb40 2024-03-04T08:13:30.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6af6c00640 1 -- 172.21.15.113:0/4251185307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6af00ccb40 msgr2=0x7f6af00c7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:30.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6af6c00640 1 --2- 172.21.15.113:0/4251185307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6af00ccb40 0x7f6af00c7370 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00097f0 tx=0x7f6ae001ec90 comp rx=0 tx=0).stop 2024-03-04T08:13:30.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6af6c00640 1 -- 172.21.15.113:0/4251185307 shutdown_connections 2024-03-04T08:13:30.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6af6c00640 1 --2- 172.21.15.113:0/4251185307 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af0154f10 0x7f6af015e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:30.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6af6c00640 1 --2- 172.21.15.113:0/4251185307 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6af00ccb40 0x7f6af00c7370 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:30.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6af6c00640 1 -- 172.21.15.113:0/4251185307 >> 172.21.15.113:0/4251185307 conn(0x7f6af006b060 msgr2=0x7f6af006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:30.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6af6c00640 1 -- 172.21.15.113:0/4251185307 shutdown_connections 2024-03-04T08:13:30.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6af6c00640 1 -- 172.21.15.113:0/4251185307 wait complete. 2024-03-04T08:13:30.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6af6c00640 1 Processor -- start 2024-03-04T08:13:30.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.809+0000 7f6af6c00640 1 -- start start 2024-03-04T08:13:30.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af6c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af00ccb40 0x7f6af00c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:30.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af6c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6af0154f10 0x7f6af01f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:30.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af6c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6af01f4520 0x7f6af0158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:30.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af6c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00c78b0 con 0x7f6af01f4520 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af6c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af015e0d0 con 0x7f6af0154f10 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af6c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00ccf20 con 0x7f6af00ccb40 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6af01f4520 0x7f6af0158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6aefe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af00ccb40 0x7f6af00c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6aefe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af00ccb40 0x7f6af00c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:39668/0 (socket says 172.21.15.113:39668) 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6aefe00640 1 -- 172.21.15.113:0/1158057143 learned_addr learned my addr 172.21.15.113:0/1158057143 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6ae7400640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6af0154f10 0x7f6af01f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af4c00640 1 -- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af00ccb40 msgr2=0x7f6af00c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af4c00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af00ccb40 0x7f6af00c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af4c00640 1 -- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6af0154f10 msgr2=0x7f6af01f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af4c00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6af0154f10 0x7f6af01f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:30.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af4c00640 1 -- 172.21.15.113:0/1158057143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af01f4a70 con 0x7f6af01f4520 2024-03-04T08:13:30.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af4c00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6af01f4520 0x7f6af0158d90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800ef90 tx=0x7f6ae800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:30.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6aecc00640 1 -- 172.21.15.113:0/1158057143 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8018020 con 0x7f6af01f4520 2024-03-04T08:13:30.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6aecc00640 1 -- 172.21.15.113:0/1158057143 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae8014480 con 0x7f6af01f4520 2024-03-04T08:13:30.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6aecc00640 1 -- 172.21.15.113:0/1158057143 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8014870 con 0x7f6af01f4520 2024-03-04T08:13:30.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af6c00640 1 -- 172.21.15.113:0/1158057143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af00c5320 con 0x7f6af01f4520 2024-03-04T08:13:30.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af6c00640 1 -- 172.21.15.113:0/1158057143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af00c58c0 con 0x7f6af01f4520 2024-03-04T08:13:30.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6af6c00640 1 -- 172.21.15.113:0/1158057143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af0158a80 con 0x7f6af01f4520 2024-03-04T08:13:30.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.813+0000 7f6aecc00640 1 -- 172.21.15.113:0/1158057143 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ae801b020 con 0x7f6af01f4520 2024-03-04T08:13:30.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.817+0000 7f6aecc00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6acc066f90 0x7f6acc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:30.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.817+0000 7f6aecc00640 1 -- 172.21.15.113:0/1158057143 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f6ae80864d0 con 0x7f6af01f4520 2024-03-04T08:13:30.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.817+0000 7f6aefe00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6acc066f90 0x7f6acc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:30.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.817+0000 7f6aefe00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6acc066f90 0x7f6acc069450 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00097f0 tx=0x7f6ae0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:30.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:30.821+0000 7f6aecc00640 1 -- 172.21.15.113:0/1158057143 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6ae8022030 con 0x7f6af01f4520 2024-03-04T08:13:31.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 -- 172.21.15.113:0/1158057143 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6af00729c0 con 0x7f6af01f4520 2024-03-04T08:13:31.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6aecc00640 1 -- 172.21.15.113:0/1158057143 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f6ae8010040 con 0x7f6af01f4520 2024-03-04T08:13:31.012 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:31.012 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7839,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7692,7519,7389]},"id":1}]} 2024-03-04T08:13:31.012 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 144 2024-03-04T08:13:31.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 -- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6acc066f90 msgr2=0x7f6acc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:31.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6acc066f90 0x7f6acc069450 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00097f0 tx=0x7f6ae0008040 comp rx=0 tx=0).stop 2024-03-04T08:13:31.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 -- 172.21.15.113:0/1158057143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6af01f4520 msgr2=0x7f6af0158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:31.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6af01f4520 0x7f6af0158d90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800ef90 tx=0x7f6ae800c490 comp rx=0 tx=0).stop 2024-03-04T08:13:31.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 -- 172.21.15.113:0/1158057143 shutdown_connections 2024-03-04T08:13:31.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6acc066f90 0x7f6acc069450 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6af01f4520 0x7f6af0158d90 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6af0154f10 0x7f6af01f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 --2- 172.21.15.113:0/1158057143 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6af00ccb40 0x7f6af00c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 -- 172.21.15.113:0/1158057143 >> 172.21.15.113:0/1158057143 conn(0x7f6af006b060 msgr2=0x7f6af00c85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:31.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 -- 172.21.15.113:0/1158057143 shutdown_connections 2024-03-04T08:13:31.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.013+0000 7f6af6c00640 1 -- 172.21.15.113:0/1158057143 wait complete. 2024-03-04T08:13:31.034 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=0) 2024-03-04T08:13:31.034 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-04T08:13:31.034 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:13:31.072 INFO:tasks.ceph.mds.a:Stopped 2024-03-04T08:13:31.072 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:31.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.249+0000 7fbbba400640 1 Processor -- start 2024-03-04T08:13:31.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.249+0000 7fbbba400640 1 -- start start 2024-03-04T08:13:31.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbba400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbbb40ce6b0 0x7fbbb415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:31.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbba400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:31.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbba400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb406c210 con 0x7fbbb40ce2d0 2024-03-04T08:13:31.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbba400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb415bbf0 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbba400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb406eb80 con 0x7fbbb406de80 2024-03-04T08:13:31.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbb3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:31.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbb3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:39678/0 (socket says 172.21.15.113:39678) 2024-03-04T08:13:31.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbb3e00640 1 -- 172.21.15.113:0/1983896483 learned_addr learned my addr 172.21.15.113:0/1983896483 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:31.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbabe00640 1 --2- 172.21.15.113:0/1983896483 >> v2:172.21.15.136:3300/0 conn(0x7fbbb40ce6b0 0x7fbbb415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:31.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbb3e00640 1 -- 172.21.15.113:0/1983896483 >> v2:172.21.15.136:3300/0 conn(0x7fbbb40ce6b0 msgr2=0x7fbbb415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:31.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbb3e00640 1 --2- 172.21.15.113:0/1983896483 >> v2:172.21.15.136:3300/0 conn(0x7fbbb40ce6b0 0x7fbbb415b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbb3e00640 1 -- 172.21.15.113:0/1983896483 >> v1:172.21.15.136:6789/0 conn(0x7fbbb40ce2d0 legacy=0x7fbbb406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:31.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.253+0000 7fbbb3e00640 1 -- 172.21.15.113:0/1983896483 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb406c4f0 con 0x7fbbb406de80 2024-03-04T08:13:31.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbb3e00640 1 --2- 172.21.15.113:0/1983896483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4154b10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009e50 tx=0x7fbbac01f210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9db67617f72ce7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbb2a00640 1 -- 172.21.15.113:0/1983896483 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0b4070 con 0x7fbbb406de80 2024-03-04T08:13:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbb2a00640 1 -- 172.21.15.113:0/1983896483 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac0af400 con 0x7fbbb406de80 2024-03-04T08:13:31.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbb2a00640 1 -- 172.21.15.113:0/1983896483 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0af7d0 con 0x7fbbb406de80 2024-03-04T08:13:31.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 -- 172.21.15.113:0/1983896483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 msgr2=0x7fbbb4154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:31.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 --2- 172.21.15.113:0/1983896483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4154b10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009e50 tx=0x7fbbac01f210 comp rx=0 tx=0).stop 2024-03-04T08:13:31.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 -- 172.21.15.113:0/1983896483 shutdown_connections 2024-03-04T08:13:31.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 --2- 172.21.15.113:0/1983896483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4154b10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 --2- 172.21.15.113:0/1983896483 >> v2:172.21.15.136:3300/0 conn(0x7fbbb40ce6b0 0x7fbbb415b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 -- 172.21.15.113:0/1983896483 >> 172.21.15.113:0/1983896483 conn(0x7fbbb40709a0 msgr2=0x7fbbb4070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:31.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 -- 172.21.15.113:0/1983896483 shutdown_connections 2024-03-04T08:13:31.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 -- 172.21.15.113:0/1983896483 wait complete. 2024-03-04T08:13:31.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 Processor -- start 2024-03-04T08:13:31.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 -- start start 2024-03-04T08:13:31.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:31.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbb40ce2d0 0x7fbbb4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:31.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.257+0000 7fbbba400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb40ce6b0 0x7fbbb4155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:31.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbba400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb406c7d0 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbba400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb406e810 con 0x7fbbb40ce2d0 2024-03-04T08:13:31.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbba400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb406f720 con 0x7fbbb406de80 2024-03-04T08:13:31.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:31.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbb3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb40ce6b0 0x7fbbb4155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:31.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbb3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:39694/0 (socket says 172.21.15.113:39694) 2024-03-04T08:13:31.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbb3400640 1 -- 172.21.15.113:0/3040706016 learned_addr learned my addr 172.21.15.113:0/3040706016 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:31.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbabe00640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbb40ce2d0 0x7fbbb4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:31.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbb3e00640 1 -- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 msgr2=0x7fbbb4155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:31.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbb3e00640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4155670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbb3e00640 1 -- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbb40ce2d0 msgr2=0x7fbbb4159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:31.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbb3e00640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbb40ce2d0 0x7fbbb4159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbb3e00640 1 -- 172.21.15.113:0/3040706016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb4149340 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbb3e00640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb40ce6b0 0x7fbbb4155ef0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0ae000 tx=0x7fbbac01bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:31.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbab400640 1 -- 172.21.15.113:0/3040706016 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0c0070 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbab400640 1 -- 172.21.15.113:0/3040706016 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac0090d0 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbab400640 1 -- 172.21.15.113:0/3040706016 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0b4040 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbba400640 1 -- 172.21.15.113:0/3040706016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb41495b0 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbba400640 1 -- 172.21.15.113:0/3040706016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb4149b70 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.261+0000 7fbbba400640 1 -- 172.21.15.113:0/3040706016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb4075c00 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.265+0000 7fbbab400640 1 -- 172.21.15.113:0/3040706016 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbac0c3070 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.265+0000 7fbbab400640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb90066f90 0x7fbb90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:31.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.265+0000 7fbbab400640 1 -- 172.21.15.113:0/3040706016 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7fbbac122ba0 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.265+0000 7fbbb3400640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb90066f90 0x7fbb90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:31.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.265+0000 7fbbb3400640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb90066f90 0x7fbb90069450 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbba4007fd0 tx=0x7fbba4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:31.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.269+0000 7fbbab400640 1 -- 172.21.15.113:0/3040706016 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbbac018d70 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 -- 172.21.15.113:0/3040706016 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbb406cb40 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbab400640 1 -- 172.21.15.113:0/3040706016 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+6621 (secure 0 0 0) 0x7fbbac0f2dd0 con 0x7fbbb40ce6b0 2024-03-04T08:13:31.459 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:31.459 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7839,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7692,7519,7389]},"id":1}]} 2024-03-04T08:13:31.460 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 144 2024-03-04T08:13:31.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 -- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb90066f90 msgr2=0x7fbb90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:31.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb90066f90 0x7fbb90069450 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbba4007fd0 tx=0x7fbba4009040 comp rx=0 tx=0).stop 2024-03-04T08:13:31.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 -- 172.21.15.113:0/3040706016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb40ce6b0 msgr2=0x7fbbb4155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:31.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb40ce6b0 0x7fbbb4155ef0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0ae000 tx=0x7fbbac01bfb0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 -- 172.21.15.113:0/3040706016 shutdown_connections 2024-03-04T08:13:31.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb90066f90 0x7fbb90069450 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbb40ce6b0 0x7fbbb4155ef0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbb40ce2d0 0x7fbbb4159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 --2- 172.21.15.113:0/3040706016 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbb406de80 0x7fbbb4155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 -- 172.21.15.113:0/3040706016 >> 172.21.15.113:0/3040706016 conn(0x7fbbb40709a0 msgr2=0x7fbbb414fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 -- 172.21.15.113:0/3040706016 shutdown_connections 2024-03-04T08:13:31.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:31.461+0000 7fbbba400640 1 -- 172.21.15.113:0/3040706016 wait complete. 2024-03-04T08:13:31.484 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:13:33.486 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:33.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.621+0000 7fd799c00640 1 Processor -- start 2024-03-04T08:13:33.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.621+0000 7fd799c00640 1 -- start start 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd794152950 0x7fd79415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd79406faa0 0x7fd7940c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd79406ed80 0x7fd79406f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7940c8370 con 0x7fd79406ed80 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7940ca0a0 con 0x7fd794152950 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940c4ce0 con 0x7fd79406faa0 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd79406ed80 0x7fd79406f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd79406ed80 0x7fd79406f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45830/0 (socket says 172.21.15.113:45830) 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799200640 1 -- 172.21.15.113:0/3585755673 learned_addr learned my addr 172.21.15.113:0/3585755673 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd78b400640 1 --2- 172.21.15.113:0/3585755673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd79406faa0 0x7fd7940c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:33.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd793e00640 1 --2- 172.21.15.113:0/3585755673 >> v2:172.21.15.136:3300/0 conn(0x7fd794152950 0x7fd79415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:33.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799200640 1 -- 172.21.15.113:0/3585755673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd79406faa0 msgr2=0x7fd7940c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:33.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799200640 1 --2- 172.21.15.113:0/3585755673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd79406faa0 0x7fd7940c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799200640 1 -- 172.21.15.113:0/3585755673 >> v2:172.21.15.136:3300/0 conn(0x7fd794152950 msgr2=0x7fd79415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:33.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799200640 1 --2- 172.21.15.113:0/3585755673 >> v2:172.21.15.136:3300/0 conn(0x7fd794152950 0x7fd79415bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.625+0000 7fd799200640 1 -- 172.21.15.113:0/3585755673 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940c4f80 con 0x7fd79406ed80 2024-03-04T08:13:33.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799200640 1 --2- 172.21.15.113:0/3585755673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd79406ed80 0x7fd79406f230 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd7940ceb90 tx=0x7fd78c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366e32129f7aabfa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:33.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd793400640 1 -- 172.21.15.113:0/3585755673 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0b5070 con 0x7fd79406ed80 2024-03-04T08:13:33.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd793400640 1 -- 172.21.15.113:0/3585755673 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c0a89d0 con 0x7fd79406ed80 2024-03-04T08:13:33.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd793400640 1 -- 172.21.15.113:0/3585755673 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0a8da0 con 0x7fd79406ed80 2024-03-04T08:13:33.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 -- 172.21.15.113:0/3585755673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd79406ed80 msgr2=0x7fd79406f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:33.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 --2- 172.21.15.113:0/3585755673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd79406ed80 0x7fd79406f230 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd7940ceb90 tx=0x7fd78c01ed10 comp rx=0 tx=0).stop 2024-03-04T08:13:33.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 -- 172.21.15.113:0/3585755673 shutdown_connections 2024-03-04T08:13:33.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 --2- 172.21.15.113:0/3585755673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd79406ed80 0x7fd79406f230 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 --2- 172.21.15.113:0/3585755673 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd79406faa0 0x7fd7940c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 --2- 172.21.15.113:0/3585755673 >> v2:172.21.15.136:3300/0 conn(0x7fd794152950 0x7fd79415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 -- 172.21.15.113:0/3585755673 >> 172.21.15.113:0/3585755673 conn(0x7fd794072f40 msgr2=0x7fd7940706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:33.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 -- 172.21.15.113:0/3585755673 shutdown_connections 2024-03-04T08:13:33.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 -- 172.21.15.113:0/3585755673 wait complete. 2024-03-04T08:13:33.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 Processor -- start 2024-03-04T08:13:33.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 -- start start 2024-03-04T08:13:33.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd79406ed80 0x7fd7941e1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:33.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd79406faa0 0x7fd7941e2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:33.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd794152950 0x7fd794158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:33.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940c9a50 con 0x7fd794152950 2024-03-04T08:13:33.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79406fe80 con 0x7fd79406faa0 2024-03-04T08:13:33.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.629+0000 7fd799c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940c9d30 con 0x7fd79406ed80 2024-03-04T08:13:33.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd794152950 0x7fd794158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:33.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd794152950 0x7fd794158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45832/0 (socket says 172.21.15.113:45832) 2024-03-04T08:13:33.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799200640 1 -- 172.21.15.113:0/736623920 learned_addr learned my addr 172.21.15.113:0/736623920 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:33.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd78b400640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd79406faa0 0x7fd7941e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:33.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd793e00640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd79406ed80 0x7fd7941e1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:33.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799200640 1 -- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd79406ed80 msgr2=0x7fd7941e1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:33.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799200640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd79406ed80 0x7fd7941e1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799200640 1 -- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd79406faa0 msgr2=0x7fd7941e2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:33.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799200640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd79406faa0 0x7fd7941e2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799200640 1 -- 172.21.15.113:0/736623920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940b30d0 con 0x7fd794152950 2024-03-04T08:13:33.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799200640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd794152950 0x7fd794158870 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd78c01f5b0 tx=0x7fd78c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:33.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd790c00640 1 -- 172.21.15.113:0/736623920 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0b5070 con 0x7fd794152950 2024-03-04T08:13:33.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd790c00640 1 -- 172.21.15.113:0/736623920 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c0b09f0 con 0x7fd794152950 2024-03-04T08:13:33.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd790c00640 1 -- 172.21.15.113:0/736623920 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0b0de0 con 0x7fd794152950 2024-03-04T08:13:33.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799c00640 1 -- 172.21.15.113:0/736623920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7940b33b0 con 0x7fd794152950 2024-03-04T08:13:33.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799c00640 1 -- 172.21.15.113:0/736623920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7941f3b00 con 0x7fd794152950 2024-03-04T08:13:33.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd799c00640 1 -- 172.21.15.113:0/736623920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd794075a80 con 0x7fd794152950 2024-03-04T08:13:33.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd790c00640 1 -- 172.21.15.113:0/736623920 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd78c0c2060 con 0x7fd794152950 2024-03-04T08:13:33.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd790c00640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd770066f60 0x7fd770069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:33.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.633+0000 7fd790c00640 1 -- 172.21.15.113:0/736623920 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7fd78c122960 con 0x7fd794152950 2024-03-04T08:13:33.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.637+0000 7fd793e00640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd770066f60 0x7fd770069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:33.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.637+0000 7fd793e00640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd770066f60 0x7fd770069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd784003fd0 tx=0x7fd7840039a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:33.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.637+0000 7fd790c00640 1 -- 172.21.15.113:0/736623920 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd78c018870 con 0x7fd794152950 2024-03-04T08:13:33.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 -- 172.21.15.113:0/736623920 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7940c9580 con 0x7fd794152950 2024-03-04T08:13:33.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd790c00640 1 -- 172.21.15.113:0/736623920 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+6621 (secure 0 0 0) 0x7fd78c0f30c0 con 0x7fd794152950 2024-03-04T08:13:33.828 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:33.828 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7839,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7692,7519,7389]},"id":1}]} 2024-03-04T08:13:33.828 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 144 2024-03-04T08:13:33.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 -- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd770066f60 msgr2=0x7fd770069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:33.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd770066f60 0x7fd770069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd784003fd0 tx=0x7fd7840039a0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 -- 172.21.15.113:0/736623920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd794152950 msgr2=0x7fd794158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:33.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd794152950 0x7fd794158870 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd78c01f5b0 tx=0x7fd78c01bab0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 -- 172.21.15.113:0/736623920 shutdown_connections 2024-03-04T08:13:33.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd770066f60 0x7fd770069420 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd794152950 0x7fd794158870 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd79406faa0 0x7fd7941e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 --2- 172.21.15.113:0/736623920 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd79406ed80 0x7fd7941e1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:33.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 -- 172.21.15.113:0/736623920 >> 172.21.15.113:0/736623920 conn(0x7fd794072f40 msgr2=0x7fd794159d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:33.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 -- 172.21.15.113:0/736623920 shutdown_connections 2024-03-04T08:13:33.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:33.829+0000 7fd799c00640 1 -- 172.21.15.113:0/736623920 wait complete. 2024-03-04T08:13:33.849 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:13:35.850 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:35.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f063ca00640 1 Processor -- start 2024-03-04T08:13:35.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f063ca00640 1 -- start start 2024-03-04T08:13:35.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f063ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06380b2dd0 0x7f06380c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:35.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f063ca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0638151980 con 0x7f06381515a0 2024-03-04T08:13:35.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f063ca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0638157740 con 0x7f0638072e10 2024-03-04T08:13:35.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f063ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06380732a0 con 0x7f06380b2dd0 2024-03-04T08:13:35.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f0637400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f06381515a0 0x7f06380c3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:39824/0 (socket says 172.21.15.113:39824) 2024-03-04T08:13:35.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f0637400640 1 -- 172.21.15.113:0/1559075078 learned_addr learned my addr 172.21.15.113:0/1559075078 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:13:35.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f062ea00640 1 --2- 172.21.15.113:0/1559075078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06380b2dd0 0x7f06380c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f0636a00640 1 -- 172.21.15.113:0/1559075078 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013693520 0 0) 0x7f0628001620 con 0x7f06381515a0 2024-03-04T08:13:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f0636a00640 1 -- 172.21.15.113:0/1559075078 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f061c005740 con 0x7f06381515a0 2024-03-04T08:13:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f062ea00640 1 -- 172.21.15.113:0/1559075078 >> v1:172.21.15.136:6789/0 conn(0x7f0638072e10 legacy=0x7f06380c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f0636a00640 1 -- 172.21.15.113:0/1559075078 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662022900 0 0) 0x7f0638157740 con 0x7f0638072e10 2024-03-04T08:13:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f062ea00640 1 -- 172.21.15.113:0/1559075078 >> v1:172.21.15.113:6789/0 conn(0x7f06381515a0 legacy=0x7f06380c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:35.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.989+0000 7f062ea00640 1 -- 172.21.15.113:0/1559075078 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638157a20 con 0x7f06380b2dd0 2024-03-04T08:13:35.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f062ea00640 1 --2- 172.21.15.113:0/1559075078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06380b2dd0 0x7f06380c8340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06200097f0 tx=0x7f062001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b08e1043bb87150 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:35.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f0636a00640 1 -- 172.21.15.113:0/1559075078 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06200b5070 con 0x7f06380b2dd0 2024-03-04T08:13:35.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f0636a00640 1 -- 172.21.15.113:0/1559075078 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06200a89d0 con 0x7f06380b2dd0 2024-03-04T08:13:35.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f0636a00640 1 -- 172.21.15.113:0/1559075078 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06200a8d80 con 0x7f06380b2dd0 2024-03-04T08:13:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f063ca00640 1 -- 172.21.15.113:0/1559075078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06380b2dd0 msgr2=0x7f06380c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f063ca00640 1 --2- 172.21.15.113:0/1559075078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06380b2dd0 0x7f06380c8340 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f06200097f0 tx=0x7f062001ecd0 comp rx=0 tx=0).stop 2024-03-04T08:13:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f063ca00640 1 -- 172.21.15.113:0/1559075078 shutdown_connections 2024-03-04T08:13:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f063ca00640 1 --2- 172.21.15.113:0/1559075078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06380b2dd0 0x7f06380c8340 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f063ca00640 1 -- 172.21.15.113:0/1559075078 >> 172.21.15.113:0/1559075078 conn(0x7f06380c9a30 msgr2=0x7f06380c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f063ca00640 1 -- 172.21.15.113:0/1559075078 shutdown_connections 2024-03-04T08:13:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.993+0000 7f063ca00640 1 -- 172.21.15.113:0/1559075078 wait complete. 2024-03-04T08:13:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f063ca00640 1 Processor -- start 2024-03-04T08:13:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f063ca00640 1 -- start start 2024-03-04T08:13:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f063ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0638072e10 0x7f06381f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:35.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f063ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06380b2dd0 0x7f06381e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f063ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06381515a0 0x7f063813a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f063ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06380734c0 con 0x7f0638072e10 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f063ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638157fe0 con 0x7f06380b2dd0 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f063ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06380c3d20 con 0x7f06381515a0 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f0637400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0638072e10 0x7f06381f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f0637400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0638072e10 0x7f06381f0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45838/0 (socket says 172.21.15.113:45838) 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f0637400640 1 -- 172.21.15.113:0/653601103 learned_addr learned my addr 172.21.15.113:0/653601103 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f0637e00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06381515a0 0x7f063813a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f062ea00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06380b2dd0 0x7f06381e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f0637400640 1 -- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06381515a0 msgr2=0x7f063813a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f0637400640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06381515a0 0x7f063813a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f0637400640 1 -- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06380b2dd0 msgr2=0x7f06381e1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:35.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f0637400640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06380b2dd0 0x7f06381e1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:35.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f0637400640 1 -- 172.21.15.113:0/653601103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06380caf80 con 0x7f0638072e10 2024-03-04T08:13:35.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f062ea00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06380b2dd0 0x7f06381e1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:13:35.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f0637400640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0638072e10 0x7f06381f0e30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f062800c7c0 tx=0x7f062800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:35.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f062fe00640 1 -- 172.21.15.113:0/653601103 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628017070 con 0x7f0638072e10 2024-03-04T08:13:35.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f062fe00640 1 -- 172.21.15.113:0/653601103 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062800a640 con 0x7f0638072e10 2024-03-04T08:13:35.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f062fe00640 1 -- 172.21.15.113:0/653601103 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062800aa50 con 0x7f0638072e10 2024-03-04T08:13:35.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f063ca00640 1 -- 172.21.15.113:0/653601103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06381f44d0 con 0x7f0638072e10 2024-03-04T08:13:35.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:35.997+0000 7f063ca00640 1 -- 172.21.15.113:0/653601103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06381f4a90 con 0x7f0638072e10 2024-03-04T08:13:35.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.001+0000 7f063ca00640 1 -- 172.21.15.113:0/653601103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0638075a80 con 0x7f0638072e10 2024-03-04T08:13:36.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.001+0000 7f062fe00640 1 -- 172.21.15.113:0/653601103 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f062801a070 con 0x7f0638072e10 2024-03-04T08:13:36.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.001+0000 7f062fe00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0614066f60 0x7f0614069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:36.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.001+0000 7f062fe00640 1 -- 172.21.15.113:0/653601103 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f0628084de0 con 0x7f0638072e10 2024-03-04T08:13:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.001+0000 7f062ea00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0614066f60 0x7f0614069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.001+0000 7f062ea00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0614066f60 0x7f0614069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f06200b2000 tx=0x7f06200aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:36.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.009+0000 7f062fe00640 1 -- 172.21.15.113:0/653601103 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0638151980 con 0x7f0638072e10 2024-03-04T08:13:36.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.197+0000 7f063ca00640 1 -- 172.21.15.113:0/653601103 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f063814b630 con 0x7f0638072e10 2024-03-04T08:13:36.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.197+0000 7f062fe00640 1 -- 172.21.15.113:0/653601103 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f0628055090 con 0x7f0638072e10 2024-03-04T08:13:36.198 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:36.199 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7839,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7692,7519,7389]},"id":1}]} 2024-03-04T08:13:36.199 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 144 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 -- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0614066f60 msgr2=0x7f0614069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0614066f60 0x7f0614069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f06200b2000 tx=0x7f06200aa070 comp rx=0 tx=0).stop 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 -- 172.21.15.113:0/653601103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0638072e10 msgr2=0x7f06381f0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0638072e10 0x7f06381f0e30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f062800c7c0 tx=0x7f062800cc80 comp rx=0 tx=0).stop 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 -- 172.21.15.113:0/653601103 shutdown_connections 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f06381515a0 0x7f063813a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06380b2dd0 0x7f06381e1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0614066f60 0x7f0614069420 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 --2- 172.21.15.113:0/653601103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0638072e10 0x7f06381f0e30 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 -- 172.21.15.113:0/653601103 >> 172.21.15.113:0/653601103 conn(0x7f06380c9a30 msgr2=0x7f06380c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 -- 172.21.15.113:0/653601103 shutdown_connections 2024-03-04T08:13:36.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:36.201+0000 7f063ca00640 1 -- 172.21.15.113:0/653601103 wait complete. 2024-03-04T08:13:36.220 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:13:37.340 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:13:37.343 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:13:38.224 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:38.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.361+0000 7f1e77e00640 1 Processor -- start 2024-03-04T08:13:38.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.361+0000 7f1e77e00640 1 -- start start 2024-03-04T08:13:38.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.361+0000 7f1e77e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e70152bd0 0x7f1e70070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:38.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e77e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e701591f0 con 0x7f1e700c8200 2024-03-04T08:13:38.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e77e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e701594d0 con 0x7f1e701527f0 2024-03-04T08:13:38.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e77e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e70071df0 con 0x7f1e70152bd0 2024-03-04T08:13:38.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e74a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1e700c8200 0x7f1e701587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45812/0 (socket says 172.21.15.113:45812) 2024-03-04T08:13:38.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e74a00640 1 -- 172.21.15.113:0/3371401469 learned_addr learned my addr 172.21.15.113:0/3371401469 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:13:38.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e75e00640 1 --2- 172.21.15.113:0/3371401469 >> v2:172.21.15.136:3300/0 conn(0x7f1e70152bd0 0x7f1e70070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:38.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e6fe00640 1 -- 172.21.15.113:0/3371401469 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519950674 0 0) 0x7f1e701591f0 con 0x7f1e700c8200 2024-03-04T08:13:38.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e6fe00640 1 -- 172.21.15.113:0/3371401469 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e58003530 con 0x7f1e700c8200 2024-03-04T08:13:38.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e6fe00640 1 -- 172.21.15.113:0/3371401469 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185865672 0 0) 0x7f1e701594d0 con 0x7f1e701527f0 2024-03-04T08:13:38.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e6fe00640 1 -- 172.21.15.113:0/3371401469 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e701591f0 con 0x7f1e701527f0 2024-03-04T08:13:38.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e75e00640 1 -- 172.21.15.113:0/3371401469 >> v1:172.21.15.136:6789/0 conn(0x7f1e701527f0 legacy=0x7f1e70072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:38.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e75e00640 1 -- 172.21.15.113:0/3371401469 >> v1:172.21.15.113:6789/0 conn(0x7f1e700c8200 legacy=0x7f1e701587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:38.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e6fe00640 1 -- 172.21.15.113:0/3371401469 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1417562110 0 0) 0x7f1e58003530 con 0x7f1e700c8200 2024-03-04T08:13:38.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.365+0000 7f1e75e00640 1 -- 172.21.15.113:0/3371401469 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700717a0 con 0x7f1e70152bd0 2024-03-04T08:13:38.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e75e00640 1 --2- 172.21.15.113:0/3371401469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e70152bd0 0x7f1e70070d40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009d90 tx=0x7f1e6801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f7a8a83ee70e8c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e6fe00640 1 -- 172.21.15.113:0/3371401469 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c001680 con 0x7f1e70152bd0 2024-03-04T08:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e6fe00640 1 -- 172.21.15.113:0/3371401469 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e680a64e0 con 0x7f1e70152bd0 2024-03-04T08:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e6fe00640 1 -- 172.21.15.113:0/3371401469 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680a6890 con 0x7f1e70152bd0 2024-03-04T08:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e77e00640 1 -- 172.21.15.113:0/3371401469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e70152bd0 msgr2=0x7f1e70070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:38.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e77e00640 1 --2- 172.21.15.113:0/3371401469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e70152bd0 0x7f1e70070d40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009d90 tx=0x7f1e6801c890 comp rx=0 tx=0).stop 2024-03-04T08:13:38.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e77e00640 1 -- 172.21.15.113:0/3371401469 shutdown_connections 2024-03-04T08:13:38.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e77e00640 1 --2- 172.21.15.113:0/3371401469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e70152bd0 0x7f1e70070d40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:38.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e77e00640 1 -- 172.21.15.113:0/3371401469 >> 172.21.15.113:0/3371401469 conn(0x7f1e7006e820 msgr2=0x7f1e7006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:38.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e77e00640 1 -- 172.21.15.113:0/3371401469 shutdown_connections 2024-03-04T08:13:38.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e77e00640 1 -- 172.21.15.113:0/3371401469 wait complete. 2024-03-04T08:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e77e00640 1 Processor -- start 2024-03-04T08:13:38.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.369+0000 7f1e77e00640 1 -- start start 2024-03-04T08:13:38.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e77e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e700c8200 0x7f1e700c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:38.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e77e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e701527f0 0x7f1e70141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:38.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e77e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70152bd0 0x7f1e70141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:38.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e77e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70166500 con 0x7f1e70152bd0 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e77e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70166380 con 0x7f1e700c8200 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e77e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70166680 con 0x7f1e701527f0 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e75e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70152bd0 0x7f1e70141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e75e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70152bd0 0x7f1e70141f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43786/0 (socket says 172.21.15.113:43786) 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e75e00640 1 -- 172.21.15.113:0/354956427 learned_addr learned my addr 172.21.15.113:0/354956427 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e74a00640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e701527f0 0x7f1e70141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e75400640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e700c8200 0x7f1e700c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e75e00640 1 -- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e701527f0 msgr2=0x7f1e70141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e75e00640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e701527f0 0x7f1e70141a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e75e00640 1 -- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e700c8200 msgr2=0x7f1e700c2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e75e00640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e700c8200 0x7f1e700c2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:38.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e75e00640 1 -- 172.21.15.113:0/354956427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70145b70 con 0x7f1e70152bd0 2024-03-04T08:13:38.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e75e00640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70152bd0 0x7f1e70141f40 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009d90 tx=0x7f1e68019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:38.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e6d600640 1 -- 172.21.15.113:0/354956427 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680a68a0 con 0x7f1e70152bd0 2024-03-04T08:13:38.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e6d600640 1 -- 172.21.15.113:0/354956427 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e680c0070 con 0x7f1e70152bd0 2024-03-04T08:13:38.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e6d600640 1 -- 172.21.15.113:0/354956427 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680a6c80 con 0x7f1e70152bd0 2024-03-04T08:13:38.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e77e00640 1 -- 172.21.15.113:0/354956427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e70146b40 con 0x7f1e70152bd0 2024-03-04T08:13:38.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e77e00640 1 -- 172.21.15.113:0/354956427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e70147100 con 0x7f1e70152bd0 2024-03-04T08:13:38.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.373+0000 7f1e77e00640 1 -- 172.21.15.113:0/354956427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e70075c00 con 0x7f1e70152bd0 2024-03-04T08:13:38.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.377+0000 7f1e6d600640 1 -- 172.21.15.113:0/354956427 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e680b7040 con 0x7f1e70152bd0 2024-03-04T08:13:38.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.377+0000 7f1e6d600640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e50066f60 0x7f1e50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:38.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.377+0000 7f1e6d600640 1 -- 172.21.15.113:0/354956427 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f1e6811fed0 con 0x7f1e70152bd0 2024-03-04T08:13:38.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.377+0000 7f1e75400640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e50066f60 0x7f1e50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:38.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.377+0000 7f1e75400640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e50066f60 0x7f1e50069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1e640011f0 tx=0x7f1e64009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:38.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.381+0000 7f1e6d600640 1 -- 172.21.15.113:0/354956427 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1e680c9020 con 0x7f1e70152bd0 2024-03-04T08:13:38.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 -- 172.21.15.113:0/354956427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e70156af0 con 0x7f1e70152bd0 2024-03-04T08:13:38.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e6d600640 1 -- 172.21.15.113:0/354956427 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f1e68016480 con 0x7f1e70152bd0 2024-03-04T08:13:38.568 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:38.568 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7839,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7692,7519,7389]},"id":1}]} 2024-03-04T08:13:38.569 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 144 2024-03-04T08:13:38.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 -- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e50066f60 msgr2=0x7f1e50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:38.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e50066f60 0x7f1e50069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1e640011f0 tx=0x7f1e64009040 comp rx=0 tx=0).stop 2024-03-04T08:13:38.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 -- 172.21.15.113:0/354956427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70152bd0 msgr2=0x7f1e70141f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:38.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70152bd0 0x7f1e70141f40 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009d90 tx=0x7f1e68019630 comp rx=0 tx=0).stop 2024-03-04T08:13:38.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 -- 172.21.15.113:0/354956427 shutdown_connections 2024-03-04T08:13:38.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1e50066f60 0x7f1e50069420 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:38.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e70152bd0 0x7f1e70141f40 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:38.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e701527f0 0x7f1e70141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:38.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 --2- 172.21.15.113:0/354956427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e700c8200 0x7f1e700c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:38.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 -- 172.21.15.113:0/354956427 >> 172.21.15.113:0/354956427 conn(0x7f1e7006e820 msgr2=0x7f1e7006ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:38.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 -- 172.21.15.113:0/354956427 shutdown_connections 2024-03-04T08:13:38.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:38.569+0000 7f1e77e00640 1 -- 172.21.15.113:0/354956427 wait complete. 2024-03-04T08:13:38.590 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:13:40.592 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.745+0000 7f71ce800640 1 Processor -- start 2024-03-04T08:13:40.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.745+0000 7f71ce800640 1 -- start start 2024-03-04T08:13:40.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71ce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f71c80b20b0 0x7f71c80c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71ce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f71c806b060 0x7f71c806b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71ce800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c80c1ec0 con 0x7f71c80c85f0 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71ce800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c80c9b30 con 0x7f71c806b060 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71ce800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c80c12a0 con 0x7f71c80b20b0 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71c7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f71c806b060 0x7f71c806b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71c7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f71c806b060 0x7f71c806b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43794/0 (socket says 172.21.15.113:43794) 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71c7e00640 1 -- 172.21.15.113:0/532214428 learned_addr learned my addr 172.21.15.113:0/532214428 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71c7400640 1 --2- 172.21.15.113:0/532214428 >> v2:172.21.15.136:3300/0 conn(0x7f71c80b20b0 0x7f71c80c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71c7e00640 1 -- 172.21.15.113:0/532214428 >> v2:172.21.15.136:3300/0 conn(0x7f71c80b20b0 msgr2=0x7f71c80c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71c7e00640 1 --2- 172.21.15.113:0/532214428 >> v2:172.21.15.136:3300/0 conn(0x7f71c80b20b0 0x7f71c80c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71c7e00640 1 -- 172.21.15.113:0/532214428 >> v1:172.21.15.136:6789/0 conn(0x7f71c80c85f0 legacy=0x7f71c8157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:40.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.749+0000 7f71c7e00640 1 -- 172.21.15.113:0/532214428 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80c1580 con 0x7f71c806b060 2024-03-04T08:13:40.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71c7e00640 1 --2- 172.21.15.113:0/532214428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c806b060 0x7f71c806b470 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f71c8070670 tx=0x7f71c001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa559bb42b744d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71c6000640 1 -- 172.21.15.113:0/532214428 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c00b4070 con 0x7f71c806b060 2024-03-04T08:13:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71c6000640 1 -- 172.21.15.113:0/532214428 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c00af400 con 0x7f71c806b060 2024-03-04T08:13:40.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71c6000640 1 -- 172.21.15.113:0/532214428 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c00af7d0 con 0x7f71c806b060 2024-03-04T08:13:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 -- 172.21.15.113:0/532214428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c806b060 msgr2=0x7f71c806b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 --2- 172.21.15.113:0/532214428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c806b060 0x7f71c806b470 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f71c8070670 tx=0x7f71c001f1e0 comp rx=0 tx=0).stop 2024-03-04T08:13:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 -- 172.21.15.113:0/532214428 shutdown_connections 2024-03-04T08:13:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 --2- 172.21.15.113:0/532214428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c806b060 0x7f71c806b470 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 --2- 172.21.15.113:0/532214428 >> v2:172.21.15.136:3300/0 conn(0x7f71c80b20b0 0x7f71c80c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 -- 172.21.15.113:0/532214428 >> 172.21.15.113:0/532214428 conn(0x7f71c806e820 msgr2=0x7f71c806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 -- 172.21.15.113:0/532214428 shutdown_connections 2024-03-04T08:13:40.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 -- 172.21.15.113:0/532214428 wait complete. 2024-03-04T08:13:40.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 Processor -- start 2024-03-04T08:13:40.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 -- start start 2024-03-04T08:13:40.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c806b060 0x7f71c8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:40.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71c80b20b0 0x7f71c81f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:40.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c80c85f0 0x7f71c81f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8157e20 con 0x7f71c80c85f0 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c81571f0 con 0x7f71c806b060 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71ce800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80c2400 con 0x7f71c80b20b0 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.753+0000 7f71c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c80c85f0 0x7f71c81f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c80c85f0 0x7f71c81f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43808/0 (socket says 172.21.15.113:43808) 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c7e00640 1 -- 172.21.15.113:0/3891569758 learned_addr learned my addr 172.21.15.113:0/3891569758 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c7400640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c806b060 0x7f71c8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c6a00640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71c80b20b0 0x7f71c81f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c7e00640 1 -- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71c80b20b0 msgr2=0x7f71c81f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c7e00640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71c80b20b0 0x7f71c81f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c7e00640 1 -- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c806b060 msgr2=0x7f71c8155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c7e00640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c806b060 0x7f71c8155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:40.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c7e00640 1 -- 172.21.15.113:0/3891569758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c8144660 con 0x7f71c80c85f0 2024-03-04T08:13:40.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c6a00640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71c80b20b0 0x7f71c81f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:13:40.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71c7e00640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c80c85f0 0x7f71c81f2500 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f71c00b2040 tx=0x7f71c001bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:40.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71bf400640 1 -- 172.21.15.113:0/3891569758 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c00b4040 con 0x7f71c80c85f0 2024-03-04T08:13:40.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71bf400640 1 -- 172.21.15.113:0/3891569758 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c00c0070 con 0x7f71c80c85f0 2024-03-04T08:13:40.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71bf400640 1 -- 172.21.15.113:0/3891569758 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c0009510 con 0x7f71c80c85f0 2024-03-04T08:13:40.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71ce800640 1 -- 172.21.15.113:0/3891569758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c8144940 con 0x7f71c80c85f0 2024-03-04T08:13:40.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71ce800640 1 -- 172.21.15.113:0/3891569758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c8152520 con 0x7f71c80c85f0 2024-03-04T08:13:40.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71ce800640 1 -- 172.21.15.113:0/3891569758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c8075c00 con 0x7f71c80c85f0 2024-03-04T08:13:40.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.757+0000 7f71bf400640 1 -- 172.21.15.113:0/3891569758 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71c00c5460 con 0x7f71c80c85f0 2024-03-04T08:13:40.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.761+0000 7f71bf400640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f71a8066f60 0x7f71a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:40.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.761+0000 7f71bf400640 1 -- 172.21.15.113:0/3891569758 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f71c0123860 con 0x7f71c80c85f0 2024-03-04T08:13:40.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.761+0000 7f71c7400640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f71a8066f60 0x7f71a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:40.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.761+0000 7f71c7400640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f71a8066f60 0x7f71a8069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f71b8006aa0 tx=0x7f71b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:40.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.765+0000 7f71bf400640 1 -- 172.21.15.113:0/3891569758 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f71c00ca020 con 0x7f71c80c85f0 2024-03-04T08:13:40.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.953+0000 7f71ce800640 1 -- 172.21.15.113:0/3891569758 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71c8152020 con 0x7f71c80c85f0 2024-03-04T08:13:40.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.953+0000 7f71bf400640 1 -- 172.21.15.113:0/3891569758 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f71c0018d40 con 0x7f71c80c85f0 2024-03-04T08:13:40.952 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:40.952 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7839,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7692,7519,7389]},"id":1}]} 2024-03-04T08:13:40.952 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 144 2024-03-04T08:13:40.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 -- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f71a8066f60 msgr2=0x7f71a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:40.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f71a8066f60 0x7f71a8069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f71b8006aa0 tx=0x7f71b8008040 comp rx=0 tx=0).stop 2024-03-04T08:13:40.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 -- 172.21.15.113:0/3891569758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c80c85f0 msgr2=0x7f71c81f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:40.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c80c85f0 0x7f71c81f2500 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f71c00b2040 tx=0x7f71c001bf80 comp rx=0 tx=0).stop 2024-03-04T08:13:40.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 -- 172.21.15.113:0/3891569758 shutdown_connections 2024-03-04T08:13:40.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f71a8066f60 0x7f71a8069420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:40.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f71c80c85f0 0x7f71c81f2500 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:40.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f71c80b20b0 0x7f71c81f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:40.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 --2- 172.21.15.113:0/3891569758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c806b060 0x7f71c8155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:40.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 -- 172.21.15.113:0/3891569758 >> 172.21.15.113:0/3891569758 conn(0x7f71c806e820 msgr2=0x7f71c81470a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:40.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 -- 172.21.15.113:0/3891569758 shutdown_connections 2024-03-04T08:13:40.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:40.957+0000 7f71ce800640 1 -- 172.21.15.113:0/3891569758 wait complete. 2024-03-04T08:13:40.975 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:13:42.977 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.112+0000 7f270b600640 1 Processor -- start 2024-03-04T08:13:43.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.112+0000 7f270b600640 1 -- start start 2024-03-04T08:13:43.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.112+0000 7f270b600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f27040c8200 0x7f2704070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:43.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.112+0000 7f270b600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2704152bd0 0x7f27041587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f270b600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27041591f0 con 0x7f27041527f0 2024-03-04T08:13:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f270b600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2704071df0 con 0x7f2704152bd0 2024-03-04T08:13:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f270b600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27041597b0 con 0x7f27040c8200 2024-03-04T08:13:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2709600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2704152bd0 0x7f27041587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2709600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2704152bd0 0x7f27041587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43820/0 (socket says 172.21.15.113:43820) 2024-03-04T08:13:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2709600640 1 -- 172.21.15.113:0/2381515969 learned_addr learned my addr 172.21.15.113:0/2381515969 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2708c00640 1 --1- 172.21.15.113:0/2381515969 >> v1:172.21.15.113:6789/0 conn(0x7f27041527f0 0x7f2704072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:45828/0 (socket says 172.21.15.113:45828) 2024-03-04T08:13:43.115 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2703e00640 1 --2- 172.21.15.113:0/2381515969 >> v2:172.21.15.136:3300/0 conn(0x7f27040c8200 0x7f2704070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2709600640 1 -- 172.21.15.113:0/2381515969 >> v2:172.21.15.136:3300/0 conn(0x7f27040c8200 msgr2=0x7f2704070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2709600640 1 --2- 172.21.15.113:0/2381515969 >> v2:172.21.15.136:3300/0 conn(0x7f27040c8200 0x7f2704070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2709600640 1 -- 172.21.15.113:0/2381515969 >> v1:172.21.15.113:6789/0 conn(0x7f27041527f0 legacy=0x7f2704072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:43.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2709600640 1 -- 172.21.15.113:0/2381515969 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27041594d0 con 0x7f2704152bd0 2024-03-04T08:13:43.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2709600640 1 --2- 172.21.15.113:0/2381515969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704152bd0 0x7f27041587d0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f27040cd230 tx=0x7f26fc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20c0841ffb7e315 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2703400640 1 -- 172.21.15.113:0/2381515969 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0a57a0 con 0x7f2704152bd0 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2703400640 1 -- 172.21.15.113:0/2381515969 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26fc0a5940 con 0x7f2704152bd0 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.116+0000 7f2703400640 1 -- 172.21.15.113:0/2381515969 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0a5d10 con 0x7f2704152bd0 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 -- 172.21.15.113:0/2381515969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704152bd0 msgr2=0x7f27041587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 --2- 172.21.15.113:0/2381515969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704152bd0 0x7f27041587d0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f27040cd230 tx=0x7f26fc01c9f0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 -- 172.21.15.113:0/2381515969 shutdown_connections 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 --2- 172.21.15.113:0/2381515969 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2704152bd0 0x7f27041587d0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 --2- 172.21.15.113:0/2381515969 >> v2:172.21.15.136:3300/0 conn(0x7f27040c8200 0x7f2704070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 -- 172.21.15.113:0/2381515969 >> 172.21.15.113:0/2381515969 conn(0x7f270406e820 msgr2=0x7f270406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 -- 172.21.15.113:0/2381515969 shutdown_connections 2024-03-04T08:13:43.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 -- 172.21.15.113:0/2381515969 wait complete. 2024-03-04T08:13:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 Processor -- start 2024-03-04T08:13:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 -- start start 2024-03-04T08:13:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27040c8200 0x7f27041f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27041527f0 0x7f2704153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2704152bd0 0x7f27041f5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27041654f0 con 0x7f27041527f0 2024-03-04T08:13:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2704165370 con 0x7f2704152bd0 2024-03-04T08:13:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f270b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2704165670 con 0x7f27040c8200 2024-03-04T08:13:43.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f2703e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27041527f0 0x7f2704153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f2703e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27041527f0 0x7f2704153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43830/0 (socket says 172.21.15.113:43830) 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f2703e00640 1 -- 172.21.15.113:0/99730173 learned_addr learned my addr 172.21.15.113:0/99730173 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f2708c00640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27040c8200 0x7f27041f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f2709600640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2704152bd0 0x7f27041f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f2703e00640 1 -- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27040c8200 msgr2=0x7f27041f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f2703e00640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27040c8200 0x7f27041f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f2703e00640 1 -- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2704152bd0 msgr2=0x7f27041f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f2703e00640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2704152bd0 0x7f27041f5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.120+0000 7f2703e00640 1 -- 172.21.15.113:0/99730173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2704157400 con 0x7f27041527f0 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.124+0000 7f2703e00640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27041527f0 0x7f2704153310 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f26f000e900 tx=0x7f26f000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.124+0000 7f2700c00640 1 -- 172.21.15.113:0/99730173 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0017070 con 0x7f27041527f0 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.124+0000 7f2700c00640 1 -- 172.21.15.113:0/99730173 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f000ca00 con 0x7f27041527f0 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.124+0000 7f270b600640 1 -- 172.21.15.113:0/99730173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2704151da0 con 0x7f27041527f0 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.124+0000 7f2700c00640 1 -- 172.21.15.113:0/99730173 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000cdf0 con 0x7f27041527f0 2024-03-04T08:13:43.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.124+0000 7f270b600640 1 -- 172.21.15.113:0/99730173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27041f1d40 con 0x7f27041527f0 2024-03-04T08:13:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.124+0000 7f270b600640 1 -- 172.21.15.113:0/99730173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2704068ad0 con 0x7f27041527f0 2024-03-04T08:13:43.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.124+0000 7f2700c00640 1 -- 172.21.15.113:0/99730173 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26f001a070 con 0x7f27041527f0 2024-03-04T08:13:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.124+0000 7f2700c00640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f26e4066f60 0x7f26e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.124+0000 7f2700c00640 1 -- 172.21.15.113:0/99730173 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7f26f0085770 con 0x7f27041527f0 2024-03-04T08:13:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.128+0000 7f2708c00640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f26e4066f60 0x7f26e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:43.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.128+0000 7f2708c00640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f26e4066f60 0x7f26e4069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f26f8000f70 tx=0x7f26f8007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:43.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.132+0000 7f2700c00640 1 -- 172.21.15.113:0/99730173 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f26f0055920 con 0x7f27041527f0 2024-03-04T08:13:43.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.392+0000 7f270b600640 1 -- 172.21.15.113:0/99730173 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2704075c00 con 0x7f27041527f0 2024-03-04T08:13:43.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.392+0000 7f2700c00640 1 -- 172.21.15.113:0/99730173 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f26f00599c0 con 0x7f27041527f0 2024-03-04T08:13:43.394 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:43.395 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7839,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7692,7519,7389]},"id":1}]} 2024-03-04T08:13:43.395 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 144 2024-03-04T08:13:43.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 -- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f26e4066f60 msgr2=0x7f26e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:43.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f26e4066f60 0x7f26e4069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f26f8000f70 tx=0x7f26f8007e90 comp rx=0 tx=0).stop 2024-03-04T08:13:43.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 -- 172.21.15.113:0/99730173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27041527f0 msgr2=0x7f2704153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:43.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27041527f0 0x7f2704153310 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f26f000e900 tx=0x7f26f000edc0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 -- 172.21.15.113:0/99730173 shutdown_connections 2024-03-04T08:13:43.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2704152bd0 0x7f27041f5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f26e4066f60 0x7f26e4069420 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27041527f0 0x7f2704153310 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 --2- 172.21.15.113:0/99730173 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27040c8200 0x7f27041f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:43.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 -- 172.21.15.113:0/99730173 >> 172.21.15.113:0/99730173 conn(0x7f270406e820 msgr2=0x7f2704157600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:43.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 -- 172.21.15.113:0/99730173 shutdown_connections 2024-03-04T08:13:43.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:43.396+0000 7f270b600640 1 -- 172.21.15.113:0/99730173 wait complete. 2024-03-04T08:13:43.417 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:13:45.418 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:45.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d8800640 1 Processor -- start 2024-03-04T08:13:45.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d8800640 1 -- start start 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d8800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3d00ce2d0 0x7ff3d015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d8800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3d00ce6b0 0x7ff3d006f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d8800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 0x7ff3d0154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d8800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d015bbf0 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d8800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d006eb80 con 0x7ff3d00ce2d0 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d8800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d006c210 con 0x7ff3d006de80 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3d00ce6b0 0x7ff3d006f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3d00ce6b0 0x7ff3d006f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43838/0 (socket says 172.21.15.113:43838) 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d5400640 1 -- 172.21.15.113:0/2824376283 learned_addr learned my addr 172.21.15.113:0/2824376283 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d6800640 1 --2- 172.21.15.113:0/2824376283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 0x7ff3d0154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:45.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d5e00640 1 --2- 172.21.15.113:0/2824376283 >> v2:172.21.15.136:3300/0 conn(0x7ff3d00ce2d0 0x7ff3d015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:45.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d5400640 1 -- 172.21.15.113:0/2824376283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 msgr2=0x7ff3d0154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:45.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d5400640 1 --2- 172.21.15.113:0/2824376283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 0x7ff3d0154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d5400640 1 -- 172.21.15.113:0/2824376283 >> v2:172.21.15.136:3300/0 conn(0x7ff3d00ce2d0 msgr2=0x7ff3d015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:45.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d5400640 1 --2- 172.21.15.113:0/2824376283 >> v2:172.21.15.136:3300/0 conn(0x7ff3d00ce2d0 0x7ff3d015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d5400640 1 -- 172.21.15.113:0/2824376283 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d006c4f0 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d6800640 1 --2- 172.21.15.113:0/2824376283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 0x7ff3d0154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:13:45.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.556+0000 7ff3d5e00640 1 --2- 172.21.15.113:0/2824376283 >> v2:172.21.15.136:3300/0 conn(0x7ff3d00ce2d0 0x7ff3d015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:13:45.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d5400640 1 --2- 172.21.15.113:0/2824376283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 0x7ff3d006f010 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00b3ca0 tx=0x7ff3c001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9229094f3fd7949 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:45.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d4a00640 1 -- 172.21.15.113:0/2824376283 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00b5070 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d4a00640 1 -- 172.21.15.113:0/2824376283 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c00a89d0 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d4a00640 1 -- 172.21.15.113:0/2824376283 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00a8da0 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d8800640 1 -- 172.21.15.113:0/2824376283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 msgr2=0x7ff3d006f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:45.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d8800640 1 --2- 172.21.15.113:0/2824376283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 0x7ff3d006f010 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00b3ca0 tx=0x7ff3c001ed10 comp rx=0 tx=0).stop 2024-03-04T08:13:45.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d8800640 1 -- 172.21.15.113:0/2824376283 shutdown_connections 2024-03-04T08:13:45.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d8800640 1 --2- 172.21.15.113:0/2824376283 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 0x7ff3d0154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d8800640 1 --2- 172.21.15.113:0/2824376283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 0x7ff3d006f010 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d8800640 1 --2- 172.21.15.113:0/2824376283 >> v2:172.21.15.136:3300/0 conn(0x7ff3d00ce2d0 0x7ff3d015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d8800640 1 -- 172.21.15.113:0/2824376283 >> 172.21.15.113:0/2824376283 conn(0x7ff3d00709a0 msgr2=0x7ff3d0070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:45.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d8800640 1 -- 172.21.15.113:0/2824376283 shutdown_connections 2024-03-04T08:13:45.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.560+0000 7ff3d8800640 1 -- 172.21.15.113:0/2824376283 wait complete. 2024-03-04T08:13:45.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d8800640 1 Processor -- start 2024-03-04T08:13:45.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d8800640 1 -- start start 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d8800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 0x7ff3d00cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d8800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d00ce2d0 0x7ff3d01f1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d8800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 0x7ff3d01f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d8800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d006c7d0 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d8800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d006e810 con 0x7ff3d00ce2d0 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d8800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d0070010 con 0x7ff3d006de80 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 0x7ff3d01f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 0x7ff3d00cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 0x7ff3d01f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43852/0 (socket says 172.21.15.113:43852) 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d6800640 1 -- 172.21.15.113:0/4256625081 learned_addr learned my addr 172.21.15.113:0/4256625081 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:45.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d5400640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d00ce2d0 0x7ff3d01f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d6800640 1 -- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 msgr2=0x7ff3d00cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d6800640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 0x7ff3d00cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d6800640 1 -- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d00ce2d0 msgr2=0x7ff3d01f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d6800640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d00ce2d0 0x7ff3d01f1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d6800640 1 -- 172.21.15.113:0/4256625081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d014f660 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d6800640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 0x7ff3d01f1650 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc007e10 tx=0x7ff3cc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3ca000640 1 -- 172.21.15.113:0/4256625081 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc01a070 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3ca000640 1 -- 172.21.15.113:0/4256625081 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3cc003ba0 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d8800640 1 -- 172.21.15.113:0/4256625081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d014f940 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3ca000640 1 -- 172.21.15.113:0/4256625081 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc004030 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.564+0000 7ff3d8800640 1 -- 172.21.15.113:0/4256625081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d014ff00 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.568+0000 7ff3d8800640 1 -- 172.21.15.113:0/4256625081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d0075c00 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.568+0000 7ff3ca000640 1 -- 172.21.15.113:0/4256625081 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3cc01d070 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.568+0000 7ff3ca000640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3b4066f60 0x7ff3b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:45.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.568+0000 7ff3ca000640 1 -- 172.21.15.113:0/4256625081 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7631+0+0 (secure 0 0 0) 0x7ff3cc085990 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.568+0000 7ff3d5e00640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3b4066f60 0x7ff3b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:45.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.568+0000 7ff3d5e00640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3b4066f60 0x7ff3b4069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4009950 tx=0x7ff3c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:45.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.572+0000 7ff3ca000640 1 -- 172.21.15.113:0/4256625081 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff3cc010070 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.756+0000 7ff3d8800640 1 -- 172.21.15.113:0/4256625081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3d006cb40 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.756+0000 7ff3ca000640 1 -- 172.21.15.113:0/4256625081 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+6621 (secure 0 0 0) 0x7ff3cc055bd0 con 0x7ff3d00ce6b0 2024-03-04T08:13:45.758 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:45.759 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7839,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":144}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:11:32.443026+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7389,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7389":{"gid":7389,"name":"a","rank":0,"incarnation":126,"state":"up:active","state_seq":32,"addr":"172.21.15.113:6836/1310057122","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1310057122},{"type":"v1","addr":"172.21.15.113:6836","nonce":1310057122}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7389,"qdb_cluster":[7692,7519,7389]},"id":1}]} 2024-03-04T08:13:45.759 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 144 2024-03-04T08:13:45.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 -- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3b4066f60 msgr2=0x7ff3b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:45.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3b4066f60 0x7ff3b4069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4009950 tx=0x7ff3c4008040 comp rx=0 tx=0).stop 2024-03-04T08:13:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 -- 172.21.15.113:0/4256625081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 msgr2=0x7ff3d01f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 0x7ff3d01f1650 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc007e10 tx=0x7ff3cc00d4a0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 -- 172.21.15.113:0/4256625081 shutdown_connections 2024-03-04T08:13:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3b4066f60 0x7ff3b4069420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3d00ce6b0 0x7ff3d01f1650 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d00ce2d0 0x7ff3d01f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 --2- 172.21.15.113:0/4256625081 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3d006de80 0x7ff3d00cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 -- 172.21.15.113:0/4256625081 >> 172.21.15.113:0/4256625081 conn(0x7ff3d00709a0 msgr2=0x7ff3d0159250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 -- 172.21.15.113:0/4256625081 shutdown_connections 2024-03-04T08:13:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:45.760+0000 7ff3d8800640 1 -- 172.21.15.113:0/4256625081 wait complete. 2024-03-04T08:13:45.780 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:13:47.781 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:47.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.916+0000 7fb4f8200640 1 Processor -- start 2024-03-04T08:13:47.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.916+0000 7fb4f8200640 1 -- start start 2024-03-04T08:13:47.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f0152a50 0x7fb4f00ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:47.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f8200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb4f015c4e0 0x7fb4f00ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:47.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f8200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f0152e30 con 0x7fb4f006ff20 2024-03-04T08:13:47.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f8200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f00c31b0 con 0x7fb4f015c4e0 2024-03-04T08:13:47.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f006f840 con 0x7fb4f0152a50 2024-03-04T08:13:47.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f4e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb4f006ff20 0x7fb4f015bdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37654/0 (socket says 172.21.15.113:37654) 2024-03-04T08:13:47.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f4e00640 1 -- 172.21.15.113:0/3809224603 learned_addr learned my addr 172.21.15.113:0/3809224603 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:13:47.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f6200640 1 --2- 172.21.15.113:0/3809224603 >> v2:172.21.15.136:3300/0 conn(0x7fb4f015c4e0 0x7fb4f00ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:47.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f5800640 1 --2- 172.21.15.113:0/3809224603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f0152a50 0x7fb4f00ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:47.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4efe00640 1 -- 172.21.15.113:0/3809224603 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502254822 0 0) 0x7fb4f0152e30 con 0x7fb4f006ff20 2024-03-04T08:13:47.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4efe00640 1 -- 172.21.15.113:0/3809224603 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4d8003530 con 0x7fb4f006ff20 2024-03-04T08:13:47.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f6200640 1 -- 172.21.15.113:0/3809224603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f0152a50 msgr2=0x7fb4f00ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:47.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f6200640 1 --2- 172.21.15.113:0/3809224603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f0152a50 0x7fb4f00ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:47.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4efe00640 1 -- 172.21.15.113:0/3809224603 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3434703265 0 0) 0x7fb4d8003530 con 0x7fb4f006ff20 2024-03-04T08:13:47.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f6200640 1 -- 172.21.15.113:0/3809224603 >> v1:172.21.15.113:6789/0 conn(0x7fb4f006ff20 legacy=0x7fb4f015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:47.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.920+0000 7fb4f6200640 1 -- 172.21.15.113:0/3809224603 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f006f280 con 0x7fb4f015c4e0 2024-03-04T08:13:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f6200640 1 --2- 172.21.15.113:0/3809224603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f015c4e0 0x7fb4f00ce660 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009d90 tx=0x7fb4e801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b58c3f19412a44 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4efe00640 1 -- 172.21.15.113:0/3809224603 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80b2070 con 0x7fb4f015c4e0 2024-03-04T08:13:47.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4efe00640 1 -- 172.21.15.113:0/3809224603 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e80a5690 con 0x7fb4f015c4e0 2024-03-04T08:13:47.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4efe00640 1 -- 172.21.15.113:0/3809224603 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc001680 con 0x7fb4f015c4e0 2024-03-04T08:13:47.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f8200640 1 -- 172.21.15.113:0/3809224603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f015c4e0 msgr2=0x7fb4f00ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:47.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f8200640 1 --2- 172.21.15.113:0/3809224603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f015c4e0 0x7fb4f00ce660 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009d90 tx=0x7fb4e801c890 comp rx=0 tx=0).stop 2024-03-04T08:13:47.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f8200640 1 -- 172.21.15.113:0/3809224603 shutdown_connections 2024-03-04T08:13:47.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f8200640 1 --2- 172.21.15.113:0/3809224603 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f015c4e0 0x7fb4f00ce660 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:47.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f8200640 1 --2- 172.21.15.113:0/3809224603 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f0152a50 0x7fb4f00ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:47.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f8200640 1 -- 172.21.15.113:0/3809224603 >> 172.21.15.113:0/3809224603 conn(0x7fb4f00734d0 msgr2=0x7fb4f0066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:47.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f8200640 1 -- 172.21.15.113:0/3809224603 shutdown_connections 2024-03-04T08:13:47.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f8200640 1 -- 172.21.15.113:0/3809224603 wait complete. 2024-03-04T08:13:47.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f8200640 1 Processor -- start 2024-03-04T08:13:47.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.924+0000 7fb4f8200640 1 -- start start 2024-03-04T08:13:47.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f8200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4f006ff20 0x7fb4f014bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:47.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f8200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f0152a50 0x7fb4f0147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:47.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f8200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f015c4e0 0x7fb4f0148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:47.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f8200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0165a00 con 0x7fb4f006ff20 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f8200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0165880 con 0x7fb4f0152a50 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f8200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f0165b80 con 0x7fb4f015c4e0 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4f006ff20 0x7fb4f014bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f5800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4f006ff20 0x7fb4f014bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33214/0 (socket says 172.21.15.113:33214) 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f5800640 1 -- 172.21.15.113:0/2997454827 learned_addr learned my addr 172.21.15.113:0/2997454827 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f6200640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f015c4e0 0x7fb4f0148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f4e00640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f0152a50 0x7fb4f0147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f5800640 1 -- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f015c4e0 msgr2=0x7fb4f0148360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f5800640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f015c4e0 0x7fb4f0148360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f5800640 1 -- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f0152a50 msgr2=0x7fb4f0147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f5800640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f0152a50 0x7fb4f0147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f5800640 1 -- 172.21.15.113:0/2997454827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0155bd0 con 0x7fb4f006ff20 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f4e00640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f0152a50 0x7fb4f0147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f6200640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f015c4e0 0x7fb4f0148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:13:47.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f5800640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4f006ff20 0x7fb4f014bb10 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e930 tx=0x7fb4e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:47.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4ed600640 1 -- 172.21.15.113:0/2997454827 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4017070 con 0x7fb4f006ff20 2024-03-04T08:13:47.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4ed600640 1 -- 172.21.15.113:0/2997454827 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e400ca00 con 0x7fb4f006ff20 2024-03-04T08:13:47.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4ed600640 1 -- 172.21.15.113:0/2997454827 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400cdb0 con 0x7fb4f006ff20 2024-03-04T08:13:47.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f8200640 1 -- 172.21.15.113:0/2997454827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f0155e90 con 0x7fb4f006ff20 2024-03-04T08:13:47.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f8200640 1 -- 172.21.15.113:0/2997454827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f006dbf0 con 0x7fb4f006ff20 2024-03-04T08:13:47.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4f8200640 1 -- 172.21.15.113:0/2997454827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f0075c00 con 0x7fb4f006ff20 2024-03-04T08:13:47.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.928+0000 7fb4ed600640 1 -- 172.21.15.113:0/2997454827 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4e401a070 con 0x7fb4f006ff20 2024-03-04T08:13:47.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.932+0000 7fb4ed600640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4d0066f60 0x7fb4d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:47.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.932+0000 7fb4ed600640 1 -- 172.21.15.113:0/2997454827 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7fb4e40858e0 con 0x7fb4f006ff20 2024-03-04T08:13:47.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.932+0000 7fb4f4e00640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4d0066f60 0x7fb4d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:47.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.932+0000 7fb4f4e00640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4d0066f60 0x7fb4d0069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0011f0 tx=0x7fb4dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:47.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:47.936+0000 7fb4ed600640 1 -- 172.21.15.113:0/2997454827 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb4e4056130 con 0x7fb4f006ff20 2024-03-04T08:13:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.124+0000 7fb4f8200640 1 -- 172.21.15.113:0/2997454827 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4f0076a60 con 0x7fb4f006ff20 2024-03-04T08:13:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4ed600640 1 -- 172.21.15.113:0/2997454827 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+5820 (secure 0 0 0) 0x7fb4e4055a30 con 0x7fb4f006ff20 2024-03-04T08:13:48.127 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:48.127 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":145,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:13:45.912080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7519,7692]},"id":1}]} 2024-03-04T08:13:48.127 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 145 2024-03-04T08:13:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 -- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4d0066f60 msgr2=0x7fb4d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4d0066f60 0x7fb4d0069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0011f0 tx=0x7fb4dc009040 comp rx=0 tx=0).stop 2024-03-04T08:13:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 -- 172.21.15.113:0/2997454827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4f006ff20 msgr2=0x7fb4f014bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4f006ff20 0x7fb4f014bb10 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e930 tx=0x7fb4e400edf0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 -- 172.21.15.113:0/2997454827 shutdown_connections 2024-03-04T08:13:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4f015c4e0 0x7fb4f0148360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4f0152a50 0x7fb4f0147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb4d0066f60 0x7fb4d0069420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 --2- 172.21.15.113:0/2997454827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4f006ff20 0x7fb4f014bb10 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 -- 172.21.15.113:0/2997454827 >> 172.21.15.113:0/2997454827 conn(0x7fb4f00734d0 msgr2=0x7fb4f00ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 -- 172.21.15.113:0/2997454827 shutdown_connections 2024-03-04T08:13:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.128+0000 7fb4f8200640 1 -- 172.21.15.113:0/2997454827 wait complete. 2024-03-04T08:13:48.148 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-03-04T08:13:48.148 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:13:48.148 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:48.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.284+0000 7fa827a00640 1 Processor -- start 2024-03-04T08:13:48.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.284+0000 7fa827a00640 1 -- start start 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.284+0000 7fa827a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa82014e4c0 0x7fa820072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.284+0000 7fa827a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa82015b900 0x7fa8200731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.284+0000 7fa827a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8200c27f0 con 0x7fa8200c22e0 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.284+0000 7fa827a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82015ad60 con 0x7fa82014e4c0 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.284+0000 7fa827a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200c2a90 con 0x7fa82015b900 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa825000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa82014e4c0 0x7fa820072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa825000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa82014e4c0 0x7fa820072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33220/0 (socket says 172.21.15.113:33220) 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa825000640 1 -- 172.21.15.113:0/2824592407 learned_addr learned my addr 172.21.15.113:0/2824592407 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa825a00640 1 --2- 172.21.15.113:0/2824592407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa82015b900 0x7fa8200731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa825000640 1 -- 172.21.15.113:0/2824592407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa82015b900 msgr2=0x7fa8200731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa825000640 1 --2- 172.21.15.113:0/2824592407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa82015b900 0x7fa8200731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa825000640 1 -- 172.21.15.113:0/2824592407 >> v1:172.21.15.136:6789/0 conn(0x7fa8200c22e0 legacy=0x7fa82015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:48.288 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa825000640 1 -- 172.21.15.113:0/2824592407 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa820070c60 con 0x7fa82014e4c0 2024-03-04T08:13:48.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa825000640 1 --2- 172.21.15.113:0/2824592407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 0x7fa820072c80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fa814009870 tx=0x7fa81401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc159c63ed80d4c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:48.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa81f400640 1 -- 172.21.15.113:0/2824592407 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8140b5070 con 0x7fa82014e4c0 2024-03-04T08:13:48.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa81f400640 1 -- 172.21.15.113:0/2824592407 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8140a8a50 con 0x7fa82014e4c0 2024-03-04T08:13:48.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.288+0000 7fa81f400640 1 -- 172.21.15.113:0/2824592407 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8140a8e20 con 0x7fa82014e4c0 2024-03-04T08:13:48.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 -- 172.21.15.113:0/2824592407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 msgr2=0x7fa820072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:48.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 --2- 172.21.15.113:0/2824592407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 0x7fa820072c80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fa814009870 tx=0x7fa81401ed50 comp rx=0 tx=0).stop 2024-03-04T08:13:48.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 -- 172.21.15.113:0/2824592407 shutdown_connections 2024-03-04T08:13:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 --2- 172.21.15.113:0/2824592407 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa82015b900 0x7fa8200731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 --2- 172.21.15.113:0/2824592407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 0x7fa820072c80 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 -- 172.21.15.113:0/2824592407 >> 172.21.15.113:0/2824592407 conn(0x7fa82006e820 msgr2=0x7fa82006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 -- 172.21.15.113:0/2824592407 shutdown_connections 2024-03-04T08:13:48.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 -- 172.21.15.113:0/2824592407 wait complete. 2024-03-04T08:13:48.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 Processor -- start 2024-03-04T08:13:48.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 -- start start 2024-03-04T08:13:48.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8200c22e0 0x7fa8201f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 0x7fa820164370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa82015b900 0x7fa8201648b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa820071500 con 0x7fa82014e4c0 2024-03-04T08:13:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa820071220 con 0x7fa8200c22e0 2024-03-04T08:13:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa827a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82014e8a0 con 0x7fa82015b900 2024-03-04T08:13:48.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa81fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 0x7fa820164370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa81fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 0x7fa820164370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33226/0 (socket says 172.21.15.113:33226) 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa81fe00640 1 -- 172.21.15.113:0/1357939592 learned_addr learned my addr 172.21.15.113:0/1357939592 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa825000640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8200c22e0 0x7fa8201f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa825a00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa82015b900 0x7fa8201648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa81fe00640 1 -- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa82015b900 msgr2=0x7fa8201648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa81fe00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa82015b900 0x7fa8201648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa81fe00640 1 -- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8200c22e0 msgr2=0x7fa8201f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa81fe00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8200c22e0 0x7fa8201f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.292+0000 7fa81fe00640 1 -- 172.21.15.113:0/1357939592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82014dc70 con 0x7fa82014e4c0 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.296+0000 7fa81fe00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 0x7fa820164370 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00c3a0 tx=0x7fa80c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.296+0000 7fa81cc00640 1 -- 172.21.15.113:0/1357939592 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0052d0 con 0x7fa82014e4c0 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.296+0000 7fa81cc00640 1 -- 172.21.15.113:0/1357939592 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c005470 con 0x7fa82014e4c0 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.296+0000 7fa81cc00640 1 -- 172.21.15.113:0/1357939592 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c007260 con 0x7fa82014e4c0 2024-03-04T08:13:48.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.296+0000 7fa827a00640 1 -- 172.21.15.113:0/1357939592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8201f5090 con 0x7fa82014e4c0 2024-03-04T08:13:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.296+0000 7fa827a00640 1 -- 172.21.15.113:0/1357939592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa820162760 con 0x7fa82014e4c0 2024-03-04T08:13:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.296+0000 7fa827a00640 1 -- 172.21.15.113:0/1357939592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa820075c00 con 0x7fa82014e4c0 2024-03-04T08:13:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.296+0000 7fa81cc00640 1 -- 172.21.15.113:0/1357939592 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa80c007400 con 0x7fa82014e4c0 2024-03-04T08:13:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.300+0000 7fa81cc00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa800066f90 0x7fa800069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.300+0000 7fa81cc00640 1 -- 172.21.15.113:0/1357939592 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7fa80c084c40 con 0x7fa82014e4c0 2024-03-04T08:13:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.300+0000 7fa825000640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa800066f90 0x7fa800069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.300+0000 7fa825000640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa800066f90 0x7fa800069450 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa8140b2000 tx=0x7fa814009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:48.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.304+0000 7fa81cc00640 1 -- 172.21.15.113:0/1357939592 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa80c054d80 con 0x7fa82014e4c0 2024-03-04T08:13:48.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 -- 172.21.15.113:0/1357939592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa820155190 con 0x7fa82014e4c0 2024-03-04T08:13:48.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa81cc00640 1 -- 172.21.15.113:0/1357939592 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+5820 (secure 0 0 0) 0x7fa80c058e20 con 0x7fa82014e4c0 2024-03-04T08:13:48.624 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:48.624 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":145,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:13:45.912080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7519,7692]},"id":1}]} 2024-03-04T08:13:48.625 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 145 2024-03-04T08:13:48.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 -- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa800066f90 msgr2=0x7fa800069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:48.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa800066f90 0x7fa800069450 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa8140b2000 tx=0x7fa814009040 comp rx=0 tx=0).stop 2024-03-04T08:13:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 -- 172.21.15.113:0/1357939592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 msgr2=0x7fa820164370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 0x7fa820164370 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa80c00c3a0 tx=0x7fa80c00c860 comp rx=0 tx=0).stop 2024-03-04T08:13:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 -- 172.21.15.113:0/1357939592 shutdown_connections 2024-03-04T08:13:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa82015b900 0x7fa8201648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa800066f90 0x7fa800069450 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa82014e4c0 0x7fa820164370 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 --2- 172.21.15.113:0/1357939592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8200c22e0 0x7fa8201f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 -- 172.21.15.113:0/1357939592 >> 172.21.15.113:0/1357939592 conn(0x7fa82006e820 msgr2=0x7fa82015b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 -- 172.21.15.113:0/1357939592 shutdown_connections 2024-03-04T08:13:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:48.624+0000 7fa827a00640 1 -- 172.21.15.113:0/1357939592 wait complete. 2024-03-04T08:13:48.654 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 145, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 145, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7519": { "addr": "172.21.15.136:6835/247269911", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 247269911, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 247269911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7519, "incarnation": 133, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 36 }, "gid_7692": { "addr": "172.21.15.113:6833/1928887637", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 1928887637, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 1928887637, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7692, "incarnation": 139, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 70, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:13:45.912080+0000", "qdb_cluster": [ 7519, 7692 ], "qdb_leader": 7519, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 7519, "mds_2": 7692 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:13:50.655 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:50.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc37a600640 1 Processor -- start 2024-03-04T08:13:50.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc37a600640 1 -- start start 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc37a600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3740ce2d0 0x7fc37415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc37a600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc3740ce6b0 0x7fc37406f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc37a600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc37406de80 0x7fc374154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc37a600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37415bbf0 con 0x7fc3740ce2d0 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc37a600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37406eb80 con 0x7fc3740ce6b0 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc37a600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37406c210 con 0x7fc37406de80 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc373400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3740ce2d0 0x7fc37415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc373400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc3740ce2d0 0x7fc37415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33232/0 (socket says 172.21.15.113:33232) 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc373400640 1 -- 172.21.15.113:0/1241103 learned_addr learned my addr 172.21.15.113:0/1241103 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc373e00640 1 --2- 172.21.15.113:0/1241103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc37406de80 0x7fc374154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:50.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc372a00640 1 --2- 172.21.15.113:0/1241103 >> v2:172.21.15.136:3300/0 conn(0x7fc3740ce6b0 0x7fc37406f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:50.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc373400640 1 -- 172.21.15.113:0/1241103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc37406de80 msgr2=0x7fc374154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:50.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc373400640 1 --2- 172.21.15.113:0/1241103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc37406de80 0x7fc374154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:50.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc373400640 1 -- 172.21.15.113:0/1241103 >> v2:172.21.15.136:3300/0 conn(0x7fc3740ce6b0 msgr2=0x7fc37406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:50.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc373400640 1 --2- 172.21.15.113:0/1241103 >> v2:172.21.15.136:3300/0 conn(0x7fc3740ce6b0 0x7fc37406f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:50.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.792+0000 7fc373400640 1 -- 172.21.15.113:0/1241103 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37406c4f0 con 0x7fc3740ce2d0 2024-03-04T08:13:50.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc373400640 1 --2- 172.21.15.113:0/1241103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3740ce2d0 0x7fc37415b6b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fc374158860 tx=0x7fc36401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21d20dc043f6b32 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:50.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc372000640 1 -- 172.21.15.113:0/1241103 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640b5070 con 0x7fc3740ce2d0 2024-03-04T08:13:50.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc372000640 1 -- 172.21.15.113:0/1241103 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3640a89d0 con 0x7fc3740ce2d0 2024-03-04T08:13:50.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc372000640 1 -- 172.21.15.113:0/1241103 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640a8da0 con 0x7fc3740ce2d0 2024-03-04T08:13:50.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc37a600640 1 -- 172.21.15.113:0/1241103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3740ce2d0 msgr2=0x7fc37415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:50.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc37a600640 1 --2- 172.21.15.113:0/1241103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3740ce2d0 0x7fc37415b6b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fc374158860 tx=0x7fc36401ed10 comp rx=0 tx=0).stop 2024-03-04T08:13:50.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc37a600640 1 -- 172.21.15.113:0/1241103 shutdown_connections 2024-03-04T08:13:50.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc37a600640 1 --2- 172.21.15.113:0/1241103 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc37406de80 0x7fc374154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:50.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc37a600640 1 --2- 172.21.15.113:0/1241103 >> v2:172.21.15.136:3300/0 conn(0x7fc3740ce6b0 0x7fc37406f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:50.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc37a600640 1 --2- 172.21.15.113:0/1241103 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3740ce2d0 0x7fc37415b6b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:50.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc37a600640 1 -- 172.21.15.113:0/1241103 >> 172.21.15.113:0/1241103 conn(0x7fc3740709a0 msgr2=0x7fc374070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:50.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc37a600640 1 -- 172.21.15.113:0/1241103 shutdown_connections 2024-03-04T08:13:50.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.796+0000 7fc37a600640 1 -- 172.21.15.113:0/1241103 wait complete. 2024-03-04T08:13:50.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc37a600640 1 Processor -- start 2024-03-04T08:13:50.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc37a600640 1 -- start start 2024-03-04T08:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc37a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37406de80 0x7fc374151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc37a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3740ce2d0 0x7fc37414ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc37a600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3740ce6b0 0x7fc374158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc37a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37406c7d0 con 0x7fc37406de80 2024-03-04T08:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc37a600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37406e810 con 0x7fc3740ce2d0 2024-03-04T08:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc37a600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc374070010 con 0x7fc3740ce6b0 2024-03-04T08:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37406de80 0x7fc374151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37406de80 0x7fc374151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33234/0 (socket says 172.21.15.113:33234) 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373400640 1 -- 172.21.15.113:0/476201972 learned_addr learned my addr 172.21.15.113:0/476201972 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc372a00640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3740ce2d0 0x7fc37414ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373e00640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3740ce6b0 0x7fc374158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373400640 1 -- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3740ce6b0 msgr2=0x7fc374158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373400640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3740ce6b0 0x7fc374158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373400640 1 -- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3740ce2d0 msgr2=0x7fc37414ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373400640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3740ce2d0 0x7fc37414ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373400640 1 -- 172.21.15.113:0/476201972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc374153520 con 0x7fc37406de80 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373e00640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3740ce6b0 0x7fc374158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc373400640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37406de80 0x7fc374151f30 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fc3640b3040 tx=0x7fc36401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc36b400640 1 -- 172.21.15.113:0/476201972 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640b5070 con 0x7fc37406de80 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc36b400640 1 -- 172.21.15.113:0/476201972 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3640b0950 con 0x7fc37406de80 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc36b400640 1 -- 172.21.15.113:0/476201972 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3640b0d40 con 0x7fc37406de80 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc37a600640 1 -- 172.21.15.113:0/476201972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc374150bb0 con 0x7fc37406de80 2024-03-04T08:13:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.800+0000 7fc37a600640 1 -- 172.21.15.113:0/476201972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc374150e50 con 0x7fc37406de80 2024-03-04T08:13:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.804+0000 7fc37a600640 1 -- 172.21.15.113:0/476201972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc374075c00 con 0x7fc37406de80 2024-03-04T08:13:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.804+0000 7fc36b400640 1 -- 172.21.15.113:0/476201972 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3640c2060 con 0x7fc37406de80 2024-03-04T08:13:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.804+0000 7fc36b400640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc354066f60 0x7fc354069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.804+0000 7fc36b400640 1 -- 172.21.15.113:0/476201972 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7fc3641228f0 con 0x7fc37406de80 2024-03-04T08:13:50.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.804+0000 7fc372a00640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc354066f60 0x7fc354069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:50.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.804+0000 7fc372a00640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc354066f60 0x7fc354069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc374157d50 tx=0x7fc360008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:50.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:50.808+0000 7fc36b400640 1 -- 172.21.15.113:0/476201972 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc374153520 con 0x7fc37406de80 2024-03-04T08:13:51.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 -- 172.21.15.113:0/476201972 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3740c47d0 con 0x7fc37406de80 2024-03-04T08:13:51.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc36b400640 1 -- 172.21.15.113:0/476201972 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+5820 (secure 0 0 0) 0x7fc364018870 con 0x7fc37406de80 2024-03-04T08:13:51.047 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:51.047 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":145,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:13:45.912080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7519,7692]},"id":1}]} 2024-03-04T08:13:51.047 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 145 2024-03-04T08:13:51.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 -- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc354066f60 msgr2=0x7fc354069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:51.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc354066f60 0x7fc354069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc374157d50 tx=0x7fc360008040 comp rx=0 tx=0).stop 2024-03-04T08:13:51.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 -- 172.21.15.113:0/476201972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37406de80 msgr2=0x7fc374151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:51.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37406de80 0x7fc374151f30 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fc3640b3040 tx=0x7fc36401bab0 comp rx=0 tx=0).stop 2024-03-04T08:13:51.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 -- 172.21.15.113:0/476201972 shutdown_connections 2024-03-04T08:13:51.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3740ce6b0 0x7fc374158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:51.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3740ce2d0 0x7fc37414ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:51.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc354066f60 0x7fc354069420 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:51.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 --2- 172.21.15.113:0/476201972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc37406de80 0x7fc374151f30 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:51.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 -- 172.21.15.113:0/476201972 >> 172.21.15.113:0/476201972 conn(0x7fc3740709a0 msgr2=0x7fc37414ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:51.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.048+0000 7fc37a600640 1 -- 172.21.15.113:0/476201972 shutdown_connections 2024-03-04T08:13:51.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:51.052+0000 7fc37a600640 1 -- 172.21.15.113:0/476201972 wait complete. 2024-03-04T08:13:51.074 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:13:53.075 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:53.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e9a200640 1 Processor -- start 2024-03-04T08:13:53.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e9a200640 1 -- start start 2024-03-04T08:13:53.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e9a200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3e941527f0 0x7f3e94070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:53.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e9a200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e940c8200 0x7f3e94072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:53.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e9a200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e941591f0 con 0x7f3e94152bd0 2024-03-04T08:13:53.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e9a200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e94071df0 con 0x7f3e941527f0 2024-03-04T08:13:53.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e9a200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e941597b0 con 0x7f3e940c8200 2024-03-04T08:13:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e93e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3e94152bd0 0x7f3e941587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37664/0 (socket says 172.21.15.113:37664) 2024-03-04T08:13:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e93e00640 1 -- 172.21.15.113:0/2241392442 learned_addr learned my addr 172.21.15.113:0/2241392442 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:13:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e93400640 1 --2- 172.21.15.113:0/2241392442 >> v2:172.21.15.136:3300/0 conn(0x7f3e941527f0 0x7f3e94070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e92a00640 1 --2- 172.21.15.113:0/2241392442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e940c8200 0x7f3e94072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e92000640 1 -- 172.21.15.113:0/2241392442 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632600794 0 0) 0x7f3e941591f0 con 0x7f3e94152bd0 2024-03-04T08:13:53.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e92000640 1 -- 172.21.15.113:0/2241392442 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e78003530 con 0x7f3e94152bd0 2024-03-04T08:13:53.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e92000640 1 -- 172.21.15.113:0/2241392442 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3221064286 0 0) 0x7f3e78003530 con 0x7f3e94152bd0 2024-03-04T08:13:53.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e93400640 1 -- 172.21.15.113:0/2241392442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e940c8200 msgr2=0x7f3e94072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:53.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e93400640 1 --2- 172.21.15.113:0/2241392442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e940c8200 0x7f3e94072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:53.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e93400640 1 -- 172.21.15.113:0/2241392442 >> v1:172.21.15.113:6789/0 conn(0x7f3e94152bd0 legacy=0x7f3e941587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:53.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.224+0000 7f3e93400640 1 -- 172.21.15.113:0/2241392442 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e941594d0 con 0x7f3e941527f0 2024-03-04T08:13:53.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e93400640 1 --2- 172.21.15.113:0/2241392442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e941527f0 0x7f3e94070d40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3e840097c0 tx=0x7f3e8401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf677ebf49f3c36 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:53.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e92000640 1 -- 172.21.15.113:0/2241392442 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e840b0070 con 0x7f3e941527f0 2024-03-04T08:13:53.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e92000640 1 -- 172.21.15.113:0/2241392442 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e840ab400 con 0x7f3e941527f0 2024-03-04T08:13:53.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e92000640 1 -- 172.21.15.113:0/2241392442 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c001a00 con 0x7f3e941527f0 2024-03-04T08:13:53.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e9a200640 1 -- 172.21.15.113:0/2241392442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e941527f0 msgr2=0x7f3e94070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:53.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e9a200640 1 --2- 172.21.15.113:0/2241392442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e941527f0 0x7f3e94070d40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3e840097c0 tx=0x7f3e8401c3c0 comp rx=0 tx=0).stop 2024-03-04T08:13:53.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e9a200640 1 -- 172.21.15.113:0/2241392442 shutdown_connections 2024-03-04T08:13:53.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e9a200640 1 --2- 172.21.15.113:0/2241392442 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e940c8200 0x7f3e94072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:53.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e9a200640 1 --2- 172.21.15.113:0/2241392442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e941527f0 0x7f3e94070d40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:53.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e9a200640 1 -- 172.21.15.113:0/2241392442 >> 172.21.15.113:0/2241392442 conn(0x7f3e9406e820 msgr2=0x7f3e9406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:53.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e9a200640 1 -- 172.21.15.113:0/2241392442 shutdown_connections 2024-03-04T08:13:53.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.228+0000 7f3e9a200640 1 -- 172.21.15.113:0/2241392442 wait complete. 2024-03-04T08:13:53.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e9a200640 1 Processor -- start 2024-03-04T08:13:53.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e9a200640 1 -- start start 2024-03-04T08:13:53.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e9a200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e940c8200 0x7f3e94145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:53.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e9a200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e941527f0 0x7f3e941490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:53.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e9a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e94152bd0 0x7f3e941f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:53.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e9a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e94165160 con 0x7f3e94152bd0 2024-03-04T08:13:53.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e9a200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e940727c0 con 0x7f3e941527f0 2024-03-04T08:13:53.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e9a200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e941652e0 con 0x7f3e940c8200 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e93e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e94152bd0 0x7f3e941f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e93e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e94152bd0 0x7f3e941f6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33248/0 (socket says 172.21.15.113:33248) 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e93e00640 1 -- 172.21.15.113:0/3613257250 learned_addr learned my addr 172.21.15.113:0/3613257250 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e92a00640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e941527f0 0x7f3e941490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e93400640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e940c8200 0x7f3e94145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e93e00640 1 -- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e940c8200 msgr2=0x7f3e94145ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e93e00640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e940c8200 0x7f3e94145ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e93e00640 1 -- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e941527f0 msgr2=0x7f3e941490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e93e00640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e941527f0 0x7f3e941490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e93e00640 1 -- 172.21.15.113:0/3613257250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e940b1af0 con 0x7f3e94152bd0 2024-03-04T08:13:53.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.232+0000 7f3e93e00640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e94152bd0 0x7f3e941f6790 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00cc40 tx=0x7f3e8c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:53.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.236+0000 7f3e8b400640 1 -- 172.21.15.113:0/3613257250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c00ce50 con 0x7f3e94152bd0 2024-03-04T08:13:53.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.236+0000 7f3e8b400640 1 -- 172.21.15.113:0/3613257250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c012400 con 0x7f3e94152bd0 2024-03-04T08:13:53.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.236+0000 7f3e8b400640 1 -- 172.21.15.113:0/3613257250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c0127f0 con 0x7f3e94152bd0 2024-03-04T08:13:53.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.236+0000 7f3e9a200640 1 -- 172.21.15.113:0/3613257250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e940b1530 con 0x7f3e94152bd0 2024-03-04T08:13:53.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.236+0000 7f3e9a200640 1 -- 172.21.15.113:0/3613257250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e941453c0 con 0x7f3e94152bd0 2024-03-04T08:13:53.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.236+0000 7f3e9a200640 1 -- 172.21.15.113:0/3613257250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e94075c00 con 0x7f3e94152bd0 2024-03-04T08:13:53.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.236+0000 7f3e8b400640 1 -- 172.21.15.113:0/3613257250 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e8c012990 con 0x7f3e94152bd0 2024-03-04T08:13:53.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.236+0000 7f3e8b400640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e74067130 0x7f3e740695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:53.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.240+0000 7f3e8b400640 1 -- 172.21.15.113:0/3613257250 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f3e8c085f20 con 0x7f3e94152bd0 2024-03-04T08:13:53.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.240+0000 7f3e93400640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e74067130 0x7f3e740695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:53.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.240+0000 7f3e93400640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e74067130 0x7f3e740695f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3e840ad000 tx=0x7f3e840a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:53.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.244+0000 7f3e8b400640 1 -- 172.21.15.113:0/3613257250 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3e8c056070 con 0x7f3e94152bd0 2024-03-04T08:13:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.516+0000 7f3e9a200640 1 -- 172.21.15.113:0/3613257250 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e940c91d0 con 0x7f3e94152bd0 2024-03-04T08:13:53.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.516+0000 7f3e8b400640 1 -- 172.21.15.113:0/3613257250 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+5820 (secure 0 0 0) 0x7f3e8c05a110 con 0x7f3e94152bd0 2024-03-04T08:13:53.516 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:53.516 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":145,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:13:45.912080+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7519,7692]},"id":1}]} 2024-03-04T08:13:53.516 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 145 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 -- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e74067130 msgr2=0x7f3e740695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e74067130 0x7f3e740695f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3e840ad000 tx=0x7f3e840a6000 comp rx=0 tx=0).stop 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 -- 172.21.15.113:0/3613257250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e94152bd0 msgr2=0x7f3e941f6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e94152bd0 0x7f3e941f6790 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00cc40 tx=0x7f3e8c00a430 comp rx=0 tx=0).stop 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 -- 172.21.15.113:0/3613257250 shutdown_connections 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e74067130 0x7f3e740695f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e94152bd0 0x7f3e941f6790 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e941527f0 0x7f3e941490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 --2- 172.21.15.113:0/3613257250 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e940c8200 0x7f3e94145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 -- 172.21.15.113:0/3613257250 >> 172.21.15.113:0/3613257250 conn(0x7f3e9406e820 msgr2=0x7f3e941564e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:53.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 -- 172.21.15.113:0/3613257250 shutdown_connections 2024-03-04T08:13:53.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:53.520+0000 7f3e9a200640 1 -- 172.21.15.113:0/3613257250 wait complete. 2024-03-04T08:13:53.547 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:13:55.547 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:55.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.708+0000 7fd96a600640 1 Processor -- start 2024-03-04T08:13:55.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.708+0000 7fd96a600640 1 -- start start 2024-03-04T08:13:55.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd96a600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd964151840 0x7fd9640c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:55.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd96a600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd96406d140 0x7fd964157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:55.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd96a600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd964157da0 con 0x7fd964151c20 2024-03-04T08:13:55.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd96a600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9640c3440 con 0x7fd96406d140 2024-03-04T08:13:55.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd96a600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd964158080 con 0x7fd964151840 2024-03-04T08:13:55.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd969c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd96406d140 0x7fd964157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:55.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd969200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd964151840 0x7fd9640c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:55.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd963e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd964151c20 0x7fd96406c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37676/0 (socket says 172.21.15.113:37676) 2024-03-04T08:13:55.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd963e00640 1 -- 172.21.15.113:0/3772262972 learned_addr learned my addr 172.21.15.113:0/3772262972 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:13:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd969c00640 1 -- 172.21.15.113:0/3772262972 >> v2:172.21.15.136:3300/0 conn(0x7fd964151840 msgr2=0x7fd9640c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd969c00640 1 --2- 172.21.15.113:0/3772262972 >> v2:172.21.15.136:3300/0 conn(0x7fd964151840 0x7fd9640c1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd969c00640 1 -- 172.21.15.113:0/3772262972 >> v1:172.21.15.113:6789/0 conn(0x7fd964151c20 legacy=0x7fd96406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:55.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.712+0000 7fd969c00640 1 -- 172.21.15.113:0/3772262972 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd964158360 con 0x7fd96406d140 2024-03-04T08:13:55.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd969c00640 1 --2- 172.21.15.113:0/3772262972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd96406d140 0x7fd964157710 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd964070ba0 tx=0x7fd95c021330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e669fd9bd76d9fee server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:55.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd963400640 1 -- 172.21.15.113:0/3772262972 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0b6070 con 0x7fd96406d140 2024-03-04T08:13:55.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd963400640 1 -- 172.21.15.113:0/3772262972 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c00b2d0 con 0x7fd96406d140 2024-03-04T08:13:55.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd963400640 1 -- 172.21.15.113:0/3772262972 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c009760 con 0x7fd96406d140 2024-03-04T08:13:55.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd96a600640 1 -- 172.21.15.113:0/3772262972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd96406d140 msgr2=0x7fd964157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:55.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd96a600640 1 --2- 172.21.15.113:0/3772262972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd96406d140 0x7fd964157710 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd964070ba0 tx=0x7fd95c021330 comp rx=0 tx=0).stop 2024-03-04T08:13:55.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd96a600640 1 -- 172.21.15.113:0/3772262972 shutdown_connections 2024-03-04T08:13:55.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd96a600640 1 --2- 172.21.15.113:0/3772262972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd96406d140 0x7fd964157710 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:55.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd96a600640 1 --2- 172.21.15.113:0/3772262972 >> v2:172.21.15.136:3300/0 conn(0x7fd964151840 0x7fd9640c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:55.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd96a600640 1 -- 172.21.15.113:0/3772262972 >> 172.21.15.113:0/3772262972 conn(0x7fd9640c9a30 msgr2=0x7fd9640c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:55.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd96a600640 1 -- 172.21.15.113:0/3772262972 shutdown_connections 2024-03-04T08:13:55.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd96a600640 1 -- 172.21.15.113:0/3772262972 wait complete. 2024-03-04T08:13:55.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd96a600640 1 Processor -- start 2024-03-04T08:13:55.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.716+0000 7fd96a600640 1 -- start start 2024-03-04T08:13:55.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd96a600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd96406d140 0x7fd9641ece30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd96a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964151840 0x7fd9640c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd96a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd964151c20 0x7fd9641f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd96a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd964071db0 con 0x7fd964151c20 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd96a600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd964071ad0 con 0x7fd964151840 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd96a600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96406c880 con 0x7fd96406d140 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd969c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd964151c20 0x7fd9641f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd969200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd96406d140 0x7fd9641ece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd969c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd964151c20 0x7fd9641f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33270/0 (socket says 172.21.15.113:33270) 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd969c00640 1 -- 172.21.15.113:0/2256565217 learned_addr learned my addr 172.21.15.113:0/2256565217 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd963e00640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964151840 0x7fd9640c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd969c00640 1 -- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd96406d140 msgr2=0x7fd9641ece30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:55.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd969c00640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd96406d140 0x7fd9641ece30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:55.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd969c00640 1 -- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964151840 msgr2=0x7fd9640c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:55.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd969c00640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964151840 0x7fd9640c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:55.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd969c00640 1 -- 172.21.15.113:0/2256565217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9640c7e30 con 0x7fd964151c20 2024-03-04T08:13:55.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd969c00640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd964151c20 0x7fd9641f0050 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0b4040 tx=0x7fd95c01e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:55.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd960c00640 1 -- 172.21.15.113:0/2256565217 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0b6040 con 0x7fd964151c20 2024-03-04T08:13:55.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd960c00640 1 -- 172.21.15.113:0/2256565217 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c00b0d0 con 0x7fd964151c20 2024-03-04T08:13:55.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd96a600640 1 -- 172.21.15.113:0/2256565217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9641ed370 con 0x7fd964151c20 2024-03-04T08:13:55.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd960c00640 1 -- 172.21.15.113:0/2256565217 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00b660 con 0x7fd964151c20 2024-03-04T08:13:55.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd96a600640 1 -- 172.21.15.113:0/2256565217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd964150ef0 con 0x7fd964151c20 2024-03-04T08:13:55.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.720+0000 7fd96a600640 1 -- 172.21.15.113:0/2256565217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd964075a80 con 0x7fd964151c20 2024-03-04T08:13:55.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.724+0000 7fd960c00640 1 -- 172.21.15.113:0/2256565217 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd95c0c4020 con 0x7fd964151c20 2024-03-04T08:13:55.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.724+0000 7fd960c00640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 0x7fd944069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:55.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.724+0000 7fd960c00640 1 -- 172.21.15.113:0/2256565217 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7fd95c1250f0 con 0x7fd964151c20 2024-03-04T08:13:55.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.724+0000 7fd969200640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 0x7fd944069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:55.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.724+0000 7fd969200640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 0x7fd944069450 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd958009b60 tx=0x7fd958009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:55.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.732+0000 7fd960c00640 1 -- 172.21.15.113:0/2256565217 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd95c0cb020 con 0x7fd964151c20 2024-03-04T08:13:55.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.980+0000 7fd96a600640 1 -- 172.21.15.113:0/2256565217 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9641ef990 con 0x7fd964151c20 2024-03-04T08:13:55.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.980+0000 7fd960c00640 1 -- 172.21.15.113:0/2256565217 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5822 (secure 0 0 0) 0x7fd9640c7e30 con 0x7fd964151c20 2024-03-04T08:13:55.977 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:55.977 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:13:55.910046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:resolve","state_seq":36,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7692,7519]},"id":1}]} 2024-03-04T08:13:55.978 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 146 2024-03-04T08:13:55.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 -- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 msgr2=0x7fd944069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:55.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 0x7fd944069450 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd958009b60 tx=0x7fd958009190 comp rx=0 tx=0).stop 2024-03-04T08:13:55.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 -- 172.21.15.113:0/2256565217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd964151c20 msgr2=0x7fd9641f0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:55.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd964151c20 0x7fd9641f0050 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0b4040 tx=0x7fd95c01e0d0 comp rx=0 tx=0).stop 2024-03-04T08:13:55.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 -- 172.21.15.113:0/2256565217 shutdown_connections 2024-03-04T08:13:55.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd944066f90 0x7fd944069450 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:55.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd964151c20 0x7fd9641f0050 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:55.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964151840 0x7fd9640c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:55.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 --2- 172.21.15.113:0/2256565217 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd96406d140 0x7fd9641ece30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:55.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 -- 172.21.15.113:0/2256565217 >> 172.21.15.113:0/2256565217 conn(0x7fd9640c9a30 msgr2=0x7fd96406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:55.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 -- 172.21.15.113:0/2256565217 shutdown_connections 2024-03-04T08:13:55.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:55.984+0000 7fd96a600640 1 -- 172.21.15.113:0/2256565217 wait complete. 2024-03-04T08:13:55.997 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:13:57.999 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:13:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd28600640 1 Processor -- start 2024-03-04T08:13:58.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd28600640 1 -- start start 2024-03-04T08:13:58.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd28600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbd20152b20 0x7fbd20070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd28600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbd200c8b00 0x7fbd2015be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd28600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd2015c340 0x7fbd2015c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd28600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd200c8f90 con 0x7fbd200c8b00 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd28600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd200c5320 con 0x7fbd20152b20 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd28600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd20152f00 con 0x7fbd2015c340 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd25200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbd200c8b00 0x7fbd2015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd25200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbd200c8b00 0x7fbd2015be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49402/0 (socket says 172.21.15.113:49402) 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd25200640 1 -- 172.21.15.113:0/3785489921 learned_addr learned my addr 172.21.15.113:0/3785489921 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd25c00640 1 --2- 172.21.15.113:0/3785489921 >> v2:172.21.15.136:3300/0 conn(0x7fbd20152b20 0x7fbd20070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd26600640 1 --2- 172.21.15.113:0/3785489921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd2015c340 0x7fbd2015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd25200640 1 -- 172.21.15.113:0/3785489921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd2015c340 msgr2=0x7fbd2015c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd25200640 1 --2- 172.21.15.113:0/3785489921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd2015c340 0x7fbd2015c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd25200640 1 -- 172.21.15.113:0/3785489921 >> v2:172.21.15.136:3300/0 conn(0x7fbd20152b20 msgr2=0x7fbd20070010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:13:58.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd25200640 1 --2- 172.21.15.113:0/3785489921 >> v2:172.21.15.136:3300/0 conn(0x7fbd20152b20 0x7fbd20070010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.140+0000 7fbd25200640 1 -- 172.21.15.113:0/3785489921 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd200c5040 con 0x7fbd200c8b00 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd25200640 1 --2- 172.21.15.113:0/3785489921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd200c8b00 0x7fbd2015be00 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbd20072a50 tx=0x7fbd0c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf9148e55874940 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd1fe00640 1 -- 172.21.15.113:0/3785489921 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0b0070 con 0x7fbd200c8b00 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd1fe00640 1 -- 172.21.15.113:0/3785489921 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c0ab600 con 0x7fbd200c8b00 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd1fe00640 1 -- 172.21.15.113:0/3785489921 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0ab9d0 con 0x7fbd200c8b00 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd28600640 1 -- 172.21.15.113:0/3785489921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd200c8b00 msgr2=0x7fbd2015be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd28600640 1 --2- 172.21.15.113:0/3785489921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd200c8b00 0x7fbd2015be00 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbd20072a50 tx=0x7fbd0c01c520 comp rx=0 tx=0).stop 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd28600640 1 -- 172.21.15.113:0/3785489921 shutdown_connections 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd28600640 1 --2- 172.21.15.113:0/3785489921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd2015c340 0x7fbd2015c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd28600640 1 --2- 172.21.15.113:0/3785489921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd200c8b00 0x7fbd2015be00 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd28600640 1 --2- 172.21.15.113:0/3785489921 >> v2:172.21.15.136:3300/0 conn(0x7fbd20152b20 0x7fbd20070010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd28600640 1 -- 172.21.15.113:0/3785489921 >> 172.21.15.113:0/3785489921 conn(0x7fbd200709a0 msgr2=0x7fbd20070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:58.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd28600640 1 -- 172.21.15.113:0/3785489921 shutdown_connections 2024-03-04T08:13:58.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.144+0000 7fbd28600640 1 -- 172.21.15.113:0/3785489921 wait complete. 2024-03-04T08:13:58.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.148+0000 7fbd28600640 1 Processor -- start 2024-03-04T08:13:58.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.148+0000 7fbd28600640 1 -- start start 2024-03-04T08:13:58.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.148+0000 7fbd28600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd200c8b00 0x7fbd20151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:58.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.148+0000 7fbd28600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd20152b20 0x7fbd2006c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:58.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.148+0000 7fbd28600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd2015c340 0x7fbd2006ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:58.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.148+0000 7fbd28600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd20158810 con 0x7fbd2015c340 2024-03-04T08:13:58.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.148+0000 7fbd28600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200c3810 con 0x7fbd200c8b00 2024-03-04T08:13:58.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.148+0000 7fbd28600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200c8490 con 0x7fbd20152b20 2024-03-04T08:13:58.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.148+0000 7fbd26600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd2015c340 0x7fbd2006ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:58.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.148+0000 7fbd25c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd200c8b00 0x7fbd20151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:58.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd25200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd20152b20 0x7fbd2006c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:58.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd26600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd2015c340 0x7fbd2006ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49416/0 (socket says 172.21.15.113:49416) 2024-03-04T08:13:58.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd26600640 1 -- 172.21.15.113:0/239653462 learned_addr learned my addr 172.21.15.113:0/239653462 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:13:58.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd26600640 1 -- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd20152b20 msgr2=0x7fbd2006c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:58.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd26600640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd20152b20 0x7fbd2006c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd26600640 1 -- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd200c8b00 msgr2=0x7fbd20151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:58.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd26600640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd200c8b00 0x7fbd20151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd26600640 1 -- 172.21.15.113:0/239653462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd201581a0 con 0x7fbd2015c340 2024-03-04T08:13:58.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd26600640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd2015c340 0x7fbd2006ca90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009d30 tx=0x7fbd1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:58.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd1d600640 1 -- 172.21.15.113:0/239653462 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18018020 con 0x7fbd2015c340 2024-03-04T08:13:58.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd1d600640 1 -- 172.21.15.113:0/239653462 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd18014480 con 0x7fbd2015c340 2024-03-04T08:13:58.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd1d600640 1 -- 172.21.15.113:0/239653462 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18014870 con 0x7fbd2015c340 2024-03-04T08:13:58.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd28600640 1 -- 172.21.15.113:0/239653462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd201f7560 con 0x7fbd2015c340 2024-03-04T08:13:58.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd28600640 1 -- 172.21.15.113:0/239653462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd201f7b20 con 0x7fbd2015c340 2024-03-04T08:13:58.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd28600640 1 -- 172.21.15.113:0/239653462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd20075c00 con 0x7fbd2015c340 2024-03-04T08:13:58.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd1d600640 1 -- 172.21.15.113:0/239653462 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd1801b020 con 0x7fbd2015c340 2024-03-04T08:13:58.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.152+0000 7fbd1d600640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd00066f90 0x7fbd00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:13:58.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.156+0000 7fbd1d600640 1 -- 172.21.15.113:0/239653462 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7fbd180861b0 con 0x7fbd2015c340 2024-03-04T08:13:58.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.156+0000 7fbd25c00640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd00066f90 0x7fbd00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:13:58.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.156+0000 7fbd25c00640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd00066f90 0x7fbd00069450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009a40 tx=0x7fbd14009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:13:58.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.164+0000 7fbd1d600640 1 -- 172.21.15.113:0/239653462 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbd18022070 con 0x7fbd2015c340 2024-03-04T08:13:58.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 -- 172.21.15.113:0/239653462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd2006fdf0 con 0x7fbd2015c340 2024-03-04T08:13:58.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd1d600640 1 -- 172.21.15.113:0/239653462 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5822 (secure 0 0 0) 0x7fbd18010040 con 0x7fbd2015c340 2024-03-04T08:13:58.348 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:13:58.348 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:13:55.910046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:resolve","state_seq":36,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7692,7519]},"id":1}]} 2024-03-04T08:13:58.348 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 146 2024-03-04T08:13:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 -- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd00066f90 msgr2=0x7fbd00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd00066f90 0x7fbd00069450 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009a40 tx=0x7fbd14009190 comp rx=0 tx=0).stop 2024-03-04T08:13:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 -- 172.21.15.113:0/239653462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd2015c340 msgr2=0x7fbd2006ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:13:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd2015c340 0x7fbd2006ca90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009d30 tx=0x7fbd1800c490 comp rx=0 tx=0).stop 2024-03-04T08:13:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 -- 172.21.15.113:0/239653462 shutdown_connections 2024-03-04T08:13:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd00066f90 0x7fbd00069450 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd2015c340 0x7fbd2006ca90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd20152b20 0x7fbd2006c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 --2- 172.21.15.113:0/239653462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd200c8b00 0x7fbd20151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:13:58.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 -- 172.21.15.113:0/239653462 >> 172.21.15.113:0/239653462 conn(0x7fbd200709a0 msgr2=0x7fbd200cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:13:58.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 -- 172.21.15.113:0/239653462 shutdown_connections 2024-03-04T08:13:58.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:13:58.352+0000 7fbd28600640 1 -- 172.21.15.113:0/239653462 wait complete. 2024-03-04T08:13:58.369 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:14:00.370 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:14:00.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fc6e00640 1 Processor -- start 2024-03-04T08:14:00.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fc6e00640 1 -- start start 2024-03-04T08:14:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fc6e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc0158010 0x7f7fc006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fc6e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7fc006ff20 0x7f7fc015b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fc6e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc00ca0e0 con 0x7f7fc015b750 2024-03-04T08:14:00.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fc6e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc006f450 con 0x7f7fc006ff20 2024-03-04T08:14:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fc6e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc015ad80 con 0x7f7fc0158010 2024-03-04T08:14:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fbf400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7fc006ff20 0x7f7fc015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fc4e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7fc015b750 0x7f7fc00ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36030/0 (socket says 172.21.15.113:36030) 2024-03-04T08:14:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fc4e00640 1 -- 172.21.15.113:0/1190843697 learned_addr learned my addr 172.21.15.113:0/1190843697 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:14:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fbf400640 1 --2- 172.21.15.113:0/1190843697 >> v2:172.21.15.113:3300/0 conn(0x7f7fc006ff20 0x7f7fc015b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49430/0 (socket says 172.21.15.113:49430) 2024-03-04T08:14:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fbfe00640 1 --2- 172.21.15.113:0/1190843697 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc0158010 0x7f7fc006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:00.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fbf400640 1 -- 172.21.15.113:0/1190843697 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc0158010 msgr2=0x7f7fc006dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:14:00.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fbf400640 1 --2- 172.21.15.113:0/1190843697 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc0158010 0x7f7fc006dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:00.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fbf400640 1 -- 172.21.15.113:0/1190843697 >> v1:172.21.15.113:6789/0 conn(0x7f7fc015b750 legacy=0x7f7fc00ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:00.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fbf400640 1 -- 172.21.15.113:0/1190843697 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc015aa30 con 0x7f7fc006ff20 2024-03-04T08:14:00.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.512+0000 7f7fbea00640 1 -- 172.21.15.113:0/1190843697 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883355493 0 0) 0x7f7fc00ca0e0 con 0x7f7fc015b750 2024-03-04T08:14:00.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fbf400640 1 --2- 172.21.15.113:0/1190843697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 0x7f7fc015b210 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0097c0 tx=0x7f7fac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480528ea756f23c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:00.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fbea00640 1 -- 172.21.15.113:0/1190843697 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0b0070 con 0x7f7fc006ff20 2024-03-04T08:14:00.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fbea00640 1 -- 172.21.15.113:0/1190843697 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac0ab600 con 0x7f7fc006ff20 2024-03-04T08:14:00.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fbea00640 1 -- 172.21.15.113:0/1190843697 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0ab9d0 con 0x7f7fc006ff20 2024-03-04T08:14:00.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1190843697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 msgr2=0x7f7fc015b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:00.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fc6e00640 1 --2- 172.21.15.113:0/1190843697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 0x7f7fc015b210 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0097c0 tx=0x7f7fac01c4a0 comp rx=0 tx=0).stop 2024-03-04T08:14:00.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1190843697 shutdown_connections 2024-03-04T08:14:00.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fc6e00640 1 --2- 172.21.15.113:0/1190843697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 0x7f7fc015b210 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:00.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fc6e00640 1 --2- 172.21.15.113:0/1190843697 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc0158010 0x7f7fc006dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:00.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1190843697 >> 172.21.15.113:0/1190843697 conn(0x7f7fc00734d0 msgr2=0x7f7fc0066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:00.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1190843697 shutdown_connections 2024-03-04T08:14:00.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.516+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1190843697 wait complete. 2024-03-04T08:14:00.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fc6e00640 1 Processor -- start 2024-03-04T08:14:00.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fc6e00640 1 -- start start 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fc6e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 0x7f7fc01549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fc6e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc0158010 0x7f7fc01f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fc6e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc015b750 0x7f7fc01f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fc6e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc006c170 con 0x7f7fc006ff20 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fc6e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc01583f0 con 0x7f7fc015b750 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fc6e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc006c2f0 con 0x7f7fc0158010 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fbfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 0x7f7fc01549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fbfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 0x7f7fc01549a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49446/0 (socket says 172.21.15.113:49446) 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fbfe00640 1 -- 172.21.15.113:0/1668884741 learned_addr learned my addr 172.21.15.113:0/1668884741 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fbf400640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc0158010 0x7f7fc01f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fc4e00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc015b750 0x7f7fc01f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:00.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fbfe00640 1 -- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc0158010 msgr2=0x7f7fc01f6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fbfe00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc0158010 0x7f7fc01f6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fbfe00640 1 -- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc015b750 msgr2=0x7f7fc01f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fbfe00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc015b750 0x7f7fc01f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fbfe00640 1 -- 172.21.15.113:0/1668884741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc00cdf30 con 0x7f7fc006ff20 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.520+0000 7f7fc4e00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc015b750 0x7f7fc01f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fbfe00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 0x7f7fc01549a0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000e960 tx=0x7f7fb000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fb7e00640 1 -- 172.21.15.113:0/1668884741 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0017070 con 0x7f7fc006ff20 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fb7e00640 1 -- 172.21.15.113:0/1668884741 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb000ca00 con 0x7f7fc006ff20 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1668884741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc015a760 con 0x7f7fc006ff20 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fb7e00640 1 -- 172.21.15.113:0/1668884741 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000cdb0 con 0x7f7fc006ff20 2024-03-04T08:14:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1668884741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc0158c50 con 0x7f7fc006ff20 2024-03-04T08:14:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1668884741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc0075c00 con 0x7f7fc006ff20 2024-03-04T08:14:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fb7e00640 1 -- 172.21.15.113:0/1668884741 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fb001a070 con 0x7f7fc006ff20 2024-03-04T08:14:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fb7e00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa0066f60 0x7f7fa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fb7e00640 1 -- 172.21.15.113:0/1668884741 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f7fb0085820 con 0x7f7fc006ff20 2024-03-04T08:14:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.524+0000 7f7fbf400640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa0066f60 0x7f7fa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:00.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.528+0000 7f7fbf400640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa0066f60 0x7f7fa0069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0ad000 tx=0x7f7fac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:00.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.528+0000 7f7fb7e00640 1 -- 172.21.15.113:0/1668884741 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7fb0055970 con 0x7f7fc006ff20 2024-03-04T08:14:00.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.720+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1668884741 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fc006fd00 con 0x7f7fc006ff20 2024-03-04T08:14:00.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.720+0000 7f7fb7e00640 1 -- 172.21.15.113:0/1668884741 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5822 (secure 0 0 0) 0x7f7fb0059a10 con 0x7f7fc006ff20 2024-03-04T08:14:00.717 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:14:00.717 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:13:55.910046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:resolve","state_seq":36,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7692,7519]},"id":1}]} 2024-03-04T08:14:00.717 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 146 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa0066f60 msgr2=0x7f7fa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa0066f60 0x7f7fa0069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0ad000 tx=0x7f7fac008040 comp rx=0 tx=0).stop 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1668884741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 msgr2=0x7f7fc01549a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 0x7f7fc01549a0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000e960 tx=0x7f7fb000ee20 comp rx=0 tx=0).stop 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1668884741 shutdown_connections 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7fc015b750 0x7f7fc01f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7fc0158010 0x7f7fc01f6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7fa0066f60 0x7f7fa0069420 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 --2- 172.21.15.113:0/1668884741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7fc006ff20 0x7f7fc01549a0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1668884741 >> 172.21.15.113:0/1668884741 conn(0x7f7fc00734d0 msgr2=0x7f7fc0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1668884741 shutdown_connections 2024-03-04T08:14:00.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:00.724+0000 7f7fc6e00640 1 -- 172.21.15.113:0/1668884741 wait complete. 2024-03-04T08:14:00.747 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 146, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 146, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7519": { "addr": "172.21.15.136:6835/247269911", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 247269911, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 247269911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7519, "incarnation": 133, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 36 }, "gid_7692": { "addr": "172.21.15.113:6833/1928887637", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 1928887637, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 1928887637, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7692, "incarnation": 139, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:resolve", "state_seq": 36 } }, "last_failure": 0, "last_failure_osd_epoch": 70, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:13:55.910046+0000", "qdb_cluster": [ 7692, 7519 ], "qdb_leader": 7519, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 7519, "mds_2": 7692 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:14:02.749 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:14:02.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.888+0000 7effafc00640 1 Processor -- start 2024-03-04T08:14:02.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.888+0000 7effafc00640 1 -- start start 2024-03-04T08:14:02.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.888+0000 7effafc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7effa81527f0 0x7effa8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:02.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.888+0000 7effafc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa81591f0 con 0x7effa80c8200 2024-03-04T08:14:02.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.888+0000 7effafc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa81594d0 con 0x7effa8152bd0 2024-03-04T08:14:02.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.888+0000 7effafc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa8071df0 con 0x7effa81527f0 2024-03-04T08:14:02.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.888+0000 7effa7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7effa80c8200 0x7effa8072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36040/0 (socket says 172.21.15.113:36040) 2024-03-04T08:14:02.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.888+0000 7effa7e00640 1 -- 172.21.15.113:0/3276324601 learned_addr learned my addr 172.21.15.113:0/3276324601 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:14:02.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.888+0000 7effad200640 1 --2- 172.21.15.113:0/3276324601 >> v2:172.21.15.136:3300/0 conn(0x7effa81527f0 0x7effa8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:02.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.892+0000 7effa7400640 1 -- 172.21.15.113:0/3276324601 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644567058 0 0) 0x7effa81591f0 con 0x7effa80c8200 2024-03-04T08:14:02.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.892+0000 7effa7400640 1 -- 172.21.15.113:0/3276324601 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff90003530 con 0x7effa80c8200 2024-03-04T08:14:02.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.892+0000 7effad200640 1 -- 172.21.15.113:0/3276324601 >> v1:172.21.15.136:6789/0 conn(0x7effa8152bd0 legacy=0x7effa81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:02.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.892+0000 7effad200640 1 -- 172.21.15.113:0/3276324601 >> v1:172.21.15.113:6789/0 conn(0x7effa80c8200 legacy=0x7effa8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:02.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.892+0000 7effad200640 1 -- 172.21.15.113:0/3276324601 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa80717a0 con 0x7effa81527f0 2024-03-04T08:14:02.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effad200640 1 --2- 172.21.15.113:0/3276324601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa81527f0 0x7effa8070d40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0097c0 tx=0x7eff9c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bbaaccbea63ab3f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:02.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effa7400640 1 -- 172.21.15.113:0/3276324601 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0b1070 con 0x7effa81527f0 2024-03-04T08:14:02.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effa7400640 1 -- 172.21.15.113:0/3276324601 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c0ac410 con 0x7effa81527f0 2024-03-04T08:14:02.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effa7400640 1 -- 172.21.15.113:0/3276324601 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0ac7c0 con 0x7effa81527f0 2024-03-04T08:14:02.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 -- 172.21.15.113:0/3276324601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa81527f0 msgr2=0x7effa8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:02.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 --2- 172.21.15.113:0/3276324601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa81527f0 0x7effa8070d40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0097c0 tx=0x7eff9c01c4a0 comp rx=0 tx=0).stop 2024-03-04T08:14:02.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 -- 172.21.15.113:0/3276324601 shutdown_connections 2024-03-04T08:14:02.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 --2- 172.21.15.113:0/3276324601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa81527f0 0x7effa8070d40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:02.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 -- 172.21.15.113:0/3276324601 >> 172.21.15.113:0/3276324601 conn(0x7effa806e820 msgr2=0x7effa806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:02.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 -- 172.21.15.113:0/3276324601 shutdown_connections 2024-03-04T08:14:02.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 -- 172.21.15.113:0/3276324601 wait complete. 2024-03-04T08:14:02.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 Processor -- start 2024-03-04T08:14:02.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 -- start start 2024-03-04T08:14:02.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effa80c8200 0x7effa8151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:02.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa81527f0 0x7effa8153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:02.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa8152bd0 0x7effa81f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:02.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa8166380 con 0x7effa80c8200 2024-03-04T08:14:02.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa81597b0 con 0x7effa81527f0 2024-03-04T08:14:02.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effafc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa8166500 con 0x7effa8152bd0 2024-03-04T08:14:02.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.896+0000 7effad200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effa80c8200 0x7effa8151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:02.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effad200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effa80c8200 0x7effa8151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49456/0 (socket says 172.21.15.113:49456) 2024-03-04T08:14:02.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effad200640 1 -- 172.21.15.113:0/3622493136 learned_addr learned my addr 172.21.15.113:0/3622493136 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:14:02.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effadc00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa8152bd0 0x7effa81f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:02.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effa7e00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa81527f0 0x7effa8153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:02.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effad200640 1 -- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa8152bd0 msgr2=0x7effa81f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:02.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effad200640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa8152bd0 0x7effa81f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:02.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effad200640 1 -- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa81527f0 msgr2=0x7effa8153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:02.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effad200640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa81527f0 0x7effa8153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:02.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effad200640 1 -- 172.21.15.113:0/3622493136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa8154fe0 con 0x7effa80c8200 2024-03-04T08:14:02.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effa7e00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa81527f0 0x7effa8153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:14:02.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effad200640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effa80c8200 0x7effa8151e20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0af040 tx=0x7eff9c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:02.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effa4c00640 1 -- 172.21.15.113:0/3622493136 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0b1040 con 0x7effa80c8200 2024-03-04T08:14:02.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effa4c00640 1 -- 172.21.15.113:0/3622493136 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c0acd70 con 0x7effa80c8200 2024-03-04T08:14:02.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effafc00640 1 -- 172.21.15.113:0/3622493136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa8154740 con 0x7effa80c8200 2024-03-04T08:14:02.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effa4c00640 1 -- 172.21.15.113:0/3622493136 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c006e20 con 0x7effa80c8200 2024-03-04T08:14:02.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effafc00640 1 -- 172.21.15.113:0/3622493136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa8154d00 con 0x7effa80c8200 2024-03-04T08:14:02.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effafc00640 1 -- 172.21.15.113:0/3622493136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa8075c00 con 0x7effa80c8200 2024-03-04T08:14:02.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effa4c00640 1 -- 172.21.15.113:0/3622493136 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff9c0be060 con 0x7effa80c8200 2024-03-04T08:14:02.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effa4c00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff88066f60 0x7eff88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:02.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.900+0000 7effa4c00640 1 -- 172.21.15.113:0/3622493136 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7eff9c11e700 con 0x7effa80c8200 2024-03-04T08:14:02.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.904+0000 7effa7e00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff88066f60 0x7eff88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:02.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.904+0000 7effa7e00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff88066f60 0x7eff88069420 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7effa8156ab0 tx=0x7eff94009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:02.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:02.908+0000 7effa4c00640 1 -- 172.21.15.113:0/3622493136 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7eff9c016090 con 0x7effa80c8200 2024-03-04T08:14:03.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 -- 172.21.15.113:0/3622493136 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effa8156800 con 0x7effa80c8200 2024-03-04T08:14:03.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effa4c00640 1 -- 172.21.15.113:0/3622493136 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 147 v147) v1 ==== 95+0+5824 (secure 0 0 0) 0x7eff9c0eed80 con 0x7effa80c8200 2024-03-04T08:14:03.091 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:14:03.091 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":147,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:01.065102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:reconnect","state_seq":38,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7519,7692]},"id":1}]} 2024-03-04T08:14:03.092 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 147 2024-03-04T08:14:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 -- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff88066f60 msgr2=0x7eff88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff88066f60 0x7eff88069420 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7effa8156ab0 tx=0x7eff94009040 comp rx=0 tx=0).stop 2024-03-04T08:14:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 -- 172.21.15.113:0/3622493136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effa80c8200 msgr2=0x7effa8151e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effa80c8200 0x7effa8151e20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0af040 tx=0x7eff9c019240 comp rx=0 tx=0).stop 2024-03-04T08:14:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 -- 172.21.15.113:0/3622493136 shutdown_connections 2024-03-04T08:14:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7effa8152bd0 0x7effa81f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa81527f0 0x7effa8153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7eff88066f60 0x7eff88069420 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 --2- 172.21.15.113:0/3622493136 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7effa80c8200 0x7effa8151e20 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 -- 172.21.15.113:0/3622493136 >> 172.21.15.113:0/3622493136 conn(0x7effa806e820 msgr2=0x7effa8158470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 -- 172.21.15.113:0/3622493136 shutdown_connections 2024-03-04T08:14:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:03.096+0000 7effafc00640 1 -- 172.21.15.113:0/3622493136 wait complete. 2024-03-04T08:14:03.113 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:14:05.113 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:14:05.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.252+0000 7fd7ff000640 1 Processor -- start 2024-03-04T08:14:05.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.252+0000 7fd7ff000640 1 -- start start 2024-03-04T08:14:05.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.252+0000 7fd7ff000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7f80c85f0 0x7fd7f806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:05.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.252+0000 7fd7ff000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f806b5a0 0x7fd7f8161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.252+0000 7fd7ff000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f80c1ec0 con 0x7fd7f80b20b0 2024-03-04T08:14:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.252+0000 7fd7ff000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f80c9b30 con 0x7fd7f80c85f0 2024-03-04T08:14:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.252+0000 7fd7ff000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80c12a0 con 0x7fd7f806b5a0 2024-03-04T08:14:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7f80c85f0 0x7fd7f806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd7f80c85f0 0x7fd7f806b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49460/0 (socket says 172.21.15.113:49460) 2024-03-04T08:14:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f7400640 1 -- 172.21.15.113:0/202873073 learned_addr learned my addr 172.21.15.113:0/202873073 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:14:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f7e00640 1 --1- 172.21.15.113:0/202873073 >> v1:172.21.15.113:6789/0 conn(0x7fd7f80b20b0 0x7fd7f8157710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36048/0 (socket says 172.21.15.113:36048) 2024-03-04T08:14:05.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7fd000640 1 --2- 172.21.15.113:0/202873073 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f806b5a0 0x7fd7f8161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f7400640 1 -- 172.21.15.113:0/202873073 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f806b5a0 msgr2=0x7fd7f8161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:14:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f7400640 1 --2- 172.21.15.113:0/202873073 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f806b5a0 0x7fd7f8161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f7400640 1 -- 172.21.15.113:0/202873073 >> v1:172.21.15.113:6789/0 conn(0x7fd7f80b20b0 legacy=0x7fd7f8157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f7400640 1 -- 172.21.15.113:0/202873073 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80c1580 con 0x7fd7f80c85f0 2024-03-04T08:14:05.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f6a00640 1 -- 172.21.15.113:0/202873073 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568352963 0 0) 0x7fd7f80c1ec0 con 0x7fd7f80b20b0 2024-03-04T08:14:05.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f7400640 1 --2- 172.21.15.113:0/202873073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f80c85f0 0x7fd7f806b060 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009870 tx=0x7fd7e401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2f72a20b639f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:05.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f6a00640 1 -- 172.21.15.113:0/202873073 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40b5070 con 0x7fd7f80c85f0 2024-03-04T08:14:05.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f6a00640 1 -- 172.21.15.113:0/202873073 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e40a8a50 con 0x7fd7f80c85f0 2024-03-04T08:14:05.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.256+0000 7fd7f6a00640 1 -- 172.21.15.113:0/202873073 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e40a8e20 con 0x7fd7f80c85f0 2024-03-04T08:14:05.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 -- 172.21.15.113:0/202873073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f80c85f0 msgr2=0x7fd7f806b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:05.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 --2- 172.21.15.113:0/202873073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f80c85f0 0x7fd7f806b060 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009870 tx=0x7fd7e401ed50 comp rx=0 tx=0).stop 2024-03-04T08:14:05.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 -- 172.21.15.113:0/202873073 shutdown_connections 2024-03-04T08:14:05.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 --2- 172.21.15.113:0/202873073 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f806b5a0 0x7fd7f8161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:05.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 --2- 172.21.15.113:0/202873073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f80c85f0 0x7fd7f806b060 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:05.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 -- 172.21.15.113:0/202873073 >> 172.21.15.113:0/202873073 conn(0x7fd7f806e820 msgr2=0x7fd7f806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:05.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 -- 172.21.15.113:0/202873073 shutdown_connections 2024-03-04T08:14:05.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 -- 172.21.15.113:0/202873073 wait complete. 2024-03-04T08:14:05.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 Processor -- start 2024-03-04T08:14:05.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 -- start start 2024-03-04T08:14:05.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f806b5a0 0x7fd7f81f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:05.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f80b20b0 0x7fd7f81f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:05.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7f80c85f0 0x7fd7f81f05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:05.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f8157e20 con 0x7fd7f806b5a0 2024-03-04T08:14:05.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81571f0 con 0x7fd7f80c85f0 2024-03-04T08:14:05.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7ff000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f80c2400 con 0x7fd7f80b20b0 2024-03-04T08:14:05.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7f7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f806b5a0 0x7fd7f81f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7f7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f806b5a0 0x7fd7f81f3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49462/0 (socket says 172.21.15.113:49462) 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7f7e00640 1 -- 172.21.15.113:0/2432687258 learned_addr learned my addr 172.21.15.113:0/2432687258 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7f7400640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f80b20b0 0x7fd7f81f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7fd000640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7f80c85f0 0x7fd7f81f05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7f7e00640 1 -- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f80b20b0 msgr2=0x7fd7f81f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7f7e00640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f80b20b0 0x7fd7f81f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7f7e00640 1 -- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7f80c85f0 msgr2=0x7fd7f81f05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7f7e00640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7f80c85f0 0x7fd7f81f05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.260+0000 7fd7f7e00640 1 -- 172.21.15.113:0/2432687258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f814ed30 con 0x7fd7f806b5a0 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7f7e00640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f806b5a0 0x7fd7f81f3b60 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80011f0 tx=0x7fd7e800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:05.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7efe00640 1 -- 172.21.15.113:0/2432687258 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8018070 con 0x7fd7f806b5a0 2024-03-04T08:14:05.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7efe00640 1 -- 172.21.15.113:0/2432687258 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e80074e0 con 0x7fd7f806b5a0 2024-03-04T08:14:05.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7efe00640 1 -- 172.21.15.113:0/2432687258 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8004a10 con 0x7fd7f806b5a0 2024-03-04T08:14:05.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7ff000640 1 -- 172.21.15.113:0/2432687258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f814efd0 con 0x7fd7f806b5a0 2024-03-04T08:14:05.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7ff000640 1 -- 172.21.15.113:0/2432687258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f81dedc0 con 0x7fd7f806b5a0 2024-03-04T08:14:05.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7ff000640 1 -- 172.21.15.113:0/2432687258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f8075c00 con 0x7fd7f806b5a0 2024-03-04T08:14:05.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7efe00640 1 -- 172.21.15.113:0/2432687258 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7e8004bb0 con 0x7fd7f806b5a0 2024-03-04T08:14:05.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7efe00640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7d8066f60 0x7fd7d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:05.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7efe00640 1 -- 172.21.15.113:0/2432687258 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7fd7e8084a40 con 0x7fd7f806b5a0 2024-03-04T08:14:05.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.264+0000 7fd7f7400640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7d8066f60 0x7fd7d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:05.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.268+0000 7fd7f7400640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7d8066f60 0x7fd7d8069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40b2000 tx=0x7fd7e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:05.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.272+0000 7fd7efe00640 1 -- 172.21.15.113:0/2432687258 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd7e8054c10 con 0x7fd7f806b5a0 2024-03-04T08:14:05.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.460+0000 7fd7ff000640 1 -- 172.21.15.113:0/2432687258 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7f80c83d0 con 0x7fd7f806b5a0 2024-03-04T08:14:05.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.460+0000 7fd7efe00640 1 -- 172.21.15.113:0/2432687258 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 147 v147) v1 ==== 95+0+5824 (secure 0 0 0) 0x7fd7e8058cb0 con 0x7fd7f806b5a0 2024-03-04T08:14:05.458 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:14:05.458 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":147,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":147,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:01.065102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:reconnect","state_seq":38,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7519,7692]},"id":1}]} 2024-03-04T08:14:05.458 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 147 2024-03-04T08:14:05.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 -- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7d8066f60 msgr2=0x7fd7d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:05.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7d8066f60 0x7fd7d8069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd7e40b2000 tx=0x7fd7e4009040 comp rx=0 tx=0).stop 2024-03-04T08:14:05.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 -- 172.21.15.113:0/2432687258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f806b5a0 msgr2=0x7fd7f81f3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:05.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f806b5a0 0x7fd7f81f3b60 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80011f0 tx=0x7fd7e800c9b0 comp rx=0 tx=0).stop 2024-03-04T08:14:05.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 -- 172.21.15.113:0/2432687258 shutdown_connections 2024-03-04T08:14:05.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7f80c85f0 0x7fd7f81f05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:05.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd7f80b20b0 0x7fd7f81f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:05.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd7d8066f60 0x7fd7d8069420 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:05.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 --2- 172.21.15.113:0/2432687258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd7f806b5a0 0x7fd7f81f3b60 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:05.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 -- 172.21.15.113:0/2432687258 >> 172.21.15.113:0/2432687258 conn(0x7fd7f806e820 msgr2=0x7fd7f80c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:05.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 -- 172.21.15.113:0/2432687258 shutdown_connections 2024-03-04T08:14:05.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:05.464+0000 7fd7ff000640 1 -- 172.21.15.113:0/2432687258 wait complete. 2024-03-04T08:14:05.479 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:14:07.468 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:14:07.472 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:14:07.480 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:14:07.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f4733400640 1 Processor -- start 2024-03-04T08:14:07.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f4733400640 1 -- start start 2024-03-04T08:14:07.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f4733400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f472c0c8200 0x7f472c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:07.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f4733400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472c1591f0 con 0x7f472c152bd0 2024-03-04T08:14:07.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f4733400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472c1594d0 con 0x7f472c1527f0 2024-03-04T08:14:07.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f4733400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472c071df0 con 0x7f472c0c8200 2024-03-04T08:14:07.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f4731400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f472c152bd0 0x7f472c1587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42356/0 (socket says 172.21.15.113:42356) 2024-03-04T08:14:07.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f4731400640 1 -- 172.21.15.113:0/1403211136 learned_addr learned my addr 172.21.15.113:0/1403211136 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:14:07.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f472be00640 1 --2- 172.21.15.113:0/1403211136 >> v2:172.21.15.136:3300/0 conn(0x7f472c0c8200 0x7f472c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:07.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f472b400640 1 -- 172.21.15.113:0/1403211136 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228351132 0 0) 0x7f472c1591f0 con 0x7f472c152bd0 2024-03-04T08:14:07.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f472b400640 1 -- 172.21.15.113:0/1403211136 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4714003530 con 0x7f472c152bd0 2024-03-04T08:14:07.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f472be00640 1 -- 172.21.15.113:0/1403211136 >> v1:172.21.15.136:6789/0 conn(0x7f472c1527f0 legacy=0x7f472c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:07.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f472be00640 1 -- 172.21.15.113:0/1403211136 >> v1:172.21.15.113:6789/0 conn(0x7f472c152bd0 legacy=0x7f472c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:07.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.660+0000 7f472be00640 1 -- 172.21.15.113:0/1403211136 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c0717a0 con 0x7f472c0c8200 2024-03-04T08:14:07.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f472be00640 1 --2- 172.21.15.113:0/1403211136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f472c0c8200 0x7f472c070d40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f47180097c0 tx=0x7f471801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c9a01cc138927e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:07.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f472b400640 1 -- 172.21.15.113:0/1403211136 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180b1070 con 0x7f472c0c8200 2024-03-04T08:14:07.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f472b400640 1 -- 172.21.15.113:0/1403211136 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47180ac410 con 0x7f472c0c8200 2024-03-04T08:14:07.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f472b400640 1 -- 172.21.15.113:0/1403211136 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180ac7e0 con 0x7f472c0c8200 2024-03-04T08:14:07.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f4733400640 1 -- 172.21.15.113:0/1403211136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f472c0c8200 msgr2=0x7f472c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:07.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f4733400640 1 --2- 172.21.15.113:0/1403211136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f472c0c8200 0x7f472c070d40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f47180097c0 tx=0x7f471801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:14:07.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f4733400640 1 -- 172.21.15.113:0/1403211136 shutdown_connections 2024-03-04T08:14:07.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f4733400640 1 --2- 172.21.15.113:0/1403211136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f472c0c8200 0x7f472c070d40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:07.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f4733400640 1 -- 172.21.15.113:0/1403211136 >> 172.21.15.113:0/1403211136 conn(0x7f472c06e820 msgr2=0x7f472c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:07.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f4733400640 1 -- 172.21.15.113:0/1403211136 shutdown_connections 2024-03-04T08:14:07.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.664+0000 7f4733400640 1 -- 172.21.15.113:0/1403211136 wait complete. 2024-03-04T08:14:07.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4733400640 1 Processor -- start 2024-03-04T08:14:07.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4733400640 1 -- start start 2024-03-04T08:14:07.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4733400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f472c0c8200 0x7f472c153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:07.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4733400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f472c1527f0 0x7f472c1f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:07.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4733400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f472c152bd0 0x7f472c0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:07.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4733400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c166380 con 0x7f472c1527f0 2024-03-04T08:14:07.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4733400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c1597b0 con 0x7f472c0c8200 2024-03-04T08:14:07.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4733400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472c166500 con 0x7f472c152bd0 2024-03-04T08:14:07.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f472be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f472c1527f0 0x7f472c1f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:07.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f472be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f472c1527f0 0x7f472c1f7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60536/0 (socket says 172.21.15.113:60536) 2024-03-04T08:14:07.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f472be00640 1 -- 172.21.15.113:0/2080665483 learned_addr learned my addr 172.21.15.113:0/2080665483 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4731400640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f472c152bd0 0x7f472c0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4730a00640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f472c0c8200 0x7f472c153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f472be00640 1 -- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f472c152bd0 msgr2=0x7f472c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f472be00640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f472c152bd0 0x7f472c0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f472be00640 1 -- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f472c0c8200 msgr2=0x7f472c153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f472be00640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f472c0c8200 0x7f472c153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f472be00640 1 -- 172.21.15.113:0/2080665483 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c1f8330 con 0x7f472c1527f0 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f472be00640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f472c1527f0 0x7f472c1f7df0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f47180190b0 tx=0x7f4718019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4728c00640 1 -- 172.21.15.113:0/2080665483 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180b1040 con 0x7f472c1527f0 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4728c00640 1 -- 172.21.15.113:0/2080665483 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47180bb070 con 0x7f472c1527f0 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4728c00640 1 -- 172.21.15.113:0/2080665483 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47180b5410 con 0x7f472c1527f0 2024-03-04T08:14:07.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4733400640 1 -- 172.21.15.113:0/2080665483 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c1544d0 con 0x7f472c1527f0 2024-03-04T08:14:07.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.668+0000 7f4733400640 1 -- 172.21.15.113:0/2080665483 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472c154a90 con 0x7f472c1527f0 2024-03-04T08:14:07.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.672+0000 7f4733400640 1 -- 172.21.15.113:0/2080665483 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472c075c00 con 0x7f472c1527f0 2024-03-04T08:14:07.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.672+0000 7f4728c00640 1 -- 172.21.15.113:0/2080665483 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47180b55b0 con 0x7f472c1527f0 2024-03-04T08:14:07.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.672+0000 7f4728c00640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f470c066f60 0x7f470c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:07.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.672+0000 7f4728c00640 1 -- 172.21.15.113:0/2080665483 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f471811f310 con 0x7f472c1527f0 2024-03-04T08:14:07.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.672+0000 7f4730a00640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f470c066f60 0x7f470c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:07.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.672+0000 7f4730a00640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f470c066f60 0x7f470c069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4720000f70 tx=0x7f4720009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:07.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.676+0000 7f4728c00640 1 -- 172.21.15.113:0/2080665483 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4718016090 con 0x7f472c1527f0 2024-03-04T08:14:07.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.876+0000 7f4733400640 1 -- 172.21.15.113:0/2080665483 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f472c070d40 con 0x7f472c1527f0 2024-03-04T08:14:07.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.876+0000 7f4728c00640 1 -- 172.21.15.113:0/2080665483 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+5821 (secure 0 0 0) 0x7f47180efaa0 con 0x7f472c1527f0 2024-03-04T08:14:07.874 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:14:07.874 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":148,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:06.409091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:rejoin","state_seq":40,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7692,7519]},"id":1}]} 2024-03-04T08:14:07.874 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 148 2024-03-04T08:14:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 -- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f470c066f60 msgr2=0x7f470c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f470c066f60 0x7f470c069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4720000f70 tx=0x7f4720009040 comp rx=0 tx=0).stop 2024-03-04T08:14:07.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 -- 172.21.15.113:0/2080665483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f472c1527f0 msgr2=0x7f472c1f7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f472c1527f0 0x7f472c1f7df0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f47180190b0 tx=0x7f4718019240 comp rx=0 tx=0).stop 2024-03-04T08:14:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 -- 172.21.15.113:0/2080665483 shutdown_connections 2024-03-04T08:14:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f472c152bd0 0x7f472c0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f470c066f60 0x7f470c069420 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f472c1527f0 0x7f472c1f7df0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 --2- 172.21.15.113:0/2080665483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f472c0c8200 0x7f472c153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 -- 172.21.15.113:0/2080665483 >> 172.21.15.113:0/2080665483 conn(0x7f472c06e820 msgr2=0x7f472c0c1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 -- 172.21.15.113:0/2080665483 shutdown_connections 2024-03-04T08:14:07.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:07.880+0000 7f4733400640 1 -- 172.21.15.113:0/2080665483 wait complete. 2024-03-04T08:14:07.895 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:14:09.896 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:14:10.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3087600640 1 Processor -- start 2024-03-04T08:14:10.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3087600640 1 -- start start 2024-03-04T08:14:10.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3087600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3080070010 0x7f30800c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:10.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3087600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f308015c510 0x7f3080163190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:10.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3087600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3080152f00 con 0x7f3080152b20 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3087600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30800c5320 con 0x7f308015c510 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3087600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30800c5040 con 0x7f3080070010 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f307fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3080070010 0x7f30800c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3085600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f308015c510 0x7f3080163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3084c00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f3080152b20 0x7f308015be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:38026/0 (socket says 172.21.15.113:38026) 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f307fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3080070010 0x7f30800c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:50034/0 (socket says 172.21.15.113:50034) 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3084c00640 1 -- 172.21.15.113:0/3388959034 learned_addr learned my addr 172.21.15.113:0/3388959034 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3085600640 1 -- 172.21.15.113:0/3388959034 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3080070010 msgr2=0x7f30800c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3085600640 1 --2- 172.21.15.113:0/3388959034 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3080070010 0x7f30800c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3085600640 1 -- 172.21.15.113:0/3388959034 >> v1:172.21.15.136:6789/0 conn(0x7f3080152b20 legacy=0x7f308015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:10.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.036+0000 7f3085600640 1 -- 172.21.15.113:0/3388959034 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30800c3250 con 0x7f308015c510 2024-03-04T08:14:10.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f3085600640 1 --2- 172.21.15.113:0/3388959034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f308015c510 0x7f3080163190 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f30800c54a0 tx=0x7f307801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f07c318713f236 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:10.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f307f400640 1 -- 172.21.15.113:0/3388959034 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30780b2070 con 0x7f308015c510 2024-03-04T08:14:10.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f307f400640 1 -- 172.21.15.113:0/3388959034 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30780a5810 con 0x7f308015c510 2024-03-04T08:14:10.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f307f400640 1 -- 172.21.15.113:0/3388959034 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30780a5be0 con 0x7f308015c510 2024-03-04T08:14:10.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f3087600640 1 -- 172.21.15.113:0/3388959034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f308015c510 msgr2=0x7f3080163190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:10.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f3087600640 1 --2- 172.21.15.113:0/3388959034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f308015c510 0x7f3080163190 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f30800c54a0 tx=0x7f307801c950 comp rx=0 tx=0).stop 2024-03-04T08:14:10.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f3087600640 1 -- 172.21.15.113:0/3388959034 shutdown_connections 2024-03-04T08:14:10.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f3087600640 1 --2- 172.21.15.113:0/3388959034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f308015c510 0x7f3080163190 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:10.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f3087600640 1 --2- 172.21.15.113:0/3388959034 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3080070010 0x7f30800c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:10.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f3087600640 1 -- 172.21.15.113:0/3388959034 >> 172.21.15.113:0/3388959034 conn(0x7f30800709a0 msgr2=0x7f3080070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:10.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f3087600640 1 -- 172.21.15.113:0/3388959034 shutdown_connections 2024-03-04T08:14:10.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.040+0000 7f3087600640 1 -- 172.21.15.113:0/3388959034 wait complete. 2024-03-04T08:14:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3087600640 1 Processor -- start 2024-03-04T08:14:10.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3087600640 1 -- start start 2024-03-04T08:14:10.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3087600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3080070010 0x7f30800cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:10.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3087600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3080152b20 0x7f308014c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:10.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3087600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308015c510 0x7f308014cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:10.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3087600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30800c8490 con 0x7f3080070010 2024-03-04T08:14:10.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3087600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3080158810 con 0x7f3080152b20 2024-03-04T08:14:10.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3087600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3080157c60 con 0x7f308015c510 2024-03-04T08:14:10.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3084c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3080070010 0x7f30800cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:10.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3084c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3080070010 0x7f30800cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60566/0 (socket says 172.21.15.113:60566) 2024-03-04T08:14:10.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3084c00640 1 -- 172.21.15.113:0/1078254365 learned_addr learned my addr 172.21.15.113:0/1078254365 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:14:10.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3085600640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308015c510 0x7f308014cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:10.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f307fe00640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3080152b20 0x7f308014c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:10.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3084c00640 1 -- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308015c510 msgr2=0x7f308014cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:10.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3084c00640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308015c510 0x7f308014cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:10.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3084c00640 1 -- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3080152b20 msgr2=0x7f308014c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:10.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3084c00640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3080152b20 0x7f308014c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:10.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3084c00640 1 -- 172.21.15.113:0/1078254365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3080074600 con 0x7f3080070010 2024-03-04T08:14:10.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f307fe00640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3080152b20 0x7f308014c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:14:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3084c00640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3080070010 0x7f30800cca90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f307400c3a0 tx=0x7f307400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f307cc00640 1 -- 172.21.15.113:0/1078254365 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074017070 con 0x7f3080070010 2024-03-04T08:14:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f307cc00640 1 -- 172.21.15.113:0/1078254365 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3074005cf0 con 0x7f3080070010 2024-03-04T08:14:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3087600640 1 -- 172.21.15.113:0/1078254365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3080156870 con 0x7f3080070010 2024-03-04T08:14:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f307cc00640 1 -- 172.21.15.113:0/1078254365 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30740072f0 con 0x7f3080070010 2024-03-04T08:14:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.044+0000 7f3087600640 1 -- 172.21.15.113:0/1078254365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30801f4a90 con 0x7f3080070010 2024-03-04T08:14:10.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.048+0000 7f3087600640 1 -- 172.21.15.113:0/1078254365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3080075c00 con 0x7f3080070010 2024-03-04T08:14:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.048+0000 7f307cc00640 1 -- 172.21.15.113:0/1078254365 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f307401a070 con 0x7f3080070010 2024-03-04T08:14:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.048+0000 7f307cc00640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3060066f60 0x7f3060069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.048+0000 7f307cc00640 1 -- 172.21.15.113:0/1078254365 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f3074084ce0 con 0x7f3080070010 2024-03-04T08:14:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.048+0000 7f307fe00640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3060066f60 0x7f3060069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:10.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.048+0000 7f307fe00640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3060066f60 0x7f3060069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f30800cd380 tx=0x7f306c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:10.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.052+0000 7f307cc00640 1 -- 172.21.15.113:0/1078254365 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3080156870 con 0x7f3080070010 2024-03-04T08:14:10.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 -- 172.21.15.113:0/1078254365 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f308006fdf0 con 0x7f3080070010 2024-03-04T08:14:10.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f307cc00640 1 -- 172.21.15.113:0/1078254365 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+5821 (secure 0 0 0) 0x7f30740553e0 con 0x7f3080070010 2024-03-04T08:14:10.300 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:14:10.300 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":148,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":148,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:06.409091+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:rejoin","state_seq":40,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7519,"qdb_cluster":[7692,7519]},"id":1}]} 2024-03-04T08:14:10.300 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 148 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 -- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3060066f60 msgr2=0x7f3060069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3060066f60 0x7f3060069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f30800cd380 tx=0x7f306c008040 comp rx=0 tx=0).stop 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 -- 172.21.15.113:0/1078254365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3080070010 msgr2=0x7f30800cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3080070010 0x7f30800cca90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f307400c3a0 tx=0x7f307400c860 comp rx=0 tx=0).stop 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 -- 172.21.15.113:0/1078254365 shutdown_connections 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f308015c510 0x7f308014cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3080152b20 0x7f308014c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3060066f60 0x7f3060069420 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 --2- 172.21.15.113:0/1078254365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3080070010 0x7f30800cca90 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 -- 172.21.15.113:0/1078254365 >> 172.21.15.113:0/1078254365 conn(0x7f30800709a0 msgr2=0x7f30800c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 -- 172.21.15.113:0/1078254365 shutdown_connections 2024-03-04T08:14:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:10.304+0000 7f3087600640 1 -- 172.21.15.113:0/1078254365 wait complete. 2024-03-04T08:14:10.321 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:14:12.323 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:14:12.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f658b200640 1 Processor -- start 2024-03-04T08:14:12.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f658b200640 1 -- start start 2024-03-04T08:14:12.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f658b200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f658c152c50 0x7f658c06b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:12.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f658b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658c15bda0 con 0x7f658c0c7930 2024-03-04T08:14:12.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f658b200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658c15c040 con 0x7f658c071a30 2024-03-04T08:14:12.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f658b200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f658c0c7fd0 con 0x7f658c152c50 2024-03-04T08:14:12.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f6589e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f658c152c50 0x7f658c06b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:12.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f6589400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f658c0c7930 0x7f658c0c8cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:42366/0 (socket says 172.21.15.113:42366) 2024-03-04T08:14:12.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f6589400640 1 -- 172.21.15.113:0/4002846139 learned_addr learned my addr 172.21.15.113:0/4002846139 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:14:12.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f6589e00640 1 -- 172.21.15.113:0/4002846139 >> v1:172.21.15.136:6789/0 conn(0x7f658c071a30 legacy=0x7f658c071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:12.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f6589e00640 1 -- 172.21.15.113:0/4002846139 >> v1:172.21.15.113:6789/0 conn(0x7f658c0c7930 legacy=0x7f658c0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:12.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.464+0000 7f6589e00640 1 -- 172.21.15.113:0/4002846139 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c15c320 con 0x7f658c152c50 2024-03-04T08:14:12.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f6589e00640 1 --2- 172.21.15.113:0/4002846139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c152c50 0x7f658c06b060 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f657c0097a0 tx=0x7f657c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ec1a6a4fc47e87 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:12.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f6588a00640 1 -- 172.21.15.113:0/4002846139 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c0b5070 con 0x7f658c152c50 2024-03-04T08:14:12.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f6588a00640 1 -- 172.21.15.113:0/4002846139 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657c0a8970 con 0x7f658c152c50 2024-03-04T08:14:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f6588a00640 1 -- 172.21.15.113:0/4002846139 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c0a8d40 con 0x7f658c152c50 2024-03-04T08:14:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f658b200640 1 -- 172.21.15.113:0/4002846139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c152c50 msgr2=0x7f658c06b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f658b200640 1 --2- 172.21.15.113:0/4002846139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c152c50 0x7f658c06b060 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f657c0097a0 tx=0x7f657c01ec70 comp rx=0 tx=0).stop 2024-03-04T08:14:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f658b200640 1 -- 172.21.15.113:0/4002846139 shutdown_connections 2024-03-04T08:14:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f658b200640 1 --2- 172.21.15.113:0/4002846139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c152c50 0x7f658c06b060 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f658b200640 1 -- 172.21.15.113:0/4002846139 >> 172.21.15.113:0/4002846139 conn(0x7f658c0c9a30 msgr2=0x7f658c0c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f658b200640 1 -- 172.21.15.113:0/4002846139 shutdown_connections 2024-03-04T08:14:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.468+0000 7f658b200640 1 -- 172.21.15.113:0/4002846139 wait complete. 2024-03-04T08:14:12.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658b200640 1 Processor -- start 2024-03-04T08:14:12.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658b200640 1 -- start start 2024-03-04T08:14:12.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f658c071a30 0x7f658c0c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:12.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c0c7930 0x7f658c0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:12.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658c152c50 0x7f658c153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:12.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c06b710 con 0x7f658c0c7930 2024-03-04T08:14:12.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c15c8e0 con 0x7f658c152c50 2024-03-04T08:14:12.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c0c9400 con 0x7f658c071a30 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6589400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c0c7930 0x7f658c0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6589400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c0c7930 0x7f658c0cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60586/0 (socket says 172.21.15.113:60586) 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6589400640 1 -- 172.21.15.113:0/621532874 learned_addr learned my addr 172.21.15.113:0/621532874 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6589e00640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f658c071a30 0x7f658c0c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658a800640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658c152c50 0x7f658c153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6589400640 1 -- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f658c071a30 msgr2=0x7f658c0c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6589400640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f658c071a30 0x7f658c0c36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6589400640 1 -- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658c152c50 msgr2=0x7f658c153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6589400640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658c152c50 0x7f658c153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6589400640 1 -- 172.21.15.113:0/621532874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c0c3bf0 con 0x7f658c0c7930 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6589400640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c0c7930 0x7f658c0cb8a0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f657800c530 tx=0x7f657800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6582000640 1 -- 172.21.15.113:0/621532874 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578017070 con 0x7f658c0c7930 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6582000640 1 -- 172.21.15.113:0/621532874 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6578005cf0 con 0x7f658c0c7930 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f6582000640 1 -- 172.21.15.113:0/621532874 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578007270 con 0x7f658c0c7930 2024-03-04T08:14:12.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658b200640 1 -- 172.21.15.113:0/621532874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c1f0960 con 0x7f658c0c7930 2024-03-04T08:14:12.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.472+0000 7f658b200640 1 -- 172.21.15.113:0/621532874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658c1f0f20 con 0x7f658c0c7930 2024-03-04T08:14:12.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.476+0000 7f658b200640 1 -- 172.21.15.113:0/621532874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658c075a80 con 0x7f658c0c7930 2024-03-04T08:14:12.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.476+0000 7f6582000640 1 -- 172.21.15.113:0/621532874 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f657801a070 con 0x7f658c0c7930 2024-03-04T08:14:12.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.476+0000 7f6582000640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f656c066f90 0x7f656c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:12.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.476+0000 7f6582000640 1 -- 172.21.15.113:0/621532874 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f6578084b00 con 0x7f658c0c7930 2024-03-04T08:14:12.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.476+0000 7f6589e00640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f656c066f90 0x7f656c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:12.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.476+0000 7f6589e00640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f656c066f90 0x7f656c069450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f657c0b2000 tx=0x7f657c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:12.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.480+0000 7f6582000640 1 -- 172.21.15.113:0/621532874 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6578054c40 con 0x7f658c0c7930 2024-03-04T08:14:12.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.712+0000 7f658b200640 1 -- 172.21.15.113:0/621532874 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f658c0c4880 con 0x7f658c0c7930 2024-03-04T08:14:12.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f6582000640 1 -- 172.21.15.113:0/621532874 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) v1 ==== 95+0+5826 (secure 0 0 0) 0x7f6578058ce0 con 0x7f658c0c7930 2024-03-04T08:14:12.711 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:14:12.711 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":149,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:11.665238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7519,7692,7839]},"id":1}]} 2024-03-04T08:14:12.712 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 149 2024-03-04T08:14:12.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 -- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f656c066f90 msgr2=0x7f656c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:12.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f656c066f90 0x7f656c069450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f657c0b2000 tx=0x7f657c008040 comp rx=0 tx=0).stop 2024-03-04T08:14:12.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 -- 172.21.15.113:0/621532874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c0c7930 msgr2=0x7f658c0cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:12.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c0c7930 0x7f658c0cb8a0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f657800c530 tx=0x7f657800c9f0 comp rx=0 tx=0).stop 2024-03-04T08:14:12.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 -- 172.21.15.113:0/621532874 shutdown_connections 2024-03-04T08:14:12.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f658c152c50 0x7f658c153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:12.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f656c066f90 0x7f656c069450 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:12.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f658c0c7930 0x7f658c0cb8a0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:12.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 --2- 172.21.15.113:0/621532874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f658c071a30 0x7f658c0c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:12.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 -- 172.21.15.113:0/621532874 >> 172.21.15.113:0/621532874 conn(0x7f658c0c9a30 msgr2=0x7f658c071e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:12.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 -- 172.21.15.113:0/621532874 shutdown_connections 2024-03-04T08:14:12.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:12.716+0000 7f658b200640 1 -- 172.21.15.113:0/621532874 wait complete. 2024-03-04T08:14:12.733 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=0, replacing gid=7389 2024-03-04T08:14:12.733 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 9 secs before reviving mds.a 2024-03-04T08:14:19.317 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T08:14:19.320+0000 7f9b59600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T08:14:19.317 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T08:14:19.320+0000 7f9b59600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T08:14:19.317 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T08:14:19.320+0000 7f9b59600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-04T08:14:21.734 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-03-04T08:14:21.734 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-04T08:14:21.735 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-04T08:14:21.739 INFO:tasks.ceph.mds.a:Started 2024-03-04T08:14:21.740 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:14:21.898 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-03-04T08:14:21.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.952+0000 7f4293e00640 1 Processor -- start 2024-03-04T08:14:21.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.952+0000 7f4293e00640 1 -- start start 2024-03-04T08:14:21.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4293e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4294152950 0x7f429415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:21.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4293e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42940c4ce0 con 0x7f42940cb540 2024-03-04T08:14:21.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4293e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42940c4f80 con 0x7f429406faa0 2024-03-04T08:14:21.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4293e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42940c8370 con 0x7f4294152950 2024-03-04T08:14:21.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4292a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4294152950 0x7f429415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:21.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4292000640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f429406faa0 0x7f429406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:48360/0 (socket says 172.21.15.113:48360) 2024-03-04T08:14:21.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4292000640 1 -- 172.21.15.113:0/581763616 learned_addr learned my addr 172.21.15.113:0/581763616 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:14:21.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4291600640 1 -- 172.21.15.113:0/581763616 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614936707 0 0) 0x7f42940c4f80 con 0x7f429406faa0 2024-03-04T08:14:21.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4291600640 1 -- 172.21.15.113:0/581763616 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4278003530 con 0x7f429406faa0 2024-03-04T08:14:21.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4291600640 1 -- 172.21.15.113:0/581763616 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112085778 0 0) 0x7f42940c4ce0 con 0x7f42940cb540 2024-03-04T08:14:21.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4292a00640 1 -- 172.21.15.113:0/581763616 >> v1:172.21.15.136:6789/0 conn(0x7f429406faa0 legacy=0x7f429406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:21.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4292a00640 1 -- 172.21.15.113:0/581763616 >> v1:172.21.15.113:6789/0 conn(0x7f42940cb540 legacy=0x7f4294162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:21.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.956+0000 7f4292a00640 1 -- 172.21.15.113:0/581763616 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42940c5220 con 0x7f4294152950 2024-03-04T08:14:21.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4292a00640 1 --2- 172.21.15.113:0/581763616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4294152950 0x7f429415bb40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f42840097c0 tx=0x7f428401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bc7be909e7d56c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:21.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4291600640 1 -- 172.21.15.113:0/581763616 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42840b5070 con 0x7f4294152950 2024-03-04T08:14:21.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4291600640 1 -- 172.21.15.113:0/581763616 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42840a8990 con 0x7f4294152950 2024-03-04T08:14:21.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4291600640 1 -- 172.21.15.113:0/581763616 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42840a8d60 con 0x7f4294152950 2024-03-04T08:14:21.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4293e00640 1 -- 172.21.15.113:0/581763616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4294152950 msgr2=0x7f429415bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:21.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4293e00640 1 --2- 172.21.15.113:0/581763616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4294152950 0x7f429415bb40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f42840097c0 tx=0x7f428401ec90 comp rx=0 tx=0).stop 2024-03-04T08:14:21.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4293e00640 1 -- 172.21.15.113:0/581763616 shutdown_connections 2024-03-04T08:14:21.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4293e00640 1 --2- 172.21.15.113:0/581763616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4294152950 0x7f429415bb40 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:21.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4293e00640 1 -- 172.21.15.113:0/581763616 >> 172.21.15.113:0/581763616 conn(0x7f4294072f40 msgr2=0x7f42940706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:21.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4293e00640 1 -- 172.21.15.113:0/581763616 shutdown_connections 2024-03-04T08:14:21.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4293e00640 1 -- 172.21.15.113:0/581763616 wait complete. 2024-03-04T08:14:21.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.960+0000 7f4293e00640 1 Processor -- start 2024-03-04T08:14:21.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4293e00640 1 -- start start 2024-03-04T08:14:21.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4293e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f429406faa0 0x7f42941edef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:21.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4293e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42940cb540 0x7f42941515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:21.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4293e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4294152950 0x7f429414b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:21.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4293e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42940c9a50 con 0x7f42940cb540 2024-03-04T08:14:21.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4293e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42940ca0a0 con 0x7f4294152950 2024-03-04T08:14:21.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4293e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42940c9d30 con 0x7f429406faa0 2024-03-04T08:14:21.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4292000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42940cb540 0x7f42941515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:21.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4292000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42940cb540 0x7f42941515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54326/0 (socket says 172.21.15.113:54326) 2024-03-04T08:14:21.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4292000640 1 -- 172.21.15.113:0/1998347528 learned_addr learned my addr 172.21.15.113:0/1998347528 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:14:21.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4292a00640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f429406faa0 0x7f42941edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:21.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4293400640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4294152950 0x7f429414b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:21.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4292000640 1 -- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f429406faa0 msgr2=0x7f42941edef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:21.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4292000640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f429406faa0 0x7f42941edef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:21.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4292000640 1 -- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4294152950 msgr2=0x7f429414b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:21.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4292000640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4294152950 0x7f429414b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:21.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4292000640 1 -- 172.21.15.113:0/1998347528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429415b070 con 0x7f42940cb540 2024-03-04T08:14:21.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4292000640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42940cb540 0x7f42941515f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f428000c460 tx=0x7f428000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:21.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f428b400640 1 -- 172.21.15.113:0/1998347528 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280017070 con 0x7f42940cb540 2024-03-04T08:14:21.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f428b400640 1 -- 172.21.15.113:0/1998347528 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4280005d10 con 0x7f42940cb540 2024-03-04T08:14:21.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f428b400640 1 -- 172.21.15.113:0/1998347528 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42800072b0 con 0x7f42940cb540 2024-03-04T08:14:21.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4293e00640 1 -- 172.21.15.113:0/1998347528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429415b350 con 0x7f42940cb540 2024-03-04T08:14:21.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.964+0000 7f4293e00640 1 -- 172.21.15.113:0/1998347528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429415b910 con 0x7f42940cb540 2024-03-04T08:14:21.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.968+0000 7f4293e00640 1 -- 172.21.15.113:0/1998347528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4294075a80 con 0x7f42940cb540 2024-03-04T08:14:21.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.968+0000 7f428b400640 1 -- 172.21.15.113:0/1998347528 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f428001a070 con 0x7f42940cb540 2024-03-04T08:14:21.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.968+0000 7f428b400640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4274066f90 0x7f4274069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:14:21.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.968+0000 7f428b400640 1 -- 172.21.15.113:0/1998347528 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f4280084bc0 con 0x7f42940cb540 2024-03-04T08:14:21.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.968+0000 7f4292a00640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4274066f90 0x7f4274069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:14:21.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.972+0000 7f4292a00640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4274066f90 0x7f4274069450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f42840b2000 tx=0x7f4284008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:14:21.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:21.972+0000 7f428b400640 1 -- 172.21.15.113:0/1998347528 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4280054c70 con 0x7f42940cb540 2024-03-04T08:14:22.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.180+0000 7f4293e00640 1 -- 172.21.15.113:0/1998347528 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42940c9580 con 0x7f42940cb540 2024-03-04T08:14:22.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.180+0000 7f428b400640 1 -- 172.21.15.113:0/1998347528 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f4280058d10 con 0x7f42940cb540 2024-03-04T08:14:22.178 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:14:22.178 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7004,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:11.665238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7519]},"id":1}]} 2024-03-04T08:14:22.178 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 150 2024-03-04T08:14:22.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 -- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4274066f90 msgr2=0x7f4274069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4274066f90 0x7f4274069450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f42840b2000 tx=0x7f4284008040 comp rx=0 tx=0).stop 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 -- 172.21.15.113:0/1998347528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42940cb540 msgr2=0x7f42941515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42940cb540 0x7f42941515f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f428000c460 tx=0x7f428000c920 comp rx=0 tx=0).stop 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 -- 172.21.15.113:0/1998347528 shutdown_connections 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4294152950 0x7f429414b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4274066f90 0x7f4274069450 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f42940cb540 0x7f42941515f0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 --2- 172.21.15.113:0/1998347528 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f429406faa0 0x7f42941edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 -- 172.21.15.113:0/1998347528 >> 172.21.15.113:0/1998347528 conn(0x7f4294072f40 msgr2=0x7f42940705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 -- 172.21.15.113:0/1998347528 shutdown_connections 2024-03-04T08:14:22.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:14:22.184+0000 7f4293e00640 1 -- 172.21.15.113:0/1998347528 wait complete. 2024-03-04T08:14:22.198 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-03-04T08:14:22.199 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 83 secs before thrashing 2024-03-04T08:14:24.396 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:14:24.396 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 25 -eq 48 ] 2024-03-04T08:14:24.396 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.25 2024-03-04T08:14:24.397 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.25 2024-03-04T08:14:24.397 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.25 2024-03-04T08:14:24.398 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.25 2024-03-04T08:14:24.398 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:14:28.465 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:14:28.465 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 26 -eq 48 ] 2024-03-04T08:14:28.465 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.26 2024-03-04T08:14:28.466 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.26 2024-03-04T08:14:28.466 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.26 2024-03-04T08:14:28.466 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.26 2024-03-04T08:14:28.466 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:14:32.254 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:14:32.254 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.27 2024-03-04T08:14:32.255 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 27 -eq 48 ] 2024-03-04T08:14:32.255 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.27 2024-03-04T08:14:32.255 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.27 2024-03-04T08:14:32.255 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.27 2024-03-04T08:14:32.255 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:14:35.364 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:14:35.365 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.28 2024-03-04T08:14:35.365 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 28 -eq 48 ] 2024-03-04T08:14:35.365 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.28 2024-03-04T08:14:35.365 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.28 2024-03-04T08:14:35.365 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.28 2024-03-04T08:14:35.365 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:14:37.589 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:14:37.593 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:14:37.789 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:14:37.789 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.29 2024-03-04T08:14:37.790 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 29 -eq 48 ] 2024-03-04T08:14:37.790 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.29 2024-03-04T08:14:37.791 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.29 2024-03-04T08:14:37.791 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.29 2024-03-04T08:14:37.791 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:14:42.154 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:14:42.155 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.30 2024-03-04T08:14:42.156 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 30 -eq 48 ] 2024-03-04T08:14:42.156 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.30 2024-03-04T08:14:42.156 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.30 2024-03-04T08:14:42.156 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.30 2024-03-04T08:14:42.156 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:14:44.416 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:14:44.417 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.31 2024-03-04T08:14:44.417 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 31 -eq 48 ] 2024-03-04T08:14:44.417 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.31 2024-03-04T08:14:44.417 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.31 2024-03-04T08:14:44.417 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.31 2024-03-04T08:14:44.417 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:14:52.007 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:14:52.008 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.32 2024-03-04T08:14:52.008 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 32 -eq 48 ] 2024-03-04T08:14:52.008 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.32 2024-03-04T08:14:52.008 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.32 2024-03-04T08:14:52.008 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.32 2024-03-04T08:14:52.008 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:14:55.143 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:14:55.143 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 33 -eq 48 ] 2024-03-04T08:14:55.144 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.33 2024-03-04T08:14:55.144 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.33 2024-03-04T08:14:55.144 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.33 2024-03-04T08:14:55.145 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.33 2024-03-04T08:14:55.145 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:01.769 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:15:01.769 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.34 2024-03-04T08:15:01.769 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 34 -eq 48 ] 2024-03-04T08:15:01.770 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.34 2024-03-04T08:15:01.770 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.34 2024-03-04T08:15:01.770 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.34 2024-03-04T08:15:01.770 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:04.688 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:15:04.688 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 35 -eq 48 ] 2024-03-04T08:15:04.689 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.35 2024-03-04T08:15:04.689 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.35 2024-03-04T08:15:04.689 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.35 2024-03-04T08:15:04.689 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.35 2024-03-04T08:15:04.689 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:07.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:15:07.721 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:15:13.022 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:15:13.023 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.36 2024-03-04T08:15:13.023 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 36 -eq 48 ] 2024-03-04T08:15:13.023 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.36 2024-03-04T08:15:13.023 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.36 2024-03-04T08:15:13.023 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.36 2024-03-04T08:15:13.023 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:17.845 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:15:17.846 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 37 -eq 48 ] 2024-03-04T08:15:17.846 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.37 2024-03-04T08:15:17.846 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.37 2024-03-04T08:15:17.846 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.37 2024-03-04T08:15:17.847 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.37 2024-03-04T08:15:17.847 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:21.473 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:15:21.474 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.38 2024-03-04T08:15:21.474 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 38 -eq 48 ] 2024-03-04T08:15:21.474 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.38 2024-03-04T08:15:21.474 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.38 2024-03-04T08:15:21.474 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.38 2024-03-04T08:15:21.474 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:30.244 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:15:30.245 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.39 2024-03-04T08:15:30.245 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 39 -eq 48 ] 2024-03-04T08:15:30.245 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.39 2024-03-04T08:15:30.246 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.39 2024-03-04T08:15:30.246 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.39 2024-03-04T08:15:30.246 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:37.009 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:15:37.009 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 40 -eq 48 ] 2024-03-04T08:15:37.010 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.40 2024-03-04T08:15:37.010 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.40 2024-03-04T08:15:37.010 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.40 2024-03-04T08:15:37.010 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.40 2024-03-04T08:15:37.010 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:37.772 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:15:37.776 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:15:41.998 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:15:41.998 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.41 2024-03-04T08:15:41.999 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 41 -eq 48 ] 2024-03-04T08:15:41.999 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.41 2024-03-04T08:15:41.999 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.41 2024-03-04T08:15:41.999 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.41 2024-03-04T08:15:41.999 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:45.200 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:15:45.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.350+0000 7f2e5ca00640 1 Processor -- start 2024-03-04T08:15:45.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.350+0000 7f2e5ca00640 1 -- start start 2024-03-04T08:15:45.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e5ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e58157f80 0x7f2e5806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e5ca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5806df00 con 0x7f2e580c8a90 2024-03-04T08:15:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e5ca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e580ce1a0 con 0x7f2e580c9ce0 2024-03-04T08:15:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e5ca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5806e1e0 con 0x7f2e58157f80 2024-03-04T08:15:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e57400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e58157f80 0x7f2e5806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e57400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e58157f80 0x7f2e5806c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35490/0 (socket says 172.21.15.113:35490) 2024-03-04T08:15:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e57400640 1 -- 172.21.15.113:0/1753897394 learned_addr learned my addr 172.21.15.113:0/1753897394 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e4fe00640 1 --1- 172.21.15.113:0/1753897394 >> v1:172.21.15.113:6789/0 conn(0x7f2e580c8a90 0x7f2e5806ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50300/0 (socket says 172.21.15.113:50300) 2024-03-04T08:15:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e57400640 1 -- 172.21.15.113:0/1753897394 >> v1:172.21.15.136:6789/0 conn(0x7f2e580c9ce0 legacy=0x7f2e580ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e57400640 1 -- 172.21.15.113:0/1753897394 >> v1:172.21.15.113:6789/0 conn(0x7f2e580c8a90 legacy=0x7f2e5806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.354+0000 7f2e57400640 1 -- 172.21.15.113:0/1753897394 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e580ce480 con 0x7f2e58157f80 2024-03-04T08:15:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e57400640 1 --2- 172.21.15.113:0/1753897394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e58157f80 0x7f2e5806c170 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009870 tx=0x7f2e4801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8497ed2b1249e2fb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e56a00640 1 -- 172.21.15.113:0/1753897394 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e480b2070 con 0x7f2e58157f80 2024-03-04T08:15:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e56a00640 1 -- 172.21.15.113:0/1753897394 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e480a9070 con 0x7f2e58157f80 2024-03-04T08:15:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e56a00640 1 -- 172.21.15.113:0/1753897394 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e480ad880 con 0x7f2e58157f80 2024-03-04T08:15:45.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 -- 172.21.15.113:0/1753897394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e58157f80 msgr2=0x7f2e5806c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 --2- 172.21.15.113:0/1753897394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e58157f80 0x7f2e5806c170 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009870 tx=0x7f2e4801c560 comp rx=0 tx=0).stop 2024-03-04T08:15:45.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 -- 172.21.15.113:0/1753897394 shutdown_connections 2024-03-04T08:15:45.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 --2- 172.21.15.113:0/1753897394 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e58157f80 0x7f2e5806c170 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 -- 172.21.15.113:0/1753897394 >> 172.21.15.113:0/1753897394 conn(0x7f2e58072f40 msgr2=0x7f2e580706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:45.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 -- 172.21.15.113:0/1753897394 shutdown_connections 2024-03-04T08:15:45.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 -- 172.21.15.113:0/1753897394 wait complete. 2024-03-04T08:15:45.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 Processor -- start 2024-03-04T08:15:45.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 -- start start 2024-03-04T08:15:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e580c8a90 0x7f2e581f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e580c9ce0 0x7f2e581f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e58157f80 0x7f2e5814c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5806fdb0 con 0x7f2e580c8a90 2024-03-04T08:15:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5806fc30 con 0x7f2e58157f80 2024-03-04T08:15:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.358+0000 7f2e5ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5806f490 con 0x7f2e580c9ce0 2024-03-04T08:15:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e57400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e580c8a90 0x7f2e581f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e57400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e580c8a90 0x7f2e581f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35500/0 (socket says 172.21.15.113:35500) 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e57400640 1 -- 172.21.15.113:0/2579475500 learned_addr learned my addr 172.21.15.113:0/2579475500 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e4fe00640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e580c9ce0 0x7f2e581f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e57e00640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e58157f80 0x7f2e5814c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e57400640 1 -- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e580c9ce0 msgr2=0x7f2e581f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e57400640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e580c9ce0 0x7f2e581f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e57400640 1 -- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e58157f80 msgr2=0x7f2e5814c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e57400640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e58157f80 0x7f2e5814c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e57400640 1 -- 172.21.15.113:0/2579475500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e581f7810 con 0x7f2e580c8a90 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e57400640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e580c8a90 0x7f2e581f6cf0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f2e480b0040 tx=0x7f2e48019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e4f400640 1 -- 172.21.15.113:0/2579475500 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e480b2040 con 0x7f2e580c8a90 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e4f400640 1 -- 172.21.15.113:0/2579475500 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e480bb070 con 0x7f2e580c8a90 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e4f400640 1 -- 172.21.15.113:0/2579475500 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e480b68b0 con 0x7f2e580c8a90 2024-03-04T08:15:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e5ca00640 1 -- 172.21.15.113:0/2579475500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5814e230 con 0x7f2e580c8a90 2024-03-04T08:15:45.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e5ca00640 1 -- 172.21.15.113:0/2579475500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5814df50 con 0x7f2e580c8a90 2024-03-04T08:15:45.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e5ca00640 1 -- 172.21.15.113:0/2579475500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e58075a80 con 0x7f2e580c8a90 2024-03-04T08:15:45.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.362+0000 7f2e4f400640 1 -- 172.21.15.113:0/2579475500 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e480ab070 con 0x7f2e580c8a90 2024-03-04T08:15:45.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.366+0000 7f2e4f400640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e34066f60 0x7f2e34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.366+0000 7f2e4f400640 1 -- 172.21.15.113:0/2579475500 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f2e4811fae0 con 0x7f2e580c8a90 2024-03-04T08:15:45.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.366+0000 7f2e4fe00640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e34066f60 0x7f2e34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.366+0000 7f2e4fe00640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e34066f60 0x7f2e34069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2e58150c20 tx=0x7f2e40007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:45.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.370+0000 7f2e4f400640 1 -- 172.21.15.113:0/2579475500 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2e480c4020 con 0x7f2e580c8a90 2024-03-04T08:15:45.396 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:15:45.396 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.42 2024-03-04T08:15:45.397 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 42 -eq 48 ] 2024-03-04T08:15:45.397 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.42 2024-03-04T08:15:45.397 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.42 2024-03-04T08:15:45.397 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.42 2024-03-04T08:15:45.397 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.678+0000 7f2e5ca00640 1 -- 172.21.15.113:0/2579475500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e5806f880 con 0x7f2e580c8a90 2024-03-04T08:15:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.678+0000 7f2e4f400640 1 -- 172.21.15.113:0/2579475500 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f2e48016150 con 0x7f2e580c8a90 2024-03-04T08:15:45.678 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:15:45.678 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7004,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:11.665238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7519]},"id":1}]} 2024-03-04T08:15:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 150 2024-03-04T08:15:45.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 -- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e34066f60 msgr2=0x7f2e34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e34066f60 0x7f2e34069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2e58150c20 tx=0x7f2e40007cb0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 -- 172.21.15.113:0/2579475500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e580c8a90 msgr2=0x7f2e581f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e580c8a90 0x7f2e581f6cf0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f2e480b0040 tx=0x7f2e48019300 comp rx=0 tx=0).stop 2024-03-04T08:15:45.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 -- 172.21.15.113:0/2579475500 shutdown_connections 2024-03-04T08:15:45.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e58157f80 0x7f2e5814c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e580c9ce0 0x7f2e581f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e34066f60 0x7f2e34069420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 --2- 172.21.15.113:0/2579475500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e580c8a90 0x7f2e581f6cf0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 -- 172.21.15.113:0/2579475500 >> 172.21.15.113:0/2579475500 conn(0x7f2e58072f40 msgr2=0x7f2e58151170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:45.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 -- 172.21.15.113:0/2579475500 shutdown_connections 2024-03-04T08:15:45.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.682+0000 7f2e5ca00640 1 -- 172.21.15.113:0/2579475500 wait complete. 2024-03-04T08:15:45.701 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=1) 2024-03-04T08:15:45.701 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-04T08:15:45.701 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:15:45.716 INFO:tasks.ceph.mds.b:Stopped 2024-03-04T08:15:45.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:15:45.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f43800640 1 Processor -- start 2024-03-04T08:15:45.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f43800640 1 -- start start 2024-03-04T08:15:45.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f43800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f3c14e4c0 0x7f6f3c072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f43800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3c0c27f0 con 0x7f6f3c0c22e0 2024-03-04T08:15:45.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f43800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3c0c2a90 con 0x7f6f3c15b900 2024-03-04T08:15:45.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f43800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c15ad60 con 0x7f6f3c14e4c0 2024-03-04T08:15:45.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6f3c0c22e0 0x7f6f3c15b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50302/0 (socket says 172.21.15.113:50302) 2024-03-04T08:15:45.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3be00640 1 -- 172.21.15.113:0/2112766097 learned_addr learned my addr 172.21.15.113:0/2112766097 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:15:45.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f40e00640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f3c14e4c0 0x7f6f3c072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510125006 0 0) 0x7f6f3c0c27f0 con 0x7f6f3c0c22e0 2024-03-04T08:15:45.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f24003530 con 0x7f6f3c0c22e0 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 723001502 0 0) 0x7f6f24003530 con 0x7f6f3c0c22e0 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f3c14e4c0 msgr2=0x7f6f3c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f3c14e4c0 0x7f6f3c072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 >> v1:172.21.15.136:6789/0 conn(0x7f6f3c15b900 legacy=0x7f6f3c1620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c070c60 con 0x7f6f3c0c22e0 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f6f28001680 con 0x7f6f3c0c22e0 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 >> v1:172.21.15.113:6789/0 conn(0x7f6f3c0c22e0 legacy=0x7f6f3c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f3c14e4c0 0x7f6f240039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24003f30 0x7f6f24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f24004d50 0x7f6f240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2400ba00 con 0x7f6f24004d50 2024-03-04T08:15:45.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2400bbb0 con 0x7f6f24003f30 2024-03-04T08:15:45.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.858+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2400bdb0 con 0x7f6f3c14e4c0 2024-03-04T08:15:45.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f41800640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f24004d50 0x7f6f240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f40e00640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f3c14e4c0 0x7f6f240039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f3be00640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24003f30 0x7f6f24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f41800640 1 -- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f3c14e4c0 msgr2=0x7f6f240039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:15:45.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f41800640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f3c14e4c0 0x7f6f240039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f41800640 1 -- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24003f30 msgr2=0x7f6f24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:15:45.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f41800640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f24003f30 0x7f6f24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f41800640 1 -- 172.21.15.113:0/2112766097 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f24003530 con 0x7f6f24004d50 2024-03-04T08:15:45.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f41800640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f24004d50 0x7f6f240051d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f6f34002010 tx=0x7f6f3401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1488e0a9fb75fe server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:45.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f340b2070 con 0x7f6f24004d50 2024-03-04T08:15:45.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f340a57f0 con 0x7f6f24004d50 2024-03-04T08:15:45.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.862+0000 7f6f3b400640 1 -- 172.21.15.113:0/2112766097 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34001c10 con 0x7f6f24004d50 2024-03-04T08:15:45.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.866+0000 7f6f43800640 1 -- 172.21.15.113:0/2112766097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f24004d50 msgr2=0x7f6f240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.866+0000 7f6f43800640 1 --2- 172.21.15.113:0/2112766097 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f24004d50 0x7f6f240051d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f6f34002010 tx=0x7f6f3401c8e0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.866+0000 7f6f41800640 1 -- 172.21.15.113:0/2112766097 reap_dead start 2024-03-04T08:15:45.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.866+0000 7f6f43800640 1 -- 172.21.15.113:0/2112766097 shutdown_connections 2024-03-04T08:15:45.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.866+0000 7f6f43800640 1 -- 172.21.15.113:0/2112766097 >> 172.21.15.113:0/2112766097 conn(0x7f6f3c06e820 msgr2=0x7f6f3c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:45.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.866+0000 7f6f43800640 1 -- 172.21.15.113:0/2112766097 shutdown_connections 2024-03-04T08:15:45.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.866+0000 7f6f43800640 1 -- 172.21.15.113:0/2112766097 wait complete. 2024-03-04T08:15:45.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.866+0000 7f6f43800640 1 Processor -- start 2024-03-04T08:15:45.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.866+0000 7f6f43800640 1 -- start start 2024-03-04T08:15:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f43800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f24004d50 0x7f6f3c1f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f43800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f3c158930 0x7f6f3c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f43800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3c14b200 0x7f6f3c13fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f43800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c071500 con 0x7f6f3c14b200 2024-03-04T08:15:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f43800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c071220 con 0x7f6f3c158930 2024-03-04T08:15:45.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f43800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c14e8a0 con 0x7f6f24004d50 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f41800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3c14b200 0x7f6f3c13fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f41800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3c14b200 0x7f6f3c13fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35524/0 (socket says 172.21.15.113:35524) 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f41800640 1 -- 172.21.15.113:0/3876793359 learned_addr learned my addr 172.21.15.113:0/3876793359 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f40e00640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f24004d50 0x7f6f3c1f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f3be00640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f3c158930 0x7f6f3c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f41800640 1 -- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f24004d50 msgr2=0x7f6f3c1f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f41800640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f24004d50 0x7f6f3c1f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f41800640 1 -- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f3c158930 msgr2=0x7f6f3c15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f41800640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f3c158930 0x7f6f3c15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f41800640 1 -- 172.21.15.113:0/3876793359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c159330 con 0x7f6f3c14b200 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f3be00640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f3c158930 0x7f6f3c15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:15:45.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f41800640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3c14b200 0x7f6f3c13fd20 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f6f34002010 tx=0x7f6f24005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f38c00640 1 -- 172.21.15.113:0/3876793359 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f340b2040 con 0x7f6f3c14b200 2024-03-04T08:15:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f38c00640 1 -- 172.21.15.113:0/3876793359 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f34001bf0 con 0x7f6f3c14b200 2024-03-04T08:15:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f38c00640 1 -- 172.21.15.113:0/3876793359 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34003bc0 con 0x7f6f3c14b200 2024-03-04T08:15:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f43800640 1 -- 172.21.15.113:0/3876793359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3c159610 con 0x7f6f3c14b200 2024-03-04T08:15:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f43800640 1 -- 172.21.15.113:0/3876793359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3c159bd0 con 0x7f6f3c14b200 2024-03-04T08:15:45.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.870+0000 7f6f43800640 1 -- 172.21.15.113:0/3876793359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3c075c00 con 0x7f6f3c14b200 2024-03-04T08:15:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.874+0000 7f6f38c00640 1 -- 172.21.15.113:0/3876793359 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f340be020 con 0x7f6f3c14b200 2024-03-04T08:15:45.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.874+0000 7f6f38c00640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6f1c066f90 0x7f6f1c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:45.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.874+0000 7f6f38c00640 1 -- 172.21.15.113:0/3876793359 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f6f3411eee0 con 0x7f6f3c14b200 2024-03-04T08:15:45.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.874+0000 7f6f40e00640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6f1c066f90 0x7f6f1c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:45.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.874+0000 7f6f40e00640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6f1c066f90 0x7f6f1c069450 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009be0 tx=0x7f6f30009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:45.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:45.882+0000 7f6f38c00640 1 -- 172.21.15.113:0/3876793359 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6f340c5020 con 0x7f6f3c14b200 2024-03-04T08:15:46.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 -- 172.21.15.113:0/3876793359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f3c155190 con 0x7f6f3c14b200 2024-03-04T08:15:46.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f38c00640 1 -- 172.21.15.113:0/3876793359 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f6f340165f0 con 0x7f6f3c14b200 2024-03-04T08:15:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:15:46.153 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7004,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:11.665238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7519]},"id":1}]} 2024-03-04T08:15:46.153 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 150 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 -- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6f1c066f90 msgr2=0x7f6f1c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6f1c066f90 0x7f6f1c069450 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009be0 tx=0x7f6f30009190 comp rx=0 tx=0).stop 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 -- 172.21.15.113:0/3876793359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3c14b200 msgr2=0x7f6f3c13fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3c14b200 0x7f6f3c13fd20 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f6f34002010 tx=0x7f6f24005840 comp rx=0 tx=0).stop 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 -- 172.21.15.113:0/3876793359 shutdown_connections 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6f1c066f90 0x7f6f1c069450 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f3c14b200 0x7f6f3c13fd20 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6f3c158930 0x7f6f3c15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 --2- 172.21.15.113:0/3876793359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6f24004d50 0x7f6f3c1f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.154+0000 7f6f43800640 1 -- 172.21.15.113:0/3876793359 >> 172.21.15.113:0/3876793359 conn(0x7f6f3c06e820 msgr2=0x7f6f3c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.158+0000 7f6f43800640 1 -- 172.21.15.113:0/3876793359 shutdown_connections 2024-03-04T08:15:46.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:46.158+0000 7f6f43800640 1 -- 172.21.15.113:0/3876793359 wait complete. 2024-03-04T08:15:46.173 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:15:48.175 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:15:48.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.314+0000 7ffa9ae00640 1 Processor -- start 2024-03-04T08:15:48.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa9ae00640 1 -- start start 2024-03-04T08:15:48.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa9ae00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 0x7ffa94072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:48.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa9ae00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffa9415b900 0x7ffa940731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:48.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa9ae00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa940c27f0 con 0x7ffa940c22e0 2024-03-04T08:15:48.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa9ae00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9415ad60 con 0x7ffa9415b900 2024-03-04T08:15:48.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa9ae00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa940c2a90 con 0x7ffa9414e4c0 2024-03-04T08:15:48.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa98e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffa9415b900 0x7ffa940731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:48.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 0x7ffa94072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:48.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 0x7ffa94072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:37882/0 (socket says 172.21.15.113:37882) 2024-03-04T08:15:48.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa98e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffa9415b900 0x7ffa940731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:60858/0 (socket says 172.21.15.113:60858) 2024-03-04T08:15:48.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa93e00640 1 -- 172.21.15.113:0/596356997 learned_addr learned my addr 172.21.15.113:0/596356997 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:48.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa98e00640 1 -- 172.21.15.113:0/596356997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 msgr2=0x7ffa94072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:15:48.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa98e00640 1 --2- 172.21.15.113:0/596356997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 0x7ffa94072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:48.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa98e00640 1 -- 172.21.15.113:0/596356997 >> v1:172.21.15.136:6789/0 conn(0x7ffa940c22e0 legacy=0x7ffa9415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:48.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa98e00640 1 -- 172.21.15.113:0/596356997 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa94070c60 con 0x7ffa9415b900 2024-03-04T08:15:48.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.318+0000 7ffa93e00640 1 --2- 172.21.15.113:0/596356997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 0x7ffa94072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:15:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa98e00640 1 --2- 172.21.15.113:0/596356997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa9415b900 0x7ffa940731c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00ba60 tx=0x7ffa8c021360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01cda290bfdc2df server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa93400640 1 -- 172.21.15.113:0/596356997 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0b6070 con 0x7ffa9415b900 2024-03-04T08:15:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa93400640 1 -- 172.21.15.113:0/596356997 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8c0b1600 con 0x7ffa9415b900 2024-03-04T08:15:48.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa93400640 1 -- 172.21.15.113:0/596356997 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0aae60 con 0x7ffa9415b900 2024-03-04T08:15:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa9ae00640 1 -- 172.21.15.113:0/596356997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa9415b900 msgr2=0x7ffa940731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa9ae00640 1 --2- 172.21.15.113:0/596356997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa9415b900 0x7ffa940731c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00ba60 tx=0x7ffa8c021360 comp rx=0 tx=0).stop 2024-03-04T08:15:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa9ae00640 1 -- 172.21.15.113:0/596356997 shutdown_connections 2024-03-04T08:15:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa9ae00640 1 --2- 172.21.15.113:0/596356997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa9415b900 0x7ffa940731c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa9ae00640 1 --2- 172.21.15.113:0/596356997 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 0x7ffa94072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa9ae00640 1 -- 172.21.15.113:0/596356997 >> 172.21.15.113:0/596356997 conn(0x7ffa9406e820 msgr2=0x7ffa9406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:48.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa9ae00640 1 -- 172.21.15.113:0/596356997 shutdown_connections 2024-03-04T08:15:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.322+0000 7ffa9ae00640 1 -- 172.21.15.113:0/596356997 wait complete. 2024-03-04T08:15:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 Processor -- start 2024-03-04T08:15:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 -- start start 2024-03-04T08:15:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa940c22e0 0x7ffa9415a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 0x7ffa94148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa9415b900 0x7ffa94146c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa94071500 con 0x7ffa940c22e0 2024-03-04T08:15:48.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa94071220 con 0x7ffa9415b900 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9414e8a0 con 0x7ffa9414e4c0 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa940c22e0 0x7ffa9415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa940c22e0 0x7ffa9415a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40166/0 (socket says 172.21.15.113:40166) 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa93e00640 1 -- 172.21.15.113:0/752127849 learned_addr learned my addr 172.21.15.113:0/752127849 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa98e00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa9415b900 0x7ffa94146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa8b400640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 0x7ffa94148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa93e00640 1 -- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 msgr2=0x7ffa94148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa93e00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 0x7ffa94148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa93e00640 1 -- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa9415b900 msgr2=0x7ffa94146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa93e00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa9415b900 0x7ffa94146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa93e00640 1 -- 172.21.15.113:0/752127849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa941ec510 con 0x7ffa940c22e0 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa98e00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa9415b900 0x7ffa94146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa93e00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa940c22e0 0x7ffa9415a350 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009e80 tx=0x7ffa8400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa90c00640 1 -- 172.21.15.113:0/752127849 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400ce60 con 0x7ffa940c22e0 2024-03-04T08:15:48.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa90c00640 1 -- 172.21.15.113:0/752127849 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa84019070 con 0x7ffa940c22e0 2024-03-04T08:15:48.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa90c00640 1 -- 172.21.15.113:0/752127849 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84014680 con 0x7ffa940c22e0 2024-03-04T08:15:48.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 -- 172.21.15.113:0/752127849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa941ec7f0 con 0x7ffa940c22e0 2024-03-04T08:15:48.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 -- 172.21.15.113:0/752127849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa941ecdb0 con 0x7ffa940c22e0 2024-03-04T08:15:48.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.326+0000 7ffa9ae00640 1 -- 172.21.15.113:0/752127849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa94075c00 con 0x7ffa940c22e0 2024-03-04T08:15:48.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.330+0000 7ffa90c00640 1 -- 172.21.15.113:0/752127849 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa84012050 con 0x7ffa940c22e0 2024-03-04T08:15:48.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.330+0000 7ffa90c00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 0x7ffa70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:48.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.330+0000 7ffa90c00640 1 -- 172.21.15.113:0/752127849 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7ffa84085d60 con 0x7ffa940c22e0 2024-03-04T08:15:48.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.330+0000 7ffa8b400640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 0x7ffa70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:48.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.330+0000 7ffa8b400640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 0x7ffa70069420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ffa9414a940 tx=0x7ffa7c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:48.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.334+0000 7ffa90c00640 1 -- 172.21.15.113:0/752127849 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ffa84010040 con 0x7ffa940c22e0 2024-03-04T08:15:48.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.626+0000 7ffa9ae00640 1 -- 172.21.15.113:0/752127849 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa94155190 con 0x7ffa940c22e0 2024-03-04T08:15:48.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.626+0000 7ffa90c00640 1 -- 172.21.15.113:0/752127849 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+6621 (secure 0 0 0) 0x7ffa840564f0 con 0x7ffa940c22e0 2024-03-04T08:15:48.626 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:15:48.627 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7004,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:11.665238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7519]},"id":1}]} 2024-03-04T08:15:48.627 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 150 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 -- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 msgr2=0x7ffa70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 0x7ffa70069420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ffa9414a940 tx=0x7ffa7c009040 comp rx=0 tx=0).stop 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 -- 172.21.15.113:0/752127849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa940c22e0 msgr2=0x7ffa9415a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa940c22e0 0x7ffa9415a350 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009e80 tx=0x7ffa8400c490 comp rx=0 tx=0).stop 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 -- 172.21.15.113:0/752127849 shutdown_connections 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa9415b900 0x7ffa94146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa9414e4c0 0x7ffa94148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 0x7ffa70069420 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 --2- 172.21.15.113:0/752127849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa940c22e0 0x7ffa9415a350 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 -- 172.21.15.113:0/752127849 >> 172.21.15.113:0/752127849 conn(0x7ffa9406e820 msgr2=0x7ffa940c8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 -- 172.21.15.113:0/752127849 shutdown_connections 2024-03-04T08:15:48.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:48.630+0000 7ffa9ae00640 1 -- 172.21.15.113:0/752127849 wait complete. 2024-03-04T08:15:48.649 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:15:50.649 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:15:50.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.802+0000 7f83a0e00640 1 Processor -- start 2024-03-04T08:15:50.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f83a0e00640 1 -- start start 2024-03-04T08:15:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f83a0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c0ce2d0 0x7f839c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f83a0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f839c0ce6b0 0x7f839c06f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f83a0e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839c06c210 con 0x7f839c06de80 2024-03-04T08:15:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f83a0e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839c15bbf0 con 0x7f839c0ce6b0 2024-03-04T08:15:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f83a0e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c06eb80 con 0x7f839c0ce2d0 2024-03-04T08:15:50.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f8391400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f839c0ce6b0 0x7f839c06f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:50.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f8391400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f839c0ce6b0 0x7f839c06f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40172/0 (socket says 172.21.15.113:40172) 2024-03-04T08:15:50.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f8391400640 1 -- 172.21.15.113:0/2055858626 learned_addr learned my addr 172.21.15.113:0/2055858626 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:50.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f8399e00640 1 --2- 172.21.15.113:0/2055858626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c0ce2d0 0x7f839c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:50.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f8391400640 1 -- 172.21.15.113:0/2055858626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c0ce2d0 msgr2=0x7f839c15b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:15:50.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f8391400640 1 --2- 172.21.15.113:0/2055858626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c0ce2d0 0x7f839c15b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:50.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f8391400640 1 -- 172.21.15.113:0/2055858626 >> v1:172.21.15.136:6789/0 conn(0x7f839c06de80 legacy=0x7f839c154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:50.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f8391400640 1 -- 172.21.15.113:0/2055858626 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c06c4f0 con 0x7f839c0ce6b0 2024-03-04T08:15:50.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.806+0000 7f8399e00640 1 --2- 172.21.15.113:0/2055858626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c0ce2d0 0x7f839c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:15:50.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f8391400640 1 --2- 172.21.15.113:0/2055858626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 0x7f839c06f010 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f8384009870 tx=0x7f838401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d4f35e761a2555 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:50.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f8399400640 1 -- 172.21.15.113:0/2055858626 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83840b5070 con 0x7f839c0ce6b0 2024-03-04T08:15:50.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f8399400640 1 -- 172.21.15.113:0/2055858626 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83840a8a50 con 0x7f839c0ce6b0 2024-03-04T08:15:50.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f8399400640 1 -- 172.21.15.113:0/2055858626 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83840a8e20 con 0x7f839c0ce6b0 2024-03-04T08:15:50.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f83a0e00640 1 -- 172.21.15.113:0/2055858626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 msgr2=0x7f839c06f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:50.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f83a0e00640 1 --2- 172.21.15.113:0/2055858626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 0x7f839c06f010 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f8384009870 tx=0x7f838401ed50 comp rx=0 tx=0).stop 2024-03-04T08:15:50.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f83a0e00640 1 -- 172.21.15.113:0/2055858626 shutdown_connections 2024-03-04T08:15:50.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f83a0e00640 1 --2- 172.21.15.113:0/2055858626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 0x7f839c06f010 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:50.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f83a0e00640 1 --2- 172.21.15.113:0/2055858626 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c0ce2d0 0x7f839c15b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:50.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f83a0e00640 1 -- 172.21.15.113:0/2055858626 >> 172.21.15.113:0/2055858626 conn(0x7f839c0709a0 msgr2=0x7f839c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:50.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f83a0e00640 1 -- 172.21.15.113:0/2055858626 shutdown_connections 2024-03-04T08:15:50.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f83a0e00640 1 -- 172.21.15.113:0/2055858626 wait complete. 2024-03-04T08:15:50.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f83a0e00640 1 Processor -- start 2024-03-04T08:15:50.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.810+0000 7f83a0e00640 1 -- start start 2024-03-04T08:15:50.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f83a0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c06de80 0x7f839c153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:50.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f83a0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f839c0ce2d0 0x7f839c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:50.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f83a0e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 0x7f839c1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:50.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f83a0e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c06c7d0 con 0x7f839c0ce6b0 2024-03-04T08:15:50.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f83a0e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c06e810 con 0x7f839c0ce2d0 2024-03-04T08:15:50.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f83a0e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c155220 con 0x7f839c06de80 2024-03-04T08:15:50.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f839a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 0x7f839c1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:50.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f839a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 0x7f839c1508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40182/0 (socket says 172.21.15.113:40182) 2024-03-04T08:15:50.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f839a800640 1 -- 172.21.15.113:0/374756888 learned_addr learned my addr 172.21.15.113:0/374756888 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:50.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f8399e00640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c06de80 0x7f839c153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:50.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f8391400640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f839c0ce2d0 0x7f839c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:50.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f839a800640 1 -- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c06de80 msgr2=0x7f839c153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:50.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f839a800640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c06de80 0x7f839c153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:50.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f839a800640 1 -- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f839c0ce2d0 msgr2=0x7f839c0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:50.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f839a800640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f839c0ce2d0 0x7f839c0c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:50.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f839a800640 1 -- 172.21.15.113:0/374756888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c1528f0 con 0x7f839c0ce6b0 2024-03-04T08:15:50.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f839a800640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 0x7f839c1508d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f839400c760 tx=0x7f839400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:50.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f8392a00640 1 -- 172.21.15.113:0/374756888 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394003b80 con 0x7f839c0ce6b0 2024-03-04T08:15:50.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f8392a00640 1 -- 172.21.15.113:0/374756888 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8394003d20 con 0x7f839c0ce6b0 2024-03-04T08:15:50.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f8392a00640 1 -- 172.21.15.113:0/374756888 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839400aa30 con 0x7f839c0ce6b0 2024-03-04T08:15:50.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f83a0e00640 1 -- 172.21.15.113:0/374756888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c1f1a00 con 0x7f839c0ce6b0 2024-03-04T08:15:50.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f83a0e00640 1 -- 172.21.15.113:0/374756888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c0c8590 con 0x7f839c0ce6b0 2024-03-04T08:15:50.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.814+0000 7f83a0e00640 1 -- 172.21.15.113:0/374756888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839c075c00 con 0x7f839c0ce6b0 2024-03-04T08:15:50.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.818+0000 7f8392a00640 1 -- 172.21.15.113:0/374756888 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f839400abd0 con 0x7f839c0ce6b0 2024-03-04T08:15:50.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.818+0000 7f8392a00640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8378066f60 0x7f8378069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:50.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.818+0000 7f8392a00640 1 -- 172.21.15.113:0/374756888 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f83940856f0 con 0x7f839c0ce6b0 2024-03-04T08:15:50.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.818+0000 7f8399e00640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8378066f60 0x7f8378069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:50.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.818+0000 7f8399e00640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8378066f60 0x7f8378069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f838c009950 tx=0x7f838c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:50.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:50.822+0000 7f8392a00640 1 -- 172.21.15.113:0/374756888 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f83940557a0 con 0x7f839c0ce6b0 2024-03-04T08:15:51.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 -- 172.21.15.113:0/374756888 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f839c06cb40 con 0x7f839c0ce6b0 2024-03-04T08:15:51.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f8392a00640 1 -- 172.21.15.113:0/374756888 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f8394059840 con 0x7f839c0ce6b0 2024-03-04T08:15:51.109 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:15:51.109 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7004,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:11.665238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7519]},"id":1}]} 2024-03-04T08:15:51.111 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 150 2024-03-04T08:15:51.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 -- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8378066f60 msgr2=0x7f8378069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8378066f60 0x7f8378069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f838c009950 tx=0x7f838c008040 comp rx=0 tx=0).stop 2024-03-04T08:15:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 -- 172.21.15.113:0/374756888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 msgr2=0x7f839c1508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 0x7f839c1508d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f839400c760 tx=0x7f839400cc20 comp rx=0 tx=0).stop 2024-03-04T08:15:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 -- 172.21.15.113:0/374756888 shutdown_connections 2024-03-04T08:15:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8378066f60 0x7f8378069420 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f839c0ce6b0 0x7f839c1508d0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f839c0ce2d0 0x7f839c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 --2- 172.21.15.113:0/374756888 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f839c06de80 0x7f839c153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 -- 172.21.15.113:0/374756888 >> 172.21.15.113:0/374756888 conn(0x7f839c0709a0 msgr2=0x7f839c154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 -- 172.21.15.113:0/374756888 shutdown_connections 2024-03-04T08:15:51.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:51.110+0000 7f83a0e00640 1 -- 172.21.15.113:0/374756888 wait complete. 2024-03-04T08:15:51.133 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:15:53.135 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:15:53.198 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:15:53.198 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 43 -eq 48 ] 2024-03-04T08:15:53.198 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.43 2024-03-04T08:15:53.199 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.43 2024-03-04T08:15:53.199 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.43 2024-03-04T08:15:53.199 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.43 2024-03-04T08:15:53.199 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:15:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.277+0000 7ff2e6a00640 1 Processor -- start 2024-03-04T08:15:53.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.277+0000 7ff2e6a00640 1 -- start start 2024-03-04T08:15:53.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2e00c8a90 0x7ff2e015e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:53.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff2e015ea60 0x7ff2e00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:53.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e6a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e0070fd0 con 0x7ff2e0157df0 2024-03-04T08:15:53.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e6a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e015e090 con 0x7ff2e015ea60 2024-03-04T08:15:53.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e6a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e006dfd0 con 0x7ff2e00c8a90 2024-03-04T08:15:53.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e5600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff2e0157df0 0x7ff2e006c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54802/0 (socket says 172.21.15.113:54802) 2024-03-04T08:15:53.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e5600640 1 -- 172.21.15.113:0/3021963130 learned_addr learned my addr 172.21.15.113:0/3021963130 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:15:53.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e6000640 1 --2- 172.21.15.113:0/3021963130 >> v2:172.21.15.113:3300/0 conn(0x7ff2e015ea60 0x7ff2e00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:53.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e4c00640 1 --2- 172.21.15.113:0/3021963130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2e00c8a90 0x7ff2e015e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:53.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2dfe00640 1 -- 172.21.15.113:0/3021963130 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125820286 0 0) 0x7ff2d4001120 con 0x7ff2e0157df0 2024-03-04T08:15:53.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2dfe00640 1 -- 172.21.15.113:0/3021963130 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c8003530 con 0x7ff2e0157df0 2024-03-04T08:15:53.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e6000640 1 -- 172.21.15.113:0/3021963130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2e00c8a90 msgr2=0x7ff2e015e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:15:53.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e6000640 1 --2- 172.21.15.113:0/3021963130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2e00c8a90 0x7ff2e015e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:53.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e6000640 1 -- 172.21.15.113:0/3021963130 >> v1:172.21.15.113:6789/0 conn(0x7ff2e0157df0 legacy=0x7ff2e006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:53.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.281+0000 7ff2e6000640 1 -- 172.21.15.113:0/3021963130 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e006fcd0 con 0x7ff2e015ea60 2024-03-04T08:15:53.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2e6000640 1 --2- 172.21.15.113:0/3021963130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e015ea60 0x7ff2e00ce020 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009d90 tx=0x7ff2d801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519bafb0c1e9b679 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:53.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2dfe00640 1 -- 172.21.15.113:0/3021963130 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80b4070 con 0x7ff2e015ea60 2024-03-04T08:15:53.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2dfe00640 1 -- 172.21.15.113:0/3021963130 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d80af600 con 0x7ff2e015ea60 2024-03-04T08:15:53.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2dfe00640 1 -- 172.21.15.113:0/3021963130 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d80af9d0 con 0x7ff2e015ea60 2024-03-04T08:15:53.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2e6a00640 1 -- 172.21.15.113:0/3021963130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e015ea60 msgr2=0x7ff2e00ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:53.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2e6a00640 1 --2- 172.21.15.113:0/3021963130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e015ea60 0x7ff2e00ce020 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009d90 tx=0x7ff2d801f150 comp rx=0 tx=0).stop 2024-03-04T08:15:53.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2e6a00640 1 -- 172.21.15.113:0/3021963130 shutdown_connections 2024-03-04T08:15:53.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2e6a00640 1 --2- 172.21.15.113:0/3021963130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e015ea60 0x7ff2e00ce020 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:53.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2e6a00640 1 --2- 172.21.15.113:0/3021963130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2e00c8a90 0x7ff2e015e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:53.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2e6a00640 1 -- 172.21.15.113:0/3021963130 >> 172.21.15.113:0/3021963130 conn(0x7ff2e0072f40 msgr2=0x7ff2e00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:53.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2e6a00640 1 -- 172.21.15.113:0/3021963130 shutdown_connections 2024-03-04T08:15:53.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.285+0000 7ff2e6a00640 1 -- 172.21.15.113:0/3021963130 wait complete. 2024-03-04T08:15:53.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.289+0000 7ff2e6a00640 1 Processor -- start 2024-03-04T08:15:53.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.289+0000 7ff2e6a00640 1 -- start start 2024-03-04T08:15:53.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2e00c8a90 0x7ff2e00cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:53.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e0157df0 0x7ff2e01f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:53.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2e015ea60 0x7ff2e00c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:53.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00c30b0 con 0x7ff2e0157df0 2024-03-04T08:15:53.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e015dd20 con 0x7ff2e015ea60 2024-03-04T08:15:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e006dc00 con 0x7ff2e00c8a90 2024-03-04T08:15:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e0157df0 0x7ff2e01f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e0157df0 0x7ff2e01f02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40202/0 (socket says 172.21.15.113:40202) 2024-03-04T08:15:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e4c00640 1 -- 172.21.15.113:0/2437240558 learned_addr learned my addr 172.21.15.113:0/2437240558 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e5600640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2e00c8a90 0x7ff2e00cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6000640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2e015ea60 0x7ff2e00c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e4c00640 1 -- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2e00c8a90 msgr2=0x7ff2e00cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e4c00640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2e00c8a90 0x7ff2e00cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e4c00640 1 -- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2e015ea60 msgr2=0x7ff2e00c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:53.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e4c00640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2e015ea60 0x7ff2e00c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e4c00640 1 -- 172.21.15.113:0/2437240558 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e00c4b60 con 0x7ff2e0157df0 2024-03-04T08:15:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6000640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2e015ea60 0x7ff2e00c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:15:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e4c00640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e0157df0 0x7ff2e01f02f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00e930 tx=0x7ff2cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2dd600640 1 -- 172.21.15.113:0/2437240558 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc017070 con 0x7ff2e0157df0 2024-03-04T08:15:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2dd600640 1 -- 172.21.15.113:0/2437240558 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc00ca00 con 0x7ff2e0157df0 2024-03-04T08:15:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2dd600640 1 -- 172.21.15.113:0/2437240558 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc00cdb0 con 0x7ff2e0157df0 2024-03-04T08:15:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6a00640 1 -- 172.21.15.113:0/2437240558 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e01f3990 con 0x7ff2e0157df0 2024-03-04T08:15:53.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6a00640 1 -- 172.21.15.113:0/2437240558 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e00c5b20 con 0x7ff2e0157df0 2024-03-04T08:15:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.293+0000 7ff2e6a00640 1 -- 172.21.15.113:0/2437240558 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e0075a80 con 0x7ff2e0157df0 2024-03-04T08:15:53.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.297+0000 7ff2dd600640 1 -- 172.21.15.113:0/2437240558 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2cc01a070 con 0x7ff2e0157df0 2024-03-04T08:15:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.297+0000 7ff2dd600640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff2c0066f60 0x7ff2c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.297+0000 7ff2dd600640 1 -- 172.21.15.113:0/2437240558 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7ff2cc085830 con 0x7ff2e0157df0 2024-03-04T08:15:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.297+0000 7ff2e5600640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff2c0066f60 0x7ff2c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:53.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.297+0000 7ff2e5600640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff2c0066f60 0x7ff2c0069420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4005890 tx=0x7ff2d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:53.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.305+0000 7ff2dd600640 1 -- 172.21.15.113:0/2437240558 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff2cc055a30 con 0x7ff2e0157df0 2024-03-04T08:15:53.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.509+0000 7ff2e6a00640 1 -- 172.21.15.113:0/2437240558 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2e00c9610 con 0x7ff2e0157df0 2024-03-04T08:15:53.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2dd600640 1 -- 172.21.15.113:0/2437240558 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+6621 (secure 0 0 0) 0x7ff2cc059ad0 con 0x7ff2e0157df0 2024-03-04T08:15:53.511 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:15:53.511 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7004,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:11.665238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7519]},"id":1}]} 2024-03-04T08:15:53.511 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 150 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 -- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff2c0066f60 msgr2=0x7ff2c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff2c0066f60 0x7ff2c0069420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4005890 tx=0x7ff2d4009040 comp rx=0 tx=0).stop 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 -- 172.21.15.113:0/2437240558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e0157df0 msgr2=0x7ff2e01f02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e0157df0 0x7ff2e01f02f0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00e930 tx=0x7ff2cc00edf0 comp rx=0 tx=0).stop 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 -- 172.21.15.113:0/2437240558 shutdown_connections 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2e015ea60 0x7ff2e00c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff2c0066f60 0x7ff2c0069420 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2e0157df0 0x7ff2e01f02f0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 --2- 172.21.15.113:0/2437240558 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2e00c8a90 0x7ff2e00cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 -- 172.21.15.113:0/2437240558 >> 172.21.15.113:0/2437240558 conn(0x7ff2e0072f40 msgr2=0x7ff2e00c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 -- 172.21.15.113:0/2437240558 shutdown_connections 2024-03-04T08:15:53.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:53.513+0000 7ff2e6a00640 1 -- 172.21.15.113:0/2437240558 wait complete. 2024-03-04T08:15:53.543 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:15:55.544 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:15:55.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.729+0000 7f4fbfe00640 1 Processor -- start 2024-03-04T08:15:55.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.729+0000 7f4fbfe00640 1 -- start start 2024-03-04T08:15:55.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fbfe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc0152b70 0x7f4fc00ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:55.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fbfe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4fc006fdb0 0x7f4fc00ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:55.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fbfe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc015cb20 con 0x7f4fc00c31b0 2024-03-04T08:15:55.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fbfe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc0152f50 con 0x7f4fc006fdb0 2024-03-04T08:15:55.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fbfe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc006f030 con 0x7f4fc0152b70 2024-03-04T08:15:55.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fb7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4fc006fdb0 0x7f4fc00ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:55.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fb7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4fc006fdb0 0x7f4fc00ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40204/0 (socket says 172.21.15.113:40204) 2024-03-04T08:15:55.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fb7e00640 1 -- 172.21.15.113:0/2662985071 learned_addr learned my addr 172.21.15.113:0/2662985071 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:55.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fbea00640 1 --2- 172.21.15.113:0/2662985071 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc0152b70 0x7f4fc00ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:55.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fb7e00640 1 -- 172.21.15.113:0/2662985071 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc0152b70 msgr2=0x7f4fc00ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:15:55.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fb7e00640 1 --2- 172.21.15.113:0/2662985071 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc0152b70 0x7f4fc00ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:55.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fb7e00640 1 -- 172.21.15.113:0/2662985071 >> v1:172.21.15.113:6789/0 conn(0x7f4fc00c31b0 legacy=0x7f4fc00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:55.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fb7e00640 1 -- 172.21.15.113:0/2662985071 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc006f280 con 0x7f4fc006fdb0 2024-03-04T08:15:55.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.733+0000 7f4fbe000640 1 -- 172.21.15.113:0/2662985071 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936975460 0 0) 0x7f4fc015cb20 con 0x7f4fc00c31b0 2024-03-04T08:15:55.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fb7e00640 1 --2- 172.21.15.113:0/2662985071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 0x7f4fc00ce660 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009870 tx=0x7f4fa801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728c950d8ceabc25 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:55.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbe000640 1 -- 172.21.15.113:0/2662985071 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80b0070 con 0x7f4fc006fdb0 2024-03-04T08:15:55.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbe000640 1 -- 172.21.15.113:0/2662985071 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa80ab600 con 0x7f4fc006fdb0 2024-03-04T08:15:55.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbe000640 1 -- 172.21.15.113:0/2662985071 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80ab9d0 con 0x7f4fc006fdb0 2024-03-04T08:15:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2662985071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 msgr2=0x7f4fc00ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbfe00640 1 --2- 172.21.15.113:0/2662985071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 0x7f4fc00ce660 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009870 tx=0x7f4fa801c4d0 comp rx=0 tx=0).stop 2024-03-04T08:15:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2662985071 shutdown_connections 2024-03-04T08:15:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbfe00640 1 --2- 172.21.15.113:0/2662985071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 0x7f4fc00ce660 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbfe00640 1 --2- 172.21.15.113:0/2662985071 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc0152b70 0x7f4fc00ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:55.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2662985071 >> 172.21.15.113:0/2662985071 conn(0x7f4fc0072e70 msgr2=0x7f4fc0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2662985071 shutdown_connections 2024-03-04T08:15:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.737+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2662985071 wait complete. 2024-03-04T08:15:55.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbfe00640 1 Processor -- start 2024-03-04T08:15:55.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbfe00640 1 -- start start 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 0x7f4fc01f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbfe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc00c31b0 0x7f4fc006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fc0152b70 0x7f4fc0162020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbfe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015c8b0 con 0x7f4fc006fdb0 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbfe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015c730 con 0x7f4fc0152b70 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbfe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc01658d0 con 0x7f4fc00c31b0 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 0x7f4fc01f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 0x7f4fc01f7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40212/0 (socket says 172.21.15.113:40212) 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbea00640 1 -- 172.21.15.113:0/2661674130 learned_addr learned my addr 172.21.15.113:0/2661674130 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fb7e00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc00c31b0 0x7f4fc006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbea00640 1 -- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc00c31b0 msgr2=0x7f4fc006cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbf400640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fc0152b70 0x7f4fc0162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:55.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbea00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc00c31b0 0x7f4fc006cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbea00640 1 -- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fc0152b70 msgr2=0x7f4fc0162020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbea00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fc0152b70 0x7f4fc0162020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbea00640 1 -- 172.21.15.113:0/2661674130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc01f7a70 con 0x7f4fc006fdb0 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fb7e00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc00c31b0 0x7f4fc006cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbf400640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fc0152b70 0x7f4fc0162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbea00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 0x7f4fc01f7530 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000e870 tx=0x7f4fb000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fb6a00640 1 -- 172.21.15.113:0/2661674130 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0017070 con 0x7f4fc006fdb0 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fb6a00640 1 -- 172.21.15.113:0/2661674130 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb000c940 con 0x7f4fc006fdb0 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fb6a00640 1 -- 172.21.15.113:0/2661674130 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb000ccf0 con 0x7f4fc006fdb0 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2661674130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc00ce780 con 0x7f4fc006fdb0 2024-03-04T08:15:55.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.741+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2661674130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc006dda0 con 0x7f4fc006fdb0 2024-03-04T08:15:55.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.745+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2661674130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc006e0e0 con 0x7f4fc006fdb0 2024-03-04T08:15:55.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.745+0000 7f4fb6a00640 1 -- 172.21.15.113:0/2661674130 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4fb001a070 con 0x7f4fc006fdb0 2024-03-04T08:15:55.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.745+0000 7f4fb6a00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f9c066f60 0x7f4f9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:55.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.745+0000 7f4fb6a00640 1 -- 172.21.15.113:0/2661674130 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f4fb0085820 con 0x7f4fc006fdb0 2024-03-04T08:15:55.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.745+0000 7f4fb7e00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f9c066f60 0x7f4f9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:55.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.749+0000 7f4fb7e00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f9c066f60 0x7f4f9c069420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80ad000 tx=0x7f4fa8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:55.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:55.749+0000 7f4fb6a00640 1 -- 172.21.15.113:0/2661674130 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4fc006e0e0 con 0x7f4fc006fdb0 2024-03-04T08:15:56.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.029+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2661674130 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fc0075a80 con 0x7f4fc006fdb0 2024-03-04T08:15:56.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.029+0000 7f4fb6a00640 1 -- 172.21.15.113:0/2661674130 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f4fb0055f20 con 0x7f4fc006fdb0 2024-03-04T08:15:56.029 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:15:56.029 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7004,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:11.665238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7519]},"id":1}]} 2024-03-04T08:15:56.030 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 150 2024-03-04T08:15:56.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f9c066f60 msgr2=0x7f4f9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f9c066f60 0x7f4f9c069420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4fa80ad000 tx=0x7f4fa8008040 comp rx=0 tx=0).stop 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2661674130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 msgr2=0x7f4fc01f7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 0x7f4fc01f7530 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000e870 tx=0x7f4fb000ec70 comp rx=0 tx=0).stop 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2661674130 shutdown_connections 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fc0152b70 0x7f4fc0162020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4fc00c31b0 0x7f4fc006cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f9c066f60 0x7f4f9c069420 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 --2- 172.21.15.113:0/2661674130 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4fc006fdb0 0x7f4fc01f7530 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2661674130 >> 172.21.15.113:0/2661674130 conn(0x7f4fc0072e70 msgr2=0x7f4fc00b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2661674130 shutdown_connections 2024-03-04T08:15:56.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:56.033+0000 7f4fbfe00640 1 -- 172.21.15.113:0/2661674130 wait complete. 2024-03-04T08:15:56.050 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:15:58.052 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:15:58.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.189+0000 7f5793200640 1 Processor -- start 2024-03-04T08:15:58.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.189+0000 7f5793200640 1 -- start start 2024-03-04T08:15:58.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5793200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f578c0c8200 0x7f578c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5793200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f578c152bd0 0x7f578c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5793200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578c1591f0 con 0x7f578c1527f0 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5793200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578c071df0 con 0x7f578c152bd0 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5793200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578c1597b0 con 0x7f578c0c8200 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5791200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f578c152bd0 0x7f578c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5791200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f578c152bd0 0x7f578c1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34870/0 (socket says 172.21.15.113:34870) 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5791200640 1 -- 172.21.15.113:0/657306014 learned_addr learned my addr 172.21.15.113:0/657306014 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5783e00640 1 --2- 172.21.15.113:0/657306014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f578c0c8200 0x7f578c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5791200640 1 -- 172.21.15.113:0/657306014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f578c0c8200 msgr2=0x7f578c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5791200640 1 --2- 172.21.15.113:0/657306014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f578c0c8200 0x7f578c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:58.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5791200640 1 -- 172.21.15.113:0/657306014 >> v1:172.21.15.136:6789/0 conn(0x7f578c1527f0 legacy=0x7f578c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:58.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.193+0000 7f5791200640 1 -- 172.21.15.113:0/657306014 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c1594d0 con 0x7f578c152bd0 2024-03-04T08:15:58.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5791200640 1 --2- 172.21.15.113:0/657306014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 0x7f578c1587d0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5784009e50 tx=0x7f578401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c302ca564c5bd761 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:58.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f578b400640 1 -- 172.21.15.113:0/657306014 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57840a57f0 con 0x7f578c152bd0 2024-03-04T08:15:58.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f578b400640 1 -- 172.21.15.113:0/657306014 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57840a5990 con 0x7f578c152bd0 2024-03-04T08:15:58.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f578b400640 1 -- 172.21.15.113:0/657306014 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57840a5d60 con 0x7f578c152bd0 2024-03-04T08:15:58.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5793200640 1 -- 172.21.15.113:0/657306014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 msgr2=0x7f578c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:58.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5793200640 1 --2- 172.21.15.113:0/657306014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 0x7f578c1587d0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5784009e50 tx=0x7f578401ca30 comp rx=0 tx=0).stop 2024-03-04T08:15:58.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5793200640 1 -- 172.21.15.113:0/657306014 shutdown_connections 2024-03-04T08:15:58.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5793200640 1 --2- 172.21.15.113:0/657306014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 0x7f578c1587d0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:58.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5793200640 1 --2- 172.21.15.113:0/657306014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f578c0c8200 0x7f578c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:58.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5793200640 1 -- 172.21.15.113:0/657306014 >> 172.21.15.113:0/657306014 conn(0x7f578c06e820 msgr2=0x7f578c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:58.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5793200640 1 -- 172.21.15.113:0/657306014 shutdown_connections 2024-03-04T08:15:58.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5793200640 1 -- 172.21.15.113:0/657306014 wait complete. 2024-03-04T08:15:58.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5793200640 1 Processor -- start 2024-03-04T08:15:58.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.197+0000 7f5793200640 1 -- start start 2024-03-04T08:15:58.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5793200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f578c0c8200 0x7f578c1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:58.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5793200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f578c1527f0 0x7f578c14d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:58.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5793200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 0x7f578c1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5793200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c1654f0 con 0x7f578c152bd0 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5793200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578c165370 con 0x7f578c1527f0 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5793200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578c165670 con 0x7f578c0c8200 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5791200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 0x7f578c1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5791200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 0x7f578c1555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34880/0 (socket says 172.21.15.113:34880) 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5791200640 1 -- 172.21.15.113:0/3327200447 learned_addr learned my addr 172.21.15.113:0/3327200447 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5783e00640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f578c1527f0 0x7f578c14d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f578be00640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f578c0c8200 0x7f578c1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5791200640 1 -- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f578c0c8200 msgr2=0x7f578c1530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5791200640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f578c0c8200 0x7f578c1530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5791200640 1 -- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f578c1527f0 msgr2=0x7f578c14d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5791200640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f578c1527f0 0x7f578c14d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5791200640 1 -- 172.21.15.113:0/3327200447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c155b20 con 0x7f578c152bd0 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5791200640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 0x7f578c1555e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f57840af040 tx=0x7f57840197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:58.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5788c00640 1 -- 172.21.15.113:0/3327200447 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57840b8670 con 0x7f578c152bd0 2024-03-04T08:15:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5788c00640 1 -- 172.21.15.113:0/3327200447 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57840be070 con 0x7f578c152bd0 2024-03-04T08:15:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5788c00640 1 -- 172.21.15.113:0/3327200447 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57840b8a50 con 0x7f578c152bd0 2024-03-04T08:15:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5793200640 1 -- 172.21.15.113:0/3327200447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578c155040 con 0x7f578c152bd0 2024-03-04T08:15:58.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5793200640 1 -- 172.21.15.113:0/3327200447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578c14cd60 con 0x7f578c152bd0 2024-03-04T08:15:58.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.201+0000 7f5793200640 1 -- 172.21.15.113:0/3327200447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578c075c00 con 0x7f578c152bd0 2024-03-04T08:15:58.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.205+0000 7f5788c00640 1 -- 172.21.15.113:0/3327200447 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57840b8bf0 con 0x7f578c152bd0 2024-03-04T08:15:58.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.205+0000 7f5788c00640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5768066f60 0x7f5768069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:15:58.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.205+0000 7f5788c00640 1 -- 172.21.15.113:0/3327200447 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f5784120ad0 con 0x7f578c152bd0 2024-03-04T08:15:58.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.205+0000 7f578be00640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5768066f60 0x7f5768069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:15:58.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.205+0000 7f578be00640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5768066f60 0x7f5768069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f577c001810 tx=0x7f577c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:15:58.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.209+0000 7f5788c00640 1 -- 172.21.15.113:0/3327200447 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5784016620 con 0x7f578c152bd0 2024-03-04T08:15:58.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.401+0000 7f5793200640 1 -- 172.21.15.113:0/3327200447 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f578c0c79d0 con 0x7f578c152bd0 2024-03-04T08:15:58.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5788c00640 1 -- 172.21.15.113:0/3327200447 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f57840f0d30 con 0x7f578c152bd0 2024-03-04T08:15:58.403 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:15:58.403 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7004,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:11.665238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7519]},"id":1}]} 2024-03-04T08:15:58.404 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 150 2024-03-04T08:15:58.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 -- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5768066f60 msgr2=0x7f5768069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:58.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5768066f60 0x7f5768069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f577c001810 tx=0x7f577c009040 comp rx=0 tx=0).stop 2024-03-04T08:15:58.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 -- 172.21.15.113:0/3327200447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 msgr2=0x7f578c1555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:15:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 0x7f578c1555e0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f57840af040 tx=0x7f57840197d0 comp rx=0 tx=0).stop 2024-03-04T08:15:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 -- 172.21.15.113:0/3327200447 shutdown_connections 2024-03-04T08:15:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5768066f60 0x7f5768069420 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f578c152bd0 0x7f578c1555e0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f578c1527f0 0x7f578c14d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 --2- 172.21.15.113:0/3327200447 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f578c0c8200 0x7f578c1530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:15:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 -- 172.21.15.113:0/3327200447 >> 172.21.15.113:0/3327200447 conn(0x7f578c06e820 msgr2=0x7f578c0c2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:15:58.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 -- 172.21.15.113:0/3327200447 shutdown_connections 2024-03-04T08:15:58.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:15:58.405+0000 7f5793200640 1 -- 172.21.15.113:0/3327200447 wait complete. 2024-03-04T08:15:58.426 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:16:00.427 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:00.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f1122200640 1 Processor -- start 2024-03-04T08:16:00.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f1122200640 1 -- start start 2024-03-04T08:16:00.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f1122200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 0x7f111c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:00.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f1122200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f111c152bd0 0x7f111c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:00.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f1122200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c1591f0 con 0x7f111c0c8200 2024-03-04T08:16:00.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f1122200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c071df0 con 0x7f111c152bd0 2024-03-04T08:16:00.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f1122200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c1597b0 con 0x7f111c1527f0 2024-03-04T08:16:00.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f111be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f111c152bd0 0x7f111c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:00.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f111be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f111c152bd0 0x7f111c1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34896/0 (socket says 172.21.15.113:34896) 2024-03-04T08:16:00.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f1112a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f111c0c8200 0x7f111c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60490/0 (socket says 172.21.15.113:60490) 2024-03-04T08:16:00.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f111be00640 1 -- 172.21.15.113:0/3014994515 learned_addr learned my addr 172.21.15.113:0/3014994515 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:00.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f111b400640 1 --2- 172.21.15.113:0/3014994515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 0x7f111c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:00.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f111be00640 1 -- 172.21.15.113:0/3014994515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 msgr2=0x7f111c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:00.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f111be00640 1 --2- 172.21.15.113:0/3014994515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 0x7f111c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:00.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f111be00640 1 -- 172.21.15.113:0/3014994515 >> v1:172.21.15.113:6789/0 conn(0x7f111c0c8200 legacy=0x7f111c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:00.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f111be00640 1 -- 172.21.15.113:0/3014994515 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c1594d0 con 0x7f111c152bd0 2024-03-04T08:16:00.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f111aa00640 1 -- 172.21.15.113:0/3014994515 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160143962 0 0) 0x7f111c1591f0 con 0x7f111c0c8200 2024-03-04T08:16:00.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.569+0000 7f111b400640 1 --2- 172.21.15.113:0/3014994515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 0x7f111c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:16:00.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f111be00640 1 --2- 172.21.15.113:0/3014994515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 0x7f111c1587d0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1114009e50 tx=0x7f111401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434207583ad97306 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:00.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f111aa00640 1 -- 172.21.15.113:0/3014994515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11140a57f0 con 0x7f111c152bd0 2024-03-04T08:16:00.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f111aa00640 1 -- 172.21.15.113:0/3014994515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11140a5990 con 0x7f111c152bd0 2024-03-04T08:16:00.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f111aa00640 1 -- 172.21.15.113:0/3014994515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11140a5d60 con 0x7f111c152bd0 2024-03-04T08:16:00.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f1122200640 1 -- 172.21.15.113:0/3014994515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 msgr2=0x7f111c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:00.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f1122200640 1 --2- 172.21.15.113:0/3014994515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 0x7f111c1587d0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1114009e50 tx=0x7f111401ca30 comp rx=0 tx=0).stop 2024-03-04T08:16:00.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f1122200640 1 -- 172.21.15.113:0/3014994515 shutdown_connections 2024-03-04T08:16:00.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f1122200640 1 --2- 172.21.15.113:0/3014994515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 0x7f111c1587d0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:00.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f1122200640 1 --2- 172.21.15.113:0/3014994515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 0x7f111c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:00.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f1122200640 1 -- 172.21.15.113:0/3014994515 >> 172.21.15.113:0/3014994515 conn(0x7f111c06e820 msgr2=0x7f111c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:00.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f1122200640 1 -- 172.21.15.113:0/3014994515 shutdown_connections 2024-03-04T08:16:00.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.573+0000 7f1122200640 1 -- 172.21.15.113:0/3014994515 wait complete. 2024-03-04T08:16:00.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f1122200640 1 Processor -- start 2024-03-04T08:16:00.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f1122200640 1 -- start start 2024-03-04T08:16:00.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f1122200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f111c0c8200 0x7f111c1f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:00.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f1122200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 0x7f111c14c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f1122200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 0x7f111c0c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f1122200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c1654f0 con 0x7f111c152bd0 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f1122200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c165370 con 0x7f111c0c8200 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f1122200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c165670 con 0x7f111c1527f0 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f111be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 0x7f111c0c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f111be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 0x7f111c0c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34898/0 (socket says 172.21.15.113:34898) 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f111be00640 1 -- 172.21.15.113:0/3459970683 learned_addr learned my addr 172.21.15.113:0/3459970683 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f111b400640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f111c0c8200 0x7f111c1f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f1112a00640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 0x7f111c14c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f111be00640 1 -- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 msgr2=0x7f111c14c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f111be00640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 0x7f111c14c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f111be00640 1 -- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f111c0c8200 msgr2=0x7f111c1f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:00.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f111be00640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f111c0c8200 0x7f111c1f38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:00.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.577+0000 7f111be00640 1 -- 172.21.15.113:0/3459970683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c1f3de0 con 0x7f111c152bd0 2024-03-04T08:16:00.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.581+0000 7f111be00640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 0x7f111c0c8610 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1114009e50 tx=0x7f11140197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:00.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.581+0000 7f1113e00640 1 -- 172.21.15.113:0/3459970683 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11140b8670 con 0x7f111c152bd0 2024-03-04T08:16:00.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.581+0000 7f1113e00640 1 -- 172.21.15.113:0/3459970683 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11140be070 con 0x7f111c152bd0 2024-03-04T08:16:00.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.581+0000 7f1113e00640 1 -- 172.21.15.113:0/3459970683 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11140b8a30 con 0x7f111c152bd0 2024-03-04T08:16:00.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.581+0000 7f1122200640 1 -- 172.21.15.113:0/3459970683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c1f4070 con 0x7f111c152bd0 2024-03-04T08:16:00.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.581+0000 7f1122200640 1 -- 172.21.15.113:0/3459970683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c1f4630 con 0x7f111c152bd0 2024-03-04T08:16:00.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.581+0000 7f1122200640 1 -- 172.21.15.113:0/3459970683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c075c00 con 0x7f111c152bd0 2024-03-04T08:16:00.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.581+0000 7f1113e00640 1 -- 172.21.15.113:0/3459970683 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11140b8bd0 con 0x7f111c152bd0 2024-03-04T08:16:00.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.581+0000 7f1113e00640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f8066f60 0x7f10f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:00.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.581+0000 7f1113e00640 1 -- 172.21.15.113:0/3459970683 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7717+0+0 (secure 0 0 0) 0x7f1114120a20 con 0x7f111c152bd0 2024-03-04T08:16:00.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.585+0000 7f111b400640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f8066f60 0x7f10f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:00.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.585+0000 7f111b400640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f8066f60 0x7f10f8069420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f110c00c900 tx=0x7f110c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:00.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.585+0000 7f1113e00640 1 -- 172.21.15.113:0/3459970683 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1114016620 con 0x7f111c152bd0 2024-03-04T08:16:00.706 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:00.706 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.44 2024-03-04T08:16:00.706 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 44 -eq 48 ] 2024-03-04T08:16:00.706 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.44 2024-03-04T08:16:00.707 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.44 2024-03-04T08:16:00.707 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.44 2024-03-04T08:16:00.707 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:00.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 -- 172.21.15.113:0/3459970683 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f111c0cb9b0 con 0x7f111c152bd0 2024-03-04T08:16:00.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1113e00640 1 -- 172.21.15.113:0/3459970683 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 150 v150) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f11140f11b0 con 0x7f111c152bd0 2024-03-04T08:16:00.808 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:00.808 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":150,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":7004,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":150}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:14:11.665238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7519,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7519":{"gid":7519,"name":"b","rank":1,"incarnation":133,"state":"up:active","state_seq":36,"addr":"172.21.15.136:6835/247269911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":247269911},{"type":"v1","addr":"172.21.15.136:6835","nonce":247269911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7519]},"id":1}]} 2024-03-04T08:16:00.808 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 150 2024-03-04T08:16:00.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 -- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f8066f60 msgr2=0x7f10f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:00.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f8066f60 0x7f10f8069420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f110c00c900 tx=0x7f110c008040 comp rx=0 tx=0).stop 2024-03-04T08:16:00.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 -- 172.21.15.113:0/3459970683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 msgr2=0x7f111c0c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:00.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 0x7f111c0c8610 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1114009e50 tx=0x7f11140197d0 comp rx=0 tx=0).stop 2024-03-04T08:16:00.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 -- 172.21.15.113:0/3459970683 shutdown_connections 2024-03-04T08:16:00.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f10f8066f60 0x7f10f8069420 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:00.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f111c152bd0 0x7f111c0c8610 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:00.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f111c1527f0 0x7f111c14c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:00.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 --2- 172.21.15.113:0/3459970683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f111c0c8200 0x7f111c1f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:00.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 -- 172.21.15.113:0/3459970683 >> 172.21.15.113:0/3459970683 conn(0x7f111c06e820 msgr2=0x7f111c0c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:00.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 -- 172.21.15.113:0/3459970683 shutdown_connections 2024-03-04T08:16:00.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:00.809+0000 7f1122200640 1 -- 172.21.15.113:0/3459970683 wait complete. 2024-03-04T08:16:00.829 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2024-03-04T08:16:02.830 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.969+0000 7ff412c00640 1 Processor -- start 2024-03-04T08:16:02.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.969+0000 7ff412c00640 1 -- start start 2024-03-04T08:16:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.969+0000 7ff412c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c06df10 0x7ff40c15e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:02.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.969+0000 7ff412c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c06c310 con 0x7ff40c158230 2024-03-04T08:16:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.969+0000 7ff412c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40c0c95f0 con 0x7ff40c0703e0 2024-03-04T08:16:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.969+0000 7ff412c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c06c5b0 con 0x7ff40c06df10 2024-03-04T08:16:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.969+0000 7ff40be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff40c158230 0x7ff40c06f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60504/0 (socket says 172.21.15.113:60504) 2024-03-04T08:16:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff40be00640 1 -- 172.21.15.113:0/2213686415 learned_addr learned my addr 172.21.15.113:0/2213686415 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:16:02.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff403400640 1 --2- 172.21.15.113:0/2213686415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c06df10 0x7ff40c15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff40b400640 1 -- 172.21.15.113:0/2213686415 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972065204 0 0) 0x7ff40c06c310 con 0x7ff40c158230 2024-03-04T08:16:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff40b400640 1 -- 172.21.15.113:0/2213686415 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f0003530 con 0x7ff40c158230 2024-03-04T08:16:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff40b400640 1 -- 172.21.15.113:0/2213686415 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220245802 0 0) 0x7ff40c0c95f0 con 0x7ff40c0703e0 2024-03-04T08:16:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff40b400640 1 -- 172.21.15.113:0/2213686415 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff40c06c310 con 0x7ff40c0703e0 2024-03-04T08:16:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff403400640 1 -- 172.21.15.113:0/2213686415 >> v1:172.21.15.136:6789/0 conn(0x7ff40c0703e0 legacy=0x7ff40c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff40b400640 1 -- 172.21.15.113:0/2213686415 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1012592860 0 0) 0x7ff3f0003530 con 0x7ff40c158230 2024-03-04T08:16:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff403400640 1 -- 172.21.15.113:0/2213686415 >> v1:172.21.15.113:6789/0 conn(0x7ff40c158230 legacy=0x7ff40c06f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:02.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff403400640 1 -- 172.21.15.113:0/2213686415 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0c9890 con 0x7ff40c06df10 2024-03-04T08:16:02.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.973+0000 7ff403400640 1 --2- 172.21.15.113:0/2213686415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c06df10 0x7ff40c15e190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40097f0 tx=0x7ff3f401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4bafd962dd4bbb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:02.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff40b400640 1 -- 172.21.15.113:0/2213686415 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40b0070 con 0x7ff40c06df10 2024-03-04T08:16:02.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff40b400640 1 -- 172.21.15.113:0/2213686415 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f40ab400 con 0x7ff40c06df10 2024-03-04T08:16:02.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff40b400640 1 -- 172.21.15.113:0/2213686415 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc001a00 con 0x7ff40c06df10 2024-03-04T08:16:02.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 -- 172.21.15.113:0/2213686415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c06df10 msgr2=0x7ff40c15e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 --2- 172.21.15.113:0/2213686415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c06df10 0x7ff40c15e190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40097f0 tx=0x7ff3f401c400 comp rx=0 tx=0).stop 2024-03-04T08:16:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 -- 172.21.15.113:0/2213686415 shutdown_connections 2024-03-04T08:16:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 --2- 172.21.15.113:0/2213686415 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c06df10 0x7ff40c15e190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 -- 172.21.15.113:0/2213686415 >> 172.21.15.113:0/2213686415 conn(0x7ff40c066db0 msgr2=0x7ff40c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 -- 172.21.15.113:0/2213686415 shutdown_connections 2024-03-04T08:16:02.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 -- 172.21.15.113:0/2213686415 wait complete. 2024-03-04T08:16:02.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 Processor -- start 2024-03-04T08:16:02.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 -- start start 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff40c06df10 0x7ff40c14fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff40c0703e0 0x7ff40c150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c158230 0x7ff40c1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c168e70 con 0x7ff40c0703e0 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c168cf0 con 0x7ff40c06df10 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff412c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c168ff0 con 0x7ff40c158230 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff403400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff40c0703e0 0x7ff40c150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff403400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff40c0703e0 0x7ff40c150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34906/0 (socket says 172.21.15.113:34906) 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff403400640 1 -- 172.21.15.113:0/3388787333 learned_addr learned my addr 172.21.15.113:0/3388787333 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff40be00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff40c06df10 0x7ff40c14fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff410c00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c158230 0x7ff40c1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.977+0000 7ff403400640 1 -- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c158230 msgr2=0x7ff40c1595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:02.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff403400640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c158230 0x7ff40c1595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:02.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff403400640 1 -- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff40c06df10 msgr2=0x7ff40c14fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:02.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff403400640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff40c06df10 0x7ff40c14fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:02.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff403400640 1 -- 172.21.15.113:0/3388787333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0c7070 con 0x7ff40c0703e0 2024-03-04T08:16:02.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff403400640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff40c0703e0 0x7ff40c150210 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40097f0 tx=0x7ff3f401ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff408c00640 1 -- 172.21.15.113:0/3388787333 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40b0050 con 0x7ff40c0703e0 2024-03-04T08:16:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff408c00640 1 -- 172.21.15.113:0/3388787333 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f40bc070 con 0x7ff40c0703e0 2024-03-04T08:16:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff408c00640 1 -- 172.21.15.113:0/3388787333 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40b7850 con 0x7ff40c0703e0 2024-03-04T08:16:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff412c00640 1 -- 172.21.15.113:0/3388787333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c0c7330 con 0x7ff40c0703e0 2024-03-04T08:16:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff412c00640 1 -- 172.21.15.113:0/3388787333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c0c78f0 con 0x7ff40c0703e0 2024-03-04T08:16:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff412c00640 1 -- 172.21.15.113:0/3388787333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c075c00 con 0x7ff40c0703e0 2024-03-04T08:16:02.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.981+0000 7ff408c00640 1 -- 172.21.15.113:0/3388787333 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3f40b79f0 con 0x7ff40c0703e0 2024-03-04T08:16:02.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.985+0000 7ff408c00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3e8066f60 0x7ff3e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:02.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.985+0000 7ff408c00640 1 -- 172.21.15.113:0/3388787333 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7ff3f411ffd0 con 0x7ff40c0703e0 2024-03-04T08:16:02.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.985+0000 7ff40be00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3e8066f60 0x7ff3e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:02.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.985+0000 7ff40be00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3e8066f60 0x7ff3e8069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc001570 tx=0x7ff3fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:02.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:02.989+0000 7ff408c00640 1 -- 172.21.15.113:0/3388787333 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff3f4015ff0 con 0x7ff40c0703e0 2024-03-04T08:16:03.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.273+0000 7ff412c00640 1 -- 172.21.15.113:0/3388787333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff40c155ad0 con 0x7ff40c0703e0 2024-03-04T08:16:03.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.273+0000 7ff408c00640 1 -- 172.21.15.113:0/3388787333 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+5823 (secure 0 0 0) 0x7ff3f40f0710 con 0x7ff40c0703e0 2024-03-04T08:16:03.274 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:03.274 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":151,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:00.936813+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839]},"id":1}]} 2024-03-04T08:16:03.274 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 151 2024-03-04T08:16:03.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 -- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3e8066f60 msgr2=0x7ff3e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:03.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3e8066f60 0x7ff3e8069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc001570 tx=0x7ff3fc009040 comp rx=0 tx=0).stop 2024-03-04T08:16:03.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 -- 172.21.15.113:0/3388787333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff40c0703e0 msgr2=0x7ff40c150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:03.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff40c0703e0 0x7ff40c150210 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40097f0 tx=0x7ff3f401ac40 comp rx=0 tx=0).stop 2024-03-04T08:16:03.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 -- 172.21.15.113:0/3388787333 shutdown_connections 2024-03-04T08:16:03.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff40c158230 0x7ff40c1595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff3e8066f60 0x7ff3e8069420 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff40c0703e0 0x7ff40c150210 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 --2- 172.21.15.113:0/3388787333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff40c06df10 0x7ff40c14fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 -- 172.21.15.113:0/3388787333 >> 172.21.15.113:0/3388787333 conn(0x7ff40c066db0 msgr2=0x7ff40c067190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:03.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 -- 172.21.15.113:0/3388787333 shutdown_connections 2024-03-04T08:16:03.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.277+0000 7ff412c00640 1 -- 172.21.15.113:0/3388787333 wait complete. 2024-03-04T08:16:03.301 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2024-03-04T08:16:03.301 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:16:03.301 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:03.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd48d600640 1 Processor -- start 2024-03-04T08:16:03.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd48d600640 1 -- start start 2024-03-04T08:16:03.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd48d600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 0x7fd488070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd48d600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd4880c8200 0x7fd488072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd48d600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4881591f0 con 0x7fd488152bd0 2024-03-04T08:16:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd48d600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd488071df0 con 0x7fd4880c8200 2024-03-04T08:16:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd48d600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4881597b0 con 0x7fd4881527f0 2024-03-04T08:16:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd47e000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd4880c8200 0x7fd488072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd486a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 0x7fd488070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd486a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 0x7fd488070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:48764/0 (socket says 172.21.15.113:48764) 2024-03-04T08:16:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd486a00640 1 -- 172.21.15.113:0/346536391 learned_addr learned my addr 172.21.15.113:0/346536391 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:03.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd47e000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd4880c8200 0x7fd488072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48024/0 (socket says 172.21.15.113:48024) 2024-03-04T08:16:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd486a00640 1 -- 172.21.15.113:0/346536391 >> v2:172.21.15.136:3300/0 conn(0x7fd4880c8200 msgr2=0x7fd488072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd486a00640 1 --2- 172.21.15.113:0/346536391 >> v2:172.21.15.136:3300/0 conn(0x7fd4880c8200 0x7fd488072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd486a00640 1 -- 172.21.15.113:0/346536391 >> v1:172.21.15.136:6789/0 conn(0x7fd488152bd0 legacy=0x7fd4881587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd486a00640 1 -- 172.21.15.113:0/346536391 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4881594d0 con 0x7fd4881527f0 2024-03-04T08:16:03.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.445+0000 7fd47e000640 1 --2- 172.21.15.113:0/346536391 >> v2:172.21.15.136:3300/0 conn(0x7fd4880c8200 0x7fd488072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:16:03.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.449+0000 7fd486a00640 1 --2- 172.21.15.113:0/346536391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 0x7fd488070d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd478009e50 tx=0x7fd47801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a730d0904efc992 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:03.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.449+0000 7fd486000640 1 -- 172.21.15.113:0/346536391 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780a5810 con 0x7fd4881527f0 2024-03-04T08:16:03.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.449+0000 7fd486000640 1 -- 172.21.15.113:0/346536391 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4780a59b0 con 0x7fd4881527f0 2024-03-04T08:16:03.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.449+0000 7fd486000640 1 -- 172.21.15.113:0/346536391 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780a5d80 con 0x7fd4881527f0 2024-03-04T08:16:03.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.449+0000 7fd48d600640 1 -- 172.21.15.113:0/346536391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 msgr2=0x7fd488070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:03.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.449+0000 7fd48d600640 1 --2- 172.21.15.113:0/346536391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 0x7fd488070d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd478009e50 tx=0x7fd47801ca30 comp rx=0 tx=0).stop 2024-03-04T08:16:03.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.449+0000 7fd48d600640 1 -- 172.21.15.113:0/346536391 shutdown_connections 2024-03-04T08:16:03.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.449+0000 7fd48d600640 1 --2- 172.21.15.113:0/346536391 >> v2:172.21.15.136:3300/0 conn(0x7fd4880c8200 0x7fd488072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.449+0000 7fd48d600640 1 --2- 172.21.15.113:0/346536391 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 0x7fd488070d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.449+0000 7fd48d600640 1 -- 172.21.15.113:0/346536391 >> 172.21.15.113:0/346536391 conn(0x7fd48806e820 msgr2=0x7fd48806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:03.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.453+0000 7fd48d600640 1 -- 172.21.15.113:0/346536391 shutdown_connections 2024-03-04T08:16:03.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.453+0000 7fd48d600640 1 -- 172.21.15.113:0/346536391 wait complete. 2024-03-04T08:16:03.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.453+0000 7fd48d600640 1 Processor -- start 2024-03-04T08:16:03.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.453+0000 7fd48d600640 1 -- start start 2024-03-04T08:16:03.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd48d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4880c8200 0x7fd488151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd48d600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 0x7fd48814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd48d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd488152bd0 0x7fd488156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd48d600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488165160 con 0x7fd488152bd0 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd48d600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4880727c0 con 0x7fd4880c8200 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd48d600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4881652e0 con 0x7fd4881527f0 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd487400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd488152bd0 0x7fd488156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd487400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd488152bd0 0x7fd488156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34922/0 (socket says 172.21.15.113:34922) 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd487400640 1 -- 172.21.15.113:0/3475173352 learned_addr learned my addr 172.21.15.113:0/3475173352 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd47e000640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 0x7fd48814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd486a00640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4880c8200 0x7fd488151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd487400640 1 -- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 msgr2=0x7fd48814a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:03.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd487400640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 0x7fd48814a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd487400640 1 -- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4880c8200 msgr2=0x7fd488151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:03.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd487400640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4880c8200 0x7fd488151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd487400640 1 -- 172.21.15.113:0/3475173352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48814fcf0 con 0x7fd488152bd0 2024-03-04T08:16:03.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd487400640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd488152bd0 0x7fd488156eb0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c340 tx=0x7fd48000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:03.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd47f400640 1 -- 172.21.15.113:0/3475173352 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480005220 con 0x7fd488152bd0 2024-03-04T08:16:03.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd47f400640 1 -- 172.21.15.113:0/3475173352 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4800053c0 con 0x7fd488152bd0 2024-03-04T08:16:03.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd47f400640 1 -- 172.21.15.113:0/3475173352 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480007270 con 0x7fd488152bd0 2024-03-04T08:16:03.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd48d600640 1 -- 172.21.15.113:0/3475173352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48814ffd0 con 0x7fd488152bd0 2024-03-04T08:16:03.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd48d600640 1 -- 172.21.15.113:0/3475173352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4881566b0 con 0x7fd488152bd0 2024-03-04T08:16:03.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.457+0000 7fd48d600640 1 -- 172.21.15.113:0/3475173352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd488075c00 con 0x7fd488152bd0 2024-03-04T08:16:03.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.461+0000 7fd47f400640 1 -- 172.21.15.113:0/3475173352 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd480019020 con 0x7fd488152bd0 2024-03-04T08:16:03.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.461+0000 7fd47f400640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd464066f60 0x7fd464069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:03.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.461+0000 7fd47f400640 1 -- 172.21.15.113:0/3475173352 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7fd480084ab0 con 0x7fd488152bd0 2024-03-04T08:16:03.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.461+0000 7fd486a00640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd464066f60 0x7fd464069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:03.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.461+0000 7fd486a00640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd464066f60 0x7fd464069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd4780ae000 tx=0x7fd4780a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:03.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.465+0000 7fd47f400640 1 -- 172.21.15.113:0/3475173352 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd480054ba0 con 0x7fd488152bd0 2024-03-04T08:16:03.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 -- 172.21.15.113:0/3475173352 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4880c79d0 con 0x7fd488152bd0 2024-03-04T08:16:03.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd47f400640 1 -- 172.21.15.113:0/3475173352 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+5823 (secure 0 0 0) 0x7fd480058c40 con 0x7fd488152bd0 2024-03-04T08:16:03.737 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:03.737 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":151,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:00.936813+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839]},"id":1}]} 2024-03-04T08:16:03.737 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 151 2024-03-04T08:16:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 -- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd464066f60 msgr2=0x7fd464069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd464066f60 0x7fd464069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd4780ae000 tx=0x7fd4780a7070 comp rx=0 tx=0).stop 2024-03-04T08:16:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 -- 172.21.15.113:0/3475173352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd488152bd0 msgr2=0x7fd488156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd488152bd0 0x7fd488156eb0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c340 tx=0x7fd48000c800 comp rx=0 tx=0).stop 2024-03-04T08:16:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 -- 172.21.15.113:0/3475173352 shutdown_connections 2024-03-04T08:16:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd464066f60 0x7fd464069420 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd488152bd0 0x7fd488156eb0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd4881527f0 0x7fd48814a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 --2- 172.21.15.113:0/3475173352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4880c8200 0x7fd488151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.737+0000 7fd48d600640 1 -- 172.21.15.113:0/3475173352 >> 172.21.15.113:0/3475173352 conn(0x7fd48806e820 msgr2=0x7fd488070b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:03.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.741+0000 7fd48d600640 1 -- 172.21.15.113:0/3475173352 shutdown_connections 2024-03-04T08:16:03.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:03.741+0000 7fd48d600640 1 -- 172.21.15.113:0/3475173352 wait complete. 2024-03-04T08:16:03.758 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 151, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 151, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7004": { "addr": "172.21.15.113:6836/4083765155", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 4083765155, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 4083765155, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7004, "incarnation": 151, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:replay", "state_seq": 1 }, "gid_7692": { "addr": "172.21.15.113:6833/1928887637", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 1928887637, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 1928887637, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7692, "incarnation": 139, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 } }, "last_failure": 0, "last_failure_osd_epoch": 71, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:16:00.936813+0000", "qdb_cluster": [ 7692, 7839 ], "qdb_leader": 7839, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 7004, "mds_2": 7692 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:16:05.475 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:05.475 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.45 2024-03-04T08:16:05.475 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 45 -eq 48 ] 2024-03-04T08:16:05.475 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.45 2024-03-04T08:16:05.476 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.45 2024-03-04T08:16:05.476 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.45 2024-03-04T08:16:05.476 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:05.759 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:05.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.905+0000 7f76e0200640 1 Processor -- start 2024-03-04T08:16:05.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.905+0000 7f76e0200640 1 -- start start 2024-03-04T08:16:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.905+0000 7f76e0200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f76d8152d60 0x7f76d80ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.905+0000 7f76e0200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d8159040 0x7f76d80ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.905+0000 7f76e0200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d80c5380 con 0x7f76d8070120 2024-03-04T08:16:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.905+0000 7f76e0200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d80c3440 con 0x7f76d8152d60 2024-03-04T08:16:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.905+0000 7f76e0200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d806f620 con 0x7f76d8159040 2024-03-04T08:16:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76dce00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f76d8070120 0x7f76d8158930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60508/0 (socket says 172.21.15.113:60508) 2024-03-04T08:16:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76dce00640 1 -- 172.21.15.113:0/1010420614 learned_addr learned my addr 172.21.15.113:0/1010420614 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:16:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76dd800640 1 --2- 172.21.15.113:0/1010420614 >> v2:172.21.15.136:3300/0 conn(0x7f76d8152d60 0x7f76d80ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:05.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76de200640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d8159040 0x7f76d80ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281527265 0 0) 0x7f76d80c5380 con 0x7f76d8070120 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76c0003530 con 0x7f76d8070120 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1555462413 0 0) 0x7f76c0003530 con 0x7f76d8070120 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d8159040 msgr2=0x7f76d80ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d8159040 0x7f76d80ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 >> v2:172.21.15.136:3300/0 conn(0x7f76d8152d60 msgr2=0x7f76d80ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 --2- 172.21.15.113:0/1010420614 >> v2:172.21.15.136:3300/0 conn(0x7f76d8152d60 0x7f76d80ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d80c36e0 con 0x7f76d8070120 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f76c4001680 con 0x7f76d8070120 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 >> v1:172.21.15.113:6789/0 conn(0x7f76d8070120 legacy=0x7f76d8158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c00039c0 0x7f76c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:05.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c0004810 0x7f76c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76c00051d0 0x7f76c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80c3440 con 0x7f76c0004810 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d806f620 con 0x7f76c00039c0 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c000bf10 con 0x7f76c00051d0 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76dce00640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c0004810 0x7f76c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76de200640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76c00051d0 0x7f76c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76dd800640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c00039c0 0x7f76c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76dce00640 1 -- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76c00051d0 msgr2=0x7f76c000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76dce00640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76c00051d0 0x7f76c000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76dce00640 1 -- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c00039c0 msgr2=0x7f76c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76dce00640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c00039c0 0x7f76c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:05.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.909+0000 7f76dce00640 1 -- 172.21.15.113:0/1010420614 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c0003530 con 0x7f76c0004810 2024-03-04T08:16:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76dce00640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c0004810 0x7f76c0004c90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f76c400c160 tx=0x7f76c401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27d89f755e64c52 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4007c90 con 0x7f76c0004810 2024-03-04T08:16:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c40a9020 con 0x7f76c0004810 2024-03-04T08:16:05.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76d7e00640 1 -- 172.21.15.113:0/1010420614 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4005ae0 con 0x7f76c0004810 2024-03-04T08:16:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76e0200640 1 -- 172.21.15.113:0/1010420614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c0004810 msgr2=0x7f76c0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76e0200640 1 --2- 172.21.15.113:0/1010420614 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c0004810 0x7f76c0004c90 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f76c400c160 tx=0x7f76c401cbe0 comp rx=0 tx=0).stop 2024-03-04T08:16:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76de200640 1 -- 172.21.15.113:0/1010420614 reap_dead start 2024-03-04T08:16:05.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76e0200640 1 -- 172.21.15.113:0/1010420614 shutdown_connections 2024-03-04T08:16:05.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76e0200640 1 -- 172.21.15.113:0/1010420614 >> 172.21.15.113:0/1010420614 conn(0x7f76d8066db0 msgr2=0x7f76d80671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:05.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76e0200640 1 -- 172.21.15.113:0/1010420614 shutdown_connections 2024-03-04T08:16:05.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76e0200640 1 -- 172.21.15.113:0/1010420614 wait complete. 2024-03-04T08:16:05.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76e0200640 1 Processor -- start 2024-03-04T08:16:05.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.913+0000 7f76e0200640 1 -- start start 2024-03-04T08:16:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76e0200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0004810 0x7f76d80b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76e0200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d80b3650 0x7f76d814cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76e0200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d80b1cf0 0x7f76d814d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76e0200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80c39c0 con 0x7f76d80b1cf0 2024-03-04T08:16:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76e0200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80c5020 con 0x7f76c0004810 2024-03-04T08:16:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76e0200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80c3b40 con 0x7f76d80b3650 2024-03-04T08:16:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76de200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d80b1cf0 0x7f76d814d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76de200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d80b1cf0 0x7f76d814d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34940/0 (socket says 172.21.15.113:34940) 2024-03-04T08:16:05.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76de200640 1 -- 172.21.15.113:0/3851566474 learned_addr learned my addr 172.21.15.113:0/3851566474 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:05.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76dce00640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d80b3650 0x7f76d814cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:05.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76dd800640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0004810 0x7f76d80b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76de200640 1 -- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d80b3650 msgr2=0x7f76d814cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76de200640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d80b3650 0x7f76d814cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76de200640 1 -- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0004810 msgr2=0x7f76d80b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76de200640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0004810 0x7f76d80b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76de200640 1 -- 172.21.15.113:0/3851566474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d81f00c0 con 0x7f76d80b1cf0 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76de200640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d80b1cf0 0x7f76d814d280 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f76d0007cf0 tx=0x7f76d00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76d5600640 1 -- 172.21.15.113:0/3851566474 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0011de0 con 0x7f76d80b1cf0 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76d5600640 1 -- 172.21.15.113:0/3851566474 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d001b070 con 0x7f76d80b1cf0 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76d5600640 1 -- 172.21.15.113:0/3851566474 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d00039a0 con 0x7f76d80b1cf0 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76e0200640 1 -- 172.21.15.113:0/3851566474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d81f03a0 con 0x7f76d80b1cf0 2024-03-04T08:16:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76e0200640 1 -- 172.21.15.113:0/3851566474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d81f0960 con 0x7f76d80b1cf0 2024-03-04T08:16:05.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76e0200640 1 -- 172.21.15.113:0/3851566474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d8075c00 con 0x7f76d80b1cf0 2024-03-04T08:16:05.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.917+0000 7f76d5600640 1 -- 172.21.15.113:0/3851566474 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76d0003b40 con 0x7f76d80b1cf0 2024-03-04T08:16:05.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.921+0000 7f76d5600640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76b8066f60 0x7f76b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:05.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.921+0000 7f76d5600640 1 -- 172.21.15.113:0/3851566474 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f76d0085da0 con 0x7f76d80b1cf0 2024-03-04T08:16:05.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.921+0000 7f76dd800640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76b8066f60 0x7f76b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:05.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.921+0000 7f76dd800640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76b8066f60 0x7f76b8069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009ae0 tx=0x7f76cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:05.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:05.925+0000 7f76d5600640 1 -- 172.21.15.113:0/3851566474 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f76d0055f20 con 0x7f76d80b1cf0 2024-03-04T08:16:06.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.201+0000 7f76e0200640 1 -- 172.21.15.113:0/3851566474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76d806ff00 con 0x7f76d80b1cf0 2024-03-04T08:16:06.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.201+0000 7f76d5600640 1 -- 172.21.15.113:0/3851566474 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5825 (secure 0 0 0) 0x7f76d0059fc0 con 0x7f76d80b1cf0 2024-03-04T08:16:06.202 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:06.202 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:05.952545+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:resolve","state_seq":28,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692]},"id":1}]} 2024-03-04T08:16:06.202 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 152 2024-03-04T08:16:06.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 -- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76b8066f60 msgr2=0x7f76b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:06.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76b8066f60 0x7f76b8069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f76cc009ae0 tx=0x7f76cc008040 comp rx=0 tx=0).stop 2024-03-04T08:16:06.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 -- 172.21.15.113:0/3851566474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d80b1cf0 msgr2=0x7f76d814d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:06.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d80b1cf0 0x7f76d814d280 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f76d0007cf0 tx=0x7f76d00115a0 comp rx=0 tx=0).stop 2024-03-04T08:16:06.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 -- 172.21.15.113:0/3851566474 shutdown_connections 2024-03-04T08:16:06.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f76b8066f60 0x7f76b8069420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:06.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d80b1cf0 0x7f76d814d280 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:06.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76d80b3650 0x7f76d814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:06.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 --2- 172.21.15.113:0/3851566474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76c0004810 0x7f76d80b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:06.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 -- 172.21.15.113:0/3851566474 >> 172.21.15.113:0/3851566474 conn(0x7f76d8066db0 msgr2=0x7f76d80cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:06.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 -- 172.21.15.113:0/3851566474 shutdown_connections 2024-03-04T08:16:06.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:06.205+0000 7f76e0200640 1 -- 172.21.15.113:0/3851566474 wait complete. 2024-03-04T08:16:06.223 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:16:07.866 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:16:07.870 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:16:08.224 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:08.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.389+0000 7f51fbe00640 1 Processor -- start 2024-03-04T08:16:08.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.389+0000 7f51fbe00640 1 -- start start 2024-03-04T08:16:08.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51fbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f51fc152ad0 0x7f51fc0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:08.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc066fb0 0x7f51fc0cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:08.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51fbe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51fc0c65b0 con 0x7f51fc1526f0 2024-03-04T08:16:08.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51fbe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc0c9530 con 0x7f51fc152ad0 2024-03-04T08:16:08.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51fbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0c8c80 con 0x7f51fc066fb0 2024-03-04T08:16:08.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51faa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f51fc1526f0 0x7f51fc0c8e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50914/0 (socket says 172.21.15.113:50914) 2024-03-04T08:16:08.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51faa00640 1 -- 172.21.15.113:0/2443318261 learned_addr learned my addr 172.21.15.113:0/2443318261 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:16:08.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51fb400640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc066fb0 0x7f51fc0cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:08.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51fa000640 1 --2- 172.21.15.113:0/2443318261 >> v2:172.21.15.136:3300/0 conn(0x7f51fc152ad0 0x7f51fc0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:08.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2905977400 0 0) 0x7f51fc0c65b0 con 0x7f51fc1526f0 2024-03-04T08:16:08.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e0003530 con 0x7f51fc1526f0 2024-03-04T08:16:08.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3862063715 0 0) 0x7f51e0003530 con 0x7f51fc1526f0 2024-03-04T08:16:08.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc066fb0 msgr2=0x7f51fc0cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:08.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc066fb0 0x7f51fc0cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 >> v2:172.21.15.136:3300/0 conn(0x7f51fc152ad0 msgr2=0x7f51fc0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:08.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 --2- 172.21.15.113:0/2443318261 >> v2:172.21.15.136:3300/0 conn(0x7f51fc152ad0 0x7f51fc0c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc0c6890 con 0x7f51fc1526f0 2024-03-04T08:16:08.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f51ec001680 con 0x7f51fc1526f0 2024-03-04T08:16:08.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 >> v1:172.21.15.113:6789/0 conn(0x7f51fc1526f0 legacy=0x7f51fc0c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:08.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e00039c0 0x7f51e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:08.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc066fb0 0x7f51e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:08.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51e0004d50 0x7f51e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:08.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0c9530 con 0x7f51e00039c0 2024-03-04T08:16:08.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0c8c80 con 0x7f51e0004d50 2024-03-04T08:16:08.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e000b9d0 con 0x7f51fc066fb0 2024-03-04T08:16:08.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51faa00640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e00039c0 0x7f51e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:08.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51fa000640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc066fb0 0x7f51e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:08.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51fb400640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51e0004d50 0x7f51e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:08.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51faa00640 1 -- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc066fb0 msgr2=0x7f51e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:08.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51faa00640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc066fb0 0x7f51e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51faa00640 1 -- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51e0004d50 msgr2=0x7f51e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:08.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51faa00640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51e0004d50 0x7f51e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.393+0000 7f51faa00640 1 -- 172.21.15.113:0/2443318261 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e0003530 con 0x7f51e00039c0 2024-03-04T08:16:08.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51faa00640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e00039c0 0x7f51e0003de0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c210 tx=0x7f51ec01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ca7a84573a3f36 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:08.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec007d60 con 0x7f51e00039c0 2024-03-04T08:16:08.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec005ae0 con 0x7f51e00039c0 2024-03-04T08:16:08.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51f9600640 1 -- 172.21.15.113:0/2443318261 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec007070 con 0x7f51e00039c0 2024-03-04T08:16:08.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51fbe00640 1 -- 172.21.15.113:0/2443318261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e00039c0 msgr2=0x7f51e0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:08.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51fbe00640 1 --2- 172.21.15.113:0/2443318261 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51e00039c0 0x7f51e0003de0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c210 tx=0x7f51ec01f3f0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51fb400640 1 -- 172.21.15.113:0/2443318261 reap_dead start 2024-03-04T08:16:08.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51fbe00640 1 -- 172.21.15.113:0/2443318261 shutdown_connections 2024-03-04T08:16:08.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51fbe00640 1 -- 172.21.15.113:0/2443318261 >> 172.21.15.113:0/2443318261 conn(0x7f51fc06b060 msgr2=0x7f51fc06b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:08.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51fbe00640 1 -- 172.21.15.113:0/2443318261 shutdown_connections 2024-03-04T08:16:08.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.397+0000 7f51fbe00640 1 -- 172.21.15.113:0/2443318261 wait complete. 2024-03-04T08:16:08.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fbe00640 1 Processor -- start 2024-03-04T08:16:08.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fbe00640 1 -- start start 2024-03-04T08:16:08.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51e00039c0 0x7f51fc1eea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:08.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc14de00 0x7f51fc1e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc1e3660 0x7f51fc15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc158ed0 con 0x7f51fc1e3660 2024-03-04T08:16:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fbe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc158bf0 con 0x7f51e00039c0 2024-03-04T08:16:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc1585c0 con 0x7f51fc14de00 2024-03-04T08:16:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc1e3660 0x7f51fc15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:08.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc1e3660 0x7f51fc15b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46824/0 (socket says 172.21.15.113:46824) 2024-03-04T08:16:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fb400640 1 -- 172.21.15.113:0/1108648621 learned_addr learned my addr 172.21.15.113:0/1108648621 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fa000640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc14de00 0x7f51fc1e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51faa00640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51e00039c0 0x7f51fc1eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fb400640 1 -- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc14de00 msgr2=0x7f51fc1e23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fb400640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc14de00 0x7f51fc1e23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fb400640 1 -- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51e00039c0 msgr2=0x7f51fc1eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fb400640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51e00039c0 0x7f51fc1eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fb400640 1 -- 172.21.15.113:0/1108648621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc1559b0 con 0x7f51fc1e3660 2024-03-04T08:16:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51fb400640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc1e3660 0x7f51fc15b760 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0ca0d0 tx=0x7f51f400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.401+0000 7f51f2a00640 1 -- 172.21.15.113:0/1108648621 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f400fd80 con 0x7f51fc1e3660 2024-03-04T08:16:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.405+0000 7f51f2a00640 1 -- 172.21.15.113:0/1108648621 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f4015430 con 0x7f51fc1e3660 2024-03-04T08:16:08.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.405+0000 7f51f2a00640 1 -- 172.21.15.113:0/1108648621 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4015820 con 0x7f51fc1e3660 2024-03-04T08:16:08.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.405+0000 7f51fbe00640 1 -- 172.21.15.113:0/1108648621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc155c90 con 0x7f51fc1e3660 2024-03-04T08:16:08.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.405+0000 7f51fbe00640 1 -- 172.21.15.113:0/1108648621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc1ee270 con 0x7f51fc1e3660 2024-03-04T08:16:08.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.405+0000 7f51fbe00640 1 -- 172.21.15.113:0/1108648621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc075a80 con 0x7f51fc1e3660 2024-03-04T08:16:08.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.405+0000 7f51f2a00640 1 -- 172.21.15.113:0/1108648621 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51f40159c0 con 0x7f51fc1e3660 2024-03-04T08:16:08.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.405+0000 7f51f2a00640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f51dc066f60 0x7f51dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:08.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.405+0000 7f51f2a00640 1 -- 172.21.15.113:0/1108648621 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f51f40862c0 con 0x7f51fc1e3660 2024-03-04T08:16:08.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.409+0000 7f51faa00640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f51dc066f60 0x7f51dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:08.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.409+0000 7f51faa00640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f51dc066f60 0x7f51dc069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0ae000 tx=0x7f51ec000ff0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:08.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.413+0000 7f51f2a00640 1 -- 172.21.15.113:0/1108648621 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f51f4024030 con 0x7f51fc1e3660 2024-03-04T08:16:08.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 -- 172.21.15.113:0/1108648621 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51fc0c4e80 con 0x7f51fc1e3660 2024-03-04T08:16:08.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51f2a00640 1 -- 172.21.15.113:0/1108648621 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5825 (secure 0 0 0) 0x7f51f4019030 con 0x7f51fc1e3660 2024-03-04T08:16:08.614 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:08.614 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:05.952545+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:resolve","state_seq":28,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692]},"id":1}]} 2024-03-04T08:16:08.616 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 152 2024-03-04T08:16:08.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 -- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f51dc066f60 msgr2=0x7f51dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:08.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f51dc066f60 0x7f51dc069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0ae000 tx=0x7f51ec000ff0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 -- 172.21.15.113:0/1108648621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc1e3660 msgr2=0x7f51fc15b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:08.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc1e3660 0x7f51fc15b760 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f51fc0ca0d0 tx=0x7f51f400f580 comp rx=0 tx=0).stop 2024-03-04T08:16:08.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 -- 172.21.15.113:0/1108648621 shutdown_connections 2024-03-04T08:16:08.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f51dc066f60 0x7f51dc069420 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51fc1e3660 0x7f51fc15b760 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51fc14de00 0x7f51fc1e23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 --2- 172.21.15.113:0/1108648621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51e00039c0 0x7f51fc1eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:08.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 -- 172.21.15.113:0/1108648621 >> 172.21.15.113:0/1108648621 conn(0x7f51fc06b060 msgr2=0x7f51fc0c8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:08.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.613+0000 7f51fbe00640 1 -- 172.21.15.113:0/1108648621 shutdown_connections 2024-03-04T08:16:08.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:08.617+0000 7f51fbe00640 1 -- 172.21.15.113:0/1108648621 wait complete. 2024-03-04T08:16:08.633 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:16:10.634 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:10.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff857600640 1 Processor -- start 2024-03-04T08:16:10.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff857600640 1 -- start start 2024-03-04T08:16:10.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff857600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 0x7ff8500ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:10.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff857600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff850159040 0x7ff8500ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:10.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff857600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8500c5380 con 0x7ff850070120 2024-03-04T08:16:10.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff857600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8500c3440 con 0x7ff850159040 2024-03-04T08:16:10.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff857600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85006f620 con 0x7ff850152d60 2024-03-04T08:16:10.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff854c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 0x7ff8500ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:10.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff855600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff850159040 0x7ff8500ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:10.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff84fe00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7ff850070120 0x7ff850158930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:42256/0 (socket says 172.21.15.113:42256) 2024-03-04T08:16:10.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff84fe00640 1 -- 172.21.15.113:0/2917456234 learned_addr learned my addr 172.21.15.113:0/2917456234 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:16:10.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff855600640 1 -- 172.21.15.113:0/2917456234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 msgr2=0x7ff8500ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:10.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff855600640 1 --2- 172.21.15.113:0/2917456234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 0x7ff8500ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:10.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff855600640 1 -- 172.21.15.113:0/2917456234 >> v1:172.21.15.136:6789/0 conn(0x7ff850070120 legacy=0x7ff850158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:10.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff855600640 1 -- 172.21.15.113:0/2917456234 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8500c36e0 con 0x7ff850159040 2024-03-04T08:16:10.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.777+0000 7ff854c00640 1 --2- 172.21.15.113:0/2917456234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 0x7ff8500ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:16:10.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.781+0000 7ff855600640 1 --2- 172.21.15.113:0/2917456234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff850159040 0x7ff8500ce850 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff848009d90 tx=0x7ff84801f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1436e0d1ece8c917 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:10.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff84f400640 1 -- 172.21.15.113:0/2917456234 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8480b4070 con 0x7ff850159040 2024-03-04T08:16:10.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff84f400640 1 -- 172.21.15.113:0/2917456234 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8480af600 con 0x7ff850159040 2024-03-04T08:16:10.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff84f400640 1 -- 172.21.15.113:0/2917456234 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8480af9d0 con 0x7ff850159040 2024-03-04T08:16:10.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff857600640 1 -- 172.21.15.113:0/2917456234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff850159040 msgr2=0x7ff8500ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:10.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff857600640 1 --2- 172.21.15.113:0/2917456234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff850159040 0x7ff8500ce850 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff848009d90 tx=0x7ff84801f150 comp rx=0 tx=0).stop 2024-03-04T08:16:10.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff857600640 1 -- 172.21.15.113:0/2917456234 shutdown_connections 2024-03-04T08:16:10.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff857600640 1 --2- 172.21.15.113:0/2917456234 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff850159040 0x7ff8500ce850 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:10.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff857600640 1 --2- 172.21.15.113:0/2917456234 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 0x7ff8500ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:10.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff857600640 1 -- 172.21.15.113:0/2917456234 >> 172.21.15.113:0/2917456234 conn(0x7ff850066db0 msgr2=0x7ff8500671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:10.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff857600640 1 -- 172.21.15.113:0/2917456234 shutdown_connections 2024-03-04T08:16:10.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff857600640 1 -- 172.21.15.113:0/2917456234 wait complete. 2024-03-04T08:16:10.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff857600640 1 Processor -- start 2024-03-04T08:16:10.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.785+0000 7ff857600640 1 -- start start 2024-03-04T08:16:10.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff857600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850070120 0x7ff8500af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:10.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff857600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 0x7ff850151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:10.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff857600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8500af7a0 0x7ff850151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:10.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff857600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500c39c0 con 0x7ff850070120 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff857600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500c5020 con 0x7ff8500af7a0 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff857600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500c3b40 con 0x7ff850152d60 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff854c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850070120 0x7ff8500af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff854c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850070120 0x7ff8500af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46838/0 (socket says 172.21.15.113:46838) 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff854c00640 1 -- 172.21.15.113:0/1367164708 learned_addr learned my addr 172.21.15.113:0/1367164708 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff84fe00640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 0x7ff850151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff855600640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8500af7a0 0x7ff850151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff854c00640 1 -- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 msgr2=0x7ff850151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff854c00640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 0x7ff850151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff854c00640 1 -- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8500af7a0 msgr2=0x7ff850151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff854c00640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8500af7a0 0x7ff850151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff854c00640 1 -- 172.21.15.113:0/1367164708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8501f0690 con 0x7ff850070120 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff854c00640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850070120 0x7ff8500af160 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff84400ec10 tx=0x7ff84400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff84cc00640 1 -- 172.21.15.113:0/1367164708 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844017070 con 0x7ff850070120 2024-03-04T08:16:10.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff84cc00640 1 -- 172.21.15.113:0/1367164708 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84400cc30 con 0x7ff850070120 2024-03-04T08:16:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff84cc00640 1 -- 172.21.15.113:0/1367164708 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844013470 con 0x7ff850070120 2024-03-04T08:16:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff857600640 1 -- 172.21.15.113:0/1367164708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8501f4b00 con 0x7ff850070120 2024-03-04T08:16:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.789+0000 7ff857600640 1 -- 172.21.15.113:0/1367164708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8500b03c0 con 0x7ff850070120 2024-03-04T08:16:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.793+0000 7ff857600640 1 -- 172.21.15.113:0/1367164708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff850075c00 con 0x7ff850070120 2024-03-04T08:16:10.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.793+0000 7ff84cc00640 1 -- 172.21.15.113:0/1367164708 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff84401a070 con 0x7ff850070120 2024-03-04T08:16:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.793+0000 7ff84cc00640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff830066f90 0x7ff830069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.793+0000 7ff84cc00640 1 -- 172.21.15.113:0/1367164708 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7ff8440857b0 con 0x7ff850070120 2024-03-04T08:16:10.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.793+0000 7ff84fe00640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff830066f90 0x7ff830069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:10.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.793+0000 7ff84fe00640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff830066f90 0x7ff830069450 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff8500ce160 tx=0x7ff83c007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:10.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:10.801+0000 7ff84cc00640 1 -- 172.21.15.113:0/1367164708 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff8501f0690 con 0x7ff850070120 2024-03-04T08:16:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 -- 172.21.15.113:0/1367164708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff850152600 con 0x7ff850070120 2024-03-04T08:16:11.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff84cc00640 1 -- 172.21.15.113:0/1367164708 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5825 (secure 0 0 0) 0x7ff8440558a0 con 0x7ff850070120 2024-03-04T08:16:11.021 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:11.021 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:05.952545+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:resolve","state_seq":28,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692]},"id":1}]} 2024-03-04T08:16:11.021 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 152 2024-03-04T08:16:11.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 -- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff830066f90 msgr2=0x7ff830069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff830066f90 0x7ff830069450 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff8500ce160 tx=0x7ff83c007e90 comp rx=0 tx=0).stop 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 -- 172.21.15.113:0/1367164708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850070120 msgr2=0x7ff8500af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850070120 0x7ff8500af160 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff84400ec10 tx=0x7ff84400c520 comp rx=0 tx=0).stop 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 -- 172.21.15.113:0/1367164708 shutdown_connections 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8500af7a0 0x7ff850151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff850152d60 0x7ff850151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff830066f90 0x7ff830069450 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 --2- 172.21.15.113:0/1367164708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff850070120 0x7ff8500af160 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 -- 172.21.15.113:0/1367164708 >> 172.21.15.113:0/1367164708 conn(0x7ff850066db0 msgr2=0x7ff8500c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 -- 172.21.15.113:0/1367164708 shutdown_connections 2024-03-04T08:16:11.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:11.021+0000 7ff857600640 1 -- 172.21.15.113:0/1367164708 wait complete. 2024-03-04T08:16:11.041 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:16:11.476 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:11.477 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.46 2024-03-04T08:16:11.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 46 -eq 48 ] 2024-03-04T08:16:11.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.46 2024-03-04T08:16:11.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.46 2024-03-04T08:16:11.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.46 2024-03-04T08:16:11.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:13.042 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:13.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.181+0000 7f858de00640 1 Processor -- start 2024-03-04T08:16:13.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.181+0000 7f858de00640 1 -- start start 2024-03-04T08:16:13.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.181+0000 7f858de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f858814e4c0 0x7f8588072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:13.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.181+0000 7f858de00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f85880c22e0 0x7f85880731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:13.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.181+0000 7f858de00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85880c27f0 con 0x7f85880c2d80 2024-03-04T08:16:13.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.181+0000 7f858de00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858815ad60 con 0x7f858814e4c0 2024-03-04T08:16:13.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.181+0000 7f858de00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85880c2a90 con 0x7f85880c22e0 2024-03-04T08:16:13.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f858814e4c0 0x7f8588072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:13.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f858814e4c0 0x7f8588072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46844/0 (socket says 172.21.15.113:46844) 2024-03-04T08:16:13.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586a00640 1 -- 172.21.15.113:0/3827697495 learned_addr learned my addr 172.21.15.113:0/3827697495 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:13.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8587400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f85880c2d80 0x7f858815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50930/0 (socket says 172.21.15.113:50930) 2024-03-04T08:16:13.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f857fe00640 1 --2- 172.21.15.113:0/3827697495 >> v2:172.21.15.136:3300/0 conn(0x7f85880c22e0 0x7f85880731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:13.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586a00640 1 -- 172.21.15.113:0/3827697495 >> v2:172.21.15.136:3300/0 conn(0x7f85880c22e0 msgr2=0x7f85880731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:13.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586a00640 1 --2- 172.21.15.113:0/3827697495 >> v2:172.21.15.136:3300/0 conn(0x7f85880c22e0 0x7f85880731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:13.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586a00640 1 -- 172.21.15.113:0/3827697495 >> v1:172.21.15.113:6789/0 conn(0x7f85880c2d80 legacy=0x7f858815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:13.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586a00640 1 -- 172.21.15.113:0/3827697495 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588070c60 con 0x7f858814e4c0 2024-03-04T08:16:13.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586a00640 1 --2- 172.21.15.113:0/3827697495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 0x7f8588072c80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f85880cd230 tx=0x7f857801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a306e7e3ab1b12 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:13.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586000640 1 -- 172.21.15.113:0/3827697495 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85780b5070 con 0x7f858814e4c0 2024-03-04T08:16:13.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586000640 1 -- 172.21.15.113:0/3827697495 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85780a8a10 con 0x7f858814e4c0 2024-03-04T08:16:13.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.185+0000 7f8586000640 1 -- 172.21.15.113:0/3827697495 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85780a8de0 con 0x7f858814e4c0 2024-03-04T08:16:13.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 -- 172.21.15.113:0/3827697495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 msgr2=0x7f8588072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:13.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 --2- 172.21.15.113:0/3827697495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 0x7f8588072c80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f85880cd230 tx=0x7f857801ed10 comp rx=0 tx=0).stop 2024-03-04T08:16:13.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 -- 172.21.15.113:0/3827697495 shutdown_connections 2024-03-04T08:16:13.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 --2- 172.21.15.113:0/3827697495 >> v2:172.21.15.136:3300/0 conn(0x7f85880c22e0 0x7f85880731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:13.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 --2- 172.21.15.113:0/3827697495 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 0x7f8588072c80 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:13.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 -- 172.21.15.113:0/3827697495 >> 172.21.15.113:0/3827697495 conn(0x7f858806e820 msgr2=0x7f858806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:13.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 -- 172.21.15.113:0/3827697495 shutdown_connections 2024-03-04T08:16:13.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 -- 172.21.15.113:0/3827697495 wait complete. 2024-03-04T08:16:13.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 Processor -- start 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 -- start start 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85880c22e0 0x7f85881f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85880c2d80 0x7f85881450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 0x7f858815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588071500 con 0x7f858814e4c0 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588071220 con 0x7f85880c22e0 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f858de00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858814e8a0 con 0x7f85880c2d80 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f8587400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 0x7f858815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f8587400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 0x7f858815a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46854/0 (socket says 172.21.15.113:46854) 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f8587400640 1 -- 172.21.15.113:0/3213260637 learned_addr learned my addr 172.21.15.113:0/3213260637 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f857fe00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85880c2d80 0x7f85881450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f8586a00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85880c22e0 0x7f85881f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f8587400640 1 -- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85880c2d80 msgr2=0x7f85881450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:13.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f8587400640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85880c2d80 0x7f85881450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:13.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f8587400640 1 -- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85880c22e0 msgr2=0x7f85881f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:13.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f8587400640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85880c22e0 0x7f85881f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:13.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f8587400640 1 -- 172.21.15.113:0/3213260637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85880cb5d0 con 0x7f858814e4c0 2024-03-04T08:16:13.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f857fe00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85880c2d80 0x7f85881450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:16:13.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.189+0000 7f8586a00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85880c22e0 0x7f85881f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:16:13.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.193+0000 7f8587400640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 0x7f858815a050 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f858000c730 tx=0x7f858000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:13.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.193+0000 7f857f400640 1 -- 172.21.15.113:0/3213260637 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580017070 con 0x7f858814e4c0 2024-03-04T08:16:13.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.193+0000 7f857f400640 1 -- 172.21.15.113:0/3213260637 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858000ce40 con 0x7f858814e4c0 2024-03-04T08:16:13.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.193+0000 7f857f400640 1 -- 172.21.15.113:0/3213260637 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858000a840 con 0x7f858814e4c0 2024-03-04T08:16:13.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.193+0000 7f858de00640 1 -- 172.21.15.113:0/3213260637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85880cb8b0 con 0x7f858814e4c0 2024-03-04T08:16:13.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.193+0000 7f858de00640 1 -- 172.21.15.113:0/3213260637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85881f4650 con 0x7f858814e4c0 2024-03-04T08:16:13.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.193+0000 7f858de00640 1 -- 172.21.15.113:0/3213260637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8588075c00 con 0x7f858814e4c0 2024-03-04T08:16:13.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.193+0000 7f857f400640 1 -- 172.21.15.113:0/3213260637 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f858001a070 con 0x7f858814e4c0 2024-03-04T08:16:13.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.197+0000 7f857f400640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8564066f90 0x7f8564069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:13.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.197+0000 7f857f400640 1 -- 172.21.15.113:0/3213260637 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f8580084da0 con 0x7f858814e4c0 2024-03-04T08:16:13.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.197+0000 7f8586a00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8564066f90 0x7f8564069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:13.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.197+0000 7f8586a00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8564066f90 0x7f8564069450 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f85780b2000 tx=0x7f8578008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:13.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.201+0000 7f857f400640 1 -- 172.21.15.113:0/3213260637 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8580054e90 con 0x7f858814e4c0 2024-03-04T08:16:13.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.461+0000 7f858de00640 1 -- 172.21.15.113:0/3213260637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8588071a60 con 0x7f858814e4c0 2024-03-04T08:16:13.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.461+0000 7f857f400640 1 -- 172.21.15.113:0/3213260637 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+5827 (secure 0 0 0) 0x7f8580082e20 con 0x7f858814e4c0 2024-03-04T08:16:13.463 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:13.463 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":153,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:11.007726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:reconnect","state_seq":30,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839]},"id":1}]} 2024-03-04T08:16:13.463 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 153 2024-03-04T08:16:13.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 -- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8564066f90 msgr2=0x7f8564069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:13.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8564066f90 0x7f8564069450 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f85780b2000 tx=0x7f8578008040 comp rx=0 tx=0).stop 2024-03-04T08:16:13.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 -- 172.21.15.113:0/3213260637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 msgr2=0x7f858815a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:13.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 0x7f858815a050 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f858000c730 tx=0x7f858000cbf0 comp rx=0 tx=0).stop 2024-03-04T08:16:13.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 -- 172.21.15.113:0/3213260637 shutdown_connections 2024-03-04T08:16:13.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8564066f90 0x7f8564069450 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:13.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858814e4c0 0x7f858815a050 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:13.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85880c2d80 0x7f85881450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:13.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 --2- 172.21.15.113:0/3213260637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85880c22e0 0x7f85881f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:13.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 -- 172.21.15.113:0/3213260637 >> 172.21.15.113:0/3213260637 conn(0x7f858806e820 msgr2=0x7f858814d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:13.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 -- 172.21.15.113:0/3213260637 shutdown_connections 2024-03-04T08:16:13.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:13.465+0000 7f858de00640 1 -- 172.21.15.113:0/3213260637 wait complete. 2024-03-04T08:16:13.483 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:16:14.464 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:14.464 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.47 2024-03-04T08:16:14.465 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 47 -eq 48 ] 2024-03-04T08:16:14.465 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.47 2024-03-04T08:16:14.465 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.47 2024-03-04T08:16:14.465 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.47 2024-03-04T08:16:14.465 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:15.484 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:15.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada7a00640 1 Processor -- start 2024-03-04T08:16:15.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada7a00640 1 -- start start 2024-03-04T08:16:15.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada7a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0152d60 0x7fada00ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:15.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada7a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fada0070120 0x7fada00ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:15.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada7a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fada00c9db0 0x7fada0158930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:15.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada7a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada00c5380 con 0x7fada00c9db0 2024-03-04T08:16:15.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada7a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada00c3440 con 0x7fada0070120 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada7a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada006f620 con 0x7fada0152d60 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada5a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fada00c9db0 0x7fada0158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada5a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fada00c9db0 0x7fada0158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46862/0 (socket says 172.21.15.113:46862) 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada5a00640 1 -- 172.21.15.113:0/1924343669 learned_addr learned my addr 172.21.15.113:0/1924343669 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fad9fe00640 1 --2- 172.21.15.113:0/1924343669 >> v2:172.21.15.136:3300/0 conn(0x7fada0070120 0x7fada00ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada5000640 1 --2- 172.21.15.113:0/1924343669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0152d60 0x7fada00ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada5a00640 1 -- 172.21.15.113:0/1924343669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0152d60 msgr2=0x7fada00ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada5a00640 1 --2- 172.21.15.113:0/1924343669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0152d60 0x7fada00ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada5a00640 1 -- 172.21.15.113:0/1924343669 >> v2:172.21.15.136:3300/0 conn(0x7fada0070120 msgr2=0x7fada00ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada5a00640 1 --2- 172.21.15.113:0/1924343669 >> v2:172.21.15.136:3300/0 conn(0x7fada0070120 0x7fada00ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fada5a00640 1 -- 172.21.15.113:0/1924343669 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada00c36e0 con 0x7fada00c9db0 2024-03-04T08:16:15.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.625+0000 7fad9fe00640 1 --2- 172.21.15.113:0/1924343669 >> v2:172.21.15.136:3300/0 conn(0x7fada0070120 0x7fada00ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:16:15.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fada5a00640 1 --2- 172.21.15.113:0/1924343669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada00c9db0 0x7fada0158930 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fada0070580 tx=0x7fad9801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e63ec496ace57c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:15.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fad9f400640 1 -- 172.21.15.113:0/1924343669 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad980b4070 con 0x7fada00c9db0 2024-03-04T08:16:15.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fad9f400640 1 -- 172.21.15.113:0/1924343669 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad980af400 con 0x7fada00c9db0 2024-03-04T08:16:15.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fad9f400640 1 -- 172.21.15.113:0/1924343669 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad980af7d0 con 0x7fada00c9db0 2024-03-04T08:16:15.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fada7a00640 1 -- 172.21.15.113:0/1924343669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada00c9db0 msgr2=0x7fada0158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:15.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fada7a00640 1 --2- 172.21.15.113:0/1924343669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada00c9db0 0x7fada0158930 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fada0070580 tx=0x7fad9801f1e0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fada7a00640 1 -- 172.21.15.113:0/1924343669 shutdown_connections 2024-03-04T08:16:15.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fada7a00640 1 --2- 172.21.15.113:0/1924343669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada00c9db0 0x7fada0158930 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fada7a00640 1 --2- 172.21.15.113:0/1924343669 >> v2:172.21.15.136:3300/0 conn(0x7fada0070120 0x7fada00ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fada7a00640 1 --2- 172.21.15.113:0/1924343669 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0152d60 0x7fada00ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fada7a00640 1 -- 172.21.15.113:0/1924343669 >> 172.21.15.113:0/1924343669 conn(0x7fada0066db0 msgr2=0x7fada00671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:15.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fada7a00640 1 -- 172.21.15.113:0/1924343669 shutdown_connections 2024-03-04T08:16:15.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.629+0000 7fada7a00640 1 -- 172.21.15.113:0/1924343669 wait complete. 2024-03-04T08:16:15.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada7a00640 1 Processor -- start 2024-03-04T08:16:15.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada7a00640 1 -- start start 2024-03-04T08:16:15.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada7a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0070120 0x7fada015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:15.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada7a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada00c9db0 0x7fada01e4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:15.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada7a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada0152d60 0x7fada0151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:15.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada7a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada00c39c0 con 0x7fada0152d60 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada7a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada00c5020 con 0x7fada00c9db0 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada7a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada00c3b40 con 0x7fada0070120 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada0152d60 0x7fada0151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0070120 0x7fada015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada0152d60 0x7fada0151580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46864/0 (socket says 172.21.15.113:46864) 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0070120 0x7fada015c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:44958/0 (socket says 172.21.15.113:44958) 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5a00640 1 -- 172.21.15.113:0/1063187897 learned_addr learned my addr 172.21.15.113:0/1063187897 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fad9fe00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada00c9db0 0x7fada01e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5a00640 1 -- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0070120 msgr2=0x7fada015c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5a00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0070120 0x7fada015c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5a00640 1 -- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada00c9db0 msgr2=0x7fada01e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:15.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5a00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada00c9db0 0x7fada01e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5a00640 1 -- 172.21.15.113:0/1063187897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada01f0870 con 0x7fada0152d60 2024-03-04T08:16:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada5a00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada0152d60 0x7fada0151580 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fad980b2040 tx=0x7fad9801bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fad9cc00640 1 -- 172.21.15.113:0/1063187897 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad980b4040 con 0x7fada0152d60 2024-03-04T08:16:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fad9cc00640 1 -- 172.21.15.113:0/1063187897 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad980afcc0 con 0x7fada0152d60 2024-03-04T08:16:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada7a00640 1 -- 172.21.15.113:0/1063187897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada01f0b50 con 0x7fada0152d60 2024-03-04T08:16:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fad9cc00640 1 -- 172.21.15.113:0/1063187897 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98009510 con 0x7fada0152d60 2024-03-04T08:16:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.633+0000 7fada7a00640 1 -- 172.21.15.113:0/1063187897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada01f1110 con 0x7fada0152d60 2024-03-04T08:16:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.637+0000 7fada7a00640 1 -- 172.21.15.113:0/1063187897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada006ff00 con 0x7fada0152d60 2024-03-04T08:16:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.637+0000 7fad9cc00640 1 -- 172.21.15.113:0/1063187897 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad980b8670 con 0x7fada0152d60 2024-03-04T08:16:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.637+0000 7fad9cc00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad80066f60 0x7fad80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.637+0000 7fad9cc00640 1 -- 172.21.15.113:0/1063187897 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7fad981238f0 con 0x7fada0152d60 2024-03-04T08:16:15.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.637+0000 7fada5000640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad80066f60 0x7fad80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:15.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.637+0000 7fada5000640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad80066f60 0x7fad80069420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fad94007a60 tx=0x7fad94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:15.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.641+0000 7fad9cc00640 1 -- 172.21.15.113:0/1063187897 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fad98018d40 con 0x7fada0152d60 2024-03-04T08:16:15.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.837+0000 7fada7a00640 1 -- 172.21.15.113:0/1063187897 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fada0075be0 con 0x7fada0152d60 2024-03-04T08:16:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fad9cc00640 1 -- 172.21.15.113:0/1063187897 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+5827 (secure 0 0 0) 0x7fad980f3a70 con 0x7fada0152d60 2024-03-04T08:16:15.839 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:15.839 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":153,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:11.007726+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:reconnect","state_seq":30,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839]},"id":1}]} 2024-03-04T08:16:15.839 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 153 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 -- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad80066f60 msgr2=0x7fad80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad80066f60 0x7fad80069420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fad94007a60 tx=0x7fad94008040 comp rx=0 tx=0).stop 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 -- 172.21.15.113:0/1063187897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada0152d60 msgr2=0x7fada0151580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada0152d60 0x7fada0151580 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fad980b2040 tx=0x7fad9801bf80 comp rx=0 tx=0).stop 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 -- 172.21.15.113:0/1063187897 shutdown_connections 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad80066f60 0x7fad80069420 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fada0152d60 0x7fada0151580 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada00c9db0 0x7fada01e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 --2- 172.21.15.113:0/1063187897 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fada0070120 0x7fada015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 -- 172.21.15.113:0/1063187897 >> 172.21.15.113:0/1063187897 conn(0x7fada0066db0 msgr2=0x7fada0067190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:15.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 -- 172.21.15.113:0/1063187897 shutdown_connections 2024-03-04T08:16:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:15.841+0000 7fada7a00640 1 -- 172.21.15.113:0/1063187897 wait complete. 2024-03-04T08:16:15.860 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 153, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 153, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7004": { "addr": "172.21.15.113:6836/4083765155", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 4083765155, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 4083765155, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7004, "incarnation": 151, "join_fscid": -1, "name": "a", "rank": 1, "state": "up:reconnect", "state_seq": 30 }, "gid_7692": { "addr": "172.21.15.113:6833/1928887637", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 1928887637, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 1928887637, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7692, "incarnation": 139, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 } }, "last_failure": 0, "last_failure_osd_epoch": 71, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:16:11.007726+0000", "qdb_cluster": [ 7692, 7839 ], "qdb_leader": 7839, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 7004, "mds_2": 7692 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:16:17.860 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.029+0000 7f0849800640 1 Processor -- start 2024-03-04T08:16:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.029+0000 7f0849800640 1 -- start start 2024-03-04T08:16:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0849800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0844152c60 0x7f084415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:18.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0849800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08440c7a80 0x7f08441630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0849800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0844072d80 con 0x7f084415c280 2024-03-04T08:16:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0849800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084406b060 con 0x7f08440c7a80 2024-03-04T08:16:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0849800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0844073510 con 0x7f0844152c60 2024-03-04T08:16:18.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0843400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08440c7a80 0x7f08441630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:18.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0843400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f08440c7a80 0x7f08441630d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35274/0 (socket says 172.21.15.113:35274) 2024-03-04T08:16:18.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0843400640 1 -- 172.21.15.113:0/2444543226 learned_addr learned my addr 172.21.15.113:0/2444543226 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:18.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0842a00640 1 --2- 172.21.15.113:0/2444543226 >> v2:172.21.15.136:3300/0 conn(0x7f0844152c60 0x7f084415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:18.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0843400640 1 -- 172.21.15.113:0/2444543226 >> v2:172.21.15.136:3300/0 conn(0x7f0844152c60 msgr2=0x7f084415bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:18.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0843400640 1 --2- 172.21.15.113:0/2444543226 >> v2:172.21.15.136:3300/0 conn(0x7f0844152c60 0x7f084415bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:18.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0843400640 1 -- 172.21.15.113:0/2444543226 >> v1:172.21.15.136:6789/0 conn(0x7f084415c280 legacy=0x7f08440cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:18.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.033+0000 7f0843400640 1 -- 172.21.15.113:0/2444543226 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084406b2b0 con 0x7f08440c7a80 2024-03-04T08:16:18.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0843400640 1 --2- 172.21.15.113:0/2444543226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 0x7f08441630d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f08440c4370 tx=0x7f083c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e80f74890072b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:18.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0841600640 1 -- 172.21.15.113:0/2444543226 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c0a57f0 con 0x7f08440c7a80 2024-03-04T08:16:18.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0841600640 1 -- 172.21.15.113:0/2444543226 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c0a5990 con 0x7f08440c7a80 2024-03-04T08:16:18.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0841600640 1 -- 172.21.15.113:0/2444543226 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c0a5d60 con 0x7f08440c7a80 2024-03-04T08:16:18.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0849800640 1 -- 172.21.15.113:0/2444543226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 msgr2=0x7f08441630d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:18.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0849800640 1 --2- 172.21.15.113:0/2444543226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 0x7f08441630d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f08440c4370 tx=0x7f083c01c930 comp rx=0 tx=0).stop 2024-03-04T08:16:18.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0849800640 1 -- 172.21.15.113:0/2444543226 shutdown_connections 2024-03-04T08:16:18.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0849800640 1 --2- 172.21.15.113:0/2444543226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 0x7f08441630d0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:18.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0849800640 1 --2- 172.21.15.113:0/2444543226 >> v2:172.21.15.136:3300/0 conn(0x7f0844152c60 0x7f084415bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:18.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0849800640 1 -- 172.21.15.113:0/2444543226 >> 172.21.15.113:0/2444543226 conn(0x7f08440671b0 msgr2=0x7f084406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:18.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0849800640 1 -- 172.21.15.113:0/2444543226 shutdown_connections 2024-03-04T08:16:18.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.037+0000 7f0849800640 1 -- 172.21.15.113:0/2444543226 wait complete. 2024-03-04T08:16:18.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0849800640 1 Processor -- start 2024-03-04T08:16:18.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0849800640 1 -- start start 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0849800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 0x7f0844141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0849800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0844152c60 0x7f0844141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0849800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084415c280 0x7f084415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0849800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08440cd720 con 0x7f08440c7a80 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0849800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08440731d0 con 0x7f0844152c60 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0849800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08440cd8a0 con 0x7f084415c280 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0842a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 0x7f0844141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0842a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 0x7f0844141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35278/0 (socket says 172.21.15.113:35278) 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0842a00640 1 -- 172.21.15.113:0/1362793049 learned_addr learned my addr 172.21.15.113:0/1362793049 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0842000640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0844152c60 0x7f0844141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:18.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0843400640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084415c280 0x7f084415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:18.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0842a00640 1 -- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084415c280 msgr2=0x7f084415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0842a00640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084415c280 0x7f084415a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0842a00640 1 -- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0844152c60 msgr2=0x7f0844141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0842a00640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0844152c60 0x7f0844141f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0842a00640 1 -- 172.21.15.113:0/1362793049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08440c4c20 con 0x7f08440c7a80 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.041+0000 7f0843400640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084415c280 0x7f084415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.045+0000 7f0842a00640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 0x7f0844141a30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f083400e8a0 tx=0x7f083400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.045+0000 7f083aa00640 1 -- 172.21.15.113:0/1362793049 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0834017070 con 0x7f08440c7a80 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.045+0000 7f083aa00640 1 -- 172.21.15.113:0/1362793049 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083400c940 con 0x7f08440c7a80 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.045+0000 7f083aa00640 1 -- 172.21.15.113:0/1362793049 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083400ccf0 con 0x7f08440c7a80 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.045+0000 7f0849800640 1 -- 172.21.15.113:0/1362793049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08441f1d70 con 0x7f08440c7a80 2024-03-04T08:16:18.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.045+0000 7f0849800640 1 -- 172.21.15.113:0/1362793049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0844159920 con 0x7f08440c7a80 2024-03-04T08:16:18.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.045+0000 7f0849800640 1 -- 172.21.15.113:0/1362793049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0844075c00 con 0x7f08440c7a80 2024-03-04T08:16:18.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.045+0000 7f083aa00640 1 -- 172.21.15.113:0/1362793049 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f083401a070 con 0x7f08440c7a80 2024-03-04T08:16:18.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.049+0000 7f083aa00640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066fe0 0x7f08240694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:18.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.049+0000 7f083aa00640 1 -- 172.21.15.113:0/1362793049 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f0834085840 con 0x7f08440c7a80 2024-03-04T08:16:18.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.049+0000 7f0842000640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066fe0 0x7f08240694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:18.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.049+0000 7f0842000640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066fe0 0x7f08240694a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f08440c6a50 tx=0x7f0830009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:18.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.053+0000 7f083aa00640 1 -- 172.21.15.113:0/1362793049 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0834055b40 con 0x7f08440c7a80 2024-03-04T08:16:18.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 -- 172.21.15.113:0/1362793049 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f084406d090 con 0x7f08440c7a80 2024-03-04T08:16:18.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f083aa00640 1 -- 172.21.15.113:0/1362793049 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f0834059be0 con 0x7f08440c7a80 2024-03-04T08:16:18.264 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:18.264 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":154,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:16.437460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:rejoin","state_seq":32,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692]},"id":1}]} 2024-03-04T08:16:18.264 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 154 2024-03-04T08:16:18.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 -- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066fe0 msgr2=0x7f08240694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:18.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066fe0 0x7f08240694a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f08440c6a50 tx=0x7f0830009040 comp rx=0 tx=0).stop 2024-03-04T08:16:18.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 -- 172.21.15.113:0/1362793049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 msgr2=0x7f0844141a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:18.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 0x7f0844141a30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f083400e8a0 tx=0x7f083400eca0 comp rx=0 tx=0).stop 2024-03-04T08:16:18.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 -- 172.21.15.113:0/1362793049 shutdown_connections 2024-03-04T08:16:18.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f084415c280 0x7f084415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:18.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0844152c60 0x7f0844141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:18.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0824066fe0 0x7f08240694a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:18.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 --2- 172.21.15.113:0/1362793049 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08440c7a80 0x7f0844141a30 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:18.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 -- 172.21.15.113:0/1362793049 >> 172.21.15.113:0/1362793049 conn(0x7f08440671b0 msgr2=0x7f0844071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:18.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 -- 172.21.15.113:0/1362793049 shutdown_connections 2024-03-04T08:16:18.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:18.265+0000 7f0849800640 1 -- 172.21.15.113:0/1362793049 wait complete. 2024-03-04T08:16:18.285 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:16:18.692 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:18.692 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 48 -eq 48 ] 2024-03-04T08:16:18.692 INFO:tasks.workunit.client.0.smithi113.stderr:+ continue 2024-03-04T08:16:18.692 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 49 -eq 48 ] 2024-03-04T08:16:18.692 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.49 2024-03-04T08:16:18.693 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.49 2024-03-04T08:16:18.693 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.49 2024-03-04T08:16:18.693 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.49 2024-03-04T08:16:18.693 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:20.286 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:20.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.473+0000 7f86e7a00640 1 Processor -- start 2024-03-04T08:16:20.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.473+0000 7f86e7a00640 1 -- start start 2024-03-04T08:16:20.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86e7a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e014e4c0 0x7f86e0072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:20.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86e7a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86e00c22e0 0x7f86e00731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:20.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86e7a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e00c27f0 con 0x7f86e00c2d80 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86e7a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e015ad60 con 0x7f86e00c22e0 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86e7a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e00c2a90 con 0x7f86e014e4c0 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86dfe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86e00c22e0 0x7f86e00731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86e5a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f86e00c2d80 0x7f86e015b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47944/0 (socket says 172.21.15.113:47944) 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86e5a00640 1 -- 172.21.15.113:0/714333506 learned_addr learned my addr 172.21.15.113:0/714333506 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86e5000640 1 --2- 172.21.15.113:0/714333506 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e014e4c0 0x7f86e0072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86dfe00640 1 -- 172.21.15.113:0/714333506 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e014e4c0 msgr2=0x7f86e0072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86dfe00640 1 --2- 172.21.15.113:0/714333506 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e014e4c0 0x7f86e0072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86dfe00640 1 -- 172.21.15.113:0/714333506 >> v1:172.21.15.113:6789/0 conn(0x7f86e00c2d80 legacy=0x7f86e015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86dfe00640 1 -- 172.21.15.113:0/714333506 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0070c60 con 0x7f86e00c22e0 2024-03-04T08:16:20.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86df400640 1 -- 172.21.15.113:0/714333506 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957111421 0 0) 0x7f86e00c27f0 con 0x7f86e00c2d80 2024-03-04T08:16:20.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.477+0000 7f86dfe00640 1 --2- 172.21.15.113:0/714333506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e00c22e0 0x7f86e00731c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0097c0 tx=0x7f86cc01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7767a40c8cea173 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:20.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86df400640 1 -- 172.21.15.113:0/714333506 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0b5070 con 0x7f86e00c22e0 2024-03-04T08:16:20.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86df400640 1 -- 172.21.15.113:0/714333506 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc0a8990 con 0x7f86e00c22e0 2024-03-04T08:16:20.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86df400640 1 -- 172.21.15.113:0/714333506 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0a8d60 con 0x7f86e00c22e0 2024-03-04T08:16:20.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86e7a00640 1 -- 172.21.15.113:0/714333506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e00c22e0 msgr2=0x7f86e00731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:20.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86e7a00640 1 --2- 172.21.15.113:0/714333506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e00c22e0 0x7f86e00731c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0097c0 tx=0x7f86cc01ec90 comp rx=0 tx=0).stop 2024-03-04T08:16:20.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86e7a00640 1 -- 172.21.15.113:0/714333506 shutdown_connections 2024-03-04T08:16:20.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86e7a00640 1 --2- 172.21.15.113:0/714333506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e00c22e0 0x7f86e00731c0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:20.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86e7a00640 1 --2- 172.21.15.113:0/714333506 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e014e4c0 0x7f86e0072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:20.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86e7a00640 1 -- 172.21.15.113:0/714333506 >> 172.21.15.113:0/714333506 conn(0x7f86e006e820 msgr2=0x7f86e006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:20.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86e7a00640 1 -- 172.21.15.113:0/714333506 shutdown_connections 2024-03-04T08:16:20.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86e7a00640 1 -- 172.21.15.113:0/714333506 wait complete. 2024-03-04T08:16:20.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86e7a00640 1 Processor -- start 2024-03-04T08:16:20.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.481+0000 7f86e7a00640 1 -- start start 2024-03-04T08:16:20.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e7a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e00c22e0 0x7f86e00c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:20.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e7a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e00c2d80 0x7f86e014d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:20.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e7a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e014e4c0 0x7f86e01524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:20.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e7a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e0071500 con 0x7f86e014e4c0 2024-03-04T08:16:20.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e7a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e0071220 con 0x7f86e00c22e0 2024-03-04T08:16:20.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e7a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e014e8a0 con 0x7f86e00c2d80 2024-03-04T08:16:20.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e014e4c0 0x7f86e01524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e014e4c0 0x7f86e01524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35288/0 (socket says 172.21.15.113:35288) 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5a00640 1 -- 172.21.15.113:0/1383364219 learned_addr learned my addr 172.21.15.113:0/1383364219 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5000640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e00c22e0 0x7f86e00c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86dfe00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e00c2d80 0x7f86e014d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5a00640 1 -- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e00c2d80 msgr2=0x7f86e014d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5a00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e00c2d80 0x7f86e014d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5a00640 1 -- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e00c22e0 msgr2=0x7f86e00c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5a00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e00c22e0 0x7f86e00c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5a00640 1 -- 172.21.15.113:0/1383364219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e00c55c0 con 0x7f86e014e4c0 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5000640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e00c22e0 0x7f86e00c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e5a00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e014e4c0 0x7f86e01524f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f86d800c7b0 tx=0x7f86d800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86dcc00640 1 -- 172.21.15.113:0/1383364219 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8018070 con 0x7f86e014e4c0 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86dcc00640 1 -- 172.21.15.113:0/1383364219 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d800a640 con 0x7f86e014e4c0 2024-03-04T08:16:20.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e7a00640 1 -- 172.21.15.113:0/1383364219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e00c58b0 con 0x7f86e014e4c0 2024-03-04T08:16:20.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86dcc00640 1 -- 172.21.15.113:0/1383364219 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d800aa70 con 0x7f86e014e4c0 2024-03-04T08:16:20.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e7a00640 1 -- 172.21.15.113:0/1383364219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e00c5e70 con 0x7f86e014e4c0 2024-03-04T08:16:20.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.485+0000 7f86e7a00640 1 -- 172.21.15.113:0/1383364219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e0075c00 con 0x7f86e014e4c0 2024-03-04T08:16:20.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.489+0000 7f86dcc00640 1 -- 172.21.15.113:0/1383364219 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86d801b070 con 0x7f86e014e4c0 2024-03-04T08:16:20.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.489+0000 7f86dcc00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c0066f90 0x7f86c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:20.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.489+0000 7f86dcc00640 1 -- 172.21.15.113:0/1383364219 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f86d8084f20 con 0x7f86e014e4c0 2024-03-04T08:16:20.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.489+0000 7f86e5000640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c0066f90 0x7f86c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:20.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.493+0000 7f86e5000640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c0066f90 0x7f86c0069450 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009b10 tx=0x7f86d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:20.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.493+0000 7f86dcc00640 1 -- 172.21.15.113:0/1383364219 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f86d80555d0 con 0x7f86e014e4c0 2024-03-04T08:16:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.741+0000 7f86e7a00640 1 -- 172.21.15.113:0/1383364219 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86e0155190 con 0x7f86e014e4c0 2024-03-04T08:16:20.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.741+0000 7f86dcc00640 1 -- 172.21.15.113:0/1383364219 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f86d8054ed0 con 0x7f86e014e4c0 2024-03-04T08:16:20.742 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:20.742 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":154,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:16.437460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:rejoin","state_seq":32,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692]},"id":1}]} 2024-03-04T08:16:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 154 2024-03-04T08:16:20.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 -- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c0066f90 msgr2=0x7f86c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:20.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c0066f90 0x7f86c0069450 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f86d4009b10 tx=0x7f86d4008040 comp rx=0 tx=0).stop 2024-03-04T08:16:20.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 -- 172.21.15.113:0/1383364219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e014e4c0 msgr2=0x7f86e01524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:20.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e014e4c0 0x7f86e01524f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f86d800c7b0 tx=0x7f86d800cc70 comp rx=0 tx=0).stop 2024-03-04T08:16:20.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 -- 172.21.15.113:0/1383364219 shutdown_connections 2024-03-04T08:16:20.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f86c0066f90 0x7f86c0069450 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:20.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86e014e4c0 0x7f86e01524f0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:20.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f86e00c2d80 0x7f86e014d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:20.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 --2- 172.21.15.113:0/1383364219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f86e00c22e0 0x7f86e00c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:20.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 -- 172.21.15.113:0/1383364219 >> 172.21.15.113:0/1383364219 conn(0x7f86e006e820 msgr2=0x7f86e00c74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:20.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 -- 172.21.15.113:0/1383364219 shutdown_connections 2024-03-04T08:16:20.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:20.745+0000 7f86e7a00640 1 -- 172.21.15.113:0/1383364219 wait complete. 2024-03-04T08:16:20.774 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:16:22.065 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:22.066 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.50 2024-03-04T08:16:22.066 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 50 -eq 48 ] 2024-03-04T08:16:22.066 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.50 2024-03-04T08:16:22.066 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.50 2024-03-04T08:16:22.066 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.50 2024-03-04T08:16:22.066 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:22.775 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.913+0000 7f25db000640 1 Processor -- start 2024-03-04T08:16:22.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.913+0000 7f25db000640 1 -- start start 2024-03-04T08:16:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25db000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 0x7f25d4072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25db000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f25d415b900 0x7f25d40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:22.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25db000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d40c27f0 con 0x7f25d40c22e0 2024-03-04T08:16:22.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25db000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d415ad60 con 0x7f25d415b900 2024-03-04T08:16:22.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25db000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d40c2a90 con 0x7f25d414e4c0 2024-03-04T08:16:22.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25cb400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f25d40c22e0 0x7f25d415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47948/0 (socket says 172.21.15.113:47948) 2024-03-04T08:16:22.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25cb400640 1 -- 172.21.15.113:0/1272371371 learned_addr learned my addr 172.21.15.113:0/1272371371 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:16:22.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25d9000640 1 --2- 172.21.15.113:0/1272371371 >> v2:172.21.15.136:3300/0 conn(0x7f25d415b900 0x7f25d40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25d3e00640 1 --2- 172.21.15.113:0/1272371371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 0x7f25d4072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25d3400640 1 -- 172.21.15.113:0/1272371371 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771527632 0 0) 0x7f25d40c27f0 con 0x7f25d40c22e0 2024-03-04T08:16:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25d3400640 1 -- 172.21.15.113:0/1272371371 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b8003530 con 0x7f25d40c22e0 2024-03-04T08:16:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25d9000640 1 -- 172.21.15.113:0/1272371371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 msgr2=0x7f25d4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25d9000640 1 --2- 172.21.15.113:0/1272371371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 0x7f25d4072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25d3400640 1 -- 172.21.15.113:0/1272371371 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1865053950 0 0) 0x7f25b8003530 con 0x7f25d40c22e0 2024-03-04T08:16:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25d9000640 1 -- 172.21.15.113:0/1272371371 >> v1:172.21.15.113:6789/0 conn(0x7f25d40c22e0 legacy=0x7f25d415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:22.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.917+0000 7f25d9000640 1 -- 172.21.15.113:0/1272371371 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d4070c60 con 0x7f25d415b900 2024-03-04T08:16:22.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25d9000640 1 --2- 172.21.15.113:0/1272371371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d415b900 0x7f25d40731c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0097c0 tx=0x7f25cc01ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81230cda43ea19d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25d3400640 1 -- 172.21.15.113:0/1272371371 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0b5070 con 0x7f25d415b900 2024-03-04T08:16:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25d3400640 1 -- 172.21.15.113:0/1272371371 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc0a85f0 con 0x7f25d415b900 2024-03-04T08:16:22.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25d3400640 1 -- 172.21.15.113:0/1272371371 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc001680 con 0x7f25d415b900 2024-03-04T08:16:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25db000640 1 -- 172.21.15.113:0/1272371371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d415b900 msgr2=0x7f25d40731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25db000640 1 --2- 172.21.15.113:0/1272371371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d415b900 0x7f25d40731c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0097c0 tx=0x7f25cc01ebb0 comp rx=0 tx=0).stop 2024-03-04T08:16:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25db000640 1 -- 172.21.15.113:0/1272371371 shutdown_connections 2024-03-04T08:16:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25db000640 1 --2- 172.21.15.113:0/1272371371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d415b900 0x7f25d40731c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25db000640 1 --2- 172.21.15.113:0/1272371371 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 0x7f25d4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25db000640 1 -- 172.21.15.113:0/1272371371 >> 172.21.15.113:0/1272371371 conn(0x7f25d406e820 msgr2=0x7f25d406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25db000640 1 -- 172.21.15.113:0/1272371371 shutdown_connections 2024-03-04T08:16:22.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.921+0000 7f25db000640 1 -- 172.21.15.113:0/1272371371 wait complete. 2024-03-04T08:16:22.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25db000640 1 Processor -- start 2024-03-04T08:16:22.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25db000640 1 -- start start 2024-03-04T08:16:22.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25db000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40c22e0 0x7f25d41489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:22.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25db000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 0x7f25d41f0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:22.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25db000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d415b900 0x7f25d41f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:22.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25db000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d4071500 con 0x7f25d40c22e0 2024-03-04T08:16:22.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25db000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d4071220 con 0x7f25d415b900 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25db000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d414e8a0 con 0x7f25d414e4c0 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40c22e0 0x7f25d41489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40c22e0 0x7f25d41489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35290/0 (socket says 172.21.15.113:35290) 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d3e00640 1 -- 172.21.15.113:0/3011011151 learned_addr learned my addr 172.21.15.113:0/3011011151 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d9000640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d415b900 0x7f25d41f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25cb400640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 0x7f25d41f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d3e00640 1 -- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 msgr2=0x7f25d41f0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d3e00640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 0x7f25d41f0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d3e00640 1 -- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d415b900 msgr2=0x7f25d41f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d3e00640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d415b900 0x7f25d41f0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d3e00640 1 -- 172.21.15.113:0/3011011151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d41f4ba0 con 0x7f25d40c22e0 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d9000640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d415b900 0x7f25d41f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:16:22.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25cb400640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 0x7f25d41f0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:16:22.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d3e00640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40c22e0 0x7f25d41489b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009d10 tx=0x7f25c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:22.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d0c00640 1 -- 172.21.15.113:0/3011011151 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4018020 con 0x7f25d40c22e0 2024-03-04T08:16:22.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d0c00640 1 -- 172.21.15.113:0/3011011151 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c4014480 con 0x7f25d40c22e0 2024-03-04T08:16:22.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25db000640 1 -- 172.21.15.113:0/3011011151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d4146a10 con 0x7f25d40c22e0 2024-03-04T08:16:22.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25d0c00640 1 -- 172.21.15.113:0/3011011151 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4014830 con 0x7f25d40c22e0 2024-03-04T08:16:22.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.925+0000 7f25db000640 1 -- 172.21.15.113:0/3011011151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d4146cb0 con 0x7f25d40c22e0 2024-03-04T08:16:22.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.929+0000 7f25db000640 1 -- 172.21.15.113:0/3011011151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d4075c00 con 0x7f25d40c22e0 2024-03-04T08:16:22.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.929+0000 7f25d0c00640 1 -- 172.21.15.113:0/3011011151 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25c401b020 con 0x7f25d40c22e0 2024-03-04T08:16:22.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.929+0000 7f25d0c00640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f25b0066f60 0x7f25b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:22.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.929+0000 7f25d0c00640 1 -- 172.21.15.113:0/3011011151 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f25c4086660 con 0x7f25d40c22e0 2024-03-04T08:16:22.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.929+0000 7f25cb400640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f25b0066f60 0x7f25b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:22.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.929+0000 7f25cb400640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f25b0066f60 0x7f25b0069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f25bc001ec0 tx=0x7f25bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:22.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:22.937+0000 7f25d0c00640 1 -- 172.21.15.113:0/3011011151 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f25c4022030 con 0x7f25d40c22e0 2024-03-04T08:16:23.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.133+0000 7f25db000640 1 -- 172.21.15.113:0/3011011151 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25d4148530 con 0x7f25d40c22e0 2024-03-04T08:16:23.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.133+0000 7f25d0c00640 1 -- 172.21.15.113:0/3011011151 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) v1 ==== 95+0+5829 (secure 0 0 0) 0x7f25c4010040 con 0x7f25d40c22e0 2024-03-04T08:16:23.133 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:23.133 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":155,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7004,7692,7839]},"id":1}]} 2024-03-04T08:16:23.133 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 155 2024-03-04T08:16:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 -- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f25b0066f60 msgr2=0x7f25b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f25b0066f60 0x7f25b0069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f25bc001ec0 tx=0x7f25bc009040 comp rx=0 tx=0).stop 2024-03-04T08:16:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 -- 172.21.15.113:0/3011011151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40c22e0 msgr2=0x7f25d41489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40c22e0 0x7f25d41489b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f25c4009d10 tx=0x7f25c400c490 comp rx=0 tx=0).stop 2024-03-04T08:16:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 -- 172.21.15.113:0/3011011151 shutdown_connections 2024-03-04T08:16:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25d415b900 0x7f25d41f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25d414e4c0 0x7f25d41f0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f25b0066f60 0x7f25b0069420 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 --2- 172.21.15.113:0/3011011151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25d40c22e0 0x7f25d41489b0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:23.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 -- 172.21.15.113:0/3011011151 >> 172.21.15.113:0/3011011151 conn(0x7f25d406e820 msgr2=0x7f25d4072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 -- 172.21.15.113:0/3011011151 shutdown_connections 2024-03-04T08:16:23.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:23.137+0000 7f25db000640 1 -- 172.21.15.113:0/3011011151 wait complete. 2024-03-04T08:16:23.163 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=1, replacing gid=7519 2024-03-04T08:16:23.163 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.b 2024-03-04T08:16:24.322 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T08:16:24.321+0000 7f9b59600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T08:16:27.365 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:27.366 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.51 2024-03-04T08:16:27.366 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 51 -eq 48 ] 2024-03-04T08:16:27.366 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.51 2024-03-04T08:16:27.366 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.51 2024-03-04T08:16:27.366 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.51 2024-03-04T08:16:27.366 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:30.844 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:30.844 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 52 -eq 48 ] 2024-03-04T08:16:30.844 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.52 2024-03-04T08:16:30.844 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.52 2024-03-04T08:16:30.845 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.52 2024-03-04T08:16:30.845 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.52 2024-03-04T08:16:30.845 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:31.164 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2024-03-04T08:16:31.164 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-04T08:16:31.165 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-04T08:16:31.168 INFO:tasks.ceph.mds.b:Started 2024-03-04T08:16:31.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:16:31.245 INFO:tasks.ceph.mds.b.smithi136.stdout:starting mds.b at 2024-03-04T08:16:31.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.321+0000 7f0feb200640 1 Processor -- start 2024-03-04T08:16:31.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0feb200640 1 -- start start 2024-03-04T08:16:31.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0feb200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0fec151760 0x7f0fec0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:31.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0feb200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec0c18d0 0x7f0fec071f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:31.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0feb200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fec06c290 con 0x7f0fec151b40 2024-03-04T08:16:31.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0feb200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fec15aef0 con 0x7f0fec151760 2024-03-04T08:16:31.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0feb200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec06ef10 con 0x7f0fec0c18d0 2024-03-04T08:16:31.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fe9400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0fec151b40 0x7f0fec15b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36942/0 (socket says 172.21.15.113:36942) 2024-03-04T08:16:31.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fe9400640 1 -- 172.21.15.113:0/186033704 learned_addr learned my addr 172.21.15.113:0/186033704 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:16:31.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fe9e00640 1 --2- 172.21.15.113:0/186033704 >> v2:172.21.15.136:3300/0 conn(0x7f0fec151760 0x7f0fec0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:31.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fea800640 1 --2- 172.21.15.113:0/186033704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec0c18d0 0x7f0fec071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:31.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fe8a00640 1 -- 172.21.15.113:0/186033704 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097317477 0 0) 0x7f0fec06c290 con 0x7f0fec151b40 2024-03-04T08:16:31.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fe8a00640 1 -- 172.21.15.113:0/186033704 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd0003530 con 0x7f0fec151b40 2024-03-04T08:16:31.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fea800640 1 -- 172.21.15.113:0/186033704 >> v2:172.21.15.136:3300/0 conn(0x7f0fec151760 msgr2=0x7f0fec0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:16:31.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fea800640 1 --2- 172.21.15.113:0/186033704 >> v2:172.21.15.136:3300/0 conn(0x7f0fec151760 0x7f0fec0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:31.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fea800640 1 -- 172.21.15.113:0/186033704 >> v1:172.21.15.113:6789/0 conn(0x7f0fec151b40 legacy=0x7f0fec15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:31.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fea800640 1 -- 172.21.15.113:0/186033704 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec15ba90 con 0x7f0fec0c18d0 2024-03-04T08:16:31.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.325+0000 7f0fe9e00640 1 --2- 172.21.15.113:0/186033704 >> v2:172.21.15.136:3300/0 conn(0x7f0fec151760 0x7f0fec0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:16:31.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0fea800640 1 --2- 172.21.15.113:0/186033704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec0c18d0 0x7f0fec071f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009d90 tx=0x7f0fe401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9186df6a9552751a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:31.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0fe8a00640 1 -- 172.21.15.113:0/186033704 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40b2070 con 0x7f0fec0c18d0 2024-03-04T08:16:31.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0fe8a00640 1 -- 172.21.15.113:0/186033704 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe40a5890 con 0x7f0fec0c18d0 2024-03-04T08:16:31.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0fe8a00640 1 -- 172.21.15.113:0/186033704 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40a5c60 con 0x7f0fec0c18d0 2024-03-04T08:16:31.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0feb200640 1 -- 172.21.15.113:0/186033704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec0c18d0 msgr2=0x7f0fec071f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:31.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0feb200640 1 --2- 172.21.15.113:0/186033704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec0c18d0 0x7f0fec071f70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009d90 tx=0x7f0fe401c8e0 comp rx=0 tx=0).stop 2024-03-04T08:16:31.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0feb200640 1 -- 172.21.15.113:0/186033704 shutdown_connections 2024-03-04T08:16:31.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0feb200640 1 --2- 172.21.15.113:0/186033704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec0c18d0 0x7f0fec071f70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:31.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0feb200640 1 --2- 172.21.15.113:0/186033704 >> v2:172.21.15.136:3300/0 conn(0x7f0fec151760 0x7f0fec0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:31.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0feb200640 1 -- 172.21.15.113:0/186033704 >> 172.21.15.113:0/186033704 conn(0x7f0fec06e2a0 msgr2=0x7f0fec06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:31.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0feb200640 1 -- 172.21.15.113:0/186033704 shutdown_connections 2024-03-04T08:16:31.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0feb200640 1 -- 172.21.15.113:0/186033704 wait complete. 2024-03-04T08:16:31.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0feb200640 1 Processor -- start 2024-03-04T08:16:31.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.333+0000 7f0feb200640 1 -- start start 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0feb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fec0c18d0 0x7f0fec0ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0feb200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec151760 0x7f0fec137410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0feb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fec151b40 0x7f0fec1f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0feb200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0728c0 con 0x7f0fec151b40 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0feb200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec15ab80 con 0x7f0fec0c18d0 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0feb200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0c1e20 con 0x7f0fec151760 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fea800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fec151b40 0x7f0fec1f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fea800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fec151b40 0x7f0fec1f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42706/0 (socket says 172.21.15.113:42706) 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fea800640 1 -- 172.21.15.113:0/2778848620 learned_addr learned my addr 172.21.15.113:0/2778848620 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fe9400640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec151760 0x7f0fec137410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fe9e00640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fec0c18d0 0x7f0fec0ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fea800640 1 -- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec151760 msgr2=0x7f0fec137410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:31.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fea800640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec151760 0x7f0fec137410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:31.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fea800640 1 -- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fec0c18d0 msgr2=0x7f0fec0ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:31.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fea800640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fec0c18d0 0x7f0fec0ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:31.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fea800640 1 -- 172.21.15.113:0/2778848620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec15a960 con 0x7f0fec151b40 2024-03-04T08:16:31.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fea800640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fec151b40 0x7f0fec1f4c60 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009d90 tx=0x7f0fe4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:31.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fe2000640 1 -- 172.21.15.113:0/2778848620 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40b2050 con 0x7f0fec151b40 2024-03-04T08:16:31.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fe2000640 1 -- 172.21.15.113:0/2778848620 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe40be070 con 0x7f0fec151b40 2024-03-04T08:16:31.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0fe2000640 1 -- 172.21.15.113:0/2778848620 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40ad870 con 0x7f0fec151b40 2024-03-04T08:16:31.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0feb200640 1 -- 172.21.15.113:0/2778848620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec15a680 con 0x7f0fec151b40 2024-03-04T08:16:31.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0feb200640 1 -- 172.21.15.113:0/2778848620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec141a40 con 0x7f0fec151b40 2024-03-04T08:16:31.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.337+0000 7f0feb200640 1 -- 172.21.15.113:0/2778848620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec075a80 con 0x7f0fec151b40 2024-03-04T08:16:31.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.341+0000 7f0fe2000640 1 -- 172.21.15.113:0/2778848620 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fe40ada10 con 0x7f0fec151b40 2024-03-04T08:16:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.341+0000 7f0fe2000640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0fcc066f60 0x7f0fcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:16:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.341+0000 7f0fe2000640 1 -- 172.21.15.113:0/2778848620 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f0fe41211d0 con 0x7f0fec151b40 2024-03-04T08:16:31.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.341+0000 7f0fe9e00640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0fcc066f60 0x7f0fcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:16:31.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.341+0000 7f0fe9e00640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0fcc066f60 0x7f0fcc069420 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009580 tx=0x7f0fdc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:16:31.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.345+0000 7f0fe2000640 1 -- 172.21.15.113:0/2778848620 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0fe4016560 con 0x7f0fec151b40 2024-03-04T08:16:31.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.565+0000 7f0feb200640 1 -- 172.21.15.113:0/2778848620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fec162d00 con 0x7f0fec151b40 2024-03-04T08:16:31.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0fe2000640 1 -- 172.21.15.113:0/2778848620 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f0fe40f13c0 con 0x7f0fec151b40 2024-03-04T08:16:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:16:31.567 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7004]},"id":1}]} 2024-03-04T08:16:31.567 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 156 2024-03-04T08:16:31.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 -- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0fcc066f60 msgr2=0x7f0fcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:31.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0fcc066f60 0x7f0fcc069420 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009580 tx=0x7f0fdc009190 comp rx=0 tx=0).stop 2024-03-04T08:16:31.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 -- 172.21.15.113:0/2778848620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fec151b40 msgr2=0x7f0fec1f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:16:31.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fec151b40 0x7f0fec1f4c60 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009d90 tx=0x7f0fe4019680 comp rx=0 tx=0).stop 2024-03-04T08:16:31.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 -- 172.21.15.113:0/2778848620 shutdown_connections 2024-03-04T08:16:31.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0fcc066f60 0x7f0fcc069420 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:31.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0fec151b40 0x7f0fec1f4c60 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:31.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0fec151760 0x7f0fec137410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:31.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 --2- 172.21.15.113:0/2778848620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0fec0c18d0 0x7f0fec0ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:16:31.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 -- 172.21.15.113:0/2778848620 >> 172.21.15.113:0/2778848620 conn(0x7f0fec06e2a0 msgr2=0x7f0fec143760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:16:31.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 -- 172.21.15.113:0/2778848620 shutdown_connections 2024-03-04T08:16:31.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:16:31.569+0000 7f0feb200640 1 -- 172.21.15.113:0/2778848620 wait complete. 2024-03-04T08:16:31.588 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2024-03-04T08:16:31.588 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 66 secs before thrashing 2024-03-04T08:16:34.946 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:34.946 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.53 2024-03-04T08:16:34.946 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 53 -eq 48 ] 2024-03-04T08:16:34.947 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.53 2024-03-04T08:16:34.947 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.53 2024-03-04T08:16:34.947 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.53 2024-03-04T08:16:34.947 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:38.130 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:16:38.134 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:16:38.937 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:38.937 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.54 2024-03-04T08:16:38.937 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 54 -eq 48 ] 2024-03-04T08:16:38.938 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.54 2024-03-04T08:16:38.938 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.54 2024-03-04T08:16:38.938 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.54 2024-03-04T08:16:38.938 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:43.003 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:43.004 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.55 2024-03-04T08:16:43.004 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 55 -eq 48 ] 2024-03-04T08:16:43.004 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.55 2024-03-04T08:16:43.004 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.55 2024-03-04T08:16:43.005 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.55 2024-03-04T08:16:43.005 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:47.841 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:47.841 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 56 -eq 48 ] 2024-03-04T08:16:47.841 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.56 2024-03-04T08:16:47.841 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.56 2024-03-04T08:16:47.842 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.56 2024-03-04T08:16:47.842 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.56 2024-03-04T08:16:47.842 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:51.870 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:51.870 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 57 -eq 48 ] 2024-03-04T08:16:51.870 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.57 2024-03-04T08:16:51.870 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.57 2024-03-04T08:16:51.871 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.57 2024-03-04T08:16:51.871 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.57 2024-03-04T08:16:51.871 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:55.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:55.478 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.58 2024-03-04T08:16:55.479 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 58 -eq 48 ] 2024-03-04T08:16:55.479 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.58 2024-03-04T08:16:55.479 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.58 2024-03-04T08:16:55.479 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.58 2024-03-04T08:16:55.480 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:16:59.486 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:16:59.487 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 59 -eq 48 ] 2024-03-04T08:16:59.487 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.59 2024-03-04T08:16:59.487 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.59 2024-03-04T08:16:59.487 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.59 2024-03-04T08:16:59.487 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.59 2024-03-04T08:16:59.488 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:04.110 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:04.110 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 60 -eq 48 ] 2024-03-04T08:17:04.110 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.60 2024-03-04T08:17:04.111 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.60 2024-03-04T08:17:04.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.60 2024-03-04T08:17:04.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.60 2024-03-04T08:17:04.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:08.109 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:08.110 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 61 -eq 48 ] 2024-03-04T08:17:08.110 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.61 2024-03-04T08:17:08.110 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.61 2024-03-04T08:17:08.110 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.61 2024-03-04T08:17:08.110 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.61 2024-03-04T08:17:08.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:08.182 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:17:08.187 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:17:11.983 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:11.983 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 62 -eq 48 ] 2024-03-04T08:17:11.983 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.62 2024-03-04T08:17:11.983 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.62 2024-03-04T08:17:11.984 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.62 2024-03-04T08:17:11.984 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.62 2024-03-04T08:17:11.984 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:15.729 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:15.729 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 63 -eq 48 ] 2024-03-04T08:17:15.730 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.63 2024-03-04T08:17:15.730 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.63 2024-03-04T08:17:15.730 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.63 2024-03-04T08:17:15.730 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.63 2024-03-04T08:17:15.730 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:20.687 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:20.687 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.64 2024-03-04T08:17:20.688 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 64 -eq 48 ] 2024-03-04T08:17:20.688 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.64 2024-03-04T08:17:20.688 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.64 2024-03-04T08:17:20.688 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.64 2024-03-04T08:17:20.688 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:26.505 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:26.506 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 65 -eq 48 ] 2024-03-04T08:17:26.506 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.65 2024-03-04T08:17:26.506 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.65 2024-03-04T08:17:26.506 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.65 2024-03-04T08:17:26.507 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.65 2024-03-04T08:17:26.507 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:31.213 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:31.213 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 66 -eq 48 ] 2024-03-04T08:17:31.213 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.66 2024-03-04T08:17:31.214 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.66 2024-03-04T08:17:31.214 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.66 2024-03-04T08:17:31.214 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.66 2024-03-04T08:17:31.214 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:35.510 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:35.510 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.67 2024-03-04T08:17:35.510 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 67 -eq 48 ] 2024-03-04T08:17:35.511 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.67 2024-03-04T08:17:35.511 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.67 2024-03-04T08:17:35.511 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.67 2024-03-04T08:17:35.511 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:37.589 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:37.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f9865c00640 1 Processor -- start 2024-03-04T08:17:37.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f9865c00640 1 -- start start 2024-03-04T08:17:37.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f9865c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9860152950 0x7f986015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:37.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f9865c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006faa0 0x7f98600c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:37.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f9865c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98600c4ce0 con 0x7f986006ed80 2024-03-04T08:17:37.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f9865c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98600c8370 con 0x7f9860152950 2024-03-04T08:17:37.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f9865c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98600ca0a0 con 0x7f986006faa0 2024-03-04T08:17:37.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f9865200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f986006ed80 0x7f98600ca530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36416/0 (socket says 172.21.15.113:36416) 2024-03-04T08:17:37.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f9865200640 1 -- 172.21.15.113:0/786138014 learned_addr learned my addr 172.21.15.113:0/786138014 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:17:37.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f985fe00640 1 --2- 172.21.15.113:0/786138014 >> v2:172.21.15.136:3300/0 conn(0x7f9860152950 0x7f986015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:37.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.739+0000 7f9857400640 1 --2- 172.21.15.113:0/786138014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006faa0 0x7f98600c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:37.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.743+0000 7f985f400640 1 -- 172.21.15.113:0/786138014 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738848315 0 0) 0x7f98600c4ce0 con 0x7f986006ed80 2024-03-04T08:17:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.743+0000 7f985f400640 1 -- 172.21.15.113:0/786138014 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9844003530 con 0x7f986006ed80 2024-03-04T08:17:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.743+0000 7f9857400640 1 -- 172.21.15.113:0/786138014 >> v2:172.21.15.136:3300/0 conn(0x7f9860152950 msgr2=0x7f986015bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:17:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.743+0000 7f9857400640 1 --2- 172.21.15.113:0/786138014 >> v2:172.21.15.136:3300/0 conn(0x7f9860152950 0x7f986015bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.743+0000 7f9857400640 1 -- 172.21.15.113:0/786138014 >> v1:172.21.15.113:6789/0 conn(0x7f986006ed80 legacy=0x7f98600ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.743+0000 7f9857400640 1 -- 172.21.15.113:0/786138014 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600c4f80 con 0x7f986006faa0 2024-03-04T08:17:37.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.743+0000 7f985fe00640 1 --2- 172.21.15.113:0/786138014 >> v2:172.21.15.136:3300/0 conn(0x7f9860152950 0x7f986015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:17:37.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9857400640 1 --2- 172.21.15.113:0/786138014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006faa0 0x7f98600c30b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98480097c0 tx=0x7f984801ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b691c38d5d36b7dd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f985f400640 1 -- 172.21.15.113:0/786138014 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98480b5070 con 0x7f986006faa0 2024-03-04T08:17:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f985f400640 1 -- 172.21.15.113:0/786138014 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98480a8990 con 0x7f986006faa0 2024-03-04T08:17:37.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f985f400640 1 -- 172.21.15.113:0/786138014 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98480a8d60 con 0x7f986006faa0 2024-03-04T08:17:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9865c00640 1 -- 172.21.15.113:0/786138014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006faa0 msgr2=0x7f98600c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9865c00640 1 --2- 172.21.15.113:0/786138014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006faa0 0x7f98600c30b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f98480097c0 tx=0x7f984801ec90 comp rx=0 tx=0).stop 2024-03-04T08:17:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9865c00640 1 -- 172.21.15.113:0/786138014 shutdown_connections 2024-03-04T08:17:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9865c00640 1 --2- 172.21.15.113:0/786138014 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006faa0 0x7f98600c30b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9865c00640 1 --2- 172.21.15.113:0/786138014 >> v2:172.21.15.136:3300/0 conn(0x7f9860152950 0x7f986015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9865c00640 1 -- 172.21.15.113:0/786138014 >> 172.21.15.113:0/786138014 conn(0x7f9860072f40 msgr2=0x7f98600706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9865c00640 1 -- 172.21.15.113:0/786138014 shutdown_connections 2024-03-04T08:17:37.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9865c00640 1 -- 172.21.15.113:0/786138014 wait complete. 2024-03-04T08:17:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9865c00640 1 Processor -- start 2024-03-04T08:17:37.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.747+0000 7f9865c00640 1 -- start start 2024-03-04T08:17:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006ed80 0x7f986015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986006faa0 0x7f986006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9860152950 0x7f986015a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98600c9a50 con 0x7f9860152950 2024-03-04T08:17:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986006fe80 con 0x7f986006faa0 2024-03-04T08:17:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98600c9d30 con 0x7f986006ed80 2024-03-04T08:17:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9860152950 0x7f986015a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f985fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006ed80 0x7f986015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9860152950 0x7f986015a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45958/0 (socket says 172.21.15.113:45958) 2024-03-04T08:17:37.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865200640 1 -- 172.21.15.113:0/1722669134 learned_addr learned my addr 172.21.15.113:0/1722669134 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9857400640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986006faa0 0x7f986006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865200640 1 -- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006ed80 msgr2=0x7f986015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865200640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006ed80 0x7f986015ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865200640 1 -- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986006faa0 msgr2=0x7f986006d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865200640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986006faa0 0x7f986006d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:37.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865200640 1 -- 172.21.15.113:0/1722669134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600b35e0 con 0x7f9860152950 2024-03-04T08:17:37.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865200640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9860152950 0x7f986015a790 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f985800cb10 tx=0x7f985800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:37.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f985cc00640 1 -- 172.21.15.113:0/1722669134 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858017070 con 0x7f9860152950 2024-03-04T08:17:37.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f985cc00640 1 -- 172.21.15.113:0/1722669134 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985800ac70 con 0x7f9860152950 2024-03-04T08:17:37.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865c00640 1 -- 172.21.15.113:0/1722669134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9860159710 con 0x7f9860152950 2024-03-04T08:17:37.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f985cc00640 1 -- 172.21.15.113:0/1722669134 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985800ae60 con 0x7f9860152950 2024-03-04T08:17:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865c00640 1 -- 172.21.15.113:0/1722669134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98601599d0 con 0x7f9860152950 2024-03-04T08:17:37.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.751+0000 7f9865c00640 1 -- 172.21.15.113:0/1722669134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9860075a80 con 0x7f9860152950 2024-03-04T08:17:37.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.755+0000 7f985cc00640 1 -- 172.21.15.113:0/1722669134 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f985801a070 con 0x7f9860152950 2024-03-04T08:17:37.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.755+0000 7f985cc00640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:37.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.755+0000 7f985cc00640 1 -- 172.21.15.113:0/1722669134 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f9858085540 con 0x7f9860152950 2024-03-04T08:17:37.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.755+0000 7f985fe00640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:37.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.755+0000 7f985fe00640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 0x7f983c069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9850009580 tx=0x7f9850009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:37.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:37.763+0000 7f985cc00640 1 -- 172.21.15.113:0/1722669134 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f98600b35e0 con 0x7f9860152950 2024-03-04T08:17:38.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 -- 172.21.15.113:0/1722669134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f986006f880 con 0x7f9860152950 2024-03-04T08:17:38.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f985cc00640 1 -- 172.21.15.113:0/1722669134 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f9858055640 con 0x7f9860152950 2024-03-04T08:17:38.132 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:38.132 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7004]},"id":1}]} 2024-03-04T08:17:38.132 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 156 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 -- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 msgr2=0x7f983c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 0x7f983c069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9850009580 tx=0x7f9850009190 comp rx=0 tx=0).stop 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 -- 172.21.15.113:0/1722669134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9860152950 msgr2=0x7f986015a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9860152950 0x7f986015a790 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f985800cb10 tx=0x7f985800a060 comp rx=0 tx=0).stop 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 -- 172.21.15.113:0/1722669134 shutdown_connections 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f983c066f60 0x7f983c069420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9860152950 0x7f986015a790 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f986006faa0 0x7f986006d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 --2- 172.21.15.113:0/1722669134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f986006ed80 0x7f986015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 -- 172.21.15.113:0/1722669134 >> 172.21.15.113:0/1722669134 conn(0x7f9860072f40 msgr2=0x7f98600705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.135+0000 7f9865c00640 1 -- 172.21.15.113:0/1722669134 shutdown_connections 2024-03-04T08:17:38.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.139+0000 7f9865c00640 1 -- 172.21.15.113:0/1722669134 wait complete. 2024-03-04T08:17:38.160 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.a (rank=1) 2024-03-04T08:17:38.160 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-04T08:17:38.160 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:17:38.175 INFO:tasks.ceph.mds.a:Stopped 2024-03-04T08:17:38.176 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:38.241 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:38.241 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.68 2024-03-04T08:17:38.242 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 68 -eq 48 ] 2024-03-04T08:17:38.242 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.68 2024-03-04T08:17:38.242 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.68 2024-03-04T08:17:38.242 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.68 2024-03-04T08:17:38.242 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:38.277 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:17:38.280 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:17:38.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f7641800640 1 Processor -- start 2024-03-04T08:17:38.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f7641800640 1 -- start start 2024-03-04T08:17:38.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f7641800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f763c1527f0 0x7f763c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:38.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f7641800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f763c0c8200 0x7f763c072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:38.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f7641800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c1591f0 con 0x7f763c152bd0 2024-03-04T08:17:38.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f7641800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c071df0 con 0x7f763c0c8200 2024-03-04T08:17:38.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f7641800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c1597b0 con 0x7f763c1527f0 2024-03-04T08:17:38.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f763a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f763c0c8200 0x7f763c072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:38.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f763aa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f763c1527f0 0x7f763c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:38.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f763a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f763c0c8200 0x7f763c072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:53744/0 (socket says 172.21.15.113:53744) 2024-03-04T08:17:38.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f763a000640 1 -- 172.21.15.113:0/4045493156 learned_addr learned my addr 172.21.15.113:0/4045493156 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:38.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f763a000640 1 -- 172.21.15.113:0/4045493156 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f763c1527f0 msgr2=0x7f763c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:17:38.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f763a000640 1 --2- 172.21.15.113:0/4045493156 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f763c1527f0 0x7f763c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f763a000640 1 -- 172.21.15.113:0/4045493156 >> v1:172.21.15.136:6789/0 conn(0x7f763c152bd0 legacy=0x7f763c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:38.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.387+0000 7f763a000640 1 -- 172.21.15.113:0/4045493156 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c1594d0 con 0x7f763c0c8200 2024-03-04T08:17:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f763a000640 1 --2- 172.21.15.113:0/4045493156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763c0c8200 0x7f763c072280 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7628009870 tx=0x7f762801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe00b6d0c9c5a218 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7639600640 1 -- 172.21.15.113:0/4045493156 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76280a7070 con 0x7f763c0c8200 2024-03-04T08:17:38.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7639600640 1 -- 172.21.15.113:0/4045493156 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76280b0070 con 0x7f763c0c8200 2024-03-04T08:17:38.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7639600640 1 -- 172.21.15.113:0/4045493156 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76280ab890 con 0x7f763c0c8200 2024-03-04T08:17:38.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7641800640 1 -- 172.21.15.113:0/4045493156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763c0c8200 msgr2=0x7f763c072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:38.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7641800640 1 --2- 172.21.15.113:0/4045493156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763c0c8200 0x7f763c072280 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7628009870 tx=0x7f762801c560 comp rx=0 tx=0).stop 2024-03-04T08:17:38.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7641800640 1 -- 172.21.15.113:0/4045493156 shutdown_connections 2024-03-04T08:17:38.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7641800640 1 --2- 172.21.15.113:0/4045493156 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763c0c8200 0x7f763c072280 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7641800640 1 --2- 172.21.15.113:0/4045493156 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f763c1527f0 0x7f763c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7641800640 1 -- 172.21.15.113:0/4045493156 >> 172.21.15.113:0/4045493156 conn(0x7f763c06e820 msgr2=0x7f763c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:38.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7641800640 1 -- 172.21.15.113:0/4045493156 shutdown_connections 2024-03-04T08:17:38.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.395+0000 7f7641800640 1 -- 172.21.15.113:0/4045493156 wait complete. 2024-03-04T08:17:38.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7641800640 1 Processor -- start 2024-03-04T08:17:38.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7641800640 1 -- start start 2024-03-04T08:17:38.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7641800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763c0c8200 0x7f763c0b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:38.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7641800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f763c1527f0 0x7f763c147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:38.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7641800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f763c152bd0 0x7f763c14c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:38.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7641800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c165160 con 0x7f763c1527f0 2024-03-04T08:17:38.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7641800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c0727c0 con 0x7f763c0c8200 2024-03-04T08:17:38.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7641800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c1652e0 con 0x7f763c152bd0 2024-03-04T08:17:38.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f763c1527f0 0x7f763c147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:38.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f763c1527f0 0x7f763c147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45972/0 (socket says 172.21.15.113:45972) 2024-03-04T08:17:38.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763a000640 1 -- 172.21.15.113:0/2669541195 learned_addr learned my addr 172.21.15.113:0/2669541195 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:38.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763aa00640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763c0c8200 0x7f763c0b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:38.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763b400640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f763c152bd0 0x7f763c14c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763a000640 1 -- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f763c152bd0 msgr2=0x7f763c14c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763a000640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f763c152bd0 0x7f763c14c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763a000640 1 -- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763c0c8200 msgr2=0x7f763c0b1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763a000640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763c0c8200 0x7f763c0b1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763a000640 1 -- 172.21.15.113:0/2669541195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c0b05d0 con 0x7f763c1527f0 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f763a000640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f763c1527f0 0x7f763c147250 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f76280ad040 tx=0x7f762801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7632a00640 1 -- 172.21.15.113:0/2669541195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76280be5f0 con 0x7f763c1527f0 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7632a00640 1 -- 172.21.15.113:0/2669541195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76280be790 con 0x7f763c1527f0 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7632a00640 1 -- 172.21.15.113:0/2669541195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76280beb80 con 0x7f763c1527f0 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.399+0000 7f7641800640 1 -- 172.21.15.113:0/2669541195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c145ef0 con 0x7f763c1527f0 2024-03-04T08:17:38.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.403+0000 7f7641800640 1 -- 172.21.15.113:0/2669541195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c146490 con 0x7f763c1527f0 2024-03-04T08:17:38.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.403+0000 7f7641800640 1 -- 172.21.15.113:0/2669541195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c075c00 con 0x7f763c1527f0 2024-03-04T08:17:38.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.403+0000 7f7632a00640 1 -- 172.21.15.113:0/2669541195 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76280b43f0 con 0x7f763c1527f0 2024-03-04T08:17:38.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.403+0000 7f7632a00640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f761c066f60 0x7f761c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:38.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.403+0000 7f7632a00640 1 -- 172.21.15.113:0/2669541195 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f7628120210 con 0x7f763c1527f0 2024-03-04T08:17:38.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.403+0000 7f763aa00640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f761c066f60 0x7f761c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:38.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.407+0000 7f763aa00640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f761c066f60 0x7f761c069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f762c009b60 tx=0x7f762c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:38.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.407+0000 7f7632a00640 1 -- 172.21.15.113:0/2669541195 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f763c0b05d0 con 0x7f763c1527f0 2024-03-04T08:17:38.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 -- 172.21.15.113:0/2669541195 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f763c145820 con 0x7f763c1527f0 2024-03-04T08:17:38.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7632a00640 1 -- 172.21.15.113:0/2669541195 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f7628016150 con 0x7f763c1527f0 2024-03-04T08:17:38.612 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:38.613 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7004]},"id":1}]} 2024-03-04T08:17:38.613 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 156 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 -- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f761c066f60 msgr2=0x7f761c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f761c066f60 0x7f761c069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f762c009b60 tx=0x7f762c009190 comp rx=0 tx=0).stop 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 -- 172.21.15.113:0/2669541195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f763c1527f0 msgr2=0x7f763c147250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f763c1527f0 0x7f763c147250 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f76280ad040 tx=0x7f762801ada0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 -- 172.21.15.113:0/2669541195 shutdown_connections 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f763c152bd0 0x7f763c14c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f761c066f60 0x7f761c069420 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f763c1527f0 0x7f763c147250 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 --2- 172.21.15.113:0/2669541195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f763c0c8200 0x7f763c0b1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.615+0000 7f7641800640 1 -- 172.21.15.113:0/2669541195 >> 172.21.15.113:0/2669541195 conn(0x7f763c06e820 msgr2=0x7f763c0c1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.619+0000 7f7641800640 1 -- 172.21.15.113:0/2669541195 shutdown_connections 2024-03-04T08:17:38.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:38.619+0000 7f7641800640 1 -- 172.21.15.113:0/2669541195 wait complete. 2024-03-04T08:17:38.633 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:17:40.634 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:40.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7e00640 1 Processor -- start 2024-03-04T08:17:40.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7e00640 1 -- start start 2024-03-04T08:17:40.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74c8152950 0x7f74c815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:40.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f74c80cb540 0x7f74c80c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80c4ce0 con 0x7f74c806faa0 2024-03-04T08:17:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c80c8370 con 0x7f74c80cb540 2024-03-04T08:17:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80ca0a0 con 0x7f74c8152950 2024-03-04T08:17:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f74c80cb540 0x7f74c80c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f74c80cb540 0x7f74c80c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45986/0 (socket says 172.21.15.113:45986) 2024-03-04T08:17:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7400640 1 -- 172.21.15.113:0/3526972504 learned_addr learned my addr 172.21.15.113:0/3526972504 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:40.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c6a00640 1 --2- 172.21.15.113:0/3526972504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74c8152950 0x7f74c815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:40.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7400640 1 -- 172.21.15.113:0/3526972504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74c8152950 msgr2=0x7f74c815bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:17:40.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7400640 1 --2- 172.21.15.113:0/3526972504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74c8152950 0x7f74c815bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:40.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7400640 1 -- 172.21.15.113:0/3526972504 >> v1:172.21.15.136:6789/0 conn(0x7f74c806faa0 legacy=0x7f74c806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:40.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.779+0000 7f74c7400640 1 -- 172.21.15.113:0/3526972504 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80c4f80 con 0x7f74c80cb540 2024-03-04T08:17:40.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c7400640 1 --2- 172.21.15.113:0/3526972504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 0x7f74c80c30b0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009870 tx=0x7f74c001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5823bc83c3ce71d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:40.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c6000640 1 -- 172.21.15.113:0/3526972504 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00b5070 con 0x7f74c80cb540 2024-03-04T08:17:40.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c6000640 1 -- 172.21.15.113:0/3526972504 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c00a8a50 con 0x7f74c80cb540 2024-03-04T08:17:40.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c6000640 1 -- 172.21.15.113:0/3526972504 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00a8e20 con 0x7f74c80cb540 2024-03-04T08:17:40.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c7e00640 1 -- 172.21.15.113:0/3526972504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 msgr2=0x7f74c80c30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:40.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c7e00640 1 --2- 172.21.15.113:0/3526972504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 0x7f74c80c30b0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009870 tx=0x7f74c001ed50 comp rx=0 tx=0).stop 2024-03-04T08:17:40.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c7e00640 1 -- 172.21.15.113:0/3526972504 shutdown_connections 2024-03-04T08:17:40.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c7e00640 1 --2- 172.21.15.113:0/3526972504 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 0x7f74c80c30b0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:40.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c7e00640 1 --2- 172.21.15.113:0/3526972504 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74c8152950 0x7f74c815bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:40.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c7e00640 1 -- 172.21.15.113:0/3526972504 >> 172.21.15.113:0/3526972504 conn(0x7f74c8072f40 msgr2=0x7f74c80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:40.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c7e00640 1 -- 172.21.15.113:0/3526972504 shutdown_connections 2024-03-04T08:17:40.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.783+0000 7f74c7e00640 1 -- 172.21.15.113:0/3526972504 wait complete. 2024-03-04T08:17:40.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c7e00640 1 Processor -- start 2024-03-04T08:17:40.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c7e00640 1 -- start start 2024-03-04T08:17:40.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74c806faa0 0x7f74c81e5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:40.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 0x7f74c81eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:40.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74c8152950 0x7f74c81f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:40.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80c9a50 con 0x7f74c80cb540 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c806fe80 con 0x7f74c8152950 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80c9d30 con 0x7f74c806faa0 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74be000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 0x7f74c81eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74be000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 0x7f74c81eff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45998/0 (socket says 172.21.15.113:45998) 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74be000640 1 -- 172.21.15.113:0/2080280000 learned_addr learned my addr 172.21.15.113:0/2080280000 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c6a00640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74c806faa0 0x7f74c81e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c7400640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74c8152950 0x7f74c81f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74be000640 1 -- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74c806faa0 msgr2=0x7f74c81e5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74be000640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74c806faa0 0x7f74c81e5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74be000640 1 -- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74c8152950 msgr2=0x7f74c81f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74be000640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74c8152950 0x7f74c81f0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74be000640 1 -- 172.21.15.113:0/2080280000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c806e0a0 con 0x7f74c80cb540 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74c7400640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74c8152950 0x7f74c81f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:17:40.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.787+0000 7f74be000640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 0x7f74c81eff00 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f74b000c3c0 tx=0x7f74b000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.791+0000 7f74bfe00640 1 -- 172.21.15.113:0/2080280000 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0017070 con 0x7f74c80cb540 2024-03-04T08:17:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.791+0000 7f74bfe00640 1 -- 172.21.15.113:0/2080280000 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b00052f0 con 0x7f74c80cb540 2024-03-04T08:17:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.791+0000 7f74bfe00640 1 -- 172.21.15.113:0/2080280000 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0003840 con 0x7f74c80cb540 2024-03-04T08:17:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.791+0000 7f74c7e00640 1 -- 172.21.15.113:0/2080280000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c8152e60 con 0x7f74c80cb540 2024-03-04T08:17:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.791+0000 7f74c7e00640 1 -- 172.21.15.113:0/2080280000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c8153420 con 0x7f74c80cb540 2024-03-04T08:17:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.791+0000 7f74c7e00640 1 -- 172.21.15.113:0/2080280000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c8075a80 con 0x7f74c80cb540 2024-03-04T08:17:40.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.791+0000 7f74bfe00640 1 -- 172.21.15.113:0/2080280000 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74b001a070 con 0x7f74c80cb540 2024-03-04T08:17:40.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.791+0000 7f74bfe00640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f74a4066f90 0x7f74a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:40.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.791+0000 7f74bfe00640 1 -- 172.21.15.113:0/2080280000 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f74b0084b80 con 0x7f74c80cb540 2024-03-04T08:17:40.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.795+0000 7f74c6a00640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f74a4066f90 0x7f74a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:40.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.795+0000 7f74c6a00640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f74a4066f90 0x7f74a4069450 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f74b80113e0 tx=0x7f74b800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:40.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:40.795+0000 7f74bfe00640 1 -- 172.21.15.113:0/2080280000 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f74b0054be0 con 0x7f74c80cb540 2024-03-04T08:17:41.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 -- 172.21.15.113:0/2080280000 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74c80c9580 con 0x7f74c80cb540 2024-03-04T08:17:41.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74bfe00640 1 -- 172.21.15.113:0/2080280000 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f74b0058c80 con 0x7f74c80cb540 2024-03-04T08:17:41.048 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:41.048 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7004]},"id":1}]} 2024-03-04T08:17:41.049 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 156 2024-03-04T08:17:41.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 -- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f74a4066f90 msgr2=0x7f74a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f74a4066f90 0x7f74a4069450 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f74b80113e0 tx=0x7f74b800b040 comp rx=0 tx=0).stop 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 -- 172.21.15.113:0/2080280000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 msgr2=0x7f74c81eff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 0x7f74c81eff00 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f74b000c3c0 tx=0x7f74b000c880 comp rx=0 tx=0).stop 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 -- 172.21.15.113:0/2080280000 shutdown_connections 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f74c8152950 0x7f74c81f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f74a4066f90 0x7f74a4069450 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74c80cb540 0x7f74c81eff00 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 --2- 172.21.15.113:0/2080280000 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f74c806faa0 0x7f74c81e5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 -- 172.21.15.113:0/2080280000 >> 172.21.15.113:0/2080280000 conn(0x7f74c8072f40 msgr2=0x7f74c8158860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.051+0000 7f74c7e00640 1 -- 172.21.15.113:0/2080280000 shutdown_connections 2024-03-04T08:17:41.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:41.055+0000 7f74c7e00640 1 -- 172.21.15.113:0/2080280000 wait complete. 2024-03-04T08:17:41.073 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:17:41.668 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:41.668 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 69 -eq 48 ] 2024-03-04T08:17:41.668 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.69 2024-03-04T08:17:41.668 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.69 2024-03-04T08:17:41.669 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.69 2024-03-04T08:17:41.669 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.69 2024-03-04T08:17:41.669 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:43.074 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:43.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4968600640 1 Processor -- start 2024-03-04T08:17:43.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4968600640 1 -- start start 2024-03-04T08:17:43.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4968600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4960154c00 0x7f49600c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:43.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4968600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49600c7700 0x7f496015abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:43.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4968600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49600c7d80 con 0x7f4960154fe0 2024-03-04T08:17:43.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4968600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496015efb0 con 0x7f49600c7700 2024-03-04T08:17:43.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4968600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49600c6b70 con 0x7f4960154c00 2024-03-04T08:17:43.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4965200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4960154fe0 0x7f49600c1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36426/0 (socket says 172.21.15.113:36426) 2024-03-04T08:17:43.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4965200640 1 -- 172.21.15.113:0/3807704030 learned_addr learned my addr 172.21.15.113:0/3807704030 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:17:43.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4966600640 1 --2- 172.21.15.113:0/3807704030 >> v2:172.21.15.113:3300/0 conn(0x7f49600c7700 0x7f496015abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:43.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4965c00640 1 --2- 172.21.15.113:0/3807704030 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4960154c00 0x7f49600c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:43.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4966600640 1 -- 172.21.15.113:0/3807704030 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4960154c00 msgr2=0x7f49600c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:17:43.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4966600640 1 --2- 172.21.15.113:0/3807704030 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4960154c00 0x7f49600c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:43.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4966600640 1 -- 172.21.15.113:0/3807704030 >> v1:172.21.15.113:6789/0 conn(0x7f4960154fe0 legacy=0x7f49600c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:43.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f4966600640 1 -- 172.21.15.113:0/3807704030 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49600c8060 con 0x7f49600c7700 2024-03-04T08:17:43.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.219+0000 7f495fe00640 1 -- 172.21.15.113:0/3807704030 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119002452 0 0) 0x7f49600c7d80 con 0x7f4960154fe0 2024-03-04T08:17:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f4966600640 1 --2- 172.21.15.113:0/3807704030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49600c7700 0x7f496015abe0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4958009d90 tx=0x7f495801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ba776c1b04cff9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f495fe00640 1 -- 172.21.15.113:0/3807704030 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49580b2070 con 0x7f49600c7700 2024-03-04T08:17:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f495fe00640 1 -- 172.21.15.113:0/3807704030 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49580a5890 con 0x7f49600c7700 2024-03-04T08:17:43.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f495fe00640 1 -- 172.21.15.113:0/3807704030 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49580a5c60 con 0x7f49600c7700 2024-03-04T08:17:43.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f4968600640 1 -- 172.21.15.113:0/3807704030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49600c7700 msgr2=0x7f496015abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:43.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f4968600640 1 --2- 172.21.15.113:0/3807704030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49600c7700 0x7f496015abe0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f4958009d90 tx=0x7f495801c8e0 comp rx=0 tx=0).stop 2024-03-04T08:17:43.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f4968600640 1 -- 172.21.15.113:0/3807704030 shutdown_connections 2024-03-04T08:17:43.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f4968600640 1 --2- 172.21.15.113:0/3807704030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49600c7700 0x7f496015abe0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:43.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f4968600640 1 --2- 172.21.15.113:0/3807704030 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4960154c00 0x7f49600c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:43.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f4968600640 1 -- 172.21.15.113:0/3807704030 >> 172.21.15.113:0/3807704030 conn(0x7f496006e820 msgr2=0x7f496006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:43.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f4968600640 1 -- 172.21.15.113:0/3807704030 shutdown_connections 2024-03-04T08:17:43.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.223+0000 7f4968600640 1 -- 172.21.15.113:0/3807704030 wait complete. 2024-03-04T08:17:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4968600640 1 Processor -- start 2024-03-04T08:17:43.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4968600640 1 -- start start 2024-03-04T08:17:43.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4968600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49600c7700 0x7f49600c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:43.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4968600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4960154c00 0x7f49601f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4968600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4960154fe0 0x7f49600c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4968600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49600c23e0 con 0x7f4960154fe0 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4968600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49600c2260 con 0x7f49600c7700 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4968600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4960167c10 con 0x7f4960154c00 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4966600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4960154fe0 0x7f49600c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4966600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4960154fe0 0x7f49600c3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46006/0 (socket says 172.21.15.113:46006) 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4966600640 1 -- 172.21.15.113:0/310656057 learned_addr learned my addr 172.21.15.113:0/310656057 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4965c00640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49600c7700 0x7f49600c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4965200640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4960154c00 0x7f49601f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4966600640 1 -- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4960154c00 msgr2=0x7f49601f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4966600640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4960154c00 0x7f49601f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4966600640 1 -- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49600c7700 msgr2=0x7f49600c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4966600640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49600c7700 0x7f49600c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:43.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4966600640 1 -- 172.21.15.113:0/310656057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49601f7e70 con 0x7f4960154fe0 2024-03-04T08:17:43.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4966600640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4960154fe0 0x7f49600c3a60 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f49580b0040 tx=0x7f4958019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:43.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f495d600640 1 -- 172.21.15.113:0/310656057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49580b2050 con 0x7f4960154fe0 2024-03-04T08:17:43.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f495d600640 1 -- 172.21.15.113:0/310656057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49580be070 con 0x7f4960154fe0 2024-03-04T08:17:43.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f495d600640 1 -- 172.21.15.113:0/310656057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49580ad850 con 0x7f4960154fe0 2024-03-04T08:17:43.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4968600640 1 -- 172.21.15.113:0/310656057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49600c63f0 con 0x7f4960154fe0 2024-03-04T08:17:43.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.227+0000 7f4968600640 1 -- 172.21.15.113:0/310656057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4960150650 con 0x7f4960154fe0 2024-03-04T08:17:43.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.231+0000 7f4968600640 1 -- 172.21.15.113:0/310656057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4960075c00 con 0x7f4960154fe0 2024-03-04T08:17:43.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.231+0000 7f495d600640 1 -- 172.21.15.113:0/310656057 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49580ad9f0 con 0x7f4960154fe0 2024-03-04T08:17:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.231+0000 7f495d600640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4940066f90 0x7f4940069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.231+0000 7f495d600640 1 -- 172.21.15.113:0/310656057 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f4958120760 con 0x7f4960154fe0 2024-03-04T08:17:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.231+0000 7f4965c00640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4940066f90 0x7f4940069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.235+0000 7f4965c00640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4940066f90 0x7f4940069450 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4954009ac0 tx=0x7f4954009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.235+0000 7f495d600640 1 -- 172.21.15.113:0/310656057 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4958016560 con 0x7f4960154fe0 2024-03-04T08:17:43.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.443+0000 7f4968600640 1 -- 172.21.15.113:0/310656057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49600c12b0 con 0x7f4960154fe0 2024-03-04T08:17:43.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.443+0000 7f495d600640 1 -- 172.21.15.113:0/310656057 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f49580f0a00 con 0x7f4960154fe0 2024-03-04T08:17:43.440 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:43.440 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7004]},"id":1}]} 2024-03-04T08:17:43.440 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 156 2024-03-04T08:17:43.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.443+0000 7f4968600640 1 -- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4940066f90 msgr2=0x7f4940069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:43.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.443+0000 7f4968600640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4940066f90 0x7f4940069450 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4954009ac0 tx=0x7f4954009190 comp rx=0 tx=0).stop 2024-03-04T08:17:43.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.443+0000 7f4968600640 1 -- 172.21.15.113:0/310656057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4960154fe0 msgr2=0x7f49600c3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:43.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.447+0000 7f4968600640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4960154fe0 0x7f49600c3a60 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f49580b0040 tx=0x7f4958019680 comp rx=0 tx=0).stop 2024-03-04T08:17:43.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.447+0000 7f4968600640 1 -- 172.21.15.113:0/310656057 shutdown_connections 2024-03-04T08:17:43.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.447+0000 7f4968600640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4940066f90 0x7f4940069450 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:43.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.447+0000 7f4968600640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4960154fe0 0x7f49600c3a60 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:43.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.447+0000 7f4968600640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4960154c00 0x7f49601f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:43.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.447+0000 7f4968600640 1 --2- 172.21.15.113:0/310656057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49600c7700 0x7f49600c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:43.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.447+0000 7f4968600640 1 -- 172.21.15.113:0/310656057 >> 172.21.15.113:0/310656057 conn(0x7f496006e820 msgr2=0x7f49600c8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:43.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.447+0000 7f4968600640 1 -- 172.21.15.113:0/310656057 shutdown_connections 2024-03-04T08:17:43.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:43.447+0000 7f4968600640 1 -- 172.21.15.113:0/310656057 wait complete. 2024-03-04T08:17:43.462 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:17:44.581 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:44.581 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.70 2024-03-04T08:17:44.581 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 70 -eq 48 ] 2024-03-04T08:17:44.582 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.70 2024-03-04T08:17:44.582 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.70 2024-03-04T08:17:44.582 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.70 2024-03-04T08:17:44.582 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:45.463 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe087200640 1 Processor -- start 2024-03-04T08:17:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe087200640 1 -- start start 2024-03-04T08:17:45.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe087200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 0x7fe0880c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:45.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe087200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe0881581d0 0x7fe08806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:45.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe087200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe08806ed80 0x7fe08806f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:45.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe087200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08806dc00 con 0x7fe08806ed80 2024-03-04T08:17:45.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe087200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08806de30 con 0x7fe0881581d0 2024-03-04T08:17:45.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe087200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08806e110 con 0x7fe088157df0 2024-03-04T08:17:45.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe086800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe08806ed80 0x7fe08806f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:45.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe086800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe08806ed80 0x7fe08806f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46016/0 (socket says 172.21.15.113:46016) 2024-03-04T08:17:45.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe086800640 1 -- 172.21.15.113:0/3892789431 learned_addr learned my addr 172.21.15.113:0/3892789431 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:45.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe07d400640 1 --2- 172.21.15.113:0/3892789431 >> v2:172.21.15.136:3300/0 conn(0x7fe0881581d0 0x7fe08806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:45.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe085e00640 1 --2- 172.21.15.113:0/3892789431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 0x7fe0880c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:45.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe086800640 1 -- 172.21.15.113:0/3892789431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 msgr2=0x7fe0880c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:17:45.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe086800640 1 --2- 172.21.15.113:0/3892789431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 0x7fe0880c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe086800640 1 -- 172.21.15.113:0/3892789431 >> v2:172.21.15.136:3300/0 conn(0x7fe0881581d0 msgr2=0x7fe08806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:17:45.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe086800640 1 --2- 172.21.15.113:0/3892789431 >> v2:172.21.15.136:3300/0 conn(0x7fe0881581d0 0x7fe08806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe086800640 1 -- 172.21.15.113:0/3892789431 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0880ce0c0 con 0x7fe08806ed80 2024-03-04T08:17:45.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.675+0000 7fe07d400640 1 --2- 172.21.15.113:0/3892789431 >> v2:172.21.15.136:3300/0 conn(0x7fe0881581d0 0x7fe08806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:17:45.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe086800640 1 --2- 172.21.15.113:0/3892789431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe08806ed80 0x7fe08806f200 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fe0880cbbd0 tx=0x7fe08001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4203c7a18440a37f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe085400640 1 -- 172.21.15.113:0/3892789431 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0800b4070 con 0x7fe08806ed80 2024-03-04T08:17:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe085400640 1 -- 172.21.15.113:0/3892789431 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0800af400 con 0x7fe08806ed80 2024-03-04T08:17:45.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe085400640 1 -- 172.21.15.113:0/3892789431 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0800af7d0 con 0x7fe08806ed80 2024-03-04T08:17:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe087200640 1 -- 172.21.15.113:0/3892789431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe08806ed80 msgr2=0x7fe08806f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe087200640 1 --2- 172.21.15.113:0/3892789431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe08806ed80 0x7fe08806f200 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fe0880cbbd0 tx=0x7fe08001f1e0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe087200640 1 -- 172.21.15.113:0/3892789431 shutdown_connections 2024-03-04T08:17:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe087200640 1 --2- 172.21.15.113:0/3892789431 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe08806ed80 0x7fe08806f200 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe087200640 1 --2- 172.21.15.113:0/3892789431 >> v2:172.21.15.136:3300/0 conn(0x7fe0881581d0 0x7fe08806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe087200640 1 --2- 172.21.15.113:0/3892789431 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 0x7fe0880c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe087200640 1 -- 172.21.15.113:0/3892789431 >> 172.21.15.113:0/3892789431 conn(0x7fe088072f40 msgr2=0x7fe0880706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe087200640 1 -- 172.21.15.113:0/3892789431 shutdown_connections 2024-03-04T08:17:45.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.679+0000 7fe087200640 1 -- 172.21.15.113:0/3892789431 wait complete. 2024-03-04T08:17:45.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe087200640 1 Processor -- start 2024-03-04T08:17:45.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe087200640 1 -- start start 2024-03-04T08:17:45.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe087200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe08806ed80 0x7fe0880ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:45.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe087200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 0x7fe088155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:45.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe087200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0881581d0 0x7fe0880cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:45.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe087200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08806fd90 con 0x7fe0881581d0 2024-03-04T08:17:45.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe087200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08806fc10 con 0x7fe08806ed80 2024-03-04T08:17:45.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe087200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880c9be0 con 0x7fe088157df0 2024-03-04T08:17:45.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe086800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0881581d0 0x7fe0880cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:45.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe086800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0881581d0 0x7fe0880cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46026/0 (socket says 172.21.15.113:46026) 2024-03-04T08:17:45.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe086800640 1 -- 172.21.15.113:0/1963004574 learned_addr learned my addr 172.21.15.113:0/1963004574 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:45.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe085e00640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe08806ed80 0x7fe0880ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:45.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.683+0000 7fe07d400640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 0x7fe088155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:45.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe086800640 1 -- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 msgr2=0x7fe088155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:45.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe086800640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 0x7fe088155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe086800640 1 -- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe08806ed80 msgr2=0x7fe0880ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:45.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe086800640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe08806ed80 0x7fe0880ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe086800640 1 -- 172.21.15.113:0/1963004574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0880cdb40 con 0x7fe0881581d0 2024-03-04T08:17:45.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe07d400640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 0x7fe088155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:17:45.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe086800640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0881581d0 0x7fe0880cc300 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fe08001fa80 tx=0x7fe08001bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:45.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe07ea00640 1 -- 172.21.15.113:0/1963004574 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0800c0070 con 0x7fe0881581d0 2024-03-04T08:17:45.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe07ea00640 1 -- 172.21.15.113:0/1963004574 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0800090d0 con 0x7fe0881581d0 2024-03-04T08:17:45.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe087200640 1 -- 172.21.15.113:0/1963004574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0880cde20 con 0x7fe0881581d0 2024-03-04T08:17:45.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe07ea00640 1 -- 172.21.15.113:0/1963004574 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0800b4040 con 0x7fe0881581d0 2024-03-04T08:17:45.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe087200640 1 -- 172.21.15.113:0/1963004574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe088150500 con 0x7fe0881581d0 2024-03-04T08:17:45.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe087200640 1 -- 172.21.15.113:0/1963004574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe088075a80 con 0x7fe0881581d0 2024-03-04T08:17:45.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.687+0000 7fe07ea00640 1 -- 172.21.15.113:0/1963004574 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0800c3070 con 0x7fe0881581d0 2024-03-04T08:17:45.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.691+0000 7fe07ea00640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe064066f90 0x7fe064069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:45.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.691+0000 7fe07ea00640 1 -- 172.21.15.113:0/1963004574 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7fe080122f90 con 0x7fe0881581d0 2024-03-04T08:17:45.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.691+0000 7fe085e00640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe064066f90 0x7fe064069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:45.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.691+0000 7fe085e00640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe064066f90 0x7fe064069450 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe07000ba30 tx=0x7fe070008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:45.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.695+0000 7fe07ea00640 1 -- 172.21.15.113:0/1963004574 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe080018d40 con 0x7fe0881581d0 2024-03-04T08:17:45.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.927+0000 7fe087200640 1 -- 172.21.15.113:0/1963004574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0880c97b0 con 0x7fe0881581d0 2024-03-04T08:17:45.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.927+0000 7fe07ea00640 1 -- 172.21.15.113:0/1963004574 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+6624 (secure 0 0 0) 0x7fe0800f36e0 con 0x7fe0881581d0 2024-03-04T08:17:45.926 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:45.926 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7004]},"id":1}]} 2024-03-04T08:17:45.926 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 156 2024-03-04T08:17:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 -- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe064066f90 msgr2=0x7fe064069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe064066f90 0x7fe064069450 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe07000ba30 tx=0x7fe070008040 comp rx=0 tx=0).stop 2024-03-04T08:17:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 -- 172.21.15.113:0/1963004574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0881581d0 msgr2=0x7fe0880cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0881581d0 0x7fe0880cc300 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fe08001fa80 tx=0x7fe08001bf80 comp rx=0 tx=0).stop 2024-03-04T08:17:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 -- 172.21.15.113:0/1963004574 shutdown_connections 2024-03-04T08:17:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe064066f90 0x7fe064069450 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0881581d0 0x7fe0880cc300 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe088157df0 0x7fe088155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 --2- 172.21.15.113:0/1963004574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe08806ed80 0x7fe0880ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 -- 172.21.15.113:0/1963004574 >> 172.21.15.113:0/1963004574 conn(0x7fe088072f40 msgr2=0x7fe0880ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 -- 172.21.15.113:0/1963004574 shutdown_connections 2024-03-04T08:17:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:45.931+0000 7fe087200640 1 -- 172.21.15.113:0/1963004574 wait complete. 2024-03-04T08:17:45.951 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:17:47.952 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.091+0000 7f6653c00640 1 Processor -- start 2024-03-04T08:17:48.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.091+0000 7f6653c00640 1 -- start start 2024-03-04T08:17:48.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6653c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f664c154c00 0x7f664c0c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:48.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6653c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f664c154fe0 0x7f664c0c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:48.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6653c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c0c7d80 con 0x7f664c0c7700 2024-03-04T08:17:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6653c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c15efb0 con 0x7f664c154c00 2024-03-04T08:17:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6653c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c0c6b70 con 0x7f664c154fe0 2024-03-04T08:17:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6651200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f664c154c00 0x7f664c0c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6651200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f664c154c00 0x7f664c0c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57430/0 (socket says 172.21.15.113:57430) 2024-03-04T08:17:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6651200640 1 -- 172.21.15.113:0/3994776232 learned_addr learned my addr 172.21.15.113:0/3994776232 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f664be00640 1 --2- 172.21.15.113:0/3994776232 >> v2:172.21.15.136:3300/0 conn(0x7f664c154fe0 0x7f664c0c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:48.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6651200640 1 -- 172.21.15.113:0/3994776232 >> v2:172.21.15.136:3300/0 conn(0x7f664c154fe0 msgr2=0x7f664c0c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:17:48.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6651200640 1 --2- 172.21.15.113:0/3994776232 >> v2:172.21.15.136:3300/0 conn(0x7f664c154fe0 0x7f664c0c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:48.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6651200640 1 -- 172.21.15.113:0/3994776232 >> v1:172.21.15.136:6789/0 conn(0x7f664c0c7700 legacy=0x7f664c15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:17:48.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f6651200640 1 -- 172.21.15.113:0/3994776232 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c0c8060 con 0x7f664c154c00 2024-03-04T08:17:48.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.095+0000 7f664be00640 1 --2- 172.21.15.113:0/3994776232 >> v2:172.21.15.136:3300/0 conn(0x7f664c154fe0 0x7f664c0c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:17:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6651200640 1 --2- 172.21.15.113:0/3994776232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c154c00 0x7f664c0c1190 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f664c15bc00 tx=0x7f664001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf9ffafb81d76e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f664b400640 1 -- 172.21.15.113:0/3994776232 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66400b2070 con 0x7f664c154c00 2024-03-04T08:17:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f664b400640 1 -- 172.21.15.113:0/3994776232 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66400ad680 con 0x7f664c154c00 2024-03-04T08:17:48.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f664b400640 1 -- 172.21.15.113:0/3994776232 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66400ada50 con 0x7f664c154c00 2024-03-04T08:17:48.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6653c00640 1 -- 172.21.15.113:0/3994776232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c154c00 msgr2=0x7f664c0c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:48.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6653c00640 1 --2- 172.21.15.113:0/3994776232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c154c00 0x7f664c0c1190 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f664c15bc00 tx=0x7f664001c490 comp rx=0 tx=0).stop 2024-03-04T08:17:48.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6653c00640 1 -- 172.21.15.113:0/3994776232 shutdown_connections 2024-03-04T08:17:48.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6653c00640 1 --2- 172.21.15.113:0/3994776232 >> v2:172.21.15.136:3300/0 conn(0x7f664c154fe0 0x7f664c0c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:48.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6653c00640 1 --2- 172.21.15.113:0/3994776232 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c154c00 0x7f664c0c1190 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:48.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6653c00640 1 -- 172.21.15.113:0/3994776232 >> 172.21.15.113:0/3994776232 conn(0x7f664c06e820 msgr2=0x7f664c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6653c00640 1 -- 172.21.15.113:0/3994776232 shutdown_connections 2024-03-04T08:17:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6653c00640 1 -- 172.21.15.113:0/3994776232 wait complete. 2024-03-04T08:17:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6653c00640 1 Processor -- start 2024-03-04T08:17:48.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.099+0000 7f6653c00640 1 -- start start 2024-03-04T08:17:48.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6653c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c0c7700 0x7f664c15cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:48.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6653c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f664c154c00 0x7f664c0aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:48.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6653c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f664c154fe0 0x7f664c0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:48.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6653c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0c2460 con 0x7f664c0c7700 2024-03-04T08:17:48.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6653c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0c22e0 con 0x7f664c154fe0 2024-03-04T08:17:48.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6653c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c167c10 con 0x7f664c154c00 2024-03-04T08:17:48.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6651200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c0c7700 0x7f664c15cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:48.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6651200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c0c7700 0x7f664c15cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57444/0 (socket says 172.21.15.113:57444) 2024-03-04T08:17:48.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6651200640 1 -- 172.21.15.113:0/3935431778 learned_addr learned my addr 172.21.15.113:0/3935431778 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:48.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6651c00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f664c154fe0 0x7f664c0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:48.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f664be00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f664c154c00 0x7f664c0aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:48.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6651200640 1 -- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f664c154c00 msgr2=0x7f664c0aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:48.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6651200640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f664c154c00 0x7f664c0aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:48.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6651200640 1 -- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f664c154fe0 msgr2=0x7f664c0af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:48.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6651200640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f664c154fe0 0x7f664c0af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:48.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6651200640 1 -- 172.21.15.113:0/3935431778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c15bf20 con 0x7f664c0c7700 2024-03-04T08:17:48.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6651200640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c0c7700 0x7f664c15cce0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f664001a0c0 tx=0x7f664001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:48.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6648c00640 1 -- 172.21.15.113:0/3935431778 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66400b2040 con 0x7f664c0c7700 2024-03-04T08:17:48.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6648c00640 1 -- 172.21.15.113:0/3935431778 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66400bd070 con 0x7f664c0c7700 2024-03-04T08:17:48.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6653c00640 1 -- 172.21.15.113:0/3935431778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c15d220 con 0x7f664c0c7700 2024-03-04T08:17:48.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6648c00640 1 -- 172.21.15.113:0/3935431778 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66400b8450 con 0x7f664c0c7700 2024-03-04T08:17:48.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6653c00640 1 -- 172.21.15.113:0/3935431778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c1f3e00 con 0x7f664c0c7700 2024-03-04T08:17:48.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.103+0000 7f6653c00640 1 -- 172.21.15.113:0/3935431778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c1f3c10 con 0x7f664c0c7700 2024-03-04T08:17:48.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.107+0000 7f6648c00640 1 -- 172.21.15.113:0/3935431778 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66400b85f0 con 0x7f664c0c7700 2024-03-04T08:17:48.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.107+0000 7f6648c00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f662c066fe0 0x7f662c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:48.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.107+0000 7f6648c00640 1 -- 172.21.15.113:0/3935431778 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f664011f900 con 0x7f664c0c7700 2024-03-04T08:17:48.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.107+0000 7f664be00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f662c066fe0 0x7f662c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:48.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.107+0000 7f664be00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f662c066fe0 0x7f662c0694a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f664c14e250 tx=0x7f6638008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:48.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.115+0000 7f6648c00640 1 -- 172.21.15.113:0/3935431778 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6640016110 con 0x7f664c0c7700 2024-03-04T08:17:48.272 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:48.273 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 71 -eq 48 ] 2024-03-04T08:17:48.273 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.71 2024-03-04T08:17:48.273 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.71 2024-03-04T08:17:48.273 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.71 2024-03-04T08:17:48.274 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.71 2024-03-04T08:17:48.274 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:48.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.411+0000 7f6653c00640 1 -- 172.21.15.113:0/3935431778 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f664c075c00 con 0x7f664c0c7700 2024-03-04T08:17:48.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.411+0000 7f6648c00640 1 -- 172.21.15.113:0/3935431778 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f66400f00c0 con 0x7f664c0c7700 2024-03-04T08:17:48.409 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:48.410 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7004]},"id":1}]} 2024-03-04T08:17:48.410 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 156 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 -- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f662c066fe0 msgr2=0x7f662c0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f662c066fe0 0x7f662c0694a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f664c14e250 tx=0x7f6638008040 comp rx=0 tx=0).stop 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 -- 172.21.15.113:0/3935431778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c0c7700 msgr2=0x7f664c15cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c0c7700 0x7f664c15cce0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f664001a0c0 tx=0x7f664001acd0 comp rx=0 tx=0).stop 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 -- 172.21.15.113:0/3935431778 shutdown_connections 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f664c154fe0 0x7f664c0af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f664c154c00 0x7f664c0aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f662c066fe0 0x7f662c0694a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 --2- 172.21.15.113:0/3935431778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f664c0c7700 0x7f664c15cce0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 -- 172.21.15.113:0/3935431778 >> 172.21.15.113:0/3935431778 conn(0x7f664c06e820 msgr2=0x7f664c0cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:48.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 -- 172.21.15.113:0/3935431778 shutdown_connections 2024-03-04T08:17:48.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:48.415+0000 7f6653c00640 1 -- 172.21.15.113:0/3935431778 wait complete. 2024-03-04T08:17:48.442 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:17:50.443 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:50.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f0125000640 1 Processor -- start 2024-03-04T08:17:50.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f0125000640 1 -- start start 2024-03-04T08:17:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f0125000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01201527f0 0x7f0120070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f0125000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0120152bd0 0x7f01201587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f0125000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01201591f0 con 0x7f01200c8200 2024-03-04T08:17:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f0125000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0120071df0 con 0x7f01201527f0 2024-03-04T08:17:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f0125000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01201597b0 con 0x7f0120152bd0 2024-03-04T08:17:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01201527f0 0x7f0120070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011d400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f01200c8200 0x7f0120072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47866/0 (socket says 172.21.15.113:47866) 2024-03-04T08:17:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f01201527f0 0x7f0120070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57456/0 (socket says 172.21.15.113:57456) 2024-03-04T08:17:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011d400640 1 -- 172.21.15.113:0/1267887018 learned_addr learned my addr 172.21.15.113:0/1267887018 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:17:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011e800640 1 --2- 172.21.15.113:0/1267887018 >> v2:172.21.15.136:3300/0 conn(0x7f0120152bd0 0x7f01201587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011de00640 1 -- 172.21.15.113:0/1267887018 >> v2:172.21.15.136:3300/0 conn(0x7f0120152bd0 msgr2=0x7f01201587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:17:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011de00640 1 --2- 172.21.15.113:0/1267887018 >> v2:172.21.15.136:3300/0 conn(0x7f0120152bd0 0x7f01201587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011de00640 1 -- 172.21.15.113:0/1267887018 >> v1:172.21.15.113:6789/0 conn(0x7f01200c8200 legacy=0x7f0120072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011de00640 1 -- 172.21.15.113:0/1267887018 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01201594d0 con 0x7f01201527f0 2024-03-04T08:17:50.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011e800640 1 --2- 172.21.15.113:0/1267887018 >> v2:172.21.15.136:3300/0 conn(0x7f0120152bd0 0x7f01201587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:17:50.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.587+0000 7f011ca00640 1 -- 172.21.15.113:0/1267887018 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969727855 0 0) 0x7f01201591f0 con 0x7f01200c8200 2024-03-04T08:17:50.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f011de00640 1 --2- 172.21.15.113:0/1267887018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 0x7f0120070d40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f01200712b0 tx=0x7f011001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a0ad3aa519c7da server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f011ca00640 1 -- 172.21.15.113:0/1267887018 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01100b1070 con 0x7f01201527f0 2024-03-04T08:17:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f011ca00640 1 -- 172.21.15.113:0/1267887018 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01100ac5f0 con 0x7f01201527f0 2024-03-04T08:17:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f011ca00640 1 -- 172.21.15.113:0/1267887018 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01100ac9c0 con 0x7f01201527f0 2024-03-04T08:17:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f0125000640 1 -- 172.21.15.113:0/1267887018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 msgr2=0x7f0120070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f0125000640 1 --2- 172.21.15.113:0/1267887018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 0x7f0120070d40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f01200712b0 tx=0x7f011001c480 comp rx=0 tx=0).stop 2024-03-04T08:17:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f0125000640 1 -- 172.21.15.113:0/1267887018 shutdown_connections 2024-03-04T08:17:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f0125000640 1 --2- 172.21.15.113:0/1267887018 >> v2:172.21.15.136:3300/0 conn(0x7f0120152bd0 0x7f01201587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f0125000640 1 --2- 172.21.15.113:0/1267887018 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 0x7f0120070d40 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f0125000640 1 -- 172.21.15.113:0/1267887018 >> 172.21.15.113:0/1267887018 conn(0x7f012006e820 msgr2=0x7f012006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f0125000640 1 -- 172.21.15.113:0/1267887018 shutdown_connections 2024-03-04T08:17:50.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.591+0000 7f0125000640 1 -- 172.21.15.113:0/1267887018 wait complete. 2024-03-04T08:17:50.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0125000640 1 Processor -- start 2024-03-04T08:17:50.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0125000640 1 -- start start 2024-03-04T08:17:50.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0125000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01200c8200 0x7f01200cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0125000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 0x7f01201501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0125000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120152bd0 0x7f01201506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0125000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01201654f0 con 0x7f01201527f0 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0125000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0120165370 con 0x7f0120152bd0 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0125000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0120165670 con 0x7f01200c8200 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 0x7f01201501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 0x7f01201501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57468/0 (socket says 172.21.15.113:57468) 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011d400640 1 -- 172.21.15.113:0/1022146225 learned_addr learned my addr 172.21.15.113:0/1022146225 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011de00640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01200c8200 0x7f01200cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011e800640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120152bd0 0x7f01201506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011d400640 1 -- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01200c8200 msgr2=0x7f01200cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011d400640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01200c8200 0x7f01200cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011d400640 1 -- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120152bd0 msgr2=0x7f01201506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:50.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011d400640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120152bd0 0x7f01201506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011d400640 1 -- 172.21.15.113:0/1022146225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01201f3c90 con 0x7f01201527f0 2024-03-04T08:17:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f011d400640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 0x7f01201501b0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f010c00c4b0 tx=0x7f010c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0116000640 1 -- 172.21.15.113:0/1022146225 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c018070 con 0x7f01201527f0 2024-03-04T08:17:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0116000640 1 -- 172.21.15.113:0/1022146225 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c007500 con 0x7f01201527f0 2024-03-04T08:17:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0125000640 1 -- 172.21.15.113:0/1022146225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01201f3f70 con 0x7f01201527f0 2024-03-04T08:17:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0116000640 1 -- 172.21.15.113:0/1022146225 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c004a30 con 0x7f01201527f0 2024-03-04T08:17:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.595+0000 7f0125000640 1 -- 172.21.15.113:0/1022146225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01201f4530 con 0x7f01201527f0 2024-03-04T08:17:50.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.599+0000 7f0125000640 1 -- 172.21.15.113:0/1022146225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0120075c00 con 0x7f01201527f0 2024-03-04T08:17:50.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.599+0000 7f0116000640 1 -- 172.21.15.113:0/1022146225 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f010c01b070 con 0x7f01201527f0 2024-03-04T08:17:50.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.599+0000 7f0116000640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0100066f60 0x7f0100069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:50.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.599+0000 7f0116000640 1 -- 172.21.15.113:0/1022146225 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f010c084a30 con 0x7f01201527f0 2024-03-04T08:17:50.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.599+0000 7f011de00640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0100066f60 0x7f0100069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:50.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.603+0000 7f011de00640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0100066f60 0x7f0100069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f01100ae000 tx=0x7f01100a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:50.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.603+0000 7f0116000640 1 -- 172.21.15.113:0/1022146225 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f010c054d10 con 0x7f01201527f0 2024-03-04T08:17:50.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.847+0000 7f0125000640 1 -- 172.21.15.113:0/1022146225 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01200c79d0 con 0x7f01201527f0 2024-03-04T08:17:50.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0116000640 1 -- 172.21.15.113:0/1022146225 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f010c058db0 con 0x7f01201527f0 2024-03-04T08:17:50.847 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:50.847 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7004]},"id":1}]} 2024-03-04T08:17:50.847 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 156 2024-03-04T08:17:50.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 -- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0100066f60 msgr2=0x7f0100069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:50.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0100066f60 0x7f0100069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f01100ae000 tx=0x7f01100a7000 comp rx=0 tx=0).stop 2024-03-04T08:17:50.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 -- 172.21.15.113:0/1022146225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 msgr2=0x7f01201501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:50.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 0x7f01201501b0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f010c00c4b0 tx=0x7f010c00c970 comp rx=0 tx=0).stop 2024-03-04T08:17:50.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 -- 172.21.15.113:0/1022146225 shutdown_connections 2024-03-04T08:17:50.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0120152bd0 0x7f01201506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:50.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0100066f60 0x7f0100069420 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:50.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01201527f0 0x7f01201501b0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:50.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 --2- 172.21.15.113:0/1022146225 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01200c8200 0x7f01200cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:50.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 -- 172.21.15.113:0/1022146225 >> 172.21.15.113:0/1022146225 conn(0x7f012006e820 msgr2=0x7f012014b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:50.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 -- 172.21.15.113:0/1022146225 shutdown_connections 2024-03-04T08:17:50.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:50.851+0000 7f0125000640 1 -- 172.21.15.113:0/1022146225 wait complete. 2024-03-04T08:17:50.870 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:17:51.792 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:51.792 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 72 -eq 48 ] 2024-03-04T08:17:51.792 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.72 2024-03-04T08:17:51.793 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.72 2024-03-04T08:17:51.793 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.72 2024-03-04T08:17:51.794 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.72 2024-03-04T08:17:51.794 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:52.872 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:53.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce15a00640 1 Processor -- start 2024-03-04T08:17:53.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce15a00640 1 -- start start 2024-03-04T08:17:53.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce15a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce1006cb30 0x7fce100c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:53.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce15a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce100cae80 con 0x7fce10154d90 2024-03-04T08:17:53.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce15a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1015e170 con 0x7fce101549b0 2024-03-04T08:17:53.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce15a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce1015e450 con 0x7fce1006cb30 2024-03-04T08:17:53.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce0f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce1006cb30 0x7fce100c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:53.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce0ea00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fce101549b0 0x7fce1015b300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:43452/0 (socket says 172.21.15.113:43452) 2024-03-04T08:17:53.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce0ea00640 1 -- 172.21.15.113:0/2958135194 learned_addr learned my addr 172.21.15.113:0/2958135194 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:17:53.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce0f400640 1 -- 172.21.15.113:0/2958135194 >> v1:172.21.15.136:6789/0 conn(0x7fce101549b0 legacy=0x7fce1015b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:53.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce0f400640 1 -- 172.21.15.113:0/2958135194 >> v1:172.21.15.113:6789/0 conn(0x7fce10154d90 legacy=0x7fce100d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:53.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.031+0000 7fce0f400640 1 -- 172.21.15.113:0/2958135194 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1015e730 con 0x7fce1006cb30 2024-03-04T08:17:53.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.035+0000 7fce0f400640 1 --2- 172.21.15.113:0/2958135194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce1006cb30 0x7fce100c58a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fce08009d90 tx=0x7fce0801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869edd823547e335 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:53.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.035+0000 7fce0d600640 1 -- 172.21.15.113:0/2958135194 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce080b4070 con 0x7fce1006cb30 2024-03-04T08:17:53.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.035+0000 7fce0d600640 1 -- 172.21.15.113:0/2958135194 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce080af600 con 0x7fce1006cb30 2024-03-04T08:17:53.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.035+0000 7fce0d600640 1 -- 172.21.15.113:0/2958135194 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce080af9d0 con 0x7fce1006cb30 2024-03-04T08:17:53.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.039+0000 7fce15a00640 1 -- 172.21.15.113:0/2958135194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce1006cb30 msgr2=0x7fce100c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:53.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.039+0000 7fce15a00640 1 --2- 172.21.15.113:0/2958135194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce1006cb30 0x7fce100c58a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fce08009d90 tx=0x7fce0801f150 comp rx=0 tx=0).stop 2024-03-04T08:17:53.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.039+0000 7fce15a00640 1 -- 172.21.15.113:0/2958135194 shutdown_connections 2024-03-04T08:17:53.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.039+0000 7fce15a00640 1 --2- 172.21.15.113:0/2958135194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce1006cb30 0x7fce100c58a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:53.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.039+0000 7fce15a00640 1 -- 172.21.15.113:0/2958135194 >> 172.21.15.113:0/2958135194 conn(0x7fce1006dc50 msgr2=0x7fce10066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:53.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.039+0000 7fce15a00640 1 -- 172.21.15.113:0/2958135194 shutdown_connections 2024-03-04T08:17:53.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.039+0000 7fce15a00640 1 -- 172.21.15.113:0/2958135194 wait complete. 2024-03-04T08:17:53.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.039+0000 7fce15a00640 1 Processor -- start 2024-03-04T08:17:53.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.039+0000 7fce15a00640 1 -- start start 2024-03-04T08:17:53.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce15a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce1006cb30 0x7fce100b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:53.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce15a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce101549b0 0x7fce101f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:53.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce15a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce10154d90 0x7fce100c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:53.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce15a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce100c5380 con 0x7fce101549b0 2024-03-04T08:17:53.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce15a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1015ecf0 con 0x7fce10154d90 2024-03-04T08:17:53.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce15a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1015de70 con 0x7fce1006cb30 2024-03-04T08:17:53.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce101549b0 0x7fce101f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:53.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce101549b0 0x7fce101f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57490/0 (socket says 172.21.15.113:57490) 2024-03-04T08:17:53.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0e000640 1 -- 172.21.15.113:0/2821781798 learned_addr learned my addr 172.21.15.113:0/2821781798 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:53.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0ea00640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce1006cb30 0x7fce100b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:53.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0f400640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce10154d90 0x7fce100c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0e000640 1 -- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce1006cb30 msgr2=0x7fce100b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0e000640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce1006cb30 0x7fce100b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0e000640 1 -- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce10154d90 msgr2=0x7fce100c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0e000640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce10154d90 0x7fce100c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0e000640 1 -- 172.21.15.113:0/2821781798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce101f3cd0 con 0x7fce101549b0 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce0e000640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce101549b0 0x7fce101f4490 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c410 tx=0x7fcdfc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce06a00640 1 -- 172.21.15.113:0/2821781798 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc017070 con 0x7fce101549b0 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce06a00640 1 -- 172.21.15.113:0/2821781798 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0052f0 con 0x7fce101549b0 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce06a00640 1 -- 172.21.15.113:0/2821781798 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc003840 con 0x7fce101549b0 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce15a00640 1 -- 172.21.15.113:0/2821781798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1015ce40 con 0x7fce101549b0 2024-03-04T08:17:53.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.043+0000 7fce15a00640 1 -- 172.21.15.113:0/2821781798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1015d400 con 0x7fce101549b0 2024-03-04T08:17:53.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.047+0000 7fce15a00640 1 -- 172.21.15.113:0/2821781798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce10075c00 con 0x7fce101549b0 2024-03-04T08:17:53.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.047+0000 7fce06a00640 1 -- 172.21.15.113:0/2821781798 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcdfc01a070 con 0x7fce101549b0 2024-03-04T08:17:53.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.047+0000 7fce06a00640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdf0066f90 0x7fcdf0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:53.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.047+0000 7fce06a00640 1 -- 172.21.15.113:0/2821781798 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7fcdfc084a60 con 0x7fce101549b0 2024-03-04T08:17:53.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.047+0000 7fce0ea00640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdf0066f90 0x7fcdf0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:53.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.047+0000 7fce0ea00640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdf0066f90 0x7fcdf0069450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fce00000f70 tx=0x7fce00007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:53.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.055+0000 7fce06a00640 1 -- 172.21.15.113:0/2821781798 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcdfc054be0 con 0x7fce101549b0 2024-03-04T08:17:53.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.331+0000 7fce15a00640 1 -- 172.21.15.113:0/2821781798 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce100ccfd0 con 0x7fce101549b0 2024-03-04T08:17:53.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce06a00640 1 -- 172.21.15.113:0/2821781798 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+6624 (secure 0 0 0) 0x7fcdfc058c80 con 0x7fce101549b0 2024-03-04T08:17:53.331 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:53.332 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7004]},"id":1}]} 2024-03-04T08:17:53.332 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 156 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 -- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdf0066f90 msgr2=0x7fcdf0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdf0066f90 0x7fcdf0069450 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fce00000f70 tx=0x7fce00007e90 comp rx=0 tx=0).stop 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 -- 172.21.15.113:0/2821781798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce101549b0 msgr2=0x7fce101f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce101549b0 0x7fce101f4490 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c410 tx=0x7fcdfc00c8d0 comp rx=0 tx=0).stop 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 -- 172.21.15.113:0/2821781798 shutdown_connections 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce10154d90 0x7fce100c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdf0066f90 0x7fcdf0069450 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce101549b0 0x7fce101f4490 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 --2- 172.21.15.113:0/2821781798 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce1006cb30 0x7fce100b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 -- 172.21.15.113:0/2821781798 >> 172.21.15.113:0/2821781798 conn(0x7fce1006dc50 msgr2=0x7fce100cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 -- 172.21.15.113:0/2821781798 shutdown_connections 2024-03-04T08:17:53.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:53.335+0000 7fce15a00640 1 -- 172.21.15.113:0/2821781798 wait complete. 2024-03-04T08:17:53.359 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:17:55.360 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:55.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a5c200640 1 Processor -- start 2024-03-04T08:17:55.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a5c200640 1 -- start start 2024-03-04T08:17:55.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a5c200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 0x7f1a5406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:55.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a5c200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a540ca0e0 con 0x7f1a540ce120 2024-03-04T08:17:55.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a5c200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5415aa30 con 0x7f1a5406ff20 2024-03-04T08:17:55.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a5c200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5406f450 con 0x7f1a54158010 2024-03-04T08:17:55.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a5a200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1a540ce120 0x7f1a541652d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47880/0 (socket says 172.21.15.113:47880) 2024-03-04T08:17:55.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a5a200640 1 -- 172.21.15.113:0/3373970757 learned_addr learned my addr 172.21.15.113:0/3373970757 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:17:55.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a59800640 1 --2- 172.21.15.113:0/3373970757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 0x7f1a5406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:55.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a53e00640 1 -- 172.21.15.113:0/3373970757 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198239559 0 0) 0x7f1a540ca0e0 con 0x7f1a540ce120 2024-03-04T08:17:55.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a53e00640 1 -- 172.21.15.113:0/3373970757 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a3c003530 con 0x7f1a540ce120 2024-03-04T08:17:55.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a59800640 1 -- 172.21.15.113:0/3373970757 >> v1:172.21.15.136:6789/0 conn(0x7f1a5406ff20 legacy=0x7f1a5415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:55.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a53e00640 1 -- 172.21.15.113:0/3373970757 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372546818 0 0) 0x7f1a5415aa30 con 0x7f1a5406ff20 2024-03-04T08:17:55.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a59800640 1 -- 172.21.15.113:0/3373970757 >> v1:172.21.15.113:6789/0 conn(0x7f1a540ce120 legacy=0x7f1a541652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:55.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.563+0000 7f1a59800640 1 -- 172.21.15.113:0/3373970757 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5406c590 con 0x7f1a54158010 2024-03-04T08:17:55.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a59800640 1 --2- 172.21.15.113:0/3373970757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 0x7f1a5406dd50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a5406f5d0 tx=0x7f1a4801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa2dd173437078a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:55.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a53e00640 1 -- 172.21.15.113:0/3373970757 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480b0070 con 0x7f1a54158010 2024-03-04T08:17:55.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a53e00640 1 -- 172.21.15.113:0/3373970757 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a480ab600 con 0x7f1a54158010 2024-03-04T08:17:55.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a53e00640 1 -- 172.21.15.113:0/3373970757 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480ab9b0 con 0x7f1a54158010 2024-03-04T08:17:55.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a5c200640 1 -- 172.21.15.113:0/3373970757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 msgr2=0x7f1a5406dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:55.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a5c200640 1 --2- 172.21.15.113:0/3373970757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 0x7f1a5406dd50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a5406f5d0 tx=0x7f1a4801c4c0 comp rx=0 tx=0).stop 2024-03-04T08:17:55.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a5c200640 1 -- 172.21.15.113:0/3373970757 shutdown_connections 2024-03-04T08:17:55.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a5c200640 1 --2- 172.21.15.113:0/3373970757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 0x7f1a5406dd50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:55.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a5c200640 1 -- 172.21.15.113:0/3373970757 >> 172.21.15.113:0/3373970757 conn(0x7f1a540734d0 msgr2=0x7f1a54066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:55.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a5c200640 1 -- 172.21.15.113:0/3373970757 shutdown_connections 2024-03-04T08:17:55.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.567+0000 7f1a5c200640 1 -- 172.21.15.113:0/3373970757 wait complete. 2024-03-04T08:17:55.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5c200640 1 Processor -- start 2024-03-04T08:17:55.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5c200640 1 -- start start 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5c200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a5406ff20 0x7f1a54147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a540ce120 0x7f1a54149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5c200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 0x7f1a54148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5415b920 con 0x7f1a540ce120 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5c200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5406e290 con 0x7f1a5406ff20 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5c200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a541583f0 con 0x7f1a54158010 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a58e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a540ce120 0x7f1a54149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a58e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a540ce120 0x7f1a54149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57498/0 (socket says 172.21.15.113:57498) 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a58e00640 1 -- 172.21.15.113:0/3404889717 learned_addr learned my addr 172.21.15.113:0/3404889717 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a59800640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a5406ff20 0x7f1a54147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5a200640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 0x7f1a54148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:55.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a58e00640 1 -- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 msgr2=0x7f1a54148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:55.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a58e00640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 0x7f1a54148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:55.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a58e00640 1 -- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a5406ff20 msgr2=0x7f1a54147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:55.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a58e00640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a5406ff20 0x7f1a54147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:55.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a58e00640 1 -- 172.21.15.113:0/3404889717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a54159130 con 0x7f1a540ce120 2024-03-04T08:17:55.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5a200640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 0x7f1a54148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:17:55.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a58e00640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a540ce120 0x7f1a54149d90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000c580 tx=0x7f1a4000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:55.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a51600640 1 -- 172.21.15.113:0/3404889717 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40017070 con 0x7f1a540ce120 2024-03-04T08:17:55.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a51600640 1 -- 172.21.15.113:0/3404889717 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a40003600 con 0x7f1a540ce120 2024-03-04T08:17:55.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5c200640 1 -- 172.21.15.113:0/3404889717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a54159410 con 0x7f1a540ce120 2024-03-04T08:17:55.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a51600640 1 -- 172.21.15.113:0/3404889717 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a400039f0 con 0x7f1a540ce120 2024-03-04T08:17:55.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.571+0000 7f1a5c200640 1 -- 172.21.15.113:0/3404889717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a54149590 con 0x7f1a540ce120 2024-03-04T08:17:55.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.575+0000 7f1a5c200640 1 -- 172.21.15.113:0/3404889717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a54075c00 con 0x7f1a540ce120 2024-03-04T08:17:55.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.575+0000 7f1a51600640 1 -- 172.21.15.113:0/3404889717 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a4001a070 con 0x7f1a540ce120 2024-03-04T08:17:55.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.575+0000 7f1a51600640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a34066f90 0x7f1a34069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:55.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.575+0000 7f1a51600640 1 -- 172.21.15.113:0/3404889717 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7803+0+0 (secure 0 0 0) 0x7f1a40084ca0 con 0x7f1a540ce120 2024-03-04T08:17:55.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.575+0000 7f1a59800640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a34066f90 0x7f1a34069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:55.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.579+0000 7f1a59800640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a34066f90 0x7f1a34069450 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1a480ad000 tx=0x7f1a480a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:55.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.579+0000 7f1a51600640 1 -- 172.21.15.113:0/3404889717 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1a40054da0 con 0x7f1a540ce120 2024-03-04T08:17:55.662 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:55.663 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.73 2024-03-04T08:17:55.663 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 73 -eq 48 ] 2024-03-04T08:17:55.663 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.73 2024-03-04T08:17:55.663 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.73 2024-03-04T08:17:55.663 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.73 2024-03-04T08:17:55.663 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:17:55.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.823+0000 7f1a5c200640 1 -- 172.21.15.113:0/3404889717 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a5406fd00 con 0x7f1a540ce120 2024-03-04T08:17:55.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.823+0000 7f1a51600640 1 -- 172.21.15.113:0/3404889717 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+6624 (secure 0 0 0) 0x7f1a40058e40 con 0x7f1a540ce120 2024-03-04T08:17:55.822 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:55.823 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":156}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:16:22.420719+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":7004,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7004":{"gid":7004,"name":"a","rank":1,"incarnation":151,"state":"up:active","state_seq":34,"addr":"172.21.15.113:6836/4083765155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":4083765155},{"type":"v1","addr":"172.21.15.113:6836","nonce":4083765155}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692,7004]},"id":1}]} 2024-03-04T08:17:55.823 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 156 2024-03-04T08:17:55.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 -- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a34066f90 msgr2=0x7f1a34069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:55.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a34066f90 0x7f1a34069450 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1a480ad000 tx=0x7f1a480a6000 comp rx=0 tx=0).stop 2024-03-04T08:17:55.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 -- 172.21.15.113:0/3404889717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a540ce120 msgr2=0x7f1a54149d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:55.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a540ce120 0x7f1a54149d90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000c580 tx=0x7f1a4000ca40 comp rx=0 tx=0).stop 2024-03-04T08:17:55.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 -- 172.21.15.113:0/3404889717 shutdown_connections 2024-03-04T08:17:55.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1a54158010 0x7f1a54148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:55.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1a34066f90 0x7f1a34069450 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:55.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a540ce120 0x7f1a54149d90 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:55.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 --2- 172.21.15.113:0/3404889717 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a5406ff20 0x7f1a54147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:55.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 -- 172.21.15.113:0/3404889717 >> 172.21.15.113:0/3404889717 conn(0x7f1a540734d0 msgr2=0x7f1a540ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:55.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 -- 172.21.15.113:0/3404889717 shutdown_connections 2024-03-04T08:17:55.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:55.827+0000 7f1a5c200640 1 -- 172.21.15.113:0/3404889717 wait complete. 2024-03-04T08:17:55.844 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is laggy/crashed, in failed state, or mds.a is removed from mdsmap 2024-03-04T08:17:57.845 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:57.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.987+0000 7f18f3e00640 1 Processor -- start 2024-03-04T08:17:57.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.987+0000 7f18f3e00640 1 -- start start 2024-03-04T08:17:57.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.987+0000 7f18f3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f18ec0c22e0 0x7f18ec15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:57.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.987+0000 7f18f3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0c27f0 con 0x7f18ec0c2d80 2024-03-04T08:17:57.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.987+0000 7f18f3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0c2a90 con 0x7f18ec14e4c0 2024-03-04T08:17:57.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.987+0000 7f18f3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec15ad60 con 0x7f18ec0c22e0 2024-03-04T08:17:57.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.987+0000 7f18f1e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f18ec0c2d80 0x7f18ec1620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33464/0 (socket says 172.21.15.113:33464) 2024-03-04T08:17:57.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.987+0000 7f18f1e00640 1 -- 172.21.15.113:0/1821256301 learned_addr learned my addr 172.21.15.113:0/1821256301 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:17:57.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.987+0000 7f18f0a00640 1 --2- 172.21.15.113:0/1821256301 >> v2:172.21.15.136:3300/0 conn(0x7f18ec0c22e0 0x7f18ec15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:57.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18ebe00640 1 -- 172.21.15.113:0/1821256301 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223667722 0 0) 0x7f18ec0c27f0 con 0x7f18ec0c2d80 2024-03-04T08:17:57.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18ebe00640 1 -- 172.21.15.113:0/1821256301 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d4003530 con 0x7f18ec0c2d80 2024-03-04T08:17:57.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18ebe00640 1 -- 172.21.15.113:0/1821256301 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054730972 0 0) 0x7f18ec0c2a90 con 0x7f18ec14e4c0 2024-03-04T08:17:57.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18ebe00640 1 -- 172.21.15.113:0/1821256301 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18ec0c27f0 con 0x7f18ec14e4c0 2024-03-04T08:17:57.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18f0a00640 1 -- 172.21.15.113:0/1821256301 >> v1:172.21.15.136:6789/0 conn(0x7f18ec14e4c0 legacy=0x7f18ec072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:57.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18f0a00640 1 -- 172.21.15.113:0/1821256301 >> v1:172.21.15.113:6789/0 conn(0x7f18ec0c2d80 legacy=0x7f18ec1620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:57.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18f0a00640 1 -- 172.21.15.113:0/1821256301 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec070c60 con 0x7f18ec0c22e0 2024-03-04T08:17:57.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18f0a00640 1 --2- 172.21.15.113:0/1821256301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18ec0c22e0 0x7f18ec15b1f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f18d80097c0 tx=0x7f18d801c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f3b7538542e2a4c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:57.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18ebe00640 1 -- 172.21.15.113:0/1821256301 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d80b0070 con 0x7f18ec0c22e0 2024-03-04T08:17:57.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18ebe00640 1 -- 172.21.15.113:0/1821256301 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d80ab400 con 0x7f18ec0c22e0 2024-03-04T08:17:57.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.991+0000 7f18ebe00640 1 -- 172.21.15.113:0/1821256301 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4001a00 con 0x7f18ec0c22e0 2024-03-04T08:17:57.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 -- 172.21.15.113:0/1821256301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18ec0c22e0 msgr2=0x7f18ec15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:57.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 --2- 172.21.15.113:0/1821256301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18ec0c22e0 0x7f18ec15b1f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f18d80097c0 tx=0x7f18d801c2e0 comp rx=0 tx=0).stop 2024-03-04T08:17:57.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 -- 172.21.15.113:0/1821256301 shutdown_connections 2024-03-04T08:17:57.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 --2- 172.21.15.113:0/1821256301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18ec0c22e0 0x7f18ec15b1f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:57.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 -- 172.21.15.113:0/1821256301 >> 172.21.15.113:0/1821256301 conn(0x7f18ec06e820 msgr2=0x7f18ec06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:57.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 -- 172.21.15.113:0/1821256301 shutdown_connections 2024-03-04T08:17:57.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 -- 172.21.15.113:0/1821256301 wait complete. 2024-03-04T08:17:57.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 Processor -- start 2024-03-04T08:17:57.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 -- start start 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18ec0c22e0 0x7f18ec141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18ec0c2d80 0x7f18ec13f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18ec14e4c0 0x7f18ec13f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec14e8a0 con 0x7f18ec0c2d80 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec15bb40 con 0x7f18ec14e4c0 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0aede0 con 0x7f18ec0c22e0 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18ec0c2d80 0x7f18ec13f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18ec0c2d80 0x7f18ec13f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36546/0 (socket says 172.21.15.113:36546) 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f0a00640 1 -- 172.21.15.113:0/81049594 learned_addr learned my addr 172.21.15.113:0/81049594 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f1400640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18ec0c22e0 0x7f18ec141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f1e00640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18ec14e4c0 0x7f18ec13f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f0a00640 1 -- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18ec0c22e0 msgr2=0x7f18ec141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:57.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f0a00640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18ec0c22e0 0x7f18ec141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:57.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f0a00640 1 -- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18ec14e4c0 msgr2=0x7f18ec13f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:57.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.995+0000 7f18f0a00640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18ec14e4c0 0x7f18ec13f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:57.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.999+0000 7f18f0a00640 1 -- 172.21.15.113:0/81049594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec14a6a0 con 0x7f18ec0c2d80 2024-03-04T08:17:57.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.999+0000 7f18f0a00640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18ec0c2d80 0x7f18ec13f1e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f18d80ae040 tx=0x7f18d801ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:57.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.999+0000 7f18e9600640 1 -- 172.21.15.113:0/81049594 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d80b0050 con 0x7f18ec0c2d80 2024-03-04T08:17:57.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.999+0000 7f18e9600640 1 -- 172.21.15.113:0/81049594 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d80bc070 con 0x7f18ec0c2d80 2024-03-04T08:17:57.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.999+0000 7f18e9600640 1 -- 172.21.15.113:0/81049594 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8006e20 con 0x7f18ec0c2d80 2024-03-04T08:17:57.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.999+0000 7f18f3e00640 1 -- 172.21.15.113:0/81049594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec14a940 con 0x7f18ec0c2d80 2024-03-04T08:17:57.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.999+0000 7f18f3e00640 1 -- 172.21.15.113:0/81049594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec14af00 con 0x7f18ec0c2d80 2024-03-04T08:17:57.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:57.999+0000 7f18f3e00640 1 -- 172.21.15.113:0/81049594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec075c00 con 0x7f18ec0c2d80 2024-03-04T08:17:57.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.003+0000 7f18e9600640 1 -- 172.21.15.113:0/81049594 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18d80b7850 con 0x7f18ec0c2d80 2024-03-04T08:17:57.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.003+0000 7f18e9600640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18cc066f60 0x7f18cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:58.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.003+0000 7f18e9600640 1 -- 172.21.15.113:0/81049594 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7f18d811fda0 con 0x7f18ec0c2d80 2024-03-04T08:17:58.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.003+0000 7f18f1400640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18cc066f60 0x7f18cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:58.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.003+0000 7f18f1400640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18cc066f60 0x7f18cc069420 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f18e00011f0 tx=0x7f18e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:58.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.011+0000 7f18e9600640 1 -- 172.21.15.113:0/81049594 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f18d8015f90 con 0x7f18ec0c2d80 2024-03-04T08:17:58.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.319+0000 7f18f3e00640 1 -- 172.21.15.113:0/81049594 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18ec155190 con 0x7f18ec0c2d80 2024-03-04T08:17:58.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18e9600640 1 -- 172.21.15.113:0/81049594 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f18d80efeb0 con 0x7f18ec0c2d80 2024-03-04T08:17:58.320 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:58.320 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:17:55.953569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839]},"id":1}]} 2024-03-04T08:17:58.320 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 157 2024-03-04T08:17:58.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 -- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18cc066f60 msgr2=0x7f18cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18cc066f60 0x7f18cc069420 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f18e00011f0 tx=0x7f18e0009040 comp rx=0 tx=0).stop 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 -- 172.21.15.113:0/81049594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18ec0c2d80 msgr2=0x7f18ec13f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18ec0c2d80 0x7f18ec13f1e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f18d80ae040 tx=0x7f18d801ab20 comp rx=0 tx=0).stop 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 -- 172.21.15.113:0/81049594 shutdown_connections 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18ec14e4c0 0x7f18ec13f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f18cc066f60 0x7f18cc069420 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18ec0c2d80 0x7f18ec13f1e0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 --2- 172.21.15.113:0/81049594 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18ec0c22e0 0x7f18ec141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 -- 172.21.15.113:0/81049594 >> 172.21.15.113:0/81049594 conn(0x7f18ec06e820 msgr2=0x7f18ec0c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 -- 172.21.15.113:0/81049594 shutdown_connections 2024-03-04T08:17:58.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.323+0000 7f18f3e00640 1 -- 172.21.15.113:0/81049594 wait complete. 2024-03-04T08:17:58.351 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a down, removed from mdsmap 2024-03-04T08:17:58.351 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:17:58.351 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:17:58.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dc4e00640 1 Processor -- start 2024-03-04T08:17:58.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dc4e00640 1 -- start start 2024-03-04T08:17:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dc4e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9dc01549b0 0x7f9dc015b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:58.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dc4e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dc0154d90 0x7f9dc00d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dc4e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc00cae80 con 0x7f9dc006cb30 2024-03-04T08:17:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dc4e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc015b840 con 0x7f9dc01549b0 2024-03-04T08:17:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dc4e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc015e170 con 0x7f9dc0154d90 2024-03-04T08:17:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dbde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9dc01549b0 0x7f9dc015b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dbde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9dc01549b0 0x7f9dc015b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36550/0 (socket says 172.21.15.113:36550) 2024-03-04T08:17:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dbde00640 1 -- 172.21.15.113:0/3646582069 learned_addr learned my addr 172.21.15.113:0/3646582069 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dbd400640 1 --2- 172.21.15.113:0/3646582069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dc0154d90 0x7f9dc00d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:58.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dbde00640 1 -- 172.21.15.113:0/3646582069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dc0154d90 msgr2=0x7f9dc00d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:17:58.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dbde00640 1 --2- 172.21.15.113:0/3646582069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dc0154d90 0x7f9dc00d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dbde00640 1 -- 172.21.15.113:0/3646582069 >> v1:172.21.15.136:6789/0 conn(0x7f9dc006cb30 legacy=0x7f9dc00c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:58.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.499+0000 7f9dbde00640 1 -- 172.21.15.113:0/3646582069 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc015e450 con 0x7f9dc01549b0 2024-03-04T08:17:58.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.503+0000 7f9dbde00640 1 --2- 172.21.15.113:0/3646582069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc01549b0 0x7f9dc015b300 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009870 tx=0x7f9db001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae4bd69930c7cb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:58.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.503+0000 7f9dbca00640 1 -- 172.21.15.113:0/3646582069 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00b5070 con 0x7f9dc01549b0 2024-03-04T08:17:58.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.503+0000 7f9dbca00640 1 -- 172.21.15.113:0/3646582069 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db00a8a50 con 0x7f9dc01549b0 2024-03-04T08:17:58.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.503+0000 7f9dbca00640 1 -- 172.21.15.113:0/3646582069 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00a8e20 con 0x7f9dc01549b0 2024-03-04T08:17:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.507+0000 7f9dc4e00640 1 -- 172.21.15.113:0/3646582069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc01549b0 msgr2=0x7f9dc015b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.507+0000 7f9dc4e00640 1 --2- 172.21.15.113:0/3646582069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc01549b0 0x7f9dc015b300 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9db0009870 tx=0x7f9db001ed50 comp rx=0 tx=0).stop 2024-03-04T08:17:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.507+0000 7f9dc4e00640 1 -- 172.21.15.113:0/3646582069 shutdown_connections 2024-03-04T08:17:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.507+0000 7f9dc4e00640 1 --2- 172.21.15.113:0/3646582069 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dc0154d90 0x7f9dc00d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.507+0000 7f9dc4e00640 1 --2- 172.21.15.113:0/3646582069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc01549b0 0x7f9dc015b300 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.507+0000 7f9dc4e00640 1 -- 172.21.15.113:0/3646582069 >> 172.21.15.113:0/3646582069 conn(0x7f9dc006dc50 msgr2=0x7f9dc0066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.507+0000 7f9dc4e00640 1 -- 172.21.15.113:0/3646582069 shutdown_connections 2024-03-04T08:17:58.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.507+0000 7f9dc4e00640 1 -- 172.21.15.113:0/3646582069 wait complete. 2024-03-04T08:17:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.507+0000 7f9dc4e00640 1 Processor -- start 2024-03-04T08:17:58.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.507+0000 7f9dc4e00640 1 -- start start 2024-03-04T08:17:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dc4e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc006cb30 0x7f9dc01f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:58.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dc4e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dc01549b0 0x7f9dc00c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dc4e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc0154d90 0x7f9dc00d0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dc4e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc015ecf0 con 0x7f9dc0154d90 2024-03-04T08:17:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dc4e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc015ea10 con 0x7f9dc006cb30 2024-03-04T08:17:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dc4e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00c5380 con 0x7f9dc01549b0 2024-03-04T08:17:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbe800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc0154d90 0x7f9dc00d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbe800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc0154d90 0x7f9dc00d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36562/0 (socket says 172.21.15.113:36562) 2024-03-04T08:17:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbe800640 1 -- 172.21.15.113:0/361875876 learned_addr learned my addr 172.21.15.113:0/361875876 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:17:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbde00640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc006cb30 0x7f9dc01f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:58.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbd400640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dc01549b0 0x7f9dc00c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:58.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbe800640 1 -- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dc01549b0 msgr2=0x7f9dc00c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:58.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbe800640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dc01549b0 0x7f9dc00c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbe800640 1 -- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc006cb30 msgr2=0x7f9dc01f0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:58.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbe800640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc006cb30 0x7f9dc01f0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbe800640 1 -- 172.21.15.113:0/361875876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00ce0e0 con 0x7f9dc0154d90 2024-03-04T08:17:58.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dbe800640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc0154d90 0x7f9dc00d0830 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c7d0 tx=0x7f9db800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:58.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9db6000640 1 -- 172.21.15.113:0/361875876 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8003b80 con 0x7f9dc0154d90 2024-03-04T08:17:58.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9db6000640 1 -- 172.21.15.113:0/361875876 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db8003d20 con 0x7f9dc0154d90 2024-03-04T08:17:58.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9db6000640 1 -- 172.21.15.113:0/361875876 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db800aa30 con 0x7f9dc0154d90 2024-03-04T08:17:58.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dc4e00640 1 -- 172.21.15.113:0/361875876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc00ce3a0 con 0x7f9dc0154d90 2024-03-04T08:17:58.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dc4e00640 1 -- 172.21.15.113:0/361875876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc00ce960 con 0x7f9dc0154d90 2024-03-04T08:17:58.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.511+0000 7f9dc4e00640 1 -- 172.21.15.113:0/361875876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc0075c00 con 0x7f9dc0154d90 2024-03-04T08:17:58.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.515+0000 7f9db6000640 1 -- 172.21.15.113:0/361875876 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9db800abd0 con 0x7f9dc0154d90 2024-03-04T08:17:58.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.515+0000 7f9db6000640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9da0066f90 0x7f9da0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:17:58.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.515+0000 7f9db6000640 1 -- 172.21.15.113:0/361875876 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7f9db8085790 con 0x7f9dc0154d90 2024-03-04T08:17:58.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.515+0000 7f9dbde00640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9da0066f90 0x7f9da0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:17:58.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.515+0000 7f9dbde00640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9da0066f90 0x7f9da0069450 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9db00b2000 tx=0x7f9db0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:17:58.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.519+0000 7f9db6000640 1 -- 172.21.15.113:0/361875876 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9db8055790 con 0x7f9dc0154d90 2024-03-04T08:17:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.707+0000 7f9dc4e00640 1 -- 172.21.15.113:0/361875876 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dc0066f50 con 0x7f9dc0154d90 2024-03-04T08:17:58.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.707+0000 7f9db6000640 1 -- 172.21.15.113:0/361875876 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f9db8059830 con 0x7f9dc0154d90 2024-03-04T08:17:58.706 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:17:58.706 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:17:55.953569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839]},"id":1}]} 2024-03-04T08:17:58.707 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 157 2024-03-04T08:17:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 -- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9da0066f90 msgr2=0x7f9da0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9da0066f90 0x7f9da0069450 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9db00b2000 tx=0x7f9db0009040 comp rx=0 tx=0).stop 2024-03-04T08:17:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 -- 172.21.15.113:0/361875876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc0154d90 msgr2=0x7f9dc00d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:17:58.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc0154d90 0x7f9dc00d0830 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c7d0 tx=0x7f9db800cc90 comp rx=0 tx=0).stop 2024-03-04T08:17:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 -- 172.21.15.113:0/361875876 shutdown_connections 2024-03-04T08:17:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9da0066f90 0x7f9da0069450 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dc0154d90 0x7f9dc00d0830 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dc01549b0 0x7f9dc00c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 --2- 172.21.15.113:0/361875876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dc006cb30 0x7f9dc01f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:17:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 -- 172.21.15.113:0/361875876 >> 172.21.15.113:0/361875876 conn(0x7f9dc006dc50 msgr2=0x7f9dc01589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:17:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 -- 172.21.15.113:0/361875876 shutdown_connections 2024-03-04T08:17:58.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:17:58.711+0000 7f9dc4e00640 1 -- 172.21.15.113:0/361875876 wait complete. 2024-03-04T08:17:58.727 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 157, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 157, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7692": { "addr": "172.21.15.113:6833/1928887637", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 1928887637, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 1928887637, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7692, "incarnation": 139, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 72, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:17:55.953569+0000", "qdb_cluster": [ 7692, 7839 ], "qdb_leader": 7839, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 8032, "mds_2": 7692 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:17:59.768 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:17:59.769 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 74 -eq 48 ] 2024-03-04T08:17:59.769 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.74 2024-03-04T08:17:59.769 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.74 2024-03-04T08:17:59.769 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.74 2024-03-04T08:17:59.769 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.74 2024-03-04T08:17:59.770 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:00.728 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:00.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f5780400640 1 Processor -- start 2024-03-04T08:18:00.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f5780400640 1 -- start start 2024-03-04T08:18:00.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f5780400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57780c8200 0x7f5778070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:00.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f5780400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5778152bd0 0x7f57781587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:00.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f5780400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57781591f0 con 0x7f57781527f0 2024-03-04T08:18:00.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f5780400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5778071df0 con 0x7f5778152bd0 2024-03-04T08:18:00.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f5780400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57781597b0 con 0x7f57780c8200 2024-03-04T08:18:00.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f577d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57780c8200 0x7f5778070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:00.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f577d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57780c8200 0x7f5778070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:41846/0 (socket says 172.21.15.113:41846) 2024-03-04T08:18:00.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f577d000640 1 -- 172.21.15.113:0/962217634 learned_addr learned my addr 172.21.15.113:0/962217634 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:00.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f577e400640 1 --2- 172.21.15.113:0/962217634 >> v2:172.21.15.136:3300/0 conn(0x7f5778152bd0 0x7f57781587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:00.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f577d000640 1 -- 172.21.15.113:0/962217634 >> v2:172.21.15.136:3300/0 conn(0x7f5778152bd0 msgr2=0x7f57781587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:00.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f577d000640 1 --2- 172.21.15.113:0/962217634 >> v2:172.21.15.136:3300/0 conn(0x7f5778152bd0 0x7f57781587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:00.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f577d000640 1 -- 172.21.15.113:0/962217634 >> v1:172.21.15.136:6789/0 conn(0x7f57781527f0 legacy=0x7f5778072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:00.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.871+0000 7f577d000640 1 -- 172.21.15.113:0/962217634 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57781594d0 con 0x7f57780c8200 2024-03-04T08:18:00.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f577d000640 1 --2- 172.21.15.113:0/962217634 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57780c8200 0x7f5778070d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5764009870 tx=0x7f576401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=209fa9ef221d10d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:00.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5777e00640 1 -- 172.21.15.113:0/962217634 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57640a7070 con 0x7f57780c8200 2024-03-04T08:18:00.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5777e00640 1 -- 172.21.15.113:0/962217634 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57640b0070 con 0x7f57780c8200 2024-03-04T08:18:00.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5777e00640 1 -- 172.21.15.113:0/962217634 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57640ab890 con 0x7f57780c8200 2024-03-04T08:18:00.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5780400640 1 -- 172.21.15.113:0/962217634 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57780c8200 msgr2=0x7f5778070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:00.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5780400640 1 --2- 172.21.15.113:0/962217634 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57780c8200 0x7f5778070d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5764009870 tx=0x7f576401c560 comp rx=0 tx=0).stop 2024-03-04T08:18:00.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5780400640 1 -- 172.21.15.113:0/962217634 shutdown_connections 2024-03-04T08:18:00.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5780400640 1 --2- 172.21.15.113:0/962217634 >> v2:172.21.15.136:3300/0 conn(0x7f5778152bd0 0x7f57781587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:00.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5780400640 1 --2- 172.21.15.113:0/962217634 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f57780c8200 0x7f5778070d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:00.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5780400640 1 -- 172.21.15.113:0/962217634 >> 172.21.15.113:0/962217634 conn(0x7f577806e820 msgr2=0x7f577806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:00.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5780400640 1 -- 172.21.15.113:0/962217634 shutdown_connections 2024-03-04T08:18:00.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.875+0000 7f5780400640 1 -- 172.21.15.113:0/962217634 wait complete. 2024-03-04T08:18:00.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5780400640 1 Processor -- start 2024-03-04T08:18:00.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5780400640 1 -- start start 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5780400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57780c8200 0x7f57781421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5780400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57781527f0 0x7f577814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5780400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5778152bd0 0x7f57780ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5780400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57781654f0 con 0x7f57780c8200 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5780400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778165370 con 0x7f57781527f0 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5780400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5778165670 con 0x7f5778152bd0 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57780c8200 0x7f57781421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57780c8200 0x7f57781421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36564/0 (socket says 172.21.15.113:36564) 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577da00640 1 -- 172.21.15.113:0/1298598903 learned_addr learned my addr 172.21.15.113:0/1298598903 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577d000640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57781527f0 0x7f577814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:00.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577e400640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5778152bd0 0x7f57780ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:00.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577da00640 1 -- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5778152bd0 msgr2=0x7f57780ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:00.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577da00640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5778152bd0 0x7f57780ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:00.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577da00640 1 -- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57781527f0 msgr2=0x7f577814ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:00.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577da00640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57781527f0 0x7f577814ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:00.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577da00640 1 -- 172.21.15.113:0/1298598903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57780ca010 con 0x7f57780c8200 2024-03-04T08:18:00.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f577da00640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57780c8200 0x7f57781421b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f576c00c450 tx=0x7f576c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:00.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5775600640 1 -- 172.21.15.113:0/1298598903 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c016020 con 0x7f57780c8200 2024-03-04T08:18:00.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5775600640 1 -- 172.21.15.113:0/1298598903 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c0052d0 con 0x7f57780c8200 2024-03-04T08:18:00.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5780400640 1 -- 172.21.15.113:0/1298598903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57781f82e0 con 0x7f57780c8200 2024-03-04T08:18:00.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5775600640 1 -- 172.21.15.113:0/1298598903 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c007260 con 0x7f57780c8200 2024-03-04T08:18:00.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.879+0000 7f5780400640 1 -- 172.21.15.113:0/1298598903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57780cbae0 con 0x7f57780c8200 2024-03-04T08:18:00.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.883+0000 7f5780400640 1 -- 172.21.15.113:0/1298598903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5778075c00 con 0x7f57780c8200 2024-03-04T08:18:00.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.883+0000 7f5775600640 1 -- 172.21.15.113:0/1298598903 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f576c007400 con 0x7f57780c8200 2024-03-04T08:18:00.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.883+0000 7f5775600640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5758066f60 0x7f5758069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:00.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.883+0000 7f5775600640 1 -- 172.21.15.113:0/1298598903 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7f576c0851c0 con 0x7f57780c8200 2024-03-04T08:18:00.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.883+0000 7f577d000640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5758066f60 0x7f5758069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:00.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.887+0000 7f577d000640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5758066f60 0x7f5758069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f57640af000 tx=0x7f57640a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:00.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:00.887+0000 7f5775600640 1 -- 172.21.15.113:0/1298598903 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f576c020020 con 0x7f57780c8200 2024-03-04T08:18:01.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.135+0000 7f5780400640 1 -- 172.21.15.113:0/1298598903 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5778162f30 con 0x7f57780c8200 2024-03-04T08:18:01.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.135+0000 7f5775600640 1 -- 172.21.15.113:0/1298598903 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f576c005480 con 0x7f57780c8200 2024-03-04T08:18:01.133 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:01.133 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":157,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:17:55.953569+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839]},"id":1}]} 2024-03-04T08:18:01.133 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 157 2024-03-04T08:18:01.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 -- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5758066f60 msgr2=0x7f5758069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:01.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5758066f60 0x7f5758069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f57640af000 tx=0x7f57640a7000 comp rx=0 tx=0).stop 2024-03-04T08:18:01.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 -- 172.21.15.113:0/1298598903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57780c8200 msgr2=0x7f57781421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:01.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57780c8200 0x7f57781421b0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f576c00c450 tx=0x7f576c00c910 comp rx=0 tx=0).stop 2024-03-04T08:18:01.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 -- 172.21.15.113:0/1298598903 shutdown_connections 2024-03-04T08:18:01.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5778152bd0 0x7f57780ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:01.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57781527f0 0x7f577814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:01.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5758066f60 0x7f5758069420 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:01.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 --2- 172.21.15.113:0/1298598903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f57780c8200 0x7f57781421b0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:01.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 -- 172.21.15.113:0/1298598903 >> 172.21.15.113:0/1298598903 conn(0x7f577806e820 msgr2=0x7f5778151840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:01.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 -- 172.21.15.113:0/1298598903 shutdown_connections 2024-03-04T08:18:01.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:01.139+0000 7f5780400640 1 -- 172.21.15.113:0/1298598903 wait complete. 2024-03-04T08:18:01.158 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:18:03.159 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:03.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.295+0000 7f363ec00640 1 Processor -- start 2024-03-04T08:18:03.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.295+0000 7f363ec00640 1 -- start start 2024-03-04T08:18:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f363ec00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380c8200 0x7f3638070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f363ec00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36381591f0 con 0x7f36381527f0 2024-03-04T08:18:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f363ec00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36381594d0 con 0x7f3638152bd0 2024-03-04T08:18:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f363ec00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3638071df0 con 0x7f36380c8200 2024-03-04T08:18:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f36381527f0 0x7f3638072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33474/0 (socket says 172.21.15.113:33474) 2024-03-04T08:18:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637e00640 1 -- 172.21.15.113:0/3585764855 learned_addr learned my addr 172.21.15.113:0/3585764855 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:18:03.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f362f400640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380c8200 0x7f3638070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:03.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101859042 0 0) 0x7f36381591f0 con 0x7f36381527f0 2024-03-04T08:18:03.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f361c003530 con 0x7f36381527f0 2024-03-04T08:18:03.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693892771 0 0) 0x7f36381594d0 con 0x7f3638152bd0 2024-03-04T08:18:03.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36381591f0 con 0x7f3638152bd0 2024-03-04T08:18:03.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1497524578 0 0) 0x7f361c003530 con 0x7f36381527f0 2024-03-04T08:18:03.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380c8200 msgr2=0x7f3638070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:03.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380c8200 0x7f3638070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:03.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 >> v1:172.21.15.136:6789/0 conn(0x7f3638152bd0 legacy=0x7f36381587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:03.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36380717a0 con 0x7f36381527f0 2024-03-04T08:18:03.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f3628001680 con 0x7f36381527f0 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 >> v1:172.21.15.113:6789/0 conn(0x7f36381527f0 legacy=0x7f3638072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 0x7f361c003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f361c004810 0x7f361c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380c8200 0x7f361c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3638071df0 con 0x7f361c003b80 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c00ba80 con 0x7f361c004810 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c00bc30 con 0x7f36380c8200 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637e00640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 0x7f361c003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f362f400640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f361c004810 0x7f361c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f363cc00640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380c8200 0x7f361c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637e00640 1 -- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380c8200 msgr2=0x7f361c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637e00640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380c8200 0x7f361c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637e00640 1 -- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f361c004810 msgr2=0x7f361c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:03.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637e00640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f361c004810 0x7f361c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:03.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.299+0000 7f3637e00640 1 -- 172.21.15.113:0/3585764855 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c003530 con 0x7f361c003b80 2024-03-04T08:18:03.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f3637e00640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 0x7f361c003f60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f3628007f80 tx=0x7f362801caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8124a353142cff01 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:03.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628007bd0 con 0x7f361c003b80 2024-03-04T08:18:03.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36280a9020 con 0x7f361c003b80 2024-03-04T08:18:03.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f3637400640 1 -- 172.21.15.113:0/3585764855 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628005b00 con 0x7f361c003b80 2024-03-04T08:18:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f363ec00640 1 -- 172.21.15.113:0/3585764855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 msgr2=0x7f361c003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f363ec00640 1 --2- 172.21.15.113:0/3585764855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 0x7f361c003f60 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f3628007f80 tx=0x7f362801caa0 comp rx=0 tx=0).stop 2024-03-04T08:18:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f363cc00640 1 -- 172.21.15.113:0/3585764855 reap_dead start 2024-03-04T08:18:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f363ec00640 1 -- 172.21.15.113:0/3585764855 shutdown_connections 2024-03-04T08:18:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f363ec00640 1 -- 172.21.15.113:0/3585764855 >> 172.21.15.113:0/3585764855 conn(0x7f363806e820 msgr2=0x7f363806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f363ec00640 1 -- 172.21.15.113:0/3585764855 shutdown_connections 2024-03-04T08:18:03.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.303+0000 7f363ec00640 1 -- 172.21.15.113:0/3585764855 wait complete. 2024-03-04T08:18:03.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363ec00640 1 Processor -- start 2024-03-04T08:18:03.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363ec00640 1 -- start start 2024-03-04T08:18:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363ec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 0x7f363814ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363ec00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380b2220 0x7f363814b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363ec00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f363814e700 0x7f36380b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363ec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3638164370 con 0x7f361c003b80 2024-03-04T08:18:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363ec00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36381597b0 con 0x7f363814e700 2024-03-04T08:18:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363ec00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36381644f0 con 0x7f36380b2220 2024-03-04T08:18:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3637e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 0x7f363814ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3637e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 0x7f363814ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36580/0 (socket says 172.21.15.113:36580) 2024-03-04T08:18:03.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3637e00640 1 -- 172.21.15.113:0/523603274 learned_addr learned my addr 172.21.15.113:0/523603274 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:03.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f362f400640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380b2220 0x7f363814b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:03.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363cc00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f363814e700 0x7f36380b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:03.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3637e00640 1 -- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380b2220 msgr2=0x7f363814b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:03.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3637e00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380b2220 0x7f363814b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:03.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3637e00640 1 -- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f363814e700 msgr2=0x7f36380b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:03.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3637e00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f363814e700 0x7f36380b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:03.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3637e00640 1 -- 172.21.15.113:0/523603274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36380cb0c0 con 0x7f361c003b80 2024-03-04T08:18:03.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3637e00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 0x7f363814ab50 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f36280ab040 tx=0x7f361c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3634c00640 1 -- 172.21.15.113:0/523603274 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628007bd0 con 0x7f361c003b80 2024-03-04T08:18:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3634c00640 1 -- 172.21.15.113:0/523603274 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36280072d0 con 0x7f361c003b80 2024-03-04T08:18:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363ec00640 1 -- 172.21.15.113:0/523603274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36380c6590 con 0x7f361c003b80 2024-03-04T08:18:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f3634c00640 1 -- 172.21.15.113:0/523603274 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628001b10 con 0x7f361c003b80 2024-03-04T08:18:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.307+0000 7f363ec00640 1 -- 172.21.15.113:0/523603274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36381f4530 con 0x7f361c003b80 2024-03-04T08:18:03.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.311+0000 7f363ec00640 1 -- 172.21.15.113:0/523603274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3638075c00 con 0x7f361c003b80 2024-03-04T08:18:03.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.311+0000 7f3634c00640 1 -- 172.21.15.113:0/523603274 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36280a55e0 con 0x7f361c003b80 2024-03-04T08:18:03.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.311+0000 7f3634c00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3614066f90 0x7f3614069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:03.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.311+0000 7f3634c00640 1 -- 172.21.15.113:0/523603274 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7f362811ef90 con 0x7f361c003b80 2024-03-04T08:18:03.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.311+0000 7f362f400640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3614066f90 0x7f3614069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:03.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.315+0000 7f362f400640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3614066f90 0x7f3614069450 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f363806f610 tx=0x7f3620009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:03.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.323+0000 7f3634c00640 1 -- 172.21.15.113:0/523603274 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3628016810 con 0x7f361c003b80 2024-03-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.519+0000 7f363ec00640 1 -- 172.21.15.113:0/523603274 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36381561d0 con 0x7f361c003b80 2024-03-04T08:18:03.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.519+0000 7f3634c00640 1 -- 172.21.15.113:0/523603274 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) v1 ==== 95+0+5825 (secure 0 0 0) 0x7f36280ef700 con 0x7f361c003b80 2024-03-04T08:18:03.519 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:03.519 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":158,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:01.861908+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:resolve","state_seq":24,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692]},"id":1}]} 2024-03-04T08:18:03.519 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 158 2024-03-04T08:18:03.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 -- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3614066f90 msgr2=0x7f3614069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:03.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3614066f90 0x7f3614069450 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f363806f610 tx=0x7f3620009190 comp rx=0 tx=0).stop 2024-03-04T08:18:03.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 -- 172.21.15.113:0/523603274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 msgr2=0x7f363814ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:03.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 0x7f363814ab50 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f36280ab040 tx=0x7f361c005840 comp rx=0 tx=0).stop 2024-03-04T08:18:03.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 -- 172.21.15.113:0/523603274 shutdown_connections 2024-03-04T08:18:03.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f363814e700 0x7f36380b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f36380b2220 0x7f363814b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3614066f90 0x7f3614069450 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 --2- 172.21.15.113:0/523603274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f361c003b80 0x7f363814ab50 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 -- 172.21.15.113:0/523603274 >> 172.21.15.113:0/523603274 conn(0x7f363806e820 msgr2=0x7f36381583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 -- 172.21.15.113:0/523603274 shutdown_connections 2024-03-04T08:18:03.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:03.523+0000 7f363ec00640 1 -- 172.21.15.113:0/523603274 wait complete. 2024-03-04T08:18:03.541 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:18:04.131 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:04.132 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.75 2024-03-04T08:18:04.132 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 75 -eq 48 ] 2024-03-04T08:18:04.132 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.75 2024-03-04T08:18:04.132 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.75 2024-03-04T08:18:04.132 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.75 2024-03-04T08:18:04.132 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:05.543 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:05.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f9a00640 1 Processor -- start 2024-03-04T08:18:05.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f9a00640 1 -- start start 2024-03-04T08:18:05.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f9a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd1f4154e50 0x7fd1f40c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:05.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f9a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40c25e0 0x7fd1f40cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:05.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f9a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f415b590 con 0x7fd1f40733f0 2024-03-04T08:18:05.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f9a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f415b850 con 0x7fd1f4154e50 2024-03-04T08:18:05.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f9a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f4072810 con 0x7fd1f40c25e0 2024-03-04T08:18:05.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40c25e0 0x7fd1f40cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:05.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40c25e0 0x7fd1f40cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:41886/0 (socket says 172.21.15.113:41886) 2024-03-04T08:18:05.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f2000640 1 -- 172.21.15.113:0/1037296510 learned_addr learned my addr 172.21.15.113:0/1037296510 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:05.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f2a00640 1 --2- 172.21.15.113:0/1037296510 >> v2:172.21.15.136:3300/0 conn(0x7fd1f4154e50 0x7fd1f40c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:05.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f2000640 1 -- 172.21.15.113:0/1037296510 >> v2:172.21.15.136:3300/0 conn(0x7fd1f4154e50 msgr2=0x7fd1f40c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:05.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f2000640 1 --2- 172.21.15.113:0/1037296510 >> v2:172.21.15.136:3300/0 conn(0x7fd1f4154e50 0x7fd1f40c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:05.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f2000640 1 -- 172.21.15.113:0/1037296510 >> v1:172.21.15.136:6789/0 conn(0x7fd1f40733f0 legacy=0x7fd1f415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:05.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.743+0000 7fd1f2000640 1 -- 172.21.15.113:0/1037296510 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f4072af0 con 0x7fd1f40c25e0 2024-03-04T08:18:05.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f2000640 1 --2- 172.21.15.113:0/1037296510 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40c25e0 0x7fd1f40cd010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009870 tx=0x7fd1e001ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d35dbe721190cc40 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:05.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f1600640 1 -- 172.21.15.113:0/1037296510 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00b5070 con 0x7fd1f40c25e0 2024-03-04T08:18:05.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f1600640 1 -- 172.21.15.113:0/1037296510 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e00a8a50 con 0x7fd1f40c25e0 2024-03-04T08:18:05.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f1600640 1 -- 172.21.15.113:0/1037296510 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00a8e20 con 0x7fd1f40c25e0 2024-03-04T08:18:05.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1037296510 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40c25e0 msgr2=0x7fd1f40cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:05.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f9a00640 1 --2- 172.21.15.113:0/1037296510 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40c25e0 0x7fd1f40cd010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0009870 tx=0x7fd1e001ed50 comp rx=0 tx=0).stop 2024-03-04T08:18:05.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1037296510 shutdown_connections 2024-03-04T08:18:05.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f9a00640 1 --2- 172.21.15.113:0/1037296510 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40c25e0 0x7fd1f40cd010 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:05.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f9a00640 1 --2- 172.21.15.113:0/1037296510 >> v2:172.21.15.136:3300/0 conn(0x7fd1f4154e50 0x7fd1f40c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:05.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1037296510 >> 172.21.15.113:0/1037296510 conn(0x7fd1f40671b0 msgr2=0x7fd1f406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:05.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1037296510 shutdown_connections 2024-03-04T08:18:05.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.747+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1037296510 wait complete. 2024-03-04T08:18:05.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f9a00640 1 Processor -- start 2024-03-04T08:18:05.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f9a00640 1 -- start start 2024-03-04T08:18:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f9a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40733f0 0x7fd1f40c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f9a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f4154e50 0x7fd1f4148e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f9a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1f40cceb0 0x7fd1f4149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f9a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f406b790 con 0x7fd1f4154e50 2024-03-04T08:18:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f9a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f4155230 con 0x7fd1f40cceb0 2024-03-04T08:18:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f9a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f40ae8e0 con 0x7fd1f40733f0 2024-03-04T08:18:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40733f0 0x7fd1f40c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f4154e50 0x7fd1f4148e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:05.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40733f0 0x7fd1f40c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:41898/0 (socket says 172.21.15.113:41898) 2024-03-04T08:18:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f4154e50 0x7fd1f4148e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36584/0 (socket says 172.21.15.113:36584) 2024-03-04T08:18:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2a00640 1 -- 172.21.15.113:0/1696886445 learned_addr learned my addr 172.21.15.113:0/1696886445 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f3400640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1f40cceb0 0x7fd1f4149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2000640 1 -- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40733f0 msgr2=0x7fd1f40c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2000640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40733f0 0x7fd1f40c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2000640 1 -- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1f40cceb0 msgr2=0x7fd1f4149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2000640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1f40cceb0 0x7fd1f4149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2000640 1 -- 172.21.15.113:0/1696886445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f40c2140 con 0x7fd1f4154e50 2024-03-04T08:18:05.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f2000640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f4154e50 0x7fd1f4148e50 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4072990 tx=0x7fd1e001baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:05.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1eaa00640 1 -- 172.21.15.113:0/1696886445 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00c0070 con 0x7fd1f4154e50 2024-03-04T08:18:05.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1eaa00640 1 -- 172.21.15.113:0/1696886445 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e00ae550 con 0x7fd1f4154e50 2024-03-04T08:18:05.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1eaa00640 1 -- 172.21.15.113:0/1696886445 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00b5070 con 0x7fd1f4154e50 2024-03-04T08:18:05.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1696886445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f41f1100 con 0x7fd1f4154e50 2024-03-04T08:18:05.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.751+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1696886445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f40aeb30 con 0x7fd1f4154e50 2024-03-04T08:18:05.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.755+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1696886445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f4075c00 con 0x7fd1f4154e50 2024-03-04T08:18:05.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.755+0000 7fd1eaa00640 1 -- 172.21.15.113:0/1696886445 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd1e00b9070 con 0x7fd1f4154e50 2024-03-04T08:18:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.755+0000 7fd1eaa00640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d4066f90 0x7fd1d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.755+0000 7fd1eaa00640 1 -- 172.21.15.113:0/1696886445 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7fd1e0122f80 con 0x7fd1f4154e50 2024-03-04T08:18:05.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.755+0000 7fd1f2a00640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d4066f90 0x7fd1d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:05.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.755+0000 7fd1f2a00640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d4066f90 0x7fd1d4069450 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009b60 tx=0x7fd1e4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:05.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:05.759+0000 7fd1eaa00640 1 -- 172.21.15.113:0/1696886445 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd1e00188b0 con 0x7fd1f4154e50 2024-03-04T08:18:06.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.039+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1696886445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1f40731d0 con 0x7fd1f4154e50 2024-03-04T08:18:06.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.039+0000 7fd1eaa00640 1 -- 172.21.15.113:0/1696886445 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) v1 ==== 95+0+5825 (secure 0 0 0) 0x7fd1e00f2f90 con 0x7fd1f4154e50 2024-03-04T08:18:06.038 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:06.038 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":158,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:01.861908+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:resolve","state_seq":24,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692]},"id":1}]} 2024-03-04T08:18:06.038 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 158 2024-03-04T08:18:06.040 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d4066f90 msgr2=0x7fd1d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d4066f90 0x7fd1d4069450 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009b60 tx=0x7fd1e4009190 comp rx=0 tx=0).stop 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1696886445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f4154e50 msgr2=0x7fd1f4148e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f4154e50 0x7fd1f4148e50 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4072990 tx=0x7fd1e001baf0 comp rx=0 tx=0).stop 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1696886445 shutdown_connections 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1f40cceb0 0x7fd1f4149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d4066f90 0x7fd1d4069450 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f4154e50 0x7fd1f4148e50 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 --2- 172.21.15.113:0/1696886445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f40733f0 0x7fd1f40c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1696886445 >> 172.21.15.113:0/1696886445 conn(0x7fd1f40671b0 msgr2=0x7fd1f40c3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1696886445 shutdown_connections 2024-03-04T08:18:06.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:06.043+0000 7fd1f9a00640 1 -- 172.21.15.113:0/1696886445 wait complete. 2024-03-04T08:18:06.060 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:18:08.061 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:08.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.243+0000 7fb672800640 1 Processor -- start 2024-03-04T08:18:08.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.243+0000 7fb672800640 1 -- start start 2024-03-04T08:18:08.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb672800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb66c1582e0 0x7fb66c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:08.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb672800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb66c06ddf0 0x7fb66c15af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:08.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb672800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66c1586c0 con 0x7fb66c0b0ed0 2024-03-04T08:18:08.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb672800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb66c06c210 con 0x7fb66c06ddf0 2024-03-04T08:18:08.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb672800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c06c4f0 con 0x7fb66c1582e0 2024-03-04T08:18:08.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb66be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb66c06ddf0 0x7fb66c15af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:08.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb66be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb66c06ddf0 0x7fb66c15af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58072/0 (socket says 172.21.15.113:58072) 2024-03-04T08:18:08.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb66be00640 1 -- 172.21.15.113:0/639293642 learned_addr learned my addr 172.21.15.113:0/639293642 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:08.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb66b400640 1 --2- 172.21.15.113:0/639293642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb66c1582e0 0x7fb66c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:08.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb66be00640 1 -- 172.21.15.113:0/639293642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb66c1582e0 msgr2=0x7fb66c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:08.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb66be00640 1 --2- 172.21.15.113:0/639293642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb66c1582e0 0x7fb66c0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:08.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb66be00640 1 -- 172.21.15.113:0/639293642 >> v1:172.21.15.136:6789/0 conn(0x7fb66c0b0ed0 legacy=0x7fb66c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:08.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.247+0000 7fb66be00640 1 -- 172.21.15.113:0/639293642 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c06f6a0 con 0x7fb66c06ddf0 2024-03-04T08:18:08.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb66be00640 1 --2- 172.21.15.113:0/639293642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c06ddf0 0x7fb66c15af40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb664009870 tx=0x7fb66401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66381a5cc1b6a555 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:08.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb66aa00640 1 -- 172.21.15.113:0/639293642 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6640b2070 con 0x7fb66c06ddf0 2024-03-04T08:18:08.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb66aa00640 1 -- 172.21.15.113:0/639293642 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6640a9070 con 0x7fb66c06ddf0 2024-03-04T08:18:08.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb66aa00640 1 -- 172.21.15.113:0/639293642 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6640ad880 con 0x7fb66c06ddf0 2024-03-04T08:18:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb672800640 1 -- 172.21.15.113:0/639293642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c06ddf0 msgr2=0x7fb66c15af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb672800640 1 --2- 172.21.15.113:0/639293642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c06ddf0 0x7fb66c15af40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb664009870 tx=0x7fb66401c560 comp rx=0 tx=0).stop 2024-03-04T08:18:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb672800640 1 -- 172.21.15.113:0/639293642 shutdown_connections 2024-03-04T08:18:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb672800640 1 --2- 172.21.15.113:0/639293642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c06ddf0 0x7fb66c15af40 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb672800640 1 --2- 172.21.15.113:0/639293642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb66c1582e0 0x7fb66c0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb672800640 1 -- 172.21.15.113:0/639293642 >> 172.21.15.113:0/639293642 conn(0x7fb66c066db0 msgr2=0x7fb66c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb672800640 1 -- 172.21.15.113:0/639293642 shutdown_connections 2024-03-04T08:18:08.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.251+0000 7fb672800640 1 -- 172.21.15.113:0/639293642 wait complete. 2024-03-04T08:18:08.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb672800640 1 Processor -- start 2024-03-04T08:18:08.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb672800640 1 -- start start 2024-03-04T08:18:08.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb672800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb66c06ddf0 0x7fb66c0cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:08.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb672800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb66c0b0ed0 0x7fb66c0ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:08.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb672800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c1582e0 0x7fb66c151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:08.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb672800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c0ce9e0 con 0x7fb66c1582e0 2024-03-04T08:18:08.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb672800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c06c7d0 con 0x7fb66c0b0ed0 2024-03-04T08:18:08.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb672800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66c0ceb60 con 0x7fb66c06ddf0 2024-03-04T08:18:08.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb66be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c1582e0 0x7fb66c151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb66be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c1582e0 0x7fb66c151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58086/0 (socket says 172.21.15.113:58086) 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb66be00640 1 -- 172.21.15.113:0/3355700072 learned_addr learned my addr 172.21.15.113:0/3355700072 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb66b400640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb66c06ddf0 0x7fb66c0cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb662a00640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb66c0b0ed0 0x7fb66c0ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb66be00640 1 -- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb66c06ddf0 msgr2=0x7fb66c0cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb66be00640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb66c06ddf0 0x7fb66c0cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb66be00640 1 -- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb66c0b0ed0 msgr2=0x7fb66c0ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb66be00640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb66c0b0ed0 0x7fb66c0ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb66be00640 1 -- 172.21.15.113:0/3355700072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66c0b28d0 con 0x7fb66c1582e0 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb662a00640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb66c0b0ed0 0x7fb66c0ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb66be00640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c1582e0 0x7fb66c151010 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb66401a190 tx=0x7fb664019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb663e00640 1 -- 172.21.15.113:0/3355700072 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6640bb070 con 0x7fb66c1582e0 2024-03-04T08:18:08.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb663e00640 1 -- 172.21.15.113:0/3355700072 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6640adbd0 con 0x7fb66c1582e0 2024-03-04T08:18:08.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb663e00640 1 -- 172.21.15.113:0/3355700072 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6640b2040 con 0x7fb66c1582e0 2024-03-04T08:18:08.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb672800640 1 -- 172.21.15.113:0/3355700072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66c0cd470 con 0x7fb66c1582e0 2024-03-04T08:18:08.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.255+0000 7fb672800640 1 -- 172.21.15.113:0/3355700072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66c0cda30 con 0x7fb66c1582e0 2024-03-04T08:18:08.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.259+0000 7fb672800640 1 -- 172.21.15.113:0/3355700072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66c068eb0 con 0x7fb66c1582e0 2024-03-04T08:18:08.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.259+0000 7fb663e00640 1 -- 172.21.15.113:0/3355700072 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6640be070 con 0x7fb66c1582e0 2024-03-04T08:18:08.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.259+0000 7fb663e00640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb648066f90 0x7fb648069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:08.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.259+0000 7fb663e00640 1 -- 172.21.15.113:0/3355700072 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7fb66411e9a0 con 0x7fb66c1582e0 2024-03-04T08:18:08.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.259+0000 7fb66b400640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb648066f90 0x7fb648069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:08.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.259+0000 7fb66b400640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb648066f90 0x7fb648069450 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009ac0 tx=0x7fb65c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:08.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.267+0000 7fb663e00640 1 -- 172.21.15.113:0/3355700072 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb664016150 con 0x7fb66c1582e0 2024-03-04T08:18:08.336 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:18:08.340 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:18:08.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.503+0000 7fb672800640 1 -- 172.21.15.113:0/3355700072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb66c0c3540 con 0x7fb66c1582e0 2024-03-04T08:18:08.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.503+0000 7fb663e00640 1 -- 172.21.15.113:0/3355700072 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 159 v159) v1 ==== 95+0+5827 (secure 0 0 0) 0x7fb6640eea30 con 0x7fb66c1582e0 2024-03-04T08:18:08.502 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:08.503 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":159,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":159,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:07.455772+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839]},"id":1}]} 2024-03-04T08:18:08.503 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 159 2024-03-04T08:18:08.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 -- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb648066f90 msgr2=0x7fb648069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:08.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb648066f90 0x7fb648069450 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009ac0 tx=0x7fb65c009190 comp rx=0 tx=0).stop 2024-03-04T08:18:08.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 -- 172.21.15.113:0/3355700072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c1582e0 msgr2=0x7fb66c151010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:08.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c1582e0 0x7fb66c151010 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb66401a190 tx=0x7fb664019300 comp rx=0 tx=0).stop 2024-03-04T08:18:08.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 -- 172.21.15.113:0/3355700072 shutdown_connections 2024-03-04T08:18:08.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb648066f90 0x7fb648069450 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:08.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb66c1582e0 0x7fb66c151010 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:08.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb66c0b0ed0 0x7fb66c0ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:08.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 --2- 172.21.15.113:0/3355700072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb66c06ddf0 0x7fb66c0cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:08.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 -- 172.21.15.113:0/3355700072 >> 172.21.15.113:0/3355700072 conn(0x7fb66c066db0 msgr2=0x7fb66c153230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:08.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 -- 172.21.15.113:0/3355700072 shutdown_connections 2024-03-04T08:18:08.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:08.507+0000 7fb672800640 1 -- 172.21.15.113:0/3355700072 wait complete. 2024-03-04T08:18:08.532 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:18:09.454 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:09.454 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 76 -eq 48 ] 2024-03-04T08:18:09.454 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.76 2024-03-04T08:18:09.454 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.76 2024-03-04T08:18:09.454 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.76 2024-03-04T08:18:09.455 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.76 2024-03-04T08:18:09.455 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:10.533 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:10.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.699+0000 7ffa99800640 1 Processor -- start 2024-03-04T08:18:10.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.699+0000 7ffa99800640 1 -- start start 2024-03-04T08:18:10.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa99800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 0x7ffa9406c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:10.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa99800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffa940c8a90 0x7ffa9406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:10.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa99800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9406df00 con 0x7ffa9406f2c0 2024-03-04T08:18:10.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa99800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9406e1e0 con 0x7ffa940c8a90 2024-03-04T08:18:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa99800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa940ce1a0 con 0x7ffa94157f80 2024-03-04T08:18:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 0x7ffa9406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa8b400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffa940c8a90 0x7ffa9406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:10.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa8b400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffa940c8a90 0x7ffa9406ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:60786/0 (socket says 172.21.15.113:60786) 2024-03-04T08:18:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 0x7ffa9406c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:43048/0 (socket says 172.21.15.113:43048) 2024-03-04T08:18:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa8b400640 1 -- 172.21.15.113:0/2013207754 learned_addr learned my addr 172.21.15.113:0/2013207754 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa8b400640 1 -- 172.21.15.113:0/2013207754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 msgr2=0x7ffa9406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa8b400640 1 --2- 172.21.15.113:0/2013207754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 0x7ffa9406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa8b400640 1 -- 172.21.15.113:0/2013207754 >> v1:172.21.15.136:6789/0 conn(0x7ffa9406f2c0 legacy=0x7ffa940c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa8b400640 1 -- 172.21.15.113:0/2013207754 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa940ce480 con 0x7ffa940c8a90 2024-03-04T08:18:10.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.703+0000 7ffa93e00640 1 --2- 172.21.15.113:0/2013207754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 0x7ffa9406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:18:10.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa8b400640 1 --2- 172.21.15.113:0/2013207754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa940c8a90 0x7ffa9406ed80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009870 tx=0x7ffa7c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd0b9214b11c1a3d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa93400640 1 -- 172.21.15.113:0/2013207754 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0b2070 con 0x7ffa940c8a90 2024-03-04T08:18:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa93400640 1 -- 172.21.15.113:0/2013207754 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c0a9070 con 0x7ffa940c8a90 2024-03-04T08:18:10.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa93400640 1 -- 172.21.15.113:0/2013207754 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0ad880 con 0x7ffa940c8a90 2024-03-04T08:18:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa99800640 1 -- 172.21.15.113:0/2013207754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa940c8a90 msgr2=0x7ffa9406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa99800640 1 --2- 172.21.15.113:0/2013207754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa940c8a90 0x7ffa9406ed80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009870 tx=0x7ffa7c01c560 comp rx=0 tx=0).stop 2024-03-04T08:18:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa99800640 1 -- 172.21.15.113:0/2013207754 shutdown_connections 2024-03-04T08:18:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa99800640 1 --2- 172.21.15.113:0/2013207754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa940c8a90 0x7ffa9406ed80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa99800640 1 --2- 172.21.15.113:0/2013207754 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 0x7ffa9406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa99800640 1 -- 172.21.15.113:0/2013207754 >> 172.21.15.113:0/2013207754 conn(0x7ffa94072f40 msgr2=0x7ffa940706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa99800640 1 -- 172.21.15.113:0/2013207754 shutdown_connections 2024-03-04T08:18:10.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.707+0000 7ffa99800640 1 -- 172.21.15.113:0/2013207754 wait complete. 2024-03-04T08:18:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 Processor -- start 2024-03-04T08:18:10.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 -- start start 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa9406f2c0 0x7ffa941491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa940c8a90 0x7ffa940cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 0x7ffa94147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9406fdb0 con 0x7ffa9406f2c0 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9406fc30 con 0x7ffa940c8a90 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa94167a60 con 0x7ffa94157f80 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa9406f2c0 0x7ffa941491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa93e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa9406f2c0 0x7ffa941491f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58098/0 (socket says 172.21.15.113:58098) 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa93e00640 1 -- 172.21.15.113:0/2256266520 learned_addr learned my addr 172.21.15.113:0/2256266520 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa8b400640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa940c8a90 0x7ffa940cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa98e00640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 0x7ffa94147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa93e00640 1 -- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 msgr2=0x7ffa94147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:10.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa93e00640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 0x7ffa94147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa93e00640 1 -- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa940c8a90 msgr2=0x7ffa940cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa93e00640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa940c8a90 0x7ffa940cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa93e00640 1 -- 172.21.15.113:0/2256266520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa941481e0 con 0x7ffa9406f2c0 2024-03-04T08:18:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa93e00640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa9406f2c0 0x7ffa941491f0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400e980 tx=0x7ffa8400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa90c00640 1 -- 172.21.15.113:0/2256266520 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84017070 con 0x7ffa9406f2c0 2024-03-04T08:18:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa90c00640 1 -- 172.21.15.113:0/2256266520 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8400ca00 con 0x7ffa9406f2c0 2024-03-04T08:18:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 -- 172.21.15.113:0/2256266520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9414da10 con 0x7ffa9406f2c0 2024-03-04T08:18:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa90c00640 1 -- 172.21.15.113:0/2256266520 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400cdd0 con 0x7ffa9406f2c0 2024-03-04T08:18:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 -- 172.21.15.113:0/2256266520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa94148840 con 0x7ffa9406f2c0 2024-03-04T08:18:10.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.711+0000 7ffa99800640 1 -- 172.21.15.113:0/2256266520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa941f43e0 con 0x7ffa9406f2c0 2024-03-04T08:18:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.715+0000 7ffa90c00640 1 -- 172.21.15.113:0/2256266520 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa8401a070 con 0x7ffa9406f2c0 2024-03-04T08:18:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.715+0000 7ffa90c00640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 0x7ffa70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.715+0000 7ffa90c00640 1 -- 172.21.15.113:0/2256266520 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7ffa84085ac0 con 0x7ffa9406f2c0 2024-03-04T08:18:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.715+0000 7ffa8b400640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 0x7ffa70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.719+0000 7ffa8b400640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 0x7ffa70069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0af000 tx=0x7ffa7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.719+0000 7ffa90c00640 1 -- 172.21.15.113:0/2256266520 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ffa941f43e0 con 0x7ffa9406f2c0 2024-03-04T08:18:10.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.955+0000 7ffa99800640 1 -- 172.21.15.113:0/2256266520 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa94075a80 con 0x7ffa9406f2c0 2024-03-04T08:18:10.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa90c00640 1 -- 172.21.15.113:0/2256266520 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 159 v159) v1 ==== 95+0+5827 (secure 0 0 0) 0x7ffa84055ba0 con 0x7ffa9406f2c0 2024-03-04T08:18:10.955 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:10.955 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":159,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":159,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:07.455772+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839]},"id":1}]} 2024-03-04T08:18:10.955 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 159 2024-03-04T08:18:10.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 -- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 msgr2=0x7ffa70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:10.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 0x7ffa70069420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0af000 tx=0x7ffa7c008040 comp rx=0 tx=0).stop 2024-03-04T08:18:10.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 -- 172.21.15.113:0/2256266520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa9406f2c0 msgr2=0x7ffa941491f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:10.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa9406f2c0 0x7ffa941491f0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400e980 tx=0x7ffa8400ee40 comp rx=0 tx=0).stop 2024-03-04T08:18:10.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 -- 172.21.15.113:0/2256266520 shutdown_connections 2024-03-04T08:18:10.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffa94157f80 0x7ffa94147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:10.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa940c8a90 0x7ffa940cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:10.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa70066f60 0x7ffa70069420 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:10.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 --2- 172.21.15.113:0/2256266520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa9406f2c0 0x7ffa941491f0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:10.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 -- 172.21.15.113:0/2256266520 >> 172.21.15.113:0/2256266520 conn(0x7ffa94072f40 msgr2=0x7ffa94151be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:10.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 -- 172.21.15.113:0/2256266520 shutdown_connections 2024-03-04T08:18:10.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:10.959+0000 7ffa99800640 1 -- 172.21.15.113:0/2256266520 wait complete. 2024-03-04T08:18:10.977 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 159, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 159, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7692": { "addr": "172.21.15.113:6833/1928887637", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 1928887637, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 1928887637, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7692, "incarnation": 139, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:active", "state_seq": 22 }, "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:reconnect", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 72, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:18:07.455772+0000", "qdb_cluster": [ 7692, 7839 ], "qdb_leader": 7839, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 8032, "mds_2": 7692 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:18:12.979 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:13.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.127+0000 7f2d67200640 1 Processor -- start 2024-03-04T08:18:13.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.127+0000 7f2d67200640 1 -- start start 2024-03-04T08:18:13.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d67200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2d60157ef0 0x7f2d600c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d67200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2d60070df0 0x7f2d6015b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d67200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d600c9c50 con 0x7f2d6015b6c0 2024-03-04T08:18:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d67200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6015a9a0 con 0x7f2d60157ef0 2024-03-04T08:18:13.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d67200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6006de30 con 0x7f2d60070df0 2024-03-04T08:18:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d65e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2d60157ef0 0x7f2d600c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d65e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2d60157ef0 0x7f2d600c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58102/0 (socket says 172.21.15.113:58102) 2024-03-04T08:18:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d65e00640 1 -- 172.21.15.113:0/3154909957 learned_addr learned my addr 172.21.15.113:0/3154909957 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d65400640 1 --2- 172.21.15.113:0/3154909957 >> v2:172.21.15.136:3300/0 conn(0x7f2d60070df0 0x7f2d6015b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d65e00640 1 -- 172.21.15.113:0/3154909957 >> v2:172.21.15.136:3300/0 conn(0x7f2d60070df0 msgr2=0x7f2d6015b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d65e00640 1 --2- 172.21.15.113:0/3154909957 >> v2:172.21.15.136:3300/0 conn(0x7f2d60070df0 0x7f2d6015b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d65e00640 1 -- 172.21.15.113:0/3154909957 >> v1:172.21.15.113:6789/0 conn(0x7f2d6015b6c0 legacy=0x7f2d600cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d64a00640 1 -- 172.21.15.113:0/3154909957 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231337611 0 0) 0x7f2d600c9c50 con 0x7f2d6015b6c0 2024-03-04T08:18:13.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.131+0000 7f2d65e00640 1 -- 172.21.15.113:0/3154909957 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6006c210 con 0x7f2d60157ef0 2024-03-04T08:18:13.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.135+0000 7f2d65e00640 1 --2- 172.21.15.113:0/3154909957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 0x7f2d600c8a90 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f2d600ceb20 tx=0x7f2d5401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e21d21d6489b0d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:13.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.135+0000 7f2d64a00640 1 -- 172.21.15.113:0/3154909957 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540ab650 con 0x7f2d60157ef0 2024-03-04T08:18:13.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.135+0000 7f2d64a00640 1 -- 172.21.15.113:0/3154909957 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d540b0070 con 0x7f2d60157ef0 2024-03-04T08:18:13.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.135+0000 7f2d64a00640 1 -- 172.21.15.113:0/3154909957 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540aba10 con 0x7f2d60157ef0 2024-03-04T08:18:13.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.139+0000 7f2d67200640 1 -- 172.21.15.113:0/3154909957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 msgr2=0x7f2d600c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:13.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.139+0000 7f2d67200640 1 --2- 172.21.15.113:0/3154909957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 0x7f2d600c8a90 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f2d600ceb20 tx=0x7f2d5401c540 comp rx=0 tx=0).stop 2024-03-04T08:18:13.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.139+0000 7f2d67200640 1 -- 172.21.15.113:0/3154909957 shutdown_connections 2024-03-04T08:18:13.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.139+0000 7f2d67200640 1 --2- 172.21.15.113:0/3154909957 >> v2:172.21.15.136:3300/0 conn(0x7f2d60070df0 0x7f2d6015b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:13.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.139+0000 7f2d67200640 1 --2- 172.21.15.113:0/3154909957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 0x7f2d600c8a90 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:13.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.139+0000 7f2d67200640 1 -- 172.21.15.113:0/3154909957 >> 172.21.15.113:0/3154909957 conn(0x7f2d60072f40 msgr2=0x7f2d600706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:13.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.139+0000 7f2d67200640 1 -- 172.21.15.113:0/3154909957 shutdown_connections 2024-03-04T08:18:13.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.139+0000 7f2d67200640 1 -- 172.21.15.113:0/3154909957 wait complete. 2024-03-04T08:18:13.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.139+0000 7f2d67200640 1 Processor -- start 2024-03-04T08:18:13.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.139+0000 7f2d67200640 1 -- start start 2024-03-04T08:18:13.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d67200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d60070df0 0x7f2d60153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:13.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d67200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 0x7f2d6014e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d67200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6015b6c0 0x7f2d601f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d67200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d60158450 con 0x7f2d60157ef0 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d67200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d601582d0 con 0x7f2d6015b6c0 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d67200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d600ce6f0 con 0x7f2d60070df0 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d65400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 0x7f2d6014e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d65400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 0x7f2d6014e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58104/0 (socket says 172.21.15.113:58104) 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d65400640 1 -- 172.21.15.113:0/2718856398 learned_addr learned my addr 172.21.15.113:0/2718856398 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d65e00640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d60070df0 0x7f2d60153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d66800640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6015b6c0 0x7f2d601f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d65400640 1 -- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d60070df0 msgr2=0x7f2d60153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d65400640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d60070df0 0x7f2d60153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d65400640 1 -- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6015b6c0 msgr2=0x7f2d601f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d65400640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6015b6c0 0x7f2d601f7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:13.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d65400640 1 -- 172.21.15.113:0/2718856398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d60148670 con 0x7f2d60157ef0 2024-03-04T08:18:13.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d65400640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 0x7f2d6014e210 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e960 tx=0x7f2d5000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:13.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d5a000640 1 -- 172.21.15.113:0/2718856398 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50017070 con 0x7f2d60157ef0 2024-03-04T08:18:13.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d5a000640 1 -- 172.21.15.113:0/2718856398 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5000c9a0 con 0x7f2d60157ef0 2024-03-04T08:18:13.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d5a000640 1 -- 172.21.15.113:0/2718856398 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5000cd50 con 0x7f2d60157ef0 2024-03-04T08:18:13.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d67200640 1 -- 172.21.15.113:0/2718856398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d601f6ba0 con 0x7f2d60157ef0 2024-03-04T08:18:13.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d67200640 1 -- 172.21.15.113:0/2718856398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d60152cc0 con 0x7f2d60157ef0 2024-03-04T08:18:13.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.143+0000 7f2d67200640 1 -- 172.21.15.113:0/2718856398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d60075a80 con 0x7f2d60157ef0 2024-03-04T08:18:13.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.147+0000 7f2d5a000640 1 -- 172.21.15.113:0/2718856398 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d5001a070 con 0x7f2d60157ef0 2024-03-04T08:18:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.147+0000 7f2d5a000640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d44066f90 0x7f2d44069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.147+0000 7f2d5a000640 1 -- 172.21.15.113:0/2718856398 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7f2d500857c0 con 0x7f2d60157ef0 2024-03-04T08:18:13.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.147+0000 7f2d65e00640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d44066f90 0x7f2d44069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:13.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.147+0000 7f2d65e00640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d44066f90 0x7f2d44069450 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2d540af000 tx=0x7f2d540a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:13.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.155+0000 7f2d5a000640 1 -- 172.21.15.113:0/2718856398 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2d500558e0 con 0x7f2d60157ef0 2024-03-04T08:18:13.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 -- 172.21.15.113:0/2718856398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d600cdf90 con 0x7f2d60157ef0 2024-03-04T08:18:13.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d5a000640 1 -- 172.21.15.113:0/2718856398 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 159 v159) v1 ==== 95+0+5827 (secure 0 0 0) 0x7f2d50059980 con 0x7f2d60157ef0 2024-03-04T08:18:13.348 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:13.348 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":159,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":159,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:07.455772+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839]},"id":1}]} 2024-03-04T08:18:13.348 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 159 2024-03-04T08:18:13.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 -- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d44066f90 msgr2=0x7f2d44069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:13.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d44066f90 0x7f2d44069450 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f2d540af000 tx=0x7f2d540a7000 comp rx=0 tx=0).stop 2024-03-04T08:18:13.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 -- 172.21.15.113:0/2718856398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 msgr2=0x7f2d6014e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:13.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 0x7f2d6014e210 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000e960 tx=0x7f2d5000ee20 comp rx=0 tx=0).stop 2024-03-04T08:18:13.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 -- 172.21.15.113:0/2718856398 shutdown_connections 2024-03-04T08:18:13.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2d6015b6c0 0x7f2d601f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:13.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2d44066f90 0x7f2d44069450 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:13.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d60157ef0 0x7f2d6014e210 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:13.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 --2- 172.21.15.113:0/2718856398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2d60070df0 0x7f2d60153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:13.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 -- 172.21.15.113:0/2718856398 >> 172.21.15.113:0/2718856398 conn(0x7f2d60072f40 msgr2=0x7f2d60156f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:13.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 -- 172.21.15.113:0/2718856398 shutdown_connections 2024-03-04T08:18:13.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:13.351+0000 7f2d67200640 1 -- 172.21.15.113:0/2718856398 wait complete. 2024-03-04T08:18:13.370 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:18:15.199 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:15.199 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.77 2024-03-04T08:18:15.200 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 77 -eq 48 ] 2024-03-04T08:18:15.200 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.77 2024-03-04T08:18:15.200 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.77 2024-03-04T08:18:15.200 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.77 2024-03-04T08:18:15.200 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:15.371 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.510+0000 7f780cc00640 1 Processor -- start 2024-03-04T08:18:15.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f780cc00640 1 -- start start 2024-03-04T08:18:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f780cc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f780814e4c0 0x7f7808072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f780cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 0x7f78080731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f780cc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78080c27f0 con 0x7f78080c2d80 2024-03-04T08:18:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f780cc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780815ad60 con 0x7f780814e4c0 2024-03-04T08:18:15.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f780cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78080c2a90 con 0x7f78080c22e0 2024-03-04T08:18:15.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f7805400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 0x7f78080731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:15.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f7805e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f780814e4c0 0x7f7808072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:15.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f7805400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 0x7f78080731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:43076/0 (socket says 172.21.15.113:43076) 2024-03-04T08:18:15.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f7805400640 1 -- 172.21.15.113:0/1646744194 learned_addr learned my addr 172.21.15.113:0/1646744194 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:15.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f7805400640 1 -- 172.21.15.113:0/1646744194 >> v2:172.21.15.136:3300/0 conn(0x7f780814e4c0 msgr2=0x7f7808072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:15.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f7805400640 1 --2- 172.21.15.113:0/1646744194 >> v2:172.21.15.136:3300/0 conn(0x7f780814e4c0 0x7f7808072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:15.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f7805400640 1 -- 172.21.15.113:0/1646744194 >> v1:172.21.15.136:6789/0 conn(0x7f78080c2d80 legacy=0x7f780815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:15.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f7805400640 1 -- 172.21.15.113:0/1646744194 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7808070c60 con 0x7f78080c22e0 2024-03-04T08:18:15.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.514+0000 7f7805e00640 1 --2- 172.21.15.113:0/1646744194 >> v2:172.21.15.136:3300/0 conn(0x7f780814e4c0 0x7f7808072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:18:15.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.518+0000 7f7805400640 1 --2- 172.21.15.113:0/1646744194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 0x7f78080731c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009870 tx=0x7f77f401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29ab325a26414a0e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:15.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.518+0000 7f7804a00640 1 -- 172.21.15.113:0/1646744194 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f40b5070 con 0x7f78080c22e0 2024-03-04T08:18:15.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.518+0000 7f7804a00640 1 -- 172.21.15.113:0/1646744194 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f40a8a50 con 0x7f78080c22e0 2024-03-04T08:18:15.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.518+0000 7f7804a00640 1 -- 172.21.15.113:0/1646744194 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f40a8e20 con 0x7f78080c22e0 2024-03-04T08:18:15.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 -- 172.21.15.113:0/1646744194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 msgr2=0x7f78080731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:15.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 --2- 172.21.15.113:0/1646744194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 0x7f78080731c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009870 tx=0x7f77f401ed50 comp rx=0 tx=0).stop 2024-03-04T08:18:15.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 -- 172.21.15.113:0/1646744194 shutdown_connections 2024-03-04T08:18:15.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 --2- 172.21.15.113:0/1646744194 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 0x7f78080731c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:15.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 --2- 172.21.15.113:0/1646744194 >> v2:172.21.15.136:3300/0 conn(0x7f780814e4c0 0x7f7808072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:15.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 -- 172.21.15.113:0/1646744194 >> 172.21.15.113:0/1646744194 conn(0x7f780806e820 msgr2=0x7f780806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:15.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 -- 172.21.15.113:0/1646744194 shutdown_connections 2024-03-04T08:18:15.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 -- 172.21.15.113:0/1646744194 wait complete. 2024-03-04T08:18:15.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 Processor -- start 2024-03-04T08:18:15.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 -- start start 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 0x7f78080c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080c2d80 0x7f780814b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f780814e4c0 0x7f78080c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7808071500 con 0x7f78080c2d80 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7808071220 con 0x7f780814e4c0 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f780cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780814e8a0 con 0x7f78080c22e0 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f7805400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080c2d80 0x7f780814b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f7805400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080c2d80 0x7f780814b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58108/0 (socket says 172.21.15.113:58108) 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f7805400640 1 -- 172.21.15.113:0/3177679676 learned_addr learned my addr 172.21.15.113:0/3177679676 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f7805e00640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 0x7f78080c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f7806800640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f780814e4c0 0x7f78080c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:15.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f7805400640 1 -- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 msgr2=0x7f78080c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f7805400640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 0x7f78080c6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f7805400640 1 -- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f780814e4c0 msgr2=0x7f78080c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.522+0000 7f7805400640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f780814e4c0 0x7f78080c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.526+0000 7f7805400640 1 -- 172.21.15.113:0/3177679676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780815a660 con 0x7f78080c2d80 2024-03-04T08:18:15.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.526+0000 7f7805400640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080c2d80 0x7f780814b4a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f77f40b3040 tx=0x7f77f401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:15.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.526+0000 7f77fe000640 1 -- 172.21.15.113:0/3177679676 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f40b5070 con 0x7f78080c2d80 2024-03-04T08:18:15.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.526+0000 7f77fe000640 1 -- 172.21.15.113:0/3177679676 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f40b0de0 con 0x7f78080c2d80 2024-03-04T08:18:15.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.526+0000 7f780cc00640 1 -- 172.21.15.113:0/3177679676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78080c4300 con 0x7f78080c2d80 2024-03-04T08:18:15.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.526+0000 7f77fe000640 1 -- 172.21.15.113:0/3177679676 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f40bb670 con 0x7f78080c2d80 2024-03-04T08:18:15.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.526+0000 7f780cc00640 1 -- 172.21.15.113:0/3177679676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78080c48c0 con 0x7f78080c2d80 2024-03-04T08:18:15.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.526+0000 7f780cc00640 1 -- 172.21.15.113:0/3177679676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7808075c00 con 0x7f78080c2d80 2024-03-04T08:18:15.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.526+0000 7f77fe000640 1 -- 172.21.15.113:0/3177679676 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77f40c2060 con 0x7f78080c2d80 2024-03-04T08:18:15.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.530+0000 7f77fe000640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77e8066f60 0x7f77e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:15.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.530+0000 7f77fe000640 1 -- 172.21.15.113:0/3177679676 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7f77f4122d90 con 0x7f78080c2d80 2024-03-04T08:18:15.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.530+0000 7f7805e00640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77e8066f60 0x7f77e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:15.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.530+0000 7f7805e00640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77e8066f60 0x7f77e8069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009580 tx=0x7f77f8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:15.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.530+0000 7f77fe000640 1 -- 172.21.15.113:0/3177679676 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f77f40188b0 con 0x7f78080c2d80 2024-03-04T08:18:15.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.770+0000 7f780cc00640 1 -- 172.21.15.113:0/3177679676 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7808155190 con 0x7f78080c2d80 2024-03-04T08:18:15.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.770+0000 7f77fe000640 1 -- 172.21.15.113:0/3177679676 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f77f40f2d90 con 0x7f78080c2d80 2024-03-04T08:18:15.770 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:15.770 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":160,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:13.108372+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692]},"id":1}]} 2024-03-04T08:18:15.770 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 160 2024-03-04T08:18:15.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 -- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77e8066f60 msgr2=0x7f77e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:15.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77e8066f60 0x7f77e8069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f77f8009580 tx=0x7f77f8009190 comp rx=0 tx=0).stop 2024-03-04T08:18:15.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 -- 172.21.15.113:0/3177679676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080c2d80 msgr2=0x7f780814b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:15.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080c2d80 0x7f780814b4a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f77f40b3040 tx=0x7f77f401baf0 comp rx=0 tx=0).stop 2024-03-04T08:18:15.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 -- 172.21.15.113:0/3177679676 shutdown_connections 2024-03-04T08:18:15.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f780814e4c0 0x7f78080c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:15.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77e8066f60 0x7f77e8069420 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:15.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78080c2d80 0x7f780814b4a0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:15.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 --2- 172.21.15.113:0/3177679676 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f78080c22e0 0x7f78080c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:15.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 -- 172.21.15.113:0/3177679676 >> 172.21.15.113:0/3177679676 conn(0x7f780806e820 msgr2=0x7f780814c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:15.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 -- 172.21.15.113:0/3177679676 shutdown_connections 2024-03-04T08:18:15.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:15.774+0000 7f780cc00640 1 -- 172.21.15.113:0/3177679676 wait complete. 2024-03-04T08:18:15.791 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:18:17.792 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.930+0000 7f29a5e00640 1 Processor -- start 2024-03-04T08:18:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.930+0000 7f29a5e00640 1 -- start start 2024-03-04T08:18:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f29a5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a0152a50 0x7f29a00ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f29a5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29a006ff20 0x7f29a00ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f29a5e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a0152e30 con 0x7f29a00c8b90 2024-03-04T08:18:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f29a5e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a00c31b0 con 0x7f29a006ff20 2024-03-04T08:18:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f29a5e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a006f840 con 0x7f29a0152a50 2024-03-04T08:18:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f299e000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29a006ff20 0x7f29a00ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f299e000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f29a006ff20 0x7f29a00ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38244/0 (socket says 172.21.15.113:38244) 2024-03-04T08:18:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f299e000640 1 -- 172.21.15.113:0/1098944705 learned_addr learned my addr 172.21.15.113:0/1098944705 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f299ea00640 1 --2- 172.21.15.113:0/1098944705 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a0152a50 0x7f29a00ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:17.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f299e000640 1 -- 172.21.15.113:0/1098944705 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a0152a50 msgr2=0x7f29a00ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:17.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f299e000640 1 --2- 172.21.15.113:0/1098944705 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a0152a50 0x7f29a00ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:17.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f299e000640 1 -- 172.21.15.113:0/1098944705 >> v1:172.21.15.136:6789/0 conn(0x7f29a00c8b90 legacy=0x7f29a015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:17.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.934+0000 7f299e000640 1 -- 172.21.15.113:0/1098944705 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a006f280 con 0x7f29a006ff20 2024-03-04T08:18:17.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f299e000640 1 --2- 172.21.15.113:0/1098944705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a006ff20 0x7f29a00ce660 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f298c009870 tx=0x7f298c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c63ac3fb6316e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:17.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f299d600640 1 -- 172.21.15.113:0/1098944705 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c0a9620 con 0x7f29a006ff20 2024-03-04T08:18:17.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f299d600640 1 -- 172.21.15.113:0/1098944705 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c0a97c0 con 0x7f29a006ff20 2024-03-04T08:18:17.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f299d600640 1 -- 172.21.15.113:0/1098944705 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c0a9b90 con 0x7f29a006ff20 2024-03-04T08:18:17.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f29a5e00640 1 -- 172.21.15.113:0/1098944705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a006ff20 msgr2=0x7f29a00ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:17.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f29a5e00640 1 --2- 172.21.15.113:0/1098944705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a006ff20 0x7f29a00ce660 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f298c009870 tx=0x7f298c01c560 comp rx=0 tx=0).stop 2024-03-04T08:18:17.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f29a5e00640 1 -- 172.21.15.113:0/1098944705 shutdown_connections 2024-03-04T08:18:17.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f29a5e00640 1 --2- 172.21.15.113:0/1098944705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a006ff20 0x7f29a00ce660 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:17.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f29a5e00640 1 --2- 172.21.15.113:0/1098944705 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a0152a50 0x7f29a00ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:17.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f29a5e00640 1 -- 172.21.15.113:0/1098944705 >> 172.21.15.113:0/1098944705 conn(0x7f29a00734d0 msgr2=0x7f29a0066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:17.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f29a5e00640 1 -- 172.21.15.113:0/1098944705 shutdown_connections 2024-03-04T08:18:17.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.938+0000 7f29a5e00640 1 -- 172.21.15.113:0/1098944705 wait complete. 2024-03-04T08:18:17.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f29a5e00640 1 Processor -- start 2024-03-04T08:18:17.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f29a5e00640 1 -- start start 2024-03-04T08:18:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f29a5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a006ff20 0x7f29a0151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f29a5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a00c8b90 0x7f29a01454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:17.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f29a5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0152a50 0x7f29a0143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:17.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f29a5e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0165a00 con 0x7f29a0152a50 2024-03-04T08:18:17.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f29a5e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0165880 con 0x7f29a006ff20 2024-03-04T08:18:17.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f29a5e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0165b80 con 0x7f29a00c8b90 2024-03-04T08:18:17.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0152a50 0x7f29a0143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:17.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0152a50 0x7f29a0143e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38256/0 (socket says 172.21.15.113:38256) 2024-03-04T08:18:17.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a006ff20 0x7f29a0151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:17.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299f400640 1 -- 172.21.15.113:0/205305546 learned_addr learned my addr 172.21.15.113:0/205305546 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:17.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299e000640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a00c8b90 0x7f29a01454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:17.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299f400640 1 -- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a00c8b90 msgr2=0x7f29a01454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:17.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299f400640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a00c8b90 0x7f29a01454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:17.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299f400640 1 -- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a006ff20 msgr2=0x7f29a0151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:17.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299f400640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a006ff20 0x7f29a0151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299f400640 1 -- 172.21.15.113:0/205305546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0161f40 con 0x7f29a0152a50 2024-03-04T08:18:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f299f400640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0152a50 0x7f29a0143e20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f299800c7a0 tx=0x7f299800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f2997400640 1 -- 172.21.15.113:0/205305546 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2998003b80 con 0x7f29a0152a50 2024-03-04T08:18:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.942+0000 7f2997400640 1 -- 172.21.15.113:0/205305546 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2998003d20 con 0x7f29a0152a50 2024-03-04T08:18:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.946+0000 7f29a5e00640 1 -- 172.21.15.113:0/205305546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a0162220 con 0x7f29a0152a50 2024-03-04T08:18:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.946+0000 7f2997400640 1 -- 172.21.15.113:0/205305546 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299800aa90 con 0x7f29a0152a50 2024-03-04T08:18:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.946+0000 7f29a5e00640 1 -- 172.21.15.113:0/205305546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a0144ce0 con 0x7f29a0152a50 2024-03-04T08:18:17.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.946+0000 7f29a5e00640 1 -- 172.21.15.113:0/205305546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a0075c00 con 0x7f29a0152a50 2024-03-04T08:18:17.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.946+0000 7f2997400640 1 -- 172.21.15.113:0/205305546 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f299800ac30 con 0x7f29a0152a50 2024-03-04T08:18:17.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.946+0000 7f2997400640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2980066f90 0x7f2980069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:17.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.946+0000 7f2997400640 1 -- 172.21.15.113:0/205305546 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7f2998086020 con 0x7f29a0152a50 2024-03-04T08:18:17.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.946+0000 7f299ea00640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2980066f90 0x7f2980069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:17.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.950+0000 7f299ea00640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2980066f90 0x7f2980069450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f29900079c0 tx=0x7f2990008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:17.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:17.950+0000 7f2997400640 1 -- 172.21.15.113:0/205305546 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f29980562c0 con 0x7f29a0152a50 2024-03-04T08:18:18.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.170+0000 7f29a5e00640 1 -- 172.21.15.113:0/205305546 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29a0162e40 con 0x7f29a0152a50 2024-03-04T08:18:18.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.170+0000 7f2997400640 1 -- 172.21.15.113:0/205305546 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f299805a360 con 0x7f29a0152a50 2024-03-04T08:18:18.170 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:18.170 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":160,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:13.108372+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,7692]},"id":1}]} 2024-03-04T08:18:18.171 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 160 2024-03-04T08:18:18.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 -- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2980066f90 msgr2=0x7f2980069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2980066f90 0x7f2980069450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f29900079c0 tx=0x7f2990008040 comp rx=0 tx=0).stop 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 -- 172.21.15.113:0/205305546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0152a50 msgr2=0x7f29a0143e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0152a50 0x7f29a0143e20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f299800c7a0 tx=0x7f299800cc60 comp rx=0 tx=0).stop 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 -- 172.21.15.113:0/205305546 shutdown_connections 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2980066f90 0x7f2980069450 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f29a0152a50 0x7f29a0143e20 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f29a00c8b90 0x7f29a01454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 --2- 172.21.15.113:0/205305546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f29a006ff20 0x7f29a0151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 -- 172.21.15.113:0/205305546 >> 172.21.15.113:0/205305546 conn(0x7f29a00734d0 msgr2=0x7f29a00cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 -- 172.21.15.113:0/205305546 shutdown_connections 2024-03-04T08:18:18.173 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:18.174+0000 7f29a5e00640 1 -- 172.21.15.113:0/205305546 wait complete. 2024-03-04T08:18:18.192 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:18:19.327 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T08:18:19.330+0000 7f9b59600640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T08:18:20.135 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:20.135 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.78 2024-03-04T08:18:20.136 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 78 -eq 48 ] 2024-03-04T08:18:20.136 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.78 2024-03-04T08:18:20.136 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.78 2024-03-04T08:18:20.136 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.78 2024-03-04T08:18:20.136 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:20.193 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:20.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd508000640 1 Processor -- start 2024-03-04T08:18:20.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd508000640 1 -- start start 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd508000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5000ce2d0 0x7fd50006ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd508000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd5000b0e30 0x7fd5000ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd508000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd50015af90 0x7fd50015b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd508000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50006c170 con 0x7fd5000ce2d0 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd508000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5000c9460 con 0x7fd5000b0e30 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd508000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50015b960 con 0x7fd50015af90 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd505600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5000ce2d0 0x7fd50006ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd505600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd5000ce2d0 0x7fd50006ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38264/0 (socket says 172.21.15.113:38264) 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd505600640 1 -- 172.21.15.113:0/3779390577 learned_addr learned my addr 172.21.15.113:0/3779390577 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd504c00640 1 --2- 172.21.15.113:0/3779390577 >> v2:172.21.15.136:3300/0 conn(0x7fd5000b0e30 0x7fd5000ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd505600640 1 -- 172.21.15.113:0/3779390577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd50015af90 msgr2=0x7fd50015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd506000640 1 --2- 172.21.15.113:0/3779390577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd50015af90 0x7fd50015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd505600640 1 --2- 172.21.15.113:0/3779390577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd50015af90 0x7fd50015b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd505600640 1 -- 172.21.15.113:0/3779390577 >> v2:172.21.15.136:3300/0 conn(0x7fd5000b0e30 msgr2=0x7fd5000ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:20.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd505600640 1 --2- 172.21.15.113:0/3779390577 >> v2:172.21.15.136:3300/0 conn(0x7fd5000b0e30 0x7fd5000ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd505600640 1 -- 172.21.15.113:0/3779390577 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50006c3c0 con 0x7fd5000ce2d0 2024-03-04T08:18:20.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.338+0000 7fd504c00640 1 --2- 172.21.15.113:0/3779390577 >> v2:172.21.15.136:3300/0 conn(0x7fd5000b0e30 0x7fd5000ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:18:20.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd505600640 1 --2- 172.21.15.113:0/3779390577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5000ce2d0 0x7fd50006ddf0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fd5000ceef0 tx=0x7fd4f401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86671fe71f24c32e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd4ffe00640 1 -- 172.21.15.113:0/3779390577 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40b5070 con 0x7fd5000ce2d0 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd4ffe00640 1 -- 172.21.15.113:0/3779390577 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f40a89d0 con 0x7fd5000ce2d0 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd4ffe00640 1 -- 172.21.15.113:0/3779390577 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f40a8da0 con 0x7fd5000ce2d0 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd508000640 1 -- 172.21.15.113:0/3779390577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5000ce2d0 msgr2=0x7fd50006ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd508000640 1 --2- 172.21.15.113:0/3779390577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5000ce2d0 0x7fd50006ddf0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fd5000ceef0 tx=0x7fd4f401ed10 comp rx=0 tx=0).stop 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd508000640 1 -- 172.21.15.113:0/3779390577 shutdown_connections 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd508000640 1 --2- 172.21.15.113:0/3779390577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd50015af90 0x7fd50015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd508000640 1 --2- 172.21.15.113:0/3779390577 >> v2:172.21.15.136:3300/0 conn(0x7fd5000b0e30 0x7fd5000ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd508000640 1 --2- 172.21.15.113:0/3779390577 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd5000ce2d0 0x7fd50006ddf0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd508000640 1 -- 172.21.15.113:0/3779390577 >> 172.21.15.113:0/3779390577 conn(0x7fd500066db0 msgr2=0x7fd5000671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:20.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd508000640 1 -- 172.21.15.113:0/3779390577 shutdown_connections 2024-03-04T08:18:20.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.342+0000 7fd508000640 1 -- 172.21.15.113:0/3779390577 wait complete. 2024-03-04T08:18:20.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd508000640 1 Processor -- start 2024-03-04T08:18:20.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd508000640 1 -- start start 2024-03-04T08:18:20.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd508000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5000b0e30 0x7fd5001e2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:20.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd508000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5000ce2d0 0x7fd5000cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:20.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd508000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50015af90 0x7fd5001f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:20.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd508000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000c9990 con 0x7fd50015af90 2024-03-04T08:18:20.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd508000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000c96b0 con 0x7fd5000ce2d0 2024-03-04T08:18:20.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd508000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5001585a0 con 0x7fd5000b0e30 2024-03-04T08:18:20.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd506000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50015af90 0x7fd5001f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:20.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd506000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50015af90 0x7fd5001f43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38268/0 (socket says 172.21.15.113:38268) 2024-03-04T08:18:20.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd506000640 1 -- 172.21.15.113:0/742428653 learned_addr learned my addr 172.21.15.113:0/742428653 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:20.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd504c00640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5000ce2d0 0x7fd5000cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:20.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd505600640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5000b0e30 0x7fd5001e2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:20.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd506000640 1 -- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5000b0e30 msgr2=0x7fd5001e2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:20.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd506000640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5000b0e30 0x7fd5001e2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd506000640 1 -- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5000ce2d0 msgr2=0x7fd5000cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:20.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd506000640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5000ce2d0 0x7fd5000cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd506000640 1 -- 172.21.15.113:0/742428653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd500152510 con 0x7fd50015af90 2024-03-04T08:18:20.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd506000640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50015af90 0x7fd5001f43a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800cd20 tx=0x7fd4f800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:20.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd4fd600640 1 -- 172.21.15.113:0/742428653 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8003ba0 con 0x7fd50015af90 2024-03-04T08:18:20.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd4fd600640 1 -- 172.21.15.113:0/742428653 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f8003d40 con 0x7fd50015af90 2024-03-04T08:18:20.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd508000640 1 -- 172.21.15.113:0/742428653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd500153490 con 0x7fd50015af90 2024-03-04T08:18:20.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd4fd600640 1 -- 172.21.15.113:0/742428653 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8004240 con 0x7fd50015af90 2024-03-04T08:18:20.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.346+0000 7fd508000640 1 -- 172.21.15.113:0/742428653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd500152230 con 0x7fd50015af90 2024-03-04T08:18:20.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.350+0000 7fd508000640 1 -- 172.21.15.113:0/742428653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd500068eb0 con 0x7fd50015af90 2024-03-04T08:18:20.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.350+0000 7fd4fd600640 1 -- 172.21.15.113:0/742428653 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4f80043e0 con 0x7fd50015af90 2024-03-04T08:18:20.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.350+0000 7fd4fd600640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd4e0066f60 0x7fd4e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:20.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.350+0000 7fd4fd600640 1 -- 172.21.15.113:0/742428653 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7fd4f80861b0 con 0x7fd50015af90 2024-03-04T08:18:20.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.350+0000 7fd505600640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd4e0066f60 0x7fd4e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:20.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.350+0000 7fd505600640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd4e0066f60 0x7fd4e0069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40b2000 tx=0x7fd4f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:20.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.354+0000 7fd4fd600640 1 -- 172.21.15.113:0/742428653 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd4f80562d0 con 0x7fd50015af90 2024-03-04T08:18:20.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 -- 172.21.15.113:0/742428653 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd500075c00 con 0x7fd50015af90 2024-03-04T08:18:20.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd4fd600640 1 -- 172.21.15.113:0/742428653 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) v1 ==== 95+0+5829 (secure 0 0 0) 0x7fd4f805a370 con 0x7fd50015af90 2024-03-04T08:18:20.616 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:20.616 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":161,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839,8032]},"id":1}]} 2024-03-04T08:18:20.616 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 161 2024-03-04T08:18:20.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 -- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd4e0066f60 msgr2=0x7fd4e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd4e0066f60 0x7fd4e0069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40b2000 tx=0x7fd4f4009040 comp rx=0 tx=0).stop 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 -- 172.21.15.113:0/742428653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50015af90 msgr2=0x7fd5001f43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50015af90 0x7fd5001f43a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800cd20 tx=0x7fd4f800a660 comp rx=0 tx=0).stop 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 -- 172.21.15.113:0/742428653 shutdown_connections 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd4e0066f60 0x7fd4e0069420 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd50015af90 0x7fd5001f43a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5000ce2d0 0x7fd5000cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 --2- 172.21.15.113:0/742428653 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd5000b0e30 0x7fd5001e2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 -- 172.21.15.113:0/742428653 >> 172.21.15.113:0/742428653 conn(0x7fd500066db0 msgr2=0x7fd5001e6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 -- 172.21.15.113:0/742428653 shutdown_connections 2024-03-04T08:18:20.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:20.618+0000 7fd508000640 1 -- 172.21.15.113:0/742428653 wait complete. 2024-03-04T08:18:20.637 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=1, replacing gid=7004 2024-03-04T08:18:20.638 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.a 2024-03-04T08:18:22.638 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.a 2024-03-04T08:18:22.639 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-04T08:18:22.639 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-04T08:18:22.642 INFO:tasks.ceph.mds.a:Started 2024-03-04T08:18:22.642 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:22.739 INFO:tasks.ceph.mds.a.smithi113.stdout:starting mds.a at 2024-03-04T08:18:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.798+0000 7fa91f000640 1 Processor -- start 2024-03-04T08:18:22.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.798+0000 7fa91f000640 1 -- start start 2024-03-04T08:18:22.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91f000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa9180c8200 0x7fa918070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:22.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91f000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa918152bd0 0x7fa9181587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:22.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91f000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9181591f0 con 0x7fa9181527f0 2024-03-04T08:18:22.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91f000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa918071df0 con 0x7fa918152bd0 2024-03-04T08:18:22.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91f000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9181597b0 con 0x7fa9180c8200 2024-03-04T08:18:22.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91d000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa918152bd0 0x7fa9181587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:22.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91d000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa918152bd0 0x7fa9181587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38288/0 (socket says 172.21.15.113:38288) 2024-03-04T08:18:22.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91d000640 1 -- 172.21.15.113:0/2454079039 learned_addr learned my addr 172.21.15.113:0/2454079039 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:22.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa90f400640 1 --2- 172.21.15.113:0/2454079039 >> v2:172.21.15.136:3300/0 conn(0x7fa9180c8200 0x7fa918070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:22.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91d000640 1 -- 172.21.15.113:0/2454079039 >> v2:172.21.15.136:3300/0 conn(0x7fa9180c8200 msgr2=0x7fa918070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:22.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91d000640 1 --2- 172.21.15.113:0/2454079039 >> v2:172.21.15.136:3300/0 conn(0x7fa9180c8200 0x7fa918070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:22.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91d000640 1 -- 172.21.15.113:0/2454079039 >> v1:172.21.15.136:6789/0 conn(0x7fa9181527f0 legacy=0x7fa918072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:22.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.802+0000 7fa91d000640 1 -- 172.21.15.113:0/2454079039 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9181594d0 con 0x7fa918152bd0 2024-03-04T08:18:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91d000640 1 --2- 172.21.15.113:0/2454079039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa918152bd0 0x7fa9181587d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa9180cd230 tx=0x7fa91001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d087f5a5da1d7bf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa917400640 1 -- 172.21.15.113:0/2454079039 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100a57a0 con 0x7fa918152bd0 2024-03-04T08:18:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa917400640 1 -- 172.21.15.113:0/2454079039 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9100a5940 con 0x7fa918152bd0 2024-03-04T08:18:22.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa917400640 1 -- 172.21.15.113:0/2454079039 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100a5d10 con 0x7fa918152bd0 2024-03-04T08:18:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91f000640 1 -- 172.21.15.113:0/2454079039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa918152bd0 msgr2=0x7fa9181587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91f000640 1 --2- 172.21.15.113:0/2454079039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa918152bd0 0x7fa9181587d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa9180cd230 tx=0x7fa91001c9f0 comp rx=0 tx=0).stop 2024-03-04T08:18:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91f000640 1 -- 172.21.15.113:0/2454079039 shutdown_connections 2024-03-04T08:18:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91f000640 1 --2- 172.21.15.113:0/2454079039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa918152bd0 0x7fa9181587d0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:22.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91f000640 1 --2- 172.21.15.113:0/2454079039 >> v2:172.21.15.136:3300/0 conn(0x7fa9180c8200 0x7fa918070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91f000640 1 -- 172.21.15.113:0/2454079039 >> 172.21.15.113:0/2454079039 conn(0x7fa91806e820 msgr2=0x7fa91806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91f000640 1 -- 172.21.15.113:0/2454079039 shutdown_connections 2024-03-04T08:18:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91f000640 1 -- 172.21.15.113:0/2454079039 wait complete. 2024-03-04T08:18:22.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91f000640 1 Processor -- start 2024-03-04T08:18:22.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.806+0000 7fa91f000640 1 -- start start 2024-03-04T08:18:22.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91f000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9180c8200 0x7fa918142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:22.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91f000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9181527f0 0x7fa9180b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:22.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91f000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa918152bd0 0x7fa9181f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:22.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91f000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9181654f0 con 0x7fa9180c8200 2024-03-04T08:18:22.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91f000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa918165370 con 0x7fa918152bd0 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91f000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa918165670 con 0x7fa9181527f0 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa917e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9180c8200 0x7fa918142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa917e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9180c8200 0x7fa918142150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38302/0 (socket says 172.21.15.113:38302) 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa917e00640 1 -- 172.21.15.113:0/3626791826 learned_addr learned my addr 172.21.15.113:0/3626791826 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa90f400640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9181527f0 0x7fa9180b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91d000640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa918152bd0 0x7fa9181f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa917e00640 1 -- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9181527f0 msgr2=0x7fa9180b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa917e00640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9181527f0 0x7fa9180b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa917e00640 1 -- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa918152bd0 msgr2=0x7fa9181f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa917e00640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa918152bd0 0x7fa9181f2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa917e00640 1 -- 172.21.15.113:0/3626791826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa918142840 con 0x7fa9180c8200 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91d000640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa918152bd0 0x7fa9181f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:18:22.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa917e00640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9180c8200 0x7fa918142150 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c420 tx=0x7fa90800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa914c00640 1 -- 172.21.15.113:0/3626791826 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9080052d0 con 0x7fa9180c8200 2024-03-04T08:18:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa914c00640 1 -- 172.21.15.113:0/3626791826 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908005470 con 0x7fa9180c8200 2024-03-04T08:18:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91f000640 1 -- 172.21.15.113:0/3626791826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa918141c50 con 0x7fa9180c8200 2024-03-04T08:18:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa914c00640 1 -- 172.21.15.113:0/3626791826 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908007260 con 0x7fa9180c8200 2024-03-04T08:18:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91f000640 1 -- 172.21.15.113:0/3626791826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9181f1980 con 0x7fa9180c8200 2024-03-04T08:18:22.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa91f000640 1 -- 172.21.15.113:0/3626791826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9180c9a50 con 0x7fa9180c8200 2024-03-04T08:18:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.810+0000 7fa914c00640 1 -- 172.21.15.113:0/3626791826 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa908007400 con 0x7fa9180c8200 2024-03-04T08:18:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.814+0000 7fa914c00640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8f4066f90 0x7fa8f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.814+0000 7fa914c00640 1 -- 172.21.15.113:0/3626791826 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7fa908084e70 con 0x7fa9180c8200 2024-03-04T08:18:22.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.814+0000 7fa90f400640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8f4066f90 0x7fa8f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:22.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.814+0000 7fa90f400640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8f4066f90 0x7fa8f4069450 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa918162c00 tx=0x7fa900008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:22.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:22.818+0000 7fa914c00640 1 -- 172.21.15.113:0/3626791826 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa9080554c0 con 0x7fa9180c8200 2024-03-04T08:18:23.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.074+0000 7fa91f000640 1 -- 172.21.15.113:0/3626791826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9180c79d0 con 0x7fa9180c8200 2024-03-04T08:18:23.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.074+0000 7fa914c00640 1 -- 172.21.15.113:0/3626791826 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) v1 ==== 95+0+5829 (secure 0 0 0) 0x7fa908054dc0 con 0x7fa9180c8200 2024-03-04T08:18:23.074 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:23.074 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":161,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7692,7839,8032]},"id":1}]} 2024-03-04T08:18:23.074 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 161 2024-03-04T08:18:23.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 -- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8f4066f90 msgr2=0x7fa8f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:23.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8f4066f90 0x7fa8f4069450 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa918162c00 tx=0x7fa900008040 comp rx=0 tx=0).stop 2024-03-04T08:18:23.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 -- 172.21.15.113:0/3626791826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9180c8200 msgr2=0x7fa918142150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:23.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9180c8200 0x7fa918142150 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c420 tx=0x7fa90800c8e0 comp rx=0 tx=0).stop 2024-03-04T08:18:23.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 -- 172.21.15.113:0/3626791826 shutdown_connections 2024-03-04T08:18:23.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa918152bd0 0x7fa9181f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:23.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa9181527f0 0x7fa9180b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:23.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa8f4066f90 0x7fa8f4069450 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:23.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 --2- 172.21.15.113:0/3626791826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9180c8200 0x7fa918142150 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:23.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 -- 172.21.15.113:0/3626791826 >> 172.21.15.113:0/3626791826 conn(0x7fa91806e820 msgr2=0x7fa9181573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:23.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 -- 172.21.15.113:0/3626791826 shutdown_connections 2024-03-04T08:18:23.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:23.078+0000 7fa91f000640 1 -- 172.21.15.113:0/3626791826 wait complete. 2024-03-04T08:18:23.094 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.a is in active, standby or standby-replay 2024-03-04T08:18:25.095 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:18:25.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e53e00640 1 Processor -- start 2024-03-04T08:18:25.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e53e00640 1 -- start start 2024-03-04T08:18:25.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e53e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2e4c1527f0 0x7f2e4c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:25.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e53e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2e4c152bd0 0x7f2e4c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:25.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e53e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c1591f0 con 0x7f2e4c0c8200 2024-03-04T08:18:25.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e53e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c071df0 con 0x7f2e4c1527f0 2024-03-04T08:18:25.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e53e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c1597b0 con 0x7f2e4c152bd0 2024-03-04T08:18:25.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e50a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2e4c0c8200 0x7f2e4c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41680/0 (socket says 172.21.15.113:41680) 2024-03-04T08:18:25.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e50a00640 1 -- 172.21.15.113:0/2295263675 learned_addr learned my addr 172.21.15.113:0/2295263675 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:18:25.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e51400640 1 --2- 172.21.15.113:0/2295263675 >> v2:172.21.15.113:3300/0 conn(0x7f2e4c1527f0 0x7f2e4c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:25.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e51e00640 1 --2- 172.21.15.113:0/2295263675 >> v2:172.21.15.136:3300/0 conn(0x7f2e4c152bd0 0x7f2e4c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:25.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.234+0000 7f2e4be00640 1 -- 172.21.15.113:0/2295263675 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435505558 0 0) 0x7f2e4c1591f0 con 0x7f2e4c0c8200 2024-03-04T08:18:25.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.238+0000 7f2e4be00640 1 -- 172.21.15.113:0/2295263675 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e34003530 con 0x7f2e4c0c8200 2024-03-04T08:18:25.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.238+0000 7f2e51400640 1 -- 172.21.15.113:0/2295263675 >> v2:172.21.15.136:3300/0 conn(0x7f2e4c152bd0 msgr2=0x7f2e4c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:18:25.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.238+0000 7f2e51400640 1 --2- 172.21.15.113:0/2295263675 >> v2:172.21.15.136:3300/0 conn(0x7f2e4c152bd0 0x7f2e4c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:25.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.238+0000 7f2e51400640 1 -- 172.21.15.113:0/2295263675 >> v1:172.21.15.113:6789/0 conn(0x7f2e4c0c8200 legacy=0x7f2e4c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:25.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.238+0000 7f2e51400640 1 -- 172.21.15.113:0/2295263675 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c1594d0 con 0x7f2e4c1527f0 2024-03-04T08:18:25.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.238+0000 7f2e51400640 1 --2- 172.21.15.113:0/2295263675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c1527f0 0x7f2e4c070d40 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0712b0 tx=0x7f2e4001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8640a1efe34f78 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:25.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.238+0000 7f2e4be00640 1 -- 172.21.15.113:0/2295263675 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e400b1070 con 0x7f2e4c1527f0 2024-03-04T08:18:25.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.238+0000 7f2e4be00640 1 -- 172.21.15.113:0/2295263675 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e400ac5f0 con 0x7f2e4c1527f0 2024-03-04T08:18:25.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.238+0000 7f2e4be00640 1 -- 172.21.15.113:0/2295263675 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e400ac9c0 con 0x7f2e4c1527f0 2024-03-04T08:18:25.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 -- 172.21.15.113:0/2295263675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c1527f0 msgr2=0x7f2e4c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:25.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 --2- 172.21.15.113:0/2295263675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c1527f0 0x7f2e4c070d40 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0712b0 tx=0x7f2e4001c480 comp rx=0 tx=0).stop 2024-03-04T08:18:25.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 -- 172.21.15.113:0/2295263675 shutdown_connections 2024-03-04T08:18:25.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 --2- 172.21.15.113:0/2295263675 >> v2:172.21.15.136:3300/0 conn(0x7f2e4c152bd0 0x7f2e4c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:25.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 --2- 172.21.15.113:0/2295263675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c1527f0 0x7f2e4c070d40 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:25.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 -- 172.21.15.113:0/2295263675 >> 172.21.15.113:0/2295263675 conn(0x7f2e4c06e820 msgr2=0x7f2e4c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:25.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 -- 172.21.15.113:0/2295263675 shutdown_connections 2024-03-04T08:18:25.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 -- 172.21.15.113:0/2295263675 wait complete. 2024-03-04T08:18:25.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 Processor -- start 2024-03-04T08:18:25.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 -- start start 2024-03-04T08:18:25.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e4c0c8200 0x7f2e4c14a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:25.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e4c1527f0 0x7f2e4c157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:25.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c152bd0 0x7f2e4c14f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:25.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c1654f0 con 0x7f2e4c152bd0 2024-03-04T08:18:25.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c165370 con 0x7f2e4c1527f0 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e53e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c165670 con 0x7f2e4c0c8200 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e51e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c152bd0 0x7f2e4c14f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e51e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c152bd0 0x7f2e4c14f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38316/0 (socket says 172.21.15.113:38316) 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e51e00640 1 -- 172.21.15.113:0/294847570 learned_addr learned my addr 172.21.15.113:0/294847570 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e51400640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e4c0c8200 0x7f2e4c14a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e50a00640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e4c1527f0 0x7f2e4c157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e51e00640 1 -- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e4c0c8200 msgr2=0x7f2e4c14a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e51e00640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e4c0c8200 0x7f2e4c14a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e51e00640 1 -- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e4c1527f0 msgr2=0x7f2e4c157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e51e00640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e4c1527f0 0x7f2e4c157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:25.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.242+0000 7f2e51e00640 1 -- 172.21.15.113:0/294847570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c1f45c0 con 0x7f2e4c152bd0 2024-03-04T08:18:25.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e51e00640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c152bd0 0x7f2e4c14f710 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400efc0 tx=0x7f2e4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:25.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e49600640 1 -- 172.21.15.113:0/294847570 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44018020 con 0x7f2e4c152bd0 2024-03-04T08:18:25.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e49600640 1 -- 172.21.15.113:0/294847570 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e44014480 con 0x7f2e4c152bd0 2024-03-04T08:18:25.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e49600640 1 -- 172.21.15.113:0/294847570 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44014870 con 0x7f2e4c152bd0 2024-03-04T08:18:25.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e53e00640 1 -- 172.21.15.113:0/294847570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c156d70 con 0x7f2e4c152bd0 2024-03-04T08:18:25.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e53e00640 1 -- 172.21.15.113:0/294847570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c157330 con 0x7f2e4c152bd0 2024-03-04T08:18:25.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e53e00640 1 -- 172.21.15.113:0/294847570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c075c00 con 0x7f2e4c152bd0 2024-03-04T08:18:25.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e49600640 1 -- 172.21.15.113:0/294847570 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e4401b020 con 0x7f2e4c152bd0 2024-03-04T08:18:25.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e49600640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e2c066f90 0x7f2e2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:18:25.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e49600640 1 -- 172.21.15.113:0/294847570 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7889+0+0 (secure 0 0 0) 0x7f2e44086650 con 0x7f2e4c152bd0 2024-03-04T08:18:25.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.246+0000 7f2e51400640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e2c066f90 0x7f2e2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:18:25.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.250+0000 7f2e51400640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e2c066f90 0x7f2e2c069450 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2e400ae000 tx=0x7f2e40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:18:25.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.254+0000 7f2e49600640 1 -- 172.21.15.113:0/294847570 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2e44022030 con 0x7f2e4c152bd0 2024-03-04T08:18:25.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.450+0000 7f2e53e00640 1 -- 172.21.15.113:0/294847570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e4c14f4f0 con 0x7f2e4c152bd0 2024-03-04T08:18:25.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e49600640 1 -- 172.21.15.113:0/294847570 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f2e44010040 con 0x7f2e4c152bd0 2024-03-04T08:18:25.451 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:18:25.451 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839,7692]},"id":1}]} 2024-03-04T08:18:25.451 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 162 2024-03-04T08:18:25.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 -- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e2c066f90 msgr2=0x7f2e2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:25.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e2c066f90 0x7f2e2c069450 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2e400ae000 tx=0x7f2e40008040 comp rx=0 tx=0).stop 2024-03-04T08:18:25.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 -- 172.21.15.113:0/294847570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c152bd0 msgr2=0x7f2e4c14f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:18:25.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c152bd0 0x7f2e4c14f710 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2e4400efc0 tx=0x7f2e4400c490 comp rx=0 tx=0).stop 2024-03-04T08:18:25.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 -- 172.21.15.113:0/294847570 shutdown_connections 2024-03-04T08:18:25.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e2c066f90 0x7f2e2c069450 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:25.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e4c152bd0 0x7f2e4c14f710 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:25.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e4c1527f0 0x7f2e4c157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:25.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 --2- 172.21.15.113:0/294847570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2e4c0c8200 0x7f2e4c14a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:18:25.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 -- 172.21.15.113:0/294847570 >> 172.21.15.113:0/294847570 conn(0x7f2e4c06e820 msgr2=0x7f2e4c06ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:18:25.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 -- 172.21.15.113:0/294847570 shutdown_connections 2024-03-04T08:18:25.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:18:25.454+0000 7f2e53e00640 1 -- 172.21.15.113:0/294847570 wait complete. 2024-03-04T08:18:25.472 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a reported in up:standby state 2024-03-04T08:18:25.472 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 83 secs before thrashing 2024-03-04T08:18:26.382 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:26.382 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.79 2024-03-04T08:18:26.382 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 79 -eq 48 ] 2024-03-04T08:18:26.383 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.79 2024-03-04T08:18:26.383 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.79 2024-03-04T08:18:26.383 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.79 2024-03-04T08:18:26.383 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:29.594 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:29.594 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.80 2024-03-04T08:18:29.594 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 80 -eq 48 ] 2024-03-04T08:18:29.594 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.80 2024-03-04T08:18:29.594 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.80 2024-03-04T08:18:29.594 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.80 2024-03-04T08:18:29.594 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:32.484 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:32.485 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.81 2024-03-04T08:18:32.485 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 81 -eq 48 ] 2024-03-04T08:18:32.485 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.81 2024-03-04T08:18:32.485 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.81 2024-03-04T08:18:32.485 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.81 2024-03-04T08:18:32.485 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:38.402 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:18:38.405 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:18:38.442 INFO:tasks.ceph.mds.c.smithi113.stderr:2024-03-04T08:18:38.442+0000 7fd116c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 50875) UID: 0 2024-03-04T08:18:38.443 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-03-04T08:18:38.442+0000 7fce6ae00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 50875) UID: 0 2024-03-04T08:18:38.443 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T08:18:38.442+0000 7f9b5e600640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 50875) UID: 0 2024-03-04T08:18:39.688 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:39.688 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 82 -eq 48 ] 2024-03-04T08:18:39.689 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.82 2024-03-04T08:18:39.689 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.82 2024-03-04T08:18:39.689 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.82 2024-03-04T08:18:39.689 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.82 2024-03-04T08:18:39.689 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:45.233 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:45.233 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 83 -eq 48 ] 2024-03-04T08:18:45.233 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.83 2024-03-04T08:18:45.233 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.83 2024-03-04T08:18:45.234 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.83 2024-03-04T08:18:45.234 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.83 2024-03-04T08:18:45.234 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:49.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:49.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 84 -eq 48 ] 2024-03-04T08:18:49.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.84 2024-03-04T08:18:49.477 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.84 2024-03-04T08:18:49.477 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.84 2024-03-04T08:18:49.478 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.84 2024-03-04T08:18:49.478 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:53.044 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:53.045 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 85 -eq 48 ] 2024-03-04T08:18:53.045 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.85 2024-03-04T08:18:53.045 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.85 2024-03-04T08:18:53.045 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.85 2024-03-04T08:18:53.046 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.85 2024-03-04T08:18:53.046 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:18:58.846 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:18:58.846 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 86 -eq 48 ] 2024-03-04T08:18:58.846 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.86 2024-03-04T08:18:58.846 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.86 2024-03-04T08:18:58.847 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.86 2024-03-04T08:18:58.847 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.86 2024-03-04T08:18:58.847 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:19:03.703 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:19:03.704 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 87 -eq 48 ] 2024-03-04T08:19:03.704 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.87 2024-03-04T08:19:03.704 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.87 2024-03-04T08:19:03.704 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.87 2024-03-04T08:19:03.705 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.87 2024-03-04T08:19:03.705 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:19:06.534 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:19:06.534 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.88 2024-03-04T08:19:06.535 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 88 -eq 48 ] 2024-03-04T08:19:06.535 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.88 2024-03-04T08:19:06.535 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.88 2024-03-04T08:19:06.536 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.88 2024-03-04T08:19:06.536 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:19:11.881 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:19:11.882 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.89 2024-03-04T08:19:11.882 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 89 -eq 48 ] 2024-03-04T08:19:11.882 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.89 2024-03-04T08:19:11.882 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.89 2024-03-04T08:19:11.882 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.89 2024-03-04T08:19:11.883 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:19:17.222 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:19:17.222 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 90 -eq 48 ] 2024-03-04T08:19:17.222 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.90 2024-03-04T08:19:17.222 INFO:tasks.workunit.client.0.smithi113.stdout:checking v0.90 2024-03-04T08:19:17.223 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo checking v0.90 2024-03-04T08:19:17.223 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd .snap/v0.90 2024-03-04T08:19:17.223 INFO:tasks.workunit.client.0.smithi113.stderr:+ git diff --exit-code 2024-03-04T08:19:23.075 INFO:tasks.workunit.client.0.smithi113.stderr:+ cd ../.. 2024-03-04T08:19:23.075 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 1 -eq 48 ] 2024-03-04T08:19:23.075 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.1 2024-03-04T08:19:23.076 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.1 2024-03-04T08:19:23.089 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 2 -eq 48 ] 2024-03-04T08:19:23.089 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.2 2024-03-04T08:19:23.089 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.2 2024-03-04T08:19:23.100 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 3 -eq 48 ] 2024-03-04T08:19:23.100 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.3 2024-03-04T08:19:23.100 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.3 2024-03-04T08:19:23.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 4 -eq 48 ] 2024-03-04T08:19:23.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.4 2024-03-04T08:19:23.111 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.4 2024-03-04T08:19:23.123 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 5 -eq 48 ] 2024-03-04T08:19:23.123 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.5 2024-03-04T08:19:23.123 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.5 2024-03-04T08:19:23.134 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 6 -eq 48 ] 2024-03-04T08:19:23.135 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.6 2024-03-04T08:19:23.135 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.6 2024-03-04T08:19:23.145 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 7 -eq 48 ] 2024-03-04T08:19:23.146 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.7 2024-03-04T08:19:23.146 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.7 2024-03-04T08:19:23.158 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 8 -eq 48 ] 2024-03-04T08:19:23.158 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.8 2024-03-04T08:19:23.158 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.8 2024-03-04T08:19:23.170 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 9 -eq 48 ] 2024-03-04T08:19:23.170 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.9 2024-03-04T08:19:23.170 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.9 2024-03-04T08:19:23.183 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 10 -eq 48 ] 2024-03-04T08:19:23.183 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.10 2024-03-04T08:19:23.183 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.10 2024-03-04T08:19:23.194 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 11 -eq 48 ] 2024-03-04T08:19:23.194 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.11 2024-03-04T08:19:23.194 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.11 2024-03-04T08:19:23.497 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 12 -eq 48 ] 2024-03-04T08:19:23.497 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.12 2024-03-04T08:19:23.497 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.12 2024-03-04T08:19:23.766 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 13 -eq 48 ] 2024-03-04T08:19:23.766 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.13 2024-03-04T08:19:23.766 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.13 2024-03-04T08:19:23.777 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 14 -eq 48 ] 2024-03-04T08:19:23.778 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.14 2024-03-04T08:19:23.778 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.14 2024-03-04T08:19:23.791 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 15 -eq 48 ] 2024-03-04T08:19:23.791 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.15 2024-03-04T08:19:23.791 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.15 2024-03-04T08:19:23.801 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 16 -eq 48 ] 2024-03-04T08:19:23.801 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.16 2024-03-04T08:19:23.801 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.16 2024-03-04T08:19:23.813 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 17 -eq 48 ] 2024-03-04T08:19:23.813 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.17 2024-03-04T08:19:23.814 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.17 2024-03-04T08:19:23.824 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 18 -eq 48 ] 2024-03-04T08:19:23.824 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.18 2024-03-04T08:19:23.824 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.18 2024-03-04T08:19:23.834 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 19 -eq 48 ] 2024-03-04T08:19:23.834 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.19 2024-03-04T08:19:23.834 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.19 2024-03-04T08:19:23.844 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 20 -eq 48 ] 2024-03-04T08:19:23.845 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.20 2024-03-04T08:19:23.845 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.20 2024-03-04T08:19:23.856 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 21 -eq 48 ] 2024-03-04T08:19:23.857 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.21 2024-03-04T08:19:23.857 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.21 2024-03-04T08:19:23.867 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 22 -eq 48 ] 2024-03-04T08:19:23.868 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.22 2024-03-04T08:19:23.868 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.22 2024-03-04T08:19:23.878 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 23 -eq 48 ] 2024-03-04T08:19:23.879 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.23 2024-03-04T08:19:23.879 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.23 2024-03-04T08:19:23.889 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 24 -eq 48 ] 2024-03-04T08:19:23.889 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.24 2024-03-04T08:19:23.889 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.24 2024-03-04T08:19:23.899 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 25 -eq 48 ] 2024-03-04T08:19:23.899 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.25 2024-03-04T08:19:23.899 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.25 2024-03-04T08:19:23.909 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 26 -eq 48 ] 2024-03-04T08:19:23.909 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.26 2024-03-04T08:19:23.909 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.26 2024-03-04T08:19:23.923 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 27 -eq 48 ] 2024-03-04T08:19:23.923 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.27 2024-03-04T08:19:23.923 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.27 2024-03-04T08:19:23.935 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 28 -eq 48 ] 2024-03-04T08:19:23.935 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.28 2024-03-04T08:19:23.935 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.28 2024-03-04T08:19:23.946 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 29 -eq 48 ] 2024-03-04T08:19:23.946 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.29 2024-03-04T08:19:23.946 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.29 2024-03-04T08:19:23.957 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 30 -eq 48 ] 2024-03-04T08:19:23.957 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.30 2024-03-04T08:19:23.958 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.30 2024-03-04T08:19:23.969 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 31 -eq 48 ] 2024-03-04T08:19:23.969 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.31 2024-03-04T08:19:23.969 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.31 2024-03-04T08:19:24.648 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 32 -eq 48 ] 2024-03-04T08:19:24.648 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.32 2024-03-04T08:19:24.648 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.32 2024-03-04T08:19:24.659 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 33 -eq 48 ] 2024-03-04T08:19:24.659 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.33 2024-03-04T08:19:24.659 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.33 2024-03-04T08:19:24.670 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 34 -eq 48 ] 2024-03-04T08:19:24.670 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.34 2024-03-04T08:19:24.670 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.34 2024-03-04T08:19:24.680 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 35 -eq 48 ] 2024-03-04T08:19:24.680 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.35 2024-03-04T08:19:24.680 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.35 2024-03-04T08:19:24.691 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 36 -eq 48 ] 2024-03-04T08:19:24.691 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.36 2024-03-04T08:19:24.691 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.36 2024-03-04T08:19:24.702 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 37 -eq 48 ] 2024-03-04T08:19:24.702 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.37 2024-03-04T08:19:24.702 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.37 2024-03-04T08:19:24.713 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 38 -eq 48 ] 2024-03-04T08:19:24.713 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.38 2024-03-04T08:19:24.714 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.38 2024-03-04T08:19:24.727 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 39 -eq 48 ] 2024-03-04T08:19:24.727 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.39 2024-03-04T08:19:24.727 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.39 2024-03-04T08:19:24.737 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 40 -eq 48 ] 2024-03-04T08:19:24.737 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.40 2024-03-04T08:19:24.737 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.40 2024-03-04T08:19:24.750 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 41 -eq 48 ] 2024-03-04T08:19:24.750 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.41 2024-03-04T08:19:24.750 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.41 2024-03-04T08:19:24.762 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 42 -eq 48 ] 2024-03-04T08:19:24.762 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.42 2024-03-04T08:19:24.762 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.42 2024-03-04T08:19:24.775 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 43 -eq 48 ] 2024-03-04T08:19:24.775 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.43 2024-03-04T08:19:24.776 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.43 2024-03-04T08:19:24.786 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 44 -eq 48 ] 2024-03-04T08:19:24.786 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.44 2024-03-04T08:19:24.786 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.44 2024-03-04T08:19:24.796 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 45 -eq 48 ] 2024-03-04T08:19:24.796 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.45 2024-03-04T08:19:24.797 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.45 2024-03-04T08:19:24.805 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 46 -eq 48 ] 2024-03-04T08:19:24.805 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.46 2024-03-04T08:19:24.805 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.46 2024-03-04T08:19:24.816 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 47 -eq 48 ] 2024-03-04T08:19:24.816 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.47 2024-03-04T08:19:24.816 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.47 2024-03-04T08:19:25.478 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 48 -eq 48 ] 2024-03-04T08:19:25.478 INFO:tasks.workunit.client.0.smithi113.stderr:+ continue 2024-03-04T08:19:25.478 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 49 -eq 48 ] 2024-03-04T08:19:25.478 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.49 2024-03-04T08:19:25.478 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.49 2024-03-04T08:19:25.779 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 50 -eq 48 ] 2024-03-04T08:19:25.779 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.50 2024-03-04T08:19:25.779 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.50 2024-03-04T08:19:25.788 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 51 -eq 48 ] 2024-03-04T08:19:25.788 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.51 2024-03-04T08:19:25.788 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.51 2024-03-04T08:19:25.796 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 52 -eq 48 ] 2024-03-04T08:19:25.797 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.52 2024-03-04T08:19:25.797 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.52 2024-03-04T08:19:25.806 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 53 -eq 48 ] 2024-03-04T08:19:25.806 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.53 2024-03-04T08:19:25.806 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.53 2024-03-04T08:19:25.816 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 54 -eq 48 ] 2024-03-04T08:19:25.816 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.54 2024-03-04T08:19:25.816 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.54 2024-03-04T08:19:25.825 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 55 -eq 48 ] 2024-03-04T08:19:25.825 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.55 2024-03-04T08:19:25.825 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.55 2024-03-04T08:19:26.441 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 56 -eq 48 ] 2024-03-04T08:19:26.441 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.56 2024-03-04T08:19:26.441 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.56 2024-03-04T08:19:26.696 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 57 -eq 48 ] 2024-03-04T08:19:26.696 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.57 2024-03-04T08:19:26.696 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.57 2024-03-04T08:19:26.707 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 58 -eq 48 ] 2024-03-04T08:19:26.708 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.58 2024-03-04T08:19:26.708 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.58 2024-03-04T08:19:26.718 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 59 -eq 48 ] 2024-03-04T08:19:26.718 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.59 2024-03-04T08:19:26.718 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.59 2024-03-04T08:19:26.728 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 60 -eq 48 ] 2024-03-04T08:19:26.729 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.60 2024-03-04T08:19:26.729 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.60 2024-03-04T08:19:26.740 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 61 -eq 48 ] 2024-03-04T08:19:26.740 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.61 2024-03-04T08:19:26.740 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.61 2024-03-04T08:19:26.751 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 62 -eq 48 ] 2024-03-04T08:19:26.751 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.62 2024-03-04T08:19:26.751 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.62 2024-03-04T08:19:26.761 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 63 -eq 48 ] 2024-03-04T08:19:26.761 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.63 2024-03-04T08:19:26.761 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.63 2024-03-04T08:19:26.772 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 64 -eq 48 ] 2024-03-04T08:19:26.772 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.64 2024-03-04T08:19:26.772 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.64 2024-03-04T08:19:26.784 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 65 -eq 48 ] 2024-03-04T08:19:26.784 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.65 2024-03-04T08:19:26.784 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.65 2024-03-04T08:19:26.795 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 66 -eq 48 ] 2024-03-04T08:19:26.796 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.66 2024-03-04T08:19:26.796 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.66 2024-03-04T08:19:26.805 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 67 -eq 48 ] 2024-03-04T08:19:26.805 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.67 2024-03-04T08:19:26.805 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.67 2024-03-04T08:19:26.816 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 68 -eq 48 ] 2024-03-04T08:19:26.816 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.68 2024-03-04T08:19:26.816 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.68 2024-03-04T08:19:26.826 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 69 -eq 48 ] 2024-03-04T08:19:26.826 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.69 2024-03-04T08:19:26.826 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.69 2024-03-04T08:19:26.837 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 70 -eq 48 ] 2024-03-04T08:19:26.837 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.70 2024-03-04T08:19:26.838 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.70 2024-03-04T08:19:26.848 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 71 -eq 48 ] 2024-03-04T08:19:26.848 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.71 2024-03-04T08:19:26.848 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.71 2024-03-04T08:19:26.860 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 72 -eq 48 ] 2024-03-04T08:19:26.860 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.72 2024-03-04T08:19:26.860 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.72 2024-03-04T08:19:26.872 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 73 -eq 48 ] 2024-03-04T08:19:26.872 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.73 2024-03-04T08:19:26.872 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.73 2024-03-04T08:19:26.883 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 74 -eq 48 ] 2024-03-04T08:19:26.883 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.74 2024-03-04T08:19:26.883 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.74 2024-03-04T08:19:26.894 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 75 -eq 48 ] 2024-03-04T08:19:26.894 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.75 2024-03-04T08:19:26.894 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.75 2024-03-04T08:19:26.905 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 76 -eq 48 ] 2024-03-04T08:19:26.905 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.76 2024-03-04T08:19:26.905 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.76 2024-03-04T08:19:26.924 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 77 -eq 48 ] 2024-03-04T08:19:26.924 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.77 2024-03-04T08:19:26.925 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.77 2024-03-04T08:19:26.945 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 78 -eq 48 ] 2024-03-04T08:19:26.945 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.78 2024-03-04T08:19:26.945 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.78 2024-03-04T08:19:26.971 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 79 -eq 48 ] 2024-03-04T08:19:26.971 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.79 2024-03-04T08:19:26.972 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.79 2024-03-04T08:19:26.994 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 80 -eq 48 ] 2024-03-04T08:19:26.994 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.80 2024-03-04T08:19:26.994 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.80 2024-03-04T08:19:27.021 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 81 -eq 48 ] 2024-03-04T08:19:27.021 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.81 2024-03-04T08:19:27.022 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.81 2024-03-04T08:19:27.055 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 82 -eq 48 ] 2024-03-04T08:19:27.056 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.82 2024-03-04T08:19:27.056 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.82 2024-03-04T08:19:27.083 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 83 -eq 48 ] 2024-03-04T08:19:27.083 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.83 2024-03-04T08:19:27.084 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.83 2024-03-04T08:19:27.312 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 84 -eq 48 ] 2024-03-04T08:19:27.312 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.84 2024-03-04T08:19:27.312 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.84 2024-03-04T08:19:27.346 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 85 -eq 48 ] 2024-03-04T08:19:27.347 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.85 2024-03-04T08:19:27.347 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.85 2024-03-04T08:19:27.375 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 86 -eq 48 ] 2024-03-04T08:19:27.376 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.86 2024-03-04T08:19:27.378 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.86 2024-03-04T08:19:27.400 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 87 -eq 48 ] 2024-03-04T08:19:27.400 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.87 2024-03-04T08:19:27.400 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.87 2024-03-04T08:19:27.425 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 88 -eq 48 ] 2024-03-04T08:19:27.425 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.88 2024-03-04T08:19:27.425 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.88 2024-03-04T08:19:27.449 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 89 -eq 48 ] 2024-03-04T08:19:27.449 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.89 2024-03-04T08:19:27.449 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.89 2024-03-04T08:19:27.474 INFO:tasks.workunit.client.0.smithi113.stderr:+ [ 90 -eq 48 ] 2024-03-04T08:19:27.475 INFO:tasks.workunit.client.0.smithi113.stderr:+ ver=v0.90 2024-03-04T08:19:27.475 INFO:tasks.workunit.client.0.smithi113.stderr:+ rmdir .snap/v0.90 2024-03-04T08:19:27.495 INFO:tasks.workunit.client.0.smithi113.stderr:+ echo OK 2024-03-04T08:19:27.496 INFO:tasks.workunit.client.0.smithi113.stdout:OK 2024-03-04T08:19:27.496 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T08:19:27.496 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T08:19:36.576 INFO:tasks.ceph.mgr.x.smithi136.stderr:2024-03-04T08:19:36.577+0000 7fc3178b6640 -1 calc_pg_upmaps abort due to max <= 0 2024-03-04T08:19:48.473 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:19:48.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.717+0000 7feee3400640 1 Processor -- start 2024-03-04T08:19:48.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.717+0000 7feee3400640 1 -- start start 2024-03-04T08:19:48.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.721+0000 7feee3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 0x7feedc0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:48.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.721+0000 7feee0a00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7feedc154140 0x7feedc0c8370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:32952/0 (socket says 172.21.15.113:32952) 2024-03-04T08:19:48.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.721+0000 7feee0a00640 1 -- 172.21.15.113:0/1599630041 learned_addr learned my addr 172.21.15.113:0/1599630041 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:19:48.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.721+0000 7feee3400640 1 --2- 172.21.15.113:0/1599630041 >> v2:172.21.15.113:3300/0 conn(0x7feedc0724a0 0x7feedc0c9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:48.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.721+0000 7feedbe00640 1 --2- 172.21.15.113:0/1599630041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 0x7feedc0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:48.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.721+0000 7feee3400640 1 -- 172.21.15.113:0/1599630041 --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc0c1230 con 0x7feedc154140 2024-03-04T08:19:48.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.721+0000 7feee3400640 1 -- 172.21.15.113:0/1599630041 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc0c14d0 con 0x7feedc0724a0 2024-03-04T08:19:48.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.721+0000 7feee3400640 1 -- 172.21.15.113:0/1599630041 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0c17b0 con 0x7feedc072060 2024-03-04T08:19:48.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.721+0000 7feee1400640 1 --2- 172.21.15.113:0/1599630041 >> v2:172.21.15.113:3300/0 conn(0x7feedc0724a0 0x7feedc0c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.725+0000 7feedb400640 1 -- 172.21.15.113:0/1599630041 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 872261995 0 0) 0x7feedc0c1230 con 0x7feedc154140 2024-03-04T08:19:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.725+0000 7feedb400640 1 -- 172.21.15.113:0/1599630041 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec4003530 con 0x7feedc154140 2024-03-04T08:19:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.725+0000 7feee1400640 1 -- 172.21.15.113:0/1599630041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 msgr2=0x7feedc0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:19:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.725+0000 7feee1400640 1 --2- 172.21.15.113:0/1599630041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 0x7feedc0c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.725+0000 7feee1400640 1 -- 172.21.15.113:0/1599630041 >> v1:172.21.15.136:6789/0 conn(0x7feedc154140 legacy=0x7feedc0c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:48.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.725+0000 7feee1400640 1 -- 172.21.15.113:0/1599630041 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc0c1a90 con 0x7feedc0724a0 2024-03-04T08:19:48.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.733+0000 7feee1400640 1 --2- 172.21.15.113:0/1599630041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feedc0724a0 0x7feedc0c9140 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7feed4009e10 tx=0x7feed4027930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b056b93759dbb29d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:48.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.733+0000 7feedb400640 1 -- 172.21.15.113:0/1599630041 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40b2710 con 0x7feedc0724a0 2024-03-04T08:19:48.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.733+0000 7feedb400640 1 -- 172.21.15.113:0/1599630041 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed40b28b0 con 0x7feedc0724a0 2024-03-04T08:19:48.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.733+0000 7feedb400640 1 -- 172.21.15.113:0/1599630041 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40b2c80 con 0x7feedc0724a0 2024-03-04T08:19:48.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.737+0000 7feee3400640 1 -- 172.21.15.113:0/1599630041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feedc0724a0 msgr2=0x7feedc0c9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:48.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.737+0000 7feee3400640 1 --2- 172.21.15.113:0/1599630041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feedc0724a0 0x7feedc0c9140 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7feed4009e10 tx=0x7feed4027930 comp rx=0 tx=0).stop 2024-03-04T08:19:48.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.737+0000 7feee3400640 1 -- 172.21.15.113:0/1599630041 shutdown_connections 2024-03-04T08:19:48.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.737+0000 7feee3400640 1 --2- 172.21.15.113:0/1599630041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feedc0724a0 0x7feedc0c9140 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:48.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.737+0000 7feee3400640 1 --2- 172.21.15.113:0/1599630041 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 0x7feedc0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:48.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.737+0000 7feee3400640 1 -- 172.21.15.113:0/1599630041 >> 172.21.15.113:0/1599630041 conn(0x7feedc06e820 msgr2=0x7feedc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:48.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.745+0000 7feee3400640 1 -- 172.21.15.113:0/1599630041 shutdown_connections 2024-03-04T08:19:48.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.745+0000 7feee3400640 1 -- 172.21.15.113:0/1599630041 wait complete. 2024-03-04T08:19:48.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.745+0000 7feee3400640 1 Processor -- start 2024-03-04T08:19:48.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.745+0000 7feee3400640 1 -- start start 2024-03-04T08:19:48.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 0x7feedc1439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:48.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feedc0724a0 0x7feedc14d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:48.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feedc154140 0x7feedc1523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:48.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee3400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0729a0 con 0x7feedc0724a0 2024-03-04T08:19:48.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee3400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc154520 con 0x7feedc154140 2024-03-04T08:19:48.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee3400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc06f340 con 0x7feedc072060 2024-03-04T08:19:48.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee0a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 0x7feedc1439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:48.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee0a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 0x7feedc1439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:42426/0 (socket says 172.21.15.113:42426) 2024-03-04T08:19:48.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee0a00640 1 -- 172.21.15.113:0/4175104855 learned_addr learned my addr 172.21.15.113:0/4175104855 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:19:48.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee1400640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feedc154140 0x7feedc1523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feedbe00640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feedc0724a0 0x7feedc14d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee1400640 1 -- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 msgr2=0x7feedc1439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee1400640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 0x7feedc1439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee1400640 1 -- 172.21.15.113:0/4175104855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feedc0724a0 msgr2=0x7feedc14d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee1400640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feedc0724a0 0x7feedc14d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.749+0000 7feee1400640 1 -- 172.21.15.113:0/4175104855 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc143ef0 con 0x7feedc154140 2024-03-04T08:19:48.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.753+0000 7feee1400640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feedc154140 0x7feedc1523f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7feedc06c2a0 tx=0x7feed40246d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.753+0000 7feed8c00640 1 -- 172.21.15.113:0/4175104855 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40c4420 con 0x7feedc154140 2024-03-04T08:19:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.753+0000 7feed8c00640 1 -- 172.21.15.113:0/4175104855 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed40c45c0 con 0x7feedc154140 2024-03-04T08:19:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.753+0000 7feed8c00640 1 -- 172.21.15.113:0/4175104855 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40c4990 con 0x7feedc154140 2024-03-04T08:19:48.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.753+0000 7feee3400640 1 -- 172.21.15.113:0/4175104855 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc1445d0 con 0x7feedc154140 2024-03-04T08:19:48.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.753+0000 7feee3400640 1 -- 172.21.15.113:0/4175104855 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc153520 con 0x7feedc154140 2024-03-04T08:19:48.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.753+0000 7feee3400640 1 -- 172.21.15.113:0/4175104855 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc075c00 con 0x7feedc154140 2024-03-04T08:19:48.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.757+0000 7feed8c00640 1 -- 172.21.15.113:0/4175104855 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feed40cd020 con 0x7feedc154140 2024-03-04T08:19:48.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.757+0000 7feed8c00640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feebc066fb0 0x7feebc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:48.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.757+0000 7feed8c00640 1 -- 172.21.15.113:0/4175104855 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7945+0+0 (secure 0 0 0) 0x7feed412c960 con 0x7feedc154140 2024-03-04T08:19:48.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.757+0000 7feee0a00640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feebc066fb0 0x7feebc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:48.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.761+0000 7feee0a00640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feebc066fb0 0x7feebc069470 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7feed0000fc0 tx=0x7feed0005bd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:48.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:48.769+0000 7feed8c00640 1 -- 172.21.15.113:0/4175104855 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7feed40d4020 con 0x7feedc154140 2024-03-04T08:19:49.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.113+0000 7feee3400640 1 -- 172.21.15.113:0/4175104855 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feedc0c7f90 con 0x7feedc154140 2024-03-04T08:19:49.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.117+0000 7feed8c00640 1 -- 172.21.15.113:0/4175104855 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+6621 (secure 0 0 0) 0x7feed4021490 con 0x7feedc154140 2024-03-04T08:19:49.120 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:19:49.120 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839,7692]},"id":1}]} 2024-03-04T08:19:49.120 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 162 2024-03-04T08:19:49.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 -- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feebc066fb0 msgr2=0x7feebc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:49.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feebc066fb0 0x7feebc069470 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7feed0000fc0 tx=0x7feed0005bd0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 -- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feedc154140 msgr2=0x7feedc1523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:49.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feedc154140 0x7feedc1523f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7feedc06c2a0 tx=0x7feed40246d0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 -- 172.21.15.113:0/4175104855 shutdown_connections 2024-03-04T08:19:49.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feedc154140 0x7feedc1523f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feebc066fb0 0x7feebc069470 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feedc0724a0 0x7feedc14d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 --2- 172.21.15.113:0/4175104855 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feedc072060 0x7feedc1439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 -- 172.21.15.113:0/4175104855 >> 172.21.15.113:0/4175104855 conn(0x7feedc06e820 msgr2=0x7feedc0c73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:49.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 -- 172.21.15.113:0/4175104855 shutdown_connections 2024-03-04T08:19:49.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.125+0000 7feee3400640 1 -- 172.21.15.113:0/4175104855 wait complete. 2024-03-04T08:19:49.159 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.c (rank=2) 2024-03-04T08:19:49.159 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-04T08:19:49.159 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:19:49.191 INFO:tasks.ceph.mds.c:Stopped 2024-03-04T08:19:49.191 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:19:49.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.437+0000 7f4be4200640 1 Processor -- start 2024-03-04T08:19:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.437+0000 7f4be4200640 1 -- start start 2024-03-04T08:19:49.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be4200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc155100 0x7f4bdc0c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:49.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be4200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4bdc06b060 0x7f4bdc0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:49.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be4200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bdc15efd0 con 0x7f4bdc0cd600 2024-03-04T08:19:49.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be4200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bdc0c8890 con 0x7f4bdc06b060 2024-03-04T08:19:49.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be4200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc06b4a0 con 0x7f4bdc155100 2024-03-04T08:19:49.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4bdc06b060 0x7f4bdc0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:49.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc155100 0x7f4bdc0c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:49.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc155100 0x7f4bdc0c08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:42434/0 (socket says 172.21.15.113:42434) 2024-03-04T08:19:49.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be1800640 1 -- 172.21.15.113:0/2655730112 learned_addr learned my addr 172.21.15.113:0/2655730112 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:19:49.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be0e00640 1 -- 172.21.15.113:0/2655730112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc155100 msgr2=0x7f4bdc0c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:19:49.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be0e00640 1 --2- 172.21.15.113:0/2655730112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc155100 0x7f4bdc0c08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be0e00640 1 -- 172.21.15.113:0/2655730112 >> v1:172.21.15.136:6789/0 conn(0x7f4bdc0cd600 legacy=0x7f4bdc15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:49.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.441+0000 7f4be0e00640 1 -- 172.21.15.113:0/2655730112 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc0c7ef0 con 0x7f4bdc06b060 2024-03-04T08:19:49.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4be0e00640 1 --2- 172.21.15.113:0/2655730112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bdc06b060 0x7f4bdc0cd0c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80097c0 tx=0x7f4bc801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33a465178d40f790 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:49.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4bdbe00640 1 -- 172.21.15.113:0/2655730112 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80b0070 con 0x7f4bdc06b060 2024-03-04T08:19:49.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4bdbe00640 1 -- 172.21.15.113:0/2655730112 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc80ab600 con 0x7f4bdc06b060 2024-03-04T08:19:49.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4bdbe00640 1 -- 172.21.15.113:0/2655730112 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80ab9d0 con 0x7f4bdc06b060 2024-03-04T08:19:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4be4200640 1 -- 172.21.15.113:0/2655730112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bdc06b060 msgr2=0x7f4bdc0cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4be4200640 1 --2- 172.21.15.113:0/2655730112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bdc06b060 0x7f4bdc0cd0c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80097c0 tx=0x7f4bc801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4be4200640 1 -- 172.21.15.113:0/2655730112 shutdown_connections 2024-03-04T08:19:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4be4200640 1 --2- 172.21.15.113:0/2655730112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bdc06b060 0x7f4bdc0cd0c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4be4200640 1 --2- 172.21.15.113:0/2655730112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc155100 0x7f4bdc0c08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4be4200640 1 -- 172.21.15.113:0/2655730112 >> 172.21.15.113:0/2655730112 conn(0x7f4bdc06deb0 msgr2=0x7f4bdc06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4be4200640 1 -- 172.21.15.113:0/2655730112 shutdown_connections 2024-03-04T08:19:49.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.445+0000 7f4be4200640 1 -- 172.21.15.113:0/2655730112 wait complete. 2024-03-04T08:19:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be4200640 1 Processor -- start 2024-03-04T08:19:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be4200640 1 -- start start 2024-03-04T08:19:49.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be4200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc06b060 0x7f4bdc0c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be4200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bdc0cd600 0x7f4bdc150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be4200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bdc155100 0x7f4bdc151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be4200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc15ed30 con 0x7f4bdc0cd600 2024-03-04T08:19:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be4200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc15ebb0 con 0x7f4bdc155100 2024-03-04T08:19:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be4200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc167d30 con 0x7f4bdc06b060 2024-03-04T08:19:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc06b060 0x7f4bdc0c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc06b060 0x7f4bdc0c16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:42442/0 (socket says 172.21.15.113:42442) 2024-03-04T08:19:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be1800640 1 -- 172.21.15.113:0/922561200 learned_addr learned my addr 172.21.15.113:0/922561200 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:19:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.449+0000 7f4be2200640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bdc155100 0x7f4bdc151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:49.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4be1800640 1 -- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bdc155100 msgr2=0x7f4bdc151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:49.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4be1800640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bdc155100 0x7f4bdc151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4be1800640 1 -- 172.21.15.113:0/922561200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bdc0cd600 msgr2=0x7f4bdc150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:49.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4be1800640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bdc0cd600 0x7f4bdc150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4be1800640 1 -- 172.21.15.113:0/922561200 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc15e0e0 con 0x7f4bdc06b060 2024-03-04T08:19:49.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4be1800640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc06b060 0x7f4bdc0c16d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000e820 tx=0x7f4bd000ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:49.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4bd9600640 1 -- 172.21.15.113:0/922561200 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0017070 con 0x7f4bdc06b060 2024-03-04T08:19:49.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4bd9600640 1 -- 172.21.15.113:0/922561200 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd000c940 con 0x7f4bdc06b060 2024-03-04T08:19:49.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4bd9600640 1 -- 172.21.15.113:0/922561200 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000cd30 con 0x7f4bdc06b060 2024-03-04T08:19:49.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4be4200640 1 -- 172.21.15.113:0/922561200 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc068670 con 0x7f4bdc06b060 2024-03-04T08:19:49.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4be4200640 1 -- 172.21.15.113:0/922561200 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bdc1f89b0 con 0x7f4bdc06b060 2024-03-04T08:19:49.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.453+0000 7f4be4200640 1 -- 172.21.15.113:0/922561200 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bdc075c00 con 0x7f4bdc06b060 2024-03-04T08:19:49.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.457+0000 7f4bd9600640 1 -- 172.21.15.113:0/922561200 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4bd001a070 con 0x7f4bdc06b060 2024-03-04T08:19:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.457+0000 7f4bd9600640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bbc066f60 0x7f4bbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.457+0000 7f4bd9600640 1 -- 172.21.15.113:0/922561200 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7945+0+0 (secure 0 0 0) 0x7f4bd0085f50 con 0x7f4bdc06b060 2024-03-04T08:19:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.457+0000 7f4be0e00640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bbc066f60 0x7f4bbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:49.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.457+0000 7f4be0e00640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bbc066f60 0x7f4bbc069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80ad000 tx=0x7f4bc8004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:49.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.469+0000 7f4bd9600640 1 -- 172.21.15.113:0/922561200 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4bd0056030 con 0x7f4bdc06b060 2024-03-04T08:19:49.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.705+0000 7f4be4200640 1 -- 172.21.15.113:0/922561200 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bdc0c7440 con 0x7f4bdc06b060 2024-03-04T08:19:49.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.705+0000 7f4bd9600640 1 -- 172.21.15.113:0/922561200 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f4bd005a0d0 con 0x7f4bdc06b060 2024-03-04T08:19:49.706 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:19:49.706 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839,7692]},"id":1}]} 2024-03-04T08:19:49.706 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 162 2024-03-04T08:19:49.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 -- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bbc066f60 msgr2=0x7f4bbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bbc066f60 0x7f4bbc069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80ad000 tx=0x7f4bc8004740 comp rx=0 tx=0).stop 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 -- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc06b060 msgr2=0x7f4bdc0c16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc06b060 0x7f4bdc0c16d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000e820 tx=0x7f4bd000ece0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 -- 172.21.15.113:0/922561200 shutdown_connections 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4bdc155100 0x7f4bdc151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4bbc066f60 0x7f4bbc069420 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4bdc0cd600 0x7f4bdc150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 --2- 172.21.15.113:0/922561200 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4bdc06b060 0x7f4bdc0c16d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 -- 172.21.15.113:0/922561200 >> 172.21.15.113:0/922561200 conn(0x7f4bdc06deb0 msgr2=0x7f4bdc06c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 -- 172.21.15.113:0/922561200 shutdown_connections 2024-03-04T08:19:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:49.709+0000 7f4be4200640 1 -- 172.21.15.113:0/922561200 wait complete. 2024-03-04T08:19:49.731 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:19:51.732 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:19:51.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.901+0000 7fc7b7e00640 1 Processor -- start 2024-03-04T08:19:51.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b7e00640 1 -- start start 2024-03-04T08:19:51.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc7b8152720 0x7fc7b80c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:51.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80cb6c0 0x7fc7b8066db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:51.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7b80ca8e0 con 0x7fc7b80672f0 2024-03-04T08:19:51.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7b815bec0 con 0x7fc7b8152720 2024-03-04T08:19:51.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b815c1a0 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b7400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc7b80672f0 0x7fc7b8158ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54058/0 (socket says 172.21.15.113:54058) 2024-03-04T08:19:51.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b7400640 1 -- 172.21.15.113:0/1738354408 learned_addr learned my addr 172.21.15.113:0/1738354408 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:19:51.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b6a00640 1 --2- 172.21.15.113:0/1738354408 >> v2:172.21.15.136:3300/0 conn(0x7fc7b8152720 0x7fc7b80c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:51.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b6000640 1 --2- 172.21.15.113:0/1738354408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80cb6c0 0x7fc7b8066db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:51.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b5600640 1 -- 172.21.15.113:0/1738354408 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667936325 0 0) 0x7fc7b80ca8e0 con 0x7fc7b80672f0 2024-03-04T08:19:51.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b5600640 1 -- 172.21.15.113:0/1738354408 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc79c003530 con 0x7fc7b80672f0 2024-03-04T08:19:51.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b6a00640 1 -- 172.21.15.113:0/1738354408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80cb6c0 msgr2=0x7fc7b8066db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:19:51.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b6a00640 1 --2- 172.21.15.113:0/1738354408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80cb6c0 0x7fc7b8066db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:51.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b6a00640 1 -- 172.21.15.113:0/1738354408 >> v1:172.21.15.113:6789/0 conn(0x7fc7b80672f0 legacy=0x7fc7b8158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:51.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b6a00640 1 -- 172.21.15.113:0/1738354408 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b815c480 con 0x7fc7b8152720 2024-03-04T08:19:51.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.905+0000 7fc7b6000640 1 --2- 172.21.15.113:0/1738354408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80cb6c0 0x7fc7b8066db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:19:51.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b6a00640 1 --2- 172.21.15.113:0/1738354408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8152720 0x7fc7b80c94a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80097c0 tx=0x7fc7a801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50129f00acc3629 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:51.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b5600640 1 -- 172.21.15.113:0/1738354408 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80b0070 con 0x7fc7b8152720 2024-03-04T08:19:51.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b5600640 1 -- 172.21.15.113:0/1738354408 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a80ab400 con 0x7fc7b8152720 2024-03-04T08:19:51.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b5600640 1 -- 172.21.15.113:0/1738354408 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80ab7b0 con 0x7fc7b8152720 2024-03-04T08:19:51.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b7e00640 1 -- 172.21.15.113:0/1738354408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8152720 msgr2=0x7fc7b80c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:51.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b7e00640 1 --2- 172.21.15.113:0/1738354408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8152720 0x7fc7b80c94a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80097c0 tx=0x7fc7a801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:19:51.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b7e00640 1 -- 172.21.15.113:0/1738354408 shutdown_connections 2024-03-04T08:19:51.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b7e00640 1 --2- 172.21.15.113:0/1738354408 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80cb6c0 0x7fc7b8066db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:51.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b7e00640 1 --2- 172.21.15.113:0/1738354408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8152720 0x7fc7b80c94a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:51.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b7e00640 1 -- 172.21.15.113:0/1738354408 >> 172.21.15.113:0/1738354408 conn(0x7fc7b806b6c0 msgr2=0x7fc7b8072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:51.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.909+0000 7fc7b7e00640 1 -- 172.21.15.113:0/1738354408 shutdown_connections 2024-03-04T08:19:51.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.913+0000 7fc7b7e00640 1 -- 172.21.15.113:0/1738354408 wait complete. 2024-03-04T08:19:51.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.913+0000 7fc7b7e00640 1 Processor -- start 2024-03-04T08:19:51.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.913+0000 7fc7b7e00640 1 -- start start 2024-03-04T08:19:51.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80672f0 0x7fc7b8145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:51.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b80cb6c0 0x7fc7b81f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:51.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8152720 0x7fc7b81f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:51.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b8152b00 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b815bb30 con 0x7fc7b8152720 2024-03-04T08:19:51.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b8152c80 con 0x7fc7b80672f0 2024-03-04T08:19:51.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b80cb6c0 0x7fc7b81f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:51.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8152720 0x7fc7b81f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:51.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80672f0 0x7fc7b8145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:51.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b80cb6c0 0x7fc7b81f20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47352/0 (socket says 172.21.15.113:47352) 2024-03-04T08:19:51.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b6000640 1 -- 172.21.15.113:0/4273370114 learned_addr learned my addr 172.21.15.113:0/4273370114 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:19:51.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b6000640 1 -- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80672f0 msgr2=0x7fc7b8145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:51.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b6000640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80672f0 0x7fc7b8145ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:51.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b6000640 1 -- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8152720 msgr2=0x7fc7b81f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:51.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b6000640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8152720 0x7fc7b81f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:51.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b6000640 1 -- 172.21.15.113:0/4273370114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b81f1c10 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b6000640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b80cb6c0 0x7fc7b81f20c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400db70 tx=0x7fc7a400b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:51.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7aea00640 1 -- 172.21.15.113:0/4273370114 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4016070 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7aea00640 1 -- 172.21.15.113:0/4273370114 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a400be30 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7aea00640 1 -- 172.21.15.113:0/4273370114 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400de60 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b7e00640 1 -- 172.21.15.113:0/4273370114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b81ef100 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b7e00640 1 -- 172.21.15.113:0/4273370114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b814a4f0 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.917+0000 7fc7b7e00640 1 -- 172.21.15.113:0/4273370114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b8075a80 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.921+0000 7fc7aea00640 1 -- 172.21.15.113:0/4273370114 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7a4019070 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.921+0000 7fc7aea00640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc798066f60 0x7fc798069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:51.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.921+0000 7fc7aea00640 1 -- 172.21.15.113:0/4273370114 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7945+0+0 (secure 0 0 0) 0x7fc7a40853c0 con 0x7fc7b80cb6c0 2024-03-04T08:19:51.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.921+0000 7fc7b6a00640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc798066f60 0x7fc798069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:51.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.921+0000 7fc7b6a00640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc798066f60 0x7fc798069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80ad000 tx=0x7fc7a80a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:51.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:51.929+0000 7fc7aea00640 1 -- 172.21.15.113:0/4273370114 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc7a40554c0 con 0x7fc7b80cb6c0 2024-03-04T08:19:52.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.209+0000 7fc7b7e00640 1 -- 172.21.15.113:0/4273370114 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7b8149b40 con 0x7fc7b80cb6c0 2024-03-04T08:19:52.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.209+0000 7fc7aea00640 1 -- 172.21.15.113:0/4273370114 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+6621 (secure 0 0 0) 0x7fc7a4059560 con 0x7fc7b80cb6c0 2024-03-04T08:19:52.210 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:19:52.210 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839,7692]},"id":1}]} 2024-03-04T08:19:52.210 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 162 2024-03-04T08:19:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 -- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc798066f60 msgr2=0x7fc798069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc798066f60 0x7fc798069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80ad000 tx=0x7fc7a80a6000 comp rx=0 tx=0).stop 2024-03-04T08:19:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 -- 172.21.15.113:0/4273370114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b80cb6c0 msgr2=0x7fc7b81f20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b80cb6c0 0x7fc7b81f20c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400db70 tx=0x7fc7a400b510 comp rx=0 tx=0).stop 2024-03-04T08:19:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 -- 172.21.15.113:0/4273370114 shutdown_connections 2024-03-04T08:19:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8152720 0x7fc7b81f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc798066f60 0x7fc798069420 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:52.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7b80cb6c0 0x7fc7b81f20c0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 --2- 172.21.15.113:0/4273370114 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7b80672f0 0x7fc7b8145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 -- 172.21.15.113:0/4273370114 >> 172.21.15.113:0/4273370114 conn(0x7fc7b806b6c0 msgr2=0x7fc7b80c9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 -- 172.21.15.113:0/4273370114 shutdown_connections 2024-03-04T08:19:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:52.213+0000 7fc7b7e00640 1 -- 172.21.15.113:0/4273370114 wait complete. 2024-03-04T08:19:52.242 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:19:54.243 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:19:54.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.428+0000 7f1143200640 1 Processor -- start 2024-03-04T08:19:54.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.428+0000 7f1143200640 1 -- start start 2024-03-04T08:19:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1143200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11440cb540 0x7f114415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1143200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f114406faa0 0x7f11440c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1143200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11440c4ce0 con 0x7f1144152950 2024-03-04T08:19:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1143200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440c8370 con 0x7f114406faa0 2024-03-04T08:19:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1143200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11440ca0a0 con 0x7f11440cb540 2024-03-04T08:19:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1139400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11440cb540 0x7f114415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1142800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f114406faa0 0x7f11440c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1142800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f114406faa0 0x7f11440c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:53372/0 (socket says 172.21.15.113:53372) 2024-03-04T08:19:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1142800640 1 -- 172.21.15.113:0/3356928612 learned_addr learned my addr 172.21.15.113:0/3356928612 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:19:54.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1141e00640 1 --1- 172.21.15.113:0/3356928612 >> v1:172.21.15.136:6789/0 conn(0x7f1144152950 0x7f114406ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:32970/0 (socket says 172.21.15.113:32970) 2024-03-04T08:19:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1139400640 1 -- 172.21.15.113:0/3356928612 >> v2:172.21.15.136:3300/0 conn(0x7f114406faa0 msgr2=0x7f11440c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:19:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1139400640 1 --2- 172.21.15.113:0/3356928612 >> v2:172.21.15.136:3300/0 conn(0x7f114406faa0 0x7f11440c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1139400640 1 -- 172.21.15.113:0/3356928612 >> v1:172.21.15.136:6789/0 conn(0x7f1144152950 legacy=0x7f114406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:54.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.432+0000 7f1139400640 1 -- 172.21.15.113:0/3356928612 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440c4f80 con 0x7f11440cb540 2024-03-04T08:19:54.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1139400640 1 --2- 172.21.15.113:0/3356928612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11440cb540 0x7f114415bb40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f112c009d90 tx=0x7f112c01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1667f99daeca5a4f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1141400640 1 -- 172.21.15.113:0/3356928612 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c0b4070 con 0x7f11440cb540 2024-03-04T08:19:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1141400640 1 -- 172.21.15.113:0/3356928612 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c0af600 con 0x7f11440cb540 2024-03-04T08:19:54.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1141400640 1 -- 172.21.15.113:0/3356928612 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c0af9d0 con 0x7f11440cb540 2024-03-04T08:19:54.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1143200640 1 -- 172.21.15.113:0/3356928612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11440cb540 msgr2=0x7f114415bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:54.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1143200640 1 --2- 172.21.15.113:0/3356928612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11440cb540 0x7f114415bb40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f112c009d90 tx=0x7f112c01f150 comp rx=0 tx=0).stop 2024-03-04T08:19:54.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1143200640 1 -- 172.21.15.113:0/3356928612 shutdown_connections 2024-03-04T08:19:54.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1143200640 1 --2- 172.21.15.113:0/3356928612 >> v2:172.21.15.136:3300/0 conn(0x7f114406faa0 0x7f11440c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:54.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1143200640 1 --2- 172.21.15.113:0/3356928612 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f11440cb540 0x7f114415bb40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:54.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1143200640 1 -- 172.21.15.113:0/3356928612 >> 172.21.15.113:0/3356928612 conn(0x7f1144072f40 msgr2=0x7f11440706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:54.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1143200640 1 -- 172.21.15.113:0/3356928612 shutdown_connections 2024-03-04T08:19:54.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1143200640 1 -- 172.21.15.113:0/3356928612 wait complete. 2024-03-04T08:19:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1143200640 1 Processor -- start 2024-03-04T08:19:54.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.436+0000 7f1143200640 1 -- start start 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1143200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114406faa0 0x7f11440c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1143200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11440cb540 0x7f1144156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1143200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1144152950 0x7f11441f02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1143200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440c9a50 con 0x7f1144152950 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1143200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440c5220 con 0x7f11440cb540 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1143200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11440c9d30 con 0x7f114406faa0 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1142800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1144152950 0x7f11441f02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1142800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1144152950 0x7f11441f02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47354/0 (socket says 172.21.15.113:47354) 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1142800640 1 -- 172.21.15.113:0/786624406 learned_addr learned my addr 172.21.15.113:0/786624406 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1141e00640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114406faa0 0x7f11440c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:54.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1139400640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11440cb540 0x7f1144156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1142800640 1 -- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114406faa0 msgr2=0x7f11440c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1142800640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114406faa0 0x7f11440c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1142800640 1 -- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11440cb540 msgr2=0x7f1144156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1142800640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11440cb540 0x7f1144156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1142800640 1 -- 172.21.15.113:0/786624406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114406df80 con 0x7f1144152950 2024-03-04T08:19:54.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1142800640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1144152950 0x7f11441f02b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f113c00e7f0 tx=0x7f113c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f113aa00640 1 -- 172.21.15.113:0/786624406 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c017070 con 0x7f1144152950 2024-03-04T08:19:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f113aa00640 1 -- 172.21.15.113:0/786624406 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c00c940 con 0x7f1144152950 2024-03-04T08:19:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f113aa00640 1 -- 172.21.15.113:0/786624406 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c00eea0 con 0x7f1144152950 2024-03-04T08:19:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1143200640 1 -- 172.21.15.113:0/786624406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11441f3a70 con 0x7f1144152950 2024-03-04T08:19:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1143200640 1 -- 172.21.15.113:0/786624406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11441f4030 con 0x7f1144152950 2024-03-04T08:19:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f1143200640 1 -- 172.21.15.113:0/786624406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1144075a80 con 0x7f1144152950 2024-03-04T08:19:54.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.440+0000 7f113aa00640 1 -- 172.21.15.113:0/786624406 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f113c01a070 con 0x7f1144152950 2024-03-04T08:19:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.444+0000 7f113aa00640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1120066f60 0x7f1120069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.444+0000 7f113aa00640 1 -- 172.21.15.113:0/786624406 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7945+0+0 (secure 0 0 0) 0x7f113c085af0 con 0x7f1144152950 2024-03-04T08:19:54.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.444+0000 7f1141e00640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1120066f60 0x7f1120069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:54.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.444+0000 7f1141e00640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1120066f60 0x7f1120069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1134000f70 tx=0x7f1134007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:54.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.448+0000 7f113aa00640 1 -- 172.21.15.113:0/786624406 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f113c055b50 con 0x7f1144152950 2024-03-04T08:19:54.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 -- 172.21.15.113:0/786624406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11440c9580 con 0x7f1144152950 2024-03-04T08:19:54.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f113aa00640 1 -- 172.21.15.113:0/786624406 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f113c059bf0 con 0x7f1144152950 2024-03-04T08:19:54.644 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:19:54.644 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839,7692]},"id":1}]} 2024-03-04T08:19:54.644 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 162 2024-03-04T08:19:54.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 -- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1120066f60 msgr2=0x7f1120069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:54.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1120066f60 0x7f1120069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1134000f70 tx=0x7f1134007e30 comp rx=0 tx=0).stop 2024-03-04T08:19:54.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 -- 172.21.15.113:0/786624406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1144152950 msgr2=0x7f11441f02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:54.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1144152950 0x7f11441f02b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f113c00e7f0 tx=0x7f113c00ecb0 comp rx=0 tx=0).stop 2024-03-04T08:19:54.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 -- 172.21.15.113:0/786624406 shutdown_connections 2024-03-04T08:19:54.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1120066f60 0x7f1120069420 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:54.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1144152950 0x7f11441f02b0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:54.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11440cb540 0x7f1144156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:54.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 --2- 172.21.15.113:0/786624406 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f114406faa0 0x7f11440c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:54.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 -- 172.21.15.113:0/786624406 >> 172.21.15.113:0/786624406 conn(0x7f1144072f40 msgr2=0x7f1144157420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:54.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 -- 172.21.15.113:0/786624406 shutdown_connections 2024-03-04T08:19:54.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:54.644+0000 7f1143200640 1 -- 172.21.15.113:0/786624406 wait complete. 2024-03-04T08:19:54.665 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:19:56.666 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:19:56.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82cce00640 1 Processor -- start 2024-03-04T08:19:56.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82cce00640 1 -- start start 2024-03-04T08:19:56.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82cce00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82c814e4c0 0x7f82c8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:56.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82cce00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82c80c22e0 0x7f82c80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:56.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82cce00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c80c27f0 con 0x7f82c80c2d80 2024-03-04T08:19:56.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82cce00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c815ad60 con 0x7f82c814e4c0 2024-03-04T08:19:56.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82cce00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c80c2a90 con 0x7f82c80c22e0 2024-03-04T08:19:56.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82c5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82c814e4c0 0x7f82c8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:56.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82c5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f82c814e4c0 0x7f82c8072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47366/0 (socket says 172.21.15.113:47366) 2024-03-04T08:19:56.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82c5e00640 1 -- 172.21.15.113:0/1224014649 learned_addr learned my addr 172.21.15.113:0/1224014649 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:19:56.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.804+0000 7f82bd400640 1 --2- 172.21.15.113:0/1224014649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82c80c22e0 0x7f82c80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:56.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.808+0000 7f82c5e00640 1 -- 172.21.15.113:0/1224014649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82c80c22e0 msgr2=0x7f82c80731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:19:56.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.808+0000 7f82c5e00640 1 --2- 172.21.15.113:0/1224014649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82c80c22e0 0x7f82c80731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:56.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.808+0000 7f82c5e00640 1 -- 172.21.15.113:0/1224014649 >> v1:172.21.15.136:6789/0 conn(0x7f82c80c2d80 legacy=0x7f82c815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:56.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.808+0000 7f82c5e00640 1 -- 172.21.15.113:0/1224014649 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c8070c60 con 0x7f82c814e4c0 2024-03-04T08:19:56.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.808+0000 7f82c5e00640 1 --2- 172.21.15.113:0/1224014649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c814e4c0 0x7f82c8072c80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009e50 tx=0x7f82b8021420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce9f0e071e3d7e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:56.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.808+0000 7f82c5400640 1 -- 172.21.15.113:0/1224014649 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80b6070 con 0x7f82c814e4c0 2024-03-04T08:19:56.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.808+0000 7f82c5400640 1 -- 172.21.15.113:0/1224014649 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b80b1400 con 0x7f82c814e4c0 2024-03-04T08:19:56.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.808+0000 7f82c5400640 1 -- 172.21.15.113:0/1224014649 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b80b17d0 con 0x7f82c814e4c0 2024-03-04T08:19:56.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.808+0000 7f82cce00640 1 -- 172.21.15.113:0/1224014649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c814e4c0 msgr2=0x7f82c8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:56.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.808+0000 7f82cce00640 1 --2- 172.21.15.113:0/1224014649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c814e4c0 0x7f82c8072c80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009e50 tx=0x7f82b8021420 comp rx=0 tx=0).stop 2024-03-04T08:19:56.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 -- 172.21.15.113:0/1224014649 shutdown_connections 2024-03-04T08:19:56.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 --2- 172.21.15.113:0/1224014649 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82c80c22e0 0x7f82c80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:56.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 --2- 172.21.15.113:0/1224014649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c814e4c0 0x7f82c8072c80 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:56.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 -- 172.21.15.113:0/1224014649 >> 172.21.15.113:0/1224014649 conn(0x7f82c806e820 msgr2=0x7f82c806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:56.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 -- 172.21.15.113:0/1224014649 shutdown_connections 2024-03-04T08:19:56.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 -- 172.21.15.113:0/1224014649 wait complete. 2024-03-04T08:19:56.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 Processor -- start 2024-03-04T08:19:56.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 -- start start 2024-03-04T08:19:56.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82c80c22e0 0x7f82c814d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:56.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80c2d80 0x7f82c81f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:56.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c814e4c0 0x7f82c8148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:56.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c8071500 con 0x7f82c80c2d80 2024-03-04T08:19:56.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c8071220 con 0x7f82c814e4c0 2024-03-04T08:19:56.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82cce00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c814e8a0 con 0x7f82c80c22e0 2024-03-04T08:19:56.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82bd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80c2d80 0x7f82c81f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82bd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80c2d80 0x7f82c81f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47378/0 (socket says 172.21.15.113:47378) 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82bd400640 1 -- 172.21.15.113:0/2802145843 learned_addr learned my addr 172.21.15.113:0/2802145843 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82c5e00640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82c80c22e0 0x7f82c814d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82c6800640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c814e4c0 0x7f82c8148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82bd400640 1 -- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82c80c22e0 msgr2=0x7f82c814d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82bd400640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82c80c22e0 0x7f82c814d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82bd400640 1 -- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c814e4c0 msgr2=0x7f82c8148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82bd400640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c814e4c0 0x7f82c8148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82bd400640 1 -- 172.21.15.113:0/2802145843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c815a500 con 0x7f82c80c2d80 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82c6800640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c814e4c0 0x7f82c8148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:19:56.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.812+0000 7f82bd400640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80c2d80 0x7f82c81f5110 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f82b000e930 tx=0x7f82b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.816+0000 7f82bea00640 1 -- 172.21.15.113:0/2802145843 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0017070 con 0x7f82c80c2d80 2024-03-04T08:19:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.816+0000 7f82bea00640 1 -- 172.21.15.113:0/2802145843 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b000ca00 con 0x7f82c80c2d80 2024-03-04T08:19:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.816+0000 7f82bea00640 1 -- 172.21.15.113:0/2802145843 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000cdd0 con 0x7f82c80c2d80 2024-03-04T08:19:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.816+0000 7f82cce00640 1 -- 172.21.15.113:0/2802145843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c8159a70 con 0x7f82c80c2d80 2024-03-04T08:19:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.816+0000 7f82cce00640 1 -- 172.21.15.113:0/2802145843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c815aa80 con 0x7f82c80c2d80 2024-03-04T08:19:56.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.816+0000 7f82cce00640 1 -- 172.21.15.113:0/2802145843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c8075c00 con 0x7f82c80c2d80 2024-03-04T08:19:56.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.816+0000 7f82bea00640 1 -- 172.21.15.113:0/2802145843 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82b001a070 con 0x7f82c80c2d80 2024-03-04T08:19:56.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.816+0000 7f82bea00640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82a4066f60 0x7f82a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:56.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.816+0000 7f82bea00640 1 -- 172.21.15.113:0/2802145843 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7917+0+0 (secure 0 0 0) 0x7f82b00859f0 con 0x7f82c80c2d80 2024-03-04T08:19:56.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.820+0000 7f82c5e00640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82a4066f60 0x7f82a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:56.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.820+0000 7f82c5e00640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82a4066f60 0x7f82a4069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f82b80b3000 tx=0x7f82b800d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:56.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:56.820+0000 7f82bea00640 1 -- 172.21.15.113:0/2802145843 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f82b00560f0 con 0x7f82c80c2d80 2024-03-04T08:19:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 -- 172.21.15.113:0/2802145843 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82c81585f0 con 0x7f82c80c2d80 2024-03-04T08:19:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82bea00640 1 -- 172.21.15.113:0/2802145843 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f82b00559f0 con 0x7f82c80c2d80 2024-03-04T08:19:57.160 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:19:57.160 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839,7692]},"id":1}]} 2024-03-04T08:19:57.160 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 162 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 -- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82a4066f60 msgr2=0x7f82a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82a4066f60 0x7f82a4069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f82b80b3000 tx=0x7f82b800d040 comp rx=0 tx=0).stop 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 -- 172.21.15.113:0/2802145843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80c2d80 msgr2=0x7f82c81f5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80c2d80 0x7f82c81f5110 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f82b000e930 tx=0x7f82b000edf0 comp rx=0 tx=0).stop 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 -- 172.21.15.113:0/2802145843 shutdown_connections 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82c814e4c0 0x7f82c8148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82a4066f60 0x7f82a4069420 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82c80c2d80 0x7f82c81f5110 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 --2- 172.21.15.113:0/2802145843 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82c80c22e0 0x7f82c814d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 -- 172.21.15.113:0/2802145843 >> 172.21.15.113:0/2802145843 conn(0x7f82c806e820 msgr2=0x7f82c80c1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 -- 172.21.15.113:0/2802145843 shutdown_connections 2024-03-04T08:19:57.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:57.160+0000 7f82cce00640 1 -- 172.21.15.113:0/2802145843 wait complete. 2024-03-04T08:19:57.190 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:19:59.192 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:19:59.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f6061800640 1 Processor -- start 2024-03-04T08:19:59.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f6061800640 1 -- start start 2024-03-04T08:19:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f6061800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f605c1527f0 0x7f605c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f6061800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605c1591f0 con 0x7f605c0c8200 2024-03-04T08:19:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f6061800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605c1594d0 con 0x7f605c152bd0 2024-03-04T08:19:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f6061800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605c071df0 con 0x7f605c1527f0 2024-03-04T08:19:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f6052000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f605c0c8200 0x7f605c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54316/0 (socket says 172.21.15.113:54316) 2024-03-04T08:19:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f6052000640 1 -- 172.21.15.113:0/3680701884 learned_addr learned my addr 172.21.15.113:0/3680701884 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:19:59.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605aa00640 1 --2- 172.21.15.113:0/3680701884 >> v2:172.21.15.136:3300/0 conn(0x7f605c1527f0 0x7f605c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005580252 0 0) 0x7f605c1591f0 con 0x7f605c0c8200 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6040003530 con 0x7f605c0c8200 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2900335560 0 0) 0x7f6040003530 con 0x7f605c0c8200 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 >> v2:172.21.15.136:3300/0 conn(0x7f605c1527f0 msgr2=0x7f605c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 --2- 172.21.15.113:0/3680701884 >> v2:172.21.15.136:3300/0 conn(0x7f605c1527f0 0x7f605c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 >> v1:172.21.15.136:6789/0 conn(0x7f605c152bd0 legacy=0x7f605c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c0717a0 con 0x7f605c0c8200 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f6044001680 con 0x7f605c0c8200 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 >> v1:172.21.15.113:6789/0 conn(0x7f605c0c8200 legacy=0x7f605c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6040003a10 0x7f6040003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6040004810 0x7f6040004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f604000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:59.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c071df0 con 0x7f60400051d0 2024-03-04T08:19:59.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604000c0d0 con 0x7f6040004810 2024-03-04T08:19:59.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604000c2d0 con 0x7f6040003a10 2024-03-04T08:19:59.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605b400640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f604000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:59.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f6052000640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6040004810 0x7f6040004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:59.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.332+0000 7f605aa00640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6040003a10 0x7f6040003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:59.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.336+0000 7f605b400640 1 -- 172.21.15.113:0/3680701884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6040003a10 msgr2=0x7f6040003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:19:59.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.336+0000 7f605b400640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6040003a10 0x7f6040003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:59.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.336+0000 7f605b400640 1 -- 172.21.15.113:0/3680701884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6040004810 msgr2=0x7f6040004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:19:59.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.336+0000 7f605b400640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6040004810 0x7f6040004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:59.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.336+0000 7f605b400640 1 -- 172.21.15.113:0/3680701884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6040003530 con 0x7f60400051d0 2024-03-04T08:19:59.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f605b400640 1 -- 172.21.15.113:0/3680701884 reap_dead start 2024-03-04T08:19:59.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f605b400640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f604000b9d0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f6054001780 tx=0x7f605401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77af7ac1e48c8517 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:59.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60540013a0 con 0x7f60400051d0 2024-03-04T08:19:59.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60540a9070 con 0x7f60400051d0 2024-03-04T08:19:59.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f605a000640 1 -- 172.21.15.113:0/3680701884 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6054003600 con 0x7f60400051d0 2024-03-04T08:19:59.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f6061800640 1 -- 172.21.15.113:0/3680701884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 msgr2=0x7f604000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:59.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f6061800640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f604000b9d0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f6054001780 tx=0x7f605401c420 comp rx=0 tx=0).stop 2024-03-04T08:19:59.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f6061800640 1 -- 172.21.15.113:0/3680701884 shutdown_connections 2024-03-04T08:19:59.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f6061800640 1 --2- 172.21.15.113:0/3680701884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f604000b9d0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:59.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f6061800640 1 -- 172.21.15.113:0/3680701884 >> 172.21.15.113:0/3680701884 conn(0x7f605c06e820 msgr2=0x7f605c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:59.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f6061800640 1 -- 172.21.15.113:0/3680701884 shutdown_connections 2024-03-04T08:19:59.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f6061800640 1 -- 172.21.15.113:0/3680701884 wait complete. 2024-03-04T08:19:59.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f6061800640 1 Processor -- start 2024-03-04T08:19:59.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f6061800640 1 -- start start 2024-03-04T08:19:59.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.340+0000 7f6061800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f605c14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:59.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6061800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f605c153fe0 0x7f605c1ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:59.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6061800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f605c14ffb0 0x7f605c150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:59.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6061800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c070d40 con 0x7f60400051d0 2024-03-04T08:19:59.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6061800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c1597b0 con 0x7f605c14ffb0 2024-03-04T08:19:59.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6061800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605c070ec0 con 0x7f605c153fe0 2024-03-04T08:19:59.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f605c14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:59.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f605c14f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52990/0 (socket says 172.21.15.113:52990) 2024-03-04T08:19:59.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605aa00640 1 -- 172.21.15.113:0/1496516610 learned_addr learned my addr 172.21.15.113:0/1496516610 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605b400640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f605c14ffb0 0x7f605c150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6052000640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f605c153fe0 0x7f605c1ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605aa00640 1 -- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f605c153fe0 msgr2=0x7f605c1ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605aa00640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f605c153fe0 0x7f605c1ec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605aa00640 1 -- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f605c14ffb0 msgr2=0x7f605c150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605aa00640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f605c14ffb0 0x7f605c150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605aa00640 1 -- 172.21.15.113:0/1496516610 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c155c70 con 0x7f60400051d0 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605b400640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f605c14ffb0 0x7f605c150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f605aa00640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f605c14f350 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f604c009fc0 tx=0x7f604c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6053400640 1 -- 172.21.15.113:0/1496516610 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c00fea0 con 0x7f60400051d0 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6053400640 1 -- 172.21.15.113:0/1496516610 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604c015630 con 0x7f60400051d0 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6053400640 1 -- 172.21.15.113:0/1496516610 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c015a20 con 0x7f60400051d0 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6061800640 1 -- 172.21.15.113:0/1496516610 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c155f50 con 0x7f60400051d0 2024-03-04T08:19:59.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6061800640 1 -- 172.21.15.113:0/1496516610 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605c153c80 con 0x7f60400051d0 2024-03-04T08:19:59.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6061800640 1 -- 172.21.15.113:0/1496516610 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605c075c00 con 0x7f60400051d0 2024-03-04T08:19:59.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.344+0000 7f6053400640 1 -- 172.21.15.113:0/1496516610 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f604c015bc0 con 0x7f60400051d0 2024-03-04T08:19:59.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.348+0000 7f6053400640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6038066f90 0x7f6038069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:19:59.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.348+0000 7f6053400640 1 -- 172.21.15.113:0/1496516610 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7917+0+0 (secure 0 0 0) 0x7f604c086a70 con 0x7f60400051d0 2024-03-04T08:19:59.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.348+0000 7f6052000640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6038066f90 0x7f6038069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:19:59.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.348+0000 7f6052000640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6038066f90 0x7f6038069450 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f605c0c9b20 tx=0x7f6044009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:19:59.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.352+0000 7f6053400640 1 -- 172.21.15.113:0/1496516610 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f604c019030 con 0x7f60400051d0 2024-03-04T08:19:59.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.548+0000 7f6061800640 1 -- 172.21.15.113:0/1496516610 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f605c071fa0 con 0x7f60400051d0 2024-03-04T08:19:59.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6053400640 1 -- 172.21.15.113:0/1496516610 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f604c010040 con 0x7f60400051d0 2024-03-04T08:19:59.551 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:19:59.551 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839,7692]},"id":1}]} 2024-03-04T08:19:59.552 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 162 2024-03-04T08:19:59.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 -- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6038066f90 msgr2=0x7f6038069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:59.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6038066f90 0x7f6038069450 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f605c0c9b20 tx=0x7f6044009040 comp rx=0 tx=0).stop 2024-03-04T08:19:59.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 -- 172.21.15.113:0/1496516610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 msgr2=0x7f605c14f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:19:59.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f605c14f350 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f604c009fc0 tx=0x7f604c00f580 comp rx=0 tx=0).stop 2024-03-04T08:19:59.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 -- 172.21.15.113:0/1496516610 shutdown_connections 2024-03-04T08:19:59.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f605c14ffb0 0x7f605c150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:59.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f605c153fe0 0x7f605c1ec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:59.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6038066f90 0x7f6038069450 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:59.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 --2- 172.21.15.113:0/1496516610 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60400051d0 0x7f605c14f350 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:19:59.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 -- 172.21.15.113:0/1496516610 >> 172.21.15.113:0/1496516610 conn(0x7f605c06e820 msgr2=0x7f605c0c2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:19:59.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 -- 172.21.15.113:0/1496516610 shutdown_connections 2024-03-04T08:19:59.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:19:59.552+0000 7f6061800640 1 -- 172.21.15.113:0/1496516610 wait complete. 2024-03-04T08:19:59.573 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:20:01.574 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.712+0000 7fcdfc600640 1 Processor -- start 2024-03-04T08:20:01.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.712+0000 7fcdfc600640 1 -- start start 2024-03-04T08:20:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.712+0000 7fcdfc600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcdf41527f0 0x7fcdf4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.712+0000 7fcdfc600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf4152bd0 0x7fcdf41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.712+0000 7fcdfc600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf41591f0 con 0x7fcdf40c8200 2024-03-04T08:20:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.712+0000 7fcdfc600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf4071df0 con 0x7fcdf41527f0 2024-03-04T08:20:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdfc600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf41597b0 con 0x7fcdf4152bd0 2024-03-04T08:20:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdfa600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf4152bd0 0x7fcdf41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdfa600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf4152bd0 0x7fcdf41587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:34702/0 (socket says 172.21.15.113:34702) 2024-03-04T08:20:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdfa600640 1 -- 172.21.15.113:0/3271491599 learned_addr learned my addr 172.21.15.113:0/3271491599 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:01.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdf9c00640 1 --2- 172.21.15.113:0/3271491599 >> v2:172.21.15.136:3300/0 conn(0x7fcdf41527f0 0x7fcdf4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdf9c00640 1 -- 172.21.15.113:0/3271491599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf4152bd0 msgr2=0x7fcdf41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdf9c00640 1 --2- 172.21.15.113:0/3271491599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf4152bd0 0x7fcdf41587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdf9c00640 1 -- 172.21.15.113:0/3271491599 >> v1:172.21.15.136:6789/0 conn(0x7fcdf40c8200 legacy=0x7fcdf4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdf9c00640 1 -- 172.21.15.113:0/3271491599 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf41594d0 con 0x7fcdf41527f0 2024-03-04T08:20:01.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdfa600640 1 --2- 172.21.15.113:0/3271491599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf4152bd0 0x7fcdf41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:20:01.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.716+0000 7fcdf9c00640 1 --2- 172.21.15.113:0/3271491599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf41527f0 0x7fcdf4070d40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcde80097c0 tx=0x7fcde801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a3a3c9101d56a1f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:01.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdf3e00640 1 -- 172.21.15.113:0/3271491599 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde80b1070 con 0x7fcdf41527f0 2024-03-04T08:20:01.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdf3e00640 1 -- 172.21.15.113:0/3271491599 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde80ac410 con 0x7fcdf41527f0 2024-03-04T08:20:01.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdf3e00640 1 -- 172.21.15.113:0/3271491599 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde80ac7e0 con 0x7fcdf41527f0 2024-03-04T08:20:01.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 -- 172.21.15.113:0/3271491599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf41527f0 msgr2=0x7fcdf4070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 --2- 172.21.15.113:0/3271491599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf41527f0 0x7fcdf4070d40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcde80097c0 tx=0x7fcde801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:20:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 -- 172.21.15.113:0/3271491599 shutdown_connections 2024-03-04T08:20:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 --2- 172.21.15.113:0/3271491599 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf4152bd0 0x7fcdf41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 --2- 172.21.15.113:0/3271491599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf41527f0 0x7fcdf4070d40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 -- 172.21.15.113:0/3271491599 >> 172.21.15.113:0/3271491599 conn(0x7fcdf406e820 msgr2=0x7fcdf406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 -- 172.21.15.113:0/3271491599 shutdown_connections 2024-03-04T08:20:01.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 -- 172.21.15.113:0/3271491599 wait complete. 2024-03-04T08:20:01.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 Processor -- start 2024-03-04T08:20:01.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 -- start start 2024-03-04T08:20:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf40c8200 0x7fcdf40c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf41527f0 0x7fcdf4148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdf4152bd0 0x7fcdf414ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf41654f0 con 0x7fcdf4152bd0 2024-03-04T08:20:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf4165370 con 0x7fcdf40c8200 2024-03-04T08:20:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfc600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf4165670 con 0x7fcdf41527f0 2024-03-04T08:20:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfa600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdf4152bd0 0x7fcdf414ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:01.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdf9200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf41527f0 0x7fcdf4148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:01.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfa600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdf4152bd0 0x7fcdf414ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53000/0 (socket says 172.21.15.113:53000) 2024-03-04T08:20:01.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdfa600640 1 -- 172.21.15.113:0/1451477478 learned_addr learned my addr 172.21.15.113:0/1451477478 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:01.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.720+0000 7fcdf9c00640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf40c8200 0x7fcdf40c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:01.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdfa600640 1 -- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf41527f0 msgr2=0x7fcdf4148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:01.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdfa600640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf41527f0 0x7fcdf4148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:01.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdfa600640 1 -- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf40c8200 msgr2=0x7fcdf40c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:01.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdfa600640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf40c8200 0x7fcdf40c1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:01.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdfa600640 1 -- 172.21.15.113:0/1451477478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf41f84e0 con 0x7fcdf4152bd0 2024-03-04T08:20:01.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdfa600640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdf4152bd0 0x7fcdf414ecf0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009e70 tx=0x7fcdec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:01.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdf1600640 1 -- 172.21.15.113:0/1451477478 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00cbe0 con 0x7fcdf4152bd0 2024-03-04T08:20:01.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdf1600640 1 -- 172.21.15.113:0/1451477478 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec00cd80 con 0x7fcdf4152bd0 2024-03-04T08:20:01.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdf1600640 1 -- 172.21.15.113:0/1451477478 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec003bc0 con 0x7fcdf4152bd0 2024-03-04T08:20:01.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdfc600640 1 -- 172.21.15.113:0/1451477478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf41f8200 con 0x7fcdf4152bd0 2024-03-04T08:20:01.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdfc600640 1 -- 172.21.15.113:0/1451477478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf414e210 con 0x7fcdf4152bd0 2024-03-04T08:20:01.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdfc600640 1 -- 172.21.15.113:0/1451477478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf4075c00 con 0x7fcdf4152bd0 2024-03-04T08:20:01.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdf1600640 1 -- 172.21.15.113:0/1451477478 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcdec01b030 con 0x7fcdf4152bd0 2024-03-04T08:20:01.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdf1600640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdd4066f90 0x7fcdd4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:01.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.724+0000 7fcdf1600640 1 -- 172.21.15.113:0/1451477478 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7917+0+0 (secure 0 0 0) 0x7fcdec085ac0 con 0x7fcdf4152bd0 2024-03-04T08:20:01.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.728+0000 7fcdf9c00640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdd4066f90 0x7fcdd4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:01.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.728+0000 7fcdf9c00640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdd4066f90 0x7fcdd4069450 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcde80ae000 tx=0x7fcde8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:01.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.732+0000 7fcdf1600640 1 -- 172.21.15.113:0/1451477478 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcdec022020 con 0x7fcdf4152bd0 2024-03-04T08:20:01.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.924+0000 7fcdfc600640 1 -- 172.21.15.113:0/1451477478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdf4162c30 con 0x7fcdf4152bd0 2024-03-04T08:20:01.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.924+0000 7fcdf1600640 1 -- 172.21.15.113:0/1451477478 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+6621 (secure 0 0 0) 0x7fcdec010040 con 0x7fcdf4152bd0 2024-03-04T08:20:01.925 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:01.925 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839,7692]},"id":1}]} 2024-03-04T08:20:01.925 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 162 2024-03-04T08:20:01.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 -- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdd4066f90 msgr2=0x7fcdd4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:01.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdd4066f90 0x7fcdd4069450 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcde80ae000 tx=0x7fcde8008040 comp rx=0 tx=0).stop 2024-03-04T08:20:01.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 -- 172.21.15.113:0/1451477478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdf4152bd0 msgr2=0x7fcdf414ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:01.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdf4152bd0 0x7fcdf414ecf0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009e70 tx=0x7fcdec00c490 comp rx=0 tx=0).stop 2024-03-04T08:20:01.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 -- 172.21.15.113:0/1451477478 shutdown_connections 2024-03-04T08:20:01.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdd4066f90 0x7fcdd4069450 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:01.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdf4152bd0 0x7fcdf414ecf0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:01.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdf41527f0 0x7fcdf4148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:01.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 --2- 172.21.15.113:0/1451477478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdf40c8200 0x7fcdf40c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:01.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 -- 172.21.15.113:0/1451477478 >> 172.21.15.113:0/1451477478 conn(0x7fcdf406e820 msgr2=0x7fcdf40ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:01.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 -- 172.21.15.113:0/1451477478 shutdown_connections 2024-03-04T08:20:01.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:01.928+0000 7fcdfc600640 1 -- 172.21.15.113:0/1451477478 wait complete. 2024-03-04T08:20:01.946 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:20:03.948 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:04.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff787000640 1 Processor -- start 2024-03-04T08:20:04.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff787000640 1 -- start start 2024-03-04T08:20:04.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff787000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff7800cb540 0x7ff78015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:04.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff787000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff78006faa0 0x7ff7800c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:04.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff787000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7800c4ce0 con 0x7ff780152950 2024-03-04T08:20:04.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff787000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78006ff20 con 0x7ff7800cb540 2024-03-04T08:20:04.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff787000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800c8370 con 0x7ff78006faa0 2024-03-04T08:20:04.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff785c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff780152950 0x7ff78006ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54328/0 (socket says 172.21.15.113:54328) 2024-03-04T08:20:04.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff785c00640 1 -- 172.21.15.113:0/3967152227 learned_addr learned my addr 172.21.15.113:0/3967152227 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:04.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff786600640 1 --2- 172.21.15.113:0/3967152227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff78006faa0 0x7ff7800c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:04.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff785200640 1 --2- 172.21.15.113:0/3967152227 >> v2:172.21.15.136:3300/0 conn(0x7ff7800cb540 0x7ff78015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:04.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff77fe00640 1 -- 172.21.15.113:0/3967152227 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635489994 0 0) 0x7ff7800c4ce0 con 0x7ff780152950 2024-03-04T08:20:04.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff77fe00640 1 -- 172.21.15.113:0/3967152227 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff768003530 con 0x7ff780152950 2024-03-04T08:20:04.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff785200640 1 -- 172.21.15.113:0/3967152227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff78006faa0 msgr2=0x7ff7800c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:04.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff785200640 1 --2- 172.21.15.113:0/3967152227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff78006faa0 0x7ff7800c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:04.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff785200640 1 -- 172.21.15.113:0/3967152227 >> v1:172.21.15.113:6789/0 conn(0x7ff780152950 legacy=0x7ff78006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:04.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.092+0000 7ff785200640 1 -- 172.21.15.113:0/3967152227 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800c4f80 con 0x7ff7800cb540 2024-03-04T08:20:04.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff785200640 1 --2- 172.21.15.113:0/3967152227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7800cb540 0x7ff78015bb40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0097c0 tx=0x7ff76c01ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=babbbedb0cf20909 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:04.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff77fe00640 1 -- 172.21.15.113:0/3967152227 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0b5070 con 0x7ff7800cb540 2024-03-04T08:20:04.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff77fe00640 1 -- 172.21.15.113:0/3967152227 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c0a8990 con 0x7ff7800cb540 2024-03-04T08:20:04.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff77fe00640 1 -- 172.21.15.113:0/3967152227 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0a8d60 con 0x7ff7800cb540 2024-03-04T08:20:04.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff787000640 1 -- 172.21.15.113:0/3967152227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7800cb540 msgr2=0x7ff78015bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:04.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff787000640 1 --2- 172.21.15.113:0/3967152227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7800cb540 0x7ff78015bb40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0097c0 tx=0x7ff76c01ec90 comp rx=0 tx=0).stop 2024-03-04T08:20:04.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff787000640 1 -- 172.21.15.113:0/3967152227 shutdown_connections 2024-03-04T08:20:04.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff787000640 1 --2- 172.21.15.113:0/3967152227 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff78006faa0 0x7ff7800c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:04.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff787000640 1 --2- 172.21.15.113:0/3967152227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7800cb540 0x7ff78015bb40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:04.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff787000640 1 -- 172.21.15.113:0/3967152227 >> 172.21.15.113:0/3967152227 conn(0x7ff780072f40 msgr2=0x7ff7800706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:04.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff787000640 1 -- 172.21.15.113:0/3967152227 shutdown_connections 2024-03-04T08:20:04.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.096+0000 7ff787000640 1 -- 172.21.15.113:0/3967152227 wait complete. 2024-03-04T08:20:04.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff787000640 1 Processor -- start 2024-03-04T08:20:04.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff787000640 1 -- start start 2024-03-04T08:20:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff787000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff78006faa0 0x7ff78014db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff787000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7800cb540 0x7ff7801f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff787000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff780152950 0x7ff78014b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff787000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800c5220 con 0x7ff7800cb540 2024-03-04T08:20:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff787000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800ca0a0 con 0x7ff780152950 2024-03-04T08:20:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff787000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7800c9a50 con 0x7ff78006faa0 2024-03-04T08:20:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff785200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7800cb540 0x7ff7801f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff785200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7800cb540 0x7ff7801f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53006/0 (socket says 172.21.15.113:53006) 2024-03-04T08:20:04.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff785200640 1 -- 172.21.15.113:0/1662202024 learned_addr learned my addr 172.21.15.113:0/1662202024 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff786600640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff780152950 0x7ff78014b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff785c00640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff78006faa0 0x7ff78014db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff785200640 1 -- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff78006faa0 msgr2=0x7ff78014db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff785200640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff78006faa0 0x7ff78014db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff785200640 1 -- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff780152950 msgr2=0x7ff78014b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff785200640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff780152950 0x7ff78014b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:04.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff785200640 1 -- 172.21.15.113:0/1662202024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7801f4520 con 0x7ff7800cb540 2024-03-04T08:20:04.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff785200640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7800cb540 0x7ff7801f3aa0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0b3040 tx=0x7ff76c01ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:04.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff77d600640 1 -- 172.21.15.113:0/1662202024 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0bf070 con 0x7ff7800cb540 2024-03-04T08:20:04.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff77d600640 1 -- 172.21.15.113:0/1662202024 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c0b0700 con 0x7ff7800cb540 2024-03-04T08:20:04.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff77d600640 1 -- 172.21.15.113:0/1662202024 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0b5040 con 0x7ff7800cb540 2024-03-04T08:20:04.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff787000640 1 -- 172.21.15.113:0/1662202024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78006cd20 con 0x7ff7800cb540 2024-03-04T08:20:04.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.100+0000 7ff787000640 1 -- 172.21.15.113:0/1662202024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78006d2e0 con 0x7ff7800cb540 2024-03-04T08:20:04.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.104+0000 7ff787000640 1 -- 172.21.15.113:0/1662202024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78006cb30 con 0x7ff7800cb540 2024-03-04T08:20:04.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.104+0000 7ff77d600640 1 -- 172.21.15.113:0/1662202024 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff76c0b0af0 con 0x7ff7800cb540 2024-03-04T08:20:04.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.104+0000 7ff77d600640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff760066f60 0x7ff760069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:04.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.104+0000 7ff77d600640 1 -- 172.21.15.113:0/1662202024 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7917+0+0 (secure 0 0 0) 0x7ff76c122110 con 0x7ff7800cb540 2024-03-04T08:20:04.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.104+0000 7ff785c00640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff760066f60 0x7ff760069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:04.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.104+0000 7ff785c00640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff760066f60 0x7ff760069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff774007fd0 tx=0x7ff774009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:04.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.108+0000 7ff77d600640 1 -- 172.21.15.113:0/1662202024 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff76c0187f0 con 0x7ff7800cb540 2024-03-04T08:20:04.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.304+0000 7ff787000640 1 -- 172.21.15.113:0/1662202024 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff780075a80 con 0x7ff7800cb540 2024-03-04T08:20:04.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.304+0000 7ff77d600640 1 -- 172.21.15.113:0/1662202024 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+6621 (secure 0 0 0) 0x7ff76c0f2190 con 0x7ff7800cb540 2024-03-04T08:20:04.305 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:04.305 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8253,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":162}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:18:18.486135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":7692},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7692":{"gid":7692,"name":"c","rank":2,"incarnation":139,"state":"up:active","state_seq":22,"addr":"172.21.15.113:6833/1928887637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":1928887637},{"type":"v1","addr":"172.21.15.113:6833","nonce":1928887637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839,7692]},"id":1}]} 2024-03-04T08:20:04.306 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 162 2024-03-04T08:20:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 -- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff760066f60 msgr2=0x7ff760069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff760066f60 0x7ff760069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff774007fd0 tx=0x7ff774009040 comp rx=0 tx=0).stop 2024-03-04T08:20:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 -- 172.21.15.113:0/1662202024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7800cb540 msgr2=0x7ff7801f3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7800cb540 0x7ff7801f3aa0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ff76c0b3040 tx=0x7ff76c01ba30 comp rx=0 tx=0).stop 2024-03-04T08:20:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 -- 172.21.15.113:0/1662202024 shutdown_connections 2024-03-04T08:20:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff780152950 0x7ff78014b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff760066f60 0x7ff760069420 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:04.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7800cb540 0x7ff7801f3aa0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 --2- 172.21.15.113:0/1662202024 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff78006faa0 0x7ff78014db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 -- 172.21.15.113:0/1662202024 >> 172.21.15.113:0/1662202024 conn(0x7ff780072f40 msgr2=0x7ff78006ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 -- 172.21.15.113:0/1662202024 shutdown_connections 2024-03-04T08:20:04.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:04.308+0000 7ff787000640 1 -- 172.21.15.113:0/1662202024 wait complete. 2024-03-04T08:20:04.325 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.c is laggy/crashed, in failed state, or mds.c is removed from mdsmap 2024-03-04T08:20:06.326 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:06.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f7412c00640 1 Processor -- start 2024-03-04T08:20:06.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f7412c00640 1 -- start start 2024-03-04T08:20:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f7412c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f740c152ab0 0x7f740c1592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f7412c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f740c0c81c0 0x7f740c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f7412c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c0c7410 con 0x7f740c0c8810 2024-03-04T08:20:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f7412c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c158e50 con 0x7f740c152ab0 2024-03-04T08:20:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f7412c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c159820 con 0x7f740c0c81c0 2024-03-04T08:20:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f740be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f740c152ab0 0x7f740c1592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f740be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f740c152ab0 0x7f740c1592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53008/0 (socket says 172.21.15.113:53008) 2024-03-04T08:20:06.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f740be00640 1 -- 172.21.15.113:0/2318193122 learned_addr learned my addr 172.21.15.113:0/2318193122 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f7410c00640 1 --1- 172.21.15.113:0/2318193122 >> v1:172.21.15.113:6789/0 conn(0x7f740c0c8810 0x7f740c0cc8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54338/0 (socket says 172.21.15.113:54338) 2024-03-04T08:20:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f740b400640 1 --2- 172.21.15.113:0/2318193122 >> v2:172.21.15.136:3300/0 conn(0x7f740c0c81c0 0x7f740c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f740be00640 1 -- 172.21.15.113:0/2318193122 >> v2:172.21.15.136:3300/0 conn(0x7f740c0c81c0 msgr2=0x7f740c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f740be00640 1 --2- 172.21.15.113:0/2318193122 >> v2:172.21.15.136:3300/0 conn(0x7f740c0c81c0 0x7f740c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f740be00640 1 -- 172.21.15.113:0/2318193122 >> v1:172.21.15.113:6789/0 conn(0x7f740c0c8810 legacy=0x7f740c0cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f740be00640 1 -- 172.21.15.113:0/2318193122 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0c76b0 con 0x7f740c152ab0 2024-03-04T08:20:06.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.468+0000 7f740b400640 1 --2- 172.21.15.113:0/2318193122 >> v2:172.21.15.136:3300/0 conn(0x7f740c0c81c0 0x7f740c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:20:06.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f740be00640 1 --2- 172.21.15.113:0/2318193122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 0x7f740c1592e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f740c158fd0 tx=0x7f73fc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3306170add900f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:06.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f740aa00640 1 -- 172.21.15.113:0/2318193122 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0b0070 con 0x7f740c152ab0 2024-03-04T08:20:06.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f740aa00640 1 -- 172.21.15.113:0/2318193122 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc0ab600 con 0x7f740c152ab0 2024-03-04T08:20:06.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f740aa00640 1 -- 172.21.15.113:0/2318193122 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0ab9d0 con 0x7f740c152ab0 2024-03-04T08:20:06.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f7412c00640 1 -- 172.21.15.113:0/2318193122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 msgr2=0x7f740c1592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f7412c00640 1 --2- 172.21.15.113:0/2318193122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 0x7f740c1592e0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f740c158fd0 tx=0x7f73fc01c500 comp rx=0 tx=0).stop 2024-03-04T08:20:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f7412c00640 1 -- 172.21.15.113:0/2318193122 shutdown_connections 2024-03-04T08:20:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f7412c00640 1 --2- 172.21.15.113:0/2318193122 >> v2:172.21.15.136:3300/0 conn(0x7f740c0c81c0 0x7f740c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f7412c00640 1 --2- 172.21.15.113:0/2318193122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 0x7f740c1592e0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f7412c00640 1 -- 172.21.15.113:0/2318193122 >> 172.21.15.113:0/2318193122 conn(0x7f740c06b060 msgr2=0x7f740c06b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f7412c00640 1 -- 172.21.15.113:0/2318193122 shutdown_connections 2024-03-04T08:20:06.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.472+0000 7f7412c00640 1 -- 172.21.15.113:0/2318193122 wait complete. 2024-03-04T08:20:06.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7412c00640 1 Processor -- start 2024-03-04T08:20:06.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7412c00640 1 -- start start 2024-03-04T08:20:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7412c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f740c0c81c0 0x7f740c072e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7412c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f740c0c8810 0x7f740c1f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7412c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 0x7f740c151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7412c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c152e90 con 0x7f740c152ab0 2024-03-04T08:20:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7412c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0c7950 con 0x7f740c0c8810 2024-03-04T08:20:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7412c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0ccff0 con 0x7f740c0c81c0 2024-03-04T08:20:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7410c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 0x7f740c151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7410c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 0x7f740c151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53016/0 (socket says 172.21.15.113:53016) 2024-03-04T08:20:06.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7410c00640 1 -- 172.21.15.113:0/1700498035 learned_addr learned my addr 172.21.15.113:0/1700498035 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f740be00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f740c0c81c0 0x7f740c072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f740b400640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f740c0c8810 0x7f740c1f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7410c00640 1 -- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f740c0c81c0 msgr2=0x7f740c072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7410c00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f740c0c81c0 0x7f740c072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7410c00640 1 -- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f740c0c8810 msgr2=0x7f740c1f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7410c00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f740c0c8810 0x7f740c1f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7410c00640 1 -- 172.21.15.113:0/1700498035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0734c0 con 0x7f740c152ab0 2024-03-04T08:20:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f740b400640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f740c0c8810 0x7f740c1f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:20:06.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7410c00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 0x7f740c151570 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f740400c730 tx=0x7f740400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7403e00640 1 -- 172.21.15.113:0/1700498035 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404017070 con 0x7f740c152ab0 2024-03-04T08:20:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7403e00640 1 -- 172.21.15.113:0/1700498035 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740400ce40 con 0x7f740c152ab0 2024-03-04T08:20:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7412c00640 1 -- 172.21.15.113:0/1700498035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c159af0 con 0x7f740c152ab0 2024-03-04T08:20:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7403e00640 1 -- 172.21.15.113:0/1700498035 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400a880 con 0x7f740c152ab0 2024-03-04T08:20:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.476+0000 7f7412c00640 1 -- 172.21.15.113:0/1700498035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c15a0b0 con 0x7f740c152ab0 2024-03-04T08:20:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.480+0000 7f7412c00640 1 -- 172.21.15.113:0/1700498035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c075c00 con 0x7f740c152ab0 2024-03-04T08:20:06.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.480+0000 7f7403e00640 1 -- 172.21.15.113:0/1700498035 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f740401a070 con 0x7f740c152ab0 2024-03-04T08:20:06.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.480+0000 7f7403e00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f73ec066f90 0x7f73ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.480+0000 7f7403e00640 1 -- 172.21.15.113:0/1700498035 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f7404084f50 con 0x7f740c152ab0 2024-03-04T08:20:06.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.480+0000 7f740be00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f73ec066f90 0x7f73ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.480+0000 7f740be00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f73ec066f90 0x7f73ec069450 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0ad000 tx=0x7f73fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:06.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.484+0000 7f7403e00640 1 -- 172.21.15.113:0/1700498035 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7404054f90 con 0x7f740c152ab0 2024-03-04T08:20:06.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 -- 172.21.15.113:0/1700498035 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f740c159600 con 0x7f740c152ab0 2024-03-04T08:20:06.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7403e00640 1 -- 172.21.15.113:0/1700498035 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 163 v163) v1 ==== 95+0+5820 (secure 0 0 0) 0x7f7404059030 con 0x7f740c152ab0 2024-03-04T08:20:06.680 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:06.680 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":163,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:05.978187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:20:06.680 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 163 2024-03-04T08:20:06.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 -- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f73ec066f90 msgr2=0x7f73ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:06.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f73ec066f90 0x7f73ec069450 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0ad000 tx=0x7f73fc008040 comp rx=0 tx=0).stop 2024-03-04T08:20:06.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 -- 172.21.15.113:0/1700498035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 msgr2=0x7f740c151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:06.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 0x7f740c151570 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f740400c730 tx=0x7f740400cbf0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 -- 172.21.15.113:0/1700498035 shutdown_connections 2024-03-04T08:20:06.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f73ec066f90 0x7f73ec069450 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f740c152ab0 0x7f740c151570 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f740c0c8810 0x7f740c1f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 --2- 172.21.15.113:0/1700498035 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f740c0c81c0 0x7f740c072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 -- 172.21.15.113:0/1700498035 >> 172.21.15.113:0/1700498035 conn(0x7f740c06b060 msgr2=0x7f740c0cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:06.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 -- 172.21.15.113:0/1700498035 shutdown_connections 2024-03-04T08:20:06.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.680+0000 7f7412c00640 1 -- 172.21.15.113:0/1700498035 wait complete. 2024-03-04T08:20:06.701 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c down, removed from mdsmap 2024-03-04T08:20:06.701 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:20:06.701 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:06.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1ad0000640 1 Processor -- start 2024-03-04T08:20:06.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1ad0000640 1 -- start start 2024-03-04T08:20:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1ad0000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 0x7f1ac8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1ad0000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1ac815b900 0x7f1ac80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1ad0000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac80c27f0 con 0x7f1ac80c22e0 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1ad0000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac815ad60 con 0x7f1ac815b900 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1ad0000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80c2a90 con 0x7f1ac814e4c0 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1accc00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1ac80c22e0 0x7f1ac815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54348/0 (socket says 172.21.15.113:54348) 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1accc00640 1 -- 172.21.15.113:0/107025838 learned_addr learned my addr 172.21.15.113:0/107025838 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1acd600640 1 --2- 172.21.15.113:0/107025838 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 0x7f1ac8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1ace000640 1 --2- 172.21.15.113:0/107025838 >> v2:172.21.15.136:3300/0 conn(0x7f1ac815b900 0x7f1ac80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1ac7e00640 1 -- 172.21.15.113:0/107025838 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206832929 0 0) 0x7f1ac80c27f0 con 0x7f1ac80c22e0 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1ac7e00640 1 -- 172.21.15.113:0/107025838 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ab0003530 con 0x7f1ac80c22e0 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1acd600640 1 -- 172.21.15.113:0/107025838 >> v2:172.21.15.136:3300/0 conn(0x7f1ac815b900 msgr2=0x7f1ac80731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1acd600640 1 --2- 172.21.15.113:0/107025838 >> v2:172.21.15.136:3300/0 conn(0x7f1ac815b900 0x7f1ac80731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1acd600640 1 -- 172.21.15.113:0/107025838 >> v1:172.21.15.113:6789/0 conn(0x7f1ac80c22e0 legacy=0x7f1ac815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:06.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.844+0000 7f1acd600640 1 -- 172.21.15.113:0/107025838 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac8070c60 con 0x7f1ac814e4c0 2024-03-04T08:20:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.848+0000 7f1acd600640 1 --2- 172.21.15.113:0/107025838 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 0x7f1ac8072c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0097c0 tx=0x7f1abc01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2113ff8ed040f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.848+0000 7f1ac7e00640 1 -- 172.21.15.113:0/107025838 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0b5070 con 0x7f1ac814e4c0 2024-03-04T08:20:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.848+0000 7f1ac7e00640 1 -- 172.21.15.113:0/107025838 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc0a8990 con 0x7f1ac814e4c0 2024-03-04T08:20:06.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.848+0000 7f1ac7e00640 1 -- 172.21.15.113:0/107025838 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0a8d40 con 0x7f1ac814e4c0 2024-03-04T08:20:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.852+0000 7f1ad0000640 1 -- 172.21.15.113:0/107025838 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 msgr2=0x7f1ac8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.852+0000 7f1ad0000640 1 --2- 172.21.15.113:0/107025838 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 0x7f1ac8072c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0097c0 tx=0x7f1abc01ec90 comp rx=0 tx=0).stop 2024-03-04T08:20:06.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.852+0000 7f1ad0000640 1 -- 172.21.15.113:0/107025838 shutdown_connections 2024-03-04T08:20:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.852+0000 7f1ad0000640 1 --2- 172.21.15.113:0/107025838 >> v2:172.21.15.136:3300/0 conn(0x7f1ac815b900 0x7f1ac80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.852+0000 7f1ad0000640 1 --2- 172.21.15.113:0/107025838 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 0x7f1ac8072c80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.852+0000 7f1ad0000640 1 -- 172.21.15.113:0/107025838 >> 172.21.15.113:0/107025838 conn(0x7f1ac806e820 msgr2=0x7f1ac806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.852+0000 7f1ad0000640 1 -- 172.21.15.113:0/107025838 shutdown_connections 2024-03-04T08:20:06.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.852+0000 7f1ad0000640 1 -- 172.21.15.113:0/107025838 wait complete. 2024-03-04T08:20:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.852+0000 7f1ad0000640 1 Processor -- start 2024-03-04T08:20:06.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.852+0000 7f1ad0000640 1 -- start start 2024-03-04T08:20:06.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ad0000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c22e0 0x7f1ac80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ad0000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 0x7f1ac814a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ad0000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac815b900 0x7f1ac806b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ad0000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8071500 con 0x7f1ac80c22e0 2024-03-04T08:20:06.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ad0000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac8071220 con 0x7f1ac815b900 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ad0000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac814e8a0 con 0x7f1ac814e4c0 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1acd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c22e0 0x7f1ac80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1acd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c22e0 0x7f1ac80cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53026/0 (socket says 172.21.15.113:53026) 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1acd600640 1 -- 172.21.15.113:0/2205802488 learned_addr learned my addr 172.21.15.113:0/2205802488 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1accc00640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 0x7f1ac814a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ace000640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac815b900 0x7f1ac806b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1acd600640 1 -- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 msgr2=0x7f1ac814a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1acd600640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 0x7f1ac814a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1acd600640 1 -- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac815b900 msgr2=0x7f1ac806b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1acd600640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac815b900 0x7f1ac806b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1acd600640 1 -- 172.21.15.113:0/2205802488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac80cbae0 con 0x7f1ac80c22e0 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1accc00640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 0x7f1ac814a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1acd600640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c22e0 0x7f1ac80cb5a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0b3040 tx=0x7f1abc01ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:06.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ac5600640 1 -- 172.21.15.113:0/2205802488 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0bf070 con 0x7f1ac80c22e0 2024-03-04T08:20:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ac5600640 1 -- 172.21.15.113:0/2205802488 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1abc004240 con 0x7f1ac80c22e0 2024-03-04T08:20:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ac5600640 1 -- 172.21.15.113:0/2205802488 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0b5040 con 0x7f1ac80c22e0 2024-03-04T08:20:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ad0000640 1 -- 172.21.15.113:0/2205802488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac814aee0 con 0x7f1ac80c22e0 2024-03-04T08:20:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ad0000640 1 -- 172.21.15.113:0/2205802488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac81e53f0 con 0x7f1ac80c22e0 2024-03-04T08:20:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ad0000640 1 -- 172.21.15.113:0/2205802488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8075c00 con 0x7f1ac80c22e0 2024-03-04T08:20:06.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.856+0000 7f1ac5600640 1 -- 172.21.15.113:0/2205802488 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1abc003620 con 0x7f1ac80c22e0 2024-03-04T08:20:06.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.860+0000 7f1ac5600640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:06.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.860+0000 7f1ac5600640 1 -- 172.21.15.113:0/2205802488 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f1abc1221b0 con 0x7f1ac80c22e0 2024-03-04T08:20:06.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.860+0000 7f1accc00640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:06.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.860+0000 7f1accc00640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 0x7f1aa8069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1ac81e7020 tx=0x7f1ab4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:06.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:06.864+0000 7f1ac5600640 1 -- 172.21.15.113:0/2205802488 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1ac80cbae0 con 0x7f1ac80c22e0 2024-03-04T08:20:07.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.172+0000 7f1ad0000640 1 -- 172.21.15.113:0/2205802488 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ac8155190 con 0x7f1ac80c22e0 2024-03-04T08:20:07.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ac5600640 1 -- 172.21.15.113:0/2205802488 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 163 v163) v1 ==== 95+0+5820 (secure 0 0 0) 0x7f1abc0f8070 con 0x7f1ac80c22e0 2024-03-04T08:20:07.175 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:07.175 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":163,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:05.978187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:20:07.175 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 163 2024-03-04T08:20:07.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 -- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 msgr2=0x7f1aa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:07.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 0x7f1aa8069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1ac81e7020 tx=0x7f1ab4009040 comp rx=0 tx=0).stop 2024-03-04T08:20:07.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 -- 172.21.15.113:0/2205802488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c22e0 msgr2=0x7f1ac80cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:07.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c22e0 0x7f1ac80cb5a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0b3040 tx=0x7f1abc01ba30 comp rx=0 tx=0).stop 2024-03-04T08:20:07.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 -- 172.21.15.113:0/2205802488 shutdown_connections 2024-03-04T08:20:07.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ac815b900 0x7f1ac806b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:07.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1ac814e4c0 0x7f1ac814a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:07.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1aa8066f60 0x7f1aa8069420 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:07.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 --2- 172.21.15.113:0/2205802488 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ac80c22e0 0x7f1ac80cb5a0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:07.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 -- 172.21.15.113:0/2205802488 >> 172.21.15.113:0/2205802488 conn(0x7f1ac806e820 msgr2=0x7f1ac80c73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:07.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 -- 172.21.15.113:0/2205802488 shutdown_connections 2024-03-04T08:20:07.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:07.176+0000 7f1ad0000640 1 -- 172.21.15.113:0/2205802488 wait complete. 2024-03-04T08:20:07.197 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 163, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 163, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8253": { "addr": "172.21.15.113:6836/893414524", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 893414524, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 893414524, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8253, "incarnation": 163, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 80, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:20:05.978187+0000", "qdb_cluster": [ 7839, 8032 ], "qdb_leader": 7839, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 8032, "mds_2": 8253 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:20:09.197 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:09.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.336+0000 7f2dca000640 1 Processor -- start 2024-03-04T08:20:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.336+0000 7f2dca000640 1 -- start start 2024-03-04T08:20:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dca000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2dc40ce6b0 0x7f2dc415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dca000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2dc406de80 0x7f2dc4154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dca000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc406c210 con 0x7f2dc40ce2d0 2024-03-04T08:20:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dca000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc415bbf0 con 0x7f2dc40ce6b0 2024-03-04T08:20:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dca000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc406eb80 con 0x7f2dc406de80 2024-03-04T08:20:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dbbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2dc40ce6b0 0x7f2dc415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dbbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2dc40ce6b0 0x7f2dc415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35860/0 (socket says 172.21.15.113:35860) 2024-03-04T08:20:09.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dbbe00640 1 -- 172.21.15.113:0/2060374987 learned_addr learned my addr 172.21.15.113:0/2060374987 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:09.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dc3e00640 1 --2- 172.21.15.113:0/2060374987 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2dc406de80 0x7f2dc4154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:09.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dbbe00640 1 -- 172.21.15.113:0/2060374987 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2dc406de80 msgr2=0x7f2dc4154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:09.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dbbe00640 1 --2- 172.21.15.113:0/2060374987 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2dc406de80 0x7f2dc4154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:09.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dbbe00640 1 -- 172.21.15.113:0/2060374987 >> v1:172.21.15.113:6789/0 conn(0x7f2dc40ce2d0 legacy=0x7f2dc406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:09.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dbbe00640 1 -- 172.21.15.113:0/2060374987 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc406c4f0 con 0x7f2dc40ce6b0 2024-03-04T08:20:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.340+0000 7f2dc2a00640 1 -- 172.21.15.113:0/2060374987 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892877886 0 0) 0x7f2dc406c210 con 0x7f2dc40ce2d0 2024-03-04T08:20:09.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dbbe00640 1 --2- 172.21.15.113:0/2060374987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc40ce6b0 0x7f2dc415b6b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63ec61bed88f3ef server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:09.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dc2a00640 1 -- 172.21.15.113:0/2060374987 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0b5070 con 0x7f2dc40ce6b0 2024-03-04T08:20:09.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dc2a00640 1 -- 172.21.15.113:0/2060374987 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac0a8a50 con 0x7f2dc40ce6b0 2024-03-04T08:20:09.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dc2a00640 1 -- 172.21.15.113:0/2060374987 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0a8e20 con 0x7f2dc40ce6b0 2024-03-04T08:20:09.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dca000640 1 -- 172.21.15.113:0/2060374987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc40ce6b0 msgr2=0x7f2dc415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:09.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dca000640 1 --2- 172.21.15.113:0/2060374987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc40ce6b0 0x7f2dc415b6b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac01ed50 comp rx=0 tx=0).stop 2024-03-04T08:20:09.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dca000640 1 -- 172.21.15.113:0/2060374987 shutdown_connections 2024-03-04T08:20:09.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dca000640 1 --2- 172.21.15.113:0/2060374987 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2dc406de80 0x7f2dc4154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:09.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dca000640 1 --2- 172.21.15.113:0/2060374987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc40ce6b0 0x7f2dc415b6b0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:09.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dca000640 1 -- 172.21.15.113:0/2060374987 >> 172.21.15.113:0/2060374987 conn(0x7f2dc40709a0 msgr2=0x7f2dc4070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:09.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dca000640 1 -- 172.21.15.113:0/2060374987 shutdown_connections 2024-03-04T08:20:09.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dca000640 1 -- 172.21.15.113:0/2060374987 wait complete. 2024-03-04T08:20:09.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dca000640 1 Processor -- start 2024-03-04T08:20:09.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.344+0000 7f2dca000640 1 -- start start 2024-03-04T08:20:09.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dca000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc406de80 0x7f2dc41514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:09.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dca000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dc40ce2d0 0x7f2dc41f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:09.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dca000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2dc40ce6b0 0x7f2dc41f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dca000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc406c7d0 con 0x7f2dc406de80 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dca000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc406e810 con 0x7f2dc40ce2d0 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dca000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc406f720 con 0x7f2dc40ce6b0 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dc3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc406de80 0x7f2dc41514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dc3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc406de80 0x7f2dc41514e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35866/0 (socket says 172.21.15.113:35866) 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dc3400640 1 -- 172.21.15.113:0/2922919527 learned_addr learned my addr 172.21.15.113:0/2922919527 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dc3e00640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2dc40ce6b0 0x7f2dc41f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dbbe00640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dc40ce2d0 0x7f2dc41f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dc3400640 1 -- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2dc40ce6b0 msgr2=0x7f2dc41f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dc3400640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2dc40ce6b0 0x7f2dc41f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dc3400640 1 -- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dc40ce2d0 msgr2=0x7f2dc41f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dc3400640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dc40ce2d0 0x7f2dc41f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dc3400640 1 -- 172.21.15.113:0/2922919527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc41f5370 con 0x7f2dc406de80 2024-03-04T08:20:09.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dc3400640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc406de80 0x7f2dc41514e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f2db400c500 tx=0x7f2db400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dbb400640 1 -- 172.21.15.113:0/2922919527 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4017070 con 0x7f2dc406de80 2024-03-04T08:20:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dbb400640 1 -- 172.21.15.113:0/2922919527 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4003600 con 0x7f2dc406de80 2024-03-04T08:20:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dbb400640 1 -- 172.21.15.113:0/2922919527 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40039f0 con 0x7f2dc406de80 2024-03-04T08:20:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dca000640 1 -- 172.21.15.113:0/2922919527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc4150b50 con 0x7f2dc406de80 2024-03-04T08:20:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dca000640 1 -- 172.21.15.113:0/2922919527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc4150880 con 0x7f2dc406de80 2024-03-04T08:20:09.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.348+0000 7f2dca000640 1 -- 172.21.15.113:0/2922919527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc4075c00 con 0x7f2dc406de80 2024-03-04T08:20:09.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.352+0000 7f2dbb400640 1 -- 172.21.15.113:0/2922919527 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2db401a070 con 0x7f2dc406de80 2024-03-04T08:20:09.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.352+0000 7f2dbb400640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2da0066f60 0x7f2da0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:09.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.352+0000 7f2dbb400640 1 -- 172.21.15.113:0/2922919527 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f2db4084c80 con 0x7f2dc406de80 2024-03-04T08:20:09.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.352+0000 7f2dbbe00640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2da0066f60 0x7f2da0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:09.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.352+0000 7f2dbbe00640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2da0066f60 0x7f2da0069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0b2000 tx=0x7f2dac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:09.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.356+0000 7f2dbb400640 1 -- 172.21.15.113:0/2922919527 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2db4054eb0 con 0x7f2dc406de80 2024-03-04T08:20:09.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 -- 172.21.15.113:0/2922919527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dc40cef20 con 0x7f2dc406de80 2024-03-04T08:20:09.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dbb400640 1 -- 172.21.15.113:0/2922919527 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 163 v163) v1 ==== 95+0+5820 (secure 0 0 0) 0x7f2db4058f50 con 0x7f2dc406de80 2024-03-04T08:20:09.596 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:09.597 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":163,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:05.978187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:20:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 163 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 -- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2da0066f60 msgr2=0x7f2da0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2da0066f60 0x7f2da0069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0b2000 tx=0x7f2dac009040 comp rx=0 tx=0).stop 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 -- 172.21.15.113:0/2922919527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc406de80 msgr2=0x7f2dc41514e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc406de80 0x7f2dc41514e0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f2db400c500 tx=0x7f2db400c9c0 comp rx=0 tx=0).stop 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 -- 172.21.15.113:0/2922919527 shutdown_connections 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2dc40ce6b0 0x7f2dc41f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2dc40ce2d0 0x7f2dc41f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2da0066f60 0x7f2da0069420 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 --2- 172.21.15.113:0/2922919527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2dc406de80 0x7f2dc41514e0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 -- 172.21.15.113:0/2922919527 >> 172.21.15.113:0/2922919527 conn(0x7f2dc40709a0 msgr2=0x7f2dc415aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 -- 172.21.15.113:0/2922919527 shutdown_connections 2024-03-04T08:20:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:09.600+0000 7f2dca000640 1 -- 172.21.15.113:0/2922919527 wait complete. 2024-03-04T08:20:09.619 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:20:11.620 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:11.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79f3000640 1 Processor -- start 2024-03-04T08:20:11.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79f3000640 1 -- start start 2024-03-04T08:20:11.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79f3000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec152d80 0x7f79ec1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:11.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79f3000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79ec0ca5c0 0x7f79ec0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:11.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79f3000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f79ec158ee0 0x7f79ec0c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:11.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79f3000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec153160 con 0x7f79ec0ca5c0 2024-03-04T08:20:11.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79f3000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec06b920 con 0x7f79ec158ee0 2024-03-04T08:20:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79f3000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec06bbc0 con 0x7f79ec152d80 2024-03-04T08:20:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79e3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79ec0ca5c0 0x7f79ec0c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79e3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f79ec0ca5c0 0x7f79ec0c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35868/0 (socket says 172.21.15.113:35868) 2024-03-04T08:20:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79e3e00640 1 -- 172.21.15.113:0/4058411763 learned_addr learned my addr 172.21.15.113:0/4058411763 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79ebe00640 1 --2- 172.21.15.113:0/4058411763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec152d80 0x7f79ec1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:11.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79f1000640 1 --2- 172.21.15.113:0/4058411763 >> v2:172.21.15.136:3300/0 conn(0x7f79ec158ee0 0x7f79ec0c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79e3e00640 1 -- 172.21.15.113:0/4058411763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec152d80 msgr2=0x7f79ec1589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79e3e00640 1 --2- 172.21.15.113:0/4058411763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec152d80 0x7f79ec1589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79e3e00640 1 -- 172.21.15.113:0/4058411763 >> v2:172.21.15.136:3300/0 conn(0x7f79ec158ee0 msgr2=0x7f79ec0c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79e3e00640 1 --2- 172.21.15.113:0/4058411763 >> v2:172.21.15.136:3300/0 conn(0x7f79ec158ee0 0x7f79ec0c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79e3e00640 1 -- 172.21.15.113:0/4058411763 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec06be80 con 0x7f79ec0ca5c0 2024-03-04T08:20:11.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.764+0000 7f79ebe00640 1 --2- 172.21.15.113:0/4058411763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec152d80 0x7f79ec1589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:20:11.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79e3e00640 1 --2- 172.21.15.113:0/4058411763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec0ca5c0 0x7f79ec0c95a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f79ec06f5d0 tx=0x7f79d401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4365cda413a9a923 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:11.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79eb400640 1 -- 172.21.15.113:0/4058411763 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d40b5070 con 0x7f79ec0ca5c0 2024-03-04T08:20:11.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79eb400640 1 -- 172.21.15.113:0/4058411763 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d40a89d0 con 0x7f79ec0ca5c0 2024-03-04T08:20:11.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79eb400640 1 -- 172.21.15.113:0/4058411763 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d40a8da0 con 0x7f79ec0ca5c0 2024-03-04T08:20:11.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79f3000640 1 -- 172.21.15.113:0/4058411763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec0ca5c0 msgr2=0x7f79ec0c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79f3000640 1 --2- 172.21.15.113:0/4058411763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec0ca5c0 0x7f79ec0c95a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f79ec06f5d0 tx=0x7f79d401ed10 comp rx=0 tx=0).stop 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79f3000640 1 -- 172.21.15.113:0/4058411763 shutdown_connections 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79f3000640 1 --2- 172.21.15.113:0/4058411763 >> v2:172.21.15.136:3300/0 conn(0x7f79ec158ee0 0x7f79ec0c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79f3000640 1 --2- 172.21.15.113:0/4058411763 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec0ca5c0 0x7f79ec0c95a0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79f3000640 1 --2- 172.21.15.113:0/4058411763 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec152d80 0x7f79ec1589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79f3000640 1 -- 172.21.15.113:0/4058411763 >> 172.21.15.113:0/4058411763 conn(0x7f79ec06e240 msgr2=0x7f79ec06e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79f3000640 1 -- 172.21.15.113:0/4058411763 shutdown_connections 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.768+0000 7f79f3000640 1 -- 172.21.15.113:0/4058411763 wait complete. 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f3000640 1 Processor -- start 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f3000640 1 -- start start 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f3000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec0ca5c0 0x7f79ec14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f3000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec152d80 0x7f79ec14c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f3000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec158ee0 0x7f79ec155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f3000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0cd7a0 con 0x7f79ec152d80 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f3000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0cd4c0 con 0x7f79ec0ca5c0 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f3000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0cda80 con 0x7f79ec158ee0 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec152d80 0x7f79ec14c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec152d80 0x7f79ec14c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35882/0 (socket says 172.21.15.113:35882) 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e3e00640 1 -- 172.21.15.113:0/1009466110 learned_addr learned my addr 172.21.15.113:0/1009466110 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79ebe00640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec0ca5c0 0x7f79ec14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f1000640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec158ee0 0x7f79ec155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e3e00640 1 -- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec158ee0 msgr2=0x7f79ec155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e3e00640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec158ee0 0x7f79ec155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e3e00640 1 -- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec0ca5c0 msgr2=0x7f79ec14bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e3e00640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec0ca5c0 0x7f79ec14bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e3e00640 1 -- 172.21.15.113:0/1009466110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec15bd40 con 0x7f79ec152d80 2024-03-04T08:20:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e3e00640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec152d80 0x7f79ec14c320 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f79d401d3c0 tx=0x7f79d401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e8c00640 1 -- 172.21.15.113:0/1009466110 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d40b5070 con 0x7f79ec152d80 2024-03-04T08:20:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e8c00640 1 -- 172.21.15.113:0/1009466110 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d40b0a70 con 0x7f79ec152d80 2024-03-04T08:20:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79e8c00640 1 -- 172.21.15.113:0/1009466110 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d40b0e60 con 0x7f79ec152d80 2024-03-04T08:20:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f3000640 1 -- 172.21.15.113:0/1009466110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec15bfe0 con 0x7f79ec152d80 2024-03-04T08:20:11.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.772+0000 7f79f3000640 1 -- 172.21.15.113:0/1009466110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec15a5b0 con 0x7f79ec152d80 2024-03-04T08:20:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.776+0000 7f79f3000640 1 -- 172.21.15.113:0/1009466110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec075c00 con 0x7f79ec152d80 2024-03-04T08:20:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.776+0000 7f79e8c00640 1 -- 172.21.15.113:0/1009466110 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79d40c2060 con 0x7f79ec152d80 2024-03-04T08:20:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.776+0000 7f79e8c00640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c8066f90 0x7f79c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.776+0000 7f79e8c00640 1 -- 172.21.15.113:0/1009466110 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f79d4122910 con 0x7f79ec152d80 2024-03-04T08:20:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.776+0000 7f79ebe00640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c8066f90 0x7f79c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.780+0000 7f79ebe00640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c8066f90 0x7f79c8069450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0098d0 tx=0x7f79dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:11.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.788+0000 7f79e8c00640 1 -- 172.21.15.113:0/1009466110 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f79d4018870 con 0x7f79ec152d80 2024-03-04T08:20:11.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 -- 172.21.15.113:0/1009466110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79ec0cac00 con 0x7f79ec152d80 2024-03-04T08:20:11.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79e8c00640 1 -- 172.21.15.113:0/1009466110 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 163 v163) v1 ==== 95+0+5820 (secure 0 0 0) 0x7f79d40f29f0 con 0x7f79ec152d80 2024-03-04T08:20:11.980 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:11.980 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":163,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:05.978187+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:20:11.980 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 163 2024-03-04T08:20:11.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 -- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c8066f90 msgr2=0x7f79c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:11.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c8066f90 0x7f79c8069450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0098d0 tx=0x7f79dc008040 comp rx=0 tx=0).stop 2024-03-04T08:20:11.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 -- 172.21.15.113:0/1009466110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec152d80 msgr2=0x7f79ec14c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:11.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec152d80 0x7f79ec14c320 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f79d401d3c0 tx=0x7f79d401bab0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 -- 172.21.15.113:0/1009466110 shutdown_connections 2024-03-04T08:20:11.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f79ec158ee0 0x7f79ec155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f79c8066f90 0x7f79c8069450 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f79ec152d80 0x7f79ec14c320 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 --2- 172.21.15.113:0/1009466110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79ec0ca5c0 0x7f79ec14bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:11.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 -- 172.21.15.113:0/1009466110 >> 172.21.15.113:0/1009466110 conn(0x7f79ec06e240 msgr2=0x7f79ec0cbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:11.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 -- 172.21.15.113:0/1009466110 shutdown_connections 2024-03-04T08:20:11.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:11.984+0000 7f79f3000640 1 -- 172.21.15.113:0/1009466110 wait complete. 2024-03-04T08:20:12.010 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:20:14.011 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:14.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8eca00640 1 Processor -- start 2024-03-04T08:20:14.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8eca00640 1 -- start start 2024-03-04T08:20:14.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8eca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8e80c42e0 0x7fc8e815b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:14.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8eca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e806d480 con 0x7fc8e806f5b0 2024-03-04T08:20:14.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8eca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e806c590 con 0x7fc8e8154d00 2024-03-04T08:20:14.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8eca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e815ba50 con 0x7fc8e80c42e0 2024-03-04T08:20:14.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8dd400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc8e806f5b0 0x7fc8e80c3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55692/0 (socket says 172.21.15.113:55692) 2024-03-04T08:20:14.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8dd400640 1 -- 172.21.15.113:0/4246918859 learned_addr learned my addr 172.21.15.113:0/4246918859 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:14.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8e6800640 1 --2- 172.21.15.113:0/4246918859 >> v2:172.21.15.113:3300/0 conn(0x7fc8e80c42e0 0x7fc8e815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:14.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8e6800640 1 -- 172.21.15.113:0/4246918859 >> v1:172.21.15.136:6789/0 conn(0x7fc8e8154d00 legacy=0x7fc8e80ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:14.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8e6800640 1 -- 172.21.15.113:0/4246918859 >> v1:172.21.15.113:6789/0 conn(0x7fc8e806f5b0 legacy=0x7fc8e80c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:14.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.156+0000 7fc8e6800640 1 -- 172.21.15.113:0/4246918859 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e806c870 con 0x7fc8e80c42e0 2024-03-04T08:20:14.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8e6800640 1 --2- 172.21.15.113:0/4246918859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e80c42e0 0x7fc8e815b510 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00097c0 tx=0x7fc8e001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234a3dcdb116ad66 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:14.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8e5400640 1 -- 172.21.15.113:0/4246918859 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00b1070 con 0x7fc8e80c42e0 2024-03-04T08:20:14.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8e5400640 1 -- 172.21.15.113:0/4246918859 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e00ac5f0 con 0x7fc8e80c42e0 2024-03-04T08:20:14.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8e5400640 1 -- 172.21.15.113:0/4246918859 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00ac9c0 con 0x7fc8e80c42e0 2024-03-04T08:20:14.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8eca00640 1 -- 172.21.15.113:0/4246918859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e80c42e0 msgr2=0x7fc8e815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:14.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8eca00640 1 --2- 172.21.15.113:0/4246918859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e80c42e0 0x7fc8e815b510 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00097c0 tx=0x7fc8e001c4a0 comp rx=0 tx=0).stop 2024-03-04T08:20:14.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8eca00640 1 -- 172.21.15.113:0/4246918859 shutdown_connections 2024-03-04T08:20:14.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8eca00640 1 --2- 172.21.15.113:0/4246918859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e80c42e0 0x7fc8e815b510 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:14.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8eca00640 1 -- 172.21.15.113:0/4246918859 >> 172.21.15.113:0/4246918859 conn(0x7fc8e8066db0 msgr2=0x7fc8e80671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:14.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8eca00640 1 -- 172.21.15.113:0/4246918859 shutdown_connections 2024-03-04T08:20:14.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8eca00640 1 -- 172.21.15.113:0/4246918859 wait complete. 2024-03-04T08:20:14.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8eca00640 1 Processor -- start 2024-03-04T08:20:14.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.164+0000 7fc8eca00640 1 -- start start 2024-03-04T08:20:14.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8eca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e806f5b0 0x7fc8e81f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:14.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8eca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8e80c42e0 0x7fc8e81f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:14.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8eca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8e8154d00 0x7fc8e814c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:14.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8eca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e806c170 con 0x7fc8e806f5b0 2024-03-04T08:20:14.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8eca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e81550e0 con 0x7fc8e8154d00 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8eca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e806c2f0 con 0x7fc8e80c42e0 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8e5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e806f5b0 0x7fc8e81f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8e5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e806f5b0 0x7fc8e81f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35896/0 (socket says 172.21.15.113:35896) 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8e5e00640 1 -- 172.21.15.113:0/980923412 learned_addr learned my addr 172.21.15.113:0/980923412 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8dd400640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8e80c42e0 0x7fc8e81f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8e6800640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8e8154d00 0x7fc8e814c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8e5e00640 1 -- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8e80c42e0 msgr2=0x7fc8e81f3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8e5e00640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8e80c42e0 0x7fc8e81f3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8e5e00640 1 -- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8e8154d00 msgr2=0x7fc8e814c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8e5e00640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8e8154d00 0x7fc8e814c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8e5e00640 1 -- 172.21.15.113:0/980923412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e80b1320 con 0x7fc8e806f5b0 2024-03-04T08:20:14.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8dd400640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8e80c42e0 0x7fc8e81f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:20:14.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8e5e00640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e806f5b0 0x7fc8e81f6ee0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c3c0 tx=0x7fc8d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:14.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8dea00640 1 -- 172.21.15.113:0/980923412 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8017070 con 0x7fc8e806f5b0 2024-03-04T08:20:14.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8dea00640 1 -- 172.21.15.113:0/980923412 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d80052d0 con 0x7fc8e806f5b0 2024-03-04T08:20:14.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8eca00640 1 -- 172.21.15.113:0/980923412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e80c5df0 con 0x7fc8e806f5b0 2024-03-04T08:20:14.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8dea00640 1 -- 172.21.15.113:0/980923412 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8003840 con 0x7fc8e806f5b0 2024-03-04T08:20:14.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8eca00640 1 -- 172.21.15.113:0/980923412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e80c5830 con 0x7fc8e806f5b0 2024-03-04T08:20:14.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8eca00640 1 -- 172.21.15.113:0/980923412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e8075c00 con 0x7fc8e806f5b0 2024-03-04T08:20:14.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.168+0000 7fc8dea00640 1 -- 172.21.15.113:0/980923412 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8d801a070 con 0x7fc8e806f5b0 2024-03-04T08:20:14.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.172+0000 7fc8dea00640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc8c4066f60 0x7fc8c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:14.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.172+0000 7fc8dea00640 1 -- 172.21.15.113:0/980923412 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fc8d8084cc0 con 0x7fc8e806f5b0 2024-03-04T08:20:14.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.172+0000 7fc8dd400640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc8c4066f60 0x7fc8c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:14.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.172+0000 7fc8dd400640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc8c4066f60 0x7fc8c4069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8068570 tx=0x7fc8d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:14.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.176+0000 7fc8dea00640 1 -- 172.21.15.113:0/980923412 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc8d808b080 con 0x7fc8e806f5b0 2024-03-04T08:20:14.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 -- 172.21.15.113:0/980923412 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8e80c4e60 con 0x7fc8e806f5b0 2024-03-04T08:20:14.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8dea00640 1 -- 172.21.15.113:0/980923412 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+5822 (secure 0 0 0) 0x7fc8d8005480 con 0x7fc8e806f5b0 2024-03-04T08:20:14.368 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:14.368 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":164,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:11.190534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:resolve","state_seq":29,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839]},"id":1}]} 2024-03-04T08:20:14.368 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 164 2024-03-04T08:20:14.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 -- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc8c4066f60 msgr2=0x7fc8c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:14.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc8c4066f60 0x7fc8c4069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8068570 tx=0x7fc8d0009040 comp rx=0 tx=0).stop 2024-03-04T08:20:14.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 -- 172.21.15.113:0/980923412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e806f5b0 msgr2=0x7fc8e81f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:14.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e806f5b0 0x7fc8e81f6ee0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800c3c0 tx=0x7fc8d800c880 comp rx=0 tx=0).stop 2024-03-04T08:20:14.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 -- 172.21.15.113:0/980923412 shutdown_connections 2024-03-04T08:20:14.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8e8154d00 0x7fc8e814c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:14.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8e80c42e0 0x7fc8e81f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:14.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc8c4066f60 0x7fc8c4069420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:14.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 --2- 172.21.15.113:0/980923412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8e806f5b0 0x7fc8e81f6ee0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:14.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 -- 172.21.15.113:0/980923412 >> 172.21.15.113:0/980923412 conn(0x7fc8e8066db0 msgr2=0x7fc8e80c3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:14.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 -- 172.21.15.113:0/980923412 shutdown_connections 2024-03-04T08:20:14.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:14.372+0000 7fc8eca00640 1 -- 172.21.15.113:0/980923412 wait complete. 2024-03-04T08:20:14.390 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:20:16.391 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:16.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.548+0000 7fa105400640 1 Processor -- start 2024-03-04T08:20:16.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa105400640 1 -- start start 2024-03-04T08:20:16.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa105400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa1001527f0 0x7fa100070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:16.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa105400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1000c8200 0x7fa100072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:16.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa105400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1001591f0 con 0x7fa100152bd0 2024-03-04T08:20:16.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa105400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa100071df0 con 0x7fa1001527f0 2024-03-04T08:20:16.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa105400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1001597b0 con 0x7fa1000c8200 2024-03-04T08:20:16.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa0f5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1000c8200 0x7fa100072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:16.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa0fe800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa1001527f0 0x7fa100070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:16.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa0ff200640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fa100152bd0 0x7fa1001587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:56994/0 (socket says 172.21.15.113:56994) 2024-03-04T08:20:16.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa0fe800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa1001527f0 0x7fa100070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:48338/0 (socket says 172.21.15.113:48338) 2024-03-04T08:20:16.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa0ff200640 1 -- 172.21.15.113:0/1370435311 learned_addr learned my addr 172.21.15.113:0/1370435311 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:16.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa0f5e00640 1 -- 172.21.15.113:0/1370435311 >> v2:172.21.15.136:3300/0 conn(0x7fa1001527f0 msgr2=0x7fa100070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:16.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa0f5e00640 1 --2- 172.21.15.113:0/1370435311 >> v2:172.21.15.136:3300/0 conn(0x7fa1001527f0 0x7fa100070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:16.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa0f5e00640 1 -- 172.21.15.113:0/1370435311 >> v1:172.21.15.136:6789/0 conn(0x7fa100152bd0 legacy=0x7fa1001587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:16.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.552+0000 7fa0f5e00640 1 -- 172.21.15.113:0/1370435311 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1001594d0 con 0x7fa1000c8200 2024-03-04T08:20:16.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa0f5e00640 1 --2- 172.21.15.113:0/1370435311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1000c8200 0x7fa100072280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80097c0 tx=0x7fa0e801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd62d8709357d73e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:16.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa0fde00640 1 -- 172.21.15.113:0/1370435311 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80b1070 con 0x7fa1000c8200 2024-03-04T08:20:16.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa0fde00640 1 -- 172.21.15.113:0/1370435311 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e80ac410 con 0x7fa1000c8200 2024-03-04T08:20:16.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa0fde00640 1 -- 172.21.15.113:0/1370435311 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80ac7e0 con 0x7fa1000c8200 2024-03-04T08:20:16.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa105400640 1 -- 172.21.15.113:0/1370435311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1000c8200 msgr2=0x7fa100072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:16.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa105400640 1 --2- 172.21.15.113:0/1370435311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1000c8200 0x7fa100072280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80097c0 tx=0x7fa0e801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:20:16.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa105400640 1 -- 172.21.15.113:0/1370435311 shutdown_connections 2024-03-04T08:20:16.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa105400640 1 --2- 172.21.15.113:0/1370435311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1000c8200 0x7fa100072280 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:16.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa105400640 1 --2- 172.21.15.113:0/1370435311 >> v2:172.21.15.136:3300/0 conn(0x7fa1001527f0 0x7fa100070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:16.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa105400640 1 -- 172.21.15.113:0/1370435311 >> 172.21.15.113:0/1370435311 conn(0x7fa10006e820 msgr2=0x7fa10006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:16.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa105400640 1 -- 172.21.15.113:0/1370435311 shutdown_connections 2024-03-04T08:20:16.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.556+0000 7fa105400640 1 -- 172.21.15.113:0/1370435311 wait complete. 2024-03-04T08:20:16.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa105400640 1 Processor -- start 2024-03-04T08:20:16.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa105400640 1 -- start start 2024-03-04T08:20:16.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa105400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1000c8200 0x7fa1001f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa105400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1001527f0 0x7fa100150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa105400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa100152bd0 0x7fa1000b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa105400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa100165160 con 0x7fa1000c8200 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa105400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1000727c0 con 0x7fa100152bd0 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa105400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1001652e0 con 0x7fa1001527f0 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0fe800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1000c8200 0x7fa1001f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0fe800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1000c8200 0x7fa1001f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35900/0 (socket says 172.21.15.113:35900) 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0fe800640 1 -- 172.21.15.113:0/342636045 learned_addr learned my addr 172.21.15.113:0/342636045 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0f5e00640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1001527f0 0x7fa100150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0ff200640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa100152bd0 0x7fa1000b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0fe800640 1 -- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1001527f0 msgr2=0x7fa100150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0fe800640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1001527f0 0x7fa100150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:16.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0fe800640 1 -- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa100152bd0 msgr2=0x7fa1000b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:16.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0fe800640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa100152bd0 0x7fa1000b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:16.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0fe800640 1 -- 172.21.15.113:0/342636045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1001f76c0 con 0x7fa1000c8200 2024-03-04T08:20:16.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0fe800640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1000c8200 0x7fa1001f4190 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000e960 tx=0x7fa0f000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:16.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0f7400640 1 -- 172.21.15.113:0/342636045 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0017070 con 0x7fa1000c8200 2024-03-04T08:20:16.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0f7400640 1 -- 172.21.15.113:0/342636045 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f000ca00 con 0x7fa1000c8200 2024-03-04T08:20:16.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa0f7400640 1 -- 172.21.15.113:0/342636045 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000cdb0 con 0x7fa1000c8200 2024-03-04T08:20:16.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa105400640 1 -- 172.21.15.113:0/342636045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1001f36b0 con 0x7fa1000c8200 2024-03-04T08:20:16.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.560+0000 7fa105400640 1 -- 172.21.15.113:0/342636045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1000685e0 con 0x7fa1000c8200 2024-03-04T08:20:16.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.564+0000 7fa105400640 1 -- 172.21.15.113:0/342636045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa100075c00 con 0x7fa1000c8200 2024-03-04T08:20:16.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.564+0000 7fa0f7400640 1 -- 172.21.15.113:0/342636045 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0f001a070 con 0x7fa1000c8200 2024-03-04T08:20:16.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.564+0000 7fa0f7400640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0dc066f60 0x7fa0dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:16.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.564+0000 7fa0f7400640 1 -- 172.21.15.113:0/342636045 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fa0f00859d0 con 0x7fa1000c8200 2024-03-04T08:20:16.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.564+0000 7fa0f5e00640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0dc066f60 0x7fa0dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:16.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.564+0000 7fa0f5e00640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0dc066f60 0x7fa0dc069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80ae000 tx=0x7fa0e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:16.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.568+0000 7fa0f7400640 1 -- 172.21.15.113:0/342636045 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa0f0055980 con 0x7fa1000c8200 2024-03-04T08:20:16.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 -- 172.21.15.113:0/342636045 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1000c79d0 con 0x7fa1000c8200 2024-03-04T08:20:16.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa0f7400640 1 -- 172.21.15.113:0/342636045 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+5822 (secure 0 0 0) 0x7fa0f0059a20 con 0x7fa1000c8200 2024-03-04T08:20:16.952 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:16.953 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":164,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:11.190534+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:resolve","state_seq":29,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839]},"id":1}]} 2024-03-04T08:20:16.953 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 164 2024-03-04T08:20:16.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 -- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0dc066f60 msgr2=0x7fa0dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0dc066f60 0x7fa0dc069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80ae000 tx=0x7fa0e8008040 comp rx=0 tx=0).stop 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 -- 172.21.15.113:0/342636045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1000c8200 msgr2=0x7fa1001f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1000c8200 0x7fa1001f4190 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000e960 tx=0x7fa0f000ee20 comp rx=0 tx=0).stop 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 -- 172.21.15.113:0/342636045 shutdown_connections 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa100152bd0 0x7fa1000b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa1001527f0 0x7fa100150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa0dc066f60 0x7fa0dc069420 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 --2- 172.21.15.113:0/342636045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa1000c8200 0x7fa1001f4190 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 -- 172.21.15.113:0/342636045 >> 172.21.15.113:0/342636045 conn(0x7fa10006e820 msgr2=0x7fa100070b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 -- 172.21.15.113:0/342636045 shutdown_connections 2024-03-04T08:20:16.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:16.956+0000 7fa105400640 1 -- 172.21.15.113:0/342636045 wait complete. 2024-03-04T08:20:16.974 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:20:18.974 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.120+0000 7f6553000640 1 Processor -- start 2024-03-04T08:20:19.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.120+0000 7f6553000640 1 -- start start 2024-03-04T08:20:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.120+0000 7f6553000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 0x7f654c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.120+0000 7f6553000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654c06c280 con 0x7f654c0c78e0 2024-03-04T08:20:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.120+0000 7f6553000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654c06ce30 con 0x7f654c154eb0 2024-03-04T08:20:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.120+0000 7f6553000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654c0c44e0 con 0x7f654c0c7240 2024-03-04T08:20:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.120+0000 7f6551000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f654c0c78e0 0x7f654c162090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58148/0 (socket says 172.21.15.113:58148) 2024-03-04T08:20:19.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.120+0000 7f6551000640 1 -- 172.21.15.113:0/2265073110 learned_addr learned my addr 172.21.15.113:0/2265073110 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:19.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.120+0000 7f6543e00640 1 --2- 172.21.15.113:0/2265073110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 0x7f654c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:19.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f654b400640 1 -- 172.21.15.113:0/2265073110 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90191977 0 0) 0x7f654c06c280 con 0x7f654c0c78e0 2024-03-04T08:20:19.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f654b400640 1 -- 172.21.15.113:0/2265073110 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6530003530 con 0x7f654c0c78e0 2024-03-04T08:20:19.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f654b400640 1 -- 172.21.15.113:0/2265073110 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6258804 0 0) 0x7f654c06ce30 con 0x7f654c154eb0 2024-03-04T08:20:19.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f654b400640 1 -- 172.21.15.113:0/2265073110 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f654c06c280 con 0x7f654c154eb0 2024-03-04T08:20:19.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f6543e00640 1 -- 172.21.15.113:0/2265073110 >> v1:172.21.15.136:6789/0 conn(0x7f654c154eb0 legacy=0x7f654c0ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:19.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f6543e00640 1 -- 172.21.15.113:0/2265073110 >> v1:172.21.15.113:6789/0 conn(0x7f654c0c78e0 legacy=0x7f654c162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:19.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f654b400640 1 -- 172.21.15.113:0/2265073110 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2562987418 0 0) 0x7f6530003530 con 0x7f654c0c78e0 2024-03-04T08:20:19.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f6543e00640 1 -- 172.21.15.113:0/2265073110 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c0c4730 con 0x7f654c0c7240 2024-03-04T08:20:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f6543e00640 1 --2- 172.21.15.113:0/2265073110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 0x7f654c15b1f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f654c0c4660 tx=0x7f653401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc85b847abf78f81 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f654b400640 1 -- 172.21.15.113:0/2265073110 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65340b0070 con 0x7f654c0c7240 2024-03-04T08:20:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f654b400640 1 -- 172.21.15.113:0/2265073110 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65340ab400 con 0x7f654c0c7240 2024-03-04T08:20:19.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.124+0000 7f654b400640 1 -- 172.21.15.113:0/2265073110 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544001a00 con 0x7f654c0c7240 2024-03-04T08:20:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 -- 172.21.15.113:0/2265073110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 msgr2=0x7f654c15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 --2- 172.21.15.113:0/2265073110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 0x7f654c15b1f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f654c0c4660 tx=0x7f653401c3e0 comp rx=0 tx=0).stop 2024-03-04T08:20:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 -- 172.21.15.113:0/2265073110 shutdown_connections 2024-03-04T08:20:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 --2- 172.21.15.113:0/2265073110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 0x7f654c15b1f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 -- 172.21.15.113:0/2265073110 >> 172.21.15.113:0/2265073110 conn(0x7f654c071d20 msgr2=0x7f654c072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 -- 172.21.15.113:0/2265073110 shutdown_connections 2024-03-04T08:20:19.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 -- 172.21.15.113:0/2265073110 wait complete. 2024-03-04T08:20:19.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 Processor -- start 2024-03-04T08:20:19.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 -- start start 2024-03-04T08:20:19.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 0x7f654c0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c0c78e0 0x7f654c1f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f654c154eb0 0x7f654c1f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c155290 con 0x7f654c154eb0 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c15ad60 con 0x7f654c0c78e0 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6553000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654c0cd680 con 0x7f654c0c7240 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6551000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f654c154eb0 0x7f654c1f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6551000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f654c154eb0 0x7f654c1f4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51114/0 (socket says 172.21.15.113:51114) 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6551000640 1 -- 172.21.15.113:0/1739547600 learned_addr learned my addr 172.21.15.113:0/1739547600 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f654be00640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 0x7f654c0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.128+0000 7f6543e00640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c0c78e0 0x7f654c1f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6551000640 1 -- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 msgr2=0x7f654c0c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6551000640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 0x7f654c0c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:19.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6551000640 1 -- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c0c78e0 msgr2=0x7f654c1f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:19.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6551000640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c0c78e0 0x7f654c1f3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:19.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6551000640 1 -- 172.21.15.113:0/1739547600 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c14dc60 con 0x7f654c154eb0 2024-03-04T08:20:19.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6551000640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f654c154eb0 0x7f654c1f4180 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f654400ccc0 tx=0x7f654400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6548c00640 1 -- 172.21.15.113:0/1739547600 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544012400 con 0x7f654c154eb0 2024-03-04T08:20:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6548c00640 1 -- 172.21.15.113:0/1739547600 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65440125a0 con 0x7f654c154eb0 2024-03-04T08:20:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6553000640 1 -- 172.21.15.113:0/1739547600 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654c14df40 con 0x7f654c154eb0 2024-03-04T08:20:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6548c00640 1 -- 172.21.15.113:0/1739547600 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544012990 con 0x7f654c154eb0 2024-03-04T08:20:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6553000640 1 -- 172.21.15.113:0/1739547600 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654c15a190 con 0x7f654c154eb0 2024-03-04T08:20:19.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6553000640 1 -- 172.21.15.113:0/1739547600 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654c075c00 con 0x7f654c154eb0 2024-03-04T08:20:19.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.132+0000 7f6548c00640 1 -- 172.21.15.113:0/1739547600 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6544012b30 con 0x7f654c154eb0 2024-03-04T08:20:19.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.136+0000 7f6548c00640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6528066f90 0x7f6528069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:19.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.136+0000 7f6548c00640 1 -- 172.21.15.113:0/1739547600 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f6544086150 con 0x7f654c154eb0 2024-03-04T08:20:19.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.136+0000 7f654be00640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6528066f90 0x7f6528069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:19.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.136+0000 7f654be00640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6528066f90 0x7f6528069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f653c0011f0 tx=0x7f653c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:19.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.140+0000 7f6548c00640 1 -- 172.21.15.113:0/1739547600 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6544012e20 con 0x7f654c154eb0 2024-03-04T08:20:19.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.332+0000 7f6553000640 1 -- 172.21.15.113:0/1739547600 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f654c1546b0 con 0x7f654c154eb0 2024-03-04T08:20:19.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.332+0000 7f6548c00640 1 -- 172.21.15.113:0/1739547600 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f6544056100 con 0x7f654c154eb0 2024-03-04T08:20:19.330 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:19.330 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":165,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:17.080433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:20:19.331 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 165 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 -- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6528066f90 msgr2=0x7f6528069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6528066f90 0x7f6528069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f653c0011f0 tx=0x7f653c009040 comp rx=0 tx=0).stop 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 -- 172.21.15.113:0/1739547600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f654c154eb0 msgr2=0x7f654c1f4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f654c154eb0 0x7f654c1f4180 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f654400ccc0 tx=0x7f654400a430 comp rx=0 tx=0).stop 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 -- 172.21.15.113:0/1739547600 shutdown_connections 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6528066f90 0x7f6528069450 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f654c154eb0 0x7f654c1f4180 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f654c0c78e0 0x7f654c1f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 --2- 172.21.15.113:0/1739547600 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f654c0c7240 0x7f654c0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 -- 172.21.15.113:0/1739547600 >> 172.21.15.113:0/1739547600 conn(0x7f654c071d20 msgr2=0x7f654c0ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 -- 172.21.15.113:0/1739547600 shutdown_connections 2024-03-04T08:20:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:19.336+0000 7f6553000640 1 -- 172.21.15.113:0/1739547600 wait complete. 2024-03-04T08:20:19.355 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 165, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 165, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8253": { "addr": "172.21.15.113:6836/893414524", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 893414524, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 893414524, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8253, "incarnation": 163, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:reconnect", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 80, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:20:17.080433+0000", "qdb_cluster": [ 7839, 8032 ], "qdb_leader": 7839, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 8032, "mds_2": 8253 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:20:21.357 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:21.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f04800640 1 Processor -- start 2024-03-04T08:20:21.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f04800640 1 -- start start 2024-03-04T08:20:21.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f04800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1efc0c8200 0x7f1efc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:21.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f04800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1efc152bd0 0x7f1efc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:21.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f04800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc1591f0 con 0x7f1efc1527f0 2024-03-04T08:20:21.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f04800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc071df0 con 0x7f1efc0c8200 2024-03-04T08:20:21.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f04800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc1597b0 con 0x7f1efc152bd0 2024-03-04T08:20:21.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f01400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1efc0c8200 0x7f1efc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:21.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f01400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1efc0c8200 0x7f1efc070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51120/0 (socket says 172.21.15.113:51120) 2024-03-04T08:20:21.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f02800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1efc152bd0 0x7f1efc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:21.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f01400640 1 -- 172.21.15.113:0/188247871 learned_addr learned my addr 172.21.15.113:0/188247871 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:21.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f01400640 1 -- 172.21.15.113:0/188247871 >> v2:172.21.15.136:3300/0 conn(0x7f1efc152bd0 msgr2=0x7f1efc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:21.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f01400640 1 --2- 172.21.15.113:0/188247871 >> v2:172.21.15.136:3300/0 conn(0x7f1efc152bd0 0x7f1efc1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:21.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f01400640 1 -- 172.21.15.113:0/188247871 >> v1:172.21.15.136:6789/0 conn(0x7f1efc1527f0 legacy=0x7f1efc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:21.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.500+0000 7f1f01400640 1 -- 172.21.15.113:0/188247871 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc1594d0 con 0x7f1efc0c8200 2024-03-04T08:20:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f01400640 1 --2- 172.21.15.113:0/188247871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 0x7f1efc070d40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0712b0 tx=0x7f1eec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918e958009ba17d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f00a00640 1 -- 172.21.15.113:0/188247871 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0ab430 con 0x7f1efc0c8200 2024-03-04T08:20:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f00a00640 1 -- 172.21.15.113:0/188247871 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec0b0070 con 0x7f1efc0c8200 2024-03-04T08:20:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f00a00640 1 -- 172.21.15.113:0/188247871 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0ab7f0 con 0x7f1efc0c8200 2024-03-04T08:20:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f04800640 1 -- 172.21.15.113:0/188247871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 msgr2=0x7f1efc070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f04800640 1 --2- 172.21.15.113:0/188247871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 0x7f1efc070d40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0712b0 tx=0x7f1eec01c520 comp rx=0 tx=0).stop 2024-03-04T08:20:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f04800640 1 -- 172.21.15.113:0/188247871 shutdown_connections 2024-03-04T08:20:21.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f04800640 1 --2- 172.21.15.113:0/188247871 >> v2:172.21.15.136:3300/0 conn(0x7f1efc152bd0 0x7f1efc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:21.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f04800640 1 --2- 172.21.15.113:0/188247871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 0x7f1efc070d40 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:21.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f04800640 1 -- 172.21.15.113:0/188247871 >> 172.21.15.113:0/188247871 conn(0x7f1efc06e820 msgr2=0x7f1efc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:21.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f04800640 1 -- 172.21.15.113:0/188247871 shutdown_connections 2024-03-04T08:20:21.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.504+0000 7f1f04800640 1 -- 172.21.15.113:0/188247871 wait complete. 2024-03-04T08:20:21.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.508+0000 7f1f04800640 1 Processor -- start 2024-03-04T08:20:21.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.508+0000 7f1f04800640 1 -- start start 2024-03-04T08:20:21.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.508+0000 7f1f04800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 0x7f1efc0b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:21.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.508+0000 7f1f04800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1efc1527f0 0x7f1efc1ef770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:21.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.508+0000 7f1f04800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1efc152bd0 0x7f1efc0b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:21.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f04800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc1654f0 con 0x7f1efc0c8200 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f04800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc165370 con 0x7f1efc152bd0 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f04800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc165670 con 0x7f1efc1527f0 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f01e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 0x7f1efc0b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f01e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 0x7f1efc0b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51130/0 (socket says 172.21.15.113:51130) 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f01e00640 1 -- 172.21.15.113:0/899873291 learned_addr learned my addr 172.21.15.113:0/899873291 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f01400640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1efc1527f0 0x7f1efc1ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f02800640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1efc152bd0 0x7f1efc0b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f01e00640 1 -- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1efc1527f0 msgr2=0x7f1efc1ef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f01e00640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1efc1527f0 0x7f1efc1ef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f01e00640 1 -- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1efc152bd0 msgr2=0x7f1efc0b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f01e00640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1efc152bd0 0x7f1efc0b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f01e00640 1 -- 172.21.15.113:0/899873291 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc0af980 con 0x7f1efc0c8200 2024-03-04T08:20:21.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f01e00640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 0x7f1efc0b0e40 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000c450 tx=0x7f1ef000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:21.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1ef6000640 1 -- 172.21.15.113:0/899873291 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0016020 con 0x7f1efc0c8200 2024-03-04T08:20:21.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1ef6000640 1 -- 172.21.15.113:0/899873291 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef00052d0 con 0x7f1efc0c8200 2024-03-04T08:20:21.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f04800640 1 -- 172.21.15.113:0/899873291 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc0c3e00 con 0x7f1efc0c8200 2024-03-04T08:20:21.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1ef6000640 1 -- 172.21.15.113:0/899873291 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0007260 con 0x7f1efc0c8200 2024-03-04T08:20:21.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f04800640 1 -- 172.21.15.113:0/899873291 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc0afc20 con 0x7f1efc0c8200 2024-03-04T08:20:21.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.512+0000 7f1f04800640 1 -- 172.21.15.113:0/899873291 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc075c00 con 0x7f1efc0c8200 2024-03-04T08:20:21.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.516+0000 7f1ef6000640 1 -- 172.21.15.113:0/899873291 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ef0007400 con 0x7f1efc0c8200 2024-03-04T08:20:21.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.516+0000 7f1ef6000640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1ee0066f60 0x7f1ee0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:21.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.516+0000 7f1ef6000640 1 -- 172.21.15.113:0/899873291 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f1ef0085160 con 0x7f1efc0c8200 2024-03-04T08:20:21.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.516+0000 7f1f01400640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1ee0066f60 0x7f1ee0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:21.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.516+0000 7f1f01400640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1ee0066f60 0x7f1ee0069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0af000 tx=0x7f1eec0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:21.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.520+0000 7f1ef6000640 1 -- 172.21.15.113:0/899873291 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1ef0020020 con 0x7f1efc0c8200 2024-03-04T08:20:21.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 -- 172.21.15.113:0/899873291 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1efc0c79d0 con 0x7f1efc0c8200 2024-03-04T08:20:21.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1ef6000640 1 -- 172.21.15.113:0/899873291 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) v1 ==== 95+0+5824 (secure 0 0 0) 0x7f1ef0005480 con 0x7f1efc0c8200 2024-03-04T08:20:21.709 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:21.709 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":165,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:17.080433+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:20:21.709 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 165 2024-03-04T08:20:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 -- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1ee0066f60 msgr2=0x7f1ee0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1ee0066f60 0x7f1ee0069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1eec0af000 tx=0x7f1eec0a7000 comp rx=0 tx=0).stop 2024-03-04T08:20:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 -- 172.21.15.113:0/899873291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 msgr2=0x7f1efc0b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 0x7f1efc0b0e40 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000c450 tx=0x7f1ef000c910 comp rx=0 tx=0).stop 2024-03-04T08:20:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 -- 172.21.15.113:0/899873291 shutdown_connections 2024-03-04T08:20:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1efc152bd0 0x7f1efc0b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1efc1527f0 0x7f1efc1ef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1ee0066f60 0x7f1ee0069420 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:21.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 --2- 172.21.15.113:0/899873291 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0c8200 0x7f1efc0b0e40 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:21.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.712+0000 7f1f04800640 1 -- 172.21.15.113:0/899873291 >> 172.21.15.113:0/899873291 conn(0x7f1efc06e820 msgr2=0x7f1efc158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:21.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.716+0000 7f1f04800640 1 -- 172.21.15.113:0/899873291 shutdown_connections 2024-03-04T08:20:21.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:21.716+0000 7f1f04800640 1 -- 172.21.15.113:0/899873291 wait complete. 2024-03-04T08:20:21.729 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:20:23.731 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:23.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb096800640 1 Processor -- start 2024-03-04T08:20:23.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb096800640 1 -- start start 2024-03-04T08:20:23.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb096800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb090152bd0 0x7fb090070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:23.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb096800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0901591f0 con 0x7fb0901527f0 2024-03-04T08:20:23.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb096800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0901594d0 con 0x7fb0900c8200 2024-03-04T08:20:23.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb096800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb090071df0 con 0x7fb090152bd0 2024-03-04T08:20:23.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb08fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb090152bd0 0x7fb090070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:23.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb08fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb090152bd0 0x7fb090070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51146/0 (socket says 172.21.15.113:51146) 2024-03-04T08:20:23.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb08fe00640 1 -- 172.21.15.113:0/679378650 learned_addr learned my addr 172.21.15.113:0/679378650 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:23.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb08f400640 1 --1- 172.21.15.113:0/679378650 >> v1:172.21.15.113:6789/0 conn(0x7fb0901527f0 0x7fb090072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58150/0 (socket says 172.21.15.113:58150) 2024-03-04T08:20:23.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb08fe00640 1 -- 172.21.15.113:0/679378650 >> v1:172.21.15.136:6789/0 conn(0x7fb0900c8200 legacy=0x7fb0901587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:23.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb08fe00640 1 -- 172.21.15.113:0/679378650 >> v1:172.21.15.113:6789/0 conn(0x7fb0901527f0 legacy=0x7fb090072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:23.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.876+0000 7fb08fe00640 1 -- 172.21.15.113:0/679378650 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0900717a0 con 0x7fb090152bd0 2024-03-04T08:20:23.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb08fe00640 1 --2- 172.21.15.113:0/679378650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb090152bd0 0x7fb090070d40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb088009870 tx=0x7fb08801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb276aabc4fead7a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:23.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb08ea00640 1 -- 172.21.15.113:0/679378650 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0880b2070 con 0x7fb090152bd0 2024-03-04T08:20:23.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb08ea00640 1 -- 172.21.15.113:0/679378650 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0880a9070 con 0x7fb090152bd0 2024-03-04T08:20:23.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb08ea00640 1 -- 172.21.15.113:0/679378650 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0880ad880 con 0x7fb090152bd0 2024-03-04T08:20:23.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb096800640 1 -- 172.21.15.113:0/679378650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb090152bd0 msgr2=0x7fb090070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:23.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb096800640 1 --2- 172.21.15.113:0/679378650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb090152bd0 0x7fb090070d40 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb088009870 tx=0x7fb08801c560 comp rx=0 tx=0).stop 2024-03-04T08:20:23.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb096800640 1 -- 172.21.15.113:0/679378650 shutdown_connections 2024-03-04T08:20:23.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb096800640 1 --2- 172.21.15.113:0/679378650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb090152bd0 0x7fb090070d40 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:23.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb096800640 1 -- 172.21.15.113:0/679378650 >> 172.21.15.113:0/679378650 conn(0x7fb09006e820 msgr2=0x7fb09006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:23.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb096800640 1 -- 172.21.15.113:0/679378650 shutdown_connections 2024-03-04T08:20:23.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.880+0000 7fb096800640 1 -- 172.21.15.113:0/679378650 wait complete. 2024-03-04T08:20:23.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb096800640 1 Processor -- start 2024-03-04T08:20:23.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb096800640 1 -- start start 2024-03-04T08:20:23.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb096800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0900c8200 0x7fb090163750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:23.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb096800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0901527f0 0x7fb090147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:23.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb096800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090152bd0 0x7fb090162a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:23.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb096800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb090166500 con 0x7fb0900c8200 2024-03-04T08:20:23.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb096800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb090166380 con 0x7fb090152bd0 2024-03-04T08:20:23.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb096800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb090166680 con 0x7fb0901527f0 2024-03-04T08:20:23.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb08f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0900c8200 0x7fb090163750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:23.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb08f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0900c8200 0x7fb090163750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51154/0 (socket says 172.21.15.113:51154) 2024-03-04T08:20:23.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb08f400640 1 -- 172.21.15.113:0/4134397395 learned_addr learned my addr 172.21.15.113:0/4134397395 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:23.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb08fe00640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090152bd0 0x7fb090162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:23.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb086a00640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0901527f0 0x7fb090147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:23.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb08f400640 1 -- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0901527f0 msgr2=0x7fb090147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:23.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb08f400640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0901527f0 0x7fb090147e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:23.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb08f400640 1 -- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090152bd0 msgr2=0x7fb090162a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:23.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb08f400640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090152bd0 0x7fb090162a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:23.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb08f400640 1 -- 172.21.15.113:0/4134397395 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090159a30 con 0x7fb0900c8200 2024-03-04T08:20:23.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb08f400640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0900c8200 0x7fb090163750 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c4c0 tx=0x7fb08000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:23.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb087e00640 1 -- 172.21.15.113:0/4134397395 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080018070 con 0x7fb0900c8200 2024-03-04T08:20:23.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb087e00640 1 -- 172.21.15.113:0/4134397395 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080007480 con 0x7fb0900c8200 2024-03-04T08:20:23.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb087e00640 1 -- 172.21.15.113:0/4134397395 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0800047f0 con 0x7fb0900c8200 2024-03-04T08:20:23.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb096800640 1 -- 172.21.15.113:0/4134397395 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09015a540 con 0x7fb0900c8200 2024-03-04T08:20:23.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.884+0000 7fb096800640 1 -- 172.21.15.113:0/4134397395 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb090159ca0 con 0x7fb0900c8200 2024-03-04T08:20:23.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.888+0000 7fb096800640 1 -- 172.21.15.113:0/4134397395 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb090075c00 con 0x7fb0900c8200 2024-03-04T08:20:23.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.888+0000 7fb087e00640 1 -- 172.21.15.113:0/4134397395 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb080004990 con 0x7fb0900c8200 2024-03-04T08:20:23.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.888+0000 7fb087e00640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb06c066f60 0x7fb06c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:23.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.888+0000 7fb087e00640 1 -- 172.21.15.113:0/4134397395 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fb080084b10 con 0x7fb0900c8200 2024-03-04T08:20:23.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.888+0000 7fb086a00640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb06c066f60 0x7fb06c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:23.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.888+0000 7fb086a00640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb06c066f60 0x7fb06c069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb0900cc0b0 tx=0x7fb078009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:23.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:23.896+0000 7fb087e00640 1 -- 172.21.15.113:0/4134397395 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb09015a540 con 0x7fb0900c8200 2024-03-04T08:20:24.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 -- 172.21.15.113:0/4134397395 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb090161cb0 con 0x7fb0900c8200 2024-03-04T08:20:24.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb087e00640 1 -- 172.21.15.113:0/4134397395 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+5821 (secure 0 0 0) 0x7fb080054b50 con 0x7fb0900c8200 2024-03-04T08:20:24.088 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:24.089 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":166,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:22.109423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839]},"id":1}]} 2024-03-04T08:20:24.089 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 166 2024-03-04T08:20:24.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 -- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb06c066f60 msgr2=0x7fb06c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:24.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb06c066f60 0x7fb06c069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb0900cc0b0 tx=0x7fb078009040 comp rx=0 tx=0).stop 2024-03-04T08:20:24.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 -- 172.21.15.113:0/4134397395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0900c8200 msgr2=0x7fb090163750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:24.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0900c8200 0x7fb090163750 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb08000c4c0 tx=0x7fb08000c980 comp rx=0 tx=0).stop 2024-03-04T08:20:24.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 -- 172.21.15.113:0/4134397395 shutdown_connections 2024-03-04T08:20:24.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb090152bd0 0x7fb090162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:24.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0901527f0 0x7fb090147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:24.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb06c066f60 0x7fb06c069420 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:24.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 --2- 172.21.15.113:0/4134397395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0900c8200 0x7fb090163750 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:24.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.092+0000 7fb096800640 1 -- 172.21.15.113:0/4134397395 >> 172.21.15.113:0/4134397395 conn(0x7fb09006e820 msgr2=0x7fb09014a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:24.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.096+0000 7fb096800640 1 -- 172.21.15.113:0/4134397395 shutdown_connections 2024-03-04T08:20:24.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:24.096+0000 7fb096800640 1 -- 172.21.15.113:0/4134397395 wait complete. 2024-03-04T08:20:24.110 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:20:26.111 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:26.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f9270400640 1 Processor -- start 2024-03-04T08:20:26.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f9270400640 1 -- start start 2024-03-04T08:20:26.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f9270400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268155100 0x7f92680c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:26.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f9270400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f926806b060 0x7f92680cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:26.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f9270400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92680cd600 0x7f926815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:26.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f9270400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92680c8890 con 0x7f92680cd600 2024-03-04T08:20:26.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f9270400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926806b4a0 con 0x7f926806b060 2024-03-04T08:20:26.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f9270400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926815efd0 con 0x7f9268155100 2024-03-04T08:20:26.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926e400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f92680cd600 0x7f926815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:26.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268155100 0x7f92680c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:26.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268155100 0x7f92680c08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:51538/0 (socket says 172.21.15.113:51538) 2024-03-04T08:20:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926da00640 1 -- 172.21.15.113:0/479021404 learned_addr learned my addr 172.21.15.113:0/479021404 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926d000640 1 --2- 172.21.15.113:0/479021404 >> v2:172.21.15.136:3300/0 conn(0x7f926806b060 0x7f92680cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926da00640 1 -- 172.21.15.113:0/479021404 >> v2:172.21.15.136:3300/0 conn(0x7f926806b060 msgr2=0x7f92680cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926da00640 1 --2- 172.21.15.113:0/479021404 >> v2:172.21.15.136:3300/0 conn(0x7f926806b060 0x7f92680cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926da00640 1 -- 172.21.15.113:0/479021404 >> v2:172.21.15.113:3300/0 conn(0x7f92680cd600 msgr2=0x7f926815ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926da00640 1 --2- 172.21.15.113:0/479021404 >> v2:172.21.15.113:3300/0 conn(0x7f92680cd600 0x7f926815ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926da00640 1 -- 172.21.15.113:0/479021404 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92680c7ef0 con 0x7f9268155100 2024-03-04T08:20:26.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.256+0000 7f926e400640 1 --2- 172.21.15.113:0/479021404 >> v2:172.21.15.113:3300/0 conn(0x7f92680cd600 0x7f926815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:20:26.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f926da00640 1 --2- 172.21.15.113:0/479021404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268155100 0x7f92680c08d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f92680cac40 tx=0x7f925c01c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90d941c40b76aec7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:26.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9267e00640 1 -- 172.21.15.113:0/479021404 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0b0070 con 0x7f9268155100 2024-03-04T08:20:26.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9267e00640 1 -- 172.21.15.113:0/479021404 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925c0ab600 con 0x7f9268155100 2024-03-04T08:20:26.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9267e00640 1 -- 172.21.15.113:0/479021404 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0ab9d0 con 0x7f9268155100 2024-03-04T08:20:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9270400640 1 -- 172.21.15.113:0/479021404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268155100 msgr2=0x7f92680c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9270400640 1 --2- 172.21.15.113:0/479021404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268155100 0x7f92680c08d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f92680cac40 tx=0x7f925c01c520 comp rx=0 tx=0).stop 2024-03-04T08:20:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9270400640 1 -- 172.21.15.113:0/479021404 shutdown_connections 2024-03-04T08:20:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9270400640 1 --2- 172.21.15.113:0/479021404 >> v2:172.21.15.113:3300/0 conn(0x7f92680cd600 0x7f926815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9270400640 1 --2- 172.21.15.113:0/479021404 >> v2:172.21.15.136:3300/0 conn(0x7f926806b060 0x7f92680cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9270400640 1 --2- 172.21.15.113:0/479021404 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9268155100 0x7f92680c08d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9270400640 1 -- 172.21.15.113:0/479021404 >> 172.21.15.113:0/479021404 conn(0x7f926806deb0 msgr2=0x7f926806e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:26.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9270400640 1 -- 172.21.15.113:0/479021404 shutdown_connections 2024-03-04T08:20:26.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.260+0000 7f9270400640 1 -- 172.21.15.113:0/479021404 wait complete. 2024-03-04T08:20:26.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9270400640 1 Processor -- start 2024-03-04T08:20:26.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9270400640 1 -- start start 2024-03-04T08:20:26.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9270400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f926806b060 0x7f9268151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:26.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9270400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92680cd600 0x7f92680c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:26.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9270400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9268155100 0x7f92680c3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:26.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9270400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926815ebb0 con 0x7f92680cd600 2024-03-04T08:20:26.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9270400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926815b680 con 0x7f9268155100 2024-03-04T08:20:26.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9270400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926815ed30 con 0x7f926806b060 2024-03-04T08:20:26.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92680cd600 0x7f92680c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f926806b060 0x7f9268151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92680cd600 0x7f92680c2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51174/0 (socket says 172.21.15.113:51174) 2024-03-04T08:20:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926d000640 1 -- 172.21.15.113:0/3188729736 learned_addr learned my addr 172.21.15.113:0/3188729736 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926e400640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9268155100 0x7f92680c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926d000640 1 -- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f926806b060 msgr2=0x7f9268151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926d000640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f926806b060 0x7f9268151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926d000640 1 -- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9268155100 msgr2=0x7f92680c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926d000640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9268155100 0x7f92680c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926d000640 1 -- 172.21.15.113:0/3188729736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9268151cb0 con 0x7f92680cd600 2024-03-04T08:20:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f926d000640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92680cd600 0x7f92680c2d40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f925400e8b0 tx=0x7f925400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9265600640 1 -- 172.21.15.113:0/3188729736 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254017070 con 0x7f92680cd600 2024-03-04T08:20:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9265600640 1 -- 172.21.15.113:0/3188729736 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925400c940 con 0x7f92680cd600 2024-03-04T08:20:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9265600640 1 -- 172.21.15.113:0/3188729736 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925400cd10 con 0x7f92680cd600 2024-03-04T08:20:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9270400640 1 -- 172.21.15.113:0/3188729736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9268151f90 con 0x7f92680cd600 2024-03-04T08:20:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.264+0000 7f9270400640 1 -- 172.21.15.113:0/3188729736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9268152550 con 0x7f92680cd600 2024-03-04T08:20:26.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.268+0000 7f9270400640 1 -- 172.21.15.113:0/3188729736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9268075c00 con 0x7f92680cd600 2024-03-04T08:20:26.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.268+0000 7f9265600640 1 -- 172.21.15.113:0/3188729736 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f925401a070 con 0x7f92680cd600 2024-03-04T08:20:26.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.268+0000 7f9265600640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9248066f60 0x7f9248069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:26.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.268+0000 7f9265600640 1 -- 172.21.15.113:0/3188729736 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f9254085c80 con 0x7f92680cd600 2024-03-04T08:20:26.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.268+0000 7f926da00640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9248066f60 0x7f9248069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:26.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.272+0000 7f926da00640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9248066f60 0x7f9248069420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f925c0ad000 tx=0x7f925c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:26.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.272+0000 7f9265600640 1 -- 172.21.15.113:0/3188729736 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9254055c30 con 0x7f92680cd600 2024-03-04T08:20:26.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 -- 172.21.15.113:0/3188729736 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f926815e800 con 0x7f92680cd600 2024-03-04T08:20:26.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9265600640 1 -- 172.21.15.113:0/3188729736 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+5821 (secure 0 0 0) 0x7f9254059cd0 con 0x7f92680cd600 2024-03-04T08:20:26.468 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:26.468 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":166,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":166,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:22.109423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839]},"id":1}]} 2024-03-04T08:20:26.469 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 166 2024-03-04T08:20:26.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 -- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9248066f60 msgr2=0x7f9248069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:26.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9248066f60 0x7f9248069420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f925c0ad000 tx=0x7f925c008040 comp rx=0 tx=0).stop 2024-03-04T08:20:26.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 -- 172.21.15.113:0/3188729736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92680cd600 msgr2=0x7f92680c2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:26.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92680cd600 0x7f92680c2d40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f925400e8b0 tx=0x7f925400ecb0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 -- 172.21.15.113:0/3188729736 shutdown_connections 2024-03-04T08:20:26.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9268155100 0x7f92680c3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9248066f60 0x7f9248069420 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f92680cd600 0x7f92680c2d40 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 --2- 172.21.15.113:0/3188729736 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f926806b060 0x7f9268151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:26.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 -- 172.21.15.113:0/3188729736 >> 172.21.15.113:0/3188729736 conn(0x7f926806deb0 msgr2=0x7f9268159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:26.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 -- 172.21.15.113:0/3188729736 shutdown_connections 2024-03-04T08:20:26.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:26.472+0000 7f9270400640 1 -- 172.21.15.113:0/3188729736 wait complete. 2024-03-04T08:20:26.489 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:20:28.490 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:28.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9ace00640 1 Processor -- start 2024-03-04T08:20:28.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9ace00640 1 -- start start 2024-03-04T08:20:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9ace00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8070df0 0x7fe9a80c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9ace00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a80c9c50 con 0x7fe9a806fb70 2024-03-04T08:20:28.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9ace00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a815a9a0 con 0x7fe9a8157ef0 2024-03-04T08:20:28.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9ace00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a806de30 con 0x7fe9a8070df0 2024-03-04T08:20:28.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9a7400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fe9a8157ef0 0x7fe9a815b180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:60084/0 (socket says 172.21.15.113:60084) 2024-03-04T08:20:28.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9a6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8070df0 0x7fe9a80c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:28.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9a7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe9a806fb70 0x7fe9a80cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35992/0 (socket says 172.21.15.113:35992) 2024-03-04T08:20:28.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9a7400640 1 -- 172.21.15.113:0/1244579854 learned_addr learned my addr 172.21.15.113:0/1244579854 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:28.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9a6000640 1 -- 172.21.15.113:0/1244579854 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820207981 0 0) 0x7fe9a80c9c50 con 0x7fe9a806fb70 2024-03-04T08:20:28.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.648+0000 7fe9a6000640 1 -- 172.21.15.113:0/1244579854 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe98c003530 con 0x7fe9a806fb70 2024-03-04T08:20:28.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.652+0000 7fe9a6a00640 1 -- 172.21.15.113:0/1244579854 >> v1:172.21.15.136:6789/0 conn(0x7fe9a8157ef0 legacy=0x7fe9a815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:28.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.652+0000 7fe9a6a00640 1 -- 172.21.15.113:0/1244579854 >> v1:172.21.15.113:6789/0 conn(0x7fe9a806fb70 legacy=0x7fe9a80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:28.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.652+0000 7fe9a6a00640 1 -- 172.21.15.113:0/1244579854 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a806c210 con 0x7fe9a8070df0 2024-03-04T08:20:28.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.652+0000 7fe9a6a00640 1 --2- 172.21.15.113:0/1244579854 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8070df0 0x7fe9a80c8a90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe9940097f0 tx=0x7fe99401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcce8ec3c6c7661c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:28.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9a6000640 1 -- 172.21.15.113:0/1244579854 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940ab5f0 con 0x7fe9a8070df0 2024-03-04T08:20:28.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9a6000640 1 -- 172.21.15.113:0/1244579854 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9940b0070 con 0x7fe9a8070df0 2024-03-04T08:20:28.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9a6000640 1 -- 172.21.15.113:0/1244579854 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940ab990 con 0x7fe9a8070df0 2024-03-04T08:20:28.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9ace00640 1 -- 172.21.15.113:0/1244579854 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8070df0 msgr2=0x7fe9a80c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:28.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9ace00640 1 --2- 172.21.15.113:0/1244579854 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8070df0 0x7fe9a80c8a90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe9940097f0 tx=0x7fe99401c4e0 comp rx=0 tx=0).stop 2024-03-04T08:20:28.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9ace00640 1 -- 172.21.15.113:0/1244579854 shutdown_connections 2024-03-04T08:20:28.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9ace00640 1 --2- 172.21.15.113:0/1244579854 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8070df0 0x7fe9a80c8a90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:28.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9ace00640 1 -- 172.21.15.113:0/1244579854 >> 172.21.15.113:0/1244579854 conn(0x7fe9a8072f40 msgr2=0x7fe9a80706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:28.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9ace00640 1 -- 172.21.15.113:0/1244579854 shutdown_connections 2024-03-04T08:20:28.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9ace00640 1 -- 172.21.15.113:0/1244579854 wait complete. 2024-03-04T08:20:28.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9ace00640 1 Processor -- start 2024-03-04T08:20:28.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.656+0000 7fe9ace00640 1 -- start start 2024-03-04T08:20:28.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9ace00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a806fb70 0x7fe9a8147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9ace00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8157ef0 0x7fe9a814bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9ace00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a8149010 0x7fe9a81494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9ace00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a81582d0 con 0x7fe9a806fb70 2024-03-04T08:20:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9ace00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a815b890 con 0x7fe9a8149010 2024-03-04T08:20:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9ace00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a8158450 con 0x7fe9a8157ef0 2024-03-04T08:20:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a806fb70 0x7fe9a8147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a6a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8157ef0 0x7fe9a814bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a806fb70 0x7fe9a8147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45944/0 (socket says 172.21.15.113:45944) 2024-03-04T08:20:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a7400640 1 -- 172.21.15.113:0/1771781918 learned_addr learned my addr 172.21.15.113:0/1771781918 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:28.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a7e00640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a8149010 0x7fe9a81494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:28.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a7400640 1 -- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8157ef0 msgr2=0x7fe9a814bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a7400640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8157ef0 0x7fe9a814bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a7400640 1 -- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a8149010 msgr2=0x7fe9a81494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a7400640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a8149010 0x7fe9a81494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a7400640 1 -- 172.21.15.113:0/1771781918 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a80ca130 con 0x7fe9a806fb70 2024-03-04T08:20:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9a7400640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a806fb70 0x7fe9a8147c70 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe99800c3d0 tx=0x7fe99800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe99f400640 1 -- 172.21.15.113:0/1771781918 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998017070 con 0x7fe9a806fb70 2024-03-04T08:20:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe99f400640 1 -- 172.21.15.113:0/1771781918 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe998005cf0 con 0x7fe9a806fb70 2024-03-04T08:20:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe99f400640 1 -- 172.21.15.113:0/1771781918 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9980072f0 con 0x7fe9a806fb70 2024-03-04T08:20:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9ace00640 1 -- 172.21.15.113:0/1771781918 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a81499e0 con 0x7fe9a806fb70 2024-03-04T08:20:28.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9ace00640 1 -- 172.21.15.113:0/1771781918 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a81f6b90 con 0x7fe9a806fb70 2024-03-04T08:20:28.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.660+0000 7fe9ace00640 1 -- 172.21.15.113:0/1771781918 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a81f7860 con 0x7fe9a806fb70 2024-03-04T08:20:28.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.664+0000 7fe99f400640 1 -- 172.21.15.113:0/1771781918 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe99801a070 con 0x7fe9a806fb70 2024-03-04T08:20:28.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.664+0000 7fe99f400640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe988066f60 0x7fe988069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:28.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.664+0000 7fe99f400640 1 -- 172.21.15.113:0/1771781918 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fe998084c60 con 0x7fe9a806fb70 2024-03-04T08:20:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.664+0000 7fe9a6a00640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe988066f60 0x7fe988069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:28.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.668+0000 7fe9a6a00640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe988066f60 0x7fe988069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe9940af000 tx=0x7fe9940b6040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:28.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.672+0000 7fe99f400640 1 -- 172.21.15.113:0/1771781918 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe998054cb0 con 0x7fe9a806fb70 2024-03-04T08:20:28.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.884+0000 7fe9ace00640 1 -- 172.21.15.113:0/1771781918 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9a8075a80 con 0x7fe9a806fb70 2024-03-04T08:20:28.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.884+0000 7fe99f400640 1 -- 172.21.15.113:0/1771781918 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 167 v167) v1 ==== 95+0+5826 (secure 0 0 0) 0x7fe998058d50 con 0x7fe9a806fb70 2024-03-04T08:20:28.882 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:28.882 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":167,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:27.135997+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:active","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032,8253]},"id":1}]} 2024-03-04T08:20:28.883 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 167 2024-03-04T08:20:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 -- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe988066f60 msgr2=0x7fe988069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:28.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe988066f60 0x7fe988069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe9940af000 tx=0x7fe9940b6040 comp rx=0 tx=0).stop 2024-03-04T08:20:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 -- 172.21.15.113:0/1771781918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a806fb70 msgr2=0x7fe9a8147c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a806fb70 0x7fe9a8147c70 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe99800c3d0 tx=0x7fe99800c890 comp rx=0 tx=0).stop 2024-03-04T08:20:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 -- 172.21.15.113:0/1771781918 shutdown_connections 2024-03-04T08:20:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a8149010 0x7fe9a81494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9a8157ef0 0x7fe9a814bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe988066f60 0x7fe988069420 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 --2- 172.21.15.113:0/1771781918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9a806fb70 0x7fe9a8147c70 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 -- 172.21.15.113:0/1771781918 >> 172.21.15.113:0/1771781918 conn(0x7fe9a8072f40 msgr2=0x7fe9a814d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 -- 172.21.15.113:0/1771781918 shutdown_connections 2024-03-04T08:20:28.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:28.888+0000 7fe9ace00640 1 -- 172.21.15.113:0/1771781918 wait complete. 2024-03-04T08:20:28.904 INFO:tasks.mds_thrash.fs.[cephfs]:mds.a has gained rank=2, replacing gid=7692 2024-03-04T08:20:28.904 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.c 2024-03-04T08:20:29.337 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T08:20:29.340+0000 7f9b59600640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-04T08:20:33.905 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.c 2024-03-04T08:20:33.906 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-04T08:20:33.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-04T08:20:33.909 INFO:tasks.ceph.mds.c:Started 2024-03-04T08:20:33.910 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:34.001 INFO:tasks.ceph.mds.c.smithi113.stdout:starting mds.c at 2024-03-04T08:20:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e5d200640 1 Processor -- start 2024-03-04T08:20:34.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e5d200640 1 -- start start 2024-03-04T08:20:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e5d200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0e58151840 0x7f0e580c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e5d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151c20 0x7f0e58157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e5d200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e58157da0 con 0x7f0e5806d140 2024-03-04T08:20:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e5d200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e580c3440 con 0x7f0e58151840 2024-03-04T08:20:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e5d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58158080 con 0x7f0e58151c20 2024-03-04T08:20:34.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e4ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151c20 0x7f0e58157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e57e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0e5806d140 0x7f0e5806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36006/0 (socket says 172.21.15.113:36006) 2024-03-04T08:20:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e57e00640 1 -- 172.21.15.113:0/2525508113 learned_addr learned my addr 172.21.15.113:0/2525508113 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e57400640 1 --2- 172.21.15.113:0/2525508113 >> v2:172.21.15.136:3300/0 conn(0x7f0e58151840 0x7f0e580c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e56a00640 1 -- 172.21.15.113:0/2525508113 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378390294 0 0) 0x7f0e58157da0 con 0x7f0e5806d140 2024-03-04T08:20:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e4ea00640 1 -- 172.21.15.113:0/2525508113 >> v2:172.21.15.136:3300/0 conn(0x7f0e58151840 msgr2=0x7f0e580c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e4ea00640 1 --2- 172.21.15.113:0/2525508113 >> v2:172.21.15.136:3300/0 conn(0x7f0e58151840 0x7f0e580c1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e4ea00640 1 -- 172.21.15.113:0/2525508113 >> v1:172.21.15.113:6789/0 conn(0x7f0e5806d140 legacy=0x7f0e5806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:34.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.096+0000 7f0e4ea00640 1 -- 172.21.15.113:0/2525508113 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e58158360 con 0x7f0e58151c20 2024-03-04T08:20:34.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.100+0000 7f0e4ea00640 1 --2- 172.21.15.113:0/2525508113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151c20 0x7f0e58157710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e400097c0 tx=0x7f0e4001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e2a68134b866d82 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:34.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.100+0000 7f0e56a00640 1 -- 172.21.15.113:0/2525508113 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e400b5070 con 0x7f0e58151c20 2024-03-04T08:20:34.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.100+0000 7f0e56a00640 1 -- 172.21.15.113:0/2525508113 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e400a8990 con 0x7f0e58151c20 2024-03-04T08:20:34.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.100+0000 7f0e56a00640 1 -- 172.21.15.113:0/2525508113 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e400a8d60 con 0x7f0e58151c20 2024-03-04T08:20:34.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.100+0000 7f0e5d200640 1 -- 172.21.15.113:0/2525508113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151c20 msgr2=0x7f0e58157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:34.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.100+0000 7f0e5d200640 1 --2- 172.21.15.113:0/2525508113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151c20 0x7f0e58157710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e400097c0 tx=0x7f0e4001ec90 comp rx=0 tx=0).stop 2024-03-04T08:20:34.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.100+0000 7f0e5d200640 1 -- 172.21.15.113:0/2525508113 shutdown_connections 2024-03-04T08:20:34.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.100+0000 7f0e5d200640 1 --2- 172.21.15.113:0/2525508113 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151c20 0x7f0e58157710 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:34.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.100+0000 7f0e5d200640 1 --2- 172.21.15.113:0/2525508113 >> v2:172.21.15.136:3300/0 conn(0x7f0e58151840 0x7f0e580c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:34.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.100+0000 7f0e5d200640 1 -- 172.21.15.113:0/2525508113 >> 172.21.15.113:0/2525508113 conn(0x7f0e580c9a30 msgr2=0x7f0e580c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:34.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e5d200640 1 -- 172.21.15.113:0/2525508113 shutdown_connections 2024-03-04T08:20:34.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e5d200640 1 -- 172.21.15.113:0/2525508113 wait complete. 2024-03-04T08:20:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e5d200640 1 Processor -- start 2024-03-04T08:20:34.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e5d200640 1 -- start start 2024-03-04T08:20:34.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e5d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5806d140 0x7f0e580c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:34.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e5d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151840 0x7f0e580cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e5d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e58151c20 0x7f0e58140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e5d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58071db0 con 0x7f0e58151c20 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e5d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58071ad0 con 0x7f0e5806d140 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e5d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5806c880 con 0x7f0e58151840 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e57e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e58151c20 0x7f0e58140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e57e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e58151c20 0x7f0e58140870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45974/0 (socket says 172.21.15.113:45974) 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e4ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151840 0x7f0e580cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e57e00640 1 -- 172.21.15.113:0/2714486838 learned_addr learned my addr 172.21.15.113:0/2714486838 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e57400640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5806d140 0x7f0e580c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e57e00640 1 -- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151840 msgr2=0x7f0e580cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e57e00640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151840 0x7f0e580cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e57e00640 1 -- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5806d140 msgr2=0x7f0e580c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e57e00640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5806d140 0x7f0e580c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:34.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e57e00640 1 -- 172.21.15.113:0/2714486838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580c9300 con 0x7f0e58151c20 2024-03-04T08:20:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e57e00640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e58151c20 0x7f0e58140870 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000c7a0 tx=0x7f0e5000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e4fe00640 1 -- 172.21.15.113:0/2714486838 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50018070 con 0x7f0e58151c20 2024-03-04T08:20:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.104+0000 7f0e4fe00640 1 -- 172.21.15.113:0/2714486838 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5000a640 con 0x7f0e58151c20 2024-03-04T08:20:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.108+0000 7f0e5d200640 1 -- 172.21.15.113:0/2714486838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e580c95e0 con 0x7f0e58151c20 2024-03-04T08:20:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.108+0000 7f0e4fe00640 1 -- 172.21.15.113:0/2714486838 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000aab0 con 0x7f0e58151c20 2024-03-04T08:20:34.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.108+0000 7f0e5d200640 1 -- 172.21.15.113:0/2714486838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5814d5b0 con 0x7f0e58151c20 2024-03-04T08:20:34.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.108+0000 7f0e5d200640 1 -- 172.21.15.113:0/2714486838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e58075a80 con 0x7f0e58151c20 2024-03-04T08:20:34.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.108+0000 7f0e4fe00640 1 -- 172.21.15.113:0/2714486838 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e5001b070 con 0x7f0e58151c20 2024-03-04T08:20:34.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.108+0000 7f0e4fe00640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e34066f90 0x7f0e34069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:34.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.108+0000 7f0e4fe00640 1 -- 172.21.15.113:0/2714486838 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f0e50084e60 con 0x7f0e58151c20 2024-03-04T08:20:34.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.108+0000 7f0e57400640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e34066f90 0x7f0e34069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:34.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.108+0000 7f0e57400640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e34066f90 0x7f0e34069450 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009b60 tx=0x7f0e48009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:34.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.116+0000 7f0e4fe00640 1 -- 172.21.15.113:0/2714486838 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0e50054f30 con 0x7f0e58151c20 2024-03-04T08:20:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.400+0000 7f0e5d200640 1 -- 172.21.15.113:0/2714486838 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e5814f7d0 con 0x7f0e58151c20 2024-03-04T08:20:34.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.400+0000 7f0e4fe00640 1 -- 172.21.15.113:0/2714486838 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f0e500589f0 con 0x7f0e58151c20 2024-03-04T08:20:34.397 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:34.397 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":168,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:27.135997+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:active","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8253,8032,7839]},"id":1}]} 2024-03-04T08:20:34.397 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 168 2024-03-04T08:20:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.400+0000 7f0e5d200640 1 -- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e34066f90 msgr2=0x7f0e34069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.400+0000 7f0e5d200640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e34066f90 0x7f0e34069450 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009b60 tx=0x7f0e48009190 comp rx=0 tx=0).stop 2024-03-04T08:20:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.400+0000 7f0e5d200640 1 -- 172.21.15.113:0/2714486838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e58151c20 msgr2=0x7f0e58140870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.400+0000 7f0e5d200640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e58151c20 0x7f0e58140870 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000c7a0 tx=0x7f0e5000cc60 comp rx=0 tx=0).stop 2024-03-04T08:20:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.404+0000 7f0e5d200640 1 -- 172.21.15.113:0/2714486838 shutdown_connections 2024-03-04T08:20:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.404+0000 7f0e5d200640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0e34066f90 0x7f0e34069450 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.404+0000 7f0e5d200640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e58151c20 0x7f0e58140870 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.404+0000 7f0e5d200640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0e58151840 0x7f0e580cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.404+0000 7f0e5d200640 1 --2- 172.21.15.113:0/2714486838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0e5806d140 0x7f0e580c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:34.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.404+0000 7f0e5d200640 1 -- 172.21.15.113:0/2714486838 >> 172.21.15.113:0/2714486838 conn(0x7f0e580c9a30 msgr2=0x7f0e5813ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:34.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.404+0000 7f0e5d200640 1 -- 172.21.15.113:0/2714486838 shutdown_connections 2024-03-04T08:20:34.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:34.404+0000 7f0e5d200640 1 -- 172.21.15.113:0/2714486838 wait complete. 2024-03-04T08:20:34.426 INFO:tasks.mds_thrash.fs.[cephfs]:mds.c reported in up:standby state 2024-03-04T08:20:34.426 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 16 secs before thrashing 2024-03-04T08:20:50.427 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:50.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e3800640 1 Processor -- start 2024-03-04T08:20:50.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e3800640 1 -- start start 2024-03-04T08:20:50.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e3800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 0x7f12dc0c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:50.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e3800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f12dc154fe0 0x7f12dc0c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:50.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e3800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12dc0c7d80 con 0x7f12dc0c7700 2024-03-04T08:20:50.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e3800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc15efb0 con 0x7f12dc154fe0 2024-03-04T08:20:50.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e3800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0c6b70 con 0x7f12dc154c00 2024-03-04T08:20:50.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e1800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f12dc0c7700 0x7f12dc15abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36072/0 (socket says 172.21.15.113:36072) 2024-03-04T08:20:50.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e1800640 1 -- 172.21.15.113:0/1841000278 learned_addr learned my addr 172.21.15.113:0/1841000278 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:50.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e0e00640 1 --2- 172.21.15.113:0/1841000278 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 0x7f12dc0c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:50.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12dbe00640 1 --2- 172.21.15.113:0/1841000278 >> v2:172.21.15.136:3300/0 conn(0x7f12dc154fe0 0x7f12dc0c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:50.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12db400640 1 -- 172.21.15.113:0/1841000278 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538952356 0 0) 0x7f12dc0c7d80 con 0x7f12dc0c7700 2024-03-04T08:20:50.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12db400640 1 -- 172.21.15.113:0/1841000278 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c4003530 con 0x7f12dc0c7700 2024-03-04T08:20:50.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12dbe00640 1 -- 172.21.15.113:0/1841000278 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 msgr2=0x7f12dc0c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:50.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12dbe00640 1 --2- 172.21.15.113:0/1841000278 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 0x7f12dc0c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:50.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12dbe00640 1 -- 172.21.15.113:0/1841000278 >> v1:172.21.15.113:6789/0 conn(0x7f12dc0c7700 legacy=0x7f12dc15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:50.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12db400640 1 -- 172.21.15.113:0/1841000278 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14922235 0 0) 0x7f12c4003530 con 0x7f12dc0c7700 2024-03-04T08:20:50.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12dbe00640 1 -- 172.21.15.113:0/1841000278 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc0c8060 con 0x7f12dc154fe0 2024-03-04T08:20:50.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.571+0000 7f12e0e00640 1 --2- 172.21.15.113:0/1841000278 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 0x7f12dc0c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:20:50.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.575+0000 7f12dbe00640 1 --2- 172.21.15.113:0/1841000278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc154fe0 0x7f12dc0c1b50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f12c80097c0 tx=0x7f12c801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f371e3e1a69cfc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:50.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.575+0000 7f12db400640 1 -- 172.21.15.113:0/1841000278 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c80b0070 con 0x7f12dc154fe0 2024-03-04T08:20:50.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.575+0000 7f12db400640 1 -- 172.21.15.113:0/1841000278 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c80ab400 con 0x7f12dc154fe0 2024-03-04T08:20:50.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.575+0000 7f12db400640 1 -- 172.21.15.113:0/1841000278 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4001a00 con 0x7f12dc154fe0 2024-03-04T08:20:50.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.579+0000 7f12e3800640 1 -- 172.21.15.113:0/1841000278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc154fe0 msgr2=0x7f12dc0c1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:50.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.579+0000 7f12e3800640 1 --2- 172.21.15.113:0/1841000278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc154fe0 0x7f12dc0c1b50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f12c80097c0 tx=0x7f12c801c330 comp rx=0 tx=0).stop 2024-03-04T08:20:50.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.579+0000 7f12e3800640 1 -- 172.21.15.113:0/1841000278 shutdown_connections 2024-03-04T08:20:50.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.579+0000 7f12e3800640 1 --2- 172.21.15.113:0/1841000278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc154fe0 0x7f12dc0c1b50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:50.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.579+0000 7f12e3800640 1 --2- 172.21.15.113:0/1841000278 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 0x7f12dc0c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:50.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.579+0000 7f12e3800640 1 -- 172.21.15.113:0/1841000278 >> 172.21.15.113:0/1841000278 conn(0x7f12dc06e820 msgr2=0x7f12dc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:50.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.579+0000 7f12e3800640 1 -- 172.21.15.113:0/1841000278 shutdown_connections 2024-03-04T08:20:50.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.579+0000 7f12e3800640 1 -- 172.21.15.113:0/1841000278 wait complete. 2024-03-04T08:20:50.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.579+0000 7f12e3800640 1 Processor -- start 2024-03-04T08:20:50.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.579+0000 7f12e3800640 1 -- start start 2024-03-04T08:20:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e3800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc0c7700 0x7f12dc15c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e3800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 0x7f12dc1454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e3800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12dc154fe0 0x7f12dc14c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e3800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc167c10 con 0x7f12dc154fe0 2024-03-04T08:20:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e3800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0c2320 con 0x7f12dc0c7700 2024-03-04T08:20:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e3800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc167d90 con 0x7f12dc154c00 2024-03-04T08:20:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12dc154fe0 0x7f12dc14c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12dc154fe0 0x7f12dc14c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51256/0 (socket says 172.21.15.113:51256) 2024-03-04T08:20:50.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e1800640 1 -- 172.21.15.113:0/1472537728 learned_addr learned my addr 172.21.15.113:0/1472537728 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:50.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e0e00640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc0c7700 0x7f12dc15c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:50.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12dbe00640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 0x7f12dc1454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:50.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e1800640 1 -- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 msgr2=0x7f12dc1454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:50.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e1800640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 0x7f12dc1454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:50.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e1800640 1 -- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc0c7700 msgr2=0x7f12dc15c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:50.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e1800640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc0c7700 0x7f12dc15c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:50.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e1800640 1 -- 172.21.15.113:0/1472537728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc15d8a0 con 0x7f12dc154fe0 2024-03-04T08:20:50.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e1800640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12dc154fe0 0x7f12dc14c250 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f12d400cc40 tx=0x7f12d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12d8c00640 1 -- 172.21.15.113:0/1472537728 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d400ce50 con 0x7f12dc154fe0 2024-03-04T08:20:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12d8c00640 1 -- 172.21.15.113:0/1472537728 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d4012400 con 0x7f12dc154fe0 2024-03-04T08:20:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e3800640 1 -- 172.21.15.113:0/1472537728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12dc0c9f70 con 0x7f12dc154fe0 2024-03-04T08:20:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12d8c00640 1 -- 172.21.15.113:0/1472537728 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d40127f0 con 0x7f12dc154fe0 2024-03-04T08:20:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e3800640 1 -- 172.21.15.113:0/1472537728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc0ca720 con 0x7f12dc154fe0 2024-03-04T08:20:50.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.583+0000 7f12e3800640 1 -- 172.21.15.113:0/1472537728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc075c00 con 0x7f12dc154fe0 2024-03-04T08:20:50.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.587+0000 7f12d8c00640 1 -- 172.21.15.113:0/1472537728 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12d4012990 con 0x7f12dc154fe0 2024-03-04T08:20:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.587+0000 7f12d8c00640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f12bc067100 0x7f12bc0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.587+0000 7f12d8c00640 1 -- 172.21.15.113:0/1472537728 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f12d40860e0 con 0x7f12dc154fe0 2024-03-04T08:20:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.587+0000 7f12e0e00640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f12bc067100 0x7f12bc0695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:50.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.587+0000 7f12e0e00640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f12bc067100 0x7f12bc0695c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009580 tx=0x7f12d0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:50.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.591+0000 7f12d8c00640 1 -- 172.21.15.113:0/1472537728 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f12d4056130 con 0x7f12dc154fe0 2024-03-04T08:20:50.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.911+0000 7f12e3800640 1 -- 172.21.15.113:0/1472537728 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12dc158d60 con 0x7f12dc154fe0 2024-03-04T08:20:50.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.911+0000 7f12d8c00640 1 -- 172.21.15.113:0/1472537728 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+6621 (secure 0 0 0) 0x7f12d405a1d0 con 0x7f12dc154fe0 2024-03-04T08:20:50.909 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:50.909 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":168,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":167,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:27.135997+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:active","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8253,8032,7839]},"id":1}]} 2024-03-04T08:20:50.910 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 168 2024-03-04T08:20:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 -- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f12bc067100 msgr2=0x7f12bc0695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f12bc067100 0x7f12bc0695c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009580 tx=0x7f12d0009190 comp rx=0 tx=0).stop 2024-03-04T08:20:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 -- 172.21.15.113:0/1472537728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12dc154fe0 msgr2=0x7f12dc14c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12dc154fe0 0x7f12dc14c250 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f12d400cc40 tx=0x7f12d400a430 comp rx=0 tx=0).stop 2024-03-04T08:20:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 -- 172.21.15.113:0/1472537728 shutdown_connections 2024-03-04T08:20:50.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f12bc067100 0x7f12bc0695c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f12dc154fe0 0x7f12dc14c250 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f12dc154c00 0x7f12dc1454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 --2- 172.21.15.113:0/1472537728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f12dc0c7700 0x7f12dc15c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 -- 172.21.15.113:0/1472537728 >> 172.21.15.113:0/1472537728 conn(0x7f12dc06e820 msgr2=0x7f12dc0c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 -- 172.21.15.113:0/1472537728 shutdown_connections 2024-03-04T08:20:50.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:50.915+0000 7f12e3800640 1 -- 172.21.15.113:0/1472537728 wait complete. 2024-03-04T08:20:50.933 INFO:tasks.mds_thrash.fs.[cephfs]:thrashing max_mds: 3 -> 2 2024-03-04T08:20:50.934 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2024-03-04T08:20:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.091+0000 7f345e400640 1 Processor -- start 2024-03-04T08:20:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.091+0000 7f345e400640 1 -- start start 2024-03-04T08:20:51.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345e400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f345806d260 0x7f34580c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:51.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d720 0x7f345815a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:51.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345e400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345815b5d0 con 0x7f34581515f0 2024-03-04T08:20:51.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345e400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345815b8b0 con 0x7f345806d260 2024-03-04T08:20:51.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34580c86c0 con 0x7f345806d720 2024-03-04T08:20:51.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345d000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f34581515f0 0x7f34580c3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36082/0 (socket says 172.21.15.113:36082) 2024-03-04T08:20:51.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345d000640 1 -- 172.21.15.113:0/1171737569 learned_addr learned my addr 172.21.15.113:0/1171737569 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:51.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345da00640 1 --2- 172.21.15.113:0/1171737569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d720 0x7f345815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f344fe00640 1 --2- 172.21.15.113:0/1171737569 >> v2:172.21.15.136:3300/0 conn(0x7f345806d260 0x7f34580c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f3457e00640 1 -- 172.21.15.113:0/1171737569 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036958256 0 0) 0x7f345815b5d0 con 0x7f34581515f0 2024-03-04T08:20:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f3457e00640 1 -- 172.21.15.113:0/1171737569 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f343c003530 con 0x7f34581515f0 2024-03-04T08:20:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345da00640 1 -- 172.21.15.113:0/1171737569 >> v2:172.21.15.136:3300/0 conn(0x7f345806d260 msgr2=0x7f34580c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345da00640 1 --2- 172.21.15.113:0/1171737569 >> v2:172.21.15.136:3300/0 conn(0x7f345806d260 0x7f34580c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345da00640 1 -- 172.21.15.113:0/1171737569 >> v1:172.21.15.113:6789/0 conn(0x7f34581515f0 legacy=0x7f34580c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.095+0000 7f345da00640 1 -- 172.21.15.113:0/1171737569 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34580c89a0 con 0x7f345806d720 2024-03-04T08:20:51.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345da00640 1 --2- 172.21.15.113:0/1171737569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d720 0x7f345815a990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3450009d90 tx=0x7f345001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc41f840cd3c28 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:51.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f3457e00640 1 -- 172.21.15.113:0/1171737569 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34500b2070 con 0x7f345806d720 2024-03-04T08:20:51.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f3457e00640 1 -- 172.21.15.113:0/1171737569 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34500a5920 con 0x7f345806d720 2024-03-04T08:20:51.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f3457e00640 1 -- 172.21.15.113:0/1171737569 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34500a5cd0 con 0x7f345806d720 2024-03-04T08:20:51.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345e400640 1 -- 172.21.15.113:0/1171737569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d720 msgr2=0x7f345815a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:51.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345e400640 1 --2- 172.21.15.113:0/1171737569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d720 0x7f345815a990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3450009d90 tx=0x7f345001c970 comp rx=0 tx=0).stop 2024-03-04T08:20:51.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345e400640 1 -- 172.21.15.113:0/1171737569 shutdown_connections 2024-03-04T08:20:51.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345e400640 1 --2- 172.21.15.113:0/1171737569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d720 0x7f345815a990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:51.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345e400640 1 --2- 172.21.15.113:0/1171737569 >> v2:172.21.15.136:3300/0 conn(0x7f345806d260 0x7f34580c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:51.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345e400640 1 -- 172.21.15.113:0/1171737569 >> 172.21.15.113:0/1171737569 conn(0x7f34580cd010 msgr2=0x7f34580cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:51.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345e400640 1 -- 172.21.15.113:0/1171737569 shutdown_connections 2024-03-04T08:20:51.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345e400640 1 -- 172.21.15.113:0/1171737569 wait complete. 2024-03-04T08:20:51.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345e400640 1 Processor -- start 2024-03-04T08:20:51.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.099+0000 7f345e400640 1 -- start start 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d260 0x7f345814cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345806d720 0x7f34581f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345e400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581515f0 0x7f34581f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3458151b50 con 0x7f345806d720 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345e400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34581519d0 con 0x7f34581515f0 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34580c3d40 con 0x7f345806d260 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f344fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345806d720 0x7f34581f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d260 0x7f345814cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d260 0x7f345814cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:58826/0 (socket says 172.21.15.113:58826) 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345d000640 1 -- 172.21.15.113:0/3332512542 learned_addr learned my addr 172.21.15.113:0/3332512542 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f344fe00640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345806d720 0x7f34581f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51258/0 (socket says 172.21.15.113:51258) 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345da00640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581515f0 0x7f34581f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:51.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f344fe00640 1 -- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d260 msgr2=0x7f345814cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f344fe00640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d260 0x7f345814cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f344fe00640 1 -- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581515f0 msgr2=0x7f34581f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f344fe00640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581515f0 0x7f34581f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f344fe00640 1 -- 172.21.15.113:0/3332512542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345814af70 con 0x7f345806d720 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f344fe00640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345806d720 0x7f34581f3500 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f344000e8a0 tx=0x7f344000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f3455600640 1 -- 172.21.15.113:0/3332512542 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440017070 con 0x7f345806d720 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345e400640 1 -- 172.21.15.113:0/3332512542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345815a660 con 0x7f345806d720 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f3455600640 1 -- 172.21.15.113:0/3332512542 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344000c940 con 0x7f345806d720 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f3455600640 1 -- 172.21.15.113:0/3332512542 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344000cd10 con 0x7f345806d720 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345e400640 1 -- 172.21.15.113:0/3332512542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34581562a0 con 0x7f345806d720 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f345e400640 1 -- 172.21.15.113:0/3332512542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3458158aa0 con 0x7f345806d720 2024-03-04T08:20:51.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.103+0000 7f3455600640 1 -- 172.21.15.113:0/3332512542 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f344001a070 con 0x7f345806d720 2024-03-04T08:20:51.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.107+0000 7f3455600640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3434066f60 0x7f3434069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:51.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.107+0000 7f3455600640 1 -- 172.21.15.113:0/3332512542 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f3440085700 con 0x7f345806d720 2024-03-04T08:20:51.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.107+0000 7f345d000640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3434066f60 0x7f3434069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:51.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.107+0000 7f345d000640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3434066f60 0x7f3434069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3448007fd0 tx=0x7f3448009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:51.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.115+0000 7f3455600640 1 -- 172.21.15.113:0/3332512542 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3440055750 con 0x7f345806d720 2024-03-04T08:20:51.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:51.315+0000 7f345e400640 1 -- 172.21.15.113:0/3332512542 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f3458075a80 con 0x7f345806d720 2024-03-04T08:20:52.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f3455600640 1 -- 172.21.15.113:0/3332512542 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v169) v1 ==== 105+0+0 (secure 0 0 0) 0x7f34400597f0 con 0x7f345806d720 2024-03-04T08:20:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f345e400640 1 -- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3434066f60 msgr2=0x7f3434069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f345e400640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3434066f60 0x7f3434069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3448007fd0 tx=0x7f3448009040 comp rx=0 tx=0).stop 2024-03-04T08:20:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f345e400640 1 -- 172.21.15.113:0/3332512542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345806d720 msgr2=0x7f34581f3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f345e400640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345806d720 0x7f34581f3500 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f344000e8a0 tx=0x7f344000ed60 comp rx=0 tx=0).stop 2024-03-04T08:20:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f345e400640 1 -- 172.21.15.113:0/3332512542 shutdown_connections 2024-03-04T08:20:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f345e400640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f34581515f0 0x7f34581f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f345e400640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3434066f60 0x7f3434069420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f345e400640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f345806d720 0x7f34581f3500 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f345e400640 1 --2- 172.21.15.113:0/3332512542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f345806d260 0x7f345814cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.207+0000 7f345e400640 1 -- 172.21.15.113:0/3332512542 >> 172.21.15.113:0/3332512542 conn(0x7f34580cd010 msgr2=0x7f34580722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.211+0000 7f345e400640 1 -- 172.21.15.113:0/3332512542 shutdown_connections 2024-03-04T08:20:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.211+0000 7f345e400640 1 -- 172.21.15.113:0/3332512542 wait complete. 2024-03-04T08:20:52.233 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:20:52.233 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:52.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5c1e00640 1 Processor -- start 2024-03-04T08:20:52.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5c1e00640 1 -- start start 2024-03-04T08:20:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5c1e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5bc158210 0x7fb5bc0c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5c1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5bc06b060 0x7fb5bc0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5c1e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc0b2230 con 0x7fb5bc0733f0 2024-03-04T08:20:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5c1e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5bc072fb0 con 0x7fb5bc158210 2024-03-04T08:20:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5c1e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0ca940 con 0x7fb5bc06b060 2024-03-04T08:20:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5baa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5bc158210 0x7fb5bc0c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5baa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb5bc158210 0x7fb5bc0c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51270/0 (socket says 172.21.15.113:51270) 2024-03-04T08:20:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5baa00640 1 -- 172.21.15.113:0/2472838233 learned_addr learned my addr 172.21.15.113:0/2472838233 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5bb400640 1 --2- 172.21.15.113:0/2472838233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5bc06b060 0x7fb5bc0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:52.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5baa00640 1 -- 172.21.15.113:0/2472838233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5bc06b060 msgr2=0x7fb5bc0ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:52.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5baa00640 1 --2- 172.21.15.113:0/2472838233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5bc06b060 0x7fb5bc0ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5baa00640 1 -- 172.21.15.113:0/2472838233 >> v1:172.21.15.136:6789/0 conn(0x7fb5bc0733f0 legacy=0x7fb5bc0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:52.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.383+0000 7fb5baa00640 1 -- 172.21.15.113:0/2472838233 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc0c7bb0 con 0x7fb5bc158210 2024-03-04T08:20:52.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5baa00640 1 --2- 172.21.15.113:0/2472838233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5bc158210 0x7fb5bc0c8bb0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009870 tx=0x7fb5ac01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0eb22e8b6dfe75 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:52.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5b9600640 1 -- 172.21.15.113:0/2472838233 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0ab650 con 0x7fb5bc158210 2024-03-04T08:20:52.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5b9600640 1 -- 172.21.15.113:0/2472838233 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac0b0070 con 0x7fb5bc158210 2024-03-04T08:20:52.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5b9600640 1 -- 172.21.15.113:0/2472838233 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0aba10 con 0x7fb5bc158210 2024-03-04T08:20:52.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2472838233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5bc158210 msgr2=0x7fb5bc0c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:52.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5c1e00640 1 --2- 172.21.15.113:0/2472838233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5bc158210 0x7fb5bc0c8bb0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009870 tx=0x7fb5ac01c560 comp rx=0 tx=0).stop 2024-03-04T08:20:52.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2472838233 shutdown_connections 2024-03-04T08:20:52.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5c1e00640 1 --2- 172.21.15.113:0/2472838233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5bc06b060 0x7fb5bc0ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5c1e00640 1 --2- 172.21.15.113:0/2472838233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5bc158210 0x7fb5bc0c8bb0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2472838233 >> 172.21.15.113:0/2472838233 conn(0x7fb5bc0671b0 msgr2=0x7fb5bc06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:52.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2472838233 shutdown_connections 2024-03-04T08:20:52.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.387+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2472838233 wait complete. 2024-03-04T08:20:52.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.391+0000 7fb5c1e00640 1 Processor -- start 2024-03-04T08:20:52.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.391+0000 7fb5c1e00640 1 -- start start 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.391+0000 7fb5c1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5bc06b060 0x7fb5bc0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.391+0000 7fb5c1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5bc0733f0 0x7fb5bc147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.391+0000 7fb5c1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5bc158210 0x7fb5bc1f5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.391+0000 7fb5c1e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0cd720 con 0x7fb5bc158210 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.391+0000 7fb5c1e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0b1ef0 con 0x7fb5bc06b060 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.391+0000 7fb5c1e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0cd8a0 con 0x7fb5bc0733f0 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5baa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5bc06b060 0x7fb5bc0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5bb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5bc158210 0x7fb5bc1f5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5ba000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5bc0733f0 0x7fb5bc147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5baa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5bc06b060 0x7fb5bc0cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:42906/0 (socket says 172.21.15.113:42906) 2024-03-04T08:20:52.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5baa00640 1 -- 172.21.15.113:0/2681197155 learned_addr learned my addr 172.21.15.113:0/2681197155 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:52.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5bb400640 1 -- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5bc0733f0 msgr2=0x7fb5bc147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5bb400640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5bc0733f0 0x7fb5bc147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5bb400640 1 -- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5bc06b060 msgr2=0x7fb5bc0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5bb400640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5bc06b060 0x7fb5bc0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5bb400640 1 -- 172.21.15.113:0/2681197155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc159f40 con 0x7fb5bc158210 2024-03-04T08:20:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5bb400640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5bc158210 0x7fb5bc1f5130 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400efc0 tx=0x7fb5b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5b3400640 1 -- 172.21.15.113:0/2681197155 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4018020 con 0x7fb5bc158210 2024-03-04T08:20:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5b3400640 1 -- 172.21.15.113:0/2681197155 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b4014480 con 0x7fb5bc158210 2024-03-04T08:20:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2681197155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc1593c0 con 0x7fb5bc158210 2024-03-04T08:20:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5b3400640 1 -- 172.21.15.113:0/2681197155 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4014870 con 0x7fb5bc158210 2024-03-04T08:20:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2681197155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5bc159980 con 0x7fb5bc158210 2024-03-04T08:20:52.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.395+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2681197155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5bc075c00 con 0x7fb5bc158210 2024-03-04T08:20:52.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.399+0000 7fb5b3400640 1 -- 172.21.15.113:0/2681197155 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5b401b020 con 0x7fb5bc158210 2024-03-04T08:20:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.399+0000 7fb5b3400640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb59c066f60 0x7fb59c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.399+0000 7fb5b3400640 1 -- 172.21.15.113:0/2681197155 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fb5b4086570 con 0x7fb5bc158210 2024-03-04T08:20:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.399+0000 7fb5baa00640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb59c066f60 0x7fb59c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:52.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.399+0000 7fb5baa00640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb59c066f60 0x7fb59c069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac01c5a0 tx=0x7fb5ac0a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.403+0000 7fb5b3400640 1 -- 172.21.15.113:0/2681197155 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb5b4022030 con 0x7fb5bc158210 2024-03-04T08:20:52.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.683+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2681197155 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5bc0731d0 con 0x7fb5bc158210 2024-03-04T08:20:52.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.683+0000 7fb5b3400640 1 -- 172.21.15.113:0/2681197155 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7fb5b4010040 con 0x7fb5bc158210 2024-03-04T08:20:52.681 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:52.681 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:20:52.681 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:20:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb59c066f60 msgr2=0x7fb59c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb59c066f60 0x7fb59c069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac01c5a0 tx=0x7fb5ac0a7000 comp rx=0 tx=0).stop 2024-03-04T08:20:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2681197155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5bc158210 msgr2=0x7fb5bc1f5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5bc158210 0x7fb5bc1f5130 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb5b400efc0 tx=0x7fb5b400c490 comp rx=0 tx=0).stop 2024-03-04T08:20:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2681197155 shutdown_connections 2024-03-04T08:20:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb59c066f60 0x7fb59c069420 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb5bc158210 0x7fb5bc1f5130 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb5bc0733f0 0x7fb5bc147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 --2- 172.21.15.113:0/2681197155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5bc06b060 0x7fb5bc0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2681197155 >> 172.21.15.113:0/2681197155 conn(0x7fb5bc0671b0 msgr2=0x7fb5bc156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2681197155 shutdown_connections 2024-03-04T08:20:52.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:52.687+0000 7fb5c1e00640 1 -- 172.21.15.113:0/2681197155 wait complete. 2024-03-04T08:20:52.704 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:20:52.705 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 170, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 170, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8253": { "addr": "172.21.15.113:6836/893414524", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 893414524, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 893414524, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8253, "incarnation": 163, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:stopping", "state_seq": 35 } }, "last_failure": 0, "last_failure_osd_epoch": 80, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:20:52.211831+0000", "qdb_cluster": [ 7839, 8032 ], "qdb_leader": 7839, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 8032, "mds_2": 8253 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.113:6833/2999156262", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2999156262, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2999156262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 168, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8388, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:20:54.705 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:54.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32bd000640 1 Processor -- start 2024-03-04T08:20:54.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32bd000640 1 -- start start 2024-03-04T08:20:54.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32bd000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f32b8152bd0 0x7f32b8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:54.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32bd000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b81591f0 con 0x7f32b81527f0 2024-03-04T08:20:54.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32bd000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b81594d0 con 0x7f32b80c8200 2024-03-04T08:20:54.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32bd000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b8071df0 con 0x7f32b8152bd0 2024-03-04T08:20:54.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32b6800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f32b8152bd0 0x7f32b8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:54.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32b5e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f32b81527f0 0x7f32b8072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:36088/0 (socket says 172.21.15.113:36088) 2024-03-04T08:20:54.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32b5e00640 1 -- 172.21.15.113:0/2791488329 learned_addr learned my addr 172.21.15.113:0/2791488329 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:54.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32b5400640 1 -- 172.21.15.113:0/2791488329 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926070471 0 0) 0x7f32b81591f0 con 0x7f32b81527f0 2024-03-04T08:20:54.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32b6800640 1 -- 172.21.15.113:0/2791488329 >> v1:172.21.15.136:6789/0 conn(0x7f32b80c8200 legacy=0x7f32b81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:54.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32b6800640 1 -- 172.21.15.113:0/2791488329 >> v1:172.21.15.113:6789/0 conn(0x7f32b81527f0 legacy=0x7f32b8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:54.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.847+0000 7f32b6800640 1 -- 172.21.15.113:0/2791488329 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b80717a0 con 0x7f32b8152bd0 2024-03-04T08:20:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32b6800640 1 --2- 172.21.15.113:0/2791488329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b8152bd0 0x7f32b8070d40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f32b00097c0 tx=0x7f32b001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46f4a0abb122084 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32b5400640 1 -- 172.21.15.113:0/2791488329 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b00b0070 con 0x7f32b8152bd0 2024-03-04T08:20:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32b5400640 1 -- 172.21.15.113:0/2791488329 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b00ab600 con 0x7f32b8152bd0 2024-03-04T08:20:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32b5400640 1 -- 172.21.15.113:0/2791488329 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b00ab9d0 con 0x7f32b8152bd0 2024-03-04T08:20:54.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32bd000640 1 -- 172.21.15.113:0/2791488329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b8152bd0 msgr2=0x7f32b8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:54.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32bd000640 1 --2- 172.21.15.113:0/2791488329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b8152bd0 0x7f32b8070d40 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f32b00097c0 tx=0x7f32b001c4a0 comp rx=0 tx=0).stop 2024-03-04T08:20:54.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32bd000640 1 -- 172.21.15.113:0/2791488329 shutdown_connections 2024-03-04T08:20:54.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32bd000640 1 --2- 172.21.15.113:0/2791488329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b8152bd0 0x7f32b8070d40 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:54.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32bd000640 1 -- 172.21.15.113:0/2791488329 >> 172.21.15.113:0/2791488329 conn(0x7f32b806e820 msgr2=0x7f32b806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:54.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32bd000640 1 -- 172.21.15.113:0/2791488329 shutdown_connections 2024-03-04T08:20:54.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.851+0000 7f32bd000640 1 -- 172.21.15.113:0/2791488329 wait complete. 2024-03-04T08:20:54.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32bd000640 1 Processor -- start 2024-03-04T08:20:54.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32bd000640 1 -- start start 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32bd000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32b80c8200 0x7f32b80cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32bd000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b81527f0 0x7f32b8142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32bd000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b8152bd0 0x7f32b8155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32bd000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8166500 con 0x7f32b81527f0 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32bd000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8166380 con 0x7f32b8152bd0 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32bd000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8166680 con 0x7f32b80c8200 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32ad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b81527f0 0x7f32b8142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32ad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b81527f0 0x7f32b8142aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51296/0 (socket says 172.21.15.113:51296) 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32ad400640 1 -- 172.21.15.113:0/4184334664 learned_addr learned my addr 172.21.15.113:0/4184334664 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32b5e00640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32b80c8200 0x7f32b80cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32b6800640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b8152bd0 0x7f32b8155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:54.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32ad400640 1 -- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32b80c8200 msgr2=0x7f32b80cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32ad400640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32b80c8200 0x7f32b80cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32ad400640 1 -- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b8152bd0 msgr2=0x7f32b8155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32ad400640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b8152bd0 0x7f32b8155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32ad400640 1 -- 172.21.15.113:0/4184334664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b81f5150 con 0x7f32b81527f0 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32ad400640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b81527f0 0x7f32b8142aa0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f32a000c500 tx=0x7f32a000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32aea00640 1 -- 172.21.15.113:0/4184334664 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0017070 con 0x7f32b81527f0 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32aea00640 1 -- 172.21.15.113:0/4184334664 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a00052d0 con 0x7f32b81527f0 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32aea00640 1 -- 172.21.15.113:0/4184334664 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0003820 con 0x7f32b81527f0 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32bd000640 1 -- 172.21.15.113:0/4184334664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b81f53f0 con 0x7f32b81527f0 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.855+0000 7f32bd000640 1 -- 172.21.15.113:0/4184334664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b81f59b0 con 0x7f32b81527f0 2024-03-04T08:20:54.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.859+0000 7f32bd000640 1 -- 172.21.15.113:0/4184334664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b8075c00 con 0x7f32b81527f0 2024-03-04T08:20:54.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.859+0000 7f32aea00640 1 -- 172.21.15.113:0/4184334664 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32a001a070 con 0x7f32b81527f0 2024-03-04T08:20:54.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.859+0000 7f32aea00640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3294066f90 0x7f3294069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:54.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.859+0000 7f32aea00640 1 -- 172.21.15.113:0/4184334664 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f32a0084f00 con 0x7f32b81527f0 2024-03-04T08:20:54.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.859+0000 7f32b5e00640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3294066f90 0x7f3294069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:54.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.863+0000 7f32b5e00640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3294066f90 0x7f3294069450 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f32a8001950 tx=0x7f32a80018e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:54.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:54.863+0000 7f32aea00640 1 -- 172.21.15.113:0/4184334664 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f32a0021020 con 0x7f32b81527f0 2024-03-04T08:20:55.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 -- 172.21.15.113:0/4184334664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32b80c7570 con 0x7f32b81527f0 2024-03-04T08:20:55.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32aea00640 1 -- 172.21.15.113:0/4184334664 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7f32a0005480 con 0x7f32b81527f0 2024-03-04T08:20:55.060 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:55.060 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:20:55.060 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:20:55.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 -- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3294066f90 msgr2=0x7f3294069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:55.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3294066f90 0x7f3294069450 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f32a8001950 tx=0x7f32a80018e0 comp rx=0 tx=0).stop 2024-03-04T08:20:55.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 -- 172.21.15.113:0/4184334664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b81527f0 msgr2=0x7f32b8142aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:55.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b81527f0 0x7f32b8142aa0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f32a000c500 tx=0x7f32a000c900 comp rx=0 tx=0).stop 2024-03-04T08:20:55.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 -- 172.21.15.113:0/4184334664 shutdown_connections 2024-03-04T08:20:55.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b8152bd0 0x7f32b8155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:55.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3294066f90 0x7f3294069450 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:55.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32b81527f0 0x7f32b8142aa0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:55.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 --2- 172.21.15.113:0/4184334664 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f32b80c8200 0x7f32b80cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:55.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 -- 172.21.15.113:0/4184334664 >> 172.21.15.113:0/4184334664 conn(0x7f32b806e820 msgr2=0x7f32b8157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:55.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 -- 172.21.15.113:0/4184334664 shutdown_connections 2024-03-04T08:20:55.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:55.063+0000 7f32bd000640 1 -- 172.21.15.113:0/4184334664 wait complete. 2024-03-04T08:20:55.082 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:20:57.083 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:57.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d63c00640 1 Processor -- start 2024-03-04T08:20:57.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d63c00640 1 -- start start 2024-03-04T08:20:57.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d63c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7d5c0b20b0 0x7f7d5c157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:57.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d63c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d5c0c85f0 0x7f7d5c161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:57.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d63c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5c0c1ec0 con 0x7f7d5c157c50 2024-03-04T08:20:57.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d63c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c0c9b30 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d63c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0c12a0 con 0x7f7d5c0c85f0 2024-03-04T08:20:57.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d5be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7d5c157c50 0x7f7d5c06b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33820/0 (socket says 172.21.15.113:33820) 2024-03-04T08:20:57.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d5be00640 1 -- 172.21.15.113:0/3300758883 learned_addr learned my addr 172.21.15.113:0/3300758883 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:20:57.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d61200640 1 --2- 172.21.15.113:0/3300758883 >> v2:172.21.15.113:3300/0 conn(0x7f7d5c0b20b0 0x7f7d5c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:57.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d61c00640 1 --2- 172.21.15.113:0/3300758883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d5c0c85f0 0x7f7d5c161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:57.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d5b400640 1 -- 172.21.15.113:0/3300758883 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701545763 0 0) 0x7f7d5c0c1ec0 con 0x7f7d5c157c50 2024-03-04T08:20:57.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d5b400640 1 -- 172.21.15.113:0/3300758883 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d44003530 con 0x7f7d5c157c50 2024-03-04T08:20:57.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d61200640 1 -- 172.21.15.113:0/3300758883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d5c0c85f0 msgr2=0x7f7d5c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:57.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d61200640 1 --2- 172.21.15.113:0/3300758883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d5c0c85f0 0x7f7d5c161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:57.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d61200640 1 -- 172.21.15.113:0/3300758883 >> v1:172.21.15.113:6789/0 conn(0x7f7d5c157c50 legacy=0x7f7d5c06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:57.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.227+0000 7f7d61200640 1 -- 172.21.15.113:0/3300758883 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c0c1580 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.231+0000 7f7d61200640 1 --2- 172.21.15.113:0/3300758883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 0x7f7d5c157710 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000a960 tx=0x7f7d50020060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137c597a43bbe28d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:57.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.231+0000 7f7d5b400640 1 -- 172.21.15.113:0/3300758883 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500b6070 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.231+0000 7f7d5b400640 1 -- 172.21.15.113:0/3300758883 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d500b1440 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.231+0000 7f7d5b400640 1 -- 172.21.15.113:0/3300758883 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000b040 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 -- 172.21.15.113:0/3300758883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 msgr2=0x7f7d5c157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:57.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 --2- 172.21.15.113:0/3300758883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 0x7f7d5c157710 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7d5000a960 tx=0x7f7d50020060 comp rx=0 tx=0).stop 2024-03-04T08:20:57.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 -- 172.21.15.113:0/3300758883 shutdown_connections 2024-03-04T08:20:57.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 --2- 172.21.15.113:0/3300758883 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d5c0c85f0 0x7f7d5c161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:57.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 --2- 172.21.15.113:0/3300758883 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 0x7f7d5c157710 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:57.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 -- 172.21.15.113:0/3300758883 >> 172.21.15.113:0/3300758883 conn(0x7f7d5c06e820 msgr2=0x7f7d5c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:57.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 -- 172.21.15.113:0/3300758883 shutdown_connections 2024-03-04T08:20:57.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 -- 172.21.15.113:0/3300758883 wait complete. 2024-03-04T08:20:57.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 Processor -- start 2024-03-04T08:20:57.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 -- start start 2024-03-04T08:20:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 0x7f7d5c1f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d5c0c85f0 0x7f7d5c06b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d5c157c50 0x7f7d5c145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0c2400 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1571f0 con 0x7f7d5c0c85f0 2024-03-04T08:20:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d63c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c156ec0 con 0x7f7d5c157c50 2024-03-04T08:20:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d61200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 0x7f7d5c1f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d61200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 0x7f7d5c1f1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52056/0 (socket says 172.21.15.113:52056) 2024-03-04T08:20:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d61200640 1 -- 172.21.15.113:0/3152675501 learned_addr learned my addr 172.21.15.113:0/3152675501 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d61c00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d5c157c50 0x7f7d5c145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d5be00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d5c0c85f0 0x7f7d5c06b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d61200640 1 -- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d5c157c50 msgr2=0x7f7d5c145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:57.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d61200640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d5c157c50 0x7f7d5c145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d61200640 1 -- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d5c0c85f0 msgr2=0x7f7d5c06b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d61200640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d5c0c85f0 0x7f7d5c06b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d61200640 1 -- 172.21.15.113:0/3152675501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c14b7b0 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.235+0000 7f7d5be00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d5c0c85f0 0x7f7d5c06b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:20:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d61200640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 0x7f7d5c1f1860 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7d500b4040 tx=0x7f7d5001ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d58c00640 1 -- 172.21.15.113:0/3152675501 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500c1070 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d58c00640 1 -- 172.21.15.113:0/3152675501 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d500bc660 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d63c00640 1 -- 172.21.15.113:0/3152675501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c1501a0 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d58c00640 1 -- 172.21.15.113:0/3152675501 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500b6070 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d63c00640 1 -- 172.21.15.113:0/3152675501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c1e2150 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d63c00640 1 -- 172.21.15.113:0/3152675501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c075c00 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d58c00640 1 -- 172.21.15.113:0/3152675501 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d500c4070 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d58c00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d3c066f60 0x7f7d3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:57.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d58c00640 1 -- 172.21.15.113:0/3152675501 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f7d50123230 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.239+0000 7f7d5be00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d3c066f60 0x7f7d3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:57.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.243+0000 7f7d5be00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d3c066f60 0x7f7d3c069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c149b80 tx=0x7f7d48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:57.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.243+0000 7f7d58c00640 1 -- 172.21.15.113:0/3152675501 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7d5c1501a0 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.431+0000 7f7d63c00640 1 -- 172.21.15.113:0/3152675501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d5c0c83d0 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d58c00640 1 -- 172.21.15.113:0/3152675501 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7f7d50019bc0 con 0x7f7d5c0b20b0 2024-03-04T08:20:57.431 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:57.431 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:20:57.431 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 -- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d3c066f60 msgr2=0x7f7d3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d3c066f60 0x7f7d3c069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c149b80 tx=0x7f7d48009040 comp rx=0 tx=0).stop 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 -- 172.21.15.113:0/3152675501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 msgr2=0x7f7d5c1f1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 0x7f7d5c1f1860 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7d500b4040 tx=0x7f7d5001ce00 comp rx=0 tx=0).stop 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 -- 172.21.15.113:0/3152675501 shutdown_connections 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7d5c157c50 0x7f7d5c145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d5c0c85f0 0x7f7d5c06b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7d3c066f60 0x7f7d3c069420 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 --2- 172.21.15.113:0/3152675501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7d5c0b20b0 0x7f7d5c1f1860 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 -- 172.21.15.113:0/3152675501 >> 172.21.15.113:0/3152675501 conn(0x7f7d5c06e820 msgr2=0x7f7d5c072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 -- 172.21.15.113:0/3152675501 shutdown_connections 2024-03-04T08:20:57.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:57.435+0000 7f7d63c00640 1 -- 172.21.15.113:0/3152675501 wait complete. 2024-03-04T08:20:57.452 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:20:59.453 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:20:59.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.591+0000 7fa989200640 1 Processor -- start 2024-03-04T08:20:59.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.591+0000 7fa989200640 1 -- start start 2024-03-04T08:20:59.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.591+0000 7fa989200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa984152d60 0x7fa9840ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:59.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.591+0000 7fa989200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa984159040 0x7fa9840ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:59.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.591+0000 7fa989200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9840c5380 con 0x7fa984070120 2024-03-04T08:20:59.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.591+0000 7fa989200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9840c3440 con 0x7fa984152d60 2024-03-04T08:20:59.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.591+0000 7fa989200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98406f620 con 0x7fa984159040 2024-03-04T08:20:59.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa984152d60 0x7fa9840ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:59.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa984152d60 0x7fa9840ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52062/0 (socket says 172.21.15.113:52062) 2024-03-04T08:20:59.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981e00640 1 -- 172.21.15.113:0/348559438 learned_addr learned my addr 172.21.15.113:0/348559438 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:59.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa97be00640 1 --1- 172.21.15.113:0/348559438 >> v1:172.21.15.136:6789/0 conn(0x7fa984070120 0x7fa984158930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:51234/0 (socket says 172.21.15.113:51234) 2024-03-04T08:20:59.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa982800640 1 --2- 172.21.15.113:0/348559438 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa984159040 0x7fa9840ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:59.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981e00640 1 -- 172.21.15.113:0/348559438 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa984159040 msgr2=0x7fa9840ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:20:59.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981e00640 1 --2- 172.21.15.113:0/348559438 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa984159040 0x7fa9840ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981e00640 1 -- 172.21.15.113:0/348559438 >> v1:172.21.15.136:6789/0 conn(0x7fa984070120 legacy=0x7fa984158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:59.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981e00640 1 -- 172.21.15.113:0/348559438 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9840c36e0 con 0x7fa984152d60 2024-03-04T08:20:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981e00640 1 --2- 172.21.15.113:0/348559438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984152d60 0x7fa9840ce310 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fa974009870 tx=0x7fa97401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37afe79b1c9fa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981400640 1 -- 172.21.15.113:0/348559438 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9740b5070 con 0x7fa984152d60 2024-03-04T08:20:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981400640 1 -- 172.21.15.113:0/348559438 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9740a8a50 con 0x7fa984152d60 2024-03-04T08:20:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.595+0000 7fa981400640 1 -- 172.21.15.113:0/348559438 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9740a8e20 con 0x7fa984152d60 2024-03-04T08:20:59.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 -- 172.21.15.113:0/348559438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984152d60 msgr2=0x7fa9840ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 --2- 172.21.15.113:0/348559438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984152d60 0x7fa9840ce310 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fa974009870 tx=0x7fa97401ed50 comp rx=0 tx=0).stop 2024-03-04T08:20:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 -- 172.21.15.113:0/348559438 shutdown_connections 2024-03-04T08:20:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 --2- 172.21.15.113:0/348559438 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa984159040 0x7fa9840ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 --2- 172.21.15.113:0/348559438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984152d60 0x7fa9840ce310 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 -- 172.21.15.113:0/348559438 >> 172.21.15.113:0/348559438 conn(0x7fa984066db0 msgr2=0x7fa9840671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 -- 172.21.15.113:0/348559438 shutdown_connections 2024-03-04T08:20:59.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 -- 172.21.15.113:0/348559438 wait complete. 2024-03-04T08:20:59.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 Processor -- start 2024-03-04T08:20:59.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 -- start start 2024-03-04T08:20:59.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984070120 0x7fa98406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa984152d60 0x7fa98406cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa984157df0 0x7fa9841e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840c39c0 con 0x7fa984070120 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840c5020 con 0x7fa984152d60 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa989200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840c3b40 con 0x7fa984157df0 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa981e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984070120 0x7fa98406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa981e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984070120 0x7fa98406d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52076/0 (socket says 172.21.15.113:52076) 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa981e00640 1 -- 172.21.15.113:0/4076045276 learned_addr learned my addr 172.21.15.113:0/4076045276 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa97be00640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa984152d60 0x7fa98406cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa982800640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa984157df0 0x7fa9841e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa981e00640 1 -- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa984157df0 msgr2=0x7fa9841e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:59.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa981e00640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa984157df0 0x7fa9841e3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa981e00640 1 -- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa984152d60 msgr2=0x7fa98406cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:59.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa981e00640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa984152d60 0x7fa98406cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.599+0000 7fa981e00640 1 -- 172.21.15.113:0/4076045276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9840c8510 con 0x7fa984070120 2024-03-04T08:20:59.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.603+0000 7fa981e00640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984070120 0x7fa98406d480 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa9740b3040 tx=0x7fa97401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:59.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.603+0000 7fa97a000640 1 -- 172.21.15.113:0/4076045276 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9740b5070 con 0x7fa984070120 2024-03-04T08:20:59.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.603+0000 7fa97a000640 1 -- 172.21.15.113:0/4076045276 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9740b0de0 con 0x7fa984070120 2024-03-04T08:20:59.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.603+0000 7fa989200640 1 -- 172.21.15.113:0/4076045276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9840c5900 con 0x7fa984070120 2024-03-04T08:20:59.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.603+0000 7fa97a000640 1 -- 172.21.15.113:0/4076045276 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9740bb670 con 0x7fa984070120 2024-03-04T08:20:59.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.603+0000 7fa989200640 1 -- 172.21.15.113:0/4076045276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9840c7510 con 0x7fa984070120 2024-03-04T08:20:59.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.603+0000 7fa989200640 1 -- 172.21.15.113:0/4076045276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98406ff00 con 0x7fa984070120 2024-03-04T08:20:59.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.603+0000 7fa97a000640 1 -- 172.21.15.113:0/4076045276 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9740c2060 con 0x7fa984070120 2024-03-04T08:20:59.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.603+0000 7fa97a000640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa960066f60 0x7fa960069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:20:59.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.603+0000 7fa97a000640 1 -- 172.21.15.113:0/4076045276 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fa974122ef0 con 0x7fa984070120 2024-03-04T08:20:59.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.607+0000 7fa97be00640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa960066f60 0x7fa960069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:20:59.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.607+0000 7fa97be00640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa960066f60 0x7fa960069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa9840c8810 tx=0x7fa96c007cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:20:59.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.607+0000 7fa97a000640 1 -- 172.21.15.113:0/4076045276 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa9740188b0 con 0x7fa984070120 2024-03-04T08:20:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 -- 172.21.15.113:0/4076045276 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa984075be0 con 0x7fa984070120 2024-03-04T08:20:59.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa97a000640 1 -- 172.21.15.113:0/4076045276 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7fa9740f69f0 con 0x7fa984070120 2024-03-04T08:20:59.800 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:20:59.800 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:20:59.800 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:20:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 -- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa960066f60 msgr2=0x7fa960069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa960066f60 0x7fa960069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa9840c8810 tx=0x7fa96c007cd0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 -- 172.21.15.113:0/4076045276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984070120 msgr2=0x7fa98406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:20:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984070120 0x7fa98406d480 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa9740b3040 tx=0x7fa97401baf0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 -- 172.21.15.113:0/4076045276 shutdown_connections 2024-03-04T08:20:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa984157df0 0x7fa9841e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa984152d60 0x7fa98406cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa960066f60 0x7fa960069420 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 --2- 172.21.15.113:0/4076045276 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa984070120 0x7fa98406d480 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:20:59.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 -- 172.21.15.113:0/4076045276 >> 172.21.15.113:0/4076045276 conn(0x7fa984066db0 msgr2=0x7fa984156740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:20:59.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 -- 172.21.15.113:0/4076045276 shutdown_connections 2024-03-04T08:20:59.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:20:59.803+0000 7fa989200640 1 -- 172.21.15.113:0/4076045276 wait complete. 2024-03-04T08:20:59.822 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:01.824 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:02.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.007+0000 7f19b0e00640 1 Processor -- start 2024-03-04T08:21:02.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.007+0000 7f19b0e00640 1 -- start start 2024-03-04T08:21:02.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19b0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19ac152b20 0x7f19ac070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:02.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19b0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19ac15c510 0x7f19ac163190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:02.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19b0e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ac152f00 con 0x7f19ac0c8b00 2024-03-04T08:21:02.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19b0e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ac0c8f90 con 0x7f19ac152b20 2024-03-04T08:21:02.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19b0e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0c5320 con 0x7f19ac15c510 2024-03-04T08:21:02.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19a9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19ac152b20 0x7f19ac070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:02.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19a9400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f19ac0c8b00 0x7f19ac15be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33832/0 (socket says 172.21.15.113:33832) 2024-03-04T08:21:02.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19a9400640 1 -- 172.21.15.113:0/1207184274 learned_addr learned my addr 172.21.15.113:0/1207184274 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:02.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19aa800640 1 --2- 172.21.15.113:0/1207184274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19ac15c510 0x7f19ac163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:02.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19a9e00640 1 -- 172.21.15.113:0/1207184274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19ac15c510 msgr2=0x7f19ac163190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:02.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19a9e00640 1 --2- 172.21.15.113:0/1207184274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19ac15c510 0x7f19ac163190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:02.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19a9e00640 1 -- 172.21.15.113:0/1207184274 >> v1:172.21.15.113:6789/0 conn(0x7f19ac0c8b00 legacy=0x7f19ac15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:02.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.011+0000 7f19a9e00640 1 -- 172.21.15.113:0/1207184274 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac0c5040 con 0x7f19ac152b20 2024-03-04T08:21:02.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19a9e00640 1 --2- 172.21.15.113:0/1207184274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac152b20 0x7f19ac070010 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f199c0097c0 tx=0x7f199c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575099fb7c19955c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:02.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19a8a00640 1 -- 172.21.15.113:0/1207184274 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c0b2070 con 0x7f19ac152b20 2024-03-04T08:21:02.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19a8a00640 1 -- 172.21.15.113:0/1207184274 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c0ad640 con 0x7f19ac152b20 2024-03-04T08:21:02.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19a8a00640 1 -- 172.21.15.113:0/1207184274 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c0ada10 con 0x7f19ac152b20 2024-03-04T08:21:02.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19b0e00640 1 -- 172.21.15.113:0/1207184274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac152b20 msgr2=0x7f19ac070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:02.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19b0e00640 1 --2- 172.21.15.113:0/1207184274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac152b20 0x7f19ac070010 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f199c0097c0 tx=0x7f199c01c4a0 comp rx=0 tx=0).stop 2024-03-04T08:21:02.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19b0e00640 1 -- 172.21.15.113:0/1207184274 shutdown_connections 2024-03-04T08:21:02.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19b0e00640 1 --2- 172.21.15.113:0/1207184274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19ac15c510 0x7f19ac163190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:02.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19b0e00640 1 --2- 172.21.15.113:0/1207184274 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac152b20 0x7f19ac070010 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:02.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19b0e00640 1 -- 172.21.15.113:0/1207184274 >> 172.21.15.113:0/1207184274 conn(0x7f19ac0709a0 msgr2=0x7f19ac070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:02.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19b0e00640 1 -- 172.21.15.113:0/1207184274 shutdown_connections 2024-03-04T08:21:02.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19b0e00640 1 -- 172.21.15.113:0/1207184274 wait complete. 2024-03-04T08:21:02.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19b0e00640 1 Processor -- start 2024-03-04T08:21:02.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.015+0000 7f19b0e00640 1 -- start start 2024-03-04T08:21:02.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19b0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19ac0c8b00 0x7f19ac153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:02.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19b0e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac152b20 0x7f19ac15a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19b0e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac15c510 0x7f19ac1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19b0e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac158810 con 0x7f19ac15c510 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19b0e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0c3810 con 0x7f19ac152b20 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19b0e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0c8490 con 0x7f19ac0c8b00 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19aa800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac15c510 0x7f19ac1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19aa800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac15c510 0x7f19ac1f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52104/0 (socket says 172.21.15.113:52104) 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19aa800640 1 -- 172.21.15.113:0/3326327517 learned_addr learned my addr 172.21.15.113:0/3326327517 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19a9e00640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19ac0c8b00 0x7f19ac153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19a9400640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac152b20 0x7f19ac15a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19aa800640 1 -- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19ac0c8b00 msgr2=0x7f19ac153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19aa800640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19ac0c8b00 0x7f19ac153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:02.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19aa800640 1 -- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac152b20 msgr2=0x7f19ac15a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:02.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19aa800640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac152b20 0x7f19ac15a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:02.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19aa800640 1 -- 172.21.15.113:0/3326327517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac15b720 con 0x7f19ac15c510 2024-03-04T08:21:02.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19a9400640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac152b20 0x7f19ac15a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:02.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19aa800640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac15c510 0x7f19ac1f4dd0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f19a400efc0 tx=0x7f19a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:02.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19a2000640 1 -- 172.21.15.113:0/3326327517 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4018020 con 0x7f19ac15c510 2024-03-04T08:21:02.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19a2000640 1 -- 172.21.15.113:0/3326327517 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4014480 con 0x7f19ac15c510 2024-03-04T08:21:02.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19b0e00640 1 -- 172.21.15.113:0/3326327517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac15ba00 con 0x7f19ac15c510 2024-03-04T08:21:02.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19a2000640 1 -- 172.21.15.113:0/3326327517 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4014870 con 0x7f19ac15c510 2024-03-04T08:21:02.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19b0e00640 1 -- 172.21.15.113:0/3326327517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac15bfc0 con 0x7f19ac15c510 2024-03-04T08:21:02.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.019+0000 7f19b0e00640 1 -- 172.21.15.113:0/3326327517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac075c00 con 0x7f19ac15c510 2024-03-04T08:21:02.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.023+0000 7f19a2000640 1 -- 172.21.15.113:0/3326327517 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19a401b020 con 0x7f19ac15c510 2024-03-04T08:21:02.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.023+0000 7f19a2000640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f198c066f60 0x7f198c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:02.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.023+0000 7f19a2000640 1 -- 172.21.15.113:0/3326327517 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f19a4086800 con 0x7f19ac15c510 2024-03-04T08:21:02.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.023+0000 7f19a9e00640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f198c066f60 0x7f198c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:02.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.023+0000 7f19a9e00640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f198c066f60 0x7f198c069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f199c0af000 tx=0x7f199c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:02.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.027+0000 7f19a2000640 1 -- 172.21.15.113:0/3326327517 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f19a4022020 con 0x7f19ac15c510 2024-03-04T08:21:02.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.271+0000 7f19b0e00640 1 -- 172.21.15.113:0/3326327517 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19ac06fdf0 con 0x7f19ac15c510 2024-03-04T08:21:02.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.271+0000 7f19a2000640 1 -- 172.21.15.113:0/3326327517 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7f19a4010040 con 0x7f19ac15c510 2024-03-04T08:21:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:02.271 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:02.271 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:21:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 -- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f198c066f60 msgr2=0x7f198c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f198c066f60 0x7f198c069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f199c0af000 tx=0x7f199c008040 comp rx=0 tx=0).stop 2024-03-04T08:21:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 -- 172.21.15.113:0/3326327517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac15c510 msgr2=0x7f19ac1f4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac15c510 0x7f19ac1f4dd0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f19a400efc0 tx=0x7f19a400c490 comp rx=0 tx=0).stop 2024-03-04T08:21:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 -- 172.21.15.113:0/3326327517 shutdown_connections 2024-03-04T08:21:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f198c066f60 0x7f198c069420 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19ac15c510 0x7f19ac1f4dd0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19ac152b20 0x7f19ac15a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:02.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 --2- 172.21.15.113:0/3326327517 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19ac0c8b00 0x7f19ac153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:02.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 -- 172.21.15.113:0/3326327517 >> 172.21.15.113:0/3326327517 conn(0x7f19ac0709a0 msgr2=0x7f19ac0c6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:02.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 -- 172.21.15.113:0/3326327517 shutdown_connections 2024-03-04T08:21:02.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:02.275+0000 7f19b0e00640 1 -- 172.21.15.113:0/3326327517 wait complete. 2024-03-04T08:21:02.293 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:04.294 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f0536000640 1 Processor -- start 2024-03-04T08:21:04.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f0536000640 1 -- start start 2024-03-04T08:21:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f0536000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f053014e4c0 0x7f0530072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f0536000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300c27f0 con 0x7f053015b900 2024-03-04T08:21:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f0536000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300c2a90 con 0x7f05300c22e0 2024-03-04T08:21:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f0536000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f053015ad60 con 0x7f053014e4c0 2024-03-04T08:21:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f052f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f053014e4c0 0x7f0530072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f052fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f053015b900 0x7f05301620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33836/0 (socket says 172.21.15.113:33836) 2024-03-04T08:21:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f052fe00640 1 -- 172.21.15.113:0/2929395258 learned_addr learned my addr 172.21.15.113:0/2929395258 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f052f400640 1 -- 172.21.15.113:0/2929395258 >> v1:172.21.15.136:6789/0 conn(0x7f05300c22e0 legacy=0x7f053015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f052f400640 1 -- 172.21.15.113:0/2929395258 >> v1:172.21.15.113:6789/0 conn(0x7f053015b900 legacy=0x7f05301620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f052f400640 1 -- 172.21.15.113:0/2929395258 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530070c60 con 0x7f053014e4c0 2024-03-04T08:21:04.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.435+0000 7f052e000640 1 -- 172.21.15.113:0/2929395258 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143529659 0 0) 0x7f05300c27f0 con 0x7f053015b900 2024-03-04T08:21:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f052f400640 1 --2- 172.21.15.113:0/2929395258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 0x7f0530072c80 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f05200097c0 tx=0x7f052001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac6c40793c73d08 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f052e000640 1 -- 172.21.15.113:0/2929395258 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200b5070 con 0x7f053014e4c0 2024-03-04T08:21:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f052e000640 1 -- 172.21.15.113:0/2929395258 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05200a8990 con 0x7f053014e4c0 2024-03-04T08:21:04.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f052e000640 1 -- 172.21.15.113:0/2929395258 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200a8d60 con 0x7f053014e4c0 2024-03-04T08:21:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f0536000640 1 -- 172.21.15.113:0/2929395258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 msgr2=0x7f0530072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f0536000640 1 --2- 172.21.15.113:0/2929395258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 0x7f0530072c80 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f05200097c0 tx=0x7f052001ec90 comp rx=0 tx=0).stop 2024-03-04T08:21:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f0536000640 1 -- 172.21.15.113:0/2929395258 shutdown_connections 2024-03-04T08:21:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f0536000640 1 --2- 172.21.15.113:0/2929395258 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 0x7f0530072c80 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f0536000640 1 -- 172.21.15.113:0/2929395258 >> 172.21.15.113:0/2929395258 conn(0x7f053006e820 msgr2=0x7f053006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f0536000640 1 -- 172.21.15.113:0/2929395258 shutdown_connections 2024-03-04T08:21:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f0536000640 1 -- 172.21.15.113:0/2929395258 wait complete. 2024-03-04T08:21:04.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.439+0000 7f0536000640 1 Processor -- start 2024-03-04T08:21:04.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0536000640 1 -- start start 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0536000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05300c22e0 0x7f0530158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0536000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 0x7f0530137f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0536000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053015b900 0x7f05301384a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0536000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530071500 con 0x7f053014e4c0 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0536000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530071220 con 0x7f053015b900 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0536000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053014e8a0 con 0x7f05300c22e0 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 0x7f0530137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052f400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05300c22e0 0x7f0530158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 0x7f0530137f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52134/0 (socket says 172.21.15.113:52134) 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052ea00640 1 -- 172.21.15.113:0/2135717465 learned_addr learned my addr 172.21.15.113:0/2135717465 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:04.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052fe00640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053015b900 0x7f05301384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052ea00640 1 -- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05300c22e0 msgr2=0x7f0530158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052ea00640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05300c22e0 0x7f0530158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052ea00640 1 -- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053015b900 msgr2=0x7f05301384a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052ea00640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053015b900 0x7f05301384a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052ea00640 1 -- 172.21.15.113:0/2135717465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053015a3f0 con 0x7f053014e4c0 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f052ea00640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 0x7f0530137f60 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f051c00c3f0 tx=0x7f051c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0527400640 1 -- 172.21.15.113:0/2135717465 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c017070 con 0x7f053014e4c0 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0527400640 1 -- 172.21.15.113:0/2135717465 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c0052d0 con 0x7f053014e4c0 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0536000640 1 -- 172.21.15.113:0/2135717465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05301f42e0 con 0x7f053014e4c0 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0527400640 1 -- 172.21.15.113:0/2135717465 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c003840 con 0x7f053014e4c0 2024-03-04T08:21:04.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0536000640 1 -- 172.21.15.113:0/2135717465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05301f48a0 con 0x7f053014e4c0 2024-03-04T08:21:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.443+0000 7f0536000640 1 -- 172.21.15.113:0/2135717465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0530075c00 con 0x7f053014e4c0 2024-03-04T08:21:04.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.447+0000 7f0527400640 1 -- 172.21.15.113:0/2135717465 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f051c01a070 con 0x7f053014e4c0 2024-03-04T08:21:04.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.447+0000 7f0527400640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0510066f90 0x7f0510069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:04.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.447+0000 7f0527400640 1 -- 172.21.15.113:0/2135717465 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f051c084c80 con 0x7f053014e4c0 2024-03-04T08:21:04.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.447+0000 7f052f400640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0510066f90 0x7f0510069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:04.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.447+0000 7f052f400640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0510066f90 0x7f0510069450 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f05200b2000 tx=0x7f0520008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:04.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.451+0000 7f0527400640 1 -- 172.21.15.113:0/2135717465 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f051c005480 con 0x7f053014e4c0 2024-03-04T08:21:04.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.651+0000 7f0536000640 1 -- 172.21.15.113:0/2135717465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0530159910 con 0x7f053014e4c0 2024-03-04T08:21:04.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.651+0000 7f0527400640 1 -- 172.21.15.113:0/2135717465 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7f051c054cc0 con 0x7f053014e4c0 2024-03-04T08:21:04.650 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:04.651 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:04.651 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 -- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0510066f90 msgr2=0x7f0510069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0510066f90 0x7f0510069450 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f05200b2000 tx=0x7f0520008040 comp rx=0 tx=0).stop 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 -- 172.21.15.113:0/2135717465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 msgr2=0x7f0530137f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 0x7f0530137f60 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f051c00c3f0 tx=0x7f051c00c8b0 comp rx=0 tx=0).stop 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 -- 172.21.15.113:0/2135717465 shutdown_connections 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f053015b900 0x7f05301384a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0510066f90 0x7f0510069450 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f053014e4c0 0x7f0530137f60 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 --2- 172.21.15.113:0/2135717465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f05300c22e0 0x7f0530158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 -- 172.21.15.113:0/2135717465 >> 172.21.15.113:0/2135717465 conn(0x7f053006e820 msgr2=0x7f0530144200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 -- 172.21.15.113:0/2135717465 shutdown_connections 2024-03-04T08:21:04.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:04.655+0000 7f0536000640 1 -- 172.21.15.113:0/2135717465 wait complete. 2024-03-04T08:21:04.672 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:21:04.673 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 170, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 170, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8253": { "addr": "172.21.15.113:6836/893414524", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 893414524, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 893414524, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8253, "incarnation": 163, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:stopping", "state_seq": 35 } }, "last_failure": 0, "last_failure_osd_epoch": 80, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:20:52.211831+0000", "qdb_cluster": [ 7839, 8032 ], "qdb_leader": 7839, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 8032, "mds_2": 8253 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.113:6833/2999156262", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2999156262, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2999156262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 168, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8388, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:21:06.673 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:06.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.811+0000 7fec81800640 1 Processor -- start 2024-03-04T08:21:06.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.811+0000 7fec81800640 1 -- start start 2024-03-04T08:21:06.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.811+0000 7fec81800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec7c152bd0 0x7fec7c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:06.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.811+0000 7fec81800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7c1591f0 con 0x7fec7c1527f0 2024-03-04T08:21:06.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.811+0000 7fec81800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7c1594d0 con 0x7fec7c0c8200 2024-03-04T08:21:06.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.811+0000 7fec81800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c071df0 con 0x7fec7c152bd0 2024-03-04T08:21:06.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.811+0000 7fec7aa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fec7c1527f0 0x7fec7c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33838/0 (socket says 172.21.15.113:33838) 2024-03-04T08:21:06.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.811+0000 7fec7aa00640 1 -- 172.21.15.113:0/2224327362 learned_addr learned my addr 172.21.15.113:0/2224327362 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:06.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.811+0000 7fec7b400640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec7c152bd0 0x7fec7c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:06.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.811+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807534077 0 0) 0x7fec7c1591f0 con 0x7fec7c1527f0 2024-03-04T08:21:06.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec60003530 con 0x7fec7c1527f0 2024-03-04T08:21:06.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186906519 0 0) 0x7fec7c1594d0 con 0x7fec7c0c8200 2024-03-04T08:21:06.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec7c1591f0 con 0x7fec7c0c8200 2024-03-04T08:21:06.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2992977459 0 0) 0x7fec60003530 con 0x7fec7c1527f0 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec7c152bd0 msgr2=0x7fec7c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec7c152bd0 0x7fec7c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 >> v1:172.21.15.136:6789/0 conn(0x7fec7c0c8200 legacy=0x7fec7c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c0717a0 con 0x7fec7c1527f0 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fec6c001680 con 0x7fec7c1527f0 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 >> v1:172.21.15.113:6789/0 conn(0x7fec7c1527f0 legacy=0x7fec7c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec7c152bd0 0x7fec60003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec600040e0 0x7fec60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec60004d50 0x7fec600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c071df0 con 0x7fec600040e0 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6000bab0 con 0x7fec60004d50 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6000bcb0 con 0x7fec7c152bd0 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec73e00640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec600040e0 0x7fec60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:06.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7aa00640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec7c152bd0 0x7fec60003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7b400640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec60004d50 0x7fec600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec73e00640 1 -- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec7c152bd0 msgr2=0x7fec60003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec73e00640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec7c152bd0 0x7fec60003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec73e00640 1 -- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec60004d50 msgr2=0x7fec600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec73e00640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec60004d50 0x7fec600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec73e00640 1 -- 172.21.15.113:0/2224327362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60003530 con 0x7fec600040e0 2024-03-04T08:21:06.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec7b400640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec60004d50 0x7fec600051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:06.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.815+0000 7fec73e00640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec600040e0 0x7fec60004810 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0c85e0 tx=0x7fec6401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d437695d32f2b7b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:06.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64001520 con 0x7fec600040e0 2024-03-04T08:21:06.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec640052f0 con 0x7fec600040e0 2024-03-04T08:21:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec7a000640 1 -- 172.21.15.113:0/2224327362 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec640a8930 con 0x7fec600040e0 2024-03-04T08:21:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 -- 172.21.15.113:0/2224327362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec600040e0 msgr2=0x7fec60004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 --2- 172.21.15.113:0/2224327362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec600040e0 0x7fec60004810 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0c85e0 tx=0x7fec6401ecd0 comp rx=0 tx=0).stop 2024-03-04T08:21:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec7b400640 1 -- 172.21.15.113:0/2224327362 reap_dead start 2024-03-04T08:21:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 -- 172.21.15.113:0/2224327362 shutdown_connections 2024-03-04T08:21:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 -- 172.21.15.113:0/2224327362 >> 172.21.15.113:0/2224327362 conn(0x7fec7c06e820 msgr2=0x7fec7c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 -- 172.21.15.113:0/2224327362 shutdown_connections 2024-03-04T08:21:06.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 -- 172.21.15.113:0/2224327362 wait complete. 2024-03-04T08:21:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 Processor -- start 2024-03-04T08:21:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 -- start start 2024-03-04T08:21:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec600040e0 0x7fec7c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:06.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec7c0c2a50 0x7fec7c156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c0c5430 0x7fec7c148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c158ee0 con 0x7fec7c0c5430 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c1597b0 con 0x7fec7c0c2a50 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec81800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c070d40 con 0x7fec600040e0 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec7b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c0c5430 0x7fec7c148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec7b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c0c5430 0x7fec7c148c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52140/0 (socket says 172.21.15.113:52140) 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec7b400640 1 -- 172.21.15.113:0/3167935863 learned_addr learned my addr 172.21.15.113:0/3167935863 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec73e00640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec7c0c2a50 0x7fec7c156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.819+0000 7fec7aa00640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec600040e0 0x7fec7c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec7b400640 1 -- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec600040e0 msgr2=0x7fec7c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec7b400640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec600040e0 0x7fec7c0b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec7b400640 1 -- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec7c0c2a50 msgr2=0x7fec7c156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:06.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec7b400640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec7c0c2a50 0x7fec7c156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec7b400640 1 -- 172.21.15.113:0/3167935863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c0b1920 con 0x7fec7c0c5430 2024-03-04T08:21:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec7aa00640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec600040e0 0x7fec7c0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec73e00640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec7c0c2a50 0x7fec7c156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec7b400640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c0c5430 0x7fec7c148c80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0c9c70 tx=0x7fec74010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec72a00640 1 -- 172.21.15.113:0/3167935863 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74010da0 con 0x7fec7c0c5430 2024-03-04T08:21:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec72a00640 1 -- 172.21.15.113:0/3167935863 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec740039a0 con 0x7fec7c0c5430 2024-03-04T08:21:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec81800640 1 -- 172.21.15.113:0/3167935863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c0b0da0 con 0x7fec7c0c5430 2024-03-04T08:21:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec72a00640 1 -- 172.21.15.113:0/3167935863 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74003d90 con 0x7fec7c0c5430 2024-03-04T08:21:06.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec81800640 1 -- 172.21.15.113:0/3167935863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c0b1640 con 0x7fec7c0c5430 2024-03-04T08:21:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec81800640 1 -- 172.21.15.113:0/3167935863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c075c00 con 0x7fec7c0c5430 2024-03-04T08:21:06.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.823+0000 7fec72a00640 1 -- 172.21.15.113:0/3167935863 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec7401e020 con 0x7fec7c0c5430 2024-03-04T08:21:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.827+0000 7fec72a00640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec58066f90 0x7fec58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.827+0000 7fec72a00640 1 -- 172.21.15.113:0/3167935863 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fec74086070 con 0x7fec7c0c5430 2024-03-04T08:21:06.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.827+0000 7fec7aa00640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec58066f90 0x7fec58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:06.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.827+0000 7fec7aa00640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec58066f90 0x7fec58069450 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c10 tx=0x7fec6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:06.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:06.827+0000 7fec72a00640 1 -- 172.21.15.113:0/3167935863 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fec74025020 con 0x7fec7c0c5430 2024-03-04T08:21:07.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.019+0000 7fec81800640 1 -- 172.21.15.113:0/3167935863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec7c0c63d0 con 0x7fec7c0c5430 2024-03-04T08:21:07.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.019+0000 7fec72a00640 1 -- 172.21.15.113:0/3167935863 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7fec74056670 con 0x7fec7c0c5430 2024-03-04T08:21:07.017 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:07.017 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:07.017 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:21:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 -- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec58066f90 msgr2=0x7fec58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec58066f90 0x7fec58069450 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c10 tx=0x7fec6c009040 comp rx=0 tx=0).stop 2024-03-04T08:21:07.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 -- 172.21.15.113:0/3167935863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c0c5430 msgr2=0x7fec7c148c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c0c5430 0x7fec7c148c80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0c9c70 tx=0x7fec74010590 comp rx=0 tx=0).stop 2024-03-04T08:21:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 -- 172.21.15.113:0/3167935863 shutdown_connections 2024-03-04T08:21:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec58066f90 0x7fec58069450 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec7c0c5430 0x7fec7c148c80 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec7c0c2a50 0x7fec7c156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 --2- 172.21.15.113:0/3167935863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec600040e0 0x7fec7c0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 -- 172.21.15.113:0/3167935863 >> 172.21.15.113:0/3167935863 conn(0x7fec7c06e820 msgr2=0x7fec7c157ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 -- 172.21.15.113:0/3167935863 shutdown_connections 2024-03-04T08:21:07.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:07.023+0000 7fec81800640 1 -- 172.21.15.113:0/3167935863 wait complete. 2024-03-04T08:21:07.046 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:09.048 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.187+0000 7fe785400640 1 Processor -- start 2024-03-04T08:21:09.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.187+0000 7fe785400640 1 -- start start 2024-03-04T08:21:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.187+0000 7fe785400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe780152bd0 0x7fe7800c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.187+0000 7fe785400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe7800c9df0 0x7fe7800ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.187+0000 7fe785400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7800c9490 con 0x7fe7800c4fd0 2024-03-04T08:21:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.187+0000 7fe785400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7800c9c50 con 0x7fe780152bd0 2024-03-04T08:21:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.187+0000 7fe785400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7800c97c0 con 0x7fe7800c9df0 2024-03-04T08:21:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.187+0000 7fe77e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe780152bd0 0x7fe7800c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.187+0000 7fe777e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe7800c4fd0 0x7fe780158980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59816/0 (socket says 172.21.15.113:59816) 2024-03-04T08:21:09.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.187+0000 7fe777e00640 1 -- 172.21.15.113:0/2327875861 learned_addr learned my addr 172.21.15.113:0/2327875861 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:09.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.191+0000 7fe77f200640 1 --2- 172.21.15.113:0/2327875861 >> v2:172.21.15.136:3300/0 conn(0x7fe7800c9df0 0x7fe7800ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:09.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.191+0000 7fe77e800640 1 -- 172.21.15.113:0/2327875861 >> v2:172.21.15.136:3300/0 conn(0x7fe7800c9df0 msgr2=0x7fe7800ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:09.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.191+0000 7fe77e800640 1 --2- 172.21.15.113:0/2327875861 >> v2:172.21.15.136:3300/0 conn(0x7fe7800c9df0 0x7fe7800ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:09.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.191+0000 7fe77e800640 1 -- 172.21.15.113:0/2327875861 >> v1:172.21.15.113:6789/0 conn(0x7fe7800c4fd0 legacy=0x7fe780158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:09.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.191+0000 7fe77e800640 1 -- 172.21.15.113:0/2327875861 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7800c54a0 con 0x7fe780152bd0 2024-03-04T08:21:09.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.191+0000 7fe77de00640 1 -- 172.21.15.113:0/2327875861 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294532703 0 0) 0x7fe7800c9490 con 0x7fe7800c4fd0 2024-03-04T08:21:09.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.191+0000 7fe77e800640 1 --2- 172.21.15.113:0/2327875861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe780152bd0 0x7fe7800c8d10 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe7800c89d0 tx=0x7fe77001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9531e8d6aa957615 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.191+0000 7fe77de00640 1 -- 172.21.15.113:0/2327875861 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700b2070 con 0x7fe780152bd0 2024-03-04T08:21:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.191+0000 7fe77de00640 1 -- 172.21.15.113:0/2327875861 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7700a9070 con 0x7fe780152bd0 2024-03-04T08:21:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.191+0000 7fe77de00640 1 -- 172.21.15.113:0/2327875861 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700ad880 con 0x7fe780152bd0 2024-03-04T08:21:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 -- 172.21.15.113:0/2327875861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe780152bd0 msgr2=0x7fe7800c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 --2- 172.21.15.113:0/2327875861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe780152bd0 0x7fe7800c8d10 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe7800c89d0 tx=0x7fe77001c450 comp rx=0 tx=0).stop 2024-03-04T08:21:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 -- 172.21.15.113:0/2327875861 shutdown_connections 2024-03-04T08:21:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 --2- 172.21.15.113:0/2327875861 >> v2:172.21.15.136:3300/0 conn(0x7fe7800c9df0 0x7fe7800ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:09.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 --2- 172.21.15.113:0/2327875861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe780152bd0 0x7fe7800c8d10 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 -- 172.21.15.113:0/2327875861 >> 172.21.15.113:0/2327875861 conn(0x7fe780066db0 msgr2=0x7fe7800671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 -- 172.21.15.113:0/2327875861 shutdown_connections 2024-03-04T08:21:09.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 -- 172.21.15.113:0/2327875861 wait complete. 2024-03-04T08:21:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 Processor -- start 2024-03-04T08:21:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 -- start start 2024-03-04T08:21:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7800c4fd0 0x7fe780162590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7800c9df0 0x7fe7801f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe780152bd0 0x7fe780150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800c3b10 con 0x7fe7800c4fd0 2024-03-04T08:21:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7800c3990 con 0x7fe7800c9df0 2024-03-04T08:21:09.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe780152fb0 con 0x7fe780152bd0 2024-03-04T08:21:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe77e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7800c4fd0 0x7fe780162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe77e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7800c4fd0 0x7fe780162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42302/0 (socket says 172.21.15.113:42302) 2024-03-04T08:21:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe77e800640 1 -- 172.21.15.113:0/912448979 learned_addr learned my addr 172.21.15.113:0/912448979 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe77f200640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe780152bd0 0x7fe780150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe777e00640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7800c9df0 0x7fe7801f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe77e800640 1 -- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe780152bd0 msgr2=0x7fe780150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe77e800640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe780152bd0 0x7fe780150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe77e800640 1 -- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7800c9df0 msgr2=0x7fe7801f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:09.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe77e800640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7800c9df0 0x7fe7801f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe77e800640 1 -- 172.21.15.113:0/912448979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78015c920 con 0x7fe7800c4fd0 2024-03-04T08:21:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe77e800640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7800c4fd0 0x7fe780162590 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b0040 tx=0x7fe77001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe776a00640 1 -- 172.21.15.113:0/912448979 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700bf070 con 0x7fe7800c4fd0 2024-03-04T08:21:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe776a00640 1 -- 172.21.15.113:0/912448979 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7700add80 con 0x7fe7800c4fd0 2024-03-04T08:21:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe785400640 1 -- 172.21.15.113:0/912448979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7801f3ea0 con 0x7fe7800c4fd0 2024-03-04T08:21:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.195+0000 7fe776a00640 1 -- 172.21.15.113:0/912448979 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700b2040 con 0x7fe7800c4fd0 2024-03-04T08:21:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.199+0000 7fe785400640 1 -- 172.21.15.113:0/912448979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7801f7bb0 con 0x7fe7800c4fd0 2024-03-04T08:21:09.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.199+0000 7fe785400640 1 -- 172.21.15.113:0/912448979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe780075c00 con 0x7fe7800c4fd0 2024-03-04T08:21:09.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.199+0000 7fe776a00640 1 -- 172.21.15.113:0/912448979 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7700c2070 con 0x7fe7800c4fd0 2024-03-04T08:21:09.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.199+0000 7fe776a00640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe75c066f60 0x7fe75c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:09.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.199+0000 7fe776a00640 1 -- 172.21.15.113:0/912448979 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fe77011f580 con 0x7fe7800c4fd0 2024-03-04T08:21:09.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.199+0000 7fe777e00640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe75c066f60 0x7fe75c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:09.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.199+0000 7fe777e00640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe75c066f60 0x7fe75c069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe78006fcb0 tx=0x7fe768009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:09.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.203+0000 7fe776a00640 1 -- 172.21.15.113:0/912448979 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe78015c920 con 0x7fe7800c4fd0 2024-03-04T08:21:09.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.395+0000 7fe785400640 1 -- 172.21.15.113:0/912448979 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7800cb930 con 0x7fe7800c4fd0 2024-03-04T08:21:09.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.395+0000 7fe776a00640 1 -- 172.21.15.113:0/912448979 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7fe770016070 con 0x7fe7800c4fd0 2024-03-04T08:21:09.394 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:09.394 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:09.394 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 -- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe75c066f60 msgr2=0x7fe75c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe75c066f60 0x7fe75c069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe78006fcb0 tx=0x7fe768009040 comp rx=0 tx=0).stop 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 -- 172.21.15.113:0/912448979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7800c4fd0 msgr2=0x7fe780162590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7800c4fd0 0x7fe780162590 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b0040 tx=0x7fe77001ac90 comp rx=0 tx=0).stop 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 -- 172.21.15.113:0/912448979 shutdown_connections 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe780152bd0 0x7fe780150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7800c9df0 0x7fe7801f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe75c066f60 0x7fe75c069420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 --2- 172.21.15.113:0/912448979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7800c4fd0 0x7fe780162590 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 -- 172.21.15.113:0/912448979 >> 172.21.15.113:0/912448979 conn(0x7fe780066db0 msgr2=0x7fe7800cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 -- 172.21.15.113:0/912448979 shutdown_connections 2024-03-04T08:21:09.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:09.399+0000 7fe785400640 1 -- 172.21.15.113:0/912448979 wait complete. 2024-03-04T08:21:09.416 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:11.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:11.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21e5c00640 1 Processor -- start 2024-03-04T08:21:11.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21e5c00640 1 -- start start 2024-03-04T08:21:11.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21e5c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f21e00c8200 0x7f21e0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:11.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21e5c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e0152bd0 0x7f21e01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:11.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21e5c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e01591f0 con 0x7f21e01527f0 2024-03-04T08:21:11.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21e5c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e0071df0 con 0x7f21e00c8200 2024-03-04T08:21:11.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21e5c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e01597b0 con 0x7f21e0152bd0 2024-03-04T08:21:11.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21dea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f21e01527f0 0x7f21e0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59818/0 (socket says 172.21.15.113:59818) 2024-03-04T08:21:11.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21dea00640 1 -- 172.21.15.113:0/967857308 learned_addr learned my addr 172.21.15.113:0/967857308 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:11.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21d7e00640 1 --2- 172.21.15.113:0/967857308 >> v2:172.21.15.136:3300/0 conn(0x7f21e00c8200 0x7f21e0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:11.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21df400640 1 --2- 172.21.15.113:0/967857308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e0152bd0 0x7f21e01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:11.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.555+0000 7f21de000640 1 -- 172.21.15.113:0/967857308 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008032556 0 0) 0x7f21e01591f0 con 0x7f21e01527f0 2024-03-04T08:21:11.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.559+0000 7f21de000640 1 -- 172.21.15.113:0/967857308 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c4003530 con 0x7f21e01527f0 2024-03-04T08:21:11.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.559+0000 7f21d7e00640 1 -- 172.21.15.113:0/967857308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e0152bd0 msgr2=0x7f21e01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:11.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.559+0000 7f21d7e00640 1 --2- 172.21.15.113:0/967857308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e0152bd0 0x7f21e01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:11.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.559+0000 7f21d7e00640 1 -- 172.21.15.113:0/967857308 >> v1:172.21.15.113:6789/0 conn(0x7f21e01527f0 legacy=0x7f21e0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:11.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.559+0000 7f21d7e00640 1 -- 172.21.15.113:0/967857308 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e01594d0 con 0x7f21e00c8200 2024-03-04T08:21:11.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.559+0000 7f21d7e00640 1 --2- 172.21.15.113:0/967857308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e00c8200 0x7f21e0070d40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f21c80097c0 tx=0x7f21c801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ed1e0c10eafc6c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:11.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.559+0000 7f21de000640 1 -- 172.21.15.113:0/967857308 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c80b1070 con 0x7f21e00c8200 2024-03-04T08:21:11.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.559+0000 7f21de000640 1 -- 172.21.15.113:0/967857308 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c80ac410 con 0x7f21e00c8200 2024-03-04T08:21:11.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.559+0000 7f21de000640 1 -- 172.21.15.113:0/967857308 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c80ac7e0 con 0x7f21e00c8200 2024-03-04T08:21:11.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 -- 172.21.15.113:0/967857308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e00c8200 msgr2=0x7f21e0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:11.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 --2- 172.21.15.113:0/967857308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e00c8200 0x7f21e0070d40 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f21c80097c0 tx=0x7f21c801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:21:11.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 -- 172.21.15.113:0/967857308 shutdown_connections 2024-03-04T08:21:11.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 --2- 172.21.15.113:0/967857308 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e0152bd0 0x7f21e01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:11.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 --2- 172.21.15.113:0/967857308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e00c8200 0x7f21e0070d40 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:11.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 -- 172.21.15.113:0/967857308 >> 172.21.15.113:0/967857308 conn(0x7f21e006e820 msgr2=0x7f21e006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:11.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 -- 172.21.15.113:0/967857308 shutdown_connections 2024-03-04T08:21:11.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 -- 172.21.15.113:0/967857308 wait complete. 2024-03-04T08:21:11.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 Processor -- start 2024-03-04T08:21:11.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 -- start start 2024-03-04T08:21:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e00c8200 0x7f21e00cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e01527f0 0x7f21e01514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e0152bd0 0x7f21e0161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e01654f0 con 0x7f21e0152bd0 2024-03-04T08:21:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e0165370 con 0x7f21e00c8200 2024-03-04T08:21:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21e5c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e0165670 con 0x7f21e01527f0 2024-03-04T08:21:11.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21df400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e0152bd0 0x7f21e0161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21df400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e0152bd0 0x7f21e0161370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42318/0 (socket says 172.21.15.113:42318) 2024-03-04T08:21:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21dea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e00c8200 0x7f21e00cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21df400640 1 -- 172.21.15.113:0/3256713884 learned_addr learned my addr 172.21.15.113:0/3256713884 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21d7e00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e01527f0 0x7f21e01514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21df400640 1 -- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e01527f0 msgr2=0x7f21e01514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21df400640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e01527f0 0x7f21e01514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21df400640 1 -- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e00c8200 msgr2=0x7f21e00cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21df400640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e00c8200 0x7f21e00cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21df400640 1 -- 172.21.15.113:0/3256713884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e0158ac0 con 0x7f21e0152bd0 2024-03-04T08:21:11.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21d7e00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e01527f0 0x7f21e01514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:11.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.563+0000 7f21df400640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e0152bd0 0x7f21e0161370 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f21d800efc0 tx=0x7f21d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:11.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.567+0000 7f21d6a00640 1 -- 172.21.15.113:0/3256713884 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8018020 con 0x7f21e0152bd0 2024-03-04T08:21:11.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.567+0000 7f21d6a00640 1 -- 172.21.15.113:0/3256713884 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d8014480 con 0x7f21e0152bd0 2024-03-04T08:21:11.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.567+0000 7f21d6a00640 1 -- 172.21.15.113:0/3256713884 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8014870 con 0x7f21e0152bd0 2024-03-04T08:21:11.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.567+0000 7f21e5c00640 1 -- 172.21.15.113:0/3256713884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e01f3e00 con 0x7f21e0152bd0 2024-03-04T08:21:11.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.567+0000 7f21e5c00640 1 -- 172.21.15.113:0/3256713884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e01f43c0 con 0x7f21e0152bd0 2024-03-04T08:21:11.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.567+0000 7f21e5c00640 1 -- 172.21.15.113:0/3256713884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e0075c00 con 0x7f21e0152bd0 2024-03-04T08:21:11.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.567+0000 7f21d6a00640 1 -- 172.21.15.113:0/3256713884 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21d801b020 con 0x7f21e0152bd0 2024-03-04T08:21:11.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.567+0000 7f21d6a00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f90 0x7f21bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:11.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.567+0000 7f21d6a00640 1 -- 172.21.15.113:0/3256713884 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f21d8086580 con 0x7f21e0152bd0 2024-03-04T08:21:11.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.567+0000 7f21dea00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f90 0x7f21bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:11.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.571+0000 7f21dea00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f90 0x7f21bc069450 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f21d0007fd0 tx=0x7f21d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:11.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.571+0000 7f21d6a00640 1 -- 172.21.15.113:0/3256713884 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f21d8022020 con 0x7f21e0152bd0 2024-03-04T08:21:11.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 -- 172.21.15.113:0/3256713884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21e00c6820 con 0x7f21e0152bd0 2024-03-04T08:21:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21d6a00640 1 -- 172.21.15.113:0/3256713884 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7f21d8010040 con 0x7f21e0152bd0 2024-03-04T08:21:11.772 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:11.772 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:11.772 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:21:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 -- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f90 msgr2=0x7f21bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f90 0x7f21bc069450 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f21d0007fd0 tx=0x7f21d0009040 comp rx=0 tx=0).stop 2024-03-04T08:21:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 -- 172.21.15.113:0/3256713884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e0152bd0 msgr2=0x7f21e0161370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e0152bd0 0x7f21e0161370 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f21d800efc0 tx=0x7f21d800c490 comp rx=0 tx=0).stop 2024-03-04T08:21:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 -- 172.21.15.113:0/3256713884 shutdown_connections 2024-03-04T08:21:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f21bc066f90 0x7f21bc069450 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:11.774 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21e0152bd0 0x7f21e0161370 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f21e01527f0 0x7f21e01514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 --2- 172.21.15.113:0/3256713884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21e00c8200 0x7f21e00cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 -- 172.21.15.113:0/3256713884 >> 172.21.15.113:0/3256713884 conn(0x7f21e006e820 msgr2=0x7f21e01565e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 -- 172.21.15.113:0/3256713884 shutdown_connections 2024-03-04T08:21:11.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:11.775+0000 7f21e5c00640 1 -- 172.21.15.113:0/3256713884 wait complete. 2024-03-04T08:21:11.794 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:13.794 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:13.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0942400640 1 Processor -- start 2024-03-04T08:21:13.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0942400640 1 -- start start 2024-03-04T08:21:13.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0942400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f093c154fe0 0x7f093c0c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:13.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0942400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:13.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0942400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c0c7d80 con 0x7f093c154c00 2024-03-04T08:21:13.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0942400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c15efb0 con 0x7f093c154fe0 2024-03-04T08:21:13.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0942400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0c6b70 con 0x7f093c0c7700 2024-03-04T08:21:13.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f093be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:13.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0933e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f093c154fe0 0x7f093c0c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:13.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f093b400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f093c154c00 0x7f093c0c1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:45824/0 (socket says 172.21.15.113:45824) 2024-03-04T08:21:13.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f093be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c15abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:36146/0 (socket says 172.21.15.113:36146) 2024-03-04T08:21:13.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f093b400640 1 -- 172.21.15.113:0/1104696570 learned_addr learned my addr 172.21.15.113:0/1104696570 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:13.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0933e00640 1 -- 172.21.15.113:0/1104696570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 msgr2=0x7f093c15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:13.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0933e00640 1 --2- 172.21.15.113:0/1104696570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c15abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:13.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0933e00640 1 -- 172.21.15.113:0/1104696570 >> v1:172.21.15.136:6789/0 conn(0x7f093c154c00 legacy=0x7f093c0c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:13.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f0933e00640 1 -- 172.21.15.113:0/1104696570 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c0c8060 con 0x7f093c154fe0 2024-03-04T08:21:13.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.935+0000 7f093be00640 1 --2- 172.21.15.113:0/1104696570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:21:13.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f0933e00640 1 --2- 172.21.15.113:0/1104696570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f093c154fe0 0x7f093c0c1190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f09240097c0 tx=0x7f092401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e8b8e571e3007e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:13.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f093aa00640 1 -- 172.21.15.113:0/1104696570 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240b0070 con 0x7f093c154fe0 2024-03-04T08:21:13.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f093aa00640 1 -- 172.21.15.113:0/1104696570 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09240ab600 con 0x7f093c154fe0 2024-03-04T08:21:13.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f093aa00640 1 -- 172.21.15.113:0/1104696570 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240ab9d0 con 0x7f093c154fe0 2024-03-04T08:21:13.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f0942400640 1 -- 172.21.15.113:0/1104696570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f093c154fe0 msgr2=0x7f093c0c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:13.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f0942400640 1 --2- 172.21.15.113:0/1104696570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f093c154fe0 0x7f093c0c1190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f09240097c0 tx=0x7f092401c410 comp rx=0 tx=0).stop 2024-03-04T08:21:13.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f0942400640 1 -- 172.21.15.113:0/1104696570 shutdown_connections 2024-03-04T08:21:13.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f0942400640 1 --2- 172.21.15.113:0/1104696570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:13.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f0942400640 1 --2- 172.21.15.113:0/1104696570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f093c154fe0 0x7f093c0c1190 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:13.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f0942400640 1 -- 172.21.15.113:0/1104696570 >> 172.21.15.113:0/1104696570 conn(0x7f093c06e820 msgr2=0x7f093c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:13.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f0942400640 1 -- 172.21.15.113:0/1104696570 shutdown_connections 2024-03-04T08:21:13.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.939+0000 7f0942400640 1 -- 172.21.15.113:0/1104696570 wait complete. 2024-03-04T08:21:13.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0942400640 1 Processor -- start 2024-03-04T08:21:13.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0942400640 1 -- start start 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0942400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c0b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0942400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f093c154c00 0x7f093c152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0942400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093c154fe0 0x7f093c153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0942400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0c23e0 con 0x7f093c154fe0 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0942400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c0c2260 con 0x7f093c154c00 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0942400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c167c10 con 0x7f093c0c7700 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093c154fe0 0x7f093c153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c0b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c0b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:36162/0 (socket says 172.21.15.113:36162) 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093c154fe0 0x7f093c153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42326/0 (socket says 172.21.15.113:42326) 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093b400640 1 -- 172.21.15.113:0/3676630615 learned_addr learned my addr 172.21.15.113:0/3676630615 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:13.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0933e00640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f093c154c00 0x7f093c152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:13.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093be00640 1 -- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 msgr2=0x7f093c0b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093be00640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c0b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093be00640 1 -- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f093c154c00 msgr2=0x7f093c152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093be00640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f093c154c00 0x7f093c152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093be00640 1 -- 172.21.15.113:0/3676630615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c0b0540 con 0x7f093c154fe0 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f093be00640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093c154fe0 0x7f093c153270 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f093400efc0 tx=0x7f093400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0933400640 1 -- 172.21.15.113:0/3676630615 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934018030 con 0x7f093c154fe0 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0933400640 1 -- 172.21.15.113:0/3676630615 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093400cd30 con 0x7f093c154fe0 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0933400640 1 -- 172.21.15.113:0/3676630615 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934003bc0 con 0x7f093c154fe0 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0942400640 1 -- 172.21.15.113:0/3676630615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c15bfe0 con 0x7f093c154fe0 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.943+0000 7f0942400640 1 -- 172.21.15.113:0/3676630615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c0aecb0 con 0x7f093c154fe0 2024-03-04T08:21:13.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.947+0000 7f0942400640 1 -- 172.21.15.113:0/3676630615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c152a20 con 0x7f093c154fe0 2024-03-04T08:21:13.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.947+0000 7f0933400640 1 -- 172.21.15.113:0/3676630615 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f093401b070 con 0x7f093c154fe0 2024-03-04T08:21:13.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.947+0000 7f0933400640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0918066f60 0x7f0918069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:13.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.947+0000 7f0933400640 1 -- 172.21.15.113:0/3676630615 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f09340860c0 con 0x7f093c154fe0 2024-03-04T08:21:13.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.947+0000 7f093b400640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0918066f60 0x7f0918069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:13.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.947+0000 7f093b400640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0918066f60 0x7f0918069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f092c007fd0 tx=0x7f092c007f20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:13.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:13.951+0000 7f0933400640 1 -- 172.21.15.113:0/3676630615 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0934022070 con 0x7f093c154fe0 2024-03-04T08:21:14.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.139+0000 7f0942400640 1 -- 172.21.15.113:0/3676630615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f093c075c00 con 0x7f093c154fe0 2024-03-04T08:21:14.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0933400640 1 -- 172.21.15.113:0/3676630615 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7f0934010040 con 0x7f093c154fe0 2024-03-04T08:21:14.139 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:14.139 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:14.139 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:21:14.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 -- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0918066f60 msgr2=0x7f0918069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:14.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0918066f60 0x7f0918069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f092c007fd0 tx=0x7f092c007f20 comp rx=0 tx=0).stop 2024-03-04T08:21:14.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 -- 172.21.15.113:0/3676630615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093c154fe0 msgr2=0x7f093c153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:14.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093c154fe0 0x7f093c153270 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f093400efc0 tx=0x7f093400c490 comp rx=0 tx=0).stop 2024-03-04T08:21:14.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 -- 172.21.15.113:0/3676630615 shutdown_connections 2024-03-04T08:21:14.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0918066f60 0x7f0918069420 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:14.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f093c154fe0 0x7f093c153270 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:14.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f093c154c00 0x7f093c152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:14.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 --2- 172.21.15.113:0/3676630615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f093c0c7700 0x7f093c0b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:14.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 -- 172.21.15.113:0/3676630615 >> 172.21.15.113:0/3676630615 conn(0x7f093c06e820 msgr2=0x7f093c151370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:14.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 -- 172.21.15.113:0/3676630615 shutdown_connections 2024-03-04T08:21:14.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:14.143+0000 7f0942400640 1 -- 172.21.15.113:0/3676630615 wait complete. 2024-03-04T08:21:14.161 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:14.293 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:21:14.296 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:21:16.162 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:16.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c32000640 1 Processor -- start 2024-03-04T08:21:16.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c32000640 1 -- start start 2024-03-04T08:21:16.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c32000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c15b900 0x7f0c2c163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c32000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2c0c27f0 con 0x7f0c2c0c22e0 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c32000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2c0c2a90 con 0x7f0c2c14e4c0 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c32000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c15ad60 con 0x7f0c2c15b900 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c2aa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0c2c0c22e0 0x7f0c2c15b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59824/0 (socket says 172.21.15.113:59824) 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c2aa00640 1 -- 172.21.15.113:0/397689508 learned_addr learned my addr 172.21.15.113:0/397689508 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c2be00640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c15b900 0x7f0c2c163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043273679 0 0) 0x7f0c2c0c27f0 con 0x7f0c2c0c22e0 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c10003530 con 0x7f0c2c0c22e0 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365585527 0 0) 0x7f0c2c0c2a90 con 0x7f0c2c14e4c0 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.303+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c2c0c27f0 con 0x7f0c2c14e4c0 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1614621476 0 0) 0x7f0c10003530 con 0x7f0c2c0c22e0 2024-03-04T08:21:16.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c15b900 msgr2=0x7f0c2c163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c15b900 0x7f0c2c163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 >> v1:172.21.15.136:6789/0 conn(0x7f0c2c14e4c0 legacy=0x7f0c2c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c070c60 con 0x7f0c2c0c22e0 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f0c18001680 con 0x7f0c2c0c22e0 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2be00640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c15b900 0x7f0c2c163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 >> v1:172.21.15.113:6789/0 conn(0x7f0c2c0c22e0 legacy=0x7f0c2c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 0x7f0c10003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c10004810 0x7f0c10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c15b900 0x7f0c100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1000b9d0 con 0x7f0c10003b80 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1000bb80 con 0x7f0c10004810 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1000bd80 con 0x7f0c2c15b900 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2b400640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 0x7f0c10003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:16.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2aa00640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c10004810 0x7f0c10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:16.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2be00640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c15b900 0x7f0c100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:16.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2b400640 1 -- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c15b900 msgr2=0x7f0c100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:16.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2b400640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c15b900 0x7f0c100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:16.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2b400640 1 -- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c10004810 msgr2=0x7f0c10004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:16.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2b400640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c10004810 0x7f0c10004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:16.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.307+0000 7f0c2b400640 1 -- 172.21.15.113:0/397689508 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10003530 con 0x7f0c10003b80 2024-03-04T08:21:16.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c2b400640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 0x7f0c10003f80 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c007fd0 tx=0x7f0c1c01ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80ff7cfc07090a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:16.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c001520 con 0x7f0c10003b80 2024-03-04T08:21:16.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0052f0 con 0x7f0c10003b80 2024-03-04T08:21:16.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c2a000640 1 -- 172.21.15.113:0/397689508 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0a8910 con 0x7f0c10003b80 2024-03-04T08:21:16.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c32000640 1 -- 172.21.15.113:0/397689508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 msgr2=0x7f0c10003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:16.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c32000640 1 --2- 172.21.15.113:0/397689508 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 0x7f0c10003f80 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c007fd0 tx=0x7f0c1c01ecb0 comp rx=0 tx=0).stop 2024-03-04T08:21:16.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c2be00640 1 -- 172.21.15.113:0/397689508 reap_dead start 2024-03-04T08:21:16.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c32000640 1 -- 172.21.15.113:0/397689508 shutdown_connections 2024-03-04T08:21:16.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c32000640 1 -- 172.21.15.113:0/397689508 >> 172.21.15.113:0/397689508 conn(0x7f0c2c06e820 msgr2=0x7f0c2c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:16.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c32000640 1 -- 172.21.15.113:0/397689508 shutdown_connections 2024-03-04T08:21:16.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.311+0000 7f0c32000640 1 -- 172.21.15.113:0/397689508 wait complete. 2024-03-04T08:21:16.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 Processor -- start 2024-03-04T08:21:16.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 -- start start 2024-03-04T08:21:16.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 0x7f0c2c1f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:16.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c2c14bc50 0x7f0c2c14c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:16.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c1468d0 0x7f0c2c146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c071500 con 0x7f0c10003b80 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c071220 con 0x7f0c2c14bc50 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c14e8a0 con 0x7f0c2c1468d0 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 0x7f0c2c1f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 0x7f0c2c1f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42340/0 (socket says 172.21.15.113:42340) 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2b400640 1 -- 172.21.15.113:0/2154044339 learned_addr learned my addr 172.21.15.113:0/2154044339 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2be00640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c1468d0 0x7f0c2c146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2aa00640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c2c14bc50 0x7f0c2c14c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2b400640 1 -- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c1468d0 msgr2=0x7f0c2c146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2b400640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c1468d0 0x7f0c2c146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2b400640 1 -- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c2c14bc50 msgr2=0x7f0c2c14c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2b400640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c2c14bc50 0x7f0c2c14c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2b400640 1 -- 172.21.15.113:0/2154044339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c1e65a0 con 0x7f0c10003b80 2024-03-04T08:21:16.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2aa00640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c2c14bc50 0x7f0c2c14c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:16.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c2b400640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 0x7f0c2c1f1060 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c09b3f0 tx=0x7f0c10005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:16.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c23400640 1 -- 172.21.15.113:0/2154044339 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0bd020 con 0x7f0c10003b80 2024-03-04T08:21:16.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c23400640 1 -- 172.21.15.113:0/2154044339 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0054a0 con 0x7f0c10003b80 2024-03-04T08:21:16.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 -- 172.21.15.113:0/2154044339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c1e6840 con 0x7f0c10003b80 2024-03-04T08:21:16.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c23400640 1 -- 172.21.15.113:0/2154044339 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c001500 con 0x7f0c10003b80 2024-03-04T08:21:16.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 -- 172.21.15.113:0/2154044339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c0caba0 con 0x7f0c10003b80 2024-03-04T08:21:16.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.315+0000 7f0c32000640 1 -- 172.21.15.113:0/2154044339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c075c00 con 0x7f0c10003b80 2024-03-04T08:21:16.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.319+0000 7f0c23400640 1 -- 172.21.15.113:0/2154044339 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c1c0c0020 con 0x7f0c10003b80 2024-03-04T08:21:16.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.319+0000 7f0c23400640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c0c066f60 0x7f0c0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:16.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.319+0000 7f0c23400640 1 -- 172.21.15.113:0/2154044339 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f0c1c122c40 con 0x7f0c10003b80 2024-03-04T08:21:16.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.319+0000 7f0c2aa00640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c0c066f60 0x7f0c0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:16.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.319+0000 7f0c2aa00640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c0c066f60 0x7f0c0c069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0c18001060 tx=0x7f0c18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:16.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.327+0000 7f0c23400640 1 -- 172.21.15.113:0/2154044339 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0c1c0c7030 con 0x7f0c10003b80 2024-03-04T08:21:16.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 -- 172.21.15.113:0/2154044339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c2c0c1260 con 0x7f0c10003b80 2024-03-04T08:21:16.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c23400640 1 -- 172.21.15.113:0/2154044339 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7f0c1c0189c0 con 0x7f0c10003b80 2024-03-04T08:21:16.517 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:16.517 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:16.517 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:21:16.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 -- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c0c066f60 msgr2=0x7f0c0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:16.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c0c066f60 0x7f0c0c069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0c18001060 tx=0x7f0c18009040 comp rx=0 tx=0).stop 2024-03-04T08:21:16.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 -- 172.21.15.113:0/2154044339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 msgr2=0x7f0c2c1f1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:16.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 0x7f0c2c1f1060 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c09b3f0 tx=0x7f0c10005840 comp rx=0 tx=0).stop 2024-03-04T08:21:16.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 -- 172.21.15.113:0/2154044339 shutdown_connections 2024-03-04T08:21:16.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c2c1468d0 0x7f0c2c146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:16.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c2c14bc50 0x7f0c2c14c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:16.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c0c066f60 0x7f0c0c069420 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:16.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 --2- 172.21.15.113:0/2154044339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c10003b80 0x7f0c2c1f1060 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:16.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 -- 172.21.15.113:0/2154044339 >> 172.21.15.113:0/2154044339 conn(0x7f0c2c06e820 msgr2=0x7f0c2c072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:16.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 -- 172.21.15.113:0/2154044339 shutdown_connections 2024-03-04T08:21:16.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:16.519+0000 7f0c32000640 1 -- 172.21.15.113:0/2154044339 wait complete. 2024-03-04T08:21:16.537 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:21:16.538 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 170, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 170, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7839": { "addr": "172.21.15.113:6837/2544977425", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 2544977425, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 2544977425, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7839, "incarnation": 145, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8253": { "addr": "172.21.15.113:6836/893414524", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6834", "nonce": 893414524, "type": "v2" }, { "addr": "172.21.15.113:6836", "nonce": 893414524, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8253, "incarnation": 163, "join_fscid": -1, "name": "a", "rank": 2, "state": "up:stopping", "state_seq": 35 } }, "last_failure": 0, "last_failure_osd_epoch": 80, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:20:52.211831+0000", "qdb_cluster": [ 7839, 8032 ], "qdb_leader": 7839, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7839, "mds_1": 8032, "mds_2": 8253 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 0, "join_fscid": -1, "name": "f", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.113:6833/2999156262", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2999156262, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2999156262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 168, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8388, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:21:18.539 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd247600640 1 Processor -- start 2024-03-04T08:21:18.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd247600640 1 -- start start 2024-03-04T08:21:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd247600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd240155100 0x7fd2400c08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd247600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd24006b060 0x7fd2400cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd247600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24015efd0 con 0x7fd2400cd600 2024-03-04T08:21:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd247600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2400c8890 con 0x7fd240155100 2024-03-04T08:21:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd247600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24006b4b0 con 0x7fd24006b060 2024-03-04T08:21:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd244c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd240155100 0x7fd2400c08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd245600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd2400cd600 0x7fd24015ad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48706/0 (socket says 172.21.15.113:48706) 2024-03-04T08:21:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd245600640 1 -- 172.21.15.113:0/3891950859 learned_addr learned my addr 172.21.15.113:0/3891950859 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:18.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd23fe00640 1 --2- 172.21.15.113:0/3891950859 >> v2:172.21.15.136:3300/0 conn(0x7fd24006b060 0x7fd2400cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:18.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd244c00640 1 -- 172.21.15.113:0/3891950859 >> v2:172.21.15.136:3300/0 conn(0x7fd24006b060 msgr2=0x7fd2400cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:18.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd244c00640 1 --2- 172.21.15.113:0/3891950859 >> v2:172.21.15.136:3300/0 conn(0x7fd24006b060 0x7fd2400cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:18.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd244c00640 1 -- 172.21.15.113:0/3891950859 >> v1:172.21.15.113:6789/0 conn(0x7fd2400cd600 legacy=0x7fd24015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:18.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd244c00640 1 -- 172.21.15.113:0/3891950859 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2400c7ef0 con 0x7fd240155100 2024-03-04T08:21:18.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.679+0000 7fd23f400640 1 -- 172.21.15.113:0/3891950859 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418354910 0 0) 0x7fd24015efd0 con 0x7fd2400cd600 2024-03-04T08:21:18.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd244c00640 1 --2- 172.21.15.113:0/3891950859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd240155100 0x7fd2400c08d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fd240071b70 tx=0x7fd23401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f94f9ef5cf7365f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:18.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd23f400640 1 -- 172.21.15.113:0/3891950859 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340b0070 con 0x7fd240155100 2024-03-04T08:21:18.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd23f400640 1 -- 172.21.15.113:0/3891950859 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2340ab600 con 0x7fd240155100 2024-03-04T08:21:18.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd23f400640 1 -- 172.21.15.113:0/3891950859 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340ab9d0 con 0x7fd240155100 2024-03-04T08:21:18.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd247600640 1 -- 172.21.15.113:0/3891950859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd240155100 msgr2=0x7fd2400c08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:18.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd247600640 1 --2- 172.21.15.113:0/3891950859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd240155100 0x7fd2400c08d0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fd240071b70 tx=0x7fd23401c480 comp rx=0 tx=0).stop 2024-03-04T08:21:18.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd247600640 1 -- 172.21.15.113:0/3891950859 shutdown_connections 2024-03-04T08:21:18.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd247600640 1 --2- 172.21.15.113:0/3891950859 >> v2:172.21.15.136:3300/0 conn(0x7fd24006b060 0x7fd2400cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:18.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd247600640 1 --2- 172.21.15.113:0/3891950859 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd240155100 0x7fd2400c08d0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:18.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd247600640 1 -- 172.21.15.113:0/3891950859 >> 172.21.15.113:0/3891950859 conn(0x7fd24006deb0 msgr2=0x7fd24006e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:18.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd247600640 1 -- 172.21.15.113:0/3891950859 shutdown_connections 2024-03-04T08:21:18.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.683+0000 7fd247600640 1 -- 172.21.15.113:0/3891950859 wait complete. 2024-03-04T08:21:18.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd247600640 1 Processor -- start 2024-03-04T08:21:18.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd247600640 1 -- start start 2024-03-04T08:21:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd247600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24006b060 0x7fd2400c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd247600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2400cd600 0x7fd2401f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd247600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd240155100 0x7fd2401f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd247600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24015ed30 con 0x7fd24006b060 2024-03-04T08:21:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd247600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24015ebb0 con 0x7fd240155100 2024-03-04T08:21:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd247600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd240167d30 con 0x7fd2400cd600 2024-03-04T08:21:18.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd244c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24006b060 0x7fd2400c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd244c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24006b060 0x7fd2400c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46490/0 (socket says 172.21.15.113:46490) 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd244c00640 1 -- 172.21.15.113:0/1211134125 learned_addr learned my addr 172.21.15.113:0/1211134125 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd245600640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd240155100 0x7fd2401f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd23fe00640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2400cd600 0x7fd2401f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd244c00640 1 -- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2400cd600 msgr2=0x7fd2401f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd244c00640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2400cd600 0x7fd2401f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd244c00640 1 -- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd240155100 msgr2=0x7fd2401f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd244c00640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd240155100 0x7fd2401f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd244c00640 1 -- 172.21.15.113:0/1211134125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24015c780 con 0x7fd24006b060 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd23fe00640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2400cd600 0x7fd2401f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd244c00640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24006b060 0x7fd2400c5cd0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd2340ae040 tx=0x7fd23401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd23cc00640 1 -- 172.21.15.113:0/1211134125 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340b0050 con 0x7fd24006b060 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd23cc00640 1 -- 172.21.15.113:0/1211134125 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2340bc070 con 0x7fd24006b060 2024-03-04T08:21:18.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd23cc00640 1 -- 172.21.15.113:0/1211134125 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340b5850 con 0x7fd24006b060 2024-03-04T08:21:18.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd247600640 1 -- 172.21.15.113:0/1211134125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2400c5590 con 0x7fd24006b060 2024-03-04T08:21:18.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.687+0000 7fd247600640 1 -- 172.21.15.113:0/1211134125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2400c6a90 con 0x7fd24006b060 2024-03-04T08:21:18.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.691+0000 7fd247600640 1 -- 172.21.15.113:0/1211134125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd240075c00 con 0x7fd24006b060 2024-03-04T08:21:18.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.691+0000 7fd23cc00640 1 -- 172.21.15.113:0/1211134125 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2340b59f0 con 0x7fd24006b060 2024-03-04T08:21:18.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.691+0000 7fd23cc00640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd220066f60 0x7fd220069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:18.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.691+0000 7fd23cc00640 1 -- 172.21.15.113:0/1211134125 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7fd234120070 con 0x7fd24006b060 2024-03-04T08:21:18.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.691+0000 7fd23fe00640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd220066f60 0x7fd220069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:18.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.691+0000 7fd23fe00640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd220066f60 0x7fd220069420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd2400c42e0 tx=0x7fd22c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:18.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.695+0000 7fd23cc00640 1 -- 172.21.15.113:0/1211134125 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd234016070 con 0x7fd24006b060 2024-03-04T08:21:18.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.883+0000 7fd247600640 1 -- 172.21.15.113:0/1211134125 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2400cdf80 con 0x7fd24006b060 2024-03-04T08:21:18.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.883+0000 7fd23cc00640 1 -- 172.21.15.113:0/1211134125 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7fd2340f0700 con 0x7fd24006b060 2024-03-04T08:21:18.883 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:18.883 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:18.883 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:21:18.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 -- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd220066f60 msgr2=0x7fd220069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:18.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd220066f60 0x7fd220069420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd2400c42e0 tx=0x7fd22c008040 comp rx=0 tx=0).stop 2024-03-04T08:21:18.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 -- 172.21.15.113:0/1211134125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24006b060 msgr2=0x7fd2400c5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:18.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24006b060 0x7fd2400c5cd0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fd2340ae040 tx=0x7fd23401acc0 comp rx=0 tx=0).stop 2024-03-04T08:21:18.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 -- 172.21.15.113:0/1211134125 shutdown_connections 2024-03-04T08:21:18.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd240155100 0x7fd2401f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:18.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2400cd600 0x7fd2401f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:18.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd220066f60 0x7fd220069420 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:18.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 --2- 172.21.15.113:0/1211134125 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd24006b060 0x7fd2400c5cd0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:18.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 -- 172.21.15.113:0/1211134125 >> 172.21.15.113:0/1211134125 conn(0x7fd24006deb0 msgr2=0x7fd24006e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:18.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 -- 172.21.15.113:0/1211134125 shutdown_connections 2024-03-04T08:21:18.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:18.887+0000 7fd247600640 1 -- 172.21.15.113:0/1211134125 wait complete. 2024-03-04T08:21:18.905 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:20.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.043+0000 7f6727e00640 1 Processor -- start 2024-03-04T08:21:21.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.043+0000 7f6727e00640 1 -- start start 2024-03-04T08:21:21.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6727e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6720158210 0x7f67200c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:21.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6727e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f67200733f0 0x7f67200cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:21.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6727e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67200b2230 con 0x7f672006b060 2024-03-04T08:21:21.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6727e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6720072fb0 con 0x7f67200733f0 2024-03-04T08:21:21.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6727e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67200c7bb0 con 0x7f6720158210 2024-03-04T08:21:21.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6724a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f67200733f0 0x7f67200cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:21.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6725400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6720158210 0x7f67200c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:21.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6725400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6720158210 0x7f67200c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:57150/0 (socket says 172.21.15.113:57150) 2024-03-04T08:21:21.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6724a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f67200733f0 0x7f67200cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:54276/0 (socket says 172.21.15.113:54276) 2024-03-04T08:21:21.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6725e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f672006b060 0x7f67200c93f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:40468/0 (socket says 172.21.15.113:40468) 2024-03-04T08:21:21.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6725400640 1 -- 172.21.15.113:0/542642949 learned_addr learned my addr 172.21.15.113:0/542642949 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:21.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6725400640 1 -- 172.21.15.113:0/542642949 >> v2:172.21.15.136:3300/0 conn(0x7f67200733f0 msgr2=0x7f67200cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:21.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6725400640 1 --2- 172.21.15.113:0/542642949 >> v2:172.21.15.136:3300/0 conn(0x7f67200733f0 0x7f67200cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:21.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6725400640 1 -- 172.21.15.113:0/542642949 >> v1:172.21.15.136:6789/0 conn(0x7f672006b060 legacy=0x7f67200c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:21.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.047+0000 7f6725400640 1 -- 172.21.15.113:0/542642949 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67200c7e50 con 0x7f6720158210 2024-03-04T08:21:21.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6725400640 1 --2- 172.21.15.113:0/542642949 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6720158210 0x7f67200c8bb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6714009870 tx=0x7f671401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105d84ab95ec6e83 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:21.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f671fe00640 1 -- 172.21.15.113:0/542642949 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67140b2070 con 0x7f6720158210 2024-03-04T08:21:21.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f671fe00640 1 -- 172.21.15.113:0/542642949 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67140a9070 con 0x7f6720158210 2024-03-04T08:21:21.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f671fe00640 1 -- 172.21.15.113:0/542642949 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67140ad880 con 0x7f6720158210 2024-03-04T08:21:21.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6727e00640 1 -- 172.21.15.113:0/542642949 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6720158210 msgr2=0x7f67200c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:21.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6727e00640 1 --2- 172.21.15.113:0/542642949 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6720158210 0x7f67200c8bb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6714009870 tx=0x7f671401c560 comp rx=0 tx=0).stop 2024-03-04T08:21:21.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6727e00640 1 -- 172.21.15.113:0/542642949 shutdown_connections 2024-03-04T08:21:21.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6727e00640 1 --2- 172.21.15.113:0/542642949 >> v2:172.21.15.136:3300/0 conn(0x7f67200733f0 0x7f67200cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:21.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6727e00640 1 --2- 172.21.15.113:0/542642949 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6720158210 0x7f67200c8bb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:21.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6727e00640 1 -- 172.21.15.113:0/542642949 >> 172.21.15.113:0/542642949 conn(0x7f67200671b0 msgr2=0x7f672006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:21.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6727e00640 1 -- 172.21.15.113:0/542642949 shutdown_connections 2024-03-04T08:21:21.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6727e00640 1 -- 172.21.15.113:0/542642949 wait complete. 2024-03-04T08:21:21.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6727e00640 1 Processor -- start 2024-03-04T08:21:21.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.051+0000 7f6727e00640 1 -- start start 2024-03-04T08:21:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6727e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f672006b060 0x7f6720156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6727e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67200733f0 0x7f67200c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6727e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6720158210 0x7f67200c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6727e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672006cff0 con 0x7f67200733f0 2024-03-04T08:21:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6727e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672006ce70 con 0x7f6720158210 2024-03-04T08:21:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6727e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6720167ad0 con 0x7f672006b060 2024-03-04T08:21:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6725400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f672006b060 0x7f6720156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6725400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f672006b060 0x7f6720156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:57152/0 (socket says 172.21.15.113:57152) 2024-03-04T08:21:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6725400640 1 -- 172.21.15.113:0/3731324730 learned_addr learned my addr 172.21.15.113:0/3731324730 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:21.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6724a00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67200733f0 0x7f67200c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:21.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6725e00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6720158210 0x7f67200c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:21.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6724a00640 1 -- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f672006b060 msgr2=0x7f6720156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:21.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6724a00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f672006b060 0x7f6720156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:21.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6724a00640 1 -- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6720158210 msgr2=0x7f67200c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:21.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6724a00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6720158210 0x7f67200c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:21.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6724a00640 1 -- 172.21.15.113:0/3731324730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67200c5110 con 0x7f67200733f0 2024-03-04T08:21:21.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6724a00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67200733f0 0x7f67200c5bb0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f670c00e8a0 tx=0x7f670c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:21.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f671d600640 1 -- 172.21.15.113:0/3731324730 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c017070 con 0x7f67200733f0 2024-03-04T08:21:21.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f671d600640 1 -- 172.21.15.113:0/3731324730 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c00c940 con 0x7f67200733f0 2024-03-04T08:21:21.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6727e00640 1 -- 172.21.15.113:0/3731324730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67200ccf90 con 0x7f67200733f0 2024-03-04T08:21:21.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f671d600640 1 -- 172.21.15.113:0/3731324730 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c00ccf0 con 0x7f67200733f0 2024-03-04T08:21:21.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6727e00640 1 -- 172.21.15.113:0/3731324730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67200cd270 con 0x7f67200733f0 2024-03-04T08:21:21.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.055+0000 7f6727e00640 1 -- 172.21.15.113:0/3731324730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6720075c00 con 0x7f67200733f0 2024-03-04T08:21:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.059+0000 7f671d600640 1 -- 172.21.15.113:0/3731324730 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f670c01a070 con 0x7f67200733f0 2024-03-04T08:21:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.059+0000 7f671d600640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6700066f90 0x7f6700069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.059+0000 7f671d600640 1 -- 172.21.15.113:0/3731324730 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f670c085850 con 0x7f67200733f0 2024-03-04T08:21:21.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.059+0000 7f6725400640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6700066f90 0x7f6700069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:21.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.059+0000 7f6725400640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6700066f90 0x7f6700069450 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f671401ac10 tx=0x7f6714008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:21.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.063+0000 7f671d600640 1 -- 172.21.15.113:0/3731324730 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f670c0558b0 con 0x7f67200733f0 2024-03-04T08:21:21.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.251+0000 7f6727e00640 1 -- 172.21.15.113:0/3731324730 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67200731d0 con 0x7f67200733f0 2024-03-04T08:21:21.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.251+0000 7f671d600640 1 -- 172.21.15.113:0/3731324730 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7f670c059950 con 0x7f67200733f0 2024-03-04T08:21:21.250 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:21.250 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:21.251 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:21:21.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 -- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6700066f90 msgr2=0x7f6700069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:21.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6700066f90 0x7f6700069450 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f671401ac10 tx=0x7f6714008040 comp rx=0 tx=0).stop 2024-03-04T08:21:21.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 -- 172.21.15.113:0/3731324730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67200733f0 msgr2=0x7f67200c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:21.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67200733f0 0x7f67200c5bb0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f670c00e8a0 tx=0x7f670c00ed60 comp rx=0 tx=0).stop 2024-03-04T08:21:21.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 -- 172.21.15.113:0/3731324730 shutdown_connections 2024-03-04T08:21:21.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6720158210 0x7f67200c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:21.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6700066f90 0x7f6700069450 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:21.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67200733f0 0x7f67200c5bb0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:21.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 --2- 172.21.15.113:0/3731324730 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f672006b060 0x7f6720156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:21.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 -- 172.21.15.113:0/3731324730 >> 172.21.15.113:0/3731324730 conn(0x7f67200671b0 msgr2=0x7f672006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:21.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 -- 172.21.15.113:0/3731324730 shutdown_connections 2024-03-04T08:21:21.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:21.255+0000 7f6727e00640 1 -- 172.21.15.113:0/3731324730 wait complete. 2024-03-04T08:21:21.272 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:23.273 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:23.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.418+0000 7f9abf600640 1 Processor -- start 2024-03-04T08:21:23.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9abf600640 1 -- start start 2024-03-04T08:21:23.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9abf600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9ab80c08d0 0x7f9ab806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:23.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9abf600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ab815ad30 0x7f9ab815b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:23.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9abf600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab815b6c0 con 0x7f9ab8155100 2024-03-04T08:21:23.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9abf600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab80c8890 con 0x7f9ab80c08d0 2024-03-04T08:21:23.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9abf600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab815efd0 con 0x7f9ab815ad30 2024-03-04T08:21:23.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9abcc00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9ab8155100 0x7f9ab80cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48710/0 (socket says 172.21.15.113:48710) 2024-03-04T08:21:23.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9abcc00640 1 -- 172.21.15.113:0/1967548655 learned_addr learned my addr 172.21.15.113:0/1967548655 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:23.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9ab7e00640 1 --2- 172.21.15.113:0/1967548655 >> v2:172.21.15.113:3300/0 conn(0x7f9ab80c08d0 0x7f9ab806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:23.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9abd600640 1 --2- 172.21.15.113:0/1967548655 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ab815ad30 0x7f9ab815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:23.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9ab7400640 1 -- 172.21.15.113:0/1967548655 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696773407 0 0) 0x7f9ab815b6c0 con 0x7f9ab8155100 2024-03-04T08:21:23.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9ab7400640 1 -- 172.21.15.113:0/1967548655 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa0003530 con 0x7f9ab8155100 2024-03-04T08:21:23.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9ab7e00640 1 -- 172.21.15.113:0/1967548655 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ab815ad30 msgr2=0x7f9ab815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:23.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9ab7e00640 1 --2- 172.21.15.113:0/1967548655 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ab815ad30 0x7f9ab815b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:23.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9ab7e00640 1 -- 172.21.15.113:0/1967548655 >> v1:172.21.15.113:6789/0 conn(0x7f9ab8155100 legacy=0x7f9ab80cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:23.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.422+0000 7f9ab7e00640 1 -- 172.21.15.113:0/1967548655 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab80c7ef0 con 0x7f9ab80c08d0 2024-03-04T08:21:23.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9ab7e00640 1 --2- 172.21.15.113:0/1967548655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 0x7f9ab806b060 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40097c0 tx=0x7f9aa401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe32bdef71e2dd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:23.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9ab7400640 1 -- 172.21.15.113:0/1967548655 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40b2070 con 0x7f9ab80c08d0 2024-03-04T08:21:23.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9ab7400640 1 -- 172.21.15.113:0/1967548655 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa40ad640 con 0x7f9ab80c08d0 2024-03-04T08:21:23.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9ab7400640 1 -- 172.21.15.113:0/1967548655 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40ad9f0 con 0x7f9ab80c08d0 2024-03-04T08:21:23.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9abf600640 1 -- 172.21.15.113:0/1967548655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 msgr2=0x7f9ab806b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:23.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9abf600640 1 --2- 172.21.15.113:0/1967548655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 0x7f9ab806b060 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40097c0 tx=0x7f9aa401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:21:23.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9abf600640 1 -- 172.21.15.113:0/1967548655 shutdown_connections 2024-03-04T08:21:23.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9abf600640 1 --2- 172.21.15.113:0/1967548655 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ab815ad30 0x7f9ab815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:23.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9abf600640 1 --2- 172.21.15.113:0/1967548655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 0x7f9ab806b060 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:23.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9abf600640 1 -- 172.21.15.113:0/1967548655 >> 172.21.15.113:0/1967548655 conn(0x7f9ab806deb0 msgr2=0x7f9ab806e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:23.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9abf600640 1 -- 172.21.15.113:0/1967548655 shutdown_connections 2024-03-04T08:21:23.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9abf600640 1 -- 172.21.15.113:0/1967548655 wait complete. 2024-03-04T08:21:23.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9abf600640 1 Processor -- start 2024-03-04T08:21:23.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.426+0000 7f9abf600640 1 -- start start 2024-03-04T08:21:23.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abf600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 0x7f9ab80c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:23.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abf600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ab8155100 0x7f9ab806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:23.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abf600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ab815ad30 0x7f9ab81f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:23.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abf600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab815ed30 con 0x7f9ab80c08d0 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abf600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab815ebb0 con 0x7f9ab815ad30 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abf600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80cd7d0 con 0x7f9ab8155100 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abcc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 0x7f9ab80c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abcc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 0x7f9ab80c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46514/0 (socket says 172.21.15.113:46514) 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abcc00640 1 -- 172.21.15.113:0/2424215498 learned_addr learned my addr 172.21.15.113:0/2424215498 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9ab7e00640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ab8155100 0x7f9ab806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abd600640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ab815ad30 0x7f9ab81f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abcc00640 1 -- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ab8155100 msgr2=0x7f9ab806feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abcc00640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ab8155100 0x7f9ab806feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abcc00640 1 -- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ab815ad30 msgr2=0x7f9ab81f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abcc00640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ab815ad30 0x7f9ab81f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abcc00640 1 -- 172.21.15.113:0/2424215498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab815dc40 con 0x7f9ab80c08d0 2024-03-04T08:21:23.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abcc00640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 0x7f9ab80c4ff0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00c450 tx=0x7f9aac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:23.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9ab4c00640 1 -- 172.21.15.113:0/2424215498 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac017070 con 0x7f9ab80c08d0 2024-03-04T08:21:23.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9ab4c00640 1 -- 172.21.15.113:0/2424215498 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac005cf0 con 0x7f9ab80c08d0 2024-03-04T08:21:23.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abf600640 1 -- 172.21.15.113:0/2424215498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab815df20 con 0x7f9ab80c08d0 2024-03-04T08:21:23.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9ab4c00640 1 -- 172.21.15.113:0/2424215498 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac007290 con 0x7f9ab80c08d0 2024-03-04T08:21:23.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abf600640 1 -- 172.21.15.113:0/2424215498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab80c5d20 con 0x7f9ab80c08d0 2024-03-04T08:21:23.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.430+0000 7f9abf600640 1 -- 172.21.15.113:0/2424215498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab8075c00 con 0x7f9ab80c08d0 2024-03-04T08:21:23.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.434+0000 7f9ab4c00640 1 -- 172.21.15.113:0/2424215498 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9aac01a070 con 0x7f9ab80c08d0 2024-03-04T08:21:23.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.434+0000 7f9ab4c00640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a98066f90 0x7f9a98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:23.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.434+0000 7f9ab4c00640 1 -- 172.21.15.113:0/2424215498 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f9aac084c40 con 0x7f9ab80c08d0 2024-03-04T08:21:23.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.434+0000 7f9ab7e00640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a98066f90 0x7f9a98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:23.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.434+0000 7f9ab7e00640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a98066f90 0x7f9a98069450 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40af000 tx=0x7f9aa4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:23.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.438+0000 7f9ab4c00640 1 -- 172.21.15.113:0/2424215498 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9aac054d10 con 0x7f9ab80c08d0 2024-03-04T08:21:23.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 -- 172.21.15.113:0/2424215498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ab80c8a20 con 0x7f9ab80c08d0 2024-03-04T08:21:23.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9ab4c00640 1 -- 172.21.15.113:0/2424215498 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+6618 (secure 0 0 0) 0x7f9aac058db0 con 0x7f9ab80c08d0 2024-03-04T08:21:23.633 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:23.633 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":8388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":168}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:20:52.211831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8253":{"gid":8253,"name":"a","rank":2,"incarnation":163,"state":"up:stopping","state_seq":35,"addr":"172.21.15.113:6836/893414524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":893414524},{"type":"v1","addr":"172.21.15.113:6836","nonce":893414524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:23.633 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 170 2024-03-04T08:21:23.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 -- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a98066f90 msgr2=0x7f9a98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a98066f90 0x7f9a98069450 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40af000 tx=0x7f9aa4008040 comp rx=0 tx=0).stop 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 -- 172.21.15.113:0/2424215498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 msgr2=0x7f9ab80c4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 0x7f9ab80c4ff0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00c450 tx=0x7f9aac00c910 comp rx=0 tx=0).stop 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 -- 172.21.15.113:0/2424215498 shutdown_connections 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ab815ad30 0x7f9ab81f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9ab8155100 0x7f9ab806feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9a98066f90 0x7f9a98069450 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 --2- 172.21.15.113:0/2424215498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ab80c08d0 0x7f9ab80c4ff0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 -- 172.21.15.113:0/2424215498 >> 172.21.15.113:0/2424215498 conn(0x7f9ab806deb0 msgr2=0x7f9ab814e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 -- 172.21.15.113:0/2424215498 shutdown_connections 2024-03-04T08:21:23.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:23.634+0000 7f9abf600640 1 -- 172.21.15.113:0/2424215498 wait complete. 2024-03-04T08:21:23.654 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:25.656 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:25.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.794+0000 7f5b19c00640 1 Processor -- start 2024-03-04T08:21:25.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.794+0000 7f5b19c00640 1 -- start start 2024-03-04T08:21:25.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b19c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1414e4c0 0x7f5b14072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:25.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b19c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5b1415b900 0x7f5b140731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:25.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b19c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b140c27f0 con 0x7f5b140c22e0 2024-03-04T08:21:25.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b19c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1415ad60 con 0x7f5b1415b900 2024-03-04T08:21:25.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b19c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140c2a90 con 0x7f5b1414e4c0 2024-03-04T08:21:25.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b13400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5b1415b900 0x7f5b140731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:25.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b12a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1414e4c0 0x7f5b14072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:25.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b0be00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f5b140c22e0 0x7f5b1415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:40482/0 (socket says 172.21.15.113:40482) 2024-03-04T08:21:25.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b13400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5b1415b900 0x7f5b140731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:54304/0 (socket says 172.21.15.113:54304) 2024-03-04T08:21:25.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b0be00640 1 -- 172.21.15.113:0/3640307617 learned_addr learned my addr 172.21.15.113:0/3640307617 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:25.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b13400640 1 -- 172.21.15.113:0/3640307617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1414e4c0 msgr2=0x7f5b14072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:25.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b13400640 1 --2- 172.21.15.113:0/3640307617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1414e4c0 0x7f5b14072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:25.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b13400640 1 -- 172.21.15.113:0/3640307617 >> v1:172.21.15.136:6789/0 conn(0x7f5b140c22e0 legacy=0x7f5b1415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:25.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b13400640 1 -- 172.21.15.113:0/3640307617 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14070c60 con 0x7f5b1415b900 2024-03-04T08:21:25.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.798+0000 7f5b12a00640 1 --2- 172.21.15.113:0/3640307617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1414e4c0 0x7f5b14072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:21:25.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b13400640 1 --2- 172.21.15.113:0/3640307617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1415b900 0x7f5b140731c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009d90 tx=0x7f5b0c01f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6b8f58f77223299 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:25.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b12000640 1 -- 172.21.15.113:0/3640307617 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0b4070 con 0x7f5b1415b900 2024-03-04T08:21:25.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b12000640 1 -- 172.21.15.113:0/3640307617 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0af600 con 0x7f5b1415b900 2024-03-04T08:21:25.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b12000640 1 -- 172.21.15.113:0/3640307617 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0af9d0 con 0x7f5b1415b900 2024-03-04T08:21:25.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 -- 172.21.15.113:0/3640307617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1415b900 msgr2=0x7f5b140731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:25.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 --2- 172.21.15.113:0/3640307617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1415b900 0x7f5b140731c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009d90 tx=0x7f5b0c01f150 comp rx=0 tx=0).stop 2024-03-04T08:21:25.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 -- 172.21.15.113:0/3640307617 shutdown_connections 2024-03-04T08:21:25.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 --2- 172.21.15.113:0/3640307617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b1415b900 0x7f5b140731c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:25.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 --2- 172.21.15.113:0/3640307617 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1414e4c0 0x7f5b14072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:25.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 -- 172.21.15.113:0/3640307617 >> 172.21.15.113:0/3640307617 conn(0x7f5b1406e820 msgr2=0x7f5b1406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:25.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 -- 172.21.15.113:0/3640307617 shutdown_connections 2024-03-04T08:21:25.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 -- 172.21.15.113:0/3640307617 wait complete. 2024-03-04T08:21:25.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 Processor -- start 2024-03-04T08:21:25.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 -- start start 2024-03-04T08:21:25.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b140c22e0 0x7f5b141e1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:25.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b1414e4c0 0x7f5b141e0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:25.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1415b900 0x7f5b14147140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:25.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14071500 con 0x7f5b1414e4c0 2024-03-04T08:21:25.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14071220 con 0x7f5b140c22e0 2024-03-04T08:21:25.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.802+0000 7f5b19c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1414e8a0 con 0x7f5b1415b900 2024-03-04T08:21:25.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b1414e4c0 0x7f5b141e0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b1414e4c0 0x7f5b141e0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46516/0 (socket says 172.21.15.113:46516) 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0be00640 1 -- 172.21.15.113:0/1118108993 learned_addr learned my addr 172.21.15.113:0/1118108993 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b13400640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1415b900 0x7f5b14147140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b12a00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b140c22e0 0x7f5b141e1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0be00640 1 -- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1415b900 msgr2=0x7f5b14147140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0be00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1415b900 0x7f5b14147140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0be00640 1 -- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b140c22e0 msgr2=0x7f5b141e1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0be00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b140c22e0 0x7f5b141e1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0be00640 1 -- 172.21.15.113:0/1118108993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14159a90 con 0x7f5b1414e4c0 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0be00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b1414e4c0 0x7f5b141e0b40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00c3a0 tx=0x7f5afc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0aa00640 1 -- 172.21.15.113:0/1118108993 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc017070 con 0x7f5b1414e4c0 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0aa00640 1 -- 172.21.15.113:0/1118108993 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc005cf0 con 0x7f5b1414e4c0 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0aa00640 1 -- 172.21.15.113:0/1118108993 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0072f0 con 0x7f5b1414e4c0 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b19c00640 1 -- 172.21.15.113:0/1118108993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b14159d70 con 0x7f5b1414e4c0 2024-03-04T08:21:25.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b19c00640 1 -- 172.21.15.113:0/1118108993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b141581e0 con 0x7f5b1414e4c0 2024-03-04T08:21:25.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b19c00640 1 -- 172.21.15.113:0/1118108993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b14075c00 con 0x7f5b1414e4c0 2024-03-04T08:21:25.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0aa00640 1 -- 172.21.15.113:0/1118108993 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5afc01a070 con 0x7f5b1414e4c0 2024-03-04T08:21:25.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0aa00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5af0066f60 0x7f5af0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:25.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.806+0000 7f5b0aa00640 1 -- 172.21.15.113:0/1118108993 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7975+0+0 (secure 0 0 0) 0x7f5afc084ce0 con 0x7f5b1414e4c0 2024-03-04T08:21:25.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.810+0000 7f5b12a00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5af0066f60 0x7f5af0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:25.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.810+0000 7f5b12a00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5af0066f60 0x7f5af0069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009580 tx=0x7f5b04009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:25.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:25.814+0000 7f5b0aa00640 1 -- 172.21.15.113:0/1118108993 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5afc054c90 con 0x7f5b1414e4c0 2024-03-04T08:21:26.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 -- 172.21.15.113:0/1118108993 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b140c1260 con 0x7f5b1414e4c0 2024-03-04T08:21:26.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b0aa00640 1 -- 172.21.15.113:0/1118108993 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f5afc058d30 con 0x7f5b1414e4c0 2024-03-04T08:21:26.004 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:26.004 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":171,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:25.990497+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839]},"id":1}]} 2024-03-04T08:21:26.004 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 171 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 -- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5af0066f60 msgr2=0x7f5af0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5af0066f60 0x7f5af0069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009580 tx=0x7f5b04009190 comp rx=0 tx=0).stop 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 -- 172.21.15.113:0/1118108993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b1414e4c0 msgr2=0x7f5b141e0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b1414e4c0 0x7f5b141e0b40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00c3a0 tx=0x7f5afc00c860 comp rx=0 tx=0).stop 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 -- 172.21.15.113:0/1118108993 shutdown_connections 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b1415b900 0x7f5b14147140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5af0066f60 0x7f5af0069420 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b1414e4c0 0x7f5b141e0b40 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 --2- 172.21.15.113:0/1118108993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b140c22e0 0x7f5b141e1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 -- 172.21.15.113:0/1118108993 >> 172.21.15.113:0/1118108993 conn(0x7f5b1406e820 msgr2=0x7f5b14147d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 -- 172.21.15.113:0/1118108993 shutdown_connections 2024-03-04T08:21:26.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.006+0000 7f5b19c00640 1 -- 172.21.15.113:0/1118108993 wait complete. 2024-03-04T08:21:26.026 INFO:tasks.mds_thrash.fs.[cephfs]:mds cluster has 2 alive and active, now stable! 2024-03-04T08:21:26.026 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2024-03-04T08:21:26.026 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-03-04T08:21:26.026 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:21:26.054 INFO:tasks.ceph.mds.e:Stopped 2024-03-04T08:21:26.055 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:26.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.238+0000 7f3102800640 1 Processor -- start 2024-03-04T08:21:26.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.238+0000 7f3102800640 1 -- start start 2024-03-04T08:21:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f3102800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc0c8200 0x7f30fc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f3102800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f30fc152bd0 0x7f30fc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f3102800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30fc1591f0 con 0x7f30fc1527f0 2024-03-04T08:21:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f3102800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc071df0 con 0x7f30fc152bd0 2024-03-04T08:21:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f3102800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc1597b0 con 0x7f30fc0c8200 2024-03-04T08:21:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30faa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc0c8200 0x7f30fc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30fbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f30fc152bd0 0x7f30fc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30fb400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f30fc1527f0 0x7f30fc072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:40488/0 (socket says 172.21.15.113:40488) 2024-03-04T08:21:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30faa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc0c8200 0x7f30fc070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:57184/0 (socket says 172.21.15.113:57184) 2024-03-04T08:21:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30fb400640 1 -- 172.21.15.113:0/22914597 learned_addr learned my addr 172.21.15.113:0/22914597 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30fbe00640 1 -- 172.21.15.113:0/22914597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc0c8200 msgr2=0x7f30fc070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:26.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30fbe00640 1 --2- 172.21.15.113:0/22914597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc0c8200 0x7f30fc070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30fbe00640 1 -- 172.21.15.113:0/22914597 >> v1:172.21.15.136:6789/0 conn(0x7f30fc1527f0 legacy=0x7f30fc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30fbe00640 1 -- 172.21.15.113:0/22914597 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc1594d0 con 0x7f30fc152bd0 2024-03-04T08:21:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30fa000640 1 -- 172.21.15.113:0/22914597 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311413300 0 0) 0x7f30fc1591f0 con 0x7f30fc1527f0 2024-03-04T08:21:26.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30faa00640 1 --2- 172.21.15.113:0/22914597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc0c8200 0x7f30fc070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:21:26.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.242+0000 7f30fbe00640 1 --2- 172.21.15.113:0/22914597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc152bd0 0x7f30fc1587d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f30f40097c0 tx=0x7f30f401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b84eb95d8fc3604 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:26.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f30fa000640 1 -- 172.21.15.113:0/22914597 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f40b2070 con 0x7f30fc152bd0 2024-03-04T08:21:26.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f30fa000640 1 -- 172.21.15.113:0/22914597 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f40ad640 con 0x7f30fc152bd0 2024-03-04T08:21:26.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f30fa000640 1 -- 172.21.15.113:0/22914597 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f40ada10 con 0x7f30fc152bd0 2024-03-04T08:21:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 -- 172.21.15.113:0/22914597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc152bd0 msgr2=0x7f30fc1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 --2- 172.21.15.113:0/22914597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc152bd0 0x7f30fc1587d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f30f40097c0 tx=0x7f30f401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 -- 172.21.15.113:0/22914597 shutdown_connections 2024-03-04T08:21:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 --2- 172.21.15.113:0/22914597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc152bd0 0x7f30fc1587d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 --2- 172.21.15.113:0/22914597 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc0c8200 0x7f30fc070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 -- 172.21.15.113:0/22914597 >> 172.21.15.113:0/22914597 conn(0x7f30fc06e820 msgr2=0x7f30fc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 -- 172.21.15.113:0/22914597 shutdown_connections 2024-03-04T08:21:26.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 -- 172.21.15.113:0/22914597 wait complete. 2024-03-04T08:21:26.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 Processor -- start 2024-03-04T08:21:26.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 -- start start 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30fc0c8200 0x7f30fc14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc1527f0 0x7f30fc14a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc152bd0 0x7f30fc0c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc1654f0 con 0x7f30fc0c8200 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc165370 con 0x7f30fc152bd0 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f3102800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc165670 con 0x7f30fc1527f0 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f30fb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30fc0c8200 0x7f30fc14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f30fb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30fc0c8200 0x7f30fc14a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46532/0 (socket says 172.21.15.113:46532) 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f30fb400640 1 -- 172.21.15.113:0/15753820 learned_addr learned my addr 172.21.15.113:0/15753820 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f30faa00640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc1527f0 0x7f30fc14a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:26.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.246+0000 7f30fbe00640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc152bd0 0x7f30fc0c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30fb400640 1 -- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc1527f0 msgr2=0x7f30fc14a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30fb400640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc1527f0 0x7f30fc14a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30fb400640 1 -- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc152bd0 msgr2=0x7f30fc0c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30fb400640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc152bd0 0x7f30fc0c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30fb400640 1 -- 172.21.15.113:0/15753820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc0c4460 con 0x7f30fc0c8200 2024-03-04T08:21:26.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30fb400640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30fc0c8200 0x7f30fc14a4b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00c890 tx=0x7f30ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30f3400640 1 -- 172.21.15.113:0/15753820 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec018070 con 0x7f30fc0c8200 2024-03-04T08:21:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30f3400640 1 -- 172.21.15.113:0/15753820 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ec00a640 con 0x7f30fc0c8200 2024-03-04T08:21:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f3102800640 1 -- 172.21.15.113:0/15753820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc0cb380 con 0x7f30fc0c8200 2024-03-04T08:21:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30f3400640 1 -- 172.21.15.113:0/15753820 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00aa30 con 0x7f30fc0c8200 2024-03-04T08:21:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f3102800640 1 -- 172.21.15.113:0/15753820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc1581e0 con 0x7f30fc0c8200 2024-03-04T08:21:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f3102800640 1 -- 172.21.15.113:0/15753820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc075c00 con 0x7f30fc0c8200 2024-03-04T08:21:26.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30f3400640 1 -- 172.21.15.113:0/15753820 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30ec00abd0 con 0x7f30fc0c8200 2024-03-04T08:21:26.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30f3400640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f30dc066f90 0x7f30dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:26.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30f3400640 1 -- 172.21.15.113:0/15753820 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8061+0+0 (secure 0 0 0) 0x7f30ec084fd0 con 0x7f30fc0c8200 2024-03-04T08:21:26.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.250+0000 7f30faa00640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f30dc066f90 0x7f30dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:26.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.254+0000 7f30faa00640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f30dc066f90 0x7f30dc069450 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009580 tx=0x7f30e8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:26.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.254+0000 7f30f3400640 1 -- 172.21.15.113:0/15753820 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f30ec054f40 con 0x7f30fc0c8200 2024-03-04T08:21:26.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 -- 172.21.15.113:0/15753820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30fc0cbee0 con 0x7f30fc0c8200 2024-03-04T08:21:26.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f30f3400640 1 -- 172.21.15.113:0/15753820 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f30ec058fe0 con 0x7f30fc0c8200 2024-03-04T08:21:26.445 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:26.445 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":171,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:25.990497+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839]},"id":1}]} 2024-03-04T08:21:26.445 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 171 2024-03-04T08:21:26.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 -- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f30dc066f90 msgr2=0x7f30dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:26.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f30dc066f90 0x7f30dc069450 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009580 tx=0x7f30e8009190 comp rx=0 tx=0).stop 2024-03-04T08:21:26.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 -- 172.21.15.113:0/15753820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30fc0c8200 msgr2=0x7f30fc14a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:26.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30fc0c8200 0x7f30fc14a4b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00c890 tx=0x7f30ec00cd50 comp rx=0 tx=0).stop 2024-03-04T08:21:26.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 -- 172.21.15.113:0/15753820 shutdown_connections 2024-03-04T08:21:26.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30fc152bd0 0x7f30fc0c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f30fc1527f0 0x7f30fc14a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f30dc066f90 0x7f30dc069450 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 --2- 172.21.15.113:0/15753820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f30fc0c8200 0x7f30fc14a4b0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:26.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 -- 172.21.15.113:0/15753820 >> 172.21.15.113:0/15753820 conn(0x7f30fc06e820 msgr2=0x7f30fc151fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:26.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 -- 172.21.15.113:0/15753820 shutdown_connections 2024-03-04T08:21:26.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:26.446+0000 7f3102800640 1 -- 172.21.15.113:0/15753820 wait complete. 2024-03-04T08:21:26.465 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:21:28.465 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:28.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e8000640 1 Processor -- start 2024-03-04T08:21:28.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e8000640 1 -- start start 2024-03-04T08:21:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e8000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 0x7f01e0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:28.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e8000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f01e0152bd0 0x7f01e01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:28.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e8000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e01591f0 con 0x7f01e01527f0 2024-03-04T08:21:28.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e8000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e0071df0 con 0x7f01e0152bd0 2024-03-04T08:21:28.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e8000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e01597b0 con 0x7f01e00c8200 2024-03-04T08:21:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e4c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 0x7f01e0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e5600640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f01e01527f0 0x7f01e0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:35406/0 (socket says 172.21.15.113:35406) 2024-03-04T08:21:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e5600640 1 -- 172.21.15.113:0/947684735 learned_addr learned my addr 172.21.15.113:0/947684735 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e6000640 1 --2- 172.21.15.113:0/947684735 >> v2:172.21.15.136:3300/0 conn(0x7f01e0152bd0 0x7f01e01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.614+0000 7f01e6000640 1 -- 172.21.15.113:0/947684735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 msgr2=0x7f01e0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e6000640 1 --2- 172.21.15.113:0/947684735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 0x7f01e0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e6000640 1 -- 172.21.15.113:0/947684735 >> v1:172.21.15.136:6789/0 conn(0x7f01e01527f0 legacy=0x7f01e0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e6000640 1 -- 172.21.15.113:0/947684735 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e01594d0 con 0x7f01e0152bd0 2024-03-04T08:21:28.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e4c00640 1 --2- 172.21.15.113:0/947684735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 0x7f01e0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:21:28.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e6000640 1 --2- 172.21.15.113:0/947684735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e0152bd0 0x7f01e01587d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009d90 tx=0x7f01d801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b85917e4e2bda0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01dfe00640 1 -- 172.21.15.113:0/947684735 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80b2070 con 0x7f01e0152bd0 2024-03-04T08:21:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01dfe00640 1 -- 172.21.15.113:0/947684735 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d80a5920 con 0x7f01e0152bd0 2024-03-04T08:21:28.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01dfe00640 1 -- 172.21.15.113:0/947684735 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80a5cf0 con 0x7f01e0152bd0 2024-03-04T08:21:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e8000640 1 -- 172.21.15.113:0/947684735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e0152bd0 msgr2=0x7f01e01587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e8000640 1 --2- 172.21.15.113:0/947684735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e0152bd0 0x7f01e01587d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009d90 tx=0x7f01d801c970 comp rx=0 tx=0).stop 2024-03-04T08:21:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e8000640 1 -- 172.21.15.113:0/947684735 shutdown_connections 2024-03-04T08:21:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e8000640 1 --2- 172.21.15.113:0/947684735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e0152bd0 0x7f01e01587d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e8000640 1 --2- 172.21.15.113:0/947684735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 0x7f01e0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e8000640 1 -- 172.21.15.113:0/947684735 >> 172.21.15.113:0/947684735 conn(0x7f01e006e820 msgr2=0x7f01e006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.618+0000 7f01e8000640 1 -- 172.21.15.113:0/947684735 shutdown_connections 2024-03-04T08:21:28.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e8000640 1 -- 172.21.15.113:0/947684735 wait complete. 2024-03-04T08:21:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e8000640 1 Processor -- start 2024-03-04T08:21:28.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e8000640 1 -- start start 2024-03-04T08:21:28.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e8000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 0x7f01e014e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e8000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e01527f0 0x7f01e01f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e8000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e0152bd0 0x7f01e014b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e8000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e01654f0 con 0x7f01e01527f0 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e8000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0165370 con 0x7f01e0152bd0 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e8000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0165670 con 0x7f01e00c8200 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e01527f0 0x7f01e01f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e01527f0 0x7f01e01f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52036/0 (socket says 172.21.15.113:52036) 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e4c00640 1 -- 172.21.15.113:0/596269434 learned_addr learned my addr 172.21.15.113:0/596269434 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e5600640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 0x7f01e014e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e6000640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e0152bd0 0x7f01e014b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e4c00640 1 -- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 msgr2=0x7f01e014e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e4c00640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 0x7f01e014e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:28.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e4c00640 1 -- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e0152bd0 msgr2=0x7f01e014b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e4c00640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e0152bd0 0x7f01e014b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e4c00640 1 -- 172.21.15.113:0/596269434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0153100 con 0x7f01e01527f0 2024-03-04T08:21:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e4c00640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e01527f0 0x7f01e01f2320 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00e9d0 tx=0x7f01cc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01dd600640 1 -- 172.21.15.113:0/596269434 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc017070 con 0x7f01e01527f0 2024-03-04T08:21:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01dd600640 1 -- 172.21.15.113:0/596269434 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01cc00ca60 con 0x7f01e01527f0 2024-03-04T08:21:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01dd600640 1 -- 172.21.15.113:0/596269434 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc00ce50 con 0x7f01e01527f0 2024-03-04T08:21:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.622+0000 7f01e8000640 1 -- 172.21.15.113:0/596269434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e01533e0 con 0x7f01e01527f0 2024-03-04T08:21:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.626+0000 7f01e8000640 1 -- 172.21.15.113:0/596269434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e014ace0 con 0x7f01e01527f0 2024-03-04T08:21:28.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.626+0000 7f01e8000640 1 -- 172.21.15.113:0/596269434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e0075c00 con 0x7f01e01527f0 2024-03-04T08:21:28.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.626+0000 7f01dd600640 1 -- 172.21.15.113:0/596269434 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01cc01a070 con 0x7f01e01527f0 2024-03-04T08:21:28.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.626+0000 7f01dd600640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c0067130 0x7f01c00695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:28.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.626+0000 7f01dd600640 1 -- 172.21.15.113:0/596269434 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8061+0+0 (secure 0 0 0) 0x7f01cc0859d0 con 0x7f01e01527f0 2024-03-04T08:21:28.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.626+0000 7f01e5600640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c0067130 0x7f01c00695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:28.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.626+0000 7f01e5600640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c0067130 0x7f01c00695f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f01d4000f70 tx=0x7f01d4007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:28.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.630+0000 7f01dd600640 1 -- 172.21.15.113:0/596269434 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f01cc055930 con 0x7f01e01527f0 2024-03-04T08:21:28.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.818+0000 7f01e8000640 1 -- 172.21.15.113:0/596269434 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01e00c79d0 con 0x7f01e01527f0 2024-03-04T08:21:28.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.818+0000 7f01dd600640 1 -- 172.21.15.113:0/596269434 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+5823 (secure 0 0 0) 0x7f01cc0599d0 con 0x7f01e01527f0 2024-03-04T08:21:28.817 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:28.817 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":171,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:25.990497+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839]},"id":1}]} 2024-03-04T08:21:28.817 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 171 2024-03-04T08:21:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 -- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c0067130 msgr2=0x7f01c00695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c0067130 0x7f01c00695f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f01d4000f70 tx=0x7f01d4007e90 comp rx=0 tx=0).stop 2024-03-04T08:21:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 -- 172.21.15.113:0/596269434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e01527f0 msgr2=0x7f01e01f2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e01527f0 0x7f01e01f2320 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00e9d0 tx=0x7f01cc00ee90 comp rx=0 tx=0).stop 2024-03-04T08:21:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 -- 172.21.15.113:0/596269434 shutdown_connections 2024-03-04T08:21:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01e0152bd0 0x7f01e014b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01c0067130 0x7f01c00695f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01e01527f0 0x7f01e01f2320 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:28.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 --2- 172.21.15.113:0/596269434 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f01e00c8200 0x7f01e014e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 -- 172.21.15.113:0/596269434 >> 172.21.15.113:0/596269434 conn(0x7f01e006e820 msgr2=0x7f01e00cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 -- 172.21.15.113:0/596269434 shutdown_connections 2024-03-04T08:21:28.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:28.822+0000 7f01e8000640 1 -- 172.21.15.113:0/596269434 wait complete. 2024-03-04T08:21:28.838 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:21:30.839 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:30.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.986+0000 7fd0a3200640 1 Processor -- start 2024-03-04T08:21:30.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.986+0000 7fd0a3200640 1 -- start start 2024-03-04T08:21:30.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd0a3200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd09c0b20b0 0x7fd09c157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:30.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd0a3200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd09c0c85f0 0x7fd09c161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:30.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd0a3200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09c0c1ec0 con 0x7fd09c157c50 2024-03-04T08:21:30.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd0a3200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09c0c9b30 con 0x7fd09c0b20b0 2024-03-04T08:21:30.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd0a3200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0c12a0 con 0x7fd09c0c85f0 2024-03-04T08:21:30.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd09be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd09c0b20b0 0x7fd09c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:30.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd093400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd09c157c50 0x7fd09c06b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50064/0 (socket says 172.21.15.113:50064) 2024-03-04T08:21:30.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd093400640 1 -- 172.21.15.113:0/4174219208 learned_addr learned my addr 172.21.15.113:0/4174219208 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:30.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd0a1200640 1 --2- 172.21.15.113:0/4174219208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd09c0c85f0 0x7fd09c161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:30.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd09be00640 1 -- 172.21.15.113:0/4174219208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd09c0c85f0 msgr2=0x7fd09c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:30.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd09be00640 1 --2- 172.21.15.113:0/4174219208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd09c0c85f0 0x7fd09c161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:30.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd09be00640 1 -- 172.21.15.113:0/4174219208 >> v1:172.21.15.113:6789/0 conn(0x7fd09c157c50 legacy=0x7fd09c06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:30.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.990+0000 7fd09be00640 1 -- 172.21.15.113:0/4174219208 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c0c1580 con 0x7fd09c0b20b0 2024-03-04T08:21:30.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd09be00640 1 --2- 172.21.15.113:0/4174219208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0b20b0 0x7fd09c157710 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00a960 tx=0x7fd08c020060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c16b3d83cb394f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:30.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd09b400640 1 -- 172.21.15.113:0/4174219208 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0b6070 con 0x7fd09c0b20b0 2024-03-04T08:21:30.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd09b400640 1 -- 172.21.15.113:0/4174219208 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c0b1440 con 0x7fd09c0b20b0 2024-03-04T08:21:30.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd09b400640 1 -- 172.21.15.113:0/4174219208 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00b040 con 0x7fd09c0b20b0 2024-03-04T08:21:30.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd0a3200640 1 -- 172.21.15.113:0/4174219208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0b20b0 msgr2=0x7fd09c157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:30.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd0a3200640 1 --2- 172.21.15.113:0/4174219208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0b20b0 0x7fd09c157710 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00a960 tx=0x7fd08c020060 comp rx=0 tx=0).stop 2024-03-04T08:21:30.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd0a3200640 1 -- 172.21.15.113:0/4174219208 shutdown_connections 2024-03-04T08:21:30.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd0a3200640 1 --2- 172.21.15.113:0/4174219208 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd09c0c85f0 0x7fd09c161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:30.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd0a3200640 1 --2- 172.21.15.113:0/4174219208 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0b20b0 0x7fd09c157710 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:30.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd0a3200640 1 -- 172.21.15.113:0/4174219208 >> 172.21.15.113:0/4174219208 conn(0x7fd09c06e820 msgr2=0x7fd09c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:30.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd0a3200640 1 -- 172.21.15.113:0/4174219208 shutdown_connections 2024-03-04T08:21:30.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd0a3200640 1 -- 172.21.15.113:0/4174219208 wait complete. 2024-03-04T08:21:30.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd0a3200640 1 Processor -- start 2024-03-04T08:21:30.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.994+0000 7fd0a3200640 1 -- start start 2024-03-04T08:21:30.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd0a3200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c0b20b0 0x7fd09c1f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:30.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd0a3200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0c85f0 0x7fd09c1e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:30.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd0a3200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd09c157c50 0x7fd09c1f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:30.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd0a3200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c0c2400 con 0x7fd09c0c85f0 2024-03-04T08:21:30.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd0a3200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c1571f0 con 0x7fd09c0b20b0 2024-03-04T08:21:30.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd0a3200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c156ec0 con 0x7fd09c157c50 2024-03-04T08:21:30.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd093400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0c85f0 0x7fd09c1e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:30.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd093400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0c85f0 0x7fd09c1e7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52058/0 (socket says 172.21.15.113:52058) 2024-03-04T08:21:30.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd093400640 1 -- 172.21.15.113:0/1183510737 learned_addr learned my addr 172.21.15.113:0/1183510737 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:30.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd0a1200640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd09c157c50 0x7fd09c1f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:31.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd09be00640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c0b20b0 0x7fd09c1f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:31.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd093400640 1 -- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd09c157c50 msgr2=0x7fd09c1f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:31.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd093400640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd09c157c50 0x7fd09c1f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:31.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd093400640 1 -- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c0b20b0 msgr2=0x7fd09c1f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:31.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd093400640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c0b20b0 0x7fd09c1f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:31.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd093400640 1 -- 172.21.15.113:0/1183510737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c0b0200 con 0x7fd09c0c85f0 2024-03-04T08:21:31.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd093400640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0c85f0 0x7fd09c1e7a10 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c370 tx=0x7fd08400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:31.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd098c00640 1 -- 172.21.15.113:0/1183510737 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084017070 con 0x7fd09c0c85f0 2024-03-04T08:21:31.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd098c00640 1 -- 172.21.15.113:0/1183510737 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084005cf0 con 0x7fd09c0c85f0 2024-03-04T08:21:31.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd098c00640 1 -- 172.21.15.113:0/1183510737 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840072f0 con 0x7fd09c0c85f0 2024-03-04T08:21:31.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd0a3200640 1 -- 172.21.15.113:0/1183510737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09c0b04a0 con 0x7fd09c0c85f0 2024-03-04T08:21:31.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd0a3200640 1 -- 172.21.15.113:0/1183510737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09c0b0a60 con 0x7fd09c0c85f0 2024-03-04T08:21:31.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:30.998+0000 7fd0a3200640 1 -- 172.21.15.113:0/1183510737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09c075c00 con 0x7fd09c0c85f0 2024-03-04T08:21:31.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.002+0000 7fd098c00640 1 -- 172.21.15.113:0/1183510737 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd08401a070 con 0x7fd09c0c85f0 2024-03-04T08:21:31.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.002+0000 7fd098c00640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd078066fe0 0x7fd0780694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:31.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.002+0000 7fd098c00640 1 -- 172.21.15.113:0/1183510737 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8061+0+0 (secure 0 0 0) 0x7fd084084d20 con 0x7fd09c0c85f0 2024-03-04T08:21:31.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.002+0000 7fd09be00640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd078066fe0 0x7fd0780694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:31.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.002+0000 7fd09be00640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd078066fe0 0x7fd0780694a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0b3000 tx=0x7fd08c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:31.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.006+0000 7fd098c00640 1 -- 172.21.15.113:0/1183510737 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd084054d00 con 0x7fd09c0c85f0 2024-03-04T08:21:31.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.194+0000 7fd0a3200640 1 -- 172.21.15.113:0/1183510737 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd09c0c83d0 con 0x7fd09c0c85f0 2024-03-04T08:21:31.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.194+0000 7fd098c00640 1 -- 172.21.15.113:0/1183510737 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+5823 (secure 0 0 0) 0x7fd084058da0 con 0x7fd09c0c85f0 2024-03-04T08:21:31.194 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:31.194 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":171,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:25.990497+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[8032,7839]},"id":1}]} 2024-03-04T08:21:31.194 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 171 2024-03-04T08:21:31.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 -- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd078066fe0 msgr2=0x7fd0780694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:31.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd078066fe0 0x7fd0780694a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0b3000 tx=0x7fd08c008040 comp rx=0 tx=0).stop 2024-03-04T08:21:31.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 -- 172.21.15.113:0/1183510737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0c85f0 msgr2=0x7fd09c1e7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:31.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0c85f0 0x7fd09c1e7a10 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c370 tx=0x7fd08400c830 comp rx=0 tx=0).stop 2024-03-04T08:21:31.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 -- 172.21.15.113:0/1183510737 shutdown_connections 2024-03-04T08:21:31.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd09c157c50 0x7fd09c1f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:31.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd078066fe0 0x7fd0780694a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:31.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd09c0c85f0 0x7fd09c1e7a10 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:31.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 --2- 172.21.15.113:0/1183510737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c0b20b0 0x7fd09c1f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:31.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 -- 172.21.15.113:0/1183510737 >> 172.21.15.113:0/1183510737 conn(0x7fd09c06e820 msgr2=0x7fd09c074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:31.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 -- 172.21.15.113:0/1183510737 shutdown_connections 2024-03-04T08:21:31.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:31.198+0000 7fd0a3200640 1 -- 172.21.15.113:0/1183510737 wait complete. 2024-03-04T08:21:31.215 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:21:33.216 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:33.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b8800640 1 Processor -- start 2024-03-04T08:21:33.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b8800640 1 -- start start 2024-03-04T08:21:33.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b8800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff2b00c8200 0x7ff2b0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:33.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b8800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b0152bd0 0x7ff2b01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:33.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b8800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b01591f0 con 0x7ff2b01527f0 2024-03-04T08:21:33.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b8800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b0071df0 con 0x7ff2b00c8200 2024-03-04T08:21:33.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b8800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b01597b0 con 0x7ff2b0152bd0 2024-03-04T08:21:33.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff2b00c8200 0x7ff2b0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:33.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b6800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b0152bd0 0x7ff2b01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:33.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff2b00c8200 0x7ff2b0070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:55148/0 (socket says 172.21.15.113:55148) 2024-03-04T08:21:33.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b5400640 1 -- 172.21.15.113:0/2678215519 learned_addr learned my addr 172.21.15.113:0/2678215519 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:33.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b5e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7ff2b01527f0 0x7ff2b0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:35410/0 (socket says 172.21.15.113:35410) 2024-03-04T08:21:33.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b5400640 1 -- 172.21.15.113:0/2678215519 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b0152bd0 msgr2=0x7ff2b01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:33.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b5400640 1 --2- 172.21.15.113:0/2678215519 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b0152bd0 0x7ff2b01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:33.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b5400640 1 -- 172.21.15.113:0/2678215519 >> v1:172.21.15.136:6789/0 conn(0x7ff2b01527f0 legacy=0x7ff2b0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:33.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.366+0000 7ff2b5400640 1 -- 172.21.15.113:0/2678215519 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b01594d0 con 0x7ff2b00c8200 2024-03-04T08:21:33.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b5400640 1 --2- 172.21.15.113:0/2678215519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b00c8200 0x7ff2b0070d40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009870 tx=0x7ff2a001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e29fd841cde1c207 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:33.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b4a00640 1 -- 172.21.15.113:0/2678215519 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00a7070 con 0x7ff2b00c8200 2024-03-04T08:21:33.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b4a00640 1 -- 172.21.15.113:0/2678215519 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00b0070 con 0x7ff2b00c8200 2024-03-04T08:21:33.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b4a00640 1 -- 172.21.15.113:0/2678215519 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00ab890 con 0x7ff2b00c8200 2024-03-04T08:21:33.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b8800640 1 -- 172.21.15.113:0/2678215519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b00c8200 msgr2=0x7ff2b0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:33.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b8800640 1 --2- 172.21.15.113:0/2678215519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b00c8200 0x7ff2b0070d40 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009870 tx=0x7ff2a001c560 comp rx=0 tx=0).stop 2024-03-04T08:21:33.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b8800640 1 -- 172.21.15.113:0/2678215519 shutdown_connections 2024-03-04T08:21:33.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b8800640 1 --2- 172.21.15.113:0/2678215519 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b0152bd0 0x7ff2b01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:33.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b8800640 1 --2- 172.21.15.113:0/2678215519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b00c8200 0x7ff2b0070d40 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:33.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b8800640 1 -- 172.21.15.113:0/2678215519 >> 172.21.15.113:0/2678215519 conn(0x7ff2b006e820 msgr2=0x7ff2b006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:33.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b8800640 1 -- 172.21.15.113:0/2678215519 shutdown_connections 2024-03-04T08:21:33.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.370+0000 7ff2b8800640 1 -- 172.21.15.113:0/2678215519 wait complete. 2024-03-04T08:21:33.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b8800640 1 Processor -- start 2024-03-04T08:21:33.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b8800640 1 -- start start 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b8800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b00c8200 0x7ff2b01547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b8800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b01527f0 0x7ff2b01f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b8800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b0152bd0 0x7ff2b01f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b8800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b01654f0 con 0x7ff2b01527f0 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b8800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0165370 con 0x7ff2b00c8200 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b8800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b0165670 con 0x7ff2b0152bd0 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b01527f0 0x7ff2b01f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b01527f0 0x7ff2b01f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52066/0 (socket says 172.21.15.113:52066) 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b5400640 1 -- 172.21.15.113:0/2403703885 learned_addr learned my addr 172.21.15.113:0/2403703885 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b5e00640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b00c8200 0x7ff2b01547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:33.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b6800640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b0152bd0 0x7ff2b01f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b5400640 1 -- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b0152bd0 msgr2=0x7ff2b01f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b5400640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b0152bd0 0x7ff2b01f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b5400640 1 -- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b00c8200 msgr2=0x7ff2b01547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b5400640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b00c8200 0x7ff2b01547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b5400640 1 -- 172.21.15.113:0/2403703885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b0151570 con 0x7ff2b01527f0 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b5400640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b01527f0 0x7ff2b01f25b0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7ff2a001a190 tx=0x7ff2a001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2aa000640 1 -- 172.21.15.113:0/2403703885 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00be5f0 con 0x7ff2b01527f0 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2aa000640 1 -- 172.21.15.113:0/2403703885 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00be790 con 0x7ff2b01527f0 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b8800640 1 -- 172.21.15.113:0/2403703885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b0159aa0 con 0x7ff2b01527f0 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2aa000640 1 -- 172.21.15.113:0/2403703885 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00beb80 con 0x7ff2b01527f0 2024-03-04T08:21:33.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.374+0000 7ff2b8800640 1 -- 172.21.15.113:0/2403703885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b015a020 con 0x7ff2b01527f0 2024-03-04T08:21:33.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.378+0000 7ff2b8800640 1 -- 172.21.15.113:0/2403703885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b0075c00 con 0x7ff2b01527f0 2024-03-04T08:21:33.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.378+0000 7ff2aa000640 1 -- 172.21.15.113:0/2403703885 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2a00b43f0 con 0x7ff2b01527f0 2024-03-04T08:21:33.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.378+0000 7ff2aa000640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f60 0x7ff294069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:33.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.378+0000 7ff2aa000640 1 -- 172.21.15.113:0/2403703885 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8061+0+0 (secure 0 0 0) 0x7ff2a011ff00 con 0x7ff2b01527f0 2024-03-04T08:21:33.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.378+0000 7ff2b5e00640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f60 0x7ff294069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:33.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.378+0000 7ff2b5e00640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f60 0x7ff294069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4000f70 tx=0x7ff2a4007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:33.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.382+0000 7ff2aa000640 1 -- 172.21.15.113:0/2403703885 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff2b0151570 con 0x7ff2b01527f0 2024-03-04T08:21:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.570+0000 7ff2b8800640 1 -- 172.21.15.113:0/2403703885 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2b00732f0 con 0x7ff2b01527f0 2024-03-04T08:21:33.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.570+0000 7ff2aa000640 1 -- 172.21.15.113:0/2403703885 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5825 (secure 0 0 0) 0x7ff2a0016150 con 0x7ff2b01527f0 2024-03-04T08:21:33.571 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:33.571 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:31.002944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:33.571 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 172 2024-03-04T08:21:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 -- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f60 msgr2=0x7ff294069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f60 0x7ff294069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4000f70 tx=0x7ff2a4007e90 comp rx=0 tx=0).stop 2024-03-04T08:21:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 -- 172.21.15.113:0/2403703885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b01527f0 msgr2=0x7ff2b01f25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b01527f0 0x7ff2b01f25b0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7ff2a001a190 tx=0x7ff2a001ada0 comp rx=0 tx=0).stop 2024-03-04T08:21:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 -- 172.21.15.113:0/2403703885 shutdown_connections 2024-03-04T08:21:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff2b0152bd0 0x7ff2b01f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff294066f60 0x7ff294069420 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2b01527f0 0x7ff2b01f25b0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 --2- 172.21.15.113:0/2403703885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2b00c8200 0x7ff2b01547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:33.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 -- 172.21.15.113:0/2403703885 >> 172.21.15.113:0/2403703885 conn(0x7ff2b006e820 msgr2=0x7ff2b0070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 -- 172.21.15.113:0/2403703885 shutdown_connections 2024-03-04T08:21:33.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:33.574+0000 7ff2b8800640 1 -- 172.21.15.113:0/2403703885 wait complete. 2024-03-04T08:21:33.592 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:21:35.593 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:35.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.738+0000 7fa51cc00640 1 Processor -- start 2024-03-04T08:21:35.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.738+0000 7fa51cc00640 1 -- start start 2024-03-04T08:21:35.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa51cc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa518152bd0 0x7fa518070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:35.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa51cc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5181591f0 con 0x7fa5181527f0 2024-03-04T08:21:35.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa51cc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5181594d0 con 0x7fa5180c8200 2024-03-04T08:21:35.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa51cc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa518071df0 con 0x7fa518152bd0 2024-03-04T08:21:35.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa515e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa5181527f0 0x7fa518072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50074/0 (socket says 172.21.15.113:50074) 2024-03-04T08:21:35.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa515e00640 1 -- 172.21.15.113:0/3396125584 learned_addr learned my addr 172.21.15.113:0/3396125584 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:35.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa516800640 1 --2- 172.21.15.113:0/3396125584 >> v2:172.21.15.136:3300/0 conn(0x7fa518152bd0 0x7fa518070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:35.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186319537 0 0) 0x7fa5181591f0 con 0x7fa5181527f0 2024-03-04T08:21:35.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4fc003530 con 0x7fa5181527f0 2024-03-04T08:21:35.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3999368103 0 0) 0x7fa4fc003530 con 0x7fa5181527f0 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 >> v2:172.21.15.136:3300/0 conn(0x7fa518152bd0 msgr2=0x7fa518070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 --2- 172.21.15.113:0/3396125584 >> v2:172.21.15.136:3300/0 conn(0x7fa518152bd0 0x7fa518070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 >> v1:172.21.15.136:6789/0 conn(0x7fa5180c8200 legacy=0x7fa5181587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5180717a0 con 0x7fa5181527f0 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fa508001680 con 0x7fa5181527f0 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 >> v1:172.21.15.113:6789/0 conn(0x7fa5181527f0 legacy=0x7fa518072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa4fc0039c0 0x7fa4fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4fc004810 0x7fa4fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa4fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa518071df0 con 0x7fa4fc0051d0 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc00c070 con 0x7fa4fc004810 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc00c270 con 0x7fa4fc0039c0 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa516800640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa4fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:35.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa515e00640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa4fc0039c0 0x7fa4fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:35.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa515400640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4fc004810 0x7fa4fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:35.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa516800640 1 -- 172.21.15.113:0/3396125584 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa4fc0039c0 msgr2=0x7fa4fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:35.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa516800640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa4fc0039c0 0x7fa4fc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa516800640 1 -- 172.21.15.113:0/3396125584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4fc004810 msgr2=0x7fa4fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:35.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa516800640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4fc004810 0x7fa4fc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.742+0000 7fa516800640 1 -- 172.21.15.113:0/3396125584 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc003530 con 0x7fa4fc0051d0 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa516800640 1 -- 172.21.15.113:0/3396125584 reap_dead start 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa516800640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa4fc00b9d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fa510009fc0 tx=0x7fa51001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9594d513e61138c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510009c10 con 0x7fa4fc0051d0 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5100a5cd0 con 0x7fa4fc0051d0 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa514a00640 1 -- 172.21.15.113:0/3396125584 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510003be0 con 0x7fa4fc0051d0 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa51cc00640 1 -- 172.21.15.113:0/3396125584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 msgr2=0x7fa4fc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa51cc00640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa4fc00b9d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fa510009fc0 tx=0x7fa51001ccc0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa51cc00640 1 -- 172.21.15.113:0/3396125584 shutdown_connections 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa51cc00640 1 --2- 172.21.15.113:0/3396125584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa4fc00b9d0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa51cc00640 1 -- 172.21.15.113:0/3396125584 >> 172.21.15.113:0/3396125584 conn(0x7fa51806e820 msgr2=0x7fa51806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa51cc00640 1 -- 172.21.15.113:0/3396125584 shutdown_connections 2024-03-04T08:21:35.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa51cc00640 1 -- 172.21.15.113:0/3396125584 wait complete. 2024-03-04T08:21:35.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa51cc00640 1 Processor -- start 2024-03-04T08:21:35.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.746+0000 7fa51cc00640 1 -- start start 2024-03-04T08:21:35.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa51cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa5180c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:35.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa51cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa518154f30 0x7fa5181f0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa51cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5181f0670 0x7fa5181f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa51cc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa518070d40 con 0x7fa4fc0051d0 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa51cc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5181597b0 con 0x7fa518154f30 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa51cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa518070ec0 con 0x7fa5181f0670 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa515e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa5180c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa515e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa5180c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52074/0 (socket says 172.21.15.113:52074) 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa515e00640 1 -- 172.21.15.113:0/1148132531 learned_addr learned my addr 172.21.15.113:0/1148132531 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa515400640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa518154f30 0x7fa5181f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa516800640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5181f0670 0x7fa5181f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa515e00640 1 -- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5181f0670 msgr2=0x7fa5181f1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa515e00640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5181f0670 0x7fa5181f1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa515e00640 1 -- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa518154f30 msgr2=0x7fa5181f0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa515e00640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa518154f30 0x7fa5181f0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa515e00640 1 -- 172.21.15.113:0/1148132531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5181f0af0 con 0x7fa4fc0051d0 2024-03-04T08:21:35.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa515e00640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa5180c6150 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fa508001060 tx=0x7fa50800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:35.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa50e000640 1 -- 172.21.15.113:0/1148132531 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800ee70 con 0x7fa4fc0051d0 2024-03-04T08:21:35.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa50e000640 1 -- 172.21.15.113:0/1148132531 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa508018070 con 0x7fa4fc0051d0 2024-03-04T08:21:35.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa51cc00640 1 -- 172.21.15.113:0/1148132531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51815bfd0 con 0x7fa4fc0051d0 2024-03-04T08:21:35.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa50e000640 1 -- 172.21.15.113:0/1148132531 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800cad0 con 0x7fa4fc0051d0 2024-03-04T08:21:35.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa51cc00640 1 -- 172.21.15.113:0/1148132531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5181ef370 con 0x7fa4fc0051d0 2024-03-04T08:21:35.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa51cc00640 1 -- 172.21.15.113:0/1148132531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa518075c00 con 0x7fa4fc0051d0 2024-03-04T08:21:35.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.750+0000 7fa50e000640 1 -- 172.21.15.113:0/1148132531 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa50800cc70 con 0x7fa4fc0051d0 2024-03-04T08:21:35.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.754+0000 7fa50e000640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa4f8066f60 0x7fa4f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:35.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.754+0000 7fa50e000640 1 -- 172.21.15.113:0/1148132531 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8061+0+0 (secure 0 0 0) 0x7fa508085790 con 0x7fa4fc0051d0 2024-03-04T08:21:35.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.754+0000 7fa515400640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa4f8066f60 0x7fa4f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:35.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.754+0000 7fa515400640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa4f8066f60 0x7fa4f8069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa504000c10 tx=0x7fa504009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:35.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.758+0000 7fa50e000640 1 -- 172.21.15.113:0/1148132531 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa5181f0af0 con 0x7fa4fc0051d0 2024-03-04T08:21:35.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.942+0000 7fa51cc00640 1 -- 172.21.15.113:0/1148132531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5180c79d0 con 0x7fa4fc0051d0 2024-03-04T08:21:35.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.942+0000 7fa50e000640 1 -- 172.21.15.113:0/1148132531 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5825 (secure 0 0 0) 0x7fa508055dd0 con 0x7fa4fc0051d0 2024-03-04T08:21:35.943 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:31.002944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:35.943 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 172 2024-03-04T08:21:35.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 -- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa4f8066f60 msgr2=0x7fa4f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa4f8066f60 0x7fa4f8069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa504000c10 tx=0x7fa504009040 comp rx=0 tx=0).stop 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 -- 172.21.15.113:0/1148132531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 msgr2=0x7fa5180c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa5180c6150 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fa508001060 tx=0x7fa50800ec80 comp rx=0 tx=0).stop 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 -- 172.21.15.113:0/1148132531 shutdown_connections 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5181f0670 0x7fa5181f1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa518154f30 0x7fa5181f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa4f8066f60 0x7fa4f8069420 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 --2- 172.21.15.113:0/1148132531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4fc0051d0 0x7fa5180c6150 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 -- 172.21.15.113:0/1148132531 >> 172.21.15.113:0/1148132531 conn(0x7fa51806e820 msgr2=0x7fa5180b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 -- 172.21.15.113:0/1148132531 shutdown_connections 2024-03-04T08:21:35.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:35.946+0000 7fa51cc00640 1 -- 172.21.15.113:0/1148132531 wait complete. 2024-03-04T08:21:35.964 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:21:37.965 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:38.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.110+0000 7fb9e2e00640 1 Processor -- start 2024-03-04T08:21:38.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.110+0000 7fb9e2e00640 1 -- start start 2024-03-04T08:21:38.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e2e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb9dc0c22e0 0x7fb9dc15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e2e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9dc0c2d80 0x7fb9dc15b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e2e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9dc0c27f0 con 0x7fb9dc14e4c0 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e2e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9dc15ad60 con 0x7fb9dc0c2d80 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e2e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9dc0c2a90 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9dc0c2d80 0x7fb9dc15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9dc0c2d80 0x7fb9dc15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46280/0 (socket says 172.21.15.113:46280) 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e0e00640 1 -- 172.21.15.113:0/3268242129 learned_addr learned my addr 172.21.15.113:0/3268242129 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9d3400640 1 --2- 172.21.15.113:0/3268242129 >> v2:172.21.15.136:3300/0 conn(0x7fb9dc0c22e0 0x7fb9dc15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e0e00640 1 -- 172.21.15.113:0/3268242129 >> v2:172.21.15.136:3300/0 conn(0x7fb9dc0c22e0 msgr2=0x7fb9dc15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e0e00640 1 --2- 172.21.15.113:0/3268242129 >> v2:172.21.15.136:3300/0 conn(0x7fb9dc0c22e0 0x7fb9dc15b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:38.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e0e00640 1 -- 172.21.15.113:0/3268242129 >> v1:172.21.15.136:6789/0 conn(0x7fb9dc14e4c0 legacy=0x7fb9dc072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:38.114 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.114+0000 7fb9e0e00640 1 -- 172.21.15.113:0/3268242129 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc070c60 con 0x7fb9dc0c2d80 2024-03-04T08:21:38.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e0e00640 1 --2- 172.21.15.113:0/3268242129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c2d80 0x7fb9dc15b730 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0cd230 tx=0x7fb9d401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06d84aa8a8e5af2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:38.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9db400640 1 -- 172.21.15.113:0/3268242129 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d40a57d0 con 0x7fb9dc0c2d80 2024-03-04T08:21:38.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9db400640 1 -- 172.21.15.113:0/3268242129 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d40a5970 con 0x7fb9dc0c2d80 2024-03-04T08:21:38.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9db400640 1 -- 172.21.15.113:0/3268242129 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d40a5d40 con 0x7fb9dc0c2d80 2024-03-04T08:21:38.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e2e00640 1 -- 172.21.15.113:0/3268242129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c2d80 msgr2=0x7fb9dc15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:38.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e2e00640 1 --2- 172.21.15.113:0/3268242129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c2d80 0x7fb9dc15b730 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0cd230 tx=0x7fb9d401c9f0 comp rx=0 tx=0).stop 2024-03-04T08:21:38.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e2e00640 1 -- 172.21.15.113:0/3268242129 shutdown_connections 2024-03-04T08:21:38.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e2e00640 1 --2- 172.21.15.113:0/3268242129 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c2d80 0x7fb9dc15b730 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:38.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e2e00640 1 --2- 172.21.15.113:0/3268242129 >> v2:172.21.15.136:3300/0 conn(0x7fb9dc0c22e0 0x7fb9dc15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:38.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e2e00640 1 -- 172.21.15.113:0/3268242129 >> 172.21.15.113:0/3268242129 conn(0x7fb9dc06e820 msgr2=0x7fb9dc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:38.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e2e00640 1 -- 172.21.15.113:0/3268242129 shutdown_connections 2024-03-04T08:21:38.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e2e00640 1 -- 172.21.15.113:0/3268242129 wait complete. 2024-03-04T08:21:38.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e2e00640 1 Processor -- start 2024-03-04T08:21:38.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.118+0000 7fb9e2e00640 1 -- start start 2024-03-04T08:21:38.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e2e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c22e0 0x7fb9dc14c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:38.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e2e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9dc0c2d80 0x7fb9dc14cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:38.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e2e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc14e4c0 0x7fb9dc146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:38.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e2e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc0aede0 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e2e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc14e8a0 con 0x7fb9dc0c2d80 2024-03-04T08:21:38.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e2e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc073390 con 0x7fb9dc14e4c0 2024-03-04T08:21:38.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c22e0 0x7fb9dc14c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c22e0 0x7fb9dc14c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46286/0 (socket says 172.21.15.113:46286) 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9dbe00640 1 -- 172.21.15.113:0/2727063347 learned_addr learned my addr 172.21.15.113:0/2727063347 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9d3400640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9dc0c2d80 0x7fb9dc14cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e0e00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc14e4c0 0x7fb9dc146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9dbe00640 1 -- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc14e4c0 msgr2=0x7fb9dc146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9dbe00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc14e4c0 0x7fb9dc146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9dbe00640 1 -- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9dc0c2d80 msgr2=0x7fb9dc14cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9dbe00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9dc0c2d80 0x7fb9dc14cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9dbe00640 1 -- 172.21.15.113:0/2727063347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc145df0 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e0e00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc14e4c0 0x7fb9dc146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9dbe00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c22e0 0x7fb9dc14c620 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00c450 tx=0x7fb9cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:38.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9d8c00640 1 -- 172.21.15.113:0/2727063347 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc016020 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9d8c00640 1 -- 172.21.15.113:0/2727063347 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc0052d0 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e2e00640 1 -- 172.21.15.113:0/2727063347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9dc147430 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9d8c00640 1 -- 172.21.15.113:0/2727063347 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc007260 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e2e00640 1 -- 172.21.15.113:0/2727063347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9dc141610 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9e2e00640 1 -- 172.21.15.113:0/2727063347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9dc141940 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.122+0000 7fb9d8c00640 1 -- 172.21.15.113:0/2727063347 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9cc007400 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.126+0000 7fb9d8c00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9b8066f60 0x7fb9b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.126+0000 7fb9d8c00640 1 -- 172.21.15.113:0/2727063347 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8061+0+0 (secure 0 0 0) 0x7fb9cc0851f0 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.126+0000 7fb9d3400640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9b8066f60 0x7fb9b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:38.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.126+0000 7fb9d3400640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9b8066f60 0x7fb9b8069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0cb400 tx=0x7fb9c4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:38.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.130+0000 7fb9d8c00640 1 -- 172.21.15.113:0/2727063347 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb9cc020020 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.318+0000 7fb9e2e00640 1 -- 172.21.15.113:0/2727063347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9dc147950 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.318+0000 7fb9d8c00640 1 -- 172.21.15.113:0/2727063347 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5825 (secure 0 0 0) 0x7fb9dc147950 con 0x7fb9dc0c22e0 2024-03-04T08:21:38.318 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:38.318 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:31.002944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:38.318 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 172 2024-03-04T08:21:38.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 -- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9b8066f60 msgr2=0x7fb9b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:38.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9b8066f60 0x7fb9b8069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0cb400 tx=0x7fb9c4009190 comp rx=0 tx=0).stop 2024-03-04T08:21:38.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 -- 172.21.15.113:0/2727063347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c22e0 msgr2=0x7fb9dc14c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:38.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c22e0 0x7fb9dc14c620 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00c450 tx=0x7fb9cc00c910 comp rx=0 tx=0).stop 2024-03-04T08:21:38.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 -- 172.21.15.113:0/2727063347 shutdown_connections 2024-03-04T08:21:38.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9dc14e4c0 0x7fb9dc146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:38.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9dc0c2d80 0x7fb9dc14cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:38.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb9b8066f60 0x7fb9b8069420 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:38.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 --2- 172.21.15.113:0/2727063347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9dc0c22e0 0x7fb9dc14c620 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:38.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 -- 172.21.15.113:0/2727063347 >> 172.21.15.113:0/2727063347 conn(0x7fb9dc06e820 msgr2=0x7fb9dc06b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:38.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 -- 172.21.15.113:0/2727063347 shutdown_connections 2024-03-04T08:21:38.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:38.322+0000 7fb9e2e00640 1 -- 172.21.15.113:0/2727063347 wait complete. 2024-03-04T08:21:38.339 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:21:40.340 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:40.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.486+0000 7fe9d5400640 1 Processor -- start 2024-03-04T08:21:40.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9d5400640 1 -- start start 2024-03-04T08:21:40.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9d5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d01527f0 0x7fe9d0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:40.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9d5400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d01591f0 con 0x7fe9d0152bd0 2024-03-04T08:21:40.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9d5400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d01594d0 con 0x7fe9d00c8200 2024-03-04T08:21:40.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9d5400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0071df0 con 0x7fe9d01527f0 2024-03-04T08:21:40.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cf200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe9d0152bd0 0x7fe9d01587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57256/0 (socket says 172.21.15.113:57256) 2024-03-04T08:21:40.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cf200640 1 -- 172.21.15.113:0/4241291773 learned_addr learned my addr 172.21.15.113:0/4241291773 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:40.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9ce800640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d01527f0 0x7fe9d0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:40.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465862005 0 0) 0x7fe9d01591f0 con 0x7fe9d0152bd0 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b4003530 con 0x7fe9d0152bd0 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914052044 0 0) 0x7fe9d01594d0 con 0x7fe9d00c8200 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9d01591f0 con 0x7fe9d00c8200 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3727538110 0 0) 0x7fe9b4003530 con 0x7fe9d0152bd0 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d01527f0 msgr2=0x7fe9d0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d01527f0 0x7fe9d0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 >> v1:172.21.15.136:6789/0 conn(0x7fe9d00c8200 legacy=0x7fe9d0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d00717a0 con 0x7fe9d0152bd0 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fe9c8001a00 con 0x7fe9d0152bd0 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 >> v1:172.21.15.113:6789/0 conn(0x7fe9d0152bd0 legacy=0x7fe9d01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d01527f0 0x7fe9b4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b40040e0 0x7fe9b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b4004d50 0x7fe9b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:40.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0071df0 con 0x7fe9b4004d50 2024-03-04T08:21:40.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b400bab0 con 0x7fe9b40040e0 2024-03-04T08:21:40.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b400bcb0 con 0x7fe9d01527f0 2024-03-04T08:21:40.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cf200640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b4004d50 0x7fe9b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:40.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9ce800640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d01527f0 0x7fe9b4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:40.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9c5e00640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b40040e0 0x7fe9b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:40.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cf200640 1 -- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d01527f0 msgr2=0x7fe9b4003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:40.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cf200640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d01527f0 0x7fe9b4003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:40.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cf200640 1 -- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b40040e0 msgr2=0x7fe9b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:40.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cf200640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b40040e0 0x7fe9b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:40.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.490+0000 7fe9cf200640 1 -- 172.21.15.113:0/4241291773 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b4003530 con 0x7fe9b4004d50 2024-03-04T08:21:40.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9cf200640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b4004d50 0x7fe9b40051d0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800a7c0 tx=0x7fe9c801fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93c1a554cdc2869 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:40.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800a410 con 0x7fe9b4004d50 2024-03-04T08:21:40.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c80aa690 con 0x7fe9b4004d50 2024-03-04T08:21:40.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9cde00640 1 -- 172.21.15.113:0/4241291773 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80aaac0 con 0x7fe9b4004d50 2024-03-04T08:21:40.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9d5400640 1 -- 172.21.15.113:0/4241291773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b4004d50 msgr2=0x7fe9b40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:40.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9d5400640 1 --2- 172.21.15.113:0/4241291773 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9b4004d50 0x7fe9b40051d0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800a7c0 tx=0x7fe9c801fa40 comp rx=0 tx=0).stop 2024-03-04T08:21:40.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9cf200640 1 -- 172.21.15.113:0/4241291773 reap_dead start 2024-03-04T08:21:40.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9d5400640 1 -- 172.21.15.113:0/4241291773 shutdown_connections 2024-03-04T08:21:40.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9d5400640 1 -- 172.21.15.113:0/4241291773 >> 172.21.15.113:0/4241291773 conn(0x7fe9d006e820 msgr2=0x7fe9d006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:40.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9d5400640 1 -- 172.21.15.113:0/4241291773 shutdown_connections 2024-03-04T08:21:40.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.494+0000 7fe9d5400640 1 -- 172.21.15.113:0/4241291773 wait complete. 2024-03-04T08:21:40.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9d5400640 1 Processor -- start 2024-03-04T08:21:40.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9d5400640 1 -- start start 2024-03-04T08:21:40.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9d5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b4004d50 0x7fe9d00c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:40.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9d5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d00c3550 0x7fe9d0157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:40.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9d5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d00c4e90 0x7fe9d00c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:40.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9d5400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0158ee0 con 0x7fe9d00c3550 2024-03-04T08:21:40.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9d5400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d01597b0 con 0x7fe9b4004d50 2024-03-04T08:21:40.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9d5400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0070d40 con 0x7fe9d00c4e90 2024-03-04T08:21:40.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d00c3550 0x7fe9d0157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:40.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9ce800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b4004d50 0x7fe9d00c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:40.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9cf200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d00c4e90 0x7fe9d00c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:40.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c5e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d00c3550 0x7fe9d0157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46298/0 (socket says 172.21.15.113:46298) 2024-03-04T08:21:40.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c5e00640 1 -- 172.21.15.113:0/3470260889 learned_addr learned my addr 172.21.15.113:0/3470260889 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:40.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c5e00640 1 -- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d00c4e90 msgr2=0x7fe9d00c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:40.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c5e00640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d00c4e90 0x7fe9d00c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:40.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c5e00640 1 -- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b4004d50 msgr2=0x7fe9d00c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:40.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c5e00640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b4004d50 0x7fe9d00c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:40.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c5e00640 1 -- 172.21.15.113:0/3470260889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d0156460 con 0x7fe9d00c3550 2024-03-04T08:21:40.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c5e00640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d00c3550 0x7fe9d0157800 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800f7d0 tx=0x7fe9b800fbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:40.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c7400640 1 -- 172.21.15.113:0/3470260889 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80050d0 con 0x7fe9d00c3550 2024-03-04T08:21:40.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c7400640 1 -- 172.21.15.113:0/3470260889 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8018070 con 0x7fe9d00c3550 2024-03-04T08:21:40.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9c7400640 1 -- 172.21.15.113:0/3470260889 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800d6d0 con 0x7fe9d00c3550 2024-03-04T08:21:40.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9d5400640 1 -- 172.21.15.113:0/3470260889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d00ae4f0 con 0x7fe9d00c3550 2024-03-04T08:21:40.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.498+0000 7fe9d5400640 1 -- 172.21.15.113:0/3470260889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d01f3e90 con 0x7fe9d00c3550 2024-03-04T08:21:40.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.502+0000 7fe9d5400640 1 -- 172.21.15.113:0/3470260889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d0075c00 con 0x7fe9d00c3550 2024-03-04T08:21:40.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.502+0000 7fe9c7400640 1 -- 172.21.15.113:0/3470260889 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9b800d870 con 0x7fe9d00c3550 2024-03-04T08:21:40.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.502+0000 7fe9c7400640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe9ac066f60 0x7fe9ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:40.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.502+0000 7fe9c7400640 1 -- 172.21.15.113:0/3470260889 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 8061+0+0 (secure 0 0 0) 0x7fe9b8085730 con 0x7fe9d00c3550 2024-03-04T08:21:40.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.502+0000 7fe9ce800640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe9ac066f60 0x7fe9ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:40.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.502+0000 7fe9ce800640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe9ac066f60 0x7fe9ac069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009be0 tx=0x7fe9c0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:40.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.506+0000 7fe9c7400640 1 -- 172.21.15.113:0/3470260889 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe9b8055d70 con 0x7fe9d00c3550 2024-03-04T08:21:40.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 -- 172.21.15.113:0/3470260889 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9d00c79d0 con 0x7fe9d00c3550 2024-03-04T08:21:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9c7400640 1 -- 172.21.15.113:0/3470260889 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5825 (secure 0 0 0) 0x7fe9b8055670 con 0x7fe9d00c3550 2024-03-04T08:21:40.696 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:40.696 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4247,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:31.002944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7839,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7839":{"gid":7839,"name":"e","rank":0,"incarnation":145,"state":"up:active","state_seq":42,"addr":"172.21.15.113:6837/2544977425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":2544977425},{"type":"v1","addr":"172.21.15.113:6837","nonce":2544977425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7839,"qdb_cluster":[7839,8032]},"id":1}]} 2024-03-04T08:21:40.696 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 172 2024-03-04T08:21:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 -- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe9ac066f60 msgr2=0x7fe9ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe9ac066f60 0x7fe9ac069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009be0 tx=0x7fe9c0009190 comp rx=0 tx=0).stop 2024-03-04T08:21:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 -- 172.21.15.113:0/3470260889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d00c3550 msgr2=0x7fe9d0157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d00c3550 0x7fe9d0157800 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800f7d0 tx=0x7fe9b800fbd0 comp rx=0 tx=0).stop 2024-03-04T08:21:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 -- 172.21.15.113:0/3470260889 shutdown_connections 2024-03-04T08:21:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe9d00c4e90 0x7fe9d00c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe9ac066f60 0x7fe9ac069420 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9d00c3550 0x7fe9d0157800 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:40.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 --2- 172.21.15.113:0/3470260889 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9b4004d50 0x7fe9d00c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 -- 172.21.15.113:0/3470260889 >> 172.21.15.113:0/3470260889 conn(0x7fe9d006e820 msgr2=0x7fe9d00c2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 -- 172.21.15.113:0/3470260889 shutdown_connections 2024-03-04T08:21:40.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:40.698+0000 7fe9d5400640 1 -- 172.21.15.113:0/3470260889 wait complete. 2024-03-04T08:21:40.717 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2024-03-04T08:21:42.718 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:42.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.866+0000 7fcf00c00640 1 Processor -- start 2024-03-04T08:21:42.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.866+0000 7fcf00c00640 1 -- start start 2024-03-04T08:21:42.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.866+0000 7fcf00c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcefc0c22e0 0x7fcefc15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:42.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.866+0000 7fcf00c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcefc0c27f0 con 0x7fcefc14e4c0 2024-03-04T08:21:42.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.866+0000 7fcf00c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcefc0c2a90 con 0x7fcefc0c2d80 2024-03-04T08:21:42.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.866+0000 7fcf00c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcefc15ad60 con 0x7fcefc0c22e0 2024-03-04T08:21:42.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.866+0000 7fcef1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcefc0c22e0 0x7fcefc15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:42.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.866+0000 7fcef9e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcefc14e4c0 0x7fcefc072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57266/0 (socket says 172.21.15.113:57266) 2024-03-04T08:21:42.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.866+0000 7fcef9e00640 1 -- 172.21.15.113:0/1860774698 learned_addr learned my addr 172.21.15.113:0/1860774698 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:42.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.870+0000 7fcef1400640 1 -- 172.21.15.113:0/1860774698 >> v1:172.21.15.136:6789/0 conn(0x7fcefc0c2d80 legacy=0x7fcefc1620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:42.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.870+0000 7fcef1400640 1 -- 172.21.15.113:0/1860774698 >> v1:172.21.15.113:6789/0 conn(0x7fcefc14e4c0 legacy=0x7fcefc072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:42.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.870+0000 7fcef9400640 1 -- 172.21.15.113:0/1860774698 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359666276 0 0) 0x7fcefc0c27f0 con 0x7fcefc14e4c0 2024-03-04T08:21:42.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.870+0000 7fcef1400640 1 -- 172.21.15.113:0/1860774698 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc070c60 con 0x7fcefc0c22e0 2024-03-04T08:21:42.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.870+0000 7fcef1400640 1 --2- 172.21.15.113:0/1860774698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c22e0 0x7fcefc15b1f0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fceec0097c0 tx=0x7fceec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8c8af457bd0e20 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:42.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.870+0000 7fcef9400640 1 -- 172.21.15.113:0/1860774698 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0b1070 con 0x7fcefc0c22e0 2024-03-04T08:21:42.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.870+0000 7fcef9400640 1 -- 172.21.15.113:0/1860774698 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec0ac5f0 con 0x7fcefc0c22e0 2024-03-04T08:21:42.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.870+0000 7fcef9400640 1 -- 172.21.15.113:0/1860774698 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0ac9c0 con 0x7fcefc0c22e0 2024-03-04T08:21:42.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.870+0000 7fcf00c00640 1 -- 172.21.15.113:0/1860774698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c22e0 msgr2=0x7fcefc15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:42.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.870+0000 7fcf00c00640 1 --2- 172.21.15.113:0/1860774698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c22e0 0x7fcefc15b1f0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fceec0097c0 tx=0x7fceec01c4a0 comp rx=0 tx=0).stop 2024-03-04T08:21:42.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 -- 172.21.15.113:0/1860774698 shutdown_connections 2024-03-04T08:21:42.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 --2- 172.21.15.113:0/1860774698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c22e0 0x7fcefc15b1f0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:42.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 -- 172.21.15.113:0/1860774698 >> 172.21.15.113:0/1860774698 conn(0x7fcefc06e820 msgr2=0x7fcefc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:42.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 -- 172.21.15.113:0/1860774698 shutdown_connections 2024-03-04T08:21:42.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 -- 172.21.15.113:0/1860774698 wait complete. 2024-03-04T08:21:42.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 Processor -- start 2024-03-04T08:21:42.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 -- start start 2024-03-04T08:21:42.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcefc0c22e0 0x7fcefc145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:42.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c2d80 0x7fcefc1f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:42.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcefc14e4c0 0x7fcefc1f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:42.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc14e8a0 con 0x7fcefc0c2d80 2024-03-04T08:21:42.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc15bb40 con 0x7fcefc0c22e0 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0aede0 con 0x7fcefc14e4c0 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c2d80 0x7fcefc1f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c2d80 0x7fcefc1f4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46308/0 (socket says 172.21.15.113:46308) 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef1400640 1 -- 172.21.15.113:0/4182953872 learned_addr learned my addr 172.21.15.113:0/4182953872 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef9e00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcefc0c22e0 0x7fcefc145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcefa800640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcefc14e4c0 0x7fcefc1f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef1400640 1 -- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcefc14e4c0 msgr2=0x7fcefc1f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef1400640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcefc14e4c0 0x7fcefc1f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef1400640 1 -- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcefc0c22e0 msgr2=0x7fcefc145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef1400640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcefc0c22e0 0x7fcefc145f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef1400640 1 -- 172.21.15.113:0/4182953872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc1492d0 con 0x7fcefc0c2d80 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef9e00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcefc0c22e0 0x7fcefc145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcefa800640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcefc14e4c0 0x7fcefc1f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:42.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef1400640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c2d80 0x7fcefc1f4500 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fceec0af040 tx=0x7fceec01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef2a00640 1 -- 172.21.15.113:0/4182953872 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0b1040 con 0x7fcefc0c2d80 2024-03-04T08:21:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef2a00640 1 -- 172.21.15.113:0/4182953872 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec004030 con 0x7fcefc0c2d80 2024-03-04T08:21:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcef2a00640 1 -- 172.21.15.113:0/4182953872 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0043e0 con 0x7fcefc0c2d80 2024-03-04T08:21:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.874+0000 7fcf00c00640 1 -- 172.21.15.113:0/4182953872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc1548e0 con 0x7fcefc0c2d80 2024-03-04T08:21:42.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.878+0000 7fcf00c00640 1 -- 172.21.15.113:0/4182953872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc14a5c0 con 0x7fcefc0c2d80 2024-03-04T08:21:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.878+0000 7fcf00c00640 1 -- 172.21.15.113:0/4182953872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcefc1f4330 con 0x7fcefc0c2d80 2024-03-04T08:21:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.878+0000 7fcef2a00640 1 -- 172.21.15.113:0/4182953872 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fceec003620 con 0x7fcefc0c2d80 2024-03-04T08:21:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.878+0000 7fcef2a00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fced8066f90 0x7fced8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.878+0000 7fcef2a00640 1 -- 172.21.15.113:0/4182953872 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fceec120180 con 0x7fcefc0c2d80 2024-03-04T08:21:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.878+0000 7fcef9e00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fced8066f90 0x7fced8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:42.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.878+0000 7fcef9e00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fced8066f90 0x7fced8069450 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcee4000f70 tx=0x7fcee4007c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:42.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:42.882+0000 7fcef2a00640 1 -- 172.21.15.113:0/4182953872 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fceec0a9070 con 0x7fcefc0c2d80 2024-03-04T08:21:43.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.070+0000 7fcf00c00640 1 -- 172.21.15.113:0/4182953872 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcefc075c00 con 0x7fcefc0c2d80 2024-03-04T08:21:43.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcef2a00640 1 -- 172.21.15.113:0/4182953872 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) v1 ==== 95+0+5026 (secure 0 0 0) 0x7fceec016090 con 0x7fcefc0c2d80 2024-03-04T08:21:43.071 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:43.071 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":173,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:40.994141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:21:43.072 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 173 2024-03-04T08:21:43.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 -- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fced8066f90 msgr2=0x7fced8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:43.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fced8066f90 0x7fced8069450 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcee4000f70 tx=0x7fcee4007c60 comp rx=0 tx=0).stop 2024-03-04T08:21:43.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 -- 172.21.15.113:0/4182953872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c2d80 msgr2=0x7fcefc1f4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:43.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c2d80 0x7fcefc1f4500 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fceec0af040 tx=0x7fceec01ace0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 -- 172.21.15.113:0/4182953872 shutdown_connections 2024-03-04T08:21:43.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcefc14e4c0 0x7fcefc1f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fced8066f90 0x7fced8069450 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcefc0c2d80 0x7fcefc1f4500 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 --2- 172.21.15.113:0/4182953872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcefc0c22e0 0x7fcefc145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 -- 172.21.15.113:0/4182953872 >> 172.21.15.113:0/4182953872 conn(0x7fcefc06e820 msgr2=0x7fcefc072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 -- 172.21.15.113:0/4182953872 shutdown_connections 2024-03-04T08:21:43.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.074+0000 7fcf00c00640 1 -- 172.21.15.113:0/4182953872 wait complete. 2024-03-04T08:21:43.092 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2024-03-04T08:21:43.092 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:21:43.092 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:43.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd19000640 1 Processor -- start 2024-03-04T08:21:43.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd19000640 1 -- start start 2024-03-04T08:21:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd19000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdd14154c00 0x7fdd140c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd19000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd140c7d80 con 0x7fdd14154fe0 2024-03-04T08:21:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd19000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd140c8060 con 0x7fdd140c7700 2024-03-04T08:21:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd19000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1415efb0 con 0x7fdd14154c00 2024-03-04T08:21:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd11e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdd14154c00 0x7fdd140c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd11400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdd14154fe0 0x7fdd140c1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57282/0 (socket says 172.21.15.113:57282) 2024-03-04T08:21:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd11400640 1 -- 172.21.15.113:0/311512551 learned_addr learned my addr 172.21.15.113:0/311512551 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd11e00640 1 -- 172.21.15.113:0/311512551 >> v1:172.21.15.136:6789/0 conn(0x7fdd140c7700 legacy=0x7fdd1415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:43.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd11e00640 1 -- 172.21.15.113:0/311512551 >> v1:172.21.15.113:6789/0 conn(0x7fdd14154fe0 legacy=0x7fdd140c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:43.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.230+0000 7fdd11e00640 1 -- 172.21.15.113:0/311512551 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd140c6dc0 con 0x7fdd14154c00 2024-03-04T08:21:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd11e00640 1 --2- 172.21.15.113:0/311512551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd14154c00 0x7fdd140c1190 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009d90 tx=0x7fdd0401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397ef5ae166bcc92 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd10a00640 1 -- 172.21.15.113:0/311512551 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040b2070 con 0x7fdd14154c00 2024-03-04T08:21:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd10a00640 1 -- 172.21.15.113:0/311512551 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd040a57a0 con 0x7fdd14154c00 2024-03-04T08:21:43.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd10a00640 1 -- 172.21.15.113:0/311512551 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040a5b70 con 0x7fdd14154c00 2024-03-04T08:21:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd19000640 1 -- 172.21.15.113:0/311512551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd14154c00 msgr2=0x7fdd140c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd19000640 1 --2- 172.21.15.113:0/311512551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd14154c00 0x7fdd140c1190 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009d90 tx=0x7fdd0401c8e0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd19000640 1 -- 172.21.15.113:0/311512551 shutdown_connections 2024-03-04T08:21:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd19000640 1 --2- 172.21.15.113:0/311512551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd14154c00 0x7fdd140c1190 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd19000640 1 -- 172.21.15.113:0/311512551 >> 172.21.15.113:0/311512551 conn(0x7fdd1406e820 msgr2=0x7fdd1406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd19000640 1 -- 172.21.15.113:0/311512551 shutdown_connections 2024-03-04T08:21:43.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.234+0000 7fdd19000640 1 -- 172.21.15.113:0/311512551 wait complete. 2024-03-04T08:21:43.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 Processor -- start 2024-03-04T08:21:43.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 -- start start 2024-03-04T08:21:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd140c7700 0x7fdd140c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd14154fe0 0x7fdd14151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:43.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd140c3500 0x7fdd1414f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd14168fb0 con 0x7fdd140c7700 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd14168e30 con 0x7fdd140c3500 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd14169130 con 0x7fdd14154fe0 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd140c7700 0x7fdd140c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd11e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd140c7700 0x7fdd140c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46318/0 (socket says 172.21.15.113:46318) 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd11e00640 1 -- 172.21.15.113:0/4053755941 learned_addr learned my addr 172.21.15.113:0/4053755941 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd12800640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd140c3500 0x7fdd1414f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd11400640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd14154fe0 0x7fdd14151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd11e00640 1 -- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd14154fe0 msgr2=0x7fdd14151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd11e00640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd14154fe0 0x7fdd14151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd11e00640 1 -- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd140c3500 msgr2=0x7fdd1414f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:43.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd11e00640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd140c3500 0x7fdd1414f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd11e00640 1 -- 172.21.15.113:0/4053755941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd140c93a0 con 0x7fdd140c7700 2024-03-04T08:21:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd11e00640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd140c7700 0x7fdd140c5a00 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdd040b0040 tx=0x7fdd04019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd0a000640 1 -- 172.21.15.113:0/4053755941 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040b2040 con 0x7fdd140c7700 2024-03-04T08:21:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd0a000640 1 -- 172.21.15.113:0/4053755941 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd040bf070 con 0x7fdd140c7700 2024-03-04T08:21:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 -- 172.21.15.113:0/4053755941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd140c9640 con 0x7fdd140c7700 2024-03-04T08:21:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd0a000640 1 -- 172.21.15.113:0/4053755941 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040b88d0 con 0x7fdd140c7700 2024-03-04T08:21:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 -- 172.21.15.113:0/4053755941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd140c4f20 con 0x7fdd140c7700 2024-03-04T08:21:43.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.238+0000 7fdd19000640 1 -- 172.21.15.113:0/4053755941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd14075c00 con 0x7fdd140c7700 2024-03-04T08:21:43.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.242+0000 7fdd0a000640 1 -- 172.21.15.113:0/4053755941 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd040b8a70 con 0x7fdd140c7700 2024-03-04T08:21:43.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.242+0000 7fdd0a000640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdcf4066f90 0x7fdcf4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:43.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.242+0000 7fdd0a000640 1 -- 172.21.15.113:0/4053755941 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fdd04120500 con 0x7fdd140c7700 2024-03-04T08:21:43.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.242+0000 7fdd11400640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdcf4066f90 0x7fdcf4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:43.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.242+0000 7fdd11400640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdcf4066f90 0x7fdcf4069450 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdd140723d0 tx=0x7fdd00009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:43.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.246+0000 7fdd0a000640 1 -- 172.21.15.113:0/4053755941 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdd040c8020 con 0x7fdd140c7700 2024-03-04T08:21:43.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 -- 172.21.15.113:0/4053755941 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd14068a60 con 0x7fdd140c7700 2024-03-04T08:21:43.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd0a000640 1 -- 172.21.15.113:0/4053755941 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) v1 ==== 95+0+5026 (secure 0 0 0) 0x7fdd04016560 con 0x7fdd140c7700 2024-03-04T08:21:43.436 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:43.436 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":173,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:40.994141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:21:43.436 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 173 2024-03-04T08:21:43.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 -- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdcf4066f90 msgr2=0x7fdcf4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:43.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdcf4066f90 0x7fdcf4069450 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fdd140723d0 tx=0x7fdd00009040 comp rx=0 tx=0).stop 2024-03-04T08:21:43.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 -- 172.21.15.113:0/4053755941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd140c7700 msgr2=0x7fdd140c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:43.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd140c7700 0x7fdd140c5a00 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fdd040b0040 tx=0x7fdd04019680 comp rx=0 tx=0).stop 2024-03-04T08:21:43.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 -- 172.21.15.113:0/4053755941 shutdown_connections 2024-03-04T08:21:43.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdd140c3500 0x7fdd1414f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdd14154fe0 0x7fdd14151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdcf4066f90 0x7fdcf4069450 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 --2- 172.21.15.113:0/4053755941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd140c7700 0x7fdd140c5a00 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:43.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 -- 172.21.15.113:0/4053755941 >> 172.21.15.113:0/4053755941 conn(0x7fdd1406e820 msgr2=0x7fdd1415abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:43.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 -- 172.21.15.113:0/4053755941 shutdown_connections 2024-03-04T08:21:43.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:43.438+0000 7fdd19000640 1 -- 172.21.15.113:0/4053755941 wait complete. 2024-03-04T08:21:43.457 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 173, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 173, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 173, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8388": { "addr": "172.21.15.113:6833/2999156262", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2999156262, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2999156262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8388, "incarnation": 171, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:resolve", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 82, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:21:40.994141+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4247, "mds_1": 8032, "mds_2": 8388 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:21:44.596 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:21:44.600 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:21:45.458 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:45.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda4b400640 1 Processor -- start 2024-03-04T08:21:45.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda4b400640 1 -- start start 2024-03-04T08:21:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda4b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fda440c8200 0x7fda44070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda4b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda44152bd0 0x7fda441587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda4b400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda441591f0 con 0x7fda441527f0 2024-03-04T08:21:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda4b400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda44071df0 con 0x7fda440c8200 2024-03-04T08:21:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda4b400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda441597b0 con 0x7fda44152bd0 2024-03-04T08:21:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda3be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fda440c8200 0x7fda44070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:45.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda3be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fda440c8200 0x7fda44070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46320/0 (socket says 172.21.15.113:46320) 2024-03-04T08:21:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda48a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fda441527f0 0x7fda44072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57298/0 (socket says 172.21.15.113:57298) 2024-03-04T08:21:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda3be00640 1 -- 172.21.15.113:0/392851575 learned_addr learned my addr 172.21.15.113:0/392851575 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda49400640 1 --2- 172.21.15.113:0/392851575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda44152bd0 0x7fda441587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda3be00640 1 -- 172.21.15.113:0/392851575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda44152bd0 msgr2=0x7fda441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda3be00640 1 --2- 172.21.15.113:0/392851575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda44152bd0 0x7fda441587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda3be00640 1 -- 172.21.15.113:0/392851575 >> v1:172.21.15.113:6789/0 conn(0x7fda441527f0 legacy=0x7fda44072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda3be00640 1 -- 172.21.15.113:0/392851575 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda441594d0 con 0x7fda440c8200 2024-03-04T08:21:45.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.606+0000 7fda43e00640 1 -- 172.21.15.113:0/392851575 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194654638 0 0) 0x7fda441591f0 con 0x7fda441527f0 2024-03-04T08:21:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda3be00640 1 --2- 172.21.15.113:0/392851575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda440c8200 0x7fda44070d40 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009870 tx=0x7fda2c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3401a7b5614d139c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda43e00640 1 -- 172.21.15.113:0/392851575 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0a7070 con 0x7fda440c8200 2024-03-04T08:21:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda43e00640 1 -- 172.21.15.113:0/392851575 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c0b0070 con 0x7fda440c8200 2024-03-04T08:21:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda43e00640 1 -- 172.21.15.113:0/392851575 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0ab890 con 0x7fda440c8200 2024-03-04T08:21:45.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda4b400640 1 -- 172.21.15.113:0/392851575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda440c8200 msgr2=0x7fda44070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:45.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda4b400640 1 --2- 172.21.15.113:0/392851575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda440c8200 0x7fda44070d40 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fda2c009870 tx=0x7fda2c01c560 comp rx=0 tx=0).stop 2024-03-04T08:21:45.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda4b400640 1 -- 172.21.15.113:0/392851575 shutdown_connections 2024-03-04T08:21:45.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda4b400640 1 --2- 172.21.15.113:0/392851575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda44152bd0 0x7fda441587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:45.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda4b400640 1 --2- 172.21.15.113:0/392851575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda440c8200 0x7fda44070d40 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:45.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda4b400640 1 -- 172.21.15.113:0/392851575 >> 172.21.15.113:0/392851575 conn(0x7fda4406e820 msgr2=0x7fda4406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:45.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda4b400640 1 -- 172.21.15.113:0/392851575 shutdown_connections 2024-03-04T08:21:45.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.610+0000 7fda4b400640 1 -- 172.21.15.113:0/392851575 wait complete. 2024-03-04T08:21:45.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 Processor -- start 2024-03-04T08:21:45.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 -- start start 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda440c8200 0x7fda44158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda441527f0 0x7fda440cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda44152bd0 0x7fda44145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda441654f0 con 0x7fda44152bd0 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda44165370 con 0x7fda441527f0 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda44165670 con 0x7fda440c8200 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda49400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda44152bd0 0x7fda44145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda49400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda44152bd0 0x7fda44145c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46322/0 (socket says 172.21.15.113:46322) 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda49400640 1 -- 172.21.15.113:0/219587429 learned_addr learned my addr 172.21.15.113:0/219587429 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda48a00640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda440c8200 0x7fda44158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda3be00640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda441527f0 0x7fda440cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:45.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda49400640 1 -- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda440c8200 msgr2=0x7fda44158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:45.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda49400640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda440c8200 0x7fda44158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:45.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda49400640 1 -- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda441527f0 msgr2=0x7fda440cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:45.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda49400640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda441527f0 0x7fda440cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:45.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda49400640 1 -- 172.21.15.113:0/219587429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda440aff00 con 0x7fda44152bd0 2024-03-04T08:21:45.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda49400640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda44152bd0 0x7fda44145c70 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009d30 tx=0x7fda3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:45.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda41600640 1 -- 172.21.15.113:0/219587429 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c018020 con 0x7fda44152bd0 2024-03-04T08:21:45.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda41600640 1 -- 172.21.15.113:0/219587429 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c014480 con 0x7fda44152bd0 2024-03-04T08:21:45.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 -- 172.21.15.113:0/219587429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda440af980 con 0x7fda44152bd0 2024-03-04T08:21:45.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda41600640 1 -- 172.21.15.113:0/219587429 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c014870 con 0x7fda44152bd0 2024-03-04T08:21:45.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 -- 172.21.15.113:0/219587429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda440b1600 con 0x7fda44152bd0 2024-03-04T08:21:45.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.614+0000 7fda4b400640 1 -- 172.21.15.113:0/219587429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda44075c00 con 0x7fda44152bd0 2024-03-04T08:21:45.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.618+0000 7fda41600640 1 -- 172.21.15.113:0/219587429 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda3c01b020 con 0x7fda44152bd0 2024-03-04T08:21:45.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.618+0000 7fda41600640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda20066f60 0x7fda20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:45.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.618+0000 7fda41600640 1 -- 172.21.15.113:0/219587429 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fda3c0864a0 con 0x7fda44152bd0 2024-03-04T08:21:45.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.618+0000 7fda48a00640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda20066f60 0x7fda20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:45.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.618+0000 7fda48a00640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda20066f60 0x7fda20069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fda34000f70 tx=0x7fda34007b60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:45.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.622+0000 7fda41600640 1 -- 172.21.15.113:0/219587429 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fda3c022070 con 0x7fda44152bd0 2024-03-04T08:21:45.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.806+0000 7fda4b400640 1 -- 172.21.15.113:0/219587429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda440c19d0 con 0x7fda44152bd0 2024-03-04T08:21:45.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.806+0000 7fda41600640 1 -- 172.21.15.113:0/219587429 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) v1 ==== 95+0+5026 (secure 0 0 0) 0x7fda440aff00 con 0x7fda44152bd0 2024-03-04T08:21:45.806 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:45.806 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":173,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:40.994141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:21:45.806 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 173 2024-03-04T08:21:45.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 -- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda20066f60 msgr2=0x7fda20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:45.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda20066f60 0x7fda20069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fda34000f70 tx=0x7fda34007b60 comp rx=0 tx=0).stop 2024-03-04T08:21:45.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 -- 172.21.15.113:0/219587429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda44152bd0 msgr2=0x7fda44145c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:45.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda44152bd0 0x7fda44145c70 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009d30 tx=0x7fda3c00c490 comp rx=0 tx=0).stop 2024-03-04T08:21:45.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 -- 172.21.15.113:0/219587429 shutdown_connections 2024-03-04T08:21:45.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fda20066f60 0x7fda20069420 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:45.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fda44152bd0 0x7fda44145c70 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:45.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fda441527f0 0x7fda440cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:45.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 --2- 172.21.15.113:0/219587429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fda440c8200 0x7fda44158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:45.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 -- 172.21.15.113:0/219587429 >> 172.21.15.113:0/219587429 conn(0x7fda4406e820 msgr2=0x7fda440c1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:45.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 -- 172.21.15.113:0/219587429 shutdown_connections 2024-03-04T08:21:45.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:45.810+0000 7fda4b400640 1 -- 172.21.15.113:0/219587429 wait complete. 2024-03-04T08:21:45.827 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:47.827 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:47.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.966+0000 7fabc8c00640 1 Processor -- start 2024-03-04T08:21:47.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc8c00640 1 -- start start 2024-03-04T08:21:47.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fabc40c7b10 0x7fabc4152b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:47.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc406b060 0x7fabc4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:47.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc4159320 con 0x7fabc4152780 2024-03-04T08:21:47.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc415be00 con 0x7fabc40c7b10 2024-03-04T08:21:47.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc415c0e0 con 0x7fabc406b060 2024-03-04T08:21:47.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc406b060 0x7fabc4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:47.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabbaa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fabc40c7b10 0x7fabc4152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:47.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc3400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fabc4152780 0x7fabc406ba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:36812/0 (socket says 172.21.15.113:36812) 2024-03-04T08:21:47.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc3400640 1 -- 172.21.15.113:0/1463241886 learned_addr learned my addr 172.21.15.113:0/1463241886 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:47.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc3e00640 1 -- 172.21.15.113:0/1463241886 >> v2:172.21.15.136:3300/0 conn(0x7fabc40c7b10 msgr2=0x7fabc4152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:47.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc3e00640 1 --2- 172.21.15.113:0/1463241886 >> v2:172.21.15.136:3300/0 conn(0x7fabc40c7b10 0x7fabc4152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:47.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc3e00640 1 -- 172.21.15.113:0/1463241886 >> v1:172.21.15.136:6789/0 conn(0x7fabc4152780 legacy=0x7fabc406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:47.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.970+0000 7fabc3e00640 1 -- 172.21.15.113:0/1463241886 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc415c3c0 con 0x7fabc406b060 2024-03-04T08:21:47.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc3e00640 1 --2- 172.21.15.113:0/1463241886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc406b060 0x7fabc4071a80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009d90 tx=0x7fabbc01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50c26a16d4b8c44 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:47.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc2a00640 1 -- 172.21.15.113:0/1463241886 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0b2070 con 0x7fabc406b060 2024-03-04T08:21:47.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc2a00640 1 -- 172.21.15.113:0/1463241886 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc0a59e0 con 0x7fabc406b060 2024-03-04T08:21:47.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc2a00640 1 -- 172.21.15.113:0/1463241886 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0a5db0 con 0x7fabc406b060 2024-03-04T08:21:47.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc8c00640 1 -- 172.21.15.113:0/1463241886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc406b060 msgr2=0x7fabc4071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:47.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc8c00640 1 --2- 172.21.15.113:0/1463241886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc406b060 0x7fabc4071a80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009d90 tx=0x7fabbc01c940 comp rx=0 tx=0).stop 2024-03-04T08:21:47.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc8c00640 1 -- 172.21.15.113:0/1463241886 shutdown_connections 2024-03-04T08:21:47.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc8c00640 1 --2- 172.21.15.113:0/1463241886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc406b060 0x7fabc4071a80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:47.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc8c00640 1 --2- 172.21.15.113:0/1463241886 >> v2:172.21.15.136:3300/0 conn(0x7fabc40c7b10 0x7fabc4152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:47.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc8c00640 1 -- 172.21.15.113:0/1463241886 >> 172.21.15.113:0/1463241886 conn(0x7fabc40cd110 msgr2=0x7fabc40cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:47.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc8c00640 1 -- 172.21.15.113:0/1463241886 shutdown_connections 2024-03-04T08:21:47.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc8c00640 1 -- 172.21.15.113:0/1463241886 wait complete. 2024-03-04T08:21:47.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc8c00640 1 Processor -- start 2024-03-04T08:21:47.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.974+0000 7fabc8c00640 1 -- start start 2024-03-04T08:21:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc8c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc406b060 0x7fabc40aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc40c7b10 0x7fabc4154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc8c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabc4152780 0x7fabc41553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc8c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc406c2b0 con 0x7fabc406b060 2024-03-04T08:21:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc8c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc406c130 con 0x7fabc4152780 2024-03-04T08:21:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc40cde30 con 0x7fabc40c7b10 2024-03-04T08:21:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc406b060 0x7fabc40aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc40c7b10 0x7fabc4154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc406b060 0x7fabc40aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33858/0 (socket says 172.21.15.113:33858) 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabbaa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc40c7b10 0x7fabc4154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:55958/0 (socket says 172.21.15.113:55958) 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc3400640 1 -- 172.21.15.113:0/4219145575 learned_addr learned my addr 172.21.15.113:0/4219145575 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc3e00640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabc4152780 0x7fabc41553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc3400640 1 -- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc40c7b10 msgr2=0x7fabc4154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc3400640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc40c7b10 0x7fabc4154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc3400640 1 -- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabc4152780 msgr2=0x7fabc41553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc3400640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabc4152780 0x7fabc41553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc3400640 1 -- 172.21.15.113:0/4219145575 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40b0f80 con 0x7fabc406b060 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc3400640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc406b060 0x7fabc40aeb40 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c3a0 tx=0x7fabb400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabbbe00640 1 -- 172.21.15.113:0/4219145575 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb4017070 con 0x7fabc406b060 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabbbe00640 1 -- 172.21.15.113:0/4219145575 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb4005cf0 con 0x7fabc406b060 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabbbe00640 1 -- 172.21.15.113:0/4219145575 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40072f0 con 0x7fabc406b060 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc8c00640 1 -- 172.21.15.113:0/4219145575 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc41516a0 con 0x7fabc406b060 2024-03-04T08:21:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc8c00640 1 -- 172.21.15.113:0/4219145575 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc40b1fe0 con 0x7fabc406b060 2024-03-04T08:21:47.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabc8c00640 1 -- 172.21.15.113:0/4219145575 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc4075a80 con 0x7fabc406b060 2024-03-04T08:21:47.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.978+0000 7fabbbe00640 1 -- 172.21.15.113:0/4219145575 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabb401a070 con 0x7fabc406b060 2024-03-04T08:21:47.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.982+0000 7fabbbe00640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faba0066f90 0x7faba0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:47.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.982+0000 7fabbbe00640 1 -- 172.21.15.113:0/4219145575 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fabb4084d00 con 0x7fabc406b060 2024-03-04T08:21:47.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.982+0000 7fabbaa00640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faba0066f90 0x7faba0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:47.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.982+0000 7fabbaa00640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faba0066f90 0x7faba0069450 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fabc4068dc0 tx=0x7fabac009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:47.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:47.986+0000 7fabbbe00640 1 -- 172.21.15.113:0/4219145575 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fabb4054c90 con 0x7fabc406b060 2024-03-04T08:21:48.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.182+0000 7fabc8c00640 1 -- 172.21.15.113:0/4219145575 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabc40cc350 con 0x7fabc406b060 2024-03-04T08:21:48.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.182+0000 7fabbbe00640 1 -- 172.21.15.113:0/4219145575 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) v1 ==== 95+0+5026 (secure 0 0 0) 0x7fabb4058d30 con 0x7fabc406b060 2024-03-04T08:21:48.182 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:48.182 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":173,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:40.994141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:21:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 173 2024-03-04T08:21:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 -- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faba0066f90 msgr2=0x7faba0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faba0066f90 0x7faba0069450 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fabc4068dc0 tx=0x7fabac009190 comp rx=0 tx=0).stop 2024-03-04T08:21:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 -- 172.21.15.113:0/4219145575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc406b060 msgr2=0x7fabc40aeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc406b060 0x7fabc40aeb40 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c3a0 tx=0x7fabb400c860 comp rx=0 tx=0).stop 2024-03-04T08:21:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 -- 172.21.15.113:0/4219145575 shutdown_connections 2024-03-04T08:21:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabc4152780 0x7fabc41553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabc40c7b10 0x7fabc4154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7faba0066f90 0x7faba0069450 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 --2- 172.21.15.113:0/4219145575 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabc406b060 0x7fabc40aeb40 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 -- 172.21.15.113:0/4219145575 >> 172.21.15.113:0/4219145575 conn(0x7fabc40cd110 msgr2=0x7fabc415b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 -- 172.21.15.113:0/4219145575 shutdown_connections 2024-03-04T08:21:48.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:48.186+0000 7fabc8c00640 1 -- 172.21.15.113:0/4219145575 wait complete. 2024-03-04T08:21:48.204 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:50.205 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:50.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5d6e00640 1 Processor -- start 2024-03-04T08:21:50.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5d6e00640 1 -- start start 2024-03-04T08:21:50.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5d6e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 0x7fc5d015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:50.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5d6e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d006c210 con 0x7fc5d006de80 2024-03-04T08:21:50.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5d6e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d006c4f0 con 0x7fc5d00ce6b0 2024-03-04T08:21:50.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5d6e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d015bbf0 con 0x7fc5d00ce2d0 2024-03-04T08:21:50.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5d4e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc5d006de80 0x7fc5d0154b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57262/0 (socket says 172.21.15.113:57262) 2024-03-04T08:21:50.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5d4e00640 1 -- 172.21.15.113:0/3874644602 learned_addr learned my addr 172.21.15.113:0/3874644602 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:50.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5cfe00640 1 --2- 172.21.15.113:0/3874644602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 0x7fc5d015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:50.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5cf400640 1 -- 172.21.15.113:0/3874644602 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358808281 0 0) 0x7fc5d006c210 con 0x7fc5d006de80 2024-03-04T08:21:50.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5cf400640 1 -- 172.21.15.113:0/3874644602 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5b4003530 con 0x7fc5d006de80 2024-03-04T08:21:50.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5cf400640 1 -- 172.21.15.113:0/3874644602 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755126991 0 0) 0x7fc5d006c4f0 con 0x7fc5d00ce6b0 2024-03-04T08:21:50.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5cfe00640 1 -- 172.21.15.113:0/3874644602 >> v1:172.21.15.136:6789/0 conn(0x7fc5d00ce6b0 legacy=0x7fc5d006f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:50.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5cfe00640 1 -- 172.21.15.113:0/3874644602 >> v1:172.21.15.113:6789/0 conn(0x7fc5d006de80 legacy=0x7fc5d0154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:50.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.342+0000 7fc5cfe00640 1 -- 172.21.15.113:0/3874644602 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d006e530 con 0x7fc5d00ce2d0 2024-03-04T08:21:50.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5cfe00640 1 --2- 172.21.15.113:0/3874644602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 0x7fc5d015b6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00097f0 tx=0x7fc5c001eb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31c13f30455d1be server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:50.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5cf400640 1 -- 172.21.15.113:0/3874644602 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00b5070 con 0x7fc5d00ce2d0 2024-03-04T08:21:50.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5cf400640 1 -- 172.21.15.113:0/3874644602 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c00a85d0 con 0x7fc5d00ce2d0 2024-03-04T08:21:50.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5cf400640 1 -- 172.21.15.113:0/3874644602 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8001a00 con 0x7fc5d00ce2d0 2024-03-04T08:21:50.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5d6e00640 1 -- 172.21.15.113:0/3874644602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 msgr2=0x7fc5d015b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:50.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5d6e00640 1 --2- 172.21.15.113:0/3874644602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 0x7fc5d015b6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00097f0 tx=0x7fc5c001eb10 comp rx=0 tx=0).stop 2024-03-04T08:21:50.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5d6e00640 1 -- 172.21.15.113:0/3874644602 shutdown_connections 2024-03-04T08:21:50.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5d6e00640 1 --2- 172.21.15.113:0/3874644602 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 0x7fc5d015b6b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:50.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5d6e00640 1 -- 172.21.15.113:0/3874644602 >> 172.21.15.113:0/3874644602 conn(0x7fc5d00709a0 msgr2=0x7fc5d0070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:50.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5d6e00640 1 -- 172.21.15.113:0/3874644602 shutdown_connections 2024-03-04T08:21:50.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.346+0000 7fc5d6e00640 1 -- 172.21.15.113:0/3874644602 wait complete. 2024-03-04T08:21:50.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 Processor -- start 2024-03-04T08:21:50.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 -- start start 2024-03-04T08:21:50.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5d006de80 0x7fc5d01f1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:50.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 0x7fc5d00c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:50.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5d00ce6b0 0x7fc5d0155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:50.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d006c7d0 con 0x7fc5d006de80 2024-03-04T08:21:50.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d006eb80 con 0x7fc5d00ce6b0 2024-03-04T08:21:50.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d006f720 con 0x7fc5d00ce2d0 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5cfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5d006de80 0x7fc5d01f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5c7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 0x7fc5d00c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5cfe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5d006de80 0x7fc5d01f1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33868/0 (socket says 172.21.15.113:33868) 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5cfe00640 1 -- 172.21.15.113:0/1224077187 learned_addr learned my addr 172.21.15.113:0/1224077187 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d4e00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5d00ce6b0 0x7fc5d0155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5cfe00640 1 -- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 msgr2=0x7fc5d00c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5cfe00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 0x7fc5d00c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5cfe00640 1 -- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5d00ce6b0 msgr2=0x7fc5d0155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5cfe00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5d00ce6b0 0x7fc5d0155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5cfe00640 1 -- 172.21.15.113:0/1224077187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d01f15d0 con 0x7fc5d006de80 2024-03-04T08:21:50.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5cfe00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5d006de80 0x7fc5d01f1090 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00b3040 tx=0x7fc5c001b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:50.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5ccc00640 1 -- 172.21.15.113:0/1224077187 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00b5040 con 0x7fc5d006de80 2024-03-04T08:21:50.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5ccc00640 1 -- 172.21.15.113:0/1224077187 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c00a9890 con 0x7fc5d006de80 2024-03-04T08:21:50.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 -- 172.21.15.113:0/1224077187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d0155a40 con 0x7fc5d006de80 2024-03-04T08:21:50.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5ccc00640 1 -- 172.21.15.113:0/1224077187 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00a9c40 con 0x7fc5d006de80 2024-03-04T08:21:50.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 -- 172.21.15.113:0/1224077187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d0150820 con 0x7fc5d006de80 2024-03-04T08:21:50.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.350+0000 7fc5d6e00640 1 -- 172.21.15.113:0/1224077187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d0075c00 con 0x7fc5d006de80 2024-03-04T08:21:50.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.354+0000 7fc5ccc00640 1 -- 172.21.15.113:0/1224077187 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5c00c2030 con 0x7fc5d006de80 2024-03-04T08:21:50.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.354+0000 7fc5ccc00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc5ac066f60 0x7fc5ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:50.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.354+0000 7fc5ccc00640 1 -- 172.21.15.113:0/1224077187 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fc5c0122230 con 0x7fc5d006de80 2024-03-04T08:21:50.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.354+0000 7fc5c7e00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc5ac066f60 0x7fc5ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:50.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.354+0000 7fc5c7e00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc5ac066f60 0x7fc5ac069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00cc470 tx=0x7fc5b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:50.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.358+0000 7fc5ccc00640 1 -- 172.21.15.113:0/1224077187 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc5c00c9060 con 0x7fc5d006de80 2024-03-04T08:21:50.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.546+0000 7fc5d6e00640 1 -- 172.21.15.113:0/1224077187 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5d0159590 con 0x7fc5d006de80 2024-03-04T08:21:50.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.546+0000 7fc5ccc00640 1 -- 172.21.15.113:0/1224077187 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) v1 ==== 95+0+5026 (secure 0 0 0) 0x7fc5c0018730 con 0x7fc5d006de80 2024-03-04T08:21:50.546 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:50.547 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":173,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:40.994141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:21:50.547 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 173 2024-03-04T08:21:50.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 -- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc5ac066f60 msgr2=0x7fc5ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:50.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc5ac066f60 0x7fc5ac069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00cc470 tx=0x7fc5b8009040 comp rx=0 tx=0).stop 2024-03-04T08:21:50.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 -- 172.21.15.113:0/1224077187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5d006de80 msgr2=0x7fc5d01f1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:50.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5d006de80 0x7fc5d01f1090 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00b3040 tx=0x7fc5c001b8b0 comp rx=0 tx=0).stop 2024-03-04T08:21:50.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 -- 172.21.15.113:0/1224077187 shutdown_connections 2024-03-04T08:21:50.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc5d00ce6b0 0x7fc5d0155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:50.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5d00ce2d0 0x7fc5d00c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:50.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc5ac066f60 0x7fc5ac069420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:50.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 --2- 172.21.15.113:0/1224077187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5d006de80 0x7fc5d01f1090 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:50.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 -- 172.21.15.113:0/1224077187 >> 172.21.15.113:0/1224077187 conn(0x7fc5d00709a0 msgr2=0x7fc5d01548e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:50.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 -- 172.21.15.113:0/1224077187 shutdown_connections 2024-03-04T08:21:50.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:50.550+0000 7fc5d6e00640 1 -- 172.21.15.113:0/1224077187 wait complete. 2024-03-04T08:21:50.568 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:52.568 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:52.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.714+0000 7f2ac9000640 1 Processor -- start 2024-03-04T08:21:52.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac9000640 1 -- start start 2024-03-04T08:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac9000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 0x7f2ac4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac9000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac41591f0 con 0x7f2ac41527f0 2024-03-04T08:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac9000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac41594d0 con 0x7f2ac4152bd0 2024-03-04T08:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac9000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac4071df0 con 0x7f2ac40c8200 2024-03-04T08:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac1e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2ac41527f0 0x7f2ac4072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57274/0 (socket says 172.21.15.113:57274) 2024-03-04T08:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac1e00640 1 -- 172.21.15.113:0/1312646718 learned_addr learned my addr 172.21.15.113:0/1312646718 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:52.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ab9400640 1 --2- 172.21.15.113:0/1312646718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 0x7f2ac4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac1400640 1 -- 172.21.15.113:0/1312646718 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355570240 0 0) 0x7f2ac41591f0 con 0x7f2ac41527f0 2024-03-04T08:21:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac1400640 1 -- 172.21.15.113:0/1312646718 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aa8003530 con 0x7f2ac41527f0 2024-03-04T08:21:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac1400640 1 -- 172.21.15.113:0/1312646718 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372571496 0 0) 0x7f2ac41594d0 con 0x7f2ac4152bd0 2024-03-04T08:21:52.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ac1400640 1 -- 172.21.15.113:0/1312646718 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ac41591f0 con 0x7f2ac4152bd0 2024-03-04T08:21:52.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ab9400640 1 -- 172.21.15.113:0/1312646718 >> v1:172.21.15.136:6789/0 conn(0x7f2ac4152bd0 legacy=0x7f2ac41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:52.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ab9400640 1 -- 172.21.15.113:0/1312646718 >> v1:172.21.15.113:6789/0 conn(0x7f2ac41527f0 legacy=0x7f2ac4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:52.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.718+0000 7f2ab9400640 1 -- 172.21.15.113:0/1312646718 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac40717a0 con 0x7f2ac40c8200 2024-03-04T08:21:52.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ab9400640 1 --2- 172.21.15.113:0/1312646718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 0x7f2ac4070d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0097f0 tx=0x7f2aac01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf1b4aa5b5195734 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:52.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac1400640 1 -- 172.21.15.113:0/1312646718 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0ab5f0 con 0x7f2ac40c8200 2024-03-04T08:21:52.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac1400640 1 -- 172.21.15.113:0/1312646718 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac0b0070 con 0x7f2ac40c8200 2024-03-04T08:21:52.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac1400640 1 -- 172.21.15.113:0/1312646718 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0ab990 con 0x7f2ac40c8200 2024-03-04T08:21:52.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac9000640 1 -- 172.21.15.113:0/1312646718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 msgr2=0x7f2ac4070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:52.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac9000640 1 --2- 172.21.15.113:0/1312646718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 0x7f2ac4070d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0097f0 tx=0x7f2aac01c4e0 comp rx=0 tx=0).stop 2024-03-04T08:21:52.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac9000640 1 -- 172.21.15.113:0/1312646718 shutdown_connections 2024-03-04T08:21:52.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac9000640 1 --2- 172.21.15.113:0/1312646718 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 0x7f2ac4070d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:52.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac9000640 1 -- 172.21.15.113:0/1312646718 >> 172.21.15.113:0/1312646718 conn(0x7f2ac406e820 msgr2=0x7f2ac406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:52.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac9000640 1 -- 172.21.15.113:0/1312646718 shutdown_connections 2024-03-04T08:21:52.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac9000640 1 -- 172.21.15.113:0/1312646718 wait complete. 2024-03-04T08:21:52.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac9000640 1 Processor -- start 2024-03-04T08:21:52.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.722+0000 7f2ac9000640 1 -- start start 2024-03-04T08:21:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac9000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 0x7f2ac4149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac41527f0 0x7f2ac4147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac9000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ac4152bd0 0x7f2ac4147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac9000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac4166380 con 0x7f2ac41527f0 2024-03-04T08:21:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac9000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac41597b0 con 0x7f2ac4152bd0 2024-03-04T08:21:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac9000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac4166500 con 0x7f2ac40c8200 2024-03-04T08:21:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ab9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac41527f0 0x7f2ac4147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ab9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac41527f0 0x7f2ac4147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33880/0 (socket says 172.21.15.113:33880) 2024-03-04T08:21:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ab9400640 1 -- 172.21.15.113:0/4232656735 learned_addr learned my addr 172.21.15.113:0/4232656735 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:52.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac2800640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ac4152bd0 0x7f2ac4147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac1e00640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 0x7f2ac4149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ab9400640 1 -- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 msgr2=0x7f2ac4149140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ab9400640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 0x7f2ac4149140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ab9400640 1 -- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ac4152bd0 msgr2=0x7f2ac4147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ab9400640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ac4152bd0 0x7f2ac4147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ab9400640 1 -- 172.21.15.113:0/4232656735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac40b0b60 con 0x7f2ac41527f0 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac2800640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ac4152bd0 0x7f2ac4147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ab9400640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac41527f0 0x7f2ac4147250 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0ad040 tx=0x7f2aac019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2abaa00640 1 -- 172.21.15.113:0/4232656735 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac003820 con 0x7f2ac41527f0 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2abaa00640 1 -- 172.21.15.113:0/4232656735 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac0039c0 con 0x7f2ac41527f0 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac9000640 1 -- 172.21.15.113:0/4232656735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac40b05a0 con 0x7f2ac41527f0 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2abaa00640 1 -- 172.21.15.113:0/4232656735 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac003d70 con 0x7f2ac41527f0 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac9000640 1 -- 172.21.15.113:0/4232656735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac4145c80 con 0x7f2ac41527f0 2024-03-04T08:21:52.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2ac9000640 1 -- 172.21.15.113:0/4232656735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac4075c00 con 0x7f2ac41527f0 2024-03-04T08:21:52.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.726+0000 7f2abaa00640 1 -- 172.21.15.113:0/4232656735 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2aac0c3060 con 0x7f2ac41527f0 2024-03-04T08:21:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.730+0000 7f2abaa00640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2aa0066f60 0x7f2aa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.730+0000 7f2abaa00640 1 -- 172.21.15.113:0/4232656735 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f2aac11f260 con 0x7f2ac41527f0 2024-03-04T08:21:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.730+0000 7f2ac1e00640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2aa0066f60 0x7f2aa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:52.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.730+0000 7f2ac1e00640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2aa0066f60 0x7f2aa0069420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40012f0 tx=0x7f2ab4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:52.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.734+0000 7f2abaa00640 1 -- 172.21.15.113:0/4232656735 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2aac0160d0 con 0x7f2ac41527f0 2024-03-04T08:21:52.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 -- 172.21.15.113:0/4232656735 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ac40c79d0 con 0x7f2ac41527f0 2024-03-04T08:21:52.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2abaa00640 1 -- 172.21.15.113:0/4232656735 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+5029 (secure 0 0 0) 0x7f2aac0ef9f0 con 0x7f2ac41527f0 2024-03-04T08:21:52.920 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:52.920 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":174,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:50.449362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:resolve","state_seq":722,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:21:52.920 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 174 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 -- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2aa0066f60 msgr2=0x7f2aa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2aa0066f60 0x7f2aa0069420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40012f0 tx=0x7f2ab4009040 comp rx=0 tx=0).stop 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 -- 172.21.15.113:0/4232656735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac41527f0 msgr2=0x7f2ac4147250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac41527f0 0x7f2ac4147250 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0ad040 tx=0x7f2aac019280 comp rx=0 tx=0).stop 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 -- 172.21.15.113:0/4232656735 shutdown_connections 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ac4152bd0 0x7f2ac4147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2aa0066f60 0x7f2aa0069420 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ac41527f0 0x7f2ac4147250 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 --2- 172.21.15.113:0/4232656735 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ac40c8200 0x7f2ac4149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 -- 172.21.15.113:0/4232656735 >> 172.21.15.113:0/4232656735 conn(0x7f2ac406e820 msgr2=0x7f2ac41580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 -- 172.21.15.113:0/4232656735 shutdown_connections 2024-03-04T08:21:52.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:52.922+0000 7f2ac9000640 1 -- 172.21.15.113:0/4232656735 wait complete. 2024-03-04T08:21:52.940 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:54.941 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:55.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3f1a00640 1 Processor -- start 2024-03-04T08:21:55.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3f1a00640 1 -- start start 2024-03-04T08:21:55.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3f1a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3ec152c50 0x7fd3ec06b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:55.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3f1a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd3ec0c7930 0x7fd3ec0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:55.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3f1a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ec15bda0 con 0x7fd3ec0c9230 2024-03-04T08:21:55.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3f1a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec0c7fd0 con 0x7fd3ec152c50 2024-03-04T08:21:55.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3f1a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec15c040 con 0x7fd3ec0c7930 2024-03-04T08:21:55.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3ebe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3ec152c50 0x7fd3ec06b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:55.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3f1000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd3ec0c9230 0x7fd3ec071a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57280/0 (socket says 172.21.15.113:57280) 2024-03-04T08:21:55.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3f1000640 1 -- 172.21.15.113:0/3293317157 learned_addr learned my addr 172.21.15.113:0/3293317157 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:21:55.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3ebe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3ec152c50 0x7fd3ec06b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33890/0 (socket says 172.21.15.113:33890) 2024-03-04T08:21:55.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3eb400640 1 --2- 172.21.15.113:0/3293317157 >> v2:172.21.15.136:3300/0 conn(0x7fd3ec0c7930 0x7fd3ec0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:55.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3ebe00640 1 -- 172.21.15.113:0/3293317157 >> v2:172.21.15.136:3300/0 conn(0x7fd3ec0c7930 msgr2=0x7fd3ec0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:55.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3ebe00640 1 --2- 172.21.15.113:0/3293317157 >> v2:172.21.15.136:3300/0 conn(0x7fd3ec0c7930 0x7fd3ec0c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:55.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3ebe00640 1 -- 172.21.15.113:0/3293317157 >> v1:172.21.15.113:6789/0 conn(0x7fd3ec0c9230 legacy=0x7fd3ec071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:55.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3ebe00640 1 -- 172.21.15.113:0/3293317157 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec15c320 con 0x7fd3ec152c50 2024-03-04T08:21:55.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.090+0000 7fd3eaa00640 1 -- 172.21.15.113:0/3293317157 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165426879 0 0) 0x7fd3ec15bda0 con 0x7fd3ec0c9230 2024-03-04T08:21:55.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3ebe00640 1 --2- 172.21.15.113:0/3293317157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec152c50 0x7fd3ec06b060 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec06cfa0 tx=0x7fd3dc01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8dfef9d6880c58 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:55.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3eaa00640 1 -- 172.21.15.113:0/3293317157 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0b5070 con 0x7fd3ec152c50 2024-03-04T08:21:55.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3eaa00640 1 -- 172.21.15.113:0/3293317157 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc0a8950 con 0x7fd3ec152c50 2024-03-04T08:21:55.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3eaa00640 1 -- 172.21.15.113:0/3293317157 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0a8d20 con 0x7fd3ec152c50 2024-03-04T08:21:55.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3f1a00640 1 -- 172.21.15.113:0/3293317157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec152c50 msgr2=0x7fd3ec06b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:55.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3f1a00640 1 --2- 172.21.15.113:0/3293317157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec152c50 0x7fd3ec06b060 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec06cfa0 tx=0x7fd3dc01ec50 comp rx=0 tx=0).stop 2024-03-04T08:21:55.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3f1a00640 1 -- 172.21.15.113:0/3293317157 shutdown_connections 2024-03-04T08:21:55.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3f1a00640 1 --2- 172.21.15.113:0/3293317157 >> v2:172.21.15.136:3300/0 conn(0x7fd3ec0c7930 0x7fd3ec0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:55.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3f1a00640 1 --2- 172.21.15.113:0/3293317157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec152c50 0x7fd3ec06b060 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:55.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3f1a00640 1 -- 172.21.15.113:0/3293317157 >> 172.21.15.113:0/3293317157 conn(0x7fd3ec0c9a30 msgr2=0x7fd3ec0c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:55.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3f1a00640 1 -- 172.21.15.113:0/3293317157 shutdown_connections 2024-03-04T08:21:55.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3f1a00640 1 -- 172.21.15.113:0/3293317157 wait complete. 2024-03-04T08:21:55.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3f1a00640 1 Processor -- start 2024-03-04T08:21:55.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.094+0000 7fd3f1a00640 1 -- start start 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3f1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec0c7930 0x7fd3ec148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3f1a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3ec0c9230 0x7fd3ec1e1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3f1a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd3ec152c50 0x7fd3ec0c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3f1a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec06b710 con 0x7fd3ec0c7930 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3f1a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec15c8e0 con 0x7fd3ec0c9230 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3f1a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec06c3e0 con 0x7fd3ec152c50 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3ebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec0c7930 0x7fd3ec148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3ebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec0c7930 0x7fd3ec148f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:33900/0 (socket says 172.21.15.113:33900) 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3ebe00640 1 -- 172.21.15.113:0/367941248 learned_addr learned my addr 172.21.15.113:0/367941248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3f1000640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd3ec152c50 0x7fd3ec0c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3eb400640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3ec0c9230 0x7fd3ec1e1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3ebe00640 1 -- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd3ec152c50 msgr2=0x7fd3ec0c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:55.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3ebe00640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd3ec152c50 0x7fd3ec0c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3ebe00640 1 -- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3ec0c9230 msgr2=0x7fd3ec1e1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3ebe00640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3ec0c9230 0x7fd3ec1e1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3ebe00640 1 -- 172.21.15.113:0/367941248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec06c120 con 0x7fd3ec0c7930 2024-03-04T08:21:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3ebe00640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec0c7930 0x7fd3ec148f60 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc01c880 tx=0x7fd3dc01b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3e3e00640 1 -- 172.21.15.113:0/367941248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0bf070 con 0x7fd3ec0c7930 2024-03-04T08:21:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3e3e00640 1 -- 172.21.15.113:0/367941248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc0b0640 con 0x7fd3ec0c7930 2024-03-04T08:21:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3e3e00640 1 -- 172.21.15.113:0/367941248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0b5040 con 0x7fd3ec0c7930 2024-03-04T08:21:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3f1a00640 1 -- 172.21.15.113:0/367941248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ec14d860 con 0x7fd3ec0c7930 2024-03-04T08:21:55.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3f1a00640 1 -- 172.21.15.113:0/367941248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ec0c20f0 con 0x7fd3ec0c7930 2024-03-04T08:21:55.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3f1a00640 1 -- 172.21.15.113:0/367941248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ec075a80 con 0x7fd3ec0c7930 2024-03-04T08:21:55.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.098+0000 7fd3e3e00640 1 -- 172.21.15.113:0/367941248 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3dc0c2070 con 0x7fd3ec0c7930 2024-03-04T08:21:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.102+0000 7fd3e3e00640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd3cc066f60 0x7fd3cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.102+0000 7fd3e3e00640 1 -- 172.21.15.113:0/367941248 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fd3dc122230 con 0x7fd3ec0c7930 2024-03-04T08:21:55.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.102+0000 7fd3eb400640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd3cc066f60 0x7fd3cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:55.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.102+0000 7fd3eb400640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd3cc066f60 0x7fd3cc069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec14e1e0 tx=0x7fd3d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:55.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.106+0000 7fd3e3e00640 1 -- 172.21.15.113:0/367941248 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd3dc0187b0 con 0x7fd3ec0c7930 2024-03-04T08:21:55.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 -- 172.21.15.113:0/367941248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3ec14c760 con 0x7fd3ec0c7930 2024-03-04T08:21:55.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3e3e00640 1 -- 172.21.15.113:0/367941248 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+5029 (secure 0 0 0) 0x7fd3dc0f28a0 con 0x7fd3ec0c7930 2024-03-04T08:21:55.292 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:55.293 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":174,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:50.449362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:resolve","state_seq":722,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:resolve","state_seq":16,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:21:55.293 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 174 2024-03-04T08:21:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 -- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd3cc066f60 msgr2=0x7fd3cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd3cc066f60 0x7fd3cc069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec14e1e0 tx=0x7fd3d8008040 comp rx=0 tx=0).stop 2024-03-04T08:21:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 -- 172.21.15.113:0/367941248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec0c7930 msgr2=0x7fd3ec148f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec0c7930 0x7fd3ec148f60 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc01c880 tx=0x7fd3dc01b9f0 comp rx=0 tx=0).stop 2024-03-04T08:21:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 -- 172.21.15.113:0/367941248 shutdown_connections 2024-03-04T08:21:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd3ec152c50 0x7fd3ec0c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3ec0c9230 0x7fd3ec1e1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd3cc066f60 0x7fd3cc069420 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 --2- 172.21.15.113:0/367941248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3ec0c7930 0x7fd3ec148f60 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 -- 172.21.15.113:0/367941248 >> 172.21.15.113:0/367941248 conn(0x7fd3ec0c9a30 msgr2=0x7fd3ec0c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 -- 172.21.15.113:0/367941248 shutdown_connections 2024-03-04T08:21:55.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:55.294+0000 7fd3f1a00640 1 -- 172.21.15.113:0/367941248 wait complete. 2024-03-04T08:21:55.314 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 174, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 174, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 173, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:resolve", "state_seq": 722 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8388": { "addr": "172.21.15.113:6833/2999156262", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2999156262, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2999156262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8388, "incarnation": 171, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:resolve", "state_seq": 16 } }, "last_failure": 0, "last_failure_osd_epoch": 82, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:21:50.449362+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4247, "mds_1": 8032, "mds_2": 8388 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:21:57.315 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:57.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.462+0000 7ff75d200640 1 Processor -- start 2024-03-04T08:21:57.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.462+0000 7ff75d200640 1 -- start start 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff75d200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff7581527f0 0x7ff758070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff75d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff7580c8200 0x7ff758072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff75d200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7581591f0 con 0x7ff758152bd0 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff75d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff758071df0 con 0x7ff7580c8200 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff75d200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7581597b0 con 0x7ff7581527f0 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff74d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff7580c8200 0x7ff758072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff74d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff7580c8200 0x7ff758072280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50788/0 (socket says 172.21.15.113:50788) 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff74d400640 1 -- 172.21.15.113:0/267153070 learned_addr learned my addr 172.21.15.113:0/267153070 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff755e00640 1 --2- 172.21.15.113:0/267153070 >> v2:172.21.15.136:3300/0 conn(0x7ff7581527f0 0x7ff758070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff74d400640 1 -- 172.21.15.113:0/267153070 >> v2:172.21.15.136:3300/0 conn(0x7ff7581527f0 msgr2=0x7ff758070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:57.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff74d400640 1 --2- 172.21.15.113:0/267153070 >> v2:172.21.15.136:3300/0 conn(0x7ff7581527f0 0x7ff758070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff74d400640 1 -- 172.21.15.113:0/267153070 >> v1:172.21.15.136:6789/0 conn(0x7ff758152bd0 legacy=0x7ff7581587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:57.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff74d400640 1 -- 172.21.15.113:0/267153070 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7581594d0 con 0x7ff7580c8200 2024-03-04T08:21:57.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.466+0000 7ff74d400640 1 --2- 172.21.15.113:0/267153070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7580c8200 0x7ff758072280 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff75806f400 tx=0x7ff74001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ff5b041166240e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:57.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff755400640 1 -- 172.21.15.113:0/267153070 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400b1070 con 0x7ff7580c8200 2024-03-04T08:21:57.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff755400640 1 -- 172.21.15.113:0/267153070 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7400ac630 con 0x7ff7580c8200 2024-03-04T08:21:57.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff755400640 1 -- 172.21.15.113:0/267153070 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400a8050 con 0x7ff7580c8200 2024-03-04T08:21:57.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 -- 172.21.15.113:0/267153070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7580c8200 msgr2=0x7ff758072280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:57.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 --2- 172.21.15.113:0/267153070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7580c8200 0x7ff758072280 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ff75806f400 tx=0x7ff74001c4e0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 -- 172.21.15.113:0/267153070 shutdown_connections 2024-03-04T08:21:57.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 --2- 172.21.15.113:0/267153070 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7580c8200 0x7ff758072280 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 --2- 172.21.15.113:0/267153070 >> v2:172.21.15.136:3300/0 conn(0x7ff7581527f0 0x7ff758070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 -- 172.21.15.113:0/267153070 >> 172.21.15.113:0/267153070 conn(0x7ff75806e820 msgr2=0x7ff75806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:57.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 -- 172.21.15.113:0/267153070 shutdown_connections 2024-03-04T08:21:57.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 -- 172.21.15.113:0/267153070 wait complete. 2024-03-04T08:21:57.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 Processor -- start 2024-03-04T08:21:57.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 -- start start 2024-03-04T08:21:57.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7580c8200 0x7ff7581f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:57.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7581527f0 0x7ff7580c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:57.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff758152bd0 0x7ff758143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:57.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff758165160 con 0x7ff758152bd0 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580727c0 con 0x7ff7580c8200 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff75d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7581652e0 con 0x7ff7581527f0 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff756800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff758152bd0 0x7ff758143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff756800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff758152bd0 0x7ff758143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50800/0 (socket says 172.21.15.113:50800) 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff756800640 1 -- 172.21.15.113:0/1572590188 learned_addr learned my addr 172.21.15.113:0/1572590188 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff755e00640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7580c8200 0x7ff7581f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff74d400640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7581527f0 0x7ff7580c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff756800640 1 -- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7581527f0 msgr2=0x7ff7580c28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff756800640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7581527f0 0x7ff7580c28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff756800640 1 -- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7580c8200 msgr2=0x7ff7581f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.470+0000 7ff756800640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7580c8200 0x7ff7581f4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff756800640 1 -- 172.21.15.113:0/1572590188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7581f72c0 con 0x7ff758152bd0 2024-03-04T08:21:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff756800640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff758152bd0 0x7ff758143d30 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff75000b440 tx=0x7ff75000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff74ea00640 1 -- 172.21.15.113:0/1572590188 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750015020 con 0x7ff758152bd0 2024-03-04T08:21:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff74ea00640 1 -- 172.21.15.113:0/1572590188 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff750011850 con 0x7ff758152bd0 2024-03-04T08:21:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff75d200640 1 -- 172.21.15.113:0/1572590188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7581f7580 con 0x7ff758152bd0 2024-03-04T08:21:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff74ea00640 1 -- 172.21.15.113:0/1572590188 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750011c40 con 0x7ff758152bd0 2024-03-04T08:21:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff75d200640 1 -- 172.21.15.113:0/1572590188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7581f7b40 con 0x7ff758152bd0 2024-03-04T08:21:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff75d200640 1 -- 172.21.15.113:0/1572590188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff758075c00 con 0x7ff758152bd0 2024-03-04T08:21:57.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff74ea00640 1 -- 172.21.15.113:0/1572590188 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff750018020 con 0x7ff758152bd0 2024-03-04T08:21:57.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff74ea00640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff734066f60 0x7ff734069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:57.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff74ea00640 1 -- 172.21.15.113:0/1572590188 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7ff750085550 con 0x7ff758152bd0 2024-03-04T08:21:57.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.474+0000 7ff755e00640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff734066f60 0x7ff734069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:57.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.478+0000 7ff755e00640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff734066f60 0x7ff734069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff748003b90 tx=0x7ff7480098d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:57.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.478+0000 7ff74ea00640 1 -- 172.21.15.113:0/1572590188 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff75001f030 con 0x7ff758152bd0 2024-03-04T08:21:57.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.666+0000 7ff75d200640 1 -- 172.21.15.113:0/1572590188 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7580c79d0 con 0x7ff758152bd0 2024-03-04T08:21:57.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.666+0000 7ff74ea00640 1 -- 172.21.15.113:0/1572590188 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+5031 (secure 0 0 0) 0x7ff750055450 con 0x7ff758152bd0 2024-03-04T08:21:57.666 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:21:57.667 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":175,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:56.412250+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:resolve","state_seq":722,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:reconnect","state_seq":23,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:21:57.667 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 175 2024-03-04T08:21:57.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 -- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff734066f60 msgr2=0x7ff734069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:57.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff734066f60 0x7ff734069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff748003b90 tx=0x7ff7480098d0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 -- 172.21.15.113:0/1572590188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff758152bd0 msgr2=0x7ff758143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:57.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff758152bd0 0x7ff758143d30 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff75000b440 tx=0x7ff75000b900 comp rx=0 tx=0).stop 2024-03-04T08:21:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 -- 172.21.15.113:0/1572590188 shutdown_connections 2024-03-04T08:21:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff734066f60 0x7ff734069420 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff758152bd0 0x7ff758143d30 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7581527f0 0x7ff7580c28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 --2- 172.21.15.113:0/1572590188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7580c8200 0x7ff7581f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 -- 172.21.15.113:0/1572590188 >> 172.21.15.113:0/1572590188 conn(0x7ff75806e820 msgr2=0x7ff7580c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 -- 172.21.15.113:0/1572590188 shutdown_connections 2024-03-04T08:21:57.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:57.670+0000 7ff75d200640 1 -- 172.21.15.113:0/1572590188 wait complete. 2024-03-04T08:21:57.688 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:21:59.689 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:21:59.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.834+0000 7f3e5d200640 1 Processor -- start 2024-03-04T08:21:59.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.834+0000 7f3e5d200640 1 -- start start 2024-03-04T08:21:59.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e5d200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3e580c22e0 0x7f3e5815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:59.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e5d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e580c2d80 0x7f3e5815b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:59.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e5d200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e580c27f0 con 0x7f3e5814e4c0 2024-03-04T08:21:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e5d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e5815ad60 con 0x7f3e580c2d80 2024-03-04T08:21:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e5d200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e580c2a90 con 0x7f3e580c22e0 2024-03-04T08:21:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e56800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e580c2d80 0x7f3e5815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e56800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e580c2d80 0x7f3e5815b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50810/0 (socket says 172.21.15.113:50810) 2024-03-04T08:21:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e56800640 1 -- 172.21.15.113:0/14654945 learned_addr learned my addr 172.21.15.113:0/14654945 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e4d400640 1 --2- 172.21.15.113:0/14654945 >> v2:172.21.15.136:3300/0 conn(0x7f3e580c22e0 0x7f3e5815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e56800640 1 -- 172.21.15.113:0/14654945 >> v2:172.21.15.136:3300/0 conn(0x7f3e580c22e0 msgr2=0x7f3e5815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e56800640 1 --2- 172.21.15.113:0/14654945 >> v2:172.21.15.136:3300/0 conn(0x7f3e580c22e0 0x7f3e5815b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e56800640 1 -- 172.21.15.113:0/14654945 >> v1:172.21.15.136:6789/0 conn(0x7f3e5814e4c0 legacy=0x7f3e58072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:21:59.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.838+0000 7f3e56800640 1 -- 172.21.15.113:0/14654945 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e58070c60 con 0x7f3e580c2d80 2024-03-04T08:21:59.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e56800640 1 --2- 172.21.15.113:0/14654945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e580c2d80 0x7f3e5815b730 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f3e580cd230 tx=0x7f3e5001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9045a0d55644f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:59.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e55400640 1 -- 172.21.15.113:0/14654945 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500ab5f0 con 0x7f3e580c2d80 2024-03-04T08:21:59.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e55400640 1 -- 172.21.15.113:0/14654945 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e500b0070 con 0x7f3e580c2d80 2024-03-04T08:21:59.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e55400640 1 -- 172.21.15.113:0/14654945 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500ab9b0 con 0x7f3e580c2d80 2024-03-04T08:21:59.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e5d200640 1 -- 172.21.15.113:0/14654945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e580c2d80 msgr2=0x7f3e5815b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:59.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e5d200640 1 --2- 172.21.15.113:0/14654945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e580c2d80 0x7f3e5815b730 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f3e580cd230 tx=0x7f3e5001c4e0 comp rx=0 tx=0).stop 2024-03-04T08:21:59.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e5d200640 1 -- 172.21.15.113:0/14654945 shutdown_connections 2024-03-04T08:21:59.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e5d200640 1 --2- 172.21.15.113:0/14654945 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e580c2d80 0x7f3e5815b730 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:59.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e5d200640 1 --2- 172.21.15.113:0/14654945 >> v2:172.21.15.136:3300/0 conn(0x7f3e580c22e0 0x7f3e5815b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:59.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e5d200640 1 -- 172.21.15.113:0/14654945 >> 172.21.15.113:0/14654945 conn(0x7f3e5806e820 msgr2=0x7f3e5806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:21:59.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e5d200640 1 -- 172.21.15.113:0/14654945 shutdown_connections 2024-03-04T08:21:59.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e5d200640 1 -- 172.21.15.113:0/14654945 wait complete. 2024-03-04T08:21:59.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e5d200640 1 Processor -- start 2024-03-04T08:21:59.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.842+0000 7f3e5d200640 1 -- start start 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e5d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e580c22e0 0x7f3e580cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e5d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e580c2d80 0x7f3e58153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e5d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5814e4c0 0x7f3e581542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e5d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580aede0 con 0x7f3e5814e4c0 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e5d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5814e8a0 con 0x7f3e580c22e0 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e5d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e58073390 con 0x7f3e580c2d80 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e56800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5814e4c0 0x7f3e581542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e4d400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e580c2d80 0x7f3e58153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e56800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5814e4c0 0x7f3e581542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50814/0 (socket says 172.21.15.113:50814) 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e56800640 1 -- 172.21.15.113:0/3994321700 learned_addr learned my addr 172.21.15.113:0/3994321700 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e55e00640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e580c22e0 0x7f3e580cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:59.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e56800640 1 -- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e580c2d80 msgr2=0x7f3e58153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:59.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e56800640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e580c2d80 0x7f3e58153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:59.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e56800640 1 -- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e580c22e0 msgr2=0x7f3e580cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:21:59.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e56800640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e580c22e0 0x7f3e580cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:21:59.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e56800640 1 -- 172.21.15.113:0/3994321700 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e580cbdf0 con 0x7f3e5814e4c0 2024-03-04T08:21:59.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e56800640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5814e4c0 0x7f3e581542a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3e500ad040 tx=0x7f3e5001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:59.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e4ea00640 1 -- 172.21.15.113:0/3994321700 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500be3f0 con 0x7f3e5814e4c0 2024-03-04T08:21:59.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e4ea00640 1 -- 172.21.15.113:0/3994321700 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e500be590 con 0x7f3e5814e4c0 2024-03-04T08:21:59.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e4ea00640 1 -- 172.21.15.113:0/3994321700 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500be980 con 0x7f3e5814e4c0 2024-03-04T08:21:59.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e5d200640 1 -- 172.21.15.113:0/3994321700 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5814d4b0 con 0x7f3e5814e4c0 2024-03-04T08:21:59.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e5d200640 1 -- 172.21.15.113:0/3994321700 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e58153280 con 0x7f3e5814e4c0 2024-03-04T08:21:59.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e5d200640 1 -- 172.21.15.113:0/3994321700 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e58155190 con 0x7f3e5814e4c0 2024-03-04T08:21:59.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.846+0000 7f3e4ea00640 1 -- 172.21.15.113:0/3994321700 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e500b43f0 con 0x7f3e5814e4c0 2024-03-04T08:21:59.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.850+0000 7f3e4ea00640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e34066f60 0x7f3e34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:21:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.850+0000 7f3e4ea00640 1 -- 172.21.15.113:0/3994321700 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f3e5011fbd0 con 0x7f3e5814e4c0 2024-03-04T08:21:59.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.850+0000 7f3e55e00640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e34066f60 0x7f3e34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:21:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.850+0000 7f3e55e00640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e34066f60 0x7f3e34069420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3e48007fd0 tx=0x7f3e48007e40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:21:59.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:21:59.854+0000 7f3e4ea00640 1 -- 172.21.15.113:0/3994321700 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3e500c4060 con 0x7f3e5814e4c0 2024-03-04T08:22:00.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.038+0000 7f3e5d200640 1 -- 172.21.15.113:0/3994321700 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e580c1260 con 0x7f3e5814e4c0 2024-03-04T08:22:00.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.038+0000 7f3e4ea00640 1 -- 172.21.15.113:0/3994321700 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+5031 (secure 0 0 0) 0x7f3e500160d0 con 0x7f3e5814e4c0 2024-03-04T08:22:00.038 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:00.038 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":175,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":175,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:21:56.412250+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:resolve","state_seq":722,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:reconnect","state_seq":23,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:22:00.039 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 175 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 -- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e34066f60 msgr2=0x7f3e34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e34066f60 0x7f3e34069420 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3e48007fd0 tx=0x7f3e48007e40 comp rx=0 tx=0).stop 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 -- 172.21.15.113:0/3994321700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5814e4c0 msgr2=0x7f3e581542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5814e4c0 0x7f3e581542a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3e500ad040 tx=0x7f3e5001ad20 comp rx=0 tx=0).stop 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 -- 172.21.15.113:0/3994321700 shutdown_connections 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3e34066f60 0x7f3e34069420 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e5814e4c0 0x7f3e581542a0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e580c2d80 0x7f3e58153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 --2- 172.21.15.113:0/3994321700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e580c22e0 0x7f3e580cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 -- 172.21.15.113:0/3994321700 >> 172.21.15.113:0/3994321700 conn(0x7f3e5806e820 msgr2=0x7f3e5806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 -- 172.21.15.113:0/3994321700 shutdown_connections 2024-03-04T08:22:00.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:00.042+0000 7f3e5d200640 1 -- 172.21.15.113:0/3994321700 wait complete. 2024-03-04T08:22:00.060 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:02.060 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:02.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676f000640 1 Processor -- start 2024-03-04T08:22:02.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676f000640 1 -- start start 2024-03-04T08:22:02.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676f000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6768151b40 0x7f67680c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:02.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676f000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676806c290 con 0x7f6768151760 2024-03-04T08:22:02.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676f000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676806ef10 con 0x7f67680c18d0 2024-03-04T08:22:02.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676f000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f676815aef0 con 0x7f6768151b40 2024-03-04T08:22:02.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6768151b40 0x7f67680c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:02.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6768151b40 0x7f67680c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50824/0 (socket says 172.21.15.113:50824) 2024-03-04T08:22:02.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676d200640 1 -- 172.21.15.113:0/315508565 learned_addr learned my addr 172.21.15.113:0/315508565 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:02.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676dc00640 1 --1- 172.21.15.113:0/315508565 >> v1:172.21.15.113:6789/0 conn(0x7f6768151760 0x7f676815b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49708/0 (socket says 172.21.15.113:49708) 2024-03-04T08:22:02.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676d200640 1 -- 172.21.15.113:0/315508565 >> v1:172.21.15.136:6789/0 conn(0x7f67680c18d0 legacy=0x7f6768071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:02.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676d200640 1 -- 172.21.15.113:0/315508565 >> v1:172.21.15.113:6789/0 conn(0x7f6768151760 legacy=0x7f676815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:02.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f676d200640 1 -- 172.21.15.113:0/315508565 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676815ba90 con 0x7f6768151b40 2024-03-04T08:22:02.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.210+0000 7f6767e00640 1 -- 172.21.15.113:0/315508565 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353941725 0 0) 0x7f676806c290 con 0x7f6768151760 2024-03-04T08:22:02.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f676d200640 1 --2- 172.21.15.113:0/315508565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 0x7f67680c83d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f675400bb10 tx=0x7f675401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed34bf2e8f19e1a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:02.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f6767e00640 1 -- 172.21.15.113:0/315508565 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540b4070 con 0x7f6768151b40 2024-03-04T08:22:02.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f6767e00640 1 -- 172.21.15.113:0/315508565 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67540a7a80 con 0x7f6768151b40 2024-03-04T08:22:02.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f6767e00640 1 -- 172.21.15.113:0/315508565 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540a7e50 con 0x7f6768151b40 2024-03-04T08:22:02.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f676f000640 1 -- 172.21.15.113:0/315508565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 msgr2=0x7f67680c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:02.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f676f000640 1 --2- 172.21.15.113:0/315508565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 0x7f67680c83d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f675400bb10 tx=0x7f675401eba0 comp rx=0 tx=0).stop 2024-03-04T08:22:02.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f676f000640 1 -- 172.21.15.113:0/315508565 shutdown_connections 2024-03-04T08:22:02.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f676f000640 1 --2- 172.21.15.113:0/315508565 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 0x7f67680c83d0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:02.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f676f000640 1 -- 172.21.15.113:0/315508565 >> 172.21.15.113:0/315508565 conn(0x7f676806e2a0 msgr2=0x7f676806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:02.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f676f000640 1 -- 172.21.15.113:0/315508565 shutdown_connections 2024-03-04T08:22:02.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f676f000640 1 -- 172.21.15.113:0/315508565 wait complete. 2024-03-04T08:22:02.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f676f000640 1 Processor -- start 2024-03-04T08:22:02.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.214+0000 7f676f000640 1 -- start start 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676f000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67680c18d0 0x7f67681444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676f000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6768151760 0x7f6768155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676f000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 0x7f6768151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676f000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67680c1e20 con 0x7f6768151b40 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676f000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676815ab80 con 0x7f67680c18d0 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676f000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67680c70c0 con 0x7f6768151760 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 0x7f6768151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 0x7f6768151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50836/0 (socket says 172.21.15.113:50836) 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676e600640 1 -- 172.21.15.113:0/2960275399 learned_addr learned my addr 172.21.15.113:0/2960275399 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676dc00640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67680c18d0 0x7f67681444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676d200640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6768151760 0x7f6768155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676e600640 1 -- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6768151760 msgr2=0x7f6768155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:02.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676e600640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6768151760 0x7f6768155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:02.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676e600640 1 -- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67680c18d0 msgr2=0x7f67681444c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:02.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676e600640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67680c18d0 0x7f67681444c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:02.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676e600640 1 -- 172.21.15.113:0/2960275399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6768154ea0 con 0x7f6768151b40 2024-03-04T08:22:02.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676e600640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 0x7f6768151f80 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f676000c7a0 tx=0x7f676000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:02.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f6765600640 1 -- 172.21.15.113:0/2960275399 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760003b80 con 0x7f6768151b40 2024-03-04T08:22:02.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f6765600640 1 -- 172.21.15.113:0/2960275399 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6760003d20 con 0x7f6768151b40 2024-03-04T08:22:02.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676f000640 1 -- 172.21.15.113:0/2960275399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6768156640 con 0x7f6768151b40 2024-03-04T08:22:02.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f6765600640 1 -- 172.21.15.113:0/2960275399 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676000aa90 con 0x7f6768151b40 2024-03-04T08:22:02.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676f000640 1 -- 172.21.15.113:0/2960275399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6768156c00 con 0x7f6768151b40 2024-03-04T08:22:02.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.218+0000 7f676f000640 1 -- 172.21.15.113:0/2960275399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6768075a80 con 0x7f6768151b40 2024-03-04T08:22:02.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.222+0000 7f6765600640 1 -- 172.21.15.113:0/2960275399 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f676000ac30 con 0x7f6768151b40 2024-03-04T08:22:02.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.222+0000 7f6765600640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6748066f60 0x7f6748069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:02.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.222+0000 7f6765600640 1 -- 172.21.15.113:0/2960275399 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f6760085980 con 0x7f6768151b40 2024-03-04T08:22:02.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.222+0000 7f676dc00640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6748066f60 0x7f6748069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:02.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.222+0000 7f676dc00640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6748066f60 0x7f6748069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f675c007fd0 tx=0x7f675c007ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:02.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.226+0000 7f6765600640 1 -- 172.21.15.113:0/2960275399 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6760055f60 con 0x7f6768151b40 2024-03-04T08:22:02.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 -- 172.21.15.113:0/2960275399 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f676814f2a0 con 0x7f6768151b40 2024-03-04T08:22:02.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f6765600640 1 -- 172.21.15.113:0/2960275399 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+5030 (secure 0 0 0) 0x7f6760055860 con 0x7f6768151b40 2024-03-04T08:22:02.412 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:02.412 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":176,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:01.441568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:reconnect","state_seq":725,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:22:02.412 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 176 2024-03-04T08:22:02.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 -- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6748066f60 msgr2=0x7f6748069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6748066f60 0x7f6748069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f675c007fd0 tx=0x7f675c007ba0 comp rx=0 tx=0).stop 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 -- 172.21.15.113:0/2960275399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 msgr2=0x7f6768151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 0x7f6768151f80 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f676000c7a0 tx=0x7f676000cc60 comp rx=0 tx=0).stop 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 -- 172.21.15.113:0/2960275399 shutdown_connections 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6748066f60 0x7f6748069420 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6768151b40 0x7f6768151f80 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6768151760 0x7f6768155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 --2- 172.21.15.113:0/2960275399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67680c18d0 0x7f67681444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 -- 172.21.15.113:0/2960275399 >> 172.21.15.113:0/2960275399 conn(0x7f676806e2a0 msgr2=0x7f676806d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 -- 172.21.15.113:0/2960275399 shutdown_connections 2024-03-04T08:22:02.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:02.414+0000 7f676f000640 1 -- 172.21.15.113:0/2960275399 wait complete. 2024-03-04T08:22:02.432 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:04.435 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.582+0000 7fce26600640 1 Processor -- start 2024-03-04T08:22:04.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.582+0000 7fce26600640 1 -- start start 2024-03-04T08:22:04.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce26600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce2014e4c0 0x7fce20072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:04.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce26600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fce2015b900 0x7fce200731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:04.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce26600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce200c27f0 con 0x7fce200c22e0 2024-03-04T08:22:04.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce26600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2015ad60 con 0x7fce2014e4c0 2024-03-04T08:22:04.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce26600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce200c2a90 con 0x7fce2015b900 2024-03-04T08:22:04.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce1f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce2014e4c0 0x7fce20072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:04.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce1f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce2014e4c0 0x7fce20072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50852/0 (socket says 172.21.15.113:50852) 2024-03-04T08:22:04.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce1f400640 1 -- 172.21.15.113:0/992084053 learned_addr learned my addr 172.21.15.113:0/992084053 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:04.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce1fe00640 1 --2- 172.21.15.113:0/992084053 >> v2:172.21.15.136:3300/0 conn(0x7fce2015b900 0x7fce200731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:04.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce1f400640 1 -- 172.21.15.113:0/992084053 >> v2:172.21.15.136:3300/0 conn(0x7fce2015b900 msgr2=0x7fce200731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:04.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce1f400640 1 --2- 172.21.15.113:0/992084053 >> v2:172.21.15.136:3300/0 conn(0x7fce2015b900 0x7fce200731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:04.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce1f400640 1 -- 172.21.15.113:0/992084053 >> v1:172.21.15.136:6789/0 conn(0x7fce200c22e0 legacy=0x7fce2015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:04.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.586+0000 7fce1f400640 1 -- 172.21.15.113:0/992084053 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20070c60 con 0x7fce2014e4c0 2024-03-04T08:22:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce1f400640 1 --2- 172.21.15.113:0/992084053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 0x7fce20072c80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fce2006f690 tx=0x7fce1001f180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a72b9419a2adf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce1ea00640 1 -- 172.21.15.113:0/992084053 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce100b4070 con 0x7fce2014e4c0 2024-03-04T08:22:04.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce1ea00640 1 -- 172.21.15.113:0/992084053 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce100af400 con 0x7fce2014e4c0 2024-03-04T08:22:04.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce1ea00640 1 -- 172.21.15.113:0/992084053 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce100af7d0 con 0x7fce2014e4c0 2024-03-04T08:22:04.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce26600640 1 -- 172.21.15.113:0/992084053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 msgr2=0x7fce20072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:04.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce26600640 1 --2- 172.21.15.113:0/992084053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 0x7fce20072c80 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fce2006f690 tx=0x7fce1001f180 comp rx=0 tx=0).stop 2024-03-04T08:22:04.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce26600640 1 -- 172.21.15.113:0/992084053 shutdown_connections 2024-03-04T08:22:04.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce26600640 1 --2- 172.21.15.113:0/992084053 >> v2:172.21.15.136:3300/0 conn(0x7fce2015b900 0x7fce200731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:04.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce26600640 1 --2- 172.21.15.113:0/992084053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 0x7fce20072c80 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:04.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce26600640 1 -- 172.21.15.113:0/992084053 >> 172.21.15.113:0/992084053 conn(0x7fce2006e820 msgr2=0x7fce2006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:04.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce26600640 1 -- 172.21.15.113:0/992084053 shutdown_connections 2024-03-04T08:22:04.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce26600640 1 -- 172.21.15.113:0/992084053 wait complete. 2024-03-04T08:22:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce26600640 1 Processor -- start 2024-03-04T08:22:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.590+0000 7fce26600640 1 -- start start 2024-03-04T08:22:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce26600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce200c22e0 0x7fce20159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce26600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 0x7fce2014d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce26600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce2015b900 0x7fce201e3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:04.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce26600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce20071500 con 0x7fce2014e4c0 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce26600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce20071220 con 0x7fce2015b900 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce26600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2014e8a0 con 0x7fce200c22e0 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce16a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 0x7fce2014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce1f400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce200c22e0 0x7fce20159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce16a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 0x7fce2014d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50858/0 (socket says 172.21.15.113:50858) 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce16a00640 1 -- 172.21.15.113:0/525711427 learned_addr learned my addr 172.21.15.113:0/525711427 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce1fe00640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce2015b900 0x7fce201e3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce16a00640 1 -- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce200c22e0 msgr2=0x7fce20159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce16a00640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce200c22e0 0x7fce20159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce16a00640 1 -- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce2015b900 msgr2=0x7fce201e3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce16a00640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce2015b900 0x7fce201e3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce16a00640 1 -- 172.21.15.113:0/525711427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2015a970 con 0x7fce2014e4c0 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce1fe00640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce2015b900 0x7fce201e3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:22:04.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce16a00640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 0x7fce2014d350 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fce0800b410 tx=0x7fce0800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:04.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce17e00640 1 -- 172.21.15.113:0/525711427 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce08015030 con 0x7fce2014e4c0 2024-03-04T08:22:04.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce17e00640 1 -- 172.21.15.113:0/525711427 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce08004020 con 0x7fce2014e4c0 2024-03-04T08:22:04.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce17e00640 1 -- 172.21.15.113:0/525711427 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce08004450 con 0x7fce2014e4c0 2024-03-04T08:22:04.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce26600640 1 -- 172.21.15.113:0/525711427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2014aa90 con 0x7fce2014e4c0 2024-03-04T08:22:04.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce26600640 1 -- 172.21.15.113:0/525711427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce200ca4a0 con 0x7fce2014e4c0 2024-03-04T08:22:04.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce26600640 1 -- 172.21.15.113:0/525711427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce20075c00 con 0x7fce2014e4c0 2024-03-04T08:22:04.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.594+0000 7fce17e00640 1 -- 172.21.15.113:0/525711427 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce08018070 con 0x7fce2014e4c0 2024-03-04T08:22:04.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.598+0000 7fce17e00640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdfc066f60 0x7fcdfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:04.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.598+0000 7fce17e00640 1 -- 172.21.15.113:0/525711427 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fce080854f0 con 0x7fce2014e4c0 2024-03-04T08:22:04.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.598+0000 7fce1f400640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdfc066f60 0x7fcdfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:04.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.598+0000 7fce1f400640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdfc066f60 0x7fcdfc069420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fce100b1000 tx=0x7fce100aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:04.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.602+0000 7fce17e00640 1 -- 172.21.15.113:0/525711427 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fce08055ad0 con 0x7fce2014e4c0 2024-03-04T08:22:04.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.790+0000 7fce26600640 1 -- 172.21.15.113:0/525711427 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce20155190 con 0x7fce2014e4c0 2024-03-04T08:22:04.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.790+0000 7fce17e00640 1 -- 172.21.15.113:0/525711427 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 176 v176) v1 ==== 95+0+5030 (secure 0 0 0) 0x7fce080553d0 con 0x7fce2014e4c0 2024-03-04T08:22:04.789 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:04.789 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":176,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":176,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:01.441568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:reconnect","state_seq":725,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:22:04.789 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 176 2024-03-04T08:22:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 -- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdfc066f60 msgr2=0x7fcdfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdfc066f60 0x7fcdfc069420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fce100b1000 tx=0x7fce100aa000 comp rx=0 tx=0).stop 2024-03-04T08:22:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 -- 172.21.15.113:0/525711427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 msgr2=0x7fce2014d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 0x7fce2014d350 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fce0800b410 tx=0x7fce0800b8d0 comp rx=0 tx=0).stop 2024-03-04T08:22:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 -- 172.21.15.113:0/525711427 shutdown_connections 2024-03-04T08:22:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce2015b900 0x7fce201e3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcdfc066f60 0x7fcdfc069420 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:04.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce2014e4c0 0x7fce2014d350 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:04.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 --2- 172.21.15.113:0/525711427 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce200c22e0 0x7fce20159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:04.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 -- 172.21.15.113:0/525711427 >> 172.21.15.113:0/525711427 conn(0x7fce2006e820 msgr2=0x7fce20072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:04.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 -- 172.21.15.113:0/525711427 shutdown_connections 2024-03-04T08:22:04.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:04.794+0000 7fce26600640 1 -- 172.21.15.113:0/525711427 wait complete. 2024-03-04T08:22:04.809 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:06.810 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:06.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00ff400640 1 Processor -- start 2024-03-04T08:22:06.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00ff400640 1 -- start start 2024-03-04T08:22:06.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00ff400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f00f814e4c0 0x7f00f8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:06.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00ff400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f815b900 0x7f00f80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:06.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00ff400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f80c27f0 con 0x7f00f80c22e0 2024-03-04T08:22:06.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00ff400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f815ad60 con 0x7f00f814e4c0 2024-03-04T08:22:06.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00ff400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f80c2a90 con 0x7f00f815b900 2024-03-04T08:22:06.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00fd400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f815b900 0x7f00f80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:06.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00fca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f00f814e4c0 0x7f00f8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:06.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00fd400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f815b900 0x7f00f80731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:41530/0 (socket says 172.21.15.113:41530) 2024-03-04T08:22:06.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00fd400640 1 -- 172.21.15.113:0/1341603328 learned_addr learned my addr 172.21.15.113:0/1341603328 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:06.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00fd400640 1 -- 172.21.15.113:0/1341603328 >> v2:172.21.15.136:3300/0 conn(0x7f00f814e4c0 msgr2=0x7f00f8072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:06.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00fd400640 1 --2- 172.21.15.113:0/1341603328 >> v2:172.21.15.136:3300/0 conn(0x7f00f814e4c0 0x7f00f8072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:06.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00fd400640 1 -- 172.21.15.113:0/1341603328 >> v1:172.21.15.136:6789/0 conn(0x7f00f80c22e0 legacy=0x7f00f815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:06.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00f7e00640 1 -- 172.21.15.113:0/1341603328 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824836291 0 0) 0x7f00f80c27f0 con 0x7f00f80c22e0 2024-03-04T08:22:06.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.954+0000 7f00fd400640 1 -- 172.21.15.113:0/1341603328 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f8070c60 con 0x7f00f815b900 2024-03-04T08:22:06.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00fd400640 1 --2- 172.21.15.113:0/1341603328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f815b900 0x7f00f80731c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009e50 tx=0x7f00f001f210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94be62356b4c2815 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:06.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00f7e00640 1 -- 172.21.15.113:0/1341603328 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f00b4070 con 0x7f00f815b900 2024-03-04T08:22:06.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00f7e00640 1 -- 172.21.15.113:0/1341603328 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f00af400 con 0x7f00f815b900 2024-03-04T08:22:06.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00f7e00640 1 -- 172.21.15.113:0/1341603328 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f00af7d0 con 0x7f00f815b900 2024-03-04T08:22:06.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00ff400640 1 -- 172.21.15.113:0/1341603328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f815b900 msgr2=0x7f00f80731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:06.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00ff400640 1 --2- 172.21.15.113:0/1341603328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f815b900 0x7f00f80731c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009e50 tx=0x7f00f001f210 comp rx=0 tx=0).stop 2024-03-04T08:22:06.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00ff400640 1 -- 172.21.15.113:0/1341603328 shutdown_connections 2024-03-04T08:22:06.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00ff400640 1 --2- 172.21.15.113:0/1341603328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f815b900 0x7f00f80731c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:06.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00ff400640 1 --2- 172.21.15.113:0/1341603328 >> v2:172.21.15.136:3300/0 conn(0x7f00f814e4c0 0x7f00f8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:06.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00ff400640 1 -- 172.21.15.113:0/1341603328 >> 172.21.15.113:0/1341603328 conn(0x7f00f806e820 msgr2=0x7f00f806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:06.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00ff400640 1 -- 172.21.15.113:0/1341603328 shutdown_connections 2024-03-04T08:22:06.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.958+0000 7f00ff400640 1 -- 172.21.15.113:0/1341603328 wait complete. 2024-03-04T08:22:06.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00ff400640 1 Processor -- start 2024-03-04T08:22:06.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00ff400640 1 -- start start 2024-03-04T08:22:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00ff400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f80c22e0 0x7f00f80b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00ff400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f814e4c0 0x7f00f80c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00ff400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00f815b900 0x7f00f806b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00ff400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f8071500 con 0x7f00f815b900 2024-03-04T08:22:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00ff400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f8071220 con 0x7f00f814e4c0 2024-03-04T08:22:06.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00ff400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f814e8a0 con 0x7f00f80c22e0 2024-03-04T08:22:06.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00fd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00f815b900 0x7f00f806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:06.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00fd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00f815b900 0x7f00f806b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50868/0 (socket says 172.21.15.113:50868) 2024-03-04T08:22:06.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00fd400640 1 -- 172.21.15.113:0/3006816846 learned_addr learned my addr 172.21.15.113:0/3006816846 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:06.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00fca00640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f80c22e0 0x7f00f80b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:06.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00efe00640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f814e4c0 0x7f00f80c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:06.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00fd400640 1 -- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f80c22e0 msgr2=0x7f00f80b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00fd400640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f80c22e0 0x7f00f80b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00fd400640 1 -- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f814e4c0 msgr2=0x7f00f80c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00fd400640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f814e4c0 0x7f00f80c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00fd400640 1 -- 172.21.15.113:0/3006816846 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f814cfa0 con 0x7f00f815b900 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00fd400640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00f815b900 0x7f00f806b060 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b2040 tx=0x7f00f001bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00f5600640 1 -- 172.21.15.113:0/3006816846 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f00b4040 con 0x7f00f815b900 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00f5600640 1 -- 172.21.15.113:0/3006816846 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f00090d0 con 0x7f00f815b900 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00ff400640 1 -- 172.21.15.113:0/3006816846 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f80b1b10 con 0x7f00f815b900 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00f5600640 1 -- 172.21.15.113:0/3006816846 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0009510 con 0x7f00f815b900 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.962+0000 7f00ff400640 1 -- 172.21.15.113:0/3006816846 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f814bc40 con 0x7f00f815b900 2024-03-04T08:22:06.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.966+0000 7f00ff400640 1 -- 172.21.15.113:0/3006816846 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f8075c00 con 0x7f00f815b900 2024-03-04T08:22:06.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.966+0000 7f00f5600640 1 -- 172.21.15.113:0/3006816846 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00f00c4060 con 0x7f00f815b900 2024-03-04T08:22:06.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.966+0000 7f00f5600640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f00d4066f60 0x7f00d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:06.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.966+0000 7f00f5600640 1 -- 172.21.15.113:0/3006816846 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f00f0122c20 con 0x7f00f815b900 2024-03-04T08:22:06.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.966+0000 7f00fca00640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f00d4066f60 0x7f00d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:06.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.966+0000 7f00fca00640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f00d4066f60 0x7f00d4069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009ac0 tx=0x7f00e8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:06.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:06.970+0000 7f00f5600640 1 -- 172.21.15.113:0/3006816846 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f00f0018d70 con 0x7f00f815b900 2024-03-04T08:22:07.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 -- 172.21.15.113:0/3006816846 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00f80c80d0 con 0x7f00f815b900 2024-03-04T08:22:07.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00f5600640 1 -- 172.21.15.113:0/3006816846 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5027 (secure 0 0 0) 0x7f00f00f2bc0 con 0x7f00f815b900 2024-03-04T08:22:07.157 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:07.158 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:06.468978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:rejoin","state_seq":727,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:22:07.159 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 177 2024-03-04T08:22:07.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 -- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f00d4066f60 msgr2=0x7f00d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:07.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f00d4066f60 0x7f00d4069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009ac0 tx=0x7f00e8009190 comp rx=0 tx=0).stop 2024-03-04T08:22:07.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 -- 172.21.15.113:0/3006816846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00f815b900 msgr2=0x7f00f806b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:07.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00f815b900 0x7f00f806b060 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f00f00b2040 tx=0x7f00f001bfb0 comp rx=0 tx=0).stop 2024-03-04T08:22:07.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 -- 172.21.15.113:0/3006816846 shutdown_connections 2024-03-04T08:22:07.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f00d4066f60 0x7f00d4069420 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:07.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00f815b900 0x7f00f806b060 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:07.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00f814e4c0 0x7f00f80c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 --2- 172.21.15.113:0/3006816846 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f00f80c22e0 0x7f00f80b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 -- 172.21.15.113:0/3006816846 >> 172.21.15.113:0/3006816846 conn(0x7f00f806e820 msgr2=0x7f00f80c8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 -- 172.21.15.113:0/3006816846 shutdown_connections 2024-03-04T08:22:07.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:07.158+0000 7f00ff400640 1 -- 172.21.15.113:0/3006816846 wait complete. 2024-03-04T08:22:07.182 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 177, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 177, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 173, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:rejoin", "state_seq": 727 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8388": { "addr": "172.21.15.113:6833/2999156262", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2999156262, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2999156262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8388, "incarnation": 171, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:rejoin", "state_seq": 25 } }, "last_failure": 0, "last_failure_osd_epoch": 82, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:22:06.468978+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4247, "mds_1": 8032, "mds_2": 8388 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:22:09.183 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab8600640 1 Processor -- start 2024-03-04T08:22:09.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab8600640 1 -- start start 2024-03-04T08:22:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab8600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffab014e4c0 0x7ffab0072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab8600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ffab00c22e0 0x7ffab00731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab8600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab00c27f0 con 0x7ffab00c2d80 2024-03-04T08:22:09.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab8600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab015ad60 con 0x7ffab014e4c0 2024-03-04T08:22:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab8600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab00c2a90 con 0x7ffab00c22e0 2024-03-04T08:22:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab5c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffab014e4c0 0x7ffab0072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab5c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffab014e4c0 0x7ffab0072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45036/0 (socket says 172.21.15.113:45036) 2024-03-04T08:22:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab5c00640 1 -- 172.21.15.113:0/604357087 learned_addr learned my addr 172.21.15.113:0/604357087 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab5200640 1 --2- 172.21.15.113:0/604357087 >> v2:172.21.15.136:3300/0 conn(0x7ffab00c22e0 0x7ffab00731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab5c00640 1 -- 172.21.15.113:0/604357087 >> v2:172.21.15.136:3300/0 conn(0x7ffab00c22e0 msgr2=0x7ffab00731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab5c00640 1 --2- 172.21.15.113:0/604357087 >> v2:172.21.15.136:3300/0 conn(0x7ffab00c22e0 0x7ffab00731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab5c00640 1 -- 172.21.15.113:0/604357087 >> v1:172.21.15.136:6789/0 conn(0x7ffab00c2d80 legacy=0x7ffab015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.325+0000 7ffab5c00640 1 -- 172.21.15.113:0/604357087 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab0070c60 con 0x7ffab014e4c0 2024-03-04T08:22:09.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffab5c00640 1 --2- 172.21.15.113:0/604357087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab014e4c0 0x7ffab0072c80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ffab00cd230 tx=0x7ffaa401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738436bf0c3d4ef2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:09.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffaafe00640 1 -- 172.21.15.113:0/604357087 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa40b5070 con 0x7ffab014e4c0 2024-03-04T08:22:09.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffaafe00640 1 -- 172.21.15.113:0/604357087 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa40a89d0 con 0x7ffab014e4c0 2024-03-04T08:22:09.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffaafe00640 1 -- 172.21.15.113:0/604357087 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa40a8da0 con 0x7ffab014e4c0 2024-03-04T08:22:09.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffab8600640 1 -- 172.21.15.113:0/604357087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab014e4c0 msgr2=0x7ffab0072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:09.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffab8600640 1 --2- 172.21.15.113:0/604357087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab014e4c0 0x7ffab0072c80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7ffab00cd230 tx=0x7ffaa401ecd0 comp rx=0 tx=0).stop 2024-03-04T08:22:09.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffab8600640 1 -- 172.21.15.113:0/604357087 shutdown_connections 2024-03-04T08:22:09.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffab8600640 1 --2- 172.21.15.113:0/604357087 >> v2:172.21.15.136:3300/0 conn(0x7ffab00c22e0 0x7ffab00731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:09.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffab8600640 1 --2- 172.21.15.113:0/604357087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab014e4c0 0x7ffab0072c80 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:09.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffab8600640 1 -- 172.21.15.113:0/604357087 >> 172.21.15.113:0/604357087 conn(0x7ffab006e820 msgr2=0x7ffab006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:09.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffab8600640 1 -- 172.21.15.113:0/604357087 shutdown_connections 2024-03-04T08:22:09.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.329+0000 7ffab8600640 1 -- 172.21.15.113:0/604357087 wait complete. 2024-03-04T08:22:09.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab8600640 1 Processor -- start 2024-03-04T08:22:09.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab8600640 1 -- start start 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab8600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab00c22e0 0x7ffab01e1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab8600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffab00c2d80 0x7ffab00cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab8600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab014e4c0 0x7ffab01f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab8600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0071500 con 0x7ffab00c22e0 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab8600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0071220 con 0x7ffab014e4c0 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab8600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab014e8a0 con 0x7ffab00c2d80 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab00c22e0 0x7ffab01e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab00c22e0 0x7ffab01e1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45050/0 (socket says 172.21.15.113:45050) 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5c00640 1 -- 172.21.15.113:0/695320061 learned_addr learned my addr 172.21.15.113:0/695320061 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5200640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffab00c2d80 0x7ffab00cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:09.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab6600640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab014e4c0 0x7ffab01f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5c00640 1 -- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffab00c2d80 msgr2=0x7ffab00cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5c00640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffab00c2d80 0x7ffab00cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5c00640 1 -- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab014e4c0 msgr2=0x7ffab01f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5c00640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab014e4c0 0x7ffab01f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5c00640 1 -- 172.21.15.113:0/695320061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab0154410 con 0x7ffab00c22e0 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5200640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffab00c2d80 0x7ffab00cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab5c00640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab00c22e0 0x7ffab01e1750 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40b3040 tx=0x7ffaa401ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffaad600640 1 -- 172.21.15.113:0/695320061 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa40b5070 con 0x7ffab00c22e0 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffaad600640 1 -- 172.21.15.113:0/695320061 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa40b0a50 con 0x7ffab00c22e0 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffaad600640 1 -- 172.21.15.113:0/695320061 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa40b0e40 con 0x7ffab00c22e0 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab8600640 1 -- 172.21.15.113:0/695320061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab01546f0 con 0x7ffab00c22e0 2024-03-04T08:22:09.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.333+0000 7ffab8600640 1 -- 172.21.15.113:0/695320061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab0154cb0 con 0x7ffab00c22e0 2024-03-04T08:22:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.337+0000 7ffab8600640 1 -- 172.21.15.113:0/695320061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab0075c00 con 0x7ffab00c22e0 2024-03-04T08:22:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.337+0000 7ffaad600640 1 -- 172.21.15.113:0/695320061 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffaa40c2060 con 0x7ffab00c22e0 2024-03-04T08:22:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.337+0000 7ffaad600640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa90066f60 0x7ffa90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.337+0000 7ffaad600640 1 -- 172.21.15.113:0/695320061 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7ffaa4122b40 con 0x7ffab00c22e0 2024-03-04T08:22:09.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.337+0000 7ffab5200640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa90066f60 0x7ffa90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:09.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.341+0000 7ffab5200640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa90066f60 0x7ffa90069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffab014d340 tx=0x7ffa9c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:09.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.341+0000 7ffaad600640 1 -- 172.21.15.113:0/695320061 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ffaa4018830 con 0x7ffab00c22e0 2024-03-04T08:22:09.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.529+0000 7ffab8600640 1 -- 172.21.15.113:0/695320061 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffab0155190 con 0x7ffab00c22e0 2024-03-04T08:22:09.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffaad600640 1 -- 172.21.15.113:0/695320061 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5027 (secure 0 0 0) 0x7ffaa40f3120 con 0x7ffab00c22e0 2024-03-04T08:22:09.531 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:09.532 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:06.468978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:rejoin","state_seq":727,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:22:09.532 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 177 2024-03-04T08:22:09.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 -- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa90066f60 msgr2=0x7ffa90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa90066f60 0x7ffa90069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffab014d340 tx=0x7ffa9c008040 comp rx=0 tx=0).stop 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 -- 172.21.15.113:0/695320061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab00c22e0 msgr2=0x7ffab01e1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab00c22e0 0x7ffab01e1750 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40b3040 tx=0x7ffaa401ba70 comp rx=0 tx=0).stop 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 -- 172.21.15.113:0/695320061 shutdown_connections 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab014e4c0 0x7ffab01f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffab00c2d80 0x7ffab00cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffa90066f60 0x7ffa90069420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 --2- 172.21.15.113:0/695320061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffab00c22e0 0x7ffab01e1750 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 -- 172.21.15.113:0/695320061 >> 172.21.15.113:0/695320061 conn(0x7ffab006e820 msgr2=0x7ffab0072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 -- 172.21.15.113:0/695320061 shutdown_connections 2024-03-04T08:22:09.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:09.533+0000 7ffab8600640 1 -- 172.21.15.113:0/695320061 wait complete. 2024-03-04T08:22:09.553 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:11.554 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:11.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.701+0000 7f2379200640 1 Processor -- start 2024-03-04T08:22:11.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.701+0000 7f2379200640 1 -- start start 2024-03-04T08:22:11.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2379200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23740c22e0 0x7f237415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:11.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2379200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f23740c2d80 0x7f237415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:11.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2379200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740c27f0 con 0x7f237414e4c0 2024-03-04T08:22:11.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2379200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237415ad60 con 0x7f23740c2d80 2024-03-04T08:22:11.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2379200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c2a90 con 0x7f23740c22e0 2024-03-04T08:22:11.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2372800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f23740c2d80 0x7f237415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:11.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2369400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23740c22e0 0x7f237415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:11.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2372800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f23740c2d80 0x7f237415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47852/0 (socket says 172.21.15.113:47852) 2024-03-04T08:22:11.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2372800640 1 -- 172.21.15.113:0/2102138458 learned_addr learned my addr 172.21.15.113:0/2102138458 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:11.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2369400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23740c22e0 0x7f237415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:54062/0 (socket says 172.21.15.113:54062) 2024-03-04T08:22:11.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2371e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f237414e4c0 0x7f2374072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:40184/0 (socket says 172.21.15.113:40184) 2024-03-04T08:22:11.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2372800640 1 -- 172.21.15.113:0/2102138458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23740c22e0 msgr2=0x7f237415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:11.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2372800640 1 --2- 172.21.15.113:0/2102138458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23740c22e0 0x7f237415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:11.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2372800640 1 -- 172.21.15.113:0/2102138458 >> v1:172.21.15.136:6789/0 conn(0x7f237414e4c0 legacy=0x7f2374072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:11.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2372800640 1 -- 172.21.15.113:0/2102138458 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2374070c60 con 0x7f23740c2d80 2024-03-04T08:22:11.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2369400640 1 --2- 172.21.15.113:0/2102138458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23740c22e0 0x7f237415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:22:11.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.705+0000 7f2371400640 1 -- 172.21.15.113:0/2102138458 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132508610 0 0) 0x7f23740c27f0 con 0x7f237414e4c0 2024-03-04T08:22:11.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2372800640 1 --2- 172.21.15.113:0/2102138458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23740c2d80 0x7f237415b730 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f236c009e50 tx=0x7f236c01f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0fc41d6fe9f7401 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2371400640 1 -- 172.21.15.113:0/2102138458 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0b4070 con 0x7f23740c2d80 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2371400640 1 -- 172.21.15.113:0/2102138458 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c0af400 con 0x7f23740c2d80 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2371400640 1 -- 172.21.15.113:0/2102138458 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0af7d0 con 0x7f23740c2d80 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2379200640 1 -- 172.21.15.113:0/2102138458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23740c2d80 msgr2=0x7f237415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2379200640 1 --2- 172.21.15.113:0/2102138458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23740c2d80 0x7f237415b730 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f236c009e50 tx=0x7f236c01f210 comp rx=0 tx=0).stop 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2379200640 1 -- 172.21.15.113:0/2102138458 shutdown_connections 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2379200640 1 --2- 172.21.15.113:0/2102138458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23740c2d80 0x7f237415b730 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2379200640 1 --2- 172.21.15.113:0/2102138458 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23740c22e0 0x7f237415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2379200640 1 -- 172.21.15.113:0/2102138458 >> 172.21.15.113:0/2102138458 conn(0x7f237406e820 msgr2=0x7f237406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2379200640 1 -- 172.21.15.113:0/2102138458 shutdown_connections 2024-03-04T08:22:11.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2379200640 1 -- 172.21.15.113:0/2102138458 wait complete. 2024-03-04T08:22:11.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2379200640 1 Processor -- start 2024-03-04T08:22:11.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.709+0000 7f2379200640 1 -- start start 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2379200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23740c22e0 0x7f237413c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2379200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23740c2d80 0x7f23741f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2379200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237414e4c0 0x7f2374142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2379200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2374071500 con 0x7f23740c2d80 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2379200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2374071220 con 0x7f23740c22e0 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2379200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237414e8a0 con 0x7f237414e4c0 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2369400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23740c2d80 0x7f23741f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2369400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23740c2d80 0x7f23741f3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45066/0 (socket says 172.21.15.113:45066) 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2369400640 1 -- 172.21.15.113:0/3363867926 learned_addr learned my addr 172.21.15.113:0/3363867926 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2371e00640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23740c22e0 0x7f237413c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2372800640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237414e4c0 0x7f2374142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2369400640 1 -- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237414e4c0 msgr2=0x7f2374142830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:11.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2369400640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237414e4c0 0x7f2374142830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:11.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2369400640 1 -- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23740c22e0 msgr2=0x7f237413c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:11.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2369400640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23740c22e0 0x7f237413c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:11.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2369400640 1 -- 172.21.15.113:0/3363867926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740cbe20 con 0x7f23740c2d80 2024-03-04T08:22:11.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2372800640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237414e4c0 0x7f2374142830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:22:11.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2369400640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23740c2d80 0x7f23741f3ed0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f235c00ed70 tx=0x7f235c00d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:11.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f236aa00640 1 -- 172.21.15.113:0/3363867926 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c010850 con 0x7f23740c2d80 2024-03-04T08:22:11.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f236aa00640 1 -- 172.21.15.113:0/3363867926 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c00be50 con 0x7f23740c2d80 2024-03-04T08:22:11.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f236aa00640 1 -- 172.21.15.113:0/3363867926 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c010bf0 con 0x7f23740c2d80 2024-03-04T08:22:11.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2379200640 1 -- 172.21.15.113:0/3363867926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2374146380 con 0x7f23740c2d80 2024-03-04T08:22:11.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2379200640 1 -- 172.21.15.113:0/3363867926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23740cba70 con 0x7f23740c2d80 2024-03-04T08:22:11.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.713+0000 7f2379200640 1 -- 172.21.15.113:0/3363867926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2374075c00 con 0x7f23740c2d80 2024-03-04T08:22:11.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.717+0000 7f236aa00640 1 -- 172.21.15.113:0/3363867926 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f235c00f3d0 con 0x7f23740c2d80 2024-03-04T08:22:11.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.717+0000 7f236aa00640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 0x7f2350069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:11.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.717+0000 7f236aa00640 1 -- 172.21.15.113:0/3363867926 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f235c086210 con 0x7f23740c2d80 2024-03-04T08:22:11.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.717+0000 7f2371e00640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 0x7f2350069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:11.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.717+0000 7f2371e00640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 0x7f2350069450 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2364007fd0 tx=0x7f2364007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:11.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.721+0000 7f236aa00640 1 -- 172.21.15.113:0/3363867926 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f235c014030 con 0x7f23740c2d80 2024-03-04T08:22:11.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.905+0000 7f2379200640 1 -- 172.21.15.113:0/3363867926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23740c85d0 con 0x7f23740c2d80 2024-03-04T08:22:11.906 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.905+0000 7f236aa00640 1 -- 172.21.15.113:0/3363867926 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+5027 (secure 0 0 0) 0x7f235c056890 con 0x7f23740c2d80 2024-03-04T08:22:11.906 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:11.906 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:06.468978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:rejoin","state_seq":727,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:rejoin","state_seq":25,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:22:11.906 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 177 2024-03-04T08:22:11.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 -- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 msgr2=0x7f2350069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:11.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 0x7f2350069450 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2364007fd0 tx=0x7f2364007d60 comp rx=0 tx=0).stop 2024-03-04T08:22:11.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 -- 172.21.15.113:0/3363867926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23740c2d80 msgr2=0x7f23741f3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:11.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23740c2d80 0x7f23741f3ed0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f235c00ed70 tx=0x7f235c00d720 comp rx=0 tx=0).stop 2024-03-04T08:22:11.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 -- 172.21.15.113:0/3363867926 shutdown_connections 2024-03-04T08:22:11.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f237414e4c0 0x7f2374142830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:11.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2350066f90 0x7f2350069450 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:11.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23740c2d80 0x7f23741f3ed0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:11.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 --2- 172.21.15.113:0/3363867926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23740c22e0 0x7f237413c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:11.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 -- 172.21.15.113:0/3363867926 >> 172.21.15.113:0/3363867926 conn(0x7f237406e820 msgr2=0x7f23740c72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:11.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 -- 172.21.15.113:0/3363867926 shutdown_connections 2024-03-04T08:22:11.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:11.909+0000 7f2379200640 1 -- 172.21.15.113:0/3363867926 wait complete. 2024-03-04T08:22:11.928 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:13.929 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:14.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a2c00640 1 Processor -- start 2024-03-04T08:22:14.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a2c00640 1 -- start start 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a2c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f269c14e4c0 0x7f269c072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a2c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f269c0c22e0 0x7f269c0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a2c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f269c0c2d80 0x7f269c15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a2c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269c15ad60 con 0x7f269c0c2d80 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a2c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269c0c27f0 con 0x7f269c14e4c0 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a2c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c0c2a90 con 0x7f269c0c22e0 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a0c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f269c0c2d80 0x7f269c15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f2693400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f269c0c22e0 0x7f269c0731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a0c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f269c0c2d80 0x7f269c15b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45072/0 (socket says 172.21.15.113:45072) 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a0c00640 1 -- 172.21.15.113:0/3323540491 learned_addr learned my addr 172.21.15.113:0/3323540491 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f269be00640 1 --2- 172.21.15.113:0/3323540491 >> v2:172.21.15.136:3300/0 conn(0x7f269c14e4c0 0x7f269c072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a0c00640 1 -- 172.21.15.113:0/3323540491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f269c0c22e0 msgr2=0x7f269c0731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a0c00640 1 --2- 172.21.15.113:0/3323540491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f269c0c22e0 0x7f269c0731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a0c00640 1 -- 172.21.15.113:0/3323540491 >> v2:172.21.15.136:3300/0 conn(0x7f269c14e4c0 msgr2=0x7f269c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:14.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a0c00640 1 --2- 172.21.15.113:0/3323540491 >> v2:172.21.15.136:3300/0 conn(0x7f269c14e4c0 0x7f269c072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f26a0c00640 1 -- 172.21.15.113:0/3323540491 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c070c60 con 0x7f269c0c2d80 2024-03-04T08:22:14.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.077+0000 7f269be00640 1 --2- 172.21.15.113:0/3323540491 >> v2:172.21.15.136:3300/0 conn(0x7f269c14e4c0 0x7f269c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:22:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f26a0c00640 1 --2- 172.21.15.113:0/3323540491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c2d80 0x7f269c15b1f0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f269c06f690 tx=0x7f269401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bdac487ffe491b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f269b400640 1 -- 172.21.15.113:0/3323540491 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940b5070 con 0x7f269c0c2d80 2024-03-04T08:22:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f269b400640 1 -- 172.21.15.113:0/3323540491 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26940a89d0 con 0x7f269c0c2d80 2024-03-04T08:22:14.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f269b400640 1 -- 172.21.15.113:0/3323540491 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940a8da0 con 0x7f269c0c2d80 2024-03-04T08:22:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f26a2c00640 1 -- 172.21.15.113:0/3323540491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c2d80 msgr2=0x7f269c15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f26a2c00640 1 --2- 172.21.15.113:0/3323540491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c2d80 0x7f269c15b1f0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f269c06f690 tx=0x7f269401ed10 comp rx=0 tx=0).stop 2024-03-04T08:22:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f26a2c00640 1 -- 172.21.15.113:0/3323540491 shutdown_connections 2024-03-04T08:22:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f26a2c00640 1 --2- 172.21.15.113:0/3323540491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c2d80 0x7f269c15b1f0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f26a2c00640 1 --2- 172.21.15.113:0/3323540491 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f269c0c22e0 0x7f269c0731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f26a2c00640 1 --2- 172.21.15.113:0/3323540491 >> v2:172.21.15.136:3300/0 conn(0x7f269c14e4c0 0x7f269c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f26a2c00640 1 -- 172.21.15.113:0/3323540491 >> 172.21.15.113:0/3323540491 conn(0x7f269c06e820 msgr2=0x7f269c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f26a2c00640 1 -- 172.21.15.113:0/3323540491 shutdown_connections 2024-03-04T08:22:14.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.081+0000 7f26a2c00640 1 -- 172.21.15.113:0/3323540491 wait complete. 2024-03-04T08:22:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a2c00640 1 Processor -- start 2024-03-04T08:22:14.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a2c00640 1 -- start start 2024-03-04T08:22:14.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a2c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c22e0 0x7f269c1496f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:14.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a2c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f269c0c2d80 0x7f269c152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a2c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f269c14e4c0 0x7f269c1ef6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a2c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c071500 con 0x7f269c0c22e0 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a2c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c071220 con 0x7f269c0c2d80 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a2c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c14e8a0 con 0x7f269c14e4c0 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f269be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c22e0 0x7f269c1496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f269be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c22e0 0x7f269c1496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45078/0 (socket says 172.21.15.113:45078) 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f269be00640 1 -- 172.21.15.113:0/3921403724 learned_addr learned my addr 172.21.15.113:0/3921403724 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a0c00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f269c14e4c0 0x7f269c1ef6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f2693400640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f269c0c2d80 0x7f269c152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f269be00640 1 -- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f269c14e4c0 msgr2=0x7f269c1ef6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f269be00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f269c14e4c0 0x7f269c1ef6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f269be00640 1 -- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f269c0c2d80 msgr2=0x7f269c152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:14.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f269be00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f269c0c2d80 0x7f269c152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f269be00640 1 -- 172.21.15.113:0/3921403724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c1efbe0 con 0x7f269c0c22e0 2024-03-04T08:22:14.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f2693400640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f269c0c2d80 0x7f269c152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:22:14.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f269be00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c22e0 0x7f269c1496f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f268c009fd0 tx=0x7f268c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:14.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f2698c00640 1 -- 172.21.15.113:0/3921403724 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c01a070 con 0x7f269c0c22e0 2024-03-04T08:22:14.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f2698c00640 1 -- 172.21.15.113:0/3921403724 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c003bc0 con 0x7f269c0c22e0 2024-03-04T08:22:14.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f2698c00640 1 -- 172.21.15.113:0/3921403724 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c004030 con 0x7f269c0c22e0 2024-03-04T08:22:14.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a2c00640 1 -- 172.21.15.113:0/3921403724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c0c3cc0 con 0x7f269c0c22e0 2024-03-04T08:22:14.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.085+0000 7f26a2c00640 1 -- 172.21.15.113:0/3921403724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269c1548e0 con 0x7f269c0c22e0 2024-03-04T08:22:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.089+0000 7f26a2c00640 1 -- 172.21.15.113:0/3921403724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269c075c00 con 0x7f269c0c22e0 2024-03-04T08:22:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.089+0000 7f2698c00640 1 -- 172.21.15.113:0/3921403724 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f268c01d070 con 0x7f269c0c22e0 2024-03-04T08:22:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.089+0000 7f2698c00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2678066f60 0x7f2678069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.089+0000 7f2698c00640 1 -- 172.21.15.113:0/3921403724 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f268c085bb0 con 0x7f269c0c22e0 2024-03-04T08:22:14.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.089+0000 7f2693400640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2678066f60 0x7f2678069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:14.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.089+0000 7f2693400640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2678066f60 0x7f2678069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f269c1e5000 tx=0x7f2684008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:14.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.093+0000 7f2698c00640 1 -- 172.21.15.113:0/3921403724 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f268c010070 con 0x7f269c0c22e0 2024-03-04T08:22:14.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.281+0000 7f26a2c00640 1 -- 172.21.15.113:0/3921403724 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f269c0c8b90 con 0x7f269c0c22e0 2024-03-04T08:22:14.283 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f2698c00640 1 -- 172.21.15.113:0/3921403724 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f268c055b40 con 0x7f269c0c22e0 2024-03-04T08:22:14.283 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:14.283 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:14.284 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:14.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 -- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2678066f60 msgr2=0x7f2678069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:14.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2678066f60 0x7f2678069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f269c1e5000 tx=0x7f2684008040 comp rx=0 tx=0).stop 2024-03-04T08:22:14.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 -- 172.21.15.113:0/3921403724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c22e0 msgr2=0x7f269c1496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c22e0 0x7f269c1496f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f268c009fd0 tx=0x7f268c00d560 comp rx=0 tx=0).stop 2024-03-04T08:22:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 -- 172.21.15.113:0/3921403724 shutdown_connections 2024-03-04T08:22:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f269c14e4c0 0x7f269c1ef6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f269c0c2d80 0x7f269c152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2678066f60 0x7f2678069420 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 --2- 172.21.15.113:0/3921403724 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f269c0c22e0 0x7f269c1496f0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 -- 172.21.15.113:0/3921403724 >> 172.21.15.113:0/3921403724 conn(0x7f269c06e820 msgr2=0x7f269c0c7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 -- 172.21.15.113:0/3921403724 shutdown_connections 2024-03-04T08:22:14.286 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:14.285+0000 7f26a2c00640 1 -- 172.21.15.113:0/3921403724 wait complete. 2024-03-04T08:22:14.305 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:14.707 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:22:14.711 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:22:16.306 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91e0400640 1 Processor -- start 2024-03-04T08:22:16.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91e0400640 1 -- start start 2024-03-04T08:22:16.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91e0400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 0x7f91d815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:16.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91e0400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d806c210 con 0x7f91d80ce2d0 2024-03-04T08:22:16.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91e0400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d806c4f0 con 0x7f91d80ce6b0 2024-03-04T08:22:16.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91e0400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d815bbf0 con 0x7f91d806de80 2024-03-04T08:22:16.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91dda00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f91d80ce2d0 0x7f91d806f010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56818/0 (socket says 172.21.15.113:56818) 2024-03-04T08:22:16.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91dda00640 1 -- 172.21.15.113:0/3298272776 learned_addr learned my addr 172.21.15.113:0/3298272776 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:22:16.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91de400640 1 --2- 172.21.15.113:0/3298272776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 0x7f91d815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:16.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91d7e00640 1 -- 172.21.15.113:0/3298272776 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128446607 0 0) 0x7f91d806c210 con 0x7f91d80ce2d0 2024-03-04T08:22:16.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91d7e00640 1 -- 172.21.15.113:0/3298272776 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c0003530 con 0x7f91d80ce2d0 2024-03-04T08:22:16.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91d7e00640 1 -- 172.21.15.113:0/3298272776 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046658914 0 0) 0x7f91d806c4f0 con 0x7f91d80ce6b0 2024-03-04T08:22:16.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91de400640 1 -- 172.21.15.113:0/3298272776 >> v1:172.21.15.136:6789/0 conn(0x7f91d80ce6b0 legacy=0x7f91d8154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:16.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91de400640 1 -- 172.21.15.113:0/3298272776 >> v1:172.21.15.113:6789/0 conn(0x7f91d80ce2d0 legacy=0x7f91d806f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:16.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.453+0000 7f91de400640 1 -- 172.21.15.113:0/3298272776 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d806e530 con 0x7f91d806de80 2024-03-04T08:22:16.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91de400640 1 --2- 172.21.15.113:0/3298272776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 0x7f91d815b6b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009dc0 tx=0x7f91d001efc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=565a090a88a85bb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:16.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91d7e00640 1 -- 172.21.15.113:0/3298272776 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00b5070 con 0x7f91d806de80 2024-03-04T08:22:16.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91d7e00640 1 -- 172.21.15.113:0/3298272776 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d00a8b90 con 0x7f91d806de80 2024-03-04T08:22:16.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91d7e00640 1 -- 172.21.15.113:0/3298272776 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc001680 con 0x7f91d806de80 2024-03-04T08:22:16.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91e0400640 1 -- 172.21.15.113:0/3298272776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 msgr2=0x7f91d815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:16.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91e0400640 1 --2- 172.21.15.113:0/3298272776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 0x7f91d815b6b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009dc0 tx=0x7f91d001efc0 comp rx=0 tx=0).stop 2024-03-04T08:22:16.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91e0400640 1 -- 172.21.15.113:0/3298272776 shutdown_connections 2024-03-04T08:22:16.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91e0400640 1 --2- 172.21.15.113:0/3298272776 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 0x7f91d815b6b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:16.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91e0400640 1 -- 172.21.15.113:0/3298272776 >> 172.21.15.113:0/3298272776 conn(0x7f91d80709a0 msgr2=0x7f91d8070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:16.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91e0400640 1 -- 172.21.15.113:0/3298272776 shutdown_connections 2024-03-04T08:22:16.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.457+0000 7f91e0400640 1 -- 172.21.15.113:0/3298272776 wait complete. 2024-03-04T08:22:16.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91e0400640 1 Processor -- start 2024-03-04T08:22:16.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91e0400640 1 -- start start 2024-03-04T08:22:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91e0400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 0x7f91d81484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91e0400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d80ce2d0 0x7f91d8155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91e0400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91d80ce6b0 0x7f91d8156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91e0400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d806c7d0 con 0x7f91d80ce6b0 2024-03-04T08:22:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91e0400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d806eb80 con 0x7f91d80ce2d0 2024-03-04T08:22:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91e0400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d806f720 con 0x7f91d806de80 2024-03-04T08:22:16.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91dda00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 0x7f91d81484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91de400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91d80ce6b0 0x7f91d8156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91dda00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 0x7f91d81484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:54126/0 (socket says 172.21.15.113:54126) 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91dda00640 1 -- 172.21.15.113:0/2807511464 learned_addr learned my addr 172.21.15.113:0/2807511464 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91dd000640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d80ce2d0 0x7f91d8155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91de400640 1 -- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 msgr2=0x7f91d81484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91de400640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 0x7f91d81484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91de400640 1 -- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d80ce2d0 msgr2=0x7f91d8155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91de400640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d80ce2d0 0x7f91d8155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91de400640 1 -- 172.21.15.113:0/2807511464 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d80c6220 con 0x7f91d80ce6b0 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91dd000640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d80ce2d0 0x7f91d8155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91de400640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91d80ce6b0 0x7f91d8156090 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f91d00af000 tx=0x7f91d001bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91d5600640 1 -- 172.21.15.113:0/2807511464 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00b5070 con 0x7f91d80ce6b0 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91d5600640 1 -- 172.21.15.113:0/2807511464 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d00c0070 con 0x7f91d80ce6b0 2024-03-04T08:22:16.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91d5600640 1 -- 172.21.15.113:0/2807511464 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0004650 con 0x7f91d80ce6b0 2024-03-04T08:22:16.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91e0400640 1 -- 172.21.15.113:0/2807511464 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d80c56a0 con 0x7f91d80ce6b0 2024-03-04T08:22:16.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.461+0000 7f91e0400640 1 -- 172.21.15.113:0/2807511464 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d80c5c60 con 0x7f91d80ce6b0 2024-03-04T08:22:16.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.465+0000 7f91e0400640 1 -- 172.21.15.113:0/2807511464 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d8075c00 con 0x7f91d80ce6b0 2024-03-04T08:22:16.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.465+0000 7f91d5600640 1 -- 172.21.15.113:0/2807511464 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91d00c5460 con 0x7f91d80ce6b0 2024-03-04T08:22:16.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.465+0000 7f91d5600640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f91b8066f90 0x7f91b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:16.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.465+0000 7f91d5600640 1 -- 172.21.15.113:0/2807511464 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f91d0123b70 con 0x7f91d80ce6b0 2024-03-04T08:22:16.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.465+0000 7f91dda00640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f91b8066f90 0x7f91b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:16.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.465+0000 7f91dda00640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f91b8066f90 0x7f91b8069450 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f91cc001da0 tx=0x7f91cc001d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:16.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.469+0000 7f91d5600640 1 -- 172.21.15.113:0/2807511464 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f91d0018be0 con 0x7f91d80ce6b0 2024-03-04T08:22:16.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.657+0000 7f91e0400640 1 -- 172.21.15.113:0/2807511464 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91d80cde20 con 0x7f91d80ce6b0 2024-03-04T08:22:16.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.657+0000 7f91d5600640 1 -- 172.21.15.113:0/2807511464 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f91d00f41e0 con 0x7f91d80ce6b0 2024-03-04T08:22:16.658 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:16.658 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:16.659 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:16.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 -- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f91b8066f90 msgr2=0x7f91b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f91b8066f90 0x7f91b8069450 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f91cc001da0 tx=0x7f91cc001d30 comp rx=0 tx=0).stop 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 -- 172.21.15.113:0/2807511464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91d80ce6b0 msgr2=0x7f91d8156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91d80ce6b0 0x7f91d8156090 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f91d00af000 tx=0x7f91d001bd60 comp rx=0 tx=0).stop 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 -- 172.21.15.113:0/2807511464 shutdown_connections 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f91b8066f90 0x7f91b8069450 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f91d80ce6b0 0x7f91d8156090 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d80ce2d0 0x7f91d8155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 --2- 172.21.15.113:0/2807511464 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f91d806de80 0x7f91d81484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 -- 172.21.15.113:0/2807511464 >> 172.21.15.113:0/2807511464 conn(0x7f91d80709a0 msgr2=0x7f91d81598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 -- 172.21.15.113:0/2807511464 shutdown_connections 2024-03-04T08:22:16.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:16.661+0000 7f91e0400640 1 -- 172.21.15.113:0/2807511464 wait complete. 2024-03-04T08:22:16.679 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:18.680 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:18.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff358800640 1 Processor -- start 2024-03-04T08:22:18.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff358800640 1 -- start start 2024-03-04T08:22:18.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff358800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff350155d80 0x7ff3500c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:18.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff358800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff350156160 con 0x7ff3500c7050 2024-03-04T08:22:18.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff358800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35015ef10 con 0x7ff3500c1860 2024-03-04T08:22:18.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff358800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff350162500 con 0x7ff350155d80 2024-03-04T08:22:18.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff356800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff3500c7050 0x7ff350169890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46836/0 (socket says 172.21.15.113:46836) 2024-03-04T08:22:18.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff356800640 1 -- 172.21.15.113:0/4014527717 learned_addr learned my addr 172.21.15.113:0/4014527717 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:22:18.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff355e00640 1 --2- 172.21.15.113:0/4014527717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff350155d80 0x7ff3500c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:18.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff354a00640 1 -- 172.21.15.113:0/4014527717 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224499806 0 0) 0x7ff350156160 con 0x7ff3500c7050 2024-03-04T08:22:18.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff354a00640 1 -- 172.21.15.113:0/4014527717 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff338003530 con 0x7ff3500c7050 2024-03-04T08:22:18.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff355e00640 1 -- 172.21.15.113:0/4014527717 >> v1:172.21.15.136:6789/0 conn(0x7ff3500c1860 legacy=0x7ff350162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:18.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff354a00640 1 -- 172.21.15.113:0/4014527717 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176294205 0 0) 0x7ff35015ef10 con 0x7ff3500c1860 2024-03-04T08:22:18.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff355e00640 1 -- 172.21.15.113:0/4014527717 >> v1:172.21.15.113:6789/0 conn(0x7ff3500c7050 legacy=0x7ff350169890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:18.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.817+0000 7ff355e00640 1 -- 172.21.15.113:0/4014527717 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35015f1b0 con 0x7ff350155d80 2024-03-04T08:22:18.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff355e00640 1 --2- 172.21.15.113:0/4014527717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff350155d80 0x7ff3500c7dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff3440097f0 tx=0x7ff34401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70488dbc1581e7c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:18.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff354a00640 1 -- 172.21.15.113:0/4014527717 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3440ab5f0 con 0x7ff350155d80 2024-03-04T08:22:18.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff354a00640 1 -- 172.21.15.113:0/4014527717 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3440b0070 con 0x7ff350155d80 2024-03-04T08:22:18.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff354a00640 1 -- 172.21.15.113:0/4014527717 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3440ab9b0 con 0x7ff350155d80 2024-03-04T08:22:18.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff358800640 1 -- 172.21.15.113:0/4014527717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff350155d80 msgr2=0x7ff3500c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:18.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff358800640 1 --2- 172.21.15.113:0/4014527717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff350155d80 0x7ff3500c7dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff3440097f0 tx=0x7ff34401c4e0 comp rx=0 tx=0).stop 2024-03-04T08:22:18.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff358800640 1 -- 172.21.15.113:0/4014527717 shutdown_connections 2024-03-04T08:22:18.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff358800640 1 --2- 172.21.15.113:0/4014527717 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff350155d80 0x7ff3500c7dd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:18.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff358800640 1 -- 172.21.15.113:0/4014527717 >> 172.21.15.113:0/4014527717 conn(0x7ff35006e820 msgr2=0x7ff35006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:18.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff358800640 1 -- 172.21.15.113:0/4014527717 shutdown_connections 2024-03-04T08:22:18.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.821+0000 7ff358800640 1 -- 172.21.15.113:0/4014527717 wait complete. 2024-03-04T08:22:18.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff358800640 1 Processor -- start 2024-03-04T08:22:18.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff358800640 1 -- start start 2024-03-04T08:22:18.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff358800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3500c1860 0x7ff3500c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff358800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3500c7050 0x7ff3500c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff358800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff350155d80 0x7ff3500c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff358800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35015fcd0 con 0x7ff350155d80 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff358800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35015f9f0 con 0x7ff3500c7050 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff358800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff350163380 con 0x7ff3500c1860 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff355e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3500c1860 0x7ff3500c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff355e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3500c1860 0x7ff3500c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:45568/0 (socket says 172.21.15.113:45568) 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff355e00640 1 -- 172.21.15.113:0/3812726039 learned_addr learned my addr 172.21.15.113:0/3812726039 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff356800640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff350155d80 0x7ff3500c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff355400640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3500c7050 0x7ff3500c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff356800640 1 -- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3500c1860 msgr2=0x7ff3500c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff356800640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3500c1860 0x7ff3500c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff356800640 1 -- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3500c7050 msgr2=0x7ff3500c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:18.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff356800640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3500c7050 0x7ff3500c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:18.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff356800640 1 -- 172.21.15.113:0/3812726039 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3501f7a30 con 0x7ff350155d80 2024-03-04T08:22:18.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff355e00640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3500c1860 0x7ff3500c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:22:18.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff356800640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff350155d80 0x7ff3500c4cc0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00cad0 tx=0x7ff34c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:18.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff34a000640 1 -- 172.21.15.113:0/3812726039 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c017070 con 0x7ff350155d80 2024-03-04T08:22:18.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff34a000640 1 -- 172.21.15.113:0/3812726039 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c00ab10 con 0x7ff350155d80 2024-03-04T08:22:18.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff34a000640 1 -- 172.21.15.113:0/3812726039 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c012400 con 0x7ff350155d80 2024-03-04T08:22:18.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff358800640 1 -- 172.21.15.113:0/3812726039 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3501f7cf0 con 0x7ff350155d80 2024-03-04T08:22:18.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.825+0000 7ff358800640 1 -- 172.21.15.113:0/3812726039 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3501f82b0 con 0x7ff350155d80 2024-03-04T08:22:18.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.829+0000 7ff358800640 1 -- 172.21.15.113:0/3812726039 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff350075c00 con 0x7ff350155d80 2024-03-04T08:22:18.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.829+0000 7ff34a000640 1 -- 172.21.15.113:0/3812726039 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff34c01a070 con 0x7ff350155d80 2024-03-04T08:22:18.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.829+0000 7ff34a000640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff334066f90 0x7ff334069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:18.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.829+0000 7ff34a000640 1 -- 172.21.15.113:0/3812726039 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7ff34c085510 con 0x7ff350155d80 2024-03-04T08:22:18.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.829+0000 7ff355e00640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff334066f90 0x7ff334069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:18.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.829+0000 7ff355e00640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff334066f90 0x7ff334069450 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff344009740 tx=0x7ff3440a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:18.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:18.833+0000 7ff34a000640 1 -- 172.21.15.113:0/3812726039 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff34c0554b0 con 0x7ff350155d80 2024-03-04T08:22:19.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.021+0000 7ff358800640 1 -- 172.21.15.113:0/3812726039 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3500722d0 con 0x7ff350155d80 2024-03-04T08:22:19.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.021+0000 7ff34a000640 1 -- 172.21.15.113:0/3812726039 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7ff34c059550 con 0x7ff350155d80 2024-03-04T08:22:19.024 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:19.025 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 -- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff334066f90 msgr2=0x7ff334069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff334066f90 0x7ff334069450 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff344009740 tx=0x7ff3440a7000 comp rx=0 tx=0).stop 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 -- 172.21.15.113:0/3812726039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff350155d80 msgr2=0x7ff3500c4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff350155d80 0x7ff3500c4cc0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00cad0 tx=0x7ff34c00ced0 comp rx=0 tx=0).stop 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 -- 172.21.15.113:0/3812726039 shutdown_connections 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff334066f90 0x7ff334069450 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff350155d80 0x7ff3500c4cc0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3500c7050 0x7ff3500c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 --2- 172.21.15.113:0/3812726039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3500c1860 0x7ff3500c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 -- 172.21.15.113:0/3812726039 >> 172.21.15.113:0/3812726039 conn(0x7ff35006e820 msgr2=0x7ff3500ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:19.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 -- 172.21.15.113:0/3812726039 shutdown_connections 2024-03-04T08:22:19.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:19.025+0000 7ff358800640 1 -- 172.21.15.113:0/3812726039 wait complete. 2024-03-04T08:22:19.043 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:22:19.043 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 179, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 179, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 173, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 729 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8388": { "addr": "172.21.15.113:6833/2999156262", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2999156262, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2999156262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8388, "incarnation": 171, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:stopping", "state_seq": 28 } }, "last_failure": 0, "last_failure_osd_epoch": 82, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:22:12.512430+0000", "qdb_cluster": [ 4247, 8032 ], "qdb_leader": 4247, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4247, "mds_1": 8032, "mds_2": 8388 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:22:20.080 INFO:tasks.ceph.mds.f.smithi136.stderr:2024-03-04T08:22:20.081+0000 7f7e8d315640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-04T08:22:20.080 INFO:tasks.ceph.mds.f.smithi136.stderr:2024-03-04T08:22:20.081+0000 7f7e8d315640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-04T08:22:21.045 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.189+0000 7f8443a00640 1 Processor -- start 2024-03-04T08:22:21.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8443a00640 1 -- start start 2024-03-04T08:22:21.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8443a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f843c152c60 0x7f843c15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8443a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f843c15c280 0x7f843c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8443a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 0x7f843c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8443a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c06b060 con 0x7f843c152c60 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8443a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c072d80 con 0x7f843c15c280 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8443a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c073510 con 0x7f843c0c7a80 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8441000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f843c152c60 0x7f843c15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8441000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f843c152c60 0x7f843c15bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59444/0 (socket says 172.21.15.113:59444) 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8441000640 1 -- 172.21.15.113:0/2943414239 learned_addr learned my addr 172.21.15.113:0/2943414239 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8433e00640 1 --2- 172.21.15.113:0/2943414239 >> v2:172.21.15.136:3300/0 conn(0x7f843c15c280 0x7f843c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8441a00640 1 --2- 172.21.15.113:0/2943414239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 0x7f843c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8441000640 1 -- 172.21.15.113:0/2943414239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 msgr2=0x7f843c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8441000640 1 --2- 172.21.15.113:0/2943414239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 0x7f843c0cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8441000640 1 -- 172.21.15.113:0/2943414239 >> v2:172.21.15.136:3300/0 conn(0x7f843c15c280 msgr2=0x7f843c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8441000640 1 --2- 172.21.15.113:0/2943414239 >> v2:172.21.15.136:3300/0 conn(0x7f843c15c280 0x7f843c0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8441000640 1 -- 172.21.15.113:0/2943414239 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c06b2b0 con 0x7f843c152c60 2024-03-04T08:22:21.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.193+0000 7f8441a00640 1 --2- 172.21.15.113:0/2943414239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 0x7f843c0cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:22:21.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8441000640 1 --2- 172.21.15.113:0/2943414239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 0x7f843c15bd40 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f843c0cdb30 tx=0x7f842c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452d15df913afea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:21.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f843be00640 1 -- 172.21.15.113:0/2943414239 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0aa610 con 0x7f843c152c60 2024-03-04T08:22:21.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f843be00640 1 -- 172.21.15.113:0/2943414239 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c0af070 con 0x7f843c152c60 2024-03-04T08:22:21.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f843be00640 1 -- 172.21.15.113:0/2943414239 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0aa9d0 con 0x7f843c152c60 2024-03-04T08:22:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 -- 172.21.15.113:0/2943414239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 msgr2=0x7f843c15bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 --2- 172.21.15.113:0/2943414239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 0x7f843c15bd40 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f843c0cdb30 tx=0x7f842c01c4f0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 -- 172.21.15.113:0/2943414239 shutdown_connections 2024-03-04T08:22:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 --2- 172.21.15.113:0/2943414239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 0x7f843c0cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 --2- 172.21.15.113:0/2943414239 >> v2:172.21.15.136:3300/0 conn(0x7f843c15c280 0x7f843c0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 --2- 172.21.15.113:0/2943414239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 0x7f843c15bd40 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 -- 172.21.15.113:0/2943414239 >> 172.21.15.113:0/2943414239 conn(0x7f843c0671b0 msgr2=0x7f843c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 -- 172.21.15.113:0/2943414239 shutdown_connections 2024-03-04T08:22:21.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 -- 172.21.15.113:0/2943414239 wait complete. 2024-03-04T08:22:21.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 Processor -- start 2024-03-04T08:22:21.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.197+0000 7f8443a00640 1 -- start start 2024-03-04T08:22:21.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8443a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 0x7f843c0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:21.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8443a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 0x7f843c1468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8443a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f843c15c280 0x7f843c146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8443a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0c8ac0 con 0x7f843c152c60 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8443a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0731d0 con 0x7f843c15c280 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8443a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0c8c40 con 0x7f843c0c7a80 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8433e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 0x7f843c1468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8433e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 0x7f843c1468c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59456/0 (socket says 172.21.15.113:59456) 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8433e00640 1 -- 172.21.15.113:0/2890039848 learned_addr learned my addr 172.21.15.113:0/2890039848 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8441000640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 0x7f843c0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8441a00640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f843c15c280 0x7f843c146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8433e00640 1 -- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 msgr2=0x7f843c0c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8433e00640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 0x7f843c0c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8433e00640 1 -- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f843c15c280 msgr2=0x7f843c146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8433e00640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f843c15c280 0x7f843c146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8433e00640 1 -- 172.21.15.113:0/2890039848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c0c5d00 con 0x7f843c152c60 2024-03-04T08:22:21.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8433e00640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 0x7f843c1468c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f842400e8a0 tx=0x7f842400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:21.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8439600640 1 -- 172.21.15.113:0/2890039848 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424017070 con 0x7f843c152c60 2024-03-04T08:22:21.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8439600640 1 -- 172.21.15.113:0/2890039848 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842400c940 con 0x7f843c152c60 2024-03-04T08:22:21.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8439600640 1 -- 172.21.15.113:0/2890039848 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842400ccf0 con 0x7f843c152c60 2024-03-04T08:22:21.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8443a00640 1 -- 172.21.15.113:0/2890039848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c0c5a20 con 0x7f843c152c60 2024-03-04T08:22:21.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8443a00640 1 -- 172.21.15.113:0/2890039848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c14d2d0 con 0x7f843c152c60 2024-03-04T08:22:21.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8443a00640 1 -- 172.21.15.113:0/2890039848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c075c00 con 0x7f843c152c60 2024-03-04T08:22:21.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.201+0000 7f8439600640 1 -- 172.21.15.113:0/2890039848 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f842401a070 con 0x7f843c152c60 2024-03-04T08:22:21.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.205+0000 7f8439600640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8418066f90 0x7f8418069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:21.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.205+0000 7f8439600640 1 -- 172.21.15.113:0/2890039848 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f8424085c70 con 0x7f843c152c60 2024-03-04T08:22:21.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.205+0000 7f8441000640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8418066f90 0x7f8418069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:21.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.205+0000 7f8441000640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8418066f90 0x7f8418069450 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f842c0ae000 tx=0x7f842c004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:21.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.209+0000 7f8439600640 1 -- 172.21.15.113:0/2890039848 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8424055b70 con 0x7f843c152c60 2024-03-04T08:22:21.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.393+0000 7f8443a00640 1 -- 172.21.15.113:0/2890039848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f843c072fb0 con 0x7f843c152c60 2024-03-04T08:22:21.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.393+0000 7f8439600640 1 -- 172.21.15.113:0/2890039848 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f8424059c10 con 0x7f843c152c60 2024-03-04T08:22:21.393 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:21.394 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:21.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 -- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8418066f90 msgr2=0x7f8418069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:21.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8418066f90 0x7f8418069450 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f842c0ae000 tx=0x7f842c004740 comp rx=0 tx=0).stop 2024-03-04T08:22:21.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 -- 172.21.15.113:0/2890039848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 msgr2=0x7f843c1468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:21.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 0x7f843c1468c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f842400e8a0 tx=0x7f842400eca0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 -- 172.21.15.113:0/2890039848 shutdown_connections 2024-03-04T08:22:21.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f843c15c280 0x7f843c146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8418066f90 0x7f8418069450 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f843c152c60 0x7f843c1468c0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 --2- 172.21.15.113:0/2890039848 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f843c0c7a80 0x7f843c0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:21.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 -- 172.21.15.113:0/2890039848 >> 172.21.15.113:0/2890039848 conn(0x7f843c0671b0 msgr2=0x7f843c151e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:21.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 -- 172.21.15.113:0/2890039848 shutdown_connections 2024-03-04T08:22:21.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:21.397+0000 7f8443a00640 1 -- 172.21.15.113:0/2890039848 wait complete. 2024-03-04T08:22:21.415 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:23.416 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:23.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.565+0000 7fad57200640 1 Processor -- start 2024-03-04T08:22:23.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.565+0000 7fad57200640 1 -- start start 2024-03-04T08:22:23.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.565+0000 7fad57200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad50152a50 0x7fad500ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:23.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad57200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad50152e30 con 0x7fad5015c4e0 2024-03-04T08:22:23.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad57200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5006f280 con 0x7fad5006ff20 2024-03-04T08:22:23.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad57200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad500c31b0 con 0x7fad50152a50 2024-03-04T08:22:23.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad4fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fad50152a50 0x7fad500ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:23.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad55200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fad5015c4e0 0x7fad50163060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46838/0 (socket says 172.21.15.113:46838) 2024-03-04T08:22:23.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad55200640 1 -- 172.21.15.113:0/2892261612 learned_addr learned my addr 172.21.15.113:0/2892261612 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:22:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad4f400640 1 -- 172.21.15.113:0/2892261612 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731151054 0 0) 0x7fad50152e30 con 0x7fad5015c4e0 2024-03-04T08:22:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad4f400640 1 -- 172.21.15.113:0/2892261612 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad34003530 con 0x7fad5015c4e0 2024-03-04T08:22:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad4fe00640 1 -- 172.21.15.113:0/2892261612 >> v1:172.21.15.136:6789/0 conn(0x7fad5006ff20 legacy=0x7fad5015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad4fe00640 1 -- 172.21.15.113:0/2892261612 >> v1:172.21.15.113:6789/0 conn(0x7fad5015c4e0 legacy=0x7fad50163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:23.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad4fe00640 1 -- 172.21.15.113:0/2892261612 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5006f560 con 0x7fad50152a50 2024-03-04T08:22:23.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad4fe00640 1 --2- 172.21.15.113:0/2892261612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad50152a50 0x7fad500ce120 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fad400097c0 tx=0x7fad4001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98e31e466a52513 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:23.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad4f400640 1 -- 172.21.15.113:0/2892261612 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad400b1070 con 0x7fad50152a50 2024-03-04T08:22:23.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad4f400640 1 -- 172.21.15.113:0/2892261612 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad400ac410 con 0x7fad50152a50 2024-03-04T08:22:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.569+0000 7fad4f400640 1 -- 172.21.15.113:0/2892261612 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad400ac7e0 con 0x7fad50152a50 2024-03-04T08:22:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 -- 172.21.15.113:0/2892261612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad50152a50 msgr2=0x7fad500ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 --2- 172.21.15.113:0/2892261612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad50152a50 0x7fad500ce120 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fad400097c0 tx=0x7fad4001c4a0 comp rx=0 tx=0).stop 2024-03-04T08:22:23.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 -- 172.21.15.113:0/2892261612 shutdown_connections 2024-03-04T08:22:23.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 --2- 172.21.15.113:0/2892261612 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad50152a50 0x7fad500ce120 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:23.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 -- 172.21.15.113:0/2892261612 >> 172.21.15.113:0/2892261612 conn(0x7fad500734d0 msgr2=0x7fad50066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:23.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 -- 172.21.15.113:0/2892261612 shutdown_connections 2024-03-04T08:22:23.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 -- 172.21.15.113:0/2892261612 wait complete. 2024-03-04T08:22:23.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 Processor -- start 2024-03-04T08:22:23.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 -- start start 2024-03-04T08:22:23.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5006ff20 0x7fad50150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:23.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad50152a50 0x7fad501f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:23.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad5015c4e0 0x7fad501f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:23.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500ce9f0 con 0x7fad5015c4e0 2024-03-04T08:22:23.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500ce870 con 0x7fad50152a50 2024-03-04T08:22:23.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad57200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad50166fa0 con 0x7fad5006ff20 2024-03-04T08:22:23.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad55200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad5015c4e0 0x7fad501f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:23.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad55200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad5015c4e0 0x7fad501f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59476/0 (socket says 172.21.15.113:59476) 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad55200640 1 -- 172.21.15.113:0/4167772311 learned_addr learned my addr 172.21.15.113:0/4167772311 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad47e00640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad50152a50 0x7fad501f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad4fe00640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5006ff20 0x7fad50150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad55200640 1 -- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5006ff20 msgr2=0x7fad50150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad55200640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5006ff20 0x7fad50150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad55200640 1 -- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad50152a50 msgr2=0x7fad501f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad55200640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad50152a50 0x7fad501f53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad55200640 1 -- 172.21.15.113:0/4167772311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5014cdf0 con 0x7fad5015c4e0 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad47e00640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad50152a50 0x7fad501f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.573+0000 7fad55200640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad5015c4e0 0x7fad501f58e0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fad4800cb40 tx=0x7fad4800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.577+0000 7fad4cc00640 1 -- 172.21.15.113:0/4167772311 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48017070 con 0x7fad5015c4e0 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.577+0000 7fad4cc00640 1 -- 172.21.15.113:0/4167772311 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4800aca0 con 0x7fad5015c4e0 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.577+0000 7fad57200640 1 -- 172.21.15.113:0/4167772311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5006ccb0 con 0x7fad5015c4e0 2024-03-04T08:22:23.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.577+0000 7fad4cc00640 1 -- 172.21.15.113:0/4167772311 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48012620 con 0x7fad5015c4e0 2024-03-04T08:22:23.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.577+0000 7fad57200640 1 -- 172.21.15.113:0/4167772311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5006d230 con 0x7fad5015c4e0 2024-03-04T08:22:23.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.577+0000 7fad57200640 1 -- 172.21.15.113:0/4167772311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad501f5080 con 0x7fad5015c4e0 2024-03-04T08:22:23.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.577+0000 7fad4cc00640 1 -- 172.21.15.113:0/4167772311 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad4801a070 con 0x7fad5015c4e0 2024-03-04T08:22:23.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.577+0000 7fad4cc00640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad2c066f60 0x7fad2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:23.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.577+0000 7fad4cc00640 1 -- 172.21.15.113:0/4167772311 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fad48085870 con 0x7fad5015c4e0 2024-03-04T08:22:23.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.577+0000 7fad4fe00640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad2c066f60 0x7fad2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:23.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.581+0000 7fad4fe00640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad2c066f60 0x7fad2c069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fad400ae000 tx=0x7fad40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:23.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.581+0000 7fad4cc00640 1 -- 172.21.15.113:0/4167772311 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fad501f5080 con 0x7fad5015c4e0 2024-03-04T08:22:23.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.769+0000 7fad57200640 1 -- 172.21.15.113:0/4167772311 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad50075c00 con 0x7fad5015c4e0 2024-03-04T08:22:23.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.769+0000 7fad4cc00640 1 -- 172.21.15.113:0/4167772311 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7fad48055e50 con 0x7fad5015c4e0 2024-03-04T08:22:23.770 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:23.770 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:23.771 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:23.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 -- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad2c066f60 msgr2=0x7fad2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad2c066f60 0x7fad2c069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fad400ae000 tx=0x7fad40008040 comp rx=0 tx=0).stop 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 -- 172.21.15.113:0/4167772311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad5015c4e0 msgr2=0x7fad501f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad5015c4e0 0x7fad501f58e0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fad4800cb40 tx=0x7fad4800a060 comp rx=0 tx=0).stop 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 -- 172.21.15.113:0/4167772311 shutdown_connections 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad2c066f60 0x7fad2c069420 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fad5015c4e0 0x7fad501f58e0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fad50152a50 0x7fad501f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 --2- 172.21.15.113:0/4167772311 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fad5006ff20 0x7fad50150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 -- 172.21.15.113:0/4167772311 >> 172.21.15.113:0/4167772311 conn(0x7fad500734d0 msgr2=0x7fad50066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 -- 172.21.15.113:0/4167772311 shutdown_connections 2024-03-04T08:22:23.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:23.773+0000 7fad57200640 1 -- 172.21.15.113:0/4167772311 wait complete. 2024-03-04T08:22:23.792 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:25.796 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:25.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.941+0000 7f0681600640 1 Processor -- start 2024-03-04T08:22:25.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.941+0000 7f0681600640 1 -- start start 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f0681600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f067c0b20b0 0x7f067c157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f0681600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f067c157c50 0x7f067c06b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f0681600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067c0c1ec0 con 0x7f067c0c85f0 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f0681600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067c0c9b30 con 0x7f067c157c50 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f0681600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067c0c12a0 con 0x7f067c0b20b0 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f067a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f067c157c50 0x7f067c06b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f067b400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f067c0c85f0 0x7f067c15a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46852/0 (socket says 172.21.15.113:46852) 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f067b400640 1 -- 172.21.15.113:0/826872864 learned_addr learned my addr 172.21.15.113:0/826872864 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f067aa00640 1 --2- 172.21.15.113:0/826872864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f067c0b20b0 0x7f067c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f067a000640 1 -- 172.21.15.113:0/826872864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f067c0b20b0 msgr2=0x7f067c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:25.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f067a000640 1 --2- 172.21.15.113:0/826872864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f067c0b20b0 0x7f067c157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:25.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f067a000640 1 -- 172.21.15.113:0/826872864 >> v1:172.21.15.113:6789/0 conn(0x7f067c0c85f0 legacy=0x7f067c15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:25.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f067a000640 1 -- 172.21.15.113:0/826872864 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c0c1580 con 0x7f067c157c50 2024-03-04T08:22:25.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f0679600640 1 -- 172.21.15.113:0/826872864 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504289803 0 0) 0x7f067c0c1ec0 con 0x7f067c0c85f0 2024-03-04T08:22:25.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.945+0000 7f067a000640 1 --2- 172.21.15.113:0/826872864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 0x7f067c06b060 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f06680097c0 tx=0x7f066801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d297ebf49dc186d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:25.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0679600640 1 -- 172.21.15.113:0/826872864 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06680b2070 con 0x7f067c157c50 2024-03-04T08:22:25.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0679600640 1 -- 172.21.15.113:0/826872864 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06680ad680 con 0x7f067c157c50 2024-03-04T08:22:25.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0679600640 1 -- 172.21.15.113:0/826872864 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06680ada50 con 0x7f067c157c50 2024-03-04T08:22:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 -- 172.21.15.113:0/826872864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 msgr2=0x7f067c06b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 --2- 172.21.15.113:0/826872864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 0x7f067c06b060 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f06680097c0 tx=0x7f066801c470 comp rx=0 tx=0).stop 2024-03-04T08:22:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 -- 172.21.15.113:0/826872864 shutdown_connections 2024-03-04T08:22:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 --2- 172.21.15.113:0/826872864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 0x7f067c06b060 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 --2- 172.21.15.113:0/826872864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f067c0b20b0 0x7f067c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 -- 172.21.15.113:0/826872864 >> 172.21.15.113:0/826872864 conn(0x7f067c06e820 msgr2=0x7f067c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 -- 172.21.15.113:0/826872864 shutdown_connections 2024-03-04T08:22:25.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 -- 172.21.15.113:0/826872864 wait complete. 2024-03-04T08:22:25.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 Processor -- start 2024-03-04T08:22:25.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 -- start start 2024-03-04T08:22:25.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f067c0b20b0 0x7f067c155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f067c0c85f0 0x7f067c144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 0x7f067c145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c0c1b80 con 0x7f067c157c50 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067c156ec0 con 0x7f067c0b20b0 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f0681600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067c0c80c0 con 0x7f067c0c85f0 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f067b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 0x7f067c145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f067b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 0x7f067c145000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59492/0 (socket says 172.21.15.113:59492) 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f067b400640 1 -- 172.21.15.113:0/4264101498 learned_addr learned my addr 172.21.15.113:0/4264101498 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f067aa00640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f067c0b20b0 0x7f067c155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.949+0000 7f067a000640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f067c0c85f0 0x7f067c144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f067b400640 1 -- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f067c0c85f0 msgr2=0x7f067c144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f067b400640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f067c0c85f0 0x7f067c144ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:25.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f067b400640 1 -- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f067c0b20b0 msgr2=0x7f067c155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f067b400640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f067c0b20b0 0x7f067c155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f067b400640 1 -- 172.21.15.113:0/4264101498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c1f3480 con 0x7f067c157c50 2024-03-04T08:22:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f067b400640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 0x7f067c145000 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f067400c7d0 tx=0x7f067400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f0672a00640 1 -- 172.21.15.113:0/4264101498 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0674018070 con 0x7f067c157c50 2024-03-04T08:22:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f0672a00640 1 -- 172.21.15.113:0/4264101498 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067400a640 con 0x7f067c157c50 2024-03-04T08:22:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f0672a00640 1 -- 172.21.15.113:0/4264101498 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067400aa70 con 0x7f067c157c50 2024-03-04T08:22:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f0681600640 1 -- 172.21.15.113:0/4264101498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067c0aeff0 con 0x7f067c157c50 2024-03-04T08:22:25.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f0681600640 1 -- 172.21.15.113:0/4264101498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067c143fe0 con 0x7f067c157c50 2024-03-04T08:22:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f0681600640 1 -- 172.21.15.113:0/4264101498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067c075c00 con 0x7f067c157c50 2024-03-04T08:22:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f0672a00640 1 -- 172.21.15.113:0/4264101498 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f067401b070 con 0x7f067c157c50 2024-03-04T08:22:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f0672a00640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f065c066f60 0x7f065c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:25.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.953+0000 7f0672a00640 1 -- 172.21.15.113:0/4264101498 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f0674084fd0 con 0x7f067c157c50 2024-03-04T08:22:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.957+0000 7f067aa00640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f065c066f60 0x7f065c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:25.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.957+0000 7f067aa00640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f065c066f60 0x7f065c069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f066c009ac0 tx=0x7f066c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:25.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:25.957+0000 7f0672a00640 1 -- 172.21.15.113:0/4264101498 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0674054f60 con 0x7f067c157c50 2024-03-04T08:22:26.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.145+0000 7f0681600640 1 -- 172.21.15.113:0/4264101498 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f067c14d780 con 0x7f067c157c50 2024-03-04T08:22:26.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.145+0000 7f0672a00640 1 -- 172.21.15.113:0/4264101498 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f0674059000 con 0x7f067c157c50 2024-03-04T08:22:26.145 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:26.145 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:26.145 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.145+0000 7f0681600640 1 -- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f065c066f60 msgr2=0x7f065c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.145+0000 7f0681600640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f065c066f60 0x7f065c069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f066c009ac0 tx=0x7f066c009190 comp rx=0 tx=0).stop 2024-03-04T08:22:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.145+0000 7f0681600640 1 -- 172.21.15.113:0/4264101498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 msgr2=0x7f067c145000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.149+0000 7f0681600640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 0x7f067c145000 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f067400c7d0 tx=0x7f067400cc90 comp rx=0 tx=0).stop 2024-03-04T08:22:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.149+0000 7f0681600640 1 -- 172.21.15.113:0/4264101498 shutdown_connections 2024-03-04T08:22:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.149+0000 7f0681600640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f065c066f60 0x7f065c069420 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:26.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.149+0000 7f0681600640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f067c157c50 0x7f067c145000 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:26.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.149+0000 7f0681600640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f067c0c85f0 0x7f067c144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:26.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.149+0000 7f0681600640 1 --2- 172.21.15.113:0/4264101498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f067c0b20b0 0x7f067c155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:26.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.149+0000 7f0681600640 1 -- 172.21.15.113:0/4264101498 >> 172.21.15.113:0/4264101498 conn(0x7f067c06e820 msgr2=0x7f067c0cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:26.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.149+0000 7f0681600640 1 -- 172.21.15.113:0/4264101498 shutdown_connections 2024-03-04T08:22:26.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:26.149+0000 7f0681600640 1 -- 172.21.15.113:0/4264101498 wait complete. 2024-03-04T08:22:26.166 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:28.167 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:28.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.313+0000 7f779b600640 1 Processor -- start 2024-03-04T08:22:28.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.313+0000 7f779b600640 1 -- start start 2024-03-04T08:22:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.313+0000 7f779b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77940c8200 0x7f7794070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.313+0000 7f779b600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7794152bd0 0x7f77941587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.313+0000 7f779b600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77941591f0 con 0x7f77941527f0 2024-03-04T08:22:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.313+0000 7f779b600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7794071df0 con 0x7f7794152bd0 2024-03-04T08:22:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.313+0000 7f779b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77941597b0 con 0x7f77940c8200 2024-03-04T08:22:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f778be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77940c8200 0x7f7794070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f778be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77940c8200 0x7f7794070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:38394/0 (socket says 172.21.15.113:38394) 2024-03-04T08:22:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f778be00640 1 -- 172.21.15.113:0/1778521674 learned_addr learned my addr 172.21.15.113:0/1778521674 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:28.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f7799600640 1 --2- 172.21.15.113:0/1778521674 >> v2:172.21.15.136:3300/0 conn(0x7f7794152bd0 0x7f77941587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:28.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f778be00640 1 -- 172.21.15.113:0/1778521674 >> v2:172.21.15.136:3300/0 conn(0x7f7794152bd0 msgr2=0x7f77941587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:28.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f778be00640 1 --2- 172.21.15.113:0/1778521674 >> v2:172.21.15.136:3300/0 conn(0x7f7794152bd0 0x7f77941587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:28.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f778be00640 1 -- 172.21.15.113:0/1778521674 >> v1:172.21.15.136:6789/0 conn(0x7f77941527f0 legacy=0x7f7794072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:28.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f778be00640 1 -- 172.21.15.113:0/1778521674 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77941594d0 con 0x7f77940c8200 2024-03-04T08:22:28.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f778be00640 1 --2- 172.21.15.113:0/1778521674 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77940c8200 0x7f7794070d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f777c009870 tx=0x7f777c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fc753472f3c0b6b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:28.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f7793e00640 1 -- 172.21.15.113:0/1778521674 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0a7070 con 0x7f77940c8200 2024-03-04T08:22:28.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.317+0000 7f7793e00640 1 -- 172.21.15.113:0/1778521674 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c0b0070 con 0x7f77940c8200 2024-03-04T08:22:28.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f7793e00640 1 -- 172.21.15.113:0/1778521674 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0ab890 con 0x7f77940c8200 2024-03-04T08:22:28.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 -- 172.21.15.113:0/1778521674 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77940c8200 msgr2=0x7f7794070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:28.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 --2- 172.21.15.113:0/1778521674 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77940c8200 0x7f7794070d40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f777c009870 tx=0x7f777c01c560 comp rx=0 tx=0).stop 2024-03-04T08:22:28.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 -- 172.21.15.113:0/1778521674 shutdown_connections 2024-03-04T08:22:28.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 --2- 172.21.15.113:0/1778521674 >> v2:172.21.15.136:3300/0 conn(0x7f7794152bd0 0x7f77941587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:28.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 --2- 172.21.15.113:0/1778521674 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77940c8200 0x7f7794070d40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:28.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 -- 172.21.15.113:0/1778521674 >> 172.21.15.113:0/1778521674 conn(0x7f779406e820 msgr2=0x7f779406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:28.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 -- 172.21.15.113:0/1778521674 shutdown_connections 2024-03-04T08:22:28.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 -- 172.21.15.113:0/1778521674 wait complete. 2024-03-04T08:22:28.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 Processor -- start 2024-03-04T08:22:28.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 -- start start 2024-03-04T08:22:28.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77940c8200 0x7f77941541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77941527f0 0x7f7794155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7794152bd0 0x7f7794155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77941654f0 con 0x7f77941527f0 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7794165370 con 0x7f77940c8200 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f779b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7794165670 con 0x7f7794152bd0 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f778be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77941527f0 0x7f7794155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f7799600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7794152bd0 0x7f7794155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f778be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77941527f0 0x7f7794155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56728/0 (socket says 172.21.15.113:56728) 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f7799600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7794152bd0 0x7f7794155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:38400/0 (socket says 172.21.15.113:38400) 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f778be00640 1 -- 172.21.15.113:0/2008209333 learned_addr learned my addr 172.21.15.113:0/2008209333 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f7798c00640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77940c8200 0x7f77941541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f778be00640 1 -- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7794152bd0 msgr2=0x7f7794155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f778be00640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7794152bd0 0x7f7794155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:28.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f778be00640 1 -- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77940c8200 msgr2=0x7f77941541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:28.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f778be00640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77940c8200 0x7f77941541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:28.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.321+0000 7f778be00640 1 -- 172.21.15.113:0/2008209333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7794153820 con 0x7f77941527f0 2024-03-04T08:22:28.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f778be00640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77941527f0 0x7f7794155820 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f777c0ad040 tx=0x7f777c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f7791600640 1 -- 172.21.15.113:0/2008209333 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0be5f0 con 0x7f77941527f0 2024-03-04T08:22:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f7791600640 1 -- 172.21.15.113:0/2008209333 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c0be790 con 0x7f77941527f0 2024-03-04T08:22:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f7791600640 1 -- 172.21.15.113:0/2008209333 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c0beb80 con 0x7f77941527f0 2024-03-04T08:22:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f779b600640 1 -- 172.21.15.113:0/2008209333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77941f2630 con 0x7f77941527f0 2024-03-04T08:22:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f779b600640 1 -- 172.21.15.113:0/2008209333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7794159f10 con 0x7f77941527f0 2024-03-04T08:22:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f779b600640 1 -- 172.21.15.113:0/2008209333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7794075c00 con 0x7f77941527f0 2024-03-04T08:22:28.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f7791600640 1 -- 172.21.15.113:0/2008209333 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f777c0b43f0 con 0x7f77941527f0 2024-03-04T08:22:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f7791600640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7770066f60 0x7f7770069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f7791600640 1 -- 172.21.15.113:0/2008209333 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f777c120360 con 0x7f77941527f0 2024-03-04T08:22:28.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.325+0000 7f7798c00640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7770066f60 0x7f7770069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:28.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.329+0000 7f7798c00640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7770066f60 0x7f7770069420 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7784001810 tx=0x7f7784009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:28.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.329+0000 7f7791600640 1 -- 172.21.15.113:0/2008209333 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f777c016150 con 0x7f77941527f0 2024-03-04T08:22:28.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.517+0000 7f779b600640 1 -- 172.21.15.113:0/2008209333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77940c79d0 con 0x7f77941527f0 2024-03-04T08:22:28.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.517+0000 7f7791600640 1 -- 172.21.15.113:0/2008209333 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f777c0f04a0 con 0x7f77941527f0 2024-03-04T08:22:28.518 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:28.518 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:28.518 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:28.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 -- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7770066f60 msgr2=0x7f7770069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:28.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7770066f60 0x7f7770069420 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7784001810 tx=0x7f7784009040 comp rx=0 tx=0).stop 2024-03-04T08:22:28.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 -- 172.21.15.113:0/2008209333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77941527f0 msgr2=0x7f7794155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:28.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77941527f0 0x7f7794155820 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f777c0ad040 tx=0x7f777c01ada0 comp rx=0 tx=0).stop 2024-03-04T08:22:28.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 -- 172.21.15.113:0/2008209333 shutdown_connections 2024-03-04T08:22:28.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7794152bd0 0x7f7794155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:28.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7770066f60 0x7f7770069420 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:28.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77941527f0 0x7f7794155820 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:28.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 --2- 172.21.15.113:0/2008209333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77940c8200 0x7f77941541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:28.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 -- 172.21.15.113:0/2008209333 >> 172.21.15.113:0/2008209333 conn(0x7f779406e820 msgr2=0x7f779406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:28.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 -- 172.21.15.113:0/2008209333 shutdown_connections 2024-03-04T08:22:28.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:28.521+0000 7f779b600640 1 -- 172.21.15.113:0/2008209333 wait complete. 2024-03-04T08:22:28.538 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:30.539 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:30.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.685+0000 7f49f4800640 1 Processor -- start 2024-03-04T08:22:30.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.685+0000 7f49f4800640 1 -- start start 2024-03-04T08:22:30.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.685+0000 7f49f4800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec14e4c0 0x7f49ec072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:30.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.685+0000 7f49f4800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f49ec0c22e0 0x7f49ec0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.685+0000 7f49f4800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ec0c27f0 con 0x7f49ec0c2d80 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.685+0000 7f49f4800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ec15ad60 con 0x7f49ec0c22e0 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.685+0000 7f49f4800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c2a90 con 0x7f49ec14e4c0 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f2800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f49ec0c2d80 0x7f49ec15b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40250/0 (socket says 172.21.15.113:40250) 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f2800640 1 -- 172.21.15.113:0/1221134247 learned_addr learned my addr 172.21.15.113:0/1221134247 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f1400640 1 --2- 172.21.15.113:0/1221134247 >> v2:172.21.15.136:3300/0 conn(0x7f49ec0c22e0 0x7f49ec0731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f1e00640 1 --2- 172.21.15.113:0/1221134247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec14e4c0 0x7f49ec072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f0a00640 1 -- 172.21.15.113:0/1221134247 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921084117 0 0) 0x7f49ec0c27f0 con 0x7f49ec0c2d80 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f0a00640 1 -- 172.21.15.113:0/1221134247 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d4003530 con 0x7f49ec0c2d80 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f1e00640 1 -- 172.21.15.113:0/1221134247 >> v2:172.21.15.136:3300/0 conn(0x7f49ec0c22e0 msgr2=0x7f49ec0731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f1e00640 1 --2- 172.21.15.113:0/1221134247 >> v2:172.21.15.136:3300/0 conn(0x7f49ec0c22e0 0x7f49ec0731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f1e00640 1 -- 172.21.15.113:0/1221134247 >> v1:172.21.15.113:6789/0 conn(0x7f49ec0c2d80 legacy=0x7f49ec15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f1e00640 1 -- 172.21.15.113:0/1221134247 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec070c60 con 0x7f49ec14e4c0 2024-03-04T08:22:30.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f1400640 1 --2- 172.21.15.113:0/1221134247 >> v2:172.21.15.136:3300/0 conn(0x7f49ec0c22e0 0x7f49ec0731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:22:30.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.689+0000 7f49f1e00640 1 --2- 172.21.15.113:0/1221134247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec14e4c0 0x7f49ec072c80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f49e00097c0 tx=0x7f49e001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b63a0eb4631a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:30.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f0a00640 1 -- 172.21.15.113:0/1221134247 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e00b5070 con 0x7f49ec14e4c0 2024-03-04T08:22:30.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f0a00640 1 -- 172.21.15.113:0/1221134247 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e00a8990 con 0x7f49ec14e4c0 2024-03-04T08:22:30.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f0a00640 1 -- 172.21.15.113:0/1221134247 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e00a8d40 con 0x7f49ec14e4c0 2024-03-04T08:22:30.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 -- 172.21.15.113:0/1221134247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec14e4c0 msgr2=0x7f49ec072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:30.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 --2- 172.21.15.113:0/1221134247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec14e4c0 0x7f49ec072c80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f49e00097c0 tx=0x7f49e001ec90 comp rx=0 tx=0).stop 2024-03-04T08:22:30.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 -- 172.21.15.113:0/1221134247 shutdown_connections 2024-03-04T08:22:30.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 --2- 172.21.15.113:0/1221134247 >> v2:172.21.15.136:3300/0 conn(0x7f49ec0c22e0 0x7f49ec0731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:30.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 --2- 172.21.15.113:0/1221134247 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec14e4c0 0x7f49ec072c80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:30.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 -- 172.21.15.113:0/1221134247 >> 172.21.15.113:0/1221134247 conn(0x7f49ec06e820 msgr2=0x7f49ec06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:30.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 -- 172.21.15.113:0/1221134247 shutdown_connections 2024-03-04T08:22:30.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 -- 172.21.15.113:0/1221134247 wait complete. 2024-03-04T08:22:30.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 Processor -- start 2024-03-04T08:22:30.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 -- start start 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec0c22e0 0x7f49ec159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49ec0c2d80 0x7f49ec15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49ec14e4c0 0x7f49ec1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec071500 con 0x7f49ec14e4c0 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec071220 con 0x7f49ec0c2d80 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f4800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec14e8a0 con 0x7f49ec0c22e0 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49ec14e4c0 0x7f49ec1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49ec14e4c0 0x7f49ec1538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56744/0 (socket says 172.21.15.113:56744) 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f2800640 1 -- 172.21.15.113:0/3894986570 learned_addr learned my addr 172.21.15.113:0/3894986570 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f1e00640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec0c22e0 0x7f49ec159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.693+0000 7f49f1400640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49ec0c2d80 0x7f49ec15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49f2800640 1 -- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec0c22e0 msgr2=0x7f49ec159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:30.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49f2800640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec0c22e0 0x7f49ec159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:30.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49f2800640 1 -- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49ec0c2d80 msgr2=0x7f49ec15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:30.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49f2800640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49ec0c2d80 0x7f49ec15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:30.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49f2800640 1 -- 172.21.15.113:0/3894986570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec0ae720 con 0x7f49ec14e4c0 2024-03-04T08:22:30.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49f2800640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49ec14e4c0 0x7f49ec1538c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f49e800ca70 tx=0x7f49e800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:30.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49e6000640 1 -- 172.21.15.113:0/3894986570 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8017070 con 0x7f49ec14e4c0 2024-03-04T08:22:30.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49e6000640 1 -- 172.21.15.113:0/3894986570 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e800abd0 con 0x7f49ec14e4c0 2024-03-04T08:22:30.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49e6000640 1 -- 172.21.15.113:0/3894986570 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8012400 con 0x7f49ec14e4c0 2024-03-04T08:22:30.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49f4800640 1 -- 172.21.15.113:0/3894986570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec0c3860 con 0x7f49ec14e4c0 2024-03-04T08:22:30.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49f4800640 1 -- 172.21.15.113:0/3894986570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec0c4ec0 con 0x7f49ec14e4c0 2024-03-04T08:22:30.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49f4800640 1 -- 172.21.15.113:0/3894986570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ec075c00 con 0x7f49ec14e4c0 2024-03-04T08:22:30.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49e6000640 1 -- 172.21.15.113:0/3894986570 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49e801a070 con 0x7f49ec14e4c0 2024-03-04T08:22:30.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49e6000640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49d0066f60 0x7f49d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:30.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49e6000640 1 -- 172.21.15.113:0/3894986570 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f49e8085600 con 0x7f49ec14e4c0 2024-03-04T08:22:30.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.697+0000 7f49f1e00640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49d0066f60 0x7f49d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:30.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.701+0000 7f49f1e00640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49d0066f60 0x7f49d0069420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f49e00b2000 tx=0x7f49e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:30.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.701+0000 7f49e6000640 1 -- 172.21.15.113:0/3894986570 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f49e8055590 con 0x7f49ec14e4c0 2024-03-04T08:22:30.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.889+0000 7f49f4800640 1 -- 172.21.15.113:0/3894986570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49ec0c1260 con 0x7f49ec14e4c0 2024-03-04T08:22:30.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.889+0000 7f49e6000640 1 -- 172.21.15.113:0/3894986570 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f49e8059630 con 0x7f49ec14e4c0 2024-03-04T08:22:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:30.890 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:30.890 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:30.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 -- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49d0066f60 msgr2=0x7f49d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:30.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49d0066f60 0x7f49d0069420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f49e00b2000 tx=0x7f49e0008040 comp rx=0 tx=0).stop 2024-03-04T08:22:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 -- 172.21.15.113:0/3894986570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49ec14e4c0 msgr2=0x7f49ec1538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49ec14e4c0 0x7f49ec1538c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f49e800ca70 tx=0x7f49e800cf30 comp rx=0 tx=0).stop 2024-03-04T08:22:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 -- 172.21.15.113:0/3894986570 shutdown_connections 2024-03-04T08:22:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49d0066f60 0x7f49d0069420 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49ec14e4c0 0x7f49ec1538c0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49ec0c2d80 0x7f49ec15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 --2- 172.21.15.113:0/3894986570 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49ec0c22e0 0x7f49ec159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 -- 172.21.15.113:0/3894986570 >> 172.21.15.113:0/3894986570 conn(0x7f49ec06e820 msgr2=0x7f49ec06b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 -- 172.21.15.113:0/3894986570 shutdown_connections 2024-03-04T08:22:30.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:30.893+0000 7f49f4800640 1 -- 172.21.15.113:0/3894986570 wait complete. 2024-03-04T08:22:30.911 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:22:30.917 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 179, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 179, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 173, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 729 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8388": { "addr": "172.21.15.113:6833/2999156262", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2999156262, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2999156262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8388, "incarnation": 171, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:stopping", "state_seq": 28 } }, "last_failure": 0, "last_failure_osd_epoch": 82, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:22:12.512430+0000", "qdb_cluster": [ 4247, 8032 ], "qdb_leader": 4247, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4247, "mds_1": 8032, "mds_2": 8388 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:22:32.918 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:33.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a6ba00640 1 Processor -- start 2024-03-04T08:22:33.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a6ba00640 1 -- start start 2024-03-04T08:22:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a6ba00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5a640ccb40 0x7f5a640c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a6ba00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a64154f10 0x7f5a6415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a6ba00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6415aed0 con 0x7f5a6415aa40 2024-03-04T08:22:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a6ba00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6415b170 con 0x7f5a64154f10 2024-03-04T08:22:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a6ba00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6415b410 con 0x7f5a640ccb40 2024-03-04T08:22:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a69a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a64154f10 0x7f5a6415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a5be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5a6415aa40 0x7f5a64154800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40254/0 (socket says 172.21.15.113:40254) 2024-03-04T08:22:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a5be00640 1 -- 172.21.15.113:0/4130661301 learned_addr learned my addr 172.21.15.113:0/4130661301 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:22:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a69000640 1 --2- 172.21.15.113:0/4130661301 >> v2:172.21.15.136:3300/0 conn(0x7f5a640ccb40 0x7f5a640c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:33.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a69a00640 1 -- 172.21.15.113:0/4130661301 >> v2:172.21.15.136:3300/0 conn(0x7f5a640ccb40 msgr2=0x7f5a640c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:33.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a69a00640 1 --2- 172.21.15.113:0/4130661301 >> v2:172.21.15.136:3300/0 conn(0x7f5a640ccb40 0x7f5a640c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:33.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a69a00640 1 -- 172.21.15.113:0/4130661301 >> v1:172.21.15.113:6789/0 conn(0x7f5a6415aa40 legacy=0x7f5a64154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:33.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a69a00640 1 -- 172.21.15.113:0/4130661301 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6415b9b0 con 0x7f5a64154f10 2024-03-04T08:22:33.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.065+0000 7f5a63e00640 1 -- 172.21.15.113:0/4130661301 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237835071 0 0) 0x7f5a6415aed0 con 0x7f5a6415aa40 2024-03-04T08:22:33.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a69a00640 1 --2- 172.21.15.113:0/4130661301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64154f10 0x7f5a6415e5f0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009d90 tx=0x7f5a5c01f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91cebe88f18e23 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:33.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a63e00640 1 -- 172.21.15.113:0/4130661301 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0b4070 con 0x7f5a64154f10 2024-03-04T08:22:33.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a63e00640 1 -- 172.21.15.113:0/4130661301 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c0af600 con 0x7f5a64154f10 2024-03-04T08:22:33.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a63e00640 1 -- 172.21.15.113:0/4130661301 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0af9d0 con 0x7f5a64154f10 2024-03-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a6ba00640 1 -- 172.21.15.113:0/4130661301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64154f10 msgr2=0x7f5a6415e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a6ba00640 1 --2- 172.21.15.113:0/4130661301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64154f10 0x7f5a6415e5f0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c009d90 tx=0x7f5a5c01f120 comp rx=0 tx=0).stop 2024-03-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a6ba00640 1 -- 172.21.15.113:0/4130661301 shutdown_connections 2024-03-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a6ba00640 1 --2- 172.21.15.113:0/4130661301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64154f10 0x7f5a6415e5f0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a6ba00640 1 --2- 172.21.15.113:0/4130661301 >> v2:172.21.15.136:3300/0 conn(0x7f5a640ccb40 0x7f5a640c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:33.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a6ba00640 1 -- 172.21.15.113:0/4130661301 >> 172.21.15.113:0/4130661301 conn(0x7f5a6406b060 msgr2=0x7f5a6406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a6ba00640 1 -- 172.21.15.113:0/4130661301 shutdown_connections 2024-03-04T08:22:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.069+0000 7f5a6ba00640 1 -- 172.21.15.113:0/4130661301 wait complete. 2024-03-04T08:22:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a6ba00640 1 Processor -- start 2024-03-04T08:22:33.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a6ba00640 1 -- start start 2024-03-04T08:22:33.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a6ba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a640ccb40 0x7f5a6415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a6ba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a64154f10 0x7f5a6414a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a6ba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64152290 0x7f5a6414abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a6ba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640c78b0 con 0x7f5a64152290 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a6ba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6415e0d0 con 0x7f5a64154f10 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a6ba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a640ccf20 con 0x7f5a640ccb40 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a69a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64152290 0x7f5a6414abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a69a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64152290 0x7f5a6414abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56758/0 (socket says 172.21.15.113:56758) 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a69a00640 1 -- 172.21.15.113:0/2525048967 learned_addr learned my addr 172.21.15.113:0/2525048967 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a69000640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a640ccb40 0x7f5a6415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a5be00640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a64154f10 0x7f5a6414a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a69a00640 1 -- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a640ccb40 msgr2=0x7f5a6415a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a69a00640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a640ccb40 0x7f5a6415a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a69a00640 1 -- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a64154f10 msgr2=0x7f5a6414a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:33.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a69a00640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a64154f10 0x7f5a6414a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a69a00640 1 -- 172.21.15.113:0/2525048967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a641f4670 con 0x7f5a64152290 2024-03-04T08:22:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a69a00640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64152290 0x7f5a6414abf0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c01cd50 tx=0x7f5a5c01bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a61600640 1 -- 172.21.15.113:0/2525048967 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0bf020 con 0x7f5a64152290 2024-03-04T08:22:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a61600640 1 -- 172.21.15.113:0/2525048967 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c0ba470 con 0x7f5a64152290 2024-03-04T08:22:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a61600640 1 -- 172.21.15.113:0/2525048967 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c0b4040 con 0x7f5a64152290 2024-03-04T08:22:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a6ba00640 1 -- 172.21.15.113:0/2525048967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a641f4910 con 0x7f5a64152290 2024-03-04T08:22:33.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.073+0000 7f5a6ba00640 1 -- 172.21.15.113:0/2525048967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a64153070 con 0x7f5a64152290 2024-03-04T08:22:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.077+0000 7f5a6ba00640 1 -- 172.21.15.113:0/2525048967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a64075c00 con 0x7f5a64152290 2024-03-04T08:22:33.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.077+0000 7f5a61600640 1 -- 172.21.15.113:0/2525048967 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a5c0c2020 con 0x7f5a64152290 2024-03-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.077+0000 7f5a61600640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a40066f60 0x7f5a40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.077+0000 7f5a61600640 1 -- 172.21.15.113:0/2525048967 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f5a5c1240f0 con 0x7f5a64152290 2024-03-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.077+0000 7f5a69000640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a40066f60 0x7f5a40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:33.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.077+0000 7f5a69000640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a40066f60 0x7f5a40069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009b60 tx=0x7f5a54009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:33.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.081+0000 7f5a61600640 1 -- 172.21.15.113:0/2525048967 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5a5c0c9030 con 0x7f5a64152290 2024-03-04T08:22:33.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 -- 172.21.15.113:0/2525048967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a640c6a30 con 0x7f5a64152290 2024-03-04T08:22:33.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a61600640 1 -- 172.21.15.113:0/2525048967 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f5a5c018c80 con 0x7f5a64152290 2024-03-04T08:22:33.268 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:33.268 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:33.269 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:33.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 -- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a40066f60 msgr2=0x7f5a40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:33.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a40066f60 0x7f5a40069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009b60 tx=0x7f5a54009190 comp rx=0 tx=0).stop 2024-03-04T08:22:33.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 -- 172.21.15.113:0/2525048967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64152290 msgr2=0x7f5a6414abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:33.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64152290 0x7f5a6414abf0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c01cd50 tx=0x7f5a5c01bec0 comp rx=0 tx=0).stop 2024-03-04T08:22:33.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 -- 172.21.15.113:0/2525048967 shutdown_connections 2024-03-04T08:22:33.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a40066f60 0x7f5a40069420 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:33.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a64152290 0x7f5a6414abf0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a64154f10 0x7f5a6414a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 --2- 172.21.15.113:0/2525048967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a640ccb40 0x7f5a6415a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 -- 172.21.15.113:0/2525048967 >> 172.21.15.113:0/2525048967 conn(0x7f5a6406b060 msgr2=0x7f5a640c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 -- 172.21.15.113:0/2525048967 shutdown_connections 2024-03-04T08:22:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:33.269+0000 7f5a6ba00640 1 -- 172.21.15.113:0/2525048967 wait complete. 2024-03-04T08:22:33.290 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:35.291 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c9200640 1 Processor -- start 2024-03-04T08:22:35.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c9200640 1 -- start start 2024-03-04T08:22:35.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c9200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77c414e4c0 0x7f77c4072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:35.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c9200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 0x7f77c40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c9200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c40c27f0 con 0x7f77c40c2d80 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c9200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c415ad60 con 0x7f77c414e4c0 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c9200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c40c2a90 con 0x7f77c40c22e0 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c1e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77c414e4c0 0x7f77c4072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c1e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f77c414e4c0 0x7f77c4072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56768/0 (socket says 172.21.15.113:56768) 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c1e00640 1 -- 172.21.15.113:0/2278053509 learned_addr learned my addr 172.21.15.113:0/2278053509 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77b9400640 1 --2- 172.21.15.113:0/2278053509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 0x7f77c40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c1e00640 1 -- 172.21.15.113:0/2278053509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 msgr2=0x7f77c40731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c1e00640 1 --2- 172.21.15.113:0/2278053509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 0x7f77c40731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c1e00640 1 -- 172.21.15.113:0/2278053509 >> v1:172.21.15.136:6789/0 conn(0x7f77c40c2d80 legacy=0x7f77c415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:35.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.437+0000 7f77c1e00640 1 -- 172.21.15.113:0/2278053509 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c4070c60 con 0x7f77c414e4c0 2024-03-04T08:22:35.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c1e00640 1 --2- 172.21.15.113:0/2278053509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c414e4c0 0x7f77c4072c80 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009e50 tx=0x7f77b401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68abf7c64181634 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:35.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c1400640 1 -- 172.21.15.113:0/2278053509 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40b4070 con 0x7f77c414e4c0 2024-03-04T08:22:35.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c1400640 1 -- 172.21.15.113:0/2278053509 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b40af400 con 0x7f77c414e4c0 2024-03-04T08:22:35.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c1400640 1 -- 172.21.15.113:0/2278053509 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40af7d0 con 0x7f77c414e4c0 2024-03-04T08:22:35.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c9200640 1 -- 172.21.15.113:0/2278053509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c414e4c0 msgr2=0x7f77c4072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:35.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c9200640 1 --2- 172.21.15.113:0/2278053509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c414e4c0 0x7f77c4072c80 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009e50 tx=0x7f77b401f210 comp rx=0 tx=0).stop 2024-03-04T08:22:35.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c9200640 1 -- 172.21.15.113:0/2278053509 shutdown_connections 2024-03-04T08:22:35.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c9200640 1 --2- 172.21.15.113:0/2278053509 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 0x7f77c40731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:35.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c9200640 1 --2- 172.21.15.113:0/2278053509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c414e4c0 0x7f77c4072c80 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:35.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c9200640 1 -- 172.21.15.113:0/2278053509 >> 172.21.15.113:0/2278053509 conn(0x7f77c406e820 msgr2=0x7f77c406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:35.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c9200640 1 -- 172.21.15.113:0/2278053509 shutdown_connections 2024-03-04T08:22:35.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.441+0000 7f77c9200640 1 -- 172.21.15.113:0/2278053509 wait complete. 2024-03-04T08:22:35.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c9200640 1 Processor -- start 2024-03-04T08:22:35.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c9200640 1 -- start start 2024-03-04T08:22:35.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c9200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 0x7f77c40c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:35.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c9200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c40c2d80 0x7f77c40ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:35.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c9200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77c414e4c0 0x7f77c40cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:35.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c9200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c4071500 con 0x7f77c40c2d80 2024-03-04T08:22:35.446 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c9200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c4071220 con 0x7f77c414e4c0 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c9200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c414e8a0 con 0x7f77c40c22e0 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77b9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c40c2d80 0x7f77c40ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77b9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c40c2d80 0x7f77c40ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56780/0 (socket says 172.21.15.113:56780) 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77b9400640 1 -- 172.21.15.113:0/2291357574 learned_addr learned my addr 172.21.15.113:0/2291357574 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c1e00640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 0x7f77c40c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c2800640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77c414e4c0 0x7f77c40cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77b9400640 1 -- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 msgr2=0x7f77c40c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77b9400640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 0x7f77c40c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77b9400640 1 -- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77c414e4c0 msgr2=0x7f77c40cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77b9400640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77c414e4c0 0x7f77c40cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77b9400640 1 -- 172.21.15.113:0/2291357574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c415a4b0 con 0x7f77c40c2d80 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c1e00640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 0x7f77c40c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c2800640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77c414e4c0 0x7f77c40cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:22:35.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77b9400640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c40c2d80 0x7f77c40ca780 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00d800 tx=0x7f77ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:35.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77baa00640 1 -- 172.21.15.113:0/2291357574 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac004020 con 0x7f77c40c2d80 2024-03-04T08:22:35.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77baa00640 1 -- 172.21.15.113:0/2291357574 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac0041c0 con 0x7f77c40c2d80 2024-03-04T08:22:35.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c9200640 1 -- 172.21.15.113:0/2291357574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c4158d00 con 0x7f77c40c2d80 2024-03-04T08:22:35.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77baa00640 1 -- 172.21.15.113:0/2291357574 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0043b0 con 0x7f77c40c2d80 2024-03-04T08:22:35.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.445+0000 7f77c9200640 1 -- 172.21.15.113:0/2291357574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c4154ed0 con 0x7f77c40c2d80 2024-03-04T08:22:35.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.449+0000 7f77c9200640 1 -- 172.21.15.113:0/2291357574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c4075c00 con 0x7f77c40c2d80 2024-03-04T08:22:35.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.449+0000 7f77baa00640 1 -- 172.21.15.113:0/2291357574 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77ac01b020 con 0x7f77c40c2d80 2024-03-04T08:22:35.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.449+0000 7f77baa00640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77a0066f60 0x7f77a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:35.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.449+0000 7f77baa00640 1 -- 172.21.15.113:0/2291357574 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f77ac0857d0 con 0x7f77c40c2d80 2024-03-04T08:22:35.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.449+0000 7f77c1e00640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77a0066f60 0x7f77a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:35.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.449+0000 7f77c1e00640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77a0066f60 0x7f77a0069420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f77b40b1000 tx=0x7f77b40aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:35.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.453+0000 7f77baa00640 1 -- 172.21.15.113:0/2291357574 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f77ac055e40 con 0x7f77c40c2d80 2024-03-04T08:22:35.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 -- 172.21.15.113:0/2291357574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77c40c8b30 con 0x7f77c40c2d80 2024-03-04T08:22:35.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77baa00640 1 -- 172.21.15.113:0/2291357574 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f77ac055740 con 0x7f77c40c2d80 2024-03-04T08:22:35.645 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:35.645 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:35.645 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:35.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 -- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77a0066f60 msgr2=0x7f77a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:35.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77a0066f60 0x7f77a0069420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f77b40b1000 tx=0x7f77b40aa000 comp rx=0 tx=0).stop 2024-03-04T08:22:35.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 -- 172.21.15.113:0/2291357574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c40c2d80 msgr2=0x7f77c40ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:35.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c40c2d80 0x7f77c40ca780 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00d800 tx=0x7f77ac00dcc0 comp rx=0 tx=0).stop 2024-03-04T08:22:35.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 -- 172.21.15.113:0/2291357574 shutdown_connections 2024-03-04T08:22:35.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77c414e4c0 0x7f77c40cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:35.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f77a0066f60 0x7f77a0069420 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:35.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f77c40c2d80 0x7f77c40ca780 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:35.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 --2- 172.21.15.113:0/2291357574 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f77c40c22e0 0x7f77c40c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:35.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 -- 172.21.15.113:0/2291357574 >> 172.21.15.113:0/2291357574 conn(0x7f77c406e820 msgr2=0x7f77c40c1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:35.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 -- 172.21.15.113:0/2291357574 shutdown_connections 2024-03-04T08:22:35.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:35.645+0000 7f77c9200640 1 -- 172.21.15.113:0/2291357574 wait complete. 2024-03-04T08:22:35.667 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:37.668 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:37.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.813+0000 7f318f600640 1 Processor -- start 2024-03-04T08:22:37.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.813+0000 7f318f600640 1 -- start start 2024-03-04T08:22:37.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318f600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f318806ff20 0x7f318806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:37.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318f600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f31880ce120 0x7f31880ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:37.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318f600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31880ca0e0 con 0x7f3188158010 2024-03-04T08:22:37.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318f600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318806f450 con 0x7f31880ce120 2024-03-04T08:22:37.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318f600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318815ad80 con 0x7f318806ff20 2024-03-04T08:22:37.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f317fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f318806ff20 0x7f318806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:37.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318d600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f31880ce120 0x7f31880ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:37.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318d600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f31880ce120 0x7f31880ce5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:39364/0 (socket says 172.21.15.113:39364) 2024-03-04T08:22:37.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318cc00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f3188158010 0x7f318815b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:52476/0 (socket says 172.21.15.113:52476) 2024-03-04T08:22:37.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f317fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f318806ff20 0x7f318806dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:42532/0 (socket says 172.21.15.113:42532) 2024-03-04T08:22:37.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318d600640 1 -- 172.21.15.113:0/1927037859 learned_addr learned my addr 172.21.15.113:0/1927037859 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:37.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318d600640 1 -- 172.21.15.113:0/1927037859 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f318806ff20 msgr2=0x7f318806dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:37.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318d600640 1 --2- 172.21.15.113:0/1927037859 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f318806ff20 0x7f318806dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:37.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318d600640 1 -- 172.21.15.113:0/1927037859 >> v1:172.21.15.136:6789/0 conn(0x7f3188158010 legacy=0x7f318815b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:37.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.817+0000 7f318d600640 1 -- 172.21.15.113:0/1927037859 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318815aa30 con 0x7f31880ce120 2024-03-04T08:22:37.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318d600640 1 --2- 172.21.15.113:0/1927037859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31880ce120 0x7f31880ce5a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3180009e50 tx=0x7f318001ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c73a17cad0fadce1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f3187e00640 1 -- 172.21.15.113:0/1927037859 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800a5700 con 0x7f31880ce120 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f3187e00640 1 -- 172.21.15.113:0/1927037859 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31800a58a0 con 0x7f31880ce120 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f3187e00640 1 -- 172.21.15.113:0/1927037859 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31800a5c70 con 0x7f31880ce120 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318f600640 1 -- 172.21.15.113:0/1927037859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31880ce120 msgr2=0x7f31880ce5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318f600640 1 --2- 172.21.15.113:0/1927037859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31880ce120 0x7f31880ce5a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3180009e50 tx=0x7f318001ca30 comp rx=0 tx=0).stop 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318f600640 1 -- 172.21.15.113:0/1927037859 shutdown_connections 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318f600640 1 --2- 172.21.15.113:0/1927037859 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f31880ce120 0x7f31880ce5a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318f600640 1 --2- 172.21.15.113:0/1927037859 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f318806ff20 0x7f318806dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318f600640 1 -- 172.21.15.113:0/1927037859 >> 172.21.15.113:0/1927037859 conn(0x7f31880734d0 msgr2=0x7f3188066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318f600640 1 -- 172.21.15.113:0/1927037859 shutdown_connections 2024-03-04T08:22:37.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318f600640 1 -- 172.21.15.113:0/1927037859 wait complete. 2024-03-04T08:22:37.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318f600640 1 Processor -- start 2024-03-04T08:22:37.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.821+0000 7f318f600640 1 -- start start 2024-03-04T08:22:37.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318f600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f318806ff20 0x7f31881407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:37.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318f600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f31880ce120 0x7f31881430f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:37.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318f600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3188158010 0x7f31881f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:37.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318f600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318815b920 con 0x7f3188158010 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318f600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318806e290 con 0x7f318806ff20 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318f600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31881583f0 con 0x7f31880ce120 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3188158010 0x7f31881f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f318806ff20 0x7f31881407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f317fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f31880ce120 0x7f31881430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3188158010 0x7f31881f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46336/0 (socket says 172.21.15.113:46336) 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318d600640 1 -- 172.21.15.113:0/139384682 learned_addr learned my addr 172.21.15.113:0/139384682 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318cc00640 1 -- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f31880ce120 msgr2=0x7f31881430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318cc00640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f31880ce120 0x7f31881430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318cc00640 1 -- 172.21.15.113:0/139384682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3188158010 msgr2=0x7f31881f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318cc00640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3188158010 0x7f31881f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318cc00640 1 -- 172.21.15.113:0/139384682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3188141310 con 0x7f318806ff20 2024-03-04T08:22:37.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318d600640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3188158010 0x7f31881f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:22:37.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318cc00640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f318806ff20 0x7f31881407d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f317800c390 tx=0x7f317800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:37.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f3185600640 1 -- 172.21.15.113:0/139384682 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31780074e0 con 0x7f318806ff20 2024-03-04T08:22:37.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318f600640 1 -- 172.21.15.113:0/139384682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31881415f0 con 0x7f318806ff20 2024-03-04T08:22:37.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f3185600640 1 -- 172.21.15.113:0/139384682 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178007680 con 0x7f318806ff20 2024-03-04T08:22:37.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f3185600640 1 -- 172.21.15.113:0/139384682 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31780049f0 con 0x7f318806ff20 2024-03-04T08:22:37.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318f600640 1 -- 172.21.15.113:0/139384682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3188141bb0 con 0x7f318806ff20 2024-03-04T08:22:37.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.825+0000 7f318f600640 1 -- 172.21.15.113:0/139384682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3188075c00 con 0x7f318806ff20 2024-03-04T08:22:37.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.829+0000 7f3185600640 1 -- 172.21.15.113:0/139384682 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f317801a020 con 0x7f318806ff20 2024-03-04T08:22:37.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.829+0000 7f3185600640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3164066f60 0x7f3164069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:37.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.829+0000 7f3185600640 1 -- 172.21.15.113:0/139384682 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f31780857a0 con 0x7f318806ff20 2024-03-04T08:22:37.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.829+0000 7f317fe00640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3164066f60 0x7f3164069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:37.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.829+0000 7f317fe00640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3164066f60 0x7f3164069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3188156820 tx=0x7f3170009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:37.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:37.841+0000 7f3185600640 1 -- 172.21.15.113:0/139384682 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3178021070 con 0x7f318806ff20 2024-03-04T08:22:38.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.025+0000 7f318f600640 1 -- 172.21.15.113:0/139384682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31880cef50 con 0x7f318806ff20 2024-03-04T08:22:38.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f3185600640 1 -- 172.21.15.113:0/139384682 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f3178055730 con 0x7f318806ff20 2024-03-04T08:22:38.027 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:38.027 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:38.027 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:38.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 -- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3164066f60 msgr2=0x7f3164069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:38.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3164066f60 0x7f3164069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3188156820 tx=0x7f3170009190 comp rx=0 tx=0).stop 2024-03-04T08:22:38.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 -- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f318806ff20 msgr2=0x7f31881407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:38.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f318806ff20 0x7f31881407d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f317800c390 tx=0x7f317800c850 comp rx=0 tx=0).stop 2024-03-04T08:22:38.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 -- 172.21.15.113:0/139384682 shutdown_connections 2024-03-04T08:22:38.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3164066f60 0x7f3164069420 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:38.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3188158010 0x7f31881f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:38.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f31880ce120 0x7f31881430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:38.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 --2- 172.21.15.113:0/139384682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f318806ff20 0x7f31881407d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:38.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 -- 172.21.15.113:0/139384682 >> 172.21.15.113:0/139384682 conn(0x7f31880734d0 msgr2=0x7f3188155a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:38.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 -- 172.21.15.113:0/139384682 shutdown_connections 2024-03-04T08:22:38.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:38.029+0000 7f318f600640 1 -- 172.21.15.113:0/139384682 wait complete. 2024-03-04T08:22:38.048 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:40.049 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:40.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2eeea00640 1 Processor -- start 2024-03-04T08:22:40.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2eeea00640 1 -- start start 2024-03-04T08:22:40.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2eeea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2ee80ce2d0 0x7f2ee815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:40.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2eeea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee806c210 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2eeea00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee806c4f0 con 0x7f2ee806de80 2024-03-04T08:22:40.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2eeea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee815bbf0 con 0x7f2ee80ce2d0 2024-03-04T08:22:40.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2ee7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2ee80ce2d0 0x7f2ee815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:40.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2ee7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2ee80ce2d0 0x7f2ee815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46344/0 (socket says 172.21.15.113:46344) 2024-03-04T08:22:40.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2edf400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2ee80ce6b0 0x7f2ee806f010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59584/0 (socket says 172.21.15.113:59584) 2024-03-04T08:22:40.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2ee7e00640 1 -- 172.21.15.113:0/1453036622 learned_addr learned my addr 172.21.15.113:0/1453036622 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:40.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2ee7e00640 1 -- 172.21.15.113:0/1453036622 >> v1:172.21.15.136:6789/0 conn(0x7f2ee806de80 legacy=0x7f2ee8154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:40.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2ee7e00640 1 -- 172.21.15.113:0/1453036622 >> v1:172.21.15.113:6789/0 conn(0x7f2ee80ce6b0 legacy=0x7f2ee806f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:40.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.197+0000 7f2ee7e00640 1 -- 172.21.15.113:0/1453036622 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee806e530 con 0x7f2ee80ce2d0 2024-03-04T08:22:40.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2ee7e00640 1 --2- 172.21.15.113:0/1453036622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce2d0 0x7f2ee815b6b0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009870 tx=0x7f2ed001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238c05d5ed7697f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:40.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2ee7400640 1 -- 172.21.15.113:0/1453036622 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00b5070 con 0x7f2ee80ce2d0 2024-03-04T08:22:40.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2ee7400640 1 -- 172.21.15.113:0/1453036622 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed00a8a50 con 0x7f2ee80ce2d0 2024-03-04T08:22:40.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2ee7400640 1 -- 172.21.15.113:0/1453036622 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00a8e20 con 0x7f2ee80ce2d0 2024-03-04T08:22:40.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2eeea00640 1 -- 172.21.15.113:0/1453036622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce2d0 msgr2=0x7f2ee815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:40.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2eeea00640 1 --2- 172.21.15.113:0/1453036622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce2d0 0x7f2ee815b6b0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009870 tx=0x7f2ed001ed50 comp rx=0 tx=0).stop 2024-03-04T08:22:40.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2eeea00640 1 -- 172.21.15.113:0/1453036622 shutdown_connections 2024-03-04T08:22:40.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2eeea00640 1 --2- 172.21.15.113:0/1453036622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce2d0 0x7f2ee815b6b0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:40.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2eeea00640 1 -- 172.21.15.113:0/1453036622 >> 172.21.15.113:0/1453036622 conn(0x7f2ee80709a0 msgr2=0x7f2ee8070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:40.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2eeea00640 1 -- 172.21.15.113:0/1453036622 shutdown_connections 2024-03-04T08:22:40.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.201+0000 7f2eeea00640 1 -- 172.21.15.113:0/1453036622 wait complete. 2024-03-04T08:22:40.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 Processor -- start 2024-03-04T08:22:40.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 -- start start 2024-03-04T08:22:40.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ee806de80 0x7f2ee814a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:40.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ee80ce2d0 0x7f2ee81f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:40.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce6b0 0x7f2ee8158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:40.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee806c7d0 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee806eb80 con 0x7f2ee806de80 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee806f720 con 0x7f2ee80ce2d0 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce6b0 0x7f2ee8158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce6b0 0x7f2ee8158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46360/0 (socket says 172.21.15.113:46360) 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeca00640 1 -- 172.21.15.113:0/3469977445 learned_addr learned my addr 172.21.15.113:0/3469977445 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2ee7e00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ee806de80 0x7f2ee814a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2edf400640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ee80ce2d0 0x7f2ee81f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeca00640 1 -- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ee80ce2d0 msgr2=0x7f2ee81f1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeca00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ee80ce2d0 0x7f2ee81f1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeca00640 1 -- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ee806de80 msgr2=0x7f2ee814a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeca00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ee806de80 0x7f2ee814a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeca00640 1 -- 172.21.15.113:0/3469977445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee81506c0 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeca00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce6b0 0x7f2ee8158aa0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000c770 tx=0x7f2ee000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:40.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2ee4c00640 1 -- 172.21.15.113:0/3469977445 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0003b80 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2ee4c00640 1 -- 172.21.15.113:0/3469977445 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee0003d20 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 -- 172.21.15.113:0/3469977445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee81503e0 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2ee4c00640 1 -- 172.21.15.113:0/3469977445 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000aa90 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 -- 172.21.15.113:0/3469977445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee81f4f40 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.205+0000 7f2eeea00640 1 -- 172.21.15.113:0/3469977445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee8075c00 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.209+0000 7f2ee4c00640 1 -- 172.21.15.113:0/3469977445 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ee000ac30 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.209+0000 7f2ee4c00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2ec4066f60 0x7f2ec4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:40.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.209+0000 7f2ee4c00640 1 -- 172.21.15.113:0/3469977445 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f2ee0085990 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.209+0000 7f2ee7e00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2ec4066f60 0x7f2ec4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:40.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.209+0000 7f2ee7e00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2ec4066f60 0x7f2ec4069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00b2000 tx=0x7f2ed0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:40.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.213+0000 7f2ee4c00640 1 -- 172.21.15.113:0/3469977445 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2ee0055ee0 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.397+0000 7f2eeea00640 1 -- 172.21.15.113:0/3469977445 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ee81f0f50 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2ee4c00640 1 -- 172.21.15.113:0/3469977445 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f2ee00557e0 con 0x7f2ee80ce6b0 2024-03-04T08:22:40.399 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:40.399 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:40.400 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 -- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2ec4066f60 msgr2=0x7f2ec4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2ec4066f60 0x7f2ec4069420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00b2000 tx=0x7f2ed0009040 comp rx=0 tx=0).stop 2024-03-04T08:22:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 -- 172.21.15.113:0/3469977445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce6b0 msgr2=0x7f2ee8158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce6b0 0x7f2ee8158aa0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000c770 tx=0x7f2ee000cc30 comp rx=0 tx=0).stop 2024-03-04T08:22:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 -- 172.21.15.113:0/3469977445 shutdown_connections 2024-03-04T08:22:40.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2ec4066f60 0x7f2ec4069420 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:40.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ee80ce6b0 0x7f2ee8158aa0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:40.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ee80ce2d0 0x7f2ee81f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:40.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 --2- 172.21.15.113:0/3469977445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ee806de80 0x7f2ee814a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:40.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 -- 172.21.15.113:0/3469977445 >> 172.21.15.113:0/3469977445 conn(0x7f2ee80709a0 msgr2=0x7f2ee815ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:40.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 -- 172.21.15.113:0/3469977445 shutdown_connections 2024-03-04T08:22:40.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:40.401+0000 7f2eeea00640 1 -- 172.21.15.113:0/3469977445 wait complete. 2024-03-04T08:22:40.420 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:42.421 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:42.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9ce600640 1 Processor -- start 2024-03-04T08:22:42.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9ce600640 1 -- start start 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9ce600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc9c814e4c0 0x7fc9c8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9ce600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9c80c22e0 0x7fc9c80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9ce600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c80c27f0 con 0x7fc9c80c2d80 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9ce600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c815ad60 con 0x7fc9c80c22e0 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9ce600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c80c2a90 con 0x7fc9c814e4c0 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9c6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc9c80c22e0 0x7fc9c80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9c7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc9c80c2d80 0x7fc9c815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59594/0 (socket says 172.21.15.113:59594) 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9c7e00640 1 -- 172.21.15.113:0/63479250 learned_addr learned my addr 172.21.15.113:0/63479250 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9c7400640 1 --2- 172.21.15.113:0/63479250 >> v2:172.21.15.136:3300/0 conn(0x7fc9c814e4c0 0x7fc9c8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9c6a00640 1 -- 172.21.15.113:0/63479250 >> v2:172.21.15.136:3300/0 conn(0x7fc9c814e4c0 msgr2=0x7fc9c8072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:42.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9c6a00640 1 --2- 172.21.15.113:0/63479250 >> v2:172.21.15.136:3300/0 conn(0x7fc9c814e4c0 0x7fc9c8072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:42.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9c6a00640 1 -- 172.21.15.113:0/63479250 >> v1:172.21.15.113:6789/0 conn(0x7fc9c80c2d80 legacy=0x7fc9c815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:42.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9c6a00640 1 -- 172.21.15.113:0/63479250 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c8070c60 con 0x7fc9c80c22e0 2024-03-04T08:22:42.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.569+0000 7fc9c6000640 1 -- 172.21.15.113:0/63479250 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030801844 0 0) 0x7fc9c80c27f0 con 0x7fc9c80c2d80 2024-03-04T08:22:42.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9c6a00640 1 --2- 172.21.15.113:0/63479250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 0x7fc9c80731c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc9c806f690 tx=0x7fc9b001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31187c6d38c0742a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:42.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9c6000640 1 -- 172.21.15.113:0/63479250 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b00b5070 con 0x7fc9c80c22e0 2024-03-04T08:22:42.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9c6000640 1 -- 172.21.15.113:0/63479250 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b00a8970 con 0x7fc9c80c22e0 2024-03-04T08:22:42.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9c6000640 1 -- 172.21.15.113:0/63479250 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b00a8d40 con 0x7fc9c80c22e0 2024-03-04T08:22:42.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9ce600640 1 -- 172.21.15.113:0/63479250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 msgr2=0x7fc9c80731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:42.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9ce600640 1 --2- 172.21.15.113:0/63479250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 0x7fc9c80731c0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc9c806f690 tx=0x7fc9b001ec70 comp rx=0 tx=0).stop 2024-03-04T08:22:42.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9ce600640 1 -- 172.21.15.113:0/63479250 shutdown_connections 2024-03-04T08:22:42.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9ce600640 1 --2- 172.21.15.113:0/63479250 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 0x7fc9c80731c0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:42.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9ce600640 1 --2- 172.21.15.113:0/63479250 >> v2:172.21.15.136:3300/0 conn(0x7fc9c814e4c0 0x7fc9c8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:42.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9ce600640 1 -- 172.21.15.113:0/63479250 >> 172.21.15.113:0/63479250 conn(0x7fc9c806e820 msgr2=0x7fc9c806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:42.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9ce600640 1 -- 172.21.15.113:0/63479250 shutdown_connections 2024-03-04T08:22:42.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.573+0000 7fc9ce600640 1 -- 172.21.15.113:0/63479250 wait complete. 2024-03-04T08:22:42.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9ce600640 1 Processor -- start 2024-03-04T08:22:42.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9ce600640 1 -- start start 2024-03-04T08:22:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9ce600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 0x7fc9c8152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9ce600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9c80c2d80 0x7fc9c81513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9ce600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9c814e4c0 0x7fc9c81f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9ce600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c8071500 con 0x7fc9c80c22e0 2024-03-04T08:22:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9ce600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c8071220 con 0x7fc9c80c2d80 2024-03-04T08:22:42.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9ce600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c814e8a0 con 0x7fc9c814e4c0 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 0x7fc9c8152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 0x7fc9c8152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46364/0 (socket says 172.21.15.113:46364) 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c7400640 1 -- 172.21.15.113:0/3517016474 learned_addr learned my addr 172.21.15.113:0/3517016474 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c7e00640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9c814e4c0 0x7fc9c81f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c6a00640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9c80c2d80 0x7fc9c81513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c7400640 1 -- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9c814e4c0 msgr2=0x7fc9c81f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c7400640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9c814e4c0 0x7fc9c81f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c7400640 1 -- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9c80c2d80 msgr2=0x7fc9c81513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c7400640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9c80c2d80 0x7fc9c81513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c7400640 1 -- 172.21.15.113:0/3517016474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c8153760 con 0x7fc9c80c22e0 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9c7400640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 0x7fc9c8152f20 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800e870 tx=0x7fc9b800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9bf400640 1 -- 172.21.15.113:0/3517016474 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8017070 con 0x7fc9c80c22e0 2024-03-04T08:22:42.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9bf400640 1 -- 172.21.15.113:0/3517016474 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b800c940 con 0x7fc9c80c22e0 2024-03-04T08:22:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9bf400640 1 -- 172.21.15.113:0/3517016474 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b800ccf0 con 0x7fc9c80c22e0 2024-03-04T08:22:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9ce600640 1 -- 172.21.15.113:0/3517016474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c8139990 con 0x7fc9c80c22e0 2024-03-04T08:22:42.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.577+0000 7fc9ce600640 1 -- 172.21.15.113:0/3517016474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c8139f50 con 0x7fc9c80c22e0 2024-03-04T08:22:42.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.581+0000 7fc9ce600640 1 -- 172.21.15.113:0/3517016474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c8075c00 con 0x7fc9c80c22e0 2024-03-04T08:22:42.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.581+0000 7fc9bf400640 1 -- 172.21.15.113:0/3517016474 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9b801a070 con 0x7fc9c80c22e0 2024-03-04T08:22:42.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.581+0000 7fc9bf400640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc9a4066f90 0x7fc9a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:42.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.581+0000 7fc9bf400640 1 -- 172.21.15.113:0/3517016474 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fc9b8085bd0 con 0x7fc9c80c22e0 2024-03-04T08:22:42.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.581+0000 7fc9c6a00640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc9a4066f90 0x7fc9a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:42.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.581+0000 7fc9c6a00640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc9a4066f90 0x7fc9a4069450 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00b2000 tx=0x7fc9b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:42.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.585+0000 7fc9bf400640 1 -- 172.21.15.113:0/3517016474 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc9b8056240 con 0x7fc9c80c22e0 2024-03-04T08:22:42.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 -- 172.21.15.113:0/3517016474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9c8155190 con 0x7fc9c80c22e0 2024-03-04T08:22:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9bf400640 1 -- 172.21.15.113:0/3517016474 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7fc9b8055b40 con 0x7fc9c80c22e0 2024-03-04T08:22:42.777 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:42.777 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:42.777 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 -- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc9a4066f90 msgr2=0x7fc9a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc9a4066f90 0x7fc9a4069450 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00b2000 tx=0x7fc9b0008040 comp rx=0 tx=0).stop 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 -- 172.21.15.113:0/3517016474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 msgr2=0x7fc9c8152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 0x7fc9c8152f20 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fc9b800e870 tx=0x7fc9b800ec70 comp rx=0 tx=0).stop 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 -- 172.21.15.113:0/3517016474 shutdown_connections 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc9c814e4c0 0x7fc9c81f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9c80c2d80 0x7fc9c81513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc9a4066f90 0x7fc9a4069450 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 --2- 172.21.15.113:0/3517016474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc9c80c22e0 0x7fc9c8152f20 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 -- 172.21.15.113:0/3517016474 >> 172.21.15.113:0/3517016474 conn(0x7fc9c806e820 msgr2=0x7fc9c80c7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 -- 172.21.15.113:0/3517016474 shutdown_connections 2024-03-04T08:22:42.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:42.777+0000 7fc9ce600640 1 -- 172.21.15.113:0/3517016474 wait complete. 2024-03-04T08:22:42.798 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:22:42.799 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 179, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 179, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4247": { "addr": "172.21.15.136:6839/1409794802", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6838", "nonce": 1409794802, "type": "v2" }, { "addr": "172.21.15.136:6839", "nonce": 1409794802, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4247, "incarnation": 173, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 729 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8388": { "addr": "172.21.15.113:6833/2999156262", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6832", "nonce": 2999156262, "type": "v2" }, { "addr": "172.21.15.113:6833", "nonce": 2999156262, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8388, "incarnation": 171, "join_fscid": -1, "name": "c", "rank": 2, "state": "up:stopping", "state_seq": 28 } }, "last_failure": 0, "last_failure_osd_epoch": 82, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:22:12.512430+0000", "qdb_cluster": [ 4247, 8032 ], "qdb_leader": 4247, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4247, "mds_1": 8032, "mds_2": 8388 } } } ], "standbys": [ { "addr": "172.21.15.136:6837/2228772746", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2228772746, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2228772746, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 4244, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2024-03-04T08:22:44.800 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:44.828 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:22:44.849 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:22:44.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb93ce00640 1 Processor -- start 2024-03-04T08:22:44.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb93ce00640 1 -- start start 2024-03-04T08:22:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb93ce00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb938154d00 0x7fb93815b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb93ce00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9380c3bd0 0x7fb9380c4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb93ce00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb93806d480 con 0x7fb93806f5b0 2024-03-04T08:22:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb93ce00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93815ba50 con 0x7fb9380c3bd0 2024-03-04T08:22:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb93ce00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9380c4540 con 0x7fb938154d00 2024-03-04T08:22:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb936800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9380c3bd0 0x7fb9380c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb935e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb938154d00 0x7fb93815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb935400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb93806f5b0 0x7fb9380ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59600/0 (socket says 172.21.15.113:59600) 2024-03-04T08:22:44.949 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb935400640 1 -- 172.21.15.113:0/490241918 learned_addr learned my addr 172.21.15.113:0/490241918 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:22:44.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb936800640 1 -- 172.21.15.113:0/490241918 >> v2:172.21.15.136:3300/0 conn(0x7fb938154d00 msgr2=0x7fb93815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:44.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb936800640 1 --2- 172.21.15.113:0/490241918 >> v2:172.21.15.136:3300/0 conn(0x7fb938154d00 0x7fb93815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:44.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb936800640 1 -- 172.21.15.113:0/490241918 >> v1:172.21.15.113:6789/0 conn(0x7fb93806f5b0 legacy=0x7fb9380ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:44.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.949+0000 7fb936800640 1 -- 172.21.15.113:0/490241918 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93806c590 con 0x7fb9380c3bd0 2024-03-04T08:22:44.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb936800640 1 --2- 172.21.15.113:0/490241918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 0x7fb9380c4000 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb938067d90 tx=0x7fb93001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7572bc0c2966402f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:44.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb934a00640 1 -- 172.21.15.113:0/490241918 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9300b0070 con 0x7fb9380c3bd0 2024-03-04T08:22:44.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb934a00640 1 -- 172.21.15.113:0/490241918 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9300ab600 con 0x7fb9380c3bd0 2024-03-04T08:22:44.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb934a00640 1 -- 172.21.15.113:0/490241918 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9300ab9d0 con 0x7fb9380c3bd0 2024-03-04T08:22:44.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb93ce00640 1 -- 172.21.15.113:0/490241918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 msgr2=0x7fb9380c4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:44.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb93ce00640 1 --2- 172.21.15.113:0/490241918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 0x7fb9380c4000 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb938067d90 tx=0x7fb93001c480 comp rx=0 tx=0).stop 2024-03-04T08:22:44.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb93ce00640 1 -- 172.21.15.113:0/490241918 shutdown_connections 2024-03-04T08:22:44.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb93ce00640 1 --2- 172.21.15.113:0/490241918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 0x7fb9380c4000 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:44.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb93ce00640 1 --2- 172.21.15.113:0/490241918 >> v2:172.21.15.136:3300/0 conn(0x7fb938154d00 0x7fb93815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:44.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb93ce00640 1 -- 172.21.15.113:0/490241918 >> 172.21.15.113:0/490241918 conn(0x7fb938066db0 msgr2=0x7fb9380671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:44.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb93ce00640 1 -- 172.21.15.113:0/490241918 shutdown_connections 2024-03-04T08:22:44.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.953+0000 7fb93ce00640 1 -- 172.21.15.113:0/490241918 wait complete. 2024-03-04T08:22:44.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 Processor -- start 2024-03-04T08:22:44.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 -- start start 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb93806f5b0 0x7fb938159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 0x7fb9381f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb938154d00 0x7fb9381f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381550e0 con 0x7fb9380c3bd0 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93806d110 con 0x7fb93806f5b0 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93806c170 con 0x7fb938154d00 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb935400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 0x7fb9381f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb935400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 0x7fb9381f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46386/0 (socket says 172.21.15.113:46386) 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb935400640 1 -- 172.21.15.113:0/1772590028 learned_addr learned my addr 172.21.15.113:0/1772590028 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb935e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb93806f5b0 0x7fb938159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb936800640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb938154d00 0x7fb9381f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:44.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb935400640 1 -- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb938154d00 msgr2=0x7fb9381f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:44.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb935400640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb938154d00 0x7fb9381f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:44.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb935400640 1 -- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb93806f5b0 msgr2=0x7fb938159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:44.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb935400640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb93806f5b0 0x7fb938159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:44.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb935400640 1 -- 172.21.15.113:0/1772590028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9381f6c60 con 0x7fb9380c3bd0 2024-03-04T08:22:44.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb935400640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 0x7fb9381f7110 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fb92400c450 tx=0x7fb92400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:44.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb92e000640 1 -- 172.21.15.113:0/1772590028 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924017070 con 0x7fb9380c3bd0 2024-03-04T08:22:44.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb92e000640 1 -- 172.21.15.113:0/1772590028 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb924005cf0 con 0x7fb9380c3bd0 2024-03-04T08:22:44.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb92e000640 1 -- 172.21.15.113:0/1772590028 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924007290 con 0x7fb9380c3bd0 2024-03-04T08:22:44.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 -- 172.21.15.113:0/1772590028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9380b1d20 con 0x7fb9380c3bd0 2024-03-04T08:22:44.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 -- 172.21.15.113:0/1772590028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9380b22e0 con 0x7fb9380c3bd0 2024-03-04T08:22:44.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.957+0000 7fb93ce00640 1 -- 172.21.15.113:0/1772590028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb938075c00 con 0x7fb9380c3bd0 2024-03-04T08:22:44.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.961+0000 7fb92e000640 1 -- 172.21.15.113:0/1772590028 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb92401a070 con 0x7fb9380c3bd0 2024-03-04T08:22:44.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.961+0000 7fb92e000640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb918066f60 0x7fb918069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:44.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.961+0000 7fb92e000640 1 -- 172.21.15.113:0/1772590028 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fb924084c70 con 0x7fb9380c3bd0 2024-03-04T08:22:44.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.961+0000 7fb935e00640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb918066f60 0x7fb918069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:44.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.961+0000 7fb935e00640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb918066f60 0x7fb918069420 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb928011340 tx=0x7fb92800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:44.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:44.965+0000 7fb92e000640 1 -- 172.21.15.113:0/1772590028 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb924054ca0 con 0x7fb9380c3bd0 2024-03-04T08:22:45.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 -- 172.21.15.113:0/1772590028 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9380717b0 con 0x7fb9380c3bd0 2024-03-04T08:22:45.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb92e000640 1 -- 172.21.15.113:0/1772590028 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7fb924058d40 con 0x7fb9380c3bd0 2024-03-04T08:22:45.152 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:45.152 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:45.152 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 -- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb918066f60 msgr2=0x7fb918069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb918066f60 0x7fb918069420 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb928011340 tx=0x7fb92800b040 comp rx=0 tx=0).stop 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 -- 172.21.15.113:0/1772590028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 msgr2=0x7fb9381f7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 0x7fb9381f7110 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fb92400c450 tx=0x7fb92400c910 comp rx=0 tx=0).stop 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 -- 172.21.15.113:0/1772590028 shutdown_connections 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb938154d00 0x7fb9381f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb918066f60 0x7fb918069420 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9380c3bd0 0x7fb9381f7110 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 --2- 172.21.15.113:0/1772590028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb93806f5b0 0x7fb938159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 -- 172.21.15.113:0/1772590028 >> 172.21.15.113:0/1772590028 conn(0x7fb938066db0 msgr2=0x7fb9380ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 -- 172.21.15.113:0/1772590028 shutdown_connections 2024-03-04T08:22:45.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:45.153+0000 7fb93ce00640 1 -- 172.21.15.113:0/1772590028 wait complete. 2024-03-04T08:22:45.173 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:47.175 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:47.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.321+0000 7fa5dbe00640 1 Processor -- start 2024-03-04T08:22:47.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.321+0000 7fa5dbe00640 1 -- start start 2024-03-04T08:22:47.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5d4152ab0 0x7fa5d41592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:47.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fa5d40c8810 0x7fa5d40cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:47.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5dbe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40c7410 con 0x7fa5d40c81c0 2024-03-04T08:22:47.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5dbe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d4158e50 con 0x7fa5d4152ab0 2024-03-04T08:22:47.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5dbe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d4159820 con 0x7fa5d40c8810 2024-03-04T08:22:47.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5d9400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5d4152ab0 0x7fa5d41592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:47.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5d9400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5d4152ab0 0x7fa5d41592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36026/0 (socket says 172.21.15.113:36026) 2024-03-04T08:22:47.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5d9400640 1 -- 172.21.15.113:0/3744402043 learned_addr learned my addr 172.21.15.113:0/3744402043 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:47.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5d8a00640 1 --1- 172.21.15.113:0/3744402043 >> v1:172.21.15.113:6789/0 conn(0x7fa5d40c81c0 0x7fa5d415c7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46552/0 (socket says 172.21.15.113:46552) 2024-03-04T08:22:47.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5d9e00640 1 --2- 172.21.15.113:0/3744402043 >> v2:172.21.15.136:3300/0 conn(0x7fa5d40c8810 0x7fa5d40cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:47.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5d9400640 1 -- 172.21.15.113:0/3744402043 >> v2:172.21.15.136:3300/0 conn(0x7fa5d40c8810 msgr2=0x7fa5d40cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:47.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5d9400640 1 --2- 172.21.15.113:0/3744402043 >> v2:172.21.15.136:3300/0 conn(0x7fa5d40c8810 0x7fa5d40cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:47.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5d9400640 1 -- 172.21.15.113:0/3744402043 >> v1:172.21.15.113:6789/0 conn(0x7fa5d40c81c0 legacy=0x7fa5d415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:47.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5d9400640 1 -- 172.21.15.113:0/3744402043 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40c76b0 con 0x7fa5d4152ab0 2024-03-04T08:22:47.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.325+0000 7fa5d3e00640 1 -- 172.21.15.113:0/3744402043 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095497008 0 0) 0x7fa5d40c7410 con 0x7fa5d40c81c0 2024-03-04T08:22:47.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5d9400640 1 --2- 172.21.15.113:0/3744402043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d4152ab0 0x7fa5d41592e0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40c6a60 tx=0x7fa5c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca7048bff9cbc93 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:47.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5d3e00640 1 -- 172.21.15.113:0/3744402043 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80b0070 con 0x7fa5d4152ab0 2024-03-04T08:22:47.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5d3e00640 1 -- 172.21.15.113:0/3744402043 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c80ab600 con 0x7fa5d4152ab0 2024-03-04T08:22:47.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5d3e00640 1 -- 172.21.15.113:0/3744402043 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80ab9d0 con 0x7fa5d4152ab0 2024-03-04T08:22:47.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 -- 172.21.15.113:0/3744402043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d4152ab0 msgr2=0x7fa5d41592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:47.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 --2- 172.21.15.113:0/3744402043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d4152ab0 0x7fa5d41592e0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40c6a60 tx=0x7fa5c801c520 comp rx=0 tx=0).stop 2024-03-04T08:22:47.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 -- 172.21.15.113:0/3744402043 shutdown_connections 2024-03-04T08:22:47.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 --2- 172.21.15.113:0/3744402043 >> v2:172.21.15.136:3300/0 conn(0x7fa5d40c8810 0x7fa5d40cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:47.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 --2- 172.21.15.113:0/3744402043 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d4152ab0 0x7fa5d41592e0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:47.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 -- 172.21.15.113:0/3744402043 >> 172.21.15.113:0/3744402043 conn(0x7fa5d406b060 msgr2=0x7fa5d406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:47.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 -- 172.21.15.113:0/3744402043 shutdown_connections 2024-03-04T08:22:47.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 -- 172.21.15.113:0/3744402043 wait complete. 2024-03-04T08:22:47.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 Processor -- start 2024-03-04T08:22:47.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 -- start start 2024-03-04T08:22:47.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c81c0 0x7fa5d414dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:47.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5d40c8810 0x7fa5d4148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:47.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5d4152ab0 0x7fa5d4157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:47.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d41659c0 con 0x7fa5d40c81c0 2024-03-04T08:22:47.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40c7950 con 0x7fa5d40c8810 2024-03-04T08:22:47.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5dbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4165b40 con 0x7fa5d4152ab0 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.329+0000 7fa5d9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c81c0 0x7fa5d414dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c81c0 0x7fa5d414dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36034/0 (socket says 172.21.15.113:36034) 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d9400640 1 -- 172.21.15.113:0/2456845835 learned_addr learned my addr 172.21.15.113:0/2456845835 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d8a00640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5d40c8810 0x7fa5d4148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d9e00640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5d4152ab0 0x7fa5d4157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d9400640 1 -- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5d4152ab0 msgr2=0x7fa5d4157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d9400640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5d4152ab0 0x7fa5d4157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d9400640 1 -- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5d40c8810 msgr2=0x7fa5d4148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d9400640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5d40c8810 0x7fa5d4148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d9400640 1 -- 172.21.15.113:0/2456845835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d41519a0 con 0x7fa5d40c81c0 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d9400640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c81c0 0x7fa5d414dbb0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa5c801ca00 tx=0x7fa5c80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d1600640 1 -- 172.21.15.113:0/2456845835 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80b0040 con 0x7fa5d40c81c0 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d1600640 1 -- 172.21.15.113:0/2456845835 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c80b9070 con 0x7fa5d40c81c0 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d1600640 1 -- 172.21.15.113:0/2456845835 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80b4880 con 0x7fa5d40c81c0 2024-03-04T08:22:47.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5dbe00640 1 -- 172.21.15.113:0/2456845835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d4151c80 con 0x7fa5d40c81c0 2024-03-04T08:22:47.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5dbe00640 1 -- 172.21.15.113:0/2456845835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d41f7210 con 0x7fa5d40c81c0 2024-03-04T08:22:47.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5dbe00640 1 -- 172.21.15.113:0/2456845835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d4075c00 con 0x7fa5d40c81c0 2024-03-04T08:22:47.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d1600640 1 -- 172.21.15.113:0/2456845835 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5c80b4a20 con 0x7fa5d40c81c0 2024-03-04T08:22:47.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d1600640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b4066f60 0x7fa5b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:47.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d1600640 1 -- 172.21.15.113:0/2456845835 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7fa5c811f6e0 con 0x7fa5d40c81c0 2024-03-04T08:22:47.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.333+0000 7fa5d8a00640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b4066f60 0x7fa5b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:47.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.337+0000 7fa5d8a00640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b4066f60 0x7fa5b4069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa5d41f1410 tx=0x7fa5c0007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:47.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.341+0000 7fa5d1600640 1 -- 172.21.15.113:0/2456845835 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa5c8016110 con 0x7fa5d40c81c0 2024-03-04T08:22:47.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 -- 172.21.15.113:0/2456845835 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5d41563d0 con 0x7fa5d40c81c0 2024-03-04T08:22:47.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5d1600640 1 -- 172.21.15.113:0/2456845835 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7fa5c80efcc0 con 0x7fa5d40c81c0 2024-03-04T08:22:47.528 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:47.528 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:47.528 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 -- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b4066f60 msgr2=0x7fa5b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b4066f60 0x7fa5b4069420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa5d41f1410 tx=0x7fa5c0007d60 comp rx=0 tx=0).stop 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 -- 172.21.15.113:0/2456845835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c81c0 msgr2=0x7fa5d414dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c81c0 0x7fa5d414dbb0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa5c801ca00 tx=0x7fa5c80192c0 comp rx=0 tx=0).stop 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 -- 172.21.15.113:0/2456845835 shutdown_connections 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa5d4152ab0 0x7fa5d4157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa5d40c8810 0x7fa5d4148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa5b4066f60 0x7fa5b4069420 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 --2- 172.21.15.113:0/2456845835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d40c81c0 0x7fa5d414dbb0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 -- 172.21.15.113:0/2456845835 >> 172.21.15.113:0/2456845835 conn(0x7fa5d406b060 msgr2=0x7fa5d40c4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 -- 172.21.15.113:0/2456845835 shutdown_connections 2024-03-04T08:22:47.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:47.529+0000 7fa5dbe00640 1 -- 172.21.15.113:0/2456845835 wait complete. 2024-03-04T08:22:47.549 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:49.550 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:49.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3e600640 1 Processor -- start 2024-03-04T08:22:49.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3e600640 1 -- start start 2024-03-04T08:22:49.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3e600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0c38151840 0x7f0c380c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:49.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3e600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c3806d140 0x7f0c38157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:49.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3e600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c38157da0 con 0x7f0c38151c20 2024-03-04T08:22:49.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3e600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c380c3440 con 0x7f0c38151840 2024-03-04T08:22:49.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3e600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c38158080 con 0x7f0c3806d140 2024-03-04T08:22:49.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3d200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0c38151840 0x7f0c380c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3dc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c3806d140 0x7f0c38157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c2fe00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f0c38151c20 0x7f0c3806c170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:40712/0 (socket says 172.21.15.113:40712) 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3d200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0c38151840 0x7f0c380c1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:50394/0 (socket says 172.21.15.113:50394) 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c2fe00640 1 -- 172.21.15.113:0/1333188860 learned_addr learned my addr 172.21.15.113:0/1333188860 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3dc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c3806d140 0x7f0c38157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:50094/0 (socket says 172.21.15.113:50094) 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c37e00640 1 -- 172.21.15.113:0/1333188860 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513919015 0 0) 0x7f0c38157da0 con 0x7f0c38151c20 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c37e00640 1 -- 172.21.15.113:0/1333188860 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c1c003530 con 0x7f0c38151c20 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3d200640 1 -- 172.21.15.113:0/1333188860 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c3806d140 msgr2=0x7f0c38157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3d200640 1 --2- 172.21.15.113:0/1333188860 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c3806d140 0x7f0c38157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3d200640 1 -- 172.21.15.113:0/1333188860 >> v1:172.21.15.136:6789/0 conn(0x7f0c38151c20 legacy=0x7f0c3806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3d200640 1 -- 172.21.15.113:0/1333188860 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c38158360 con 0x7f0c38151840 2024-03-04T08:22:49.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.697+0000 7f0c3dc00640 1 --2- 172.21.15.113:0/1333188860 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c3806d140 0x7f0c38157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:22:49.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c3d200640 1 --2- 172.21.15.113:0/1333188860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c38151840 0x7f0c380c1f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0c280097c0 tx=0x7f0c2801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d25501ae4c747f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:49.701 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c37e00640 1 -- 172.21.15.113:0/1333188860 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c280b5070 con 0x7f0c38151840 2024-03-04T08:22:49.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c37e00640 1 -- 172.21.15.113:0/1333188860 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c280a8990 con 0x7f0c38151840 2024-03-04T08:22:49.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c37e00640 1 -- 172.21.15.113:0/1333188860 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c280a8d40 con 0x7f0c38151840 2024-03-04T08:22:49.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c3e600640 1 -- 172.21.15.113:0/1333188860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c38151840 msgr2=0x7f0c380c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:49.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c3e600640 1 --2- 172.21.15.113:0/1333188860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c38151840 0x7f0c380c1f50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0c280097c0 tx=0x7f0c2801ec90 comp rx=0 tx=0).stop 2024-03-04T08:22:49.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c3e600640 1 -- 172.21.15.113:0/1333188860 shutdown_connections 2024-03-04T08:22:49.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c3e600640 1 --2- 172.21.15.113:0/1333188860 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c3806d140 0x7f0c38157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:49.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c3e600640 1 --2- 172.21.15.113:0/1333188860 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c38151840 0x7f0c380c1f50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:49.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c3e600640 1 -- 172.21.15.113:0/1333188860 >> 172.21.15.113:0/1333188860 conn(0x7f0c380c9a30 msgr2=0x7f0c380c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:49.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c3e600640 1 -- 172.21.15.113:0/1333188860 shutdown_connections 2024-03-04T08:22:49.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.701+0000 7f0c3e600640 1 -- 172.21.15.113:0/1333188860 wait complete. 2024-03-04T08:22:49.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3e600640 1 Processor -- start 2024-03-04T08:22:49.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3e600640 1 -- start start 2024-03-04T08:22:49.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3e600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3806d140 0x7f0c38150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:49.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3e600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c38151840 0x7f0c380c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3e600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c38151c20 0x7f0c3813b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3e600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c38071db0 con 0x7f0c3806d140 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3e600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c38071ad0 con 0x7f0c38151840 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3e600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3806c880 con 0x7f0c38151c20 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3806d140 0x7f0c38150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3806d140 0x7f0c38150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36040/0 (socket says 172.21.15.113:36040) 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3d200640 1 -- 172.21.15.113:0/3330909971 learned_addr learned my addr 172.21.15.113:0/3330909971 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c2fe00640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c38151840 0x7f0c380c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3dc00640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c38151c20 0x7f0c3813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3d200640 1 -- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c38151c20 msgr2=0x7f0c3813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3d200640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c38151c20 0x7f0c3813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3d200640 1 -- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c38151840 msgr2=0x7f0c380c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3d200640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c38151840 0x7f0c380c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3d200640 1 -- 172.21.15.113:0/3330909971 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c380c8f30 con 0x7f0c3806d140 2024-03-04T08:22:49.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3d200640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3806d140 0x7f0c38150450 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0c2801c8c0 tx=0x7f0c2801ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:49.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c35600640 1 -- 172.21.15.113:0/3330909971 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c280b5040 con 0x7f0c3806d140 2024-03-04T08:22:49.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c35600640 1 -- 172.21.15.113:0/3330909971 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c280bf070 con 0x7f0c3806d140 2024-03-04T08:22:49.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c35600640 1 -- 172.21.15.113:0/3330909971 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c28004440 con 0x7f0c3806d140 2024-03-04T08:22:49.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3e600640 1 -- 172.21.15.113:0/3330909971 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3814b670 con 0x7f0c3806d140 2024-03-04T08:22:49.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.705+0000 7f0c3e600640 1 -- 172.21.15.113:0/3330909971 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c380c94b0 con 0x7f0c3806d140 2024-03-04T08:22:49.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.709+0000 7f0c3e600640 1 -- 172.21.15.113:0/3330909971 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c38075a80 con 0x7f0c3806d140 2024-03-04T08:22:49.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.709+0000 7f0c35600640 1 -- 172.21.15.113:0/3330909971 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c28003620 con 0x7f0c3806d140 2024-03-04T08:22:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.709+0000 7f0c35600640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c14066f60 0x7f0c14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.709+0000 7f0c35600640 1 -- 172.21.15.113:0/3330909971 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8147+0+0 (secure 0 0 0) 0x7f0c28122da0 con 0x7f0c3806d140 2024-03-04T08:22:49.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.709+0000 7f0c2fe00640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c14066f60 0x7f0c14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:49.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.709+0000 7f0c2fe00640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c14066f60 0x7f0c14069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0c20007fd0 tx=0x7f0c20009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:49.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.713+0000 7f0c35600640 1 -- 172.21.15.113:0/3330909971 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0c280187f0 con 0x7f0c3806d140 2024-03-04T08:22:49.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.897+0000 7f0c3e600640 1 -- 172.21.15.113:0/3330909971 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c380c80c0 con 0x7f0c3806d140 2024-03-04T08:22:49.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c35600640 1 -- 172.21.15.113:0/3330909971 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f0c280f34a0 con 0x7f0c3806d140 2024-03-04T08:22:49.899 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:49.900 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:12.512430+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8388":{"gid":8388,"name":"c","rank":2,"incarnation":171,"state":"up:stopping","state_seq":28,"addr":"172.21.15.113:6833/2999156262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":2999156262},{"type":"v1","addr":"172.21.15.113:6833","nonce":2999156262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:49.900 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 179 2024-03-04T08:22:49.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 -- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c14066f60 msgr2=0x7f0c14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:49.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c14066f60 0x7f0c14069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0c20007fd0 tx=0x7f0c20009040 comp rx=0 tx=0).stop 2024-03-04T08:22:49.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 -- 172.21.15.113:0/3330909971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3806d140 msgr2=0x7f0c38150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:49.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3806d140 0x7f0c38150450 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f0c2801c8c0 tx=0x7f0c2801ba30 comp rx=0 tx=0).stop 2024-03-04T08:22:49.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 -- 172.21.15.113:0/3330909971 shutdown_connections 2024-03-04T08:22:49.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0c38151c20 0x7f0c3813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:49.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c38151840 0x7f0c380c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:49.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0c14066f60 0x7f0c14069420 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:49.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 --2- 172.21.15.113:0/3330909971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c3806d140 0x7f0c38150450 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:49.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 -- 172.21.15.113:0/3330909971 >> 172.21.15.113:0/3330909971 conn(0x7f0c380c9a30 msgr2=0x7f0c38141000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:49.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 -- 172.21.15.113:0/3330909971 shutdown_connections 2024-03-04T08:22:49.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:49.901+0000 7f0c3e600640 1 -- 172.21.15.113:0/3330909971 wait complete. 2024-03-04T08:22:49.920 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:22:51.922 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b8a00640 1 Processor -- start 2024-03-04T08:22:52.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b8a00640 1 -- start start 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b8a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69b415b900 0x7f69b4163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b8a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b40c27f0 con 0x7f69b414e4c0 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b8a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b40c2a90 con 0x7f69b40c22e0 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b8a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b415ad60 con 0x7f69b415b900 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b1e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f69b414e4c0 0x7f69b4072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:46562/0 (socket says 172.21.15.113:46562) 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b1e00640 1 -- 172.21.15.113:0/1440118799 learned_addr learned my addr 172.21.15.113:0/1440118799 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b2800640 1 --2- 172.21.15.113:0/1440118799 >> v2:172.21.15.113:3300/0 conn(0x7f69b415b900 0x7f69b4163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b0a00640 1 -- 172.21.15.113:0/1440118799 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092815764 0 0) 0x7f69b40c27f0 con 0x7f69b414e4c0 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b0a00640 1 -- 172.21.15.113:0/1440118799 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6998003530 con 0x7f69b414e4c0 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b2800640 1 -- 172.21.15.113:0/1440118799 >> v1:172.21.15.136:6789/0 conn(0x7f69b40c22e0 legacy=0x7f69b415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:52.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b2800640 1 -- 172.21.15.113:0/1440118799 >> v1:172.21.15.113:6789/0 conn(0x7f69b414e4c0 legacy=0x7f69b4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.069+0000 7f69b2800640 1 -- 172.21.15.113:0/1440118799 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b4070c60 con 0x7f69b415b900 2024-03-04T08:22:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b2800640 1 --2- 172.21.15.113:0/1440118799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b415b900 0x7f69b4163100 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0097c0 tx=0x7f69ac01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545b32f28d702eb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b0a00640 1 -- 172.21.15.113:0/1440118799 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0b5070 con 0x7f69b415b900 2024-03-04T08:22:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b0a00640 1 -- 172.21.15.113:0/1440118799 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac0a8990 con 0x7f69b415b900 2024-03-04T08:22:52.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b0a00640 1 -- 172.21.15.113:0/1440118799 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0a8d60 con 0x7f69b415b900 2024-03-04T08:22:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b8a00640 1 -- 172.21.15.113:0/1440118799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b415b900 msgr2=0x7f69b4163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b8a00640 1 --2- 172.21.15.113:0/1440118799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b415b900 0x7f69b4163100 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0097c0 tx=0x7f69ac01ec90 comp rx=0 tx=0).stop 2024-03-04T08:22:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b8a00640 1 -- 172.21.15.113:0/1440118799 shutdown_connections 2024-03-04T08:22:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b8a00640 1 --2- 172.21.15.113:0/1440118799 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b415b900 0x7f69b4163100 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b8a00640 1 -- 172.21.15.113:0/1440118799 >> 172.21.15.113:0/1440118799 conn(0x7f69b406e820 msgr2=0x7f69b406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b8a00640 1 -- 172.21.15.113:0/1440118799 shutdown_connections 2024-03-04T08:22:52.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.073+0000 7f69b8a00640 1 -- 172.21.15.113:0/1440118799 wait complete. 2024-03-04T08:22:52.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 Processor -- start 2024-03-04T08:22:52.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 -- start start 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b40c22e0 0x7f69b40c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b414e4c0 0x7f69b4152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69b415b900 0x7f69b41f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b4071500 con 0x7f69b40c22e0 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b4071220 con 0x7f69b414e4c0 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b414e8a0 con 0x7f69b415b900 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b40c22e0 0x7f69b40c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b2800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69b415b900 0x7f69b41f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b40c22e0 0x7f69b40c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36052/0 (socket says 172.21.15.113:36052) 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b1e00640 1 -- 172.21.15.113:0/3353222873 learned_addr learned my addr 172.21.15.113:0/3353222873 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b1400640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b414e4c0 0x7f69b4152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b1e00640 1 -- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69b415b900 msgr2=0x7f69b41f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:52.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b1e00640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69b415b900 0x7f69b41f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:52.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b1e00640 1 -- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b414e4c0 msgr2=0x7f69b4152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:52.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b1e00640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b414e4c0 0x7f69b4152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:52.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b1e00640 1 -- 172.21.15.113:0/3353222873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b4159f90 con 0x7f69b40c22e0 2024-03-04T08:22:52.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b1e00640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b40c22e0 0x7f69b40c6ac0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f69a400cbf0 tx=0x7f69a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:52.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69aa000640 1 -- 172.21.15.113:0/3353222873 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4017070 con 0x7f69b40c22e0 2024-03-04T08:22:52.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69aa000640 1 -- 172.21.15.113:0/3353222873 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a400ad50 con 0x7f69b40c22e0 2024-03-04T08:22:52.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 -- 172.21.15.113:0/3353222873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b415a270 con 0x7f69b40c22e0 2024-03-04T08:22:52.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69aa000640 1 -- 172.21.15.113:0/3353222873 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a4012620 con 0x7f69b40c22e0 2024-03-04T08:22:52.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 -- 172.21.15.113:0/3353222873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b41593e0 con 0x7f69b40c22e0 2024-03-04T08:22:52.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.077+0000 7f69b8a00640 1 -- 172.21.15.113:0/3353222873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b4075c00 con 0x7f69b40c22e0 2024-03-04T08:22:52.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.081+0000 7f69aa000640 1 -- 172.21.15.113:0/3353222873 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69a401a070 con 0x7f69b40c22e0 2024-03-04T08:22:52.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.081+0000 7f69aa000640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6994066f60 0x7f6994069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:52.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.081+0000 7f69aa000640 1 -- 172.21.15.113:0/3353222873 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7f69a40858e0 con 0x7f69b40c22e0 2024-03-04T08:22:52.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.081+0000 7f69b1400640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6994066f60 0x7f6994069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:52.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.081+0000 7f69b1400640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6994066f60 0x7f6994069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f69b40c8f40 tx=0x7f69a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:52.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.085+0000 7f69aa000640 1 -- 172.21.15.113:0/3353222873 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f69a40557e0 con 0x7f69b40c22e0 2024-03-04T08:22:52.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 -- 172.21.15.113:0/3353222873 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69b40c8970 con 0x7f69b40c22e0 2024-03-04T08:22:52.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69aa000640 1 -- 172.21.15.113:0/3353222873 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 180 v180) v1 ==== 95+0+4239 (secure 0 0 0) 0x7f69a4059880 con 0x7f69b40c22e0 2024-03-04T08:22:52.272 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:52.272 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":180,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":180,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:51.004972+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[8032,4247]},"id":1}]} 2024-03-04T08:22:52.273 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 180 2024-03-04T08:22:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 -- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6994066f60 msgr2=0x7f6994069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6994066f60 0x7f6994069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f69b40c8f40 tx=0x7f69a0009040 comp rx=0 tx=0).stop 2024-03-04T08:22:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 -- 172.21.15.113:0/3353222873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b40c22e0 msgr2=0x7f69b40c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:52.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b40c22e0 0x7f69b40c6ac0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f69a400cbf0 tx=0x7f69a400a430 comp rx=0 tx=0).stop 2024-03-04T08:22:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 -- 172.21.15.113:0/3353222873 shutdown_connections 2024-03-04T08:22:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69b415b900 0x7f69b41f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69b414e4c0 0x7f69b4152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6994066f60 0x7f6994069420 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 --2- 172.21.15.113:0/3353222873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69b40c22e0 0x7f69b40c6ac0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 -- 172.21.15.113:0/3353222873 >> 172.21.15.113:0/3353222873 conn(0x7f69b406e820 msgr2=0x7f69b414bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 -- 172.21.15.113:0/3353222873 shutdown_connections 2024-03-04T08:22:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:52.273+0000 7f69b8a00640 1 -- 172.21.15.113:0/3353222873 wait complete. 2024-03-04T08:22:52.294 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=7839 2024-03-04T08:22:52.294 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.e 2024-03-04T08:22:59.295 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2024-03-04T08:22:59.295 INFO:tasks.ceph.mds.e:Restarting daemon 2024-03-04T08:22:59.295 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-03-04T08:22:59.299 INFO:tasks.ceph.mds.e:Started 2024-03-04T08:22:59.299 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:22:59.391 INFO:tasks.ceph.mds.e.smithi113.stdout:starting mds.e at 2024-03-04T08:22:59.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.444+0000 7fd2cc200640 1 Processor -- start 2024-03-04T08:22:59.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.444+0000 7fd2cc200640 1 -- start start 2024-03-04T08:22:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2cc200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd2c40c8200 0x7fd2c4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2cc200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2c4152bd0 0x7fd2c41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2cc200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c41591f0 con 0x7fd2c41527f0 2024-03-04T08:22:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2cc200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c4071df0 con 0x7fd2c4152bd0 2024-03-04T08:22:59.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2cc200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c41597b0 con 0x7fd2c40c8200 2024-03-04T08:22:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2ca200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2c4152bd0 0x7fd2c41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2ca200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2c4152bd0 0x7fd2c41587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39700/0 (socket says 172.21.15.113:39700) 2024-03-04T08:22:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2c9800640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fd2c41527f0 0x7fd2c4072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:47644/0 (socket says 172.21.15.113:47644) 2024-03-04T08:22:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2ca200640 1 -- 172.21.15.113:0/2492678805 learned_addr learned my addr 172.21.15.113:0/2492678805 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2c8e00640 1 --2- 172.21.15.113:0/2492678805 >> v2:172.21.15.136:3300/0 conn(0x7fd2c40c8200 0x7fd2c4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2ca200640 1 -- 172.21.15.113:0/2492678805 >> v2:172.21.15.136:3300/0 conn(0x7fd2c40c8200 msgr2=0x7fd2c4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:22:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2ca200640 1 --2- 172.21.15.113:0/2492678805 >> v2:172.21.15.136:3300/0 conn(0x7fd2c40c8200 0x7fd2c4070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2ca200640 1 -- 172.21.15.113:0/2492678805 >> v1:172.21.15.136:6789/0 conn(0x7fd2c41527f0 legacy=0x7fd2c4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:59.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.448+0000 7fd2ca200640 1 -- 172.21.15.113:0/2492678805 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c41594d0 con 0x7fd2c4152bd0 2024-03-04T08:22:59.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2ca200640 1 --2- 172.21.15.113:0/2492678805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c4152bd0 0x7fd2c41587d0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40cd230 tx=0x7fd2bc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819caac8a2b2553 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:59.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2c3e00640 1 -- 172.21.15.113:0/2492678805 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0a57a0 con 0x7fd2c4152bd0 2024-03-04T08:22:59.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2c3e00640 1 -- 172.21.15.113:0/2492678805 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc0a5940 con 0x7fd2c4152bd0 2024-03-04T08:22:59.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2c3e00640 1 -- 172.21.15.113:0/2492678805 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0a5d10 con 0x7fd2c4152bd0 2024-03-04T08:22:59.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2cc200640 1 -- 172.21.15.113:0/2492678805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c4152bd0 msgr2=0x7fd2c41587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:59.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2cc200640 1 --2- 172.21.15.113:0/2492678805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c4152bd0 0x7fd2c41587d0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40cd230 tx=0x7fd2bc01c9f0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2cc200640 1 -- 172.21.15.113:0/2492678805 shutdown_connections 2024-03-04T08:22:59.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2cc200640 1 --2- 172.21.15.113:0/2492678805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c4152bd0 0x7fd2c41587d0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2cc200640 1 --2- 172.21.15.113:0/2492678805 >> v2:172.21.15.136:3300/0 conn(0x7fd2c40c8200 0x7fd2c4070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2cc200640 1 -- 172.21.15.113:0/2492678805 >> 172.21.15.113:0/2492678805 conn(0x7fd2c406e820 msgr2=0x7fd2c406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:59.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2cc200640 1 -- 172.21.15.113:0/2492678805 shutdown_connections 2024-03-04T08:22:59.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2cc200640 1 -- 172.21.15.113:0/2492678805 wait complete. 2024-03-04T08:22:59.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2cc200640 1 Processor -- start 2024-03-04T08:22:59.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.452+0000 7fd2cc200640 1 -- start start 2024-03-04T08:22:59.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2cc200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c40c8200 0x7fd2c41f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:59.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2cc200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c41527f0 0x7fd2c414dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2cc200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c4152bd0 0x7fd2c40afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2cc200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c41654f0 con 0x7fd2c40c8200 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2cc200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c4165370 con 0x7fd2c4152bd0 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2cc200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c4165670 con 0x7fd2c41527f0 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c40c8200 0x7fd2c41f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c8e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c41527f0 0x7fd2c414dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2ca200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c4152bd0 0x7fd2c40afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c9800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c40c8200 0x7fd2c41f79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39708/0 (socket says 172.21.15.113:39708) 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c9800640 1 -- 172.21.15.113:0/2152381829 learned_addr learned my addr 172.21.15.113:0/2152381829 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c9800640 1 -- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c41527f0 msgr2=0x7fd2c414dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c9800640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c41527f0 0x7fd2c414dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c9800640 1 -- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c4152bd0 msgr2=0x7fd2c40afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c9800640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c4152bd0 0x7fd2c40afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c9800640 1 -- 172.21.15.113:0/2152381829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c41f3a40 con 0x7fd2c40c8200 2024-03-04T08:22:59.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c9800640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c40c8200 0x7fd2c41f79c0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c420 tx=0x7fd2b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:59.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c1600640 1 -- 172.21.15.113:0/2152381829 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80074e0 con 0x7fd2c40c8200 2024-03-04T08:22:59.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c1600640 1 -- 172.21.15.113:0/2152381829 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b8007680 con 0x7fd2c40c8200 2024-03-04T08:22:59.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c1600640 1 -- 172.21.15.113:0/2152381829 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8004a30 con 0x7fd2c40c8200 2024-03-04T08:22:59.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2cc200640 1 -- 172.21.15.113:0/2152381829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c41f3d00 con 0x7fd2c40c8200 2024-03-04T08:22:59.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2cc200640 1 -- 172.21.15.113:0/2152381829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c41f42c0 con 0x7fd2c40c8200 2024-03-04T08:22:59.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2cc200640 1 -- 172.21.15.113:0/2152381829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c4075c00 con 0x7fd2c40c8200 2024-03-04T08:22:59.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.456+0000 7fd2c1600640 1 -- 172.21.15.113:0/2152381829 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2b801a020 con 0x7fd2c40c8200 2024-03-04T08:22:59.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.460+0000 7fd2c1600640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd2a4066f60 0x7fd2a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:22:59.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.460+0000 7fd2c1600640 1 -- 172.21.15.113:0/2152381829 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7fd2b8085120 con 0x7fd2c40c8200 2024-03-04T08:22:59.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.460+0000 7fd2c8e00640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd2a4066f60 0x7fd2a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:22:59.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.460+0000 7fd2c8e00640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd2a4066f60 0x7fd2a4069420 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40c2230 tx=0x7fd2b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:22:59.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.464+0000 7fd2c1600640 1 -- 172.21.15.113:0/2152381829 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd2b80550e0 con 0x7fd2c40c8200 2024-03-04T08:22:59.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.656+0000 7fd2cc200640 1 -- 172.21.15.113:0/2152381829 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2c40c79d0 con 0x7fd2c40c8200 2024-03-04T08:22:59.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.656+0000 7fd2c1600640 1 -- 172.21.15.113:0/2152381829 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+4242 (secure 0 0 0) 0x7fd2b8059180 con 0x7fd2c40c8200 2024-03-04T08:22:59.658 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:22:59.658 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":181,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:22:56.019109+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:resolve","state_seq":738,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:22:59.658 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 181 2024-03-04T08:22:59.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 -- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd2a4066f60 msgr2=0x7fd2a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:59.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd2a4066f60 0x7fd2a4069420 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40c2230 tx=0x7fd2b0008040 comp rx=0 tx=0).stop 2024-03-04T08:22:59.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 -- 172.21.15.113:0/2152381829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c40c8200 msgr2=0x7fd2c41f79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:22:59.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c40c8200 0x7fd2c41f79c0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800c420 tx=0x7fd2b800c8e0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 -- 172.21.15.113:0/2152381829 shutdown_connections 2024-03-04T08:22:59.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2c4152bd0 0x7fd2c40afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd2c41527f0 0x7fd2c414dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd2a4066f60 0x7fd2a4069420 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 --2- 172.21.15.113:0/2152381829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2c40c8200 0x7fd2c41f79c0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:22:59.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 -- 172.21.15.113:0/2152381829 >> 172.21.15.113:0/2152381829 conn(0x7fd2c406e820 msgr2=0x7fd2c40c2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:22:59.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 -- 172.21.15.113:0/2152381829 shutdown_connections 2024-03-04T08:22:59.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:22:59.660+0000 7fd2cc200640 1 -- 172.21.15.113:0/2152381829 wait complete. 2024-03-04T08:22:59.681 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2024-03-04T08:23:01.681 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:01.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.824+0000 7fccf4600640 1 Processor -- start 2024-03-04T08:23:01.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.824+0000 7fccf4600640 1 -- start start 2024-03-04T08:23:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.824+0000 7fccf4600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fccec154c00 0x7fccec0c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.824+0000 7fccf4600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec154fe0 0x7fccec0c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf4600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccec0c7d80 con 0x7fccec0c7700 2024-03-04T08:23:01.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf4600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec15efb0 con 0x7fccec154c00 2024-03-04T08:23:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf4600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0c6b70 con 0x7fccec154fe0 2024-03-04T08:23:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf1200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec154fe0 0x7fccec0c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf1200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec154fe0 0x7fccec0c1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:52104/0 (socket says 172.21.15.113:52104) 2024-03-04T08:23:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf1200640 1 -- 172.21.15.113:0/1265926593 learned_addr learned my addr 172.21.15.113:0/1265926593 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf1c00640 1 --2- 172.21.15.113:0/1265926593 >> v2:172.21.15.136:3300/0 conn(0x7fccec154c00 0x7fccec0c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf1200640 1 -- 172.21.15.113:0/1265926593 >> v2:172.21.15.136:3300/0 conn(0x7fccec154c00 msgr2=0x7fccec0c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf1200640 1 --2- 172.21.15.113:0/1265926593 >> v2:172.21.15.136:3300/0 conn(0x7fccec154c00 0x7fccec0c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf1200640 1 -- 172.21.15.113:0/1265926593 >> v1:172.21.15.136:6789/0 conn(0x7fccec0c7700 legacy=0x7fccec15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:01.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf1200640 1 -- 172.21.15.113:0/1265926593 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec0c8060 con 0x7fccec154fe0 2024-03-04T08:23:01.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccf1200640 1 --2- 172.21.15.113:0/1265926593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec154fe0 0x7fccec0c1b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009870 tx=0x7fccd801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf388f07d7d627 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.828+0000 7fccebe00640 1 -- 172.21.15.113:0/1265926593 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80b1070 con 0x7fccec154fe0 2024-03-04T08:23:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccebe00640 1 -- 172.21.15.113:0/1265926593 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd80ac620 con 0x7fccec154fe0 2024-03-04T08:23:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccebe00640 1 -- 172.21.15.113:0/1265926593 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80ac9f0 con 0x7fccec154fe0 2024-03-04T08:23:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 -- 172.21.15.113:0/1265926593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec154fe0 msgr2=0x7fccec0c1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:01.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 --2- 172.21.15.113:0/1265926593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec154fe0 0x7fccec0c1b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009870 tx=0x7fccd801c4d0 comp rx=0 tx=0).stop 2024-03-04T08:23:01.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 -- 172.21.15.113:0/1265926593 shutdown_connections 2024-03-04T08:23:01.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 --2- 172.21.15.113:0/1265926593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec154fe0 0x7fccec0c1b50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:01.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 --2- 172.21.15.113:0/1265926593 >> v2:172.21.15.136:3300/0 conn(0x7fccec154c00 0x7fccec0c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:01.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 -- 172.21.15.113:0/1265926593 >> 172.21.15.113:0/1265926593 conn(0x7fccec06e820 msgr2=0x7fccec06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:01.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 -- 172.21.15.113:0/1265926593 shutdown_connections 2024-03-04T08:23:01.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 -- 172.21.15.113:0/1265926593 wait complete. 2024-03-04T08:23:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 Processor -- start 2024-03-04T08:23:01.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 -- start start 2024-03-04T08:23:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec0c7700 0x7fccec1f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccec154c00 0x7fccec0c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:01.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccec154fe0 0x7fccec14e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec167c10 con 0x7fccec154c00 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0c2320 con 0x7fccec154fe0 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf4600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec167d90 con 0x7fccec0c7700 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf1c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec0c7700 0x7fccec1f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf1c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec0c7700 0x7fccec1f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:52120/0 (socket says 172.21.15.113:52120) 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf1c00640 1 -- 172.21.15.113:0/3533456207 learned_addr learned my addr 172.21.15.113:0/3533456207 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf1200640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccec154c00 0x7fccec0c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.832+0000 7fccf2600640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccec154fe0 0x7fccec14e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fccf1c00640 1 -- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccec154fe0 msgr2=0x7fccec14e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fccf1c00640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccec154fe0 0x7fccec14e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fccf1c00640 1 -- 172.21.15.113:0/3533456207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccec154c00 msgr2=0x7fccec0c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fccf1c00640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccec154c00 0x7fccec0c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:01.836 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fccf1c00640 1 -- 172.21.15.113:0/3533456207 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec1f3da0 con 0x7fccec0c7700 2024-03-04T08:23:01.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fccf1c00640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec0c7700 0x7fccec1f4820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcce000e8a0 tx=0x7fcce000eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:01.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fcce9600640 1 -- 172.21.15.113:0/3533456207 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce0017070 con 0x7fccec0c7700 2024-03-04T08:23:01.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fccf4600640 1 -- 172.21.15.113:0/3533456207 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec0caf90 con 0x7fccec0c7700 2024-03-04T08:23:01.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fcce9600640 1 -- 172.21.15.113:0/3533456207 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce000c940 con 0x7fccec0c7700 2024-03-04T08:23:01.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fcce9600640 1 -- 172.21.15.113:0/3533456207 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce000ccf0 con 0x7fccec0c7700 2024-03-04T08:23:01.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fccf4600640 1 -- 172.21.15.113:0/3533456207 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec1f4040 con 0x7fccec0c7700 2024-03-04T08:23:01.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fccf4600640 1 -- 172.21.15.113:0/3533456207 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec075c00 con 0x7fccec0c7700 2024-03-04T08:23:01.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fcce9600640 1 -- 172.21.15.113:0/3533456207 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcce001a070 con 0x7fccec0c7700 2024-03-04T08:23:01.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fcce9600640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcccc066f60 0x7fcccc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:01.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.836+0000 7fcce9600640 1 -- 172.21.15.113:0/3533456207 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7fcce0086120 con 0x7fccec0c7700 2024-03-04T08:23:01.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.840+0000 7fccf1200640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcccc066f60 0x7fcccc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:01.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.840+0000 7fccf1200640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcccc066f60 0x7fcccc069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ae000 tx=0x7fccd80a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:01.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:01.848+0000 7fcce9600640 1 -- 172.21.15.113:0/3533456207 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcce00560b0 con 0x7fccec0c7700 2024-03-04T08:23:02.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.032+0000 7fccf4600640 1 -- 172.21.15.113:0/3533456207 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccec1512e0 con 0x7fccec0c7700 2024-03-04T08:23:02.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fcce9600640 1 -- 172.21.15.113:0/3533456207 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) v1 ==== 95+0+5038 (secure 0 0 0) 0x7fcce005a150 con 0x7fccec0c7700 2024-03-04T08:23:02.036 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:02.036 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":183,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8760,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:01.660417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:reconnect","state_seq":740,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:23:02.036 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 183 2024-03-04T08:23:02.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 -- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcccc066f60 msgr2=0x7fcccc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:02.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcccc066f60 0x7fcccc069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ae000 tx=0x7fccd80a7000 comp rx=0 tx=0).stop 2024-03-04T08:23:02.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 -- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec0c7700 msgr2=0x7fccec1f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:02.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec0c7700 0x7fccec1f4820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcce000e8a0 tx=0x7fcce000eca0 comp rx=0 tx=0).stop 2024-03-04T08:23:02.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 -- 172.21.15.113:0/3533456207 shutdown_connections 2024-03-04T08:23:02.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccec154fe0 0x7fccec14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:02.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcccc066f60 0x7fcccc069420 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:02.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccec154c00 0x7fccec0c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:02.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 --2- 172.21.15.113:0/3533456207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccec0c7700 0x7fccec1f4820 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 -- 172.21.15.113:0/3533456207 >> 172.21.15.113:0/3533456207 conn(0x7fccec06e820 msgr2=0x7fccec1588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 -- 172.21.15.113:0/3533456207 shutdown_connections 2024-03-04T08:23:02.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:02.036+0000 7fccf4600640 1 -- 172.21.15.113:0/3533456207 wait complete. 2024-03-04T08:23:02.058 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2024-03-04T08:23:02.058 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before thrashing 2024-03-04T08:23:09.059 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc2b000640 1 Processor -- start 2024-03-04T08:23:09.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc2b000640 1 -- start start 2024-03-04T08:23:09.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc2b000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc2406de80 0x7fcc2415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc2b000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2406c210 con 0x7fcc240ce6b0 2024-03-04T08:23:09.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc2b000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc2406c4f0 con 0x7fcc240ce2d0 2024-03-04T08:23:09.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc2b000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2415bbf0 con 0x7fcc2406de80 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc23400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcc240ce6b0 0x7fcc24154b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60264/0 (socket says 172.21.15.113:60264) 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc23400640 1 -- 172.21.15.113:0/205356869 learned_addr learned my addr 172.21.15.113:0/205356869 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc29000640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc2406de80 0x7fcc2415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358644052 0 0) 0x7fcc2406c210 con 0x7fcc240ce6b0 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc08003530 con 0x7fcc240ce6b0 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099375100 0 0) 0x7fcc2406c4f0 con 0x7fcc240ce2d0 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc2406c210 con 0x7fcc240ce2d0 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4191273567 0 0) 0x7fcc08003530 con 0x7fcc240ce6b0 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc2406de80 msgr2=0x7fcc2415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc2406de80 0x7fcc2415b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 >> v1:172.21.15.136:6789/0 conn(0x7fcc240ce2d0 legacy=0x7fcc2406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2406e530 con 0x7fcc240ce6b0 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fcc10001680 con 0x7fcc240ce6b0 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 >> v1:172.21.15.113:6789/0 conn(0x7fcc240ce6b0 legacy=0x7fcc24154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc2406de80 0x7fcc08004be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc08005190 0x7fcc080037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 0x7fcc08004170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2415bbf0 con 0x7fcc08003cf0 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0800bb10 con 0x7fcc08005190 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0800bd10 con 0x7fcc2406de80 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc29000640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 0x7fcc08004170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc23400640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc08005190 0x7fcc080037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc23e00640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc2406de80 0x7fcc08004be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc29000640 1 -- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc2406de80 msgr2=0x7fcc08004be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc29000640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc2406de80 0x7fcc08004be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc29000640 1 -- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc08005190 msgr2=0x7fcc080037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc29000640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc08005190 0x7fcc080037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.208+0000 7fcc29000640 1 -- 172.21.15.113:0/205356869 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc08003530 con 0x7fcc08003cf0 2024-03-04T08:23:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc29000640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 0x7fcc08004170 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009fc0 tx=0x7fcc1c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5942f976930521e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0b2070 con 0x7fcc08003cf0 2024-03-04T08:23:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c0a5c90 con 0x7fcc08003cf0 2024-03-04T08:23:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc22a00640 1 -- 172.21.15.113:0/205356869 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c009c10 con 0x7fcc08003cf0 2024-03-04T08:23:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc2b000640 1 -- 172.21.15.113:0/205356869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 msgr2=0x7fcc08004170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc2b000640 1 --2- 172.21.15.113:0/205356869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 0x7fcc08004170 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009fc0 tx=0x7fcc1c01cc80 comp rx=0 tx=0).stop 2024-03-04T08:23:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc29000640 1 -- 172.21.15.113:0/205356869 reap_dead start 2024-03-04T08:23:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc2b000640 1 -- 172.21.15.113:0/205356869 shutdown_connections 2024-03-04T08:23:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc2b000640 1 -- 172.21.15.113:0/205356869 >> 172.21.15.113:0/205356869 conn(0x7fcc240709a0 msgr2=0x7fcc24070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:09.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc2b000640 1 -- 172.21.15.113:0/205356869 shutdown_connections 2024-03-04T08:23:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.212+0000 7fcc2b000640 1 -- 172.21.15.113:0/205356869 wait complete. 2024-03-04T08:23:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc2b000640 1 Processor -- start 2024-03-04T08:23:09.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc2b000640 1 -- start start 2024-03-04T08:23:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc2b000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 0x7fcc24156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc2b000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc241f1960 0x7fcc241f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc2b000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc241f0a30 0x7fcc241f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc2b000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2406c7d0 con 0x7fcc08003cf0 2024-03-04T08:23:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc2b000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2406eb80 con 0x7fcc241f0a30 2024-03-04T08:23:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc2b000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2406f720 con 0x7fcc241f1960 2024-03-04T08:23:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 0x7fcc24156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc29000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc241f0a30 0x7fcc241f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc23e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 0x7fcc24156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40500/0 (socket says 172.21.15.113:40500) 2024-03-04T08:23:09.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc23e00640 1 -- 172.21.15.113:0/3408558474 learned_addr learned my addr 172.21.15.113:0/3408558474 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc23400640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc241f1960 0x7fcc241f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc23e00640 1 -- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc241f1960 msgr2=0x7fcc241f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc23e00640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc241f1960 0x7fcc241f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc23e00640 1 -- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc241f0a30 msgr2=0x7fcc241f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc23e00640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc241f0a30 0x7fcc241f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc23e00640 1 -- 172.21.15.113:0/3408558474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc241556b0 con 0x7fcc08003cf0 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc23e00640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 0x7fcc24156cf0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e810 tx=0x7fcc1400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc1be00640 1 -- 172.21.15.113:0/3408558474 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc140050d0 con 0x7fcc08003cf0 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc1be00640 1 -- 172.21.15.113:0/3408558474 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc14018070 con 0x7fcc08003cf0 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc2b000640 1 -- 172.21.15.113:0/3408558474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc24155970 con 0x7fcc08003cf0 2024-03-04T08:23:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc1be00640 1 -- 172.21.15.113:0/3408558474 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400c660 con 0x7fcc08003cf0 2024-03-04T08:23:09.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.216+0000 7fcc2b000640 1 -- 172.21.15.113:0/3408558474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc24155f30 con 0x7fcc08003cf0 2024-03-04T08:23:09.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.220+0000 7fcc2b000640 1 -- 172.21.15.113:0/3408558474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc24075c00 con 0x7fcc08003cf0 2024-03-04T08:23:09.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.220+0000 7fcc1be00640 1 -- 172.21.15.113:0/3408558474 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc1400c800 con 0x7fcc08003cf0 2024-03-04T08:23:09.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.220+0000 7fcc1be00640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcc04066f60 0x7fcc04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.220+0000 7fcc1be00640 1 -- 172.21.15.113:0/3408558474 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7fcc140856f0 con 0x7fcc08003cf0 2024-03-04T08:23:09.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.220+0000 7fcc23400640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcc04066f60 0x7fcc04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.220+0000 7fcc23400640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcc04066f60 0x7fcc04069420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcc240cbab0 tx=0x7fcc10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:09.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.224+0000 7fcc1be00640 1 -- 172.21.15.113:0/3408558474 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcc14055d00 con 0x7fcc08003cf0 2024-03-04T08:23:09.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.412+0000 7fcc2b000640 1 -- 172.21.15.113:0/3408558474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc241585d0 con 0x7fcc08003cf0 2024-03-04T08:23:09.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.412+0000 7fcc1be00640 1 -- 172.21.15.113:0/3408558474 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+5035 (secure 0 0 0) 0x7fcc14055600 con 0x7fcc08003cf0 2024-03-04T08:23:09.415 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:09.415 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":184,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8760,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:06.689466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:rejoin","state_seq":742,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[8032,4247]},"id":1}]} 2024-03-04T08:23:09.415 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 184 2024-03-04T08:23:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 -- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcc04066f60 msgr2=0x7fcc04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcc04066f60 0x7fcc04069420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcc240cbab0 tx=0x7fcc10009040 comp rx=0 tx=0).stop 2024-03-04T08:23:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 -- 172.21.15.113:0/3408558474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 msgr2=0x7fcc24156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 0x7fcc24156cf0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e810 tx=0x7fcc1400ecd0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 -- 172.21.15.113:0/3408558474 shutdown_connections 2024-03-04T08:23:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcc241f0a30 0x7fcc241f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcc241f1960 0x7fcc241f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcc04066f60 0x7fcc04069420 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 --2- 172.21.15.113:0/3408558474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc08003cf0 0x7fcc24156cf0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 -- 172.21.15.113:0/3408558474 >> 172.21.15.113:0/3408558474 conn(0x7fcc240709a0 msgr2=0x7fcc24158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 -- 172.21.15.113:0/3408558474 shutdown_connections 2024-03-04T08:23:09.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.416+0000 7fcc2b000640 1 -- 172.21.15.113:0/3408558474 wait complete. 2024-03-04T08:23:09.436 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=2) 2024-03-04T08:23:09.436 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-04T08:23:09.436 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:23:09.450 INFO:tasks.ceph.mds.d:Stopped 2024-03-04T08:23:09.450 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:09.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.584+0000 7fc2c9e00640 1 Processor -- start 2024-03-04T08:23:09.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.584+0000 7fc2c9e00640 1 -- start start 2024-03-04T08:23:09.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.584+0000 7fc2c9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c415b900 0x7fc2c4163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.584+0000 7fc2c9e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c40c27f0 con 0x7fc2c414e4c0 2024-03-04T08:23:09.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.584+0000 7fc2c9e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c40c2a90 con 0x7fc2c40c22e0 2024-03-04T08:23:09.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.584+0000 7fc2c9e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c415ad60 con 0x7fc2c415b900 2024-03-04T08:23:09.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc2c414e4c0 0x7fc2c4072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60266/0 (socket says 172.21.15.113:60266) 2024-03-04T08:23:09.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2a00640 1 -- 172.21.15.113:0/462450886 learned_addr learned my addr 172.21.15.113:0/462450886 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:23:09.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c3400640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c415b900 0x7fc2c4163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741970310 0 0) 0x7fc2c40c27f0 con 0x7fc2c414e4c0 2024-03-04T08:23:09.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a8003530 con 0x7fc2c414e4c0 2024-03-04T08:23:09.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707329837 0 0) 0x7fc2c40c2a90 con 0x7fc2c40c22e0 2024-03-04T08:23:09.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2c40c27f0 con 0x7fc2c40c22e0 2024-03-04T08:23:09.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 561644968 0 0) 0x7fc2a8003530 con 0x7fc2c414e4c0 2024-03-04T08:23:09.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c415b900 msgr2=0x7fc2c4163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c415b900 0x7fc2c4163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 >> v1:172.21.15.136:6789/0 conn(0x7fc2c40c22e0 legacy=0x7fc2c415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c4070c60 con 0x7fc2c414e4c0 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fc2b4001680 con 0x7fc2c414e4c0 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 >> v1:172.21.15.113:6789/0 conn(0x7fc2c414e4c0 legacy=0x7fc2c4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2a8003b80 0x7fc2a8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c415b900 0x7fc2a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2a8004d50 0x7fc2a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a800ba80 con 0x7fc2a8003b80 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a800de10 con 0x7fc2a8004d50 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a800dfc0 con 0x7fc2c415b900 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2a00640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2a8003b80 0x7fc2a8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2ba000640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c415b900 0x7fc2a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c3400640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2a8004d50 0x7fc2a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2a00640 1 -- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c415b900 msgr2=0x7fc2a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:09.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2a00640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c415b900 0x7fc2a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2a00640 1 -- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2a8004d50 msgr2=0x7fc2a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:09.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2a00640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2a8004d50 0x7fc2a80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c2a00640 1 -- 172.21.15.113:0/462450886 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8003530 con 0x7fc2a8003b80 2024-03-04T08:23:09.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.588+0000 7fc2c3400640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2a8004d50 0x7fc2a80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:23:09.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c2a00640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2a8003b80 0x7fc2a8003f80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400c210 tx=0x7fc2b401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4581bc497b6300 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4007cf0 con 0x7fc2a8003b80 2024-03-04T08:23:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b40a9020 con 0x7fc2a8003b80 2024-03-04T08:23:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c2000640 1 -- 172.21.15.113:0/462450886 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40a5b40 con 0x7fc2a8003b80 2024-03-04T08:23:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 -- 172.21.15.113:0/462450886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2a8003b80 msgr2=0x7fc2a8003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 --2- 172.21.15.113:0/462450886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2a8003b80 0x7fc2a8003f80 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400c210 tx=0x7fc2b401cc20 comp rx=0 tx=0).stop 2024-03-04T08:23:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c3400640 1 -- 172.21.15.113:0/462450886 reap_dead start 2024-03-04T08:23:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 -- 172.21.15.113:0/462450886 shutdown_connections 2024-03-04T08:23:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 -- 172.21.15.113:0/462450886 >> 172.21.15.113:0/462450886 conn(0x7fc2c406e820 msgr2=0x7fc2c406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 -- 172.21.15.113:0/462450886 shutdown_connections 2024-03-04T08:23:09.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 -- 172.21.15.113:0/462450886 wait complete. 2024-03-04T08:23:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 Processor -- start 2024-03-04T08:23:09.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 -- start start 2024-03-04T08:23:09.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2a8003b80 0x7fc2c414fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c4152060 0x7fc2c414d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c4146cb0 0x7fc2c4147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c4071500 con 0x7fc2c4146cb0 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c4071220 con 0x7fc2a8003b80 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.592+0000 7fc2c9e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c414e8a0 con 0x7fc2c4152060 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c4146cb0 0x7fc2c4147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2ba000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c4152060 0x7fc2c414d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c4146cb0 0x7fc2c4147130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40522/0 (socket says 172.21.15.113:40522) 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c3400640 1 -- 172.21.15.113:0/2441547330 learned_addr learned my addr 172.21.15.113:0/2441547330 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c2a00640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2a8003b80 0x7fc2c414fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c3400640 1 -- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c4152060 msgr2=0x7fc2c414d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c3400640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c4152060 0x7fc2c414d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c3400640 1 -- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2a8003b80 msgr2=0x7fc2c414fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c3400640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2a8003b80 0x7fc2c414fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c3400640 1 -- 172.21.15.113:0/2441547330 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c414eb60 con 0x7fc2c4146cb0 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c3400640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c4146cb0 0x7fc2c4147130 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40c80a0 tx=0x7fc2bc010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2bbe00640 1 -- 172.21.15.113:0/2441547330 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc010da0 con 0x7fc2c4146cb0 2024-03-04T08:23:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2bbe00640 1 -- 172.21.15.113:0/2441547330 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc0039a0 con 0x7fc2c4146cb0 2024-03-04T08:23:09.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2bbe00640 1 -- 172.21.15.113:0/2441547330 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc003dd0 con 0x7fc2c4146cb0 2024-03-04T08:23:09.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c9e00640 1 -- 172.21.15.113:0/2441547330 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c414ede0 con 0x7fc2c4146cb0 2024-03-04T08:23:09.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c9e00640 1 -- 172.21.15.113:0/2441547330 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c4158cb0 con 0x7fc2c4146cb0 2024-03-04T08:23:09.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2c9e00640 1 -- 172.21.15.113:0/2441547330 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c4075c00 con 0x7fc2c4146cb0 2024-03-04T08:23:09.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2bbe00640 1 -- 172.21.15.113:0/2441547330 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2bc01e030 con 0x7fc2c4146cb0 2024-03-04T08:23:09.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2bbe00640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2a0066f60 0x7fc2a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.596+0000 7fc2bbe00640 1 -- 172.21.15.113:0/2441547330 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7fc2bc085fb0 con 0x7fc2c4146cb0 2024-03-04T08:23:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.600+0000 7fc2c2a00640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2a0066f60 0x7fc2a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:09.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.600+0000 7fc2c2a00640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2a0066f60 0x7fc2a0069420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40aa000 tx=0x7fc2c4072ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:09.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.600+0000 7fc2bbe00640 1 -- 172.21.15.113:0/2441547330 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc2bc025020 con 0x7fc2c4146cb0 2024-03-04T08:23:09.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 -- 172.21.15.113:0/2441547330 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2c40c1260 con 0x7fc2c4146cb0 2024-03-04T08:23:09.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2bbe00640 1 -- 172.21.15.113:0/2441547330 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+5035 (secure 0 0 0) 0x7fc2c414eb60 con 0x7fc2c4146cb0 2024-03-04T08:23:09.792 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:09.792 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":184,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8760,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:06.689466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:rejoin","state_seq":742,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[8032,4247]},"id":1}]} 2024-03-04T08:23:09.792 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 184 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 -- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2a0066f60 msgr2=0x7fc2a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2a0066f60 0x7fc2a0069420 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc2b40aa000 tx=0x7fc2c4072ba0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 -- 172.21.15.113:0/2441547330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c4146cb0 msgr2=0x7fc2c4147130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c4146cb0 0x7fc2c4147130 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40c80a0 tx=0x7fc2bc010590 comp rx=0 tx=0).stop 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 -- 172.21.15.113:0/2441547330 shutdown_connections 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2a0066f60 0x7fc2a0069420 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2c4146cb0 0x7fc2c4147130 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2c4152060 0x7fc2c414d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 --2- 172.21.15.113:0/2441547330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2a8003b80 0x7fc2c414fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 -- 172.21.15.113:0/2441547330 >> 172.21.15.113:0/2441547330 conn(0x7fc2c406e820 msgr2=0x7fc2c40c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 -- 172.21.15.113:0/2441547330 shutdown_connections 2024-03-04T08:23:09.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:09.792+0000 7fc2c9e00640 1 -- 172.21.15.113:0/2441547330 wait complete. 2024-03-04T08:23:09.813 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-03-04T08:23:11.814 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:11.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f414b200640 1 Processor -- start 2024-03-04T08:23:11.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f414b200640 1 -- start start 2024-03-04T08:23:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f414b200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f41440c8200 0x7f4144070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f414b200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41441591f0 con 0x7f4144152bd0 2024-03-04T08:23:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f414b200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41441594d0 con 0x7f41441527f0 2024-03-04T08:23:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f414b200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4144071df0 con 0x7f41440c8200 2024-03-04T08:23:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f4149200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4144152bd0 0x7f41441587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60280/0 (socket says 172.21.15.113:60280) 2024-03-04T08:23:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f4149200640 1 -- 172.21.15.113:0/2347889214 learned_addr learned my addr 172.21.15.113:0/2347889214 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:23:11.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f4143400640 1 --2- 172.21.15.113:0/2347889214 >> v2:172.21.15.136:3300/0 conn(0x7f41440c8200 0x7f4144070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:11.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f4142a00640 1 -- 172.21.15.113:0/2347889214 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589983640 0 0) 0x7f41441591f0 con 0x7f4144152bd0 2024-03-04T08:23:11.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f4142a00640 1 -- 172.21.15.113:0/2347889214 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4128003530 con 0x7f4144152bd0 2024-03-04T08:23:11.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f4143400640 1 -- 172.21.15.113:0/2347889214 >> v1:172.21.15.136:6789/0 conn(0x7f41441527f0 legacy=0x7f4144072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:11.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f4143400640 1 -- 172.21.15.113:0/2347889214 >> v1:172.21.15.113:6789/0 conn(0x7f4144152bd0 legacy=0x7f41441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:11.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.964+0000 7f4143400640 1 -- 172.21.15.113:0/2347889214 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41440717a0 con 0x7f41440c8200 2024-03-04T08:23:11.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f4143400640 1 --2- 172.21.15.113:0/2347889214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41440c8200 0x7f4144070d40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f41300097c0 tx=0x7f413001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9348dcb4dc39f06 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:11.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f4142a00640 1 -- 172.21.15.113:0/2347889214 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300b1070 con 0x7f41440c8200 2024-03-04T08:23:11.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f4142a00640 1 -- 172.21.15.113:0/2347889214 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41300ac410 con 0x7f41440c8200 2024-03-04T08:23:11.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f4142a00640 1 -- 172.21.15.113:0/2347889214 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300ac7c0 con 0x7f41440c8200 2024-03-04T08:23:11.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f414b200640 1 -- 172.21.15.113:0/2347889214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41440c8200 msgr2=0x7f4144070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:11.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f414b200640 1 --2- 172.21.15.113:0/2347889214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41440c8200 0x7f4144070d40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f41300097c0 tx=0x7f413001c4a0 comp rx=0 tx=0).stop 2024-03-04T08:23:11.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f414b200640 1 -- 172.21.15.113:0/2347889214 shutdown_connections 2024-03-04T08:23:11.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f414b200640 1 --2- 172.21.15.113:0/2347889214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41440c8200 0x7f4144070d40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:11.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f414b200640 1 -- 172.21.15.113:0/2347889214 >> 172.21.15.113:0/2347889214 conn(0x7f414406e820 msgr2=0x7f414406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:11.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f414b200640 1 -- 172.21.15.113:0/2347889214 shutdown_connections 2024-03-04T08:23:11.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.968+0000 7f414b200640 1 -- 172.21.15.113:0/2347889214 wait complete. 2024-03-04T08:23:11.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 Processor -- start 2024-03-04T08:23:11.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 -- start start 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41440c8200 0x7f4144149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41441527f0 0x7f414414a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4144152bd0 0x7f4144142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4144166380 con 0x7f41440c8200 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41441597b0 con 0x7f4144152bd0 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4144166500 con 0x7f41441527f0 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4143e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41440c8200 0x7f4144149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4143e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41440c8200 0x7f4144149ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40538/0 (socket says 172.21.15.113:40538) 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4143e00640 1 -- 172.21.15.113:0/953995581 learned_addr learned my addr 172.21.15.113:0/953995581 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4149200640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4144152bd0 0x7f4144142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4143400640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41441527f0 0x7f414414a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:11.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4143e00640 1 -- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41441527f0 msgr2=0x7f414414a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:11.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4143e00640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41441527f0 0x7f414414a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:11.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4143e00640 1 -- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4144152bd0 msgr2=0x7f4144142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:11.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4143e00640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4144152bd0 0x7f4144142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:11.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4143e00640 1 -- 172.21.15.113:0/953995581 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4144143960 con 0x7f41440c8200 2024-03-04T08:23:11.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f4143e00640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41440c8200 0x7f4144149ee0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f413400c420 tx=0x7f413400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:11.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f413be00640 1 -- 172.21.15.113:0/953995581 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41340052d0 con 0x7f41440c8200 2024-03-04T08:23:11.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f413be00640 1 -- 172.21.15.113:0/953995581 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4134005470 con 0x7f41440c8200 2024-03-04T08:23:11.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 -- 172.21.15.113:0/953995581 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41441f1790 con 0x7f41440c8200 2024-03-04T08:23:11.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f413be00640 1 -- 172.21.15.113:0/953995581 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134007260 con 0x7f41440c8200 2024-03-04T08:23:11.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 -- 172.21.15.113:0/953995581 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41441496e0 con 0x7f41440c8200 2024-03-04T08:23:11.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.972+0000 7f414b200640 1 -- 172.21.15.113:0/953995581 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4144075c00 con 0x7f41440c8200 2024-03-04T08:23:11.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.976+0000 7f413be00640 1 -- 172.21.15.113:0/953995581 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4134007400 con 0x7f41440c8200 2024-03-04T08:23:11.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.976+0000 7f413be00640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4124066f60 0x7f4124069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:11.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.976+0000 7f413be00640 1 -- 172.21.15.113:0/953995581 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7f4134084eb0 con 0x7f41440c8200 2024-03-04T08:23:11.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.976+0000 7f4143400640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4124066f60 0x7f4124069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:11.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.976+0000 7f4143400640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4124066f60 0x7f4124069420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f41300ae000 tx=0x7f41300a6040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:11.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:11.980+0000 7f413be00640 1 -- 172.21.15.113:0/953995581 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4134054e40 con 0x7f41440c8200 2024-03-04T08:23:12.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.168+0000 7f414b200640 1 -- 172.21.15.113:0/953995581 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41440c79d0 con 0x7f41440c8200 2024-03-04T08:23:12.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.168+0000 7f413be00640 1 -- 172.21.15.113:0/953995581 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+5035 (secure 0 0 0) 0x7f4134058ee0 con 0x7f41440c8200 2024-03-04T08:23:12.166 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:12.166 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":184,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8760,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:06.689466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:rejoin","state_seq":742,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[8032,4247]},"id":1}]} 2024-03-04T08:23:12.167 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 184 2024-03-04T08:23:12.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 -- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4124066f60 msgr2=0x7f4124069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:12.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4124066f60 0x7f4124069420 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f41300ae000 tx=0x7f41300a6040 comp rx=0 tx=0).stop 2024-03-04T08:23:12.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 -- 172.21.15.113:0/953995581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41440c8200 msgr2=0x7f4144149ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:12.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41440c8200 0x7f4144149ee0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f413400c420 tx=0x7f413400c8e0 comp rx=0 tx=0).stop 2024-03-04T08:23:12.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 -- 172.21.15.113:0/953995581 shutdown_connections 2024-03-04T08:23:12.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4144152bd0 0x7f4144142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:12.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41441527f0 0x7f414414a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:12.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4124066f60 0x7f4124069420 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:12.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 --2- 172.21.15.113:0/953995581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41440c8200 0x7f4144149ee0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:12.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 -- 172.21.15.113:0/953995581 >> 172.21.15.113:0/953995581 conn(0x7f414406e820 msgr2=0x7f41441574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:12.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 -- 172.21.15.113:0/953995581 shutdown_connections 2024-03-04T08:23:12.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:12.172+0000 7f414b200640 1 -- 172.21.15.113:0/953995581 wait complete. 2024-03-04T08:23:12.187 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-03-04T08:23:14.188 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:14.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2e5600640 1 Processor -- start 2024-03-04T08:23:14.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2e5600640 1 -- start start 2024-03-04T08:23:14.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2e5600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e015bb40 0x7fc2e006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:14.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2e5600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e00c4ce0 con 0x7fc2e00cb540 2024-03-04T08:23:14.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2e5600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e00c4f80 con 0x7fc2e0152950 2024-03-04T08:23:14.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2e5600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00c8370 con 0x7fc2e015bb40 2024-03-04T08:23:14.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2df400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc2e00cb540 0x7fc2e0160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60292/0 (socket says 172.21.15.113:60292) 2024-03-04T08:23:14.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2df400640 1 -- 172.21.15.113:0/797158176 learned_addr learned my addr 172.21.15.113:0/797158176 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:23:14.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2e4c00640 1 --2- 172.21.15.113:0/797158176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e015bb40 0x7fc2e006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:14.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2dea00640 1 -- 172.21.15.113:0/797158176 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300353680 0 0) 0x7fc2e00c4ce0 con 0x7fc2e00cb540 2024-03-04T08:23:14.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2dea00640 1 -- 172.21.15.113:0/797158176 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2c4003530 con 0x7fc2e00cb540 2024-03-04T08:23:14.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2e4c00640 1 -- 172.21.15.113:0/797158176 >> v1:172.21.15.136:6789/0 conn(0x7fc2e0152950 legacy=0x7fc2e006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:14.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2dea00640 1 -- 172.21.15.113:0/797158176 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890559070 0 0) 0x7fc2e00c4f80 con 0x7fc2e0152950 2024-03-04T08:23:14.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2e4c00640 1 -- 172.21.15.113:0/797158176 >> v1:172.21.15.113:6789/0 conn(0x7fc2e00cb540 legacy=0x7fc2e0160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:14.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.340+0000 7fc2e4c00640 1 -- 172.21.15.113:0/797158176 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00cb920 con 0x7fc2e015bb40 2024-03-04T08:23:14.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2e4c00640 1 --2- 172.21.15.113:0/797158176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e015bb40 0x7fc2e006faa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80097f0 tx=0x7fc2d801ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=363d53750f6fb9c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:14.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2dea00640 1 -- 172.21.15.113:0/797158176 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d80b5070 con 0x7fc2e015bb40 2024-03-04T08:23:14.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2dea00640 1 -- 172.21.15.113:0/797158176 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d80a89d0 con 0x7fc2e015bb40 2024-03-04T08:23:14.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2dea00640 1 -- 172.21.15.113:0/797158176 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d80a8d80 con 0x7fc2e015bb40 2024-03-04T08:23:14.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2e5600640 1 -- 172.21.15.113:0/797158176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e015bb40 msgr2=0x7fc2e006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:14.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2e5600640 1 --2- 172.21.15.113:0/797158176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e015bb40 0x7fc2e006faa0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc2d80097f0 tx=0x7fc2d801ecd0 comp rx=0 tx=0).stop 2024-03-04T08:23:14.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2e5600640 1 -- 172.21.15.113:0/797158176 shutdown_connections 2024-03-04T08:23:14.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2e5600640 1 --2- 172.21.15.113:0/797158176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e015bb40 0x7fc2e006faa0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:14.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2e5600640 1 -- 172.21.15.113:0/797158176 >> 172.21.15.113:0/797158176 conn(0x7fc2e0072f40 msgr2=0x7fc2e00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:14.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2e5600640 1 -- 172.21.15.113:0/797158176 shutdown_connections 2024-03-04T08:23:14.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.344+0000 7fc2e5600640 1 -- 172.21.15.113:0/797158176 wait complete. 2024-03-04T08:23:14.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e5600640 1 Processor -- start 2024-03-04T08:23:14.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e5600640 1 -- start start 2024-03-04T08:23:14.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e5600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e00cb540 0x7fc2e014e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:14.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e5600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e0152950 0x7fc2e01e4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:14.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e5600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e015bb40 0x7fc2e0144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:14.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e5600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00c9a50 con 0x7fc2e015bb40 2024-03-04T08:23:14.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e5600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00ca0a0 con 0x7fc2e0152950 2024-03-04T08:23:14.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e5600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e00c9d30 con 0x7fc2e00cb540 2024-03-04T08:23:14.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e015bb40 0x7fc2e0144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:14.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e4c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e015bb40 0x7fc2e0144880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40542/0 (socket says 172.21.15.113:40542) 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2df400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e0152950 0x7fc2e01e4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e4c00640 1 -- 172.21.15.113:0/1371937964 learned_addr learned my addr 172.21.15.113:0/1371937964 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2dfe00640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e00cb540 0x7fc2e014e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e4c00640 1 -- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e00cb540 msgr2=0x7fc2e014e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e4c00640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e00cb540 0x7fc2e014e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e4c00640 1 -- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e0152950 msgr2=0x7fc2e01e4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e4c00640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e0152950 0x7fc2e01e4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e4c00640 1 -- 172.21.15.113:0/1371937964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e01f1890 con 0x7fc2e015bb40 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e4c00640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e015bb40 0x7fc2e0144880 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc2d801c900 tx=0x7fc2d801ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2d7e00640 1 -- 172.21.15.113:0/1371937964 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d80b5040 con 0x7fc2e015bb40 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2d7e00640 1 -- 172.21.15.113:0/1371937964 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d80bf070 con 0x7fc2e015bb40 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e5600640 1 -- 172.21.15.113:0/1371937964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e0150fd0 con 0x7fc2e015bb40 2024-03-04T08:23:14.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2d7e00640 1 -- 172.21.15.113:0/1371937964 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8003620 con 0x7fc2e015bb40 2024-03-04T08:23:14.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.348+0000 7fc2e5600640 1 -- 172.21.15.113:0/1371937964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e01f1de0 con 0x7fc2e015bb40 2024-03-04T08:23:14.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.352+0000 7fc2e5600640 1 -- 172.21.15.113:0/1371937964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e0075a80 con 0x7fc2e015bb40 2024-03-04T08:23:14.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.352+0000 7fc2d7e00640 1 -- 172.21.15.113:0/1371937964 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2d80037c0 con 0x7fc2e015bb40 2024-03-04T08:23:14.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.352+0000 7fc2d7e00640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2c0066f90 0x7fc2c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:14.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.352+0000 7fc2d7e00640 1 -- 172.21.15.113:0/1371937964 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7fc2d8122cf0 con 0x7fc2e015bb40 2024-03-04T08:23:14.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.352+0000 7fc2dfe00640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2c0066f90 0x7fc2c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:14.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.352+0000 7fc2dfe00640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2c0066f90 0x7fc2c0069450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00012f0 tx=0x7fc2d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:14.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.356+0000 7fc2d7e00640 1 -- 172.21.15.113:0/1371937964 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc2d8018830 con 0x7fc2e015bb40 2024-03-04T08:23:14.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 -- 172.21.15.113:0/1371937964 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2e00c9580 con 0x7fc2e015bb40 2024-03-04T08:23:14.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2d7e00640 1 -- 172.21.15.113:0/1371937964 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+5035 (secure 0 0 0) 0x7fc2d80f2d40 con 0x7fc2e015bb40 2024-03-04T08:23:14.541 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:14.541 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":184,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8760,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:06.689466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:rejoin","state_seq":742,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[8032,4247]},"id":1}]} 2024-03-04T08:23:14.541 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 184 2024-03-04T08:23:14.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 -- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2c0066f90 msgr2=0x7fc2c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:14.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2c0066f90 0x7fc2c0069450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00012f0 tx=0x7fc2d0009040 comp rx=0 tx=0).stop 2024-03-04T08:23:14.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 -- 172.21.15.113:0/1371937964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e015bb40 msgr2=0x7fc2e0144880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:14.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e015bb40 0x7fc2e0144880 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc2d801c900 tx=0x7fc2d801ba70 comp rx=0 tx=0).stop 2024-03-04T08:23:14.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 -- 172.21.15.113:0/1371937964 shutdown_connections 2024-03-04T08:23:14.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc2c0066f90 0x7fc2c0069450 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:14.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc2e015bb40 0x7fc2e0144880 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:14.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2e0152950 0x7fc2e01e4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:14.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 --2- 172.21.15.113:0/1371937964 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2e00cb540 0x7fc2e014e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:14.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 -- 172.21.15.113:0/1371937964 >> 172.21.15.113:0/1371937964 conn(0x7fc2e0072f40 msgr2=0x7fc2e00c5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:14.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 -- 172.21.15.113:0/1371937964 shutdown_connections 2024-03-04T08:23:14.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:14.544+0000 7fc2e5600640 1 -- 172.21.15.113:0/1371937964 wait complete. 2024-03-04T08:23:14.562 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-03-04T08:23:14.974 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:23:14.977 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:23:16.563 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:16.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.716+0000 7f23ab400640 1 Processor -- start 2024-03-04T08:23:16.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.716+0000 7f23ab400640 1 -- start start 2024-03-04T08:23:16.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.716+0000 7f23ab400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f23a415b900 0x7f23a4163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:16.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.716+0000 7f23ab400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a40c27f0 con 0x7f23a414e4c0 2024-03-04T08:23:16.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.716+0000 7f23ab400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a40c2a90 con 0x7f23a40c22e0 2024-03-04T08:23:16.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.716+0000 7f23ab400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a415ad60 con 0x7f23a415b900 2024-03-04T08:23:16.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.716+0000 7f23a8a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f23a414e4c0 0x7f23a4072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60306/0 (socket says 172.21.15.113:60306) 2024-03-04T08:23:16.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.716+0000 7f23a8a00640 1 -- 172.21.15.113:0/323331159 learned_addr learned my addr 172.21.15.113:0/323331159 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:23:16.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.716+0000 7f23a9400640 1 --2- 172.21.15.113:0/323331159 >> v2:172.21.15.136:3300/0 conn(0x7f23a415b900 0x7f23a4163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:16.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.720+0000 7f23a3e00640 1 -- 172.21.15.113:0/323331159 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971623760 0 0) 0x7f23a40c27f0 con 0x7f23a414e4c0 2024-03-04T08:23:16.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.720+0000 7f23a3e00640 1 -- 172.21.15.113:0/323331159 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2388005740 con 0x7f23a414e4c0 2024-03-04T08:23:16.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.720+0000 7f23a3e00640 1 -- 172.21.15.113:0/323331159 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666565566 0 0) 0x7f23a40c2a90 con 0x7f23a40c22e0 2024-03-04T08:23:16.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.720+0000 7f23a9400640 1 -- 172.21.15.113:0/323331159 >> v1:172.21.15.136:6789/0 conn(0x7f23a40c22e0 legacy=0x7f23a415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:16.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.720+0000 7f23a9400640 1 -- 172.21.15.113:0/323331159 >> v1:172.21.15.113:6789/0 conn(0x7f23a414e4c0 legacy=0x7f23a4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:16.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.720+0000 7f23a9400640 1 -- 172.21.15.113:0/323331159 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a4070c60 con 0x7f23a415b900 2024-03-04T08:23:16.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.720+0000 7f23a9400640 1 --2- 172.21.15.113:0/323331159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a415b900 0x7f23a4163100 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f239c009d90 tx=0x7f239c01f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfdf2657e7239390 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:16.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.720+0000 7f23a3e00640 1 -- 172.21.15.113:0/323331159 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c0b4070 con 0x7f23a415b900 2024-03-04T08:23:16.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.720+0000 7f23a3e00640 1 -- 172.21.15.113:0/323331159 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239c0af600 con 0x7f23a415b900 2024-03-04T08:23:16.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.720+0000 7f23a3e00640 1 -- 172.21.15.113:0/323331159 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239c0af9b0 con 0x7f23a415b900 2024-03-04T08:23:16.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 -- 172.21.15.113:0/323331159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a415b900 msgr2=0x7f23a4163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:16.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 --2- 172.21.15.113:0/323331159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a415b900 0x7f23a4163100 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f239c009d90 tx=0x7f239c01f150 comp rx=0 tx=0).stop 2024-03-04T08:23:16.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 -- 172.21.15.113:0/323331159 shutdown_connections 2024-03-04T08:23:16.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 --2- 172.21.15.113:0/323331159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a415b900 0x7f23a4163100 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:16.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 -- 172.21.15.113:0/323331159 >> 172.21.15.113:0/323331159 conn(0x7f23a406e820 msgr2=0x7f23a406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:16.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 -- 172.21.15.113:0/323331159 shutdown_connections 2024-03-04T08:23:16.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 -- 172.21.15.113:0/323331159 wait complete. 2024-03-04T08:23:16.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 Processor -- start 2024-03-04T08:23:16.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 -- start start 2024-03-04T08:23:16.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23a40c22e0 0x7f23a41ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:16.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a414e4c0 0x7f23a41429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:16.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a415b900 0x7f23a4142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:16.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4071500 con 0x7f23a414e4c0 2024-03-04T08:23:16.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a4071220 con 0x7f23a415b900 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23ab400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a414e8a0 con 0x7f23a40c22e0 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f239be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a414e4c0 0x7f23a41429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f239be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a414e4c0 0x7f23a41429a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40554/0 (socket says 172.21.15.113:40554) 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f239be00640 1 -- 172.21.15.113:0/610251204 learned_addr learned my addr 172.21.15.113:0/610251204 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23a8a00640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23a40c22e0 0x7f23a41ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23a9400640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a415b900 0x7f23a4142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f239be00640 1 -- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23a40c22e0 msgr2=0x7f23a41ecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f239be00640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23a40c22e0 0x7f23a41ecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f239be00640 1 -- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a415b900 msgr2=0x7f23a4142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f239be00640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a415b900 0x7f23a4142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f239be00640 1 -- 172.21.15.113:0/610251204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a41459a0 con 0x7f23a414e4c0 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f23a9400640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a415b900 0x7f23a4142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:23:16.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.724+0000 7f239be00640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a414e4c0 0x7f23a41429a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c440 tx=0x7f238c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:16.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.728+0000 7f23a1600640 1 -- 172.21.15.113:0/610251204 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c005480 con 0x7f23a414e4c0 2024-03-04T08:23:16.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.728+0000 7f23a1600640 1 -- 172.21.15.113:0/610251204 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c003600 con 0x7f23a414e4c0 2024-03-04T08:23:16.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.728+0000 7f23a1600640 1 -- 172.21.15.113:0/610251204 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c0039f0 con 0x7f23a414e4c0 2024-03-04T08:23:16.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.728+0000 7f23ab400640 1 -- 172.21.15.113:0/610251204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a41453e0 con 0x7f23a414e4c0 2024-03-04T08:23:16.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.728+0000 7f23ab400640 1 -- 172.21.15.113:0/610251204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a41ec490 con 0x7f23a414e4c0 2024-03-04T08:23:16.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.728+0000 7f23ab400640 1 -- 172.21.15.113:0/610251204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a4075c00 con 0x7f23a414e4c0 2024-03-04T08:23:16.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.728+0000 7f23a1600640 1 -- 172.21.15.113:0/610251204 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f238c003b90 con 0x7f23a414e4c0 2024-03-04T08:23:16.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.728+0000 7f23a1600640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2380066f60 0x7f2380069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:16.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.728+0000 7f23a1600640 1 -- 172.21.15.113:0/610251204 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7f238c0854d0 con 0x7f23a414e4c0 2024-03-04T08:23:16.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.728+0000 7f23a8a00640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2380066f60 0x7f2380069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:16.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.732+0000 7f23a8a00640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2380066f60 0x7f2380069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2394007fd0 tx=0x7f2394009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:16.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.732+0000 7f23a1600640 1 -- 172.21.15.113:0/610251204 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f238c055400 con 0x7f23a414e4c0 2024-03-04T08:23:16.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.924+0000 7f23ab400640 1 -- 172.21.15.113:0/610251204 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23a40c20c0 con 0x7f23a414e4c0 2024-03-04T08:23:16.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.924+0000 7f23a1600640 1 -- 172.21.15.113:0/610251204 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+5035 (secure 0 0 0) 0x7f238c0594a0 con 0x7f23a414e4c0 2024-03-04T08:23:16.922 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:16.922 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":184,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8760,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:06.689466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:rejoin","state_seq":742,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[8032,4247]},"id":1}]} 2024-03-04T08:23:16.922 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 184 2024-03-04T08:23:16.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 -- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2380066f60 msgr2=0x7f2380069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:16.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2380066f60 0x7f2380069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2394007fd0 tx=0x7f2394009040 comp rx=0 tx=0).stop 2024-03-04T08:23:16.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 -- 172.21.15.113:0/610251204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a414e4c0 msgr2=0x7f23a41429a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:16.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a414e4c0 0x7f23a41429a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c440 tx=0x7f238c00c900 comp rx=0 tx=0).stop 2024-03-04T08:23:16.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 -- 172.21.15.113:0/610251204 shutdown_connections 2024-03-04T08:23:16.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23a415b900 0x7f23a4142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:16.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2380066f60 0x7f2380069420 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:16.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23a414e4c0 0x7f23a41429a0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:16.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 --2- 172.21.15.113:0/610251204 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23a40c22e0 0x7f23a41ecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:16.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 -- 172.21.15.113:0/610251204 >> 172.21.15.113:0/610251204 conn(0x7f23a406e820 msgr2=0x7f23a4147a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:16.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 -- 172.21.15.113:0/610251204 shutdown_connections 2024-03-04T08:23:16.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:16.928+0000 7f23ab400640 1 -- 172.21.15.113:0/610251204 wait complete. 2024-03-04T08:23:16.944 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-03-04T08:23:18.945 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:19.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.092+0000 7f9e84c00640 1 Processor -- start 2024-03-04T08:23:19.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e84c00640 1 -- start start 2024-03-04T08:23:19.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e84c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151840 0x7f9e800c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:19.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e84c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e80157da0 con 0x7f9e8006d140 2024-03-04T08:23:19.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e84c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e80158080 con 0x7f9e80151c20 2024-03-04T08:23:19.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e84c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800c3440 con 0x7f9e80151840 2024-03-04T08:23:19.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e7fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9e8006d140 0x7f9e800722d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44362/0 (socket says 172.21.15.113:44362) 2024-03-04T08:23:19.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e7fe00640 1 -- 172.21.15.113:0/2327151340 learned_addr learned my addr 172.21.15.113:0/2327151340 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:23:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e7f400640 1 --2- 172.21.15.113:0/2327151340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151840 0x7f9e800c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e7ea00640 1 -- 172.21.15.113:0/2327151340 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897945015 0 0) 0x7f9e80157da0 con 0x7f9e8006d140 2024-03-04T08:23:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e7ea00640 1 -- 172.21.15.113:0/2327151340 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e64003530 con 0x7f9e8006d140 2024-03-04T08:23:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e7ea00640 1 -- 172.21.15.113:0/2327151340 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 997176371 0 0) 0x7f9e64003530 con 0x7f9e8006d140 2024-03-04T08:23:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e7f400640 1 -- 172.21.15.113:0/2327151340 >> v1:172.21.15.136:6789/0 conn(0x7f9e80151c20 legacy=0x7f9e8006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e7f400640 1 -- 172.21.15.113:0/2327151340 >> v1:172.21.15.113:6789/0 conn(0x7f9e8006d140 legacy=0x7f9e800722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:19.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.096+0000 7f9e7f400640 1 -- 172.21.15.113:0/2327151340 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e80158360 con 0x7f9e80151840 2024-03-04T08:23:19.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e7f400640 1 --2- 172.21.15.113:0/2327151340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151840 0x7f9e800c1f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009770 tx=0x7f9e7001c2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f405d2528452cfca server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:19.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e7ea00640 1 -- 172.21.15.113:0/2327151340 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700b0070 con 0x7f9e80151840 2024-03-04T08:23:19.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e7ea00640 1 -- 172.21.15.113:0/2327151340 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e700ab400 con 0x7f9e80151840 2024-03-04T08:23:19.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e7ea00640 1 -- 172.21.15.113:0/2327151340 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78001a00 con 0x7f9e80151840 2024-03-04T08:23:19.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e84c00640 1 -- 172.21.15.113:0/2327151340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151840 msgr2=0x7f9e800c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:19.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e84c00640 1 --2- 172.21.15.113:0/2327151340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151840 0x7f9e800c1f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009770 tx=0x7f9e7001c2c0 comp rx=0 tx=0).stop 2024-03-04T08:23:19.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e84c00640 1 -- 172.21.15.113:0/2327151340 shutdown_connections 2024-03-04T08:23:19.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e84c00640 1 --2- 172.21.15.113:0/2327151340 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151840 0x7f9e800c1f50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:19.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e84c00640 1 -- 172.21.15.113:0/2327151340 >> 172.21.15.113:0/2327151340 conn(0x7f9e800c9a30 msgr2=0x7f9e800c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:19.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e84c00640 1 -- 172.21.15.113:0/2327151340 shutdown_connections 2024-03-04T08:23:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e84c00640 1 -- 172.21.15.113:0/2327151340 wait complete. 2024-03-04T08:23:19.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.100+0000 7f9e84c00640 1 Processor -- start 2024-03-04T08:23:19.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e84c00640 1 -- start start 2024-03-04T08:23:19.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e84c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8006d140 0x7f9e801f0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:19.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e84c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e80151840 0x7f9e8006bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:19.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e84c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151c20 0x7f9e8006c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e84c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e80071db0 con 0x7f9e8006d140 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e84c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e80071ad0 con 0x7f9e80151840 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e84c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8006c880 con 0x7f9e80151c20 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e7f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8006d140 0x7f9e801f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e7f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8006d140 0x7f9e801f0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50864/0 (socket says 172.21.15.113:50864) 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e7f400640 1 -- 172.21.15.113:0/3796892967 learned_addr learned my addr 172.21.15.113:0/3796892967 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e7fe00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151c20 0x7f9e8006c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e76a00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e80151840 0x7f9e8006bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e7f400640 1 -- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151c20 msgr2=0x7f9e8006c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e7f400640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151c20 0x7f9e8006c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e7f400640 1 -- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e80151840 msgr2=0x7f9e8006bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e7f400640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e80151840 0x7f9e8006bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e7f400640 1 -- 172.21.15.113:0/3796892967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e80156850 con 0x7f9e8006d140 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e7f400640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8006d140 0x7f9e801f0020 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f9e7001cb60 tx=0x7f9e70019060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:19.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e77e00640 1 -- 172.21.15.113:0/3796892967 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700b0040 con 0x7f9e8006d140 2024-03-04T08:23:19.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e77e00640 1 -- 172.21.15.113:0/3796892967 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e700bb070 con 0x7f9e8006d140 2024-03-04T08:23:19.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e84c00640 1 -- 172.21.15.113:0/3796892967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e801f36c0 con 0x7f9e8006d140 2024-03-04T08:23:19.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e77e00640 1 -- 172.21.15.113:0/3796892967 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70006e20 con 0x7f9e8006d140 2024-03-04T08:23:19.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e84c00640 1 -- 172.21.15.113:0/3796892967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e801f0560 con 0x7f9e8006d140 2024-03-04T08:23:19.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.104+0000 7f9e84c00640 1 -- 172.21.15.113:0/3796892967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e80075a80 con 0x7f9e8006d140 2024-03-04T08:23:19.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.108+0000 7f9e77e00640 1 -- 172.21.15.113:0/3796892967 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e700b3070 con 0x7f9e8006d140 2024-03-04T08:23:19.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.108+0000 7f9e77e00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e5c066f60 0x7f9e5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:19.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.108+0000 7f9e77e00640 1 -- 172.21.15.113:0/3796892967 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7f9e7011fad0 con 0x7f9e8006d140 2024-03-04T08:23:19.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.108+0000 7f9e76a00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e5c066f60 0x7f9e5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:19.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.108+0000 7f9e76a00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e5c066f60 0x7f9e5c069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9e68001ec0 tx=0x7f9e6800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:19.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.112+0000 7f9e77e00640 1 -- 172.21.15.113:0/3796892967 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9e70015f70 con 0x7f9e8006d140 2024-03-04T08:23:19.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 -- 172.21.15.113:0/3796892967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e801444a0 con 0x7f9e8006d140 2024-03-04T08:23:19.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e77e00640 1 -- 172.21.15.113:0/3796892967 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+5035 (secure 0 0 0) 0x7f9e700f00e0 con 0x7f9e8006d140 2024-03-04T08:23:19.297 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:19.297 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":184,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8760,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:06.689466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:rejoin","state_seq":742,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[8032,4247]},"id":1}]} 2024-03-04T08:23:19.297 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 184 2024-03-04T08:23:19.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 -- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e5c066f60 msgr2=0x7f9e5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:19.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e5c066f60 0x7f9e5c069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9e68001ec0 tx=0x7f9e6800b040 comp rx=0 tx=0).stop 2024-03-04T08:23:19.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 -- 172.21.15.113:0/3796892967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8006d140 msgr2=0x7f9e801f0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:19.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8006d140 0x7f9e801f0020 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f9e7001cb60 tx=0x7f9e70019060 comp rx=0 tx=0).stop 2024-03-04T08:23:19.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 -- 172.21.15.113:0/3796892967 shutdown_connections 2024-03-04T08:23:19.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9e80151c20 0x7f9e8006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:19.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e80151840 0x7f9e8006bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:19.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9e5c066f60 0x7f9e5c069420 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:19.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 --2- 172.21.15.113:0/3796892967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e8006d140 0x7f9e801f0020 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:19.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 -- 172.21.15.113:0/3796892967 >> 172.21.15.113:0/3796892967 conn(0x7f9e800c9a30 msgr2=0x7f9e800c1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:19.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 -- 172.21.15.113:0/3796892967 shutdown_connections 2024-03-04T08:23:19.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:19.300+0000 7f9e84c00640 1 -- 172.21.15.113:0/3796892967 wait complete. 2024-03-04T08:23:19.318 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-03-04T08:23:21.319 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:21.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.468+0000 7fec90000640 1 Processor -- start 2024-03-04T08:23:21.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.468+0000 7fec90000640 1 -- start start 2024-03-04T08:23:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.468+0000 7fec90000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fec880c85f0 0x7fec8806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.468+0000 7fec90000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec880c1ec0 con 0x7fec880b20b0 2024-03-04T08:23:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.468+0000 7fec90000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec880c9b30 con 0x7fec8806b5a0 2024-03-04T08:23:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.468+0000 7fec90000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec880c12a0 con 0x7fec880c85f0 2024-03-04T08:23:21.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.468+0000 7fec8d600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fec880b20b0 0x7fec88157710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44376/0 (socket says 172.21.15.113:44376) 2024-03-04T08:23:21.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.468+0000 7fec8d600640 1 -- 172.21.15.113:0/2274584445 learned_addr learned my addr 172.21.15.113:0/2274584445 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:23:21.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.468+0000 7fec8cc00640 1 --2- 172.21.15.113:0/2274584445 >> v2:172.21.15.136:3300/0 conn(0x7fec880c85f0 0x7fec8806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:21.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec87e00640 1 -- 172.21.15.113:0/2274584445 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267154149 0 0) 0x7fec880c1ec0 con 0x7fec880b20b0 2024-03-04T08:23:21.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec87e00640 1 -- 172.21.15.113:0/2274584445 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec70003530 con 0x7fec880b20b0 2024-03-04T08:23:21.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec87e00640 1 -- 172.21.15.113:0/2274584445 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740889373 0 0) 0x7fec880c9b30 con 0x7fec8806b5a0 2024-03-04T08:23:21.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec87e00640 1 -- 172.21.15.113:0/2274584445 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec880c1ec0 con 0x7fec8806b5a0 2024-03-04T08:23:21.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec8cc00640 1 -- 172.21.15.113:0/2274584445 >> v1:172.21.15.136:6789/0 conn(0x7fec8806b5a0 legacy=0x7fec88161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:21.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec8cc00640 1 -- 172.21.15.113:0/2274584445 >> v1:172.21.15.113:6789/0 conn(0x7fec880b20b0 legacy=0x7fec88157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:21.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec8cc00640 1 -- 172.21.15.113:0/2274584445 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880c1580 con 0x7fec880c85f0 2024-03-04T08:23:21.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec8cc00640 1 --2- 172.21.15.113:0/2274584445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec880c85f0 0x7fec8806b060 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fec740097c0 tx=0x7fec7401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbdfbed06d553b62 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:21.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec87e00640 1 -- 172.21.15.113:0/2274584445 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740b5070 con 0x7fec880c85f0 2024-03-04T08:23:21.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec87e00640 1 -- 172.21.15.113:0/2274584445 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec740a8990 con 0x7fec880c85f0 2024-03-04T08:23:21.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.472+0000 7fec87e00640 1 -- 172.21.15.113:0/2274584445 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740a8d40 con 0x7fec880c85f0 2024-03-04T08:23:21.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 -- 172.21.15.113:0/2274584445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec880c85f0 msgr2=0x7fec8806b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:21.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 --2- 172.21.15.113:0/2274584445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec880c85f0 0x7fec8806b060 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fec740097c0 tx=0x7fec7401ec90 comp rx=0 tx=0).stop 2024-03-04T08:23:21.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 -- 172.21.15.113:0/2274584445 shutdown_connections 2024-03-04T08:23:21.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 --2- 172.21.15.113:0/2274584445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec880c85f0 0x7fec8806b060 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:21.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 -- 172.21.15.113:0/2274584445 >> 172.21.15.113:0/2274584445 conn(0x7fec8806e820 msgr2=0x7fec8806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:21.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 -- 172.21.15.113:0/2274584445 shutdown_connections 2024-03-04T08:23:21.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 -- 172.21.15.113:0/2274584445 wait complete. 2024-03-04T08:23:21.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 Processor -- start 2024-03-04T08:23:21.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 -- start start 2024-03-04T08:23:21.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8806b5a0 0x7fec881461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:21.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec880b20b0 0x7fec88151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:21.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec880c85f0 0x7fec88141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:21.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec88157e20 con 0x7fec8806b5a0 2024-03-04T08:23:21.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec881571f0 con 0x7fec880c85f0 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec90000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec880c2400 con 0x7fec880b20b0 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8806b5a0 0x7fec881461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8806b5a0 0x7fec881461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50870/0 (socket says 172.21.15.113:50870) 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec880b20b0 0x7fec88151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8d600640 1 -- 172.21.15.113:0/2340662704 learned_addr learned my addr 172.21.15.113:0/2340662704 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8e000640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec880c85f0 0x7fec88141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8d600640 1 -- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec880b20b0 msgr2=0x7fec88151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8d600640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec880b20b0 0x7fec88151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8d600640 1 -- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec880c85f0 msgr2=0x7fec88141dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8d600640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec880c85f0 0x7fec88141dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8d600640 1 -- 172.21.15.113:0/2340662704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880c7c10 con 0x7fec8806b5a0 2024-03-04T08:23:21.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.476+0000 7fec8d600640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8806b5a0 0x7fec881461c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00c500 tx=0x7fec7c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:21.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.480+0000 7fec85600640 1 -- 172.21.15.113:0/2340662704 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c017070 con 0x7fec8806b5a0 2024-03-04T08:23:21.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.480+0000 7fec85600640 1 -- 172.21.15.113:0/2340662704 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c005cf0 con 0x7fec8806b5a0 2024-03-04T08:23:21.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.480+0000 7fec85600640 1 -- 172.21.15.113:0/2340662704 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c007330 con 0x7fec8806b5a0 2024-03-04T08:23:21.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.480+0000 7fec90000640 1 -- 172.21.15.113:0/2340662704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec88144580 con 0x7fec8806b5a0 2024-03-04T08:23:21.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.480+0000 7fec90000640 1 -- 172.21.15.113:0/2340662704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec88144b40 con 0x7fec8806b5a0 2024-03-04T08:23:21.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.480+0000 7fec90000640 1 -- 172.21.15.113:0/2340662704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec88075c00 con 0x7fec8806b5a0 2024-03-04T08:23:21.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.480+0000 7fec85600640 1 -- 172.21.15.113:0/2340662704 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec7c01a070 con 0x7fec8806b5a0 2024-03-04T08:23:21.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.480+0000 7fec85600640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec68066f60 0x7fec68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:21.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.480+0000 7fec85600640 1 -- 172.21.15.113:0/2340662704 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7fec7c084e10 con 0x7fec8806b5a0 2024-03-04T08:23:21.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.480+0000 7fec8cc00640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec68066f60 0x7fec68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:21.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.484+0000 7fec8cc00640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec68066f60 0x7fec68069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fec740b2000 tx=0x7fec74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:21.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.484+0000 7fec85600640 1 -- 172.21.15.113:0/2340662704 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fec7c088030 con 0x7fec8806b5a0 2024-03-04T08:23:21.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.672+0000 7fec90000640 1 -- 172.21.15.113:0/2340662704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec88146a50 con 0x7fec8806b5a0 2024-03-04T08:23:21.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.672+0000 7fec85600640 1 -- 172.21.15.113:0/2340662704 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+5035 (secure 0 0 0) 0x7fec7c054dd0 con 0x7fec8806b5a0 2024-03-04T08:23:21.669 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:21.669 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":184,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8760,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:06.689466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:rejoin","state_seq":742,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[8032,4247]},"id":1}]} 2024-03-04T08:23:21.669 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 184 2024-03-04T08:23:21.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.672+0000 7fec90000640 1 -- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec68066f60 msgr2=0x7fec68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:21.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.672+0000 7fec90000640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec68066f60 0x7fec68069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fec740b2000 tx=0x7fec74008040 comp rx=0 tx=0).stop 2024-03-04T08:23:21.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.676+0000 7fec90000640 1 -- 172.21.15.113:0/2340662704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8806b5a0 msgr2=0x7fec881461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:21.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.676+0000 7fec90000640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8806b5a0 0x7fec881461c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00c500 tx=0x7fec7c00c9c0 comp rx=0 tx=0).stop 2024-03-04T08:23:21.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.676+0000 7fec90000640 1 -- 172.21.15.113:0/2340662704 shutdown_connections 2024-03-04T08:23:21.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.676+0000 7fec90000640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec880c85f0 0x7fec88141dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:21.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.676+0000 7fec90000640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec880b20b0 0x7fec88151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:21.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.676+0000 7fec90000640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fec68066f60 0x7fec68069420 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:21.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.676+0000 7fec90000640 1 --2- 172.21.15.113:0/2340662704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec8806b5a0 0x7fec881461c0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:21.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.676+0000 7fec90000640 1 -- 172.21.15.113:0/2340662704 >> 172.21.15.113:0/2340662704 conn(0x7fec8806e820 msgr2=0x7fec880b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:21.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.676+0000 7fec90000640 1 -- 172.21.15.113:0/2340662704 shutdown_connections 2024-03-04T08:23:21.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:21.676+0000 7fec90000640 1 -- 172.21.15.113:0/2340662704 wait complete. 2024-03-04T08:23:21.690 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-03-04T08:23:23.691 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:23.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f9c00640 1 Processor -- start 2024-03-04T08:23:23.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f9c00640 1 -- start start 2024-03-04T08:23:23.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f9c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc6f40ccb40 0x7fc6f40c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:23.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f9c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f4154f10 0x7fc6f415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:23.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f9c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f415aed0 con 0x7fc6f415aa40 2024-03-04T08:23:23.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f9c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f415b170 con 0x7fc6f40ccb40 2024-03-04T08:23:23.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f9c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f415b410 con 0x7fc6f4154f10 2024-03-04T08:23:23.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f4154f10 0x7fc6f415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:23.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc6f40ccb40 0x7fc6f40c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:23.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc6f40ccb40 0x7fc6f40c7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:52248/0 (socket says 172.21.15.113:52248) 2024-03-04T08:23:23.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f2a00640 1 -- 172.21.15.113:0/272646516 learned_addr learned my addr 172.21.15.113:0/272646516 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:23.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f4154f10 0x7fc6f415e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:43074/0 (socket says 172.21.15.113:43074) 2024-03-04T08:23:23.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f3400640 1 -- 172.21.15.113:0/272646516 >> v2:172.21.15.136:3300/0 conn(0x7fc6f40ccb40 msgr2=0x7fc6f40c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:23.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f3400640 1 --2- 172.21.15.113:0/272646516 >> v2:172.21.15.136:3300/0 conn(0x7fc6f40ccb40 0x7fc6f40c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:23.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f3400640 1 -- 172.21.15.113:0/272646516 >> v1:172.21.15.136:6789/0 conn(0x7fc6f415aa40 legacy=0x7fc6f4154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:23.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.844+0000 7fc6f3400640 1 -- 172.21.15.113:0/272646516 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f415b9b0 con 0x7fc6f4154f10 2024-03-04T08:23:23.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f3400640 1 --2- 172.21.15.113:0/272646516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f4154f10 0x7fc6f415e5f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009dc0 tx=0x7fc6ec01f140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64d46b7458651b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f2000640 1 -- 172.21.15.113:0/272646516 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0b4070 con 0x7fc6f4154f10 2024-03-04T08:23:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f2000640 1 -- 172.21.15.113:0/272646516 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec0af600 con 0x7fc6f4154f10 2024-03-04T08:23:23.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f2000640 1 -- 172.21.15.113:0/272646516 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0af9d0 con 0x7fc6f4154f10 2024-03-04T08:23:23.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f9c00640 1 -- 172.21.15.113:0/272646516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f4154f10 msgr2=0x7fc6f415e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:23.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f9c00640 1 --2- 172.21.15.113:0/272646516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f4154f10 0x7fc6f415e5f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009dc0 tx=0x7fc6ec01f140 comp rx=0 tx=0).stop 2024-03-04T08:23:23.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f9c00640 1 -- 172.21.15.113:0/272646516 shutdown_connections 2024-03-04T08:23:23.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f9c00640 1 --2- 172.21.15.113:0/272646516 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f4154f10 0x7fc6f415e5f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:23.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f9c00640 1 --2- 172.21.15.113:0/272646516 >> v2:172.21.15.136:3300/0 conn(0x7fc6f40ccb40 0x7fc6f40c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:23.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f9c00640 1 -- 172.21.15.113:0/272646516 >> 172.21.15.113:0/272646516 conn(0x7fc6f406b060 msgr2=0x7fc6f406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:23.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f9c00640 1 -- 172.21.15.113:0/272646516 shutdown_connections 2024-03-04T08:23:23.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.848+0000 7fc6f9c00640 1 -- 172.21.15.113:0/272646516 wait complete. 2024-03-04T08:23:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f9c00640 1 Processor -- start 2024-03-04T08:23:23.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f9c00640 1 -- start start 2024-03-04T08:23:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6f40ccb40 0x7fc6f41f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f9c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6f4154f10 0x7fc6f40c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f9c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f40c5170 0x7fc6f40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f9c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40c78b0 con 0x7fc6f4154f10 2024-03-04T08:23:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f9c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f415e0d0 con 0x7fc6f40ccb40 2024-03-04T08:23:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f9c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f40ccf20 con 0x7fc6f40c5170 2024-03-04T08:23:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f40c5170 0x7fc6f40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:23.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6f4154f10 0x7fc6f40c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f40c5170 0x7fc6f40cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:43090/0 (socket says 172.21.15.113:43090) 2024-03-04T08:23:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f3400640 1 -- 172.21.15.113:0/2268435593 learned_addr learned my addr 172.21.15.113:0/2268435593 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f2a00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6f40ccb40 0x7fc6f41f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6ebe00640 1 -- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f40c5170 msgr2=0x7fc6f40cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6ebe00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f40c5170 0x7fc6f40cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6ebe00640 1 -- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6f40ccb40 msgr2=0x7fc6f41f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6ebe00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6f40ccb40 0x7fc6f41f1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6ebe00640 1 -- 172.21.15.113:0/2268435593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f41587e0 con 0x7fc6f4154f10 2024-03-04T08:23:23.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6ebe00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6f4154f10 0x7fc6f40c60a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fc6f415b0a0 tx=0x7fc6dc00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6eaa00640 1 -- 172.21.15.113:0/2268435593 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc017070 con 0x7fc6f4154f10 2024-03-04T08:23:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6eaa00640 1 -- 172.21.15.113:0/2268435593 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc0052d0 con 0x7fc6f4154f10 2024-03-04T08:23:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6eaa00640 1 -- 172.21.15.113:0/2268435593 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc003840 con 0x7fc6f4154f10 2024-03-04T08:23:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f9c00640 1 -- 172.21.15.113:0/2268435593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f4153ed0 con 0x7fc6f4154f10 2024-03-04T08:23:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.852+0000 7fc6f9c00640 1 -- 172.21.15.113:0/2268435593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f41f0870 con 0x7fc6f4154f10 2024-03-04T08:23:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.856+0000 7fc6f9c00640 1 -- 172.21.15.113:0/2268435593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f4075c00 con 0x7fc6f4154f10 2024-03-04T08:23:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.856+0000 7fc6eaa00640 1 -- 172.21.15.113:0/2268435593 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6dc01a070 con 0x7fc6f4154f10 2024-03-04T08:23:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.856+0000 7fc6eaa00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc6d0066f90 0x7fc6d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.856+0000 7fc6eaa00640 1 -- 172.21.15.113:0/2268435593 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8233+0+0 (secure 0 0 0) 0x7fc6dc084d10 con 0x7fc6f4154f10 2024-03-04T08:23:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.856+0000 7fc6f2a00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc6d0066f90 0x7fc6d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:23.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.856+0000 7fc6f2a00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc6d0066f90 0x7fc6d0069450 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009b50 tx=0x7fc6e4009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:23.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:23.860+0000 7fc6eaa00640 1 -- 172.21.15.113:0/2268435593 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc6dc08a020 con 0x7fc6f4154f10 2024-03-04T08:23:24.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 -- 172.21.15.113:0/2268435593 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6f40c6d40 con 0x7fc6f4154f10 2024-03-04T08:23:24.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6eaa00640 1 -- 172.21.15.113:0/2268435593 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) v1 ==== 95+0+5035 (secure 0 0 0) 0x7fc6dc005480 con 0x7fc6f4154f10 2024-03-04T08:23:24.049 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:24.049 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":184,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8760,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":182}],"filesystems":[{"mdsmap":{"epoch":184,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:06.689466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":4244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4244":{"gid":4244,"name":"d","rank":2,"incarnation":180,"state":"up:rejoin","state_seq":742,"addr":"172.21.15.136:6837/2228772746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2228772746},{"type":"v1","addr":"172.21.15.136:6837","nonce":2228772746}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[8032,4247]},"id":1}]} 2024-03-04T08:23:24.049 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 184 2024-03-04T08:23:24.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 -- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc6d0066f90 msgr2=0x7fc6d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:24.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc6d0066f90 0x7fc6d0069450 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009b50 tx=0x7fc6e4009240 comp rx=0 tx=0).stop 2024-03-04T08:23:24.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 -- 172.21.15.113:0/2268435593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6f4154f10 msgr2=0x7fc6f40c60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:24.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6f4154f10 0x7fc6f40c60a0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fc6f415b0a0 tx=0x7fc6dc00c820 comp rx=0 tx=0).stop 2024-03-04T08:23:24.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 -- 172.21.15.113:0/2268435593 shutdown_connections 2024-03-04T08:23:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6f40c5170 0x7fc6f40cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc6d0066f90 0x7fc6d0069450 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6f4154f10 0x7fc6f40c60a0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 --2- 172.21.15.113:0/2268435593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6f40ccb40 0x7fc6f41f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.052+0000 7fc6f9c00640 1 -- 172.21.15.113:0/2268435593 >> 172.21.15.113:0/2268435593 conn(0x7fc6f406b060 msgr2=0x7fc6f414cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.056+0000 7fc6f9c00640 1 -- 172.21.15.113:0/2268435593 shutdown_connections 2024-03-04T08:23:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:24.056+0000 7fc6f9c00640 1 -- 172.21.15.113:0/2268435593 wait complete. 2024-03-04T08:23:24.070 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2024-03-04T08:23:26.072 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:26.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.216+0000 7f8cd0c00640 1 Processor -- start 2024-03-04T08:23:26.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.216+0000 7f8cd0c00640 1 -- start start 2024-03-04T08:23:26.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cd0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc1527f0 0x7f8ccc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cd0c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8ccc152bd0 0x7f8ccc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cd0c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ccc1591f0 con 0x7f8ccc0c8200 2024-03-04T08:23:26.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cd0c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc071df0 con 0x7f8ccc152bd0 2024-03-04T08:23:26.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cd0c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc1597b0 con 0x7f8ccc1527f0 2024-03-04T08:23:26.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cc9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc1527f0 0x7f8ccc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cc3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8ccc0c8200 0x7f8ccc072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44386/0 (socket says 172.21.15.113:44386) 2024-03-04T08:23:26.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cc3e00640 1 -- 172.21.15.113:0/2679716439 learned_addr learned my addr 172.21.15.113:0/2679716439 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:23:26.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cca800640 1 --2- 172.21.15.113:0/2679716439 >> v2:172.21.15.136:3300/0 conn(0x7f8ccc152bd0 0x7f8ccc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cc9400640 1 -- 172.21.15.113:0/2679716439 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840462085 0 0) 0x7f8ccc1591f0 con 0x7f8ccc0c8200 2024-03-04T08:23:26.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cc9400640 1 -- 172.21.15.113:0/2679716439 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cb0003530 con 0x7f8ccc0c8200 2024-03-04T08:23:26.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cc9e00640 1 -- 172.21.15.113:0/2679716439 >> v2:172.21.15.136:3300/0 conn(0x7f8ccc152bd0 msgr2=0x7f8ccc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:26.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cc9e00640 1 --2- 172.21.15.113:0/2679716439 >> v2:172.21.15.136:3300/0 conn(0x7f8ccc152bd0 0x7f8ccc1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cc9e00640 1 -- 172.21.15.113:0/2679716439 >> v1:172.21.15.113:6789/0 conn(0x7f8ccc0c8200 legacy=0x7f8ccc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.220+0000 7f8cc9e00640 1 -- 172.21.15.113:0/2679716439 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc1594d0 con 0x7f8ccc1527f0 2024-03-04T08:23:26.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cc9e00640 1 --2- 172.21.15.113:0/2679716439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc1527f0 0x7f8ccc070d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0097c0 tx=0x7f8cbc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f00e3b82acb075a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:26.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cc9400640 1 -- 172.21.15.113:0/2679716439 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc0b1070 con 0x7f8ccc1527f0 2024-03-04T08:23:26.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cc9400640 1 -- 172.21.15.113:0/2679716439 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc0ac410 con 0x7f8ccc1527f0 2024-03-04T08:23:26.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cc9400640 1 -- 172.21.15.113:0/2679716439 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc0ac7c0 con 0x7f8ccc1527f0 2024-03-04T08:23:26.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cd0c00640 1 -- 172.21.15.113:0/2679716439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc1527f0 msgr2=0x7f8ccc070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cd0c00640 1 --2- 172.21.15.113:0/2679716439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc1527f0 0x7f8ccc070d40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0097c0 tx=0x7f8cbc01c4a0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cd0c00640 1 -- 172.21.15.113:0/2679716439 shutdown_connections 2024-03-04T08:23:26.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cd0c00640 1 --2- 172.21.15.113:0/2679716439 >> v2:172.21.15.136:3300/0 conn(0x7f8ccc152bd0 0x7f8ccc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cd0c00640 1 --2- 172.21.15.113:0/2679716439 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc1527f0 0x7f8ccc070d40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cd0c00640 1 -- 172.21.15.113:0/2679716439 >> 172.21.15.113:0/2679716439 conn(0x7f8ccc06e820 msgr2=0x7f8ccc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:26.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cd0c00640 1 -- 172.21.15.113:0/2679716439 shutdown_connections 2024-03-04T08:23:26.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cd0c00640 1 -- 172.21.15.113:0/2679716439 wait complete. 2024-03-04T08:23:26.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cd0c00640 1 Processor -- start 2024-03-04T08:23:26.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.224+0000 7f8cd0c00640 1 -- start start 2024-03-04T08:23:26.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cd0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cd0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ccc1527f0 0x7f8ccc14ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cd0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc152bd0 0x7f8ccc14fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cd0c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc1654f0 con 0x7f8ccc1527f0 2024-03-04T08:23:26.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cd0c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc165370 con 0x7f8ccc0c8200 2024-03-04T08:23:26.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cd0c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc165670 con 0x7f8ccc152bd0 2024-03-04T08:23:26.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ccc1527f0 0x7f8ccc14ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:52270/0 (socket says 172.21.15.113:52270) 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc9e00640 1 -- 172.21.15.113:0/3802573916 learned_addr learned my addr 172.21.15.113:0/3802573916 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cca800640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc152bd0 0x7f8ccc14fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc9e00640 1 -- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc152bd0 msgr2=0x7f8ccc14fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc9e00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc152bd0 0x7f8ccc14fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc9e00640 1 -- 172.21.15.113:0/3802573916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ccc1527f0 msgr2=0x7f8ccc14ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc9e00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ccc1527f0 0x7f8ccc14ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc9e00640 1 -- 172.21.15.113:0/3802573916 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc0c9480 con 0x7f8ccc0c8200 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc3e00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ccc1527f0 0x7f8ccc14ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc9e00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0c8dd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc01ab50 tx=0x7f8cbc019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc2000640 1 -- 172.21.15.113:0/3802573916 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc0b1040 con 0x7f8ccc0c8200 2024-03-04T08:23:26.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc2000640 1 -- 172.21.15.113:0/3802573916 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc0ba070 con 0x7f8ccc0c8200 2024-03-04T08:23:26.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc2000640 1 -- 172.21.15.113:0/3802573916 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc003920 con 0x7f8ccc0c8200 2024-03-04T08:23:26.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cd0c00640 1 -- 172.21.15.113:0/3802573916 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc155780 con 0x7f8ccc0c8200 2024-03-04T08:23:26.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cd0c00640 1 -- 172.21.15.113:0/3802573916 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc155d40 con 0x7f8ccc0c8200 2024-03-04T08:23:26.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cd0c00640 1 -- 172.21.15.113:0/3802573916 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc0c79d0 con 0x7f8ccc0c8200 2024-03-04T08:23:26.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.228+0000 7f8cc2000640 1 -- 172.21.15.113:0/3802573916 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8cbc003ac0 con 0x7f8ccc0c8200 2024-03-04T08:23:26.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.232+0000 7f8cc2000640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ca8066f90 0x7f8ca8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.232+0000 7f8cc2000640 1 -- 172.21.15.113:0/3802573916 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7f8cbc11f550 con 0x7f8ccc0c8200 2024-03-04T08:23:26.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.232+0000 7f8cc3e00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ca8066f90 0x7f8ca8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.232+0000 7f8cc3e00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ca8066f90 0x7f8ca8069450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0c13a0 tx=0x7f8cb4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:26.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.236+0000 7f8cc2000640 1 -- 172.21.15.113:0/3802573916 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8cbc016090 con 0x7f8ccc0c8200 2024-03-04T08:23:26.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 -- 172.21.15.113:0/3802573916 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ccc1f71a0 con 0x7f8ccc0c8200 2024-03-04T08:23:26.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cc2000640 1 -- 172.21.15.113:0/3802573916 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+4239 (secure 0 0 0) 0x7f8cbc0ef540 con 0x7f8ccc0c8200 2024-03-04T08:23:26.420 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:26.421 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":185,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:26.011632+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:23:26.421 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 185 2024-03-04T08:23:26.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 -- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ca8066f90 msgr2=0x7f8ca8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ca8066f90 0x7f8ca8069450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0c13a0 tx=0x7f8cb4009040 comp rx=0 tx=0).stop 2024-03-04T08:23:26.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 -- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ccc0c8200 msgr2=0x7f8ccc0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0c8dd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc01ab50 tx=0x7f8cbc019240 comp rx=0 tx=0).stop 2024-03-04T08:23:26.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 -- 172.21.15.113:0/3802573916 shutdown_connections 2024-03-04T08:23:26.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ccc152bd0 0x7f8ccc14fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8ca8066f90 0x7f8ca8069450 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ccc1527f0 0x7f8ccc14ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 --2- 172.21.15.113:0/3802573916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ccc0c8200 0x7f8ccc0c8dd0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 -- 172.21.15.113:0/3802573916 >> 172.21.15.113:0/3802573916 conn(0x7f8ccc06e820 msgr2=0x7f8ccc0c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:26.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 -- 172.21.15.113:0/3802573916 shutdown_connections 2024-03-04T08:23:26.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.424+0000 7f8cd0c00640 1 -- 172.21.15.113:0/3802573916 wait complete. 2024-03-04T08:23:26.441 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2024-03-04T08:23:26.441 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:23:26.441 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:26.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f5715200640 1 Processor -- start 2024-03-04T08:23:26.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f5715200640 1 -- start start 2024-03-04T08:23:26.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f5715200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5710154c00 0x7f57100c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f5715200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5710154fe0 0x7f57100c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f5715200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57100c7d80 con 0x7f57100c7700 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f5715200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571015efb0 con 0x7f5710154fe0 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f5715200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100c6b70 con 0x7f5710154c00 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f5707e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5710154fe0 0x7f57100c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f5707e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5710154fe0 0x7f57100c1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50904/0 (socket says 172.21.15.113:50904) 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f5707e00640 1 -- 172.21.15.113:0/4219045679 learned_addr learned my addr 172.21.15.113:0/4219045679 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f570e800640 1 --1- 172.21.15.113:0/4219045679 >> v1:172.21.15.113:6789/0 conn(0x7f57100c7700 0x7f571015abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:44388/0 (socket says 172.21.15.113:44388) 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.580+0000 7f570de00640 1 --2- 172.21.15.113:0/4219045679 >> v2:172.21.15.136:3300/0 conn(0x7f5710154c00 0x7f57100c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5707e00640 1 -- 172.21.15.113:0/4219045679 >> v2:172.21.15.136:3300/0 conn(0x7f5710154c00 msgr2=0x7f57100c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:26.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5707e00640 1 --2- 172.21.15.113:0/4219045679 >> v2:172.21.15.136:3300/0 conn(0x7f5710154c00 0x7f57100c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5707e00640 1 -- 172.21.15.113:0/4219045679 >> v1:172.21.15.113:6789/0 conn(0x7f57100c7700 legacy=0x7f571015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5707e00640 1 -- 172.21.15.113:0/4219045679 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100c8060 con 0x7f5710154fe0 2024-03-04T08:23:26.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5707e00640 1 --2- 172.21.15.113:0/4219045679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 0x7f57100c1b50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f57100cd200 tx=0x7f56f801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd11fed17ca84d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:26.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f570d400640 1 -- 172.21.15.113:0/4219045679 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80b2070 con 0x7f5710154fe0 2024-03-04T08:23:26.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f570d400640 1 -- 172.21.15.113:0/4219045679 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f80ad680 con 0x7f5710154fe0 2024-03-04T08:23:26.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f570d400640 1 -- 172.21.15.113:0/4219045679 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80ada50 con 0x7f5710154fe0 2024-03-04T08:23:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5715200640 1 -- 172.21.15.113:0/4219045679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 msgr2=0x7f57100c1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5715200640 1 --2- 172.21.15.113:0/4219045679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 0x7f57100c1b50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f57100cd200 tx=0x7f56f801c490 comp rx=0 tx=0).stop 2024-03-04T08:23:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5715200640 1 -- 172.21.15.113:0/4219045679 shutdown_connections 2024-03-04T08:23:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5715200640 1 --2- 172.21.15.113:0/4219045679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 0x7f57100c1b50 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5715200640 1 --2- 172.21.15.113:0/4219045679 >> v2:172.21.15.136:3300/0 conn(0x7f5710154c00 0x7f57100c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5715200640 1 -- 172.21.15.113:0/4219045679 >> 172.21.15.113:0/4219045679 conn(0x7f571006e820 msgr2=0x7f571006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.584+0000 7f5715200640 1 -- 172.21.15.113:0/4219045679 shutdown_connections 2024-03-04T08:23:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 -- 172.21.15.113:0/4219045679 wait complete. 2024-03-04T08:23:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 Processor -- start 2024-03-04T08:23:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 -- start start 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57100c7700 0x7f571015cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5710154c00 0x7f57100c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 0x7f57101f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100c2460 con 0x7f5710154fe0 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57100c22e0 con 0x7f57100c7700 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5710167c10 con 0x7f5710154c00 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 0x7f57101f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5707e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5710154c00 0x7f57100c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 0x7f57101f7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50908/0 (socket says 172.21.15.113:50908) 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570e800640 1 -- 172.21.15.113:0/745435551 learned_addr learned my addr 172.21.15.113:0/745435551 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570de00640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57100c7700 0x7f571015cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570e800640 1 -- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5710154c00 msgr2=0x7f57100c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570e800640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5710154c00 0x7f57100c36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570e800640 1 -- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57100c7700 msgr2=0x7f571015cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570e800640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57100c7700 0x7f571015cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570e800640 1 -- 172.21.15.113:0/745435551 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57101f3d90 con 0x7f5710154fe0 2024-03-04T08:23:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570de00640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57100c7700 0x7f571015cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:23:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f570e800640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 0x7f57101f7300 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f570800c6f0 tx=0x7f570800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5706000640 1 -- 172.21.15.113:0/745435551 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708017070 con 0x7f5710154fe0 2024-03-04T08:23:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5706000640 1 -- 172.21.15.113:0/745435551 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5708005db0 con 0x7f5710154fe0 2024-03-04T08:23:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5706000640 1 -- 172.21.15.113:0/745435551 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570800a840 con 0x7f5710154fe0 2024-03-04T08:23:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 -- 172.21.15.113:0/745435551 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571015bd60 con 0x7f5710154fe0 2024-03-04T08:23:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.588+0000 7f5715200640 1 -- 172.21.15.113:0/745435551 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57101f4050 con 0x7f5710154fe0 2024-03-04T08:23:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.592+0000 7f5715200640 1 -- 172.21.15.113:0/745435551 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5710075c00 con 0x7f5710154fe0 2024-03-04T08:23:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.592+0000 7f5706000640 1 -- 172.21.15.113:0/745435551 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f570801a070 con 0x7f5710154fe0 2024-03-04T08:23:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.592+0000 7f5706000640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56ec066f60 0x7f56ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.592+0000 7f5706000640 1 -- 172.21.15.113:0/745435551 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7f5708084f30 con 0x7f5710154fe0 2024-03-04T08:23:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.592+0000 7f570de00640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56ec066f60 0x7f56ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:26.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.592+0000 7f570de00640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56ec066f60 0x7f56ec069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5700009b10 tx=0x7f5700008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:26.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.596+0000 7f5706000640 1 -- 172.21.15.113:0/745435551 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5708054e10 con 0x7f5710154fe0 2024-03-04T08:23:26.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.784+0000 7f5715200640 1 -- 172.21.15.113:0/745435551 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5710158b60 con 0x7f5710154fe0 2024-03-04T08:23:26.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.784+0000 7f5706000640 1 -- 172.21.15.113:0/745435551 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+4239 (secure 0 0 0) 0x7f5708058eb0 con 0x7f5710154fe0 2024-03-04T08:23:26.782 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:26.782 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":185,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:26.011632+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:23:26.782 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 185 2024-03-04T08:23:26.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 -- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56ec066f60 msgr2=0x7f56ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56ec066f60 0x7f56ec069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5700009b10 tx=0x7f5700008040 comp rx=0 tx=0).stop 2024-03-04T08:23:26.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 -- 172.21.15.113:0/745435551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 msgr2=0x7f57101f7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:26.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 0x7f57101f7300 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f570800c6f0 tx=0x7f570800cbb0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 -- 172.21.15.113:0/745435551 shutdown_connections 2024-03-04T08:23:26.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f56ec066f60 0x7f56ec069420 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5710154fe0 0x7f57101f7300 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5710154c00 0x7f57100c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 --2- 172.21.15.113:0/745435551 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f57100c7700 0x7f571015cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:26.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 -- 172.21.15.113:0/745435551 >> 172.21.15.113:0/745435551 conn(0x7f571006e820 msgr2=0x7f5710159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:26.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 -- 172.21.15.113:0/745435551 shutdown_connections 2024-03-04T08:23:26.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:26.788+0000 7f5715200640 1 -- 172.21.15.113:0/745435551 wait complete. 2024-03-04T08:23:26.803 INFO:tasks.mds_thrash.fs.[cephfs]:cluster has 2 actives (max_mds is 2), no MDS can replace rank 2 2024-03-04T08:23:26.804 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 3 secs before reviving mds.d 2024-03-04T08:23:29.804 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2024-03-04T08:23:29.804 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-04T08:23:29.805 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-04T08:23:29.807 INFO:tasks.ceph.mds.d:Started 2024-03-04T08:23:29.807 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:29.875 INFO:tasks.ceph.mds.d.smithi136.stdout:starting mds.d at 2024-03-04T08:23:29.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b7c000640 1 Processor -- start 2024-03-04T08:23:29.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b7c000640 1 -- start start 2024-03-04T08:23:29.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b7c000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b7414e4c0 0x7f7b74072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:29.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b7c000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7b740c22e0 0x7f7b740731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b7c000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b740c27f0 con 0x7f7b740c2d80 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b7c000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b7415ad60 con 0x7f7b740c22e0 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b7c000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740c2a90 con 0x7f7b7414e4c0 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b79600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b7414e4c0 0x7f7b74072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b79600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b7414e4c0 0x7f7b74072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47352/0 (socket says 172.21.15.113:47352) 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b78c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7b740c22e0 0x7f7b740731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b79600640 1 -- 172.21.15.113:0/1220752764 learned_addr learned my addr 172.21.15.113:0/1220752764 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b79600640 1 -- 172.21.15.113:0/1220752764 >> v2:172.21.15.136:3300/0 conn(0x7f7b740c22e0 msgr2=0x7f7b740731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b79600640 1 --2- 172.21.15.113:0/1220752764 >> v2:172.21.15.136:3300/0 conn(0x7f7b740c22e0 0x7f7b740731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b79600640 1 -- 172.21.15.113:0/1220752764 >> v1:172.21.15.136:6789/0 conn(0x7f7b740c2d80 legacy=0x7f7b7415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:29.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.956+0000 7f7b79600640 1 -- 172.21.15.113:0/1220752764 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74070c60 con 0x7f7b7414e4c0 2024-03-04T08:23:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b79600640 1 --2- 172.21.15.113:0/1220752764 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b7414e4c0 0x7f7b74072c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800d150 tx=0x7f7b68021420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbd2358fdc300011 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b73e00640 1 -- 172.21.15.113:0/1220752764 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680b6070 con 0x7f7b7414e4c0 2024-03-04T08:23:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b73e00640 1 -- 172.21.15.113:0/1220752764 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6800b2d0 con 0x7f7b7414e4c0 2024-03-04T08:23:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b73e00640 1 -- 172.21.15.113:0/1220752764 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800b480 con 0x7f7b7414e4c0 2024-03-04T08:23:29.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b7c000640 1 -- 172.21.15.113:0/1220752764 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b7414e4c0 msgr2=0x7f7b74072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b7c000640 1 --2- 172.21.15.113:0/1220752764 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b7414e4c0 0x7f7b74072c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7b6800d150 tx=0x7f7b68021420 comp rx=0 tx=0).stop 2024-03-04T08:23:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b7c000640 1 -- 172.21.15.113:0/1220752764 shutdown_connections 2024-03-04T08:23:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b7c000640 1 --2- 172.21.15.113:0/1220752764 >> v2:172.21.15.136:3300/0 conn(0x7f7b740c22e0 0x7f7b740731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b7c000640 1 --2- 172.21.15.113:0/1220752764 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b7414e4c0 0x7f7b74072c80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b7c000640 1 -- 172.21.15.113:0/1220752764 >> 172.21.15.113:0/1220752764 conn(0x7f7b7406e820 msgr2=0x7f7b7406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b7c000640 1 -- 172.21.15.113:0/1220752764 shutdown_connections 2024-03-04T08:23:29.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.960+0000 7f7b7c000640 1 -- 172.21.15.113:0/1220752764 wait complete. 2024-03-04T08:23:29.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7c000640 1 Processor -- start 2024-03-04T08:23:29.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7c000640 1 -- start start 2024-03-04T08:23:29.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7c000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b740c22e0 0x7f7b74157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:29.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7c000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b740c2d80 0x7f7b741553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:29.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7c000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b7414e4c0 0x7f7b74154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:29.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7c000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74071500 con 0x7f7b7414e4c0 2024-03-04T08:23:29.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7c000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74071220 con 0x7f7b740c2d80 2024-03-04T08:23:29.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7c000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7414e8a0 con 0x7f7b740c22e0 2024-03-04T08:23:29.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7a000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b7414e4c0 0x7f7b74154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b79600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b740c22e0 0x7f7b74157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b79600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b740c22e0 0x7f7b74157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47368/0 (socket says 172.21.15.113:47368) 2024-03-04T08:23:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b78c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b740c2d80 0x7f7b741553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:29.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b79600640 1 -- 172.21.15.113:0/2686585412 learned_addr learned my addr 172.21.15.113:0/2686585412 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b79600640 1 -- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b740c2d80 msgr2=0x7f7b741553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b79600640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b740c2d80 0x7f7b741553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b79600640 1 -- 172.21.15.113:0/2686585412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b7414e4c0 msgr2=0x7f7b74154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b79600640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b7414e4c0 0x7f7b74154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b79600640 1 -- 172.21.15.113:0/2686585412 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b740c5050 con 0x7f7b740c22e0 2024-03-04T08:23:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7a000640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b7414e4c0 0x7f7b74154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:23:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b79600640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b740c22e0 0x7f7b74157fb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7b680b4040 tx=0x7f7b6801e1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:29.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b71600640 1 -- 172.21.15.113:0/2686585412 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680b6070 con 0x7f7b740c22e0 2024-03-04T08:23:29.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7c000640 1 -- 172.21.15.113:0/2686585412 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b740c52f0 con 0x7f7b740c22e0 2024-03-04T08:23:29.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b71600640 1 -- 172.21.15.113:0/2686585412 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b680bc420 con 0x7f7b740c22e0 2024-03-04T08:23:29.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b71600640 1 -- 172.21.15.113:0/2686585412 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800b480 con 0x7f7b740c22e0 2024-03-04T08:23:29.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.964+0000 7f7b7c000640 1 -- 172.21.15.113:0/2686585412 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b740c58b0 con 0x7f7b740c22e0 2024-03-04T08:23:29.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.968+0000 7f7b7c000640 1 -- 172.21.15.113:0/2686585412 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b74075c00 con 0x7f7b740c22e0 2024-03-04T08:23:29.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.968+0000 7f7b71600640 1 -- 172.21.15.113:0/2686585412 <== mon.2 v2:172.21.15.136:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b680ba030 con 0x7f7b740c22e0 2024-03-04T08:23:29.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.968+0000 7f7b71600640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7b54066f60 0x7f7b54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:29.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.968+0000 7f7b71600640 1 -- 172.21.15.113:0/2686585412 <== mon.2 v2:172.21.15.136:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7f7b681236b0 con 0x7f7b740c22e0 2024-03-04T08:23:29.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.968+0000 7f7b78c00640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7b54066f60 0x7f7b54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:29.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.972+0000 7f7b78c00640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7b54066f60 0x7f7b54069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7b7414bc60 tx=0x7f7b60009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:29.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:29.980+0000 7f7b71600640 1 -- 172.21.15.113:0/2686585412 <== mon.2 v2:172.21.15.136:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7b6801aef0 con 0x7f7b740c22e0 2024-03-04T08:23:30.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.168+0000 7f7b7c000640 1 -- 172.21.15.113:0/2686585412 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b74155190 con 0x7f7b740c22e0 2024-03-04T08:23:30.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b71600640 1 -- 172.21.15.113:0/2686585412 <== mon.2 v2:172.21.15.136:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+4239 (secure 0 0 0) 0x7f7b680f3590 con 0x7f7b740c22e0 2024-03-04T08:23:30.167 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:30.167 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":185,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:26.011632+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:replay","state_seq":1,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:23:30.167 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 185 2024-03-04T08:23:30.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 -- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7b54066f60 msgr2=0x7f7b54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:30.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7b54066f60 0x7f7b54069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7b7414bc60 tx=0x7f7b60009190 comp rx=0 tx=0).stop 2024-03-04T08:23:30.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 -- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b740c22e0 msgr2=0x7f7b74157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:30.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b740c22e0 0x7f7b74157fb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7b680b4040 tx=0x7f7b6801e1c0 comp rx=0 tx=0).stop 2024-03-04T08:23:30.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 -- 172.21.15.113:0/2686585412 shutdown_connections 2024-03-04T08:23:30.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7b54066f60 0x7f7b54069420 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:30.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b7414e4c0 0x7f7b74154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:30.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b740c2d80 0x7f7b741553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:30.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 --2- 172.21.15.113:0/2686585412 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7b740c22e0 0x7f7b74157fb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:30.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 -- 172.21.15.113:0/2686585412 >> 172.21.15.113:0/2686585412 conn(0x7f7b7406e820 msgr2=0x7f7b7414c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:30.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 -- 172.21.15.113:0/2686585412 shutdown_connections 2024-03-04T08:23:30.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:30.172+0000 7f7b7c000640 1 -- 172.21.15.113:0/2686585412 wait complete. 2024-03-04T08:23:30.188 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2024-03-04T08:23:32.189 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:23:32.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.336+0000 7f6281400640 1 Processor -- start 2024-03-04T08:23:32.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.336+0000 7f6281400640 1 -- start start 2024-03-04T08:23:32.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f6281400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f627c160820 0x7f627c162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:32.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f6281400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627c0c1ec0 con 0x7f627c0c85f0 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f6281400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627c0c9b30 con 0x7f627c0b20b0 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f6281400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627c0c12a0 con 0x7f627c160820 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f6271e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f627c0c85f0 0x7f627c06b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56660/0 (socket says 172.21.15.113:56660) 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f6271e00640 1 -- 172.21.15.113:0/1190999952 learned_addr learned my addr 172.21.15.113:0/1190999952 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f627b200640 1 --2- 172.21.15.113:0/1190999952 >> v2:172.21.15.136:3300/0 conn(0x7f627c160820 0x7f627c162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f6279e00640 1 -- 172.21.15.113:0/1190999952 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922341294 0 0) 0x7f627c0c1ec0 con 0x7f627c0c85f0 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f6279e00640 1 -- 172.21.15.113:0/1190999952 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6260003530 con 0x7f627c0c85f0 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f627b200640 1 -- 172.21.15.113:0/1190999952 >> v1:172.21.15.136:6789/0 conn(0x7f627c0b20b0 legacy=0x7f627c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f6279e00640 1 -- 172.21.15.113:0/1190999952 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1020769255 0 0) 0x7f6260003530 con 0x7f627c0c85f0 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f627b200640 1 -- 172.21.15.113:0/1190999952 >> v1:172.21.15.113:6789/0 conn(0x7f627c0c85f0 legacy=0x7f627c06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:32.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.340+0000 7f627b200640 1 -- 172.21.15.113:0/1190999952 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c0c1580 con 0x7f627c160820 2024-03-04T08:23:32.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f627b200640 1 --2- 172.21.15.113:0/1190999952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f627c160820 0x7f627c162c10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f62740097c0 tx=0x7f627401ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5618f184a6cdfcb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:32.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6279e00640 1 -- 172.21.15.113:0/1190999952 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62740b5070 con 0x7f627c160820 2024-03-04T08:23:32.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6279e00640 1 -- 172.21.15.113:0/1190999952 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62740a8630 con 0x7f627c160820 2024-03-04T08:23:32.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6279e00640 1 -- 172.21.15.113:0/1190999952 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640016a0 con 0x7f627c160820 2024-03-04T08:23:32.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6281400640 1 -- 172.21.15.113:0/1190999952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f627c160820 msgr2=0x7f627c162c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:32.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6281400640 1 --2- 172.21.15.113:0/1190999952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f627c160820 0x7f627c162c10 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f62740097c0 tx=0x7f627401ebb0 comp rx=0 tx=0).stop 2024-03-04T08:23:32.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6281400640 1 -- 172.21.15.113:0/1190999952 shutdown_connections 2024-03-04T08:23:32.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6281400640 1 --2- 172.21.15.113:0/1190999952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f627c160820 0x7f627c162c10 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:32.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6281400640 1 -- 172.21.15.113:0/1190999952 >> 172.21.15.113:0/1190999952 conn(0x7f627c06e820 msgr2=0x7f627c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:32.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6281400640 1 -- 172.21.15.113:0/1190999952 shutdown_connections 2024-03-04T08:23:32.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6281400640 1 -- 172.21.15.113:0/1190999952 wait complete. 2024-03-04T08:23:32.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6281400640 1 Processor -- start 2024-03-04T08:23:32.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.344+0000 7f6281400640 1 -- start start 2024-03-04T08:23:32.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6281400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f627c0b20b0 0x7f627c0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:32.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6281400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c85f0 0x7f627c1f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:32.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6281400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f627c160820 0x7f627c1e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:32.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6281400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c157e20 con 0x7f627c0c85f0 2024-03-04T08:23:32.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6281400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c1571f0 con 0x7f627c160820 2024-03-04T08:23:32.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6281400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c0c2400 con 0x7f627c0b20b0 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6271e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c85f0 0x7f627c1f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6271e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c85f0 0x7f627c1f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45764/0 (socket says 172.21.15.113:45764) 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6271e00640 1 -- 172.21.15.113:0/4148073357 learned_addr learned my addr 172.21.15.113:0/4148073357 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f627b200640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f627c160820 0x7f627c1e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f627a800640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f627c0b20b0 0x7f627c0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6271e00640 1 -- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f627c0b20b0 msgr2=0x7f627c0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6271e00640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f627c0b20b0 0x7f627c0cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6271e00640 1 -- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f627c160820 msgr2=0x7f627c1e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6271e00640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f627c160820 0x7f627c1e5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6271e00640 1 -- 172.21.15.113:0/4148073357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c146650 con 0x7f627c0c85f0 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6271e00640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c85f0 0x7f627c1f3f40 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f626400c760 tx=0x7f626400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6273400640 1 -- 172.21.15.113:0/4148073357 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6264018070 con 0x7f627c0c85f0 2024-03-04T08:23:32.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6273400640 1 -- 172.21.15.113:0/4148073357 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62640072f0 con 0x7f627c0c85f0 2024-03-04T08:23:32.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6281400640 1 -- 172.21.15.113:0/4148073357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627c14f8d0 con 0x7f627c0c85f0 2024-03-04T08:23:32.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6273400640 1 -- 172.21.15.113:0/4148073357 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62640047f0 con 0x7f627c0c85f0 2024-03-04T08:23:32.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6281400640 1 -- 172.21.15.113:0/4148073357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c0ca180 con 0x7f627c0c85f0 2024-03-04T08:23:32.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6281400640 1 -- 172.21.15.113:0/4148073357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c075c00 con 0x7f627c0c85f0 2024-03-04T08:23:32.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.348+0000 7f6273400640 1 -- 172.21.15.113:0/4148073357 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6264004990 con 0x7f627c0c85f0 2024-03-04T08:23:32.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.352+0000 7f6273400640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:23:32.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.352+0000 7f6273400640 1 -- 172.21.15.113:0/4148073357 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7f6264084de0 con 0x7f627c0c85f0 2024-03-04T08:23:32.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.352+0000 7f627a800640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:23:32.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.352+0000 7f627a800640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6258066f60 0x7f6258069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f626c0012f0 tx=0x7f626c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:23:32.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.356+0000 7f6273400640 1 -- 172.21.15.113:0/4148073357 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6264054d30 con 0x7f627c0c85f0 2024-03-04T08:23:32.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.544+0000 7f6281400640 1 -- 172.21.15.113:0/4148073357 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f627c072f10 con 0x7f627c0c85f0 2024-03-04T08:23:32.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.544+0000 7f6273400640 1 -- 172.21.15.113:0/4148073357 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+5034 (secure 0 0 0) 0x7f626405b080 con 0x7f627c0c85f0 2024-03-04T08:23:32.542 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:23:32.542 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":187,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":186}],"filesystems":[{"mdsmap":{"epoch":187,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:31.025171+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:resolve","state_seq":9,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:23:32.542 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 187 2024-03-04T08:23:32.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 -- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6258066f60 msgr2=0x7f6258069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:32.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6258066f60 0x7f6258069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f626c0012f0 tx=0x7f626c009040 comp rx=0 tx=0).stop 2024-03-04T08:23:32.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 -- 172.21.15.113:0/4148073357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c85f0 msgr2=0x7f627c1f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:23:32.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c85f0 0x7f627c1f3f40 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f626400c760 tx=0x7f626400cc20 comp rx=0 tx=0).stop 2024-03-04T08:23:32.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 -- 172.21.15.113:0/4148073357 shutdown_connections 2024-03-04T08:23:32.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f627c160820 0x7f627c1e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:32.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6258066f60 0x7f6258069420 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:32.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f627c0c85f0 0x7f627c1f3f40 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:32.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 --2- 172.21.15.113:0/4148073357 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f627c0b20b0 0x7f627c0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:23:32.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 -- 172.21.15.113:0/4148073357 >> 172.21.15.113:0/4148073357 conn(0x7f627c06e820 msgr2=0x7f627c148030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:23:32.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 -- 172.21.15.113:0/4148073357 shutdown_connections 2024-03-04T08:23:32.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:23:32.548+0000 7f6281400640 1 -- 172.21.15.113:0/4148073357 wait complete. 2024-03-04T08:23:32.563 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2024-03-04T08:23:32.563 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 38 secs before thrashing 2024-03-04T08:23:45.035 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:23:45.039 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:24:10.564 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.711+0000 7f2deac00640 1 Processor -- start 2024-03-04T08:24:10.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.711+0000 7f2deac00640 1 -- start start 2024-03-04T08:24:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2deac00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 0x7f2de4072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2deac00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de40c27f0 con 0x7f2de415b900 2024-03-04T08:24:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2deac00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de40c2a90 con 0x7f2de40c22e0 2024-03-04T08:24:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2deac00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de415ad60 con 0x7f2de414e4c0 2024-03-04T08:24:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de8c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f2de415b900 0x7f2de41620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59476/0 (socket says 172.21.15.113:59476) 2024-03-04T08:24:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de8c00640 1 -- 172.21.15.113:0/2971692361 learned_addr learned my addr 172.21.15.113:0/2971692361 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:10.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de3e00640 1 --2- 172.21.15.113:0/2971692361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 0x7f2de4072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de3400640 1 -- 172.21.15.113:0/2971692361 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522308037 0 0) 0x7f2de40c27f0 con 0x7f2de415b900 2024-03-04T08:24:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de3400640 1 -- 172.21.15.113:0/2971692361 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dc8005740 con 0x7f2de415b900 2024-03-04T08:24:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de3400640 1 -- 172.21.15.113:0/2971692361 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193295788 0 0) 0x7f2de40c2a90 con 0x7f2de40c22e0 2024-03-04T08:24:10.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de3400640 1 -- 172.21.15.113:0/2971692361 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2de40c27f0 con 0x7f2de40c22e0 2024-03-04T08:24:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de3e00640 1 -- 172.21.15.113:0/2971692361 >> v1:172.21.15.136:6789/0 conn(0x7f2de40c22e0 legacy=0x7f2de415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de3e00640 1 -- 172.21.15.113:0/2971692361 >> v1:172.21.15.113:6789/0 conn(0x7f2de415b900 legacy=0x7f2de41620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de3400640 1 -- 172.21.15.113:0/2971692361 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 742740739 0 0) 0x7f2dc8005740 con 0x7f2de415b900 2024-03-04T08:24:10.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.715+0000 7f2de3e00640 1 -- 172.21.15.113:0/2971692361 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4070c60 con 0x7f2de414e4c0 2024-03-04T08:24:10.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2de3e00640 1 --2- 172.21.15.113:0/2971692361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 0x7f2de4072c80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009820 tx=0x7f2dd401ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ceee9c2fee8e08 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:10.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2de3400640 1 -- 172.21.15.113:0/2971692361 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40b5070 con 0x7f2de414e4c0 2024-03-04T08:24:10.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2de3400640 1 -- 172.21.15.113:0/2971692361 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd40a8690 con 0x7f2de414e4c0 2024-03-04T08:24:10.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2de3400640 1 -- 172.21.15.113:0/2971692361 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc001a00 con 0x7f2de414e4c0 2024-03-04T08:24:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2deac00640 1 -- 172.21.15.113:0/2971692361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 msgr2=0x7f2de4072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2deac00640 1 --2- 172.21.15.113:0/2971692361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 0x7f2de4072c80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009820 tx=0x7f2dd401ebd0 comp rx=0 tx=0).stop 2024-03-04T08:24:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2deac00640 1 -- 172.21.15.113:0/2971692361 shutdown_connections 2024-03-04T08:24:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2deac00640 1 --2- 172.21.15.113:0/2971692361 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 0x7f2de4072c80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2deac00640 1 -- 172.21.15.113:0/2971692361 >> 172.21.15.113:0/2971692361 conn(0x7f2de406e820 msgr2=0x7f2de406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2deac00640 1 -- 172.21.15.113:0/2971692361 shutdown_connections 2024-03-04T08:24:10.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2deac00640 1 -- 172.21.15.113:0/2971692361 wait complete. 2024-03-04T08:24:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2deac00640 1 Processor -- start 2024-03-04T08:24:10.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.719+0000 7f2deac00640 1 -- start start 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2deac00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de40c22e0 0x7f2de40c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2deac00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 0x7f2de41469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2deac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de415b900 0x7f2de4146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2deac00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4071500 con 0x7f2de40c22e0 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2deac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4071220 con 0x7f2de415b900 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2deac00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de414e8a0 con 0x7f2de414e4c0 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de40c22e0 0x7f2de40c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de40c22e0 0x7f2de40c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55346/0 (socket says 172.21.15.113:55346) 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de3e00640 1 -- 172.21.15.113:0/1325659108 learned_addr learned my addr 172.21.15.113:0/1325659108 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2ddb400640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 0x7f2de41469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de8c00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de415b900 0x7f2de4146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:10.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de3e00640 1 -- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 msgr2=0x7f2de41469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de3e00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 0x7f2de41469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de3e00640 1 -- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de415b900 msgr2=0x7f2de4146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de3e00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de415b900 0x7f2de4146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de3e00640 1 -- 172.21.15.113:0/1325659108 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de414a690 con 0x7f2de40c22e0 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de8c00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de415b900 0x7f2de4146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de3e00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de40c22e0 0x7f2de40c5a90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009820 tx=0x7f2dd401b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de0c00640 1 -- 172.21.15.113:0/1325659108 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40bd020 con 0x7f2de40c22e0 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de0c00640 1 -- 172.21.15.113:0/1325659108 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd40b0640 con 0x7f2de40c22e0 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de0c00640 1 -- 172.21.15.113:0/1325659108 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40b5040 con 0x7f2de40c22e0 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2deac00640 1 -- 172.21.15.113:0/1325659108 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de41494b0 con 0x7f2de40c22e0 2024-03-04T08:24:10.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2deac00640 1 -- 172.21.15.113:0/1325659108 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de4149a30 con 0x7f2de40c22e0 2024-03-04T08:24:10.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2deac00640 1 -- 172.21.15.113:0/1325659108 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de4075c00 con 0x7f2de40c22e0 2024-03-04T08:24:10.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.723+0000 7f2de0c00640 1 -- 172.21.15.113:0/1325659108 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2dd40c0020 con 0x7f2de40c22e0 2024-03-04T08:24:10.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.727+0000 7f2de0c00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2dc0066f60 0x7f2dc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:10.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.727+0000 7f2de0c00640 1 -- 172.21.15.113:0/1325659108 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7f2dd4122ac0 con 0x7f2de40c22e0 2024-03-04T08:24:10.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.727+0000 7f2ddb400640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2dc0066f60 0x7f2dc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:10.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.727+0000 7f2ddb400640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2dc0066f60 0x7f2dc0069420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2de40aeca0 tx=0x7f2dcc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:10.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.731+0000 7f2de0c00640 1 -- 172.21.15.113:0/1325659108 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2dd40c7030 con 0x7f2de40c22e0 2024-03-04T08:24:10.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.915+0000 7f2deac00640 1 -- 172.21.15.113:0/1325659108 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2de40c7be0 con 0x7f2de40c22e0 2024-03-04T08:24:10.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.915+0000 7f2de0c00640 1 -- 172.21.15.113:0/1325659108 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) v1 ==== 95+0+5036 (secure 0 0 0) 0x7f2dd4018770 con 0x7f2de40c22e0 2024-03-04T08:24:10.914 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:10.914 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":191,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":186}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:46.847290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:24:10.915 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 191 2024-03-04T08:24:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 -- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2dc0066f60 msgr2=0x7f2dc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:10.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2dc0066f60 0x7f2dc0069420 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2de40aeca0 tx=0x7f2dcc009040 comp rx=0 tx=0).stop 2024-03-04T08:24:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 -- 172.21.15.113:0/1325659108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de40c22e0 msgr2=0x7f2de40c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de40c22e0 0x7f2de40c5a90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009820 tx=0x7f2dd401b970 comp rx=0 tx=0).stop 2024-03-04T08:24:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 -- 172.21.15.113:0/1325659108 shutdown_connections 2024-03-04T08:24:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2de415b900 0x7f2de4146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2de414e4c0 0x7f2de41469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2dc0066f60 0x7f2dc0069420 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 --2- 172.21.15.113:0/1325659108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de40c22e0 0x7f2de40c5a90 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 -- 172.21.15.113:0/1325659108 >> 172.21.15.113:0/1325659108 conn(0x7f2de406e820 msgr2=0x7f2de40c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 -- 172.21.15.113:0/1325659108 shutdown_connections 2024-03-04T08:24:10.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:10.919+0000 7f2deac00640 1 -- 172.21.15.113:0/1325659108 wait complete. 2024-03-04T08:24:10.936 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2024-03-04T08:24:10.936 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-03-04T08:24:10.936 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:24:10.952 INFO:tasks.ceph.mds.f:Stopped 2024-03-04T08:24:10.953 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:11.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe881800640 1 Processor -- start 2024-03-04T08:24:11.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe881800640 1 -- start start 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe881800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe87c152b20 0x7fe87c070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe881800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c15c510 0x7fe87c163190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe881800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87c152f00 con 0x7fe87c0c8b00 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe881800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87c0c8f90 con 0x7fe87c152b20 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe881800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c0c5320 con 0x7fe87c15c510 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe87a000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe87c0c8b00 0x7fe87c15be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59490/0 (socket says 172.21.15.113:59490) 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe87a000640 1 -- 172.21.15.113:0/1538698561 learned_addr learned my addr 172.21.15.113:0/1538698561 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe87aa00640 1 --2- 172.21.15.113:0/1538698561 >> v2:172.21.15.136:3300/0 conn(0x7fe87c152b20 0x7fe87c070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe87b400640 1 --2- 172.21.15.113:0/1538698561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c15c510 0x7fe87c163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe879600640 1 -- 172.21.15.113:0/1538698561 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515985122 0 0) 0x7fe87c152f00 con 0x7fe87c0c8b00 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe879600640 1 -- 172.21.15.113:0/1538698561 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe860003530 con 0x7fe87c0c8b00 2024-03-04T08:24:11.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe87aa00640 1 -- 172.21.15.113:0/1538698561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c15c510 msgr2=0x7fe87c163190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:11.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe87aa00640 1 --2- 172.21.15.113:0/1538698561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c15c510 0x7fe87c163190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:11.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe87aa00640 1 -- 172.21.15.113:0/1538698561 >> v1:172.21.15.113:6789/0 conn(0x7fe87c0c8b00 legacy=0x7fe87c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:11.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe879600640 1 -- 172.21.15.113:0/1538698561 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4229314942 0 0) 0x7fe860003530 con 0x7fe87c0c8b00 2024-03-04T08:24:11.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe87aa00640 1 -- 172.21.15.113:0/1538698561 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c0c5040 con 0x7fe87c152b20 2024-03-04T08:24:11.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.095+0000 7fe87b400640 1 --2- 172.21.15.113:0/1538698561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c15c510 0x7fe87c163190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:24:11.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe87aa00640 1 --2- 172.21.15.113:0/1538698561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87c152b20 0x7fe87c070010 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009d90 tx=0x7fe86c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b04e4d690841dd63 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:11.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe879600640 1 -- 172.21.15.113:0/1538698561 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0b2070 con 0x7fe87c152b20 2024-03-04T08:24:11.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe879600640 1 -- 172.21.15.113:0/1538698561 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c0a5580 con 0x7fe87c152b20 2024-03-04T08:24:11.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe879600640 1 -- 172.21.15.113:0/1538698561 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8680016a0 con 0x7fe87c152b20 2024-03-04T08:24:11.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe881800640 1 -- 172.21.15.113:0/1538698561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87c152b20 msgr2=0x7fe87c070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:11.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe881800640 1 --2- 172.21.15.113:0/1538698561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87c152b20 0x7fe87c070010 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009d90 tx=0x7fe86c01c890 comp rx=0 tx=0).stop 2024-03-04T08:24:11.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe881800640 1 -- 172.21.15.113:0/1538698561 shutdown_connections 2024-03-04T08:24:11.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe881800640 1 --2- 172.21.15.113:0/1538698561 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c15c510 0x7fe87c163190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:11.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe881800640 1 --2- 172.21.15.113:0/1538698561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87c152b20 0x7fe87c070010 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:11.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe881800640 1 -- 172.21.15.113:0/1538698561 >> 172.21.15.113:0/1538698561 conn(0x7fe87c0709a0 msgr2=0x7fe87c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:11.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe881800640 1 -- 172.21.15.113:0/1538698561 shutdown_connections 2024-03-04T08:24:11.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.099+0000 7fe881800640 1 -- 172.21.15.113:0/1538698561 wait complete. 2024-03-04T08:24:11.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe881800640 1 Processor -- start 2024-03-04T08:24:11.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe881800640 1 -- start start 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe881800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87c0c8b00 0x7fe87c158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe881800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c152b20 0x7fe87c1eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe881800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe87c15c510 0x7fe87c1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe881800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c158810 con 0x7fe87c15c510 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe881800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c0c3810 con 0x7fe87c0c8b00 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe881800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c0c8490 con 0x7fe87c152b20 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe87c15c510 0x7fe87c1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe87c15c510 0x7fe87c1550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55350/0 (socket says 172.21.15.113:55350) 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87b400640 1 -- 172.21.15.113:0/3031832569 learned_addr learned my addr 172.21.15.113:0/3031832569 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87a000640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c152b20 0x7fe87c1eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87aa00640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87c0c8b00 0x7fe87c158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:11.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87b400640 1 -- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c152b20 msgr2=0x7fe87c1eb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87b400640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c152b20 0x7fe87c1eb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87b400640 1 -- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87c0c8b00 msgr2=0x7fe87c158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87b400640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87c0c8b00 0x7fe87c158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87b400640 1 -- 172.21.15.113:0/3031832569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c1ebc20 con 0x7fe87c15c510 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87a000640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c152b20 0x7fe87c1eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87aa00640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87c0c8b00 0x7fe87c158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe87b400640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe87c15c510 0x7fe87c1550f0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fe87400e980 tx=0x7fe87400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe872a00640 1 -- 172.21.15.113:0/3031832569 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874017070 con 0x7fe87c15c510 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe872a00640 1 -- 172.21.15.113:0/3031832569 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87400ca00 con 0x7fe87c15c510 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe881800640 1 -- 172.21.15.113:0/3031832569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87c1f24c0 con 0x7fe87c15c510 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe872a00640 1 -- 172.21.15.113:0/3031832569 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87400cdd0 con 0x7fe87c15c510 2024-03-04T08:24:11.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.103+0000 7fe881800640 1 -- 172.21.15.113:0/3031832569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c1534a0 con 0x7fe87c15c510 2024-03-04T08:24:11.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.107+0000 7fe881800640 1 -- 172.21.15.113:0/3031832569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87c1eb3c0 con 0x7fe87c15c510 2024-03-04T08:24:11.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.107+0000 7fe872a00640 1 -- 172.21.15.113:0/3031832569 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe874013470 con 0x7fe87c15c510 2024-03-04T08:24:11.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.107+0000 7fe872a00640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe85c066f60 0x7fe85c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:11.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.107+0000 7fe872a00640 1 -- 172.21.15.113:0/3031832569 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7fe874085660 con 0x7fe87c15c510 2024-03-04T08:24:11.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.107+0000 7fe87aa00640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe85c066f60 0x7fe85c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:11.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.111+0000 7fe87aa00640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe85c066f60 0x7fe85c069420 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0af000 tx=0x7fe86c0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:11.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.111+0000 7fe872a00640 1 -- 172.21.15.113:0/3031832569 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe87c1eb3c0 con 0x7fe87c15c510 2024-03-04T08:24:11.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.299+0000 7fe881800640 1 -- 172.21.15.113:0/3031832569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe87c075c00 con 0x7fe87c15c510 2024-03-04T08:24:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.299+0000 7fe872a00640 1 -- 172.21.15.113:0/3031832569 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) v1 ==== 95+0+5036 (secure 0 0 0) 0x7fe874055b60 con 0x7fe87c15c510 2024-03-04T08:24:11.298 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:11.298 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":191,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":186}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:46.847290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:24:11.298 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 191 2024-03-04T08:24:11.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 -- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe85c066f60 msgr2=0x7fe85c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:11.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe85c066f60 0x7fe85c069420 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0af000 tx=0x7fe86c0a7070 comp rx=0 tx=0).stop 2024-03-04T08:24:11.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 -- 172.21.15.113:0/3031832569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe87c15c510 msgr2=0x7fe87c1550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:11.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe87c15c510 0x7fe87c1550f0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fe87400e980 tx=0x7fe87400ee40 comp rx=0 tx=0).stop 2024-03-04T08:24:11.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 -- 172.21.15.113:0/3031832569 shutdown_connections 2024-03-04T08:24:11.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe85c066f60 0x7fe85c069420 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:11.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe87c15c510 0x7fe87c1550f0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:11.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe87c152b20 0x7fe87c1eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:11.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 --2- 172.21.15.113:0/3031832569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe87c0c8b00 0x7fe87c158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:11.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 -- 172.21.15.113:0/3031832569 >> 172.21.15.113:0/3031832569 conn(0x7fe87c0709a0 msgr2=0x7fe87c0cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:11.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 -- 172.21.15.113:0/3031832569 shutdown_connections 2024-03-04T08:24:11.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:11.303+0000 7fe881800640 1 -- 172.21.15.113:0/3031832569 wait complete. 2024-03-04T08:24:11.319 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-03-04T08:24:13.323 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:13.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.467+0000 7fc408200640 1 Processor -- start 2024-03-04T08:24:13.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.467+0000 7fc408200640 1 -- start start 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc408200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc40014e4c0 0x7fc400072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc408200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc40015b900 0x7fc4000731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc408200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4000c27f0 con 0x7fc4000c22e0 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc408200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc40015ad60 con 0x7fc40014e4c0 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc408200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4000c2a90 con 0x7fc40015b900 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc404e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc4000c22e0 0x7fc40015b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:59492/0 (socket says 172.21.15.113:59492) 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc404e00640 1 -- 172.21.15.113:0/2334562111 learned_addr learned my addr 172.21.15.113:0/2334562111 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc406200640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc40015b900 0x7fc4000731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc405800640 1 --2- 172.21.15.113:0/2334562111 >> v2:172.21.15.136:3300/0 conn(0x7fc40014e4c0 0x7fc400072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979174375 0 0) 0x7fc4000c27f0 con 0x7fc4000c22e0 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3e8003530 con 0x7fc4000c22e0 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4156663562 0 0) 0x7fc3e8003530 con 0x7fc4000c22e0 2024-03-04T08:24:13.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc40015b900 msgr2=0x7fc4000731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc40015b900 0x7fc4000731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 >> v2:172.21.15.136:3300/0 conn(0x7fc40014e4c0 msgr2=0x7fc400072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 --2- 172.21.15.113:0/2334562111 >> v2:172.21.15.136:3300/0 conn(0x7fc40014e4c0 0x7fc400072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc400070c60 con 0x7fc4000c22e0 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fc3ec001680 con 0x7fc4000c22e0 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 >> v1:172.21.15.113:6789/0 conn(0x7fc4000c22e0 legacy=0x7fc40015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3e80039c0 0x7fc3e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 0x7fc3e8004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3e8004f00 0x7fc3e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4000c2a90 con 0x7fc3e80043b0 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e800bf10 con 0x7fc3e8004f00 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e800c090 con 0x7fc3e80039c0 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc404e00640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 0x7fc3e8004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:13.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc406200640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3e8004f00 0x7fc3e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:13.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc405800640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3e80039c0 0x7fc3e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:13.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc404e00640 1 -- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3e80039c0 msgr2=0x7fc3e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:13.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc404e00640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3e80039c0 0x7fc3e8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc404e00640 1 -- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3e8004f00 msgr2=0x7fc3e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:13.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc404e00640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc3e8004f00 0x7fc3e800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc404e00640 1 -- 172.21.15.113:0/2334562111 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e8003530 con 0x7fc3e80043b0 2024-03-04T08:24:13.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.471+0000 7fc405800640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc3e80039c0 0x7fc3e8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:24:13.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc404e00640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 0x7fc3e8004890 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00c270 tx=0x7fc3ec01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf6b90d4c06335f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:13.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec007d40 con 0x7fc3e80043b0 2024-03-04T08:24:13.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0a9020 con 0x7fc3e80043b0 2024-03-04T08:24:13.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc3ffe00640 1 -- 172.21.15.113:0/2334562111 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0a5ba0 con 0x7fc3e80043b0 2024-03-04T08:24:13.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc408200640 1 -- 172.21.15.113:0/2334562111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 msgr2=0x7fc3e8004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:13.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc408200640 1 --2- 172.21.15.113:0/2334562111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 0x7fc3e8004890 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00c270 tx=0x7fc3ec01cca0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc406200640 1 -- 172.21.15.113:0/2334562111 reap_dead start 2024-03-04T08:24:13.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc408200640 1 -- 172.21.15.113:0/2334562111 shutdown_connections 2024-03-04T08:24:13.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc408200640 1 -- 172.21.15.113:0/2334562111 >> 172.21.15.113:0/2334562111 conn(0x7fc40006e820 msgr2=0x7fc40006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:13.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc408200640 1 -- 172.21.15.113:0/2334562111 shutdown_connections 2024-03-04T08:24:13.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc408200640 1 -- 172.21.15.113:0/2334562111 wait complete. 2024-03-04T08:24:13.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc408200640 1 Processor -- start 2024-03-04T08:24:13.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.475+0000 7fc408200640 1 -- start start 2024-03-04T08:24:13.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc408200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 0x7fc40014c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:13.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc408200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc400154630 0x7fc400154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:13.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc408200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc400141d60 0x7fc4001421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:13.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc408200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc400071500 con 0x7fc3e80043b0 2024-03-04T08:24:13.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc408200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc400071220 con 0x7fc400141d60 2024-03-04T08:24:13.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc408200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40014e8a0 con 0x7fc400154630 2024-03-04T08:24:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc405800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 0x7fc40014c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc405800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 0x7fc40014c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55360/0 (socket says 172.21.15.113:55360) 2024-03-04T08:24:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc405800640 1 -- 172.21.15.113:0/3864731755 learned_addr learned my addr 172.21.15.113:0/3864731755 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc404e00640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc400154630 0x7fc400154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:13.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc406200640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc400141d60 0x7fc4001421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:13.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc405800640 1 -- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc400154630 msgr2=0x7fc400154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:13.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc405800640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc400154630 0x7fc400154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc405800640 1 -- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc400141d60 msgr2=0x7fc4001421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:13.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc405800640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc400141d60 0x7fc4001421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc405800640 1 -- 172.21.15.113:0/3864731755 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4001f09b0 con 0x7fc3e80043b0 2024-03-04T08:24:13.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc405800640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 0x7fc40014c030 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40115d0 tx=0x7fc3f4011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:13.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc3fd600640 1 -- 172.21.15.113:0/3864731755 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4004220 con 0x7fc3e80043b0 2024-03-04T08:24:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc3fd600640 1 -- 172.21.15.113:0/3864731755 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f40043c0 con 0x7fc3e80043b0 2024-03-04T08:24:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc408200640 1 -- 172.21.15.113:0/3864731755 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4001f0c70 con 0x7fc3e80043b0 2024-03-04T08:24:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc3fd600640 1 -- 172.21.15.113:0/3864731755 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4015890 con 0x7fc3e80043b0 2024-03-04T08:24:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc408200640 1 -- 172.21.15.113:0/3864731755 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4001f1230 con 0x7fc3e80043b0 2024-03-04T08:24:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc408200640 1 -- 172.21.15.113:0/3864731755 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc400075c00 con 0x7fc3e80043b0 2024-03-04T08:24:13.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.479+0000 7fc3fd600640 1 -- 172.21.15.113:0/3864731755 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3f4015a30 con 0x7fc3e80043b0 2024-03-04T08:24:13.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.483+0000 7fc3fd600640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc3e0066f90 0x7fc3e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:13.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.483+0000 7fc3fd600640 1 -- 172.21.15.113:0/3864731755 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7fc3f4086050 con 0x7fc3e80043b0 2024-03-04T08:24:13.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.483+0000 7fc404e00640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc3e0066f90 0x7fc3e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:13.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.483+0000 7fc404e00640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc3e0066f90 0x7fc3e0069450 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0aa000 tx=0x7fc400072ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:13.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.487+0000 7fc3fd600640 1 -- 172.21.15.113:0/3864731755 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc3f4019030 con 0x7fc3e80043b0 2024-03-04T08:24:13.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.671+0000 7fc408200640 1 -- 172.21.15.113:0/3864731755 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4000ae640 con 0x7fc3e80043b0 2024-03-04T08:24:13.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc3fd600640 1 -- 172.21.15.113:0/3864731755 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) v1 ==== 95+0+5036 (secure 0 0 0) 0x7fc3f4055f00 con 0x7fc3e80043b0 2024-03-04T08:24:13.671 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:13.671 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":191,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":186}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:46.847290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:24:13.671 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 191 2024-03-04T08:24:13.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 -- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc3e0066f90 msgr2=0x7fc3e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:13.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc3e0066f90 0x7fc3e0069450 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0aa000 tx=0x7fc400072ba0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 -- 172.21.15.113:0/3864731755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 msgr2=0x7fc40014c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:13.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 0x7fc40014c030 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40115d0 tx=0x7fc3f4011a90 comp rx=0 tx=0).stop 2024-03-04T08:24:13.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 -- 172.21.15.113:0/3864731755 shutdown_connections 2024-03-04T08:24:13.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc400141d60 0x7fc4001421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc400154630 0x7fc400154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc3e0066f90 0x7fc3e0069450 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 --2- 172.21.15.113:0/3864731755 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc3e80043b0 0x7fc40014c030 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:13.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 -- 172.21.15.113:0/3864731755 >> 172.21.15.113:0/3864731755 conn(0x7fc40006e820 msgr2=0x7fc4000c6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:13.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 -- 172.21.15.113:0/3864731755 shutdown_connections 2024-03-04T08:24:13.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:13.675+0000 7fc408200640 1 -- 172.21.15.113:0/3864731755 wait complete. 2024-03-04T08:24:13.692 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-03-04T08:24:15.323 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:24:15.327 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:24:15.693 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:15.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed3a800640 1 Processor -- start 2024-03-04T08:24:15.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed3a800640 1 -- start start 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed3a800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fed340ce2d0 0x7fed3415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed3a800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed340ce6b0 0x7fed3406f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed3a800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fed3406de80 0x7fed34154b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed3a800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3415bbf0 con 0x7fed340ce2d0 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed3a800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3406eb80 con 0x7fed3406de80 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed3a800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3406c210 con 0x7fed340ce6b0 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed33400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fed340ce2d0 0x7fed3415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed33400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fed340ce2d0 0x7fed3415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55368/0 (socket says 172.21.15.113:55368) 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed33400640 1 -- 172.21.15.113:0/2991371452 learned_addr learned my addr 172.21.15.113:0/2991371452 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed33e00640 1 --2- 172.21.15.113:0/2991371452 >> v2:172.21.15.136:3300/0 conn(0x7fed3406de80 0x7fed34154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:15.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed2aa00640 1 --2- 172.21.15.113:0/2991371452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed340ce6b0 0x7fed3406f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:15.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed33400640 1 -- 172.21.15.113:0/2991371452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed340ce6b0 msgr2=0x7fed3406f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:15.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed33400640 1 --2- 172.21.15.113:0/2991371452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed340ce6b0 0x7fed3406f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:15.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed33400640 1 -- 172.21.15.113:0/2991371452 >> v2:172.21.15.136:3300/0 conn(0x7fed3406de80 msgr2=0x7fed34154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:15.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed33400640 1 --2- 172.21.15.113:0/2991371452 >> v2:172.21.15.136:3300/0 conn(0x7fed3406de80 0x7fed34154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:15.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed33400640 1 -- 172.21.15.113:0/2991371452 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3406c4f0 con 0x7fed340ce2d0 2024-03-04T08:24:15.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.843+0000 7fed33e00640 1 --2- 172.21.15.113:0/2991371452 >> v2:172.21.15.136:3300/0 conn(0x7fed3406de80 0x7fed34154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:24:15.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed33400640 1 --2- 172.21.15.113:0/2991371452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed340ce2d0 0x7fed3415b6b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fed340ceed0 tx=0x7fed1c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe31b592ea57934 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:15.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed32a00640 1 -- 172.21.15.113:0/2991371452 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0b5070 con 0x7fed340ce2d0 2024-03-04T08:24:15.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed32a00640 1 -- 172.21.15.113:0/2991371452 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c0a89d0 con 0x7fed340ce2d0 2024-03-04T08:24:15.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed32a00640 1 -- 172.21.15.113:0/2991371452 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0a8da0 con 0x7fed340ce2d0 2024-03-04T08:24:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed3a800640 1 -- 172.21.15.113:0/2991371452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed340ce2d0 msgr2=0x7fed3415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed3a800640 1 --2- 172.21.15.113:0/2991371452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed340ce2d0 0x7fed3415b6b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fed340ceed0 tx=0x7fed1c01ed10 comp rx=0 tx=0).stop 2024-03-04T08:24:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed3a800640 1 -- 172.21.15.113:0/2991371452 shutdown_connections 2024-03-04T08:24:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed3a800640 1 --2- 172.21.15.113:0/2991371452 >> v2:172.21.15.136:3300/0 conn(0x7fed3406de80 0x7fed34154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed3a800640 1 --2- 172.21.15.113:0/2991371452 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed340ce6b0 0x7fed3406f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed3a800640 1 --2- 172.21.15.113:0/2991371452 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed340ce2d0 0x7fed3415b6b0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed3a800640 1 -- 172.21.15.113:0/2991371452 >> 172.21.15.113:0/2991371452 conn(0x7fed340709a0 msgr2=0x7fed34070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed3a800640 1 -- 172.21.15.113:0/2991371452 shutdown_connections 2024-03-04T08:24:15.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.847+0000 7fed3a800640 1 -- 172.21.15.113:0/2991371452 wait complete. 2024-03-04T08:24:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 Processor -- start 2024-03-04T08:24:15.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 -- start start 2024-03-04T08:24:15.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed3406de80 0x7fed340c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:15.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed340ce2d0 0x7fed341f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed340ce6b0 0x7fed34150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3406c7d0 con 0x7fed3406de80 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3406e810 con 0x7fed340ce6b0 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed34070010 con 0x7fed340ce2d0 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed33400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed3406de80 0x7fed340c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed33400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed3406de80 0x7fed340c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55374/0 (socket says 172.21.15.113:55374) 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed33400640 1 -- 172.21.15.113:0/1034713092 learned_addr learned my addr 172.21.15.113:0/1034713092 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed2aa00640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed340ce2d0 0x7fed341f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed33e00640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed340ce6b0 0x7fed34150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed33400640 1 -- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed340ce2d0 msgr2=0x7fed341f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed33400640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed340ce2d0 0x7fed341f4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed33400640 1 -- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed340ce6b0 msgr2=0x7fed34150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:15.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed33400640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed340ce6b0 0x7fed34150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:15.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed33400640 1 -- 172.21.15.113:0/1034713092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed34151310 con 0x7fed3406de80 2024-03-04T08:24:15.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed33400640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed3406de80 0x7fed340c6b10 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fed1c01d3c0 tx=0x7fed1c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:15.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed2be00640 1 -- 172.21.15.113:0/1034713092 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0b5070 con 0x7fed3406de80 2024-03-04T08:24:15.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed2be00640 1 -- 172.21.15.113:0/1034713092 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c0b0a70 con 0x7fed3406de80 2024-03-04T08:24:15.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 -- 172.21.15.113:0/1034713092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed340c7210 con 0x7fed3406de80 2024-03-04T08:24:15.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed2be00640 1 -- 172.21.15.113:0/1034713092 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0b0e60 con 0x7fed3406de80 2024-03-04T08:24:15.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 -- 172.21.15.113:0/1034713092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed341f0ec0 con 0x7fed3406de80 2024-03-04T08:24:15.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.851+0000 7fed3a800640 1 -- 172.21.15.113:0/1034713092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed341f0cd0 con 0x7fed3406de80 2024-03-04T08:24:15.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.855+0000 7fed2be00640 1 -- 172.21.15.113:0/1034713092 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed1c0c2060 con 0x7fed3406de80 2024-03-04T08:24:15.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.855+0000 7fed2be00640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed10066f60 0x7fed10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:15.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.855+0000 7fed2be00640 1 -- 172.21.15.113:0/1034713092 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7fed1c122d90 con 0x7fed3406de80 2024-03-04T08:24:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.855+0000 7fed2aa00640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed10066f60 0x7fed10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:15.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.855+0000 7fed2aa00640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed10066f60 0x7fed10069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fed34154060 tx=0x7fed24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:15.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:15.859+0000 7fed2be00640 1 -- 172.21.15.113:0/1034713092 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fed1c018870 con 0x7fed3406de80 2024-03-04T08:24:16.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.043+0000 7fed3a800640 1 -- 172.21.15.113:0/1034713092 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed34075c00 con 0x7fed3406de80 2024-03-04T08:24:16.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.043+0000 7fed2be00640 1 -- 172.21.15.113:0/1034713092 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) v1 ==== 95+0+5036 (secure 0 0 0) 0x7fed1c0f3230 con 0x7fed3406de80 2024-03-04T08:24:16.043 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:16.043 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":191,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":186}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:46.847290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:24:16.043 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 191 2024-03-04T08:24:16.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 -- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed10066f60 msgr2=0x7fed10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed10066f60 0x7fed10069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fed34154060 tx=0x7fed24008040 comp rx=0 tx=0).stop 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 -- 172.21.15.113:0/1034713092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed3406de80 msgr2=0x7fed340c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed3406de80 0x7fed340c6b10 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fed1c01d3c0 tx=0x7fed1c01bab0 comp rx=0 tx=0).stop 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 -- 172.21.15.113:0/1034713092 shutdown_connections 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fed340ce6b0 0x7fed34150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fed340ce2d0 0x7fed341f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed10066f60 0x7fed10069420 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 --2- 172.21.15.113:0/1034713092 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fed3406de80 0x7fed340c6b10 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 -- 172.21.15.113:0/1034713092 >> 172.21.15.113:0/1034713092 conn(0x7fed340709a0 msgr2=0x7fed3406f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 -- 172.21.15.113:0/1034713092 shutdown_connections 2024-03-04T08:24:16.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:16.047+0000 7fed3a800640 1 -- 172.21.15.113:0/1034713092 wait complete. 2024-03-04T08:24:16.064 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-03-04T08:24:18.065 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:18.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.211+0000 7f8169400640 1 Processor -- start 2024-03-04T08:24:18.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f8169400640 1 -- start start 2024-03-04T08:24:18.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f8169400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81640c9ab0 0x7f816415c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:18.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f8169400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f81640c8fe0 0x7f81640c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:18.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f8169400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81640c5630 con 0x7f8164152ac0 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f8169400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81640c5bb0 con 0x7f81640c9ab0 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f8169400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f816406fc70 con 0x7f81640c8fe0 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f815b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81640c9ab0 0x7f816415c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f815b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f81640c9ab0 0x7f816415c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35158/0 (socket says 172.21.15.113:35158) 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f815b400640 1 -- 172.21.15.113:0/1877819105 learned_addr learned my addr 172.21.15.113:0/1877819105 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f8163e00640 1 --1- 172.21.15.113:0/1877819105 >> v1:172.21.15.113:6789/0 conn(0x7f8164152ac0 0x7f816406ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43518/0 (socket says 172.21.15.113:43518) 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f8168a00640 1 --2- 172.21.15.113:0/1877819105 >> v2:172.21.15.136:3300/0 conn(0x7f81640c8fe0 0x7f81640c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f815b400640 1 -- 172.21.15.113:0/1877819105 >> v2:172.21.15.136:3300/0 conn(0x7f81640c8fe0 msgr2=0x7f81640c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f815b400640 1 --2- 172.21.15.113:0/1877819105 >> v2:172.21.15.136:3300/0 conn(0x7f81640c8fe0 0x7f81640c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f815b400640 1 -- 172.21.15.113:0/1877819105 >> v1:172.21.15.113:6789/0 conn(0x7f8164152ac0 legacy=0x7f816406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:18.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.215+0000 7f815b400640 1 -- 172.21.15.113:0/1877819105 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81640c58d0 con 0x7f81640c9ab0 2024-03-04T08:24:18.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f815b400640 1 --2- 172.21.15.113:0/1877819105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c9ab0 0x7f816415c6b0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f816406f660 tx=0x7f814c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c91d7b42391e8f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:18.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8163400640 1 -- 172.21.15.113:0/1877819105 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0b0070 con 0x7f81640c9ab0 2024-03-04T08:24:18.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8163400640 1 -- 172.21.15.113:0/1877819105 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c0ab600 con 0x7f81640c9ab0 2024-03-04T08:24:18.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8163400640 1 -- 172.21.15.113:0/1877819105 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0ab9d0 con 0x7f81640c9ab0 2024-03-04T08:24:18.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8169400640 1 -- 172.21.15.113:0/1877819105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c9ab0 msgr2=0x7f816415c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:18.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8169400640 1 --2- 172.21.15.113:0/1877819105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c9ab0 0x7f816415c6b0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f816406f660 tx=0x7f814c01c490 comp rx=0 tx=0).stop 2024-03-04T08:24:18.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8169400640 1 -- 172.21.15.113:0/1877819105 shutdown_connections 2024-03-04T08:24:18.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8169400640 1 --2- 172.21.15.113:0/1877819105 >> v2:172.21.15.136:3300/0 conn(0x7f81640c8fe0 0x7f81640c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:18.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8169400640 1 --2- 172.21.15.113:0/1877819105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c9ab0 0x7f816415c6b0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:18.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8169400640 1 -- 172.21.15.113:0/1877819105 >> 172.21.15.113:0/1877819105 conn(0x7f8164072e30 msgr2=0x7f8164066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:18.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8169400640 1 -- 172.21.15.113:0/1877819105 shutdown_connections 2024-03-04T08:24:18.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8169400640 1 -- 172.21.15.113:0/1877819105 wait complete. 2024-03-04T08:24:18.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8169400640 1 Processor -- start 2024-03-04T08:24:18.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.219+0000 7f8169400640 1 -- start start 2024-03-04T08:24:18.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8169400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c8fe0 0x7f816414f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:18.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8169400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8164152ac0 0x7f81640cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:18.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8169400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81640cc860 0x7f8164146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8169400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8164152ea0 con 0x7f81640c8fe0 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8169400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81640c8a00 con 0x7f81640cc860 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8169400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f816415cbf0 con 0x7f8164152ac0 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8163e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c8fe0 0x7f816414f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8163e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c8fe0 0x7f816414f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35172/0 (socket says 172.21.15.113:35172) 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8163e00640 1 -- 172.21.15.113:0/599232601 learned_addr learned my addr 172.21.15.113:0/599232601 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f815b400640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8164152ac0 0x7f81640cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8168a00640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81640cc860 0x7f8164146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8163e00640 1 -- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8164152ac0 msgr2=0x7f81640cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8163e00640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8164152ac0 0x7f81640cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8163e00640 1 -- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81640cc860 msgr2=0x7f8164146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8163e00640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81640cc860 0x7f8164146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8163e00640 1 -- 172.21.15.113:0/599232601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8164148af0 con 0x7f81640c8fe0 2024-03-04T08:24:18.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8163e00640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c8fe0 0x7f816414f700 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f815400c370 tx=0x7f815400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:18.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8160c00640 1 -- 172.21.15.113:0/599232601 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81540074e0 con 0x7f81640c8fe0 2024-03-04T08:24:18.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8160c00640 1 -- 172.21.15.113:0/599232601 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8154007680 con 0x7f81640c8fe0 2024-03-04T08:24:18.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8169400640 1 -- 172.21.15.113:0/599232601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816414b0f0 con 0x7f81640c8fe0 2024-03-04T08:24:18.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8160c00640 1 -- 172.21.15.113:0/599232601 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154004a30 con 0x7f81640c8fe0 2024-03-04T08:24:18.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8169400640 1 -- 172.21.15.113:0/599232601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8164147d30 con 0x7f81640c8fe0 2024-03-04T08:24:18.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8169400640 1 -- 172.21.15.113:0/599232601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8164075a80 con 0x7f81640c8fe0 2024-03-04T08:24:18.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.223+0000 7f8160c00640 1 -- 172.21.15.113:0/599232601 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f815401a020 con 0x7f81640c8fe0 2024-03-04T08:24:18.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.227+0000 7f8160c00640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8140066f60 0x7f8140069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:18.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.227+0000 7f8160c00640 1 -- 172.21.15.113:0/599232601 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7f81540852d0 con 0x7f81640c8fe0 2024-03-04T08:24:18.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.227+0000 7f815b400640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8140066f60 0x7f8140069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:18.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.227+0000 7f815b400640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8140066f60 0x7f8140069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f814c0ad000 tx=0x7f814c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:18.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.231+0000 7f8160c00640 1 -- 172.21.15.113:0/599232601 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8154055800 con 0x7f81640c8fe0 2024-03-04T08:24:18.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.415+0000 7f8169400640 1 -- 172.21.15.113:0/599232601 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8164074ce0 con 0x7f81640c8fe0 2024-03-04T08:24:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.415+0000 7f8160c00640 1 -- 172.21.15.113:0/599232601 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) v1 ==== 95+0+5036 (secure 0 0 0) 0x7f8154055100 con 0x7f81640c8fe0 2024-03-04T08:24:18.413 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:18.413 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":191,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":186}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:46.847290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:24:18.413 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 191 2024-03-04T08:24:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 -- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8140066f60 msgr2=0x7f8140069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8140066f60 0x7f8140069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f814c0ad000 tx=0x7f814c008040 comp rx=0 tx=0).stop 2024-03-04T08:24:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 -- 172.21.15.113:0/599232601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c8fe0 msgr2=0x7f816414f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c8fe0 0x7f816414f700 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f815400c370 tx=0x7f815400c830 comp rx=0 tx=0).stop 2024-03-04T08:24:18.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 -- 172.21.15.113:0/599232601 shutdown_connections 2024-03-04T08:24:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f81640cc860 0x7f8164146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8164152ac0 0x7f81640cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8140066f60 0x7f8140069420 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 --2- 172.21.15.113:0/599232601 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f81640c8fe0 0x7f816414f700 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 -- 172.21.15.113:0/599232601 >> 172.21.15.113:0/599232601 conn(0x7f8164072e30 msgr2=0x7f8164161210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 -- 172.21.15.113:0/599232601 shutdown_connections 2024-03-04T08:24:18.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:18.419+0000 7f8169400640 1 -- 172.21.15.113:0/599232601 wait complete. 2024-03-04T08:24:18.435 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-03-04T08:24:20.436 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.583+0000 7f8e25000640 1 Processor -- start 2024-03-04T08:24:20.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.583+0000 7f8e25000640 1 -- start start 2024-03-04T08:24:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e25000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e2014e4c0 0x7f8e20072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e25000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8e2015b900 0x7f8e200731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e25000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e200c27f0 con 0x7f8e200c22e0 2024-03-04T08:24:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e25000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2015ad60 con 0x7f8e2015b900 2024-03-04T08:24:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e25000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e200c2a90 con 0x7f8e2014e4c0 2024-03-04T08:24:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e1e800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8e2015b900 0x7f8e200731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e15400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8e200c22e0 0x7f8e2015b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43532/0 (socket says 172.21.15.113:43532) 2024-03-04T08:24:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e15400640 1 -- 172.21.15.113:0/262330670 learned_addr learned my addr 172.21.15.113:0/262330670 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e1de00640 1 --2- 172.21.15.113:0/262330670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e2014e4c0 0x7f8e20072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:20.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e1d400640 1 -- 172.21.15.113:0/262330670 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938795052 0 0) 0x7f8e200c27f0 con 0x7f8e200c22e0 2024-03-04T08:24:20.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e1d400640 1 -- 172.21.15.113:0/262330670 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e04003530 con 0x7f8e200c22e0 2024-03-04T08:24:20.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e1e800640 1 -- 172.21.15.113:0/262330670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e2014e4c0 msgr2=0x7f8e20072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:20.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e1e800640 1 --2- 172.21.15.113:0/262330670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e2014e4c0 0x7f8e20072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:20.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e1e800640 1 -- 172.21.15.113:0/262330670 >> v1:172.21.15.113:6789/0 conn(0x7f8e200c22e0 legacy=0x7f8e2015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:20.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.587+0000 7f8e1e800640 1 -- 172.21.15.113:0/262330670 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e20070c60 con 0x7f8e2015b900 2024-03-04T08:24:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e1e800640 1 --2- 172.21.15.113:0/262330670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2015b900 0x7f8e200731c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009d90 tx=0x7f8e1801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51693b946221b99 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e1d400640 1 -- 172.21.15.113:0/262330670 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180b4070 con 0x7f8e2015b900 2024-03-04T08:24:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e1d400640 1 -- 172.21.15.113:0/262330670 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e180af600 con 0x7f8e2015b900 2024-03-04T08:24:20.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e1d400640 1 -- 172.21.15.113:0/262330670 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180af9d0 con 0x7f8e2015b900 2024-03-04T08:24:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 -- 172.21.15.113:0/262330670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2015b900 msgr2=0x7f8e200731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 --2- 172.21.15.113:0/262330670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2015b900 0x7f8e200731c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009d90 tx=0x7f8e1801f150 comp rx=0 tx=0).stop 2024-03-04T08:24:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 -- 172.21.15.113:0/262330670 shutdown_connections 2024-03-04T08:24:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 --2- 172.21.15.113:0/262330670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e2015b900 0x7f8e200731c0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 --2- 172.21.15.113:0/262330670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e2014e4c0 0x7f8e20072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 -- 172.21.15.113:0/262330670 >> 172.21.15.113:0/262330670 conn(0x7f8e2006e820 msgr2=0x7f8e2006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 -- 172.21.15.113:0/262330670 shutdown_connections 2024-03-04T08:24:20.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 -- 172.21.15.113:0/262330670 wait complete. 2024-03-04T08:24:20.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 Processor -- start 2024-03-04T08:24:20.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 -- start start 2024-03-04T08:24:20.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e200c22e0 0x7f8e201ecea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:20.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e2014e4c0 0x7f8e201ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:20.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e2015b900 0x7f8e200c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20071500 con 0x7f8e200c22e0 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20071220 con 0x7f8e2015b900 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e25000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2014e8a0 con 0x7f8e2014e4c0 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e1de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e200c22e0 0x7f8e201ecea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e1de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e200c22e0 0x7f8e201ecea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35192/0 (socket says 172.21.15.113:35192) 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e1de00640 1 -- 172.21.15.113:0/1441785348 learned_addr learned my addr 172.21.15.113:0/1441785348 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e1e800640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e2015b900 0x7f8e200c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.591+0000 7f8e15400640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e2014e4c0 0x7f8e201ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e1de00640 1 -- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e2014e4c0 msgr2=0x7f8e201ef8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e1de00640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e2014e4c0 0x7f8e201ef8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e1de00640 1 -- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e2015b900 msgr2=0x7f8e200c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e1de00640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e2015b900 0x7f8e200c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e1de00640 1 -- 172.21.15.113:0/1441785348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2014a500 con 0x7f8e200c22e0 2024-03-04T08:24:20.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e1de00640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e200c22e0 0x7f8e201ecea0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f8e10010df0 tx=0x7f8e1000e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:20.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e16a00640 1 -- 172.21.15.113:0/1441785348 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1001e070 con 0x7f8e200c22e0 2024-03-04T08:24:20.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e16a00640 1 -- 172.21.15.113:0/1441785348 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10011860 con 0x7f8e200c22e0 2024-03-04T08:24:20.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e16a00640 1 -- 172.21.15.113:0/1441785348 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10011c10 con 0x7f8e200c22e0 2024-03-04T08:24:20.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e25000640 1 -- 172.21.15.113:0/1441785348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e200cacc0 con 0x7f8e200c22e0 2024-03-04T08:24:20.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e25000640 1 -- 172.21.15.113:0/1441785348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e201ec3c0 con 0x7f8e200c22e0 2024-03-04T08:24:20.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e25000640 1 -- 172.21.15.113:0/1441785348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e20075c00 con 0x7f8e200c22e0 2024-03-04T08:24:20.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e16a00640 1 -- 172.21.15.113:0/1441785348 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e10021070 con 0x7f8e200c22e0 2024-03-04T08:24:20.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e16a00640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8dfc066f60 0x7f8dfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:20.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e16a00640 1 -- 172.21.15.113:0/1441785348 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7f8e10088600 con 0x7f8e200c22e0 2024-03-04T08:24:20.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.595+0000 7f8e15400640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8dfc066f60 0x7f8dfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:20.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.599+0000 7f8e15400640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8dfc066f60 0x7f8dfc069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8e201588c0 tx=0x7f8e08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:20.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.599+0000 7f8e16a00640 1 -- 172.21.15.113:0/1441785348 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8e10058b30 con 0x7f8e200c22e0 2024-03-04T08:24:20.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.791+0000 7f8e25000640 1 -- 172.21.15.113:0/1441785348 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e20155190 con 0x7f8e200c22e0 2024-03-04T08:24:20.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.791+0000 7f8e16a00640 1 -- 172.21.15.113:0/1441785348 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) v1 ==== 95+0+5036 (secure 0 0 0) 0x7f8e10058430 con 0x7f8e200c22e0 2024-03-04T08:24:20.790 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:20.790 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":191,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":186}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:46.847290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:24:20.790 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 191 2024-03-04T08:24:20.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 -- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8dfc066f60 msgr2=0x7f8dfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:20.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8dfc066f60 0x7f8dfc069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8e201588c0 tx=0x7f8e08009040 comp rx=0 tx=0).stop 2024-03-04T08:24:20.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 -- 172.21.15.113:0/1441785348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e200c22e0 msgr2=0x7f8e201ecea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e200c22e0 0x7f8e201ecea0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f8e10010df0 tx=0x7f8e1000e730 comp rx=0 tx=0).stop 2024-03-04T08:24:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 -- 172.21.15.113:0/1441785348 shutdown_connections 2024-03-04T08:24:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e2015b900 0x7f8e200c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8e2014e4c0 0x7f8e201ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8dfc066f60 0x7f8dfc069420 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 --2- 172.21.15.113:0/1441785348 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8e200c22e0 0x7f8e201ecea0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 -- 172.21.15.113:0/1441785348 >> 172.21.15.113:0/1441785348 conn(0x7f8e2006e820 msgr2=0x7f8e200c7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 -- 172.21.15.113:0/1441785348 shutdown_connections 2024-03-04T08:24:20.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:20.795+0000 7f8e25000640 1 -- 172.21.15.113:0/1441785348 wait complete. 2024-03-04T08:24:20.811 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-03-04T08:24:22.812 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:22.959 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.959+0000 7fc1d6e00640 1 Processor -- start 2024-03-04T08:24:22.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d6e00640 1 -- start start 2024-03-04T08:24:22.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d6e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1d0154d00 0x7fc1d0071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:22.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d6e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1d015a990 0x7fc1d015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:22.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d6e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d015b720 con 0x7fc1d00c9640 2024-03-04T08:24:22.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d6e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d00c8630 con 0x7fc1d0154d00 2024-03-04T08:24:22.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d6e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01550e0 con 0x7fc1d015a990 2024-03-04T08:24:22.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d5a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1d0154d00 0x7fc1d0071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:22.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d5000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc1d00c9640 0x7fc1d00c2900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43542/0 (socket says 172.21.15.113:43542) 2024-03-04T08:24:22.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d5000640 1 -- 172.21.15.113:0/1479885036 learned_addr learned my addr 172.21.15.113:0/1479885036 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:22.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d5a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1d0154d00 0x7fc1d0071a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35206/0 (socket says 172.21.15.113:35206) 2024-03-04T08:24:22.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d6400640 1 --2- 172.21.15.113:0/1479885036 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1d015a990 0x7fc1d015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:22.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d5a00640 1 -- 172.21.15.113:0/1479885036 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1d015a990 msgr2=0x7fc1d015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:22.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d5a00640 1 --2- 172.21.15.113:0/1479885036 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1d015a990 0x7fc1d015ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:22.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d5a00640 1 -- 172.21.15.113:0/1479885036 >> v1:172.21.15.113:6789/0 conn(0x7fc1d00c9640 legacy=0x7fc1d00c2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:22.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1d5a00640 1 -- 172.21.15.113:0/1479885036 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d00c8880 con 0x7fc1d0154d00 2024-03-04T08:24:22.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.963+0000 7fc1cfe00640 1 -- 172.21.15.113:0/1479885036 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720358269 0 0) 0x7fc1d015b720 con 0x7fc1d00c9640 2024-03-04T08:24:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d5a00640 1 --2- 172.21.15.113:0/1479885036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d0154d00 0x7fc1d0071a30 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009d90 tx=0x7fc1c401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44de5a0f305d0293 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1cfe00640 1 -- 172.21.15.113:0/1479885036 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c40b4070 con 0x7fc1d0154d00 2024-03-04T08:24:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1cfe00640 1 -- 172.21.15.113:0/1479885036 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c40af600 con 0x7fc1d0154d00 2024-03-04T08:24:22.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1cfe00640 1 -- 172.21.15.113:0/1479885036 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c40af9d0 con 0x7fc1d0154d00 2024-03-04T08:24:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1479885036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d0154d00 msgr2=0x7fc1d0071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d6e00640 1 --2- 172.21.15.113:0/1479885036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d0154d00 0x7fc1d0071a30 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009d90 tx=0x7fc1c401f150 comp rx=0 tx=0).stop 2024-03-04T08:24:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1479885036 shutdown_connections 2024-03-04T08:24:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d6e00640 1 --2- 172.21.15.113:0/1479885036 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1d015a990 0x7fc1d015ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d6e00640 1 --2- 172.21.15.113:0/1479885036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d0154d00 0x7fc1d0071a30 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1479885036 >> 172.21.15.113:0/1479885036 conn(0x7fc1d00c9a30 msgr2=0x7fc1d00c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1479885036 shutdown_connections 2024-03-04T08:24:22.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1479885036 wait complete. 2024-03-04T08:24:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d6e00640 1 Processor -- start 2024-03-04T08:24:22.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.967+0000 7fc1d6e00640 1 -- start start 2024-03-04T08:24:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1d0154d00 0x7fc1d0156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1d015a990 0x7fc1d0155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:22.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d015e780 0x7fc1d0072d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d0071f70 con 0x7fc1d015e780 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d00c8e40 con 0x7fc1d015a990 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d015b320 con 0x7fc1d0154d00 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d015e780 0x7fc1d0072d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d015e780 0x7fc1d0072d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35214/0 (socket says 172.21.15.113:35214) 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6400640 1 -- 172.21.15.113:0/1942949608 learned_addr learned my addr 172.21.15.113:0/1942949608 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d5a00640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1d0154d00 0x7fc1d0156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d5000640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1d015a990 0x7fc1d0155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6400640 1 -- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1d0154d00 msgr2=0x7fc1d0156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6400640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1d0154d00 0x7fc1d0156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6400640 1 -- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1d015a990 msgr2=0x7fc1d0155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6400640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1d015a990 0x7fc1d0155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:22.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6400640 1 -- 172.21.15.113:0/1942949608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d015eca0 con 0x7fc1d015e780 2024-03-04T08:24:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6400640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d015e780 0x7fc1d0072d60 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800d560 tx=0x7fc1c800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1cd600640 1 -- 172.21.15.113:0/1942949608 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80125f0 con 0x7fc1d015e780 2024-03-04T08:24:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1cd600640 1 -- 172.21.15.113:0/1942949608 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c8019070 con 0x7fc1d015e780 2024-03-04T08:24:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1cd600640 1 -- 172.21.15.113:0/1942949608 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80129f0 con 0x7fc1d015e780 2024-03-04T08:24:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1942949608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d01f03a0 con 0x7fc1d015e780 2024-03-04T08:24:22.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1942949608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d01f0960 con 0x7fc1d015e780 2024-03-04T08:24:22.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1942949608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d0075a80 con 0x7fc1d015e780 2024-03-04T08:24:22.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.971+0000 7fc1cd600640 1 -- 172.21.15.113:0/1942949608 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1c8012b90 con 0x7fc1d015e780 2024-03-04T08:24:22.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.975+0000 7fc1cd600640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1b0066f90 0x7fc1b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:22.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.975+0000 7fc1cd600640 1 -- 172.21.15.113:0/1942949608 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7fc1c8085e70 con 0x7fc1d015e780 2024-03-04T08:24:22.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.975+0000 7fc1d5a00640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1b0066f90 0x7fc1b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:22.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.975+0000 7fc1d5a00640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1b0066f90 0x7fc1b0069450 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40b1000 tx=0x7fc1c40aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:22.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:22.979+0000 7fc1cd600640 1 -- 172.21.15.113:0/1942949608 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc1c80563a0 con 0x7fc1d015e780 2024-03-04T08:24:23.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.163+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1942949608 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1d00c8410 con 0x7fc1d015e780 2024-03-04T08:24:23.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.163+0000 7fc1cd600640 1 -- 172.21.15.113:0/1942949608 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) v1 ==== 95+0+5036 (secure 0 0 0) 0x7fc1c8055ca0 con 0x7fc1d015e780 2024-03-04T08:24:23.163 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:23.163 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":191,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":186}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:46.847290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:24:23.163 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 191 2024-03-04T08:24:23.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1b0066f90 msgr2=0x7fc1b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:23.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1b0066f90 0x7fc1b0069450 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40b1000 tx=0x7fc1c40aa000 comp rx=0 tx=0).stop 2024-03-04T08:24:23.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1942949608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d015e780 msgr2=0x7fc1d0072d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:23.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d015e780 0x7fc1d0072d60 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800d560 tx=0x7fc1c800da20 comp rx=0 tx=0).stop 2024-03-04T08:24:23.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1942949608 shutdown_connections 2024-03-04T08:24:23.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1b0066f90 0x7fc1b0069450 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:23.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d015e780 0x7fc1d0072d60 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:23.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1d015a990 0x7fc1d0155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 --2- 172.21.15.113:0/1942949608 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc1d0154d00 0x7fc1d0156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1942949608 >> 172.21.15.113:0/1942949608 conn(0x7fc1d00c9a30 msgr2=0x7fc1d0153200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1942949608 shutdown_connections 2024-03-04T08:24:23.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:23.167+0000 7fc1d6e00640 1 -- 172.21.15.113:0/1942949608 wait complete. 2024-03-04T08:24:23.184 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-03-04T08:24:25.186 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:25.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67cd400640 1 Processor -- start 2024-03-04T08:24:25.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67cd400640 1 -- start start 2024-03-04T08:24:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67cd400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 0x7f67c81662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67cd400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c80ca0e0 con 0x7f67c806ff20 2024-03-04T08:24:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67cd400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c815aa30 con 0x7f67c8158010 2024-03-04T08:24:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67cd400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c806f450 con 0x7f67c806dd50 2024-03-04T08:24:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67bde00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f67c806ff20 0x7f67c80ce120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43554/0 (socket says 172.21.15.113:43554) 2024-03-04T08:24:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67bde00640 1 -- 172.21.15.113:0/3400274513 learned_addr learned my addr 172.21.15.113:0/3400274513 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67c7200640 1 --2- 172.21.15.113:0/3400274513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 0x7f67c81662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67c5e00640 1 -- 172.21.15.113:0/3400274513 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295710534 0 0) 0x7f67c80ca0e0 con 0x7f67c806ff20 2024-03-04T08:24:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67c5e00640 1 -- 172.21.15.113:0/3400274513 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ac003530 con 0x7f67c806ff20 2024-03-04T08:24:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67c5e00640 1 -- 172.21.15.113:0/3400274513 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671719367 0 0) 0x7f67c815aa30 con 0x7f67c8158010 2024-03-04T08:24:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67c5e00640 1 -- 172.21.15.113:0/3400274513 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c80ca0e0 con 0x7f67c8158010 2024-03-04T08:24:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67c7200640 1 -- 172.21.15.113:0/3400274513 >> v1:172.21.15.136:6789/0 conn(0x7f67c8158010 legacy=0x7f67c815b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67c7200640 1 -- 172.21.15.113:0/3400274513 >> v1:172.21.15.113:6789/0 conn(0x7f67c806ff20 legacy=0x7f67c80ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67c5e00640 1 -- 172.21.15.113:0/3400274513 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 164116187 0 0) 0x7f67ac003530 con 0x7f67c806ff20 2024-03-04T08:24:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.335+0000 7f67c7200640 1 -- 172.21.15.113:0/3400274513 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c806c590 con 0x7f67c806dd50 2024-03-04T08:24:25.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67c7200640 1 --2- 172.21.15.113:0/3400274513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 0x7f67c81662e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009dc0 tx=0x7f67c001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2359a9e299daaa88 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67c5e00640 1 -- 172.21.15.113:0/3400274513 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0001680 con 0x7f67c806dd50 2024-03-04T08:24:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67c5e00640 1 -- 172.21.15.113:0/3400274513 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c00a6500 con 0x7f67c806dd50 2024-03-04T08:24:25.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67c5e00640 1 -- 172.21.15.113:0/3400274513 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c00a68b0 con 0x7f67c806dd50 2024-03-04T08:24:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67cd400640 1 -- 172.21.15.113:0/3400274513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 msgr2=0x7f67c81662e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67cd400640 1 --2- 172.21.15.113:0/3400274513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 0x7f67c81662e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009dc0 tx=0x7f67c001c8b0 comp rx=0 tx=0).stop 2024-03-04T08:24:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67cd400640 1 -- 172.21.15.113:0/3400274513 shutdown_connections 2024-03-04T08:24:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67cd400640 1 --2- 172.21.15.113:0/3400274513 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 0x7f67c81662e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67cd400640 1 -- 172.21.15.113:0/3400274513 >> 172.21.15.113:0/3400274513 conn(0x7f67c80734d0 msgr2=0x7f67c8066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:25.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67cd400640 1 -- 172.21.15.113:0/3400274513 shutdown_connections 2024-03-04T08:24:25.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.339+0000 7f67cd400640 1 -- 172.21.15.113:0/3400274513 wait complete. 2024-03-04T08:24:25.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 Processor -- start 2024-03-04T08:24:25.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 -- start start 2024-03-04T08:24:25.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 0x7f67c80b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:25.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c806ff20 0x7f67c81f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:25.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c8158010 0x7f67c81570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:25.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c81583f0 con 0x7f67c8158010 2024-03-04T08:24:25.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c806e1d0 con 0x7f67c806ff20 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c806c170 con 0x7f67c806dd50 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67c7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c8158010 0x7f67c81570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67c7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c8158010 0x7f67c81570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35226/0 (socket says 172.21.15.113:35226) 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67c7200640 1 -- 172.21.15.113:0/3463557986 learned_addr learned my addr 172.21.15.113:0/3463557986 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67c6800640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 0x7f67c80b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67bde00640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c806ff20 0x7f67c81f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67c7200640 1 -- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 msgr2=0x7f67c80b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67c7200640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 0x7f67c80b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67c7200640 1 -- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c806ff20 msgr2=0x7f67c81f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67c7200640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c806ff20 0x7f67c81f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67c7200640 1 -- 172.21.15.113:0/3463557986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c814dce0 con 0x7f67c8158010 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67bde00640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c806ff20 0x7f67c81f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:24:25.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67c7200640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c8158010 0x7f67c81570e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f67c001a4e0 tx=0x7f67c0019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:25.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67bf400640 1 -- 172.21.15.113:0/3463557986 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c00a6ac0 con 0x7f67c8158010 2024-03-04T08:24:25.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67bf400640 1 -- 172.21.15.113:0/3463557986 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c00bf070 con 0x7f67c8158010 2024-03-04T08:24:25.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67bf400640 1 -- 172.21.15.113:0/3463557986 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0004040 con 0x7f67c8158010 2024-03-04T08:24:25.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 -- 172.21.15.113:0/3463557986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c80b0c30 con 0x7f67c8158010 2024-03-04T08:24:25.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 -- 172.21.15.113:0/3463557986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c80b11d0 con 0x7f67c8158010 2024-03-04T08:24:25.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.343+0000 7f67cd400640 1 -- 172.21.15.113:0/3463557986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c8075c00 con 0x7f67c8158010 2024-03-04T08:24:25.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.347+0000 7f67bf400640 1 -- 172.21.15.113:0/3463557986 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67c00041e0 con 0x7f67c8158010 2024-03-04T08:24:25.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.347+0000 7f67bf400640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f67a4066f90 0x7f67a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:25.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.347+0000 7f67bf400640 1 -- 172.21.15.113:0/3463557986 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8319+0+0 (secure 0 0 0) 0x7f67c0120ab0 con 0x7f67c8158010 2024-03-04T08:24:25.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.347+0000 7f67c6800640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f67a4066f90 0x7f67a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:25.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.347+0000 7f67c6800640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f67a4066f90 0x7f67a4069450 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f67b80011f0 tx=0x7f67b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:25.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.351+0000 7f67bf400640 1 -- 172.21.15.113:0/3463557986 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f67c00164a0 con 0x7f67c8158010 2024-03-04T08:24:25.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.535+0000 7f67cd400640 1 -- 172.21.15.113:0/3463557986 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67c8159a50 con 0x7f67c8158010 2024-03-04T08:24:25.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.535+0000 7f67bf400640 1 -- 172.21.15.113:0/3463557986 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 191 v191) v1 ==== 95+0+5036 (secure 0 0 0) 0x7f67c00f10d0 con 0x7f67c8158010 2024-03-04T08:24:25.535 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:25.535 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":191,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":186}],"filesystems":[{"mdsmap":{"epoch":191,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:23:46.847290+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":4247,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4247":{"gid":4247,"name":"f","rank":0,"incarnation":173,"state":"up:active","state_seq":729,"addr":"172.21.15.136:6839/1409794802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":1409794802},{"type":"v1","addr":"172.21.15.136:6839","nonce":1409794802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4247,"qdb_cluster":[4247,8032]},"id":1}]} 2024-03-04T08:24:25.535 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 191 2024-03-04T08:24:25.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 -- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f67a4066f90 msgr2=0x7f67a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f67a4066f90 0x7f67a4069450 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f67b80011f0 tx=0x7f67b8009040 comp rx=0 tx=0).stop 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 -- 172.21.15.113:0/3463557986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c8158010 msgr2=0x7f67c81570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c8158010 0x7f67c81570e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f67c001a4e0 tx=0x7f67c0019650 comp rx=0 tx=0).stop 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 -- 172.21.15.113:0/3463557986 shutdown_connections 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f67a4066f90 0x7f67a4069450 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67c8158010 0x7f67c81570e0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67c806ff20 0x7f67c81f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 --2- 172.21.15.113:0/3463557986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67c806dd50 0x7f67c80b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 -- 172.21.15.113:0/3463557986 >> 172.21.15.113:0/3463557986 conn(0x7f67c80734d0 msgr2=0x7f67c80afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 -- 172.21.15.113:0/3463557986 shutdown_connections 2024-03-04T08:24:25.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:25.539+0000 7f67cd400640 1 -- 172.21.15.113:0/3463557986 wait complete. 2024-03-04T08:24:25.556 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2024-03-04T08:24:27.558 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:27.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.707+0000 7efc57e00640 1 Processor -- start 2024-03-04T08:24:27.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.707+0000 7efc57e00640 1 -- start start 2024-03-04T08:24:27.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc57e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efc500c1860 0x7efc50162c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:27.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc57e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc50156160 con 0x7efc50155d80 2024-03-04T08:24:27.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc57e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5015ef10 con 0x7efc501631b0 2024-03-04T08:24:27.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc57e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc50162500 con 0x7efc500c1860 2024-03-04T08:24:27.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc54a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efc500c1860 0x7efc50162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:27.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc55400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efc50155d80 0x7efc500c7dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49162/0 (socket says 172.21.15.113:49162) 2024-03-04T08:24:27.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc55400640 1 -- 172.21.15.113:0/1387708629 learned_addr learned my addr 172.21.15.113:0/1387708629 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:27.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc4fe00640 1 -- 172.21.15.113:0/1387708629 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236286062 0 0) 0x7efc50156160 con 0x7efc50155d80 2024-03-04T08:24:27.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc4fe00640 1 -- 172.21.15.113:0/1387708629 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc38003530 con 0x7efc50155d80 2024-03-04T08:24:27.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc54a00640 1 -- 172.21.15.113:0/1387708629 >> v1:172.21.15.136:6789/0 conn(0x7efc501631b0 legacy=0x7efc500c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:27.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc54a00640 1 -- 172.21.15.113:0/1387708629 >> v1:172.21.15.113:6789/0 conn(0x7efc50155d80 legacy=0x7efc500c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:27.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.711+0000 7efc54a00640 1 -- 172.21.15.113:0/1387708629 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5015f1b0 con 0x7efc500c1860 2024-03-04T08:24:27.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc54a00640 1 --2- 172.21.15.113:0/1387708629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc500c1860 0x7efc50162c70 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7efc50156330 tx=0x7efc3c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932d46e2b65f0921 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:27.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc4fe00640 1 -- 172.21.15.113:0/1387708629 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0b1070 con 0x7efc500c1860 2024-03-04T08:24:27.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc4fe00640 1 -- 172.21.15.113:0/1387708629 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c0ac5f0 con 0x7efc500c1860 2024-03-04T08:24:27.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc4fe00640 1 -- 172.21.15.113:0/1387708629 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0ac9c0 con 0x7efc500c1860 2024-03-04T08:24:27.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc57e00640 1 -- 172.21.15.113:0/1387708629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc500c1860 msgr2=0x7efc50162c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:27.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc57e00640 1 --2- 172.21.15.113:0/1387708629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc500c1860 0x7efc50162c70 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7efc50156330 tx=0x7efc3c01c4a0 comp rx=0 tx=0).stop 2024-03-04T08:24:27.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc57e00640 1 -- 172.21.15.113:0/1387708629 shutdown_connections 2024-03-04T08:24:27.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc57e00640 1 --2- 172.21.15.113:0/1387708629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc500c1860 0x7efc50162c70 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:27.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc57e00640 1 -- 172.21.15.113:0/1387708629 >> 172.21.15.113:0/1387708629 conn(0x7efc5006e820 msgr2=0x7efc5006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:27.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc57e00640 1 -- 172.21.15.113:0/1387708629 shutdown_connections 2024-03-04T08:24:27.713 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc57e00640 1 -- 172.21.15.113:0/1387708629 wait complete. 2024-03-04T08:24:27.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc57e00640 1 Processor -- start 2024-03-04T08:24:27.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc57e00640 1 -- start start 2024-03-04T08:24:27.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.715+0000 7efc57e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc500c1860 0x7efc500ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc57e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50155d80 0x7efc501f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc57e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc501631b0 0x7efc501fb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc57e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc500c1e10 con 0x7efc50155d80 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc57e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5015fcd0 con 0x7efc500c1860 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc57e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc500c7760 con 0x7efc501631b0 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc54a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50155d80 0x7efc501f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc54a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50155d80 0x7efc501f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43536/0 (socket says 172.21.15.113:43536) 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc54a00640 1 -- 172.21.15.113:0/466352248 learned_addr learned my addr 172.21.15.113:0/466352248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc55400640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc500c1860 0x7efc500ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc55e00640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc501631b0 0x7efc501fb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc54a00640 1 -- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc501631b0 msgr2=0x7efc501fb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:27.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc54a00640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc501631b0 0x7efc501fb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:27.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc54a00640 1 -- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc500c1860 msgr2=0x7efc500ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:27.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc54a00640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc500c1860 0x7efc500ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:27.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc54a00640 1 -- 172.21.15.113:0/466352248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5015c4b0 con 0x7efc50155d80 2024-03-04T08:24:27.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc54a00640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50155d80 0x7efc501f7a70 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0af040 tx=0x7efc3c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:27.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc4d600640 1 -- 172.21.15.113:0/466352248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0b1040 con 0x7efc50155d80 2024-03-04T08:24:27.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc4d600640 1 -- 172.21.15.113:0/466352248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c0bb070 con 0x7efc50155d80 2024-03-04T08:24:27.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc4d600640 1 -- 172.21.15.113:0/466352248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0b5870 con 0x7efc50155d80 2024-03-04T08:24:27.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc57e00640 1 -- 172.21.15.113:0/466352248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc50155600 con 0x7efc50155d80 2024-03-04T08:24:27.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc57e00640 1 -- 172.21.15.113:0/466352248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5015b9d0 con 0x7efc50155d80 2024-03-04T08:24:27.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc57e00640 1 -- 172.21.15.113:0/466352248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc50075c00 con 0x7efc50155d80 2024-03-04T08:24:27.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc4d600640 1 -- 172.21.15.113:0/466352248 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc3c0b5a10 con 0x7efc50155d80 2024-03-04T08:24:27.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.719+0000 7efc4d600640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc30066f60 0x7efc30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:27.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.723+0000 7efc4d600640 1 -- 172.21.15.113:0/466352248 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7efc3c11f7b0 con 0x7efc50155d80 2024-03-04T08:24:27.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.723+0000 7efc55400640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc30066f60 0x7efc30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:27.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.723+0000 7efc55400640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc30066f60 0x7efc30069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7efc440011f0 tx=0x7efc44007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:27.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.723+0000 7efc4d600640 1 -- 172.21.15.113:0/466352248 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efc3c0a9070 con 0x7efc50155d80 2024-03-04T08:24:27.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.911+0000 7efc57e00640 1 -- 172.21.15.113:0/466352248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc500c7bb0 con 0x7efc50155d80 2024-03-04T08:24:27.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.911+0000 7efc4d600640 1 -- 172.21.15.113:0/466352248 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+4283 (secure 0 0 0) 0x7efc3c016090 con 0x7efc50155d80 2024-03-04T08:24:27.909 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:27.909 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:26.023887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:27.910 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 192 2024-03-04T08:24:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 -- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc30066f60 msgr2=0x7efc30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc30066f60 0x7efc30069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7efc440011f0 tx=0x7efc44007d40 comp rx=0 tx=0).stop 2024-03-04T08:24:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 -- 172.21.15.113:0/466352248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50155d80 msgr2=0x7efc501f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:27.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50155d80 0x7efc501f7a70 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0af040 tx=0x7efc3c019240 comp rx=0 tx=0).stop 2024-03-04T08:24:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 -- 172.21.15.113:0/466352248 shutdown_connections 2024-03-04T08:24:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efc501631b0 0x7efc501fb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efc30066f60 0x7efc30069420 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc50155d80 0x7efc501f7a70 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 --2- 172.21.15.113:0/466352248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc500c1860 0x7efc500ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 -- 172.21.15.113:0/466352248 >> 172.21.15.113:0/466352248 conn(0x7efc5006e820 msgr2=0x7efc50072440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 -- 172.21.15.113:0/466352248 shutdown_connections 2024-03-04T08:24:27.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:27.915+0000 7efc57e00640 1 -- 172.21.15.113:0/466352248 wait complete. 2024-03-04T08:24:27.930 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2024-03-04T08:24:27.930 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2024-03-04T08:24:27.930 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.067+0000 7f6777e00640 1 Processor -- start 2024-03-04T08:24:28.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.067+0000 7f6777e00640 1 -- start start 2024-03-04T08:24:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6777e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67701527f0 0x7f6770070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6777e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f67700c8200 0x7f6770072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:28.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6777e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67701591f0 con 0x7f6770152bd0 2024-03-04T08:24:28.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6777e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6770071df0 con 0x7f67700c8200 2024-03-04T08:24:28.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6777e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67701597b0 con 0x7f67701527f0 2024-03-04T08:24:28.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6775400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67701527f0 0x7f6770070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:28.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6774a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f67700c8200 0x7f6770072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:28.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6775400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67701527f0 0x7f6770070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:50424/0 (socket says 172.21.15.113:50424) 2024-03-04T08:24:28.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6775e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f6770152bd0 0x7f67701587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:39462/0 (socket says 172.21.15.113:39462) 2024-03-04T08:24:28.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6775400640 1 -- 172.21.15.113:0/3176076700 learned_addr learned my addr 172.21.15.113:0/3176076700 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6775400640 1 -- 172.21.15.113:0/3176076700 >> v2:172.21.15.136:3300/0 conn(0x7f67700c8200 msgr2=0x7f6770072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6775400640 1 --2- 172.21.15.113:0/3176076700 >> v2:172.21.15.136:3300/0 conn(0x7f67700c8200 0x7f6770072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6775400640 1 -- 172.21.15.113:0/3176076700 >> v1:172.21.15.136:6789/0 conn(0x7f6770152bd0 legacy=0x7f67701587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:28.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.071+0000 7f6775400640 1 -- 172.21.15.113:0/3176076700 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67701594d0 con 0x7f67701527f0 2024-03-04T08:24:28.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6775400640 1 --2- 172.21.15.113:0/3176076700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67701527f0 0x7f6770070d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6760009870 tx=0x7f676001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec1bc1a274260c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:28.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f676fe00640 1 -- 172.21.15.113:0/3176076700 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600a7070 con 0x7f67701527f0 2024-03-04T08:24:28.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f676fe00640 1 -- 172.21.15.113:0/3176076700 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67600b0070 con 0x7f67701527f0 2024-03-04T08:24:28.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f676fe00640 1 -- 172.21.15.113:0/3176076700 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600ab890 con 0x7f67701527f0 2024-03-04T08:24:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 -- 172.21.15.113:0/3176076700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67701527f0 msgr2=0x7f6770070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 --2- 172.21.15.113:0/3176076700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67701527f0 0x7f6770070d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6760009870 tx=0x7f676001c560 comp rx=0 tx=0).stop 2024-03-04T08:24:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 -- 172.21.15.113:0/3176076700 shutdown_connections 2024-03-04T08:24:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 --2- 172.21.15.113:0/3176076700 >> v2:172.21.15.136:3300/0 conn(0x7f67700c8200 0x7f6770072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 --2- 172.21.15.113:0/3176076700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67701527f0 0x7f6770070d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 -- 172.21.15.113:0/3176076700 >> 172.21.15.113:0/3176076700 conn(0x7f677006e820 msgr2=0x7f677006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:28.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 -- 172.21.15.113:0/3176076700 shutdown_connections 2024-03-04T08:24:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 -- 172.21.15.113:0/3176076700 wait complete. 2024-03-04T08:24:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 Processor -- start 2024-03-04T08:24:28.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 -- start start 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67700c8200 0x7f677014f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67701527f0 0x7f677014ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6770152bd0 0x7f677014a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6770165160 con 0x7f6770152bd0 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700727c0 con 0x7f67701527f0 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.075+0000 7f6777e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67701652e0 con 0x7f67700c8200 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6775e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6770152bd0 0x7f677014a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6775e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6770152bd0 0x7f677014a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43552/0 (socket says 172.21.15.113:43552) 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6775e00640 1 -- 172.21.15.113:0/4049205093 learned_addr learned my addr 172.21.15.113:0/4049205093 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6775400640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67700c8200 0x7f677014f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6774a00640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67701527f0 0x7f677014ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:28.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6775e00640 1 -- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67700c8200 msgr2=0x7f677014f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:28.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6775e00640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67700c8200 0x7f677014f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:28.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6775e00640 1 -- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67701527f0 msgr2=0x7f677014ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:28.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6775e00640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67701527f0 0x7f677014ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:28.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6775e00640 1 -- 172.21.15.113:0/4049205093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67701f1f90 con 0x7f6770152bd0 2024-03-04T08:24:28.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6775e00640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6770152bd0 0x7f677014a320 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f676800c730 tx=0x7f676800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:28.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f676d600640 1 -- 172.21.15.113:0/4049205093 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768017070 con 0x7f6770152bd0 2024-03-04T08:24:28.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f676d600640 1 -- 172.21.15.113:0/4049205093 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676800ce40 con 0x7f6770152bd0 2024-03-04T08:24:28.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6777e00640 1 -- 172.21.15.113:0/4049205093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67701f2250 con 0x7f6770152bd0 2024-03-04T08:24:28.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f676d600640 1 -- 172.21.15.113:0/4049205093 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676800a7e0 con 0x7f6770152bd0 2024-03-04T08:24:28.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6777e00640 1 -- 172.21.15.113:0/4049205093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67701f2af0 con 0x7f6770152bd0 2024-03-04T08:24:28.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f6777e00640 1 -- 172.21.15.113:0/4049205093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6770075c00 con 0x7f6770152bd0 2024-03-04T08:24:28.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f676d600640 1 -- 172.21.15.113:0/4049205093 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f676801a070 con 0x7f6770152bd0 2024-03-04T08:24:28.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f676d600640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6754066f90 0x7f6754069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:28.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.079+0000 7f676d600640 1 -- 172.21.15.113:0/4049205093 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f6768084ed0 con 0x7f6770152bd0 2024-03-04T08:24:28.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.083+0000 7f6775400640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6754066f90 0x7f6754069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:28.079 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.083+0000 7f6775400640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6754066f90 0x7f6754069450 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f67600af000 tx=0x7f67600a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:28.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.083+0000 7f676d600640 1 -- 172.21.15.113:0/4049205093 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6768054de0 con 0x7f6770152bd0 2024-03-04T08:24:28.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 -- 172.21.15.113:0/4049205093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67700c79d0 con 0x7f6770152bd0 2024-03-04T08:24:28.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f676d600640 1 -- 172.21.15.113:0/4049205093 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+4283 (secure 0 0 0) 0x7f6768058e80 con 0x7f6770152bd0 2024-03-04T08:24:28.272 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:28.272 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:26.023887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:28.272 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 192 2024-03-04T08:24:28.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 -- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6754066f90 msgr2=0x7f6754069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:28.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6754066f90 0x7f6754069450 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f67600af000 tx=0x7f67600a7000 comp rx=0 tx=0).stop 2024-03-04T08:24:28.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 -- 172.21.15.113:0/4049205093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6770152bd0 msgr2=0x7f677014a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:28.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6770152bd0 0x7f677014a320 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f676800c730 tx=0x7f676800cbf0 comp rx=0 tx=0).stop 2024-03-04T08:24:28.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 -- 172.21.15.113:0/4049205093 shutdown_connections 2024-03-04T08:24:28.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6754066f90 0x7f6754069450 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6770152bd0 0x7f677014a320 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67701527f0 0x7f677014ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 --2- 172.21.15.113:0/4049205093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67700c8200 0x7f677014f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 -- 172.21.15.113:0/4049205093 >> 172.21.15.113:0/4049205093 conn(0x7f677006e820 msgr2=0x7f67700c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 -- 172.21.15.113:0/4049205093 shutdown_connections 2024-03-04T08:24:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:28.275+0000 7f6777e00640 1 -- 172.21.15.113:0/4049205093 wait complete. 2024-03-04T08:24:28.293 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:24:28.294 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 192, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 192, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:26.023887+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:24:30.295 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:30.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.443+0000 7f537ea00640 1 Processor -- start 2024-03-04T08:24:30.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.443+0000 7f537ea00640 1 -- start start 2024-03-04T08:24:30.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f53780c85f0 0x7f537806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:30.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f537806b5a0 0x7f5378161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:30.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ea00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53780c1ec0 con 0x7f53780b20b0 2024-03-04T08:24:30.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53780c9b30 con 0x7f537806b5a0 2024-03-04T08:24:30.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ea00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53780c12a0 con 0x7f53780c85f0 2024-03-04T08:24:30.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f537806b5a0 0x7f5378161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:30.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f537806b5a0 0x7f5378161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43564/0 (socket says 172.21.15.113:43564) 2024-03-04T08:24:30.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ca00640 1 -- 172.21.15.113:0/2670759937 learned_addr learned my addr 172.21.15.113:0/2670759937 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:30.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f536f400640 1 --2- 172.21.15.113:0/2670759937 >> v2:172.21.15.136:3300/0 conn(0x7f53780c85f0 0x7f537806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:30.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ca00640 1 -- 172.21.15.113:0/2670759937 >> v2:172.21.15.136:3300/0 conn(0x7f53780c85f0 msgr2=0x7f537806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:30.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ca00640 1 --2- 172.21.15.113:0/2670759937 >> v2:172.21.15.136:3300/0 conn(0x7f53780c85f0 0x7f537806b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:30.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ca00640 1 -- 172.21.15.113:0/2670759937 >> v1:172.21.15.136:6789/0 conn(0x7f53780b20b0 legacy=0x7f5378157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:30.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f537ca00640 1 -- 172.21.15.113:0/2670759937 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53780c1580 con 0x7f537806b5a0 2024-03-04T08:24:30.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.447+0000 7f536f400640 1 --2- 172.21.15.113:0/2670759937 >> v2:172.21.15.136:3300/0 conn(0x7f53780c85f0 0x7f537806b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:24:30.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ca00640 1 --2- 172.21.15.113:0/2670759937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f537806b5a0 0x7f5378161820 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f537806f5b0 tx=0x7f53700200e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6706cc124265e520 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:30.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f5377400640 1 -- 172.21.15.113:0/2670759937 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53700b6070 con 0x7f537806b5a0 2024-03-04T08:24:30.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f5377400640 1 -- 172.21.15.113:0/2670759937 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53700b1440 con 0x7f537806b5a0 2024-03-04T08:24:30.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f5377400640 1 -- 172.21.15.113:0/2670759937 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537000b040 con 0x7f537806b5a0 2024-03-04T08:24:30.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 -- 172.21.15.113:0/2670759937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f537806b5a0 msgr2=0x7f5378161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:30.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 --2- 172.21.15.113:0/2670759937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f537806b5a0 0x7f5378161820 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f537806f5b0 tx=0x7f53700200e0 comp rx=0 tx=0).stop 2024-03-04T08:24:30.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 -- 172.21.15.113:0/2670759937 shutdown_connections 2024-03-04T08:24:30.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 --2- 172.21.15.113:0/2670759937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f537806b5a0 0x7f5378161820 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:30.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 --2- 172.21.15.113:0/2670759937 >> v2:172.21.15.136:3300/0 conn(0x7f53780c85f0 0x7f537806b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:30.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 -- 172.21.15.113:0/2670759937 >> 172.21.15.113:0/2670759937 conn(0x7f537806e820 msgr2=0x7f537806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:30.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 -- 172.21.15.113:0/2670759937 shutdown_connections 2024-03-04T08:24:30.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 -- 172.21.15.113:0/2670759937 wait complete. 2024-03-04T08:24:30.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 Processor -- start 2024-03-04T08:24:30.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 -- start start 2024-03-04T08:24:30.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f537806b5a0 0x7f53781f0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:30.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53780b20b0 0x7f5378072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:30.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780c85f0 0x7f53780b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:30.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5378157e20 con 0x7f53780c85f0 2024-03-04T08:24:30.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53781571f0 con 0x7f537806b5a0 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.451+0000 7f537ea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53780c2400 con 0x7f53780b20b0 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780c85f0 0x7f53780b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780c85f0 0x7f53780b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43578/0 (socket says 172.21.15.113:43578) 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ca00640 1 -- 172.21.15.113:0/2938599050 learned_addr learned my addr 172.21.15.113:0/2938599050 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f536f400640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53780b20b0 0x7f5378072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f5377e00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f537806b5a0 0x7f53781f0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ca00640 1 -- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53780b20b0 msgr2=0x7f5378072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ca00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53780b20b0 0x7f5378072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ca00640 1 -- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f537806b5a0 msgr2=0x7f53781f0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ca00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f537806b5a0 0x7f53781f0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ca00640 1 -- 172.21.15.113:0/2938599050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53781f1210 con 0x7f53780c85f0 2024-03-04T08:24:30.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ca00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780c85f0 0x7f53780b0900 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f537000a1c0 tx=0x7f537001e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:30.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f5374c00640 1 -- 172.21.15.113:0/2938599050 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53700bf070 con 0x7f53780c85f0 2024-03-04T08:24:30.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f5374c00640 1 -- 172.21.15.113:0/2938599050 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53700af550 con 0x7f53780c85f0 2024-03-04T08:24:30.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ea00640 1 -- 172.21.15.113:0/2938599050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5378155820 con 0x7f53780c85f0 2024-03-04T08:24:30.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f5374c00640 1 -- 172.21.15.113:0/2938599050 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53700b6030 con 0x7f53780c85f0 2024-03-04T08:24:30.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ea00640 1 -- 172.21.15.113:0/2938599050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537814f090 con 0x7f53780c85f0 2024-03-04T08:24:30.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f537ea00640 1 -- 172.21.15.113:0/2938599050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5378075c00 con 0x7f53780c85f0 2024-03-04T08:24:30.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f5374c00640 1 -- 172.21.15.113:0/2938599050 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53700c2070 con 0x7f53780c85f0 2024-03-04T08:24:30.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.455+0000 7f5374c00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5354066f90 0x7f5354069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:30.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.459+0000 7f5374c00640 1 -- 172.21.15.113:0/2938599050 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f5370122ce0 con 0x7f53780c85f0 2024-03-04T08:24:30.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.459+0000 7f5377e00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5354066f90 0x7f5354069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:30.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.459+0000 7f5377e00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5354066f90 0x7f5354069450 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5368001ea0 tx=0x7f5368009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:30.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.463+0000 7f5374c00640 1 -- 172.21.15.113:0/2938599050 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5370019c40 con 0x7f53780c85f0 2024-03-04T08:24:30.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 -- 172.21.15.113:0/2938599050 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53780c83d0 con 0x7f53780c85f0 2024-03-04T08:24:30.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f5374c00640 1 -- 172.21.15.113:0/2938599050 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+4283 (secure 0 0 0) 0x7f53700f2bc0 con 0x7f53780c85f0 2024-03-04T08:24:30.648 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:30.648 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:26.023887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:30.648 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 192 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 -- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5354066f90 msgr2=0x7f5354069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5354066f90 0x7f5354069450 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5368001ea0 tx=0x7f5368009040 comp rx=0 tx=0).stop 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 -- 172.21.15.113:0/2938599050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780c85f0 msgr2=0x7f53780b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780c85f0 0x7f53780b0900 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f537000a1c0 tx=0x7f537001e920 comp rx=0 tx=0).stop 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 -- 172.21.15.113:0/2938599050 shutdown_connections 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5354066f90 0x7f5354069450 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f53780c85f0 0x7f53780b0900 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f53780b20b0 0x7f5378072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 --2- 172.21.15.113:0/2938599050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f537806b5a0 0x7f53781f0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 -- 172.21.15.113:0/2938599050 >> 172.21.15.113:0/2938599050 conn(0x7f537806e820 msgr2=0x7f53780cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 -- 172.21.15.113:0/2938599050 shutdown_connections 2024-03-04T08:24:30.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:30.651+0000 7f537ea00640 1 -- 172.21.15.113:0/2938599050 wait complete. 2024-03-04T08:24:30.668 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:24:32.669 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:32.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.807+0000 7ffb33800640 1 Processor -- start 2024-03-04T08:24:32.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.807+0000 7ffb33800640 1 -- start start 2024-03-04T08:24:32.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb33800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb2c14e4c0 0x7ffb2c072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:32.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb33800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c15b900 0x7ffb2c0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:32.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb33800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2c0c27f0 con 0x7ffb2c0c22e0 2024-03-04T08:24:32.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb33800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2c15ad60 con 0x7ffb2c14e4c0 2024-03-04T08:24:32.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb33800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c0c2a90 con 0x7ffb2c15b900 2024-03-04T08:24:32.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb30e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb2c14e4c0 0x7ffb2c072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:32.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb2be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ffb2c0c22e0 0x7ffb2c15b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:49178/0 (socket says 172.21.15.113:49178) 2024-03-04T08:24:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb2be00640 1 -- 172.21.15.113:0/3678272542 learned_addr learned my addr 172.21.15.113:0/3678272542 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb30e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffb2c14e4c0 0x7ffb2c072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43584/0 (socket says 172.21.15.113:43584) 2024-03-04T08:24:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb31800640 1 --2- 172.21.15.113:0/3678272542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c15b900 0x7ffb2c0731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb2b400640 1 -- 172.21.15.113:0/3678272542 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187774591 0 0) 0x7ffb2c0c27f0 con 0x7ffb2c0c22e0 2024-03-04T08:24:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb30e00640 1 -- 172.21.15.113:0/3678272542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c15b900 msgr2=0x7ffb2c0731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb30e00640 1 --2- 172.21.15.113:0/3678272542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c15b900 0x7ffb2c0731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb30e00640 1 -- 172.21.15.113:0/3678272542 >> v1:172.21.15.113:6789/0 conn(0x7ffb2c0c22e0 legacy=0x7ffb2c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.811+0000 7ffb30e00640 1 -- 172.21.15.113:0/3678272542 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c070c60 con 0x7ffb2c14e4c0 2024-03-04T08:24:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb30e00640 1 --2- 172.21.15.113:0/3678272542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c14e4c0 0x7ffb2c072c80 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009d90 tx=0x7ffb1c01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a172879e4a3fc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb2b400640 1 -- 172.21.15.113:0/3678272542 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0b4070 con 0x7ffb2c14e4c0 2024-03-04T08:24:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb2b400640 1 -- 172.21.15.113:0/3678272542 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c0af600 con 0x7ffb2c14e4c0 2024-03-04T08:24:32.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb2b400640 1 -- 172.21.15.113:0/3678272542 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0af9d0 con 0x7ffb2c14e4c0 2024-03-04T08:24:32.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb33800640 1 -- 172.21.15.113:0/3678272542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c14e4c0 msgr2=0x7ffb2c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:32.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb33800640 1 --2- 172.21.15.113:0/3678272542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c14e4c0 0x7ffb2c072c80 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009d90 tx=0x7ffb1c01f150 comp rx=0 tx=0).stop 2024-03-04T08:24:32.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb33800640 1 -- 172.21.15.113:0/3678272542 shutdown_connections 2024-03-04T08:24:32.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb33800640 1 --2- 172.21.15.113:0/3678272542 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c15b900 0x7ffb2c0731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:32.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb33800640 1 --2- 172.21.15.113:0/3678272542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c14e4c0 0x7ffb2c072c80 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb33800640 1 -- 172.21.15.113:0/3678272542 >> 172.21.15.113:0/3678272542 conn(0x7ffb2c06e820 msgr2=0x7ffb2c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb33800640 1 -- 172.21.15.113:0/3678272542 shutdown_connections 2024-03-04T08:24:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.815+0000 7ffb33800640 1 -- 172.21.15.113:0/3678272542 wait complete. 2024-03-04T08:24:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb33800640 1 Processor -- start 2024-03-04T08:24:32.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb33800640 1 -- start start 2024-03-04T08:24:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb33800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb2c0c22e0 0x7ffb2c13dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb33800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c14e4c0 0x7ffb2c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb33800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c15b900 0x7ffb2c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:32.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb33800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c071500 con 0x7ffb2c15b900 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb33800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c071220 con 0x7ffb2c0c22e0 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb33800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c14e8a0 con 0x7ffb2c14e4c0 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb31800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c15b900 0x7ffb2c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb31800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c15b900 0x7ffb2c0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43598/0 (socket says 172.21.15.113:43598) 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb31800640 1 -- 172.21.15.113:0/277817248 learned_addr learned my addr 172.21.15.113:0/277817248 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb2be00640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c14e4c0 0x7ffb2c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb30e00640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb2c0c22e0 0x7ffb2c13dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb31800640 1 -- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c14e4c0 msgr2=0x7ffb2c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb31800640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c14e4c0 0x7ffb2c0cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb31800640 1 -- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb2c0c22e0 msgr2=0x7ffb2c13dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb31800640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb2c0c22e0 0x7ffb2c13dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb31800640 1 -- 172.21.15.113:0/277817248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c0b0d60 con 0x7ffb2c15b900 2024-03-04T08:24:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb2be00640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c14e4c0 0x7ffb2c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:24:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb30e00640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb2c0c22e0 0x7ffb2c13dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:24:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb31800640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c15b900 0x7ffb2c0cb630 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400e8b0 tx=0x7ffb2400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb28c00640 1 -- 172.21.15.113:0/277817248 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24017070 con 0x7ffb2c15b900 2024-03-04T08:24:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb28c00640 1 -- 172.21.15.113:0/277817248 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2400c940 con 0x7ffb2c15b900 2024-03-04T08:24:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb28c00640 1 -- 172.21.15.113:0/277817248 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400cd10 con 0x7ffb2c15b900 2024-03-04T08:24:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb33800640 1 -- 172.21.15.113:0/277817248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c153110 con 0x7ffb2c15b900 2024-03-04T08:24:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.819+0000 7ffb33800640 1 -- 172.21.15.113:0/277817248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c1503f0 con 0x7ffb2c15b900 2024-03-04T08:24:32.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.823+0000 7ffb33800640 1 -- 172.21.15.113:0/277817248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c075c00 con 0x7ffb2c15b900 2024-03-04T08:24:32.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.823+0000 7ffb28c00640 1 -- 172.21.15.113:0/277817248 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb2401a070 con 0x7ffb2c15b900 2024-03-04T08:24:32.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.823+0000 7ffb28c00640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb08066f90 0x7ffb08069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:32.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.823+0000 7ffb28c00640 1 -- 172.21.15.113:0/277817248 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7ffb24085be0 con 0x7ffb2c15b900 2024-03-04T08:24:32.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.823+0000 7ffb30e00640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb08066f90 0x7ffb08069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:32.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.823+0000 7ffb30e00640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb08066f90 0x7ffb08069450 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0b1000 tx=0x7ffb1c0aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:32.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:32.827+0000 7ffb28c00640 1 -- 172.21.15.113:0/277817248 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ffb24056020 con 0x7ffb2c15b900 2024-03-04T08:24:33.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.026+0000 7ffb33800640 1 -- 172.21.15.113:0/277817248 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb2c14a750 con 0x7ffb2c15b900 2024-03-04T08:24:33.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb28c00640 1 -- 172.21.15.113:0/277817248 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+4283 (secure 0 0 0) 0x7ffb24055920 con 0x7ffb2c15b900 2024-03-04T08:24:33.027 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:33.028 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:26.023887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:replay","state_seq":1,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:33.028 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 192 2024-03-04T08:24:33.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 -- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb08066f90 msgr2=0x7ffb08069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:33.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb08066f90 0x7ffb08069450 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0b1000 tx=0x7ffb1c0aa000 comp rx=0 tx=0).stop 2024-03-04T08:24:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 -- 172.21.15.113:0/277817248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c15b900 msgr2=0x7ffb2c0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c15b900 0x7ffb2c0cb630 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400e8b0 tx=0x7ffb2400ed70 comp rx=0 tx=0).stop 2024-03-04T08:24:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 -- 172.21.15.113:0/277817248 shutdown_connections 2024-03-04T08:24:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ffb08066f90 0x7ffb08069450 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffb2c15b900 0x7ffb2c0cb630 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ffb2c14e4c0 0x7ffb2c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 --2- 172.21.15.113:0/277817248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb2c0c22e0 0x7ffb2c13dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 -- 172.21.15.113:0/277817248 >> 172.21.15.113:0/277817248 conn(0x7ffb2c06e820 msgr2=0x7ffb2c072ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 -- 172.21.15.113:0/277817248 shutdown_connections 2024-03-04T08:24:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:33.030+0000 7ffb33800640 1 -- 172.21.15.113:0/277817248 wait complete. 2024-03-04T08:24:33.049 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:24:35.050 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:35.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4125e00640 1 Processor -- start 2024-03-04T08:24:35.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4125e00640 1 -- start start 2024-03-04T08:24:35.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4125e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4120152950 0x7f412006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:35.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4125e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 0x7f4120071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:35.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4125e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f412006b890 0x7f412006bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:35.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4125e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412006b5a0 con 0x7f4120152950 2024-03-04T08:24:35.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4125e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412015c580 con 0x7f412006b890 2024-03-04T08:24:35.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4125e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412006efc0 con 0x7f41200c94c0 2024-03-04T08:24:35.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4124a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4120152950 0x7f412006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:35.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4124a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4120152950 0x7f412006b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43604/0 (socket says 172.21.15.113:43604) 2024-03-04T08:24:35.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4125400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f412006b890 0x7f412006bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:35.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4124a00640 1 -- 172.21.15.113:0/3712471096 learned_addr learned my addr 172.21.15.113:0/3712471096 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:35.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f411fe00640 1 --2- 172.21.15.113:0/3712471096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 0x7f4120071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:35.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4124a00640 1 -- 172.21.15.113:0/3712471096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 msgr2=0x7f4120071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:35.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4124a00640 1 --2- 172.21.15.113:0/3712471096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 0x7f4120071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4124a00640 1 -- 172.21.15.113:0/3712471096 >> v2:172.21.15.136:3300/0 conn(0x7f412006b890 msgr2=0x7f412006bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:35.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4124a00640 1 --2- 172.21.15.113:0/3712471096 >> v2:172.21.15.136:3300/0 conn(0x7f412006b890 0x7f412006bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f4124a00640 1 -- 172.21.15.113:0/3712471096 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120152d30 con 0x7f4120152950 2024-03-04T08:24:35.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.194+0000 7f411fe00640 1 --2- 172.21.15.113:0/3712471096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 0x7f4120071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:24:35.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f4124a00640 1 --2- 172.21.15.113:0/3712471096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4120152950 0x7f412006b060 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f41200cd010 tx=0x7f411401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8e4d85fed16b68 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:35.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f411f400640 1 -- 172.21.15.113:0/3712471096 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41140b5070 con 0x7f4120152950 2024-03-04T08:24:35.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f411f400640 1 -- 172.21.15.113:0/3712471096 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41140a89d0 con 0x7f4120152950 2024-03-04T08:24:35.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f411f400640 1 -- 172.21.15.113:0/3712471096 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41140a8da0 con 0x7f4120152950 2024-03-04T08:24:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f4125e00640 1 -- 172.21.15.113:0/3712471096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4120152950 msgr2=0x7f412006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f4125e00640 1 --2- 172.21.15.113:0/3712471096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4120152950 0x7f412006b060 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f41200cd010 tx=0x7f411401ed10 comp rx=0 tx=0).stop 2024-03-04T08:24:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f4125e00640 1 -- 172.21.15.113:0/3712471096 shutdown_connections 2024-03-04T08:24:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f4125e00640 1 --2- 172.21.15.113:0/3712471096 >> v2:172.21.15.136:3300/0 conn(0x7f412006b890 0x7f412006bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f4125e00640 1 --2- 172.21.15.113:0/3712471096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 0x7f4120071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f4125e00640 1 --2- 172.21.15.113:0/3712471096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4120152950 0x7f412006b060 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f4125e00640 1 -- 172.21.15.113:0/3712471096 >> 172.21.15.113:0/3712471096 conn(0x7f41200cd080 msgr2=0x7f41200cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f4125e00640 1 -- 172.21.15.113:0/3712471096 shutdown_connections 2024-03-04T08:24:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.198+0000 7f4125e00640 1 -- 172.21.15.113:0/3712471096 wait complete. 2024-03-04T08:24:35.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 Processor -- start 2024-03-04T08:24:35.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 -- start start 2024-03-04T08:24:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f412006b890 0x7f4120151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 0x7f41201e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4120152950 0x7f41201e4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412015bc50 con 0x7f412006b890 2024-03-04T08:24:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41200c9260 con 0x7f4120152950 2024-03-04T08:24:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412015bf30 con 0x7f41200c94c0 2024-03-04T08:24:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4124a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f412006b890 0x7f4120151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4124a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f412006b890 0x7f4120151e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43608/0 (socket says 172.21.15.113:43608) 2024-03-04T08:24:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4124a00640 1 -- 172.21.15.113:0/3362023670 learned_addr learned my addr 172.21.15.113:0/3362023670 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125400640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4120152950 0x7f41201e4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f411fe00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 0x7f41201e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4124a00640 1 -- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 msgr2=0x7f41201e3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4124a00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 0x7f41201e3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4124a00640 1 -- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4120152950 msgr2=0x7f41201e4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4124a00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4120152950 0x7f41201e4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4124a00640 1 -- 172.21.15.113:0/3362023670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120155f90 con 0x7f412006b890 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4124a00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f412006b890 0x7f4120151e70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f41140b3040 tx=0x7f411401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f411cc00640 1 -- 172.21.15.113:0/3362023670 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41140b5070 con 0x7f412006b890 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f411cc00640 1 -- 172.21.15.113:0/3362023670 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41140b0a30 con 0x7f412006b890 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 -- 172.21.15.113:0/3362023670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41201f3bd0 con 0x7f412006b890 2024-03-04T08:24:35.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f411cc00640 1 -- 172.21.15.113:0/3362023670 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41140b0e20 con 0x7f412006b890 2024-03-04T08:24:35.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 -- 172.21.15.113:0/3362023670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41201f4170 con 0x7f412006b890 2024-03-04T08:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.202+0000 7f4125e00640 1 -- 172.21.15.113:0/3362023670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4120075a80 con 0x7f412006b890 2024-03-04T08:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.206+0000 7f411cc00640 1 -- 172.21.15.113:0/3362023670 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f41140c2060 con 0x7f412006b890 2024-03-04T08:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.206+0000 7f411cc00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4100066f60 0x7f4100069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.206+0000 7f411cc00640 1 -- 172.21.15.113:0/3362023670 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f4114122be0 con 0x7f412006b890 2024-03-04T08:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.206+0000 7f411fe00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4100066f60 0x7f4100069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:35.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.206+0000 7f411fe00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4100066f60 0x7f4100069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f41200c8540 tx=0x7f410c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:35.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.210+0000 7f411cc00640 1 -- 172.21.15.113:0/3362023670 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4114018870 con 0x7f412006b890 2024-03-04T08:24:35.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.394+0000 7f4125e00640 1 -- 172.21.15.113:0/3362023670 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41200727b0 con 0x7f412006b890 2024-03-04T08:24:35.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f411cc00640 1 -- 172.21.15.113:0/3362023670 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f41140f3020 con 0x7f412006b890 2024-03-04T08:24:35.395 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:35.396 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:35.396 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:35.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 -- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4100066f60 msgr2=0x7f4100069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:35.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4100066f60 0x7f4100069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f41200c8540 tx=0x7f410c008040 comp rx=0 tx=0).stop 2024-03-04T08:24:35.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 -- 172.21.15.113:0/3362023670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f412006b890 msgr2=0x7f4120151e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:35.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f412006b890 0x7f4120151e70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f41140b3040 tx=0x7f411401bab0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 -- 172.21.15.113:0/3362023670 shutdown_connections 2024-03-04T08:24:35.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4120152950 0x7f41201e4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41200c94c0 0x7f41201e3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4100066f60 0x7f4100069420 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 --2- 172.21.15.113:0/3362023670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f412006b890 0x7f4120151e70 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:35.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 -- 172.21.15.113:0/3362023670 >> 172.21.15.113:0/3362023670 conn(0x7f41200cd080 msgr2=0x7f41200cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:35.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 -- 172.21.15.113:0/3362023670 shutdown_connections 2024-03-04T08:24:35.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:35.398+0000 7f4125e00640 1 -- 172.21.15.113:0/3362023670 wait complete. 2024-03-04T08:24:35.416 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:24:37.417 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:37.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbea00640 1 Processor -- start 2024-03-04T08:24:37.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbea00640 1 -- start start 2024-03-04T08:24:37.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 0x7fadb80c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:37.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fadb80c18d0 0x7fadb8071f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:37.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbea00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb806c290 con 0x7fadb8151b40 2024-03-04T08:24:37.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbea00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb815aef0 con 0x7fadb80c18d0 2024-03-04T08:24:37.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb806ef10 con 0x7fadb8151760 2024-03-04T08:24:37.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbd600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 0x7fadb80c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:37.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbe000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fadb80c18d0 0x7fadb8071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:37.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbd600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 0x7fadb80c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:47984/0 (socket says 172.21.15.113:47984) 2024-03-04T08:24:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbd600640 1 -- 172.21.15.113:0/2914729207 learned_addr learned my addr 172.21.15.113:0/2914729207 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbd600640 1 -- 172.21.15.113:0/2914729207 >> v2:172.21.15.136:3300/0 conn(0x7fadb80c18d0 msgr2=0x7fadb8071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbd600640 1 --2- 172.21.15.113:0/2914729207 >> v2:172.21.15.136:3300/0 conn(0x7fadb80c18d0 0x7fadb8071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbd600640 1 -- 172.21.15.113:0/2914729207 >> v1:172.21.15.136:6789/0 conn(0x7fadb8151b40 legacy=0x7fadb815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:37.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.566+0000 7fadbd600640 1 -- 172.21.15.113:0/2914729207 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb815ba90 con 0x7fadb8151760 2024-03-04T08:24:37.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadbd600640 1 --2- 172.21.15.113:0/2914729207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 0x7fadb80c83d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fadac009870 tx=0x7fadac01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab987e98a2f193e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:37.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadb7e00640 1 -- 172.21.15.113:0/2914729207 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac0b1070 con 0x7fadb8151760 2024-03-04T08:24:37.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadb7e00640 1 -- 172.21.15.113:0/2914729207 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac0a8050 con 0x7fadb8151760 2024-03-04T08:24:37.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadb7e00640 1 -- 172.21.15.113:0/2914729207 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac0ac860 con 0x7fadb8151760 2024-03-04T08:24:37.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadbea00640 1 -- 172.21.15.113:0/2914729207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 msgr2=0x7fadb80c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:37.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadbea00640 1 --2- 172.21.15.113:0/2914729207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 0x7fadb80c83d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fadac009870 tx=0x7fadac01c4d0 comp rx=0 tx=0).stop 2024-03-04T08:24:37.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadbea00640 1 -- 172.21.15.113:0/2914729207 shutdown_connections 2024-03-04T08:24:37.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadbea00640 1 --2- 172.21.15.113:0/2914729207 >> v2:172.21.15.136:3300/0 conn(0x7fadb80c18d0 0x7fadb8071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:37.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadbea00640 1 --2- 172.21.15.113:0/2914729207 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 0x7fadb80c83d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:37.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadbea00640 1 -- 172.21.15.113:0/2914729207 >> 172.21.15.113:0/2914729207 conn(0x7fadb806e2a0 msgr2=0x7fadb806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:37.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadbea00640 1 -- 172.21.15.113:0/2914729207 shutdown_connections 2024-03-04T08:24:37.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.570+0000 7fadbea00640 1 -- 172.21.15.113:0/2914729207 wait complete. 2024-03-04T08:24:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbea00640 1 Processor -- start 2024-03-04T08:24:37.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbea00640 1 -- start start 2024-03-04T08:24:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb80c18d0 0x7fadb81f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 0x7fadb81f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadb8151b40 0x7fadb813dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80728c0 con 0x7fadb80c18d0 2024-03-04T08:24:37.574 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb815ab80 con 0x7fadb8151b40 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80c1e20 con 0x7fadb8151760 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb80c18d0 0x7fadb81f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbd600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb80c18d0 0x7fadb81f4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38634/0 (socket says 172.21.15.113:38634) 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbd600640 1 -- 172.21.15.113:0/595435894 learned_addr learned my addr 172.21.15.113:0/595435894 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbcc00640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 0x7fadb81f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbe000640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadb8151b40 0x7fadb813dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbd600640 1 -- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 msgr2=0x7fadb81f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbd600640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 0x7fadb81f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbd600640 1 -- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadb8151b40 msgr2=0x7fadb813dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbd600640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadb8151b40 0x7fadb813dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbd600640 1 -- 172.21.15.113:0/595435894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb81f7dd0 con 0x7fadb80c18d0 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbe000640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadb8151b40 0x7fadb813dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbd600640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb80c18d0 0x7fadb81f4e90 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fadac0af040 tx=0x7fadac01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadb5600640 1 -- 172.21.15.113:0/595435894 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac0b1040 con 0x7fadb80c18d0 2024-03-04T08:24:37.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadb5600640 1 -- 172.21.15.113:0/595435894 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac0bd070 con 0x7fadb80c18d0 2024-03-04T08:24:37.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbea00640 1 -- 172.21.15.113:0/595435894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb815a100 con 0x7fadb80c18d0 2024-03-04T08:24:37.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadb5600640 1 -- 172.21.15.113:0/595435894 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac0b7660 con 0x7fadb80c18d0 2024-03-04T08:24:37.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.574+0000 7fadbea00640 1 -- 172.21.15.113:0/595435894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb815a6c0 con 0x7fadb80c18d0 2024-03-04T08:24:37.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.578+0000 7fadbea00640 1 -- 172.21.15.113:0/595435894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb8074e10 con 0x7fadb80c18d0 2024-03-04T08:24:37.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.578+0000 7fadb5600640 1 -- 172.21.15.113:0/595435894 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fadac0b7800 con 0x7fadb80c18d0 2024-03-04T08:24:37.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.578+0000 7fadb5600640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad98066f90 0x7fad98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:37.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.578+0000 7fadb5600640 1 -- 172.21.15.113:0/595435894 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fadac11fe00 con 0x7fadb80c18d0 2024-03-04T08:24:37.577 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.578+0000 7fadbcc00640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad98066f90 0x7fad98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:37.578 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.578+0000 7fadbcc00640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad98066f90 0x7fad98069450 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fadb806de10 tx=0x7fada4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:37.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.582+0000 7fadb5600640 1 -- 172.21.15.113:0/595435894 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fadac016150 con 0x7fadb80c18d0 2024-03-04T08:24:37.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.766+0000 7fadbea00640 1 -- 172.21.15.113:0/595435894 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadb814faf0 con 0x7fadb80c18d0 2024-03-04T08:24:37.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadb5600640 1 -- 172.21.15.113:0/595435894 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fadac0f03f0 con 0x7fadb80c18d0 2024-03-04T08:24:37.768 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:37.769 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:37.770 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:37.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 -- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad98066f90 msgr2=0x7fad98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:37.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad98066f90 0x7fad98069450 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fadb806de10 tx=0x7fada4009040 comp rx=0 tx=0).stop 2024-03-04T08:24:37.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 -- 172.21.15.113:0/595435894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb80c18d0 msgr2=0x7fadb81f4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:37.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb80c18d0 0x7fadb81f4e90 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fadac0af040 tx=0x7fadac01ad10 comp rx=0 tx=0).stop 2024-03-04T08:24:37.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 -- 172.21.15.113:0/595435894 shutdown_connections 2024-03-04T08:24:37.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadb8151b40 0x7fadb813dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:37.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fadb8151760 0x7fadb81f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:37.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fad98066f90 0x7fad98069450 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:37.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 --2- 172.21.15.113:0/595435894 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadb80c18d0 0x7fadb81f4e90 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:37.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 -- 172.21.15.113:0/595435894 >> 172.21.15.113:0/595435894 conn(0x7fadb806e2a0 msgr2=0x7fadb8074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:37.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 -- 172.21.15.113:0/595435894 shutdown_connections 2024-03-04T08:24:37.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:37.770+0000 7fadbea00640 1 -- 172.21.15.113:0/595435894 wait complete. 2024-03-04T08:24:37.788 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:24:39.789 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:39.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.934+0000 7f094b600640 1 Processor -- start 2024-03-04T08:24:39.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.934+0000 7f094b600640 1 -- start start 2024-03-04T08:24:39.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f094b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 0x7f0944070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:39.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f094b600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09441591f0 con 0x7f0944152bd0 2024-03-04T08:24:39.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f094b600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09441594d0 con 0x7f09441527f0 2024-03-04T08:24:39.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f094b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0944071df0 con 0x7f09440c8200 2024-03-04T08:24:39.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f0949600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0944152bd0 0x7f09441587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43112/0 (socket says 172.21.15.113:43112) 2024-03-04T08:24:39.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f0949600640 1 -- 172.21.15.113:0/807318889 learned_addr learned my addr 172.21.15.113:0/807318889 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:39.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f093be00640 1 --2- 172.21.15.113:0/807318889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 0x7f0944070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:39.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f0943e00640 1 -- 172.21.15.113:0/807318889 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739894461 0 0) 0x7f09441591f0 con 0x7f0944152bd0 2024-03-04T08:24:39.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f0943e00640 1 -- 172.21.15.113:0/807318889 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0928003530 con 0x7f0944152bd0 2024-03-04T08:24:39.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f0943e00640 1 -- 172.21.15.113:0/807318889 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519915517 0 0) 0x7f09441594d0 con 0x7f09441527f0 2024-03-04T08:24:39.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f0943e00640 1 -- 172.21.15.113:0/807318889 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09441591f0 con 0x7f09441527f0 2024-03-04T08:24:39.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f0943e00640 1 -- 172.21.15.113:0/807318889 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2478563888 0 0) 0x7f0928003530 con 0x7f0944152bd0 2024-03-04T08:24:39.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f093be00640 1 -- 172.21.15.113:0/807318889 >> v1:172.21.15.136:6789/0 conn(0x7f09441527f0 legacy=0x7f0944072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:39.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f093be00640 1 -- 172.21.15.113:0/807318889 >> v1:172.21.15.113:6789/0 conn(0x7f0944152bd0 legacy=0x7f09441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:39.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.938+0000 7f093be00640 1 -- 172.21.15.113:0/807318889 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09440717a0 con 0x7f09440c8200 2024-03-04T08:24:39.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f093be00640 1 --2- 172.21.15.113:0/807318889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 0x7f0944070d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f092c0097f0 tx=0x7f092c01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35e4c4dea4df7e65 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:39.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f0943e00640 1 -- 172.21.15.113:0/807318889 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0b0070 con 0x7f09440c8200 2024-03-04T08:24:39.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f0943e00640 1 -- 172.21.15.113:0/807318889 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c0ab400 con 0x7f09440c8200 2024-03-04T08:24:39.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f0943e00640 1 -- 172.21.15.113:0/807318889 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c001a00 con 0x7f09440c8200 2024-03-04T08:24:39.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 -- 172.21.15.113:0/807318889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 msgr2=0x7f0944070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:39.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 --2- 172.21.15.113:0/807318889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 0x7f0944070d40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f092c0097f0 tx=0x7f092c01c400 comp rx=0 tx=0).stop 2024-03-04T08:24:39.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 -- 172.21.15.113:0/807318889 shutdown_connections 2024-03-04T08:24:39.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 --2- 172.21.15.113:0/807318889 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 0x7f0944070d40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:39.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 -- 172.21.15.113:0/807318889 >> 172.21.15.113:0/807318889 conn(0x7f094406e820 msgr2=0x7f094406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:39.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 -- 172.21.15.113:0/807318889 shutdown_connections 2024-03-04T08:24:39.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 -- 172.21.15.113:0/807318889 wait complete. 2024-03-04T08:24:39.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 Processor -- start 2024-03-04T08:24:39.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 -- start start 2024-03-04T08:24:39.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 0x7f09440b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:39.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.942+0000 7f094b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09441527f0 0x7f09441f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:39.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f094b600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0944152bd0 0x7f094414a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:39.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f094b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0944166500 con 0x7f09441527f0 2024-03-04T08:24:39.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f094b600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0944166380 con 0x7f0944152bd0 2024-03-04T08:24:39.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f094b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0944166680 con 0x7f09440c8200 2024-03-04T08:24:39.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f093be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09441527f0 0x7f09441f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:39.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f093be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09441527f0 0x7f09441f68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38640/0 (socket says 172.21.15.113:38640) 2024-03-04T08:24:39.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f093be00640 1 -- 172.21.15.113:0/1938966077 learned_addr learned my addr 172.21.15.113:0/1938966077 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:39.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f0948c00640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 0x7f09440b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:39.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f0949600640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0944152bd0 0x7f094414a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f093be00640 1 -- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 msgr2=0x7f09440b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f093be00640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 0x7f09440b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f093be00640 1 -- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0944152bd0 msgr2=0x7f094414a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f093be00640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0944152bd0 0x7f094414a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f093be00640 1 -- 172.21.15.113:0/1938966077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094415c200 con 0x7f09441527f0 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f0948c00640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 0x7f09440b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f0949600640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0944152bd0 0x7f094414a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f093be00640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09441527f0 0x7f09441f68d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f092c0ae040 tx=0x7f092c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f0941600640 1 -- 172.21.15.113:0/1938966077 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0b0040 con 0x7f09441527f0 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f0941600640 1 -- 172.21.15.113:0/1938966077 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c003620 con 0x7f09441527f0 2024-03-04T08:24:39.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f0941600640 1 -- 172.21.15.113:0/1938966077 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0039d0 con 0x7f09441527f0 2024-03-04T08:24:39.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f094b600640 1 -- 172.21.15.113:0/1938966077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094414ded0 con 0x7f09441527f0 2024-03-04T08:24:39.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f094b600640 1 -- 172.21.15.113:0/1938966077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094415ba00 con 0x7f09441527f0 2024-03-04T08:24:39.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f094b600640 1 -- 172.21.15.113:0/1938966077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0944075c00 con 0x7f09441527f0 2024-03-04T08:24:39.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.946+0000 7f0941600640 1 -- 172.21.15.113:0/1938966077 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f092c0bb030 con 0x7f09441527f0 2024-03-04T08:24:39.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.950+0000 7f0941600640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0920066f60 0x7f0920069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:39.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.950+0000 7f0941600640 1 -- 172.21.15.113:0/1938966077 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f092c11ed30 con 0x7f09441527f0 2024-03-04T08:24:39.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.950+0000 7f0948c00640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0920066f60 0x7f0920069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:39.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.950+0000 7f0948c00640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0920066f60 0x7f0920069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f09340011f0 tx=0x7f0934009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:39.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:39.954+0000 7f0941600640 1 -- 172.21.15.113:0/1938966077 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f092c0c2060 con 0x7f09441527f0 2024-03-04T08:24:40.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 -- 172.21.15.113:0/1938966077 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09440c79d0 con 0x7f09441527f0 2024-03-04T08:24:40.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f0941600640 1 -- 172.21.15.113:0/1938966077 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f092c015ff0 con 0x7f09441527f0 2024-03-04T08:24:40.140 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:40.140 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:40.140 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:40.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 -- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0920066f60 msgr2=0x7f0920069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:40.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0920066f60 0x7f0920069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f09340011f0 tx=0x7f0934009040 comp rx=0 tx=0).stop 2024-03-04T08:24:40.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 -- 172.21.15.113:0/1938966077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09441527f0 msgr2=0x7f09441f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:40.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09441527f0 0x7f09441f68d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f092c0ae040 tx=0x7f092c0191a0 comp rx=0 tx=0).stop 2024-03-04T08:24:40.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 -- 172.21.15.113:0/1938966077 shutdown_connections 2024-03-04T08:24:40.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0944152bd0 0x7f094414a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:40.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0920066f60 0x7f0920069420 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:40.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09441527f0 0x7f09441f68d0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:40.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 --2- 172.21.15.113:0/1938966077 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f09440c8200 0x7f09440b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:40.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 -- 172.21.15.113:0/1938966077 >> 172.21.15.113:0/1938966077 conn(0x7f094406e820 msgr2=0x7f09441563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:40.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 -- 172.21.15.113:0/1938966077 shutdown_connections 2024-03-04T08:24:40.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:40.142+0000 7f094b600640 1 -- 172.21.15.113:0/1938966077 wait complete. 2024-03-04T08:24:40.162 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:24:40.163 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:24:42.164 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:42.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f438e000640 1 Processor -- start 2024-03-04T08:24:42.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f438e000640 1 -- start start 2024-03-04T08:24:42.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f438e000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f43880c9f60 0x7f438815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:42.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f438e000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43880c5c40 con 0x7f4388157870 2024-03-04T08:24:42.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f438e000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4388157cb0 con 0x7f4388151b30 2024-03-04T08:24:42.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f438e000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43880c5f00 con 0x7f43880c9f60 2024-03-04T08:24:42.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4387e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4388157870 0x7f4388162110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43126/0 (socket says 172.21.15.113:43126) 2024-03-04T08:24:42.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4387e00640 1 -- 172.21.15.113:0/2663998689 learned_addr learned my addr 172.21.15.113:0/2663998689 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:42.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386a00640 1 --2- 172.21.15.113:0/2663998689 >> v2:172.21.15.136:3300/0 conn(0x7f43880c9f60 0x7f438815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:42.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472664239 0 0) 0x7f43880c5c40 con 0x7f4388157870 2024-03-04T08:24:42.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f436c003530 con 0x7f4388157870 2024-03-04T08:24:42.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1763987152 0 0) 0x7f436c003530 con 0x7f4388157870 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 >> v2:172.21.15.136:3300/0 conn(0x7f43880c9f60 msgr2=0x7f438815b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 --2- 172.21.15.113:0/2663998689 >> v2:172.21.15.136:3300/0 conn(0x7f43880c9f60 0x7f438815b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 >> v1:172.21.15.136:6789/0 conn(0x7f4388151b30 legacy=0x7f43880724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43880c61e0 con 0x7f4388157870 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f4380001a00 con 0x7f4388157870 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 >> v1:172.21.15.113:6789/0 conn(0x7f4388157870 legacy=0x7f4388162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f436c003a10 0x7f436c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f436c004810 0x7f436c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f436c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43880c5f00 con 0x7f436c0051d0 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c00c0b0 con 0x7f436c004810 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c00e4c0 con 0x7f436c003a10 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4387e00640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f436c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:42.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4387400640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f436c003a10 0x7f436c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:42.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386a00640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f436c004810 0x7f436c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:42.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4387e00640 1 -- 172.21.15.113:0/2663998689 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f436c003a10 msgr2=0x7f436c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:42.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4387e00640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f436c003a10 0x7f436c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4387e00640 1 -- 172.21.15.113:0/2663998689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f436c004810 msgr2=0x7f436c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:42.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4387e00640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f436c004810 0x7f436c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4387e00640 1 -- 172.21.15.113:0/2663998689 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c003530 con 0x7f436c0051d0 2024-03-04T08:24:42.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.314+0000 7f4386a00640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f436c004810 0x7f436c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:24:42.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f4387e00640 1 -- 172.21.15.113:0/2663998689 reap_dead start 2024-03-04T08:24:42.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f4387e00640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f436c00b9d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f438000a790 tx=0x7f438001d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14a5366682ca43d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:42.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438000a3b0 con 0x7f436c0051d0 2024-03-04T08:24:42.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43800ad670 con 0x7f436c0051d0 2024-03-04T08:24:42.317 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f4386000640 1 -- 172.21.15.113:0/2663998689 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43800ada60 con 0x7f436c0051d0 2024-03-04T08:24:42.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f438e000640 1 -- 172.21.15.113:0/2663998689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 msgr2=0x7f436c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:42.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f438e000640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f436c00b9d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f438000a790 tx=0x7f438001d2c0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f438e000640 1 -- 172.21.15.113:0/2663998689 shutdown_connections 2024-03-04T08:24:42.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f438e000640 1 --2- 172.21.15.113:0/2663998689 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f436c00b9d0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f438e000640 1 -- 172.21.15.113:0/2663998689 >> 172.21.15.113:0/2663998689 conn(0x7f438806ef60 msgr2=0x7f438806f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:42.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f438e000640 1 -- 172.21.15.113:0/2663998689 shutdown_connections 2024-03-04T08:24:42.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.318+0000 7f438e000640 1 -- 172.21.15.113:0/2663998689 wait complete. 2024-03-04T08:24:42.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 Processor -- start 2024-03-04T08:24:42.319 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 -- start start 2024-03-04T08:24:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f43881e1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388159d80 0x7f43881f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4388159830 0x7f438814e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:42.321 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438815b3a0 con 0x7f436c0051d0 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438815b0c0 con 0x7f4388159d80 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438815adb0 con 0x7f4388159830 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4387400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f43881e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4387400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f43881e1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38662/0 (socket says 172.21.15.113:38662) 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4387400640 1 -- 172.21.15.113:0/2746201868 learned_addr learned my addr 172.21.15.113:0/2746201868 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4386a00640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388159d80 0x7f43881f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4387e00640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4388159830 0x7f438814e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4387400640 1 -- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4388159830 msgr2=0x7f438814e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4387400640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4388159830 0x7f438814e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4387400640 1 -- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388159d80 msgr2=0x7f43881f0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4387400640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388159d80 0x7f43881f0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4387400640 1 -- 172.21.15.113:0/2746201868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438815a4b0 con 0x7f436c0051d0 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f4387400640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f43881e1340 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f437800e500 tx=0x7f437800e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f437f400640 1 -- 172.21.15.113:0/2746201868 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43780052e0 con 0x7f436c0051d0 2024-03-04T08:24:42.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f437f400640 1 -- 172.21.15.113:0/2746201868 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4378018070 con 0x7f436c0051d0 2024-03-04T08:24:42.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f437f400640 1 -- 172.21.15.113:0/2746201868 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378003820 con 0x7f436c0051d0 2024-03-04T08:24:42.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 -- 172.21.15.113:0/2746201868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438815a770 con 0x7f436c0051d0 2024-03-04T08:24:42.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 -- 172.21.15.113:0/2746201868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4388158d50 con 0x7f436c0051d0 2024-03-04T08:24:42.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.322+0000 7f438e000640 1 -- 172.21.15.113:0/2746201868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4388075c00 con 0x7f436c0051d0 2024-03-04T08:24:42.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.326+0000 7f437f400640 1 -- 172.21.15.113:0/2746201868 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43780039c0 con 0x7f436c0051d0 2024-03-04T08:24:42.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.326+0000 7f437f400640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4368066f60 0x7f4368069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:42.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.326+0000 7f437f400640 1 -- 172.21.15.113:0/2746201868 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f43780857d0 con 0x7f436c0051d0 2024-03-04T08:24:42.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.326+0000 7f4386a00640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4368066f60 0x7f4368069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:42.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.326+0000 7f4386a00640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4368066f60 0x7f4368069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f438814b710 tx=0x7f4374008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:42.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.330+0000 7f437f400640 1 -- 172.21.15.113:0/2746201868 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4378055cb0 con 0x7f436c0051d0 2024-03-04T08:24:42.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 -- 172.21.15.113:0/2746201868 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43880cd640 con 0x7f436c0051d0 2024-03-04T08:24:42.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f437f400640 1 -- 172.21.15.113:0/2746201868 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f4378005480 con 0x7f436c0051d0 2024-03-04T08:24:42.516 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:42.516 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:42.517 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 -- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4368066f60 msgr2=0x7f4368069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4368066f60 0x7f4368069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f438814b710 tx=0x7f4374008040 comp rx=0 tx=0).stop 2024-03-04T08:24:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 -- 172.21.15.113:0/2746201868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 msgr2=0x7f43881e1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f43881e1340 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f437800e500 tx=0x7f437800e9c0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 -- 172.21.15.113:0/2746201868 shutdown_connections 2024-03-04T08:24:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4388159830 0x7f438814e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4388159d80 0x7f43881f0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4368066f60 0x7f4368069420 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 --2- 172.21.15.113:0/2746201868 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f436c0051d0 0x7f43881e1340 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 -- 172.21.15.113:0/2746201868 >> 172.21.15.113:0/2746201868 conn(0x7f438806ef60 msgr2=0x7f43880713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 -- 172.21.15.113:0/2746201868 shutdown_connections 2024-03-04T08:24:42.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:42.518+0000 7f438e000640 1 -- 172.21.15.113:0/2746201868 wait complete. 2024-03-04T08:24:42.537 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:24:44.538 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:44.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.686+0000 7fca34e00640 1 Processor -- start 2024-03-04T08:24:44.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca34e00640 1 -- start start 2024-03-04T08:24:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca34e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca30157df0 0x7fca300c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca34e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fca301581d0 0x7fca3006c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca34e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3006de30 con 0x7fca3006ed80 2024-03-04T08:24:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca34e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3006dc00 con 0x7fca301581d0 2024-03-04T08:24:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca34e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3006e110 con 0x7fca30157df0 2024-03-04T08:24:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fca3006ed80 0x7fca300c9be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43140/0 (socket says 172.21.15.113:43140) 2024-03-04T08:24:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2fe00640 1 -- 172.21.15.113:0/1229110091 learned_addr learned my addr 172.21.15.113:0/1229110091 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2f400640 1 --2- 172.21.15.113:0/1229110091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca30157df0 0x7fca300c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:44.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2ea00640 1 --2- 172.21.15.113:0/1229110091 >> v2:172.21.15.136:3300/0 conn(0x7fca301581d0 0x7fca3006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:44.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2e000640 1 -- 172.21.15.113:0/1229110091 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030240127 0 0) 0x7fca3006de30 con 0x7fca3006ed80 2024-03-04T08:24:44.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2e000640 1 -- 172.21.15.113:0/1229110091 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca14003530 con 0x7fca3006ed80 2024-03-04T08:24:44.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2ea00640 1 -- 172.21.15.113:0/1229110091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca30157df0 msgr2=0x7fca300c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:44.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2ea00640 1 --2- 172.21.15.113:0/1229110091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca30157df0 0x7fca300c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:44.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2ea00640 1 -- 172.21.15.113:0/1229110091 >> v1:172.21.15.113:6789/0 conn(0x7fca3006ed80 legacy=0x7fca300c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:44.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2e000640 1 -- 172.21.15.113:0/1229110091 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3722223674 0 0) 0x7fca14003530 con 0x7fca3006ed80 2024-03-04T08:24:44.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2ea00640 1 -- 172.21.15.113:0/1229110091 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300ce0c0 con 0x7fca301581d0 2024-03-04T08:24:44.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.690+0000 7fca2f400640 1 --2- 172.21.15.113:0/1229110091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca30157df0 0x7fca300c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:24:44.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca2ea00640 1 --2- 172.21.15.113:0/1229110091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca301581d0 0x7fca3006c170 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2551bbe11cc51b9e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:44.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca2e000640 1 -- 172.21.15.113:0/1229110091 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0b5070 con 0x7fca301581d0 2024-03-04T08:24:44.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca2e000640 1 -- 172.21.15.113:0/1229110091 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c0a85f0 con 0x7fca301581d0 2024-03-04T08:24:44.692 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca2e000640 1 -- 172.21.15.113:0/1229110091 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28001a00 con 0x7fca301581d0 2024-03-04T08:24:44.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca34e00640 1 -- 172.21.15.113:0/1229110091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca301581d0 msgr2=0x7fca3006c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:44.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca34e00640 1 --2- 172.21.15.113:0/1229110091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca301581d0 0x7fca3006c170 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01ebb0 comp rx=0 tx=0).stop 2024-03-04T08:24:44.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca34e00640 1 -- 172.21.15.113:0/1229110091 shutdown_connections 2024-03-04T08:24:44.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca34e00640 1 --2- 172.21.15.113:0/1229110091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca301581d0 0x7fca3006c170 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:44.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca34e00640 1 --2- 172.21.15.113:0/1229110091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca30157df0 0x7fca300c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:44.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca34e00640 1 -- 172.21.15.113:0/1229110091 >> 172.21.15.113:0/1229110091 conn(0x7fca30072f40 msgr2=0x7fca300706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:44.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca34e00640 1 -- 172.21.15.113:0/1229110091 shutdown_connections 2024-03-04T08:24:44.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca34e00640 1 -- 172.21.15.113:0/1229110091 wait complete. 2024-03-04T08:24:44.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.694+0000 7fca34e00640 1 Processor -- start 2024-03-04T08:24:44.695 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca34e00640 1 -- start start 2024-03-04T08:24:44.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca34e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca3006ed80 0x7fca300c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:44.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca34e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30157df0 0x7fca301544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:44.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca34e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca301581d0 0x7fca301f0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:44.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca34e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3006fd90 con 0x7fca30157df0 2024-03-04T08:24:44.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca34e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3006fc10 con 0x7fca3006ed80 2024-03-04T08:24:44.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca34e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca300ca2f0 con 0x7fca301581d0 2024-03-04T08:24:44.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30157df0 0x7fca301544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:44.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2ea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30157df0 0x7fca301544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38666/0 (socket says 172.21.15.113:38666) 2024-03-04T08:24:44.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2ea00640 1 -- 172.21.15.113:0/121130201 learned_addr learned my addr 172.21.15.113:0/121130201 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:44.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2f400640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca3006ed80 0x7fca300c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2fe00640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca301581d0 0x7fca301f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2ea00640 1 -- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca301581d0 msgr2=0x7fca301f0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2ea00640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca301581d0 0x7fca301f0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2ea00640 1 -- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca3006ed80 msgr2=0x7fca300c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2ea00640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca3006ed80 0x7fca300c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2ea00640 1 -- 172.21.15.113:0/121130201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca301f3a50 con 0x7fca30157df0 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca2ea00640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30157df0 0x7fca301544f0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fca1c01d260 tx=0x7fca1c01b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca27400640 1 -- 172.21.15.113:0/121130201 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0c1070 con 0x7fca30157df0 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca27400640 1 -- 172.21.15.113:0/121130201 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c0b0640 con 0x7fca30157df0 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca27400640 1 -- 172.21.15.113:0/121130201 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0b5040 con 0x7fca30157df0 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca34e00640 1 -- 172.21.15.113:0/121130201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca301e7470 con 0x7fca30157df0 2024-03-04T08:24:44.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca34e00640 1 -- 172.21.15.113:0/121130201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca301f3ff0 con 0x7fca30157df0 2024-03-04T08:24:44.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.698+0000 7fca34e00640 1 -- 172.21.15.113:0/121130201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca30075a80 con 0x7fca30157df0 2024-03-04T08:24:44.699 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.702+0000 7fca27400640 1 -- 172.21.15.113:0/121130201 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca1c0b09f0 con 0x7fca30157df0 2024-03-04T08:24:44.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.702+0000 7fca27400640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:44.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.702+0000 7fca27400640 1 -- 172.21.15.113:0/121130201 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fca1c122890 con 0x7fca30157df0 2024-03-04T08:24:44.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.702+0000 7fca2f400640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:44.700 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.702+0000 7fca2f400640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca10066f60 0x7fca10069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fca20009580 tx=0x7fca20009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:44.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.706+0000 7fca27400640 1 -- 172.21.15.113:0/121130201 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fca1c018710 con 0x7fca30157df0 2024-03-04T08:24:44.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.890+0000 7fca34e00640 1 -- 172.21.15.113:0/121130201 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca300c97b0 con 0x7fca30157df0 2024-03-04T08:24:44.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.890+0000 7fca27400640 1 -- 172.21.15.113:0/121130201 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fca1c0f2680 con 0x7fca30157df0 2024-03-04T08:24:44.891 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:44.892 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:44.893 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:44.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 -- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca10066f60 msgr2=0x7fca10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca10066f60 0x7fca10069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fca20009580 tx=0x7fca20009190 comp rx=0 tx=0).stop 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 -- 172.21.15.113:0/121130201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30157df0 msgr2=0x7fca301544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30157df0 0x7fca301544f0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fca1c01d260 tx=0x7fca1c01b950 comp rx=0 tx=0).stop 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 -- 172.21.15.113:0/121130201 shutdown_connections 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fca301581d0 0x7fca301f0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca30157df0 0x7fca301544f0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 --2- 172.21.15.113:0/121130201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fca3006ed80 0x7fca300c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 -- 172.21.15.113:0/121130201 >> 172.21.15.113:0/121130201 conn(0x7fca30072f40 msgr2=0x7fca3015a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 -- 172.21.15.113:0/121130201 shutdown_connections 2024-03-04T08:24:44.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:44.894+0000 7fca34e00640 1 -- 172.21.15.113:0/121130201 wait complete. 2024-03-04T08:24:44.911 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:24:45.376 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:24:45.380 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:24:46.912 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.058+0000 7f9d2d400640 1 Processor -- start 2024-03-04T08:24:47.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.058+0000 7f9d2d400640 1 -- start start 2024-03-04T08:24:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d2d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9d280c8200 0x7f9d28070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d2d400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9d28152bd0 0x7f9d281587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d2d400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d281591f0 con 0x7f9d281527f0 2024-03-04T08:24:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d2d400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d28071df0 con 0x7f9d280c8200 2024-03-04T08:24:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d2d400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d281597b0 con 0x7f9d28152bd0 2024-03-04T08:24:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d1de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9d280c8200 0x7f9d28070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d1de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9d280c8200 0x7f9d28070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38668/0 (socket says 172.21.15.113:38668) 2024-03-04T08:24:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d1de00640 1 -- 172.21.15.113:0/3752912878 learned_addr learned my addr 172.21.15.113:0/3752912878 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d27200640 1 --2- 172.21.15.113:0/3752912878 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9d28152bd0 0x7f9d281587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d1de00640 1 -- 172.21.15.113:0/3752912878 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9d28152bd0 msgr2=0x7f9d281587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d1de00640 1 --2- 172.21.15.113:0/3752912878 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9d28152bd0 0x7f9d281587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d1de00640 1 -- 172.21.15.113:0/3752912878 >> v1:172.21.15.136:6789/0 conn(0x7f9d281527f0 legacy=0x7f9d28072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.062+0000 7f9d1de00640 1 -- 172.21.15.113:0/3752912878 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d281594d0 con 0x7f9d280c8200 2024-03-04T08:24:47.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d1de00640 1 --2- 172.21.15.113:0/3752912878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d280c8200 0x7f9d28070d40 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009870 tx=0x7f9d1001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7a35e270e840d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:47.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d25e00640 1 -- 172.21.15.113:0/3752912878 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d100a7070 con 0x7f9d280c8200 2024-03-04T08:24:47.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d25e00640 1 -- 172.21.15.113:0/3752912878 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d100b0070 con 0x7f9d280c8200 2024-03-04T08:24:47.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d25e00640 1 -- 172.21.15.113:0/3752912878 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d100ab890 con 0x7f9d280c8200 2024-03-04T08:24:47.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 -- 172.21.15.113:0/3752912878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d280c8200 msgr2=0x7f9d28070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:47.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 --2- 172.21.15.113:0/3752912878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d280c8200 0x7f9d28070d40 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009870 tx=0x7f9d1001c560 comp rx=0 tx=0).stop 2024-03-04T08:24:47.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 -- 172.21.15.113:0/3752912878 shutdown_connections 2024-03-04T08:24:47.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 --2- 172.21.15.113:0/3752912878 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9d28152bd0 0x7f9d281587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:47.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 --2- 172.21.15.113:0/3752912878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d280c8200 0x7f9d28070d40 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:47.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 -- 172.21.15.113:0/3752912878 >> 172.21.15.113:0/3752912878 conn(0x7f9d2806e820 msgr2=0x7f9d2806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:47.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 -- 172.21.15.113:0/3752912878 shutdown_connections 2024-03-04T08:24:47.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 -- 172.21.15.113:0/3752912878 wait complete. 2024-03-04T08:24:47.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 Processor -- start 2024-03-04T08:24:47.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 -- start start 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d280c8200 0x7f9d281612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9d281527f0 0x7f9d280b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d28152bd0 0x7f9d280c1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d281654f0 con 0x7f9d28152bd0 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d28165370 con 0x7f9d280c8200 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.066+0000 7f9d2d400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d28165670 con 0x7f9d281527f0 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d27200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d28152bd0 0x7f9d280c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d27200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d28152bd0 0x7f9d280c1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:38680/0 (socket says 172.21.15.113:38680) 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d27200640 1 -- 172.21.15.113:0/2280288704 learned_addr learned my addr 172.21.15.113:0/2280288704 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d1de00640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9d281527f0 0x7f9d280b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:47.068 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d26800640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d280c8200 0x7f9d281612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d27200640 1 -- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9d281527f0 msgr2=0x7f9d280b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d27200640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9d281527f0 0x7f9d280b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d27200640 1 -- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d280c8200 msgr2=0x7f9d281612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d27200640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d280c8200 0x7f9d281612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d27200640 1 -- 172.21.15.113:0/2280288704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d280cb1c0 con 0x7f9d28152bd0 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d27200640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d28152bd0 0x7f9d280c1760 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9d20007e60 tx=0x7f9d2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d1f400640 1 -- 172.21.15.113:0/2280288704 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20014420 con 0x7f9d28152bd0 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d1f400640 1 -- 172.21.15.113:0/2280288704 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d20019070 con 0x7f9d28152bd0 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d1f400640 1 -- 172.21.15.113:0/2280288704 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20014820 con 0x7f9d28152bd0 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d2d400640 1 -- 172.21.15.113:0/2280288704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d28161970 con 0x7f9d28152bd0 2024-03-04T08:24:47.069 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d2d400640 1 -- 172.21.15.113:0/2280288704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d28161f30 con 0x7f9d28152bd0 2024-03-04T08:24:47.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d2d400640 1 -- 172.21.15.113:0/2280288704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d28075c00 con 0x7f9d28152bd0 2024-03-04T08:24:47.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.070+0000 7f9d1f400640 1 -- 172.21.15.113:0/2280288704 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d200149c0 con 0x7f9d28152bd0 2024-03-04T08:24:47.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.074+0000 7f9d1f400640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9d04066f60 0x7f9d04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:47.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.074+0000 7f9d1f400640 1 -- 172.21.15.113:0/2280288704 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f9d20086330 con 0x7f9d28152bd0 2024-03-04T08:24:47.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.074+0000 7f9d26800640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9d04066f60 0x7f9d04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:47.072 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.074+0000 7f9d26800640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9d04066f60 0x7f9d04069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9d18001650 tx=0x7f9d18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:47.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.078+0000 7f9d1f400640 1 -- 172.21.15.113:0/2280288704 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9d20056800 con 0x7f9d28152bd0 2024-03-04T08:24:47.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 -- 172.21.15.113:0/2280288704 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d280c79d0 con 0x7f9d28152bd0 2024-03-04T08:24:47.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d1f400640 1 -- 172.21.15.113:0/2280288704 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f9d20056100 con 0x7f9d28152bd0 2024-03-04T08:24:47.268 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:47.268 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:47.268 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 -- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9d04066f60 msgr2=0x7f9d04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9d04066f60 0x7f9d04069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f9d18001650 tx=0x7f9d18009040 comp rx=0 tx=0).stop 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 -- 172.21.15.113:0/2280288704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d28152bd0 msgr2=0x7f9d280c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d28152bd0 0x7f9d280c1760 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9d20007e60 tx=0x7f9d2000e4b0 comp rx=0 tx=0).stop 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 -- 172.21.15.113:0/2280288704 shutdown_connections 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9d04066f60 0x7f9d04069420 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d28152bd0 0x7f9d280c1760 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9d281527f0 0x7f9d280b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 --2- 172.21.15.113:0/2280288704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d280c8200 0x7f9d281612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 -- 172.21.15.113:0/2280288704 >> 172.21.15.113:0/2280288704 conn(0x7f9d2806e820 msgr2=0x7f9d28158310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 -- 172.21.15.113:0/2280288704 shutdown_connections 2024-03-04T08:24:47.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:47.270+0000 7f9d2d400640 1 -- 172.21.15.113:0/2280288704 wait complete. 2024-03-04T08:24:47.289 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:24:49.291 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:49.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.438+0000 7f196d200640 1 Processor -- start 2024-03-04T08:24:49.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.438+0000 7f196d200640 1 -- start start 2024-03-04T08:24:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.438+0000 7f196d200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f19680ccb40 0x7f19680c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.438+0000 7f196d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1968154f10 0x7f196815e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.438+0000 7f196d200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196815aed0 con 0x7f196815aa40 2024-03-04T08:24:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f196d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196815b170 con 0x7f1968154f10 2024-03-04T08:24:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f196d200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196815b410 con 0x7f19680ccb40 2024-03-04T08:24:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1966800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1968154f10 0x7f196815e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1966800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1968154f10 0x7f196815e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35246/0 (socket says 172.21.15.113:35246) 2024-03-04T08:24:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f195d400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f196815aa40 0x7f1968154800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34786/0 (socket says 172.21.15.113:34786) 2024-03-04T08:24:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1966800640 1 -- 172.21.15.113:0/1467398688 learned_addr learned my addr 172.21.15.113:0/1467398688 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:49.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1965e00640 1 --2- 172.21.15.113:0/1467398688 >> v2:172.21.15.136:3300/0 conn(0x7f19680ccb40 0x7f19680c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:49.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1966800640 1 -- 172.21.15.113:0/1467398688 >> v2:172.21.15.136:3300/0 conn(0x7f19680ccb40 msgr2=0x7f19680c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:49.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1966800640 1 --2- 172.21.15.113:0/1467398688 >> v2:172.21.15.136:3300/0 conn(0x7f19680ccb40 0x7f19680c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:49.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1966800640 1 -- 172.21.15.113:0/1467398688 >> v1:172.21.15.113:6789/0 conn(0x7f196815aa40 legacy=0x7f1968154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:49.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1966800640 1 -- 172.21.15.113:0/1467398688 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196815b9b0 con 0x7f1968154f10 2024-03-04T08:24:49.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1966800640 1 --2- 172.21.15.113:0/1467398688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1968154f10 0x7f196815e5f0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f1960009e50 tx=0x7f196001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71a94fd27efe4da server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:49.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1965400640 1 -- 172.21.15.113:0/1467398688 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19600b4070 con 0x7f1968154f10 2024-03-04T08:24:49.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1965400640 1 -- 172.21.15.113:0/1467398688 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19600af400 con 0x7f1968154f10 2024-03-04T08:24:49.442 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.442+0000 7f1965400640 1 -- 172.21.15.113:0/1467398688 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19600af7d0 con 0x7f1968154f10 2024-03-04T08:24:49.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 -- 172.21.15.113:0/1467398688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1968154f10 msgr2=0x7f196815e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:49.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 --2- 172.21.15.113:0/1467398688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1968154f10 0x7f196815e5f0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f1960009e50 tx=0x7f196001f1e0 comp rx=0 tx=0).stop 2024-03-04T08:24:49.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 -- 172.21.15.113:0/1467398688 shutdown_connections 2024-03-04T08:24:49.443 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 --2- 172.21.15.113:0/1467398688 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1968154f10 0x7f196815e5f0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:49.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 --2- 172.21.15.113:0/1467398688 >> v2:172.21.15.136:3300/0 conn(0x7f19680ccb40 0x7f19680c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:49.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 -- 172.21.15.113:0/1467398688 >> 172.21.15.113:0/1467398688 conn(0x7f196806b060 msgr2=0x7f196806b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:49.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 -- 172.21.15.113:0/1467398688 shutdown_connections 2024-03-04T08:24:49.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 -- 172.21.15.113:0/1467398688 wait complete. 2024-03-04T08:24:49.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 Processor -- start 2024-03-04T08:24:49.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 -- start start 2024-03-04T08:24:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19680ccb40 0x7f1968156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1968154f10 0x7f1968155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19681f15a0 0x7f19681e36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680c78b0 con 0x7f19681f15a0 2024-03-04T08:24:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196815e0d0 con 0x7f1968154f10 2024-03-04T08:24:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f196d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19680ccf20 con 0x7f19680ccb40 2024-03-04T08:24:49.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f1966800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19681f15a0 0x7f19681e36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f1966800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19681f15a0 0x7f19681e36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35252/0 (socket says 172.21.15.113:35252) 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f1966800640 1 -- 172.21.15.113:0/312613032 learned_addr learned my addr 172.21.15.113:0/312613032 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f1965e00640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19680ccb40 0x7f1968156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f195d400640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1968154f10 0x7f1968155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f1966800640 1 -- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19680ccb40 msgr2=0x7f1968156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f1966800640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19680ccb40 0x7f1968156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f1966800640 1 -- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1968154f10 msgr2=0x7f1968155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f1966800640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1968154f10 0x7f1968155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.446+0000 7f1966800640 1 -- 172.21.15.113:0/312613032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196815a310 con 0x7f19681f15a0 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f1966800640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19681f15a0 0x7f19681e36e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f196001bdf0 tx=0x7f196001bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f195ea00640 1 -- 172.21.15.113:0/312613032 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19600b4040 con 0x7f19681f15a0 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f195ea00640 1 -- 172.21.15.113:0/312613032 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19600c0070 con 0x7f19681f15a0 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f196d200640 1 -- 172.21.15.113:0/312613032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1968156e70 con 0x7f19681f15a0 2024-03-04T08:24:49.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f195ea00640 1 -- 172.21.15.113:0/312613032 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960009510 con 0x7f19681f15a0 2024-03-04T08:24:49.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f196d200640 1 -- 172.21.15.113:0/312613032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19681f0ac0 con 0x7f19681f15a0 2024-03-04T08:24:49.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f196d200640 1 -- 172.21.15.113:0/312613032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19681f08b0 con 0x7f19681f15a0 2024-03-04T08:24:49.449 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f195ea00640 1 -- 172.21.15.113:0/312613032 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19600c5460 con 0x7f19681f15a0 2024-03-04T08:24:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f195ea00640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1944066f90 0x7f1944069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f195ea00640 1 -- 172.21.15.113:0/312613032 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f1960123d40 con 0x7f19681f15a0 2024-03-04T08:24:49.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.450+0000 7f1965e00640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1944066f90 0x7f1944069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:49.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.454+0000 7f1965e00640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1944066f90 0x7f1944069450 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1958009af0 tx=0x7f1958008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:49.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.454+0000 7f195ea00640 1 -- 172.21.15.113:0/312613032 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1960018d40 con 0x7f19681f15a0 2024-03-04T08:24:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.642+0000 7f196d200640 1 -- 172.21.15.113:0/312613032 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1968075c00 con 0x7f19681f15a0 2024-03-04T08:24:49.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.642+0000 7f195ea00640 1 -- 172.21.15.113:0/312613032 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f19600f42d0 con 0x7f19681f15a0 2024-03-04T08:24:49.642 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:49.643 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:49.643 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:49.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 -- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1944066f90 msgr2=0x7f1944069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:49.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1944066f90 0x7f1944069450 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1958009af0 tx=0x7f1958008040 comp rx=0 tx=0).stop 2024-03-04T08:24:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 -- 172.21.15.113:0/312613032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19681f15a0 msgr2=0x7f19681e36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19681f15a0 0x7f19681e36e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f196001bdf0 tx=0x7f196001bf80 comp rx=0 tx=0).stop 2024-03-04T08:24:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 -- 172.21.15.113:0/312613032 shutdown_connections 2024-03-04T08:24:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1944066f90 0x7f1944069450 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19681f15a0 0x7f19681e36e0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1968154f10 0x7f1968155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 --2- 172.21.15.113:0/312613032 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19680ccb40 0x7f1968156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 -- 172.21.15.113:0/312613032 >> 172.21.15.113:0/312613032 conn(0x7f196806b060 msgr2=0x7f19680ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 -- 172.21.15.113:0/312613032 shutdown_connections 2024-03-04T08:24:49.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:49.646+0000 7f196d200640 1 -- 172.21.15.113:0/312613032 wait complete. 2024-03-04T08:24:49.663 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:24:51.664 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:51.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.810+0000 7f8593c00640 1 Processor -- start 2024-03-04T08:24:51.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.810+0000 7f8593c00640 1 -- start start 2024-03-04T08:24:51.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8593c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f858c0ce570 0x7f858c154c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:51.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8593c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858c06c860 con 0x7f858c070120 2024-03-04T08:24:51.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8593c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858c06f8d0 con 0x7f858c0c8d80 2024-03-04T08:24:51.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8593c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858c06f310 con 0x7f858c0ce570 2024-03-04T08:24:51.811 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8591c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f858c070120 0x7f858c1656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34796/0 (socket says 172.21.15.113:34796) 2024-03-04T08:24:51.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8591c00640 1 -- 172.21.15.113:0/1450862861 learned_addr learned my addr 172.21.15.113:0/1450862861 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:51.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8591200640 1 --2- 172.21.15.113:0/1450862861 >> v2:172.21.15.136:3300/0 conn(0x7f858c0ce570 0x7f858c154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:51.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078047962 0 0) 0x7f858c06c860 con 0x7f858c070120 2024-03-04T08:24:51.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8570003530 con 0x7f858c070120 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2419339494 0 0) 0x7f8570003530 con 0x7f858c070120 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 >> v2:172.21.15.136:3300/0 conn(0x7f858c0ce570 msgr2=0x7f858c154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 --2- 172.21.15.113:0/1450862861 >> v2:172.21.15.136:3300/0 conn(0x7f858c0ce570 0x7f858c154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 >> v1:172.21.15.136:6789/0 conn(0x7f858c0c8d80 legacy=0x7f858c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c06f5f0 con 0x7f858c070120 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f8584001680 con 0x7f858c070120 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8591200640 1 --2- 172.21.15.113:0/1450862861 >> v2:172.21.15.136:3300/0 conn(0x7f858c0ce570 0x7f858c154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 >> v1:172.21.15.113:6789/0 conn(0x7f858c070120 legacy=0x7f858c1656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85700039c0 0x7f8570003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8570004810 0x7f8570004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85700051d0 0x7f857000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c06f310 con 0x7f8570004810 2024-03-04T08:24:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857000c080 con 0x7f85700039c0 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857000c280 con 0x7f85700051d0 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858be00640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8570004810 0x7f8570004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8591200640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85700039c0 0x7f8570003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8591c00640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85700051d0 0x7f857000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858be00640 1 -- 172.21.15.113:0/1450862861 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85700051d0 msgr2=0x7f857000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858be00640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85700051d0 0x7f857000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858be00640 1 -- 172.21.15.113:0/1450862861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85700039c0 msgr2=0x7f8570003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858be00640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85700039c0 0x7f8570003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f858be00640 1 -- 172.21.15.113:0/1450862861 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8570003530 con 0x7f8570004810 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8591c00640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85700051d0 0x7f857000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:24:51.815 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.814+0000 7f8591c00640 1 -- 172.21.15.113:0/1450862861 reap_dead start 2024-03-04T08:24:51.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f858be00640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8570004810 0x7f8570004c90 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8578001780 tx=0x7f857801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb14701a0975e66 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:51.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85780013a0 con 0x7f8570004810 2024-03-04T08:24:51.816 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85780052f0 con 0x7f8570004810 2024-03-04T08:24:51.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f858b400640 1 -- 172.21.15.113:0/1450862861 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578003840 con 0x7f8570004810 2024-03-04T08:24:51.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f8593c00640 1 -- 172.21.15.113:0/1450862861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8570004810 msgr2=0x7f8570004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:51.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f8593c00640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8570004810 0x7f8570004c90 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8578001780 tx=0x7f857801c400 comp rx=0 tx=0).stop 2024-03-04T08:24:51.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f8593c00640 1 -- 172.21.15.113:0/1450862861 shutdown_connections 2024-03-04T08:24:51.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f8593c00640 1 --2- 172.21.15.113:0/1450862861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8570004810 0x7f8570004c90 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:51.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f8593c00640 1 -- 172.21.15.113:0/1450862861 >> 172.21.15.113:0/1450862861 conn(0x7f858c066db0 msgr2=0x7f858c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:51.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f8593c00640 1 -- 172.21.15.113:0/1450862861 shutdown_connections 2024-03-04T08:24:51.817 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f8593c00640 1 -- 172.21.15.113:0/1450862861 wait complete. 2024-03-04T08:24:51.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f8593c00640 1 Processor -- start 2024-03-04T08:24:51.818 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.818+0000 7f8593c00640 1 -- start start 2024-03-04T08:24:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8593c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8570004810 0x7f858c0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8593c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f858c0cdbc0 0x7f858c1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8593c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858c1f0370 0x7f858c15e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8593c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c06fb80 con 0x7f858c1f0370 2024-03-04T08:24:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8593c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c06c4f0 con 0x7f8570004810 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8593c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c06fd00 con 0x7f858c0cdbc0 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8591c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858c1f0370 0x7f858c15e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8591c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858c1f0370 0x7f858c15e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35258/0 (socket says 172.21.15.113:35258) 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8591c00640 1 -- 172.21.15.113:0/11059916 learned_addr learned my addr 172.21.15.113:0/11059916 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8591200640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8570004810 0x7f858c0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f858be00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f858c0cdbc0 0x7f858c1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8591c00640 1 -- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f858c0cdbc0 msgr2=0x7f858c1534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8591c00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f858c0cdbc0 0x7f858c1534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8591c00640 1 -- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8570004810 msgr2=0x7f858c0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8591c00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8570004810 0x7f858c0ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8591c00640 1 -- 172.21.15.113:0/11059916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0cd0d0 con 0x7f858c1f0370 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8591c00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858c1f0370 0x7f858c15e8d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f858c0cbef0 tx=0x7f858400d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8588c00640 1 -- 172.21.15.113:0/11059916 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584007440 con 0x7f858c1f0370 2024-03-04T08:24:51.821 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8588c00640 1 -- 172.21.15.113:0/11059916 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8584007600 con 0x7f858c1f0370 2024-03-04T08:24:51.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8588c00640 1 -- 172.21.15.113:0/11059916 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584004a30 con 0x7f858c1f0370 2024-03-04T08:24:51.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8593c00640 1 -- 172.21.15.113:0/11059916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858c148990 con 0x7f858c1f0370 2024-03-04T08:24:51.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8593c00640 1 -- 172.21.15.113:0/11059916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858c1f3ce0 con 0x7f858c1f0370 2024-03-04T08:24:51.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8593c00640 1 -- 172.21.15.113:0/11059916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858c075c00 con 0x7f858c1f0370 2024-03-04T08:24:51.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.822+0000 7f8588c00640 1 -- 172.21.15.113:0/11059916 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f858401b020 con 0x7f858c1f0370 2024-03-04T08:24:51.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.826+0000 7f8588c00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f856c066f90 0x7f856c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:51.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.826+0000 7f8588c00640 1 -- 172.21.15.113:0/11059916 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f85840854f0 con 0x7f858c1f0370 2024-03-04T08:24:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.826+0000 7f8591200640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f856c066f90 0x7f856c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.826+0000 7f8591200640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f856c066f90 0x7f856c069450 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f857c009fd0 tx=0x7f857c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:51.830+0000 7f8588c00640 1 -- 172.21.15.113:0/11059916 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f85840552e0 con 0x7f858c1f0370 2024-03-04T08:24:52.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.014+0000 7f8593c00640 1 -- 172.21.15.113:0/11059916 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f858c152440 con 0x7f858c1f0370 2024-03-04T08:24:52.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8588c00640 1 -- 172.21.15.113:0/11059916 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f8584059380 con 0x7f858c1f0370 2024-03-04T08:24:52.015 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:52.015 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:52.016 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 -- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f856c066f90 msgr2=0x7f856c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f856c066f90 0x7f856c069450 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f857c009fd0 tx=0x7f857c00b040 comp rx=0 tx=0).stop 2024-03-04T08:24:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 -- 172.21.15.113:0/11059916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858c1f0370 msgr2=0x7f858c15e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:52.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858c1f0370 0x7f858c15e8d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f858c0cbef0 tx=0x7f858400d8a0 comp rx=0 tx=0).stop 2024-03-04T08:24:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 -- 172.21.15.113:0/11059916 shutdown_connections 2024-03-04T08:24:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f856c066f90 0x7f856c069450 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f858c1f0370 0x7f858c15e8d0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f858c0cdbc0 0x7f858c1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 --2- 172.21.15.113:0/11059916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8570004810 0x7f858c0ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 -- 172.21.15.113:0/11059916 >> 172.21.15.113:0/11059916 conn(0x7f858c066db0 msgr2=0x7f858c152720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 -- 172.21.15.113:0/11059916 shutdown_connections 2024-03-04T08:24:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:52.018+0000 7f8593c00640 1 -- 172.21.15.113:0/11059916 wait complete. 2024-03-04T08:24:52.036 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:24:52.037 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:24:54.038 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d44a00640 1 Processor -- start 2024-03-04T08:24:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d44a00640 1 -- start start 2024-03-04T08:24:54.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d44a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d40152950 0x7f6d4015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:54.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d44a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6d400cb540 0x7f6d400c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:54.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d44a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d400c4ce0 con 0x7f6d4006faa0 2024-03-04T08:24:54.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d44a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d400c8370 con 0x7f6d400cb540 2024-03-04T08:24:54.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d44a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400ca0a0 con 0x7f6d40152950 2024-03-04T08:24:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d3fe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6d400cb540 0x7f6d400c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d3f400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d40152950 0x7f6d4015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d3ea00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f6d4006faa0 0x7f6d4006ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:34560/0 (socket says 172.21.15.113:34560) 2024-03-04T08:24:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d3ea00640 1 -- 172.21.15.113:0/978883759 learned_addr learned my addr 172.21.15.113:0/978883759 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.186+0000 7f6d3fe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f6d400cb540 0x7f6d400c30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47266/0 (socket says 172.21.15.113:47266) 2024-03-04T08:24:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.190+0000 7f6d3f400640 1 -- 172.21.15.113:0/978883759 >> v2:172.21.15.136:3300/0 conn(0x7f6d400cb540 msgr2=0x7f6d400c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.190+0000 7f6d3f400640 1 --2- 172.21.15.113:0/978883759 >> v2:172.21.15.136:3300/0 conn(0x7f6d400cb540 0x7f6d400c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.190+0000 7f6d3f400640 1 -- 172.21.15.113:0/978883759 >> v1:172.21.15.136:6789/0 conn(0x7f6d4006faa0 legacy=0x7f6d4006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.190+0000 7f6d3f400640 1 -- 172.21.15.113:0/978883759 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d400c4f80 con 0x7f6d40152950 2024-03-04T08:24:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.190+0000 7f6d3fe00640 1 --2- 172.21.15.113:0/978883759 >> v2:172.21.15.136:3300/0 conn(0x7f6d400cb540 0x7f6d400c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:24:54.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.190+0000 7f6d3f400640 1 --2- 172.21.15.113:0/978883759 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d40152950 0x7f6d4015bb40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d300097c0 tx=0x7f6d3001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b492adb5c14d1f06 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:54.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.190+0000 7f6d3e000640 1 -- 172.21.15.113:0/978883759 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300b5070 con 0x7f6d40152950 2024-03-04T08:24:54.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.190+0000 7f6d3e000640 1 -- 172.21.15.113:0/978883759 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d300a8990 con 0x7f6d40152950 2024-03-04T08:24:54.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.190+0000 7f6d3e000640 1 -- 172.21.15.113:0/978883759 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300a8d60 con 0x7f6d40152950 2024-03-04T08:24:54.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 -- 172.21.15.113:0/978883759 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d40152950 msgr2=0x7f6d4015bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:54.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 --2- 172.21.15.113:0/978883759 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d40152950 0x7f6d4015bb40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6d300097c0 tx=0x7f6d3001ec90 comp rx=0 tx=0).stop 2024-03-04T08:24:54.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 -- 172.21.15.113:0/978883759 shutdown_connections 2024-03-04T08:24:54.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 --2- 172.21.15.113:0/978883759 >> v2:172.21.15.136:3300/0 conn(0x7f6d400cb540 0x7f6d400c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:54.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 --2- 172.21.15.113:0/978883759 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d40152950 0x7f6d4015bb40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:54.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 -- 172.21.15.113:0/978883759 >> 172.21.15.113:0/978883759 conn(0x7f6d40072f40 msgr2=0x7f6d400706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:54.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 -- 172.21.15.113:0/978883759 shutdown_connections 2024-03-04T08:24:54.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 -- 172.21.15.113:0/978883759 wait complete. 2024-03-04T08:24:54.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 Processor -- start 2024-03-04T08:24:54.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 -- start start 2024-03-04T08:24:54.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d4006faa0 0x7f6d401e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:54.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d400cb540 0x7f6d40156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:54.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d40152950 0x7f6d401547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400c9a50 con 0x7f6d40152950 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4006fe80 con 0x7f6d4006faa0 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d44a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400c9d30 con 0x7f6d400cb540 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d3fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d40152950 0x7f6d401547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d3fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d40152950 0x7f6d401547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35266/0 (socket says 172.21.15.113:35266) 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d3fe00640 1 -- 172.21.15.113:0/3937338720 learned_addr learned my addr 172.21.15.113:0/3937338720 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d3ea00640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d400cb540 0x7f6d40156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d3f400640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d4006faa0 0x7f6d401e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d3fe00640 1 -- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d400cb540 msgr2=0x7f6d40156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d3fe00640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d400cb540 0x7f6d40156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d3fe00640 1 -- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d4006faa0 msgr2=0x7f6d401e1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d3fe00640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d4006faa0 0x7f6d401e1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.194+0000 7f6d3fe00640 1 -- 172.21.15.113:0/3937338720 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4015ae00 con 0x7f6d40152950 2024-03-04T08:24:54.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d3fe00640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d40152950 0x7f6d401547f0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009f00 tx=0x7f6d3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:54.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d37400640 1 -- 172.21.15.113:0/3937338720 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38018030 con 0x7f6d40152950 2024-03-04T08:24:54.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d37400640 1 -- 172.21.15.113:0/3937338720 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3800ce50 con 0x7f6d40152950 2024-03-04T08:24:54.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d44a00640 1 -- 172.21.15.113:0/3937338720 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4015b0e0 con 0x7f6d40152950 2024-03-04T08:24:54.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d37400640 1 -- 172.21.15.113:0/3937338720 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38003bc0 con 0x7f6d40152950 2024-03-04T08:24:54.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d44a00640 1 -- 172.21.15.113:0/3937338720 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4015b6a0 con 0x7f6d40152950 2024-03-04T08:24:54.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d44a00640 1 -- 172.21.15.113:0/3937338720 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d40075a80 con 0x7f6d40152950 2024-03-04T08:24:54.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d37400640 1 -- 172.21.15.113:0/3937338720 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d3801b070 con 0x7f6d40152950 2024-03-04T08:24:54.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d37400640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d20066f60 0x7f6d20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:54.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d37400640 1 -- 172.21.15.113:0/3937338720 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f6d38086470 con 0x7f6d40152950 2024-03-04T08:24:54.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.198+0000 7f6d3f400640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d20066f60 0x7f6d20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:54.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.202+0000 7f6d3f400640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d20066f60 0x7f6d20069420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6d300b2000 tx=0x7f6d300aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:54.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.202+0000 7f6d37400640 1 -- 172.21.15.113:0/3937338720 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6d38010040 con 0x7f6d40152950 2024-03-04T08:24:54.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 -- 172.21.15.113:0/3937338720 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d400c9580 con 0x7f6d40152950 2024-03-04T08:24:54.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d37400640 1 -- 172.21.15.113:0/3937338720 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f6d380562f0 con 0x7f6d40152950 2024-03-04T08:24:54.388 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:54.388 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:54.388 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 -- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d20066f60 msgr2=0x7f6d20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d20066f60 0x7f6d20069420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6d300b2000 tx=0x7f6d300aa070 comp rx=0 tx=0).stop 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 -- 172.21.15.113:0/3937338720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d40152950 msgr2=0x7f6d401547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d40152950 0x7f6d401547f0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009f00 tx=0x7f6d3800c490 comp rx=0 tx=0).stop 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 -- 172.21.15.113:0/3937338720 shutdown_connections 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6d20066f60 0x7f6d20069420 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6d40152950 0x7f6d401547f0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f6d400cb540 0x7f6d40156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 --2- 172.21.15.113:0/3937338720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6d4006faa0 0x7f6d401e1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 -- 172.21.15.113:0/3937338720 >> 172.21.15.113:0/3937338720 conn(0x7f6d40072f40 msgr2=0x7f6d400c71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 -- 172.21.15.113:0/3937338720 shutdown_connections 2024-03-04T08:24:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:54.390+0000 7f6d44a00640 1 -- 172.21.15.113:0/3937338720 wait complete. 2024-03-04T08:24:54.409 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:24:56.410 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:56.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a42200640 1 Processor -- start 2024-03-04T08:24:56.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a42200640 1 -- start start 2024-03-04T08:24:56.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a42200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5a3c152a90 0x7f5a3c159370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:56.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a42200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a3c06cc20 0x7f5a3c159de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:56.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a42200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a3c0cd010 0x7f5a3c0cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:56.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a42200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3c06b740 con 0x7f5a3c06cc20 2024-03-04T08:24:56.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a42200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3c1598b0 con 0x7f5a3c152a90 2024-03-04T08:24:56.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a42200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c06b460 con 0x7f5a3c0cd010 2024-03-04T08:24:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a3aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a3c06cc20 0x7f5a3c159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a3aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a3c06cc20 0x7f5a3c159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35278/0 (socket says 172.21.15.113:35278) 2024-03-04T08:24:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a3aa00640 1 -- 172.21.15.113:0/1754401326 learned_addr learned my addr 172.21.15.113:0/1754401326 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a3be00640 1 --2- 172.21.15.113:0/1754401326 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a3c0cd010 0x7f5a3c0cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a3b400640 1 --2- 172.21.15.113:0/1754401326 >> v2:172.21.15.136:3300/0 conn(0x7f5a3c152a90 0x7f5a3c159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a3aa00640 1 -- 172.21.15.113:0/1754401326 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a3c0cd010 msgr2=0x7f5a3c0cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a3aa00640 1 --2- 172.21.15.113:0/1754401326 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a3c0cd010 0x7f5a3c0cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a3aa00640 1 -- 172.21.15.113:0/1754401326 >> v2:172.21.15.136:3300/0 conn(0x7f5a3c152a90 msgr2=0x7f5a3c159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a3aa00640 1 --2- 172.21.15.113:0/1754401326 >> v2:172.21.15.136:3300/0 conn(0x7f5a3c152a90 0x7f5a3c159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.558+0000 7f5a3aa00640 1 -- 172.21.15.113:0/1754401326 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c072660 con 0x7f5a3c06cc20 2024-03-04T08:24:56.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a3aa00640 1 --2- 172.21.15.113:0/1754401326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c06cc20 0x7f5a3c159de0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0cdcc0 tx=0x7f5a2c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ce337d87f471c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:56.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a3a000640 1 -- 172.21.15.113:0/1754401326 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0b1070 con 0x7f5a3c06cc20 2024-03-04T08:24:56.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a3a000640 1 -- 172.21.15.113:0/1754401326 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2c0ac630 con 0x7f5a3c06cc20 2024-03-04T08:24:56.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a3a000640 1 -- 172.21.15.113:0/1754401326 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0a8050 con 0x7f5a3c06cc20 2024-03-04T08:24:56.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a42200640 1 -- 172.21.15.113:0/1754401326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c06cc20 msgr2=0x7f5a3c159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:56.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a42200640 1 --2- 172.21.15.113:0/1754401326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c06cc20 0x7f5a3c159de0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0cdcc0 tx=0x7f5a2c01c520 comp rx=0 tx=0).stop 2024-03-04T08:24:56.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a42200640 1 -- 172.21.15.113:0/1754401326 shutdown_connections 2024-03-04T08:24:56.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a42200640 1 --2- 172.21.15.113:0/1754401326 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a3c0cd010 0x7f5a3c0cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a42200640 1 --2- 172.21.15.113:0/1754401326 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c06cc20 0x7f5a3c159de0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a42200640 1 --2- 172.21.15.113:0/1754401326 >> v2:172.21.15.136:3300/0 conn(0x7f5a3c152a90 0x7f5a3c159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a42200640 1 -- 172.21.15.113:0/1754401326 >> 172.21.15.113:0/1754401326 conn(0x7f5a3c0671e0 msgr2=0x7f5a3c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:56.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a42200640 1 -- 172.21.15.113:0/1754401326 shutdown_connections 2024-03-04T08:24:56.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.562+0000 7f5a42200640 1 -- 172.21.15.113:0/1754401326 wait complete. 2024-03-04T08:24:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 Processor -- start 2024-03-04T08:24:56.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 -- start start 2024-03-04T08:24:56.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a3c06cc20 0x7f5a3c15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:56.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a3c0cd010 0x7f5a3c0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:56.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c152a90 0x7f5a3c14d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:56.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0735a0 con 0x7f5a3c152a90 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c073420 con 0x7f5a3c0cd010 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c06b060 con 0x7f5a3c06cc20 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c152a90 0x7f5a3c14d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c152a90 0x7f5a3c14d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35286/0 (socket says 172.21.15.113:35286) 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3be00640 1 -- 172.21.15.113:0/2506037786 learned_addr learned my addr 172.21.15.113:0/2506037786 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3b400640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a3c06cc20 0x7f5a3c15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3aa00640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a3c0cd010 0x7f5a3c0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3be00640 1 -- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a3c06cc20 msgr2=0x7f5a3c15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3be00640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a3c06cc20 0x7f5a3c15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3be00640 1 -- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a3c0cd010 msgr2=0x7f5a3c0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3be00640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a3c0cd010 0x7f5a3c0c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3be00640 1 -- 172.21.15.113:0/2506037786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c15aa00 con 0x7f5a3c152a90 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a3be00640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c152a90 0x7f5a3c14d700 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400ef00 tx=0x7f5a3400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a33400640 1 -- 172.21.15.113:0/2506037786 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a34018020 con 0x7f5a3c152a90 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a33400640 1 -- 172.21.15.113:0/2506037786 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a34014400 con 0x7f5a3c152a90 2024-03-04T08:24:56.566 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a33400640 1 -- 172.21.15.113:0/2506037786 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a340147f0 con 0x7f5a3c152a90 2024-03-04T08:24:56.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 -- 172.21.15.113:0/2506037786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c15bc30 con 0x7f5a3c152a90 2024-03-04T08:24:56.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 -- 172.21.15.113:0/2506037786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c0c9ac0 con 0x7f5a3c152a90 2024-03-04T08:24:56.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.566+0000 7f5a42200640 1 -- 172.21.15.113:0/2506037786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c075c00 con 0x7f5a3c152a90 2024-03-04T08:24:56.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.570+0000 7f5a33400640 1 -- 172.21.15.113:0/2506037786 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a3401b020 con 0x7f5a3c152a90 2024-03-04T08:24:56.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.570+0000 7f5a33400640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a18066f60 0x7f5a18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:56.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.570+0000 7f5a33400640 1 -- 172.21.15.113:0/2506037786 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f5a340860f0 con 0x7f5a3c152a90 2024-03-04T08:24:56.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.570+0000 7f5a3b400640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a18066f60 0x7f5a18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:56.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.570+0000 7f5a3b400640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a18066f60 0x7f5a18069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009ac0 tx=0x7f5a24009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:56.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.574+0000 7f5a33400640 1 -- 172.21.15.113:0/2506037786 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5a34022070 con 0x7f5a3c152a90 2024-03-04T08:24:56.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.758+0000 7f5a42200640 1 -- 172.21.15.113:0/2506037786 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a3c0c8640 con 0x7f5a3c152a90 2024-03-04T08:24:56.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a33400640 1 -- 172.21.15.113:0/2506037786 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f5a34055f40 con 0x7f5a3c152a90 2024-03-04T08:24:56.759 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:56.759 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:56.759 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 -- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a18066f60 msgr2=0x7f5a18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a18066f60 0x7f5a18069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009ac0 tx=0x7f5a24009190 comp rx=0 tx=0).stop 2024-03-04T08:24:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 -- 172.21.15.113:0/2506037786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c152a90 msgr2=0x7f5a3c14d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c152a90 0x7f5a3c14d700 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400ef00 tx=0x7f5a3400b480 comp rx=0 tx=0).stop 2024-03-04T08:24:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 -- 172.21.15.113:0/2506037786 shutdown_connections 2024-03-04T08:24:56.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5a18066f60 0x7f5a18069420 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a3c152a90 0x7f5a3c14d700 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a3c0cd010 0x7f5a3c0c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 --2- 172.21.15.113:0/2506037786 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5a3c06cc20 0x7f5a3c15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 -- 172.21.15.113:0/2506037786 >> 172.21.15.113:0/2506037786 conn(0x7f5a3c0671e0 msgr2=0x7f5a3c0cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 -- 172.21.15.113:0/2506037786 shutdown_connections 2024-03-04T08:24:56.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:56.762+0000 7f5a42200640 1 -- 172.21.15.113:0/2506037786 wait complete. 2024-03-04T08:24:56.781 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:24:58.781 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:24:58.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b4800640 1 Processor -- start 2024-03-04T08:24:58.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b4800640 1 -- start start 2024-03-04T08:24:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b4800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f98ac06cb60 0x7f98ac1640e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b4800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac072d80 con 0x7f98ac152c60 2024-03-04T08:24:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b4800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac073510 con 0x7f98ac0c7a80 2024-03-04T08:24:58.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b4800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac06b060 con 0x7f98ac06cb60 2024-03-04T08:24:58.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b1e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f98ac152c60 0x7f98ac15bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43296/0 (socket says 172.21.15.113:43296) 2024-03-04T08:24:58.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b1e00640 1 -- 172.21.15.113:0/3328622816 learned_addr learned my addr 172.21.15.113:0/3328622816 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:24:58.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b2800640 1 --2- 172.21.15.113:0/3328622816 >> v2:172.21.15.136:3300/0 conn(0x7f98ac06cb60 0x7f98ac1640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:58.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491787211 0 0) 0x7f98ac072d80 con 0x7f98ac152c60 2024-03-04T08:24:58.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9894003530 con 0x7f98ac152c60 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021924267 0 0) 0x7f98ac073510 con 0x7f98ac0c7a80 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98ac072d80 con 0x7f98ac0c7a80 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2636429526 0 0) 0x7f9894003530 con 0x7f98ac152c60 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 >> v2:172.21.15.136:3300/0 conn(0x7f98ac06cb60 msgr2=0x7f98ac1640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 --2- 172.21.15.113:0/3328622816 >> v2:172.21.15.136:3300/0 conn(0x7f98ac06cb60 0x7f98ac1640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 >> v1:172.21.15.136:6789/0 conn(0x7f98ac0c7a80 legacy=0x7f98ac0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac06b2b0 con 0x7f98ac152c60 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f98a0001680 con 0x7f98ac152c60 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 >> v1:172.21.15.113:6789/0 conn(0x7f98ac152c60 legacy=0x7f98ac15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9894003ba0 0x7f9894003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f9894004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9894005220 0x7f989400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac06b060 con 0x7f9894004810 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989400c150 con 0x7f9894005220 2024-03-04T08:24:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989400c350 con 0x7f9894003ba0 2024-03-04T08:24:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b1400640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f9894004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b2800640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9894005220 0x7f989400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b1e00640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9894003ba0 0x7f9894003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b1400640 1 -- 172.21.15.113:0/3328622816 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9894003ba0 msgr2=0x7f9894003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b1400640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9894003ba0 0x7f9894003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b1400640 1 -- 172.21.15.113:0/3328622816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9894005220 msgr2=0x7f989400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:24:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b1400640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9894005220 0x7f989400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b2800640 1 -- 172.21.15.113:0/3328622816 reap_dead start 2024-03-04T08:24:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.930+0000 7f98b1400640 1 -- 172.21.15.113:0/3328622816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894003530 con 0x7f9894004810 2024-03-04T08:24:58.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b1400640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f9894004c90 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f989c0018d0 tx=0x7f989c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3086b11ef3b9bfa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:58.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c0b0070 con 0x7f9894004810 2024-03-04T08:24:58.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c0052f0 con 0x7f9894004810 2024-03-04T08:24:58.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b0a00640 1 -- 172.21.15.113:0/3328622816 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c001520 con 0x7f9894004810 2024-03-04T08:24:58.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b4800640 1 -- 172.21.15.113:0/3328622816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 msgr2=0x7f9894004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:58.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b4800640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f9894004c90 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f989c0018d0 tx=0x7f989c01c4f0 comp rx=0 tx=0).stop 2024-03-04T08:24:58.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b4800640 1 -- 172.21.15.113:0/3328622816 shutdown_connections 2024-03-04T08:24:58.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b4800640 1 --2- 172.21.15.113:0/3328622816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f9894004c90 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:58.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b4800640 1 -- 172.21.15.113:0/3328622816 >> 172.21.15.113:0/3328622816 conn(0x7f98ac0671b0 msgr2=0x7f98ac06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:58.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b4800640 1 -- 172.21.15.113:0/3328622816 shutdown_connections 2024-03-04T08:24:58.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.934+0000 7f98b4800640 1 -- 172.21.15.113:0/3328622816 wait complete. 2024-03-04T08:24:58.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b4800640 1 Processor -- start 2024-03-04T08:24:58.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b4800640 1 -- start start 2024-03-04T08:24:58.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b4800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f98ac1e8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:58.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b4800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98ac154510 0x7f98ac14cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:58.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b4800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98ac14d260 0x7f98ac1495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b4800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c8ac0 con 0x7f9894004810 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b4800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0731d0 con 0x7f98ac154510 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b4800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c8c40 con 0x7f98ac14d260 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f98ac1e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b1e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f98ac1e8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42480/0 (socket says 172.21.15.113:42480) 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b1e00640 1 -- 172.21.15.113:0/654072474 learned_addr learned my addr 172.21.15.113:0/654072474 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b2800640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98ac14d260 0x7f98ac1495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b1400640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98ac154510 0x7f98ac14cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b1e00640 1 -- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98ac14d260 msgr2=0x7f98ac1495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b1e00640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98ac14d260 0x7f98ac1495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b1e00640 1 -- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98ac154510 msgr2=0x7f98ac14cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b1e00640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98ac154510 0x7f98ac14cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b1e00640 1 -- 172.21.15.113:0/654072474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac14d710 con 0x7f9894004810 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b2800640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98ac14d260 0x7f98ac1495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:24:58.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b1e00640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f98ac1e8460 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f98a0004e00 tx=0x7f98a0007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:58.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98a6000640 1 -- 172.21.15.113:0/654072474 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a000f7b0 con 0x7f9894004810 2024-03-04T08:24:58.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98a6000640 1 -- 172.21.15.113:0/654072474 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a0018070 con 0x7f9894004810 2024-03-04T08:24:58.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b4800640 1 -- 172.21.15.113:0/654072474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac1f6840 con 0x7f9894004810 2024-03-04T08:24:58.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98a6000640 1 -- 172.21.15.113:0/654072474 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a000fbd0 con 0x7f9894004810 2024-03-04T08:24:58.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.938+0000 7f98b4800640 1 -- 172.21.15.113:0/654072474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac1f6e00 con 0x7f9894004810 2024-03-04T08:24:58.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.942+0000 7f98b4800640 1 -- 172.21.15.113:0/654072474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac075c00 con 0x7f9894004810 2024-03-04T08:24:58.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.942+0000 7f98a6000640 1 -- 172.21.15.113:0/654072474 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98a001d460 con 0x7f9894004810 2024-03-04T08:24:58.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.942+0000 7f98a6000640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9890066f60 0x7f9890069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:24:58.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.942+0000 7f98a6000640 1 -- 172.21.15.113:0/654072474 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f98a00859e0 con 0x7f9894004810 2024-03-04T08:24:58.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.942+0000 7f98b1400640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9890066f60 0x7f9890069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:24:58.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.942+0000 7f98b1400640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9890066f60 0x7f9890069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f989c0ad000 tx=0x7f989c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:24:58.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:58.946+0000 7f98a6000640 1 -- 172.21.15.113:0/654072474 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f98a00558c0 con 0x7f9894004810 2024-03-04T08:24:59.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 -- 172.21.15.113:0/654072474 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98ac151210 con 0x7f9894004810 2024-03-04T08:24:59.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98a6000640 1 -- 172.21.15.113:0/654072474 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f98a0059960 con 0x7f9894004810 2024-03-04T08:24:59.132 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:24:59.132 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:24:59.133 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:24:59.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 -- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9890066f60 msgr2=0x7f9890069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:59.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9890066f60 0x7f9890069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f989c0ad000 tx=0x7f989c009040 comp rx=0 tx=0).stop 2024-03-04T08:24:59.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 -- 172.21.15.113:0/654072474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 msgr2=0x7f98ac1e8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f98ac1e8460 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f98a0004e00 tx=0x7f98a0007430 comp rx=0 tx=0).stop 2024-03-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 -- 172.21.15.113:0/654072474 shutdown_connections 2024-03-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f98ac14d260 0x7f98ac1495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98ac154510 0x7f98ac14cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9890066f60 0x7f9890069420 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 --2- 172.21.15.113:0/654072474 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9894004810 0x7f98ac1e8460 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 -- 172.21.15.113:0/654072474 >> 172.21.15.113:0/654072474 conn(0x7f98ac0671b0 msgr2=0x7f98ac0cb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 -- 172.21.15.113:0/654072474 shutdown_connections 2024-03-04T08:24:59.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:24:59.134+0000 7f98b4800640 1 -- 172.21.15.113:0/654072474 wait complete. 2024-03-04T08:24:59.153 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:01.154 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.302+0000 7f1b6da00640 1 Processor -- start 2024-03-04T08:25:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.302+0000 7f1b6da00640 1 -- start start 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.302+0000 7f1b6da00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b680b20b0 0x7f1b68157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.302+0000 7f1b6da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b68157c50 0x7f1b6806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.302+0000 7f1b6da00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b680c1ec0 con 0x7f1b680c85f0 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.302+0000 7f1b6da00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b680c9b30 con 0x7f1b680b20b0 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.302+0000 7f1b6da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b680c12a0 con 0x7f1b68157c50 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.302+0000 7f1b66a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b680b20b0 0x7f1b68157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.302+0000 7f1b66a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1b680b20b0 0x7f1b68157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42490/0 (socket says 172.21.15.113:42490) 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.302+0000 7f1b66a00640 1 -- 172.21.15.113:0/4252735691 learned_addr learned my addr 172.21.15.113:0/4252735691 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b5e000640 1 --2- 172.21.15.113:0/4252735691 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b68157c50 0x7f1b6806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b66a00640 1 -- 172.21.15.113:0/4252735691 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b68157c50 msgr2=0x7f1b6806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b66a00640 1 --2- 172.21.15.113:0/4252735691 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b68157c50 0x7f1b6806b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:01.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b66a00640 1 -- 172.21.15.113:0/4252735691 >> v1:172.21.15.136:6789/0 conn(0x7f1b680c85f0 legacy=0x7f1b6815a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:01.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b66a00640 1 -- 172.21.15.113:0/4252735691 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b680c1580 con 0x7f1b680b20b0 2024-03-04T08:25:01.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b66a00640 1 --2- 172.21.15.113:0/4252735691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680b20b0 0x7f1b68157710 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800af60 tx=0x7f1b5801db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401225874e743657 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b66000640 1 -- 172.21.15.113:0/4252735691 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800a2d0 con 0x7f1b680b20b0 2024-03-04T08:25:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b66000640 1 -- 172.21.15.113:0/4252735691 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b580a6870 con 0x7f1b680b20b0 2024-03-04T08:25:01.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b66000640 1 -- 172.21.15.113:0/4252735691 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b580a6c40 con 0x7f1b680b20b0 2024-03-04T08:25:01.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b6da00640 1 -- 172.21.15.113:0/4252735691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680b20b0 msgr2=0x7f1b68157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:01.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.306+0000 7f1b6da00640 1 --2- 172.21.15.113:0/4252735691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680b20b0 0x7f1b68157710 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800af60 tx=0x7f1b5801db00 comp rx=0 tx=0).stop 2024-03-04T08:25:01.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 -- 172.21.15.113:0/4252735691 shutdown_connections 2024-03-04T08:25:01.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 --2- 172.21.15.113:0/4252735691 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b68157c50 0x7f1b6806b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:01.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 --2- 172.21.15.113:0/4252735691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680b20b0 0x7f1b68157710 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:01.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 -- 172.21.15.113:0/4252735691 >> 172.21.15.113:0/4252735691 conn(0x7f1b6806e820 msgr2=0x7f1b6806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:01.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 -- 172.21.15.113:0/4252735691 shutdown_connections 2024-03-04T08:25:01.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 -- 172.21.15.113:0/4252735691 wait complete. 2024-03-04T08:25:01.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 Processor -- start 2024-03-04T08:25:01.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 -- start start 2024-03-04T08:25:01.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b680b20b0 0x7f1b68152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680c85f0 0x7f1b6814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b68157c50 0x7f1b68072400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b680c1b80 con 0x7f1b680c85f0 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b68156ec0 con 0x7f1b680b20b0 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b6da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b680c80c0 con 0x7f1b68157c50 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b5e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680c85f0 0x7f1b6814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b5e000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680c85f0 0x7f1b6814a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42506/0 (socket says 172.21.15.113:42506) 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b5e000640 1 -- 172.21.15.113:0/592844269 learned_addr learned my addr 172.21.15.113:0/592844269 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b66a00640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b680b20b0 0x7f1b68152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b67400640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b68157c50 0x7f1b68072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b5e000640 1 -- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b68157c50 msgr2=0x7f1b68072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b5e000640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b68157c50 0x7f1b68072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b5e000640 1 -- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b680b20b0 msgr2=0x7f1b68152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b5e000640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b680b20b0 0x7f1b68152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:01.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b5e000640 1 -- 172.21.15.113:0/592844269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b68151e80 con 0x7f1b680c85f0 2024-03-04T08:25:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b5e000640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680c85f0 0x7f1b6814a540 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000d7d0 tx=0x7f1b5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.310+0000 7f1b5f400640 1 -- 172.21.15.113:0/592844269 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5000b6c0 con 0x7f1b680c85f0 2024-03-04T08:25:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.314+0000 7f1b5f400640 1 -- 172.21.15.113:0/592844269 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b50019070 con 0x7f1b680c85f0 2024-03-04T08:25:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.314+0000 7f1b6da00640 1 -- 172.21.15.113:0/592844269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b681f6fc0 con 0x7f1b680c85f0 2024-03-04T08:25:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.314+0000 7f1b5f400640 1 -- 172.21.15.113:0/592844269 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5000baa0 con 0x7f1b680c85f0 2024-03-04T08:25:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.314+0000 7f1b6da00640 1 -- 172.21.15.113:0/592844269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b681f7580 con 0x7f1b680c85f0 2024-03-04T08:25:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.314+0000 7f1b6da00640 1 -- 172.21.15.113:0/592844269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b68075c00 con 0x7f1b680c85f0 2024-03-04T08:25:01.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.314+0000 7f1b5f400640 1 -- 172.21.15.113:0/592844269 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b5000bc40 con 0x7f1b680c85f0 2024-03-04T08:25:01.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.314+0000 7f1b5f400640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b44066f60 0x7f1b44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:01.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.314+0000 7f1b5f400640 1 -- 172.21.15.113:0/592844269 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f1b50085f20 con 0x7f1b680c85f0 2024-03-04T08:25:01.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.314+0000 7f1b66a00640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b44066f60 0x7f1b44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:01.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.318+0000 7f1b66a00640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b44066f60 0x7f1b44069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1b580af000 tx=0x7f1b580a8070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:01.318 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.318+0000 7f1b5f400640 1 -- 172.21.15.113:0/592844269 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1b50056450 con 0x7f1b680c85f0 2024-03-04T08:25:01.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 -- 172.21.15.113:0/592844269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b680c83d0 con 0x7f1b680c85f0 2024-03-04T08:25:01.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b5f400640 1 -- 172.21.15.113:0/592844269 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f1b50055d50 con 0x7f1b680c85f0 2024-03-04T08:25:01.507 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:01.507 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:01.508 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:01.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 -- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b44066f60 msgr2=0x7f1b44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b44066f60 0x7f1b44069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1b580af000 tx=0x7f1b580a8070 comp rx=0 tx=0).stop 2024-03-04T08:25:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 -- 172.21.15.113:0/592844269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680c85f0 msgr2=0x7f1b6814a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680c85f0 0x7f1b6814a540 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000d7d0 tx=0x7f1b5000dc90 comp rx=0 tx=0).stop 2024-03-04T08:25:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 -- 172.21.15.113:0/592844269 shutdown_connections 2024-03-04T08:25:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1b68157c50 0x7f1b68072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1b44066f60 0x7f1b44069420 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1b680c85f0 0x7f1b6814a540 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 --2- 172.21.15.113:0/592844269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b680b20b0 0x7f1b68152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 -- 172.21.15.113:0/592844269 >> 172.21.15.113:0/592844269 conn(0x7f1b6806e820 msgr2=0x7f1b680c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:01.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 -- 172.21.15.113:0/592844269 shutdown_connections 2024-03-04T08:25:01.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:01.510+0000 7f1b6da00640 1 -- 172.21.15.113:0/592844269 wait complete. 2024-03-04T08:25:01.528 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:03.531 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:03.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd563a00640 1 Processor -- start 2024-03-04T08:25:03.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd563a00640 1 -- start start 2024-03-04T08:25:03.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd563a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd55c0c8200 0x7fd55c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:03.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd563a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd55c152bd0 0x7fd55c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:03.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd563a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55c1591f0 con 0x7fd55c1527f0 2024-03-04T08:25:03.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd563a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55c071df0 con 0x7fd55c0c8200 2024-03-04T08:25:03.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd563a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55c1597b0 con 0x7fd55c152bd0 2024-03-04T08:25:03.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd55be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd55c0c8200 0x7fd55c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:03.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd55be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd55c0c8200 0x7fd55c070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42516/0 (socket says 172.21.15.113:42516) 2024-03-04T08:25:03.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd55be00640 1 -- 172.21.15.113:0/455913656 learned_addr learned my addr 172.21.15.113:0/455913656 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:03.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd561a00640 1 --2- 172.21.15.113:0/455913656 >> v2:172.21.15.136:3300/0 conn(0x7fd55c152bd0 0x7fd55c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:03.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd55be00640 1 -- 172.21.15.113:0/455913656 >> v2:172.21.15.136:3300/0 conn(0x7fd55c152bd0 msgr2=0x7fd55c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:03.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd55be00640 1 --2- 172.21.15.113:0/455913656 >> v2:172.21.15.136:3300/0 conn(0x7fd55c152bd0 0x7fd55c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:03.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd55be00640 1 -- 172.21.15.113:0/455913656 >> v1:172.21.15.136:6789/0 conn(0x7fd55c1527f0 legacy=0x7fd55c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:03.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.678+0000 7fd55be00640 1 -- 172.21.15.113:0/455913656 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c1594d0 con 0x7fd55c0c8200 2024-03-04T08:25:03.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd55be00640 1 --2- 172.21.15.113:0/455913656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c0c8200 0x7fd55c070d40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0712b0 tx=0x7fd54801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1316f60e4543fa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:03.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd55b400640 1 -- 172.21.15.113:0/455913656 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5480ab430 con 0x7fd55c0c8200 2024-03-04T08:25:03.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd55b400640 1 -- 172.21.15.113:0/455913656 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5480b0070 con 0x7fd55c0c8200 2024-03-04T08:25:03.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd55b400640 1 -- 172.21.15.113:0/455913656 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5480ab7f0 con 0x7fd55c0c8200 2024-03-04T08:25:03.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd563a00640 1 -- 172.21.15.113:0/455913656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c0c8200 msgr2=0x7fd55c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:03.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd563a00640 1 --2- 172.21.15.113:0/455913656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c0c8200 0x7fd55c070d40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0712b0 tx=0x7fd54801c520 comp rx=0 tx=0).stop 2024-03-04T08:25:03.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd563a00640 1 -- 172.21.15.113:0/455913656 shutdown_connections 2024-03-04T08:25:03.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd563a00640 1 --2- 172.21.15.113:0/455913656 >> v2:172.21.15.136:3300/0 conn(0x7fd55c152bd0 0x7fd55c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:03.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd563a00640 1 --2- 172.21.15.113:0/455913656 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c0c8200 0x7fd55c070d40 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:03.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd563a00640 1 -- 172.21.15.113:0/455913656 >> 172.21.15.113:0/455913656 conn(0x7fd55c06e820 msgr2=0x7fd55c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:03.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd563a00640 1 -- 172.21.15.113:0/455913656 shutdown_connections 2024-03-04T08:25:03.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.682+0000 7fd563a00640 1 -- 172.21.15.113:0/455913656 wait complete. 2024-03-04T08:25:03.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd563a00640 1 Processor -- start 2024-03-04T08:25:03.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd563a00640 1 -- start start 2024-03-04T08:25:03.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd563a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd55c0c8200 0x7fd55c0c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:03.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd563a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c1527f0 0x7fd55c148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:03.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd563a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c152bd0 0x7fd55c15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:03.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd563a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c1654f0 con 0x7fd55c1527f0 2024-03-04T08:25:03.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd563a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c165370 con 0x7fd55c152bd0 2024-03-04T08:25:03.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd563a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c165670 con 0x7fd55c0c8200 2024-03-04T08:25:03.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd55be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c1527f0 0x7fd55c148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:03.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd55be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c1527f0 0x7fd55c148990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42526/0 (socket says 172.21.15.113:42526) 2024-03-04T08:25:03.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd55be00640 1 -- 172.21.15.113:0/2532721228 learned_addr learned my addr 172.21.15.113:0/2532721228 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:03.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd561a00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c152bd0 0x7fd55c15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:03.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd561000640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd55c0c8200 0x7fd55c0c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd55be00640 1 -- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd55c0c8200 msgr2=0x7fd55c0c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd55be00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd55c0c8200 0x7fd55c0c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd55be00640 1 -- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c152bd0 msgr2=0x7fd55c15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd55be00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c152bd0 0x7fd55c15ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd55be00640 1 -- 172.21.15.113:0/2532721228 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c1f4b50 con 0x7fd55c1527f0 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd55be00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c1527f0 0x7fd55c148990 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd54801a150 tx=0x7fd54801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd558c00640 1 -- 172.21.15.113:0/2532721228 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5480be5f0 con 0x7fd55c1527f0 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd558c00640 1 -- 172.21.15.113:0/2532721228 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5480be790 con 0x7fd55c1527f0 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd558c00640 1 -- 172.21.15.113:0/2532721228 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5480beb60 con 0x7fd55c1527f0 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd563a00640 1 -- 172.21.15.113:0/2532721228 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55c1f7dc0 con 0x7fd55c1527f0 2024-03-04T08:25:03.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.686+0000 7fd563a00640 1 -- 172.21.15.113:0/2532721228 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55c1f4070 con 0x7fd55c1527f0 2024-03-04T08:25:03.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.690+0000 7fd563a00640 1 -- 172.21.15.113:0/2532721228 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55c075c00 con 0x7fd55c1527f0 2024-03-04T08:25:03.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.690+0000 7fd558c00640 1 -- 172.21.15.113:0/2532721228 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5480b43f0 con 0x7fd55c1527f0 2024-03-04T08:25:03.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.690+0000 7fd558c00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd53c066f60 0x7fd53c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:03.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.690+0000 7fd558c00640 1 -- 172.21.15.113:0/2532721228 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fd5480a8070 con 0x7fd55c1527f0 2024-03-04T08:25:03.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.690+0000 7fd561000640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd53c066f60 0x7fd53c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:03.690 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.690+0000 7fd561000640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd53c066f60 0x7fd53c069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd550001810 tx=0x7fd550009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:03.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.694+0000 7fd558c00640 1 -- 172.21.15.113:0/2532721228 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd548121e20 con 0x7fd55c1527f0 2024-03-04T08:25:03.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.878+0000 7fd563a00640 1 -- 172.21.15.113:0/2532721228 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd55c0c79d0 con 0x7fd55c1527f0 2024-03-04T08:25:03.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.878+0000 7fd558c00640 1 -- 172.21.15.113:0/2532721228 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fd548016110 con 0x7fd55c1527f0 2024-03-04T08:25:03.878 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:03.878 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:03.878 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 -- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd53c066f60 msgr2=0x7fd53c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd53c066f60 0x7fd53c069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fd550001810 tx=0x7fd550009040 comp rx=0 tx=0).stop 2024-03-04T08:25:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 -- 172.21.15.113:0/2532721228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c1527f0 msgr2=0x7fd55c148990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c1527f0 0x7fd55c148990 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd54801a150 tx=0x7fd54801ad60 comp rx=0 tx=0).stop 2024-03-04T08:25:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 -- 172.21.15.113:0/2532721228 shutdown_connections 2024-03-04T08:25:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd55c152bd0 0x7fd55c15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd53c066f60 0x7fd53c069420 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd55c1527f0 0x7fd55c148990 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 --2- 172.21.15.113:0/2532721228 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd55c0c8200 0x7fd55c0c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:03.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 -- 172.21.15.113:0/2532721228 >> 172.21.15.113:0/2532721228 conn(0x7fd55c06e820 msgr2=0x7fd55c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 -- 172.21.15.113:0/2532721228 shutdown_connections 2024-03-04T08:25:03.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:03.882+0000 7fd563a00640 1 -- 172.21.15.113:0/2532721228 wait complete. 2024-03-04T08:25:03.900 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:25:03.900 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:25:05.901 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:06.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba7400640 1 Processor -- start 2024-03-04T08:25:06.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba7400640 1 -- start start 2024-03-04T08:25:06.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba7400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feba01527f0 0x7feba0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:06.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feba0152bd0 0x7feba01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:06.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba7400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba01591f0 con 0x7feba00c8200 2024-03-04T08:25:06.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba7400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba0071df0 con 0x7feba0152bd0 2024-03-04T08:25:06.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba7400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba01597b0 con 0x7feba01527f0 2024-03-04T08:25:06.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feba0152bd0 0x7feba01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:06.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feb9fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7feba00c8200 0x7feba0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:43312/0 (socket says 172.21.15.113:43312) 2024-03-04T08:25:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feba0152bd0 0x7feba01587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42530/0 (socket says 172.21.15.113:42530) 2024-03-04T08:25:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feb9fe00640 1 -- 172.21.15.113:0/3377711781 learned_addr learned my addr 172.21.15.113:0/3377711781 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba4a00640 1 --2- 172.21.15.113:0/3377711781 >> v2:172.21.15.136:3300/0 conn(0x7feba01527f0 0x7feba0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba5400640 1 -- 172.21.15.113:0/3377711781 >> v2:172.21.15.136:3300/0 conn(0x7feba01527f0 msgr2=0x7feba0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba5400640 1 --2- 172.21.15.113:0/3377711781 >> v2:172.21.15.136:3300/0 conn(0x7feba01527f0 0x7feba0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba5400640 1 -- 172.21.15.113:0/3377711781 >> v1:172.21.15.113:6789/0 conn(0x7feba00c8200 legacy=0x7feba0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.050+0000 7feba5400640 1 -- 172.21.15.113:0/3377711781 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba01594d0 con 0x7feba0152bd0 2024-03-04T08:25:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feba5400640 1 --2- 172.21.15.113:0/3377711781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 0x7feba01587d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7feba006f400 tx=0x7feb9801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3095b4d2a6d1006 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feb9f400640 1 -- 172.21.15.113:0/3377711781 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980b0070 con 0x7feba0152bd0 2024-03-04T08:25:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feb9f400640 1 -- 172.21.15.113:0/3377711781 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb980ab600 con 0x7feba0152bd0 2024-03-04T08:25:06.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feb9f400640 1 -- 172.21.15.113:0/3377711781 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980ab9d0 con 0x7feba0152bd0 2024-03-04T08:25:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feba7400640 1 -- 172.21.15.113:0/3377711781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 msgr2=0x7feba01587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feba7400640 1 --2- 172.21.15.113:0/3377711781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 0x7feba01587d0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7feba006f400 tx=0x7feb9801c480 comp rx=0 tx=0).stop 2024-03-04T08:25:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feba7400640 1 -- 172.21.15.113:0/3377711781 shutdown_connections 2024-03-04T08:25:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feba7400640 1 --2- 172.21.15.113:0/3377711781 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 0x7feba01587d0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feba7400640 1 --2- 172.21.15.113:0/3377711781 >> v2:172.21.15.136:3300/0 conn(0x7feba01527f0 0x7feba0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feba7400640 1 -- 172.21.15.113:0/3377711781 >> 172.21.15.113:0/3377711781 conn(0x7feba006e820 msgr2=0x7feba006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:06.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feba7400640 1 -- 172.21.15.113:0/3377711781 shutdown_connections 2024-03-04T08:25:06.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.054+0000 7feba7400640 1 -- 172.21.15.113:0/3377711781 wait complete. 2024-03-04T08:25:06.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba7400640 1 Processor -- start 2024-03-04T08:25:06.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba7400640 1 -- start start 2024-03-04T08:25:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba00c8200 0x7feba01548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba7400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feba01527f0 0x7feba0153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 0x7feba0150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba7400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba01654f0 con 0x7feba0152bd0 2024-03-04T08:25:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba7400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba0165370 con 0x7feba00c8200 2024-03-04T08:25:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba7400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba0165670 con 0x7feba01527f0 2024-03-04T08:25:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 0x7feba0150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 0x7feba0150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42536/0 (socket says 172.21.15.113:42536) 2024-03-04T08:25:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba5400640 1 -- 172.21.15.113:0/591046044 learned_addr learned my addr 172.21.15.113:0/591046044 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba4a00640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba00c8200 0x7feba01548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feb9fe00640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feba01527f0 0x7feba0153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba5400640 1 -- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feba01527f0 msgr2=0x7feba0153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba5400640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feba01527f0 0x7feba0153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba5400640 1 -- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba00c8200 msgr2=0x7feba01548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba5400640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba00c8200 0x7feba01548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba5400640 1 -- 172.21.15.113:0/591046044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba014cf50 con 0x7feba0152bd0 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba4a00640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba00c8200 0x7feba01548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba5400640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 0x7feba0150aa0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7feb980ae040 tx=0x7feb98019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feb9cc00640 1 -- 172.21.15.113:0/591046044 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980b0040 con 0x7feba0152bd0 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feb9cc00640 1 -- 172.21.15.113:0/591046044 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb980b9070 con 0x7feba0152bd0 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba7400640 1 -- 172.21.15.113:0/591046044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba01502e0 con 0x7feba0152bd0 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feb9cc00640 1 -- 172.21.15.113:0/591046044 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb980b4650 con 0x7feba0152bd0 2024-03-04T08:25:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.058+0000 7feba7400640 1 -- 172.21.15.113:0/591046044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba014d4b0 con 0x7feba0152bd0 2024-03-04T08:25:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.062+0000 7feba7400640 1 -- 172.21.15.113:0/591046044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba0075c00 con 0x7feba0152bd0 2024-03-04T08:25:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.062+0000 7feb9cc00640 1 -- 172.21.15.113:0/591046044 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb980b47f0 con 0x7feba0152bd0 2024-03-04T08:25:06.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.062+0000 7feb9cc00640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb80066f60 0x7feb80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:06.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.062+0000 7feb9cc00640 1 -- 172.21.15.113:0/591046044 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7feb9811f740 con 0x7feba0152bd0 2024-03-04T08:25:06.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.062+0000 7feba4a00640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb80066f60 0x7feb80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:06.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.062+0000 7feba4a00640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb80066f60 0x7feb80069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feb900074c0 tx=0x7feb9000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:06.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.066+0000 7feb9cc00640 1 -- 172.21.15.113:0/591046044 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7feb98016070 con 0x7feba0152bd0 2024-03-04T08:25:06.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.254+0000 7feba7400640 1 -- 172.21.15.113:0/591046044 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feba0157a10 con 0x7feba0152bd0 2024-03-04T08:25:06.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.254+0000 7feb9cc00640 1 -- 172.21.15.113:0/591046044 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7feb980efca0 con 0x7feba0152bd0 2024-03-04T08:25:06.254 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:06.254 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:06.254 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:06.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 -- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb80066f60 msgr2=0x7feb80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:06.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb80066f60 0x7feb80069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7feb900074c0 tx=0x7feb9000b040 comp rx=0 tx=0).stop 2024-03-04T08:25:06.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 -- 172.21.15.113:0/591046044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 msgr2=0x7feba0150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:06.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 0x7feba0150aa0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7feb980ae040 tx=0x7feb98019220 comp rx=0 tx=0).stop 2024-03-04T08:25:06.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 -- 172.21.15.113:0/591046044 shutdown_connections 2024-03-04T08:25:06.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb80066f60 0x7feb80069420 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:06.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feba0152bd0 0x7feba0150aa0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:06.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feba01527f0 0x7feba0153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:06.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 --2- 172.21.15.113:0/591046044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba00c8200 0x7feba01548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:06.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 -- 172.21.15.113:0/591046044 >> 172.21.15.113:0/591046044 conn(0x7feba006e820 msgr2=0x7feba006ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:06.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 -- 172.21.15.113:0/591046044 shutdown_connections 2024-03-04T08:25:06.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:06.258+0000 7feba7400640 1 -- 172.21.15.113:0/591046044 wait complete. 2024-03-04T08:25:06.275 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:08.275 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.422+0000 7f5fd5400640 1 Processor -- start 2024-03-04T08:25:08.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.422+0000 7f5fd5400640 1 -- start start 2024-03-04T08:25:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fd5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd00c8200 0x7f5fd0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fd5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5fd0152bd0 0x7f5fd01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:08.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fd5400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd01591f0 con 0x7f5fd01527f0 2024-03-04T08:25:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fd5400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd0071df0 con 0x7f5fd0152bd0 2024-03-04T08:25:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fd5400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd01597b0 con 0x7f5fd00c8200 2024-03-04T08:25:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fce800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5fd01527f0 0x7f5fd0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38216/0 (socket says 172.21.15.113:38216) 2024-03-04T08:25:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fce800640 1 -- 172.21.15.113:0/3569380484 learned_addr learned my addr 172.21.15.113:0/3569380484 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fc5e00640 1 --2- 172.21.15.113:0/3569380484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd00c8200 0x7f5fd0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:08.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fcf200640 1 --2- 172.21.15.113:0/3569380484 >> v2:172.21.15.136:3300/0 conn(0x7f5fd0152bd0 0x7f5fd01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:08.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fcde00640 1 -- 172.21.15.113:0/3569380484 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287719575 0 0) 0x7f5fd01591f0 con 0x7f5fd01527f0 2024-03-04T08:25:08.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fcde00640 1 -- 172.21.15.113:0/3569380484 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb4003530 con 0x7f5fd01527f0 2024-03-04T08:25:08.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fc5e00640 1 -- 172.21.15.113:0/3569380484 >> v2:172.21.15.136:3300/0 conn(0x7f5fd0152bd0 msgr2=0x7f5fd01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:08.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fc5e00640 1 --2- 172.21.15.113:0/3569380484 >> v2:172.21.15.136:3300/0 conn(0x7f5fd0152bd0 0x7f5fd01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:08.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fc5e00640 1 -- 172.21.15.113:0/3569380484 >> v1:172.21.15.113:6789/0 conn(0x7f5fd01527f0 legacy=0x7f5fd0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:08.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fc5e00640 1 -- 172.21.15.113:0/3569380484 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd01594d0 con 0x7f5fd00c8200 2024-03-04T08:25:08.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.426+0000 7f5fcf200640 1 --2- 172.21.15.113:0/3569380484 >> v2:172.21.15.136:3300/0 conn(0x7f5fd0152bd0 0x7f5fd01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:25:08.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fc5e00640 1 --2- 172.21.15.113:0/3569380484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd00c8200 0x7f5fd0070d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80097c0 tx=0x7f5fb801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ed7910759887c9c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:08.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fcde00640 1 -- 172.21.15.113:0/3569380484 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb80b1070 con 0x7f5fd00c8200 2024-03-04T08:25:08.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fcde00640 1 -- 172.21.15.113:0/3569380484 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb80ac410 con 0x7f5fd00c8200 2024-03-04T08:25:08.427 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fcde00640 1 -- 172.21.15.113:0/3569380484 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb80ac7c0 con 0x7f5fd00c8200 2024-03-04T08:25:08.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 -- 172.21.15.113:0/3569380484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd00c8200 msgr2=0x7f5fd0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:08.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 --2- 172.21.15.113:0/3569380484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd00c8200 0x7f5fd0070d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80097c0 tx=0x7f5fb801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:25:08.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 -- 172.21.15.113:0/3569380484 shutdown_connections 2024-03-04T08:25:08.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 --2- 172.21.15.113:0/3569380484 >> v2:172.21.15.136:3300/0 conn(0x7f5fd0152bd0 0x7f5fd01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:08.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 --2- 172.21.15.113:0/3569380484 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd00c8200 0x7f5fd0070d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:08.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 -- 172.21.15.113:0/3569380484 >> 172.21.15.113:0/3569380484 conn(0x7f5fd006e820 msgr2=0x7f5fd006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:08.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 -- 172.21.15.113:0/3569380484 shutdown_connections 2024-03-04T08:25:08.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 -- 172.21.15.113:0/3569380484 wait complete. 2024-03-04T08:25:08.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 Processor -- start 2024-03-04T08:25:08.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 -- start start 2024-03-04T08:25:08.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd00c8200 0x7f5fd00b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:08.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd01527f0 0x7f5fd0150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:08.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd0152bd0 0x7f5fd014a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:08.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd01654f0 con 0x7f5fd00c8200 2024-03-04T08:25:08.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0165370 con 0x7f5fd01527f0 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.430+0000 7f5fd5400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0165670 con 0x7f5fd0152bd0 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fce800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd00c8200 0x7f5fd00b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fce800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd00c8200 0x7f5fd00b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39956/0 (socket says 172.21.15.113:39956) 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fce800640 1 -- 172.21.15.113:0/1250307451 learned_addr learned my addr 172.21.15.113:0/1250307451 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fc5e00640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd01527f0 0x7f5fd0150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fcf200640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd0152bd0 0x7f5fd014a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fce800640 1 -- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd0152bd0 msgr2=0x7f5fd014a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fce800640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd0152bd0 0x7f5fd014a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fce800640 1 -- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd01527f0 msgr2=0x7f5fd0150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fce800640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd01527f0 0x7f5fd0150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:08.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fce800640 1 -- 172.21.15.113:0/1250307451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd00b1030 con 0x7f5fd00c8200 2024-03-04T08:25:08.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fce800640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd00c8200 0x7f5fd00b1410 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c450 tx=0x7f5fc000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:08.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fc7400640 1 -- 172.21.15.113:0/1250307451 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0017070 con 0x7f5fd00c8200 2024-03-04T08:25:08.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fc7400640 1 -- 172.21.15.113:0/1250307451 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc0005cf0 con 0x7f5fd00c8200 2024-03-04T08:25:08.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fd5400640 1 -- 172.21.15.113:0/1250307451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd00cbb40 con 0x7f5fd00c8200 2024-03-04T08:25:08.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fc7400640 1 -- 172.21.15.113:0/1250307451 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00072f0 con 0x7f5fd00c8200 2024-03-04T08:25:08.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fd5400640 1 -- 172.21.15.113:0/1250307451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd00ca570 con 0x7f5fd00c8200 2024-03-04T08:25:08.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fd5400640 1 -- 172.21.15.113:0/1250307451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd0075c00 con 0x7f5fd00c8200 2024-03-04T08:25:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.434+0000 7f5fc7400640 1 -- 172.21.15.113:0/1250307451 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fc001a070 con 0x7f5fd00c8200 2024-03-04T08:25:08.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.438+0000 7f5fc7400640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fac066f60 0x7f5fac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:08.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.438+0000 7f5fc7400640 1 -- 172.21.15.113:0/1250307451 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f5fc0084e80 con 0x7f5fd00c8200 2024-03-04T08:25:08.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.438+0000 7f5fc5e00640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fac066f60 0x7f5fac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:08.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.438+0000 7f5fc5e00640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fac066f60 0x7f5fac069420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80ae000 tx=0x7f5fb80a6040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:08.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.442+0000 7f5fc7400640 1 -- 172.21.15.113:0/1250307451 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5fc0054d00 con 0x7f5fd00c8200 2024-03-04T08:25:08.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.626+0000 7f5fd5400640 1 -- 172.21.15.113:0/1250307451 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fd00c79d0 con 0x7f5fd00c8200 2024-03-04T08:25:08.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fc7400640 1 -- 172.21.15.113:0/1250307451 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f5fc0058da0 con 0x7f5fd00c8200 2024-03-04T08:25:08.627 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:08.628 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:08.628 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:08.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 -- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fac066f60 msgr2=0x7f5fac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fac066f60 0x7f5fac069420 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80ae000 tx=0x7f5fb80a6040 comp rx=0 tx=0).stop 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 -- 172.21.15.113:0/1250307451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd00c8200 msgr2=0x7f5fd00b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd00c8200 0x7f5fd00b1410 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c450 tx=0x7f5fc000c850 comp rx=0 tx=0).stop 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 -- 172.21.15.113:0/1250307451 shutdown_connections 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5fd0152bd0 0x7f5fd014a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fd01527f0 0x7f5fd0150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fac066f60 0x7f5fac069420 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 --2- 172.21.15.113:0/1250307451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fd00c8200 0x7f5fd00b1410 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 -- 172.21.15.113:0/1250307451 >> 172.21.15.113:0/1250307451 conn(0x7f5fd006e820 msgr2=0x7f5fd00c2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 -- 172.21.15.113:0/1250307451 shutdown_connections 2024-03-04T08:25:08.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:08.630+0000 7f5fd5400640 1 -- 172.21.15.113:0/1250307451 wait complete. 2024-03-04T08:25:08.649 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:10.649 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:10.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f38000640 1 Processor -- start 2024-03-04T08:25:10.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f38000640 1 -- start start 2024-03-04T08:25:10.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f38000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5f3015aa40 0x7f5f30154800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:10.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f38000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f300ccf20 0x7f5f30154d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:10.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f38000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3015aed0 con 0x7f5f300ccb40 2024-03-04T08:25:10.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f38000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3015b170 con 0x7f5f300ccf20 2024-03-04T08:25:10.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f38000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3015b410 con 0x7f5f3015aa40 2024-03-04T08:25:10.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f36000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f300ccf20 0x7f5f30154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:10.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f36000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f300ccf20 0x7f5f30154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39970/0 (socket says 172.21.15.113:39970) 2024-03-04T08:25:10.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f36000640 1 -- 172.21.15.113:0/514255450 learned_addr learned my addr 172.21.15.113:0/514255450 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:10.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f34c00640 1 --2- 172.21.15.113:0/514255450 >> v2:172.21.15.136:3300/0 conn(0x7f5f3015aa40 0x7f5f30154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:10.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f36000640 1 -- 172.21.15.113:0/514255450 >> v2:172.21.15.136:3300/0 conn(0x7f5f3015aa40 msgr2=0x7f5f30154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:10.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f36000640 1 --2- 172.21.15.113:0/514255450 >> v2:172.21.15.136:3300/0 conn(0x7f5f3015aa40 0x7f5f30154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:10.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f36000640 1 -- 172.21.15.113:0/514255450 >> v1:172.21.15.136:6789/0 conn(0x7f5f300ccb40 legacy=0x7f5f300c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:10.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.798+0000 7f5f36000640 1 -- 172.21.15.113:0/514255450 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3015b9b0 con 0x7f5f300ccf20 2024-03-04T08:25:10.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f36000640 1 --2- 172.21.15.113:0/514255450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 0x7f5f30154d40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009e50 tx=0x7f5f2801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb1aa49d380e29e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:10.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f2fe00640 1 -- 172.21.15.113:0/514255450 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280b4070 con 0x7f5f300ccf20 2024-03-04T08:25:10.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f2fe00640 1 -- 172.21.15.113:0/514255450 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f280af400 con 0x7f5f300ccf20 2024-03-04T08:25:10.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f2fe00640 1 -- 172.21.15.113:0/514255450 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280af7d0 con 0x7f5f300ccf20 2024-03-04T08:25:10.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f38000640 1 -- 172.21.15.113:0/514255450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 msgr2=0x7f5f30154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:10.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f38000640 1 --2- 172.21.15.113:0/514255450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 0x7f5f30154d40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5f28009e50 tx=0x7f5f2801f1e0 comp rx=0 tx=0).stop 2024-03-04T08:25:10.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f38000640 1 -- 172.21.15.113:0/514255450 shutdown_connections 2024-03-04T08:25:10.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f38000640 1 --2- 172.21.15.113:0/514255450 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 0x7f5f30154d40 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:10.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f38000640 1 --2- 172.21.15.113:0/514255450 >> v2:172.21.15.136:3300/0 conn(0x7f5f3015aa40 0x7f5f30154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:10.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f38000640 1 -- 172.21.15.113:0/514255450 >> 172.21.15.113:0/514255450 conn(0x7f5f3006b060 msgr2=0x7f5f3006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:10.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f38000640 1 -- 172.21.15.113:0/514255450 shutdown_connections 2024-03-04T08:25:10.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.802+0000 7f5f38000640 1 -- 172.21.15.113:0/514255450 wait complete. 2024-03-04T08:25:10.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 Processor -- start 2024-03-04T08:25:10.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 -- start start 2024-03-04T08:25:10.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f300ccb40 0x7f5f300c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:10.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 0x7f5f30153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f300cc290 0x7f5f3014bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300c9fd0 con 0x7f5f300ccf20 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3015e0d0 con 0x7f5f300cc290 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300ca150 con 0x7f5f300ccb40 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f34c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 0x7f5f30153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f34c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 0x7f5f30153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39976/0 (socket says 172.21.15.113:39976) 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f34c00640 1 -- 172.21.15.113:0/498638149 learned_addr learned my addr 172.21.15.113:0/498638149 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f35600640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f300ccb40 0x7f5f300c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f36000640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f300cc290 0x7f5f3014bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f34c00640 1 -- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f300ccb40 msgr2=0x7f5f300c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f34c00640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f300ccb40 0x7f5f300c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f34c00640 1 -- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f300cc290 msgr2=0x7f5f3014bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f34c00640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f300cc290 0x7f5f3014bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f34c00640 1 -- 172.21.15.113:0/498638149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f301f13f0 con 0x7f5f300ccf20 2024-03-04T08:25:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f34c00640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 0x7f5f30153250 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00e8a0 tx=0x7f5f1c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:10.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f2d600640 1 -- 172.21.15.113:0/498638149 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c017070 con 0x7f5f300ccf20 2024-03-04T08:25:10.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f2d600640 1 -- 172.21.15.113:0/498638149 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00c940 con 0x7f5f300ccf20 2024-03-04T08:25:10.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 -- 172.21.15.113:0/498638149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f30152490 con 0x7f5f300ccf20 2024-03-04T08:25:10.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f2d600640 1 -- 172.21.15.113:0/498638149 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00ccf0 con 0x7f5f300ccf20 2024-03-04T08:25:10.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 -- 172.21.15.113:0/498638149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f30152a50 con 0x7f5f300ccf20 2024-03-04T08:25:10.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.806+0000 7f5f38000640 1 -- 172.21.15.113:0/498638149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f30075c00 con 0x7f5f300ccf20 2024-03-04T08:25:10.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.810+0000 7f5f2d600640 1 -- 172.21.15.113:0/498638149 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f1c01a070 con 0x7f5f300ccf20 2024-03-04T08:25:10.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.810+0000 7f5f2d600640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f10066f60 0x7f5f10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:10.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.810+0000 7f5f2d600640 1 -- 172.21.15.113:0/498638149 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f5f1c0859b0 con 0x7f5f300ccf20 2024-03-04T08:25:10.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.810+0000 7f5f35600640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f10066f60 0x7f5f10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:10.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.810+0000 7f5f35600640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f10066f60 0x7f5f10069420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5f24001810 tx=0x7f5f24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:10.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.814+0000 7f5f2d600640 1 -- 172.21.15.113:0/498638149 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5f1c055890 con 0x7f5f300ccf20 2024-03-04T08:25:10.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.998+0000 7f5f38000640 1 -- 172.21.15.113:0/498638149 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f30151bb0 con 0x7f5f300ccf20 2024-03-04T08:25:10.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:10.998+0000 7f5f2d600640 1 -- 172.21.15.113:0/498638149 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f5f1c059930 con 0x7f5f300ccf20 2024-03-04T08:25:10.997 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:10.997 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:10.997 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:10.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 -- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f10066f60 msgr2=0x7f5f10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:10.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f10066f60 0x7f5f10069420 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5f24001810 tx=0x7f5f24009040 comp rx=0 tx=0).stop 2024-03-04T08:25:10.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 -- 172.21.15.113:0/498638149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 msgr2=0x7f5f30153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:10.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 0x7f5f30153250 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00e8a0 tx=0x7f5f1c00eca0 comp rx=0 tx=0).stop 2024-03-04T08:25:10.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 -- 172.21.15.113:0/498638149 shutdown_connections 2024-03-04T08:25:10.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f300cc290 0x7f5f3014bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:10.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f10066f60 0x7f5f10069420 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:10.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300ccf20 0x7f5f30153250 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:11.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 --2- 172.21.15.113:0/498638149 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f300ccb40 0x7f5f300c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:11.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 -- 172.21.15.113:0/498638149 >> 172.21.15.113:0/498638149 conn(0x7f5f3006b060 msgr2=0x7f5f30071e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:11.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 -- 172.21.15.113:0/498638149 shutdown_connections 2024-03-04T08:25:11.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:11.002+0000 7f5f38000640 1 -- 172.21.15.113:0/498638149 wait complete. 2024-03-04T08:25:11.018 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:13.019 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c0e400640 1 Processor -- start 2024-03-04T08:25:13.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c0e400640 1 -- start start 2024-03-04T08:25:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c0e400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3c080c8200 0x7f3c08070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c0e400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c081591f0 con 0x7f3c081527f0 2024-03-04T08:25:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c0e400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c081594d0 con 0x7f3c08152bd0 2024-03-04T08:25:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c0e400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c08071df0 con 0x7f3c080c8200 2024-03-04T08:25:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c07400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3c081527f0 0x7f3c08072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38220/0 (socket says 172.21.15.113:38220) 2024-03-04T08:25:13.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c07400640 1 -- 172.21.15.113:0/3478094611 learned_addr learned my addr 172.21.15.113:0/3478094611 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:13.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3bfea00640 1 --2- 172.21.15.113:0/3478094611 >> v2:172.21.15.136:3300/0 conn(0x7f3c080c8200 0x7f3c08070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276488372 0 0) 0x7f3c081591f0 con 0x7f3c081527f0 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bec003530 con 0x7f3c081527f0 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 501392655 0 0) 0x7f3bec003530 con 0x7f3c081527f0 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 >> v2:172.21.15.136:3300/0 conn(0x7f3c080c8200 msgr2=0x7f3c08070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 --2- 172.21.15.113:0/3478094611 >> v2:172.21.15.136:3300/0 conn(0x7f3c080c8200 0x7f3c08070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 >> v1:172.21.15.136:6789/0 conn(0x7f3c08152bd0 legacy=0x7f3c081587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c080717a0 con 0x7f3c081527f0 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f3bf8001680 con 0x7f3c081527f0 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 >> v1:172.21.15.113:6789/0 conn(0x7f3c081527f0 legacy=0x7f3c08072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bec0039c0 0x7f3bec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3bec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bec0051d0 0x7f3bec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08071df0 con 0x7f3bec004810 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec00c070 con 0x7f3bec0051d0 2024-03-04T08:25:13.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec00c270 con 0x7f3bec0039c0 2024-03-04T08:25:13.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3bfea00640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3bec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:13.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c07400640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bec0039c0 0x7f3bec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:13.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c07e00640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bec0051d0 0x7f3bec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:13.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3bfea00640 1 -- 172.21.15.113:0/3478094611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bec0039c0 msgr2=0x7f3bec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:13.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3bfea00640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3bec0039c0 0x7f3bec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:13.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3bfea00640 1 -- 172.21.15.113:0/3478094611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bec0051d0 msgr2=0x7f3bec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:13.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3bfea00640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bec0051d0 0x7f3bec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:13.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3bfea00640 1 -- 172.21.15.113:0/3478094611 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec003530 con 0x7f3bec004810 2024-03-04T08:25:13.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.166+0000 7f3c07e00640 1 -- 172.21.15.113:0/3478094611 reap_dead start 2024-03-04T08:25:13.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3bfea00640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3bec004c90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009ab0 tx=0x7f3bf001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f0cb8f0dc8c569 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:13.169 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00b2070 con 0x7f3bec004810 2024-03-04T08:25:13.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf00a5530 con 0x7f3bec004810 2024-03-04T08:25:13.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3c06a00640 1 -- 172.21.15.113:0/3478094611 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00096b0 con 0x7f3bec004810 2024-03-04T08:25:13.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3c0e400640 1 -- 172.21.15.113:0/3478094611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 msgr2=0x7f3bec004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:13.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3c0e400640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3bec004c90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0009ab0 tx=0x7f3bf001ca50 comp rx=0 tx=0).stop 2024-03-04T08:25:13.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3c0e400640 1 -- 172.21.15.113:0/3478094611 shutdown_connections 2024-03-04T08:25:13.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3c0e400640 1 --2- 172.21.15.113:0/3478094611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3bec004c90 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:13.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3c0e400640 1 -- 172.21.15.113:0/3478094611 >> 172.21.15.113:0/3478094611 conn(0x7f3c0806e820 msgr2=0x7f3c0806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:13.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3c0e400640 1 -- 172.21.15.113:0/3478094611 shutdown_connections 2024-03-04T08:25:13.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.170+0000 7f3c0e400640 1 -- 172.21.15.113:0/3478094611 wait complete. 2024-03-04T08:25:13.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c0e400640 1 Processor -- start 2024-03-04T08:25:13.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c0e400640 1 -- start start 2024-03-04T08:25:13.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c0e400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3c081f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:13.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c0e400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c080b0250 0x7f3c08157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:13.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c0e400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c081eeab0 0x7f3c081eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:13.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c0e400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08070d40 con 0x7f3bec004810 2024-03-04T08:25:13.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c0e400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c081597b0 con 0x7f3c080b0250 2024-03-04T08:25:13.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c0e400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08070ec0 con 0x7f3c081eeab0 2024-03-04T08:25:13.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c07400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3c081f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:13.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c07400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3c081f2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39996/0 (socket says 172.21.15.113:39996) 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c07400640 1 -- 172.21.15.113:0/1413587809 learned_addr learned my addr 172.21.15.113:0/1413587809 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3bfea00640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c080b0250 0x7f3c08157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c07e00640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c081eeab0 0x7f3c081eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c07400640 1 -- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c081eeab0 msgr2=0x7f3c081eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c07400640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c081eeab0 0x7f3c081eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c07400640 1 -- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c080b0250 msgr2=0x7f3c08157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c07400640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c080b0250 0x7f3c08157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c07400640 1 -- 172.21.15.113:0/1413587809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c080b2170 con 0x7f3bec004810 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3bfea00640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c080b0250 0x7f3c08157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c07400640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3c081f2400 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80119b0 tx=0x7f3bf8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3bffe00640 1 -- 172.21.15.113:0/1413587809 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8010040 con 0x7f3bec004810 2024-03-04T08:25:13.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3bffe00640 1 -- 172.21.15.113:0/1413587809 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf801a070 con 0x7f3bec004810 2024-03-04T08:25:13.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3bffe00640 1 -- 172.21.15.113:0/1413587809 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf800fe20 con 0x7f3bec004810 2024-03-04T08:25:13.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c0e400640 1 -- 172.21.15.113:0/1413587809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c080b2410 con 0x7f3bec004810 2024-03-04T08:25:13.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.174+0000 7f3c0e400640 1 -- 172.21.15.113:0/1413587809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c08155fd0 con 0x7f3bec004810 2024-03-04T08:25:13.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.178+0000 7f3c0e400640 1 -- 172.21.15.113:0/1413587809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c08075c00 con 0x7f3bec004810 2024-03-04T08:25:13.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.178+0000 7f3bffe00640 1 -- 172.21.15.113:0/1413587809 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3bf801f460 con 0x7f3bec004810 2024-03-04T08:25:13.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.178+0000 7f3bffe00640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3be4066f60 0x7f3be4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:13.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.178+0000 7f3bffe00640 1 -- 172.21.15.113:0/1413587809 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f3bf8088f70 con 0x7f3bec004810 2024-03-04T08:25:13.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.178+0000 7f3bfea00640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3be4066f60 0x7f3be4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:13.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.178+0000 7f3bfea00640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3be4066f60 0x7f3be4069420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00af000 tx=0x7f3bf0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:13.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.182+0000 7f3bffe00640 1 -- 172.21.15.113:0/1413587809 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3bf80593b0 con 0x7f3bec004810 2024-03-04T08:25:13.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 -- 172.21.15.113:0/1413587809 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c080c79d0 con 0x7f3bec004810 2024-03-04T08:25:13.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3bffe00640 1 -- 172.21.15.113:0/1413587809 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f3bf8086e20 con 0x7f3bec004810 2024-03-04T08:25:13.373 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:13.373 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:13.373 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 -- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3be4066f60 msgr2=0x7f3be4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3be4066f60 0x7f3be4069420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00af000 tx=0x7f3bf0008040 comp rx=0 tx=0).stop 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 -- 172.21.15.113:0/1413587809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 msgr2=0x7f3c081f2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3c081f2400 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80119b0 tx=0x7f3bf8011e70 comp rx=0 tx=0).stop 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 -- 172.21.15.113:0/1413587809 shutdown_connections 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c081eeab0 0x7f3c081eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c080b0250 0x7f3c08157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3be4066f60 0x7f3be4069420 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 --2- 172.21.15.113:0/1413587809 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bec004810 0x7f3c081f2400 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 -- 172.21.15.113:0/1413587809 >> 172.21.15.113:0/1413587809 conn(0x7f3c0806e820 msgr2=0x7f3c080c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 -- 172.21.15.113:0/1413587809 shutdown_connections 2024-03-04T08:25:13.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:13.374+0000 7f3c0e400640 1 -- 172.21.15.113:0/1413587809 wait complete. 2024-03-04T08:25:13.394 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:15.395 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:15.439 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:25:15.445 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:25:15.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.542+0000 7f9f07800640 1 Processor -- start 2024-03-04T08:25:15.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.542+0000 7f9f07800640 1 -- start start 2024-03-04T08:25:15.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9f07800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9f0015b900 0x7f9f00163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:15.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9f07800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f000c27f0 con 0x7f9f0014e4c0 2024-03-04T08:25:15.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9f07800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f000c2a90 con 0x7f9f000c22e0 2024-03-04T08:25:15.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9f07800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0015ad60 con 0x7f9f0015b900 2024-03-04T08:25:15.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9f04e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9f0014e4c0 0x7f9f00072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38224/0 (socket says 172.21.15.113:38224) 2024-03-04T08:25:15.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9f04e00640 1 -- 172.21.15.113:0/1753271390 learned_addr learned my addr 172.21.15.113:0/1753271390 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:15.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9f05800640 1 --2- 172.21.15.113:0/1753271390 >> v2:172.21.15.113:3300/0 conn(0x7f9f0015b900 0x7f9f00163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:15.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9f05800640 1 -- 172.21.15.113:0/1753271390 >> v1:172.21.15.136:6789/0 conn(0x7f9f000c22e0 legacy=0x7f9f0015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:15.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9f05800640 1 -- 172.21.15.113:0/1753271390 >> v1:172.21.15.113:6789/0 conn(0x7f9f0014e4c0 legacy=0x7f9f00072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:15.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9f05800640 1 -- 172.21.15.113:0/1753271390 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f00070c60 con 0x7f9f0015b900 2024-03-04T08:25:15.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.546+0000 7f9effe00640 1 -- 172.21.15.113:0/1753271390 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956287332 0 0) 0x7f9f000c27f0 con 0x7f9f0014e4c0 2024-03-04T08:25:15.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f05800640 1 --2- 172.21.15.113:0/1753271390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0015b900 0x7f9f00163100 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009d90 tx=0x7f9ef801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6c31a4eaaf7f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:15.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9effe00640 1 -- 172.21.15.113:0/1753271390 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80b4070 con 0x7f9f0015b900 2024-03-04T08:25:15.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9effe00640 1 -- 172.21.15.113:0/1753271390 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef80af600 con 0x7f9f0015b900 2024-03-04T08:25:15.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9effe00640 1 -- 172.21.15.113:0/1753271390 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80af9d0 con 0x7f9f0015b900 2024-03-04T08:25:15.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 -- 172.21.15.113:0/1753271390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0015b900 msgr2=0x7f9f00163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:15.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 --2- 172.21.15.113:0/1753271390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0015b900 0x7f9f00163100 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009d90 tx=0x7f9ef801f150 comp rx=0 tx=0).stop 2024-03-04T08:25:15.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 -- 172.21.15.113:0/1753271390 shutdown_connections 2024-03-04T08:25:15.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 --2- 172.21.15.113:0/1753271390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0015b900 0x7f9f00163100 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:15.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 -- 172.21.15.113:0/1753271390 >> 172.21.15.113:0/1753271390 conn(0x7f9f0006e820 msgr2=0x7f9f0006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:15.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 -- 172.21.15.113:0/1753271390 shutdown_connections 2024-03-04T08:25:15.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 -- 172.21.15.113:0/1753271390 wait complete. 2024-03-04T08:25:15.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 Processor -- start 2024-03-04T08:25:15.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 -- start start 2024-03-04T08:25:15.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f000c22e0 0x7f9f0015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:15.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0014e4c0 0x7f9f001f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:15.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0015b900 0x7f9f001468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:15.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f00071500 con 0x7f9f0014e4c0 2024-03-04T08:25:15.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f00071220 con 0x7f9f0015b900 2024-03-04T08:25:15.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9f07800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0014e8a0 con 0x7f9f000c22e0 2024-03-04T08:25:15.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9ef7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0014e4c0 0x7f9f001f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:15.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9ef7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0014e4c0 0x7f9f001f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40008/0 (socket says 172.21.15.113:40008) 2024-03-04T08:25:15.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.550+0000 7f9ef7e00640 1 -- 172.21.15.113:0/4252512708 learned_addr learned my addr 172.21.15.113:0/4252512708 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:15.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9f04e00640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f000c22e0 0x7f9f0015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:15.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9f05800640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0015b900 0x7f9f001468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:15.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9ef7e00640 1 -- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f000c22e0 msgr2=0x7f9f0015a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:15.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9ef7e00640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f000c22e0 0x7f9f0015a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:15.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9ef7e00640 1 -- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0015b900 msgr2=0x7f9f001468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:15.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9ef7e00640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0015b900 0x7f9f001468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:15.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9ef7e00640 1 -- 172.21.15.113:0/4252512708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f001f1ac0 con 0x7f9f0014e4c0 2024-03-04T08:25:15.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9f04e00640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f000c22e0 0x7f9f0015a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:15.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9ef7e00640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0014e4c0 0x7f9f001f1580 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800c3c0 tx=0x7f9ee800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:15.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9efd600640 1 -- 172.21.15.113:0/4252512708 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8017070 con 0x7f9f0014e4c0 2024-03-04T08:25:15.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9efd600640 1 -- 172.21.15.113:0/4252512708 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee80052d0 con 0x7f9f0014e4c0 2024-03-04T08:25:15.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9f07800640 1 -- 172.21.15.113:0/4252512708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f000b0730 con 0x7f9f0014e4c0 2024-03-04T08:25:15.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9efd600640 1 -- 172.21.15.113:0/4252512708 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8003840 con 0x7f9f0014e4c0 2024-03-04T08:25:15.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9f07800640 1 -- 172.21.15.113:0/4252512708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f000afb70 con 0x7f9f0014e4c0 2024-03-04T08:25:15.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9f07800640 1 -- 172.21.15.113:0/4252512708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f00075c00 con 0x7f9f0014e4c0 2024-03-04T08:25:15.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9efd600640 1 -- 172.21.15.113:0/4252512708 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ee801a070 con 0x7f9f0014e4c0 2024-03-04T08:25:15.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9efd600640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9edc066f90 0x7f9edc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:15.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.554+0000 7f9efd600640 1 -- 172.21.15.113:0/4252512708 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f9ee8084d90 con 0x7f9f0014e4c0 2024-03-04T08:25:15.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.558+0000 7f9f04e00640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9edc066f90 0x7f9edc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:15.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.558+0000 7f9f04e00640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9edc066f90 0x7f9edc069450 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0000f70 tx=0x7f9ef0007c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:15.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.562+0000 7f9efd600640 1 -- 172.21.15.113:0/4252512708 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9ee8005480 con 0x7f9f0014e4c0 2024-03-04T08:25:15.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.750+0000 7f9f07800640 1 -- 172.21.15.113:0/4252512708 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f000b0900 con 0x7f9f0014e4c0 2024-03-04T08:25:15.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.750+0000 7f9efd600640 1 -- 172.21.15.113:0/4252512708 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f9ee8054c10 con 0x7f9f0014e4c0 2024-03-04T08:25:15.750 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:15.750 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:15.751 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:15.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 -- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9edc066f90 msgr2=0x7f9edc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:15.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9edc066f90 0x7f9edc069450 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0000f70 tx=0x7f9ef0007c60 comp rx=0 tx=0).stop 2024-03-04T08:25:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 -- 172.21.15.113:0/4252512708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0014e4c0 msgr2=0x7f9f001f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0014e4c0 0x7f9f001f1580 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800c3c0 tx=0x7f9ee800c880 comp rx=0 tx=0).stop 2024-03-04T08:25:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 -- 172.21.15.113:0/4252512708 shutdown_connections 2024-03-04T08:25:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f0015b900 0x7f9f001468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9edc066f90 0x7f9edc069450 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f0014e4c0 0x7f9f001f1580 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 --2- 172.21.15.113:0/4252512708 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f000c22e0 0x7f9f0015a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 -- 172.21.15.113:0/4252512708 >> 172.21.15.113:0/4252512708 conn(0x7f9f0006e820 msgr2=0x7f9f00072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 -- 172.21.15.113:0/4252512708 shutdown_connections 2024-03-04T08:25:15.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:15.754+0000 7f9f07800640 1 -- 172.21.15.113:0/4252512708 wait complete. 2024-03-04T08:25:15.771 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:25:15.772 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:25:17.774 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6881800640 1 Processor -- start 2024-03-04T08:25:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6881800640 1 -- start start 2024-03-04T08:25:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6881800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f687c151760 0x7f687c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6881800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f687c0c18d0 0x7f687c071f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6881800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c06c290 con 0x7f687c151b40 2024-03-04T08:25:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6881800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687c15aef0 con 0x7f687c0c18d0 2024-03-04T08:25:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6881800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c06ef10 con 0x7f687c151760 2024-03-04T08:25:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6880e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f687c0c18d0 0x7f687c071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:17.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6880e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f687c0c18d0 0x7f687c071f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32836/0 (socket says 172.21.15.113:32836) 2024-03-04T08:25:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6880e00640 1 -- 172.21.15.113:0/415281748 learned_addr learned my addr 172.21.15.113:0/415281748 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f687be00640 1 --2- 172.21.15.113:0/415281748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f687c151760 0x7f687c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6880e00640 1 -- 172.21.15.113:0/415281748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f687c151760 msgr2=0x7f687c0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6880e00640 1 --2- 172.21.15.113:0/415281748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f687c151760 0x7f687c0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6880e00640 1 -- 172.21.15.113:0/415281748 >> v1:172.21.15.136:6789/0 conn(0x7f687c151b40 legacy=0x7f687c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:17.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.922+0000 7f6880e00640 1 -- 172.21.15.113:0/415281748 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c15ba90 con 0x7f687c0c18d0 2024-03-04T08:25:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f6880e00640 1 --2- 172.21.15.113:0/415281748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c0c18d0 0x7f687c071f70 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f6874009e50 tx=0x7f687401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780413c0c74b3c24 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f687b400640 1 -- 172.21.15.113:0/415281748 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68740a5750 con 0x7f687c0c18d0 2024-03-04T08:25:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f687b400640 1 -- 172.21.15.113:0/415281748 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68740a58f0 con 0x7f687c0c18d0 2024-03-04T08:25:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f687b400640 1 -- 172.21.15.113:0/415281748 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68740a5cc0 con 0x7f687c0c18d0 2024-03-04T08:25:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f6881800640 1 -- 172.21.15.113:0/415281748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c0c18d0 msgr2=0x7f687c071f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f6881800640 1 --2- 172.21.15.113:0/415281748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c0c18d0 0x7f687c071f70 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f6874009e50 tx=0x7f687401c9a0 comp rx=0 tx=0).stop 2024-03-04T08:25:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f6881800640 1 -- 172.21.15.113:0/415281748 shutdown_connections 2024-03-04T08:25:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f6881800640 1 --2- 172.21.15.113:0/415281748 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c0c18d0 0x7f687c071f70 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f6881800640 1 --2- 172.21.15.113:0/415281748 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f687c151760 0x7f687c0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f6881800640 1 -- 172.21.15.113:0/415281748 >> 172.21.15.113:0/415281748 conn(0x7f687c06e2a0 msgr2=0x7f687c06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f6881800640 1 -- 172.21.15.113:0/415281748 shutdown_connections 2024-03-04T08:25:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.926+0000 7f6881800640 1 -- 172.21.15.113:0/415281748 wait complete. 2024-03-04T08:25:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6881800640 1 Processor -- start 2024-03-04T08:25:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6881800640 1 -- start start 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6881800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f687c0c18d0 0x7f687c13b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6881800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f687c151760 0x7f687c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6881800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c151b40 0x7f687c13c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6881800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0728c0 con 0x7f687c151b40 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6881800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c15ab80 con 0x7f687c151760 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6881800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0c1e20 con 0x7f687c0c18d0 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6880e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c151b40 0x7f687c13c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6880e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c151b40 0x7f687c13c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32840/0 (socket says 172.21.15.113:32840) 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6880e00640 1 -- 172.21.15.113:0/2622571428 learned_addr learned my addr 172.21.15.113:0/2622571428 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f687be00640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f687c0c18d0 0x7f687c13b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6873400640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f687c151760 0x7f687c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:17.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6880e00640 1 -- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f687c0c18d0 msgr2=0x7f687c13b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6880e00640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f687c0c18d0 0x7f687c13b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6880e00640 1 -- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f687c151760 msgr2=0x7f687c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6880e00640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f687c151760 0x7f687c159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6880e00640 1 -- 172.21.15.113:0/2622571428 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c15a3f0 con 0x7f687c151b40 2024-03-04T08:25:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6880e00640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c151b40 0x7f687c13c550 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f687401a5d0 tx=0x7f6874019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6878c00640 1 -- 172.21.15.113:0/2622571428 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68740bc020 con 0x7f687c151b40 2024-03-04T08:25:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6878c00640 1 -- 172.21.15.113:0/2622571428 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68740b8670 con 0x7f687c151b40 2024-03-04T08:25:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6881800640 1 -- 172.21.15.113:0/2622571428 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c15a6d0 con 0x7f687c151b40 2024-03-04T08:25:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6878c00640 1 -- 172.21.15.113:0/2622571428 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6874009510 con 0x7f687c151b40 2024-03-04T08:25:17.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.930+0000 7f6881800640 1 -- 172.21.15.113:0/2622571428 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c1f4b00 con 0x7f687c151b40 2024-03-04T08:25:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.934+0000 7f6881800640 1 -- 172.21.15.113:0/2622571428 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687c075a80 con 0x7f687c151b40 2024-03-04T08:25:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.934+0000 7f6878c00640 1 -- 172.21.15.113:0/2622571428 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68740bf020 con 0x7f687c151b40 2024-03-04T08:25:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.934+0000 7f6878c00640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6858066f90 0x7f6858069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:17.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.934+0000 7f6878c00640 1 -- 172.21.15.113:0/2622571428 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f68741209a0 con 0x7f687c151b40 2024-03-04T08:25:17.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.934+0000 7f687be00640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6858066f90 0x7f6858069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:17.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.934+0000 7f687be00640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6858066f90 0x7f6858069450 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f686c009af0 tx=0x7f686c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:17.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:17.938+0000 7f6878c00640 1 -- 172.21.15.113:0/2622571428 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6874016620 con 0x7f687c151b40 2024-03-04T08:25:18.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 -- 172.21.15.113:0/2622571428 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f687c0c7850 con 0x7f687c151b40 2024-03-04T08:25:18.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6878c00640 1 -- 172.21.15.113:0/2622571428 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f68740f0f60 con 0x7f687c151b40 2024-03-04T08:25:18.124 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:18.124 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:18.124 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 -- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6858066f90 msgr2=0x7f6858069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6858066f90 0x7f6858069450 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f686c009af0 tx=0x7f686c008040 comp rx=0 tx=0).stop 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 -- 172.21.15.113:0/2622571428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c151b40 msgr2=0x7f687c13c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c151b40 0x7f687c13c550 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f687401a5d0 tx=0x7f6874019740 comp rx=0 tx=0).stop 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 -- 172.21.15.113:0/2622571428 shutdown_connections 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6858066f90 0x7f6858069450 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f687c151b40 0x7f687c13c550 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f687c151760 0x7f687c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 --2- 172.21.15.113:0/2622571428 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f687c0c18d0 0x7f687c13b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 -- 172.21.15.113:0/2622571428 >> 172.21.15.113:0/2622571428 conn(0x7f687c06e2a0 msgr2=0x7f687c0c65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 -- 172.21.15.113:0/2622571428 shutdown_connections 2024-03-04T08:25:18.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:18.126+0000 7f6881800640 1 -- 172.21.15.113:0/2622571428 wait complete. 2024-03-04T08:25:18.145 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:20.146 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:20.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feba3c00640 1 Processor -- start 2024-03-04T08:25:20.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feba3c00640 1 -- start start 2024-03-04T08:25:20.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feba3c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 0x7feb9c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:20.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feba3c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb9c0b0ed0 0x7feb9c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:20.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feba3c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9c1586c0 con 0x7feb9c06ddf0 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feba3c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb9c06c210 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feba3c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c06c4f0 con 0x7feb9c1582e0 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feb93e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb9c0b0ed0 0x7feb9c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feb93e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb9c0b0ed0 0x7feb9c0ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32844/0 (socket says 172.21.15.113:32844) 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feb93e00640 1 -- 172.21.15.113:0/64541145 learned_addr learned my addr 172.21.15.113:0/64541145 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feba1200640 1 --2- 172.21.15.113:0/64541145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 0x7feb9c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feb93e00640 1 -- 172.21.15.113:0/64541145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 msgr2=0x7feb9c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feb93e00640 1 --2- 172.21.15.113:0/64541145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 0x7feb9c0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feb93e00640 1 -- 172.21.15.113:0/64541145 >> v1:172.21.15.136:6789/0 conn(0x7feb9c06ddf0 legacy=0x7feb9c15af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feb93e00640 1 -- 172.21.15.113:0/64541145 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c06f6a0 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.294+0000 7feba1200640 1 --2- 172.21.15.113:0/64541145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 0x7feb9c0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:20.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feb93e00640 1 --2- 172.21.15.113:0/64541145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 0x7feb9c0ce2d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7feb84009870 tx=0x7feb8401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ebe6f621634e22 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:20.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feb9be00640 1 -- 172.21.15.113:0/64541145 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb840b2070 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feb9be00640 1 -- 172.21.15.113:0/64541145 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb840a9070 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feb9be00640 1 -- 172.21.15.113:0/64541145 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb840ad880 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feba3c00640 1 -- 172.21.15.113:0/64541145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 msgr2=0x7feb9c0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:20.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feba3c00640 1 --2- 172.21.15.113:0/64541145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 0x7feb9c0ce2d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7feb84009870 tx=0x7feb8401c560 comp rx=0 tx=0).stop 2024-03-04T08:25:20.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feba3c00640 1 -- 172.21.15.113:0/64541145 shutdown_connections 2024-03-04T08:25:20.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feba3c00640 1 --2- 172.21.15.113:0/64541145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 0x7feb9c0ce2d0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:20.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feba3c00640 1 --2- 172.21.15.113:0/64541145 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 0x7feb9c0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:20.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feba3c00640 1 -- 172.21.15.113:0/64541145 >> 172.21.15.113:0/64541145 conn(0x7feb9c066db0 msgr2=0x7feb9c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:20.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feba3c00640 1 -- 172.21.15.113:0/64541145 shutdown_connections 2024-03-04T08:25:20.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.298+0000 7feba3c00640 1 -- 172.21.15.113:0/64541145 wait complete. 2024-03-04T08:25:20.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba3c00640 1 Processor -- start 2024-03-04T08:25:20.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba3c00640 1 -- start start 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba3c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c06ddf0 0x7feb9c1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba3c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 0x7feb9c1f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba3c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 0x7feb9c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba3c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c167ed0 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba3c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c167d50 con 0x7feb9c06ddf0 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba3c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9c168050 con 0x7feb9c1582e0 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb93e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 0x7feb9c1f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb93e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 0x7feb9c1f6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32850/0 (socket says 172.21.15.113:32850) 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb93e00640 1 -- 172.21.15.113:0/1584722410 learned_addr learned my addr 172.21.15.113:0/1584722410 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba1200640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c06ddf0 0x7feb9c1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba1c00640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 0x7feb9c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb93e00640 1 -- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 msgr2=0x7feb9c159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb93e00640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 0x7feb9c159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb93e00640 1 -- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c06ddf0 msgr2=0x7feb9c1f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb93e00640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c06ddf0 0x7feb9c1f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb93e00640 1 -- 172.21.15.113:0/1584722410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c1ecee0 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb93e00640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 0x7feb9c1f6cd0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7feb840b0040 tx=0x7feb84019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb99600640 1 -- 172.21.15.113:0/1584722410 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb840bb070 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb99600640 1 -- 172.21.15.113:0/1584722410 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb840add90 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba3c00640 1 -- 172.21.15.113:0/1584722410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9c1f72d0 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feb99600640 1 -- 172.21.15.113:0/1584722410 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb840b2040 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.302+0000 7feba3c00640 1 -- 172.21.15.113:0/1584722410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9c1f7890 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.306+0000 7feba3c00640 1 -- 172.21.15.113:0/1584722410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb9c068eb0 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.306+0000 7feb99600640 1 -- 172.21.15.113:0/1584722410 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb840be070 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.306+0000 7feb99600640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb78066f60 0x7feb78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:20.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.306+0000 7feb99600640 1 -- 172.21.15.113:0/1584722410 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7feb8411ea40 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.306+0000 7feba1200640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb78066f60 0x7feb78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:20.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.306+0000 7feba1200640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb78066f60 0x7feb78069420 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0045f0 tx=0x7feb8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:20.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.310+0000 7feb99600640 1 -- 172.21.15.113:0/1584722410 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7feb84016150 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.498+0000 7feba3c00640 1 -- 172.21.15.113:0/1584722410 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb9c075c00 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.498+0000 7feb99600640 1 -- 172.21.15.113:0/1584722410 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7feb840ee8c0 con 0x7feb9c0b0ed0 2024-03-04T08:25:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:20.498 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:20.498 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:20.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 -- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb78066f60 msgr2=0x7feb78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:20.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb78066f60 0x7feb78069420 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0045f0 tx=0x7feb8c008040 comp rx=0 tx=0).stop 2024-03-04T08:25:20.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 -- 172.21.15.113:0/1584722410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 msgr2=0x7feb9c1f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:20.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 0x7feb9c1f6cd0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7feb840b0040 tx=0x7feb84019300 comp rx=0 tx=0).stop 2024-03-04T08:25:20.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 -- 172.21.15.113:0/1584722410 shutdown_connections 2024-03-04T08:25:20.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb9c1582e0 0x7feb9c159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:20.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb78066f60 0x7feb78069420 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:20.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb9c0b0ed0 0x7feb9c1f6cd0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:20.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 --2- 172.21.15.113:0/1584722410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb9c06ddf0 0x7feb9c1f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:20.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 -- 172.21.15.113:0/1584722410 >> 172.21.15.113:0/1584722410 conn(0x7feb9c066db0 msgr2=0x7feb9c156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:20.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 -- 172.21.15.113:0/1584722410 shutdown_connections 2024-03-04T08:25:20.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:20.502+0000 7feba3c00640 1 -- 172.21.15.113:0/1584722410 wait complete. 2024-03-04T08:25:20.519 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:22.520 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:22.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.666+0000 7f5ff6200640 1 Processor -- start 2024-03-04T08:25:22.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.666+0000 7f5ff6200640 1 -- start start 2024-03-04T08:25:22.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.666+0000 7f5ff6200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ff00b20b0 0x7f5ff0157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:22.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.666+0000 7f5ff6200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5ff0157c50 0x7f5ff006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:22.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.666+0000 7f5ff6200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5ff00c85f0 0x7f5ff006b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:22.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.666+0000 7f5ff6200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff00c1ec0 con 0x7f5ff0157c50 2024-03-04T08:25:22.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.666+0000 7f5ff6200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff00c9b30 con 0x7f5ff00c85f0 2024-03-04T08:25:22.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.666+0000 7f5ff6200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00c12a0 con 0x7f5ff00b20b0 2024-03-04T08:25:22.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5feea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5ff0157c50 0x7f5ff006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:22.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5feea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5ff0157c50 0x7f5ff006b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32858/0 (socket says 172.21.15.113:32858) 2024-03-04T08:25:22.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5feea00640 1 -- 172.21.15.113:0/1892303303 learned_addr learned my addr 172.21.15.113:0/1892303303 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:22.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5fefe00640 1 --2- 172.21.15.113:0/1892303303 >> v2:172.21.15.136:3300/0 conn(0x7f5ff00c85f0 0x7f5ff006b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:22.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5fef400640 1 --2- 172.21.15.113:0/1892303303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ff00b20b0 0x7f5ff0157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:22.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5feea00640 1 -- 172.21.15.113:0/1892303303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ff00b20b0 msgr2=0x7f5ff0157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:22.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5feea00640 1 --2- 172.21.15.113:0/1892303303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ff00b20b0 0x7f5ff0157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5feea00640 1 -- 172.21.15.113:0/1892303303 >> v2:172.21.15.136:3300/0 conn(0x7f5ff00c85f0 msgr2=0x7f5ff006b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:22.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5feea00640 1 --2- 172.21.15.113:0/1892303303 >> v2:172.21.15.136:3300/0 conn(0x7f5ff00c85f0 0x7f5ff006b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5feea00640 1 -- 172.21.15.113:0/1892303303 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00c1580 con 0x7f5ff0157c50 2024-03-04T08:25:22.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5feea00640 1 --2- 172.21.15.113:0/1892303303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff0157c50 0x7f5ff006b060 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0070a20 tx=0x7f5fdc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44557e07805fe16 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:22.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5fee000640 1 -- 172.21.15.113:0/1892303303 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0ab430 con 0x7f5ff0157c50 2024-03-04T08:25:22.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5fee000640 1 -- 172.21.15.113:0/1892303303 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc0b0070 con 0x7f5ff0157c50 2024-03-04T08:25:22.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.670+0000 7f5fee000640 1 -- 172.21.15.113:0/1892303303 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0ab7f0 con 0x7f5ff0157c50 2024-03-04T08:25:22.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 -- 172.21.15.113:0/1892303303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff0157c50 msgr2=0x7f5ff006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:22.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 --2- 172.21.15.113:0/1892303303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff0157c50 0x7f5ff006b060 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0070a20 tx=0x7f5fdc01c4f0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 -- 172.21.15.113:0/1892303303 shutdown_connections 2024-03-04T08:25:22.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 --2- 172.21.15.113:0/1892303303 >> v2:172.21.15.136:3300/0 conn(0x7f5ff00c85f0 0x7f5ff006b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 --2- 172.21.15.113:0/1892303303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff0157c50 0x7f5ff006b060 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 --2- 172.21.15.113:0/1892303303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ff00b20b0 0x7f5ff0157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 -- 172.21.15.113:0/1892303303 >> 172.21.15.113:0/1892303303 conn(0x7f5ff006e820 msgr2=0x7f5ff006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:22.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 -- 172.21.15.113:0/1892303303 shutdown_connections 2024-03-04T08:25:22.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 -- 172.21.15.113:0/1892303303 wait complete. 2024-03-04T08:25:22.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 Processor -- start 2024-03-04T08:25:22.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 -- start start 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff00b20b0 0x7f5ff0151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ff00c85f0 0x7f5ff014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ff0157c50 0x7f5ff0142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00c1b80 con 0x7f5ff00b20b0 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0156ec0 con 0x7f5ff00c85f0 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5ff6200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00c80c0 con 0x7f5ff0157c50 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5fef400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff00b20b0 0x7f5ff0151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5fef400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff00b20b0 0x7f5ff0151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32874/0 (socket says 172.21.15.113:32874) 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5fef400640 1 -- 172.21.15.113:0/1379504933 learned_addr learned my addr 172.21.15.113:0/1379504933 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5feea00640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ff00c85f0 0x7f5ff014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:22.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5fefe00640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ff0157c50 0x7f5ff0142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5fef400640 1 -- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ff0157c50 msgr2=0x7f5ff0142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5fef400640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ff0157c50 0x7f5ff0142d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5fef400640 1 -- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ff00c85f0 msgr2=0x7f5ff014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5fef400640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ff00c85f0 0x7f5ff014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5fef400640 1 -- 172.21.15.113:0/1379504933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff014b690 con 0x7f5ff00b20b0 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5feea00640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ff00c85f0 0x7f5ff014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.674+0000 7f5fef400640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff00b20b0 0x7f5ff0151830 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000d800 tx=0x7f5fe000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.678+0000 7f5fe7400640 1 -- 172.21.15.113:0/1379504933 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0018030 con 0x7f5ff00b20b0 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.678+0000 7f5fe7400640 1 -- 172.21.15.113:0/1379504933 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe00096a0 con 0x7f5ff00b20b0 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.678+0000 7f5fe7400640 1 -- 172.21.15.113:0/1379504933 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0009870 con 0x7f5ff00b20b0 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.678+0000 7f5ff6200640 1 -- 172.21.15.113:0/1379504933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff01f1b20 con 0x7f5ff00b20b0 2024-03-04T08:25:22.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.678+0000 7f5ff6200640 1 -- 172.21.15.113:0/1379504933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff0153b50 con 0x7f5ff00b20b0 2024-03-04T08:25:22.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.678+0000 7f5ff6200640 1 -- 172.21.15.113:0/1379504933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff0075c00 con 0x7f5ff00b20b0 2024-03-04T08:25:22.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.678+0000 7f5fe7400640 1 -- 172.21.15.113:0/1379504933 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fe001b070 con 0x7f5ff00b20b0 2024-03-04T08:25:22.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.678+0000 7f5fe7400640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fd0066f60 0x7f5fd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:22.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.678+0000 7f5fe7400640 1 -- 172.21.15.113:0/1379504933 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f5fe0085d10 con 0x7f5ff00b20b0 2024-03-04T08:25:22.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.678+0000 7f5feea00640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fd0066f60 0x7f5fd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:22.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.682+0000 7f5feea00640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fd0066f60 0x7f5fd0069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0af000 tx=0x7f5fdc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:22.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.686+0000 7f5fe7400640 1 -- 172.21.15.113:0/1379504933 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5fe008b020 con 0x7f5ff00b20b0 2024-03-04T08:25:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.870+0000 7f5ff6200640 1 -- 172.21.15.113:0/1379504933 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ff00c83d0 con 0x7f5ff00b20b0 2024-03-04T08:25:22.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.870+0000 7f5fe7400640 1 -- 172.21.15.113:0/1379504933 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f5fe0056270 con 0x7f5ff00b20b0 2024-03-04T08:25:22.870 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:22.870 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:22.870 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:22.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 -- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fd0066f60 msgr2=0x7f5fd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:22.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fd0066f60 0x7f5fd0069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0af000 tx=0x7f5fdc008040 comp rx=0 tx=0).stop 2024-03-04T08:25:22.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 -- 172.21.15.113:0/1379504933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff00b20b0 msgr2=0x7f5ff0151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:22.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff00b20b0 0x7f5ff0151830 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000d800 tx=0x7f5fe000dcc0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 -- 172.21.15.113:0/1379504933 shutdown_connections 2024-03-04T08:25:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5ff0157c50 0x7f5ff0142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ff00c85f0 0x7f5ff014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fd0066f60 0x7f5fd0069420 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 --2- 172.21.15.113:0/1379504933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff00b20b0 0x7f5ff0151830 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 -- 172.21.15.113:0/1379504933 >> 172.21.15.113:0/1379504933 conn(0x7f5ff006e820 msgr2=0x7f5ff0156650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 -- 172.21.15.113:0/1379504933 shutdown_connections 2024-03-04T08:25:22.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:22.874+0000 7f5ff6200640 1 -- 172.21.15.113:0/1379504933 wait complete. 2024-03-04T08:25:22.891 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:24.892 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc628600640 1 Processor -- start 2024-03-04T08:25:25.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc628600640 1 -- start start 2024-03-04T08:25:25.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc628600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc620152ac0 0x7fc6200c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc628600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc6200c3ff0 0x7fc6200ce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc628600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6200c4440 con 0x7fc6200ca8c0 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc628600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62006f900 con 0x7fc620152ac0 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc628600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc620159900 con 0x7fc6200c3ff0 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc625c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc620152ac0 0x7fc6200c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc625200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc6200ca8c0 0x7fc620158920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:55256/0 (socket says 172.21.15.113:55256) 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc625200640 1 -- 172.21.15.113:0/2580110515 learned_addr learned my addr 172.21.15.113:0/2580110515 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc625c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc620152ac0 0x7fc6200c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32886/0 (socket says 172.21.15.113:32886) 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc626600640 1 --2- 172.21.15.113:0/2580110515 >> v2:172.21.15.136:3300/0 conn(0x7fc6200c3ff0 0x7fc6200ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc625c00640 1 -- 172.21.15.113:0/2580110515 >> v2:172.21.15.136:3300/0 conn(0x7fc6200c3ff0 msgr2=0x7fc6200ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc625c00640 1 --2- 172.21.15.113:0/2580110515 >> v2:172.21.15.136:3300/0 conn(0x7fc6200c3ff0 0x7fc6200ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc625c00640 1 -- 172.21.15.113:0/2580110515 >> v1:172.21.15.113:6789/0 conn(0x7fc6200ca8c0 legacy=0x7fc620158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:25.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.041+0000 7fc625c00640 1 -- 172.21.15.113:0/2580110515 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc620159340 con 0x7fc620152ac0 2024-03-04T08:25:25.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc625c00640 1 --2- 172.21.15.113:0/2580110515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc620152ac0 0x7fc6200c8c00 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fc6200b4530 tx=0x7fc61401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a248c85a27a41bd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:25.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc61fe00640 1 -- 172.21.15.113:0/2580110515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6140b0070 con 0x7fc620152ac0 2024-03-04T08:25:25.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc61fe00640 1 -- 172.21.15.113:0/2580110515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6140ab400 con 0x7fc620152ac0 2024-03-04T08:25:25.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc61fe00640 1 -- 172.21.15.113:0/2580110515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6140ab7d0 con 0x7fc620152ac0 2024-03-04T08:25:25.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc628600640 1 -- 172.21.15.113:0/2580110515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc620152ac0 msgr2=0x7fc6200c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:25.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc628600640 1 --2- 172.21.15.113:0/2580110515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc620152ac0 0x7fc6200c8c00 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fc6200b4530 tx=0x7fc61401c480 comp rx=0 tx=0).stop 2024-03-04T08:25:25.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc628600640 1 -- 172.21.15.113:0/2580110515 shutdown_connections 2024-03-04T08:25:25.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc628600640 1 --2- 172.21.15.113:0/2580110515 >> v2:172.21.15.136:3300/0 conn(0x7fc6200c3ff0 0x7fc6200ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:25.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc628600640 1 --2- 172.21.15.113:0/2580110515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc620152ac0 0x7fc6200c8c00 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:25.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc628600640 1 -- 172.21.15.113:0/2580110515 >> 172.21.15.113:0/2580110515 conn(0x7fc620066db0 msgr2=0x7fc6200671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:25.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc628600640 1 -- 172.21.15.113:0/2580110515 shutdown_connections 2024-03-04T08:25:25.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc628600640 1 -- 172.21.15.113:0/2580110515 wait complete. 2024-03-04T08:25:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc628600640 1 Processor -- start 2024-03-04T08:25:25.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.045+0000 7fc628600640 1 -- start start 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc628600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6200c3ff0 0x7fc6200b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc628600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6200ca8c0 0x7fc6200cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc628600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc620152ac0 0x7fc62014cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc628600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc620159030 con 0x7fc6200ca8c0 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc628600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6200c3ad0 con 0x7fc620152ac0 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc628600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc620152ea0 con 0x7fc6200c3ff0 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc625200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6200ca8c0 0x7fc6200cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc625200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6200ca8c0 0x7fc6200cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:32902/0 (socket says 172.21.15.113:32902) 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc625200640 1 -- 172.21.15.113:0/1446780277 learned_addr learned my addr 172.21.15.113:0/1446780277 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc626600640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc620152ac0 0x7fc62014cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc625c00640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6200c3ff0 0x7fc6200b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc625200640 1 -- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6200c3ff0 msgr2=0x7fc6200b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:25.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc625200640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6200c3ff0 0x7fc6200b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc625200640 1 -- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc620152ac0 msgr2=0x7fc62014cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc625200640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc620152ac0 0x7fc62014cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc625200640 1 -- 172.21.15.113:0/1446780277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62006d670 con 0x7fc6200ca8c0 2024-03-04T08:25:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc626600640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc620152ac0 0x7fc62014cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc625200640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6200ca8c0 0x7fc6200cd2b0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00c420 tx=0x7fc60c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc61d600640 1 -- 172.21.15.113:0/1446780277 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c017070 con 0x7fc6200ca8c0 2024-03-04T08:25:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc61d600640 1 -- 172.21.15.113:0/1446780277 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60c005cf0 con 0x7fc6200ca8c0 2024-03-04T08:25:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc61d600640 1 -- 172.21.15.113:0/1446780277 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0072f0 con 0x7fc6200ca8c0 2024-03-04T08:25:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc628600640 1 -- 172.21.15.113:0/1446780277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62014e000 con 0x7fc6200ca8c0 2024-03-04T08:25:25.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc628600640 1 -- 172.21.15.113:0/1446780277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62014e5c0 con 0x7fc6200ca8c0 2024-03-04T08:25:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc628600640 1 -- 172.21.15.113:0/1446780277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc620075c00 con 0x7fc6200ca8c0 2024-03-04T08:25:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.049+0000 7fc61d600640 1 -- 172.21.15.113:0/1446780277 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc60c01a070 con 0x7fc6200ca8c0 2024-03-04T08:25:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.053+0000 7fc61d600640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc600066f90 0x7fc600069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.053+0000 7fc61d600640 1 -- 172.21.15.113:0/1446780277 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fc60c084ec0 con 0x7fc6200ca8c0 2024-03-04T08:25:25.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.053+0000 7fc625c00640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc600066f90 0x7fc600069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:25.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.053+0000 7fc625c00640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc600066f90 0x7fc600069450 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc6140ad000 tx=0x7fc614008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:25.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.057+0000 7fc61d600640 1 -- 172.21.15.113:0/1446780277 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc60c055390 con 0x7fc6200ca8c0 2024-03-04T08:25:25.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.245+0000 7fc628600640 1 -- 172.21.15.113:0/1446780277 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6200c9170 con 0x7fc6200ca8c0 2024-03-04T08:25:25.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.245+0000 7fc61d600640 1 -- 172.21.15.113:0/1446780277 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fc60c054c90 con 0x7fc6200ca8c0 2024-03-04T08:25:25.248 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:25.248 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:25.249 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:25.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 -- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc600066f90 msgr2=0x7fc600069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:25.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc600066f90 0x7fc600069450 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc6140ad000 tx=0x7fc614008040 comp rx=0 tx=0).stop 2024-03-04T08:25:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 -- 172.21.15.113:0/1446780277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6200ca8c0 msgr2=0x7fc6200cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6200ca8c0 0x7fc6200cd2b0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fc60c00c420 tx=0x7fc60c00c8e0 comp rx=0 tx=0).stop 2024-03-04T08:25:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 -- 172.21.15.113:0/1446780277 shutdown_connections 2024-03-04T08:25:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc620152ac0 0x7fc62014cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc600066f90 0x7fc600069450 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc6200ca8c0 0x7fc6200cd2b0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 --2- 172.21.15.113:0/1446780277 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc6200c3ff0 0x7fc6200b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 -- 172.21.15.113:0/1446780277 >> 172.21.15.113:0/1446780277 conn(0x7fc620066db0 msgr2=0x7fc6201ea7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 -- 172.21.15.113:0/1446780277 shutdown_connections 2024-03-04T08:25:25.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:25.249+0000 7fc628600640 1 -- 172.21.15.113:0/1446780277 wait complete. 2024-03-04T08:25:25.268 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:27.269 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd43000640 1 Processor -- start 2024-03-04T08:25:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd43000640 1 -- start start 2024-03-04T08:25:27.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd43000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbd3c0c8200 0x7fbd3c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:27.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd43000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c152bd0 0x7fbd3c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:27.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd43000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3c1591f0 con 0x7fbd3c1527f0 2024-03-04T08:25:27.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd43000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3c071df0 con 0x7fbd3c0c8200 2024-03-04T08:25:27.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd43000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c1597b0 con 0x7fbd3c152bd0 2024-03-04T08:25:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd33e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbd3c0c8200 0x7fbd3c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd41000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c152bd0 0x7fbd3c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd33e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbd3c0c8200 0x7fbd3c070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:33356/0 (socket says 172.21.15.113:33356) 2024-03-04T08:25:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd33e00640 1 -- 172.21.15.113:0/2947029152 learned_addr learned my addr 172.21.15.113:0/2947029152 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd41000640 1 -- 172.21.15.113:0/2947029152 >> v2:172.21.15.136:3300/0 conn(0x7fbd3c0c8200 msgr2=0x7fbd3c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd41000640 1 --2- 172.21.15.113:0/2947029152 >> v2:172.21.15.136:3300/0 conn(0x7fbd3c0c8200 0x7fbd3c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd41000640 1 -- 172.21.15.113:0/2947029152 >> v1:172.21.15.136:6789/0 conn(0x7fbd3c1527f0 legacy=0x7fbd3c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd41000640 1 -- 172.21.15.113:0/2947029152 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c1594d0 con 0x7fbd3c152bd0 2024-03-04T08:25:27.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.417+0000 7fbd33e00640 1 --2- 172.21.15.113:0/2947029152 >> v2:172.21.15.136:3300/0 conn(0x7fbd3c0c8200 0x7fbd3c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:25:27.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd41000640 1 --2- 172.21.15.113:0/2947029152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c152bd0 0x7fbd3c1587d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009d90 tx=0x7fbd3401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7eaba5087347abc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd3b400640 1 -- 172.21.15.113:0/2947029152 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340b2070 con 0x7fbd3c152bd0 2024-03-04T08:25:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd3b400640 1 -- 172.21.15.113:0/2947029152 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd340a5920 con 0x7fbd3c152bd0 2024-03-04T08:25:27.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd3b400640 1 -- 172.21.15.113:0/2947029152 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340a5cf0 con 0x7fbd3c152bd0 2024-03-04T08:25:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd43000640 1 -- 172.21.15.113:0/2947029152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c152bd0 msgr2=0x7fbd3c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd43000640 1 --2- 172.21.15.113:0/2947029152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c152bd0 0x7fbd3c1587d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009d90 tx=0x7fbd3401c970 comp rx=0 tx=0).stop 2024-03-04T08:25:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd43000640 1 -- 172.21.15.113:0/2947029152 shutdown_connections 2024-03-04T08:25:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd43000640 1 --2- 172.21.15.113:0/2947029152 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c152bd0 0x7fbd3c1587d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd43000640 1 --2- 172.21.15.113:0/2947029152 >> v2:172.21.15.136:3300/0 conn(0x7fbd3c0c8200 0x7fbd3c070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:27.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd43000640 1 -- 172.21.15.113:0/2947029152 >> 172.21.15.113:0/2947029152 conn(0x7fbd3c06e820 msgr2=0x7fbd3c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd43000640 1 -- 172.21.15.113:0/2947029152 shutdown_connections 2024-03-04T08:25:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.421+0000 7fbd43000640 1 -- 172.21.15.113:0/2947029152 wait complete. 2024-03-04T08:25:27.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd43000640 1 Processor -- start 2024-03-04T08:25:27.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd43000640 1 -- start start 2024-03-04T08:25:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd43000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd3c0c8200 0x7fbd3c1f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd43000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c1527f0 0x7fbd3c157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:27.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd43000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd3c152bd0 0x7fbd3c153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd43000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c1654f0 con 0x7fbd3c152bd0 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd43000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c165370 con 0x7fbd3c0c8200 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd43000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c165670 con 0x7fbd3c1527f0 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd41000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd3c152bd0 0x7fbd3c153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd41000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd3c152bd0 0x7fbd3c153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44446/0 (socket says 172.21.15.113:44446) 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd41000640 1 -- 172.21.15.113:0/771080492 learned_addr learned my addr 172.21.15.113:0/771080492 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd3be00640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd3c0c8200 0x7fbd3c1f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd33e00640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c1527f0 0x7fbd3c157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd41000640 1 -- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c1527f0 msgr2=0x7fbd3c157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd41000640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c1527f0 0x7fbd3c157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd41000640 1 -- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd3c0c8200 msgr2=0x7fbd3c1f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd41000640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd3c0c8200 0x7fbd3c1f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd41000640 1 -- 172.21.15.113:0/771080492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c1500b0 con 0x7fbd3c152bd0 2024-03-04T08:25:27.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd41000640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd3c152bd0 0x7fbd3c153200 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fbd340b0040 tx=0x7fbd34019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:27.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd38c00640 1 -- 172.21.15.113:0/771080492 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340b2050 con 0x7fbd3c152bd0 2024-03-04T08:25:27.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd38c00640 1 -- 172.21.15.113:0/771080492 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd340be070 con 0x7fbd3c152bd0 2024-03-04T08:25:27.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd38c00640 1 -- 172.21.15.113:0/771080492 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340ada90 con 0x7fbd3c152bd0 2024-03-04T08:25:27.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd43000640 1 -- 172.21.15.113:0/771080492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c150350 con 0x7fbd3c152bd0 2024-03-04T08:25:27.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.425+0000 7fbd43000640 1 -- 172.21.15.113:0/771080492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c150910 con 0x7fbd3c152bd0 2024-03-04T08:25:27.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.429+0000 7fbd43000640 1 -- 172.21.15.113:0/771080492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c075c00 con 0x7fbd3c152bd0 2024-03-04T08:25:27.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.429+0000 7fbd38c00640 1 -- 172.21.15.113:0/771080492 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd340adc30 con 0x7fbd3c152bd0 2024-03-04T08:25:27.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.429+0000 7fbd38c00640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd18066f60 0x7fbd18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:27.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.429+0000 7fbd38c00640 1 -- 172.21.15.113:0/771080492 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fbd34120c30 con 0x7fbd3c152bd0 2024-03-04T08:25:27.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.429+0000 7fbd3be00640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd18066f60 0x7fbd18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:27.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.429+0000 7fbd3be00640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd18066f60 0x7fbd18069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c001810 tx=0x7fbd2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:27.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.433+0000 7fbd38c00640 1 -- 172.21.15.113:0/771080492 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbd34016560 con 0x7fbd3c152bd0 2024-03-04T08:25:27.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 -- 172.21.15.113:0/771080492 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd3c0c79d0 con 0x7fbd3c152bd0 2024-03-04T08:25:27.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd38c00640 1 -- 172.21.15.113:0/771080492 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fbd340f0b40 con 0x7fbd3c152bd0 2024-03-04T08:25:27.621 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:27.621 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:27.621 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:27.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 -- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd18066f60 msgr2=0x7fbd18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd18066f60 0x7fbd18069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c001810 tx=0x7fbd2c009040 comp rx=0 tx=0).stop 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 -- 172.21.15.113:0/771080492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd3c152bd0 msgr2=0x7fbd3c153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd3c152bd0 0x7fbd3c153200 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fbd340b0040 tx=0x7fbd34019710 comp rx=0 tx=0).stop 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 -- 172.21.15.113:0/771080492 shutdown_connections 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbd18066f60 0x7fbd18069420 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbd3c152bd0 0x7fbd3c153200 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbd3c1527f0 0x7fbd3c157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 --2- 172.21.15.113:0/771080492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd3c0c8200 0x7fbd3c1f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 -- 172.21.15.113:0/771080492 >> 172.21.15.113:0/771080492 conn(0x7fbd3c06e820 msgr2=0x7fbd3c0c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 -- 172.21.15.113:0/771080492 shutdown_connections 2024-03-04T08:25:27.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:27.621+0000 7fbd43000640 1 -- 172.21.15.113:0/771080492 wait complete. 2024-03-04T08:25:27.642 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:25:27.642 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:25:29.643 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:29.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff466c00640 1 Processor -- start 2024-03-04T08:25:29.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff466c00640 1 -- start start 2024-03-04T08:25:29.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff466c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4600c85f0 0x7ff46006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:29.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff466c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 0x7ff460161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff466c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4600c1ec0 con 0x7ff4600b20b0 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff466c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4600c9b30 con 0x7ff4600c85f0 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff466c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600c12a0 con 0x7ff46006b5a0 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff457400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4600c85f0 0x7ff46006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff457400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff4600c85f0 0x7ff46006b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44454/0 (socket says 172.21.15.113:44454) 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff457400640 1 -- 172.21.15.113:0/1455774328 learned_addr learned my addr 172.21.15.113:0/1455774328 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff464c00640 1 --2- 172.21.15.113:0/1455774328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 0x7ff460161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff457400640 1 -- 172.21.15.113:0/1455774328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 msgr2=0x7ff460161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff457400640 1 --2- 172.21.15.113:0/1455774328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 0x7ff460161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff457400640 1 -- 172.21.15.113:0/1455774328 >> v1:172.21.15.136:6789/0 conn(0x7ff4600b20b0 legacy=0x7ff460157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff457400640 1 -- 172.21.15.113:0/1455774328 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600c1580 con 0x7ff4600c85f0 2024-03-04T08:25:29.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.789+0000 7ff464c00640 1 --2- 172.21.15.113:0/1455774328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 0x7ff460161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:29.793 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff457400640 1 --2- 172.21.15.113:0/1455774328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 0x7ff46006b060 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ff448009870 tx=0x7ff44801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1013c8bdc5719311 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff45f400640 1 -- 172.21.15.113:0/1455774328 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4480b5070 con 0x7ff4600c85f0 2024-03-04T08:25:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff45f400640 1 -- 172.21.15.113:0/1455774328 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4480a8a50 con 0x7ff4600c85f0 2024-03-04T08:25:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff45f400640 1 -- 172.21.15.113:0/1455774328 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4480a8e20 con 0x7ff4600c85f0 2024-03-04T08:25:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff466c00640 1 -- 172.21.15.113:0/1455774328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 msgr2=0x7ff46006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff466c00640 1 --2- 172.21.15.113:0/1455774328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 0x7ff46006b060 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ff448009870 tx=0x7ff44801ed50 comp rx=0 tx=0).stop 2024-03-04T08:25:29.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff466c00640 1 -- 172.21.15.113:0/1455774328 shutdown_connections 2024-03-04T08:25:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff466c00640 1 --2- 172.21.15.113:0/1455774328 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 0x7ff460161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff466c00640 1 --2- 172.21.15.113:0/1455774328 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 0x7ff46006b060 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff466c00640 1 -- 172.21.15.113:0/1455774328 >> 172.21.15.113:0/1455774328 conn(0x7ff46006e820 msgr2=0x7ff46006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff466c00640 1 -- 172.21.15.113:0/1455774328 shutdown_connections 2024-03-04T08:25:29.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.793+0000 7ff466c00640 1 -- 172.21.15.113:0/1455774328 wait complete. 2024-03-04T08:25:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff466c00640 1 Processor -- start 2024-03-04T08:25:29.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff466c00640 1 -- start start 2024-03-04T08:25:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff466c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 0x7ff460072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:29.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff466c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4600b20b0 0x7ff4601f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff466c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 0x7ff4601f1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff466c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff460157e20 con 0x7ff4600c85f0 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff466c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4601571f0 con 0x7ff4600b20b0 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff466c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600c2400 con 0x7ff46006b5a0 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff464c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 0x7ff4601f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff464c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 0x7ff4601f1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44470/0 (socket says 172.21.15.113:44470) 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff464c00640 1 -- 172.21.15.113:0/2853638110 learned_addr learned my addr 172.21.15.113:0/2853638110 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff457400640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4600b20b0 0x7ff4601f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff45fe00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 0x7ff460072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff464c00640 1 -- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 msgr2=0x7ff460072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff464c00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 0x7ff460072560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff464c00640 1 -- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4600b20b0 msgr2=0x7ff4601f0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:29.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff464c00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4600b20b0 0x7ff4601f0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:29.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff464c00640 1 -- 172.21.15.113:0/2853638110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600c80c0 con 0x7ff4600c85f0 2024-03-04T08:25:29.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff464c00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 0x7ff4601f1390 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ff45800e840 tx=0x7ff45800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:29.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff45cc00640 1 -- 172.21.15.113:0/2853638110 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458017070 con 0x7ff4600c85f0 2024-03-04T08:25:29.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff45cc00640 1 -- 172.21.15.113:0/2853638110 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45800c6d0 con 0x7ff4600c85f0 2024-03-04T08:25:29.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff466c00640 1 -- 172.21.15.113:0/2853638110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4600b13a0 con 0x7ff4600c85f0 2024-03-04T08:25:29.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff45cc00640 1 -- 172.21.15.113:0/2853638110 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800f040 con 0x7ff4600c85f0 2024-03-04T08:25:29.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.797+0000 7ff466c00640 1 -- 172.21.15.113:0/2853638110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4601f43a0 con 0x7ff4600c85f0 2024-03-04T08:25:29.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.801+0000 7ff466c00640 1 -- 172.21.15.113:0/2853638110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff460075c00 con 0x7ff4600c85f0 2024-03-04T08:25:29.800 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.801+0000 7ff45cc00640 1 -- 172.21.15.113:0/2853638110 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff45801a070 con 0x7ff4600c85f0 2024-03-04T08:25:29.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.801+0000 7ff45cc00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff43c066f60 0x7ff43c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:29.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.801+0000 7ff45cc00640 1 -- 172.21.15.113:0/2853638110 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7ff4580856e0 con 0x7ff4600c85f0 2024-03-04T08:25:29.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.801+0000 7ff45fe00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff43c066f60 0x7ff43c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:29.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.801+0000 7ff45fe00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff43c066f60 0x7ff43c069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff450001ea0 tx=0x7ff450009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:29.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.805+0000 7ff45cc00640 1 -- 172.21.15.113:0/2853638110 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff4580554d0 con 0x7ff4600c85f0 2024-03-04T08:25:29.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.993+0000 7ff466c00640 1 -- 172.21.15.113:0/2853638110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff46006f050 con 0x7ff4600c85f0 2024-03-04T08:25:29.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.993+0000 7ff45cc00640 1 -- 172.21.15.113:0/2853638110 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7ff458059570 con 0x7ff4600c85f0 2024-03-04T08:25:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:29.995 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:29.995 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:29.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 -- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff43c066f60 msgr2=0x7ff43c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:29.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff43c066f60 0x7ff43c069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff450001ea0 tx=0x7ff450009040 comp rx=0 tx=0).stop 2024-03-04T08:25:29.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 -- 172.21.15.113:0/2853638110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 msgr2=0x7ff4601f1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:29.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 0x7ff4601f1390 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ff45800e840 tx=0x7ff45800ed00 comp rx=0 tx=0).stop 2024-03-04T08:25:29.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 -- 172.21.15.113:0/2853638110 shutdown_connections 2024-03-04T08:25:29.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff43c066f60 0x7ff43c069420 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:29.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4600c85f0 0x7ff4601f1390 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:29.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4600b20b0 0x7ff4601f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:29.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 --2- 172.21.15.113:0/2853638110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff46006b5a0 0x7ff460072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:29.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 -- 172.21.15.113:0/2853638110 >> 172.21.15.113:0/2853638110 conn(0x7ff46006e820 msgr2=0x7ff4600cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:29.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 -- 172.21.15.113:0/2853638110 shutdown_connections 2024-03-04T08:25:29.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:29.997+0000 7ff466c00640 1 -- 172.21.15.113:0/2853638110 wait complete. 2024-03-04T08:25:30.016 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:32.017 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:32.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83b5400640 1 Processor -- start 2024-03-04T08:25:32.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83b5400640 1 -- start start 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83b5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83b00c22e0 0x7f83b015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83b5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83b00c2d80 0x7f83b015b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83b5400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b00c27f0 con 0x7f83b014e4c0 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83b5400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b015ad60 con 0x7f83b00c2d80 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83b5400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b00c2a90 con 0x7f83b00c22e0 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83af200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83b00c2d80 0x7f83b015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83af200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83b00c2d80 0x7f83b015b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44484/0 (socket says 172.21.15.113:44484) 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83af200640 1 -- 172.21.15.113:0/2656089758 learned_addr learned my addr 172.21.15.113:0/2656089758 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.165+0000 7f83a5e00640 1 --2- 172.21.15.113:0/2656089758 >> v2:172.21.15.136:3300/0 conn(0x7f83b00c22e0 0x7f83b015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83af200640 1 -- 172.21.15.113:0/2656089758 >> v2:172.21.15.136:3300/0 conn(0x7f83b00c22e0 msgr2=0x7f83b015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83af200640 1 --2- 172.21.15.113:0/2656089758 >> v2:172.21.15.136:3300/0 conn(0x7f83b00c22e0 0x7f83b015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:32.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83af200640 1 -- 172.21.15.113:0/2656089758 >> v1:172.21.15.136:6789/0 conn(0x7f83b014e4c0 legacy=0x7f83b0072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:32.168 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83af200640 1 -- 172.21.15.113:0/2656089758 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b0070c60 con 0x7f83b00c2d80 2024-03-04T08:25:32.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83af200640 1 --2- 172.21.15.113:0/2656089758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b00c2d80 0x7f83b015b730 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f83b00cd230 tx=0x7f83a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da3ee8ed4777893 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:32.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83ade00640 1 -- 172.21.15.113:0/2656089758 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80ab430 con 0x7f83b00c2d80 2024-03-04T08:25:32.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83ade00640 1 -- 172.21.15.113:0/2656089758 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a80b0070 con 0x7f83b00c2d80 2024-03-04T08:25:32.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83ade00640 1 -- 172.21.15.113:0/2656089758 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80ab7f0 con 0x7f83b00c2d80 2024-03-04T08:25:32.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83b5400640 1 -- 172.21.15.113:0/2656089758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b00c2d80 msgr2=0x7f83b015b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:32.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83b5400640 1 --2- 172.21.15.113:0/2656089758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b00c2d80 0x7f83b015b730 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f83b00cd230 tx=0x7f83a801c520 comp rx=0 tx=0).stop 2024-03-04T08:25:32.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83b5400640 1 -- 172.21.15.113:0/2656089758 shutdown_connections 2024-03-04T08:25:32.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83b5400640 1 --2- 172.21.15.113:0/2656089758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b00c2d80 0x7f83b015b730 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:32.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83b5400640 1 --2- 172.21.15.113:0/2656089758 >> v2:172.21.15.136:3300/0 conn(0x7f83b00c22e0 0x7f83b015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:32.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83b5400640 1 -- 172.21.15.113:0/2656089758 >> 172.21.15.113:0/2656089758 conn(0x7f83b006e820 msgr2=0x7f83b006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:32.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.169+0000 7f83b5400640 1 -- 172.21.15.113:0/2656089758 shutdown_connections 2024-03-04T08:25:32.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83b5400640 1 -- 172.21.15.113:0/2656089758 wait complete. 2024-03-04T08:25:32.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83b5400640 1 Processor -- start 2024-03-04T08:25:32.172 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83b5400640 1 -- start start 2024-03-04T08:25:32.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83b5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83b00c22e0 0x7f83b0152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83b5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b00c2d80 0x7f83b00cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83b5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b014e4c0 0x7f83b0152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83b5400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b00aede0 con 0x7f83b014e4c0 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83b5400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b014e8a0 con 0x7f83b00c2d80 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83b5400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b0073390 con 0x7f83b00c22e0 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83af200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b014e4c0 0x7f83b0152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83af200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b014e4c0 0x7f83b0152fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44492/0 (socket says 172.21.15.113:44492) 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83af200640 1 -- 172.21.15.113:0/2230028638 learned_addr learned my addr 172.21.15.113:0/2230028638 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83a5e00640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b00c2d80 0x7f83b00cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83ae800640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83b00c22e0 0x7f83b0152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83af200640 1 -- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83b00c22e0 msgr2=0x7f83b0152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83af200640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83b00c22e0 0x7f83b0152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:32.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83af200640 1 -- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b00c2d80 msgr2=0x7f83b00cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:32.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83af200640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b00c2d80 0x7f83b00cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:32.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83af200640 1 -- 172.21.15.113:0/2230028638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b01517c0 con 0x7f83b014e4c0 2024-03-04T08:25:32.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83af200640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b014e4c0 0x7f83b0152fe0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f83a80ad040 tx=0x7f83a801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:32.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83a7400640 1 -- 172.21.15.113:0/2230028638 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80be5f0 con 0x7f83b014e4c0 2024-03-04T08:25:32.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83a7400640 1 -- 172.21.15.113:0/2230028638 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a80be7b0 con 0x7f83b014e4c0 2024-03-04T08:25:32.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83b5400640 1 -- 172.21.15.113:0/2230028638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b01f4db0 con 0x7f83b014e4c0 2024-03-04T08:25:32.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.173+0000 7f83a7400640 1 -- 172.21.15.113:0/2230028638 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80beba0 con 0x7f83b014e4c0 2024-03-04T08:25:32.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.177+0000 7f83b5400640 1 -- 172.21.15.113:0/2230028638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b01f1c90 con 0x7f83b014e4c0 2024-03-04T08:25:32.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.177+0000 7f83b5400640 1 -- 172.21.15.113:0/2230028638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b01f17a0 con 0x7f83b014e4c0 2024-03-04T08:25:32.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.177+0000 7f83a7400640 1 -- 172.21.15.113:0/2230028638 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83a80b43f0 con 0x7f83b014e4c0 2024-03-04T08:25:32.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.177+0000 7f83a7400640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f838c066f90 0x7f838c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:32.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.177+0000 7f83a7400640 1 -- 172.21.15.113:0/2230028638 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f83a80a8070 con 0x7f83b014e4c0 2024-03-04T08:25:32.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.177+0000 7f83ae800640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f838c066f90 0x7f838c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:32.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.177+0000 7f83ae800640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f838c066f90 0x7f838c069450 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f83a0001650 tx=0x7f83a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:32.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.181+0000 7f83a7400640 1 -- 172.21.15.113:0/2230028638 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f83a8121e20 con 0x7f83b014e4c0 2024-03-04T08:25:32.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.369+0000 7f83b5400640 1 -- 172.21.15.113:0/2230028638 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83b0075c00 con 0x7f83b014e4c0 2024-03-04T08:25:32.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.369+0000 7f83a7400640 1 -- 172.21.15.113:0/2230028638 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f83a8016110 con 0x7f83b014e4c0 2024-03-04T08:25:32.370 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:32.370 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:32.370 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:32.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 -- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f838c066f90 msgr2=0x7f838c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:32.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f838c066f90 0x7f838c069450 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f83a0001650 tx=0x7f83a0009040 comp rx=0 tx=0).stop 2024-03-04T08:25:32.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 -- 172.21.15.113:0/2230028638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b014e4c0 msgr2=0x7f83b0152fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:32.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b014e4c0 0x7f83b0152fe0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f83a80ad040 tx=0x7f83a801ad60 comp rx=0 tx=0).stop 2024-03-04T08:25:32.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 -- 172.21.15.113:0/2230028638 shutdown_connections 2024-03-04T08:25:32.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f838c066f90 0x7f838c069450 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:32.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83b014e4c0 0x7f83b0152fe0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:32.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83b00c2d80 0x7f83b00cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:32.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 --2- 172.21.15.113:0/2230028638 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83b00c22e0 0x7f83b0152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:32.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 -- 172.21.15.113:0/2230028638 >> 172.21.15.113:0/2230028638 conn(0x7f83b006e820 msgr2=0x7f83b015b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:32.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 -- 172.21.15.113:0/2230028638 shutdown_connections 2024-03-04T08:25:32.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:32.373+0000 7f83b5400640 1 -- 172.21.15.113:0/2230028638 wait complete. 2024-03-04T08:25:32.392 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:34.392 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:34.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe52de00640 1 Processor -- start 2024-03-04T08:25:34.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe52de00640 1 -- start start 2024-03-04T08:25:34.538 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe52de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5280c22e0 0x7fe52815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe52de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c2d80 0x7fe52815b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe52de00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5280c27f0 con 0x7fe52814e4c0 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe52de00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52815ad60 con 0x7fe5280c22e0 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe52de00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280c2a90 con 0x7fe5280c2d80 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe526000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5280c22e0 0x7fe52815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe526a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe52814e4c0 0x7fe528072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41350/0 (socket says 172.21.15.113:41350) 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe526000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5280c22e0 0x7fe52815b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44494/0 (socket says 172.21.15.113:44494) 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe526a00640 1 -- 172.21.15.113:0/2844334480 learned_addr learned my addr 172.21.15.113:0/2844334480 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.537+0000 7fe527400640 1 --2- 172.21.15.113:0/2844334480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c2d80 0x7fe52815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe526000640 1 -- 172.21.15.113:0/2844334480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c2d80 msgr2=0x7fe52815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe526000640 1 --2- 172.21.15.113:0/2844334480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c2d80 0x7fe52815b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe526000640 1 -- 172.21.15.113:0/2844334480 >> v1:172.21.15.113:6789/0 conn(0x7fe52814e4c0 legacy=0x7fe528072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe526000640 1 -- 172.21.15.113:0/2844334480 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe528070c60 con 0x7fe5280c22e0 2024-03-04T08:25:34.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe525600640 1 -- 172.21.15.113:0/2844334480 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1606009963 0 0) 0x7fe5280c27f0 con 0x7fe52814e4c0 2024-03-04T08:25:34.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe526000640 1 --2- 172.21.15.113:0/2844334480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c22e0 0x7fe52815b1f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fe5140097c0 tx=0x7fe51401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9762c4574d41e33 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:34.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe525600640 1 -- 172.21.15.113:0/2844334480 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5140b5070 con 0x7fe5280c22e0 2024-03-04T08:25:34.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe525600640 1 -- 172.21.15.113:0/2844334480 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5140a8990 con 0x7fe5280c22e0 2024-03-04T08:25:34.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe525600640 1 -- 172.21.15.113:0/2844334480 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5140a8d60 con 0x7fe5280c22e0 2024-03-04T08:25:34.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe52de00640 1 -- 172.21.15.113:0/2844334480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c22e0 msgr2=0x7fe52815b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:34.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe52de00640 1 --2- 172.21.15.113:0/2844334480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c22e0 0x7fe52815b1f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fe5140097c0 tx=0x7fe51401ec90 comp rx=0 tx=0).stop 2024-03-04T08:25:34.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe52de00640 1 -- 172.21.15.113:0/2844334480 shutdown_connections 2024-03-04T08:25:34.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe52de00640 1 --2- 172.21.15.113:0/2844334480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c2d80 0x7fe52815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:34.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe52de00640 1 --2- 172.21.15.113:0/2844334480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c22e0 0x7fe52815b1f0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:34.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe52de00640 1 -- 172.21.15.113:0/2844334480 >> 172.21.15.113:0/2844334480 conn(0x7fe52806e820 msgr2=0x7fe52806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:34.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe52de00640 1 -- 172.21.15.113:0/2844334480 shutdown_connections 2024-03-04T08:25:34.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.541+0000 7fe52de00640 1 -- 172.21.15.113:0/2844334480 wait complete. 2024-03-04T08:25:34.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe52de00640 1 Processor -- start 2024-03-04T08:25:34.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe52de00640 1 -- start start 2024-03-04T08:25:34.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe52de00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c22e0 0x7fe5280c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:34.546 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe52de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c2d80 0x7fe5281f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe52de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe52814e4c0 0x7fe52815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe52de00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe528071500 con 0x7fe5280c2d80 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe52de00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe528071220 con 0x7fe52814e4c0 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe52de00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52814e8a0 con 0x7fe5280c22e0 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c2d80 0x7fe5281f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c2d80 0x7fe5281f3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44502/0 (socket says 172.21.15.113:44502) 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526000640 1 -- 172.21.15.113:0/3462533456 learned_addr learned my addr 172.21.15.113:0/3462533456 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe527400640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe52814e4c0 0x7fe52815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526a00640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c22e0 0x7fe5280c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526000640 1 -- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c22e0 msgr2=0x7fe5280c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526000640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c22e0 0x7fe5280c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526000640 1 -- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe52814e4c0 msgr2=0x7fe52815a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526000640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe52814e4c0 0x7fe52815a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526000640 1 -- 172.21.15.113:0/3462533456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe528153800 con 0x7fe5280c2d80 2024-03-04T08:25:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe527400640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe52814e4c0 0x7fe52815a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526a00640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c22e0 0x7fe5280c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe526000640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c2d80 0x7fe5281f3a00 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe51401c8c0 tx=0x7fe51401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe51f400640 1 -- 172.21.15.113:0/3462533456 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5140bf070 con 0x7fe5280c2d80 2024-03-04T08:25:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe51f400640 1 -- 172.21.15.113:0/3462533456 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5140b07a0 con 0x7fe5280c2d80 2024-03-04T08:25:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe51f400640 1 -- 172.21.15.113:0/3462533456 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5140b5040 con 0x7fe5280c2d80 2024-03-04T08:25:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.545+0000 7fe52de00640 1 -- 172.21.15.113:0/3462533456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5281f3f40 con 0x7fe5280c2d80 2024-03-04T08:25:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.549+0000 7fe52de00640 1 -- 172.21.15.113:0/3462533456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe528153a70 con 0x7fe5280c2d80 2024-03-04T08:25:34.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.549+0000 7fe52de00640 1 -- 172.21.15.113:0/3462533456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe528075c00 con 0x7fe5280c2d80 2024-03-04T08:25:34.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.549+0000 7fe51f400640 1 -- 172.21.15.113:0/3462533456 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5140c2070 con 0x7fe5280c2d80 2024-03-04T08:25:34.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.549+0000 7fe51f400640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe508066f90 0x7fe508069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:34.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.549+0000 7fe51f400640 1 -- 172.21.15.113:0/3462533456 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fe514122250 con 0x7fe5280c2d80 2024-03-04T08:25:34.549 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.549+0000 7fe526a00640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe508066f90 0x7fe508069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.549+0000 7fe526a00640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe508066f90 0x7fe508069450 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe518005fd0 tx=0x7fe518005b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:34.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.553+0000 7fe51f400640 1 -- 172.21.15.113:0/3462533456 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe5140187f0 con 0x7fe5280c2d80 2024-03-04T08:25:34.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 -- 172.21.15.113:0/3462533456 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe528155190 con 0x7fe5280c2d80 2024-03-04T08:25:34.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe51f400640 1 -- 172.21.15.113:0/3462533456 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fe5140f27b0 con 0x7fe5280c2d80 2024-03-04T08:25:34.741 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:34.741 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:34.741 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 -- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe508066f90 msgr2=0x7fe508069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe508066f90 0x7fe508069450 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe518005fd0 tx=0x7fe518005b80 comp rx=0 tx=0).stop 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 -- 172.21.15.113:0/3462533456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c2d80 msgr2=0x7fe5281f3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c2d80 0x7fe5281f3a00 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe51401c8c0 tx=0x7fe51401ba30 comp rx=0 tx=0).stop 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 -- 172.21.15.113:0/3462533456 shutdown_connections 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe52814e4c0 0x7fe52815a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe508066f90 0x7fe508069450 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5280c2d80 0x7fe5281f3a00 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 --2- 172.21.15.113:0/3462533456 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe5280c22e0 0x7fe5280c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.741+0000 7fe52de00640 1 -- 172.21.15.113:0/3462533456 >> 172.21.15.113:0/3462533456 conn(0x7fe52806e820 msgr2=0x7fe52814abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.745+0000 7fe52de00640 1 -- 172.21.15.113:0/3462533456 shutdown_connections 2024-03-04T08:25:34.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:34.745+0000 7fe52de00640 1 -- 172.21.15.113:0/3462533456 wait complete. 2024-03-04T08:25:34.761 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:36.762 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:36.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0213000640 1 Processor -- start 2024-03-04T08:25:36.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0213000640 1 -- start start 2024-03-04T08:25:36.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0213000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f020c152c60 0x7f020c15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:36.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0213000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f020c15c280 0x7f020c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:36.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0213000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c072d80 con 0x7f020c0c7a80 2024-03-04T08:25:36.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0213000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c06b060 con 0x7f020c15c280 2024-03-04T08:25:36.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0213000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c073510 con 0x7f020c152c60 2024-03-04T08:25:36.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0203400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f020c15c280 0x7f020c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:36.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0211000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f020c0c7a80 0x7f020c158770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41358/0 (socket says 172.21.15.113:41358) 2024-03-04T08:25:36.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0211000640 1 -- 172.21.15.113:0/2447004430 learned_addr learned my addr 172.21.15.113:0/2447004430 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:36.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f020be00640 1 --2- 172.21.15.113:0/2447004430 >> v2:172.21.15.136:3300/0 conn(0x7f020c152c60 0x7f020c15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:36.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0203400640 1 -- 172.21.15.113:0/2447004430 >> v2:172.21.15.136:3300/0 conn(0x7f020c152c60 msgr2=0x7f020c15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:36.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0203400640 1 --2- 172.21.15.113:0/2447004430 >> v2:172.21.15.136:3300/0 conn(0x7f020c152c60 0x7f020c15bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:36.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0203400640 1 -- 172.21.15.113:0/2447004430 >> v1:172.21.15.113:6789/0 conn(0x7f020c0c7a80 legacy=0x7f020c158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:36.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f020b400640 1 -- 172.21.15.113:0/2447004430 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892999196 0 0) 0x7f020c072d80 con 0x7f020c0c7a80 2024-03-04T08:25:36.911 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.909+0000 7f0203400640 1 -- 172.21.15.113:0/2447004430 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c06b2b0 con 0x7f020c15c280 2024-03-04T08:25:36.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f0203400640 1 --2- 172.21.15.113:0/2447004430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c15c280 0x7f020c0cd010 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f020c0c4370 tx=0x7f01f401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ac4c58a4bf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:36.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f020b400640 1 -- 172.21.15.113:0/2447004430 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40b1070 con 0x7f020c15c280 2024-03-04T08:25:36.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f020b400640 1 -- 172.21.15.113:0/2447004430 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f40ac620 con 0x7f020c15c280 2024-03-04T08:25:36.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f020b400640 1 -- 172.21.15.113:0/2447004430 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f40ac9f0 con 0x7f020c15c280 2024-03-04T08:25:36.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f0213000640 1 -- 172.21.15.113:0/2447004430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c15c280 msgr2=0x7f020c0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:36.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f0213000640 1 --2- 172.21.15.113:0/2447004430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c15c280 0x7f020c0cd010 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f020c0c4370 tx=0x7f01f401c3c0 comp rx=0 tx=0).stop 2024-03-04T08:25:36.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f0213000640 1 -- 172.21.15.113:0/2447004430 shutdown_connections 2024-03-04T08:25:36.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f0213000640 1 --2- 172.21.15.113:0/2447004430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c15c280 0x7f020c0cd010 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:36.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f0213000640 1 --2- 172.21.15.113:0/2447004430 >> v2:172.21.15.136:3300/0 conn(0x7f020c152c60 0x7f020c15bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:36.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f0213000640 1 -- 172.21.15.113:0/2447004430 >> 172.21.15.113:0/2447004430 conn(0x7f020c0671b0 msgr2=0x7f020c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:36.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f0213000640 1 -- 172.21.15.113:0/2447004430 shutdown_connections 2024-03-04T08:25:36.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.913+0000 7f0213000640 1 -- 172.21.15.113:0/2447004430 wait complete. 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 Processor -- start 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 -- start start 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c0c7a80 0x7f020c1f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f020c152c60 0x7f020c1f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f020c15c280 0x7f020c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0c8ac0 con 0x7f020c0c7a80 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0731d0 con 0x7f020c15c280 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0c8c40 con 0x7f020c152c60 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f020be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c0c7a80 0x7f020c1f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f020be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c0c7a80 0x7f020c1f7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44518/0 (socket says 172.21.15.113:44518) 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f020be00640 1 -- 172.21.15.113:0/2500420710 learned_addr learned my addr 172.21.15.113:0/2500420710 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0211000640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f020c15c280 0x7f020c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:36.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0203400640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f020c152c60 0x7f020c1f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f020be00640 1 -- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f020c152c60 msgr2=0x7f020c1f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f020be00640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f020c152c60 0x7f020c1f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f020be00640 1 -- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f020c15c280 msgr2=0x7f020c15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f020be00640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f020c15c280 0x7f020c15a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f020be00640 1 -- 172.21.15.113:0/2500420710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c068470 con 0x7f020c0c7a80 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f020be00640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c0c7a80 0x7f020c1f7b20 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00e930 tx=0x7f01fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0208c00640 1 -- 172.21.15.113:0/2500420710 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc017070 con 0x7f020c0c7a80 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0208c00640 1 -- 172.21.15.113:0/2500420710 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc00ca00 con 0x7f020c0c7a80 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0208c00640 1 -- 172.21.15.113:0/2500420710 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00cdb0 con 0x7f020c0c7a80 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 -- 172.21.15.113:0/2500420710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c158380 con 0x7f020c0c7a80 2024-03-04T08:25:36.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 -- 172.21.15.113:0/2500420710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c1f7040 con 0x7f020c0c7a80 2024-03-04T08:25:36.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.917+0000 7f0213000640 1 -- 172.21.15.113:0/2500420710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c075c00 con 0x7f020c0c7a80 2024-03-04T08:25:36.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.921+0000 7f0208c00640 1 -- 172.21.15.113:0/2500420710 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01fc01a070 con 0x7f020c0c7a80 2024-03-04T08:25:36.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.921+0000 7f0208c00640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01e8066f60 0x7f01e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:36.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.921+0000 7f0208c00640 1 -- 172.21.15.113:0/2500420710 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f01fc085ec0 con 0x7f020c0c7a80 2024-03-04T08:25:36.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.921+0000 7f0203400640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01e8066f60 0x7f01e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:36.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.921+0000 7f0203400640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01e8066f60 0x7f01e8069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f01f40ae000 tx=0x7f01f40a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:36.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:36.925+0000 7f0208c00640 1 -- 172.21.15.113:0/2500420710 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f01fc0564e0 con 0x7f020c0c7a80 2024-03-04T08:25:37.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.117+0000 7f0213000640 1 -- 172.21.15.113:0/2500420710 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f020c1563c0 con 0x7f020c0c7a80 2024-03-04T08:25:37.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0208c00640 1 -- 172.21.15.113:0/2500420710 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f01fc055de0 con 0x7f020c0c7a80 2024-03-04T08:25:37.119 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:37.120 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:37.120 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:37.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 -- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01e8066f60 msgr2=0x7f01e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01e8066f60 0x7f01e8069420 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f01f40ae000 tx=0x7f01f40a7000 comp rx=0 tx=0).stop 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 -- 172.21.15.113:0/2500420710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c0c7a80 msgr2=0x7f020c1f7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c0c7a80 0x7f020c1f7b20 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00e930 tx=0x7f01fc00edf0 comp rx=0 tx=0).stop 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 -- 172.21.15.113:0/2500420710 shutdown_connections 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f020c15c280 0x7f020c15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f020c152c60 0x7f020c1f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f01e8066f60 0x7f01e8069420 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 --2- 172.21.15.113:0/2500420710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f020c0c7a80 0x7f020c1f7b20 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 -- 172.21.15.113:0/2500420710 >> 172.21.15.113:0/2500420710 conn(0x7f020c0671b0 msgr2=0x7f020c151d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 -- 172.21.15.113:0/2500420710 shutdown_connections 2024-03-04T08:25:37.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:37.121+0000 7f0213000640 1 -- 172.21.15.113:0/2500420710 wait complete. 2024-03-04T08:25:37.141 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:39.141 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:39.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04d2c00640 1 Processor -- start 2024-03-04T08:25:39.287 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04d2c00640 1 -- start start 2024-03-04T08:25:39.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04d2c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f04cc152bd0 0x7f04cc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:39.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04d2c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04cc1591f0 con 0x7f04cc0c8200 2024-03-04T08:25:39.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04d2c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04cc1594d0 con 0x7f04cc1527f0 2024-03-04T08:25:39.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04d2c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04cc071df0 con 0x7f04cc152bd0 2024-03-04T08:25:39.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04c3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f04cc0c8200 0x7f04cc1587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41964/0 (socket says 172.21.15.113:41964) 2024-03-04T08:25:39.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04c3400640 1 -- 172.21.15.113:0/1389929354 learned_addr learned my addr 172.21.15.113:0/1389929354 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:39.289 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04d0c00640 1 --2- 172.21.15.113:0/1389929354 >> v2:172.21.15.136:3300/0 conn(0x7f04cc152bd0 0x7f04cc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:39.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659819051 0 0) 0x7f04cc1591f0 con 0x7f04cc0c8200 2024-03-04T08:25:39.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04b0003530 con 0x7f04cc0c8200 2024-03-04T08:25:39.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1924583668 0 0) 0x7f04b0003530 con 0x7f04cc0c8200 2024-03-04T08:25:39.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 >> v2:172.21.15.136:3300/0 conn(0x7f04cc152bd0 msgr2=0x7f04cc070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:39.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 --2- 172.21.15.113:0/1389929354 >> v2:172.21.15.136:3300/0 conn(0x7f04cc152bd0 0x7f04cc070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 >> v1:172.21.15.136:6789/0 conn(0x7f04cc1527f0 legacy=0x7f04cc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:39.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc0717a0 con 0x7f04cc0c8200 2024-03-04T08:25:39.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f04b4001680 con 0x7f04cc0c8200 2024-03-04T08:25:39.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 >> v1:172.21.15.113:6789/0 conn(0x7f04cc0c8200 legacy=0x7f04cc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:39.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04b00039c0 0x7f04b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b00051d0 0x7f04b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc071df0 con 0x7f04b0004810 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b000c070 con 0x7f04b00051d0 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b000c270 con 0x7f04b00039c0 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04c3400640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04d0c00640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b00051d0 0x7f04b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04cbe00640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04b00039c0 0x7f04b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04c3400640 1 -- 172.21.15.113:0/1389929354 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04b00039c0 msgr2=0x7f04b0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04c3400640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04b00039c0 0x7f04b0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04c3400640 1 -- 172.21.15.113:0/1389929354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b00051d0 msgr2=0x7f04b000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04c3400640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04b00051d0 0x7f04b000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04c3400640 1 -- 172.21.15.113:0/1389929354 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b0003530 con 0x7f04b0004810 2024-03-04T08:25:39.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.289+0000 7f04d0c00640 1 -- 172.21.15.113:0/1389929354 reap_dead start 2024-03-04T08:25:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04c3400640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04b0004c90 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f04b400c190 tx=0x7f04b401f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756ffdf7c1571265 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4007d00 con 0x7f04b0004810 2024-03-04T08:25:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b4005b00 con 0x7f04b0004810 2024-03-04T08:25:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04cb400640 1 -- 172.21.15.113:0/1389929354 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b4007070 con 0x7f04b0004810 2024-03-04T08:25:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04d2c00640 1 -- 172.21.15.113:0/1389929354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 msgr2=0x7f04b0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04d2c00640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04b0004c90 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f04b400c190 tx=0x7f04b401f350 comp rx=0 tx=0).stop 2024-03-04T08:25:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04d2c00640 1 -- 172.21.15.113:0/1389929354 shutdown_connections 2024-03-04T08:25:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04d2c00640 1 --2- 172.21.15.113:0/1389929354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04b0004c90 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04d2c00640 1 -- 172.21.15.113:0/1389929354 >> 172.21.15.113:0/1389929354 conn(0x7f04cc06e820 msgr2=0x7f04cc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04d2c00640 1 -- 172.21.15.113:0/1389929354 shutdown_connections 2024-03-04T08:25:39.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.293+0000 7f04d2c00640 1 -- 172.21.15.113:0/1389929354 wait complete. 2024-03-04T08:25:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d2c00640 1 Processor -- start 2024-03-04T08:25:39.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d2c00640 1 -- start start 2024-03-04T08:25:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d2c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04cc0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d2c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04cc0c9840 0x7f04cc0c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d2c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04cc0c4e10 0x7f04cc0c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d2c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc070d40 con 0x7f04b0004810 2024-03-04T08:25:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d2c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc1597b0 con 0x7f04cc0c9840 2024-03-04T08:25:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d2c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc070ec0 con 0x7f04cc0c4e10 2024-03-04T08:25:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04cbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04cc0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:39.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04cbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04cc0c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55044/0 (socket says 172.21.15.113:55044) 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04cbe00640 1 -- 172.21.15.113:0/1438641552 learned_addr learned my addr 172.21.15.113:0/1438641552 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d0c00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04cc0c4e10 0x7f04cc0c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04c3400640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04cc0c9840 0x7f04cc0c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04cbe00640 1 -- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04cc0c4e10 msgr2=0x7f04cc0c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04cbe00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04cc0c4e10 0x7f04cc0c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04cbe00640 1 -- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04cc0c9840 msgr2=0x7f04cc0c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04cbe00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04cc0c9840 0x7f04cc0c5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04cbe00640 1 -- 172.21.15.113:0/1438641552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc1f01f0 con 0x7f04b0004810 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04c3400640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04cc0c9840 0x7f04cc0c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04cbe00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04cc0c5740 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00e700 tx=0x7f04bc00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:39.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04c8c00640 1 -- 172.21.15.113:0/1438641552 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00ee10 con 0x7f04b0004810 2024-03-04T08:25:39.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04c8c00640 1 -- 172.21.15.113:0/1438641552 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc018070 con 0x7f04b0004810 2024-03-04T08:25:39.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04c8c00640 1 -- 172.21.15.113:0/1438641552 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc003dd0 con 0x7f04b0004810 2024-03-04T08:25:39.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d2c00640 1 -- 172.21.15.113:0/1438641552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc1f38e0 con 0x7f04b0004810 2024-03-04T08:25:39.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.297+0000 7f04d2c00640 1 -- 172.21.15.113:0/1438641552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04cc1f0750 con 0x7f04b0004810 2024-03-04T08:25:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.301+0000 7f04d2c00640 1 -- 172.21.15.113:0/1438641552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04cc075c00 con 0x7f04b0004810 2024-03-04T08:25:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.301+0000 7f04c8c00640 1 -- 172.21.15.113:0/1438641552 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04bc0050d0 con 0x7f04b0004810 2024-03-04T08:25:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.301+0000 7f04c8c00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04a8066f90 0x7f04a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.301+0000 7f04c8c00640 1 -- 172.21.15.113:0/1438641552 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f04bc0858c0 con 0x7f04b0004810 2024-03-04T08:25:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.301+0000 7f04c3400640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04a8066f90 0x7f04a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.301+0000 7f04c3400640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04a8066f90 0x7f04a8069450 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f04b40ae000 tx=0x7f04b4007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.305+0000 7f04c8c00640 1 -- 172.21.15.113:0/1438641552 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f04cc1f01f0 con 0x7f04b0004810 2024-03-04T08:25:39.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.489+0000 7f04d2c00640 1 -- 172.21.15.113:0/1438641552 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04cc0c79d0 con 0x7f04b0004810 2024-03-04T08:25:39.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04c8c00640 1 -- 172.21.15.113:0/1438641552 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f04bc055df0 con 0x7f04b0004810 2024-03-04T08:25:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:39.491 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:39.491 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:39.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 -- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04a8066f90 msgr2=0x7f04a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04a8066f90 0x7f04a8069450 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f04b40ae000 tx=0x7f04b4007040 comp rx=0 tx=0).stop 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 -- 172.21.15.113:0/1438641552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 msgr2=0x7f04cc0c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04cc0c5740 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00e700 tx=0x7f04bc00ebc0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 -- 172.21.15.113:0/1438641552 shutdown_connections 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04cc0c4e10 0x7f04cc0c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04cc0c9840 0x7f04cc0c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04a8066f90 0x7f04a8069450 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 --2- 172.21.15.113:0/1438641552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04b0004810 0x7f04cc0c5740 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 -- 172.21.15.113:0/1438641552 >> 172.21.15.113:0/1438641552 conn(0x7f04cc06e820 msgr2=0x7f04cc0cad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 -- 172.21.15.113:0/1438641552 shutdown_connections 2024-03-04T08:25:39.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:39.493+0000 7f04d2c00640 1 -- 172.21.15.113:0/1438641552 wait complete. 2024-03-04T08:25:39.512 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:25:39.513 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:25:41.513 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:41.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f415bc00640 1 Processor -- start 2024-03-04T08:25:41.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f415bc00640 1 -- start start 2024-03-04T08:25:41.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f415bc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f415414e4c0 0x7f4154072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:41.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f415bc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 0x7f41540731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:41.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f415bc00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41540c27f0 con 0x7f41540c22e0 2024-03-04T08:25:41.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f415bc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415415ad60 con 0x7f415414e4c0 2024-03-04T08:25:41.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f415bc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41540c2a90 con 0x7f415415b900 2024-03-04T08:25:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f4159c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 0x7f41540731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f4159200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f415414e4c0 0x7f4154072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:41.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f414be00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f41540c22e0 0x7f415415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:35744/0 (socket says 172.21.15.113:35744) 2024-03-04T08:25:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f414be00640 1 -- 172.21.15.113:0/255856039 learned_addr learned my addr 172.21.15.113:0/255856039 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f4159200640 1 -- 172.21.15.113:0/255856039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 msgr2=0x7f41540731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f4159200640 1 --2- 172.21.15.113:0/255856039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 0x7f41540731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f4159200640 1 -- 172.21.15.113:0/255856039 >> v1:172.21.15.136:6789/0 conn(0x7f41540c22e0 legacy=0x7f415415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f4159200640 1 -- 172.21.15.113:0/255856039 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4154070c60 con 0x7f415414e4c0 2024-03-04T08:25:41.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.661+0000 7f4159c00640 1 --2- 172.21.15.113:0/255856039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 0x7f41540731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:25:41.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f4159200640 1 --2- 172.21.15.113:0/255856039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f415414e4c0 0x7f4154072c80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f41440097c0 tx=0x7f414401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=168106c207c192d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:41.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f4153e00640 1 -- 172.21.15.113:0/255856039 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440b5070 con 0x7f415414e4c0 2024-03-04T08:25:41.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f4153e00640 1 -- 172.21.15.113:0/255856039 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41440a8990 con 0x7f415414e4c0 2024-03-04T08:25:41.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f4153e00640 1 -- 172.21.15.113:0/255856039 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440a8d60 con 0x7f415414e4c0 2024-03-04T08:25:41.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f415bc00640 1 -- 172.21.15.113:0/255856039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f415414e4c0 msgr2=0x7f4154072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:41.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f415bc00640 1 --2- 172.21.15.113:0/255856039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f415414e4c0 0x7f4154072c80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f41440097c0 tx=0x7f414401ec90 comp rx=0 tx=0).stop 2024-03-04T08:25:41.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f415bc00640 1 -- 172.21.15.113:0/255856039 shutdown_connections 2024-03-04T08:25:41.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f415bc00640 1 --2- 172.21.15.113:0/255856039 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 0x7f41540731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:41.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f415bc00640 1 --2- 172.21.15.113:0/255856039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f415414e4c0 0x7f4154072c80 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:41.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f415bc00640 1 -- 172.21.15.113:0/255856039 >> 172.21.15.113:0/255856039 conn(0x7f415406e820 msgr2=0x7f415406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:41.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f415bc00640 1 -- 172.21.15.113:0/255856039 shutdown_connections 2024-03-04T08:25:41.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.665+0000 7f415bc00640 1 -- 172.21.15.113:0/255856039 wait complete. 2024-03-04T08:25:41.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f415bc00640 1 Processor -- start 2024-03-04T08:25:41.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f415bc00640 1 -- start start 2024-03-04T08:25:41.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f415bc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41540c22e0 0x7f4154147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:41.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f415bc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415414e4c0 0x7f4154147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:41.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f415bc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 0x7f4154153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:41.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f415bc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4154071500 con 0x7f415414e4c0 2024-03-04T08:25:41.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f415bc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4154071220 con 0x7f41540c22e0 2024-03-04T08:25:41.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f415bc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415414e8a0 con 0x7f415415b900 2024-03-04T08:25:41.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f414be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415414e4c0 0x7f4154147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:41.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f414be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415414e4c0 0x7f4154147fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55056/0 (socket says 172.21.15.113:55056) 2024-03-04T08:25:41.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f414be00640 1 -- 172.21.15.113:0/441663252 learned_addr learned my addr 172.21.15.113:0/441663252 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:41.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f4159200640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41540c22e0 0x7f4154147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:41.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f4159c00640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 0x7f4154153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:41.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f414be00640 1 -- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 msgr2=0x7f4154153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:41.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f414be00640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 0x7f4154153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:41.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f414be00640 1 -- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41540c22e0 msgr2=0x7f4154147a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:41.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f414be00640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41540c22e0 0x7f4154147a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:41.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f414be00640 1 -- 172.21.15.113:0/441663252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41541f1f30 con 0x7f415414e4c0 2024-03-04T08:25:41.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f414be00640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415414e4c0 0x7f4154147fa0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c370 tx=0x7f413c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:41.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f4151600640 1 -- 172.21.15.113:0/441663252 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c017070 con 0x7f415414e4c0 2024-03-04T08:25:41.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f4151600640 1 -- 172.21.15.113:0/441663252 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413c005cf0 con 0x7f415414e4c0 2024-03-04T08:25:41.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f4151600640 1 -- 172.21.15.113:0/441663252 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413c0072f0 con 0x7f415414e4c0 2024-03-04T08:25:41.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.669+0000 7f415bc00640 1 -- 172.21.15.113:0/441663252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41541f2210 con 0x7f415414e4c0 2024-03-04T08:25:41.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.673+0000 7f415bc00640 1 -- 172.21.15.113:0/441663252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41541f27d0 con 0x7f415414e4c0 2024-03-04T08:25:41.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.673+0000 7f415bc00640 1 -- 172.21.15.113:0/441663252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4154075c00 con 0x7f415414e4c0 2024-03-04T08:25:41.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.673+0000 7f4151600640 1 -- 172.21.15.113:0/441663252 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f413c01a070 con 0x7f415414e4c0 2024-03-04T08:25:41.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.673+0000 7f4151600640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4130066f60 0x7f4130069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:41.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.673+0000 7f4151600640 1 -- 172.21.15.113:0/441663252 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f413c084d70 con 0x7f415414e4c0 2024-03-04T08:25:41.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.673+0000 7f4159200640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4130066f60 0x7f4130069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:41.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.677+0000 7f4159200640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4130066f60 0x7f4130069420 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f41440b2000 tx=0x7f4144008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:41.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.677+0000 7f4151600640 1 -- 172.21.15.113:0/441663252 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f413c054c50 con 0x7f415414e4c0 2024-03-04T08:25:41.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 -- 172.21.15.113:0/441663252 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4154155190 con 0x7f415414e4c0 2024-03-04T08:25:41.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f4151600640 1 -- 172.21.15.113:0/441663252 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f413c058cf0 con 0x7f415414e4c0 2024-03-04T08:25:41.865 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:41.865 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:41.865 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:41.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 -- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4130066f60 msgr2=0x7f4130069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4130066f60 0x7f4130069420 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f41440b2000 tx=0x7f4144008040 comp rx=0 tx=0).stop 2024-03-04T08:25:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 -- 172.21.15.113:0/441663252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415414e4c0 msgr2=0x7f4154147fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415414e4c0 0x7f4154147fa0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f413c00c370 tx=0x7f413c00c830 comp rx=0 tx=0).stop 2024-03-04T08:25:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 -- 172.21.15.113:0/441663252 shutdown_connections 2024-03-04T08:25:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f415415b900 0x7f4154153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4130066f60 0x7f4130069420 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f415414e4c0 0x7f4154147fa0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 --2- 172.21.15.113:0/441663252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41540c22e0 0x7f4154147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 -- 172.21.15.113:0/441663252 >> 172.21.15.113:0/441663252 conn(0x7f415406e820 msgr2=0x7f415414d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:41.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 -- 172.21.15.113:0/441663252 shutdown_connections 2024-03-04T08:25:41.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:41.865+0000 7f415bc00640 1 -- 172.21.15.113:0/441663252 wait complete. 2024-03-04T08:25:41.885 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:43.886 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:44.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde61a00640 1 Processor -- start 2024-03-04T08:25:44.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde61a00640 1 -- start start 2024-03-04T08:25:44.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde61a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde5c15b900 0x7fde5c163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:44.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde61a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c0c27f0 con 0x7fde5c14e4c0 2024-03-04T08:25:44.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde61a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c0c2a90 con 0x7fde5c0c22e0 2024-03-04T08:25:44.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde61a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5c15ad60 con 0x7fde5c15b900 2024-03-04T08:25:44.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde5aa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fde5c14e4c0 0x7fde5c072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:41976/0 (socket says 172.21.15.113:41976) 2024-03-04T08:25:44.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde5b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde5c15b900 0x7fde5c163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:44.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde5aa00640 1 -- 172.21.15.113:0/2291932841 learned_addr learned my addr 172.21.15.113:0/2291932841 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:44.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde5b400640 1 -- 172.21.15.113:0/2291932841 >> v1:172.21.15.136:6789/0 conn(0x7fde5c0c22e0 legacy=0x7fde5c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:44.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde5b400640 1 -- 172.21.15.113:0/2291932841 >> v1:172.21.15.113:6789/0 conn(0x7fde5c14e4c0 legacy=0x7fde5c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:44.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde5b400640 1 -- 172.21.15.113:0/2291932841 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c070c60 con 0x7fde5c15b900 2024-03-04T08:25:44.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.033+0000 7fde59600640 1 -- 172.21.15.113:0/2291932841 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408604939 0 0) 0x7fde5c0c27f0 con 0x7fde5c14e4c0 2024-03-04T08:25:44.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde5b400640 1 --2- 172.21.15.113:0/2291932841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 0x7fde5c163100 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fde54009d90 tx=0x7fde5401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40de0f52624324b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:44.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde59600640 1 -- 172.21.15.113:0/2291932841 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540b4070 con 0x7fde5c15b900 2024-03-04T08:25:44.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde59600640 1 -- 172.21.15.113:0/2291932841 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde540af600 con 0x7fde5c15b900 2024-03-04T08:25:44.036 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde59600640 1 -- 172.21.15.113:0/2291932841 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540af9d0 con 0x7fde5c15b900 2024-03-04T08:25:44.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde61a00640 1 -- 172.21.15.113:0/2291932841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 msgr2=0x7fde5c163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:44.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde61a00640 1 --2- 172.21.15.113:0/2291932841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 0x7fde5c163100 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fde54009d90 tx=0x7fde5401f150 comp rx=0 tx=0).stop 2024-03-04T08:25:44.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde61a00640 1 -- 172.21.15.113:0/2291932841 shutdown_connections 2024-03-04T08:25:44.037 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde61a00640 1 --2- 172.21.15.113:0/2291932841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 0x7fde5c163100 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:44.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde61a00640 1 -- 172.21.15.113:0/2291932841 >> 172.21.15.113:0/2291932841 conn(0x7fde5c06e820 msgr2=0x7fde5c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:44.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde61a00640 1 -- 172.21.15.113:0/2291932841 shutdown_connections 2024-03-04T08:25:44.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.037+0000 7fde61a00640 1 -- 172.21.15.113:0/2291932841 wait complete. 2024-03-04T08:25:44.038 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 Processor -- start 2024-03-04T08:25:44.039 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 -- start start 2024-03-04T08:25:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde5c0c22e0 0x7fde5c145560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde5c14e4c0 0x7fde5c141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 0x7fde5c153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c071500 con 0x7fde5c15b900 2024-03-04T08:25:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c071220 con 0x7fde5c0c22e0 2024-03-04T08:25:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c14e8a0 con 0x7fde5c14e4c0 2024-03-04T08:25:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 0x7fde5c153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:44.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 0x7fde5c153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55086/0 (socket says 172.21.15.113:55086) 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5b400640 1 -- 172.21.15.113:0/2661915350 learned_addr learned my addr 172.21.15.113:0/2661915350 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5a000640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde5c14e4c0 0x7fde5c141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5aa00640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde5c0c22e0 0x7fde5c145560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5b400640 1 -- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde5c14e4c0 msgr2=0x7fde5c141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5b400640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde5c14e4c0 0x7fde5c141500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5b400640 1 -- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde5c0c22e0 msgr2=0x7fde5c145560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5b400640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde5c0c22e0 0x7fde5c145560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5b400640 1 -- 172.21.15.113:0/2661915350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c1f5010 con 0x7fde5c15b900 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde5b400640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 0x7fde5c153310 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fde5401cd80 tx=0x7fde5401bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde52a00640 1 -- 172.21.15.113:0/2661915350 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540b4040 con 0x7fde5c15b900 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde52a00640 1 -- 172.21.15.113:0/2661915350 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde540c1070 con 0x7fde5c15b900 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde52a00640 1 -- 172.21.15.113:0/2661915350 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540ba690 con 0x7fde5c15b900 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 -- 172.21.15.113:0/2661915350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c154bf0 con 0x7fde5c15b900 2024-03-04T08:25:44.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 -- 172.21.15.113:0/2661915350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c152df0 con 0x7fde5c15b900 2024-03-04T08:25:44.043 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.041+0000 7fde61a00640 1 -- 172.21.15.113:0/2661915350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5c075c00 con 0x7fde5c15b900 2024-03-04T08:25:44.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.045+0000 7fde52a00640 1 -- 172.21.15.113:0/2661915350 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde540ba830 con 0x7fde5c15b900 2024-03-04T08:25:44.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.045+0000 7fde52a00640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde3c066f90 0x7fde3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:44.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.045+0000 7fde52a00640 1 -- 172.21.15.113:0/2661915350 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fde541246e0 con 0x7fde5c15b900 2024-03-04T08:25:44.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.045+0000 7fde5aa00640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde3c066f90 0x7fde3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:44.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.045+0000 7fde5aa00640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde3c066f90 0x7fde3c069450 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fde4c007fd0 tx=0x7fde4c007ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:44.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.049+0000 7fde52a00640 1 -- 172.21.15.113:0/2661915350 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fde54018cb0 con 0x7fde5c15b900 2024-03-04T08:25:44.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.237+0000 7fde61a00640 1 -- 172.21.15.113:0/2661915350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde5c0c1260 con 0x7fde5c15b900 2024-03-04T08:25:44.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.237+0000 7fde52a00640 1 -- 172.21.15.113:0/2661915350 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fde540f4e20 con 0x7fde5c15b900 2024-03-04T08:25:44.237 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:44.238 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:44.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 -- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde3c066f90 msgr2=0x7fde3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:44.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde3c066f90 0x7fde3c069450 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fde4c007fd0 tx=0x7fde4c007ba0 comp rx=0 tx=0).stop 2024-03-04T08:25:44.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 -- 172.21.15.113:0/2661915350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 msgr2=0x7fde5c153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:44.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 0x7fde5c153310 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fde5401cd80 tx=0x7fde5401bef0 comp rx=0 tx=0).stop 2024-03-04T08:25:44.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 -- 172.21.15.113:0/2661915350 shutdown_connections 2024-03-04T08:25:44.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fde3c066f90 0x7fde3c069450 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:44.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde5c15b900 0x7fde5c153310 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:44.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fde5c14e4c0 0x7fde5c141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:44.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 --2- 172.21.15.113:0/2661915350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fde5c0c22e0 0x7fde5c145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:44.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 -- 172.21.15.113:0/2661915350 >> 172.21.15.113:0/2661915350 conn(0x7fde5c06e820 msgr2=0x7fde5c144090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:44.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 -- 172.21.15.113:0/2661915350 shutdown_connections 2024-03-04T08:25:44.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:44.241+0000 7fde61a00640 1 -- 172.21.15.113:0/2661915350 wait complete. 2024-03-04T08:25:44.259 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:45.566 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:25:45.570 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:25:46.259 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:46.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.405+0000 7f2676e00640 1 Processor -- start 2024-03-04T08:25:46.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.405+0000 7f2676e00640 1 -- start start 2024-03-04T08:25:46.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2676e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f26700b20b0 0x7f2670157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:46.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2676e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700c85f0 0x7f2670161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:46.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2676e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26700c1ec0 con 0x7f2670157c50 2024-03-04T08:25:46.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2676e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26700c9b30 con 0x7f26700b20b0 2024-03-04T08:25:46.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2676e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26700c12a0 con 0x7f26700c85f0 2024-03-04T08:25:46.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2674e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700c85f0 0x7f2670161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:46.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f266fe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f26700b20b0 0x7f2670157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:46.408 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f266fe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f26700b20b0 0x7f2670157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:36504/0 (socket says 172.21.15.113:36504) 2024-03-04T08:25:46.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2674e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700c85f0 0x7f2670161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:58144/0 (socket says 172.21.15.113:58144) 2024-03-04T08:25:46.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2667400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f2670157c50 0x7f267006b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:35756/0 (socket says 172.21.15.113:35756) 2024-03-04T08:25:46.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f266fe00640 1 -- 172.21.15.113:0/3556558416 learned_addr learned my addr 172.21.15.113:0/3556558416 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:46.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2674e00640 1 -- 172.21.15.113:0/3556558416 >> v2:172.21.15.136:3300/0 conn(0x7f26700b20b0 msgr2=0x7f2670157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:46.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2674e00640 1 --2- 172.21.15.113:0/3556558416 >> v2:172.21.15.136:3300/0 conn(0x7f26700b20b0 0x7f2670157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:46.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2674e00640 1 -- 172.21.15.113:0/3556558416 >> v1:172.21.15.136:6789/0 conn(0x7f2670157c50 legacy=0x7f267006b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:46.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f2674e00640 1 -- 172.21.15.113:0/3556558416 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26700c1580 con 0x7f26700c85f0 2024-03-04T08:25:46.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.409+0000 7f266fe00640 1 --2- 172.21.15.113:0/3556558416 >> v2:172.21.15.136:3300/0 conn(0x7f26700b20b0 0x7f2670157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:25:46.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2674e00640 1 --2- 172.21.15.113:0/3556558416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700c85f0 0x7f2670161820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f26680097c0 tx=0x7f266801ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01af09430e8089c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:46.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f266f400640 1 -- 172.21.15.113:0/3556558416 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680b5070 con 0x7f26700c85f0 2024-03-04T08:25:46.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f266f400640 1 -- 172.21.15.113:0/3556558416 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26680a8990 con 0x7f26700c85f0 2024-03-04T08:25:46.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f266f400640 1 -- 172.21.15.113:0/3556558416 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680a8d60 con 0x7f26700c85f0 2024-03-04T08:25:46.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2676e00640 1 -- 172.21.15.113:0/3556558416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700c85f0 msgr2=0x7f2670161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:46.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2676e00640 1 --2- 172.21.15.113:0/3556558416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700c85f0 0x7f2670161820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f26680097c0 tx=0x7f266801ec90 comp rx=0 tx=0).stop 2024-03-04T08:25:46.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2676e00640 1 -- 172.21.15.113:0/3556558416 shutdown_connections 2024-03-04T08:25:46.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2676e00640 1 --2- 172.21.15.113:0/3556558416 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700c85f0 0x7f2670161820 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:46.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2676e00640 1 --2- 172.21.15.113:0/3556558416 >> v2:172.21.15.136:3300/0 conn(0x7f26700b20b0 0x7f2670157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:46.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2676e00640 1 -- 172.21.15.113:0/3556558416 >> 172.21.15.113:0/3556558416 conn(0x7f267006e820 msgr2=0x7f267006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:46.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2676e00640 1 -- 172.21.15.113:0/3556558416 shutdown_connections 2024-03-04T08:25:46.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2676e00640 1 -- 172.21.15.113:0/3556558416 wait complete. 2024-03-04T08:25:46.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2676e00640 1 Processor -- start 2024-03-04T08:25:46.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.413+0000 7f2676e00640 1 -- start start 2024-03-04T08:25:46.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2676e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700b20b0 0x7f26701f0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:46.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2676e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26700c85f0 0x7f267014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2676e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2670157c50 0x7f26700cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2676e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26700c2400 con 0x7f2670157c50 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2676e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26701571f0 con 0x7f26700c85f0 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2676e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2670156ec0 con 0x7f26700b20b0 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2674e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2670157c50 0x7f26700cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2667400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26700c85f0 0x7f267014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2674e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2670157c50 0x7f26700cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55096/0 (socket says 172.21.15.113:55096) 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2674e00640 1 -- 172.21.15.113:0/3350763956 learned_addr learned my addr 172.21.15.113:0/3350763956 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f266fe00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700b20b0 0x7f26701f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2674e00640 1 -- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700b20b0 msgr2=0x7f26701f0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2674e00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700b20b0 0x7f26701f0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2674e00640 1 -- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26700c85f0 msgr2=0x7f267014e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2674e00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26700c85f0 0x7f267014e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:46.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2674e00640 1 -- 172.21.15.113:0/3350763956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267014e940 con 0x7f2670157c50 2024-03-04T08:25:46.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f266fe00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700b20b0 0x7f26701f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:46.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2674e00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2670157c50 0x7f26700cad10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f266801c8c0 tx=0x7f266801ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:46.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f266cc00640 1 -- 172.21.15.113:0/3350763956 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680bf070 con 0x7f2670157c50 2024-03-04T08:25:46.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f266cc00640 1 -- 172.21.15.113:0/3350763956 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26680b0640 con 0x7f2670157c50 2024-03-04T08:25:46.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f266cc00640 1 -- 172.21.15.113:0/3350763956 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680b5040 con 0x7f2670157c50 2024-03-04T08:25:46.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2676e00640 1 -- 172.21.15.113:0/3350763956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267014c4d0 con 0x7f2670157c50 2024-03-04T08:25:46.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2676e00640 1 -- 172.21.15.113:0/3350763956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267014eeb0 con 0x7f2670157c50 2024-03-04T08:25:46.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f2676e00640 1 -- 172.21.15.113:0/3350763956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2670075c00 con 0x7f2670157c50 2024-03-04T08:25:46.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.417+0000 7f266cc00640 1 -- 172.21.15.113:0/3350763956 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26680b0a10 con 0x7f2670157c50 2024-03-04T08:25:46.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.421+0000 7f266cc00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f264c066f60 0x7f264c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:46.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.421+0000 7f266cc00640 1 -- 172.21.15.113:0/3350763956 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f26681224d0 con 0x7f2670157c50 2024-03-04T08:25:46.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.421+0000 7f266fe00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f264c066f60 0x7f264c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:46.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.421+0000 7f266fe00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f264c066f60 0x7f264c069420 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2660009240 tx=0x7f266000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:46.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.425+0000 7f266cc00640 1 -- 172.21.15.113:0/3350763956 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f26680187f0 con 0x7f2670157c50 2024-03-04T08:25:46.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.609+0000 7f2676e00640 1 -- 172.21.15.113:0/3350763956 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26700c83d0 con 0x7f2670157c50 2024-03-04T08:25:46.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.609+0000 7f266cc00640 1 -- 172.21.15.113:0/3350763956 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f26680f2240 con 0x7f2670157c50 2024-03-04T08:25:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:46.610 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:46.610 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:46.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 -- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f264c066f60 msgr2=0x7f264c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:46.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f264c066f60 0x7f264c069420 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2660009240 tx=0x7f266000b040 comp rx=0 tx=0).stop 2024-03-04T08:25:46.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 -- 172.21.15.113:0/3350763956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2670157c50 msgr2=0x7f26700cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:46.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2670157c50 0x7f26700cad10 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f266801c8c0 tx=0x7f266801ba30 comp rx=0 tx=0).stop 2024-03-04T08:25:46.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 -- 172.21.15.113:0/3350763956 shutdown_connections 2024-03-04T08:25:46.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f264c066f60 0x7f264c069420 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:46.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2670157c50 0x7f26700cad10 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:46.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f26700c85f0 0x7f267014e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:46.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 --2- 172.21.15.113:0/3350763956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f26700b20b0 0x7f26701f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:46.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 -- 172.21.15.113:0/3350763956 >> 172.21.15.113:0/3350763956 conn(0x7f267006e820 msgr2=0x7f2670150e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:46.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 -- 172.21.15.113:0/3350763956 shutdown_connections 2024-03-04T08:25:46.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:46.613+0000 7f2676e00640 1 -- 172.21.15.113:0/3350763956 wait complete. 2024-03-04T08:25:46.631 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:48.632 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4956800640 1 Processor -- start 2024-03-04T08:25:48.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4956800640 1 -- start start 2024-03-04T08:25:48.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4956800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49500ca760 0x7f495006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:48.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4956800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495006cd00 0x7f49500ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4956800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4950156430 con 0x7f4950156050 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4956800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495006b5a0 con 0x7f49500ca760 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4956800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4950158bb0 con 0x7f495006cd00 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4946a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49500ca760 0x7f495006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4946a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49500ca760 0x7f495006b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58170/0 (socket says 172.21.15.113:58170) 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4946a00640 1 -- 172.21.15.113:0/2154508540 learned_addr learned my addr 172.21.15.113:0/2154508540 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f494f400640 1 --1- 172.21.15.113:0/2154508540 >> v1:172.21.15.113:6789/0 conn(0x7f4950156050 0x7f495006dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54078/0 (socket says 172.21.15.113:54078) 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f494fe00640 1 --2- 172.21.15.113:0/2154508540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495006cd00 0x7f49500ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4946a00640 1 -- 172.21.15.113:0/2154508540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495006cd00 msgr2=0x7f49500ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4946a00640 1 --2- 172.21.15.113:0/2154508540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495006cd00 0x7f49500ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4946a00640 1 -- 172.21.15.113:0/2154508540 >> v1:172.21.15.113:6789/0 conn(0x7f4950156050 legacy=0x7f495006dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f4946a00640 1 -- 172.21.15.113:0/2154508540 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4950158e90 con 0x7f49500ca760 2024-03-04T08:25:48.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.777+0000 7f494ea00640 1 -- 172.21.15.113:0/2154508540 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572507867 0 0) 0x7f4950156430 con 0x7f4950156050 2024-03-04T08:25:48.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f4946a00640 1 --2- 172.21.15.113:0/2154508540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49500ca760 0x7f495006b060 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f4938009870 tx=0x7f493801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2438493045b96053 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:48.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f494ea00640 1 -- 172.21.15.113:0/2154508540 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380b2070 con 0x7f49500ca760 2024-03-04T08:25:48.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f494ea00640 1 -- 172.21.15.113:0/2154508540 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49380a9070 con 0x7f49500ca760 2024-03-04T08:25:48.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f494ea00640 1 -- 172.21.15.113:0/2154508540 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380ad880 con 0x7f49500ca760 2024-03-04T08:25:48.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f4956800640 1 -- 172.21.15.113:0/2154508540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49500ca760 msgr2=0x7f495006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:48.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f4956800640 1 --2- 172.21.15.113:0/2154508540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49500ca760 0x7f495006b060 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f4938009870 tx=0x7f493801c560 comp rx=0 tx=0).stop 2024-03-04T08:25:48.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f4956800640 1 -- 172.21.15.113:0/2154508540 shutdown_connections 2024-03-04T08:25:48.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f4956800640 1 --2- 172.21.15.113:0/2154508540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495006cd00 0x7f49500ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f4956800640 1 --2- 172.21.15.113:0/2154508540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49500ca760 0x7f495006b060 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f4956800640 1 -- 172.21.15.113:0/2154508540 >> 172.21.15.113:0/2154508540 conn(0x7f4950071d20 msgr2=0x7f4950072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:48.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f4956800640 1 -- 172.21.15.113:0/2154508540 shutdown_connections 2024-03-04T08:25:48.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.781+0000 7f4956800640 1 -- 172.21.15.113:0/2154508540 wait complete. 2024-03-04T08:25:48.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 Processor -- start 2024-03-04T08:25:48.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 -- start start 2024-03-04T08:25:48.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495006cd00 0x7f49501548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:48.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49500ca760 0x7f495014b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:48.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4950156050 0x7f495014bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495006e590 con 0x7f4950156050 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495006e410 con 0x7f495006cd00 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4950165a80 con 0x7f49500ca760 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4950156050 0x7f495014bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4950156050 0x7f495014bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58172/0 (socket says 172.21.15.113:58172) 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494fe00640 1 -- 172.21.15.113:0/2586796110 learned_addr learned my addr 172.21.15.113:0/2586796110 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494f400640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495006cd00 0x7f49501548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4946a00640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49500ca760 0x7f495014b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494fe00640 1 -- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49500ca760 msgr2=0x7f495014b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494fe00640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49500ca760 0x7f495014b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494fe00640 1 -- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495006cd00 msgr2=0x7f49501548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494fe00640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495006cd00 0x7f49501548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494fe00640 1 -- 172.21.15.113:0/2586796110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4950155750 con 0x7f4950156050 2024-03-04T08:25:48.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494f400640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495006cd00 0x7f49501548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:48.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4946a00640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49500ca760 0x7f495014b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:48.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f494fe00640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4950156050 0x7f495014bc40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f4948007e40 tx=0x7f494800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:48.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4947e00640 1 -- 172.21.15.113:0/2586796110 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948014620 con 0x7f4950156050 2024-03-04T08:25:48.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4947e00640 1 -- 172.21.15.113:0/2586796110 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948019070 con 0x7f4950156050 2024-03-04T08:25:48.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 -- 172.21.15.113:0/2586796110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49501f3640 con 0x7f4950156050 2024-03-04T08:25:48.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4947e00640 1 -- 172.21.15.113:0/2586796110 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948014a20 con 0x7f4950156050 2024-03-04T08:25:48.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 -- 172.21.15.113:0/2586796110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49501551f0 con 0x7f4950156050 2024-03-04T08:25:48.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.785+0000 7f4956800640 1 -- 172.21.15.113:0/2586796110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4950075c00 con 0x7f4950156050 2024-03-04T08:25:48.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.789+0000 7f4947e00640 1 -- 172.21.15.113:0/2586796110 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4948014bc0 con 0x7f4950156050 2024-03-04T08:25:48.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.789+0000 7f4947e00640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f492c066f60 0x7f492c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:48.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.789+0000 7f4947e00640 1 -- 172.21.15.113:0/2586796110 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f4948086410 con 0x7f4950156050 2024-03-04T08:25:48.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.789+0000 7f494f400640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f492c066f60 0x7f492c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:48.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.789+0000 7f494f400640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f492c066f60 0x7f492c069420 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4940000f70 tx=0x7f4940007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:48.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.793+0000 7f4947e00640 1 -- 172.21.15.113:0/2586796110 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4948011260 con 0x7f4950156050 2024-03-04T08:25:48.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.977+0000 7f4956800640 1 -- 172.21.15.113:0/2586796110 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4950163040 con 0x7f4950156050 2024-03-04T08:25:48.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4947e00640 1 -- 172.21.15.113:0/2586796110 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f49480568e0 con 0x7f4950156050 2024-03-04T08:25:48.979 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:48.979 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:48.980 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:48.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 -- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f492c066f60 msgr2=0x7f492c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:48.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f492c066f60 0x7f492c069420 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4940000f70 tx=0x7f4940007cb0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 -- 172.21.15.113:0/2586796110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4950156050 msgr2=0x7f495014bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:48.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4950156050 0x7f495014bc40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f4948007e40 tx=0x7f494800e4b0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 -- 172.21.15.113:0/2586796110 shutdown_connections 2024-03-04T08:25:48.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f492c066f60 0x7f492c069420 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4950156050 0x7f495014bc40 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f49500ca760 0x7f495014b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 --2- 172.21.15.113:0/2586796110 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495006cd00 0x7f49501548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:48.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 -- 172.21.15.113:0/2586796110 >> 172.21.15.113:0/2586796110 conn(0x7f4950071d20 msgr2=0x7f4950072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:48.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 -- 172.21.15.113:0/2586796110 shutdown_connections 2024-03-04T08:25:48.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:48.981+0000 7f4956800640 1 -- 172.21.15.113:0/2586796110 wait complete. 2024-03-04T08:25:49.000 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:51.001 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:51.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f3200640 1 Processor -- start 2024-03-04T08:25:51.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f3200640 1 -- start start 2024-03-04T08:25:51.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f3200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f95ec152a90 0x7f95ec0730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f3200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95ec15bce0 0x7f95ec15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f3200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec06f1b0 con 0x7f95ec06cd10 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f3200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ec152e70 con 0x7f95ec15bce0 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f3200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ec06c320 con 0x7f95ec152a90 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95ec15bce0 0x7f95ec15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f95ec15bce0 0x7f95ec15c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58188/0 (socket says 172.21.15.113:58188) 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f2800640 1 -- 172.21.15.113:0/3344028539 learned_addr learned my addr 172.21.15.113:0/3344028539 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f1e00640 1 --2- 172.21.15.113:0/3344028539 >> v2:172.21.15.136:3300/0 conn(0x7f95ec152a90 0x7f95ec0730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f2800640 1 -- 172.21.15.113:0/3344028539 >> v2:172.21.15.136:3300/0 conn(0x7f95ec152a90 msgr2=0x7f95ec0730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f2800640 1 --2- 172.21.15.113:0/3344028539 >> v2:172.21.15.136:3300/0 conn(0x7f95ec152a90 0x7f95ec0730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f2800640 1 -- 172.21.15.113:0/3344028539 >> v1:172.21.15.136:6789/0 conn(0x7f95ec06cd10 legacy=0x7f95ec06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f2800640 1 -- 172.21.15.113:0/3344028539 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec071580 con 0x7f95ec15bce0 2024-03-04T08:25:51.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.149+0000 7f95f1e00640 1 --2- 172.21.15.113:0/3344028539 >> v2:172.21.15.136:3300/0 conn(0x7f95ec152a90 0x7f95ec0730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:51.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f2800640 1 --2- 172.21.15.113:0/3344028539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 0x7f95ec15c0f0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f95ec06df40 tx=0x7f95e801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be409d260f1af0f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:51.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f0a00640 1 -- 172.21.15.113:0/3344028539 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e80a56c0 con 0x7f95ec15bce0 2024-03-04T08:25:51.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f0a00640 1 -- 172.21.15.113:0/3344028539 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e80a5860 con 0x7f95ec15bce0 2024-03-04T08:25:51.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f0a00640 1 -- 172.21.15.113:0/3344028539 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e80a5c30 con 0x7f95ec15bce0 2024-03-04T08:25:51.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f3200640 1 -- 172.21.15.113:0/3344028539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 msgr2=0x7f95ec15c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:51.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f3200640 1 --2- 172.21.15.113:0/3344028539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 0x7f95ec15c0f0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f95ec06df40 tx=0x7f95e801c9f0 comp rx=0 tx=0).stop 2024-03-04T08:25:51.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f3200640 1 -- 172.21.15.113:0/3344028539 shutdown_connections 2024-03-04T08:25:51.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f3200640 1 --2- 172.21.15.113:0/3344028539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 0x7f95ec15c0f0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:51.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f3200640 1 --2- 172.21.15.113:0/3344028539 >> v2:172.21.15.136:3300/0 conn(0x7f95ec152a90 0x7f95ec0730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:51.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f3200640 1 -- 172.21.15.113:0/3344028539 >> 172.21.15.113:0/3344028539 conn(0x7f95ec06e2a0 msgr2=0x7f95ec06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:51.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f3200640 1 -- 172.21.15.113:0/3344028539 shutdown_connections 2024-03-04T08:25:51.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f3200640 1 -- 172.21.15.113:0/3344028539 wait complete. 2024-03-04T08:25:51.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f3200640 1 Processor -- start 2024-03-04T08:25:51.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.153+0000 7f95f3200640 1 -- start start 2024-03-04T08:25:51.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f3200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95ec06cd10 0x7f95ec143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:51.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f3200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ec152a90 0x7f95ec0c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:51.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f3200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 0x7f95ec0c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:51.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f3200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec165890 con 0x7f95ec15bce0 2024-03-04T08:25:51.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f3200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec071330 con 0x7f95ec152a90 2024-03-04T08:25:51.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f3200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec165a10 con 0x7f95ec06cd10 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 0x7f95ec0c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 0x7f95ec0c4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58192/0 (socket says 172.21.15.113:58192) 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f2800640 1 -- 172.21.15.113:0/2901774362 learned_addr learned my addr 172.21.15.113:0/2901774362 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f1400640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ec152a90 0x7f95ec0c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f1e00640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95ec06cd10 0x7f95ec143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f2800640 1 -- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95ec06cd10 msgr2=0x7f95ec143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f2800640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95ec06cd10 0x7f95ec143850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f2800640 1 -- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ec152a90 msgr2=0x7f95ec0c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f2800640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ec152a90 0x7f95ec0c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f2800640 1 -- 172.21.15.113:0/2901774362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec072030 con 0x7f95ec15bce0 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f2800640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 0x7f95ec0c4ba0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f95e801a620 tx=0x7f95e8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95e6000640 1 -- 172.21.15.113:0/2901774362 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e8009510 con 0x7f95ec15bce0 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95e6000640 1 -- 172.21.15.113:0/2901774362 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e80be070 con 0x7f95ec15bce0 2024-03-04T08:25:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f3200640 1 -- 172.21.15.113:0/2901774362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec151ff0 con 0x7f95ec15bce0 2024-03-04T08:25:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95e6000640 1 -- 172.21.15.113:0/2901774362 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e80b88b0 con 0x7f95ec15bce0 2024-03-04T08:25:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f3200640 1 -- 172.21.15.113:0/2901774362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ec1445b0 con 0x7f95ec15bce0 2024-03-04T08:25:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95f3200640 1 -- 172.21.15.113:0/2901774362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ec075a80 con 0x7f95ec15bce0 2024-03-04T08:25:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.157+0000 7f95e6000640 1 -- 172.21.15.113:0/2901774362 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95e80b8a50 con 0x7f95ec15bce0 2024-03-04T08:25:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.161+0000 7f95e6000640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95d0066f90 0x7f95d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.161+0000 7f95e6000640 1 -- 172.21.15.113:0/2901774362 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f95e8120d20 con 0x7f95ec15bce0 2024-03-04T08:25:51.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.161+0000 7f95f1e00640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95d0066f90 0x7f95d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:51.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.161+0000 7f95f1e00640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95d0066f90 0x7f95d0069450 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f95e000ba10 tx=0x7f95e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:51.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.165+0000 7f95e6000640 1 -- 172.21.15.113:0/2901774362 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f95e80165e0 con 0x7f95ec15bce0 2024-03-04T08:25:51.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 -- 172.21.15.113:0/2901774362 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95ec0c5f30 con 0x7f95ec15bce0 2024-03-04T08:25:51.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95e6000640 1 -- 172.21.15.113:0/2901774362 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f95e80f1280 con 0x7f95ec15bce0 2024-03-04T08:25:51.352 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:51.352 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:51.352 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:51.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 -- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95d0066f90 msgr2=0x7f95d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:51.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95d0066f90 0x7f95d0069450 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f95e000ba10 tx=0x7f95e0008040 comp rx=0 tx=0).stop 2024-03-04T08:25:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 -- 172.21.15.113:0/2901774362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 msgr2=0x7f95ec0c4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 0x7f95ec0c4ba0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f95e801a620 tx=0x7f95e8019790 comp rx=0 tx=0).stop 2024-03-04T08:25:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 -- 172.21.15.113:0/2901774362 shutdown_connections 2024-03-04T08:25:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f95d0066f90 0x7f95d0069450 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95ec15bce0 0x7f95ec0c4ba0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ec152a90 0x7f95ec0c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 --2- 172.21.15.113:0/2901774362 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f95ec06cd10 0x7f95ec143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 -- 172.21.15.113:0/2901774362 >> 172.21.15.113:0/2901774362 conn(0x7f95ec06e2a0 msgr2=0x7f95ec0cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 -- 172.21.15.113:0/2901774362 shutdown_connections 2024-03-04T08:25:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:51.353+0000 7f95f3200640 1 -- 172.21.15.113:0/2901774362 wait complete. 2024-03-04T08:25:51.374 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:25:51.374 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:25:53.375 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:53.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3b0a00640 1 Processor -- start 2024-03-04T08:25:53.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3b0a00640 1 -- start start 2024-03-04T08:25:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3b0a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 0x7fb3ac0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3b0a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3ac070120 0x7fb3ac0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3b0a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac0c5380 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3b0a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ac0c3440 con 0x7fb3ac070120 2024-03-04T08:25:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3b0a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac06f620 con 0x7fb3ac152d60 2024-03-04T08:25:53.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3a1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3ac070120 0x7fb3ac0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3a1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb3ac070120 0x7fb3ac0ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58196/0 (socket says 172.21.15.113:58196) 2024-03-04T08:25:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3a1400640 1 -- 172.21.15.113:0/663561090 learned_addr learned my addr 172.21.15.113:0/663561090 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3a9e00640 1 --2- 172.21.15.113:0/663561090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 0x7fb3ac0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3a1400640 1 -- 172.21.15.113:0/663561090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 msgr2=0x7fb3ac0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3a1400640 1 --2- 172.21.15.113:0/663561090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 0x7fb3ac0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3a1400640 1 -- 172.21.15.113:0/663561090 >> v1:172.21.15.136:6789/0 conn(0x7fb3ac0c9db0 legacy=0x7fb3ac158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:53.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.525+0000 7fb3a1400640 1 -- 172.21.15.113:0/663561090 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac0c36e0 con 0x7fb3ac070120 2024-03-04T08:25:53.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3a1400640 1 --2- 172.21.15.113:0/663561090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac070120 0x7fb3ac0ce850 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fb394009870 tx=0x7fb39401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd3d16d0365b0c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:53.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3a9400640 1 -- 172.21.15.113:0/663561090 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940b5070 con 0x7fb3ac070120 2024-03-04T08:25:53.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3a9400640 1 -- 172.21.15.113:0/663561090 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3940a8a50 con 0x7fb3ac070120 2024-03-04T08:25:53.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3a9400640 1 -- 172.21.15.113:0/663561090 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940a8e20 con 0x7fb3ac070120 2024-03-04T08:25:53.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3b0a00640 1 -- 172.21.15.113:0/663561090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac070120 msgr2=0x7fb3ac0ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:53.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3b0a00640 1 --2- 172.21.15.113:0/663561090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac070120 0x7fb3ac0ce850 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fb394009870 tx=0x7fb39401ed50 comp rx=0 tx=0).stop 2024-03-04T08:25:53.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3b0a00640 1 -- 172.21.15.113:0/663561090 shutdown_connections 2024-03-04T08:25:53.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3b0a00640 1 --2- 172.21.15.113:0/663561090 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac070120 0x7fb3ac0ce850 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:53.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3b0a00640 1 --2- 172.21.15.113:0/663561090 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 0x7fb3ac0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:53.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3b0a00640 1 -- 172.21.15.113:0/663561090 >> 172.21.15.113:0/663561090 conn(0x7fb3ac066db0 msgr2=0x7fb3ac0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:53.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3b0a00640 1 -- 172.21.15.113:0/663561090 shutdown_connections 2024-03-04T08:25:53.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.529+0000 7fb3b0a00640 1 -- 172.21.15.113:0/663561090 wait complete. 2024-03-04T08:25:53.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3b0a00640 1 Processor -- start 2024-03-04T08:25:53.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3b0a00640 1 -- start start 2024-03-04T08:25:53.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3b0a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3ac070120 0x7fb3ac14f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:53.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3b0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac0c9db0 0x7fb3ac14fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:53.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3b0a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 0x7fb3ac14ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:53.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3b0a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0c39c0 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3b0a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0c5020 con 0x7fb3ac070120 2024-03-04T08:25:53.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3b0a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac0c3b40 con 0x7fb3ac152d60 2024-03-04T08:25:53.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac0c9db0 0x7fb3ac14fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:53.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac0c9db0 0x7fb3ac14fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58208/0 (socket says 172.21.15.113:58208) 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a1400640 1 -- 172.21.15.113:0/3494091046 learned_addr learned my addr 172.21.15.113:0/3494091046 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a9e00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3ac070120 0x7fb3ac14f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3aa800640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 0x7fb3ac14ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a1400640 1 -- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 msgr2=0x7fb3ac14ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a1400640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 0x7fb3ac14ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a1400640 1 -- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3ac070120 msgr2=0x7fb3ac14f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a1400640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3ac070120 0x7fb3ac14f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a1400640 1 -- 172.21.15.113:0/3494091046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac0af900 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a9e00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3ac070120 0x7fb3ac14f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3aa800640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 0x7fb3ac14ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a1400640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac0c9db0 0x7fb3ac14fe40 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fb3940b3040 tx=0x7fb39401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a2a00640 1 -- 172.21.15.113:0/3494091046 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940b5070 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a2a00640 1 -- 172.21.15.113:0/3494091046 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3940b0de0 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3a2a00640 1 -- 172.21.15.113:0/3494091046 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940bb670 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3b0a00640 1 -- 172.21.15.113:0/3494091046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac0afbe0 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.533+0000 7fb3b0a00640 1 -- 172.21.15.113:0/3494091046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac14ee20 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.537+0000 7fb3b0a00640 1 -- 172.21.15.113:0/3494091046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac075c00 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.537+0000 7fb3a2a00640 1 -- 172.21.15.113:0/3494091046 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3940c2060 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.537+0000 7fb3a2a00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb388066f60 0x7fb388069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.537+0000 7fb3a2a00640 1 -- 172.21.15.113:0/3494091046 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fb394122f30 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.537+0000 7fb3a9e00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb388066f60 0x7fb388069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:53.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.537+0000 7fb3a9e00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb388066f60 0x7fb388069420 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009bb0 tx=0x7fb39c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:53.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.541+0000 7fb3a2a00640 1 -- 172.21.15.113:0/3494091046 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb3940188b0 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 -- 172.21.15.113:0/3494091046 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3ac06ff00 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3a2a00640 1 -- 172.21.15.113:0/3494091046 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fb3940f3490 con 0x7fb3ac0c9db0 2024-03-04T08:25:53.730 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:53.730 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:53.731 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:53.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 -- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb388066f60 msgr2=0x7fb388069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb388066f60 0x7fb388069420 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009bb0 tx=0x7fb39c008040 comp rx=0 tx=0).stop 2024-03-04T08:25:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 -- 172.21.15.113:0/3494091046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac0c9db0 msgr2=0x7fb3ac14fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac0c9db0 0x7fb3ac14fe40 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fb3940b3040 tx=0x7fb39401baf0 comp rx=0 tx=0).stop 2024-03-04T08:25:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 -- 172.21.15.113:0/3494091046 shutdown_connections 2024-03-04T08:25:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb3ac152d60 0x7fb3ac14ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:53.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb388066f60 0x7fb388069420 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:53.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb3ac0c9db0 0x7fb3ac14fe40 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:53.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 --2- 172.21.15.113:0/3494091046 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3ac070120 0x7fb3ac14f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:53.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.729+0000 7fb3b0a00640 1 -- 172.21.15.113:0/3494091046 >> 172.21.15.113:0/3494091046 conn(0x7fb3ac066db0 msgr2=0x7fb3ac0cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:53.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.733+0000 7fb3b0a00640 1 -- 172.21.15.113:0/3494091046 shutdown_connections 2024-03-04T08:25:53.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:53.733+0000 7fb3b0a00640 1 -- 172.21.15.113:0/3494091046 wait complete. 2024-03-04T08:25:53.750 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:55.751 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:55.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.897+0000 7fcdb1a00640 1 Processor -- start 2024-03-04T08:25:55.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.897+0000 7fcdb1a00640 1 -- start start 2024-03-04T08:25:55.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.897+0000 7fcdb1a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcdac155d80 0x7fcdac0c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:55.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.897+0000 7fcdb1a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdac0c7050 0x7fcdac0c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:55.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.897+0000 7fcdb1a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac156160 con 0x7fcdac0c1860 2024-03-04T08:25:55.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.897+0000 7fcdb1a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac162500 con 0x7fcdac155d80 2024-03-04T08:25:55.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.897+0000 7fcdb1a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac15ef10 con 0x7fcdac0c7050 2024-03-04T08:25:55.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.897+0000 7fcdaaa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcdac155d80 0x7fcdac0c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:55.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.897+0000 7fcda2000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcdac0c1860 0x7fcdac162c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:54094/0 (socket says 172.21.15.113:54094) 2024-03-04T08:25:55.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.897+0000 7fcda2000640 1 -- 172.21.15.113:0/2891032775 learned_addr learned my addr 172.21.15.113:0/2891032775 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:25:55.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.901+0000 7fcdab400640 1 --2- 172.21.15.113:0/2891032775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdac0c7050 0x7fcdac0c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:55.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.901+0000 7fcdaaa00640 1 -- 172.21.15.113:0/2891032775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdac0c7050 msgr2=0x7fcdac0c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:25:55.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.901+0000 7fcdaaa00640 1 --2- 172.21.15.113:0/2891032775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdac0c7050 0x7fcdac0c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:55.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.901+0000 7fcdaaa00640 1 -- 172.21.15.113:0/2891032775 >> v1:172.21.15.113:6789/0 conn(0x7fcdac0c1860 legacy=0x7fcdac162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:55.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.901+0000 7fcdaaa00640 1 -- 172.21.15.113:0/2891032775 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac15f1b0 con 0x7fcdac155d80 2024-03-04T08:25:55.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.901+0000 7fcdaa000640 1 -- 172.21.15.113:0/2891032775 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600442662 0 0) 0x7fcdac156160 con 0x7fcdac0c1860 2024-03-04T08:25:55.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.901+0000 7fcdaaa00640 1 --2- 172.21.15.113:0/2891032775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac155d80 0x7fcdac0c7dd0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009d90 tx=0x7fcd9c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b451b1cbee6dd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:55.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.901+0000 7fcdaa000640 1 -- 172.21.15.113:0/2891032775 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0b2070 con 0x7fcdac155d80 2024-03-04T08:25:55.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.901+0000 7fcdaa000640 1 -- 172.21.15.113:0/2891032775 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c0a5940 con 0x7fcdac155d80 2024-03-04T08:25:55.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.901+0000 7fcdaa000640 1 -- 172.21.15.113:0/2891032775 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0a5d10 con 0x7fcdac155d80 2024-03-04T08:25:55.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 -- 172.21.15.113:0/2891032775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac155d80 msgr2=0x7fcdac0c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:55.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 --2- 172.21.15.113:0/2891032775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac155d80 0x7fcdac0c7dd0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009d90 tx=0x7fcd9c01c970 comp rx=0 tx=0).stop 2024-03-04T08:25:55.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 -- 172.21.15.113:0/2891032775 shutdown_connections 2024-03-04T08:25:55.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 --2- 172.21.15.113:0/2891032775 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdac0c7050 0x7fcdac0c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:55.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 --2- 172.21.15.113:0/2891032775 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac155d80 0x7fcdac0c7dd0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:55.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 -- 172.21.15.113:0/2891032775 >> 172.21.15.113:0/2891032775 conn(0x7fcdac06e820 msgr2=0x7fcdac06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:55.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 -- 172.21.15.113:0/2891032775 shutdown_connections 2024-03-04T08:25:55.903 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 -- 172.21.15.113:0/2891032775 wait complete. 2024-03-04T08:25:55.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 Processor -- start 2024-03-04T08:25:55.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 -- start start 2024-03-04T08:25:55.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac0c1860 0x7fcdac15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:55.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdac0c7050 0x7fcdac15e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:55.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdac155d80 0x7fcdac0c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:55.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac15fcd0 con 0x7fcdac0c1860 2024-03-04T08:25:55.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac15f9f0 con 0x7fcdac0c7050 2024-03-04T08:25:55.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdb1a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac163380 con 0x7fcdac155d80 2024-03-04T08:25:55.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdaaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac0c1860 0x7fcdac15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:55.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdaaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac0c1860 0x7fcdac15d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58226/0 (socket says 172.21.15.113:58226) 2024-03-04T08:25:55.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdaaa00640 1 -- 172.21.15.113:0/3893986662 learned_addr learned my addr 172.21.15.113:0/3893986662 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:55.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcda2000640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdac0c7050 0x7fcdac15e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdab400640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdac155d80 0x7fcdac0c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdaaa00640 1 -- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdac155d80 msgr2=0x7fcdac0c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdaaa00640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdac155d80 0x7fcdac0c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdaaa00640 1 -- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdac0c7050 msgr2=0x7fcdac15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdaaa00640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdac0c7050 0x7fcdac15e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdaaa00640 1 -- 172.21.15.113:0/3893986662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac1554c0 con 0x7fcdac0c1860 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcda2000640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdac0c7050 0x7fcdac15e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.905+0000 7fcdaaa00640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac0c1860 0x7fcdac15d880 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c01a5a0 tx=0x7fcd9c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.909+0000 7fcda3400640 1 -- 172.21.15.113:0/3893986662 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0bd020 con 0x7fcdac0c1860 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.909+0000 7fcda3400640 1 -- 172.21.15.113:0/3893986662 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c0ad7f0 con 0x7fcdac0c1860 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.909+0000 7fcda3400640 1 -- 172.21.15.113:0/3893986662 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0b2040 con 0x7fcdac0c1860 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.909+0000 7fcdb1a00640 1 -- 172.21.15.113:0/3893986662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac155740 con 0x7fcdac0c1860 2024-03-04T08:25:55.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.909+0000 7fcdb1a00640 1 -- 172.21.15.113:0/3893986662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac15cda0 con 0x7fcdac0c1860 2024-03-04T08:25:55.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.909+0000 7fcdb1a00640 1 -- 172.21.15.113:0/3893986662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac075c00 con 0x7fcdac0c1860 2024-03-04T08:25:55.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.909+0000 7fcda3400640 1 -- 172.21.15.113:0/3893986662 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd9c0c0020 con 0x7fcdac0c1860 2024-03-04T08:25:55.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.909+0000 7fcda3400640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcd88066f60 0x7fcd88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:55.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.909+0000 7fcda3400640 1 -- 172.21.15.113:0/3893986662 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fcd9c120d90 con 0x7fcdac0c1860 2024-03-04T08:25:55.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.909+0000 7fcda2000640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcd88066f60 0x7fcd88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:55.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.913+0000 7fcda2000640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcd88066f60 0x7fcd88069420 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fcdac15c3b0 tx=0x7fcd94007cb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:55.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:55.913+0000 7fcda3400640 1 -- 172.21.15.113:0/3893986662 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcd9c0c7030 con 0x7fcdac0c1860 2024-03-04T08:25:56.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.097+0000 7fcdb1a00640 1 -- 172.21.15.113:0/3893986662 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdac169670 con 0x7fcdac0c1860 2024-03-04T08:25:56.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.097+0000 7fcda3400640 1 -- 172.21.15.113:0/3893986662 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fcdac1554c0 con 0x7fcdac0c1860 2024-03-04T08:25:56.098 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:56.099 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:56.099 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 -- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcd88066f60 msgr2=0x7fcd88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcd88066f60 0x7fcd88069420 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fcdac15c3b0 tx=0x7fcd94007cb0 comp rx=0 tx=0).stop 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 -- 172.21.15.113:0/3893986662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac0c1860 msgr2=0x7fcdac15d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac0c1860 0x7fcdac15d880 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c01a5a0 tx=0x7fcd9c019710 comp rx=0 tx=0).stop 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 -- 172.21.15.113:0/3893986662 shutdown_connections 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcdac155d80 0x7fcdac0c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcdac0c7050 0x7fcdac15e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcd88066f60 0x7fcd88069420 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 --2- 172.21.15.113:0/3893986662 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdac0c1860 0x7fcdac15d880 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 -- 172.21.15.113:0/3893986662 >> 172.21.15.113:0/3893986662 conn(0x7fcdac06e820 msgr2=0x7fcdac0729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 -- 172.21.15.113:0/3893986662 shutdown_connections 2024-03-04T08:25:56.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:56.101+0000 7fcdb1a00640 1 -- 172.21.15.113:0/3893986662 wait complete. 2024-03-04T08:25:56.119 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:25:58.121 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:25:58.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4960a00640 1 Processor -- start 2024-03-04T08:25:58.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4960a00640 1 -- start start 2024-03-04T08:25:58.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4960a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f495c0ce570 0x7f495c154c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:58.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4960a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495c06c860 con 0x7f495c070120 2024-03-04T08:25:58.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4960a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495c06f8d0 con 0x7f495c0c8d80 2024-03-04T08:25:58.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4960a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495c06f310 con 0x7f495c0ce570 2024-03-04T08:25:58.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4959e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f495c0ce570 0x7f495c154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:58.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4959e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f495c0ce570 0x7f495c154c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41558/0 (socket says 172.21.15.113:41558) 2024-03-04T08:25:58.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4959e00640 1 -- 172.21.15.113:0/40031318 learned_addr learned my addr 172.21.15.113:0/40031318 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:58.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4959e00640 1 -- 172.21.15.113:0/40031318 >> v1:172.21.15.136:6789/0 conn(0x7f495c0c8d80 legacy=0x7f495c15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:58.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4959e00640 1 -- 172.21.15.113:0/40031318 >> v1:172.21.15.113:6789/0 conn(0x7f495c070120 legacy=0x7f495c1656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:58.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.269+0000 7f4959e00640 1 -- 172.21.15.113:0/40031318 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c06f5f0 con 0x7f495c0ce570 2024-03-04T08:25:58.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4959e00640 1 --2- 172.21.15.113:0/40031318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 0x7f495c154c90 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f494c009e50 tx=0x7f494c01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc91c1e7b87848f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:58.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4959400640 1 -- 172.21.15.113:0/40031318 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c0b4070 con 0x7f495c0ce570 2024-03-04T08:25:58.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4959400640 1 -- 172.21.15.113:0/40031318 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c0af400 con 0x7f495c0ce570 2024-03-04T08:25:58.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4959400640 1 -- 172.21.15.113:0/40031318 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c0af7d0 con 0x7f495c0ce570 2024-03-04T08:25:58.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4960a00640 1 -- 172.21.15.113:0/40031318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 msgr2=0x7f495c154c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:58.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4960a00640 1 --2- 172.21.15.113:0/40031318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 0x7f495c154c90 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f494c009e50 tx=0x7f494c01f210 comp rx=0 tx=0).stop 2024-03-04T08:25:58.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4960a00640 1 -- 172.21.15.113:0/40031318 shutdown_connections 2024-03-04T08:25:58.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4960a00640 1 --2- 172.21.15.113:0/40031318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 0x7f495c154c90 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:58.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4960a00640 1 -- 172.21.15.113:0/40031318 >> 172.21.15.113:0/40031318 conn(0x7f495c066db0 msgr2=0x7f495c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:58.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4960a00640 1 -- 172.21.15.113:0/40031318 shutdown_connections 2024-03-04T08:25:58.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.273+0000 7f4960a00640 1 -- 172.21.15.113:0/40031318 wait complete. 2024-03-04T08:25:58.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4960a00640 1 Processor -- start 2024-03-04T08:25:58.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4960a00640 1 -- start start 2024-03-04T08:25:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4960a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c070120 0x7f495c14eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4960a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495c0c8d80 0x7f495c14bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4960a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 0x7f495c1e0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4960a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c06fb80 con 0x7f495c0ce570 2024-03-04T08:25:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4960a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c06c4f0 con 0x7f495c070120 2024-03-04T08:25:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4960a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495c06fd00 con 0x7f495c0c8d80 2024-03-04T08:25:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f495a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 0x7f495c1e0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:58.278 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f495a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 0x7f495c1e0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41562/0 (socket says 172.21.15.113:41562) 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f495a800640 1 -- 172.21.15.113:0/3010020254 learned_addr learned my addr 172.21.15.113:0/3010020254 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4959e00640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c070120 0x7f495c14eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4951400640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495c0c8d80 0x7f495c14bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f495a800640 1 -- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495c0c8d80 msgr2=0x7f495c14bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f495a800640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495c0c8d80 0x7f495c14bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f495a800640 1 -- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c070120 msgr2=0x7f495c14eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f495a800640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c070120 0x7f495c14eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f495a800640 1 -- 172.21.15.113:0/3010020254 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c1510b0 con 0x7f495c0ce570 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4951400640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495c0c8d80 0x7f495c14bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f495a800640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 0x7f495c1e0e80 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f495400c3c0 tx=0x7f495400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4952a00640 1 -- 172.21.15.113:0/3010020254 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954005220 con 0x7f495c0ce570 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4952a00640 1 -- 172.21.15.113:0/3010020254 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49540053c0 con 0x7f495c0ce570 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4960a00640 1 -- 172.21.15.113:0/3010020254 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495c151350 con 0x7f495c0ce570 2024-03-04T08:25:58.279 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4952a00640 1 -- 172.21.15.113:0/3010020254 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954007270 con 0x7f495c0ce570 2024-03-04T08:25:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.277+0000 7f4960a00640 1 -- 172.21.15.113:0/3010020254 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495c1501c0 con 0x7f495c0ce570 2024-03-04T08:25:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.281+0000 7f4960a00640 1 -- 172.21.15.113:0/3010020254 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495c075c00 con 0x7f495c0ce570 2024-03-04T08:25:58.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.281+0000 7f4952a00640 1 -- 172.21.15.113:0/3010020254 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4954019020 con 0x7f495c0ce570 2024-03-04T08:25:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.281+0000 7f4952a00640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4938066f60 0x7f4938069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:25:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.281+0000 7f4952a00640 1 -- 172.21.15.113:0/3010020254 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f4954084d10 con 0x7f495c0ce570 2024-03-04T08:25:58.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.281+0000 7f4959e00640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4938066f60 0x7f4938069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:25:58.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.281+0000 7f4959e00640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4938066f60 0x7f4938069420 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f494c0b1000 tx=0x7f494c0aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:25:58.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.285+0000 7f4952a00640 1 -- 172.21.15.113:0/3010020254 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f495408a020 con 0x7f495c0ce570 2024-03-04T08:25:58.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.477+0000 7f4960a00640 1 -- 172.21.15.113:0/3010020254 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f495c06ff00 con 0x7f495c0ce570 2024-03-04T08:25:58.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.477+0000 7f4952a00640 1 -- 172.21.15.113:0/3010020254 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f4954055270 con 0x7f495c0ce570 2024-03-04T08:25:58.478 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:25:58.478 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:25:58.478 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:25:58.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 -- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4938066f60 msgr2=0x7f4938069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:58.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4938066f60 0x7f4938069420 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f494c0b1000 tx=0x7f494c0aa000 comp rx=0 tx=0).stop 2024-03-04T08:25:58.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 -- 172.21.15.113:0/3010020254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 msgr2=0x7f495c1e0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:25:58.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 0x7f495c1e0e80 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f495400c3c0 tx=0x7f495400c7c0 comp rx=0 tx=0).stop 2024-03-04T08:25:58.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 -- 172.21.15.113:0/3010020254 shutdown_connections 2024-03-04T08:25:58.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4938066f60 0x7f4938069420 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:58.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f495c0ce570 0x7f495c1e0e80 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:58.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f495c0c8d80 0x7f495c14bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:58.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 --2- 172.21.15.113:0/3010020254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c070120 0x7f495c14eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:25:58.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 -- 172.21.15.113:0/3010020254 >> 172.21.15.113:0/3010020254 conn(0x7f495c066db0 msgr2=0x7f495c0c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:25:58.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 -- 172.21.15.113:0/3010020254 shutdown_connections 2024-03-04T08:25:58.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:25:58.481+0000 7f4960a00640 1 -- 172.21.15.113:0/3010020254 wait complete. 2024-03-04T08:25:58.499 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:00.500 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:00.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.641+0000 7fdc6e200640 1 Processor -- start 2024-03-04T08:26:00.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.641+0000 7fdc6e200640 1 -- start start 2024-03-04T08:26:00.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.641+0000 7fdc6e200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdc680c8200 0x7fdc68070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:00.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.641+0000 7fdc6e200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdc68152bd0 0x7fdc681587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:00.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.641+0000 7fdc6e200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc681591f0 con 0x7fdc681527f0 2024-03-04T08:26:00.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.641+0000 7fdc6e200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc68071df0 con 0x7fdc680c8200 2024-03-04T08:26:00.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.641+0000 7fdc6e200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc681597b0 con 0x7fdc68152bd0 2024-03-04T08:26:00.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.641+0000 7fdc5fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdc680c8200 0x7fdc68070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:00.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc5fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdc680c8200 0x7fdc68070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41572/0 (socket says 172.21.15.113:41572) 2024-03-04T08:26:00.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc5fe00640 1 -- 172.21.15.113:0/569009285 learned_addr learned my addr 172.21.15.113:0/569009285 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:00.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc67e00640 1 --2- 172.21.15.113:0/569009285 >> v2:172.21.15.136:3300/0 conn(0x7fdc68152bd0 0x7fdc681587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:00.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc5fe00640 1 -- 172.21.15.113:0/569009285 >> v2:172.21.15.136:3300/0 conn(0x7fdc68152bd0 msgr2=0x7fdc681587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:00.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc5fe00640 1 --2- 172.21.15.113:0/569009285 >> v2:172.21.15.136:3300/0 conn(0x7fdc68152bd0 0x7fdc681587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:00.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc5fe00640 1 -- 172.21.15.113:0/569009285 >> v1:172.21.15.136:6789/0 conn(0x7fdc681527f0 legacy=0x7fdc68072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:00.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc5fe00640 1 -- 172.21.15.113:0/569009285 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc681594d0 con 0x7fdc680c8200 2024-03-04T08:26:00.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc5fe00640 1 --2- 172.21.15.113:0/569009285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc680c8200 0x7fdc68070d40 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fdc680712b0 tx=0x7fdc5001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec29557b6ae044d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:00.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc66a00640 1 -- 172.21.15.113:0/569009285 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500ab430 con 0x7fdc680c8200 2024-03-04T08:26:00.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc66a00640 1 -- 172.21.15.113:0/569009285 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc500b0070 con 0x7fdc680c8200 2024-03-04T08:26:00.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc66a00640 1 -- 172.21.15.113:0/569009285 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500ab7f0 con 0x7fdc680c8200 2024-03-04T08:26:00.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc6e200640 1 -- 172.21.15.113:0/569009285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc680c8200 msgr2=0x7fdc68070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:00.647 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.645+0000 7fdc6e200640 1 --2- 172.21.15.113:0/569009285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc680c8200 0x7fdc68070d40 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fdc680712b0 tx=0x7fdc5001c520 comp rx=0 tx=0).stop 2024-03-04T08:26:00.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 -- 172.21.15.113:0/569009285 shutdown_connections 2024-03-04T08:26:00.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 --2- 172.21.15.113:0/569009285 >> v2:172.21.15.136:3300/0 conn(0x7fdc68152bd0 0x7fdc681587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:00.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 --2- 172.21.15.113:0/569009285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc680c8200 0x7fdc68070d40 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:00.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 -- 172.21.15.113:0/569009285 >> 172.21.15.113:0/569009285 conn(0x7fdc6806e820 msgr2=0x7fdc6806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:00.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 -- 172.21.15.113:0/569009285 shutdown_connections 2024-03-04T08:26:00.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 -- 172.21.15.113:0/569009285 wait complete. 2024-03-04T08:26:00.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 Processor -- start 2024-03-04T08:26:00.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 -- start start 2024-03-04T08:26:00.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc680c8200 0x7fdc680c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:00.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc681527f0 0x7fdc68148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:00.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc68152bd0 0x7fdc6815ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:00.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc681654f0 con 0x7fdc681527f0 2024-03-04T08:26:00.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68165370 con 0x7fdc68152bd0 2024-03-04T08:26:00.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc68165670 con 0x7fdc680c8200 2024-03-04T08:26:00.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc681527f0 0x7fdc68148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc681527f0 0x7fdc68148990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41578/0 (socket says 172.21.15.113:41578) 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5fe00640 1 -- 172.21.15.113:0/2891294409 learned_addr learned my addr 172.21.15.113:0/2891294409 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc67e00640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc68152bd0 0x7fdc6815ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc67400640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc680c8200 0x7fdc680c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5fe00640 1 -- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc680c8200 msgr2=0x7fdc680c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5fe00640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc680c8200 0x7fdc680c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5fe00640 1 -- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc68152bd0 msgr2=0x7fdc6815ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5fe00640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc68152bd0 0x7fdc6815ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5fe00640 1 -- 172.21.15.113:0/2891294409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc681f4b50 con 0x7fdc681527f0 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5fe00640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc681527f0 0x7fdc68148990 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fdc5001a150 tx=0x7fdc5001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5f400640 1 -- 172.21.15.113:0/2891294409 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500be5f0 con 0x7fdc681527f0 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5f400640 1 -- 172.21.15.113:0/2891294409 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc500be790 con 0x7fdc681527f0 2024-03-04T08:26:00.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc5f400640 1 -- 172.21.15.113:0/2891294409 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500beb60 con 0x7fdc681527f0 2024-03-04T08:26:00.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.649+0000 7fdc6e200640 1 -- 172.21.15.113:0/2891294409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc681f7dc0 con 0x7fdc681527f0 2024-03-04T08:26:00.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.653+0000 7fdc6e200640 1 -- 172.21.15.113:0/2891294409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc681f4070 con 0x7fdc681527f0 2024-03-04T08:26:00.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.653+0000 7fdc6e200640 1 -- 172.21.15.113:0/2891294409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc68075c00 con 0x7fdc681527f0 2024-03-04T08:26:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.653+0000 7fdc5f400640 1 -- 172.21.15.113:0/2891294409 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc500b43f0 con 0x7fdc681527f0 2024-03-04T08:26:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.653+0000 7fdc5f400640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc44066f60 0x7fdc44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.653+0000 7fdc5f400640 1 -- 172.21.15.113:0/2891294409 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fdc500a8030 con 0x7fdc681527f0 2024-03-04T08:26:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.653+0000 7fdc67400640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc44066f60 0x7fdc44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:00.654 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.653+0000 7fdc67400640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc44066f60 0x7fdc44069420 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fdc58001650 tx=0x7fdc58009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:00.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.657+0000 7fdc5f400640 1 -- 172.21.15.113:0/2891294409 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdc50016110 con 0x7fdc681527f0 2024-03-04T08:26:00.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 -- 172.21.15.113:0/2891294409 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc680c79d0 con 0x7fdc681527f0 2024-03-04T08:26:00.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc5f400640 1 -- 172.21.15.113:0/2891294409 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fdc500f0370 con 0x7fdc681527f0 2024-03-04T08:26:00.845 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:00.845 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:00.845 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:00.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 -- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc44066f60 msgr2=0x7fdc44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc44066f60 0x7fdc44069420 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fdc58001650 tx=0x7fdc58009040 comp rx=0 tx=0).stop 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 -- 172.21.15.113:0/2891294409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc681527f0 msgr2=0x7fdc68148990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc681527f0 0x7fdc68148990 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fdc5001a150 tx=0x7fdc5001ad60 comp rx=0 tx=0).stop 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 -- 172.21.15.113:0/2891294409 shutdown_connections 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc68152bd0 0x7fdc6815ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdc44066f60 0x7fdc44069420 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc681527f0 0x7fdc68148990 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 --2- 172.21.15.113:0/2891294409 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdc680c8200 0x7fdc680c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 -- 172.21.15.113:0/2891294409 >> 172.21.15.113:0/2891294409 conn(0x7fdc6806e820 msgr2=0x7fdc68072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 -- 172.21.15.113:0/2891294409 shutdown_connections 2024-03-04T08:26:00.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:00.845+0000 7fdc6e200640 1 -- 172.21.15.113:0/2891294409 wait complete. 2024-03-04T08:26:00.866 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:02.867 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:03.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.013+0000 7f5b9c200640 1 Processor -- start 2024-03-04T08:26:03.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.013+0000 7f5b9c200640 1 -- start start 2024-03-04T08:26:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b9c200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b940c22e0 0x7f5b9415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b9c200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5b940c2d80 0x7f5b9415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b9c200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b940c27f0 con 0x7f5b9414e4c0 2024-03-04T08:26:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b9c200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9415ad60 con 0x7f5b940c22e0 2024-03-04T08:26:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b9c200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b940c2a90 con 0x7f5b940c2d80 2024-03-04T08:26:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b98e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b940c22e0 0x7f5b9415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b98e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b940c22e0 0x7f5b9415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41584/0 (socket says 172.21.15.113:41584) 2024-03-04T08:26:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b99800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f5b9414e4c0 0x7f5b94072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60552/0 (socket says 172.21.15.113:60552) 2024-03-04T08:26:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b98e00640 1 -- 172.21.15.113:0/3665852831 learned_addr learned my addr 172.21.15.113:0/3665852831 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:03.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b9a200640 1 --2- 172.21.15.113:0/3665852831 >> v2:172.21.15.136:3300/0 conn(0x7f5b940c2d80 0x7f5b9415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:03.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b98e00640 1 -- 172.21.15.113:0/3665852831 >> v2:172.21.15.136:3300/0 conn(0x7f5b940c2d80 msgr2=0x7f5b9415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:03.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b98e00640 1 --2- 172.21.15.113:0/3665852831 >> v2:172.21.15.136:3300/0 conn(0x7f5b940c2d80 0x7f5b9415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b98e00640 1 -- 172.21.15.113:0/3665852831 >> v1:172.21.15.113:6789/0 conn(0x7f5b9414e4c0 legacy=0x7f5b94072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:03.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b98e00640 1 -- 172.21.15.113:0/3665852831 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94070c60 con 0x7f5b940c22e0 2024-03-04T08:26:03.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b9a200640 1 --2- 172.21.15.113:0/3665852831 >> v2:172.21.15.136:3300/0 conn(0x7f5b940c2d80 0x7f5b9415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:03.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b93e00640 1 -- 172.21.15.113:0/3665852831 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306789991 0 0) 0x7f5b940c27f0 con 0x7f5b9414e4c0 2024-03-04T08:26:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b98e00640 1 --2- 172.21.15.113:0/3665852831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940c22e0 0x7f5b9415b1f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5b940c2700 tx=0x7f5b8001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6f97b63ee5a19e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b93e00640 1 -- 172.21.15.113:0/3665852831 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800b4070 con 0x7f5b940c22e0 2024-03-04T08:26:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b93e00640 1 -- 172.21.15.113:0/3665852831 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b800a7ad0 con 0x7f5b940c22e0 2024-03-04T08:26:03.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.017+0000 7f5b93e00640 1 -- 172.21.15.113:0/3665852831 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b800af440 con 0x7f5b940c22e0 2024-03-04T08:26:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 -- 172.21.15.113:0/3665852831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940c22e0 msgr2=0x7f5b9415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 --2- 172.21.15.113:0/3665852831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940c22e0 0x7f5b9415b1f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5b940c2700 tx=0x7f5b8001ebf0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 -- 172.21.15.113:0/3665852831 shutdown_connections 2024-03-04T08:26:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 --2- 172.21.15.113:0/3665852831 >> v2:172.21.15.136:3300/0 conn(0x7f5b940c2d80 0x7f5b9415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 --2- 172.21.15.113:0/3665852831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b940c22e0 0x7f5b9415b1f0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 -- 172.21.15.113:0/3665852831 >> 172.21.15.113:0/3665852831 conn(0x7f5b9406e820 msgr2=0x7f5b9406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 -- 172.21.15.113:0/3665852831 shutdown_connections 2024-03-04T08:26:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 -- 172.21.15.113:0/3665852831 wait complete. 2024-03-04T08:26:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 Processor -- start 2024-03-04T08:26:03.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 -- start start 2024-03-04T08:26:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b940c22e0 0x7f5b94153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b940c2d80 0x7f5b940b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b9414e4c0 0x7f5b94159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b940aede0 con 0x7f5b9414e4c0 2024-03-04T08:26:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9414e8a0 con 0x7f5b940c2d80 2024-03-04T08:26:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9c200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b94073390 con 0x7f5b940c22e0 2024-03-04T08:26:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b9414e4c0 0x7f5b94159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:03.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b9414e4c0 0x7f5b94159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41600/0 (socket says 172.21.15.113:41600) 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9a200640 1 -- 172.21.15.113:0/4243683350 learned_addr learned my addr 172.21.15.113:0/4243683350 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b99800640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b940c22e0 0x7f5b94153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b98e00640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b940c2d80 0x7f5b940b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9a200640 1 -- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b940c22e0 msgr2=0x7f5b94153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9a200640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b940c22e0 0x7f5b94153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9a200640 1 -- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b940c2d80 msgr2=0x7f5b940b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9a200640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b940c2d80 0x7f5b940b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b9a200640 1 -- 172.21.15.113:0/4243683350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b941f7980 con 0x7f5b9414e4c0 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.021+0000 7f5b99800640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b940c22e0 0x7f5b94153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b9a200640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b9414e4c0 0x7f5b94159df0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00d800 tx=0x7f5b8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b91600640 1 -- 172.21.15.113:0/4243683350 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00b6c0 con 0x7f5b9414e4c0 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b91600640 1 -- 172.21.15.113:0/4243683350 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c019070 con 0x7f5b9414e4c0 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b91600640 1 -- 172.21.15.113:0/4243683350 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00baa0 con 0x7f5b9414e4c0 2024-03-04T08:26:03.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b9c200640 1 -- 172.21.15.113:0/4243683350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b941f76a0 con 0x7f5b9414e4c0 2024-03-04T08:26:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b9c200640 1 -- 172.21.15.113:0/4243683350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b94153010 con 0x7f5b9414e4c0 2024-03-04T08:26:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b9c200640 1 -- 172.21.15.113:0/4243683350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b94075c00 con 0x7f5b9414e4c0 2024-03-04T08:26:03.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b91600640 1 -- 172.21.15.113:0/4243683350 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b8c00bc40 con 0x7f5b9414e4c0 2024-03-04T08:26:03.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b91600640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b74066f90 0x7f5b74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:03.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b91600640 1 -- 172.21.15.113:0/4243683350 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f5b8c085e20 con 0x7f5b9414e4c0 2024-03-04T08:26:03.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.025+0000 7f5b99800640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b74066f90 0x7f5b74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.029+0000 7f5b99800640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b74066f90 0x7f5b74069450 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5b88005fd0 tx=0x7f5b88005b80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:03.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.029+0000 7f5b91600640 1 -- 172.21.15.113:0/4243683350 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5b8c0563e0 con 0x7f5b9414e4c0 2024-03-04T08:26:03.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.217+0000 7f5b9c200640 1 -- 172.21.15.113:0/4243683350 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b94155190 con 0x7f5b9414e4c0 2024-03-04T08:26:03.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.217+0000 7f5b91600640 1 -- 172.21.15.113:0/4243683350 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f5b8c055ce0 con 0x7f5b9414e4c0 2024-03-04T08:26:03.218 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:03.218 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:03.219 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:03.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 -- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b74066f90 msgr2=0x7f5b74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:03.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b74066f90 0x7f5b74069450 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f5b88005fd0 tx=0x7f5b88005b80 comp rx=0 tx=0).stop 2024-03-04T08:26:03.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 -- 172.21.15.113:0/4243683350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b9414e4c0 msgr2=0x7f5b94159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:03.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b9414e4c0 0x7f5b94159df0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00d800 tx=0x7f5b8c00dcc0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 -- 172.21.15.113:0/4243683350 shutdown_connections 2024-03-04T08:26:03.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5b74066f90 0x7f5b74069450 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b9414e4c0 0x7f5b94159df0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b940c2d80 0x7f5b940b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 --2- 172.21.15.113:0/4243683350 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5b940c22e0 0x7f5b94153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:03.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 -- 172.21.15.113:0/4243683350 >> 172.21.15.113:0/4243683350 conn(0x7f5b9406e820 msgr2=0x7f5b9414b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:03.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 -- 172.21.15.113:0/4243683350 shutdown_connections 2024-03-04T08:26:03.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:03.221+0000 7f5b9c200640 1 -- 172.21.15.113:0/4243683350 wait complete. 2024-03-04T08:26:03.240 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:26:03.242 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:26:05.243 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:05.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983ec00640 1 Processor -- start 2024-03-04T08:26:05.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983ec00640 1 -- start start 2024-03-04T08:26:05.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983ec00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9838152a90 0x7f98380730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:05.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983ec00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f983815bce0 0x7f983815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:05.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983ec00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983806f1b0 con 0x7f983806cd10 2024-03-04T08:26:05.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983ec00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9838152e70 con 0x7f9838152a90 2024-03-04T08:26:05.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983ec00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983806c320 con 0x7f983815bce0 2024-03-04T08:26:05.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983d800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9838152a90 0x7f98380730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:05.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983ce00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f983806cd10 0x7f983806d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60558/0 (socket says 172.21.15.113:60558) 2024-03-04T08:26:05.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983ce00640 1 -- 172.21.15.113:0/3603459869 learned_addr learned my addr 172.21.15.113:0/3603459869 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:05.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983e200640 1 --2- 172.21.15.113:0/3603459869 >> v2:172.21.15.136:3300/0 conn(0x7f983815bce0 0x7f983815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:05.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983d800640 1 -- 172.21.15.113:0/3603459869 >> v2:172.21.15.136:3300/0 conn(0x7f983815bce0 msgr2=0x7f983815c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:05.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983d800640 1 --2- 172.21.15.113:0/3603459869 >> v2:172.21.15.136:3300/0 conn(0x7f983815bce0 0x7f983815c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:05.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983d800640 1 -- 172.21.15.113:0/3603459869 >> v1:172.21.15.113:6789/0 conn(0x7f983806cd10 legacy=0x7f983806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:05.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f983d800640 1 -- 172.21.15.113:0/3603459869 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9838071580 con 0x7f9838152a90 2024-03-04T08:26:05.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.393+0000 7f9837e00640 1 -- 172.21.15.113:0/3603459869 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658296003 0 0) 0x7f983806f1b0 con 0x7f983806cd10 2024-03-04T08:26:05.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983d800640 1 --2- 172.21.15.113:0/3603459869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 0x7f98380730d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f983815cde0 tx=0x7f982c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa285a40f5f9a38 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:05.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f9837e00640 1 -- 172.21.15.113:0/3603459869 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0b0070 con 0x7f9838152a90 2024-03-04T08:26:05.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f9837e00640 1 -- 172.21.15.113:0/3603459869 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c0ab600 con 0x7f9838152a90 2024-03-04T08:26:05.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f9837e00640 1 -- 172.21.15.113:0/3603459869 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c0ab9d0 con 0x7f9838152a90 2024-03-04T08:26:05.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983ec00640 1 -- 172.21.15.113:0/3603459869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 msgr2=0x7f98380730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:05.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983ec00640 1 --2- 172.21.15.113:0/3603459869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 0x7f98380730d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f983815cde0 tx=0x7f982c01c480 comp rx=0 tx=0).stop 2024-03-04T08:26:05.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983ec00640 1 -- 172.21.15.113:0/3603459869 shutdown_connections 2024-03-04T08:26:05.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983ec00640 1 --2- 172.21.15.113:0/3603459869 >> v2:172.21.15.136:3300/0 conn(0x7f983815bce0 0x7f983815c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:05.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983ec00640 1 --2- 172.21.15.113:0/3603459869 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 0x7f98380730d0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:05.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983ec00640 1 -- 172.21.15.113:0/3603459869 >> 172.21.15.113:0/3603459869 conn(0x7f983806e2a0 msgr2=0x7f983806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:05.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983ec00640 1 -- 172.21.15.113:0/3603459869 shutdown_connections 2024-03-04T08:26:05.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983ec00640 1 -- 172.21.15.113:0/3603459869 wait complete. 2024-03-04T08:26:05.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983ec00640 1 Processor -- start 2024-03-04T08:26:05.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.397+0000 7f983ec00640 1 -- start start 2024-03-04T08:26:05.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ec00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f983806cd10 0x7f9838151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:05.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 0x7f98380cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ec00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f983815bce0 0x7f98381f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9838165890 con 0x7f9838152a90 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ec00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9838071330 con 0x7f983815bce0 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ec00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9838165a10 con 0x7f983806cd10 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 0x7f98380cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 0x7f98380cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41614/0 (socket says 172.21.15.113:41614) 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ce00640 1 -- 172.21.15.113:0/2192492078 learned_addr learned my addr 172.21.15.113:0/2192492078 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983d800640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f983806cd10 0x7f9838151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983e200640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f983815bce0 0x7f98381f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ce00640 1 -- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f983806cd10 msgr2=0x7f9838151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ce00640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f983806cd10 0x7f9838151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ce00640 1 -- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f983815bce0 msgr2=0x7f98381f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:05.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ce00640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f983815bce0 0x7f98381f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:05.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ce00640 1 -- 172.21.15.113:0/2192492078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380c0ed0 con 0x7f9838152a90 2024-03-04T08:26:05.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ce00640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 0x7f98380cb570 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f982400c480 tx=0x7f982400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:05.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f9835600640 1 -- 172.21.15.113:0/2192492078 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824018070 con 0x7f9838152a90 2024-03-04T08:26:05.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f9835600640 1 -- 172.21.15.113:0/2192492078 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9824007500 con 0x7f9838152a90 2024-03-04T08:26:05.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ec00640 1 -- 172.21.15.113:0/2192492078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98380c11b0 con 0x7f9838152a90 2024-03-04T08:26:05.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f9835600640 1 -- 172.21.15.113:0/2192492078 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824004a30 con 0x7f9838152a90 2024-03-04T08:26:05.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ec00640 1 -- 172.21.15.113:0/2192492078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983813fc10 con 0x7f9838152a90 2024-03-04T08:26:05.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f983ec00640 1 -- 172.21.15.113:0/2192492078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9838075a80 con 0x7f9838152a90 2024-03-04T08:26:05.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.401+0000 7f9835600640 1 -- 172.21.15.113:0/2192492078 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f982401b070 con 0x7f9838152a90 2024-03-04T08:26:05.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.405+0000 7f9835600640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9818066f60 0x7f9818069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:05.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.405+0000 7f9835600640 1 -- 172.21.15.113:0/2192492078 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f9824084f00 con 0x7f9838152a90 2024-03-04T08:26:05.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.405+0000 7f983d800640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9818066f60 0x7f9818069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:05.404 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.405+0000 7f983d800640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9818066f60 0x7f9818069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f982c0ad000 tx=0x7f982c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:05.407 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.409+0000 7f9835600640 1 -- 172.21.15.113:0/2192492078 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f98240553d0 con 0x7f9838152a90 2024-03-04T08:26:05.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 -- 172.21.15.113:0/2192492078 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98380c5f30 con 0x7f9838152a90 2024-03-04T08:26:05.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f9835600640 1 -- 172.21.15.113:0/2192492078 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f9824054cd0 con 0x7f9838152a90 2024-03-04T08:26:05.597 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:05.597 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:05.598 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:05.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 -- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9818066f60 msgr2=0x7f9818069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:05.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9818066f60 0x7f9818069420 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f982c0ad000 tx=0x7f982c0a6000 comp rx=0 tx=0).stop 2024-03-04T08:26:05.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 -- 172.21.15.113:0/2192492078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 msgr2=0x7f98380cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:05.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 0x7f98380cb570 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f982400c480 tx=0x7f982400c940 comp rx=0 tx=0).stop 2024-03-04T08:26:05.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 -- 172.21.15.113:0/2192492078 shutdown_connections 2024-03-04T08:26:05.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f983815bce0 0x7f98381f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:05.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9818066f60 0x7f9818069420 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:05.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9838152a90 0x7f98380cb570 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:05.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 --2- 172.21.15.113:0/2192492078 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f983806cd10 0x7f9838151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:05.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 -- 172.21.15.113:0/2192492078 >> 172.21.15.113:0/2192492078 conn(0x7f983806e2a0 msgr2=0x7f983806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:05.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 -- 172.21.15.113:0/2192492078 shutdown_connections 2024-03-04T08:26:05.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:05.597+0000 7f983ec00640 1 -- 172.21.15.113:0/2192492078 wait complete. 2024-03-04T08:26:05.617 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:07.619 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:07.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.765+0000 7f3c59600640 1 Processor -- start 2024-03-04T08:26:07.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.765+0000 7f3c59600640 1 -- start start 2024-03-04T08:26:07.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c59600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c5414e4c0 0x7f3c54072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:07.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c59600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3c540c22e0 0x7f3c540731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:07.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c59600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540c27f0 con 0x7f3c540c2d80 2024-03-04T08:26:07.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c59600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c5415ad60 con 0x7f3c5414e4c0 2024-03-04T08:26:07.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c59600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c540c2a90 con 0x7f3c540c22e0 2024-03-04T08:26:07.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c52a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c5414e4c0 0x7f3c54072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:07.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c52a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3c5414e4c0 0x7f3c54072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57620/0 (socket says 172.21.15.113:57620) 2024-03-04T08:26:07.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c52a00640 1 -- 172.21.15.113:0/1245532305 learned_addr learned my addr 172.21.15.113:0/1245532305 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:07.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c53400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3c540c2d80 0x7f3c5415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34506/0 (socket says 172.21.15.113:34506) 2024-03-04T08:26:07.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c4a000640 1 --2- 172.21.15.113:0/1245532305 >> v2:172.21.15.136:3300/0 conn(0x7f3c540c22e0 0x7f3c540731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:07.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c52a00640 1 -- 172.21.15.113:0/1245532305 >> v2:172.21.15.136:3300/0 conn(0x7f3c540c22e0 msgr2=0x7f3c540731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:07.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c52a00640 1 --2- 172.21.15.113:0/1245532305 >> v2:172.21.15.136:3300/0 conn(0x7f3c540c22e0 0x7f3c540731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:07.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c52a00640 1 -- 172.21.15.113:0/1245532305 >> v1:172.21.15.113:6789/0 conn(0x7f3c540c2d80 legacy=0x7f3c5415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:07.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c52a00640 1 -- 172.21.15.113:0/1245532305 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c54070c60 con 0x7f3c5414e4c0 2024-03-04T08:26:07.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.769+0000 7f3c52000640 1 -- 172.21.15.113:0/1245532305 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665184112 0 0) 0x7f3c540c27f0 con 0x7f3c540c2d80 2024-03-04T08:26:07.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c52a00640 1 --2- 172.21.15.113:0/1245532305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c5414e4c0 0x7f3c54072c80 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f3c540cd230 tx=0x7f3c440213e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff2af250bc7442d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:07.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c52000640 1 -- 172.21.15.113:0/1245532305 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c440b6070 con 0x7f3c5414e4c0 2024-03-04T08:26:07.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c52000640 1 -- 172.21.15.113:0/1245532305 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c440b1600 con 0x7f3c5414e4c0 2024-03-04T08:26:07.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c52000640 1 -- 172.21.15.113:0/1245532305 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c440b19d0 con 0x7f3c5414e4c0 2024-03-04T08:26:07.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 -- 172.21.15.113:0/1245532305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c5414e4c0 msgr2=0x7f3c54072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:07.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 --2- 172.21.15.113:0/1245532305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c5414e4c0 0x7f3c54072c80 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f3c540cd230 tx=0x7f3c440213e0 comp rx=0 tx=0).stop 2024-03-04T08:26:07.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 -- 172.21.15.113:0/1245532305 shutdown_connections 2024-03-04T08:26:07.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 --2- 172.21.15.113:0/1245532305 >> v2:172.21.15.136:3300/0 conn(0x7f3c540c22e0 0x7f3c540731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:07.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 --2- 172.21.15.113:0/1245532305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c5414e4c0 0x7f3c54072c80 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:07.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 -- 172.21.15.113:0/1245532305 >> 172.21.15.113:0/1245532305 conn(0x7f3c5406e820 msgr2=0x7f3c5406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:07.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 -- 172.21.15.113:0/1245532305 shutdown_connections 2024-03-04T08:26:07.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 -- 172.21.15.113:0/1245532305 wait complete. 2024-03-04T08:26:07.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 Processor -- start 2024-03-04T08:26:07.773 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 -- start start 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c540c22e0 0x7f3c5414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c540c2d80 0x7f3c54143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c5414e4c0 0x7f3c54144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54071500 con 0x7f3c540c22e0 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54071220 con 0x7f3c540c2d80 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.773+0000 7f3c59600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5414e8a0 con 0x7f3c5414e4c0 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c52a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c540c22e0 0x7f3c5414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c52a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c540c22e0 0x7f3c5414d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57632/0 (socket says 172.21.15.113:57632) 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c52a00640 1 -- 172.21.15.113:0/679651263 learned_addr learned my addr 172.21.15.113:0/679651263 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c4a000640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c540c2d80 0x7f3c54143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c52a00640 1 -- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c5414e4c0 msgr2=0x7f3c54144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c53400640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c5414e4c0 0x7f3c54144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:07.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c52a00640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c5414e4c0 0x7f3c54144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c52a00640 1 -- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c540c2d80 msgr2=0x7f3c54143b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c52a00640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c540c2d80 0x7f3c54143b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c52a00640 1 -- 172.21.15.113:0/679651263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c54154980 con 0x7f3c540c22e0 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c4a000640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c540c2d80 0x7f3c54143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c53400640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c5414e4c0 0x7f3c54144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c52a00640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c540c22e0 0x7f3c5414d1b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f3c4401dff0 tx=0x7f3c4401fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c4b400640 1 -- 172.21.15.113:0/679651263 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c440b6070 con 0x7f3c540c22e0 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c4b400640 1 -- 172.21.15.113:0/679651263 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c440b07d0 con 0x7f3c540c22e0 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c4b400640 1 -- 172.21.15.113:0/679651263 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c440b0bc0 con 0x7f3c540c22e0 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c59600640 1 -- 172.21.15.113:0/679651263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c541f1d40 con 0x7f3c540c22e0 2024-03-04T08:26:07.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c59600640 1 -- 172.21.15.113:0/679651263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c541f1fe0 con 0x7f3c540c22e0 2024-03-04T08:26:07.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c59600640 1 -- 172.21.15.113:0/679651263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c54075c00 con 0x7f3c540c22e0 2024-03-04T08:26:07.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c4b400640 1 -- 172.21.15.113:0/679651263 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c440c3060 con 0x7f3c540c22e0 2024-03-04T08:26:07.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.777+0000 7f3c4b400640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c30066f60 0x7f3c30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:07.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.781+0000 7f3c4b400640 1 -- 172.21.15.113:0/679651263 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f3c44122e60 con 0x7f3c540c22e0 2024-03-04T08:26:07.779 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.781+0000 7f3c4a000640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c30066f60 0x7f3c30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:07.780 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.781+0000 7f3c4a000640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c30066f60 0x7f3c30069420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3c541e8030 tx=0x7f3c3c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:07.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.785+0000 7f3c4b400640 1 -- 172.21.15.113:0/679651263 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3c4401af40 con 0x7f3c540c22e0 2024-03-04T08:26:07.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.969+0000 7f3c59600640 1 -- 172.21.15.113:0/679651263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c54068a60 con 0x7f3c540c22e0 2024-03-04T08:26:07.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.969+0000 7f3c4b400640 1 -- 172.21.15.113:0/679651263 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f3c440f34e0 con 0x7f3c540c22e0 2024-03-04T08:26:07.970 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:07.970 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:07.970 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 -- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c30066f60 msgr2=0x7f3c30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c30066f60 0x7f3c30069420 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3c541e8030 tx=0x7f3c3c008040 comp rx=0 tx=0).stop 2024-03-04T08:26:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 -- 172.21.15.113:0/679651263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c540c22e0 msgr2=0x7f3c5414d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c540c22e0 0x7f3c5414d1b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f3c4401dff0 tx=0x7f3c4401fc20 comp rx=0 tx=0).stop 2024-03-04T08:26:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 -- 172.21.15.113:0/679651263 shutdown_connections 2024-03-04T08:26:07.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3c5414e4c0 0x7f3c54144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:07.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c540c2d80 0x7f3c54143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:07.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c30066f60 0x7f3c30069420 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:07.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 --2- 172.21.15.113:0/679651263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3c540c22e0 0x7f3c5414d1b0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:07.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 -- 172.21.15.113:0/679651263 >> 172.21.15.113:0/679651263 conn(0x7f3c5406e820 msgr2=0x7f3c5406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:07.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 -- 172.21.15.113:0/679651263 shutdown_connections 2024-03-04T08:26:07.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:07.973+0000 7f3c59600640 1 -- 172.21.15.113:0/679651263 wait complete. 2024-03-04T08:26:07.991 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:09.992 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:10.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.137+0000 7f94d7e00640 1 Processor -- start 2024-03-04T08:26:10.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.137+0000 7f94d7e00640 1 -- start start 2024-03-04T08:26:10.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154c00 0x7f94d00c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:10.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f94d00c7700 0x7f94d015abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:10.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d00c7d80 con 0x7f94d0154fe0 2024-03-04T08:26:10.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d015efb0 con 0x7f94d00c7700 2024-03-04T08:26:10.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00c6b70 con 0x7f94d0154c00 2024-03-04T08:26:10.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154c00 0x7f94d00c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:10.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f94d00c7700 0x7f94d015abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:10.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154c00 0x7f94d00c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:33550/0 (socket says 172.21.15.113:33550) 2024-03-04T08:26:10.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d5400640 1 -- 172.21.15.113:0/3629581983 learned_addr learned my addr 172.21.15.113:0/3629581983 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:10.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d5400640 1 -- 172.21.15.113:0/3629581983 >> v2:172.21.15.136:3300/0 conn(0x7f94d00c7700 msgr2=0x7f94d015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:10.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d5400640 1 --2- 172.21.15.113:0/3629581983 >> v2:172.21.15.136:3300/0 conn(0x7f94d00c7700 0x7f94d015abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:10.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d5400640 1 -- 172.21.15.113:0/3629581983 >> v1:172.21.15.136:6789/0 conn(0x7f94d0154fe0 legacy=0x7f94d00c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:10.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.141+0000 7f94d5400640 1 -- 172.21.15.113:0/3629581983 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d00c8060 con 0x7f94d0154c00 2024-03-04T08:26:10.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d5400640 1 --2- 172.21.15.113:0/3629581983 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154c00 0x7f94d00c1190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009870 tx=0x7f94c401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c25a84db802a11cc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:10.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94cfe00640 1 -- 172.21.15.113:0/3629581983 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c40b1070 con 0x7f94d0154c00 2024-03-04T08:26:10.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94cfe00640 1 -- 172.21.15.113:0/3629581983 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c40ac620 con 0x7f94d0154c00 2024-03-04T08:26:10.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94cfe00640 1 -- 172.21.15.113:0/3629581983 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c40ac9f0 con 0x7f94d0154c00 2024-03-04T08:26:10.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d7e00640 1 -- 172.21.15.113:0/3629581983 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154c00 msgr2=0x7f94d00c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:10.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d7e00640 1 --2- 172.21.15.113:0/3629581983 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154c00 0x7f94d00c1190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009870 tx=0x7f94c401c4d0 comp rx=0 tx=0).stop 2024-03-04T08:26:10.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d7e00640 1 -- 172.21.15.113:0/3629581983 shutdown_connections 2024-03-04T08:26:10.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d7e00640 1 --2- 172.21.15.113:0/3629581983 >> v2:172.21.15.136:3300/0 conn(0x7f94d00c7700 0x7f94d015abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:10.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d7e00640 1 --2- 172.21.15.113:0/3629581983 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154c00 0x7f94d00c1190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:10.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d7e00640 1 -- 172.21.15.113:0/3629581983 >> 172.21.15.113:0/3629581983 conn(0x7f94d006e820 msgr2=0x7f94d006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:10.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d7e00640 1 -- 172.21.15.113:0/3629581983 shutdown_connections 2024-03-04T08:26:10.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d7e00640 1 -- 172.21.15.113:0/3629581983 wait complete. 2024-03-04T08:26:10.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d7e00640 1 Processor -- start 2024-03-04T08:26:10.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.145+0000 7f94d7e00640 1 -- start start 2024-03-04T08:26:10.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d00c7700 0x7f94d0150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d0154c00 0x7f94d01508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154fe0 0x7f94d01f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00c23e0 con 0x7f94d0154c00 2024-03-04T08:26:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00c2260 con 0x7f94d00c7700 2024-03-04T08:26:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d0167c10 con 0x7f94d0154fe0 2024-03-04T08:26:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d4a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d0154c00 0x7f94d01508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d4a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d0154c00 0x7f94d01508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57642/0 (socket says 172.21.15.113:57642) 2024-03-04T08:26:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d4a00640 1 -- 172.21.15.113:0/2723255493 learned_addr learned my addr 172.21.15.113:0/2723255493 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d5400640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d00c7700 0x7f94d0150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:10.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d5e00640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154fe0 0x7f94d01f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:10.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d4a00640 1 -- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154fe0 msgr2=0x7f94d01f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:10.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d4a00640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154fe0 0x7f94d01f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:10.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d4a00640 1 -- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d00c7700 msgr2=0x7f94d0150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:10.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d4a00640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d00c7700 0x7f94d0150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:10.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d4a00640 1 -- 172.21.15.113:0/2723255493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d0150190 con 0x7f94d0154c00 2024-03-04T08:26:10.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d4a00640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d0154c00 0x7f94d01508c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00c370 tx=0x7f94bc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:10.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94cd600640 1 -- 172.21.15.113:0/2723255493 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0052d0 con 0x7f94d0154c00 2024-03-04T08:26:10.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94cd600640 1 -- 172.21.15.113:0/2723255493 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc005470 con 0x7f94d0154c00 2024-03-04T08:26:10.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d7e00640 1 -- 172.21.15.113:0/2723255493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d014fbf0 con 0x7f94d0154c00 2024-03-04T08:26:10.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94cd600640 1 -- 172.21.15.113:0/2723255493 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc007260 con 0x7f94d0154c00 2024-03-04T08:26:10.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d7e00640 1 -- 172.21.15.113:0/2723255493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d01554f0 con 0x7f94d0154c00 2024-03-04T08:26:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94d7e00640 1 -- 172.21.15.113:0/2723255493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d0075c00 con 0x7f94d0154c00 2024-03-04T08:26:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.149+0000 7f94cd600640 1 -- 172.21.15.113:0/2723255493 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94bc007400 con 0x7f94d0154c00 2024-03-04T08:26:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.153+0000 7f94cd600640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94b0066f60 0x7f94b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.153+0000 7f94cd600640 1 -- 172.21.15.113:0/2723255493 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f94bc084fe0 con 0x7f94d0154c00 2024-03-04T08:26:10.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.153+0000 7f94d5400640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94b0066f60 0x7f94b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:10.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.153+0000 7f94d5400640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94b0066f60 0x7f94b0069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f94c40ae000 tx=0x7f94c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:10.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.157+0000 7f94cd600640 1 -- 172.21.15.113:0/2723255493 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f94bc054e00 con 0x7f94d0154c00 2024-03-04T08:26:10.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.341+0000 7f94d7e00640 1 -- 172.21.15.113:0/2723255493 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94d00cc740 con 0x7f94d0154c00 2024-03-04T08:26:10.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.341+0000 7f94cd600640 1 -- 172.21.15.113:0/2723255493 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f94bc058ea0 con 0x7f94d0154c00 2024-03-04T08:26:10.343 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:10.343 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:10.343 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:10.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 -- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94b0066f60 msgr2=0x7f94b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:10.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94b0066f60 0x7f94b0069420 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f94c40ae000 tx=0x7f94c4008040 comp rx=0 tx=0).stop 2024-03-04T08:26:10.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 -- 172.21.15.113:0/2723255493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d0154c00 msgr2=0x7f94d01508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:10.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d0154c00 0x7f94d01508c0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00c370 tx=0x7f94bc00c830 comp rx=0 tx=0).stop 2024-03-04T08:26:10.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 -- 172.21.15.113:0/2723255493 shutdown_connections 2024-03-04T08:26:10.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f94d0154fe0 0x7f94d01f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:10.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f94b0066f60 0x7f94b0069420 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:10.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f94d0154c00 0x7f94d01508c0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:10.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 --2- 172.21.15.113:0/2723255493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94d00c7700 0x7f94d0150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:10.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 -- 172.21.15.113:0/2723255493 >> 172.21.15.113:0/2723255493 conn(0x7f94d006e820 msgr2=0x7f94d0074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:10.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 -- 172.21.15.113:0/2723255493 shutdown_connections 2024-03-04T08:26:10.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:10.345+0000 7f94d7e00640 1 -- 172.21.15.113:0/2723255493 wait complete. 2024-03-04T08:26:10.364 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:12.365 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:12.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cbb000640 1 Processor -- start 2024-03-04T08:26:12.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cbb000640 1 -- start start 2024-03-04T08:26:12.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cbb000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3cb41527f0 0x7f3cb4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:12.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cbb000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb4152bd0 0x7f3cb41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:12.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cbb000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb41591f0 con 0x7f3cb40c8200 2024-03-04T08:26:12.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cbb000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb4071df0 con 0x7f3cb41527f0 2024-03-04T08:26:12.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cbb000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb41597b0 con 0x7f3cb4152bd0 2024-03-04T08:26:12.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cab400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3cb40c8200 0x7f3cb4072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34518/0 (socket says 172.21.15.113:34518) 2024-03-04T08:26:12.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cab400640 1 -- 172.21.15.113:0/1988237635 learned_addr learned my addr 172.21.15.113:0/1988237635 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:12.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cb3e00640 1 --2- 172.21.15.113:0/1988237635 >> v2:172.21.15.136:3300/0 conn(0x7f3cb41527f0 0x7f3cb4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:12.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cb9000640 1 --2- 172.21.15.113:0/1988237635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb4152bd0 0x7f3cb41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:12.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cb3400640 1 -- 172.21.15.113:0/1988237635 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352356565 0 0) 0x7f3cb41591f0 con 0x7f3cb40c8200 2024-03-04T08:26:12.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.513+0000 7f3cb3400640 1 -- 172.21.15.113:0/1988237635 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c98003530 con 0x7f3cb40c8200 2024-03-04T08:26:12.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.517+0000 7f3cb3e00640 1 -- 172.21.15.113:0/1988237635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb4152bd0 msgr2=0x7f3cb41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:12.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.517+0000 7f3cb3e00640 1 --2- 172.21.15.113:0/1988237635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb4152bd0 0x7f3cb41587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:12.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.517+0000 7f3cb3e00640 1 -- 172.21.15.113:0/1988237635 >> v1:172.21.15.113:6789/0 conn(0x7f3cb40c8200 legacy=0x7f3cb4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:12.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.517+0000 7f3cb3e00640 1 -- 172.21.15.113:0/1988237635 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb41594d0 con 0x7f3cb41527f0 2024-03-04T08:26:12.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.517+0000 7f3cb3e00640 1 --2- 172.21.15.113:0/1988237635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb41527f0 0x7f3cb4070d40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40097c0 tx=0x7f3ca401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a549698a584925c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:12.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.517+0000 7f3cb3400640 1 -- 172.21.15.113:0/1988237635 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40b1070 con 0x7f3cb41527f0 2024-03-04T08:26:12.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.517+0000 7f3cb3400640 1 -- 172.21.15.113:0/1988237635 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca40ac410 con 0x7f3cb41527f0 2024-03-04T08:26:12.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.517+0000 7f3cb3400640 1 -- 172.21.15.113:0/1988237635 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40ac7e0 con 0x7f3cb41527f0 2024-03-04T08:26:12.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.517+0000 7f3cbb000640 1 -- 172.21.15.113:0/1988237635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb41527f0 msgr2=0x7f3cb4070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:12.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.517+0000 7f3cbb000640 1 --2- 172.21.15.113:0/1988237635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb41527f0 0x7f3cb4070d40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40097c0 tx=0x7f3ca401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:26:12.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 -- 172.21.15.113:0/1988237635 shutdown_connections 2024-03-04T08:26:12.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 --2- 172.21.15.113:0/1988237635 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb4152bd0 0x7f3cb41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:12.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 --2- 172.21.15.113:0/1988237635 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb41527f0 0x7f3cb4070d40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:12.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 -- 172.21.15.113:0/1988237635 >> 172.21.15.113:0/1988237635 conn(0x7f3cb406e820 msgr2=0x7f3cb406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:12.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 -- 172.21.15.113:0/1988237635 shutdown_connections 2024-03-04T08:26:12.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 -- 172.21.15.113:0/1988237635 wait complete. 2024-03-04T08:26:12.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 Processor -- start 2024-03-04T08:26:12.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 -- start start 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb40c8200 0x7f3cb41f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb41527f0 0x7f3cb4143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb4152bd0 0x7f3cb4157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb41654f0 con 0x7f3cb4152bd0 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4165370 con 0x7f3cb40c8200 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4165670 con 0x7f3cb41527f0 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb4152bd0 0x7f3cb4157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb4152bd0 0x7f3cb4157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57658/0 (socket says 172.21.15.113:57658) 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb9000640 1 -- 172.21.15.113:0/2086488004 learned_addr learned my addr 172.21.15.113:0/2086488004 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb3e00640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb40c8200 0x7f3cb41f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cab400640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb41527f0 0x7f3cb4143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:12.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb9000640 1 -- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb41527f0 msgr2=0x7f3cb4143d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:12.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb9000640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb41527f0 0x7f3cb4143d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:12.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb9000640 1 -- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb40c8200 msgr2=0x7f3cb41f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:12.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb9000640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb40c8200 0x7f3cb41f7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:12.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb9000640 1 -- 172.21.15.113:0/2086488004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb41f8110 con 0x7f3cb4152bd0 2024-03-04T08:26:12.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb9000640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb4152bd0 0x7f3cb4157a80 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00efc0 tx=0x7f3cac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:12.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb0c00640 1 -- 172.21.15.113:0/2086488004 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac018020 con 0x7f3cb4152bd0 2024-03-04T08:26:12.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb0c00640 1 -- 172.21.15.113:0/2086488004 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac014480 con 0x7f3cb4152bd0 2024-03-04T08:26:12.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cb0c00640 1 -- 172.21.15.113:0/2086488004 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac014870 con 0x7f3cb4152bd0 2024-03-04T08:26:12.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.521+0000 7f3cbb000640 1 -- 172.21.15.113:0/2086488004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb41f3c80 con 0x7f3cb4152bd0 2024-03-04T08:26:12.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.525+0000 7f3cbb000640 1 -- 172.21.15.113:0/2086488004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb41f3f40 con 0x7f3cb4152bd0 2024-03-04T08:26:12.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.525+0000 7f3cbb000640 1 -- 172.21.15.113:0/2086488004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb4075c00 con 0x7f3cb4152bd0 2024-03-04T08:26:12.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.525+0000 7f3cb0c00640 1 -- 172.21.15.113:0/2086488004 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cac01b020 con 0x7f3cb4152bd0 2024-03-04T08:26:12.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.525+0000 7f3cb0c00640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c90066f60 0x7f3c90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:12.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.525+0000 7f3cb0c00640 1 -- 172.21.15.113:0/2086488004 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f3cac086500 con 0x7f3cb4152bd0 2024-03-04T08:26:12.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.525+0000 7f3cb3e00640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c90066f60 0x7f3c90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:12.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.529+0000 7f3cb3e00640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c90066f60 0x7f3c90069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40ae000 tx=0x7f3ca4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:12.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.529+0000 7f3cb0c00640 1 -- 172.21.15.113:0/2086488004 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3cac022070 con 0x7f3cb4152bd0 2024-03-04T08:26:12.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.721+0000 7f3cbb000640 1 -- 172.21.15.113:0/2086488004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cb40732f0 con 0x7f3cb4152bd0 2024-03-04T08:26:12.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cb0c00640 1 -- 172.21.15.113:0/2086488004 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f3cac010040 con 0x7f3cb4152bd0 2024-03-04T08:26:12.723 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:12.723 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:12.723 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:12.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 -- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c90066f60 msgr2=0x7f3c90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:12.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c90066f60 0x7f3c90069420 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3ca40ae000 tx=0x7f3ca4008040 comp rx=0 tx=0).stop 2024-03-04T08:26:12.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 -- 172.21.15.113:0/2086488004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb4152bd0 msgr2=0x7f3cb4157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:12.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb4152bd0 0x7f3cb4157a80 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00efc0 tx=0x7f3cac00c490 comp rx=0 tx=0).stop 2024-03-04T08:26:12.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 -- 172.21.15.113:0/2086488004 shutdown_connections 2024-03-04T08:26:12.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3c90066f60 0x7f3c90069420 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:12.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3cb4152bd0 0x7f3cb4157a80 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:12.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3cb41527f0 0x7f3cb4143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:12.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 --2- 172.21.15.113:0/2086488004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb40c8200 0x7f3cb41f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:12.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 -- 172.21.15.113:0/2086488004 >> 172.21.15.113:0/2086488004 conn(0x7f3cb406e820 msgr2=0x7f3cb40c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:12.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 -- 172.21.15.113:0/2086488004 shutdown_connections 2024-03-04T08:26:12.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:12.725+0000 7f3cbb000640 1 -- 172.21.15.113:0/2086488004 wait complete. 2024-03-04T08:26:12.743 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:14.743 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:14.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.888+0000 7f17af400640 1 Processor -- start 2024-03-04T08:26:14.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.888+0000 7f17af400640 1 -- start start 2024-03-04T08:26:14.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.888+0000 7f17af400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17a80c8200 0x7f17a8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:14.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.888+0000 7f17af400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a81591f0 con 0x7f17a81527f0 2024-03-04T08:26:14.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17af400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a81594d0 con 0x7f17a8152bd0 2024-03-04T08:26:14.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17af400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a8071df0 con 0x7f17a80c8200 2024-03-04T08:26:14.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17a7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17a80c8200 0x7f17a8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:14.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17a7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f17a80c8200 0x7f17a8070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57664/0 (socket says 172.21.15.113:57664) 2024-03-04T08:26:14.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17a7e00640 1 -- 172.21.15.113:0/3516089916 learned_addr learned my addr 172.21.15.113:0/3516089916 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:14.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17aca00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f17a81527f0 0x7f17a8072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:34534/0 (socket says 172.21.15.113:34534) 2024-03-04T08:26:14.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17a7e00640 1 -- 172.21.15.113:0/3516089916 >> v1:172.21.15.136:6789/0 conn(0x7f17a8152bd0 legacy=0x7f17a81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:14.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17a7e00640 1 -- 172.21.15.113:0/3516089916 >> v1:172.21.15.113:6789/0 conn(0x7f17a81527f0 legacy=0x7f17a8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:14.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17a7e00640 1 -- 172.21.15.113:0/3516089916 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a80717a0 con 0x7f17a80c8200 2024-03-04T08:26:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17a7e00640 1 --2- 172.21.15.113:0/3516089916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a80c8200 0x7f17a8070d40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f1794009870 tx=0x7f179401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973372570863f8c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17a7400640 1 -- 172.21.15.113:0/3516089916 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17940a7070 con 0x7f17a80c8200 2024-03-04T08:26:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17a7400640 1 -- 172.21.15.113:0/3516089916 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17940b0070 con 0x7f17a80c8200 2024-03-04T08:26:14.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.892+0000 7f17a7400640 1 -- 172.21.15.113:0/3516089916 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17940ab890 con 0x7f17a80c8200 2024-03-04T08:26:14.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 -- 172.21.15.113:0/3516089916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a80c8200 msgr2=0x7f17a8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:14.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 --2- 172.21.15.113:0/3516089916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a80c8200 0x7f17a8070d40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f1794009870 tx=0x7f179401c560 comp rx=0 tx=0).stop 2024-03-04T08:26:14.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 -- 172.21.15.113:0/3516089916 shutdown_connections 2024-03-04T08:26:14.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 --2- 172.21.15.113:0/3516089916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a80c8200 0x7f17a8070d40 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:14.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 -- 172.21.15.113:0/3516089916 >> 172.21.15.113:0/3516089916 conn(0x7f17a806e820 msgr2=0x7f17a806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:14.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 -- 172.21.15.113:0/3516089916 shutdown_connections 2024-03-04T08:26:14.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 -- 172.21.15.113:0/3516089916 wait complete. 2024-03-04T08:26:14.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 Processor -- start 2024-03-04T08:26:14.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 -- start start 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a80c8200 0x7f17a81f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a81527f0 0x7f17a80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a8152bd0 0x7f17a814ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a8166380 con 0x7f17a8152bd0 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a81597b0 con 0x7f17a80c8200 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17af400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a8166500 con 0x7f17a81527f0 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17ad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a8152bd0 0x7f17a814ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17ad400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a8152bd0 0x7f17a814ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:57678/0 (socket says 172.21.15.113:57678) 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17ad400640 1 -- 172.21.15.113:0/1672081131 learned_addr learned my addr 172.21.15.113:0/1672081131 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17a7e00640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a81527f0 0x7f17a80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:14.899 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17aca00640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a80c8200 0x7f17a81f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17ad400640 1 -- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a81527f0 msgr2=0x7f17a80b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17ad400640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a81527f0 0x7f17a80b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17ad400640 1 -- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a80c8200 msgr2=0x7f17a81f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17ad400640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a80c8200 0x7f17a81f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17ad400640 1 -- 172.21.15.113:0/1672081131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a8154f40 con 0x7f17a8152bd0 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.896+0000 7f17aca00640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a80c8200 0x7f17a81f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17ad400640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a8152bd0 0x7f17a814ada0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c370 tx=0x7f17a000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17a4c00640 1 -- 172.21.15.113:0/1672081131 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0005220 con 0x7f17a8152bd0 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17a4c00640 1 -- 172.21.15.113:0/1672081131 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a00053c0 con 0x7f17a8152bd0 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17a4c00640 1 -- 172.21.15.113:0/1672081131 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0007270 con 0x7f17a8152bd0 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17af400640 1 -- 172.21.15.113:0/1672081131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a8155220 con 0x7f17a8152bd0 2024-03-04T08:26:14.900 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17af400640 1 -- 172.21.15.113:0/1672081131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a81557e0 con 0x7f17a8152bd0 2024-03-04T08:26:14.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17af400640 1 -- 172.21.15.113:0/1672081131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a8075c00 con 0x7f17a8152bd0 2024-03-04T08:26:14.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17a4c00640 1 -- 172.21.15.113:0/1672081131 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17a0019020 con 0x7f17a8152bd0 2024-03-04T08:26:14.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17a4c00640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1788066f60 0x7f1788069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:14.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17a4c00640 1 -- 172.21.15.113:0/1672081131 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f17a0084d50 con 0x7f17a8152bd0 2024-03-04T08:26:14.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.900+0000 7f17aca00640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1788066f60 0x7f1788069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:14.902 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.904+0000 7f17aca00640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1788066f60 0x7f1788069420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f17980012f0 tx=0x7f1798005be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:14.905 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:14.904+0000 7f17a4c00640 1 -- 172.21.15.113:0/1672081131 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f17a00552b0 con 0x7f17a8152bd0 2024-03-04T08:26:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.092+0000 7f17af400640 1 -- 172.21.15.113:0/1672081131 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17a80c79d0 con 0x7f17a8152bd0 2024-03-04T08:26:15.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.092+0000 7f17a4c00640 1 -- 172.21.15.113:0/1672081131 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f17a0054bb0 con 0x7f17a8152bd0 2024-03-04T08:26:15.094 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:15.094 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 -- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1788066f60 msgr2=0x7f1788069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1788066f60 0x7f1788069420 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f17980012f0 tx=0x7f1798005be0 comp rx=0 tx=0).stop 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 -- 172.21.15.113:0/1672081131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a8152bd0 msgr2=0x7f17a814ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a8152bd0 0x7f17a814ada0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c370 tx=0x7f17a000c830 comp rx=0 tx=0).stop 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 -- 172.21.15.113:0/1672081131 shutdown_connections 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1788066f60 0x7f1788069420 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17a8152bd0 0x7f17a814ada0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f17a81527f0 0x7f17a80b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 --2- 172.21.15.113:0/1672081131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a80c8200 0x7f17a81f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 -- 172.21.15.113:0/1672081131 >> 172.21.15.113:0/1672081131 conn(0x7f17a806e820 msgr2=0x7f17a80cb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 -- 172.21.15.113:0/1672081131 shutdown_connections 2024-03-04T08:26:15.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:15.096+0000 7f17af400640 1 -- 172.21.15.113:0/1672081131 wait complete. 2024-03-04T08:26:15.115 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:26:15.116 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:26:15.645 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:26:15.649 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:26:17.117 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:17.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda7200640 1 Processor -- start 2024-03-04T08:26:17.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda7200640 1 -- start start 2024-03-04T08:26:17.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda7200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feda815b240 0x7feda8166170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:17.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda7200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda80c9160 con 0x7feda80c2180 2024-03-04T08:26:17.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda7200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda80c9400 con 0x7feda81547a0 2024-03-04T08:26:17.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda7200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda80c96e0 con 0x7feda815b240 2024-03-04T08:26:17.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda5400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7feda80c2180 0x7feda815ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33022/0 (socket says 172.21.15.113:33022) 2024-03-04T08:26:17.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda5400640 1 -- 172.21.15.113:0/3761727718 learned_addr learned my addr 172.21.15.113:0/3761727718 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:17.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda6800640 1 --2- 172.21.15.113:0/3761727718 >> v2:172.21.15.136:3300/0 conn(0x7feda815b240 0x7feda8166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:17.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda4a00640 1 -- 172.21.15.113:0/3761727718 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018422965 0 0) 0x7feda80c9160 con 0x7feda80c2180 2024-03-04T08:26:17.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda4a00640 1 -- 172.21.15.113:0/3761727718 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed8c003530 con 0x7feda80c2180 2024-03-04T08:26:17.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda6800640 1 -- 172.21.15.113:0/3761727718 >> v1:172.21.15.136:6789/0 conn(0x7feda81547a0 legacy=0x7feda80c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:17.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda6800640 1 -- 172.21.15.113:0/3761727718 >> v1:172.21.15.113:6789/0 conn(0x7feda80c2180 legacy=0x7feda815ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:17.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda4a00640 1 -- 172.21.15.113:0/3761727718 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 948264050 0 0) 0x7fed8c003530 con 0x7feda80c2180 2024-03-04T08:26:17.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.264+0000 7feda6800640 1 -- 172.21.15.113:0/3761727718 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda80c7c00 con 0x7feda815b240 2024-03-04T08:26:17.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda6800640 1 --2- 172.21.15.113:0/3761727718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feda815b240 0x7feda8166170 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7feda0009d90 tx=0x7feda001f080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8b609178b4871f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:17.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda4a00640 1 -- 172.21.15.113:0/3761727718 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda00b5070 con 0x7feda815b240 2024-03-04T08:26:17.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda4a00640 1 -- 172.21.15.113:0/3761727718 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda00a8c10 con 0x7feda815b240 2024-03-04T08:26:17.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda4a00640 1 -- 172.21.15.113:0/3761727718 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed940016a0 con 0x7feda815b240 2024-03-04T08:26:17.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda7200640 1 -- 172.21.15.113:0/3761727718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feda815b240 msgr2=0x7feda8166170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:17.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda7200640 1 --2- 172.21.15.113:0/3761727718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feda815b240 0x7feda8166170 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7feda0009d90 tx=0x7feda001f080 comp rx=0 tx=0).stop 2024-03-04T08:26:17.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda7200640 1 -- 172.21.15.113:0/3761727718 shutdown_connections 2024-03-04T08:26:17.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda7200640 1 --2- 172.21.15.113:0/3761727718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feda815b240 0x7feda8166170 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:17.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda7200640 1 -- 172.21.15.113:0/3761727718 >> 172.21.15.113:0/3761727718 conn(0x7feda80cd080 msgr2=0x7feda80cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:17.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda7200640 1 -- 172.21.15.113:0/3761727718 shutdown_connections 2024-03-04T08:26:17.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.268+0000 7feda7200640 1 -- 172.21.15.113:0/3761727718 wait complete. 2024-03-04T08:26:17.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 Processor -- start 2024-03-04T08:26:17.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 -- start start 2024-03-04T08:26:17.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feda80c2180 0x7feda81e7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:17.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feda81547a0 0x7feda81f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:17.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda815b240 0x7feda81f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:17.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda8154b80 con 0x7feda815b240 2024-03-04T08:26:17.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda80c81c0 con 0x7feda80c2180 2024-03-04T08:26:17.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda8154d00 con 0x7feda81547a0 2024-03-04T08:26:17.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda815b240 0x7feda81f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:17.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda815b240 0x7feda81f14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50018/0 (socket says 172.21.15.113:50018) 2024-03-04T08:26:17.273 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda6800640 1 -- 172.21.15.113:0/1842898417 learned_addr learned my addr 172.21.15.113:0/1842898417 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda5400640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feda81547a0 0x7feda81f0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda5e00640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feda80c2180 0x7feda81e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda6800640 1 -- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feda81547a0 msgr2=0x7feda81f0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda6800640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feda81547a0 0x7feda81f0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda6800640 1 -- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feda80c2180 msgr2=0x7feda81e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda6800640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feda80c2180 0x7feda81e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda6800640 1 -- 172.21.15.113:0/1842898417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda81f0a90 con 0x7feda815b240 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda6800640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda815b240 0x7feda81f14d0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7feda00b3040 tx=0x7feda001be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7fed9e000640 1 -- 172.21.15.113:0/1842898417 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda00b5040 con 0x7feda815b240 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7fed9e000640 1 -- 172.21.15.113:0/1842898417 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda00b0b50 con 0x7feda815b240 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 -- 172.21.15.113:0/1842898417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda814e860 con 0x7feda815b240 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7fed9e000640 1 -- 172.21.15.113:0/1842898417 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0009450 con 0x7feda815b240 2024-03-04T08:26:17.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 -- 172.21.15.113:0/1842898417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda80caf20 con 0x7feda815b240 2024-03-04T08:26:17.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.272+0000 7feda7200640 1 -- 172.21.15.113:0/1842898417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda81f0c60 con 0x7feda815b240 2024-03-04T08:26:17.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.276+0000 7fed9e000640 1 -- 172.21.15.113:0/1842898417 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feda00c4020 con 0x7feda815b240 2024-03-04T08:26:17.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.276+0000 7fed9e000640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed88066f60 0x7fed88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:17.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.276+0000 7fed9e000640 1 -- 172.21.15.113:0/1842898417 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7feda0123a30 con 0x7feda815b240 2024-03-04T08:26:17.276 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.276+0000 7feda5e00640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed88066f60 0x7fed88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:17.277 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.276+0000 7feda5e00640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed88066f60 0x7fed88069420 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fed98000f70 tx=0x7fed98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:17.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.280+0000 7fed9e000640 1 -- 172.21.15.113:0/1842898417 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7feda00cb020 con 0x7feda815b240 2024-03-04T08:26:17.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.464+0000 7feda7200640 1 -- 172.21.15.113:0/1842898417 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feda8075a80 con 0x7feda815b240 2024-03-04T08:26:17.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.464+0000 7fed9e000640 1 -- 172.21.15.113:0/1842898417 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7feda0018be0 con 0x7feda815b240 2024-03-04T08:26:17.466 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:17.466 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:17.466 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 -- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed88066f60 msgr2=0x7fed88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed88066f60 0x7fed88069420 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fed98000f70 tx=0x7fed98009040 comp rx=0 tx=0).stop 2024-03-04T08:26:17.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 -- 172.21.15.113:0/1842898417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda815b240 msgr2=0x7feda81f14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:17.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda815b240 0x7feda81f14d0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7feda00b3040 tx=0x7feda001be20 comp rx=0 tx=0).stop 2024-03-04T08:26:17.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 -- 172.21.15.113:0/1842898417 shutdown_connections 2024-03-04T08:26:17.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fed88066f60 0x7fed88069420 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:17.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feda815b240 0x7feda81f14d0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:17.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feda81547a0 0x7feda81f0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:17.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 --2- 172.21.15.113:0/1842898417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feda80c2180 0x7feda81e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:17.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 -- 172.21.15.113:0/1842898417 >> 172.21.15.113:0/1842898417 conn(0x7feda80cd080 msgr2=0x7feda80c8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:17.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 -- 172.21.15.113:0/1842898417 shutdown_connections 2024-03-04T08:26:17.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:17.468+0000 7feda7200640 1 -- 172.21.15.113:0/1842898417 wait complete. 2024-03-04T08:26:17.488 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:19.490 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:19.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.636+0000 7fb4baa00640 1 Processor -- start 2024-03-04T08:26:19.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.636+0000 7fb4baa00640 1 -- start start 2024-03-04T08:26:19.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4baa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b4160820 0x7fb4b4162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:19.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4baa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b40c1ec0 con 0x7fb4b40c85f0 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4baa00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b40c9b30 con 0x7fb4b40b20b0 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4baa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40c12a0 con 0x7fb4b4160820 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4b3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb4b40c85f0 0x7fb4b406b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33030/0 (socket says 172.21.15.113:33030) 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4b3400640 1 -- 172.21.15.113:0/2163876289 learned_addr learned my addr 172.21.15.113:0/2163876289 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4b8a00640 1 --2- 172.21.15.113:0/2163876289 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b4160820 0x7fb4b4162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4b2a00640 1 -- 172.21.15.113:0/2163876289 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 673452665 0 0) 0x7fb4b40c1ec0 con 0x7fb4b40c85f0 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4b2a00640 1 -- 172.21.15.113:0/2163876289 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb498003530 con 0x7fb4b40c85f0 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4b2a00640 1 -- 172.21.15.113:0/2163876289 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181594538 0 0) 0x7fb4b40c9b30 con 0x7fb4b40b20b0 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4b2a00640 1 -- 172.21.15.113:0/2163876289 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b40c1ec0 con 0x7fb4b40b20b0 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4b8a00640 1 -- 172.21.15.113:0/2163876289 >> v1:172.21.15.136:6789/0 conn(0x7fb4b40b20b0 legacy=0x7fb4b4157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4b8a00640 1 -- 172.21.15.113:0/2163876289 >> v1:172.21.15.113:6789/0 conn(0x7fb4b40c85f0 legacy=0x7fb4b406b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:19.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.640+0000 7fb4b8a00640 1 -- 172.21.15.113:0/2163876289 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b40c1580 con 0x7fb4b4160820 2024-03-04T08:26:19.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4b8a00640 1 --2- 172.21.15.113:0/2163876289 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b4160820 0x7fb4b4162c10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00aed0 tx=0x7fb4ac01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c089f992c29b71d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:19.644 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4b2a00640 1 -- 172.21.15.113:0/2163876289 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0b4070 con 0x7fb4b4160820 2024-03-04T08:26:19.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4b2a00640 1 -- 172.21.15.113:0/2163876289 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac008440 con 0x7fb4b4160820 2024-03-04T08:26:19.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4b2a00640 1 -- 172.21.15.113:0/2163876289 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac008810 con 0x7fb4b4160820 2024-03-04T08:26:19.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4baa00640 1 -- 172.21.15.113:0/2163876289 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b4160820 msgr2=0x7fb4b4162c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:19.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4baa00640 1 --2- 172.21.15.113:0/2163876289 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b4160820 0x7fb4b4162c10 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00aed0 tx=0x7fb4ac01f150 comp rx=0 tx=0).stop 2024-03-04T08:26:19.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4baa00640 1 -- 172.21.15.113:0/2163876289 shutdown_connections 2024-03-04T08:26:19.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4baa00640 1 --2- 172.21.15.113:0/2163876289 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b4160820 0x7fb4b4162c10 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:19.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4baa00640 1 -- 172.21.15.113:0/2163876289 >> 172.21.15.113:0/2163876289 conn(0x7fb4b406e820 msgr2=0x7fb4b406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:19.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4baa00640 1 -- 172.21.15.113:0/2163876289 shutdown_connections 2024-03-04T08:26:19.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4baa00640 1 -- 172.21.15.113:0/2163876289 wait complete. 2024-03-04T08:26:19.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4baa00640 1 Processor -- start 2024-03-04T08:26:19.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.644+0000 7fb4baa00640 1 -- start start 2024-03-04T08:26:19.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4baa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b40b20b0 0x7fb4b40c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:19.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4baa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b40c85f0 0x7fb4b40c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:19.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4baa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b4160820 0x7fb4b4146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:19.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4baa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b4157e20 con 0x7fb4b4160820 2024-03-04T08:26:19.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4baa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b41571f0 con 0x7fb4b40c85f0 2024-03-04T08:26:19.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4baa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40c2400 con 0x7fb4b40b20b0 2024-03-04T08:26:19.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b4160820 0x7fb4b4146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:19.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b4160820 0x7fb4b4146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50020/0 (socket says 172.21.15.113:50020) 2024-03-04T08:26:19.648 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b8a00640 1 -- 172.21.15.113:0/490440804 learned_addr learned my addr 172.21.15.113:0/490440804 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:19.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b3e00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b40b20b0 0x7fb4b40c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:19.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b3400640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b40c85f0 0x7fb4b40c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:19.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b8a00640 1 -- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b40b20b0 msgr2=0x7fb4b40c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:19.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b8a00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b40b20b0 0x7fb4b40c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:19.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b8a00640 1 -- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b40c85f0 msgr2=0x7fb4b40c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:19.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b8a00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b40c85f0 0x7fb4b40c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:19.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b8a00640 1 -- 172.21.15.113:0/490440804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b40caca0 con 0x7fb4b4160820 2024-03-04T08:26:19.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b3400640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b40c85f0 0x7fb4b40c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:19.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4b8a00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b4160820 0x7fb4b4146b30 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0b2040 tx=0x7fb4ac01bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:19.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4abe00640 1 -- 172.21.15.113:0/490440804 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0b4040 con 0x7fb4b4160820 2024-03-04T08:26:19.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4abe00640 1 -- 172.21.15.113:0/490440804 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac0c0070 con 0x7fb4b4160820 2024-03-04T08:26:19.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4baa00640 1 -- 172.21.15.113:0/490440804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b40cb7d0 con 0x7fb4b4160820 2024-03-04T08:26:19.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4abe00640 1 -- 172.21.15.113:0/490440804 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00a560 con 0x7fb4b4160820 2024-03-04T08:26:19.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4baa00640 1 -- 172.21.15.113:0/490440804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b41ec100 con 0x7fb4b4160820 2024-03-04T08:26:19.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4baa00640 1 -- 172.21.15.113:0/490440804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b4075c00 con 0x7fb4b4160820 2024-03-04T08:26:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.648+0000 7fb4abe00640 1 -- 172.21.15.113:0/490440804 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4ac0c5460 con 0x7fb4b4160820 2024-03-04T08:26:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.652+0000 7fb4abe00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb494066f60 0x7fb494069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.652+0000 7fb4abe00640 1 -- 172.21.15.113:0/490440804 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fb4ac123c00 con 0x7fb4b4160820 2024-03-04T08:26:19.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.652+0000 7fb4b3e00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb494066f60 0x7fb494069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:19.652 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.652+0000 7fb4b3e00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb494066f60 0x7fb494069420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4007fd0 tx=0x7fb4a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:19.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.656+0000 7fb4abe00640 1 -- 172.21.15.113:0/490440804 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb4ac018cb0 con 0x7fb4b4160820 2024-03-04T08:26:19.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 -- 172.21.15.113:0/490440804 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4b40c83d0 con 0x7fb4b4160820 2024-03-04T08:26:19.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4abe00640 1 -- 172.21.15.113:0/490440804 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fb4ac0f4160 con 0x7fb4b4160820 2024-03-04T08:26:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:19.844 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:19.844 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 -- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb494066f60 msgr2=0x7fb494069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb494066f60 0x7fb494069420 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4007fd0 tx=0x7fb4a4009040 comp rx=0 tx=0).stop 2024-03-04T08:26:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 -- 172.21.15.113:0/490440804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b4160820 msgr2=0x7fb4b4146b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b4160820 0x7fb4b4146b30 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0b2040 tx=0x7fb4ac01bef0 comp rx=0 tx=0).stop 2024-03-04T08:26:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 -- 172.21.15.113:0/490440804 shutdown_connections 2024-03-04T08:26:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb494066f60 0x7fb494069420 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb4b4160820 0x7fb4b4146b30 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:19.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4b40c85f0 0x7fb4b40c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:19.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 --2- 172.21.15.113:0/490440804 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb4b40b20b0 0x7fb4b40c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:19.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 -- 172.21.15.113:0/490440804 >> 172.21.15.113:0/490440804 conn(0x7fb4b406e820 msgr2=0x7fb4b4149e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:19.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 -- 172.21.15.113:0/490440804 shutdown_connections 2024-03-04T08:26:19.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:19.844+0000 7fb4baa00640 1 -- 172.21.15.113:0/490440804 wait complete. 2024-03-04T08:26:19.866 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:21.866 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:22.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f932a400640 1 Processor -- start 2024-03-04T08:26:22.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f932a400640 1 -- start start 2024-03-04T08:26:22.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f932a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f932414e4c0 0x7f9324072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:22.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f932a400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93240c27f0 con 0x7f93240c22e0 2024-03-04T08:26:22.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f932a400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93240c2a90 con 0x7f932415b900 2024-03-04T08:26:22.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f932a400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932415ad60 con 0x7f932414e4c0 2024-03-04T08:26:22.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f9323400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f932414e4c0 0x7f9324072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:22.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f9323400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f932414e4c0 0x7f9324072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50034/0 (socket says 172.21.15.113:50034) 2024-03-04T08:26:22.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f9322a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f93240c22e0 0x7f932415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33038/0 (socket says 172.21.15.113:33038) 2024-03-04T08:26:22.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f9323400640 1 -- 172.21.15.113:0/2361429037 learned_addr learned my addr 172.21.15.113:0/2361429037 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:22.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f9323400640 1 -- 172.21.15.113:0/2361429037 >> v1:172.21.15.136:6789/0 conn(0x7f932415b900 legacy=0x7f93241620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:22.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f9323400640 1 -- 172.21.15.113:0/2361429037 >> v1:172.21.15.113:6789/0 conn(0x7f93240c22e0 legacy=0x7f932415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:22.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.012+0000 7f9323400640 1 -- 172.21.15.113:0/2361429037 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9324070c60 con 0x7f932414e4c0 2024-03-04T08:26:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f9323400640 1 --2- 172.21.15.113:0/2361429037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932414e4c0 0x7f9324072c80 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9314009870 tx=0x7f931401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608e99cb61ed00f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f9322000640 1 -- 172.21.15.113:0/2361429037 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140b5070 con 0x7f932414e4c0 2024-03-04T08:26:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f9322000640 1 -- 172.21.15.113:0/2361429037 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93140a8a50 con 0x7f932414e4c0 2024-03-04T08:26:22.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f9322000640 1 -- 172.21.15.113:0/2361429037 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140a8e20 con 0x7f932414e4c0 2024-03-04T08:26:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f932a400640 1 -- 172.21.15.113:0/2361429037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932414e4c0 msgr2=0x7f9324072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f932a400640 1 --2- 172.21.15.113:0/2361429037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932414e4c0 0x7f9324072c80 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9314009870 tx=0x7f931401ed50 comp rx=0 tx=0).stop 2024-03-04T08:26:22.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f932a400640 1 -- 172.21.15.113:0/2361429037 shutdown_connections 2024-03-04T08:26:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f932a400640 1 --2- 172.21.15.113:0/2361429037 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932414e4c0 0x7f9324072c80 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f932a400640 1 -- 172.21.15.113:0/2361429037 >> 172.21.15.113:0/2361429037 conn(0x7f932406e820 msgr2=0x7f932406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f932a400640 1 -- 172.21.15.113:0/2361429037 shutdown_connections 2024-03-04T08:26:22.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.016+0000 7f932a400640 1 -- 172.21.15.113:0/2361429037 wait complete. 2024-03-04T08:26:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f932a400640 1 Processor -- start 2024-03-04T08:26:22.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f932a400640 1 -- start start 2024-03-04T08:26:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f932a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93240c22e0 0x7f93240cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f932a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f932414e4c0 0x7f932414c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f932a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932415b900 0x7f932414ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f932a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324071500 con 0x7f932415b900 2024-03-04T08:26:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f932a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324071220 con 0x7f932414e4c0 2024-03-04T08:26:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f932a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932414e8a0 con 0x7f93240c22e0 2024-03-04T08:26:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f9323e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932415b900 0x7f932414ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:22.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f9323e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932415b900 0x7f932414ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50038/0 (socket says 172.21.15.113:50038) 2024-03-04T08:26:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f9323e00640 1 -- 172.21.15.113:0/3560524622 learned_addr learned my addr 172.21.15.113:0/3560524622 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f9323e00640 1 -- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93240c22e0 msgr2=0x7f93240cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f9323e00640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93240c22e0 0x7f93240cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f9323e00640 1 -- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f932414e4c0 msgr2=0x7f932414c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f9323e00640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f932414e4c0 0x7f932414c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f9323e00640 1 -- 172.21.15.113:0/3560524622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93241e7a30 con 0x7f932415b900 2024-03-04T08:26:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f9323e00640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932415b900 0x7f932414ca80 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c7a0 tx=0x7f931c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f931b400640 1 -- 172.21.15.113:0/3560524622 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c003b80 con 0x7f932415b900 2024-03-04T08:26:22.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f931b400640 1 -- 172.21.15.113:0/3560524622 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c003d20 con 0x7f932415b900 2024-03-04T08:26:22.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f931b400640 1 -- 172.21.15.113:0/3560524622 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c00aa90 con 0x7f932415b900 2024-03-04T08:26:22.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f932a400640 1 -- 172.21.15.113:0/3560524622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93241e7d10 con 0x7f932415b900 2024-03-04T08:26:22.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.020+0000 7f932a400640 1 -- 172.21.15.113:0/3560524622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9324158db0 con 0x7f932415b900 2024-03-04T08:26:22.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.024+0000 7f932a400640 1 -- 172.21.15.113:0/3560524622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9324075c00 con 0x7f932415b900 2024-03-04T08:26:22.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.024+0000 7f931b400640 1 -- 172.21.15.113:0/3560524622 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f931c00ac30 con 0x7f932415b900 2024-03-04T08:26:22.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.024+0000 7f931b400640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9300066f90 0x7f9300069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:22.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.024+0000 7f931b400640 1 -- 172.21.15.113:0/3560524622 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f931c085b00 con 0x7f932415b900 2024-03-04T08:26:22.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.024+0000 7f9323400640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9300066f90 0x7f9300069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:22.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.024+0000 7f9323400640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9300066f90 0x7f9300069450 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f93140b2000 tx=0x7f9314004740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:22.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.028+0000 7f931b400640 1 -- 172.21.15.113:0/3560524622 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f931c055fa0 con 0x7f932415b900 2024-03-04T08:26:22.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.216+0000 7f932a400640 1 -- 172.21.15.113:0/3560524622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f932415b4a0 con 0x7f932415b900 2024-03-04T08:26:22.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.216+0000 7f931b400640 1 -- 172.21.15.113:0/3560524622 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f931c0558a0 con 0x7f932415b900 2024-03-04T08:26:22.217 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:22.217 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:22.217 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:22.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 -- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9300066f90 msgr2=0x7f9300069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:22.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9300066f90 0x7f9300069450 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f93140b2000 tx=0x7f9314004740 comp rx=0 tx=0).stop 2024-03-04T08:26:22.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 -- 172.21.15.113:0/3560524622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932415b900 msgr2=0x7f932414ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:22.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932415b900 0x7f932414ca80 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c7a0 tx=0x7f931c00cc60 comp rx=0 tx=0).stop 2024-03-04T08:26:22.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 -- 172.21.15.113:0/3560524622 shutdown_connections 2024-03-04T08:26:22.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9300066f90 0x7f9300069450 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:22.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f932415b900 0x7f932414ca80 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:22.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f932414e4c0 0x7f932414c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:22.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 --2- 172.21.15.113:0/3560524622 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f93240c22e0 0x7f93240cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:22.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 -- 172.21.15.113:0/3560524622 >> 172.21.15.113:0/3560524622 conn(0x7f932406e820 msgr2=0x7f93240c8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:22.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 -- 172.21.15.113:0/3560524622 shutdown_connections 2024-03-04T08:26:22.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:22.220+0000 7f932a400640 1 -- 172.21.15.113:0/3560524622 wait complete. 2024-03-04T08:26:22.238 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:24.239 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:24.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.384+0000 7f6113e00640 1 Processor -- start 2024-03-04T08:26:24.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.384+0000 7f6113e00640 1 -- start start 2024-03-04T08:26:24.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c9ab0 0x7f611415c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:24.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f61140c8fe0 0x7f61140c9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:24.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61140c5630 con 0x7f6114152ac0 2024-03-04T08:26:24.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61140c5bb0 con 0x7f61140c8fe0 2024-03-04T08:26:24.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611406fc70 con 0x7f61140c9ab0 2024-03-04T08:26:24.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f61140c8fe0 0x7f61140c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:24.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f61140c8fe0 0x7f61140c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50052/0 (socket says 172.21.15.113:50052) 2024-03-04T08:26:24.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113400640 1 -- 172.21.15.113:0/1684041864 learned_addr learned my addr 172.21.15.113:0/1684041864 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:24.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f610a000640 1 --2- 172.21.15.113:0/1684041864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c9ab0 0x7f611415c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:24.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113400640 1 -- 172.21.15.113:0/1684041864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c9ab0 msgr2=0x7f611415c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:24.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113400640 1 --2- 172.21.15.113:0/1684041864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c9ab0 0x7f611415c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:24.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113400640 1 -- 172.21.15.113:0/1684041864 >> v1:172.21.15.136:6789/0 conn(0x7f6114152ac0 legacy=0x7f611406ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:24.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f6113400640 1 -- 172.21.15.113:0/1684041864 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61140c58d0 con 0x7f61140c8fe0 2024-03-04T08:26:24.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.388+0000 7f610a000640 1 --2- 172.21.15.113:0/1684041864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c9ab0 0x7f611415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:24.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113400640 1 --2- 172.21.15.113:0/1684041864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c8fe0 0x7f61140c9450 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f610c009870 tx=0x7f610c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20341581f64b5d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:24.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6112000640 1 -- 172.21.15.113:0/1684041864 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c0b1070 con 0x7f61140c8fe0 2024-03-04T08:26:24.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6112000640 1 -- 172.21.15.113:0/1684041864 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610c0ac620 con 0x7f61140c8fe0 2024-03-04T08:26:24.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6112000640 1 -- 172.21.15.113:0/1684041864 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c0ac9f0 con 0x7f61140c8fe0 2024-03-04T08:26:24.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 -- 172.21.15.113:0/1684041864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c8fe0 msgr2=0x7f61140c9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:24.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 --2- 172.21.15.113:0/1684041864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c8fe0 0x7f61140c9450 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f610c009870 tx=0x7f610c01c4d0 comp rx=0 tx=0).stop 2024-03-04T08:26:24.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 -- 172.21.15.113:0/1684041864 shutdown_connections 2024-03-04T08:26:24.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 --2- 172.21.15.113:0/1684041864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c8fe0 0x7f61140c9450 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:24.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 --2- 172.21.15.113:0/1684041864 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c9ab0 0x7f611415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:24.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 -- 172.21.15.113:0/1684041864 >> 172.21.15.113:0/1684041864 conn(0x7f6114072e30 msgr2=0x7f6114066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:24.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 -- 172.21.15.113:0/1684041864 shutdown_connections 2024-03-04T08:26:24.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 -- 172.21.15.113:0/1684041864 wait complete. 2024-03-04T08:26:24.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 Processor -- start 2024-03-04T08:26:24.393 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 -- start start 2024-03-04T08:26:24.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c8fe0 0x7f61141f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:24.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6114152ac0 0x7f611415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:24.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c4e10 0x7f61140cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:24.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6114152ea0 con 0x7f61140c4e10 2024-03-04T08:26:24.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61140c8a00 con 0x7f6114152ac0 2024-03-04T08:26:24.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.392+0000 7f6113e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611415cbf0 con 0x7f61140c8fe0 2024-03-04T08:26:24.396 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c4e10 0x7f61140cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c4e10 0x7f61140cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50060/0 (socket says 172.21.15.113:50060) 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113400640 1 -- 172.21.15.113:0/519793303 learned_addr learned my addr 172.21.15.113:0/519793303 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6112a00640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c8fe0 0x7f61141f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f610a000640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6114152ac0 0x7f611415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113400640 1 -- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c8fe0 msgr2=0x7f61141f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113400640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c8fe0 0x7f61141f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113400640 1 -- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6114152ac0 msgr2=0x7f611415a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113400640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6114152ac0 0x7f611415a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113400640 1 -- 172.21.15.113:0/519793303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611415aae0 con 0x7f61140c4e10 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f610a000640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6114152ac0 0x7f611415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113400640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c4e10 0x7f61140cc3b0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f610c009870 tx=0x7f610c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f610b400640 1 -- 172.21.15.113:0/519793303 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c0b1040 con 0x7f61140c4e10 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f610b400640 1 -- 172.21.15.113:0/519793303 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610c0b5420 con 0x7f61140c4e10 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113e00640 1 -- 172.21.15.113:0/519793303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611415b710 con 0x7f61140c4e10 2024-03-04T08:26:24.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f610b400640 1 -- 172.21.15.113:0/519793303 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c0b57d0 con 0x7f61140c4e10 2024-03-04T08:26:24.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113e00640 1 -- 172.21.15.113:0/519793303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611415b060 con 0x7f61140c4e10 2024-03-04T08:26:24.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f6113e00640 1 -- 172.21.15.113:0/519793303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6114075a80 con 0x7f61140c4e10 2024-03-04T08:26:24.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.396+0000 7f610b400640 1 -- 172.21.15.113:0/519793303 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f610c0b5970 con 0x7f61140c4e10 2024-03-04T08:26:24.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.400+0000 7f610b400640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f60f0066f90 0x7f60f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:24.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.400+0000 7f610b400640 1 -- 172.21.15.113:0/519793303 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f610c11f7a0 con 0x7f61140c4e10 2024-03-04T08:26:24.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.400+0000 7f6112a00640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f60f0066f90 0x7f60f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:24.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.400+0000 7f6112a00640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f60f0066f90 0x7f60f0069450 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6104001d50 tx=0x7f6104009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:24.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.404+0000 7f610b400640 1 -- 172.21.15.113:0/519793303 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f610c016150 con 0x7f61140c4e10 2024-03-04T08:26:24.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.592+0000 7f6113e00640 1 -- 172.21.15.113:0/519793303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6114161b30 con 0x7f61140c4e10 2024-03-04T08:26:24.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.592+0000 7f610b400640 1 -- 172.21.15.113:0/519793303 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f610c0efc70 con 0x7f61140c4e10 2024-03-04T08:26:24.594 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:24.594 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:24.595 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:24.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 -- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f60f0066f90 msgr2=0x7f60f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:24.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f60f0066f90 0x7f60f0069450 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6104001d50 tx=0x7f6104009040 comp rx=0 tx=0).stop 2024-03-04T08:26:24.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 -- 172.21.15.113:0/519793303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c4e10 msgr2=0x7f61140cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:24.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c4e10 0x7f61140cc3b0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f610c009870 tx=0x7f610c019270 comp rx=0 tx=0).stop 2024-03-04T08:26:24.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 -- 172.21.15.113:0/519793303 shutdown_connections 2024-03-04T08:26:24.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f60f0066f90 0x7f60f0069450 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:24.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61140c4e10 0x7f61140cc3b0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:24.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6114152ac0 0x7f611415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:24.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 --2- 172.21.15.113:0/519793303 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61140c8fe0 0x7f61141f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:24.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 -- 172.21.15.113:0/519793303 >> 172.21.15.113:0/519793303 conn(0x7f6114072e30 msgr2=0x7f611414e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:24.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 -- 172.21.15.113:0/519793303 shutdown_connections 2024-03-04T08:26:24.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:24.596+0000 7f6113e00640 1 -- 172.21.15.113:0/519793303 wait complete. 2024-03-04T08:26:24.615 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:26.617 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:26.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab5cc00640 1 Processor -- start 2024-03-04T08:26:26.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab5cc00640 1 -- start start 2024-03-04T08:26:26.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab5cc00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fab58157df0 0x7fab580c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:26.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab5cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab5806c170 0x7fab580c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:26.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab5cc00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5806de30 con 0x7fab581581d0 2024-03-04T08:26:26.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab5cc00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5806dc00 con 0x7fab58157df0 2024-03-04T08:26:26.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab5cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5806c600 con 0x7fab5806c170 2024-03-04T08:26:26.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab4ea00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fab581581d0 0x7fab5806ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:33044/0 (socket says 172.21.15.113:33044) 2024-03-04T08:26:26.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab4ea00640 1 -- 172.21.15.113:0/980160426 learned_addr learned my addr 172.21.15.113:0/980160426 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:26.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab57400640 1 --2- 172.21.15.113:0/980160426 >> v2:172.21.15.136:3300/0 conn(0x7fab58157df0 0x7fab580c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:26.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab57e00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab5806c170 0x7fab580c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:26.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142489028 0 0) 0x7fab5806de30 con 0x7fab581581d0 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab3c003530 con 0x7fab581581d0 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 120468478 0 0) 0x7fab3c003530 con 0x7fab581581d0 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab5806c170 msgr2=0x7fab580c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab5806c170 0x7fab580c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 >> v2:172.21.15.136:3300/0 conn(0x7fab58157df0 msgr2=0x7fab580c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 --2- 172.21.15.113:0/980160426 >> v2:172.21.15.136:3300/0 conn(0x7fab58157df0 0x7fab580c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5806e110 con 0x7fab581581d0 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fab40001680 con 0x7fab581581d0 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 >> v1:172.21.15.113:6789/0 conn(0x7fab581581d0 legacy=0x7fab5806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab3c0039c0 0x7fab3c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab3c004810 0x7fab3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab5806c170 0x7fab3c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5806c600 con 0x7fab3c004810 2024-03-04T08:26:26.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c00b9d0 con 0x7fab3c0039c0 2024-03-04T08:26:26.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3c00bb50 con 0x7fab5806c170 2024-03-04T08:26:26.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab4ea00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab3c004810 0x7fab3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:26.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.764+0000 7fab57400640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab3c0039c0 0x7fab3c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:26.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab57e00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab5806c170 0x7fab3c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:26.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab4ea00640 1 -- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab5806c170 msgr2=0x7fab3c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:26.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab4ea00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab5806c170 0x7fab3c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab4ea00640 1 -- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab3c0039c0 msgr2=0x7fab3c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:26.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab4ea00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab3c0039c0 0x7fab3c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.769 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab4ea00640 1 -- 172.21.15.113:0/980160426 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3c003530 con 0x7fab3c004810 2024-03-04T08:26:26.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab4ea00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab3c004810 0x7fab3c004c90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fab4000c240 tx=0x7fab4001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f133697287d5dc25 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:26.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40007d40 con 0x7fab3c004810 2024-03-04T08:26:26.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab400a9020 con 0x7fab3c004810 2024-03-04T08:26:26.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab56a00640 1 -- 172.21.15.113:0/980160426 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40005ae0 con 0x7fab3c004810 2024-03-04T08:26:26.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab5cc00640 1 -- 172.21.15.113:0/980160426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab3c004810 msgr2=0x7fab3c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:26.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab5cc00640 1 --2- 172.21.15.113:0/980160426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab3c004810 0x7fab3c004c90 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fab4000c240 tx=0x7fab4001cca0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.768+0000 7fab57e00640 1 -- 172.21.15.113:0/980160426 reap_dead start 2024-03-04T08:26:26.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 -- 172.21.15.113:0/980160426 shutdown_connections 2024-03-04T08:26:26.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 -- 172.21.15.113:0/980160426 >> 172.21.15.113:0/980160426 conn(0x7fab58072f40 msgr2=0x7fab580706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:26.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 -- 172.21.15.113:0/980160426 shutdown_connections 2024-03-04T08:26:26.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 -- 172.21.15.113:0/980160426 wait complete. 2024-03-04T08:26:26.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 Processor -- start 2024-03-04T08:26:26.772 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 -- start start 2024-03-04T08:26:26.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab3c004810 0x7fab5814c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:26.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5814de50 0x7fab580cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:26.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab581f3a10 0x7fab581f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:26.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5806f490 con 0x7fab581f3a10 2024-03-04T08:26:26.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab580ce3a0 con 0x7fab5814de50 2024-03-04T08:26:26.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5806f610 con 0x7fab3c004810 2024-03-04T08:26:26.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab581f3a10 0x7fab581f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:26.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab581f3a10 0x7fab581f3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50070/0 (socket says 172.21.15.113:50070) 2024-03-04T08:26:26.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57e00640 1 -- 172.21.15.113:0/4202451759 learned_addr learned my addr 172.21.15.113:0/4202451759 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:26.775 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57400640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab3c004810 0x7fab5814c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab4ea00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5814de50 0x7fab580cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57e00640 1 -- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab3c004810 msgr2=0x7fab5814c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57e00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab3c004810 0x7fab5814c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57e00640 1 -- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5814de50 msgr2=0x7fab580cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57e00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5814de50 0x7fab580cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57e00640 1 -- 172.21.15.113:0/4202451759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab581e2d30 con 0x7fab581f3a10 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57400640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab3c004810 0x7fab5814c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab4ea00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5814de50 0x7fab580cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab57e00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab581f3a10 0x7fab581f3e90 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fab580b3810 tx=0x7fab50011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab4fe00640 1 -- 172.21.15.113:0/4202451759 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50004240 con 0x7fab581f3a10 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab4fe00640 1 -- 172.21.15.113:0/4202451759 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5001b070 con 0x7fab581f3a10 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab5cc00640 1 -- 172.21.15.113:0/4202451759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab581e2fd0 con 0x7fab581f3a10 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.772+0000 7fab4fe00640 1 -- 172.21.15.113:0/4202451759 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000f710 con 0x7fab581f3a10 2024-03-04T08:26:26.776 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.776+0000 7fab5cc00640 1 -- 172.21.15.113:0/4202451759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab581e3290 con 0x7fab581f3a10 2024-03-04T08:26:26.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.776+0000 7fab5cc00640 1 -- 172.21.15.113:0/4202451759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab58075a80 con 0x7fab581f3a10 2024-03-04T08:26:26.777 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.776+0000 7fab4fe00640 1 -- 172.21.15.113:0/4202451759 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab50012040 con 0x7fab581f3a10 2024-03-04T08:26:26.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.776+0000 7fab4fe00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab34066f60 0x7fab34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:26.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.776+0000 7fab4fe00640 1 -- 172.21.15.113:0/4202451759 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fab50085940 con 0x7fab581f3a10 2024-03-04T08:26:26.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.776+0000 7fab57400640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab34066f60 0x7fab34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:26.778 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.780+0000 7fab57400640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab34066f60 0x7fab34069420 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fab48007520 tx=0x7fab4800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:26.781 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.780+0000 7fab4fe00640 1 -- 172.21.15.113:0/4202451759 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fab50055d80 con 0x7fab581f3a10 2024-03-04T08:26:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 -- 172.21.15.113:0/4202451759 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab58068350 con 0x7fab581f3a10 2024-03-04T08:26:26.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab4fe00640 1 -- 172.21.15.113:0/4202451759 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fab50055680 con 0x7fab581f3a10 2024-03-04T08:26:26.969 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:26.969 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:26.969 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 -- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab34066f60 msgr2=0x7fab34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab34066f60 0x7fab34069420 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fab48007520 tx=0x7fab4800b040 comp rx=0 tx=0).stop 2024-03-04T08:26:26.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 -- 172.21.15.113:0/4202451759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab581f3a10 msgr2=0x7fab581f3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab581f3a10 0x7fab581f3e90 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fab580b3810 tx=0x7fab50011cf0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 -- 172.21.15.113:0/4202451759 shutdown_connections 2024-03-04T08:26:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fab34066f60 0x7fab34069420 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab581f3a10 0x7fab581f3e90 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5814de50 0x7fab580cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 --2- 172.21.15.113:0/4202451759 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fab3c004810 0x7fab5814c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 -- 172.21.15.113:0/4202451759 >> 172.21.15.113:0/4202451759 conn(0x7fab58072f40 msgr2=0x7fab5814f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 -- 172.21.15.113:0/4202451759 shutdown_connections 2024-03-04T08:26:26.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:26.968+0000 7fab5cc00640 1 -- 172.21.15.113:0/4202451759 wait complete. 2024-03-04T08:26:26.990 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:26:26.990 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:26:28.991 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:29.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.136+0000 7fafcbe00640 1 Processor -- start 2024-03-04T08:26:29.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.136+0000 7fafcbe00640 1 -- start start 2024-03-04T08:26:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafc41527f0 0x7fafc4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fafc4152bd0 0x7fafc41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafcbe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc41591f0 con 0x7fafc40c8200 2024-03-04T08:26:29.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafcbe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc4071df0 con 0x7fafc4152bd0 2024-03-04T08:26:29.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafcbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc41597b0 con 0x7fafc41527f0 2024-03-04T08:26:29.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc8a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fafc40c8200 0x7fafc4072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40568/0 (socket says 172.21.15.113:40568) 2024-03-04T08:26:29.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc8a00640 1 -- 172.21.15.113:0/3136108726 learned_addr learned my addr 172.21.15.113:0/3136108726 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:29.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc9e00640 1 --2- 172.21.15.113:0/3136108726 >> v2:172.21.15.136:3300/0 conn(0x7fafc4152bd0 0x7fafc41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:29.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc9400640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafc41527f0 0x7fafc4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:29.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907204462 0 0) 0x7fafc41591f0 con 0x7fafc40c8200 2024-03-04T08:26:29.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafac003530 con 0x7fafc40c8200 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1849092251 0 0) 0x7fafac003530 con 0x7fafc40c8200 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafc41527f0 msgr2=0x7fafc4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafc41527f0 0x7fafc4070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 >> v2:172.21.15.136:3300/0 conn(0x7fafc4152bd0 msgr2=0x7fafc41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 --2- 172.21.15.113:0/3136108726 >> v2:172.21.15.136:3300/0 conn(0x7fafc4152bd0 0x7fafc41587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc41594d0 con 0x7fafc40c8200 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fafb0001680 con 0x7fafc40c8200 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc9e00640 1 --2- 172.21.15.113:0/3136108726 >> v2:172.21.15.136:3300/0 conn(0x7fafc4152bd0 0x7fafc41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 >> v1:172.21.15.113:6789/0 conn(0x7fafc40c8200 legacy=0x7fafc4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafc41527f0 0x7fafac0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:29.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafac003f40 0x7fafac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafac004d50 0x7fafac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc4071df0 con 0x7fafac003f40 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc41597b0 con 0x7fafac004d50 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac00b9d0 con 0x7fafc41527f0 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc8a00640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafac003f40 0x7fafac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc9400640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafc41527f0 0x7fafac0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc9e00640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafac004d50 0x7fafac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc8a00640 1 -- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafc41527f0 msgr2=0x7fafac0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc8a00640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafc41527f0 0x7fafac0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc8a00640 1 -- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafac004d50 msgr2=0x7fafac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc8a00640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafac004d50 0x7fafac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:29.144 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.140+0000 7fafc8a00640 1 -- 172.21.15.113:0/3136108726 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac003530 con 0x7fafac003f40 2024-03-04T08:26:29.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafc8a00640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafac003f40 0x7fafac004810 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c210 tx=0x7fafb001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f831a69126bbc1a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0007d40 con 0x7fafac003f40 2024-03-04T08:26:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0005b00 con 0x7fafac003f40 2024-03-04T08:26:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafc3e00640 1 -- 172.21.15.113:0/3136108726 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0007070 con 0x7fafac003f40 2024-03-04T08:26:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafcbe00640 1 -- 172.21.15.113:0/3136108726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafac003f40 msgr2=0x7fafac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafcbe00640 1 --2- 172.21.15.113:0/3136108726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafac003f40 0x7fafac004810 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c210 tx=0x7fafb001f370 comp rx=0 tx=0).stop 2024-03-04T08:26:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafc9e00640 1 -- 172.21.15.113:0/3136108726 reap_dead start 2024-03-04T08:26:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafcbe00640 1 -- 172.21.15.113:0/3136108726 shutdown_connections 2024-03-04T08:26:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafcbe00640 1 -- 172.21.15.113:0/3136108726 >> 172.21.15.113:0/3136108726 conn(0x7fafc406e820 msgr2=0x7fafc406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafcbe00640 1 -- 172.21.15.113:0/3136108726 shutdown_connections 2024-03-04T08:26:29.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.144+0000 7fafcbe00640 1 -- 172.21.15.113:0/3136108726 wait complete. 2024-03-04T08:26:29.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 Processor -- start 2024-03-04T08:26:29.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 -- start start 2024-03-04T08:26:29.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafac003f40 0x7fafc4153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc41f1e20 0x7fafc41557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4155d20 0x7fafc41480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc4158d10 con 0x7fafc41f1e20 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc4071a80 con 0x7fafc4155d20 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc4158e90 con 0x7fafac003f40 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc41f1e20 0x7fafc41557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc8a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc41f1e20 0x7fafc41557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48168/0 (socket says 172.21.15.113:48168) 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc8a00640 1 -- 172.21.15.113:0/77796104 learned_addr learned my addr 172.21.15.113:0/77796104 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc9400640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafac003f40 0x7fafc4153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc9e00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4155d20 0x7fafc41480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc8a00640 1 -- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafac003f40 msgr2=0x7fafc4153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc8a00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafac003f40 0x7fafc4153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc8a00640 1 -- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4155d20 msgr2=0x7fafc41480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:29.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc8a00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4155d20 0x7fafc41480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc8a00640 1 -- 172.21.15.113:0/77796104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40af850 con 0x7fafc41f1e20 2024-03-04T08:26:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc9400640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafac003f40 0x7fafc4153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc9e00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4155d20 0x7fafc41480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc8a00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc41f1e20 0x7fafc41557e0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fafb001f1a0 tx=0x7fafac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc1600640 1 -- 172.21.15.113:0/77796104 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0000ee0 con 0x7fafc41f1e20 2024-03-04T08:26:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc1600640 1 -- 172.21.15.113:0/77796104 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0005cb0 con 0x7fafc41f1e20 2024-03-04T08:26:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 -- 172.21.15.113:0/77796104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc4155110 con 0x7fafc41f1e20 2024-03-04T08:26:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc1600640 1 -- 172.21.15.113:0/77796104 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0005e60 con 0x7fafc41f1e20 2024-03-04T08:26:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 -- 172.21.15.113:0/77796104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc40afdf0 con 0x7fafc41f1e20 2024-03-04T08:26:29.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafcbe00640 1 -- 172.21.15.113:0/77796104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc40affc0 con 0x7fafc41f1e20 2024-03-04T08:26:29.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc1600640 1 -- 172.21.15.113:0/77796104 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafb00ba030 con 0x7fafc41f1e20 2024-03-04T08:26:29.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc1600640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafa4067100 0x7fafa40695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:29.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc1600640 1 -- 172.21.15.113:0/77796104 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fafb01225b0 con 0x7fafc41f1e20 2024-03-04T08:26:29.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc9400640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafa4067100 0x7fafa40695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:29.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.152+0000 7fafc9400640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafa4067100 0x7fafa40695c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009a80 tx=0x7fafb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:29.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.156+0000 7fafc1600640 1 -- 172.21.15.113:0/77796104 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fafb00c1030 con 0x7fafc41f1e20 2024-03-04T08:26:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 -- 172.21.15.113:0/77796104 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafc40c79d0 con 0x7fafc41f1e20 2024-03-04T08:26:29.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafc1600640 1 -- 172.21.15.113:0/77796104 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fafb00190e0 con 0x7fafc41f1e20 2024-03-04T08:26:29.344 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:29.344 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:29.344 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:29.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 -- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafa4067100 msgr2=0x7fafa40695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafa4067100 0x7fafa40695c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009a80 tx=0x7fafb8008040 comp rx=0 tx=0).stop 2024-03-04T08:26:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 -- 172.21.15.113:0/77796104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc41f1e20 msgr2=0x7fafc41557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc41f1e20 0x7fafc41557e0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fafb001f1a0 tx=0x7fafac005840 comp rx=0 tx=0).stop 2024-03-04T08:26:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 -- 172.21.15.113:0/77796104 shutdown_connections 2024-03-04T08:26:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fafc4155d20 0x7fafc41480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fafa4067100 0x7fafa40695c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fafc41f1e20 0x7fafc41557e0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 --2- 172.21.15.113:0/77796104 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fafac003f40 0x7fafc4153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:29.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 -- 172.21.15.113:0/77796104 >> 172.21.15.113:0/77796104 conn(0x7fafc406e820 msgr2=0x7fafc40ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 -- 172.21.15.113:0/77796104 shutdown_connections 2024-03-04T08:26:29.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:29.344+0000 7fafcbe00640 1 -- 172.21.15.113:0/77796104 wait complete. 2024-03-04T08:26:29.366 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:31.366 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:31.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41f0400640 1 Processor -- start 2024-03-04T08:26:31.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41f0400640 1 -- start start 2024-03-04T08:26:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41f0400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f41e81527f0 0x7f41e8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41f0400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e81591f0 con 0x7f41e80c8200 2024-03-04T08:26:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41f0400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e81594d0 con 0x7f41e8152bd0 2024-03-04T08:26:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41f0400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e8071df0 con 0x7f41e81527f0 2024-03-04T08:26:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41ed000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f41e80c8200 0x7f41e8072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40570/0 (socket says 172.21.15.113:40570) 2024-03-04T08:26:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41ed000640 1 -- 172.21.15.113:0/3426844434 learned_addr learned my addr 172.21.15.113:0/3426844434 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:31.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41eda00640 1 --2- 172.21.15.113:0/3426844434 >> v2:172.21.15.136:3300/0 conn(0x7f41e81527f0 0x7f41e8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:31.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41e7e00640 1 -- 172.21.15.113:0/3426844434 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530607444 0 0) 0x7f41e81591f0 con 0x7f41e80c8200 2024-03-04T08:26:31.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.516+0000 7f41e7e00640 1 -- 172.21.15.113:0/3426844434 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d0003530 con 0x7f41e80c8200 2024-03-04T08:26:31.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41eda00640 1 -- 172.21.15.113:0/3426844434 >> v1:172.21.15.136:6789/0 conn(0x7f41e8152bd0 legacy=0x7f41e81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:31.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41e7e00640 1 -- 172.21.15.113:0/3426844434 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 781826031 0 0) 0x7f41d0003530 con 0x7f41e80c8200 2024-03-04T08:26:31.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41eda00640 1 -- 172.21.15.113:0/3426844434 >> v1:172.21.15.113:6789/0 conn(0x7f41e80c8200 legacy=0x7f41e8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:31.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41eda00640 1 -- 172.21.15.113:0/3426844434 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80717a0 con 0x7f41e81527f0 2024-03-04T08:26:31.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41eda00640 1 --2- 172.21.15.113:0/3426844434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e81527f0 0x7f41e8070d40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097c0 tx=0x7f41dc01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b6f16c026c22779 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:31.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41e7e00640 1 -- 172.21.15.113:0/3426844434 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0b0070 con 0x7f41e81527f0 2024-03-04T08:26:31.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41e7e00640 1 -- 172.21.15.113:0/3426844434 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc0ab400 con 0x7f41e81527f0 2024-03-04T08:26:31.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41e7e00640 1 -- 172.21.15.113:0/3426844434 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4001680 con 0x7f41e81527f0 2024-03-04T08:26:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41f0400640 1 -- 172.21.15.113:0/3426844434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e81527f0 msgr2=0x7f41e8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41f0400640 1 --2- 172.21.15.113:0/3426844434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e81527f0 0x7f41e8070d40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097c0 tx=0x7f41dc01c3c0 comp rx=0 tx=0).stop 2024-03-04T08:26:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41f0400640 1 -- 172.21.15.113:0/3426844434 shutdown_connections 2024-03-04T08:26:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41f0400640 1 --2- 172.21.15.113:0/3426844434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e81527f0 0x7f41e8070d40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.520+0000 7f41f0400640 1 -- 172.21.15.113:0/3426844434 >> 172.21.15.113:0/3426844434 conn(0x7f41e806e820 msgr2=0x7f41e806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 -- 172.21.15.113:0/3426844434 shutdown_connections 2024-03-04T08:26:31.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 -- 172.21.15.113:0/3426844434 wait complete. 2024-03-04T08:26:31.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 Processor -- start 2024-03-04T08:26:31.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 -- start start 2024-03-04T08:26:31.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41e80c8200 0x7f41e80c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:31.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41e81527f0 0x7f41e80b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:31.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e8152bd0 0x7f41e80b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:31.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8166380 con 0x7f41e80c8200 2024-03-04T08:26:31.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e81597b0 con 0x7f41e8152bd0 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8166500 con 0x7f41e81527f0 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41eda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41e80c8200 0x7f41e80c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41eda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41e80c8200 0x7f41e80c3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48180/0 (socket says 172.21.15.113:48180) 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41eda00640 1 -- 172.21.15.113:0/26585233 learned_addr learned my addr 172.21.15.113:0/26585233 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41ee400640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e8152bd0 0x7f41e80b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41ed000640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41e81527f0 0x7f41e80b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41eda00640 1 -- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41e81527f0 msgr2=0x7f41e80b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41eda00640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41e81527f0 0x7f41e80b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41eda00640 1 -- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e8152bd0 msgr2=0x7f41e80b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41eda00640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e8152bd0 0x7f41e80b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41eda00640 1 -- 172.21.15.113:0/26585233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e814ad90 con 0x7f41e80c8200 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41ee400640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e8152bd0 0x7f41e80b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41eda00640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41e80c8200 0x7f41e80c3530 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0ae040 tx=0x7f41dc01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:31.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41e5600640 1 -- 172.21.15.113:0/26585233 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0b0050 con 0x7f41e80c8200 2024-03-04T08:26:31.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41e5600640 1 -- 172.21.15.113:0/26585233 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc0bc070 con 0x7f41e80c8200 2024-03-04T08:26:31.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 -- 172.21.15.113:0/26585233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e8149d30 con 0x7f41e80c8200 2024-03-04T08:26:31.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41e5600640 1 -- 172.21.15.113:0/26585233 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0b7850 con 0x7f41e80c8200 2024-03-04T08:26:31.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.524+0000 7f41f0400640 1 -- 172.21.15.113:0/26585233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e814b050 con 0x7f41e80c8200 2024-03-04T08:26:31.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.528+0000 7f41f0400640 1 -- 172.21.15.113:0/26585233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e8075c00 con 0x7f41e80c8200 2024-03-04T08:26:31.524 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.528+0000 7f41e5600640 1 -- 172.21.15.113:0/26585233 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f41dc0b79f0 con 0x7f41e80c8200 2024-03-04T08:26:31.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.528+0000 7f41e5600640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f41c8066f60 0x7f41c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:31.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.528+0000 7f41e5600640 1 -- 172.21.15.113:0/26585233 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f41dc120050 con 0x7f41e80c8200 2024-03-04T08:26:31.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.528+0000 7f41ed000640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f41c8066f60 0x7f41c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:31.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.528+0000 7f41ed000640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f41c8066f60 0x7f41c8069420 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f41e814b590 tx=0x7f41d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:31.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.532+0000 7f41e5600640 1 -- 172.21.15.113:0/26585233 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f41dc015fb0 con 0x7f41e80c8200 2024-03-04T08:26:31.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 -- 172.21.15.113:0/26585233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41e80c79d0 con 0x7f41e80c8200 2024-03-04T08:26:31.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41e5600640 1 -- 172.21.15.113:0/26585233 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f41dc0f0730 con 0x7f41e80c8200 2024-03-04T08:26:31.716 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:31.716 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:31.716 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 -- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f41c8066f60 msgr2=0x7f41c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f41c8066f60 0x7f41c8069420 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f41e814b590 tx=0x7f41d4009040 comp rx=0 tx=0).stop 2024-03-04T08:26:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 -- 172.21.15.113:0/26585233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41e80c8200 msgr2=0x7f41e80c3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41e80c8200 0x7f41e80c3530 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0ae040 tx=0x7f41dc01ac00 comp rx=0 tx=0).stop 2024-03-04T08:26:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 -- 172.21.15.113:0/26585233 shutdown_connections 2024-03-04T08:26:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41e8152bd0 0x7f41e80b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f41e81527f0 0x7f41e80b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:31.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f41c8066f60 0x7f41c8069420 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:31.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 --2- 172.21.15.113:0/26585233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f41e80c8200 0x7f41e80c3530 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:31.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 -- 172.21.15.113:0/26585233 >> 172.21.15.113:0/26585233 conn(0x7f41e806e820 msgr2=0x7f41e80cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:31.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 -- 172.21.15.113:0/26585233 shutdown_connections 2024-03-04T08:26:31.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:31.720+0000 7f41f0400640 1 -- 172.21.15.113:0/26585233 wait complete. 2024-03-04T08:26:31.737 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:33.739 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:33.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5109400640 1 Processor -- start 2024-03-04T08:26:33.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5109400640 1 -- start start 2024-03-04T08:26:33.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5109400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51040b20b0 0x7f5104157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:33.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5109400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5104157c50 0x7f510406b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:33.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5109400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51040c1ec0 con 0x7f51040c85f0 2024-03-04T08:26:33.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5109400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51040c9b30 con 0x7f5104157c50 2024-03-04T08:26:33.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5109400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51040c12a0 con 0x7f51040b20b0 2024-03-04T08:26:33.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5101e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5104157c50 0x7f510406b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:33.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5101e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5104157c50 0x7f510406b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48190/0 (socket says 172.21.15.113:48190) 2024-03-04T08:26:33.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5101e00640 1 -- 172.21.15.113:0/3865188001 learned_addr learned my addr 172.21.15.113:0/3865188001 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:33.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5103200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f51040c85f0 0x7f510415a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40586/0 (socket says 172.21.15.113:40586) 2024-03-04T08:26:33.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.888+0000 7f5102800640 1 --2- 172.21.15.113:0/3865188001 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51040b20b0 0x7f5104157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:33.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5101e00640 1 -- 172.21.15.113:0/3865188001 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51040b20b0 msgr2=0x7f5104157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:33.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5101e00640 1 --2- 172.21.15.113:0/3865188001 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51040b20b0 0x7f5104157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:33.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5101e00640 1 -- 172.21.15.113:0/3865188001 >> v1:172.21.15.113:6789/0 conn(0x7f51040c85f0 legacy=0x7f510415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:33.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5101e00640 1 -- 172.21.15.113:0/3865188001 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51040c1580 con 0x7f5104157c50 2024-03-04T08:26:33.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5101400640 1 -- 172.21.15.113:0/3865188001 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126614725 0 0) 0x7f51040c1ec0 con 0x7f51040c85f0 2024-03-04T08:26:33.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5101e00640 1 --2- 172.21.15.113:0/3865188001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5104157c50 0x7f510406b060 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f50f0006fd0 tx=0x7f50f001e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bb395fe47c71a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5101400640 1 -- 172.21.15.113:0/3865188001 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00ad070 con 0x7f5104157c50 2024-03-04T08:26:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5101400640 1 -- 172.21.15.113:0/3865188001 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0004230 con 0x7f5104157c50 2024-03-04T08:26:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5101400640 1 -- 172.21.15.113:0/3865188001 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00a7720 con 0x7f5104157c50 2024-03-04T08:26:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5109400640 1 -- 172.21.15.113:0/3865188001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5104157c50 msgr2=0x7f510406b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:33.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5109400640 1 --2- 172.21.15.113:0/3865188001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5104157c50 0x7f510406b060 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f50f0006fd0 tx=0x7f50f001e9c0 comp rx=0 tx=0).stop 2024-03-04T08:26:33.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5109400640 1 -- 172.21.15.113:0/3865188001 shutdown_connections 2024-03-04T08:26:33.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5109400640 1 --2- 172.21.15.113:0/3865188001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5104157c50 0x7f510406b060 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:33.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5109400640 1 --2- 172.21.15.113:0/3865188001 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f51040b20b0 0x7f5104157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:33.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5109400640 1 -- 172.21.15.113:0/3865188001 >> 172.21.15.113:0/3865188001 conn(0x7f510406e820 msgr2=0x7f510406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:33.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5109400640 1 -- 172.21.15.113:0/3865188001 shutdown_connections 2024-03-04T08:26:33.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.892+0000 7f5109400640 1 -- 172.21.15.113:0/3865188001 wait complete. 2024-03-04T08:26:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5109400640 1 Processor -- start 2024-03-04T08:26:33.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5109400640 1 -- start start 2024-03-04T08:26:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5109400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51040b20b0 0x7f5104156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5109400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51040c85f0 0x7f510414ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5109400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5104157c50 0x7f510414f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5109400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51040c1b80 con 0x7f51040b20b0 2024-03-04T08:26:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5109400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5104156ec0 con 0x7f51040c85f0 2024-03-04T08:26:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5109400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51040c80c0 con 0x7f5104157c50 2024-03-04T08:26:33.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5102800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51040b20b0 0x7f5104156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5102800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51040b20b0 0x7f5104156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48198/0 (socket says 172.21.15.113:48198) 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5102800640 1 -- 172.21.15.113:0/1755697008 learned_addr learned my addr 172.21.15.113:0/1755697008 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5101e00640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51040c85f0 0x7f510414ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5103200640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5104157c50 0x7f510414f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5102800640 1 -- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5104157c50 msgr2=0x7f510414f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5102800640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5104157c50 0x7f510414f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5102800640 1 -- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51040c85f0 msgr2=0x7f510414ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5102800640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51040c85f0 0x7f510414ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5102800640 1 -- 172.21.15.113:0/1755697008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51040c33b0 con 0x7f51040b20b0 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5102800640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51040b20b0 0x7f5104156540 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009d10 tx=0x7f50f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f50faa00640 1 -- 172.21.15.113:0/1755697008 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4018020 con 0x7f51040b20b0 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f50faa00640 1 -- 172.21.15.113:0/1755697008 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f4014480 con 0x7f51040b20b0 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f50faa00640 1 -- 172.21.15.113:0/1755697008 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f4014830 con 0x7f51040b20b0 2024-03-04T08:26:33.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5109400640 1 -- 172.21.15.113:0/1755697008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51040c3930 con 0x7f51040b20b0 2024-03-04T08:26:33.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.896+0000 7f5109400640 1 -- 172.21.15.113:0/1755697008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51040c3650 con 0x7f51040b20b0 2024-03-04T08:26:33.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.900+0000 7f5109400640 1 -- 172.21.15.113:0/1755697008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5104075c00 con 0x7f51040b20b0 2024-03-04T08:26:33.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.900+0000 7f50faa00640 1 -- 172.21.15.113:0/1755697008 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50f401b020 con 0x7f51040b20b0 2024-03-04T08:26:33.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.900+0000 7f50faa00640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50e4066f90 0x7f50e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:33.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.900+0000 7f50faa00640 1 -- 172.21.15.113:0/1755697008 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f50f4086560 con 0x7f51040b20b0 2024-03-04T08:26:33.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.900+0000 7f5101e00640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50e4066f90 0x7f50e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:33.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.900+0000 7f5101e00640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50e4066f90 0x7f50e4069450 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f50f00b1000 tx=0x7f50f00a9070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:33.901 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:33.904+0000 7f50faa00640 1 -- 172.21.15.113:0/1755697008 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f50f4022070 con 0x7f51040b20b0 2024-03-04T08:26:34.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.092+0000 7f5109400640 1 -- 172.21.15.113:0/1755697008 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51040cba70 con 0x7f51040b20b0 2024-03-04T08:26:34.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.092+0000 7f50faa00640 1 -- 172.21.15.113:0/1755697008 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f50f4010040 con 0x7f51040b20b0 2024-03-04T08:26:34.089 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:34.089 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:34.090 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 -- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50e4066f90 msgr2=0x7f50e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50e4066f90 0x7f50e4069450 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f50f00b1000 tx=0x7f50f00a9070 comp rx=0 tx=0).stop 2024-03-04T08:26:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 -- 172.21.15.113:0/1755697008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51040b20b0 msgr2=0x7f5104156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51040b20b0 0x7f5104156540 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009d10 tx=0x7f50f400c490 comp rx=0 tx=0).stop 2024-03-04T08:26:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 -- 172.21.15.113:0/1755697008 shutdown_connections 2024-03-04T08:26:34.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5104157c50 0x7f510414f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f51040c85f0 0x7f510414ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f50e4066f90 0x7f50e4069450 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 --2- 172.21.15.113:0/1755697008 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51040b20b0 0x7f5104156540 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 -- 172.21.15.113:0/1755697008 >> 172.21.15.113:0/1755697008 conn(0x7f510406e820 msgr2=0x7f5104072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 -- 172.21.15.113:0/1755697008 shutdown_connections 2024-03-04T08:26:34.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:34.096+0000 7f5109400640 1 -- 172.21.15.113:0/1755697008 wait complete. 2024-03-04T08:26:34.110 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:36.112 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:36.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f822c200640 1 Processor -- start 2024-03-04T08:26:36.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f822c200640 1 -- start start 2024-03-04T08:26:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f822c200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f82240c8200 0x7f8224070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f822c200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8224152bd0 0x7f82241587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f822c200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82241591f0 con 0x7f82241527f0 2024-03-04T08:26:36.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f822c200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8224071df0 con 0x7f82240c8200 2024-03-04T08:26:36.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f822c200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82241597b0 con 0x7f8224152bd0 2024-03-04T08:26:36.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f8229800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f82241527f0 0x7f8224072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40594/0 (socket says 172.21.15.113:40594) 2024-03-04T08:26:36.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f8229800640 1 -- 172.21.15.113:0/617721274 learned_addr learned my addr 172.21.15.113:0/617721274 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:36.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f822a200640 1 --2- 172.21.15.113:0/617721274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8224152bd0 0x7f82241587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:36.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f8228e00640 1 --2- 172.21.15.113:0/617721274 >> v2:172.21.15.136:3300/0 conn(0x7f82240c8200 0x7f8224070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:36.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f8223e00640 1 -- 172.21.15.113:0/617721274 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378554927 0 0) 0x7f82241591f0 con 0x7f82241527f0 2024-03-04T08:26:36.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f8223e00640 1 -- 172.21.15.113:0/617721274 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f820c003530 con 0x7f82241527f0 2024-03-04T08:26:36.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f8223e00640 1 -- 172.21.15.113:0/617721274 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 512501389 0 0) 0x7f820c003530 con 0x7f82241527f0 2024-03-04T08:26:36.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f8228e00640 1 -- 172.21.15.113:0/617721274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8224152bd0 msgr2=0x7f82241587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:36.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f8228e00640 1 --2- 172.21.15.113:0/617721274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8224152bd0 0x7f82241587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:36.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f8228e00640 1 -- 172.21.15.113:0/617721274 >> v1:172.21.15.113:6789/0 conn(0x7f82241527f0 legacy=0x7f8224072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:36.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.260+0000 7f8228e00640 1 -- 172.21.15.113:0/617721274 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82241594d0 con 0x7f82240c8200 2024-03-04T08:26:36.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f8228e00640 1 --2- 172.21.15.113:0/617721274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82240c8200 0x7f8224070d40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f82100097c0 tx=0x7f821001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=475ead0f13ee0324 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:36.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f8223e00640 1 -- 172.21.15.113:0/617721274 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82100b0070 con 0x7f82240c8200 2024-03-04T08:26:36.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f8223e00640 1 -- 172.21.15.113:0/617721274 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82100ab400 con 0x7f82240c8200 2024-03-04T08:26:36.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f8223e00640 1 -- 172.21.15.113:0/617721274 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218001680 con 0x7f82240c8200 2024-03-04T08:26:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f822c200640 1 -- 172.21.15.113:0/617721274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82240c8200 msgr2=0x7f8224070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f822c200640 1 --2- 172.21.15.113:0/617721274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82240c8200 0x7f8224070d40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f82100097c0 tx=0x7f821001c3c0 comp rx=0 tx=0).stop 2024-03-04T08:26:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f822c200640 1 -- 172.21.15.113:0/617721274 shutdown_connections 2024-03-04T08:26:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f822c200640 1 --2- 172.21.15.113:0/617721274 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8224152bd0 0x7f82241587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f822c200640 1 --2- 172.21.15.113:0/617721274 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82240c8200 0x7f8224070d40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f822c200640 1 -- 172.21.15.113:0/617721274 >> 172.21.15.113:0/617721274 conn(0x7f822406e820 msgr2=0x7f822406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f822c200640 1 -- 172.21.15.113:0/617721274 shutdown_connections 2024-03-04T08:26:36.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.264+0000 7f822c200640 1 -- 172.21.15.113:0/617721274 wait complete. 2024-03-04T08:26:36.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 Processor -- start 2024-03-04T08:26:36.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 -- start start 2024-03-04T08:26:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82240c8200 0x7f82240b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82241527f0 0x7f82240af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8224152bd0 0x7f8224141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82241654f0 con 0x7f8224152bd0 2024-03-04T08:26:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8224165370 con 0x7f82241527f0 2024-03-04T08:26:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8224165670 con 0x7f82240c8200 2024-03-04T08:26:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8224152bd0 0x7f8224141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8224152bd0 0x7f8224141d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48202/0 (socket says 172.21.15.113:48202) 2024-03-04T08:26:36.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822a200640 1 -- 172.21.15.113:0/2095663682 learned_addr learned my addr 172.21.15.113:0/2095663682 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f8229800640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82240c8200 0x7f82240b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f8228e00640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82241527f0 0x7f82240af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822a200640 1 -- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82240c8200 msgr2=0x7f82240b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822a200640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82240c8200 0x7f82240b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822a200640 1 -- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82241527f0 msgr2=0x7f82240af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822a200640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82241527f0 0x7f82240af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822a200640 1 -- 172.21.15.113:0/2095663682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8224156660 con 0x7f8224152bd0 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f8228e00640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82241527f0 0x7f82240af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822a200640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8224152bd0 0x7f8224141d20 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f821c00efc0 tx=0x7f821c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f8221600640 1 -- 172.21.15.113:0/2095663682 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c018020 con 0x7f8224152bd0 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f8221600640 1 -- 172.21.15.113:0/2095663682 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c014480 con 0x7f8224152bd0 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 -- 172.21.15.113:0/2095663682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8224156920 con 0x7f8224152bd0 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f8221600640 1 -- 172.21.15.113:0/2095663682 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c014870 con 0x7f8224152bd0 2024-03-04T08:26:36.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 -- 172.21.15.113:0/2095663682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822414b9e0 con 0x7f8224152bd0 2024-03-04T08:26:36.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.268+0000 7f822c200640 1 -- 172.21.15.113:0/2095663682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8224075c00 con 0x7f8224152bd0 2024-03-04T08:26:36.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.272+0000 7f8221600640 1 -- 172.21.15.113:0/2095663682 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f821c01b020 con 0x7f8224152bd0 2024-03-04T08:26:36.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.272+0000 7f8221600640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8204066f60 0x7f8204069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:36.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.272+0000 7f8221600640 1 -- 172.21.15.113:0/2095663682 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f821c086600 con 0x7f8224152bd0 2024-03-04T08:26:36.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.272+0000 7f8229800640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8204066f60 0x7f8204069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:36.269 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.272+0000 7f8229800640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8204066f60 0x7f8204069420 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8218007a10 tx=0x7f8218009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:36.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.276+0000 7f8221600640 1 -- 172.21.15.113:0/2095663682 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f821c022070 con 0x7f8224152bd0 2024-03-04T08:26:36.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.460+0000 7f822c200640 1 -- 172.21.15.113:0/2095663682 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82240c79d0 con 0x7f8224152bd0 2024-03-04T08:26:36.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f8221600640 1 -- 172.21.15.113:0/2095663682 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f821c010040 con 0x7f8224152bd0 2024-03-04T08:26:36.459 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:36.459 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:36.459 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:36.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 -- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8204066f60 msgr2=0x7f8204069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:36.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8204066f60 0x7f8204069420 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8218007a10 tx=0x7f8218009040 comp rx=0 tx=0).stop 2024-03-04T08:26:36.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 -- 172.21.15.113:0/2095663682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8224152bd0 msgr2=0x7f8224141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:36.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8224152bd0 0x7f8224141d20 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f821c00efc0 tx=0x7f821c00c490 comp rx=0 tx=0).stop 2024-03-04T08:26:36.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 -- 172.21.15.113:0/2095663682 shutdown_connections 2024-03-04T08:26:36.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8204066f60 0x7f8204069420 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:36.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8224152bd0 0x7f8224141d20 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:36.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82241527f0 0x7f82240af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:36.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 --2- 172.21.15.113:0/2095663682 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82240c8200 0x7f82240b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:36.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 -- 172.21.15.113:0/2095663682 >> 172.21.15.113:0/2095663682 conn(0x7f822406e820 msgr2=0x7f82240c1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:36.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 -- 172.21.15.113:0/2095663682 shutdown_connections 2024-03-04T08:26:36.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:36.464+0000 7f822c200640 1 -- 172.21.15.113:0/2095663682 wait complete. 2024-03-04T08:26:36.480 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:38.481 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:38.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.632+0000 7f27e2800640 1 Processor -- start 2024-03-04T08:26:38.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.632+0000 7f27e2800640 1 -- start start 2024-03-04T08:26:38.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.632+0000 7f27e2800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27dc0c22e0 0x7f27dc15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.632+0000 7f27e2800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27dc0c27f0 con 0x7f27dc0c2d80 2024-03-04T08:26:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.632+0000 7f27e2800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27dc0c2a90 con 0x7f27dc14e4c0 2024-03-04T08:26:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.632+0000 7f27e2800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27dc15ad60 con 0x7f27dc0c22e0 2024-03-04T08:26:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.632+0000 7f27d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27dc0c22e0 0x7f27dc15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27dbe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f27dc0c2d80 0x7f27dc1620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40388/0 (socket says 172.21.15.113:40388) 2024-03-04T08:26:38.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27dbe00640 1 -- 172.21.15.113:0/3116496108 learned_addr learned my addr 172.21.15.113:0/3116496108 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27daa00640 1 -- 172.21.15.113:0/3116496108 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3544412261 0 0) 0x7f27dc0c27f0 con 0x7f27dc0c2d80 2024-03-04T08:26:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27daa00640 1 -- 172.21.15.113:0/3116496108 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27c0005740 con 0x7f27dc0c2d80 2024-03-04T08:26:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27d2a00640 1 -- 172.21.15.113:0/3116496108 >> v1:172.21.15.136:6789/0 conn(0x7f27dc14e4c0 legacy=0x7f27dc072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27daa00640 1 -- 172.21.15.113:0/3116496108 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649682569 0 0) 0x7f27dc0c2a90 con 0x7f27dc14e4c0 2024-03-04T08:26:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27d2a00640 1 -- 172.21.15.113:0/3116496108 >> v1:172.21.15.113:6789/0 conn(0x7f27dc0c2d80 legacy=0x7f27dc1620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:38.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27d2a00640 1 -- 172.21.15.113:0/3116496108 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc070c60 con 0x7f27dc0c22e0 2024-03-04T08:26:38.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27d2a00640 1 --2- 172.21.15.113:0/3116496108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc0c22e0 0x7f27dc15b1f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f27c40097c0 tx=0x7f27c401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f221df6753334ef server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:38.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27daa00640 1 -- 172.21.15.113:0/3116496108 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40b0070 con 0x7f27dc0c22e0 2024-03-04T08:26:38.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27daa00640 1 -- 172.21.15.113:0/3116496108 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c40ab600 con 0x7f27dc0c22e0 2024-03-04T08:26:38.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.636+0000 7f27daa00640 1 -- 172.21.15.113:0/3116496108 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40ab9d0 con 0x7f27dc0c22e0 2024-03-04T08:26:38.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 -- 172.21.15.113:0/3116496108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc0c22e0 msgr2=0x7f27dc15b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 --2- 172.21.15.113:0/3116496108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc0c22e0 0x7f27dc15b1f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f27c40097c0 tx=0x7f27c401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:26:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 -- 172.21.15.113:0/3116496108 shutdown_connections 2024-03-04T08:26:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 --2- 172.21.15.113:0/3116496108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc0c22e0 0x7f27dc15b1f0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 -- 172.21.15.113:0/3116496108 >> 172.21.15.113:0/3116496108 conn(0x7f27dc06e820 msgr2=0x7f27dc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 -- 172.21.15.113:0/3116496108 shutdown_connections 2024-03-04T08:26:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 -- 172.21.15.113:0/3116496108 wait complete. 2024-03-04T08:26:38.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 Processor -- start 2024-03-04T08:26:38.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 -- start start 2024-03-04T08:26:38.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27dc0c22e0 0x7f27dc146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:38.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27dc0c2d80 0x7f27dc0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:38.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc14e4c0 0x7f27dc1536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:38.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc14e8a0 con 0x7f27dc14e4c0 2024-03-04T08:26:38.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc15bb40 con 0x7f27dc0c22e0 2024-03-04T08:26:38.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27e2800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc0aede0 con 0x7f27dc0c2d80 2024-03-04T08:26:38.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc14e4c0 0x7f27dc1536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:38.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc14e4c0 0x7f27dc1536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42742/0 (socket says 172.21.15.113:42742) 2024-03-04T08:26:38.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27dbe00640 1 -- 172.21.15.113:0/3303025494 learned_addr learned my addr 172.21.15.113:0/3303025494 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27d2a00640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27dc0c2d80 0x7f27dc0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27db400640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27dc0c22e0 0x7f27dc146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27dbe00640 1 -- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27dc0c2d80 msgr2=0x7f27dc0cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27dbe00640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27dc0c2d80 0x7f27dc0cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27dbe00640 1 -- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27dc0c22e0 msgr2=0x7f27dc146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27dbe00640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27dc0c22e0 0x7f27dc146db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27dbe00640 1 -- 172.21.15.113:0/3303025494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc145e30 con 0x7f27dc14e4c0 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.640+0000 7f27db400640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27dc0c22e0 0x7f27dc146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27dbe00640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc14e4c0 0x7f27dc1536f0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f27d400cad0 tx=0x7f27d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27d3e00640 1 -- 172.21.15.113:0/3303025494 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4017070 con 0x7f27dc14e4c0 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27d3e00640 1 -- 172.21.15.113:0/3303025494 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d400ac30 con 0x7f27dc14e4c0 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27d3e00640 1 -- 172.21.15.113:0/3303025494 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4012400 con 0x7f27dc14e4c0 2024-03-04T08:26:38.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27e2800640 1 -- 172.21.15.113:0/3303025494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27dc146110 con 0x7f27dc14e4c0 2024-03-04T08:26:38.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27e2800640 1 -- 172.21.15.113:0/3303025494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc152ef0 con 0x7f27dc14e4c0 2024-03-04T08:26:38.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27e2800640 1 -- 172.21.15.113:0/3303025494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc075c00 con 0x7f27dc14e4c0 2024-03-04T08:26:38.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27d3e00640 1 -- 172.21.15.113:0/3303025494 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27d401a070 con 0x7f27dc14e4c0 2024-03-04T08:26:38.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27d3e00640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27b8066f60 0x7f27b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:38.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27d3e00640 1 -- 172.21.15.113:0/3303025494 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f27d40857c0 con 0x7f27dc14e4c0 2024-03-04T08:26:38.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.644+0000 7f27db400640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27b8066f60 0x7f27b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:38.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.648+0000 7f27db400640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27b8066f60 0x7f27b8069420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f27cc007fd0 tx=0x7f27cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:38.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.648+0000 7f27d3e00640 1 -- 172.21.15.113:0/3303025494 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f27d40556d0 con 0x7f27dc14e4c0 2024-03-04T08:26:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.836+0000 7f27e2800640 1 -- 172.21.15.113:0/3303025494 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27dc0c8b90 con 0x7f27dc14e4c0 2024-03-04T08:26:38.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.836+0000 7f27d3e00640 1 -- 172.21.15.113:0/3303025494 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f27d4059770 con 0x7f27dc14e4c0 2024-03-04T08:26:38.835 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:38.835 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:38.835 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 -- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27b8066f60 msgr2=0x7f27b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27b8066f60 0x7f27b8069420 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f27cc007fd0 tx=0x7f27cc009040 comp rx=0 tx=0).stop 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 -- 172.21.15.113:0/3303025494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc14e4c0 msgr2=0x7f27dc1536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc14e4c0 0x7f27dc1536f0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f27d400cad0 tx=0x7f27d400cf90 comp rx=0 tx=0).stop 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 -- 172.21.15.113:0/3303025494 shutdown_connections 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27b8066f60 0x7f27b8069420 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27dc14e4c0 0x7f27dc1536f0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27dc0c2d80 0x7f27dc0cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 --2- 172.21.15.113:0/3303025494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27dc0c22e0 0x7f27dc146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 -- 172.21.15.113:0/3303025494 >> 172.21.15.113:0/3303025494 conn(0x7f27dc06e820 msgr2=0x7f27dc072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:38.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 -- 172.21.15.113:0/3303025494 shutdown_connections 2024-03-04T08:26:38.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:38.840+0000 7f27e2800640 1 -- 172.21.15.113:0/3303025494 wait complete. 2024-03-04T08:26:38.855 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:26:38.856 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:26:40.857 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:41.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb661c00640 1 Processor -- start 2024-03-04T08:26:41.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb661c00640 1 -- start start 2024-03-04T08:26:41.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb661c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb65c0ca7c0 0x7fb65c15b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:41.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb661c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c072530 con 0x7fb65c06fd80 2024-03-04T08:26:41.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb661c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c071cb0 con 0x7fb65c06f9a0 2024-03-04T08:26:41.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb661c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65c071f70 con 0x7fb65c0ca7c0 2024-03-04T08:26:41.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb652000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb65c0ca7c0 0x7fb65c15b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:41.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb652000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb65c0ca7c0 0x7fb65c15b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42750/0 (socket says 172.21.15.113:42750) 2024-03-04T08:26:41.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb652000640 1 -- 172.21.15.113:0/4190763368 learned_addr learned my addr 172.21.15.113:0/4190763368 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:41.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb65b400640 1 --1- 172.21.15.113:0/4190763368 >> v1:172.21.15.113:6789/0 conn(0x7fb65c06fd80 0x7fb65c154920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40398/0 (socket says 172.21.15.113:40398) 2024-03-04T08:26:41.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb652000640 1 -- 172.21.15.113:0/4190763368 >> v1:172.21.15.136:6789/0 conn(0x7fb65c06f9a0 legacy=0x7fb65c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:41.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb652000640 1 -- 172.21.15.113:0/4190763368 >> v1:172.21.15.113:6789/0 conn(0x7fb65c06fd80 legacy=0x7fb65c154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:41.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb652000640 1 -- 172.21.15.113:0/4190763368 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c072250 con 0x7fb65c0ca7c0 2024-03-04T08:26:41.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.008+0000 7fb65a000640 1 -- 172.21.15.113:0/4190763368 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721904680 0 0) 0x7fb6540014a0 con 0x7fb65c06fd80 2024-03-04T08:26:41.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb652000640 1 --2- 172.21.15.113:0/4190763368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c0ca7c0 0x7fb65c15b600 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb6440098a0 tx=0x7fb64401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48158442d8e79342 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:41.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb65a000640 1 -- 172.21.15.113:0/4190763368 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440b0070 con 0x7fb65c0ca7c0 2024-03-04T08:26:41.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb65a000640 1 -- 172.21.15.113:0/4190763368 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6440ab600 con 0x7fb65c0ca7c0 2024-03-04T08:26:41.008 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb65a000640 1 -- 172.21.15.113:0/4190763368 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440ab9d0 con 0x7fb65c0ca7c0 2024-03-04T08:26:41.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb661c00640 1 -- 172.21.15.113:0/4190763368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c0ca7c0 msgr2=0x7fb65c15b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:41.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb661c00640 1 --2- 172.21.15.113:0/4190763368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c0ca7c0 0x7fb65c15b600 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb6440098a0 tx=0x7fb64401c4b0 comp rx=0 tx=0).stop 2024-03-04T08:26:41.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb661c00640 1 -- 172.21.15.113:0/4190763368 shutdown_connections 2024-03-04T08:26:41.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb661c00640 1 --2- 172.21.15.113:0/4190763368 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c0ca7c0 0x7fb65c15b600 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:41.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb661c00640 1 -- 172.21.15.113:0/4190763368 >> 172.21.15.113:0/4190763368 conn(0x7fb65c06e240 msgr2=0x7fb65c06e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:41.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb661c00640 1 -- 172.21.15.113:0/4190763368 shutdown_connections 2024-03-04T08:26:41.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.012+0000 7fb661c00640 1 -- 172.21.15.113:0/4190763368 wait complete. 2024-03-04T08:26:41.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 Processor -- start 2024-03-04T08:26:41.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 -- start start 2024-03-04T08:26:41.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c06f9a0 0x7fb65c150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:41.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65c06fd80 0x7fb65c0c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:41.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0ca7c0 0x7fb65c14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:41.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c168d30 con 0x7fb65c06f9a0 2024-03-04T08:26:41.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c06b2b0 con 0x7fb65c0ca7c0 2024-03-04T08:26:41.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c168eb0 con 0x7fb65c06fd80 2024-03-04T08:26:41.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c06f9a0 0x7fb65c150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:41.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c06f9a0 0x7fb65c150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42766/0 (socket says 172.21.15.113:42766) 2024-03-04T08:26:41.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65aa00640 1 -- 172.21.15.113:0/1929216231 learned_addr learned my addr 172.21.15.113:0/1929216231 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:41.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb652000640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65c06fd80 0x7fb65c0c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65b400640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0ca7c0 0x7fb65c14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65aa00640 1 -- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65c06fd80 msgr2=0x7fb65c0c22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65aa00640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65c06fd80 0x7fb65c0c22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65aa00640 1 -- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0ca7c0 msgr2=0x7fb65c14e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65aa00640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0ca7c0 0x7fb65c14e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65aa00640 1 -- 172.21.15.113:0/1929216231 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c150b60 con 0x7fb65c06f9a0 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65b400640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0ca7c0 0x7fb65c14e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb65aa00640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c06f9a0 0x7fb65c150090 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00c530 tx=0x7fb64c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb653400640 1 -- 172.21.15.113:0/1929216231 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c016020 con 0x7fb65c06f9a0 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb653400640 1 -- 172.21.15.113:0/1929216231 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64c005220 con 0x7fb65c06f9a0 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb653400640 1 -- 172.21.15.113:0/1929216231 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64c007030 con 0x7fb65c06f9a0 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 -- 172.21.15.113:0/1929216231 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c06cda0 con 0x7fb65c06f9a0 2024-03-04T08:26:41.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 -- 172.21.15.113:0/1929216231 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c14eff0 con 0x7fb65c06f9a0 2024-03-04T08:26:41.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.016+0000 7fb661c00640 1 -- 172.21.15.113:0/1929216231 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65c075c00 con 0x7fb65c06f9a0 2024-03-04T08:26:41.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.020+0000 7fb653400640 1 -- 172.21.15.113:0/1929216231 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb64c0071d0 con 0x7fb65c06f9a0 2024-03-04T08:26:41.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.020+0000 7fb653400640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 0x7fb638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:41.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.020+0000 7fb653400640 1 -- 172.21.15.113:0/1929216231 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fb64c085430 con 0x7fb65c06f9a0 2024-03-04T08:26:41.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.020+0000 7fb652000640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 0x7fb638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:41.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.020+0000 7fb652000640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 0x7fb638069420 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb6440ad000 tx=0x7fb644008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:41.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.024+0000 7fb653400640 1 -- 172.21.15.113:0/1929216231 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb64c020020 con 0x7fb65c06f9a0 2024-03-04T08:26:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 -- 172.21.15.113:0/1929216231 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb65c150750 con 0x7fb65c06f9a0 2024-03-04T08:26:41.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb653400640 1 -- 172.21.15.113:0/1929216231 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fb64c0074c0 con 0x7fb65c06f9a0 2024-03-04T08:26:41.209 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:41.209 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:41.209 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:41.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 -- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 msgr2=0x7fb638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:41.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 0x7fb638069420 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb6440ad000 tx=0x7fb644008040 comp rx=0 tx=0).stop 2024-03-04T08:26:41.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 -- 172.21.15.113:0/1929216231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c06f9a0 msgr2=0x7fb65c150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:41.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c06f9a0 0x7fb65c150090 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00c530 tx=0x7fb64c00c9f0 comp rx=0 tx=0).stop 2024-03-04T08:26:41.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 -- 172.21.15.113:0/1929216231 shutdown_connections 2024-03-04T08:26:41.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0ca7c0 0x7fb65c14e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:41.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65c06fd80 0x7fb65c0c22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:41.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 0x7fb638069420 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:41.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 --2- 172.21.15.113:0/1929216231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c06f9a0 0x7fb65c150090 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:41.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 -- 172.21.15.113:0/1929216231 >> 172.21.15.113:0/1929216231 conn(0x7fb65c06e240 msgr2=0x7fb65c06e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:41.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 -- 172.21.15.113:0/1929216231 shutdown_connections 2024-03-04T08:26:41.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:41.212+0000 7fb661c00640 1 -- 172.21.15.113:0/1929216231 wait complete. 2024-03-04T08:26:41.230 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:43.231 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:43.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.380+0000 7fc54b600640 1 Processor -- start 2024-03-04T08:26:43.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.380+0000 7fc54b600640 1 -- start start 2024-03-04T08:26:43.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.380+0000 7fc54b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 0x7fc544070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:43.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.380+0000 7fc54b600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5441591f0 con 0x7fc5440c8200 2024-03-04T08:26:43.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.380+0000 7fc54b600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5441594d0 con 0x7fc544152bd0 2024-03-04T08:26:43.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.380+0000 7fc54b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc544071df0 con 0x7fc5441527f0 2024-03-04T08:26:43.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc543e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc5440c8200 0x7fc544072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40406/0 (socket says 172.21.15.113:40406) 2024-03-04T08:26:43.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc543e00640 1 -- 172.21.15.113:0/2925404131 learned_addr learned my addr 172.21.15.113:0/2925404131 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:43.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc548c00640 1 --2- 172.21.15.113:0/2925404131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 0x7fc544070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:43.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc543400640 1 -- 172.21.15.113:0/2925404131 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530597718 0 0) 0x7fc5441591f0 con 0x7fc5440c8200 2024-03-04T08:26:43.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc543400640 1 -- 172.21.15.113:0/2925404131 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc528003530 con 0x7fc5440c8200 2024-03-04T08:26:43.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc543400640 1 -- 172.21.15.113:0/2925404131 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310639216 0 0) 0x7fc5441594d0 con 0x7fc544152bd0 2024-03-04T08:26:43.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc543400640 1 -- 172.21.15.113:0/2925404131 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5441591f0 con 0x7fc544152bd0 2024-03-04T08:26:43.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc548c00640 1 -- 172.21.15.113:0/2925404131 >> v1:172.21.15.136:6789/0 conn(0x7fc544152bd0 legacy=0x7fc5441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:43.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc548c00640 1 -- 172.21.15.113:0/2925404131 >> v1:172.21.15.113:6789/0 conn(0x7fc5440c8200 legacy=0x7fc544072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:43.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc548c00640 1 -- 172.21.15.113:0/2925404131 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440717a0 con 0x7fc5441527f0 2024-03-04T08:26:43.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.384+0000 7fc548c00640 1 --2- 172.21.15.113:0/2925404131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 0x7fc544070d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc5340097f0 tx=0x7fc53401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e510ec83dfef659c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:43.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc543400640 1 -- 172.21.15.113:0/2925404131 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340ab5f0 con 0x7fc5441527f0 2024-03-04T08:26:43.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc543400640 1 -- 172.21.15.113:0/2925404131 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5340b0070 con 0x7fc5441527f0 2024-03-04T08:26:43.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc543400640 1 -- 172.21.15.113:0/2925404131 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340ab990 con 0x7fc5441527f0 2024-03-04T08:26:43.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 -- 172.21.15.113:0/2925404131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 msgr2=0x7fc544070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:43.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 --2- 172.21.15.113:0/2925404131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 0x7fc544070d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc5340097f0 tx=0x7fc53401c4e0 comp rx=0 tx=0).stop 2024-03-04T08:26:43.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 -- 172.21.15.113:0/2925404131 shutdown_connections 2024-03-04T08:26:43.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 --2- 172.21.15.113:0/2925404131 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 0x7fc544070d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:43.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 -- 172.21.15.113:0/2925404131 >> 172.21.15.113:0/2925404131 conn(0x7fc54406e820 msgr2=0x7fc54406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:43.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 -- 172.21.15.113:0/2925404131 shutdown_connections 2024-03-04T08:26:43.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 -- 172.21.15.113:0/2925404131 wait complete. 2024-03-04T08:26:43.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 Processor -- start 2024-03-04T08:26:43.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 -- start start 2024-03-04T08:26:43.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5440c8200 0x7fc54414f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:43.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 0x7fc5441554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc544152bd0 0x7fc5440c1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc544166380 con 0x7fc5440c8200 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5441597b0 con 0x7fc544152bd0 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc54b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc544166500 con 0x7fc5441527f0 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc548c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5440c8200 0x7fc54414f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc548c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5440c8200 0x7fc54414f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42770/0 (socket says 172.21.15.113:42770) 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc548c00640 1 -- 172.21.15.113:0/1429577579 learned_addr learned my addr 172.21.15.113:0/1429577579 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc543e00640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 0x7fc5441554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc549600640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc544152bd0 0x7fc5440c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc548c00640 1 -- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 msgr2=0x7fc5441554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.388+0000 7fc548c00640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 0x7fc5441554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc548c00640 1 -- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc544152bd0 msgr2=0x7fc5440c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc548c00640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc544152bd0 0x7fc5440c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:43.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc548c00640 1 -- 172.21.15.113:0/1429577579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54414aa30 con 0x7fc5440c8200 2024-03-04T08:26:43.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc548c00640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5440c8200 0x7fc54414f8e0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fc5340097f0 tx=0x7fc53401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:43.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc540c00640 1 -- 172.21.15.113:0/1429577579 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340bf410 con 0x7fc5440c8200 2024-03-04T08:26:43.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc540c00640 1 -- 172.21.15.113:0/1429577579 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5340bf5b0 con 0x7fc5440c8200 2024-03-04T08:26:43.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc540c00640 1 -- 172.21.15.113:0/1429577579 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5340bf960 con 0x7fc5440c8200 2024-03-04T08:26:43.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc54b600640 1 -- 172.21.15.113:0/1429577579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5440c27e0 con 0x7fc5440c8200 2024-03-04T08:26:43.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc54b600640 1 -- 172.21.15.113:0/1429577579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5441f4240 con 0x7fc5440c8200 2024-03-04T08:26:43.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc54b600640 1 -- 172.21.15.113:0/1429577579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc544075c00 con 0x7fc5440c8200 2024-03-04T08:26:43.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc540c00640 1 -- 172.21.15.113:0/1429577579 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5340bd020 con 0x7fc5440c8200 2024-03-04T08:26:43.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc540c00640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc524066f60 0x7fc524069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:43.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc540c00640 1 -- 172.21.15.113:0/1429577579 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fc53411fd20 con 0x7fc5440c8200 2024-03-04T08:26:43.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.392+0000 7fc543e00640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc524066f60 0x7fc524069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:43.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.396+0000 7fc543e00640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc524066f60 0x7fc524069420 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc544164130 tx=0x7fc530009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:43.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.396+0000 7fc540c00640 1 -- 172.21.15.113:0/1429577579 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc5340160d0 con 0x7fc5440c8200 2024-03-04T08:26:43.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 -- 172.21.15.113:0/1429577579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5440c79d0 con 0x7fc5440c8200 2024-03-04T08:26:43.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc540c00640 1 -- 172.21.15.113:0/1429577579 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fc5340f03a0 con 0x7fc5440c8200 2024-03-04T08:26:43.580 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:43.580 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:43.580 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:43.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 -- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc524066f60 msgr2=0x7fc524069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:43.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc524066f60 0x7fc524069420 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc544164130 tx=0x7fc530009040 comp rx=0 tx=0).stop 2024-03-04T08:26:43.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 -- 172.21.15.113:0/1429577579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5440c8200 msgr2=0x7fc54414f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:43.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5440c8200 0x7fc54414f8e0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fc5340097f0 tx=0x7fc53401ad20 comp rx=0 tx=0).stop 2024-03-04T08:26:43.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 -- 172.21.15.113:0/1429577579 shutdown_connections 2024-03-04T08:26:43.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc544152bd0 0x7fc5440c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:43.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc5441527f0 0x7fc5441554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:43.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc524066f60 0x7fc524069420 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:43.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 --2- 172.21.15.113:0/1429577579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5440c8200 0x7fc54414f8e0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:43.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 -- 172.21.15.113:0/1429577579 >> 172.21.15.113:0/1429577579 conn(0x7fc54406e820 msgr2=0x7fc54406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:43.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 -- 172.21.15.113:0/1429577579 shutdown_connections 2024-03-04T08:26:43.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:43.584+0000 7fc54b600640 1 -- 172.21.15.113:0/1429577579 wait complete. 2024-03-04T08:26:43.601 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:45.602 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:45.708 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:26:45.712 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:26:45.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a7e00640 1 Processor -- start 2024-03-04T08:26:45.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a7e00640 1 -- start start 2024-03-04T08:26:45.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd6a8152900 0x7fd6a8071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:45.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd6a806c480 0x7fd6a80c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:45.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a8152ce0 con 0x7fd6a80c7930 2024-03-04T08:26:45.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a806b060 con 0x7fd6a8152900 2024-03-04T08:26:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a806b300 con 0x7fd6a806c480 2024-03-04T08:26:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a6a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd6a8152900 0x7fd6a8071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a6000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd6a806c480 0x7fd6a80c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a7400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd6a80c7930 0x7fd6a80c9290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40412/0 (socket says 172.21.15.113:40412) 2024-03-04T08:26:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a7400640 1 -- 172.21.15.113:0/2571735438 learned_addr learned my addr 172.21.15.113:0/2571735438 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:45.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a6a00640 1 -- 172.21.15.113:0/2571735438 >> v2:172.21.15.136:3300/0 conn(0x7fd6a806c480 msgr2=0x7fd6a80c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a6a00640 1 --2- 172.21.15.113:0/2571735438 >> v2:172.21.15.136:3300/0 conn(0x7fd6a806c480 0x7fd6a80c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a6a00640 1 -- 172.21.15.113:0/2571735438 >> v1:172.21.15.113:6789/0 conn(0x7fd6a80c7930 legacy=0x7fd6a80c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a6a00640 1 -- 172.21.15.113:0/2571735438 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a806b5e0 con 0x7fd6a8152900 2024-03-04T08:26:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.756+0000 7fd6a5600640 1 -- 172.21.15.113:0/2571735438 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581353852 0 0) 0x7fd6a00014a0 con 0x7fd6a80c7930 2024-03-04T08:26:45.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a6a00640 1 --2- 172.21.15.113:0/2571735438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8152900 0x7fd6a8071a30 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80715c0 tx=0x7fd69801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101bfdbdfaefe519 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:45.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a5600640 1 -- 172.21.15.113:0/2571735438 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6980b1070 con 0x7fd6a8152900 2024-03-04T08:26:45.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a5600640 1 -- 172.21.15.113:0/2571735438 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6980ac5f0 con 0x7fd6a8152900 2024-03-04T08:26:45.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a5600640 1 -- 172.21.15.113:0/2571735438 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6980ac9c0 con 0x7fd6a8152900 2024-03-04T08:26:45.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2571735438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8152900 msgr2=0x7fd6a8071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:45.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a7e00640 1 --2- 172.21.15.113:0/2571735438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8152900 0x7fd6a8071a30 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80715c0 tx=0x7fd69801c480 comp rx=0 tx=0).stop 2024-03-04T08:26:45.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2571735438 shutdown_connections 2024-03-04T08:26:45.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a7e00640 1 --2- 172.21.15.113:0/2571735438 >> v2:172.21.15.136:3300/0 conn(0x7fd6a806c480 0x7fd6a80c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:45.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a7e00640 1 --2- 172.21.15.113:0/2571735438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a8152900 0x7fd6a8071a30 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:45.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2571735438 >> 172.21.15.113:0/2571735438 conn(0x7fd6a80c9a30 msgr2=0x7fd6a80c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:45.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2571735438 shutdown_connections 2024-03-04T08:26:45.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2571735438 wait complete. 2024-03-04T08:26:45.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a7e00640 1 Processor -- start 2024-03-04T08:26:45.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.760+0000 7fd6a7e00640 1 -- start start 2024-03-04T08:26:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6a806c480 0x7fd6a80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a80c7930 0x7fd6a81f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6a8152900 0x7fd6a8149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a8071f70 con 0x7fd6a80c7930 2024-03-04T08:26:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a8158800 con 0x7fd6a8152900 2024-03-04T08:26:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a806d450 con 0x7fd6a806c480 2024-03-04T08:26:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a80c7930 0x7fd6a81f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a80c7930 0x7fd6a81f7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42798/0 (socket says 172.21.15.113:42798) 2024-03-04T08:26:45.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a6000640 1 -- 172.21.15.113:0/2594060633 learned_addr learned my addr 172.21.15.113:0/2594060633 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a7400640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6a8152900 0x7fd6a8149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a6a00640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6a806c480 0x7fd6a80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a6000640 1 -- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6a806c480 msgr2=0x7fd6a80c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a6000640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6a806c480 0x7fd6a80c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a6000640 1 -- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6a8152900 msgr2=0x7fd6a8149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a6000640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6a8152900 0x7fd6a8149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a6000640 1 -- 172.21.15.113:0/2594060633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a81f73c0 con 0x7fd6a80c7930 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a6000640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a80c7930 0x7fd6a81f7e30 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fd69400e8a0 tx=0x7fd69400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd69ea00640 1 -- 172.21.15.113:0/2594060633 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd694017070 con 0x7fd6a80c7930 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd69ea00640 1 -- 172.21.15.113:0/2594060633 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69400c940 con 0x7fd6a80c7930 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd69ea00640 1 -- 172.21.15.113:0/2594060633 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69400ccf0 con 0x7fd6a80c7930 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2594060633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a80cc970 con 0x7fd6a80c7930 2024-03-04T08:26:45.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2594060633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a81f7910 con 0x7fd6a80c7930 2024-03-04T08:26:45.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.764+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2594060633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a8075a80 con 0x7fd6a80c7930 2024-03-04T08:26:45.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.768+0000 7fd69ea00640 1 -- 172.21.15.113:0/2594060633 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd69401a070 con 0x7fd6a80c7930 2024-03-04T08:26:45.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.768+0000 7fd69ea00640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd688066f60 0x7fd688069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:45.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.768+0000 7fd69ea00640 1 -- 172.21.15.113:0/2594060633 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fd694085d50 con 0x7fd6a80c7930 2024-03-04T08:26:45.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.768+0000 7fd6a6a00640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd688066f60 0x7fd688069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:45.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.768+0000 7fd6a6a00640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd688066f60 0x7fd688069420 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd6980ae000 tx=0x7fd6980a7000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:45.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.772+0000 7fd69ea00640 1 -- 172.21.15.113:0/2594060633 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd694089030 con 0x7fd6a80c7930 2024-03-04T08:26:45.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.956+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2594060633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6a8161a60 con 0x7fd6a80c7930 2024-03-04T08:26:45.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd69ea00640 1 -- 172.21.15.113:0/2594060633 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fd694055ba0 con 0x7fd6a80c7930 2024-03-04T08:26:45.955 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:45.955 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:45.955 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:45.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd688066f60 msgr2=0x7fd688069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:45.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd688066f60 0x7fd688069420 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd6980ae000 tx=0x7fd6980a7000 comp rx=0 tx=0).stop 2024-03-04T08:26:45.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2594060633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a80c7930 msgr2=0x7fd6a81f7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:45.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a80c7930 0x7fd6a81f7e30 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fd69400e8a0 tx=0x7fd69400eca0 comp rx=0 tx=0).stop 2024-03-04T08:26:45.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2594060633 shutdown_connections 2024-03-04T08:26:45.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6a8152900 0x7fd6a8149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:45.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd688066f60 0x7fd688069420 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:45.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a80c7930 0x7fd6a81f7e30 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:45.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 --2- 172.21.15.113:0/2594060633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6a806c480 0x7fd6a80c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:45.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2594060633 >> 172.21.15.113:0/2594060633 conn(0x7fd6a80c9a30 msgr2=0x7fd6a806b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:45.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2594060633 shutdown_connections 2024-03-04T08:26:45.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:45.960+0000 7fd6a7e00640 1 -- 172.21.15.113:0/2594060633 wait complete. 2024-03-04T08:26:45.975 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:47.976 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:48.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.124+0000 7f90c2a00640 1 Processor -- start 2024-03-04T08:26:48.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c2a00640 1 -- start start 2024-03-04T08:26:48.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f90bc157ef0 0x7f90bc0c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:48.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06fb70 0x7f90bc0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:48.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c2a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90bc0c9c50 con 0x7f90bc070df0 2024-03-04T08:26:48.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c2a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90bc15a9a0 con 0x7f90bc157ef0 2024-03-04T08:26:48.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c2a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc06de30 con 0x7f90bc06fb70 2024-03-04T08:26:48.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06fb70 0x7f90bc0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:48.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c1600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f90bc157ef0 0x7f90bc0c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:48.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06fb70 0x7f90bc0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:55078/0 (socket says 172.21.15.113:55078) 2024-03-04T08:26:48.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c2000640 1 -- 172.21.15.113:0/615663471 learned_addr learned my addr 172.21.15.113:0/615663471 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c1600640 1 -- 172.21.15.113:0/615663471 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06fb70 msgr2=0x7f90bc0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c1600640 1 --2- 172.21.15.113:0/615663471 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06fb70 0x7f90bc0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c1600640 1 -- 172.21.15.113:0/615663471 >> v1:172.21.15.136:6789/0 conn(0x7f90bc070df0 legacy=0x7f90bc15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c1600640 1 -- 172.21.15.113:0/615663471 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc06c210 con 0x7f90bc157ef0 2024-03-04T08:26:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.128+0000 7f90c2000640 1 --2- 172.21.15.113:0/615663471 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06fb70 0x7f90bc0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:26:48.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90c1600640 1 --2- 172.21.15.113:0/615663471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc157ef0 0x7f90bc0c8a90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f90b00097c0 tx=0x7f90b001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f825f8b929506ea server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:48.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90bbe00640 1 -- 172.21.15.113:0/615663471 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00b1070 con 0x7f90bc157ef0 2024-03-04T08:26:48.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90bbe00640 1 -- 172.21.15.113:0/615663471 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b00ac410 con 0x7f90bc157ef0 2024-03-04T08:26:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90bbe00640 1 -- 172.21.15.113:0/615663471 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00ac7e0 con 0x7f90bc157ef0 2024-03-04T08:26:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90c2a00640 1 -- 172.21.15.113:0/615663471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc157ef0 msgr2=0x7f90bc0c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90c2a00640 1 --2- 172.21.15.113:0/615663471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc157ef0 0x7f90bc0c8a90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f90b00097c0 tx=0x7f90b001c4a0 comp rx=0 tx=0).stop 2024-03-04T08:26:48.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90c2a00640 1 -- 172.21.15.113:0/615663471 shutdown_connections 2024-03-04T08:26:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90c2a00640 1 --2- 172.21.15.113:0/615663471 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc06fb70 0x7f90bc0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90c2a00640 1 --2- 172.21.15.113:0/615663471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc157ef0 0x7f90bc0c8a90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90c2a00640 1 -- 172.21.15.113:0/615663471 >> 172.21.15.113:0/615663471 conn(0x7f90bc072f40 msgr2=0x7f90bc0706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90c2a00640 1 -- 172.21.15.113:0/615663471 shutdown_connections 2024-03-04T08:26:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.132+0000 7f90c2a00640 1 -- 172.21.15.113:0/615663471 wait complete. 2024-03-04T08:26:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 Processor -- start 2024-03-04T08:26:48.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 -- start start 2024-03-04T08:26:48.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06fb70 0x7f90bc146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:48.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90bc070df0 0x7f90bc146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:48.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc157ef0 0x7f90bc1f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:48.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc1582d0 con 0x7f90bc070df0 2024-03-04T08:26:48.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc15b890 con 0x7f90bc06fb70 2024-03-04T08:26:48.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc158450 con 0x7f90bc157ef0 2024-03-04T08:26:48.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90bc070df0 0x7f90bc146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c0c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90bc070df0 0x7f90bc146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50392/0 (socket says 172.21.15.113:50392) 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c0c00640 1 -- 172.21.15.113:0/3319576592 learned_addr learned my addr 172.21.15.113:0/3319576592 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c1600640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06fb70 0x7f90bc146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2000640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc157ef0 0x7f90bc1f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c0c00640 1 -- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc157ef0 msgr2=0x7f90bc1f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c0c00640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc157ef0 0x7f90bc1f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c0c00640 1 -- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06fb70 msgr2=0x7f90bc146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c0c00640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06fb70 0x7f90bc146630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c0c00640 1 -- 172.21.15.113:0/3319576592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc148c30 con 0x7f90bc070df0 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c1600640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06fb70 0x7f90bc146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c0c00640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90bc070df0 0x7f90bc146b70 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f90a800c450 tx=0x7f90a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90b9600640 1 -- 172.21.15.113:0/3319576592 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a8016020 con 0x7f90bc070df0 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90b9600640 1 -- 172.21.15.113:0/3319576592 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a80052d0 con 0x7f90bc070df0 2024-03-04T08:26:48.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90b9600640 1 -- 172.21.15.113:0/3319576592 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a8007260 con 0x7f90bc070df0 2024-03-04T08:26:48.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 -- 172.21.15.113:0/3319576592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc148f10 con 0x7f90bc070df0 2024-03-04T08:26:48.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 -- 172.21.15.113:0/3319576592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc1494d0 con 0x7f90bc070df0 2024-03-04T08:26:48.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.136+0000 7f90c2a00640 1 -- 172.21.15.113:0/3319576592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc075a80 con 0x7f90bc070df0 2024-03-04T08:26:48.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.140+0000 7f90b9600640 1 -- 172.21.15.113:0/3319576592 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90a8007400 con 0x7f90bc070df0 2024-03-04T08:26:48.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.140+0000 7f90b9600640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f909c066f90 0x7f909c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:48.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.140+0000 7f90b9600640 1 -- 172.21.15.113:0/3319576592 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f90a80854f0 con 0x7f90bc070df0 2024-03-04T08:26:48.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.140+0000 7f90c1600640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f909c066f90 0x7f909c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:48.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.140+0000 7f90c1600640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f909c066f90 0x7f909c069450 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f90b00ae000 tx=0x7f90b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:48.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.144+0000 7f90b9600640 1 -- 172.21.15.113:0/3319576592 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f90a8020020 con 0x7f90bc070df0 2024-03-04T08:26:48.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 -- 172.21.15.113:0/3319576592 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90bc0c8560 con 0x7f90bc070df0 2024-03-04T08:26:48.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90b9600640 1 -- 172.21.15.113:0/3319576592 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f90a8005480 con 0x7f90bc070df0 2024-03-04T08:26:48.328 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:48.329 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:48.329 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:48.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 -- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f909c066f90 msgr2=0x7f909c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:48.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f909c066f90 0x7f909c069450 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f90b00ae000 tx=0x7f90b0008040 comp rx=0 tx=0).stop 2024-03-04T08:26:48.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 -- 172.21.15.113:0/3319576592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90bc070df0 msgr2=0x7f90bc146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:48.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90bc070df0 0x7f90bc146b70 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f90a800c450 tx=0x7f90a800c910 comp rx=0 tx=0).stop 2024-03-04T08:26:48.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 -- 172.21.15.113:0/3319576592 shutdown_connections 2024-03-04T08:26:48.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f90bc157ef0 0x7f90bc1f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:48.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f909c066f90 0x7f909c069450 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:48.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90bc070df0 0x7f90bc146b70 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:48.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 --2- 172.21.15.113:0/3319576592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc06fb70 0x7f90bc146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:48.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 -- 172.21.15.113:0/3319576592 >> 172.21.15.113:0/3319576592 conn(0x7f90bc072f40 msgr2=0x7f90bc0ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:48.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 -- 172.21.15.113:0/3319576592 shutdown_connections 2024-03-04T08:26:48.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:48.332+0000 7f90c2a00640 1 -- 172.21.15.113:0/3319576592 wait complete. 2024-03-04T08:26:48.352 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:50.353 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:50.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd39ca00640 1 Processor -- start 2024-03-04T08:26:50.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd39ca00640 1 -- start start 2024-03-04T08:26:50.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd39ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39806ff20 0x7fd39815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:50.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd39ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3980c8b90 0x7fd39815c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:50.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd39ca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd398152e30 con 0x7fd398152a50 2024-03-04T08:26:50.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd39ca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3980c31b0 con 0x7fd3980c8b90 2024-03-04T08:26:50.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd39ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39806f840 con 0x7fd39806ff20 2024-03-04T08:26:50.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd396800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3980c8b90 0x7fd39815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:50.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd396800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3980c8b90 0x7fd39815c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50394/0 (socket says 172.21.15.113:50394) 2024-03-04T08:26:50.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd396800640 1 -- 172.21.15.113:0/3792900465 learned_addr learned my addr 172.21.15.113:0/3792900465 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:50.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd38d400640 1 --2- 172.21.15.113:0/3792900465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39806ff20 0x7fd39815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:50.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd396800640 1 -- 172.21.15.113:0/3792900465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39806ff20 msgr2=0x7fd39815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:50.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd396800640 1 --2- 172.21.15.113:0/3792900465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39806ff20 0x7fd39815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd396800640 1 -- 172.21.15.113:0/3792900465 >> v1:172.21.15.136:6789/0 conn(0x7fd398152a50 legacy=0x7fd3980ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:50.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.504+0000 7fd396800640 1 -- 172.21.15.113:0/3792900465 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39806f280 con 0x7fd3980c8b90 2024-03-04T08:26:50.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd396800640 1 --2- 172.21.15.113:0/3792900465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 0x7fd39815c310 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd390009870 tx=0x7fd39001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76dc53b71aba19 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:50.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd395400640 1 -- 172.21.15.113:0/3792900465 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3900aa650 con 0x7fd3980c8b90 2024-03-04T08:26:50.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd395400640 1 -- 172.21.15.113:0/3792900465 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3900af070 con 0x7fd3980c8b90 2024-03-04T08:26:50.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd395400640 1 -- 172.21.15.113:0/3792900465 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3900aaa10 con 0x7fd3980c8b90 2024-03-04T08:26:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd39ca00640 1 -- 172.21.15.113:0/3792900465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 msgr2=0x7fd39815c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd39ca00640 1 --2- 172.21.15.113:0/3792900465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 0x7fd39815c310 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd390009870 tx=0x7fd39001c4d0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd39ca00640 1 -- 172.21.15.113:0/3792900465 shutdown_connections 2024-03-04T08:26:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd39ca00640 1 --2- 172.21.15.113:0/3792900465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 0x7fd39815c310 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd39ca00640 1 --2- 172.21.15.113:0/3792900465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39806ff20 0x7fd39815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd39ca00640 1 -- 172.21.15.113:0/3792900465 >> 172.21.15.113:0/3792900465 conn(0x7fd3980734d0 msgr2=0x7fd398066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd39ca00640 1 -- 172.21.15.113:0/3792900465 shutdown_connections 2024-03-04T08:26:50.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.508+0000 7fd39ca00640 1 -- 172.21.15.113:0/3792900465 wait complete. 2024-03-04T08:26:50.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 Processor -- start 2024-03-04T08:26:50.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 -- start start 2024-03-04T08:26:50.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39806ff20 0x7fd3981f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 0x7fd3981557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398152a50 0x7fd398155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980ce9b0 con 0x7fd3980c8b90 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980ce830 con 0x7fd398152a50 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd398165880 con 0x7fd39806ff20 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 0x7fd3981557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 0x7fd3981557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50402/0 (socket says 172.21.15.113:50402) 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38d400640 1 -- 172.21.15.113:0/3909131473 learned_addr learned my addr 172.21.15.113:0/3909131473 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd395e00640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39806ff20 0x7fd3981f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd396800640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398152a50 0x7fd398155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:50.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38d400640 1 -- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39806ff20 msgr2=0x7fd3981f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38d400640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39806ff20 0x7fd3981f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38d400640 1 -- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398152a50 msgr2=0x7fd398155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38d400640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398152a50 0x7fd398155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38d400640 1 -- 172.21.15.113:0/3909131473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3981531f0 con 0x7fd3980c8b90 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd396800640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398152a50 0x7fd398155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38d400640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 0x7fd3981557f0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd38000d620 tx=0x7fd38000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38ea00640 1 -- 172.21.15.113:0/3909131473 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800125f0 con 0x7fd3980c8b90 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38ea00640 1 -- 172.21.15.113:0/3909131473 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380019070 con 0x7fd3980c8b90 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd38ea00640 1 -- 172.21.15.113:0/3909131473 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800129b0 con 0x7fd3980c8b90 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 -- 172.21.15.113:0/3909131473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3981534d0 con 0x7fd3980c8b90 2024-03-04T08:26:50.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 -- 172.21.15.113:0/3909131473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd398153a90 con 0x7fd3980c8b90 2024-03-04T08:26:50.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.512+0000 7fd39ca00640 1 -- 172.21.15.113:0/3909131473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd398075c00 con 0x7fd3980c8b90 2024-03-04T08:26:50.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.516+0000 7fd38ea00640 1 -- 172.21.15.113:0/3909131473 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd380012b50 con 0x7fd3980c8b90 2024-03-04T08:26:50.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.516+0000 7fd38ea00640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd374066f60 0x7fd374069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:50.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.516+0000 7fd38ea00640 1 -- 172.21.15.113:0/3909131473 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fd380085e30 con 0x7fd3980c8b90 2024-03-04T08:26:50.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.516+0000 7fd395e00640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd374066f60 0x7fd374069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:50.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.516+0000 7fd395e00640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd374066f60 0x7fd374069420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd388001ea0 tx=0x7fd388009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:50.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.520+0000 7fd38ea00640 1 -- 172.21.15.113:0/3909131473 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd380056390 con 0x7fd3980c8b90 2024-03-04T08:26:50.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.708+0000 7fd39ca00640 1 -- 172.21.15.113:0/3909131473 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd39806eaa0 con 0x7fd3980c8b90 2024-03-04T08:26:50.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd38ea00640 1 -- 172.21.15.113:0/3909131473 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fd380055c90 con 0x7fd3980c8b90 2024-03-04T08:26:50.708 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:50.708 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:50.709 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 -- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd374066f60 msgr2=0x7fd374069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd374066f60 0x7fd374069420 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fd388001ea0 tx=0x7fd388009040 comp rx=0 tx=0).stop 2024-03-04T08:26:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 -- 172.21.15.113:0/3909131473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 msgr2=0x7fd3981557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 0x7fd3981557f0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd38000d620 tx=0x7fd38000dae0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 -- 172.21.15.113:0/3909131473 shutdown_connections 2024-03-04T08:26:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd398152a50 0x7fd398155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd374066f60 0x7fd374069420 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3980c8b90 0x7fd3981557f0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 --2- 172.21.15.113:0/3909131473 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd39806ff20 0x7fd3981f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:50.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 -- 172.21.15.113:0/3909131473 >> 172.21.15.113:0/3909131473 conn(0x7fd3980734d0 msgr2=0x7fd3980cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:50.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 -- 172.21.15.113:0/3909131473 shutdown_connections 2024-03-04T08:26:50.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:50.712+0000 7fd39ca00640 1 -- 172.21.15.113:0/3909131473 wait complete. 2024-03-04T08:26:50.729 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:26:50.730 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:26:52.730 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:52.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f0801a00640 1 Processor -- start 2024-03-04T08:26:52.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f0801a00640 1 -- start start 2024-03-04T08:26:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f0801a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f07fc0b20b0 0x7f07fc157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:52.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f0801a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07fc0c1ec0 con 0x7f07fc157c50 2024-03-04T08:26:52.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f0801a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07fc0c9b30 con 0x7f07fc0c85f0 2024-03-04T08:26:52.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f0801a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07fc0c12a0 con 0x7f07fc0b20b0 2024-03-04T08:26:52.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f07f2000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f07fc157c50 0x7f07fc06b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48530/0 (socket says 172.21.15.113:48530) 2024-03-04T08:26:52.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f07f2000640 1 -- 172.21.15.113:0/3338552401 learned_addr learned my addr 172.21.15.113:0/3338552401 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:52.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f07faa00640 1 --2- 172.21.15.113:0/3338552401 >> v2:172.21.15.136:3300/0 conn(0x7f07fc0b20b0 0x7f07fc157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:52.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f07fa000640 1 -- 172.21.15.113:0/3338552401 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895687035 0 0) 0x7f07fc0c1ec0 con 0x7f07fc157c50 2024-03-04T08:26:52.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f07fa000640 1 -- 172.21.15.113:0/3338552401 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e0003450 con 0x7f07fc157c50 2024-03-04T08:26:52.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f07faa00640 1 -- 172.21.15.113:0/3338552401 >> v1:172.21.15.136:6789/0 conn(0x7f07fc0c85f0 legacy=0x7f07fc161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:52.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f07faa00640 1 -- 172.21.15.113:0/3338552401 >> v1:172.21.15.113:6789/0 conn(0x7f07fc157c50 legacy=0x7f07fc06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:52.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.880+0000 7f07faa00640 1 -- 172.21.15.113:0/3338552401 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc0c1580 con 0x7f07fc0b20b0 2024-03-04T08:26:52.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.884+0000 7f07faa00640 1 --2- 172.21.15.113:0/3338552401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07fc0b20b0 0x7f07fc157710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009d90 tx=0x7f07ec01f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c05b378e47d19f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:52.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f07fa000640 1 -- 172.21.15.113:0/3338552401 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec0b4070 con 0x7f07fc0b20b0 2024-03-04T08:26:52.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f07fa000640 1 -- 172.21.15.113:0/3338552401 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec0af600 con 0x7f07fc0b20b0 2024-03-04T08:26:52.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f07fa000640 1 -- 172.21.15.113:0/3338552401 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec0af9d0 con 0x7f07fc0b20b0 2024-03-04T08:26:52.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f0801a00640 1 -- 172.21.15.113:0/3338552401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07fc0b20b0 msgr2=0x7f07fc157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:52.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f0801a00640 1 --2- 172.21.15.113:0/3338552401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07fc0b20b0 0x7f07fc157710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f07ec009d90 tx=0x7f07ec01f150 comp rx=0 tx=0).stop 2024-03-04T08:26:52.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f0801a00640 1 -- 172.21.15.113:0/3338552401 shutdown_connections 2024-03-04T08:26:52.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f0801a00640 1 --2- 172.21.15.113:0/3338552401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07fc0b20b0 0x7f07fc157710 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:52.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f0801a00640 1 -- 172.21.15.113:0/3338552401 >> 172.21.15.113:0/3338552401 conn(0x7f07fc06e820 msgr2=0x7f07fc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:52.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f0801a00640 1 -- 172.21.15.113:0/3338552401 shutdown_connections 2024-03-04T08:26:52.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f0801a00640 1 -- 172.21.15.113:0/3338552401 wait complete. 2024-03-04T08:26:52.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f0801a00640 1 Processor -- start 2024-03-04T08:26:52.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.888+0000 7f0801a00640 1 -- start start 2024-03-04T08:26:52.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f0801a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07fc0b20b0 0x7f07fc1f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:52.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f0801a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07fc0c85f0 0x7f07fc1e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:52.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f0801a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07fc157c50 0x7f07fc06b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:52.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f0801a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc0c2400 con 0x7f07fc157c50 2024-03-04T08:26:52.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f0801a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc1571f0 con 0x7f07fc0c85f0 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f0801a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc156ec0 con 0x7f07fc0b20b0 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07fb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07fc157c50 0x7f07fc06b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07fb400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07fc157c50 0x7f07fc06b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50410/0 (socket says 172.21.15.113:50410) 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07fb400640 1 -- 172.21.15.113:0/3436372162 learned_addr learned my addr 172.21.15.113:0/3436372162 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07f2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07fc0c85f0 0x7f07fc1e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07faa00640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07fc0b20b0 0x7f07fc1f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07fb400640 1 -- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07fc0b20b0 msgr2=0x7f07fc1f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07fb400640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07fc0b20b0 0x7f07fc1f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07fb400640 1 -- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07fc0c85f0 msgr2=0x7f07fc1e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07fb400640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07fc0c85f0 0x7f07fc1e3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07fb400640 1 -- 172.21.15.113:0/3436372162 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc0aeb10 con 0x7f07fc157c50 2024-03-04T08:26:52.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07fb400640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07fc157c50 0x7f07fc06b2c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f07f400c430 tx=0x7f07f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:52.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07f3400640 1 -- 172.21.15.113:0/3436372162 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4005220 con 0x7f07fc157c50 2024-03-04T08:26:52.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07f3400640 1 -- 172.21.15.113:0/3436372162 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f40053c0 con 0x7f07fc157c50 2024-03-04T08:26:52.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f0801a00640 1 -- 172.21.15.113:0/3436372162 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc1e5980 con 0x7f07fc157c50 2024-03-04T08:26:52.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07f3400640 1 -- 172.21.15.113:0/3436372162 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f4007270 con 0x7f07fc157c50 2024-03-04T08:26:52.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f0801a00640 1 -- 172.21.15.113:0/3436372162 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc1e36b0 con 0x7f07fc157c50 2024-03-04T08:26:52.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f0801a00640 1 -- 172.21.15.113:0/3436372162 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc075c00 con 0x7f07fc157c50 2024-03-04T08:26:52.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.892+0000 7f07f3400640 1 -- 172.21.15.113:0/3436372162 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07f4019020 con 0x7f07fc157c50 2024-03-04T08:26:52.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.896+0000 7f07f3400640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f07d8066f60 0x7f07d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:52.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.896+0000 7f07f3400640 1 -- 172.21.15.113:0/3436372162 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f07f4084d90 con 0x7f07fc157c50 2024-03-04T08:26:52.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.896+0000 7f07faa00640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f07d8066f60 0x7f07d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:52.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.896+0000 7f07faa00640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f07d8066f60 0x7f07d8069420 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0b1000 tx=0x7f07ec0aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:52.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:52.900+0000 7f07f3400640 1 -- 172.21.15.113:0/3436372162 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f07f4054c10 con 0x7f07fc157c50 2024-03-04T08:26:53.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 -- 172.21.15.113:0/3436372162 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07fc1e3880 con 0x7f07fc157c50 2024-03-04T08:26:53.084 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f07f3400640 1 -- 172.21.15.113:0/3436372162 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f07f4058cb0 con 0x7f07fc157c50 2024-03-04T08:26:53.084 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:53.084 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:53.085 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:53.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 -- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f07d8066f60 msgr2=0x7f07d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:53.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f07d8066f60 0x7f07d8069420 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0b1000 tx=0x7f07ec0aa000 comp rx=0 tx=0).stop 2024-03-04T08:26:53.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 -- 172.21.15.113:0/3436372162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07fc157c50 msgr2=0x7f07fc06b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:53.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07fc157c50 0x7f07fc06b2c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f07f400c430 tx=0x7f07f400c8f0 comp rx=0 tx=0).stop 2024-03-04T08:26:53.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 -- 172.21.15.113:0/3436372162 shutdown_connections 2024-03-04T08:26:53.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f07d8066f60 0x7f07d8069420 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:53.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07fc157c50 0x7f07fc06b2c0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:53.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07fc0c85f0 0x7f07fc1e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:53.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 --2- 172.21.15.113:0/3436372162 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f07fc0b20b0 0x7f07fc1f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:53.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 -- 172.21.15.113:0/3436372162 >> 172.21.15.113:0/3436372162 conn(0x7f07fc06e820 msgr2=0x7f07fc0728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:53.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 -- 172.21.15.113:0/3436372162 shutdown_connections 2024-03-04T08:26:53.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:53.088+0000 7f0801a00640 1 -- 172.21.15.113:0/3436372162 wait complete. 2024-03-04T08:26:53.106 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:55.107 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:55.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.256+0000 7fb02a600640 1 Processor -- start 2024-03-04T08:26:55.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.256+0000 7fb02a600640 1 -- start start 2024-03-04T08:26:55.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb02a600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb0240b20b0 0x7fb024157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb02a600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb0240c85f0 0x7fb024161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb02a600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0240c1ec0 con 0x7fb024157c50 2024-03-04T08:26:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb02a600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0240c9b30 con 0x7fb0240c85f0 2024-03-04T08:26:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb02a600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0240c12a0 con 0x7fb0240b20b0 2024-03-04T08:26:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb023e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb0240c85f0 0x7fb024161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb022a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb024157c50 0x7fb02406b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48536/0 (socket says 172.21.15.113:48536) 2024-03-04T08:26:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb022a00640 1 -- 172.21.15.113:0/2420081275 learned_addr learned my addr 172.21.15.113:0/2420081275 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:26:55.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb023e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb0240c85f0 0x7fb024161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50416/0 (socket says 172.21.15.113:50416) 2024-03-04T08:26:55.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb023400640 1 --2- 172.21.15.113:0/2420081275 >> v2:172.21.15.136:3300/0 conn(0x7fb0240b20b0 0x7fb024157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:55.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb023e00640 1 -- 172.21.15.113:0/2420081275 >> v2:172.21.15.136:3300/0 conn(0x7fb0240b20b0 msgr2=0x7fb024157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:55.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb023e00640 1 --2- 172.21.15.113:0/2420081275 >> v2:172.21.15.136:3300/0 conn(0x7fb0240b20b0 0x7fb024157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:55.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb023e00640 1 -- 172.21.15.113:0/2420081275 >> v1:172.21.15.113:6789/0 conn(0x7fb024157c50 legacy=0x7fb02406b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:55.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb023e00640 1 -- 172.21.15.113:0/2420081275 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0240c1580 con 0x7fb0240c85f0 2024-03-04T08:26:55.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.260+0000 7fb023400640 1 --2- 172.21.15.113:0/2420081275 >> v2:172.21.15.136:3300/0 conn(0x7fb0240b20b0 0x7fb024157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb023e00640 1 --2- 172.21.15.113:0/2420081275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0240c85f0 0x7fb024161820 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fb024070670 tx=0x7fb01c01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c77e92716fb07f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb022000640 1 -- 172.21.15.113:0/2420081275 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0b4070 con 0x7fb0240c85f0 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb022000640 1 -- 172.21.15.113:0/2420081275 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c0af600 con 0x7fb0240c85f0 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb022000640 1 -- 172.21.15.113:0/2420081275 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0af9d0 con 0x7fb0240c85f0 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 -- 172.21.15.113:0/2420081275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0240c85f0 msgr2=0x7fb024161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 --2- 172.21.15.113:0/2420081275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0240c85f0 0x7fb024161820 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fb024070670 tx=0x7fb01c01f110 comp rx=0 tx=0).stop 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 -- 172.21.15.113:0/2420081275 shutdown_connections 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 --2- 172.21.15.113:0/2420081275 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0240c85f0 0x7fb024161820 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 --2- 172.21.15.113:0/2420081275 >> v2:172.21.15.136:3300/0 conn(0x7fb0240b20b0 0x7fb024157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 -- 172.21.15.113:0/2420081275 >> 172.21.15.113:0/2420081275 conn(0x7fb02406e820 msgr2=0x7fb02406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 -- 172.21.15.113:0/2420081275 shutdown_connections 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 -- 172.21.15.113:0/2420081275 wait complete. 2024-03-04T08:26:55.265 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 Processor -- start 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 -- start start 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0240b20b0 0x7fb0241e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0240c85f0 0x7fb0240c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb024157c50 0x7fb02414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0240c2400 con 0x7fb024157c50 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0241571f0 con 0x7fb0240b20b0 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb02a600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb024156ec0 con 0x7fb0240c85f0 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.264+0000 7fb023e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb024157c50 0x7fb02414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb023e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb024157c50 0x7fb02414c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:50426/0 (socket says 172.21.15.113:50426) 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb023400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0240b20b0 0x7fb0241e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb023e00640 1 -- 172.21.15.113:0/3541426912 learned_addr learned my addr 172.21.15.113:0/3541426912 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb022a00640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0240c85f0 0x7fb0240c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb023e00640 1 -- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0240c85f0 msgr2=0x7fb0240c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb023e00640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0240c85f0 0x7fb0240c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb023e00640 1 -- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0240b20b0 msgr2=0x7fb0241e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb023e00640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0240b20b0 0x7fb0241e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb023e00640 1 -- 172.21.15.113:0/3541426912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0241e90d0 con 0x7fb024157c50 2024-03-04T08:26:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb023e00640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb024157c50 0x7fb02414c220 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fb01c01bd20 tx=0x7fb01c01beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb01b400640 1 -- 172.21.15.113:0/3541426912 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0b4040 con 0x7fb024157c50 2024-03-04T08:26:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb01b400640 1 -- 172.21.15.113:0/3541426912 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01c0c1070 con 0x7fb024157c50 2024-03-04T08:26:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb02a600640 1 -- 172.21.15.113:0/3541426912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb024152d60 con 0x7fb024157c50 2024-03-04T08:26:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb01b400640 1 -- 172.21.15.113:0/3541426912 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01c009450 con 0x7fb024157c50 2024-03-04T08:26:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb02a600640 1 -- 172.21.15.113:0/3541426912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb024153000 con 0x7fb024157c50 2024-03-04T08:26:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb02a600640 1 -- 172.21.15.113:0/3541426912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb024075c00 con 0x7fb024157c50 2024-03-04T08:26:55.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.268+0000 7fb01b400640 1 -- 172.21.15.113:0/3541426912 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb01c0ba6b0 con 0x7fb024157c50 2024-03-04T08:26:55.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.272+0000 7fb01b400640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb000066f90 0x7fb000069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:55.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.272+0000 7fb01b400640 1 -- 172.21.15.113:0/3541426912 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fb01c124580 con 0x7fb024157c50 2024-03-04T08:26:55.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.272+0000 7fb023400640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb000066f90 0x7fb000069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:55.268 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.272+0000 7fb023400640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb000066f90 0x7fb000069450 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0045f0 tx=0x7fb00c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:55.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.272+0000 7fb01b400640 1 -- 172.21.15.113:0/3541426912 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb01c018c70 con 0x7fb024157c50 2024-03-04T08:26:55.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.460+0000 7fb02a600640 1 -- 172.21.15.113:0/3541426912 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb024151400 con 0x7fb024157c50 2024-03-04T08:26:55.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb01b400640 1 -- 172.21.15.113:0/3541426912 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fb01c0f44f0 con 0x7fb024157c50 2024-03-04T08:26:55.459 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:55.459 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:55.460 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:55.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 -- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb000066f90 msgr2=0x7fb000069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:55.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb000066f90 0x7fb000069450 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0045f0 tx=0x7fb00c008040 comp rx=0 tx=0).stop 2024-03-04T08:26:55.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 -- 172.21.15.113:0/3541426912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb024157c50 msgr2=0x7fb02414c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:55.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb024157c50 0x7fb02414c220 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fb01c01bd20 tx=0x7fb01c01beb0 comp rx=0 tx=0).stop 2024-03-04T08:26:55.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 -- 172.21.15.113:0/3541426912 shutdown_connections 2024-03-04T08:26:55.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb000066f90 0x7fb000069450 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:55.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb024157c50 0x7fb02414c220 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:55.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0240c85f0 0x7fb0240c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:55.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 --2- 172.21.15.113:0/3541426912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0240b20b0 0x7fb0241e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:55.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 -- 172.21.15.113:0/3541426912 >> 172.21.15.113:0/3541426912 conn(0x7fb02406e820 msgr2=0x7fb0240ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:55.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 -- 172.21.15.113:0/3541426912 shutdown_connections 2024-03-04T08:26:55.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:55.464+0000 7fb02a600640 1 -- 172.21.15.113:0/3541426912 wait complete. 2024-03-04T08:26:55.481 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:57.482 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:57.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efda1800640 1 Processor -- start 2024-03-04T08:26:57.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efda1800640 1 -- start start 2024-03-04T08:26:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efda1800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd9c14e4c0 0x7efd9c072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:57.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efda1800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9c0c27f0 con 0x7efd9c0c22e0 2024-03-04T08:26:57.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efda1800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9c0c2a90 con 0x7efd9c15b900 2024-03-04T08:26:57.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efda1800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9c15ad60 con 0x7efd9c14e4c0 2024-03-04T08:26:57.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efd9aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd9c14e4c0 0x7efd9c072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:57.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efd9aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efd9c14e4c0 0x7efd9c072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59314/0 (socket says 172.21.15.113:59314) 2024-03-04T08:26:57.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efd9aa00640 1 -- 172.21.15.113:0/317251000 learned_addr learned my addr 172.21.15.113:0/317251000 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:57.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efd92000640 1 --1- 172.21.15.113:0/317251000 >> v1:172.21.15.113:6789/0 conn(0x7efd9c0c22e0 0x7efd9c15b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60554/0 (socket says 172.21.15.113:60554) 2024-03-04T08:26:57.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.632+0000 7efd9aa00640 1 -- 172.21.15.113:0/317251000 >> v1:172.21.15.136:6789/0 conn(0x7efd9c15b900 legacy=0x7efd9c1620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:26:57.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efd9aa00640 1 -- 172.21.15.113:0/317251000 >> v1:172.21.15.113:6789/0 conn(0x7efd9c0c22e0 legacy=0x7efd9c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:57.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efd9aa00640 1 -- 172.21.15.113:0/317251000 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c070c60 con 0x7efd9c14e4c0 2024-03-04T08:26:57.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efd9aa00640 1 --2- 172.21.15.113:0/317251000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c14e4c0 0x7efd9c072c80 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009870 tx=0x7efd8c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9b0f81a7c2c412 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:57.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efd9a000640 1 -- 172.21.15.113:0/317251000 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0b5070 con 0x7efd9c14e4c0 2024-03-04T08:26:57.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efd9a000640 1 -- 172.21.15.113:0/317251000 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c0a8a50 con 0x7efd9c14e4c0 2024-03-04T08:26:57.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efd9a000640 1 -- 172.21.15.113:0/317251000 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0a8e20 con 0x7efd9c14e4c0 2024-03-04T08:26:57.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efda1800640 1 -- 172.21.15.113:0/317251000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c14e4c0 msgr2=0x7efd9c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:57.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efda1800640 1 --2- 172.21.15.113:0/317251000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c14e4c0 0x7efd9c072c80 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009870 tx=0x7efd8c01ed50 comp rx=0 tx=0).stop 2024-03-04T08:26:57.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efda1800640 1 -- 172.21.15.113:0/317251000 shutdown_connections 2024-03-04T08:26:57.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efda1800640 1 --2- 172.21.15.113:0/317251000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c14e4c0 0x7efd9c072c80 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:57.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efda1800640 1 -- 172.21.15.113:0/317251000 >> 172.21.15.113:0/317251000 conn(0x7efd9c06e820 msgr2=0x7efd9c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:57.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efda1800640 1 -- 172.21.15.113:0/317251000 shutdown_connections 2024-03-04T08:26:57.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.636+0000 7efda1800640 1 -- 172.21.15.113:0/317251000 wait complete. 2024-03-04T08:26:57.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efda1800640 1 Processor -- start 2024-03-04T08:26:57.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efda1800640 1 -- start start 2024-03-04T08:26:57.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efda1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c0c22e0 0x7efd9c159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:57.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efda1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd9c14e4c0 0x7efd9c149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:57.638 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efda1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9c15b900 0x7efd9c1499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efda1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c071500 con 0x7efd9c0c22e0 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efda1800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c071220 con 0x7efd9c15b900 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efda1800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c14e8a0 con 0x7efd9c14e4c0 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd9aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c0c22e0 0x7efd9c159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd9aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c0c22e0 0x7efd9c159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59316/0 (socket says 172.21.15.113:59316) 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd9aa00640 1 -- 172.21.15.113:0/1203634182 learned_addr learned my addr 172.21.15.113:0/1203634182 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd9b400640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9c15b900 0x7efd9c1499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd92000640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd9c14e4c0 0x7efd9c149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd9aa00640 1 -- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd9c14e4c0 msgr2=0x7efd9c149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd9aa00640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd9c14e4c0 0x7efd9c149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:57.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd9aa00640 1 -- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9c15b900 msgr2=0x7efd9c1499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:57.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd9aa00640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9c15b900 0x7efd9c1499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:57.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd9aa00640 1 -- 172.21.15.113:0/1203634182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c0c7bb0 con 0x7efd9c0c22e0 2024-03-04T08:26:57.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd9aa00640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c0c22e0 0x7efd9c159df0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7efd8c01c980 tx=0x7efd8c01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:57.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd93400640 1 -- 172.21.15.113:0/1203634182 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0c0070 con 0x7efd9c0c22e0 2024-03-04T08:26:57.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd93400640 1 -- 172.21.15.113:0/1203634182 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c0ae790 con 0x7efd9c0c22e0 2024-03-04T08:26:57.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efd93400640 1 -- 172.21.15.113:0/1203634182 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c0b5070 con 0x7efd9c0c22e0 2024-03-04T08:26:57.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efda1800640 1 -- 172.21.15.113:0/1203634182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c1f13d0 con 0x7efd9c0c22e0 2024-03-04T08:26:57.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.640+0000 7efda1800640 1 -- 172.21.15.113:0/1203634182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c06bbf0 con 0x7efd9c0c22e0 2024-03-04T08:26:57.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.644+0000 7efda1800640 1 -- 172.21.15.113:0/1203634182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c06be30 con 0x7efd9c0c22e0 2024-03-04T08:26:57.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.644+0000 7efd93400640 1 -- 172.21.15.113:0/1203634182 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd8c0b9070 con 0x7efd9c0c22e0 2024-03-04T08:26:57.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.644+0000 7efd93400640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd78066f90 0x7efd78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:26:57.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.644+0000 7efd93400640 1 -- 172.21.15.113:0/1203634182 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7efd8c123430 con 0x7efd9c0c22e0 2024-03-04T08:26:57.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.644+0000 7efd92000640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd78066f90 0x7efd78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:26:57.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.648+0000 7efd92000640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd78066f90 0x7efd78069450 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7efd9c15a7b0 tx=0x7efd84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:26:57.645 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.648+0000 7efd93400640 1 -- 172.21.15.113:0/1203634182 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efd9c06be30 con 0x7efd9c0c22e0 2024-03-04T08:26:57.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.832+0000 7efda1800640 1 -- 172.21.15.113:0/1203634182 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd9c075c00 con 0x7efd9c0c22e0 2024-03-04T08:26:57.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.832+0000 7efd93400640 1 -- 172.21.15.113:0/1203634182 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7efd8c0188b0 con 0x7efd9c0c22e0 2024-03-04T08:26:57.830 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:26:57.830 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:26:57.830 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:26:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 -- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd78066f90 msgr2=0x7efd78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd78066f90 0x7efd78069450 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7efd9c15a7b0 tx=0x7efd84009040 comp rx=0 tx=0).stop 2024-03-04T08:26:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 -- 172.21.15.113:0/1203634182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c0c22e0 msgr2=0x7efd9c159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:26:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c0c22e0 0x7efd9c159df0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7efd8c01c980 tx=0x7efd8c01baf0 comp rx=0 tx=0).stop 2024-03-04T08:26:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 -- 172.21.15.113:0/1203634182 shutdown_connections 2024-03-04T08:26:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd9c15b900 0x7efd9c1499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:57.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efd9c14e4c0 0x7efd9c149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:57.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efd78066f90 0x7efd78069450 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:57.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 --2- 172.21.15.113:0/1203634182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd9c0c22e0 0x7efd9c159df0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:26:57.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 -- 172.21.15.113:0/1203634182 >> 172.21.15.113:0/1203634182 conn(0x7efd9c06e820 msgr2=0x7efd9c072bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:26:57.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 -- 172.21.15.113:0/1203634182 shutdown_connections 2024-03-04T08:26:57.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:26:57.836+0000 7efda1800640 1 -- 172.21.15.113:0/1203634182 wait complete. 2024-03-04T08:26:57.851 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:26:59.852 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:26:59.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.000+0000 7f82fea00640 1 Processor -- start 2024-03-04T08:26:59.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.000+0000 7f82fea00640 1 -- start start 2024-03-04T08:27:00.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82fea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f82f81527f0 0x7f82f8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:00.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82fea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8152bd0 0x7f82f81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:00.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82fea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f81591f0 con 0x7f82f80c8200 2024-03-04T08:27:00.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82fea00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f8071df0 con 0x7f82f81527f0 2024-03-04T08:27:00.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82fea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81597b0 con 0x7f82f8152bd0 2024-03-04T08:27:00.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82efe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f82f80c8200 0x7f82f8072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60560/0 (socket says 172.21.15.113:60560) 2024-03-04T08:27:00.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82efe00640 1 -- 172.21.15.113:0/3387700387 learned_addr learned my addr 172.21.15.113:0/3387700387 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:00.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82fca00640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8152bd0 0x7f82f81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:00.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7e00640 1 --2- 172.21.15.113:0/3387700387 >> v2:172.21.15.136:3300/0 conn(0x7f82f81527f0 0x7f82f8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:00.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2613288104 0 0) 0x7f82f81591f0 con 0x7f82f80c8200 2024-03-04T08:27:00.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82dc003530 con 0x7f82f80c8200 2024-03-04T08:27:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 152578860 0 0) 0x7f82dc003530 con 0x7f82f80c8200 2024-03-04T08:27:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8152bd0 msgr2=0x7f82f81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8152bd0 0x7f82f81587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 >> v2:172.21.15.136:3300/0 conn(0x7f82f81527f0 msgr2=0x7f82f8070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 --2- 172.21.15.113:0/3387700387 >> v2:172.21.15.136:3300/0 conn(0x7f82f81527f0 0x7f82f8070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f81594d0 con 0x7f82f80c8200 2024-03-04T08:27:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f82e0001680 con 0x7f82f80c8200 2024-03-04T08:27:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82fca00640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8152bd0 0x7f82f81587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:27:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 >> v1:172.21.15.113:6789/0 conn(0x7f82f80c8200 legacy=0x7f82f8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:00.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8152bd0 0x7f82dc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc003f50 0x7f82dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc004d50 0x7f82dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f8071df0 con 0x7f82dc003f50 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81597b0 con 0x7f82dc004d50 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc00b9d0 con 0x7f82f8152bd0 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82efe00640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc003f50 0x7f82dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7e00640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8152bd0 0x7f82dc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82efe00640 1 -- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8152bd0 msgr2=0x7f82dc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82efe00640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8152bd0 0x7f82dc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82efe00640 1 -- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc004d50 msgr2=0x7f82dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82efe00640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc004d50 0x7f82dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82efe00640 1 -- 172.21.15.113:0/3387700387 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc003530 con 0x7f82dc003f50 2024-03-04T08:27:00.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.004+0000 7f82f7e00640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8152bd0 0x7f82dc0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:00.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82efe00640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc003f50 0x7f82dc004810 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f82e000c160 tx=0x7f82e001f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61a2821e61cffb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:00.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0007c70 con 0x7f82dc003f50 2024-03-04T08:27:00.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e0005b00 con 0x7f82dc003f50 2024-03-04T08:27:00.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82f7400640 1 -- 172.21.15.113:0/3387700387 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0007070 con 0x7f82dc003f50 2024-03-04T08:27:00.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82fea00640 1 -- 172.21.15.113:0/3387700387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc003f50 msgr2=0x7f82dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:00.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82fea00640 1 --2- 172.21.15.113:0/3387700387 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82dc003f50 0x7f82dc004810 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f82e000c160 tx=0x7f82e001f310 comp rx=0 tx=0).stop 2024-03-04T08:27:00.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82fca00640 1 -- 172.21.15.113:0/3387700387 reap_dead start 2024-03-04T08:27:00.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82fea00640 1 -- 172.21.15.113:0/3387700387 shutdown_connections 2024-03-04T08:27:00.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82fea00640 1 -- 172.21.15.113:0/3387700387 >> 172.21.15.113:0/3387700387 conn(0x7f82f806e820 msgr2=0x7f82f806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:00.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82fea00640 1 -- 172.21.15.113:0/3387700387 shutdown_connections 2024-03-04T08:27:00.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.008+0000 7f82fea00640 1 -- 172.21.15.113:0/3387700387 wait complete. 2024-03-04T08:27:00.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 Processor -- start 2024-03-04T08:27:00.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 -- start start 2024-03-04T08:27:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc003f50 0x7f82f81566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f81f0be0 0x7f82f81f1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8154570 0x7f82f8154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:00.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81587d0 con 0x7f82f81f0be0 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f8071a80 con 0x7f82dc003f50 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f8158950 con 0x7f82f8154570 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82efe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f81f0be0 0x7f82f81f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82efe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f81f0be0 0x7f82f81f1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59334/0 (socket says 172.21.15.113:59334) 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82efe00640 1 -- 172.21.15.113:0/1805817382 learned_addr learned my addr 172.21.15.113:0/1805817382 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82f7e00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc003f50 0x7f82f81566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fca00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8154570 0x7f82f8154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82efe00640 1 -- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8154570 msgr2=0x7f82f8154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82efe00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8154570 0x7f82f8154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82efe00640 1 -- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc003f50 msgr2=0x7f82f81566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82efe00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc003f50 0x7f82f81566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82efe00640 1 -- 172.21.15.113:0/1805817382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f814f380 con 0x7f82f81f0be0 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82efe00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f81f0be0 0x7f82f81f1020 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f82e001f140 tx=0x7f82dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:00.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82f4c00640 1 -- 172.21.15.113:0/1805817382 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0005d70 con 0x7f82f81f0be0 2024-03-04T08:27:00.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82f4c00640 1 -- 172.21.15.113:0/1805817382 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e00047f0 con 0x7f82f81f0be0 2024-03-04T08:27:00.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 -- 172.21.15.113:0/1805817382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f81f4310 con 0x7f82f81f0be0 2024-03-04T08:27:00.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82f4c00640 1 -- 172.21.15.113:0/1805817382 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0004be0 con 0x7f82f81f0be0 2024-03-04T08:27:00.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 -- 172.21.15.113:0/1805817382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f81f48d0 con 0x7f82f81f0be0 2024-03-04T08:27:00.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.012+0000 7f82fea00640 1 -- 172.21.15.113:0/1805817382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f8075c00 con 0x7f82f81f0be0 2024-03-04T08:27:00.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.016+0000 7f82f4c00640 1 -- 172.21.15.113:0/1805817382 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82e00ba030 con 0x7f82f81f0be0 2024-03-04T08:27:00.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.016+0000 7f82f4c00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82d4066f90 0x7f82d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:00.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.016+0000 7f82f4c00640 1 -- 172.21.15.113:0/1805817382 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f82e0122680 con 0x7f82f81f0be0 2024-03-04T08:27:00.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.016+0000 7f82f7e00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82d4066f90 0x7f82d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:00.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.016+0000 7f82f7e00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82d4066f90 0x7f82d4069450 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f82e80099b0 tx=0x7f82e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:00.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.020+0000 7f82f4c00640 1 -- 172.21.15.113:0/1805817382 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f82e00c1030 con 0x7f82f81f0be0 2024-03-04T08:27:00.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 -- 172.21.15.113:0/1805817382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82f8070cf0 con 0x7f82f81f0be0 2024-03-04T08:27:00.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82f4c00640 1 -- 172.21.15.113:0/1805817382 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f82e0019020 con 0x7f82f81f0be0 2024-03-04T08:27:00.207 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:00.208 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:00.208 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:00.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 -- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82d4066f90 msgr2=0x7f82d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:00.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82d4066f90 0x7f82d4069450 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f82e80099b0 tx=0x7f82e8008040 comp rx=0 tx=0).stop 2024-03-04T08:27:00.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 -- 172.21.15.113:0/1805817382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f81f0be0 msgr2=0x7f82f81f1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:00.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f81f0be0 0x7f82f81f1020 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f82e001f140 tx=0x7f82dc005840 comp rx=0 tx=0).stop 2024-03-04T08:27:00.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 -- 172.21.15.113:0/1805817382 shutdown_connections 2024-03-04T08:27:00.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f82f8154570 0x7f82f8154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:00.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f82d4066f90 0x7f82d4069450 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:00.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f82f81f0be0 0x7f82f81f1020 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:00.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 --2- 172.21.15.113:0/1805817382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82dc003f50 0x7f82f81566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:00.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 -- 172.21.15.113:0/1805817382 >> 172.21.15.113:0/1805817382 conn(0x7f82f806e820 msgr2=0x7f82f81570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:00.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 -- 172.21.15.113:0/1805817382 shutdown_connections 2024-03-04T08:27:00.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:00.212+0000 7f82fea00640 1 -- 172.21.15.113:0/1805817382 wait complete. 2024-03-04T08:27:00.229 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:02.230 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:02.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d3e00640 1 Processor -- start 2024-03-04T08:27:02.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d3e00640 1 -- start start 2024-03-04T08:27:02.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6cc1527f0 0x7fe6cc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:02.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe6cc0c8200 0x7fe6cc072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:02.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6cc1591f0 con 0x7fe6cc152bd0 2024-03-04T08:27:02.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6cc071df0 con 0x7fe6cc1527f0 2024-03-04T08:27:02.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc1597b0 con 0x7fe6cc0c8200 2024-03-04T08:27:02.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6cc1527f0 0x7fe6cc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:02.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d1e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe6cc152bd0 0x7fe6cc1587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60574/0 (socket says 172.21.15.113:60574) 2024-03-04T08:27:02.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d1e00640 1 -- 172.21.15.113:0/3005831531 learned_addr learned my addr 172.21.15.113:0/3005831531 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:02.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d0a00640 1 --2- 172.21.15.113:0/3005831531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe6cc0c8200 0x7fe6cc072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d1400640 1 -- 172.21.15.113:0/3005831531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe6cc0c8200 msgr2=0x7fe6cc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d1400640 1 --2- 172.21.15.113:0/3005831531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe6cc0c8200 0x7fe6cc072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d1400640 1 -- 172.21.15.113:0/3005831531 >> v1:172.21.15.113:6789/0 conn(0x7fe6cc152bd0 legacy=0x7fe6cc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6d1400640 1 -- 172.21.15.113:0/3005831531 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc1594d0 con 0x7fe6cc1527f0 2024-03-04T08:27:02.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.380+0000 7fe6cbe00640 1 -- 172.21.15.113:0/3005831531 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38558901 0 0) 0x7fe6cc1591f0 con 0x7fe6cc152bd0 2024-03-04T08:27:02.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6d1400640 1 --2- 172.21.15.113:0/3005831531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc1527f0 0x7fe6cc070d40 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009d90 tx=0x7fe6c001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=421342ab74b010cd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:02.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6cbe00640 1 -- 172.21.15.113:0/3005831531 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00b2070 con 0x7fe6cc1527f0 2024-03-04T08:27:02.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6cbe00640 1 -- 172.21.15.113:0/3005831531 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c00a5920 con 0x7fe6cc1527f0 2024-03-04T08:27:02.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6cbe00640 1 -- 172.21.15.113:0/3005831531 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00a5cf0 con 0x7fe6cc1527f0 2024-03-04T08:27:02.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6d3e00640 1 -- 172.21.15.113:0/3005831531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc1527f0 msgr2=0x7fe6cc070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:02.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6d3e00640 1 --2- 172.21.15.113:0/3005831531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc1527f0 0x7fe6cc070d40 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009d90 tx=0x7fe6c001c970 comp rx=0 tx=0).stop 2024-03-04T08:27:02.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6d3e00640 1 -- 172.21.15.113:0/3005831531 shutdown_connections 2024-03-04T08:27:02.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6d3e00640 1 --2- 172.21.15.113:0/3005831531 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe6cc0c8200 0x7fe6cc072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:02.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6d3e00640 1 --2- 172.21.15.113:0/3005831531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc1527f0 0x7fe6cc070d40 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:02.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6d3e00640 1 -- 172.21.15.113:0/3005831531 >> 172.21.15.113:0/3005831531 conn(0x7fe6cc06e820 msgr2=0x7fe6cc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:02.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6d3e00640 1 -- 172.21.15.113:0/3005831531 shutdown_connections 2024-03-04T08:27:02.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.384+0000 7fe6d3e00640 1 -- 172.21.15.113:0/3005831531 wait complete. 2024-03-04T08:27:02.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 Processor -- start 2024-03-04T08:27:02.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 -- start start 2024-03-04T08:27:02.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0c8200 0x7fe6cc154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:02.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe6cc1527f0 0x7fe6cc0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:02.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6cc152bd0 0x7fe6cc14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc165160 con 0x7fe6cc0c8200 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc0727c0 con 0x7fe6cc152bd0 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc1652e0 con 0x7fe6cc1527f0 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0c8200 0x7fe6cc154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0c8200 0x7fe6cc154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59354/0 (socket says 172.21.15.113:59354) 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d1400640 1 -- 172.21.15.113:0/2005823526 learned_addr learned my addr 172.21.15.113:0/2005823526 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d1e00640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6cc152bd0 0x7fe6cc14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d0a00640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe6cc1527f0 0x7fe6cc0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d1400640 1 -- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe6cc1527f0 msgr2=0x7fe6cc0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d1400640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe6cc1527f0 0x7fe6cc0af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d1400640 1 -- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6cc152bd0 msgr2=0x7fe6cc14dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d1400640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6cc152bd0 0x7fe6cc14dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:02.385 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d1400640 1 -- 172.21.15.113:0/2005823526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc154350 con 0x7fe6cc0c8200 2024-03-04T08:27:02.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d1400640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0c8200 0x7fe6cc154af0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00b0040 tx=0x7fe6c0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:02.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6c9600640 1 -- 172.21.15.113:0/2005823526 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00bd020 con 0x7fe6cc0c8200 2024-03-04T08:27:02.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6c9600640 1 -- 172.21.15.113:0/2005823526 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c00b8670 con 0x7fe6cc0c8200 2024-03-04T08:27:02.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6c9600640 1 -- 172.21.15.113:0/2005823526 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00b2040 con 0x7fe6cc0c8200 2024-03-04T08:27:02.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 -- 172.21.15.113:0/2005823526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc0b1580 con 0x7fe6cc0c8200 2024-03-04T08:27:02.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 -- 172.21.15.113:0/2005823526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc1535f0 con 0x7fe6cc0c8200 2024-03-04T08:27:02.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.388+0000 7fe6d3e00640 1 -- 172.21.15.113:0/2005823526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6cc075c00 con 0x7fe6cc0c8200 2024-03-04T08:27:02.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.392+0000 7fe6c9600640 1 -- 172.21.15.113:0/2005823526 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6c00c0020 con 0x7fe6cc0c8200 2024-03-04T08:27:02.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.392+0000 7fe6c9600640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe6ac066f60 0x7fe6ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:02.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.392+0000 7fe6c9600640 1 -- 172.21.15.113:0/2005823526 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fe6c0120bd0 con 0x7fe6cc0c8200 2024-03-04T08:27:02.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.392+0000 7fe6d0a00640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe6ac066f60 0x7fe6ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:02.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.392+0000 7fe6d0a00640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe6ac066f60 0x7fe6ac069420 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0724e0 tx=0x7fe6b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:02.392 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.396+0000 7fe6c9600640 1 -- 172.21.15.113:0/2005823526 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe6c00c7030 con 0x7fe6cc0c8200 2024-03-04T08:27:02.580 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.584+0000 7fe6d3e00640 1 -- 172.21.15.113:0/2005823526 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6cc0712e0 con 0x7fe6cc0c8200 2024-03-04T08:27:02.581 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.584+0000 7fe6c9600640 1 -- 172.21.15.113:0/2005823526 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fe6cc154350 con 0x7fe6cc0c8200 2024-03-04T08:27:02.581 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:02.581 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:02.582 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:02.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 -- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe6ac066f60 msgr2=0x7fe6ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:02.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe6ac066f60 0x7fe6ac069420 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0724e0 tx=0x7fe6b8008040 comp rx=0 tx=0).stop 2024-03-04T08:27:02.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 -- 172.21.15.113:0/2005823526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0c8200 msgr2=0x7fe6cc154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0c8200 0x7fe6cc154af0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00b0040 tx=0x7fe6c0019710 comp rx=0 tx=0).stop 2024-03-04T08:27:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 -- 172.21.15.113:0/2005823526 shutdown_connections 2024-03-04T08:27:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe6cc152bd0 0x7fe6cc14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe6cc1527f0 0x7fe6cc0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe6ac066f60 0x7fe6ac069420 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 --2- 172.21.15.113:0/2005823526 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6cc0c8200 0x7fe6cc154af0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 -- 172.21.15.113:0/2005823526 >> 172.21.15.113:0/2005823526 conn(0x7fe6cc06e820 msgr2=0x7fe6cc070b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 -- 172.21.15.113:0/2005823526 shutdown_connections 2024-03-04T08:27:02.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:02.588+0000 7fe6d3e00640 1 -- 172.21.15.113:0/2005823526 wait complete. 2024-03-04T08:27:02.603 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:27:02.604 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:27:04.605 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:04.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f2855800640 1 Processor -- start 2024-03-04T08:27:04.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f2855800640 1 -- start start 2024-03-04T08:27:04.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f2855800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f28500b2dd0 0x7f28500c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:04.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f2855800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f2850072e10 0x7f28500c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:04.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f2855800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2850151980 con 0x7f28501515a0 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f2855800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2850158350 con 0x7f28500b2dd0 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f2855800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28500734c0 con 0x7f2850072e10 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f284f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f28500b2dd0 0x7f28500c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f284fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f28501515a0 0x7f28500c3610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60578/0 (socket says 172.21.15.113:60578) 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f284fe00640 1 -- 172.21.15.113:0/1637074300 learned_addr learned my addr 172.21.15.113:0/1637074300 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f2854e00640 1 --2- 172.21.15.113:0/1637074300 >> v2:172.21.15.136:3300/0 conn(0x7f2850072e10 0x7f28500c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f284f400640 1 -- 172.21.15.113:0/1637074300 >> v2:172.21.15.136:3300/0 conn(0x7f2850072e10 msgr2=0x7f28500c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f284f400640 1 --2- 172.21.15.113:0/1637074300 >> v2:172.21.15.136:3300/0 conn(0x7f2850072e10 0x7f28500c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f284f400640 1 -- 172.21.15.113:0/1637074300 >> v1:172.21.15.113:6789/0 conn(0x7f28501515a0 legacy=0x7f28500c3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f284f400640 1 -- 172.21.15.113:0/1637074300 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2850073240 con 0x7f28500b2dd0 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f284ea00640 1 -- 172.21.15.113:0/1637074300 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891821276 0 0) 0x7f28400012a0 con 0x7f28501515a0 2024-03-04T08:27:04.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.756+0000 7f2854e00640 1 --2- 172.21.15.113:0/1637074300 >> v2:172.21.15.136:3300/0 conn(0x7f2850072e10 0x7f28500c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:04.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f284f400640 1 --2- 172.21.15.113:0/1637074300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 0x7f28500c8340 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f28500c99e0 tx=0x7f283c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4019feb8d4dd532e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:04.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f284ea00640 1 -- 172.21.15.113:0/1637074300 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0b5070 con 0x7f28500b2dd0 2024-03-04T08:27:04.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f284ea00640 1 -- 172.21.15.113:0/1637074300 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c0a8970 con 0x7f28500b2dd0 2024-03-04T08:27:04.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f284ea00640 1 -- 172.21.15.113:0/1637074300 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0a8d40 con 0x7f28500b2dd0 2024-03-04T08:27:04.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f2855800640 1 -- 172.21.15.113:0/1637074300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 msgr2=0x7f28500c8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:04.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f2855800640 1 --2- 172.21.15.113:0/1637074300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 0x7f28500c8340 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f28500c99e0 tx=0x7f283c01ec70 comp rx=0 tx=0).stop 2024-03-04T08:27:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f2855800640 1 -- 172.21.15.113:0/1637074300 shutdown_connections 2024-03-04T08:27:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f2855800640 1 --2- 172.21.15.113:0/1637074300 >> v2:172.21.15.136:3300/0 conn(0x7f2850072e10 0x7f28500c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f2855800640 1 --2- 172.21.15.113:0/1637074300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 0x7f28500c8340 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f2855800640 1 -- 172.21.15.113:0/1637074300 >> 172.21.15.113:0/1637074300 conn(0x7f28500c9a30 msgr2=0x7f28500c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f2855800640 1 -- 172.21.15.113:0/1637074300 shutdown_connections 2024-03-04T08:27:04.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f2855800640 1 -- 172.21.15.113:0/1637074300 wait complete. 2024-03-04T08:27:04.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f2855800640 1 Processor -- start 2024-03-04T08:27:04.758 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.760+0000 7f2855800640 1 -- start start 2024-03-04T08:27:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2855800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850072e10 0x7f28500c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2855800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 0x7f285014c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2855800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28501515a0 0x7f28500c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2855800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850157d00 con 0x7f28500b2dd0 2024-03-04T08:27:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2855800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850157a20 con 0x7f2850072e10 2024-03-04T08:27:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2855800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2850157fe0 con 0x7f28501515a0 2024-03-04T08:27:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 0x7f285014c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:04.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 0x7f285014c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59364/0 (socket says 172.21.15.113:59364) 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284f400640 1 -- 172.21.15.113:0/853620112 learned_addr learned my addr 172.21.15.113:0/853620112 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284fe00640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850072e10 0x7f28500c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2854e00640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28501515a0 0x7f28500c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284f400640 1 -- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28501515a0 msgr2=0x7f28500c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284f400640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28501515a0 0x7f28500c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284f400640 1 -- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850072e10 msgr2=0x7f28500c3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284f400640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850072e10 0x7f28500c3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284f400640 1 -- 172.21.15.113:0/853620112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28501535f0 con 0x7f28500b2dd0 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284fe00640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850072e10 0x7f28500c3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2854e00640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28501515a0 0x7f28500c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f284f400640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 0x7f285014c5e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f283c0b3040 tx=0x7f283c01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2847e00640 1 -- 172.21.15.113:0/853620112 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0bf070 con 0x7f28500b2dd0 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2847e00640 1 -- 172.21.15.113:0/853620112 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c0b0740 con 0x7f28500b2dd0 2024-03-04T08:27:04.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2855800640 1 -- 172.21.15.113:0/853620112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28501538d0 con 0x7f28500b2dd0 2024-03-04T08:27:04.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2847e00640 1 -- 172.21.15.113:0/853620112 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0b5040 con 0x7f28500b2dd0 2024-03-04T08:27:04.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2855800640 1 -- 172.21.15.113:0/853620112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2850153e90 con 0x7f28500b2dd0 2024-03-04T08:27:04.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2855800640 1 -- 172.21.15.113:0/853620112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2850075a80 con 0x7f28500b2dd0 2024-03-04T08:27:04.762 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.764+0000 7f2847e00640 1 -- 172.21.15.113:0/853620112 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f283c0c2070 con 0x7f28500b2dd0 2024-03-04T08:27:04.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.768+0000 7f2847e00640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2830066f60 0x7f2830069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:04.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.768+0000 7f2847e00640 1 -- 172.21.15.113:0/853620112 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f283c1223d0 con 0x7f28500b2dd0 2024-03-04T08:27:04.765 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.768+0000 7f284fe00640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2830066f60 0x7f2830069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:04.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.772+0000 7f2847e00640 1 -- 172.21.15.113:0/853620112 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2850073240 con 0x7f28500b2dd0 2024-03-04T08:27:04.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.772+0000 7f284fe00640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2830066f60 0x7f2830069420 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2840001670 tx=0x7f2840009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:04.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.956+0000 7f2855800640 1 -- 172.21.15.113:0/853620112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28500c1f50 con 0x7f28500b2dd0 2024-03-04T08:27:04.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.956+0000 7f2847e00640 1 -- 172.21.15.113:0/853620112 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f283c0187d0 con 0x7f28500b2dd0 2024-03-04T08:27:04.953 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:04.953 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:04.954 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:04.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 -- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2830066f60 msgr2=0x7f2830069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:04.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2830066f60 0x7f2830069420 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2840001670 tx=0x7f2840009040 comp rx=0 tx=0).stop 2024-03-04T08:27:04.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 -- 172.21.15.113:0/853620112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 msgr2=0x7f285014c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:04.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 0x7f285014c5e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f283c0b3040 tx=0x7f283c01ba10 comp rx=0 tx=0).stop 2024-03-04T08:27:04.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 -- 172.21.15.113:0/853620112 shutdown_connections 2024-03-04T08:27:04.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28501515a0 0x7f28500c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:04.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2830066f60 0x7f2830069420 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:04.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f28500b2dd0 0x7f285014c5e0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:04.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 --2- 172.21.15.113:0/853620112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2850072e10 0x7f28500c3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:04.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 -- 172.21.15.113:0/853620112 >> 172.21.15.113:0/853620112 conn(0x7f28500c9a30 msgr2=0x7f2850154880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:04.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 -- 172.21.15.113:0/853620112 shutdown_connections 2024-03-04T08:27:04.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:04.960+0000 7f2855800640 1 -- 172.21.15.113:0/853620112 wait complete. 2024-03-04T08:27:04.975 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:06.976 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:07.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75fa400640 1 Processor -- start 2024-03-04T08:27:07.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75fa400640 1 -- start start 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75fa400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f75f40c22e0 0x7f75f415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75fa400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75f40c2d80 0x7f75f415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75fa400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f40c27f0 con 0x7f75f414e4c0 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75fa400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f415ad60 con 0x7f75f40c2d80 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75fa400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f40c2a90 con 0x7f75f40c22e0 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f3400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f75f414e4c0 0x7f75f4072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60580/0 (socket says 172.21.15.113:60580) 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f3400640 1 -- 172.21.15.113:0/3255198810 learned_addr learned my addr 172.21.15.113:0/3255198810 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f3e00640 1 --2- 172.21.15.113:0/3255198810 >> v2:172.21.15.113:3300/0 conn(0x7f75f40c2d80 0x7f75f415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f2a00640 1 --2- 172.21.15.113:0/3255198810 >> v2:172.21.15.136:3300/0 conn(0x7f75f40c22e0 0x7f75f415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f2000640 1 -- 172.21.15.113:0/3255198810 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597796085 0 0) 0x7f75f40c27f0 con 0x7f75f414e4c0 2024-03-04T08:27:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f2000640 1 -- 172.21.15.113:0/3255198810 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75d8003530 con 0x7f75f414e4c0 2024-03-04T08:27:07.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f3e00640 1 -- 172.21.15.113:0/3255198810 >> v2:172.21.15.136:3300/0 conn(0x7f75f40c22e0 msgr2=0x7f75f415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:07.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f3e00640 1 --2- 172.21.15.113:0/3255198810 >> v2:172.21.15.136:3300/0 conn(0x7f75f40c22e0 0x7f75f415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:07.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f3e00640 1 -- 172.21.15.113:0/3255198810 >> v1:172.21.15.113:6789/0 conn(0x7f75f414e4c0 legacy=0x7f75f4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:07.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f3e00640 1 -- 172.21.15.113:0/3255198810 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f4070c60 con 0x7f75f40c2d80 2024-03-04T08:27:07.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.127+0000 7f75f2a00640 1 --2- 172.21.15.113:0/3255198810 >> v2:172.21.15.136:3300/0 conn(0x7f75f40c22e0 0x7f75f415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:07.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75f3e00640 1 --2- 172.21.15.113:0/3255198810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f40c2d80 0x7f75f415b730 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f75f40cd230 tx=0x7f75ec01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1b53ff19e48584 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75f2000640 1 -- 172.21.15.113:0/3255198810 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0b2070 con 0x7f75f40c2d80 2024-03-04T08:27:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75f2000640 1 -- 172.21.15.113:0/3255198810 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec0a5900 con 0x7f75f40c2d80 2024-03-04T08:27:07.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75f2000640 1 -- 172.21.15.113:0/3255198810 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0a5cd0 con 0x7f75f40c2d80 2024-03-04T08:27:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75fa400640 1 -- 172.21.15.113:0/3255198810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f40c2d80 msgr2=0x7f75f415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75fa400640 1 --2- 172.21.15.113:0/3255198810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f40c2d80 0x7f75f415b730 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f75f40cd230 tx=0x7f75ec01c930 comp rx=0 tx=0).stop 2024-03-04T08:27:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75fa400640 1 -- 172.21.15.113:0/3255198810 shutdown_connections 2024-03-04T08:27:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75fa400640 1 --2- 172.21.15.113:0/3255198810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f40c2d80 0x7f75f415b730 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75fa400640 1 --2- 172.21.15.113:0/3255198810 >> v2:172.21.15.136:3300/0 conn(0x7f75f40c22e0 0x7f75f415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75fa400640 1 -- 172.21.15.113:0/3255198810 >> 172.21.15.113:0/3255198810 conn(0x7f75f406e820 msgr2=0x7f75f406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75fa400640 1 -- 172.21.15.113:0/3255198810 shutdown_connections 2024-03-04T08:27:07.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75fa400640 1 -- 172.21.15.113:0/3255198810 wait complete. 2024-03-04T08:27:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.131+0000 7f75fa400640 1 Processor -- start 2024-03-04T08:27:07.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75fa400640 1 -- start start 2024-03-04T08:27:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75fa400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f40c22e0 0x7f75f40cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75fa400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75f40c2d80 0x7f75f4159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75fa400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f414e4c0 0x7f75f4146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75fa400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40aede0 con 0x7f75f414e4c0 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75fa400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f414e8a0 con 0x7f75f40c22e0 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75fa400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f4073390 con 0x7f75f40c2d80 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f414e4c0 0x7f75f4146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f414e4c0 0x7f75f4146b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:59388/0 (socket says 172.21.15.113:59388) 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f3e00640 1 -- 172.21.15.113:0/750978246 learned_addr learned my addr 172.21.15.113:0/750978246 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f2a00640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75f40c2d80 0x7f75f4159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f3400640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f40c22e0 0x7f75f40cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f3e00640 1 -- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75f40c2d80 msgr2=0x7f75f4159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f3e00640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75f40c2d80 0x7f75f4159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f3e00640 1 -- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f40c22e0 msgr2=0x7f75f40cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:07.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f3e00640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f40c22e0 0x7f75f40cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f3e00640 1 -- 172.21.15.113:0/750978246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f41f6b80 con 0x7f75f414e4c0 2024-03-04T08:27:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75f3e00640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f414e4c0 0x7f75f4146b50 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01a560 tx=0x7f75ec0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75eb400640 1 -- 172.21.15.113:0/750978246 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0b2050 con 0x7f75f414e4c0 2024-03-04T08:27:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75eb400640 1 -- 172.21.15.113:0/750978246 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec0be070 con 0x7f75f414e4c0 2024-03-04T08:27:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75fa400640 1 -- 172.21.15.113:0/750978246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f4152fb0 con 0x7f75f414e4c0 2024-03-04T08:27:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75eb400640 1 -- 172.21.15.113:0/750978246 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0ada90 con 0x7f75f414e4c0 2024-03-04T08:27:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75fa400640 1 -- 172.21.15.113:0/750978246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f4153570 con 0x7f75f414e4c0 2024-03-04T08:27:07.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75fa400640 1 -- 172.21.15.113:0/750978246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f4075c00 con 0x7f75f414e4c0 2024-03-04T08:27:07.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.135+0000 7f75eb400640 1 -- 172.21.15.113:0/750978246 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75ec0a5780 con 0x7f75f414e4c0 2024-03-04T08:27:07.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.139+0000 7f75eb400640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75d4066f90 0x7f75d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:07.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.139+0000 7f75eb400640 1 -- 172.21.15.113:0/750978246 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f75ec120b70 con 0x7f75f414e4c0 2024-03-04T08:27:07.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.139+0000 7f75f3400640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75d4066f90 0x7f75d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:07.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.139+0000 7f75f3400640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75d4066f90 0x7f75d4069450 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f75e4005890 tx=0x7f75e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:07.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.143+0000 7f75eb400640 1 -- 172.21.15.113:0/750978246 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f75ec016520 con 0x7f75f414e4c0 2024-03-04T08:27:07.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.331+0000 7f75fa400640 1 -- 172.21.15.113:0/750978246 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75f4155190 con 0x7f75f414e4c0 2024-03-04T08:27:07.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.331+0000 7f75eb400640 1 -- 172.21.15.113:0/750978246 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f75ec0f1020 con 0x7f75f414e4c0 2024-03-04T08:27:07.329 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:07.329 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:07.329 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.331+0000 7f75fa400640 1 -- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75d4066f90 msgr2=0x7f75d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.331+0000 7f75fa400640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75d4066f90 0x7f75d4069450 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f75e4005890 tx=0x7f75e4009040 comp rx=0 tx=0).stop 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.331+0000 7f75fa400640 1 -- 172.21.15.113:0/750978246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f414e4c0 msgr2=0x7f75f4146b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.331+0000 7f75fa400640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f414e4c0 0x7f75f4146b50 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01a560 tx=0x7f75ec0196d0 comp rx=0 tx=0).stop 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.335+0000 7f75fa400640 1 -- 172.21.15.113:0/750978246 shutdown_connections 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.335+0000 7f75fa400640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75d4066f90 0x7f75d4069450 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.335+0000 7f75fa400640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f414e4c0 0x7f75f4146b50 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.335+0000 7f75fa400640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f75f40c2d80 0x7f75f4159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.335+0000 7f75fa400640 1 --2- 172.21.15.113:0/750978246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f40c22e0 0x7f75f40cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.335+0000 7f75fa400640 1 -- 172.21.15.113:0/750978246 >> 172.21.15.113:0/750978246 conn(0x7f75f406e820 msgr2=0x7f75f4072c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.335+0000 7f75fa400640 1 -- 172.21.15.113:0/750978246 shutdown_connections 2024-03-04T08:27:07.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:07.335+0000 7f75fa400640 1 -- 172.21.15.113:0/750978246 wait complete. 2024-03-04T08:27:07.349 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:09.350 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:09.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.499+0000 7f4d6fe00640 1 Processor -- start 2024-03-04T08:27:09.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.499+0000 7f4d6fe00640 1 -- start start 2024-03-04T08:27:09.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.499+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6814e4c0 0x7f4d68072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:09.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.499+0000 7f4d6fe00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d680c27f0 con 0x7f4d6815b900 2024-03-04T08:27:09.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.499+0000 7f4d6fe00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d680c2a90 con 0x7f4d680c22e0 2024-03-04T08:27:09.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.499+0000 7f4d6fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6815ad60 con 0x7f4d6814e4c0 2024-03-04T08:27:09.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.503+0000 7f4d6de00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f4d6815b900 0x7f4d681620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37422/0 (socket says 172.21.15.113:37422) 2024-03-04T08:27:09.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.503+0000 7f4d6de00640 1 -- 172.21.15.113:0/3113237986 learned_addr learned my addr 172.21.15.113:0/3113237986 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:09.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.503+0000 7f4d6d400640 1 --2- 172.21.15.113:0/3113237986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6814e4c0 0x7f4d68072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:09.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.503+0000 7f4d67e00640 1 -- 172.21.15.113:0/3113237986 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773459871 0 0) 0x7f4d680c27f0 con 0x7f4d6815b900 2024-03-04T08:27:09.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.503+0000 7f4d67e00640 1 -- 172.21.15.113:0/3113237986 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d50003530 con 0x7f4d6815b900 2024-03-04T08:27:09.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.503+0000 7f4d67e00640 1 -- 172.21.15.113:0/3113237986 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2748281341 0 0) 0x7f4d50003530 con 0x7f4d6815b900 2024-03-04T08:27:09.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.503+0000 7f4d6d400640 1 -- 172.21.15.113:0/3113237986 >> v1:172.21.15.136:6789/0 conn(0x7f4d680c22e0 legacy=0x7f4d6815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:09.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.503+0000 7f4d6d400640 1 -- 172.21.15.113:0/3113237986 >> v1:172.21.15.113:6789/0 conn(0x7f4d6815b900 legacy=0x7f4d681620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:09.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.503+0000 7f4d6d400640 1 -- 172.21.15.113:0/3113237986 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d68070c60 con 0x7f4d6814e4c0 2024-03-04T08:27:09.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.503+0000 7f4d6d400640 1 --2- 172.21.15.113:0/3113237986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6814e4c0 0x7f4d68072c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009820 tx=0x7f4d5c01ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928f22401be90860 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:09.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d67e00640 1 -- 172.21.15.113:0/3113237986 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0b5070 con 0x7f4d6814e4c0 2024-03-04T08:27:09.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d67e00640 1 -- 172.21.15.113:0/3113237986 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c0a8690 con 0x7f4d6814e4c0 2024-03-04T08:27:09.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d67e00640 1 -- 172.21.15.113:0/3113237986 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60001a00 con 0x7f4d6814e4c0 2024-03-04T08:27:09.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3113237986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6814e4c0 msgr2=0x7f4d68072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 --2- 172.21.15.113:0/3113237986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6814e4c0 0x7f4d68072c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009820 tx=0x7f4d5c01ebd0 comp rx=0 tx=0).stop 2024-03-04T08:27:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3113237986 shutdown_connections 2024-03-04T08:27:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 --2- 172.21.15.113:0/3113237986 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6814e4c0 0x7f4d68072c80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3113237986 >> 172.21.15.113:0/3113237986 conn(0x7f4d6806e820 msgr2=0x7f4d6806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3113237986 shutdown_connections 2024-03-04T08:27:09.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3113237986 wait complete. 2024-03-04T08:27:09.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 Processor -- start 2024-03-04T08:27:09.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 -- start start 2024-03-04T08:27:09.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d680c22e0 0x7f4d68166240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:09.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d6814e4c0 0x7f4d68166780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:09.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6815b900 0x7f4d68141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:09.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d68071500 con 0x7f4d680c22e0 2024-03-04T08:27:09.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d68071220 con 0x7f4d6814e4c0 2024-03-04T08:27:09.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6814e8a0 con 0x7f4d6815b900 2024-03-04T08:27:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d680c22e0 0x7f4d68166240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6d400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d680c22e0 0x7f4d68166240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43970/0 (socket says 172.21.15.113:43970) 2024-03-04T08:27:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6d400640 1 -- 172.21.15.113:0/3184752339 learned_addr learned my addr 172.21.15.113:0/3184752339 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6de00640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6815b900 0x7f4d68141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.507+0000 7f4d6ca00640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d6814e4c0 0x7f4d68166780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d6d400640 1 -- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6815b900 msgr2=0x7f4d68141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d6d400640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6815b900 0x7f4d68141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d6d400640 1 -- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d6814e4c0 msgr2=0x7f4d68166780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d6d400640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d6814e4c0 0x7f4d68166780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:09.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d6d400640 1 -- 172.21.15.113:0/3184752339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d68165260 con 0x7f4d680c22e0 2024-03-04T08:27:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d6d400640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d680c22e0 0x7f4d68166240 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009740 tx=0x7f4d5c01b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d65600640 1 -- 172.21.15.113:0/3184752339 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0b5040 con 0x7f4d680c22e0 2024-03-04T08:27:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d65600640 1 -- 172.21.15.113:0/3184752339 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c0bf070 con 0x7f4d680c22e0 2024-03-04T08:27:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d65600640 1 -- 172.21.15.113:0/3184752339 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0b0840 con 0x7f4d680c22e0 2024-03-04T08:27:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3184752339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d680c3570 con 0x7f4d680c22e0 2024-03-04T08:27:09.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3184752339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d680c3b10 con 0x7f4d680c22e0 2024-03-04T08:27:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3184752339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d68075c00 con 0x7f4d680c22e0 2024-03-04T08:27:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d65600640 1 -- 172.21.15.113:0/3184752339 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d5c0b09e0 con 0x7f4d680c22e0 2024-03-04T08:27:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d65600640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d48066f60 0x7f4d48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d65600640 1 -- 172.21.15.113:0/3184752339 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f4d5c122e70 con 0x7f4d680c22e0 2024-03-04T08:27:09.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.511+0000 7f4d6ca00640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d48066f60 0x7f4d48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:09.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.515+0000 7f4d6ca00640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d48066f60 0x7f4d48069420 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4d680c86f0 tx=0x7f4d54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:09.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.515+0000 7f4d65600640 1 -- 172.21.15.113:0/3184752339 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4d5c018770 con 0x7f4d680c22e0 2024-03-04T08:27:09.702 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.703+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3184752339 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d6806ee40 con 0x7f4d680c22e0 2024-03-04T08:27:09.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d65600640 1 -- 172.21.15.113:0/3184752339 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f4d5c0f33d0 con 0x7f4d680c22e0 2024-03-04T08:27:09.703 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:09.703 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:09.703 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d48066f60 msgr2=0x7f4d48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d48066f60 0x7f4d48069420 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4d680c86f0 tx=0x7f4d54009040 comp rx=0 tx=0).stop 2024-03-04T08:27:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3184752339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d680c22e0 msgr2=0x7f4d68166240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:09.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d680c22e0 0x7f4d68166240 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009740 tx=0x7f4d5c01b970 comp rx=0 tx=0).stop 2024-03-04T08:27:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3184752339 shutdown_connections 2024-03-04T08:27:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4d6815b900 0x7f4d68141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4d6814e4c0 0x7f4d68166780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d48066f60 0x7f4d48069420 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 --2- 172.21.15.113:0/3184752339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d680c22e0 0x7f4d68166240 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3184752339 >> 172.21.15.113:0/3184752339 conn(0x7f4d6806e820 msgr2=0x7f4d68074700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3184752339 shutdown_connections 2024-03-04T08:27:09.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:09.707+0000 7f4d6fe00640 1 -- 172.21.15.113:0/3184752339 wait complete. 2024-03-04T08:27:09.725 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:11.726 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:11.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb9b600640 1 Processor -- start 2024-03-04T08:27:11.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb9b600640 1 -- start start 2024-03-04T08:27:11.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb9b600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbb9414e4c0 0x7fbb94072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:11.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb9b600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb9415b900 0x7fbb940731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:11.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb9b600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb940c27f0 con 0x7fbb940c22e0 2024-03-04T08:27:11.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb9b600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb9415ad60 con 0x7fbb9415b900 2024-03-04T08:27:11.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb9b600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb940c2a90 con 0x7fbb9414e4c0 2024-03-04T08:27:11.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb9415b900 0x7fbb940731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:11.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbb9415b900 0x7fbb940731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43984/0 (socket says 172.21.15.113:43984) 2024-03-04T08:27:11.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb99600640 1 -- 172.21.15.113:0/3102261314 learned_addr learned my addr 172.21.15.113:0/3102261314 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:11.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb93e00640 1 --1- 172.21.15.113:0/3102261314 >> v1:172.21.15.113:6789/0 conn(0x7fbb940c22e0 0x7fbb9415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37426/0 (socket says 172.21.15.113:37426) 2024-03-04T08:27:11.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb98c00640 1 --2- 172.21.15.113:0/3102261314 >> v2:172.21.15.136:3300/0 conn(0x7fbb9414e4c0 0x7fbb94072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:11.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb99600640 1 -- 172.21.15.113:0/3102261314 >> v2:172.21.15.136:3300/0 conn(0x7fbb9414e4c0 msgr2=0x7fbb94072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:11.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb99600640 1 --2- 172.21.15.113:0/3102261314 >> v2:172.21.15.136:3300/0 conn(0x7fbb9414e4c0 0x7fbb94072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:11.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb99600640 1 -- 172.21.15.113:0/3102261314 >> v1:172.21.15.113:6789/0 conn(0x7fbb940c22e0 legacy=0x7fbb9415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:11.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb99600640 1 -- 172.21.15.113:0/3102261314 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb94070c60 con 0x7fbb9415b900 2024-03-04T08:27:11.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.875+0000 7fbb98c00640 1 --2- 172.21.15.113:0/3102261314 >> v2:172.21.15.136:3300/0 conn(0x7fbb9414e4c0 0x7fbb94072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:11.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb99600640 1 --2- 172.21.15.113:0/3102261314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb9415b900 0x7fbb940731c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fbb9406f690 tx=0x7fbb8c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1a121b62ceb178 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:11.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb93400640 1 -- 172.21.15.113:0/3102261314 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0b4070 con 0x7fbb9415b900 2024-03-04T08:27:11.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb93400640 1 -- 172.21.15.113:0/3102261314 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c0af400 con 0x7fbb9415b900 2024-03-04T08:27:11.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb93400640 1 -- 172.21.15.113:0/3102261314 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0af7d0 con 0x7fbb9415b900 2024-03-04T08:27:11.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb9b600640 1 -- 172.21.15.113:0/3102261314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb9415b900 msgr2=0x7fbb940731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:11.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb9b600640 1 --2- 172.21.15.113:0/3102261314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb9415b900 0x7fbb940731c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fbb9406f690 tx=0x7fbb8c01f1e0 comp rx=0 tx=0).stop 2024-03-04T08:27:11.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb9b600640 1 -- 172.21.15.113:0/3102261314 shutdown_connections 2024-03-04T08:27:11.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb9b600640 1 --2- 172.21.15.113:0/3102261314 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb9415b900 0x7fbb940731c0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:11.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb9b600640 1 --2- 172.21.15.113:0/3102261314 >> v2:172.21.15.136:3300/0 conn(0x7fbb9414e4c0 0x7fbb94072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:11.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb9b600640 1 -- 172.21.15.113:0/3102261314 >> 172.21.15.113:0/3102261314 conn(0x7fbb9406e820 msgr2=0x7fbb9406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:11.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb9b600640 1 -- 172.21.15.113:0/3102261314 shutdown_connections 2024-03-04T08:27:11.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.879+0000 7fbb9b600640 1 -- 172.21.15.113:0/3102261314 wait complete. 2024-03-04T08:27:11.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 Processor -- start 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 -- start start 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb940c22e0 0x7fbb94148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb9414e4c0 0x7fbb941e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb9415b900 0x7fbb941f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94071500 con 0x7fbb940c22e0 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94071220 con 0x7fbb9415b900 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9414e8a0 con 0x7fbb9414e4c0 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb98c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb940c22e0 0x7fbb94148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb98c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb940c22e0 0x7fbb94148e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43988/0 (socket says 172.21.15.113:43988) 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb98c00640 1 -- 172.21.15.113:0/2956482985 learned_addr learned my addr 172.21.15.113:0/2956482985 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb99600640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb9415b900 0x7fbb941f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb93e00640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb9414e4c0 0x7fbb941e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb98c00640 1 -- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb9414e4c0 msgr2=0x7fbb941e71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:11.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb98c00640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb9414e4c0 0x7fbb941e71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb98c00640 1 -- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb9415b900 msgr2=0x7fbb941f0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb98c00640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb9415b900 0x7fbb941f0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb98c00640 1 -- 172.21.15.113:0/2956482985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb941f4210 con 0x7fbb940c22e0 2024-03-04T08:27:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb98c00640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb940c22e0 0x7fbb94148e10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800e810 tx=0x7fbb8800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb90c00640 1 -- 172.21.15.113:0/2956482985 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88017070 con 0x7fbb940c22e0 2024-03-04T08:27:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb90c00640 1 -- 172.21.15.113:0/2956482985 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8800c8d0 con 0x7fbb940c22e0 2024-03-04T08:27:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb90c00640 1 -- 172.21.15.113:0/2956482985 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800f040 con 0x7fbb940c22e0 2024-03-04T08:27:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 -- 172.21.15.113:0/2956482985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb941f44d0 con 0x7fbb940c22e0 2024-03-04T08:27:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 -- 172.21.15.113:0/2956482985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb941f4a90 con 0x7fbb940c22e0 2024-03-04T08:27:11.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.883+0000 7fbb9b600640 1 -- 172.21.15.113:0/2956482985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb94075c00 con 0x7fbb940c22e0 2024-03-04T08:27:11.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.887+0000 7fbb90c00640 1 -- 172.21.15.113:0/2956482985 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb8801a070 con 0x7fbb940c22e0 2024-03-04T08:27:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.887+0000 7fbb90c00640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb74066f90 0x7fbb74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.887+0000 7fbb90c00640 1 -- 172.21.15.113:0/2956482985 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fbb88085670 con 0x7fbb940c22e0 2024-03-04T08:27:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.887+0000 7fbb93e00640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb74066f90 0x7fbb74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:11.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.887+0000 7fbb93e00640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb74066f90 0x7fbb74069450 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbb940aee40 tx=0x7fbb80007c50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:11.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:11.891+0000 7fbb90c00640 1 -- 172.21.15.113:0/2956482985 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbb88055bd0 con 0x7fbb940c22e0 2024-03-04T08:27:12.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.075+0000 7fbb9b600640 1 -- 172.21.15.113:0/2956482985 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb9414b260 con 0x7fbb940c22e0 2024-03-04T08:27:12.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb90c00640 1 -- 172.21.15.113:0/2956482985 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fbb880554d0 con 0x7fbb940c22e0 2024-03-04T08:27:12.075 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:12.075 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:12.075 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 -- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb74066f90 msgr2=0x7fbb74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb74066f90 0x7fbb74069450 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbb940aee40 tx=0x7fbb80007c50 comp rx=0 tx=0).stop 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 -- 172.21.15.113:0/2956482985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb940c22e0 msgr2=0x7fbb94148e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb940c22e0 0x7fbb94148e10 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800e810 tx=0x7fbb8800ecd0 comp rx=0 tx=0).stop 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 -- 172.21.15.113:0/2956482985 shutdown_connections 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb9415b900 0x7fbb941f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbb9414e4c0 0x7fbb941e71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbb74066f90 0x7fbb74069450 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 --2- 172.21.15.113:0/2956482985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbb940c22e0 0x7fbb94148e10 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 -- 172.21.15.113:0/2956482985 >> 172.21.15.113:0/2956482985 conn(0x7fbb9406e820 msgr2=0x7fbb940c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 -- 172.21.15.113:0/2956482985 shutdown_connections 2024-03-04T08:27:12.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:12.079+0000 7fbb9b600640 1 -- 172.21.15.113:0/2956482985 wait complete. 2024-03-04T08:27:12.097 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:14.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:14.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe759c00640 1 Processor -- start 2024-03-04T08:27:14.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe759c00640 1 -- start start 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe759c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe75414e4c0 0x7fe754072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe759c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe75415b900 0x7fe7540731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe759c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7540c27f0 con 0x7fe7540c22e0 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe759c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75415ad60 con 0x7fe75414e4c0 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe759c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540c2a90 con 0x7fe75415b900 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe74a000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe7540c22e0 0x7fe75415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37430/0 (socket says 172.21.15.113:37430) 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe74a000640 1 -- 172.21.15.113:0/2723488485 learned_addr learned my addr 172.21.15.113:0/2723488485 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe752a00640 1 --2- 172.21.15.113:0/2723488485 >> v2:172.21.15.136:3300/0 conn(0x7fe75414e4c0 0x7fe754072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe753400640 1 --2- 172.21.15.113:0/2723488485 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe75415b900 0x7fe7540731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe752000640 1 -- 172.21.15.113:0/2723488485 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747564819 0 0) 0x7fe7540c27f0 con 0x7fe7540c22e0 2024-03-04T08:27:14.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe752000640 1 -- 172.21.15.113:0/2723488485 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe738003530 con 0x7fe7540c22e0 2024-03-04T08:27:14.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe752a00640 1 -- 172.21.15.113:0/2723488485 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe75415b900 msgr2=0x7fe7540731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:14.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe752a00640 1 --2- 172.21.15.113:0/2723488485 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe75415b900 0x7fe7540731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:14.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe752a00640 1 -- 172.21.15.113:0/2723488485 >> v1:172.21.15.113:6789/0 conn(0x7fe7540c22e0 legacy=0x7fe75415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:14.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe752000640 1 -- 172.21.15.113:0/2723488485 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 349177035 0 0) 0x7fe738003530 con 0x7fe7540c22e0 2024-03-04T08:27:14.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.247+0000 7fe752a00640 1 -- 172.21.15.113:0/2723488485 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754070c60 con 0x7fe75414e4c0 2024-03-04T08:27:14.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe752a00640 1 --2- 172.21.15.113:0/2723488485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe75414e4c0 0x7fe754072c80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe744009d90 tx=0x7fe74401f080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84421e1f1088e52b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:14.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe752000640 1 -- 172.21.15.113:0/2723488485 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7440b5070 con 0x7fe75414e4c0 2024-03-04T08:27:14.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe752000640 1 -- 172.21.15.113:0/2723488485 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7440a8bd0 con 0x7fe75414e4c0 2024-03-04T08:27:14.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe752000640 1 -- 172.21.15.113:0/2723488485 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c001680 con 0x7fe75414e4c0 2024-03-04T08:27:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe759c00640 1 -- 172.21.15.113:0/2723488485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe75414e4c0 msgr2=0x7fe754072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe759c00640 1 --2- 172.21.15.113:0/2723488485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe75414e4c0 0x7fe754072c80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe744009d90 tx=0x7fe74401f080 comp rx=0 tx=0).stop 2024-03-04T08:27:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe759c00640 1 -- 172.21.15.113:0/2723488485 shutdown_connections 2024-03-04T08:27:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe759c00640 1 --2- 172.21.15.113:0/2723488485 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe75415b900 0x7fe7540731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe759c00640 1 --2- 172.21.15.113:0/2723488485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe75414e4c0 0x7fe754072c80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe759c00640 1 -- 172.21.15.113:0/2723488485 >> 172.21.15.113:0/2723488485 conn(0x7fe75406e820 msgr2=0x7fe75406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe759c00640 1 -- 172.21.15.113:0/2723488485 shutdown_connections 2024-03-04T08:27:14.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.251+0000 7fe759c00640 1 -- 172.21.15.113:0/2723488485 wait complete. 2024-03-04T08:27:14.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 Processor -- start 2024-03-04T08:27:14.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 -- start start 2024-03-04T08:27:14.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7540c22e0 0x7fe75414ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:14.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe75414e4c0 0x7fe75414f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75415b900 0x7fe754159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe754071500 con 0x7fe75415b900 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe754071220 con 0x7fe75414e4c0 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75414e8a0 con 0x7fe7540c22e0 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe753400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75415b900 0x7fe754159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe753400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75415b900 0x7fe754159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44004/0 (socket says 172.21.15.113:44004) 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe753400640 1 -- 172.21.15.113:0/1485365637 learned_addr learned my addr 172.21.15.113:0/1485365637 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe752a00640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7540c22e0 0x7fe75414ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe74a000640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe75414e4c0 0x7fe75414f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe753400640 1 -- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7540c22e0 msgr2=0x7fe75414ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe753400640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7540c22e0 0x7fe75414ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe753400640 1 -- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe75414e4c0 msgr2=0x7fe75414f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:14.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe753400640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe75414e4c0 0x7fe75414f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:14.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe753400640 1 -- 172.21.15.113:0/1485365637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754152760 con 0x7fe75415b900 2024-03-04T08:27:14.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe753400640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75415b900 0x7fe754159df0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00e880 tx=0x7fe74c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:14.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe74b400640 1 -- 172.21.15.113:0/1485365637 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c017070 con 0x7fe75415b900 2024-03-04T08:27:14.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe74b400640 1 -- 172.21.15.113:0/1485365637 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c00c8e0 con 0x7fe75415b900 2024-03-04T08:27:14.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe74b400640 1 -- 172.21.15.113:0/1485365637 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c00ccb0 con 0x7fe75415b900 2024-03-04T08:27:14.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 -- 172.21.15.113:0/1485365637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe754152480 con 0x7fe75415b900 2024-03-04T08:27:14.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 -- 172.21.15.113:0/1485365637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe754153ab0 con 0x7fe75415b900 2024-03-04T08:27:14.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.255+0000 7fe759c00640 1 -- 172.21.15.113:0/1485365637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe754075c00 con 0x7fe75415b900 2024-03-04T08:27:14.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.259+0000 7fe74b400640 1 -- 172.21.15.113:0/1485365637 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe74c01a070 con 0x7fe75415b900 2024-03-04T08:27:14.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.259+0000 7fe74b400640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe730066f90 0x7fe730069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:14.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.259+0000 7fe74b400640 1 -- 172.21.15.113:0/1485365637 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fe74c085980 con 0x7fe75415b900 2024-03-04T08:27:14.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.259+0000 7fe752a00640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe730066f90 0x7fe730069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:14.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.259+0000 7fe752a00640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe730066f90 0x7fe730069450 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe7440b2000 tx=0x7fe7440aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:14.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.263+0000 7fe74b400640 1 -- 172.21.15.113:0/1485365637 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe74c055890 con 0x7fe75415b900 2024-03-04T08:27:14.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 -- 172.21.15.113:0/1485365637 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe754155190 con 0x7fe75415b900 2024-03-04T08:27:14.448 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe74b400640 1 -- 172.21.15.113:0/1485365637 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fe74c059930 con 0x7fe75415b900 2024-03-04T08:27:14.448 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:14.448 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:14.448 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:14.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 -- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe730066f90 msgr2=0x7fe730069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:14.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe730066f90 0x7fe730069450 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe7440b2000 tx=0x7fe7440aa070 comp rx=0 tx=0).stop 2024-03-04T08:27:14.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 -- 172.21.15.113:0/1485365637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75415b900 msgr2=0x7fe754159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:14.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75415b900 0x7fe754159df0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00e880 tx=0x7fe74c00ed40 comp rx=0 tx=0).stop 2024-03-04T08:27:14.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 -- 172.21.15.113:0/1485365637 shutdown_connections 2024-03-04T08:27:14.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe730066f90 0x7fe730069450 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:14.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe75415b900 0x7fe754159df0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:14.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe75414e4c0 0x7fe75414f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:14.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 --2- 172.21.15.113:0/1485365637 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe7540c22e0 0x7fe75414ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:14.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 -- 172.21.15.113:0/1485365637 >> 172.21.15.113:0/1485365637 conn(0x7fe75406e820 msgr2=0x7fe75406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:14.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 -- 172.21.15.113:0/1485365637 shutdown_connections 2024-03-04T08:27:14.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:14.451+0000 7fe759c00640 1 -- 172.21.15.113:0/1485365637 wait complete. 2024-03-04T08:27:14.469 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:27:14.470 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:27:15.776 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:27:15.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:27:16.471 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:16.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d0a400640 1 Processor -- start 2024-03-04T08:27:16.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d0a400640 1 -- start start 2024-03-04T08:27:16.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d0a400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5d041527f0 0x7f5d04070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:16.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d0a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d040c8200 0x7f5d04072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:16.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d0a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d04152bd0 0x7f5d041587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:16.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d0a400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d04071df0 con 0x7f5d04152bd0 2024-03-04T08:27:16.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d0a400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d041597b0 con 0x7f5d041527f0 2024-03-04T08:27:16.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d0a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d041591f0 con 0x7f5d040c8200 2024-03-04T08:27:16.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d03e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d04152bd0 0x7f5d041587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:16.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d03e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d04152bd0 0x7f5d041587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44008/0 (socket says 172.21.15.113:44008) 2024-03-04T08:27:16.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d03e00640 1 -- 172.21.15.113:0/3459064642 learned_addr learned my addr 172.21.15.113:0/3459064642 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:16.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d03400640 1 --2- 172.21.15.113:0/3459064642 >> v2:172.21.15.136:3300/0 conn(0x7f5d041527f0 0x7f5d04070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d02a00640 1 --2- 172.21.15.113:0/3459064642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d040c8200 0x7f5d04072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d03e00640 1 -- 172.21.15.113:0/3459064642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d040c8200 msgr2=0x7f5d04072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d03e00640 1 --2- 172.21.15.113:0/3459064642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d040c8200 0x7f5d04072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d03e00640 1 -- 172.21.15.113:0/3459064642 >> v2:172.21.15.136:3300/0 conn(0x7f5d041527f0 msgr2=0x7f5d04070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d03e00640 1 --2- 172.21.15.113:0/3459064642 >> v2:172.21.15.136:3300/0 conn(0x7f5d041527f0 0x7f5d04070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.623+0000 7f5d03e00640 1 -- 172.21.15.113:0/3459064642 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d041594d0 con 0x7f5d04152bd0 2024-03-04T08:27:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d03e00640 1 --2- 172.21.15.113:0/3459064642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d04152bd0 0x7f5d041587d0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f5d0406f400 tx=0x7f5cfc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975711844d244495 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d02000640 1 -- 172.21.15.113:0/3459064642 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc0a5760 con 0x7f5d04152bd0 2024-03-04T08:27:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d02000640 1 -- 172.21.15.113:0/3459064642 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc0a5900 con 0x7f5d04152bd0 2024-03-04T08:27:16.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d02000640 1 -- 172.21.15.113:0/3459064642 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc0a5cd0 con 0x7f5d04152bd0 2024-03-04T08:27:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d0a400640 1 -- 172.21.15.113:0/3459064642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d04152bd0 msgr2=0x7f5d041587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d0a400640 1 --2- 172.21.15.113:0/3459064642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d04152bd0 0x7f5d041587d0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f5d0406f400 tx=0x7f5cfc01c9f0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d0a400640 1 -- 172.21.15.113:0/3459064642 shutdown_connections 2024-03-04T08:27:16.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d0a400640 1 --2- 172.21.15.113:0/3459064642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d04152bd0 0x7f5d041587d0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d0a400640 1 --2- 172.21.15.113:0/3459064642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d040c8200 0x7f5d04072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d0a400640 1 --2- 172.21.15.113:0/3459064642 >> v2:172.21.15.136:3300/0 conn(0x7f5d041527f0 0x7f5d04070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d0a400640 1 -- 172.21.15.113:0/3459064642 >> 172.21.15.113:0/3459064642 conn(0x7f5d0406e820 msgr2=0x7f5d0406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d0a400640 1 -- 172.21.15.113:0/3459064642 shutdown_connections 2024-03-04T08:27:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.627+0000 7f5d0a400640 1 -- 172.21.15.113:0/3459064642 wait complete. 2024-03-04T08:27:16.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 Processor -- start 2024-03-04T08:27:16.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 -- start start 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d040c8200 0x7f5d041f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d041527f0 0x7f5d0414fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d04152bd0 0x7f5d04150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0415bf40 con 0x7f5d040c8200 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0415bdc0 con 0x7f5d04152bd0 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0415c0c0 con 0x7f5d041527f0 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d03400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d040c8200 0x7f5d041f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d03400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d040c8200 0x7f5d041f79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44010/0 (socket says 172.21.15.113:44010) 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d03400640 1 -- 172.21.15.113:0/2147910401 learned_addr learned my addr 172.21.15.113:0/2147910401 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d02a00640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d041527f0 0x7f5d0414fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d03e00640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d04152bd0 0x7f5d04150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:16.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d03400640 1 -- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d041527f0 msgr2=0x7f5d0414fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d03400640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d041527f0 0x7f5d0414fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d03400640 1 -- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d04152bd0 msgr2=0x7f5d04150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d03400640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d04152bd0 0x7f5d04150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d03400640 1 -- 172.21.15.113:0/2147910401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d041589f0 con 0x7f5d040c8200 2024-03-04T08:27:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d03400640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d040c8200 0x7f5d041f79c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400e960 tx=0x7f5cf400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5cfb400640 1 -- 172.21.15.113:0/2147910401 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4017070 con 0x7f5d040c8200 2024-03-04T08:27:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5cfb400640 1 -- 172.21.15.113:0/2147910401 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf400ca00 con 0x7f5d040c8200 2024-03-04T08:27:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 -- 172.21.15.113:0/2147910401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d04158710 con 0x7f5d040c8200 2024-03-04T08:27:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5cfb400640 1 -- 172.21.15.113:0/2147910401 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400cdb0 con 0x7f5d040c8200 2024-03-04T08:27:16.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 -- 172.21.15.113:0/2147910401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d041f74a0 con 0x7f5d040c8200 2024-03-04T08:27:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.631+0000 7f5d0a400640 1 -- 172.21.15.113:0/2147910401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d04075c00 con 0x7f5d040c8200 2024-03-04T08:27:16.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.635+0000 7f5cfb400640 1 -- 172.21.15.113:0/2147910401 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cf401a070 con 0x7f5d040c8200 2024-03-04T08:27:16.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.635+0000 7f5cfb400640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce4066f60 0x7f5ce4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:16.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.635+0000 7f5cfb400640 1 -- 172.21.15.113:0/2147910401 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f5cf4085ba0 con 0x7f5d040c8200 2024-03-04T08:27:16.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.635+0000 7f5d02a00640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce4066f60 0x7f5ce4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:16.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.635+0000 7f5d02a00640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce4066f60 0x7f5ce4069420 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5d041f17e0 tx=0x7f5cf0009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:16.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.639+0000 7f5cfb400640 1 -- 172.21.15.113:0/2147910401 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5cf4055990 con 0x7f5d040c8200 2024-03-04T08:27:16.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 -- 172.21.15.113:0/2147910401 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d040c6820 con 0x7f5d040c8200 2024-03-04T08:27:16.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5cfb400640 1 -- 172.21.15.113:0/2147910401 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f5cf4059a30 con 0x7f5d040c8200 2024-03-04T08:27:16.824 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:16.824 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:16.824 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 -- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce4066f60 msgr2=0x7f5ce4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce4066f60 0x7f5ce4069420 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5d041f17e0 tx=0x7f5cf0009190 comp rx=0 tx=0).stop 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 -- 172.21.15.113:0/2147910401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d040c8200 msgr2=0x7f5d041f79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d040c8200 0x7f5d041f79c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400e960 tx=0x7f5cf400ee20 comp rx=0 tx=0).stop 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 -- 172.21.15.113:0/2147910401 shutdown_connections 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d04152bd0 0x7f5d04150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5d041527f0 0x7f5d0414fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5ce4066f60 0x7f5ce4069420 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 --2- 172.21.15.113:0/2147910401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d040c8200 0x7f5d041f79c0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 -- 172.21.15.113:0/2147910401 >> 172.21.15.113:0/2147910401 conn(0x7f5d0406e820 msgr2=0x7f5d04151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:16.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 -- 172.21.15.113:0/2147910401 shutdown_connections 2024-03-04T08:27:16.827 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:16.827+0000 7f5d0a400640 1 -- 172.21.15.113:0/2147910401 wait complete. 2024-03-04T08:27:16.845 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:18.846 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:18.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeb1c00640 1 Processor -- start 2024-03-04T08:27:18.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeb1c00640 1 -- start start 2024-03-04T08:27:18.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeb1c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac0c08d0 0x7efeac06b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:18.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeb1c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7efeac15ad30 0x7efeac15b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:18.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeb1c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeac15efd0 con 0x7efeac155100 2024-03-04T08:27:18.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeb1c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeac0c8890 con 0x7efeac15ad30 2024-03-04T08:27:18.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeb1c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0c7ef0 con 0x7efeac0c08d0 2024-03-04T08:27:18.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeaaa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7efeac155100 0x7efeac0cd0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38892/0 (socket says 172.21.15.113:38892) 2024-03-04T08:27:18.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeaaa00640 1 -- 172.21.15.113:0/3320464796 learned_addr learned my addr 172.21.15.113:0/3320464796 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:18.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeab400640 1 --2- 172.21.15.113:0/3320464796 >> v2:172.21.15.136:3300/0 conn(0x7efeac15ad30 0x7efeac15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:18.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efea2000640 1 --2- 172.21.15.113:0/3320464796 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac0c08d0 0x7efeac06b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:18.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeaa000640 1 -- 172.21.15.113:0/3320464796 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897692903 0 0) 0x7efeac15efd0 con 0x7efeac155100 2024-03-04T08:27:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeaa000640 1 -- 172.21.15.113:0/3320464796 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe90003530 con 0x7efeac155100 2024-03-04T08:27:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeaa000640 1 -- 172.21.15.113:0/3320464796 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3840416187 0 0) 0x7efe90003530 con 0x7efeac155100 2024-03-04T08:27:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeab400640 1 -- 172.21.15.113:0/3320464796 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac0c08d0 msgr2=0x7efeac06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeab400640 1 --2- 172.21.15.113:0/3320464796 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac0c08d0 0x7efeac06b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeab400640 1 -- 172.21.15.113:0/3320464796 >> v1:172.21.15.113:6789/0 conn(0x7efeac155100 legacy=0x7efeac0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:18.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.995+0000 7efeab400640 1 -- 172.21.15.113:0/3320464796 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac0c81d0 con 0x7efeac15ad30 2024-03-04T08:27:18.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeab400640 1 --2- 172.21.15.113:0/3320464796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeac15ad30 0x7efeac15b140 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efea40097c0 tx=0x7efea401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fef145b4a86d9cf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:18.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeaa000640 1 -- 172.21.15.113:0/3320464796 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea40b0070 con 0x7efeac15ad30 2024-03-04T08:27:18.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeaa000640 1 -- 172.21.15.113:0/3320464796 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea40ab400 con 0x7efeac15ad30 2024-03-04T08:27:18.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeaa000640 1 -- 172.21.15.113:0/3320464796 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c001a00 con 0x7efeac15ad30 2024-03-04T08:27:18.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeb1c00640 1 -- 172.21.15.113:0/3320464796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeac15ad30 msgr2=0x7efeac15b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:18.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeb1c00640 1 --2- 172.21.15.113:0/3320464796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeac15ad30 0x7efeac15b140 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efea40097c0 tx=0x7efea401c3c0 comp rx=0 tx=0).stop 2024-03-04T08:27:18.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeb1c00640 1 -- 172.21.15.113:0/3320464796 shutdown_connections 2024-03-04T08:27:18.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeb1c00640 1 --2- 172.21.15.113:0/3320464796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeac15ad30 0x7efeac15b140 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:18.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeb1c00640 1 --2- 172.21.15.113:0/3320464796 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac0c08d0 0x7efeac06b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:18.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeb1c00640 1 -- 172.21.15.113:0/3320464796 >> 172.21.15.113:0/3320464796 conn(0x7efeac06deb0 msgr2=0x7efeac06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:18.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeb1c00640 1 -- 172.21.15.113:0/3320464796 shutdown_connections 2024-03-04T08:27:18.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:18.999+0000 7efeb1c00640 1 -- 172.21.15.113:0/3320464796 wait complete. 2024-03-04T08:27:18.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeb1c00640 1 Processor -- start 2024-03-04T08:27:18.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeb1c00640 1 -- start start 2024-03-04T08:27:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeb1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efeac0c08d0 0x7efeac0c1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeb1c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac155100 0x7efeac15d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeb1c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeac15ad30 0x7efeac14d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeb1c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac15ed30 con 0x7efeac0c08d0 2024-03-04T08:27:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeb1c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac15ebb0 con 0x7efeac15ad30 2024-03-04T08:27:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeb1c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeac0cd7d0 con 0x7efeac155100 2024-03-04T08:27:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeaaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efeac0c08d0 0x7efeac0c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeaaa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efeac0c08d0 0x7efeac0c1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39598/0 (socket says 172.21.15.113:39598) 2024-03-04T08:27:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeaaa00640 1 -- 172.21.15.113:0/3618006242 learned_addr learned my addr 172.21.15.113:0/3618006242 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:19.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efea2000640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac155100 0x7efeac15d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeab400640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeac15ad30 0x7efeac14d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeaaa00640 1 -- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac155100 msgr2=0x7efeac15d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeaaa00640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac155100 0x7efeac15d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeaaa00640 1 -- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeac15ad30 msgr2=0x7efeac14d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeaaa00640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeac15ad30 0x7efeac14d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeaaa00640 1 -- 172.21.15.113:0/3618006242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeac0c6610 con 0x7efeac0c08d0 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efea2000640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac155100 0x7efeac15d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeaaa00640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efeac0c08d0 0x7efeac0c1600 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7efe9c001f60 tx=0x7efe9c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efea3400640 1 -- 172.21.15.113:0/3618006242 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c017070 con 0x7efeac0c08d0 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efea3400640 1 -- 172.21.15.113:0/3618006242 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c012400 con 0x7efeac0c08d0 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efea3400640 1 -- 172.21.15.113:0/3618006242 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0127d0 con 0x7efeac0c08d0 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeb1c00640 1 -- 172.21.15.113:0/3618006242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac0c58f0 con 0x7efeac0c08d0 2024-03-04T08:27:19.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.003+0000 7efeb1c00640 1 -- 172.21.15.113:0/3618006242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac14cb00 con 0x7efeac0c08d0 2024-03-04T08:27:19.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.007+0000 7efeb1c00640 1 -- 172.21.15.113:0/3618006242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac075c00 con 0x7efeac0c08d0 2024-03-04T08:27:19.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.007+0000 7efea3400640 1 -- 172.21.15.113:0/3618006242 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe9c01a070 con 0x7efeac0c08d0 2024-03-04T08:27:19.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.007+0000 7efea3400640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efe88066f90 0x7efe88069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:19.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.007+0000 7efea3400640 1 -- 172.21.15.113:0/3618006242 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7efe9c085ab0 con 0x7efeac0c08d0 2024-03-04T08:27:19.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.007+0000 7efea2000640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efe88066f90 0x7efe88069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:19.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.007+0000 7efea2000640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efe88066f90 0x7efe88069450 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7efeac0ce120 tx=0x7efe94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:19.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.011+0000 7efea3400640 1 -- 172.21.15.113:0/3618006242 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efe9c056010 con 0x7efeac0c08d0 2024-03-04T08:27:19.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.195+0000 7efeb1c00640 1 -- 172.21.15.113:0/3618006242 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efeac15aa80 con 0x7efeac0c08d0 2024-03-04T08:27:19.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efea3400640 1 -- 172.21.15.113:0/3618006242 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7efe9c055910 con 0x7efeac0c08d0 2024-03-04T08:27:19.195 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:19.195 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:19.196 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 -- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efe88066f90 msgr2=0x7efe88069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efe88066f90 0x7efe88069450 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7efeac0ce120 tx=0x7efe94008040 comp rx=0 tx=0).stop 2024-03-04T08:27:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 -- 172.21.15.113:0/3618006242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efeac0c08d0 msgr2=0x7efeac0c1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efeac0c08d0 0x7efeac0c1600 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7efe9c001f60 tx=0x7efe9c00a430 comp rx=0 tx=0).stop 2024-03-04T08:27:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 -- 172.21.15.113:0/3618006242 shutdown_connections 2024-03-04T08:27:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efeac15ad30 0x7efeac14d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:19.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7efeac155100 0x7efeac15d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7efe88066f90 0x7efe88069450 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 --2- 172.21.15.113:0/3618006242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efeac0c08d0 0x7efeac0c1600 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 -- 172.21.15.113:0/3618006242 >> 172.21.15.113:0/3618006242 conn(0x7efeac06deb0 msgr2=0x7efeac06e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 -- 172.21.15.113:0/3618006242 shutdown_connections 2024-03-04T08:27:19.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:19.199+0000 7efeb1c00640 1 -- 172.21.15.113:0/3618006242 wait complete. 2024-03-04T08:27:19.217 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:21.218 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:21.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b0d600640 1 Processor -- start 2024-03-04T08:27:21.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b0d600640 1 -- start start 2024-03-04T08:27:21.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b0d600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4b0814e4c0 0x7f4b08072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:21.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b0d600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4b0815b900 0x7f4b080731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:21.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b0d600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b080c27f0 con 0x7f4b080c22e0 2024-03-04T08:27:21.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b0d600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0815ad60 con 0x7f4b0815b900 2024-03-04T08:27:21.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b0d600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b080c2a90 con 0x7f4b0814e4c0 2024-03-04T08:27:21.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b07400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4b0815b900 0x7f4b080731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:21.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b07400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4b0815b900 0x7f4b080731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39606/0 (socket says 172.21.15.113:39606) 2024-03-04T08:27:21.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b07400640 1 -- 172.21.15.113:0/1377721055 learned_addr learned my addr 172.21.15.113:0/1377721055 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:21.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4afe000640 1 --1- 172.21.15.113:0/1377721055 >> v1:172.21.15.113:6789/0 conn(0x7f4b080c22e0 0x7f4b0815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38900/0 (socket says 172.21.15.113:38900) 2024-03-04T08:27:21.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.367+0000 7f4b06a00640 1 --2- 172.21.15.113:0/1377721055 >> v2:172.21.15.136:3300/0 conn(0x7f4b0814e4c0 0x7f4b08072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:21.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b07400640 1 -- 172.21.15.113:0/1377721055 >> v2:172.21.15.136:3300/0 conn(0x7f4b0814e4c0 msgr2=0x7f4b08072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:21.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b07400640 1 --2- 172.21.15.113:0/1377721055 >> v2:172.21.15.136:3300/0 conn(0x7f4b0814e4c0 0x7f4b08072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:21.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b07400640 1 -- 172.21.15.113:0/1377721055 >> v1:172.21.15.113:6789/0 conn(0x7f4b080c22e0 legacy=0x7f4b0815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:21.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b07400640 1 -- 172.21.15.113:0/1377721055 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08070c60 con 0x7f4b0815b900 2024-03-04T08:27:21.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b07400640 1 --2- 172.21.15.113:0/1377721055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 0x7f4b080731c0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f4b0806f690 tx=0x7f4b0001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb579cebfcb3572 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:21.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b06000640 1 -- 172.21.15.113:0/1377721055 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b000b4070 con 0x7f4b0815b900 2024-03-04T08:27:21.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b06000640 1 -- 172.21.15.113:0/1377721055 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b000af400 con 0x7f4b0815b900 2024-03-04T08:27:21.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b06000640 1 -- 172.21.15.113:0/1377721055 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b000af7d0 con 0x7f4b0815b900 2024-03-04T08:27:21.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b0d600640 1 -- 172.21.15.113:0/1377721055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 msgr2=0x7f4b080731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:21.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b0d600640 1 --2- 172.21.15.113:0/1377721055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 0x7f4b080731c0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f4b0806f690 tx=0x7f4b0001f1e0 comp rx=0 tx=0).stop 2024-03-04T08:27:21.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b0d600640 1 -- 172.21.15.113:0/1377721055 shutdown_connections 2024-03-04T08:27:21.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b0d600640 1 --2- 172.21.15.113:0/1377721055 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 0x7f4b080731c0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:21.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b0d600640 1 --2- 172.21.15.113:0/1377721055 >> v2:172.21.15.136:3300/0 conn(0x7f4b0814e4c0 0x7f4b08072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:21.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b0d600640 1 -- 172.21.15.113:0/1377721055 >> 172.21.15.113:0/1377721055 conn(0x7f4b0806e820 msgr2=0x7f4b0806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:21.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b0d600640 1 -- 172.21.15.113:0/1377721055 shutdown_connections 2024-03-04T08:27:21.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.371+0000 7f4b0d600640 1 -- 172.21.15.113:0/1377721055 wait complete. 2024-03-04T08:27:21.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b0d600640 1 Processor -- start 2024-03-04T08:27:21.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b0d600640 1 -- start start 2024-03-04T08:27:21.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b0d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b080c22e0 0x7f4b081458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:21.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b0d600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b0814e4c0 0x7f4b08152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:21.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b0d600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 0x7f4b081525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:21.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b0d600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08071500 con 0x7f4b0815b900 2024-03-04T08:27:21.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b0d600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08071220 con 0x7f4b080c22e0 2024-03-04T08:27:21.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b0d600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0814e8a0 con 0x7f4b0814e4c0 2024-03-04T08:27:21.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b07400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 0x7f4b081525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:21.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b07400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 0x7f4b081525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39622/0 (socket says 172.21.15.113:39622) 2024-03-04T08:27:21.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b07400640 1 -- 172.21.15.113:0/2318598553 learned_addr learned my addr 172.21.15.113:0/2318598553 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:21.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b06a00640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b080c22e0 0x7f4b081458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:21.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4afe000640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b0814e4c0 0x7f4b08152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:21.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b07400640 1 -- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b0814e4c0 msgr2=0x7f4b08152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:21.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b07400640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b0814e4c0 0x7f4b08152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:21.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b07400640 1 -- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b080c22e0 msgr2=0x7f4b081458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:21.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b07400640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b080c22e0 0x7f4b081458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:21.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b07400640 1 -- 172.21.15.113:0/2318598553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b081543a0 con 0x7f4b0815b900 2024-03-04T08:27:21.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4b07400640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 0x7f4b081525a0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f4b0001ce10 tx=0x7f4b0001bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:21.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4aff400640 1 -- 172.21.15.113:0/2318598553 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b000b4040 con 0x7f4b0815b900 2024-03-04T08:27:21.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4aff400640 1 -- 172.21.15.113:0/2318598553 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b000afd00 con 0x7f4b0815b900 2024-03-04T08:27:21.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.375+0000 7f4aff400640 1 -- 172.21.15.113:0/2318598553 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00009510 con 0x7f4b0815b900 2024-03-04T08:27:21.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.379+0000 7f4b0d600640 1 -- 172.21.15.113:0/2318598553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b08154640 con 0x7f4b0815b900 2024-03-04T08:27:21.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.379+0000 7f4b0d600640 1 -- 172.21.15.113:0/2318598553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b08154c00 con 0x7f4b0815b900 2024-03-04T08:27:21.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.379+0000 7f4b0d600640 1 -- 172.21.15.113:0/2318598553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b08075c00 con 0x7f4b0815b900 2024-03-04T08:27:21.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.379+0000 7f4aff400640 1 -- 172.21.15.113:0/2318598553 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b000c3020 con 0x7f4b0815b900 2024-03-04T08:27:21.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.379+0000 7f4aff400640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ae4066f60 0x7f4ae4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:21.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.379+0000 7f4aff400640 1 -- 172.21.15.113:0/2318598553 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f4b00123650 con 0x7f4b0815b900 2024-03-04T08:27:21.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.379+0000 7f4b06a00640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ae4066f60 0x7f4ae4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:21.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.383+0000 7f4b06a00640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ae4066f60 0x7f4ae4069420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009b60 tx=0x7f4af8009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:21.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.383+0000 7f4aff400640 1 -- 172.21.15.113:0/2318598553 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4b000ca020 con 0x7f4b0815b900 2024-03-04T08:27:21.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.571+0000 7f4b0d600640 1 -- 172.21.15.113:0/2318598553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b08155190 con 0x7f4b0815b900 2024-03-04T08:27:21.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.571+0000 7f4aff400640 1 -- 172.21.15.113:0/2318598553 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f4b00018d40 con 0x7f4b0815b900 2024-03-04T08:27:21.570 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:21.570 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:21.570 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:21.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 -- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ae4066f60 msgr2=0x7f4ae4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:21.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ae4066f60 0x7f4ae4069420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009b60 tx=0x7f4af8009190 comp rx=0 tx=0).stop 2024-03-04T08:27:21.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 -- 172.21.15.113:0/2318598553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 msgr2=0x7f4b081525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:21.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 0x7f4b081525a0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f4b0001ce10 tx=0x7f4b0001bf80 comp rx=0 tx=0).stop 2024-03-04T08:27:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 -- 172.21.15.113:0/2318598553 shutdown_connections 2024-03-04T08:27:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4ae4066f60 0x7f4ae4069420 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b0815b900 0x7f4b081525a0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4b0814e4c0 0x7f4b08152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 --2- 172.21.15.113:0/2318598553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b080c22e0 0x7f4b081458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 -- 172.21.15.113:0/2318598553 >> 172.21.15.113:0/2318598553 conn(0x7f4b0806e820 msgr2=0x7f4b080c67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 -- 172.21.15.113:0/2318598553 shutdown_connections 2024-03-04T08:27:21.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:21.575+0000 7f4b0d600640 1 -- 172.21.15.113:0/2318598553 wait complete. 2024-03-04T08:27:21.592 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:23.593 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:23.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.743+0000 7fef87600640 1 Processor -- start 2024-03-04T08:27:23.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.743+0000 7fef87600640 1 -- start start 2024-03-04T08:27:23.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef87600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fef800c22e0 0x7fef8015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:23.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef87600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fef800c2d80 0x7fef8015b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:23.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef87600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef800c27f0 con 0x7fef8014e4c0 2024-03-04T08:27:23.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef87600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8015ad60 con 0x7fef800c22e0 2024-03-04T08:27:23.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef87600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef800c2a90 con 0x7fef800c2d80 2024-03-04T08:27:23.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef77e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fef800c22e0 0x7fef8015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:23.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef84c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fef8014e4c0 0x7fef80072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38908/0 (socket says 172.21.15.113:38908) 2024-03-04T08:27:23.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef84c00640 1 -- 172.21.15.113:0/3652658994 learned_addr learned my addr 172.21.15.113:0/3652658994 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:23.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef85600640 1 --2- 172.21.15.113:0/3652658994 >> v2:172.21.15.136:3300/0 conn(0x7fef800c2d80 0x7fef8015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:23.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef77e00640 1 -- 172.21.15.113:0/3652658994 >> v2:172.21.15.136:3300/0 conn(0x7fef800c2d80 msgr2=0x7fef8015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:23.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef77e00640 1 --2- 172.21.15.113:0/3652658994 >> v2:172.21.15.136:3300/0 conn(0x7fef800c2d80 0x7fef8015b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:23.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef77e00640 1 -- 172.21.15.113:0/3652658994 >> v1:172.21.15.113:6789/0 conn(0x7fef8014e4c0 legacy=0x7fef80072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:23.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef77e00640 1 -- 172.21.15.113:0/3652658994 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef80070c60 con 0x7fef800c22e0 2024-03-04T08:27:23.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.747+0000 7fef7fe00640 1 -- 172.21.15.113:0/3652658994 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221558112 0 0) 0x7fef800c27f0 con 0x7fef8014e4c0 2024-03-04T08:27:23.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef77e00640 1 --2- 172.21.15.113:0/3652658994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef800c22e0 0x7fef8015b1f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fef800c2700 tx=0x7fef6801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a797e8a2adf71f92 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:23.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef7fe00640 1 -- 172.21.15.113:0/3652658994 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680b1070 con 0x7fef800c22e0 2024-03-04T08:27:23.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef7fe00640 1 -- 172.21.15.113:0/3652658994 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef680ac5f0 con 0x7fef800c22e0 2024-03-04T08:27:23.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef7fe00640 1 -- 172.21.15.113:0/3652658994 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef680ac9c0 con 0x7fef800c22e0 2024-03-04T08:27:23.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 -- 172.21.15.113:0/3652658994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef800c22e0 msgr2=0x7fef8015b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:23.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 --2- 172.21.15.113:0/3652658994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef800c22e0 0x7fef8015b1f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fef800c2700 tx=0x7fef6801c480 comp rx=0 tx=0).stop 2024-03-04T08:27:23.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 -- 172.21.15.113:0/3652658994 shutdown_connections 2024-03-04T08:27:23.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 --2- 172.21.15.113:0/3652658994 >> v2:172.21.15.136:3300/0 conn(0x7fef800c2d80 0x7fef8015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:23.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 --2- 172.21.15.113:0/3652658994 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef800c22e0 0x7fef8015b1f0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:23.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 -- 172.21.15.113:0/3652658994 >> 172.21.15.113:0/3652658994 conn(0x7fef8006e820 msgr2=0x7fef8006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:23.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 -- 172.21.15.113:0/3652658994 shutdown_connections 2024-03-04T08:27:23.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 -- 172.21.15.113:0/3652658994 wait complete. 2024-03-04T08:27:23.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 Processor -- start 2024-03-04T08:27:23.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 -- start start 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef800c22e0 0x7fef801f1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef800c2d80 0x7fef801f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef8014e4c0 0x7fef80146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef800aede0 con 0x7fef8014e4c0 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8014e8a0 con 0x7fef800c22e0 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef87600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef80073390 con 0x7fef800c2d80 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef85600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef8014e4c0 0x7fef80146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef85600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef8014e4c0 0x7fef80146030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39636/0 (socket says 172.21.15.113:39636) 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.751+0000 7fef85600640 1 -- 172.21.15.113:0/4214849147 learned_addr learned my addr 172.21.15.113:0/4214849147 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef77e00640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef800c2d80 0x7fef801f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef84c00640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef800c22e0 0x7fef801f1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef85600640 1 -- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef800c2d80 msgr2=0x7fef801f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef85600640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef800c2d80 0x7fef801f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef85600640 1 -- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef800c22e0 msgr2=0x7fef801f1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:23.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef85600640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef800c22e0 0x7fef801f1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:23.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef85600640 1 -- 172.21.15.113:0/4214849147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef80143bc0 con 0x7fef8014e4c0 2024-03-04T08:27:23.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef84c00640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef800c22e0 0x7fef801f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:23.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef77e00640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef800c2d80 0x7fef801f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:23.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef85600640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef8014e4c0 0x7fef80146030 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fef78007e60 tx=0x7fef7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:23.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef7d600640 1 -- 172.21.15.113:0/4214849147 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78014420 con 0x7fef8014e4c0 2024-03-04T08:27:23.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef7d600640 1 -- 172.21.15.113:0/4214849147 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef78019070 con 0x7fef8014e4c0 2024-03-04T08:27:23.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef87600640 1 -- 172.21.15.113:0/4214849147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef80143ea0 con 0x7fef8014e4c0 2024-03-04T08:27:23.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef7d600640 1 -- 172.21.15.113:0/4214849147 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78014820 con 0x7fef8014e4c0 2024-03-04T08:27:23.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef87600640 1 -- 172.21.15.113:0/4214849147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef80144460 con 0x7fef8014e4c0 2024-03-04T08:27:23.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef87600640 1 -- 172.21.15.113:0/4214849147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef801f19e0 con 0x7fef8014e4c0 2024-03-04T08:27:23.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef7d600640 1 -- 172.21.15.113:0/4214849147 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef780149c0 con 0x7fef8014e4c0 2024-03-04T08:27:23.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef7d600640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef5c066f90 0x7fef5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:23.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef7d600640 1 -- 172.21.15.113:0/4214849147 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fef78086450 con 0x7fef8014e4c0 2024-03-04T08:27:23.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.755+0000 7fef84c00640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef5c066f90 0x7fef5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:23.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.759+0000 7fef84c00640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef5c066f90 0x7fef5c069450 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fef70007fd0 tx=0x7fef70007c20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:23.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.759+0000 7fef7d600640 1 -- 172.21.15.113:0/4214849147 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fef801f19e0 con 0x7fef8014e4c0 2024-03-04T08:27:23.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 -- 172.21.15.113:0/4214849147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef80075c00 con 0x7fef8014e4c0 2024-03-04T08:27:23.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef7d600640 1 -- 172.21.15.113:0/4214849147 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fef78056890 con 0x7fef8014e4c0 2024-03-04T08:27:23.952 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:23.952 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:23.953 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 -- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef5c066f90 msgr2=0x7fef5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef5c066f90 0x7fef5c069450 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fef70007fd0 tx=0x7fef70007c20 comp rx=0 tx=0).stop 2024-03-04T08:27:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 -- 172.21.15.113:0/4214849147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef8014e4c0 msgr2=0x7fef80146030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef8014e4c0 0x7fef80146030 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fef78007e60 tx=0x7fef7800e4b0 comp rx=0 tx=0).stop 2024-03-04T08:27:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 -- 172.21.15.113:0/4214849147 shutdown_connections 2024-03-04T08:27:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fef5c066f90 0x7fef5c069450 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:23.954 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef8014e4c0 0x7fef80146030 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:23.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fef800c2d80 0x7fef801f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:23.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 --2- 172.21.15.113:0/4214849147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef800c22e0 0x7fef801f1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:23.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 -- 172.21.15.113:0/4214849147 >> 172.21.15.113:0/4214849147 conn(0x7fef8006e820 msgr2=0x7fef800c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:23.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 -- 172.21.15.113:0/4214849147 shutdown_connections 2024-03-04T08:27:23.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:23.955+0000 7fef87600640 1 -- 172.21.15.113:0/4214849147 wait complete. 2024-03-04T08:27:23.973 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:25.975 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.123+0000 7f1cfc400640 1 Processor -- start 2024-03-04T08:27:26.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.123+0000 7f1cfc400640 1 -- start start 2024-03-04T08:27:26.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cfc400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cf40c8d80 0x7f1cf415b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:26.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cfc400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf406c860 con 0x7f1cf40ce570 2024-03-04T08:27:26.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cfc400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf406f8d0 con 0x7f1cf4070120 2024-03-04T08:27:26.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cfc400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf406f310 con 0x7f1cf40c8d80 2024-03-04T08:27:26.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cf9000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1cf40c8d80 0x7f1cf415b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:26.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cf9a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1cf40ce570 0x7f1cf4154c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:38922/0 (socket says 172.21.15.113:38922) 2024-03-04T08:27:26.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cf9a00640 1 -- 172.21.15.113:0/745865626 learned_addr learned my addr 172.21.15.113:0/745865626 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:26.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cf9000640 1 -- 172.21.15.113:0/745865626 >> v1:172.21.15.136:6789/0 conn(0x7f1cf4070120 legacy=0x7f1cf41656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:26.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cf9000640 1 -- 172.21.15.113:0/745865626 >> v1:172.21.15.113:6789/0 conn(0x7f1cf40ce570 legacy=0x7f1cf4154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:26.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cf9000640 1 -- 172.21.15.113:0/745865626 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf406f5f0 con 0x7f1cf40c8d80 2024-03-04T08:27:26.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cf9000640 1 --2- 172.21.15.113:0/745865626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40c8d80 0x7f1cf415b150 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00097c0 tx=0x7f1ce001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b613861b9f479e67 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:26.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.127+0000 7f1cf3e00640 1 -- 172.21.15.113:0/745865626 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce00b5070 con 0x7f1cf40c8d80 2024-03-04T08:27:26.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cf3e00640 1 -- 172.21.15.113:0/745865626 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce00a8950 con 0x7f1cf40c8d80 2024-03-04T08:27:26.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cf3e00640 1 -- 172.21.15.113:0/745865626 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce00a8d20 con 0x7f1cf40c8d80 2024-03-04T08:27:26.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 -- 172.21.15.113:0/745865626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40c8d80 msgr2=0x7f1cf415b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:26.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 --2- 172.21.15.113:0/745865626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40c8d80 0x7f1cf415b150 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00097c0 tx=0x7f1ce001ec50 comp rx=0 tx=0).stop 2024-03-04T08:27:26.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 -- 172.21.15.113:0/745865626 shutdown_connections 2024-03-04T08:27:26.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 --2- 172.21.15.113:0/745865626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40c8d80 0x7f1cf415b150 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:26.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 -- 172.21.15.113:0/745865626 >> 172.21.15.113:0/745865626 conn(0x7f1cf4066db0 msgr2=0x7f1cf40671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:26.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 -- 172.21.15.113:0/745865626 shutdown_connections 2024-03-04T08:27:26.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 -- 172.21.15.113:0/745865626 wait complete. 2024-03-04T08:27:26.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 Processor -- start 2024-03-04T08:27:26.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 -- start start 2024-03-04T08:27:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cf4070120 0x7f1cf415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cf40c8d80 0x7f1cf40c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40ce570 0x7f1cf40c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf406fb80 con 0x7f1cf40ce570 2024-03-04T08:27:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf406c4f0 con 0x7f1cf4070120 2024-03-04T08:27:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfc400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf406fd00 con 0x7f1cf40c8d80 2024-03-04T08:27:26.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfa400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40ce570 0x7f1cf40c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfa400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40ce570 0x7f1cf40c65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:39660/0 (socket says 172.21.15.113:39660) 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfa400640 1 -- 172.21.15.113:0/2543218950 learned_addr learned my addr 172.21.15.113:0/2543218950 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cf9a00640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cf4070120 0x7f1cf415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cf9000640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cf40c8d80 0x7f1cf40c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfa400640 1 -- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cf40c8d80 msgr2=0x7f1cf40c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfa400640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cf40c8d80 0x7f1cf40c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfa400640 1 -- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cf4070120 msgr2=0x7f1cf415a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfa400640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cf4070120 0x7f1cf415a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cfa400640 1 -- 172.21.15.113:0/2543218950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf40c3e50 con 0x7f1cf40ce570 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.131+0000 7f1cf9000640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cf40c8d80 0x7f1cf40c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cfa400640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40ce570 0x7f1cf40c65f0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00c410 tx=0x7f1cec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cf1600640 1 -- 172.21.15.113:0/2543218950 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec017070 con 0x7f1cf40ce570 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cf1600640 1 -- 172.21.15.113:0/2543218950 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec003600 con 0x7f1cf40ce570 2024-03-04T08:27:26.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cf1600640 1 -- 172.21.15.113:0/2543218950 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0039f0 con 0x7f1cf40ce570 2024-03-04T08:27:26.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cfc400640 1 -- 172.21.15.113:0/2543218950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf41f08d0 con 0x7f1cf40ce570 2024-03-04T08:27:26.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cfc400640 1 -- 172.21.15.113:0/2543218950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf40c85a0 con 0x7f1cf40ce570 2024-03-04T08:27:26.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cfc400640 1 -- 172.21.15.113:0/2543218950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf4075c00 con 0x7f1cf40ce570 2024-03-04T08:27:26.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cf1600640 1 -- 172.21.15.113:0/2543218950 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cec01a070 con 0x7f1cf40ce570 2024-03-04T08:27:26.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cf1600640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1cd4066fb0 0x7f1cd4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:26.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cf1600640 1 -- 172.21.15.113:0/2543218950 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f1cec084de0 con 0x7f1cf40ce570 2024-03-04T08:27:26.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.135+0000 7f1cf9a00640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1cd4066fb0 0x7f1cd4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:26.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.139+0000 7f1cf9a00640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1cd4066fb0 0x7f1cd4069470 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80012f0 tx=0x7f1ce80019a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:26.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.139+0000 7f1cf1600640 1 -- 172.21.15.113:0/2543218950 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1cf41f08d0 con 0x7f1cf40ce570 2024-03-04T08:27:26.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.327+0000 7f1cfc400640 1 -- 172.21.15.113:0/2543218950 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cf4154470 con 0x7f1cf40ce570 2024-03-04T08:27:26.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.327+0000 7f1cf1600640 1 -- 172.21.15.113:0/2543218950 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f1cec054d70 con 0x7f1cf40ce570 2024-03-04T08:27:26.326 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:26.326 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:26.327 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:26.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 -- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1cd4066fb0 msgr2=0x7f1cd4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:26.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1cd4066fb0 0x7f1cd4069470 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80012f0 tx=0x7f1ce80019a0 comp rx=0 tx=0).stop 2024-03-04T08:27:26.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 -- 172.21.15.113:0/2543218950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40ce570 msgr2=0x7f1cf40c65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:26.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40ce570 0x7f1cf40c65f0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00c410 tx=0x7f1cec00c8d0 comp rx=0 tx=0).stop 2024-03-04T08:27:26.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 -- 172.21.15.113:0/2543218950 shutdown_connections 2024-03-04T08:27:26.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1cd4066fb0 0x7f1cd4069470 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:26.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1cf40ce570 0x7f1cf40c65f0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:26.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1cf40c8d80 0x7f1cf40c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:26.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 --2- 172.21.15.113:0/2543218950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1cf4070120 0x7f1cf415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:26.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 -- 172.21.15.113:0/2543218950 >> 172.21.15.113:0/2543218950 conn(0x7f1cf4066db0 msgr2=0x7f1cf40c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:26.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 -- 172.21.15.113:0/2543218950 shutdown_connections 2024-03-04T08:27:26.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:26.331+0000 7f1cfc400640 1 -- 172.21.15.113:0/2543218950 wait complete. 2024-03-04T08:27:26.348 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:27:26.348 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:27:28.350 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:28.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.499+0000 7f9002c00640 1 Processor -- start 2024-03-04T08:27:28.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.499+0000 7f9002c00640 1 -- start start 2024-03-04T08:27:28.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9002c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ffc0c22e0 0x7f8ffc15b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:28.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9002c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8ffc0c2d80 0x7f8ffc15b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:28.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9002c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ffc0c27f0 con 0x7f8ffc14e4c0 2024-03-04T08:27:28.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9002c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ffc15ad60 con 0x7f8ffc0c2d80 2024-03-04T08:27:28.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9002c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0c2a90 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9000c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8ffc0c2d80 0x7f8ffc15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9000c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f8ffc0c2d80 0x7f8ffc15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:36652/0 (socket says 172.21.15.113:36652) 2024-03-04T08:27:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9000c00640 1 -- 172.21.15.113:0/1400159221 learned_addr learned my addr 172.21.15.113:0/1400159221 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f8ff3400640 1 --2- 172.21.15.113:0/1400159221 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ffc0c22e0 0x7f8ffc15b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9000c00640 1 -- 172.21.15.113:0/1400159221 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ffc0c22e0 msgr2=0x7f8ffc15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9000c00640 1 --2- 172.21.15.113:0/1400159221 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ffc0c22e0 0x7f8ffc15b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9000c00640 1 -- 172.21.15.113:0/1400159221 >> v1:172.21.15.136:6789/0 conn(0x7f8ffc14e4c0 legacy=0x7f8ffc072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:28.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.503+0000 7f9000c00640 1 -- 172.21.15.113:0/1400159221 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc070c60 con 0x7f8ffc0c2d80 2024-03-04T08:27:28.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9000c00640 1 --2- 172.21.15.113:0/1400159221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ffc0c2d80 0x7f8ffc15b730 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009870 tx=0x7f8ff401ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b2f0e8b5013957 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:28.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f8ffb400640 1 -- 172.21.15.113:0/1400159221 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40b5070 con 0x7f8ffc0c2d80 2024-03-04T08:27:28.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f8ffb400640 1 -- 172.21.15.113:0/1400159221 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff40a8a50 con 0x7f8ffc0c2d80 2024-03-04T08:27:28.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f8ffb400640 1 -- 172.21.15.113:0/1400159221 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40a8e20 con 0x7f8ffc0c2d80 2024-03-04T08:27:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 -- 172.21.15.113:0/1400159221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ffc0c2d80 msgr2=0x7f8ffc15b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 --2- 172.21.15.113:0/1400159221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ffc0c2d80 0x7f8ffc15b730 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4009870 tx=0x7f8ff401ed50 comp rx=0 tx=0).stop 2024-03-04T08:27:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 -- 172.21.15.113:0/1400159221 shutdown_connections 2024-03-04T08:27:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 --2- 172.21.15.113:0/1400159221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ffc0c2d80 0x7f8ffc15b730 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 --2- 172.21.15.113:0/1400159221 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ffc0c22e0 0x7f8ffc15b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 -- 172.21.15.113:0/1400159221 >> 172.21.15.113:0/1400159221 conn(0x7f8ffc06e820 msgr2=0x7f8ffc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 -- 172.21.15.113:0/1400159221 shutdown_connections 2024-03-04T08:27:28.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 -- 172.21.15.113:0/1400159221 wait complete. 2024-03-04T08:27:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 Processor -- start 2024-03-04T08:27:28.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 -- start start 2024-03-04T08:27:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ffc0c22e0 0x7f8ffc1f14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ffc0c2d80 0x7f8ffc14b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:28.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ffc14e4c0 0x7f8ffc1497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:28.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc071500 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc071220 con 0x7f8ffc14e4c0 2024-03-04T08:27:28.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f9002c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc14e8a0 con 0x7f8ffc0c2d80 2024-03-04T08:27:28.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f8ffbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ffc0c22e0 0x7f8ffc1f14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:28.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f8ffbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ffc0c22e0 0x7f8ffc1f14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41984/0 (socket says 172.21.15.113:41984) 2024-03-04T08:27:28.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.507+0000 7f8ffbe00640 1 -- 172.21.15.113:0/1186369645 learned_addr learned my addr 172.21.15.113:0/1186369645 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:28.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ff3400640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ffc0c2d80 0x7f8ffc14b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:28.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f9000c00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ffc14e4c0 0x7f8ffc1497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:28.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ffbe00640 1 -- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ffc0c2d80 msgr2=0x7f8ffc14b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:28.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ffbe00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ffc0c2d80 0x7f8ffc14b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:28.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ffbe00640 1 -- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ffc14e4c0 msgr2=0x7f8ffc1497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:28.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ffbe00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ffc14e4c0 0x7f8ffc1497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:28.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ffbe00640 1 -- 172.21.15.113:0/1186369645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc1f1a20 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ffbe00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ffc0c22e0 0x7f8ffc1f14e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00c7c0 tx=0x7f8fec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:28.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ff8c00640 1 -- 172.21.15.113:0/1186369645 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec016020 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ff8c00640 1 -- 172.21.15.113:0/1186369645 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec006d60 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f9002c00640 1 -- 172.21.15.113:0/1186369645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc1f4ad0 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ff8c00640 1 -- 172.21.15.113:0/1186369645 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec003bc0 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f9002c00640 1 -- 172.21.15.113:0/1186369645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc1f0ce0 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f9002c00640 1 -- 172.21.15.113:0/1186369645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ffc1f0b10 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.511+0000 7f8ff8c00640 1 -- 172.21.15.113:0/1186369645 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fec00a850 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.515+0000 7f8ff8c00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8fd8066f60 0x7f8fd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:28.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.515+0000 7f8ff8c00640 1 -- 172.21.15.113:0/1186369645 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f8fec085720 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.515+0000 7f8ff3400640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8fd8066f60 0x7f8fd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:28.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.515+0000 7f8ff3400640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8fd8066f60 0x7f8fd8069420 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc14d020 tx=0x7f8fe4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:28.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.519+0000 7f8ff8c00640 1 -- 172.21.15.113:0/1186369645 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8fec020020 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 -- 172.21.15.113:0/1186369645 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ffc0c1260 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f8ff8c00640 1 -- 172.21.15.113:0/1186369645 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f8fec0555a0 con 0x7f8ffc0c22e0 2024-03-04T08:27:28.704 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:28.704 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:28.704 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 -- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8fd8066f60 msgr2=0x7f8fd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8fd8066f60 0x7f8fd8069420 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc14d020 tx=0x7f8fe4009190 comp rx=0 tx=0).stop 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 -- 172.21.15.113:0/1186369645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ffc0c22e0 msgr2=0x7f8ffc1f14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ffc0c22e0 0x7f8ffc1f14e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00c7c0 tx=0x7f8fec00cc80 comp rx=0 tx=0).stop 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 -- 172.21.15.113:0/1186369645 shutdown_connections 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ffc14e4c0 0x7f8ffc1497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8ffc0c2d80 0x7f8ffc14b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8fd8066f60 0x7f8fd8069420 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 --2- 172.21.15.113:0/1186369645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ffc0c22e0 0x7f8ffc1f14e0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 -- 172.21.15.113:0/1186369645 >> 172.21.15.113:0/1186369645 conn(0x7f8ffc06e820 msgr2=0x7f8ffc0c14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 -- 172.21.15.113:0/1186369645 shutdown_connections 2024-03-04T08:27:28.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:28.707+0000 7f9002c00640 1 -- 172.21.15.113:0/1186369645 wait complete. 2024-03-04T08:27:28.725 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:30.726 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:30.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f8600640 1 Processor -- start 2024-03-04T08:27:30.872 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f8600640 1 -- start start 2024-03-04T08:27:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f8600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1f0154d50 0x7fd1f00c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f8600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f006b6b0 0x7fd1f00cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f8600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f0072710 con 0x7fd1f006cde0 2024-03-04T08:27:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f8600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f0072c90 con 0x7fd1f0154d50 2024-03-04T08:27:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f8600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f006b220 con 0x7fd1f006b6b0 2024-03-04T08:27:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f5c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1f0154d50 0x7fd1f00c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f5c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd1f0154d50 0x7fd1f00c20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41986/0 (socket says 172.21.15.113:41986) 2024-03-04T08:27:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f6600640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd1f006cde0 0x7fd1f015e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48578/0 (socket says 172.21.15.113:48578) 2024-03-04T08:27:30.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f5c00640 1 -- 172.21.15.113:0/3809653432 learned_addr learned my addr 172.21.15.113:0/3809653432 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:30.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f5200640 1 --2- 172.21.15.113:0/3809653432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f006b6b0 0x7fd1f00cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:30.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f5c00640 1 -- 172.21.15.113:0/3809653432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f006b6b0 msgr2=0x7fd1f00cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:30.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f5c00640 1 --2- 172.21.15.113:0/3809653432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f006b6b0 0x7fd1f00cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:30.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f5c00640 1 -- 172.21.15.113:0/3809653432 >> v1:172.21.15.113:6789/0 conn(0x7fd1f006cde0 legacy=0x7fd1f015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:30.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1efe00640 1 -- 172.21.15.113:0/3809653432 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583398504 0 0) 0x7fd1f0072710 con 0x7fd1f006cde0 2024-03-04T08:27:30.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.875+0000 7fd1f5c00640 1 -- 172.21.15.113:0/3809653432 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f015b710 con 0x7fd1f0154d50 2024-03-04T08:27:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1f5c00640 1 --2- 172.21.15.113:0/3809653432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f0154d50 0x7fd1f00c20a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009870 tx=0x7fd1e401cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787bea983cdf9d6a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:30.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1efe00640 1 -- 172.21.15.113:0/3809653432 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40a9610 con 0x7fd1f0154d50 2024-03-04T08:27:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1efe00640 1 -- 172.21.15.113:0/3809653432 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e40a97b0 con 0x7fd1f0154d50 2024-03-04T08:27:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1efe00640 1 -- 172.21.15.113:0/3809653432 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40a9b80 con 0x7fd1f0154d50 2024-03-04T08:27:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1f8600640 1 -- 172.21.15.113:0/3809653432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f0154d50 msgr2=0x7fd1f00c20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:30.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1f8600640 1 --2- 172.21.15.113:0/3809653432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f0154d50 0x7fd1f00c20a0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009870 tx=0x7fd1e401cf90 comp rx=0 tx=0).stop 2024-03-04T08:27:30.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1f8600640 1 -- 172.21.15.113:0/3809653432 shutdown_connections 2024-03-04T08:27:30.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1f8600640 1 --2- 172.21.15.113:0/3809653432 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f006b6b0 0x7fd1f00cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:30.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1f8600640 1 --2- 172.21.15.113:0/3809653432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f0154d50 0x7fd1f00c20a0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:30.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1f8600640 1 -- 172.21.15.113:0/3809653432 >> 172.21.15.113:0/3809653432 conn(0x7fd1f00671b0 msgr2=0x7fd1f006dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:30.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1f8600640 1 -- 172.21.15.113:0/3809653432 shutdown_connections 2024-03-04T08:27:30.878 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.879+0000 7fd1f8600640 1 -- 172.21.15.113:0/3809653432 wait complete. 2024-03-04T08:27:30.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 Processor -- start 2024-03-04T08:27:30.879 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 -- start start 2024-03-04T08:27:30.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f006cde0 0x7fd1f00cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:30.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f0154d50 0x7fd1f00c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:30.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1f0146cc0 0x7fd1f01f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:30.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f01552b0 con 0x7fd1f006cde0 2024-03-04T08:27:30.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0155130 con 0x7fd1f0146cc0 2024-03-04T08:27:30.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00cd550 con 0x7fd1f0154d50 2024-03-04T08:27:30.880 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f5c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f006cde0 0x7fd1f00cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:30.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f5c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f006cde0 0x7fd1f00cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41990/0 (socket says 172.21.15.113:41990) 2024-03-04T08:27:30.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f5c00640 1 -- 172.21.15.113:0/4065039515 learned_addr learned my addr 172.21.15.113:0/4065039515 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:30.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f5200640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f0154d50 0x7fd1f00c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f5c00640 1 -- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f0154d50 msgr2=0x7fd1f00c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f5c00640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f0154d50 0x7fd1f00c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f5c00640 1 -- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1f0146cc0 msgr2=0x7fd1f01f3530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f5c00640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1f0146cc0 0x7fd1f01f3530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f5c00640 1 -- 172.21.15.113:0/4065039515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f0159ce0 con 0x7fd1f006cde0 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f5c00640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f006cde0 0x7fd1f00cc090 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd1e401a160 tx=0x7fd1e40192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1ed600640 1 -- 172.21.15.113:0/4065039515 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40a9610 con 0x7fd1f006cde0 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1ed600640 1 -- 172.21.15.113:0/4065039515 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e40ba680 con 0x7fd1f006cde0 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1ed600640 1 -- 172.21.15.113:0/4065039515 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40baa70 con 0x7fd1f006cde0 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 -- 172.21.15.113:0/4065039515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f0159fc0 con 0x7fd1f006cde0 2024-03-04T08:27:30.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 -- 172.21.15.113:0/4065039515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f00cb890 con 0x7fd1f006cde0 2024-03-04T08:27:30.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.883+0000 7fd1f8600640 1 -- 172.21.15.113:0/4065039515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f0075c00 con 0x7fd1f006cde0 2024-03-04T08:27:30.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.887+0000 7fd1ed600640 1 -- 172.21.15.113:0/4065039515 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd1e40bac10 con 0x7fd1f006cde0 2024-03-04T08:27:30.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.887+0000 7fd1ed600640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d0066f90 0x7fd1d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:30.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.887+0000 7fd1ed600640 1 -- 172.21.15.113:0/4065039515 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fd1e411fc70 con 0x7fd1f006cde0 2024-03-04T08:27:30.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.887+0000 7fd1f5200640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d0066f90 0x7fd1d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:30.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.887+0000 7fd1f5200640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d0066f90 0x7fd1d0069450 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00cb310 tx=0x7fd1dc009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:30.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:30.891+0000 7fd1ed600640 1 -- 172.21.15.113:0/4065039515 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd1e40c1030 con 0x7fd1f006cde0 2024-03-04T08:27:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 -- 172.21.15.113:0/4065039515 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1f00c3eb0 con 0x7fd1f006cde0 2024-03-04T08:27:31.075 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1ed600640 1 -- 172.21.15.113:0/4065039515 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fd1e40ad020 con 0x7fd1f006cde0 2024-03-04T08:27:31.075 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:31.075 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:31.076 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:31.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 -- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d0066f90 msgr2=0x7fd1d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:31.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d0066f90 0x7fd1d0069450 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00cb310 tx=0x7fd1dc009190 comp rx=0 tx=0).stop 2024-03-04T08:27:31.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 -- 172.21.15.113:0/4065039515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f006cde0 msgr2=0x7fd1f00cc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:31.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f006cde0 0x7fd1f00cc090 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fd1e401a160 tx=0x7fd1e40192d0 comp rx=0 tx=0).stop 2024-03-04T08:27:31.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 -- 172.21.15.113:0/4065039515 shutdown_connections 2024-03-04T08:27:31.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1f0146cc0 0x7fd1f01f3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:31.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd1f0154d50 0x7fd1f00c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:31.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd1d0066f90 0x7fd1d0069450 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:31.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 --2- 172.21.15.113:0/4065039515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1f006cde0 0x7fd1f00cc090 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:31.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 -- 172.21.15.113:0/4065039515 >> 172.21.15.113:0/4065039515 conn(0x7fd1f00671b0 msgr2=0x7fd1f015d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:31.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 -- 172.21.15.113:0/4065039515 shutdown_connections 2024-03-04T08:27:31.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:31.079+0000 7fd1f8600640 1 -- 172.21.15.113:0/4065039515 wait complete. 2024-03-04T08:27:31.096 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:33.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:33.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd471e00640 1 Processor -- start 2024-03-04T08:27:33.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd471e00640 1 -- start start 2024-03-04T08:27:33.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd471e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd46c1527f0 0x7fd46c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:33.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd471e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46c1591f0 con 0x7fd46c0c8200 2024-03-04T08:27:33.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd471e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46c1594d0 con 0x7fd46c152bd0 2024-03-04T08:27:33.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd471e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46c071df0 con 0x7fd46c1527f0 2024-03-04T08:27:33.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd46aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd46c1527f0 0x7fd46c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:33.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd462000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd46c0c8200 0x7fd46c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48592/0 (socket says 172.21.15.113:48592) 2024-03-04T08:27:33.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd462000640 1 -- 172.21.15.113:0/1994242712 learned_addr learned my addr 172.21.15.113:0/1994242712 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:33.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd46aa00640 1 -- 172.21.15.113:0/1994242712 >> v1:172.21.15.136:6789/0 conn(0x7fd46c152bd0 legacy=0x7fd46c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:33.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd46aa00640 1 -- 172.21.15.113:0/1994242712 >> v1:172.21.15.113:6789/0 conn(0x7fd46c0c8200 legacy=0x7fd46c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:33.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.247+0000 7fd46aa00640 1 -- 172.21.15.113:0/1994242712 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c0717a0 con 0x7fd46c1527f0 2024-03-04T08:27:33.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd46aa00640 1 --2- 172.21.15.113:0/1994242712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c1527f0 0x7fd46c070d40 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009d90 tx=0x7fd45c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4730a66c797212f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:33.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd46a000640 1 -- 172.21.15.113:0/1994242712 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0b2070 con 0x7fd46c1527f0 2024-03-04T08:27:33.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd46a000640 1 -- 172.21.15.113:0/1994242712 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c0a5940 con 0x7fd46c1527f0 2024-03-04T08:27:33.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd46a000640 1 -- 172.21.15.113:0/1994242712 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0a5d10 con 0x7fd46c1527f0 2024-03-04T08:27:33.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd471e00640 1 -- 172.21.15.113:0/1994242712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c1527f0 msgr2=0x7fd46c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:33.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd471e00640 1 --2- 172.21.15.113:0/1994242712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c1527f0 0x7fd46c070d40 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009d90 tx=0x7fd45c01c970 comp rx=0 tx=0).stop 2024-03-04T08:27:33.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd471e00640 1 -- 172.21.15.113:0/1994242712 shutdown_connections 2024-03-04T08:27:33.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd471e00640 1 --2- 172.21.15.113:0/1994242712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c1527f0 0x7fd46c070d40 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:33.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd471e00640 1 -- 172.21.15.113:0/1994242712 >> 172.21.15.113:0/1994242712 conn(0x7fd46c06e820 msgr2=0x7fd46c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:33.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd471e00640 1 -- 172.21.15.113:0/1994242712 shutdown_connections 2024-03-04T08:27:33.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.251+0000 7fd471e00640 1 -- 172.21.15.113:0/1994242712 wait complete. 2024-03-04T08:27:33.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd471e00640 1 Processor -- start 2024-03-04T08:27:33.251 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd471e00640 1 -- start start 2024-03-04T08:27:33.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd471e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0c8200 0x7fd46c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:33.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd471e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c1527f0 0x7fd46c149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd471e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd46c152bd0 0x7fd46c149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd471e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c166380 con 0x7fd46c0c8200 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd471e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c1597b0 con 0x7fd46c1527f0 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd471e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c166500 con 0x7fd46c152bd0 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd46aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0c8200 0x7fd46c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd46aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0c8200 0x7fd46c151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42016/0 (socket says 172.21.15.113:42016) 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd46aa00640 1 -- 172.21.15.113:0/3439789967 learned_addr learned my addr 172.21.15.113:0/3439789967 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd462000640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c1527f0 0x7fd46c149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd46b400640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd46c152bd0 0x7fd46c149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd46aa00640 1 -- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd46c152bd0 msgr2=0x7fd46c149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd46aa00640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd46c152bd0 0x7fd46c149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd46aa00640 1 -- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c1527f0 msgr2=0x7fd46c149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:33.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd46aa00640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c1527f0 0x7fd46c149910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd46aa00640 1 -- 172.21.15.113:0/3439789967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c14edf0 con 0x7fd46c0c8200 2024-03-04T08:27:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd46aa00640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0c8200 0x7fd46c151f90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009d90 tx=0x7fd45c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd463e00640 1 -- 172.21.15.113:0/3439789967 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0b2050 con 0x7fd46c0c8200 2024-03-04T08:27:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd463e00640 1 -- 172.21.15.113:0/3439789967 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c0be070 con 0x7fd46c0c8200 2024-03-04T08:27:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd463e00640 1 -- 172.21.15.113:0/3439789967 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c0adb90 con 0x7fd46c0c8200 2024-03-04T08:27:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd471e00640 1 -- 172.21.15.113:0/3439789967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c1f4620 con 0x7fd46c0c8200 2024-03-04T08:27:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.255+0000 7fd471e00640 1 -- 172.21.15.113:0/3439789967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c14e000 con 0x7fd46c0c8200 2024-03-04T08:27:33.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.259+0000 7fd471e00640 1 -- 172.21.15.113:0/3439789967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c075c00 con 0x7fd46c0c8200 2024-03-04T08:27:33.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.259+0000 7fd463e00640 1 -- 172.21.15.113:0/3439789967 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd45c0b9450 con 0x7fd46c0c8200 2024-03-04T08:27:33.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.259+0000 7fd463e00640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd448066f90 0x7fd448069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:33.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.259+0000 7fd463e00640 1 -- 172.21.15.113:0/3439789967 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fd45c1207a0 con 0x7fd46c0c8200 2024-03-04T08:27:33.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.259+0000 7fd462000640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd448066f90 0x7fd448069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:33.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.259+0000 7fd462000640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd448066f90 0x7fd448069450 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd46c06ec00 tx=0x7fd454009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:33.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.263+0000 7fd463e00640 1 -- 172.21.15.113:0/3439789967 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd45c0c7020 con 0x7fd46c0c8200 2024-03-04T08:27:33.450 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.451+0000 7fd471e00640 1 -- 172.21.15.113:0/3439789967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd46c0c79d0 con 0x7fd46c0c8200 2024-03-04T08:27:33.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd463e00640 1 -- 172.21.15.113:0/3439789967 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fd45c016560 con 0x7fd46c0c8200 2024-03-04T08:27:33.451 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:33.451 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:33.452 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 -- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd448066f90 msgr2=0x7fd448069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd448066f90 0x7fd448069450 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd46c06ec00 tx=0x7fd454009040 comp rx=0 tx=0).stop 2024-03-04T08:27:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 -- 172.21.15.113:0/3439789967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0c8200 msgr2=0x7fd46c151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0c8200 0x7fd46c151f90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009d90 tx=0x7fd45c019710 comp rx=0 tx=0).stop 2024-03-04T08:27:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 -- 172.21.15.113:0/3439789967 shutdown_connections 2024-03-04T08:27:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd46c152bd0 0x7fd46c149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd46c1527f0 0x7fd46c149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd448066f90 0x7fd448069450 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:33.453 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 --2- 172.21.15.113:0/3439789967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd46c0c8200 0x7fd46c151f90 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 -- 172.21.15.113:0/3439789967 >> 172.21.15.113:0/3439789967 conn(0x7fd46c06e820 msgr2=0x7fd46c0cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 -- 172.21.15.113:0/3439789967 shutdown_connections 2024-03-04T08:27:33.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:33.455+0000 7fd471e00640 1 -- 172.21.15.113:0/3439789967 wait complete. 2024-03-04T08:27:33.472 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:35.473 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:35.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f58800640 1 Processor -- start 2024-03-04T08:27:35.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f58800640 1 -- start start 2024-03-04T08:27:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f58800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f501527f0 0x7f3f50070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f58800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3f500c8200 0x7f3f50072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f58800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f501591f0 con 0x7f3f50152bd0 2024-03-04T08:27:35.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f58800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f50071df0 con 0x7f3f501527f0 2024-03-04T08:27:35.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f58800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f501597b0 con 0x7f3f500c8200 2024-03-04T08:27:35.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f55e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f501527f0 0x7f3f50070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:35.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f55e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f501527f0 0x7f3f50070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42030/0 (socket says 172.21.15.113:42030) 2024-03-04T08:27:35.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f55e00640 1 -- 172.21.15.113:0/1175920685 learned_addr learned my addr 172.21.15.113:0/1175920685 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:35.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f56800640 1 --1- 172.21.15.113:0/1175920685 >> v1:172.21.15.113:6789/0 conn(0x7f3f50152bd0 0x7f3f501587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48600/0 (socket says 172.21.15.113:48600) 2024-03-04T08:27:35.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f55400640 1 --2- 172.21.15.113:0/1175920685 >> v2:172.21.15.136:3300/0 conn(0x7f3f500c8200 0x7f3f50072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:35.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f55e00640 1 -- 172.21.15.113:0/1175920685 >> v2:172.21.15.136:3300/0 conn(0x7f3f500c8200 msgr2=0x7f3f50072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:35.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f55e00640 1 --2- 172.21.15.113:0/1175920685 >> v2:172.21.15.136:3300/0 conn(0x7f3f500c8200 0x7f3f50072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:35.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f55e00640 1 -- 172.21.15.113:0/1175920685 >> v1:172.21.15.113:6789/0 conn(0x7f3f50152bd0 legacy=0x7f3f501587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:35.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.623+0000 7f3f55e00640 1 -- 172.21.15.113:0/1175920685 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f501594d0 con 0x7f3f501527f0 2024-03-04T08:27:35.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f55e00640 1 --2- 172.21.15.113:0/1175920685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f501527f0 0x7f3f50070d40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f3f500cd230 tx=0x7f3f4401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef2cdde9189b565 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:35.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f54a00640 1 -- 172.21.15.113:0/1175920685 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440ab430 con 0x7f3f501527f0 2024-03-04T08:27:35.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f54a00640 1 -- 172.21.15.113:0/1175920685 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f440b0070 con 0x7f3f501527f0 2024-03-04T08:27:35.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f54a00640 1 -- 172.21.15.113:0/1175920685 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440ab7f0 con 0x7f3f501527f0 2024-03-04T08:27:35.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f58800640 1 -- 172.21.15.113:0/1175920685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f501527f0 msgr2=0x7f3f50070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:35.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f58800640 1 --2- 172.21.15.113:0/1175920685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f501527f0 0x7f3f50070d40 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f3f500cd230 tx=0x7f3f4401c520 comp rx=0 tx=0).stop 2024-03-04T08:27:35.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f58800640 1 -- 172.21.15.113:0/1175920685 shutdown_connections 2024-03-04T08:27:35.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f58800640 1 --2- 172.21.15.113:0/1175920685 >> v2:172.21.15.136:3300/0 conn(0x7f3f500c8200 0x7f3f50072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:35.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f58800640 1 --2- 172.21.15.113:0/1175920685 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f501527f0 0x7f3f50070d40 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:35.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f58800640 1 -- 172.21.15.113:0/1175920685 >> 172.21.15.113:0/1175920685 conn(0x7f3f5006e820 msgr2=0x7f3f5006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:35.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f58800640 1 -- 172.21.15.113:0/1175920685 shutdown_connections 2024-03-04T08:27:35.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.627+0000 7f3f58800640 1 -- 172.21.15.113:0/1175920685 wait complete. 2024-03-04T08:27:35.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f58800640 1 Processor -- start 2024-03-04T08:27:35.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f58800640 1 -- start start 2024-03-04T08:27:35.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f58800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f500c8200 0x7f3f50155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:35.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f58800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f501527f0 0x7f3f501516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f58800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3f50152bd0 0x7f3f500b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f58800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f50165160 con 0x7f3f500c8200 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f58800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500727c0 con 0x7f3f501527f0 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f58800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f501652e0 con 0x7f3f50152bd0 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f55e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f500c8200 0x7f3f50155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f55e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f500c8200 0x7f3f50155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42044/0 (socket says 172.21.15.113:42044) 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f55e00640 1 -- 172.21.15.113:0/3337229398 learned_addr learned my addr 172.21.15.113:0/3337229398 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f56800640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3f50152bd0 0x7f3f500b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f55400640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f501527f0 0x7f3f501516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f55e00640 1 -- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3f50152bd0 msgr2=0x7f3f500b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f55e00640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3f50152bd0 0x7f3f500b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:35.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f55e00640 1 -- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f501527f0 msgr2=0x7f3f501516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:35.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f55e00640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f501527f0 0x7f3f501516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:35.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f55e00640 1 -- 172.21.15.113:0/3337229398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f500b0ed0 con 0x7f3f500c8200 2024-03-04T08:27:35.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f55e00640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f500c8200 0x7f3f50155370 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f3f440ad040 tx=0x7f3f4401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:35.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f4a000640 1 -- 172.21.15.113:0/3337229398 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440be5f0 con 0x7f3f500c8200 2024-03-04T08:27:35.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f4a000640 1 -- 172.21.15.113:0/3337229398 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f440be790 con 0x7f3f500c8200 2024-03-04T08:27:35.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f4a000640 1 -- 172.21.15.113:0/3337229398 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440beb60 con 0x7f3f500c8200 2024-03-04T08:27:35.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f58800640 1 -- 172.21.15.113:0/3337229398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5014df30 con 0x7f3f500c8200 2024-03-04T08:27:35.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.631+0000 7f3f58800640 1 -- 172.21.15.113:0/3337229398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f501fb7f0 con 0x7f3f500c8200 2024-03-04T08:27:35.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.635+0000 7f3f58800640 1 -- 172.21.15.113:0/3337229398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f50075c00 con 0x7f3f500c8200 2024-03-04T08:27:35.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.635+0000 7f3f4a000640 1 -- 172.21.15.113:0/3337229398 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f440b43f0 con 0x7f3f500c8200 2024-03-04T08:27:35.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.635+0000 7f3f4a000640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3f34066f60 0x7f3f34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:35.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.635+0000 7f3f4a000640 1 -- 172.21.15.113:0/3337229398 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f3f440a8070 con 0x7f3f500c8200 2024-03-04T08:27:35.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.635+0000 7f3f55400640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3f34066f60 0x7f3f34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:35.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.635+0000 7f3f55400640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3f34066f60 0x7f3f34069420 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3f500c6a40 tx=0x7f3f40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:35.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.639+0000 7f3f4a000640 1 -- 172.21.15.113:0/3337229398 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3f44016110 con 0x7f3f500c8200 2024-03-04T08:27:35.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.823+0000 7f3f58800640 1 -- 172.21.15.113:0/3337229398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f500cbdc0 con 0x7f3f500c8200 2024-03-04T08:27:35.822 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f4a000640 1 -- 172.21.15.113:0/3337229398 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f3f440f0300 con 0x7f3f500c8200 2024-03-04T08:27:35.823 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:35.823 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:35.823 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:35.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 -- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3f34066f60 msgr2=0x7f3f34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:35.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3f34066f60 0x7f3f34069420 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3f500c6a40 tx=0x7f3f40008040 comp rx=0 tx=0).stop 2024-03-04T08:27:35.824 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 -- 172.21.15.113:0/3337229398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f500c8200 msgr2=0x7f3f50155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:35.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f500c8200 0x7f3f50155370 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f3f440ad040 tx=0x7f3f4401ad60 comp rx=0 tx=0).stop 2024-03-04T08:27:35.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 -- 172.21.15.113:0/3337229398 shutdown_connections 2024-03-04T08:27:35.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3f50152bd0 0x7f3f500b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:35.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f501527f0 0x7f3f501516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:35.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3f34066f60 0x7f3f34069420 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:35.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 --2- 172.21.15.113:0/3337229398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f500c8200 0x7f3f50155370 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:35.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 -- 172.21.15.113:0/3337229398 >> 172.21.15.113:0/3337229398 conn(0x7f3f5006e820 msgr2=0x7f3f500c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:35.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 -- 172.21.15.113:0/3337229398 shutdown_connections 2024-03-04T08:27:35.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:35.827+0000 7f3f58800640 1 -- 172.21.15.113:0/3337229398 wait complete. 2024-03-04T08:27:35.844 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:37.845 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:37.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.991+0000 7f8f3d800640 1 Processor -- start 2024-03-04T08:27:37.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.991+0000 7f8f3d800640 1 -- start start 2024-03-04T08:27:37.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f3d800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f381527f0 0x7f8f38070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:37.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f3d800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f381591f0 con 0x7f8f380c8200 2024-03-04T08:27:37.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f3d800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f381594d0 con 0x7f8f38152bd0 2024-03-04T08:27:37.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f3d800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38071df0 con 0x7f8f381527f0 2024-03-04T08:27:37.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f36000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f8f380c8200 0x7f8f38072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53012/0 (socket says 172.21.15.113:53012) 2024-03-04T08:27:37.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f36000640 1 -- 172.21.15.113:0/1296822173 learned_addr learned my addr 172.21.15.113:0/1296822173 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:37.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f36a00640 1 --2- 172.21.15.113:0/1296822173 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f381527f0 0x7f8f38070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:37.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f35600640 1 -- 172.21.15.113:0/1296822173 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968095140 0 0) 0x7f8f381591f0 con 0x7f8f380c8200 2024-03-04T08:27:37.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f35600640 1 -- 172.21.15.113:0/1296822173 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f1c003530 con 0x7f8f380c8200 2024-03-04T08:27:37.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f35600640 1 -- 172.21.15.113:0/1296822173 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773499368 0 0) 0x7f8f381594d0 con 0x7f8f38152bd0 2024-03-04T08:27:37.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f35600640 1 -- 172.21.15.113:0/1296822173 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f381591f0 con 0x7f8f38152bd0 2024-03-04T08:27:37.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f35600640 1 -- 172.21.15.113:0/1296822173 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 23275512 0 0) 0x7f8f1c003530 con 0x7f8f380c8200 2024-03-04T08:27:37.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f36a00640 1 -- 172.21.15.113:0/1296822173 >> v1:172.21.15.136:6789/0 conn(0x7f8f38152bd0 legacy=0x7f8f381587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:37.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f36a00640 1 -- 172.21.15.113:0/1296822173 >> v1:172.21.15.113:6789/0 conn(0x7f8f380c8200 legacy=0x7f8f38072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:37.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.995+0000 7f8f36a00640 1 -- 172.21.15.113:0/1296822173 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380717a0 con 0x7f8f381527f0 2024-03-04T08:27:37.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f36a00640 1 --2- 172.21.15.113:0/1296822173 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f381527f0 0x7f8f38070d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009dc0 tx=0x7f8f2801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8086e12c2c6f894 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:37.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f35600640 1 -- 172.21.15.113:0/1296822173 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f280b2070 con 0x7f8f381527f0 2024-03-04T08:27:37.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f35600640 1 -- 172.21.15.113:0/1296822173 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f280a5600 con 0x7f8f381527f0 2024-03-04T08:27:37.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f35600640 1 -- 172.21.15.113:0/1296822173 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24001680 con 0x7f8f381527f0 2024-03-04T08:27:37.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 -- 172.21.15.113:0/1296822173 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f381527f0 msgr2=0x7f8f38070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:37.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 --2- 172.21.15.113:0/1296822173 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f381527f0 0x7f8f38070d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f28009dc0 tx=0x7f8f2801c8b0 comp rx=0 tx=0).stop 2024-03-04T08:27:37.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 -- 172.21.15.113:0/1296822173 shutdown_connections 2024-03-04T08:27:37.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 --2- 172.21.15.113:0/1296822173 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f381527f0 0x7f8f38070d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:37.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 -- 172.21.15.113:0/1296822173 >> 172.21.15.113:0/1296822173 conn(0x7f8f3806e820 msgr2=0x7f8f3806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:37.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 -- 172.21.15.113:0/1296822173 shutdown_connections 2024-03-04T08:27:37.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 -- 172.21.15.113:0/1296822173 wait complete. 2024-03-04T08:27:37.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 Processor -- start 2024-03-04T08:27:37.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 -- start start 2024-03-04T08:27:37.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f380c8200 0x7f8f38157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:37.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f381527f0 0x7f8f3814cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:37.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f38152bd0 0x7f8f3814d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:37.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38166380 con 0x7f8f381527f0 2024-03-04T08:27:37.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381597b0 con 0x7f8f380c8200 2024-03-04T08:27:37.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f3d800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38166500 con 0x7f8f38152bd0 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f36000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f381527f0 0x7f8f3814cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f36000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f381527f0 0x7f8f3814cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44166/0 (socket says 172.21.15.113:44166) 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f36000640 1 -- 172.21.15.113:0/2846910895 learned_addr learned my addr 172.21.15.113:0/2846910895 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f37400640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f38152bd0 0x7f8f3814d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:37.999+0000 7f8f36a00640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f380c8200 0x7f8f38157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f36000640 1 -- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f38152bd0 msgr2=0x7f8f3814d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f36000640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f38152bd0 0x7f8f3814d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f36000640 1 -- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f380c8200 msgr2=0x7f8f38157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f36000640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f380c8200 0x7f8f38157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f36000640 1 -- 172.21.15.113:0/2846910895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f381f7960 con 0x7f8f381527f0 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f36a00640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f380c8200 0x7f8f38157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f36000640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f381527f0 0x7f8f3814cae0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f8f24000fe0 tx=0x7f8f2400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:38.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f2ea00640 1 -- 172.21.15.113:0/2846910895 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24017070 con 0x7f8f381527f0 2024-03-04T08:27:38.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f2ea00640 1 -- 172.21.15.113:0/2846910895 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f24005ae0 con 0x7f8f381527f0 2024-03-04T08:27:38.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f3d800640 1 -- 172.21.15.113:0/2846910895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f381f36b0 con 0x7f8f381527f0 2024-03-04T08:27:38.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f2ea00640 1 -- 172.21.15.113:0/2846910895 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24007070 con 0x7f8f381527f0 2024-03-04T08:27:38.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f3d800640 1 -- 172.21.15.113:0/2846910895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f381f3c70 con 0x7f8f381527f0 2024-03-04T08:27:38.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f3d800640 1 -- 172.21.15.113:0/2846910895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f38075c00 con 0x7f8f381527f0 2024-03-04T08:27:38.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f2ea00640 1 -- 172.21.15.113:0/2846910895 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f2401a070 con 0x7f8f381527f0 2024-03-04T08:27:38.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f2ea00640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f18066f90 0x7f8f18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:38.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f2ea00640 1 -- 172.21.15.113:0/2846910895 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f8f24084f50 con 0x7f8f381527f0 2024-03-04T08:27:38.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.003+0000 7f8f36a00640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f18066f90 0x7f8f18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:38.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.007+0000 7f8f36a00640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f18066f90 0x7f8f18069450 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8f280af000 tx=0x7f8f280a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:38.006 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.007+0000 7f8f2ea00640 1 -- 172.21.15.113:0/2846910895 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8f24054d40 con 0x7f8f381527f0 2024-03-04T08:27:38.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.195+0000 7f8f3d800640 1 -- 172.21.15.113:0/2846910895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f380c79d0 con 0x7f8f381527f0 2024-03-04T08:27:38.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.195+0000 7f8f2ea00640 1 -- 172.21.15.113:0/2846910895 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f8f24058de0 con 0x7f8f381527f0 2024-03-04T08:27:38.194 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:38.195 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:38.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 -- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f18066f90 msgr2=0x7f8f18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:38.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f18066f90 0x7f8f18069450 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8f280af000 tx=0x7f8f280a7070 comp rx=0 tx=0).stop 2024-03-04T08:27:38.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 -- 172.21.15.113:0/2846910895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f381527f0 msgr2=0x7f8f3814cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:38.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f381527f0 0x7f8f3814cae0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f8f24000fe0 tx=0x7f8f2400cbe0 comp rx=0 tx=0).stop 2024-03-04T08:27:38.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 -- 172.21.15.113:0/2846910895 shutdown_connections 2024-03-04T08:27:38.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f8f38152bd0 0x7f8f3814d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:38.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8f18066f90 0x7f8f18069450 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:38.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f381527f0 0x7f8f3814cae0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:38.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 --2- 172.21.15.113:0/2846910895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f380c8200 0x7f8f38157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:38.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 -- 172.21.15.113:0/2846910895 >> 172.21.15.113:0/2846910895 conn(0x7f8f3806e820 msgr2=0x7f8f380c1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:38.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 -- 172.21.15.113:0/2846910895 shutdown_connections 2024-03-04T08:27:38.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:38.199+0000 7f8f3d800640 1 -- 172.21.15.113:0/2846910895 wait complete. 2024-03-04T08:27:38.216 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:27:38.217 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:27:40.217 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:40.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.367+0000 7fb881400640 1 Processor -- start 2024-03-04T08:27:40.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.367+0000 7fb881400640 1 -- start start 2024-03-04T08:27:40.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb881400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c160820 0x7fb87c162c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:40.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb881400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c0c1ec0 con 0x7fb87c0b20b0 2024-03-04T08:27:40.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb881400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87c0c9b30 con 0x7fb87c0c85f0 2024-03-04T08:27:40.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb881400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0c12a0 con 0x7fb87c160820 2024-03-04T08:27:40.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb87a800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb87c0b20b0 0x7fb87c157710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53016/0 (socket says 172.21.15.113:53016) 2024-03-04T08:27:40.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb87a800640 1 -- 172.21.15.113:0/3433161514 learned_addr learned my addr 172.21.15.113:0/3433161514 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:40.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb87b200640 1 --2- 172.21.15.113:0/3433161514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c160820 0x7fb87c162c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:40.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb879400640 1 -- 172.21.15.113:0/3433161514 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999539606 0 0) 0x7fb87c0c1ec0 con 0x7fb87c0b20b0 2024-03-04T08:27:40.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb879400640 1 -- 172.21.15.113:0/3433161514 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb860003450 con 0x7fb87c0b20b0 2024-03-04T08:27:40.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb879400640 1 -- 172.21.15.113:0/3433161514 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128306918 0 0) 0x7fb87c0c9b30 con 0x7fb87c0c85f0 2024-03-04T08:27:40.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb879400640 1 -- 172.21.15.113:0/3433161514 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb87c0c1ec0 con 0x7fb87c0c85f0 2024-03-04T08:27:40.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb87b200640 1 -- 172.21.15.113:0/3433161514 >> v1:172.21.15.136:6789/0 conn(0x7fb87c0c85f0 legacy=0x7fb87c06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:40.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb87b200640 1 -- 172.21.15.113:0/3433161514 >> v1:172.21.15.113:6789/0 conn(0x7fb87c0b20b0 legacy=0x7fb87c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:40.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.371+0000 7fb87b200640 1 -- 172.21.15.113:0/3433161514 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0c1580 con 0x7fb87c160820 2024-03-04T08:27:40.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb87b200640 1 --2- 172.21.15.113:0/3433161514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c160820 0x7fb87c162c10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb874009dc0 tx=0x7fb87401f170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8884dbb872d94469 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb879400640 1 -- 172.21.15.113:0/3433161514 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740b4070 con 0x7fb87c160820 2024-03-04T08:27:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb879400640 1 -- 172.21.15.113:0/3433161514 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8740af600 con 0x7fb87c160820 2024-03-04T08:27:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb879400640 1 -- 172.21.15.113:0/3433161514 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740af9b0 con 0x7fb87c160820 2024-03-04T08:27:40.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 -- 172.21.15.113:0/3433161514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c160820 msgr2=0x7fb87c162c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:40.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 --2- 172.21.15.113:0/3433161514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c160820 0x7fb87c162c10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb874009dc0 tx=0x7fb87401f170 comp rx=0 tx=0).stop 2024-03-04T08:27:40.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 -- 172.21.15.113:0/3433161514 shutdown_connections 2024-03-04T08:27:40.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 --2- 172.21.15.113:0/3433161514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c160820 0x7fb87c162c10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:40.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 -- 172.21.15.113:0/3433161514 >> 172.21.15.113:0/3433161514 conn(0x7fb87c06e820 msgr2=0x7fb87c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:40.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 -- 172.21.15.113:0/3433161514 shutdown_connections 2024-03-04T08:27:40.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 -- 172.21.15.113:0/3433161514 wait complete. 2024-03-04T08:27:40.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 Processor -- start 2024-03-04T08:27:40.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 -- start start 2024-03-04T08:27:40.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0b20b0 0x7fb87c0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:40.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c0c85f0 0x7fb87c1439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:40.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb87c160820 0x7fb87c143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c157e20 con 0x7fb87c0b20b0 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c1571f0 con 0x7fb87c160820 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb881400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c0c2400 con 0x7fb87c0c85f0 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb87a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0b20b0 0x7fb87c0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb87a800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0b20b0 0x7fb87c0c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44176/0 (socket says 172.21.15.113:44176) 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb87a800640 1 -- 172.21.15.113:0/3531152088 learned_addr learned my addr 172.21.15.113:0/3531152088 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb87b200640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb87c160820 0x7fb87c143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.375+0000 7fb879e00640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c0c85f0 0x7fb87c1439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb87a800640 1 -- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c0c85f0 msgr2=0x7fb87c1439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb87a800640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c0c85f0 0x7fb87c1439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb87a800640 1 -- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb87c160820 msgr2=0x7fb87c143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:40.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb87a800640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb87c160820 0x7fb87c143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:40.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb87a800640 1 -- 172.21.15.113:0/3531152088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c14a760 con 0x7fb87c0b20b0 2024-03-04T08:27:40.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb87a800640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0b20b0 0x7fb87c0c8e60 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fb86400c4d0 tx=0x7fb86400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:40.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb872a00640 1 -- 172.21.15.113:0/3531152088 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864017070 con 0x7fb87c0b20b0 2024-03-04T08:27:40.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb872a00640 1 -- 172.21.15.113:0/3531152088 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb864005cf0 con 0x7fb87c0b20b0 2024-03-04T08:27:40.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb881400640 1 -- 172.21.15.113:0/3531152088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87c14d640 con 0x7fb87c0b20b0 2024-03-04T08:27:40.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb872a00640 1 -- 172.21.15.113:0/3531152088 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864007270 con 0x7fb87c0b20b0 2024-03-04T08:27:40.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb881400640 1 -- 172.21.15.113:0/3531152088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87c14dc00 con 0x7fb87c0b20b0 2024-03-04T08:27:40.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb881400640 1 -- 172.21.15.113:0/3531152088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb87c14df40 con 0x7fb87c0b20b0 2024-03-04T08:27:40.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb872a00640 1 -- 172.21.15.113:0/3531152088 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb86401a070 con 0x7fb87c0b20b0 2024-03-04T08:27:40.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb872a00640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb858066f90 0x7fb858069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:40.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.379+0000 7fb872a00640 1 -- 172.21.15.113:0/3531152088 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fb864084d00 con 0x7fb87c0b20b0 2024-03-04T08:27:40.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.383+0000 7fb879e00640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb858066f90 0x7fb858069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:40.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.383+0000 7fb879e00640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb858066f90 0x7fb858069450 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb87c14d290 tx=0x7fb86c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:40.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.383+0000 7fb872a00640 1 -- 172.21.15.113:0/3531152088 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb87c14df40 con 0x7fb87c0b20b0 2024-03-04T08:27:40.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.571+0000 7fb881400640 1 -- 172.21.15.113:0/3531152088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb87c075c00 con 0x7fb87c0b20b0 2024-03-04T08:27:40.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb872a00640 1 -- 172.21.15.113:0/3531152088 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fb864054c10 con 0x7fb87c0b20b0 2024-03-04T08:27:40.571 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:40.571 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:40.571 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:40.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 -- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb858066f90 msgr2=0x7fb858069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:40.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb858066f90 0x7fb858069450 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb87c14d290 tx=0x7fb86c009040 comp rx=0 tx=0).stop 2024-03-04T08:27:40.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 -- 172.21.15.113:0/3531152088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0b20b0 msgr2=0x7fb87c0c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:40.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0b20b0 0x7fb87c0c8e60 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fb86400c4d0 tx=0x7fb86400c990 comp rx=0 tx=0).stop 2024-03-04T08:27:40.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 -- 172.21.15.113:0/3531152088 shutdown_connections 2024-03-04T08:27:40.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb87c160820 0x7fb87c143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:40.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb87c0c85f0 0x7fb87c1439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:40.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb858066f90 0x7fb858069450 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:40.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 --2- 172.21.15.113:0/3531152088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb87c0b20b0 0x7fb87c0c8e60 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:40.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 -- 172.21.15.113:0/3531152088 >> 172.21.15.113:0/3531152088 conn(0x7fb87c06e820 msgr2=0x7fb87c14f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:40.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 -- 172.21.15.113:0/3531152088 shutdown_connections 2024-03-04T08:27:40.573 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:40.575+0000 7fb881400640 1 -- 172.21.15.113:0/3531152088 wait complete. 2024-03-04T08:27:40.592 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:42.595 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:42.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f99200640 1 Processor -- start 2024-03-04T08:27:42.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f99200640 1 -- start start 2024-03-04T08:27:42.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f99200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f94154fe0 0x7f1f940c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:42.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f99200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f940c7d80 con 0x7f1f94154c00 2024-03-04T08:27:42.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f99200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f940c8060 con 0x7f1f940c7700 2024-03-04T08:27:42.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f99200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9415efb0 con 0x7f1f94154fe0 2024-03-04T08:27:42.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f89400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f94154fe0 0x7f1f940c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:42.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f91e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1f94154c00 0x7f1f940c1b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:53028/0 (socket says 172.21.15.113:53028) 2024-03-04T08:27:42.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f89400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1f94154fe0 0x7f1f940c1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44192/0 (socket says 172.21.15.113:44192) 2024-03-04T08:27:42.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f91e00640 1 -- 172.21.15.113:0/1042808300 learned_addr learned my addr 172.21.15.113:0/1042808300 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:42.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f89400640 1 -- 172.21.15.113:0/1042808300 >> v1:172.21.15.136:6789/0 conn(0x7f1f940c7700 legacy=0x7f1f9415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:42.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f89400640 1 -- 172.21.15.113:0/1042808300 >> v1:172.21.15.113:6789/0 conn(0x7f1f94154c00 legacy=0x7f1f940c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:42.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.743+0000 7f1f89400640 1 -- 172.21.15.113:0/1042808300 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f940c6dc0 con 0x7f1f94154fe0 2024-03-04T08:27:42.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f89400640 1 --2- 172.21.15.113:0/1042808300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f94154fe0 0x7f1f940c1190 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0097c0 tx=0x7f1f7c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f214b09691ae5a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:42.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f91400640 1 -- 172.21.15.113:0/1042808300 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0b0070 con 0x7f1f94154fe0 2024-03-04T08:27:42.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f91400640 1 -- 172.21.15.113:0/1042808300 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c0ab600 con 0x7f1f94154fe0 2024-03-04T08:27:42.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f91400640 1 -- 172.21.15.113:0/1042808300 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0ab9d0 con 0x7f1f94154fe0 2024-03-04T08:27:42.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f99200640 1 -- 172.21.15.113:0/1042808300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f94154fe0 msgr2=0x7f1f940c1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:42.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f99200640 1 --2- 172.21.15.113:0/1042808300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f94154fe0 0x7f1f940c1190 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0097c0 tx=0x7f1f7c01c410 comp rx=0 tx=0).stop 2024-03-04T08:27:42.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f99200640 1 -- 172.21.15.113:0/1042808300 shutdown_connections 2024-03-04T08:27:42.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f99200640 1 --2- 172.21.15.113:0/1042808300 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f94154fe0 0x7f1f940c1190 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:42.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f99200640 1 -- 172.21.15.113:0/1042808300 >> 172.21.15.113:0/1042808300 conn(0x7f1f9406e820 msgr2=0x7f1f9406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:42.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f99200640 1 -- 172.21.15.113:0/1042808300 shutdown_connections 2024-03-04T08:27:42.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.747+0000 7f1f99200640 1 -- 172.21.15.113:0/1042808300 wait complete. 2024-03-04T08:27:42.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f99200640 1 Processor -- start 2024-03-04T08:27:42.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f99200640 1 -- start start 2024-03-04T08:27:42.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f99200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f940c7700 0x7f1f941566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:42.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f99200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1f94154c00 0x7f1f94156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:42.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f99200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94154fe0 0x7f1f94152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:42.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f99200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f94168fb0 con 0x7f1f940c7700 2024-03-04T08:27:42.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f99200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f94168e30 con 0x7f1f94154fe0 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f99200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f94169130 con 0x7f1f94154c00 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f91e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f940c7700 0x7f1f941566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f91e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f940c7700 0x7f1f941566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44208/0 (socket says 172.21.15.113:44208) 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f91e00640 1 -- 172.21.15.113:0/2571603784 learned_addr learned my addr 172.21.15.113:0/2571603784 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f92800640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94154fe0 0x7f1f94152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f89400640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1f94154c00 0x7f1f94156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f91e00640 1 -- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1f94154c00 msgr2=0x7f1f94156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f91e00640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1f94154c00 0x7f1f94156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f91e00640 1 -- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94154fe0 msgr2=0x7f1f94152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f91e00640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94154fe0 0x7f1f94152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f91e00640 1 -- 172.21.15.113:0/2571603784 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f94151a00 con 0x7f1f940c7700 2024-03-04T08:27:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f91e00640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f940c7700 0x7f1f941566d0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400cbc0 tx=0x7f1f8400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:42.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f8aa00640 1 -- 172.21.15.113:0/2571603784 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84017070 con 0x7f1f940c7700 2024-03-04T08:27:42.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f8aa00640 1 -- 172.21.15.113:0/2571603784 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8400aca0 con 0x7f1f940c7700 2024-03-04T08:27:42.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f8aa00640 1 -- 172.21.15.113:0/2571603784 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84012600 con 0x7f1f940c7700 2024-03-04T08:27:42.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f99200640 1 -- 172.21.15.113:0/2571603784 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f94151ce0 con 0x7f1f940c7700 2024-03-04T08:27:42.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.751+0000 7f1f99200640 1 -- 172.21.15.113:0/2571603784 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f941522a0 con 0x7f1f940c7700 2024-03-04T08:27:42.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.755+0000 7f1f99200640 1 -- 172.21.15.113:0/2571603784 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f94075c00 con 0x7f1f940c7700 2024-03-04T08:27:42.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.755+0000 7f1f8aa00640 1 -- 172.21.15.113:0/2571603784 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f8401a070 con 0x7f1f940c7700 2024-03-04T08:27:42.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.755+0000 7f1f8aa00640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1f70066f60 0x7f1f70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:42.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.755+0000 7f1f8aa00640 1 -- 172.21.15.113:0/2571603784 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f1f840857a0 con 0x7f1f940c7700 2024-03-04T08:27:42.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.755+0000 7f1f89400640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1f70066f60 0x7f1f70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:42.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.755+0000 7f1f89400640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1f70066f60 0x7f1f70069420 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0ad000 tx=0x7f1f7c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:42.757 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.759+0000 7f1f8aa00640 1 -- 172.21.15.113:0/2571603784 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1f84055620 con 0x7f1f940c7700 2024-03-04T08:27:42.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.947+0000 7f1f99200640 1 -- 172.21.15.113:0/2571603784 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f940c1710 con 0x7f1f940c7700 2024-03-04T08:27:42.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.947+0000 7f1f8aa00640 1 -- 172.21.15.113:0/2571603784 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f1f840596c0 con 0x7f1f940c7700 2024-03-04T08:27:42.945 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:42.945 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:42.945 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:42.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 -- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1f70066f60 msgr2=0x7f1f70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:42.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1f70066f60 0x7f1f70069420 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0ad000 tx=0x7f1f7c0a6000 comp rx=0 tx=0).stop 2024-03-04T08:27:42.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 -- 172.21.15.113:0/2571603784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f940c7700 msgr2=0x7f1f941566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:42.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f940c7700 0x7f1f941566d0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400cbc0 tx=0x7f1f8400a060 comp rx=0 tx=0).stop 2024-03-04T08:27:42.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 -- 172.21.15.113:0/2571603784 shutdown_connections 2024-03-04T08:27:42.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94154fe0 0x7f1f94152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:42.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1f94154c00 0x7f1f94156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:42.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1f70066f60 0x7f1f70069420 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:42.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 --2- 172.21.15.113:0/2571603784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1f940c7700 0x7f1f941566d0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:42.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 -- 172.21.15.113:0/2571603784 >> 172.21.15.113:0/2571603784 conn(0x7f1f9406e820 msgr2=0x7f1f940c93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:42.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 -- 172.21.15.113:0/2571603784 shutdown_connections 2024-03-04T08:27:42.948 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:42.951+0000 7f1f99200640 1 -- 172.21.15.113:0/2571603784 wait complete. 2024-03-04T08:27:42.966 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:44.967 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:45.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb3a400640 1 Processor -- start 2024-03-04T08:27:45.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb3a400640 1 -- start start 2024-03-04T08:27:45.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb3a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb3406d420 0x7feb340730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:45.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb3a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb3415bce0 0x7feb3415c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:45.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb3a400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3406f1b0 con 0x7feb34152a90 2024-03-04T08:27:45.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb3a400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb34152e70 con 0x7feb3415bce0 2024-03-04T08:27:45.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb3a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3406c320 con 0x7feb3406d420 2024-03-04T08:27:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb39a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb3415bce0 0x7feb3415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb39a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feb3415bce0 0x7feb3415c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44216/0 (socket says 172.21.15.113:44216) 2024-03-04T08:27:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb39a00640 1 -- 172.21.15.113:0/3530382163 learned_addr learned my addr 172.21.15.113:0/3530382163 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb33e00640 1 --2- 172.21.15.113:0/3530382163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb3406d420 0x7feb340730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb39a00640 1 -- 172.21.15.113:0/3530382163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb3406d420 msgr2=0x7feb340730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb39a00640 1 --2- 172.21.15.113:0/3530382163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb3406d420 0x7feb340730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb39a00640 1 -- 172.21.15.113:0/3530382163 >> v1:172.21.15.113:6789/0 conn(0x7feb34152a90 legacy=0x7feb3406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:45.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb39a00640 1 -- 172.21.15.113:0/3530382163 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb34071580 con 0x7feb3415bce0 2024-03-04T08:27:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.119+0000 7feb33400640 1 -- 172.21.15.113:0/3530382163 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952701467 0 0) 0x7feb3406f1b0 con 0x7feb34152a90 2024-03-04T08:27:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb39a00640 1 --2- 172.21.15.113:0/3530382163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 0x7feb3415c150 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009e50 tx=0x7feb2c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b1b84a16b07eaa server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb33400640 1 -- 172.21.15.113:0/3530382163 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0a5700 con 0x7feb3415bce0 2024-03-04T08:27:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb33400640 1 -- 172.21.15.113:0/3530382163 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c0a58a0 con 0x7feb3415bce0 2024-03-04T08:27:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb33400640 1 -- 172.21.15.113:0/3530382163 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0a5c70 con 0x7feb3415bce0 2024-03-04T08:27:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb3a400640 1 -- 172.21.15.113:0/3530382163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 msgr2=0x7feb3415c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:45.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb3a400640 1 --2- 172.21.15.113:0/3530382163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 0x7feb3415c150 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009e50 tx=0x7feb2c01ca30 comp rx=0 tx=0).stop 2024-03-04T08:27:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb3a400640 1 -- 172.21.15.113:0/3530382163 shutdown_connections 2024-03-04T08:27:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb3a400640 1 --2- 172.21.15.113:0/3530382163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 0x7feb3415c150 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb3a400640 1 --2- 172.21.15.113:0/3530382163 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb3406d420 0x7feb340730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb3a400640 1 -- 172.21.15.113:0/3530382163 >> 172.21.15.113:0/3530382163 conn(0x7feb3406e2a0 msgr2=0x7feb3406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb3a400640 1 -- 172.21.15.113:0/3530382163 shutdown_connections 2024-03-04T08:27:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb3a400640 1 -- 172.21.15.113:0/3530382163 wait complete. 2024-03-04T08:27:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.123+0000 7feb3a400640 1 Processor -- start 2024-03-04T08:27:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb3a400640 1 -- start start 2024-03-04T08:27:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb3a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb3406d420 0x7feb3415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb3a400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb34152a90 0x7feb341f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb3a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 0x7feb34141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb3a400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3415caa0 con 0x7feb3415bce0 2024-03-04T08:27:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb3a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb34071330 con 0x7feb3406d420 2024-03-04T08:27:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb3a400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb34165aa0 con 0x7feb34152a90 2024-03-04T08:27:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb39a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 0x7feb34141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb39a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 0x7feb34141ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:44220/0 (socket says 172.21.15.113:44220) 2024-03-04T08:27:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb39a00640 1 -- 172.21.15.113:0/1504034707 learned_addr learned my addr 172.21.15.113:0/1504034707 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb39000640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb3406d420 0x7feb3415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb33e00640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb34152a90 0x7feb341f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb39a00640 1 -- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb34152a90 msgr2=0x7feb341f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb39a00640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb34152a90 0x7feb341f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb39a00640 1 -- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb3406d420 msgr2=0x7feb3415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb39a00640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb3406d420 0x7feb3415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb39a00640 1 -- 172.21.15.113:0/1504034707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb341f5c50 con 0x7feb3415bce0 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb39a00640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 0x7feb34141ec0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009e50 tx=0x7feb2c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb30c00640 1 -- 172.21.15.113:0/1504034707 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0b8470 con 0x7feb3415bce0 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb30c00640 1 -- 172.21.15.113:0/1504034707 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c0b8610 con 0x7feb3415bce0 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb30c00640 1 -- 172.21.15.113:0/1504034707 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0b89e0 con 0x7feb3415bce0 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb3a400640 1 -- 172.21.15.113:0/1504034707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb34147f10 con 0x7feb3415bce0 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb3a400640 1 -- 172.21.15.113:0/1504034707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb341f1b20 con 0x7feb3415bce0 2024-03-04T08:27:45.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.127+0000 7feb3a400640 1 -- 172.21.15.113:0/1504034707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb34075a80 con 0x7feb3415bce0 2024-03-04T08:27:45.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.131+0000 7feb30c00640 1 -- 172.21.15.113:0/1504034707 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb2c0c0020 con 0x7feb3415bce0 2024-03-04T08:27:45.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.131+0000 7feb30c00640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb14066f60 0x7feb14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:45.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.131+0000 7feb30c00640 1 -- 172.21.15.113:0/1504034707 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7feb2c120240 con 0x7feb3415bce0 2024-03-04T08:27:45.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.131+0000 7feb39000640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb14066f60 0x7feb14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:45.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.131+0000 7feb39000640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb14066f60 0x7feb14069420 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7feb28007fd0 tx=0x7feb28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:45.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.135+0000 7feb30c00640 1 -- 172.21.15.113:0/1504034707 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7feb2c0c7020 con 0x7feb3415bce0 2024-03-04T08:27:45.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 -- 172.21.15.113:0/1504034707 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb340c7590 con 0x7feb3415bce0 2024-03-04T08:27:45.320 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb30c00640 1 -- 172.21.15.113:0/1504034707 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7feb2c016620 con 0x7feb3415bce0 2024-03-04T08:27:45.321 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:45.321 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:45.323 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:45.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 -- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb14066f60 msgr2=0x7feb14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:45.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb14066f60 0x7feb14069420 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7feb28007fd0 tx=0x7feb28009040 comp rx=0 tx=0).stop 2024-03-04T08:27:45.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 -- 172.21.15.113:0/1504034707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 msgr2=0x7feb34141ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:45.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 0x7feb34141ec0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009e50 tx=0x7feb2c0197d0 comp rx=0 tx=0).stop 2024-03-04T08:27:45.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 -- 172.21.15.113:0/1504034707 shutdown_connections 2024-03-04T08:27:45.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7feb14066f60 0x7feb14069420 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:45.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feb3415bce0 0x7feb34141ec0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:45.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feb34152a90 0x7feb341f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:45.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 --2- 172.21.15.113:0/1504034707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feb3406d420 0x7feb3415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:45.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 -- 172.21.15.113:0/1504034707 >> 172.21.15.113:0/1504034707 conn(0x7feb3406e2a0 msgr2=0x7feb34144fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:45.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 -- 172.21.15.113:0/1504034707 shutdown_connections 2024-03-04T08:27:45.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:45.323+0000 7feb3a400640 1 -- 172.21.15.113:0/1504034707 wait complete. 2024-03-04T08:27:45.342 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:45.829 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:27:45.834 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:27:47.342 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:47.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99c600640 1 Processor -- start 2024-03-04T08:27:47.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99c600640 1 -- start start 2024-03-04T08:27:47.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99c600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb9940c22e0 0x7fb99415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:47.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99c600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9940c2d80 0x7fb99415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:47.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99c600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9940c27f0 con 0x7fb99414e4c0 2024-03-04T08:27:47.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99c600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99415ad60 con 0x7fb9940c2d80 2024-03-04T08:27:47.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99c600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9940c2a90 con 0x7fb9940c22e0 2024-03-04T08:27:47.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb999c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb99414e4c0 0x7fb994072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37966/0 (socket says 172.21.15.113:37966) 2024-03-04T08:27:47.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb999c00640 1 -- 172.21.15.113:0/2946593766 learned_addr learned my addr 172.21.15.113:0/2946593766 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:47.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99a600640 1 --2- 172.21.15.113:0/2946593766 >> v2:172.21.15.113:3300/0 conn(0x7fb9940c2d80 0x7fb99415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:47.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb999200640 1 --2- 172.21.15.113:0/2946593766 >> v2:172.21.15.136:3300/0 conn(0x7fb9940c22e0 0x7fb99415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:47.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb993e00640 1 -- 172.21.15.113:0/2946593766 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169343107 0 0) 0x7fb9940c27f0 con 0x7fb99414e4c0 2024-03-04T08:27:47.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb993e00640 1 -- 172.21.15.113:0/2946593766 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb97c003530 con 0x7fb99414e4c0 2024-03-04T08:27:47.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99a600640 1 -- 172.21.15.113:0/2946593766 >> v2:172.21.15.136:3300/0 conn(0x7fb9940c22e0 msgr2=0x7fb99415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:47.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99a600640 1 --2- 172.21.15.113:0/2946593766 >> v2:172.21.15.136:3300/0 conn(0x7fb9940c22e0 0x7fb99415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:47.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99a600640 1 -- 172.21.15.113:0/2946593766 >> v1:172.21.15.113:6789/0 conn(0x7fb99414e4c0 legacy=0x7fb994072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:47.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.491+0000 7fb99a600640 1 -- 172.21.15.113:0/2946593766 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb994070c60 con 0x7fb9940c2d80 2024-03-04T08:27:47.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb99a600640 1 --2- 172.21.15.113:0/2946593766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c2d80 0x7fb99415b730 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fb9940cd230 tx=0x7fb98c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3358038ce1ec26 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:47.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb993e00640 1 -- 172.21.15.113:0/2946593766 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0b2070 con 0x7fb9940c2d80 2024-03-04T08:27:47.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb993e00640 1 -- 172.21.15.113:0/2946593766 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c0a5900 con 0x7fb9940c2d80 2024-03-04T08:27:47.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb993e00640 1 -- 172.21.15.113:0/2946593766 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0a5cd0 con 0x7fb9940c2d80 2024-03-04T08:27:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb99c600640 1 -- 172.21.15.113:0/2946593766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c2d80 msgr2=0x7fb99415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb99c600640 1 --2- 172.21.15.113:0/2946593766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c2d80 0x7fb99415b730 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fb9940cd230 tx=0x7fb98c01c930 comp rx=0 tx=0).stop 2024-03-04T08:27:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb99c600640 1 -- 172.21.15.113:0/2946593766 shutdown_connections 2024-03-04T08:27:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb99c600640 1 --2- 172.21.15.113:0/2946593766 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c2d80 0x7fb99415b730 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb99c600640 1 --2- 172.21.15.113:0/2946593766 >> v2:172.21.15.136:3300/0 conn(0x7fb9940c22e0 0x7fb99415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb99c600640 1 -- 172.21.15.113:0/2946593766 >> 172.21.15.113:0/2946593766 conn(0x7fb99406e820 msgr2=0x7fb99406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb99c600640 1 -- 172.21.15.113:0/2946593766 shutdown_connections 2024-03-04T08:27:47.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.495+0000 7fb99c600640 1 -- 172.21.15.113:0/2946593766 wait complete. 2024-03-04T08:27:47.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 Processor -- start 2024-03-04T08:27:47.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 -- start start 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c22e0 0x7fb994140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9940c2d80 0x7fb99414ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb99414e4c0 0x7fb99414cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9940aede0 con 0x7fb9940c22e0 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99414e8a0 con 0x7fb99414e4c0 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb994073390 con 0x7fb9940c2d80 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb999c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c22e0 0x7fb994140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb999c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c22e0 0x7fb994140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55810/0 (socket says 172.21.15.113:55810) 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb999c00640 1 -- 172.21.15.113:0/3965588811 learned_addr learned my addr 172.21.15.113:0/3965588811 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb999200640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9940c2d80 0x7fb99414ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:47.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99a600640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb99414e4c0 0x7fb99414cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb999c00640 1 -- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9940c2d80 msgr2=0x7fb99414ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb999c00640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9940c2d80 0x7fb99414ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb999c00640 1 -- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb99414e4c0 msgr2=0x7fb99414cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb999c00640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb99414e4c0 0x7fb99414cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb999c00640 1 -- 172.21.15.113:0/3965588811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99406bab0 con 0x7fb9940c22e0 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb999c00640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c22e0 0x7fb994140df0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb98800c450 tx=0x7fb98800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb991600640 1 -- 172.21.15.113:0/3965588811 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988017070 con 0x7fb9940c22e0 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb991600640 1 -- 172.21.15.113:0/3965588811 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb988005cf0 con 0x7fb9940c22e0 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 -- 172.21.15.113:0/3965588811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb994157c00 con 0x7fb9940c22e0 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb991600640 1 -- 172.21.15.113:0/3965588811 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988007290 con 0x7fb9940c22e0 2024-03-04T08:27:47.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 -- 172.21.15.113:0/3965588811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9941581c0 con 0x7fb9940c22e0 2024-03-04T08:27:47.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.499+0000 7fb99c600640 1 -- 172.21.15.113:0/3965588811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb994158400 con 0x7fb9940c22e0 2024-03-04T08:27:47.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.503+0000 7fb991600640 1 -- 172.21.15.113:0/3965588811 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb98801a070 con 0x7fb9940c22e0 2024-03-04T08:27:47.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.503+0000 7fb991600640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb974066f60 0x7fb974069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:47.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.503+0000 7fb991600640 1 -- 172.21.15.113:0/3965588811 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fb988084ff0 con 0x7fb9940c22e0 2024-03-04T08:27:47.500 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.503+0000 7fb999200640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb974066f60 0x7fb974069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:47.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.503+0000 7fb999200640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb974066f60 0x7fb974069420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb9941447e0 tx=0x7fb980008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:47.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.507+0000 7fb991600640 1 -- 172.21.15.113:0/3965588811 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb994158400 con 0x7fb9940c22e0 2024-03-04T08:27:47.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.695+0000 7fb99c600640 1 -- 172.21.15.113:0/3965588811 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb994075c00 con 0x7fb9940c22e0 2024-03-04T08:27:47.694 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.695+0000 7fb991600640 1 -- 172.21.15.113:0/3965588811 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fb988055450 con 0x7fb9940c22e0 2024-03-04T08:27:47.695 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:47.695 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:47.695 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 -- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb974066f60 msgr2=0x7fb974069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb974066f60 0x7fb974069420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb9941447e0 tx=0x7fb980008040 comp rx=0 tx=0).stop 2024-03-04T08:27:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 -- 172.21.15.113:0/3965588811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c22e0 msgr2=0x7fb994140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c22e0 0x7fb994140df0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fb98800c450 tx=0x7fb98800c910 comp rx=0 tx=0).stop 2024-03-04T08:27:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 -- 172.21.15.113:0/3965588811 shutdown_connections 2024-03-04T08:27:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb99414e4c0 0x7fb99414cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb9940c2d80 0x7fb99414ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:47.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb974066f60 0x7fb974069420 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 --2- 172.21.15.113:0/3965588811 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9940c22e0 0x7fb994140df0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 -- 172.21.15.113:0/3965588811 >> 172.21.15.113:0/3965588811 conn(0x7fb99406e820 msgr2=0x7fb9940c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 -- 172.21.15.113:0/3965588811 shutdown_connections 2024-03-04T08:27:47.698 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:47.699+0000 7fb99c600640 1 -- 172.21.15.113:0/3965588811 wait complete. 2024-03-04T08:27:47.716 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:49.716 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:49.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb223600640 1 Processor -- start 2024-03-04T08:27:49.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb223600640 1 -- start start 2024-03-04T08:27:49.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb223600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb21c06dd50 0x7fb21c1662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:49.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb223600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c0ca0e0 con 0x7fb21c158010 2024-03-04T08:27:49.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb223600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb21c15aa30 con 0x7fb21c06ff20 2024-03-04T08:27:49.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb223600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb21c06f450 con 0x7fb21c06dd50 2024-03-04T08:27:49.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb220c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb21c158010 0x7fb21c15b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37974/0 (socket says 172.21.15.113:37974) 2024-03-04T08:27:49.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb220c00640 1 -- 172.21.15.113:0/979843981 learned_addr learned my addr 172.21.15.113:0/979843981 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:49.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb221600640 1 --2- 172.21.15.113:0/979843981 >> v2:172.21.15.136:3300/0 conn(0x7fb21c06dd50 0x7fb21c1662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:49.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb21be00640 1 -- 172.21.15.113:0/979843981 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453635154 0 0) 0x7fb21c0ca0e0 con 0x7fb21c158010 2024-03-04T08:27:49.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb21be00640 1 -- 172.21.15.113:0/979843981 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb200003530 con 0x7fb21c158010 2024-03-04T08:27:49.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb21be00640 1 -- 172.21.15.113:0/979843981 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 860700441 0 0) 0x7fb200003530 con 0x7fb21c158010 2024-03-04T08:27:49.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb221600640 1 -- 172.21.15.113:0/979843981 >> v1:172.21.15.136:6789/0 conn(0x7fb21c06ff20 legacy=0x7fb21c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:49.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb221600640 1 -- 172.21.15.113:0/979843981 >> v1:172.21.15.113:6789/0 conn(0x7fb21c158010 legacy=0x7fb21c15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:49.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.867+0000 7fb221600640 1 -- 172.21.15.113:0/979843981 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c06c590 con 0x7fb21c06dd50 2024-03-04T08:27:49.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb221600640 1 --2- 172.21.15.113:0/979843981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb21c06dd50 0x7fb21c1662e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb214009d90 tx=0x7fb21401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1262dfb8e910aafd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:49.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb21be00640 1 -- 172.21.15.113:0/979843981 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c001680 con 0x7fb21c06dd50 2024-03-04T08:27:49.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb21be00640 1 -- 172.21.15.113:0/979843981 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2140a64e0 con 0x7fb21c06dd50 2024-03-04T08:27:49.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb21be00640 1 -- 172.21.15.113:0/979843981 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140a6890 con 0x7fb21c06dd50 2024-03-04T08:27:49.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb223600640 1 -- 172.21.15.113:0/979843981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb21c06dd50 msgr2=0x7fb21c1662e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:49.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb223600640 1 --2- 172.21.15.113:0/979843981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb21c06dd50 0x7fb21c1662e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb214009d90 tx=0x7fb21401c890 comp rx=0 tx=0).stop 2024-03-04T08:27:49.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb223600640 1 -- 172.21.15.113:0/979843981 shutdown_connections 2024-03-04T08:27:49.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb223600640 1 --2- 172.21.15.113:0/979843981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb21c06dd50 0x7fb21c1662e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:49.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb223600640 1 -- 172.21.15.113:0/979843981 >> 172.21.15.113:0/979843981 conn(0x7fb21c0734d0 msgr2=0x7fb21c066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:49.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb223600640 1 -- 172.21.15.113:0/979843981 shutdown_connections 2024-03-04T08:27:49.870 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.871+0000 7fb223600640 1 -- 172.21.15.113:0/979843981 wait complete. 2024-03-04T08:27:49.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb223600640 1 Processor -- start 2024-03-04T08:27:49.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb223600640 1 -- start start 2024-03-04T08:27:49.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb223600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb21c06dd50 0x7fb21c14bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:49.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb223600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb21c06ff20 0x7fb21c146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:49.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb223600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21c158010 0x7fb21c147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:49.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb223600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c15b920 con 0x7fb21c158010 2024-03-04T08:27:49.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb223600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c15ad80 con 0x7fb21c06dd50 2024-03-04T08:27:49.874 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb223600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21c1583f0 con 0x7fb21c06ff20 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb221600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21c158010 0x7fb21c147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb221600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21c158010 0x7fb21c147050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55818/0 (socket says 172.21.15.113:55818) 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb221600640 1 -- 172.21.15.113:0/2411498233 learned_addr learned my addr 172.21.15.113:0/2411498233 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb220c00640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb21c06dd50 0x7fb21c14bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb213e00640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb21c06ff20 0x7fb21c146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb221600640 1 -- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb21c06ff20 msgr2=0x7fb21c146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb221600640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb21c06ff20 0x7fb21c146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb221600640 1 -- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb21c06dd50 msgr2=0x7fb21c14bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb221600640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb21c06dd50 0x7fb21c14bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb221600640 1 -- 172.21.15.113:0/2411498233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21c1f8040 con 0x7fb21c158010 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb221600640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21c158010 0x7fb21c147050 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fb21401a4c0 tx=0x7fb214019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:49.875 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb219600640 1 -- 172.21.15.113:0/2411498233 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140bd070 con 0x7fb21c158010 2024-03-04T08:27:49.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb219600640 1 -- 172.21.15.113:0/2411498233 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2140a69a0 con 0x7fb21c158010 2024-03-04T08:27:49.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb219600640 1 -- 172.21.15.113:0/2411498233 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140a6d70 con 0x7fb21c158010 2024-03-04T08:27:49.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb223600640 1 -- 172.21.15.113:0/2411498233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21c0cd780 con 0x7fb21c158010 2024-03-04T08:27:49.876 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.875+0000 7fb223600640 1 -- 172.21.15.113:0/2411498233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21c0cdd40 con 0x7fb21c158010 2024-03-04T08:27:49.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.879+0000 7fb223600640 1 -- 172.21.15.113:0/2411498233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb21c075c00 con 0x7fb21c158010 2024-03-04T08:27:49.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.879+0000 7fb219600640 1 -- 172.21.15.113:0/2411498233 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2140c0070 con 0x7fb21c158010 2024-03-04T08:27:49.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.879+0000 7fb219600640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f8066f60 0x7fb1f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:49.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.879+0000 7fb219600640 1 -- 172.21.15.113:0/2411498233 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fb214120900 con 0x7fb21c158010 2024-03-04T08:27:49.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.879+0000 7fb220c00640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f8066f60 0x7fb1f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:49.877 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.879+0000 7fb220c00640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f8066f60 0x7fb1f8069420 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb20c005890 tx=0x7fb20c005800 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:49.881 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:49.883+0000 7fb219600640 1 -- 172.21.15.113:0/2411498233 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb2140c7070 con 0x7fb21c158010 2024-03-04T08:27:50.070 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.071+0000 7fb223600640 1 -- 172.21.15.113:0/2411498233 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb21c071770 con 0x7fb21c158010 2024-03-04T08:27:50.071 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb219600640 1 -- 172.21.15.113:0/2411498233 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fb214016480 con 0x7fb21c158010 2024-03-04T08:27:50.071 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:50.071 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:50.071 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 -- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f8066f60 msgr2=0x7fb1f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f8066f60 0x7fb1f8069420 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fb20c005890 tx=0x7fb20c005800 comp rx=0 tx=0).stop 2024-03-04T08:27:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 -- 172.21.15.113:0/2411498233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21c158010 msgr2=0x7fb21c147050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21c158010 0x7fb21c147050 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fb21401a4c0 tx=0x7fb214019630 comp rx=0 tx=0).stop 2024-03-04T08:27:50.073 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 -- 172.21.15.113:0/2411498233 shutdown_connections 2024-03-04T08:27:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb1f8066f60 0x7fb1f8069420 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb21c158010 0x7fb21c147050 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb21c06ff20 0x7fb21c146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 --2- 172.21.15.113:0/2411498233 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb21c06dd50 0x7fb21c14bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 -- 172.21.15.113:0/2411498233 >> 172.21.15.113:0/2411498233 conn(0x7fb21c0734d0 msgr2=0x7fb21c066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 -- 172.21.15.113:0/2411498233 shutdown_connections 2024-03-04T08:27:50.074 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:50.075+0000 7fb223600640 1 -- 172.21.15.113:0/2411498233 wait complete. 2024-03-04T08:27:50.092 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:27:50.092 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:27:52.093 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:52.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f1954400640 1 Processor -- start 2024-03-04T08:27:52.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f1954400640 1 -- start start 2024-03-04T08:27:52.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f1954400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f194c1527f0 0x7f194c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:52.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f1954400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c1591f0 con 0x7f194c0c8200 2024-03-04T08:27:52.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f1954400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c1594d0 con 0x7f194c152bd0 2024-03-04T08:27:52.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f1954400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c071df0 con 0x7f194c1527f0 2024-03-04T08:27:52.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f1951000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f194c0c8200 0x7f194c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37986/0 (socket says 172.21.15.113:37986) 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f1951000640 1 -- 172.21.15.113:0/2500278692 learned_addr learned my addr 172.21.15.113:0/2500278692 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f1951a00640 1 --2- 172.21.15.113:0/2500278692 >> v2:172.21.15.136:3300/0 conn(0x7f194c1527f0 0x7f194c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166513460 0 0) 0x7f194c1591f0 con 0x7f194c0c8200 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1934003530 con 0x7f194c0c8200 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3184831990 0 0) 0x7f1934003530 con 0x7f194c0c8200 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 >> v2:172.21.15.136:3300/0 conn(0x7f194c1527f0 msgr2=0x7f194c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f194be00640 1 --2- 172.21.15.113:0/2500278692 >> v2:172.21.15.136:3300/0 conn(0x7f194c1527f0 0x7f194c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 >> v1:172.21.15.136:6789/0 conn(0x7f194c152bd0 legacy=0x7f194c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0717a0 con 0x7f194c0c8200 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.243+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f1938001680 con 0x7f194c0c8200 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 >> v1:172.21.15.113:6789/0 conn(0x7f194c0c8200 legacy=0x7f194c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f194be00640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19340039c0 0x7f1934003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f194be00640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934004810 0x7f1934004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f194be00640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19340051d0 0x7f193400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c071df0 con 0x7f1934004810 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193400c070 con 0x7f19340051d0 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193400c270 con 0x7f19340039c0 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f1951000640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934004810 0x7f1934004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f1951a00640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19340039c0 0x7f1934003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f1952400640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19340051d0 0x7f193400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f1951000640 1 -- 172.21.15.113:0/2500278692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19340039c0 msgr2=0x7f1934003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f1951000640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19340039c0 0x7f1934003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f1951000640 1 -- 172.21.15.113:0/2500278692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19340051d0 msgr2=0x7f193400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:52.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f1951000640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19340051d0 0x7f193400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f1951000640 1 -- 172.21.15.113:0/2500278692 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934003530 con 0x7f1934004810 2024-03-04T08:27:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f1952400640 1 -- 172.21.15.113:0/2500278692 reap_dead start 2024-03-04T08:27:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f1951000640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934004810 0x7f1934004c90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f193800c190 tx=0x7f193801f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fab036aca4d730 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938007d00 con 0x7f1934004810 2024-03-04T08:27:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1938005b00 con 0x7f1934004810 2024-03-04T08:27:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.247+0000 7f194be00640 1 -- 172.21.15.113:0/2500278692 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938007070 con 0x7f1934004810 2024-03-04T08:27:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 -- 172.21.15.113:0/2500278692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934004810 msgr2=0x7f1934004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934004810 0x7f1934004c90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f193800c190 tx=0x7f193801f390 comp rx=0 tx=0).stop 2024-03-04T08:27:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 -- 172.21.15.113:0/2500278692 shutdown_connections 2024-03-04T08:27:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 --2- 172.21.15.113:0/2500278692 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1934004810 0x7f1934004c90 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 -- 172.21.15.113:0/2500278692 >> 172.21.15.113:0/2500278692 conn(0x7f194c06e820 msgr2=0x7f194c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 -- 172.21.15.113:0/2500278692 shutdown_connections 2024-03-04T08:27:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 -- 172.21.15.113:0/2500278692 wait complete. 2024-03-04T08:27:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 Processor -- start 2024-03-04T08:27:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 -- start start 2024-03-04T08:27:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1934004810 0x7f194c0c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f194c1570f0 0x7f194c0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194c151740 0x7f194c156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c070d40 con 0x7f194c1570f0 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c1597b0 con 0x7f194c151740 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1954400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c070ec0 con 0x7f1934004810 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f194c1570f0 0x7f194c0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f194c1570f0 0x7f194c0c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55848/0 (socket says 172.21.15.113:55848) 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951000640 1 -- 172.21.15.113:0/814369359 learned_addr learned my addr 172.21.15.113:0/814369359 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951a00640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1934004810 0x7f194c0c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1952400640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194c151740 0x7f194c156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:52.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951000640 1 -- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1934004810 msgr2=0x7f194c0c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951000640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1934004810 0x7f194c0c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951000640 1 -- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194c151740 msgr2=0x7f194c156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951000640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194c151740 0x7f194c156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951000640 1 -- 172.21.15.113:0/814369359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0c75d0 con 0x7f194c1570f0 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951a00640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1934004810 0x7f194c0c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.251+0000 7f1951000640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f194c1570f0 0x7f194c0c3550 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f193801f1c0 tx=0x7f1934005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.255+0000 7f1949600640 1 -- 172.21.15.113:0/814369359 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938007d00 con 0x7f194c1570f0 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.255+0000 7f1949600640 1 -- 172.21.15.113:0/814369359 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1938000ee0 con 0x7f194c1570f0 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.255+0000 7f1954400640 1 -- 172.21.15.113:0/814369359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c0c72f0 con 0x7f194c1570f0 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.255+0000 7f1949600640 1 -- 172.21.15.113:0/814369359 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938001480 con 0x7f194c1570f0 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.255+0000 7f1954400640 1 -- 172.21.15.113:0/814369359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c1ecc70 con 0x7f194c1570f0 2024-03-04T08:27:52.254 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.255+0000 7f1954400640 1 -- 172.21.15.113:0/814369359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c0c62d0 con 0x7f194c1570f0 2024-03-04T08:27:52.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.255+0000 7f1949600640 1 -- 172.21.15.113:0/814369359 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19380047f0 con 0x7f194c1570f0 2024-03-04T08:27:52.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.255+0000 7f1949600640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f192c066f60 0x7f192c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:52.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.255+0000 7f1949600640 1 -- 172.21.15.113:0/814369359 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f1938123150 con 0x7f194c1570f0 2024-03-04T08:27:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.255+0000 7f1951a00640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f192c066f60 0x7f192c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.259+0000 7f1951a00640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f192c066f60 0x7f192c069420 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f194c050c80 tx=0x7f1940008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:52.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.259+0000 7f1949600640 1 -- 172.21.15.113:0/814369359 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f194c0c75d0 con 0x7f194c1570f0 2024-03-04T08:27:52.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 -- 172.21.15.113:0/814369359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f194c075c00 con 0x7f194c1570f0 2024-03-04T08:27:52.447 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1949600640 1 -- 172.21.15.113:0/814369359 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f19380190a0 con 0x7f194c1570f0 2024-03-04T08:27:52.448 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:52.448 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:52.448 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:52.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 -- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f192c066f60 msgr2=0x7f192c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:52.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f192c066f60 0x7f192c069420 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f194c050c80 tx=0x7f1940008040 comp rx=0 tx=0).stop 2024-03-04T08:27:52.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 -- 172.21.15.113:0/814369359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f194c1570f0 msgr2=0x7f194c0c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:52.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f194c1570f0 0x7f194c0c3550 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f193801f1c0 tx=0x7f1934005840 comp rx=0 tx=0).stop 2024-03-04T08:27:52.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 -- 172.21.15.113:0/814369359 shutdown_connections 2024-03-04T08:27:52.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f194c151740 0x7f194c156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:52.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f192c066f60 0x7f192c069420 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:52.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f194c1570f0 0x7f194c0c3550 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:52.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 --2- 172.21.15.113:0/814369359 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1934004810 0x7f194c0c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:52.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 -- 172.21.15.113:0/814369359 >> 172.21.15.113:0/814369359 conn(0x7f194c06e820 msgr2=0x7f194c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:52.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 -- 172.21.15.113:0/814369359 shutdown_connections 2024-03-04T08:27:52.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:52.451+0000 7f1954400640 1 -- 172.21.15.113:0/814369359 wait complete. 2024-03-04T08:27:52.469 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:54.470 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:54.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.619+0000 7f287d200640 1 Processor -- start 2024-03-04T08:27:54.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.619+0000 7f287d200640 1 -- start start 2024-03-04T08:27:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f287d200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f287814e4c0 0x7f2878072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f287d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28780c22e0 0x7f28780731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f287d200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28780c27f0 con 0x7f28780c2d80 2024-03-04T08:27:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f287d200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287815ad60 con 0x7f287814e4c0 2024-03-04T08:27:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f287d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28780c2a90 con 0x7f28780c22e0 2024-03-04T08:27:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f2875e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f287814e4c0 0x7f2878072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f2875e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f287814e4c0 0x7f2878072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55856/0 (socket says 172.21.15.113:55856) 2024-03-04T08:27:54.619 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f2875e00640 1 -- 172.21.15.113:0/635820029 learned_addr learned my addr 172.21.15.113:0/635820029 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f2875e00640 1 -- 172.21.15.113:0/635820029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28780c22e0 msgr2=0x7f28780731c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-04T08:27:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f2875e00640 1 --2- 172.21.15.113:0/635820029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28780c22e0 0x7f28780731c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f2875e00640 1 -- 172.21.15.113:0/635820029 >> v1:172.21.15.136:6789/0 conn(0x7f28780c2d80 legacy=0x7f287815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:54.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f2875e00640 1 -- 172.21.15.113:0/635820029 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2878070c60 con 0x7f287814e4c0 2024-03-04T08:27:54.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.623+0000 7f2875e00640 1 --2- 172.21.15.113:0/635820029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 0x7f2878072c80 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2868009870 tx=0x7f286801ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b581761fb9812c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:54.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f2874a00640 1 -- 172.21.15.113:0/635820029 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680b5070 con 0x7f287814e4c0 2024-03-04T08:27:54.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f2874a00640 1 -- 172.21.15.113:0/635820029 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28680a8a70 con 0x7f287814e4c0 2024-03-04T08:27:54.623 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f2874a00640 1 -- 172.21.15.113:0/635820029 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680a8e40 con 0x7f287814e4c0 2024-03-04T08:27:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 -- 172.21.15.113:0/635820029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 msgr2=0x7f2878072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 --2- 172.21.15.113:0/635820029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 0x7f2878072c80 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2868009870 tx=0x7f286801ed70 comp rx=0 tx=0).stop 2024-03-04T08:27:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 -- 172.21.15.113:0/635820029 shutdown_connections 2024-03-04T08:27:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 --2- 172.21.15.113:0/635820029 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28780c22e0 0x7f28780731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 --2- 172.21.15.113:0/635820029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 0x7f2878072c80 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 -- 172.21.15.113:0/635820029 >> 172.21.15.113:0/635820029 conn(0x7f287806e820 msgr2=0x7f287806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 -- 172.21.15.113:0/635820029 shutdown_connections 2024-03-04T08:27:54.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 -- 172.21.15.113:0/635820029 wait complete. 2024-03-04T08:27:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 Processor -- start 2024-03-04T08:27:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 -- start start 2024-03-04T08:27:54.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28780c22e0 0x7f2878145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:54.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28780c2d80 0x7f28781f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:54.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 0x7f287813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:54.627 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2878071500 con 0x7f287814e4c0 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2878071220 con 0x7f28780c22e0 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f287d200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287814e8a0 con 0x7f28780c2d80 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f2876800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 0x7f287813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f2876800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 0x7f287813ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55858/0 (socket says 172.21.15.113:55858) 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f2876800640 1 -- 172.21.15.113:0/675484863 learned_addr learned my addr 172.21.15.113:0/675484863 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.627+0000 7f2875e00640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28780c22e0 0x7f2878145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f2875400640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28780c2d80 0x7f28781f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f2876800640 1 -- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28780c2d80 msgr2=0x7f28781f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f2876800640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28780c2d80 0x7f28781f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f2876800640 1 -- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28780c22e0 msgr2=0x7f2878145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f2876800640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28780c22e0 0x7f2878145370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:54.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f2876800640 1 -- 172.21.15.113:0/675484863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28780b0fa0 con 0x7f287814e4c0 2024-03-04T08:27:54.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f2875400640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28780c2d80 0x7f28781f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:27:54.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f2876800640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 0x7f287813ed90 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f287000c7d0 tx=0x7f287000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:54.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f286e000640 1 -- 172.21.15.113:0/675484863 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870003b80 con 0x7f287814e4c0 2024-03-04T08:27:54.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f286e000640 1 -- 172.21.15.113:0/675484863 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2870003d20 con 0x7f287814e4c0 2024-03-04T08:27:54.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f287d200640 1 -- 172.21.15.113:0/675484863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28780afd60 con 0x7f287814e4c0 2024-03-04T08:27:54.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f286e000640 1 -- 172.21.15.113:0/675484863 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287000aa30 con 0x7f287814e4c0 2024-03-04T08:27:54.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f287d200640 1 -- 172.21.15.113:0/675484863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2878148fe0 con 0x7f287814e4c0 2024-03-04T08:27:54.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f287d200640 1 -- 172.21.15.113:0/675484863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2878075c00 con 0x7f287814e4c0 2024-03-04T08:27:54.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f286e000640 1 -- 172.21.15.113:0/675484863 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f287000abd0 con 0x7f287814e4c0 2024-03-04T08:27:54.630 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f286e000640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2854066f60 0x7f2854069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:54.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.631+0000 7f286e000640 1 -- 172.21.15.113:0/675484863 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f28700859e0 con 0x7f287814e4c0 2024-03-04T08:27:54.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.635+0000 7f2875e00640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2854066f60 0x7f2854069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:54.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.635+0000 7f286e000640 1 -- 172.21.15.113:0/675484863 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f28780b0fa0 con 0x7f287814e4c0 2024-03-04T08:27:54.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.639+0000 7f2875e00640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2854066f60 0x7f2854069420 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f28680b2000 tx=0x7f2868008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:54.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 -- 172.21.15.113:0/675484863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2878155190 con 0x7f287814e4c0 2024-03-04T08:27:54.823 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f286e000640 1 -- 172.21.15.113:0/675484863 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f2870055f10 con 0x7f287814e4c0 2024-03-04T08:27:54.824 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:54.824 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:54.824 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:54.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 -- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2854066f60 msgr2=0x7f2854069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:54.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2854066f60 0x7f2854069420 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f28680b2000 tx=0x7f2868008040 comp rx=0 tx=0).stop 2024-03-04T08:27:54.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 -- 172.21.15.113:0/675484863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 msgr2=0x7f287813ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:54.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 0x7f287813ed90 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f287000c7d0 tx=0x7f287000cc90 comp rx=0 tx=0).stop 2024-03-04T08:27:54.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 -- 172.21.15.113:0/675484863 shutdown_connections 2024-03-04T08:27:54.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2854066f60 0x7f2854069420 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:54.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f287814e4c0 0x7f287813ed90 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:54.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f28780c2d80 0x7f28781f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:54.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 --2- 172.21.15.113:0/675484863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f28780c22e0 0x7f2878145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:54.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 -- 172.21.15.113:0/675484863 >> 172.21.15.113:0/675484863 conn(0x7f287806e820 msgr2=0x7f28780c1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:54.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 -- 172.21.15.113:0/675484863 shutdown_connections 2024-03-04T08:27:54.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:54.827+0000 7f287d200640 1 -- 172.21.15.113:0/675484863 wait complete. 2024-03-04T08:27:54.845 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:56.846 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:56.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4ea800640 1 Processor -- start 2024-03-04T08:27:56.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4ea800640 1 -- start start 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4ea800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4e414e4c0 0x7fe4e4072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4ea800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4e40c22e0 0x7fe4e40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4ea800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e40c27f0 con 0x7fe4e40c2d80 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4ea800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e415ad60 con 0x7fe4e414e4c0 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4ea800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40c2a90 con 0x7fe4e40c22e0 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4e3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4e414e4c0 0x7fe4e4072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4e3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe4e414e4c0 0x7fe4e4072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55874/0 (socket says 172.21.15.113:55874) 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4e3400640 1 -- 172.21.15.113:0/1135711956 learned_addr learned my addr 172.21.15.113:0/1135711956 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4e2a00640 1 --2- 172.21.15.113:0/1135711956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4e40c22e0 0x7fe4e40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4e3400640 1 -- 172.21.15.113:0/1135711956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4e40c22e0 msgr2=0x7fe4e40731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:56.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4e3400640 1 --2- 172.21.15.113:0/1135711956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4e40c22e0 0x7fe4e40731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:56.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4e3400640 1 -- 172.21.15.113:0/1135711956 >> v1:172.21.15.136:6789/0 conn(0x7fe4e40c2d80 legacy=0x7fe4e415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:56.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.995+0000 7fe4e3400640 1 -- 172.21.15.113:0/1135711956 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e4070c60 con 0x7fe4e414e4c0 2024-03-04T08:27:56.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4e3400640 1 --2- 172.21.15.113:0/1135711956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e414e4c0 0x7fe4e4072c80 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009e50 tx=0x7fe4d401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d5839b3c13b83f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:56.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4e2000640 1 -- 172.21.15.113:0/1135711956 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40b4070 con 0x7fe4e414e4c0 2024-03-04T08:27:56.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4e2000640 1 -- 172.21.15.113:0/1135711956 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d40af400 con 0x7fe4e414e4c0 2024-03-04T08:27:56.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4e2000640 1 -- 172.21.15.113:0/1135711956 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40af7d0 con 0x7fe4e414e4c0 2024-03-04T08:27:56.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4ea800640 1 -- 172.21.15.113:0/1135711956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e414e4c0 msgr2=0x7fe4e4072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:56.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4ea800640 1 --2- 172.21.15.113:0/1135711956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e414e4c0 0x7fe4e4072c80 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009e50 tx=0x7fe4d401f210 comp rx=0 tx=0).stop 2024-03-04T08:27:56.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4ea800640 1 -- 172.21.15.113:0/1135711956 shutdown_connections 2024-03-04T08:27:56.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4ea800640 1 --2- 172.21.15.113:0/1135711956 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4e40c22e0 0x7fe4e40731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:56.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4ea800640 1 --2- 172.21.15.113:0/1135711956 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e414e4c0 0x7fe4e4072c80 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:56.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4ea800640 1 -- 172.21.15.113:0/1135711956 >> 172.21.15.113:0/1135711956 conn(0x7fe4e406e820 msgr2=0x7fe4e406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:56.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4ea800640 1 -- 172.21.15.113:0/1135711956 shutdown_connections 2024-03-04T08:27:56.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:56.999+0000 7fe4ea800640 1 -- 172.21.15.113:0/1135711956 wait complete. 2024-03-04T08:27:56.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4ea800640 1 Processor -- start 2024-03-04T08:27:56.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4ea800640 1 -- start start 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4ea800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e40c22e0 0x7fe4e4152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4ea800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4e40c2d80 0x7fe4e41f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4ea800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4e414e4c0 0x7fe4e4142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4ea800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e4071500 con 0x7fe4e40c22e0 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4ea800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e4071220 con 0x7fe4e40c2d80 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4ea800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e414e8a0 con 0x7fe4e414e4c0 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e40c22e0 0x7fe4e4152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e40c22e0 0x7fe4e4152060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:55882/0 (socket says 172.21.15.113:55882) 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e3400640 1 -- 172.21.15.113:0/2362342757 learned_addr learned my addr 172.21.15.113:0/2362342757 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e2a00640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4e40c2d80 0x7fe4e41f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:57.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e3e00640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4e414e4c0 0x7fe4e4142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e3400640 1 -- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4e414e4c0 msgr2=0x7fe4e4142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e3400640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4e414e4c0 0x7fe4e4142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e3400640 1 -- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4e40c2d80 msgr2=0x7fe4e41f5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e3400640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4e40c2d80 0x7fe4e41f5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e3400640 1 -- 172.21.15.113:0/2362342757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e4165110 con 0x7fe4e40c22e0 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4e3400640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e40c22e0 0x7fe4e4152060 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe4d401d8c0 tx=0x7fe4d401bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4db400640 1 -- 172.21.15.113:0/2362342757 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40c0070 con 0x7fe4e40c22e0 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4db400640 1 -- 172.21.15.113:0/2362342757 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d40afe10 con 0x7fe4e40c22e0 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4ea800640 1 -- 172.21.15.113:0/2362342757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e4163310 con 0x7fe4e40c22e0 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4db400640 1 -- 172.21.15.113:0/2362342757 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40b4040 con 0x7fe4e40c22e0 2024-03-04T08:27:57.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.003+0000 7fe4ea800640 1 -- 172.21.15.113:0/2362342757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e41f4910 con 0x7fe4e40c22e0 2024-03-04T08:27:57.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.007+0000 7fe4ea800640 1 -- 172.21.15.113:0/2362342757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e41f4700 con 0x7fe4e40c22e0 2024-03-04T08:27:57.004 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.007+0000 7fe4db400640 1 -- 172.21.15.113:0/2362342757 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4d40c3070 con 0x7fe4e40c22e0 2024-03-04T08:27:57.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.007+0000 7fe4db400640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4c4066f90 0x7fe4c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:57.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.007+0000 7fe4db400640 1 -- 172.21.15.113:0/2362342757 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fe4d41231c0 con 0x7fe4e40c22e0 2024-03-04T08:27:57.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.007+0000 7fe4e2a00640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4c4066f90 0x7fe4c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:57.005 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.007+0000 7fe4e2a00640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4c4066f90 0x7fe4c4069450 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40cba20 tx=0x7fe4d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:57.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.011+0000 7fe4db400640 1 -- 172.21.15.113:0/2362342757 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe4d4018d70 con 0x7fe4e40c22e0 2024-03-04T08:27:57.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 -- 172.21.15.113:0/2362342757 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4e4075c00 con 0x7fe4e40c22e0 2024-03-04T08:27:57.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4db400640 1 -- 172.21.15.113:0/2362342757 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fe4d40f36c0 con 0x7fe4e40c22e0 2024-03-04T08:27:57.196 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:57.196 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:57.196 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:57.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 -- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4c4066f90 msgr2=0x7fe4c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:57.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4c4066f90 0x7fe4c4069450 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40cba20 tx=0x7fe4d0008040 comp rx=0 tx=0).stop 2024-03-04T08:27:57.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 -- 172.21.15.113:0/2362342757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e40c22e0 msgr2=0x7fe4e4152060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:57.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e40c22e0 0x7fe4e4152060 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe4d401d8c0 tx=0x7fe4d401bfb0 comp rx=0 tx=0).stop 2024-03-04T08:27:57.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 -- 172.21.15.113:0/2362342757 shutdown_connections 2024-03-04T08:27:57.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe4e414e4c0 0x7fe4e4142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:57.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe4e40c2d80 0x7fe4e41f5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:57.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe4c4066f90 0x7fe4c4069450 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 --2- 172.21.15.113:0/2362342757 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4e40c22e0 0x7fe4e4152060 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 -- 172.21.15.113:0/2362342757 >> 172.21.15.113:0/2362342757 conn(0x7fe4e406e820 msgr2=0x7fe4e40c8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 -- 172.21.15.113:0/2362342757 shutdown_connections 2024-03-04T08:27:57.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:57.199+0000 7fe4ea800640 1 -- 172.21.15.113:0/2362342757 wait complete. 2024-03-04T08:27:57.217 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:27:59.218 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:27:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.366+0000 7f0a55000640 1 Processor -- start 2024-03-04T08:27:59.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.366+0000 7f0a55000640 1 -- start start 2024-03-04T08:27:59.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.366+0000 7f0a55000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c22e0 0x7f0a5015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:59.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.366+0000 7f0a55000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f0a500c2d80 0x7f0a5015b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:59.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.366+0000 7f0a55000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a500c27f0 con 0x7f0a5014e4c0 2024-03-04T08:27:59.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.366+0000 7f0a55000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5015ad60 con 0x7f0a500c2d80 2024-03-04T08:27:59.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.366+0000 7f0a55000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a500c2a90 con 0x7f0a500c22e0 2024-03-04T08:27:59.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4de00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f0a5014e4c0 0x7f0a50072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:46010/0 (socket says 172.21.15.113:46010) 2024-03-04T08:27:59.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4de00640 1 -- 172.21.15.113:0/363061153 learned_addr learned my addr 172.21.15.113:0/363061153 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:27:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4e800640 1 --2- 172.21.15.113:0/363061153 >> v2:172.21.15.136:3300/0 conn(0x7f0a500c2d80 0x7f0a5015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4d400640 1 --2- 172.21.15.113:0/363061153 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c22e0 0x7f0a5015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4ca00640 1 -- 172.21.15.113:0/363061153 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852045887 0 0) 0x7f0a500c27f0 con 0x7f0a5014e4c0 2024-03-04T08:27:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4ca00640 1 -- 172.21.15.113:0/363061153 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a34003530 con 0x7f0a5014e4c0 2024-03-04T08:27:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4d400640 1 -- 172.21.15.113:0/363061153 >> v2:172.21.15.136:3300/0 conn(0x7f0a500c2d80 msgr2=0x7f0a5015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:27:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4d400640 1 --2- 172.21.15.113:0/363061153 >> v2:172.21.15.136:3300/0 conn(0x7f0a500c2d80 0x7f0a5015b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4d400640 1 -- 172.21.15.113:0/363061153 >> v1:172.21.15.136:6789/0 conn(0x7f0a5014e4c0 legacy=0x7f0a50072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4d400640 1 -- 172.21.15.113:0/363061153 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a50070c60 con 0x7f0a500c22e0 2024-03-04T08:27:59.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4e800640 1 --2- 172.21.15.113:0/363061153 >> v2:172.21.15.136:3300/0 conn(0x7f0a500c2d80 0x7f0a5015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:27:59.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.370+0000 7f0a4d400640 1 --2- 172.21.15.113:0/363061153 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c22e0 0x7f0a5015b1f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0097c0 tx=0x7f0a3c01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93626b2577498c10 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:59.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a4ca00640 1 -- 172.21.15.113:0/363061153 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c0b5070 con 0x7f0a500c22e0 2024-03-04T08:27:59.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a4ca00640 1 -- 172.21.15.113:0/363061153 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c0a8990 con 0x7f0a500c22e0 2024-03-04T08:27:59.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a4ca00640 1 -- 172.21.15.113:0/363061153 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c0a8d40 con 0x7f0a500c22e0 2024-03-04T08:27:59.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 -- 172.21.15.113:0/363061153 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c22e0 msgr2=0x7f0a5015b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:59.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 --2- 172.21.15.113:0/363061153 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c22e0 0x7f0a5015b1f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0097c0 tx=0x7f0a3c01ec90 comp rx=0 tx=0).stop 2024-03-04T08:27:59.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 -- 172.21.15.113:0/363061153 shutdown_connections 2024-03-04T08:27:59.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 --2- 172.21.15.113:0/363061153 >> v2:172.21.15.136:3300/0 conn(0x7f0a500c2d80 0x7f0a5015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:59.371 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 --2- 172.21.15.113:0/363061153 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c22e0 0x7f0a5015b1f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:59.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 -- 172.21.15.113:0/363061153 >> 172.21.15.113:0/363061153 conn(0x7f0a5006e820 msgr2=0x7f0a5006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:59.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 -- 172.21.15.113:0/363061153 shutdown_connections 2024-03-04T08:27:59.372 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 -- 172.21.15.113:0/363061153 wait complete. 2024-03-04T08:27:59.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 Processor -- start 2024-03-04T08:27:59.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 -- start start 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a500c22e0 0x7f0a500cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c2d80 0x7f0a50152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a5014e4c0 0x7f0a50152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a50071500 con 0x7f0a500c22e0 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a50071220 con 0x7f0a5014e4c0 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a55000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5014e8a0 con 0x7f0a500c2d80 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a4de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a500c22e0 0x7f0a500cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a4de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a500c22e0 0x7f0a500cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60322/0 (socket says 172.21.15.113:60322) 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a4de00640 1 -- 172.21.15.113:0/1478794514 learned_addr learned my addr 172.21.15.113:0/1478794514 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a4e800640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a5014e4c0 0x7f0a50152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:59.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.374+0000 7f0a4d400640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c2d80 0x7f0a50152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a4de00640 1 -- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c2d80 msgr2=0x7f0a50152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a4de00640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c2d80 0x7f0a50152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a4de00640 1 -- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a5014e4c0 msgr2=0x7f0a50152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a4de00640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a5014e4c0 0x7f0a50152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a4de00640 1 -- 172.21.15.113:0/1478794514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a500c6110 con 0x7f0a500c22e0 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a4e800640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a5014e4c0 0x7f0a50152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a4de00640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a500c22e0 0x7f0a500cb350 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000cb40 tx=0x7f0a4000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a46000640 1 -- 172.21.15.113:0/1478794514 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40017070 con 0x7f0a500c22e0 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a46000640 1 -- 172.21.15.113:0/1478794514 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4000aca0 con 0x7f0a500c22e0 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a46000640 1 -- 172.21.15.113:0/1478794514 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40012620 con 0x7f0a500c22e0 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a55000640 1 -- 172.21.15.113:0/1478794514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a500c63f0 con 0x7f0a500c22e0 2024-03-04T08:27:59.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a55000640 1 -- 172.21.15.113:0/1478794514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a50152590 con 0x7f0a500c22e0 2024-03-04T08:27:59.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a55000640 1 -- 172.21.15.113:0/1478794514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a50075c00 con 0x7f0a500c22e0 2024-03-04T08:27:59.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a46000640 1 -- 172.21.15.113:0/1478794514 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a4001a070 con 0x7f0a500c22e0 2024-03-04T08:27:59.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a46000640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a30066f90 0x7f0a30069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:27:59.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a46000640 1 -- 172.21.15.113:0/1478794514 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f0a40085920 con 0x7f0a500c22e0 2024-03-04T08:27:59.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.378+0000 7f0a4d400640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a30066f90 0x7f0a30069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:27:59.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.382+0000 7f0a4d400640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a30066f90 0x7f0a30069450 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0b2000 tx=0x7f0a3c0aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:27:59.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.382+0000 7f0a46000640 1 -- 172.21.15.113:0/1478794514 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0a40055e50 con 0x7f0a500c22e0 2024-03-04T08:27:59.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.570+0000 7f0a55000640 1 -- 172.21.15.113:0/1478794514 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a5014d380 con 0x7f0a500c22e0 2024-03-04T08:27:59.569 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.570+0000 7f0a46000640 1 -- 172.21.15.113:0/1478794514 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f0a40055750 con 0x7f0a500c22e0 2024-03-04T08:27:59.569 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:27:59.569 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:27:59.570 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:27:59.570 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.570+0000 7f0a55000640 1 -- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a30066f90 msgr2=0x7f0a30069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.570+0000 7f0a55000640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a30066f90 0x7f0a30069450 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0b2000 tx=0x7f0a3c0aa070 comp rx=0 tx=0).stop 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.570+0000 7f0a55000640 1 -- 172.21.15.113:0/1478794514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a500c22e0 msgr2=0x7f0a500cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.570+0000 7f0a55000640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a500c22e0 0x7f0a500cb350 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000cb40 tx=0x7f0a4000a060 comp rx=0 tx=0).stop 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.574+0000 7f0a55000640 1 -- 172.21.15.113:0/1478794514 shutdown_connections 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.574+0000 7f0a55000640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a5014e4c0 0x7f0a50152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.574+0000 7f0a55000640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0a500c2d80 0x7f0a50152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.574+0000 7f0a55000640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0a30066f90 0x7f0a30069450 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.574+0000 7f0a55000640 1 --2- 172.21.15.113:0/1478794514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0a500c22e0 0x7f0a500cb350 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.574+0000 7f0a55000640 1 -- 172.21.15.113:0/1478794514 >> 172.21.15.113:0/1478794514 conn(0x7f0a5006e820 msgr2=0x7f0a500c74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.574+0000 7f0a55000640 1 -- 172.21.15.113:0/1478794514 shutdown_connections 2024-03-04T08:27:59.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:27:59.574+0000 7f0a55000640 1 -- 172.21.15.113:0/1478794514 wait complete. 2024-03-04T08:27:59.590 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:01.591 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.738+0000 7f1628c00640 1 Processor -- start 2024-03-04T08:28:01.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.738+0000 7f1628c00640 1 -- start start 2024-03-04T08:28:01.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1628c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f162415b900 0x7f1624163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:01.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1628c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16240c27f0 con 0x7f162414e4c0 2024-03-04T08:28:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1628c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16240c2a90 con 0x7f16240c22e0 2024-03-04T08:28:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1628c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162415ad60 con 0x7f162415b900 2024-03-04T08:28:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1621e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f162414e4c0 0x7f1624072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35066/0 (socket says 172.21.15.113:35066) 2024-03-04T08:28:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1621e00640 1 -- 172.21.15.113:0/210694017 learned_addr learned my addr 172.21.15.113:0/210694017 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1622800640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f162415b900 0x7f1624163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042494358 0 0) 0x7f16240c27f0 con 0x7f162414e4c0 2024-03-04T08:28:01.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1608003530 con 0x7f162414e4c0 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525689785 0 0) 0x7f1608003530 con 0x7f162414e4c0 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f162415b900 msgr2=0x7f1624163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f162415b900 0x7f1624163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 >> v1:172.21.15.136:6789/0 conn(0x7f16240c22e0 legacy=0x7f162415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1624070c60 con 0x7f162414e4c0 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f160c001680 con 0x7f162414e4c0 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 >> v1:172.21.15.113:6789/0 conn(0x7f162414e4c0 legacy=0x7f1624072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16080039f0 0x7f1608003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1608004810 0x7f1608004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f162415b900 0x7f16080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:01.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160800ba80 con 0x7f16080039f0 2024-03-04T08:28:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160800bc00 con 0x7f1608004810 2024-03-04T08:28:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160800bdb0 con 0x7f162415b900 2024-03-04T08:28:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1621e00640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16080039f0 0x7f1608003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1621400640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1608004810 0x7f1608004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1622800640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f162415b900 0x7f16080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1621e00640 1 -- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f162415b900 msgr2=0x7f16080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1621e00640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f162415b900 0x7f16080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1621e00640 1 -- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1608004810 msgr2=0x7f1608004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1621e00640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1608004810 0x7f1608004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:01.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.742+0000 7f1621e00640 1 -- 172.21.15.113:0/210694017 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1608003530 con 0x7f16080039f0 2024-03-04T08:28:01.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1621e00640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16080039f0 0x7f1608003dd0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f160c00c0e0 tx=0x7f160c01cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8a234699fea933 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:01.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c007c50 con 0x7f16080039f0 2024-03-04T08:28:01.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c0a9020 con 0x7f16080039f0 2024-03-04T08:28:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1620a00640 1 -- 172.21.15.113:0/210694017 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c005b20 con 0x7f16080039f0 2024-03-04T08:28:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1628c00640 1 -- 172.21.15.113:0/210694017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16080039f0 msgr2=0x7f1608003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1628c00640 1 --2- 172.21.15.113:0/210694017 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16080039f0 0x7f1608003dd0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f160c00c0e0 tx=0x7f160c01cb00 comp rx=0 tx=0).stop 2024-03-04T08:28:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1622800640 1 -- 172.21.15.113:0/210694017 reap_dead start 2024-03-04T08:28:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1628c00640 1 -- 172.21.15.113:0/210694017 shutdown_connections 2024-03-04T08:28:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1628c00640 1 -- 172.21.15.113:0/210694017 >> 172.21.15.113:0/210694017 conn(0x7f162406e820 msgr2=0x7f162406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1628c00640 1 -- 172.21.15.113:0/210694017 shutdown_connections 2024-03-04T08:28:01.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1628c00640 1 -- 172.21.15.113:0/210694017 wait complete. 2024-03-04T08:28:01.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1628c00640 1 Processor -- start 2024-03-04T08:28:01.746 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.746+0000 7f1628c00640 1 -- start start 2024-03-04T08:28:01.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1628c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16080039f0 0x7f162414d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:01.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1628c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f162414c070 0x7f162414af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:01.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1628c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16241f3ce0 0x7f16241f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:01.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1628c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1624071500 con 0x7f162414c070 2024-03-04T08:28:01.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1628c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1624071220 con 0x7f16080039f0 2024-03-04T08:28:01.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1628c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162414e8a0 con 0x7f16241f3ce0 2024-03-04T08:28:01.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f162414c070 0x7f162414af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:01.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16080039f0 0x7f162414d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:01.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f162414c070 0x7f162414af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60346/0 (socket says 172.21.15.113:60346) 2024-03-04T08:28:01.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16080039f0 0x7f162414d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:55620/0 (socket says 172.21.15.113:55620) 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621400640 1 -- 172.21.15.113:0/1668555329 learned_addr learned my addr 172.21.15.113:0/1668555329 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1622800640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16241f3ce0 0x7f16241f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621400640 1 -- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16241f3ce0 msgr2=0x7f16241f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621400640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16241f3ce0 0x7f16241f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621400640 1 -- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16080039f0 msgr2=0x7f162414d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621400640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16080039f0 0x7f162414d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621400640 1 -- 172.21.15.113:0/1668555329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162414c5e0 con 0x7f162414c070 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1621400640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f162414c070 0x7f162414af00 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f1614010fa0 tx=0x7f161400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f161a000640 1 -- 172.21.15.113:0/1668555329 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614018020 con 0x7f162414c070 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f161a000640 1 -- 172.21.15.113:0/1668555329 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614005070 con 0x7f162414c070 2024-03-04T08:28:01.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f161a000640 1 -- 172.21.15.113:0/1668555329 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614005460 con 0x7f162414c070 2024-03-04T08:28:01.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1628c00640 1 -- 172.21.15.113:0/1668555329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16241f07d0 con 0x7f162414c070 2024-03-04T08:28:01.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1628c00640 1 -- 172.21.15.113:0/1668555329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16241f0d90 con 0x7f162414c070 2024-03-04T08:28:01.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.750+0000 7f1628c00640 1 -- 172.21.15.113:0/1668555329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1624075c00 con 0x7f162414c070 2024-03-04T08:28:01.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.754+0000 7f161a000640 1 -- 172.21.15.113:0/1668555329 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f161401b020 con 0x7f162414c070 2024-03-04T08:28:01.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.754+0000 7f161a000640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1600066f90 0x7f1600069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:01.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.754+0000 7f161a000640 1 -- 172.21.15.113:0/1668555329 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f1614089110 con 0x7f162414c070 2024-03-04T08:28:01.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.754+0000 7f1621e00640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1600066f90 0x7f1600069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:01.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.754+0000 7f1621e00640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1600066f90 0x7f1600069450 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f160c0aa000 tx=0x7f1624072ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:01.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.758+0000 7f161a000640 1 -- 172.21.15.113:0/1668555329 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1614022020 con 0x7f162414c070 2024-03-04T08:28:01.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 -- 172.21.15.113:0/1668555329 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1624155190 con 0x7f162414c070 2024-03-04T08:28:01.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f161a000640 1 -- 172.21.15.113:0/1668555329 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f1614058ff0 con 0x7f162414c070 2024-03-04T08:28:01.940 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:01.941 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:01.941 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:01.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 -- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1600066f90 msgr2=0x7f1600069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:01.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1600066f90 0x7f1600069450 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f160c0aa000 tx=0x7f1624072ba0 comp rx=0 tx=0).stop 2024-03-04T08:28:01.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 -- 172.21.15.113:0/1668555329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f162414c070 msgr2=0x7f162414af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:01.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f162414c070 0x7f162414af00 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f1614010fa0 tx=0x7f161400e530 comp rx=0 tx=0).stop 2024-03-04T08:28:01.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 -- 172.21.15.113:0/1668555329 shutdown_connections 2024-03-04T08:28:01.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f16241f3ce0 0x7f16241f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:01.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1600066f90 0x7f1600069450 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:01.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f162414c070 0x7f162414af00 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:01.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 --2- 172.21.15.113:0/1668555329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16080039f0 0x7f162414d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:01.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 -- 172.21.15.113:0/1668555329 >> 172.21.15.113:0/1668555329 conn(0x7f162406e820 msgr2=0x7f16240c1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:01.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 -- 172.21.15.113:0/1668555329 shutdown_connections 2024-03-04T08:28:01.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:01.942+0000 7f1628c00640 1 -- 172.21.15.113:0/1668555329 wait complete. 2024-03-04T08:28:01.962 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:28:01.963 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:28:03.964 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:04.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f8515800640 1 Processor -- start 2024-03-04T08:28:04.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f8515800640 1 -- start start 2024-03-04T08:28:04.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f8515800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8510152a50 0x7f85100ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:04.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f8515800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851015c4e0 0x7f85100ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:04.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f8515800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8510152e30 con 0x7f851006ff20 2024-03-04T08:28:04.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f8515800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85100c31b0 con 0x7f8510152a50 2024-03-04T08:28:04.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f8515800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851006f840 con 0x7f851015c4e0 2024-03-04T08:28:04.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f850ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8510152a50 0x7f85100ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:04.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f850ea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8510152a50 0x7f85100ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60356/0 (socket says 172.21.15.113:60356) 2024-03-04T08:28:04.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f850ea00640 1 -- 172.21.15.113:0/4278423240 learned_addr learned my addr 172.21.15.113:0/4278423240 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:04.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f850f400640 1 --2- 172.21.15.113:0/4278423240 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851015c4e0 0x7f85100ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:04.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f850ea00640 1 -- 172.21.15.113:0/4278423240 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851015c4e0 msgr2=0x7f85100ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:04.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f850ea00640 1 --2- 172.21.15.113:0/4278423240 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851015c4e0 0x7f85100ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:04.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f850ea00640 1 -- 172.21.15.113:0/4278423240 >> v1:172.21.15.136:6789/0 conn(0x7f851006ff20 legacy=0x7f851015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:04.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.110+0000 7f850ea00640 1 -- 172.21.15.113:0/4278423240 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851006f280 con 0x7f8510152a50 2024-03-04T08:28:04.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f850ea00640 1 --2- 172.21.15.113:0/4278423240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8510152a50 0x7f85100ce120 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f8500009e50 tx=0x7f850001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f523bbf1a584b5e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:04.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f850e000640 1 -- 172.21.15.113:0/4278423240 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85000a5900 con 0x7f8510152a50 2024-03-04T08:28:04.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f850e000640 1 -- 172.21.15.113:0/4278423240 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85000a5aa0 con 0x7f8510152a50 2024-03-04T08:28:04.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f850e000640 1 -- 172.21.15.113:0/4278423240 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85000ad3f0 con 0x7f8510152a50 2024-03-04T08:28:04.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f8515800640 1 -- 172.21.15.113:0/4278423240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8510152a50 msgr2=0x7f85100ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f8515800640 1 --2- 172.21.15.113:0/4278423240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8510152a50 0x7f85100ce120 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f8500009e50 tx=0x7f850001ca30 comp rx=0 tx=0).stop 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f8515800640 1 -- 172.21.15.113:0/4278423240 shutdown_connections 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f8515800640 1 --2- 172.21.15.113:0/4278423240 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851015c4e0 0x7f85100ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f8515800640 1 --2- 172.21.15.113:0/4278423240 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8510152a50 0x7f85100ce120 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f8515800640 1 -- 172.21.15.113:0/4278423240 >> 172.21.15.113:0/4278423240 conn(0x7f85100734d0 msgr2=0x7f8510066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f8515800640 1 -- 172.21.15.113:0/4278423240 shutdown_connections 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.114+0000 7f8515800640 1 -- 172.21.15.113:0/4278423240 wait complete. 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8515800640 1 Processor -- start 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8515800640 1 -- start start 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8515800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851006ff20 0x7f8510159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8515800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8510152a50 0x7f8510158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8515800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f851015c4e0 0x7f8510158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8515800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510165a00 con 0x7f8510152a50 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8515800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510165880 con 0x7f851015c4e0 2024-03-04T08:28:04.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8515800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8510165b80 con 0x7f851006ff20 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8506000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8510152a50 0x7f8510158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f850ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851006ff20 0x7f8510159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f850ea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851006ff20 0x7f8510159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:39230/0 (socket says 172.21.15.113:39230) 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f850ea00640 1 -- 172.21.15.113:0/982861857 learned_addr learned my addr 172.21.15.113:0/982861857 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f850f400640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f851015c4e0 0x7f8510158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8506000640 1 -- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851006ff20 msgr2=0x7f8510159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8506000640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851006ff20 0x7f8510159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8506000640 1 -- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f851015c4e0 msgr2=0x7f8510158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8506000640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f851015c4e0 0x7f8510158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8506000640 1 -- 172.21.15.113:0/982861857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8510158eb0 con 0x7f8510152a50 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8506000640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8510152a50 0x7f8510158330 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c3f0 tx=0x7f84f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8507400640 1 -- 172.21.15.113:0/982861857 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f80052d0 con 0x7f8510152a50 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8507400640 1 -- 172.21.15.113:0/982861857 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f8005470 con 0x7f8510152a50 2024-03-04T08:28:04.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8507400640 1 -- 172.21.15.113:0/982861857 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8007260 con 0x7f8510152a50 2024-03-04T08:28:04.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8515800640 1 -- 172.21.15.113:0/982861857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85101f1e70 con 0x7f8510152a50 2024-03-04T08:28:04.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.118+0000 7f8515800640 1 -- 172.21.15.113:0/982861857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8510157b30 con 0x7f8510152a50 2024-03-04T08:28:04.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.122+0000 7f8515800640 1 -- 172.21.15.113:0/982861857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8510075c00 con 0x7f8510152a50 2024-03-04T08:28:04.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.122+0000 7f8507400640 1 -- 172.21.15.113:0/982861857 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84f8007400 con 0x7f8510152a50 2024-03-04T08:28:04.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.122+0000 7f8507400640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f84ec066f60 0x7f84ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:04.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.122+0000 7f8507400640 1 -- 172.21.15.113:0/982861857 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f84f8084e30 con 0x7f8510152a50 2024-03-04T08:28:04.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.122+0000 7f850ea00640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f84ec066f60 0x7f84ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:04.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.122+0000 7f850ea00640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f84ec066f60 0x7f84ec069420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8500009e50 tx=0x7f85000a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:04.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.126+0000 7f8507400640 1 -- 172.21.15.113:0/982861857 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f84f8054d40 con 0x7f8510152a50 2024-03-04T08:28:04.311 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 -- 172.21.15.113:0/982861857 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8510162e40 con 0x7f8510152a50 2024-03-04T08:28:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8507400640 1 -- 172.21.15.113:0/982861857 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f84f8058de0 con 0x7f8510152a50 2024-03-04T08:28:04.312 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:04.312 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:04.312 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 -- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f84ec066f60 msgr2=0x7f84ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f84ec066f60 0x7f84ec069420 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8500009e50 tx=0x7f85000a7070 comp rx=0 tx=0).stop 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 -- 172.21.15.113:0/982861857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8510152a50 msgr2=0x7f8510158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8510152a50 0x7f8510158330 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c3f0 tx=0x7f84f800c8b0 comp rx=0 tx=0).stop 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 -- 172.21.15.113:0/982861857 shutdown_connections 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f851015c4e0 0x7f8510158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f84ec066f60 0x7f84ec069420 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8510152a50 0x7f8510158330 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 --2- 172.21.15.113:0/982861857 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f851006ff20 0x7f8510159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 -- 172.21.15.113:0/982861857 >> 172.21.15.113:0/982861857 conn(0x7f85100734d0 msgr2=0x7f85100cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 -- 172.21.15.113:0/982861857 shutdown_connections 2024-03-04T08:28:04.314 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:04.314+0000 7f8515800640 1 -- 172.21.15.113:0/982861857 wait complete. 2024-03-04T08:28:04.333 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:06.334 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:06.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.482+0000 7fc7d0200640 1 Processor -- start 2024-03-04T08:28:06.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.482+0000 7fc7d0200640 1 -- start start 2024-03-04T08:28:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7d0200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7c8152bd0 0x7fc7c8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7d0200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c81591f0 con 0x7fc7c81527f0 2024-03-04T08:28:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7d0200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c81594d0 con 0x7fc7c80c8200 2024-03-04T08:28:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7d0200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c8071df0 con 0x7fc7c8152bd0 2024-03-04T08:28:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7ce200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc7c8152bd0 0x7fc7c8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7cd800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc7c81527f0 0x7fc7c8072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:35078/0 (socket says 172.21.15.113:35078) 2024-03-04T08:28:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7cd800640 1 -- 172.21.15.113:0/40904269 learned_addr learned my addr 172.21.15.113:0/40904269 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7ce200640 1 -- 172.21.15.113:0/40904269 >> v1:172.21.15.136:6789/0 conn(0x7fc7c80c8200 legacy=0x7fc7c81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7ce200640 1 -- 172.21.15.113:0/40904269 >> v1:172.21.15.113:6789/0 conn(0x7fc7c81527f0 legacy=0x7fc7c8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:06.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7ce200640 1 -- 172.21.15.113:0/40904269 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80717a0 con 0x7fc7c8152bd0 2024-03-04T08:28:06.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7ce200640 1 --2- 172.21.15.113:0/40904269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c8152bd0 0x7fc7c8070d40 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009d90 tx=0x7fc7c001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8eb64d3c8a61a18 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:06.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7c7e00640 1 -- 172.21.15.113:0/40904269 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00b2070 con 0x7fc7c8152bd0 2024-03-04T08:28:06.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7c7e00640 1 -- 172.21.15.113:0/40904269 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c00a5830 con 0x7fc7c8152bd0 2024-03-04T08:28:06.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.486+0000 7fc7c7e00640 1 -- 172.21.15.113:0/40904269 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00a5c00 con 0x7fc7c8152bd0 2024-03-04T08:28:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 -- 172.21.15.113:0/40904269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c8152bd0 msgr2=0x7fc7c8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 --2- 172.21.15.113:0/40904269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c8152bd0 0x7fc7c8070d40 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009d90 tx=0x7fc7c001c970 comp rx=0 tx=0).stop 2024-03-04T08:28:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 -- 172.21.15.113:0/40904269 shutdown_connections 2024-03-04T08:28:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 --2- 172.21.15.113:0/40904269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c8152bd0 0x7fc7c8070d40 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 -- 172.21.15.113:0/40904269 >> 172.21.15.113:0/40904269 conn(0x7fc7c806e820 msgr2=0x7fc7c806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 -- 172.21.15.113:0/40904269 shutdown_connections 2024-03-04T08:28:06.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 -- 172.21.15.113:0/40904269 wait complete. 2024-03-04T08:28:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 Processor -- start 2024-03-04T08:28:06.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 -- start start 2024-03-04T08:28:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c80c8200 0x7fc7c80b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:06.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c81527f0 0x7fc7c81574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7c8152bd0 0x7fc7c8157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8166500 con 0x7fc7c81527f0 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8166380 con 0x7fc7c80c8200 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7d0200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8166680 con 0x7fc7c8152bd0 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7cce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c81527f0 0x7fc7c81574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7cce00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c81527f0 0x7fc7c81574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60398/0 (socket says 172.21.15.113:60398) 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7cce00640 1 -- 172.21.15.113:0/1146816664 learned_addr learned my addr 172.21.15.113:0/1146816664 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7cd800640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c80c8200 0x7fc7c80b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7ce200640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7c8152bd0 0x7fc7c8157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7cce00640 1 -- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7c8152bd0 msgr2=0x7fc7c8157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7cce00640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7c8152bd0 0x7fc7c8157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7cce00640 1 -- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c80c8200 msgr2=0x7fc7c80b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:06.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7cce00640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c80c8200 0x7fc7c80b0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7cce00640 1 -- 172.21.15.113:0/1146816664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80af980 con 0x7fc7c81527f0 2024-03-04T08:28:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.490+0000 7fc7cd800640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c80c8200 0x7fc7c80b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7cce00640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c81527f0 0x7fc7c81574e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400c4d0 tx=0x7fc7b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7c5600640 1 -- 172.21.15.113:0/1146816664 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40052d0 con 0x7fc7c81527f0 2024-03-04T08:28:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7c5600640 1 -- 172.21.15.113:0/1146816664 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b4005470 con 0x7fc7c81527f0 2024-03-04T08:28:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7c5600640 1 -- 172.21.15.113:0/1146816664 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4012830 con 0x7fc7c81527f0 2024-03-04T08:28:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7d0200640 1 -- 172.21.15.113:0/1146816664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c80b0960 con 0x7fc7c81527f0 2024-03-04T08:28:06.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7d0200640 1 -- 172.21.15.113:0/1146816664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c81f45d0 con 0x7fc7c81527f0 2024-03-04T08:28:06.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7d0200640 1 -- 172.21.15.113:0/1146816664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c8075c00 con 0x7fc7c81527f0 2024-03-04T08:28:06.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7c5600640 1 -- 172.21.15.113:0/1146816664 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7b40129d0 con 0x7fc7c81527f0 2024-03-04T08:28:06.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7c5600640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7a8066f60 0x7fc7a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:06.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7c5600640 1 -- 172.21.15.113:0/1146816664 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fc7b40857f0 con 0x7fc7c81527f0 2024-03-04T08:28:06.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.494+0000 7fc7cd800640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7a8066f60 0x7fc7a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.498+0000 7fc7cd800640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7a8066f60 0x7fc7a8069420 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc005890 tx=0x7fc7bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:06.497 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.498+0000 7fc7c5600640 1 -- 172.21.15.113:0/1146816664 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc7b4055c30 con 0x7fc7c81527f0 2024-03-04T08:28:06.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.686+0000 7fc7d0200640 1 -- 172.21.15.113:0/1146816664 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7c80c7de0 con 0x7fc7c81527f0 2024-03-04T08:28:06.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7c5600640 1 -- 172.21.15.113:0/1146816664 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fc7b4055530 con 0x7fc7c81527f0 2024-03-04T08:28:06.687 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:06.687 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:06.687 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:06.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 -- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7a8066f60 msgr2=0x7fc7a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:06.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7a8066f60 0x7fc7a8069420 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc005890 tx=0x7fc7bc009040 comp rx=0 tx=0).stop 2024-03-04T08:28:06.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 -- 172.21.15.113:0/1146816664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c81527f0 msgr2=0x7fc7c81574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:06.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c81527f0 0x7fc7c81574e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400c4d0 tx=0x7fc7b400c990 comp rx=0 tx=0).stop 2024-03-04T08:28:06.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 -- 172.21.15.113:0/1146816664 shutdown_connections 2024-03-04T08:28:06.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc7c8152bd0 0x7fc7c8157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:06.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc7a8066f60 0x7fc7a8069420 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:06.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc7c81527f0 0x7fc7c81574e0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:06.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 --2- 172.21.15.113:0/1146816664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7c80c8200 0x7fc7c80b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:06.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 -- 172.21.15.113:0/1146816664 >> 172.21.15.113:0/1146816664 conn(0x7fc7c806e820 msgr2=0x7fc7c806ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:06.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 -- 172.21.15.113:0/1146816664 shutdown_connections 2024-03-04T08:28:06.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:06.690+0000 7fc7d0200640 1 -- 172.21.15.113:0/1146816664 wait complete. 2024-03-04T08:28:06.707 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:08.708 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:08.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.854+0000 7fc122000640 1 Processor -- start 2024-03-04T08:28:08.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc122000640 1 -- start start 2024-03-04T08:28:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc122000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc11c152ac0 0x7fc11c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc122000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc11c0ca8c0 0x7fc11c0c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc122000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c159340 con 0x7fc11c158920 2024-03-04T08:28:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc122000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c06f900 con 0x7fc11c152ac0 2024-03-04T08:28:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc122000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c159900 con 0x7fc11c0ca8c0 2024-03-04T08:28:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc11b400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc11c152ac0 0x7fc11c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc11be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fc11c158920 0x7fc11c0ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51252/0 (socket says 172.21.15.113:51252) 2024-03-04T08:28:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc11be00640 1 -- 172.21.15.113:0/673278397 learned_addr learned my addr 172.21.15.113:0/673278397 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:08.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc112a00640 1 --2- 172.21.15.113:0/673278397 >> v2:172.21.15.136:3300/0 conn(0x7fc11c0ca8c0 0x7fc11c0c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:08.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc11b400640 1 -- 172.21.15.113:0/673278397 >> v2:172.21.15.136:3300/0 conn(0x7fc11c0ca8c0 msgr2=0x7fc11c0c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:08.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc11b400640 1 --2- 172.21.15.113:0/673278397 >> v2:172.21.15.136:3300/0 conn(0x7fc11c0ca8c0 0x7fc11c0c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:08.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc11b400640 1 -- 172.21.15.113:0/673278397 >> v1:172.21.15.113:6789/0 conn(0x7fc11c158920 legacy=0x7fc11c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:08.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc11b400640 1 -- 172.21.15.113:0/673278397 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c159620 con 0x7fc11c152ac0 2024-03-04T08:28:08.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc112a00640 1 --2- 172.21.15.113:0/673278397 >> v2:172.21.15.136:3300/0 conn(0x7fc11c0ca8c0 0x7fc11c0c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:08.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.858+0000 7fc11aa00640 1 -- 172.21.15.113:0/673278397 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840771001 0 0) 0x7fc11c159340 con 0x7fc11c158920 2024-03-04T08:28:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc11b400640 1 --2- 172.21.15.113:0/673278397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c152ac0 0x7fc11c0c8c00 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0c9140 tx=0x7fc10c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f09f4ecf14e7f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc11aa00640 1 -- 172.21.15.113:0/673278397 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0b0070 con 0x7fc11c152ac0 2024-03-04T08:28:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc11aa00640 1 -- 172.21.15.113:0/673278397 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c0ab400 con 0x7fc11c152ac0 2024-03-04T08:28:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc11aa00640 1 -- 172.21.15.113:0/673278397 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0ab7d0 con 0x7fc11c152ac0 2024-03-04T08:28:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc122000640 1 -- 172.21.15.113:0/673278397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c152ac0 msgr2=0x7fc11c0c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc122000640 1 --2- 172.21.15.113:0/673278397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c152ac0 0x7fc11c0c8c00 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0c9140 tx=0x7fc10c01c480 comp rx=0 tx=0).stop 2024-03-04T08:28:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc122000640 1 -- 172.21.15.113:0/673278397 shutdown_connections 2024-03-04T08:28:08.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc122000640 1 --2- 172.21.15.113:0/673278397 >> v2:172.21.15.136:3300/0 conn(0x7fc11c0ca8c0 0x7fc11c0c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc122000640 1 --2- 172.21.15.113:0/673278397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c152ac0 0x7fc11c0c8c00 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc122000640 1 -- 172.21.15.113:0/673278397 >> 172.21.15.113:0/673278397 conn(0x7fc11c066db0 msgr2=0x7fc11c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc122000640 1 -- 172.21.15.113:0/673278397 shutdown_connections 2024-03-04T08:28:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc122000640 1 -- 172.21.15.113:0/673278397 wait complete. 2024-03-04T08:28:08.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc122000640 1 Processor -- start 2024-03-04T08:28:08.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.862+0000 7fc122000640 1 -- start start 2024-03-04T08:28:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc122000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0ca8c0 0x7fc11c14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc122000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc11c152ac0 0x7fc11c0b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc122000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c158920 0x7fc11c1f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc122000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0ceb20 con 0x7fc11c158920 2024-03-04T08:28:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc122000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0ce9a0 con 0x7fc11c0ca8c0 2024-03-04T08:28:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc122000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c152ea0 con 0x7fc11c152ac0 2024-03-04T08:28:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc11be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c158920 0x7fc11c1f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc11be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c158920 0x7fc11c1f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48538/0 (socket says 172.21.15.113:48538) 2024-03-04T08:28:08.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc11be00640 1 -- 172.21.15.113:0/2870187429 learned_addr learned my addr 172.21.15.113:0/2870187429 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc11b400640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0ca8c0 0x7fc11c14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc112a00640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc11c152ac0 0x7fc11c0b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc11be00640 1 -- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc11c152ac0 msgr2=0x7fc11c0b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc11be00640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc11c152ac0 0x7fc11c0b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc11be00640 1 -- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0ca8c0 msgr2=0x7fc11c14d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc11be00640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0ca8c0 0x7fc11c14d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc11be00640 1 -- 172.21.15.113:0/2870187429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c157ae0 con 0x7fc11c158920 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc112a00640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc11c152ac0 0x7fc11c0b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc11be00640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c158920 0x7fc11c1f33c0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c730 tx=0x7fc11400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc113e00640 1 -- 172.21.15.113:0/2870187429 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114018070 con 0x7fc11c158920 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc113e00640 1 -- 172.21.15.113:0/2870187429 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11400ce40 con 0x7fc11c158920 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc113e00640 1 -- 172.21.15.113:0/2870187429 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400a820 con 0x7fc11c158920 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc122000640 1 -- 172.21.15.113:0/2870187429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c157fe0 con 0x7fc11c158920 2024-03-04T08:28:08.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc122000640 1 -- 172.21.15.113:0/2870187429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c1585a0 con 0x7fc11c158920 2024-03-04T08:28:08.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc122000640 1 -- 172.21.15.113:0/2870187429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c075c00 con 0x7fc11c158920 2024-03-04T08:28:08.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.866+0000 7fc113e00640 1 -- 172.21.15.113:0/2870187429 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc11401b070 con 0x7fc11c158920 2024-03-04T08:28:08.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.870+0000 7fc113e00640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc0f8066f60 0x7fc0f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:08.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.870+0000 7fc113e00640 1 -- 172.21.15.113:0/2870187429 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fc114084f80 con 0x7fc11c158920 2024-03-04T08:28:08.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.870+0000 7fc11b400640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc0f8066f60 0x7fc0f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:08.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.870+0000 7fc11b400640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc0f8066f60 0x7fc0f8069420 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0ad000 tx=0x7fc10c0a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:08.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:08.874+0000 7fc113e00640 1 -- 172.21.15.113:0/2870187429 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc114054e60 con 0x7fc11c158920 2024-03-04T08:28:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.058+0000 7fc122000640 1 -- 172.21.15.113:0/2870187429 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc11c0684f0 con 0x7fc11c158920 2024-03-04T08:28:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.058+0000 7fc113e00640 1 -- 172.21.15.113:0/2870187429 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fc114059e20 con 0x7fc11c158920 2024-03-04T08:28:09.057 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:09.057 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 -- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc0f8066f60 msgr2=0x7fc0f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc0f8066f60 0x7fc0f8069420 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0ad000 tx=0x7fc10c0a6000 comp rx=0 tx=0).stop 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 -- 172.21.15.113:0/2870187429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c158920 msgr2=0x7fc11c1f33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c158920 0x7fc11c1f33c0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc11400c730 tx=0x7fc11400cbf0 comp rx=0 tx=0).stop 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 -- 172.21.15.113:0/2870187429 shutdown_connections 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc0f8066f60 0x7fc0f8069420 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc11c158920 0x7fc11c1f33c0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc11c152ac0 0x7fc11c0b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 --2- 172.21.15.113:0/2870187429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc11c0ca8c0 0x7fc11c14d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 -- 172.21.15.113:0/2870187429 >> 172.21.15.113:0/2870187429 conn(0x7fc11c066db0 msgr2=0x7fc11c0cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:09.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 -- 172.21.15.113:0/2870187429 shutdown_connections 2024-03-04T08:28:09.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:09.062+0000 7fc122000640 1 -- 172.21.15.113:0/2870187429 wait complete. 2024-03-04T08:28:09.079 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:11.080 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:11.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.226+0000 7f04f7a00640 1 Processor -- start 2024-03-04T08:28:11.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.226+0000 7f04f7a00640 1 -- start start 2024-03-04T08:28:11.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f7a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04f00c08d0 0x7f04f006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:11.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f7a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04f015ad30 0x7f04f015b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:11.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f7a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f015efd0 con 0x7f04f0155100 2024-03-04T08:28:11.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f7a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f00c8890 con 0x7f04f015ad30 2024-03-04T08:28:11.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f7a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00c7ef0 con 0x7f04f00c08d0 2024-03-04T08:28:11.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f5a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04f015ad30 0x7f04f015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:11.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f5a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04f015ad30 0x7f04f015b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48554/0 (socket says 172.21.15.113:48554) 2024-03-04T08:28:11.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f5a00640 1 -- 172.21.15.113:0/108861547 learned_addr learned my addr 172.21.15.113:0/108861547 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:11.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04e7e00640 1 --2- 172.21.15.113:0/108861547 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04f00c08d0 0x7f04f006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:11.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f5a00640 1 -- 172.21.15.113:0/108861547 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04f00c08d0 msgr2=0x7f04f006b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:11.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f5a00640 1 --2- 172.21.15.113:0/108861547 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04f00c08d0 0x7f04f006b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:11.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f5a00640 1 -- 172.21.15.113:0/108861547 >> v1:172.21.15.136:6789/0 conn(0x7f04f0155100 legacy=0x7f04f00cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:11.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.230+0000 7f04f5a00640 1 -- 172.21.15.113:0/108861547 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f00c81d0 con 0x7f04f015ad30 2024-03-04T08:28:11.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f5a00640 1 --2- 172.21.15.113:0/108861547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f015ad30 0x7f04f015b140 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009e50 tx=0x7f04e801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba8f05f51d8b4e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:11.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04efe00640 1 -- 172.21.15.113:0/108861547 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e80a5700 con 0x7f04f015ad30 2024-03-04T08:28:11.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04efe00640 1 -- 172.21.15.113:0/108861547 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e80a58a0 con 0x7f04f015ad30 2024-03-04T08:28:11.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04efe00640 1 -- 172.21.15.113:0/108861547 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e80a5c70 con 0x7f04f015ad30 2024-03-04T08:28:11.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 -- 172.21.15.113:0/108861547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f015ad30 msgr2=0x7f04f015b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:11.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 --2- 172.21.15.113:0/108861547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f015ad30 0x7f04f015b140 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009e50 tx=0x7f04e801ca30 comp rx=0 tx=0).stop 2024-03-04T08:28:11.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 -- 172.21.15.113:0/108861547 shutdown_connections 2024-03-04T08:28:11.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 --2- 172.21.15.113:0/108861547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f015ad30 0x7f04f015b140 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:11.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 --2- 172.21.15.113:0/108861547 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04f00c08d0 0x7f04f006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:11.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 -- 172.21.15.113:0/108861547 >> 172.21.15.113:0/108861547 conn(0x7f04f006deb0 msgr2=0x7f04f006e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:11.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 -- 172.21.15.113:0/108861547 shutdown_connections 2024-03-04T08:28:11.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 -- 172.21.15.113:0/108861547 wait complete. 2024-03-04T08:28:11.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 Processor -- start 2024-03-04T08:28:11.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 -- start start 2024-03-04T08:28:11.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f00c08d0 0x7f04f00c1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f0155100 0x7f04f01ed450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04f015ad30 0x7f04f01f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f015ed30 con 0x7f04f00c08d0 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f015ebb0 con 0x7f04f0155100 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f7a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00cd7d0 con 0x7f04f015ad30 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.234+0000 7f04f5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f00c08d0 0x7f04f00c1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f00c08d0 0x7f04f00c1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48560/0 (socket says 172.21.15.113:48560) 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f5000640 1 -- 172.21.15.113:0/3687466176 learned_addr learned my addr 172.21.15.113:0/3687466176 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04e7e00640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f0155100 0x7f04f01ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f5a00640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04f015ad30 0x7f04f01f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f5000640 1 -- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04f015ad30 msgr2=0x7f04f01f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:11.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f5000640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04f015ad30 0x7f04f01f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:11.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f5000640 1 -- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f0155100 msgr2=0x7f04f01ed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:11.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f5000640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f0155100 0x7f04f01ed450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:11.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f5000640 1 -- 172.21.15.113:0/3687466176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f00c5ba0 con 0x7f04f00c08d0 2024-03-04T08:28:11.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f5000640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f00c08d0 0x7f04f00c1fb0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c530 tx=0x7f04e000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:11.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04ed600640 1 -- 172.21.15.113:0/3687466176 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e0016020 con 0x7f04f00c08d0 2024-03-04T08:28:11.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04ed600640 1 -- 172.21.15.113:0/3687466176 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e0005220 con 0x7f04f00c08d0 2024-03-04T08:28:11.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04ed600640 1 -- 172.21.15.113:0/3687466176 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e0007030 con 0x7f04f00c08d0 2024-03-04T08:28:11.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f7a00640 1 -- 172.21.15.113:0/3687466176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f00c5e40 con 0x7f04f00c08d0 2024-03-04T08:28:11.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f7a00640 1 -- 172.21.15.113:0/3687466176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f00c66e0 con 0x7f04f00c08d0 2024-03-04T08:28:11.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04f7a00640 1 -- 172.21.15.113:0/3687466176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f0075c00 con 0x7f04f00c08d0 2024-03-04T08:28:11.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04ed600640 1 -- 172.21.15.113:0/3687466176 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04e00071d0 con 0x7f04f00c08d0 2024-03-04T08:28:11.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04ed600640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04cc066f60 0x7f04cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:11.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.238+0000 7f04ed600640 1 -- 172.21.15.113:0/3687466176 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f04e0085550 con 0x7f04f00c08d0 2024-03-04T08:28:11.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.242+0000 7f04e7e00640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04cc066f60 0x7f04cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:11.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.242+0000 7f04e7e00640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04cc066f60 0x7f04cc069420 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f04f015dba0 tx=0x7f04d8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:11.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.242+0000 7f04ed600640 1 -- 172.21.15.113:0/3687466176 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f04e0020020 con 0x7f04f00c08d0 2024-03-04T08:28:11.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 -- 172.21.15.113:0/3687466176 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04f00c8a20 con 0x7f04f00c08d0 2024-03-04T08:28:11.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04ed600640 1 -- 172.21.15.113:0/3687466176 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f04e00074c0 con 0x7f04f00c08d0 2024-03-04T08:28:11.436 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:11.436 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:11.436 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:11.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 -- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04cc066f60 msgr2=0x7f04cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:11.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04cc066f60 0x7f04cc069420 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f04f015dba0 tx=0x7f04d8008040 comp rx=0 tx=0).stop 2024-03-04T08:28:11.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 -- 172.21.15.113:0/3687466176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f00c08d0 msgr2=0x7f04f00c1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:11.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f00c08d0 0x7f04f00c1fb0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c530 tx=0x7f04e000c9f0 comp rx=0 tx=0).stop 2024-03-04T08:28:11.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 -- 172.21.15.113:0/3687466176 shutdown_connections 2024-03-04T08:28:11.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f04f015ad30 0x7f04f01f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:11.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04f0155100 0x7f04f01ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:11.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f04cc066f60 0x7f04cc069420 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:11.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 --2- 172.21.15.113:0/3687466176 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04f00c08d0 0x7f04f00c1fb0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:11.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 -- 172.21.15.113:0/3687466176 >> 172.21.15.113:0/3687466176 conn(0x7f04f006deb0 msgr2=0x7f04f006e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:11.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 -- 172.21.15.113:0/3687466176 shutdown_connections 2024-03-04T08:28:11.439 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:11.438+0000 7f04f7a00640 1 -- 172.21.15.113:0/3687466176 wait complete. 2024-03-04T08:28:11.457 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:13.458 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:13.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.606+0000 7f61c6000640 1 Processor -- start 2024-03-04T08:28:13.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.606+0000 7f61c6000640 1 -- start start 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.606+0000 7f61c6000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f61c01527f0 0x7f61c0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.606+0000 7f61c6000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f61c0152bd0 0x7f61c01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.606+0000 7f61c6000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c01591f0 con 0x7f61c00c8200 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.606+0000 7f61c6000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c0071df0 con 0x7f61c0152bd0 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.606+0000 7f61c6000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c01597b0 con 0x7f61c01527f0 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.606+0000 7f61b7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f61c00c8200 0x7f61c0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51268/0 (socket says 172.21.15.113:51268) 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.606+0000 7f61b7e00640 1 -- 172.21.15.113:0/2643910973 learned_addr learned my addr 172.21.15.113:0/2643910973 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.606+0000 7f61bfe00640 1 --2- 172.21.15.113:0/2643910973 >> v2:172.21.15.113:3300/0 conn(0x7f61c0152bd0 0x7f61c01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.610+0000 7f61bf400640 1 --2- 172.21.15.113:0/2643910973 >> v2:172.21.15.136:3300/0 conn(0x7f61c01527f0 0x7f61c0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.610+0000 7f61bfe00640 1 -- 172.21.15.113:0/2643910973 >> v2:172.21.15.136:3300/0 conn(0x7f61c01527f0 msgr2=0x7f61c0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.610+0000 7f61bfe00640 1 --2- 172.21.15.113:0/2643910973 >> v2:172.21.15.136:3300/0 conn(0x7f61c01527f0 0x7f61c0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.610+0000 7f61bfe00640 1 -- 172.21.15.113:0/2643910973 >> v1:172.21.15.113:6789/0 conn(0x7f61c00c8200 legacy=0x7f61c0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:13.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.610+0000 7f61bfe00640 1 -- 172.21.15.113:0/2643910973 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c01594d0 con 0x7f61c0152bd0 2024-03-04T08:28:13.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.610+0000 7f61bea00640 1 -- 172.21.15.113:0/2643910973 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639834523 0 0) 0x7f61c01591f0 con 0x7f61c00c8200 2024-03-04T08:28:13.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.610+0000 7f61bfe00640 1 --2- 172.21.15.113:0/2643910973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c0152bd0 0x7f61c01587d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f61c006f400 tx=0x7f61b801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44db422d9a3fb6e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:13.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.610+0000 7f61bea00640 1 -- 172.21.15.113:0/2643910973 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80b2070 con 0x7f61c0152bd0 2024-03-04T08:28:13.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.610+0000 7f61bea00640 1 -- 172.21.15.113:0/2643910973 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b80a58e0 con 0x7f61c0152bd0 2024-03-04T08:28:13.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.610+0000 7f61bea00640 1 -- 172.21.15.113:0/2643910973 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80a5cb0 con 0x7f61c0152bd0 2024-03-04T08:28:13.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 -- 172.21.15.113:0/2643910973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c0152bd0 msgr2=0x7f61c01587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:13.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 --2- 172.21.15.113:0/2643910973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c0152bd0 0x7f61c01587d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f61c006f400 tx=0x7f61b801c930 comp rx=0 tx=0).stop 2024-03-04T08:28:13.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 -- 172.21.15.113:0/2643910973 shutdown_connections 2024-03-04T08:28:13.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 --2- 172.21.15.113:0/2643910973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c0152bd0 0x7f61c01587d0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:13.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 --2- 172.21.15.113:0/2643910973 >> v2:172.21.15.136:3300/0 conn(0x7f61c01527f0 0x7f61c0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:13.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 -- 172.21.15.113:0/2643910973 >> 172.21.15.113:0/2643910973 conn(0x7f61c006e820 msgr2=0x7f61c006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:13.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 -- 172.21.15.113:0/2643910973 shutdown_connections 2024-03-04T08:28:13.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 -- 172.21.15.113:0/2643910973 wait complete. 2024-03-04T08:28:13.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 Processor -- start 2024-03-04T08:28:13.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 -- start start 2024-03-04T08:28:13.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61c00c8200 0x7f61c0153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:13.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c01527f0 0x7f61c00b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:13.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61c0152bd0 0x7f61c0149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:13.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c01654f0 con 0x7f61c01527f0 2024-03-04T08:28:13.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c0165370 con 0x7f61c0152bd0 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61c6000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c0165670 con 0x7f61c00c8200 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c01527f0 0x7f61c00b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61b7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c01527f0 0x7f61c00b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48566/0 (socket says 172.21.15.113:48566) 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61b7e00640 1 -- 172.21.15.113:0/797637633 learned_addr learned my addr 172.21.15.113:0/797637633 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61bfe00640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61c0152bd0 0x7f61c0149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61bf400640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61c00c8200 0x7f61c0153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61b7e00640 1 -- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61c00c8200 msgr2=0x7f61c0153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61b7e00640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61c00c8200 0x7f61c0153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61b7e00640 1 -- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61c0152bd0 msgr2=0x7f61c0149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61b7e00640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61c0152bd0 0x7f61c0149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61b7e00640 1 -- 172.21.15.113:0/797637633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c01503f0 con 0x7f61c01527f0 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.614+0000 7f61b7e00640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c01527f0 0x7f61c00b0b90 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c4b0 tx=0x7f61a800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.618+0000 7f61b7400640 1 -- 172.21.15.113:0/797637633 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8018070 con 0x7f61c01527f0 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.618+0000 7f61b7400640 1 -- 172.21.15.113:0/797637633 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a8007500 con 0x7f61c01527f0 2024-03-04T08:28:13.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.618+0000 7f61c6000640 1 -- 172.21.15.113:0/797637633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c01f7a30 con 0x7f61c01527f0 2024-03-04T08:28:13.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.618+0000 7f61b7400640 1 -- 172.21.15.113:0/797637633 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8004a30 con 0x7f61c01527f0 2024-03-04T08:28:13.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.618+0000 7f61c6000640 1 -- 172.21.15.113:0/797637633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c0150b70 con 0x7f61c01527f0 2024-03-04T08:28:13.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.618+0000 7f61c6000640 1 -- 172.21.15.113:0/797637633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c0075c00 con 0x7f61c01527f0 2024-03-04T08:28:13.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.618+0000 7f61b7400640 1 -- 172.21.15.113:0/797637633 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61a801b070 con 0x7f61c01527f0 2024-03-04T08:28:13.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.618+0000 7f61b7400640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f619c066f60 0x7f619c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:13.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.618+0000 7f61b7400640 1 -- 172.21.15.113:0/797637633 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f61a8084d40 con 0x7f61c01527f0 2024-03-04T08:28:13.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.618+0000 7f61bf400640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f619c066f60 0x7f619c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:13.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.622+0000 7f61bf400640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f619c066f60 0x7f619c069420 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009af0 tx=0x7f61b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:13.621 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.622+0000 7f61b7400640 1 -- 172.21.15.113:0/797637633 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f61a8054d10 con 0x7f61c01527f0 2024-03-04T08:28:13.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.810+0000 7f61c6000640 1 -- 172.21.15.113:0/797637633 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61c00c79d0 con 0x7f61c01527f0 2024-03-04T08:28:13.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61b7400640 1 -- 172.21.15.113:0/797637633 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f61a8058db0 con 0x7f61c01527f0 2024-03-04T08:28:13.811 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:13.811 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:13.811 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:13.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 -- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f619c066f60 msgr2=0x7f619c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f619c066f60 0x7f619c069420 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009af0 tx=0x7f61b0008040 comp rx=0 tx=0).stop 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 -- 172.21.15.113:0/797637633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c01527f0 msgr2=0x7f61c00b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c01527f0 0x7f61c00b0b90 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f61a800c4b0 tx=0x7f61a800c970 comp rx=0 tx=0).stop 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 -- 172.21.15.113:0/797637633 shutdown_connections 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61c0152bd0 0x7f61c0149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f619c066f60 0x7f619c069420 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f61c01527f0 0x7f61c00b0b90 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 --2- 172.21.15.113:0/797637633 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f61c00c8200 0x7f61c0153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 -- 172.21.15.113:0/797637633 >> 172.21.15.113:0/797637633 conn(0x7f61c006e820 msgr2=0x7f61c00ca120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 -- 172.21.15.113:0/797637633 shutdown_connections 2024-03-04T08:28:13.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:13.814+0000 7f61c6000640 1 -- 172.21.15.113:0/797637633 wait complete. 2024-03-04T08:28:13.831 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:28:13.832 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:28:15.833 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:15.945 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:28:15.950 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:28:15.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc21b800640 1 Processor -- start 2024-03-04T08:28:15.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc21b800640 1 -- start start 2024-03-04T08:28:15.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc21b800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 0x7fc214072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc21b800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc2140731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc21b800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2140c27f0 con 0x7fc2140c2d80 2024-03-04T08:28:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc21b800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21415ad60 con 0x7fc21414e4c0 2024-03-04T08:28:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc21b800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140c2a90 con 0x7fc2140c22e0 2024-03-04T08:28:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc218e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 0x7fc214072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc20be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc2140731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc219800640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fc2140c2d80 0x7fc21415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:59078/0 (socket says 172.21.15.113:59078) 2024-03-04T08:28:15.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.982+0000 7fc219800640 1 -- 172.21.15.113:0/1273642985 learned_addr learned my addr 172.21.15.113:0/1273642985 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:15.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc213e00640 1 -- 172.21.15.113:0/1273642985 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722058163 0 0) 0x7fc2140c27f0 con 0x7fc2140c2d80 2024-03-04T08:28:15.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc213e00640 1 -- 172.21.15.113:0/1273642985 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f8003530 con 0x7fc2140c2d80 2024-03-04T08:28:15.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc20be00640 1 -- 172.21.15.113:0/1273642985 >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 msgr2=0x7fc214072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:15.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc20be00640 1 --2- 172.21.15.113:0/1273642985 >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 0x7fc214072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:15.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc20be00640 1 -- 172.21.15.113:0/1273642985 >> v1:172.21.15.136:6789/0 conn(0x7fc2140c2d80 legacy=0x7fc21415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:15.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc20be00640 1 -- 172.21.15.113:0/1273642985 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214070c60 con 0x7fc2140c22e0 2024-03-04T08:28:15.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc218e00640 1 --2- 172.21.15.113:0/1273642985 >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 0x7fc214072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:28:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc20be00640 1 --2- 172.21.15.113:0/1273642985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc2140731c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0097c0 tx=0x7fc1fc01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5604576815fcdae4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc213e00640 1 -- 172.21.15.113:0/1273642985 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc0b5070 con 0x7fc2140c22e0 2024-03-04T08:28:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc213e00640 1 -- 172.21.15.113:0/1273642985 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1fc0a8990 con 0x7fc2140c22e0 2024-03-04T08:28:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.986+0000 7fc213e00640 1 -- 172.21.15.113:0/1273642985 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc0a8d60 con 0x7fc2140c22e0 2024-03-04T08:28:15.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 -- 172.21.15.113:0/1273642985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 msgr2=0x7fc2140731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 --2- 172.21.15.113:0/1273642985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc2140731c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0097c0 tx=0x7fc1fc01ec90 comp rx=0 tx=0).stop 2024-03-04T08:28:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 -- 172.21.15.113:0/1273642985 shutdown_connections 2024-03-04T08:28:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 --2- 172.21.15.113:0/1273642985 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c22e0 0x7fc2140731c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 --2- 172.21.15.113:0/1273642985 >> v2:172.21.15.136:3300/0 conn(0x7fc21414e4c0 0x7fc214072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 -- 172.21.15.113:0/1273642985 >> 172.21.15.113:0/1273642985 conn(0x7fc21406e820 msgr2=0x7fc21406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 -- 172.21.15.113:0/1273642985 shutdown_connections 2024-03-04T08:28:15.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 -- 172.21.15.113:0/1273642985 wait complete. 2024-03-04T08:28:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 Processor -- start 2024-03-04T08:28:15.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 -- start start 2024-03-04T08:28:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c22e0 0x7fc2141542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c2d80 0x7fc2141636f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:15.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214163c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:15.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc214071500 con 0x7fc21414e4c0 2024-03-04T08:28:15.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc214071220 con 0x7fc2140c22e0 2024-03-04T08:28:15.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc21b800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21414e8a0 con 0x7fc2140c2d80 2024-03-04T08:28:15.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc20be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c2d80 0x7fc2141636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:15.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc218e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c22e0 0x7fc2141542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:15.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc219800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:15.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc20be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c2d80 0x7fc2141636f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:39198/0 (socket says 172.21.15.113:39198) 2024-03-04T08:28:15.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc20be00640 1 -- 172.21.15.113:0/1617337832 learned_addr learned my addr 172.21.15.113:0/1617337832 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:15.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc218e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c22e0 0x7fc2141542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:58106/0 (socket says 172.21.15.113:58106) 2024-03-04T08:28:15.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc219800640 1 -- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c2d80 msgr2=0x7fc2141636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:15.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc219800640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c2d80 0x7fc2141636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:15.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc219800640 1 -- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c22e0 msgr2=0x7fc2141542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:15.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc219800640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c22e0 0x7fc2141542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:15.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc219800640 1 -- 172.21.15.113:0/1617337832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214164280 con 0x7fc21414e4c0 2024-03-04T08:28:15.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.990+0000 7fc219800640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214163c30 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00ca80 tx=0x7fc20c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc211600640 1 -- 172.21.15.113:0/1617337832 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c017070 con 0x7fc21414e4c0 2024-03-04T08:28:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc211600640 1 -- 172.21.15.113:0/1617337832 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20c00ac30 con 0x7fc21414e4c0 2024-03-04T08:28:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc211600640 1 -- 172.21.15.113:0/1617337832 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20c012400 con 0x7fc21414e4c0 2024-03-04T08:28:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc21b800640 1 -- 172.21.15.113:0/1617337832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc214164560 con 0x7fc21414e4c0 2024-03-04T08:28:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc21b800640 1 -- 172.21.15.113:0/1617337832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc214158ad0 con 0x7fc21414e4c0 2024-03-04T08:28:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc21b800640 1 -- 172.21.15.113:0/1617337832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc214164a80 con 0x7fc21414e4c0 2024-03-04T08:28:15.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc211600640 1 -- 172.21.15.113:0/1617337832 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc20c01a070 con 0x7fc21414e4c0 2024-03-04T08:28:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc211600640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f0066f90 0x7fc1f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc211600640 1 -- 172.21.15.113:0/1617337832 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fc20c0856b0 con 0x7fc21414e4c0 2024-03-04T08:28:15.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc218e00640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f0066f90 0x7fc1f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:15.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.994+0000 7fc218e00640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f0066f90 0x7fc1f0069450 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc204009580 tx=0x7fc204009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:15.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:15.998+0000 7fc211600640 1 -- 172.21.15.113:0/1617337832 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc20c0555c0 con 0x7fc21414e4c0 2024-03-04T08:28:16.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.186+0000 7fc21b800640 1 -- 172.21.15.113:0/1617337832 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc214075c00 con 0x7fc21414e4c0 2024-03-04T08:28:16.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.186+0000 7fc211600640 1 -- 172.21.15.113:0/1617337832 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fc20c059660 con 0x7fc21414e4c0 2024-03-04T08:28:16.185 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:16.185 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:16.185 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 -- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f0066f90 msgr2=0x7fc1f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f0066f90 0x7fc1f0069450 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc204009580 tx=0x7fc204009190 comp rx=0 tx=0).stop 2024-03-04T08:28:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 -- 172.21.15.113:0/1617337832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 msgr2=0x7fc214163c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214163c30 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00ca80 tx=0x7fc20c00cf40 comp rx=0 tx=0).stop 2024-03-04T08:28:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 -- 172.21.15.113:0/1617337832 shutdown_connections 2024-03-04T08:28:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fc1f0066f90 0x7fc1f0069450 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc21414e4c0 0x7fc214163c30 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc2140c2d80 0x7fc2141636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 --2- 172.21.15.113:0/1617337832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc2140c22e0 0x7fc2141542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:16.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 -- 172.21.15.113:0/1617337832 >> 172.21.15.113:0/1617337832 conn(0x7fc21406e820 msgr2=0x7fc2140c15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:16.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 -- 172.21.15.113:0/1617337832 shutdown_connections 2024-03-04T08:28:16.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:16.190+0000 7fc21b800640 1 -- 172.21.15.113:0/1617337832 wait complete. 2024-03-04T08:28:16.207 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:18.207 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:18.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.354+0000 7fb7c9e00640 1 Processor -- start 2024-03-04T08:28:18.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.354+0000 7fb7c9e00640 1 -- start start 2024-03-04T08:28:18.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb7c40c22e0 0x7fb7c415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:18.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb7c40c2d80 0x7fb7c415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:18.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c9e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c40c27f0 con 0x7fb7c414e4c0 2024-03-04T08:28:18.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c9e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c415ad60 con 0x7fb7c40c22e0 2024-03-04T08:28:18.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c9e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c40c2a90 con 0x7fb7c40c2d80 2024-03-04T08:28:18.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb7c40c22e0 0x7fb7c415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:18.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb7c40c22e0 0x7fb7c415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34898/0 (socket says 172.21.15.113:34898) 2024-03-04T08:28:18.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c2000640 1 -- 172.21.15.113:0/1977664684 learned_addr learned my addr 172.21.15.113:0/1977664684 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:18.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c3400640 1 --2- 172.21.15.113:0/1977664684 >> v2:172.21.15.136:3300/0 conn(0x7fb7c40c2d80 0x7fb7c415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:18.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c2000640 1 -- 172.21.15.113:0/1977664684 >> v2:172.21.15.136:3300/0 conn(0x7fb7c40c2d80 msgr2=0x7fb7c415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:18.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c2000640 1 --2- 172.21.15.113:0/1977664684 >> v2:172.21.15.136:3300/0 conn(0x7fb7c40c2d80 0x7fb7c415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:18.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c2000640 1 -- 172.21.15.113:0/1977664684 >> v1:172.21.15.136:6789/0 conn(0x7fb7c414e4c0 legacy=0x7fb7c4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:18.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.358+0000 7fb7c2000640 1 -- 172.21.15.113:0/1977664684 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4070c60 con 0x7fb7c40c22e0 2024-03-04T08:28:18.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c2000640 1 --2- 172.21.15.113:0/1977664684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 0x7fb7c415b1f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40c2700 tx=0x7fb7b001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e91785866bcee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:18.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c1600640 1 -- 172.21.15.113:0/1977664684 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00b0070 con 0x7fb7c40c22e0 2024-03-04T08:28:18.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c1600640 1 -- 172.21.15.113:0/1977664684 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b00ab600 con 0x7fb7c40c22e0 2024-03-04T08:28:18.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c1600640 1 -- 172.21.15.113:0/1977664684 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00ab9d0 con 0x7fb7c40c22e0 2024-03-04T08:28:18.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 -- 172.21.15.113:0/1977664684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 msgr2=0x7fb7c415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:18.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 --2- 172.21.15.113:0/1977664684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 0x7fb7c415b1f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40c2700 tx=0x7fb7b001c520 comp rx=0 tx=0).stop 2024-03-04T08:28:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 -- 172.21.15.113:0/1977664684 shutdown_connections 2024-03-04T08:28:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 --2- 172.21.15.113:0/1977664684 >> v2:172.21.15.136:3300/0 conn(0x7fb7c40c2d80 0x7fb7c415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 --2- 172.21.15.113:0/1977664684 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 0x7fb7c415b1f0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 -- 172.21.15.113:0/1977664684 >> 172.21.15.113:0/1977664684 conn(0x7fb7c406e820 msgr2=0x7fb7c406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 -- 172.21.15.113:0/1977664684 shutdown_connections 2024-03-04T08:28:18.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 -- 172.21.15.113:0/1977664684 wait complete. 2024-03-04T08:28:18.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 Processor -- start 2024-03-04T08:28:18.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 -- start start 2024-03-04T08:28:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 0x7fb7c41513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7c40c2d80 0x7fb7c414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7c414e4c0 0x7fb7c414ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c40aede0 con 0x7fb7c40c22e0 2024-03-04T08:28:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c414e8a0 con 0x7fb7c40c2d80 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c9e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c4073390 con 0x7fb7c414e4c0 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 0x7fb7c41513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c2a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 0x7fb7c41513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34902/0 (socket says 172.21.15.113:34902) 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.362+0000 7fb7c2a00640 1 -- 172.21.15.113:0/2655036962 learned_addr learned my addr 172.21.15.113:0/2655036962 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c2000640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7c40c2d80 0x7fb7c414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c3400640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7c414e4c0 0x7fb7c414ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c2a00640 1 -- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7c414e4c0 msgr2=0x7fb7c414ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c2a00640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7c414e4c0 0x7fb7c414ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c2a00640 1 -- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7c40c2d80 msgr2=0x7fb7c414faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c2a00640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7c40c2d80 0x7fb7c414faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c2a00640 1 -- 172.21.15.113:0/2655036962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c414b490 con 0x7fb7c40c22e0 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c2000640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7c40c2d80 0x7fb7c414faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c2a00640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 0x7fb7c41513d0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400c450 tx=0x7fb7b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:18.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7bb400640 1 -- 172.21.15.113:0/2655036962 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4016020 con 0x7fb7c40c22e0 2024-03-04T08:28:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7bb400640 1 -- 172.21.15.113:0/2655036962 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b40052d0 con 0x7fb7c40c22e0 2024-03-04T08:28:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7bb400640 1 -- 172.21.15.113:0/2655036962 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4007260 con 0x7fb7c40c22e0 2024-03-04T08:28:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c9e00640 1 -- 172.21.15.113:0/2655036962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c41f0190 con 0x7fb7c40c22e0 2024-03-04T08:28:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c9e00640 1 -- 172.21.15.113:0/2655036962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c414efc0 con 0x7fb7c40c22e0 2024-03-04T08:28:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7c9e00640 1 -- 172.21.15.113:0/2655036962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c4075c00 con 0x7fb7c40c22e0 2024-03-04T08:28:18.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7bb400640 1 -- 172.21.15.113:0/2655036962 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7b4007400 con 0x7fb7c40c22e0 2024-03-04T08:28:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7bb400640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb7a4066f60 0x7fb7a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.366+0000 7fb7bb400640 1 -- 172.21.15.113:0/2655036962 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fb7b40854b0 con 0x7fb7c40c22e0 2024-03-04T08:28:18.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.370+0000 7fb7c2000640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb7a4066f60 0x7fb7a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:18.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.370+0000 7fb7c2000640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb7a4066f60 0x7fb7a4069420 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00ad000 tx=0x7fb7b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:18.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.374+0000 7fb7bb400640 1 -- 172.21.15.113:0/2655036962 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb7b4020020 con 0x7fb7c40c22e0 2024-03-04T08:28:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.558+0000 7fb7c9e00640 1 -- 172.21.15.113:0/2655036962 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7c40c88b0 con 0x7fb7c40c22e0 2024-03-04T08:28:18.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.558+0000 7fb7bb400640 1 -- 172.21.15.113:0/2655036962 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fb7b4005480 con 0x7fb7c40c22e0 2024-03-04T08:28:18.558 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:18.558 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:18.558 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 -- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb7a4066f60 msgr2=0x7fb7a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb7a4066f60 0x7fb7a4069420 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00ad000 tx=0x7fb7b0008040 comp rx=0 tx=0).stop 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 -- 172.21.15.113:0/2655036962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 msgr2=0x7fb7c41513d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 0x7fb7c41513d0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400c450 tx=0x7fb7b400c910 comp rx=0 tx=0).stop 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 -- 172.21.15.113:0/2655036962 shutdown_connections 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb7c414e4c0 0x7fb7c414ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb7c40c2d80 0x7fb7c414faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb7a4066f60 0x7fb7a4069420 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 --2- 172.21.15.113:0/2655036962 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb7c40c22e0 0x7fb7c41513d0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 -- 172.21.15.113:0/2655036962 >> 172.21.15.113:0/2655036962 conn(0x7fb7c406e820 msgr2=0x7fb7c4072c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:18.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 -- 172.21.15.113:0/2655036962 shutdown_connections 2024-03-04T08:28:18.561 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:18.562+0000 7fb7c9e00640 1 -- 172.21.15.113:0/2655036962 wait complete. 2024-03-04T08:28:18.578 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:20.580 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:20.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bfa00640 1 Processor -- start 2024-03-04T08:28:20.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bfa00640 1 -- start start 2024-03-04T08:28:20.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bfa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25b815b900 0x7f25b8163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:20.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bfa00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b80c27f0 con 0x7f25b814e4c0 2024-03-04T08:28:20.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bfa00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b80c2a90 con 0x7f25b80c22e0 2024-03-04T08:28:20.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bfa00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b815ad60 con 0x7f25b815b900 2024-03-04T08:28:20.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bda00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25b815b900 0x7f25b8163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:20.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bda00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f25b815b900 0x7f25b8163100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34914/0 (socket says 172.21.15.113:34914) 2024-03-04T08:28:20.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bd000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f25b814e4c0 0x7f25b8072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48912/0 (socket says 172.21.15.113:48912) 2024-03-04T08:28:20.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bda00640 1 -- 172.21.15.113:0/3244949225 learned_addr learned my addr 172.21.15.113:0/3244949225 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:20.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bda00640 1 -- 172.21.15.113:0/3244949225 >> v1:172.21.15.136:6789/0 conn(0x7f25b80c22e0 legacy=0x7f25b815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:20.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bda00640 1 -- 172.21.15.113:0/3244949225 >> v1:172.21.15.113:6789/0 conn(0x7f25b814e4c0 legacy=0x7f25b8072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:20.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25bda00640 1 -- 172.21.15.113:0/3244949225 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b8070c60 con 0x7f25b815b900 2024-03-04T08:28:20.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.734+0000 7f25b7e00640 1 -- 172.21.15.113:0/3244949225 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30327027 0 0) 0x7f25b80c27f0 con 0x7f25b814e4c0 2024-03-04T08:28:20.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25bda00640 1 --2- 172.21.15.113:0/3244949225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 0x7f25b8163100 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009e50 tx=0x7f25b001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54396316364c84ff server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:20.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25b7e00640 1 -- 172.21.15.113:0/3244949225 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b00b4070 con 0x7f25b815b900 2024-03-04T08:28:20.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25b7e00640 1 -- 172.21.15.113:0/3244949225 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b00af400 con 0x7f25b815b900 2024-03-04T08:28:20.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25b7e00640 1 -- 172.21.15.113:0/3244949225 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b00af7d0 con 0x7f25b815b900 2024-03-04T08:28:20.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25bfa00640 1 -- 172.21.15.113:0/3244949225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 msgr2=0x7f25b8163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:20.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25bfa00640 1 --2- 172.21.15.113:0/3244949225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 0x7f25b8163100 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009e50 tx=0x7f25b001f210 comp rx=0 tx=0).stop 2024-03-04T08:28:20.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25bfa00640 1 -- 172.21.15.113:0/3244949225 shutdown_connections 2024-03-04T08:28:20.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25bfa00640 1 --2- 172.21.15.113:0/3244949225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 0x7f25b8163100 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:20.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25bfa00640 1 -- 172.21.15.113:0/3244949225 >> 172.21.15.113:0/3244949225 conn(0x7f25b806e820 msgr2=0x7f25b806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:20.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25bfa00640 1 -- 172.21.15.113:0/3244949225 shutdown_connections 2024-03-04T08:28:20.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.738+0000 7f25bfa00640 1 -- 172.21.15.113:0/3244949225 wait complete. 2024-03-04T08:28:20.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bfa00640 1 Processor -- start 2024-03-04T08:28:20.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bfa00640 1 -- start start 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bfa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25b80c22e0 0x7f25b8153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bfa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b814e4c0 0x7f25b806b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bfa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 0x7f25b814cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bfa00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8071500 con 0x7f25b815b900 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bfa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8071220 con 0x7f25b80c22e0 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bfa00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b814e8a0 con 0x7f25b814e4c0 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 0x7f25b814cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 0x7f25b814cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34920/0 (socket says 172.21.15.113:34920) 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bda00640 1 -- 172.21.15.113:0/3731125548 learned_addr learned my addr 172.21.15.113:0/3731125548 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bd000640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25b80c22e0 0x7f25b8153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25afe00640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b814e4c0 0x7f25b806b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bda00640 1 -- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b814e4c0 msgr2=0x7f25b806b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:20.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bda00640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b814e4c0 0x7f25b806b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:20.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bda00640 1 -- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25b80c22e0 msgr2=0x7f25b8153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:20.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bda00640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25b80c22e0 0x7f25b8153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:20.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bda00640 1 -- 172.21.15.113:0/3731125548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b80cb090 con 0x7f25b815b900 2024-03-04T08:28:20.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bda00640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 0x7f25b814cb30 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009e50 tx=0x7f25b001bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25b5600640 1 -- 172.21.15.113:0/3731125548 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b00b4040 con 0x7f25b815b900 2024-03-04T08:28:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25b5600640 1 -- 172.21.15.113:0/3731125548 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b00090d0 con 0x7f25b815b900 2024-03-04T08:28:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25b5600640 1 -- 172.21.15.113:0/3731125548 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b0003ba0 con 0x7f25b815b900 2024-03-04T08:28:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bfa00640 1 -- 172.21.15.113:0/3731125548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b80cadb0 con 0x7f25b815b900 2024-03-04T08:28:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.742+0000 7f25bfa00640 1 -- 172.21.15.113:0/3731125548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b815a140 con 0x7f25b815b900 2024-03-04T08:28:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.746+0000 7f25bfa00640 1 -- 172.21.15.113:0/3731125548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b8075c00 con 0x7f25b815b900 2024-03-04T08:28:20.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.746+0000 7f25b5600640 1 -- 172.21.15.113:0/3731125548 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25b00c3060 con 0x7f25b815b900 2024-03-04T08:28:20.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.746+0000 7f25b5600640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2594066f60 0x7f2594069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:20.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.746+0000 7f25b5600640 1 -- 172.21.15.113:0/3731125548 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f25b01233f0 con 0x7f25b815b900 2024-03-04T08:28:20.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.746+0000 7f25bd000640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2594066f60 0x7f2594069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:20.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.746+0000 7f25bd000640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2594066f60 0x7f2594069420 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f25a8007fd0 tx=0x7f25a8007ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:20.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.750+0000 7f25b5600640 1 -- 172.21.15.113:0/3731125548 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f25b0018d70 con 0x7f25b815b900 2024-03-04T08:28:20.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.938+0000 7f25bfa00640 1 -- 172.21.15.113:0/3731125548 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25b8155190 con 0x7f25b815b900 2024-03-04T08:28:20.938 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.938+0000 7f25b5600640 1 -- 172.21.15.113:0/3731125548 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f25b00c43b0 con 0x7f25b815b900 2024-03-04T08:28:20.938 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:20.938 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:20.939 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:20.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 -- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2594066f60 msgr2=0x7f2594069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:20.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2594066f60 0x7f2594069420 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f25a8007fd0 tx=0x7f25a8007ba0 comp rx=0 tx=0).stop 2024-03-04T08:28:20.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 -- 172.21.15.113:0/3731125548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 msgr2=0x7f25b814cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:20.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 0x7f25b814cb30 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009e50 tx=0x7f25b001bfb0 comp rx=0 tx=0).stop 2024-03-04T08:28:20.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 -- 172.21.15.113:0/3731125548 shutdown_connections 2024-03-04T08:28:20.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2594066f60 0x7f2594069420 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:20.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f25b815b900 0x7f25b814cb30 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:20.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f25b814e4c0 0x7f25b806b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:20.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 --2- 172.21.15.113:0/3731125548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f25b80c22e0 0x7f25b8153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:20.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 -- 172.21.15.113:0/3731125548 >> 172.21.15.113:0/3731125548 conn(0x7f25b806e820 msgr2=0x7f25b80c1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:20.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 -- 172.21.15.113:0/3731125548 shutdown_connections 2024-03-04T08:28:20.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:20.942+0000 7f25bfa00640 1 -- 172.21.15.113:0/3731125548 wait complete. 2024-03-04T08:28:20.959 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:22.960 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.106+0000 7f15dda00640 1 Processor -- start 2024-03-04T08:28:23.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.106+0000 7f15dda00640 1 -- start start 2024-03-04T08:28:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15dda00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f15d81527f0 0x7f15d8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15dda00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15d80c8200 0x7f15d8072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15dda00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15d8152bd0 0x7f15d81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15dda00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d8071df0 con 0x7f15d8152bd0 2024-03-04T08:28:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15dda00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d81597b0 con 0x7f15d81527f0 2024-03-04T08:28:23.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15dda00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d81591f0 con 0x7f15d80c8200 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15d7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15d8152bd0 0x7f15d81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15d7400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f15d8152bd0 0x7f15d81587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34936/0 (socket says 172.21.15.113:34936) 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15d7400640 1 -- 172.21.15.113:0/2392857005 learned_addr learned my addr 172.21.15.113:0/2392857005 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15d6a00640 1 --2- 172.21.15.113:0/2392857005 >> v2:172.21.15.136:3300/0 conn(0x7f15d81527f0 0x7f15d8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15ce000640 1 --2- 172.21.15.113:0/2392857005 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15d80c8200 0x7f15d8072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15d7400640 1 -- 172.21.15.113:0/2392857005 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15d80c8200 msgr2=0x7f15d8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15d7400640 1 --2- 172.21.15.113:0/2392857005 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15d80c8200 0x7f15d8072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15d7400640 1 -- 172.21.15.113:0/2392857005 >> v2:172.21.15.136:3300/0 conn(0x7f15d81527f0 msgr2=0x7f15d8070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15d7400640 1 --2- 172.21.15.113:0/2392857005 >> v2:172.21.15.136:3300/0 conn(0x7f15d81527f0 0x7f15d8070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15d7400640 1 -- 172.21.15.113:0/2392857005 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d81594d0 con 0x7f15d8152bd0 2024-03-04T08:28:23.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.110+0000 7f15d6a00640 1 --2- 172.21.15.113:0/2392857005 >> v2:172.21.15.136:3300/0 conn(0x7f15d81527f0 0x7f15d8070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:23.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15d7400640 1 --2- 172.21.15.113:0/2392857005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 0x7f15d81587d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f15d806f400 tx=0x7f15d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c192a1ddfd3abb38 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:23.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15d6000640 1 -- 172.21.15.113:0/2392857005 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00a5760 con 0x7f15d8152bd0 2024-03-04T08:28:23.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15d6000640 1 -- 172.21.15.113:0/2392857005 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d00a5900 con 0x7f15d8152bd0 2024-03-04T08:28:23.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15d6000640 1 -- 172.21.15.113:0/2392857005 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00a5cd0 con 0x7f15d8152bd0 2024-03-04T08:28:23.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 -- 172.21.15.113:0/2392857005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 msgr2=0x7f15d81587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:23.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 --2- 172.21.15.113:0/2392857005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 0x7f15d81587d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f15d806f400 tx=0x7f15d001c9f0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 -- 172.21.15.113:0/2392857005 shutdown_connections 2024-03-04T08:28:23.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 --2- 172.21.15.113:0/2392857005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 0x7f15d81587d0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 --2- 172.21.15.113:0/2392857005 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15d80c8200 0x7f15d8072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 --2- 172.21.15.113:0/2392857005 >> v2:172.21.15.136:3300/0 conn(0x7f15d81527f0 0x7f15d8070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 -- 172.21.15.113:0/2392857005 >> 172.21.15.113:0/2392857005 conn(0x7f15d806e820 msgr2=0x7f15d806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 -- 172.21.15.113:0/2392857005 shutdown_connections 2024-03-04T08:28:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 -- 172.21.15.113:0/2392857005 wait complete. 2024-03-04T08:28:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 Processor -- start 2024-03-04T08:28:23.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.114+0000 7f15dda00640 1 -- start start 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15dda00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15d80c8200 0x7f15d8149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15dda00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15d81527f0 0x7f15d814a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15dda00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 0x7f15d8157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15dda00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d815bf40 con 0x7f15d8152bd0 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15dda00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d815bdc0 con 0x7f15d80c8200 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15dda00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d815c0c0 con 0x7f15d81527f0 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15d7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 0x7f15d8157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15d7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 0x7f15d8157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34948/0 (socket says 172.21.15.113:34948) 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15d7400640 1 -- 172.21.15.113:0/4070742953 learned_addr learned my addr 172.21.15.113:0/4070742953 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15d6a00640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15d80c8200 0x7f15d8149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15ce000640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15d81527f0 0x7f15d814a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:23.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15d7400640 1 -- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15d81527f0 msgr2=0x7f15d814a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15d7400640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15d81527f0 0x7f15d814a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15d7400640 1 -- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15d80c8200 msgr2=0x7f15d8149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15d7400640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15d80c8200 0x7f15d8149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15d7400640 1 -- 172.21.15.113:0/4070742953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d80b0920 con 0x7f15d8152bd0 2024-03-04T08:28:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15d7400640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 0x7f15d8157450 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f15d001d290 tx=0x7f15d0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15cf400640 1 -- 172.21.15.113:0/4070742953 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00b8670 con 0x7f15d8152bd0 2024-03-04T08:28:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15cf400640 1 -- 172.21.15.113:0/4070742953 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d00be070 con 0x7f15d8152bd0 2024-03-04T08:28:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15cf400640 1 -- 172.21.15.113:0/4070742953 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00b8ab0 con 0x7f15d8152bd0 2024-03-04T08:28:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15dda00640 1 -- 172.21.15.113:0/4070742953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d80b0be0 con 0x7f15d8152bd0 2024-03-04T08:28:23.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15dda00640 1 -- 172.21.15.113:0/4070742953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d80b11a0 con 0x7f15d8152bd0 2024-03-04T08:28:23.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15dda00640 1 -- 172.21.15.113:0/4070742953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d8075c00 con 0x7f15d8152bd0 2024-03-04T08:28:23.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.118+0000 7f15cf400640 1 -- 172.21.15.113:0/4070742953 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15d00b8c50 con 0x7f15d8152bd0 2024-03-04T08:28:23.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.122+0000 7f15cf400640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f15b4066f60 0x7f15b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:23.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.122+0000 7f15cf400640 1 -- 172.21.15.113:0/4070742953 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f15d0120d80 con 0x7f15d8152bd0 2024-03-04T08:28:23.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.122+0000 7f15d6a00640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f15b4066f60 0x7f15b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:23.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.122+0000 7f15d6a00640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f15b4066f60 0x7f15b4069420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c900 tx=0x7f15c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:23.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.126+0000 7f15cf400640 1 -- 172.21.15.113:0/4070742953 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f15d00165e0 con 0x7f15d8152bd0 2024-03-04T08:28:23.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.310+0000 7f15dda00640 1 -- 172.21.15.113:0/4070742953 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15d80687e0 con 0x7f15d8152bd0 2024-03-04T08:28:23.310 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.310+0000 7f15cf400640 1 -- 172.21.15.113:0/4070742953 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f15d00f1430 con 0x7f15d8152bd0 2024-03-04T08:28:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:23.311 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:23.311 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 -- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f15b4066f60 msgr2=0x7f15b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f15b4066f60 0x7f15b4069420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c900 tx=0x7f15c8008040 comp rx=0 tx=0).stop 2024-03-04T08:28:23.312 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 -- 172.21.15.113:0/4070742953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 msgr2=0x7f15d8157450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 0x7f15d8157450 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f15d001d290 tx=0x7f15d0019790 comp rx=0 tx=0).stop 2024-03-04T08:28:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 -- 172.21.15.113:0/4070742953 shutdown_connections 2024-03-04T08:28:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f15b4066f60 0x7f15b4069420 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f15d8152bd0 0x7f15d8157450 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f15d81527f0 0x7f15d814a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 --2- 172.21.15.113:0/4070742953 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f15d80c8200 0x7f15d8149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 -- 172.21.15.113:0/4070742953 >> 172.21.15.113:0/4070742953 conn(0x7f15d806e820 msgr2=0x7f15d814fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 -- 172.21.15.113:0/4070742953 shutdown_connections 2024-03-04T08:28:23.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:23.314+0000 7f15dda00640 1 -- 172.21.15.113:0/4070742953 wait complete. 2024-03-04T08:28:23.331 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:25.332 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:25.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a8800640 1 Processor -- start 2024-03-04T08:28:25.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a8800640 1 -- start start 2024-03-04T08:28:25.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a8800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff9a014e4c0 0x7ff9a0072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:25.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a8800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9a015b900 0x7ff9a00731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:25.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a8800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a00c27f0 con 0x7ff9a00c22e0 2024-03-04T08:28:25.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a8800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a015ad60 con 0x7ff9a014e4c0 2024-03-04T08:28:25.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a8800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a00c2a90 con 0x7ff9a015b900 2024-03-04T08:28:25.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff9a014e4c0 0x7ff9a0072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:25.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff9a014e4c0 0x7ff9a0072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34952/0 (socket says 172.21.15.113:34952) 2024-03-04T08:28:25.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a5e00640 1 -- 172.21.15.113:0/2510561705 learned_addr learned my addr 172.21.15.113:0/2510561705 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:25.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a6800640 1 --2- 172.21.15.113:0/2510561705 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9a015b900 0x7ff9a00731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:25.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a5e00640 1 -- 172.21.15.113:0/2510561705 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9a015b900 msgr2=0x7ff9a00731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:25.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a5e00640 1 --2- 172.21.15.113:0/2510561705 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9a015b900 0x7ff9a00731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:25.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a5e00640 1 -- 172.21.15.113:0/2510561705 >> v1:172.21.15.136:6789/0 conn(0x7ff9a00c22e0 legacy=0x7ff9a015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:25.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.482+0000 7ff9a5e00640 1 -- 172.21.15.113:0/2510561705 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0070c60 con 0x7ff9a014e4c0 2024-03-04T08:28:25.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a5e00640 1 --2- 172.21.15.113:0/2510561705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 0x7ff9a0072c80 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff994009870 tx=0x7ff99401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b07378a61a5dfb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:25.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a4a00640 1 -- 172.21.15.113:0/2510561705 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9940b5070 con 0x7ff9a014e4c0 2024-03-04T08:28:25.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a4a00640 1 -- 172.21.15.113:0/2510561705 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9940a8a50 con 0x7ff9a014e4c0 2024-03-04T08:28:25.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a4a00640 1 -- 172.21.15.113:0/2510561705 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9940a8e20 con 0x7ff9a014e4c0 2024-03-04T08:28:25.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a8800640 1 -- 172.21.15.113:0/2510561705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 msgr2=0x7ff9a0072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:25.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a8800640 1 --2- 172.21.15.113:0/2510561705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 0x7ff9a0072c80 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff994009870 tx=0x7ff99401ed50 comp rx=0 tx=0).stop 2024-03-04T08:28:25.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a8800640 1 -- 172.21.15.113:0/2510561705 shutdown_connections 2024-03-04T08:28:25.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a8800640 1 --2- 172.21.15.113:0/2510561705 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9a015b900 0x7ff9a00731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:25.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a8800640 1 --2- 172.21.15.113:0/2510561705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 0x7ff9a0072c80 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:25.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a8800640 1 -- 172.21.15.113:0/2510561705 >> 172.21.15.113:0/2510561705 conn(0x7ff9a006e820 msgr2=0x7ff9a006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:25.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a8800640 1 -- 172.21.15.113:0/2510561705 shutdown_connections 2024-03-04T08:28:25.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.486+0000 7ff9a8800640 1 -- 172.21.15.113:0/2510561705 wait complete. 2024-03-04T08:28:25.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 Processor -- start 2024-03-04T08:28:25.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 -- start start 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9a00c22e0 0x7ff9a006b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 0x7ff9a014a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9a015b900 0x7ff9a01f4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a0071500 con 0x7ff9a014e4c0 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a0071220 con 0x7ff9a00c22e0 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a014e8a0 con 0x7ff9a015b900 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 0x7ff9a014a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 0x7ff9a014a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:34960/0 (socket says 172.21.15.113:34960) 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5400640 1 -- 172.21.15.113:0/3190787733 learned_addr learned my addr 172.21.15.113:0/3190787733 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5e00640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9a00c22e0 0x7ff9a006b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a6800640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9a015b900 0x7ff9a01f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:25.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5400640 1 -- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9a015b900 msgr2=0x7ff9a01f4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5400640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9a015b900 0x7ff9a01f4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5400640 1 -- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9a00c22e0 msgr2=0x7ff9a006b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5400640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9a00c22e0 0x7ff9a006b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5400640 1 -- 172.21.15.113:0/3190787733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a01524a0 con 0x7ff9a014e4c0 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5e00640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9a00c22e0 0x7ff9a006b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a5400640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 0x7ff9a014a060 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ff99000c370 tx=0x7ff99000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff99a000640 1 -- 172.21.15.113:0/3190787733 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9900052d0 con 0x7ff9a014e4c0 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff99a000640 1 -- 172.21.15.113:0/3190787733 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff990005470 con 0x7ff9a014e4c0 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 -- 172.21.15.113:0/3190787733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a0152780 con 0x7ff9a014e4c0 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff99a000640 1 -- 172.21.15.113:0/3190787733 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff990007260 con 0x7ff9a014e4c0 2024-03-04T08:28:25.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 -- 172.21.15.113:0/3190787733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a0152d40 con 0x7ff9a014e4c0 2024-03-04T08:28:25.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.490+0000 7ff9a8800640 1 -- 172.21.15.113:0/3190787733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a0152f80 con 0x7ff9a014e4c0 2024-03-04T08:28:25.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.494+0000 7ff99a000640 1 -- 172.21.15.113:0/3190787733 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff990007400 con 0x7ff9a014e4c0 2024-03-04T08:28:25.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.494+0000 7ff99a000640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff984066f90 0x7ff984069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:25.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.494+0000 7ff99a000640 1 -- 172.21.15.113:0/3190787733 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7ff990084f70 con 0x7ff9a014e4c0 2024-03-04T08:28:25.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.494+0000 7ff9a5e00640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff984066f90 0x7ff984069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:25.493 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.494+0000 7ff9a5e00640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff984066f90 0x7ff984069450 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff9940b2000 tx=0x7ff994008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:25.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.498+0000 7ff99a000640 1 -- 172.21.15.113:0/3190787733 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff9900554d0 con 0x7ff9a014e4c0 2024-03-04T08:28:25.683 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 -- 172.21.15.113:0/3190787733 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9a0075c00 con 0x7ff9a014e4c0 2024-03-04T08:28:25.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff99a000640 1 -- 172.21.15.113:0/3190787733 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7ff990054dd0 con 0x7ff9a014e4c0 2024-03-04T08:28:25.684 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:25.684 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:25.684 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:25.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 -- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff984066f90 msgr2=0x7ff984069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:25.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff984066f90 0x7ff984069450 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff9940b2000 tx=0x7ff994008040 comp rx=0 tx=0).stop 2024-03-04T08:28:25.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 -- 172.21.15.113:0/3190787733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 msgr2=0x7ff9a014a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:25.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 0x7ff9a014a060 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7ff99000c370 tx=0x7ff99000c830 comp rx=0 tx=0).stop 2024-03-04T08:28:25.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 -- 172.21.15.113:0/3190787733 shutdown_connections 2024-03-04T08:28:25.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff9a015b900 0x7ff9a01f4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:25.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff984066f90 0x7ff984069450 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:25.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9a014e4c0 0x7ff9a014a060 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:25.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 --2- 172.21.15.113:0/3190787733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff9a00c22e0 0x7ff9a006b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:25.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 -- 172.21.15.113:0/3190787733 >> 172.21.15.113:0/3190787733 conn(0x7ff9a006e820 msgr2=0x7ff9a00c75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:25.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 -- 172.21.15.113:0/3190787733 shutdown_connections 2024-03-04T08:28:25.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:25.686+0000 7ff9a8800640 1 -- 172.21.15.113:0/3190787733 wait complete. 2024-03-04T08:28:25.705 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:28:25.706 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:28:27.707 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:27.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.854+0000 7f8547e00640 1 Processor -- start 2024-03-04T08:28:27.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.854+0000 7f8547e00640 1 -- start start 2024-03-04T08:28:27.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8547e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f854815d260 0x7f85481663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:27.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8547e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854806c310 con 0x7f854815b8a0 2024-03-04T08:28:27.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8547e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854806c5f0 con 0x7f854806dcb0 2024-03-04T08:28:27.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8547e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854815c6c0 con 0x7f854815d260 2024-03-04T08:28:27.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f854815b8a0 0x7f854806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58244/0 (socket says 172.21.15.113:58244) 2024-03-04T08:28:27.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546a00640 1 -- 172.21.15.113:0/82649732 learned_addr learned my addr 172.21.15.113:0/82649732 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:27.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8547400640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f854815d260 0x7f85481663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:27.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216062684 0 0) 0x7f854806c310 con 0x7f854815b8a0 2024-03-04T08:28:27.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f852c003530 con 0x7f854815b8a0 2024-03-04T08:28:27.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922975837 0 0) 0x7f854806c5f0 con 0x7f854806dcb0 2024-03-04T08:28:27.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f854806c310 con 0x7f854806dcb0 2024-03-04T08:28:27.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3382287096 0 0) 0x7f852c003530 con 0x7f854815b8a0 2024-03-04T08:28:27.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f854815d260 msgr2=0x7f85481663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:27.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f854815d260 0x7f85481663a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:27.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 >> v1:172.21.15.136:6789/0 conn(0x7f854806dcb0 legacy=0x7f854815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854815c070 con 0x7f854815b8a0 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f8538001680 con 0x7f854815b8a0 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 >> v1:172.21.15.113:6789/0 conn(0x7f854815b8a0 legacy=0x7f854806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 0x7f852c003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f852c004810 0x7f852c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f854815d260 0x7f852c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854815c6c0 con 0x7f852c003b80 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852c00bab0 con 0x7f852c004810 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852c00bcb0 con 0x7f854815d260 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546a00640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 0x7f852c003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f853e000640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f852c004810 0x7f852c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8547400640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f854815d260 0x7f852c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546a00640 1 -- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f854815d260 msgr2=0x7f852c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546a00640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f854815d260 0x7f852c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:27.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546a00640 1 -- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f852c004810 msgr2=0x7f852c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546a00640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f852c004810 0x7f852c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:27.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.858+0000 7f8546a00640 1 -- 172.21.15.113:0/82649732 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c003530 con 0x7f852c003b80 2024-03-04T08:28:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8546a00640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 0x7f852c003fa0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f8538007fa0 tx=0x7f853801cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78e7cb05900d3ef server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538007bf0 con 0x7f852c003b80 2024-03-04T08:28:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85380a9070 con 0x7f852c003b80 2024-03-04T08:28:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8546000640 1 -- 172.21.15.113:0/82649732 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538005d40 con 0x7f852c003b80 2024-03-04T08:28:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8547e00640 1 -- 172.21.15.113:0/82649732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 msgr2=0x7f852c003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8547e00640 1 --2- 172.21.15.113:0/82649732 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 0x7f852c003fa0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f8538007fa0 tx=0x7f853801cb00 comp rx=0 tx=0).stop 2024-03-04T08:28:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8547400640 1 -- 172.21.15.113:0/82649732 reap_dead start 2024-03-04T08:28:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8547e00640 1 -- 172.21.15.113:0/82649732 shutdown_connections 2024-03-04T08:28:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8547e00640 1 -- 172.21.15.113:0/82649732 >> 172.21.15.113:0/82649732 conn(0x7f8548072e30 msgr2=0x7f8548066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:27.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8547e00640 1 -- 172.21.15.113:0/82649732 shutdown_connections 2024-03-04T08:28:27.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8547e00640 1 -- 172.21.15.113:0/82649732 wait complete. 2024-03-04T08:28:27.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8547e00640 1 Processor -- start 2024-03-04T08:28:27.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.862+0000 7f8547e00640 1 -- start start 2024-03-04T08:28:27.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8547e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 0x7f85481545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:27.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8547e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85480c6c70 0x7f85480c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:27.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8547e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85480c4530 0x7f85480c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8547e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854806d840 con 0x7f852c003b80 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8547e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854815bc80 con 0x7f85480c4530 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8547e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854806f660 con 0x7f85480c6c70 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8546a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 0x7f85481545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8546a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 0x7f85481545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48414/0 (socket says 172.21.15.113:48414) 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8546a00640 1 -- 172.21.15.113:0/1286700347 learned_addr learned my addr 172.21.15.113:0/1286700347 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f853e000640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85480c6c70 0x7f85480c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8547400640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85480c4530 0x7f85480c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8546a00640 1 -- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85480c6c70 msgr2=0x7f85480c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8546a00640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85480c6c70 0x7f85480c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8546a00640 1 -- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85480c4530 msgr2=0x7f85480c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8546a00640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85480c4530 0x7f85480c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8546a00640 1 -- 172.21.15.113:0/1286700347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85480c4ef0 con 0x7f852c003b80 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f853e000640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85480c6c70 0x7f85480c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:27.865 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8546a00640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 0x7f85481545b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f8538007fa0 tx=0x7f852c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f853f400640 1 -- 172.21.15.113:0/1286700347 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538007bf0 con 0x7f852c003b80 2024-03-04T08:28:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f853f400640 1 -- 172.21.15.113:0/1286700347 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85380a6d50 con 0x7f852c003b80 2024-03-04T08:28:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8547e00640 1 -- 172.21.15.113:0/1286700347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85480c5d00 con 0x7f852c003b80 2024-03-04T08:28:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f853f400640 1 -- 172.21.15.113:0/1286700347 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538004a10 con 0x7f852c003b80 2024-03-04T08:28:27.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8547e00640 1 -- 172.21.15.113:0/1286700347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8548157260 con 0x7f852c003b80 2024-03-04T08:28:27.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f8547e00640 1 -- 172.21.15.113:0/1286700347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8548075a80 con 0x7f852c003b80 2024-03-04T08:28:27.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.866+0000 7f853f400640 1 -- 172.21.15.113:0/1286700347 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8538004bb0 con 0x7f852c003b80 2024-03-04T08:28:27.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.870+0000 7f853f400640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8524066f60 0x7f8524069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:27.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.870+0000 7f853f400640 1 -- 172.21.15.113:0/1286700347 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f853811fa10 con 0x7f852c003b80 2024-03-04T08:28:27.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.870+0000 7f853e000640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8524066f60 0x7f8524069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:27.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.870+0000 7f853e000640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8524066f60 0x7f8524069420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8530005630 tx=0x7f8530009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:27.871 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:27.874+0000 7f853f400640 1 -- 172.21.15.113:0/1286700347 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8538016810 con 0x7f852c003b80 2024-03-04T08:28:28.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 -- 172.21.15.113:0/1286700347 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f854806d4d0 con 0x7f852c003b80 2024-03-04T08:28:28.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f853f400640 1 -- 172.21.15.113:0/1286700347 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f85380eff40 con 0x7f852c003b80 2024-03-04T08:28:28.060 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:28.060 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:28.060 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 -- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8524066f60 msgr2=0x7f8524069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8524066f60 0x7f8524069420 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f8530005630 tx=0x7f8530009040 comp rx=0 tx=0).stop 2024-03-04T08:28:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 -- 172.21.15.113:0/1286700347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 msgr2=0x7f85481545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 0x7f85481545b0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f8538007fa0 tx=0x7f852c005840 comp rx=0 tx=0).stop 2024-03-04T08:28:28.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 -- 172.21.15.113:0/1286700347 shutdown_connections 2024-03-04T08:28:28.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85480c4530 0x7f85480c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:28.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f85480c6c70 0x7f85480c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:28.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f8524066f60 0x7f8524069420 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:28.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 --2- 172.21.15.113:0/1286700347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f852c003b80 0x7f85481545b0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:28.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 -- 172.21.15.113:0/1286700347 >> 172.21.15.113:0/1286700347 conn(0x7f8548072e30 msgr2=0x7f85480cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:28.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 -- 172.21.15.113:0/1286700347 shutdown_connections 2024-03-04T08:28:28.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:28.062+0000 7f8547e00640 1 -- 172.21.15.113:0/1286700347 wait complete. 2024-03-04T08:28:28.082 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:30.083 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:30.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.230+0000 7f19fea00640 1 Processor -- start 2024-03-04T08:28:30.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.230+0000 7f19fea00640 1 -- start start 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19fea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19f814e4c0 0x7f19f8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19fea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f80c22e0 0x7f19f80731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19fea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f19f80c2d80 0x7f19f815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19fea00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f815ad60 con 0x7f19f814e4c0 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19fea00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f80c27f0 con 0x7f19f80c2d80 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19fea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f80c2a90 con 0x7f19f80c22e0 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19f7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19f814e4c0 0x7f19f8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19f7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f19f814e4c0 0x7f19f8072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48422/0 (socket says 172.21.15.113:48422) 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19f7e00640 1 -- 172.21.15.113:0/2860628556 learned_addr learned my addr 172.21.15.113:0/2860628556 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19ef400640 1 --2- 172.21.15.113:0/2860628556 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f80c22e0 0x7f19f80731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19fca00640 1 --2- 172.21.15.113:0/2860628556 >> v2:172.21.15.136:3300/0 conn(0x7f19f80c2d80 0x7f19f815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:30.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19f7e00640 1 -- 172.21.15.113:0/2860628556 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f80c22e0 msgr2=0x7f19f80731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19f7e00640 1 --2- 172.21.15.113:0/2860628556 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f80c22e0 0x7f19f80731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19f7e00640 1 -- 172.21.15.113:0/2860628556 >> v2:172.21.15.136:3300/0 conn(0x7f19f80c2d80 msgr2=0x7f19f815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19f7e00640 1 --2- 172.21.15.113:0/2860628556 >> v2:172.21.15.136:3300/0 conn(0x7f19f80c2d80 0x7f19f815b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19f7e00640 1 -- 172.21.15.113:0/2860628556 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f8070c60 con 0x7f19f814e4c0 2024-03-04T08:28:30.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.234+0000 7f19ef400640 1 --2- 172.21.15.113:0/2860628556 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f80c22e0 0x7f19f80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:30.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19f7e00640 1 --2- 172.21.15.113:0/2860628556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f814e4c0 0x7f19f8072c80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f19f80c2700 tx=0x7f19e801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e490d36f2e12a7ac server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:30.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19f7400640 1 -- 172.21.15.113:0/2860628556 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e80b5070 con 0x7f19f814e4c0 2024-03-04T08:28:30.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19f7400640 1 -- 172.21.15.113:0/2860628556 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e80a89d0 con 0x7f19f814e4c0 2024-03-04T08:28:30.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19f7400640 1 -- 172.21.15.113:0/2860628556 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e80a8da0 con 0x7f19f814e4c0 2024-03-04T08:28:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 -- 172.21.15.113:0/2860628556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f814e4c0 msgr2=0x7f19f8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 --2- 172.21.15.113:0/2860628556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f814e4c0 0x7f19f8072c80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f19f80c2700 tx=0x7f19e801ed10 comp rx=0 tx=0).stop 2024-03-04T08:28:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 -- 172.21.15.113:0/2860628556 shutdown_connections 2024-03-04T08:28:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 --2- 172.21.15.113:0/2860628556 >> v2:172.21.15.136:3300/0 conn(0x7f19f80c2d80 0x7f19f815b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 --2- 172.21.15.113:0/2860628556 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f80c22e0 0x7f19f80731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 --2- 172.21.15.113:0/2860628556 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f814e4c0 0x7f19f8072c80 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 -- 172.21.15.113:0/2860628556 >> 172.21.15.113:0/2860628556 conn(0x7f19f806e820 msgr2=0x7f19f806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 -- 172.21.15.113:0/2860628556 shutdown_connections 2024-03-04T08:28:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 -- 172.21.15.113:0/2860628556 wait complete. 2024-03-04T08:28:30.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 Processor -- start 2024-03-04T08:28:30.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 -- start start 2024-03-04T08:28:30.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f80c22e0 0x7f19f81541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f80c2d80 0x7f19f80cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.238+0000 7f19fea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f814e4c0 0x7f19f81588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19fea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f8071500 con 0x7f19f80c2d80 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19fea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f8071220 con 0x7f19f80c22e0 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19fea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f814e8a0 con 0x7f19f814e4c0 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19ef400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f80c2d80 0x7f19f80cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19ef400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f80c2d80 0x7f19f80cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48436/0 (socket says 172.21.15.113:48436) 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19ef400640 1 -- 172.21.15.113:0/339770870 learned_addr learned my addr 172.21.15.113:0/339770870 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19fca00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f814e4c0 0x7f19f81588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19f7e00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f80c22e0 0x7f19f81541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19ef400640 1 -- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f814e4c0 msgr2=0x7f19f81588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:30.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19ef400640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f814e4c0 0x7f19f81588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19ef400640 1 -- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f80c22e0 msgr2=0x7f19f81541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:30.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19ef400640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f80c22e0 0x7f19f81541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19ef400640 1 -- 172.21.15.113:0/339770870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f8154720 con 0x7f19f80c2d80 2024-03-04T08:28:30.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19f7e00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f80c22e0 0x7f19f81541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:30.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19ef400640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f80c2d80 0x7f19f80cb550 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e7e0 tx=0x7f19e000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:30.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19f4c00640 1 -- 172.21.15.113:0/339770870 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0017070 con 0x7f19f80c2d80 2024-03-04T08:28:30.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19f4c00640 1 -- 172.21.15.113:0/339770870 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e000c6d0 con 0x7f19f80c2d80 2024-03-04T08:28:30.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19fea00640 1 -- 172.21.15.113:0/339770870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f81523e0 con 0x7f19f80c2d80 2024-03-04T08:28:30.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19f4c00640 1 -- 172.21.15.113:0/339770870 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e000f040 con 0x7f19f80c2d80 2024-03-04T08:28:30.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19fea00640 1 -- 172.21.15.113:0/339770870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f81529a0 con 0x7f19f80c2d80 2024-03-04T08:28:30.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19fea00640 1 -- 172.21.15.113:0/339770870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f8075c00 con 0x7f19f80c2d80 2024-03-04T08:28:30.244 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.242+0000 7f19f4c00640 1 -- 172.21.15.113:0/339770870 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19e001a070 con 0x7f19f80c2d80 2024-03-04T08:28:30.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.246+0000 7f19f4c00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d4066f90 0x7f19d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:30.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.246+0000 7f19f4c00640 1 -- 172.21.15.113:0/339770870 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f19e0085640 con 0x7f19f80c2d80 2024-03-04T08:28:30.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.246+0000 7f19f7e00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d4066f90 0x7f19d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:30.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.246+0000 7f19f7e00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d4066f90 0x7f19d4069450 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f19e80b2000 tx=0x7f19e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:30.246 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.250+0000 7f19f4c00640 1 -- 172.21.15.113:0/339770870 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f19e0055550 con 0x7f19f80c2d80 2024-03-04T08:28:30.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.434+0000 7f19fea00640 1 -- 172.21.15.113:0/339770870 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19f80c8b90 con 0x7f19f80c2d80 2024-03-04T08:28:30.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19f4c00640 1 -- 172.21.15.113:0/339770870 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f19e00595f0 con 0x7f19f80c2d80 2024-03-04T08:28:30.435 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:30.435 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:30.435 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:30.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 -- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d4066f90 msgr2=0x7f19d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:30.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d4066f90 0x7f19d4069450 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f19e80b2000 tx=0x7f19e8008040 comp rx=0 tx=0).stop 2024-03-04T08:28:30.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 -- 172.21.15.113:0/339770870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f80c2d80 msgr2=0x7f19f80cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:30.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f80c2d80 0x7f19f80cb550 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e7e0 tx=0x7f19e000eca0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 -- 172.21.15.113:0/339770870 shutdown_connections 2024-03-04T08:28:30.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f19f814e4c0 0x7f19f81588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f19d4066f90 0x7f19d4069450 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f19f80c2d80 0x7f19f80cb550 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 --2- 172.21.15.113:0/339770870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f80c22e0 0x7f19f81541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:30.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 -- 172.21.15.113:0/339770870 >> 172.21.15.113:0/339770870 conn(0x7f19f806e820 msgr2=0x7f19f814c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:30.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 -- 172.21.15.113:0/339770870 shutdown_connections 2024-03-04T08:28:30.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:30.438+0000 7f19fea00640 1 -- 172.21.15.113:0/339770870 wait complete. 2024-03-04T08:28:30.457 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:32.458 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:32.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabf6000640 1 Processor -- start 2024-03-04T08:28:32.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabf6000640 1 -- start start 2024-03-04T08:28:32.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabf6000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fabf00c08d0 0x7fabf006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:32.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabf6000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fabf015ad30 0x7fabf015b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:32.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabf6000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf015b6c0 con 0x7fabf0155100 2024-03-04T08:28:32.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabf6000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf00c8890 con 0x7fabf015ad30 2024-03-04T08:28:32.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabf6000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf015efd0 con 0x7fabf00c08d0 2024-03-04T08:28:32.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabefe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fabf015ad30 0x7fabf015b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:32.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabefe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fabf015ad30 0x7fabf015b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48444/0 (socket says 172.21.15.113:48444) 2024-03-04T08:28:32.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabefe00640 1 -- 172.21.15.113:0/2185072358 learned_addr learned my addr 172.21.15.113:0/2185072358 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:32.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabeea00640 1 --2- 172.21.15.113:0/2185072358 >> v2:172.21.15.136:3300/0 conn(0x7fabf00c08d0 0x7fabf006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:32.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabefe00640 1 -- 172.21.15.113:0/2185072358 >> v2:172.21.15.136:3300/0 conn(0x7fabf00c08d0 msgr2=0x7fabf006b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:32.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabefe00640 1 --2- 172.21.15.113:0/2185072358 >> v2:172.21.15.136:3300/0 conn(0x7fabf00c08d0 0x7fabf006b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:32.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabefe00640 1 -- 172.21.15.113:0/2185072358 >> v1:172.21.15.136:6789/0 conn(0x7fabf0155100 legacy=0x7fabf00cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:32.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.606+0000 7fabefe00640 1 -- 172.21.15.113:0/2185072358 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf00c7ef0 con 0x7fabf015ad30 2024-03-04T08:28:32.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabefe00640 1 --2- 172.21.15.113:0/2185072358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 0x7fabf015b180 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fabf0071b70 tx=0x7fabe801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1592ab4eca0f499e server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:32.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabee000640 1 -- 172.21.15.113:0/2185072358 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80a56c0 con 0x7fabf015ad30 2024-03-04T08:28:32.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabee000640 1 -- 172.21.15.113:0/2185072358 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe80a5860 con 0x7fabf015ad30 2024-03-04T08:28:32.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabee000640 1 -- 172.21.15.113:0/2185072358 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80a5c30 con 0x7fabf015ad30 2024-03-04T08:28:32.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabf6000640 1 -- 172.21.15.113:0/2185072358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 msgr2=0x7fabf015b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:32.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabf6000640 1 --2- 172.21.15.113:0/2185072358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 0x7fabf015b180 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fabf0071b70 tx=0x7fabe801c9f0 comp rx=0 tx=0).stop 2024-03-04T08:28:32.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabf6000640 1 -- 172.21.15.113:0/2185072358 shutdown_connections 2024-03-04T08:28:32.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabf6000640 1 --2- 172.21.15.113:0/2185072358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 0x7fabf015b180 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:32.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabf6000640 1 --2- 172.21.15.113:0/2185072358 >> v2:172.21.15.136:3300/0 conn(0x7fabf00c08d0 0x7fabf006b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:32.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabf6000640 1 -- 172.21.15.113:0/2185072358 >> 172.21.15.113:0/2185072358 conn(0x7fabf006deb0 msgr2=0x7fabf006e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:32.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabf6000640 1 -- 172.21.15.113:0/2185072358 shutdown_connections 2024-03-04T08:28:32.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.610+0000 7fabf6000640 1 -- 172.21.15.113:0/2185072358 wait complete. 2024-03-04T08:28:32.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 Processor -- start 2024-03-04T08:28:32.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 -- start start 2024-03-04T08:28:32.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabf00c08d0 0x7fabf01f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:32.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabf0155100 0x7fabf00c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:32.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 0x7fabf00c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:32.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf015ed30 con 0x7fabf015ad30 2024-03-04T08:28:32.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf015ebb0 con 0x7fabf0155100 2024-03-04T08:28:32.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00cd7d0 con 0x7fabf00c08d0 2024-03-04T08:28:32.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabefe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 0x7fabf00c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:32.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabefe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 0x7fabf00c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48452/0 (socket says 172.21.15.113:48452) 2024-03-04T08:28:32.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabefe00640 1 -- 172.21.15.113:0/1711144465 learned_addr learned my addr 172.21.15.113:0/1711144465 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabeea00640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabf0155100 0x7fabf00c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabef400640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabf00c08d0 0x7fabf01f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabefe00640 1 -- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabf00c08d0 msgr2=0x7fabf01f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabefe00640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabf00c08d0 0x7fabf01f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabefe00640 1 -- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabf0155100 msgr2=0x7fabf00c6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabefe00640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabf0155100 0x7fabf00c6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabefe00640 1 -- 172.21.15.113:0/1711144465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf006c950 con 0x7fabf015ad30 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabefe00640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 0x7fabf00c5250 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fabe801a620 tx=0x7fabe8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabe7400640 1 -- 172.21.15.113:0/1711144465 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80bc020 con 0x7fabf015ad30 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabe7400640 1 -- 172.21.15.113:0/1711144465 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe80b8670 con 0x7fabf015ad30 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabe7400640 1 -- 172.21.15.113:0/1711144465 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8009510 con 0x7fabf015ad30 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 -- 172.21.15.113:0/1711144465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf014a620 con 0x7fabf015ad30 2024-03-04T08:28:32.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 -- 172.21.15.113:0/1711144465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf014abe0 con 0x7fabf015ad30 2024-03-04T08:28:32.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.614+0000 7fabf6000640 1 -- 172.21.15.113:0/1711144465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf0075c00 con 0x7fabf015ad30 2024-03-04T08:28:32.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.618+0000 7fabe7400640 1 -- 172.21.15.113:0/1711144465 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabe80bf020 con 0x7fabf015ad30 2024-03-04T08:28:32.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.618+0000 7fabe7400640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabcc066f60 0x7fabcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:32.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.618+0000 7fabe7400640 1 -- 172.21.15.113:0/1711144465 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fabe8120a50 con 0x7fabf015ad30 2024-03-04T08:28:32.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.618+0000 7fabef400640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabcc066f60 0x7fabcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:32.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.618+0000 7fabef400640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabcc066f60 0x7fabcc069420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fabd8001650 tx=0x7fabd8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:32.620 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.622+0000 7fabe7400640 1 -- 172.21.15.113:0/1711144465 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fabe80c6030 con 0x7fabf015ad30 2024-03-04T08:28:32.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.806+0000 7fabf6000640 1 -- 172.21.15.113:0/1711144465 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabf015aab0 con 0x7fabf015ad30 2024-03-04T08:28:32.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabe7400640 1 -- 172.21.15.113:0/1711144465 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fabe80165e0 con 0x7fabf015ad30 2024-03-04T08:28:32.807 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:32.807 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:32.807 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 -- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabcc066f60 msgr2=0x7fabcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabcc066f60 0x7fabcc069420 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fabd8001650 tx=0x7fabd8009040 comp rx=0 tx=0).stop 2024-03-04T08:28:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 -- 172.21.15.113:0/1711144465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 msgr2=0x7fabf00c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 0x7fabf00c5250 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fabe801a620 tx=0x7fabe8019790 comp rx=0 tx=0).stop 2024-03-04T08:28:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 -- 172.21.15.113:0/1711144465 shutdown_connections 2024-03-04T08:28:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fabcc066f60 0x7fabcc069420 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fabf015ad30 0x7fabf00c5250 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabf0155100 0x7fabf00c6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 --2- 172.21.15.113:0/1711144465 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fabf00c08d0 0x7fabf01f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:32.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 -- 172.21.15.113:0/1711144465 >> 172.21.15.113:0/1711144465 conn(0x7fabf006deb0 msgr2=0x7fabf006e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 -- 172.21.15.113:0/1711144465 shutdown_connections 2024-03-04T08:28:32.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:32.810+0000 7fabf6000640 1 -- 172.21.15.113:0/1711144465 wait complete. 2024-03-04T08:28:32.828 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:34.829 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:34.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.974+0000 7fe0aa400640 1 Processor -- start 2024-03-04T08:28:34.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.974+0000 7fe0aa400640 1 -- start start 2024-03-04T08:28:34.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0aa400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0a41527f0 0x7fe0a4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:34.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0aa400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe0a40c8200 0x7fe0a4072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:34.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0aa400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0a4152bd0 0x7fe0a41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:34.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0aa400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a4071df0 con 0x7fe0a4152bd0 2024-03-04T08:28:34.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0aa400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a41597b0 con 0x7fe0a40c8200 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0aa400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a41591f0 con 0x7fe0a41527f0 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0a3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0a4152bd0 0x7fe0a41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0a3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0a4152bd0 0x7fe0a41587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48466/0 (socket says 172.21.15.113:48466) 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0a3e00640 1 -- 172.21.15.113:0/1322245722 learned_addr learned my addr 172.21.15.113:0/1322245722 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0a3400640 1 --2- 172.21.15.113:0/1322245722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0a41527f0 0x7fe0a4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0a2a00640 1 --2- 172.21.15.113:0/1322245722 >> v2:172.21.15.136:3300/0 conn(0x7fe0a40c8200 0x7fe0a4072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0a3e00640 1 -- 172.21.15.113:0/1322245722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0a41527f0 msgr2=0x7fe0a4070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0a3e00640 1 --2- 172.21.15.113:0/1322245722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0a41527f0 0x7fe0a4070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0a3e00640 1 -- 172.21.15.113:0/1322245722 >> v2:172.21.15.136:3300/0 conn(0x7fe0a40c8200 msgr2=0x7fe0a4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0a3e00640 1 --2- 172.21.15.113:0/1322245722 >> v2:172.21.15.136:3300/0 conn(0x7fe0a40c8200 0x7fe0a4072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:34.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.978+0000 7fe0a3e00640 1 -- 172.21.15.113:0/1322245722 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a41594d0 con 0x7fe0a4152bd0 2024-03-04T08:28:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0a3e00640 1 --2- 172.21.15.113:0/1322245722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 0x7fe0a41587d0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40c3520 tx=0x7fe09c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee4736e351e0c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0a2000640 1 -- 172.21.15.113:0/1322245722 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0b1070 con 0x7fe0a4152bd0 2024-03-04T08:28:34.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0a2000640 1 -- 172.21.15.113:0/1322245722 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09c0ac630 con 0x7fe0a4152bd0 2024-03-04T08:28:34.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0a2000640 1 -- 172.21.15.113:0/1322245722 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0a8050 con 0x7fe0a4152bd0 2024-03-04T08:28:34.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 -- 172.21.15.113:0/1322245722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 msgr2=0x7fe0a41587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:34.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 --2- 172.21.15.113:0/1322245722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 0x7fe0a41587d0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40c3520 tx=0x7fe09c01c520 comp rx=0 tx=0).stop 2024-03-04T08:28:34.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 -- 172.21.15.113:0/1322245722 shutdown_connections 2024-03-04T08:28:34.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 --2- 172.21.15.113:0/1322245722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 0x7fe0a41587d0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:34.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 --2- 172.21.15.113:0/1322245722 >> v2:172.21.15.136:3300/0 conn(0x7fe0a40c8200 0x7fe0a4072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:34.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 --2- 172.21.15.113:0/1322245722 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0a41527f0 0x7fe0a4070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:34.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 -- 172.21.15.113:0/1322245722 >> 172.21.15.113:0/1322245722 conn(0x7fe0a406e820 msgr2=0x7fe0a406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:34.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 -- 172.21.15.113:0/1322245722 shutdown_connections 2024-03-04T08:28:34.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 -- 172.21.15.113:0/1322245722 wait complete. 2024-03-04T08:28:34.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 Processor -- start 2024-03-04T08:28:34.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.982+0000 7fe0aa400640 1 -- start start 2024-03-04T08:28:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0aa400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a40c8200 0x7fe0a4158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0aa400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0a41527f0 0x7fe0a4156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0aa400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 0x7fe0a41f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0aa400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a415bf40 con 0x7fe0a4152bd0 2024-03-04T08:28:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0aa400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a415bdc0 con 0x7fe0a40c8200 2024-03-04T08:28:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0aa400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a415c0c0 con 0x7fe0a41527f0 2024-03-04T08:28:34.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 0x7fe0a41f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 0x7fe0a41f14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:48482/0 (socket says 172.21.15.113:48482) 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a3e00640 1 -- 172.21.15.113:0/604739333 learned_addr learned my addr 172.21.15.113:0/604739333 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a2a00640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0a41527f0 0x7fe0a4156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a3400640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a40c8200 0x7fe0a4158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a3e00640 1 -- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0a41527f0 msgr2=0x7fe0a4156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a3e00640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0a41527f0 0x7fe0a4156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a3e00640 1 -- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a40c8200 msgr2=0x7fe0a4158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a3e00640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a40c8200 0x7fe0a4158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a3e00640 1 -- 172.21.15.113:0/604739333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a4157fa0 con 0x7fe0a4152bd0 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0a3e00640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 0x7fe0a41f14b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe09c01cdc0 tx=0x7fe09c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe09b400640 1 -- 172.21.15.113:0/604739333 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0bc070 con 0x7fe0a4152bd0 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe09b400640 1 -- 172.21.15.113:0/604739333 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09c0ab5e0 con 0x7fe0a4152bd0 2024-03-04T08:28:34.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe09b400640 1 -- 172.21.15.113:0/604739333 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0b1070 con 0x7fe0a4152bd0 2024-03-04T08:28:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0aa400640 1 -- 172.21.15.113:0/604739333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a40b1150 con 0x7fe0a4152bd0 2024-03-04T08:28:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0aa400640 1 -- 172.21.15.113:0/604739333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a40b1710 con 0x7fe0a4152bd0 2024-03-04T08:28:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe0aa400640 1 -- 172.21.15.113:0/604739333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a4075c00 con 0x7fe0a4152bd0 2024-03-04T08:28:34.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.986+0000 7fe09b400640 1 -- 172.21.15.113:0/604739333 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe09c0bf070 con 0x7fe0a4152bd0 2024-03-04T08:28:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.990+0000 7fe09b400640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe084066f60 0x7fe084069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.990+0000 7fe09b400640 1 -- 172.21.15.113:0/604739333 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fe09c11f6f0 con 0x7fe0a4152bd0 2024-03-04T08:28:34.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.990+0000 7fe0a3400640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe084066f60 0x7fe084069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:34.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.990+0000 7fe0a3400640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe084066f60 0x7fe084069420 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe0940074a0 tx=0x7fe09400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:34.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:34.994+0000 7fe09b400640 1 -- 172.21.15.113:0/604739333 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe09c016110 con 0x7fe0a4152bd0 2024-03-04T08:28:35.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.178+0000 7fe0aa400640 1 -- 172.21.15.113:0/604739333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0a40b1950 con 0x7fe0a4152bd0 2024-03-04T08:28:35.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.178+0000 7fe09b400640 1 -- 172.21.15.113:0/604739333 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fe09c0efb90 con 0x7fe0a4152bd0 2024-03-04T08:28:35.177 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:35.177 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:35.177 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:35.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 -- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe084066f60 msgr2=0x7fe084069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:35.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe084066f60 0x7fe084069420 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe0940074a0 tx=0x7fe09400b040 comp rx=0 tx=0).stop 2024-03-04T08:28:35.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 -- 172.21.15.113:0/604739333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 msgr2=0x7fe0a41f14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:35.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 0x7fe0a41f14b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe09c01cdc0 tx=0x7fe09c0192c0 comp rx=0 tx=0).stop 2024-03-04T08:28:35.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 -- 172.21.15.113:0/604739333 shutdown_connections 2024-03-04T08:28:35.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe084066f60 0x7fe084069420 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:35.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0a4152bd0 0x7fe0a41f14b0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:35.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe0a41527f0 0x7fe0a4156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:35.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 --2- 172.21.15.113:0/604739333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe0a40c8200 0x7fe0a4158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:35.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 -- 172.21.15.113:0/604739333 >> 172.21.15.113:0/604739333 conn(0x7fe0a406e820 msgr2=0x7fe0a40c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:35.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 -- 172.21.15.113:0/604739333 shutdown_connections 2024-03-04T08:28:35.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:35.182+0000 7fe0aa400640 1 -- 172.21.15.113:0/604739333 wait complete. 2024-03-04T08:28:35.198 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:37.199 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:37.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.346+0000 7fa43be00640 1 Processor -- start 2024-03-04T08:28:37.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.346+0000 7fa43be00640 1 -- start start 2024-03-04T08:28:37.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.346+0000 7fa43be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa43c06d260 0x7fa43c06ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:37.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.346+0000 7fa43be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa43c06d640 0x7fa43c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:37.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.346+0000 7fa43be00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa43c156fc0 con 0x7fa43c0c0ed0 2024-03-04T08:28:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.346+0000 7fa43be00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c06f2d0 con 0x7fa43c06d260 2024-03-04T08:28:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.346+0000 7fa43be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c157280 con 0x7fa43c06d640 2024-03-04T08:28:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.346+0000 7fa43aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa43c06d260 0x7fa43c06ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.346+0000 7fa43b400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa43c0c0ed0 0x7fa43c1508d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37498/0 (socket says 172.21.15.113:37498) 2024-03-04T08:28:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.346+0000 7fa43b400640 1 -- 172.21.15.113:0/2899337382 learned_addr learned my addr 172.21.15.113:0/2899337382 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa43a000640 1 --2- 172.21.15.113:0/2899337382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa43c06d640 0x7fa43c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa43aa00640 1 -- 172.21.15.113:0/2899337382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa43c06d640 msgr2=0x7fa43c15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa43aa00640 1 --2- 172.21.15.113:0/2899337382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa43c06d640 0x7fa43c15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa43aa00640 1 -- 172.21.15.113:0/2899337382 >> v1:172.21.15.113:6789/0 conn(0x7fa43c0c0ed0 legacy=0x7fa43c1508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:37.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa43aa00640 1 -- 172.21.15.113:0/2899337382 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c157560 con 0x7fa43c06d260 2024-03-04T08:28:37.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa43aa00640 1 --2- 172.21.15.113:0/2899337382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 0x7fa43c06ed90 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fa4240097c0 tx=0x7fa42401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa7148465cc8064 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:37.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa439600640 1 -- 172.21.15.113:0/2899337382 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4240b5070 con 0x7fa43c06d260 2024-03-04T08:28:37.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa439600640 1 -- 172.21.15.113:0/2899337382 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4240a8990 con 0x7fa43c06d260 2024-03-04T08:28:37.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa439600640 1 -- 172.21.15.113:0/2899337382 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4240a8d60 con 0x7fa43c06d260 2024-03-04T08:28:37.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa43be00640 1 -- 172.21.15.113:0/2899337382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 msgr2=0x7fa43c06ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:37.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.350+0000 7fa43be00640 1 --2- 172.21.15.113:0/2899337382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 0x7fa43c06ed90 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fa4240097c0 tx=0x7fa42401ec90 comp rx=0 tx=0).stop 2024-03-04T08:28:37.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 -- 172.21.15.113:0/2899337382 shutdown_connections 2024-03-04T08:28:37.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 --2- 172.21.15.113:0/2899337382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa43c06d640 0x7fa43c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:37.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 --2- 172.21.15.113:0/2899337382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 0x7fa43c06ed90 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:37.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 -- 172.21.15.113:0/2899337382 >> 172.21.15.113:0/2899337382 conn(0x7fa43c06e2a0 msgr2=0x7fa43c06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:37.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 -- 172.21.15.113:0/2899337382 shutdown_connections 2024-03-04T08:28:37.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 -- 172.21.15.113:0/2899337382 wait complete. 2024-03-04T08:28:37.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 Processor -- start 2024-03-04T08:28:37.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 -- start start 2024-03-04T08:28:37.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 0x7fa43c1f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:37.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa43c06d640 0x7fa43c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:37.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c0c0ed0 0x7fa43c0c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c06c150 con 0x7fa43c06d260 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c157b20 con 0x7fa43c0c0ed0 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c06c2d0 con 0x7fa43c06d640 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 0x7fa43c1f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43aa00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 0x7fa43c1f1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53332/0 (socket says 172.21.15.113:53332) 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43aa00640 1 -- 172.21.15.113:0/835321480 learned_addr learned my addr 172.21.15.113:0/835321480 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43a000640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa43c06d640 0x7fa43c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43b400640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c0c0ed0 0x7fa43c0c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43aa00640 1 -- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa43c06d640 msgr2=0x7fa43c1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43aa00640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa43c06d640 0x7fa43c1597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43aa00640 1 -- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c0c0ed0 msgr2=0x7fa43c0c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43aa00640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c0c0ed0 0x7fa43c0c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43aa00640 1 -- 172.21.15.113:0/835321480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c0c6500 con 0x7fa43c06d260 2024-03-04T08:28:37.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43b400640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c0c0ed0 0x7fa43c0c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:28:37.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa43aa00640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 0x7fa43c1f1470 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fa4240b3040 tx=0x7fa42401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:37.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa433400640 1 -- 172.21.15.113:0/835321480 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4240bf070 con 0x7fa43c06d260 2024-03-04T08:28:37.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa433400640 1 -- 172.21.15.113:0/835321480 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4240b0820 con 0x7fa43c06d260 2024-03-04T08:28:37.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.354+0000 7fa433400640 1 -- 172.21.15.113:0/835321480 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4240b5040 con 0x7fa43c06d260 2024-03-04T08:28:37.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.358+0000 7fa43be00640 1 -- 172.21.15.113:0/835321480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c1f19b0 con 0x7fa43c06d260 2024-03-04T08:28:37.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.358+0000 7fa43be00640 1 -- 172.21.15.113:0/835321480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c1f0c70 con 0x7fa43c06d260 2024-03-04T08:28:37.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.358+0000 7fa43be00640 1 -- 172.21.15.113:0/835321480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c0c3820 con 0x7fa43c06d260 2024-03-04T08:28:37.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.358+0000 7fa433400640 1 -- 172.21.15.113:0/835321480 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa4240b0c10 con 0x7fa43c06d260 2024-03-04T08:28:37.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.358+0000 7fa433400640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa418066f60 0x7fa418069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:37.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.358+0000 7fa433400640 1 -- 172.21.15.113:0/835321480 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fa424122520 con 0x7fa43c06d260 2024-03-04T08:28:37.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.358+0000 7fa43a000640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa418066f60 0x7fa418069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:37.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.358+0000 7fa43a000640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa418066f60 0x7fa418069420 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa43c06e9d0 tx=0x7fa42c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:37.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.362+0000 7fa433400640 1 -- 172.21.15.113:0/835321480 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa43c0c3820 con 0x7fa43c06d260 2024-03-04T08:28:37.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.550+0000 7fa43be00640 1 -- 172.21.15.113:0/835321480 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa43c1f0a60 con 0x7fa43c06d260 2024-03-04T08:28:37.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa433400640 1 -- 172.21.15.113:0/835321480 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fa4240187f0 con 0x7fa43c06d260 2024-03-04T08:28:37.551 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:37.551 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:37.551 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 -- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa418066f60 msgr2=0x7fa418069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa418066f60 0x7fa418069420 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa43c06e9d0 tx=0x7fa42c008040 comp rx=0 tx=0).stop 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 -- 172.21.15.113:0/835321480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 msgr2=0x7fa43c1f1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 0x7fa43c1f1470 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fa4240b3040 tx=0x7fa42401ba30 comp rx=0 tx=0).stop 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 -- 172.21.15.113:0/835321480 shutdown_connections 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa43c0c0ed0 0x7fa43c0c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa43c06d640 0x7fa43c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa418066f60 0x7fa418069420 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 --2- 172.21.15.113:0/835321480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa43c06d260 0x7fa43c1f1470 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 -- 172.21.15.113:0/835321480 >> 172.21.15.113:0/835321480 conn(0x7fa43c06e2a0 msgr2=0x7fa43c0c2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 -- 172.21.15.113:0/835321480 shutdown_connections 2024-03-04T08:28:37.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:37.554+0000 7fa43be00640 1 -- 172.21.15.113:0/835321480 wait complete. 2024-03-04T08:28:37.571 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:28:37.572 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:28:39.573 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:39.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc7e00640 1 Processor -- start 2024-03-04T08:28:39.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc7e00640 1 -- start start 2024-03-04T08:28:39.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc806d640 0x7f1fc815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc8156fc0 con 0x7f1fc806d260 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc8157280 con 0x7f1fc80c0ed0 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc8157560 con 0x7f1fc806d640 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1fc806d260 0x7f1fc806ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37502/0 (socket says 172.21.15.113:37502) 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6a00640 1 -- 172.21.15.113:0/4288529500 learned_addr learned my addr 172.21.15.113:0/4288529500 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fbe000640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc806d640 0x7f1fc815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259520589 0 0) 0x7f1fc8156fc0 con 0x7f1fc806d260 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fac003530 con 0x7f1fc806d260 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1442707769 0 0) 0x7f1fac003530 con 0x7f1fc806d260 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc806d640 msgr2=0x7f1fc815a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc806d640 0x7f1fc815a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 >> v1:172.21.15.136:6789/0 conn(0x7f1fc80c0ed0 legacy=0x7f1fc81508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:39.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc8157840 con 0x7f1fc806d260 2024-03-04T08:28:39.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f1fb8001680 con 0x7f1fc806d260 2024-03-04T08:28:39.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 >> v1:172.21.15.113:6789/0 conn(0x7f1fc806d260 legacy=0x7f1fc806ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:39.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fac0039f0 0x7f1fac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:39.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fac0043a0 0x7f1fac004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc806d640 0x7f1fac0050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc8157560 con 0x7f1fac0039f0 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac00cae0 con 0x7f1fac0043a0 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fac00ba50 con 0x7f1fc806d640 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.722+0000 7f1fc6a00640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fac0039f0 0x7f1fac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fbe000640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fac0043a0 0x7f1fac004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc7400640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc806d640 0x7f1fac0050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc6a00640 1 -- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc806d640 msgr2=0x7f1fac0050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc6a00640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc806d640 0x7f1fac0050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc6a00640 1 -- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fac0043a0 msgr2=0x7f1fac004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc6a00640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fac0043a0 0x7f1fac004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:39.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc6a00640 1 -- 172.21.15.113:0/4288529500 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac003530 con 0x7f1fac0039f0 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc6a00640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fac0039f0 0x7f1fac003dd0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800c2c0 tx=0x7f1fb801cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dbf4eb47ec1a69 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8007d60 con 0x7f1fac0039f0 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb80a9070 con 0x7f1fac0039f0 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc6000640 1 -- 172.21.15.113:0/4288529500 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80a5e10 con 0x7f1fac0039f0 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc7e00640 1 -- 172.21.15.113:0/4288529500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fac0039f0 msgr2=0x7f1fac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/4288529500 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fac0039f0 0x7f1fac003dd0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800c2c0 tx=0x7f1fb801cd40 comp rx=0 tx=0).stop 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc7400640 1 -- 172.21.15.113:0/4288529500 reap_dead start 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc7e00640 1 -- 172.21.15.113:0/4288529500 shutdown_connections 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.726+0000 7f1fc7e00640 1 -- 172.21.15.113:0/4288529500 >> 172.21.15.113:0/4288529500 conn(0x7f1fc806e2a0 msgr2=0x7f1fc806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 -- 172.21.15.113:0/4288529500 shutdown_connections 2024-03-04T08:28:39.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 -- 172.21.15.113:0/4288529500 wait complete. 2024-03-04T08:28:39.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 Processor -- start 2024-03-04T08:28:39.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 -- start start 2024-03-04T08:28:39.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fac0039f0 0x7f1fc814b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:39.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc814cf80 0x7f1fc814be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:39.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc80af4a0 0x7f1fc80af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:39.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc806c2d0 con 0x7f1fc814cf80 2024-03-04T08:28:39.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc806c150 con 0x7f1fac0039f0 2024-03-04T08:28:39.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc8150fe0 con 0x7f1fc80af4a0 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbe000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc814cf80 0x7f1fc814be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbe000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc814cf80 0x7f1fc814be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53340/0 (socket says 172.21.15.113:53340) 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbe000640 1 -- 172.21.15.113:0/28891091 learned_addr learned my addr 172.21.15.113:0/28891091 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7400640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc80af4a0 0x7f1fc80af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc6a00640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fac0039f0 0x7f1fc814b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbe000640 1 -- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc80af4a0 msgr2=0x7f1fc80af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbe000640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc80af4a0 0x7f1fc80af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbe000640 1 -- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fac0039f0 msgr2=0x7f1fc814b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbe000640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fac0039f0 0x7f1fc814b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbe000640 1 -- 172.21.15.113:0/28891091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc814b410 con 0x7f1fc814cf80 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbe000640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc814cf80 0x7f1fc814be00 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00115e0 tx=0x7f1fb0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbf400640 1 -- 172.21.15.113:0/28891091 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb000f8f0 con 0x7f1fc814cf80 2024-03-04T08:28:39.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbf400640 1 -- 172.21.15.113:0/28891091 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb000fa90 con 0x7f1fc814cf80 2024-03-04T08:28:39.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fbf400640 1 -- 172.21.15.113:0/28891091 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb000fe40 con 0x7f1fc814cf80 2024-03-04T08:28:39.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.730+0000 7f1fc7e00640 1 -- 172.21.15.113:0/28891091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc814e3b0 con 0x7f1fc814cf80 2024-03-04T08:28:39.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.734+0000 7f1fc7e00640 1 -- 172.21.15.113:0/28891091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc80afe70 con 0x7f1fc814cf80 2024-03-04T08:28:39.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.734+0000 7f1fc7e00640 1 -- 172.21.15.113:0/28891091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc8075a80 con 0x7f1fc814cf80 2024-03-04T08:28:39.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.734+0000 7f1fbf400640 1 -- 172.21.15.113:0/28891091 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fb0015460 con 0x7f1fc814cf80 2024-03-04T08:28:39.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.734+0000 7f1fbf400640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 0x7f1fa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:39.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.734+0000 7f1fbf400640 1 -- 172.21.15.113:0/28891091 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f1fb0086450 con 0x7f1fc814cf80 2024-03-04T08:28:39.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.734+0000 7f1fc6a00640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 0x7f1fa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:39.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.738+0000 7f1fc6a00640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 0x7f1fa4069420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800c2c0 tx=0x7f1fb80b1040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:39.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.738+0000 7f1fbf400640 1 -- 172.21.15.113:0/28891091 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1fb0019030 con 0x7f1fc814cf80 2024-03-04T08:28:39.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 -- 172.21.15.113:0/28891091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fc814abc0 con 0x7f1fc814cf80 2024-03-04T08:28:39.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fbf400640 1 -- 172.21.15.113:0/28891091 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f1fb0056920 con 0x7f1fc814cf80 2024-03-04T08:28:39.923 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:39.923 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:39.924 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 -- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 msgr2=0x7f1fa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 0x7f1fa4069420 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800c2c0 tx=0x7f1fb80b1040 comp rx=0 tx=0).stop 2024-03-04T08:28:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 -- 172.21.15.113:0/28891091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc814cf80 msgr2=0x7f1fc814be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc814cf80 0x7f1fc814be00 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00115e0 tx=0x7f1fb0011aa0 comp rx=0 tx=0).stop 2024-03-04T08:28:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 -- 172.21.15.113:0/28891091 shutdown_connections 2024-03-04T08:28:39.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1fc80af4a0 0x7f1fc80af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:39.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1fa4066f60 0x7f1fa4069420 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:39.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fc814cf80 0x7f1fc814be00 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:39.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 --2- 172.21.15.113:0/28891091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fac0039f0 0x7f1fc814b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:39.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 -- 172.21.15.113:0/28891091 >> 172.21.15.113:0/28891091 conn(0x7f1fc806e2a0 msgr2=0x7f1fc8146ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:39.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 -- 172.21.15.113:0/28891091 shutdown_connections 2024-03-04T08:28:39.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:39.926+0000 7f1fc7e00640 1 -- 172.21.15.113:0/28891091 wait complete. 2024-03-04T08:28:39.944 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:41.945 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:42.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4daa200640 1 Processor -- start 2024-03-04T08:28:42.091 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4daa200640 1 -- start start 2024-03-04T08:28:42.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4daa200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4da414e4c0 0x7f4da4072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:42.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4daa200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4da40c22e0 0x7f4da40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:42.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4daa200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da40c27f0 con 0x7f4da40c2d80 2024-03-04T08:28:42.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4daa200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da415ad60 con 0x7f4da40c22e0 2024-03-04T08:28:42.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4daa200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40c2a90 con 0x7f4da414e4c0 2024-03-04T08:28:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4d9aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4da40c22e0 0x7f4da40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4d9aa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4da40c22e0 0x7f4da40731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53356/0 (socket says 172.21.15.113:53356) 2024-03-04T08:28:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4da3e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f4da40c2d80 0x7f4da415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:43738/0 (socket says 172.21.15.113:43738) 2024-03-04T08:28:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4d9aa00640 1 -- 172.21.15.113:0/428495134 learned_addr learned my addr 172.21.15.113:0/428495134 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4da3400640 1 --2- 172.21.15.113:0/428495134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4da414e4c0 0x7f4da4072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4d9aa00640 1 -- 172.21.15.113:0/428495134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4da414e4c0 msgr2=0x7f4da4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4d9aa00640 1 --2- 172.21.15.113:0/428495134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4da414e4c0 0x7f4da4072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4d9aa00640 1 -- 172.21.15.113:0/428495134 >> v1:172.21.15.136:6789/0 conn(0x7f4da40c2d80 legacy=0x7f4da415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4d9aa00640 1 -- 172.21.15.113:0/428495134 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da4070c60 con 0x7f4da40c22e0 2024-03-04T08:28:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.094+0000 7f4da2a00640 1 -- 172.21.15.113:0/428495134 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768740332 0 0) 0x7f4da40c27f0 con 0x7f4da40c2d80 2024-03-04T08:28:42.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4d9aa00640 1 --2- 172.21.15.113:0/428495134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da40c22e0 0x7f4da40731c0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009870 tx=0x7f4d8c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c46a17fb125ccb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:42.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4da2a00640 1 -- 172.21.15.113:0/428495134 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0b5070 con 0x7f4da40c22e0 2024-03-04T08:28:42.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4da2a00640 1 -- 172.21.15.113:0/428495134 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c0a8a50 con 0x7f4da40c22e0 2024-03-04T08:28:42.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4da2a00640 1 -- 172.21.15.113:0/428495134 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0a8e20 con 0x7f4da40c22e0 2024-03-04T08:28:42.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4daa200640 1 -- 172.21.15.113:0/428495134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da40c22e0 msgr2=0x7f4da40731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:42.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4daa200640 1 --2- 172.21.15.113:0/428495134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da40c22e0 0x7f4da40731c0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009870 tx=0x7f4d8c01ed50 comp rx=0 tx=0).stop 2024-03-04T08:28:42.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4daa200640 1 -- 172.21.15.113:0/428495134 shutdown_connections 2024-03-04T08:28:42.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4daa200640 1 --2- 172.21.15.113:0/428495134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da40c22e0 0x7f4da40731c0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:42.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4daa200640 1 --2- 172.21.15.113:0/428495134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4da414e4c0 0x7f4da4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:42.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4daa200640 1 -- 172.21.15.113:0/428495134 >> 172.21.15.113:0/428495134 conn(0x7f4da406e820 msgr2=0x7f4da406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:42.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4daa200640 1 -- 172.21.15.113:0/428495134 shutdown_connections 2024-03-04T08:28:42.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.098+0000 7f4daa200640 1 -- 172.21.15.113:0/428495134 wait complete. 2024-03-04T08:28:42.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 Processor -- start 2024-03-04T08:28:42.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 -- start start 2024-03-04T08:28:42.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4da40c22e0 0x7f4da40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:42.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4da40c2d80 0x7f4da41f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:42.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da414e4c0 0x7f4da41548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:42.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da4071500 con 0x7f4da414e4c0 2024-03-04T08:28:42.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da4071220 con 0x7f4da40c2d80 2024-03-04T08:28:42.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da414e8a0 con 0x7f4da40c22e0 2024-03-04T08:28:42.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4da3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da414e4c0 0x7f4da41548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:42.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4da40c2d80 0x7f4da41f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:42.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4da40c2d80 0x7f4da41f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:44882/0 (socket says 172.21.15.113:44882) 2024-03-04T08:28:42.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9aa00640 1 -- 172.21.15.113:0/15134019 learned_addr learned my addr 172.21.15.113:0/15134019 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:42.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4da3400640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4da40c22e0 0x7f4da40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:42.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9aa00640 1 -- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4da40c22e0 msgr2=0x7f4da40af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:42.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9aa00640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4da40c22e0 0x7f4da40af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:42.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9aa00640 1 -- 172.21.15.113:0/15134019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da414e4c0 msgr2=0x7f4da41548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:42.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9aa00640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da414e4c0 0x7f4da41548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:42.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9aa00640 1 -- 172.21.15.113:0/15134019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da40b1440 con 0x7f4da40c2d80 2024-03-04T08:28:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9aa00640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4da40c2d80 0x7f4da41f4010 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0b3040 tx=0x7f4d8c01baf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9be00640 1 -- 172.21.15.113:0/15134019 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0c0070 con 0x7f4da40c2d80 2024-03-04T08:28:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9be00640 1 -- 172.21.15.113:0/15134019 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c0b0e20 con 0x7f4da40c2d80 2024-03-04T08:28:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4d9be00640 1 -- 172.21.15.113:0/15134019 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0b5070 con 0x7f4da40c2d80 2024-03-04T08:28:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 -- 172.21.15.113:0/15134019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da40b16e0 con 0x7f4da40c2d80 2024-03-04T08:28:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 -- 172.21.15.113:0/15134019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da40c9790 con 0x7f4da40c2d80 2024-03-04T08:28:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.102+0000 7f4daa200640 1 -- 172.21.15.113:0/15134019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da4075c00 con 0x7f4da40c2d80 2024-03-04T08:28:42.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.106+0000 7f4d9be00640 1 -- 172.21.15.113:0/15134019 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d8c0ba3e0 con 0x7f4da40c2d80 2024-03-04T08:28:42.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.106+0000 7f4d9be00640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d80066f60 0x7f4d80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:42.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.106+0000 7f4d9be00640 1 -- 172.21.15.113:0/15134019 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f4d8c123590 con 0x7f4da40c2d80 2024-03-04T08:28:42.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.106+0000 7f4da3400640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d80066f60 0x7f4d80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:42.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.106+0000 7f4da3400640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d80066f60 0x7f4d80069420 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009ac0 tx=0x7f4d94009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:42.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.118+0000 7f4d9be00640 1 -- 172.21.15.113:0/15134019 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4d8c0188b0 con 0x7f4da40c2d80 2024-03-04T08:28:42.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 -- 172.21.15.113:0/15134019 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4da4071ad0 con 0x7f4da40c2d80 2024-03-04T08:28:42.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4d9be00640 1 -- 172.21.15.113:0/15134019 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f4d8c0f34a0 con 0x7f4da40c2d80 2024-03-04T08:28:42.304 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:42.304 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:42.305 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:42.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 -- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d80066f60 msgr2=0x7f4d80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:42.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d80066f60 0x7f4d80069420 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009ac0 tx=0x7f4d94009190 comp rx=0 tx=0).stop 2024-03-04T08:28:42.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 -- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4da40c2d80 msgr2=0x7f4da41f4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:42.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4da40c2d80 0x7f4da41f4010 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0b3040 tx=0x7f4d8c01baf0 comp rx=0 tx=0).stop 2024-03-04T08:28:42.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 -- 172.21.15.113:0/15134019 shutdown_connections 2024-03-04T08:28:42.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4d80066f60 0x7f4d80069420 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:42.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4da414e4c0 0x7f4da41548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:42.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4da40c2d80 0x7f4da41f4010 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:42.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 --2- 172.21.15.113:0/15134019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4da40c22e0 0x7f4da40af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:42.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 -- 172.21.15.113:0/15134019 >> 172.21.15.113:0/15134019 conn(0x7f4da406e820 msgr2=0x7f4da40c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:42.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 -- 172.21.15.113:0/15134019 shutdown_connections 2024-03-04T08:28:42.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:42.306+0000 7f4daa200640 1 -- 172.21.15.113:0/15134019 wait complete. 2024-03-04T08:28:42.325 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:44.326 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:44.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.474+0000 7f23eea00640 1 Processor -- start 2024-03-04T08:28:44.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.474+0000 7f23eea00640 1 -- start start 2024-03-04T08:28:44.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.474+0000 7f23eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e814e4c0 0x7f23e8072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:44.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.474+0000 7f23eea00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e80c27f0 con 0x7f23e815b900 2024-03-04T08:28:44.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.474+0000 7f23eea00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e80c2a90 con 0x7f23e80c22e0 2024-03-04T08:28:44.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.474+0000 7f23eea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e815ad60 con 0x7f23e814e4c0 2024-03-04T08:28:44.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.474+0000 7f23eca00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f23e815b900 0x7f23e81620f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37518/0 (socket says 172.21.15.113:37518) 2024-03-04T08:28:44.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.474+0000 7f23eca00640 1 -- 172.21.15.113:0/1151406096 learned_addr learned my addr 172.21.15.113:0/1151406096 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:44.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.478+0000 7f23e7e00640 1 --2- 172.21.15.113:0/1151406096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e814e4c0 0x7f23e8072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:44.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.478+0000 7f23e7e00640 1 -- 172.21.15.113:0/1151406096 >> v1:172.21.15.136:6789/0 conn(0x7f23e80c22e0 legacy=0x7f23e815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:44.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.478+0000 7f23e7e00640 1 -- 172.21.15.113:0/1151406096 >> v1:172.21.15.113:6789/0 conn(0x7f23e815b900 legacy=0x7f23e81620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:44.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.478+0000 7f23e7e00640 1 -- 172.21.15.113:0/1151406096 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e8070c60 con 0x7f23e814e4c0 2024-03-04T08:28:44.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.478+0000 7f23e7e00640 1 --2- 172.21.15.113:0/1151406096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e814e4c0 0x7f23e8072c80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009df0 tx=0x7f23d801f180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa66c9b3db10f91 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:44.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23e7400640 1 -- 172.21.15.113:0/1151406096 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80b4070 con 0x7f23e814e4c0 2024-03-04T08:28:44.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23e7400640 1 -- 172.21.15.113:0/1151406096 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d80af600 con 0x7f23e814e4c0 2024-03-04T08:28:44.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23e7400640 1 -- 172.21.15.113:0/1151406096 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80af9d0 con 0x7f23e814e4c0 2024-03-04T08:28:44.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23eea00640 1 -- 172.21.15.113:0/1151406096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e814e4c0 msgr2=0x7f23e8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:44.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23eea00640 1 --2- 172.21.15.113:0/1151406096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e814e4c0 0x7f23e8072c80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009df0 tx=0x7f23d801f180 comp rx=0 tx=0).stop 2024-03-04T08:28:44.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23eea00640 1 -- 172.21.15.113:0/1151406096 shutdown_connections 2024-03-04T08:28:44.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23eea00640 1 --2- 172.21.15.113:0/1151406096 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e814e4c0 0x7f23e8072c80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:44.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23eea00640 1 -- 172.21.15.113:0/1151406096 >> 172.21.15.113:0/1151406096 conn(0x7f23e806e820 msgr2=0x7f23e806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:44.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23eea00640 1 -- 172.21.15.113:0/1151406096 shutdown_connections 2024-03-04T08:28:44.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23eea00640 1 -- 172.21.15.113:0/1151406096 wait complete. 2024-03-04T08:28:44.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23eea00640 1 Processor -- start 2024-03-04T08:28:44.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.482+0000 7f23eea00640 1 -- start start 2024-03-04T08:28:44.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eea00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80c22e0 0x7f23e81f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:44.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e814e4c0 0x7f23e806b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:44.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e815b900 0x7f23e8146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eea00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8071500 con 0x7f23e80c22e0 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e8071220 con 0x7f23e814e4c0 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eea00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e814e8a0 con 0x7f23e815b900 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80c22e0 0x7f23e81f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e7e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80c22e0 0x7f23e81f0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53366/0 (socket says 172.21.15.113:53366) 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e7e00640 1 -- 172.21.15.113:0/1313773436 learned_addr learned my addr 172.21.15.113:0/1313773436 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23df400640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e814e4c0 0x7f23e806b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eca00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e815b900 0x7f23e8146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e7e00640 1 -- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e815b900 msgr2=0x7f23e8146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e7e00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e815b900 0x7f23e8146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e7e00640 1 -- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e814e4c0 msgr2=0x7f23e806b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e7e00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e814e4c0 0x7f23e806b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e7e00640 1 -- 172.21.15.113:0/1313773436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e81f0f90 con 0x7f23e80c22e0 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eca00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e815b900 0x7f23e8146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:44.485 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e7e00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80c22e0 0x7f23e81f0a50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009df0 tx=0x7f23d801bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:44.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e4c00640 1 -- 172.21.15.113:0/1313773436 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80b4040 con 0x7f23e80c22e0 2024-03-04T08:28:44.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e4c00640 1 -- 172.21.15.113:0/1313773436 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d80c1070 con 0x7f23e80c22e0 2024-03-04T08:28:44.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e4c00640 1 -- 172.21.15.113:0/1313773436 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80ba670 con 0x7f23e80c22e0 2024-03-04T08:28:44.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eea00640 1 -- 172.21.15.113:0/1313773436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e81e64b0 con 0x7f23e80c22e0 2024-03-04T08:28:44.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eea00640 1 -- 172.21.15.113:0/1313773436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e81e5cb0 con 0x7f23e80c22e0 2024-03-04T08:28:44.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23eea00640 1 -- 172.21.15.113:0/1313773436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e8075c00 con 0x7f23e80c22e0 2024-03-04T08:28:44.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.486+0000 7f23e4c00640 1 -- 172.21.15.113:0/1313773436 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23d80ba810 con 0x7f23e80c22e0 2024-03-04T08:28:44.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.490+0000 7f23e4c00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23c4066f90 0x7f23c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:44.487 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.490+0000 7f23e4c00640 1 -- 172.21.15.113:0/1313773436 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f23d8124500 con 0x7f23e80c22e0 2024-03-04T08:28:44.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.490+0000 7f23df400640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23c4066f90 0x7f23c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:44.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.490+0000 7f23df400640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23c4066f90 0x7f23c4069450 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f23e81e5500 tx=0x7f23d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:44.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.494+0000 7f23e4c00640 1 -- 172.21.15.113:0/1313773436 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f23d8018ce0 con 0x7f23e80c22e0 2024-03-04T08:28:44.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.686+0000 7f23eea00640 1 -- 172.21.15.113:0/1313773436 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23e80c6f50 con 0x7f23e80c22e0 2024-03-04T08:28:44.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.686+0000 7f23e4c00640 1 -- 172.21.15.113:0/1313773436 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f23d80f49d0 con 0x7f23e80c22e0 2024-03-04T08:28:44.686 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:44.686 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:44.686 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:44.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 -- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23c4066f90 msgr2=0x7f23c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:44.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23c4066f90 0x7f23c4069450 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f23e81e5500 tx=0x7f23d0009040 comp rx=0 tx=0).stop 2024-03-04T08:28:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 -- 172.21.15.113:0/1313773436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80c22e0 msgr2=0x7f23e81f0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80c22e0 0x7f23e81f0a50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009df0 tx=0x7f23d801bf20 comp rx=0 tx=0).stop 2024-03-04T08:28:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 -- 172.21.15.113:0/1313773436 shutdown_connections 2024-03-04T08:28:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f23e815b900 0x7f23e8146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e814e4c0 0x7f23e806b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23c4066f90 0x7f23c4069450 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 --2- 172.21.15.113:0/1313773436 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e80c22e0 0x7f23e81f0a50 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 -- 172.21.15.113:0/1313773436 >> 172.21.15.113:0/1313773436 conn(0x7f23e806e820 msgr2=0x7f23e814b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 -- 172.21.15.113:0/1313773436 shutdown_connections 2024-03-04T08:28:44.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:44.690+0000 7f23eea00640 1 -- 172.21.15.113:0/1313773436 wait complete. 2024-03-04T08:28:44.708 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:46.192 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:28:46.196 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:28:46.708 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:46.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce72c00640 1 Processor -- start 2024-03-04T08:28:46.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce72c00640 1 -- start start 2024-03-04T08:28:46.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce72c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c0c8b00 0x7fce6c1641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:46.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce72c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6c0c8f50 con 0x7fce6c152b20 2024-03-04T08:28:46.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce72c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6c15cc20 con 0x7fce6c070010 2024-03-04T08:28:46.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce72c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0c5320 con 0x7fce6c0c8b00 2024-03-04T08:28:46.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce6be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fce6c152b20 0x7fce6c15be00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:37520/0 (socket says 172.21.15.113:37520) 2024-03-04T08:28:46.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce6be00640 1 -- 172.21.15.113:0/397541882 learned_addr learned my addr 172.21.15.113:0/397541882 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:46.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce70c00640 1 --2- 172.21.15.113:0/397541882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c0c8b00 0x7fce6c1641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:46.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce6aa00640 1 -- 172.21.15.113:0/397541882 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409410444 0 0) 0x7fce5c001120 con 0x7fce6c152b20 2024-03-04T08:28:46.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce6aa00640 1 -- 172.21.15.113:0/397541882 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce50003530 con 0x7fce6c152b20 2024-03-04T08:28:46.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce6aa00640 1 -- 172.21.15.113:0/397541882 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704752242 0 0) 0x7fce6c15cc20 con 0x7fce6c070010 2024-03-04T08:28:46.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce6aa00640 1 -- 172.21.15.113:0/397541882 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce5c001120 con 0x7fce6c070010 2024-03-04T08:28:46.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce6aa00640 1 -- 172.21.15.113:0/397541882 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1520784590 0 0) 0x7fce50003530 con 0x7fce6c152b20 2024-03-04T08:28:46.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce70c00640 1 -- 172.21.15.113:0/397541882 >> v1:172.21.15.136:6789/0 conn(0x7fce6c070010 legacy=0x7fce6c15c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:46.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce70c00640 1 -- 172.21.15.113:0/397541882 >> v1:172.21.15.113:6789/0 conn(0x7fce6c152b20 legacy=0x7fce6c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:46.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.858+0000 7fce70c00640 1 -- 172.21.15.113:0/397541882 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c152f00 con 0x7fce6c0c8b00 2024-03-04T08:28:46.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce70c00640 1 --2- 172.21.15.113:0/397541882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c0c8b00 0x7fce6c1641a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fce64009dc0 tx=0x7fce6401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879d4ee3b2d98ee9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:46.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce6aa00640 1 -- 172.21.15.113:0/397541882 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c001870 con 0x7fce6c0c8b00 2024-03-04T08:28:46.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce6aa00640 1 -- 172.21.15.113:0/397541882 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce640a6500 con 0x7fce6c0c8b00 2024-03-04T08:28:46.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce6aa00640 1 -- 172.21.15.113:0/397541882 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640a68b0 con 0x7fce6c0c8b00 2024-03-04T08:28:46.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce72c00640 1 -- 172.21.15.113:0/397541882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c0c8b00 msgr2=0x7fce6c1641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:46.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce72c00640 1 --2- 172.21.15.113:0/397541882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c0c8b00 0x7fce6c1641a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fce64009dc0 tx=0x7fce6401c8b0 comp rx=0 tx=0).stop 2024-03-04T08:28:46.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce72c00640 1 -- 172.21.15.113:0/397541882 shutdown_connections 2024-03-04T08:28:46.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce72c00640 1 --2- 172.21.15.113:0/397541882 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c0c8b00 0x7fce6c1641a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:46.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce72c00640 1 -- 172.21.15.113:0/397541882 >> 172.21.15.113:0/397541882 conn(0x7fce6c0709a0 msgr2=0x7fce6c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:46.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce72c00640 1 -- 172.21.15.113:0/397541882 shutdown_connections 2024-03-04T08:28:46.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.862+0000 7fce72c00640 1 -- 172.21.15.113:0/397541882 wait complete. 2024-03-04T08:28:46.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce72c00640 1 Processor -- start 2024-03-04T08:28:46.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce72c00640 1 -- start start 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce72c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c070010 0x7fce6c1f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce72c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce6c0c8b00 0x7fce6c1f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce72c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce6c152b20 0x7fce6c06cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce72c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0c3810 con 0x7fce6c0c8b00 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce72c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0c92f0 con 0x7fce6c152b20 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce72c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c158810 con 0x7fce6c070010 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce6c0c8b00 0x7fce6c1f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce6c0c8b00 0x7fce6c1f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53370/0 (socket says 172.21.15.113:53370) 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6b400640 1 -- 172.21.15.113:0/2793871826 learned_addr learned my addr 172.21.15.113:0/2793871826 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6be00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c070010 0x7fce6c1f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce70c00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce6c152b20 0x7fce6c06cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6b400640 1 -- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c070010 msgr2=0x7fce6c1f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:46.866 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6b400640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c070010 0x7fce6c1f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:46.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6b400640 1 -- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce6c152b20 msgr2=0x7fce6c06cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:46.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6b400640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce6c152b20 0x7fce6c06cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:46.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6b400640 1 -- 172.21.15.113:0/2793871826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0cc8e0 con 0x7fce6c0c8b00 2024-03-04T08:28:46.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6be00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c070010 0x7fce6c1f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:28:46.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce6b400640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce6c0c8b00 0x7fce6c1f7240 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c430 tx=0x7fce5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:46.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce63e00640 1 -- 172.21.15.113:0/2793871826 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58017070 con 0x7fce6c0c8b00 2024-03-04T08:28:46.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce63e00640 1 -- 172.21.15.113:0/2793871826 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce580052f0 con 0x7fce6c0c8b00 2024-03-04T08:28:46.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce63e00640 1 -- 172.21.15.113:0/2793871826 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58003840 con 0x7fce6c0c8b00 2024-03-04T08:28:46.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce72c00640 1 -- 172.21.15.113:0/2793871826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c0cc320 con 0x7fce6c0c8b00 2024-03-04T08:28:46.867 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.866+0000 7fce72c00640 1 -- 172.21.15.113:0/2793871826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c1f33d0 con 0x7fce6c0c8b00 2024-03-04T08:28:46.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.870+0000 7fce72c00640 1 -- 172.21.15.113:0/2793871826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c075c00 con 0x7fce6c0c8b00 2024-03-04T08:28:46.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.870+0000 7fce63e00640 1 -- 172.21.15.113:0/2793871826 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce5801a070 con 0x7fce6c0c8b00 2024-03-04T08:28:46.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.870+0000 7fce63e00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fce4c066f90 0x7fce4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:46.868 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.870+0000 7fce63e00640 1 -- 172.21.15.113:0/2793871826 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fce58084f60 con 0x7fce6c0c8b00 2024-03-04T08:28:46.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.870+0000 7fce6be00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fce4c066f90 0x7fce4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:46.869 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.870+0000 7fce6be00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fce4c066f90 0x7fce4c069450 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0013e0 tx=0x7fce5c001dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:46.873 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:46.874+0000 7fce63e00640 1 -- 172.21.15.113:0/2793871826 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fce58055430 con 0x7fce6c0c8b00 2024-03-04T08:28:47.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 -- 172.21.15.113:0/2793871826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce6c06fdf0 con 0x7fce6c0c8b00 2024-03-04T08:28:47.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce63e00640 1 -- 172.21.15.113:0/2793871826 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fce58054d30 con 0x7fce6c0c8b00 2024-03-04T08:28:47.059 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:47.059 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:47.060 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 -- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fce4c066f90 msgr2=0x7fce4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fce4c066f90 0x7fce4c069450 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0013e0 tx=0x7fce5c001dc0 comp rx=0 tx=0).stop 2024-03-04T08:28:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 -- 172.21.15.113:0/2793871826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce6c0c8b00 msgr2=0x7fce6c1f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce6c0c8b00 0x7fce6c1f7240 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c430 tx=0x7fce5800c8f0 comp rx=0 tx=0).stop 2024-03-04T08:28:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 -- 172.21.15.113:0/2793871826 shutdown_connections 2024-03-04T08:28:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce6c152b20 0x7fce6c06cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fce4c066f90 0x7fce4c069450 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:47.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce6c0c8b00 0x7fce6c1f7240 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 --2- 172.21.15.113:0/2793871826 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fce6c070010 0x7fce6c1f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 -- 172.21.15.113:0/2793871826 >> 172.21.15.113:0/2793871826 conn(0x7fce6c0709a0 msgr2=0x7fce6c1580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 -- 172.21.15.113:0/2793871826 shutdown_connections 2024-03-04T08:28:47.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:47.062+0000 7fce72c00640 1 -- 172.21.15.113:0/2793871826 wait complete. 2024-03-04T08:28:47.080 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:49.081 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:49.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6af200640 1 Processor -- start 2024-03-04T08:28:49.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6af200640 1 -- start start 2024-03-04T08:28:49.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6af200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd6b0152950 0x7fd6b015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:49.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6af200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b00cb540 0x7fd6b00c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:49.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6af200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b00c4ce0 con 0x7fd6b006faa0 2024-03-04T08:28:49.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6af200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b00c8370 con 0x7fd6b0152950 2024-03-04T08:28:49.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6af200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00ca0a0 con 0x7fd6b00cb540 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6ade00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd6b0152950 0x7fd6b015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6ae800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b00cb540 0x7fd6b00c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6a5400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7fd6b006faa0 0x7fd6b006ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:33530/0 (socket says 172.21.15.113:33530) 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6a5400640 1 -- 172.21.15.113:0/3684305562 learned_addr learned my addr 172.21.15.113:0/3684305562 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6ad400640 1 -- 172.21.15.113:0/3684305562 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525529504 0 0) 0x7fd6b00c4ce0 con 0x7fd6b006faa0 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6ad400640 1 -- 172.21.15.113:0/3684305562 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd694003530 con 0x7fd6b006faa0 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6ade00640 1 -- 172.21.15.113:0/3684305562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b00cb540 msgr2=0x7fd6b00c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6ade00640 1 --2- 172.21.15.113:0/3684305562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b00cb540 0x7fd6b00c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6ade00640 1 -- 172.21.15.113:0/3684305562 >> v1:172.21.15.136:6789/0 conn(0x7fd6b006faa0 legacy=0x7fd6b006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6ade00640 1 -- 172.21.15.113:0/3684305562 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b00c4f80 con 0x7fd6b0152950 2024-03-04T08:28:49.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.230+0000 7fd6ae800640 1 --2- 172.21.15.113:0/3684305562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b00cb540 0x7fd6b00c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:28:49.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6ade00640 1 --2- 172.21.15.113:0/3684305562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b0152950 0x7fd6b015bb40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009d90 tx=0x7fd6a001f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98caf73c8731d542 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6ad400640 1 -- 172.21.15.113:0/3684305562 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00b4070 con 0x7fd6b0152950 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6ad400640 1 -- 172.21.15.113:0/3684305562 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a00af600 con 0x7fd6b0152950 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6ad400640 1 -- 172.21.15.113:0/3684305562 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a00af9d0 con 0x7fd6b0152950 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6af200640 1 -- 172.21.15.113:0/3684305562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b0152950 msgr2=0x7fd6b015bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6af200640 1 --2- 172.21.15.113:0/3684305562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b0152950 0x7fd6b015bb40 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009d90 tx=0x7fd6a001f150 comp rx=0 tx=0).stop 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6af200640 1 -- 172.21.15.113:0/3684305562 shutdown_connections 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6af200640 1 --2- 172.21.15.113:0/3684305562 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b00cb540 0x7fd6b00c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6af200640 1 --2- 172.21.15.113:0/3684305562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b0152950 0x7fd6b015bb40 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6af200640 1 -- 172.21.15.113:0/3684305562 >> 172.21.15.113:0/3684305562 conn(0x7fd6b0072f40 msgr2=0x7fd6b00706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6af200640 1 -- 172.21.15.113:0/3684305562 shutdown_connections 2024-03-04T08:28:49.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.234+0000 7fd6af200640 1 -- 172.21.15.113:0/3684305562 wait complete. 2024-03-04T08:28:49.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6af200640 1 Processor -- start 2024-03-04T08:28:49.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6af200640 1 -- start start 2024-03-04T08:28:49.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6af200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b006faa0 0x7fd6b0067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:49.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6af200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b00cb540 0x7fd6b01f1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:49.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6af200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b0152950 0x7fd6b01f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:49.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6af200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00c9a50 con 0x7fd6b0152950 2024-03-04T08:28:49.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6af200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b006fe80 con 0x7fd6b00cb540 2024-03-04T08:28:49.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6af200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00c9d30 con 0x7fd6b006faa0 2024-03-04T08:28:49.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6ae800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b0152950 0x7fd6b01f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:49.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6ae800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b0152950 0x7fd6b01f1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49510/0 (socket says 172.21.15.113:49510) 2024-03-04T08:28:49.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6ae800640 1 -- 172.21.15.113:0/4237300921 learned_addr learned my addr 172.21.15.113:0/4237300921 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:49.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6a5400640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b00cb540 0x7fd6b01f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6ade00640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b006faa0 0x7fd6b0067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6ae800640 1 -- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b006faa0 msgr2=0x7fd6b0067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6ae800640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b006faa0 0x7fd6b0067520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6ae800640 1 -- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b00cb540 msgr2=0x7fd6b01f1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6ae800640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b00cb540 0x7fd6b01f1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6ae800640 1 -- 172.21.15.113:0/4237300921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b01e1a40 con 0x7fd6b0152950 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6ae800640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b0152950 0x7fd6b01f1850 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800e980 tx=0x7fd6a800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6a6a00640 1 -- 172.21.15.113:0/4237300921 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8017070 con 0x7fd6b0152950 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6a6a00640 1 -- 172.21.15.113:0/4237300921 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a800ca00 con 0x7fd6b0152950 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6af200640 1 -- 172.21.15.113:0/4237300921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b00c5880 con 0x7fd6b0152950 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6a6a00640 1 -- 172.21.15.113:0/4237300921 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a800cdd0 con 0x7fd6b0152950 2024-03-04T08:28:49.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.238+0000 7fd6af200640 1 -- 172.21.15.113:0/4237300921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b01e1f90 con 0x7fd6b0152950 2024-03-04T08:28:49.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.242+0000 7fd6af200640 1 -- 172.21.15.113:0/4237300921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b0075a80 con 0x7fd6b0152950 2024-03-04T08:28:49.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.242+0000 7fd6a6a00640 1 -- 172.21.15.113:0/4237300921 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6a8013470 con 0x7fd6b0152950 2024-03-04T08:28:49.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.242+0000 7fd6a6a00640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd68c066f90 0x7fd68c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:49.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.242+0000 7fd6a6a00640 1 -- 172.21.15.113:0/4237300921 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fd6a80855f0 con 0x7fd6b0152950 2024-03-04T08:28:49.241 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.242+0000 7fd6ade00640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd68c066f90 0x7fd68c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:49.242 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.246+0000 7fd6ade00640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd68c066f90 0x7fd68c069450 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00b1000 tx=0x7fd6a00aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:49.245 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.246+0000 7fd6a6a00640 1 -- 172.21.15.113:0/4237300921 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd6a80553e0 con 0x7fd6b0152950 2024-03-04T08:28:49.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.434+0000 7fd6af200640 1 -- 172.21.15.113:0/4237300921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6b006f880 con 0x7fd6b0152950 2024-03-04T08:28:49.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.434+0000 7fd6a6a00640 1 -- 172.21.15.113:0/4237300921 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fd6a8059480 con 0x7fd6b0152950 2024-03-04T08:28:49.433 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:49.433 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:49.433 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:49.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 -- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd68c066f90 msgr2=0x7fd68c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:49.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd68c066f90 0x7fd68c069450 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00b1000 tx=0x7fd6a00aa000 comp rx=0 tx=0).stop 2024-03-04T08:28:49.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 -- 172.21.15.113:0/4237300921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b0152950 msgr2=0x7fd6b01f1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:49.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b0152950 0x7fd6b01f1850 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800e980 tx=0x7fd6a800ee40 comp rx=0 tx=0).stop 2024-03-04T08:28:49.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 -- 172.21.15.113:0/4237300921 shutdown_connections 2024-03-04T08:28:49.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd68c066f90 0x7fd68c069450 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:49.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b0152950 0x7fd6b01f1850 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:49.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6b00cb540 0x7fd6b01f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:49.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 --2- 172.21.15.113:0/4237300921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd6b006faa0 0x7fd6b0067520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:49.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 -- 172.21.15.113:0/4237300921 >> 172.21.15.113:0/4237300921 conn(0x7fd6b0072f40 msgr2=0x7fd6b0159b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:49.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 -- 172.21.15.113:0/4237300921 shutdown_connections 2024-03-04T08:28:49.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:49.438+0000 7fd6af200640 1 -- 172.21.15.113:0/4237300921 wait complete. 2024-03-04T08:28:49.454 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:28:49.455 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:28:51.456 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f6014c00640 1 Processor -- start 2024-03-04T08:28:51.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f6014c00640 1 -- start start 2024-03-04T08:28:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f6014c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f60101527f0 0x7f6010070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f6014c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60100c8200 0x7f6010072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f6014c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60101591f0 con 0x7f6010152bd0 2024-03-04T08:28:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f6014c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6010071df0 con 0x7f60101527f0 2024-03-04T08:28:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f6014c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60101597b0 con 0x7f60100c8200 2024-03-04T08:28:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f600e800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6010152bd0 0x7f60101587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57172/0 (socket says 172.21.15.113:57172) 2024-03-04T08:28:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f600e800640 1 -- 172.21.15.113:0/1410776455 learned_addr learned my addr 172.21.15.113:0/1410776455 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f600de00640 1 --2- 172.21.15.113:0/1410776455 >> v2:172.21.15.136:3300/0 conn(0x7f60101527f0 0x7f6010070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.606+0000 7f6005400640 1 --2- 172.21.15.113:0/1410776455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60100c8200 0x7f6010072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600d400640 1 -- 172.21.15.113:0/1410776455 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998476540 0 0) 0x7f60101591f0 con 0x7f6010152bd0 2024-03-04T08:28:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600d400640 1 -- 172.21.15.113:0/1410776455 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff4003530 con 0x7f6010152bd0 2024-03-04T08:28:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600d400640 1 -- 172.21.15.113:0/1410776455 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2079157825 0 0) 0x7f5ff4003530 con 0x7f6010152bd0 2024-03-04T08:28:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600de00640 1 -- 172.21.15.113:0/1410776455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60100c8200 msgr2=0x7f6010072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600de00640 1 --2- 172.21.15.113:0/1410776455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60100c8200 0x7f6010072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600de00640 1 -- 172.21.15.113:0/1410776455 >> v1:172.21.15.113:6789/0 conn(0x7f6010152bd0 legacy=0x7f60101587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600de00640 1 -- 172.21.15.113:0/1410776455 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60101594d0 con 0x7f60101527f0 2024-03-04T08:28:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600de00640 1 --2- 172.21.15.113:0/1410776455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60101527f0 0x7f6010070d40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f60000097c0 tx=0x7f600001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd34515796c2acc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:51.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600d400640 1 -- 172.21.15.113:0/1410776455 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60000b0070 con 0x7f60101527f0 2024-03-04T08:28:51.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600d400640 1 -- 172.21.15.113:0/1410776455 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60000ab400 con 0x7f60101527f0 2024-03-04T08:28:51.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.610+0000 7f600d400640 1 -- 172.21.15.113:0/1410776455 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008001a00 con 0x7f60101527f0 2024-03-04T08:28:51.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 -- 172.21.15.113:0/1410776455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60101527f0 msgr2=0x7f6010070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:51.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 --2- 172.21.15.113:0/1410776455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60101527f0 0x7f6010070d40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f60000097c0 tx=0x7f600001c3c0 comp rx=0 tx=0).stop 2024-03-04T08:28:51.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 -- 172.21.15.113:0/1410776455 shutdown_connections 2024-03-04T08:28:51.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 --2- 172.21.15.113:0/1410776455 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60100c8200 0x7f6010072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:51.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 --2- 172.21.15.113:0/1410776455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60101527f0 0x7f6010070d40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:51.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 -- 172.21.15.113:0/1410776455 >> 172.21.15.113:0/1410776455 conn(0x7f601006e820 msgr2=0x7f601006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:51.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 -- 172.21.15.113:0/1410776455 shutdown_connections 2024-03-04T08:28:51.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 -- 172.21.15.113:0/1410776455 wait complete. 2024-03-04T08:28:51.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 Processor -- start 2024-03-04T08:28:51.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 -- start start 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60100c8200 0x7f6010158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60101527f0 0x7f60101f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6010152bd0 0x7f601014a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6010165160 con 0x7f6010152bd0 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60100727c0 con 0x7f60100c8200 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6014c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60101652e0 con 0x7f60101527f0 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f600e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6010152bd0 0x7f601014a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f600e800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6010152bd0 0x7f601014a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49526/0 (socket says 172.21.15.113:49526) 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f600e800640 1 -- 172.21.15.113:0/7980544 learned_addr learned my addr 172.21.15.113:0/7980544 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f600de00640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60100c8200 0x7f6010158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f6005400640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60101527f0 0x7f60101f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f600e800640 1 -- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60101527f0 msgr2=0x7f60101f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f600e800640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60101527f0 0x7f60101f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f600e800640 1 -- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60100c8200 msgr2=0x7f6010158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f600e800640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60100c8200 0x7f6010158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f600e800640 1 -- 172.21.15.113:0/7980544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60100aec30 con 0x7f6010152bd0 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.614+0000 7f600e800640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6010152bd0 0x7f601014a2e0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f600800ccc0 tx=0x7f600800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.618+0000 7f6006a00640 1 -- 172.21.15.113:0/7980544 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6008012400 con 0x7f6010152bd0 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.618+0000 7f6006a00640 1 -- 172.21.15.113:0/7980544 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60080125c0 con 0x7f6010152bd0 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.618+0000 7f6006a00640 1 -- 172.21.15.113:0/7980544 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60080129b0 con 0x7f6010152bd0 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.618+0000 7f6014c00640 1 -- 172.21.15.113:0/7980544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6010158c50 con 0x7f6010152bd0 2024-03-04T08:28:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.618+0000 7f6014c00640 1 -- 172.21.15.113:0/7980544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601014cf70 con 0x7f6010152bd0 2024-03-04T08:28:51.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.618+0000 7f6014c00640 1 -- 172.21.15.113:0/7980544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6010075c00 con 0x7f6010152bd0 2024-03-04T08:28:51.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.618+0000 7f6006a00640 1 -- 172.21.15.113:0/7980544 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6008012b50 con 0x7f6010152bd0 2024-03-04T08:28:51.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.618+0000 7f6006a00640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fec066f90 0x7f5fec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:51.617 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.618+0000 7f6006a00640 1 -- 172.21.15.113:0/7980544 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f6008086210 con 0x7f6010152bd0 2024-03-04T08:28:51.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.618+0000 7f600de00640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fec066f90 0x7f5fec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:51.618 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.622+0000 7f600de00640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fec066f90 0x7f5fec069450 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f60000ad000 tx=0x7f60000a6000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:51.622 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.622+0000 7f6006a00640 1 -- 172.21.15.113:0/7980544 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6008056770 con 0x7f6010152bd0 2024-03-04T08:28:51.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.809+0000 7f6014c00640 1 -- 172.21.15.113:0/7980544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60100c79d0 con 0x7f6010152bd0 2024-03-04T08:28:51.810 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.809+0000 7f6006a00640 1 -- 172.21.15.113:0/7980544 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f6008056070 con 0x7f6010152bd0 2024-03-04T08:28:51.810 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:51.810 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:51.811 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:51.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 -- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fec066f90 msgr2=0x7f5fec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:51.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fec066f90 0x7f5fec069450 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f60000ad000 tx=0x7f60000a6000 comp rx=0 tx=0).stop 2024-03-04T08:28:51.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 -- 172.21.15.113:0/7980544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6010152bd0 msgr2=0x7f601014a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:51.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6010152bd0 0x7f601014a2e0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f600800ccc0 tx=0x7f600800a430 comp rx=0 tx=0).stop 2024-03-04T08:28:51.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 -- 172.21.15.113:0/7980544 shutdown_connections 2024-03-04T08:28:51.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5fec066f90 0x7f5fec069450 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:51.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6010152bd0 0x7f601014a2e0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:51.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f60101527f0 0x7f60101f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:51.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 --2- 172.21.15.113:0/7980544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60100c8200 0x7f6010158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:51.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 -- 172.21.15.113:0/7980544 >> 172.21.15.113:0/7980544 conn(0x7f601006e820 msgr2=0x7f60100c2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:51.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 -- 172.21.15.113:0/7980544 shutdown_connections 2024-03-04T08:28:51.813 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:51.813+0000 7f6014c00640 1 -- 172.21.15.113:0/7980544 wait complete. 2024-03-04T08:28:51.835 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:53.836 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:53.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.981+0000 7f182c400640 1 Processor -- start 2024-03-04T08:28:53.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.981+0000 7f182c400640 1 -- start start 2024-03-04T08:28:53.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182c400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f18240b20b0 0x7f1824157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:53.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182c400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18240c85f0 0x7f1824161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:53.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182c400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18240c1ec0 con 0x7f1824157c50 2024-03-04T08:28:53.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182c400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240c9b30 con 0x7f18240c85f0 2024-03-04T08:28:53.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182c400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240c12a0 con 0x7f18240b20b0 2024-03-04T08:28:53.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18240c85f0 0x7f1824161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:53.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f1829a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f18240b20b0 0x7f1824157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:53.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182a400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18240c85f0 0x7f1824161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49538/0 (socket says 172.21.15.113:49538) 2024-03-04T08:28:53.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182a400640 1 -- 172.21.15.113:0/3492929528 learned_addr learned my addr 172.21.15.113:0/3492929528 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:53.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182a400640 1 -- 172.21.15.113:0/3492929528 >> v2:172.21.15.136:3300/0 conn(0x7f18240b20b0 msgr2=0x7f1824157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:53.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182a400640 1 --2- 172.21.15.113:0/3492929528 >> v2:172.21.15.136:3300/0 conn(0x7f18240b20b0 0x7f1824157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:53.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182a400640 1 -- 172.21.15.113:0/3492929528 >> v1:172.21.15.136:6789/0 conn(0x7f1824157c50 legacy=0x7f182406b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:53.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.985+0000 7f182a400640 1 -- 172.21.15.113:0/3492929528 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240c1580 con 0x7f18240c85f0 2024-03-04T08:28:53.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182a400640 1 --2- 172.21.15.113:0/3492929528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18240c85f0 0x7f1824161820 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f1824070670 tx=0x7f181c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dffad66a1a54c34 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:53.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f1823e00640 1 -- 172.21.15.113:0/3492929528 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c0b4070 con 0x7f18240c85f0 2024-03-04T08:28:53.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f1823e00640 1 -- 172.21.15.113:0/3492929528 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c0af400 con 0x7f18240c85f0 2024-03-04T08:28:53.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f1823e00640 1 -- 172.21.15.113:0/3492929528 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c0af7d0 con 0x7f18240c85f0 2024-03-04T08:28:53.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182c400640 1 -- 172.21.15.113:0/3492929528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18240c85f0 msgr2=0x7f1824161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:53.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182c400640 1 --2- 172.21.15.113:0/3492929528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18240c85f0 0x7f1824161820 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f1824070670 tx=0x7f181c01f1e0 comp rx=0 tx=0).stop 2024-03-04T08:28:53.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182c400640 1 -- 172.21.15.113:0/3492929528 shutdown_connections 2024-03-04T08:28:53.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182c400640 1 --2- 172.21.15.113:0/3492929528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18240c85f0 0x7f1824161820 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:53.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182c400640 1 --2- 172.21.15.113:0/3492929528 >> v2:172.21.15.136:3300/0 conn(0x7f18240b20b0 0x7f1824157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182c400640 1 -- 172.21.15.113:0/3492929528 >> 172.21.15.113:0/3492929528 conn(0x7f182406e820 msgr2=0x7f182406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182c400640 1 -- 172.21.15.113:0/3492929528 shutdown_connections 2024-03-04T08:28:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182c400640 1 -- 172.21.15.113:0/3492929528 wait complete. 2024-03-04T08:28:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182c400640 1 Processor -- start 2024-03-04T08:28:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.989+0000 7f182c400640 1 -- start start 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182c400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18240b20b0 0x7f18241e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182c400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18240c85f0 0x7f18240c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182c400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824157c50 0x7f182414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182c400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c2400 con 0x7f1824157c50 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182c400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18241571f0 con 0x7f18240c85f0 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182c400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1824156ec0 con 0x7f18240b20b0 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824157c50 0x7f182414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f1829a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18240b20b0 0x7f18241e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182a400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824157c50 0x7f182414c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49548/0 (socket says 172.21.15.113:49548) 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182a400640 1 -- 172.21.15.113:0/2679012840 learned_addr learned my addr 172.21.15.113:0/2679012840 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:53.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f1829000640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18240c85f0 0x7f18240c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182a400640 1 -- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18240b20b0 msgr2=0x7f18241e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182a400640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18240b20b0 0x7f18241e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182a400640 1 -- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18240c85f0 msgr2=0x7f18240c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182a400640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18240c85f0 0x7f18240c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182a400640 1 -- 172.21.15.113:0/2679012840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18241e90d0 con 0x7f1824157c50 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182a400640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824157c50 0x7f182414c220 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f181c0b2040 tx=0x7f181c01bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f1821600640 1 -- 172.21.15.113:0/2679012840 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c0c0070 con 0x7f1824157c50 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f1821600640 1 -- 172.21.15.113:0/2679012840 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c0afcc0 con 0x7f1824157c50 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f1821600640 1 -- 172.21.15.113:0/2679012840 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c0b4040 con 0x7f1824157c50 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182c400640 1 -- 172.21.15.113:0/2679012840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1824152d60 con 0x7f1824157c50 2024-03-04T08:28:53.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182c400640 1 -- 172.21.15.113:0/2679012840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1824153000 con 0x7f1824157c50 2024-03-04T08:28:53.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f182c400640 1 -- 172.21.15.113:0/2679012840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1824075c00 con 0x7f1824157c50 2024-03-04T08:28:53.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.993+0000 7f1821600640 1 -- 172.21.15.113:0/2679012840 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f181c0b8670 con 0x7f1824157c50 2024-03-04T08:28:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.997+0000 7f1821600640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1808066f90 0x7f1808069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.997+0000 7f1821600640 1 -- 172.21.15.113:0/2679012840 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f181c123270 con 0x7f1824157c50 2024-03-04T08:28:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.997+0000 7f1829a00640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1808066f90 0x7f1808069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:53.997+0000 7f1829a00640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1808066f90 0x7f1808069450 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1814009ac0 tx=0x7f1814009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:53.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.001+0000 7f1821600640 1 -- 172.21.15.113:0/2679012840 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f181c018d40 con 0x7f1824157c50 2024-03-04T08:28:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.185+0000 7f182c400640 1 -- 172.21.15.113:0/2679012840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1824151400 con 0x7f1824157c50 2024-03-04T08:28:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.185+0000 7f1821600640 1 -- 172.21.15.113:0/2679012840 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f181c0f3060 con 0x7f1824157c50 2024-03-04T08:28:54.185 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:54.185 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.185+0000 7f182c400640 1 -- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1808066f90 msgr2=0x7f1808069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.185+0000 7f182c400640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1808066f90 0x7f1808069450 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1814009ac0 tx=0x7f1814009190 comp rx=0 tx=0).stop 2024-03-04T08:28:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.185+0000 7f182c400640 1 -- 172.21.15.113:0/2679012840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824157c50 msgr2=0x7f182414c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.185+0000 7f182c400640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824157c50 0x7f182414c220 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f181c0b2040 tx=0x7f181c01bf80 comp rx=0 tx=0).stop 2024-03-04T08:28:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.189+0000 7f182c400640 1 -- 172.21.15.113:0/2679012840 shutdown_connections 2024-03-04T08:28:54.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.189+0000 7f182c400640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1808066f90 0x7f1808069450 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.189+0000 7f182c400640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1824157c50 0x7f182414c220 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.189+0000 7f182c400640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18240c85f0 0x7f18240c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.189+0000 7f182c400640 1 --2- 172.21.15.113:0/2679012840 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18240b20b0 0x7f18241e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.189+0000 7f182c400640 1 -- 172.21.15.113:0/2679012840 >> 172.21.15.113:0/2679012840 conn(0x7f182406e820 msgr2=0x7f18240ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.189+0000 7f182c400640 1 -- 172.21.15.113:0/2679012840 shutdown_connections 2024-03-04T08:28:54.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:54.189+0000 7f182c400640 1 -- 172.21.15.113:0/2679012840 wait complete. 2024-03-04T08:28:54.205 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:56.206 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:56.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6799000640 1 Processor -- start 2024-03-04T08:28:56.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6799000640 1 -- start start 2024-03-04T08:28:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6799000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f679414e4c0 0x7f6794072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6799000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67940c22e0 0x7f67940731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6799000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67940c27f0 con 0x7f67940c2d80 2024-03-04T08:28:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6799000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679415ad60 con 0x7f67940c22e0 2024-03-04T08:28:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6799000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67940c2a90 con 0x7f679414e4c0 2024-03-04T08:28:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6789400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67940c22e0 0x7f67940731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6789400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67940c22e0 0x7f67940731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:49550/0 (socket says 172.21.15.113:49550) 2024-03-04T08:28:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6789400640 1 -- 172.21.15.113:0/2110635413 learned_addr learned my addr 172.21.15.113:0/2110635413 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:56.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6791e00640 1 --2- 172.21.15.113:0/2110635413 >> v2:172.21.15.136:3300/0 conn(0x7f679414e4c0 0x7f6794072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:56.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6789400640 1 -- 172.21.15.113:0/2110635413 >> v2:172.21.15.136:3300/0 conn(0x7f679414e4c0 msgr2=0x7f6794072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:28:56.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6789400640 1 --2- 172.21.15.113:0/2110635413 >> v2:172.21.15.136:3300/0 conn(0x7f679414e4c0 0x7f6794072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:56.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6789400640 1 -- 172.21.15.113:0/2110635413 >> v1:172.21.15.136:6789/0 conn(0x7f67940c2d80 legacy=0x7f679415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:56.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.353+0000 7f6789400640 1 -- 172.21.15.113:0/2110635413 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6794070c60 con 0x7f67940c22e0 2024-03-04T08:28:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6789400640 1 --2- 172.21.15.113:0/2110635413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67940c22e0 0x7f67940731c0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f679406f690 tx=0x7f677c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646477e66e6709b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6791400640 1 -- 172.21.15.113:0/2110635413 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0b5070 con 0x7f67940c22e0 2024-03-04T08:28:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6791400640 1 -- 172.21.15.113:0/2110635413 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c0a8a10 con 0x7f67940c22e0 2024-03-04T08:28:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6791400640 1 -- 172.21.15.113:0/2110635413 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0a8de0 con 0x7f67940c22e0 2024-03-04T08:28:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6799000640 1 -- 172.21.15.113:0/2110635413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67940c22e0 msgr2=0x7f67940731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6799000640 1 --2- 172.21.15.113:0/2110635413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67940c22e0 0x7f67940731c0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f679406f690 tx=0x7f677c01ed10 comp rx=0 tx=0).stop 2024-03-04T08:28:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6799000640 1 -- 172.21.15.113:0/2110635413 shutdown_connections 2024-03-04T08:28:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6799000640 1 --2- 172.21.15.113:0/2110635413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67940c22e0 0x7f67940731c0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6799000640 1 --2- 172.21.15.113:0/2110635413 >> v2:172.21.15.136:3300/0 conn(0x7f679414e4c0 0x7f6794072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:56.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6799000640 1 -- 172.21.15.113:0/2110635413 >> 172.21.15.113:0/2110635413 conn(0x7f679406e820 msgr2=0x7f679406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6799000640 1 -- 172.21.15.113:0/2110635413 shutdown_connections 2024-03-04T08:28:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.357+0000 7f6799000640 1 -- 172.21.15.113:0/2110635413 wait complete. 2024-03-04T08:28:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6799000640 1 Processor -- start 2024-03-04T08:28:56.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6799000640 1 -- start start 2024-03-04T08:28:56.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6799000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67940c22e0 0x7f67941f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:56.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6799000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67940c2d80 0x7f6794159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:56.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6799000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f679414e4c0 0x7f6794163100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:56.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6799000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6794071500 con 0x7f679414e4c0 2024-03-04T08:28:56.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6799000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6794071220 con 0x7f67940c22e0 2024-03-04T08:28:56.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6799000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679414e8a0 con 0x7f67940c2d80 2024-03-04T08:28:56.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6791e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67940c22e0 0x7f67941f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:56.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6792800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f679414e4c0 0x7f6794163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:56.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6791e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67940c22e0 0x7f67941f4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:55634/0 (socket says 172.21.15.113:55634) 2024-03-04T08:28:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6791e00640 1 -- 172.21.15.113:0/2167446777 learned_addr learned my addr 172.21.15.113:0/2167446777 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6789400640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67940c2d80 0x7f6794159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:56.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6792800640 1 -- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67940c2d80 msgr2=0x7f6794159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6792800640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67940c2d80 0x7f6794159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6792800640 1 -- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67940c22e0 msgr2=0x7f67941f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6792800640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67940c22e0 0x7f67941f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6792800640 1 -- 172.21.15.113:0/2167446777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67940cb720 con 0x7f679414e4c0 2024-03-04T08:28:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6792800640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f679414e4c0 0x7f6794163100 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c730 tx=0x7f678c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f678aa00640 1 -- 172.21.15.113:0/2167446777 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c003b80 con 0x7f679414e4c0 2024-03-04T08:28:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f678aa00640 1 -- 172.21.15.113:0/2167446777 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c003d20 con 0x7f679414e4c0 2024-03-04T08:28:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f6799000640 1 -- 172.21.15.113:0/2167446777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67941647e0 con 0x7f679414e4c0 2024-03-04T08:28:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.361+0000 7f678aa00640 1 -- 172.21.15.113:0/2167446777 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c005310 con 0x7f679414e4c0 2024-03-04T08:28:56.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.365+0000 7f6799000640 1 -- 172.21.15.113:0/2167446777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67941650c0 con 0x7f679414e4c0 2024-03-04T08:28:56.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.365+0000 7f6799000640 1 -- 172.21.15.113:0/2167446777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67941f4b10 con 0x7f679414e4c0 2024-03-04T08:28:56.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.365+0000 7f678aa00640 1 -- 172.21.15.113:0/2167446777 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f678c0056c0 con 0x7f679414e4c0 2024-03-04T08:28:56.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.365+0000 7f678aa00640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6770066f60 0x7f6770069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:56.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.365+0000 7f678aa00640 1 -- 172.21.15.113:0/2167446777 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f678c085a40 con 0x7f679414e4c0 2024-03-04T08:28:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.365+0000 7f6791e00640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6770066f60 0x7f6770069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:56.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.369+0000 7f6791e00640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6770066f60 0x7f6770069420 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6784009b80 tx=0x7f6784009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:56.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.369+0000 7f678aa00640 1 -- 172.21.15.113:0/2167446777 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f67941f4b10 con 0x7f679414e4c0 2024-03-04T08:28:56.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.557+0000 7f6799000640 1 -- 172.21.15.113:0/2167446777 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6794075c00 con 0x7f679414e4c0 2024-03-04T08:28:56.557 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.557+0000 7f678aa00640 1 -- 172.21.15.113:0/2167446777 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f678c085e20 con 0x7f679414e4c0 2024-03-04T08:28:56.557 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:56.558 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:56.558 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:56.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 -- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6770066f60 msgr2=0x7f6770069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:56.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6770066f60 0x7f6770069420 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6784009b80 tx=0x7f6784009190 comp rx=0 tx=0).stop 2024-03-04T08:28:56.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 -- 172.21.15.113:0/2167446777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f679414e4c0 msgr2=0x7f6794163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:56.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f679414e4c0 0x7f6794163100 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c730 tx=0x7f678c00cbf0 comp rx=0 tx=0).stop 2024-03-04T08:28:56.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 -- 172.21.15.113:0/2167446777 shutdown_connections 2024-03-04T08:28:56.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6770066f60 0x7f6770069420 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:56.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f679414e4c0 0x7f6794163100 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:56.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f67940c2d80 0x7f6794159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:56.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 --2- 172.21.15.113:0/2167446777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67940c22e0 0x7f67941f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:56.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 -- 172.21.15.113:0/2167446777 >> 172.21.15.113:0/2167446777 conn(0x7f679406e820 msgr2=0x7f679414c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:56.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 -- 172.21.15.113:0/2167446777 shutdown_connections 2024-03-04T08:28:56.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:56.561+0000 7f6799000640 1 -- 172.21.15.113:0/2167446777 wait complete. 2024-03-04T08:28:56.578 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:28:58.579 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:28:58.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.725+0000 7f277da00640 1 Processor -- start 2024-03-04T08:28:58.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.725+0000 7f277da00640 1 -- start start 2024-03-04T08:28:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f277da00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27781527f0 0x7f2778070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f277da00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27781591f0 con 0x7f27780c8200 2024-03-04T08:28:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f277da00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27781594d0 con 0x7f2778152bd0 2024-03-04T08:28:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f277da00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2778071df0 con 0x7f27781527f0 2024-03-04T08:28:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f276e000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f27780c8200 0x7f2778072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56184/0 (socket says 172.21.15.113:56184) 2024-03-04T08:28:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f2776a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27781527f0 0x7f2778070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:58.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f276e000640 1 -- 172.21.15.113:0/205606507 learned_addr learned my addr 172.21.15.113:0/205606507 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:28:58.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f2776a00640 1 -- 172.21.15.113:0/205606507 >> v1:172.21.15.136:6789/0 conn(0x7f2778152bd0 legacy=0x7f27781587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:58.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f2776a00640 1 -- 172.21.15.113:0/205606507 >> v1:172.21.15.113:6789/0 conn(0x7f27780c8200 legacy=0x7f2778072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:58.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f2776a00640 1 -- 172.21.15.113:0/205606507 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27780717a0 con 0x7f27781527f0 2024-03-04T08:28:58.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.729+0000 7f2776000640 1 -- 172.21.15.113:0/205606507 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678253948 0 0) 0x7f27781591f0 con 0x7f27780c8200 2024-03-04T08:28:58.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f2776a00640 1 --2- 172.21.15.113:0/205606507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27781527f0 0x7f2778070d40 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f27680097c0 tx=0x7f276801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed5969815cea766 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:58.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f2776000640 1 -- 172.21.15.113:0/205606507 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680b1070 con 0x7f27781527f0 2024-03-04T08:28:58.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f2776000640 1 -- 172.21.15.113:0/205606507 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27680ac410 con 0x7f27781527f0 2024-03-04T08:28:58.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f2776000640 1 -- 172.21.15.113:0/205606507 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680ac7e0 con 0x7f27781527f0 2024-03-04T08:28:58.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f277da00640 1 -- 172.21.15.113:0/205606507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27781527f0 msgr2=0x7f2778070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:58.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f277da00640 1 --2- 172.21.15.113:0/205606507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27781527f0 0x7f2778070d40 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f27680097c0 tx=0x7f276801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:28:58.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f277da00640 1 -- 172.21.15.113:0/205606507 shutdown_connections 2024-03-04T08:28:58.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f277da00640 1 --2- 172.21.15.113:0/205606507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27781527f0 0x7f2778070d40 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:58.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f277da00640 1 -- 172.21.15.113:0/205606507 >> 172.21.15.113:0/205606507 conn(0x7f277806e820 msgr2=0x7f277806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:58.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f277da00640 1 -- 172.21.15.113:0/205606507 shutdown_connections 2024-03-04T08:28:58.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f277da00640 1 -- 172.21.15.113:0/205606507 wait complete. 2024-03-04T08:28:58.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f277da00640 1 Processor -- start 2024-03-04T08:28:58.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f277da00640 1 -- start start 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.733+0000 7f277da00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27780c8200 0x7f277814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f277da00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27781527f0 0x7f2778155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f277da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2778152bd0 0x7f27780c13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f277da00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2778166380 con 0x7f27780c8200 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f277da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27781597b0 con 0x7f2778152bd0 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f277da00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2778166500 con 0x7f27781527f0 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2776a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27780c8200 0x7f277814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2776a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27780c8200 0x7f277814e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41726/0 (socket says 172.21.15.113:41726) 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2776a00640 1 -- 172.21.15.113:0/309798505 learned_addr learned my addr 172.21.15.113:0/309798505 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f276e000640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27781527f0 0x7f2778155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:58.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2777400640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2778152bd0 0x7f27780c13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:58.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2776a00640 1 -- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27781527f0 msgr2=0x7f2778155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:58.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2776a00640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27781527f0 0x7f2778155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:58.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2776a00640 1 -- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2778152bd0 msgr2=0x7f27780c13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:58.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2776a00640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2778152bd0 0x7f27780c13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:58.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2776a00640 1 -- 172.21.15.113:0/309798505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27780c3b00 con 0x7f27780c8200 2024-03-04T08:28:58.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2777400640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2778152bd0 0x7f27780c13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:28:58.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f2776a00640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27780c8200 0x7f277814e490 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f27680190b0 tx=0x7f2768019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:58.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f276f400640 1 -- 172.21.15.113:0/309798505 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680b9020 con 0x7f27780c8200 2024-03-04T08:28:58.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f276f400640 1 -- 172.21.15.113:0/309798505 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27680ace40 con 0x7f27780c8200 2024-03-04T08:28:58.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f276f400640 1 -- 172.21.15.113:0/309798505 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27680b1040 con 0x7f27780c8200 2024-03-04T08:28:58.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f277da00640 1 -- 172.21.15.113:0/309798505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27781f5d50 con 0x7f27780c8200 2024-03-04T08:28:58.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f277da00640 1 -- 172.21.15.113:0/309798505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27781f6310 con 0x7f27780c8200 2024-03-04T08:28:58.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f277da00640 1 -- 172.21.15.113:0/309798505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2778075c00 con 0x7f27780c8200 2024-03-04T08:28:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.737+0000 7f276f400640 1 -- 172.21.15.113:0/309798505 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27680bc020 con 0x7f27780c8200 2024-03-04T08:28:58.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.741+0000 7f276f400640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2754066f60 0x7f2754069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:28:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.741+0000 7f276f400640 1 -- 172.21.15.113:0/309798505 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f276811f4b0 con 0x7f27780c8200 2024-03-04T08:28:58.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.741+0000 7f276e000640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2754066f60 0x7f2754069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:28:58.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.741+0000 7f276e000640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2754066f60 0x7f2754069420 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2760007fd0 tx=0x7f2760007ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:28:58.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.741+0000 7f276f400640 1 -- 172.21.15.113:0/309798505 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f27680c3030 con 0x7f27780c8200 2024-03-04T08:28:58.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.929+0000 7f277da00640 1 -- 172.21.15.113:0/309798505 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2778158cc0 con 0x7f27780c8200 2024-03-04T08:28:58.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.929+0000 7f276f400640 1 -- 172.21.15.113:0/309798505 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f27780c3b00 con 0x7f27780c8200 2024-03-04T08:28:58.929 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:28:58.929 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:28:58.929 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:28:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 -- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2754066f60 msgr2=0x7f2754069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2754066f60 0x7f2754069420 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2760007fd0 tx=0x7f2760007ba0 comp rx=0 tx=0).stop 2024-03-04T08:28:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 -- 172.21.15.113:0/309798505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27780c8200 msgr2=0x7f277814e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:28:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27780c8200 0x7f277814e490 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f27680190b0 tx=0x7f2768019240 comp rx=0 tx=0).stop 2024-03-04T08:28:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 -- 172.21.15.113:0/309798505 shutdown_connections 2024-03-04T08:28:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2778152bd0 0x7f27780c13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:58.931 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27781527f0 0x7f2778155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2754066f60 0x7f2754069420 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 --2- 172.21.15.113:0/309798505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27780c8200 0x7f277814e490 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:28:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 -- 172.21.15.113:0/309798505 >> 172.21.15.113:0/309798505 conn(0x7f277806e820 msgr2=0x7f27780c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:28:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 -- 172.21.15.113:0/309798505 shutdown_connections 2024-03-04T08:28:58.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:28:58.933+0000 7f277da00640 1 -- 172.21.15.113:0/309798505 wait complete. 2024-03-04T08:28:58.950 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:00.951 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.097+0000 7f4f86400640 1 Processor -- start 2024-03-04T08:29:01.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.097+0000 7f4f86400640 1 -- start start 2024-03-04T08:29:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.097+0000 7f4f86400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4f800c22e0 0x7f4f8015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.097+0000 7f4f86400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4f800c2d80 0x7f4f8015b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.097+0000 7f4f86400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f800c27f0 con 0x7f4f8014e4c0 2024-03-04T08:29:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.097+0000 7f4f86400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8015ad60 con 0x7f4f800c2d80 2024-03-04T08:29:01.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.097+0000 7f4f86400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f800c2a90 con 0x7f4f800c22e0 2024-03-04T08:29:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.097+0000 7f4f7fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4f800c2d80 0x7f4f8015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.097+0000 7f4f7fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4f800c2d80 0x7f4f8015b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41738/0 (socket says 172.21.15.113:41738) 2024-03-04T08:29:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.097+0000 7f4f7fe00640 1 -- 172.21.15.113:0/3028184652 learned_addr learned my addr 172.21.15.113:0/3028184652 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f7ea00640 1 --2- 172.21.15.113:0/3028184652 >> v2:172.21.15.136:3300/0 conn(0x7f4f800c22e0 0x7f4f8015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f7fe00640 1 -- 172.21.15.113:0/3028184652 >> v2:172.21.15.136:3300/0 conn(0x7f4f800c22e0 msgr2=0x7f4f8015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f7fe00640 1 --2- 172.21.15.113:0/3028184652 >> v2:172.21.15.136:3300/0 conn(0x7f4f800c22e0 0x7f4f8015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f7fe00640 1 -- 172.21.15.113:0/3028184652 >> v1:172.21.15.136:6789/0 conn(0x7f4f8014e4c0 legacy=0x7f4f80072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:01.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f7fe00640 1 -- 172.21.15.113:0/3028184652 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80070c60 con 0x7f4f800c2d80 2024-03-04T08:29:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f7fe00640 1 --2- 172.21.15.113:0/3028184652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f800c2d80 0x7f4f8015b730 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f4f800cd230 tx=0x7f4f7801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e490a3489561090b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:01.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f7e000640 1 -- 172.21.15.113:0/3028184652 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f780a57d0 con 0x7f4f800c2d80 2024-03-04T08:29:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f7e000640 1 -- 172.21.15.113:0/3028184652 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f780a5970 con 0x7f4f800c2d80 2024-03-04T08:29:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f7e000640 1 -- 172.21.15.113:0/3028184652 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f780a5d40 con 0x7f4f800c2d80 2024-03-04T08:29:01.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f86400640 1 -- 172.21.15.113:0/3028184652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f800c2d80 msgr2=0x7f4f8015b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.101+0000 7f4f86400640 1 --2- 172.21.15.113:0/3028184652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f800c2d80 0x7f4f8015b730 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f4f800cd230 tx=0x7f4f7801c9f0 comp rx=0 tx=0).stop 2024-03-04T08:29:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 -- 172.21.15.113:0/3028184652 shutdown_connections 2024-03-04T08:29:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 --2- 172.21.15.113:0/3028184652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f800c2d80 0x7f4f8015b730 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 --2- 172.21.15.113:0/3028184652 >> v2:172.21.15.136:3300/0 conn(0x7f4f800c22e0 0x7f4f8015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 -- 172.21.15.113:0/3028184652 >> 172.21.15.113:0/3028184652 conn(0x7f4f8006e820 msgr2=0x7f4f8006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 -- 172.21.15.113:0/3028184652 shutdown_connections 2024-03-04T08:29:01.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 -- 172.21.15.113:0/3028184652 wait complete. 2024-03-04T08:29:01.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 Processor -- start 2024-03-04T08:29:01.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 -- start start 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f800c22e0 0x7f4f8015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f800c2d80 0x7f4f80141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f8014e4c0 0x7f4f801f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800aede0 con 0x7f4f8014e4c0 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8014e8a0 con 0x7f4f800c2d80 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f86400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80073390 con 0x7f4f800c22e0 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f8014e4c0 0x7f4f801f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f8014e4c0 0x7f4f801f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41742/0 (socket says 172.21.15.113:41742) 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7fe00640 1 -- 172.21.15.113:0/229528715 learned_addr learned my addr 172.21.15.113:0/229528715 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7f400640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f800c22e0 0x7f4f8015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7ea00640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f800c2d80 0x7f4f80141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7fe00640 1 -- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f800c22e0 msgr2=0x7f4f8015a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:01.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7fe00640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f800c22e0 0x7f4f8015a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:01.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7fe00640 1 -- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f800c2d80 msgr2=0x7f4f80141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:01.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7fe00640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f800c2d80 0x7f4f80141940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:01.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7fe00640 1 -- 172.21.15.113:0/229528715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800cbe30 con 0x7f4f8014e4c0 2024-03-04T08:29:01.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7ea00640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f800c2d80 0x7f4f80141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:01.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.105+0000 7f4f7fe00640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f8014e4c0 0x7f4f801f72d0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f4f78019600 tx=0x7f4f78019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:01.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.109+0000 7f4f77400640 1 -- 172.21.15.113:0/229528715 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f780b8470 con 0x7f4f8014e4c0 2024-03-04T08:29:01.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.109+0000 7f4f77400640 1 -- 172.21.15.113:0/229528715 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f780b8610 con 0x7f4f8014e4c0 2024-03-04T08:29:01.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.109+0000 7f4f77400640 1 -- 172.21.15.113:0/229528715 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f780b8a00 con 0x7f4f8014e4c0 2024-03-04T08:29:01.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.109+0000 7f4f86400640 1 -- 172.21.15.113:0/229528715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f800cb2d0 con 0x7f4f8014e4c0 2024-03-04T08:29:01.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.109+0000 7f4f86400640 1 -- 172.21.15.113:0/229528715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f800cb870 con 0x7f4f8014e4c0 2024-03-04T08:29:01.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.109+0000 7f4f86400640 1 -- 172.21.15.113:0/229528715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f80075c00 con 0x7f4f8014e4c0 2024-03-04T08:29:01.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.109+0000 7f4f77400640 1 -- 172.21.15.113:0/229528715 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f780c0020 con 0x7f4f8014e4c0 2024-03-04T08:29:01.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.109+0000 7f4f77400640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f5c066f60 0x7f4f5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:01.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.109+0000 7f4f77400640 1 -- 172.21.15.113:0/229528715 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f4f781204e0 con 0x7f4f8014e4c0 2024-03-04T08:29:01.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.109+0000 7f4f7f400640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f5c066f60 0x7f4f5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:01.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.113+0000 7f4f7f400640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f5c066f60 0x7f4f5c069420 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4f70007fd0 tx=0x7f4f70009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:01.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.113+0000 7f4f77400640 1 -- 172.21.15.113:0/229528715 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4f780c7020 con 0x7f4f8014e4c0 2024-03-04T08:29:01.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 -- 172.21.15.113:0/229528715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f80155190 con 0x7f4f8014e4c0 2024-03-04T08:29:01.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f77400640 1 -- 172.21.15.113:0/229528715 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f4f780165e0 con 0x7f4f8014e4c0 2024-03-04T08:29:01.299 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:01.299 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:01.299 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 -- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f5c066f60 msgr2=0x7f4f5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f5c066f60 0x7f4f5c069420 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4f70007fd0 tx=0x7f4f70009040 comp rx=0 tx=0).stop 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 -- 172.21.15.113:0/229528715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f8014e4c0 msgr2=0x7f4f801f72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f8014e4c0 0x7f4f801f72d0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f4f78019600 tx=0x7f4f78019790 comp rx=0 tx=0).stop 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 -- 172.21.15.113:0/229528715 shutdown_connections 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4f5c066f60 0x7f4f5c069420 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4f8014e4c0 0x7f4f801f72d0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4f800c2d80 0x7f4f80141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 --2- 172.21.15.113:0/229528715 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4f800c22e0 0x7f4f8015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 -- 172.21.15.113:0/229528715 >> 172.21.15.113:0/229528715 conn(0x7f4f8006e820 msgr2=0x7f4f801601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:01.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 -- 172.21.15.113:0/229528715 shutdown_connections 2024-03-04T08:29:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:01.301+0000 7f4f86400640 1 -- 172.21.15.113:0/229528715 wait complete. 2024-03-04T08:29:01.320 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:29:01.321 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:29:03.322 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:03.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c3e00640 1 Processor -- start 2024-03-04T08:29:03.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c3e00640 1 -- start start 2024-03-04T08:29:03.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69bc14e4c0 0x7f69bc072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:03.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69bc0c22e0 0x7f69bc0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:03.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69bc0c27f0 con 0x7f69bc0c2d80 2024-03-04T08:29:03.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69bc15ad60 con 0x7f69bc0c22e0 2024-03-04T08:29:03.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc0c2a90 con 0x7f69bc14e4c0 2024-03-04T08:29:03.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c0a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69bc0c22e0 0x7f69bc0731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:03.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c0a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f69bc0c22e0 0x7f69bc0731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41748/0 (socket says 172.21.15.113:41748) 2024-03-04T08:29:03.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c0a00640 1 -- 172.21.15.113:0/228427874 learned_addr learned my addr 172.21.15.113:0/228427874 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:03.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c1400640 1 --2- 172.21.15.113:0/228427874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69bc14e4c0 0x7f69bc072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:03.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c0a00640 1 -- 172.21.15.113:0/228427874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69bc14e4c0 msgr2=0x7f69bc072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:03.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c0a00640 1 --2- 172.21.15.113:0/228427874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69bc14e4c0 0x7f69bc072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:03.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c0a00640 1 -- 172.21.15.113:0/228427874 >> v1:172.21.15.136:6789/0 conn(0x7f69bc0c2d80 legacy=0x7f69bc15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:03.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.469+0000 7f69c0a00640 1 -- 172.21.15.113:0/228427874 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc070c60 con 0x7f69bc0c22e0 2024-03-04T08:29:03.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69c0a00640 1 --2- 172.21.15.113:0/228427874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 0x7f69bc0731c0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009870 tx=0x7f69a801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9f9046b6f676c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:03.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69bbe00640 1 -- 172.21.15.113:0/228427874 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a80b5070 con 0x7f69bc0c22e0 2024-03-04T08:29:03.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69bbe00640 1 -- 172.21.15.113:0/228427874 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a80a8a50 con 0x7f69bc0c22e0 2024-03-04T08:29:03.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69bbe00640 1 -- 172.21.15.113:0/228427874 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a80a8e20 con 0x7f69bc0c22e0 2024-03-04T08:29:03.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69c3e00640 1 -- 172.21.15.113:0/228427874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 msgr2=0x7f69bc0731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:03.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69c3e00640 1 --2- 172.21.15.113:0/228427874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 0x7f69bc0731c0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009870 tx=0x7f69a801ed50 comp rx=0 tx=0).stop 2024-03-04T08:29:03.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69c3e00640 1 -- 172.21.15.113:0/228427874 shutdown_connections 2024-03-04T08:29:03.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69c3e00640 1 --2- 172.21.15.113:0/228427874 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 0x7f69bc0731c0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:03.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69c3e00640 1 --2- 172.21.15.113:0/228427874 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69bc14e4c0 0x7f69bc072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:03.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69c3e00640 1 -- 172.21.15.113:0/228427874 >> 172.21.15.113:0/228427874 conn(0x7f69bc06e820 msgr2=0x7f69bc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:03.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69c3e00640 1 -- 172.21.15.113:0/228427874 shutdown_connections 2024-03-04T08:29:03.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.473+0000 7f69c3e00640 1 -- 172.21.15.113:0/228427874 wait complete. 2024-03-04T08:29:03.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c3e00640 1 Processor -- start 2024-03-04T08:29:03.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c3e00640 1 -- start start 2024-03-04T08:29:03.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 0x7f69bc15ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:03.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69bc0c2d80 0x7f69bc1644a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:03.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69bc14e4c0 0x7f69bc140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc071500 con 0x7f69bc0c22e0 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c3e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc071220 con 0x7f69bc0c2d80 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69bc14e8a0 con 0x7f69bc14e4c0 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 0x7f69bc15ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 0x7f69bc15ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41756/0 (socket says 172.21.15.113:41756) 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c1400640 1 -- 172.21.15.113:0/2287672440 learned_addr learned my addr 172.21.15.113:0/2287672440 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c0a00640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69bc0c2d80 0x7f69bc1644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c1e00640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69bc14e4c0 0x7f69bc140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c1400640 1 -- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69bc14e4c0 msgr2=0x7f69bc140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c1400640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69bc14e4c0 0x7f69bc140980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c1400640 1 -- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69bc0c2d80 msgr2=0x7f69bc1644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c1400640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69bc0c2d80 0x7f69bc1644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:03.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c1400640 1 -- 172.21.15.113:0/2287672440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc1604e0 con 0x7f69bc0c22e0 2024-03-04T08:29:03.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c1400640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 0x7f69bc15ffa0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f69b000e8a0 tx=0x7f69b000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:03.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69b9600640 1 -- 172.21.15.113:0/2287672440 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0017070 con 0x7f69bc0c22e0 2024-03-04T08:29:03.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69b9600640 1 -- 172.21.15.113:0/2287672440 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b000c940 con 0x7f69bc0c22e0 2024-03-04T08:29:03.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69b9600640 1 -- 172.21.15.113:0/2287672440 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b000ccf0 con 0x7f69bc0c22e0 2024-03-04T08:29:03.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c3e00640 1 -- 172.21.15.113:0/2287672440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69bc1f13a0 con 0x7f69bc0c22e0 2024-03-04T08:29:03.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.477+0000 7f69c3e00640 1 -- 172.21.15.113:0/2287672440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69bc0cb5c0 con 0x7f69bc0c22e0 2024-03-04T08:29:03.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.481+0000 7f69c3e00640 1 -- 172.21.15.113:0/2287672440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69bc075c00 con 0x7f69bc0c22e0 2024-03-04T08:29:03.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.481+0000 7f69b9600640 1 -- 172.21.15.113:0/2287672440 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69b001a070 con 0x7f69bc0c22e0 2024-03-04T08:29:03.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.481+0000 7f69b9600640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f699c066f90 0x7f699c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:03.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.481+0000 7f69b9600640 1 -- 172.21.15.113:0/2287672440 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f69b0085d20 con 0x7f69bc0c22e0 2024-03-04T08:29:03.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.481+0000 7f69c0a00640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f699c066f90 0x7f699c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:03.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.481+0000 7f69c0a00640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f699c066f90 0x7f699c069450 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f69a80b2000 tx=0x7f69a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:03.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.485+0000 7f69b9600640 1 -- 172.21.15.113:0/2287672440 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f69b0055b70 con 0x7f69bc0c22e0 2024-03-04T08:29:03.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 -- 172.21.15.113:0/2287672440 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69bc155190 con 0x7f69bc0c22e0 2024-03-04T08:29:03.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69b9600640 1 -- 172.21.15.113:0/2287672440 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f69b0059c10 con 0x7f69bc0c22e0 2024-03-04T08:29:03.671 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:03.671 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:03.671 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:03.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 -- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f699c066f90 msgr2=0x7f699c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:03.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f699c066f90 0x7f699c069450 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f69a80b2000 tx=0x7f69a8009040 comp rx=0 tx=0).stop 2024-03-04T08:29:03.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 -- 172.21.15.113:0/2287672440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 msgr2=0x7f69bc15ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:03.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 0x7f69bc15ffa0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f69b000e8a0 tx=0x7f69b000eca0 comp rx=0 tx=0).stop 2024-03-04T08:29:03.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 -- 172.21.15.113:0/2287672440 shutdown_connections 2024-03-04T08:29:03.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f69bc14e4c0 0x7f69bc140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:03.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69bc0c2d80 0x7f69bc1644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:03.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f699c066f90 0x7f699c069450 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:03.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 --2- 172.21.15.113:0/2287672440 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f69bc0c22e0 0x7f69bc15ffa0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:03.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 -- 172.21.15.113:0/2287672440 >> 172.21.15.113:0/2287672440 conn(0x7f69bc06e820 msgr2=0x7f69bc14c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:03.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 -- 172.21.15.113:0/2287672440 shutdown_connections 2024-03-04T08:29:03.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:03.673+0000 7f69c3e00640 1 -- 172.21.15.113:0/2287672440 wait complete. 2024-03-04T08:29:03.692 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:05.692 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.837+0000 7f0bf7000640 1 Processor -- start 2024-03-04T08:29:05.838 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.837+0000 7f0bf7000640 1 -- start start 2024-03-04T08:29:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0bf7000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 0x7f0bf0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0bf7000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf01591f0 con 0x7f0bf01527f0 2024-03-04T08:29:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0bf7000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf01594d0 con 0x7f0bf0152bd0 2024-03-04T08:29:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0bf7000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0071df0 con 0x7f0bf00c8200 2024-03-04T08:29:05.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0befe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0bf01527f0 0x7f0bf0072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56188/0 (socket says 172.21.15.113:56188) 2024-03-04T08:29:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0befe00640 1 -- 172.21.15.113:0/788116693 learned_addr learned my addr 172.21.15.113:0/788116693 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0bef400640 1 --2- 172.21.15.113:0/788116693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 0x7f0bf0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0beea00640 1 -- 172.21.15.113:0/788116693 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266687217 0 0) 0x7f0bf01591f0 con 0x7f0bf01527f0 2024-03-04T08:29:05.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0beea00640 1 -- 172.21.15.113:0/788116693 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bd4003530 con 0x7f0bf01527f0 2024-03-04T08:29:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0beea00640 1 -- 172.21.15.113:0/788116693 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766101741 0 0) 0x7f0bf01594d0 con 0x7f0bf0152bd0 2024-03-04T08:29:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0bef400640 1 -- 172.21.15.113:0/788116693 >> v1:172.21.15.136:6789/0 conn(0x7f0bf0152bd0 legacy=0x7f0bf01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0bef400640 1 -- 172.21.15.113:0/788116693 >> v1:172.21.15.113:6789/0 conn(0x7f0bf01527f0 legacy=0x7f0bf0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:05.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.841+0000 7f0bef400640 1 -- 172.21.15.113:0/788116693 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf00717a0 con 0x7f0bf00c8200 2024-03-04T08:29:05.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0bef400640 1 --2- 172.21.15.113:0/788116693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 0x7f0bf0070d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0be00097f0 tx=0x7f0be001c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3be3184085445d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:05.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0beea00640 1 -- 172.21.15.113:0/788116693 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00b0070 con 0x7f0bf00c8200 2024-03-04T08:29:05.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0beea00640 1 -- 172.21.15.113:0/788116693 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be00ab400 con 0x7f0bf00c8200 2024-03-04T08:29:05.843 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0beea00640 1 -- 172.21.15.113:0/788116693 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8001680 con 0x7f0bf00c8200 2024-03-04T08:29:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0bf7000640 1 -- 172.21.15.113:0/788116693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 msgr2=0x7f0bf0070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0bf7000640 1 --2- 172.21.15.113:0/788116693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 0x7f0bf0070d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0be00097f0 tx=0x7f0be001c320 comp rx=0 tx=0).stop 2024-03-04T08:29:05.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0bf7000640 1 -- 172.21.15.113:0/788116693 shutdown_connections 2024-03-04T08:29:05.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0bf7000640 1 --2- 172.21.15.113:0/788116693 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 0x7f0bf0070d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:05.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0bf7000640 1 -- 172.21.15.113:0/788116693 >> 172.21.15.113:0/788116693 conn(0x7f0bf006e820 msgr2=0x7f0bf006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:05.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0bf7000640 1 -- 172.21.15.113:0/788116693 shutdown_connections 2024-03-04T08:29:05.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0bf7000640 1 -- 172.21.15.113:0/788116693 wait complete. 2024-03-04T08:29:05.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0bf7000640 1 Processor -- start 2024-03-04T08:29:05.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.845+0000 7f0bf7000640 1 -- start start 2024-03-04T08:29:05.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf7000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 0x7f0bf01585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:05.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf7000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bf01527f0 0x7f0bf01f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:05.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf7000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf0152bd0 0x7f0bf014b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:05.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf7000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0166500 con 0x7f0bf0152bd0 2024-03-04T08:29:05.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf7000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0166380 con 0x7f0bf01527f0 2024-03-04T08:29:05.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf7000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0166680 con 0x7f0bf00c8200 2024-03-04T08:29:05.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf0152bd0 0x7f0bf014b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:05.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0befe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 0x7f0bf01585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf0152bd0 0x7f0bf014b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41760/0 (socket says 172.21.15.113:41760) 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf5000640 1 -- 172.21.15.113:0/477794091 learned_addr learned my addr 172.21.15.113:0/477794091 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bef400640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bf01527f0 0x7f0bf01f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf5000640 1 -- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 msgr2=0x7f0bf01585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf5000640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 0x7f0bf01585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf5000640 1 -- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bf01527f0 msgr2=0x7f0bf01f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf5000640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bf01527f0 0x7f0bf01f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf5000640 1 -- 172.21.15.113:0/477794091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf01f3940 con 0x7f0bf0152bd0 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf5000640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf0152bd0 0x7f0bf014b5b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f0be800c7b0 tx=0x7f0be800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0be7e00640 1 -- 172.21.15.113:0/477794091 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be800ce60 con 0x7f0bf0152bd0 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0be7e00640 1 -- 172.21.15.113:0/477794091 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be8003db0 con 0x7f0bf0152bd0 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf7000640 1 -- 172.21.15.113:0/477794091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf01f7950 con 0x7f0bf0152bd0 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0be7e00640 1 -- 172.21.15.113:0/477794091 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be80052b0 con 0x7f0bf0152bd0 2024-03-04T08:29:05.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf7000640 1 -- 172.21.15.113:0/477794091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf01f3ec0 con 0x7f0bf0152bd0 2024-03-04T08:29:05.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0bf7000640 1 -- 172.21.15.113:0/477794091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf0075c00 con 0x7f0bf0152bd0 2024-03-04T08:29:05.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.849+0000 7f0be7e00640 1 -- 172.21.15.113:0/477794091 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0be8005450 con 0x7f0bf0152bd0 2024-03-04T08:29:05.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.853+0000 7f0be7e00640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bcc066f60 0x7f0bcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:05.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.853+0000 7f0be7e00640 1 -- 172.21.15.113:0/477794091 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f0be80858d0 con 0x7f0bf0152bd0 2024-03-04T08:29:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.853+0000 7f0befe00640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bcc066f60 0x7f0bcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.853+0000 7f0befe00640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bcc066f60 0x7f0bcc069420 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8001ec0 tx=0x7f0bd8001b30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:05.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:05.857+0000 7f0be7e00640 1 -- 172.21.15.113:0/477794091 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0be8055720 con 0x7f0bf0152bd0 2024-03-04T08:29:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.057+0000 7f0bf7000640 1 -- 172.21.15.113:0/477794091 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bf00c1540 con 0x7f0bf0152bd0 2024-03-04T08:29:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.057+0000 7f0be7e00640 1 -- 172.21.15.113:0/477794091 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f0be80597c0 con 0x7f0bf0152bd0 2024-03-04T08:29:06.057 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:06.058 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 -- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bcc066f60 msgr2=0x7f0bcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bcc066f60 0x7f0bcc069420 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8001ec0 tx=0x7f0bd8001b30 comp rx=0 tx=0).stop 2024-03-04T08:29:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 -- 172.21.15.113:0/477794091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf0152bd0 msgr2=0x7f0bf014b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf0152bd0 0x7f0bf014b5b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f0be800c7b0 tx=0x7f0be800cc70 comp rx=0 tx=0).stop 2024-03-04T08:29:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 -- 172.21.15.113:0/477794091 shutdown_connections 2024-03-04T08:29:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f0bcc066f60 0x7f0bcc069420 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0bf0152bd0 0x7f0bf014b5b0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0bf01527f0 0x7f0bf01f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 --2- 172.21.15.113:0/477794091 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0bf00c8200 0x7f0bf01585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 -- 172.21.15.113:0/477794091 >> 172.21.15.113:0/477794091 conn(0x7f0bf006e820 msgr2=0x7f0bf0150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:06.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 -- 172.21.15.113:0/477794091 shutdown_connections 2024-03-04T08:29:06.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:06.061+0000 7f0bf7000640 1 -- 172.21.15.113:0/477794091 wait complete. 2024-03-04T08:29:06.079 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:08.080 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:08.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.225+0000 7fcceba00640 1 Processor -- start 2024-03-04T08:29:08.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.225+0000 7fcceba00640 1 -- start start 2024-03-04T08:29:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fcceba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcce414e4c0 0x7fcce4072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fcceba00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fcce40c22e0 0x7fcce40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:08.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fcceba00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce40c27f0 con 0x7fcce40c2d80 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fcceba00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce415ad60 con 0x7fcce40c22e0 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fcceba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40c2a90 con 0x7fcce414e4c0 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fcce9a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcce40c2d80 0x7fcce415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47232/0 (socket says 172.21.15.113:47232) 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fcce9a00640 1 -- 172.21.15.113:0/3097833788 learned_addr learned my addr 172.21.15.113:0/3097833788 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fccdbe00640 1 --2- 172.21.15.113:0/3097833788 >> v2:172.21.15.136:3300/0 conn(0x7fcce40c22e0 0x7fcce40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fcce9000640 1 --2- 172.21.15.113:0/3097833788 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcce414e4c0 0x7fcce4072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fcce3e00640 1 -- 172.21.15.113:0/3097833788 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987159843 0 0) 0x7fcce40c27f0 con 0x7fcce40c2d80 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fcce3e00640 1 -- 172.21.15.113:0/3097833788 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccc8003530 con 0x7fcce40c2d80 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fccdbe00640 1 -- 172.21.15.113:0/3097833788 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcce414e4c0 msgr2=0x7fcce4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fccdbe00640 1 --2- 172.21.15.113:0/3097833788 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcce414e4c0 0x7fcce4072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:08.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fccdbe00640 1 -- 172.21.15.113:0/3097833788 >> v1:172.21.15.113:6789/0 conn(0x7fcce40c2d80 legacy=0x7fcce415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:08.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.229+0000 7fccdbe00640 1 -- 172.21.15.113:0/3097833788 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce4070c60 con 0x7fcce40c22e0 2024-03-04T08:29:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fccdbe00640 1 --2- 172.21.15.113:0/3097833788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcce40c22e0 0x7fcce40731c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0097c0 tx=0x7fcccc01ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17bb1b591b14ccb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcce3e00640 1 -- 172.21.15.113:0/3097833788 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0b5070 con 0x7fcce40c22e0 2024-03-04T08:29:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcce3e00640 1 -- 172.21.15.113:0/3097833788 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc0a8990 con 0x7fcce40c22e0 2024-03-04T08:29:08.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcce3e00640 1 -- 172.21.15.113:0/3097833788 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0a8d40 con 0x7fcce40c22e0 2024-03-04T08:29:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcceba00640 1 -- 172.21.15.113:0/3097833788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcce40c22e0 msgr2=0x7fcce40731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcceba00640 1 --2- 172.21.15.113:0/3097833788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcce40c22e0 0x7fcce40731c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0097c0 tx=0x7fcccc01ec90 comp rx=0 tx=0).stop 2024-03-04T08:29:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcceba00640 1 -- 172.21.15.113:0/3097833788 shutdown_connections 2024-03-04T08:29:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcceba00640 1 --2- 172.21.15.113:0/3097833788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcce40c22e0 0x7fcce40731c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcceba00640 1 --2- 172.21.15.113:0/3097833788 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcce414e4c0 0x7fcce4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:08.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcceba00640 1 -- 172.21.15.113:0/3097833788 >> 172.21.15.113:0/3097833788 conn(0x7fcce406e820 msgr2=0x7fcce406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:08.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcceba00640 1 -- 172.21.15.113:0/3097833788 shutdown_connections 2024-03-04T08:29:08.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcceba00640 1 -- 172.21.15.113:0/3097833788 wait complete. 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcceba00640 1 Processor -- start 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.233+0000 7fcceba00640 1 -- start start 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcceba00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce40c22e0 0x7fcce40af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcceba00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcce40c2d80 0x7fcce415a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcceba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcce414e4c0 0x7fcce41594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcceba00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4071500 con 0x7fcce40c22e0 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcceba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4071220 con 0x7fcce414e4c0 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcceba00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce414e8a0 con 0x7fcce40c2d80 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce40c22e0 0x7fcce40af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce40c22e0 0x7fcce40af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58088/0 (socket says 172.21.15.113:58088) 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce9000640 1 -- 172.21.15.113:0/952532866 learned_addr learned my addr 172.21.15.113:0/952532866 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce9a00640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcce414e4c0 0x7fcce41594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:08.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fccdbe00640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcce40c2d80 0x7fcce415a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce9000640 1 -- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcce40c2d80 msgr2=0x7fcce415a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce9000640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcce40c2d80 0x7fcce415a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce9000640 1 -- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcce414e4c0 msgr2=0x7fcce41594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce9000640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcce414e4c0 0x7fcce41594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce9000640 1 -- 172.21.15.113:0/952532866 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce4159f20 con 0x7fcce40c22e0 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce9000640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce40c22e0 0x7fcce40af220 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fccd400e870 tx=0x7fccd400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce1600640 1 -- 172.21.15.113:0/952532866 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd4017070 con 0x7fcce40c22e0 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce1600640 1 -- 172.21.15.113:0/952532866 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd400c940 con 0x7fcce40c22e0 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcceba00640 1 -- 172.21.15.113:0/952532866 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce41e1ba0 con 0x7fcce40c22e0 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce1600640 1 -- 172.21.15.113:0/952532866 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd400ccf0 con 0x7fcce40c22e0 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcceba00640 1 -- 172.21.15.113:0/952532866 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce41e1e40 con 0x7fcce40c22e0 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcceba00640 1 -- 172.21.15.113:0/952532866 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce4075c00 con 0x7fcce40c22e0 2024-03-04T08:29:08.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.237+0000 7fcce1600640 1 -- 172.21.15.113:0/952532866 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fccd401a070 con 0x7fcce40c22e0 2024-03-04T08:29:08.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.241+0000 7fcce1600640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccc0066f60 0x7fccc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:08.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.241+0000 7fcce1600640 1 -- 172.21.15.113:0/952532866 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fccd4085970 con 0x7fcce40c22e0 2024-03-04T08:29:08.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.241+0000 7fccdbe00640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccc0066f60 0x7fccc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:08.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.241+0000 7fccdbe00640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccc0066f60 0x7fccc0069420 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0b2000 tx=0x7fcccc0aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:08.243 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.245+0000 7fcce1600640 1 -- 172.21.15.113:0/952532866 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fccd4055ed0 con 0x7fcce40c22e0 2024-03-04T08:29:08.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 -- 172.21.15.113:0/952532866 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcce4155190 con 0x7fcce40c22e0 2024-03-04T08:29:08.431 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcce1600640 1 -- 172.21.15.113:0/952532866 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fccd40557d0 con 0x7fcce40c22e0 2024-03-04T08:29:08.431 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:08.431 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:08.431 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:08.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 -- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccc0066f60 msgr2=0x7fccc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:08.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccc0066f60 0x7fccc0069420 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0b2000 tx=0x7fcccc0aa070 comp rx=0 tx=0).stop 2024-03-04T08:29:08.435 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 -- 172.21.15.113:0/952532866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce40c22e0 msgr2=0x7fcce40af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:08.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce40c22e0 0x7fcce40af220 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fccd400e870 tx=0x7fccd400ec70 comp rx=0 tx=0).stop 2024-03-04T08:29:08.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 -- 172.21.15.113:0/952532866 shutdown_connections 2024-03-04T08:29:08.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcce414e4c0 0x7fcce41594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:08.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcce40c2d80 0x7fcce415a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:08.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccc0066f60 0x7fccc0069420 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:08.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 --2- 172.21.15.113:0/952532866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce40c22e0 0x7fcce40af220 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:08.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 -- 172.21.15.113:0/952532866 >> 172.21.15.113:0/952532866 conn(0x7fcce406e820 msgr2=0x7fcce40c8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:08.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 -- 172.21.15.113:0/952532866 shutdown_connections 2024-03-04T08:29:08.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:08.433+0000 7fcceba00640 1 -- 172.21.15.113:0/952532866 wait complete. 2024-03-04T08:29:08.452 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:10.453 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:10.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2fe00640 1 Processor -- start 2024-03-04T08:29:10.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2fe00640 1 -- start start 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a2814e4c0 0x7f4a28072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a280c22e0 0x7f4a280731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2fe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4a280c2d80 0x7f4a2815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2fe00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2815ad60 con 0x7f4a280c22e0 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2fe00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a280c27f0 con 0x7f4a280c2d80 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a280c2a90 con 0x7f4a2814e4c0 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a280c22e0 0x7f4a280731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2ca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a280c22e0 0x7f4a280731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58090/0 (socket says 172.21.15.113:58090) 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2ca00640 1 -- 172.21.15.113:0/2404064019 learned_addr learned my addr 172.21.15.113:0/2404064019 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2d400640 1 --2- 172.21.15.113:0/2404064019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a2814e4c0 0x7f4a28072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2de00640 1 --2- 172.21.15.113:0/2404064019 >> v2:172.21.15.136:3300/0 conn(0x7f4a280c2d80 0x7f4a2815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:10.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2ca00640 1 -- 172.21.15.113:0/2404064019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a2814e4c0 msgr2=0x7f4a28072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:10.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2ca00640 1 --2- 172.21.15.113:0/2404064019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a2814e4c0 0x7f4a28072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2ca00640 1 -- 172.21.15.113:0/2404064019 >> v2:172.21.15.136:3300/0 conn(0x7f4a280c2d80 msgr2=0x7f4a2815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:10.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2ca00640 1 --2- 172.21.15.113:0/2404064019 >> v2:172.21.15.136:3300/0 conn(0x7f4a280c2d80 0x7f4a2815b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2ca00640 1 -- 172.21.15.113:0/2404064019 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28070c60 con 0x7f4a280c22e0 2024-03-04T08:29:10.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.601+0000 7f4a2de00640 1 --2- 172.21.15.113:0/2404064019 >> v2:172.21.15.136:3300/0 conn(0x7f4a280c2d80 0x7f4a2815b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:10.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a2ca00640 1 --2- 172.21.15.113:0/2404064019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c22e0 0x7f4a280731c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f4a280c2700 tx=0x7f4a1401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f714fbd2a8f19393 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a27e00640 1 -- 172.21.15.113:0/2404064019 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140b5070 con 0x7f4a280c22e0 2024-03-04T08:29:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a27e00640 1 -- 172.21.15.113:0/2404064019 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a140a89d0 con 0x7f4a280c22e0 2024-03-04T08:29:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a27e00640 1 -- 172.21.15.113:0/2404064019 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140a8da0 con 0x7f4a280c22e0 2024-03-04T08:29:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a2fe00640 1 -- 172.21.15.113:0/2404064019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c22e0 msgr2=0x7f4a280731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:10.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a2fe00640 1 --2- 172.21.15.113:0/2404064019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c22e0 0x7f4a280731c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f4a280c2700 tx=0x7f4a1401ed10 comp rx=0 tx=0).stop 2024-03-04T08:29:10.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a2fe00640 1 -- 172.21.15.113:0/2404064019 shutdown_connections 2024-03-04T08:29:10.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a2fe00640 1 --2- 172.21.15.113:0/2404064019 >> v2:172.21.15.136:3300/0 conn(0x7f4a280c2d80 0x7f4a2815b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a2fe00640 1 --2- 172.21.15.113:0/2404064019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c22e0 0x7f4a280731c0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a2fe00640 1 --2- 172.21.15.113:0/2404064019 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a2814e4c0 0x7f4a28072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a2fe00640 1 -- 172.21.15.113:0/2404064019 >> 172.21.15.113:0/2404064019 conn(0x7f4a2806e820 msgr2=0x7f4a2806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:10.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a2fe00640 1 -- 172.21.15.113:0/2404064019 shutdown_connections 2024-03-04T08:29:10.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.605+0000 7f4a2fe00640 1 -- 172.21.15.113:0/2404064019 wait complete. 2024-03-04T08:29:10.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 Processor -- start 2024-03-04T08:29:10.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 -- start start 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a280c22e0 0x7f4a281e5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c2d80 0x7f4a280c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a2814e4c0 0x7f4a281f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a28071500 con 0x7f4a280c2d80 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a28071220 con 0x7f4a280c22e0 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2814e8a0 con 0x7f4a2814e4c0 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c2d80 0x7f4a280c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2ca00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c2d80 0x7f4a280c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58092/0 (socket says 172.21.15.113:58092) 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2ca00640 1 -- 172.21.15.113:0/831431623 learned_addr learned my addr 172.21.15.113:0/831431623 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2d400640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a280c22e0 0x7f4a281e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2de00640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a2814e4c0 0x7f4a281f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2ca00640 1 -- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a2814e4c0 msgr2=0x7f4a281f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2ca00640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a2814e4c0 0x7f4a281f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2ca00640 1 -- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a280c22e0 msgr2=0x7f4a281e5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:10.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2ca00640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a280c22e0 0x7f4a281e5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2ca00640 1 -- 172.21.15.113:0/831431623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28153e80 con 0x7f4a280c2d80 2024-03-04T08:29:10.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2d400640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a280c22e0 0x7f4a281e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:10.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2ca00640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c2d80 0x7f4a280c5a90 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f4a1401d3c0 tx=0x7f4a1401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:10.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a25600640 1 -- 172.21.15.113:0/831431623 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140b5070 con 0x7f4a280c2d80 2024-03-04T08:29:10.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a25600640 1 -- 172.21.15.113:0/831431623 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a140b0a70 con 0x7f4a280c2d80 2024-03-04T08:29:10.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 -- 172.21.15.113:0/831431623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a28158ab0 con 0x7f4a280c2d80 2024-03-04T08:29:10.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a25600640 1 -- 172.21.15.113:0/831431623 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a140b0e60 con 0x7f4a280c2d80 2024-03-04T08:29:10.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 -- 172.21.15.113:0/831431623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a281540f0 con 0x7f4a280c2d80 2024-03-04T08:29:10.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.609+0000 7f4a2fe00640 1 -- 172.21.15.113:0/831431623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a28075c00 con 0x7f4a280c2d80 2024-03-04T08:29:10.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.613+0000 7f4a25600640 1 -- 172.21.15.113:0/831431623 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a140c2060 con 0x7f4a280c2d80 2024-03-04T08:29:10.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.613+0000 7f4a25600640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4a08066f60 0x7f4a08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:10.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.613+0000 7f4a25600640 1 -- 172.21.15.113:0/831431623 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f4a14122d10 con 0x7f4a280c2d80 2024-03-04T08:29:10.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.613+0000 7f4a2d400640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4a08066f60 0x7f4a08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:10.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.613+0000 7f4a2d400640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4a08066f60 0x7f4a08069420 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00b970 tx=0x7f4a1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:10.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.617+0000 7f4a25600640 1 -- 172.21.15.113:0/831431623 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4a14128020 con 0x7f4a280c2d80 2024-03-04T08:29:10.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 -- 172.21.15.113:0/831431623 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a28155190 con 0x7f4a280c2d80 2024-03-04T08:29:10.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a25600640 1 -- 172.21.15.113:0/831431623 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f4a14018870 con 0x7f4a280c2d80 2024-03-04T08:29:10.804 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:10.804 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:10.804 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:10.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 -- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4a08066f60 msgr2=0x7f4a08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4a08066f60 0x7f4a08069420 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00b970 tx=0x7f4a1c008040 comp rx=0 tx=0).stop 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 -- 172.21.15.113:0/831431623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c2d80 msgr2=0x7f4a280c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c2d80 0x7f4a280c5a90 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f4a1401d3c0 tx=0x7f4a1401bab0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 -- 172.21.15.113:0/831431623 shutdown_connections 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a2814e4c0 0x7f4a281f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4a08066f60 0x7f4a08069420 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a280c2d80 0x7f4a280c5a90 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 --2- 172.21.15.113:0/831431623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a280c22e0 0x7f4a281e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 -- 172.21.15.113:0/831431623 >> 172.21.15.113:0/831431623 conn(0x7f4a2806e820 msgr2=0x7f4a280c6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 -- 172.21.15.113:0/831431623 shutdown_connections 2024-03-04T08:29:10.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:10.805+0000 7f4a2fe00640 1 -- 172.21.15.113:0/831431623 wait complete. 2024-03-04T08:29:10.825 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:12.825 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d9000640 1 Processor -- start 2024-03-04T08:29:12.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d9000640 1 -- start start 2024-03-04T08:29:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d9000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d41527f0 0x7f62d4070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d9000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f62d4152bd0 0x7f62d41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d9000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d41591f0 con 0x7f62d40c8200 2024-03-04T08:29:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d9000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d4071df0 con 0x7f62d4152bd0 2024-03-04T08:29:12.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d9000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d41597b0 con 0x7f62d41527f0 2024-03-04T08:29:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d41527f0 0x7f62d4070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d2800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f62d4152bd0 0x7f62d41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d2800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f62d4152bd0 0x7f62d41587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:34352/0 (socket says 172.21.15.113:34352) 2024-03-04T08:29:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d2800640 1 -- 172.21.15.113:0/2460601437 learned_addr learned my addr 172.21.15.113:0/2460601437 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d1e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d41527f0 0x7f62d4070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3301/0 says I am v2:172.21.15.113:37786/0 (socket says 172.21.15.113:37786) 2024-03-04T08:29:12.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62c9400640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f62d40c8200 0x7f62d4072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:54440/0 (socket says 172.21.15.113:54440) 2024-03-04T08:29:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d1e00640 1 -- 172.21.15.113:0/2460601437 >> v2:172.21.15.136:3300/0 conn(0x7f62d4152bd0 msgr2=0x7f62d41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d1e00640 1 --2- 172.21.15.113:0/2460601437 >> v2:172.21.15.136:3300/0 conn(0x7f62d4152bd0 0x7f62d41587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d1e00640 1 -- 172.21.15.113:0/2460601437 >> v1:172.21.15.136:6789/0 conn(0x7f62d40c8200 legacy=0x7f62d4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d1e00640 1 -- 172.21.15.113:0/2460601437 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d41594d0 con 0x7f62d41527f0 2024-03-04T08:29:12.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.973+0000 7f62d2800640 1 --2- 172.21.15.113:0/2460601437 >> v2:172.21.15.136:3300/0 conn(0x7f62d4152bd0 0x7f62d41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:29:12.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d1e00640 1 --2- 172.21.15.113:0/2460601437 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d41527f0 0x7f62d4070d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f62c40097c0 tx=0x7f62c401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a38e54c1206c8b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d1400640 1 -- 172.21.15.113:0/2460601437 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c40b1070 con 0x7f62d41527f0 2024-03-04T08:29:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d1400640 1 -- 172.21.15.113:0/2460601437 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c40ac410 con 0x7f62d41527f0 2024-03-04T08:29:12.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d1400640 1 -- 172.21.15.113:0/2460601437 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c40ac7e0 con 0x7f62d41527f0 2024-03-04T08:29:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d9000640 1 -- 172.21.15.113:0/2460601437 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d41527f0 msgr2=0x7f62d4070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d9000640 1 --2- 172.21.15.113:0/2460601437 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d41527f0 0x7f62d4070d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f62c40097c0 tx=0x7f62c401c4a0 comp rx=0 tx=0).stop 2024-03-04T08:29:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d9000640 1 -- 172.21.15.113:0/2460601437 shutdown_connections 2024-03-04T08:29:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d9000640 1 --2- 172.21.15.113:0/2460601437 >> v2:172.21.15.136:3300/0 conn(0x7f62d4152bd0 0x7f62d41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d9000640 1 --2- 172.21.15.113:0/2460601437 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d41527f0 0x7f62d4070d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d9000640 1 -- 172.21.15.113:0/2460601437 >> 172.21.15.113:0/2460601437 conn(0x7f62d406e820 msgr2=0x7f62d406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d9000640 1 -- 172.21.15.113:0/2460601437 shutdown_connections 2024-03-04T08:29:12.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.977+0000 7f62d9000640 1 -- 172.21.15.113:0/2460601437 wait complete. 2024-03-04T08:29:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d9000640 1 Processor -- start 2024-03-04T08:29:12.979 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d9000640 1 -- start start 2024-03-04T08:29:12.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d9000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d40c8200 0x7f62d414d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d9000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62d41527f0 0x7f62d4148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d9000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d4152bd0 0x7f62d414e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d9000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d41654f0 con 0x7f62d41527f0 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d9000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d4165370 con 0x7f62d40c8200 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d9000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d4165670 con 0x7f62d4152bd0 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62c9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62d41527f0 0x7f62d4148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62c9400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62d41527f0 0x7f62d4148990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58102/0 (socket says 172.21.15.113:58102) 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62c9400640 1 -- 172.21.15.113:0/711097958 learned_addr learned my addr 172.21.15.113:0/711097958 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d1e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d40c8200 0x7f62d414d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d2800640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d4152bd0 0x7f62d414e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62c9400640 1 -- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d4152bd0 msgr2=0x7f62d414e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62c9400640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d4152bd0 0x7f62d414e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62c9400640 1 -- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d40c8200 msgr2=0x7f62d414d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:12.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62c9400640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d40c8200 0x7f62d414d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:12.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62c9400640 1 -- 172.21.15.113:0/711097958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d41f6290 con 0x7f62d41527f0 2024-03-04T08:29:12.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62c9400640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62d41527f0 0x7f62d4148990 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00c430 tx=0x7f62bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:12.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62caa00640 1 -- 172.21.15.113:0/711097958 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0074e0 con 0x7f62d41527f0 2024-03-04T08:29:12.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62caa00640 1 -- 172.21.15.113:0/711097958 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc007680 con 0x7f62d41527f0 2024-03-04T08:29:12.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62caa00640 1 -- 172.21.15.113:0/711097958 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc004a30 con 0x7f62d41527f0 2024-03-04T08:29:12.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d9000640 1 -- 172.21.15.113:0/711097958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d4151da0 con 0x7f62d41527f0 2024-03-04T08:29:12.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.981+0000 7f62d9000640 1 -- 172.21.15.113:0/711097958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d414de40 con 0x7f62d41527f0 2024-03-04T08:29:12.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.985+0000 7f62d9000640 1 -- 172.21.15.113:0/711097958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d4075c00 con 0x7f62d41527f0 2024-03-04T08:29:12.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.985+0000 7f62caa00640 1 -- 172.21.15.113:0/711097958 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62bc01a020 con 0x7f62d41527f0 2024-03-04T08:29:12.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.985+0000 7f62caa00640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f62b0066f60 0x7f62b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:12.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.985+0000 7f62caa00640 1 -- 172.21.15.113:0/711097958 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f62bc085150 con 0x7f62d41527f0 2024-03-04T08:29:12.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.985+0000 7f62d1e00640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f62b0066f60 0x7f62b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:12.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.985+0000 7f62d1e00640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f62b0066f60 0x7f62b0069420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f62c40ae000 tx=0x7f62c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:12.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:12.989+0000 7f62caa00640 1 -- 172.21.15.113:0/711097958 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f62d41f6290 con 0x7f62d41527f0 2024-03-04T08:29:13.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.177+0000 7f62d9000640 1 -- 172.21.15.113:0/711097958 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62d40b24a0 con 0x7f62d41527f0 2024-03-04T08:29:13.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.177+0000 7f62caa00640 1 -- 172.21.15.113:0/711097958 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f62bc0550c0 con 0x7f62d41527f0 2024-03-04T08:29:13.177 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:13.177 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:13.177 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 -- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f62b0066f60 msgr2=0x7f62b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f62b0066f60 0x7f62b0069420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f62c40ae000 tx=0x7f62c4008040 comp rx=0 tx=0).stop 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 -- 172.21.15.113:0/711097958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62d41527f0 msgr2=0x7f62d4148990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62d41527f0 0x7f62d4148990 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00c430 tx=0x7f62bc00c8f0 comp rx=0 tx=0).stop 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 -- 172.21.15.113:0/711097958 shutdown_connections 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f62d4152bd0 0x7f62d414e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f62b0066f60 0x7f62b0069420 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62d41527f0 0x7f62d4148990 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 --2- 172.21.15.113:0/711097958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62d40c8200 0x7f62d414d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 -- 172.21.15.113:0/711097958 >> 172.21.15.113:0/711097958 conn(0x7f62d406e820 msgr2=0x7f62d4156a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 -- 172.21.15.113:0/711097958 shutdown_connections 2024-03-04T08:29:13.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:13.181+0000 7f62d9000640 1 -- 172.21.15.113:0/711097958 wait complete. 2024-03-04T08:29:13.198 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:29:13.199 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:29:15.200 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:15.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.349+0000 7f7c12600640 1 Processor -- start 2024-03-04T08:29:15.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.349+0000 7f7c12600640 1 -- start start 2024-03-04T08:29:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.349+0000 7f7c12600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7c0c15aa40 0x7f7c0c154800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.349+0000 7f7c12600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0c15aed0 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.349+0000 7f7c12600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0c15b170 con 0x7f7c0c0ccf20 2024-03-04T08:29:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.349+0000 7f7c12600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0c15b410 con 0x7f7c0c15aa40 2024-03-04T08:29:15.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.349+0000 7f7c0b400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f7c0c0ccb40 0x7f7c0c0c7370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:47244/0 (socket says 172.21.15.113:47244) 2024-03-04T08:29:15.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.353+0000 7f7c0b400640 1 -- 172.21.15.113:0/1075938145 learned_addr learned my addr 172.21.15.113:0/1075938145 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:15.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.353+0000 7f7c02a00640 1 --2- 172.21.15.113:0/1075938145 >> v2:172.21.15.136:3300/0 conn(0x7f7c0c15aa40 0x7f7c0c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:15.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.353+0000 7f7c0aa00640 1 -- 172.21.15.113:0/1075938145 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199817215 0 0) 0x7f7c0c15aed0 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.353+0000 7f7c0aa00640 1 -- 172.21.15.113:0/1075938145 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bf0003530 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.353+0000 7f7c02a00640 1 -- 172.21.15.113:0/1075938145 >> v1:172.21.15.136:6789/0 conn(0x7f7c0c0ccf20 legacy=0x7f7c0c15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:15.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.353+0000 7f7c02a00640 1 -- 172.21.15.113:0/1075938145 >> v1:172.21.15.113:6789/0 conn(0x7f7c0c0ccb40 legacy=0x7f7c0c0c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:15.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.353+0000 7f7c02a00640 1 -- 172.21.15.113:0/1075938145 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c15b9b0 con 0x7f7c0c15aa40 2024-03-04T08:29:15.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.353+0000 7f7c02a00640 1 --2- 172.21.15.113:0/1075938145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0c15aa40 0x7f7c0c154800 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40097f0 tx=0x7f7bf401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b6737851d4175fb server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:15.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.353+0000 7f7c0aa00640 1 -- 172.21.15.113:0/1075938145 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40b5070 con 0x7f7c0c15aa40 2024-03-04T08:29:15.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c0aa00640 1 -- 172.21.15.113:0/1075938145 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf40a8990 con 0x7f7c0c15aa40 2024-03-04T08:29:15.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c0aa00640 1 -- 172.21.15.113:0/1075938145 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40a8d40 con 0x7f7c0c15aa40 2024-03-04T08:29:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 -- 172.21.15.113:0/1075938145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0c15aa40 msgr2=0x7f7c0c154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 --2- 172.21.15.113:0/1075938145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0c15aa40 0x7f7c0c154800 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40097f0 tx=0x7f7bf401ec90 comp rx=0 tx=0).stop 2024-03-04T08:29:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 -- 172.21.15.113:0/1075938145 shutdown_connections 2024-03-04T08:29:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 --2- 172.21.15.113:0/1075938145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0c15aa40 0x7f7c0c154800 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 -- 172.21.15.113:0/1075938145 >> 172.21.15.113:0/1075938145 conn(0x7f7c0c06b060 msgr2=0x7f7c0c06b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:15.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 -- 172.21.15.113:0/1075938145 shutdown_connections 2024-03-04T08:29:15.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 -- 172.21.15.113:0/1075938145 wait complete. 2024-03-04T08:29:15.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 Processor -- start 2024-03-04T08:29:15.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 -- start start 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0ccb40 0x7f7c0c0cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0c0ccf20 0x7f7c0c1483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0c14a040 0x7f7c0c0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c15ed00 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c15e0d0 con 0x7f7c0c0ccf20 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c12600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c15ee80 con 0x7f7c0c14a040 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c0b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0ccb40 0x7f7c0c0cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c0b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0ccb40 0x7f7c0c0cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:58116/0 (socket says 172.21.15.113:58116) 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c0b400640 1 -- 172.21.15.113:0/2964754820 learned_addr learned my addr 172.21.15.113:0/2964754820 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c02a00640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0c0ccf20 0x7f7c0c1483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:15.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.357+0000 7f7c0be00640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0c14a040 0x7f7c0c0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c0b400640 1 -- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0c14a040 msgr2=0x7f7c0c0cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c0b400640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0c14a040 0x7f7c0c0cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c0b400640 1 -- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0c0ccf20 msgr2=0x7f7c0c1483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c0b400640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0c0ccf20 0x7f7c0c1483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c0b400640 1 -- 172.21.15.113:0/2964754820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c14c180 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c0be00640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0c14a040 0x7f7c0c0cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c0b400640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0ccb40 0x7f7c0c0cc030 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00c4d0 tx=0x7f7bfc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c03e00640 1 -- 172.21.15.113:0/2964754820 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc017070 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c03e00640 1 -- 172.21.15.113:0/2964754820 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bfc005cf0 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c03e00640 1 -- 172.21.15.113:0/2964754820 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc007270 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c12600640 1 -- 172.21.15.113:0/2964754820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0c14c460 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c12600640 1 -- 172.21.15.113:0/2964754820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c14ca20 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c12600640 1 -- 172.21.15.113:0/2964754820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0c14cc60 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c03e00640 1 -- 172.21.15.113:0/2964754820 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7bfc01a070 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c03e00640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be8066f60 0x7f7be8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:15.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c03e00640 1 -- 172.21.15.113:0/2964754820 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f7bfc084f50 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.361+0000 7f7c02a00640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be8066f60 0x7f7be8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:15.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.365+0000 7f7c02a00640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be8066f60 0x7f7be8069420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40b2000 tx=0x7f7bf40aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:15.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.365+0000 7f7c03e00640 1 -- 172.21.15.113:0/2964754820 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7bfc054d40 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 -- 172.21.15.113:0/2964754820 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c0c075c00 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c03e00640 1 -- 172.21.15.113:0/2964754820 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f7bfc058de0 con 0x7f7c0c0ccb40 2024-03-04T08:29:15.552 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:15.552 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:15.553 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:15.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 -- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be8066f60 msgr2=0x7f7be8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:15.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be8066f60 0x7f7be8069420 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40b2000 tx=0x7f7bf40aa070 comp rx=0 tx=0).stop 2024-03-04T08:29:15.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 -- 172.21.15.113:0/2964754820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0ccb40 msgr2=0x7f7c0c0cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:15.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0ccb40 0x7f7c0c0cc030 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00c4d0 tx=0x7f7bfc00c990 comp rx=0 tx=0).stop 2024-03-04T08:29:15.554 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 -- 172.21.15.113:0/2964754820 shutdown_connections 2024-03-04T08:29:15.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7c0c14a040 0x7f7c0c0cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:15.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c0c0ccf20 0x7f7c0c1483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:15.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f7be8066f60 0x7f7be8069420 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:15.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 --2- 172.21.15.113:0/2964754820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7c0c0ccb40 0x7f7c0c0cc030 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:15.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 -- 172.21.15.113:0/2964754820 >> 172.21.15.113:0/2964754820 conn(0x7f7c0c06b060 msgr2=0x7f7c0c0c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:15.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 -- 172.21.15.113:0/2964754820 shutdown_connections 2024-03-04T08:29:15.555 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:15.553+0000 7f7c12600640 1 -- 172.21.15.113:0/2964754820 wait complete. 2024-03-04T08:29:15.573 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:16.252 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:29:16.256 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:29:17.574 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:17.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.721+0000 7ff346200640 1 Processor -- start 2024-03-04T08:29:17.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.721+0000 7ff346200640 1 -- start start 2024-03-04T08:29:17.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.721+0000 7ff346200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 0x7ff3400c1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:17.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.721+0000 7ff346200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff340154fe0 0x7ff3400c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:17.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.721+0000 7ff346200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3400c7d80 con 0x7ff3400c7700 2024-03-04T08:29:17.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.721+0000 7ff346200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34015efb0 con 0x7ff340154fe0 2024-03-04T08:29:17.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.721+0000 7ff346200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400c6b70 con 0x7ff340154c00 2024-03-04T08:29:17.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff33fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff3400c7700 0x7ff34015abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52592/0 (socket says 172.21.15.113:52592) 2024-03-04T08:29:17.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff33fe00640 1 -- 172.21.15.113:0/2620824275 learned_addr learned my addr 172.21.15.113:0/2620824275 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:17.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff336a00640 1 --2- 172.21.15.113:0/2620824275 >> v2:172.21.15.136:3300/0 conn(0x7ff340154fe0 0x7ff3400c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:17.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff33f400640 1 --2- 172.21.15.113:0/2620824275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 0x7ff3400c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:17.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff33ea00640 1 -- 172.21.15.113:0/2620824275 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137929198 0 0) 0x7ff3400c7d80 con 0x7ff3400c7700 2024-03-04T08:29:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff33ea00640 1 -- 172.21.15.113:0/2620824275 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff324003530 con 0x7ff3400c7700 2024-03-04T08:29:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff33ea00640 1 -- 172.21.15.113:0/2620824275 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1067424738 0 0) 0x7ff324003530 con 0x7ff3400c7700 2024-03-04T08:29:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff336a00640 1 -- 172.21.15.113:0/2620824275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 msgr2=0x7ff3400c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff336a00640 1 --2- 172.21.15.113:0/2620824275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 0x7ff3400c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff336a00640 1 -- 172.21.15.113:0/2620824275 >> v1:172.21.15.113:6789/0 conn(0x7ff3400c7700 legacy=0x7ff34015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:17.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.725+0000 7ff336a00640 1 -- 172.21.15.113:0/2620824275 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3400c8060 con 0x7ff340154fe0 2024-03-04T08:29:17.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff336a00640 1 --2- 172.21.15.113:0/2620824275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff340154fe0 0x7ff3400c1b50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff328009d90 tx=0x7ff32801c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85bb935b24a27e06 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:17.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff33ea00640 1 -- 172.21.15.113:0/2620824275 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338001680 con 0x7ff340154fe0 2024-03-04T08:29:17.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff33ea00640 1 -- 172.21.15.113:0/2620824275 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3280a6500 con 0x7ff340154fe0 2024-03-04T08:29:17.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff33ea00640 1 -- 172.21.15.113:0/2620824275 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3280a68b0 con 0x7ff340154fe0 2024-03-04T08:29:17.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 -- 172.21.15.113:0/2620824275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff340154fe0 msgr2=0x7ff3400c1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:17.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 --2- 172.21.15.113:0/2620824275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff340154fe0 0x7ff3400c1b50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff328009d90 tx=0x7ff32801c800 comp rx=0 tx=0).stop 2024-03-04T08:29:17.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 -- 172.21.15.113:0/2620824275 shutdown_connections 2024-03-04T08:29:17.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 --2- 172.21.15.113:0/2620824275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff340154fe0 0x7ff3400c1b50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:17.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 --2- 172.21.15.113:0/2620824275 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 0x7ff3400c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:17.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 -- 172.21.15.113:0/2620824275 >> 172.21.15.113:0/2620824275 conn(0x7ff34006e820 msgr2=0x7ff34006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:17.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 -- 172.21.15.113:0/2620824275 shutdown_connections 2024-03-04T08:29:17.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 -- 172.21.15.113:0/2620824275 wait complete. 2024-03-04T08:29:17.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 Processor -- start 2024-03-04T08:29:17.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 -- start start 2024-03-04T08:29:17.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3400c7700 0x7ff3401f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:17.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 0x7ff34015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:17.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff340154fe0 0x7ff3401505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:17.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340167c10 con 0x7ff340154fe0 2024-03-04T08:29:17.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400c2320 con 0x7ff3400c7700 2024-03-04T08:29:17.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff346200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff340167d90 con 0x7ff340154c00 2024-03-04T08:29:17.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff33fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff340154fe0 0x7ff3401505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:17.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff33fe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff340154fe0 0x7ff3401505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60454/0 (socket says 172.21.15.113:60454) 2024-03-04T08:29:17.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff33fe00640 1 -- 172.21.15.113:0/3089589263 learned_addr learned my addr 172.21.15.113:0/3089589263 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.729+0000 7ff33f400640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3400c7700 0x7ff3401f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff336a00640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 0x7ff34015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff33fe00640 1 -- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 msgr2=0x7ff34015da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff33fe00640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 0x7ff34015da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff33fe00640 1 -- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3400c7700 msgr2=0x7ff3401f40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff33fe00640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3400c7700 0x7ff3401f40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff33fe00640 1 -- 172.21.15.113:0/3089589263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff340155600 con 0x7ff340154fe0 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff336a00640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 0x7ff34015da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff33fe00640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff340154fe0 0x7ff3401505a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7ff33800c600 tx=0x7ff33800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff337e00640 1 -- 172.21.15.113:0/3089589263 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338005cf0 con 0x7ff340154fe0 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff337e00640 1 -- 172.21.15.113:0/3089589263 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff338007070 con 0x7ff340154fe0 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff337e00640 1 -- 172.21.15.113:0/3089589263 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338007460 con 0x7ff340154fe0 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff346200640 1 -- 172.21.15.113:0/3089589263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3400c37d0 con 0x7ff340154fe0 2024-03-04T08:29:17.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff346200640 1 -- 172.21.15.113:0/3089589263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3400c3d90 con 0x7ff340154fe0 2024-03-04T08:29:17.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff346200640 1 -- 172.21.15.113:0/3089589263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff340075c00 con 0x7ff340154fe0 2024-03-04T08:29:17.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff337e00640 1 -- 172.21.15.113:0/3089589263 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff338007600 con 0x7ff340154fe0 2024-03-04T08:29:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff337e00640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 0x7ff31c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff337e00640 1 -- 172.21.15.113:0/3089589263 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7ff3380857c0 con 0x7ff340154fe0 2024-03-04T08:29:17.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.733+0000 7ff33f400640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 0x7ff31c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:17.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.737+0000 7ff33f400640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 0x7ff31c069420 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff330009ac0 tx=0x7ff330009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:17.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.737+0000 7ff337e00640 1 -- 172.21.15.113:0/3089589263 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff338055d20 con 0x7ff340154fe0 2024-03-04T08:29:17.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.925+0000 7ff346200640 1 -- 172.21.15.113:0/3089589263 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3400c1230 con 0x7ff340154fe0 2024-03-04T08:29:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.925+0000 7ff337e00640 1 -- 172.21.15.113:0/3089589263 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7ff338055620 con 0x7ff340154fe0 2024-03-04T08:29:17.925 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:17.925 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 -- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 msgr2=0x7ff31c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 0x7ff31c069420 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff330009ac0 tx=0x7ff330009190 comp rx=0 tx=0).stop 2024-03-04T08:29:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 -- 172.21.15.113:0/3089589263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff340154fe0 msgr2=0x7ff3401505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff340154fe0 0x7ff3401505a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7ff33800c600 tx=0x7ff33800cac0 comp rx=0 tx=0).stop 2024-03-04T08:29:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 -- 172.21.15.113:0/3089589263 shutdown_connections 2024-03-04T08:29:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff31c066f60 0x7ff31c069420 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff340154fe0 0x7ff3401505a0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff340154c00 0x7ff34015da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 --2- 172.21.15.113:0/3089589263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3400c7700 0x7ff3401f40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 -- 172.21.15.113:0/3089589263 >> 172.21.15.113:0/3089589263 conn(0x7ff34006e820 msgr2=0x7ff340153e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 -- 172.21.15.113:0/3089589263 shutdown_connections 2024-03-04T08:29:17.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:17.929+0000 7ff346200640 1 -- 172.21.15.113:0/3089589263 wait complete. 2024-03-04T08:29:17.947 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:19.948 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:20.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe692e00640 1 Processor -- start 2024-03-04T08:29:20.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe692e00640 1 -- start start 2024-03-04T08:29:20.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe692e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c15b900 0x7fe68c163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:20.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe692e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c0c27f0 con 0x7fe68c0c22e0 2024-03-04T08:29:20.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe692e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c0c2a90 con 0x7fe68c14e4c0 2024-03-04T08:29:20.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe692e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c15ad60 con 0x7fe68c15b900 2024-03-04T08:29:20.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe68b400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe68c0c22e0 0x7fe68c15b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52608/0 (socket says 172.21.15.113:52608) 2024-03-04T08:29:20.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe68b400640 1 -- 172.21.15.113:0/3451284466 learned_addr learned my addr 172.21.15.113:0/3451284466 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:20.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe690e00640 1 --2- 172.21.15.113:0/3451284466 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c15b900 0x7fe68c163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:20.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe68aa00640 1 -- 172.21.15.113:0/3451284466 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545675530 0 0) 0x7fe68c0c27f0 con 0x7fe68c0c22e0 2024-03-04T08:29:20.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe68aa00640 1 -- 172.21.15.113:0/3451284466 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe670003530 con 0x7fe68c0c22e0 2024-03-04T08:29:20.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe68aa00640 1 -- 172.21.15.113:0/3451284466 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602640632 0 0) 0x7fe68c0c2a90 con 0x7fe68c14e4c0 2024-03-04T08:29:20.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe690e00640 1 -- 172.21.15.113:0/3451284466 >> v1:172.21.15.136:6789/0 conn(0x7fe68c14e4c0 legacy=0x7fe68c072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:20.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe690e00640 1 -- 172.21.15.113:0/3451284466 >> v1:172.21.15.113:6789/0 conn(0x7fe68c0c22e0 legacy=0x7fe68c15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:20.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.097+0000 7fe690e00640 1 -- 172.21.15.113:0/3451284466 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c070c60 con 0x7fe68c15b900 2024-03-04T08:29:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe690e00640 1 --2- 172.21.15.113:0/3451284466 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c15b900 0x7fe68c163100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe684009dc0 tx=0x7fe68401efc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2cbb6b26d63f248 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe68aa00640 1 -- 172.21.15.113:0/3451284466 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6840b5070 con 0x7fe68c15b900 2024-03-04T08:29:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe68aa00640 1 -- 172.21.15.113:0/3451284466 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6840a8b90 con 0x7fe68c15b900 2024-03-04T08:29:20.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe68aa00640 1 -- 172.21.15.113:0/3451284466 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678001680 con 0x7fe68c15b900 2024-03-04T08:29:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe692e00640 1 -- 172.21.15.113:0/3451284466 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c15b900 msgr2=0x7fe68c163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe692e00640 1 --2- 172.21.15.113:0/3451284466 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c15b900 0x7fe68c163100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe684009dc0 tx=0x7fe68401efc0 comp rx=0 tx=0).stop 2024-03-04T08:29:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe692e00640 1 -- 172.21.15.113:0/3451284466 shutdown_connections 2024-03-04T08:29:20.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe692e00640 1 --2- 172.21.15.113:0/3451284466 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c15b900 0x7fe68c163100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:20.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe692e00640 1 -- 172.21.15.113:0/3451284466 >> 172.21.15.113:0/3451284466 conn(0x7fe68c06e820 msgr2=0x7fe68c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:20.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe692e00640 1 -- 172.21.15.113:0/3451284466 shutdown_connections 2024-03-04T08:29:20.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.101+0000 7fe692e00640 1 -- 172.21.15.113:0/3451284466 wait complete. 2024-03-04T08:29:20.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe692e00640 1 Processor -- start 2024-03-04T08:29:20.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe692e00640 1 -- start start 2024-03-04T08:29:20.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe692e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe68c0c22e0 0x7fe68c14bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:20.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe692e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c14e4c0 0x7fe68c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:20.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe692e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe68c15b900 0x7fe68c159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:20.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe692e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c071500 con 0x7fe68c15b900 2024-03-04T08:29:20.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe692e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c071220 con 0x7fe68c0c22e0 2024-03-04T08:29:20.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe692e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c14e8a0 con 0x7fe68c14e4c0 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe690e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe68c15b900 0x7fe68c159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe690e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe68c15b900 0x7fe68c159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60468/0 (socket says 172.21.15.113:60468) 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe690e00640 1 -- 172.21.15.113:0/2166497134 learned_addr learned my addr 172.21.15.113:0/2166497134 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe68b400640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c14e4c0 0x7fe68c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe68be00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe68c0c22e0 0x7fe68c14bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe690e00640 1 -- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c14e4c0 msgr2=0x7fe68c159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe690e00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c14e4c0 0x7fe68c159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe690e00640 1 -- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe68c0c22e0 msgr2=0x7fe68c14bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe690e00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe68c0c22e0 0x7fe68c14bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe690e00640 1 -- 172.21.15.113:0/2166497134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c1524e0 con 0x7fe68c15b900 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe68b400640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c14e4c0 0x7fe68c159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe690e00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe68c15b900 0x7fe68c159dd0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fe68401d670 tx=0x7fe68401bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe683e00640 1 -- 172.21.15.113:0/2166497134 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6840b5040 con 0x7fe68c15b900 2024-03-04T08:29:20.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe683e00640 1 -- 172.21.15.113:0/2166497134 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6840c1070 con 0x7fe68c15b900 2024-03-04T08:29:20.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe692e00640 1 -- 172.21.15.113:0/2166497134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c156c90 con 0x7fe68c15b900 2024-03-04T08:29:20.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe683e00640 1 -- 172.21.15.113:0/2166497134 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684004440 con 0x7fe68c15b900 2024-03-04T08:29:20.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.105+0000 7fe692e00640 1 -- 172.21.15.113:0/2166497134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c158ad0 con 0x7fe68c15b900 2024-03-04T08:29:20.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.109+0000 7fe692e00640 1 -- 172.21.15.113:0/2166497134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68c075c00 con 0x7fe68c15b900 2024-03-04T08:29:20.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.109+0000 7fe683e00640 1 -- 172.21.15.113:0/2166497134 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6840045e0 con 0x7fe68c15b900 2024-03-04T08:29:20.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.109+0000 7fe683e00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe66c066f90 0x7fe66c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:20.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.109+0000 7fe683e00640 1 -- 172.21.15.113:0/2166497134 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fe684123fb0 con 0x7fe68c15b900 2024-03-04T08:29:20.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.109+0000 7fe68be00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe66c066f90 0x7fe66c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:20.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.109+0000 7fe68be00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe66c066f90 0x7fe66c069450 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe67c000f70 tx=0x7fe67c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:20.112 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.113+0000 7fe683e00640 1 -- 172.21.15.113:0/2166497134 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe684018be0 con 0x7fe68c15b900 2024-03-04T08:29:20.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 -- 172.21.15.113:0/2166497134 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe68c155190 con 0x7fe68c15b900 2024-03-04T08:29:20.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe683e00640 1 -- 172.21.15.113:0/2166497134 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fe6840ae340 con 0x7fe68c15b900 2024-03-04T08:29:20.299 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:20.299 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:20.300 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:20.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 -- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe66c066f90 msgr2=0x7fe66c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:20.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe66c066f90 0x7fe66c069450 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe67c000f70 tx=0x7fe67c009040 comp rx=0 tx=0).stop 2024-03-04T08:29:20.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 -- 172.21.15.113:0/2166497134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe68c15b900 msgr2=0x7fe68c159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:20.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe68c15b900 0x7fe68c159dd0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fe68401d670 tx=0x7fe68401bd60 comp rx=0 tx=0).stop 2024-03-04T08:29:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 -- 172.21.15.113:0/2166497134 shutdown_connections 2024-03-04T08:29:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe66c066f90 0x7fe66c069450 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe68c15b900 0x7fe68c159dd0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe68c14e4c0 0x7fe68c159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 --2- 172.21.15.113:0/2166497134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe68c0c22e0 0x7fe68c14bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 -- 172.21.15.113:0/2166497134 >> 172.21.15.113:0/2166497134 conn(0x7fe68c06e820 msgr2=0x7fe68c15b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 -- 172.21.15.113:0/2166497134 shutdown_connections 2024-03-04T08:29:20.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:20.301+0000 7fe692e00640 1 -- 172.21.15.113:0/2166497134 wait complete. 2024-03-04T08:29:20.321 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:22.322 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:22.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.469+0000 7f648c000640 1 Processor -- start 2024-03-04T08:29:22.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.469+0000 7f648c000640 1 -- start start 2024-03-04T08:29:22.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f648c000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f648414e4c0 0x7f6484072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:22.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f648c000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f64840c22e0 0x7f64840731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:22.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f648c000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64840c27f0 con 0x7f64840c2d80 2024-03-04T08:29:22.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f648c000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f648415ad60 con 0x7f648414e4c0 2024-03-04T08:29:22.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f648c000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64840c2a90 con 0x7f64840c22e0 2024-03-04T08:29:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f6489600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f648414e4c0 0x7f6484072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f6489600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f648414e4c0 0x7f6484072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60472/0 (socket says 172.21.15.113:60472) 2024-03-04T08:29:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f6489600640 1 -- 172.21.15.113:0/136746195 learned_addr learned my addr 172.21.15.113:0/136746195 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f6488c00640 1 --2- 172.21.15.113:0/136746195 >> v2:172.21.15.136:3300/0 conn(0x7f64840c22e0 0x7f64840731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f6489600640 1 -- 172.21.15.113:0/136746195 >> v2:172.21.15.136:3300/0 conn(0x7f64840c22e0 msgr2=0x7f64840731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f6489600640 1 --2- 172.21.15.113:0/136746195 >> v2:172.21.15.136:3300/0 conn(0x7f64840c22e0 0x7f64840731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f6489600640 1 -- 172.21.15.113:0/136746195 >> v1:172.21.15.136:6789/0 conn(0x7f64840c2d80 legacy=0x7f648415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:22.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.473+0000 7f6489600640 1 -- 172.21.15.113:0/136746195 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6484070c60 con 0x7f648414e4c0 2024-03-04T08:29:22.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f6489600640 1 --2- 172.21.15.113:0/136746195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f648414e4c0 0x7f6484072c80 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f64840cd230 tx=0x7f647801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96982bb5e65d2aa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:22.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f6483e00640 1 -- 172.21.15.113:0/136746195 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64780b5070 con 0x7f648414e4c0 2024-03-04T08:29:22.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f6483e00640 1 -- 172.21.15.113:0/136746195 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64780a89d0 con 0x7f648414e4c0 2024-03-04T08:29:22.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f6483e00640 1 -- 172.21.15.113:0/136746195 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64780a8da0 con 0x7f648414e4c0 2024-03-04T08:29:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 -- 172.21.15.113:0/136746195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f648414e4c0 msgr2=0x7f6484072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 --2- 172.21.15.113:0/136746195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f648414e4c0 0x7f6484072c80 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f64840cd230 tx=0x7f647801ecd0 comp rx=0 tx=0).stop 2024-03-04T08:29:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 -- 172.21.15.113:0/136746195 shutdown_connections 2024-03-04T08:29:22.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 --2- 172.21.15.113:0/136746195 >> v2:172.21.15.136:3300/0 conn(0x7f64840c22e0 0x7f64840731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:22.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 --2- 172.21.15.113:0/136746195 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f648414e4c0 0x7f6484072c80 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:22.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 -- 172.21.15.113:0/136746195 >> 172.21.15.113:0/136746195 conn(0x7f648406e820 msgr2=0x7f648406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:22.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 -- 172.21.15.113:0/136746195 shutdown_connections 2024-03-04T08:29:22.476 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 -- 172.21.15.113:0/136746195 wait complete. 2024-03-04T08:29:22.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 Processor -- start 2024-03-04T08:29:22.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 -- start start 2024-03-04T08:29:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64840c22e0 0x7f64840c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64840c2d80 0x7f64841f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:22.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f648414e4c0 0x7f64840c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6484071500 con 0x7f64840c22e0 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6484071220 con 0x7f64840c2d80 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.477+0000 7f648c000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648414e8a0 con 0x7f648414e4c0 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6489600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64840c22e0 0x7f64840c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6489600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64840c22e0 0x7f64840c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60486/0 (socket says 172.21.15.113:60486) 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6489600640 1 -- 172.21.15.113:0/2477004996 learned_addr learned my addr 172.21.15.113:0/2477004996 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f648a000640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f648414e4c0 0x7f64840c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6488c00640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64840c2d80 0x7f64841f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6489600640 1 -- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f648414e4c0 msgr2=0x7f64840c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6489600640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f648414e4c0 0x7f64840c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:22.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6489600640 1 -- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64840c2d80 msgr2=0x7f64841f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6489600640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64840c2d80 0x7f64841f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6489600640 1 -- 172.21.15.113:0/2477004996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f648414b9d0 con 0x7f64840c22e0 2024-03-04T08:29:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6489600640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64840c22e0 0x7f64840c6260 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f64780b3040 tx=0x7f647801ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6481600640 1 -- 172.21.15.113:0/2477004996 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64780c0070 con 0x7f64840c22e0 2024-03-04T08:29:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6481600640 1 -- 172.21.15.113:0/2477004996 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64780b0a50 con 0x7f64840c22e0 2024-03-04T08:29:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f648c000640 1 -- 172.21.15.113:0/2477004996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6484149d00 con 0x7f64840c22e0 2024-03-04T08:29:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6481600640 1 -- 172.21.15.113:0/2477004996 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64780b5070 con 0x7f64840c22e0 2024-03-04T08:29:22.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f648c000640 1 -- 172.21.15.113:0/2477004996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648415a5e0 con 0x7f64840c22e0 2024-03-04T08:29:22.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f648c000640 1 -- 172.21.15.113:0/2477004996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6484075c00 con 0x7f64840c22e0 2024-03-04T08:29:22.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6481600640 1 -- 172.21.15.113:0/2477004996 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64780c3070 con 0x7f64840c22e0 2024-03-04T08:29:22.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6481600640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6464066f90 0x7f6464069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:22.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.481+0000 7f6481600640 1 -- 172.21.15.113:0/2477004996 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f6478122b60 con 0x7f64840c22e0 2024-03-04T08:29:22.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.485+0000 7f6488c00640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6464066f90 0x7f6464069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:22.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.485+0000 7f6488c00640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6464066f90 0x7f6464069450 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f648415ad30 tx=0x7f6470008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:22.486 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.489+0000 7f6481600640 1 -- 172.21.15.113:0/2477004996 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f648414b9d0 con 0x7f64840c22e0 2024-03-04T08:29:22.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.677+0000 7f648c000640 1 -- 172.21.15.113:0/2477004996 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64840c8fa0 con 0x7f64840c22e0 2024-03-04T08:29:22.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.677+0000 7f6481600640 1 -- 172.21.15.113:0/2477004996 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f6478018830 con 0x7f64840c22e0 2024-03-04T08:29:22.678 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:22.678 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:22.678 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:22.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 -- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6464066f90 msgr2=0x7f6464069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:22.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6464066f90 0x7f6464069450 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f648415ad30 tx=0x7f6470008040 comp rx=0 tx=0).stop 2024-03-04T08:29:22.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 -- 172.21.15.113:0/2477004996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64840c22e0 msgr2=0x7f64840c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:22.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64840c22e0 0x7f64840c6260 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f64780b3040 tx=0x7f647801ba70 comp rx=0 tx=0).stop 2024-03-04T08:29:22.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 -- 172.21.15.113:0/2477004996 shutdown_connections 2024-03-04T08:29:22.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f648414e4c0 0x7f64840c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:22.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64840c2d80 0x7f64841f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:22.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6464066f90 0x7f6464069450 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:22.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 --2- 172.21.15.113:0/2477004996 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64840c22e0 0x7f64840c6260 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:22.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 -- 172.21.15.113:0/2477004996 >> 172.21.15.113:0/2477004996 conn(0x7f648406e820 msgr2=0x7f6484071920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:22.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 -- 172.21.15.113:0/2477004996 shutdown_connections 2024-03-04T08:29:22.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:22.681+0000 7f648c000640 1 -- 172.21.15.113:0/2477004996 wait complete. 2024-03-04T08:29:22.700 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:24.701 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:24.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd15a200640 1 Processor -- start 2024-03-04T08:29:24.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd15a200640 1 -- start start 2024-03-04T08:29:24.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd15a200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd15414e4c0 0x7fd154072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:24.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd15a200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd1540c22e0 0x7fd1540731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:24.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd15a200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1540c27f0 con 0x7fd1540c2d80 2024-03-04T08:29:24.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd15a200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15415ad60 con 0x7fd15414e4c0 2024-03-04T08:29:24.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd15a200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1540c2a90 con 0x7fd1540c22e0 2024-03-04T08:29:24.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd153400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd15414e4c0 0x7fd154072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:24.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd153e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd1540c2d80 0x7fd15415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:52622/0 (socket says 172.21.15.113:52622) 2024-03-04T08:29:24.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd153400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd15414e4c0 0x7fd154072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60502/0 (socket says 172.21.15.113:60502) 2024-03-04T08:29:24.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd153e00640 1 -- 172.21.15.113:0/2447046846 learned_addr learned my addr 172.21.15.113:0/2447046846 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:24.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd152a00640 1 --2- 172.21.15.113:0/2447046846 >> v2:172.21.15.136:3300/0 conn(0x7fd1540c22e0 0x7fd1540731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:24.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd153400640 1 -- 172.21.15.113:0/2447046846 >> v2:172.21.15.136:3300/0 conn(0x7fd1540c22e0 msgr2=0x7fd1540731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:24.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd153400640 1 --2- 172.21.15.113:0/2447046846 >> v2:172.21.15.136:3300/0 conn(0x7fd1540c22e0 0x7fd1540731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:24.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd153400640 1 -- 172.21.15.113:0/2447046846 >> v1:172.21.15.113:6789/0 conn(0x7fd1540c2d80 legacy=0x7fd15415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:24.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd153400640 1 -- 172.21.15.113:0/2447046846 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd154070c60 con 0x7fd15414e4c0 2024-03-04T08:29:24.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.849+0000 7fd152000640 1 -- 172.21.15.113:0/2447046846 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898825416 0 0) 0x7fd1540c27f0 con 0x7fd1540c2d80 2024-03-04T08:29:24.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd153400640 1 --2- 172.21.15.113:0/2447046846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15414e4c0 0x7fd154072c80 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fd1540cd230 tx=0x7fd14401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1774b1898047f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:24.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd152000640 1 -- 172.21.15.113:0/2447046846 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440b5070 con 0x7fd15414e4c0 2024-03-04T08:29:24.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd152000640 1 -- 172.21.15.113:0/2447046846 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1440a8970 con 0x7fd15414e4c0 2024-03-04T08:29:24.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd152000640 1 -- 172.21.15.113:0/2447046846 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440a8d40 con 0x7fd15414e4c0 2024-03-04T08:29:24.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd15a200640 1 -- 172.21.15.113:0/2447046846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15414e4c0 msgr2=0x7fd154072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:24.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd15a200640 1 --2- 172.21.15.113:0/2447046846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15414e4c0 0x7fd154072c80 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fd1540cd230 tx=0x7fd14401ec70 comp rx=0 tx=0).stop 2024-03-04T08:29:24.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd15a200640 1 -- 172.21.15.113:0/2447046846 shutdown_connections 2024-03-04T08:29:24.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd15a200640 1 --2- 172.21.15.113:0/2447046846 >> v2:172.21.15.136:3300/0 conn(0x7fd1540c22e0 0x7fd1540731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:24.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd15a200640 1 --2- 172.21.15.113:0/2447046846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd15414e4c0 0x7fd154072c80 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:24.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd15a200640 1 -- 172.21.15.113:0/2447046846 >> 172.21.15.113:0/2447046846 conn(0x7fd15406e820 msgr2=0x7fd15406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:24.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd15a200640 1 -- 172.21.15.113:0/2447046846 shutdown_connections 2024-03-04T08:29:24.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.853+0000 7fd15a200640 1 -- 172.21.15.113:0/2447046846 wait complete. 2024-03-04T08:29:24.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 Processor -- start 2024-03-04T08:29:24.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 -- start start 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1540c22e0 0x7fd1540b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1540c2d80 0x7fd15414a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15414e4c0 0x7fd1540cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd154071500 con 0x7fd1540c2d80 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd154071220 con 0x7fd1540c22e0 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15414e8a0 con 0x7fd15414e4c0 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd152a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1540c2d80 0x7fd15414a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd152a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1540c2d80 0x7fd15414a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:60508/0 (socket says 172.21.15.113:60508) 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd152a00640 1 -- 172.21.15.113:0/352380642 learned_addr learned my addr 172.21.15.113:0/352380642 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd153400640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1540c22e0 0x7fd1540b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:24.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd153e00640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15414e4c0 0x7fd1540cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd152a00640 1 -- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15414e4c0 msgr2=0x7fd1540cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd152a00640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15414e4c0 0x7fd1540cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd152a00640 1 -- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1540c22e0 msgr2=0x7fd1540b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd152a00640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1540c22e0 0x7fd1540b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd152a00640 1 -- 172.21.15.113:0/352380642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1540b0df0 con 0x7fd1540c2d80 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd152a00640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1540c2d80 0x7fd15414a470 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd14000d5d0 tx=0x7fd14000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd14b400640 1 -- 172.21.15.113:0/352380642 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140004240 con 0x7fd1540c2d80 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd14b400640 1 -- 172.21.15.113:0/352380642 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1400043e0 con 0x7fd1540c2d80 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 -- 172.21.15.113:0/352380642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1540b2100 con 0x7fd1540c2d80 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd14b400640 1 -- 172.21.15.113:0/352380642 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140003840 con 0x7fd1540c2d80 2024-03-04T08:29:24.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 -- 172.21.15.113:0/352380642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1540af680 con 0x7fd1540c2d80 2024-03-04T08:29:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.857+0000 7fd15a200640 1 -- 172.21.15.113:0/352380642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1540af110 con 0x7fd1540c2d80 2024-03-04T08:29:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.861+0000 7fd14b400640 1 -- 172.21.15.113:0/352380642 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd1400039e0 con 0x7fd1540c2d80 2024-03-04T08:29:24.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.861+0000 7fd14b400640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd134066f60 0x7fd134069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.861+0000 7fd14b400640 1 -- 172.21.15.113:0/352380642 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fd140085d10 con 0x7fd1540c2d80 2024-03-04T08:29:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.861+0000 7fd153400640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd134066f60 0x7fd134069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:24.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.861+0000 7fd153400640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd134066f60 0x7fd134069420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd1440b2000 tx=0x7fd144008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:24.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:24.865+0000 7fd14b400640 1 -- 172.21.15.113:0/352380642 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd140056240 con 0x7fd1540c2d80 2024-03-04T08:29:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.053+0000 7fd15a200640 1 -- 172.21.15.113:0/352380642 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd154075c00 con 0x7fd1540c2d80 2024-03-04T08:29:25.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.053+0000 7fd14b400640 1 -- 172.21.15.113:0/352380642 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fd140055b40 con 0x7fd1540c2d80 2024-03-04T08:29:25.054 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:25.054 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:25.054 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 -- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd134066f60 msgr2=0x7fd134069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd134066f60 0x7fd134069420 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd1440b2000 tx=0x7fd144008040 comp rx=0 tx=0).stop 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 -- 172.21.15.113:0/352380642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1540c2d80 msgr2=0x7fd15414a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1540c2d80 0x7fd15414a470 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd14000d5d0 tx=0x7fd14000da90 comp rx=0 tx=0).stop 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 -- 172.21.15.113:0/352380642 shutdown_connections 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd15414e4c0 0x7fd1540cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd134066f60 0x7fd134069420 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1540c2d80 0x7fd15414a470 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 --2- 172.21.15.113:0/352380642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1540c22e0 0x7fd1540b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 -- 172.21.15.113:0/352380642 >> 172.21.15.113:0/352380642 conn(0x7fd15406e820 msgr2=0x7fd15414d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 -- 172.21.15.113:0/352380642 shutdown_connections 2024-03-04T08:29:25.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:25.057+0000 7fd15a200640 1 -- 172.21.15.113:0/352380642 wait complete. 2024-03-04T08:29:25.074 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:29:25.075 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:29:27.076 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:27.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59f0a00640 1 Processor -- start 2024-03-04T08:29:27.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59f0a00640 1 -- start start 2024-03-04T08:29:27.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59f0a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f59ec0c8200 0x7f59ec070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:27.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59f0a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec1591f0 con 0x7f59ec152bd0 2024-03-04T08:29:27.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59f0a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec1594d0 con 0x7f59ec1527f0 2024-03-04T08:29:27.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59f0a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec071df0 con 0x7f59ec0c8200 2024-03-04T08:29:27.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59ea800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f59ec152bd0 0x7f59ec1587d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48314/0 (socket says 172.21.15.113:48314) 2024-03-04T08:29:27.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59ea800640 1 -- 172.21.15.113:0/2979320005 learned_addr learned my addr 172.21.15.113:0/2979320005 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:27.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e1400640 1 --2- 172.21.15.113:0/2979320005 >> v2:172.21.15.136:3300/0 conn(0x7f59ec0c8200 0x7f59ec070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298208414 0 0) 0x7f59ec1591f0 con 0x7f59ec152bd0 2024-03-04T08:29:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59d0003530 con 0x7f59ec152bd0 2024-03-04T08:29:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2742445377 0 0) 0x7f59d0003530 con 0x7f59ec152bd0 2024-03-04T08:29:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 >> v2:172.21.15.136:3300/0 conn(0x7f59ec0c8200 msgr2=0x7f59ec070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 --2- 172.21.15.113:0/2979320005 >> v2:172.21.15.136:3300/0 conn(0x7f59ec0c8200 0x7f59ec070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 >> v1:172.21.15.136:6789/0 conn(0x7f59ec1527f0 legacy=0x7f59ec072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0717a0 con 0x7f59ec152bd0 2024-03-04T08:29:27.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f59e4001680 con 0x7f59ec152bd0 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e1400640 1 --2- 172.21.15.113:0/2979320005 >> v2:172.21.15.136:3300/0 conn(0x7f59ec0c8200 0x7f59ec070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 >> v1:172.21.15.113:6789/0 conn(0x7f59ec152bd0 legacy=0x7f59ec1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59d0003a10 0x7f59d0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59d0004810 0x7f59d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59d00051d0 0x7f59d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec071df0 con 0x7f59d0004810 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d000c0d0 con 0x7f59d0003a10 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d000c2d0 con 0x7f59d00051d0 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e1400640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59d0004810 0x7f59d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.225+0000 7f59e9e00640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59d0003a10 0x7f59d0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59ea800640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59d00051d0 0x7f59d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59e1400640 1 -- 172.21.15.113:0/2979320005 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59d00051d0 msgr2=0x7f59d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59e1400640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59d00051d0 0x7f59d000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:27.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59e1400640 1 -- 172.21.15.113:0/2979320005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59d0003a10 msgr2=0x7f59d0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:27.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59e1400640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59d0003a10 0x7f59d0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:27.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59e1400640 1 -- 172.21.15.113:0/2979320005 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0003530 con 0x7f59d0004810 2024-03-04T08:29:27.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59ea800640 1 -- 172.21.15.113:0/2979320005 reap_dead start 2024-03-04T08:29:27.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59e9e00640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59d0003a10 0x7f59d0003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59e1400640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59d0004810 0x7f59d0004c90 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f59d400e320 tx=0x7f59d401cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cb4681dbd8a9ad server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40b2070 con 0x7f59d0004810 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d40a5de0 con 0x7f59d0004810 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59e9400640 1 -- 172.21.15.113:0/2979320005 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4009de0 con 0x7f59d0004810 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59f0a00640 1 -- 172.21.15.113:0/2979320005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59d0004810 msgr2=0x7f59d0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59f0a00640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59d0004810 0x7f59d0004c90 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f59d400e320 tx=0x7f59d401cd70 comp rx=0 tx=0).stop 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59f0a00640 1 -- 172.21.15.113:0/2979320005 shutdown_connections 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59f0a00640 1 --2- 172.21.15.113:0/2979320005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59d0004810 0x7f59d0004c90 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59f0a00640 1 -- 172.21.15.113:0/2979320005 >> 172.21.15.113:0/2979320005 conn(0x7f59ec06e820 msgr2=0x7f59ec06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59f0a00640 1 -- 172.21.15.113:0/2979320005 shutdown_connections 2024-03-04T08:29:27.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.229+0000 7f59f0a00640 1 -- 172.21.15.113:0/2979320005 wait complete. 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59f0a00640 1 Processor -- start 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59f0a00640 1 -- start start 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59f0a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59d0004810 0x7f59ec0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59f0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59ec154620 0x7f59ec1f0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59f0a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59ec1500c0 0x7f59ec1f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59f0a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec070d40 con 0x7f59ec154620 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59f0a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec1597b0 con 0x7f59d0004810 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59f0a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec070ec0 con 0x7f59ec1500c0 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59ec154620 0x7f59ec1f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59ec154620 0x7f59ec1f0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43252/0 (socket says 172.21.15.113:43252) 2024-03-04T08:29:27.234 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e1400640 1 -- 172.21.15.113:0/2074583396 learned_addr learned my addr 172.21.15.113:0/2074583396 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e9e00640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59d0004810 0x7f59ec0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59ea800640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59ec1500c0 0x7f59ec1f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e1400640 1 -- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59ec1500c0 msgr2=0x7f59ec1f0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e1400640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59ec1500c0 0x7f59ec1f0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e1400640 1 -- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59d0004810 msgr2=0x7f59ec0c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e1400640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59d0004810 0x7f59ec0c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e1400640 1 -- 172.21.15.113:0/2074583396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0c53c0 con 0x7f59ec154620 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e1400640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59ec154620 0x7f59ec1f0500 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f59d40b0040 tx=0x7f59d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e2a00640 1 -- 172.21.15.113:0/2074583396 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40bb070 con 0x7f59ec154620 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e2a00640 1 -- 172.21.15.113:0/2074583396 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d40adb60 con 0x7f59ec154620 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59f0a00640 1 -- 172.21.15.113:0/2074583396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec0b0e40 con 0x7f59ec154620 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59e2a00640 1 -- 172.21.15.113:0/2074583396 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d40b2040 con 0x7f59ec154620 2024-03-04T08:29:27.235 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.233+0000 7f59f0a00640 1 -- 172.21.15.113:0/2074583396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec0c5dc0 con 0x7f59ec154620 2024-03-04T08:29:27.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.237+0000 7f59f0a00640 1 -- 172.21.15.113:0/2074583396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec075c00 con 0x7f59ec154620 2024-03-04T08:29:27.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.237+0000 7f59e2a00640 1 -- 172.21.15.113:0/2074583396 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59d40be070 con 0x7f59ec154620 2024-03-04T08:29:27.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.237+0000 7f59e2a00640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59c8066f60 0x7f59c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:27.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.237+0000 7f59e2a00640 1 -- 172.21.15.113:0/2074583396 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f59d411fc10 con 0x7f59ec154620 2024-03-04T08:29:27.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.237+0000 7f59e9e00640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59c8066f60 0x7f59c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:27.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.237+0000 7f59e9e00640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59c8066f60 0x7f59c8069420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f59dc000c10 tx=0x7f59dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:27.240 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.241+0000 7f59e2a00640 1 -- 172.21.15.113:0/2074583396 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f59d4016ad0 con 0x7f59ec154620 2024-03-04T08:29:27.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.429+0000 7f59f0a00640 1 -- 172.21.15.113:0/2074583396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59ec0c1610 con 0x7f59ec154620 2024-03-04T08:29:27.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59e2a00640 1 -- 172.21.15.113:0/2074583396 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f59d40efa90 con 0x7f59ec154620 2024-03-04T08:29:27.430 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:27.431 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:27.431 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:27.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 -- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59c8066f60 msgr2=0x7f59c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:27.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59c8066f60 0x7f59c8069420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f59dc000c10 tx=0x7f59dc009040 comp rx=0 tx=0).stop 2024-03-04T08:29:27.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 -- 172.21.15.113:0/2074583396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59ec154620 msgr2=0x7f59ec1f0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:27.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59ec154620 0x7f59ec1f0500 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f59d40b0040 tx=0x7f59d0005840 comp rx=0 tx=0).stop 2024-03-04T08:29:27.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 -- 172.21.15.113:0/2074583396 shutdown_connections 2024-03-04T08:29:27.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f59ec1500c0 0x7f59ec1f0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:27.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f59c8066f60 0x7f59c8069420 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:27.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59ec154620 0x7f59ec1f0500 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:27.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 --2- 172.21.15.113:0/2074583396 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f59d0004810 0x7f59ec0c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:27.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 -- 172.21.15.113:0/2074583396 >> 172.21.15.113:0/2074583396 conn(0x7f59ec06e820 msgr2=0x7f59ec0b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:27.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 -- 172.21.15.113:0/2074583396 shutdown_connections 2024-03-04T08:29:27.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:27.433+0000 7f59f0a00640 1 -- 172.21.15.113:0/2074583396 wait complete. 2024-03-04T08:29:27.452 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:29.453 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:29.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df9600640 1 Processor -- start 2024-03-04T08:29:29.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df9600640 1 -- start start 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4df414e4c0 0x7f4df4072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4df40c22e0 0x7f4df40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df40c27f0 con 0x7f4df40c2d80 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df415ad60 con 0x7f4df40c22e0 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df40c2a90 con 0x7f4df414e4c0 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4df40c22e0 0x7f4df40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4df40c22e0 0x7f4df40731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43258/0 (socket says 172.21.15.113:43258) 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df2000640 1 -- 172.21.15.113:0/1589760750 learned_addr learned my addr 172.21.15.113:0/1589760750 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df2a00640 1 --2- 172.21.15.113:0/1589760750 >> v2:172.21.15.136:3300/0 conn(0x7f4df414e4c0 0x7f4df4072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df2000640 1 -- 172.21.15.113:0/1589760750 >> v2:172.21.15.136:3300/0 conn(0x7f4df414e4c0 msgr2=0x7f4df4072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df2000640 1 --2- 172.21.15.113:0/1589760750 >> v2:172.21.15.136:3300/0 conn(0x7f4df414e4c0 0x7f4df4072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df2000640 1 -- 172.21.15.113:0/1589760750 >> v1:172.21.15.113:6789/0 conn(0x7f4df40c2d80 legacy=0x7f4df415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df2000640 1 -- 172.21.15.113:0/1589760750 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df4070c60 con 0x7f4df40c22e0 2024-03-04T08:29:29.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.601+0000 7f4df1600640 1 -- 172.21.15.113:0/1589760750 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088749308 0 0) 0x7f4df40c27f0 con 0x7f4df40c2d80 2024-03-04T08:29:29.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df2000640 1 --2- 172.21.15.113:0/1589760750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c22e0 0x7f4df40731c0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4df406f690 tx=0x7f4de00213e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe54712aba6bdde server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:29.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df1600640 1 -- 172.21.15.113:0/1589760750 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de00b6070 con 0x7f4df40c22e0 2024-03-04T08:29:29.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df1600640 1 -- 172.21.15.113:0/1589760750 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de00b1600 con 0x7f4df40c22e0 2024-03-04T08:29:29.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df1600640 1 -- 172.21.15.113:0/1589760750 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de00b19d0 con 0x7f4df40c22e0 2024-03-04T08:29:29.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df9600640 1 -- 172.21.15.113:0/1589760750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c22e0 msgr2=0x7f4df40731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:29.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df9600640 1 --2- 172.21.15.113:0/1589760750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c22e0 0x7f4df40731c0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f4df406f690 tx=0x7f4de00213e0 comp rx=0 tx=0).stop 2024-03-04T08:29:29.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df9600640 1 -- 172.21.15.113:0/1589760750 shutdown_connections 2024-03-04T08:29:29.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df9600640 1 --2- 172.21.15.113:0/1589760750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c22e0 0x7f4df40731c0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:29.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df9600640 1 --2- 172.21.15.113:0/1589760750 >> v2:172.21.15.136:3300/0 conn(0x7f4df414e4c0 0x7f4df4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:29.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df9600640 1 -- 172.21.15.113:0/1589760750 >> 172.21.15.113:0/1589760750 conn(0x7f4df406e820 msgr2=0x7f4df406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:29.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df9600640 1 -- 172.21.15.113:0/1589760750 shutdown_connections 2024-03-04T08:29:29.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.605+0000 7f4df9600640 1 -- 172.21.15.113:0/1589760750 wait complete. 2024-03-04T08:29:29.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df9600640 1 Processor -- start 2024-03-04T08:29:29.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df9600640 1 -- start start 2024-03-04T08:29:29.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4df40c22e0 0x7f4df40cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:29.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df9600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c2d80 0x7f4df41f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df9600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df414e4c0 0x7f4df41f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df9600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4071500 con 0x7f4df40c2d80 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df9600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4071220 con 0x7f4df414e4c0 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df414e8a0 con 0x7f4df40c22e0 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c2d80 0x7f4df41f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df2000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c2d80 0x7f4df41f4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43262/0 (socket says 172.21.15.113:43262) 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df2000640 1 -- 172.21.15.113:0/2698365052 learned_addr learned my addr 172.21.15.113:0/2698365052 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df2a00640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4df40c22e0 0x7f4df40cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df3400640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df414e4c0 0x7f4df41f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df2000640 1 -- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4df40c22e0 msgr2=0x7f4df40cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df2000640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4df40c22e0 0x7f4df40cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df2000640 1 -- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df414e4c0 msgr2=0x7f4df41f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df2000640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df414e4c0 0x7f4df41f4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:29.610 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df2000640 1 -- 172.21.15.113:0/2698365052 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40c5510 con 0x7f4df40c2d80 2024-03-04T08:29:29.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df3400640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df414e4c0 0x7f4df41f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:29.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df2000640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c2d80 0x7f4df41f4560 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f4de000b2c0 tx=0x7f4de001fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:29.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4deaa00640 1 -- 172.21.15.113:0/2698365052 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de00c1070 con 0x7f4df40c2d80 2024-03-04T08:29:29.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4deaa00640 1 -- 172.21.15.113:0/2698365052 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de0004030 con 0x7f4df40c2d80 2024-03-04T08:29:29.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4deaa00640 1 -- 172.21.15.113:0/2698365052 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de00b6040 con 0x7f4df40c2d80 2024-03-04T08:29:29.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df9600640 1 -- 172.21.15.113:0/2698365052 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df40c3610 con 0x7f4df40c2d80 2024-03-04T08:29:29.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.609+0000 7f4df9600640 1 -- 172.21.15.113:0/2698365052 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df40c3bd0 con 0x7f4df40c2d80 2024-03-04T08:29:29.611 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.613+0000 7f4df9600640 1 -- 172.21.15.113:0/2698365052 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df4075c00 con 0x7f4df40c2d80 2024-03-04T08:29:29.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.613+0000 7f4deaa00640 1 -- 172.21.15.113:0/2698365052 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4de0003620 con 0x7f4df40c2d80 2024-03-04T08:29:29.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.613+0000 7f4deaa00640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4dd4066f60 0x7f4dd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:29.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.613+0000 7f4deaa00640 1 -- 172.21.15.113:0/2698365052 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f4de01231d0 con 0x7f4df40c2d80 2024-03-04T08:29:29.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.613+0000 7f4df2a00640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4dd4066f60 0x7f4dd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:29.613 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.613+0000 7f4df2a00640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4dd4066f60 0x7f4dd4069420 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009ac0 tx=0x7f4de4009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:29.616 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.617+0000 7f4deaa00640 1 -- 172.21.15.113:0/2698365052 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4de001af40 con 0x7f4df40c2d80 2024-03-04T08:29:29.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.805+0000 7f4df9600640 1 -- 172.21.15.113:0/2698365052 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4df40733d0 con 0x7f4df40c2d80 2024-03-04T08:29:29.806 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4deaa00640 1 -- 172.21.15.113:0/2698365052 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f4de00f3050 con 0x7f4df40c2d80 2024-03-04T08:29:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:29.807 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:29.807 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:29.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 -- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4dd4066f60 msgr2=0x7f4dd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4dd4066f60 0x7f4dd4069420 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009ac0 tx=0x7f4de4009190 comp rx=0 tx=0).stop 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 -- 172.21.15.113:0/2698365052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c2d80 msgr2=0x7f4df41f4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c2d80 0x7f4df41f4560 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f4de000b2c0 tx=0x7f4de001fc20 comp rx=0 tx=0).stop 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 -- 172.21.15.113:0/2698365052 shutdown_connections 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4df414e4c0 0x7f4df41f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4dd4066f60 0x7f4dd4069420 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4df40c2d80 0x7f4df41f4560 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 --2- 172.21.15.113:0/2698365052 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4df40c22e0 0x7f4df40cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 -- 172.21.15.113:0/2698365052 >> 172.21.15.113:0/2698365052 conn(0x7f4df406e820 msgr2=0x7f4df40c80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 -- 172.21.15.113:0/2698365052 shutdown_connections 2024-03-04T08:29:29.809 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:29.809+0000 7f4df9600640 1 -- 172.21.15.113:0/2698365052 wait complete. 2024-03-04T08:29:29.828 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:31.829 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:31.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf13e00640 1 Processor -- start 2024-03-04T08:29:31.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf13e00640 1 -- start start 2024-03-04T08:29:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf13e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcf0c0c8200 0x7fcf0c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf13e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0c1591f0 con 0x7fcf0c1527f0 2024-03-04T08:29:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf13e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf0c1594d0 con 0x7fcf0c152bd0 2024-03-04T08:29:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf13e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0c071df0 con 0x7fcf0c0c8200 2024-03-04T08:29:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf10a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcf0c0c8200 0x7fcf0c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf10a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcf0c0c8200 0x7fcf0c070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43270/0 (socket says 172.21.15.113:43270) 2024-03-04T08:29:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf11400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fcf0c1527f0 0x7fcf0c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48324/0 (socket says 172.21.15.113:48324) 2024-03-04T08:29:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf10a00640 1 -- 172.21.15.113:0/1316996395 learned_addr learned my addr 172.21.15.113:0/1316996395 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf10a00640 1 -- 172.21.15.113:0/1316996395 >> v1:172.21.15.136:6789/0 conn(0x7fcf0c152bd0 legacy=0x7fcf0c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf10a00640 1 -- 172.21.15.113:0/1316996395 >> v1:172.21.15.113:6789/0 conn(0x7fcf0c1527f0 legacy=0x7fcf0c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:31.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.977+0000 7fcf10a00640 1 -- 172.21.15.113:0/1316996395 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c0717a0 con 0x7fcf0c0c8200 2024-03-04T08:29:31.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf10a00640 1 --2- 172.21.15.113:0/1316996395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c0c8200 0x7fcf0c070d40 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009870 tx=0x7fcf0001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9e27ec8027e998 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:31.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf0be00640 1 -- 172.21.15.113:0/1316996395 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf000a7070 con 0x7fcf0c0c8200 2024-03-04T08:29:31.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf0be00640 1 -- 172.21.15.113:0/1316996395 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf000b0070 con 0x7fcf0c0c8200 2024-03-04T08:29:31.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf0be00640 1 -- 172.21.15.113:0/1316996395 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf000ab890 con 0x7fcf0c0c8200 2024-03-04T08:29:31.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf13e00640 1 -- 172.21.15.113:0/1316996395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c0c8200 msgr2=0x7fcf0c070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:31.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf13e00640 1 --2- 172.21.15.113:0/1316996395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c0c8200 0x7fcf0c070d40 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009870 tx=0x7fcf0001c560 comp rx=0 tx=0).stop 2024-03-04T08:29:31.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf13e00640 1 -- 172.21.15.113:0/1316996395 shutdown_connections 2024-03-04T08:29:31.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf13e00640 1 --2- 172.21.15.113:0/1316996395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c0c8200 0x7fcf0c070d40 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:31.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf13e00640 1 -- 172.21.15.113:0/1316996395 >> 172.21.15.113:0/1316996395 conn(0x7fcf0c06e820 msgr2=0x7fcf0c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:31.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf13e00640 1 -- 172.21.15.113:0/1316996395 shutdown_connections 2024-03-04T08:29:31.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.981+0000 7fcf13e00640 1 -- 172.21.15.113:0/1316996395 wait complete. 2024-03-04T08:29:31.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 Processor -- start 2024-03-04T08:29:31.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 -- start start 2024-03-04T08:29:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf0c0c8200 0x7fcf0c155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf0c1527f0 0x7fcf0c149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c152bd0 0x7fcf0c149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c166380 con 0x7fcf0c152bd0 2024-03-04T08:29:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c1597b0 con 0x7fcf0c1527f0 2024-03-04T08:29:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c166500 con 0x7fcf0c0c8200 2024-03-04T08:29:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf11e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c152bd0 0x7fcf0c149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf11e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c152bd0 0x7fcf0c149a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43276/0 (socket says 172.21.15.113:43276) 2024-03-04T08:29:31.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf11e00640 1 -- 172.21.15.113:0/1755823501 learned_addr learned my addr 172.21.15.113:0/1755823501 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf11400640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf0c0c8200 0x7fcf0c155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf10a00640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf0c1527f0 0x7fcf0c149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf11e00640 1 -- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf0c0c8200 msgr2=0x7fcf0c155a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf11e00640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf0c0c8200 0x7fcf0c155a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf11e00640 1 -- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf0c1527f0 msgr2=0x7fcf0c149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf11e00640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf0c1527f0 0x7fcf0c149540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf11e00640 1 -- 172.21.15.113:0/1755823501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c149fc0 con 0x7fcf0c152bd0 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf11e00640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c152bd0 0x7fcf0c149a80 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400c760 tx=0x7fcf0400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf09600640 1 -- 172.21.15.113:0/1755823501 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04003b80 con 0x7fcf0c152bd0 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf09600640 1 -- 172.21.15.113:0/1755823501 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf04003d20 con 0x7fcf0c152bd0 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 -- 172.21.15.113:0/1755823501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c14a270 con 0x7fcf0c152bd0 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf09600640 1 -- 172.21.15.113:0/1755823501 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400aa30 con 0x7fcf0c152bd0 2024-03-04T08:29:31.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 -- 172.21.15.113:0/1755823501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c15ac20 con 0x7fcf0c152bd0 2024-03-04T08:29:31.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.985+0000 7fcf13e00640 1 -- 172.21.15.113:0/1755823501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0c075c00 con 0x7fcf0c152bd0 2024-03-04T08:29:31.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.989+0000 7fcf09600640 1 -- 172.21.15.113:0/1755823501 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf0400abd0 con 0x7fcf0c152bd0 2024-03-04T08:29:31.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.989+0000 7fcf09600640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcef0066f60 0x7fcef0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:31.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.989+0000 7fcf09600640 1 -- 172.21.15.113:0/1755823501 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fcf04085820 con 0x7fcf0c152bd0 2024-03-04T08:29:31.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.989+0000 7fcf11400640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcef0066f60 0x7fcef0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:31.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.989+0000 7fcf11400640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcef0066f60 0x7fcef0069420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcefc000f70 tx=0x7fcefc007e50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:31.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:31.993+0000 7fcf09600640 1 -- 172.21.15.113:0/1755823501 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcf04055730 con 0x7fcf0c152bd0 2024-03-04T08:29:32.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.177+0000 7fcf13e00640 1 -- 172.21.15.113:0/1755823501 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf0c157650 con 0x7fcf0c152bd0 2024-03-04T08:29:32.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.177+0000 7fcf09600640 1 -- 172.21.15.113:0/1755823501 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fcf040597d0 con 0x7fcf0c152bd0 2024-03-04T08:29:32.177 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:32.177 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:32.178 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 -- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcef0066f60 msgr2=0x7fcef0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcef0066f60 0x7fcef0069420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcefc000f70 tx=0x7fcefc007e50 comp rx=0 tx=0).stop 2024-03-04T08:29:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 -- 172.21.15.113:0/1755823501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c152bd0 msgr2=0x7fcf0c149a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c152bd0 0x7fcf0c149a80 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400c760 tx=0x7fcf0400cc20 comp rx=0 tx=0).stop 2024-03-04T08:29:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 -- 172.21.15.113:0/1755823501 shutdown_connections 2024-03-04T08:29:32.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fcef0066f60 0x7fcef0069420 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcf0c152bd0 0x7fcf0c149a80 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf0c1527f0 0x7fcf0c149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 --2- 172.21.15.113:0/1755823501 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fcf0c0c8200 0x7fcf0c155a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 -- 172.21.15.113:0/1755823501 >> 172.21.15.113:0/1755823501 conn(0x7fcf0c06e820 msgr2=0x7fcf0c0cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 -- 172.21.15.113:0/1755823501 shutdown_connections 2024-03-04T08:29:32.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:32.181+0000 7fcf13e00640 1 -- 172.21.15.113:0/1755823501 wait complete. 2024-03-04T08:29:32.199 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:34.200 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:34.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.345+0000 7fec16400640 1 Processor -- start 2024-03-04T08:29:34.346 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.345+0000 7fec16400640 1 -- start start 2024-03-04T08:29:34.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec16400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec10152d80 0x7fec101589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:34.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec16400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1006b920 con 0x7fec10158ee0 2024-03-04T08:29:34.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec16400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1006bbc0 con 0x7fec100ca5c0 2024-03-04T08:29:34.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec16400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec10153160 con 0x7fec10152d80 2024-03-04T08:29:34.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fec10158ee0 0x7fec10163320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48340/0 (socket says 172.21.15.113:48340) 2024-03-04T08:29:34.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0fe00640 1 -- 172.21.15.113:0/3845566072 learned_addr learned my addr 172.21.15.113:0/3845566072 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:34.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0f400640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec10152d80 0x7fec101589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:34.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337404980 0 0) 0x7fec1006b920 con 0x7fec10158ee0 2024-03-04T08:29:34.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febf4003530 con 0x7fec10158ee0 2024-03-04T08:29:34.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1433426734 0 0) 0x7febf4003530 con 0x7fec10158ee0 2024-03-04T08:29:34.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec10152d80 msgr2=0x7fec101589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:34.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec10152d80 0x7fec101589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 >> v1:172.21.15.136:6789/0 conn(0x7fec100ca5c0 legacy=0x7fec100c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1006be80 con 0x7fec10158ee0 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fec08001680 con 0x7fec10158ee0 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 >> v1:172.21.15.113:6789/0 conn(0x7fec10158ee0 legacy=0x7fec10163320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec10152d80 0x7febf40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febf4003f30 0x7febf4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf4004d50 0x7febf40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf400ba00 con 0x7febf4004d50 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf400bbb0 con 0x7febf4003f30 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf400bdb0 con 0x7fec10152d80 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0fe00640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf4004d50 0x7febf40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0f400640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec10152d80 0x7febf40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:34.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec06a00640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febf4003f30 0x7febf4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:34.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0fe00640 1 -- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec10152d80 msgr2=0x7febf40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:34.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0fe00640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fec10152d80 0x7febf40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:34.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0fe00640 1 -- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febf4003f30 msgr2=0x7febf4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:34.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0fe00640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7febf4003f30 0x7febf4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:34.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.349+0000 7fec0fe00640 1 -- 172.21.15.113:0/3845566072 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4003530 con 0x7febf4004d50 2024-03-04T08:29:34.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec0fe00640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf4004d50 0x7febf40051d0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fec0800c210 tx=0x7fec0801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6314e104532fc508 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:34.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec08007cf0 con 0x7febf4004d50 2024-03-04T08:29:34.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec080a9070 con 0x7febf4004d50 2024-03-04T08:29:34.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec0ea00640 1 -- 172.21.15.113:0/3845566072 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec080a5dc0 con 0x7febf4004d50 2024-03-04T08:29:34.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec16400640 1 -- 172.21.15.113:0/3845566072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf4004d50 msgr2=0x7febf40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:34.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec16400640 1 --2- 172.21.15.113:0/3845566072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7febf4004d50 0x7febf40051d0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fec0800c210 tx=0x7fec0801cc20 comp rx=0 tx=0).stop 2024-03-04T08:29:34.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec0fe00640 1 -- 172.21.15.113:0/3845566072 reap_dead start 2024-03-04T08:29:34.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec16400640 1 -- 172.21.15.113:0/3845566072 shutdown_connections 2024-03-04T08:29:34.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec16400640 1 -- 172.21.15.113:0/3845566072 >> 172.21.15.113:0/3845566072 conn(0x7fec1006e240 msgr2=0x7fec1006e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:34.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec16400640 1 -- 172.21.15.113:0/3845566072 shutdown_connections 2024-03-04T08:29:34.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.353+0000 7fec16400640 1 -- 172.21.15.113:0/3845566072 wait complete. 2024-03-04T08:29:34.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 Processor -- start 2024-03-04T08:29:34.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 -- start start 2024-03-04T08:29:34.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febf4004d50 0x7fec100c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:34.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec100c7980 0x7fec1015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:34.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec101efd30 0x7fec100ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:34.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec100cd7a0 con 0x7fec100c7980 2024-03-04T08:29:34.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec100cd4c0 con 0x7fec101efd30 2024-03-04T08:29:34.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec100cda80 con 0x7febf4004d50 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec06a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec100c7980 0x7fec1015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec06a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec100c7980 0x7fec1015c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43296/0 (socket says 172.21.15.113:43296) 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec06a00640 1 -- 172.21.15.113:0/1216788060 learned_addr learned my addr 172.21.15.113:0/1216788060 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec0fe00640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec101efd30 0x7fec100ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec0f400640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febf4004d50 0x7fec100c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec06a00640 1 -- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febf4004d50 msgr2=0x7fec100c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec06a00640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febf4004d50 0x7fec100c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec06a00640 1 -- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec101efd30 msgr2=0x7fec100ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec06a00640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec101efd30 0x7fec100ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec06a00640 1 -- 172.21.15.113:0/1216788060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec100c48d0 con 0x7fec100c7980 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec0fe00640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec101efd30 0x7fec100ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec06a00640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec100c7980 0x7fec1015c1a0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7febf800f870 tx=0x7febf800fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec07e00640 1 -- 172.21.15.113:0/1216788060 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf80052e0 con 0x7fec100c7980 2024-03-04T08:29:34.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec07e00640 1 -- 172.21.15.113:0/1216788060 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf8018070 con 0x7fec100c7980 2024-03-04T08:29:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec07e00640 1 -- 172.21.15.113:0/1216788060 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf800d970 con 0x7fec100c7980 2024-03-04T08:29:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 -- 172.21.15.113:0/1216788060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec100c4bb0 con 0x7fec100c7980 2024-03-04T08:29:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 -- 172.21.15.113:0/1216788060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec100b2780 con 0x7fec100c7980 2024-03-04T08:29:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.357+0000 7fec16400640 1 -- 172.21.15.113:0/1216788060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec10075c00 con 0x7fec100c7980 2024-03-04T08:29:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.361+0000 7fec07e00640 1 -- 172.21.15.113:0/1216788060 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7febf800db10 con 0x7fec100c7980 2024-03-04T08:29:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.361+0000 7fec07e00640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7febec066f90 0x7febec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.361+0000 7fec07e00640 1 -- 172.21.15.113:0/1216788060 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7febf8085930 con 0x7fec100c7980 2024-03-04T08:29:34.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.361+0000 7fec0f400640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7febec066f90 0x7febec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:34.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.361+0000 7fec0f400640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7febec066f90 0x7febec069450 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fec00014510 tx=0x7fec0000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:34.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.365+0000 7fec07e00640 1 -- 172.21.15.113:0/1216788060 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7febf8055e90 con 0x7fec100c7980 2024-03-04T08:29:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.549+0000 7fec16400640 1 -- 172.21.15.113:0/1216788060 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec100cac00 con 0x7fec100c7980 2024-03-04T08:29:34.550 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec07e00640 1 -- 172.21.15.113:0/1216788060 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7febf8005480 con 0x7fec100c7980 2024-03-04T08:29:34.551 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:34.551 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:34.551 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:34.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 -- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7febec066f90 msgr2=0x7febec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:34.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7febec066f90 0x7febec069450 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fec00014510 tx=0x7fec0000b040 comp rx=0 tx=0).stop 2024-03-04T08:29:34.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 -- 172.21.15.113:0/1216788060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec100c7980 msgr2=0x7fec1015c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:34.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec100c7980 0x7fec1015c1a0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7febf800f870 tx=0x7febf800fd30 comp rx=0 tx=0).stop 2024-03-04T08:29:34.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 -- 172.21.15.113:0/1216788060 shutdown_connections 2024-03-04T08:29:34.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec101efd30 0x7fec100ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:34.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7febec066f90 0x7febec069450 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:34.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec100c7980 0x7fec1015c1a0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:34.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 --2- 172.21.15.113:0/1216788060 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7febf4004d50 0x7fec100c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:34.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 -- 172.21.15.113:0/1216788060 >> 172.21.15.113:0/1216788060 conn(0x7fec1006e240 msgr2=0x7fec1006f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:34.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 -- 172.21.15.113:0/1216788060 shutdown_connections 2024-03-04T08:29:34.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:34.553+0000 7fec16400640 1 -- 172.21.15.113:0/1216788060 wait complete. 2024-03-04T08:29:34.572 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:36.573 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:36.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f240d800640 1 Processor -- start 2024-03-04T08:29:36.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f240d800640 1 -- start start 2024-03-04T08:29:36.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f240d800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408152bd0 0x7f2408070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:36.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f240d800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24081591f0 con 0x7f24081527f0 2024-03-04T08:29:36.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f240d800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24081594d0 con 0x7f24080c8200 2024-03-04T08:29:36.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f240d800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2408071df0 con 0x7f2408152bd0 2024-03-04T08:29:36.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f2406a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f24081527f0 0x7f2408072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:48344/0 (socket says 172.21.15.113:48344) 2024-03-04T08:29:36.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f2406a00640 1 -- 172.21.15.113:0/3169865527 learned_addr learned my addr 172.21.15.113:0/3169865527 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:36.722 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f2407400640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408152bd0 0x7f2408070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:36.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441043046 0 0) 0x7f24081591f0 con 0x7f24081527f0 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23e8003530 con 0x7f24081527f0 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895775019 0 0) 0x7f24081594d0 con 0x7f24080c8200 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.721+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24081591f0 con 0x7f24080c8200 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 933539468 0 0) 0x7f23e8003530 con 0x7f24081527f0 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408152bd0 msgr2=0x7f2408070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408152bd0 0x7f2408070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 >> v1:172.21.15.136:6789/0 conn(0x7f24080c8200 legacy=0x7f24081587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24080717a0 con 0x7f24081527f0 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f23f8001680 con 0x7f24081527f0 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 >> v1:172.21.15.113:6789/0 conn(0x7f24081527f0 legacy=0x7f2408072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e8003b80 0x7f23e8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:36.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 0x7f23e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408152bd0 0x7f23e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2408071df0 con 0x7f23e8004810 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e800bab0 con 0x7f23e8003b80 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e800bcb0 con 0x7f2408152bd0 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f23ffe00640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 0x7f23e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2406a00640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e8003b80 0x7f23e8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2407400640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408152bd0 0x7f23e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f23ffe00640 1 -- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408152bd0 msgr2=0x7f23e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f23ffe00640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408152bd0 0x7f23e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f23ffe00640 1 -- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e8003b80 msgr2=0x7f23e8003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:36.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f23ffe00640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23e8003b80 0x7f23e8003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f23ffe00640 1 -- 172.21.15.113:0/3169865527 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e8003530 con 0x7f23e8004810 2024-03-04T08:29:36.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f2407400640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2408152bd0 0x7f23e80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.725+0000 7f23ffe00640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 0x7f23e8004c90 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f24080c85e0 tx=0x7f23f001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c20d91b90d0ae1 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0001520 con 0x7f23e8004810 2024-03-04T08:29:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f00052f0 con 0x7f23e8004810 2024-03-04T08:29:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406000640 1 -- 172.21.15.113:0/3169865527 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f00a8930 con 0x7f23e8004810 2024-03-04T08:29:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 -- 172.21.15.113:0/3169865527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 msgr2=0x7f23e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 --2- 172.21.15.113:0/3169865527 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 0x7f23e8004c90 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f24080c85e0 tx=0x7f23f001ecd0 comp rx=0 tx=0).stop 2024-03-04T08:29:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2407400640 1 -- 172.21.15.113:0/3169865527 reap_dead start 2024-03-04T08:29:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 -- 172.21.15.113:0/3169865527 shutdown_connections 2024-03-04T08:29:36.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 -- 172.21.15.113:0/3169865527 >> 172.21.15.113:0/3169865527 conn(0x7f240806e820 msgr2=0x7f240806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:36.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 -- 172.21.15.113:0/3169865527 shutdown_connections 2024-03-04T08:29:36.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 -- 172.21.15.113:0/3169865527 wait complete. 2024-03-04T08:29:36.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 Processor -- start 2024-03-04T08:29:36.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 -- start start 2024-03-04T08:29:36.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 0x7f2408156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:36.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f240815bea0 0x7f24081f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24081f1f30 0x7f240814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2408158ee0 con 0x7f23e8004810 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24081597b0 con 0x7f240815bea0 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f240d800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2408070d40 con 0x7f24081f1f30 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 0x7f2408156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 0x7f2408156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43318/0 (socket says 172.21.15.113:43318) 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406a00640 1 -- 172.21.15.113:0/3242021641 learned_addr learned my addr 172.21.15.113:0/3242021641 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f23ffe00640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f240815bea0 0x7f24081f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2407400640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24081f1f30 0x7f240814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406a00640 1 -- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24081f1f30 msgr2=0x7f240814bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406a00640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24081f1f30 0x7f240814bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406a00640 1 -- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f240815bea0 msgr2=0x7f24081f19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406a00640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f240815bea0 0x7f24081f19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:36.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.729+0000 7f2406a00640 1 -- 172.21.15.113:0/3242021641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24081501e0 con 0x7f23e8004810 2024-03-04T08:29:36.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f2406a00640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 0x7f2408156fd0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f23f8004ee0 tx=0x7f23f8007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:36.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f23fea00640 1 -- 172.21.15.113:0/3242021641 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f800f720 con 0x7f23e8004810 2024-03-04T08:29:36.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f23fea00640 1 -- 172.21.15.113:0/3242021641 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f8018070 con 0x7f23e8004810 2024-03-04T08:29:36.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f23fea00640 1 -- 172.21.15.113:0/3242021641 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f800fb20 con 0x7f23e8004810 2024-03-04T08:29:36.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f240d800640 1 -- 172.21.15.113:0/3242021641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2408151b30 con 0x7f23e8004810 2024-03-04T08:29:36.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f240d800640 1 -- 172.21.15.113:0/3242021641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2408151260 con 0x7f23e8004810 2024-03-04T08:29:36.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f240d800640 1 -- 172.21.15.113:0/3242021641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2408075c00 con 0x7f23e8004810 2024-03-04T08:29:36.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f23fea00640 1 -- 172.21.15.113:0/3242021641 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23f800fcc0 con 0x7f23e8004810 2024-03-04T08:29:36.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f23fea00640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e4066f60 0x7f23e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:36.733 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f23fea00640 1 -- 172.21.15.113:0/3242021641 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f23f8085a60 con 0x7f23e8004810 2024-03-04T08:29:36.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.733+0000 7f23ffe00640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e4066f60 0x7f23e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:36.734 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.737+0000 7f23ffe00640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e4066f60 0x7f23e4069420 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f23f00b0000 tx=0x7f23f00aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:36.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.737+0000 7f23fea00640 1 -- 172.21.15.113:0/3242021641 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f24081501e0 con 0x7f23e8004810 2024-03-04T08:29:36.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 -- 172.21.15.113:0/3242021641 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24080c79d0 con 0x7f23e8004810 2024-03-04T08:29:36.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f23fea00640 1 -- 172.21.15.113:0/3242021641 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f23f8055f90 con 0x7f23e8004810 2024-03-04T08:29:36.924 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:36.924 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:36.925 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:36.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 -- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e4066f60 msgr2=0x7f23e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:36.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e4066f60 0x7f23e4069420 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f23f00b0000 tx=0x7f23f00aa070 comp rx=0 tx=0).stop 2024-03-04T08:29:36.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 -- 172.21.15.113:0/3242021641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 msgr2=0x7f2408156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:36.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 0x7f2408156fd0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f23f8004ee0 tx=0x7f23f8007430 comp rx=0 tx=0).stop 2024-03-04T08:29:36.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 -- 172.21.15.113:0/3242021641 shutdown_connections 2024-03-04T08:29:36.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f24081f1f30 0x7f240814bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:36.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f240815bea0 0x7f24081f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:36.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f23e4066f60 0x7f23e4069420 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:36.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 --2- 172.21.15.113:0/3242021641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f23e8004810 0x7f2408156fd0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:36.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 -- 172.21.15.113:0/3242021641 >> 172.21.15.113:0/3242021641 conn(0x7f240806e820 msgr2=0x7f2408157e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:36.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 -- 172.21.15.113:0/3242021641 shutdown_connections 2024-03-04T08:29:36.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:36.925+0000 7f240d800640 1 -- 172.21.15.113:0/3242021641 wait complete. 2024-03-04T08:29:36.945 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:29:36.946 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:29:38.947 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.093+0000 7faca3c00640 1 Processor -- start 2024-03-04T08:29:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7faca3c00640 1 -- start start 2024-03-04T08:29:39.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7faca3c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c0c7a80 0x7fac9c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:39.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7faca3c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c072d80 con 0x7fac9c06cb60 2024-03-04T08:29:39.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7faca3c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c073510 con 0x7fac9c152c60 2024-03-04T08:29:39.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7faca3c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c06b060 con 0x7fac9c0c7a80 2024-03-04T08:29:39.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7faca1c00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fac9c06cb60 0x7fac9c1630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56582/0 (socket says 172.21.15.113:56582) 2024-03-04T08:29:39.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7faca1c00640 1 -- 172.21.15.113:0/766982802 learned_addr learned my addr 172.21.15.113:0/766982802 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:39.096 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7fac93e00640 1 --2- 172.21.15.113:0/766982802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c0c7a80 0x7fac9c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:39.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7fac9be00640 1 -- 172.21.15.113:0/766982802 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942420398 0 0) 0x7fac9c072d80 con 0x7fac9c06cb60 2024-03-04T08:29:39.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7fac9be00640 1 -- 172.21.15.113:0/766982802 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac80003530 con 0x7fac9c06cb60 2024-03-04T08:29:39.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7fac93e00640 1 -- 172.21.15.113:0/766982802 >> v1:172.21.15.136:6789/0 conn(0x7fac9c152c60 legacy=0x7fac9c15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:39.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7fac93e00640 1 -- 172.21.15.113:0/766982802 >> v1:172.21.15.113:6789/0 conn(0x7fac9c06cb60 legacy=0x7fac9c1630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:39.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.097+0000 7fac93e00640 1 -- 172.21.15.113:0/766982802 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c06b2b0 con 0x7fac9c0c7a80 2024-03-04T08:29:39.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7fac93e00640 1 --2- 172.21.15.113:0/766982802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c0c7a80 0x7fac9c0cd010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac840097f0 tx=0x7fac8401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b7bb9d661dd2b68 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:39.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7fac9be00640 1 -- 172.21.15.113:0/766982802 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac840b1070 con 0x7fac9c0c7a80 2024-03-04T08:29:39.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7fac9be00640 1 -- 172.21.15.113:0/766982802 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac840ac620 con 0x7fac9c0c7a80 2024-03-04T08:29:39.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7fac9be00640 1 -- 172.21.15.113:0/766982802 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac840ac9f0 con 0x7fac9c0c7a80 2024-03-04T08:29:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7faca3c00640 1 -- 172.21.15.113:0/766982802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c0c7a80 msgr2=0x7fac9c0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7faca3c00640 1 --2- 172.21.15.113:0/766982802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c0c7a80 0x7fac9c0cd010 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fac840097f0 tx=0x7fac8401c4b0 comp rx=0 tx=0).stop 2024-03-04T08:29:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7faca3c00640 1 -- 172.21.15.113:0/766982802 shutdown_connections 2024-03-04T08:29:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7faca3c00640 1 --2- 172.21.15.113:0/766982802 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c0c7a80 0x7fac9c0cd010 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7faca3c00640 1 -- 172.21.15.113:0/766982802 >> 172.21.15.113:0/766982802 conn(0x7fac9c0671b0 msgr2=0x7fac9c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7faca3c00640 1 -- 172.21.15.113:0/766982802 shutdown_connections 2024-03-04T08:29:39.101 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.101+0000 7faca3c00640 1 -- 172.21.15.113:0/766982802 wait complete. 2024-03-04T08:29:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 Processor -- start 2024-03-04T08:29:39.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 -- start start 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c06cb60 0x7fac9c0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac9c0c7a80 0x7fac9c142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac9c152c60 0x7fac9c143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0731d0 con 0x7fac9c152c60 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c06b590 con 0x7fac9c0c7a80 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0c8ac0 con 0x7fac9c06cb60 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac9c152c60 0x7fac9c143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca1c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac9c152c60 0x7fac9c143740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51024/0 (socket says 172.21.15.113:51024) 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca1c00640 1 -- 172.21.15.113:0/3151763093 learned_addr learned my addr 172.21.15.113:0/3151763093 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca1200640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c06cb60 0x7fac9c0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7fac93e00640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac9c0c7a80 0x7fac9c142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:39.105 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca1c00640 1 -- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c06cb60 msgr2=0x7fac9c0c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:39.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca1c00640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c06cb60 0x7fac9c0c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:39.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca1c00640 1 -- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac9c0c7a80 msgr2=0x7fac9c142880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:39.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca1c00640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac9c0c7a80 0x7fac9c142880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:39.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca1c00640 1 -- 172.21.15.113:0/3151763093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c143290 con 0x7fac9c152c60 2024-03-04T08:29:39.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca1c00640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac9c152c60 0x7fac9c143740 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fac9400cb70 tx=0x7fac9400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:39.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7fac99600640 1 -- 172.21.15.113:0/3151763093 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94017070 con 0x7fac9c152c60 2024-03-04T08:29:39.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7fac99600640 1 -- 172.21.15.113:0/3151763093 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9400ad40 con 0x7fac9c152c60 2024-03-04T08:29:39.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 -- 172.21.15.113:0/3151763093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c1f1710 con 0x7fac9c152c60 2024-03-04T08:29:39.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7fac99600640 1 -- 172.21.15.113:0/3151763093 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94012620 con 0x7fac9c152c60 2024-03-04T08:29:39.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 -- 172.21.15.113:0/3151763093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c1f1cd0 con 0x7fac9c152c60 2024-03-04T08:29:39.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.105+0000 7faca3c00640 1 -- 172.21.15.113:0/3151763093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9c075c00 con 0x7fac9c152c60 2024-03-04T08:29:39.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.109+0000 7fac99600640 1 -- 172.21.15.113:0/3151763093 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac9401a070 con 0x7fac9c152c60 2024-03-04T08:29:39.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.109+0000 7fac99600640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f90 0x7fac78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:39.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.109+0000 7fac99600640 1 -- 172.21.15.113:0/3151763093 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fac94085870 con 0x7fac9c152c60 2024-03-04T08:29:39.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.109+0000 7faca1200640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f90 0x7fac78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:39.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.109+0000 7faca1200640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f90 0x7fac78069450 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fac8c001770 tx=0x7fac8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:39.111 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.113+0000 7fac99600640 1 -- 172.21.15.113:0/3151763093 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fac94055750 con 0x7fac9c152c60 2024-03-04T08:29:39.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.301+0000 7faca3c00640 1 -- 172.21.15.113:0/3151763093 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac9c0c6d80 con 0x7fac9c152c60 2024-03-04T08:29:39.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.301+0000 7fac99600640 1 -- 172.21.15.113:0/3151763093 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fac940597f0 con 0x7fac9c152c60 2024-03-04T08:29:39.302 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:39.302 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:39.302 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:39.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 -- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f90 msgr2=0x7fac78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:39.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f90 0x7fac78069450 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fac8c001770 tx=0x7fac8c009040 comp rx=0 tx=0).stop 2024-03-04T08:29:39.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 -- 172.21.15.113:0/3151763093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac9c152c60 msgr2=0x7fac9c143740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:39.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac9c152c60 0x7fac9c143740 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fac9400cb70 tx=0x7fac9400a430 comp rx=0 tx=0).stop 2024-03-04T08:29:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 -- 172.21.15.113:0/3151763093 shutdown_connections 2024-03-04T08:29:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f90 0x7fac78069450 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac9c152c60 0x7fac9c143740 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac9c0c7a80 0x7fac9c142880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 --2- 172.21.15.113:0/3151763093 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac9c06cb60 0x7fac9c0c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 -- 172.21.15.113:0/3151763093 >> 172.21.15.113:0/3151763093 conn(0x7fac9c0671b0 msgr2=0x7fac9c0cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 -- 172.21.15.113:0/3151763093 shutdown_connections 2024-03-04T08:29:39.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:39.305+0000 7faca3c00640 1 -- 172.21.15.113:0/3151763093 wait complete. 2024-03-04T08:29:39.323 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:41.324 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:41.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.469+0000 7f184ee00640 1 Processor -- start 2024-03-04T08:29:41.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.469+0000 7f184ee00640 1 -- start start 2024-03-04T08:29:41.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ee00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f184814e4c0 0x7f1848072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:41.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ee00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18480c22e0 0x7f18480731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:41.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ee00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18480c27f0 con 0x7f18480c2d80 2024-03-04T08:29:41.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ee00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184815ad60 con 0x7f184814e4c0 2024-03-04T08:29:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ee00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18480c2a90 con 0x7f18480c22e0 2024-03-04T08:29:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ce00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f18480c2d80 0x7f184815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56596/0 (socket says 172.21.15.113:56596) 2024-03-04T08:29:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ce00640 1 -- 172.21.15.113:0/2113506147 learned_addr learned my addr 172.21.15.113:0/2113506147 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1847400640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18480c22e0 0x7f18480731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:41.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1847e00640 1 --2- 172.21.15.113:0/2113506147 >> v2:172.21.15.136:3300/0 conn(0x7f184814e4c0 0x7f1848072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575256393 0 0) 0x7f18480c27f0 con 0x7f18480c2d80 2024-03-04T08:29:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f182c003530 con 0x7f18480c2d80 2024-03-04T08:29:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034241196 0 0) 0x7f182c003530 con 0x7f18480c2d80 2024-03-04T08:29:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18480c22e0 msgr2=0x7f18480731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18480c22e0 0x7f18480731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 >> v2:172.21.15.136:3300/0 conn(0x7f184814e4c0 msgr2=0x7f1848072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 --2- 172.21.15.113:0/2113506147 >> v2:172.21.15.136:3300/0 conn(0x7f184814e4c0 0x7f1848072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1848070c60 con 0x7f18480c2d80 2024-03-04T08:29:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f1840001a00 con 0x7f18480c2d80 2024-03-04T08:29:41.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 >> v1:172.21.15.113:6789/0 conn(0x7f18480c2d80 legacy=0x7f184815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f182c0039c0 0x7f182c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18480c22e0 0x7f182c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f182c004d50 0x7f182c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18480c2a90 con 0x7f182c004d50 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c00dbe0 con 0x7f182c0039c0 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c00dd60 con 0x7f18480c22e0 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ce00640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f182c004d50 0x7f182c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1847400640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18480c22e0 0x7f182c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f1847e00640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f182c0039c0 0x7f182c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ce00640 1 -- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18480c22e0 msgr2=0x7f182c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ce00640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f18480c22e0 0x7f182c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ce00640 1 -- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f182c0039c0 msgr2=0x7f182c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ce00640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f182c0039c0 0x7f182c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:41.475 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.473+0000 7f184ce00640 1 -- 172.21.15.113:0/2113506147 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c003530 con 0x7f182c004d50 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f184ce00640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f182c004d50 0x7f182c0051d0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f184000a850 tx=0x7f184001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d092c7576a4d52 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184000a4a0 con 0x7f182c004d50 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18400ad650 con 0x7f182c004d50 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f1846a00640 1 -- 172.21.15.113:0/2113506147 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18400ada40 con 0x7f182c004d50 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f184ee00640 1 -- 172.21.15.113:0/2113506147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f182c004d50 msgr2=0x7f182c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f184ee00640 1 --2- 172.21.15.113:0/2113506147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f182c004d50 0x7f182c0051d0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f184000a850 tx=0x7f184001d380 comp rx=0 tx=0).stop 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f184ce00640 1 -- 172.21.15.113:0/2113506147 reap_dead start 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f184ee00640 1 -- 172.21.15.113:0/2113506147 shutdown_connections 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f184ee00640 1 -- 172.21.15.113:0/2113506147 >> 172.21.15.113:0/2113506147 conn(0x7f184806e820 msgr2=0x7f184806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f184ee00640 1 -- 172.21.15.113:0/2113506147 shutdown_connections 2024-03-04T08:29:41.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.477+0000 7f184ee00640 1 -- 172.21.15.113:0/2113506147 wait complete. 2024-03-04T08:29:41.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 Processor -- start 2024-03-04T08:29:41.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 -- start start 2024-03-04T08:29:41.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f182c004d50 0x7f1848149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:41.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18481f4740 0x7f1848159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:41.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1848153460 0x7f18481538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:41.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1848071500 con 0x7f18481f4740 2024-03-04T08:29:41.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1848071220 con 0x7f1848153460 2024-03-04T08:29:41.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184814e8a0 con 0x7f182c004d50 2024-03-04T08:29:41.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f1847400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18481f4740 0x7f1848159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:41.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f1847400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18481f4740 0x7f1848159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51052/0 (socket says 172.21.15.113:51052) 2024-03-04T08:29:41.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f1847400640 1 -- 172.21.15.113:0/1356106566 learned_addr learned my addr 172.21.15.113:0/1356106566 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:41.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ce00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1848153460 0x7f18481538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f1847e00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f182c004d50 0x7f1848149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f1847400640 1 -- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f182c004d50 msgr2=0x7f1848149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f1847400640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f182c004d50 0x7f1848149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f1847400640 1 -- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1848153460 msgr2=0x7f18481538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f1847400640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1848153460 0x7f18481538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f1847400640 1 -- 172.21.15.113:0/1356106566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184815a790 con 0x7f18481f4740 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f1847400640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18481f4740 0x7f1848159e60 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1834011990 tx=0x7f1834011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f183fe00640 1 -- 172.21.15.113:0/1356106566 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183400fad0 con 0x7f18481f4740 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f183fe00640 1 -- 172.21.15.113:0/1356106566 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183400fc70 con 0x7f18481f4740 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f183fe00640 1 -- 172.21.15.113:0/1356106566 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834014440 con 0x7f18481f4740 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 -- 172.21.15.113:0/1356106566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18481f09d0 con 0x7f18481f4740 2024-03-04T08:29:41.482 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 -- 172.21.15.113:0/1356106566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18481f0c70 con 0x7f18481f4740 2024-03-04T08:29:41.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.481+0000 7f184ee00640 1 -- 172.21.15.113:0/1356106566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1848075c00 con 0x7f18481f4740 2024-03-04T08:29:41.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.485+0000 7f183fe00640 1 -- 172.21.15.113:0/1356106566 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1834012040 con 0x7f18481f4740 2024-03-04T08:29:41.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.485+0000 7f183fe00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1828066f60 0x7f1828069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:41.483 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.485+0000 7f183fe00640 1 -- 172.21.15.113:0/1356106566 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f1834086310 con 0x7f18481f4740 2024-03-04T08:29:41.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.485+0000 7f1847e00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1828066f60 0x7f1828069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:41.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.485+0000 7f1847e00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1828066f60 0x7f1828069420 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1838009b00 tx=0x7f1838008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:41.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.489+0000 7f183fe00640 1 -- 172.21.15.113:0/1356106566 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1834019030 con 0x7f18481f4740 2024-03-04T08:29:41.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 -- 172.21.15.113:0/1356106566 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18480c1260 con 0x7f18481f4740 2024-03-04T08:29:41.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f183fe00640 1 -- 172.21.15.113:0/1356106566 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f1834056750 con 0x7f18481f4740 2024-03-04T08:29:41.676 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:41.676 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:41.676 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 -- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1828066f60 msgr2=0x7f1828069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1828066f60 0x7f1828069420 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f1838009b00 tx=0x7f1838008040 comp rx=0 tx=0).stop 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 -- 172.21.15.113:0/1356106566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18481f4740 msgr2=0x7f1848159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18481f4740 0x7f1848159e60 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1834011990 tx=0x7f1834011e50 comp rx=0 tx=0).stop 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 -- 172.21.15.113:0/1356106566 shutdown_connections 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1848153460 0x7f18481538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1828066f60 0x7f1828069420 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18481f4740 0x7f1848159e60 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 --2- 172.21.15.113:0/1356106566 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f182c004d50 0x7f1848149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 -- 172.21.15.113:0/1356106566 >> 172.21.15.113:0/1356106566 conn(0x7f184806e820 msgr2=0x7f18480c88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 -- 172.21.15.113:0/1356106566 shutdown_connections 2024-03-04T08:29:41.678 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:41.677+0000 7f184ee00640 1 -- 172.21.15.113:0/1356106566 wait complete. 2024-03-04T08:29:41.697 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:43.698 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:43.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d47200640 1 Processor -- start 2024-03-04T08:29:43.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d47200640 1 -- start start 2024-03-04T08:29:43.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d47200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3d4806d640 0x7f3d4815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:43.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d47200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d480c0ed0 0x7f3d4815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:43.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d47200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d48156fc0 con 0x7f3d4806d260 2024-03-04T08:29:43.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d47200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d48157280 con 0x7f3d4806d640 2024-03-04T08:29:43.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d47200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d48157560 con 0x7f3d480c0ed0 2024-03-04T08:29:43.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d46800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d480c0ed0 0x7f3d4815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:43.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d45400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3d4806d640 0x7f3d4815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:43.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d45e00640 1 --1- >> v1:172.21.15.136:6789/0 conn(0x7f3d4806d260 0x7f3d4806ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.136:6789/0 says I am v1:172.21.15.113:57570/0 (socket says 172.21.15.113:57570) 2024-03-04T08:29:43.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d45e00640 1 -- 172.21.15.113:0/2920871159 learned_addr learned my addr 172.21.15.113:0/2920871159 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:43.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d45400640 1 -- 172.21.15.113:0/2920871159 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d480c0ed0 msgr2=0x7f3d4815ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:43.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d45400640 1 --2- 172.21.15.113:0/2920871159 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d480c0ed0 0x7f3d4815ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:43.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d45400640 1 -- 172.21.15.113:0/2920871159 >> v1:172.21.15.136:6789/0 conn(0x7f3d4806d260 legacy=0x7f3d4806ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:43.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.845+0000 7f3d45400640 1 -- 172.21.15.113:0/2920871159 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d48157840 con 0x7f3d4806d640 2024-03-04T08:29:43.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d45400640 1 --2- 172.21.15.113:0/2920871159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d4806d640 0x7f3d4815a7a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3d340097c0 tx=0x7f3d3401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e4aa5b974633b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:43.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d44a00640 1 -- 172.21.15.113:0/2920871159 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340b5070 con 0x7f3d4806d640 2024-03-04T08:29:43.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d44a00640 1 -- 172.21.15.113:0/2920871159 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d340a8990 con 0x7f3d4806d640 2024-03-04T08:29:43.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d44a00640 1 -- 172.21.15.113:0/2920871159 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d340a8d60 con 0x7f3d4806d640 2024-03-04T08:29:43.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d47200640 1 -- 172.21.15.113:0/2920871159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d4806d640 msgr2=0x7f3d4815a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:43.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d47200640 1 --2- 172.21.15.113:0/2920871159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d4806d640 0x7f3d4815a7a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3d340097c0 tx=0x7f3d3401ec90 comp rx=0 tx=0).stop 2024-03-04T08:29:43.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d47200640 1 -- 172.21.15.113:0/2920871159 shutdown_connections 2024-03-04T08:29:43.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d47200640 1 --2- 172.21.15.113:0/2920871159 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d480c0ed0 0x7f3d4815ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:43.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d47200640 1 --2- 172.21.15.113:0/2920871159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d4806d640 0x7f3d4815a7a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:43.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d47200640 1 -- 172.21.15.113:0/2920871159 >> 172.21.15.113:0/2920871159 conn(0x7f3d4806e2a0 msgr2=0x7f3d4806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:43.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d47200640 1 -- 172.21.15.113:0/2920871159 shutdown_connections 2024-03-04T08:29:43.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.849+0000 7f3d47200640 1 -- 172.21.15.113:0/2920871159 wait complete. 2024-03-04T08:29:43.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d47200640 1 Processor -- start 2024-03-04T08:29:43.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d47200640 1 -- start start 2024-03-04T08:29:43.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d47200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d4806d260 0x7f3d48155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d47200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d4806d640 0x7f3d481ef300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d47200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c0ed0 0x7f3d481378a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d47200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4806c2d0 con 0x7f3d480c0ed0 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d47200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4806c150 con 0x7f3d4806d260 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d47200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4806c770 con 0x7f3d4806d640 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d46800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c0ed0 0x7f3d481378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d46800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c0ed0 0x7f3d481378a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51068/0 (socket says 172.21.15.113:51068) 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d46800640 1 -- 172.21.15.113:0/3597388342 learned_addr learned my addr 172.21.15.113:0/3597388342 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d45e00640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d4806d260 0x7f3d48155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d45400640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d4806d640 0x7f3d481ef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:43.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d46800640 1 -- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d4806d640 msgr2=0x7f3d481ef300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d46800640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d4806d640 0x7f3d481ef300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d46800640 1 -- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d4806d260 msgr2=0x7f3d48155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d46800640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d4806d260 0x7f3d48155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d46800640 1 -- 172.21.15.113:0/3597388342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4814abe0 con 0x7f3d480c0ed0 2024-03-04T08:29:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d46800640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c0ed0 0x7f3d481378a0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009d30 tx=0x7f3d4000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d3e000640 1 -- 172.21.15.113:0/3597388342 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40018020 con 0x7f3d480c0ed0 2024-03-04T08:29:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d3e000640 1 -- 172.21.15.113:0/3597388342 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d40014480 con 0x7f3d480c0ed0 2024-03-04T08:29:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d3e000640 1 -- 172.21.15.113:0/3597388342 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40014870 con 0x7f3d480c0ed0 2024-03-04T08:29:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d47200640 1 -- 172.21.15.113:0/3597388342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d481ee840 con 0x7f3d480c0ed0 2024-03-04T08:29:43.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.853+0000 7f3d47200640 1 -- 172.21.15.113:0/3597388342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4814cd60 con 0x7f3d480c0ed0 2024-03-04T08:29:43.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.857+0000 7f3d47200640 1 -- 172.21.15.113:0/3597388342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d48075a80 con 0x7f3d480c0ed0 2024-03-04T08:29:43.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.857+0000 7f3d3e000640 1 -- 172.21.15.113:0/3597388342 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d4001b020 con 0x7f3d480c0ed0 2024-03-04T08:29:43.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.857+0000 7f3d3e000640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d28066f60 0x7f3d28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:43.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.857+0000 7f3d3e000640 1 -- 172.21.15.113:0/3597388342 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f3d400864e0 con 0x7f3d480c0ed0 2024-03-04T08:29:43.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.857+0000 7f3d45e00640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d28066f60 0x7f3d28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:43.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.857+0000 7f3d45e00640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d28066f60 0x7f3d28069420 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f3d38000f70 tx=0x7f3d38007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:43.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:43.861+0000 7f3d3e000640 1 -- 172.21.15.113:0/3597388342 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3d40022070 con 0x7f3d480c0ed0 2024-03-04T08:29:44.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 -- 172.21.15.113:0/3597388342 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d48154a20 con 0x7f3d480c0ed0 2024-03-04T08:29:44.047 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d3e000640 1 -- 172.21.15.113:0/3597388342 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f3d4814abe0 con 0x7f3d480c0ed0 2024-03-04T08:29:44.047 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:44.047 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:44.047 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:44.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 -- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d28066f60 msgr2=0x7f3d28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:44.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d28066f60 0x7f3d28069420 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f3d38000f70 tx=0x7f3d38007e90 comp rx=0 tx=0).stop 2024-03-04T08:29:44.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 -- 172.21.15.113:0/3597388342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c0ed0 msgr2=0x7f3d481378a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:44.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c0ed0 0x7f3d481378a0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f3d40009d30 tx=0x7f3d4000c490 comp rx=0 tx=0).stop 2024-03-04T08:29:44.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 -- 172.21.15.113:0/3597388342 shutdown_connections 2024-03-04T08:29:44.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3d28066f60 0x7f3d28069420 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:44.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d480c0ed0 0x7f3d481378a0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:44.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3d4806d640 0x7f3d481ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:44.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 --2- 172.21.15.113:0/3597388342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3d4806d260 0x7f3d48155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:44.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 -- 172.21.15.113:0/3597388342 >> 172.21.15.113:0/3597388342 conn(0x7f3d4806e2a0 msgr2=0x7f3d480c6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:44.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 -- 172.21.15.113:0/3597388342 shutdown_connections 2024-03-04T08:29:44.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:44.049+0000 7f3d47200640 1 -- 172.21.15.113:0/3597388342 wait complete. 2024-03-04T08:29:44.069 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:46.070 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:46.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.216+0000 7fbeaec00640 1 Processor -- start 2024-03-04T08:29:46.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.216+0000 7fbeaec00640 1 -- start start 2024-03-04T08:29:46.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeaec00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbea80c85f0 0x7fbea806b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:46.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeaec00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbea806b5a0 0x7fbea8161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:46.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeaec00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea80c1ec0 con 0x7fbea80b20b0 2024-03-04T08:29:46.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeaec00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea80c9b30 con 0x7fbea806b5a0 2024-03-04T08:29:46.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeaec00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea80c12a0 con 0x7fbea80c85f0 2024-03-04T08:29:46.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeacc00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbea806b5a0 0x7fbea8161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:46.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbea7e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbea80b20b0 0x7fbea8157710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:56604/0 (socket says 172.21.15.113:56604) 2024-03-04T08:29:46.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbea7e00640 1 -- 172.21.15.113:0/20058814 learned_addr learned my addr 172.21.15.113:0/20058814 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:46.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbe9f400640 1 --2- 172.21.15.113:0/20058814 >> v2:172.21.15.136:3300/0 conn(0x7fbea80c85f0 0x7fbea806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:46.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbea7400640 1 -- 172.21.15.113:0/20058814 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003776140 0 0) 0x7fbea80c1ec0 con 0x7fbea80b20b0 2024-03-04T08:29:46.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeacc00640 1 -- 172.21.15.113:0/20058814 >> v2:172.21.15.136:3300/0 conn(0x7fbea80c85f0 msgr2=0x7fbea806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:46.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeacc00640 1 --2- 172.21.15.113:0/20058814 >> v2:172.21.15.136:3300/0 conn(0x7fbea80c85f0 0x7fbea806b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:46.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeacc00640 1 -- 172.21.15.113:0/20058814 >> v1:172.21.15.113:6789/0 conn(0x7fbea80b20b0 legacy=0x7fbea8157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:46.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeacc00640 1 -- 172.21.15.113:0/20058814 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea80c1580 con 0x7fbea806b5a0 2024-03-04T08:29:46.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.220+0000 7fbeacc00640 1 --2- 172.21.15.113:0/20058814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea806b5a0 0x7fbea8161820 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fbea806f5b0 tx=0x7fbea001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1137b5a4fc0e4402 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:46.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbea7400640 1 -- 172.21.15.113:0/20058814 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea00b4070 con 0x7fbea806b5a0 2024-03-04T08:29:46.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbea7400640 1 -- 172.21.15.113:0/20058814 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea00af600 con 0x7fbea806b5a0 2024-03-04T08:29:46.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbea7400640 1 -- 172.21.15.113:0/20058814 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea00af9d0 con 0x7fbea806b5a0 2024-03-04T08:29:46.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 -- 172.21.15.113:0/20058814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea806b5a0 msgr2=0x7fbea8161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:46.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 --2- 172.21.15.113:0/20058814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea806b5a0 0x7fbea8161820 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fbea806f5b0 tx=0x7fbea001f110 comp rx=0 tx=0).stop 2024-03-04T08:29:46.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 -- 172.21.15.113:0/20058814 shutdown_connections 2024-03-04T08:29:46.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 --2- 172.21.15.113:0/20058814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea806b5a0 0x7fbea8161820 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:46.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 --2- 172.21.15.113:0/20058814 >> v2:172.21.15.136:3300/0 conn(0x7fbea80c85f0 0x7fbea806b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:46.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 -- 172.21.15.113:0/20058814 >> 172.21.15.113:0/20058814 conn(0x7fbea806e820 msgr2=0x7fbea806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:46.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 -- 172.21.15.113:0/20058814 shutdown_connections 2024-03-04T08:29:46.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 -- 172.21.15.113:0/20058814 wait complete. 2024-03-04T08:29:46.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 Processor -- start 2024-03-04T08:29:46.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 -- start start 2024-03-04T08:29:46.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbea806b5a0 0x7fbea8155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:46.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbea80b20b0 0x7fbea80af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:46.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea80c85f0 0x7fbea80afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:46.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea8157e20 con 0x7fbea80c85f0 2024-03-04T08:29:46.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea81571f0 con 0x7fbea80b20b0 2024-03-04T08:29:46.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeaec00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea80c2400 con 0x7fbea806b5a0 2024-03-04T08:29:46.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeacc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea80c85f0 0x7fbea80afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:46.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeacc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea80c85f0 0x7fbea80afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:51088/0 (socket says 172.21.15.113:51088) 2024-03-04T08:29:46.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbeacc00640 1 -- 172.21.15.113:0/1681364197 learned_addr learned my addr 172.21.15.113:0/1681364197 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:46.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbea7e00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbea806b5a0 0x7fbea8155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:46.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.224+0000 7fbe9f400640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbea80b20b0 0x7fbea80af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:46.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbeacc00640 1 -- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbea806b5a0 msgr2=0x7fbea8155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:46.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbeacc00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbea806b5a0 0x7fbea8155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:46.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbeacc00640 1 -- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbea80b20b0 msgr2=0x7fbea80af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:46.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbeacc00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbea80b20b0 0x7fbea80af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:46.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbeacc00640 1 -- 172.21.15.113:0/1681364197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea8148320 con 0x7fbea80c85f0 2024-03-04T08:29:46.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbeacc00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea80c85f0 0x7fbea80afea0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fbea001bd20 tx=0x7fbea001beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:46.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbea4c00640 1 -- 172.21.15.113:0/1681364197 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea00b4040 con 0x7fbea80c85f0 2024-03-04T08:29:46.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbea4c00640 1 -- 172.21.15.113:0/1681364197 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea00c1070 con 0x7fbea80c85f0 2024-03-04T08:29:46.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbeaec00640 1 -- 172.21.15.113:0/1681364197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea81477a0 con 0x7fbea80c85f0 2024-03-04T08:29:46.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbea4c00640 1 -- 172.21.15.113:0/1681364197 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0009450 con 0x7fbea80c85f0 2024-03-04T08:29:46.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbeaec00640 1 -- 172.21.15.113:0/1681364197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea8147d60 con 0x7fbea80c85f0 2024-03-04T08:29:46.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbeaec00640 1 -- 172.21.15.113:0/1681364197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea8075c00 con 0x7fbea80c85f0 2024-03-04T08:29:46.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbea4c00640 1 -- 172.21.15.113:0/1681364197 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbea00ba6b0 con 0x7fbea80c85f0 2024-03-04T08:29:46.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbea4c00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe84066f60 0x7fbe84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:46.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbea4c00640 1 -- 172.21.15.113:0/1681364197 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fbea0124370 con 0x7fbea80c85f0 2024-03-04T08:29:46.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.228+0000 7fbea7e00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe84066f60 0x7fbe84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:46.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.232+0000 7fbea7e00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe84066f60 0x7fbe84069420 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fbe98007fd0 tx=0x7fbe98007d60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:46.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.232+0000 7fbea4c00640 1 -- 172.21.15.113:0/1681364197 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbea0018c70 con 0x7fbea80c85f0 2024-03-04T08:29:46.402 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:29:46.406 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:29:46.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.424+0000 7fbeaec00640 1 -- 172.21.15.113:0/1681364197 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbea80ca1f0 con 0x7fbea80c85f0 2024-03-04T08:29:46.425 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.424+0000 7fbea4c00640 1 -- 172.21.15.113:0/1681364197 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fbea00f4250 con 0x7fbea80c85f0 2024-03-04T08:29:46.426 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:46.426 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:46.426 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:46.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 -- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe84066f60 msgr2=0x7fbe84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:46.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe84066f60 0x7fbe84069420 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fbe98007fd0 tx=0x7fbe98007d60 comp rx=0 tx=0).stop 2024-03-04T08:29:46.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 -- 172.21.15.113:0/1681364197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea80c85f0 msgr2=0x7fbea80afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:46.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea80c85f0 0x7fbea80afea0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fbea001bd20 tx=0x7fbea001beb0 comp rx=0 tx=0).stop 2024-03-04T08:29:46.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 -- 172.21.15.113:0/1681364197 shutdown_connections 2024-03-04T08:29:46.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe84066f60 0x7fbe84069420 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:46.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbea80c85f0 0x7fbea80afea0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:46.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbea80b20b0 0x7fbea80af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:46.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 --2- 172.21.15.113:0/1681364197 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbea806b5a0 0x7fbea8155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:46.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 -- 172.21.15.113:0/1681364197 >> 172.21.15.113:0/1681364197 conn(0x7fbea806e820 msgr2=0x7fbea8074690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:46.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 -- 172.21.15.113:0/1681364197 shutdown_connections 2024-03-04T08:29:46.430 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:46.428+0000 7fbeaec00640 1 -- 172.21.15.113:0/1681364197 wait complete. 2024-03-04T08:29:46.446 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:48.448 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:48.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7617e00640 1 Processor -- start 2024-03-04T08:29:48.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7617e00640 1 -- start start 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7617e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76100c7050 0x7f76100c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7617e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7610156160 con 0x7f76100c1860 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7617e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f761015ef10 con 0x7f7610155d80 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7617e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7610162500 con 0x7f76100c7050 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7614a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f76100c1860 0x7f7610162c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:58614/0 (socket says 172.21.15.113:58614) 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7614a00640 1 -- 172.21.15.113:0/2088133057 learned_addr learned my addr 172.21.15.113:0/2088133057 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7615e00640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76100c7050 0x7f76100c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483135603 0 0) 0x7f7610156160 con 0x7f76100c1860 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75f8003530 con 0x7f76100c1860 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1282199631 0 0) 0x7f75f8003530 con 0x7f76100c1860 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76100c7050 msgr2=0x7f76100c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:48.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76100c7050 0x7f76100c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 >> v1:172.21.15.136:6789/0 conn(0x7f7610155d80 legacy=0x7f76100c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:48.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761015f1b0 con 0x7f76100c1860 2024-03-04T08:29:48.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f75fc001680 con 0x7f76100c1860 2024-03-04T08:29:48.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 >> v1:172.21.15.113:6789/0 conn(0x7f76100c1860 legacy=0x7f7610162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:48.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f80039f0 0x7f75f8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:48.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f8004810 0x7f75f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:48.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76100c7050 0x7f75f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:48.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7610162500 con 0x7f75f80039f0 2024-03-04T08:29:48.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f800ba80 con 0x7f75f8004810 2024-03-04T08:29:48.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f800bc80 con 0x7f76100c7050 2024-03-04T08:29:48.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7615400640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f80039f0 0x7f75f8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:48.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7614a00640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f8004810 0x7f75f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:48.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7615e00640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76100c7050 0x7f75f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:48.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7615400640 1 -- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76100c7050 msgr2=0x7f75f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:48.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7615400640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f76100c7050 0x7f75f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7615400640 1 -- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f8004810 msgr2=0x7f75f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:48.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7615400640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f8004810 0x7f75f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.596+0000 7f7615400640 1 -- 172.21.15.113:0/2088133057 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f8003530 con 0x7f75f80039f0 2024-03-04T08:29:48.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f7615400640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f80039f0 0x7f75f8003df0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f7604007fd0 tx=0x7f760401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e40517ec856c21 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:48.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76040b0070 con 0x7f75f80039f0 2024-03-04T08:29:48.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76040ab600 con 0x7f75f80039f0 2024-03-04T08:29:48.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f760fe00640 1 -- 172.21.15.113:0/2088133057 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604001460 con 0x7f75f80039f0 2024-03-04T08:29:48.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f7617e00640 1 -- 172.21.15.113:0/2088133057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f80039f0 msgr2=0x7f75f8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:48.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f7617e00640 1 --2- 172.21.15.113:0/2088133057 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75f80039f0 0x7f75f8003df0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f7604007fd0 tx=0x7f760401c4c0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f7615e00640 1 -- 172.21.15.113:0/2088133057 reap_dead start 2024-03-04T08:29:48.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f7617e00640 1 -- 172.21.15.113:0/2088133057 shutdown_connections 2024-03-04T08:29:48.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f7617e00640 1 -- 172.21.15.113:0/2088133057 >> 172.21.15.113:0/2088133057 conn(0x7f761006e820 msgr2=0x7f761006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:48.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f7617e00640 1 -- 172.21.15.113:0/2088133057 shutdown_connections 2024-03-04T08:29:48.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.600+0000 7f7617e00640 1 -- 172.21.15.113:0/2088133057 wait complete. 2024-03-04T08:29:48.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7617e00640 1 Processor -- start 2024-03-04T08:29:48.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7617e00640 1 -- start start 2024-03-04T08:29:48.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7617e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f80039f0 0x7f76101f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:48.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7617e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f761015ebd0 0x7f7610149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:48.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7617e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f761015c500 0x7f761014f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:48.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7617e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7610163380 con 0x7f761015c500 2024-03-04T08:29:48.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7617e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761015fcd0 con 0x7f75f80039f0 2024-03-04T08:29:48.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7617e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76100c1e10 con 0x7f761015ebd0 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f80039f0 0x7f76101f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f761015c500 0x7f761014f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f761015c500 0x7f761014f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36690/0 (socket says 172.21.15.113:36690) 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f80039f0 0x7f76101f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.113:47636/0 (socket says 172.21.15.113:47636) 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615e00640 1 -- 172.21.15.113:0/3863849895 learned_addr learned my addr 172.21.15.113:0/3863849895 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7614a00640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f761015ebd0 0x7f7610149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615e00640 1 -- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f761015ebd0 msgr2=0x7f7610149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615e00640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f761015ebd0 0x7f7610149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615e00640 1 -- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f80039f0 msgr2=0x7f76101f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615e00640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f80039f0 0x7f76101f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615e00640 1 -- 172.21.15.113:0/3863849895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7610149bd0 con 0x7f761015c500 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7615e00640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f761015c500 0x7f761014f690 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f761006c100 tx=0x7f76080115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f760d600640 1 -- 172.21.15.113:0/3863849895 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7608011e70 con 0x7f761015c500 2024-03-04T08:29:48.607 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f760d600640 1 -- 172.21.15.113:0/3863849895 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760801b070 con 0x7f761015c500 2024-03-04T08:29:48.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7617e00640 1 -- 172.21.15.113:0/3863849895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76101580b0 con 0x7f761015c500 2024-03-04T08:29:48.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f760d600640 1 -- 172.21.15.113:0/3863849895 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76080039a0 con 0x7f761015c500 2024-03-04T08:29:48.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.604+0000 7f7617e00640 1 -- 172.21.15.113:0/3863849895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7610158330 con 0x7f761015c500 2024-03-04T08:29:48.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.608+0000 7f7617e00640 1 -- 172.21.15.113:0/3863849895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7610075c00 con 0x7f761015c500 2024-03-04T08:29:48.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.608+0000 7f760d600640 1 -- 172.21.15.113:0/3863849895 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7608003b40 con 0x7f761015c500 2024-03-04T08:29:48.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.608+0000 7f760d600640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75f0066f90 0x7f75f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:48.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.608+0000 7f760d600640 1 -- 172.21.15.113:0/3863849895 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f7608086230 con 0x7f761015c500 2024-03-04T08:29:48.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.608+0000 7f7615400640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75f0066f90 0x7f75f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:48.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.608+0000 7f7615400640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75f0066f90 0x7f75f0069450 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7604007fd0 tx=0x7f7604005040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:48.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.612+0000 7f760d600640 1 -- 172.21.15.113:0/3863849895 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7608025020 con 0x7f761015c500 2024-03-04T08:29:48.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.804+0000 7f7617e00640 1 -- 172.21.15.113:0/3863849895 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76100c8370 con 0x7f761015c500 2024-03-04T08:29:48.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.804+0000 7f760d600640 1 -- 172.21.15.113:0/3863849895 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f76101580b0 con 0x7f761015c500 2024-03-04T08:29:48.805 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:48.805 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:48.806 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:48.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 -- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75f0066f90 msgr2=0x7f75f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:48.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75f0066f90 0x7f75f0069450 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7604007fd0 tx=0x7f7604005040 comp rx=0 tx=0).stop 2024-03-04T08:29:48.807 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 -- 172.21.15.113:0/3863849895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f761015c500 msgr2=0x7f761014f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f761015c500 0x7f761014f690 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f761006c100 tx=0x7f76080115a0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 -- 172.21.15.113:0/3863849895 shutdown_connections 2024-03-04T08:29:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f75f0066f90 0x7f75f0069450 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f761015c500 0x7f761014f690 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f761015ebd0 0x7f7610149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 --2- 172.21.15.113:0/3863849895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75f80039f0 0x7f76101f7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 -- 172.21.15.113:0/3863849895 >> 172.21.15.113:0/3863849895 conn(0x7f761006e820 msgr2=0x7f7610161d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 -- 172.21.15.113:0/3863849895 shutdown_connections 2024-03-04T08:29:48.808 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:48.808+0000 7f7617e00640 1 -- 172.21.15.113:0/3863849895 wait complete. 2024-03-04T08:29:48.826 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:29:48.827 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:29:50.828 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:50.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb65be00640 1 Processor -- start 2024-03-04T08:29:50.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb65be00640 1 -- start start 2024-03-04T08:29:50.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb65be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb65c151b40 0x7fb65c0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:50.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb65be00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb65c0c18d0 0x7fb65c071f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:50.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb65be00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c06c290 con 0x7fb65c151760 2024-03-04T08:29:50.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb65be00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65c15aef0 con 0x7fb65c151b40 2024-03-04T08:29:50.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb65be00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65c06ef10 con 0x7fb65c0c18d0 2024-03-04T08:29:50.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb653e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb65c151b40 0x7fb65c0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:50.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb653e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb65c151b40 0x7fb65c0c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36700/0 (socket says 172.21.15.113:36700) 2024-03-04T08:29:50.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb653e00640 1 -- 172.21.15.113:0/2861053706 learned_addr learned my addr 172.21.15.113:0/2861053706 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:50.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb65b400640 1 --2- 172.21.15.113:0/2861053706 >> v2:172.21.15.136:3300/0 conn(0x7fb65c0c18d0 0x7fb65c071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:50.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb653e00640 1 -- 172.21.15.113:0/2861053706 >> v2:172.21.15.136:3300/0 conn(0x7fb65c0c18d0 msgr2=0x7fb65c071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:50.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb653e00640 1 --2- 172.21.15.113:0/2861053706 >> v2:172.21.15.136:3300/0 conn(0x7fb65c0c18d0 0x7fb65c071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:50.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb653e00640 1 -- 172.21.15.113:0/2861053706 >> v1:172.21.15.136:6789/0 conn(0x7fb65c151760 legacy=0x7fb65c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:50.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.976+0000 7fb653e00640 1 -- 172.21.15.113:0/2861053706 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c15ba90 con 0x7fb65c151b40 2024-03-04T08:29:50.980 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb653e00640 1 --2- 172.21.15.113:0/2861053706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 0x7fb65c0c83d0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fb65c070260 tx=0x7fb64401eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a300e0c4914e20a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:50.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65a000640 1 -- 172.21.15.113:0/2861053706 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440b4070 con 0x7fb65c151b40 2024-03-04T08:29:50.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65a000640 1 -- 172.21.15.113:0/2861053706 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6440a7a40 con 0x7fb65c151b40 2024-03-04T08:29:50.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65a000640 1 -- 172.21.15.113:0/2861053706 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440a7e10 con 0x7fb65c151b40 2024-03-04T08:29:50.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65be00640 1 -- 172.21.15.113:0/2861053706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 msgr2=0x7fb65c0c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:50.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65be00640 1 --2- 172.21.15.113:0/2861053706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 0x7fb65c0c83d0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fb65c070260 tx=0x7fb64401eb60 comp rx=0 tx=0).stop 2024-03-04T08:29:50.981 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65be00640 1 -- 172.21.15.113:0/2861053706 shutdown_connections 2024-03-04T08:29:50.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65be00640 1 --2- 172.21.15.113:0/2861053706 >> v2:172.21.15.136:3300/0 conn(0x7fb65c0c18d0 0x7fb65c071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:50.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65be00640 1 --2- 172.21.15.113:0/2861053706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 0x7fb65c0c83d0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:50.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65be00640 1 -- 172.21.15.113:0/2861053706 >> 172.21.15.113:0/2861053706 conn(0x7fb65c06e2a0 msgr2=0x7fb65c06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:50.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65be00640 1 -- 172.21.15.113:0/2861053706 shutdown_connections 2024-03-04T08:29:50.982 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.980+0000 7fb65be00640 1 -- 172.21.15.113:0/2861053706 wait complete. 2024-03-04T08:29:50.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65be00640 1 Processor -- start 2024-03-04T08:29:50.983 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65be00640 1 -- start start 2024-03-04T08:29:50.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0c18d0 0x7fb65c15a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:50.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65be00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65c151760 0x7fb65c1efa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:50.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65be00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 0x7fb65c0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:50.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65be00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0728c0 con 0x7fb65c151b40 2024-03-04T08:29:50.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c15ab80 con 0x7fb65c0c18d0 2024-03-04T08:29:50.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65be00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c0c1e20 con 0x7fb65c151760 2024-03-04T08:29:50.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 0x7fb65c0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 0x7fb65c0c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36710/0 (socket says 172.21.15.113:36710) 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65b400640 1 -- 172.21.15.113:0/107774942 learned_addr learned my addr 172.21.15.113:0/107774942 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65aa00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0c18d0 0x7fb65c15a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb653e00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65c151760 0x7fb65c1efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65b400640 1 -- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65c151760 msgr2=0x7fb65c1efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65b400640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65c151760 0x7fb65c1efa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65b400640 1 -- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0c18d0 msgr2=0x7fb65c15a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65b400640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0c18d0 0x7fb65c15a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65b400640 1 -- 172.21.15.113:0/107774942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c1f26a0 con 0x7fb65c151b40 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65aa00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0c18d0 0x7fb65c15a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65b400640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 0x7fb65c0c8100 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fb65400efc0 tx=0x7fb65400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:50.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb652a00640 1 -- 172.21.15.113:0/107774942 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654018020 con 0x7fb65c151b40 2024-03-04T08:29:50.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb652a00640 1 -- 172.21.15.113:0/107774942 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb654014480 con 0x7fb65c151b40 2024-03-04T08:29:50.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb652a00640 1 -- 172.21.15.113:0/107774942 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654014870 con 0x7fb65c151b40 2024-03-04T08:29:50.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65be00640 1 -- 172.21.15.113:0/107774942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65c1effa0 con 0x7fb65c151b40 2024-03-04T08:29:50.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.984+0000 7fb65be00640 1 -- 172.21.15.113:0/107774942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c1f2c20 con 0x7fb65c151b40 2024-03-04T08:29:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.988+0000 7fb65be00640 1 -- 172.21.15.113:0/107774942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65c075a80 con 0x7fb65c151b40 2024-03-04T08:29:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.988+0000 7fb652a00640 1 -- 172.21.15.113:0/107774942 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb65401b020 con 0x7fb65c151b40 2024-03-04T08:29:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.988+0000 7fb652a00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 0x7fb638069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.988+0000 7fb652a00640 1 -- 172.21.15.113:0/107774942 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fb654086850 con 0x7fb65c151b40 2024-03-04T08:29:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.988+0000 7fb65aa00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 0x7fb638069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:50.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.988+0000 7fb65aa00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 0x7fb638069420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb64c001710 tx=0x7fb64c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:50.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:50.992+0000 7fb652a00640 1 -- 172.21.15.113:0/107774942 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb654022030 con 0x7fb65c151b40 2024-03-04T08:29:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 -- 172.21.15.113:0/107774942 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb65c073220 con 0x7fb65c151b40 2024-03-04T08:29:51.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb652a00640 1 -- 172.21.15.113:0/107774942 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fb654010040 con 0x7fb65c151b40 2024-03-04T08:29:51.180 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:51.180 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:51.180 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 -- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 msgr2=0x7fb638069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 0x7fb638069420 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb64c001710 tx=0x7fb64c009040 comp rx=0 tx=0).stop 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 -- 172.21.15.113:0/107774942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 msgr2=0x7fb65c0c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 0x7fb65c0c8100 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fb65400efc0 tx=0x7fb65400c490 comp rx=0 tx=0).stop 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 -- 172.21.15.113:0/107774942 shutdown_connections 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb638066f60 0x7fb638069420 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb65c151b40 0x7fb65c0c8100 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb65c151760 0x7fb65c1efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 --2- 172.21.15.113:0/107774942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb65c0c18d0 0x7fb65c15a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 -- 172.21.15.113:0/107774942 >> 172.21.15.113:0/107774942 conn(0x7fb65c06e2a0 msgr2=0x7fb65c14fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:51.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 -- 172.21.15.113:0/107774942 shutdown_connections 2024-03-04T08:29:51.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:51.180+0000 7fb65be00640 1 -- 172.21.15.113:0/107774942 wait complete. 2024-03-04T08:29:51.202 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:53.203 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:53.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.348+0000 7f22e2400640 1 Processor -- start 2024-03-04T08:29:53.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.348+0000 7f22e2400640 1 -- start start 2024-03-04T08:29:53.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22e2400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f22dc14e4c0 0x7f22dc072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:53.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22e2400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22dc15b900 0x7f22dc0731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:53.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22e2400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22dc0c27f0 con 0x7f22dc0c22e0 2024-03-04T08:29:53.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22e2400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc15ad60 con 0x7f22dc15b900 2024-03-04T08:29:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22e2400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc0c2a90 con 0x7f22dc14e4c0 2024-03-04T08:29:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22dc15b900 0x7f22dc0731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22dc15b900 0x7f22dc0731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36726/0 (socket says 172.21.15.113:36726) 2024-03-04T08:29:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22dbe00640 1 -- 172.21.15.113:0/4129452796 learned_addr learned my addr 172.21.15.113:0/4129452796 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22db400640 1 --2- 172.21.15.113:0/4129452796 >> v2:172.21.15.136:3300/0 conn(0x7f22dc14e4c0 0x7f22dc072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22dbe00640 1 -- 172.21.15.113:0/4129452796 >> v2:172.21.15.136:3300/0 conn(0x7f22dc14e4c0 msgr2=0x7f22dc072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22dbe00640 1 --2- 172.21.15.113:0/4129452796 >> v2:172.21.15.136:3300/0 conn(0x7f22dc14e4c0 0x7f22dc072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22dbe00640 1 -- 172.21.15.113:0/4129452796 >> v1:172.21.15.136:6789/0 conn(0x7f22dc0c22e0 legacy=0x7f22dc15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22dbe00640 1 -- 172.21.15.113:0/4129452796 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc070c60 con 0x7f22dc15b900 2024-03-04T08:29:53.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.352+0000 7f22db400640 1 --2- 172.21.15.113:0/4129452796 >> v2:172.21.15.136:3300/0 conn(0x7f22dc14e4c0 0x7f22dc072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:53.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22dbe00640 1 --2- 172.21.15.113:0/4129452796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 0x7f22dc0731c0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f22dc06f690 tx=0x7f22d401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f57c07e36f4f08 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:53.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22daa00640 1 -- 172.21.15.113:0/4129452796 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40b4070 con 0x7f22dc15b900 2024-03-04T08:29:53.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22daa00640 1 -- 172.21.15.113:0/4129452796 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d40af400 con 0x7f22dc15b900 2024-03-04T08:29:53.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22daa00640 1 -- 172.21.15.113:0/4129452796 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40af7d0 con 0x7f22dc15b900 2024-03-04T08:29:53.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 -- 172.21.15.113:0/4129452796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 msgr2=0x7f22dc0731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:53.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 --2- 172.21.15.113:0/4129452796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 0x7f22dc0731c0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f22dc06f690 tx=0x7f22d401f1e0 comp rx=0 tx=0).stop 2024-03-04T08:29:53.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 -- 172.21.15.113:0/4129452796 shutdown_connections 2024-03-04T08:29:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 --2- 172.21.15.113:0/4129452796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 0x7f22dc0731c0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 --2- 172.21.15.113:0/4129452796 >> v2:172.21.15.136:3300/0 conn(0x7f22dc14e4c0 0x7f22dc072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 -- 172.21.15.113:0/4129452796 >> 172.21.15.113:0/4129452796 conn(0x7f22dc06e820 msgr2=0x7f22dc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 -- 172.21.15.113:0/4129452796 shutdown_connections 2024-03-04T08:29:53.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 -- 172.21.15.113:0/4129452796 wait complete. 2024-03-04T08:29:53.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 Processor -- start 2024-03-04T08:29:53.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 -- start start 2024-03-04T08:29:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22dc0c22e0 0x7f22dc1f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22dc14e4c0 0x7f22dc1488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 0x7f22dc1f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc071500 con 0x7f22dc15b900 2024-03-04T08:29:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc071220 con 0x7f22dc0c22e0 2024-03-04T08:29:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.356+0000 7f22e2400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc14e8a0 con 0x7f22dc14e4c0 2024-03-04T08:29:53.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 0x7f22dc1f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22dbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 0x7f22dc1f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36742/0 (socket says 172.21.15.113:36742) 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22dbe00640 1 -- 172.21.15.113:0/260969863 learned_addr learned my addr 172.21.15.113:0/260969863 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22db400640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22dc0c22e0 0x7f22dc1f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22d2a00640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22dc14e4c0 0x7f22dc1488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22dbe00640 1 -- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22dc14e4c0 msgr2=0x7f22dc1488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22dbe00640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22dc14e4c0 0x7f22dc1488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22dbe00640 1 -- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22dc0c22e0 msgr2=0x7f22dc1f13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22dbe00640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22dc0c22e0 0x7f22dc1f13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22dbe00640 1 -- 172.21.15.113:0/260969863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc1f4610 con 0x7f22dc15b900 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22dbe00640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 0x7f22dc1f3df0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f22d40b2040 tx=0x7f22d401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22d3e00640 1 -- 172.21.15.113:0/260969863 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40b4040 con 0x7f22dc15b900 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22d3e00640 1 -- 172.21.15.113:0/260969863 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d40afdb0 con 0x7f22dc15b900 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22e2400640 1 -- 172.21.15.113:0/260969863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc1f48f0 con 0x7f22dc15b900 2024-03-04T08:29:53.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22d3e00640 1 -- 172.21.15.113:0/260969863 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d4009510 con 0x7f22dc15b900 2024-03-04T08:29:53.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22e2400640 1 -- 172.21.15.113:0/260969863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc1f0bd0 con 0x7f22dc15b900 2024-03-04T08:29:53.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22e2400640 1 -- 172.21.15.113:0/260969863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22dc075c00 con 0x7f22dc15b900 2024-03-04T08:29:53.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.360+0000 7f22d3e00640 1 -- 172.21.15.113:0/260969863 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22d40c3020 con 0x7f22dc15b900 2024-03-04T08:29:53.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.364+0000 7f22d3e00640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22b8066f90 0x7f22b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:53.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.364+0000 7f22d3e00640 1 -- 172.21.15.113:0/260969863 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f22d4123670 con 0x7f22dc15b900 2024-03-04T08:29:53.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.364+0000 7f22db400640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22b8066f90 0x7f22b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:53.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.364+0000 7f22db400640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22b8066f90 0x7f22b8069450 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f22cc006e90 tx=0x7f22cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:53.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.368+0000 7f22d3e00640 1 -- 172.21.15.113:0/260969863 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f22d40ca020 con 0x7f22dc15b900 2024-03-04T08:29:53.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.552+0000 7f22e2400640 1 -- 172.21.15.113:0/260969863 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22dc0c86f0 con 0x7f22dc15b900 2024-03-04T08:29:53.553 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.552+0000 7f22d3e00640 1 -- 172.21.15.113:0/260969863 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f22d4018d40 con 0x7f22dc15b900 2024-03-04T08:29:53.554 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:53.554 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:53.554 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 -- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22b8066f90 msgr2=0x7f22b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22b8066f90 0x7f22b8069450 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f22cc006e90 tx=0x7f22cc008040 comp rx=0 tx=0).stop 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 -- 172.21.15.113:0/260969863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 msgr2=0x7f22dc1f3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 0x7f22dc1f3df0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f22d40b2040 tx=0x7f22d401bf80 comp rx=0 tx=0).stop 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 -- 172.21.15.113:0/260969863 shutdown_connections 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f22b8066f90 0x7f22b8069450 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22dc15b900 0x7f22dc1f3df0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f22dc14e4c0 0x7f22dc1488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 --2- 172.21.15.113:0/260969863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22dc0c22e0 0x7f22dc1f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 -- 172.21.15.113:0/260969863 >> 172.21.15.113:0/260969863 conn(0x7f22dc06e820 msgr2=0x7f22dc072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 -- 172.21.15.113:0/260969863 shutdown_connections 2024-03-04T08:29:53.556 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:53.556+0000 7f22e2400640 1 -- 172.21.15.113:0/260969863 wait complete. 2024-03-04T08:29:53.575 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:55.576 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:55.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb08b600640 1 Processor -- start 2024-03-04T08:29:55.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb08b600640 1 -- start start 2024-03-04T08:29:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb08b600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fb08414e4c0 0x7fb084072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb08b600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb0840c22e0 0x7fb0840731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb08b600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0840c27f0 con 0x7fb0840c2d80 2024-03-04T08:29:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb08b600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08415ad60 con 0x7fb0840c22e0 2024-03-04T08:29:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb08b600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0840c2a90 con 0x7fb08414e4c0 2024-03-04T08:29:55.726 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb07be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb0840c22e0 0x7fb0840731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb07be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb0840c22e0 0x7fb0840731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36754/0 (socket says 172.21.15.113:36754) 2024-03-04T08:29:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb07be00640 1 -- 172.21.15.113:0/1967680707 learned_addr learned my addr 172.21.15.113:0/1967680707 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.724+0000 7fb088c00640 1 --2- 172.21.15.113:0/1967680707 >> v2:172.21.15.136:3300/0 conn(0x7fb08414e4c0 0x7fb084072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb07be00640 1 -- 172.21.15.113:0/1967680707 >> v2:172.21.15.136:3300/0 conn(0x7fb08414e4c0 msgr2=0x7fb084072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb07be00640 1 --2- 172.21.15.113:0/1967680707 >> v2:172.21.15.136:3300/0 conn(0x7fb08414e4c0 0x7fb084072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb07be00640 1 -- 172.21.15.113:0/1967680707 >> v1:172.21.15.113:6789/0 conn(0x7fb0840c2d80 legacy=0x7fb08415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:55.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb07be00640 1 -- 172.21.15.113:0/1967680707 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb084070c60 con 0x7fb0840c22e0 2024-03-04T08:29:55.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb07be00640 1 --2- 172.21.15.113:0/1967680707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0840c22e0 0x7fb0840731c0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fb08406f690 tx=0x7fb06c020f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce77f84d12c09188 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:55.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb083e00640 1 -- 172.21.15.113:0/1967680707 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0b7070 con 0x7fb0840c22e0 2024-03-04T08:29:55.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb083e00640 1 -- 172.21.15.113:0/1967680707 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06c0aac20 con 0x7fb0840c22e0 2024-03-04T08:29:55.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb083e00640 1 -- 172.21.15.113:0/1967680707 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0b2440 con 0x7fb0840c22e0 2024-03-04T08:29:55.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb08b600640 1 -- 172.21.15.113:0/1967680707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0840c22e0 msgr2=0x7fb0840731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:55.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb08b600640 1 --2- 172.21.15.113:0/1967680707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0840c22e0 0x7fb0840731c0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fb08406f690 tx=0x7fb06c020f20 comp rx=0 tx=0).stop 2024-03-04T08:29:55.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb08b600640 1 -- 172.21.15.113:0/1967680707 shutdown_connections 2024-03-04T08:29:55.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb08b600640 1 --2- 172.21.15.113:0/1967680707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb0840c22e0 0x7fb0840731c0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:55.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb08b600640 1 --2- 172.21.15.113:0/1967680707 >> v2:172.21.15.136:3300/0 conn(0x7fb08414e4c0 0x7fb084072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:55.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb08b600640 1 -- 172.21.15.113:0/1967680707 >> 172.21.15.113:0/1967680707 conn(0x7fb08406e820 msgr2=0x7fb08406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:55.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb08b600640 1 -- 172.21.15.113:0/1967680707 shutdown_connections 2024-03-04T08:29:55.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.728+0000 7fb08b600640 1 -- 172.21.15.113:0/1967680707 wait complete. 2024-03-04T08:29:55.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb08b600640 1 Processor -- start 2024-03-04T08:29:55.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb08b600640 1 -- start start 2024-03-04T08:29:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb08b600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0840c22e0 0x7fb084140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb08b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0840c2d80 0x7fb08413b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb08b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08414e4c0 0x7fb08413b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb08b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb084071500 con 0x7fb08414e4c0 2024-03-04T08:29:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb08b600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb084071220 con 0x7fb0840c22e0 2024-03-04T08:29:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb08b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08414e8a0 con 0x7fb0840c2d80 2024-03-04T08:29:55.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb089600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08414e4c0 0x7fb08413b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb089600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08414e4c0 0x7fb08413b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36764/0 (socket says 172.21.15.113:36764) 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb089600640 1 -- 172.21.15.113:0/120374709 learned_addr learned my addr 172.21.15.113:0/120374709 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb088c00640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0840c22e0 0x7fb084140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb07be00640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0840c2d80 0x7fb08413b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb089600640 1 -- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0840c2d80 msgr2=0x7fb08413b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb089600640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0840c2d80 0x7fb08413b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb089600640 1 -- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0840c22e0 msgr2=0x7fb084140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb089600640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0840c22e0 0x7fb084140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb089600640 1 -- 172.21.15.113:0/120374709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08413fdc0 con 0x7fb08414e4c0 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb089600640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08414e4c0 0x7fb08413b560 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00c760 tx=0x7fb07c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb081600640 1 -- 172.21.15.113:0/120374709 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c003b80 con 0x7fb08414e4c0 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb081600640 1 -- 172.21.15.113:0/120374709 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c003d20 con 0x7fb08414e4c0 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb081600640 1 -- 172.21.15.113:0/120374709 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00aa30 con 0x7fb08414e4c0 2024-03-04T08:29:55.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb08b600640 1 -- 172.21.15.113:0/120374709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb084140e70 con 0x7fb08414e4c0 2024-03-04T08:29:55.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.732+0000 7fb08b600640 1 -- 172.21.15.113:0/120374709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0841471a0 con 0x7fb08414e4c0 2024-03-04T08:29:55.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.736+0000 7fb08b600640 1 -- 172.21.15.113:0/120374709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb084158820 con 0x7fb08414e4c0 2024-03-04T08:29:55.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.736+0000 7fb081600640 1 -- 172.21.15.113:0/120374709 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb07c00abd0 con 0x7fb08414e4c0 2024-03-04T08:29:55.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.736+0000 7fb081600640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb060066f60 0x7fb060069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:55.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.736+0000 7fb081600640 1 -- 172.21.15.113:0/120374709 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fb07c085960 con 0x7fb08414e4c0 2024-03-04T08:29:55.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.736+0000 7fb088c00640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb060066f60 0x7fb060069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:55.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.736+0000 7fb088c00640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb060066f60 0x7fb060069420 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb0740079c0 tx=0x7fb074008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:55.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.740+0000 7fb081600640 1 -- 172.21.15.113:0/120374709 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb07c055750 con 0x7fb08414e4c0 2024-03-04T08:29:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 -- 172.21.15.113:0/120374709 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb084075c00 con 0x7fb08414e4c0 2024-03-04T08:29:55.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb081600640 1 -- 172.21.15.113:0/120374709 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fb07c0597f0 con 0x7fb08414e4c0 2024-03-04T08:29:55.927 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:55.928 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:55.928 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:55.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 -- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb060066f60 msgr2=0x7fb060069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb060066f60 0x7fb060069420 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb0740079c0 tx=0x7fb074008040 comp rx=0 tx=0).stop 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 -- 172.21.15.113:0/120374709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08414e4c0 msgr2=0x7fb08413b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08414e4c0 0x7fb08413b560 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fb07c00c760 tx=0x7fb07c00cc20 comp rx=0 tx=0).stop 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 -- 172.21.15.113:0/120374709 shutdown_connections 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fb060066f60 0x7fb060069420 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb08414e4c0 0x7fb08413b560 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fb0840c2d80 0x7fb08413b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 --2- 172.21.15.113:0/120374709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0840c22e0 0x7fb084140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 -- 172.21.15.113:0/120374709 >> 172.21.15.113:0/120374709 conn(0x7fb08406e820 msgr2=0x7fb08414ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 -- 172.21.15.113:0/120374709 shutdown_connections 2024-03-04T08:29:55.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:55.928+0000 7fb08b600640 1 -- 172.21.15.113:0/120374709 wait complete. 2024-03-04T08:29:55.950 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:29:57.951 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:29:58.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.096+0000 7f466b600640 1 Processor -- start 2024-03-04T08:29:58.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.096+0000 7f466b600640 1 -- start start 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f466b600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46640c22e0 0x7f466415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f466b600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f46640c2d80 0x7f466415b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f466b600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46640c27f0 con 0x7f466414e4c0 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f466b600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466415ad60 con 0x7f46640c22e0 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f466b600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46640c2a90 con 0x7f46640c2d80 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f465be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46640c22e0 0x7f466415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f465be00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46640c22e0 0x7f466415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41360/0 (socket says 172.21.15.113:41360) 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f465be00640 1 -- 172.21.15.113:0/1297392050 learned_addr learned my addr 172.21.15.113:0/1297392050 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f4669600640 1 --2- 172.21.15.113:0/1297392050 >> v2:172.21.15.136:3300/0 conn(0x7f46640c2d80 0x7f466415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f465be00640 1 -- 172.21.15.113:0/1297392050 >> v2:172.21.15.136:3300/0 conn(0x7f46640c2d80 msgr2=0x7f466415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:29:58.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f465be00640 1 --2- 172.21.15.113:0/1297392050 >> v2:172.21.15.136:3300/0 conn(0x7f46640c2d80 0x7f466415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:58.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f465be00640 1 -- 172.21.15.113:0/1297392050 >> v1:172.21.15.136:6789/0 conn(0x7f466414e4c0 legacy=0x7f4664072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:58.100 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f465be00640 1 -- 172.21.15.113:0/1297392050 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4664070c60 con 0x7f46640c22e0 2024-03-04T08:29:58.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.100+0000 7f465be00640 1 --2- 172.21.15.113:0/1297392050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46640c22e0 0x7f466415b1f0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f46640c2700 tx=0x7f464c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fef5772b4ef82a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:58.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f4663e00640 1 -- 172.21.15.113:0/1297392050 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0ab430 con 0x7f46640c22e0 2024-03-04T08:29:58.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f4663e00640 1 -- 172.21.15.113:0/1297392050 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f464c0b0070 con 0x7f46640c22e0 2024-03-04T08:29:58.102 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f4663e00640 1 -- 172.21.15.113:0/1297392050 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0ab7f0 con 0x7f46640c22e0 2024-03-04T08:29:58.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 -- 172.21.15.113:0/1297392050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46640c22e0 msgr2=0x7f466415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:58.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 --2- 172.21.15.113:0/1297392050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46640c22e0 0x7f466415b1f0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f46640c2700 tx=0x7f464c01c520 comp rx=0 tx=0).stop 2024-03-04T08:29:58.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 -- 172.21.15.113:0/1297392050 shutdown_connections 2024-03-04T08:29:58.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 --2- 172.21.15.113:0/1297392050 >> v2:172.21.15.136:3300/0 conn(0x7f46640c2d80 0x7f466415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:58.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 --2- 172.21.15.113:0/1297392050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46640c22e0 0x7f466415b1f0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:58.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 -- 172.21.15.113:0/1297392050 >> 172.21.15.113:0/1297392050 conn(0x7f466406e820 msgr2=0x7f466406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:58.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 -- 172.21.15.113:0/1297392050 shutdown_connections 2024-03-04T08:29:58.103 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 -- 172.21.15.113:0/1297392050 wait complete. 2024-03-04T08:29:58.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 Processor -- start 2024-03-04T08:29:58.104 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 -- start start 2024-03-04T08:29:58.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46640c22e0 0x7f46640cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:58.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46640c2d80 0x7f4664153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:58.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466414e4c0 0x7f4664153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:58.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46640aede0 con 0x7f466414e4c0 2024-03-04T08:29:58.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466414e8a0 con 0x7f46640c2d80 2024-03-04T08:29:58.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f466b600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4664073390 con 0x7f46640c22e0 2024-03-04T08:29:58.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f4669600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466414e4c0 0x7f4664153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:58.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f4669600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466414e4c0 0x7f4664153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41374/0 (socket says 172.21.15.113:41374) 2024-03-04T08:29:58.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f4669600640 1 -- 172.21.15.113:0/805414064 learned_addr learned my addr 172.21.15.113:0/805414064 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:29:58.106 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f465be00640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46640c2d80 0x7f4664153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:58.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.104+0000 7f4668c00640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46640c22e0 0x7f46640cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:58.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4669600640 1 -- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46640c22e0 msgr2=0x7f46640cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:58.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4669600640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46640c22e0 0x7f46640cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:58.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4669600640 1 -- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46640c2d80 msgr2=0x7f4664153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:58.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4669600640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46640c2d80 0x7f4664153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:58.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4669600640 1 -- 172.21.15.113:0/805414064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46641f5750 con 0x7f466414e4c0 2024-03-04T08:29:58.107 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4668c00640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46640c22e0 0x7f46640cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:29:58.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4669600640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466414e4c0 0x7f4664153da0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f465c007e60 tx=0x7f465c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:58.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4661600640 1 -- 172.21.15.113:0/805414064 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c014420 con 0x7f466414e4c0 2024-03-04T08:29:58.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4661600640 1 -- 172.21.15.113:0/805414064 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c019070 con 0x7f466414e4c0 2024-03-04T08:29:58.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4661600640 1 -- 172.21.15.113:0/805414064 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c014820 con 0x7f466414e4c0 2024-03-04T08:29:58.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f466b600640 1 -- 172.21.15.113:0/805414064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46641f5a30 con 0x7f466414e4c0 2024-03-04T08:29:58.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f466b600640 1 -- 172.21.15.113:0/805414064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46641f5ff0 con 0x7f466414e4c0 2024-03-04T08:29:58.108 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f466b600640 1 -- 172.21.15.113:0/805414064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4664075c00 con 0x7f466414e4c0 2024-03-04T08:29:58.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4661600640 1 -- 172.21.15.113:0/805414064 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f465c0149c0 con 0x7f466414e4c0 2024-03-04T08:29:58.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4661600640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640066f90 0x7f4640069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:29:58.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4661600640 1 -- 172.21.15.113:0/805414064 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f465c086290 con 0x7f466414e4c0 2024-03-04T08:29:58.109 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.108+0000 7f4668c00640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640066f90 0x7f4640069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:29:58.110 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.112+0000 7f4668c00640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640066f90 0x7f4640069450 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4654000f70 tx=0x7f4654009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:29:58.113 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.112+0000 7f4661600640 1 -- 172.21.15.113:0/805414064 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f465c056760 con 0x7f466414e4c0 2024-03-04T08:29:58.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 -- 172.21.15.113:0/805414064 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46640b22c0 con 0x7f466414e4c0 2024-03-04T08:29:58.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f4661600640 1 -- 172.21.15.113:0/805414064 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f465c056060 con 0x7f466414e4c0 2024-03-04T08:29:58.303 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:29:58.304 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:29:58.304 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:29:58.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 -- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640066f90 msgr2=0x7f4640069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:58.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640066f90 0x7f4640069450 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4654000f70 tx=0x7f4654009040 comp rx=0 tx=0).stop 2024-03-04T08:29:58.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 -- 172.21.15.113:0/805414064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466414e4c0 msgr2=0x7f4664153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:29:58.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466414e4c0 0x7f4664153da0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f465c007e60 tx=0x7f465c00e4b0 comp rx=0 tx=0).stop 2024-03-04T08:29:58.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 -- 172.21.15.113:0/805414064 shutdown_connections 2024-03-04T08:29:58.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f4640066f90 0x7f4640069450 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:58.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f466414e4c0 0x7f4664153da0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:58.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46640c2d80 0x7f4664153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:58.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 --2- 172.21.15.113:0/805414064 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f46640c22e0 0x7f46640cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:29:58.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 -- 172.21.15.113:0/805414064 >> 172.21.15.113:0/805414064 conn(0x7f466406e820 msgr2=0x7f46640c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:29:58.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 -- 172.21.15.113:0/805414064 shutdown_connections 2024-03-04T08:29:58.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:29:58.304+0000 7f466b600640 1 -- 172.21.15.113:0/805414064 wait complete. 2024-03-04T08:29:58.324 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:30:00.325 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:00.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70cec00640 1 Processor -- start 2024-03-04T08:30:00.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70cec00640 1 -- start start 2024-03-04T08:30:00.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70cec00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f70c81527f0 0x7f70c8070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:00.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70cec00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f70c80c8200 0x7f70c8072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:00.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70cec00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c81591f0 con 0x7f70c8152bd0 2024-03-04T08:30:00.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70cec00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c8071df0 con 0x7f70c81527f0 2024-03-04T08:30:00.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70cec00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c81597b0 con 0x7f70c80c8200 2024-03-04T08:30:00.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70c7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f70c81527f0 0x7f70c8070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:00.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70c7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f70c81527f0 0x7f70c8070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41378/0 (socket says 172.21.15.113:41378) 2024-03-04T08:30:00.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70c7e00640 1 -- 172.21.15.113:0/2060795691 learned_addr learned my addr 172.21.15.113:0/2060795691 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:00.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70c7400640 1 --2- 172.21.15.113:0/2060795691 >> v2:172.21.15.136:3300/0 conn(0x7f70c80c8200 0x7f70c8072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:00.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70c7e00640 1 -- 172.21.15.113:0/2060795691 >> v2:172.21.15.136:3300/0 conn(0x7f70c80c8200 msgr2=0x7f70c8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:00.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70c7e00640 1 --2- 172.21.15.113:0/2060795691 >> v2:172.21.15.136:3300/0 conn(0x7f70c80c8200 0x7f70c8072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:00.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70c7e00640 1 -- 172.21.15.113:0/2060795691 >> v1:172.21.15.136:6789/0 conn(0x7f70c8152bd0 legacy=0x7f70c81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:00.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.468+0000 7f70c7e00640 1 -- 172.21.15.113:0/2060795691 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c81594d0 con 0x7f70c81527f0 2024-03-04T08:30:00.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70c7e00640 1 --2- 172.21.15.113:0/2060795691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c81527f0 0x7f70c8070d40 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f70c80cd230 tx=0x7f70b801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87515d74c059f785 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:00.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70c6a00640 1 -- 172.21.15.113:0/2060795691 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b80a57d0 con 0x7f70c81527f0 2024-03-04T08:30:00.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70c6a00640 1 -- 172.21.15.113:0/2060795691 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b80a5970 con 0x7f70c81527f0 2024-03-04T08:30:00.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70c6a00640 1 -- 172.21.15.113:0/2060795691 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b80a5d40 con 0x7f70c81527f0 2024-03-04T08:30:00.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70cec00640 1 -- 172.21.15.113:0/2060795691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c81527f0 msgr2=0x7f70c8070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:00.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70cec00640 1 --2- 172.21.15.113:0/2060795691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c81527f0 0x7f70c8070d40 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f70c80cd230 tx=0x7f70b801c9f0 comp rx=0 tx=0).stop 2024-03-04T08:30:00.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70cec00640 1 -- 172.21.15.113:0/2060795691 shutdown_connections 2024-03-04T08:30:00.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70cec00640 1 --2- 172.21.15.113:0/2060795691 >> v2:172.21.15.136:3300/0 conn(0x7f70c80c8200 0x7f70c8072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:00.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70cec00640 1 --2- 172.21.15.113:0/2060795691 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c81527f0 0x7f70c8070d40 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:00.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70cec00640 1 -- 172.21.15.113:0/2060795691 >> 172.21.15.113:0/2060795691 conn(0x7f70c806e820 msgr2=0x7f70c806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:00.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70cec00640 1 -- 172.21.15.113:0/2060795691 shutdown_connections 2024-03-04T08:30:00.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.472+0000 7f70cec00640 1 -- 172.21.15.113:0/2060795691 wait complete. 2024-03-04T08:30:00.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70cec00640 1 Processor -- start 2024-03-04T08:30:00.474 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70cec00640 1 -- start start 2024-03-04T08:30:00.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70cec00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c80c8200 0x7f70c814a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:00.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70cec00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70c81527f0 0x7f70c80b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:00.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70cec00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c8152bd0 0x7f70c80b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:00.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70cec00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c8165160 con 0x7f70c8152bd0 2024-03-04T08:30:00.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70cec00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c80727c0 con 0x7f70c80c8200 2024-03-04T08:30:00.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70cec00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c81652e0 con 0x7f70c81527f0 2024-03-04T08:30:00.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70ccc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c8152bd0 0x7f70c80b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:00.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70ccc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c8152bd0 0x7f70c80b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41380/0 (socket says 172.21.15.113:41380) 2024-03-04T08:30:00.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70ccc00640 1 -- 172.21.15.113:0/2659503652 learned_addr learned my addr 172.21.15.113:0/2659503652 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:00.477 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70c7e00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c80c8200 0x7f70c814a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70c7400640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70c81527f0 0x7f70c80b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70ccc00640 1 -- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70c81527f0 msgr2=0x7f70c80b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70ccc00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70c81527f0 0x7f70c80b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70ccc00640 1 -- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c80c8200 msgr2=0x7f70c814a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70ccc00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c80c8200 0x7f70c814a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70ccc00640 1 -- 172.21.15.113:0/2659503652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c81f6b90 con 0x7f70c8152bd0 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70c7400640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70c81527f0 0x7f70c80b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70ccc00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c8152bd0 0x7f70c80b0fa0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f70c000c370 tx=0x7f70c000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70bfe00640 1 -- 172.21.15.113:0/2659503652 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0005220 con 0x7f70c8152bd0 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70bfe00640 1 -- 172.21.15.113:0/2659503652 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c00053c0 con 0x7f70c8152bd0 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70bfe00640 1 -- 172.21.15.113:0/2659503652 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0007270 con 0x7f70c8152bd0 2024-03-04T08:30:00.478 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70cec00640 1 -- 172.21.15.113:0/2659503652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c81f7a10 con 0x7f70c8152bd0 2024-03-04T08:30:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.476+0000 7f70cec00640 1 -- 172.21.15.113:0/2659503652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c81f70e0 con 0x7f70c8152bd0 2024-03-04T08:30:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.480+0000 7f70cec00640 1 -- 172.21.15.113:0/2659503652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c8075c00 con 0x7f70c8152bd0 2024-03-04T08:30:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.480+0000 7f70bfe00640 1 -- 172.21.15.113:0/2659503652 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70c0019020 con 0x7f70c8152bd0 2024-03-04T08:30:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.480+0000 7f70bfe00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70a8066f60 0x7f70a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:00.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.480+0000 7f70bfe00640 1 -- 172.21.15.113:0/2659503652 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f70c0084bc0 con 0x7f70c8152bd0 2024-03-04T08:30:00.480 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.480+0000 7f70c7e00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70a8066f60 0x7f70a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:00.481 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.480+0000 7f70c7e00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70a8066f60 0x7f70a8069420 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f70b80ae000 tx=0x7f70b80a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:00.484 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.484+0000 7f70bfe00640 1 -- 172.21.15.113:0/2659503652 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f70c0055180 con 0x7f70c8152bd0 2024-03-04T08:30:00.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.668+0000 7f70cec00640 1 -- 172.21.15.113:0/2659503652 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70c81f35f0 con 0x7f70c8152bd0 2024-03-04T08:30:00.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70bfe00640 1 -- 172.21.15.113:0/2659503652 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f70c0054a80 con 0x7f70c8152bd0 2024-03-04T08:30:00.671 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:00.671 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:00.671 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:00.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 -- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70a8066f60 msgr2=0x7f70a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:00.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70a8066f60 0x7f70a8069420 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f70b80ae000 tx=0x7f70b80a7070 comp rx=0 tx=0).stop 2024-03-04T08:30:00.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 -- 172.21.15.113:0/2659503652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c8152bd0 msgr2=0x7f70c80b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:00.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c8152bd0 0x7f70c80b0fa0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f70c000c370 tx=0x7f70c000c830 comp rx=0 tx=0).stop 2024-03-04T08:30:00.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 -- 172.21.15.113:0/2659503652 shutdown_connections 2024-03-04T08:30:00.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f70a8066f60 0x7f70a8069420 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:00.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70c8152bd0 0x7f70c80b0fa0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:00.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f70c81527f0 0x7f70c80b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:00.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 --2- 172.21.15.113:0/2659503652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c80c8200 0x7f70c814a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:00.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 -- 172.21.15.113:0/2659503652 >> 172.21.15.113:0/2659503652 conn(0x7f70c806e820 msgr2=0x7f70c80c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:00.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 -- 172.21.15.113:0/2659503652 shutdown_connections 2024-03-04T08:30:00.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:00.672+0000 7f70cec00640 1 -- 172.21.15.113:0/2659503652 wait complete. 2024-03-04T08:30:00.692 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:30:00.693 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:30:02.693 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:02.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dd1400640 1 Processor -- start 2024-03-04T08:30:02.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dd1400640 1 -- start start 2024-03-04T08:30:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dd1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9dcc1527f0 0x7f9dcc070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:02.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dd1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9dcc0c8200 0x7f9dcc072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dd1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dcc152bd0 0x7f9dcc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dd1400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc071df0 con 0x7f9dcc1527f0 2024-03-04T08:30:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dd1400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dcc1597b0 con 0x7f9dcc0c8200 2024-03-04T08:30:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dd1400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc1591f0 con 0x7f9dcc152bd0 2024-03-04T08:30:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dca800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9dcc1527f0 0x7f9dcc070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dca800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9dcc1527f0 0x7f9dcc070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41388/0 (socket says 172.21.15.113:41388) 2024-03-04T08:30:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dca800640 1 -- 172.21.15.113:0/1379800461 learned_addr learned my addr 172.21.15.113:0/1379800461 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dcb200640 1 --2- 172.21.15.113:0/1379800461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dcc152bd0 0x7f9dcc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:02.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dc9e00640 1 --2- 172.21.15.113:0/1379800461 >> v2:172.21.15.136:3300/0 conn(0x7f9dcc0c8200 0x7f9dcc072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:02.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dca800640 1 -- 172.21.15.113:0/1379800461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dcc152bd0 msgr2=0x7f9dcc1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:02.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dca800640 1 --2- 172.21.15.113:0/1379800461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dcc152bd0 0x7f9dcc1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:02.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dca800640 1 -- 172.21.15.113:0/1379800461 >> v2:172.21.15.136:3300/0 conn(0x7f9dcc0c8200 msgr2=0x7f9dcc072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:02.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dca800640 1 --2- 172.21.15.113:0/1379800461 >> v2:172.21.15.136:3300/0 conn(0x7f9dcc0c8200 0x7f9dcc072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:02.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.844+0000 7f9dca800640 1 -- 172.21.15.113:0/1379800461 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc1594d0 con 0x7f9dcc1527f0 2024-03-04T08:30:02.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dca800640 1 --2- 172.21.15.113:0/1379800461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc1527f0 0x7f9dcc070d40 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0cd230 tx=0x7f9dbc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b14d6fdfe0b838 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:02.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dc9400640 1 -- 172.21.15.113:0/1379800461 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc0a5780 con 0x7f9dcc1527f0 2024-03-04T08:30:02.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dc9400640 1 -- 172.21.15.113:0/1379800461 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc0a5920 con 0x7f9dcc1527f0 2024-03-04T08:30:02.844 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dc9400640 1 -- 172.21.15.113:0/1379800461 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc0a5cf0 con 0x7f9dcc1527f0 2024-03-04T08:30:02.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dd1400640 1 -- 172.21.15.113:0/1379800461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc1527f0 msgr2=0x7f9dcc070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:02.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dd1400640 1 --2- 172.21.15.113:0/1379800461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc1527f0 0x7f9dcc070d40 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0cd230 tx=0x7f9dbc01c9f0 comp rx=0 tx=0).stop 2024-03-04T08:30:02.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dd1400640 1 -- 172.21.15.113:0/1379800461 shutdown_connections 2024-03-04T08:30:02.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dd1400640 1 --2- 172.21.15.113:0/1379800461 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dcc152bd0 0x7f9dcc1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:02.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dd1400640 1 --2- 172.21.15.113:0/1379800461 >> v2:172.21.15.136:3300/0 conn(0x7f9dcc0c8200 0x7f9dcc072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:02.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dd1400640 1 --2- 172.21.15.113:0/1379800461 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc1527f0 0x7f9dcc070d40 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:02.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dd1400640 1 -- 172.21.15.113:0/1379800461 >> 172.21.15.113:0/1379800461 conn(0x7f9dcc06e820 msgr2=0x7f9dcc06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:02.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dd1400640 1 -- 172.21.15.113:0/1379800461 shutdown_connections 2024-03-04T08:30:02.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.848+0000 7f9dd1400640 1 -- 172.21.15.113:0/1379800461 wait complete. 2024-03-04T08:30:02.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 Processor -- start 2024-03-04T08:30:02.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 -- start start 2024-03-04T08:30:02.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dcc0c8200 0x7f9dcc0c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:02.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dcc1527f0 0x7f9dcc1f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:02.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc152bd0 0x7f9dcc1f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:02.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc15bf40 con 0x7f9dcc152bd0 2024-03-04T08:30:02.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc15bdc0 con 0x7f9dcc0c8200 2024-03-04T08:30:02.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dcc15c0c0 con 0x7f9dcc1527f0 2024-03-04T08:30:02.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dcb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc152bd0 0x7f9dcc1f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:02.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dcb200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc152bd0 0x7f9dcc1f43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41390/0 (socket says 172.21.15.113:41390) 2024-03-04T08:30:02.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dcb200640 1 -- 172.21.15.113:0/3408275239 learned_addr learned my addr 172.21.15.113:0/3408275239 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:02.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dca800640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dcc0c8200 0x7f9dcc0c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:02.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dc9e00640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dcc1527f0 0x7f9dcc1f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:02.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dcb200640 1 -- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dcc1527f0 msgr2=0x7f9dcc1f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:02.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dcb200640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dcc1527f0 0x7f9dcc1f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:02.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dcb200640 1 -- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dcc0c8200 msgr2=0x7f9dcc0c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:02.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dcb200640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dcc0c8200 0x7f9dcc0c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:02.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dcb200640 1 -- 172.21.15.113:0/3408275239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dcc1f78e0 con 0x7f9dcc152bd0 2024-03-04T08:30:02.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dcb200640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc152bd0 0x7f9dcc1f43a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400e820 tx=0x7f9dc400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:02.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dc2a00640 1 -- 172.21.15.113:0/3408275239 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4017070 con 0x7f9dcc152bd0 2024-03-04T08:30:02.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dc2a00640 1 -- 172.21.15.113:0/3408275239 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc400c940 con 0x7f9dcc152bd0 2024-03-04T08:30:02.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 -- 172.21.15.113:0/3408275239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dcc158030 con 0x7f9dcc152bd0 2024-03-04T08:30:02.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dc2a00640 1 -- 172.21.15.113:0/3408275239 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400cd10 con 0x7f9dcc152bd0 2024-03-04T08:30:02.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 -- 172.21.15.113:0/3408275239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dcc1f3940 con 0x7f9dcc152bd0 2024-03-04T08:30:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.852+0000 7f9dd1400640 1 -- 172.21.15.113:0/3408275239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dcc075c00 con 0x7f9dcc152bd0 2024-03-04T08:30:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.856+0000 7f9dc2a00640 1 -- 172.21.15.113:0/3408275239 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9dc401a070 con 0x7f9dcc152bd0 2024-03-04T08:30:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.856+0000 7f9dc2a00640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9dac066f60 0x7f9dac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.856+0000 7f9dc2a00640 1 -- 172.21.15.113:0/3408275239 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f9dc4085cc0 con 0x7f9dcc152bd0 2024-03-04T08:30:02.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.856+0000 7f9dca800640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9dac066f60 0x7f9dac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:02.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.856+0000 7f9dca800640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9dac066f60 0x7f9dac069420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0ae000 tx=0x7f9dbc0a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:02.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:02.860+0000 7f9dc2a00640 1 -- 172.21.15.113:0/3408275239 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9dc408c080 con 0x7f9dcc152bd0 2024-03-04T08:30:03.041 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.044+0000 7f9dd1400640 1 -- 172.21.15.113:0/3408275239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dcc0c79d0 con 0x7f9dcc152bd0 2024-03-04T08:30:03.042 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dc2a00640 1 -- 172.21.15.113:0/3408275239 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f9dc4055bd0 con 0x7f9dcc152bd0 2024-03-04T08:30:03.042 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:03.043 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:03.043 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:03.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 -- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9dac066f60 msgr2=0x7f9dac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:03.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9dac066f60 0x7f9dac069420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0ae000 tx=0x7f9dbc0a7070 comp rx=0 tx=0).stop 2024-03-04T08:30:03.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 -- 172.21.15.113:0/3408275239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc152bd0 msgr2=0x7f9dcc1f43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:03.044 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc152bd0 0x7f9dcc1f43a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f9dc400e820 tx=0x7f9dc400ece0 comp rx=0 tx=0).stop 2024-03-04T08:30:03.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 -- 172.21.15.113:0/3408275239 shutdown_connections 2024-03-04T08:30:03.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9dac066f60 0x7f9dac069420 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:03.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dcc152bd0 0x7f9dcc1f43a0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:03.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9dcc1527f0 0x7f9dcc1f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:03.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 --2- 172.21.15.113:0/3408275239 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9dcc0c8200 0x7f9dcc0c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:03.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 -- 172.21.15.113:0/3408275239 >> 172.21.15.113:0/3408275239 conn(0x7f9dcc06e820 msgr2=0x7f9dcc0c9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:03.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 -- 172.21.15.113:0/3408275239 shutdown_connections 2024-03-04T08:30:03.045 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:03.048+0000 7f9dd1400640 1 -- 172.21.15.113:0/3408275239 wait complete. 2024-03-04T08:30:03.063 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:30:05.064 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:05.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea8800640 1 Processor -- start 2024-03-04T08:30:05.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea8800640 1 -- start start 2024-03-04T08:30:05.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea8800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2ea015aa40 0x7f2ea0154800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:05.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea8800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea015aed0 con 0x7f2ea00ccf20 2024-03-04T08:30:05.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea8800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea015b170 con 0x7f2ea00ccb40 2024-03-04T08:30:05.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea8800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea015b410 con 0x7f2ea015aa40 2024-03-04T08:30:05.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2ea015aa40 0x7f2ea0154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:05.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2ea015aa40 0x7f2ea0154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41396/0 (socket says 172.21.15.113:41396) 2024-03-04T08:30:05.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea5400640 1 -- 172.21.15.113:0/3728603204 learned_addr learned my addr 172.21.15.113:0/3728603204 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:05.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea5400640 1 -- 172.21.15.113:0/3728603204 >> v1:172.21.15.136:6789/0 conn(0x7f2ea00ccb40 legacy=0x7f2ea00c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:05.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea5400640 1 -- 172.21.15.113:0/3728603204 >> v1:172.21.15.113:6789/0 conn(0x7f2ea00ccf20 legacy=0x7f2ea015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:05.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.216+0000 7f2ea5400640 1 -- 172.21.15.113:0/3728603204 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea015b9b0 con 0x7f2ea015aa40 2024-03-04T08:30:05.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea5400640 1 --2- 172.21.15.113:0/3728603204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea015aa40 0x7f2ea0154800 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f2e900098a0 tx=0x7f2e9001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da4a0dc2fe9706d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:05.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea4a00640 1 -- 172.21.15.113:0/3728603204 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e900b5070 con 0x7f2ea015aa40 2024-03-04T08:30:05.215 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea4a00640 1 -- 172.21.15.113:0/3728603204 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e900a8a30 con 0x7f2ea015aa40 2024-03-04T08:30:05.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea4a00640 1 -- 172.21.15.113:0/3728603204 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e900a8e00 con 0x7f2ea015aa40 2024-03-04T08:30:05.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea8800640 1 -- 172.21.15.113:0/3728603204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea015aa40 msgr2=0x7f2ea0154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:05.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea8800640 1 --2- 172.21.15.113:0/3728603204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea015aa40 0x7f2ea0154800 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f2e900098a0 tx=0x7f2e9001ed30 comp rx=0 tx=0).stop 2024-03-04T08:30:05.216 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea8800640 1 -- 172.21.15.113:0/3728603204 shutdown_connections 2024-03-04T08:30:05.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea8800640 1 --2- 172.21.15.113:0/3728603204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea015aa40 0x7f2ea0154800 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:05.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea8800640 1 -- 172.21.15.113:0/3728603204 >> 172.21.15.113:0/3728603204 conn(0x7f2ea006b060 msgr2=0x7f2ea006b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:05.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea8800640 1 -- 172.21.15.113:0/3728603204 shutdown_connections 2024-03-04T08:30:05.217 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea8800640 1 -- 172.21.15.113:0/3728603204 wait complete. 2024-03-04T08:30:05.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea8800640 1 Processor -- start 2024-03-04T08:30:05.218 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.220+0000 7f2ea8800640 1 -- start start 2024-03-04T08:30:05.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea8800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ea00ccb40 0x7f2ea00730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:05.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea8800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ea00ccf20 0x7f2ea015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:05.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea8800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea014a060 0x7f2ea014a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:05.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea8800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea015ed00 con 0x7f2ea014a060 2024-03-04T08:30:05.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea8800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea015e0d0 con 0x7f2ea00ccf20 2024-03-04T08:30:05.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea8800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea015ee80 con 0x7f2ea00ccb40 2024-03-04T08:30:05.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea014a060 0x7f2ea014a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:05.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea6800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea014a060 0x7f2ea014a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:41412/0 (socket says 172.21.15.113:41412) 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea6800640 1 -- 172.21.15.113:0/14207646 learned_addr learned my addr 172.21.15.113:0/14207646 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea5400640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ea00ccf20 0x7f2ea015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea5e00640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ea00ccb40 0x7f2ea00730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea6800640 1 -- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ea00ccb40 msgr2=0x7f2ea00730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea6800640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ea00ccb40 0x7f2ea00730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea6800640 1 -- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ea00ccf20 msgr2=0x7f2ea015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea6800640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ea00ccf20 0x7f2ea015bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea6800640 1 -- 172.21.15.113:0/14207646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea0153330 con 0x7f2ea014a060 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea5400640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ea00ccf20 0x7f2ea015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea6800640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea014a060 0x7f2ea014a4e0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00c770 tx=0x7f2e9c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2e9a000640 1 -- 172.21.15.113:0/14207646 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c003b80 con 0x7f2ea014a060 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2e9a000640 1 -- 172.21.15.113:0/14207646 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c003d20 con 0x7f2ea014a060 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2e9a000640 1 -- 172.21.15.113:0/14207646 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00aa30 con 0x7f2ea014a060 2024-03-04T08:30:05.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea8800640 1 -- 172.21.15.113:0/14207646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea01f10f0 con 0x7f2ea014a060 2024-03-04T08:30:05.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea8800640 1 -- 172.21.15.113:0/14207646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea01538b0 con 0x7f2ea014a060 2024-03-04T08:30:05.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.224+0000 7f2ea8800640 1 -- 172.21.15.113:0/14207646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea0075c00 con 0x7f2ea014a060 2024-03-04T08:30:05.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.228+0000 7f2e9a000640 1 -- 172.21.15.113:0/14207646 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e9c00abd0 con 0x7f2ea014a060 2024-03-04T08:30:05.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.228+0000 7f2e9a000640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e84066f90 0x7f2e84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:05.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.228+0000 7f2e9a000640 1 -- 172.21.15.113:0/14207646 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f2e9c085b70 con 0x7f2ea014a060 2024-03-04T08:30:05.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.228+0000 7f2ea5e00640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e84066f90 0x7f2e84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:05.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.228+0000 7f2ea5e00640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e84066f90 0x7f2e84069450 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2e94000f70 tx=0x7f2e94009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:05.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.232+0000 7f2e9a000640 1 -- 172.21.15.113:0/14207646 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2e9c055fb0 con 0x7f2ea014a060 2024-03-04T08:30:05.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.416+0000 7f2ea8800640 1 -- 172.21.15.113:0/14207646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ea0071620 con 0x7f2ea014a060 2024-03-04T08:30:05.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.416+0000 7f2e9a000640 1 -- 172.21.15.113:0/14207646 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f2e9c0558b0 con 0x7f2ea014a060 2024-03-04T08:30:05.414 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:05.414 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:05.414 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:05.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 -- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e84066f90 msgr2=0x7f2e84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e84066f90 0x7f2e84069450 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2e94000f70 tx=0x7f2e94009040 comp rx=0 tx=0).stop 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 -- 172.21.15.113:0/14207646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea014a060 msgr2=0x7f2ea014a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea014a060 0x7f2ea014a4e0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00c770 tx=0x7f2e9c00cc30 comp rx=0 tx=0).stop 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 -- 172.21.15.113:0/14207646 shutdown_connections 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f2e84066f90 0x7f2e84069450 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ea014a060 0x7f2ea014a4e0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ea00ccf20 0x7f2ea015bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 --2- 172.21.15.113:0/14207646 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f2ea00ccb40 0x7f2ea00730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 -- 172.21.15.113:0/14207646 >> 172.21.15.113:0/14207646 conn(0x7f2ea006b060 msgr2=0x7f2ea00cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 -- 172.21.15.113:0/14207646 shutdown_connections 2024-03-04T08:30:05.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:05.420+0000 7f2ea8800640 1 -- 172.21.15.113:0/14207646 wait complete. 2024-03-04T08:30:05.435 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:30:07.436 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:07.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.584+0000 7fa653200640 1 Processor -- start 2024-03-04T08:30:07.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa653200640 1 -- start start 2024-03-04T08:30:07.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa653200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 0x7fa65415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:07.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa653200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6540c4ce0 con 0x7fa6540cb540 2024-03-04T08:30:07.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa653200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6540c4f80 con 0x7fa65406faa0 2024-03-04T08:30:07.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa653200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540c8370 con 0x7fa654152950 2024-03-04T08:30:07.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa652800640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fa6540cb540 0x7fa654162ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57700/0 (socket says 172.21.15.113:57700) 2024-03-04T08:30:07.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa652800640 1 -- 172.21.15.113:0/540977925 learned_addr learned my addr 172.21.15.113:0/540977925 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:07.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa651e00640 1 --2- 172.21.15.113:0/540977925 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 0x7fa65415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:07.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa651400640 1 -- 172.21.15.113:0/540977925 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553590844 0 0) 0x7fa6540c4ce0 con 0x7fa6540cb540 2024-03-04T08:30:07.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa651400640 1 -- 172.21.15.113:0/540977925 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa638003530 con 0x7fa6540cb540 2024-03-04T08:30:07.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa651e00640 1 -- 172.21.15.113:0/540977925 >> v1:172.21.15.136:6789/0 conn(0x7fa65406faa0 legacy=0x7fa65406ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:07.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa651e00640 1 -- 172.21.15.113:0/540977925 >> v1:172.21.15.113:6789/0 conn(0x7fa6540cb540 legacy=0x7fa654162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:07.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa651400640 1 -- 172.21.15.113:0/540977925 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274441515 0 0) 0x7fa6540c4f80 con 0x7fa65406faa0 2024-03-04T08:30:07.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.588+0000 7fa651e00640 1 -- 172.21.15.113:0/540977925 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6540c5220 con 0x7fa654152950 2024-03-04T08:30:07.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa651e00640 1 --2- 172.21.15.113:0/540977925 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 0x7fa65415bb40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa6440097f0 tx=0x7fa64401ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f8ff4b7e9d89fde server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:07.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa651400640 1 -- 172.21.15.113:0/540977925 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440b5070 con 0x7fa654152950 2024-03-04T08:30:07.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa651400640 1 -- 172.21.15.113:0/540977925 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6440a89d0 con 0x7fa654152950 2024-03-04T08:30:07.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa651400640 1 -- 172.21.15.113:0/540977925 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440a8d80 con 0x7fa654152950 2024-03-04T08:30:07.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa653200640 1 -- 172.21.15.113:0/540977925 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 msgr2=0x7fa65415bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:07.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa653200640 1 --2- 172.21.15.113:0/540977925 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 0x7fa65415bb40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa6440097f0 tx=0x7fa64401ecd0 comp rx=0 tx=0).stop 2024-03-04T08:30:07.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa653200640 1 -- 172.21.15.113:0/540977925 shutdown_connections 2024-03-04T08:30:07.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa653200640 1 --2- 172.21.15.113:0/540977925 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 0x7fa65415bb40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:07.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa653200640 1 -- 172.21.15.113:0/540977925 >> 172.21.15.113:0/540977925 conn(0x7fa654072f40 msgr2=0x7fa6540706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:07.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa653200640 1 -- 172.21.15.113:0/540977925 shutdown_connections 2024-03-04T08:30:07.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa653200640 1 -- 172.21.15.113:0/540977925 wait complete. 2024-03-04T08:30:07.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa653200640 1 Processor -- start 2024-03-04T08:30:07.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.592+0000 7fa653200640 1 -- start start 2024-03-04T08:30:07.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa653200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa65406faa0 0x7fa6541f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:07.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa653200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6540cb540 0x7fa6540ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:07.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa653200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 0x7fa65414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:07.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa653200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540c9a50 con 0x7fa6540cb540 2024-03-04T08:30:07.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa653200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540ca0a0 con 0x7fa65406faa0 2024-03-04T08:30:07.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa653200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540c9d30 con 0x7fa654152950 2024-03-04T08:30:07.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa649400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6540cb540 0x7fa6540ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:07.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa649400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6540cb540 0x7fa6540ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37362/0 (socket says 172.21.15.113:37362) 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa649400640 1 -- 172.21.15.113:0/1640618025 learned_addr learned my addr 172.21.15.113:0/1640618025 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa651e00640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa65406faa0 0x7fa6541f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa652800640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 0x7fa65414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa649400640 1 -- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 msgr2=0x7fa65414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa649400640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 0x7fa65414eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa649400640 1 -- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa65406faa0 msgr2=0x7fa6541f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa649400640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa65406faa0 0x7fa6541f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa649400640 1 -- 172.21.15.113:0/1640618025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65415af60 con 0x7fa6540cb540 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa651e00640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa65406faa0 0x7fa6541f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa649400640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6540cb540 0x7fa6540ac880 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00c4d0 tx=0x7fa63c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa64aa00640 1 -- 172.21.15.113:0/1640618025 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c017070 con 0x7fa6540cb540 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa64aa00640 1 -- 172.21.15.113:0/1640618025 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c005480 con 0x7fa6540cb540 2024-03-04T08:30:07.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa64aa00640 1 -- 172.21.15.113:0/1640618025 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c003840 con 0x7fa6540cb540 2024-03-04T08:30:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa653200640 1 -- 172.21.15.113:0/1640618025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65415b240 con 0x7fa6540cb540 2024-03-04T08:30:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa653200640 1 -- 172.21.15.113:0/1640618025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65415b800 con 0x7fa6540cb540 2024-03-04T08:30:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa653200640 1 -- 172.21.15.113:0/1640618025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa654075a80 con 0x7fa6540cb540 2024-03-04T08:30:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.596+0000 7fa64aa00640 1 -- 172.21.15.113:0/1640618025 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa63c01a070 con 0x7fa6540cb540 2024-03-04T08:30:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.600+0000 7fa64aa00640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa630066f90 0x7fa630069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.600+0000 7fa64aa00640 1 -- 172.21.15.113:0/1640618025 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fa63c084de0 con 0x7fa6540cb540 2024-03-04T08:30:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.600+0000 7fa651e00640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa630066f90 0x7fa630069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.600+0000 7fa651e00640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa630066f90 0x7fa630069450 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa6440b2000 tx=0x7fa6440aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:07.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.604+0000 7fa64aa00640 1 -- 172.21.15.113:0/1640618025 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa63c055400 con 0x7fa6540cb540 2024-03-04T08:30:07.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.788+0000 7fa653200640 1 -- 172.21.15.113:0/1640618025 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6540c9580 con 0x7fa6540cb540 2024-03-04T08:30:07.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa64aa00640 1 -- 172.21.15.113:0/1640618025 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fa63c054d00 con 0x7fa6540cb540 2024-03-04T08:30:07.787 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:07.787 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:07.787 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 -- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa630066f90 msgr2=0x7fa630069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa630066f90 0x7fa630069450 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa6440b2000 tx=0x7fa6440aa070 comp rx=0 tx=0).stop 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 -- 172.21.15.113:0/1640618025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6540cb540 msgr2=0x7fa6540ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6540cb540 0x7fa6540ac880 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00c4d0 tx=0x7fa63c00c990 comp rx=0 tx=0).stop 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 -- 172.21.15.113:0/1640618025 shutdown_connections 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fa654152950 0x7fa65414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fa630066f90 0x7fa630069450 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa6540cb540 0x7fa6540ac880 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 --2- 172.21.15.113:0/1640618025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa65406faa0 0x7fa6541f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 -- 172.21.15.113:0/1640618025 >> 172.21.15.113:0/1640618025 conn(0x7fa654072f40 msgr2=0x7fa654151f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 -- 172.21.15.113:0/1640618025 shutdown_connections 2024-03-04T08:30:07.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:07.792+0000 7fa653200640 1 -- 172.21.15.113:0/1640618025 wait complete. 2024-03-04T08:30:07.808 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:30:09.809 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:09.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd622000640 1 Processor -- start 2024-03-04T08:30:09.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd622000640 1 -- start start 2024-03-04T08:30:09.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd622000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fd61c0b20b0 0x7fd61c157710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:09.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd622000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd61c0c85f0 0x7fd61c161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:09.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd622000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61c0c1ec0 con 0x7fd61c157c50 2024-03-04T08:30:09.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd622000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61c0c9b30 con 0x7fd61c0c85f0 2024-03-04T08:30:09.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd622000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61c0c12a0 con 0x7fd61c0b20b0 2024-03-04T08:30:09.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd612a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd61c157c50 0x7fd61c06b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57702/0 (socket says 172.21.15.113:57702) 2024-03-04T08:30:09.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd612a00640 1 -- 172.21.15.113:0/1199978604 learned_addr learned my addr 172.21.15.113:0/1199978604 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:09.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd61be00640 1 --2- 172.21.15.113:0/1199978604 >> v2:172.21.15.113:3300/0 conn(0x7fd61c0c85f0 0x7fd61c161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:09.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd61b400640 1 --2- 172.21.15.113:0/1199978604 >> v2:172.21.15.136:3300/0 conn(0x7fd61c0b20b0 0x7fd61c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:09.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd61be00640 1 -- 172.21.15.113:0/1199978604 >> v2:172.21.15.136:3300/0 conn(0x7fd61c0b20b0 msgr2=0x7fd61c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:09.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd61be00640 1 --2- 172.21.15.113:0/1199978604 >> v2:172.21.15.136:3300/0 conn(0x7fd61c0b20b0 0x7fd61c157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:09.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd61be00640 1 -- 172.21.15.113:0/1199978604 >> v1:172.21.15.113:6789/0 conn(0x7fd61c157c50 legacy=0x7fd61c06b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:09.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd61be00640 1 -- 172.21.15.113:0/1199978604 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c0c1580 con 0x7fd61c0c85f0 2024-03-04T08:30:09.958 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.960+0000 7fd61b400640 1 --2- 172.21.15.113:0/1199978604 >> v2:172.21.15.136:3300/0 conn(0x7fd61c0b20b0 0x7fd61c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:09.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd61be00640 1 --2- 172.21.15.113:0/1199978604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0c85f0 0x7fd61c161820 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fd61c070670 tx=0x7fd614020e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d5ab75e4834a03 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:09.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd61aa00640 1 -- 172.21.15.113:0/1199978604 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140b7070 con 0x7fd61c0c85f0 2024-03-04T08:30:09.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd61aa00640 1 -- 172.21.15.113:0/1199978604 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6140aab80 con 0x7fd61c0c85f0 2024-03-04T08:30:09.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd61aa00640 1 -- 172.21.15.113:0/1199978604 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140b2440 con 0x7fd61c0c85f0 2024-03-04T08:30:09.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd622000640 1 -- 172.21.15.113:0/1199978604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0c85f0 msgr2=0x7fd61c161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:09.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd622000640 1 --2- 172.21.15.113:0/1199978604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0c85f0 0x7fd61c161820 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fd61c070670 tx=0x7fd614020e80 comp rx=0 tx=0).stop 2024-03-04T08:30:09.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd622000640 1 -- 172.21.15.113:0/1199978604 shutdown_connections 2024-03-04T08:30:09.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd622000640 1 --2- 172.21.15.113:0/1199978604 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0c85f0 0x7fd61c161820 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:09.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd622000640 1 --2- 172.21.15.113:0/1199978604 >> v2:172.21.15.136:3300/0 conn(0x7fd61c0b20b0 0x7fd61c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:09.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd622000640 1 -- 172.21.15.113:0/1199978604 >> 172.21.15.113:0/1199978604 conn(0x7fd61c06e820 msgr2=0x7fd61c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:09.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd622000640 1 -- 172.21.15.113:0/1199978604 shutdown_connections 2024-03-04T08:30:09.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.964+0000 7fd622000640 1 -- 172.21.15.113:0/1199978604 wait complete. 2024-03-04T08:30:09.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd622000640 1 Processor -- start 2024-03-04T08:30:09.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd622000640 1 -- start start 2024-03-04T08:30:09.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd622000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0b20b0 0x7fd61c14dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:09.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd622000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c0c85f0 0x7fd61c072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:09.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd622000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd61c157c50 0x7fd61c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:09.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd622000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c0c2400 con 0x7fd61c0b20b0 2024-03-04T08:30:09.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd622000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c1571f0 con 0x7fd61c0c85f0 2024-03-04T08:30:09.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd622000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61c156ec0 con 0x7fd61c157c50 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd61b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0b20b0 0x7fd61c14dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd61b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0b20b0 0x7fd61c14dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37374/0 (socket says 172.21.15.113:37374) 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd61b400640 1 -- 172.21.15.113:0/4084767921 learned_addr learned my addr 172.21.15.113:0/4084767921 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd612a00640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c0c85f0 0x7fd61c072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd61be00640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd61c157c50 0x7fd61c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd61b400640 1 -- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd61c157c50 msgr2=0x7fd61c0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd61b400640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd61c157c50 0x7fd61c0c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd61b400640 1 -- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c0c85f0 msgr2=0x7fd61c072660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd61b400640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c0c85f0 0x7fd61c072660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd61b400640 1 -- 172.21.15.113:0/4084767921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61c1547b0 con 0x7fd61c0b20b0 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd61b400640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0b20b0 0x7fd61c14dad0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fd60c007f20 tx=0x7fd60c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd613e00640 1 -- 172.21.15.113:0/4084767921 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01a070 con 0x7fd61c0b20b0 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd613e00640 1 -- 172.21.15.113:0/4084767921 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c003ba0 con 0x7fd61c0b20b0 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd613e00640 1 -- 172.21.15.113:0/4084767921 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c004030 con 0x7fd61c0b20b0 2024-03-04T08:30:09.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd622000640 1 -- 172.21.15.113:0/4084767921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61c14f010 con 0x7fd61c0b20b0 2024-03-04T08:30:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.968+0000 7fd622000640 1 -- 172.21.15.113:0/4084767921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61c14f5d0 con 0x7fd61c0b20b0 2024-03-04T08:30:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.972+0000 7fd622000640 1 -- 172.21.15.113:0/4084767921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61c075c00 con 0x7fd61c0b20b0 2024-03-04T08:30:09.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.972+0000 7fd613e00640 1 -- 172.21.15.113:0/4084767921 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd60c01d070 con 0x7fd61c0b20b0 2024-03-04T08:30:09.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.972+0000 7fd613e00640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5f8066f60 0x7fd5f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:09.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.972+0000 7fd613e00640 1 -- 172.21.15.113:0/4084767921 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fd60c085cd0 con 0x7fd61c0b20b0 2024-03-04T08:30:09.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.972+0000 7fd612a00640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5f8066f60 0x7fd5f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:09.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.972+0000 7fd612a00640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5f8066f60 0x7fd5f8069420 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd61c150710 tx=0x7fd604009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:09.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:09.976+0000 7fd613e00640 1 -- 172.21.15.113:0/4084767921 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd60c010070 con 0x7fd61c0b20b0 2024-03-04T08:30:10.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.164+0000 7fd622000640 1 -- 172.21.15.113:0/4084767921 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd61c0c83d0 con 0x7fd61c0b20b0 2024-03-04T08:30:10.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd613e00640 1 -- 172.21.15.113:0/4084767921 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fd60c055b50 con 0x7fd61c0b20b0 2024-03-04T08:30:10.163 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:10.163 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:10.163 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 -- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5f8066f60 msgr2=0x7fd5f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5f8066f60 0x7fd5f8069420 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd61c150710 tx=0x7fd604009040 comp rx=0 tx=0).stop 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 -- 172.21.15.113:0/4084767921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0b20b0 msgr2=0x7fd61c14dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0b20b0 0x7fd61c14dad0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fd60c007f20 tx=0x7fd60c00d560 comp rx=0 tx=0).stop 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 -- 172.21.15.113:0/4084767921 shutdown_connections 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd61c157c50 0x7fd61c0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c0c85f0 0x7fd61c072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd5f8066f60 0x7fd5f8069420 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 --2- 172.21.15.113:0/4084767921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd61c0b20b0 0x7fd61c14dad0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 -- 172.21.15.113:0/4084767921 >> 172.21.15.113:0/4084767921 conn(0x7fd61c06e820 msgr2=0x7fd61c0ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 -- 172.21.15.113:0/4084767921 shutdown_connections 2024-03-04T08:30:10.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:10.168+0000 7fd622000640 1 -- 172.21.15.113:0/4084767921 wait complete. 2024-03-04T08:30:10.185 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:30:12.186 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:12.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.332+0000 7ff7f7200640 1 Processor -- start 2024-03-04T08:30:12.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.332+0000 7ff7f7200640 1 -- start start 2024-03-04T08:30:12.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7f7200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f015b900 0x7ff7f0163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:12.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7f7200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f00c27f0 con 0x7ff7f00c22e0 2024-03-04T08:30:12.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7f7200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f00c2a90 con 0x7ff7f014e4c0 2024-03-04T08:30:12.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7f7200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f015ad60 con 0x7ff7f015b900 2024-03-04T08:30:12.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7e7400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff7f00c22e0 0x7ff7f015b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57712/0 (socket says 172.21.15.113:57712) 2024-03-04T08:30:12.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7e7400640 1 -- 172.21.15.113:0/751443122 learned_addr learned my addr 172.21.15.113:0/751443122 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:12.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7f5200640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f015b900 0x7ff7f0163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:12.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049834276 0 0) 0x7ff7f00c27f0 con 0x7ff7f00c22e0 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d4003530 con 0x7ff7f00c22e0 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 156160653 0 0) 0x7ff7d4003530 con 0x7ff7f00c22e0 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f015b900 msgr2=0x7ff7f0163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f015b900 0x7ff7f0163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 >> v1:172.21.15.136:6789/0 conn(0x7ff7f014e4c0 legacy=0x7ff7f0072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f0070c60 con 0x7ff7f00c22e0 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7ff7d8001680 con 0x7ff7f00c22e0 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 >> v1:172.21.15.113:6789/0 conn(0x7ff7f00c22e0 legacy=0x7ff7f015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d40039f0 0x7ff7d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d40043a0 0x7ff7d4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f015b900 0x7ff7d40050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d400b9d0 con 0x7ff7d40043a0 2024-03-04T08:30:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d400bb50 con 0x7ff7d40039f0 2024-03-04T08:30:12.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d400df60 con 0x7ff7f015b900 2024-03-04T08:30:12.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7e7400640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d40043a0 0x7ff7d4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:12.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7efe00640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d40039f0 0x7ff7d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:12.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7f5200640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f015b900 0x7ff7d40050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:12.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7e7400640 1 -- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f015b900 msgr2=0x7ff7d40050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:12.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7e7400640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f015b900 0x7ff7d40050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:12.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7e7400640 1 -- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d40039f0 msgr2=0x7ff7d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:12.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7e7400640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d40039f0 0x7ff7d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:12.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7e7400640 1 -- 172.21.15.113:0/751443122 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4003530 con 0x7ff7d40043a0 2024-03-04T08:30:12.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.336+0000 7ff7f5200640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f015b900 0x7ff7d40050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:12.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7e7400640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d40043a0 0x7ff7d4004870 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800c160 tx=0x7ff7d801f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2a1b24e411810 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:12.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8007cb0 con 0x7ff7d40043a0 2024-03-04T08:30:12.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d8005ae0 con 0x7ff7d40043a0 2024-03-04T08:30:12.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7ef400640 1 -- 172.21.15.113:0/751443122 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8007070 con 0x7ff7d40043a0 2024-03-04T08:30:12.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7f7200640 1 -- 172.21.15.113:0/751443122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d40043a0 msgr2=0x7ff7d4004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:12.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7f7200640 1 --2- 172.21.15.113:0/751443122 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7d40043a0 0x7ff7d4004870 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800c160 tx=0x7ff7d801f350 comp rx=0 tx=0).stop 2024-03-04T08:30:12.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7f5200640 1 -- 172.21.15.113:0/751443122 reap_dead start 2024-03-04T08:30:12.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7f7200640 1 -- 172.21.15.113:0/751443122 shutdown_connections 2024-03-04T08:30:12.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7f7200640 1 -- 172.21.15.113:0/751443122 >> 172.21.15.113:0/751443122 conn(0x7ff7f006e820 msgr2=0x7ff7f006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:12.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7f7200640 1 -- 172.21.15.113:0/751443122 shutdown_connections 2024-03-04T08:30:12.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7f7200640 1 -- 172.21.15.113:0/751443122 wait complete. 2024-03-04T08:30:12.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7f7200640 1 Processor -- start 2024-03-04T08:30:12.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.340+0000 7ff7f7200640 1 -- start start 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7f7200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d40043a0 0x7ff7f014f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7f7200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7f0156470 0x7ff7f01568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7f7200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f00cae80 0x7ff7f00cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7f7200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f0071500 con 0x7ff7f0156470 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7f7200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f0071220 con 0x7ff7d40043a0 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7f7200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f014e8a0 con 0x7ff7f00cae80 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7e7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7f0156470 0x7ff7f01568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7e7400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7f0156470 0x7ff7f01568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37404/0 (socket says 172.21.15.113:37404) 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7e7400640 1 -- 172.21.15.113:0/1571049982 learned_addr learned my addr 172.21.15.113:0/1571049982 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7efe00640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d40043a0 0x7ff7f014f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:12.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7f5200640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f00cae80 0x7ff7f00cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:12.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7e7400640 1 -- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f00cae80 msgr2=0x7ff7f00cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:12.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7e7400640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f00cae80 0x7ff7f00cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:12.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7e7400640 1 -- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d40043a0 msgr2=0x7ff7f014f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:12.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7e7400640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d40043a0 0x7ff7f014f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:12.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7e7400640 1 -- 172.21.15.113:0/1571049982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f00cb840 con 0x7ff7f0156470 2024-03-04T08:30:12.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7efe00640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d40043a0 0x7ff7f014f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7e7400640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7f0156470 0x7ff7f01568b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7ff7d801f180 tx=0x7ff7d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7ecc00640 1 -- 172.21.15.113:0/1571049982 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8005c90 con 0x7ff7f0156470 2024-03-04T08:30:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7ecc00640 1 -- 172.21.15.113:0/1571049982 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d8005e30 con 0x7ff7f0156470 2024-03-04T08:30:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7ecc00640 1 -- 172.21.15.113:0/1571049982 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8004a10 con 0x7ff7f0156470 2024-03-04T08:30:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7f7200640 1 -- 172.21.15.113:0/1571049982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f01571a0 con 0x7ff7f0156470 2024-03-04T08:30:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7f7200640 1 -- 172.21.15.113:0/1571049982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f01efd20 con 0x7ff7f0156470 2024-03-04T08:30:12.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7f7200640 1 -- 172.21.15.113:0/1571049982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f0075c00 con 0x7ff7f0156470 2024-03-04T08:30:12.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.344+0000 7ff7ecc00640 1 -- 172.21.15.113:0/1571049982 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7d8007040 con 0x7ff7f0156470 2024-03-04T08:30:12.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.348+0000 7ff7ecc00640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7cc066fb0 0x7ff7cc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:12.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.348+0000 7ff7ecc00640 1 -- 172.21.15.113:0/1571049982 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7ff7d8122be0 con 0x7ff7f0156470 2024-03-04T08:30:12.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.348+0000 7ff7efe00640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7cc066fb0 0x7ff7cc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:12.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.348+0000 7ff7efe00640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7cc066fb0 0x7ff7cc069470 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0007fd0 tx=0x7ff7e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:12.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.352+0000 7ff7ecc00640 1 -- 172.21.15.113:0/1571049982 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff7f00cb840 con 0x7ff7f0156470 2024-03-04T08:30:12.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.536+0000 7ff7f7200640 1 -- 172.21.15.113:0/1571049982 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7f0073530 con 0x7ff7f0156470 2024-03-04T08:30:12.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.536+0000 7ff7ecc00640 1 -- 172.21.15.113:0/1571049982 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7ff7d8019060 con 0x7ff7f0156470 2024-03-04T08:30:12.534 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:12.534 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:12.534 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:12.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 -- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7cc066fb0 msgr2=0x7ff7cc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:12.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7cc066fb0 0x7ff7cc069470 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0007fd0 tx=0x7ff7e0009040 comp rx=0 tx=0).stop 2024-03-04T08:30:12.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 -- 172.21.15.113:0/1571049982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7f0156470 msgr2=0x7ff7f01568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:12.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7f0156470 0x7ff7f01568b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7ff7d801f180 tx=0x7ff7d4005840 comp rx=0 tx=0).stop 2024-03-04T08:30:12.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 -- 172.21.15.113:0/1571049982 shutdown_connections 2024-03-04T08:30:12.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff7f00cae80 0x7ff7f00cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:12.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7ff7cc066fb0 0x7ff7cc069470 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:12.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7f0156470 0x7ff7f01568b0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:12.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 --2- 172.21.15.113:0/1571049982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff7d40043a0 0x7ff7f014f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:12.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 -- 172.21.15.113:0/1571049982 >> 172.21.15.113:0/1571049982 conn(0x7ff7f006e820 msgr2=0x7ff7f006bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:12.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 -- 172.21.15.113:0/1571049982 shutdown_connections 2024-03-04T08:30:12.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:12.540+0000 7ff7f7200640 1 -- 172.21.15.113:0/1571049982 wait complete. 2024-03-04T08:30:12.555 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:30:12.556 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:30:14.557 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:14.703 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f6703200640 1 Processor -- start 2024-03-04T08:30:14.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f6703200640 1 -- start start 2024-03-04T08:30:14.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f6703200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc070010 0x7f66fc0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:14.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f6703200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc152f00 con 0x7f66fc15c510 2024-03-04T08:30:14.704 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f6703200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc0c5040 con 0x7f66fc152b20 2024-03-04T08:30:14.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f6703200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0c5320 con 0x7f66fc070010 2024-03-04T08:30:14.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f6701200640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f66fc15c510 0x7f66fc163190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:57724/0 (socket says 172.21.15.113:57724) 2024-03-04T08:30:14.705 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f6701200640 1 -- 172.21.15.113:0/960352370 learned_addr learned my addr 172.21.15.113:0/960352370 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66f3400640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc070010 0x7f66fc0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206723662 0 0) 0x7f66fc152f00 con 0x7f66fc15c510 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66e0003530 con 0x7f66fc15c510 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 725978895 0 0) 0x7f66e0003530 con 0x7f66fc15c510 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc070010 msgr2=0x7f66fc0c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc070010 0x7f66fc0c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 >> v1:172.21.15.136:6789/0 conn(0x7f66fc152b20 legacy=0x7f66fc15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc0c3250 con 0x7f66fc15c510 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f66f4001a00 con 0x7f66fc15c510 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 >> v1:172.21.15.113:6789/0 conn(0x7f66fc15c510 legacy=0x7f66fc163190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:14.707 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 0x7f66e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc070010 0x7f66e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66e0004d50 0x7f66e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0c5320 con 0x7f66e00039f0 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e000b9d0 con 0x7f66e0004d50 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e000bbd0 con 0x7f66fc070010 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.708+0000 7f66fbe00640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 0x7f66e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f66f3400640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc070010 0x7f66e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f6701200640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66e0004d50 0x7f66e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f66fbe00640 1 -- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc070010 msgr2=0x7f66e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f66fbe00640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc070010 0x7f66e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f66fbe00640 1 -- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66e0004d50 msgr2=0x7f66e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f66fbe00640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66e0004d50 0x7f66e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f66fbe00640 1 -- 172.21.15.113:0/960352370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e0003530 con 0x7f66e00039f0 2024-03-04T08:30:14.708 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f6701200640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66e0004d50 0x7f66e00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:14.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f66fbe00640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 0x7f66e0003dd0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f66ec001910 tx=0x7f66ec01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be34430c396d9768 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:14.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0b0070 con 0x7f66e00039f0 2024-03-04T08:30:14.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec0ab620 con 0x7f66e00039f0 2024-03-04T08:30:14.709 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f66fb400640 1 -- 172.21.15.113:0/960352370 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec001510 con 0x7f66e00039f0 2024-03-04T08:30:14.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f6703200640 1 -- 172.21.15.113:0/960352370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 msgr2=0x7f66e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:14.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.712+0000 7f6703200640 1 --2- 172.21.15.113:0/960352370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 0x7f66e0003dd0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f66ec001910 tx=0x7f66ec01c4c0 comp rx=0 tx=0).stop 2024-03-04T08:30:14.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6701200640 1 -- 172.21.15.113:0/960352370 reap_dead start 2024-03-04T08:30:14.710 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 -- 172.21.15.113:0/960352370 shutdown_connections 2024-03-04T08:30:14.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 -- 172.21.15.113:0/960352370 >> 172.21.15.113:0/960352370 conn(0x7f66fc0709a0 msgr2=0x7f66fc070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:14.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 -- 172.21.15.113:0/960352370 shutdown_connections 2024-03-04T08:30:14.711 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 -- 172.21.15.113:0/960352370 wait complete. 2024-03-04T08:30:14.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 Processor -- start 2024-03-04T08:30:14.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 -- start start 2024-03-04T08:30:14.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 0x7f66fc0c6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:14.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66fc155a80 0x7f66fc151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:14.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc0c54d0 0x7f66fc153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:14.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc158810 con 0x7f66e00039f0 2024-03-04T08:30:14.714 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0c3810 con 0x7f66fc155a80 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6703200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0c8490 con 0x7f66fc0c54d0 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f66fbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 0x7f66fc0c6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f66fbe00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 0x7f66fc0c6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37430/0 (socket says 172.21.15.113:37430) 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f66fbe00640 1 -- 172.21.15.113:0/1428885172 learned_addr learned my addr 172.21.15.113:0/1428885172 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f66f3400640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66fc155a80 0x7f66fc151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f6701200640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc0c54d0 0x7f66fc153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f66fbe00640 1 -- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc0c54d0 msgr2=0x7f66fc153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f66fbe00640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc0c54d0 0x7f66fc153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f66fbe00640 1 -- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66fc155a80 msgr2=0x7f66fc151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f66fbe00640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66fc155a80 0x7f66fc151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f66fbe00640 1 -- 172.21.15.113:0/1428885172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc1f3fe0 con 0x7f66e00039f0 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.716+0000 7f66fbe00640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 0x7f66fc0c6140 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f66ec01c9a0 tx=0x7f66ec019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.720+0000 7f66f8c00640 1 -- 172.21.15.113:0/1428885172 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0bb070 con 0x7f66e00039f0 2024-03-04T08:30:14.715 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.720+0000 7f66f8c00640 1 -- 172.21.15.113:0/1428885172 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec0ab400 con 0x7f66e00039f0 2024-03-04T08:30:14.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.720+0000 7f66f8c00640 1 -- 172.21.15.113:0/1428885172 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0b0040 con 0x7f66e00039f0 2024-03-04T08:30:14.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.720+0000 7f6703200640 1 -- 172.21.15.113:0/1428885172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc15bfb0 con 0x7f66e00039f0 2024-03-04T08:30:14.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.720+0000 7f6703200640 1 -- 172.21.15.113:0/1428885172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc15ec70 con 0x7f66e00039f0 2024-03-04T08:30:14.716 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.720+0000 7f6703200640 1 -- 172.21.15.113:0/1428885172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc075c00 con 0x7f66e00039f0 2024-03-04T08:30:14.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.720+0000 7f66f8c00640 1 -- 172.21.15.113:0/1428885172 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66ec0052b0 con 0x7f66e00039f0 2024-03-04T08:30:14.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.720+0000 7f66f8c00640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66d8066f90 0x7f66d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:14.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.720+0000 7f66f8c00640 1 -- 172.21.15.113:0/1428885172 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f66ec11ee80 con 0x7f66e00039f0 2024-03-04T08:30:14.717 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.720+0000 7f66f3400640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66d8066f90 0x7f66d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:14.718 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.724+0000 7f66f3400640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66d8066f90 0x7f66d8069450 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f66fc159490 tx=0x7f66e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:14.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.724+0000 7f66f8c00640 1 -- 172.21.15.113:0/1428885172 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f66ec0161d0 con 0x7f66e00039f0 2024-03-04T08:30:14.907 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 -- 172.21.15.113:0/1428885172 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66fc152450 con 0x7f66e00039f0 2024-03-04T08:30:14.908 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f66f8c00640 1 -- 172.21.15.113:0/1428885172 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f66ec0015b0 con 0x7f66e00039f0 2024-03-04T08:30:14.908 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:14.908 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:14.908 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 -- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66d8066f90 msgr2=0x7f66d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66d8066f90 0x7f66d8069450 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f66fc159490 tx=0x7f66e4008040 comp rx=0 tx=0).stop 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 -- 172.21.15.113:0/1428885172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 msgr2=0x7f66fc0c6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 0x7f66fc0c6140 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f66ec01c9a0 tx=0x7f66ec019260 comp rx=0 tx=0).stop 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 -- 172.21.15.113:0/1428885172 shutdown_connections 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f66fc0c54d0 0x7f66fc153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f66fc155a80 0x7f66fc151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f66d8066f90 0x7f66d8069450 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 --2- 172.21.15.113:0/1428885172 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f66e00039f0 0x7f66fc0c6140 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 -- 172.21.15.113:0/1428885172 >> 172.21.15.113:0/1428885172 conn(0x7f66fc0709a0 msgr2=0x7f66fc157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 -- 172.21.15.113:0/1428885172 shutdown_connections 2024-03-04T08:30:14.910 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:14.912+0000 7f6703200640 1 -- 172.21.15.113:0/1428885172 wait complete. 2024-03-04T08:30:14.929 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:30:16.461 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:30:16.465 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:30:16.930 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:17.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a08e00640 1 Processor -- start 2024-03-04T08:30:17.076 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a08e00640 1 -- start start 2024-03-04T08:30:17.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a08e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f4a0414e4c0 0x7f4a04072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:17.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a08e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a040c22e0 0x7f4a040731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:17.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a08e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a040c27f0 con 0x7f4a040c2d80 2024-03-04T08:30:17.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a08e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a0415ad60 con 0x7f4a040c22e0 2024-03-04T08:30:17.077 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a08e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a040c2a90 con 0x7f4a0414e4c0 2024-03-04T08:30:17.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a01400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a040c22e0 0x7f4a040731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:17.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a01400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4a040c22e0 0x7f4a040731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37446/0 (socket says 172.21.15.113:37446) 2024-03-04T08:30:17.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a01400640 1 -- 172.21.15.113:0/1403263350 learned_addr learned my addr 172.21.15.113:0/1403263350 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:17.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a01e00640 1 --2- 172.21.15.113:0/1403263350 >> v2:172.21.15.136:3300/0 conn(0x7f4a0414e4c0 0x7f4a04072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:17.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a01400640 1 -- 172.21.15.113:0/1403263350 >> v2:172.21.15.136:3300/0 conn(0x7f4a0414e4c0 msgr2=0x7f4a04072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:17.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a01400640 1 --2- 172.21.15.113:0/1403263350 >> v2:172.21.15.136:3300/0 conn(0x7f4a0414e4c0 0x7f4a04072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:17.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a01400640 1 -- 172.21.15.113:0/1403263350 >> v1:172.21.15.113:6789/0 conn(0x7f4a040c2d80 legacy=0x7f4a0415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:17.078 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.080+0000 7f4a01400640 1 -- 172.21.15.113:0/1403263350 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a04070c60 con 0x7f4a040c22e0 2024-03-04T08:30:17.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a01400640 1 --2- 172.21.15.113:0/1403263350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 0x7f4a040731c0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4a0406f690 tx=0x7f49ec01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766eb7ae640125ab server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:17.080 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a00a00640 1 -- 172.21.15.113:0/1403263350 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0b5070 con 0x7f4a040c22e0 2024-03-04T08:30:17.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a00a00640 1 -- 172.21.15.113:0/1403263350 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ec0a8a10 con 0x7f4a040c22e0 2024-03-04T08:30:17.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a00a00640 1 -- 172.21.15.113:0/1403263350 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec0a8de0 con 0x7f4a040c22e0 2024-03-04T08:30:17.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a08e00640 1 -- 172.21.15.113:0/1403263350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 msgr2=0x7f4a040731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:17.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a08e00640 1 --2- 172.21.15.113:0/1403263350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 0x7f4a040731c0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f4a0406f690 tx=0x7f49ec01ed10 comp rx=0 tx=0).stop 2024-03-04T08:30:17.081 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a08e00640 1 -- 172.21.15.113:0/1403263350 shutdown_connections 2024-03-04T08:30:17.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a08e00640 1 --2- 172.21.15.113:0/1403263350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 0x7f4a040731c0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:17.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a08e00640 1 --2- 172.21.15.113:0/1403263350 >> v2:172.21.15.136:3300/0 conn(0x7f4a0414e4c0 0x7f4a04072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:17.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a08e00640 1 -- 172.21.15.113:0/1403263350 >> 172.21.15.113:0/1403263350 conn(0x7f4a0406e820 msgr2=0x7f4a0406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:17.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a08e00640 1 -- 172.21.15.113:0/1403263350 shutdown_connections 2024-03-04T08:30:17.082 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.084+0000 7f4a08e00640 1 -- 172.21.15.113:0/1403263350 wait complete. 2024-03-04T08:30:17.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a08e00640 1 Processor -- start 2024-03-04T08:30:17.083 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a08e00640 1 -- start start 2024-03-04T08:30:17.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a08e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 0x7f4a040b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:17.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a08e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a040c2d80 0x7f4a040af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:17.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a08e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a0414e4c0 0x7f4a0414f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:17.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a08e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04071500 con 0x7f4a040c22e0 2024-03-04T08:30:17.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a08e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a04071220 con 0x7f4a040c2d80 2024-03-04T08:30:17.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a08e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0414e8a0 con 0x7f4a0414e4c0 2024-03-04T08:30:17.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a01e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 0x7f4a040b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:17.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a01e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 0x7f4a040b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:37452/0 (socket says 172.21.15.113:37452) 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a01e00640 1 -- 172.21.15.113:0/1934418444 learned_addr learned my addr 172.21.15.113:0/1934418444 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a02800640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a0414e4c0 0x7f4a0414f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a01400640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a040c2d80 0x7f4a040af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a01e00640 1 -- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a0414e4c0 msgr2=0x7f4a0414f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a01e00640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a0414e4c0 0x7f4a0414f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a01e00640 1 -- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a040c2d80 msgr2=0x7f4a040af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a01e00640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a040c2d80 0x7f4a040af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a01e00640 1 -- 172.21.15.113:0/1934418444 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a041f1670 con 0x7f4a040c22e0 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a01e00640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 0x7f4a040b2010 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f49f400d620 tx=0x7f49f400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f49fa000640 1 -- 172.21.15.113:0/1934418444 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f40125f0 con 0x7f4a040c22e0 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f49fa000640 1 -- 172.21.15.113:0/1934418444 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f4019070 con 0x7f4a040c22e0 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f49fa000640 1 -- 172.21.15.113:0/1934418444 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f40129b0 con 0x7f4a040c22e0 2024-03-04T08:30:17.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a08e00640 1 -- 172.21.15.113:0/1934418444 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0414ec90 con 0x7f4a040c22e0 2024-03-04T08:30:17.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.088+0000 7f4a08e00640 1 -- 172.21.15.113:0/1934418444 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0414f250 con 0x7f4a040c22e0 2024-03-04T08:30:17.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.092+0000 7f4a08e00640 1 -- 172.21.15.113:0/1934418444 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a04075c00 con 0x7f4a040c22e0 2024-03-04T08:30:17.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.092+0000 7f49fa000640 1 -- 172.21.15.113:0/1934418444 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49f4012b50 con 0x7f4a040c22e0 2024-03-04T08:30:17.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.092+0000 7f49fa000640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49e0066f60 0x7f49e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:17.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.092+0000 7f49fa000640 1 -- 172.21.15.113:0/1934418444 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f49f4085f00 con 0x7f4a040c22e0 2024-03-04T08:30:17.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.092+0000 7f4a01400640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49e0066f60 0x7f49e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:17.089 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.092+0000 7f4a01400640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49e0066f60 0x7f49e0069420 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0b2000 tx=0x7f49ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:17.092 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.096+0000 7f49fa000640 1 -- 172.21.15.113:0/1934418444 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f49f40563d0 con 0x7f4a040c22e0 2024-03-04T08:30:17.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.284+0000 7f4a08e00640 1 -- 172.21.15.113:0/1934418444 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a04155190 con 0x7f4a040c22e0 2024-03-04T08:30:17.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.284+0000 7f49fa000640 1 -- 172.21.15.113:0/1934418444 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f49f405c0d0 con 0x7f4a040c22e0 2024-03-04T08:30:17.283 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:17.283 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:17.283 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:17.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 -- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49e0066f60 msgr2=0x7f49e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:17.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49e0066f60 0x7f49e0069420 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0b2000 tx=0x7f49ec009040 comp rx=0 tx=0).stop 2024-03-04T08:30:17.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 -- 172.21.15.113:0/1934418444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 msgr2=0x7f4a040b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:17.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 0x7f4a040b2010 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f49f400d620 tx=0x7f49f400dae0 comp rx=0 tx=0).stop 2024-03-04T08:30:17.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 -- 172.21.15.113:0/1934418444 shutdown_connections 2024-03-04T08:30:17.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f4a0414e4c0 0x7f4a0414f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:17.284 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4a040c2d80 0x7f4a040af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:17.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f49e0066f60 0x7f49e0069420 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:17.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 --2- 172.21.15.113:0/1934418444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4a040c22e0 0x7f4a040b2010 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:17.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 -- 172.21.15.113:0/1934418444 >> 172.21.15.113:0/1934418444 conn(0x7f4a0406e820 msgr2=0x7f4a0406b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:17.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 -- 172.21.15.113:0/1934418444 shutdown_connections 2024-03-04T08:30:17.285 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:17.288+0000 7f4a08e00640 1 -- 172.21.15.113:0/1934418444 wait complete. 2024-03-04T08:30:17.303 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:30:19.304 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:19.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f91a00640 1 Processor -- start 2024-03-04T08:30:19.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f91a00640 1 -- start start 2024-03-04T08:30:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f91a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f9f8c0c7a80 0x7f9f8c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f91a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f8c06cb60 0x7f9f8c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f91a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f8c072d80 con 0x7f9f8c152c60 2024-03-04T08:30:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f91a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f8c06b060 con 0x7f9f8c0c7a80 2024-03-04T08:30:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f91a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c073510 con 0x7f9f8c06cb60 2024-03-04T08:30:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8aa00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9f8c152c60 0x7f9f8c15bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60822/0 (socket says 172.21.15.113:60822) 2024-03-04T08:30:19.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8aa00640 1 -- 172.21.15.113:0/2586539529 learned_addr learned my addr 172.21.15.113:0/2586539529 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f82000640 1 --2- 172.21.15.113:0/2586539529 >> v2:172.21.15.136:3300/0 conn(0x7f9f8c0c7a80 0x7f9f8c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8b400640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f8c06cb60 0x7f9f8c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280589937 0 0) 0x7f9f8c072d80 con 0x7f9f8c152c60 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f70003530 con 0x7f9f8c152c60 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2873309056 0 0) 0x7f9f70003530 con 0x7f9f8c152c60 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f8c06cb60 msgr2=0x7f9f8c0cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f8c06cb60 0x7f9f8c0cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 >> v2:172.21.15.136:3300/0 conn(0x7f9f8c0c7a80 msgr2=0x7f9f8c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 --2- 172.21.15.113:0/2586539529 >> v2:172.21.15.136:3300/0 conn(0x7f9f8c0c7a80 0x7f9f8c0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c06b2b0 con 0x7f9f8c152c60 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7f9f7c001a00 con 0x7f9f8c152c60 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 >> v1:172.21.15.113:6789/0 conn(0x7f9f8c152c60 legacy=0x7f9f8c15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:19.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f700039c0 0x7f9f70003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f8c06cb60 0x7f9f70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f70004d50 0x7f9f700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c06b060 con 0x7f9f70004d50 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c073510 con 0x7f9f700039c0 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7000b9d0 con 0x7f9f8c06cb60 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8b400640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f70004d50 0x7f9f700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f82000640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f8c06cb60 0x7f9f70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.456+0000 7f9f8aa00640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f700039c0 0x7f9f70003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8b400640 1 -- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f8c06cb60 msgr2=0x7f9f70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8b400640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f8c06cb60 0x7f9f70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8b400640 1 -- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f700039c0 msgr2=0x7f9f70003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8b400640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f700039c0 0x7f9f70003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:19.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8b400640 1 -- 172.21.15.113:0/2586539529 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f70003530 con 0x7f9f70004d50 2024-03-04T08:30:19.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8aa00640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f700039c0 0x7f9f70003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8b400640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f70004d50 0x7f9f700051d0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009ae0 tx=0x7f9f8401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279bed8abda13576 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f840b2070 con 0x7f9f70004d50 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f840a5660 con 0x7f9f70004d50 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8a000640 1 -- 172.21.15.113:0/2586539529 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f840096b0 con 0x7f9f70004d50 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f91a00640 1 -- 172.21.15.113:0/2586539529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f70004d50 msgr2=0x7f9f700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f91a00640 1 --2- 172.21.15.113:0/2586539529 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f70004d50 0x7f9f700051d0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009ae0 tx=0x7f9f8401ca70 comp rx=0 tx=0).stop 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f8b400640 1 -- 172.21.15.113:0/2586539529 reap_dead start 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f91a00640 1 -- 172.21.15.113:0/2586539529 shutdown_connections 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f91a00640 1 -- 172.21.15.113:0/2586539529 >> 172.21.15.113:0/2586539529 conn(0x7f9f8c0671b0 msgr2=0x7f9f8c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.460+0000 7f9f91a00640 1 -- 172.21.15.113:0/2586539529 shutdown_connections 2024-03-04T08:30:19.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f91a00640 1 -- 172.21.15.113:0/2586539529 wait complete. 2024-03-04T08:30:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f91a00640 1 Processor -- start 2024-03-04T08:30:19.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f91a00640 1 -- start start 2024-03-04T08:30:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f91a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f70004d50 0x7f9f8c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f91a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f8c15bfd0 0x7f9f8c1e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f91a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f8c14fa70 0x7f9f8c1e6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f91a00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c15c450 con 0x7f9f8c14fa70 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f91a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c072860 con 0x7f9f8c15bfd0 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f91a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8c06b590 con 0x7f9f70004d50 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f8b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f8c14fa70 0x7f9f8c1e6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f8b400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f8c14fa70 0x7f9f8c1e6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35280/0 (socket says 172.21.15.113:35280) 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f8aa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f70004d50 0x7f9f8c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f8b400640 1 -- 172.21.15.113:0/1436717774 learned_addr learned my addr 172.21.15.113:0/1436717774 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f82000640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f8c15bfd0 0x7f9f8c1e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f8b400640 1 -- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f70004d50 msgr2=0x7f9f8c15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f8b400640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f70004d50 0x7f9f8c15ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:19.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f8b400640 1 -- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f8c15bfd0 msgr2=0x7f9f8c1e5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f8b400640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f8c15bfd0 0x7f9f8c1e5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f8b400640 1 -- 172.21.15.113:0/1436717774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c159f30 con 0x7f9f8c14fa70 2024-03-04T08:30:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f8b400640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f8c14fa70 0x7f9f8c1e6140 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009ae0 tx=0x7f9f70005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f83400640 1 -- 172.21.15.113:0/1436717774 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f840b2040 con 0x7f9f8c14fa70 2024-03-04T08:30:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f83400640 1 -- 172.21.15.113:0/1436717774 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f84000e50 con 0x7f9f8c14fa70 2024-03-04T08:30:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f91a00640 1 -- 172.21.15.113:0/1436717774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f8c1f4100 con 0x7f9f8c14fa70 2024-03-04T08:30:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.464+0000 7f9f83400640 1 -- 172.21.15.113:0/1436717774 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f840096b0 con 0x7f9f8c14fa70 2024-03-04T08:30:19.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.468+0000 7f9f91a00640 1 -- 172.21.15.113:0/1436717774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8c1f43a0 con 0x7f9f8c14fa70 2024-03-04T08:30:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.468+0000 7f9f91a00640 1 -- 172.21.15.113:0/1436717774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8c075c00 con 0x7f9f8c14fa70 2024-03-04T08:30:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.468+0000 7f9f83400640 1 -- 172.21.15.113:0/1436717774 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f84004260 con 0x7f9f8c14fa70 2024-03-04T08:30:19.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.468+0000 7f9f83400640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f68066f90 0x7f9f68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.468+0000 7f9f83400640 1 -- 172.21.15.113:0/1436717774 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f9f8411f200 con 0x7f9f8c14fa70 2024-03-04T08:30:19.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.468+0000 7f9f8aa00640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f68066f90 0x7f9f68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:19.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.472+0000 7f9f8aa00640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f68066f90 0x7f9f68069450 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c001340 tx=0x7f9f7c001740 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.472+0000 7f9f83400640 1 -- 172.21.15.113:0/1436717774 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9f8c159f30 con 0x7f9f8c14fa70 2024-03-04T08:30:19.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 -- 172.21.15.113:0/1436717774 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f8c072fb0 con 0x7f9f8c14fa70 2024-03-04T08:30:19.659 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f83400640 1 -- 172.21.15.113:0/1436717774 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f9f84016780 con 0x7f9f8c14fa70 2024-03-04T08:30:19.659 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:19.659 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:19.660 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:19.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 -- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f68066f90 msgr2=0x7f9f68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:19.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f68066f90 0x7f9f68069450 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c001340 tx=0x7f9f7c001740 comp rx=0 tx=0).stop 2024-03-04T08:30:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 -- 172.21.15.113:0/1436717774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f8c14fa70 msgr2=0x7f9f8c1e6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f8c14fa70 0x7f9f8c1e6140 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009ae0 tx=0x7f9f70005840 comp rx=0 tx=0).stop 2024-03-04T08:30:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 -- 172.21.15.113:0/1436717774 shutdown_connections 2024-03-04T08:30:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9f68066f90 0x7f9f68069450 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9f8c14fa70 0x7f9f8c1e6140 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f8c15bfd0 0x7f9f8c1e5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 --2- 172.21.15.113:0/1436717774 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9f70004d50 0x7f9f8c15ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 -- 172.21.15.113:0/1436717774 >> 172.21.15.113:0/1436717774 conn(0x7f9f8c0671b0 msgr2=0x7f9f8c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 -- 172.21.15.113:0/1436717774 shutdown_connections 2024-03-04T08:30:19.662 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:19.664+0000 7f9f91a00640 1 -- 172.21.15.113:0/1436717774 wait complete. 2024-03-04T08:30:19.682 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:30:21.683 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.832+0000 7fbbf3400640 1 Processor -- start 2024-03-04T08:30:21.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.832+0000 7fbbf3400640 1 -- start start 2024-03-04T08:30:21.830 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbf3400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbec14e4c0 0x7fbbec072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbf3400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbec0c27f0 con 0x7fbbec0c22e0 2024-03-04T08:30:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbf3400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbec0c2a90 con 0x7fbbec15b900 2024-03-04T08:30:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbf3400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbec15ad60 con 0x7fbbec14e4c0 2024-03-04T08:30:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbf0a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbec14e4c0 0x7fbbec072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbe3e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbbec0c22e0 0x7fbbec15b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60834/0 (socket says 172.21.15.113:60834) 2024-03-04T08:30:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbf0a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbbec14e4c0 0x7fbbec072c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35296/0 (socket says 172.21.15.113:35296) 2024-03-04T08:30:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbe3e00640 1 -- 172.21.15.113:0/1396015223 learned_addr learned my addr 172.21.15.113:0/1396015223 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbf0a00640 1 -- 172.21.15.113:0/1396015223 >> v1:172.21.15.136:6789/0 conn(0x7fbbec15b900 legacy=0x7fbbec1620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:21.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbf0a00640 1 -- 172.21.15.113:0/1396015223 >> v1:172.21.15.113:6789/0 conn(0x7fbbec0c22e0 legacy=0x7fbbec15b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:21.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.836+0000 7fbbf0a00640 1 -- 172.21.15.113:0/1396015223 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec070c60 con 0x7fbbec14e4c0 2024-03-04T08:30:21.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf0a00640 1 --2- 172.21.15.113:0/1396015223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec14e4c0 0x7fbbec072c80 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00ba60 tx=0x7fbbdc021320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b7247bfe9c57c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:21.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbebe00640 1 -- 172.21.15.113:0/1396015223 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0b6070 con 0x7fbbec14e4c0 2024-03-04T08:30:21.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbebe00640 1 -- 172.21.15.113:0/1396015223 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc0b1600 con 0x7fbbec14e4c0 2024-03-04T08:30:21.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbebe00640 1 -- 172.21.15.113:0/1396015223 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0b19d0 con 0x7fbbec14e4c0 2024-03-04T08:30:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 -- 172.21.15.113:0/1396015223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec14e4c0 msgr2=0x7fbbec072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 --2- 172.21.15.113:0/1396015223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec14e4c0 0x7fbbec072c80 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00ba60 tx=0x7fbbdc021320 comp rx=0 tx=0).stop 2024-03-04T08:30:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 -- 172.21.15.113:0/1396015223 shutdown_connections 2024-03-04T08:30:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 --2- 172.21.15.113:0/1396015223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec14e4c0 0x7fbbec072c80 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 -- 172.21.15.113:0/1396015223 >> 172.21.15.113:0/1396015223 conn(0x7fbbec06e820 msgr2=0x7fbbec06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 -- 172.21.15.113:0/1396015223 shutdown_connections 2024-03-04T08:30:21.835 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 -- 172.21.15.113:0/1396015223 wait complete. 2024-03-04T08:30:21.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 Processor -- start 2024-03-04T08:30:21.837 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 -- start start 2024-03-04T08:30:21.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c22e0 0x7fbbec0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:21.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbec14e4c0 0x7fbbec153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:21.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec15b900 0x7fbbec0afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:21.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec071500 con 0x7fbbec0c22e0 2024-03-04T08:30:21.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec071220 con 0x7fbbec15b900 2024-03-04T08:30:21.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf3400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbec14e8a0 con 0x7fbbec14e4c0 2024-03-04T08:30:21.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c22e0 0x7fbbec0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:21.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf0a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c22e0 0x7fbbec0c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35312/0 (socket says 172.21.15.113:35312) 2024-03-04T08:30:21.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.840+0000 7fbbf0a00640 1 -- 172.21.15.113:0/29361891 learned_addr learned my addr 172.21.15.113:0/29361891 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:21.839 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbe3e00640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbec14e4c0 0x7fbbec153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf1400640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec15b900 0x7fbbec0afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf0a00640 1 -- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbec14e4c0 msgr2=0x7fbbec153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf0a00640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbec14e4c0 0x7fbbec153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf0a00640 1 -- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec15b900 msgr2=0x7fbbec0afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf0a00640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec15b900 0x7fbbec0afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf0a00640 1 -- 172.21.15.113:0/29361891 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec0af220 con 0x7fbbec0c22e0 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf1400640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec15b900 0x7fbbec0afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf0a00640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c22e0 0x7fbbec0c6e20 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00ba60 tx=0x7fbbdc01e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbe9600640 1 -- 172.21.15.113:0/29361891 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0b6040 con 0x7fbbec0c22e0 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbe9600640 1 -- 172.21.15.113:0/29361891 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc0b1400 con 0x7fbbec0c22e0 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf3400640 1 -- 172.21.15.113:0/29361891 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbec0af7c0 con 0x7fbbec0c22e0 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbe9600640 1 -- 172.21.15.113:0/29361891 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0ba660 con 0x7fbbec0c22e0 2024-03-04T08:30:21.840 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf3400640 1 -- 172.21.15.113:0/29361891 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbec154000 con 0x7fbbec0c22e0 2024-03-04T08:30:21.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbf3400640 1 -- 172.21.15.113:0/29361891 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec075c00 con 0x7fbbec0c22e0 2024-03-04T08:30:21.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbe9600640 1 -- 172.21.15.113:0/29361891 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbdc0c1020 con 0x7fbbec0c22e0 2024-03-04T08:30:21.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbe9600640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbc8066f60 0x7fbbc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:21.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbe9600640 1 -- 172.21.15.113:0/29361891 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fbbdc123320 con 0x7fbbec0c22e0 2024-03-04T08:30:21.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.844+0000 7fbbe3e00640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbc8066f60 0x7fbbc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:21.842 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.848+0000 7fbbe3e00640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbc8066f60 0x7fbbc8069420 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbbec147820 tx=0x7fbbd4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:21.845 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:21.848+0000 7fbbe9600640 1 -- 172.21.15.113:0/29361891 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbbdc0aae20 con 0x7fbbec0c22e0 2024-03-04T08:30:22.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.036+0000 7fbbf3400640 1 -- 172.21.15.113:0/29361891 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbec155190 con 0x7fbbec0c22e0 2024-03-04T08:30:22.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.036+0000 7fbbe9600640 1 -- 172.21.15.113:0/29361891 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fbbdc01ae80 con 0x7fbbec0c22e0 2024-03-04T08:30:22.032 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:22.033 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:22.033 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:22.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 -- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbc8066f60 msgr2=0x7fbbc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:22.034 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbc8066f60 0x7fbbc8069420 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbbec147820 tx=0x7fbbd4009040 comp rx=0 tx=0).stop 2024-03-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 -- 172.21.15.113:0/29361891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c22e0 msgr2=0x7fbbec0c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c22e0 0x7fbbec0c6e20 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00ba60 tx=0x7fbbdc01e0c0 comp rx=0 tx=0).stop 2024-03-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 -- 172.21.15.113:0/29361891 shutdown_connections 2024-03-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbbec15b900 0x7fbbec0afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbbec14e4c0 0x7fbbec153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbbc8066f60 0x7fbbc8069420 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 --2- 172.21.15.113:0/29361891 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbbec0c22e0 0x7fbbec0c6e20 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 -- 172.21.15.113:0/29361891 >> 172.21.15.113:0/29361891 conn(0x7fbbec06e820 msgr2=0x7fbbec14d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 -- 172.21.15.113:0/29361891 shutdown_connections 2024-03-04T08:30:22.035 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:22.040+0000 7fbbf3400640 1 -- 172.21.15.113:0/29361891 wait complete. 2024-03-04T08:30:22.053 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2024-03-04T08:30:24.054 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:24.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.204+0000 7f6899800640 1 Processor -- start 2024-03-04T08:30:24.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.204+0000 7f6899800640 1 -- start start 2024-03-04T08:30:24.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.204+0000 7f6899800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f68940c22e0 0x7f689415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:24.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6899800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68940c27f0 con 0x7f689414e4c0 2024-03-04T08:30:24.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6899800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68940c2a90 con 0x7f68940c2d80 2024-03-04T08:30:24.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6899800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689415ad60 con 0x7f68940c22e0 2024-03-04T08:30:24.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6892000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f68940c22e0 0x7f689415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:24.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6892a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f689414e4c0 0x7f6894072c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:60840/0 (socket says 172.21.15.113:60840) 2024-03-04T08:30:24.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6892000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f68940c22e0 0x7f689415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35314/0 (socket says 172.21.15.113:35314) 2024-03-04T08:30:24.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6892a00640 1 -- 172.21.15.113:0/3903204605 learned_addr learned my addr 172.21.15.113:0/3903204605 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:24.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6892000640 1 -- 172.21.15.113:0/3903204605 >> v1:172.21.15.136:6789/0 conn(0x7f68940c2d80 legacy=0x7f68941620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:24.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6892000640 1 -- 172.21.15.113:0/3903204605 >> v1:172.21.15.113:6789/0 conn(0x7f689414e4c0 legacy=0x7f6894072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:24.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6892000640 1 -- 172.21.15.113:0/3903204605 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6894070c60 con 0x7f68940c22e0 2024-03-04T08:30:24.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6892000640 1 --2- 172.21.15.113:0/3903204605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c22e0 0x7f689415b1f0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f68800097c0 tx=0x7f688001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3878129fc43dff79 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:24.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6891600640 1 -- 172.21.15.113:0/3903204605 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68800b1070 con 0x7f68940c22e0 2024-03-04T08:30:24.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6891600640 1 -- 172.21.15.113:0/3903204605 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68800ac5f0 con 0x7f68940c22e0 2024-03-04T08:30:24.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.208+0000 7f6891600640 1 -- 172.21.15.113:0/3903204605 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68800ac9c0 con 0x7f68940c22e0 2024-03-04T08:30:24.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 -- 172.21.15.113:0/3903204605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c22e0 msgr2=0x7f689415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:24.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 --2- 172.21.15.113:0/3903204605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c22e0 0x7f689415b1f0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f68800097c0 tx=0x7f688001c4a0 comp rx=0 tx=0).stop 2024-03-04T08:30:24.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 -- 172.21.15.113:0/3903204605 shutdown_connections 2024-03-04T08:30:24.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 --2- 172.21.15.113:0/3903204605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c22e0 0x7f689415b1f0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:24.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 -- 172.21.15.113:0/3903204605 >> 172.21.15.113:0/3903204605 conn(0x7f689406e820 msgr2=0x7f689406ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:24.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 -- 172.21.15.113:0/3903204605 shutdown_connections 2024-03-04T08:30:24.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 -- 172.21.15.113:0/3903204605 wait complete. 2024-03-04T08:30:24.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 Processor -- start 2024-03-04T08:30:24.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 -- start start 2024-03-04T08:30:24.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68940c22e0 0x7f68941593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:24.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c2d80 0x7f689415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:24.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f689414e4c0 0x7f68940af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:24.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689414e8a0 con 0x7f68940c2d80 2024-03-04T08:30:24.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689415bb40 con 0x7f68940c22e0 2024-03-04T08:30:24.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6899800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68940aede0 con 0x7f689414e4c0 2024-03-04T08:30:24.210 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c2d80 0x7f689415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c2d80 0x7f689415a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35330/0 (socket says 172.21.15.113:35330) 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892000640 1 -- 172.21.15.113:0/1857412615 learned_addr learned my addr 172.21.15.113:0/1857412615 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892a00640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68940c22e0 0x7f68941593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6893400640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f689414e4c0 0x7f68940af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892000640 1 -- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f689414e4c0 msgr2=0x7f68940af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892000640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f689414e4c0 0x7f68940af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892000640 1 -- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68940c22e0 msgr2=0x7f68941593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892000640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68940c22e0 0x7f68941593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892000640 1 -- 172.21.15.113:0/1857412615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689413d010 con 0x7f68940c2d80 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892a00640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68940c22e0 0x7f68941593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.212+0000 7f6892000640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c2d80 0x7f689415a3d0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f688001a0d0 tx=0x7f6880019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.216+0000 7f688aa00640 1 -- 172.21.15.113:0/1857412615 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68800b9070 con 0x7f68940c2d80 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.216+0000 7f688aa00640 1 -- 172.21.15.113:0/1857412615 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68800b5610 con 0x7f68940c2d80 2024-03-04T08:30:24.211 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.216+0000 7f688aa00640 1 -- 172.21.15.113:0/1857412615 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68800b1040 con 0x7f68940c2d80 2024-03-04T08:30:24.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.216+0000 7f6899800640 1 -- 172.21.15.113:0/1857412615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689414b9f0 con 0x7f68940c2d80 2024-03-04T08:30:24.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.216+0000 7f6899800640 1 -- 172.21.15.113:0/1857412615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6894146e90 con 0x7f68940c2d80 2024-03-04T08:30:24.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.216+0000 7f6899800640 1 -- 172.21.15.113:0/1857412615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6894075c00 con 0x7f68940c2d80 2024-03-04T08:30:24.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.216+0000 7f688aa00640 1 -- 172.21.15.113:0/1857412615 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68800bc070 con 0x7f68940c2d80 2024-03-04T08:30:24.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.216+0000 7f688aa00640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6874066f60 0x7f6874069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:24.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.216+0000 7f688aa00640 1 -- 172.21.15.113:0/1857412615 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f688011f740 con 0x7f68940c2d80 2024-03-04T08:30:24.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.216+0000 7f6892a00640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6874066f60 0x7f6874069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:24.214 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.220+0000 7f6892a00640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6874066f60 0x7f6874069420 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6884005c10 tx=0x7f6884009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:24.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.220+0000 7f688aa00640 1 -- 172.21.15.113:0/1857412615 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f68800a9070 con 0x7f68940c2d80 2024-03-04T08:30:24.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 -- 172.21.15.113:0/1857412615 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6894140480 con 0x7f68940c2d80 2024-03-04T08:30:24.411 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f688aa00640 1 -- 172.21.15.113:0/1857412615 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7f6880016090 con 0x7f68940c2d80 2024-03-04T08:30:24.411 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:24.412 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:24.412 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:24.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 -- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6874066f60 msgr2=0x7f6874069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6874066f60 0x7f6874069420 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6884005c10 tx=0x7f6884009040 comp rx=0 tx=0).stop 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 -- 172.21.15.113:0/1857412615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c2d80 msgr2=0x7f689415a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c2d80 0x7f689415a3d0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f688001a0d0 tx=0x7f6880019240 comp rx=0 tx=0).stop 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 -- 172.21.15.113:0/1857412615 shutdown_connections 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f689414e4c0 0x7f68940af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f6874066f60 0x7f6874069420 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f68940c2d80 0x7f689415a3d0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 --2- 172.21.15.113:0/1857412615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f68940c22e0 0x7f68941593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 -- 172.21.15.113:0/1857412615 >> 172.21.15.113:0/1857412615 conn(0x7f689406e820 msgr2=0x7f689414c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 -- 172.21.15.113:0/1857412615 shutdown_connections 2024-03-04T08:30:24.414 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:24.416+0000 7f6899800640 1 -- 172.21.15.113:0/1857412615 wait complete. 2024-03-04T08:30:24.433 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2024-03-04T08:30:24.434 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 193, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2024-03-04T07:33:55.904916+0000", "damaged": [], "data_pools": [ 3 ], "enabled": true, "epoch": 193, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "balance_automate": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7661": { "addr": "172.21.15.136:6837/2373169282", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6836", "nonce": 2373169282, "type": "v2" }, { "addr": "172.21.15.136:6837", "nonce": 2373169282, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 7661, "incarnation": 192, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:resolve", "state_seq": 18 }, "gid_8032": { "addr": "172.21.15.136:6835/1399979527", "addrs": { "addrvec": [ { "addr": "172.21.15.136:6834", "nonce": 1399979527, "type": "v2" }, { "addr": "172.21.15.136:6835", "nonce": 1399979527, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8032, "incarnation": 157, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 30 }, "gid_8760": { "addr": "172.21.15.113:6837/3564557636", "addrs": { "addrvec": [ { "addr": "172.21.15.113:6835", "nonce": 3564557636, "type": "v2" }, { "addr": "172.21.15.113:6837", "nonce": 3564557636, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540701547738038271, "flags": 0, "gid": 8760, "incarnation": 185, "join_fscid": -1, "laggy_since": "2024-03-04T08:24:26.023625+0000", "name": "e", "rank": 2, "state": "up:stopping", "state_seq": 15 } }, "last_failure": 0, "last_failure_osd_epoch": 85, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 2, "modified": "2024-03-04T08:24:34.702167+0000", "qdb_cluster": [ 8032 ], "qdb_leader": 8032, "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7661, "mds_1": 8032, "mds_2": 8760 } } } ], "standbys": [] } 2024-03-04T08:30:26.435 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-04T08:30:26.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.584+0000 7fbeb1800640 1 Processor -- start 2024-03-04T08:30:26.582 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb1800640 1 -- start start 2024-03-04T08:30:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb1800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbeac06dcb0 0x7fbeac15cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb1800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbeac0c8990 0x7fbeac15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb1800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeac06c310 con 0x7fbeac15b8a0 2024-03-04T08:30:26.583 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb1800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeac15c6c0 con 0x7fbeac0c8990 2024-03-04T08:30:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb1800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeac06c5f0 con 0x7fbeac06dcb0 2024-03-04T08:30:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbeac0c8990 0x7fbeac15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbeac0c8990 0x7fbeac15d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35332/0 (socket says 172.21.15.113:35332) 2024-03-04T08:30:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb0e00640 1 -- 172.21.15.113:0/1460632828 learned_addr learned my addr 172.21.15.113:0/1460632828 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbea3400640 1 --2- 172.21.15.113:0/1460632828 >> v2:172.21.15.136:3300/0 conn(0x7fbeac06dcb0 0x7fbeac15cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb0e00640 1 -- 172.21.15.113:0/1460632828 >> v2:172.21.15.136:3300/0 conn(0x7fbeac06dcb0 msgr2=0x7fbeac15cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb0e00640 1 --2- 172.21.15.113:0/1460632828 >> v2:172.21.15.136:3300/0 conn(0x7fbeac06dcb0 0x7fbeac15cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:26.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb0e00640 1 -- 172.21.15.113:0/1460632828 >> v1:172.21.15.136:6789/0 conn(0x7fbeac15b8a0 legacy=0x7fbeac06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:26.585 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.588+0000 7fbeb0e00640 1 -- 172.21.15.113:0/1460632828 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac15c070 con 0x7fbeac0c8990 2024-03-04T08:30:26.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb0e00640 1 --2- 172.21.15.113:0/1460632828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac0c8990 0x7fbeac15d090 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0cec10 tx=0x7fbea401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9561dd7642a00c87 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeab400640 1 -- 172.21.15.113:0/1460632828 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40a57d0 con 0x7fbeac0c8990 2024-03-04T08:30:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeab400640 1 -- 172.21.15.113:0/1460632828 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea40a5970 con 0x7fbeac0c8990 2024-03-04T08:30:26.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeab400640 1 -- 172.21.15.113:0/1460632828 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40a5d40 con 0x7fbeac0c8990 2024-03-04T08:30:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb1800640 1 -- 172.21.15.113:0/1460632828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac0c8990 msgr2=0x7fbeac15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb1800640 1 --2- 172.21.15.113:0/1460632828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac0c8990 0x7fbeac15d090 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0cec10 tx=0x7fbea401c9f0 comp rx=0 tx=0).stop 2024-03-04T08:30:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb1800640 1 -- 172.21.15.113:0/1460632828 shutdown_connections 2024-03-04T08:30:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb1800640 1 --2- 172.21.15.113:0/1460632828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac0c8990 0x7fbeac15d090 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb1800640 1 --2- 172.21.15.113:0/1460632828 >> v2:172.21.15.136:3300/0 conn(0x7fbeac06dcb0 0x7fbeac15cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb1800640 1 -- 172.21.15.113:0/1460632828 >> 172.21.15.113:0/1460632828 conn(0x7fbeac072e30 msgr2=0x7fbeac066db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb1800640 1 -- 172.21.15.113:0/1460632828 shutdown_connections 2024-03-04T08:30:26.588 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb1800640 1 -- 172.21.15.113:0/1460632828 wait complete. 2024-03-04T08:30:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb1800640 1 Processor -- start 2024-03-04T08:30:26.589 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.592+0000 7fbeb1800640 1 -- start start 2024-03-04T08:30:26.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb1800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbeac06dcb0 0x7fbeac1f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:26.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbeac0c8990 0x7fbeac0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb1800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac15b8a0 0x7fbeac0c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb1800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac06f7e0 con 0x7fbeac15b8a0 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb1800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac06f660 con 0x7fbeac0c8990 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb1800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac167bb0 con 0x7fbeac06dcb0 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb0e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac15b8a0 0x7fbeac0c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb0e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac15b8a0 0x7fbeac0c2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:35334/0 (socket says 172.21.15.113:35334) 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb0e00640 1 -- 172.21.15.113:0/639786932 learned_addr learned my addr 172.21.15.113:0/639786932 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbea3400640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbeac0c8990 0x7fbeac0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeabe00640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbeac06dcb0 0x7fbeac1f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb0e00640 1 -- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbeac06dcb0 msgr2=0x7fbeac1f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb0e00640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbeac06dcb0 0x7fbeac1f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb0e00640 1 -- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbeac0c8990 msgr2=0x7fbeac0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:26.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb0e00640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbeac0c8990 0x7fbeac0c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:26.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb0e00640 1 -- 172.21.15.113:0/639786932 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac0c3c30 con 0x7fbeac15b8a0 2024-03-04T08:30:26.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeabe00640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbeac06dcb0 0x7fbeac1f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:26.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb0e00640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac15b8a0 0x7fbeac0c2530 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fbea4019600 tx=0x7fbea4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:26.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbea8c00640 1 -- 172.21.15.113:0/639786932 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40b8670 con 0x7fbeac15b8a0 2024-03-04T08:30:26.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbea8c00640 1 -- 172.21.15.113:0/639786932 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea40be070 con 0x7fbeac15b8a0 2024-03-04T08:30:26.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbea8c00640 1 -- 172.21.15.113:0/639786932 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40b8a70 con 0x7fbeac15b8a0 2024-03-04T08:30:26.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb1800640 1 -- 172.21.15.113:0/639786932 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac0c3f10 con 0x7fbeac15b8a0 2024-03-04T08:30:26.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb1800640 1 -- 172.21.15.113:0/639786932 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac0c1d30 con 0x7fbeac15b8a0 2024-03-04T08:30:26.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbeb1800640 1 -- 172.21.15.113:0/639786932 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeac075a80 con 0x7fbeac15b8a0 2024-03-04T08:30:26.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.596+0000 7fbea8c00640 1 -- 172.21.15.113:0/639786932 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbea40b8c10 con 0x7fbeac15b8a0 2024-03-04T08:30:26.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.600+0000 7fbea8c00640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe88066f60 0x7fbe88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:26.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.600+0000 7fbea8c00640 1 -- 172.21.15.113:0/639786932 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fbea4120b70 con 0x7fbeac15b8a0 2024-03-04T08:30:26.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.600+0000 7fbeabe00640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe88066f60 0x7fbe88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:26.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.600+0000 7fbeabe00640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe88066f60 0x7fbe88069420 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c001670 tx=0x7fbe9c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:26.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.604+0000 7fbea8c00640 1 -- 172.21.15.113:0/639786932 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbea40165e0 con 0x7fbeac15b8a0 2024-03-04T08:30:26.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 -- 172.21.15.113:0/639786932 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbeac06d4d0 con 0x7fbeac15b8a0 2024-03-04T08:30:26.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbea8c00640 1 -- 172.21.15.113:0/639786932 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+4285 (secure 0 0 0) 0x7fbea40f1040 con 0x7fbeac15b8a0 2024-03-04T08:30:26.787 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:26.787 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-04T07:33:55.904916+0000","modified":"2024-03-04T08:24:34.702167+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":85,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7661,"mds_1":8032,"mds_2":8760},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7661":{"gid":7661,"name":"d","rank":0,"incarnation":192,"state":"up:resolve","state_seq":18,"addr":"172.21.15.136:6837/2373169282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":2373169282},{"type":"v1","addr":"172.21.15.136:6837","nonce":2373169282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8032":{"gid":8032,"name":"b","rank":1,"incarnation":157,"state":"up:active","state_seq":30,"addr":"172.21.15.136:6835/1399979527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":1399979527},{"type":"v1","addr":"172.21.15.136:6835","nonce":1399979527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_8760":{"gid":8760,"name":"e","rank":2,"incarnation":185,"state":"up:stopping","state_seq":15,"addr":"172.21.15.113:6837/3564557636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6835","nonce":3564557636},{"type":"v1","addr":"172.21.15.113:6837","nonce":3564557636}]},"join_fscid":-1,"laggy_since":"2024-03-04T08:24:26.023625+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8032,"qdb_cluster":[8032]},"id":1}]} 2024-03-04T08:30:26.787 INFO:teuthology.orchestra.run.smithi113.stderr:dumped fsmap epoch 193 2024-03-04T08:30:26.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 -- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe88066f60 msgr2=0x7fbe88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:26.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe88066f60 0x7fbe88069420 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c001670 tx=0x7fbe9c009040 comp rx=0 tx=0).stop 2024-03-04T08:30:26.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 -- 172.21.15.113:0/639786932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac15b8a0 msgr2=0x7fbeac0c2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:26.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac15b8a0 0x7fbeac0c2530 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fbea4019600 tx=0x7fbea4019790 comp rx=0 tx=0).stop 2024-03-04T08:30:26.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 -- 172.21.15.113:0/639786932 shutdown_connections 2024-03-04T08:30:26.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe88066f60 0x7fbe88069420 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:26.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbeac15b8a0 0x7fbeac0c2530 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:26.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbeac0c8990 0x7fbeac0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:26.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 --2- 172.21.15.113:0/639786932 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbeac06dcb0 0x7fbeac1f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:26.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 -- 172.21.15.113:0/639786932 >> 172.21.15.113:0/639786932 conn(0x7fbeac072e30 msgr2=0x7fbeac0ca130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:26.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 -- 172.21.15.113:0/639786932 shutdown_connections 2024-03-04T08:30:26.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:26.792+0000 7fbeb1800640 1 -- 172.21.15.113:0/639786932 wait complete. 2024-03-04T08:30:26.808 ERROR:tasks.mds_thrash.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/mds_thrash.py", line 124, in _run self.do_thrash() File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/mds_thrash.py", line 312, in do_thrash status = self.wait_for_stable(rank, gid) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/mds_thrash.py", line 217, in wait_for_stable raise RuntimeError('timeout waiting for cluster to stabilize') RuntimeError: timeout waiting for cluster to stabilize 2024-03-04T08:30:30.700 INFO:tasks.daemonwatchdog.daemon_watchdog:thrasher.fs.[cephfs] failed 2024-03-04T08:30:30.700 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-03-04T08:30:30.700 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T08:30:30.700 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T08:30:30.710 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T08:30:30.710 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T08:30:30.763 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:30:30.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe56d000640 1 Processor -- start 2024-03-04T08:30:30.950 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe56d000640 1 -- start start 2024-03-04T08:30:30.951 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe56d000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5680cb540 0x7fe56815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:30.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe56d000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fe56806faa0 0x7fe5680c30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:30.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe56d000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5680c4ce0 con 0x7fe568152950 2024-03-04T08:30:30.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe56d000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5680c8370 con 0x7fe5680cb540 2024-03-04T08:30:30.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe56d000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5680ca0a0 con 0x7fe56806faa0 2024-03-04T08:30:30.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe566a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5680cb540 0x7fe56815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:30.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe566a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe5680cb540 0x7fe56815bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56366/0 (socket says 172.21.15.113:56366) 2024-03-04T08:30:30.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe566a00640 1 -- 172.21.15.113:0/799930582 learned_addr learned my addr 172.21.15.113:0/799930582 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:30.952 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe567e00640 1 --2- 172.21.15.113:0/799930582 >> v2:172.21.15.136:3300/0 conn(0x7fe56806faa0 0x7fe5680c30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:30.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe566a00640 1 -- 172.21.15.113:0/799930582 >> v2:172.21.15.136:3300/0 conn(0x7fe56806faa0 msgr2=0x7fe5680c30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:30.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe566a00640 1 --2- 172.21.15.113:0/799930582 >> v2:172.21.15.136:3300/0 conn(0x7fe56806faa0 0x7fe5680c30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:30.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe566a00640 1 -- 172.21.15.113:0/799930582 >> v1:172.21.15.136:6789/0 conn(0x7fe568152950 legacy=0x7fe56806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:30.953 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.956+0000 7fe566a00640 1 -- 172.21.15.113:0/799930582 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680c4f80 con 0x7fe5680cb540 2024-03-04T08:30:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe566a00640 1 --2- 172.21.15.113:0/799930582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5680cb540 0x7fe56815bb40 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fe5680c8530 tx=0x7fe55401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3b12d537b77559 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe566000640 1 -- 172.21.15.113:0/799930582 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5540b5070 con 0x7fe5680cb540 2024-03-04T08:30:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe566000640 1 -- 172.21.15.113:0/799930582 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5540a8a10 con 0x7fe5680cb540 2024-03-04T08:30:30.955 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe566000640 1 -- 172.21.15.113:0/799930582 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5540a8de0 con 0x7fe5680cb540 2024-03-04T08:30:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe56d000640 1 -- 172.21.15.113:0/799930582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5680cb540 msgr2=0x7fe56815bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe56d000640 1 --2- 172.21.15.113:0/799930582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5680cb540 0x7fe56815bb40 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fe5680c8530 tx=0x7fe55401ed10 comp rx=0 tx=0).stop 2024-03-04T08:30:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe56d000640 1 -- 172.21.15.113:0/799930582 shutdown_connections 2024-03-04T08:30:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe56d000640 1 --2- 172.21.15.113:0/799930582 >> v2:172.21.15.136:3300/0 conn(0x7fe56806faa0 0x7fe5680c30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe56d000640 1 --2- 172.21.15.113:0/799930582 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5680cb540 0x7fe56815bb40 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe56d000640 1 -- 172.21.15.113:0/799930582 >> 172.21.15.113:0/799930582 conn(0x7fe568072f40 msgr2=0x7fe5680706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe56d000640 1 -- 172.21.15.113:0/799930582 shutdown_connections 2024-03-04T08:30:30.956 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe56d000640 1 -- 172.21.15.113:0/799930582 wait complete. 2024-03-04T08:30:30.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe56d000640 1 Processor -- start 2024-03-04T08:30:30.957 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.960+0000 7fe56d000640 1 -- start start 2024-03-04T08:30:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe56d000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56806faa0 0x7fe5680c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe56d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5680cb540 0x7fe5680c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe56d000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe568152950 0x7fe56814a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe56d000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680c9a50 con 0x7fe56806faa0 2024-03-04T08:30:30.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe56d000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680c5220 con 0x7fe5680cb540 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe56d000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680c9d30 con 0x7fe568152950 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe567400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56806faa0 0x7fe5680c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe567400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56806faa0 0x7fe5680c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56378/0 (socket says 172.21.15.113:56378) 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe567400640 1 -- 172.21.15.113:0/3462041099 learned_addr learned my addr 172.21.15.113:0/3462041099 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe566a00640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5680cb540 0x7fe5680c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe567e00640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe568152950 0x7fe56814a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe567400640 1 -- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe568152950 msgr2=0x7fe56814a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe567400640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe568152950 0x7fe56814a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe567400640 1 -- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5680cb540 msgr2=0x7fe5680c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe567400640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5680cb540 0x7fe5680c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:30.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe567400640 1 -- 172.21.15.113:0/3462041099 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5681e25c0 con 0x7fe56806faa0 2024-03-04T08:30:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe567400640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56806faa0 0x7fe5680c5e80 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c420 tx=0x7fe55800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe55f400640 1 -- 172.21.15.113:0/3462041099 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5580052d0 con 0x7fe56806faa0 2024-03-04T08:30:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe55f400640 1 -- 172.21.15.113:0/3462041099 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe558005470 con 0x7fe56806faa0 2024-03-04T08:30:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe55f400640 1 -- 172.21.15.113:0/3462041099 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558007260 con 0x7fe56806faa0 2024-03-04T08:30:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe56d000640 1 -- 172.21.15.113:0/3462041099 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5681e2860 con 0x7fe56806faa0 2024-03-04T08:30:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe56d000640 1 -- 172.21.15.113:0/3462041099 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5681e2e20 con 0x7fe56806faa0 2024-03-04T08:30:30.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.964+0000 7fe56d000640 1 -- 172.21.15.113:0/3462041099 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe568075a80 con 0x7fe56806faa0 2024-03-04T08:30:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.968+0000 7fe55f400640 1 -- 172.21.15.113:0/3462041099 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe558007400 con 0x7fe56806faa0 2024-03-04T08:30:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.968+0000 7fe55f400640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe548066f60 0x7fe548069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.968+0000 7fe55f400640 1 -- 172.21.15.113:0/3462041099 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fe558084ff0 con 0x7fe56806faa0 2024-03-04T08:30:30.963 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.968+0000 7fe566a00640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe548066f60 0x7fe548069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:30.964 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.968+0000 7fe566a00640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe548066f60 0x7fe548069420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe5540b2000 tx=0x7fe554009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:30.968 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:30.972+0000 7fe55f400640 1 -- 172.21.15.113:0/3462041099 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe558054e70 con 0x7fe56806faa0 2024-03-04T08:30:31.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.132+0000 7fe56d000640 1 -- 172.21.15.113:0/3462041099 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5680c9580 con 0x7fe56806faa0 2024-03-04T08:30:31.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.132+0000 7fe55f400640 1 -- 172.21.15.113:0/3462041099 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+20859 (secure 0 0 0) 0x7fe558058f10 con 0x7fe56806faa0 2024-03-04T08:30:31.129 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:31.129 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T08:24:26.023707+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":683,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.136:6839/1409794802":"2024-03-05T08:24:26.023653+0000","172.21.15.136:6834/2368734308":"2024-03-05T07:58:45.780423+0000","172.21.15.113:6832/3005395426":"2024-03-05T07:37:40.596420+0000","172.21.15.113:6836/986515123":"2024-03-05T08:00:40.798160+0000","172.21.15.113:6834/2299268553":"2024-03-05T07:52:10.722331+0000","172.21.15.136:6834/3944001911":"2024-03-05T07:45:20.664657+0000","172.21.15.113:6836/2299268553":"2024-03-05T07:52:10.722331+0000","172.21.15.136:6835/2522634751":"2024-03-05T08:07:30.861130+0000","172.21.15.136:6834/1691762848":"2024-03-05T07:49:30.698378+0000","172.21.15.113:6832/2732565639":"2024-03-05T08:02:45.817095+0000","172.21.15.113:6837/742633123":"2024-03-05T07:47:40.683981+0000","172.21.15.113:6836/4038214481":"2024-03-05T07:36:05.584380+0000","172.21.15.136:6835/3944001911":"2024-03-05T07:45:20.664657+0000","172.21.15.113:6834/986515123":"2024-03-05T08:00:40.798160+0000","172.21.15.113:6833/605142559":"2024-03-05T07:44:20.656043+0000","172.21.15.113:6836/1035982016":"2024-03-05T08:05:45.840907+0000","172.21.15.113:6833/3005395426":"2024-03-05T07:37:40.596420+0000","172.21.15.113:6832/1928887637":"2024-03-05T08:20:05.977949+0000","172.21.15.113:6833/810789479":"2024-03-05T07:54:40.746387+0000","172.21.15.113:6835/2544977425":"2024-03-05T08:21:40.993913+0000","172.21.15.113:6837/4197507712":"2024-03-05T08:04:30.830241+0000","172.21.15.136:6834/3348592059":"2024-03-05T07:40:15.619912+0000","172.21.15.113:6835/1310042629":"2024-03-05T07:38:50.604670+0000","172.21.15.136:6834/2522634751":"2024-03-05T08:07:30.861130+0000","172.21.15.113:6835/742633123":"2024-03-05T07:47:40.683981+0000","172.21.15.113:6834/1310057122":"2024-03-05T08:13:45.911820+0000","172.21.15.113:6834/4038214481":"2024-03-05T07:36:05.584380+0000","172.21.15.136:6835/1691762848":"2024-03-05T07:49:30.698378+0000","172.21.15.113:6832/605142559":"2024-03-05T07:44:20.656043+0000","172.21.15.113:6837/2544977425":"2024-03-05T08:21:40.993913+0000","172.21.15.113:6835/344022786":"2024-03-05T07:57:05.766508+0000","172.21.15.113:6832/2999156262":"2024-03-05T08:22:51.004939+0000","172.21.15.113:6837/1401330284":"2024-03-05T08:11:10.891335+0000","172.21.15.113:6837/1310042629":"2024-03-05T07:38:50.604670+0000","172.21.15.113:6834/2997246507":"2024-03-05T07:41:40.631034+0000","172.21.15.136:6835/2368734308":"2024-03-05T07:58:45.780423+0000","172.21.15.113:6836/1310057122":"2024-03-05T08:13:45.911820+0000","172.21.15.136:6835/3348592059":"2024-03-05T07:40:15.619912+0000","172.21.15.136:6837/2228772746":"2024-03-05T08:23:26.011590+0000","172.21.15.113:6836/2997246507":"2024-03-05T07:41:40.631034+0000","172.21.15.136:6835/3388011401":"2024-03-05T08:01:30.805387+0000","172.21.15.136:6835/247269911":"2024-03-05T08:16:00.936524+0000","172.21.15.113:6833/2732565639":"2024-03-05T08:02:45.817095+0000","172.21.15.113:6835/4197507712":"2024-03-05T08:04:30.830241+0000","172.21.15.136:6838/1409794802":"2024-03-05T08:24:26.023653+0000","172.21.15.113:6834/1035982016":"2024-03-05T08:05:45.840907+0000","172.21.15.113:6832/3825257303":"2024-03-05T08:09:55.880790+0000","172.21.15.136:6834/3388011401":"2024-03-05T08:01:30.805387+0000","172.21.15.113:6833/3825257303":"2024-03-05T08:09:55.880790+0000","172.21.15.113:6835/1401330284":"2024-03-05T08:11:10.891335+0000","172.21.15.136:6834/247269911":"2024-03-05T08:16:00.936524+0000","172.21.15.113:6833/1928887637":"2024-03-05T08:20:05.977949+0000","172.21.15.113:6834/4083765155":"2024-03-05T08:17:55.953381+0000","172.21.15.113:6833/2999156262":"2024-03-05T08:22:51.004939+0000","172.21.15.113:6836/4083765155":"2024-03-05T08:17:55.953381+0000","172.21.15.113:6832/810789479":"2024-03-05T07:54:40.746387+0000","172.21.15.113:6834/893414524":"2024-03-05T08:21:25.990484+0000","172.21.15.113:6837/344022786":"2024-03-05T07:57:05.766508+0000","172.21.15.113:6836/893414524":"2024-03-05T08:21:25.990484+0000","172.21.15.136:6836/2228772746":"2024-03-05T08:23:26.011590+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:30:31.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 -- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe548066f60 msgr2=0x7fe548069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe548066f60 0x7fe548069420 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe5540b2000 tx=0x7fe554009040 comp rx=0 tx=0).stop 2024-03-04T08:30:31.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 -- 172.21.15.113:0/3462041099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56806faa0 msgr2=0x7fe5680c5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56806faa0 0x7fe5680c5e80 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c420 tx=0x7fe55800c8e0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 -- 172.21.15.113:0/3462041099 shutdown_connections 2024-03-04T08:30:31.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fe568152950 0x7fe56814a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5680cb540 0x7fe5680c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fe548066f60 0x7fe548069420 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 --2- 172.21.15.113:0/3462041099 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe56806faa0 0x7fe5680c5e80 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 -- 172.21.15.113:0/3462041099 >> 172.21.15.113:0/3462041099 conn(0x7fe568072f40 msgr2=0x7fe5681563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:31.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 -- 172.21.15.113:0/3462041099 shutdown_connections 2024-03-04T08:30:31.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.136+0000 7fe56d000640 1 -- 172.21.15.113:0/3462041099 wait complete. 2024-03-04T08:30:31.151 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T07:33:41.507700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T07:33:53.680465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T07:33:54.892043+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 683, 'snap_epoch': 75, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T08:30:31.153 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T08:30:31.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f34e00640 1 Processor -- start 2024-03-04T08:30:31.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f34e00640 1 -- start start 2024-03-04T08:30:31.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f34e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f300cc500 0x7f5f300c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f34e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f30073360 0x7f5f3006bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f34e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f5f300cc8e0 0x7f5f30070f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f34e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f301588b0 con 0x7f5f300cc500 2024-03-04T08:30:31.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f34e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f30158b70 con 0x7f5f300cc8e0 2024-03-04T08:30:31.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f34e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30158e50 con 0x7f5f30073360 2024-03-04T08:30:31.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f2de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f300cc500 0x7f5f300c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f2de00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5f300cc500 0x7f5f300c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56390/0 (socket says 172.21.15.113:56390) 2024-03-04T08:30:31.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f2de00640 1 -- 172.21.15.113:0/1019968317 learned_addr learned my addr 172.21.15.113:0/1019968317 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:31.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f25400640 1 --2- 172.21.15.113:0/1019968317 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f30073360 0x7f5f3006bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.300+0000 7f5f2e800640 1 --2- 172.21.15.113:0/1019968317 >> v2:172.21.15.136:3300/0 conn(0x7f5f300cc8e0 0x7f5f30070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f2de00640 1 -- 172.21.15.113:0/1019968317 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f30073360 msgr2=0x7f5f3006bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:31.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f2de00640 1 --2- 172.21.15.113:0/1019968317 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f30073360 0x7f5f3006bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f2de00640 1 -- 172.21.15.113:0/1019968317 >> v2:172.21.15.136:3300/0 conn(0x7f5f300cc8e0 msgr2=0x7f5f30070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:31.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f2de00640 1 --2- 172.21.15.113:0/1019968317 >> v2:172.21.15.136:3300/0 conn(0x7f5f300cc8e0 0x7f5f30070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f2de00640 1 -- 172.21.15.113:0/1019968317 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30159130 con 0x7f5f300cc500 2024-03-04T08:30:31.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f2e800640 1 --2- 172.21.15.113:0/1019968317 >> v2:172.21.15.136:3300/0 conn(0x7f5f300cc8e0 0x7f5f30070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:31.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f2de00640 1 --2- 172.21.15.113:0/1019968317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 0x7f5f300c7480 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009e50 tx=0x7f5f2001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b6734699bc25d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:31.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f2d400640 1 -- 172.21.15.113:0/1019968317 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200b4070 con 0x7f5f300cc500 2024-03-04T08:30:31.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f2d400640 1 -- 172.21.15.113:0/1019968317 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f200af400 con 0x7f5f300cc500 2024-03-04T08:30:31.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f2d400640 1 -- 172.21.15.113:0/1019968317 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f200af7d0 con 0x7f5f300cc500 2024-03-04T08:30:31.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f34e00640 1 -- 172.21.15.113:0/1019968317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 msgr2=0x7f5f300c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f34e00640 1 --2- 172.21.15.113:0/1019968317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 0x7f5f300c7480 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009e50 tx=0x7f5f2001f1e0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f34e00640 1 -- 172.21.15.113:0/1019968317 shutdown_connections 2024-03-04T08:30:31.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f34e00640 1 --2- 172.21.15.113:0/1019968317 >> v2:172.21.15.136:3300/0 conn(0x7f5f300cc8e0 0x7f5f30070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f34e00640 1 --2- 172.21.15.113:0/1019968317 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f30073360 0x7f5f3006bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f34e00640 1 --2- 172.21.15.113:0/1019968317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 0x7f5f300c7480 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f34e00640 1 -- 172.21.15.113:0/1019968317 >> 172.21.15.113:0/1019968317 conn(0x7f5f30066db0 msgr2=0x7f5f300671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:31.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f34e00640 1 -- 172.21.15.113:0/1019968317 shutdown_connections 2024-03-04T08:30:31.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.304+0000 7f5f34e00640 1 -- 172.21.15.113:0/1019968317 wait complete. 2024-03-04T08:30:31.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f34e00640 1 Processor -- start 2024-03-04T08:30:31.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f34e00640 1 -- start start 2024-03-04T08:30:31.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f34e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f30073360 0x7f5f3015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f34e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 0x7f5f3014ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f34e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f300cc8e0 0x7f5f3014d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f34e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f300715e0 con 0x7f5f300cc500 2024-03-04T08:30:31.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f34e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30071460 con 0x7f5f30073360 2024-03-04T08:30:31.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f34e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30071760 con 0x7f5f300cc8e0 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f25400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 0x7f5f3014ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f25400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 0x7f5f3014ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56392/0 (socket says 172.21.15.113:56392) 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f25400640 1 -- 172.21.15.113:0/1646387700 learned_addr learned my addr 172.21.15.113:0/1646387700 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f2de00640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f30073360 0x7f5f3015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f2e800640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f300cc8e0 0x7f5f3014d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f25400640 1 -- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f300cc8e0 msgr2=0x7f5f3014d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f25400640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f300cc8e0 0x7f5f3014d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f25400640 1 -- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f30073360 msgr2=0x7f5f3015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f25400640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f30073360 0x7f5f3015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f25400640 1 -- 172.21.15.113:0/1646387700 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f301ee540 con 0x7f5f300cc500 2024-03-04T08:30:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f25400640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 0x7f5f3014ae20 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e8a0 tx=0x7f5f1800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:31.307 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f26a00640 1 -- 172.21.15.113:0/1646387700 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18017070 con 0x7f5f300cc500 2024-03-04T08:30:31.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f26a00640 1 -- 172.21.15.113:0/1646387700 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1800c940 con 0x7f5f300cc500 2024-03-04T08:30:31.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f34e00640 1 -- 172.21.15.113:0/1646387700 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f300c5070 con 0x7f5f300cc500 2024-03-04T08:30:31.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.308+0000 7f5f26a00640 1 -- 172.21.15.113:0/1646387700 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18009e60 con 0x7f5f300cc500 2024-03-04T08:30:31.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.312+0000 7f5f34e00640 1 -- 172.21.15.113:0/1646387700 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3014d1b0 con 0x7f5f300cc500 2024-03-04T08:30:31.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.312+0000 7f5f34e00640 1 -- 172.21.15.113:0/1646387700 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f30075c00 con 0x7f5f300cc500 2024-03-04T08:30:31.308 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.312+0000 7f5f26a00640 1 -- 172.21.15.113:0/1646387700 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f1801a070 con 0x7f5f300cc500 2024-03-04T08:30:31.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.312+0000 7f5f26a00640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f0c066f60 0x7f5f0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.312+0000 7f5f26a00640 1 -- 172.21.15.113:0/1646387700 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f5f18085a00 con 0x7f5f300cc500 2024-03-04T08:30:31.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.312+0000 7f5f2de00640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f0c066f60 0x7f5f0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.312+0000 7f5f2de00640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f0c066f60 0x7f5f0c069420 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f5f200b1000 tx=0x7f5f200aa000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:31.313 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.316+0000 7f5f26a00640 1 -- 172.21.15.113:0/1646387700 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5f301ee540 con 0x7f5f300cc500 2024-03-04T08:30:31.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.472+0000 7f5f34e00640 1 -- 172.21.15.113:0/1646387700 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f3006ba20 con 0x7f5f300cc500 2024-03-04T08:30:31.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.472+0000 7f5f26a00640 1 -- 172.21.15.113:0/1646387700 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f180557f0 con 0x7f5f300cc500 2024-03-04T08:30:31.470 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-03-04T08:30:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 -- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f0c066f60 msgr2=0x7f5f0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f0c066f60 0x7f5f0c069420 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f5f200b1000 tx=0x7f5f200aa000 comp rx=0 tx=0).stop 2024-03-04T08:30:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 -- 172.21.15.113:0/1646387700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 msgr2=0x7f5f3014ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 0x7f5f3014ae20 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e8a0 tx=0x7f5f1800ed60 comp rx=0 tx=0).stop 2024-03-04T08:30:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 -- 172.21.15.113:0/1646387700 shutdown_connections 2024-03-04T08:30:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f5f300cc8e0 0x7f5f3014d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f5f0c066f60 0x7f5f0c069420 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f300cc500 0x7f5f3014ae20 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 --2- 172.21.15.113:0/1646387700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f30073360 0x7f5f3015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 -- 172.21.15.113:0/1646387700 >> 172.21.15.113:0/1646387700 conn(0x7f5f30066db0 msgr2=0x7f5f30157460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:31.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 -- 172.21.15.113:0/1646387700 shutdown_connections 2024-03-04T08:30:31.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.476+0000 7f5f34e00640 1 -- 172.21.15.113:0/1646387700 wait complete. 2024-03-04T08:30:31.490 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T08:30:31.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.632+0000 7f3e07e00640 1 Processor -- start 2024-03-04T08:30:31.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.632+0000 7f3e07e00640 1 -- start start 2024-03-04T08:30:31.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e07e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f3e0806e6a0 0x7f3e080c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e080c3900 0x7f3e080c2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e07e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e080c9530 con 0x7f3e0806ea80 2024-03-04T08:30:31.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e07e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0815b7d0 con 0x7f3e0806e6a0 2024-03-04T08:30:31.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e07e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0815abe0 con 0x7f3e080c3900 2024-03-04T08:30:31.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3dfe000640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f3e0806ea80 0x7f3e080c31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50300/0 (socket says 172.21.15.113:50300) 2024-03-04T08:30:31.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3dfe000640 1 -- 172.21.15.113:0/967862577 learned_addr learned my addr 172.21.15.113:0/967862577 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:31.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e07400640 1 --2- 172.21.15.113:0/967862577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e080c3900 0x7f3e080c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.631 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e06a00640 1 --2- 172.21.15.113:0/967862577 >> v2:172.21.15.136:3300/0 conn(0x7f3e0806e6a0 0x7f3e080c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e06000640 1 -- 172.21.15.113:0/967862577 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695316239 0 0) 0x7f3e080c9530 con 0x7f3e0806ea80 2024-03-04T08:30:31.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e06000640 1 -- 172.21.15.113:0/967862577 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dec003530 con 0x7f3e0806ea80 2024-03-04T08:30:31.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e06a00640 1 -- 172.21.15.113:0/967862577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e080c3900 msgr2=0x7f3e080c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:31.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e06a00640 1 --2- 172.21.15.113:0/967862577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e080c3900 0x7f3e080c2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e06000640 1 -- 172.21.15.113:0/967862577 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1516434209 0 0) 0x7f3dec003530 con 0x7f3e0806ea80 2024-03-04T08:30:31.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e06a00640 1 -- 172.21.15.113:0/967862577 >> v1:172.21.15.113:6789/0 conn(0x7f3e0806ea80 legacy=0x7f3e080c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.636+0000 7f3e06a00640 1 -- 172.21.15.113:0/967862577 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0815aea0 con 0x7f3e0806e6a0 2024-03-04T08:30:31.634 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e06a00640 1 --2- 172.21.15.113:0/967862577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e0806e6a0 0x7f3e080c1f50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3df00097c0 tx=0x7f3df001ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c40f4c2a8085111 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:31.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e06000640 1 -- 172.21.15.113:0/967862577 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df00b5070 con 0x7f3e0806e6a0 2024-03-04T08:30:31.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e06000640 1 -- 172.21.15.113:0/967862577 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df00a85f0 con 0x7f3e0806e6a0 2024-03-04T08:30:31.635 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e06000640 1 -- 172.21.15.113:0/967862577 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8001680 con 0x7f3e0806e6a0 2024-03-04T08:30:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e07e00640 1 -- 172.21.15.113:0/967862577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e0806e6a0 msgr2=0x7f3e080c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e07e00640 1 --2- 172.21.15.113:0/967862577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e0806e6a0 0x7f3e080c1f50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3df00097c0 tx=0x7f3df001ebb0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e07e00640 1 -- 172.21.15.113:0/967862577 shutdown_connections 2024-03-04T08:30:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e07e00640 1 --2- 172.21.15.113:0/967862577 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e080c3900 0x7f3e080c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e07e00640 1 --2- 172.21.15.113:0/967862577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e0806e6a0 0x7f3e080c1f50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e07e00640 1 -- 172.21.15.113:0/967862577 >> 172.21.15.113:0/967862577 conn(0x7f3e0806f460 msgr2=0x7f3e0806f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e07e00640 1 -- 172.21.15.113:0/967862577 shutdown_connections 2024-03-04T08:30:31.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e07e00640 1 -- 172.21.15.113:0/967862577 wait complete. 2024-03-04T08:30:31.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e07e00640 1 Processor -- start 2024-03-04T08:30:31.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.640+0000 7f3e07e00640 1 -- start start 2024-03-04T08:30:31.639 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e0806e6a0 0x7f3e0814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e0806ea80 0x7f3e080ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e080c3900 0x7f3e081f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080c97d0 con 0x7f3e0806ea80 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0815b460 con 0x7f3e080c3900 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080c9140 con 0x7f3e0806e6a0 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dfe000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e0806ea80 0x7f3e080ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dfe000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e0806ea80 0x7f3e080ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56396/0 (socket says 172.21.15.113:56396) 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dfe000640 1 -- 172.21.15.113:0/1768215518 learned_addr learned my addr 172.21.15.113:0/1768215518 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07400640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e080c3900 0x7f3e081f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e06a00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e0806e6a0 0x7f3e0814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dfe000640 1 -- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e0806e6a0 msgr2=0x7f3e0814d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dfe000640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e0806e6a0 0x7f3e0814d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dfe000640 1 -- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e080c3900 msgr2=0x7f3e081f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dfe000640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e080c3900 0x7f3e081f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dfe000640 1 -- 172.21.15.113:0/1768215518 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0814d4a0 con 0x7f3e0806ea80 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e06a00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e0806e6a0 0x7f3e0814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07400640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e080c3900 0x7f3e081f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dfe000640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e0806ea80 0x7f3e080ae8e0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c760 tx=0x7f3df800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dffe00640 1 -- 172.21.15.113:0/1768215518 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8017070 con 0x7f3e0806ea80 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dffe00640 1 -- 172.21.15.113:0/1768215518 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df8005b00 con 0x7f3e0806ea80 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dffe00640 1 -- 172.21.15.113:0/1768215518 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3df8007070 con 0x7f3e0806ea80 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07e00640 1 -- 172.21.15.113:0/1768215518 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e080aef70 con 0x7f3e0806ea80 2024-03-04T08:30:31.641 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07e00640 1 -- 172.21.15.113:0/1768215518 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e081f10e0 con 0x7f3e0806ea80 2024-03-04T08:30:31.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3e07e00640 1 -- 172.21.15.113:0/1768215518 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e08075a80 con 0x7f3e0806ea80 2024-03-04T08:30:31.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.644+0000 7f3dffe00640 1 -- 172.21.15.113:0/1768215518 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3df801a070 con 0x7f3e0806ea80 2024-03-04T08:30:31.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.648+0000 7f3dffe00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3de4066f60 0x7f3de4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.648+0000 7f3dffe00640 1 -- 172.21.15.113:0/1768215518 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f3df8085100 con 0x7f3e0806ea80 2024-03-04T08:30:31.642 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.648+0000 7f3e06a00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3de4066f60 0x7f3de4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.643 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.648+0000 7f3e06a00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3de4066f60 0x7f3de4069420 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3df00b2000 tx=0x7f3df00aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:31.646 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.652+0000 7f3dffe00640 1 -- 172.21.15.113:0/1768215518 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3df80555a0 con 0x7f3e0806ea80 2024-03-04T08:30:31.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.804+0000 7f3e07e00640 1 -- 172.21.15.113:0/1768215518 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e080c4d80 con 0x7f3e0806ea80 2024-03-04T08:30:31.802 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.804+0000 7f3dffe00640 1 -- 172.21.15.113:0/1768215518 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3df8054ea0 con 0x7f3e0806ea80 2024-03-04T08:30:31.802 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-03-04T08:30:31.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 -- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3de4066f60 msgr2=0x7f3de4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3de4066f60 0x7f3de4069420 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3df00b2000 tx=0x7f3df00aa070 comp rx=0 tx=0).stop 2024-03-04T08:30:31.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 -- 172.21.15.113:0/1768215518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e0806ea80 msgr2=0x7f3e080ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e0806ea80 0x7f3e080ae8e0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f3df800c760 tx=0x7f3df800cc20 comp rx=0 tx=0).stop 2024-03-04T08:30:31.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 -- 172.21.15.113:0/1768215518 shutdown_connections 2024-03-04T08:30:31.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3e080c3900 0x7f3e081f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f3de4066f60 0x7f3de4069420 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.804 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e0806ea80 0x7f3e080ae8e0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 --2- 172.21.15.113:0/1768215518 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f3e0806e6a0 0x7f3e0814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 -- 172.21.15.113:0/1768215518 >> 172.21.15.113:0/1768215518 conn(0x7f3e0806f460 msgr2=0x7f3e080c7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:31.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 -- 172.21.15.113:0/1768215518 shutdown_connections 2024-03-04T08:30:31.805 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.808+0000 7f3e07e00640 1 -- 172.21.15.113:0/1768215518 wait complete. 2024-03-04T08:30:31.822 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T08:30:31.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea3e00640 1 Processor -- start 2024-03-04T08:30:31.960 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea3e00640 1 -- start start 2024-03-04T08:30:31.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe9c0cc500 0x7fbe9c0c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c1588b0 con 0x7fbe9c073360 2024-03-04T08:30:31.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c158b70 con 0x7fbe9c0cc8e0 2024-03-04T08:30:31.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c158e50 con 0x7fbe9c0cc500 2024-03-04T08:30:31.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe9c0cc500 0x7fbe9c0c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe9c0cc500 0x7fbe9c0c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56402/0 (socket says 172.21.15.113:56402) 2024-03-04T08:30:31.961 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea1400640 1 -- 172.21.15.113:0/400464366 learned_addr learned my addr 172.21.15.113:0/400464366 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:31.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea0a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fbe9c073360 0x7fbe9c070f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50306/0 (socket says 172.21.15.113:50306) 2024-03-04T08:30:31.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea1400640 1 -- 172.21.15.113:0/400464366 >> v1:172.21.15.136:6789/0 conn(0x7fbe9c0cc8e0 legacy=0x7fbe9c1524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.964+0000 7fbea1400640 1 -- 172.21.15.113:0/400464366 >> v1:172.21.15.113:6789/0 conn(0x7fbe9c073360 legacy=0x7fbe9c070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbea1400640 1 -- 172.21.15.113:0/400464366 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c159130 con 0x7fbe9c0cc500 2024-03-04T08:30:31.962 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbe9be00640 1 -- 172.21.15.113:0/400464366 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646778288 0 0) 0x7fbe9c1588b0 con 0x7fbe9c073360 2024-03-04T08:30:31.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbea1400640 1 --2- 172.21.15.113:0/400464366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c0cc500 0x7fbe9c0c7480 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009870 tx=0x7fbe9001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c99177e1b47eba6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:31.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbe9be00640 1 -- 172.21.15.113:0/400464366 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900b5070 con 0x7fbe9c0cc500 2024-03-04T08:30:31.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbe9be00640 1 -- 172.21.15.113:0/400464366 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe900a8a50 con 0x7fbe9c0cc500 2024-03-04T08:30:31.965 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbe9be00640 1 -- 172.21.15.113:0/400464366 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900a8e20 con 0x7fbe9c0cc500 2024-03-04T08:30:31.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbea3e00640 1 -- 172.21.15.113:0/400464366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c0cc500 msgr2=0x7fbe9c0c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbea3e00640 1 --2- 172.21.15.113:0/400464366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c0cc500 0x7fbe9c0c7480 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009870 tx=0x7fbe9001ed50 comp rx=0 tx=0).stop 2024-03-04T08:30:31.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbea3e00640 1 -- 172.21.15.113:0/400464366 shutdown_connections 2024-03-04T08:30:31.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbea3e00640 1 --2- 172.21.15.113:0/400464366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c0cc500 0x7fbe9c0c7480 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbea3e00640 1 -- 172.21.15.113:0/400464366 >> 172.21.15.113:0/400464366 conn(0x7fbe9c066db0 msgr2=0x7fbe9c0671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:31.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbea3e00640 1 -- 172.21.15.113:0/400464366 shutdown_connections 2024-03-04T08:30:31.966 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.968+0000 7fbea3e00640 1 -- 172.21.15.113:0/400464366 wait complete. 2024-03-04T08:30:31.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea3e00640 1 Processor -- start 2024-03-04T08:30:31.967 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea3e00640 1 -- start start 2024-03-04T08:30:31.969 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea3e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c073360 0x7fbe9c0c1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe9c0cc500 0x7fbe9c1f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe9c0cc8e0 0x7fbe9c1f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea3e00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0717b0 con 0x7fbe9c073360 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea3e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c071630 con 0x7fbe9c0cc8e0 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c151a70 con 0x7fbe9c0cc500 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c073360 0x7fbe9c0c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea1400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c073360 0x7fbe9c0c1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56418/0 (socket says 172.21.15.113:56418) 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea1400640 1 -- 172.21.15.113:0/345596036 learned_addr learned my addr 172.21.15.113:0/345596036 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea1e00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe9c0cc8e0 0x7fbe9c1f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea0a00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe9c0cc500 0x7fbe9c1f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea1400640 1 -- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe9c0cc500 msgr2=0x7fbe9c1f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea1400640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe9c0cc500 0x7fbe9c1f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea1400640 1 -- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe9c0cc8e0 msgr2=0x7fbe9c1f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea1400640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe9c0cc8e0 0x7fbe9c1f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:31.970 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea1400640 1 -- 172.21.15.113:0/345596036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c150940 con 0x7fbe9c073360 2024-03-04T08:30:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea0a00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe9c0cc500 0x7fbe9c1f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:30:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea1400640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c073360 0x7fbe9c0c1090 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fbe900b3040 tx=0x7fbe9001baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbe99600640 1 -- 172.21.15.113:0/345596036 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900c0070 con 0x7fbe9c073360 2024-03-04T08:30:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbe99600640 1 -- 172.21.15.113:0/345596036 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe900b0e40 con 0x7fbe9c073360 2024-03-04T08:30:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbe99600640 1 -- 172.21.15.113:0/345596036 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900b5040 con 0x7fbe9c073360 2024-03-04T08:30:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea3e00640 1 -- 172.21.15.113:0/345596036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c150c20 con 0x7fbe9c073360 2024-03-04T08:30:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.972+0000 7fbea3e00640 1 -- 172.21.15.113:0/345596036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c156190 con 0x7fbe9c073360 2024-03-04T08:30:31.971 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.976+0000 7fbea3e00640 1 -- 172.21.15.113:0/345596036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c0c5ad0 con 0x7fbe9c073360 2024-03-04T08:30:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.976+0000 7fbe99600640 1 -- 172.21.15.113:0/345596036 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe900a8910 con 0x7fbe9c073360 2024-03-04T08:30:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.976+0000 7fbe99600640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe7c066f60 0x7fbe7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.976+0000 7fbe99600640 1 -- 172.21.15.113:0/345596036 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fbe90122f90 con 0x7fbe9c073360 2024-03-04T08:30:31.972 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.976+0000 7fbea0a00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe7c066f60 0x7fbe7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:31.973 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.976+0000 7fbea0a00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe7c066f60 0x7fbe7c069420 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0c7250 tx=0x7fbe88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:31.976 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:31.980+0000 7fbe99600640 1 -- 172.21.15.113:0/345596036 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbe900188b0 con 0x7fbe9c073360 2024-03-04T08:30:32.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 -- 172.21.15.113:0/345596036 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe9c06ba20 con 0x7fbe9c073360 2024-03-04T08:30:32.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbe99600640 1 -- 172.21.15.113:0/345596036 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe900f3550 con 0x7fbe9c073360 2024-03-04T08:30:32.131 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-03-04T08:30:32.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 -- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe7c066f60 msgr2=0x7fbe7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:32.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe7c066f60 0x7fbe7c069420 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0c7250 tx=0x7fbe88009040 comp rx=0 tx=0).stop 2024-03-04T08:30:32.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 -- 172.21.15.113:0/345596036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c073360 msgr2=0x7fbe9c0c1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:32.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c073360 0x7fbe9c0c1090 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fbe900b3040 tx=0x7fbe9001baf0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 -- 172.21.15.113:0/345596036 shutdown_connections 2024-03-04T08:30:32.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe9c0cc8e0 0x7fbe9c1f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbe9c0cc500 0x7fbe9c1f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fbe7c066f60 0x7fbe7c069420 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 --2- 172.21.15.113:0/345596036 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe9c073360 0x7fbe9c0c1090 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 -- 172.21.15.113:0/345596036 >> 172.21.15.113:0/345596036 conn(0x7fbe9c066db0 msgr2=0x7fbe9c074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:32.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 -- 172.21.15.113:0/345596036 shutdown_connections 2024-03-04T08:30:32.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.136+0000 7fbea3e00640 1 -- 172.21.15.113:0/345596036 wait complete. 2024-03-04T08:30:32.153 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T08:30:32.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.292+0000 7f116f600640 1 Processor -- start 2024-03-04T08:30:32.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.292+0000 7f116f600640 1 -- start start 2024-03-04T08:30:32.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f116f600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f116814e4c0 0x7f1168072c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f116f600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f11680c22e0 0x7f11680731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f116f600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f11680c2d80 0x7f116815b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f116f600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116815ad60 con 0x7f11680c22e0 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f116f600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11680c27f0 con 0x7f11680c2d80 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f116f600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11680c2a90 con 0x7f116814e4c0 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f115fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f11680c22e0 0x7f11680731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f115fe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f11680c22e0 0x7f11680731c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56420/0 (socket says 172.21.15.113:56420) 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f115fe00640 1 -- 172.21.15.113:0/2323956683 learned_addr learned my addr 172.21.15.113:0/2323956683 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f116d600640 1 --2- 172.21.15.113:0/2323956683 >> v2:172.21.15.136:3300/0 conn(0x7f11680c2d80 0x7f116815b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f116cc00640 1 --2- 172.21.15.113:0/2323956683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f116814e4c0 0x7f1168072c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f115fe00640 1 -- 172.21.15.113:0/2323956683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f116814e4c0 msgr2=0x7f1168072c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f115fe00640 1 --2- 172.21.15.113:0/2323956683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f116814e4c0 0x7f1168072c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f115fe00640 1 -- 172.21.15.113:0/2323956683 >> v2:172.21.15.136:3300/0 conn(0x7f11680c2d80 msgr2=0x7f116815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:32.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f115fe00640 1 --2- 172.21.15.113:0/2323956683 >> v2:172.21.15.136:3300/0 conn(0x7f11680c2d80 0x7f116815b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.293 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.296+0000 7f115fe00640 1 -- 172.21.15.113:0/2323956683 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168070c60 con 0x7f11680c22e0 2024-03-04T08:30:32.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f115fe00640 1 --2- 172.21.15.113:0/2323956683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 0x7f11680731c0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f11680c2700 tx=0x7f115001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1ffde6b42d0abd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:32.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f1167e00640 1 -- 172.21.15.113:0/2323956683 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500b5070 con 0x7f11680c22e0 2024-03-04T08:30:32.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f1167e00640 1 -- 172.21.15.113:0/2323956683 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11500a89d0 con 0x7f11680c22e0 2024-03-04T08:30:32.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f1167e00640 1 -- 172.21.15.113:0/2323956683 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500a8da0 con 0x7f11680c22e0 2024-03-04T08:30:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 -- 172.21.15.113:0/2323956683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 msgr2=0x7f11680731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 --2- 172.21.15.113:0/2323956683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 0x7f11680731c0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f11680c2700 tx=0x7f115001ed10 comp rx=0 tx=0).stop 2024-03-04T08:30:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 -- 172.21.15.113:0/2323956683 shutdown_connections 2024-03-04T08:30:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 --2- 172.21.15.113:0/2323956683 >> v2:172.21.15.136:3300/0 conn(0x7f11680c2d80 0x7f116815b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 --2- 172.21.15.113:0/2323956683 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 0x7f11680731c0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 --2- 172.21.15.113:0/2323956683 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f116814e4c0 0x7f1168072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 -- 172.21.15.113:0/2323956683 >> 172.21.15.113:0/2323956683 conn(0x7f116806e820 msgr2=0x7f116806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 -- 172.21.15.113:0/2323956683 shutdown_connections 2024-03-04T08:30:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 -- 172.21.15.113:0/2323956683 wait complete. 2024-03-04T08:30:32.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 Processor -- start 2024-03-04T08:30:32.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.300+0000 7f116f600640 1 -- start start 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116f600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 0x7f11681e2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116f600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11680c2d80 0x7f1168147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116f600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f116814e4c0 0x7f11681ef310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116f600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1168071500 con 0x7f11680c22e0 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116f600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1168071220 con 0x7f11680c2d80 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116f600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116814e8a0 con 0x7f116814e4c0 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 0x7f11681e2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116cc00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 0x7f11681e2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56428/0 (socket says 172.21.15.113:56428) 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116cc00640 1 -- 172.21.15.113:0/1389510382 learned_addr learned my addr 172.21.15.113:0/1389510382 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f115fe00640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11680c2d80 0x7f1168147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116d600640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f116814e4c0 0x7f11681ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116cc00640 1 -- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f116814e4c0 msgr2=0x7f11681ef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116cc00640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f116814e4c0 0x7f11681ef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116cc00640 1 -- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11680c2d80 msgr2=0x7f1168147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116cc00640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11680c2d80 0x7f1168147e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116cc00640 1 -- 172.21.15.113:0/1389510382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11681eee30 con 0x7f11680c22e0 2024-03-04T08:30:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116cc00640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 0x7f11681e2750 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f115800d5f0 tx=0x7f115800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:32.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f1165600640 1 -- 172.21.15.113:0/1389510382 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11580125f0 con 0x7f11680c22e0 2024-03-04T08:30:32.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f1165600640 1 -- 172.21.15.113:0/1389510382 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1158019070 con 0x7f11680c22e0 2024-03-04T08:30:32.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f1165600640 1 -- 172.21.15.113:0/1389510382 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11580129b0 con 0x7f11680c22e0 2024-03-04T08:30:32.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116f600640 1 -- 172.21.15.113:0/1389510382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11681e1bd0 con 0x7f11680c22e0 2024-03-04T08:30:32.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116f600640 1 -- 172.21.15.113:0/1389510382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11681e2190 con 0x7f11680c22e0 2024-03-04T08:30:32.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f116f600640 1 -- 172.21.15.113:0/1389510382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11681eec20 con 0x7f11680c22e0 2024-03-04T08:30:32.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.304+0000 7f1165600640 1 -- 172.21.15.113:0/1389510382 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1158012b50 con 0x7f11680c22e0 2024-03-04T08:30:32.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.308+0000 7f1165600640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1144066f60 0x7f1144069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:32.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.308+0000 7f1165600640 1 -- 172.21.15.113:0/1389510382 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f1158085cf0 con 0x7f11680c22e0 2024-03-04T08:30:32.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.308+0000 7f115fe00640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1144066f60 0x7f1144069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:32.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.308+0000 7f115fe00640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1144066f60 0x7f1144069420 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f11500b2000 tx=0x7f1150009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:32.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.312+0000 7f1165600640 1 -- 172.21.15.113:0/1389510382 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f115808a070 con 0x7f11680c22e0 2024-03-04T08:30:32.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.476+0000 7f116f600640 1 -- 172.21.15.113:0/1389510382 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1168075c00 con 0x7f11680c22e0 2024-03-04T08:30:32.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.476+0000 7f1165600640 1 -- 172.21.15.113:0/1389510382 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 60 entries v85) v1 ==== 81+0+3706 (secure 0 0 0) 0x7f11580562b0 con 0x7f11680c22e0 2024-03-04T08:30:32.473 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6839/1409794802 2024-03-05T08:24:26.023653+0000 2024-03-04T08:30:32.473 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/2368734308 2024-03-05T07:58:45.780423+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/3005395426 2024-03-05T07:37:40.596420+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/986515123 2024-03-05T08:00:40.798160+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/2299268553 2024-03-05T07:52:10.722331+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/3944001911 2024-03-05T07:45:20.664657+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/2299268553 2024-03-05T07:52:10.722331+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/2522634751 2024-03-05T08:07:30.861130+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/1691762848 2024-03-05T07:49:30.698378+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/2732565639 2024-03-05T08:02:45.817095+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/742633123 2024-03-05T07:47:40.683981+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/4038214481 2024-03-05T07:36:05.584380+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/3944001911 2024-03-05T07:45:20.664657+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/986515123 2024-03-05T08:00:40.798160+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/605142559 2024-03-05T07:44:20.656043+0000 2024-03-04T08:30:32.474 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/1035982016 2024-03-05T08:05:45.840907+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/3005395426 2024-03-05T07:37:40.596420+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/1928887637 2024-03-05T08:20:05.977949+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/810789479 2024-03-05T07:54:40.746387+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/2544977425 2024-03-05T08:21:40.993913+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/4197507712 2024-03-05T08:04:30.830241+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/3348592059 2024-03-05T07:40:15.619912+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/1310042629 2024-03-05T07:38:50.604670+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/2522634751 2024-03-05T08:07:30.861130+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/742633123 2024-03-05T07:47:40.683981+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/1310057122 2024-03-05T08:13:45.911820+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/4038214481 2024-03-05T07:36:05.584380+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/1691762848 2024-03-05T07:49:30.698378+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/605142559 2024-03-05T07:44:20.656043+0000 2024-03-04T08:30:32.475 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/2544977425 2024-03-05T08:21:40.993913+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/344022786 2024-03-05T07:57:05.766508+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/2999156262 2024-03-05T08:22:51.004939+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/1401330284 2024-03-05T08:11:10.891335+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/1310042629 2024-03-05T07:38:50.604670+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/2997246507 2024-03-05T07:41:40.631034+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/2368734308 2024-03-05T07:58:45.780423+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/1310057122 2024-03-05T08:13:45.911820+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/3348592059 2024-03-05T07:40:15.619912+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6837/2228772746 2024-03-05T08:23:26.011590+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/2997246507 2024-03-05T07:41:40.631034+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/3388011401 2024-03-05T08:01:30.805387+0000 2024-03-04T08:30:32.476 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/247269911 2024-03-05T08:16:00.936524+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/2732565639 2024-03-05T08:02:45.817095+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/4197507712 2024-03-05T08:04:30.830241+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6838/1409794802 2024-03-05T08:24:26.023653+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/1035982016 2024-03-05T08:05:45.840907+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/3825257303 2024-03-05T08:09:55.880790+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/3388011401 2024-03-05T08:01:30.805387+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/3825257303 2024-03-05T08:09:55.880790+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/1401330284 2024-03-05T08:11:10.891335+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/247269911 2024-03-05T08:16:00.936524+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/1928887637 2024-03-05T08:20:05.977949+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/4083765155 2024-03-05T08:17:55.953381+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/2999156262 2024-03-05T08:22:51.004939+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/4083765155 2024-03-05T08:17:55.953381+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/810789479 2024-03-05T07:54:40.746387+0000 2024-03-04T08:30:32.477 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/893414524 2024-03-05T08:21:25.990484+0000 2024-03-04T08:30:32.478 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/344022786 2024-03-05T07:57:05.766508+0000 2024-03-04T08:30:32.478 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/893414524 2024-03-05T08:21:25.990484+0000 2024-03-04T08:30:32.478 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6836/2228772746 2024-03-05T08:23:26.011590+0000 2024-03-04T08:30:32.478 INFO:teuthology.orchestra.run.smithi113.stderr:listed 60 entries 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 -- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1144066f60 msgr2=0x7f1144069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1144066f60 0x7f1144069420 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f11500b2000 tx=0x7f1150009040 comp rx=0 tx=0).stop 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 -- 172.21.15.113:0/1389510382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 msgr2=0x7f11681e2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 0x7f11681e2750 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f115800d5f0 tx=0x7f115800dab0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 -- 172.21.15.113:0/1389510382 shutdown_connections 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f116814e4c0 0x7f11681ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11680c2d80 0x7f1168147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1144066f60 0x7f1144069420 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 --2- 172.21.15.113:0/1389510382 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f11680c22e0 0x7f11681e2750 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 -- 172.21.15.113:0/1389510382 >> 172.21.15.113:0/1389510382 conn(0x7f116806e820 msgr2=0x7f116806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 -- 172.21.15.113:0/1389510382 shutdown_connections 2024-03-04T08:30:32.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:32.480+0000 7f116f600640 1 -- 172.21.15.113:0/1389510382 wait complete. 2024-03-04T08:30:32.494 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-04T08:30:32.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:30:32.494 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-04T08:30:32.537 INFO:teuthology.orchestra.run.smithi113.stderr:umount: /home/ubuntu/cephtest/mnt.0: target is busy. 2024-03-04T08:30:32.539 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/CodingStyle': Input/output error 2024-03-04T08:30:32.539 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/install-deps.sh': Input/output error 2024-03-04T08:30:32.539 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/PendingReleaseNotes': Input/output error 2024-03-04T08:30:32.539 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/m4': Input/output error 2024-03-04T08:30:32.539 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/.git': Input/output error 2024-03-04T08:30:32.540 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/.mailmap': Input/output error 2024-03-04T08:30:32.540 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/rpm': Input/output error 2024-03-04T08:30:32.540 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/Makefile.am': Input/output error 2024-03-04T08:30:32.540 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/INSTALL': Input/output error 2024-03-04T08:30:32.540 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/CONTRIBUTING.rst': Input/output error 2024-03-04T08:30:32.540 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/doc_deps.deb.txt': Input/output error 2024-03-04T08:30:32.541 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/AUTHORS': Input/output error 2024-03-04T08:30:32.541 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/udev': Input/output error 2024-03-04T08:30:32.541 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/.gitmodule_mirrors': Input/output error 2024-03-04T08:30:32.541 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/debian': Input/output error 2024-03-04T08:30:32.541 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/README': Input/output error 2024-03-04T08:30:32.541 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/Doxyfile': Input/output error 2024-03-04T08:30:32.541 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/configure.ac': Input/output error 2024-03-04T08:30:32.542 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/qa': Input/output error 2024-03-04T08:30:32.542 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/.gitmodules': Input/output error 2024-03-04T08:30:32.542 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/.gitignore': Input/output error 2024-03-04T08:30:32.542 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/admin': Input/output error 2024-03-04T08:30:32.542 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/COPYING': Input/output error 2024-03-04T08:30:32.542 INFO:tasks.workunit.client.0.smithi113.stderr:rm: cannot remove '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/autogen.sh': Input/output error 2024-03-04T08:30:32.542 INFO:tasks.workunit.client.0.smithi113.stderr:rm: fts_read failed: Input/output error 2024-03-04T08:30:32.543 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-04T08:30:32.543 DEBUG:tasks.cephfs.kernel_mount:Killing processes on client.0... 2024-03-04T08:30:32.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:30:32.543 DEBUG:teuthology.orchestra.run.smithi113:> sudo PATH=/usr/sbin:$PATH lsof ; ps auxf 2024-03-04T08:30:32.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:30:32.544 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2024-03-04T08:30:32.545 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T08:30:32.637 INFO:teuthology.orchestra.run.smithi113.stderr:lsof: WARNING: can't stat() ceph file system /home/ubuntu/cephtest/mnt.0 2024-03-04T08:30:32.637 INFO:teuthology.orchestra.run.smithi113.stderr: Output information may be incomplete. 2024-03-04T08:30:33.441 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 441, in _run_tests remote.run(logger=log.getChild(role), args=args, timeout=(60*60)) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' 2024-03-04T08:30:33.685 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5dda335f72604facba74f52871b27d41 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/workunit.py", line 441, in _run_tests remote.run(logger=log.getChild(role), args=args, timeout=(60*60)) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 1: 'sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp' 2024-03-04T08:30:33.688 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2024-03-04T08:30:33.699 INFO:tasks.mds_thrash:joining mds_thrasher 2024-03-04T08:30:33.699 ERROR:teuthology.run_tasks:Manager failed: mds_thrash Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/mds_thrash.py", line 432, in task raise RuntimeError('error during thrashing') RuntimeError: error during thrashing 2024-03-04T08:30:33.705 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-04T08:30:33.742 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-04T08:30:33.742 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T08:30:33.742 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T08:30:33.756 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T08:30:33.756 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T08:30:33.806 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:30:33.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.988+0000 7fcd01000640 1 Processor -- start 2024-03-04T08:30:33.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.988+0000 7fcd01000640 1 -- start start 2024-03-04T08:30:33.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fcd01000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fccfc0ce6b0 0x7fccfc15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:33.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fcd01000640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccfc06c210 con 0x7fccfc0ce2d0 2024-03-04T08:30:33.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fcd01000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccfc06c4f0 con 0x7fccfc06de80 2024-03-04T08:30:33.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fcd01000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccfc15bbf0 con 0x7fccfc0ce6b0 2024-03-04T08:30:33.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fccf9e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fccfc0ce2d0 0x7fccfc06f010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50316/0 (socket says 172.21.15.113:50316) 2024-03-04T08:30:33.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fccf9e00640 1 -- 172.21.15.113:0/2308397264 learned_addr learned my addr 172.21.15.113:0/2308397264 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:33.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fccf3e00640 1 --2- 172.21.15.113:0/2308397264 >> v2:172.21.15.136:3300/0 conn(0x7fccfc0ce6b0 0x7fccfc15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:33.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fccf9400640 1 -- 172.21.15.113:0/2308397264 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596120241 0 0) 0x7fccfc06c210 con 0x7fccfc0ce2d0 2024-03-04T08:30:33.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fccf9400640 1 -- 172.21.15.113:0/2308397264 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce0003530 con 0x7fccfc0ce2d0 2024-03-04T08:30:33.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fccf9400640 1 -- 172.21.15.113:0/2308397264 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 526088649 0 0) 0x7fcce0003530 con 0x7fccfc0ce2d0 2024-03-04T08:30:33.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fccf3e00640 1 -- 172.21.15.113:0/2308397264 >> v1:172.21.15.136:6789/0 conn(0x7fccfc06de80 legacy=0x7fccfc154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:33.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fccf3e00640 1 -- 172.21.15.113:0/2308397264 >> v1:172.21.15.113:6789/0 conn(0x7fccfc0ce2d0 legacy=0x7fccfc06f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:33.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.992+0000 7fccf3e00640 1 -- 172.21.15.113:0/2308397264 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc06e530 con 0x7fccfc0ce6b0 2024-03-04T08:30:33.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fccf3e00640 1 --2- 172.21.15.113:0/2308397264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccfc0ce6b0 0x7fccfc15b6b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcce40097c0 tx=0x7fcce401ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27abe09f6ba13899 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:33.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fccf9400640 1 -- 172.21.15.113:0/2308397264 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce40b5070 con 0x7fccfc0ce6b0 2024-03-04T08:30:33.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fccf9400640 1 -- 172.21.15.113:0/2308397264 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce40a8630 con 0x7fccfc0ce6b0 2024-03-04T08:30:33.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fccf9400640 1 -- 172.21.15.113:0/2308397264 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec001680 con 0x7fccfc0ce6b0 2024-03-04T08:30:33.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fcd01000640 1 -- 172.21.15.113:0/2308397264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccfc0ce6b0 msgr2=0x7fccfc15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:33.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fcd01000640 1 --2- 172.21.15.113:0/2308397264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccfc0ce6b0 0x7fccfc15b6b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcce40097c0 tx=0x7fcce401ebb0 comp rx=0 tx=0).stop 2024-03-04T08:30:33.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fcd01000640 1 -- 172.21.15.113:0/2308397264 shutdown_connections 2024-03-04T08:30:33.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fcd01000640 1 --2- 172.21.15.113:0/2308397264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccfc0ce6b0 0x7fccfc15b6b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:33.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fcd01000640 1 -- 172.21.15.113:0/2308397264 >> 172.21.15.113:0/2308397264 conn(0x7fccfc0709a0 msgr2=0x7fccfc070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:33.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fcd01000640 1 -- 172.21.15.113:0/2308397264 shutdown_connections 2024-03-04T08:30:33.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fcd01000640 1 -- 172.21.15.113:0/2308397264 wait complete. 2024-03-04T08:30:33.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fcd01000640 1 Processor -- start 2024-03-04T08:30:33.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:33.996+0000 7fcd01000640 1 -- start start 2024-03-04T08:30:33.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fcd01000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccfc06de80 0x7fccfc0c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:33.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fcd01000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccfc0ce2d0 0x7fccfc152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:33.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fcd01000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccfc0ce6b0 0x7fccfc153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:33.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fcd01000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc06c7d0 con 0x7fccfc06de80 2024-03-04T08:30:33.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fcd01000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc06eb80 con 0x7fccfc0ce6b0 2024-03-04T08:30:33.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fcd01000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc06f720 con 0x7fccfc0ce2d0 2024-03-04T08:30:33.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccfc06de80 0x7fccfc0c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:33.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf9e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccfc06de80 0x7fccfc0c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56440/0 (socket says 172.21.15.113:56440) 2024-03-04T08:30:33.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf9e00640 1 -- 172.21.15.113:0/2377682369 learned_addr learned my addr 172.21.15.113:0/2377682369 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf3e00640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccfc0ce2d0 0x7fccfc152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccfa800640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccfc0ce6b0 0x7fccfc153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf9e00640 1 -- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccfc0ce2d0 msgr2=0x7fccfc152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf9e00640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccfc0ce2d0 0x7fccfc152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:33.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf9e00640 1 -- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccfc0ce6b0 msgr2=0x7fccfc153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf9e00640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccfc0ce6b0 0x7fccfc153490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf9e00640 1 -- 172.21.15.113:0/2377682369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc1f4040 con 0x7fccfc06de80 2024-03-04T08:30:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf3e00640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccfc0ce2d0 0x7fccfc152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccfa800640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccfc0ce6b0 0x7fccfc153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:30:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf9e00640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccfc06de80 0x7fccfc0c7310 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fccec00c730 tx=0x7fccec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf2000640 1 -- 172.21.15.113:0/2377682369 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec017070 con 0x7fccfc06de80 2024-03-04T08:30:33.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf2000640 1 -- 172.21.15.113:0/2377682369 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec005b00 con 0x7fccfc06de80 2024-03-04T08:30:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fcd01000640 1 -- 172.21.15.113:0/2377682369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccfc1f4320 con 0x7fccfc06de80 2024-03-04T08:30:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fccf2000640 1 -- 172.21.15.113:0/2377682369 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec007070 con 0x7fccfc06de80 2024-03-04T08:30:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fcd01000640 1 -- 172.21.15.113:0/2377682369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccfc1f48e0 con 0x7fccfc06de80 2024-03-04T08:30:33.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.000+0000 7fcd01000640 1 -- 172.21.15.113:0/2377682369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccfc075c00 con 0x7fccfc06de80 2024-03-04T08:30:34.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.004+0000 7fccf2000640 1 -- 172.21.15.113:0/2377682369 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fccec01a070 con 0x7fccfc06de80 2024-03-04T08:30:34.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.004+0000 7fccf2000640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccd8066f60 0x7fccd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.004+0000 7fccf2000640 1 -- 172.21.15.113:0/2377682369 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fccec085120 con 0x7fccfc06de80 2024-03-04T08:30:34.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.004+0000 7fccf3e00640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccd8066f60 0x7fccd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.004+0000 7fccf3e00640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccd8066f60 0x7fccd8069420 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcce40b2000 tx=0x7fcce40aa070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:34.007 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.012+0000 7fccf2000640 1 -- 172.21.15.113:0/2377682369 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fccec0555c0 con 0x7fccfc06de80 2024-03-04T08:30:34.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.168+0000 7fcd01000640 1 -- 172.21.15.113:0/2377682369 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccfc153a20 con 0x7fccfc06de80 2024-03-04T08:30:34.167 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fccf2000640 1 -- 172.21.15.113:0/2377682369 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+20859 (secure 0 0 0) 0x7fccec054ec0 con 0x7fccfc06de80 2024-03-04T08:30:34.168 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:34.168 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T08:24:26.023707+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":683,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.136:6839/1409794802":"2024-03-05T08:24:26.023653+0000","172.21.15.136:6834/2368734308":"2024-03-05T07:58:45.780423+0000","172.21.15.113:6832/3005395426":"2024-03-05T07:37:40.596420+0000","172.21.15.113:6836/986515123":"2024-03-05T08:00:40.798160+0000","172.21.15.113:6834/2299268553":"2024-03-05T07:52:10.722331+0000","172.21.15.136:6834/3944001911":"2024-03-05T07:45:20.664657+0000","172.21.15.113:6836/2299268553":"2024-03-05T07:52:10.722331+0000","172.21.15.136:6835/2522634751":"2024-03-05T08:07:30.861130+0000","172.21.15.136:6834/1691762848":"2024-03-05T07:49:30.698378+0000","172.21.15.113:6832/2732565639":"2024-03-05T08:02:45.817095+0000","172.21.15.113:6837/742633123":"2024-03-05T07:47:40.683981+0000","172.21.15.113:6836/4038214481":"2024-03-05T07:36:05.584380+0000","172.21.15.136:6835/3944001911":"2024-03-05T07:45:20.664657+0000","172.21.15.113:6834/986515123":"2024-03-05T08:00:40.798160+0000","172.21.15.113:6833/605142559":"2024-03-05T07:44:20.656043+0000","172.21.15.113:6836/1035982016":"2024-03-05T08:05:45.840907+0000","172.21.15.113:6833/3005395426":"2024-03-05T07:37:40.596420+0000","172.21.15.113:6832/1928887637":"2024-03-05T08:20:05.977949+0000","172.21.15.113:6833/810789479":"2024-03-05T07:54:40.746387+0000","172.21.15.113:6835/2544977425":"2024-03-05T08:21:40.993913+0000","172.21.15.113:6837/4197507712":"2024-03-05T08:04:30.830241+0000","172.21.15.136:6834/3348592059":"2024-03-05T07:40:15.619912+0000","172.21.15.113:6835/1310042629":"2024-03-05T07:38:50.604670+0000","172.21.15.136:6834/2522634751":"2024-03-05T08:07:30.861130+0000","172.21.15.113:6835/742633123":"2024-03-05T07:47:40.683981+0000","172.21.15.113:6834/1310057122":"2024-03-05T08:13:45.911820+0000","172.21.15.113:6834/4038214481":"2024-03-05T07:36:05.584380+0000","172.21.15.136:6835/1691762848":"2024-03-05T07:49:30.698378+0000","172.21.15.113:6832/605142559":"2024-03-05T07:44:20.656043+0000","172.21.15.113:6837/2544977425":"2024-03-05T08:21:40.993913+0000","172.21.15.113:6835/344022786":"2024-03-05T07:57:05.766508+0000","172.21.15.113:6832/2999156262":"2024-03-05T08:22:51.004939+0000","172.21.15.113:6837/1401330284":"2024-03-05T08:11:10.891335+0000","172.21.15.113:6837/1310042629":"2024-03-05T07:38:50.604670+0000","172.21.15.113:6834/2997246507":"2024-03-05T07:41:40.631034+0000","172.21.15.136:6835/2368734308":"2024-03-05T07:58:45.780423+0000","172.21.15.113:6836/1310057122":"2024-03-05T08:13:45.911820+0000","172.21.15.136:6835/3348592059":"2024-03-05T07:40:15.619912+0000","172.21.15.136:6837/2228772746":"2024-03-05T08:23:26.011590+0000","172.21.15.113:6836/2997246507":"2024-03-05T07:41:40.631034+0000","172.21.15.136:6835/3388011401":"2024-03-05T08:01:30.805387+0000","172.21.15.136:6835/247269911":"2024-03-05T08:16:00.936524+0000","172.21.15.113:6833/2732565639":"2024-03-05T08:02:45.817095+0000","172.21.15.113:6835/4197507712":"2024-03-05T08:04:30.830241+0000","172.21.15.136:6838/1409794802":"2024-03-05T08:24:26.023653+0000","172.21.15.113:6834/1035982016":"2024-03-05T08:05:45.840907+0000","172.21.15.113:6832/3825257303":"2024-03-05T08:09:55.880790+0000","172.21.15.136:6834/3388011401":"2024-03-05T08:01:30.805387+0000","172.21.15.113:6833/3825257303":"2024-03-05T08:09:55.880790+0000","172.21.15.113:6835/1401330284":"2024-03-05T08:11:10.891335+0000","172.21.15.136:6834/247269911":"2024-03-05T08:16:00.936524+0000","172.21.15.113:6833/1928887637":"2024-03-05T08:20:05.977949+0000","172.21.15.113:6834/4083765155":"2024-03-05T08:17:55.953381+0000","172.21.15.113:6833/2999156262":"2024-03-05T08:22:51.004939+0000","172.21.15.113:6836/4083765155":"2024-03-05T08:17:55.953381+0000","172.21.15.113:6832/810789479":"2024-03-05T07:54:40.746387+0000","172.21.15.113:6834/893414524":"2024-03-05T08:21:25.990484+0000","172.21.15.113:6837/344022786":"2024-03-05T07:57:05.766508+0000","172.21.15.113:6836/893414524":"2024-03-05T08:21:25.990484+0000","172.21.15.136:6836/2228772746":"2024-03-05T08:23:26.011590+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:30:34.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 -- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccd8066f60 msgr2=0x7fccd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccd8066f60 0x7fccd8069420 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcce40b2000 tx=0x7fcce40aa070 comp rx=0 tx=0).stop 2024-03-04T08:30:34.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 -- 172.21.15.113:0/2377682369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccfc06de80 msgr2=0x7fccfc0c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccfc06de80 0x7fccfc0c7310 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fccec00c730 tx=0x7fccec00cbf0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 -- 172.21.15.113:0/2377682369 shutdown_connections 2024-03-04T08:30:34.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccfc0ce6b0 0x7fccfc153490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fccfc0ce2d0 0x7fccfc152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.170 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fccd8066f60 0x7fccd8069420 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 --2- 172.21.15.113:0/2377682369 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fccfc06de80 0x7fccfc0c7310 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 -- 172.21.15.113:0/2377682369 >> 172.21.15.113:0/2377682369 conn(0x7fccfc0709a0 msgr2=0x7fccfc15add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:34.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 -- 172.21.15.113:0/2377682369 shutdown_connections 2024-03-04T08:30:34.171 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.172+0000 7fcd01000640 1 -- 172.21.15.113:0/2377682369 wait complete. 2024-03-04T08:30:34.189 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T07:33:41.507700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-04T07:33:53.680465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-04T07:33:54.892043+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 683, 'snap_epoch': 75, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T08:30:34.190 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-04T08:30:34.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf3a600640 1 Processor -- start 2024-03-04T08:30:34.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf3a600640 1 -- start start 2024-03-04T08:30:34.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf3a600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fdf3406ff60 0x7fdf340ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf3a600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf340c9ce0 con 0x7fdf340ce770 2024-03-04T08:30:34.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf3a600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3406e110 con 0x7fdf34158320 2024-03-04T08:30:34.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf3a600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf340c9160 con 0x7fdf3406ff60 2024-03-04T08:30:34.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf33e00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fdf340ce770 0x7fdf3415ae90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50322/0 (socket says 172.21.15.113:50322) 2024-03-04T08:30:34.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf33e00640 1 -- 172.21.15.113:0/2647352730 learned_addr learned my addr 172.21.15.113:0/2647352730 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:34.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf2be00640 1 --2- 172.21.15.113:0/2647352730 >> v2:172.21.15.136:3300/0 conn(0x7fdf3406ff60 0x7fdf340ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf32a00640 1 -- 172.21.15.113:0/2647352730 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472801619 0 0) 0x7fdf340c9ce0 con 0x7fdf340ce770 2024-03-04T08:30:34.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf32a00640 1 -- 172.21.15.113:0/2647352730 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf18003530 con 0x7fdf340ce770 2024-03-04T08:30:34.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf32a00640 1 -- 172.21.15.113:0/2647352730 <== mon.1 v1:172.21.15.136:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493609035 0 0) 0x7fdf3406e110 con 0x7fdf34158320 2024-03-04T08:30:34.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf32a00640 1 -- 172.21.15.113:0/2647352730 --> v1:172.21.15.136:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf340c9ce0 con 0x7fdf34158320 2024-03-04T08:30:34.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf2be00640 1 -- 172.21.15.113:0/2647352730 >> v1:172.21.15.136:6789/0 conn(0x7fdf34158320 legacy=0x7fdf3406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf2be00640 1 -- 172.21.15.113:0/2647352730 >> v1:172.21.15.113:6789/0 conn(0x7fdf340ce770 legacy=0x7fdf3415ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf32a00640 1 -- 172.21.15.113:0/2647352730 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3953297610 0 0) 0x7fdf18003530 con 0x7fdf340ce770 2024-03-04T08:30:34.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.336+0000 7fdf2be00640 1 -- 172.21.15.113:0/2647352730 --> v2:172.21.15.136:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf340c93b0 con 0x7fdf3406ff60 2024-03-04T08:30:34.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf2be00640 1 --2- 172.21.15.113:0/2647352730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf3406ff60 0x7fdf340ce230 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0097c0 tx=0x7fdf1c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab53e3df73ddfa09 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:34.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf32a00640 1 -- 172.21.15.113:0/2647352730 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c0b0070 con 0x7fdf3406ff60 2024-03-04T08:30:34.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf32a00640 1 -- 172.21.15.113:0/2647352730 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c0ab400 con 0x7fdf3406ff60 2024-03-04T08:30:34.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf32a00640 1 -- 172.21.15.113:0/2647352730 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c001a00 con 0x7fdf3406ff60 2024-03-04T08:30:34.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf3a600640 1 -- 172.21.15.113:0/2647352730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf3406ff60 msgr2=0x7fdf340ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf3a600640 1 --2- 172.21.15.113:0/2647352730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf3406ff60 0x7fdf340ce230 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0097c0 tx=0x7fdf1c01c3c0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf3a600640 1 -- 172.21.15.113:0/2647352730 shutdown_connections 2024-03-04T08:30:34.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf3a600640 1 --2- 172.21.15.113:0/2647352730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf3406ff60 0x7fdf340ce230 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf3a600640 1 -- 172.21.15.113:0/2647352730 >> 172.21.15.113:0/2647352730 conn(0x7fdf3406b820 msgr2=0x7fdf34070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:34.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf3a600640 1 -- 172.21.15.113:0/2647352730 shutdown_connections 2024-03-04T08:30:34.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.340+0000 7fdf3a600640 1 -- 172.21.15.113:0/2647352730 wait complete. 2024-03-04T08:30:34.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf3a600640 1 Processor -- start 2024-03-04T08:30:34.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf3a600640 1 -- start start 2024-03-04T08:30:34.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf3a600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf3406ff60 0x7fdf340b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf3a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf340ce770 0x7fdf34148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf3a600640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf34158320 0x7fdf34148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf3a600640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34168d90 con 0x7fdf34158320 2024-03-04T08:30:34.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf3a600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34168c10 con 0x7fdf340ce770 2024-03-04T08:30:34.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf3a600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34168f10 con 0x7fdf3406ff60 2024-03-04T08:30:34.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf33e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf34158320 0x7fdf34148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf33e00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf34158320 0x7fdf34148d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56454/0 (socket says 172.21.15.113:56454) 2024-03-04T08:30:34.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf33e00640 1 -- 172.21.15.113:0/962688884 learned_addr learned my addr 172.21.15.113:0/962688884 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:34.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf33400640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf3406ff60 0x7fdf340b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf2be00640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf340ce770 0x7fdf34148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf33e00640 1 -- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf3406ff60 msgr2=0x7fdf340b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf33e00640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf3406ff60 0x7fdf340b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf33e00640 1 -- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf340ce770 msgr2=0x7fdf34148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf33e00640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf340ce770 0x7fdf34148820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf33e00640 1 -- 172.21.15.113:0/962688884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf341482a0 con 0x7fdf34158320 2024-03-04T08:30:34.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf33e00640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf34158320 0x7fdf34148d60 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c001570 tx=0x7fdf2c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:34.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf2b400640 1 -- 172.21.15.113:0/962688884 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c012400 con 0x7fdf34158320 2024-03-04T08:30:34.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf2b400640 1 -- 172.21.15.113:0/962688884 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c0125c0 con 0x7fdf34158320 2024-03-04T08:30:34.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf2b400640 1 -- 172.21.15.113:0/962688884 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0129b0 con 0x7fdf34158320 2024-03-04T08:30:34.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf3a600640 1 -- 172.21.15.113:0/962688884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf340af9f0 con 0x7fdf34158320 2024-03-04T08:30:34.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.344+0000 7fdf3a600640 1 -- 172.21.15.113:0/962688884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf340afc90 con 0x7fdf34158320 2024-03-04T08:30:34.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.348+0000 7fdf3a600640 1 -- 172.21.15.113:0/962688884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf34075c00 con 0x7fdf34158320 2024-03-04T08:30:34.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.348+0000 7fdf2b400640 1 -- 172.21.15.113:0/962688884 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf2c012b50 con 0x7fdf34158320 2024-03-04T08:30:34.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.348+0000 7fdf2b400640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf10066f90 0x7fdf10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.348+0000 7fdf2b400640 1 -- 172.21.15.113:0/962688884 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fdf2c086120 con 0x7fdf34158320 2024-03-04T08:30:34.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.348+0000 7fdf33400640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf10066f90 0x7fdf10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.345 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.348+0000 7fdf33400640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf10066f90 0x7fdf10069450 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fdf240011f0 tx=0x7fdf24009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:34.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.356+0000 7fdf2b400640 1 -- 172.21.15.113:0/962688884 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fdf2c056030 con 0x7fdf34158320 2024-03-04T08:30:34.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.512+0000 7fdf3a600640 1 -- 172.21.15.113:0/962688884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf3406fd40 con 0x7fdf34158320 2024-03-04T08:30:34.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.512+0000 7fdf2b400640 1 -- 172.21.15.113:0/962688884 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf2c05a0d0 con 0x7fdf34158320 2024-03-04T08:30:34.510 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 1 2024-03-04T08:30:34.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 -- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf10066f90 msgr2=0x7fdf10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf10066f90 0x7fdf10069450 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fdf240011f0 tx=0x7fdf24009040 comp rx=0 tx=0).stop 2024-03-04T08:30:34.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 -- 172.21.15.113:0/962688884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf34158320 msgr2=0x7fdf34148d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf34158320 0x7fdf34148d60 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c001570 tx=0x7fdf2c00a430 comp rx=0 tx=0).stop 2024-03-04T08:30:34.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 -- 172.21.15.113:0/962688884 shutdown_connections 2024-03-04T08:30:34.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fdf10066f90 0x7fdf10069450 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf34158320 0x7fdf34148d60 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf340ce770 0x7fdf34148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 --2- 172.21.15.113:0/962688884 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fdf3406ff60 0x7fdf340b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 -- 172.21.15.113:0/962688884 >> 172.21.15.113:0/962688884 conn(0x7fdf3406b820 msgr2=0x7fdf34070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:34.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 -- 172.21.15.113:0/962688884 shutdown_connections 2024-03-04T08:30:34.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.516+0000 7fdf3a600640 1 -- 172.21.15.113:0/962688884 wait complete. 2024-03-04T08:30:34.532 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-04T08:30:34.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.676+0000 7f0360800640 1 Processor -- start 2024-03-04T08:30:34.673 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.676+0000 7f0360800640 1 -- start start 2024-03-04T08:30:34.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.676+0000 7f0360800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 0x7f035806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.676+0000 7f0360800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f035806ff60 0x7f03580ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.676+0000 7f0360800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f03580ce770 0x7f035815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.676+0000 7f0360800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03580c9160 con 0x7f035806ff60 2024-03-04T08:30:34.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.676+0000 7f0360800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035806c6b0 con 0x7f03580ce770 2024-03-04T08:30:34.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.676+0000 7f0360800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03580c9ce0 con 0x7f0358158320 2024-03-04T08:30:34.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.680+0000 7f035d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f035806ff60 0x7f03580ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.680+0000 7f035d400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f035806ff60 0x7f03580ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56468/0 (socket says 172.21.15.113:56468) 2024-03-04T08:30:34.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.680+0000 7f035d400640 1 -- 172.21.15.113:0/1960109698 learned_addr learned my addr 172.21.15.113:0/1960109698 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:34.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.680+0000 7f035e800640 1 --2- 172.21.15.113:0/1960109698 >> v2:172.21.15.136:3300/0 conn(0x7f03580ce770 0x7f035815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.680+0000 7f035de00640 1 --2- 172.21.15.113:0/1960109698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 0x7f035806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.680+0000 7f035d400640 1 -- 172.21.15.113:0/1960109698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 msgr2=0x7f035806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:34.676 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.680+0000 7f035d400640 1 --2- 172.21.15.113:0/1960109698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 0x7f035806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.680+0000 7f035d400640 1 -- 172.21.15.113:0/1960109698 >> v2:172.21.15.136:3300/0 conn(0x7f03580ce770 msgr2=0x7f035815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:34.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.680+0000 7f035d400640 1 --2- 172.21.15.113:0/1960109698 >> v2:172.21.15.136:3300/0 conn(0x7f03580ce770 0x7f035815ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.677 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.680+0000 7f035d400640 1 -- 172.21.15.113:0/1960109698 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035806e110 con 0x7f035806ff60 2024-03-04T08:30:34.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f035d400640 1 --2- 172.21.15.113:0/1960109698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 0x7f03580ce230 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f03580cf1f0 tx=0x7f034801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2393440aa48cfbc server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:34.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f035ca00640 1 -- 172.21.15.113:0/1960109698 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03480b0070 con 0x7f035806ff60 2024-03-04T08:30:34.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f035ca00640 1 -- 172.21.15.113:0/1960109698 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03480ab600 con 0x7f035806ff60 2024-03-04T08:30:34.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f035ca00640 1 -- 172.21.15.113:0/1960109698 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03480ab9d0 con 0x7f035806ff60 2024-03-04T08:30:34.680 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f0360800640 1 -- 172.21.15.113:0/1960109698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 msgr2=0x7f03580ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f0360800640 1 --2- 172.21.15.113:0/1960109698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 0x7f03580ce230 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f03580cf1f0 tx=0x7f034801c520 comp rx=0 tx=0).stop 2024-03-04T08:30:34.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f0360800640 1 -- 172.21.15.113:0/1960109698 shutdown_connections 2024-03-04T08:30:34.681 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f0360800640 1 --2- 172.21.15.113:0/1960109698 >> v2:172.21.15.136:3300/0 conn(0x7f03580ce770 0x7f035815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f0360800640 1 --2- 172.21.15.113:0/1960109698 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 0x7f03580ce230 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f0360800640 1 --2- 172.21.15.113:0/1960109698 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 0x7f035806c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f0360800640 1 -- 172.21.15.113:0/1960109698 >> 172.21.15.113:0/1960109698 conn(0x7f035806b820 msgr2=0x7f0358070920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:34.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f0360800640 1 -- 172.21.15.113:0/1960109698 shutdown_connections 2024-03-04T08:30:34.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f0360800640 1 -- 172.21.15.113:0/1960109698 wait complete. 2024-03-04T08:30:34.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.684+0000 7f0360800640 1 Processor -- start 2024-03-04T08:30:34.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0360800640 1 -- start start 2024-03-04T08:30:34.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0360800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 0x7f03581573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0360800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03580ce770 0x7f0358144940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0360800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 0x7f0358144e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0360800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035806dcb0 con 0x7f035806ff60 2024-03-04T08:30:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0360800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035815b7e0 con 0x7f03580ce770 2024-03-04T08:30:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0360800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035806de30 con 0x7f0358158320 2024-03-04T08:30:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 0x7f03581573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035de00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 0x7f03581573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56484/0 (socket says 172.21.15.113:56484) 2024-03-04T08:30:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035de00640 1 -- 172.21.15.113:0/2302167088 learned_addr learned my addr 172.21.15.113:0/2302167088 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:34.685 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035d400640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03580ce770 0x7f0358144940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035e800640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 0x7f0358144e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035de00640 1 -- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 msgr2=0x7f0358144e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035de00640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 0x7f0358144e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035de00640 1 -- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03580ce770 msgr2=0x7f0358144940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035de00640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03580ce770 0x7f0358144940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035de00640 1 -- 172.21.15.113:0/2302167088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035814fab0 con 0x7f035806ff60 2024-03-04T08:30:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035e800640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 0x7f0358144e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:34.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f035de00640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 0x7f03581573e0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f034c00e8a0 tx=0x7f034c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0352000640 1 -- 172.21.15.113:0/2302167088 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c017070 con 0x7f035806ff60 2024-03-04T08:30:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0352000640 1 -- 172.21.15.113:0/2302167088 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c00c940 con 0x7f035806ff60 2024-03-04T08:30:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0360800640 1 -- 172.21.15.113:0/2302167088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035815c170 con 0x7f035806ff60 2024-03-04T08:30:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0352000640 1 -- 172.21.15.113:0/2302167088 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c00ccf0 con 0x7f035806ff60 2024-03-04T08:30:34.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0360800640 1 -- 172.21.15.113:0/2302167088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035814fd20 con 0x7f035806ff60 2024-03-04T08:30:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.688+0000 7f0360800640 1 -- 172.21.15.113:0/2302167088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0358075c00 con 0x7f035806ff60 2024-03-04T08:30:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.692+0000 7f0352000640 1 -- 172.21.15.113:0/2302167088 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f034c01a070 con 0x7f035806ff60 2024-03-04T08:30:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.692+0000 7f0352000640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f033c066f90 0x7f033c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:34.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.692+0000 7f0352000640 1 -- 172.21.15.113:0/2302167088 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f034c0859d0 con 0x7f035806ff60 2024-03-04T08:30:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.692+0000 7f035d400640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f033c066f90 0x7f033c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:34.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.692+0000 7f035d400640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f033c066f90 0x7f033c069450 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f03480ad000 tx=0x7f0348008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:34.691 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.696+0000 7f0352000640 1 -- 172.21.15.113:0/2302167088 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f034c055820 con 0x7f035806ff60 2024-03-04T08:30:34.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.852+0000 7f0360800640 1 -- 172.21.15.113:0/2302167088 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0358165170 con 0x7f035806ff60 2024-03-04T08:30:34.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.852+0000 7f0352000640 1 -- 172.21.15.113:0/2302167088 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) v1 ==== 104+0+11 (secure 0 0 0) 0x7f034c0598c0 con 0x7f035806ff60 2024-03-04T08:30:34.849 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 32 2024-03-04T08:30:34.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 -- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f033c066f90 msgr2=0x7f033c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f033c066f90 0x7f033c069450 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f03480ad000 tx=0x7f0348008040 comp rx=0 tx=0).stop 2024-03-04T08:30:34.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 -- 172.21.15.113:0/2302167088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 msgr2=0x7f03581573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:34.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 0x7f03581573e0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f034c00e8a0 tx=0x7f034c00eca0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 -- 172.21.15.113:0/2302167088 shutdown_connections 2024-03-04T08:30:34.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f0358158320 0x7f0358144e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03580ce770 0x7f0358144940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f033c066f90 0x7f033c069450 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 --2- 172.21.15.113:0/2302167088 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f035806ff60 0x7f03581573e0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:34.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 -- 172.21.15.113:0/2302167088 >> 172.21.15.113:0/2302167088 conn(0x7f035806b820 msgr2=0x7f0358147ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:34.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 -- 172.21.15.113:0/2302167088 shutdown_connections 2024-03-04T08:30:34.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:34.856+0000 7f0360800640 1 -- 172.21.15.113:0/2302167088 wait complete. 2024-03-04T08:30:34.871 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-04T08:30:35.009 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.012+0000 7fd08c400640 1 Processor -- start 2024-03-04T08:30:35.010 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.012+0000 7fd08c400640 1 -- start start 2024-03-04T08:30:35.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd08c400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0840cc8e0 0x7fd0840c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:35.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd08c400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0841588b0 con 0x7fd0840cc500 2024-03-04T08:30:35.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd08c400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd084158b70 con 0x7fd084073360 2024-03-04T08:30:35.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd08c400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd084158e50 con 0x7fd0840cc8e0 2024-03-04T08:30:35.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd089a00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fd0840cc500 0x7fd084070f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50338/0 (socket says 172.21.15.113:50338) 2024-03-04T08:30:35.011 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd089a00640 1 -- 172.21.15.113:0/285374420 learned_addr learned my addr 172.21.15.113:0/285374420 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:35.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd08a400640 1 --2- 172.21.15.113:0/285374420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0840cc8e0 0x7fd0840c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:35.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd083e00640 1 -- 172.21.15.113:0/285374420 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160955194 0 0) 0x7fd0841588b0 con 0x7fd0840cc500 2024-03-04T08:30:35.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd083e00640 1 -- 172.21.15.113:0/285374420 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd06c003530 con 0x7fd0840cc500 2024-03-04T08:30:35.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd08a400640 1 -- 172.21.15.113:0/285374420 >> v1:172.21.15.136:6789/0 conn(0x7fd084073360 legacy=0x7fd0841524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:35.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd08a400640 1 -- 172.21.15.113:0/285374420 >> v1:172.21.15.113:6789/0 conn(0x7fd0840cc500 legacy=0x7fd084070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:35.013 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.016+0000 7fd08a400640 1 -- 172.21.15.113:0/285374420 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd084159130 con 0x7fd0840cc8e0 2024-03-04T08:30:35.014 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd08a400640 1 --2- 172.21.15.113:0/285374420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0840cc8e0 0x7fd0840c7480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009df0 tx=0x7fd07c01f180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de18edb8f504d16 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:35.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd083e00640 1 -- 172.21.15.113:0/285374420 <== mon.2 v2:172.21.15.136:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0b4070 con 0x7fd0840cc8e0 2024-03-04T08:30:35.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd083e00640 1 -- 172.21.15.113:0/285374420 <== mon.2 v2:172.21.15.136:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c0af600 con 0x7fd0840cc8e0 2024-03-04T08:30:35.015 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd083e00640 1 -- 172.21.15.113:0/285374420 <== mon.2 v2:172.21.15.136:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0af9b0 con 0x7fd0840cc8e0 2024-03-04T08:30:35.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd08c400640 1 -- 172.21.15.113:0/285374420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0840cc8e0 msgr2=0x7fd0840c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:35.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd08c400640 1 --2- 172.21.15.113:0/285374420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0840cc8e0 0x7fd0840c7480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009df0 tx=0x7fd07c01f180 comp rx=0 tx=0).stop 2024-03-04T08:30:35.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd08c400640 1 -- 172.21.15.113:0/285374420 shutdown_connections 2024-03-04T08:30:35.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd08c400640 1 --2- 172.21.15.113:0/285374420 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd0840cc8e0 0x7fd0840c7480 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:35.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd08c400640 1 -- 172.21.15.113:0/285374420 >> 172.21.15.113:0/285374420 conn(0x7fd084066db0 msgr2=0x7fd0840671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:35.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd08c400640 1 -- 172.21.15.113:0/285374420 shutdown_connections 2024-03-04T08:30:35.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd08c400640 1 -- 172.21.15.113:0/285374420 wait complete. 2024-03-04T08:30:35.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd08c400640 1 Processor -- start 2024-03-04T08:30:35.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.020+0000 7fd08c400640 1 -- start start 2024-03-04T08:30:35.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd08c400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd084073360 0x7fd084159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:35.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd08c400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0840cc500 0x7fd08414b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:35.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd08c400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0840cc8e0 0x7fd08414bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:35.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd08c400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840717b0 con 0x7fd0840cc500 2024-03-04T08:30:35.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd08c400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd084071630 con 0x7fd0840cc8e0 2024-03-04T08:30:35.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd08c400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0840b0650 con 0x7fd084073360 2024-03-04T08:30:35.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd089000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0840cc500 0x7fd08414b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:35.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd089a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd084073360 0x7fd084159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:35.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd089000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0840cc500 0x7fd08414b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56492/0 (socket says 172.21.15.113:56492) 2024-03-04T08:30:35.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd089000640 1 -- 172.21.15.113:0/2226272257 learned_addr learned my addr 172.21.15.113:0/2226272257 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:35.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd08a400640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0840cc8e0 0x7fd08414bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:35.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd089000640 1 -- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd084073360 msgr2=0x7fd084159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:35.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd089000640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd084073360 0x7fd084159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:35.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd089000640 1 -- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0840cc8e0 msgr2=0x7fd08414bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:35.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd089000640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0840cc8e0 0x7fd08414bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:35.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd089000640 1 -- 172.21.15.113:0/2226272257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0841f09b0 con 0x7fd0840cc500 2024-03-04T08:30:35.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd089000640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0840cc500 0x7fd08414b9a0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fd07000c500 tx=0x7fd07000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:35.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd081600640 1 -- 172.21.15.113:0/2226272257 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0700052d0 con 0x7fd0840cc500 2024-03-04T08:30:35.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd081600640 1 -- 172.21.15.113:0/2226272257 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd070005470 con 0x7fd0840cc500 2024-03-04T08:30:35.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd081600640 1 -- 172.21.15.113:0/2226272257 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070012830 con 0x7fd0840cc500 2024-03-04T08:30:35.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd08c400640 1 -- 172.21.15.113:0/2226272257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0841f0c90 con 0x7fd0840cc500 2024-03-04T08:30:35.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd08c400640 1 -- 172.21.15.113:0/2226272257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0841f1250 con 0x7fd0840cc500 2024-03-04T08:30:35.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.024+0000 7fd08c400640 1 -- 172.21.15.113:0/2226272257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd084075c00 con 0x7fd0840cc500 2024-03-04T08:30:35.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.028+0000 7fd081600640 1 -- 172.21.15.113:0/2226272257 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0700129d0 con 0x7fd0840cc500 2024-03-04T08:30:35.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.028+0000 7fd081600640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd064066f90 0x7fd064069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:35.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.028+0000 7fd081600640 1 -- 172.21.15.113:0/2226272257 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fd070085640 con 0x7fd0840cc500 2024-03-04T08:30:35.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.028+0000 7fd089a00640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd064066f90 0x7fd064069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:35.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.028+0000 7fd089a00640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd064066f90 0x7fd064069450 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd0780011f0 tx=0x7fd078007d40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:35.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.032+0000 7fd081600640 1 -- 172.21.15.113:0/2226272257 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd0700554d0 con 0x7fd0840cc500 2024-03-04T08:30:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.200+0000 7fd08c400640 1 -- 172.21.15.113:0/2226272257 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0841568c0 con 0x7fd0840cc500 2024-03-04T08:30:35.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.200+0000 7fd081600640 1 -- 172.21.15.113:0/2226272257 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd070059570 con 0x7fd0840cc500 2024-03-04T08:30:35.198 INFO:teuthology.orchestra.run.smithi113.stdout:pg_num: 64 2024-03-04T08:30:35.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 -- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd064066f90 msgr2=0x7fd064069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:35.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd064066f90 0x7fd064069450 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd0780011f0 tx=0x7fd078007d40 comp rx=0 tx=0).stop 2024-03-04T08:30:35.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 -- 172.21.15.113:0/2226272257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0840cc500 msgr2=0x7fd08414b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0840cc500 0x7fd08414b9a0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fd07000c500 tx=0x7fd07000c9c0 comp rx=0 tx=0).stop 2024-03-04T08:30:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 -- 172.21.15.113:0/2226272257 shutdown_connections 2024-03-04T08:30:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0840cc8e0 0x7fd08414bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fd064066f90 0x7fd064069450 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd0840cc500 0x7fd08414b9a0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 --2- 172.21.15.113:0/2226272257 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fd084073360 0x7fd084159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 -- 172.21.15.113:0/2226272257 >> 172.21.15.113:0/2226272257 conn(0x7fd084066db0 msgr2=0x7fd0840c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:35.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 -- 172.21.15.113:0/2226272257 shutdown_connections 2024-03-04T08:30:35.202 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.204+0000 7fd08c400640 1 -- 172.21.15.113:0/2226272257 wait complete. 2024-03-04T08:30:35.219 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-04T08:30:35.271 INFO:teuthology.orchestra.run.smithi113.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-03-04T08:30:35.271 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.271 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.272 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root txt REG 8,1 1849992 19775 /usr/lib/systemd/systemd 2024-03-04T08:30:35.272 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.272 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.272 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.272 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.272 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.272 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.272 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 2840280 9114 /usr/lib/systemd/libsystemd-shared-249.so 2024-03-04T08:30:35.273 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.273 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.273 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.273 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 35440 15410 /usr/lib/x86_64-linux-gnu/libip4tc.so.2.0.0 2024-03-04T08:30:35.273 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.273 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.273 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.273 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 34888 16814 /usr/lib/x86_64-linux-gnu/libacl.so.1.1.2301 2024-03-04T08:30:35.273 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 80768 8466 /usr/lib/x86_64-linux-gnu/libapparmor.so.1.8.2 2024-03-04T08:30:35.274 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.274 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.274 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.274 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.274 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.274 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 125360 9036 /usr/lib/x86_64-linux-gnu/libseccomp.so.2.5.3 2024-03-04T08:30:35.274 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.274 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.274 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.275 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.275 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 3w CHR 1,11 0t0 10 /dev/kmsg 2024-03-04T08:30:35.275 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 4u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.275 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 5u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.275 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 6r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.275 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 7r DIR 0,26 0 1 /sys/fs/cgroup 2024-03-04T08:30:35.275 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 8u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.275 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 9u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.275 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 10r REG 0,21 0 397 /proc/1/mountinfo 2024-03-04T08:30:35.276 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 11r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.276 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 12r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.276 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 13r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.276 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 14r REG 0,21 0 4026532078 /proc/swaps 2024-03-04T08:30:35.276 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 15u netlink 0t0 398 KOBJECT_UEVENT 2024-03-04T08:30:35.276 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 16r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.276 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 17r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.276 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 18u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.277 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 19u unix 0xffff8aaf09bb2580 0t0 402 /run/systemd/private type=STREAM 2024-03-04T08:30:35.277 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 20u unix 0xffff8aaf09bb2d00 0t0 404 /run/systemd/userdb/io.systemd.DynamicUser type=STREAM 2024-03-04T08:30:35.277 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 21u unix 0xffff8aaf09bb3480 0t0 405 /run/systemd/io.system.ManagedOOM type=STREAM 2024-03-04T08:30:35.277 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 22u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.277 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 23u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.277 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 24u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.277 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 25u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.277 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 26u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.277 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 27u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.278 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 28r CHR 10,235 0t0 442 /dev/autofs 2024-03-04T08:30:35.278 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 29u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.278 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 30u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.278 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 31u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.278 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 32u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.278 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 33u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.278 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 34u a_inode 0,14 0 2070 bpf-prog 2024-03-04T08:30:35.278 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 40u unix 0xffff8aaf09bb0f00 0t0 399 /run/systemd/notify type=DGRAM 2024-03-04T08:30:35.279 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 42u unix 0xffff8aaf09bb1680 0t0 400 type=DGRAM 2024-03-04T08:30:35.279 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 45u unix 0xffff8aaf09749680 0t0 99628 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.279 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 49u unix 0xffff8aaf09bb1e00 0t0 401 type=DGRAM 2024-03-04T08:30:35.279 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 52u unix 0xffff8aaf03728780 0t0 6430 type=DGRAM 2024-03-04T08:30:35.279 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 53u netlink 0t0 6431 AUDIT 2024-03-04T08:30:35.279 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 54u unix 0xffff8aaf03729680 0t0 6596 type=DGRAM 2024-03-04T08:30:35.279 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 55u unix 0xffff8aaf03728f00 0t0 6597 type=DGRAM 2024-03-04T08:30:35.279 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 58u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.279 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 70u unix 0xffff8aaf7d7f2d00 0t0 4719 type=STREAM 2024-03-04T08:30:35.280 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 85u unix 0xffff8aaf719e1680 0t0 99501 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.280 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 86u unix 0xffff8aaf036ae180 0t0 93622 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.280 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 87u unix 0xffff8aaf029eda00 0t0 45303 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.280 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 88u unix 0xffff8aaf09bd8f00 0t0 9592 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.280 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 89u unix 0xffff8aaf0974f080 0t0 7614 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.280 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 90u unix 0xffff8aaf0bf52580 0t0 5488 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.280 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 91u unix 0xffff8aaf09bdd280 0t0 1434 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.280 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 92u unix 0xffff8aaf0974cb00 0t0 5478 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.281 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 95u unix 0xffff8aaf07bf0780 0t0 5433 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.281 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 96u unix 0xffff8aaf0372e180 0t0 4739 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.281 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 97u unix 0xffff8aaf0ad26900 0t0 4736 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.281 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 98u unix 0xffff8aaf06486900 0t0 4731 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.281 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 99u unix 0xffff8aaf7d7f7800 0t0 658 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.281 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 100u unix 0xffff8aaf0ad20f00 0t0 4727 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.281 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 101u unix 0xffff8aaf0ad22580 0t0 8441 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.281 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 102u unix 0xffff8aaf0372a580 0t0 8440 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.281 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 114u unix 0xffff8aaf06484380 0t0 3067 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.282 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 115u unix 0xffff8aaf0ad26180 0t0 4688 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.282 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 118u unix 0xffff8aaf09f95a00 0t0 7286 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.282 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 119u unix 0xffff8aaf029ead00 0t0 83620 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.282 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 120u unix 0xffff8aaf0bf54380 0t0 95695 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.292 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 125u unix 0xffff8aaf09bb4b00 0t0 427 /run/systemd/journal/syslog type=DGRAM 2024-03-04T08:30:35.292 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 126u unix 0xffff8aaf09bb6180 0t0 438 /run/systemd/journal/socket type=DGRAM 2024-03-04T08:30:35.293 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 127u unix 0xffff8aaf09bb6900 0t0 440 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.293 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 128u unix 0xffff8aaf09bb3c00 0t0 413 /run/lvm/lvmpolld.socket type=STREAM 2024-03-04T08:30:35.293 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 129r FIFO 0,13 0t0 410 pipe 2024-03-04T08:30:35.293 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 130u unix 0xffff8aaf09f91e00 0t0 1281 /run/uuidd/request type=STREAM 2024-03-04T08:30:35.293 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 131u unix 0xffff8aaf09bb7080 0t0 443 /run/udev/control type=SEQPACKET 2024-03-04T08:30:35.293 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 132u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.293 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 133u netlink 0t0 442 ROUTE 2024-03-04T08:30:35.294 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 134u unix 0xffff8aaf09bb4380 0t0 415 /run/rpcbind.sock type=STREAM 2024-03-04T08:30:35.294 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 135u IPv4 4452 0t0 TCP *:sunrpc (LISTEN) 2024-03-04T08:30:35.294 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 136u IPv4 5231 0t0 UDP *:sunrpc 2024-03-04T08:30:35.294 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 137u IPv6 6411 0t0 TCP *:sunrpc (LISTEN) 2024-03-04T08:30:35.294 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 138u IPv6 4453 0t0 UDP *:sunrpc 2024-03-04T08:30:35.294 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 139u netlink 0t0 435 AUDIT 2024-03-04T08:30:35.294 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 150u netlink 0t0 445 KOBJECT_UEVENT 2024-03-04T08:30:35.294 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 151u FIFO 0,23 0t0 429 /run/dmeventd-server 2024-03-04T08:30:35.295 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 152u FIFO 0,23 0t0 430 /run/dmeventd-client 2024-03-04T08:30:35.295 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 153u unix 0xffff8aaf09f97800 0t0 1280 @ISCSIADM_ABSTRACT_NAMESPACE type=STREAM 2024-03-04T08:30:35.295 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 154u unix 0xffff8aaf09bb5a00 0t0 436 /run/systemd/journal/dev-log type=DGRAM 2024-03-04T08:30:35.295 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 155u FIFO 0,23 0t0 436 /run/initctl 2024-03-04T08:30:35.295 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 156u unix 0xffff8aaf09f92d00 0t0 1276 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.295 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1 root 157u unix 0xffff8aaf09bb5280 0t0 429 /run/systemd/fsck.progress type=STREAM 2024-03-04T08:30:35.295 INFO:teuthology.orchestra.run.smithi113.stdout:kthreadd 2 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.295 INFO:teuthology.orchestra.run.smithi113.stdout:kthreadd 2 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.296 INFO:teuthology.orchestra.run.smithi113.stdout:kthreadd 2 root txt unknown /proc/2/exe 2024-03-04T08:30:35.296 INFO:teuthology.orchestra.run.smithi113.stdout:pool_work 3 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.296 INFO:teuthology.orchestra.run.smithi113.stdout:pool_work 3 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.296 INFO:teuthology.orchestra.run.smithi113.stdout:pool_work 3 root txt unknown /proc/3/exe 2024-03-04T08:30:35.296 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 4 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.296 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 4 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.296 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 4 root txt unknown /proc/4/exe 2024-03-04T08:30:35.296 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 5 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.297 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 5 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.297 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 5 root txt unknown /proc/5/exe 2024-03-04T08:30:35.297 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 6 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.297 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 6 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.297 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 6 root txt unknown /proc/6/exe 2024-03-04T08:30:35.297 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.297 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.297 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7 root txt unknown /proc/7/exe 2024-03-04T08:30:35.298 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 10 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.298 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 10 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.298 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 10 root txt unknown /proc/10/exe 2024-03-04T08:30:35.298 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 12 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.298 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 12 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.298 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 12 root txt unknown /proc/12/exe 2024-03-04T08:30:35.298 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_tasks 13 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.298 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_tasks 13 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.298 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_tasks 13 root txt unknown /proc/13/exe 2024-03-04T08:30:35.299 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_tasks 14 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.299 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_tasks 14 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.299 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_tasks 14 root txt unknown /proc/14/exe 2024-03-04T08:30:35.299 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_tasks 15 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.299 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_tasks 15 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.299 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_tasks 15 root txt unknown /proc/15/exe 2024-03-04T08:30:35.299 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 16 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.299 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 16 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.299 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 16 root txt unknown /proc/16/exe 2024-03-04T08:30:35.300 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_preem 17 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.300 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_preem 17 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.300 INFO:teuthology.orchestra.run.smithi113.stdout:rcu_preem 17 root txt unknown /proc/17/exe 2024-03-04T08:30:35.300 INFO:teuthology.orchestra.run.smithi113.stdout:migration 18 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.300 INFO:teuthology.orchestra.run.smithi113.stdout:migration 18 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.300 INFO:teuthology.orchestra.run.smithi113.stdout:migration 18 root txt unknown /proc/18/exe 2024-03-04T08:30:35.300 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 19 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.300 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 19 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.301 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 19 root txt unknown /proc/19/exe 2024-03-04T08:30:35.301 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/0 20 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.301 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/0 20 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.301 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/0 20 root txt unknown /proc/20/exe 2024-03-04T08:30:35.301 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/1 21 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.301 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/1 21 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.301 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/1 21 root txt unknown /proc/21/exe 2024-03-04T08:30:35.301 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 22 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.301 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 22 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.302 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 22 root txt unknown /proc/22/exe 2024-03-04T08:30:35.302 INFO:teuthology.orchestra.run.smithi113.stdout:migration 23 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.302 INFO:teuthology.orchestra.run.smithi113.stdout:migration 23 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.302 INFO:teuthology.orchestra.run.smithi113.stdout:migration 23 root txt unknown /proc/23/exe 2024-03-04T08:30:35.302 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 24 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.302 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 24 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.302 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 24 root txt unknown /proc/24/exe 2024-03-04T08:30:35.302 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 26 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.303 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 26 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.303 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 26 root txt unknown /proc/26/exe 2024-03-04T08:30:35.303 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/2 27 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.303 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/2 27 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.303 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/2 27 root txt unknown /proc/27/exe 2024-03-04T08:30:35.303 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 28 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.303 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 28 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.303 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 28 root txt unknown /proc/28/exe 2024-03-04T08:30:35.303 INFO:teuthology.orchestra.run.smithi113.stdout:migration 29 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.304 INFO:teuthology.orchestra.run.smithi113.stdout:migration 29 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.304 INFO:teuthology.orchestra.run.smithi113.stdout:migration 29 root txt unknown /proc/29/exe 2024-03-04T08:30:35.304 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 30 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.304 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 30 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.304 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 30 root txt unknown /proc/30/exe 2024-03-04T08:30:35.304 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 32 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.304 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 32 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.304 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 32 root txt unknown /proc/32/exe 2024-03-04T08:30:35.305 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/3 33 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.305 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/3 33 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.305 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/3 33 root txt unknown /proc/33/exe 2024-03-04T08:30:35.305 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 34 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.305 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 34 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.305 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 34 root txt unknown /proc/34/exe 2024-03-04T08:30:35.305 INFO:teuthology.orchestra.run.smithi113.stdout:migration 35 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.305 INFO:teuthology.orchestra.run.smithi113.stdout:migration 35 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.306 INFO:teuthology.orchestra.run.smithi113.stdout:migration 35 root txt unknown /proc/35/exe 2024-03-04T08:30:35.306 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 36 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.306 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 36 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.306 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 36 root txt unknown /proc/36/exe 2024-03-04T08:30:35.306 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 38 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.306 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 38 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.306 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 38 root txt unknown /proc/38/exe 2024-03-04T08:30:35.306 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/4 39 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.306 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/4 39 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.307 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/4 39 root txt unknown /proc/39/exe 2024-03-04T08:30:35.307 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 40 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.307 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 40 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.307 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 40 root txt unknown /proc/40/exe 2024-03-04T08:30:35.307 INFO:teuthology.orchestra.run.smithi113.stdout:migration 41 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.307 INFO:teuthology.orchestra.run.smithi113.stdout:migration 41 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.307 INFO:teuthology.orchestra.run.smithi113.stdout:migration 41 root txt unknown /proc/41/exe 2024-03-04T08:30:35.307 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 42 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.308 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 42 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.308 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 42 root txt unknown /proc/42/exe 2024-03-04T08:30:35.308 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 44 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.308 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 44 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.308 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 44 root txt unknown /proc/44/exe 2024-03-04T08:30:35.308 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/5 45 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.308 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/5 45 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.308 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/5 45 root txt unknown /proc/45/exe 2024-03-04T08:30:35.308 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 46 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.309 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 46 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.309 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 46 root txt unknown /proc/46/exe 2024-03-04T08:30:35.309 INFO:teuthology.orchestra.run.smithi113.stdout:migration 47 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.309 INFO:teuthology.orchestra.run.smithi113.stdout:migration 47 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.309 INFO:teuthology.orchestra.run.smithi113.stdout:migration 47 root txt unknown /proc/47/exe 2024-03-04T08:30:35.309 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 48 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.309 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 48 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.309 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 48 root txt unknown /proc/48/exe 2024-03-04T08:30:35.310 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 50 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.310 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 50 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.310 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 50 root txt unknown /proc/50/exe 2024-03-04T08:30:35.310 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/6 51 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.310 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/6 51 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.310 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/6 51 root txt unknown /proc/51/exe 2024-03-04T08:30:35.310 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 52 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.310 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 52 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.311 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 52 root txt unknown /proc/52/exe 2024-03-04T08:30:35.311 INFO:teuthology.orchestra.run.smithi113.stdout:migration 53 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.311 INFO:teuthology.orchestra.run.smithi113.stdout:migration 53 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.311 INFO:teuthology.orchestra.run.smithi113.stdout:migration 53 root txt unknown /proc/53/exe 2024-03-04T08:30:35.311 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 54 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.311 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 54 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.311 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 54 root txt unknown /proc/54/exe 2024-03-04T08:30:35.311 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 56 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.312 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 56 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.312 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 56 root txt unknown /proc/56/exe 2024-03-04T08:30:35.312 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/7 57 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.312 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/7 57 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.312 INFO:teuthology.orchestra.run.smithi113.stdout:cpuhp/7 57 root txt unknown /proc/57/exe 2024-03-04T08:30:35.312 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 58 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.312 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 58 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.312 INFO:teuthology.orchestra.run.smithi113.stdout:idle_inje 58 root txt unknown /proc/58/exe 2024-03-04T08:30:35.312 INFO:teuthology.orchestra.run.smithi113.stdout:migration 59 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.313 INFO:teuthology.orchestra.run.smithi113.stdout:migration 59 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.313 INFO:teuthology.orchestra.run.smithi113.stdout:migration 59 root txt unknown /proc/59/exe 2024-03-04T08:30:35.313 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 60 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.313 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 60 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.313 INFO:teuthology.orchestra.run.smithi113.stdout:ksoftirqd 60 root txt unknown /proc/60/exe 2024-03-04T08:30:35.313 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 62 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.313 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 62 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.313 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 62 root txt unknown /proc/62/exe 2024-03-04T08:30:35.314 INFO:teuthology.orchestra.run.smithi113.stdout:kdevtmpfs 70 root cwd DIR 0,5 4320 1 / 2024-03-04T08:30:35.314 INFO:teuthology.orchestra.run.smithi113.stdout:kdevtmpfs 70 root rtd DIR 0,5 4320 1 / 2024-03-04T08:30:35.314 INFO:teuthology.orchestra.run.smithi113.stdout:kdevtmpfs 70 root txt unknown /proc/70/exe 2024-03-04T08:30:35.314 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 71 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.314 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 71 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.314 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 71 root txt unknown /proc/71/exe 2024-03-04T08:30:35.314 INFO:teuthology.orchestra.run.smithi113.stdout:kauditd 72 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.314 INFO:teuthology.orchestra.run.smithi113.stdout:kauditd 72 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.315 INFO:teuthology.orchestra.run.smithi113.stdout:kauditd 72 root txt unknown /proc/72/exe 2024-03-04T08:30:35.315 INFO:teuthology.orchestra.run.smithi113.stdout:khungtask 73 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.315 INFO:teuthology.orchestra.run.smithi113.stdout:khungtask 73 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.315 INFO:teuthology.orchestra.run.smithi113.stdout:khungtask 73 root txt unknown /proc/73/exe 2024-03-04T08:30:35.315 INFO:teuthology.orchestra.run.smithi113.stdout:oom_reape 75 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.315 INFO:teuthology.orchestra.run.smithi113.stdout:oom_reape 75 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.316 INFO:teuthology.orchestra.run.smithi113.stdout:oom_reape 75 root txt unknown /proc/75/exe 2024-03-04T08:30:35.316 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 77 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.316 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 77 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.316 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 77 root txt unknown /proc/77/exe 2024-03-04T08:30:35.316 INFO:teuthology.orchestra.run.smithi113.stdout:kcompactd 78 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.316 INFO:teuthology.orchestra.run.smithi113.stdout:kcompactd 78 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.316 INFO:teuthology.orchestra.run.smithi113.stdout:kcompactd 78 root txt unknown /proc/78/exe 2024-03-04T08:30:35.316 INFO:teuthology.orchestra.run.smithi113.stdout:ksmd 79 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.316 INFO:teuthology.orchestra.run.smithi113.stdout:ksmd 79 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.317 INFO:teuthology.orchestra.run.smithi113.stdout:ksmd 79 root txt unknown /proc/79/exe 2024-03-04T08:30:35.317 INFO:teuthology.orchestra.run.smithi113.stdout:khugepage 80 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.317 INFO:teuthology.orchestra.run.smithi113.stdout:khugepage 80 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.317 INFO:teuthology.orchestra.run.smithi113.stdout:khugepage 80 root txt unknown /proc/80/exe 2024-03-04T08:30:35.317 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 81 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.317 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 81 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.317 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 81 root txt unknown /proc/81/exe 2024-03-04T08:30:35.317 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 82 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.317 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 82 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.318 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 82 root txt unknown /proc/82/exe 2024-03-04T08:30:35.318 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 83 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.318 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 83 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.318 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 83 root txt unknown /proc/83/exe 2024-03-04T08:30:35.318 INFO:teuthology.orchestra.run.smithi113.stdout:irq/9-acp 84 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.318 INFO:teuthology.orchestra.run.smithi113.stdout:irq/9-acp 84 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.318 INFO:teuthology.orchestra.run.smithi113.stdout:irq/9-acp 84 root txt unknown /proc/84/exe 2024-03-04T08:30:35.318 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 85 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.318 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 85 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.319 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 85 root txt unknown /proc/85/exe 2024-03-04T08:30:35.319 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 86 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.319 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 86 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.319 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 86 root txt unknown /proc/86/exe 2024-03-04T08:30:35.319 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 87 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.319 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 87 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.319 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 87 root txt unknown /proc/87/exe 2024-03-04T08:30:35.319 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 88 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.320 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 88 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.320 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 88 root txt unknown /proc/88/exe 2024-03-04T08:30:35.320 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 89 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.320 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 89 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.320 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 89 root txt unknown /proc/89/exe 2024-03-04T08:30:35.320 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 90 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.320 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 90 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.320 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 90 root txt unknown /proc/90/exe 2024-03-04T08:30:35.320 INFO:teuthology.orchestra.run.smithi113.stdout:watchdogd 91 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.321 INFO:teuthology.orchestra.run.smithi113.stdout:watchdogd 91 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.321 INFO:teuthology.orchestra.run.smithi113.stdout:watchdogd 91 root txt unknown /proc/91/exe 2024-03-04T08:30:35.321 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 92 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.321 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 92 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.321 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 92 root txt unknown /proc/92/exe 2024-03-04T08:30:35.321 INFO:teuthology.orchestra.run.smithi113.stdout:kswapd0 93 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.321 INFO:teuthology.orchestra.run.smithi113.stdout:kswapd0 93 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.321 INFO:teuthology.orchestra.run.smithi113.stdout:kswapd0 93 root txt unknown /proc/93/exe 2024-03-04T08:30:35.322 INFO:teuthology.orchestra.run.smithi113.stdout:ecryptfs- 94 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.322 INFO:teuthology.orchestra.run.smithi113.stdout:ecryptfs- 94 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.322 INFO:teuthology.orchestra.run.smithi113.stdout:ecryptfs- 94 root txt unknown /proc/94/exe 2024-03-04T08:30:35.322 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 95 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.322 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 95 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.322 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 95 root txt unknown /proc/95/exe 2024-03-04T08:30:35.322 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 96 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.322 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 96 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.322 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 96 root txt unknown /proc/96/exe 2024-03-04T08:30:35.323 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 100 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.323 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 100 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.323 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 100 root txt unknown /proc/100/exe 2024-03-04T08:30:35.323 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 101 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.323 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 101 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.323 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 101 root txt unknown /proc/101/exe 2024-03-04T08:30:35.323 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 105 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.323 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 105 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.324 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 105 root txt unknown /proc/105/exe 2024-03-04T08:30:35.324 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 136 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.324 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 136 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.324 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 136 root txt unknown /proc/136/exe 2024-03-04T08:30:35.324 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 140 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.324 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 140 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.324 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 140 root txt unknown /proc/140/exe 2024-03-04T08:30:35.324 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 141 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.324 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 141 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.325 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 141 root txt unknown /proc/141/exe 2024-03-04T08:30:35.325 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 153 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.325 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 153 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.325 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 153 root txt unknown /proc/153/exe 2024-03-04T08:30:35.325 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 155 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.325 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 155 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.325 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 155 root txt unknown /proc/155/exe 2024-03-04T08:30:35.325 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 156 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.325 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 156 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.326 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 156 root txt unknown /proc/156/exe 2024-03-04T08:30:35.326 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 157 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.326 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 157 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.326 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 157 root txt unknown /proc/157/exe 2024-03-04T08:30:35.326 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 167 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.326 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 167 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.326 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 167 root txt unknown /proc/167/exe 2024-03-04T08:30:35.326 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 168 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.327 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 168 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.327 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 168 root txt unknown /proc/168/exe 2024-03-04T08:30:35.327 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 169 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.327 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 169 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.327 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 169 root txt unknown /proc/169/exe 2024-03-04T08:30:35.327 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_0 181 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.327 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_0 181 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.327 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_0 181 root txt unknown /proc/181/exe 2024-03-04T08:30:35.327 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 182 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.328 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 182 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.328 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 182 root txt unknown /proc/182/exe 2024-03-04T08:30:35.328 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_1 183 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.328 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_1 183 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.328 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_1 183 root txt unknown /proc/183/exe 2024-03-04T08:30:35.328 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 184 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.328 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 184 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.328 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 184 root txt unknown /proc/184/exe 2024-03-04T08:30:35.329 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_2 185 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.329 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_2 185 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.329 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_2 185 root txt unknown /proc/185/exe 2024-03-04T08:30:35.329 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 186 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.329 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 186 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.329 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 186 root txt unknown /proc/186/exe 2024-03-04T08:30:35.329 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_3 187 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.329 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_3 187 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.329 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_3 187 root txt unknown /proc/187/exe 2024-03-04T08:30:35.330 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 188 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.330 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 188 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.330 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 188 root txt unknown /proc/188/exe 2024-03-04T08:30:35.330 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 189 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.330 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 189 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.330 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 189 root txt unknown /proc/189/exe 2024-03-04T08:30:35.330 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_4 214 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.330 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_4 214 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.330 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_4 214 root txt unknown /proc/214/exe 2024-03-04T08:30:35.331 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 215 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.331 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 215 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.331 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 215 root txt unknown /proc/215/exe 2024-03-04T08:30:35.331 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_5 216 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.331 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_5 216 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.331 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_5 216 root txt unknown /proc/216/exe 2024-03-04T08:30:35.331 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 217 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.331 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 217 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.332 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 217 root txt unknown /proc/217/exe 2024-03-04T08:30:35.332 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_6 218 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.332 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_6 218 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.332 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_6 218 root txt unknown /proc/218/exe 2024-03-04T08:30:35.332 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 219 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.332 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 219 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.332 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 219 root txt unknown /proc/219/exe 2024-03-04T08:30:35.332 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_7 220 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_7 220 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_7 220 root txt unknown /proc/220/exe 2024-03-04T08:30:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 221 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 221 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 221 root txt unknown /proc/221/exe 2024-03-04T08:30:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_8 222 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_8 222 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_8 222 root txt unknown /proc/222/exe 2024-03-04T08:30:35.333 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 223 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 223 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 223 root txt unknown /proc/223/exe 2024-03-04T08:30:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_9 224 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_9 224 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:scsi_eh_9 224 root txt unknown /proc/224/exe 2024-03-04T08:30:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 225 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 225 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 225 root txt unknown /proc/225/exe 2024-03-04T08:30:35.334 INFO:teuthology.orchestra.run.smithi113.stdout:jbd2/sda1 260 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:jbd2/sda1 260 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:jbd2/sda1 260 root txt unknown /proc/260/exe 2024-03-04T08:30:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 261 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 261 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 261 root txt unknown /proc/261/exe 2024-03-04T08:30:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.335 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root txt REG 8,1 162176 19789 /usr/lib/systemd/systemd-journald 2024-03-04T08:30:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 16777216 6947 /var/log/journal/5896a171b9d64a25b2db7eb12a64f597/user-1000.journal 2024-03-04T08:30:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 16777216 655414 /var/log/journal/5896a171b9d64a25b2db7eb12a64f597/system.journal 2024-03-04T08:30:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.336 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 2840280 9114 /usr/lib/systemd/libsystemd-shared-249.so 2024-03-04T08:30:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 125360 9036 /usr/lib/x86_64-linux-gnu/libseccomp.so.2.5.3 2024-03-04T08:30:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.337 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 35440 15410 /usr/lib/x86_64-linux-gnu/libip4tc.so.2.0.0 2024-03-04T08:30:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 34888 16814 /usr/lib/x86_64-linux-gnu/libacl.so.1.1.2301 2024-03-04T08:30:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root mem REG 0,23 8 449 /run/systemd/journal/kernel-seqnum 2024-03-04T08:30:35.339 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.339 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 1w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.339 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 2w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.339 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 3u unix 0xffff8aaf09bb5a00 0t0 436 /run/systemd/journal/dev-log type=DGRAM 2024-03-04T08:30:35.339 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 4u unix 0xffff8aaf09bb6180 0t0 438 /run/systemd/journal/socket type=DGRAM 2024-03-04T08:30:35.339 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 5u unix 0xffff8aaf09bb6900 0t0 440 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.339 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 6u netlink 0t0 435 AUDIT 2024-03-04T08:30:35.339 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 7w CHR 1,11 0t0 10 /dev/kmsg 2024-03-04T08:30:35.339 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.340 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 9u CHR 1,11 0t0 10 /dev/kmsg 2024-03-04T08:30:35.340 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 10u unix 0xffff8aaf7cb81e00 0t0 8245 /run/systemd/journal/io.systemd.journal type=STREAM 2024-03-04T08:30:35.340 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 11r REG 0,21 0 8246 /proc/sys/kernel/hostname 2024-03-04T08:30:35.340 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 12u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.340 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 13u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.340 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 14u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.340 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 15u unix 0xffff8aaf7cb82580 0t0 8247 type=DGRAM 2024-03-04T08:30:35.340 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 16u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.341 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 17u unix 0xffff8aaf0ad26180 0t0 4688 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.341 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 18u unix 0xffff8aaf06484380 0t0 3067 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.341 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 19u unix 0xffff8aaf0974f080 0t0 7614 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.341 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 20u unix 0xffff8aaf09f95a00 0t0 7286 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.341 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 21u unix 0xffff8aaf0372a580 0t0 8440 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.341 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 22u unix 0xffff8aaf0ad22580 0t0 8441 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.341 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 23u REG 8,1 16777216 655414 /var/log/journal/5896a171b9d64a25b2db7eb12a64f597/system.journal 2024-03-04T08:30:35.341 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 24u unix 0xffff8aaf029ead00 0t0 83620 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.341 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 25u unix 0xffff8aaf0ad20f00 0t0 4727 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.342 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 26u unix 0xffff8aaf7d7f7800 0t0 658 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.342 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 27u unix 0xffff8aaf06486900 0t0 4731 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.342 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 28u unix 0xffff8aaf09bd8f00 0t0 9592 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.342 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 29u REG 8,1 16777216 6947 /var/log/journal/5896a171b9d64a25b2db7eb12a64f597/user-1000.journal 2024-03-04T08:30:35.342 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 30u unix 0xffff8aaf029eda00 0t0 45303 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.342 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 31u unix 0xffff8aaf0ad26900 0t0 4736 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.342 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 32u unix 0xffff8aaf07bf0780 0t0 5433 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.342 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 33u unix 0xffff8aaf0974cb00 0t0 5478 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.342 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 34u unix 0xffff8aaf0372e180 0t0 4739 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.343 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 35u unix 0xffff8aaf0bf52580 0t0 5488 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.343 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 36u unix 0xffff8aaf036ae180 0t0 93622 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.343 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 37u unix 0xffff8aaf09bdd280 0t0 1434 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.343 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 38u unix 0xffff8aaf0bf54380 0t0 95695 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.343 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 39u unix 0xffff8aaf719e1680 0t0 99501 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.343 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-j 308 root 42u unix 0xffff8aaf09749680 0t0 99628 /run/systemd/journal/stdout type=STREAM 2024-03-04T08:30:35.343 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 309 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.343 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 309 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.344 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 309 root txt unknown /proc/309/exe 2024-03-04T08:30:35.344 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 310 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.344 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 310 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.344 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 310 root txt unknown /proc/310/exe 2024-03-04T08:30:35.344 INFO:teuthology.orchestra.run.smithi113.stdout:ext4lazyi 339 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.344 INFO:teuthology.orchestra.run.smithi113.stdout:ext4lazyi 339 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.344 INFO:teuthology.orchestra.run.smithi113.stdout:ext4lazyi 339 root txt unknown /proc/339/exe 2024-03-04T08:30:35.344 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 340 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.345 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 340 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.345 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 340 root txt unknown /proc/340/exe 2024-03-04T08:30:35.345 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.345 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.345 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root txt REG 8,1 1131856 13517 /usr/bin/udevadm 2024-03-04T08:30:35.345 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 11124932 33003 /usr/lib/udev/hwdb.bin 2024-03-04T08:30:35.345 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.345 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.345 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.346 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.346 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.346 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.346 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.346 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.346 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.346 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.346 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.346 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 34888 16814 /usr/lib/x86_64-linux-gnu/libacl.so.1.1.2301 2024-03-04T08:30:35.347 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.347 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.347 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.347 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.347 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.347 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 1u unix 0xffff8aaf09f95280 0t0 1218 type=STREAM 2024-03-04T08:30:35.347 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 2u unix 0xffff8aaf09f95280 0t0 1218 type=STREAM 2024-03-04T08:30:35.347 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 3u netlink 0t0 445 KOBJECT_UEVENT 2024-03-04T08:30:35.347 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 4u unix 0xffff8aaf09bb7080 0t0 443 /run/udev/control type=SEQPACKET 2024-03-04T08:30:35.348 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 5u unix 0xffff8aaf09f96180 0t0 1227 type=DGRAM 2024-03-04T08:30:35.348 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 6u unix 0xffff8aaf09f96900 0t0 1231 type=DGRAM 2024-03-04T08:30:35.348 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 7u unix 0xffff8aaf09f97080 0t0 1232 type=DGRAM 2024-03-04T08:30:35.348 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 8r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.348 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 9u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.348 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 10u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.348 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 11u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.348 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 12r REG 8,1 11124932 33003 /usr/lib/udev/hwdb.bin 2024-03-04T08:30:35.349 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-u 379 root 13u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.349 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 431 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.349 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 431 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.349 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 431 root txt unknown /proc/431/exe 2024-03-04T08:30:35.349 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 446 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.349 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 446 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.349 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 446 root txt unknown /proc/446/exe 2024-03-04T08:30:35.349 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.349 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.350 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc txt REG 8,1 63912 4337 /usr/sbin/rpcbind 2024-03-04T08:30:35.350 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.350 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.350 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.350 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.350 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.350 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.350 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.351 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:35.351 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.351 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.351 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.351 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.351 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.351 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.351 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.351 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 44872 1165 /usr/lib/x86_64-linux-gnu/libwrap.so.0.7.6 2024-03-04T08:30:35.352 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.352 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:35.352 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.352 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.352 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc 1u unix 0xffff8aaf0ad23480 0t0 602 type=STREAM 2024-03-04T08:30:35.352 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc 2u unix 0xffff8aaf0ad23480 0t0 602 type=STREAM 2024-03-04T08:30:35.352 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc 3u unix 0xffff8aaf09bb4380 0t0 415 /run/rpcbind.sock type=STREAM 2024-03-04T08:30:35.353 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc 4u IPv4 4452 0t0 TCP *:sunrpc (LISTEN) 2024-03-04T08:30:35.353 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc 5u IPv4 5231 0t0 UDP *:sunrpc 2024-03-04T08:30:35.353 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc 6u IPv6 6411 0t0 TCP *:sunrpc (LISTEN) 2024-03-04T08:30:35.353 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc 7u IPv6 4453 0t0 UDP *:sunrpc 2024-03-04T08:30:35.353 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc 8rW REG 0,23 0 1146 /run/rpcbind.lock 2024-03-04T08:30:35.353 INFO:teuthology.orchestra.run.smithi113.stdout:rpcbind 483 _rpc 9u unix 0xffff8aaf05ac9680 0t0 7611 /run/rpcbind.sock type=STREAM 2024-03-04T08:30:35.353 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.353 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.353 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network txt REG 8,1 1513936 19795 /usr/lib/systemd/systemd-networkd 2024-03-04T08:30:35.354 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.354 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.354 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.354 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.354 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.354 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.354 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.354 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.354 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 125360 9036 /usr/lib/x86_64-linux-gnu/libseccomp.so.2.5.3 2024-03-04T08:30:35.355 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.355 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.355 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.355 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.355 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 2840280 9114 /usr/lib/systemd/libsystemd-shared-249.so 2024-03-04T08:30:35.355 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.355 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 35440 15410 /usr/lib/x86_64-linux-gnu/libip4tc.so.2.0.0 2024-03-04T08:30:35.355 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.355 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.356 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.356 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.356 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 34888 16814 /usr/lib/x86_64-linux-gnu/libacl.so.1.1.2301 2024-03-04T08:30:35.356 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.356 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.356 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 1u unix 0xffff8aaf06483c00 0t0 7409 type=STREAM 2024-03-04T08:30:35.356 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 2u unix 0xffff8aaf06483c00 0t0 7409 type=STREAM 2024-03-04T08:30:35.356 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 3u netlink 0t0 442 ROUTE 2024-03-04T08:30:35.357 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 4u unix 0xffff8aaf06482580 0t0 7423 type=DGRAM 2024-03-04T08:30:35.357 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.357 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 6u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.357 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 7u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.357 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 8u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.357 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 9u netlink 0t0 7427 GENERIC 2024-03-04T08:30:35.357 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 10u netlink 0t0 7428 KOBJECT_UEVENT 2024-03-04T08:30:35.357 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 11u unix 0xffff8aaf06481680 0t0 7429 type=DGRAM 2024-03-04T08:30:35.358 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 12u unix 0xffff8aaf06480f00 0t0 7430 type=DGRAM 2024-03-04T08:30:35.358 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 13u unix 0xffff8aaf06480780 0t0 7431 type=DGRAM 2024-03-04T08:30:35.358 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 14u unix 0xffff8aaf06480000 0t0 7432 type=DGRAM 2024-03-04T08:30:35.358 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 15u IPv4 9260 0t0 UDP smithi113.front.sepia.ceph.com:bootpc 2024-03-04T08:30:35.358 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 16u sock 0,8 0t0 3068 protocol: UDP 2024-03-04T08:30:35.358 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 17u pack 7435 0t0 35020 type=SOCK_RAW 2024-03-04T08:30:35.358 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 18u unix 0xffff8aaf06481e00 0t0 7436 type=STREAM 2024-03-04T08:30:35.358 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 19u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.359 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-n 492 systemd-network 20u raw6 0t0 9259 00000000000000000000000000000000:003A->00000000000000000000000000000000:0000 st=07 2024-03-04T08:30:35.359 INFO:teuthology.orchestra.run.smithi113.stdout:kipmi0 500 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.359 INFO:teuthology.orchestra.run.smithi113.stdout:kipmi0 500 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.359 INFO:teuthology.orchestra.run.smithi113.stdout:kipmi0 500 root txt unknown /proc/500/exe 2024-03-04T08:30:35.359 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.359 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.359 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root txt REG 8,1 39224 2570 /usr/sbin/blkmapd 2024-03-04T08:30:35.359 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.359 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.360 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.360 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.360 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.360 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.360 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.360 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.360 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.360 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.361 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root 3wW REG 0,23 4 1190 /run/blkmapd.pid 2024-03-04T08:30:35.361 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root 4r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.361 INFO:teuthology.orchestra.run.smithi113.stdout:blkmapd 509 root 5u unix 0xffff8aaf06486180 0t0 7442 type=DGRAM 2024-03-04T08:30:35.361 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.361 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.361 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root txt REG 8,1 47592 3135 /usr/sbin/rpc.idmapd 2024-03-04T08:30:35.361 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.361 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root mem REG 8,1 211136 486 /usr/lib/x86_64-linux-gnu/libevent_core-2.1.so.7.0.1 2024-03-04T08:30:35.362 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root mem REG 8,1 39584 4155 /usr/lib/x86_64-linux-gnu/libnfsidmap.so.1.0.0 2024-03-04T08:30:35.362 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root mem REG 8,1 35480 48372 /usr/lib/x86_64-linux-gnu/libnfsidmap/nsswitch.so 2024-03-04T08:30:35.362 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.362 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.362 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.362 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.362 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root 4u unix 0xffff8aaf06484b00 0t0 7455 type=DGRAM 2024-03-04T08:30:35.362 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.362 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root 6r FIFO 0,13 0t0 7459 pipe 2024-03-04T08:30:35.363 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root 7w FIFO 0,13 0t0 7459 pipe 2024-03-04T08:30:35.363 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root 8u REG 0,21 0 4026532452 /proc/516/net/rpc/nfs4.nametoid/channel 2024-03-04T08:30:35.363 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root 9u REG 0,21 0 4026532448 /proc/516/net/rpc/nfs4.idtoname/channel 2024-03-04T08:30:35.363 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.idmap 516 root 10r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.363 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.363 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.363 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root txt REG 8,1 55864 3888 /usr/sbin/nfsdcld 2024-03-04T08:30:35.363 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.364 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.364 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.364 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:35.364 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root mem REG 8,1 211136 486 /usr/lib/x86_64-linux-gnu/libevent_core-2.1.so.7.0.1 2024-03-04T08:30:35.364 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.364 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.364 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.365 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.365 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.365 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root 4r FIFO 0,13 0t0 624 pipe 2024-03-04T08:30:35.365 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root 5w FIFO 0,13 0t0 624 pipe 2024-03-04T08:30:35.365 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root 6u REG 8,1 32768 139416 /var/lib/nfs/nfsdcld/main.sqlite 2024-03-04T08:30:35.365 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.365 INFO:teuthology.orchestra.run.smithi113.stdout:nfsdcld 518 root 8u FIFO 0,35 0t0 113 /run/rpc_pipefs/nfsd/cld 2024-03-04T08:30:35.365 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.366 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.366 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root txt REG 8,1 453264 21677 /usr/sbin/dhclient 2024-03-04T08:30:35.366 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.366 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.366 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root mem REG 8,1 2269568 2084 /usr/lib/x86_64-linux-gnu/libdns-export.so.1110.0.2 2024-03-04T08:30:35.366 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root mem REG 8,1 475768 12518 /usr/lib/x86_64-linux-gnu/libisc-export.so.1105.0.2 2024-03-04T08:30:35.366 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.366 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.366 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.367 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.367 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root 3u unix 0xffff8aaf7cb85a00 0t0 8429 type=DGRAM 2024-03-04T08:30:35.367 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root 4r FIFO 0,13 0t0 9238 pipe 2024-03-04T08:30:35.367 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root 6w FIFO 0,13 0t0 9238 pipe 2024-03-04T08:30:35.367 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root 7w REG 8,1 1120 460 /var/lib/dhcp/dhclient.enp3s0f1.leases 2024-03-04T08:30:35.367 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root 8u pack 9257 0t0 ALL type=SOCK_RAW 2024-03-04T08:30:35.367 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 root 9u IPv4 9258 0t0 UDP *:bootpc 2024-03-04T08:30:35.367 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.368 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.368 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root txt REG 8,1 453264 21677 /usr/sbin/dhclient 2024-03-04T08:30:35.368 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.368 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.368 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root mem REG 8,1 2269568 2084 /usr/lib/x86_64-linux-gnu/libdns-export.so.1110.0.2 2024-03-04T08:30:35.368 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root mem REG 8,1 475768 12518 /usr/lib/x86_64-linux-gnu/libisc-export.so.1105.0.2 2024-03-04T08:30:35.368 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.368 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.368 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.369 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.369 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root 3u unix 0xffff8aaf7cb85a00 0t0 8429 type=DGRAM 2024-03-04T08:30:35.369 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root 4r FIFO 0,13 0t0 9238 pipe 2024-03-04T08:30:35.369 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root 6w FIFO 0,13 0t0 9238 pipe 2024-03-04T08:30:35.369 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root 7w REG 8,1 1120 460 /var/lib/dhcp/dhclient.enp3s0f1.leases 2024-03-04T08:30:35.369 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root 8u pack 9257 0t0 ALL type=SOCK_RAW 2024-03-04T08:30:35.369 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 521 isc-worke root 9u IPv4 9258 0t0 UDP *:bootpc 2024-03-04T08:30:35.369 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.369 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.370 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root txt REG 8,1 453264 21677 /usr/sbin/dhclient 2024-03-04T08:30:35.370 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.370 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.370 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root mem REG 8,1 2269568 2084 /usr/lib/x86_64-linux-gnu/libdns-export.so.1110.0.2 2024-03-04T08:30:35.370 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root mem REG 8,1 475768 12518 /usr/lib/x86_64-linux-gnu/libisc-export.so.1105.0.2 2024-03-04T08:30:35.370 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.370 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.370 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.371 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.371 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root 3u unix 0xffff8aaf7cb85a00 0t0 8429 type=DGRAM 2024-03-04T08:30:35.371 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root 4r FIFO 0,13 0t0 9238 pipe 2024-03-04T08:30:35.371 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root 6w FIFO 0,13 0t0 9238 pipe 2024-03-04T08:30:35.371 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root 7w REG 8,1 1120 460 /var/lib/dhcp/dhclient.enp3s0f1.leases 2024-03-04T08:30:35.371 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root 8u pack 9257 0t0 ALL type=SOCK_RAW 2024-03-04T08:30:35.371 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 522 isc-socke root 9u IPv4 9258 0t0 UDP *:bootpc 2024-03-04T08:30:35.371 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.371 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.372 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root txt REG 8,1 453264 21677 /usr/sbin/dhclient 2024-03-04T08:30:35.372 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.372 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.372 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root mem REG 8,1 2269568 2084 /usr/lib/x86_64-linux-gnu/libdns-export.so.1110.0.2 2024-03-04T08:30:35.372 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root mem REG 8,1 475768 12518 /usr/lib/x86_64-linux-gnu/libisc-export.so.1105.0.2 2024-03-04T08:30:35.372 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.372 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.372 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.372 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.373 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root 3u unix 0xffff8aaf7cb85a00 0t0 8429 type=DGRAM 2024-03-04T08:30:35.373 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root 4r FIFO 0,13 0t0 9238 pipe 2024-03-04T08:30:35.373 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root 6w FIFO 0,13 0t0 9238 pipe 2024-03-04T08:30:35.373 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root 7w REG 8,1 1120 460 /var/lib/dhcp/dhclient.enp3s0f1.leases 2024-03-04T08:30:35.373 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root 8u pack 9257 0t0 ALL type=SOCK_RAW 2024-03-04T08:30:35.373 INFO:teuthology.orchestra.run.smithi113.stdout:dhclient 519 523 isc-timer root 9u IPv4 9258 0t0 UDP *:bootpc 2024-03-04T08:30:35.373 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.373 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.374 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root txt REG 8,1 166328 9721 /usr/libexec/accounts-daemon 2024-03-04T08:30:35.374 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.374 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.374 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.374 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.374 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.374 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.374 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.374 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.375 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.375 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.375 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.375 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.375 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.375 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.375 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.375 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.376 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.376 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.376 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.376 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.376 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.376 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.376 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.376 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.376 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.377 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.377 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root 1u unix 0xffff8aaf03729e00 0t0 6598 type=STREAM 2024-03-04T08:30:35.377 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root 2u unix 0xffff8aaf03729e00 0t0 6598 type=STREAM 2024-03-04T08:30:35.377 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.377 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.377 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root 5u sock 0,8 0t0 7464 protocol: UNIX-STREAM 2024-03-04T08:30:35.377 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root 6u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.377 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.378 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.378 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.378 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root txt REG 8,1 166328 9721 /usr/libexec/accounts-daemon 2024-03-04T08:30:35.378 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.378 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.378 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.378 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.378 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.378 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.379 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.379 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.379 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.379 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.379 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.379 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.379 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.379 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.380 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.380 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.380 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.380 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.380 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.380 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.380 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.380 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.380 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.381 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.381 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.381 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.381 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root 1u unix 0xffff8aaf03729e00 0t0 6598 type=STREAM 2024-03-04T08:30:35.381 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root 2u unix 0xffff8aaf03729e00 0t0 6598 type=STREAM 2024-03-04T08:30:35.381 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.381 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.381 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root 5u sock 0,8 0t0 7464 protocol: UNIX-STREAM 2024-03-04T08:30:35.382 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root 6u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.382 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 531 gmain root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.382 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.382 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.382 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root txt REG 8,1 166328 9721 /usr/libexec/accounts-daemon 2024-03-04T08:30:35.382 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.382 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.382 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.382 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.383 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.383 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.383 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.383 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.383 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.383 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.383 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.383 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.383 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.384 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.384 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.384 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.384 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.384 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.384 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.384 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.384 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.385 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.385 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.385 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.385 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.385 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.385 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root 1u unix 0xffff8aaf03729e00 0t0 6598 type=STREAM 2024-03-04T08:30:35.385 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root 2u unix 0xffff8aaf03729e00 0t0 6598 type=STREAM 2024-03-04T08:30:35.385 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.385 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.386 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root 5u sock 0,8 0t0 7464 protocol: UNIX-STREAM 2024-03-04T08:30:35.386 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root 6u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.386 INFO:teuthology.orchestra.run.smithi113.stdout:accounts- 520 547 gdbus root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.386 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.386 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.386 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus txt REG 8,1 232776 9094 /usr/bin/dbus-daemon 2024-03-04T08:30:35.386 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.386 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.387 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.387 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.387 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.387 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.387 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.387 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.387 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.387 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 80768 8466 /usr/lib/x86_64-linux-gnu/libapparmor.so.1.8.2 2024-03-04T08:30:35.387 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.388 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.388 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.388 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:35.388 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.388 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 313656 8637 /usr/lib/x86_64-linux-gnu/libdbus-1.so.3.19.13 2024-03-04T08:30:35.388 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.388 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.388 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 1u unix 0xffff8aaf0ad22d00 0t0 625 type=STREAM 2024-03-04T08:30:35.388 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 2u unix 0xffff8aaf0ad22d00 0t0 625 type=STREAM 2024-03-04T08:30:35.389 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 3u unix 0xffff8aaf09f92d00 0t0 1276 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.389 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 4u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.389 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 5u sock 0,8 0t0 6626 protocol: NETLINK 2024-03-04T08:30:35.389 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 6r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.389 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 7u unix 0xffff8aaf0372ad00 0t0 6627 type=STREAM 2024-03-04T08:30:35.389 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 8u unix 0xffff8aaf0372b480 0t0 6628 type=STREAM 2024-03-04T08:30:35.389 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 9u unix 0xffff8aaf06482d00 0t0 6629 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.389 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 10u unix 0xffff8aaf7d7f3480 0t0 6630 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.389 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 11u sock 0,8 0t0 6631 protocol: UNIX-STREAM 2024-03-04T08:30:35.390 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 12u unix 0xffff8aaf0ad24b00 0t0 6657 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.390 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 13u unix 0xffff8aaf7cb80f00 0t0 737 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.390 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 14u unix 0xffff8aaf0ad21e00 0t0 740 type=DGRAM 2024-03-04T08:30:35.390 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 15u unix 0xffff8aaf09748000 0t0 6677 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.390 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 16u unix 0xffff8aaf07bf3c00 0t0 792 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.390 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 17u unix 0xffff8aaf0bf55280 0t0 9372 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.390 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 18u unix 0xffff8aaf0bf57800 0t0 98893 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.390 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 19u unix 0xffff8aaf09bdcb00 0t0 946 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.391 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 20u unix 0xffff8aaf719e3480 0t0 37351 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.391 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 21u unix 0xffff8aaf719e6180 0t0 98953 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.391 INFO:teuthology.orchestra.run.smithi113.stdout:dbus-daem 526 messagebus 24u unix 0xffff8aaf0dbd1680 0t0 99105 /run/dbus/system_bus_socket type=STREAM 2024-03-04T08:30:35.391 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.391 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.391 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root txt REG 8,1 68464 2451 /usr/sbin/irqbalance 2024-03-04T08:30:35.391 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.391 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.391 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.392 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.392 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.392 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.392 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.392 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.392 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.392 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.392 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.392 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.393 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.393 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.393 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.393 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root 1u unix 0xffff8aaf0ad21680 0t0 653 type=STREAM 2024-03-04T08:30:35.393 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root 2u unix 0xffff8aaf0ad21680 0t0 653 type=STREAM 2024-03-04T08:30:35.393 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.393 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.393 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 root 5u unix 0xffff8aaf0ad20780 0t0 660 /run/irqbalance/irqbalance534.sock type=STREAM 2024-03-04T08:30:35.394 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.394 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.394 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root txt REG 8,1 68464 2451 /usr/sbin/irqbalance 2024-03-04T08:30:35.394 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.394 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.394 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.394 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.394 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.395 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.395 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.395 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.395 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.395 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.395 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.395 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.395 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.395 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.396 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.396 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root 1u unix 0xffff8aaf0ad21680 0t0 653 type=STREAM 2024-03-04T08:30:35.396 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root 2u unix 0xffff8aaf0ad21680 0t0 653 type=STREAM 2024-03-04T08:30:35.396 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.396 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.396 INFO:teuthology.orchestra.run.smithi113.stdout:irqbalanc 534 537 gmain root 5u unix 0xffff8aaf0ad20780 0t0 660 /run/irqbalance/irqbalance534.sock type=STREAM 2024-03-04T08:30:35.396 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.396 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.397 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.397 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.397 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.397 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.397 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.397 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.397 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.397 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.397 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.398 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.398 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.398 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.398 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.398 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.398 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.398 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.398 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.398 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.399 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.399 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.399 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.399 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.399 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.399 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.399 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.399 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.399 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.400 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.400 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.400 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.400 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.400 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.400 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.400 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.400 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.401 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.401 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.401 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.401 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.401 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.401 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.401 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.401 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.401 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.402 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.402 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.402 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.402 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.402 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.402 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.402 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.402 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.402 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.403 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.403 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.403 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.403 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.403 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.403 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.403 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.403 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.403 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.404 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.404 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.404 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.404 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.404 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.404 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.404 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.404 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.405 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.405 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.405 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.405 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.405 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.405 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.405 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.405 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.405 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.406 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.406 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.406 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.406 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.406 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.406 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.406 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.406 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.407 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.407 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.407 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.407 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.407 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.407 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.407 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.407 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.407 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.408 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.408 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.408 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.408 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.408 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.408 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.408 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.409 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.409 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.409 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.409 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.409 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.409 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.409 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.409 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.409 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.410 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.410 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.410 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.410 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.410 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.410 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.410 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.410 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.411 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.411 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.411 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.411 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.411 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.411 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.411 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.411 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.412 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.412 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.412 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.412 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.412 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.412 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.412 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.412 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.413 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.413 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.413 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.413 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.413 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.413 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.413 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.413 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.413 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.414 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.414 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.414 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.414 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.414 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.414 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.414 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.414 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.415 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.415 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.415 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.415 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.415 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.415 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.415 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.415 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.416 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.416 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.416 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.416 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.416 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.416 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.416 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.416 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.417 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.417 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.417 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.417 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.417 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.417 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.417 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.417 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.418 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.418 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.418 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.418 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.418 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.418 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.418 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 597 HT\x20cle root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.418 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.418 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.419 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.419 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.419 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.419 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.419 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.419 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.419 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.419 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.419 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.420 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.420 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.420 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.420 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.420 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.420 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.420 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.420 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.421 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.421 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.421 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.421 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.421 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.421 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.421 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.421 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.422 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.423 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.423 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.423 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.423 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.423 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.423 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.423 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.423 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.424 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.424 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.424 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.424 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.424 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.424 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.424 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.424 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.425 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.425 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.425 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.425 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.425 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.425 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.425 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.425 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.426 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.426 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.426 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.426 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.426 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.426 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.426 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.426 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.426 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.427 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.427 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.427 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.427 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.427 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.427 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.427 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.427 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.428 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.428 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.428 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.428 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.428 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.428 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.428 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.429 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.429 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.429 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.429 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.429 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.429 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.429 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.429 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.429 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 598 lttng-ses root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.430 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.430 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.430 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.430 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.430 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.430 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.430 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.430 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.431 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.432 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.432 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.432 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.432 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.432 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.432 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.432 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.432 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.433 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.433 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.433 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.433 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.433 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.433 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.433 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.433 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.433 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.434 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.434 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.434 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.434 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.434 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.434 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.435 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.435 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.435 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.435 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.435 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.435 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.435 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.436 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.436 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.436 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.436 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.436 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.436 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.436 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.436 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.437 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.437 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.437 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.437 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.437 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.437 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.437 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.437 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.438 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.438 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.438 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.438 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.438 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.438 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.438 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.438 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.439 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.439 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.439 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.439 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.439 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.439 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.439 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.439 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.440 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.440 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.440 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.440 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.440 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.440 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.440 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.440 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.441 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.441 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.441 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.441 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.441 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.441 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.441 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 599 Health root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.441 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.442 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.442 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.442 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.442 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.442 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.442 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.442 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.442 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.443 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.443 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.443 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.443 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.443 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.443 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.443 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.443 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.444 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.444 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.444 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.444 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.444 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.444 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.444 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.444 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.445 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.445 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.445 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.445 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.445 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.445 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.445 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.445 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.446 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.446 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.446 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.446 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.446 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.446 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.446 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.446 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.447 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.447 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.447 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.447 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.447 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.447 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.447 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.447 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.448 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.448 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.448 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.448 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.448 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.448 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.448 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.448 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.449 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.449 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.449 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.449 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.449 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.449 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.449 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.449 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.450 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.450 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.450 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.450 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.450 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.450 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.450 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.450 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.451 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.451 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.451 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.451 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.451 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.451 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.451 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.451 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.451 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.452 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.452 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.452 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.452 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.452 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.452 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.452 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.452 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.453 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.453 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.453 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 600 Notificat root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.453 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.453 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.453 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.453 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.453 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.454 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.454 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.454 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.454 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.454 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.454 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.454 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.455 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.455 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.455 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.455 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.455 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.455 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.455 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.455 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.456 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.456 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.456 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.456 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.456 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.456 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.456 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.456 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.457 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.457 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.457 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.457 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.457 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.457 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.457 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.457 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.458 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.458 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.458 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.458 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.458 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.458 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.458 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.458 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.459 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.459 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.459 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.459 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.459 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.459 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.459 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.459 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.460 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.460 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.460 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.460 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.460 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.460 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.460 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.460 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.461 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.461 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.461 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.461 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.461 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.461 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.461 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.461 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.461 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.462 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.462 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.462 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.462 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.462 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.462 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.462 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.462 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.462 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.463 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.463 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.463 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.463 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.463 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.463 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.463 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.463 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.463 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.464 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.464 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.464 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.464 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.464 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.465 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 601 Action root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.465 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.465 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.465 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.465 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.465 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.465 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.465 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.465 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.466 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.466 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.466 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.466 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.466 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.466 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.466 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.466 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.466 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.467 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.467 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.467 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.467 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.467 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.467 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.467 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.467 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.467 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.468 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.468 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.468 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.468 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.468 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.468 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.468 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.468 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.469 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.469 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.469 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.469 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.469 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.469 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.469 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.469 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.469 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.470 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.470 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.470 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.470 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.470 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.470 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.470 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.470 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.471 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.471 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.471 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.471 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.471 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.471 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.471 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.471 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.472 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.472 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.472 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.472 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.472 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.472 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.472 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.472 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.473 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.473 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.473 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.473 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.473 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.473 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.473 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.474 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.474 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.474 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.474 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.474 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.474 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.474 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.474 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.474 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.475 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.475 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.475 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.475 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.475 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.475 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.475 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.475 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.476 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.476 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 602 Timer root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.476 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.476 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.476 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.476 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.476 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.476 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.477 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.477 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.477 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.477 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.477 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.477 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.477 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.477 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.477 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.478 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.478 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.478 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.478 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.478 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.478 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.478 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.478 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.479 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.479 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.479 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.479 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.479 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.479 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.480 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.480 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.480 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.480 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.480 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.480 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.480 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.480 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.481 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.481 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.481 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.481 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.481 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.481 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.481 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.481 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.482 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.482 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.482 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.482 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.482 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.482 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.482 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.482 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.483 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.483 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.483 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.483 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.483 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.483 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.483 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.483 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.483 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.484 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.484 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.484 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.484 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.484 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.484 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.484 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.485 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.485 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.485 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.485 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.485 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.485 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.485 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.485 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.486 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.486 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.486 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.486 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.486 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.486 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.486 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.486 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.486 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.487 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.487 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.487 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.487 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.487 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.487 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.487 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 603 Rotation root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.487 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.488 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.488 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.488 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.488 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.488 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.488 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.488 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.488 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.489 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.489 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.489 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.489 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.489 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.489 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.489 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.489 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.490 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.490 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.490 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.490 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.490 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.490 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.490 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.490 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.490 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.491 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.491 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.491 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.491 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.491 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.491 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.491 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.491 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.492 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.492 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.492 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.492 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.492 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.492 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.492 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.492 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.493 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.493 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.493 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.493 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.493 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.495 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.495 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.495 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.495 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.495 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.495 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.495 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.496 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.496 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.496 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.496 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.496 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.496 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.496 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.496 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.497 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.497 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.497 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.497 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.497 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.497 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.497 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.497 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.498 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.498 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.498 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.498 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.498 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.498 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.498 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.498 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.499 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.499 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.499 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.499 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.499 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.499 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.499 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.499 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.499 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.500 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.500 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.500 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.500 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.500 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.500 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 604 Client root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.500 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.501 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.501 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.501 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.501 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.501 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.501 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.501 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.501 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.502 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.502 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.502 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.502 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.502 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.502 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.502 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.502 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.503 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.503 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.503 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.503 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.503 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.503 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.503 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.503 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.504 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.504 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.504 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.504 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.504 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.504 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.504 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.504 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.505 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.505 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.505 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.505 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.505 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.505 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.505 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.505 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.506 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.506 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.506 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.506 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.506 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.506 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.506 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.506 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.506 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.507 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.507 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.507 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.507 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.507 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.507 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.507 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.507 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.508 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.508 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.508 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.508 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.508 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.508 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.508 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.508 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.509 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.509 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.509 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.509 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.509 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.509 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.509 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.509 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.510 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.510 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.510 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.510 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.510 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.510 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.510 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.510 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.510 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.511 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.511 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.511 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.511 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.511 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.511 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.511 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.511 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.512 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.512 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 605 UST\x20re root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.512 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.512 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.512 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.512 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.512 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.512 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.513 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.513 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.513 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.513 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.513 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.513 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.513 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.513 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.514 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.514 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.514 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.514 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.514 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.514 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.514 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.514 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.515 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.515 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.515 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.515 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.515 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.515 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.515 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.515 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.516 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.516 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.516 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.516 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.516 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.516 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.516 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.516 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.517 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.517 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.517 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.517 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.517 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.517 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.517 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.517 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.517 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.518 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.518 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.518 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.518 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.518 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.518 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.518 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.519 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.519 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.519 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.519 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.519 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.519 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.519 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.519 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.519 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.520 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.520 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.520 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.520 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.520 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.520 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.520 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.520 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.520 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.521 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.521 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.521 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.521 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.521 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.521 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.521 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.521 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.522 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.522 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.522 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.522 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.522 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.522 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.522 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.522 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.522 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.523 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.523 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.523 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.523 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 606 UST\x20ap root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.523 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.523 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.523 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.523 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.523 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.524 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.524 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.524 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.524 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.524 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.524 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.524 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.524 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.525 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.525 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.525 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.525 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.525 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.525 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.525 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.525 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.525 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.526 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.526 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.526 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.526 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.526 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.526 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.526 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.526 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.526 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.527 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.527 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.527 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.527 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.527 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.527 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.527 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.527 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.527 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.528 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.528 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.528 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.528 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.528 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.528 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.528 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.529 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.529 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.529 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.529 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.529 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.529 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.529 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.529 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.529 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.530 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.530 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.530 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.530 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.530 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.530 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.530 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.530 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.531 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.531 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.531 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.531 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.531 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.531 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.531 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.531 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.532 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.532 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.532 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.532 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.532 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.532 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.532 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.532 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.533 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.533 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.533 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.533 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.533 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.533 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.533 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.533 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.534 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.534 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.534 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.534 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.534 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 607 UST\x20ap root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.534 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.534 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.534 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.534 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.535 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.535 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.535 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.535 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.535 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.535 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.535 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.536 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.536 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.536 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.536 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.536 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.536 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.536 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.536 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.537 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.537 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.537 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.537 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.537 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.537 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.537 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.537 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.538 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.539 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.539 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.539 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.539 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.539 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.539 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.539 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.539 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.539 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.540 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.540 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.540 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.540 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.540 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.540 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.540 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.540 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.541 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.541 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.541 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.541 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.541 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.541 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.541 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.541 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.541 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.542 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.542 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.542 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.542 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.542 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.542 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.542 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.542 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.542 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.543 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.543 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.543 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.543 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.543 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.543 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.543 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.543 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.543 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.544 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.544 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.544 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.544 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.544 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.544 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.544 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.545 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.545 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.545 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.545 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.545 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.545 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 608 Applicati root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.545 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.545 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.546 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.546 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.546 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.546 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.546 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.546 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.546 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.546 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.546 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.547 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.547 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.547 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.547 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.547 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.547 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.547 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.547 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.548 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.548 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.548 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.548 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.548 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.548 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.548 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.548 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.549 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.549 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.549 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.549 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.549 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.549 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.549 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.549 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.550 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.550 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.550 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.550 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.550 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.550 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.550 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.550 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.551 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.551 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.551 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.551 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.551 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.551 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.551 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.551 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.552 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.552 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.552 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.552 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.552 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.552 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.552 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.552 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.552 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.553 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.553 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.553 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.553 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.553 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.553 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.553 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.553 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.554 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.554 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.554 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.554 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.554 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.554 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.554 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.554 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.555 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.555 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.555 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.555 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.555 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.555 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.555 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.555 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.556 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.556 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.556 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.556 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.556 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.556 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.556 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.556 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.557 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 609 Agent\x20 root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.557 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.557 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.557 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.557 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.557 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.557 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 312782 529372 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.symbols.bin 2024-03-04T08:30:35.557 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.558 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.558 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 625640 529361 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.alias.bin 2024-03-04T08:30:35.558 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 233213 529367 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.dep.bin 2024-03-04T08:30:35.558 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.558 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.558 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.558 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.558 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.559 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.559 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.559 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.559 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.559 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.559 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.559 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.559 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.560 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.560 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.560 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 9916 529364 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.bin 2024-03-04T08:30:35.560 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 23498 529363 /usr/lib/modules/6.8.0-rc6-g786939134d7e/modules.builtin.alias.bin 2024-03-04T08:30:35.560 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.560 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root mem REG 0,24 4096 2 /dev/shm/lttng-ust-wait-8 2024-03-04T08:30:35.560 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.560 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 3wW REG 0,23 0 1252 /run/lttng/lttng-sessiond.lck 2024-03-04T08:30:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 4u unix 0xffff8aaf7cb87080 0t0 8458 type=STREAM 2024-03-04T08:30:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 5r FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 6w FIFO 0,13 0t0 8460 pipe 2024-03-04T08:30:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 7r FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 8w FIFO 0,13 0t0 8461 pipe 2024-03-04T08:30:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 9r FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.561 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 10w FIFO 0,13 0t0 8462 pipe 2024-03-04T08:30:35.562 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 11r FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.562 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 12w FIFO 0,13 0t0 8463 pipe 2024-03-04T08:30:35.562 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 13r FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.562 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 14w FIFO 0,13 0t0 8464 pipe 2024-03-04T08:30:35.562 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 15r FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.562 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 16w FIFO 0,13 0t0 8465 pipe 2024-03-04T08:30:35.562 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 17r FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.562 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 18w FIFO 0,13 0t0 8466 pipe 2024-03-04T08:30:35.563 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.563 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 20u unix 0xffff8aaf7cb86900 0t0 8469 /var/run/lttng/kconsumerd/error type=STREAM 2024-03-04T08:30:35.563 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 21u unix 0xffff8aaf7cb86180 0t0 8470 /var/run/lttng/ustconsumerd64/error type=STREAM 2024-03-04T08:30:35.563 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 22u unix 0xffff8aaf7cb84380 0t0 8471 /var/run/lttng/ustconsumerd32/error type=STREAM 2024-03-04T08:30:35.563 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 23r FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.563 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 24w FIFO 0,13 0t0 8472 pipe 2024-03-04T08:30:35.563 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 25r FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.564 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 26w FIFO 0,13 0t0 8473 pipe 2024-03-04T08:30:35.564 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 27r FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.564 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 28w FIFO 0,13 0t0 8474 pipe 2024-03-04T08:30:35.564 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 29r FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.564 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 30w FIFO 0,13 0t0 8477 pipe 2024-03-04T08:30:35.564 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 31u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.565 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 32u unix 0xffff8aaf09f94b00 0t0 1333 /var/run/lttng/sessiond-health type=STREAM 2024-03-04T08:30:35.565 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 33r FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.565 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 34w FIFO 0,13 0t0 8478 pipe 2024-03-04T08:30:35.565 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 35u unix 0xffff8aaf07bf3480 0t0 7506 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.565 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 36u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.565 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 37r FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.565 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 38w FIFO 0,13 0t0 727 pipe 2024-03-04T08:30:35.565 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 39r FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.566 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 40w FIFO 0,13 0t0 728 pipe 2024-03-04T08:30:35.566 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 41u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.566 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 42u unix 0xffff8aaf0ad20000 0t0 729 /var/run/lttng/client-lttng-sessiond type=STREAM 2024-03-04T08:30:35.566 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 43u unix 0xffff8aaf7cb83c00 0t0 8479 type=STREAM 2024-03-04T08:30:35.566 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 44u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.566 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 45u unix 0xffff8aaf7cb83480 0t0 730 /var/run/lttng/sessiond-notification type=STREAM 2024-03-04T08:30:35.566 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 46r FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.566 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 47w FIFO 0,13 0t0 8480 pipe 2024-03-04T08:30:35.566 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 48u unix 0xffff8aaf7cb82d00 0t0 8481 /var/run/lttng/lttng-ust-sock-8 type=STREAM 2024-03-04T08:30:35.567 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 49u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.567 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 50r FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.567 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 51w FIFO 0,13 0t0 8482 pipe 2024-03-04T08:30:35.567 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 52r FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.567 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 53w FIFO 0,13 0t0 8483 pipe 2024-03-04T08:30:35.567 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 54u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.567 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 55r FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.567 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 56w FIFO 0,13 0t0 8484 pipe 2024-03-04T08:30:35.567 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 57u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.568 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 58u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.568 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 59u IPv4 733 0t0 TCP localhost:5345 (LISTEN) 2024-03-04T08:30:35.568 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 60r FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.568 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 61w FIFO 0,13 0t0 8485 pipe 2024-03-04T08:30:35.568 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 62u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.568 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-ses 535 610 Kernel root 63u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.568 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.568 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.569 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:35.569 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.569 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:35.569 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.569 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.569 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.569 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.569 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.569 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.569 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.570 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.570 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 313656 8637 /usr/lib/x86_64-linux-gnu/libdbus-1.so.3.19.13 2024-03-04T08:30:35.570 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 168152 4430 /usr/lib/python3/dist-packages/_dbus_bindings.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.570 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 206984 100435 /usr/lib/x86_64-linux-gnu/girepository-1.0/GLib-2.0.typelib 2024-03-04T08:30:35.570 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.570 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.570 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.570 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.571 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.571 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.571 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.571 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.571 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 348656 25613 /usr/lib/python3/dist-packages/gi/_gi.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.571 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.571 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.571 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.572 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 137288 25527 /usr/lib/x86_64-linux-gnu/libgirepository-1.0.so.1.0.0 2024-03-04T08:30:35.572 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.572 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 23040 4497 /usr/lib/python3/dist-packages/_dbus_glib_bindings.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.572 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.572 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.572 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.572 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:35.572 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.573 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.573 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.573 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.573 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root 1u unix 0xffff8aaf06487080 0t0 7465 type=STREAM 2024-03-04T08:30:35.573 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root 2u unix 0xffff8aaf06487080 0t0 7465 type=STREAM 2024-03-04T08:30:35.573 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root 3u unix 0xffff8aaf07bf2580 0t0 7585 type=STREAM 2024-03-04T08:30:35.573 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.573 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root 5u unix 0xffff8aaf07bf5280 0t0 7587 type=STREAM 2024-03-04T08:30:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:networkd- 536 root 6u unix 0xffff8aaf07bf5a00 0t0 7588 type=STREAM 2024-03-04T08:30:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root txt REG 8,1 264712 19791 /usr/lib/systemd/systemd-logind 2024-03-04T08:30:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.574 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.575 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.575 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.575 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.575 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.575 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.575 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.575 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 2840280 9114 /usr/lib/systemd/libsystemd-shared-249.so 2024-03-04T08:30:35.576 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.576 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 125360 9036 /usr/lib/x86_64-linux-gnu/libseccomp.so.2.5.3 2024-03-04T08:30:35.576 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.576 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.576 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.576 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.576 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 35440 15410 /usr/lib/x86_64-linux-gnu/libip4tc.so.2.0.0 2024-03-04T08:30:35.576 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.577 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.577 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.577 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 34888 16814 /usr/lib/x86_64-linux-gnu/libacl.so.1.1.2301 2024-03-04T08:30:35.577 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.577 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.577 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 1u unix 0xffff8aaf0ad25280 0t0 669 type=STREAM 2024-03-04T08:30:35.577 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 2u unix 0xffff8aaf0ad25280 0t0 669 type=STREAM 2024-03-04T08:30:35.577 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 3u unix 0xffff8aaf0ad27080 0t0 691 type=DGRAM 2024-03-04T08:30:35.577 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 4u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.578 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 5u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.578 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 6u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.578 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 7r REG 0,20 4096 25594 /sys/devices/virtual/tty/tty0/active 2024-03-04T08:30:35.578 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 8u netlink 0t0 699 KOBJECT_UEVENT 2024-03-04T08:30:35.578 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 9r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.578 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 10r FIFO 0,23 0t0 1394 /run/systemd/inhibit/1.ref 2024-03-04T08:30:35.578 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 11u netlink 0t0 700 KOBJECT_UEVENT 2024-03-04T08:30:35.578 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 12u netlink 0t0 701 KOBJECT_UEVENT 2024-03-04T08:30:35.579 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 13u netlink 0t0 702 KOBJECT_UEVENT 2024-03-04T08:30:35.579 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 14u unix 0xffff8aaf0ad27800 0t0 703 type=STREAM 2024-03-04T08:30:35.579 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 15u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.579 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 16u CHR 13,65 0t0 164 /dev/input/event1 2024-03-04T08:30:35.579 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 17u CHR 13,64 0t0 163 /dev/input/event0 2024-03-04T08:30:35.579 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 18u CHR 13,66 0t0 412 /dev/input/event2 2024-03-04T08:30:35.579 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 19u CHR 4,6 0t0 25 /dev/tty6 2024-03-04T08:30:35.579 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 20r FIFO 0,23 0t0 2010 /run/systemd/sessions/5.ref 2024-03-04T08:30:35.579 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 22r FIFO 0,23 0t0 3225 /run/systemd/inhibit/9.ref 2024-03-04T08:30:35.580 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-l 561 root 23r FIFO 0,23 0t0 3526 /run/systemd/inhibit/10.ref 2024-03-04T08:30:35.580 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.580 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.580 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve txt REG 8,1 489904 19802 /usr/lib/systemd/systemd-resolved 2024-03-04T08:30:35.580 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.580 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.580 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.581 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.581 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.581 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.581 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.581 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.581 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.581 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.581 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.582 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.582 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.582 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.582 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.582 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.582 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 125360 9036 /usr/lib/x86_64-linux-gnu/libseccomp.so.2.5.3 2024-03-04T08:30:35.582 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.582 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.583 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.583 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.583 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.583 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 2840280 9114 /usr/lib/systemd/libsystemd-shared-249.so 2024-03-04T08:30:35.583 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.583 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.583 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 35440 15410 /usr/lib/x86_64-linux-gnu/libip4tc.so.2.0.0 2024-03-04T08:30:35.583 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.583 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.584 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.584 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 34888 16814 /usr/lib/x86_64-linux-gnu/libacl.so.1.1.2301 2024-03-04T08:30:35.584 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.584 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.584 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.584 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 1u unix 0xffff8aaf0372da00 0t0 6645 type=STREAM 2024-03-04T08:30:35.584 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 2u unix 0xffff8aaf0372da00 0t0 6645 type=STREAM 2024-03-04T08:30:35.584 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 3u unix 0xffff8aaf0372e900 0t0 6668 type=DGRAM 2024-03-04T08:30:35.585 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 4u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.585 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 5u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.585 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 6u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.585 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 7r REG 0,41 0 6672 /proc/sys/kernel/hostname 2024-03-04T08:30:35.585 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 8r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.585 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 9u netlink 0t0 6673 ROUTE 2024-03-04T08:30:35.585 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 10u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.585 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 12u unix 0xffff8aaf7cb81680 0t0 8487 type=STREAM 2024-03-04T08:30:35.586 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 13u IPv4 8488 0t0 UDP localhost:domain 2024-03-04T08:30:35.586 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 14u IPv4 8489 0t0 TCP localhost:domain (LISTEN) 2024-03-04T08:30:35.586 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 15u unix 0xffff8aaf7cb80780 0t0 8490 /run/systemd/resolve/io.systemd.Resolve type=STREAM 2024-03-04T08:30:35.586 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 16u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.586 INFO:teuthology.orchestra.run.smithi113.stdout:systemd-r 563 systemd-resolve 17u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.586 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.586 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.586 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root txt REG 8,1 1602304 16720 /usr/bin/lttng-sessiond 2024-03-04T08:30:35.587 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.587 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.587 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.587 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.587 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.587 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.587 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.587 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 48152 27877 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 2024-03-04T08:30:35.587 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.588 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.588 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.588 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:35.588 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 224432 16686 /usr/lib/x86_64-linux-gnu/liblttng-ust-ctl.so.5.0.0 2024-03-04T08:30:35.588 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.588 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 43288 16693 /usr/lib/x86_64-linux-gnu/liburcu-cds.so.8.0.0 2024-03-04T08:30:35.588 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 22672 16694 /usr/lib/x86_64-linux-gnu/liburcu-common.so.8.0.0 2024-03-04T08:30:35.588 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 35536 16699 /usr/lib/x86_64-linux-gnu/liburcu.so.8.0.0 2024-03-04T08:30:35.589 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 530864 3680 /usr/lib/x86_64-linux-gnu/liblttng-ctl.so.0.0.0 2024-03-04T08:30:35.589 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.589 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.589 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.589 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root 1u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.589 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root 2u unix 0xffff8aaf7d7f5280 0t0 4728 type=STREAM 2024-03-04T08:30:35.589 INFO:teuthology.orchestra.run.smithi113.stdout:lttng-run 596 root 5u unix 0xffff8aaf7cb87800 0t0 8459 type=STREAM 2024-03-04T08:30:35.589 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.589 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.590 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root txt REG 8,1 121504 11947 /usr/libexec/polkitd 2024-03-04T08:30:35.590 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.590 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.590 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.590 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.590 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.590 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.590 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.591 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.591 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.591 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.591 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.591 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.591 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.591 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.591 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.591 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.592 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:35.592 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.592 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.592 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.592 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.592 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.592 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.592 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.593 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.593 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.593 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.593 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.593 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root 4u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.593 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root 5u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.593 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root 6u unix 0xffff8aaf0ad25a00 0t0 741 type=STREAM 2024-03-04T08:30:35.593 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root 7u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.594 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root 8r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.594 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 root 9r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.594 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.594 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.594 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root txt REG 8,1 121504 11947 /usr/libexec/polkitd 2024-03-04T08:30:35.594 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.594 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.594 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.594 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.595 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.595 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.595 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.595 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.595 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.595 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.595 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.595 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.596 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.596 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.596 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.596 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.596 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:35.596 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.596 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.596 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.597 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.597 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.597 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.597 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.597 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.597 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.597 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.597 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.598 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root 4u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.598 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root 5u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.598 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root 6u unix 0xffff8aaf0ad25a00 0t0 741 type=STREAM 2024-03-04T08:30:35.598 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root 7u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.598 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root 8r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.598 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 615 gmain root 9r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.598 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.598 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.599 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root txt REG 8,1 121504 11947 /usr/libexec/polkitd 2024-03-04T08:30:35.599 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.599 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.599 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.599 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.599 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.599 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.599 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.599 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.600 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.600 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.600 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.600 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.600 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.600 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.600 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.600 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.601 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:35.601 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.601 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.601 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.601 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.601 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.601 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.601 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.602 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.602 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.602 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.602 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.602 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root 4u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.602 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root 5u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.602 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root 6u unix 0xffff8aaf0ad25a00 0t0 741 type=STREAM 2024-03-04T08:30:35.602 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root 7u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.603 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root 8r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.603 INFO:teuthology.orchestra.run.smithi113.stdout:polkitd 614 617 gdbus root 9r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.603 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.603 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.603 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root txt REG 8,1 120640 3891 /usr/sbin/rpc.mountd 2024-03-04T08:30:35.603 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.603 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.603 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.603 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.603 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.604 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.604 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.604 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.604 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:35.604 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:35.604 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:35.604 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.604 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 44872 1165 /usr/lib/x86_64-linux-gnu/libwrap.so.0.7.6 2024-03-04T08:30:35.605 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.605 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.605 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.605 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.605 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 3u REG 0,21 0 4026532419 /proc/822/net/rpc/auth.unix.ip/channel 2024-03-04T08:30:35.605 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 4u IPv4 7551 0t0 UDP *:42000 2024-03-04T08:30:35.605 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 5u IPv4 7553 0t0 TCP *:52665 (LISTEN) 2024-03-04T08:30:35.605 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 6u IPv6 7555 0t0 UDP *:41916 2024-03-04T08:30:35.606 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 7u IPv6 7557 0t0 TCP *:55747 (LISTEN) 2024-03-04T08:30:35.606 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 8u IPv4 7559 0t0 UDP *:43221 2024-03-04T08:30:35.606 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 9u IPv4 7561 0t0 TCP *:58447 (LISTEN) 2024-03-04T08:30:35.606 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 10u IPv6 7563 0t0 UDP *:48566 2024-03-04T08:30:35.606 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 11u IPv6 7565 0t0 TCP *:35703 (LISTEN) 2024-03-04T08:30:35.606 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 12u IPv4 7567 0t0 UDP *:55113 2024-03-04T08:30:35.606 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 13u IPv4 7569 0t0 TCP *:50525 (LISTEN) 2024-03-04T08:30:35.606 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 14u IPv6 7571 0t0 UDP *:33095 2024-03-04T08:30:35.606 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 15u IPv6 7573 0t0 TCP *:34881 (LISTEN) 2024-03-04T08:30:35.607 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 16u REG 0,21 0 4026532423 /proc/822/net/rpc/auth.unix.gid/channel 2024-03-04T08:30:35.607 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 17u REG 0,21 0 4026532440 /proc/822/net/rpc/nfsd.export/channel 2024-03-04T08:30:35.607 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 18u REG 0,21 0 4026532444 /proc/822/net/rpc/nfsd.fh/channel 2024-03-04T08:30:35.607 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 19r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.607 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.mount 822 root 20u unix 0xffff8aaf03728000 0t0 6705 type=DGRAM 2024-03-04T08:30:35.607 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root cwd DIR 8,1 4096 1160 /var/spool/cron 2024-03-04T08:30:35.607 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.607 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root txt REG 8,1 51792 1461 /usr/sbin/cron 2024-03-04T08:30:35.608 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.608 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.608 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.608 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.608 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.608 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.608 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.608 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.609 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.609 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root 1u unix 0xffff8aaf07bf0f00 0t0 7575 type=STREAM 2024-03-04T08:30:35.609 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root 2u unix 0xffff8aaf07bf0f00 0t0 7575 type=STREAM 2024-03-04T08:30:35.609 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root 3uW REG 0,23 4 1346 /run/crond.pid 2024-03-04T08:30:35.609 INFO:teuthology.orchestra.run.smithi113.stdout:cron 839 root 4u unix 0xffff8aaf07bf1e00 0t0 7578 type=DGRAM 2024-03-04T08:30:35.609 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.609 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.609 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios txt REG 8,1 79096 10043 /usr/sbin/nrpe 2024-03-04T08:30:35.610 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.610 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.610 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.610 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.610 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.610 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.610 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.610 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.610 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.611 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:35.611 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:35.611 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 44872 1165 /usr/lib/x86_64-linux-gnu/libwrap.so.0.7.6 2024-03-04T08:30:35.611 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios DEL REG 8,1 24692 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:35.611 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.611 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.611 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios 1w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.611 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios 2w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.611 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios 3u unix 0xffff8aaf7d7f0f00 0t0 10463 type=DGRAM 2024-03-04T08:30:35.612 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios 4u IPv4 10471 0t0 TCP *:nrpe (LISTEN) 2024-03-04T08:30:35.612 INFO:teuthology.orchestra.run.smithi113.stdout:nrpe 844 nagios 5u IPv6 10472 0t0 TCP *:nrpe (LISTEN) 2024-03-04T08:30:35.612 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd cwd DIR 8,1 4096 5393 /var/lib/nfs 2024-03-04T08:30:35.612 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.612 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd txt REG 8,1 80944 2384 /usr/sbin/rpc.statd 2024-03-04T08:30:35.612 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.612 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.612 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.613 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.613 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.613 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:35.613 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.613 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.613 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.613 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:35.613 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.614 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 44872 1165 /usr/lib/x86_64-linux-gnu/libwrap.so.0.7.6 2024-03-04T08:30:35.614 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.614 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.614 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.614 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.614 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd 4u unix 0xffff8aaf09f90000 0t0 1416 type=DGRAM 2024-03-04T08:30:35.614 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd 5u IPv4 1417 0t0 UDP localhost:603 2024-03-04T08:30:35.614 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd 6w REG 0,23 4 1355 /run/rpc.statd.pid 2024-03-04T08:30:35.615 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd 8u IPv4 1421 0t0 UDP *:41822 2024-03-04T08:30:35.615 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd 9u IPv4 1423 0t0 TCP *:37081 (LISTEN) 2024-03-04T08:30:35.615 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd 10u IPv6 1425 0t0 UDP *:50928 2024-03-04T08:30:35.615 INFO:teuthology.orchestra.run.smithi113.stdout:rpc.statd 851 statd 11u IPv6 1427 0t0 TCP *:44107 (LISTEN) 2024-03-04T08:30:35.615 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.615 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.615 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root txt REG 8,1 436848 16646 /usr/sbin/tgtd 2024-03-04T08:30:35.615 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.615 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.616 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.616 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.616 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.616 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.616 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.616 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:35.616 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:35.616 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.617 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:35.617 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:35.617 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:35.617 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.617 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.617 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root 1u unix 0xffff8aaf0974c380 0t0 798 type=STREAM 2024-03-04T08:30:35.617 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root 2u unix 0xffff8aaf0974c380 0t0 798 type=STREAM 2024-03-04T08:30:35.617 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.618 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root 4ww REG 0,23 0 1376 /run/tgtd/socket.0.lock 2024-03-04T08:30:35.618 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root 5u unix 0xffff8aaf0974d280 0t0 806 /var/run/tgtd/socket.0 type=STREAM 2024-03-04T08:30:35.618 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root 6u IPv4 9363 0t0 TCP *:iscsi-target (LISTEN) 2024-03-04T08:30:35.618 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root 7u IPv6 9364 0t0 TCP *:iscsi-target (LISTEN) 2024-03-04T08:30:35.618 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root 8u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.618 INFO:teuthology.orchestra.run.smithi113.stdout:tgtd 857 root 9u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.618 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.618 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.618 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 80080 589 /usr/lib/x86_64-linux-gnu/libxxhash.so.0.8.1 2024-03-04T08:30:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root DEL REG 8,1 855 /usr/lib/x86_64-linux-gnu/libapt-pkg.so.6.0.0 2024-03-04T08:30:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 347096 12961 /usr/lib/python3/dist-packages/apt_pkg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 206984 100435 /usr/lib/x86_64-linux-gnu/girepository-1.0/GLib-2.0.typelib 2024-03-04T08:30:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.619 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.620 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.620 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.620 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.620 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 137288 25527 /usr/lib/x86_64-linux-gnu/libgirepository-1.0.so.1.0.0 2024-03-04T08:30:35.620 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.620 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 348656 25613 /usr/lib/python3/dist-packages/gi/_gi.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.620 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.620 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.621 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.621 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.621 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.621 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.621 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.621 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.621 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 313656 8637 /usr/lib/x86_64-linux-gnu/libdbus-1.so.3.19.13 2024-03-04T08:30:35.621 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 168152 4430 /usr/lib/python3/dist-packages/_dbus_bindings.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.621 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.622 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.622 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.622 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.622 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 23040 4497 /usr/lib/python3/dist-packages/_dbus_glib_bindings.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.622 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.622 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.622 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.622 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.623 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:35.623 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.623 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.623 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.623 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.623 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root 1u unix 0xffff8aaf09bdda00 0t0 5480 type=STREAM 2024-03-04T08:30:35.623 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root 2u unix 0xffff8aaf09bdda00 0t0 5480 type=STREAM 2024-03-04T08:30:35.623 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root 3w REG 8,1 113 655466 /var/log/unattended-upgrades/unattended-upgrades-shutdown.log 2024-03-04T08:30:35.623 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root 4u unix 0xffff8aaf0bf54b00 0t0 6816 type=STREAM 2024-03-04T08:30:35.624 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root 5u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.624 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root 6u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.624 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root 7u unix 0xffff8aaf09bdbc00 0t0 5564 type=STREAM 2024-03-04T08:30:35.624 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root 8w FIFO 0,23 0t0 1394 /run/systemd/inhibit/1.ref 2024-03-04T08:30:35.624 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 root 9u unix 0xffff8aaf09bdb480 0t0 5565 type=STREAM 2024-03-04T08:30:35.624 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.624 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.624 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:35.625 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.625 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 80080 589 /usr/lib/x86_64-linux-gnu/libxxhash.so.0.8.1 2024-03-04T08:30:35.625 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.625 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:35.625 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.625 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root DEL REG 8,1 855 /usr/lib/x86_64-linux-gnu/libapt-pkg.so.6.0.0 2024-03-04T08:30:35.625 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 347096 12961 /usr/lib/python3/dist-packages/apt_pkg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.625 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 206984 100435 /usr/lib/x86_64-linux-gnu/girepository-1.0/GLib-2.0.typelib 2024-03-04T08:30:35.626 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.626 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.626 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.626 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.626 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.626 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 137288 25527 /usr/lib/x86_64-linux-gnu/libgirepository-1.0.so.1.0.0 2024-03-04T08:30:35.626 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.626 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 348656 25613 /usr/lib/python3/dist-packages/gi/_gi.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.627 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.627 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.627 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.627 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.627 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.627 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.627 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 313656 8637 /usr/lib/x86_64-linux-gnu/libdbus-1.so.3.19.13 2024-03-04T08:30:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 168152 4430 /usr/lib/python3/dist-packages/_dbus_bindings.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 23040 4497 /usr/lib/python3/dist-packages/_dbus_glib_bindings.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.628 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.629 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.629 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.629 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.629 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:35.629 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.629 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.629 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.630 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.630 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root 1u unix 0xffff8aaf09bdda00 0t0 5480 type=STREAM 2024-03-04T08:30:35.630 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root 2u unix 0xffff8aaf09bdda00 0t0 5480 type=STREAM 2024-03-04T08:30:35.630 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root 3w REG 8,1 113 655466 /var/log/unattended-upgrades/unattended-upgrades-shutdown.log 2024-03-04T08:30:35.630 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root 4u unix 0xffff8aaf0bf54b00 0t0 6816 type=STREAM 2024-03-04T08:30:35.630 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root 5u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.630 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root 6u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.630 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root 7u unix 0xffff8aaf09bdbc00 0t0 5564 type=STREAM 2024-03-04T08:30:35.631 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root 8w FIFO 0,23 0t0 1394 /run/systemd/inhibit/1.ref 2024-03-04T08:30:35.631 INFO:teuthology.orchestra.run.smithi113.stdout:unattende 860 1065 gmain root 9u unix 0xffff8aaf09bdb480 0t0 5565 type=STREAM 2024-03-04T08:30:35.631 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.631 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.631 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root txt REG 8,1 917192 24518 /usr/sbin/sshd 2024-03-04T08:30:35.631 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.631 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:35.631 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.632 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.632 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.632 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.632 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.632 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.632 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.632 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.632 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.633 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.633 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.633 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:35.633 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.633 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.633 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.633 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.633 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.634 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.634 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.634 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.634 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.634 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.634 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.634 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 44872 1165 /usr/lib/x86_64-linux-gnu/libwrap.so.0.7.6 2024-03-04T08:30:35.634 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.635 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.635 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root 1u unix 0xffff8aaf0bf51e00 0t0 6782 type=STREAM 2024-03-04T08:30:35.635 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root 2u unix 0xffff8aaf0bf51e00 0t0 6782 type=STREAM 2024-03-04T08:30:35.635 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root 3u IPv4 6791 0t0 TCP *:ssh (LISTEN) 2024-03-04T08:30:35.635 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 875 root 4u IPv6 6793 0t0 TCP *:ssh (LISTEN) 2024-03-04T08:30:35.635 INFO:teuthology.orchestra.run.smithi113.stdout:lockd 880 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.635 INFO:teuthology.orchestra.run.smithi113.stdout:lockd 880 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.635 INFO:teuthology.orchestra.run.smithi113.stdout:lockd 880 root txt unknown /proc/880/exe 2024-03-04T08:30:35.636 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.636 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.636 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.636 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.636 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.636 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.636 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.636 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.637 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.637 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.637 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.637 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.637 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.637 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.637 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.638 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.638 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.638 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.638 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.638 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.638 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.638 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.638 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.639 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.639 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.639 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.639 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.639 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.639 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.639 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.639 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.640 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.640 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.640 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.640 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.640 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.640 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.640 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.640 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.641 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.641 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.641 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.641 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.641 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1070 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.641 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.641 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.641 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.642 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.642 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.642 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.642 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.642 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.642 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.642 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.642 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.642 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.643 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.643 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.643 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.643 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.643 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.643 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.643 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.643 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.644 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.644 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1071 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.644 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.644 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.644 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.644 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.644 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.644 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.645 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.645 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.645 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.645 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.645 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.645 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.645 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.645 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.645 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.646 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.646 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.646 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.646 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.646 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.646 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.646 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1072 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.646 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.647 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.647 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.647 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.647 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.647 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.647 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.647 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.647 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.647 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.648 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.648 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.648 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.648 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.648 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.648 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.648 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.648 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.649 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.649 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.649 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.649 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1073 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.649 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.649 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.649 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.649 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.650 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.650 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.650 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.650 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.650 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.650 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.650 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.650 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.651 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.651 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.651 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.651 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.651 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.651 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.651 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.651 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.651 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.652 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1074 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.652 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.652 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.652 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.652 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.652 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.652 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.652 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.653 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.653 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.653 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.653 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.653 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.653 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.653 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.653 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.653 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.654 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.654 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.654 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.654 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.654 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.654 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1132 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.654 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.654 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.655 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.655 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.655 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.655 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.655 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.655 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.655 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.655 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.656 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.656 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.656 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.656 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.656 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.656 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.656 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.656 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.656 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.657 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.657 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.657 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1133 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.657 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.657 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.657 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.657 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.657 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.658 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.658 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.658 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.658 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.658 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.658 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.658 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.658 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.658 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.659 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.659 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.659 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.659 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.659 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.659 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.659 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.659 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1134 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.660 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.660 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.660 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.660 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.660 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.660 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.660 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.660 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.661 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.661 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.661 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.661 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.661 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.661 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.661 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.661 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.662 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.662 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.662 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.662 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.662 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.662 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1135 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.662 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.662 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.662 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.663 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.663 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.663 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.663 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.663 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.663 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.663 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.663 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.664 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.664 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.664 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.664 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.664 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.664 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.664 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.665 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.665 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.665 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.665 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1136 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.665 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.665 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.665 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.666 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.666 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.666 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.666 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.666 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.666 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.666 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.666 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.666 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.667 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.667 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.667 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.667 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.667 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.667 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.667 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.667 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.667 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.668 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1137 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.668 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.668 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.668 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.668 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.668 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.668 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.668 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.669 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.670 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.670 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.670 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.670 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.670 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1138 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.670 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.670 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.670 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.671 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.671 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.671 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.671 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.671 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.671 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.671 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.671 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.672 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.672 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.672 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.672 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.672 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.672 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.672 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.672 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.673 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.673 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.673 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 1150 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.673 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.673 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.673 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root txt REG 8,1 42870592 17011 /usr/bin/containerd 2024-03-04T08:30:35.673 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.673 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root mem-W REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.673 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.674 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.674 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 1u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.674 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 2u unix 0xffff8aaf0974e900 0t0 825 type=STREAM 2024-03-04T08:30:35.674 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 3u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.674 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 4r FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.674 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 5w FIFO 0,13 0t0 6831 pipe 2024-03-04T08:30:35.674 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 6uW REG 8,1 32768 5734 /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db 2024-03-04T08:30:35.674 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.674 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.675 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 9r FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.675 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 10w FIFO 0,13 0t0 5852 pipe 2024-03-04T08:30:35.675 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 11u unix 0xffff8aaf09bdad00 0t0 5853 /run/containerd/containerd.sock.ttrpc type=STREAM 2024-03-04T08:30:35.675 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 12u unix 0xffff8aaf05ac8000 0t0 1531 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.675 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 13u IPv4 10502 0t0 TCP localhost:40279 (LISTEN) 2024-03-04T08:30:35.675 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 14u unix 0xffff8aaf08400f00 0t0 88071 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.675 INFO:teuthology.orchestra.run.smithi113.stdout:container 983 49282 container root 15u unix 0xffff8aaf08400000 0t0 86124 /run/containerd/containerd.sock type=STREAM 2024-03-04T08:30:35.675 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1037 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.676 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1037 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.676 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1037 root txt unknown /proc/1037/exe 2024-03-04T08:30:35.676 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1038 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.676 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1038 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.676 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1038 root txt unknown /proc/1038/exe 2024-03-04T08:30:35.676 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1039 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.676 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1039 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.676 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1039 root txt unknown /proc/1039/exe 2024-03-04T08:30:35.677 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1040 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.677 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1040 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.677 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1040 root txt unknown /proc/1040/exe 2024-03-04T08:30:35.677 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1043 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.677 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1043 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.677 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 1043 root txt unknown /proc/1043/exe 2024-03-04T08:30:35.677 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1049 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.677 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1049 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.677 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1049 root txt unknown /proc/1049/exe 2024-03-04T08:30:35.678 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1050 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.678 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1050 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.678 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1050 root txt unknown /proc/1050/exe 2024-03-04T08:30:35.678 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1051 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.678 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1051 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.678 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1051 root txt unknown /proc/1051/exe 2024-03-04T08:30:35.678 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1052 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.679 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1052 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.679 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1052 root txt unknown /proc/1052/exe 2024-03-04T08:30:35.679 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1053 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.679 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1053 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.679 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1053 root txt unknown /proc/1053/exe 2024-03-04T08:30:35.679 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1054 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.679 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1054 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.680 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1054 root txt unknown /proc/1054/exe 2024-03-04T08:30:35.680 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1055 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.680 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1055 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.680 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1055 root txt unknown /proc/1055/exe 2024-03-04T08:30:35.680 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1056 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.680 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1056 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.680 INFO:teuthology.orchestra.run.smithi113.stdout:nfsd 1056 root txt unknown /proc/1056/exe 2024-03-04T08:30:35.680 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1086 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.681 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1086 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.681 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1086 root txt REG 8,1 56888 3161 /usr/sbin/agetty 2024-03-04T08:30:35.681 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1086 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.681 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1086 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.681 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1086 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.681 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1086 root 0u CHR 4,65 0t0 89 /dev/ttyS1 2024-03-04T08:30:35.681 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1086 root 1u CHR 4,65 0t0 89 /dev/ttyS1 2024-03-04T08:30:35.681 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1086 root 2u CHR 4,65 0t0 89 /dev/ttyS1 2024-03-04T08:30:35.682 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1086 root 4r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.682 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.682 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.682 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root txt REG 8,1 56888 3161 /usr/sbin/agetty 2024-03-04T08:30:35.682 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.682 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.682 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root mem REG 8,1 353616 44555 /usr/lib/locale/C.utf8/LC_CTYPE 2024-03-04T08:30:35.682 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.683 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.683 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root 0u CHR 4,1 0t0 20 /dev/tty1 2024-03-04T08:30:35.683 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root 1u CHR 4,1 0t0 20 /dev/tty1 2024-03-04T08:30:35.683 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root 2u CHR 4,1 0t0 20 /dev/tty1 2024-03-04T08:30:35.683 INFO:teuthology.orchestra.run.smithi113.stdout:agetty 1117 root 4r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.683 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.683 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.683 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root txt REG 8,1 3798008 15961 /usr/bin/perl 2024-03-04T08:30:35.683 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.684 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.684 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 35416 47035 /usr/lib/x86_64-linux-gnu/perl/5.34.0/auto/Time/HiRes/HiRes.so 2024-03-04T08:30:35.684 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 68400 46935 /usr/lib/x86_64-linux-gnu/perl/5.34.0/auto/Compress/Raw/Zlib/Zlib.so 2024-03-04T08:30:35.684 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 60360 33580 /usr/lib/x86_64-linux-gnu/perl-base/auto/List/Util/Util.so 2024-03-04T08:30:35.684 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.684 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 23064 47032 /usr/lib/x86_64-linux-gnu/perl/5.34.0/auto/Sys/Syslog/Syslog.so 2024-03-04T08:30:35.684 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 31568 33577 /usr/lib/x86_64-linux-gnu/perl-base/auto/File/Glob/Glob.so 2024-03-04T08:30:35.684 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 18992 33575 /usr/lib/x86_64-linux-gnu/perl-base/auto/Cwd/Cwd.so 2024-03-04T08:30:35.685 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 47816 33582 /usr/lib/x86_64-linux-gnu/perl-base/auto/Socket/Socket.so 2024-03-04T08:30:35.685 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 23152 33579 /usr/lib/x86_64-linux-gnu/perl-base/auto/IO/IO.so 2024-03-04T08:30:35.685 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 110424 33581 /usr/lib/x86_64-linux-gnu/perl-base/auto/POSIX/POSIX.so 2024-03-04T08:30:35.685 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.685 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.685 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 22896 33576 /usr/lib/x86_64-linux-gnu/perl-base/auto/Fcntl/Fcntl.so 2024-03-04T08:30:35.685 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.685 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.686 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root 1w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.686 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root 2w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.686 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root 3r DIR 0,21 0 1 /proc 2024-03-04T08:30:35.686 INFO:teuthology.orchestra.run.smithi113.stdout:collectl 1151 root 4w REG 8,1 3120998 655430 /var/log/collectl/smithi113-20240304-072146.raw.gz 2024-03-04T08:30:35.686 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.686 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.686 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu txt REG 8,1 1849992 19775 /usr/lib/systemd/systemd 2024-03-04T08:30:35.686 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.687 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.687 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.687 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.687 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.687 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.687 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.687 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.687 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.688 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 35440 15410 /usr/lib/x86_64-linux-gnu/libip4tc.so.2.0.0 2024-03-04T08:30:35.688 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.688 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.688 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.688 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 80768 8466 /usr/lib/x86_64-linux-gnu/libapparmor.so.1.8.2 2024-03-04T08:30:35.688 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.688 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.689 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.689 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.689 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 2840280 9114 /usr/lib/systemd/libsystemd-shared-249.so 2024-03-04T08:30:35.689 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 34888 16814 /usr/lib/x86_64-linux-gnu/libacl.so.1.1.2301 2024-03-04T08:30:35.689 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.689 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 125360 9036 /usr/lib/x86_64-linux-gnu/libseccomp.so.2.5.3 2024-03-04T08:30:35.689 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.689 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.689 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 1u unix 0xffff8aaf09bd9680 0t0 5894 type=STREAM 2024-03-04T08:30:35.690 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 2u unix 0xffff8aaf09bd9680 0t0 5894 type=STREAM 2024-03-04T08:30:35.690 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 3u unix 0xffff8aaf09bda580 0t0 5915 type=DGRAM 2024-03-04T08:30:35.690 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 4u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.690 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 5u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.690 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 6r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.690 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 7r DIR 0,26 0 5331 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-03-04T08:30:35.690 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 8u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.690 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 9u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.691 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 10r REG 0,21 0 5926 /proc/1317/mountinfo 2024-03-04T08:30:35.691 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 11r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.691 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 12u unix 0xffff8aaf09bd9e00 0t0 5942 /run/user/1000/bus type=STREAM 2024-03-04T08:30:35.691 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 13r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.691 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 14r REG 0,21 0 4026532078 /proc/swaps 2024-03-04T08:30:35.691 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 15u netlink 0t0 5927 KOBJECT_UEVENT 2024-03-04T08:30:35.691 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 16u unix 0xffff8aaf09bdf080 0t0 5933 /run/user/1000/systemd/notify type=DGRAM 2024-03-04T08:30:35.691 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 17u unix 0xffff8aaf09bdf800 0t0 5934 type=DGRAM 2024-03-04T08:30:35.692 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 18u unix 0xffff8aaf09bde180 0t0 5935 type=DGRAM 2024-03-04T08:30:35.692 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 19u unix 0xffff8aaf09bde900 0t0 5936 /run/user/1000/systemd/private type=STREAM 2024-03-04T08:30:35.692 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 20u unix 0xffff8aaf09bdc380 0t0 5938 type=STREAM 2024-03-04T08:30:35.692 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 21u a_inode 0,14 0 2070 [timerfd] 2024-03-04T08:30:35.692 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 26u unix 0xffff8aaf09bd8780 0t0 5944 /run/user/1000/gnupg/S.dirmngr type=STREAM 2024-03-04T08:30:35.692 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 27u unix 0xffff8aaf029e8000 0t0 5946 /run/user/1000/gnupg/S.gpg-agent.browser type=STREAM 2024-03-04T08:30:35.692 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 28u unix 0xffff8aaf029e8780 0t0 5948 /run/user/1000/gnupg/S.gpg-agent.extra type=STREAM 2024-03-04T08:30:35.692 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 29u unix 0xffff8aaf029e8f00 0t0 5950 /run/user/1000/gnupg/S.gpg-agent.ssh type=STREAM 2024-03-04T08:30:35.693 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 30u unix 0xffff8aaf029e9680 0t0 5952 /run/user/1000/gnupg/S.gpg-agent type=STREAM 2024-03-04T08:30:35.693 INFO:teuthology.orchestra.run.smithi113.stdout:systemd 1317 ubuntu 31u unix 0xffff8aaf029e9e00 0t0 5954 /run/user/1000/pk-debconf-socket type=STREAM 2024-03-04T08:30:35.693 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.693 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.693 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu txt REG 8,1 1849992 19775 /usr/lib/systemd/systemd 2024-03-04T08:30:35.693 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:35.693 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:35.693 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:35.694 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 14328 14474 /usr/lib/x86_64-linux-gnu/security/pam_keyinit.so 2024-03-04T08:30:35.694 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:35.694 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:35.694 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 14328 14479 /usr/lib/x86_64-linux-gnu/security/pam_loginuid.so 2024-03-04T08:30:35.694 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 26616 14492 /usr/lib/x86_64-linux-gnu/security/pam_selinux.so 2024-03-04T08:30:35.694 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.694 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.694 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.695 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.695 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.695 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.695 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.695 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.695 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:35.695 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:35.696 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:35.696 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.696 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.696 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.696 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.696 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.696 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.697 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 2840280 9114 /usr/lib/systemd/libsystemd-shared-249.so 2024-03-04T08:30:35.697 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.697 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.697 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.697 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 35440 15410 /usr/lib/x86_64-linux-gnu/libip4tc.so.2.0.0 2024-03-04T08:30:35.697 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.697 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.697 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.697 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 34888 16814 /usr/lib/x86_64-linux-gnu/libacl.so.1.1.2301 2024-03-04T08:30:35.698 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 80768 8466 /usr/lib/x86_64-linux-gnu/libapparmor.so.1.8.2 2024-03-04T08:30:35.698 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 108760 9108 /usr/lib/x86_64-linux-gnu/libkmod.so.2.3.7 2024-03-04T08:30:35.698 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.698 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.698 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.698 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.698 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 125360 9036 /usr/lib/x86_64-linux-gnu/libseccomp.so.2.5.3 2024-03-04T08:30:35.698 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:35.699 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:35.699 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.699 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.699 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu 1u unix 0xffff8aaf09bd9680 0t0 5894 type=STREAM 2024-03-04T08:30:35.699 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu 2u unix 0xffff8aaf09bd9680 0t0 5894 type=STREAM 2024-03-04T08:30:35.699 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.699 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.699 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu 6w FIFO 0,13 0t0 5897 pipe 2024-03-04T08:30:35.700 INFO:teuthology.orchestra.run.smithi113.stdout:(sd-pam) 1318 ubuntu 7u unix 0xffff8aaf09bd8000 0t0 5905 type=DGRAM 2024-03-04T08:30:35.700 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.700 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.700 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog txt REG 8,1 785600 25944 /usr/sbin/rsyslogd 2024-03-04T08:30:35.700 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.700 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 28136 25376 /usr/lib/x86_64-linux-gnu/rsyslog/imklog.so 2024-03-04T08:30:35.700 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 41080 25918 /usr/lib/x86_64-linux-gnu/rsyslog/imuxsock.so 2024-03-04T08:30:35.700 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.700 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.701 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.701 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.701 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.701 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.701 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.701 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:35.703 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.703 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 51816 12678 /usr/lib/x86_64-linux-gnu/libfastjson.so.4.3.0 2024-03-04T08:30:35.704 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 14496 10543 /usr/lib/x86_64-linux-gnu/libestr.so.0.0.0 2024-03-04T08:30:35.704 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.704 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.704 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 31704 25919 /usr/lib/x86_64-linux-gnu/rsyslog/lmnet.so 2024-03-04T08:30:35.704 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.704 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.704 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 1w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.704 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 2w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.705 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 3u unix 0xffff8aaf09bb4b00 0t0 427 /run/systemd/journal/syslog type=DGRAM 2024-03-04T08:30:35.705 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 4r CHR 1,9 0t0 9 /dev/urandom 2024-03-04T08:30:35.705 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 5r REG 0,21 0 4026532037 /proc/kmsg 2024-03-04T08:30:35.705 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 6u unix 0xffff8aaf719e4380 0t0 7757 type=DGRAM 2024-03-04T08:30:35.705 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 7w REG 8,1 1138731 115293 /var/log/syslog 2024-03-04T08:30:35.705 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 8w REG 8,1 1278492 658122 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T08:30:35.705 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 9w REG 8,1 711939 655464 /var/log/auth.log 2024-03-04T08:30:35.705 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 10w REG 8,1 579317 115294 /var/log/kern.log 2024-03-04T08:30:35.706 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 syslog 11w REG 8,1 542705 658121 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T08:30:35.706 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.706 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.706 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog txt REG 8,1 785600 25944 /usr/sbin/rsyslogd 2024-03-04T08:30:35.706 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.706 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 28136 25376 /usr/lib/x86_64-linux-gnu/rsyslog/imklog.so 2024-03-04T08:30:35.706 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 41080 25918 /usr/lib/x86_64-linux-gnu/rsyslog/imuxsock.so 2024-03-04T08:30:35.706 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.706 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.707 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.707 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.707 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.707 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.707 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.707 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:35.707 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.707 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 51816 12678 /usr/lib/x86_64-linux-gnu/libfastjson.so.4.3.0 2024-03-04T08:30:35.707 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 14496 10543 /usr/lib/x86_64-linux-gnu/libestr.so.0.0.0 2024-03-04T08:30:35.708 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.708 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.708 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 31704 25919 /usr/lib/x86_64-linux-gnu/rsyslog/lmnet.so 2024-03-04T08:30:35.708 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.708 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.708 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 1w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.708 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 2w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.709 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 3u unix 0xffff8aaf09bb4b00 0t0 427 /run/systemd/journal/syslog type=DGRAM 2024-03-04T08:30:35.709 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 4r CHR 1,9 0t0 9 /dev/urandom 2024-03-04T08:30:35.709 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 5r REG 0,21 0 4026532037 /proc/kmsg 2024-03-04T08:30:35.709 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 6u unix 0xffff8aaf719e4380 0t0 7757 type=DGRAM 2024-03-04T08:30:35.709 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 7w REG 8,1 1138731 115293 /var/log/syslog 2024-03-04T08:30:35.709 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 8w REG 8,1 1278492 658122 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T08:30:35.709 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 9w REG 8,1 711939 655464 /var/log/auth.log 2024-03-04T08:30:35.709 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 10w REG 8,1 579317 115294 /var/log/kern.log 2024-03-04T08:30:35.709 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1367 in:imuxso syslog 11w REG 8,1 542705 658121 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T08:30:35.710 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.710 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.710 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog txt REG 8,1 785600 25944 /usr/sbin/rsyslogd 2024-03-04T08:30:35.710 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.710 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 28136 25376 /usr/lib/x86_64-linux-gnu/rsyslog/imklog.so 2024-03-04T08:30:35.710 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 41080 25918 /usr/lib/x86_64-linux-gnu/rsyslog/imuxsock.so 2024-03-04T08:30:35.710 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.710 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.711 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.711 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.711 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.711 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.711 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.711 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:35.711 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.711 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 51816 12678 /usr/lib/x86_64-linux-gnu/libfastjson.so.4.3.0 2024-03-04T08:30:35.712 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 14496 10543 /usr/lib/x86_64-linux-gnu/libestr.so.0.0.0 2024-03-04T08:30:35.712 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.712 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.712 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 31704 25919 /usr/lib/x86_64-linux-gnu/rsyslog/lmnet.so 2024-03-04T08:30:35.712 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.712 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.712 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 1w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.712 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 2w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.713 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 3u unix 0xffff8aaf09bb4b00 0t0 427 /run/systemd/journal/syslog type=DGRAM 2024-03-04T08:30:35.713 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 4r CHR 1,9 0t0 9 /dev/urandom 2024-03-04T08:30:35.713 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 5r REG 0,21 0 4026532037 /proc/kmsg 2024-03-04T08:30:35.713 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 6u unix 0xffff8aaf719e4380 0t0 7757 type=DGRAM 2024-03-04T08:30:35.713 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 7w REG 8,1 1138731 115293 /var/log/syslog 2024-03-04T08:30:35.713 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 8w REG 8,1 1278492 658122 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T08:30:35.713 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 9w REG 8,1 711939 655464 /var/log/auth.log 2024-03-04T08:30:35.713 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 10w REG 8,1 579317 115294 /var/log/kern.log 2024-03-04T08:30:35.713 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1368 in:imklog syslog 11w REG 8,1 542705 658121 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T08:30:35.714 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.714 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.714 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog txt REG 8,1 785600 25944 /usr/sbin/rsyslogd 2024-03-04T08:30:35.714 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.714 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 28136 25376 /usr/lib/x86_64-linux-gnu/rsyslog/imklog.so 2024-03-04T08:30:35.714 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 41080 25918 /usr/lib/x86_64-linux-gnu/rsyslog/imuxsock.so 2024-03-04T08:30:35.714 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.714 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.714 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.715 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.715 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.715 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.715 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.715 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:35.715 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.715 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 51816 12678 /usr/lib/x86_64-linux-gnu/libfastjson.so.4.3.0 2024-03-04T08:30:35.715 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 14496 10543 /usr/lib/x86_64-linux-gnu/libestr.so.0.0.0 2024-03-04T08:30:35.716 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.716 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.716 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 31704 25919 /usr/lib/x86_64-linux-gnu/rsyslog/lmnet.so 2024-03-04T08:30:35.716 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.716 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.716 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 1w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.716 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 2w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.716 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 3u unix 0xffff8aaf09bb4b00 0t0 427 /run/systemd/journal/syslog type=DGRAM 2024-03-04T08:30:35.716 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 4r CHR 1,9 0t0 9 /dev/urandom 2024-03-04T08:30:35.717 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 5r REG 0,21 0 4026532037 /proc/kmsg 2024-03-04T08:30:35.717 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 6u unix 0xffff8aaf719e4380 0t0 7757 type=DGRAM 2024-03-04T08:30:35.717 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 7w REG 8,1 1138731 115293 /var/log/syslog 2024-03-04T08:30:35.717 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 8w REG 8,1 1278492 658122 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T08:30:35.717 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 9w REG 8,1 711939 655464 /var/log/auth.log 2024-03-04T08:30:35.717 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 10w REG 8,1 579317 115294 /var/log/kern.log 2024-03-04T08:30:35.717 INFO:teuthology.orchestra.run.smithi113.stdout:rsyslogd 1366 1369 rs:main syslog 11w REG 8,1 542705 658121 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T08:30:35.717 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.718 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.718 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root txt REG 8,1 301576 16829 /usr/libexec/packagekitd 2024-03-04T08:30:35.718 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.718 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.718 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.718 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.718 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 31440 4482 /usr/lib/x86_64-linux-gnu/libdebuginfod-0.186.so 2024-03-04T08:30:35.718 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.719 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:35.719 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.719 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.719 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.719 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.719 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.719 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.719 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.720 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.720 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.720 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.720 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.720 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.720 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.720 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.720 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:35.721 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.721 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:35.721 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:35.721 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.721 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:35.721 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 825848 1712 /usr/lib/x86_64-linux-gnu/libstemmer.so.0d.0.0 2024-03-04T08:30:35.721 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:35.721 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 117400 4466 /usr/lib/x86_64-linux-gnu/libelf-0.186.so 2024-03-04T08:30:35.721 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:35.722 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 170056 4111 /usr/lib/x86_64-linux-gnu/libxmlb.so.2.0.0 2024-03-04T08:30:35.722 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 653080 24484 /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.7.0 2024-03-04T08:30:35.722 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 700800 596 /usr/lib/x86_64-linux-gnu/libdw-0.186.so 2024-03-04T08:30:35.722 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:35.722 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 80080 589 /usr/lib/x86_64-linux-gnu/libxxhash.so.0.8.1 2024-03-04T08:30:35.722 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.722 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 531656 4130 /usr/lib/x86_64-linux-gnu/libappstream.so.0.15.2 2024-03-04T08:30:35.722 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 1371192 18416 /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0.2003.0 2024-03-04T08:30:35.723 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.723 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:35.723 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.723 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root DEL REG 8,1 855 /usr/lib/x86_64-linux-gnu/libapt-pkg.so.6.0.0 2024-03-04T08:30:35.723 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.723 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.723 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.723 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 358272 16234 /usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_aptcc.so 2024-03-04T08:30:35.723 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.724 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.724 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.724 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.724 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.724 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.724 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.724 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.724 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.725 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.725 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.725 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.725 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.725 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.725 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.725 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:35.725 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.726 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.726 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.726 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.726 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.726 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.726 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 418280 18355 /usr/lib/x86_64-linux-gnu/libpackagekit-glib2.so.18.1.3 2024-03-04T08:30:35.726 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.726 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.726 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.727 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 1u unix 0xffff8aaf029ef800 0t0 46147 type=STREAM 2024-03-04T08:30:35.727 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 2u unix 0xffff8aaf029ef800 0t0 46147 type=STREAM 2024-03-04T08:30:35.727 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 3u unix 0xffff8aaf7d7f4380 0t0 40634 type=DGRAM 2024-03-04T08:30:35.727 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.727 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 5u netlink 0t0 40637 ROUTE 2024-03-04T08:30:35.727 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 6u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.727 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.727 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 8u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.727 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 9u unix 0xffff8aaf719e5a00 0t0 42711 type=STREAM 2024-03-04T08:30:35.728 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 root 10u REG 8,1 32768 5265 /var/lib/PackageKit/transactions.db 2024-03-04T08:30:35.728 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.728 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.728 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root txt REG 8,1 301576 16829 /usr/libexec/packagekitd 2024-03-04T08:30:35.728 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.728 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.728 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.728 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.728 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 31440 4482 /usr/lib/x86_64-linux-gnu/libdebuginfod-0.186.so 2024-03-04T08:30:35.729 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.729 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:35.729 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.729 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.729 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.729 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.729 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.729 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.730 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.730 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.730 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.730 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.730 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.730 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.730 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.730 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.730 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:35.730 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.731 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:35.731 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:35.731 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.731 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:35.731 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 825848 1712 /usr/lib/x86_64-linux-gnu/libstemmer.so.0d.0.0 2024-03-04T08:30:35.731 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:35.731 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 117400 4466 /usr/lib/x86_64-linux-gnu/libelf-0.186.so 2024-03-04T08:30:35.731 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:35.732 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 170056 4111 /usr/lib/x86_64-linux-gnu/libxmlb.so.2.0.0 2024-03-04T08:30:35.732 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 653080 24484 /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.7.0 2024-03-04T08:30:35.732 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 700800 596 /usr/lib/x86_64-linux-gnu/libdw-0.186.so 2024-03-04T08:30:35.732 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:35.732 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 80080 589 /usr/lib/x86_64-linux-gnu/libxxhash.so.0.8.1 2024-03-04T08:30:35.732 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.732 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 531656 4130 /usr/lib/x86_64-linux-gnu/libappstream.so.0.15.2 2024-03-04T08:30:35.732 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 1371192 18416 /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0.2003.0 2024-03-04T08:30:35.733 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.733 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:35.733 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.733 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root DEL REG 8,1 855 /usr/lib/x86_64-linux-gnu/libapt-pkg.so.6.0.0 2024-03-04T08:30:35.733 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.733 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.733 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.733 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 358272 16234 /usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_aptcc.so 2024-03-04T08:30:35.733 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.734 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.734 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.734 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.734 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.734 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.734 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.734 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.734 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.734 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.735 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.735 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.735 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.735 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.735 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.735 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:35.735 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.735 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.736 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.736 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.736 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.736 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.736 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 418280 18355 /usr/lib/x86_64-linux-gnu/libpackagekit-glib2.so.18.1.3 2024-03-04T08:30:35.736 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.736 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.736 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.737 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 1u unix 0xffff8aaf029ef800 0t0 46147 type=STREAM 2024-03-04T08:30:35.737 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 2u unix 0xffff8aaf029ef800 0t0 46147 type=STREAM 2024-03-04T08:30:35.737 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 3u unix 0xffff8aaf7d7f4380 0t0 40634 type=DGRAM 2024-03-04T08:30:35.737 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.737 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 5u netlink 0t0 40637 ROUTE 2024-03-04T08:30:35.737 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 6u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.737 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.741 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 8u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.741 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 9u unix 0xffff8aaf719e5a00 0t0 42711 type=STREAM 2024-03-04T08:30:35.741 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5010 gmain root 10u REG 8,1 32768 5265 /var/lib/PackageKit/transactions.db 2024-03-04T08:30:35.741 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.741 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.742 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root txt REG 8,1 301576 16829 /usr/libexec/packagekitd 2024-03-04T08:30:35.742 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.742 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.742 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.742 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.742 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 31440 4482 /usr/lib/x86_64-linux-gnu/libdebuginfod-0.186.so 2024-03-04T08:30:35.742 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.742 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:35.743 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.743 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.743 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.743 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.743 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.743 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.743 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.743 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.743 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.744 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.744 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.744 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.744 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.744 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.744 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:35.744 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.744 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:35.745 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:35.745 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.745 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:35.745 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 825848 1712 /usr/lib/x86_64-linux-gnu/libstemmer.so.0d.0.0 2024-03-04T08:30:35.745 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:35.745 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 117400 4466 /usr/lib/x86_64-linux-gnu/libelf-0.186.so 2024-03-04T08:30:35.745 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:35.745 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 170056 4111 /usr/lib/x86_64-linux-gnu/libxmlb.so.2.0.0 2024-03-04T08:30:35.746 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 653080 24484 /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.7.0 2024-03-04T08:30:35.746 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 700800 596 /usr/lib/x86_64-linux-gnu/libdw-0.186.so 2024-03-04T08:30:35.746 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:35.746 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 80080 589 /usr/lib/x86_64-linux-gnu/libxxhash.so.0.8.1 2024-03-04T08:30:35.746 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.746 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 531656 4130 /usr/lib/x86_64-linux-gnu/libappstream.so.0.15.2 2024-03-04T08:30:35.746 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 1371192 18416 /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0.2003.0 2024-03-04T08:30:35.746 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.746 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:35.747 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.747 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root DEL REG 8,1 855 /usr/lib/x86_64-linux-gnu/libapt-pkg.so.6.0.0 2024-03-04T08:30:35.747 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.747 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.747 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.747 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 358272 16234 /usr/lib/x86_64-linux-gnu/packagekit-backend/libpk_backend_aptcc.so 2024-03-04T08:30:35.747 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.747 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.748 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.748 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.748 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.748 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.748 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.748 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.748 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.748 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.749 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.749 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.749 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.749 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.749 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.749 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:35.749 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.749 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.749 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.750 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.750 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.750 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.750 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 418280 18355 /usr/lib/x86_64-linux-gnu/libpackagekit-glib2.so.18.1.3 2024-03-04T08:30:35.750 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.750 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.750 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.750 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 1u unix 0xffff8aaf029ef800 0t0 46147 type=STREAM 2024-03-04T08:30:35.750 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 2u unix 0xffff8aaf029ef800 0t0 46147 type=STREAM 2024-03-04T08:30:35.751 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 3u unix 0xffff8aaf7d7f4380 0t0 40634 type=DGRAM 2024-03-04T08:30:35.751 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.751 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 5u netlink 0t0 40637 ROUTE 2024-03-04T08:30:35.751 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 6u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.751 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 7r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.751 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 8u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.751 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 9u unix 0xffff8aaf719e5a00 0t0 42711 type=STREAM 2024-03-04T08:30:35.751 INFO:teuthology.orchestra.run.smithi113.stdout:packageki 5009 5011 gdbus root 10u REG 8,1 32768 5265 /var/lib/PackageKit/transactions.db 2024-03-04T08:30:35.751 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.751 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.752 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root txt REG 8,1 601248 11124 /usr/sbin/smartd 2024-03-04T08:30:35.752 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.752 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.752 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.752 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.752 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.752 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.752 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.753 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.753 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.753 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.753 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.753 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.753 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.753 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.753 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.753 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.754 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.754 INFO:teuthology.orchestra.run.smithi113.stdout:smartd 5215 root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.754 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7006 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.754 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7006 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.754 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7006 root txt unknown /proc/7006/exe 2024-03-04T08:30:35.754 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7020 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.754 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7020 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.754 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7020 root txt unknown /proc/7020/exe 2024-03-04T08:30:35.754 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7034 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.755 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7034 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.755 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7034 root txt unknown /proc/7034/exe 2024-03-04T08:30:35.755 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7048 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.755 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7048 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.755 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7048 root txt unknown /proc/7048/exe 2024-03-04T08:30:35.755 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7062 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.755 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7062 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.755 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7062 root txt unknown /proc/7062/exe 2024-03-04T08:30:35.755 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7112 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.756 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7112 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.756 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7112 root txt unknown /proc/7112/exe 2024-03-04T08:30:35.756 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7113 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.756 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7113 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.756 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7113 root txt unknown /proc/7113/exe 2024-03-04T08:30:35.756 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7114 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.756 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7114 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.756 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7114 root txt unknown /proc/7114/exe 2024-03-04T08:30:35.757 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7115 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.757 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7115 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.757 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7115 root txt unknown /proc/7115/exe 2024-03-04T08:30:35.757 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7116 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.757 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7116 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.757 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7116 root txt unknown /proc/7116/exe 2024-03-04T08:30:35.757 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7117 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.757 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7117 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.757 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7117 root txt unknown /proc/7117/exe 2024-03-04T08:30:35.758 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7118 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.758 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7118 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.758 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7118 root txt unknown /proc/7118/exe 2024-03-04T08:30:35.758 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7119 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.758 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7119 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.758 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7119 root txt unknown /proc/7119/exe 2024-03-04T08:30:35.758 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7120 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.758 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7120 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.759 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 7120 root txt unknown /proc/7120/exe 2024-03-04T08:30:35.759 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 7121 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.759 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 7121 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.759 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 7121 root txt unknown /proc/7121/exe 2024-03-04T08:30:35.759 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.759 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.759 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.759 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.759 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.760 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.760 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.760 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.760 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.760 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.760 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.760 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.761 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.761 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.761 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.761 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.761 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.761 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.761 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.761 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.761 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.762 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.762 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.762 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.762 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.762 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.762 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.762 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.762 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.763 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.763 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.763 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.763 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.763 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.763 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.763 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.763 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.763 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.764 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.764 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.764 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.764 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.764 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.764 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.764 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.764 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.765 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.765 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.765 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.765 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.765 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.765 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.765 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.765 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.766 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.766 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.766 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.766 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.766 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.766 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.766 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.766 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.767 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.767 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.767 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.767 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.767 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.767 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.767 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.767 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.768 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.768 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.768 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.768 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.768 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.768 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.768 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.768 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.769 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.769 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.769 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.769 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.769 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.769 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.769 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.770 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.770 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.770 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.770 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.770 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16928 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.770 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.770 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.770 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.771 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.771 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.771 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.771 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.771 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.771 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.771 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.772 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.772 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.772 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.772 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.772 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.772 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.772 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.772 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.773 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.773 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.773 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.773 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.773 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.773 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.773 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.773 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.773 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.774 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.774 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.774 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.774 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.774 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.774 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.774 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.774 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.775 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.775 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.775 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.775 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.775 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.775 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.775 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.775 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.775 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.776 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16929 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.776 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.776 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.776 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.776 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.776 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.776 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.776 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.777 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.777 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.777 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.777 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.777 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.777 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.777 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.777 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.778 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.778 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.778 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.778 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.778 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.778 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.778 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.778 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.779 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.779 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.779 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.779 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.779 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.779 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.779 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.779 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.780 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.780 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.780 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.780 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.780 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.780 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.780 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.780 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.780 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.781 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.781 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.781 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.781 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.781 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16930 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.781 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.781 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.781 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.782 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.782 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.782 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.782 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.782 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.782 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.782 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.782 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.782 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.783 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.783 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.783 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.783 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.783 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.783 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.783 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.783 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.784 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.784 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.784 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.784 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.784 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.784 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.784 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.784 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.784 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.785 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.785 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.785 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.785 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.785 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.785 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.785 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.786 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.786 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.786 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.786 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.786 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.786 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.786 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.786 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.787 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16931 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.787 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.787 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.787 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.787 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.787 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.787 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.787 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.787 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.788 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.788 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.788 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.788 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.788 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.788 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.788 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.788 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.789 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.789 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.789 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.789 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.789 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.789 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.789 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.789 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.789 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.790 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.790 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.790 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.790 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.790 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.790 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.790 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.790 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.791 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.791 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.791 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.791 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.791 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.791 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.791 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.791 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.791 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.792 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.792 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.792 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16932 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.792 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.792 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.792 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.792 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.793 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.793 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.793 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.793 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.793 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.793 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.793 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.793 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.793 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.794 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.794 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.794 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.794 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.794 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.794 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.794 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.794 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.794 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.795 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.795 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.795 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.795 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.795 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.795 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.795 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.795 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.796 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.796 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.796 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.796 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.796 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.796 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.796 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.796 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.797 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.797 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.797 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.797 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.797 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.797 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.797 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16933 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.797 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.797 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.798 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.798 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.798 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.798 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.798 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.798 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.798 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.798 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.798 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.799 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.799 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.799 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.799 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.799 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.799 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.799 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.799 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.800 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.800 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.800 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.800 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.800 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.800 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.800 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.800 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.801 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.801 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.801 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.801 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.801 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.801 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.801 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.801 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.801 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.802 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.802 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.802 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.802 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.802 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.802 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.802 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.802 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.803 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16934 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.803 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.803 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.803 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.803 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.803 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.803 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.803 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.803 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.804 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.804 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.804 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.804 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.804 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.804 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.804 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.804 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.805 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.805 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.805 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.805 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.805 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.805 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.805 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.805 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.805 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.806 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.807 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.807 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.807 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.807 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.807 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.807 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.807 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.807 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.808 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.808 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.808 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.808 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16935 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.808 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.808 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.808 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.808 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.809 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.809 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.809 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.809 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.809 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.809 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.809 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.809 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.809 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.810 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.810 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.810 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.810 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.810 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.810 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.810 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.810 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.811 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.811 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.811 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.811 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.811 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.811 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.811 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.811 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.811 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.812 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.812 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.812 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.812 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.812 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.812 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.812 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.812 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.813 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.813 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.813 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.813 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.813 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.813 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.813 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16936 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.813 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.813 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.814 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.814 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.814 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.814 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.814 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.814 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.814 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.814 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.815 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.815 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.815 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.815 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.815 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.815 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.815 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.815 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.816 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.816 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.816 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.816 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.816 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.816 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.816 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.816 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.816 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.817 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.817 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.817 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.817 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.817 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.817 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.817 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.817 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.818 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.818 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.818 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.818 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.818 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.818 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.818 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.818 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.819 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.819 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16937 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.819 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.819 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.819 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.819 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.819 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.819 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.819 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.820 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.820 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.820 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.820 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.820 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.820 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.820 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.820 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.821 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.821 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.821 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.821 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.821 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.821 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.821 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.821 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.822 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.822 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.822 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.822 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.822 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.822 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.822 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.822 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.823 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.823 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.823 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.823 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.823 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.823 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.823 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.823 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.824 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.824 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.824 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.824 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.824 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.824 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16938 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.824 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.824 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.825 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root txt REG 8,1 83568864 25354 /usr/bin/dockerd 2024-03-04T08:30:35.825 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.825 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem-W REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.825 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem-W REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.825 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem-W REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.825 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem-W REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.825 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem-W REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.825 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.825 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.826 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.826 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.826 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.826 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.826 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.826 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.826 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.826 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.827 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.827 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:35.827 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.827 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem-W REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.827 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.827 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.827 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 1u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.827 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 2u unix 0xffff8aaf029ed280 0t0 87280 type=STREAM 2024-03-04T08:30:35.827 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 3u unix 0xffff8aaf09748780 0t0 83624 /var/run/docker/metrics.sock type=STREAM 2024-03-04T08:30:35.828 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 4u unix 0xffff8aaf09f92580 0t0 1278 /run/docker.sock type=STREAM 2024-03-04T08:30:35.828 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.828 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 6r FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.828 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 7w FIFO 0,13 0t0 83622 pipe 2024-03-04T08:30:35.828 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 8u unix 0xffff8aaf08406180 0t0 85222 type=STREAM 2024-03-04T08:30:35.828 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 9u unix 0xffff8aaf08403c00 0t0 85223 type=STREAM 2024-03-04T08:30:35.828 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 10uW REG 8,1 32768 5823 /var/lib/docker/volumes/metadata.db 2024-03-04T08:30:35.828 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 11r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.829 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 12u netlink 0t0 86193 ROUTE 2024-03-04T08:30:35.829 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 13u netlink 0t0 86194 XFRM 2024-03-04T08:30:35.829 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 14u netlink 0t0 86195 NETFILTER 2024-03-04T08:30:35.829 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 15u unix 0xffff8aaf05acda00 0t0 86240 /var/run/docker/libnetwork/33c9e3ac4c5f.sock type=STREAM 2024-03-04T08:30:35.829 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 16uW REG 8,1 16384 5846 /var/lib/docker/buildkit/containerdmeta.db 2024-03-04T08:30:35.829 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 17uW REG 8,1 16384 5848 /var/lib/docker/buildkit/snapshots.db 2024-03-04T08:30:35.829 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 18uW REG 8,1 16384 5849 /var/lib/docker/buildkit/metadata_v2.db 2024-03-04T08:30:35.829 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 19uW REG 8,1 32768 5851 /var/lib/docker/buildkit/cache.db 2024-03-04T08:30:35.830 INFO:teuthology.orchestra.run.smithi113.stdout:dockerd 16927 16939 dockerd root 20uW REG 8,1 16384 23370 /var/lib/docker/buildkit/history.db 2024-03-04T08:30:35.830 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.830 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.830 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root txt REG 8,1 917192 24518 /usr/sbin/sshd 2024-03-04T08:30:35.830 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.830 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:35.830 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:35.830 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 18424 14480 /usr/lib/x86_64-linux-gnu/security/pam_mail.so 2024-03-04T08:30:35.830 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:35.831 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 18424 14482 /usr/lib/x86_64-linux-gnu/security/pam_motd.so 2024-03-04T08:30:35.831 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:35.831 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:35.831 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 14328 14474 /usr/lib/x86_64-linux-gnu/security/pam_keyinit.so 2024-03-04T08:30:35.831 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 14328 14479 /usr/lib/x86_64-linux-gnu/security/pam_loginuid.so 2024-03-04T08:30:35.831 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 26616 14492 /usr/lib/x86_64-linux-gnu/security/pam_selinux.so 2024-03-04T08:30:35.831 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 14328 14484 /usr/lib/x86_64-linux-gnu/security/pam_nologin.so 2024-03-04T08:30:35.831 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:35.831 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:35.832 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.832 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:35.832 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.832 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.832 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.832 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.832 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.832 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.833 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.833 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.833 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.833 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.833 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.833 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.833 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.833 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.834 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:35.834 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.834 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.834 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.834 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.834 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.834 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.834 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.834 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.835 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 44872 1165 /usr/lib/x86_64-linux-gnu/libwrap.so.0.7.6 2024-03-04T08:30:35.835 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:35.835 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:35.835 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.835 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.835 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.835 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.835 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root 3u unix 0xffff8aaf029ee900 0t0 87668 type=DGRAM 2024-03-04T08:30:35.836 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root 4u IPv4 84786 0t0 TCP smithi113.front.sepia.ceph.com:ssh->teuthology.front.sepia.ceph.com:49234 (ESTABLISHED) 2024-03-04T08:30:35.836 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root 6u unix 0xffff8aaf029ef080 0t0 87654 type=STREAM 2024-03-04T08:30:35.836 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root 7u unix 0xffff8aaf0bf57080 0t0 84795 type=STREAM 2024-03-04T08:30:35.836 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17162 root 8w FIFO 0,23 0t0 2010 /run/systemd/sessions/5.ref 2024-03-04T08:30:35.836 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.836 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.836 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu txt REG 8,1 917192 24518 /usr/sbin/sshd 2024-03-04T08:30:35.836 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.837 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:35.837 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:35.837 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 18424 14480 /usr/lib/x86_64-linux-gnu/security/pam_mail.so 2024-03-04T08:30:35.837 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:35.837 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 18424 14482 /usr/lib/x86_64-linux-gnu/security/pam_motd.so 2024-03-04T08:30:35.837 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:35.837 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:35.837 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 14328 14474 /usr/lib/x86_64-linux-gnu/security/pam_keyinit.so 2024-03-04T08:30:35.838 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 14328 14479 /usr/lib/x86_64-linux-gnu/security/pam_loginuid.so 2024-03-04T08:30:35.838 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 26616 14492 /usr/lib/x86_64-linux-gnu/security/pam_selinux.so 2024-03-04T08:30:35.838 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 14328 14484 /usr/lib/x86_64-linux-gnu/security/pam_nologin.so 2024-03-04T08:30:35.838 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:35.838 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:35.838 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.838 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:35.838 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.839 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.839 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.839 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.839 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.839 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.839 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.839 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.839 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.839 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.840 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.840 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.840 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.840 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.840 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:35.840 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.840 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.840 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.841 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.841 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.841 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.841 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.841 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.841 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 44872 1165 /usr/lib/x86_64-linux-gnu/libwrap.so.0.7.6 2024-03-04T08:30:35.841 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:35.841 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:35.842 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.842 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 0u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.842 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 1u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.842 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 2u CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.842 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 3u unix 0xffff8aaf029ee900 0t0 87668 type=DGRAM 2024-03-04T08:30:35.842 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 4u IPv4 84786 0t0 TCP smithi113.front.sepia.ceph.com:ssh->teuthology.front.sepia.ceph.com:49234 (ESTABLISHED) 2024-03-04T08:30:35.842 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 5u unix 0xffff8aaf0bf53c00 0t0 84794 type=STREAM 2024-03-04T08:30:35.842 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 6u unix 0xffff8aaf029ef080 0t0 87654 type=STREAM 2024-03-04T08:30:35.843 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 8w FIFO 0,23 0t0 2010 /run/systemd/sessions/5.ref 2024-03-04T08:30:35.843 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 9w FIFO 0,13 0t0 104619 pipe 2024-03-04T08:30:35.843 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 10r FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:35.843 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 11w FIFO 0,13 0t0 104622 pipe 2024-03-04T08:30:35.843 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 12r FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:35.843 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 13r FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:35.843 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 14w FIFO 0,13 0t0 104663 pipe 2024-03-04T08:30:35.843 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 15r FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:35.844 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 16r FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:35.844 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 17w FIFO 0,13 0t0 104666 pipe 2024-03-04T08:30:35.844 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 18r FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:35.844 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 19r FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:35.844 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 20w FIFO 0,13 0t0 104669 pipe 2024-03-04T08:30:35.844 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 21r FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:35.844 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 22r FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:35.844 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 23w FIFO 0,13 0t0 104672 pipe 2024-03-04T08:30:35.845 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 24r FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:35.845 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 25r FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:35.845 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 26w FIFO 0,13 0t0 171256 pipe 2024-03-04T08:30:35.845 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 27r FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:35.845 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 28w FIFO 0,13 0t0 168872 pipe 2024-03-04T08:30:35.845 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 29r FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:35.845 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 30r FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:35.845 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 31w FIFO 0,13 0t0 179462 pipe 2024-03-04T08:30:35.845 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 32r FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:35.846 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 34r FIFO 0,13 0t0 186983 pipe 2024-03-04T08:30:35.846 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 35r FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:35.846 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 36r FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:35.846 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 37r FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:35.846 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 39r FIFO 0,13 0t0 186984 pipe 2024-03-04T08:30:35.846 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 40r FIFO 0,13 0t0 186986 pipe 2024-03-04T08:30:35.846 INFO:teuthology.orchestra.run.smithi113.stdout:sshd 17183 ubuntu 42r FIFO 0,13 0t0 186987 pipe 2024-03-04T08:30:35.846 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.847 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.847 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp txt REG 8,1 879688 5353 /usr/sbin/ntpd 2024-03-04T08:30:35.847 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.847 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.847 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.847 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.847 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.847 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp mem REG 8,1 140800 30041 /usr/lib/x86_64-linux-gnu/libopts.so.25.17.1 2024-03-04T08:30:35.848 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.848 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.848 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.848 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 1r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.848 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 2r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.848 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 3u unix 0xffff8aaf0bf56180 0t0 84797 type=DGRAM 2024-03-04T08:30:35.848 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 4w REG 8,1 67932 365 /var/log/ntpstats/rawstats.20240304 2024-03-04T08:30:35.848 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 5w REG 8,1 47 2455 /var/log/ntpstats/sysstats.20240304 2024-03-04T08:30:35.849 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 6w REG 8,1 6785 463 /var/log/ntpstats/peerstats.20240304 2024-03-04T08:30:35.849 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 7w REG 8,1 1066 115261 /var/log/ntpstats/loopstats.20240304 2024-03-04T08:30:35.849 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 16u IPv6 84805 0t0 UDP *:ntp 2024-03-04T08:30:35.849 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 17u IPv4 84808 0t0 UDP *:ntp 2024-03-04T08:30:35.849 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 18u IPv4 84812 0t0 UDP localhost:ntp 2024-03-04T08:30:35.849 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 19u IPv4 84814 0t0 UDP smithi113.front.sepia.ceph.com:ntp 2024-03-04T08:30:35.849 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 20u IPv6 84816 0t0 UDP localhost:ntp 2024-03-04T08:30:35.849 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 21u IPv6 84818 0t0 UDP smithi113:ntp 2024-03-04T08:30:35.849 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 22u netlink 0t0 84819 ROUTE 2024-03-04T08:30:35.850 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 23u unix 0xffff8aaf0bf53480 0t0 84820 type=STREAM 2024-03-04T08:30:35.850 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 24u unix 0xffff8aaf0bf56900 0t0 84821 type=STREAM 2024-03-04T08:30:35.850 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 25u IPv4 119886 0t0 UDP smithi113:ntp 2024-03-04T08:30:35.850 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 26u IPv6 119890 0t0 UDP smithi113:ntp 2024-03-04T08:30:35.850 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 ntp 27u IPv6 119892 0t0 UDP smithi113:ntp 2024-03-04T08:30:35.850 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.850 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.850 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp txt REG 8,1 879688 5353 /usr/sbin/ntpd 2024-03-04T08:30:35.851 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.851 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp DEL REG 8,1 24691 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.851 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.851 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.851 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.851 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp mem REG 8,1 140800 30041 /usr/lib/x86_64-linux-gnu/libopts.so.25.17.1 2024-03-04T08:30:35.851 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.851 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.852 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.852 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 1r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.852 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 2r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.852 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 3u unix 0xffff8aaf0bf56180 0t0 84797 type=DGRAM 2024-03-04T08:30:35.852 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 4w REG 8,1 67932 365 /var/log/ntpstats/rawstats.20240304 2024-03-04T08:30:35.852 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 5w REG 8,1 47 2455 /var/log/ntpstats/sysstats.20240304 2024-03-04T08:30:35.852 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 6w REG 8,1 6785 463 /var/log/ntpstats/peerstats.20240304 2024-03-04T08:30:35.852 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 7w REG 8,1 1066 115261 /var/log/ntpstats/loopstats.20240304 2024-03-04T08:30:35.852 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 16u IPv6 84805 0t0 UDP *:ntp 2024-03-04T08:30:35.853 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 17u IPv4 84808 0t0 UDP *:ntp 2024-03-04T08:30:35.853 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 18u IPv4 84812 0t0 UDP localhost:ntp 2024-03-04T08:30:35.853 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 19u IPv4 84814 0t0 UDP smithi113.front.sepia.ceph.com:ntp 2024-03-04T08:30:35.853 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 20u IPv6 84816 0t0 UDP localhost:ntp 2024-03-04T08:30:35.853 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 21u IPv6 84818 0t0 UDP smithi113:ntp 2024-03-04T08:30:35.853 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 22u netlink 0t0 84819 ROUTE 2024-03-04T08:30:35.853 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 23u unix 0xffff8aaf0bf53480 0t0 84820 type=STREAM 2024-03-04T08:30:35.853 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 24u unix 0xffff8aaf0bf56900 0t0 84821 type=STREAM 2024-03-04T08:30:35.854 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 25u IPv4 119886 0t0 UDP smithi113:ntp 2024-03-04T08:30:35.854 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 26u IPv6 119890 0t0 UDP smithi113:ntp 2024-03-04T08:30:35.854 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd 17203 17204 ntpd ntp 27u IPv6 119892 0t0 UDP smithi113:ntp 2024-03-04T08:30:35.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:35.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:35.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:35.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph 1u unix 0xffff8aaf036ab480 0t0 98376 type=STREAM 2024-03-04T08:30:35.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-cras 19937 ceph 2u unix 0xffff8aaf036ab480 0t0 98376 type=STREAM 2024-03-04T08:30:35.856 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.856 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.856 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root txt REG 8,1 3397048 676923 /usr/sbin/wpa_supplicant 2024-03-04T08:30:35.856 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.857 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.857 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.857 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.857 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.857 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.857 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.857 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.857 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 313656 8637 /usr/lib/x86_64-linux-gnu/libdbus-1.so.3.19.13 2024-03-04T08:30:35.857 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.858 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:35.858 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 43064 16927 /usr/lib/x86_64-linux-gnu/libpcsclite.so.1.0.0 2024-03-04T08:30:35.858 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.858 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:35.858 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 32256 26981 /usr/lib/x86_64-linux-gnu/libnl-genl-3.so.200.26.0 2024-03-04T08:30:35.858 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:35.858 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.858 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.859 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root 1u unix 0xffff8aaf0bf50f00 0t0 96763 type=STREAM 2024-03-04T08:30:35.859 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root 2u unix 0xffff8aaf0bf50f00 0t0 96763 type=STREAM 2024-03-04T08:30:35.859 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root 3u unix 0xffff8aaf0bf50780 0t0 96766 type=DGRAM 2024-03-04T08:30:35.859 INFO:teuthology.orchestra.run.smithi113.stdout:wpa_suppl 21605 root 4u unix 0xffff8aaf0bf51680 0t0 96767 type=STREAM 2024-03-04T08:30:35.859 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.859 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.859 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root txt REG 8,1 3255560 674882 /usr/sbin/NetworkManager 2024-03-04T08:30:35.859 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 43160 674880 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-settings-plugin-ifupdown.so 2024-03-04T08:30:35.860 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 39168 674874 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-adsl.so 2024-03-04T08:30:35.860 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 298240 674877 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-wifi.so 2024-03-04T08:30:35.860 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 313656 8637 /usr/lib/x86_64-linux-gnu/libdbus-1.so.3.19.13 2024-03-04T08:30:35.860 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.860 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 59864 74945 /usr/lib/x86_64-linux-gnu/libjansson.so.4.13.0 2024-03-04T08:30:35.860 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 27144 674692 /usr/lib/x86_64-linux-gnu/libteamdctl.so.0.1.5 2024-03-04T08:30:35.860 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 47400 674876 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-team.so 2024-03-04T08:30:35.860 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 1144904 674461 /usr/lib/x86_64-linux-gnu/libmm-glib.so.0.9.0 2024-03-04T08:30:35.861 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 100584 674881 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-wwan.so 2024-03-04T08:30:35.861 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 194000 674333 /usr/lib/x86_64-linux-gnu/libbluetooth.so.3.19.6 2024-03-04T08:30:35.861 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 35120 674878 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-wwan.so 2024-03-04T08:30:35.861 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 96544 674875 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-bluetooth.so 2024-03-04T08:30:35.861 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.861 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.861 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:35.861 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.862 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.862 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.862 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.862 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.862 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.862 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.862 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.862 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.863 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.863 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.863 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.863 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.863 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.863 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:35.863 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.863 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.864 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.864 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:35.864 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:35.864 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:35.864 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.864 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.864 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.864 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.865 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.865 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.865 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.865 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.865 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.865 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.865 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.865 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.866 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.866 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.866 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.866 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.866 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.866 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.866 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.866 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 653080 24484 /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.7.0 2024-03-04T08:30:35.866 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:35.867 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.867 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.867 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 27280 674474 /usr/lib/x86_64-linux-gnu/libndp.so.0.2.0 2024-03-04T08:30:35.867 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.867 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.867 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.867 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.868 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.868 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.868 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 1u unix 0xffff8aaf719e2d00 0t0 95765 type=STREAM 2024-03-04T08:30:35.868 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 2u unix 0xffff8aaf719e2d00 0t0 95765 type=STREAM 2024-03-04T08:30:35.868 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.868 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.868 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 5u unix 0xffff8aaf719e4b00 0t0 95770 type=DGRAM 2024-03-04T08:30:35.868 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 6u unix 0xffff8aaf719e6900 0t0 95771 type=STREAM 2024-03-04T08:30:35.869 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 7u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.869 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 8r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.869 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 9r REG 0,4 0 4026532703 mnt 2024-03-04T08:30:35.869 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 10u netlink 0t0 101507 KOBJECT_UEVENT 2024-03-04T08:30:35.869 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 11u netlink 0t0 101508 GENERIC 2024-03-04T08:30:35.869 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 12u netlink 0t0 101509 ROUTE 2024-03-04T08:30:35.869 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 13r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.869 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 14r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.870 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 15u netlink 0t0 101546 KOBJECT_UEVENT 2024-03-04T08:30:35.870 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 16u netlink 0t0 101547 KOBJECT_UEVENT 2024-03-04T08:30:35.870 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 17u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.870 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 root 19w FIFO 0,23 0t0 3225 /run/systemd/inhibit/9.ref 2024-03-04T08:30:35.870 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.870 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.870 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root txt REG 8,1 3255560 674882 /usr/sbin/NetworkManager 2024-03-04T08:30:35.870 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 43160 674880 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-settings-plugin-ifupdown.so 2024-03-04T08:30:35.870 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 39168 674874 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-adsl.so 2024-03-04T08:30:35.871 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 298240 674877 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-wifi.so 2024-03-04T08:30:35.871 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 313656 8637 /usr/lib/x86_64-linux-gnu/libdbus-1.so.3.19.13 2024-03-04T08:30:35.871 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.871 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 59864 74945 /usr/lib/x86_64-linux-gnu/libjansson.so.4.13.0 2024-03-04T08:30:35.871 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 27144 674692 /usr/lib/x86_64-linux-gnu/libteamdctl.so.0.1.5 2024-03-04T08:30:35.871 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 47400 674876 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-team.so 2024-03-04T08:30:35.871 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 1144904 674461 /usr/lib/x86_64-linux-gnu/libmm-glib.so.0.9.0 2024-03-04T08:30:35.872 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 100584 674881 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-wwan.so 2024-03-04T08:30:35.872 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 194000 674333 /usr/lib/x86_64-linux-gnu/libbluetooth.so.3.19.6 2024-03-04T08:30:35.872 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 35120 674878 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-wwan.so 2024-03-04T08:30:35.872 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 96544 674875 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-bluetooth.so 2024-03-04T08:30:35.872 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.872 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.872 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:35.873 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.873 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.873 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.873 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.873 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.873 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.873 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.873 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.874 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.874 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.874 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.874 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.874 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.874 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.874 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:35.874 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.875 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.875 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.875 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:35.875 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:35.875 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:35.875 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.875 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.875 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.876 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.876 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.876 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.876 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.876 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.876 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.876 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.876 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.877 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.877 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.877 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.877 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.877 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.877 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.877 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.877 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.877 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 653080 24484 /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.7.0 2024-03-04T08:30:35.878 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:35.878 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.878 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.878 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 27280 674474 /usr/lib/x86_64-linux-gnu/libndp.so.0.2.0 2024-03-04T08:30:35.878 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.878 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.878 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.878 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.879 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.879 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.879 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 1u unix 0xffff8aaf719e2d00 0t0 95765 type=STREAM 2024-03-04T08:30:35.879 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 2u unix 0xffff8aaf719e2d00 0t0 95765 type=STREAM 2024-03-04T08:30:35.879 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.879 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.879 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 5u unix 0xffff8aaf719e4b00 0t0 95770 type=DGRAM 2024-03-04T08:30:35.880 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 6u unix 0xffff8aaf719e6900 0t0 95771 type=STREAM 2024-03-04T08:30:35.880 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 7u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.880 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 8r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.880 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 9r REG 0,4 0 4026532703 mnt 2024-03-04T08:30:35.880 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 10u netlink 0t0 101507 KOBJECT_UEVENT 2024-03-04T08:30:35.880 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 11u netlink 0t0 101508 GENERIC 2024-03-04T08:30:35.880 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 12u netlink 0t0 101509 ROUTE 2024-03-04T08:30:35.880 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 13r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.881 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 14r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.881 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 15u netlink 0t0 101546 KOBJECT_UEVENT 2024-03-04T08:30:35.881 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 16u netlink 0t0 101547 KOBJECT_UEVENT 2024-03-04T08:30:35.881 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 17u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.881 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21788 gmain root 19w FIFO 0,23 0t0 3225 /run/systemd/inhibit/9.ref 2024-03-04T08:30:35.881 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.881 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.881 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root txt REG 8,1 3255560 674882 /usr/sbin/NetworkManager 2024-03-04T08:30:35.882 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 43160 674880 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-settings-plugin-ifupdown.so 2024-03-04T08:30:35.882 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 39168 674874 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-adsl.so 2024-03-04T08:30:35.882 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 298240 674877 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-wifi.so 2024-03-04T08:30:35.882 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 313656 8637 /usr/lib/x86_64-linux-gnu/libdbus-1.so.3.19.13 2024-03-04T08:30:35.882 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.882 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 59864 74945 /usr/lib/x86_64-linux-gnu/libjansson.so.4.13.0 2024-03-04T08:30:35.882 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 27144 674692 /usr/lib/x86_64-linux-gnu/libteamdctl.so.0.1.5 2024-03-04T08:30:35.882 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 47400 674876 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-team.so 2024-03-04T08:30:35.883 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 1144904 674461 /usr/lib/x86_64-linux-gnu/libmm-glib.so.0.9.0 2024-03-04T08:30:35.883 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 100584 674881 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-wwan.so 2024-03-04T08:30:35.883 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 194000 674333 /usr/lib/x86_64-linux-gnu/libbluetooth.so.3.19.6 2024-03-04T08:30:35.883 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 35120 674878 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-wwan.so 2024-03-04T08:30:35.883 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 96544 674875 /usr/lib/x86_64-linux-gnu/NetworkManager/1.36.6/libnm-device-plugin-bluetooth.so 2024-03-04T08:30:35.883 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.883 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.883 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:35.883 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.884 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.884 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.884 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.884 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.884 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.884 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.884 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.884 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.885 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.885 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.885 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.885 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.885 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.885 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:35.885 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.885 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.886 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.886 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:35.886 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:35.886 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:35.886 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.886 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.886 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.886 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.887 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.887 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.887 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.887 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.887 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.887 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.887 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.887 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.887 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.888 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.888 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.888 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.888 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.888 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.888 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.888 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 653080 24484 /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4.7.0 2024-03-04T08:30:35.888 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:35.889 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.889 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.889 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 27280 674474 /usr/lib/x86_64-linux-gnu/libndp.so.0.2.0 2024-03-04T08:30:35.889 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.889 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.889 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.889 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.889 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.890 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.890 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 1u unix 0xffff8aaf719e2d00 0t0 95765 type=STREAM 2024-03-04T08:30:35.890 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 2u unix 0xffff8aaf719e2d00 0t0 95765 type=STREAM 2024-03-04T08:30:35.890 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.890 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.890 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 5u unix 0xffff8aaf719e4b00 0t0 95770 type=DGRAM 2024-03-04T08:30:35.890 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 6u unix 0xffff8aaf719e6900 0t0 95771 type=STREAM 2024-03-04T08:30:35.890 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 7u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.890 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 8r REG 0,4 0 4026531840 net 2024-03-04T08:30:35.891 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 9r REG 0,4 0 4026532703 mnt 2024-03-04T08:30:35.891 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 10u netlink 0t0 101507 KOBJECT_UEVENT 2024-03-04T08:30:35.891 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 11u netlink 0t0 101508 GENERIC 2024-03-04T08:30:35.891 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 12u netlink 0t0 101509 ROUTE 2024-03-04T08:30:35.891 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 13r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.891 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 14r a_inode 0,14 0 2070 inotify 2024-03-04T08:30:35.891 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 15u netlink 0t0 101546 KOBJECT_UEVENT 2024-03-04T08:30:35.891 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 16u netlink 0t0 101547 KOBJECT_UEVENT 2024-03-04T08:30:35.892 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 17u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.892 INFO:teuthology.orchestra.run.smithi113.stdout:NetworkMa 21787 21789 gdbus root 19w FIFO 0,23 0t0 3225 /run/systemd/inhibit/9.ref 2024-03-04T08:30:35.892 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres cwd DIR 8,1 4096 786704 /var/lib/postgresql/14/main 2024-03-04T08:30:35.892 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.892 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres txt REG 8,1 9522144 675561 /usr/lib/postgresql/14/bin/postgres 2024-03-04T08:30:35.892 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres DEL REG 0,1 18 /dev/zero 2024-03-04T08:30:35.892 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.892 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.893 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.893 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.893 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.893 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.893 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.893 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.893 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.893 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.894 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.894 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.894 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.894 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.894 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.894 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.894 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.894 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.895 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.895 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.895 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.895 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.895 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.895 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.895 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.895 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 3328472 2387 /usr/lib/x86_64-linux-gnu/libicui18n.so.70.1 2024-03-04T08:30:35.895 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.896 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.896 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.896 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.896 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.896 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.896 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.896 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.896 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.897 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.897 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.897 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:35.897 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.897 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.897 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.897 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 0,24 26976 18 /dev/shm/PostgreSQL.2029366278 2024-03-04T08:30:35.897 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.898 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres DEL REG 0,1 4 /SYSV000c0110 2024-03-04T08:30:35.898 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.898 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres 1w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.898 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres 2w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.898 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres 3r FIFO 0,13 0t0 98231 pipe 2024-03-04T08:30:35.898 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres 4w FIFO 0,13 0t0 98231 pipe 2024-03-04T08:30:35.898 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres 5u IPv6 98237 0t0 TCP localhost:postgresql (LISTEN) 2024-03-04T08:30:35.898 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres 6u IPv4 98238 0t0 TCP localhost:postgresql (LISTEN) 2024-03-04T08:30:35.899 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres 7u unix 0xffff8aaf0974f800 0t0 98239 /var/run/postgresql/.s.PGSQL.5432 type=STREAM 2024-03-04T08:30:35.899 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22317 postgres 8u IPv6 98243 0t0 UDP localhost:41827->localhost:41827 2024-03-04T08:30:35.899 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres cwd DIR 8,1 4096 786704 /var/lib/postgresql/14/main 2024-03-04T08:30:35.899 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.899 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres txt REG 8,1 9522144 675561 /usr/lib/postgresql/14/bin/postgres 2024-03-04T08:30:35.899 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres DEL REG 0,1 18 /dev/zero 2024-03-04T08:30:35.899 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.899 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.899 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.900 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.900 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.900 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.900 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.900 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.900 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.900 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.900 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.901 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.901 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.901 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.901 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.901 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.901 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.901 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.901 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.902 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.902 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.902 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.902 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.902 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.902 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.902 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 3328472 2387 /usr/lib/x86_64-linux-gnu/libicui18n.so.70.1 2024-03-04T08:30:35.902 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.903 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.903 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.903 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.903 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.903 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.903 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.903 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.903 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.903 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.904 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.904 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:35.904 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.904 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.904 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.904 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 0,24 26976 18 /dev/shm/PostgreSQL.2029366278 2024-03-04T08:30:35.904 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.904 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres DEL REG 0,1 4 /SYSV000c0110 2024-03-04T08:30:35.905 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.905 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres 1w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.905 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres 2w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.905 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres 3r FIFO 0,13 0t0 98231 pipe 2024-03-04T08:30:35.905 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres 4u REG 8,1 16777216 786734 /var/lib/postgresql/14/main/pg_wal/000000010000000000000001 2024-03-04T08:30:35.905 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres 8u IPv6 98243 0t0 UDP localhost:41827->localhost:41827 2024-03-04T08:30:35.905 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres 9u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.905 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22319 postgres 10u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.906 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres cwd DIR 8,1 4096 786704 /var/lib/postgresql/14/main 2024-03-04T08:30:35.906 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.906 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres txt REG 8,1 9522144 675561 /usr/lib/postgresql/14/bin/postgres 2024-03-04T08:30:35.906 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres DEL REG 0,1 18 /dev/zero 2024-03-04T08:30:35.906 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.906 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.906 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.906 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.906 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.907 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.907 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.907 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.907 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.907 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.907 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.907 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.907 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.908 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.908 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.908 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.908 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.908 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.908 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.908 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.908 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.909 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.909 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.909 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.909 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.909 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 3328472 2387 /usr/lib/x86_64-linux-gnu/libicui18n.so.70.1 2024-03-04T08:30:35.909 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.909 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.911 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.911 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.911 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.911 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.911 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.911 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.911 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.912 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.912 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.912 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:35.912 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.912 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.912 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.912 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 0,24 26976 18 /dev/shm/PostgreSQL.2029366278 2024-03-04T08:30:35.912 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.912 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres DEL REG 0,1 4 /SYSV000c0110 2024-03-04T08:30:35.913 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.913 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres 1w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.913 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres 2w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.913 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres 3r FIFO 0,13 0t0 98231 pipe 2024-03-04T08:30:35.913 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres 8u IPv6 98243 0t0 UDP localhost:41827->localhost:41827 2024-03-04T08:30:35.913 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres 9u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.913 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22320 postgres 10u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.913 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres cwd DIR 8,1 4096 786704 /var/lib/postgresql/14/main 2024-03-04T08:30:35.914 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.914 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres txt REG 8,1 9522144 675561 /usr/lib/postgresql/14/bin/postgres 2024-03-04T08:30:35.914 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres DEL REG 0,1 18 /dev/zero 2024-03-04T08:30:35.914 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.914 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.914 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.914 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.914 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.914 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.915 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.915 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.915 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.915 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.915 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.915 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.915 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.915 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.916 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.916 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.916 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.916 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.916 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.916 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.916 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.916 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.917 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.917 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.917 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.917 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 3328472 2387 /usr/lib/x86_64-linux-gnu/libicui18n.so.70.1 2024-03-04T08:30:35.917 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.917 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.917 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.917 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.917 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.918 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.918 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.918 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.918 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.918 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.918 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.918 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:35.918 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.919 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.919 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.919 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 0,24 26976 18 /dev/shm/PostgreSQL.2029366278 2024-03-04T08:30:35.919 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.919 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres DEL REG 0,1 4 /SYSV000c0110 2024-03-04T08:30:35.919 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.919 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres 1w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.919 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres 2w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.920 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres 3r FIFO 0,13 0t0 98231 pipe 2024-03-04T08:30:35.920 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres 4u REG 8,1 16777216 786734 /var/lib/postgresql/14/main/pg_wal/000000010000000000000001 2024-03-04T08:30:35.920 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres 8u IPv6 98243 0t0 UDP localhost:41827->localhost:41827 2024-03-04T08:30:35.920 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres 9u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.920 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22321 postgres 10u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.920 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres cwd DIR 8,1 4096 786704 /var/lib/postgresql/14/main 2024-03-04T08:30:35.920 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.920 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres txt REG 8,1 9522144 675561 /usr/lib/postgresql/14/bin/postgres 2024-03-04T08:30:35.921 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres DEL REG 0,1 18 /dev/zero 2024-03-04T08:30:35.921 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.921 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.921 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.921 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.921 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.921 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.921 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.921 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.922 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.922 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.922 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.922 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.922 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.922 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.922 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.922 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.923 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.923 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.923 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.923 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.923 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.923 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.923 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.923 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.924 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.924 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 3328472 2387 /usr/lib/x86_64-linux-gnu/libicui18n.so.70.1 2024-03-04T08:30:35.924 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.924 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.924 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.924 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.924 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.924 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.925 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.925 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.925 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.925 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.925 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.925 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:35.925 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.925 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.926 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.926 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 0,24 26976 18 /dev/shm/PostgreSQL.2029366278 2024-03-04T08:30:35.926 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.926 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres DEL REG 0,1 4 /SYSV000c0110 2024-03-04T08:30:35.926 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.926 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres 1w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.926 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres 2w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.926 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres 3r FIFO 0,13 0t0 98231 pipe 2024-03-04T08:30:35.927 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres 4u REG 8,1 8192 786782 /var/lib/postgresql/14/main/global/1262 2024-03-04T08:30:35.927 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres 8u IPv6 98243 0t0 UDP localhost:41827->localhost:41827 2024-03-04T08:30:35.927 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres 9u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.927 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22322 postgres 10u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.927 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres cwd DIR 8,1 4096 786704 /var/lib/postgresql/14/main 2024-03-04T08:30:35.927 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.927 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres txt REG 8,1 9522144 675561 /usr/lib/postgresql/14/bin/postgres 2024-03-04T08:30:35.927 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.927 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.928 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.928 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.928 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.928 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.928 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.928 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.928 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.928 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.929 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.929 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.929 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.929 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.929 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.929 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.929 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.929 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.930 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.930 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.930 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.930 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.930 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.930 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.930 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.930 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 3328472 2387 /usr/lib/x86_64-linux-gnu/libicui18n.so.70.1 2024-03-04T08:30:35.931 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.931 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.931 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.931 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.931 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.931 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.931 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.931 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.932 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.932 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.932 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.932 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:35.932 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.932 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.932 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.932 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.933 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.933 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres 1w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.933 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres 2w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.933 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres 3r FIFO 0,13 0t0 98231 pipe 2024-03-04T08:30:35.933 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres 4u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.933 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres 8u IPv6 98243 0t0 UDP localhost:41827->localhost:41827 2024-03-04T08:30:35.933 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres 9u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.933 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22323 postgres 10u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.934 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres cwd DIR 8,1 4096 786704 /var/lib/postgresql/14/main 2024-03-04T08:30:35.934 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.934 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres txt REG 8,1 9522144 675561 /usr/lib/postgresql/14/bin/postgres 2024-03-04T08:30:35.934 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres DEL REG 0,1 18 /dev/zero 2024-03-04T08:30:35.934 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.934 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:35.934 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:35.934 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 29476472 2385 /usr/lib/x86_64-linux-gnu/libicudata.so.70.1 2024-03-04T08:30:35.934 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.935 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.935 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:35.935 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:35.935 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:35.935 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:35.935 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.935 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:35.935 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.936 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.936 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:35.936 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:35.936 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.936 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.936 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.936 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:35.937 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:35.937 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.937 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.937 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.937 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 2062664 75701 /usr/lib/x86_64-linux-gnu/libicuuc.so.70.1 2024-03-04T08:30:35.937 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 3328472 2387 /usr/lib/x86_64-linux-gnu/libicui18n.so.70.1 2024-03-04T08:30:35.937 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.937 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:35.938 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.938 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.938 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.938 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.938 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:35.938 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.938 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.938 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:35.939 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.939 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:35.939 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.939 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 1967384 79643 /usr/lib/x86_64-linux-gnu/libxml2.so.2.9.13 2024-03-04T08:30:35.939 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.939 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 0,24 26976 18 /dev/shm/PostgreSQL.2029366278 2024-03-04T08:30:35.939 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.939 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres DEL REG 0,1 4 /SYSV000c0110 2024-03-04T08:30:35.939 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.940 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres 1w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.940 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres 2w REG 8,1 653 677236 /var/log/postgresql/postgresql-14-main.log 2024-03-04T08:30:35.940 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres 3r FIFO 0,13 0t0 98231 pipe 2024-03-04T08:30:35.940 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres 4u REG 8,1 0 786813 /var/lib/postgresql/14/main/global/6100 2024-03-04T08:30:35.940 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres 8u IPv6 98243 0t0 UDP localhost:41827->localhost:41827 2024-03-04T08:30:35.940 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres 9u a_inode 0,14 0 2070 [signalfd] 2024-03-04T08:30:35.940 INFO:teuthology.orchestra.run.smithi113.stdout:postgres 22324 postgres 10u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:35.940 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.941 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.941 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root txt REG 8,1 2194240 674788 /usr/sbin/ModemManager 2024-03-04T08:30:35.941 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14384 674742 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-broadmobi.so 2024-03-04T08:30:35.941 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 63776 674771 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-simtech.so 2024-03-04T08:30:35.941 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14352 674751 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-haier.so 2024-03-04T08:30:35.941 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 100632 674775 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-ublox.so 2024-03-04T08:30:35.941 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 18608 674755 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-linktop.so 2024-03-04T08:30:35.941 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14440 674759 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-nokia-icera.so 2024-03-04T08:30:35.942 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 47384 674763 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-option-hso.so 2024-03-04T08:30:35.942 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14392 674749 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-generic.so 2024-03-04T08:30:35.942 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 22736 674741 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-anydata.so 2024-03-04T08:30:35.942 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 30912 674777 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-wavecom.so 2024-03-04T08:30:35.942 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 22872 674760 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-nokia.so 2024-03-04T08:30:35.942 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14512 674757 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-motorola.so 2024-03-04T08:30:35.942 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 47392 674740 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-altair-lte.so 2024-03-04T08:30:35.942 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 22816 674744 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-dell.so 2024-03-04T08:30:35.942 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 18760 674769 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-sierra-legacy.so 2024-03-04T08:30:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 43336 674767 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-quectel.so 2024-03-04T08:30:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 18720 674765 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-pantech.so 2024-03-04T08:30:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 30992 674758 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-mtk.so 2024-03-04T08:30:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14504 674772 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-telit.so 2024-03-04T08:30:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14464 674762 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-novatel.so 2024-03-04T08:30:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 26944 674778 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-x22x.so 2024-03-04T08:30:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14416 674770 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-sierra.so 2024-03-04T08:30:35.943 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14424 674766 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-qcom-soc.so 2024-03-04T08:30:35.944 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 31008 674779 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-zte.so 2024-03-04T08:30:35.944 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14392 674748 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-foxconn.so 2024-03-04T08:30:35.944 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14360 674768 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-samsung.so 2024-03-04T08:30:35.944 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 22784 674776 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-via.so 2024-03-04T08:30:35.944 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 18616 674753 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-intel.so 2024-03-04T08:30:35.944 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 137544 674743 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-cinterion.so 2024-03-04T08:30:35.944 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14384 674774 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-tplink.so 2024-03-04T08:30:35.945 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 129376 674752 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-huawei.so 2024-03-04T08:30:35.945 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 22704 674773 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-thuraya.so 2024-03-04T08:30:35.945 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 26944 674754 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-iridium.so 2024-03-04T08:30:35.945 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14384 674745 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-dlink.so 2024-03-04T08:30:35.945 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 39352 674761 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-novatel-lte.so 2024-03-04T08:30:35.945 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 63808 674746 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-ericsson-mbm.so 2024-03-04T08:30:35.945 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 43280 674747 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-fibocom.so 2024-03-04T08:30:35.945 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 22800 674756 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-longcheer.so 2024-03-04T08:30:35.945 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 43176 674783 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-novatel.so 2024-03-04T08:30:35.946 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 80120 674782 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-icera.so 2024-03-04T08:30:35.946 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 88312 674786 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-telit.so 2024-03-04T08:30:35.946 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 30968 674784 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-option.so 2024-03-04T08:30:35.946 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 63736 674785 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-sierra.so 2024-03-04T08:30:35.946 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 22696 674781 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-foxconn.so 2024-03-04T08:30:35.946 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 63736 674787 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-xmm.so 2024-03-04T08:30:35.946 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 18704 29008 /usr/lib/x86_64-linux-gnu/gconv/IBM850.so 2024-03-04T08:30:35.946 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 18680 674780 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-fibocom.so 2024-03-04T08:30:35.947 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 18720 29113 /usr/lib/x86_64-linux-gnu/gconv/UTF-16.so 2024-03-04T08:30:35.947 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.947 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.947 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.947 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.947 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.947 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.947 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.948 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.948 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.948 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.948 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14416 674750 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-gosuncn.so 2024-03-04T08:30:35.948 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.948 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.948 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.948 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.949 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.949 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 632904 674437 /usr/lib/x86_64-linux-gnu/libmbim-glib.so.4.7.0 2024-03-04T08:30:35.949 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 3528776 674513 /usr/lib/x86_64-linux-gnu/libqmi-glib.so.5.9.0 2024-03-04T08:30:35.949 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.949 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.949 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.949 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 51272 18682 /usr/lib/x86_64-linux-gnu/libgudev-1.0.so.0.3.0 2024-03-04T08:30:35.949 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.950 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.950 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.950 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.950 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 1144904 674461 /usr/lib/x86_64-linux-gnu/libmm-glib.so.0.9.0 2024-03-04T08:30:35.950 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 14400 674764 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-option.so 2024-03-04T08:30:35.950 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.950 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.950 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.951 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root 1u unix 0xffff8aaf09749e00 0t0 98281 type=STREAM 2024-03-04T08:30:35.951 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root 2w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.951 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.951 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.951 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root 5u unix 0xffff8aaf0974e180 0t0 98286 type=DGRAM 2024-03-04T08:30:35.951 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root 6u unix 0xffff8aaf0dbd0f00 0t0 100866 type=STREAM 2024-03-04T08:30:35.951 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root 7u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.951 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root 8u netlink 0t0 97071 KOBJECT_UEVENT 2024-03-04T08:30:35.952 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 root 10w FIFO 0,23 0t0 3526 /run/systemd/inhibit/10.ref 2024-03-04T08:30:35.952 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.952 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.952 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root txt REG 8,1 2194240 674788 /usr/sbin/ModemManager 2024-03-04T08:30:35.952 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14384 674742 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-broadmobi.so 2024-03-04T08:30:35.952 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 63776 674771 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-simtech.so 2024-03-04T08:30:35.952 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14352 674751 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-haier.so 2024-03-04T08:30:35.952 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 100632 674775 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-ublox.so 2024-03-04T08:30:35.953 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 18608 674755 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-linktop.so 2024-03-04T08:30:35.953 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14440 674759 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-nokia-icera.so 2024-03-04T08:30:35.953 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 47384 674763 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-option-hso.so 2024-03-04T08:30:35.953 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14392 674749 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-generic.so 2024-03-04T08:30:35.953 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 22736 674741 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-anydata.so 2024-03-04T08:30:35.953 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 30912 674777 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-wavecom.so 2024-03-04T08:30:35.953 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 22872 674760 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-nokia.so 2024-03-04T08:30:35.953 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14512 674757 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-motorola.so 2024-03-04T08:30:35.954 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 47392 674740 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-altair-lte.so 2024-03-04T08:30:35.954 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 22816 674744 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-dell.so 2024-03-04T08:30:35.954 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 18760 674769 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-sierra-legacy.so 2024-03-04T08:30:35.954 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 43336 674767 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-quectel.so 2024-03-04T08:30:35.954 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 18720 674765 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-pantech.so 2024-03-04T08:30:35.954 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 30992 674758 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-mtk.so 2024-03-04T08:30:35.954 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14504 674772 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-telit.so 2024-03-04T08:30:35.955 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14464 674762 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-novatel.so 2024-03-04T08:30:35.955 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 26944 674778 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-x22x.so 2024-03-04T08:30:35.955 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14416 674770 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-sierra.so 2024-03-04T08:30:35.955 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14424 674766 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-qcom-soc.so 2024-03-04T08:30:35.955 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 31008 674779 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-zte.so 2024-03-04T08:30:35.955 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14392 674748 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-foxconn.so 2024-03-04T08:30:35.955 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14360 674768 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-samsung.so 2024-03-04T08:30:35.955 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 22784 674776 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-via.so 2024-03-04T08:30:35.956 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 18616 674753 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-intel.so 2024-03-04T08:30:35.956 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 137544 674743 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-cinterion.so 2024-03-04T08:30:35.956 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14384 674774 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-tplink.so 2024-03-04T08:30:35.956 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 129376 674752 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-huawei.so 2024-03-04T08:30:35.956 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 22704 674773 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-thuraya.so 2024-03-04T08:30:35.956 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 26944 674754 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-iridium.so 2024-03-04T08:30:35.956 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14384 674745 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-dlink.so 2024-03-04T08:30:35.957 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 39352 674761 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-novatel-lte.so 2024-03-04T08:30:35.957 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 63808 674746 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-ericsson-mbm.so 2024-03-04T08:30:35.957 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 43280 674747 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-fibocom.so 2024-03-04T08:30:35.957 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 22800 674756 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-longcheer.so 2024-03-04T08:30:35.957 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 43176 674783 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-novatel.so 2024-03-04T08:30:35.957 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 80120 674782 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-icera.so 2024-03-04T08:30:35.957 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 88312 674786 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-telit.so 2024-03-04T08:30:35.957 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 30968 674784 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-option.so 2024-03-04T08:30:35.958 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 63736 674785 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-sierra.so 2024-03-04T08:30:35.958 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 22696 674781 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-foxconn.so 2024-03-04T08:30:35.958 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 63736 674787 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-xmm.so 2024-03-04T08:30:35.958 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 18704 29008 /usr/lib/x86_64-linux-gnu/gconv/IBM850.so 2024-03-04T08:30:35.958 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 18680 674780 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-fibocom.so 2024-03-04T08:30:35.958 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 18720 29113 /usr/lib/x86_64-linux-gnu/gconv/UTF-16.so 2024-03-04T08:30:35.958 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.958 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.959 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.959 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.959 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.959 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.959 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.959 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.959 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.959 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.959 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14416 674750 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-gosuncn.so 2024-03-04T08:30:35.960 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.960 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.960 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.960 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.960 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.960 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 632904 674437 /usr/lib/x86_64-linux-gnu/libmbim-glib.so.4.7.0 2024-03-04T08:30:35.960 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 3528776 674513 /usr/lib/x86_64-linux-gnu/libqmi-glib.so.5.9.0 2024-03-04T08:30:35.960 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.961 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.961 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.961 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 51272 18682 /usr/lib/x86_64-linux-gnu/libgudev-1.0.so.0.3.0 2024-03-04T08:30:35.961 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.961 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.961 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.961 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.961 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 1144904 674461 /usr/lib/x86_64-linux-gnu/libmm-glib.so.0.9.0 2024-03-04T08:30:35.961 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 14400 674764 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-option.so 2024-03-04T08:30:35.962 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.962 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.962 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.962 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root 1u unix 0xffff8aaf09749e00 0t0 98281 type=STREAM 2024-03-04T08:30:35.962 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root 2w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.962 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.962 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.962 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root 5u unix 0xffff8aaf0974e180 0t0 98286 type=DGRAM 2024-03-04T08:30:35.963 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root 6u unix 0xffff8aaf0dbd0f00 0t0 100866 type=STREAM 2024-03-04T08:30:35.963 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root 7u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.963 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root 8u netlink 0t0 97071 KOBJECT_UEVENT 2024-03-04T08:30:35.963 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22398 gmain root 10w FIFO 0,23 0t0 3526 /run/systemd/inhibit/10.ref 2024-03-04T08:30:35.963 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.963 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.963 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root txt REG 8,1 2194240 674788 /usr/sbin/ModemManager 2024-03-04T08:30:35.963 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14384 674742 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-broadmobi.so 2024-03-04T08:30:35.964 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 63776 674771 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-simtech.so 2024-03-04T08:30:35.964 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14352 674751 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-haier.so 2024-03-04T08:30:35.964 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 100632 674775 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-ublox.so 2024-03-04T08:30:35.964 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 18608 674755 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-linktop.so 2024-03-04T08:30:35.964 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14440 674759 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-nokia-icera.so 2024-03-04T08:30:35.964 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 47384 674763 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-option-hso.so 2024-03-04T08:30:35.964 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14392 674749 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-generic.so 2024-03-04T08:30:35.964 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 22736 674741 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-anydata.so 2024-03-04T08:30:35.965 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 30912 674777 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-wavecom.so 2024-03-04T08:30:35.965 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 22872 674760 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-nokia.so 2024-03-04T08:30:35.965 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14512 674757 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-motorola.so 2024-03-04T08:30:35.965 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 47392 674740 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-altair-lte.so 2024-03-04T08:30:35.965 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 22816 674744 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-dell.so 2024-03-04T08:30:35.965 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 18760 674769 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-sierra-legacy.so 2024-03-04T08:30:35.965 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 43336 674767 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-quectel.so 2024-03-04T08:30:35.965 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 18720 674765 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-pantech.so 2024-03-04T08:30:35.966 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 30992 674758 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-mtk.so 2024-03-04T08:30:35.966 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14504 674772 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-telit.so 2024-03-04T08:30:35.966 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14464 674762 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-novatel.so 2024-03-04T08:30:35.966 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 26944 674778 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-x22x.so 2024-03-04T08:30:35.966 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14416 674770 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-sierra.so 2024-03-04T08:30:35.966 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14424 674766 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-qcom-soc.so 2024-03-04T08:30:35.966 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 31008 674779 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-zte.so 2024-03-04T08:30:35.966 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14392 674748 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-foxconn.so 2024-03-04T08:30:35.967 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14360 674768 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-samsung.so 2024-03-04T08:30:35.967 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 22784 674776 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-via.so 2024-03-04T08:30:35.967 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 18616 674753 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-intel.so 2024-03-04T08:30:35.967 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 137544 674743 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-cinterion.so 2024-03-04T08:30:35.967 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14384 674774 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-tplink.so 2024-03-04T08:30:35.967 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 129376 674752 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-huawei.so 2024-03-04T08:30:35.967 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 22704 674773 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-thuraya.so 2024-03-04T08:30:35.967 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 26944 674754 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-iridium.so 2024-03-04T08:30:35.968 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14384 674745 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-dlink.so 2024-03-04T08:30:35.968 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 39352 674761 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-novatel-lte.so 2024-03-04T08:30:35.968 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 63808 674746 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-ericsson-mbm.so 2024-03-04T08:30:35.968 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 43280 674747 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-fibocom.so 2024-03-04T08:30:35.968 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 22800 674756 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-longcheer.so 2024-03-04T08:30:35.968 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 43176 674783 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-novatel.so 2024-03-04T08:30:35.968 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 80120 674782 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-icera.so 2024-03-04T08:30:35.968 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 88312 674786 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-telit.so 2024-03-04T08:30:35.969 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 30968 674784 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-option.so 2024-03-04T08:30:35.969 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 63736 674785 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-sierra.so 2024-03-04T08:30:35.969 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 22696 674781 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-foxconn.so 2024-03-04T08:30:35.969 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 63736 674787 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-xmm.so 2024-03-04T08:30:35.969 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 18704 29008 /usr/lib/x86_64-linux-gnu/gconv/IBM850.so 2024-03-04T08:30:35.969 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 18680 674780 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-shared-fibocom.so 2024-03-04T08:30:35.969 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 18720 29113 /usr/lib/x86_64-linux-gnu/gconv/UTF-16.so 2024-03-04T08:30:35.969 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 149760 632 /usr/lib/x86_64-linux-gnu/libgpg-error.so.0.32.1 2024-03-04T08:30:35.969 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.970 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:35.970 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 1296312 890 /usr/lib/x86_64-linux-gnu/libgcrypt.so.20.3.4 2024-03-04T08:30:35.970 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:35.970 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:35.970 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.970 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.970 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 477296 18878 /usr/lib/x86_64-linux-gnu/libpcre.so.3.13.3 2024-03-04T08:30:35.970 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.971 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14416 674750 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-gosuncn.so 2024-03-04T08:30:35.971 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.971 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:35.971 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:35.971 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.971 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 807936 9152 /usr/lib/x86_64-linux-gnu/libsystemd.so.0.32.0 2024-03-04T08:30:35.971 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 632904 674437 /usr/lib/x86_64-linux-gnu/libmbim-glib.so.4.7.0 2024-03-04T08:30:35.971 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 3528776 674513 /usr/lib/x86_64-linux-gnu/libqmi-glib.so.5.9.0 2024-03-04T08:30:35.972 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 273056 1087 /usr/lib/x86_64-linux-gnu/libmount.so.1.1.0 2024-03-04T08:30:35.972 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.972 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 126472 11980 /usr/lib/x86_64-linux-gnu/libpolkit-gobject-1.so.0.0.0 2024-03-04T08:30:35.972 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 51272 18682 /usr/lib/x86_64-linux-gnu/libgudev-1.0.so.0.3.0 2024-03-04T08:30:35.972 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 387464 1159 /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.7200.4 2024-03-04T08:30:35.972 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 1920400 1059 /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.7200.4 2024-03-04T08:30:35.972 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 1277712 1078 /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4 2024-03-04T08:30:35.972 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 22736 1082 /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0.7200.4 2024-03-04T08:30:35.972 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 1144904 674461 /usr/lib/x86_64-linux-gnu/libmm-glib.so.0.9.0 2024-03-04T08:30:35.973 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 14400 674764 /usr/lib/x86_64-linux-gnu/ModemManager/libmm-plugin-option.so 2024-03-04T08:30:35.973 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.973 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.973 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.973 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root 1u unix 0xffff8aaf09749e00 0t0 98281 type=STREAM 2024-03-04T08:30:35.973 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root 2w CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:35.973 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root 3u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.973 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root 4u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.974 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root 5u unix 0xffff8aaf0974e180 0t0 98286 type=DGRAM 2024-03-04T08:30:35.974 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root 6u unix 0xffff8aaf0dbd0f00 0t0 100866 type=STREAM 2024-03-04T08:30:35.974 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root 7u a_inode 0,14 0 2070 [eventfd] 2024-03-04T08:30:35.974 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root 8u netlink 0t0 97071 KOBJECT_UEVENT 2024-03-04T08:30:35.974 INFO:teuthology.orchestra.run.smithi113.stdout:ModemMana 22397 22401 gdbus root 10w FIFO 0,23 0t0 3526 /run/systemd/inhibit/10.ref 2024-03-04T08:30:35.974 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22661 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.974 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22661 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.974 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22661 root txt unknown /proc/22661/exe 2024-03-04T08:30:35.975 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22662 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.975 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22662 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.975 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22662 root txt unknown /proc/22662/exe 2024-03-04T08:30:35.975 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22663 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.975 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22663 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.975 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22663 root txt unknown /proc/22663/exe 2024-03-04T08:30:35.975 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22664 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.975 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22664 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.976 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22664 root txt unknown /proc/22664/exe 2024-03-04T08:30:35.976 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22665 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.976 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22665 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.976 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22665 root txt unknown /proc/22665/exe 2024-03-04T08:30:35.976 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22666 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.976 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22666 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.976 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22666 root txt unknown /proc/22666/exe 2024-03-04T08:30:35.976 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22667 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.977 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22667 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.977 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22667 root txt unknown /proc/22667/exe 2024-03-04T08:30:35.977 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22668 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.977 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22668 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.977 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22668 root txt unknown /proc/22668/exe 2024-03-04T08:30:35.977 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22679 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.977 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22679 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.977 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22679 root txt unknown /proc/22679/exe 2024-03-04T08:30:35.977 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22680 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.978 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22680 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.978 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22680 root txt unknown /proc/22680/exe 2024-03-04T08:30:35.978 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22681 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.978 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22681 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.978 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22681 root txt unknown /proc/22681/exe 2024-03-04T08:30:35.978 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22682 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.978 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22682 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.978 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22682 root txt unknown /proc/22682/exe 2024-03-04T08:30:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22683 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22683 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22683 root txt unknown /proc/22683/exe 2024-03-04T08:30:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22684 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22684 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22684 root txt unknown /proc/22684/exe 2024-03-04T08:30:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22685 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22685 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22685 root txt unknown /proc/22685/exe 2024-03-04T08:30:35.980 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22686 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.980 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22686 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.980 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22686 root txt unknown /proc/22686/exe 2024-03-04T08:30:35.980 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22697 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.980 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22697 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.980 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22697 root txt unknown /proc/22697/exe 2024-03-04T08:30:35.980 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22698 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.980 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22698 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.981 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22698 root txt unknown /proc/22698/exe 2024-03-04T08:30:35.981 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22699 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.981 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22699 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.981 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22699 root txt unknown /proc/22699/exe 2024-03-04T08:30:35.981 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22700 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.981 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22700 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.981 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22700 root txt unknown /proc/22700/exe 2024-03-04T08:30:35.981 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22701 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.982 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22701 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.982 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22701 root txt unknown /proc/22701/exe 2024-03-04T08:30:35.982 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22702 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.982 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22702 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.982 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22702 root txt unknown /proc/22702/exe 2024-03-04T08:30:35.982 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22703 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.982 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22703 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.982 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22703 root txt unknown /proc/22703/exe 2024-03-04T08:30:35.982 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22704 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.983 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22704 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.983 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22704 root txt unknown /proc/22704/exe 2024-03-04T08:30:35.983 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22715 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.983 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22715 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.983 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22715 root txt unknown /proc/22715/exe 2024-03-04T08:30:35.983 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22716 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.983 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22716 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.983 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22716 root txt unknown /proc/22716/exe 2024-03-04T08:30:35.984 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22717 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.984 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22717 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.984 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22717 root txt unknown /proc/22717/exe 2024-03-04T08:30:35.984 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22718 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.984 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22718 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.984 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22718 root txt unknown /proc/22718/exe 2024-03-04T08:30:35.984 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22719 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.984 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22719 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.985 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22719 root txt unknown /proc/22719/exe 2024-03-04T08:30:35.985 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22720 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.985 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22720 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.985 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22720 root txt unknown /proc/22720/exe 2024-03-04T08:30:35.985 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22721 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.985 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22721 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.985 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22721 root txt unknown /proc/22721/exe 2024-03-04T08:30:35.985 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22722 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.986 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22722 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.986 INFO:teuthology.orchestra.run.smithi113.stdout:xfsaild/d 22722 root txt unknown /proc/22722/exe 2024-03-04T08:30:35.986 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22737 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.986 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22737 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.986 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 22737 root txt unknown /proc/22737/exe 2024-03-04T08:30:35.986 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 23878 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.986 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 23878 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.986 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 23878 root txt unknown /proc/23878/exe 2024-03-04T08:30:35.986 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 25019 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.987 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 25019 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.987 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 25019 root txt unknown /proc/25019/exe 2024-03-04T08:30:35.987 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 26160 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:35.987 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 26160 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.987 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 26160 root txt unknown /proc/26160/exe 2024-03-04T08:30:35.987 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:35.987 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.987 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root txt REG 8,1 232416 12806 /usr/bin/sudo 2024-03-04T08:30:35.988 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:35.988 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.988 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:35.988 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:35.988 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:35.988 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.988 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.988 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.989 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.989 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.989 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.989 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:35.989 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:35.989 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.989 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.989 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.990 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.990 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:35.990 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:35.990 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:35.990 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:35.990 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.990 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.990 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.990 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 516992 95600 /usr/libexec/sudo/sudoers.so 2024-03-04T08:30:35.991 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.991 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.991 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 130040 95595 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-03-04T08:30:35.991 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.991 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.991 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:35.991 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.991 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root 0r FIFO 0,13 0t0 104619 pipe 2024-03-04T08:30:35.992 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:35.992 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:35.992 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root 3r FIFO 0,13 0t0 103220 pipe 2024-03-04T08:30:35.992 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root 4w FIFO 0,13 0t0 103220 pipe 2024-03-04T08:30:35.992 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root 5r REG 8,1 1698 658119 /etc/sudoers 2024-03-04T08:30:35.992 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root 6u netlink 0t0 103229 AUDIT 2024-03-04T08:30:35.992 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27425 root 7u unix 0xffff8aaf0974a580 0t0 103230 type=DGRAM 2024-03-04T08:30:35.992 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:35.993 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.993 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:35.993 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.993 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:35.993 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.993 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:35.993 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.993 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:35.993 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:35.994 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.994 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:35.994 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:35.994 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:35.994 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.994 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root 0r FIFO 0,13 0t0 104619 pipe 2024-03-04T08:30:35.994 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:35.994 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27426 root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:35.995 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:35.995 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:35.995 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root txt REG 8,1 232416 12806 /usr/bin/sudo 2024-03-04T08:30:35.995 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:35.995 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:35.995 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:35.995 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:35.995 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:35.996 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:35.996 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:35.996 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:35.996 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:35.996 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:35.996 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:35.996 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:35.996 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:35.997 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:35.997 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:35.997 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:35.997 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:35.997 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:35.997 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:35.997 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 516992 95600 /usr/libexec/sudo/sudoers.so 2024-03-04T08:30:35.997 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:35.998 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:35.998 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:35.998 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:35.998 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:35.998 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:35.998 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:35.998 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 130040 95595 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-03-04T08:30:35.998 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:35.998 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:35.999 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:35.999 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:35.999 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root 0r FIFO 0,13 0t0 104622 pipe 2024-03-04T08:30:35.999 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:35.999 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:35.999 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root 3r FIFO 0,13 0t0 104357 pipe 2024-03-04T08:30:35.999 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root 4w FIFO 0,13 0t0 104357 pipe 2024-03-04T08:30:35.999 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root 5r REG 8,1 1698 658119 /etc/sudoers 2024-03-04T08:30:36.000 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root 6u netlink 0t0 104366 AUDIT 2024-03-04T08:30:36.000 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27428 root 7u unix 0xffff8aaf05acb480 0t0 104367 type=DGRAM 2024-03-04T08:30:36.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27434 log root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27435 service root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27436 admin_soc root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.376+0000 7f9b6d800640 1 Processor -- start 2024-03-04T08:30:36.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27437 signal_ha root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27438 rocksdb:l root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.376+0000 7f9b6d800640 1 -- start start 2024-03-04T08:30:36.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27439 rocksdb:h root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27455 ceph-mon root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.376+0000 7f9b6d800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b681582e0 0x7f9b680701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:36.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27456 fn_monsto root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27457 msgr-work root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27458 msgr-work root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.376+0000 7f9b6d800640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b681586c0 con 0x7f9b6806ddf0 2024-03-04T08:30:36.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.376+0000 7f9b6d800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b6806c210 con 0x7f9b680b0ed0 2024-03-04T08:30:36.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.376+0000 7f9b6d800640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6806c4f0 con 0x7f9b681582e0 2024-03-04T08:30:36.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.376+0000 7f9b66a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9b681582e0 0x7f9b680701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:36.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.376+0000 7f9b67400640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f9b6806ddf0 0x7f9b6815af40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:50352/0 (socket says 172.21.15.113:50352) 2024-03-04T08:30:36.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.376+0000 7f9b67400640 1 -- 172.21.15.113:0/1134470810 learned_addr learned my addr 172.21.15.113:0/1134470810 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:36.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.380+0000 7f9b66a00640 1 -- 172.21.15.113:0/1134470810 >> v1:172.21.15.136:6789/0 conn(0x7f9b680b0ed0 legacy=0x7f9b680ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:36.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.380+0000 7f9b66a00640 1 -- 172.21.15.113:0/1134470810 >> v1:172.21.15.113:6789/0 conn(0x7f9b6806ddf0 legacy=0x7f9b6815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:36.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.380+0000 7f9b66a00640 1 -- 172.21.15.113:0/1134470810 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6806f6a0 con 0x7f9b681582e0 2024-03-04T08:30:36.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.380+0000 7f9b66a00640 1 --2- 172.21.15.113:0/1134470810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b681582e0 0x7f9b680701c0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f9b580097c0 tx=0x7f9b5801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830f395b2cc27748 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:36.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.380+0000 7f9b65600640 1 -- 172.21.15.113:0/1134470810 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b580b0070 con 0x7f9b681582e0 2024-03-04T08:30:36.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.380+0000 7f9b65600640 1 -- 172.21.15.113:0/1134470810 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b580ab600 con 0x7f9b681582e0 2024-03-04T08:30:36.146 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.380+0000 7f9b65600640 1 -- 172.21.15.113:0/1134470810 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b580ab9d0 con 0x7f9b681582e0 2024-03-04T08:30:36.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.380+0000 7f9b6d800640 1 -- 172.21.15.113:0/1134470810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b681582e0 msgr2=0x7f9b680701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:36.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.380+0000 7f9b6d800640 1 --2- 172.21.15.113:0/1134470810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b681582e0 0x7f9b680701c0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f9b580097c0 tx=0x7f9b5801c4a0 comp rx=0 tx=0).stop 2024-03-04T08:30:36.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 -- 172.21.15.113:0/1134470810 shutdown_connections 2024-03-04T08:30:36.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 --2- 172.21.15.113:0/1134470810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b681582e0 0x7f9b680701c0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:36.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 -- 172.21.15.113:0/1134470810 >> 172.21.15.113:0/1134470810 conn(0x7f9b68066db0 msgr2=0x7f9b680671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:36.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 -- 172.21.15.113:0/1134470810 shutdown_connections 2024-03-04T08:30:36.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 -- 172.21.15.113:0/1134470810 wait complete. 2024-03-04T08:30:36.147 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 Processor -- start 2024-03-04T08:30:36.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 -- start start 2024-03-04T08:30:36.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b6806ddf0 0x7f9b6814f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:36.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b680b0ed0 0x7f9b68150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:36.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b681582e0 0x7f9b68147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:36.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b680ceb60 con 0x7f9b6806ddf0 2024-03-04T08:30:36.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b680ce9e0 con 0x7f9b680b0ed0 2024-03-04T08:30:36.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b68168d60 con 0x7f9b681582e0 2024-03-04T08:30:36.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b6806ddf0 0x7f9b6814f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:36.148 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66a00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b6806ddf0 0x7f9b6814f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:56498/0 (socket says 172.21.15.113:56498) 2024-03-04T08:30:36.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66a00640 1 -- 172.21.15.113:0/4101059398 learned_addr learned my addr 172.21.15.113:0/4101059398 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:36.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b67400640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b681582e0 0x7f9b68147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:36.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66000640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b680b0ed0 0x7f9b68150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:36.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66a00640 1 -- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b681582e0 msgr2=0x7f9b68147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:36.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66a00640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b681582e0 0x7f9b68147f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:36.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66a00640 1 -- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b680b0ed0 msgr2=0x7f9b68150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:36.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66a00640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b680b0ed0 0x7f9b68150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:36.149 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66a00640 1 -- 172.21.15.113:0/4101059398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b68151d00 con 0x7f9b6806ddf0 2024-03-04T08:30:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66000640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b680b0ed0 0x7f9b68150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b66a00640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b6806ddf0 0x7f9b6814f2a0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f9b580ae040 tx=0x7f9b58019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b5ea00640 1 -- 172.21.15.113:0/4101059398 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b580b8030 con 0x7f9b6806ddf0 2024-03-04T08:30:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b5ea00640 1 -- 172.21.15.113:0/4101059398 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b580b4410 con 0x7f9b6806ddf0 2024-03-04T08:30:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b6d800640 1 -- 172.21.15.113:0/4101059398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b68151fc0 con 0x7f9b6806ddf0 2024-03-04T08:30:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.384+0000 7f9b5ea00640 1 -- 172.21.15.113:0/4101059398 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b580b0040 con 0x7f9b6806ddf0 2024-03-04T08:30:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.388+0000 7f9b6d800640 1 -- 172.21.15.113:0/4101059398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b681506a0 con 0x7f9b6806ddf0 2024-03-04T08:30:36.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.388+0000 7f9b6d800640 1 -- 172.21.15.113:0/4101059398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b68068e90 con 0x7f9b6806ddf0 2024-03-04T08:30:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.388+0000 7f9b5ea00640 1 -- 172.21.15.113:0/4101059398 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b580bb070 con 0x7f9b6806ddf0 2024-03-04T08:30:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.388+0000 7f9b5ea00640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b48066f60 0x7f9b48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.388+0000 7f9b5ea00640 1 -- 172.21.15.113:0/4101059398 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f9b5811ee70 con 0x7f9b6806ddf0 2024-03-04T08:30:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.388+0000 7f9b66000640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b48066f60 0x7f9b48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.388+0000 7f9b66000640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b48066f60 0x7f9b48069420 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9b68149880 tx=0x7f9b54009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:36.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.392+0000 7f9b5ea00640 1 -- 172.21.15.113:0/4101059398 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9b580c3020 con 0x7f9b6806ddf0 2024-03-04T08:30:36.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27459 msgr-work root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27460 OpHistory root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27461 fin root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27462 safe_time root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27463 cpu_tp root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27464 cpu_tp root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27465 cpu_tp root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27466 cpu_tp root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27467 ms_dispat root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27468 ms_local root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27469 safe_time root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27470 ms_dispat root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 27471 ms_local root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root txt REG 8,1 11493912 80329 /usr/bin/ceph-mon 2024-03-04T08:30:36.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 0,14 104640 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 1w FIFO 0,13 0t0 104620 pipe 2024-03-04T08:30:36.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 2w FIFO 0,13 0t0 104621 pipe 2024-03-04T08:30:36.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 3r FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 4w FIFO 0,13 0t0 104636 pipe 2024-03-04T08:30:36.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 5w REG 8,1 197023592 677284 /var/log/ceph/ceph-mon.a.log 2024-03-04T08:30:36.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 6uW REG 0,23 6 3564 /run/ceph/ceph-mon.a.pid 2024-03-04T08:30:36.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 7r FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 8w FIFO 0,13 0t0 104637 pipe 2024-03-04T08:30:36.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 9u unix 0xffff8aaf09bb0000 0t0 104638 /var/run/ceph/ceph-mon.a.asok type=STREAM 2024-03-04T08:30:36.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 10r FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 11w FIFO 0,13 0t0 104639 pipe 2024-03-04T08:30:36.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 12u a_inode 0,14 0 104640 [io_uring] 2024-03-04T08:30:36.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 13r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 14uW REG 252,4 0 50331783 /var/lib/ceph/mon/ceph-a/store.db/LOCK 2024-03-04T08:30:36.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 15r DIR 252,4 152 50331782 /var/lib/ceph/mon/ceph-a/store.db 2024-03-04T08:30:36.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 16u IPv4 186069 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:41744 (ESTABLISHED) 2024-03-04T08:30:36.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 17r REG 252,4 4885694 50331788 /var/lib/ceph/mon/ceph-a/store.db/000081.sst 2024-03-04T08:30:36.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 18w REG 252,4 5798 50331791 /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T08:30:36.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 19u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 20r FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 21w FIFO 0,13 0t0 104646 pipe 2024-03-04T08:30:36.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 22u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 23r FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 24w FIFO 0,13 0t0 104647 pipe 2024-03-04T08:30:36.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 25u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 26r FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 27w FIFO 0,13 0t0 104648 pipe 2024-03-04T08:30:36.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 28u IPv4 103234 0t0 TCP smithi113.front.sepia.ceph.com:3300 (LISTEN) 2024-03-04T08:30:36.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 29u IPv4 103235 0t0 TCP smithi113.front.sepia.ceph.com:6789 (LISTEN) 2024-03-04T08:30:36.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 30u IPv4 101352 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:54104 (ESTABLISHED) 2024-03-04T08:30:36.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 31r FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 32w FIFO 0,13 0t0 101334 pipe 2024-03-04T08:30:36.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 33u IPv4 103247 0t0 TCP smithi113.front.sepia.ceph.com:38558->smithi136.front.sepia.ceph.com:3301 (ESTABLISHED) 2024-03-04T08:30:36.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 34r FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 35w FIFO 0,13 0t0 101335 pipe 2024-03-04T08:30:36.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 36r FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 37w FIFO 0,13 0t0 101336 pipe 2024-03-04T08:30:36.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 38w REG 8,1 194904 677287 /var/log/ceph/ceph.audit.log 2024-03-04T08:30:36.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 39u IPv4 101340 0t0 TCP smithi113.front.sepia.ceph.com:49910->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 40w REG 8,1 399047 677285 /var/log/ceph/ceph.log 2024-03-04T08:30:36.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 41u IPv4 172202 0t0 TCP smithi113.front.sepia.ceph.com:52846->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 42u IPv4 104730 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:33156 (ESTABLISHED) 2024-03-04T08:30:36.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 43u IPv4 109352 0t0 TCP smithi113.front.sepia.ceph.com:3300->192.168.0.1:34816 (ESTABLISHED) 2024-03-04T08:30:36.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 44u IPv4 134793 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:53616 (ESTABLISHED) 2024-03-04T08:30:36.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 45u IPv4 161239 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:38502 (ESTABLISHED) 2024-03-04T08:30:36.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 46u IPv4 185156 0t0 TCP smithi113.front.sepia.ceph.com:3300->smithi113.front.sepia.ceph.com:35254 (ESTABLISHED) 2024-03-04T08:30:36.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mon 27430 38749 rstore_co root 47w REG 252,4 591047 50331785 /var/lib/ceph/mon/ceph-a/store.db/000079.log 2024-03-04T08:30:36.331 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.331 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.331 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:36.331 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.331 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.331 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.331 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.332 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.332 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.332 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.332 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.332 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.332 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.332 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.332 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.333 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root 0r FIFO 0,13 0t0 104622 pipe 2024-03-04T08:30:36.333 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.333 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27431 root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.560+0000 7f9b6d800640 1 -- 172.21.15.113:0/4101059398 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9b68075c00 con 0x7f9b6806ddf0 2024-03-04T08:30:36.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27473 log root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.560+0000 7f9b5ea00640 1 -- 172.21.15.113:0/4101059398 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 60 entries v85) v1 ==== 81+0+3706 (secure 0 0 0) 0x7f9b58016090 con 0x7f9b6806ddf0 2024-03-04T08:30:36.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27475 msgr-work root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.402 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6839/1409794802 2024-03-05T08:24:26.023653+0000 2024-03-04T08:30:36.402 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/2368734308 2024-03-05T07:58:45.780423+0000 2024-03-04T08:30:36.402 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/3005395426 2024-03-05T07:37:40.596420+0000 2024-03-04T08:30:36.402 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/986515123 2024-03-05T08:00:40.798160+0000 2024-03-04T08:30:36.402 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/2299268553 2024-03-05T07:52:10.722331+0000 2024-03-04T08:30:36.402 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/3944001911 2024-03-05T07:45:20.664657+0000 2024-03-04T08:30:36.402 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/2299268553 2024-03-05T07:52:10.722331+0000 2024-03-04T08:30:36.402 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/2522634751 2024-03-05T08:07:30.861130+0000 2024-03-04T08:30:36.403 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/1691762848 2024-03-05T07:49:30.698378+0000 2024-03-04T08:30:36.403 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/2732565639 2024-03-05T08:02:45.817095+0000 2024-03-04T08:30:36.403 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/742633123 2024-03-05T07:47:40.683981+0000 2024-03-04T08:30:36.403 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/4038214481 2024-03-05T07:36:05.584380+0000 2024-03-04T08:30:36.403 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/3944001911 2024-03-05T07:45:20.664657+0000 2024-03-04T08:30:36.403 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/986515123 2024-03-05T08:00:40.798160+0000 2024-03-04T08:30:36.403 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/605142559 2024-03-05T07:44:20.656043+0000 2024-03-04T08:30:36.403 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/1035982016 2024-03-05T08:05:45.840907+0000 2024-03-04T08:30:36.404 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/3005395426 2024-03-05T07:37:40.596420+0000 2024-03-04T08:30:36.404 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/1928887637 2024-03-05T08:20:05.977949+0000 2024-03-04T08:30:36.404 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/810789479 2024-03-05T07:54:40.746387+0000 2024-03-04T08:30:36.404 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/2544977425 2024-03-05T08:21:40.993913+0000 2024-03-04T08:30:36.404 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/4197507712 2024-03-05T08:04:30.830241+0000 2024-03-04T08:30:36.404 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/3348592059 2024-03-05T07:40:15.619912+0000 2024-03-04T08:30:36.404 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/1310042629 2024-03-05T07:38:50.604670+0000 2024-03-04T08:30:36.404 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/2522634751 2024-03-05T08:07:30.861130+0000 2024-03-04T08:30:36.405 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/742633123 2024-03-05T07:47:40.683981+0000 2024-03-04T08:30:36.405 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/1310057122 2024-03-05T08:13:45.911820+0000 2024-03-04T08:30:36.405 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/4038214481 2024-03-05T07:36:05.584380+0000 2024-03-04T08:30:36.405 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/1691762848 2024-03-05T07:49:30.698378+0000 2024-03-04T08:30:36.405 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/605142559 2024-03-05T07:44:20.656043+0000 2024-03-04T08:30:36.405 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/2544977425 2024-03-05T08:21:40.993913+0000 2024-03-04T08:30:36.405 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/344022786 2024-03-05T07:57:05.766508+0000 2024-03-04T08:30:36.405 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/2999156262 2024-03-05T08:22:51.004939+0000 2024-03-04T08:30:36.406 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/1401330284 2024-03-05T08:11:10.891335+0000 2024-03-04T08:30:36.406 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/1310042629 2024-03-05T07:38:50.604670+0000 2024-03-04T08:30:36.406 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/2997246507 2024-03-05T07:41:40.631034+0000 2024-03-04T08:30:36.406 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/2368734308 2024-03-05T07:58:45.780423+0000 2024-03-04T08:30:36.406 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/1310057122 2024-03-05T08:13:45.911820+0000 2024-03-04T08:30:36.406 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/3348592059 2024-03-05T07:40:15.619912+0000 2024-03-04T08:30:36.406 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6837/2228772746 2024-03-05T08:23:26.011590+0000 2024-03-04T08:30:36.406 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/2997246507 2024-03-05T07:41:40.631034+0000 2024-03-04T08:30:36.407 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/3388011401 2024-03-05T08:01:30.805387+0000 2024-03-04T08:30:36.407 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6835/247269911 2024-03-05T08:16:00.936524+0000 2024-03-04T08:30:36.407 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/2732565639 2024-03-05T08:02:45.817095+0000 2024-03-04T08:30:36.407 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/4197507712 2024-03-05T08:04:30.830241+0000 2024-03-04T08:30:36.407 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6838/1409794802 2024-03-05T08:24:26.023653+0000 2024-03-04T08:30:36.407 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/1035982016 2024-03-05T08:05:45.840907+0000 2024-03-04T08:30:36.407 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/3825257303 2024-03-05T08:09:55.880790+0000 2024-03-04T08:30:36.407 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/3388011401 2024-03-05T08:01:30.805387+0000 2024-03-04T08:30:36.408 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/3825257303 2024-03-05T08:09:55.880790+0000 2024-03-04T08:30:36.408 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6835/1401330284 2024-03-05T08:11:10.891335+0000 2024-03-04T08:30:36.408 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6834/247269911 2024-03-05T08:16:00.936524+0000 2024-03-04T08:30:36.408 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/1928887637 2024-03-05T08:20:05.977949+0000 2024-03-04T08:30:36.408 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/4083765155 2024-03-05T08:17:55.953381+0000 2024-03-04T08:30:36.408 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6833/2999156262 2024-03-05T08:22:51.004939+0000 2024-03-04T08:30:36.408 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/4083765155 2024-03-05T08:17:55.953381+0000 2024-03-04T08:30:36.408 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6832/810789479 2024-03-05T07:54:40.746387+0000 2024-03-04T08:30:36.409 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6834/893414524 2024-03-05T08:21:25.990484+0000 2024-03-04T08:30:36.409 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6837/344022786 2024-03-05T07:57:05.766508+0000 2024-03-04T08:30:36.409 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.113:6836/893414524 2024-03-05T08:21:25.990484+0000 2024-03-04T08:30:36.409 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.15.136:6836/2228772746 2024-03-05T08:23:26.011590+0000 2024-03-04T08:30:36.410 INFO:teuthology.orchestra.run.smithi113.stderr:listed 60 entries 2024-03-04T08:30:36.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27476 msgr-work root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27477 msgr-work root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 -- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b48066f60 msgr2=0x7f9b48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:36.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b48066f60 0x7f9b48069420 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9b68149880 tx=0x7f9b54009040 comp rx=0 tx=0).stop 2024-03-04T08:30:36.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 -- 172.21.15.113:0/4101059398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b6806ddf0 msgr2=0x7f9b6814f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:36.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b6806ddf0 0x7f9b6814f2a0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f9b580ae040 tx=0x7f9b58019240 comp rx=0 tx=0).stop 2024-03-04T08:30:36.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 -- 172.21.15.113:0/4101059398 shutdown_connections 2024-03-04T08:30:36.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f9b681582e0 0x7f9b68147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:36.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b680b0ed0 0x7f9b68150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:36.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f9b48066f60 0x7f9b48069420 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:36.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 --2- 172.21.15.113:0/4101059398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b6806ddf0 0x7f9b6814f2a0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:36.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 -- 172.21.15.113:0/4101059398 >> 172.21.15.113:0/4101059398 conn(0x7f9b68066db0 msgr2=0x7f9b68067190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:36.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 -- 172.21.15.113:0/4101059398 shutdown_connections 2024-03-04T08:30:36.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:35.564+0000 7f9b6d800640 1 -- 172.21.15.113:0/4101059398 wait complete. 2024-03-04T08:30:36.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27511 service root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27512 admin_soc root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27513 ceph_time root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.491 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-04T08:30:36.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:30:36.492 DEBUG:teuthology.orchestra.run.smithi113:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-04T08:30:36.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27514 fn_anonym root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27515 signal_ha root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27516 mgrsb-fin root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27517 ms_dispat root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27518 ms_local root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27519 io_contex root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27520 io_contex root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27521 safe_time root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27522 safe_time root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27523 safe_time root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27524 fn_anonym root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27525 flusher root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27526 safe_time root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27625 ceph-mgr root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27626 ceph-mgr root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27627 ceph-mgr root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27628 ceph-mgr root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27629 ceph-mgr root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27630 ceph-mgr root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root txt REG 8,1 4448208 80322 /usr/bin/ceph-mgr 2024-03-04T08:30:36.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 7092680 47993 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 2024-03-04T08:30:36.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 2989432 4414 /usr/lib/x86_64-linux-gnu/libgfortran.so.5.0.0 2024-03-04T08:30:36.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 38017672 47994 /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so 2024-03-04T08:30:36.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 3804808 22387 /usr/lib/python3/dist-packages/numpy/core/_multiarray_umath.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 7561792 50517 /usr/lib/librbd.so.1.19.0 2024-03-04T08:30:36.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 1112896 50562 /usr/lib/python3/dist-packages/rbd.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 822512 80156 /usr/lib/python3/dist-packages/numpy/random/_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 78624 50069 /usr/lib/python3.10/lib-dynload/_asyncio.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 350576 80152 /usr/lib/python3/dist-packages/numpy/random/_bounded_integers.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 230960 80154 /usr/lib/python3/dist-packages/numpy/random/_common.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 170840 80167 /usr/lib/python3/dist-packages/numpy/random/bit_generator.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 31264 50108 /usr/lib/python3.10/lib-dynload/termios.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 63840 66923 /usr/lib/python3/dist-packages/scipy/_lib/_ccallback_c.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 1956376 50527 /usr/lib/libcephfs.so.2.0.0 2024-03-04T08:30:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 24336 50089 /usr/lib/python3.10/lib-dynload/_multiprocessing.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 694312 80171 /usr/lib/python3/dist-packages/numpy/random/mtrand.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 148960 80076 /usr/lib/python3/dist-packages/numpy/linalg/_umath_linalg.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 125544 22386 /usr/lib/python3/dist-packages/numpy/core/_multiarray_tests.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 96328 80160 /usr/lib/python3/dist-packages/numpy/random/_pcg64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 289592 4405 /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0 2024-03-04T08:30:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 395776 47992 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 2024-03-04T08:30:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 129240 4237 /usr/lib/x86_64-linux-gnu/libyaml-0.so.2.0.6 2024-03-04T08:30:36.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 1116728 50533 /usr/lib/python3/dist-packages/rados.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 246624 4535 /usr/lib/python3/dist-packages/yaml/_yaml.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 765704 5526 /usr/lib/python3/dist-packages/_brotli.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 14536 50077 /usr/lib/python3.10/lib-dynload/_contextvars.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 81464 80162 /usr/lib/python3/dist-packages/numpy/random/_philox.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 136216 50079 /usr/lib/python3.10/lib-dynload/_ctypes.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 94408 80158 /usr/lib/python3/dist-packages/numpy/random/_mt19937.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 438864 641 /usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02.1 2024-03-04T08:30:36.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 484720 9046 /usr/lib/x86_64-linux-gnu/libcryptsetup.so.12.7.0 2024-03-04T08:30:36.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 54360 80165 /usr/lib/python3/dist-packages/numpy/random/_sfc64.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 72080 875 /usr/lib/x86_64-linux-gnu/libjson-c.so.5.1.0 2024-03-04T08:30:36.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 35000 2009 /usr/lib/x86_64-linux-gnu/libargon2.so.1 2024-03-04T08:30:36.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 858960 43846 /usr/lib/python3/dist-packages/cryptography/hazmat/bindings/_openssl.abi3.so 2024-03-04T08:30:36.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 72128 79990 /usr/lib/python3/dist-packages/numpy/fft/_pocketfft_internal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 104672 673965 /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so 2024-03-04T08:30:36.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 49936 670586 /usr/lib/python3/dist-packages/simplejson/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 187200 12815 /usr/lib/x86_64-linux-gnu/libmpdec.so.2.5.1 2024-03-04T08:30:36.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 19136 80077 /usr/lib/python3/dist-packages/numpy/linalg/lapack_lite.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 14848 664525 /usr/lib/python3/dist-packages/markupsafe/_speedups.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 180632 50084 /usr/lib/python3.10/lib-dynload/_decimal.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 216560 48613 /usr/lib/python3.10/lib-dynload/_ssl.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 63728 48611 /usr/lib/python3.10/lib-dynload/_hashlib.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 23664 50091 /usr/lib/python3.10/lib-dynload/_queue.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 509272 50547 /usr/lib/python3/dist-packages/cephfs.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 194472 5651 /usr/lib/python3/dist-packages/_cffi_backend.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 30920 2951 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 2024-03-04T08:30:36.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 43336 665388 /usr/lib/python3/dist-packages/bcrypt/_bcrypt.abi3.so 2024-03-04T08:30:36.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 110888 50092 /usr/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 14688 50098 /usr/lib/python3.10/lib-dynload/_uuid.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 49160 50085 /usr/lib/python3.10/lib-dynload/_json.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 14672 48612 /usr/lib/python3.10/lib-dynload/_opcode.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 1666136 50508 /usr/lib/librados.so.2.0.0 2024-03-04T08:30:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 5846784 85677 /usr/lib/x86_64-linux-gnu/libpython3.10.so.1.0 2024-03-04T08:30:36.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 1358520 20539 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6 2024-03-04T08:30:36.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 303840 80307 /usr/lib/libcephsqlite.so 2024-03-04T08:30:36.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 1w FIFO 0,13 0t0 104623 pipe 2024-03-04T08:30:36.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 2w FIFO 0,13 0t0 104624 pipe 2024-03-04T08:30:36.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 3r FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 4w FIFO 0,13 0t0 106510 pipe 2024-03-04T08:30:36.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 6r FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 7w FIFO 0,13 0t0 103237 pipe 2024-03-04T08:30:36.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 9r FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 10w FIFO 0,13 0t0 103238 pipe 2024-03-04T08:30:36.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 12r FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 13w FIFO 0,13 0t0 103239 pipe 2024-03-04T08:30:36.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 14w REG 8,1 3494565 677286 /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:30:36.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 15uW REG 0,23 6 3567 /run/ceph/ceph-mgr.y.pid 2024-03-04T08:30:36.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 16r FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 17w FIFO 0,13 0t0 105544 pipe 2024-03-04T08:30:36.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 18u unix 0xffff8aaf719e2580 0t0 105545 /var/run/ceph/ceph-mgr.y.asok type=STREAM 2024-03-04T08:30:36.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 19r FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 20w FIFO 0,13 0t0 105549 pipe 2024-03-04T08:30:36.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 21r FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 22w FIFO 0,13 0t0 105550 pipe 2024-03-04T08:30:36.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 23u IPv4 149460 0t0 TCP smithi113.front.sepia.ceph.com:43838->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mgr 27472 27631 ceph-mgr root 24u IPv4 104397 0t0 TCP smithi113.front.sepia.ceph.com:54104->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.857 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.858 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.858 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root txt REG 8,1 232416 12806 /usr/bin/sudo 2024-03-04T08:30:36.858 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:36.858 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:36.858 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.858 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.858 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.859 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.859 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:36.859 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:36.859 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:36.859 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:36.859 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.859 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:36.859 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:36.860 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 516992 95600 /usr/libexec/sudo/sudoers.so 2024-03-04T08:30:36.860 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.860 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.860 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.860 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:36.860 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:36.860 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:36.860 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.861 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:36.861 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:36.861 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.861 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:36.861 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.861 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:36.861 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 130040 95595 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-03-04T08:30:36.861 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.861 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:36.862 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:36.862 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.862 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root 0r FIFO 0,13 0t0 104663 pipe 2024-03-04T08:30:36.862 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:36.862 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:36.862 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root 3r FIFO 0,13 0t0 106540 pipe 2024-03-04T08:30:36.862 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root 4w FIFO 0,13 0t0 106540 pipe 2024-03-04T08:30:36.862 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root 5r REG 8,1 1698 658119 /etc/sudoers 2024-03-04T08:30:36.862 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root 6u netlink 0t0 106549 AUDIT 2024-03-04T08:30:36.863 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27527 root 7u unix 0xffff8aaf036a8000 0t0 106550 type=DGRAM 2024-03-04T08:30:36.863 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.863 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.863 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root txt REG 8,1 232416 12806 /usr/bin/sudo 2024-03-04T08:30:36.863 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:36.863 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:36.863 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:36.863 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:36.863 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.864 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.864 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.864 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.864 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:36.864 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.864 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:36.864 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.864 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.865 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.865 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:36.865 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:36.865 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:36.865 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:36.865 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:36.865 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:36.866 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:36.866 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.866 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:36.866 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 516992 95600 /usr/libexec/sudo/sudoers.so 2024-03-04T08:30:36.866 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.866 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.866 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:36.866 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 130040 95595 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-03-04T08:30:36.867 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.867 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:36.867 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:36.867 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.867 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root 0r FIFO 0,13 0t0 104666 pipe 2024-03-04T08:30:36.867 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:36.867 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:36.867 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root 3r FIFO 0,13 0t0 102232 pipe 2024-03-04T08:30:36.867 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root 4w FIFO 0,13 0t0 102232 pipe 2024-03-04T08:30:36.868 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root 5r REG 8,1 1698 658119 /etc/sudoers 2024-03-04T08:30:36.868 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root 6u netlink 0t0 102241 AUDIT 2024-03-04T08:30:36.868 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27528 root 7u unix 0xffff8aaf7d7f3c00 0t0 102242 type=DGRAM 2024-03-04T08:30:36.868 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.868 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.868 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root txt REG 8,1 232416 12806 /usr/bin/sudo 2024-03-04T08:30:36.868 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:36.868 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:36.869 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:36.869 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:36.869 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:36.869 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.869 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.869 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.869 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.869 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:36.870 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.870 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.870 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:36.870 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:36.870 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 516992 95600 /usr/libexec/sudo/sudoers.so 2024-03-04T08:30:36.870 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.870 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:36.870 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.870 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.871 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:36.871 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:36.871 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:36.871 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:36.871 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.871 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:36.871 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.871 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:36.871 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 130040 95595 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-03-04T08:30:36.872 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.872 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:36.872 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:36.872 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.872 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root 0r FIFO 0,13 0t0 104669 pipe 2024-03-04T08:30:36.872 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:36.872 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:36.873 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root 3r FIFO 0,13 0t0 104680 pipe 2024-03-04T08:30:36.873 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root 4w FIFO 0,13 0t0 104680 pipe 2024-03-04T08:30:36.873 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root 5r REG 8,1 1698 658119 /etc/sudoers 2024-03-04T08:30:36.873 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root 6u netlink 0t0 104689 AUDIT 2024-03-04T08:30:36.873 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27529 root 7u unix 0xffff8aaf09bb7800 0t0 104690 type=DGRAM 2024-03-04T08:30:36.873 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.873 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.873 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root txt REG 8,1 232416 12806 /usr/bin/sudo 2024-03-04T08:30:36.873 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:36.874 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:36.874 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:36.874 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:36.874 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:36.874 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.874 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.874 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.874 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.875 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.875 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.875 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.875 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:36.875 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:36.875 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.875 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:36.875 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:36.875 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:36.876 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:36.876 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:36.876 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 516992 95600 /usr/libexec/sudo/sudoers.so 2024-03-04T08:30:36.876 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:36.876 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.876 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:36.876 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.876 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:36.877 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:36.877 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 130040 95595 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-03-04T08:30:36.877 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:36.877 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:36.877 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:36.877 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.877 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root 0r FIFO 0,13 0t0 104672 pipe 2024-03-04T08:30:36.877 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:36.878 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:36.878 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root 3r FIFO 0,13 0t0 104403 pipe 2024-03-04T08:30:36.878 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root 4w FIFO 0,13 0t0 104403 pipe 2024-03-04T08:30:36.878 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root 5r REG 8,1 1698 658119 /etc/sudoers 2024-03-04T08:30:36.878 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root 6u netlink 0t0 104412 AUDIT 2024-03-04T08:30:36.878 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 27530 root 7u unix 0xffff8aaf05acbc00 0t0 104413 type=DGRAM 2024-03-04T08:30:36.878 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.878 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.879 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:36.879 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.879 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.879 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.879 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.879 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.879 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.879 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.880 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.880 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.880 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.880 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.880 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.880 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root 0r FIFO 0,13 0t0 104663 pipe 2024-03-04T08:30:36.880 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:36.880 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27531 root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:36.881 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.881 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.881 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:36.881 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.881 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.881 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.881 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.881 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.882 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.882 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.882 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.882 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.882 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.882 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.882 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.882 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root 0r FIFO 0,13 0t0 104666 pipe 2024-03-04T08:30:36.883 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:36.883 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27533 root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:36.883 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.883 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.883 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:36.883 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.883 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.883 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.883 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.884 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.884 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.884 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.884 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.884 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.884 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.884 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.884 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root 0r FIFO 0,13 0t0 104669 pipe 2024-03-04T08:30:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27534 root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:36.885 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:36.886 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.886 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.886 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:36.886 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:36.886 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.886 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:36.886 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.886 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:36.887 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.887 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root 0r FIFO 0,13 0t0 104672 pipe 2024-03-04T08:30:36.887 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:36.887 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 27538 root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:36.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:36.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:36.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:36.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:36.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:36.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:36.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:36.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:36.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:36.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:36.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:36.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:36.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:36.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:36.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:36.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:36.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:36.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:36.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:36.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:36.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:36.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:36.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:36.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:36.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:36.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:36.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:36.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:36.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:36.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:36.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:36.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:36.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:36.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:36.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:36.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:36.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:36.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:36.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:36.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:36.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:36.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:36.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:36.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:36.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:36.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:36.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:36.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:36.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:36.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:36.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:36.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:36.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:36.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:36.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:36.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:36.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:36.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:36.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:36.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:36.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:36.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:36.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:36.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:36.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:36.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:36.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:36.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:36.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:36.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:36.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:36.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:36.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:36.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:36.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:36.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:36.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:36.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:36.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:36.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:36.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:36.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:36.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:36.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:36.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:36.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:36.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:36.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:36.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:36.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:36.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:36.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:36.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:36.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:36.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:36.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:36.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:36.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:36.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:36.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:36.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:36.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:36.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:36.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:36.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:36.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:36.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:36.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:36.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:36.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:36.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:36.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:36.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:36.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:36.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:36.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:36.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:36.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:36.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:36.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:36.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:36.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:36.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:36.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:36.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:36.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:36.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:36.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:36.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:36.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:36.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:36.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 14w REG 8,1 774829697 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:36.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:36.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:36.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:36.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:36.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:36.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:36.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:36.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:36.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:36.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:36.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:36.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:36.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:36.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:36.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:36.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:36.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:36.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:36.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:36.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:36.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:36.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:36.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:36.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:36.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:36.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:36.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:36.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:36.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:36.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:36.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:36.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:36.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:36.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:36.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:36.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:36.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:36.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:36.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:36.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:36.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:36.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:36.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:36.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:36.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:36.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:36.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:36.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:36.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:36.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:36.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:36.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:36.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:36.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:36.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:36.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:36.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:36.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:36.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:36.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:36.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:36.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:36.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27547 log root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:36.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:36.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:36.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:36.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:36.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:36.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:36.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:36.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:36.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:36.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:36.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:36.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:36.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:36.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:36.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:36.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:36.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:36.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:36.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:36.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:36.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:36.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:36.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:36.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:36.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:36.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:36.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:36.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:36.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:36.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:36.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:36.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:36.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:36.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:36.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:36.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:36.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:36.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:36.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 14w REG 8,1 774829697 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:36.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:36.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:36.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:36.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:36.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:36.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:36.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:36.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:36.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:36.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:36.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:36.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:36.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:36.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:36.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:36.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:36.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:36.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:36.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:36.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:36.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:36.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:36.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:36.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:36.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:36.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:36.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:36.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:36.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:36.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:36.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:36.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:36.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:36.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:36.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:36.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:36.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:36.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:36.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:36.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:36.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:36.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:36.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:36.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:36.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:36.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:36.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:36.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:36.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:36.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:36.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:36.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:36.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:36.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27549 msgr-work root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:36.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:36.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:36.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:36.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:36.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:36.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:36.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:36.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:36.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:36.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:36.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:36.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:36.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:36.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:36.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:36.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:36.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:36.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:36.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:36.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:36.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:36.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:36.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:36.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:36.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:36.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:36.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:36.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:36.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:36.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:36.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:36.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:36.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:36.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:36.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:36.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:36.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:36.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:36.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 14w REG 8,1 774829697 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:36.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:36.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:36.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:36.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:36.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:36.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:36.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:36.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:36.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:36.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:36.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:36.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:36.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:36.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:36.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:36.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:36.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:36.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:36.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:36.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:36.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:36.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:36.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:36.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:36.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:36.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:36.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:36.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:36.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:36.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:36.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:36.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:36.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:36.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:36.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:36.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:36.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:36.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:36.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:36.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:36.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:36.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:36.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:36.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:36.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:36.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:36.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:36.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:36.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:36.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:36.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:36.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:36.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:36.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:36.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:36.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:36.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:36.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:36.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:36.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:36.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:36.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:36.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27550 msgr-work root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:36.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:36.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:36.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:36.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:36.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:36.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:36.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:36.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:36.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:36.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:36.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:36.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:36.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:36.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:36.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:36.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:36.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:36.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:36.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:36.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:36.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:36.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:36.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:36.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:36.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:36.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:36.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:36.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:36.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:36.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:36.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:36.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:36.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:36.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:36.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:36.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:36.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:36.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:36.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:36.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:36.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:36.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:36.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:36.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:36.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:36.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:36.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:36.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:36.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:36.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:36.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:36.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:36.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:36.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:36.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:36.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:36.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:36.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:36.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:36.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:36.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:36.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:36.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:36.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:36.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:36.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:36.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:36.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:36.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:36.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:36.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:36.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:36.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:36.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:36.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:36.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:36.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:36.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:36.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:36.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:36.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:36.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:36.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:36.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:36.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:36.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:36.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:36.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:36.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:36.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:36.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:36.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:36.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:36.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:36.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 14w REG 8,1 774830210 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:36.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:36.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:36.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:36.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:36.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:36.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:36.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:36.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:36.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:36.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:36.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:36.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:36.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:36.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:36.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:36.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:36.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:36.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:36.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:36.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:36.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:36.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:36.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:36.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:36.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:36.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:36.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:36.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:36.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:36.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:36.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:36.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:36.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:36.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:36.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:36.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:36.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:36.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:36.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:36.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:36.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:36.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:36.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:36.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:36.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:36.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:36.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:36.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:36.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:36.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27551 msgr-work root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 14w REG 8,1 774832297 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27555 service root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 14w REG 8,1 774832297 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27556 admin_soc root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 14w REG 8,1 774832297 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27632 signal_ha root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27633 io_contex root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27634 io_contex root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27652 OpHistory root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27653 ceph_time root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27690 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27707 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27724 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27735 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27747 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27762 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 27774 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28178 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28179 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28180 safe_time root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28181 safe_time root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28182 safe_time root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28183 safe_time root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28184 fn_anonym root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28259 rocksdb:l root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28262 rocksdb:l root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28264 rocksdb:l root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 28267 rocksdb:h root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29661 bstore_ai root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 29662 bstore_di root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30477 bstore_ai root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30478 bstore_di root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30846 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30847 cfin root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30848 bstore_kv root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30849 bstore_kv root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30850 bstore_me root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30871 ms_dispat root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 14w REG 8,1 774832998 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30873 ms_local root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30874 safe_time root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30876 safe_time root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30879 ms_dispat root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30881 ms_local root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30884 ms_dispat root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30886 ms_local root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30888 ms_dispat root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:37.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:37.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:37.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:37.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:37.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:37.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:37.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:37.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:37.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:37.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:37.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:37.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:37.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:37.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:37.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:37.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:37.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:37.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:37.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:37.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:37.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:37.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:37.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:37.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:37.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:37.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:37.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:37.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:37.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:37.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:37.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:37.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:37.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:37.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:37.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:37.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:37.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30890 ms_local root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:37.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:37.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:37.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:37.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:37.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:37.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:37.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:37.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:37.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:37.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:37.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:37.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:37.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:37.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:37.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:37.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:37.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:37.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:37.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:37.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:37.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:37.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:37.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:37.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:37.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:37.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:37.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:37.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:37.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:37.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:37.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:37.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:37.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:37.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:37.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:37.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:37.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:37.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:37.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:37.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:37.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:37.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:37.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:37.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:37.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:37.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:37.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:37.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:37.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:37.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:37.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:37.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:37.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:37.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:37.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:37.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:37.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:37.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:37.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:37.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:37.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:37.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:37.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:37.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:37.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:37.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:37.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:37.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:37.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:37.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:37.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:37.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:37.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:37.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:37.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:37.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:37.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:37.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:37.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:37.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:37.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:37.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:37.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:37.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:37.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:37.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:37.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:37.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:37.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:37.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:37.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:37.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:37.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:37.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:37.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:37.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:37.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:37.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:37.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:37.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:37.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:37.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:37.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:37.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:37.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:37.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:37.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:37.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:37.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30893 ms_dispat root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30895 ms_local root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30897 ms_dispat root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30899 ms_local root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30902 ms_dispat root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30904 ms_local root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30906 fn_anonym root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.158 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T08:30:38.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:30:38.158 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-04T08:30:38.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30907 finisher root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30908 safe_time root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30909 safe_time root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30910 ceph-osd root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30911 osd_srv_a root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30912 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30913 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30914 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30915 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30916 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30917 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30918 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30919 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30920 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30921 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30922 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30923 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30924 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30925 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30926 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30927 tp_osd_tp root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root DEL REG 0,18 103337 /[aio] 2024-03-04T08:30:38.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root DEL REG 0,18 108582 /[aio] 2024-03-04T08:30:38.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 0,14 108572 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 1w FIFO 0,13 0t0 104667 pipe 2024-03-04T08:30:38.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 2w FIFO 0,13 0t0 104668 pipe 2024-03-04T08:30:38.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 3r FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 4w FIFO 0,13 0t0 100302 pipe 2024-03-04T08:30:38.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 6r FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 7w FIFO 0,13 0t0 100303 pipe 2024-03-04T08:30:38.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 9r FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 10w FIFO 0,13 0t0 100304 pipe 2024-03-04T08:30:38.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 12r FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 13w FIFO 0,13 0t0 100305 pipe 2024-03-04T08:30:38.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 14w REG 8,1 774836440 677281 /var/log/ceph/ceph-osd.1.log 2024-03-04T08:30:38.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 15uW REG 0,23 6 3569 /run/ceph/ceph-osd.1.pid 2024-03-04T08:30:38.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 16r FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 17w FIFO 0,13 0t0 100313 pipe 2024-03-04T08:30:38.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 18u unix 0xffff8aaf08400780 0t0 100314 /var/run/ceph/ceph-osd.1.asok type=STREAM 2024-03-04T08:30:38.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 19u IPv4 100329 0t0 TCP *:6800 (LISTEN) 2024-03-04T08:30:38.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 20u IPv4 100331 0t0 TCP *:6801 (LISTEN) 2024-03-04T08:30:38.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 21u IPv4 100334 0t0 TCP *:6802 (LISTEN) 2024-03-04T08:30:38.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 22u IPv4 100338 0t0 TCP *:6803 (LISTEN) 2024-03-04T08:30:38.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 23u IPv4 100343 0t0 TCP *:6804 (LISTEN) 2024-03-04T08:30:38.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 24u IPv4 100349 0t0 TCP *:6805 (LISTEN) 2024-03-04T08:30:38.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 25u IPv4 108548 0t0 TCP *:6806 (LISTEN) 2024-03-04T08:30:38.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 26u IPv4 108556 0t0 TCP *:6807 (LISTEN) 2024-03-04T08:30:38.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 27r FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 28w FIFO 0,13 0t0 104714 pipe 2024-03-04T08:30:38.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 29r FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 30w FIFO 0,13 0t0 104715 pipe 2024-03-04T08:30:38.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 31u sock 0,8 0t0 108569 protocol: UDPv6 2024-03-04T08:30:38.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 32r DIR 252,1 357 64 /var/lib/ceph/osd/ceph-1 2024-03-04T08:30:38.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 33uW REG 252,1 37 69 /var/lib/ceph/osd/ceph-1/fsid 2024-03-04T08:30:38.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 34u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 35u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 36u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 37u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 38u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 39u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 40u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 41u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 42u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 43u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 44u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 45u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 46u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 47u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 48u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 49u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 50u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 51u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 52u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 53u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 54u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 55u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 56u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 57u REG 252,1 96636764160 70 /var/lib/ceph/osd/ceph-1/block 2024-03-04T08:30:38.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 58u a_inode 0,14 0 108572 [io_uring] 2024-03-04T08:30:38.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 59r FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 60w FIFO 0,13 0t0 106920 pipe 2024-03-04T08:30:38.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 61u IPv4 102318 0t0 TCP smithi113.front.sepia.ceph.com:40302->smithi136.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 62r FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 63w FIFO 0,13 0t0 106921 pipe 2024-03-04T08:30:38.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 64u IPv4 189730 0t0 TCP smithi113.front.sepia.ceph.com:32872->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 65u IPv4 182892 0t0 TCP smithi113.front.sepia.ceph.com:50144->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 66u IPv4 184988 0t0 TCP smithi113.front.sepia.ceph.com:60402->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 67u IPv4 180113 0t0 TCP smithi113.front.sepia.ceph.com:38906->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 68u IPv4 184772 0t0 TCP smithi113.front.sepia.ceph.com:49866->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 69u IPv4 189731 0t0 TCP smithi113.front.sepia.ceph.com:37934->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 70u IPv4 187556 0t0 TCP smithi113.front.sepia.ceph.com:38988->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 71u IPv4 178916 0t0 TCP smithi113.front.sepia.ceph.com:58770->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 72u IPv4 182804 0t0 TCP smithi113.front.sepia.ceph.com:47740->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 73u IPv4 178558 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:55108 (ESTABLISHED) 2024-03-04T08:30:38.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 74u IPv4 179434 0t0 TCP smithi113.front.sepia.ceph.com:58768->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 75u IPv4 180114 0t0 TCP smithi113.front.sepia.ceph.com:43866->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 76u IPv4 184989 0t0 TCP smithi113.front.sepia.ceph.com:57800->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 77u IPv4 186240 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36874 (ESTABLISHED) 2024-03-04T08:30:38.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 78u IPv4 178745 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:39580 (ESTABLISHED) 2024-03-04T08:30:38.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 79u IPv4 178561 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:56426 (ESTABLISHED) 2024-03-04T08:30:38.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 80u IPv4 178742 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:33134 (ESTABLISHED) 2024-03-04T08:30:38.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 81u IPv4 184079 0t0 TCP smithi113.front.sepia.ceph.com:47502->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 82u IPv4 179435 0t0 TCP smithi113.front.sepia.ceph.com:37248->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 83u IPv4 186243 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58402 (ESTABLISHED) 2024-03-04T08:30:38.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 84u IPv4 171676 0t0 TCP smithi113.front.sepia.ceph.com:51008->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:38.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 85u IPv4 186744 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi113.front.sepia.ceph.com:36860 (ESTABLISHED) 2024-03-04T08:30:38.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 86u IPv4 149461 0t0 TCP smithi113.front.sepia.ceph.com:43848->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 87u IPv4 176773 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:38350 (ESTABLISHED) 2024-03-04T08:30:38.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 88u IPv4 175556 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:51316 (ESTABLISHED) 2024-03-04T08:30:38.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 89u IPv4 172612 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:38034 (ESTABLISHED) 2024-03-04T08:30:38.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 90u IPv4 181370 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:57394 (ESTABLISHED) 2024-03-04T08:30:38.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 91u IPv4 175562 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:37170 (ESTABLISHED) 2024-03-04T08:30:38.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 92u IPv4 189465 0t0 TCP smithi113.front.sepia.ceph.com:6806->smithi136.front.sepia.ceph.com:59594 (ESTABLISHED) 2024-03-04T08:30:38.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 93u IPv4 189462 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:37936 (ESTABLISHED) 2024-03-04T08:30:38.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 94u IPv4 187756 0t0 TCP smithi113.front.sepia.ceph.com:44718->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:38.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 95u IPv4 184972 0t0 TCP smithi113.front.sepia.ceph.com:59190->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 96u IPv4 181367 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi136.front.sepia.ceph.com:39736 (ESTABLISHED) 2024-03-04T08:30:38.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 97u IPv4 170202 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:46578 (ESTABLISHED) 2024-03-04T08:30:38.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 98u IPv4 172219 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi136.front.sepia.ceph.com:36112 (ESTABLISHED) 2024-03-04T08:30:38.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 99u IPv4 174606 0t0 TCP smithi113.front.sepia.ceph.com:6800->smithi113.front.sepia.ceph.com:33866 (ESTABLISHED) 2024-03-04T08:30:38.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 100u IPv4 186747 0t0 TCP smithi113.front.sepia.ceph.com:6804->smithi113.front.sepia.ceph.com:58396 (ESTABLISHED) 2024-03-04T08:30:38.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 101u IPv4 168888 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:59596 (ESTABLISHED) 2024-03-04T08:30:38.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 102u IPv4 181528 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi113.front.sepia.ceph.com:51272 (ESTABLISHED) 2024-03-04T08:30:38.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27543 30928 osd_srv_h root 104u IPv4 178548 0t0 TCP smithi113.front.sepia.ceph.com:6802->smithi136.front.sepia.ceph.com:36924 (ESTABLISHED) 2024-03-04T08:30:38.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 14w REG 8,1 913145871 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27569 log root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27573 msgr-work root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27574 msgr-work root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27575 msgr-work root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27591 service root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27592 admin_soc root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27647 signal_ha root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27648 io_contex root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27649 io_contex root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27664 OpHistory root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27665 ceph_time root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27697 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:38.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:38.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:38.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:38.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:38.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:38.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:38.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:38.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:38.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:38.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:38.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:38.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:38.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:38.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:38.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:38.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:38.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:38.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27708 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:38.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:38.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:38.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:38.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:38.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:38.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:38.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:38.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:38.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:38.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:38.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:38.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:38.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:38.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:38.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:38.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:38.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:38.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:38.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:38.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:38.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:38.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:38.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:38.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:38.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:38.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:38.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:38.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:38.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:38.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:38.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:38.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:38.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:38.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:38.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:38.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:38.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:38.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:38.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:38.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:38.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:38.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:38.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:38.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:38.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:38.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:38.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:38.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:38.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:38.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:38.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:38.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:38.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:38.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:38.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:38.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:38.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:38.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:38.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:38.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:38.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:38.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:38.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:38.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:38.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:38.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:38.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:38.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:38.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:38.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:38.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:38.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:38.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:38.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:38.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:38.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:38.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:38.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:38.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:38.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:38.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:38.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:38.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:38.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:38.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:38.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:38.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:38.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:38.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:38.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:38.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:38.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:38.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:38.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:38.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:38.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:38.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:38.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:38.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:38.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:38.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:38.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:38.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:38.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:38.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:38.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:38.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:38.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:38.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:38.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:38.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:38.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:38.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:38.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:38.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:38.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:38.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:38.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:38.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:38.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:38.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:38.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:38.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:38.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27720 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27732 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27744 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27756 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27768 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27792 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27793 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27796 safe_time root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27799 safe_time root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27801 safe_time root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27804 safe_time root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27807 fn_anonym root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27970 rocksdb:l root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27974 rocksdb:l root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27976 rocksdb:l root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 27979 rocksdb:h root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29473 bstore_ai root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 29474 bstore_di root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30207 bstore_ai root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30208 bstore_di root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30433 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30434 cfin root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30435 bstore_kv root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30436 bstore_kv root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 14w REG 8,1 913139983 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30437 bstore_me root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30438 ms_dispat root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30439 ms_local root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30440 safe_time root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30441 safe_time root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30442 ms_dispat root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30443 ms_local root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30444 ms_dispat root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:30:39.715 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-03-04T08:30:39.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30445 ms_local root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30446 ms_dispat root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30447 ms_local root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30448 ms_dispat root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30449 ms_local root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30450 ms_dispat root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30451 ms_local root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30452 ms_dispat root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30453 ms_local root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30454 fn_anonym root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:39.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:39.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:39.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:39.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:39.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:39.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:39.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:39.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:39.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:39.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:39.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:39.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:39.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:39.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:39.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:39.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:39.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:39.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:39.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:39.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:39.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:39.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:39.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:39.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:39.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:39.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:39.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:39.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:39.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:39.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:39.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:39.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:39.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:39.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:39.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30455 finisher root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:39.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:39.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:39.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:39.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:39.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:39.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:39.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:39.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:39.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:39.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:39.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:39.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:39.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:39.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:39.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:39.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:39.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:39.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:39.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:39.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:39.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:39.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:39.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:39.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:39.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:39.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:39.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:39.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:39.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:39.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:39.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:39.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:39.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:39.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:39.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:39.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:39.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:39.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:39.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:39.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:39.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:39.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:39.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:39.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:39.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:39.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:39.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:39.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:39.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:39.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:39.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:39.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:39.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:39.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:39.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:39.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:39.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:39.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:39.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:39.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:39.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:39.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:39.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:39.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:39.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:39.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:39.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:39.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:39.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:39.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:39.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:39.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:39.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:39.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:39.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:39.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:39.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:39.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:39.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:39.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:39.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:39.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:39.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:39.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:39.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:39.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:39.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:39.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:39.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:39.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:39.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:39.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:39.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:39.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:39.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:39.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:39.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:39.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:39.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:39.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:39.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:39.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:39.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:39.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:39.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:39.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:39.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:39.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:39.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:39.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:39.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:39.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:39.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30456 safe_time root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30457 safe_time root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30458 ceph-osd root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30459 osd_srv_a root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30460 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30461 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30462 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30463 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30464 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30465 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30466 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30467 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30468 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30469 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 14w REG 8,1 913140194 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30470 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 14w REG 8,1 913142792 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30471 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 14w REG 8,1 913142792 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30472 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 14w REG 8,1 913142792 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30473 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 14w REG 8,1 913142792 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30474 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 14w REG 8,1 913142868 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30475 tp_osd_tp root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root DEL REG 0,18 102315 /[aio] 2024-03-04T08:30:40.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root DEL REG 0,18 105558 /[aio] 2024-03-04T08:30:40.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 0,14 107865 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 1w FIFO 0,13 0t0 104664 pipe 2024-03-04T08:30:40.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 2w FIFO 0,13 0t0 104665 pipe 2024-03-04T08:30:40.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 3r FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 4w FIFO 0,13 0t0 102253 pipe 2024-03-04T08:30:40.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 6r FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 7w FIFO 0,13 0t0 102254 pipe 2024-03-04T08:30:40.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 9r FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 10w FIFO 0,13 0t0 102255 pipe 2024-03-04T08:30:40.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 12r FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 13w FIFO 0,13 0t0 102256 pipe 2024-03-04T08:30:40.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 14w REG 8,1 913145871 677280 /var/log/ceph/ceph-osd.0.log 2024-03-04T08:30:40.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 15uW REG 0,23 6 3573 /run/ceph/ceph-osd.0.pid 2024-03-04T08:30:40.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 16r FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 17w FIFO 0,13 0t0 102259 pipe 2024-03-04T08:30:40.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 18u unix 0xffff8aaf7d7f2580 0t0 102260 /var/run/ceph/ceph-osd.0.asok type=STREAM 2024-03-04T08:30:40.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 19u IPv4 107655 0t0 TCP *:6824 (LISTEN) 2024-03-04T08:30:40.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 20u IPv4 107681 0t0 TCP *:6825 (LISTEN) 2024-03-04T08:30:40.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 21u IPv4 107708 0t0 TCP *:6826 (LISTEN) 2024-03-04T08:30:40.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 22u IPv4 107736 0t0 TCP *:6827 (LISTEN) 2024-03-04T08:30:40.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 23u IPv4 107765 0t0 TCP *:6828 (LISTEN) 2024-03-04T08:30:40.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 24u IPv4 107795 0t0 TCP *:6829 (LISTEN) 2024-03-04T08:30:40.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 25u IPv4 107826 0t0 TCP *:6830 (LISTEN) 2024-03-04T08:30:40.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 26u IPv4 107858 0t0 TCP *:6831 (LISTEN) 2024-03-04T08:30:40.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 27r FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 28w FIFO 0,13 0t0 106799 pipe 2024-03-04T08:30:40.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 29r FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 30w FIFO 0,13 0t0 106800 pipe 2024-03-04T08:30:40.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 31u sock 0,8 0t0 102294 protocol: UDPv6 2024-03-04T08:30:40.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 32r DIR 252,0 357 64 /var/lib/ceph/osd/ceph-0 2024-03-04T08:30:40.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 33uW REG 252,0 37 69 /var/lib/ceph/osd/ceph-0/fsid 2024-03-04T08:30:40.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 34u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 35u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 36u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 37u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 38u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 39u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 40u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 41u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 42u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 43u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 44u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 45u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 46u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 47u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 48u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 49u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 50u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 51u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 52u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 53u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 54u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 55u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 56u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 57u REG 252,0 96636764160 70 /var/lib/ceph/osd/ceph-0/block 2024-03-04T08:30:40.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 58u a_inode 0,14 0 107865 [io_uring] 2024-03-04T08:30:40.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 59r FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 60w FIFO 0,13 0t0 108588 pipe 2024-03-04T08:30:40.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 61u IPv4 181088 0t0 TCP smithi113.front.sepia.ceph.com:57870->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 62r FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 63w FIFO 0,13 0t0 108589 pipe 2024-03-04T08:30:40.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 64u IPv4 177083 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:60582 (ESTABLISHED) 2024-03-04T08:30:40.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 65u IPv4 186177 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:53910 (ESTABLISHED) 2024-03-04T08:30:40.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 66u IPv4 178696 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:34904 (ESTABLISHED) 2024-03-04T08:30:40.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 67u IPv4 178699 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38838 (ESTABLISHED) 2024-03-04T08:30:40.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 68u IPv4 176062 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:52464 (ESTABLISHED) 2024-03-04T08:30:40.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 69u IPv4 176065 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:38112 (ESTABLISHED) 2024-03-04T08:30:40.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 70u IPv4 180398 0t0 TCP smithi113.front.sepia.ceph.com:49128->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 71u IPv4 175860 0t0 TCP smithi113.front.sepia.ceph.com:49516->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 72u IPv4 173541 0t0 TCP smithi113.front.sepia.ceph.com:40650->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 73u IPv4 181089 0t0 TCP smithi113.front.sepia.ceph.com:39828->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 74u IPv4 178741 0t0 TCP smithi113.front.sepia.ceph.com:52778->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 75u IPv4 169854 0t0 TCP smithi113.front.sepia.ceph.com:43746->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 76u IPv4 186298 0t0 TCP smithi113.front.sepia.ceph.com:44946->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 77u IPv4 182689 0t0 TCP smithi113.front.sepia.ceph.com:34742->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 78u IPv4 189681 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33580 (ESTABLISHED) 2024-03-04T08:30:40.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 79u IPv4 186299 0t0 TCP smithi113.front.sepia.ceph.com:37426->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 80u IPv4 186180 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:41238 (ESTABLISHED) 2024-03-04T08:30:40.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 81u IPv4 185455 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:52000 (ESTABLISHED) 2024-03-04T08:30:40.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 82u IPv4 189684 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi136.front.sepia.ceph.com:52492 (ESTABLISHED) 2024-03-04T08:30:40.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 83u IPv4 189687 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi136.front.sepia.ceph.com:53208 (ESTABLISHED) 2024-03-04T08:30:40.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 84u IPv4 177224 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:40812 (ESTABLISHED) 2024-03-04T08:30:40.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 85u IPv4 167875 0t0 TCP smithi113.front.sepia.ceph.com:48156->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 86u IPv4 182102 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:41786 (ESTABLISHED) 2024-03-04T08:30:40.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 87u IPv4 173968 0t0 TCP smithi113.front.sepia.ceph.com:6830->smithi113.front.sepia.ceph.com:58768 (ESTABLISHED) 2024-03-04T08:30:40.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 88u IPv4 182105 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:41218 (ESTABLISHED) 2024-03-04T08:30:40.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 89u IPv4 173971 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:37248 (ESTABLISHED) 2024-03-04T08:30:40.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 90u IPv4 184134 0t0 TCP smithi113.front.sepia.ceph.com:35046->smithi136.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 91u IPv4 187840 0t0 TCP smithi113.front.sepia.ceph.com:42416->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 92u IPv4 177086 0t0 TCP smithi113.front.sepia.ceph.com:6828->smithi113.front.sepia.ceph.com:38612 (ESTABLISHED) 2024-03-04T08:30:40.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 93u IPv4 174105 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:59086 (ESTABLISHED) 2024-03-04T08:30:40.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 94u IPv4 171306 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:43612 (ESTABLISHED) 2024-03-04T08:30:40.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 95u IPv4 185004 0t0 TCP smithi113.front.sepia.ceph.com:36860->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 96u IPv4 184309 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44718 (ESTABLISHED) 2024-03-04T08:30:40.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 97u IPv4 175816 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:35680 (ESTABLISHED) 2024-03-04T08:30:40.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 98u IPv4 178506 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi136.front.sepia.ceph.com:46338 (ESTABLISHED) 2024-03-04T08:30:40.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 99u IPv4 175947 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi136.front.sepia.ceph.com:33122 (ESTABLISHED) 2024-03-04T08:30:40.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 100u IPv4 169602 0t0 TCP smithi113.front.sepia.ceph.com:6824->smithi113.front.sepia.ceph.com:41410 (ESTABLISHED) 2024-03-04T08:30:40.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 101u IPv4 185005 0t0 TCP smithi113.front.sepia.ceph.com:58396->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 102u IPv4 188515 0t0 TCP smithi113.front.sepia.ceph.com:36276->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 103u IPv4 185096 0t0 TCP smithi113.front.sepia.ceph.com:34310->smithi136.front.sepia.ceph.com:6810 (ESTABLISHED) 2024-03-04T08:30:40.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 104u IPv4 137988 0t0 TCP smithi113.front.sepia.ceph.com:53616->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27544 30476 osd_srv_h root 106u IPv4 187621 0t0 TCP smithi113.front.sepia.ceph.com:6826->smithi113.front.sepia.ceph.com:44866 (ESTABLISHED) 2024-03-04T08:30:40.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 14w REG 8,1 984900926 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 14w REG 8,1 984886419 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27570 log root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 14w REG 8,1 984886419 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27576 msgr-work root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27577 msgr-work root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27578 msgr-work root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27593 service root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27594 admin_soc root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27637 signal_ha root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27638 io_contex root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27639 io_contex root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27656 OpHistory root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27657 ceph_time root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27696 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27713 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27723 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27736 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27748 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27759 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 27771 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28167 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28168 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28169 safe_time root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:40.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:40.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:40.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:40.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:40.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:40.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:40.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:40.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:40.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:40.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:40.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28170 safe_time root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:40.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:40.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:40.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:40.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:40.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:40.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:40.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:40.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:40.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:40.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:40.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:40.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:40.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:40.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:40.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:40.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:40.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:40.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:40.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:40.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:40.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:40.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:40.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:40.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:40.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:40.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:40.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:40.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:40.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:40.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:40.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:40.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:40.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:40.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:40.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:40.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:40.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:40.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:40.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:40.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:40.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:40.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:40.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:40.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:40.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:40.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:40.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:40.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:40.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:40.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:40.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:40.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:40.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:40.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:40.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:40.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:40.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:40.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:40.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:40.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:40.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:40.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:40.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:40.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:40.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:40.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:40.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:40.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:40.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:40.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:40.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:40.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:40.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:40.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:40.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:40.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:40.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:40.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:40.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:40.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:40.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:40.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:40.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:40.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:40.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:40.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:40.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:40.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:40.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:40.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:40.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 14w REG 8,1 984889025 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:40.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:40.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:40.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:40.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:40.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:40.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:40.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:40.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:40.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:40.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:40.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:40.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:40.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:40.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:40.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:40.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:40.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:40.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:40.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:40.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:40.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:40.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:40.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:40.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:40.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:40.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:40.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:40.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:40.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:40.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:40.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:40.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:40.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:40.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:40.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:40.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:40.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:40.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:40.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:40.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:40.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:40.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:40.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:40.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28171 safe_time root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28172 safe_time root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28173 fn_anonym root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28189 rocksdb:l root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28190 rocksdb:l root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28191 rocksdb:l root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 28192 rocksdb:h root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29657 bstore_ai root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 29658 bstore_di root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30479 bstore_ai root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30480 bstore_di root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30841 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30842 cfin root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-04T08:30:41.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:30:41.292 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-04T08:30:41.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30843 bstore_kv root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30844 bstore_kv root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30845 bstore_me root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30851 ms_dispat root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30852 ms_local root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30853 safe_time root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30854 safe_time root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30855 ms_dispat root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30856 ms_local root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30857 ms_dispat root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30858 ms_local root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30859 ms_dispat root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30860 ms_local root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30861 ms_dispat root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30862 ms_local root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30863 ms_dispat root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30864 ms_local root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30865 ms_dispat root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30866 ms_local root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30867 fn_anonym root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30868 finisher root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30869 safe_time root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30870 safe_time root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30872 ceph-osd root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30875 osd_srv_a root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30877 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30878 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30880 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30882 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:41.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:41.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:41.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:41.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:41.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:41.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:41.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:41.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:41.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:41.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:41.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:41.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:41.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:41.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:41.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:41.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:41.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:41.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:41.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:41.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:41.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:41.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:41.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:41.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:41.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:41.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:41.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:41.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:41.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:41.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:41.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:41.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:41.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:41.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:41.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:41.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:41.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30883 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:41.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:41.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:41.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:41.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:41.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:41.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:41.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:41.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:41.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:41.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:41.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:41.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:41.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:41.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:41.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:41.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:41.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:41.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:41.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:41.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:41.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:41.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:41.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:41.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:41.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:41.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:41.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:41.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:41.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:41.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:41.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:41.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:41.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:41.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:41.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:41.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:41.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:41.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:41.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:41.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:41.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:41.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:41.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:41.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:41.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:41.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:41.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:41.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:41.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:41.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:41.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:41.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:41.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:41.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:41.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:41.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:41.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:41.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:41.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:41.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:41.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:41.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:41.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:41.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:41.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:41.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:41.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:41.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:41.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:41.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:41.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:41.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:41.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:41.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:41.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:41.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:41.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:41.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:41.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:41.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:41.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:41.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 14w REG 8,1 984891625 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:41.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:41.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:41.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:41.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:41.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:41.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:41.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:41.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:41.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:41.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:41.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:41.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:41.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:41.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:41.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:41.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:41.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:41.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30885 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 14w REG 8,1 984898316 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30887 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 14w REG 8,1 984898316 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30889 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 14w REG 8,1 984898316 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30891 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 14w REG 8,1 984900926 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30892 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 14w REG 8,1 984900926 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30894 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 14w REG 8,1 984900926 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30896 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 14w REG 8,1 984900926 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30898 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 14w REG 8,1 984900926 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30900 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 14w REG 8,1 984900926 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30901 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 14w REG 8,1 984900926 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30903 tp_osd_tp root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root DEL REG 0,18 105560 /[aio] 2024-03-04T08:30:42.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root DEL REG 0,18 104444 /[aio] 2024-03-04T08:30:42.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 0,14 102305 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 1w FIFO 0,13 0t0 104673 pipe 2024-03-04T08:30:42.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 2w FIFO 0,13 0t0 104674 pipe 2024-03-04T08:30:42.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 3r FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 4w FIFO 0,13 0t0 104428 pipe 2024-03-04T08:30:42.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 6r FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 7w FIFO 0,13 0t0 104429 pipe 2024-03-04T08:30:42.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 9r FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 10w FIFO 0,13 0t0 104430 pipe 2024-03-04T08:30:42.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 12r FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 13w FIFO 0,13 0t0 104431 pipe 2024-03-04T08:30:42.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 14w REG 8,1 984900926 677283 /var/log/ceph/ceph-osd.3.log 2024-03-04T08:30:42.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 15uW REG 0,23 6 3575 /run/ceph/ceph-osd.3.pid 2024-03-04T08:30:42.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 16r FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 17w FIFO 0,13 0t0 103294 pipe 2024-03-04T08:30:42.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 18u unix 0xffff8aaf09748f00 0t0 103295 /var/run/ceph/ceph-osd.3.asok type=STREAM 2024-03-04T08:30:42.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 19u IPv4 106575 0t0 TCP *:6808 (LISTEN) 2024-03-04T08:30:42.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 20u IPv4 106585 0t0 TCP *:6809 (LISTEN) 2024-03-04T08:30:42.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 21u IPv4 106596 0t0 TCP *:6810 (LISTEN) 2024-03-04T08:30:42.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 22u IPv4 106608 0t0 TCP *:6811 (LISTEN) 2024-03-04T08:30:42.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 23u IPv4 106621 0t0 TCP *:6812 (LISTEN) 2024-03-04T08:30:42.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 24u IPv4 106635 0t0 TCP *:6813 (LISTEN) 2024-03-04T08:30:42.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 25u IPv4 107613 0t0 TCP *:6814 (LISTEN) 2024-03-04T08:30:42.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 26u IPv4 107630 0t0 TCP *:6816 (LISTEN) 2024-03-04T08:30:42.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 27r FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 28w FIFO 0,13 0t0 103316 pipe 2024-03-04T08:30:42.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 29r FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 30w FIFO 0,13 0t0 103317 pipe 2024-03-04T08:30:42.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 31u sock 0,8 0t0 102302 protocol: UDPv6 2024-03-04T08:30:42.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 32r DIR 252,3 357 64 /var/lib/ceph/osd/ceph-3 2024-03-04T08:30:42.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 33uW REG 252,3 37 69 /var/lib/ceph/osd/ceph-3/fsid 2024-03-04T08:30:42.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 34u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 35u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 36u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 37u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 38u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 39u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 40u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 41u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 42u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 43u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 44u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 45u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 46u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 47u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 48u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 49u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 50u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 51u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 52u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 53u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 54u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 55u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 56u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 57u REG 252,3 96636764160 70 /var/lib/ceph/osd/ceph-3/block 2024-03-04T08:30:42.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 58u a_inode 0,14 0 102305 [io_uring] 2024-03-04T08:30:42.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 59u IPv4 176867 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:41196 (ESTABLISHED) 2024-03-04T08:30:42.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 60r FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 61w FIFO 0,13 0t0 108591 pipe 2024-03-04T08:30:42.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 62r FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 63w FIFO 0,13 0t0 108592 pipe 2024-03-04T08:30:42.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 64u IPv4 182145 0t0 TCP smithi113.front.sepia.ceph.com:34016->smithi113.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 65u IPv4 185386 0t0 TCP smithi113.front.sepia.ceph.com:33522->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 66u IPv4 175507 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48382 (ESTABLISHED) 2024-03-04T08:30:42.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 67u IPv4 187478 0t0 TCP smithi113.front.sepia.ceph.com:58232->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 68u IPv4 188561 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:60402 (ESTABLISHED) 2024-03-04T08:30:42.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 69u IPv4 185387 0t0 TCP smithi113.front.sepia.ceph.com:59538->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 70u IPv4 186690 0t0 TCP smithi113.front.sepia.ceph.com:41786->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 71u IPv4 182146 0t0 TCP smithi113.front.sepia.ceph.com:43506->smithi113.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 72u IPv4 183013 0t0 TCP smithi113.front.sepia.ceph.com:41218->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 73u IPv4 178030 0t0 TCP smithi113.front.sepia.ceph.com:55108->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 74u IPv4 188564 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:57800 (ESTABLISHED) 2024-03-04T08:30:42.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 75u IPv4 182834 0t0 TCP smithi113.front.sepia.ceph.com:41200->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 76u IPv4 186702 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:42416 (ESTABLISHED) 2024-03-04T08:30:42.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 77u IPv4 184940 0t0 TCP smithi113.front.sepia.ceph.com:40246->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 78u IPv4 175971 0t0 TCP smithi113.front.sepia.ceph.com:43138->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 79u IPv4 178031 0t0 TCP smithi113.front.sepia.ceph.com:56426->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 80u IPv4 186575 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:52858 (ESTABLISHED) 2024-03-04T08:30:42.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 81u IPv4 184052 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:32886 (ESTABLISHED) 2024-03-04T08:30:42.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 82u IPv4 184055 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:46752 (ESTABLISHED) 2024-03-04T08:30:42.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 83u IPv4 171714 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:48820 (ESTABLISHED) 2024-03-04T08:30:42.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 84u IPv4 186705 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi113.front.sepia.ceph.com:36276 (ESTABLISHED) 2024-03-04T08:30:42.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 85u IPv4 168827 0t0 TCP smithi113.front.sepia.ceph.com:36390->smithi113.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 86u IPv4 154685 0t0 TCP smithi113.front.sepia.ceph.com:42780->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 87u IPv4 170738 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:38280 (ESTABLISHED) 2024-03-04T08:30:42.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 88u IPv4 170741 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:44856 (ESTABLISHED) 2024-03-04T08:30:42.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 89u IPv4 182651 0t0 TCP smithi113.front.sepia.ceph.com:6810->smithi136.front.sepia.ceph.com:46778 (ESTABLISHED) 2024-03-04T08:30:42.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 90u IPv4 176870 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:35406 (ESTABLISHED) 2024-03-04T08:30:42.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 91u IPv4 186661 0t0 TCP smithi113.front.sepia.ceph.com:39890->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 92u IPv4 180752 0t0 TCP smithi113.front.sepia.ceph.com:56440->smithi136.front.sepia.ceph.com:6818 (ESTABLISHED) 2024-03-04T08:30:42.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 93u IPv4 189590 0t0 TCP smithi113.front.sepia.ceph.com:6812->smithi136.front.sepia.ceph.com:47296 (ESTABLISHED) 2024-03-04T08:30:42.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 94u IPv4 181950 0t0 TCP smithi113.front.sepia.ceph.com:51942->smithi136.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 95u IPv4 182654 0t0 TCP smithi113.front.sepia.ceph.com:51272->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 96u IPv4 178510 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:60100 (ESTABLISHED) 2024-03-04T08:30:42.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 97u IPv4 189593 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi136.front.sepia.ceph.com:51640 (ESTABLISHED) 2024-03-04T08:30:42.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 98u IPv4 173270 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:44992 (ESTABLISHED) 2024-03-04T08:30:42.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 99u IPv4 186556 0t0 TCP smithi113.front.sepia.ceph.com:44866->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 100u IPv4 170199 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi136.front.sepia.ceph.com:35268 (ESTABLISHED) 2024-03-04T08:30:42.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 101u IPv4 180803 0t0 TCP smithi113.front.sepia.ceph.com:34882->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 102u IPv4 175215 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:48700 (ESTABLISHED) 2024-03-04T08:30:42.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 103u IPv4 180295 0t0 TCP smithi113.front.sepia.ceph.com:6808->smithi113.front.sepia.ceph.com:54628 (ESTABLISHED) 2024-03-04T08:30:42.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 104u IPv4 186572 0t0 TCP smithi113.front.sepia.ceph.com:6814->smithi113.front.sepia.ceph.com:59058 (ESTABLISHED) 2024-03-04T08:30:42.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27545 30905 osd_srv_h root 106u IPv4 160497 0t0 TCP smithi113.front.sepia.ceph.com:38502->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27561 log root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27563 msgr-work root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27564 msgr-work root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27565 msgr-work root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27581 service root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27584 admin_soc root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27642 signal_ha root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27643 io_contex root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27644 io_contex root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27660 OpHistory root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27661 ceph_time root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27693 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27704 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 14w REG 8,1 566493493 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27716 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 14w REG 8,1 566496105 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27729 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 14w REG 8,1 566496105 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27741 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 14w REG 8,1 566496105 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27753 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 14w REG 8,1 566496105 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27765 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 14w REG 8,1 566496105 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27777 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 14w REG 8,1 566496105 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27778 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27779 safe_time root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27780 safe_time root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27781 safe_time root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27782 safe_time root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27783 fn_anonym root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:30:42.854 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-03-04T08:30:42.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27788 rocksdb:l root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27789 rocksdb:l root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27790 rocksdb:l root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 27791 rocksdb:h root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:42.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:42.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:42.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:42.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:42.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:42.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:42.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:42.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:42.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:42.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:42.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:42.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:42.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:42.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:42.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:42.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:42.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:42.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:42.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:42.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:42.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:42.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:42.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:42.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:42.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:42.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:42.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:42.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:42.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:42.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:42.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:42.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:42.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:42.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:42.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:42.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:42.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:42.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:42.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29289 bstore_ai root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:42.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:42.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:42.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:42.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:42.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:42.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:42.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:42.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:42.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:42.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:42.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:42.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:42.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:42.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:42.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:42.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:42.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:42.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:42.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:42.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:42.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:42.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:42.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:42.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:42.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:42.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:42.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:42.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:42.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:42.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:42.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:42.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:42.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:42.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:42.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:42.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:42.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:42.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:42.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:42.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:42.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:42.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:42.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:42.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:42.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:42.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:42.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:42.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:42.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:42.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:42.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:42.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:42.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:42.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:42.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:42.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:42.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:42.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:42.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:42.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:42.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:42.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:42.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:42.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:42.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:42.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:42.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:42.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:42.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:42.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:42.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:42.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:42.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:42.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:42.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:42.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:42.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:42.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:42.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:42.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:42.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:42.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:42.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:42.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:42.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:42.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:42.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:42.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:42.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:42.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:42.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:42.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:42.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:42.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:42.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:42.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:42.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:42.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:42.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:42.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:42.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:42.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:42.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:42.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:42.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:42.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:42.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:42.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:42.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 29290 bstore_di root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30025 bstore_ai root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.046 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.047 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.048 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.049 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.050 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.051 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30026 bstore_di root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.052 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.053 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.054 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.055 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.056 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.057 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.058 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.059 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30214 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.097 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.098 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30223 cfin root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30225 bstore_kv root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30226 bstore_kv root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30228 bstore_me root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30394 ms_dispat root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30395 ms_local root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30396 safe_time root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30397 safe_time root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30398 ms_dispat root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30399 ms_local root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.302 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30400 ms_dispat root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30401 ms_local root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.360 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.361 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.362 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.363 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.364 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.365 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.366 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30402 ms_dispat root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30403 ms_local root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30404 ms_dispat root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30405 ms_local root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.448 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.449 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30406 ms_dispat root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30407 ms_local root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30408 ms_dispat root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.505 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.506 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30409 ms_local root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30410 fn_anonym root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30411 finisher root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30412 safe_time root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30413 safe_time root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.635 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.636 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30414 ceph-osd root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.637 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.638 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.639 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.640 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.641 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.642 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.643 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30415 osd_srv_a root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30416 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.682 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.683 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.684 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.685 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.687 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30417 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.710 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.711 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30418 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30419 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30420 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30421 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30422 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30423 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.862 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30424 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30425 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30426 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30427 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30428 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:43.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:43.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:43.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:43.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:43.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:43.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:43.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:43.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:43.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:43.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30429 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:43.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:43.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:43.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:43.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:43.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:43.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:43.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:43.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:43.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:43.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:43.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:43.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:43.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:43.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:43.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:43.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:43.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:43.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:43.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:43.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:43.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:43.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:43.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:43.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:43.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:43.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:43.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:43.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:43.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:43.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:43.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:43.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:43.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:43.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:43.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:43.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:43.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:43.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:43.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:43.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:43.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:43.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:43.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:43.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:43.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:43.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:43.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:43.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:43.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:43.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:43.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:43.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:43.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:43.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:43.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:43.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:43.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:43.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:43.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:43.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:43.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:43.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:43.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:43.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:43.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:43.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:43.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:43.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:43.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:43.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:43.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:43.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:43.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:43.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:43.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:43.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:43.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:43.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:43.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:43.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:43.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:43.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:43.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:43.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:43.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:43.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:43.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:43.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:43.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:43.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:43.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:43.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:43.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:43.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:43.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:43.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:43.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:43.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:43.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:43.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:43.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:43.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:43.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:43.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:43.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:43.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:43.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:43.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:43.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:43.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:43.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:43.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:43.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:43.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:43.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:43.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:43.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:43.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:43.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:43.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:44.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:44.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:44.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:44.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:44.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:44.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:44.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:44.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:44.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:44.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30430 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:44.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:44.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:44.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:44.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:44.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:44.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:44.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:44.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:44.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:44.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:44.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:44.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:44.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:44.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:44.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:44.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:44.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:44.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:44.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:44.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:44.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:44.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:44.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:44.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:44.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:44.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:44.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:44.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:44.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:44.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:44.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:44.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:44.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.010 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:44.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:44.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:44.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:44.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:44.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:44.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.011 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:44.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:44.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:44.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:44.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:44.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:44.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:44.012 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:44.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:44.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:44.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:44.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:44.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:44.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:44.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:44.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:44.013 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:44.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:44.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:44.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:44.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:44.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:44.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:44.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:44.014 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.015 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.016 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:44.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:44.017 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:44.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:44.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:44.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:44.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:44.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:44.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:44.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:44.018 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:44.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:44.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:44.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:44.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:44.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:44.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:44.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:44.019 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:44.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:44.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:44.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:44.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:44.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:44.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:44.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.020 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:44.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:44.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:44.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:44.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:44.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:44.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:44.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:44.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:44.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:44.021 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:44.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:44.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:44.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:44.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:44.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:44.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:44.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:44.022 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:44.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:44.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30431 tp_osd_tp root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:44.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root txt REG 8,1 24724304 80344 /usr/bin/ceph-osd 2024-03-04T08:30:44.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 113624 664165 /usr/lib/rados-classes/libcls_rgw_gc.so.1.0.0 2024-03-04T08:30:44.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 1285744 664164 /usr/lib/rados-classes/libcls_rgw.so.1.0.0 2024-03-04T08:30:44.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 43800 664168 /usr/lib/rados-classes/libcls_timeindex.so.1.0.0 2024-03-04T08:30:44.023 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 68688 664150 /usr/lib/rados-classes/libcls_cas.so.1.0.0 2024-03-04T08:30:44.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 216168 664153 /usr/lib/rados-classes/libcls_fifo.so.1.0.0 2024-03-04T08:30:44.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 68400 664169 /usr/lib/rados-classes/libcls_user.so.1.0.0 2024-03-04T08:30:44.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 84992 664161 /usr/lib/rados-classes/libcls_queue.so.1.0.0 2024-03-04T08:30:44.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 35648 664154 /usr/lib/rados-classes/libcls_hello.so.1.0.0 2024-03-04T08:30:44.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 39688 664151 /usr/lib/rados-classes/libcls_cephfs.so.1.0.0 2024-03-04T08:30:44.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 117792 664149 /usr/lib/rados-classes/libcls_2pc_queue.so.1.0.0 2024-03-04T08:30:44.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 806216 664163 /usr/lib/rados-classes/libcls_refcount.so.1.0.0 2024-03-04T08:30:44.024 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 72688 664156 /usr/lib/rados-classes/libcls_lock.so.1.0.0 2024-03-04T08:30:44.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 64384 664152 /usr/lib/rados-classes/libcls_cmpomap.so.1.0.0 2024-03-04T08:30:44.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 396768 664162 /usr/lib/rados-classes/libcls_rbd.so.1.0.0 2024-03-04T08:30:44.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 47920 664157 /usr/lib/rados-classes/libcls_log.so.1.0.0 2024-03-04T08:30:44.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 80872 664155 /usr/lib/rados-classes/libcls_journal.so.1.0.0 2024-03-04T08:30:44.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 67736 50692 /usr/lib/x86_64-linux-gnu/liboath.so.0.1.3 2024-03-04T08:30:44.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 27352 664159 /usr/lib/rados-classes/libcls_numops.so.1.0.0 2024-03-04T08:30:44.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 68520 664160 /usr/lib/rados-classes/libcls_otp.so.1.0.0 2024-03-04T08:30:44.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 1392696 664141 /usr/lib/ceph/erasure-code/libec_lrc.so 2024-03-04T08:30:44.025 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 339328 664137 /usr/lib/ceph/erasure-code/libec_jerasure.so 2024-03-04T08:30:44.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 35560 664170 /usr/lib/rados-classes/libcls_version.so.1.0.0 2024-03-04T08:30:44.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root DEL REG 0,18 106891 /[aio] 2024-03-04T08:30:44.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root DEL REG 0,18 106889 /[aio] 2024-03-04T08:30:44.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 64464 664147 /usr/lib/ceph/extblkdev/libceph_ebd_vdo.so 2024-03-04T08:30:44.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 31432 664093 /usr/lib/ceph/compressor/libceph_snappy.so.2.0.0 2024-03-04T08:30:44.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.026 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.027 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.028 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.029 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.030 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:44.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.031 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 125152 8837 /usr/lib/x86_64-linux-gnu/liblz4.so.1.9.3 2024-03-04T08:30:44.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 35184 16894 /usr/lib/x86_64-linux-gnu/libsnappy.so.1.1.8 2024-03-04T08:30:44.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 14456 12608 /usr/lib/x86_64-linux-gnu/libaio.so.1.0.1 2024-03-04T08:30:44.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 260376 4930 /usr/lib/x86_64-linux-gnu/libfuse.so.2.9.9 2024-03-04T08:30:44.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 0,14 106879 anon_inode:[io_uring] (stat: No such file or directory) 2024-03-04T08:30:44.032 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 1w FIFO 0,13 0t0 104670 pipe 2024-03-04T08:30:44.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 2w FIFO 0,13 0t0 104671 pipe 2024-03-04T08:30:44.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 3r FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:44.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 4w FIFO 0,13 0t0 103288 pipe 2024-03-04T08:30:44.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 6r FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:44.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 7w FIFO 0,13 0t0 103289 pipe 2024-03-04T08:30:44.033 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 9r FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:44.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 10w FIFO 0,13 0t0 103290 pipe 2024-03-04T08:30:44.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 12r FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:44.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 13w FIFO 0,13 0t0 103291 pipe 2024-03-04T08:30:44.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 14w REG 8,1 566498554 677282 /var/log/ceph/ceph-osd.2.log 2024-03-04T08:30:44.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 15uW REG 0,23 6 3571 /run/ceph/ceph-osd.2.pid 2024-03-04T08:30:44.034 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 16r FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:44.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 17w FIFO 0,13 0t0 106563 pipe 2024-03-04T08:30:44.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 18u unix 0xffff8aaf036ada00 0t0 106564 /var/run/ceph/ceph-osd.2.asok type=STREAM 2024-03-04T08:30:44.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 19u IPv4 106651 0t0 TCP *:6815 (LISTEN) 2024-03-04T08:30:44.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 20u IPv4 106669 0t0 TCP *:6817 (LISTEN) 2024-03-04T08:30:44.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 21u IPv4 106688 0t0 TCP *:6818 (LISTEN) 2024-03-04T08:30:44.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 22u IPv4 106708 0t0 TCP *:6819 (LISTEN) 2024-03-04T08:30:44.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 23u IPv4 106729 0t0 TCP *:6820 (LISTEN) 2024-03-04T08:30:44.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 24u IPv4 106751 0t0 TCP *:6821 (LISTEN) 2024-03-04T08:30:44.035 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 25u IPv4 106774 0t0 TCP *:6822 (LISTEN) 2024-03-04T08:30:44.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 26u IPv4 106798 0t0 TCP *:6823 (LISTEN) 2024-03-04T08:30:44.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 27r FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:44.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 28w FIFO 0,13 0t0 108557 pipe 2024-03-04T08:30:44.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 29r FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:44.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 30w FIFO 0,13 0t0 108558 pipe 2024-03-04T08:30:44.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 31u sock 0,8 0t0 106876 protocol: UDPv6 2024-03-04T08:30:44.036 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 32r DIR 252,2 357 64 /var/lib/ceph/osd/ceph-2 2024-03-04T08:30:44.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 33uW REG 252,2 37 69 /var/lib/ceph/osd/ceph-2/fsid 2024-03-04T08:30:44.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 34u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 35u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 36u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 37u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 38u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 39u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 40u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.037 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 41u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 42u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 43u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 44u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 45u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 46u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 47u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 48u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 49u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.038 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 50u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 51u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 52u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 53u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 54u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 55u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 56u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 57u REG 252,2 96636764160 70 /var/lib/ceph/osd/ceph-2/block 2024-03-04T08:30:44.039 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 58u a_inode 0,14 0 106879 [io_uring] 2024-03-04T08:30:44.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 59r FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:44.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 60u IPv4 103336 0t0 TCP smithi113.front.sepia.ceph.com:33156->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 61w FIFO 0,13 0t0 104747 pipe 2024-03-04T08:30:44.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 62r FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:44.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 63w FIFO 0,13 0t0 104748 pipe 2024-03-04T08:30:44.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 64u IPv4 175993 0t0 TCP smithi113.front.sepia.ceph.com:38612->smithi113.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:44.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 65u IPv4 180849 0t0 TCP smithi113.front.sepia.ceph.com:60582->smithi113.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:44.040 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 66u IPv4 163031 0t0 TCP smithi113.front.sepia.ceph.com:60016->smithi136.front.sepia.ceph.com:6830 (ESTABLISHED) 2024-03-04T08:30:44.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 67u IPv4 187685 0t0 TCP smithi113.front.sepia.ceph.com:51910->smithi136.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:44.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 68u IPv4 186521 0t0 TCP smithi113.front.sepia.ceph.com:59498->smithi136.front.sepia.ceph.com:6822 (ESTABLISHED) 2024-03-04T08:30:44.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 69u IPv4 186685 0t0 TCP smithi113.front.sepia.ceph.com:53858->smithi136.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:44.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 70u IPv4 163032 0t0 TCP smithi113.front.sepia.ceph.com:54478->smithi136.front.sepia.ceph.com:6828 (ESTABLISHED) 2024-03-04T08:30:44.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 71u IPv4 182883 0t0 TCP smithi113.front.sepia.ceph.com:58508->smithi136.front.sepia.ceph.com:6820 (ESTABLISHED) 2024-03-04T08:30:44.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 72u IPv4 186613 0t0 TCP smithi113.front.sepia.ceph.com:35438->smithi136.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:44.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 73u IPv4 171484 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi113.front.sepia.ceph.com:36390 (ESTABLISHED) 2024-03-04T08:30:44.041 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 74u IPv4 184787 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:51522 (ESTABLISHED) 2024-03-04T08:30:44.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 75u IPv4 184790 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:60348 (ESTABLISHED) 2024-03-04T08:30:44.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 76u IPv4 185550 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:52494 (ESTABLISHED) 2024-03-04T08:30:44.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 77u IPv4 185553 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:33734 (ESTABLISHED) 2024-03-04T08:30:44.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 78u IPv4 183367 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:32856 (ESTABLISHED) 2024-03-04T08:30:44.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 79u IPv4 173007 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:49516 (ESTABLISHED) 2024-03-04T08:30:44.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 80u IPv4 173739 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:55730 (ESTABLISHED) 2024-03-04T08:30:44.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 81u IPv4 183364 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:39350 (ESTABLISHED) 2024-03-04T08:30:44.042 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 82u IPv4 173880 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi136.front.sepia.ceph.com:50332 (ESTABLISHED) 2024-03-04T08:30:44.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 83u IPv4 173883 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi136.front.sepia.ceph.com:45924 (ESTABLISHED) 2024-03-04T08:30:44.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 84u IPv4 166353 0t0 TCP smithi113.front.sepia.ceph.com:53106->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 85u IPv4 188595 0t0 TCP smithi113.front.sepia.ceph.com:58402->smithi113.front.sepia.ceph.com:6804 (ESTABLISHED) 2024-03-04T08:30:44.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 86u IPv4 189507 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:34016 (ESTABLISHED) 2024-03-04T08:30:44.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 87u IPv4 181417 0t0 TCP smithi113.front.sepia.ceph.com:6822->smithi113.front.sepia.ceph.com:38906 (ESTABLISHED) 2024-03-04T08:30:44.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 88u IPv4 186763 0t0 TCP smithi113.front.sepia.ceph.com:36874->smithi113.front.sepia.ceph.com:6806 (ESTABLISHED) 2024-03-04T08:30:44.043 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 89u IPv4 187827 0t0 TCP smithi113.front.sepia.ceph.com:39888->smithi136.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:44.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 90u IPv4 182922 0t0 TCP smithi113.front.sepia.ceph.com:52858->smithi113.front.sepia.ceph.com:6812 (ESTABLISHED) 2024-03-04T08:30:44.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 91u IPv4 173010 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:49128 (ESTABLISHED) 2024-03-04T08:30:44.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 92u IPv4 170467 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:40938 (ESTABLISHED) 2024-03-04T08:30:44.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 93u IPv4 181420 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43866 (ESTABLISHED) 2024-03-04T08:30:44.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 94u IPv4 184802 0t0 TCP smithi113.front.sepia.ceph.com:59058->smithi113.front.sepia.ceph.com:6814 (ESTABLISHED) 2024-03-04T08:30:44.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 95u IPv4 189510 0t0 TCP smithi113.front.sepia.ceph.com:6820->smithi113.front.sepia.ceph.com:43506 (ESTABLISHED) 2024-03-04T08:30:44.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 96u IPv4 170574 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:55722 (ESTABLISHED) 2024-03-04T08:30:44.044 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 97u IPv4 181087 0t0 TCP smithi113.front.sepia.ceph.com:52000->smithi113.front.sepia.ceph.com:6826 (ESTABLISHED) 2024-03-04T08:30:44.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 98u IPv4 175452 0t0 TCP smithi113.front.sepia.ceph.com:38034->smithi113.front.sepia.ceph.com:6802 (ESTABLISHED) 2024-03-04T08:30:44.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 99u IPv4 186129 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:52964 (ESTABLISHED) 2024-03-04T08:30:44.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 100u IPv4 173453 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:40700 (ESTABLISHED) 2024-03-04T08:30:44.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 101u IPv4 166853 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:55894 (ESTABLISHED) 2024-03-04T08:30:44.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 102u IPv4 171450 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:48214 (ESTABLISHED) 2024-03-04T08:30:44.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 103u IPv4 176739 0t0 TCP smithi113.front.sepia.ceph.com:6818->smithi136.front.sepia.ceph.com:38202 (ESTABLISHED) 2024-03-04T08:30:44.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 105u IPv4 175812 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi113.front.sepia.ceph.com:52502 (ESTABLISHED) 2024-03-04T08:30:44.045 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-osd 27546 30432 osd_srv_h root 106u IPv4 181306 0t0 TCP smithi113.front.sepia.ceph.com:6815->smithi136.front.sepia.ceph.com:46642 (ESTABLISHED) 2024-03-04T08:30:44.045 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 31391 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.046 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 31391 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.046 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 31391 root txt unknown /proc/31391/exe 2024-03-04T08:30:44.046 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 33557 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.046 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 33557 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.046 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 33557 root txt unknown /proc/33557/exe 2024-03-04T08:30:44.046 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 33570 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.046 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 33570 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.046 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 33570 root txt unknown /proc/33570/exe 2024-03-04T08:30:44.046 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 33571 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.047 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 33571 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.047 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/R 33571 root txt unknown /proc/33571/exe 2024-03-04T08:30:44.047 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 42259 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.047 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 42259 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.047 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 42259 root txt unknown /proc/42259/exe 2024-03-04T08:30:44.047 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 44042 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.047 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 44042 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.047 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 44042 root txt unknown /proc/44042/exe 2024-03-04T08:30:44.048 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 45156 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.048 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 45156 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.048 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/4 45156 root txt unknown /proc/45156/exe 2024-03-04T08:30:44.048 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 45873 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.048 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 45873 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.048 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/3 45873 root txt unknown /proc/45873/exe 2024-03-04T08:30:44.048 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 45884 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.048 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 45884 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.048 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 45884 root txt unknown /proc/45884/exe 2024-03-04T08:30:44.049 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 45885 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.049 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 45885 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.049 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 45885 root txt unknown /proc/45885/exe 2024-03-04T08:30:44.049 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 46444 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.049 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 46444 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.049 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/1 46444 root txt unknown /proc/46444/exe 2024-03-04T08:30:44.049 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 46451 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.049 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 46451 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.050 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 46451 root txt unknown /proc/46451/exe 2024-03-04T08:30:44.050 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 46993 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.050 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 46993 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.050 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 46993 root txt unknown /proc/46993/exe 2024-03-04T08:30:44.050 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 47011 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.050 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 47011 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.050 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 47011 root txt unknown /proc/47011/exe 2024-03-04T08:30:44.050 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 47731 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.051 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 47731 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.051 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 47731 root txt unknown /proc/47731/exe 2024-03-04T08:30:44.051 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 47738 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.051 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 47738 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.051 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 47738 root txt unknown /proc/47738/exe 2024-03-04T08:30:44.051 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 47739 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.051 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 47739 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.051 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/6 47739 root txt unknown /proc/47739/exe 2024-03-04T08:30:44.051 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 48291 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.052 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 48291 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.052 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 48291 root txt unknown /proc/48291/exe 2024-03-04T08:30:44.052 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 49946 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.052 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 49946 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.052 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/2 49946 root txt unknown /proc/49946/exe 2024-03-04T08:30:44.052 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 50632 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.052 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 50632 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.052 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/0 50632 root txt unknown /proc/50632/exe 2024-03-04T08:30:44.053 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.053 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.053 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root txt REG 8,1 232416 12806 /usr/bin/sudo 2024-03-04T08:30:44.053 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:44.053 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:44.053 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:44.053 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.053 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.053 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.054 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:44.054 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:44.054 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:44.054 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.054 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.054 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:44.054 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 516992 95600 /usr/libexec/sudo/sudoers.so 2024-03-04T08:30:44.054 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:44.054 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:44.055 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.055 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:44.055 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.055 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.055 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:44.055 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:44.055 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:44.055 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:44.056 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.056 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:44.056 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:44.056 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:44.056 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 130040 95595 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-03-04T08:30:44.056 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:44.056 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:44.056 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:44.057 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.057 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root 0r FIFO 0,13 0t0 171256 pipe 2024-03-04T08:30:44.057 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.057 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.057 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root 3r FIFO 0,13 0t0 169344 pipe 2024-03-04T08:30:44.057 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root 4w FIFO 0,13 0t0 169344 pipe 2024-03-04T08:30:44.057 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root 5r REG 8,1 1698 658119 /etc/sudoers 2024-03-04T08:30:44.057 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root 6u netlink 0t0 169353 AUDIT 2024-03-04T08:30:44.058 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 50782 root 7u unix 0xffff8aaf0dbd0000 0t0 169354 type=DGRAM 2024-03-04T08:30:44.058 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.058 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.058 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:44.058 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.058 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:44.058 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:44.058 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:44.058 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:44.059 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.059 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:44.059 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.059 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:44.059 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.059 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:44.059 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.059 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root 0r FIFO 0,13 0t0 171256 pipe 2024-03-04T08:30:44.060 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.060 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 50784 root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.060 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.061 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.062 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.063 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.064 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.065 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.066 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.067 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.068 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.069 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.070 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.071 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.072 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.073 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.074 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.075 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.076 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.078 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.079 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.080 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50790 log root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.081 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.082 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.083 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.084 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.085 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.086 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.087 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.088 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.089 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.090 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.091 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50792 msgr-work root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.092 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.093 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.094 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.095 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.096 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.099 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.100 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.101 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.102 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.103 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.104 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50793 msgr-work root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.105 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.106 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.107 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.108 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.109 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.110 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.111 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.112 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.113 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.114 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50794 msgr-work root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.115 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.116 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.117 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.118 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.119 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.120 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.121 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.122 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.123 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.124 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.125 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50798 service root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.126 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.127 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.128 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.129 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.130 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.131 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.132 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.133 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.134 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.135 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50799 admin_soc root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.136 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.137 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.138 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.139 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.140 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.141 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.142 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.143 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.144 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.145 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.146 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50800 signal_ha root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.147 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.148 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.149 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.150 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.151 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.152 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.153 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.154 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.155 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.156 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50801 io_contex root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.157 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.158 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.159 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.160 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.161 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.162 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.163 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.164 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.165 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.166 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50802 io_contex root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.167 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.168 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.169 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.170 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.171 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.172 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.173 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.174 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.175 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.176 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50803 ms_dispat root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.177 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.178 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.179 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.180 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.181 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.182 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.183 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.184 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.185 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.186 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.187 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50804 ms_local root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.188 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.189 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.190 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.191 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.192 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.193 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.194 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.195 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.196 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.197 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.198 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50805 safe_time root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.199 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.200 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.201 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.202 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.203 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.204 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.205 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.206 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.207 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.208 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50806 safe_time root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.209 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.210 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.211 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.212 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.213 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.214 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.215 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.216 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.217 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.218 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.219 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50807 ceph-mds root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.220 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.221 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.222 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.223 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.224 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.225 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.226 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.227 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.228 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.229 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 50830 safe_time root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.230 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.231 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.232 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.233 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.234 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.235 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.236 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.237 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.238 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.239 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.240 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51228 ceph_time root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.241 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.242 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.243 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.244 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.245 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.246 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.247 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.248 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.249 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.250 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51229 OpHistory root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.251 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.252 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.253 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.254 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.255 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.256 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.257 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.258 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.259 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.260 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.261 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51230 ms_dispat root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.262 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.263 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.264 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.265 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.266 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.267 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.268 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.269 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.270 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.271 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.272 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51231 ms_dispat root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.273 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.274 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.275 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.276 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.277 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.278 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.279 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.280 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.281 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.282 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51232 mds_rank_ root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.283 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.284 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.285 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.286 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.287 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.288 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.289 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.290 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.291 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.292 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.293 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51233 PQ_Finish root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.294 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.295 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.296 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.297 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.298 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.299 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.300 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.301 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.303 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.304 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.305 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.306 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51234 safe_time root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.307 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.308 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.309 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.310 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.311 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.312 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.313 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.314 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.315 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.316 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51235 MR_Finish root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.317 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.318 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.319 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.320 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.321 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.322 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.323 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.324 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.325 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.326 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.327 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51315 md_submit root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.328 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.329 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.330 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.332 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.333 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.334 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.335 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.336 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.337 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51525 ms_dispat root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.338 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.339 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.340 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.341 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.342 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.343 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.344 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.345 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.346 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.347 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.348 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51526 quiesce.a root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.349 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.350 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.351 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.352 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.353 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.354 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 1w FIFO 0,13 0t0 171257 pipe 2024-03-04T08:30:44.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 2w FIFO 0,13 0t0 171258 pipe 2024-03-04T08:30:44.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 3r FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.355 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 4w FIFO 0,13 0t0 168592 pipe 2024-03-04T08:30:44.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 6r FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 7w FIFO 0,13 0t0 168593 pipe 2024-03-04T08:30:44.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 9r FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 10w FIFO 0,13 0t0 168594 pipe 2024-03-04T08:30:44.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 12r FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.356 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 13w FIFO 0,13 0t0 168595 pipe 2024-03-04T08:30:44.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 14w REG 8,1 3413791722 677293 /var/log/ceph/ceph-mds.a.log 2024-03-04T08:30:44.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 15uW REG 0,23 6 3589 /run/ceph/ceph-mds.a.pid 2024-03-04T08:30:44.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 16r FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 17w FIFO 0,13 0t0 168597 pipe 2024-03-04T08:30:44.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 18u unix 0xffff8aaf7d7f1e00 0t0 168598 /var/run/ceph/ceph-mds.a.asok type=STREAM 2024-03-04T08:30:44.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 19u IPv4 173222 0t0 TCP *:6834 (LISTEN) 2024-03-04T08:30:44.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 20u IPv4 173259 0t0 TCP *:6836 (LISTEN) 2024-03-04T08:30:44.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 21r FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.357 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 22w FIFO 0,13 0t0 167861 pipe 2024-03-04T08:30:44.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 23r FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 24w FIFO 0,13 0t0 167862 pipe 2024-03-04T08:30:44.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 25r FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 26w FIFO 0,13 0t0 168600 pipe 2024-03-04T08:30:44.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 28r FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 29w FIFO 0,13 0t0 168601 pipe 2024-03-04T08:30:44.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 31u IPv4 174494 0t0 TCP smithi113.front.sepia.ceph.com:44266->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.358 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 32u IPv4 170664 0t0 TCP smithi113.front.sepia.ceph.com:48700->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 33u IPv4 175214 0t0 TCP smithi113.front.sepia.ceph.com:40700->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 34u IPv4 172449 0t0 TCP smithi113.front.sepia.ceph.com:46404->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 35u IPv4 170665 0t0 TCP smithi113.front.sepia.ceph.com:37820->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 36u IPv4 170666 0t0 TCP smithi113.front.sepia.ceph.com:54778->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 37u IPv4 175218 0t0 TCP smithi113.front.sepia.ceph.com:41410->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.359 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 50789 51527 quiesce_d root 41u IPv4 170708 0t0 TCP smithi113.front.sepia.ceph.com:33866->smithi113.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.359 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 50914 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:44.359 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 50914 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.359 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 50914 root txt unknown /proc/50914/exe 2024-03-04T08:30:44.360 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.360 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.360 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root txt REG 8,1 232416 12806 /usr/bin/sudo 2024-03-04T08:30:44.360 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:44.360 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:44.360 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:44.360 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.360 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.361 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.361 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.361 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:44.361 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:44.361 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.361 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:44.361 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:44.361 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:44.361 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.362 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.362 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:44.362 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:44.362 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:44.362 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.362 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:44.362 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:44.362 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:44.363 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.363 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:44.363 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 516992 95600 /usr/libexec/sudo/sudoers.so 2024-03-04T08:30:44.363 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:44.363 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:44.363 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 130040 95595 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-03-04T08:30:44.363 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:44.363 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:44.364 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:44.364 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.364 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root 0r FIFO 0,13 0t0 168872 pipe 2024-03-04T08:30:44.364 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.364 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.364 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root 3r FIFO 0,13 0t0 169740 pipe 2024-03-04T08:30:44.364 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root 4w FIFO 0,13 0t0 169740 pipe 2024-03-04T08:30:44.364 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root 5r REG 8,1 1698 658119 /etc/sudoers 2024-03-04T08:30:44.365 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root 6u netlink 0t0 169749 AUDIT 2024-03-04T08:30:44.365 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 51528 root 7u unix 0xffff8aaf0dbd1e00 0t0 169750 type=DGRAM 2024-03-04T08:30:44.365 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.365 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.365 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:44.365 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.365 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:44.365 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:44.365 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.366 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:44.366 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:44.366 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:44.366 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.366 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:44.366 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.366 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:44.366 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.366 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root 0r FIFO 0,13 0t0 168872 pipe 2024-03-04T08:30:44.367 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.367 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 51530 root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.367 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.368 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.369 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.370 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.371 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.372 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.373 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 14w REG 8,1 2989759007 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.377 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.378 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.379 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.380 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.381 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.383 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.384 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.385 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.386 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.387 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51536 log root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.388 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.389 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.390 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.391 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.392 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.393 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.394 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.395 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.396 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.397 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.398 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51538 msgr-work root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.399 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.400 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.401 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.402 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.403 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.404 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.405 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.406 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.407 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.408 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51539 msgr-work root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.409 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.410 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.411 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.412 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.413 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.414 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.415 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.416 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.417 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.418 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51540 msgr-work root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.419 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.420 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.421 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.422 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.423 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.424 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.425 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.426 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.427 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.428 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51544 service root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.430 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.431 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.432 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.433 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.434 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.435 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.436 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.437 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.438 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.439 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.440 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51545 admin_soc root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.441 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.442 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.443 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.444 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.445 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.446 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.447 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.450 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.451 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.452 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.453 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.454 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51546 signal_ha root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.455 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.456 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.457 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.458 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.459 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.460 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.461 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.462 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.463 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.464 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.465 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51547 io_contex root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.466 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.467 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.468 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.469 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.470 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.471 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.472 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.473 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.474 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.475 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.476 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51548 io_contex root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.477 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.478 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.479 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.480 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.481 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.482 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.483 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.484 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.485 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.486 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.487 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51549 ms_dispat root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.488 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.489 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.490 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.491 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.492 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.493 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.494 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.495 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.496 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.497 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51550 ms_local root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.498 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.499 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.500 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.501 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.502 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.503 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.504 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.507 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.508 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.509 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.510 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.511 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51551 safe_time root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.512 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.513 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.514 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.515 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.516 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.517 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.518 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 14w REG 8,1 2989758394 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.519 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.520 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.521 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51552 safe_time root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.522 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.523 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.524 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.525 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.526 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.527 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.528 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.529 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.530 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.531 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51553 ceph-mds root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.532 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.533 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.534 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.535 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.536 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.537 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.538 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.539 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.540 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.541 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 51574 safe_time root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.542 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.543 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.544 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.545 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.546 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.547 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.548 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.549 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.550 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.551 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.552 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52020 ceph_time root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.553 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.554 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.555 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.556 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.557 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.558 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.559 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.560 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.561 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.562 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52021 OpHistory root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.563 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.564 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.565 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.566 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.567 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.568 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.570 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.571 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.572 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.573 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52022 ms_dispat root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.574 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.575 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.576 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.577 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.578 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.579 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.580 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.581 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.582 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.583 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52023 ms_dispat root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.584 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.585 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.586 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.587 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.588 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.589 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.590 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.591 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.592 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.593 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.594 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52024 mds_rank_ root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.595 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.596 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.597 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.598 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.600 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.601 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.602 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.603 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.604 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.605 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52025 PQ_Finish root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.606 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.607 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.608 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.609 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.610 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.611 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.612 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.613 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.614 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.615 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.616 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52026 safe_time root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.617 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.618 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.619 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.620 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.621 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.622 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.623 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.624 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.625 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.626 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52027 MR_Finish root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.627 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.628 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.629 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.630 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.631 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.632 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.635 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-04T08:30:44.635 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T08:30:44.635 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T08:30:44.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.644 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.645 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.646 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.647 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.648 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52097 md_submit root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.649 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.650 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.651 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.652 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.653 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.654 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.655 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.656 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.657 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.658 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.659 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52636 ms_dispat root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.660 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.661 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.662 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.663 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.664 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.665 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.666 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 14w REG 8,1 2989758637 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.668 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.669 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52637 quiesce.a root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.670 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.671 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.672 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.673 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.674 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.675 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.676 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 1w FIFO 0,13 0t0 168873 pipe 2024-03-04T08:30:44.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 2w FIFO 0,13 0t0 168874 pipe 2024-03-04T08:30:44.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 3r FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 4w FIFO 0,13 0t0 169764 pipe 2024-03-04T08:30:44.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 6r FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.677 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 7w FIFO 0,13 0t0 169765 pipe 2024-03-04T08:30:44.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 9r FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 10w FIFO 0,13 0t0 169766 pipe 2024-03-04T08:30:44.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 12r FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 13w FIFO 0,13 0t0 169767 pipe 2024-03-04T08:30:44.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 14w REG 8,1 2989759007 677292 /var/log/ceph/ceph-mds.c.log 2024-03-04T08:30:44.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 15uW REG 0,23 6 3587 /run/ceph/ceph-mds.c.pid 2024-03-04T08:30:44.678 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 16r FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 17w FIFO 0,13 0t0 174618 pipe 2024-03-04T08:30:44.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 18u unix 0xffff8aaf0ad23c00 0t0 174619 /var/run/ceph/ceph-mds.c.asok type=STREAM 2024-03-04T08:30:44.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 19u IPv4 169800 0t0 TCP *:6832 (LISTEN) 2024-03-04T08:30:44.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 20u IPv4 169834 0t0 TCP *:6833 (LISTEN) 2024-03-04T08:30:44.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 21r FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 22w FIFO 0,13 0t0 174621 pipe 2024-03-04T08:30:44.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 23r FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.679 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 24w FIFO 0,13 0t0 174622 pipe 2024-03-04T08:30:44.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 25r FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 27w FIFO 0,13 0t0 169835 pipe 2024-03-04T08:30:44.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 28r FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 29w FIFO 0,13 0t0 169836 pipe 2024-03-04T08:30:44.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 31u IPv4 173738 0t0 TCP smithi113.front.sepia.ceph.com:50356->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 32u IPv4 175505 0t0 TCP smithi113.front.sepia.ceph.com:51062->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.680 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 33u IPv4 171870 0t0 TCP smithi113.front.sepia.ceph.com:55730->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 34u IPv4 175506 0t0 TCP smithi113.front.sepia.ceph.com:59506->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 35u IPv4 174765 0t0 TCP smithi113.front.sepia.ceph.com:48382->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 36u IPv4 174766 0t0 TCP smithi113.front.sepia.ceph.com:50038->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 37u IPv4 175511 0t0 TCP smithi113.front.sepia.ceph.com:40812->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 51535 52638 quiesce_d root 38u IPv4 190549 0t0 TCP smithi113.front.sepia.ceph.com:35254->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.681 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.681 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.681 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root txt REG 8,1 232416 12806 /usr/bin/sudo 2024-03-04T08:30:44.681 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:44.682 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:44.682 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:44.682 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.682 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.682 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.682 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:44.682 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:44.682 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:44.682 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:44.683 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.683 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.683 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:44.683 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:44.683 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:44.683 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.683 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.683 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.684 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:44.684 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:44.684 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:44.684 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:44.684 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.684 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:44.684 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 516992 95600 /usr/libexec/sudo/sudoers.so 2024-03-04T08:30:44.684 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:44.684 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:44.685 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 130040 95595 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-03-04T08:30:44.685 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:44.685 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:44.685 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:44.685 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.685 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root 0r FIFO 0,13 0t0 179462 pipe 2024-03-04T08:30:44.685 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.685 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.685 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root 3r FIFO 0,13 0t0 176446 pipe 2024-03-04T08:30:44.686 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root 4w FIFO 0,13 0t0 176446 pipe 2024-03-04T08:30:44.686 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root 5r REG 8,1 1698 658119 /etc/sudoers 2024-03-04T08:30:44.686 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root 6u netlink 0t0 176455 AUDIT 2024-03-04T08:30:44.686 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 53034 root 7u unix 0xffff8aaf7d7f6180 0t0 176456 type=DGRAM 2024-03-04T08:30:44.686 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.686 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.686 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root txt REG 8,1 5921160 4454 /usr/bin/python3.10 2024-03-04T08:30:44.687 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.687 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:44.687 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 74848 12212 /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4 2024-03-04T08:30:44.687 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 45240 50087 /usr/lib/python3.10/lib-dynload/_lzma.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:44.687 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.687 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 32120 50070 /usr/lib/python3.10/lib-dynload/_bz2.cpython-310-x86_64-linux-gnu.so 2024-03-04T08:30:44.687 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:44.687 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.688 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 194872 8481 /usr/lib/x86_64-linux-gnu/libexpat.so.1.8.7 2024-03-04T08:30:44.688 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.688 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:44.688 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.688 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root 0r FIFO 0,13 0t0 179462 pipe 2024-03-04T08:30:44.688 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.688 INFO:teuthology.orchestra.run.smithi113.stdout:daemon-he 53036 root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.688 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.689 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.690 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.691 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.692 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.693 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.694 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.695 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.696 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.697 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.698 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.699 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.700 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.701 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.702 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.703 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.704 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.705 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.706 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.707 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.708 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.709 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.712 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53042 log root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.713 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.714 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.715 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.716 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.717 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.718 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.719 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.720 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.721 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.722 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.723 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.724 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.725 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.726 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53044 msgr-work root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.727 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.728 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.729 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.730 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.731 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.732 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.733 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.734 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.735 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.736 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.737 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.738 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53045 msgr-work root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.739 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.740 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.741 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.742 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.743 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.744 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.745 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.746 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.747 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.748 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.749 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53046 msgr-work root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.750 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.751 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.752 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.753 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.754 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.755 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.756 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.757 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.758 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.759 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.760 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53050 service root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.761 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.762 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.763 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.764 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.765 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.766 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.767 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.768 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.769 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.770 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.771 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53051 admin_soc root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.772 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.773 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.774 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.775 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.776 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.777 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.778 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.779 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.780 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.781 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.782 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53052 signal_ha root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.783 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.784 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.785 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.786 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.787 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.789 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.790 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.791 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.792 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.793 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.794 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53053 io_contex root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.795 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.796 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.797 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.798 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.799 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.800 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.801 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.802 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.803 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.804 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.805 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53054 io_contex root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.806 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.807 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.808 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.809 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.810 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.811 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.812 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.813 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.814 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.815 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.816 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53055 ms_dispat root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.817 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.818 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.819 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.820 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.821 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.822 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.823 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.824 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.825 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.826 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.827 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53056 ms_local root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.828 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.829 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.830 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.831 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.832 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.833 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.834 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.835 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.836 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.837 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.838 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53057 safe_time root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.839 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.840 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.841 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.842 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.843 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.844 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.845 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.846 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.847 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.848 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.849 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53058 safe_time root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.850 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.851 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.852 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.853 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.854 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.855 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.856 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.857 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.858 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.859 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.860 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.861 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53059 ceph-mds root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.863 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.864 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.865 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.866 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.867 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.868 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.869 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.870 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.871 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.872 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.873 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.874 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53081 safe_time root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.875 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.876 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.877 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.878 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.879 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.880 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.881 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.882 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.883 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.885 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.886 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53320 ceph_time root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.887 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.888 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.889 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.890 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.891 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.892 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.893 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.894 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.895 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.896 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.897 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53321 OpHistory root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.898 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.899 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.900 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.901 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.902 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.903 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.904 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.905 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.906 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.907 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.908 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53322 ms_dispat root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.909 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.910 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.911 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.912 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.913 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.914 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.915 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.916 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.917 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.918 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53323 ms_dispat root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.921 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.922 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.923 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.924 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.925 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.926 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.927 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.928 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.929 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.930 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.931 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53324 mds_rank_ root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.932 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.933 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.934 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.935 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.936 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.937 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.938 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.939 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.940 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.941 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.942 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53325 PQ_Finish root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.943 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.944 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.945 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.946 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.947 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.948 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.949 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.950 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.951 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.952 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.953 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53326 safe_time root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.954 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.955 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.956 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.957 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.958 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.959 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.960 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.961 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.962 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.963 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.964 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53327 MR_Finish root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.965 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.966 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.967 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.968 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.969 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.970 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.971 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.972 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.973 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.974 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.975 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53407 md_submit root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.976 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.977 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.978 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.979 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.980 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.981 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.982 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.983 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.984 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.985 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.986 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53438 ms_dispat root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.987 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.988 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:44.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:44.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:44.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:44.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:44.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:44.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:44.989 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:44.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:44.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:44.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:44.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:44.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:44.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:44.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:44.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:44.990 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:44.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:44.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:44.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:44.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:44.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:44.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:44.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:44.991 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:44.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:44.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:44.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:44.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:44.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:44.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:44.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:44.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:44.992 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:44.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:44.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:44.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:44.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:44.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:44.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:44.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:44.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:44.993 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:44.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:44.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:44.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.994 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:44.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:44.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:44.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:44.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:44.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:44.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:44.995 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:44.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:44.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:44.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:44.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:44.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.996 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:44.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:44.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:44.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:44.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:44.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:44.997 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:44.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:44.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:44.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:44.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53439 quiesce.a root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:44.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:44.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:44.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root txt REG 8,1 7005960 665485 /usr/bin/ceph-mds 2024-03-04T08:30:44.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 51896 5507 /usr/lib/x86_64-linux-gnu/liblttng-ust-common.so.1.0.0 2024-03-04T08:30:44.998 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 51568 50444 /usr/lib/x86_64-linux-gnu/liblttng-ust-tracepoint.so.1.0.0 2024-03-04T08:30:44.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 47688 718 /usr/lib/x86_64-linux-gnu/libffi.so.8.1.0 2024-03-04T08:30:44.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:44.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 92312 811 /usr/lib/x86_64-linux-gnu/libtasn1.so.6.6.2 2024-03-04T08:30:44.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 1285888 821 /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.3.0 2024-03-04T08:30:44.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 137560 16841 /usr/lib/x86_64-linux-gnu/libbrotlicommon.so.1.0.9 2024-03-04T08:30:44.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:44.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:44.999 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 526896 794 /usr/lib/x86_64-linux-gnu/libgmp.so.10.4.1 2024-03-04T08:30:45.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 2000320 795 /usr/lib/x86_64-linux-gnu/libgnutls.so.30.31.0 2024-03-04T08:30:45.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 1743016 721 /usr/lib/x86_64-linux-gnu/libunistring.so.2.2.0 2024-03-04T08:30:45.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 841808 646 /usr/lib/x86_64-linux-gnu/libzstd.so.1.4.8 2024-03-04T08:30:45.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 4455728 673963 /usr/lib/x86_64-linux-gnu/libcrypto.so.3 2024-03-04T08:30:45.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 105376 20591 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25 2024-03-04T08:30:45.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:45.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 281000 977 /usr/lib/x86_64-linux-gnu/libnettle.so.8.4 2024-03-04T08:30:45.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 289800 594 /usr/lib/x86_64-linux-gnu/libhogweed.so.6.4 2024-03-04T08:30:45.000 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 376512 16064 /usr/lib/x86_64-linux-gnu/libldap-2.5.so.0.1.8 2024-03-04T08:30:45.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 667864 673964 /usr/lib/x86_64-linux-gnu/libssl.so.3 2024-03-04T08:30:45.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:45.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 63992 2956 /usr/lib/x86_64-linux-gnu/liblber-2.5.so.0.1.8 2024-03-04T08:30:45.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:45.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 75768 74580 /usr/lib/x86_64-linux-gnu/libpsl.so.5.3.2 2024-03-04T08:30:45.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 441944 16075 /usr/lib/x86_64-linux-gnu/libssh.so.4.8.7 2024-03-04T08:30:45.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 121864 15508 /usr/lib/x86_64-linux-gnu/librtmp.so.1 2024-03-04T08:30:45.001 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 129096 730 /usr/lib/x86_64-linux-gnu/libidn2.so.0.3.7 2024-03-04T08:30:45.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 166256 15296 /usr/lib/x86_64-linux-gnu/libnghttp2.so.14.20.1 2024-03-04T08:30:45.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 524448 13345 /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200.26.0 2024-03-04T08:30:45.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 2252096 806 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.30 2024-03-04T08:30:45.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:45.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 138136 26992 /usr/lib/x86_64-linux-gnu/libnl-3.so.200.26.0 2024-03-04T08:30:45.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 170456 595 /usr/lib/x86_64-linux-gnu/liblzma.so.5.2.5 2024-03-04T08:30:45.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 620728 50500 /usr/lib/x86_64-linux-gnu/libthrift-0.16.0.so 2024-03-04T08:30:45.002 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 673560 16077 /usr/lib/x86_64-linux-gnu/libcurl.so.4.7.0 2024-03-04T08:30:45.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:45.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 122144 3645 /usr/lib/x86_64-linux-gnu/librdmacm.so.1.3.39.0 2024-03-04T08:30:45.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 9733520 663410 /usr/lib/ceph/libceph-common.so.2 2024-03-04T08:30:45.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 142112 5024 /usr/lib/x86_64-linux-gnu/libibverbs.so.1.14.39.0 2024-03-04T08:30:45.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 166240 11009 /usr/lib/x86_64-linux-gnu/libudev.so.1.7.2 2024-03-04T08:30:45.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 220192 1156 /usr/lib/x86_64-linux-gnu/libblkid.so.1.1.0 2024-03-04T08:30:45.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 303168 5182 /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4.5.9 2024-03-04T08:30:45.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 51512 27605 /usr/lib/x86_64-linux-gnu/libbrotlidec.so.1.0.9 2024-03-04T08:30:45.003 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:45.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 63744 2646 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 2024-03-04T08:30:45.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 125488 805 /usr/lib/x86_64-linux-gnu/libgcc_s.so.1 2024-03-04T08:30:45.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 940560 11166 /usr/lib/x86_64-linux-gnu/libm.so.6 2024-03-04T08:30:45.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 244664 1523 /usr/lib/x86_64-linux-gnu/liblua5.3.so.0.0.0 2024-03-04T08:30:45.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:45.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 0r CHR 1,3 0t0 4 /dev/null 2024-03-04T08:30:45.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 1w FIFO 0,13 0t0 179463 pipe 2024-03-04T08:30:45.004 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 2w FIFO 0,13 0t0 179464 pipe 2024-03-04T08:30:45.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 3r FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:45.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 4w FIFO 0,13 0t0 175012 pipe 2024-03-04T08:30:45.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 5u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:45.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 6r FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:45.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 7w FIFO 0,13 0t0 175013 pipe 2024-03-04T08:30:45.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 8u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:45.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 9r FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:45.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 10w FIFO 0,13 0t0 175014 pipe 2024-03-04T08:30:45.005 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 11u a_inode 0,14 0 2070 [eventpoll] 2024-03-04T08:30:45.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 12r FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:45.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 13w FIFO 0,13 0t0 175015 pipe 2024-03-04T08:30:45.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 14w REG 8,1 1346155703 677887 /var/log/ceph/ceph-mds.e.log 2024-03-04T08:30:45.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 15uW REG 0,23 6 3591 /run/ceph/ceph-mds.e.pid 2024-03-04T08:30:45.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 16r FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:45.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 17w FIFO 0,13 0t0 175017 pipe 2024-03-04T08:30:45.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 18u unix 0xffff8aaf0974b480 0t0 175018 /var/run/ceph/ceph-mds.e.asok type=STREAM 2024-03-04T08:30:45.006 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 19u IPv4 176497 0t0 TCP *:6835 (LISTEN) 2024-03-04T08:30:45.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 20u IPv4 176535 0t0 TCP *:6837 (LISTEN) 2024-03-04T08:30:45.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 21r FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:45.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 22w FIFO 0,13 0t0 175020 pipe 2024-03-04T08:30:45.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 23r FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:45.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 24w FIFO 0,13 0t0 175021 pipe 2024-03-04T08:30:45.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 25u IPv4 172980 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:58902 (ESTABLISHED) 2024-03-04T08:30:45.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 26r FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:45.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 27w FIFO 0,13 0t0 175764 pipe 2024-03-04T08:30:45.007 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 28r FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:45.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 29w FIFO 0,13 0t0 175765 pipe 2024-03-04T08:30:45.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 30u IPv4 174029 0t0 TCP smithi113.front.sepia.ceph.com:48270->smithi136.front.sepia.ceph.com:6832 (ESTABLISHED) 2024-03-04T08:30:45.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 31u IPv4 176641 0t0 TCP smithi113.front.sepia.ceph.com:59316->smithi136.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:45.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 32u IPv4 172972 0t0 TCP smithi113.front.sepia.ceph.com:55722->smithi136.front.sepia.ceph.com:6816 (ESTABLISHED) 2024-03-04T08:30:45.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 33u IPv4 178416 0t0 TCP smithi113.front.sepia.ceph.com:52502->smithi113.front.sepia.ceph.com:6815 (ESTABLISHED) 2024-03-04T08:30:45.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 34u IPv4 176642 0t0 TCP smithi113.front.sepia.ceph.com:46508->smithi136.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:45.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 35u IPv4 172973 0t0 TCP smithi113.front.sepia.ceph.com:54628->smithi113.front.sepia.ceph.com:6808 (ESTABLISHED) 2024-03-04T08:30:45.008 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 36u IPv4 178417 0t0 TCP smithi113.front.sepia.ceph.com:43224->smithi136.front.sepia.ceph.com:6800 (ESTABLISHED) 2024-03-04T08:30:45.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 37u IPv4 175815 0t0 TCP smithi113.front.sepia.ceph.com:35680->smithi113.front.sepia.ceph.com:6824 (ESTABLISHED) 2024-03-04T08:30:45.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 38u IPv4 176817 0t0 TCP smithi113.front.sepia.ceph.com:6835->smithi136.front.sepia.ceph.com:36154 (ESTABLISHED) 2024-03-04T08:30:45.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 39u IPv4 181747 0t0 TCP smithi113.front.sepia.ceph.com:6835->192.168.0.1:44748 (ESTABLISHED) 2024-03-04T08:30:45.009 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-mds 53041 53440 quiesce_d root 41u IPv4 182076 0t0 TCP smithi113.front.sepia.ceph.com:41744->smithi113.front.sepia.ceph.com:3300 (ESTABLISHED) 2024-03-04T08:30:45.009 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 53161 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:45.009 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 53161 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:45.009 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/5 53161 root txt unknown /proc/53161/exe 2024-03-04T08:30:45.009 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 54836 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:45.009 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 54836 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:45.010 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/7 54836 root txt unknown /proc/54836/exe 2024-03-04T08:30:45.010 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 56225 root cwd DIR 8,1 4096 2 / 2024-03-04T08:30:45.010 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 56225 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:45.010 INFO:teuthology.orchestra.run.smithi113.stdout:kworker/u 56225 root txt unknown /proc/56225/exe 2024-03-04T08:30:45.010 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:45.010 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu rtd DIR 8,1 4096 2 / 2024-03-04T08:30:45.010 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu txt REG 8,1 1396520 886 /usr/bin/bash 2024-03-04T08:30:45.010 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:45.010 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:45.011 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu mem REG 8,1 18704 29062 /usr/lib/x86_64-linux-gnu/gconv/ISO8859-1.so 2024-03-04T08:30:45.011 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu mem REG 8,1 200136 8423 /usr/lib/x86_64-linux-gnu/libtinfo.so.6.3 2024-03-04T08:30:45.011 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu mem REG 8,1 27002 29118 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 2024-03-04T08:30:45.011 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:45.011 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu 0r FIFO 0,13 0t0 186982 pipe 2024-03-04T08:30:45.011 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu 1w FIFO 0,13 0t0 186983 pipe 2024-03-04T08:30:45.011 INFO:teuthology.orchestra.run.smithi113.stdout:bash 57800 ubuntu 2w FIFO 0,13 0t0 186984 pipe 2024-03-04T08:30:45.011 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:45.012 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:45.012 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root txt REG 8,1 232416 12806 /usr/bin/sudo 2024-03-04T08:30:45.012 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 472008 9123 /usr/lib/x86_64-linux-gnu/security/pam_systemd.so 2024-03-04T08:30:45.012 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 39024 9015 /usr/lib/x86_64-linux-gnu/libcap.so.2.44 2024-03-04T08:30:45.012 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:45.012 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:45.012 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:45.012 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:45.012 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 309600 1358 /usr/lib/x86_64-linux-gnu/libnss_systemd.so.2 2024-03-04T08:30:45.013 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 18528 650 /usr/lib/x86_64-linux-gnu/libpam_misc.so.0.82.1 2024-03-04T08:30:45.013 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 14328 23146 /usr/lib/x86_64-linux-gnu/security/pam_umask.so 2024-03-04T08:30:45.013 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 14328 841 /usr/lib/x86_64-linux-gnu/security/pam_cap.so 2024-03-04T08:30:45.013 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:45.013 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:45.013 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 198664 2137 /usr/lib/x86_64-linux-gnu/libcrypt.so.1.1.0 2024-03-04T08:30:45.013 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:45.013 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:45.014 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:45.014 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 93280 17230 /usr/lib/x86_64-linux-gnu/libnsl.so.2.0.1 2024-03-04T08:30:45.014 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 14328 14485 /usr/lib/x86_64-linux-gnu/security/pam_permit.so 2024-03-04T08:30:45.014 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 13960 8913 /usr/lib/x86_64-linux-gnu/security/pam_deny.so 2024-03-04T08:30:45.014 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:45.014 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 59464 23148 /usr/lib/x86_64-linux-gnu/security/pam_unix.so 2024-03-04T08:30:45.014 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 18424 8915 /usr/lib/x86_64-linux-gnu/security/pam_env.so 2024-03-04T08:30:45.014 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 108936 12946 /usr/lib/x86_64-linux-gnu/libz.so.1.2.11 2024-03-04T08:30:45.014 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 67736 629 /usr/lib/x86_64-linux-gnu/libpam.so.0.85.1 2024-03-04T08:30:45.015 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 516992 95600 /usr/libexec/sudo/sudoers.so 2024-03-04T08:30:45.015 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:45.015 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 27072 1559 /usr/lib/x86_64-linux-gnu/libcap-ng.so.0.0.0 2024-03-04T08:30:45.015 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 130040 95595 /usr/libexec/sudo/libsudo_util.so.0.0.0 2024-03-04T08:30:45.015 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:45.015 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 133200 775 /usr/lib/x86_64-linux-gnu/libaudit.so.1.0.0 2024-03-04T08:30:45.015 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 26696 14476 /usr/lib/x86_64-linux-gnu/security/pam_limits.so 2024-03-04T08:30:45.015 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:45.016 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root 0r FIFO 0,13 0t0 186982 pipe 2024-03-04T08:30:45.016 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root 1w FIFO 0,13 0t0 186983 pipe 2024-03-04T08:30:45.016 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root 2w FIFO 0,13 0t0 186984 pipe 2024-03-04T08:30:45.016 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root 3r FIFO 0,13 0t0 188147 pipe 2024-03-04T08:30:45.016 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root 4w FIFO 0,13 0t0 188147 pipe 2024-03-04T08:30:45.016 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root 5r REG 8,1 1698 658119 /etc/sudoers 2024-03-04T08:30:45.016 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root 6u netlink 0t0 188156 AUDIT 2024-03-04T08:30:45.016 INFO:teuthology.orchestra.run.smithi113.stdout:sudo 57802 root 7u unix 0xffff8aaf0974ad00 0t0 188157 type=DGRAM 2024-03-04T08:30:45.016 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:45.017 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:45.017 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root txt REG 8,1 167544 14991 /usr/bin/lsof 2024-03-04T08:30:45.017 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:45.017 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:45.017 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:45.017 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:45.017 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:45.017 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:45.018 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:45.018 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:45.018 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:45.018 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:45.018 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:45.018 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:45.018 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:45.018 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root 0r FIFO 0,13 0t0 186982 pipe 2024-03-04T08:30:45.018 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root 1w FIFO 0,13 0t0 186983 pipe 2024-03-04T08:30:45.019 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root 2w FIFO 0,13 0t0 186984 pipe 2024-03-04T08:30:45.019 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root 3r DIR 0,21 0 1 /proc 2024-03-04T08:30:45.019 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root 4r DIR 0,21 7 191732 /proc/57803/fd 2024-03-04T08:30:45.019 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root 5w FIFO 0,13 0t0 191737 pipe 2024-03-04T08:30:45.019 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57803 root 6r FIFO 0,13 0t0 191738 pipe 2024-03-04T08:30:45.019 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root cwd DIR 8,1 4096 6935 /home/ubuntu 2024-03-04T08:30:45.019 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root rtd DIR 8,1 4096 2 / 2024-03-04T08:30:45.019 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root txt REG 8,1 167544 14991 /usr/bin/lsof 2024-03-04T08:30:45.019 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 6039648 4922 /usr/lib/locale/locale-archive 2024-03-04T08:30:45.020 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 68552 27621 /usr/lib/x86_64-linux-gnu/libresolv.so.2 2024-03-04T08:30:45.020 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 182928 2341 /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1 2024-03-04T08:30:45.020 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 2216304 8857 /usr/lib/x86_64-linux-gnu/libc.so.6 2024-03-04T08:30:45.020 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 22600 499 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.9 2024-03-04T08:30:45.020 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 52080 11612 /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1 2024-03-04T08:30:45.020 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 18504 16447 /usr/lib/x86_64-linux-gnu/libcom_err.so.2.1 2024-03-04T08:30:45.020 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 828000 9998 /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3 2024-03-04T08:30:45.020 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 613064 813 /usr/lib/x86_64-linux-gnu/libpcre2-8.so.0.10.4 2024-03-04T08:30:45.021 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 338712 15290 /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2 2024-03-04T08:30:45.021 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 166280 834 /usr/lib/x86_64-linux-gnu/libselinux.so.1 2024-03-04T08:30:45.021 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 182912 17184 /usr/lib/x86_64-linux-gnu/libtirpc.so.3.0.0 2024-03-04T08:30:45.021 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root mem REG 8,1 240936 8851 /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 2024-03-04T08:30:45.021 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root 4r FIFO 0,13 0t0 191737 pipe 2024-03-04T08:30:45.021 INFO:teuthology.orchestra.run.smithi113.stdout:lsof 57805 root 7w FIFO 0,13 0t0 191738 pipe 2024-03-04T08:30:45.021 INFO:teuthology.orchestra.run.smithi113.stdout:USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 2024-03-04T08:30:45.021 INFO:teuthology.orchestra.run.smithi113.stdout:root 2 0.0 0.0 0 0 ? S 07:21 0:00 [kthreadd] 2024-03-04T08:30:45.022 INFO:teuthology.orchestra.run.smithi113.stdout:root 3 0.0 0.0 0 0 ? S 07:21 0:00 \_ [pool_workqueue_release] 2024-03-04T08:30:45.022 INFO:teuthology.orchestra.run.smithi113.stdout:root 4 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-rcu_g] 2024-03-04T08:30:45.022 INFO:teuthology.orchestra.run.smithi113.stdout:root 5 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-rcu_p] 2024-03-04T08:30:45.022 INFO:teuthology.orchestra.run.smithi113.stdout:root 6 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-slub_] 2024-03-04T08:30:45.022 INFO:teuthology.orchestra.run.smithi113.stdout:root 7 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-netns] 2024-03-04T08:30:45.022 INFO:teuthology.orchestra.run.smithi113.stdout:root 10 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/0:0H-events_highpri] 2024-03-04T08:30:45.022 INFO:teuthology.orchestra.run.smithi113.stdout:root 12 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-mm_pe] 2024-03-04T08:30:45.022 INFO:teuthology.orchestra.run.smithi113.stdout:root 13 0.0 0.0 0 0 ? I 07:21 0:00 \_ [rcu_tasks_kthread] 2024-03-04T08:30:45.023 INFO:teuthology.orchestra.run.smithi113.stdout:root 14 0.0 0.0 0 0 ? I 07:21 0:00 \_ [rcu_tasks_rude_kthread] 2024-03-04T08:30:45.023 INFO:teuthology.orchestra.run.smithi113.stdout:root 15 0.0 0.0 0 0 ? I 07:21 0:00 \_ [rcu_tasks_trace_kthread] 2024-03-04T08:30:45.023 INFO:teuthology.orchestra.run.smithi113.stdout:root 16 0.0 0.0 0 0 ? S 07:21 0:00 \_ [ksoftirqd/0] 2024-03-04T08:30:45.023 INFO:teuthology.orchestra.run.smithi113.stdout:root 17 0.0 0.0 0 0 ? I 07:21 0:01 \_ [rcu_preempt] 2024-03-04T08:30:45.023 INFO:teuthology.orchestra.run.smithi113.stdout:root 18 0.0 0.0 0 0 ? S 07:21 0:00 \_ [migration/0] 2024-03-04T08:30:45.023 INFO:teuthology.orchestra.run.smithi113.stdout:root 19 0.0 0.0 0 0 ? S 07:21 0:00 \_ [idle_inject/0] 2024-03-04T08:30:45.023 INFO:teuthology.orchestra.run.smithi113.stdout:root 20 0.0 0.0 0 0 ? S 07:21 0:00 \_ [cpuhp/0] 2024-03-04T08:30:45.023 INFO:teuthology.orchestra.run.smithi113.stdout:root 21 0.0 0.0 0 0 ? S 07:21 0:00 \_ [cpuhp/1] 2024-03-04T08:30:45.023 INFO:teuthology.orchestra.run.smithi113.stdout:root 22 0.0 0.0 0 0 ? S 07:21 0:00 \_ [idle_inject/1] 2024-03-04T08:30:45.024 INFO:teuthology.orchestra.run.smithi113.stdout:root 23 0.0 0.0 0 0 ? S 07:21 0:00 \_ [migration/1] 2024-03-04T08:30:45.024 INFO:teuthology.orchestra.run.smithi113.stdout:root 24 0.0 0.0 0 0 ? S 07:21 0:00 \_ [ksoftirqd/1] 2024-03-04T08:30:45.024 INFO:teuthology.orchestra.run.smithi113.stdout:root 26 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/1:0H-events_highpri] 2024-03-04T08:30:45.024 INFO:teuthology.orchestra.run.smithi113.stdout:root 27 0.0 0.0 0 0 ? S 07:21 0:00 \_ [cpuhp/2] 2024-03-04T08:30:45.024 INFO:teuthology.orchestra.run.smithi113.stdout:root 28 0.0 0.0 0 0 ? S 07:21 0:00 \_ [idle_inject/2] 2024-03-04T08:30:45.024 INFO:teuthology.orchestra.run.smithi113.stdout:root 29 0.0 0.0 0 0 ? S 07:21 0:00 \_ [migration/2] 2024-03-04T08:30:45.024 INFO:teuthology.orchestra.run.smithi113.stdout:root 30 0.0 0.0 0 0 ? S 07:21 0:00 \_ [ksoftirqd/2] 2024-03-04T08:30:45.024 INFO:teuthology.orchestra.run.smithi113.stdout:root 32 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/2:0H-events_highpri] 2024-03-04T08:30:45.024 INFO:teuthology.orchestra.run.smithi113.stdout:root 33 0.0 0.0 0 0 ? S 07:21 0:00 \_ [cpuhp/3] 2024-03-04T08:30:45.025 INFO:teuthology.orchestra.run.smithi113.stdout:root 34 0.0 0.0 0 0 ? S 07:21 0:00 \_ [idle_inject/3] 2024-03-04T08:30:45.025 INFO:teuthology.orchestra.run.smithi113.stdout:root 35 0.0 0.0 0 0 ? S 07:21 0:00 \_ [migration/3] 2024-03-04T08:30:45.025 INFO:teuthology.orchestra.run.smithi113.stdout:root 36 0.0 0.0 0 0 ? S 07:21 0:00 \_ [ksoftirqd/3] 2024-03-04T08:30:45.025 INFO:teuthology.orchestra.run.smithi113.stdout:root 38 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/3:0H-events_highpri] 2024-03-04T08:30:45.025 INFO:teuthology.orchestra.run.smithi113.stdout:root 39 0.0 0.0 0 0 ? S 07:21 0:00 \_ [cpuhp/4] 2024-03-04T08:30:45.025 INFO:teuthology.orchestra.run.smithi113.stdout:root 40 0.0 0.0 0 0 ? S 07:21 0:00 \_ [idle_inject/4] 2024-03-04T08:30:45.025 INFO:teuthology.orchestra.run.smithi113.stdout:root 41 0.0 0.0 0 0 ? S 07:21 0:00 \_ [migration/4] 2024-03-04T08:30:45.025 INFO:teuthology.orchestra.run.smithi113.stdout:root 42 0.0 0.0 0 0 ? S 07:21 0:00 \_ [ksoftirqd/4] 2024-03-04T08:30:45.026 INFO:teuthology.orchestra.run.smithi113.stdout:root 44 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/4:0H-events_highpri] 2024-03-04T08:30:45.026 INFO:teuthology.orchestra.run.smithi113.stdout:root 45 0.0 0.0 0 0 ? S 07:21 0:00 \_ [cpuhp/5] 2024-03-04T08:30:45.026 INFO:teuthology.orchestra.run.smithi113.stdout:root 46 0.0 0.0 0 0 ? S 07:21 0:00 \_ [idle_inject/5] 2024-03-04T08:30:45.026 INFO:teuthology.orchestra.run.smithi113.stdout:root 47 0.0 0.0 0 0 ? S 07:21 0:00 \_ [migration/5] 2024-03-04T08:30:45.026 INFO:teuthology.orchestra.run.smithi113.stdout:root 48 0.0 0.0 0 0 ? S 07:21 0:00 \_ [ksoftirqd/5] 2024-03-04T08:30:45.026 INFO:teuthology.orchestra.run.smithi113.stdout:root 50 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/5:0H-events_highpri] 2024-03-04T08:30:45.026 INFO:teuthology.orchestra.run.smithi113.stdout:root 51 0.0 0.0 0 0 ? S 07:21 0:00 \_ [cpuhp/6] 2024-03-04T08:30:45.026 INFO:teuthology.orchestra.run.smithi113.stdout:root 52 0.0 0.0 0 0 ? S 07:21 0:00 \_ [idle_inject/6] 2024-03-04T08:30:45.026 INFO:teuthology.orchestra.run.smithi113.stdout:root 53 0.0 0.0 0 0 ? S 07:21 0:00 \_ [migration/6] 2024-03-04T08:30:45.027 INFO:teuthology.orchestra.run.smithi113.stdout:root 54 0.0 0.0 0 0 ? S 07:21 0:00 \_ [ksoftirqd/6] 2024-03-04T08:30:45.027 INFO:teuthology.orchestra.run.smithi113.stdout:root 56 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/6:0H-events_highpri] 2024-03-04T08:30:45.027 INFO:teuthology.orchestra.run.smithi113.stdout:root 57 0.0 0.0 0 0 ? S 07:21 0:00 \_ [cpuhp/7] 2024-03-04T08:30:45.027 INFO:teuthology.orchestra.run.smithi113.stdout:root 58 0.0 0.0 0 0 ? S 07:21 0:00 \_ [idle_inject/7] 2024-03-04T08:30:45.027 INFO:teuthology.orchestra.run.smithi113.stdout:root 59 0.0 0.0 0 0 ? S 07:21 0:00 \_ [migration/7] 2024-03-04T08:30:45.027 INFO:teuthology.orchestra.run.smithi113.stdout:root 60 0.0 0.0 0 0 ? S 07:21 0:00 \_ [ksoftirqd/7] 2024-03-04T08:30:45.027 INFO:teuthology.orchestra.run.smithi113.stdout:root 62 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/7:0H-events_highpri] 2024-03-04T08:30:45.027 INFO:teuthology.orchestra.run.smithi113.stdout:root 70 0.0 0.0 0 0 ? S 07:21 0:00 \_ [kdevtmpfs] 2024-03-04T08:30:45.028 INFO:teuthology.orchestra.run.smithi113.stdout:root 71 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-inet_] 2024-03-04T08:30:45.028 INFO:teuthology.orchestra.run.smithi113.stdout:root 72 0.0 0.0 0 0 ? S 07:21 0:00 \_ [kauditd] 2024-03-04T08:30:45.028 INFO:teuthology.orchestra.run.smithi113.stdout:root 73 0.0 0.0 0 0 ? S 07:21 0:00 \_ [khungtaskd] 2024-03-04T08:30:45.028 INFO:teuthology.orchestra.run.smithi113.stdout:root 75 0.0 0.0 0 0 ? S 07:21 0:00 \_ [oom_reaper] 2024-03-04T08:30:45.028 INFO:teuthology.orchestra.run.smithi113.stdout:root 77 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-write] 2024-03-04T08:30:45.028 INFO:teuthology.orchestra.run.smithi113.stdout:root 78 0.0 0.0 0 0 ? S 07:21 0:00 \_ [kcompactd0] 2024-03-04T08:30:45.028 INFO:teuthology.orchestra.run.smithi113.stdout:root 79 0.0 0.0 0 0 ? SN 07:21 0:00 \_ [ksmd] 2024-03-04T08:30:45.028 INFO:teuthology.orchestra.run.smithi113.stdout:root 80 0.0 0.0 0 0 ? SN 07:21 0:00 \_ [khugepaged] 2024-03-04T08:30:45.028 INFO:teuthology.orchestra.run.smithi113.stdout:root 81 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-kinte] 2024-03-04T08:30:45.029 INFO:teuthology.orchestra.run.smithi113.stdout:root 82 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-kbloc] 2024-03-04T08:30:45.029 INFO:teuthology.orchestra.run.smithi113.stdout:root 83 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-blkcg] 2024-03-04T08:30:45.029 INFO:teuthology.orchestra.run.smithi113.stdout:root 84 0.0 0.0 0 0 ? S 07:21 0:00 \_ [irq/9-acpi] 2024-03-04T08:30:45.029 INFO:teuthology.orchestra.run.smithi113.stdout:root 85 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-tpm_d] 2024-03-04T08:30:45.029 INFO:teuthology.orchestra.run.smithi113.stdout:root 86 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-ata_s] 2024-03-04T08:30:45.029 INFO:teuthology.orchestra.run.smithi113.stdout:root 87 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-md] 2024-03-04T08:30:45.029 INFO:teuthology.orchestra.run.smithi113.stdout:root 88 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-md_bi] 2024-03-04T08:30:45.029 INFO:teuthology.orchestra.run.smithi113.stdout:root 89 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-edac-] 2024-03-04T08:30:45.029 INFO:teuthology.orchestra.run.smithi113.stdout:root 90 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-devfr] 2024-03-04T08:30:45.030 INFO:teuthology.orchestra.run.smithi113.stdout:root 91 0.0 0.0 0 0 ? S 07:21 0:00 \_ [watchdogd] 2024-03-04T08:30:45.030 INFO:teuthology.orchestra.run.smithi113.stdout:root 92 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/4:1H-xfs-log/dm-4] 2024-03-04T08:30:45.030 INFO:teuthology.orchestra.run.smithi113.stdout:root 93 0.0 0.0 0 0 ? S 07:21 0:00 \_ [kswapd0] 2024-03-04T08:30:45.030 INFO:teuthology.orchestra.run.smithi113.stdout:root 94 0.0 0.0 0 0 ? S 07:21 0:00 \_ [ecryptfs-kthread] 2024-03-04T08:30:45.030 INFO:teuthology.orchestra.run.smithi113.stdout:root 95 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-kthro] 2024-03-04T08:30:45.030 INFO:teuthology.orchestra.run.smithi113.stdout:root 96 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-acpi_] 2024-03-04T08:30:45.030 INFO:teuthology.orchestra.run.smithi113.stdout:root 100 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-mld] 2024-03-04T08:30:45.030 INFO:teuthology.orchestra.run.smithi113.stdout:root 101 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-ipv6_] 2024-03-04T08:30:45.030 INFO:teuthology.orchestra.run.smithi113.stdout:root 105 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/u17:0] 2024-03-04T08:30:45.031 INFO:teuthology.orchestra.run.smithi113.stdout:root 136 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/1:1H-kblockd] 2024-03-04T08:30:45.031 INFO:teuthology.orchestra.run.smithi113.stdout:root 140 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/3:1H-xfs-log/dm-4] 2024-03-04T08:30:45.031 INFO:teuthology.orchestra.run.smithi113.stdout:root 141 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/2:1H-xfs-log/dm-4] 2024-03-04T08:30:45.031 INFO:teuthology.orchestra.run.smithi113.stdout:root 153 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/7:1H-xfs-log/dm-4] 2024-03-04T08:30:45.031 INFO:teuthology.orchestra.run.smithi113.stdout:root 155 0.0 0.0 0 0 ? I< 07:21 0:01 \_ [kworker/6:1H-kblockd] 2024-03-04T08:30:45.031 INFO:teuthology.orchestra.run.smithi113.stdout:root 156 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/5:1H-xfs-log/dm-4] 2024-03-04T08:30:45.031 INFO:teuthology.orchestra.run.smithi113.stdout:root 157 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/0:1H-kblockd] 2024-03-04T08:30:45.031 INFO:teuthology.orchestra.run.smithi113.stdout:root 167 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-nvme-] 2024-03-04T08:30:45.031 INFO:teuthology.orchestra.run.smithi113.stdout:root 168 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-nvme-] 2024-03-04T08:30:45.032 INFO:teuthology.orchestra.run.smithi113.stdout:root 169 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-nvme-] 2024-03-04T08:30:45.032 INFO:teuthology.orchestra.run.smithi113.stdout:root 181 0.0 0.0 0 0 ? S 07:21 0:00 \_ [scsi_eh_0] 2024-03-04T08:30:45.032 INFO:teuthology.orchestra.run.smithi113.stdout:root 182 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-scsi_] 2024-03-04T08:30:45.032 INFO:teuthology.orchestra.run.smithi113.stdout:root 183 0.0 0.0 0 0 ? S 07:21 0:00 \_ [scsi_eh_1] 2024-03-04T08:30:45.032 INFO:teuthology.orchestra.run.smithi113.stdout:root 184 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-scsi_] 2024-03-04T08:30:45.032 INFO:teuthology.orchestra.run.smithi113.stdout:root 185 0.0 0.0 0 0 ? S 07:21 0:00 \_ [scsi_eh_2] 2024-03-04T08:30:45.032 INFO:teuthology.orchestra.run.smithi113.stdout:root 186 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-scsi_] 2024-03-04T08:30:45.032 INFO:teuthology.orchestra.run.smithi113.stdout:root 187 0.0 0.0 0 0 ? S 07:21 0:00 \_ [scsi_eh_3] 2024-03-04T08:30:45.033 INFO:teuthology.orchestra.run.smithi113.stdout:root 188 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-scsi_] 2024-03-04T08:30:45.033 INFO:teuthology.orchestra.run.smithi113.stdout:root 189 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-ixgbe] 2024-03-04T08:30:45.033 INFO:teuthology.orchestra.run.smithi113.stdout:root 214 0.0 0.0 0 0 ? S 07:21 0:00 \_ [scsi_eh_4] 2024-03-04T08:30:45.033 INFO:teuthology.orchestra.run.smithi113.stdout:root 215 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-scsi_] 2024-03-04T08:30:45.033 INFO:teuthology.orchestra.run.smithi113.stdout:root 216 0.0 0.0 0 0 ? S 07:21 0:00 \_ [scsi_eh_5] 2024-03-04T08:30:45.033 INFO:teuthology.orchestra.run.smithi113.stdout:root 217 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-scsi_] 2024-03-04T08:30:45.033 INFO:teuthology.orchestra.run.smithi113.stdout:root 218 0.0 0.0 0 0 ? S 07:21 0:00 \_ [scsi_eh_6] 2024-03-04T08:30:45.033 INFO:teuthology.orchestra.run.smithi113.stdout:root 219 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-scsi_] 2024-03-04T08:30:45.033 INFO:teuthology.orchestra.run.smithi113.stdout:root 220 0.0 0.0 0 0 ? S 07:21 0:00 \_ [scsi_eh_7] 2024-03-04T08:30:45.034 INFO:teuthology.orchestra.run.smithi113.stdout:root 221 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-scsi_] 2024-03-04T08:30:45.034 INFO:teuthology.orchestra.run.smithi113.stdout:root 222 0.0 0.0 0 0 ? S 07:21 0:00 \_ [scsi_eh_8] 2024-03-04T08:30:45.034 INFO:teuthology.orchestra.run.smithi113.stdout:root 223 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-scsi_] 2024-03-04T08:30:45.034 INFO:teuthology.orchestra.run.smithi113.stdout:root 224 0.0 0.0 0 0 ? S 07:21 0:00 \_ [scsi_eh_9] 2024-03-04T08:30:45.034 INFO:teuthology.orchestra.run.smithi113.stdout:root 225 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-scsi_] 2024-03-04T08:30:45.034 INFO:teuthology.orchestra.run.smithi113.stdout:root 260 0.0 0.0 0 0 ? S 07:21 0:01 \_ [jbd2/sda1-8] 2024-03-04T08:30:45.034 INFO:teuthology.orchestra.run.smithi113.stdout:root 261 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-ext4-] 2024-03-04T08:30:45.034 INFO:teuthology.orchestra.run.smithi113.stdout:root 309 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-rpcio] 2024-03-04T08:30:45.034 INFO:teuthology.orchestra.run.smithi113.stdout:root 310 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-xprti] 2024-03-04T08:30:45.035 INFO:teuthology.orchestra.run.smithi113.stdout:root 339 0.0 0.0 0 0 ? S 07:21 0:00 \_ [ext4lazyinit] 2024-03-04T08:30:45.035 INFO:teuthology.orchestra.run.smithi113.stdout:root 340 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-iscsi] 2024-03-04T08:30:45.035 INFO:teuthology.orchestra.run.smithi113.stdout:root 431 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-ipmi-] 2024-03-04T08:30:45.035 INFO:teuthology.orchestra.run.smithi113.stdout:root 446 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-crypt] 2024-03-04T08:30:45.035 INFO:teuthology.orchestra.run.smithi113.stdout:root 500 0.0 0.0 0 0 ? SN 07:21 0:00 \_ [kipmi0] 2024-03-04T08:30:45.035 INFO:teuthology.orchestra.run.smithi113.stdout:root 880 0.0 0.0 0 0 ? I 07:21 0:00 \_ [lockd] 2024-03-04T08:30:45.035 INFO:teuthology.orchestra.run.smithi113.stdout:root 1037 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-ib-co] 2024-03-04T08:30:45.035 INFO:teuthology.orchestra.run.smithi113.stdout:root 1038 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-ib-co] 2024-03-04T08:30:45.035 INFO:teuthology.orchestra.run.smithi113.stdout:root 1039 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-ib_mc] 2024-03-04T08:30:45.036 INFO:teuthology.orchestra.run.smithi113.stdout:root 1040 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-ib_nl] 2024-03-04T08:30:45.036 INFO:teuthology.orchestra.run.smithi113.stdout:root 1043 0.0 0.0 0 0 ? I< 07:21 0:00 \_ [kworker/R-rdma_] 2024-03-04T08:30:45.036 INFO:teuthology.orchestra.run.smithi113.stdout:root 1049 0.0 0.0 0 0 ? I 07:21 0:00 \_ [nfsd] 2024-03-04T08:30:45.036 INFO:teuthology.orchestra.run.smithi113.stdout:root 1050 0.0 0.0 0 0 ? I 07:21 0:00 \_ [nfsd] 2024-03-04T08:30:45.036 INFO:teuthology.orchestra.run.smithi113.stdout:root 1051 0.0 0.0 0 0 ? I 07:21 0:00 \_ [nfsd] 2024-03-04T08:30:45.036 INFO:teuthology.orchestra.run.smithi113.stdout:root 1052 0.0 0.0 0 0 ? I 07:21 0:00 \_ [nfsd] 2024-03-04T08:30:45.036 INFO:teuthology.orchestra.run.smithi113.stdout:root 1053 0.0 0.0 0 0 ? I 07:21 0:00 \_ [nfsd] 2024-03-04T08:30:45.036 INFO:teuthology.orchestra.run.smithi113.stdout:root 1054 0.0 0.0 0 0 ? I 07:21 0:00 \_ [nfsd] 2024-03-04T08:30:45.037 INFO:teuthology.orchestra.run.smithi113.stdout:root 1055 0.0 0.0 0 0 ? I 07:21 0:00 \_ [nfsd] 2024-03-04T08:30:45.037 INFO:teuthology.orchestra.run.smithi113.stdout:root 1056 0.0 0.0 0 0 ? I 07:21 0:00 \_ [nfsd] 2024-03-04T08:30:45.037 INFO:teuthology.orchestra.run.smithi113.stdout:root 7006 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-kdmfl] 2024-03-04T08:30:45.037 INFO:teuthology.orchestra.run.smithi113.stdout:root 7020 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-kdmfl] 2024-03-04T08:30:45.037 INFO:teuthology.orchestra.run.smithi113.stdout:root 7034 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-kdmfl] 2024-03-04T08:30:45.037 INFO:teuthology.orchestra.run.smithi113.stdout:root 7048 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-kdmfl] 2024-03-04T08:30:45.037 INFO:teuthology.orchestra.run.smithi113.stdout:root 7062 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-kdmfl] 2024-03-04T08:30:45.037 INFO:teuthology.orchestra.run.smithi113.stdout:root 7112 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-xfsal] 2024-03-04T08:30:45.037 INFO:teuthology.orchestra.run.smithi113.stdout:root 7113 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-xfs_m] 2024-03-04T08:30:45.038 INFO:teuthology.orchestra.run.smithi113.stdout:root 7114 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-xfs-b] 2024-03-04T08:30:45.038 INFO:teuthology.orchestra.run.smithi113.stdout:root 7115 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-xfs-c] 2024-03-04T08:30:45.038 INFO:teuthology.orchestra.run.smithi113.stdout:root 7116 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-xfs-r] 2024-03-04T08:30:45.038 INFO:teuthology.orchestra.run.smithi113.stdout:root 7117 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-xfs-b] 2024-03-04T08:30:45.038 INFO:teuthology.orchestra.run.smithi113.stdout:root 7118 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-xfs-i] 2024-03-04T08:30:45.038 INFO:teuthology.orchestra.run.smithi113.stdout:root 7119 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-xfs-l] 2024-03-04T08:30:45.038 INFO:teuthology.orchestra.run.smithi113.stdout:root 7120 0.0 0.0 0 0 ? I< 07:26 0:00 \_ [kworker/R-xfs-c] 2024-03-04T08:30:45.038 INFO:teuthology.orchestra.run.smithi113.stdout:root 7121 0.0 0.0 0 0 ? S 07:26 0:00 \_ [xfsaild/dm-4] 2024-03-04T08:30:45.039 INFO:teuthology.orchestra.run.smithi113.stdout:root 22661 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-b] 2024-03-04T08:30:45.039 INFO:teuthology.orchestra.run.smithi113.stdout:root 22662 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-c] 2024-03-04T08:30:45.039 INFO:teuthology.orchestra.run.smithi113.stdout:root 22663 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-r] 2024-03-04T08:30:45.039 INFO:teuthology.orchestra.run.smithi113.stdout:root 22664 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-b] 2024-03-04T08:30:45.039 INFO:teuthology.orchestra.run.smithi113.stdout:root 22665 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-i] 2024-03-04T08:30:45.039 INFO:teuthology.orchestra.run.smithi113.stdout:root 22666 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-l] 2024-03-04T08:30:45.039 INFO:teuthology.orchestra.run.smithi113.stdout:root 22667 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-c] 2024-03-04T08:30:45.039 INFO:teuthology.orchestra.run.smithi113.stdout:root 22668 0.0 0.0 0 0 ? S 07:32 0:00 \_ [xfsaild/dm-0] 2024-03-04T08:30:45.040 INFO:teuthology.orchestra.run.smithi113.stdout:root 22679 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-b] 2024-03-04T08:30:45.040 INFO:teuthology.orchestra.run.smithi113.stdout:root 22680 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-c] 2024-03-04T08:30:45.040 INFO:teuthology.orchestra.run.smithi113.stdout:root 22681 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-r] 2024-03-04T08:30:45.040 INFO:teuthology.orchestra.run.smithi113.stdout:root 22682 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-b] 2024-03-04T08:30:45.040 INFO:teuthology.orchestra.run.smithi113.stdout:root 22683 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-i] 2024-03-04T08:30:45.040 INFO:teuthology.orchestra.run.smithi113.stdout:root 22684 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-l] 2024-03-04T08:30:45.040 INFO:teuthology.orchestra.run.smithi113.stdout:root 22685 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-c] 2024-03-04T08:30:45.040 INFO:teuthology.orchestra.run.smithi113.stdout:root 22686 0.0 0.0 0 0 ? S 07:32 0:00 \_ [xfsaild/dm-1] 2024-03-04T08:30:45.040 INFO:teuthology.orchestra.run.smithi113.stdout:root 22697 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-b] 2024-03-04T08:30:45.041 INFO:teuthology.orchestra.run.smithi113.stdout:root 22698 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-c] 2024-03-04T08:30:45.041 INFO:teuthology.orchestra.run.smithi113.stdout:root 22699 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-r] 2024-03-04T08:30:45.041 INFO:teuthology.orchestra.run.smithi113.stdout:root 22700 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-b] 2024-03-04T08:30:45.041 INFO:teuthology.orchestra.run.smithi113.stdout:root 22701 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-i] 2024-03-04T08:30:45.041 INFO:teuthology.orchestra.run.smithi113.stdout:root 22702 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-l] 2024-03-04T08:30:45.041 INFO:teuthology.orchestra.run.smithi113.stdout:root 22703 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-c] 2024-03-04T08:30:45.041 INFO:teuthology.orchestra.run.smithi113.stdout:root 22704 0.0 0.0 0 0 ? S 07:32 0:00 \_ [xfsaild/dm-2] 2024-03-04T08:30:45.041 INFO:teuthology.orchestra.run.smithi113.stdout:root 22715 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-b] 2024-03-04T08:30:45.042 INFO:teuthology.orchestra.run.smithi113.stdout:root 22716 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-c] 2024-03-04T08:30:45.042 INFO:teuthology.orchestra.run.smithi113.stdout:root 22717 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-r] 2024-03-04T08:30:45.042 INFO:teuthology.orchestra.run.smithi113.stdout:root 22718 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-b] 2024-03-04T08:30:45.042 INFO:teuthology.orchestra.run.smithi113.stdout:root 22719 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-i] 2024-03-04T08:30:45.042 INFO:teuthology.orchestra.run.smithi113.stdout:root 22720 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-l] 2024-03-04T08:30:45.042 INFO:teuthology.orchestra.run.smithi113.stdout:root 22721 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-xfs-c] 2024-03-04T08:30:45.042 INFO:teuthology.orchestra.run.smithi113.stdout:root 22722 0.0 0.0 0 0 ? S 07:32 0:00 \_ [xfsaild/dm-3] 2024-03-04T08:30:45.042 INFO:teuthology.orchestra.run.smithi113.stdout:root 22737 0.0 0.0 0 0 ? I< 07:32 0:00 \_ [kworker/R-dio/d] 2024-03-04T08:30:45.043 INFO:teuthology.orchestra.run.smithi113.stdout:root 23878 0.0 0.0 0 0 ? I< 07:33 0:00 \_ [kworker/R-dio/d] 2024-03-04T08:30:45.043 INFO:teuthology.orchestra.run.smithi113.stdout:root 25019 0.0 0.0 0 0 ? I< 07:33 0:00 \_ [kworker/R-dio/d] 2024-03-04T08:30:45.043 INFO:teuthology.orchestra.run.smithi113.stdout:root 26160 0.0 0.0 0 0 ? I< 07:33 0:00 \_ [kworker/R-dio/d] 2024-03-04T08:30:45.043 INFO:teuthology.orchestra.run.smithi113.stdout:root 31391 0.0 0.0 0 0 ? I 07:33 0:03 \_ [kworker/3:58-dio/dm-1] 2024-03-04T08:30:45.043 INFO:teuthology.orchestra.run.smithi113.stdout:root 33557 0.0 0.0 0 0 ? I< 07:34 0:00 \_ [kworker/R-ceph-] 2024-03-04T08:30:45.043 INFO:teuthology.orchestra.run.smithi113.stdout:root 42259 0.1 0.0 0 0 ? I 07:53 0:03 \_ [kworker/u16:8-ext4-rsv-conversion] 2024-03-04T08:30:45.043 INFO:teuthology.orchestra.run.smithi113.stdout:root 44042 0.1 0.0 0 0 ? I 07:59 0:03 \_ [kworker/4:2-mm_percpu_wq] 2024-03-04T08:30:45.043 INFO:teuthology.orchestra.run.smithi113.stdout:root 45156 0.0 0.0 0 0 ? I 08:02 0:00 \_ [kworker/4:3-rcu_gp] 2024-03-04T08:30:45.044 INFO:teuthology.orchestra.run.smithi113.stdout:root 45873 0.1 0.0 0 0 ? I 08:03 0:03 \_ [kworker/3:1-mm_percpu_wq] 2024-03-04T08:30:45.044 INFO:teuthology.orchestra.run.smithi113.stdout:root 45884 0.0 0.0 0 0 ? I 08:04 0:00 \_ [kworker/1:0-dio/dm-0] 2024-03-04T08:30:45.044 INFO:teuthology.orchestra.run.smithi113.stdout:root 45885 0.2 0.0 0 0 ? I 08:04 0:04 \_ [kworker/2:3-mm_percpu_wq] 2024-03-04T08:30:45.044 INFO:teuthology.orchestra.run.smithi113.stdout:root 46444 0.2 0.0 0 0 ? I 08:05 0:03 \_ [kworker/1:5-mm_percpu_wq] 2024-03-04T08:30:45.044 INFO:teuthology.orchestra.run.smithi113.stdout:root 46451 0.1 0.0 0 0 ? I 08:05 0:01 \_ [kworker/u16:4-ixgbe] 2024-03-04T08:30:45.044 INFO:teuthology.orchestra.run.smithi113.stdout:root 46993 0.2 0.0 0 0 ? I 08:06 0:03 \_ [kworker/0:6-mm_percpu_wq] 2024-03-04T08:30:45.044 INFO:teuthology.orchestra.run.smithi113.stdout:root 47011 0.0 0.0 0 0 ? I 08:06 0:01 \_ [kworker/5:5-mm_percpu_wq] 2024-03-04T08:30:45.044 INFO:teuthology.orchestra.run.smithi113.stdout:root 47731 0.2 0.0 0 0 ? I 08:08 0:03 \_ [kworker/7:8-ceph-msgr] 2024-03-04T08:30:45.044 INFO:teuthology.orchestra.run.smithi113.stdout:root 47738 0.0 0.0 0 0 ? I 08:08 0:00 \_ [kworker/6:6-dio/dm-0] 2024-03-04T08:30:45.045 INFO:teuthology.orchestra.run.smithi113.stdout:root 47739 0.2 0.0 0 0 ? I 08:08 0:03 \_ [kworker/6:7-mm_percpu_wq] 2024-03-04T08:30:45.045 INFO:teuthology.orchestra.run.smithi113.stdout:root 48291 0.3 0.0 0 0 ? I 08:10 0:03 \_ [kworker/u16:15-xfs-cil/dm-4] 2024-03-04T08:30:45.045 INFO:teuthology.orchestra.run.smithi113.stdout:root 49946 0.0 0.0 0 0 ? I 08:16 0:00 \_ [kworker/2:0-events] 2024-03-04T08:30:45.045 INFO:teuthology.orchestra.run.smithi113.stdout:root 50632 0.0 0.0 0 0 ? I 08:18 0:00 \_ [kworker/0:1] 2024-03-04T08:30:45.045 INFO:teuthology.orchestra.run.smithi113.stdout:root 50914 0.0 0.0 0 0 ? I 08:19 0:00 \_ [kworker/7:0] 2024-03-04T08:30:45.045 INFO:teuthology.orchestra.run.smithi113.stdout:root 53161 0.0 0.0 0 0 ? I 08:23 0:00 \_ [kworker/5:1-xfs-sync/dm-3] 2024-03-04T08:30:45.045 INFO:teuthology.orchestra.run.smithi113.stdout:root 54836 0.0 0.0 0 0 ? I 08:26 0:00 \_ [kworker/7:1-mm_percpu_wq] 2024-03-04T08:30:45.045 INFO:teuthology.orchestra.run.smithi113.stdout:root 56225 0.0 0.0 0 0 ? I 08:28 0:00 \_ [kworker/u16:0] 2024-03-04T08:30:45.046 INFO:teuthology.orchestra.run.smithi113.stdout:root 57946 0.0 0.0 0 0 ? I 08:30 0:00 \_ [kworker/4:0-rcu_gp] 2024-03-04T08:30:45.046 INFO:teuthology.orchestra.run.smithi113.stdout:root 57947 0.0 0.0 0 0 ? I 08:30 0:00 \_ [kworker/2:1-inet_frag_wq] 2024-03-04T08:30:45.046 INFO:teuthology.orchestra.run.smithi113.stdout:root 1 0.3 0.0 167688 12624 ? Ss 07:21 0:13 /sbin/init 2024-03-04T08:30:45.046 INFO:teuthology.orchestra.run.smithi113.stdout:root 308 0.0 0.0 65044 31872 ? S sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2024-03-04T08:30:45.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=410 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 2024-03-04T08:30:45.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:30:45.065 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-03-04T08:30:45.078 INFO:teuthology.orchestra.run.smithi113.stderr:umount: /home/ubuntu/cephtest/mnt.0: no mount point specified. 2024-03-04T08:30:45.080 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-04T08:30:45.081 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-04T08:30:45.081 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-04T08:30:45.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-04T08:30:45.087 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-03-04T08:30:45.138 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=410 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275364k,nr_inodes=818841,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275368k,mode=755 0 0 2024-03-04T08:30:45.138 ERROR:tasks.daemonwatchdog.daemon_watchdog:ignoring exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/kernel_mount.py", line 191, in umount_wait self.umount(force) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/kernel_mount.py", line 170, in umount raise e File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/kernel_mount.py", line 163, in umount self.client_remote.run(args=cmd, timeout=UMOUNT_TIMEOUT, omit_sudo=False) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 32: 'sudo umount /home/ubuntu/cephtest/mnt.0 -f' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/daemonwatchdog.py", line 61, in bark mount.umount_wait(force=True) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/kernel_mount.py", line 197, in umount_wait self._run_umount_lf() File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/cephfs/mount.py", line 519, in _run_umount_lf return proc UnboundLocalError: local variable 'proc' referenced before assignment 2024-03-04T08:30:45.156 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-03-04T08:30:45.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:30:45.159 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-04T08:30:45.168 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-04T08:30:45.168 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T08:30:45.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d46c00640 1 Processor -- start 2024-03-04T08:30:45.348 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d46c00640 1 -- start start 2024-03-04T08:30:45.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d46c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1d400c85f0 0x7f1d4006b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d46c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1d4006b5a0 0x7f1d40161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d46c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d400c1ec0 con 0x7f1d400b20b0 2024-03-04T08:30:45.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d46c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d400c9b30 con 0x7f1d400c85f0 2024-03-04T08:30:45.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d46c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d400c12a0 con 0x7f1d4006b5a0 2024-03-04T08:30:45.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d3f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1d400c85f0 0x7f1d4006b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d3fe00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1d400b20b0 0x7f1d40157710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40004/0 (socket says 172.21.15.113:40004) 2024-03-04T08:30:45.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d3f400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1d400c85f0 0x7f1d4006b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52962/0 (socket says 172.21.15.113:52962) 2024-03-04T08:30:45.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d3fe00640 1 -- 172.21.15.113:0/1952043283 learned_addr learned my addr 172.21.15.113:0/1952043283 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:45.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d44c00640 1 --2- 172.21.15.113:0/1952043283 >> v2:172.21.15.136:3300/0 conn(0x7f1d4006b5a0 0x7f1d40161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d3f400640 1 -- 172.21.15.113:0/1952043283 >> v2:172.21.15.136:3300/0 conn(0x7f1d4006b5a0 msgr2=0x7f1d40161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:45.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d3f400640 1 --2- 172.21.15.113:0/1952043283 >> v2:172.21.15.136:3300/0 conn(0x7f1d4006b5a0 0x7f1d40161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.350 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d3f400640 1 -- 172.21.15.113:0/1952043283 >> v1:172.21.15.113:6789/0 conn(0x7f1d400b20b0 legacy=0x7f1d40157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.351+0000 7f1d3f400640 1 -- 172.21.15.113:0/1952043283 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400c1580 con 0x7f1d400c85f0 2024-03-04T08:30:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d3f400640 1 --2- 172.21.15.113:0/1952043283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400c85f0 0x7f1d4006b060 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f1d400c96d0 tx=0x7f1d2c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c653de723f1ca9c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:45.352 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d3ea00640 1 -- 172.21.15.113:0/1952043283 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0b5070 con 0x7f1d400c85f0 2024-03-04T08:30:45.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d3ea00640 1 -- 172.21.15.113:0/1952043283 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c0a8970 con 0x7f1d400c85f0 2024-03-04T08:30:45.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d3ea00640 1 -- 172.21.15.113:0/1952043283 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0a8d40 con 0x7f1d400c85f0 2024-03-04T08:30:45.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d46c00640 1 -- 172.21.15.113:0/1952043283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400c85f0 msgr2=0x7f1d4006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d46c00640 1 --2- 172.21.15.113:0/1952043283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400c85f0 0x7f1d4006b060 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f1d400c96d0 tx=0x7f1d2c01ec70 comp rx=0 tx=0).stop 2024-03-04T08:30:45.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d46c00640 1 -- 172.21.15.113:0/1952043283 shutdown_connections 2024-03-04T08:30:45.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d46c00640 1 --2- 172.21.15.113:0/1952043283 >> v2:172.21.15.136:3300/0 conn(0x7f1d4006b5a0 0x7f1d40161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d46c00640 1 --2- 172.21.15.113:0/1952043283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400c85f0 0x7f1d4006b060 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d46c00640 1 -- 172.21.15.113:0/1952043283 >> 172.21.15.113:0/1952043283 conn(0x7f1d4006e820 msgr2=0x7f1d4006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:45.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d46c00640 1 -- 172.21.15.113:0/1952043283 shutdown_connections 2024-03-04T08:30:45.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.355+0000 7f1d46c00640 1 -- 172.21.15.113:0/1952043283 wait complete. 2024-03-04T08:30:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d46c00640 1 Processor -- start 2024-03-04T08:30:45.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d46c00640 1 -- start start 2024-03-04T08:30:45.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d46c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d4006b5a0 0x7f1d400c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d46c00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400b20b0 0x7f1d400c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d46c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1d400c85f0 0x7f1d40165840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d46c00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40157e20 con 0x7f1d400b20b0 2024-03-04T08:30:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d46c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401571f0 con 0x7f1d4006b5a0 2024-03-04T08:30:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d46c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400c2400 con 0x7f1d400c85f0 2024-03-04T08:30:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d3f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400b20b0 0x7f1d400c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d3f400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400b20b0 0x7f1d400c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52974/0 (socket says 172.21.15.113:52974) 2024-03-04T08:30:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d3f400640 1 -- 172.21.15.113:0/2649591540 learned_addr learned my addr 172.21.15.113:0/2649591540 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d3fe00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d4006b5a0 0x7f1d400c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d44c00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1d400c85f0 0x7f1d40165840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d3f400640 1 -- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1d400c85f0 msgr2=0x7f1d40165840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d3f400640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1d400c85f0 0x7f1d40165840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d3f400640 1 -- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d4006b5a0 msgr2=0x7f1d400c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d3f400640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d4006b5a0 0x7f1d400c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d3f400640 1 -- 172.21.15.113:0/2649591540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d40156150 con 0x7f1d400b20b0 2024-03-04T08:30:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d44c00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1d400c85f0 0x7f1d40165840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T08:30:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d3f400640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400b20b0 0x7f1d400c6230 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0b3040 tx=0x7f1d2c01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d37e00640 1 -- 172.21.15.113:0/2649591540 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0bf070 con 0x7f1d400b20b0 2024-03-04T08:30:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d37e00640 1 -- 172.21.15.113:0/2649591540 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c0b07a0 con 0x7f1d400b20b0 2024-03-04T08:30:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d46c00640 1 -- 172.21.15.113:0/2649591540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d40156430 con 0x7f1d400b20b0 2024-03-04T08:30:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d37e00640 1 -- 172.21.15.113:0/2649591540 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0b5040 con 0x7f1d400b20b0 2024-03-04T08:30:45.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.359+0000 7f1d46c00640 1 -- 172.21.15.113:0/2649591540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d40144540 con 0x7f1d400b20b0 2024-03-04T08:30:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.363+0000 7f1d46c00640 1 -- 172.21.15.113:0/2649591540 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d40075c00 con 0x7f1d400b20b0 2024-03-04T08:30:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.363+0000 7f1d37e00640 1 -- 172.21.15.113:0/2649591540 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d2c0c2070 con 0x7f1d400b20b0 2024-03-04T08:30:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.363+0000 7f1d37e00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d20066f60 0x7f1d20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.363+0000 7f1d37e00640 1 -- 172.21.15.113:0/2649591540 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f1d2c1223b0 con 0x7f1d400b20b0 2024-03-04T08:30:45.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.363+0000 7f1d3fe00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d20066f60 0x7f1d20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.363+0000 7f1d3fe00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d20066f60 0x7f1d20069420 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000f70 tx=0x7f1d30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:45.365 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.367+0000 7f1d37e00640 1 -- 172.21.15.113:0/2649591540 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1d2c0187d0 con 0x7f1d400b20b0 2024-03-04T08:30:45.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.519+0000 7f1d46c00640 1 -- 172.21.15.113:0/2649591540 --> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d400c83d0 con 0x7f1d20066f60 2024-03-04T08:30:45.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.539+0000 7f1d37e00640 1 -- 172.21.15.113:0/2649591540 <== mgr.4103 v2:172.21.15.136:6832/3353359815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256615 (secure 0 0 0) 0x7f1d400c83d0 con 0x7f1d20066f60 2024-03-04T08:30:45.538 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:45.540 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-03-04T08:30:45.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 -- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d20066f60 msgr2=0x7f1d20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d20066f60 0x7f1d20069420 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000f70 tx=0x7f1d30009040 comp rx=0 tx=0).stop 2024-03-04T08:30:45.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 -- 172.21.15.113:0/2649591540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400b20b0 msgr2=0x7f1d400c6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400b20b0 0x7f1d400c6230 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0b3040 tx=0x7f1d2c01ba10 comp rx=0 tx=0).stop 2024-03-04T08:30:45.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 -- 172.21.15.113:0/2649591540 shutdown_connections 2024-03-04T08:30:45.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1d400c85f0 0x7f1d40165840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f1d20066f60 0x7f1d20069420 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d400b20b0 0x7f1d400c6230 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 --2- 172.21.15.113:0/2649591540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d4006b5a0 0x7f1d400c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 -- 172.21.15.113:0/2649591540 >> 172.21.15.113:0/2649591540 conn(0x7f1d4006e820 msgr2=0x7f1d400ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:45.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 -- 172.21.15.113:0/2649591540 shutdown_connections 2024-03-04T08:30:45.542 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.543+0000 7f1d46c00640 1 -- 172.21.15.113:0/2649591540 wait complete. 2024-03-04T08:30:45.558 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":1789,"stamp":"2024-03-04T08:30:44.638868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1349012084,"num_objects":6234,"num_object_clones":0,"num_object_copies":12468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6234,"num_whiteouts":0,"num_read":16745,"num_read_kb":5730603,"num_write":120386,"num_write_kb":3613053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1203,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164786,"ondisk_log_size":164786,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3969344,"kb_used_data":2644656,"kb_used_omap":12,"kb_used_meta":1324595,"kb_avail":751005376,"statfs":{"total":773094113280,"available":769029505024,"internally_reserved":0,"allocated":2708127744,"data_stored":2702402544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1356385846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004182"},"pg_stats":[{"pgid":"3.3f","version":"82'1995","reported_seq":4635,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.029763+0000","last_change":"2024-03-04T08:19:36.158827+0000","last_active":"2024-03-04T08:28:07.029763+0000","last_peered":"2024-03-04T08:28:07.029763+0000","last_clean":"2024-03-04T08:28:07.029763+0000","last_became_active":"2024-03-04T07:33:56.559031+0000","last_became_peered":"2024-03-04T07:33:56.559031+0000","last_unstale":"2024-03-04T08:28:07.029763+0000","last_undegraded":"2024-03-04T08:28:07.029763+0000","last_fullsized":"2024-03-04T08:28:07.029763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1995,"log_dups_size":0,"ondisk_log_size":1995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:14:15.903092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17246109400000001,"stat_sum":{"num_bytes":4212510,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":96,"num_read_kb":14433,"num_write":1280,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"80'1982","reported_seq":4617,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057758+0000","last_change":"2024-03-04T08:19:35.479098+0000","last_active":"2024-03-04T08:28:08.057758+0000","last_peered":"2024-03-04T08:28:08.057758+0000","last_clean":"2024-03-04T08:28:08.057758+0000","last_became_active":"2024-03-04T07:33:56.546547+0000","last_became_peered":"2024-03-04T07:33:56.546547+0000","last_unstale":"2024-03-04T08:28:08.057758+0000","last_undegraded":"2024-03-04T08:28:08.057758+0000","last_fullsized":"2024-03-04T08:28:08.057758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1982,"log_dups_size":0,"ondisk_log_size":1982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:14:54.677102+0000","scrub_duration":0,"objects_trimmed":173,"snaptrim_duration":2.7820693949999997,"stat_sum":{"num_bytes":8393198,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":106,"num_read_kb":21761,"num_write":1218,"num_write_kb":15425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"80'2113","reported_seq":4724,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142019+0000","last_change":"2024-03-04T08:19:49.392733+0000","last_active":"2024-03-04T08:28:07.142019+0000","last_peered":"2024-03-04T08:28:07.142019+0000","last_clean":"2024-03-04T08:28:07.142019+0000","last_became_active":"2024-03-04T07:33:56.530488+0000","last_became_peered":"2024-03-04T07:33:56.530488+0000","last_unstale":"2024-03-04T08:28:07.142019+0000","last_undegraded":"2024-03-04T08:28:07.142019+0000","last_fullsized":"2024-03-04T08:28:07.142019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2113,"log_dups_size":0,"ondisk_log_size":2113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:02:21.539335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.9646412140000002,"stat_sum":{"num_bytes":12588036,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":103,"num_read_kb":26195,"num_write":1342,"num_write_kb":31711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'2188","reported_seq":4775,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:42.133523+0000","last_change":"2024-03-04T08:19:32.727352+0000","last_active":"2024-03-04T08:26:42.133523+0000","last_peered":"2024-03-04T08:26:42.133523+0000","last_clean":"2024-03-04T08:26:42.133523+0000","last_became_active":"2024-03-04T07:34:36.767991+0000","last_became_peered":"2024-03-04T07:34:36.767991+0000","last_unstale":"2024-03-04T08:26:42.133523+0000","last_undegraded":"2024-03-04T08:26:42.133523+0000","last_fullsized":"2024-03-04T08:26:42.133523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2188,"log_dups_size":0,"ondisk_log_size":2188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:52.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019148479999999999,"stat_sum":{"num_bytes":12586995,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":106,"num_read_kb":25892,"num_write":1360,"num_write_kb":20414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"82'2280","reported_seq":4856,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014533+0000","last_change":"2024-03-04T08:19:36.167121+0000","last_active":"2024-03-04T08:26:27.014533+0000","last_peered":"2024-03-04T08:26:27.014533+0000","last_clean":"2024-03-04T08:26:27.014533+0000","last_became_active":"2024-03-04T07:33:56.540351+0000","last_became_peered":"2024-03-04T07:33:56.540351+0000","last_unstale":"2024-03-04T08:26:27.014533+0000","last_undegraded":"2024-03-04T08:26:27.014533+0000","last_fullsized":"2024-03-04T08:26:27.014533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2280,"log_dups_size":0,"ondisk_log_size":2280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:34:51.364307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.185369957,"stat_sum":{"num_bytes":16781857,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":103,"num_read_kb":39876,"num_write":1435,"num_write_kb":24971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"80'1985","reported_seq":4627,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.984410+0000","last_change":"2024-03-04T08:19:36.851047+0000","last_active":"2024-03-04T08:26:26.984410+0000","last_peered":"2024-03-04T08:26:26.984410+0000","last_clean":"2024-03-04T08:26:26.984410+0000","last_became_active":"2024-03-04T07:33:56.558046+0000","last_became_peered":"2024-03-04T07:33:56.558046+0000","last_unstale":"2024-03-04T08:26:26.984410+0000","last_undegraded":"2024-03-04T08:26:26.984410+0000","last_fullsized":"2024-03-04T08:26:26.984410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1985,"log_dups_size":0,"ondisk_log_size":1985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:00:43.652391+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.86040996599999997,"stat_sum":{"num_bytes":8388608,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":103,"num_read_kb":25220,"num_write":1249,"num_write_kb":17763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"82'2102","reported_seq":4728,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014493+0000","last_change":"2024-03-04T08:19:36.170530+0000","last_active":"2024-03-04T08:26:27.014493+0000","last_peered":"2024-03-04T08:26:27.014493+0000","last_clean":"2024-03-04T08:26:27.014493+0000","last_became_active":"2024-03-04T07:33:56.543176+0000","last_became_peered":"2024-03-04T07:33:56.543176+0000","last_unstale":"2024-03-04T08:26:27.014493+0000","last_undegraded":"2024-03-04T08:26:27.014493+0000","last_fullsized":"2024-03-04T08:26:27.014493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2102,"log_dups_size":0,"ondisk_log_size":2102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:03:35.061415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.188850779,"stat_sum":{"num_bytes":12603630,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":109,"num_read_kb":34094,"num_write":1333,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'2020","reported_seq":4661,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:28.507759+0000","last_change":"2024-03-04T08:19:52.268424+0000","last_active":"2024-03-04T08:26:28.507759+0000","last_peered":"2024-03-04T08:26:28.507759+0000","last_clean":"2024-03-04T08:26:28.507759+0000","last_became_active":"2024-03-04T07:33:56.559172+0000","last_became_peered":"2024-03-04T07:33:56.559172+0000","last_unstale":"2024-03-04T08:26:28.507759+0000","last_undegraded":"2024-03-04T08:26:28.507759+0000","last_fullsized":"2024-03-04T08:26:28.507759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2020,"log_dups_size":0,"ondisk_log_size":2020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:02:26.219884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.1976442450000002,"stat_sum":{"num_bytes":12587627,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":103,"num_read_kb":17785,"num_write":1294,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'2213","reported_seq":4820,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:22.023180+0000","last_change":"2024-03-04T08:19:36.163414+0000","last_active":"2024-03-04T08:27:22.023180+0000","last_peered":"2024-03-04T08:27:22.023180+0000","last_clean":"2024-03-04T08:27:22.023180+0000","last_became_active":"2024-03-04T07:33:56.560306+0000","last_became_peered":"2024-03-04T07:33:56.560306+0000","last_unstale":"2024-03-04T08:27:22.023180+0000","last_undegraded":"2024-03-04T08:27:22.023180+0000","last_fullsized":"2024-03-04T08:27:22.023180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2213,"log_dups_size":0,"ondisk_log_size":2213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:32:42.699565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17780130699999999,"stat_sum":{"num_bytes":8400582,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":109,"num_read_kb":22139,"num_write":1436,"num_write_kb":20580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"80'2060","reported_seq":4723,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.994336+0000","last_change":"2024-03-04T08:19:34.383564+0000","last_active":"2024-03-04T08:27:06.994336+0000","last_peered":"2024-03-04T08:27:06.994336+0000","last_clean":"2024-03-04T08:27:06.994336+0000","last_became_active":"2024-03-04T07:33:56.560592+0000","last_became_peered":"2024-03-04T07:33:56.560592+0000","last_unstale":"2024-03-04T08:27:06.994336+0000","last_undegraded":"2024-03-04T08:27:06.994336+0000","last_fullsized":"2024-03-04T08:27:06.994336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2060,"log_dups_size":0,"ondisk_log_size":2060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:40:27.691315+0000","scrub_duration":0,"objects_trimmed":120,"snaptrim_duration":1.6725091160000001,"stat_sum":{"num_bytes":25176920,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":134,"num_read_kb":52032,"num_write":1326,"num_write_kb":32831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"82'2220","reported_seq":4778,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.994298+0000","last_change":"2024-03-04T08:19:34.375018+0000","last_active":"2024-03-04T08:27:06.994298+0000","last_peered":"2024-03-04T08:27:06.994298+0000","last_clean":"2024-03-04T08:27:06.994298+0000","last_became_active":"2024-03-04T07:33:56.560719+0000","last_became_peered":"2024-03-04T07:33:56.560719+0000","last_unstale":"2024-03-04T08:27:06.994298+0000","last_undegraded":"2024-03-04T08:27:06.994298+0000","last_fullsized":"2024-03-04T08:27:06.994298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2220,"log_dups_size":0,"ondisk_log_size":2220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:33:12.566752+0000","scrub_duration":0,"objects_trimmed":134,"snaptrim_duration":1.6642866970000001,"stat_sum":{"num_bytes":8390989,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":101,"num_read_kb":24838,"num_write":1357,"num_write_kb":15528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"80'2201","reported_seq":4804,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:07.125104+0000","last_change":"2024-03-04T08:19:33.340351+0000","last_active":"2024-03-04T08:27:07.125104+0000","last_peered":"2024-03-04T08:27:07.125104+0000","last_clean":"2024-03-04T08:27:07.125104+0000","last_became_active":"2024-03-04T07:33:56.549568+0000","last_became_peered":"2024-03-04T07:33:56.549568+0000","last_unstale":"2024-03-04T08:27:07.125104+0000","last_undegraded":"2024-03-04T08:27:07.125104+0000","last_fullsized":"2024-03-04T08:27:07.125104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2201,"log_dups_size":0,"ondisk_log_size":2201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:29:31.376806+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.65008146899999997,"stat_sum":{"num_bytes":25170332,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":131,"num_read_kb":58959,"num_write":1343,"num_write_kb":36929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'2124","reported_seq":4745,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:08.512078+0000","last_change":"2024-03-04T08:19:52.261777+0000","last_active":"2024-03-04T08:27:08.512078+0000","last_peered":"2024-03-04T08:27:08.512078+0000","last_clean":"2024-03-04T08:27:08.512078+0000","last_became_active":"2024-03-04T07:33:56.558567+0000","last_became_peered":"2024-03-04T07:33:56.558567+0000","last_unstale":"2024-03-04T08:27:08.512078+0000","last_undegraded":"2024-03-04T08:27:08.512078+0000","last_fullsized":"2024-03-04T08:27:08.512078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2124,"log_dups_size":0,"ondisk_log_size":2124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:59:16.961107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.1904157900000003,"stat_sum":{"num_bytes":16788130,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":113,"num_read_kb":35043,"num_write":1370,"num_write_kb":33439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"80'2140","reported_seq":4759,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:47.140060+0000","last_change":"2024-03-04T08:19:49.427543+0000","last_active":"2024-03-04T08:27:47.140060+0000","last_peered":"2024-03-04T08:27:47.140060+0000","last_clean":"2024-03-04T08:27:47.140060+0000","last_became_active":"2024-03-04T07:33:56.543936+0000","last_became_peered":"2024-03-04T07:33:56.543936+0000","last_unstale":"2024-03-04T08:27:47.140060+0000","last_undegraded":"2024-03-04T08:27:47.140060+0000","last_fullsized":"2024-03-04T08:27:47.140060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2140,"log_dups_size":0,"ondisk_log_size":2140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:29:34.488147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0299552260000002,"stat_sum":{"num_bytes":8390003,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":16051,"num_write":1328,"num_write_kb":18720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'2145","reported_seq":4756,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:38.054541+0000","last_change":"2024-03-04T08:19:35.446076+0000","last_active":"2024-03-04T08:27:38.054541+0000","last_peered":"2024-03-04T08:27:38.054541+0000","last_clean":"2024-03-04T08:27:38.054541+0000","last_became_active":"2024-03-04T07:33:56.544837+0000","last_became_peered":"2024-03-04T07:33:56.544837+0000","last_unstale":"2024-03-04T08:27:38.054541+0000","last_undegraded":"2024-03-04T08:27:38.054541+0000","last_fullsized":"2024-03-04T08:27:38.054541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2145,"log_dups_size":0,"ondisk_log_size":2145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:01:02.809197+0000","scrub_duration":0,"objects_trimmed":196,"snaptrim_duration":2.7488653730000001,"stat_sum":{"num_bytes":8391933,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":90,"num_read_kb":10258,"num_write":1381,"num_write_kb":16511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"82'2151","reported_seq":4784,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.994588+0000","last_change":"2024-03-04T08:19:36.848889+0000","last_active":"2024-03-04T08:27:36.994588+0000","last_peered":"2024-03-04T08:27:36.994588+0000","last_clean":"2024-03-04T08:27:36.994588+0000","last_became_active":"2024-03-04T07:33:56.557962+0000","last_became_peered":"2024-03-04T07:33:56.557962+0000","last_unstale":"2024-03-04T08:27:36.994588+0000","last_undegraded":"2024-03-04T08:27:36.994588+0000","last_fullsized":"2024-03-04T08:27:36.994588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2151,"log_dups_size":0,"ondisk_log_size":2151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:33:13.908448+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.86055390200000004,"stat_sum":{"num_bytes":12654141,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":117,"num_read_kb":36882,"num_write":1403,"num_write_kb":24709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"82'2229","reported_seq":4853,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.994692+0000","last_change":"2024-03-04T08:19:36.842387+0000","last_active":"2024-03-04T08:27:36.994692+0000","last_peered":"2024-03-04T08:27:36.994692+0000","last_clean":"2024-03-04T08:27:36.994692+0000","last_became_active":"2024-03-04T07:33:56.558719+0000","last_became_peered":"2024-03-04T07:33:56.558719+0000","last_unstale":"2024-03-04T08:27:36.994692+0000","last_undegraded":"2024-03-04T08:27:36.994692+0000","last_fullsized":"2024-03-04T08:27:36.994692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2229,"log_dups_size":0,"ondisk_log_size":2229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:39:07.391195+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.85227727200000003,"stat_sum":{"num_bytes":16780080,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":129,"num_read_kb":21167,"num_write":1413,"num_write_kb":30886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"80'2047","reported_seq":4676,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138840+0000","last_change":"2024-03-04T08:19:32.728091+0000","last_active":"2024-03-04T08:27:37.138840+0000","last_peered":"2024-03-04T08:27:37.138840+0000","last_clean":"2024-03-04T08:27:37.138840+0000","last_became_active":"2024-03-04T07:33:56.528213+0000","last_became_peered":"2024-03-04T07:33:56.528213+0000","last_unstale":"2024-03-04T08:27:37.138840+0000","last_undegraded":"2024-03-04T08:27:37.138840+0000","last_fullsized":"2024-03-04T08:27:37.138840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2047,"log_dups_size":0,"ondisk_log_size":2047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:49.272879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019434987000000001,"stat_sum":{"num_bytes":4194351,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":120,"num_read_kb":13790,"num_write":1261,"num_write_kb":11549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"80'2288","reported_seq":4870,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:12.030625+0000","last_change":"2024-03-04T08:19:36.169663+0000","last_active":"2024-03-04T08:28:12.030625+0000","last_peered":"2024-03-04T08:28:12.030625+0000","last_clean":"2024-03-04T08:28:12.030625+0000","last_became_active":"2024-03-04T07:33:56.545467+0000","last_became_peered":"2024-03-04T07:33:56.545467+0000","last_unstale":"2024-03-04T08:28:12.030625+0000","last_undegraded":"2024-03-04T08:28:12.030625+0000","last_fullsized":"2024-03-04T08:28:12.030625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2288,"log_dups_size":0,"ondisk_log_size":2288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:22:01.957883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18267682599999999,"stat_sum":{"num_bytes":12645111,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":119,"num_read_kb":30029,"num_write":1423,"num_write_kb":26760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"80'2119","reported_seq":4737,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.001524+0000","last_change":"2024-03-04T08:19:34.382758+0000","last_active":"2024-03-04T08:27:57.001524+0000","last_peered":"2024-03-04T08:27:57.001524+0000","last_clean":"2024-03-04T08:27:57.001524+0000","last_became_active":"2024-03-04T07:33:56.548783+0000","last_became_peered":"2024-03-04T07:33:56.548783+0000","last_unstale":"2024-03-04T08:27:57.001524+0000","last_undegraded":"2024-03-04T08:27:57.001524+0000","last_fullsized":"2024-03-04T08:27:57.001524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2119,"log_dups_size":0,"ondisk_log_size":2119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:16:42.338152+0000","scrub_duration":0,"objects_trimmed":116,"snaptrim_duration":1.671257062,"stat_sum":{"num_bytes":8450701,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":102,"num_read_kb":25514,"num_write":1341,"num_write_kb":30030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"82'2119","reported_seq":4723,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.141072+0000","last_change":"2024-03-04T08:19:49.225914+0000","last_active":"2024-03-04T08:27:57.141072+0000","last_peered":"2024-03-04T08:27:57.141072+0000","last_clean":"2024-03-04T08:27:57.141072+0000","last_became_active":"2024-03-04T07:33:56.539403+0000","last_became_peered":"2024-03-04T07:33:56.539403+0000","last_unstale":"2024-03-04T08:27:57.141072+0000","last_undegraded":"2024-03-04T08:27:57.141072+0000","last_fullsized":"2024-03-04T08:27:57.141072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2119,"log_dups_size":0,"ondisk_log_size":2119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:43:54.215133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.827802057,"stat_sum":{"num_bytes":4204891,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":101,"num_read_kb":9693,"num_write":1325,"num_write_kb":13878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"80'2234","reported_seq":4838,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:22.131357+0000","last_change":"2024-03-04T08:19:32.717851+0000","last_active":"2024-03-04T08:26:22.131357+0000","last_peered":"2024-03-04T08:26:22.131357+0000","last_clean":"2024-03-04T08:26:22.131357+0000","last_became_active":"2024-03-04T07:34:37.644727+0000","last_became_peered":"2024-03-04T07:34:37.644727+0000","last_unstale":"2024-03-04T08:26:22.131357+0000","last_undegraded":"2024-03-04T08:26:22.131357+0000","last_fullsized":"2024-03-04T08:26:22.131357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:01:11.097920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010246078,"stat_sum":{"num_bytes":12594611,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":123,"num_read_kb":29265,"num_write":1418,"num_write_kb":23716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"82'2041","reported_seq":4709,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:58.516950+0000","last_change":"2024-03-04T08:19:52.301927+0000","last_active":"2024-03-04T08:27:58.516950+0000","last_peered":"2024-03-04T08:27:58.516950+0000","last_clean":"2024-03-04T08:27:58.516950+0000","last_became_active":"2024-03-04T07:33:56.545270+0000","last_became_peered":"2024-03-04T07:33:56.545270+0000","last_unstale":"2024-03-04T08:27:58.516950+0000","last_undegraded":"2024-03-04T08:27:58.516950+0000","last_fullsized":"2024-03-04T08:27:58.516950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2041,"log_dups_size":0,"ondisk_log_size":2041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:29:34.456689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.2311050080000001,"stat_sum":{"num_bytes":12602373,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":129,"num_read_kb":39775,"num_write":1326,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'2103","reported_seq":4737,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.001274+0000","last_change":"2024-03-04T08:19:36.847630+0000","last_active":"2024-03-04T08:28:22.001274+0000","last_peered":"2024-03-04T08:28:22.001274+0000","last_clean":"2024-03-04T08:28:22.001274+0000","last_became_active":"2024-03-04T07:33:56.554942+0000","last_became_peered":"2024-03-04T07:33:56.554942+0000","last_unstale":"2024-03-04T08:28:22.001274+0000","last_undegraded":"2024-03-04T08:28:22.001274+0000","last_fullsized":"2024-03-04T08:28:22.001274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2103,"log_dups_size":0,"ondisk_log_size":2103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:11:44.174822+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.85911008,"stat_sum":{"num_bytes":4342145,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":112,"num_read_kb":14015,"num_write":1342,"num_write_kb":11768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2416","reported_seq":4914,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.518032+0000","last_change":"2024-03-04T08:19:52.525743+0000","last_active":"2024-03-04T08:28:08.518032+0000","last_peered":"2024-03-04T08:28:08.518032+0000","last_clean":"2024-03-04T08:28:08.518032+0000","last_became_active":"2024-03-04T07:33:56.548790+0000","last_became_peered":"2024-03-04T07:33:56.548790+0000","last_unstale":"2024-03-04T08:28:08.518032+0000","last_undegraded":"2024-03-04T08:28:08.518032+0000","last_fullsized":"2024-03-04T08:28:08.518032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2416,"log_dups_size":0,"ondisk_log_size":2416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:53:43.350229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.454029974,"stat_sum":{"num_bytes":8428929,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":109,"num_read_kb":26630,"num_write":1382,"num_write_kb":16670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'2182","reported_seq":4787,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.002961+0000","last_change":"2024-03-04T08:19:34.384174+0000","last_active":"2024-03-04T08:28:07.002961+0000","last_peered":"2024-03-04T08:28:07.002961+0000","last_clean":"2024-03-04T08:28:07.002961+0000","last_became_active":"2024-03-04T07:33:56.553012+0000","last_became_peered":"2024-03-04T07:33:56.553012+0000","last_unstale":"2024-03-04T08:28:07.002961+0000","last_undegraded":"2024-03-04T08:28:07.002961+0000","last_fullsized":"2024-03-04T08:28:07.002961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2182,"log_dups_size":0,"ondisk_log_size":2182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:17:05.626254+0000","scrub_duration":0,"objects_trimmed":141,"snaptrim_duration":1.673071669,"stat_sum":{"num_bytes":4199673,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":99,"num_read_kb":13996,"num_write":1374,"num_write_kb":14419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'2031","reported_seq":4665,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057745+0000","last_change":"2024-03-04T08:19:35.448280+0000","last_active":"2024-03-04T08:28:08.057745+0000","last_peered":"2024-03-04T08:28:08.057745+0000","last_clean":"2024-03-04T08:28:08.057745+0000","last_became_active":"2024-03-04T07:33:56.550616+0000","last_became_peered":"2024-03-04T07:33:56.550616+0000","last_unstale":"2024-03-04T08:28:08.057745+0000","last_undegraded":"2024-03-04T08:28:08.057745+0000","last_fullsized":"2024-03-04T08:28:08.057745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2031,"log_dups_size":0,"ondisk_log_size":2031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:49:50.704680+0000","scrub_duration":0,"objects_trimmed":153,"snaptrim_duration":2.7491079520000001,"stat_sum":{"num_bytes":20984835,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":87,"num_read_kb":29030,"num_write":1330,"num_write_kb":28787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"80'2058","reported_seq":4675,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.518017+0000","last_change":"2024-03-04T08:19:52.538416+0000","last_active":"2024-03-04T08:28:08.518017+0000","last_peered":"2024-03-04T08:28:08.518017+0000","last_clean":"2024-03-04T08:28:08.518017+0000","last_became_active":"2024-03-04T07:33:56.543632+0000","last_became_peered":"2024-03-04T07:33:56.543632+0000","last_unstale":"2024-03-04T08:28:08.518017+0000","last_undegraded":"2024-03-04T08:28:08.518017+0000","last_fullsized":"2024-03-04T08:28:08.518017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2058,"log_dups_size":0,"ondisk_log_size":2058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:14:48.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.4338148220000004,"stat_sum":{"num_bytes":4197645,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":105,"num_read_kb":17328,"num_write":1252,"num_write_kb":17889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"80'2259","reported_seq":4843,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:17.143069+0000","last_change":"2024-03-04T08:19:49.701932+0000","last_active":"2024-03-04T08:28:17.143069+0000","last_peered":"2024-03-04T08:28:17.143069+0000","last_clean":"2024-03-04T08:28:17.143069+0000","last_became_active":"2024-03-04T07:33:56.546652+0000","last_became_peered":"2024-03-04T07:33:56.546652+0000","last_unstale":"2024-03-04T08:28:17.143069+0000","last_undegraded":"2024-03-04T08:28:17.143069+0000","last_fullsized":"2024-03-04T08:28:17.143069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2259,"log_dups_size":0,"ondisk_log_size":2259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:36:41.107790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.303629049,"stat_sum":{"num_bytes":8400817,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":112,"num_read_kb":14343,"num_write":1411,"num_write_kb":25423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'2098","reported_seq":4725,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:06.999085+0000","last_change":"2024-03-04T08:19:36.849222+0000","last_active":"2024-03-04T08:28:06.999085+0000","last_peered":"2024-03-04T08:28:06.999085+0000","last_clean":"2024-03-04T08:28:06.999085+0000","last_became_active":"2024-03-04T07:33:56.552189+0000","last_became_peered":"2024-03-04T07:33:56.552189+0000","last_unstale":"2024-03-04T08:28:06.999085+0000","last_undegraded":"2024-03-04T08:28:06.999085+0000","last_fullsized":"2024-03-04T08:28:06.999085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2098,"log_dups_size":0,"ondisk_log_size":2098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:38:31.691317+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.85857668899999995,"stat_sum":{"num_bytes":8407168,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":95,"num_read_kb":14091,"num_write":1358,"num_write_kb":17493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"80'2246","reported_seq":4842,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.141974+0000","last_change":"2024-03-04T08:19:32.724206+0000","last_active":"2024-03-04T08:28:07.141974+0000","last_peered":"2024-03-04T08:28:07.141974+0000","last_clean":"2024-03-04T08:28:07.141974+0000","last_became_active":"2024-03-04T07:33:56.549351+0000","last_became_peered":"2024-03-04T07:33:56.549351+0000","last_unstale":"2024-03-04T08:28:07.141974+0000","last_undegraded":"2024-03-04T08:28:07.141974+0000","last_fullsized":"2024-03-04T08:28:07.141974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2246,"log_dups_size":0,"ondisk_log_size":2246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:25.101476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.016086315,"stat_sum":{"num_bytes":20978563,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":109,"num_read_kb":27431,"num_write":1407,"num_write_kb":36516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'2497","reported_seq":4993,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.002897+0000","last_change":"2024-03-04T08:19:34.447685+0000","last_active":"2024-03-04T08:28:07.002897+0000","last_peered":"2024-03-04T08:28:07.002897+0000","last_clean":"2024-03-04T08:28:07.002897+0000","last_became_active":"2024-03-04T07:33:56.532251+0000","last_became_peered":"2024-03-04T07:33:56.532251+0000","last_unstale":"2024-03-04T08:28:07.002897+0000","last_undegraded":"2024-03-04T08:28:07.002897+0000","last_fullsized":"2024-03-04T08:28:07.002897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:03:48.692559+0000","scrub_duration":0,"objects_trimmed":183,"snaptrim_duration":1.7333053779999998,"stat_sum":{"num_bytes":12598132,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":114,"num_read_kb":36166,"num_write":1454,"num_write_kb":20545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"82'2217","reported_seq":4804,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.119475+0000","last_change":"2024-03-04T08:19:33.346715+0000","last_active":"2024-03-04T08:26:27.119475+0000","last_peered":"2024-03-04T08:26:27.119475+0000","last_clean":"2024-03-04T08:26:27.119475+0000","last_became_active":"2024-03-04T07:33:56.542937+0000","last_became_peered":"2024-03-04T07:33:56.542937+0000","last_unstale":"2024-03-04T08:26:27.119475+0000","last_undegraded":"2024-03-04T08:26:27.119475+0000","last_fullsized":"2024-03-04T08:26:27.119475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2217,"log_dups_size":0,"ondisk_log_size":2217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:17:50.886040+0000","scrub_duration":0,"objects_trimmed":58,"snaptrim_duration":0.65684712700000003,"stat_sum":{"num_bytes":12587619,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":118,"num_read_kb":25719,"num_write":1362,"num_write_kb":26221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"85'680","reported_seq":983,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:58.522965+0000","last_change":"2024-03-04T07:33:55.566130+0000","last_active":"2024-03-04T08:28:58.522965+0000","last_peered":"2024-03-04T08:28:58.522965+0000","last_clean":"2024-03-04T08:28:58.522965+0000","last_became_active":"2024-03-04T07:33:55.565894+0000","last_became_peered":"2024-03-04T07:33:55.565894+0000","last_unstale":"2024-03-04T08:28:58.522965+0000","last_undegraded":"2024-03-04T08:28:58.522965+0000","last_fullsized":"2024-03-04T08:28:58.522965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:36.530949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20973469,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":251,"num_read_kb":140597,"num_write":805,"num_write_kb":83805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"82'2160","reported_seq":4871,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.132021+0000","last_change":"2024-03-04T08:19:49.377440+0000","last_active":"2024-03-04T08:26:27.132021+0000","last_peered":"2024-03-04T08:26:27.132021+0000","last_clean":"2024-03-04T08:26:27.132021+0000","last_became_active":"2024-03-04T07:33:56.539738+0000","last_became_peered":"2024-03-04T07:33:56.539738+0000","last_unstale":"2024-03-04T08:26:27.132021+0000","last_undegraded":"2024-03-04T08:26:27.132021+0000","last_fullsized":"2024-03-04T08:26:27.132021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2160,"log_dups_size":0,"ondisk_log_size":2160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:44:38.647134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.9789417120000001,"stat_sum":{"num_bytes":8388926,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":220,"num_read_kb":14085,"num_write":1362,"num_write_kb":17052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1189","reported_seq":1504,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.988431+0000","last_change":"2024-03-04T07:33:55.554956+0000","last_active":"2024-03-04T08:26:26.988431+0000","last_peered":"2024-03-04T08:26:26.988431+0000","last_clean":"2024-03-04T08:26:26.988431+0000","last_became_active":"2024-03-04T07:33:55.554151+0000","last_became_peered":"2024-03-04T07:33:55.554151+0000","last_unstale":"2024-03-04T08:26:26.988431+0000","last_undegraded":"2024-03-04T08:26:26.988431+0000","last_fullsized":"2024-03-04T08:26:26.988431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1189,"log_dups_size":0,"ondisk_log_size":1189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:06:19.621510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":322,"num_read_kb":118780,"num_write":1380,"num_write_kb":72133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"82'487","reported_seq":770,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:13.512566+0000","last_change":"2024-03-04T07:33:55.542200+0000","last_active":"2024-03-04T08:27:13.512566+0000","last_peered":"2024-03-04T08:27:13.512566+0000","last_clean":"2024-03-04T08:27:13.512566+0000","last_became_active":"2024-03-04T07:33:55.541902+0000","last_became_peered":"2024-03-04T07:33:55.541902+0000","last_unstale":"2024-03-04T08:27:13.512566+0000","last_undegraded":"2024-03-04T08:27:13.512566+0000","last_fullsized":"2024-03-04T08:27:13.512566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:55:50.251288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":258,"num_read_kb":115971,"num_write":610,"num_write_kb":58869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"80'2411","reported_seq":4973,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:08.512031+0000","last_change":"2024-03-04T08:19:52.693084+0000","last_active":"2024-03-04T08:27:08.512031+0000","last_peered":"2024-03-04T08:27:08.512031+0000","last_clean":"2024-03-04T08:27:08.512031+0000","last_became_active":"2024-03-04T07:33:56.546087+0000","last_became_peered":"2024-03-04T07:33:56.546087+0000","last_unstale":"2024-03-04T08:27:08.512031+0000","last_undegraded":"2024-03-04T08:27:08.512031+0000","last_fullsized":"2024-03-04T08:27:08.512031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2411,"log_dups_size":0,"ondisk_log_size":2411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:42:05.213414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.6221017409999998,"stat_sum":{"num_bytes":4200161,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":109,"num_read_kb":16756,"num_write":1524,"num_write_kb":21226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1610","reported_seq":1897,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.990193+0000","last_change":"2024-03-04T07:33:55.568508+0000","last_active":"2024-03-04T08:27:06.990193+0000","last_peered":"2024-03-04T08:27:06.990193+0000","last_clean":"2024-03-04T08:27:06.990193+0000","last_became_active":"2024-03-04T07:33:55.566073+0000","last_became_peered":"2024-03-04T07:33:55.566073+0000","last_unstale":"2024-03-04T08:27:06.990193+0000","last_undegraded":"2024-03-04T08:27:06.990193+0000","last_fullsized":"2024-03-04T08:27:06.990193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1610,"log_dups_size":0,"ondisk_log_size":1610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:34:17.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":228,"num_read_kb":146204,"num_write":1735,"num_write_kb":76344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'2138","reported_seq":4771,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:18.052787+0000","last_change":"2024-03-04T08:19:35.448277+0000","last_active":"2024-03-04T08:27:18.052787+0000","last_peered":"2024-03-04T08:27:18.052787+0000","last_clean":"2024-03-04T08:27:18.052787+0000","last_became_active":"2024-03-04T07:33:56.552533+0000","last_became_peered":"2024-03-04T07:33:56.552533+0000","last_unstale":"2024-03-04T08:27:18.052787+0000","last_undegraded":"2024-03-04T08:27:18.052787+0000","last_fullsized":"2024-03-04T08:27:18.052787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2138,"log_dups_size":0,"ondisk_log_size":2138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:15:02.557608+0000","scrub_duration":0,"objects_trimmed":178,"snaptrim_duration":2.7512426310000002,"stat_sum":{"num_bytes":25170167,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":124,"num_read_kb":49299,"num_write":1360,"num_write_kb":40715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'786","reported_seq":1224,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:29:32.015555+0000","last_change":"2024-03-04T07:33:55.559395+0000","last_active":"2024-03-04T08:29:32.015555+0000","last_peered":"2024-03-04T08:29:32.015555+0000","last_clean":"2024-03-04T08:29:32.015555+0000","last_became_active":"2024-03-04T07:33:55.558593+0000","last_became_peered":"2024-03-04T07:33:55.558593+0000","last_unstale":"2024-03-04T08:29:32.015555+0000","last_undegraded":"2024-03-04T08:29:32.015555+0000","last_fullsized":"2024-03-04T08:29:32.015555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:00:20.205445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17355948,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":385,"num_read_kb":100065,"num_write":866,"num_write_kb":59600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"82'2101","reported_seq":4721,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.990264+0000","last_change":"2024-03-04T08:19:36.839345+0000","last_active":"2024-03-04T08:27:06.990264+0000","last_peered":"2024-03-04T08:27:06.990264+0000","last_clean":"2024-03-04T08:27:06.990264+0000","last_became_active":"2024-03-04T07:33:56.552952+0000","last_became_peered":"2024-03-04T07:33:56.552952+0000","last_unstale":"2024-03-04T08:27:06.990264+0000","last_undegraded":"2024-03-04T08:27:06.990264+0000","last_fullsized":"2024-03-04T08:27:06.990264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2101,"log_dups_size":0,"ondisk_log_size":2101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:53:29.724551+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.84949035500000003,"stat_sum":{"num_bytes":8307065,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":95,"num_read_kb":25119,"num_write":1361,"num_write_kb":16142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"82'705","reported_seq":1001,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.990287+0000","last_change":"2024-03-04T07:33:55.568220+0000","last_active":"2024-03-04T08:27:06.990287+0000","last_peered":"2024-03-04T08:27:06.990287+0000","last_clean":"2024-03-04T08:27:06.990287+0000","last_became_active":"2024-03-04T07:33:55.567569+0000","last_became_peered":"2024-03-04T07:33:55.567569+0000","last_unstale":"2024-03-04T08:27:06.990287+0000","last_undegraded":"2024-03-04T08:27:06.990287+0000","last_fullsized":"2024-03-04T08:27:06.990287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:40:49.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":256,"num_read_kb":181413,"num_write":818,"num_write_kb":87860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"82'2131","reported_seq":4730,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:07.136092+0000","last_change":"2024-03-04T08:19:49.095825+0000","last_active":"2024-03-04T08:27:07.136092+0000","last_peered":"2024-03-04T08:27:07.136092+0000","last_clean":"2024-03-04T08:27:07.136092+0000","last_became_active":"2024-03-04T07:33:56.559456+0000","last_became_peered":"2024-03-04T07:33:56.559456+0000","last_unstale":"2024-03-04T08:27:07.136092+0000","last_undegraded":"2024-03-04T08:27:07.136092+0000","last_fullsized":"2024-03-04T08:27:07.136092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2131,"log_dups_size":0,"ondisk_log_size":2131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:50:29.399119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.6973493710000001,"stat_sum":{"num_bytes":8393873,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":101,"num_read_kb":25726,"num_write":1348,"num_write_kb":16222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'1400","reported_seq":1705,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:08.051648+0000","last_change":"2024-03-04T07:33:55.550625+0000","last_active":"2024-03-04T08:27:08.051648+0000","last_peered":"2024-03-04T08:27:08.051648+0000","last_clean":"2024-03-04T08:27:08.051648+0000","last_became_active":"2024-03-04T07:33:55.550327+0000","last_became_peered":"2024-03-04T07:33:55.550327+0000","last_unstale":"2024-03-04T08:27:08.051648+0000","last_undegraded":"2024-03-04T08:27:08.051648+0000","last_fullsized":"2024-03-04T08:27:08.051648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:20.282585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":274,"num_read_kb":137054,"num_write":1577,"num_write_kb":60435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"80'2174","reported_seq":4785,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:47.130659+0000","last_change":"2024-03-04T08:19:33.352033+0000","last_active":"2024-03-04T08:27:47.130659+0000","last_peered":"2024-03-04T08:27:47.130659+0000","last_clean":"2024-03-04T08:27:47.130659+0000","last_became_active":"2024-03-04T07:33:56.536870+0000","last_became_peered":"2024-03-04T07:33:56.536870+0000","last_unstale":"2024-03-04T08:27:47.130659+0000","last_undegraded":"2024-03-04T08:27:47.130659+0000","last_fullsized":"2024-03-04T08:27:47.130659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2174,"log_dups_size":0,"ondisk_log_size":2174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:56:44.867605+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.66166569399999997,"stat_sum":{"num_bytes":12846648,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":118,"num_read_kb":29225,"num_write":1393,"num_write_kb":24887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"82'1283","reported_seq":1627,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:52.000760+0000","last_change":"2024-03-04T07:33:55.538915+0000","last_active":"2024-03-04T08:27:52.000760+0000","last_peered":"2024-03-04T08:27:52.000760+0000","last_clean":"2024-03-04T08:27:52.000760+0000","last_became_active":"2024-03-04T07:33:55.536263+0000","last_became_peered":"2024-03-04T07:33:55.536263+0000","last_unstale":"2024-03-04T08:27:52.000760+0000","last_undegraded":"2024-03-04T08:27:52.000760+0000","last_fullsized":"2024-03-04T08:27:52.000760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1283,"log_dups_size":0,"ondisk_log_size":1283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:09.612704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360800,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":354,"num_read_kb":206595,"num_write":1442,"num_write_kb":96944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"80'2051","reported_seq":4680,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:07.020807+0000","last_change":"2024-03-04T08:19:36.168875+0000","last_active":"2024-03-04T08:27:07.020807+0000","last_peered":"2024-03-04T08:27:07.020807+0000","last_clean":"2024-03-04T08:27:07.020807+0000","last_became_active":"2024-03-04T07:33:56.550767+0000","last_became_peered":"2024-03-04T07:33:56.550767+0000","last_unstale":"2024-03-04T08:27:07.020807+0000","last_undegraded":"2024-03-04T08:27:07.020807+0000","last_fullsized":"2024-03-04T08:27:07.020807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2051,"log_dups_size":0,"ondisk_log_size":2051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:41:38.658644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.187228595,"stat_sum":{"num_bytes":12597061,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":105,"num_read_kb":35778,"num_write":1291,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"82'483","reported_seq":772,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138903+0000","last_change":"2024-03-04T07:33:55.560382+0000","last_active":"2024-03-04T08:27:37.138903+0000","last_peered":"2024-03-04T08:27:37.138903+0000","last_clean":"2024-03-04T08:27:37.138903+0000","last_became_active":"2024-03-04T07:33:55.560078+0000","last_became_peered":"2024-03-04T07:33:55.560078+0000","last_unstale":"2024-03-04T08:27:37.138903+0000","last_undegraded":"2024-03-04T08:27:37.138903+0000","last_fullsized":"2024-03-04T08:27:37.138903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:41:10.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":271,"num_read_kb":107276,"num_write":561,"num_write_kb":58233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"80'2019","reported_seq":4649,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138868+0000","last_change":"2024-03-04T08:19:49.480210+0000","last_active":"2024-03-04T08:27:37.138868+0000","last_peered":"2024-03-04T08:27:37.138868+0000","last_clean":"2024-03-04T08:27:37.138868+0000","last_became_active":"2024-03-04T07:33:56.535960+0000","last_became_peered":"2024-03-04T07:33:56.535960+0000","last_unstale":"2024-03-04T08:27:37.138868+0000","last_undegraded":"2024-03-04T08:27:37.138868+0000","last_fullsized":"2024-03-04T08:27:37.138868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2019,"log_dups_size":0,"ondisk_log_size":2019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:17:01.637891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0525318239999999,"stat_sum":{"num_bytes":16780560,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":39118,"num_write":1277,"num_write_kb":32917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"82'440","reported_seq":737,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.994642+0000","last_change":"2024-03-04T07:33:55.567682+0000","last_active":"2024-03-04T08:27:36.994642+0000","last_peered":"2024-03-04T08:27:36.994642+0000","last_clean":"2024-03-04T08:27:36.994642+0000","last_became_active":"2024-03-04T07:33:55.566915+0000","last_became_peered":"2024-03-04T07:33:55.566915+0000","last_unstale":"2024-03-04T08:27:36.994642+0000","last_undegraded":"2024-03-04T08:27:36.994642+0000","last_fullsized":"2024-03-04T08:27:36.994642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:33:47.010346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":276,"num_read_kb":100025,"num_write":608,"num_write_kb":47104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'2205","reported_seq":4818,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.025397+0000","last_change":"2024-03-04T08:19:36.162956+0000","last_active":"2024-03-04T08:27:37.025397+0000","last_peered":"2024-03-04T08:27:37.025397+0000","last_clean":"2024-03-04T08:27:37.025397+0000","last_became_active":"2024-03-04T07:33:56.547459+0000","last_became_peered":"2024-03-04T07:33:56.547459+0000","last_unstale":"2024-03-04T08:27:37.025397+0000","last_undegraded":"2024-03-04T08:27:37.025397+0000","last_fullsized":"2024-03-04T08:27:37.025397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2205,"log_dups_size":0,"ondisk_log_size":2205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:48:06.725806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.181152964,"stat_sum":{"num_bytes":4213693,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":118,"num_read_kb":1396,"num_write":1423,"num_write_kb":14566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'723","reported_seq":1018,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.998645+0000","last_change":"2024-03-04T07:33:55.557041+0000","last_active":"2024-03-04T08:27:36.998645+0000","last_peered":"2024-03-04T08:27:36.998645+0000","last_clean":"2024-03-04T08:27:36.998645+0000","last_became_active":"2024-03-04T07:33:55.556416+0000","last_became_peered":"2024-03-04T07:33:55.556416+0000","last_unstale":"2024-03-04T08:27:36.998645+0000","last_undegraded":"2024-03-04T08:27:36.998645+0000","last_fullsized":"2024-03-04T08:27:36.998645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:31.472235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8390846,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":242,"num_read_kb":79450,"num_write":832,"num_write_kb":51201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"82'2099","reported_seq":4736,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.998604+0000","last_change":"2024-03-04T08:19:34.394444+0000","last_active":"2024-03-04T08:27:36.998604+0000","last_peered":"2024-03-04T08:27:36.998604+0000","last_clean":"2024-03-04T08:27:36.998604+0000","last_became_active":"2024-03-04T07:33:56.531565+0000","last_became_peered":"2024-03-04T07:33:56.531565+0000","last_unstale":"2024-03-04T08:27:36.998604+0000","last_undegraded":"2024-03-04T08:27:36.998604+0000","last_fullsized":"2024-03-04T08:27:36.998604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2099,"log_dups_size":0,"ondisk_log_size":2099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:02:41.845508+0000","scrub_duration":0,"objects_trimmed":121,"snaptrim_duration":1.6809537379999999,"stat_sum":{"num_bytes":16783369,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":112,"num_read_kb":39998,"num_write":1341,"num_write_kb":27959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'797","reported_seq":1153,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138789+0000","last_change":"2024-03-04T07:33:55.538469+0000","last_active":"2024-03-04T08:27:37.138789+0000","last_peered":"2024-03-04T08:27:37.138789+0000","last_clean":"2024-03-04T08:27:37.138789+0000","last_became_active":"2024-03-04T07:33:55.538165+0000","last_became_peered":"2024-03-04T07:33:55.538165+0000","last_unstale":"2024-03-04T08:27:37.138789+0000","last_undegraded":"2024-03-04T08:27:37.138789+0000","last_fullsized":"2024-03-04T08:27:37.138789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:00:01.263768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165936,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":419,"num_read_kb":136829,"num_write":955,"num_write_kb":63584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"80'2116","reported_seq":4728,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:18.058628+0000","last_change":"2024-03-04T08:19:35.444496+0000","last_active":"2024-03-04T08:28:18.058628+0000","last_peered":"2024-03-04T08:28:18.058628+0000","last_clean":"2024-03-04T08:28:18.058628+0000","last_became_active":"2024-03-04T07:34:36.770794+0000","last_became_peered":"2024-03-04T07:34:36.770794+0000","last_unstale":"2024-03-04T08:28:18.058628+0000","last_undegraded":"2024-03-04T08:28:18.058628+0000","last_fullsized":"2024-03-04T08:28:18.058628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2116,"log_dups_size":0,"ondisk_log_size":2116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:08:17.614159+0000","scrub_duration":0,"objects_trimmed":157,"snaptrim_duration":2.747632485,"stat_sum":{"num_bytes":16781418,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":100,"num_read_kb":34292,"num_write":1333,"num_write_kb":33482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"84'741","reported_seq":1039,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.133542+0000","last_change":"2024-03-04T07:33:55.544356+0000","last_active":"2024-03-04T08:28:07.133542+0000","last_peered":"2024-03-04T08:28:07.133542+0000","last_clean":"2024-03-04T08:28:07.133542+0000","last_became_active":"2024-03-04T07:33:55.541295+0000","last_became_peered":"2024-03-04T07:33:55.541295+0000","last_unstale":"2024-03-04T08:28:07.133542+0000","last_undegraded":"2024-03-04T08:28:07.133542+0000","last_fullsized":"2024-03-04T08:28:07.133542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:22.953982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":266,"num_read_kb":99836,"num_write":937,"num_write_kb":59254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"80'2189","reported_seq":4759,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.028167+0000","last_change":"2024-03-04T08:19:36.163661+0000","last_active":"2024-03-04T08:27:57.028167+0000","last_peered":"2024-03-04T08:27:57.028167+0000","last_clean":"2024-03-04T08:27:57.028167+0000","last_became_active":"2024-03-04T07:33:56.548113+0000","last_became_peered":"2024-03-04T07:33:56.548113+0000","last_unstale":"2024-03-04T08:27:57.028167+0000","last_undegraded":"2024-03-04T08:27:57.028167+0000","last_fullsized":"2024-03-04T08:27:57.028167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2189,"log_dups_size":0,"ondisk_log_size":2189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:13:34.466627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18041650100000001,"stat_sum":{"num_bytes":12585055,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":11871,"num_write":1338,"num_write_kb":24818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'700","reported_seq":976,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.141049+0000","last_change":"2024-03-04T07:33:55.538871+0000","last_active":"2024-03-04T08:27:57.141049+0000","last_peered":"2024-03-04T08:27:57.141049+0000","last_clean":"2024-03-04T08:27:57.141049+0000","last_became_active":"2024-03-04T07:33:55.537221+0000","last_became_peered":"2024-03-04T07:33:55.537221+0000","last_unstale":"2024-03-04T08:27:57.141049+0000","last_undegraded":"2024-03-04T08:27:57.141049+0000","last_fullsized":"2024-03-04T08:27:57.141049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:02:34.533330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":222,"num_read_kb":137755,"num_write":800,"num_write_kb":83574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"82'2101","reported_seq":4719,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:12.030686+0000","last_change":"2024-03-04T08:19:36.165896+0000","last_active":"2024-03-04T08:28:12.030686+0000","last_peered":"2024-03-04T08:28:12.030686+0000","last_clean":"2024-03-04T08:28:12.030686+0000","last_became_active":"2024-03-04T07:33:56.543354+0000","last_became_peered":"2024-03-04T07:33:56.543354+0000","last_unstale":"2024-03-04T08:28:12.030686+0000","last_undegraded":"2024-03-04T08:28:12.030686+0000","last_fullsized":"2024-03-04T08:28:12.030686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2101,"log_dups_size":0,"ondisk_log_size":2101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:52:28.820568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17976278000000001,"stat_sum":{"num_bytes":25168099,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":104,"num_read_kb":70965,"num_write":1341,"num_write_kb":36170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'895","reported_seq":1177,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:58.056491+0000","last_change":"2024-03-04T07:33:55.552856+0000","last_active":"2024-03-04T08:27:58.056491+0000","last_peered":"2024-03-04T08:27:58.056491+0000","last_clean":"2024-03-04T08:27:58.056491+0000","last_became_active":"2024-03-04T07:33:55.552472+0000","last_became_peered":"2024-03-04T07:33:55.552472+0000","last_unstale":"2024-03-04T08:27:58.056491+0000","last_undegraded":"2024-03-04T08:27:58.056491+0000","last_fullsized":"2024-03-04T08:27:58.056491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:36:50.409567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364919,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":192,"num_read_kb":160923,"num_write":993,"num_write_kb":79673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"82'2097","reported_seq":4734,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:56.997663+0000","last_change":"2024-03-04T08:19:36.847924+0000","last_active":"2024-03-04T08:27:56.997663+0000","last_peered":"2024-03-04T08:27:56.997663+0000","last_clean":"2024-03-04T08:27:56.997663+0000","last_became_active":"2024-03-04T07:34:36.774424+0000","last_became_peered":"2024-03-04T07:34:36.774424+0000","last_unstale":"2024-03-04T08:27:56.997663+0000","last_undegraded":"2024-03-04T08:27:56.997663+0000","last_fullsized":"2024-03-04T08:27:56.997663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2097,"log_dups_size":0,"ondisk_log_size":2097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:08:56.055772+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.85871435699999998,"stat_sum":{"num_bytes":8390169,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":104,"num_read_kb":28439,"num_write":1349,"num_write_kb":17437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"82'815","reported_seq":1145,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.141018+0000","last_change":"2024-03-04T07:33:55.560332+0000","last_active":"2024-03-04T08:27:57.141018+0000","last_peered":"2024-03-04T08:27:57.141018+0000","last_clean":"2024-03-04T08:27:57.141018+0000","last_became_active":"2024-03-04T07:33:55.560086+0000","last_became_peered":"2024-03-04T07:33:55.560086+0000","last_unstale":"2024-03-04T08:27:57.141018+0000","last_undegraded":"2024-03-04T08:27:57.141018+0000","last_fullsized":"2024-03-04T08:27:57.141018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:25:46.280657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":353,"num_read_kb":116715,"num_write":1050,"num_write_kb":64078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'2110","reported_seq":4723,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.028131+0000","last_change":"2024-03-04T08:19:36.163052+0000","last_active":"2024-03-04T08:27:57.028131+0000","last_peered":"2024-03-04T08:27:57.028131+0000","last_clean":"2024-03-04T08:27:57.028131+0000","last_became_active":"2024-03-04T07:33:56.559589+0000","last_became_peered":"2024-03-04T07:33:56.559589+0000","last_unstale":"2024-03-04T08:27:57.028131+0000","last_undegraded":"2024-03-04T08:27:57.028131+0000","last_fullsized":"2024-03-04T08:27:57.028131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2110,"log_dups_size":0,"ondisk_log_size":2110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:48:29.367330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17959140500000001,"stat_sum":{"num_bytes":4211072,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":108,"num_read_kb":1329,"num_write":1312,"num_write_kb":17303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":326,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:02.028901+0000","last_change":"2024-03-04T07:33:43.559630+0000","last_active":"2024-03-04T08:28:02.028901+0000","last_peered":"2024-03-04T08:28:02.028901+0000","last_clean":"2024-03-04T08:28:02.028901+0000","last_became_active":"2024-03-04T07:33:43.557489+0000","last_became_peered":"2024-03-04T07:33:43.557489+0000","last_unstale":"2024-03-04T08:28:02.028901+0000","last_undegraded":"2024-03-04T08:28:02.028901+0000","last_fullsized":"2024-03-04T08:28:02.028901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_clean_scrub_stamp":"2024-03-04T07:33:42.183823+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:55:35.762857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'444","reported_seq":744,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:06.999157+0000","last_change":"2024-03-04T07:33:55.548616+0000","last_active":"2024-03-04T08:28:06.999157+0000","last_peered":"2024-03-04T08:28:06.999157+0000","last_clean":"2024-03-04T08:28:06.999157+0000","last_became_active":"2024-03-04T07:33:55.547322+0000","last_became_peered":"2024-03-04T07:33:55.547322+0000","last_unstale":"2024-03-04T08:28:06.999157+0000","last_undegraded":"2024-03-04T08:28:06.999157+0000","last_fullsized":"2024-03-04T08:28:06.999157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:20:16.717483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165858,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":253,"num_read_kb":128383,"num_write":599,"num_write_kb":63285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"82'2192","reported_seq":4778,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.517999+0000","last_change":"2024-03-04T08:19:52.492136+0000","last_active":"2024-03-04T08:28:08.517999+0000","last_peered":"2024-03-04T08:28:08.517999+0000","last_clean":"2024-03-04T08:28:08.517999+0000","last_became_active":"2024-03-04T07:33:56.559581+0000","last_became_peered":"2024-03-04T07:33:56.559581+0000","last_unstale":"2024-03-04T08:28:08.517999+0000","last_undegraded":"2024-03-04T08:28:08.517999+0000","last_fullsized":"2024-03-04T08:28:08.517999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2192,"log_dups_size":0,"ondisk_log_size":2192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:24:09.497555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.4208662280000004,"stat_sum":{"num_bytes":12628568,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":97,"num_read_kb":23350,"num_write":1376,"num_write_kb":26319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'785","reported_seq":1076,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:12.030530+0000","last_change":"2024-03-04T07:33:55.563905+0000","last_active":"2024-03-04T08:28:12.030530+0000","last_peered":"2024-03-04T08:28:12.030530+0000","last_clean":"2024-03-04T08:28:12.030530+0000","last_became_active":"2024-03-04T07:33:55.563117+0000","last_became_peered":"2024-03-04T07:33:55.563117+0000","last_unstale":"2024-03-04T08:28:12.030530+0000","last_undegraded":"2024-03-04T08:28:12.030530+0000","last_fullsized":"2024-03-04T08:28:12.030530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:53:02.522757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":261,"num_read_kb":111921,"num_write":932,"num_write_kb":63511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'2022","reported_seq":4660,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.141979+0000","last_change":"2024-03-04T08:19:49.113482+0000","last_active":"2024-03-04T08:28:07.141979+0000","last_peered":"2024-03-04T08:28:07.141979+0000","last_clean":"2024-03-04T08:28:07.141979+0000","last_became_active":"2024-03-04T07:33:56.530776+0000","last_became_peered":"2024-03-04T07:33:56.530776+0000","last_unstale":"2024-03-04T08:28:07.141979+0000","last_undegraded":"2024-03-04T08:28:07.141979+0000","last_fullsized":"2024-03-04T08:28:07.141979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2022,"log_dups_size":0,"ondisk_log_size":2022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:59:39.265011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.715426973,"stat_sum":{"num_bytes":4196564,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":94,"num_read_kb":11883,"num_write":1291,"num_write_kb":12915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"84'1859","reported_seq":3279,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.141993+0000","last_change":"2024-03-04T07:33:55.538631+0000","last_active":"2024-03-04T08:28:07.141993+0000","last_peered":"2024-03-04T08:28:07.141993+0000","last_clean":"2024-03-04T08:28:07.141993+0000","last_became_active":"2024-03-04T07:33:55.537104+0000","last_became_peered":"2024-03-04T07:33:55.537104+0000","last_unstale":"2024-03-04T08:28:07.141993+0000","last_undegraded":"2024-03-04T08:28:07.141993+0000","last_fullsized":"2024-03-04T08:28:07.141993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1859,"log_dups_size":0,"ondisk_log_size":1859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:27:06.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165914,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1355,"num_read_kb":144971,"num_write":1920,"num_write_kb":84003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"82'2174","reported_seq":4784,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057686+0000","last_change":"2024-03-04T08:19:35.433750+0000","last_active":"2024-03-04T08:28:08.057686+0000","last_peered":"2024-03-04T08:28:08.057686+0000","last_clean":"2024-03-04T08:28:08.057686+0000","last_became_active":"2024-03-04T07:33:56.540536+0000","last_became_peered":"2024-03-04T07:33:56.540536+0000","last_unstale":"2024-03-04T08:28:08.057686+0000","last_undegraded":"2024-03-04T08:28:08.057686+0000","last_fullsized":"2024-03-04T08:28:08.057686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2174,"log_dups_size":0,"ondisk_log_size":2174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:23:13.702668+0000","scrub_duration":0,"objects_trimmed":166,"snaptrim_duration":2.7381961829999999,"stat_sum":{"num_bytes":19342025,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":108,"num_read_kb":53174,"num_write":1367,"num_write_kb":31960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"84'854","reported_seq":1121,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:06.999129+0000","last_change":"2024-03-04T07:33:55.574740+0000","last_active":"2024-03-04T08:28:06.999129+0000","last_peered":"2024-03-04T08:28:06.999129+0000","last_clean":"2024-03-04T08:28:06.999129+0000","last_became_active":"2024-03-04T07:33:55.574129+0000","last_became_peered":"2024-03-04T07:33:55.574129+0000","last_unstale":"2024-03-04T08:28:06.999129+0000","last_undegraded":"2024-03-04T08:28:06.999129+0000","last_fullsized":"2024-03-04T08:28:06.999129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:40:57.891803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":189,"num_read_kb":128430,"num_write":957,"num_write_kb":67580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"82'2115","reported_seq":4827,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.133511+0000","last_change":"2024-03-04T08:19:33.358585+0000","last_active":"2024-03-04T08:28:07.133511+0000","last_peered":"2024-03-04T08:28:07.133511+0000","last_clean":"2024-03-04T08:28:07.133511+0000","last_became_active":"2024-03-04T07:33:56.541422+0000","last_became_peered":"2024-03-04T07:33:56.541422+0000","last_unstale":"2024-03-04T08:28:07.133511+0000","last_undegraded":"2024-03-04T08:28:07.133511+0000","last_fullsized":"2024-03-04T08:28:07.133511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2115,"log_dups_size":0,"ondisk_log_size":2115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:02:25.299647+0000","scrub_duration":0,"objects_trimmed":76,"snaptrim_duration":0.66844205800000001,"stat_sum":{"num_bytes":8391536,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":186,"num_read_kb":5163,"num_write":1360,"num_write_kb":16715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"85'649","reported_seq":906,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057737+0000","last_change":"2024-03-04T07:33:55.539317+0000","last_active":"2024-03-04T08:28:08.057737+0000","last_peered":"2024-03-04T08:28:08.057737+0000","last_clean":"2024-03-04T08:28:08.057737+0000","last_became_active":"2024-03-04T07:33:55.539017+0000","last_became_peered":"2024-03-04T07:33:55.539017+0000","last_unstale":"2024-03-04T08:28:08.057737+0000","last_undegraded":"2024-03-04T08:28:08.057737+0000","last_fullsized":"2024-03-04T08:28:08.057737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:10.787316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":165,"num_read_kb":131936,"num_write":747,"num_write_kb":79565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"82'2168","reported_seq":4796,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.001381+0000","last_change":"2024-03-04T08:19:36.855176+0000","last_active":"2024-03-04T08:28:22.001381+0000","last_peered":"2024-03-04T08:28:22.001381+0000","last_clean":"2024-03-04T08:28:22.001381+0000","last_became_active":"2024-03-04T07:33:56.553742+0000","last_became_peered":"2024-03-04T07:33:56.553742+0000","last_unstale":"2024-03-04T08:28:22.001381+0000","last_undegraded":"2024-03-04T08:28:22.001381+0000","last_fullsized":"2024-03-04T08:28:22.001381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2168,"log_dups_size":0,"ondisk_log_size":2168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:27.955824+0000","scrub_duration":0,"objects_trimmed":68,"snaptrim_duration":0.86588785899999998,"stat_sum":{"num_bytes":25169512,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":118,"num_read_kb":50143,"num_write":1386,"num_write_kb":35653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"82'874","reported_seq":1154,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.032175+0000","last_change":"2024-03-04T07:33:55.569442+0000","last_active":"2024-03-04T08:28:22.032175+0000","last_peered":"2024-03-04T08:28:22.032175+0000","last_clean":"2024-03-04T08:28:22.032175+0000","last_became_active":"2024-03-04T07:33:55.567809+0000","last_became_peered":"2024-03-04T07:33:55.567809+0000","last_unstale":"2024-03-04T08:28:22.032175+0000","last_undegraded":"2024-03-04T08:28:22.032175+0000","last_fullsized":"2024-03-04T08:28:22.032175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:58.551866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971542,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":208,"num_read_kb":126955,"num_write":1015,"num_write_kb":67473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"80'2113","reported_seq":4760,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057728+0000","last_change":"2024-03-04T08:19:35.449422+0000","last_active":"2024-03-04T08:28:08.057728+0000","last_peered":"2024-03-04T08:28:08.057728+0000","last_clean":"2024-03-04T08:28:08.057728+0000","last_became_active":"2024-03-04T07:33:56.542775+0000","last_became_peered":"2024-03-04T07:33:56.542775+0000","last_unstale":"2024-03-04T08:28:08.057728+0000","last_undegraded":"2024-03-04T08:28:08.057728+0000","last_fullsized":"2024-03-04T08:28:08.057728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2113,"log_dups_size":0,"ondisk_log_size":2113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:24:08.926542+0000","scrub_duration":0,"objects_trimmed":162,"snaptrim_duration":2.754112347,"stat_sum":{"num_bytes":21241084,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":124,"num_read_kb":50364,"num_write":1380,"num_write_kb":30457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"82'1819","reported_seq":2086,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057769+0000","last_change":"2024-03-04T07:33:55.545598+0000","last_active":"2024-03-04T08:28:08.057769+0000","last_peered":"2024-03-04T08:28:08.057769+0000","last_clean":"2024-03-04T08:28:08.057769+0000","last_became_active":"2024-03-04T07:33:55.545262+0000","last_became_peered":"2024-03-04T07:33:55.545262+0000","last_unstale":"2024-03-04T08:28:08.057769+0000","last_undegraded":"2024-03-04T08:28:08.057769+0000","last_fullsized":"2024-03-04T08:28:08.057769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1819,"log_dups_size":0,"ondisk_log_size":1819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:01.667243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":214,"num_read_kb":99465,"num_write":3832,"num_write_kb":59968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"80'1975","reported_seq":4630,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.518040+0000","last_change":"2024-03-04T08:19:52.454759+0000","last_active":"2024-03-04T08:28:08.518040+0000","last_peered":"2024-03-04T08:28:08.518040+0000","last_clean":"2024-03-04T08:28:08.518040+0000","last_became_active":"2024-03-04T07:33:56.550604+0000","last_became_peered":"2024-03-04T07:33:56.550604+0000","last_unstale":"2024-03-04T08:28:08.518040+0000","last_undegraded":"2024-03-04T08:28:08.518040+0000","last_fullsized":"2024-03-04T08:28:08.518040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1975,"log_dups_size":0,"ondisk_log_size":1975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:17:11.625595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.366898226,"stat_sum":{"num_bytes":4218390,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":111,"num_read_kb":1574,"num_write":1260,"num_write_kb":17353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'484","reported_seq":758,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.002985+0000","last_change":"2024-03-04T07:33:55.567220+0000","last_active":"2024-03-04T08:28:07.002985+0000","last_peered":"2024-03-04T08:28:07.002985+0000","last_clean":"2024-03-04T08:28:07.002985+0000","last_became_active":"2024-03-04T07:33:55.566600+0000","last_became_peered":"2024-03-04T07:33:55.566600+0000","last_unstale":"2024-03-04T08:28:07.002985+0000","last_undegraded":"2024-03-04T08:28:07.002985+0000","last_fullsized":"2024-03-04T08:28:07.002985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.876938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":246,"num_read_kb":67057,"num_write":625,"num_write_kb":42779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"80'2039","reported_seq":4677,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142035+0000","last_change":"2024-03-04T08:19:49.347949+0000","last_active":"2024-03-04T08:28:07.142035+0000","last_peered":"2024-03-04T08:28:07.142035+0000","last_clean":"2024-03-04T08:28:07.142035+0000","last_became_active":"2024-03-04T07:33:56.543836+0000","last_became_peered":"2024-03-04T07:33:56.543836+0000","last_unstale":"2024-03-04T08:28:07.142035+0000","last_undegraded":"2024-03-04T08:28:07.142035+0000","last_fullsized":"2024-03-04T08:28:07.142035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2039,"log_dups_size":0,"ondisk_log_size":2039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:44:01.157492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.94967019,"stat_sum":{"num_bytes":8393261,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":91,"num_read_kb":20301,"num_write":1337,"num_write_kb":17823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'627","reported_seq":911,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.029678+0000","last_change":"2024-03-04T07:33:55.566886+0000","last_active":"2024-03-04T08:28:07.029678+0000","last_peered":"2024-03-04T08:28:07.029678+0000","last_clean":"2024-03-04T08:28:07.029678+0000","last_became_active":"2024-03-04T07:33:55.566048+0000","last_became_peered":"2024-03-04T07:33:55.566048+0000","last_unstale":"2024-03-04T08:28:07.029678+0000","last_undegraded":"2024-03-04T08:28:07.029678+0000","last_fullsized":"2024-03-04T08:28:07.029678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:45:27.190438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":244,"num_read_kb":91476,"num_write":742,"num_write_kb":42693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"80'2170","reported_seq":4766,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.133468+0000","last_change":"2024-03-04T08:19:33.369171+0000","last_active":"2024-03-04T08:28:07.133468+0000","last_peered":"2024-03-04T08:28:07.133468+0000","last_clean":"2024-03-04T08:28:07.133468+0000","last_became_active":"2024-03-04T07:33:56.531901+0000","last_became_peered":"2024-03-04T07:33:56.531901+0000","last_unstale":"2024-03-04T08:28:07.133468+0000","last_undegraded":"2024-03-04T08:28:07.133468+0000","last_fullsized":"2024-03-04T08:28:07.133468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2170,"log_dups_size":0,"ondisk_log_size":2170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:43:17.466639+0000","scrub_duration":0,"objects_trimmed":68,"snaptrim_duration":0.67861760500000001,"stat_sum":{"num_bytes":12583051,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":98,"num_read_kb":23140,"num_write":1358,"num_write_kb":19821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"85'800","reported_seq":1103,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057715+0000","last_change":"2024-03-04T07:33:55.557568+0000","last_active":"2024-03-04T08:28:08.057715+0000","last_peered":"2024-03-04T08:28:08.057715+0000","last_clean":"2024-03-04T08:28:08.057715+0000","last_became_active":"2024-03-04T07:33:55.557330+0000","last_became_peered":"2024-03-04T07:33:55.557330+0000","last_unstale":"2024-03-04T08:28:08.057715+0000","last_undegraded":"2024-03-04T08:28:08.057715+0000","last_fullsized":"2024-03-04T08:28:08.057715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:34.956318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18594593,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":285,"num_read_kb":97953,"num_write":952,"num_write_kb":44586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"80'2245","reported_seq":4842,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.517974+0000","last_change":"2024-03-04T08:19:52.542997+0000","last_active":"2024-03-04T08:28:08.517974+0000","last_peered":"2024-03-04T08:28:08.517974+0000","last_clean":"2024-03-04T08:28:08.517974+0000","last_became_active":"2024-03-04T07:33:56.552658+0000","last_became_peered":"2024-03-04T07:33:56.552658+0000","last_unstale":"2024-03-04T08:28:08.517974+0000","last_undegraded":"2024-03-04T08:28:08.517974+0000","last_fullsized":"2024-03-04T08:28:08.517974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2245,"log_dups_size":0,"ondisk_log_size":2245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:02:38.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.4718202119999999,"stat_sum":{"num_bytes":2645,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":101,"num_read_kb":1927,"num_write":1438,"num_write_kb":12677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'827","reported_seq":1131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:18.058654+0000","last_change":"2024-03-04T07:33:55.536480+0000","last_active":"2024-03-04T08:28:18.058654+0000","last_peered":"2024-03-04T08:28:18.058654+0000","last_clean":"2024-03-04T08:28:18.058654+0000","last_became_active":"2024-03-04T07:33:55.536139+0000","last_became_peered":"2024-03-04T07:33:55.536139+0000","last_unstale":"2024-03-04T08:28:18.058654+0000","last_undegraded":"2024-03-04T08:28:18.058654+0000","last_fullsized":"2024-03-04T08:28:18.058654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:12.778026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":276,"num_read_kb":124915,"num_write":1046,"num_write_kb":68013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"80'2139","reported_seq":4762,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:58.516982+0000","last_change":"2024-03-04T08:19:52.574166+0000","last_active":"2024-03-04T08:27:58.516982+0000","last_peered":"2024-03-04T08:27:58.516982+0000","last_clean":"2024-03-04T08:27:58.516982+0000","last_became_active":"2024-03-04T07:33:56.546280+0000","last_became_peered":"2024-03-04T07:33:56.546280+0000","last_unstale":"2024-03-04T08:27:58.516982+0000","last_undegraded":"2024-03-04T08:27:58.516982+0000","last_fullsized":"2024-03-04T08:27:58.516982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2139,"log_dups_size":0,"ondisk_log_size":2139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:04:37.188813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.5027303859999996,"stat_sum":{"num_bytes":8389588,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":116,"num_read_kb":19661,"num_write":1373,"num_write_kb":18499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'895","reported_seq":1206,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.028091+0000","last_change":"2024-03-04T07:33:55.572540+0000","last_active":"2024-03-04T08:27:57.028091+0000","last_peered":"2024-03-04T08:27:57.028091+0000","last_clean":"2024-03-04T08:27:57.028091+0000","last_became_active":"2024-03-04T07:33:55.571612+0000","last_became_peered":"2024-03-04T07:33:55.571612+0000","last_unstale":"2024-03-04T08:27:57.028091+0000","last_undegraded":"2024-03-04T08:27:57.028091+0000","last_fullsized":"2024-03-04T08:27:57.028091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:01:43.741423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":286,"num_read_kb":176599,"num_write":1032,"num_write_kb":79834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"80'2075","reported_seq":4727,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.005213+0000","last_change":"2024-03-04T08:19:34.379981+0000","last_active":"2024-03-04T08:28:22.005213+0000","last_peered":"2024-03-04T08:28:22.005213+0000","last_clean":"2024-03-04T08:28:22.005213+0000","last_became_active":"2024-03-04T07:33:56.532245+0000","last_became_peered":"2024-03-04T07:33:56.532245+0000","last_unstale":"2024-03-04T08:28:22.005213+0000","last_undegraded":"2024-03-04T08:28:22.005213+0000","last_fullsized":"2024-03-04T08:28:22.005213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2075,"log_dups_size":0,"ondisk_log_size":2075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:17:47.216527+0000","scrub_duration":0,"objects_trimmed":129,"snaptrim_duration":1.668147552,"stat_sum":{"num_bytes":16789440,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":116,"num_read_kb":38061,"num_write":1331,"num_write_kb":29084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"82'2080","reported_seq":4692,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.984492+0000","last_change":"2024-03-04T08:19:36.839132+0000","last_active":"2024-03-04T08:26:26.984492+0000","last_peered":"2024-03-04T08:26:26.984492+0000","last_clean":"2024-03-04T08:26:26.984492+0000","last_became_active":"2024-03-04T07:33:56.563473+0000","last_became_peered":"2024-03-04T07:33:56.563473+0000","last_unstale":"2024-03-04T08:26:26.984492+0000","last_undegraded":"2024-03-04T08:26:26.984492+0000","last_fullsized":"2024-03-04T08:26:26.984492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2080,"log_dups_size":0,"ondisk_log_size":2080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:46:20.229965+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.84949308899999998,"stat_sum":{"num_bytes":8388810,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":102,"num_read_kb":13463,"num_write":1308,"num_write_kb":22727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'689","reported_seq":944,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014567+0000","last_change":"2024-03-04T07:33:55.575688+0000","last_active":"2024-03-04T08:26:27.014567+0000","last_peered":"2024-03-04T08:26:27.014567+0000","last_clean":"2024-03-04T08:26:27.014567+0000","last_became_active":"2024-03-04T07:33:55.574766+0000","last_became_peered":"2024-03-04T07:33:55.574766+0000","last_unstale":"2024-03-04T08:26:27.014567+0000","last_undegraded":"2024-03-04T08:26:27.014567+0000","last_fullsized":"2024-03-04T08:26:27.014567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:32:19.708016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":185,"num_read_kb":103365,"num_write":788,"num_write_kb":63233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"80'2223","reported_seq":4823,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.984465+0000","last_change":"2024-03-04T08:19:36.844614+0000","last_active":"2024-03-04T08:26:26.984465+0000","last_peered":"2024-03-04T08:26:26.984465+0000","last_clean":"2024-03-04T08:26:26.984465+0000","last_became_active":"2024-03-04T07:33:56.563474+0000","last_became_peered":"2024-03-04T07:33:56.563474+0000","last_unstale":"2024-03-04T08:26:26.984465+0000","last_undegraded":"2024-03-04T08:26:26.984465+0000","last_fullsized":"2024-03-04T08:26:26.984465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2223,"log_dups_size":0,"ondisk_log_size":2223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:11:22.319394+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.85581109,"stat_sum":{"num_bytes":20978408,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":117,"num_read_kb":36831,"num_write":1409,"num_write_kb":34422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"85'764","reported_seq":1068,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014595+0000","last_change":"2024-03-04T07:33:55.570016+0000","last_active":"2024-03-04T08:26:27.014595+0000","last_peered":"2024-03-04T08:26:27.014595+0000","last_clean":"2024-03-04T08:26:27.014595+0000","last_became_active":"2024-03-04T07:33:55.568749+0000","last_became_peered":"2024-03-04T07:33:55.568749+0000","last_unstale":"2024-03-04T08:26:27.014595+0000","last_undegraded":"2024-03-04T08:26:27.014595+0000","last_fullsized":"2024-03-04T08:26:27.014595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:01.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":279,"num_read_kb":188218,"num_write":863,"num_write_kb":100106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"80'2088","reported_seq":4689,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142049+0000","last_change":"2024-03-04T08:19:49.462793+0000","last_active":"2024-03-04T08:28:07.142049+0000","last_peered":"2024-03-04T08:28:07.142049+0000","last_clean":"2024-03-04T08:28:07.142049+0000","last_became_active":"2024-03-04T07:33:56.561675+0000","last_became_peered":"2024-03-04T07:33:56.561675+0000","last_unstale":"2024-03-04T08:28:07.142049+0000","last_undegraded":"2024-03-04T08:28:07.142049+0000","last_fullsized":"2024-03-04T08:28:07.142049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2088,"log_dups_size":0,"ondisk_log_size":2088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:36:21.385199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0645439379999999,"stat_sum":{"num_bytes":8398756,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":91,"num_read_kb":13948,"num_write":1308,"num_write_kb":18354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'844","reported_seq":1117,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:03.517495+0000","last_change":"2024-03-04T07:33:55.538337+0000","last_active":"2024-03-04T08:28:03.517495+0000","last_peered":"2024-03-04T08:28:03.517495+0000","last_clean":"2024-03-04T08:28:03.517495+0000","last_became_active":"2024-03-04T07:33:55.538010+0000","last_became_peered":"2024-03-04T07:33:55.538010+0000","last_unstale":"2024-03-04T08:28:03.517495+0000","last_undegraded":"2024-03-04T08:28:03.517495+0000","last_fullsized":"2024-03-04T08:28:03.517495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:47:15.711205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":199,"num_read_kb":128064,"num_write":938,"num_write_kb":63013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'2071","reported_seq":4691,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:03.517472+0000","last_change":"2024-03-04T08:19:52.402904+0000","last_active":"2024-03-04T08:28:03.517472+0000","last_peered":"2024-03-04T08:28:03.517472+0000","last_clean":"2024-03-04T08:28:03.517472+0000","last_became_active":"2024-03-04T07:33:56.558799+0000","last_became_peered":"2024-03-04T07:33:56.558799+0000","last_unstale":"2024-03-04T08:28:03.517472+0000","last_undegraded":"2024-03-04T08:28:03.517472+0000","last_fullsized":"2024-03-04T08:28:03.517472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2071,"log_dups_size":0,"ondisk_log_size":2071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:34:51.677019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.3308552689999997,"stat_sum":{"num_bytes":16779138,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":99,"num_read_kb":47695,"num_write":1322,"num_write_kb":28136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'627","reported_seq":937,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142064+0000","last_change":"2024-03-04T07:33:55.566294+0000","last_active":"2024-03-04T08:28:07.142064+0000","last_peered":"2024-03-04T08:28:07.142064+0000","last_clean":"2024-03-04T08:28:07.142064+0000","last_became_active":"2024-03-04T07:33:55.565941+0000","last_became_peered":"2024-03-04T07:33:55.565941+0000","last_unstale":"2024-03-04T08:28:07.142064+0000","last_undegraded":"2024-03-04T08:28:07.142064+0000","last_fullsized":"2024-03-04T08:28:07.142064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:03.115085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":306,"num_read_kb":111793,"num_write":679,"num_write_kb":46654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":758811247,"num_objects":4817,"num_object_clones":0,"num_object_copies":9634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4817,"num_whiteouts":0,"num_read":7109,"num_read_kb":1683514,"num_write":86591,"num_write_kb":1471650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1519542272,"data_stored":1517622494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137094,"ondisk_log_size":137094,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":589610469,"num_objects":1415,"num_object_clones":0,"num_object_copies":2830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1415,"num_whiteouts":0,"num_read":9520,"num_read_kb":4046989,"num_write":33636,"num_write_kb":2138987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1203,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1179369472,"data_stored":1179220938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27575,"ondisk_log_size":27575,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424510129,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537624,"kb_used_data":362128,"kb_used_omap":1,"kb_used_meta":175486,"kb_avail":93834216,"statfs":{"total":96636764160,"available":96086237184,"internally_reserved":0,"allocated":370819072,"data_stored":370026846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":179698119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510128,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466432,"kb_used_data":300664,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":93905408,"statfs":{"total":96636764160,"available":96159137792,"internally_reserved":0,"allocated":307879936,"data_stored":307237685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542833,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564504,"kb_used_data":389072,"kb_used_omap":1,"kb_used_meta":175422,"kb_avail":93807336,"statfs":{"total":96636764160,"available":96058712064,"internally_reserved":0,"allocated":398409728,"data_stored":397634775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":179632586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542832,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505096,"kb_used_data":321344,"kb_used_omap":1,"kb_used_meta":183742,"kb_avail":93866744,"statfs":{"total":96636764160,"available":96119545856,"internally_reserved":0,"allocated":329056256,"data_stored":328434784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188152262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542832,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583216,"kb_used_data":403556,"kb_used_omap":1,"kb_used_meta":179646,"kb_avail":93788624,"statfs":{"total":96636764160,"available":96039550976,"internally_reserved":0,"allocated":413241344,"data_stored":412455445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183957957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542833,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509020,"kb_used_data":329360,"kb_used_omap":1,"kb_used_meta":179646,"kb_avail":93862820,"statfs":{"total":96636764160,"available":96115527680,"internally_reserved":0,"allocated":337264640,"data_stored":336553162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183957957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542833,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340760,"kb_used_data":241612,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94031080,"statfs":{"total":96636764160,"available":96287825920,"internally_reserved":0,"allocated":247410688,"data_stored":246748656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542831,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462692,"kb_used_data":296920,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":93909148,"statfs":{"total":96636764160,"available":96162967552,"internally_reserved":0,"allocated":304046080,"data_stored":303311191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":169736651},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121651200,"data_stored":121635476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157036544,"data_stored":157012200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92295168,"data_stored":92275506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":151015424,"data_stored":150997655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":104878080,"data_stored":104864685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180957184,"data_stored":180935092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":168370176,"data_stored":168353802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":203165696,"data_stored":203146522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":180797440,"data_stored":180538050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149839872,"data_stored":149678188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":154112000,"data_stored":153925853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":218800128,"data_stored":218481894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":231383040,"data_stored":231141180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":231280640,"data_stored":230973056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":159682560,"data_stored":159533685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":193646592,"data_stored":193350588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T08:30:45.563 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T08:30:45.737 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.739+0000 7f27d5000640 1 Processor -- start 2024-03-04T08:30:45.738 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.739+0000 7f27d5000640 1 -- start start 2024-03-04T08:30:45.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27d5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27d00c8200 0x7f27d0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27d5000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27d0152bd0 0x7f27d01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27d5000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d01591f0 con 0x7f27d01527f0 2024-03-04T08:30:45.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27d5000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27d0071df0 con 0x7f27d0152bd0 2024-03-04T08:30:45.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27d5000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01597b0 con 0x7f27d00c8200 2024-03-04T08:30:45.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27ce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27d0152bd0 0x7f27d01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27ce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f27d0152bd0 0x7f27d01587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52980/0 (socket says 172.21.15.113:52980) 2024-03-04T08:30:45.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27ce800640 1 -- 172.21.15.113:0/165015834 learned_addr learned my addr 172.21.15.113:0/165015834 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:45.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27c5400640 1 --2- 172.21.15.113:0/165015834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27d00c8200 0x7f27d0070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27ce800640 1 -- 172.21.15.113:0/165015834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27d00c8200 msgr2=0x7f27d0070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:45.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27ce800640 1 --2- 172.21.15.113:0/165015834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27d00c8200 0x7f27d0070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27ce800640 1 -- 172.21.15.113:0/165015834 >> v1:172.21.15.136:6789/0 conn(0x7f27d01527f0 legacy=0x7f27d0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.743+0000 7f27ce800640 1 -- 172.21.15.113:0/165015834 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d01594d0 con 0x7f27d0152bd0 2024-03-04T08:30:45.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27ce800640 1 --2- 172.21.15.113:0/165015834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d0152bd0 0x7f27d01587d0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009e50 tx=0x7f27c801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f6d39da7a52965 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:45.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27cd400640 1 -- 172.21.15.113:0/165015834 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80a57f0 con 0x7f27d0152bd0 2024-03-04T08:30:45.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27cd400640 1 -- 172.21.15.113:0/165015834 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c80a5990 con 0x7f27d0152bd0 2024-03-04T08:30:45.742 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27cd400640 1 -- 172.21.15.113:0/165015834 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80a5d60 con 0x7f27d0152bd0 2024-03-04T08:30:45.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 -- 172.21.15.113:0/165015834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d0152bd0 msgr2=0x7f27d01587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 --2- 172.21.15.113:0/165015834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d0152bd0 0x7f27d01587d0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f27c8009e50 tx=0x7f27c801ca30 comp rx=0 tx=0).stop 2024-03-04T08:30:45.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 -- 172.21.15.113:0/165015834 shutdown_connections 2024-03-04T08:30:45.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 --2- 172.21.15.113:0/165015834 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d0152bd0 0x7f27d01587d0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 --2- 172.21.15.113:0/165015834 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27d00c8200 0x7f27d0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 -- 172.21.15.113:0/165015834 >> 172.21.15.113:0/165015834 conn(0x7f27d006e820 msgr2=0x7f27d006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:45.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 -- 172.21.15.113:0/165015834 shutdown_connections 2024-03-04T08:30:45.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 -- 172.21.15.113:0/165015834 wait complete. 2024-03-04T08:30:45.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 Processor -- start 2024-03-04T08:30:45.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 -- start start 2024-03-04T08:30:45.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27d00c8200 0x7f27d0154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d01527f0 0x7f27d0148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27d0152bd0 0x7f27d0148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d01654f0 con 0x7f27d01527f0 2024-03-04T08:30:45.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d0165370 con 0x7f27d0152bd0 2024-03-04T08:30:45.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.747+0000 7f27d5000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d0165670 con 0x7f27d00c8200 2024-03-04T08:30:45.747 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d01527f0 0x7f27d0148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c5400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d01527f0 0x7f27d0148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52984/0 (socket says 172.21.15.113:52984) 2024-03-04T08:30:45.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c5400640 1 -- 172.21.15.113:0/1848551569 learned_addr learned my addr 172.21.15.113:0/1848551569 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:45.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27cde00640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27d00c8200 0x7f27d0154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27ce800640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27d0152bd0 0x7f27d0148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c5400640 1 -- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27d00c8200 msgr2=0x7f27d0154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c5400640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27d00c8200 0x7f27d0154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c5400640 1 -- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27d0152bd0 msgr2=0x7f27d0148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.748 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c5400640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27d0152bd0 0x7f27d0148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c5400640 1 -- 172.21.15.113:0/1848551569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d00c4550 con 0x7f27d01527f0 2024-03-04T08:30:45.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27ce800640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27d0152bd0 0x7f27d0148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T08:30:45.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c5400640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d01527f0 0x7f27d0148a70 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f27b800e930 tx=0x7f27b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:45.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c6a00640 1 -- 172.21.15.113:0/1848551569 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8017070 con 0x7f27d01527f0 2024-03-04T08:30:45.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c6a00640 1 -- 172.21.15.113:0/1848551569 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b800ca00 con 0x7f27d01527f0 2024-03-04T08:30:45.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c6a00640 1 -- 172.21.15.113:0/1848551569 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b800cdf0 con 0x7f27d01527f0 2024-03-04T08:30:45.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27d5000640 1 -- 172.21.15.113:0/1848551569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d00c47f0 con 0x7f27d01527f0 2024-03-04T08:30:45.749 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27d5000640 1 -- 172.21.15.113:0/1848551569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d00c4db0 con 0x7f27d01527f0 2024-03-04T08:30:45.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27d5000640 1 -- 172.21.15.113:0/1848551569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d0075c00 con 0x7f27d01527f0 2024-03-04T08:30:45.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.751+0000 7f27c6a00640 1 -- 172.21.15.113:0/1848551569 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27b801a070 con 0x7f27d01527f0 2024-03-04T08:30:45.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.755+0000 7f27c6a00640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27ac066f60 0x7f27ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:45.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.755+0000 7f27c6a00640 1 -- 172.21.15.113:0/1848551569 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7f27b8085b70 con 0x7f27d01527f0 2024-03-04T08:30:45.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.755+0000 7f27cde00640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27ac066f60 0x7f27ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:45.751 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.755+0000 7f27cde00640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27ac066f60 0x7f27ac069420 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f27c0001650 tx=0x7f27c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:45.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.759+0000 7f27c6a00640 1 -- 172.21.15.113:0/1848551569 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f27b8056100 con 0x7f27d01527f0 2024-03-04T08:30:45.904 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.907+0000 7f27d5000640 1 -- 172.21.15.113:0/1848551569 --> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f27d0162f30 con 0x7f27ac066f60 2024-03-04T08:30:45.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.927+0000 7f27c6a00640 1 -- 172.21.15.113:0/1848551569 <== mgr.4103 v2:172.21.15.136:6832/3353359815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256615 (secure 0 0 0) 0x7f27d0162f30 con 0x7f27ac066f60 2024-03-04T08:30:45.925 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:45.927 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-03-04T08:30:45.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 -- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27ac066f60 msgr2=0x7f27ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27ac066f60 0x7f27ac069420 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f27c0001650 tx=0x7f27c0009040 comp rx=0 tx=0).stop 2024-03-04T08:30:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 -- 172.21.15.113:0/1848551569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d01527f0 msgr2=0x7f27d0148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d01527f0 0x7f27d0148a70 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f27b800e930 tx=0x7f27b800edf0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 -- 172.21.15.113:0/1848551569 shutdown_connections 2024-03-04T08:30:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f27d0152bd0 0x7f27d0148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7f27ac066f60 0x7f27ac069420 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27d01527f0 0x7f27d0148a70 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 --2- 172.21.15.113:0/1848551569 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f27d00c8200 0x7f27d0154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:45.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 -- 172.21.15.113:0/1848551569 >> 172.21.15.113:0/1848551569 conn(0x7f27d006e820 msgr2=0x7f27d0158280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 -- 172.21.15.113:0/1848551569 shutdown_connections 2024-03-04T08:30:45.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:45.931+0000 7f27d5000640 1 -- 172.21.15.113:0/1848551569 wait complete. 2024-03-04T08:30:45.946 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":1789,"stamp":"2024-03-04T08:30:44.638868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1349012084,"num_objects":6234,"num_object_clones":0,"num_object_copies":12468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6234,"num_whiteouts":0,"num_read":16745,"num_read_kb":5730603,"num_write":120386,"num_write_kb":3613053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1203,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164786,"ondisk_log_size":164786,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3969344,"kb_used_data":2644656,"kb_used_omap":12,"kb_used_meta":1324595,"kb_avail":751005376,"statfs":{"total":773094113280,"available":769029505024,"internally_reserved":0,"allocated":2708127744,"data_stored":2702402544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1356385846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004182"},"pg_stats":[{"pgid":"3.3f","version":"82'1995","reported_seq":4635,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.029763+0000","last_change":"2024-03-04T08:19:36.158827+0000","last_active":"2024-03-04T08:28:07.029763+0000","last_peered":"2024-03-04T08:28:07.029763+0000","last_clean":"2024-03-04T08:28:07.029763+0000","last_became_active":"2024-03-04T07:33:56.559031+0000","last_became_peered":"2024-03-04T07:33:56.559031+0000","last_unstale":"2024-03-04T08:28:07.029763+0000","last_undegraded":"2024-03-04T08:28:07.029763+0000","last_fullsized":"2024-03-04T08:28:07.029763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1995,"log_dups_size":0,"ondisk_log_size":1995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:14:15.903092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17246109400000001,"stat_sum":{"num_bytes":4212510,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":96,"num_read_kb":14433,"num_write":1280,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"80'1982","reported_seq":4617,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057758+0000","last_change":"2024-03-04T08:19:35.479098+0000","last_active":"2024-03-04T08:28:08.057758+0000","last_peered":"2024-03-04T08:28:08.057758+0000","last_clean":"2024-03-04T08:28:08.057758+0000","last_became_active":"2024-03-04T07:33:56.546547+0000","last_became_peered":"2024-03-04T07:33:56.546547+0000","last_unstale":"2024-03-04T08:28:08.057758+0000","last_undegraded":"2024-03-04T08:28:08.057758+0000","last_fullsized":"2024-03-04T08:28:08.057758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1982,"log_dups_size":0,"ondisk_log_size":1982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:14:54.677102+0000","scrub_duration":0,"objects_trimmed":173,"snaptrim_duration":2.7820693949999997,"stat_sum":{"num_bytes":8393198,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":106,"num_read_kb":21761,"num_write":1218,"num_write_kb":15425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"80'2113","reported_seq":4724,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142019+0000","last_change":"2024-03-04T08:19:49.392733+0000","last_active":"2024-03-04T08:28:07.142019+0000","last_peered":"2024-03-04T08:28:07.142019+0000","last_clean":"2024-03-04T08:28:07.142019+0000","last_became_active":"2024-03-04T07:33:56.530488+0000","last_became_peered":"2024-03-04T07:33:56.530488+0000","last_unstale":"2024-03-04T08:28:07.142019+0000","last_undegraded":"2024-03-04T08:28:07.142019+0000","last_fullsized":"2024-03-04T08:28:07.142019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2113,"log_dups_size":0,"ondisk_log_size":2113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:02:21.539335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.9646412140000002,"stat_sum":{"num_bytes":12588036,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":103,"num_read_kb":26195,"num_write":1342,"num_write_kb":31711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'2188","reported_seq":4775,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:42.133523+0000","last_change":"2024-03-04T08:19:32.727352+0000","last_active":"2024-03-04T08:26:42.133523+0000","last_peered":"2024-03-04T08:26:42.133523+0000","last_clean":"2024-03-04T08:26:42.133523+0000","last_became_active":"2024-03-04T07:34:36.767991+0000","last_became_peered":"2024-03-04T07:34:36.767991+0000","last_unstale":"2024-03-04T08:26:42.133523+0000","last_undegraded":"2024-03-04T08:26:42.133523+0000","last_fullsized":"2024-03-04T08:26:42.133523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2188,"log_dups_size":0,"ondisk_log_size":2188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:52.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019148479999999999,"stat_sum":{"num_bytes":12586995,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":106,"num_read_kb":25892,"num_write":1360,"num_write_kb":20414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"82'2280","reported_seq":4856,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014533+0000","last_change":"2024-03-04T08:19:36.167121+0000","last_active":"2024-03-04T08:26:27.014533+0000","last_peered":"2024-03-04T08:26:27.014533+0000","last_clean":"2024-03-04T08:26:27.014533+0000","last_became_active":"2024-03-04T07:33:56.540351+0000","last_became_peered":"2024-03-04T07:33:56.540351+0000","last_unstale":"2024-03-04T08:26:27.014533+0000","last_undegraded":"2024-03-04T08:26:27.014533+0000","last_fullsized":"2024-03-04T08:26:27.014533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2280,"log_dups_size":0,"ondisk_log_size":2280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:34:51.364307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.185369957,"stat_sum":{"num_bytes":16781857,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":103,"num_read_kb":39876,"num_write":1435,"num_write_kb":24971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"80'1985","reported_seq":4627,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.984410+0000","last_change":"2024-03-04T08:19:36.851047+0000","last_active":"2024-03-04T08:26:26.984410+0000","last_peered":"2024-03-04T08:26:26.984410+0000","last_clean":"2024-03-04T08:26:26.984410+0000","last_became_active":"2024-03-04T07:33:56.558046+0000","last_became_peered":"2024-03-04T07:33:56.558046+0000","last_unstale":"2024-03-04T08:26:26.984410+0000","last_undegraded":"2024-03-04T08:26:26.984410+0000","last_fullsized":"2024-03-04T08:26:26.984410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1985,"log_dups_size":0,"ondisk_log_size":1985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:00:43.652391+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.86040996599999997,"stat_sum":{"num_bytes":8388608,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":103,"num_read_kb":25220,"num_write":1249,"num_write_kb":17763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"82'2102","reported_seq":4728,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014493+0000","last_change":"2024-03-04T08:19:36.170530+0000","last_active":"2024-03-04T08:26:27.014493+0000","last_peered":"2024-03-04T08:26:27.014493+0000","last_clean":"2024-03-04T08:26:27.014493+0000","last_became_active":"2024-03-04T07:33:56.543176+0000","last_became_peered":"2024-03-04T07:33:56.543176+0000","last_unstale":"2024-03-04T08:26:27.014493+0000","last_undegraded":"2024-03-04T08:26:27.014493+0000","last_fullsized":"2024-03-04T08:26:27.014493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2102,"log_dups_size":0,"ondisk_log_size":2102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:03:35.061415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.188850779,"stat_sum":{"num_bytes":12603630,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":109,"num_read_kb":34094,"num_write":1333,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'2020","reported_seq":4661,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:28.507759+0000","last_change":"2024-03-04T08:19:52.268424+0000","last_active":"2024-03-04T08:26:28.507759+0000","last_peered":"2024-03-04T08:26:28.507759+0000","last_clean":"2024-03-04T08:26:28.507759+0000","last_became_active":"2024-03-04T07:33:56.559172+0000","last_became_peered":"2024-03-04T07:33:56.559172+0000","last_unstale":"2024-03-04T08:26:28.507759+0000","last_undegraded":"2024-03-04T08:26:28.507759+0000","last_fullsized":"2024-03-04T08:26:28.507759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2020,"log_dups_size":0,"ondisk_log_size":2020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:02:26.219884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.1976442450000002,"stat_sum":{"num_bytes":12587627,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":103,"num_read_kb":17785,"num_write":1294,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'2213","reported_seq":4820,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:22.023180+0000","last_change":"2024-03-04T08:19:36.163414+0000","last_active":"2024-03-04T08:27:22.023180+0000","last_peered":"2024-03-04T08:27:22.023180+0000","last_clean":"2024-03-04T08:27:22.023180+0000","last_became_active":"2024-03-04T07:33:56.560306+0000","last_became_peered":"2024-03-04T07:33:56.560306+0000","last_unstale":"2024-03-04T08:27:22.023180+0000","last_undegraded":"2024-03-04T08:27:22.023180+0000","last_fullsized":"2024-03-04T08:27:22.023180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2213,"log_dups_size":0,"ondisk_log_size":2213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:32:42.699565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17780130699999999,"stat_sum":{"num_bytes":8400582,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":109,"num_read_kb":22139,"num_write":1436,"num_write_kb":20580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"80'2060","reported_seq":4723,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.994336+0000","last_change":"2024-03-04T08:19:34.383564+0000","last_active":"2024-03-04T08:27:06.994336+0000","last_peered":"2024-03-04T08:27:06.994336+0000","last_clean":"2024-03-04T08:27:06.994336+0000","last_became_active":"2024-03-04T07:33:56.560592+0000","last_became_peered":"2024-03-04T07:33:56.560592+0000","last_unstale":"2024-03-04T08:27:06.994336+0000","last_undegraded":"2024-03-04T08:27:06.994336+0000","last_fullsized":"2024-03-04T08:27:06.994336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2060,"log_dups_size":0,"ondisk_log_size":2060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:40:27.691315+0000","scrub_duration":0,"objects_trimmed":120,"snaptrim_duration":1.6725091160000001,"stat_sum":{"num_bytes":25176920,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":134,"num_read_kb":52032,"num_write":1326,"num_write_kb":32831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"82'2220","reported_seq":4778,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.994298+0000","last_change":"2024-03-04T08:19:34.375018+0000","last_active":"2024-03-04T08:27:06.994298+0000","last_peered":"2024-03-04T08:27:06.994298+0000","last_clean":"2024-03-04T08:27:06.994298+0000","last_became_active":"2024-03-04T07:33:56.560719+0000","last_became_peered":"2024-03-04T07:33:56.560719+0000","last_unstale":"2024-03-04T08:27:06.994298+0000","last_undegraded":"2024-03-04T08:27:06.994298+0000","last_fullsized":"2024-03-04T08:27:06.994298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2220,"log_dups_size":0,"ondisk_log_size":2220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:33:12.566752+0000","scrub_duration":0,"objects_trimmed":134,"snaptrim_duration":1.6642866970000001,"stat_sum":{"num_bytes":8390989,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":101,"num_read_kb":24838,"num_write":1357,"num_write_kb":15528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"80'2201","reported_seq":4804,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:07.125104+0000","last_change":"2024-03-04T08:19:33.340351+0000","last_active":"2024-03-04T08:27:07.125104+0000","last_peered":"2024-03-04T08:27:07.125104+0000","last_clean":"2024-03-04T08:27:07.125104+0000","last_became_active":"2024-03-04T07:33:56.549568+0000","last_became_peered":"2024-03-04T07:33:56.549568+0000","last_unstale":"2024-03-04T08:27:07.125104+0000","last_undegraded":"2024-03-04T08:27:07.125104+0000","last_fullsized":"2024-03-04T08:27:07.125104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2201,"log_dups_size":0,"ondisk_log_size":2201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:29:31.376806+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.65008146899999997,"stat_sum":{"num_bytes":25170332,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":131,"num_read_kb":58959,"num_write":1343,"num_write_kb":36929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'2124","reported_seq":4745,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:08.512078+0000","last_change":"2024-03-04T08:19:52.261777+0000","last_active":"2024-03-04T08:27:08.512078+0000","last_peered":"2024-03-04T08:27:08.512078+0000","last_clean":"2024-03-04T08:27:08.512078+0000","last_became_active":"2024-03-04T07:33:56.558567+0000","last_became_peered":"2024-03-04T07:33:56.558567+0000","last_unstale":"2024-03-04T08:27:08.512078+0000","last_undegraded":"2024-03-04T08:27:08.512078+0000","last_fullsized":"2024-03-04T08:27:08.512078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2124,"log_dups_size":0,"ondisk_log_size":2124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:59:16.961107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.1904157900000003,"stat_sum":{"num_bytes":16788130,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":113,"num_read_kb":35043,"num_write":1370,"num_write_kb":33439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"80'2140","reported_seq":4759,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:47.140060+0000","last_change":"2024-03-04T08:19:49.427543+0000","last_active":"2024-03-04T08:27:47.140060+0000","last_peered":"2024-03-04T08:27:47.140060+0000","last_clean":"2024-03-04T08:27:47.140060+0000","last_became_active":"2024-03-04T07:33:56.543936+0000","last_became_peered":"2024-03-04T07:33:56.543936+0000","last_unstale":"2024-03-04T08:27:47.140060+0000","last_undegraded":"2024-03-04T08:27:47.140060+0000","last_fullsized":"2024-03-04T08:27:47.140060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2140,"log_dups_size":0,"ondisk_log_size":2140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:29:34.488147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0299552260000002,"stat_sum":{"num_bytes":8390003,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":16051,"num_write":1328,"num_write_kb":18720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'2145","reported_seq":4756,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:38.054541+0000","last_change":"2024-03-04T08:19:35.446076+0000","last_active":"2024-03-04T08:27:38.054541+0000","last_peered":"2024-03-04T08:27:38.054541+0000","last_clean":"2024-03-04T08:27:38.054541+0000","last_became_active":"2024-03-04T07:33:56.544837+0000","last_became_peered":"2024-03-04T07:33:56.544837+0000","last_unstale":"2024-03-04T08:27:38.054541+0000","last_undegraded":"2024-03-04T08:27:38.054541+0000","last_fullsized":"2024-03-04T08:27:38.054541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2145,"log_dups_size":0,"ondisk_log_size":2145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:01:02.809197+0000","scrub_duration":0,"objects_trimmed":196,"snaptrim_duration":2.7488653730000001,"stat_sum":{"num_bytes":8391933,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":90,"num_read_kb":10258,"num_write":1381,"num_write_kb":16511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"82'2151","reported_seq":4784,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.994588+0000","last_change":"2024-03-04T08:19:36.848889+0000","last_active":"2024-03-04T08:27:36.994588+0000","last_peered":"2024-03-04T08:27:36.994588+0000","last_clean":"2024-03-04T08:27:36.994588+0000","last_became_active":"2024-03-04T07:33:56.557962+0000","last_became_peered":"2024-03-04T07:33:56.557962+0000","last_unstale":"2024-03-04T08:27:36.994588+0000","last_undegraded":"2024-03-04T08:27:36.994588+0000","last_fullsized":"2024-03-04T08:27:36.994588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2151,"log_dups_size":0,"ondisk_log_size":2151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:33:13.908448+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.86055390200000004,"stat_sum":{"num_bytes":12654141,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":117,"num_read_kb":36882,"num_write":1403,"num_write_kb":24709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"82'2229","reported_seq":4853,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.994692+0000","last_change":"2024-03-04T08:19:36.842387+0000","last_active":"2024-03-04T08:27:36.994692+0000","last_peered":"2024-03-04T08:27:36.994692+0000","last_clean":"2024-03-04T08:27:36.994692+0000","last_became_active":"2024-03-04T07:33:56.558719+0000","last_became_peered":"2024-03-04T07:33:56.558719+0000","last_unstale":"2024-03-04T08:27:36.994692+0000","last_undegraded":"2024-03-04T08:27:36.994692+0000","last_fullsized":"2024-03-04T08:27:36.994692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2229,"log_dups_size":0,"ondisk_log_size":2229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:39:07.391195+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.85227727200000003,"stat_sum":{"num_bytes":16780080,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":129,"num_read_kb":21167,"num_write":1413,"num_write_kb":30886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"80'2047","reported_seq":4676,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138840+0000","last_change":"2024-03-04T08:19:32.728091+0000","last_active":"2024-03-04T08:27:37.138840+0000","last_peered":"2024-03-04T08:27:37.138840+0000","last_clean":"2024-03-04T08:27:37.138840+0000","last_became_active":"2024-03-04T07:33:56.528213+0000","last_became_peered":"2024-03-04T07:33:56.528213+0000","last_unstale":"2024-03-04T08:27:37.138840+0000","last_undegraded":"2024-03-04T08:27:37.138840+0000","last_fullsized":"2024-03-04T08:27:37.138840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2047,"log_dups_size":0,"ondisk_log_size":2047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:49.272879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019434987000000001,"stat_sum":{"num_bytes":4194351,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":120,"num_read_kb":13790,"num_write":1261,"num_write_kb":11549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"80'2288","reported_seq":4870,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:12.030625+0000","last_change":"2024-03-04T08:19:36.169663+0000","last_active":"2024-03-04T08:28:12.030625+0000","last_peered":"2024-03-04T08:28:12.030625+0000","last_clean":"2024-03-04T08:28:12.030625+0000","last_became_active":"2024-03-04T07:33:56.545467+0000","last_became_peered":"2024-03-04T07:33:56.545467+0000","last_unstale":"2024-03-04T08:28:12.030625+0000","last_undegraded":"2024-03-04T08:28:12.030625+0000","last_fullsized":"2024-03-04T08:28:12.030625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2288,"log_dups_size":0,"ondisk_log_size":2288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:22:01.957883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18267682599999999,"stat_sum":{"num_bytes":12645111,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":119,"num_read_kb":30029,"num_write":1423,"num_write_kb":26760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"80'2119","reported_seq":4737,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.001524+0000","last_change":"2024-03-04T08:19:34.382758+0000","last_active":"2024-03-04T08:27:57.001524+0000","last_peered":"2024-03-04T08:27:57.001524+0000","last_clean":"2024-03-04T08:27:57.001524+0000","last_became_active":"2024-03-04T07:33:56.548783+0000","last_became_peered":"2024-03-04T07:33:56.548783+0000","last_unstale":"2024-03-04T08:27:57.001524+0000","last_undegraded":"2024-03-04T08:27:57.001524+0000","last_fullsized":"2024-03-04T08:27:57.001524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2119,"log_dups_size":0,"ondisk_log_size":2119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:16:42.338152+0000","scrub_duration":0,"objects_trimmed":116,"snaptrim_duration":1.671257062,"stat_sum":{"num_bytes":8450701,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":102,"num_read_kb":25514,"num_write":1341,"num_write_kb":30030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"82'2119","reported_seq":4723,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.141072+0000","last_change":"2024-03-04T08:19:49.225914+0000","last_active":"2024-03-04T08:27:57.141072+0000","last_peered":"2024-03-04T08:27:57.141072+0000","last_clean":"2024-03-04T08:27:57.141072+0000","last_became_active":"2024-03-04T07:33:56.539403+0000","last_became_peered":"2024-03-04T07:33:56.539403+0000","last_unstale":"2024-03-04T08:27:57.141072+0000","last_undegraded":"2024-03-04T08:27:57.141072+0000","last_fullsized":"2024-03-04T08:27:57.141072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2119,"log_dups_size":0,"ondisk_log_size":2119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:43:54.215133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.827802057,"stat_sum":{"num_bytes":4204891,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":101,"num_read_kb":9693,"num_write":1325,"num_write_kb":13878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"80'2234","reported_seq":4838,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:22.131357+0000","last_change":"2024-03-04T08:19:32.717851+0000","last_active":"2024-03-04T08:26:22.131357+0000","last_peered":"2024-03-04T08:26:22.131357+0000","last_clean":"2024-03-04T08:26:22.131357+0000","last_became_active":"2024-03-04T07:34:37.644727+0000","last_became_peered":"2024-03-04T07:34:37.644727+0000","last_unstale":"2024-03-04T08:26:22.131357+0000","last_undegraded":"2024-03-04T08:26:22.131357+0000","last_fullsized":"2024-03-04T08:26:22.131357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:01:11.097920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010246078,"stat_sum":{"num_bytes":12594611,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":123,"num_read_kb":29265,"num_write":1418,"num_write_kb":23716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"82'2041","reported_seq":4709,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:58.516950+0000","last_change":"2024-03-04T08:19:52.301927+0000","last_active":"2024-03-04T08:27:58.516950+0000","last_peered":"2024-03-04T08:27:58.516950+0000","last_clean":"2024-03-04T08:27:58.516950+0000","last_became_active":"2024-03-04T07:33:56.545270+0000","last_became_peered":"2024-03-04T07:33:56.545270+0000","last_unstale":"2024-03-04T08:27:58.516950+0000","last_undegraded":"2024-03-04T08:27:58.516950+0000","last_fullsized":"2024-03-04T08:27:58.516950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2041,"log_dups_size":0,"ondisk_log_size":2041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:29:34.456689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.2311050080000001,"stat_sum":{"num_bytes":12602373,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":129,"num_read_kb":39775,"num_write":1326,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'2103","reported_seq":4737,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.001274+0000","last_change":"2024-03-04T08:19:36.847630+0000","last_active":"2024-03-04T08:28:22.001274+0000","last_peered":"2024-03-04T08:28:22.001274+0000","last_clean":"2024-03-04T08:28:22.001274+0000","last_became_active":"2024-03-04T07:33:56.554942+0000","last_became_peered":"2024-03-04T07:33:56.554942+0000","last_unstale":"2024-03-04T08:28:22.001274+0000","last_undegraded":"2024-03-04T08:28:22.001274+0000","last_fullsized":"2024-03-04T08:28:22.001274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2103,"log_dups_size":0,"ondisk_log_size":2103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:11:44.174822+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.85911008,"stat_sum":{"num_bytes":4342145,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":112,"num_read_kb":14015,"num_write":1342,"num_write_kb":11768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2416","reported_seq":4914,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.518032+0000","last_change":"2024-03-04T08:19:52.525743+0000","last_active":"2024-03-04T08:28:08.518032+0000","last_peered":"2024-03-04T08:28:08.518032+0000","last_clean":"2024-03-04T08:28:08.518032+0000","last_became_active":"2024-03-04T07:33:56.548790+0000","last_became_peered":"2024-03-04T07:33:56.548790+0000","last_unstale":"2024-03-04T08:28:08.518032+0000","last_undegraded":"2024-03-04T08:28:08.518032+0000","last_fullsized":"2024-03-04T08:28:08.518032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2416,"log_dups_size":0,"ondisk_log_size":2416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:53:43.350229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.454029974,"stat_sum":{"num_bytes":8428929,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":109,"num_read_kb":26630,"num_write":1382,"num_write_kb":16670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'2182","reported_seq":4787,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.002961+0000","last_change":"2024-03-04T08:19:34.384174+0000","last_active":"2024-03-04T08:28:07.002961+0000","last_peered":"2024-03-04T08:28:07.002961+0000","last_clean":"2024-03-04T08:28:07.002961+0000","last_became_active":"2024-03-04T07:33:56.553012+0000","last_became_peered":"2024-03-04T07:33:56.553012+0000","last_unstale":"2024-03-04T08:28:07.002961+0000","last_undegraded":"2024-03-04T08:28:07.002961+0000","last_fullsized":"2024-03-04T08:28:07.002961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2182,"log_dups_size":0,"ondisk_log_size":2182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:17:05.626254+0000","scrub_duration":0,"objects_trimmed":141,"snaptrim_duration":1.673071669,"stat_sum":{"num_bytes":4199673,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":99,"num_read_kb":13996,"num_write":1374,"num_write_kb":14419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'2031","reported_seq":4665,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057745+0000","last_change":"2024-03-04T08:19:35.448280+0000","last_active":"2024-03-04T08:28:08.057745+0000","last_peered":"2024-03-04T08:28:08.057745+0000","last_clean":"2024-03-04T08:28:08.057745+0000","last_became_active":"2024-03-04T07:33:56.550616+0000","last_became_peered":"2024-03-04T07:33:56.550616+0000","last_unstale":"2024-03-04T08:28:08.057745+0000","last_undegraded":"2024-03-04T08:28:08.057745+0000","last_fullsized":"2024-03-04T08:28:08.057745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2031,"log_dups_size":0,"ondisk_log_size":2031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:49:50.704680+0000","scrub_duration":0,"objects_trimmed":153,"snaptrim_duration":2.7491079520000001,"stat_sum":{"num_bytes":20984835,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":87,"num_read_kb":29030,"num_write":1330,"num_write_kb":28787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"80'2058","reported_seq":4675,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.518017+0000","last_change":"2024-03-04T08:19:52.538416+0000","last_active":"2024-03-04T08:28:08.518017+0000","last_peered":"2024-03-04T08:28:08.518017+0000","last_clean":"2024-03-04T08:28:08.518017+0000","last_became_active":"2024-03-04T07:33:56.543632+0000","last_became_peered":"2024-03-04T07:33:56.543632+0000","last_unstale":"2024-03-04T08:28:08.518017+0000","last_undegraded":"2024-03-04T08:28:08.518017+0000","last_fullsized":"2024-03-04T08:28:08.518017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2058,"log_dups_size":0,"ondisk_log_size":2058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:14:48.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.4338148220000004,"stat_sum":{"num_bytes":4197645,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":105,"num_read_kb":17328,"num_write":1252,"num_write_kb":17889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"80'2259","reported_seq":4843,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:17.143069+0000","last_change":"2024-03-04T08:19:49.701932+0000","last_active":"2024-03-04T08:28:17.143069+0000","last_peered":"2024-03-04T08:28:17.143069+0000","last_clean":"2024-03-04T08:28:17.143069+0000","last_became_active":"2024-03-04T07:33:56.546652+0000","last_became_peered":"2024-03-04T07:33:56.546652+0000","last_unstale":"2024-03-04T08:28:17.143069+0000","last_undegraded":"2024-03-04T08:28:17.143069+0000","last_fullsized":"2024-03-04T08:28:17.143069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2259,"log_dups_size":0,"ondisk_log_size":2259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:36:41.107790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.303629049,"stat_sum":{"num_bytes":8400817,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":112,"num_read_kb":14343,"num_write":1411,"num_write_kb":25423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'2098","reported_seq":4725,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:06.999085+0000","last_change":"2024-03-04T08:19:36.849222+0000","last_active":"2024-03-04T08:28:06.999085+0000","last_peered":"2024-03-04T08:28:06.999085+0000","last_clean":"2024-03-04T08:28:06.999085+0000","last_became_active":"2024-03-04T07:33:56.552189+0000","last_became_peered":"2024-03-04T07:33:56.552189+0000","last_unstale":"2024-03-04T08:28:06.999085+0000","last_undegraded":"2024-03-04T08:28:06.999085+0000","last_fullsized":"2024-03-04T08:28:06.999085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2098,"log_dups_size":0,"ondisk_log_size":2098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:38:31.691317+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.85857668899999995,"stat_sum":{"num_bytes":8407168,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":95,"num_read_kb":14091,"num_write":1358,"num_write_kb":17493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"80'2246","reported_seq":4842,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.141974+0000","last_change":"2024-03-04T08:19:32.724206+0000","last_active":"2024-03-04T08:28:07.141974+0000","last_peered":"2024-03-04T08:28:07.141974+0000","last_clean":"2024-03-04T08:28:07.141974+0000","last_became_active":"2024-03-04T07:33:56.549351+0000","last_became_peered":"2024-03-04T07:33:56.549351+0000","last_unstale":"2024-03-04T08:28:07.141974+0000","last_undegraded":"2024-03-04T08:28:07.141974+0000","last_fullsized":"2024-03-04T08:28:07.141974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2246,"log_dups_size":0,"ondisk_log_size":2246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:25.101476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.016086315,"stat_sum":{"num_bytes":20978563,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":109,"num_read_kb":27431,"num_write":1407,"num_write_kb":36516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'2497","reported_seq":4993,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.002897+0000","last_change":"2024-03-04T08:19:34.447685+0000","last_active":"2024-03-04T08:28:07.002897+0000","last_peered":"2024-03-04T08:28:07.002897+0000","last_clean":"2024-03-04T08:28:07.002897+0000","last_became_active":"2024-03-04T07:33:56.532251+0000","last_became_peered":"2024-03-04T07:33:56.532251+0000","last_unstale":"2024-03-04T08:28:07.002897+0000","last_undegraded":"2024-03-04T08:28:07.002897+0000","last_fullsized":"2024-03-04T08:28:07.002897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:03:48.692559+0000","scrub_duration":0,"objects_trimmed":183,"snaptrim_duration":1.7333053779999998,"stat_sum":{"num_bytes":12598132,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":114,"num_read_kb":36166,"num_write":1454,"num_write_kb":20545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"82'2217","reported_seq":4804,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.119475+0000","last_change":"2024-03-04T08:19:33.346715+0000","last_active":"2024-03-04T08:26:27.119475+0000","last_peered":"2024-03-04T08:26:27.119475+0000","last_clean":"2024-03-04T08:26:27.119475+0000","last_became_active":"2024-03-04T07:33:56.542937+0000","last_became_peered":"2024-03-04T07:33:56.542937+0000","last_unstale":"2024-03-04T08:26:27.119475+0000","last_undegraded":"2024-03-04T08:26:27.119475+0000","last_fullsized":"2024-03-04T08:26:27.119475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2217,"log_dups_size":0,"ondisk_log_size":2217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:17:50.886040+0000","scrub_duration":0,"objects_trimmed":58,"snaptrim_duration":0.65684712700000003,"stat_sum":{"num_bytes":12587619,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":118,"num_read_kb":25719,"num_write":1362,"num_write_kb":26221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"85'680","reported_seq":983,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:58.522965+0000","last_change":"2024-03-04T07:33:55.566130+0000","last_active":"2024-03-04T08:28:58.522965+0000","last_peered":"2024-03-04T08:28:58.522965+0000","last_clean":"2024-03-04T08:28:58.522965+0000","last_became_active":"2024-03-04T07:33:55.565894+0000","last_became_peered":"2024-03-04T07:33:55.565894+0000","last_unstale":"2024-03-04T08:28:58.522965+0000","last_undegraded":"2024-03-04T08:28:58.522965+0000","last_fullsized":"2024-03-04T08:28:58.522965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:36.530949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20973469,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":251,"num_read_kb":140597,"num_write":805,"num_write_kb":83805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"82'2160","reported_seq":4871,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.132021+0000","last_change":"2024-03-04T08:19:49.377440+0000","last_active":"2024-03-04T08:26:27.132021+0000","last_peered":"2024-03-04T08:26:27.132021+0000","last_clean":"2024-03-04T08:26:27.132021+0000","last_became_active":"2024-03-04T07:33:56.539738+0000","last_became_peered":"2024-03-04T07:33:56.539738+0000","last_unstale":"2024-03-04T08:26:27.132021+0000","last_undegraded":"2024-03-04T08:26:27.132021+0000","last_fullsized":"2024-03-04T08:26:27.132021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2160,"log_dups_size":0,"ondisk_log_size":2160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:44:38.647134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.9789417120000001,"stat_sum":{"num_bytes":8388926,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":220,"num_read_kb":14085,"num_write":1362,"num_write_kb":17052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1189","reported_seq":1504,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.988431+0000","last_change":"2024-03-04T07:33:55.554956+0000","last_active":"2024-03-04T08:26:26.988431+0000","last_peered":"2024-03-04T08:26:26.988431+0000","last_clean":"2024-03-04T08:26:26.988431+0000","last_became_active":"2024-03-04T07:33:55.554151+0000","last_became_peered":"2024-03-04T07:33:55.554151+0000","last_unstale":"2024-03-04T08:26:26.988431+0000","last_undegraded":"2024-03-04T08:26:26.988431+0000","last_fullsized":"2024-03-04T08:26:26.988431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1189,"log_dups_size":0,"ondisk_log_size":1189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:06:19.621510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":322,"num_read_kb":118780,"num_write":1380,"num_write_kb":72133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"82'487","reported_seq":770,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:13.512566+0000","last_change":"2024-03-04T07:33:55.542200+0000","last_active":"2024-03-04T08:27:13.512566+0000","last_peered":"2024-03-04T08:27:13.512566+0000","last_clean":"2024-03-04T08:27:13.512566+0000","last_became_active":"2024-03-04T07:33:55.541902+0000","last_became_peered":"2024-03-04T07:33:55.541902+0000","last_unstale":"2024-03-04T08:27:13.512566+0000","last_undegraded":"2024-03-04T08:27:13.512566+0000","last_fullsized":"2024-03-04T08:27:13.512566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:55:50.251288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":258,"num_read_kb":115971,"num_write":610,"num_write_kb":58869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"80'2411","reported_seq":4973,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:08.512031+0000","last_change":"2024-03-04T08:19:52.693084+0000","last_active":"2024-03-04T08:27:08.512031+0000","last_peered":"2024-03-04T08:27:08.512031+0000","last_clean":"2024-03-04T08:27:08.512031+0000","last_became_active":"2024-03-04T07:33:56.546087+0000","last_became_peered":"2024-03-04T07:33:56.546087+0000","last_unstale":"2024-03-04T08:27:08.512031+0000","last_undegraded":"2024-03-04T08:27:08.512031+0000","last_fullsized":"2024-03-04T08:27:08.512031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2411,"log_dups_size":0,"ondisk_log_size":2411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:42:05.213414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.6221017409999998,"stat_sum":{"num_bytes":4200161,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":109,"num_read_kb":16756,"num_write":1524,"num_write_kb":21226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1610","reported_seq":1897,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.990193+0000","last_change":"2024-03-04T07:33:55.568508+0000","last_active":"2024-03-04T08:27:06.990193+0000","last_peered":"2024-03-04T08:27:06.990193+0000","last_clean":"2024-03-04T08:27:06.990193+0000","last_became_active":"2024-03-04T07:33:55.566073+0000","last_became_peered":"2024-03-04T07:33:55.566073+0000","last_unstale":"2024-03-04T08:27:06.990193+0000","last_undegraded":"2024-03-04T08:27:06.990193+0000","last_fullsized":"2024-03-04T08:27:06.990193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1610,"log_dups_size":0,"ondisk_log_size":1610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:34:17.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":228,"num_read_kb":146204,"num_write":1735,"num_write_kb":76344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'2138","reported_seq":4771,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:18.052787+0000","last_change":"2024-03-04T08:19:35.448277+0000","last_active":"2024-03-04T08:27:18.052787+0000","last_peered":"2024-03-04T08:27:18.052787+0000","last_clean":"2024-03-04T08:27:18.052787+0000","last_became_active":"2024-03-04T07:33:56.552533+0000","last_became_peered":"2024-03-04T07:33:56.552533+0000","last_unstale":"2024-03-04T08:27:18.052787+0000","last_undegraded":"2024-03-04T08:27:18.052787+0000","last_fullsized":"2024-03-04T08:27:18.052787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2138,"log_dups_size":0,"ondisk_log_size":2138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:15:02.557608+0000","scrub_duration":0,"objects_trimmed":178,"snaptrim_duration":2.7512426310000002,"stat_sum":{"num_bytes":25170167,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":124,"num_read_kb":49299,"num_write":1360,"num_write_kb":40715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'786","reported_seq":1224,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:29:32.015555+0000","last_change":"2024-03-04T07:33:55.559395+0000","last_active":"2024-03-04T08:29:32.015555+0000","last_peered":"2024-03-04T08:29:32.015555+0000","last_clean":"2024-03-04T08:29:32.015555+0000","last_became_active":"2024-03-04T07:33:55.558593+0000","last_became_peered":"2024-03-04T07:33:55.558593+0000","last_unstale":"2024-03-04T08:29:32.015555+0000","last_undegraded":"2024-03-04T08:29:32.015555+0000","last_fullsized":"2024-03-04T08:29:32.015555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:00:20.205445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17355948,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":385,"num_read_kb":100065,"num_write":866,"num_write_kb":59600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"82'2101","reported_seq":4721,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.990264+0000","last_change":"2024-03-04T08:19:36.839345+0000","last_active":"2024-03-04T08:27:06.990264+0000","last_peered":"2024-03-04T08:27:06.990264+0000","last_clean":"2024-03-04T08:27:06.990264+0000","last_became_active":"2024-03-04T07:33:56.552952+0000","last_became_peered":"2024-03-04T07:33:56.552952+0000","last_unstale":"2024-03-04T08:27:06.990264+0000","last_undegraded":"2024-03-04T08:27:06.990264+0000","last_fullsized":"2024-03-04T08:27:06.990264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2101,"log_dups_size":0,"ondisk_log_size":2101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:53:29.724551+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.84949035500000003,"stat_sum":{"num_bytes":8307065,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":95,"num_read_kb":25119,"num_write":1361,"num_write_kb":16142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"82'705","reported_seq":1001,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.990287+0000","last_change":"2024-03-04T07:33:55.568220+0000","last_active":"2024-03-04T08:27:06.990287+0000","last_peered":"2024-03-04T08:27:06.990287+0000","last_clean":"2024-03-04T08:27:06.990287+0000","last_became_active":"2024-03-04T07:33:55.567569+0000","last_became_peered":"2024-03-04T07:33:55.567569+0000","last_unstale":"2024-03-04T08:27:06.990287+0000","last_undegraded":"2024-03-04T08:27:06.990287+0000","last_fullsized":"2024-03-04T08:27:06.990287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:40:49.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":256,"num_read_kb":181413,"num_write":818,"num_write_kb":87860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"82'2131","reported_seq":4730,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:07.136092+0000","last_change":"2024-03-04T08:19:49.095825+0000","last_active":"2024-03-04T08:27:07.136092+0000","last_peered":"2024-03-04T08:27:07.136092+0000","last_clean":"2024-03-04T08:27:07.136092+0000","last_became_active":"2024-03-04T07:33:56.559456+0000","last_became_peered":"2024-03-04T07:33:56.559456+0000","last_unstale":"2024-03-04T08:27:07.136092+0000","last_undegraded":"2024-03-04T08:27:07.136092+0000","last_fullsized":"2024-03-04T08:27:07.136092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2131,"log_dups_size":0,"ondisk_log_size":2131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:50:29.399119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.6973493710000001,"stat_sum":{"num_bytes":8393873,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":101,"num_read_kb":25726,"num_write":1348,"num_write_kb":16222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'1400","reported_seq":1705,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:08.051648+0000","last_change":"2024-03-04T07:33:55.550625+0000","last_active":"2024-03-04T08:27:08.051648+0000","last_peered":"2024-03-04T08:27:08.051648+0000","last_clean":"2024-03-04T08:27:08.051648+0000","last_became_active":"2024-03-04T07:33:55.550327+0000","last_became_peered":"2024-03-04T07:33:55.550327+0000","last_unstale":"2024-03-04T08:27:08.051648+0000","last_undegraded":"2024-03-04T08:27:08.051648+0000","last_fullsized":"2024-03-04T08:27:08.051648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:20.282585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":274,"num_read_kb":137054,"num_write":1577,"num_write_kb":60435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"80'2174","reported_seq":4785,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:47.130659+0000","last_change":"2024-03-04T08:19:33.352033+0000","last_active":"2024-03-04T08:27:47.130659+0000","last_peered":"2024-03-04T08:27:47.130659+0000","last_clean":"2024-03-04T08:27:47.130659+0000","last_became_active":"2024-03-04T07:33:56.536870+0000","last_became_peered":"2024-03-04T07:33:56.536870+0000","last_unstale":"2024-03-04T08:27:47.130659+0000","last_undegraded":"2024-03-04T08:27:47.130659+0000","last_fullsized":"2024-03-04T08:27:47.130659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2174,"log_dups_size":0,"ondisk_log_size":2174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:56:44.867605+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.66166569399999997,"stat_sum":{"num_bytes":12846648,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":118,"num_read_kb":29225,"num_write":1393,"num_write_kb":24887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"82'1283","reported_seq":1627,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:52.000760+0000","last_change":"2024-03-04T07:33:55.538915+0000","last_active":"2024-03-04T08:27:52.000760+0000","last_peered":"2024-03-04T08:27:52.000760+0000","last_clean":"2024-03-04T08:27:52.000760+0000","last_became_active":"2024-03-04T07:33:55.536263+0000","last_became_peered":"2024-03-04T07:33:55.536263+0000","last_unstale":"2024-03-04T08:27:52.000760+0000","last_undegraded":"2024-03-04T08:27:52.000760+0000","last_fullsized":"2024-03-04T08:27:52.000760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1283,"log_dups_size":0,"ondisk_log_size":1283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:09.612704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360800,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":354,"num_read_kb":206595,"num_write":1442,"num_write_kb":96944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"80'2051","reported_seq":4680,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:07.020807+0000","last_change":"2024-03-04T08:19:36.168875+0000","last_active":"2024-03-04T08:27:07.020807+0000","last_peered":"2024-03-04T08:27:07.020807+0000","last_clean":"2024-03-04T08:27:07.020807+0000","last_became_active":"2024-03-04T07:33:56.550767+0000","last_became_peered":"2024-03-04T07:33:56.550767+0000","last_unstale":"2024-03-04T08:27:07.020807+0000","last_undegraded":"2024-03-04T08:27:07.020807+0000","last_fullsized":"2024-03-04T08:27:07.020807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2051,"log_dups_size":0,"ondisk_log_size":2051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:41:38.658644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.187228595,"stat_sum":{"num_bytes":12597061,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":105,"num_read_kb":35778,"num_write":1291,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"82'483","reported_seq":772,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138903+0000","last_change":"2024-03-04T07:33:55.560382+0000","last_active":"2024-03-04T08:27:37.138903+0000","last_peered":"2024-03-04T08:27:37.138903+0000","last_clean":"2024-03-04T08:27:37.138903+0000","last_became_active":"2024-03-04T07:33:55.560078+0000","last_became_peered":"2024-03-04T07:33:55.560078+0000","last_unstale":"2024-03-04T08:27:37.138903+0000","last_undegraded":"2024-03-04T08:27:37.138903+0000","last_fullsized":"2024-03-04T08:27:37.138903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:41:10.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":271,"num_read_kb":107276,"num_write":561,"num_write_kb":58233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"80'2019","reported_seq":4649,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138868+0000","last_change":"2024-03-04T08:19:49.480210+0000","last_active":"2024-03-04T08:27:37.138868+0000","last_peered":"2024-03-04T08:27:37.138868+0000","last_clean":"2024-03-04T08:27:37.138868+0000","last_became_active":"2024-03-04T07:33:56.535960+0000","last_became_peered":"2024-03-04T07:33:56.535960+0000","last_unstale":"2024-03-04T08:27:37.138868+0000","last_undegraded":"2024-03-04T08:27:37.138868+0000","last_fullsized":"2024-03-04T08:27:37.138868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2019,"log_dups_size":0,"ondisk_log_size":2019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:17:01.637891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0525318239999999,"stat_sum":{"num_bytes":16780560,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":39118,"num_write":1277,"num_write_kb":32917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"82'440","reported_seq":737,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.994642+0000","last_change":"2024-03-04T07:33:55.567682+0000","last_active":"2024-03-04T08:27:36.994642+0000","last_peered":"2024-03-04T08:27:36.994642+0000","last_clean":"2024-03-04T08:27:36.994642+0000","last_became_active":"2024-03-04T07:33:55.566915+0000","last_became_peered":"2024-03-04T07:33:55.566915+0000","last_unstale":"2024-03-04T08:27:36.994642+0000","last_undegraded":"2024-03-04T08:27:36.994642+0000","last_fullsized":"2024-03-04T08:27:36.994642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:33:47.010346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":276,"num_read_kb":100025,"num_write":608,"num_write_kb":47104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'2205","reported_seq":4818,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.025397+0000","last_change":"2024-03-04T08:19:36.162956+0000","last_active":"2024-03-04T08:27:37.025397+0000","last_peered":"2024-03-04T08:27:37.025397+0000","last_clean":"2024-03-04T08:27:37.025397+0000","last_became_active":"2024-03-04T07:33:56.547459+0000","last_became_peered":"2024-03-04T07:33:56.547459+0000","last_unstale":"2024-03-04T08:27:37.025397+0000","last_undegraded":"2024-03-04T08:27:37.025397+0000","last_fullsized":"2024-03-04T08:27:37.025397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2205,"log_dups_size":0,"ondisk_log_size":2205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:48:06.725806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.181152964,"stat_sum":{"num_bytes":4213693,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":118,"num_read_kb":1396,"num_write":1423,"num_write_kb":14566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'723","reported_seq":1018,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.998645+0000","last_change":"2024-03-04T07:33:55.557041+0000","last_active":"2024-03-04T08:27:36.998645+0000","last_peered":"2024-03-04T08:27:36.998645+0000","last_clean":"2024-03-04T08:27:36.998645+0000","last_became_active":"2024-03-04T07:33:55.556416+0000","last_became_peered":"2024-03-04T07:33:55.556416+0000","last_unstale":"2024-03-04T08:27:36.998645+0000","last_undegraded":"2024-03-04T08:27:36.998645+0000","last_fullsized":"2024-03-04T08:27:36.998645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:31.472235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8390846,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":242,"num_read_kb":79450,"num_write":832,"num_write_kb":51201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"82'2099","reported_seq":4736,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.998604+0000","last_change":"2024-03-04T08:19:34.394444+0000","last_active":"2024-03-04T08:27:36.998604+0000","last_peered":"2024-03-04T08:27:36.998604+0000","last_clean":"2024-03-04T08:27:36.998604+0000","last_became_active":"2024-03-04T07:33:56.531565+0000","last_became_peered":"2024-03-04T07:33:56.531565+0000","last_unstale":"2024-03-04T08:27:36.998604+0000","last_undegraded":"2024-03-04T08:27:36.998604+0000","last_fullsized":"2024-03-04T08:27:36.998604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2099,"log_dups_size":0,"ondisk_log_size":2099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:02:41.845508+0000","scrub_duration":0,"objects_trimmed":121,"snaptrim_duration":1.6809537379999999,"stat_sum":{"num_bytes":16783369,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":112,"num_read_kb":39998,"num_write":1341,"num_write_kb":27959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'797","reported_seq":1153,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138789+0000","last_change":"2024-03-04T07:33:55.538469+0000","last_active":"2024-03-04T08:27:37.138789+0000","last_peered":"2024-03-04T08:27:37.138789+0000","last_clean":"2024-03-04T08:27:37.138789+0000","last_became_active":"2024-03-04T07:33:55.538165+0000","last_became_peered":"2024-03-04T07:33:55.538165+0000","last_unstale":"2024-03-04T08:27:37.138789+0000","last_undegraded":"2024-03-04T08:27:37.138789+0000","last_fullsized":"2024-03-04T08:27:37.138789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:00:01.263768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165936,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":419,"num_read_kb":136829,"num_write":955,"num_write_kb":63584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"80'2116","reported_seq":4728,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:18.058628+0000","last_change":"2024-03-04T08:19:35.444496+0000","last_active":"2024-03-04T08:28:18.058628+0000","last_peered":"2024-03-04T08:28:18.058628+0000","last_clean":"2024-03-04T08:28:18.058628+0000","last_became_active":"2024-03-04T07:34:36.770794+0000","last_became_peered":"2024-03-04T07:34:36.770794+0000","last_unstale":"2024-03-04T08:28:18.058628+0000","last_undegraded":"2024-03-04T08:28:18.058628+0000","last_fullsized":"2024-03-04T08:28:18.058628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2116,"log_dups_size":0,"ondisk_log_size":2116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:08:17.614159+0000","scrub_duration":0,"objects_trimmed":157,"snaptrim_duration":2.747632485,"stat_sum":{"num_bytes":16781418,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":100,"num_read_kb":34292,"num_write":1333,"num_write_kb":33482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"84'741","reported_seq":1039,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.133542+0000","last_change":"2024-03-04T07:33:55.544356+0000","last_active":"2024-03-04T08:28:07.133542+0000","last_peered":"2024-03-04T08:28:07.133542+0000","last_clean":"2024-03-04T08:28:07.133542+0000","last_became_active":"2024-03-04T07:33:55.541295+0000","last_became_peered":"2024-03-04T07:33:55.541295+0000","last_unstale":"2024-03-04T08:28:07.133542+0000","last_undegraded":"2024-03-04T08:28:07.133542+0000","last_fullsized":"2024-03-04T08:28:07.133542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:22.953982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":266,"num_read_kb":99836,"num_write":937,"num_write_kb":59254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"80'2189","reported_seq":4759,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.028167+0000","last_change":"2024-03-04T08:19:36.163661+0000","last_active":"2024-03-04T08:27:57.028167+0000","last_peered":"2024-03-04T08:27:57.028167+0000","last_clean":"2024-03-04T08:27:57.028167+0000","last_became_active":"2024-03-04T07:33:56.548113+0000","last_became_peered":"2024-03-04T07:33:56.548113+0000","last_unstale":"2024-03-04T08:27:57.028167+0000","last_undegraded":"2024-03-04T08:27:57.028167+0000","last_fullsized":"2024-03-04T08:27:57.028167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2189,"log_dups_size":0,"ondisk_log_size":2189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:13:34.466627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18041650100000001,"stat_sum":{"num_bytes":12585055,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":11871,"num_write":1338,"num_write_kb":24818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'700","reported_seq":976,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.141049+0000","last_change":"2024-03-04T07:33:55.538871+0000","last_active":"2024-03-04T08:27:57.141049+0000","last_peered":"2024-03-04T08:27:57.141049+0000","last_clean":"2024-03-04T08:27:57.141049+0000","last_became_active":"2024-03-04T07:33:55.537221+0000","last_became_peered":"2024-03-04T07:33:55.537221+0000","last_unstale":"2024-03-04T08:27:57.141049+0000","last_undegraded":"2024-03-04T08:27:57.141049+0000","last_fullsized":"2024-03-04T08:27:57.141049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:02:34.533330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":222,"num_read_kb":137755,"num_write":800,"num_write_kb":83574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"82'2101","reported_seq":4719,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:12.030686+0000","last_change":"2024-03-04T08:19:36.165896+0000","last_active":"2024-03-04T08:28:12.030686+0000","last_peered":"2024-03-04T08:28:12.030686+0000","last_clean":"2024-03-04T08:28:12.030686+0000","last_became_active":"2024-03-04T07:33:56.543354+0000","last_became_peered":"2024-03-04T07:33:56.543354+0000","last_unstale":"2024-03-04T08:28:12.030686+0000","last_undegraded":"2024-03-04T08:28:12.030686+0000","last_fullsized":"2024-03-04T08:28:12.030686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2101,"log_dups_size":0,"ondisk_log_size":2101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:52:28.820568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17976278000000001,"stat_sum":{"num_bytes":25168099,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":104,"num_read_kb":70965,"num_write":1341,"num_write_kb":36170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'895","reported_seq":1177,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:58.056491+0000","last_change":"2024-03-04T07:33:55.552856+0000","last_active":"2024-03-04T08:27:58.056491+0000","last_peered":"2024-03-04T08:27:58.056491+0000","last_clean":"2024-03-04T08:27:58.056491+0000","last_became_active":"2024-03-04T07:33:55.552472+0000","last_became_peered":"2024-03-04T07:33:55.552472+0000","last_unstale":"2024-03-04T08:27:58.056491+0000","last_undegraded":"2024-03-04T08:27:58.056491+0000","last_fullsized":"2024-03-04T08:27:58.056491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:36:50.409567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364919,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":192,"num_read_kb":160923,"num_write":993,"num_write_kb":79673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"82'2097","reported_seq":4734,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:56.997663+0000","last_change":"2024-03-04T08:19:36.847924+0000","last_active":"2024-03-04T08:27:56.997663+0000","last_peered":"2024-03-04T08:27:56.997663+0000","last_clean":"2024-03-04T08:27:56.997663+0000","last_became_active":"2024-03-04T07:34:36.774424+0000","last_became_peered":"2024-03-04T07:34:36.774424+0000","last_unstale":"2024-03-04T08:27:56.997663+0000","last_undegraded":"2024-03-04T08:27:56.997663+0000","last_fullsized":"2024-03-04T08:27:56.997663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2097,"log_dups_size":0,"ondisk_log_size":2097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:08:56.055772+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.85871435699999998,"stat_sum":{"num_bytes":8390169,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":104,"num_read_kb":28439,"num_write":1349,"num_write_kb":17437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"82'815","reported_seq":1145,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.141018+0000","last_change":"2024-03-04T07:33:55.560332+0000","last_active":"2024-03-04T08:27:57.141018+0000","last_peered":"2024-03-04T08:27:57.141018+0000","last_clean":"2024-03-04T08:27:57.141018+0000","last_became_active":"2024-03-04T07:33:55.560086+0000","last_became_peered":"2024-03-04T07:33:55.560086+0000","last_unstale":"2024-03-04T08:27:57.141018+0000","last_undegraded":"2024-03-04T08:27:57.141018+0000","last_fullsized":"2024-03-04T08:27:57.141018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:25:46.280657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":353,"num_read_kb":116715,"num_write":1050,"num_write_kb":64078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'2110","reported_seq":4723,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.028131+0000","last_change":"2024-03-04T08:19:36.163052+0000","last_active":"2024-03-04T08:27:57.028131+0000","last_peered":"2024-03-04T08:27:57.028131+0000","last_clean":"2024-03-04T08:27:57.028131+0000","last_became_active":"2024-03-04T07:33:56.559589+0000","last_became_peered":"2024-03-04T07:33:56.559589+0000","last_unstale":"2024-03-04T08:27:57.028131+0000","last_undegraded":"2024-03-04T08:27:57.028131+0000","last_fullsized":"2024-03-04T08:27:57.028131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2110,"log_dups_size":0,"ondisk_log_size":2110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:48:29.367330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17959140500000001,"stat_sum":{"num_bytes":4211072,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":108,"num_read_kb":1329,"num_write":1312,"num_write_kb":17303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":326,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:02.028901+0000","last_change":"2024-03-04T07:33:43.559630+0000","last_active":"2024-03-04T08:28:02.028901+0000","last_peered":"2024-03-04T08:28:02.028901+0000","last_clean":"2024-03-04T08:28:02.028901+0000","last_became_active":"2024-03-04T07:33:43.557489+0000","last_became_peered":"2024-03-04T07:33:43.557489+0000","last_unstale":"2024-03-04T08:28:02.028901+0000","last_undegraded":"2024-03-04T08:28:02.028901+0000","last_fullsized":"2024-03-04T08:28:02.028901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_clean_scrub_stamp":"2024-03-04T07:33:42.183823+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:55:35.762857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'444","reported_seq":744,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:06.999157+0000","last_change":"2024-03-04T07:33:55.548616+0000","last_active":"2024-03-04T08:28:06.999157+0000","last_peered":"2024-03-04T08:28:06.999157+0000","last_clean":"2024-03-04T08:28:06.999157+0000","last_became_active":"2024-03-04T07:33:55.547322+0000","last_became_peered":"2024-03-04T07:33:55.547322+0000","last_unstale":"2024-03-04T08:28:06.999157+0000","last_undegraded":"2024-03-04T08:28:06.999157+0000","last_fullsized":"2024-03-04T08:28:06.999157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:20:16.717483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165858,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":253,"num_read_kb":128383,"num_write":599,"num_write_kb":63285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"82'2192","reported_seq":4778,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.517999+0000","last_change":"2024-03-04T08:19:52.492136+0000","last_active":"2024-03-04T08:28:08.517999+0000","last_peered":"2024-03-04T08:28:08.517999+0000","last_clean":"2024-03-04T08:28:08.517999+0000","last_became_active":"2024-03-04T07:33:56.559581+0000","last_became_peered":"2024-03-04T07:33:56.559581+0000","last_unstale":"2024-03-04T08:28:08.517999+0000","last_undegraded":"2024-03-04T08:28:08.517999+0000","last_fullsized":"2024-03-04T08:28:08.517999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2192,"log_dups_size":0,"ondisk_log_size":2192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:24:09.497555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.4208662280000004,"stat_sum":{"num_bytes":12628568,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":97,"num_read_kb":23350,"num_write":1376,"num_write_kb":26319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'785","reported_seq":1076,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:12.030530+0000","last_change":"2024-03-04T07:33:55.563905+0000","last_active":"2024-03-04T08:28:12.030530+0000","last_peered":"2024-03-04T08:28:12.030530+0000","last_clean":"2024-03-04T08:28:12.030530+0000","last_became_active":"2024-03-04T07:33:55.563117+0000","last_became_peered":"2024-03-04T07:33:55.563117+0000","last_unstale":"2024-03-04T08:28:12.030530+0000","last_undegraded":"2024-03-04T08:28:12.030530+0000","last_fullsized":"2024-03-04T08:28:12.030530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:53:02.522757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":261,"num_read_kb":111921,"num_write":932,"num_write_kb":63511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'2022","reported_seq":4660,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.141979+0000","last_change":"2024-03-04T08:19:49.113482+0000","last_active":"2024-03-04T08:28:07.141979+0000","last_peered":"2024-03-04T08:28:07.141979+0000","last_clean":"2024-03-04T08:28:07.141979+0000","last_became_active":"2024-03-04T07:33:56.530776+0000","last_became_peered":"2024-03-04T07:33:56.530776+0000","last_unstale":"2024-03-04T08:28:07.141979+0000","last_undegraded":"2024-03-04T08:28:07.141979+0000","last_fullsized":"2024-03-04T08:28:07.141979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2022,"log_dups_size":0,"ondisk_log_size":2022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:59:39.265011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.715426973,"stat_sum":{"num_bytes":4196564,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":94,"num_read_kb":11883,"num_write":1291,"num_write_kb":12915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"84'1859","reported_seq":3279,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.141993+0000","last_change":"2024-03-04T07:33:55.538631+0000","last_active":"2024-03-04T08:28:07.141993+0000","last_peered":"2024-03-04T08:28:07.141993+0000","last_clean":"2024-03-04T08:28:07.141993+0000","last_became_active":"2024-03-04T07:33:55.537104+0000","last_became_peered":"2024-03-04T07:33:55.537104+0000","last_unstale":"2024-03-04T08:28:07.141993+0000","last_undegraded":"2024-03-04T08:28:07.141993+0000","last_fullsized":"2024-03-04T08:28:07.141993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1859,"log_dups_size":0,"ondisk_log_size":1859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:27:06.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165914,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1355,"num_read_kb":144971,"num_write":1920,"num_write_kb":84003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"82'2174","reported_seq":4784,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057686+0000","last_change":"2024-03-04T08:19:35.433750+0000","last_active":"2024-03-04T08:28:08.057686+0000","last_peered":"2024-03-04T08:28:08.057686+0000","last_clean":"2024-03-04T08:28:08.057686+0000","last_became_active":"2024-03-04T07:33:56.540536+0000","last_became_peered":"2024-03-04T07:33:56.540536+0000","last_unstale":"2024-03-04T08:28:08.057686+0000","last_undegraded":"2024-03-04T08:28:08.057686+0000","last_fullsized":"2024-03-04T08:28:08.057686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2174,"log_dups_size":0,"ondisk_log_size":2174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:23:13.702668+0000","scrub_duration":0,"objects_trimmed":166,"snaptrim_duration":2.7381961829999999,"stat_sum":{"num_bytes":19342025,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":108,"num_read_kb":53174,"num_write":1367,"num_write_kb":31960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"84'854","reported_seq":1121,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:06.999129+0000","last_change":"2024-03-04T07:33:55.574740+0000","last_active":"2024-03-04T08:28:06.999129+0000","last_peered":"2024-03-04T08:28:06.999129+0000","last_clean":"2024-03-04T08:28:06.999129+0000","last_became_active":"2024-03-04T07:33:55.574129+0000","last_became_peered":"2024-03-04T07:33:55.574129+0000","last_unstale":"2024-03-04T08:28:06.999129+0000","last_undegraded":"2024-03-04T08:28:06.999129+0000","last_fullsized":"2024-03-04T08:28:06.999129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:40:57.891803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":189,"num_read_kb":128430,"num_write":957,"num_write_kb":67580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"82'2115","reported_seq":4827,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.133511+0000","last_change":"2024-03-04T08:19:33.358585+0000","last_active":"2024-03-04T08:28:07.133511+0000","last_peered":"2024-03-04T08:28:07.133511+0000","last_clean":"2024-03-04T08:28:07.133511+0000","last_became_active":"2024-03-04T07:33:56.541422+0000","last_became_peered":"2024-03-04T07:33:56.541422+0000","last_unstale":"2024-03-04T08:28:07.133511+0000","last_undegraded":"2024-03-04T08:28:07.133511+0000","last_fullsized":"2024-03-04T08:28:07.133511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2115,"log_dups_size":0,"ondisk_log_size":2115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:02:25.299647+0000","scrub_duration":0,"objects_trimmed":76,"snaptrim_duration":0.66844205800000001,"stat_sum":{"num_bytes":8391536,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":186,"num_read_kb":5163,"num_write":1360,"num_write_kb":16715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"85'649","reported_seq":906,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057737+0000","last_change":"2024-03-04T07:33:55.539317+0000","last_active":"2024-03-04T08:28:08.057737+0000","last_peered":"2024-03-04T08:28:08.057737+0000","last_clean":"2024-03-04T08:28:08.057737+0000","last_became_active":"2024-03-04T07:33:55.539017+0000","last_became_peered":"2024-03-04T07:33:55.539017+0000","last_unstale":"2024-03-04T08:28:08.057737+0000","last_undegraded":"2024-03-04T08:28:08.057737+0000","last_fullsized":"2024-03-04T08:28:08.057737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:10.787316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":165,"num_read_kb":131936,"num_write":747,"num_write_kb":79565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"82'2168","reported_seq":4796,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.001381+0000","last_change":"2024-03-04T08:19:36.855176+0000","last_active":"2024-03-04T08:28:22.001381+0000","last_peered":"2024-03-04T08:28:22.001381+0000","last_clean":"2024-03-04T08:28:22.001381+0000","last_became_active":"2024-03-04T07:33:56.553742+0000","last_became_peered":"2024-03-04T07:33:56.553742+0000","last_unstale":"2024-03-04T08:28:22.001381+0000","last_undegraded":"2024-03-04T08:28:22.001381+0000","last_fullsized":"2024-03-04T08:28:22.001381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2168,"log_dups_size":0,"ondisk_log_size":2168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:27.955824+0000","scrub_duration":0,"objects_trimmed":68,"snaptrim_duration":0.86588785899999998,"stat_sum":{"num_bytes":25169512,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":118,"num_read_kb":50143,"num_write":1386,"num_write_kb":35653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"82'874","reported_seq":1154,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.032175+0000","last_change":"2024-03-04T07:33:55.569442+0000","last_active":"2024-03-04T08:28:22.032175+0000","last_peered":"2024-03-04T08:28:22.032175+0000","last_clean":"2024-03-04T08:28:22.032175+0000","last_became_active":"2024-03-04T07:33:55.567809+0000","last_became_peered":"2024-03-04T07:33:55.567809+0000","last_unstale":"2024-03-04T08:28:22.032175+0000","last_undegraded":"2024-03-04T08:28:22.032175+0000","last_fullsized":"2024-03-04T08:28:22.032175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:58.551866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971542,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":208,"num_read_kb":126955,"num_write":1015,"num_write_kb":67473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"80'2113","reported_seq":4760,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057728+0000","last_change":"2024-03-04T08:19:35.449422+0000","last_active":"2024-03-04T08:28:08.057728+0000","last_peered":"2024-03-04T08:28:08.057728+0000","last_clean":"2024-03-04T08:28:08.057728+0000","last_became_active":"2024-03-04T07:33:56.542775+0000","last_became_peered":"2024-03-04T07:33:56.542775+0000","last_unstale":"2024-03-04T08:28:08.057728+0000","last_undegraded":"2024-03-04T08:28:08.057728+0000","last_fullsized":"2024-03-04T08:28:08.057728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2113,"log_dups_size":0,"ondisk_log_size":2113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:24:08.926542+0000","scrub_duration":0,"objects_trimmed":162,"snaptrim_duration":2.754112347,"stat_sum":{"num_bytes":21241084,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":124,"num_read_kb":50364,"num_write":1380,"num_write_kb":30457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"82'1819","reported_seq":2086,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057769+0000","last_change":"2024-03-04T07:33:55.545598+0000","last_active":"2024-03-04T08:28:08.057769+0000","last_peered":"2024-03-04T08:28:08.057769+0000","last_clean":"2024-03-04T08:28:08.057769+0000","last_became_active":"2024-03-04T07:33:55.545262+0000","last_became_peered":"2024-03-04T07:33:55.545262+0000","last_unstale":"2024-03-04T08:28:08.057769+0000","last_undegraded":"2024-03-04T08:28:08.057769+0000","last_fullsized":"2024-03-04T08:28:08.057769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1819,"log_dups_size":0,"ondisk_log_size":1819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:01.667243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":214,"num_read_kb":99465,"num_write":3832,"num_write_kb":59968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"80'1975","reported_seq":4630,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.518040+0000","last_change":"2024-03-04T08:19:52.454759+0000","last_active":"2024-03-04T08:28:08.518040+0000","last_peered":"2024-03-04T08:28:08.518040+0000","last_clean":"2024-03-04T08:28:08.518040+0000","last_became_active":"2024-03-04T07:33:56.550604+0000","last_became_peered":"2024-03-04T07:33:56.550604+0000","last_unstale":"2024-03-04T08:28:08.518040+0000","last_undegraded":"2024-03-04T08:28:08.518040+0000","last_fullsized":"2024-03-04T08:28:08.518040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1975,"log_dups_size":0,"ondisk_log_size":1975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:17:11.625595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.366898226,"stat_sum":{"num_bytes":4218390,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":111,"num_read_kb":1574,"num_write":1260,"num_write_kb":17353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'484","reported_seq":758,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.002985+0000","last_change":"2024-03-04T07:33:55.567220+0000","last_active":"2024-03-04T08:28:07.002985+0000","last_peered":"2024-03-04T08:28:07.002985+0000","last_clean":"2024-03-04T08:28:07.002985+0000","last_became_active":"2024-03-04T07:33:55.566600+0000","last_became_peered":"2024-03-04T07:33:55.566600+0000","last_unstale":"2024-03-04T08:28:07.002985+0000","last_undegraded":"2024-03-04T08:28:07.002985+0000","last_fullsized":"2024-03-04T08:28:07.002985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.876938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":246,"num_read_kb":67057,"num_write":625,"num_write_kb":42779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"80'2039","reported_seq":4677,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142035+0000","last_change":"2024-03-04T08:19:49.347949+0000","last_active":"2024-03-04T08:28:07.142035+0000","last_peered":"2024-03-04T08:28:07.142035+0000","last_clean":"2024-03-04T08:28:07.142035+0000","last_became_active":"2024-03-04T07:33:56.543836+0000","last_became_peered":"2024-03-04T07:33:56.543836+0000","last_unstale":"2024-03-04T08:28:07.142035+0000","last_undegraded":"2024-03-04T08:28:07.142035+0000","last_fullsized":"2024-03-04T08:28:07.142035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2039,"log_dups_size":0,"ondisk_log_size":2039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:44:01.157492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.94967019,"stat_sum":{"num_bytes":8393261,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":91,"num_read_kb":20301,"num_write":1337,"num_write_kb":17823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'627","reported_seq":911,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.029678+0000","last_change":"2024-03-04T07:33:55.566886+0000","last_active":"2024-03-04T08:28:07.029678+0000","last_peered":"2024-03-04T08:28:07.029678+0000","last_clean":"2024-03-04T08:28:07.029678+0000","last_became_active":"2024-03-04T07:33:55.566048+0000","last_became_peered":"2024-03-04T07:33:55.566048+0000","last_unstale":"2024-03-04T08:28:07.029678+0000","last_undegraded":"2024-03-04T08:28:07.029678+0000","last_fullsized":"2024-03-04T08:28:07.029678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:45:27.190438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":244,"num_read_kb":91476,"num_write":742,"num_write_kb":42693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"80'2170","reported_seq":4766,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.133468+0000","last_change":"2024-03-04T08:19:33.369171+0000","last_active":"2024-03-04T08:28:07.133468+0000","last_peered":"2024-03-04T08:28:07.133468+0000","last_clean":"2024-03-04T08:28:07.133468+0000","last_became_active":"2024-03-04T07:33:56.531901+0000","last_became_peered":"2024-03-04T07:33:56.531901+0000","last_unstale":"2024-03-04T08:28:07.133468+0000","last_undegraded":"2024-03-04T08:28:07.133468+0000","last_fullsized":"2024-03-04T08:28:07.133468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2170,"log_dups_size":0,"ondisk_log_size":2170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:43:17.466639+0000","scrub_duration":0,"objects_trimmed":68,"snaptrim_duration":0.67861760500000001,"stat_sum":{"num_bytes":12583051,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":98,"num_read_kb":23140,"num_write":1358,"num_write_kb":19821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"85'800","reported_seq":1103,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057715+0000","last_change":"2024-03-04T07:33:55.557568+0000","last_active":"2024-03-04T08:28:08.057715+0000","last_peered":"2024-03-04T08:28:08.057715+0000","last_clean":"2024-03-04T08:28:08.057715+0000","last_became_active":"2024-03-04T07:33:55.557330+0000","last_became_peered":"2024-03-04T07:33:55.557330+0000","last_unstale":"2024-03-04T08:28:08.057715+0000","last_undegraded":"2024-03-04T08:28:08.057715+0000","last_fullsized":"2024-03-04T08:28:08.057715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:34.956318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18594593,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":285,"num_read_kb":97953,"num_write":952,"num_write_kb":44586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"80'2245","reported_seq":4842,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.517974+0000","last_change":"2024-03-04T08:19:52.542997+0000","last_active":"2024-03-04T08:28:08.517974+0000","last_peered":"2024-03-04T08:28:08.517974+0000","last_clean":"2024-03-04T08:28:08.517974+0000","last_became_active":"2024-03-04T07:33:56.552658+0000","last_became_peered":"2024-03-04T07:33:56.552658+0000","last_unstale":"2024-03-04T08:28:08.517974+0000","last_undegraded":"2024-03-04T08:28:08.517974+0000","last_fullsized":"2024-03-04T08:28:08.517974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2245,"log_dups_size":0,"ondisk_log_size":2245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:02:38.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.4718202119999999,"stat_sum":{"num_bytes":2645,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":101,"num_read_kb":1927,"num_write":1438,"num_write_kb":12677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'827","reported_seq":1131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:18.058654+0000","last_change":"2024-03-04T07:33:55.536480+0000","last_active":"2024-03-04T08:28:18.058654+0000","last_peered":"2024-03-04T08:28:18.058654+0000","last_clean":"2024-03-04T08:28:18.058654+0000","last_became_active":"2024-03-04T07:33:55.536139+0000","last_became_peered":"2024-03-04T07:33:55.536139+0000","last_unstale":"2024-03-04T08:28:18.058654+0000","last_undegraded":"2024-03-04T08:28:18.058654+0000","last_fullsized":"2024-03-04T08:28:18.058654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:12.778026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":276,"num_read_kb":124915,"num_write":1046,"num_write_kb":68013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"80'2139","reported_seq":4762,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:58.516982+0000","last_change":"2024-03-04T08:19:52.574166+0000","last_active":"2024-03-04T08:27:58.516982+0000","last_peered":"2024-03-04T08:27:58.516982+0000","last_clean":"2024-03-04T08:27:58.516982+0000","last_became_active":"2024-03-04T07:33:56.546280+0000","last_became_peered":"2024-03-04T07:33:56.546280+0000","last_unstale":"2024-03-04T08:27:58.516982+0000","last_undegraded":"2024-03-04T08:27:58.516982+0000","last_fullsized":"2024-03-04T08:27:58.516982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2139,"log_dups_size":0,"ondisk_log_size":2139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:04:37.188813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.5027303859999996,"stat_sum":{"num_bytes":8389588,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":116,"num_read_kb":19661,"num_write":1373,"num_write_kb":18499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'895","reported_seq":1206,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.028091+0000","last_change":"2024-03-04T07:33:55.572540+0000","last_active":"2024-03-04T08:27:57.028091+0000","last_peered":"2024-03-04T08:27:57.028091+0000","last_clean":"2024-03-04T08:27:57.028091+0000","last_became_active":"2024-03-04T07:33:55.571612+0000","last_became_peered":"2024-03-04T07:33:55.571612+0000","last_unstale":"2024-03-04T08:27:57.028091+0000","last_undegraded":"2024-03-04T08:27:57.028091+0000","last_fullsized":"2024-03-04T08:27:57.028091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:01:43.741423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":286,"num_read_kb":176599,"num_write":1032,"num_write_kb":79834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"80'2075","reported_seq":4727,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.005213+0000","last_change":"2024-03-04T08:19:34.379981+0000","last_active":"2024-03-04T08:28:22.005213+0000","last_peered":"2024-03-04T08:28:22.005213+0000","last_clean":"2024-03-04T08:28:22.005213+0000","last_became_active":"2024-03-04T07:33:56.532245+0000","last_became_peered":"2024-03-04T07:33:56.532245+0000","last_unstale":"2024-03-04T08:28:22.005213+0000","last_undegraded":"2024-03-04T08:28:22.005213+0000","last_fullsized":"2024-03-04T08:28:22.005213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2075,"log_dups_size":0,"ondisk_log_size":2075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:17:47.216527+0000","scrub_duration":0,"objects_trimmed":129,"snaptrim_duration":1.668147552,"stat_sum":{"num_bytes":16789440,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":116,"num_read_kb":38061,"num_write":1331,"num_write_kb":29084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"82'2080","reported_seq":4692,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.984492+0000","last_change":"2024-03-04T08:19:36.839132+0000","last_active":"2024-03-04T08:26:26.984492+0000","last_peered":"2024-03-04T08:26:26.984492+0000","last_clean":"2024-03-04T08:26:26.984492+0000","last_became_active":"2024-03-04T07:33:56.563473+0000","last_became_peered":"2024-03-04T07:33:56.563473+0000","last_unstale":"2024-03-04T08:26:26.984492+0000","last_undegraded":"2024-03-04T08:26:26.984492+0000","last_fullsized":"2024-03-04T08:26:26.984492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2080,"log_dups_size":0,"ondisk_log_size":2080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:46:20.229965+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.84949308899999998,"stat_sum":{"num_bytes":8388810,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":102,"num_read_kb":13463,"num_write":1308,"num_write_kb":22727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'689","reported_seq":944,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014567+0000","last_change":"2024-03-04T07:33:55.575688+0000","last_active":"2024-03-04T08:26:27.014567+0000","last_peered":"2024-03-04T08:26:27.014567+0000","last_clean":"2024-03-04T08:26:27.014567+0000","last_became_active":"2024-03-04T07:33:55.574766+0000","last_became_peered":"2024-03-04T07:33:55.574766+0000","last_unstale":"2024-03-04T08:26:27.014567+0000","last_undegraded":"2024-03-04T08:26:27.014567+0000","last_fullsized":"2024-03-04T08:26:27.014567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:32:19.708016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":185,"num_read_kb":103365,"num_write":788,"num_write_kb":63233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"80'2223","reported_seq":4823,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.984465+0000","last_change":"2024-03-04T08:19:36.844614+0000","last_active":"2024-03-04T08:26:26.984465+0000","last_peered":"2024-03-04T08:26:26.984465+0000","last_clean":"2024-03-04T08:26:26.984465+0000","last_became_active":"2024-03-04T07:33:56.563474+0000","last_became_peered":"2024-03-04T07:33:56.563474+0000","last_unstale":"2024-03-04T08:26:26.984465+0000","last_undegraded":"2024-03-04T08:26:26.984465+0000","last_fullsized":"2024-03-04T08:26:26.984465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2223,"log_dups_size":0,"ondisk_log_size":2223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:11:22.319394+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.85581109,"stat_sum":{"num_bytes":20978408,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":117,"num_read_kb":36831,"num_write":1409,"num_write_kb":34422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"85'764","reported_seq":1068,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014595+0000","last_change":"2024-03-04T07:33:55.570016+0000","last_active":"2024-03-04T08:26:27.014595+0000","last_peered":"2024-03-04T08:26:27.014595+0000","last_clean":"2024-03-04T08:26:27.014595+0000","last_became_active":"2024-03-04T07:33:55.568749+0000","last_became_peered":"2024-03-04T07:33:55.568749+0000","last_unstale":"2024-03-04T08:26:27.014595+0000","last_undegraded":"2024-03-04T08:26:27.014595+0000","last_fullsized":"2024-03-04T08:26:27.014595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:01.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":279,"num_read_kb":188218,"num_write":863,"num_write_kb":100106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"80'2088","reported_seq":4689,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142049+0000","last_change":"2024-03-04T08:19:49.462793+0000","last_active":"2024-03-04T08:28:07.142049+0000","last_peered":"2024-03-04T08:28:07.142049+0000","last_clean":"2024-03-04T08:28:07.142049+0000","last_became_active":"2024-03-04T07:33:56.561675+0000","last_became_peered":"2024-03-04T07:33:56.561675+0000","last_unstale":"2024-03-04T08:28:07.142049+0000","last_undegraded":"2024-03-04T08:28:07.142049+0000","last_fullsized":"2024-03-04T08:28:07.142049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2088,"log_dups_size":0,"ondisk_log_size":2088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:36:21.385199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0645439379999999,"stat_sum":{"num_bytes":8398756,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":91,"num_read_kb":13948,"num_write":1308,"num_write_kb":18354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'844","reported_seq":1117,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:03.517495+0000","last_change":"2024-03-04T07:33:55.538337+0000","last_active":"2024-03-04T08:28:03.517495+0000","last_peered":"2024-03-04T08:28:03.517495+0000","last_clean":"2024-03-04T08:28:03.517495+0000","last_became_active":"2024-03-04T07:33:55.538010+0000","last_became_peered":"2024-03-04T07:33:55.538010+0000","last_unstale":"2024-03-04T08:28:03.517495+0000","last_undegraded":"2024-03-04T08:28:03.517495+0000","last_fullsized":"2024-03-04T08:28:03.517495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:47:15.711205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":199,"num_read_kb":128064,"num_write":938,"num_write_kb":63013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'2071","reported_seq":4691,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:03.517472+0000","last_change":"2024-03-04T08:19:52.402904+0000","last_active":"2024-03-04T08:28:03.517472+0000","last_peered":"2024-03-04T08:28:03.517472+0000","last_clean":"2024-03-04T08:28:03.517472+0000","last_became_active":"2024-03-04T07:33:56.558799+0000","last_became_peered":"2024-03-04T07:33:56.558799+0000","last_unstale":"2024-03-04T08:28:03.517472+0000","last_undegraded":"2024-03-04T08:28:03.517472+0000","last_fullsized":"2024-03-04T08:28:03.517472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2071,"log_dups_size":0,"ondisk_log_size":2071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:34:51.677019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.3308552689999997,"stat_sum":{"num_bytes":16779138,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":99,"num_read_kb":47695,"num_write":1322,"num_write_kb":28136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'627","reported_seq":937,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142064+0000","last_change":"2024-03-04T07:33:55.566294+0000","last_active":"2024-03-04T08:28:07.142064+0000","last_peered":"2024-03-04T08:28:07.142064+0000","last_clean":"2024-03-04T08:28:07.142064+0000","last_became_active":"2024-03-04T07:33:55.565941+0000","last_became_peered":"2024-03-04T07:33:55.565941+0000","last_unstale":"2024-03-04T08:28:07.142064+0000","last_undegraded":"2024-03-04T08:28:07.142064+0000","last_fullsized":"2024-03-04T08:28:07.142064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:03.115085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":306,"num_read_kb":111793,"num_write":679,"num_write_kb":46654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":758811247,"num_objects":4817,"num_object_clones":0,"num_object_copies":9634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4817,"num_whiteouts":0,"num_read":7109,"num_read_kb":1683514,"num_write":86591,"num_write_kb":1471650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1519542272,"data_stored":1517622494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137094,"ondisk_log_size":137094,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":589610469,"num_objects":1415,"num_object_clones":0,"num_object_copies":2830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1415,"num_whiteouts":0,"num_read":9520,"num_read_kb":4046989,"num_write":33636,"num_write_kb":2138987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1203,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1179369472,"data_stored":1179220938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27575,"ondisk_log_size":27575,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424510129,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537624,"kb_used_data":362128,"kb_used_omap":1,"kb_used_meta":175486,"kb_avail":93834216,"statfs":{"total":96636764160,"available":96086237184,"internally_reserved":0,"allocated":370819072,"data_stored":370026846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":179698119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510128,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466432,"kb_used_data":300664,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":93905408,"statfs":{"total":96636764160,"available":96159137792,"internally_reserved":0,"allocated":307879936,"data_stored":307237685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542833,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564504,"kb_used_data":389072,"kb_used_omap":1,"kb_used_meta":175422,"kb_avail":93807336,"statfs":{"total":96636764160,"available":96058712064,"internally_reserved":0,"allocated":398409728,"data_stored":397634775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":179632586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542832,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505096,"kb_used_data":321344,"kb_used_omap":1,"kb_used_meta":183742,"kb_avail":93866744,"statfs":{"total":96636764160,"available":96119545856,"internally_reserved":0,"allocated":329056256,"data_stored":328434784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188152262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542832,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583216,"kb_used_data":403556,"kb_used_omap":1,"kb_used_meta":179646,"kb_avail":93788624,"statfs":{"total":96636764160,"available":96039550976,"internally_reserved":0,"allocated":413241344,"data_stored":412455445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183957957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542833,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509020,"kb_used_data":329360,"kb_used_omap":1,"kb_used_meta":179646,"kb_avail":93862820,"statfs":{"total":96636764160,"available":96115527680,"internally_reserved":0,"allocated":337264640,"data_stored":336553162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183957957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542833,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340760,"kb_used_data":241612,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94031080,"statfs":{"total":96636764160,"available":96287825920,"internally_reserved":0,"allocated":247410688,"data_stored":246748656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542831,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462692,"kb_used_data":296920,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":93909148,"statfs":{"total":96636764160,"available":96162967552,"internally_reserved":0,"allocated":304046080,"data_stored":303311191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":169736651},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121651200,"data_stored":121635476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157036544,"data_stored":157012200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92295168,"data_stored":92275506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":151015424,"data_stored":150997655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":104878080,"data_stored":104864685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180957184,"data_stored":180935092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":168370176,"data_stored":168353802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":203165696,"data_stored":203146522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":180797440,"data_stored":180538050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149839872,"data_stored":149678188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":154112000,"data_stored":153925853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":218800128,"data_stored":218481894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":231383040,"data_stored":231141180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":231280640,"data_stored":230973056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":159682560,"data_stored":159533685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":193646592,"data_stored":193350588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T08:30:45.950 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-04T08:30:45.950 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T08:30:46.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.131+0000 7fac9f200640 1 Processor -- start 2024-03-04T08:30:46.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.131+0000 7fac9f200640 1 -- start start 2024-03-04T08:30:46.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac9f200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fac981527f0 0x7fac98070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac9f200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fac980c8200 0x7fac98072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac9f200640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac981591f0 con 0x7fac98152bd0 2024-03-04T08:30:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac9f200640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac98071df0 con 0x7fac981527f0 2024-03-04T08:30:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac9f200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac981597b0 con 0x7fac980c8200 2024-03-04T08:30:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac97e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fac981527f0 0x7fac98070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac97e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fac981527f0 0x7fac98070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52998/0 (socket says 172.21.15.113:52998) 2024-03-04T08:30:46.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac97e00640 1 -- 172.21.15.113:0/2933352959 learned_addr learned my addr 172.21.15.113:0/2933352959 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:46.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac97400640 1 --2- 172.21.15.113:0/2933352959 >> v2:172.21.15.136:3300/0 conn(0x7fac980c8200 0x7fac98072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac97e00640 1 -- 172.21.15.113:0/2933352959 >> v2:172.21.15.136:3300/0 conn(0x7fac980c8200 msgr2=0x7fac98072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:46.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac97e00640 1 --2- 172.21.15.113:0/2933352959 >> v2:172.21.15.136:3300/0 conn(0x7fac980c8200 0x7fac98072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac97e00640 1 -- 172.21.15.113:0/2933352959 >> v1:172.21.15.136:6789/0 conn(0x7fac98152bd0 legacy=0x7fac981587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.135+0000 7fac97e00640 1 -- 172.21.15.113:0/2933352959 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac981594d0 con 0x7fac981527f0 2024-03-04T08:30:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac97e00640 1 --2- 172.21.15.113:0/2933352959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 0x7fac98070d40 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fac980cd230 tx=0x7fac8801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d55baa1d962dd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:46.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac96a00640 1 -- 172.21.15.113:0/2933352959 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac880a57d0 con 0x7fac981527f0 2024-03-04T08:30:46.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac96a00640 1 -- 172.21.15.113:0/2933352959 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac880a5970 con 0x7fac981527f0 2024-03-04T08:30:46.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac96a00640 1 -- 172.21.15.113:0/2933352959 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac880a5d40 con 0x7fac981527f0 2024-03-04T08:30:46.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 -- 172.21.15.113:0/2933352959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 msgr2=0x7fac98070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 --2- 172.21.15.113:0/2933352959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 0x7fac98070d40 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fac980cd230 tx=0x7fac8801c9f0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 -- 172.21.15.113:0/2933352959 shutdown_connections 2024-03-04T08:30:46.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 --2- 172.21.15.113:0/2933352959 >> v2:172.21.15.136:3300/0 conn(0x7fac980c8200 0x7fac98072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 --2- 172.21.15.113:0/2933352959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 0x7fac98070d40 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 -- 172.21.15.113:0/2933352959 >> 172.21.15.113:0/2933352959 conn(0x7fac9806e820 msgr2=0x7fac9806ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:46.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 -- 172.21.15.113:0/2933352959 shutdown_connections 2024-03-04T08:30:46.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 -- 172.21.15.113:0/2933352959 wait complete. 2024-03-04T08:30:46.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 Processor -- start 2024-03-04T08:30:46.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 -- start start 2024-03-04T08:30:46.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac980c8200 0x7fac9814a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 0x7fac980b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac98152bd0 0x7fac980b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac98165160 con 0x7fac981527f0 2024-03-04T08:30:46.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac980727c0 con 0x7fac980c8200 2024-03-04T08:30:46.139 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac9f200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac981652e0 con 0x7fac98152bd0 2024-03-04T08:30:46.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac97400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 0x7fac980b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac97400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 0x7fac980b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53002/0 (socket says 172.21.15.113:53002) 2024-03-04T08:30:46.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac97400640 1 -- 172.21.15.113:0/2926739462 learned_addr learned my addr 172.21.15.113:0/2926739462 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:46.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.139+0000 7fac97e00640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac980c8200 0x7fac9814a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac9d200640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac98152bd0 0x7fac980b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac97400640 1 -- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac98152bd0 msgr2=0x7fac980b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac97400640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac98152bd0 0x7fac980b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.140 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac97400640 1 -- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac980c8200 msgr2=0x7fac9814a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac97400640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac980c8200 0x7fac9814a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac97400640 1 -- 172.21.15.113:0/2926739462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac981f6b90 con 0x7fac981527f0 2024-03-04T08:30:46.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac97400640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 0x7fac980b0a60 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fac8400e930 tx=0x7fac8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:46.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac8fe00640 1 -- 172.21.15.113:0/2926739462 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac84017070 con 0x7fac981527f0 2024-03-04T08:30:46.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac8fe00640 1 -- 172.21.15.113:0/2926739462 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8400ca00 con 0x7fac981527f0 2024-03-04T08:30:46.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac9f200640 1 -- 172.21.15.113:0/2926739462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac981f7a10 con 0x7fac981527f0 2024-03-04T08:30:46.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac8fe00640 1 -- 172.21.15.113:0/2926739462 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400cdf0 con 0x7fac981527f0 2024-03-04T08:30:46.141 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac9f200640 1 -- 172.21.15.113:0/2926739462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac981f70e0 con 0x7fac981527f0 2024-03-04T08:30:46.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac9f200640 1 -- 172.21.15.113:0/2926739462 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac98075c00 con 0x7fac981527f0 2024-03-04T08:30:46.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac8fe00640 1 -- 172.21.15.113:0/2926739462 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac8401a070 con 0x7fac981527f0 2024-03-04T08:30:46.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac8fe00640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f60 0x7fac78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac8fe00640 1 -- 172.21.15.113:0/2926739462 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fac84085e70 con 0x7fac981527f0 2024-03-04T08:30:46.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.143+0000 7fac97e00640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f60 0x7fac78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.143 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.147+0000 7fac97e00640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f60 0x7fac78069420 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fac880ae000 tx=0x7fac880a7070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:46.145 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.147+0000 7fac8fe00640 1 -- 172.21.15.113:0/2926739462 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fac84056490 con 0x7fac981527f0 2024-03-04T08:30:46.297 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.299+0000 7fac9f200640 1 -- 172.21.15.113:0/2926739462 --> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac98157890 con 0x7fac78066f60 2024-03-04T08:30:46.309 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.311+0000 7fac8fe00640 1 -- 172.21.15.113:0/2926739462 <== mgr.4103 v2:172.21.15.136:6832/3353359815 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256615 (secure 0 0 0) 0x7fac880091d0 con 0x7fac78066f60 2024-03-04T08:30:46.312 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:46.313 INFO:teuthology.orchestra.run.smithi113.stderr:dumped all 2024-03-04T08:30:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 -- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f60 msgr2=0x7fac78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f60 0x7fac78069420 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fac880ae000 tx=0x7fac880a7070 comp rx=0 tx=0).stop 2024-03-04T08:30:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 -- 172.21.15.113:0/2926739462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 msgr2=0x7fac980b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 0x7fac980b0a60 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fac8400e930 tx=0x7fac8400edf0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 -- 172.21.15.113:0/2926739462 shutdown_connections 2024-03-04T08:30:46.315 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac98152bd0 0x7fac980b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac78066f60 0x7fac78069420 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac981527f0 0x7fac980b0a60 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 --2- 172.21.15.113:0/2926739462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac980c8200 0x7fac9814a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 -- 172.21.15.113:0/2926739462 >> 172.21.15.113:0/2926739462 conn(0x7fac9806e820 msgr2=0x7fac980c9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 -- 172.21.15.113:0/2926739462 shutdown_connections 2024-03-04T08:30:46.316 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.315+0000 7fac9f200640 1 -- 172.21.15.113:0/2926739462 wait complete. 2024-03-04T08:30:46.332 INFO:teuthology.orchestra.run.smithi113.stdout:{"pg_ready":true,"pg_map":{"version":1789,"stamp":"2024-03-04T08:30:44.638868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1349012084,"num_objects":6234,"num_object_clones":0,"num_object_copies":12468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6234,"num_whiteouts":0,"num_read":16745,"num_read_kb":5730603,"num_write":120386,"num_write_kb":3613053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1203,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164786,"ondisk_log_size":164786,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":3969344,"kb_used_data":2644656,"kb_used_omap":12,"kb_used_meta":1324595,"kb_avail":751005376,"statfs":{"total":773094113280,"available":769029505024,"internally_reserved":0,"allocated":2708127744,"data_stored":2702402544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12746,"internal_metadata":1356385846},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004182"},"pg_stats":[{"pgid":"3.3f","version":"82'1995","reported_seq":4635,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.029763+0000","last_change":"2024-03-04T08:19:36.158827+0000","last_active":"2024-03-04T08:28:07.029763+0000","last_peered":"2024-03-04T08:28:07.029763+0000","last_clean":"2024-03-04T08:28:07.029763+0000","last_became_active":"2024-03-04T07:33:56.559031+0000","last_became_peered":"2024-03-04T07:33:56.559031+0000","last_unstale":"2024-03-04T08:28:07.029763+0000","last_undegraded":"2024-03-04T08:28:07.029763+0000","last_fullsized":"2024-03-04T08:28:07.029763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1995,"log_dups_size":0,"ondisk_log_size":1995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:14:15.903092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17246109400000001,"stat_sum":{"num_bytes":4212510,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":96,"num_read_kb":14433,"num_write":1280,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"80'1982","reported_seq":4617,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057758+0000","last_change":"2024-03-04T08:19:35.479098+0000","last_active":"2024-03-04T08:28:08.057758+0000","last_peered":"2024-03-04T08:28:08.057758+0000","last_clean":"2024-03-04T08:28:08.057758+0000","last_became_active":"2024-03-04T07:33:56.546547+0000","last_became_peered":"2024-03-04T07:33:56.546547+0000","last_unstale":"2024-03-04T08:28:08.057758+0000","last_undegraded":"2024-03-04T08:28:08.057758+0000","last_fullsized":"2024-03-04T08:28:08.057758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1982,"log_dups_size":0,"ondisk_log_size":1982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:14:54.677102+0000","scrub_duration":0,"objects_trimmed":173,"snaptrim_duration":2.7820693949999997,"stat_sum":{"num_bytes":8393198,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":106,"num_read_kb":21761,"num_write":1218,"num_write_kb":15425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"80'2113","reported_seq":4724,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142019+0000","last_change":"2024-03-04T08:19:49.392733+0000","last_active":"2024-03-04T08:28:07.142019+0000","last_peered":"2024-03-04T08:28:07.142019+0000","last_clean":"2024-03-04T08:28:07.142019+0000","last_became_active":"2024-03-04T07:33:56.530488+0000","last_became_peered":"2024-03-04T07:33:56.530488+0000","last_unstale":"2024-03-04T08:28:07.142019+0000","last_undegraded":"2024-03-04T08:28:07.142019+0000","last_fullsized":"2024-03-04T08:28:07.142019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2113,"log_dups_size":0,"ondisk_log_size":2113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:02:21.539335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.9646412140000002,"stat_sum":{"num_bytes":12588036,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":103,"num_read_kb":26195,"num_write":1342,"num_write_kb":31711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'2188","reported_seq":4775,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:42.133523+0000","last_change":"2024-03-04T08:19:32.727352+0000","last_active":"2024-03-04T08:26:42.133523+0000","last_peered":"2024-03-04T08:26:42.133523+0000","last_clean":"2024-03-04T08:26:42.133523+0000","last_became_active":"2024-03-04T07:34:36.767991+0000","last_became_peered":"2024-03-04T07:34:36.767991+0000","last_unstale":"2024-03-04T08:26:42.133523+0000","last_undegraded":"2024-03-04T08:26:42.133523+0000","last_fullsized":"2024-03-04T08:26:42.133523+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2188,"log_dups_size":0,"ondisk_log_size":2188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:23:52.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019148479999999999,"stat_sum":{"num_bytes":12586995,"num_objects":81,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":0,"num_read":106,"num_read_kb":25892,"num_write":1360,"num_write_kb":20414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"82'2280","reported_seq":4856,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014533+0000","last_change":"2024-03-04T08:19:36.167121+0000","last_active":"2024-03-04T08:26:27.014533+0000","last_peered":"2024-03-04T08:26:27.014533+0000","last_clean":"2024-03-04T08:26:27.014533+0000","last_became_active":"2024-03-04T07:33:56.540351+0000","last_became_peered":"2024-03-04T07:33:56.540351+0000","last_unstale":"2024-03-04T08:26:27.014533+0000","last_undegraded":"2024-03-04T08:26:27.014533+0000","last_fullsized":"2024-03-04T08:26:27.014533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2280,"log_dups_size":0,"ondisk_log_size":2280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:34:51.364307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.185369957,"stat_sum":{"num_bytes":16781857,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":103,"num_read_kb":39876,"num_write":1435,"num_write_kb":24971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"80'1985","reported_seq":4627,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.984410+0000","last_change":"2024-03-04T08:19:36.851047+0000","last_active":"2024-03-04T08:26:26.984410+0000","last_peered":"2024-03-04T08:26:26.984410+0000","last_clean":"2024-03-04T08:26:26.984410+0000","last_became_active":"2024-03-04T07:33:56.558046+0000","last_became_peered":"2024-03-04T07:33:56.558046+0000","last_unstale":"2024-03-04T08:26:26.984410+0000","last_undegraded":"2024-03-04T08:26:26.984410+0000","last_fullsized":"2024-03-04T08:26:26.984410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1985,"log_dups_size":0,"ondisk_log_size":1985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:00:43.652391+0000","scrub_duration":0,"objects_trimmed":64,"snaptrim_duration":0.86040996599999997,"stat_sum":{"num_bytes":8388608,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":103,"num_read_kb":25220,"num_write":1249,"num_write_kb":17763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"82'2102","reported_seq":4728,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014493+0000","last_change":"2024-03-04T08:19:36.170530+0000","last_active":"2024-03-04T08:26:27.014493+0000","last_peered":"2024-03-04T08:26:27.014493+0000","last_clean":"2024-03-04T08:26:27.014493+0000","last_became_active":"2024-03-04T07:33:56.543176+0000","last_became_peered":"2024-03-04T07:33:56.543176+0000","last_unstale":"2024-03-04T08:26:27.014493+0000","last_undegraded":"2024-03-04T08:26:27.014493+0000","last_fullsized":"2024-03-04T08:26:27.014493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2102,"log_dups_size":0,"ondisk_log_size":2102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:03:35.061415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.188850779,"stat_sum":{"num_bytes":12603630,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":109,"num_read_kb":34094,"num_write":1333,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"80'2020","reported_seq":4661,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:28.507759+0000","last_change":"2024-03-04T08:19:52.268424+0000","last_active":"2024-03-04T08:26:28.507759+0000","last_peered":"2024-03-04T08:26:28.507759+0000","last_clean":"2024-03-04T08:26:28.507759+0000","last_became_active":"2024-03-04T07:33:56.559172+0000","last_became_peered":"2024-03-04T07:33:56.559172+0000","last_unstale":"2024-03-04T08:26:28.507759+0000","last_undegraded":"2024-03-04T08:26:28.507759+0000","last_fullsized":"2024-03-04T08:26:28.507759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2020,"log_dups_size":0,"ondisk_log_size":2020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:02:26.219884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.1976442450000002,"stat_sum":{"num_bytes":12587627,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":103,"num_read_kb":17785,"num_write":1294,"num_write_kb":19994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'2213","reported_seq":4820,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:22.023180+0000","last_change":"2024-03-04T08:19:36.163414+0000","last_active":"2024-03-04T08:27:22.023180+0000","last_peered":"2024-03-04T08:27:22.023180+0000","last_clean":"2024-03-04T08:27:22.023180+0000","last_became_active":"2024-03-04T07:33:56.560306+0000","last_became_peered":"2024-03-04T07:33:56.560306+0000","last_unstale":"2024-03-04T08:27:22.023180+0000","last_undegraded":"2024-03-04T08:27:22.023180+0000","last_fullsized":"2024-03-04T08:27:22.023180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2213,"log_dups_size":0,"ondisk_log_size":2213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:32:42.699565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17780130699999999,"stat_sum":{"num_bytes":8400582,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":109,"num_read_kb":22139,"num_write":1436,"num_write_kb":20580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"80'2060","reported_seq":4723,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.994336+0000","last_change":"2024-03-04T08:19:34.383564+0000","last_active":"2024-03-04T08:27:06.994336+0000","last_peered":"2024-03-04T08:27:06.994336+0000","last_clean":"2024-03-04T08:27:06.994336+0000","last_became_active":"2024-03-04T07:33:56.560592+0000","last_became_peered":"2024-03-04T07:33:56.560592+0000","last_unstale":"2024-03-04T08:27:06.994336+0000","last_undegraded":"2024-03-04T08:27:06.994336+0000","last_fullsized":"2024-03-04T08:27:06.994336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2060,"log_dups_size":0,"ondisk_log_size":2060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:40:27.691315+0000","scrub_duration":0,"objects_trimmed":120,"snaptrim_duration":1.6725091160000001,"stat_sum":{"num_bytes":25176920,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":134,"num_read_kb":52032,"num_write":1326,"num_write_kb":32831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"82'2220","reported_seq":4778,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.994298+0000","last_change":"2024-03-04T08:19:34.375018+0000","last_active":"2024-03-04T08:27:06.994298+0000","last_peered":"2024-03-04T08:27:06.994298+0000","last_clean":"2024-03-04T08:27:06.994298+0000","last_became_active":"2024-03-04T07:33:56.560719+0000","last_became_peered":"2024-03-04T07:33:56.560719+0000","last_unstale":"2024-03-04T08:27:06.994298+0000","last_undegraded":"2024-03-04T08:27:06.994298+0000","last_fullsized":"2024-03-04T08:27:06.994298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2220,"log_dups_size":0,"ondisk_log_size":2220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:33:12.566752+0000","scrub_duration":0,"objects_trimmed":134,"snaptrim_duration":1.6642866970000001,"stat_sum":{"num_bytes":8390989,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":101,"num_read_kb":24838,"num_write":1357,"num_write_kb":15528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"80'2201","reported_seq":4804,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:07.125104+0000","last_change":"2024-03-04T08:19:33.340351+0000","last_active":"2024-03-04T08:27:07.125104+0000","last_peered":"2024-03-04T08:27:07.125104+0000","last_clean":"2024-03-04T08:27:07.125104+0000","last_became_active":"2024-03-04T07:33:56.549568+0000","last_became_peered":"2024-03-04T07:33:56.549568+0000","last_unstale":"2024-03-04T08:27:07.125104+0000","last_undegraded":"2024-03-04T08:27:07.125104+0000","last_fullsized":"2024-03-04T08:27:07.125104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2201,"log_dups_size":0,"ondisk_log_size":2201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:29:31.376806+0000","scrub_duration":0,"objects_trimmed":10,"snaptrim_duration":0.65008146899999997,"stat_sum":{"num_bytes":25170332,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":131,"num_read_kb":58959,"num_write":1343,"num_write_kb":36929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"80'2124","reported_seq":4745,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:08.512078+0000","last_change":"2024-03-04T08:19:52.261777+0000","last_active":"2024-03-04T08:27:08.512078+0000","last_peered":"2024-03-04T08:27:08.512078+0000","last_clean":"2024-03-04T08:27:08.512078+0000","last_became_active":"2024-03-04T07:33:56.558567+0000","last_became_peered":"2024-03-04T07:33:56.558567+0000","last_unstale":"2024-03-04T08:27:08.512078+0000","last_undegraded":"2024-03-04T08:27:08.512078+0000","last_fullsized":"2024-03-04T08:27:08.512078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2124,"log_dups_size":0,"ondisk_log_size":2124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:59:16.961107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.1904157900000003,"stat_sum":{"num_bytes":16788130,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":113,"num_read_kb":35043,"num_write":1370,"num_write_kb":33439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"80'2140","reported_seq":4759,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:47.140060+0000","last_change":"2024-03-04T08:19:49.427543+0000","last_active":"2024-03-04T08:27:47.140060+0000","last_peered":"2024-03-04T08:27:47.140060+0000","last_clean":"2024-03-04T08:27:47.140060+0000","last_became_active":"2024-03-04T07:33:56.543936+0000","last_became_peered":"2024-03-04T07:33:56.543936+0000","last_unstale":"2024-03-04T08:27:47.140060+0000","last_undegraded":"2024-03-04T08:27:47.140060+0000","last_fullsized":"2024-03-04T08:27:47.140060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2140,"log_dups_size":0,"ondisk_log_size":2140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:29:34.488147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0299552260000002,"stat_sum":{"num_bytes":8390003,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":121,"num_read_kb":16051,"num_write":1328,"num_write_kb":18720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'2145","reported_seq":4756,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:38.054541+0000","last_change":"2024-03-04T08:19:35.446076+0000","last_active":"2024-03-04T08:27:38.054541+0000","last_peered":"2024-03-04T08:27:38.054541+0000","last_clean":"2024-03-04T08:27:38.054541+0000","last_became_active":"2024-03-04T07:33:56.544837+0000","last_became_peered":"2024-03-04T07:33:56.544837+0000","last_unstale":"2024-03-04T08:27:38.054541+0000","last_undegraded":"2024-03-04T08:27:38.054541+0000","last_fullsized":"2024-03-04T08:27:38.054541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2145,"log_dups_size":0,"ondisk_log_size":2145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:01:02.809197+0000","scrub_duration":0,"objects_trimmed":196,"snaptrim_duration":2.7488653730000001,"stat_sum":{"num_bytes":8391933,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":90,"num_read_kb":10258,"num_write":1381,"num_write_kb":16511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"82'2151","reported_seq":4784,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.994588+0000","last_change":"2024-03-04T08:19:36.848889+0000","last_active":"2024-03-04T08:27:36.994588+0000","last_peered":"2024-03-04T08:27:36.994588+0000","last_clean":"2024-03-04T08:27:36.994588+0000","last_became_active":"2024-03-04T07:33:56.557962+0000","last_became_peered":"2024-03-04T07:33:56.557962+0000","last_unstale":"2024-03-04T08:27:36.994588+0000","last_undegraded":"2024-03-04T08:27:36.994588+0000","last_fullsized":"2024-03-04T08:27:36.994588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2151,"log_dups_size":0,"ondisk_log_size":2151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:33:13.908448+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.86055390200000004,"stat_sum":{"num_bytes":12654141,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":117,"num_read_kb":36882,"num_write":1403,"num_write_kb":24709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"82'2229","reported_seq":4853,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.994692+0000","last_change":"2024-03-04T08:19:36.842387+0000","last_active":"2024-03-04T08:27:36.994692+0000","last_peered":"2024-03-04T08:27:36.994692+0000","last_clean":"2024-03-04T08:27:36.994692+0000","last_became_active":"2024-03-04T07:33:56.558719+0000","last_became_peered":"2024-03-04T07:33:56.558719+0000","last_unstale":"2024-03-04T08:27:36.994692+0000","last_undegraded":"2024-03-04T08:27:36.994692+0000","last_fullsized":"2024-03-04T08:27:36.994692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2229,"log_dups_size":0,"ondisk_log_size":2229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:39:07.391195+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.85227727200000003,"stat_sum":{"num_bytes":16780080,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":129,"num_read_kb":21167,"num_write":1413,"num_write_kb":30886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"80'2047","reported_seq":4676,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138840+0000","last_change":"2024-03-04T08:19:32.728091+0000","last_active":"2024-03-04T08:27:37.138840+0000","last_peered":"2024-03-04T08:27:37.138840+0000","last_clean":"2024-03-04T08:27:37.138840+0000","last_became_active":"2024-03-04T07:33:56.528213+0000","last_became_peered":"2024-03-04T07:33:56.528213+0000","last_unstale":"2024-03-04T08:27:37.138840+0000","last_undegraded":"2024-03-04T08:27:37.138840+0000","last_fullsized":"2024-03-04T08:27:37.138840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2047,"log_dups_size":0,"ondisk_log_size":2047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:37:49.272879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019434987000000001,"stat_sum":{"num_bytes":4194351,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":120,"num_read_kb":13790,"num_write":1261,"num_write_kb":11549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"80'2288","reported_seq":4870,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:12.030625+0000","last_change":"2024-03-04T08:19:36.169663+0000","last_active":"2024-03-04T08:28:12.030625+0000","last_peered":"2024-03-04T08:28:12.030625+0000","last_clean":"2024-03-04T08:28:12.030625+0000","last_became_active":"2024-03-04T07:33:56.545467+0000","last_became_peered":"2024-03-04T07:33:56.545467+0000","last_unstale":"2024-03-04T08:28:12.030625+0000","last_undegraded":"2024-03-04T08:28:12.030625+0000","last_fullsized":"2024-03-04T08:28:12.030625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2288,"log_dups_size":0,"ondisk_log_size":2288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:22:01.957883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18267682599999999,"stat_sum":{"num_bytes":12645111,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":119,"num_read_kb":30029,"num_write":1423,"num_write_kb":26760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"80'2119","reported_seq":4737,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.001524+0000","last_change":"2024-03-04T08:19:34.382758+0000","last_active":"2024-03-04T08:27:57.001524+0000","last_peered":"2024-03-04T08:27:57.001524+0000","last_clean":"2024-03-04T08:27:57.001524+0000","last_became_active":"2024-03-04T07:33:56.548783+0000","last_became_peered":"2024-03-04T07:33:56.548783+0000","last_unstale":"2024-03-04T08:27:57.001524+0000","last_undegraded":"2024-03-04T08:27:57.001524+0000","last_fullsized":"2024-03-04T08:27:57.001524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2119,"log_dups_size":0,"ondisk_log_size":2119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:16:42.338152+0000","scrub_duration":0,"objects_trimmed":116,"snaptrim_duration":1.671257062,"stat_sum":{"num_bytes":8450701,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":102,"num_read_kb":25514,"num_write":1341,"num_write_kb":30030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"82'2119","reported_seq":4723,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.141072+0000","last_change":"2024-03-04T08:19:49.225914+0000","last_active":"2024-03-04T08:27:57.141072+0000","last_peered":"2024-03-04T08:27:57.141072+0000","last_clean":"2024-03-04T08:27:57.141072+0000","last_became_active":"2024-03-04T07:33:56.539403+0000","last_became_peered":"2024-03-04T07:33:56.539403+0000","last_unstale":"2024-03-04T08:27:57.141072+0000","last_undegraded":"2024-03-04T08:27:57.141072+0000","last_fullsized":"2024-03-04T08:27:57.141072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2119,"log_dups_size":0,"ondisk_log_size":2119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:43:54.215133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.827802057,"stat_sum":{"num_bytes":4204891,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":101,"num_read_kb":9693,"num_write":1325,"num_write_kb":13878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"80'2234","reported_seq":4838,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:22.131357+0000","last_change":"2024-03-04T08:19:32.717851+0000","last_active":"2024-03-04T08:26:22.131357+0000","last_peered":"2024-03-04T08:26:22.131357+0000","last_clean":"2024-03-04T08:26:22.131357+0000","last_became_active":"2024-03-04T07:34:37.644727+0000","last_became_peered":"2024-03-04T07:34:37.644727+0000","last_unstale":"2024-03-04T08:26:22.131357+0000","last_undegraded":"2024-03-04T08:26:22.131357+0000","last_fullsized":"2024-03-04T08:26:22.131357+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:01:11.097920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010246078,"stat_sum":{"num_bytes":12594611,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":123,"num_read_kb":29265,"num_write":1418,"num_write_kb":23716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"82'2041","reported_seq":4709,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:58.516950+0000","last_change":"2024-03-04T08:19:52.301927+0000","last_active":"2024-03-04T08:27:58.516950+0000","last_peered":"2024-03-04T08:27:58.516950+0000","last_clean":"2024-03-04T08:27:58.516950+0000","last_became_active":"2024-03-04T07:33:56.545270+0000","last_became_peered":"2024-03-04T07:33:56.545270+0000","last_unstale":"2024-03-04T08:27:58.516950+0000","last_undegraded":"2024-03-04T08:27:58.516950+0000","last_fullsized":"2024-03-04T08:27:58.516950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2041,"log_dups_size":0,"ondisk_log_size":2041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:29:34.456689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.2311050080000001,"stat_sum":{"num_bytes":12602373,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":129,"num_read_kb":39775,"num_write":1326,"num_write_kb":24556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'2103","reported_seq":4737,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.001274+0000","last_change":"2024-03-04T08:19:36.847630+0000","last_active":"2024-03-04T08:28:22.001274+0000","last_peered":"2024-03-04T08:28:22.001274+0000","last_clean":"2024-03-04T08:28:22.001274+0000","last_became_active":"2024-03-04T07:33:56.554942+0000","last_became_peered":"2024-03-04T07:33:56.554942+0000","last_unstale":"2024-03-04T08:28:22.001274+0000","last_undegraded":"2024-03-04T08:28:22.001274+0000","last_fullsized":"2024-03-04T08:28:22.001274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2103,"log_dups_size":0,"ondisk_log_size":2103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:11:44.174822+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.85911008,"stat_sum":{"num_bytes":4342145,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":112,"num_read_kb":14015,"num_write":1342,"num_write_kb":11768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"80'2416","reported_seq":4914,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.518032+0000","last_change":"2024-03-04T08:19:52.525743+0000","last_active":"2024-03-04T08:28:08.518032+0000","last_peered":"2024-03-04T08:28:08.518032+0000","last_clean":"2024-03-04T08:28:08.518032+0000","last_became_active":"2024-03-04T07:33:56.548790+0000","last_became_peered":"2024-03-04T07:33:56.548790+0000","last_unstale":"2024-03-04T08:28:08.518032+0000","last_undegraded":"2024-03-04T08:28:08.518032+0000","last_fullsized":"2024-03-04T08:28:08.518032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2416,"log_dups_size":0,"ondisk_log_size":2416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:53:43.350229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.454029974,"stat_sum":{"num_bytes":8428929,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":109,"num_read_kb":26630,"num_write":1382,"num_write_kb":16670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'2182","reported_seq":4787,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.002961+0000","last_change":"2024-03-04T08:19:34.384174+0000","last_active":"2024-03-04T08:28:07.002961+0000","last_peered":"2024-03-04T08:28:07.002961+0000","last_clean":"2024-03-04T08:28:07.002961+0000","last_became_active":"2024-03-04T07:33:56.553012+0000","last_became_peered":"2024-03-04T07:33:56.553012+0000","last_unstale":"2024-03-04T08:28:07.002961+0000","last_undegraded":"2024-03-04T08:28:07.002961+0000","last_fullsized":"2024-03-04T08:28:07.002961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2182,"log_dups_size":0,"ondisk_log_size":2182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:17:05.626254+0000","scrub_duration":0,"objects_trimmed":141,"snaptrim_duration":1.673071669,"stat_sum":{"num_bytes":4199673,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":99,"num_read_kb":13996,"num_write":1374,"num_write_kb":14419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"82'2031","reported_seq":4665,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057745+0000","last_change":"2024-03-04T08:19:35.448280+0000","last_active":"2024-03-04T08:28:08.057745+0000","last_peered":"2024-03-04T08:28:08.057745+0000","last_clean":"2024-03-04T08:28:08.057745+0000","last_became_active":"2024-03-04T07:33:56.550616+0000","last_became_peered":"2024-03-04T07:33:56.550616+0000","last_unstale":"2024-03-04T08:28:08.057745+0000","last_undegraded":"2024-03-04T08:28:08.057745+0000","last_fullsized":"2024-03-04T08:28:08.057745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2031,"log_dups_size":0,"ondisk_log_size":2031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:49:50.704680+0000","scrub_duration":0,"objects_trimmed":153,"snaptrim_duration":2.7491079520000001,"stat_sum":{"num_bytes":20984835,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":87,"num_read_kb":29030,"num_write":1330,"num_write_kb":28787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"80'2058","reported_seq":4675,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.518017+0000","last_change":"2024-03-04T08:19:52.538416+0000","last_active":"2024-03-04T08:28:08.518017+0000","last_peered":"2024-03-04T08:28:08.518017+0000","last_clean":"2024-03-04T08:28:08.518017+0000","last_became_active":"2024-03-04T07:33:56.543632+0000","last_became_peered":"2024-03-04T07:33:56.543632+0000","last_unstale":"2024-03-04T08:28:08.518017+0000","last_undegraded":"2024-03-04T08:28:08.518017+0000","last_fullsized":"2024-03-04T08:28:08.518017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2058,"log_dups_size":0,"ondisk_log_size":2058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:14:48.206953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.4338148220000004,"stat_sum":{"num_bytes":4197645,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":105,"num_read_kb":17328,"num_write":1252,"num_write_kb":17889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"80'2259","reported_seq":4843,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:17.143069+0000","last_change":"2024-03-04T08:19:49.701932+0000","last_active":"2024-03-04T08:28:17.143069+0000","last_peered":"2024-03-04T08:28:17.143069+0000","last_clean":"2024-03-04T08:28:17.143069+0000","last_became_active":"2024-03-04T07:33:56.546652+0000","last_became_peered":"2024-03-04T07:33:56.546652+0000","last_unstale":"2024-03-04T08:28:17.143069+0000","last_undegraded":"2024-03-04T08:28:17.143069+0000","last_fullsized":"2024-03-04T08:28:17.143069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2259,"log_dups_size":0,"ondisk_log_size":2259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:36:41.107790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.303629049,"stat_sum":{"num_bytes":8400817,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":112,"num_read_kb":14343,"num_write":1411,"num_write_kb":25423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"80'2098","reported_seq":4725,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:06.999085+0000","last_change":"2024-03-04T08:19:36.849222+0000","last_active":"2024-03-04T08:28:06.999085+0000","last_peered":"2024-03-04T08:28:06.999085+0000","last_clean":"2024-03-04T08:28:06.999085+0000","last_became_active":"2024-03-04T07:33:56.552189+0000","last_became_peered":"2024-03-04T07:33:56.552189+0000","last_unstale":"2024-03-04T08:28:06.999085+0000","last_undegraded":"2024-03-04T08:28:06.999085+0000","last_fullsized":"2024-03-04T08:28:06.999085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2098,"log_dups_size":0,"ondisk_log_size":2098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:38:31.691317+0000","scrub_duration":0,"objects_trimmed":46,"snaptrim_duration":0.85857668899999995,"stat_sum":{"num_bytes":8407168,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":95,"num_read_kb":14091,"num_write":1358,"num_write_kb":17493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"80'2246","reported_seq":4842,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.141974+0000","last_change":"2024-03-04T08:19:32.724206+0000","last_active":"2024-03-04T08:28:07.141974+0000","last_peered":"2024-03-04T08:28:07.141974+0000","last_clean":"2024-03-04T08:28:07.141974+0000","last_became_active":"2024-03-04T07:33:56.549351+0000","last_became_peered":"2024-03-04T07:33:56.549351+0000","last_unstale":"2024-03-04T08:28:07.141974+0000","last_undegraded":"2024-03-04T08:28:07.141974+0000","last_fullsized":"2024-03-04T08:28:07.141974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2246,"log_dups_size":0,"ondisk_log_size":2246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:25.101476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.016086315,"stat_sum":{"num_bytes":20978563,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":109,"num_read_kb":27431,"num_write":1407,"num_write_kb":36516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"80'2497","reported_seq":4993,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.002897+0000","last_change":"2024-03-04T08:19:34.447685+0000","last_active":"2024-03-04T08:28:07.002897+0000","last_peered":"2024-03-04T08:28:07.002897+0000","last_clean":"2024-03-04T08:28:07.002897+0000","last_became_active":"2024-03-04T07:33:56.532251+0000","last_became_peered":"2024-03-04T07:33:56.532251+0000","last_unstale":"2024-03-04T08:28:07.002897+0000","last_undegraded":"2024-03-04T08:28:07.002897+0000","last_fullsized":"2024-03-04T08:28:07.002897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:03:48.692559+0000","scrub_duration":0,"objects_trimmed":183,"snaptrim_duration":1.7333053779999998,"stat_sum":{"num_bytes":12598132,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":114,"num_read_kb":36166,"num_write":1454,"num_write_kb":20545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"82'2217","reported_seq":4804,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.119475+0000","last_change":"2024-03-04T08:19:33.346715+0000","last_active":"2024-03-04T08:26:27.119475+0000","last_peered":"2024-03-04T08:26:27.119475+0000","last_clean":"2024-03-04T08:26:27.119475+0000","last_became_active":"2024-03-04T07:33:56.542937+0000","last_became_peered":"2024-03-04T07:33:56.542937+0000","last_unstale":"2024-03-04T08:26:27.119475+0000","last_undegraded":"2024-03-04T08:26:27.119475+0000","last_fullsized":"2024-03-04T08:26:27.119475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2217,"log_dups_size":0,"ondisk_log_size":2217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:17:50.886040+0000","scrub_duration":0,"objects_trimmed":58,"snaptrim_duration":0.65684712700000003,"stat_sum":{"num_bytes":12587619,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":118,"num_read_kb":25719,"num_write":1362,"num_write_kb":26221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"85'680","reported_seq":983,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:58.522965+0000","last_change":"2024-03-04T07:33:55.566130+0000","last_active":"2024-03-04T08:28:58.522965+0000","last_peered":"2024-03-04T08:28:58.522965+0000","last_clean":"2024-03-04T08:28:58.522965+0000","last_became_active":"2024-03-04T07:33:55.565894+0000","last_became_peered":"2024-03-04T07:33:55.565894+0000","last_unstale":"2024-03-04T08:28:58.522965+0000","last_undegraded":"2024-03-04T08:28:58.522965+0000","last_fullsized":"2024-03-04T08:28:58.522965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:01:36.530949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20973469,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":251,"num_read_kb":140597,"num_write":805,"num_write_kb":83805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"82'2160","reported_seq":4871,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.132021+0000","last_change":"2024-03-04T08:19:49.377440+0000","last_active":"2024-03-04T08:26:27.132021+0000","last_peered":"2024-03-04T08:26:27.132021+0000","last_clean":"2024-03-04T08:26:27.132021+0000","last_became_active":"2024-03-04T07:33:56.539738+0000","last_became_peered":"2024-03-04T07:33:56.539738+0000","last_unstale":"2024-03-04T08:26:27.132021+0000","last_undegraded":"2024-03-04T08:26:27.132021+0000","last_fullsized":"2024-03-04T08:26:27.132021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2160,"log_dups_size":0,"ondisk_log_size":2160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:44:38.647134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.9789417120000001,"stat_sum":{"num_bytes":8388926,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":220,"num_read_kb":14085,"num_write":1362,"num_write_kb":17052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1189","reported_seq":1504,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.988431+0000","last_change":"2024-03-04T07:33:55.554956+0000","last_active":"2024-03-04T08:26:26.988431+0000","last_peered":"2024-03-04T08:26:26.988431+0000","last_clean":"2024-03-04T08:26:26.988431+0000","last_became_active":"2024-03-04T07:33:55.554151+0000","last_became_peered":"2024-03-04T07:33:55.554151+0000","last_unstale":"2024-03-04T08:26:26.988431+0000","last_undegraded":"2024-03-04T08:26:26.988431+0000","last_fullsized":"2024-03-04T08:26:26.988431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1189,"log_dups_size":0,"ondisk_log_size":1189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:06:19.621510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":322,"num_read_kb":118780,"num_write":1380,"num_write_kb":72133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"82'487","reported_seq":770,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:13.512566+0000","last_change":"2024-03-04T07:33:55.542200+0000","last_active":"2024-03-04T08:27:13.512566+0000","last_peered":"2024-03-04T08:27:13.512566+0000","last_clean":"2024-03-04T08:27:13.512566+0000","last_became_active":"2024-03-04T07:33:55.541902+0000","last_became_peered":"2024-03-04T07:33:55.541902+0000","last_unstale":"2024-03-04T08:27:13.512566+0000","last_undegraded":"2024-03-04T08:27:13.512566+0000","last_fullsized":"2024-03-04T08:27:13.512566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:55:50.251288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":258,"num_read_kb":115971,"num_write":610,"num_write_kb":58869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"80'2411","reported_seq":4973,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:08.512031+0000","last_change":"2024-03-04T08:19:52.693084+0000","last_active":"2024-03-04T08:27:08.512031+0000","last_peered":"2024-03-04T08:27:08.512031+0000","last_clean":"2024-03-04T08:27:08.512031+0000","last_became_active":"2024-03-04T07:33:56.546087+0000","last_became_peered":"2024-03-04T07:33:56.546087+0000","last_unstale":"2024-03-04T08:27:08.512031+0000","last_undegraded":"2024-03-04T08:27:08.512031+0000","last_fullsized":"2024-03-04T08:27:08.512031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2411,"log_dups_size":0,"ondisk_log_size":2411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:42:05.213414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.6221017409999998,"stat_sum":{"num_bytes":4200161,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":109,"num_read_kb":16756,"num_write":1524,"num_write_kb":21226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1610","reported_seq":1897,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.990193+0000","last_change":"2024-03-04T07:33:55.568508+0000","last_active":"2024-03-04T08:27:06.990193+0000","last_peered":"2024-03-04T08:27:06.990193+0000","last_clean":"2024-03-04T08:27:06.990193+0000","last_became_active":"2024-03-04T07:33:55.566073+0000","last_became_peered":"2024-03-04T07:33:55.566073+0000","last_unstale":"2024-03-04T08:27:06.990193+0000","last_undegraded":"2024-03-04T08:27:06.990193+0000","last_fullsized":"2024-03-04T08:27:06.990193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1610,"log_dups_size":0,"ondisk_log_size":1610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:34:17.471471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":228,"num_read_kb":146204,"num_write":1735,"num_write_kb":76344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'2138","reported_seq":4771,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:18.052787+0000","last_change":"2024-03-04T08:19:35.448277+0000","last_active":"2024-03-04T08:27:18.052787+0000","last_peered":"2024-03-04T08:27:18.052787+0000","last_clean":"2024-03-04T08:27:18.052787+0000","last_became_active":"2024-03-04T07:33:56.552533+0000","last_became_peered":"2024-03-04T07:33:56.552533+0000","last_unstale":"2024-03-04T08:27:18.052787+0000","last_undegraded":"2024-03-04T08:27:18.052787+0000","last_fullsized":"2024-03-04T08:27:18.052787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2138,"log_dups_size":0,"ondisk_log_size":2138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:15:02.557608+0000","scrub_duration":0,"objects_trimmed":178,"snaptrim_duration":2.7512426310000002,"stat_sum":{"num_bytes":25170167,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":124,"num_read_kb":49299,"num_write":1360,"num_write_kb":40715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'786","reported_seq":1224,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:29:32.015555+0000","last_change":"2024-03-04T07:33:55.559395+0000","last_active":"2024-03-04T08:29:32.015555+0000","last_peered":"2024-03-04T08:29:32.015555+0000","last_clean":"2024-03-04T08:29:32.015555+0000","last_became_active":"2024-03-04T07:33:55.558593+0000","last_became_peered":"2024-03-04T07:33:55.558593+0000","last_unstale":"2024-03-04T08:29:32.015555+0000","last_undegraded":"2024-03-04T08:29:32.015555+0000","last_fullsized":"2024-03-04T08:29:32.015555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:00:20.205445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17355948,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":385,"num_read_kb":100065,"num_write":866,"num_write_kb":59600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"82'2101","reported_seq":4721,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.990264+0000","last_change":"2024-03-04T08:19:36.839345+0000","last_active":"2024-03-04T08:27:06.990264+0000","last_peered":"2024-03-04T08:27:06.990264+0000","last_clean":"2024-03-04T08:27:06.990264+0000","last_became_active":"2024-03-04T07:33:56.552952+0000","last_became_peered":"2024-03-04T07:33:56.552952+0000","last_unstale":"2024-03-04T08:27:06.990264+0000","last_undegraded":"2024-03-04T08:27:06.990264+0000","last_fullsized":"2024-03-04T08:27:06.990264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2101,"log_dups_size":0,"ondisk_log_size":2101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:53:29.724551+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.84949035500000003,"stat_sum":{"num_bytes":8307065,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":95,"num_read_kb":25119,"num_write":1361,"num_write_kb":16142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"82'705","reported_seq":1001,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:06.990287+0000","last_change":"2024-03-04T07:33:55.568220+0000","last_active":"2024-03-04T08:27:06.990287+0000","last_peered":"2024-03-04T08:27:06.990287+0000","last_clean":"2024-03-04T08:27:06.990287+0000","last_became_active":"2024-03-04T07:33:55.567569+0000","last_became_peered":"2024-03-04T07:33:55.567569+0000","last_unstale":"2024-03-04T08:27:06.990287+0000","last_undegraded":"2024-03-04T08:27:06.990287+0000","last_fullsized":"2024-03-04T08:27:06.990287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":705,"log_dups_size":0,"ondisk_log_size":705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:40:49.131861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":256,"num_read_kb":181413,"num_write":818,"num_write_kb":87860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"82'2131","reported_seq":4730,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:07.136092+0000","last_change":"2024-03-04T08:19:49.095825+0000","last_active":"2024-03-04T08:27:07.136092+0000","last_peered":"2024-03-04T08:27:07.136092+0000","last_clean":"2024-03-04T08:27:07.136092+0000","last_became_active":"2024-03-04T07:33:56.559456+0000","last_became_peered":"2024-03-04T07:33:56.559456+0000","last_unstale":"2024-03-04T08:27:07.136092+0000","last_undegraded":"2024-03-04T08:27:07.136092+0000","last_fullsized":"2024-03-04T08:27:07.136092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2131,"log_dups_size":0,"ondisk_log_size":2131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:50:29.399119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.6973493710000001,"stat_sum":{"num_bytes":8393873,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":101,"num_read_kb":25726,"num_write":1348,"num_write_kb":16222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'1400","reported_seq":1705,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:08.051648+0000","last_change":"2024-03-04T07:33:55.550625+0000","last_active":"2024-03-04T08:27:08.051648+0000","last_peered":"2024-03-04T08:27:08.051648+0000","last_clean":"2024-03-04T08:27:08.051648+0000","last_became_active":"2024-03-04T07:33:55.550327+0000","last_became_peered":"2024-03-04T07:33:55.550327+0000","last_unstale":"2024-03-04T08:27:08.051648+0000","last_undegraded":"2024-03-04T08:27:08.051648+0000","last_fullsized":"2024-03-04T08:27:08.051648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:44:20.282585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972192,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":274,"num_read_kb":137054,"num_write":1577,"num_write_kb":60435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"80'2174","reported_seq":4785,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:47.130659+0000","last_change":"2024-03-04T08:19:33.352033+0000","last_active":"2024-03-04T08:27:47.130659+0000","last_peered":"2024-03-04T08:27:47.130659+0000","last_clean":"2024-03-04T08:27:47.130659+0000","last_became_active":"2024-03-04T07:33:56.536870+0000","last_became_peered":"2024-03-04T07:33:56.536870+0000","last_unstale":"2024-03-04T08:27:47.130659+0000","last_undegraded":"2024-03-04T08:27:47.130659+0000","last_fullsized":"2024-03-04T08:27:47.130659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2174,"log_dups_size":0,"ondisk_log_size":2174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:56:44.867605+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.66166569399999997,"stat_sum":{"num_bytes":12846648,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":118,"num_read_kb":29225,"num_write":1393,"num_write_kb":24887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"82'1283","reported_seq":1627,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:52.000760+0000","last_change":"2024-03-04T07:33:55.538915+0000","last_active":"2024-03-04T08:27:52.000760+0000","last_peered":"2024-03-04T08:27:52.000760+0000","last_clean":"2024-03-04T08:27:52.000760+0000","last_became_active":"2024-03-04T07:33:55.536263+0000","last_became_peered":"2024-03-04T07:33:55.536263+0000","last_unstale":"2024-03-04T08:27:52.000760+0000","last_undegraded":"2024-03-04T08:27:52.000760+0000","last_fullsized":"2024-03-04T08:27:52.000760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1283,"log_dups_size":0,"ondisk_log_size":1283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:43:09.612704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360800,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":354,"num_read_kb":206595,"num_write":1442,"num_write_kb":96944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"80'2051","reported_seq":4680,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:07.020807+0000","last_change":"2024-03-04T08:19:36.168875+0000","last_active":"2024-03-04T08:27:07.020807+0000","last_peered":"2024-03-04T08:27:07.020807+0000","last_clean":"2024-03-04T08:27:07.020807+0000","last_became_active":"2024-03-04T07:33:56.550767+0000","last_became_peered":"2024-03-04T07:33:56.550767+0000","last_unstale":"2024-03-04T08:27:07.020807+0000","last_undegraded":"2024-03-04T08:27:07.020807+0000","last_fullsized":"2024-03-04T08:27:07.020807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2051,"log_dups_size":0,"ondisk_log_size":2051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:41:38.658644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.187228595,"stat_sum":{"num_bytes":12597061,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":105,"num_read_kb":35778,"num_write":1291,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"82'483","reported_seq":772,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138903+0000","last_change":"2024-03-04T07:33:55.560382+0000","last_active":"2024-03-04T08:27:37.138903+0000","last_peered":"2024-03-04T08:27:37.138903+0000","last_clean":"2024-03-04T08:27:37.138903+0000","last_became_active":"2024-03-04T07:33:55.560078+0000","last_became_peered":"2024-03-04T07:33:55.560078+0000","last_unstale":"2024-03-04T08:27:37.138903+0000","last_undegraded":"2024-03-04T08:27:37.138903+0000","last_fullsized":"2024-03-04T08:27:37.138903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:41:10.894953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194338,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":271,"num_read_kb":107276,"num_write":561,"num_write_kb":58233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"80'2019","reported_seq":4649,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138868+0000","last_change":"2024-03-04T08:19:49.480210+0000","last_active":"2024-03-04T08:27:37.138868+0000","last_peered":"2024-03-04T08:27:37.138868+0000","last_clean":"2024-03-04T08:27:37.138868+0000","last_became_active":"2024-03-04T07:33:56.535960+0000","last_became_peered":"2024-03-04T07:33:56.535960+0000","last_unstale":"2024-03-04T08:27:37.138868+0000","last_undegraded":"2024-03-04T08:27:37.138868+0000","last_fullsized":"2024-03-04T08:27:37.138868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2019,"log_dups_size":0,"ondisk_log_size":2019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:17:01.637891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0525318239999999,"stat_sum":{"num_bytes":16780560,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":39118,"num_write":1277,"num_write_kb":32917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"82'440","reported_seq":737,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.994642+0000","last_change":"2024-03-04T07:33:55.567682+0000","last_active":"2024-03-04T08:27:36.994642+0000","last_peered":"2024-03-04T08:27:36.994642+0000","last_clean":"2024-03-04T08:27:36.994642+0000","last_became_active":"2024-03-04T07:33:55.566915+0000","last_became_peered":"2024-03-04T07:33:55.566915+0000","last_unstale":"2024-03-04T08:27:36.994642+0000","last_undegraded":"2024-03-04T08:27:36.994642+0000","last_fullsized":"2024-03-04T08:27:36.994642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:33:47.010346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":276,"num_read_kb":100025,"num_write":608,"num_write_kb":47104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'2205","reported_seq":4818,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.025397+0000","last_change":"2024-03-04T08:19:36.162956+0000","last_active":"2024-03-04T08:27:37.025397+0000","last_peered":"2024-03-04T08:27:37.025397+0000","last_clean":"2024-03-04T08:27:37.025397+0000","last_became_active":"2024-03-04T07:33:56.547459+0000","last_became_peered":"2024-03-04T07:33:56.547459+0000","last_unstale":"2024-03-04T08:27:37.025397+0000","last_undegraded":"2024-03-04T08:27:37.025397+0000","last_fullsized":"2024-03-04T08:27:37.025397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2205,"log_dups_size":0,"ondisk_log_size":2205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:48:06.725806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.181152964,"stat_sum":{"num_bytes":4213693,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":118,"num_read_kb":1396,"num_write":1423,"num_write_kb":14566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'723","reported_seq":1018,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.998645+0000","last_change":"2024-03-04T07:33:55.557041+0000","last_active":"2024-03-04T08:27:36.998645+0000","last_peered":"2024-03-04T08:27:36.998645+0000","last_clean":"2024-03-04T08:27:36.998645+0000","last_became_active":"2024-03-04T07:33:55.556416+0000","last_became_peered":"2024-03-04T07:33:55.556416+0000","last_unstale":"2024-03-04T08:27:36.998645+0000","last_undegraded":"2024-03-04T08:27:36.998645+0000","last_fullsized":"2024-03-04T08:27:36.998645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:13:31.472235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8390846,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":242,"num_read_kb":79450,"num_write":832,"num_write_kb":51201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"82'2099","reported_seq":4736,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:36.998604+0000","last_change":"2024-03-04T08:19:34.394444+0000","last_active":"2024-03-04T08:27:36.998604+0000","last_peered":"2024-03-04T08:27:36.998604+0000","last_clean":"2024-03-04T08:27:36.998604+0000","last_became_active":"2024-03-04T07:33:56.531565+0000","last_became_peered":"2024-03-04T07:33:56.531565+0000","last_unstale":"2024-03-04T08:27:36.998604+0000","last_undegraded":"2024-03-04T08:27:36.998604+0000","last_fullsized":"2024-03-04T08:27:36.998604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2099,"log_dups_size":0,"ondisk_log_size":2099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:02:41.845508+0000","scrub_duration":0,"objects_trimmed":121,"snaptrim_duration":1.6809537379999999,"stat_sum":{"num_bytes":16783369,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":112,"num_read_kb":39998,"num_write":1341,"num_write_kb":27959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"85'797","reported_seq":1153,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:37.138789+0000","last_change":"2024-03-04T07:33:55.538469+0000","last_active":"2024-03-04T08:27:37.138789+0000","last_peered":"2024-03-04T08:27:37.138789+0000","last_clean":"2024-03-04T08:27:37.138789+0000","last_became_active":"2024-03-04T07:33:55.538165+0000","last_became_peered":"2024-03-04T07:33:55.538165+0000","last_unstale":"2024-03-04T08:27:37.138789+0000","last_undegraded":"2024-03-04T08:27:37.138789+0000","last_fullsized":"2024-03-04T08:27:37.138789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:00:01.263768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165936,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":419,"num_read_kb":136829,"num_write":955,"num_write_kb":63584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"80'2116","reported_seq":4728,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:18.058628+0000","last_change":"2024-03-04T08:19:35.444496+0000","last_active":"2024-03-04T08:28:18.058628+0000","last_peered":"2024-03-04T08:28:18.058628+0000","last_clean":"2024-03-04T08:28:18.058628+0000","last_became_active":"2024-03-04T07:34:36.770794+0000","last_became_peered":"2024-03-04T07:34:36.770794+0000","last_unstale":"2024-03-04T08:28:18.058628+0000","last_undegraded":"2024-03-04T08:28:18.058628+0000","last_fullsized":"2024-03-04T08:28:18.058628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2116,"log_dups_size":0,"ondisk_log_size":2116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:08:17.614159+0000","scrub_duration":0,"objects_trimmed":157,"snaptrim_duration":2.747632485,"stat_sum":{"num_bytes":16781418,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":100,"num_read_kb":34292,"num_write":1333,"num_write_kb":33482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.f","version":"84'741","reported_seq":1039,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.133542+0000","last_change":"2024-03-04T07:33:55.544356+0000","last_active":"2024-03-04T08:28:07.133542+0000","last_peered":"2024-03-04T08:28:07.133542+0000","last_clean":"2024-03-04T08:28:07.133542+0000","last_became_active":"2024-03-04T07:33:55.541295+0000","last_became_peered":"2024-03-04T07:33:55.541295+0000","last_unstale":"2024-03-04T08:28:07.133542+0000","last_undegraded":"2024-03-04T08:28:07.133542+0000","last_fullsized":"2024-03-04T08:28:07.133542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:22.953982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":266,"num_read_kb":99836,"num_write":937,"num_write_kb":59254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":43,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"80'2189","reported_seq":4759,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.028167+0000","last_change":"2024-03-04T08:19:36.163661+0000","last_active":"2024-03-04T08:27:57.028167+0000","last_peered":"2024-03-04T08:27:57.028167+0000","last_clean":"2024-03-04T08:27:57.028167+0000","last_became_active":"2024-03-04T07:33:56.548113+0000","last_became_peered":"2024-03-04T07:33:56.548113+0000","last_unstale":"2024-03-04T08:27:57.028167+0000","last_undegraded":"2024-03-04T08:27:57.028167+0000","last_fullsized":"2024-03-04T08:27:57.028167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2189,"log_dups_size":0,"ondisk_log_size":2189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:13:34.466627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.18041650100000001,"stat_sum":{"num_bytes":12585055,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":97,"num_read_kb":11871,"num_write":1338,"num_write_kb":24818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'700","reported_seq":976,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.141049+0000","last_change":"2024-03-04T07:33:55.538871+0000","last_active":"2024-03-04T08:27:57.141049+0000","last_peered":"2024-03-04T08:27:57.141049+0000","last_clean":"2024-03-04T08:27:57.141049+0000","last_became_active":"2024-03-04T07:33:55.537221+0000","last_became_peered":"2024-03-04T07:33:55.537221+0000","last_unstale":"2024-03-04T08:27:57.141049+0000","last_undegraded":"2024-03-04T08:27:57.141049+0000","last_fullsized":"2024-03-04T08:27:57.141049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:02:34.533330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":222,"num_read_kb":137755,"num_write":800,"num_write_kb":83574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"82'2101","reported_seq":4719,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:12.030686+0000","last_change":"2024-03-04T08:19:36.165896+0000","last_active":"2024-03-04T08:28:12.030686+0000","last_peered":"2024-03-04T08:28:12.030686+0000","last_clean":"2024-03-04T08:28:12.030686+0000","last_became_active":"2024-03-04T07:33:56.543354+0000","last_became_peered":"2024-03-04T07:33:56.543354+0000","last_unstale":"2024-03-04T08:28:12.030686+0000","last_undegraded":"2024-03-04T08:28:12.030686+0000","last_fullsized":"2024-03-04T08:28:12.030686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2101,"log_dups_size":0,"ondisk_log_size":2101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:52:28.820568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17976278000000001,"stat_sum":{"num_bytes":25168099,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":104,"num_read_kb":70965,"num_write":1341,"num_write_kb":36170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'895","reported_seq":1177,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:58.056491+0000","last_change":"2024-03-04T07:33:55.552856+0000","last_active":"2024-03-04T08:27:58.056491+0000","last_peered":"2024-03-04T08:27:58.056491+0000","last_clean":"2024-03-04T08:27:58.056491+0000","last_became_active":"2024-03-04T07:33:55.552472+0000","last_became_peered":"2024-03-04T07:33:55.552472+0000","last_unstale":"2024-03-04T08:27:58.056491+0000","last_undegraded":"2024-03-04T08:27:58.056491+0000","last_fullsized":"2024-03-04T08:27:58.056491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:36:50.409567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364919,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":192,"num_read_kb":160923,"num_write":993,"num_write_kb":79673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"82'2097","reported_seq":4734,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:56.997663+0000","last_change":"2024-03-04T08:19:36.847924+0000","last_active":"2024-03-04T08:27:56.997663+0000","last_peered":"2024-03-04T08:27:56.997663+0000","last_clean":"2024-03-04T08:27:56.997663+0000","last_became_active":"2024-03-04T07:34:36.774424+0000","last_became_peered":"2024-03-04T07:34:36.774424+0000","last_unstale":"2024-03-04T08:27:56.997663+0000","last_undegraded":"2024-03-04T08:27:56.997663+0000","last_fullsized":"2024-03-04T08:27:56.997663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2097,"log_dups_size":0,"ondisk_log_size":2097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:08:56.055772+0000","scrub_duration":0,"objects_trimmed":52,"snaptrim_duration":0.85871435699999998,"stat_sum":{"num_bytes":8390169,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":104,"num_read_kb":28439,"num_write":1349,"num_write_kb":17437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"82'815","reported_seq":1145,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.141018+0000","last_change":"2024-03-04T07:33:55.560332+0000","last_active":"2024-03-04T08:27:57.141018+0000","last_peered":"2024-03-04T08:27:57.141018+0000","last_clean":"2024-03-04T08:27:57.141018+0000","last_became_active":"2024-03-04T07:33:55.560086+0000","last_became_peered":"2024-03-04T07:33:55.560086+0000","last_unstale":"2024-03-04T08:27:57.141018+0000","last_undegraded":"2024-03-04T08:27:57.141018+0000","last_fullsized":"2024-03-04T08:27:57.141018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:25:46.280657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":353,"num_read_kb":116715,"num_write":1050,"num_write_kb":64078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"82'2110","reported_seq":4723,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.028131+0000","last_change":"2024-03-04T08:19:36.163052+0000","last_active":"2024-03-04T08:27:57.028131+0000","last_peered":"2024-03-04T08:27:57.028131+0000","last_clean":"2024-03-04T08:27:57.028131+0000","last_became_active":"2024-03-04T07:33:56.559589+0000","last_became_peered":"2024-03-04T07:33:56.559589+0000","last_unstale":"2024-03-04T08:27:57.028131+0000","last_undegraded":"2024-03-04T08:27:57.028131+0000","last_fullsized":"2024-03-04T08:27:57.028131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2110,"log_dups_size":0,"ondisk_log_size":2110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:48:29.367330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.17959140500000001,"stat_sum":{"num_bytes":4211072,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":108,"num_read_kb":1329,"num_write":1312,"num_write_kb":17303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":326,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:02.028901+0000","last_change":"2024-03-04T07:33:43.559630+0000","last_active":"2024-03-04T08:28:02.028901+0000","last_peered":"2024-03-04T08:28:02.028901+0000","last_clean":"2024-03-04T08:28:02.028901+0000","last_became_active":"2024-03-04T07:33:43.557489+0000","last_became_peered":"2024-03-04T07:33:43.557489+0000","last_unstale":"2024-03-04T08:28:02.028901+0000","last_undegraded":"2024-03-04T08:28:02.028901+0000","last_fullsized":"2024-03-04T08:28:02.028901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:42.183823+0000","last_clean_scrub_stamp":"2024-03-04T07:33:42.183823+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:55:35.762857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"82'444","reported_seq":744,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:06.999157+0000","last_change":"2024-03-04T07:33:55.548616+0000","last_active":"2024-03-04T08:28:06.999157+0000","last_peered":"2024-03-04T08:28:06.999157+0000","last_clean":"2024-03-04T08:28:06.999157+0000","last_became_active":"2024-03-04T07:33:55.547322+0000","last_became_peered":"2024-03-04T07:33:55.547322+0000","last_unstale":"2024-03-04T08:28:06.999157+0000","last_undegraded":"2024-03-04T08:28:06.999157+0000","last_fullsized":"2024-03-04T08:28:06.999157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:20:16.717483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165858,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":253,"num_read_kb":128383,"num_write":599,"num_write_kb":63285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"82'2192","reported_seq":4778,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.517999+0000","last_change":"2024-03-04T08:19:52.492136+0000","last_active":"2024-03-04T08:28:08.517999+0000","last_peered":"2024-03-04T08:28:08.517999+0000","last_clean":"2024-03-04T08:28:08.517999+0000","last_became_active":"2024-03-04T07:33:56.559581+0000","last_became_peered":"2024-03-04T07:33:56.559581+0000","last_unstale":"2024-03-04T08:28:08.517999+0000","last_undegraded":"2024-03-04T08:28:08.517999+0000","last_fullsized":"2024-03-04T08:28:08.517999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2192,"log_dups_size":0,"ondisk_log_size":2192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:24:09.497555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.4208662280000004,"stat_sum":{"num_bytes":12628568,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":97,"num_read_kb":23350,"num_write":1376,"num_write_kb":26319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"82'785","reported_seq":1076,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:12.030530+0000","last_change":"2024-03-04T07:33:55.563905+0000","last_active":"2024-03-04T08:28:12.030530+0000","last_peered":"2024-03-04T08:28:12.030530+0000","last_clean":"2024-03-04T08:28:12.030530+0000","last_became_active":"2024-03-04T07:33:55.563117+0000","last_became_peered":"2024-03-04T07:33:55.563117+0000","last_unstale":"2024-03-04T08:28:12.030530+0000","last_undegraded":"2024-03-04T08:28:12.030530+0000","last_fullsized":"2024-03-04T08:28:12.030530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:53:02.522757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":261,"num_read_kb":111921,"num_write":932,"num_write_kb":63511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"80'2022","reported_seq":4660,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.141979+0000","last_change":"2024-03-04T08:19:49.113482+0000","last_active":"2024-03-04T08:28:07.141979+0000","last_peered":"2024-03-04T08:28:07.141979+0000","last_clean":"2024-03-04T08:28:07.141979+0000","last_became_active":"2024-03-04T07:33:56.530776+0000","last_became_peered":"2024-03-04T07:33:56.530776+0000","last_unstale":"2024-03-04T08:28:07.141979+0000","last_undegraded":"2024-03-04T08:28:07.141979+0000","last_fullsized":"2024-03-04T08:28:07.141979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2022,"log_dups_size":0,"ondisk_log_size":2022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T07:59:39.265011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.715426973,"stat_sum":{"num_bytes":4196564,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":94,"num_read_kb":11883,"num_write":1291,"num_write_kb":12915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"84'1859","reported_seq":3279,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.141993+0000","last_change":"2024-03-04T07:33:55.538631+0000","last_active":"2024-03-04T08:28:07.141993+0000","last_peered":"2024-03-04T08:28:07.141993+0000","last_clean":"2024-03-04T08:28:07.141993+0000","last_became_active":"2024-03-04T07:33:55.537104+0000","last_became_peered":"2024-03-04T07:33:55.537104+0000","last_unstale":"2024-03-04T08:28:07.141993+0000","last_undegraded":"2024-03-04T08:28:07.141993+0000","last_fullsized":"2024-03-04T08:28:07.141993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1859,"log_dups_size":0,"ondisk_log_size":1859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:27:06.175808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165914,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1355,"num_read_kb":144971,"num_write":1920,"num_write_kb":84003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"82'2174","reported_seq":4784,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057686+0000","last_change":"2024-03-04T08:19:35.433750+0000","last_active":"2024-03-04T08:28:08.057686+0000","last_peered":"2024-03-04T08:28:08.057686+0000","last_clean":"2024-03-04T08:28:08.057686+0000","last_became_active":"2024-03-04T07:33:56.540536+0000","last_became_peered":"2024-03-04T07:33:56.540536+0000","last_unstale":"2024-03-04T08:28:08.057686+0000","last_undegraded":"2024-03-04T08:28:08.057686+0000","last_fullsized":"2024-03-04T08:28:08.057686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2174,"log_dups_size":0,"ondisk_log_size":2174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:23:13.702668+0000","scrub_duration":0,"objects_trimmed":166,"snaptrim_duration":2.7381961829999999,"stat_sum":{"num_bytes":19342025,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":108,"num_read_kb":53174,"num_write":1367,"num_write_kb":31960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"84'854","reported_seq":1121,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:06.999129+0000","last_change":"2024-03-04T07:33:55.574740+0000","last_active":"2024-03-04T08:28:06.999129+0000","last_peered":"2024-03-04T08:28:06.999129+0000","last_clean":"2024-03-04T08:28:06.999129+0000","last_became_active":"2024-03-04T07:33:55.574129+0000","last_became_peered":"2024-03-04T07:33:55.574129+0000","last_unstale":"2024-03-04T08:28:06.999129+0000","last_undegraded":"2024-03-04T08:28:06.999129+0000","last_fullsized":"2024-03-04T08:28:06.999129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:40:57.891803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":189,"num_read_kb":128430,"num_write":957,"num_write_kb":67580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"82'2115","reported_seq":4827,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.133511+0000","last_change":"2024-03-04T08:19:33.358585+0000","last_active":"2024-03-04T08:28:07.133511+0000","last_peered":"2024-03-04T08:28:07.133511+0000","last_clean":"2024-03-04T08:28:07.133511+0000","last_became_active":"2024-03-04T07:33:56.541422+0000","last_became_peered":"2024-03-04T07:33:56.541422+0000","last_unstale":"2024-03-04T08:28:07.133511+0000","last_undegraded":"2024-03-04T08:28:07.133511+0000","last_fullsized":"2024-03-04T08:28:07.133511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2115,"log_dups_size":0,"ondisk_log_size":2115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:02:25.299647+0000","scrub_duration":0,"objects_trimmed":76,"snaptrim_duration":0.66844205800000001,"stat_sum":{"num_bytes":8391536,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":186,"num_read_kb":5163,"num_write":1360,"num_write_kb":16715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"85'649","reported_seq":906,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057737+0000","last_change":"2024-03-04T07:33:55.539317+0000","last_active":"2024-03-04T08:28:08.057737+0000","last_peered":"2024-03-04T08:28:08.057737+0000","last_clean":"2024-03-04T08:28:08.057737+0000","last_became_active":"2024-03-04T07:33:55.539017+0000","last_became_peered":"2024-03-04T07:33:55.539017+0000","last_unstale":"2024-03-04T08:28:08.057737+0000","last_undegraded":"2024-03-04T08:28:08.057737+0000","last_fullsized":"2024-03-04T08:28:08.057737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:20:10.787316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":165,"num_read_kb":131936,"num_write":747,"num_write_kb":79565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"82'2168","reported_seq":4796,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.001381+0000","last_change":"2024-03-04T08:19:36.855176+0000","last_active":"2024-03-04T08:28:22.001381+0000","last_peered":"2024-03-04T08:28:22.001381+0000","last_clean":"2024-03-04T08:28:22.001381+0000","last_became_active":"2024-03-04T07:33:56.553742+0000","last_became_peered":"2024-03-04T07:33:56.553742+0000","last_unstale":"2024-03-04T08:28:22.001381+0000","last_undegraded":"2024-03-04T08:28:22.001381+0000","last_fullsized":"2024-03-04T08:28:22.001381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2168,"log_dups_size":0,"ondisk_log_size":2168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:55:27.955824+0000","scrub_duration":0,"objects_trimmed":68,"snaptrim_duration":0.86588785899999998,"stat_sum":{"num_bytes":25169512,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":118,"num_read_kb":50143,"num_write":1386,"num_write_kb":35653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"82'874","reported_seq":1154,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.032175+0000","last_change":"2024-03-04T07:33:55.569442+0000","last_active":"2024-03-04T08:28:22.032175+0000","last_peered":"2024-03-04T08:28:22.032175+0000","last_clean":"2024-03-04T08:28:22.032175+0000","last_became_active":"2024-03-04T07:33:55.567809+0000","last_became_peered":"2024-03-04T07:33:55.567809+0000","last_unstale":"2024-03-04T08:28:22.032175+0000","last_undegraded":"2024-03-04T08:28:22.032175+0000","last_fullsized":"2024-03-04T08:28:22.032175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T13:04:58.551866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971542,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":208,"num_read_kb":126955,"num_write":1015,"num_write_kb":67473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"80'2113","reported_seq":4760,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057728+0000","last_change":"2024-03-04T08:19:35.449422+0000","last_active":"2024-03-04T08:28:08.057728+0000","last_peered":"2024-03-04T08:28:08.057728+0000","last_clean":"2024-03-04T08:28:08.057728+0000","last_became_active":"2024-03-04T07:33:56.542775+0000","last_became_peered":"2024-03-04T07:33:56.542775+0000","last_unstale":"2024-03-04T08:28:08.057728+0000","last_undegraded":"2024-03-04T08:28:08.057728+0000","last_fullsized":"2024-03-04T08:28:08.057728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2113,"log_dups_size":0,"ondisk_log_size":2113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:24:08.926542+0000","scrub_duration":0,"objects_trimmed":162,"snaptrim_duration":2.754112347,"stat_sum":{"num_bytes":21241084,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":124,"num_read_kb":50364,"num_write":1380,"num_write_kb":30457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"82'1819","reported_seq":2086,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057769+0000","last_change":"2024-03-04T07:33:55.545598+0000","last_active":"2024-03-04T08:28:08.057769+0000","last_peered":"2024-03-04T08:28:08.057769+0000","last_clean":"2024-03-04T08:28:08.057769+0000","last_became_active":"2024-03-04T07:33:55.545262+0000","last_became_peered":"2024-03-04T07:33:55.545262+0000","last_unstale":"2024-03-04T08:28:08.057769+0000","last_undegraded":"2024-03-04T08:28:08.057769+0000","last_fullsized":"2024-03-04T08:28:08.057769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":1819,"log_dups_size":0,"ondisk_log_size":1819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:59:01.667243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":214,"num_read_kb":99465,"num_write":3832,"num_write_kb":59968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"80'1975","reported_seq":4630,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.518040+0000","last_change":"2024-03-04T08:19:52.454759+0000","last_active":"2024-03-04T08:28:08.518040+0000","last_peered":"2024-03-04T08:28:08.518040+0000","last_clean":"2024-03-04T08:28:08.518040+0000","last_became_active":"2024-03-04T07:33:56.550604+0000","last_became_peered":"2024-03-04T07:33:56.550604+0000","last_unstale":"2024-03-04T08:28:08.518040+0000","last_undegraded":"2024-03-04T08:28:08.518040+0000","last_fullsized":"2024-03-04T08:28:08.518040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":1975,"log_dups_size":0,"ondisk_log_size":1975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:17:11.625595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.366898226,"stat_sum":{"num_bytes":4218390,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":111,"num_read_kb":1574,"num_write":1260,"num_write_kb":17353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'484","reported_seq":758,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.002985+0000","last_change":"2024-03-04T07:33:55.567220+0000","last_active":"2024-03-04T08:28:07.002985+0000","last_peered":"2024-03-04T08:28:07.002985+0000","last_clean":"2024-03-04T08:28:07.002985+0000","last_became_active":"2024-03-04T07:33:55.566600+0000","last_became_peered":"2024-03-04T07:33:55.566600+0000","last_unstale":"2024-03-04T08:28:07.002985+0000","last_undegraded":"2024-03-04T08:28:07.002985+0000","last_fullsized":"2024-03-04T08:28:07.002985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.876938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":246,"num_read_kb":67057,"num_write":625,"num_write_kb":42779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"80'2039","reported_seq":4677,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142035+0000","last_change":"2024-03-04T08:19:49.347949+0000","last_active":"2024-03-04T08:28:07.142035+0000","last_peered":"2024-03-04T08:28:07.142035+0000","last_clean":"2024-03-04T08:28:07.142035+0000","last_became_active":"2024-03-04T07:33:56.543836+0000","last_became_peered":"2024-03-04T07:33:56.543836+0000","last_unstale":"2024-03-04T08:28:07.142035+0000","last_undegraded":"2024-03-04T08:28:07.142035+0000","last_fullsized":"2024-03-04T08:28:07.142035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2039,"log_dups_size":0,"ondisk_log_size":2039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:44:01.157492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":2.94967019,"stat_sum":{"num_bytes":8393261,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":91,"num_read_kb":20301,"num_write":1337,"num_write_kb":17823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'627","reported_seq":911,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.029678+0000","last_change":"2024-03-04T07:33:55.566886+0000","last_active":"2024-03-04T08:28:07.029678+0000","last_peered":"2024-03-04T08:28:07.029678+0000","last_clean":"2024-03-04T08:28:07.029678+0000","last_became_active":"2024-03-04T07:33:55.566048+0000","last_became_peered":"2024-03-04T07:33:55.566048+0000","last_unstale":"2024-03-04T08:28:07.029678+0000","last_undegraded":"2024-03-04T08:28:07.029678+0000","last_fullsized":"2024-03-04T08:28:07.029678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:45:27.190438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":244,"num_read_kb":91476,"num_write":742,"num_write_kb":42693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"80'2170","reported_seq":4766,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.133468+0000","last_change":"2024-03-04T08:19:33.369171+0000","last_active":"2024-03-04T08:28:07.133468+0000","last_peered":"2024-03-04T08:28:07.133468+0000","last_clean":"2024-03-04T08:28:07.133468+0000","last_became_active":"2024-03-04T07:33:56.531901+0000","last_became_peered":"2024-03-04T07:33:56.531901+0000","last_unstale":"2024-03-04T08:28:07.133468+0000","last_undegraded":"2024-03-04T08:28:07.133468+0000","last_fullsized":"2024-03-04T08:28:07.133468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2170,"log_dups_size":0,"ondisk_log_size":2170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:43:17.466639+0000","scrub_duration":0,"objects_trimmed":68,"snaptrim_duration":0.67861760500000001,"stat_sum":{"num_bytes":12583051,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":98,"num_read_kb":23140,"num_write":1358,"num_write_kb":19821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"85'800","reported_seq":1103,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.057715+0000","last_change":"2024-03-04T07:33:55.557568+0000","last_active":"2024-03-04T08:28:08.057715+0000","last_peered":"2024-03-04T08:28:08.057715+0000","last_clean":"2024-03-04T08:28:08.057715+0000","last_became_active":"2024-03-04T07:33:55.557330+0000","last_became_peered":"2024-03-04T07:33:55.557330+0000","last_unstale":"2024-03-04T08:28:08.057715+0000","last_undegraded":"2024-03-04T08:28:08.057715+0000","last_fullsized":"2024-03-04T08:28:08.057715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T12:02:34.956318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18594593,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":285,"num_read_kb":97953,"num_write":952,"num_write_kb":44586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"80'2245","reported_seq":4842,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:08.517974+0000","last_change":"2024-03-04T08:19:52.542997+0000","last_active":"2024-03-04T08:28:08.517974+0000","last_peered":"2024-03-04T08:28:08.517974+0000","last_clean":"2024-03-04T08:28:08.517974+0000","last_became_active":"2024-03-04T07:33:56.552658+0000","last_became_peered":"2024-03-04T07:33:56.552658+0000","last_unstale":"2024-03-04T08:28:08.517974+0000","last_undegraded":"2024-03-04T08:28:08.517974+0000","last_fullsized":"2024-03-04T08:28:08.517974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2245,"log_dups_size":0,"ondisk_log_size":2245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:02:38.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.4718202119999999,"stat_sum":{"num_bytes":2645,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":101,"num_read_kb":1927,"num_write":1438,"num_write_kb":12677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'827","reported_seq":1131,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:18.058654+0000","last_change":"2024-03-04T07:33:55.536480+0000","last_active":"2024-03-04T08:28:18.058654+0000","last_peered":"2024-03-04T08:28:18.058654+0000","last_clean":"2024-03-04T08:28:18.058654+0000","last_became_active":"2024-03-04T07:33:55.536139+0000","last_became_peered":"2024-03-04T07:33:55.536139+0000","last_unstale":"2024-03-04T08:28:18.058654+0000","last_undegraded":"2024-03-04T08:28:18.058654+0000","last_fullsized":"2024-03-04T08:28:18.058654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:50:12.778026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":276,"num_read_kb":124915,"num_write":1046,"num_write_kb":68013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"80'2139","reported_seq":4762,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:58.516982+0000","last_change":"2024-03-04T08:19:52.574166+0000","last_active":"2024-03-04T08:27:58.516982+0000","last_peered":"2024-03-04T08:27:58.516982+0000","last_clean":"2024-03-04T08:27:58.516982+0000","last_became_active":"2024-03-04T07:33:56.546280+0000","last_became_peered":"2024-03-04T07:33:56.546280+0000","last_unstale":"2024-03-04T08:27:58.516982+0000","last_undegraded":"2024-03-04T08:27:58.516982+0000","last_fullsized":"2024-03-04T08:27:58.516982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2139,"log_dups_size":0,"ondisk_log_size":2139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T14:04:37.188813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.5027303859999996,"stat_sum":{"num_bytes":8389588,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":116,"num_read_kb":19661,"num_write":1373,"num_write_kb":18499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'895","reported_seq":1206,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:27:57.028091+0000","last_change":"2024-03-04T07:33:55.572540+0000","last_active":"2024-03-04T08:27:57.028091+0000","last_peered":"2024-03-04T08:27:57.028091+0000","last_clean":"2024-03-04T08:27:57.028091+0000","last_became_active":"2024-03-04T07:33:55.571612+0000","last_became_peered":"2024-03-04T07:33:55.571612+0000","last_unstale":"2024-03-04T08:27:57.028091+0000","last_undegraded":"2024-03-04T08:27:57.028091+0000","last_fullsized":"2024-03-04T08:27:57.028091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:01:43.741423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":286,"num_read_kb":176599,"num_write":1032,"num_write_kb":79834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"80'2075","reported_seq":4727,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:22.005213+0000","last_change":"2024-03-04T08:19:34.379981+0000","last_active":"2024-03-04T08:28:22.005213+0000","last_peered":"2024-03-04T08:28:22.005213+0000","last_clean":"2024-03-04T08:28:22.005213+0000","last_became_active":"2024-03-04T07:33:56.532245+0000","last_became_peered":"2024-03-04T07:33:56.532245+0000","last_unstale":"2024-03-04T08:28:22.005213+0000","last_undegraded":"2024-03-04T08:28:22.005213+0000","last_fullsized":"2024-03-04T08:28:22.005213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2075,"log_dups_size":0,"ondisk_log_size":2075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:17:47.216527+0000","scrub_duration":0,"objects_trimmed":129,"snaptrim_duration":1.668147552,"stat_sum":{"num_bytes":16789440,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":116,"num_read_kb":38061,"num_write":1331,"num_write_kb":29084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"82'2080","reported_seq":4692,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.984492+0000","last_change":"2024-03-04T08:19:36.839132+0000","last_active":"2024-03-04T08:26:26.984492+0000","last_peered":"2024-03-04T08:26:26.984492+0000","last_clean":"2024-03-04T08:26:26.984492+0000","last_became_active":"2024-03-04T07:33:56.563473+0000","last_became_peered":"2024-03-04T07:33:56.563473+0000","last_unstale":"2024-03-04T08:26:26.984492+0000","last_undegraded":"2024-03-04T08:26:26.984492+0000","last_fullsized":"2024-03-04T08:26:26.984492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2080,"log_dups_size":0,"ondisk_log_size":2080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T08:46:20.229965+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.84949308899999998,"stat_sum":{"num_bytes":8388810,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":102,"num_read_kb":13463,"num_write":1308,"num_write_kb":22727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"82'689","reported_seq":944,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014567+0000","last_change":"2024-03-04T07:33:55.575688+0000","last_active":"2024-03-04T08:26:27.014567+0000","last_peered":"2024-03-04T08:26:27.014567+0000","last_clean":"2024-03-04T08:26:27.014567+0000","last_became_active":"2024-03-04T07:33:55.574766+0000","last_became_peered":"2024-03-04T07:33:55.574766+0000","last_unstale":"2024-03-04T08:26:27.014567+0000","last_undegraded":"2024-03-04T08:26:27.014567+0000","last_fullsized":"2024-03-04T08:26:27.014567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:32:19.708016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":185,"num_read_kb":103365,"num_write":788,"num_write_kb":63233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":31,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"80'2223","reported_seq":4823,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:26.984465+0000","last_change":"2024-03-04T08:19:36.844614+0000","last_active":"2024-03-04T08:26:26.984465+0000","last_peered":"2024-03-04T08:26:26.984465+0000","last_clean":"2024-03-04T08:26:26.984465+0000","last_became_active":"2024-03-04T07:33:56.563474+0000","last_became_peered":"2024-03-04T07:33:56.563474+0000","last_unstale":"2024-03-04T08:26:26.984465+0000","last_undegraded":"2024-03-04T08:26:26.984465+0000","last_fullsized":"2024-03-04T08:26:26.984465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2223,"log_dups_size":0,"ondisk_log_size":2223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:11:22.319394+0000","scrub_duration":0,"objects_trimmed":34,"snaptrim_duration":0.85581109,"stat_sum":{"num_bytes":20978408,"num_objects":84,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":0,"num_read":117,"num_read_kb":36831,"num_write":1409,"num_write_kb":34422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"85'764","reported_seq":1068,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:26:27.014595+0000","last_change":"2024-03-04T07:33:55.570016+0000","last_active":"2024-03-04T08:26:27.014595+0000","last_peered":"2024-03-04T08:26:27.014595+0000","last_clean":"2024-03-04T08:26:27.014595+0000","last_became_active":"2024-03-04T07:33:55.568749+0000","last_became_peered":"2024-03-04T07:33:55.568749+0000","last_unstale":"2024-03-04T08:26:27.014595+0000","last_undegraded":"2024-03-04T08:26:27.014595+0000","last_fullsized":"2024-03-04T08:26:27.014595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:29:01.214561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":279,"num_read_kb":188218,"num_write":863,"num_write_kb":100106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"80'2088","reported_seq":4689,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142049+0000","last_change":"2024-03-04T08:19:49.462793+0000","last_active":"2024-03-04T08:28:07.142049+0000","last_peered":"2024-03-04T08:28:07.142049+0000","last_clean":"2024-03-04T08:28:07.142049+0000","last_became_active":"2024-03-04T07:33:56.561675+0000","last_became_peered":"2024-03-04T07:33:56.561675+0000","last_unstale":"2024-03-04T08:28:07.142049+0000","last_undegraded":"2024-03-04T08:28:07.142049+0000","last_fullsized":"2024-03-04T08:28:07.142049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2088,"log_dups_size":0,"ondisk_log_size":2088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T11:36:21.385199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":3.0645439379999999,"stat_sum":{"num_bytes":8398756,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":91,"num_read_kb":13948,"num_write":1308,"num_write_kb":18354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"84'844","reported_seq":1117,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:03.517495+0000","last_change":"2024-03-04T07:33:55.538337+0000","last_active":"2024-03-04T08:28:03.517495+0000","last_peered":"2024-03-04T08:28:03.517495+0000","last_clean":"2024-03-04T08:28:03.517495+0000","last_became_active":"2024-03-04T07:33:55.538010+0000","last_became_peered":"2024-03-04T07:33:55.538010+0000","last_unstale":"2024-03-04T08:28:03.517495+0000","last_undegraded":"2024-03-04T08:28:03.517495+0000","last_fullsized":"2024-03-04T08:28:03.517495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":844,"log_dups_size":0,"ondisk_log_size":844,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T10:47:15.711205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":199,"num_read_kb":128064,"num_write":938,"num_write_kb":63013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'2071","reported_seq":4691,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:03.517472+0000","last_change":"2024-03-04T08:19:52.402904+0000","last_active":"2024-03-04T08:28:03.517472+0000","last_peered":"2024-03-04T08:28:03.517472+0000","last_clean":"2024-03-04T08:28:03.517472+0000","last_became_active":"2024-03-04T07:33:56.558799+0000","last_became_peered":"2024-03-04T07:33:56.558799+0000","last_unstale":"2024-03-04T08:28:03.517472+0000","last_undegraded":"2024-03-04T08:28:03.517472+0000","last_fullsized":"2024-03-04T08:28:03.517472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:55.505128+0000","last_clean_scrub_stamp":"2024-03-04T07:33:55.505128+0000","objects_scrubbed":0,"log_size":2071,"log_dups_size":0,"ondisk_log_size":2071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T09:34:51.677019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":6.3308552689999997,"stat_sum":{"num_bytes":16779138,"num_objects":75,"num_object_clones":0,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":99,"num_read_kb":47695,"num_write":1322,"num_write_kb":28136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"84'627","reported_seq":937,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-04T08:28:07.142064+0000","last_change":"2024-03-04T07:33:55.566294+0000","last_active":"2024-03-04T08:28:07.142064+0000","last_peered":"2024-03-04T08:28:07.142064+0000","last_clean":"2024-03-04T08:28:07.142064+0000","last_became_active":"2024-03-04T07:33:55.565941+0000","last_became_peered":"2024-03-04T07:33:55.565941+0000","last_unstale":"2024-03-04T08:28:07.142064+0000","last_undegraded":"2024-03-04T08:28:07.142064+0000","last_fullsized":"2024-03-04T08:28:07.142064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T07:33:54.497279+0000","last_clean_scrub_stamp":"2024-03-04T07:33:54.497279+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:04:03.115085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360710,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":306,"num_read_kb":111793,"num_write":679,"num_write_kb":46654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":758811247,"num_objects":4817,"num_object_clones":0,"num_object_copies":9634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4817,"num_whiteouts":0,"num_read":7109,"num_read_kb":1683514,"num_write":86591,"num_write_kb":1471650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1519542272,"data_stored":1517622494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137094,"ondisk_log_size":137094,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":589610469,"num_objects":1415,"num_object_clones":0,"num_object_copies":2830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1415,"num_whiteouts":0,"num_read":9520,"num_read_kb":4046989,"num_write":33636,"num_write_kb":2138987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1203,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1179369472,"data_stored":1179220938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27575,"ondisk_log_size":27575,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424510129,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537624,"kb_used_data":362128,"kb_used_omap":1,"kb_used_meta":175486,"kb_avail":93834216,"statfs":{"total":96636764160,"available":96086237184,"internally_reserved":0,"allocated":370819072,"data_stored":370026846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":179698119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424510128,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466432,"kb_used_data":300664,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":93905408,"statfs":{"total":96636764160,"available":96159137792,"internally_reserved":0,"allocated":307879936,"data_stored":307237685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":169736645},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542833,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564504,"kb_used_data":389072,"kb_used_omap":1,"kb_used_meta":175422,"kb_avail":93807336,"statfs":{"total":96636764160,"available":96058712064,"internally_reserved":0,"allocated":398409728,"data_stored":397634775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":179632586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542832,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505096,"kb_used_data":321344,"kb_used_omap":1,"kb_used_meta":183742,"kb_avail":93866744,"statfs":{"total":96636764160,"available":96119545856,"internally_reserved":0,"allocated":329056256,"data_stored":328434784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188152262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542832,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583216,"kb_used_data":403556,"kb_used_omap":1,"kb_used_meta":179646,"kb_avail":93788624,"statfs":{"total":96636764160,"available":96039550976,"internally_reserved":0,"allocated":413241344,"data_stored":412455445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183957957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542833,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509020,"kb_used_data":329360,"kb_used_omap":1,"kb_used_meta":179646,"kb_avail":93862820,"statfs":{"total":96636764160,"available":96115527680,"internally_reserved":0,"allocated":337264640,"data_stored":336553162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":183957957},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542833,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340760,"kb_used_data":241612,"kb_used_omap":1,"kb_used_meta":99134,"kb_avail":94031080,"statfs":{"total":96636764160,"available":96287825920,"internally_reserved":0,"allocated":247410688,"data_stored":246748656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101513669},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542831,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462692,"kb_used_data":296920,"kb_used_omap":1,"kb_used_meta":165758,"kb_avail":93909148,"statfs":{"total":96636764160,"available":96162967552,"internally_reserved":0,"allocated":304046080,"data_stored":303311191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":169736651},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":121651200,"data_stored":121635476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157036544,"data_stored":157012200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92295168,"data_stored":92275506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":151015424,"data_stored":150997655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":104878080,"data_stored":104864685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180957184,"data_stored":180935092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":168370176,"data_stored":168353802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":203165696,"data_stored":203146522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":180797440,"data_stored":180538050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149839872,"data_stored":149678188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":154112000,"data_stored":153925853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":218800128,"data_stored":218481894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":231383040,"data_stored":231141180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":231280640,"data_stored":230973056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":159682560,"data_stored":159533685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":193646592,"data_stored":193350588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T08:30:46.336 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-04T08:30:46.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.511+0000 7fac53200640 1 Processor -- start 2024-03-04T08:30:46.508 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.511+0000 7fac53200640 1 -- start start 2024-03-04T08:30:46.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.511+0000 7fac53200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c8200 0x7fac4c070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.511+0000 7fac53200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fac4c152bd0 0x7fac4c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.511+0000 7fac53200640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c1591f0 con 0x7fac4c1527f0 2024-03-04T08:30:46.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.511+0000 7fac53200640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c071df0 con 0x7fac4c152bd0 2024-03-04T08:30:46.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.511+0000 7fac53200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c1597b0 con 0x7fac4c0c8200 2024-03-04T08:30:46.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.511+0000 7fac4be00640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fac4c1527f0 0x7fac4c072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:40018/0 (socket says 172.21.15.113:40018) 2024-03-04T08:30:46.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.511+0000 7fac4be00640 1 -- 172.21.15.113:0/683443168 learned_addr learned my addr 172.21.15.113:0/683443168 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-04T08:30:46.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac51200640 1 --2- 172.21.15.113:0/683443168 >> v2:172.21.15.136:3300/0 conn(0x7fac4c152bd0 0x7fac4c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac43400640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c8200 0x7fac4c070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 <== mon.0 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251856531 0 0) 0x7fac4c1591f0 con 0x7fac4c1527f0 2024-03-04T08:30:46.511 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac30003530 con 0x7fac4c1527f0 2024-03-04T08:30:46.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 <== mon.0 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4229639400 0 0) 0x7fac30003530 con 0x7fac4c1527f0 2024-03-04T08:30:46.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c8200 msgr2=0x7fac4c070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:46.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c8200 0x7fac4c070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 >> v2:172.21.15.136:3300/0 conn(0x7fac4c152bd0 msgr2=0x7fac4c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:46.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 --2- 172.21.15.113:0/683443168 >> v2:172.21.15.136:3300/0 conn(0x7fac4c152bd0 0x7fac4c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c1594d0 con 0x7fac4c1527f0 2024-03-04T08:30:46.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 <== mon.0 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2986594773 0 0) 0x7fac3c001680 con 0x7fac4c1527f0 2024-03-04T08:30:46.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 >> v1:172.21.15.113:6789/0 conn(0x7fac4c1527f0 legacy=0x7fac4c072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac300039c0 0x7fac30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac30004810 0x7fac30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c8200 0x7fac300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c071df0 con 0x7fac30004810 2024-03-04T08:30:46.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c1597b0 con 0x7fac300039c0 2024-03-04T08:30:46.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac3000b9d0 con 0x7fac4c0c8200 2024-03-04T08:30:46.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac43400640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac30004810 0x7fac30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac4be00640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac300039c0 0x7fac30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac51200640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c8200 0x7fac300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac43400640 1 -- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c8200 msgr2=0x7fac300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac43400640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c8200 0x7fac300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac43400640 1 -- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac300039c0 msgr2=0x7fac30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-04T08:30:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac43400640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac300039c0 0x7fac30003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.515 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.515+0000 7fac43400640 1 -- 172.21.15.113:0/683443168 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac30003530 con 0x7fac30004810 2024-03-04T08:30:46.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac43400640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac30004810 0x7fac30004c90 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fac34009ab0 tx=0x7fac3401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacc6f1fdbf8f02c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:46.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340096b0 con 0x7fac30004810 2024-03-04T08:30:46.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac340a6460 con 0x7fac30004810 2024-03-04T08:30:46.516 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac4b400640 1 -- 172.21.15.113:0/683443168 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340a6850 con 0x7fac30004810 2024-03-04T08:30:46.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 -- 172.21.15.113:0/683443168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac30004810 msgr2=0x7fac30004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 --2- 172.21.15.113:0/683443168 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac30004810 0x7fac30004c90 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fac34009ab0 tx=0x7fac3401ca50 comp rx=0 tx=0).stop 2024-03-04T08:30:46.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac51200640 1 -- 172.21.15.113:0/683443168 reap_dead start 2024-03-04T08:30:46.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 -- 172.21.15.113:0/683443168 shutdown_connections 2024-03-04T08:30:46.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 -- 172.21.15.113:0/683443168 >> 172.21.15.113:0/683443168 conn(0x7fac4c06e820 msgr2=0x7fac4c06ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:46.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 -- 172.21.15.113:0/683443168 shutdown_connections 2024-03-04T08:30:46.517 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 -- 172.21.15.113:0/683443168 wait complete. 2024-03-04T08:30:46.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 Processor -- start 2024-03-04T08:30:46.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 -- start start 2024-03-04T08:30:46.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30004810 0x7fac4c0b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac4c0b1710 0x7fac4c0c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c2cb0 0x7fac4c0c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c158d10 con 0x7fac4c0b1710 2024-03-04T08:30:46.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c071a80 con 0x7fac30004810 2024-03-04T08:30:46.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac53200640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c158e90 con 0x7fac4c0c2cb0 2024-03-04T08:30:46.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.519+0000 7fac43400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac4c0b1710 0x7fac4c0c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac43400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac4c0b1710 0x7fac4c0c21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:53018/0 (socket says 172.21.15.113:53018) 2024-03-04T08:30:46.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac43400640 1 -- 172.21.15.113:0/721049926 learned_addr learned my addr 172.21.15.113:0/721049926 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-04T08:30:46.520 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac4be00640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30004810 0x7fac4c0b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac51200640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c2cb0 0x7fac4c0c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac43400640 1 -- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c2cb0 msgr2=0x7fac4c0c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac43400640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c2cb0 0x7fac4c0c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac43400640 1 -- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30004810 msgr2=0x7fac4c0b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac43400640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30004810 0x7fac4c0b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac43400640 1 -- 172.21.15.113:0/721049926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c1e4880 con 0x7fac4c0b1710 2024-03-04T08:30:46.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac43400640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac4c0b1710 0x7fac4c0c21e0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fac34009ab0 tx=0x7fac30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:46.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac48c00640 1 -- 172.21.15.113:0/721049926 <== mon.0 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340c0070 con 0x7fac4c0b1710 2024-03-04T08:30:46.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac48c00640 1 -- 172.21.15.113:0/721049926 <== mon.0 v2:172.21.15.113:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac340a6e20 con 0x7fac4c0b1710 2024-03-04T08:30:46.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac48c00640 1 -- 172.21.15.113:0/721049926 <== mon.0 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340096b0 con 0x7fac4c0b1710 2024-03-04T08:30:46.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac53200640 1 -- 172.21.15.113:0/721049926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c1f1160 con 0x7fac4c0b1710 2024-03-04T08:30:46.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac53200640 1 -- 172.21.15.113:0/721049926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c0afcd0 con 0x7fac4c0b1710 2024-03-04T08:30:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac53200640 1 -- 172.21.15.113:0/721049926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c075c00 con 0x7fac4c0b1710 2024-03-04T08:30:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.523+0000 7fac48c00640 1 -- 172.21.15.113:0/721049926 <== mon.0 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac340c3070 con 0x7fac4c0b1710 2024-03-04T08:30:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.527+0000 7fac48c00640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac28066f60 0x7fac28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.527+0000 7fac48c00640 1 -- 172.21.15.113:0/721049926 <== mon.0 v2:172.21.15.113:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8405+0+0 (secure 0 0 0) 0x7fac3411fc70 con 0x7fac4c0b1710 2024-03-04T08:30:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.527+0000 7fac4be00640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac28066f60 0x7fac28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T08:30:46.523 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.527+0000 7fac4be00640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac28066f60 0x7fac28069420 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fac3c005300 tx=0x7fac3c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T08:30:46.525 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:30:46.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.531+0000 7fac48c00640 1 -- 172.21.15.113:0/721049926 <== mon.0 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fac340ac020 con 0x7fac4c0b1710 2024-03-04T08:30:46.529 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:30:46.684 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.687+0000 7fac53200640 1 -- 172.21.15.113:0/721049926 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac4c0c79d0 con 0x7fac4c0b1710 2024-03-04T08:30:46.686 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.687+0000 7fac48c00640 1 -- 172.21.15.113:0/721049926 <== mon.0 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+20859 (secure 0 0 0) 0x7fac34016760 con 0x7fac4c0b1710 2024-03-04T08:30:46.686 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-04T08:30:46.686 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":85,"fsid":"43451c9f-4d9f-4f78-a5ec-a82e7d344007","created":"2024-03-04T07:33:30.835110+0000","modified":"2024-03-04T08:24:26.023707+0000","last_up_change":"2024-03-04T07:33:42.183823+0000","last_in_change":"2024-03-04T07:33:33.280839+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T07:33:41.507700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-04T07:33:53.680465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-04T07:33:54.892043+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":683,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cbf2ee75-ec28-483b-bb34-349b80272fcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6825","nonce":596399521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6827","nonce":596399521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6831","nonce":596399521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":596399521},{"type":"v1","addr":"172.21.15.113:6829","nonce":596399521}]},"public_addr":"172.21.15.113:6825/596399521","cluster_addr":"172.21.15.113:6827/596399521","heartbeat_back_addr":"172.21.15.113:6831/596399521","heartbeat_front_addr":"172.21.15.113:6829/596399521","state":["exists","up"]},{"osd":1,"uuid":"49418806-cd59-4b3f-a3f1-7ea1736b54fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6801","nonce":3516786891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6803","nonce":3516786891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6807","nonce":3516786891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":3516786891},{"type":"v1","addr":"172.21.15.113:6805","nonce":3516786891}]},"public_addr":"172.21.15.113:6801/3516786891","cluster_addr":"172.21.15.113:6803/3516786891","heartbeat_back_addr":"172.21.15.113:6807/3516786891","heartbeat_front_addr":"172.21.15.113:6805/3516786891","state":["exists","up"]},{"osd":2,"uuid":"0513256b-863e-406e-88da-d1ad6902ed24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6817","nonce":4286907884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6819","nonce":4286907884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6823","nonce":4286907884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":4286907884},{"type":"v1","addr":"172.21.15.113:6821","nonce":4286907884}]},"public_addr":"172.21.15.113:6817/4286907884","cluster_addr":"172.21.15.113:6819/4286907884","heartbeat_back_addr":"172.21.15.113:6823/4286907884","heartbeat_front_addr":"172.21.15.113:6821/4286907884","state":["exists","up"]},{"osd":3,"uuid":"32c27a18-acb8-4089-a959-5a8545aabf68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6809","nonce":2541121417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6811","nonce":2541121417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6816","nonce":2541121417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":2541121417},{"type":"v1","addr":"172.21.15.113:6813","nonce":2541121417}]},"public_addr":"172.21.15.113:6809/2541121417","cluster_addr":"172.21.15.113:6811/2541121417","heartbeat_back_addr":"172.21.15.113:6816/2541121417","heartbeat_front_addr":"172.21.15.113:6813/2541121417","state":["exists","up"]},{"osd":4,"uuid":"94deb7a1-ed05-429c-9dd6-cf1e4b8e7fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6809","nonce":196578205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6811","nonce":196578205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6815","nonce":196578205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":196578205},{"type":"v1","addr":"172.21.15.136:6813","nonce":196578205}]},"public_addr":"172.21.15.136:6809/196578205","cluster_addr":"172.21.15.136:6811/196578205","heartbeat_back_addr":"172.21.15.136:6815/196578205","heartbeat_front_addr":"172.21.15.136:6813/196578205","state":["exists","up"]},{"osd":5,"uuid":"2949efcf-b78e-411c-87da-f27685b009bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6825","nonce":1056870180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6827","nonce":1056870180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6831","nonce":1056870180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":1056870180},{"type":"v1","addr":"172.21.15.136:6829","nonce":1056870180}]},"public_addr":"172.21.15.136:6825/1056870180","cluster_addr":"172.21.15.136:6827/1056870180","heartbeat_back_addr":"172.21.15.136:6831/1056870180","heartbeat_front_addr":"172.21.15.136:6829/1056870180","state":["exists","up"]},{"osd":6,"uuid":"9121e020-dcbd-4aea-af25-ce8e580d3968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6817","nonce":1386829518}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6819","nonce":1386829518}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6823","nonce":1386829518}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":1386829518},{"type":"v1","addr":"172.21.15.136:6821","nonce":1386829518}]},"public_addr":"172.21.15.136:6817/1386829518","cluster_addr":"172.21.15.136:6819/1386829518","heartbeat_back_addr":"172.21.15.136:6823/1386829518","heartbeat_front_addr":"172.21.15.136:6821/1386829518","state":["exists","up"]},{"osd":7,"uuid":"c73d46d6-a743-462e-913a-2cb43fec5a6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6801","nonce":1911155961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6803","nonce":1911155961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6807","nonce":1911155961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1911155961},{"type":"v1","addr":"172.21.15.136:6805","nonce":1911155961}]},"public_addr":"172.21.15.136:6801/1911155961","cluster_addr":"172.21.15.136:6803/1911155961","heartbeat_back_addr":"172.21.15.136:6807/1911155961","heartbeat_front_addr":"172.21.15.136:6805/1911155961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.819394+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.960543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.783151+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.938712+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.689485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.891548+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.833248+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T07:33:38.985983+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":2}]},{"pgid":"3.11","mappings":[{"from":7,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.136:6839/1409794802":"2024-03-05T08:24:26.023653+0000","172.21.15.136:6834/2368734308":"2024-03-05T07:58:45.780423+0000","172.21.15.113:6832/3005395426":"2024-03-05T07:37:40.596420+0000","172.21.15.113:6836/986515123":"2024-03-05T08:00:40.798160+0000","172.21.15.113:6834/2299268553":"2024-03-05T07:52:10.722331+0000","172.21.15.136:6834/3944001911":"2024-03-05T07:45:20.664657+0000","172.21.15.113:6836/2299268553":"2024-03-05T07:52:10.722331+0000","172.21.15.136:6835/2522634751":"2024-03-05T08:07:30.861130+0000","172.21.15.136:6834/1691762848":"2024-03-05T07:49:30.698378+0000","172.21.15.113:6832/2732565639":"2024-03-05T08:02:45.817095+0000","172.21.15.113:6837/742633123":"2024-03-05T07:47:40.683981+0000","172.21.15.113:6836/4038214481":"2024-03-05T07:36:05.584380+0000","172.21.15.136:6835/3944001911":"2024-03-05T07:45:20.664657+0000","172.21.15.113:6834/986515123":"2024-03-05T08:00:40.798160+0000","172.21.15.113:6833/605142559":"2024-03-05T07:44:20.656043+0000","172.21.15.113:6836/1035982016":"2024-03-05T08:05:45.840907+0000","172.21.15.113:6833/3005395426":"2024-03-05T07:37:40.596420+0000","172.21.15.113:6832/1928887637":"2024-03-05T08:20:05.977949+0000","172.21.15.113:6833/810789479":"2024-03-05T07:54:40.746387+0000","172.21.15.113:6835/2544977425":"2024-03-05T08:21:40.993913+0000","172.21.15.113:6837/4197507712":"2024-03-05T08:04:30.830241+0000","172.21.15.136:6834/3348592059":"2024-03-05T07:40:15.619912+0000","172.21.15.113:6835/1310042629":"2024-03-05T07:38:50.604670+0000","172.21.15.136:6834/2522634751":"2024-03-05T08:07:30.861130+0000","172.21.15.113:6835/742633123":"2024-03-05T07:47:40.683981+0000","172.21.15.113:6834/1310057122":"2024-03-05T08:13:45.911820+0000","172.21.15.113:6834/4038214481":"2024-03-05T07:36:05.584380+0000","172.21.15.136:6835/1691762848":"2024-03-05T07:49:30.698378+0000","172.21.15.113:6832/605142559":"2024-03-05T07:44:20.656043+0000","172.21.15.113:6837/2544977425":"2024-03-05T08:21:40.993913+0000","172.21.15.113:6835/344022786":"2024-03-05T07:57:05.766508+0000","172.21.15.113:6832/2999156262":"2024-03-05T08:22:51.004939+0000","172.21.15.113:6837/1401330284":"2024-03-05T08:11:10.891335+0000","172.21.15.113:6837/1310042629":"2024-03-05T07:38:50.604670+0000","172.21.15.113:6834/2997246507":"2024-03-05T07:41:40.631034+0000","172.21.15.136:6835/2368734308":"2024-03-05T07:58:45.780423+0000","172.21.15.113:6836/1310057122":"2024-03-05T08:13:45.911820+0000","172.21.15.136:6835/3348592059":"2024-03-05T07:40:15.619912+0000","172.21.15.136:6837/2228772746":"2024-03-05T08:23:26.011590+0000","172.21.15.113:6836/2997246507":"2024-03-05T07:41:40.631034+0000","172.21.15.136:6835/3388011401":"2024-03-05T08:01:30.805387+0000","172.21.15.136:6835/247269911":"2024-03-05T08:16:00.936524+0000","172.21.15.113:6833/2732565639":"2024-03-05T08:02:45.817095+0000","172.21.15.113:6835/4197507712":"2024-03-05T08:04:30.830241+0000","172.21.15.136:6838/1409794802":"2024-03-05T08:24:26.023653+0000","172.21.15.113:6834/1035982016":"2024-03-05T08:05:45.840907+0000","172.21.15.113:6832/3825257303":"2024-03-05T08:09:55.880790+0000","172.21.15.136:6834/3388011401":"2024-03-05T08:01:30.805387+0000","172.21.15.113:6833/3825257303":"2024-03-05T08:09:55.880790+0000","172.21.15.113:6835/1401330284":"2024-03-05T08:11:10.891335+0000","172.21.15.136:6834/247269911":"2024-03-05T08:16:00.936524+0000","172.21.15.113:6833/1928887637":"2024-03-05T08:20:05.977949+0000","172.21.15.113:6834/4083765155":"2024-03-05T08:17:55.953381+0000","172.21.15.113:6833/2999156262":"2024-03-05T08:22:51.004939+0000","172.21.15.113:6836/4083765155":"2024-03-05T08:17:55.953381+0000","172.21.15.113:6832/810789479":"2024-03-05T07:54:40.746387+0000","172.21.15.113:6834/893414524":"2024-03-05T08:21:25.990484+0000","172.21.15.113:6837/344022786":"2024-03-05T07:57:05.766508+0000","172.21.15.113:6836/893414524":"2024-03-05T08:21:25.990484+0000","172.21.15.136:6836/2228772746":"2024-03-05T08:23:26.011590+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T08:30:46.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 -- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac28066f60 msgr2=0x7fac28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac28066f60 0x7fac28069420 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fac3c005300 tx=0x7fac3c009040 comp rx=0 tx=0).stop 2024-03-04T08:30:46.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 -- 172.21.15.113:0/721049926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac4c0b1710 msgr2=0x7fac4c0c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T08:30:46.688 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac4c0b1710 0x7fac4c0c21e0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fac34009ab0 tx=0x7fac30005840 comp rx=0 tx=0).stop 2024-03-04T08:30:46.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 -- 172.21.15.113:0/721049926 shutdown_connections 2024-03-04T08:30:46.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fac4c0c2cb0 0x7fac4c0c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:6832/3353359815,v1:172.21.15.136:6833/3353359815] conn(0x7fac28066f60 0x7fac28069420 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac4c0b1710 0x7fac4c0c21e0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 --2- 172.21.15.113:0/721049926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fac30004810 0x7fac4c0b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:46.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 -- 172.21.15.113:0/721049926 >> 172.21.15.113:0/721049926 conn(0x7fac4c06e820 msgr2=0x7fac4c0ca7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T08:30:46.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 -- 172.21.15.113:0/721049926 shutdown_connections 2024-03-04T08:30:46.689 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:46.691+0000 7fac53200640 1 -- 172.21.15.113:0/721049926 wait complete. 2024-03-04T08:30:46.959 INFO:tasks.ceph.osd.0:Sent signal 15 2024-03-04T08:30:46.960 INFO:tasks.ceph.osd.1:Sent signal 15 2024-03-04T08:30:46.960 INFO:tasks.ceph.osd.2:Sent signal 15 2024-03-04T08:30:46.963 INFO:tasks.ceph.osd.3:Sent signal 15 2024-03-04T08:30:46.963 INFO:tasks.ceph.osd.4:Sent signal 15 2024-03-04T08:30:46.963 INFO:tasks.ceph.osd.5:Sent signal 15 2024-03-04T08:30:46.964 INFO:tasks.ceph.osd.6:Sent signal 15 2024-03-04T08:30:46.964 INFO:tasks.ceph.osd.7:Sent signal 15 2024-03-04T08:30:46.964 INFO:tasks.ceph.mds.a:Sent signal 15 2024-03-04T08:30:46.964 INFO:tasks.ceph.mds.c:Sent signal 15 2024-03-04T08:30:46.965 INFO:tasks.ceph.mds.e:Sent signal 15 2024-03-04T08:30:46.965 INFO:tasks.ceph.mds.b:Sent signal 15 2024-03-04T08:30:46.965 INFO:tasks.ceph.mds.d:Sent signal 15 2024-03-04T08:30:46.965 INFO:tasks.ceph.mon.a:Sent signal 15 2024-03-04T08:30:46.966 INFO:tasks.ceph.mon.b:Sent signal 15 2024-03-04T08:30:46.966 INFO:tasks.ceph.mon.c:Sent signal 15 2024-03-04T08:30:46.966 INFO:tasks.ceph.mgr.y:Sent signal 15 2024-03-04T08:30:46.966 INFO:tasks.ceph.mgr.x:Sent signal 15 2024-03-04T08:30:46.967 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-03-04T08:30:46.965+0000 7fbce7c8c640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 28193) UID: 0 2024-03-04T08:30:46.968 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-03-04T08:30:46.965+0000 7fbce7c8c640 -1 osd.4 85 *** Got signal Terminated *** 2024-03-04T08:30:46.968 INFO:tasks.ceph.osd.4.smithi136.stderr:2024-03-04T08:30:46.965+0000 7fbce7c8c640 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T08:30:46.968 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-03-04T08:30:46.963+0000 7fdf13c00640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 27531) UID: 0 2024-03-04T08:30:46.968 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-03-04T08:30:46.963+0000 7fdf13c00640 -1 osd.0 85 *** Got signal Terminated *** 2024-03-04T08:30:46.968 INFO:tasks.ceph.osd.0.smithi113.stderr:2024-03-04T08:30:46.963+0000 7fdf13c00640 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T08:30:46.969 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-03-04T08:30:46.963+0000 7f687d600640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 27533) UID: 0 2024-03-04T08:30:46.969 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-03-04T08:30:46.963+0000 7f687d600640 -1 osd.1 85 *** Got signal Terminated *** 2024-03-04T08:30:46.969 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-03-04T08:30:46.963+0000 7f687d600640 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T08:30:46.970 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-03-04T08:30:46.963+0000 7f05afa00640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 27534) UID: 0 2024-03-04T08:30:46.970 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-03-04T08:30:46.963+0000 7f05afa00640 -1 osd.2 85 *** Got signal Terminated *** 2024-03-04T08:30:46.970 INFO:tasks.ceph.osd.2.smithi113.stderr:2024-03-04T08:30:46.963+0000 7f05afa00640 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T08:30:46.970 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-03-04T08:30:46.965+0000 7f854fca9640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 28196) UID: 0 2024-03-04T08:30:46.971 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-03-04T08:30:46.965+0000 7f854fca9640 -1 osd.5 85 *** Got signal Terminated *** 2024-03-04T08:30:46.971 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-03-04T08:30:46.965+0000 7f854fca9640 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T08:30:46.971 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-03-04T08:30:46.967+0000 7f0820400640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 27538) UID: 0 2024-03-04T08:30:46.971 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-03-04T08:30:46.967+0000 7f0820400640 -1 osd.3 85 *** Got signal Terminated *** 2024-03-04T08:30:46.971 INFO:tasks.ceph.osd.3.smithi113.stderr:2024-03-04T08:30:46.967+0000 7f0820400640 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T08:30:46.972 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-03-04T08:30:46.967+0000 7fce6ae00640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 50784) UID: 0 2024-03-04T08:30:46.972 INFO:tasks.ceph.mds.a.smithi113.stderr:2024-03-04T08:30:46.967+0000 7fce6ae00640 -1 mds.a *** got signal Terminated *** 2024-03-04T08:30:46.972 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-03-04T08:30:46.969+0000 7fe7b1e7c640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 28198) UID: 0 2024-03-04T08:30:46.972 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-03-04T08:30:46.969+0000 7fe7b1e7c640 -1 osd.6 85 *** Got signal Terminated *** 2024-03-04T08:30:46.973 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-03-04T08:30:46.969+0000 7fe7b1e7c640 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T08:30:46.973 INFO:tasks.ceph.mds.c.smithi113.stderr:2024-03-04T08:30:46.967+0000 7fedde800640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 51530) UID: 0 2024-03-04T08:30:46.973 INFO:tasks.ceph.mds.c.smithi113.stderr:2024-03-04T08:30:46.967+0000 7fedde800640 -1 mds.c *** got signal Terminated *** 2024-03-04T08:30:46.974 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T08:30:46.967+0000 7fb5d7600640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mds -f --cluster ceph -i e (PID: 53036) UID: 0 2024-03-04T08:30:46.974 INFO:tasks.ceph.mds.e.smithi113.stderr:2024-03-04T08:30:46.967+0000 7fb5d7600640 -1 mds.e *** got signal Terminated *** 2024-03-04T08:30:46.974 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-03-04T08:30:46.969+0000 7fa16cb5e640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 28200) UID: 0 2024-03-04T08:30:46.974 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-03-04T08:30:46.969+0000 7fa16cb5e640 -1 osd.7 85 *** Got signal Terminated *** 2024-03-04T08:30:46.974 INFO:tasks.ceph.osd.7.smithi136.stderr:2024-03-04T08:30:46.969+0000 7fa16cb5e640 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T08:30:46.975 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-03-04T08:30:46.971+0000 7fbb38e00640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 27426) UID: 0 2024-03-04T08:30:46.975 INFO:tasks.ceph.mon.a.smithi113.stderr:2024-03-04T08:30:46.971+0000 7fbb38e00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-04T08:30:46.975 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T08:30:46.969+0000 7fec6ecfe640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 33643) UID: 0 2024-03-04T08:30:46.975 INFO:tasks.ceph.mds.b.smithi136.stderr:2024-03-04T08:30:46.969+0000 7fec6ecfe640 -1 mds.b *** got signal Terminated *** 2024-03-04T08:30:46.976 INFO:tasks.ceph.mds.d.smithi136.stderr:2024-03-04T08:30:46.969+0000 7f33396b0640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 33746) UID: 0 2024-03-04T08:30:46.976 INFO:tasks.ceph.mds.d.smithi136.stderr:2024-03-04T08:30:46.969+0000 7f33396b0640 -1 mds.d *** got signal Terminated *** 2024-03-04T08:30:46.976 INFO:tasks.ceph.mon.b.smithi136.stderr:2024-03-04T08:30:46.969+0000 7f39af1b0640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 27873) UID: 0 2024-03-04T08:30:46.976 INFO:tasks.ceph.mon.b.smithi136.stderr:2024-03-04T08:30:46.969+0000 7f39af1b0640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-03-04T08:30:46.977 INFO:tasks.ceph.mon.c.smithi136.stderr:2024-03-04T08:30:46.969+0000 7fc9ba3e1640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 27876) UID: 0 2024-03-04T08:30:46.977 INFO:tasks.ceph.mon.c.smithi136.stderr:2024-03-04T08:30:46.969+0000 7fc9ba3e1640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2024-03-04T08:30:47.168 INFO:tasks.ceph.mgr.y.smithi113.stderr:daemon-helper: command crashed with signal 15 2024-03-04T08:30:47.708 INFO:tasks.ceph:Scrubbing osd.0 2024-03-04T08:30:47.708 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-04T08:30:47.846 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8f5e00640 1 Processor -- start 2024-03-04T08:30:47.847 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8f5e00640 1 -- start start 2024-03-04T08:30:47.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8f5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 0x7fc8f01548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:47.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8f5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 0x7fc8f0154e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:47.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8f5e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8f00c8b80 2024-03-04T08:30:47.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8f5e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8f006b0d0 2024-03-04T08:30:47.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8f5e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8f00c8f60 2024-03-04T08:30:47.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 msgr2=0x7fc8f01548c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:30:47.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 0x7fc8f01548c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:30:47.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8f006df20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:30:47.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 msgr2=0x7fc8f0154e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:30:47.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:47.851+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 0x7fc8f0154e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:30:48.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:48.055+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 msgr2=0x7fc8f01548c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:30:48.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:48.055+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 0x7fc8f01548c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:30:48.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:48.055+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8f006df20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:30:48.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:48.055+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 msgr2=0x7fc8f0154e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:30:48.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:48.055+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 0x7fc8f0154e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:30:48.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:48.459+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 msgr2=0x7fc8f01548c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:30:48.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:48.459+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 0x7fc8f01548c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:30:48.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:48.459+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 msgr2=0x7fc8f0154e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:30:48.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:48.459+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8f006df20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:30:48.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:48.459+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 0x7fc8f0154e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:30:49.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:49.263+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 msgr2=0x7fc8f01548c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:30:49.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:49.263+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 0x7fc8f01548c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:30:49.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:49.263+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 msgr2=0x7fc8f0154e00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:30:49.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:49.263+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8f006df20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:30:49.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:49.263+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 0x7fc8f0154e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:30:50.848 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 msgr2=0x7fc8f0154e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:30:50.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8f00c8f60 0x7fc8f0154e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:50.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 msgr2=0x7fc8f01548c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:30:50.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8f006b0d0 0x7fc8f01548c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:50.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8f006df20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:30:50.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:50.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8d00036d0 2024-03-04T08:30:50.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d00073b0 2024-03-04T08:30:50.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d00036d0 legacy=0x7fc8d0005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:30:50.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:30:50.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:50.851+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:30:51.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:51.055+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d00036d0 legacy=0x7fc8d0005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:30:51.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:51.055+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:30:51.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:51.055+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:30:51.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:51.459+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d00036d0 legacy=0x7fc8d0005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:30:51.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:51.459+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:30:51.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:51.459+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:30:52.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:52.263+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d00036d0 legacy=0x7fc8d0005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:30:52.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:52.263+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:30:52.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:52.263+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:30:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:30:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d00073b0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:53.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d00036d0 legacy=0x7fc8d0005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:30:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:30:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d0009f40 2024-03-04T08:30:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8f00c8b80 2024-03-04T08:30:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000aac0 2024-03-04T08:30:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d0009f40 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:30:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:30:53.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:30:53.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:53.851+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:30:54.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:54.055+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:30:54.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:54.055+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d0009f40 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:30:54.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:54.055+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:30:54.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:54.055+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:30:54.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:54.459+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:30:54.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:54.459+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d0009f40 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:30:54.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:54.459+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:30:54.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:54.459+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:30:55.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:55.263+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:30:55.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:55.263+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d0009f40 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:30:55.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:55.263+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:30:55.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:55.263+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:30:56.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:30:56.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000aac0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:56.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:30:56.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d0009f40 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:30:56.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:56.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:56.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000e3e0 2024-03-04T08:30:56.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000ee60 2024-03-04T08:30:56.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 msgr2=0x7fc8d000f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:30:56.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 msgr2=0x7fc8d000e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:30:56.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:30:56.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:56.851+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:30:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:57.055+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 msgr2=0x7fc8d000f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:30:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:57.055+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 msgr2=0x7fc8d000e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:30:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:57.055+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:30:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:57.055+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:30:57.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:57.459+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 msgr2=0x7fc8d000f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:30:57.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:57.459+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 msgr2=0x7fc8d000e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:30:57.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:57.459+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:30:57.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:57.459+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:30:58.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:58.263+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 msgr2=0x7fc8d000f380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:30:58.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:58.263+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 msgr2=0x7fc8d000e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:30:58.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:58.263+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:30:58.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:58.263+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:30:59.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 msgr2=0x7fc8d000f380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:30:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 msgr2=0x7fc8d000e7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:30:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:30:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:30:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:30:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8f00c8b80 2024-03-04T08:30:59.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000e3e0 2024-03-04T08:30:59.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000ee60 2024-03-04T08:30:59.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:30:59.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 msgr2=0x7fc8d000f9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:30:59.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 msgr2=0x7fc8d000e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:30:59.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:30:59.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:30:59.855+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:00.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:00.059+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:00.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:00.059+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 msgr2=0x7fc8d000e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:00.059+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 msgr2=0x7fc8d000f9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:00.059+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:00.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:00.059+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:00.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:00.463+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 msgr2=0x7fc8d000e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:00.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:00.463+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:00.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:00.463+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 msgr2=0x7fc8d000f9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:00.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:00.463+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:00.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:00.463+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:01.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:01.267+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:01.267+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 msgr2=0x7fc8d000e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:01.267+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 msgr2=0x7fc8d000f9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:01.267+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:01.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:01.267+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:02.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 msgr2=0x7fc8d000f9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:02.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000ee60 0x7fc8d000f9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 msgr2=0x7fc8d000e7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e3e0 0x7fc8d000e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000f4e0 2024-03-04T08:31:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:31:02.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:02.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:02.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:02.851+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:03.055+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:03.055+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:03.055+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:03.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:03.459+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:03.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:03.459+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:03.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:03.459+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:04.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:04.263+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:04.258 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:04.263+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:04.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:04.263+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:05.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:05.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:05.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:05.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:05.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:31:05.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000f4e0 2024-03-04T08:31:05.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8f00c8b80 2024-03-04T08:31:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:31:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:05.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:05.855+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:06.059+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:06.059+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:06.059+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:06.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:06.059+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:06.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:06.463+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:06.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:06.463+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:06.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:06.463+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:06.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:06.463+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:07.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:07.267+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:07.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:07.267+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:07.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:07.267+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:07.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:07.267+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:08.850 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8f00c8b80 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d000e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000d310 2024-03-04T08:31:08.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000f9c0 2024-03-04T08:31:08.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:08.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d000e7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:08.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:08.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:08.855+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d000e7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:09.059+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:09.059+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d000e7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:09.059+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:09.059+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d000e7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:09.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:09.463+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:09.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:09.463+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d000e7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:09.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:09.463+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:09.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:09.463+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d000e7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:10.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:10.267+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:10.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:10.267+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d000e7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:10.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:10.267+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:10.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:10.267+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d000e7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:11.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d000e7c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:11.851 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d000e7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:11.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:11.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000d310 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:11.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:31:11.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d0017620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:11.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:11.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8d000f4e0 2024-03-04T08:31:11.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:31:11.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000dc60 2024-03-04T08:31:11.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:11.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:11.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:11.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d0017620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:11.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:11.855+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d0017620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:12.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:12.059+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:12.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:12.059+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:12.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:12.059+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:12.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:12.059+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d0017620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:12.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:12.059+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d0017620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:12.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:12.463+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:12.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:12.463+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:12.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:12.463+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:12.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:12.463+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d0017620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:12.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:12.463+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d0017620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:13.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:13.267+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:13.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:13.267+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:13.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:13.267+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:13.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:13.267+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d0017620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:13.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:13.267+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d0017620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:14.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d0017620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:14.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d0017620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:14.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:14.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:14.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:14.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:14.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000e840 2024-03-04T08:31:14.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f9c0 2024-03-04T08:31:14.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:14.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:14.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:14.855+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:15.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:15.059+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:15.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:15.059+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:15.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:15.059+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:15.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:15.463+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:15.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:15.463+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:15.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:15.463+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:16.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:16.267+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:16.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:16.267+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:16.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:16.267+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:16.585 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:31:16.589 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:31:17.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:17.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:17.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:17.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:31:17.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:17.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:17.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8d000e840 2024-03-04T08:31:17.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:31:17.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000f9c0 2024-03-04T08:31:17.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:17.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:17.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d00063d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:17.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:17.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:17.855+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:18.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:18.059+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:18.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:18.059+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:18.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:18.059+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:18.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:18.059+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d00063d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:18.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:18.059+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:18.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:18.463+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:18.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:18.463+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:18.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:18.463+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:18.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:18.463+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d00063d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:18.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:18.463+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:19.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:19.267+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:19.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:19.267+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:19.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:19.267+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:19.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:19.267+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d00063d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:19.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:19.267+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:20.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:20.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:20.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:20.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:20.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d00063d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:20.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:20.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000dc60 2024-03-04T08:31:20.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f4e0 2024-03-04T08:31:20.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:20.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:20.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:20.855+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:21.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:21.059+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:21.059+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:21.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:21.059+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:21.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:21.463+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:21.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:21.463+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:21.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:21.463+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:22.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:22.267+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:22.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:22.267+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:22.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:22.267+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:23.852 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:31:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8d000dc60 2024-03-04T08:31:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000e840 2024-03-04T08:31:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f9c0 2024-03-04T08:31:23.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:23.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:23.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d00063d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:23.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:23.855+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:24.059+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:24.059+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d00063d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:24.059+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:24.059+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:24.463+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:24.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:24.463+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d00063d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:24.463+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:24.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:24.463+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:25.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:25.267+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:25.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:25.267+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d00063d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:25.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:25.267+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:25.492 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:25.267+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:26.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0007860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:26.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0007860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:26.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d00063d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:26.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:26.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:26.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:26.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f4e0 2024-03-04T08:31:26.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000ff60 2024-03-04T08:31:26.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:26.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:26.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:26.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:26.855+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:27.059+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:27.059+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:27.059+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:27.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:27.059+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:27.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:27.463+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:27.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:27.463+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:27.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:27.463+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:27.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:27.463+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:28.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:28.267+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:28.262 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:28.267+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:28.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:28.267+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:28.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:28.267+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:29.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:29.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:29.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:29.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:29.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:29.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:31:29.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:29.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000dc60 2024-03-04T08:31:29.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f4e0 2024-03-04T08:31:29.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000ff60 2024-03-04T08:31:29.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:29.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:29.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:29.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:29.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:29.859+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:30.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:30.063+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:30.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:30.063+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:30.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:30.063+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:30.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:30.063+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:30.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:30.063+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:30.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:30.467+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:30.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:30.467+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:30.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:30.467+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:30.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:30.467+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:30.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:30.467+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:31.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:31.271+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:31.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:31.271+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:31.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:31.271+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:31.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:31.271+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:31.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:31.271+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:32.853 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d00063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:32.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8d000e840 2024-03-04T08:31:32.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f9c0 2024-03-04T08:31:32.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:32.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:32.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:32.859+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d00063a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:33.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:33.063+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:33.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:33.063+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:33.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:33.063+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d00063a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:33.466+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:33.466+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:33.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:33.466+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d00063a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:34.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:34.270+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:34.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:34.270+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:34.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:34.270+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d00063a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:35.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:35.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d00063a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:35.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:35.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:31:35.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:35.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000dc60 2024-03-04T08:31:35.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8d000e840 2024-03-04T08:31:35.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f9c0 2024-03-04T08:31:35.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:35.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:35.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:35.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:35.858+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:36.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:36.062+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:36.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:36.062+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:36.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:36.062+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:36.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:36.062+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:36.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:36.466+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:36.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:36.466+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:36.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:36.466+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:36.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:36.466+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:37.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:37.270+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:37.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:37.270+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:37.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:37.270+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:37.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:37.270+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:38.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.854+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:38.854 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.854+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:38.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.854+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:38.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.854+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:38.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.854+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:38.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.854+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:38.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.854+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:31:38.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.854+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000f4e0 2024-03-04T08:31:38.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.854+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:38.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.858+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:38.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.858+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:38.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:38.858+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:39.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:39.062+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:39.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:39.062+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:39.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:39.062+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:39.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:39.062+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:39.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:39.466+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:39.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:39.466+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:39.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:39.466+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:39.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:39.466+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:40.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:40.270+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:40.270+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:40.270+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:40.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:40.270+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:41.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:41.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:41.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:41.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:41.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:41.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:31:41.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:41.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000e840 2024-03-04T08:31:41.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:31:41.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000dc60 2024-03-04T08:31:41.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:41.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:41.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:41.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8ef400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:41.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:41.858+0000 7fc8ef400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:42.062+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:42.062+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:42.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:42.062+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:42.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:42.062+0000 7fc8ef400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:42.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:42.062+0000 7fc8ef400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:42.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:42.466+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:42.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:42.466+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:42.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:42.466+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:42.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:42.466+0000 7fc8ef400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:42.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:42.466+0000 7fc8ef400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:43.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:43.270+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:43.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:43.270+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:43.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:43.270+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:43.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:43.270+0000 7fc8ef400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:43.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:43.270+0000 7fc8ef400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:44.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:44.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000dc60 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:44.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:44.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:44.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:44.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:44.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000f9c0 2024-03-04T08:31:44.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f4e0 2024-03-04T08:31:44.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:44.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:44.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:44.858+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:45.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:45.062+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:45.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:45.062+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:45.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:45.062+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:45.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:45.466+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:45.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:45.466+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:45.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:45.466+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:46.270+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:46.270+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a380 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:46.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:46.270+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:46.665 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:31:46.669 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:31:47.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:47.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:47.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d00063a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:47.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:47.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:31:47.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:47.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000e840 2024-03-04T08:31:47.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:31:47.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000f4e0 2024-03-04T08:31:47.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:47.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:47.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:47.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:47.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:47.858+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:48.062+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:48.062+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:48.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:48.062+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:48.062+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:48.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:48.062+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:48.466+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:48.466+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:48.466+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:48.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:48.466+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:48.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:48.466+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:49.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:49.270+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:49.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:49.270+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:49.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:49.270+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:49.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:49.270+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:49.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:49.270+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:50.855 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:50.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f4e0 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:50.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:50.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:50.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:50.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:50.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000dc60 2024-03-04T08:31:50.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f9c0 2024-03-04T08:31:50.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:50.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:50.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:50.858+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:51.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:51.062+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:51.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:51.062+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:51.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:51.062+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:51.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:51.466+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:51.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:51.466+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:51.462 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:51.466+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:52.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:52.270+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006b20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:52.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:52.270+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:52.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:52.270+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:53.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:53.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:53.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:31:53.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:53.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000e840 2024-03-04T08:31:53.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000dc60 2024-03-04T08:31:53.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000f9c0 2024-03-04T08:31:53.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:53.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:53.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:53.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:53.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:53.858+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:54.062+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:54.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:54.062+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:54.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:54.062+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:54.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:54.062+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:54.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:54.062+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:54.463 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:54.466+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:54.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:54.466+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:54.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:54.466+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:54.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:54.466+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:54.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:54.466+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:55.270+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:31:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:55.270+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:55.266 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:55.270+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:31:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:55.270+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:55.267 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:55.270+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:56.856 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:56.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000f9c0 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:56.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:56.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:56.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:56.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:56.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000f4e0 2024-03-04T08:31:56.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:31:56.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:56.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:56.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:56.862+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:57.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:57.066+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:57.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:57.066+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:31:57.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:57.066+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:57.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:57.470+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:57.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:57.470+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:31:57.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:57.470+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:58.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:58.274+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:58.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:58.274+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:31:58.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:58.274+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:59.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:59.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:31:59.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000f4e0 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:31:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:31:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:31:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d0006580 2024-03-04T08:31:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:31:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000dc60 2024-03-04T08:31:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:31:59.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:31:59.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d0006580 legacy=0x7fc8d00185c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:31:59.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:31:59.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:31:59.862+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:00.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:00.066+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:00.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:00.066+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d0006580 legacy=0x7fc8d00185c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:00.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:00.066+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:00.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:00.066+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:00.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:00.066+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:00.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:00.470+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:00.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:00.470+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:00.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:00.470+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d0006580 legacy=0x7fc8d00185c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:00.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:00.470+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:00.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:00.470+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:01.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:01.274+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:01.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:01.274+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:01.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:01.274+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d0006580 legacy=0x7fc8d00185c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:01.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:01.274+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:01.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:01.274+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:02.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:02.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:02.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:02.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:02.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d0006580 legacy=0x7fc8d00185c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:02.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:02.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000f9c0 2024-03-04T08:32:02.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000e840 2024-03-04T08:32:02.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:02.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:02.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:02.862+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:03.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:03.066+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:03.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:03.066+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:03.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:03.066+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:03.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:03.470+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:03.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:03.470+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:03.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:03.470+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:04.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:04.274+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:04.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:04.274+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:04.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:04.274+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:05.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:05.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:05.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:05.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:05.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:32:05.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:05.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000f9c0 2024-03-04T08:32:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:32:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000dc60 2024-03-04T08:32:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:05.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:05.862+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:06.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:06.066+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:06.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:06.066+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:06.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:06.066+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:06.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:06.066+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:06.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:06.066+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:06.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:06.470+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:06.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:06.470+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:06.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:06.470+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:06.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:06.470+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:06.467 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:06.470+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:07.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:07.274+0000 7fc8ef400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:07.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:07.274+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:07.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:07.274+0000 7fc8ef400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:07.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:07.274+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:07.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:07.274+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:08.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:08.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000dc60 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:08.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:08.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:08.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000e840 2024-03-04T08:32:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d0006580 2024-03-04T08:32:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:08.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:08.862+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:09.066+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:09.066+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:09.066+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:09.470+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:09.470+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:09.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:09.470+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:10.274+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:10.274+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:10.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:10.274+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:11.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:11.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:11.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:11.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:11.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:11.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:32:11.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000dc60 2024-03-04T08:32:11.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:32:11.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d0006580 2024-03-04T08:32:11.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:11.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:11.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:11.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:11.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:11.862+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:12.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:12.066+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:12.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:12.066+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:12.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:12.066+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:12.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:12.066+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:12.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:12.066+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:12.470+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:12.470+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:12.470+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:12.470+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:12.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:12.470+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:13.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:13.274+0000 7fc8eea00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:13.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:13.274+0000 7fc8eea00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:13.274+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:13.274+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:13.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:13.274+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:14.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:14.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:14.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:14.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:14.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:14.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:14.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000f9c0 2024-03-04T08:32:14.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000e840 2024-03-04T08:32:14.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:14.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:14.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:14.862+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:15.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:15.066+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:15.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:15.066+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:15.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:15.066+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:15.470+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:15.470+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:15.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:15.470+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:16.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:16.274+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:16.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:16.274+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:16.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:16.274+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:16.802 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:32:16.806 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:32:17.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:17.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:17.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000f9c0 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:17.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:17.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:32:17.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000e840 2024-03-04T08:32:17.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000dc60 2024-03-04T08:32:17.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d0006580 2024-03-04T08:32:17.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:17.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:17.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:17.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:17.862+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:18.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:18.066+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:18.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:18.066+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:18.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:18.066+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:18.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:18.066+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:18.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:18.470+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:18.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:18.470+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:18.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:18.470+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:18.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:18.470+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:19.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:19.274+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:19.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:19.274+0000 7fc8ee000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:19.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:19.274+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:19.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:19.274+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:20.859 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000ff60 2024-03-04T08:32:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000f9c0 2024-03-04T08:32:20.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:20.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:20.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:20.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:20.862+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:21.066+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:21.066+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:21.066+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:21.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:21.066+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:21.470+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:21.470+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:21.470+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:21.466 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:21.470+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:22.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:22.274+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:22.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:22.274+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:22.270 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:22.274+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:22.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:22.274+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:23.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:23.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:23.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 msgr2=0x7fc8d000a360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000ff60 0x7fc8d000a360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:32:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8d000e840 2024-03-04T08:32:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f9c0 2024-03-04T08:32:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d0006580 2024-03-04T08:32:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:23.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:23.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:23.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:23.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:23.866+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:24.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:24.069+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:24.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:24.069+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:24.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:24.069+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:24.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:24.069+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:24.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:24.069+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:24.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:24.473+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:24.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:24.473+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:24.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:24.473+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:24.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:24.473+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:24.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:24.473+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:25.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:25.277+0000 7fc8ef400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:25.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:25.277+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:25.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:25.277+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:25.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:25.277+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:25.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:25.277+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:26.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000e840 legacy=0x7fc8d0005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 0x7fc8d00165b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000ff60 2024-03-04T08:32:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000dc60 2024-03-04T08:32:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d00165b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:26.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 0x7fc8d00165b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:26.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:26.865+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000ff60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:27.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:27.069+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d00165b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:27.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:27.069+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 0x7fc8d00165b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:27.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:27.069+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000ff60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:27.473+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d00165b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:27.473+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 0x7fc8d00165b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:27.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:27.473+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000ff60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:28.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:28.277+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d00165b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:28.277+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 0x7fc8d00165b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:28.275 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:28.277+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000ff60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:29.860 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 msgr2=0x7fc8d00165b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:29.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000dc60 0x7fc8d00165b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:29.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000ff60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:29.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:32:29.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:29.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:29.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8d000a450 2024-03-04T08:32:29.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f9c0 2024-03-04T08:32:29.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d0006580 2024-03-04T08:32:29.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0018b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:29.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:29.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:29.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:29.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:29.865+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:30.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:30.069+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:30.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:30.069+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0018b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:30.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:30.069+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:30.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:30.069+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:30.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:30.069+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:30.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:30.473+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:30.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:30.473+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:30.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:30.473+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0018b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:30.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:30.473+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:30.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:30.473+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:31.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:31.277+0000 7fc8ef400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:31.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:31.277+0000 7fc8eea00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0018b00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:31.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:31.277+0000 7fc8ee000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:31.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:31.277+0000 7fc8ef400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:31.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:31.277+0000 7fc8ee000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:32.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:32.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:32.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:32.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000f9c0 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:32.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0018b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:32.861 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 0x7fc8d0004af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:32.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000dc60 2024-03-04T08:32:32.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000e840 2024-03-04T08:32:32.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:32.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:32.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:32.865+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 0x7fc8d0004af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:33.069+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:33.069+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:33.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:33.069+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 0x7fc8d0004af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:33.473+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:33.473+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:33.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:33.473+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 0x7fc8d0004af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:34.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:34.277+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:34.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:34.277+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:34.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:34.277+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 0x7fc8d0004af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:35.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:35.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000e840 0x7fc8d0004af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:35.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:35.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:35.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:32:35.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8d000dc60 2024-03-04T08:32:35.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d00103b0 con 0x7fc8d000a450 2024-03-04T08:32:35.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d0006580 2024-03-04T08:32:35.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:35.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:35.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:35.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:35.865+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:36.069+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:36.069+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:36.069+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:36.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:36.069+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:36.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:36.473+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:36.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:36.473+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:36.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:36.473+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:36.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:36.473+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:37.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:37.277+0000 7fc8ee000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:37.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:37.277+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:37.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:37.277+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:37.274 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:37.277+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:38.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8e7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:38.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d0006580 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:38.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:38.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:38.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:38.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:38.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f9c0 2024-03-04T08:32:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000e840 2024-03-04T08:32:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d0005200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:38.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:38.865+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:39.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:39.069+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d0005200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:39.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:39.069+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:39.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:39.069+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:39.067 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:39.069+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:39.473+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d0005200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:39.473+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:39.473+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:39.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:39.473+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:40.277+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d0005200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:40.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:40.277+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:40.277+0000 7fc8eea00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:40.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:40.277+0000 7fc8eea00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:41.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8e7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:41.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d000e840 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:41.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d0005200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:41.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:41.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8ef400640 1 -- reap_dead start 2024-03-04T08:32:41.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:41.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8e7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f015ba40 con 0x7fc8d000dc60 2024-03-04T08:32:41.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8e7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f006baa0 con 0x7fc8d000a450 2024-03-04T08:32:41.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8e7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d000f9c0 2024-03-04T08:32:41.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:41.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d0005200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:41.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:41.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:41.865+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:42.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:42.069+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:42.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:42.069+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d0005200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:42.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:42.069+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:42.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:42.069+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:42.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:42.473+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:42.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:42.473+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d0005200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:42.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:42.473+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:42.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:42.473+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:43.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:43.277+0000 7fc8eea00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:43.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:43.277+0000 7fc8ee000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d0005200 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:43.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:43.277+0000 7fc8ee000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:43.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:43.277+0000 7fc8ef400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:44.862 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8e7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 msgr2=0x7fc8d0005200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8d000f9c0 0x7fc8d0005200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8e7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fc8d000a450 legacy=0x7fc8d0004af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8e7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fc8d000dc60 legacy=0x7fc8d0006a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8e7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8e7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d0006580 0x7fc8d000d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8e7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f006e630 con 0x7fc8d0006580 2024-03-04T08:32:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8e7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f015b760 con 0x7fc8d000e840 2024-03-04T08:32:44.863 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d0006580 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:44.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:44.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d0006580 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:44.864 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:44.865+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:45.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:45.069+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d0006580 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:45.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:45.069+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:45.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:45.069+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d0006580 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:45.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:45.069+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:45.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:45.473+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d0006580 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:45.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:45.473+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:45.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:45.473+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d0006580 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:45.470 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:45.473+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:46.277+0000 7fc8ee000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fc8d0006580 msgr2=0x7fc8d000d6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:46.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:46.277+0000 7fc8eea00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 msgr2=0x7fc8d0009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:46.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:46.277+0000 7fc8ee000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fc8d0006580 0x7fc8d000d6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:46.272 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:46.277+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fc8d000e840 0x7fc8d0009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:46.929 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:32:46.933 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:32:47.734 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-03-04T08:32:47.735 INFO:tasks.ceph:Scrubbing osd.1 2024-03-04T08:32:47.736 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-04T08:32:47.882 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdd2e00640 1 Processor -- start 2024-03-04T08:32:47.883 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdd2e00640 1 -- start start 2024-03-04T08:32:47.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdd2e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 0x7fbdcc072b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:47.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdd2e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdcc0730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:47.884 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdd2e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 0x7fbdcc15b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:47.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdd2e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdcc0c2e00 2024-03-04T08:32:47.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdd2e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdcc0c18e0 2024-03-04T08:32:47.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdd2e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2360 con 0x7fbdcc0c1500 2024-03-04T08:32:47.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 msgr2=0x7fbdcc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:47.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 0x7fbdcc15b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:47.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 msgr2=0x7fbdcc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:47.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 msgr2=0x7fbdcc0730d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:47.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 0x7fbdcc072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:47.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:47.885+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdcc0730d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.089+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 msgr2=0x7fbdcc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.089+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 0x7fbdcc15b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.089+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 msgr2=0x7fbdcc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.089+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 msgr2=0x7fbdcc0730d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.089+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 0x7fbdcc072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:48.085 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.089+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdcc0730d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:48.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.493+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 msgr2=0x7fbdcc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:48.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.493+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 0x7fbdcc15b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:48.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.493+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 msgr2=0x7fbdcc0730d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:48.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.493+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 msgr2=0x7fbdcc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:48.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.493+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdcc0730d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:48.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:48.493+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 0x7fbdcc072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:49.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:49.297+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 msgr2=0x7fbdcc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:49.297+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 0x7fbdcc15b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:49.297+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 msgr2=0x7fbdcc0730d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:49.297+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 msgr2=0x7fbdcc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:49.297+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdcc0730d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:49.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:49.297+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 0x7fbdcc072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:50.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:50.889+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 msgr2=0x7fbdcc072b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:50.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:50.889+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdcc0c1500 0x7fbdcc072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:50.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:50.889+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 msgr2=0x7fbdcc0730d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:50.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:50.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdcc0730d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:50.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:50.889+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 msgr2=0x7fbdcc15b290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:50.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:50.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdcc0c2e00 0x7fbdcc15b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:50.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:50.889+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac009580 2024-03-04T08:32:50.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:50.889+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac0058d0 2024-03-04T08:32:50.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:50.889+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac009580 legacy=0x7fbdac00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:50.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:50.889+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:51.093+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac009580 legacy=0x7fbdac00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:51.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:51.093+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:51.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:51.497+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac009580 legacy=0x7fbdac00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:51.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:51.497+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:52.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:52.301+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac009580 legacy=0x7fbdac00ba40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:52.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:52.301+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:53.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:53.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac009580 legacy=0x7fbdac00ba40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:53.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:32:53.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:53.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00f5a0 con 0x7fbdac00e520 2024-03-04T08:32:53.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac0058d0 2024-03-04T08:32:53.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdcc0c18e0 2024-03-04T08:32:53.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:53.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:53.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:53.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:53.889+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:54.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:54.093+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:54.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:54.093+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:54.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:54.093+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:54.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:54.093+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:54.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:54.497+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:54.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:54.497+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:54.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:54.497+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:54.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:54.497+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:55.301+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:55.301+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:55.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:55.301+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:32:55.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:55.301+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:56.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:56.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdcc0c18e0 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:56.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:56.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:56.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 0x7fbdac00c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:56.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 0x7fbdac015900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:56.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00cb90 2024-03-04T08:32:56.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac00c210 2024-03-04T08:32:56.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 msgr2=0x7fbdac015900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:56.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 0x7fbdac015900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:56.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 msgr2=0x7fbdac00c650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:56.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:56.889+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 0x7fbdac00c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:32:57.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:57.093+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 msgr2=0x7fbdac015900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:57.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:57.093+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 0x7fbdac015900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:57.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:57.093+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 msgr2=0x7fbdac00c650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:57.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:57.093+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 0x7fbdac00c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:32:57.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:57.497+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 msgr2=0x7fbdac015900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:57.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:57.497+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 0x7fbdac015900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:57.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:57.497+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 msgr2=0x7fbdac00c650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:57.491 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:57.497+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 0x7fbdac00c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:32:58.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:58.301+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 msgr2=0x7fbdac015900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:32:58.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:58.301+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 0x7fbdac015900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:58.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:58.301+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 msgr2=0x7fbdac00c650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:58.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:58.301+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 0x7fbdac00c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:32:59.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 msgr2=0x7fbdac00c650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:59.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00c210 0x7fbdac00c650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:59.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 msgr2=0x7fbdac015900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:32:59.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00cb90 0x7fbdac015900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:32:59.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:32:59.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac018230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:32:59.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac00e520 2024-03-04T08:32:59.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00f5a0 con 0x7fbdac0058d0 2024-03-04T08:32:59.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac015e40 2024-03-04T08:32:59.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:32:59.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 msgr2=0x7fbdac018230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:32:59.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:32:59.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:32:59.889+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac018230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:00.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:00.093+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:00.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:00.093+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:00.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:00.093+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 msgr2=0x7fbdac018230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:00.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:00.093+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac018230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:00.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:00.497+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:00.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:00.497+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 msgr2=0x7fbdac018230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:00.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:00.497+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:00.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:00.497+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac018230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:01.301+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:01.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:01.301+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:01.301+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 msgr2=0x7fbdac018230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:01.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:01.301+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac018230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:02.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 msgr2=0x7fbdac018230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:02.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac018230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:02.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac0058d0 legacy=0x7fbdac007d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:02.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:02.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:02.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 0x7fbdac013d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:02.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac0118e0 2024-03-04T08:33:02.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac00c650 2024-03-04T08:33:02.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 msgr2=0x7fbdac013d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:02.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 0x7fbdac013d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:02.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:02.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:02.889+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:03.086 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:03.093+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 msgr2=0x7fbdac013d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:03.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:03.093+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 0x7fbdac013d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:03.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:03.093+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:03.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:03.093+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:03.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:03.497+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 msgr2=0x7fbdac013d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:03.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:03.497+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 0x7fbdac013d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:03.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:03.497+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:03.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:03.497+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:04.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:04.301+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 msgr2=0x7fbdac013d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:04.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:04.301+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 0x7fbdac013d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:04.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:04.301+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:04.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:04.301+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:05.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:05.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:05.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 msgr2=0x7fbdac013d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:05.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac0118e0 0x7fbdac013d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:05.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac014240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:05.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:33:05.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:05.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac00e520 2024-03-04T08:33:05.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00c650 2024-03-04T08:33:05.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac015e40 2024-03-04T08:33:05.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:05.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 msgr2=0x7fbdac014240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:05.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:05.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac014240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:05.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:05.889+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:06.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:06.093+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:06.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:06.093+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 msgr2=0x7fbdac014240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:06.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:06.093+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:06.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:06.093+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac014240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:06.088 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:06.093+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:06.497+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:06.497+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:06.497+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 msgr2=0x7fbdac014240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:06.497+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac014240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:06.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:06.497+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:07.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:07.301+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:07.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:07.301+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:07.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:07.301+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 msgr2=0x7fbdac014240 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:07.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:07.301+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac014240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:07.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:07.301+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:08.886 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 msgr2=0x7fbdac014240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:08.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015e40 0x7fbdac014240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:08.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:08.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:08.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:08.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 0x7fbdac013d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:08.887 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac008480 2024-03-04T08:33:08.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac009030 2024-03-04T08:33:08.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 msgr2=0x7fbdac013d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:08.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 0x7fbdac013d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:08.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:08.889+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008480 legacy=0x7fbdac008860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:09.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:09.093+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 msgr2=0x7fbdac013d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:09.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:09.093+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 0x7fbdac013d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:09.087 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:09.093+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008480 legacy=0x7fbdac008860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:09.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:09.497+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 msgr2=0x7fbdac013d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:09.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:09.497+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 0x7fbdac013d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:09.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:09.497+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008480 legacy=0x7fbdac008860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:10.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:10.301+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 msgr2=0x7fbdac013d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:10.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:10.301+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 0x7fbdac013d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:10.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:10.301+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008480 legacy=0x7fbdac008860 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:11.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 msgr2=0x7fbdac013d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:11.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac009030 0x7fbdac013d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008480 legacy=0x7fbdac008860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:33:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac00e520 2024-03-04T08:33:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac00c650 2024-03-04T08:33:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac009030 2024-03-04T08:33:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:11.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00c650 legacy=0x7fbdac007d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:11.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:11.893+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:12.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:12.097+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:12.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:12.097+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00c650 legacy=0x7fbdac007d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:12.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:12.097+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:12.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:12.097+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:12.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:12.501+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:12.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:12.501+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:12.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:12.501+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00c650 legacy=0x7fbdac007d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:12.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:12.501+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:13.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:13.305+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:13.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:13.305+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00c650 legacy=0x7fbdac007d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:13.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:13.305+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:13.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:13.305+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:14.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:14.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:14.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00c650 legacy=0x7fbdac007d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:14.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:14.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:14.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 0x7fbdac008460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:14.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac015830 2024-03-04T08:33:14.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac015250 2024-03-04T08:33:14.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:14.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:14.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 msgr2=0x7fbdac008460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:14.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:14.893+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 0x7fbdac008460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:15.097+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:15.097+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:15.097+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 msgr2=0x7fbdac008460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:15.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:15.097+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 0x7fbdac008460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:15.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:15.501+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:15.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:15.501+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:15.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:15.501+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 msgr2=0x7fbdac008460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:15.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:15.501+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 0x7fbdac008460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:16.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:16.304+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:16.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:16.304+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:16.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:16.304+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 msgr2=0x7fbdac008460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:16.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:16.304+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 0x7fbdac008460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:17.031 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:33:17.035 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:33:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 msgr2=0x7fbdac008460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015250 0x7fbdac008460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:17.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:33:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:17.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00f5a0 con 0x7fbdac00e520 2024-03-04T08:33:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac015830 2024-03-04T08:33:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac009030 2024-03-04T08:33:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:17.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdd0e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:17.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:17.892+0000 7fbdd0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:18.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:18.096+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:18.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:18.096+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:18.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:18.096+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:18.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:18.096+0000 7fbdd0e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:18.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:18.096+0000 7fbdd0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:18.500+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:18.500+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:18.500+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:18.500+0000 7fbdd0e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:18.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:18.500+0000 7fbdd0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:19.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:19.304+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:19.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:19.304+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:19.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:19.304+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:19.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:19.304+0000 7fbdd0e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:19.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:19.304+0000 7fbdd0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:20.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac011cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:20.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac015250 2024-03-04T08:33:20.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00c650 2024-03-04T08:33:20.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac011cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:20.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac015250 legacy=0x7fbdac012230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:20.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:20.892+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac011cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:21.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:21.096+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac015250 legacy=0x7fbdac012230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:21.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:21.096+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac011cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:21.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:21.096+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac011cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:21.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:21.500+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac015250 legacy=0x7fbdac012230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:21.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:21.500+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac011cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:21.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:21.500+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac011cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:22.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:22.304+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac011cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:22.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:22.304+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac015250 legacy=0x7fbdac012230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:22.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:22.304+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac011cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:23.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac011cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:23.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac011cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:23.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac015250 legacy=0x7fbdac012230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:23.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:23.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:33:23.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:23.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00f5a0 con 0x7fbdac00e520 2024-03-04T08:33:23.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac015830 2024-03-04T08:33:23.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac009030 2024-03-04T08:33:23.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:23.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:23.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:23.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdd0e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:23.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:23.892+0000 7fbdd0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:24.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:24.096+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:24.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:24.096+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:24.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:24.096+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:24.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:24.096+0000 7fbdd0e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:24.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:24.096+0000 7fbdd0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:24.500+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:24.500+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:24.500+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:24.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:24.500+0000 7fbdd0e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:24.496 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:24.500+0000 7fbdd0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:25.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:25.304+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:25.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:25.304+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014950 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:25.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:25.304+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:25.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:25.304+0000 7fbdd0e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:25.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:25.304+0000 7fbdd0e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:26.888 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:26.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:26.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 msgr2=0x7fbdac014950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:26.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac015830 0x7fbdac014950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:26.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00e520 legacy=0x7fbdac00e900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:26.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac014e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:26.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac011cf0 2024-03-04T08:33:26.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00c650 2024-03-04T08:33:26.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac014e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:26.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:26.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:26.892+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac014e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:27.096+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:27.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:27.096+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac014e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:27.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:27.096+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac014e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:27.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:27.500+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac014e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:27.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:27.500+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:27.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:27.500+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac014e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:28.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:28.304+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:28.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:28.304+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac014e90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:28.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:28.304+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac014e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:29.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 msgr2=0x7fbdac014e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:29.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00c650 0x7fbdac014e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:29.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:29.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 0x7fbdac0080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:33:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00f5a0 con 0x7fbdac011cf0 2024-03-04T08:33:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac005d00 2024-03-04T08:33:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac009030 2024-03-04T08:33:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:29.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 msgr2=0x7fbdac0080f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:29.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:29.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:29.892+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 0x7fbdac0080f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:30.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:30.096+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:30.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:30.096+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 msgr2=0x7fbdac0080f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:30.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:30.096+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:30.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:30.096+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:30.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:30.096+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 0x7fbdac0080f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:30.500+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:30.500+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 msgr2=0x7fbdac0080f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:30.500+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:30.500+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:30.495 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:30.500+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 0x7fbdac0080f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:31.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:31.304+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 msgr2=0x7fbdac0080f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:31.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:31.304+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:31.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:31.304+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:31.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:31.304+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:31.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:31.304+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 0x7fbdac0080f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:32.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:32.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac009030 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:32.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 msgr2=0x7fbdac0080f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:32.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac005d00 0x7fbdac0080f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:32.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:32.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 0x7fbdac00e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:32.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac015830 2024-03-04T08:33:32.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00e520 2024-03-04T08:33:32.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 msgr2=0x7fbdac00e9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:32.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac015830 legacy=0x7fbdac008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:32.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:32.892+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 0x7fbdac00e9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:33.096+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac015830 legacy=0x7fbdac008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:33.096+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 msgr2=0x7fbdac00e9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:33.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:33.096+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 0x7fbdac00e9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:33.500+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 msgr2=0x7fbdac00e9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:33.500+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac015830 legacy=0x7fbdac008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:33.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:33.500+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 0x7fbdac00e9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:34.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:34.304+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac015830 legacy=0x7fbdac008630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:34.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:34.304+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 msgr2=0x7fbdac00e9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:34.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:34.304+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 0x7fbdac00e9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:35.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 msgr2=0x7fbdac00e9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:35.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00e520 0x7fbdac00e9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:35.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac015830 legacy=0x7fbdac008630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:35.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 0x7fbdac014d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:35.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:33:35.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00f5a0 con 0x7fbdac009030 2024-03-04T08:33:35.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac011cf0 2024-03-04T08:33:35.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac012940 2024-03-04T08:33:35.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 msgr2=0x7fbdac014d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:35.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 0x7fbdac014d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:35.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac009030 legacy=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:35.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:35.892+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:36.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:36.096+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac009030 legacy=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:36.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:36.096+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 msgr2=0x7fbdac014d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:36.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:36.096+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 0x7fbdac014d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:36.090 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:36.096+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:36.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:36.500+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac009030 legacy=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:36.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:36.500+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 msgr2=0x7fbdac014d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:36.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:36.500+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 0x7fbdac014d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:36.494 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:36.500+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:37.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:37.304+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac009030 legacy=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:37.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:37.304+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 msgr2=0x7fbdac014d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:37.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:37.304+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 0x7fbdac014d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:37.298 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:37.304+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:38.889 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.892+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 msgr2=0x7fbdac014d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac012940 0x7fbdac014d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.892+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac011cf0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.892+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac009030 legacy=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.892+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.892+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:38.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.892+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac015790 2024-03-04T08:33:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.892+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac00e950 2024-03-04T08:33:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.896+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.896+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac00d990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.896+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:38.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:38.896+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:39.100+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:39.100+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac00d990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:39.100+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:39.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:39.100+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:39.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:39.504+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:39.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:39.504+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac00d990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:39.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:39.504+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:39.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:39.504+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:40.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:40.308+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac00d990 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:40.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:40.308+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:40.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:40.308+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:40.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:40.308+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:41.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac00d990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:41.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:41.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:41.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:41.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:41.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:33:41.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac012170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:41.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:41.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac011cc0 2024-03-04T08:33:41.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac015790 2024-03-04T08:33:41.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2360 con 0x7fbdac00e950 2024-03-04T08:33:41.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:41.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:41.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:41.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:41.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:41.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:41.896+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac012170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.100+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.100+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.100+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.100+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.100+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac012170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:42.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.100+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.504+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.504+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.504+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.504+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.504+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac012170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:42.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:42.504+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:43.308+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:43.308+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:43.308+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:43.308+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:43.308+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac012170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:43.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:43.308+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:44.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:44.896+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:44.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:44.896+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac012170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:44.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:44.896+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:44.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:44.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:44.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:44.896+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:44.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:44.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:44.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:44.896+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac00d7c0 2024-03-04T08:33:44.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:44.896+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00f5a0 con 0x7fbdac015040 2024-03-04T08:33:44.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:44.896+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac014930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:44.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:44.896+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac015040 legacy=0x7fbdac005ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:45.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:45.100+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac014930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:45.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:45.100+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac015040 legacy=0x7fbdac005ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:45.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:45.504+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac014930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:45.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:45.504+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac015040 legacy=0x7fbdac005ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:46.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:46.308+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac014930 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:46.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:46.308+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac015040 legacy=0x7fbdac005ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:47.142 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:33:47.146 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:33:47.890 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac015040 legacy=0x7fbdac005ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:47.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac014930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:47.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:33:47.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac005ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:47.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac00d7c0 2024-03-04T08:33:47.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac00dba0 2024-03-04T08:33:47.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac015790 2024-03-04T08:33:47.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:47.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00dba0 legacy=0x7fbdac01d650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:47.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac005ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:47.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:47.896+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:48.100+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:48.100+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac005ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:48.100+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00dba0 legacy=0x7fbdac01d650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:48.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:48.100+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:48.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:48.504+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:48.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:48.504+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00dba0 legacy=0x7fbdac01d650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:48.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:48.504+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac005ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:48.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:48.504+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:49.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:49.308+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:49.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:49.308+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00dba0 legacy=0x7fbdac01d650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:49.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:49.308+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac005ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:49.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:49.308+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 msgr2=0x7fbdac005ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac015790 0x7fbdac005ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00dba0 legacy=0x7fbdac01d650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:50.891 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:50.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:50.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:50.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac011cc0 2024-03-04T08:33:50.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac00e950 2024-03-04T08:33:50.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:50.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:50.896+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:51.100+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:51.100+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:51.100+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:51.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:51.100+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:51.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:51.504+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:51.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:51.504+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:51.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:51.504+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:51.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:51.504+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:52.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:52.308+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:52.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:52.308+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:52.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:52.308+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:52.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:52.308+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:53.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:53.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:53.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:53.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:53.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:33:53.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:53.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00f5a0 con 0x7fbdac00d7c0 2024-03-04T08:33:53.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac00e950 2024-03-04T08:33:53.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac011cc0 2024-03-04T08:33:53.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:53.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:53.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:53.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:53.896+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac00f830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:54.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:54.100+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:54.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:54.100+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:54.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:54.100+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:54.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:54.100+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac00f830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:54.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:54.504+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:54.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:54.504+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:54.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:54.504+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:54.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:54.504+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac00f830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:55.308+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:55.308+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:55.308+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:55.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:55.308+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac00f830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:33:56.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:56.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:56.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac00f830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:56.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac012170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:56.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:56.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac014890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:56.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00dba0 2024-03-04T08:33:56.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac015790 2024-03-04T08:33:56.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 msgr2=0x7fbdac014890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:56.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:56.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:56.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:56.896+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac014890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:57.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:57.100+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 msgr2=0x7fbdac014890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:57.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:57.100+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:57.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:57.100+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:57.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:57.100+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac014890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:33:57.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:57.504+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 msgr2=0x7fbdac014890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:57.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:57.504+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:57.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:57.504+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:57.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:57.504+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac014890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:33:58.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:58.308+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 msgr2=0x7fbdac014890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:33:58.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:58.308+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:58.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:58.308+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:58.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:58.308+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac014890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:33:59.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:59.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac015790 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:59.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 msgr2=0x7fbdac014890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:33:59.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac014890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:33:59.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:33:59.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 0x7fbdac011330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:59.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:33:59.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac00d7c0 2024-03-04T08:33:59.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac011cc0 2024-03-04T08:33:59.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac010f50 2024-03-04T08:33:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:33:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:33:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:33:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 msgr2=0x7fbdac011330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:33:59.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:33:59.896+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 0x7fbdac011330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:00.100+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:00.100+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:00.100+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:00.100+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 msgr2=0x7fbdac011330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:00.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:00.100+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 0x7fbdac011330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:00.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:00.504+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:00.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:00.504+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:00.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:00.504+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:00.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:00.504+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 msgr2=0x7fbdac011330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:00.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:00.504+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 0x7fbdac011330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:01.308+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:01.308+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:01.308+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:01.308+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 msgr2=0x7fbdac011330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:01.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:01.308+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 0x7fbdac011330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:02.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 msgr2=0x7fbdac011330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:02.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010f50 0x7fbdac011330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:02.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:02.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:02.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:02.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:02.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00f5a0 con 0x7fbdac00e950 2024-03-04T08:34:02.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00dba0 2024-03-04T08:34:02.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:02.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:02.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:02.896+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:03.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:03.100+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:03.100+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:03.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:03.100+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:03.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:03.504+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:03.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:03.504+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:03.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:03.504+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:04.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:04.308+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:04.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:04.308+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:04.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:04.308+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:05.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 msgr2=0x7fbdac00b9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:05.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00dba0 0x7fbdac00b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:05.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:05.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac018940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:05.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:34:05.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac00d7c0 2024-03-04T08:34:05.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00f5a0 con 0x7fbdac00e950 2024-03-04T08:34:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac010840 2024-03-04T08:34:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac018940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:05.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:05.896+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac018940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:06.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:06.100+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:06.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:06.100+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:06.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:06.100+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac018940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:06.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:06.100+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac018940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:06.504+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:06.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:06.504+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac018940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:06.504+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:06.499 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:06.504+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac018940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:07.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:07.308+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:07.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:07.308+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac018940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:07.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:07.308+0000 7fbdd0e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:07.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:07.308+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac018940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:08.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac018940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:08.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac018940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:08.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:08.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac016220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:08.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:08.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac010c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:08.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00b9a0 2024-03-04T08:34:08.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac011cc0 2024-03-04T08:34:08.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:08.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:08.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:08.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:08.896+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:09.093 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:09.100+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:09.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:09.100+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:09.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:09.100+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:09.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:09.100+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:09.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:09.504+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:09.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:09.504+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:09.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:09.504+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:09.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:09.504+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:10.308+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:10.308+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:10.308+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:10.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:10.308+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011cc0 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 0x7fbdac014180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:34:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac00e950 2024-03-04T08:34:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac010840 2024-03-04T08:34:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac011180 2024-03-04T08:34:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 msgr2=0x7fbdac014180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 0x7fbdac014180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:11.895+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:12.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:12.099+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:12.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:12.099+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 msgr2=0x7fbdac014180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:12.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:12.099+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:12.094 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:12.099+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 0x7fbdac014180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:12.095 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:12.099+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:12.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:12.503+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:12.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:12.503+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 msgr2=0x7fbdac014180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:12.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:12.503+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:12.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:12.503+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 0x7fbdac014180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:12.498 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:12.503+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:13.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:13.307+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:13.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:13.307+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:13.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:13.307+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 msgr2=0x7fbdac014180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:13.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:13.307+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 0x7fbdac014180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:13.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:13.307+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:14.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 msgr2=0x7fbdac014180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:14.893 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac011180 0x7fbdac014180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac008d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac0146c0 2024-03-04T08:34:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00d7c0 2024-03-04T08:34:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:14.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:14.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:14.899+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:15.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:15.103+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:15.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:15.103+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:15.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:15.103+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:15.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:15.507+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:15.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:15.507+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:15.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:15.507+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:16.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:16.311+0000 7fbdcbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:16.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:16.311+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:16.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:16.311+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:17.199 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:34:17.203 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:34:17.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.895+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:17.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.895+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:17.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.895+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:34:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac0146c0 2024-03-04T08:34:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00d7c0 2024-03-04T08:34:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac010840 2024-03-04T08:34:17.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:17.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:17.899+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:18.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:18.103+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:18.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:18.103+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:18.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:18.103+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:18.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:18.103+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:18.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:18.103+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:18.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:18.507+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:18.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:18.507+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:18.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:18.507+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:18.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:18.507+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:18.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:18.507+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:19.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:19.311+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:19.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:19.311+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:19.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:19.311+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:19.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:19.311+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:19.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:19.311+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:20.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:20.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac010840 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:20.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 msgr2=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:20.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00d7c0 0x7fbdac010c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac00e950 2024-03-04T08:34:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00b9a0 2024-03-04T08:34:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:20.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:20.899+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:21.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:21.103+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:21.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:21.103+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:21.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:21.103+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:21.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:21.507+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:21.507+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:21.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:21.507+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:22.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:22.311+0000 7fbdcb400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac011180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:22.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:22.311+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:22.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:22.311+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:23.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:23.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:23.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00e950 legacy=0x7fbdac011180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:23.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:34:23.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:23.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:23.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac0146c0 2024-03-04T08:34:23.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00e950 2024-03-04T08:34:23.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac00b9a0 2024-03-04T08:34:23.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:23.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:23.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:23.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:23.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:23.899+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:24.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:24.103+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:24.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:24.103+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:24.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:24.103+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:24.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:24.103+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:24.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:24.103+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:24.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:24.507+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:24.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:24.507+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:24.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:24.507+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:24.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:24.507+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:24.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:24.507+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:25.311+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:25.311+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:25.311+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:25.311+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:25.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:25.311+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:26.894 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:26.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:26.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:26.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:26.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:26.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 0x7fbdac014140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:26.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac00d7c0 2024-03-04T08:34:26.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac011350 2024-03-04T08:34:26.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 msgr2=0x7fbdac014140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:26.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 0x7fbdac014140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:26.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:26.899+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:27.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:27.103+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 msgr2=0x7fbdac014140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:27.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:27.103+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 0x7fbdac014140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:27.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:27.103+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:27.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:27.507+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 msgr2=0x7fbdac014140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:27.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:27.507+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 0x7fbdac014140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:27.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:27.507+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:28.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:28.311+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 msgr2=0x7fbdac014140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:28.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:28.311+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 0x7fbdac014140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:28.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:28.311+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:29.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 msgr2=0x7fbdac014140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:29.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac011350 0x7fbdac014140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac010c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:34:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac0146c0 2024-03-04T08:34:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00e950 2024-03-04T08:34:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac00b9a0 2024-03-04T08:34:29.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:29.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:29.899+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:30.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:30.103+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:30.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:30.103+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:30.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:30.103+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:30.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:30.103+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:30.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:30.103+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:30.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:30.507+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:30.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:30.507+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:30.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:30.507+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:30.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:30.507+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:30.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:30.507+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:31.311+0000 7fbdcbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:31.311+0000 7fbdd0e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:31.311+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:31.311+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:31.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:31.311+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:32.895 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:32.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:32.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:32.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:32.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:32.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 0x7fbdac008f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:32.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac00d7c0 2024-03-04T08:34:32.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac008b10 2024-03-04T08:34:32.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 msgr2=0x7fbdac008f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:32.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 0x7fbdac008f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:32.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:32.899+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:33.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:33.103+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 msgr2=0x7fbdac008f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:33.103+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 0x7fbdac008f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:33.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:33.103+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:33.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:33.507+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 msgr2=0x7fbdac008f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:33.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:33.507+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 0x7fbdac008f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:33.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:33.507+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:34.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:34.311+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 msgr2=0x7fbdac008f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:34.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:34.311+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 0x7fbdac008f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:34.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:34.311+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:35.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 msgr2=0x7fbdac008f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:35.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac008b10 0x7fbdac008f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:35.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac00d7c0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:35.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:35.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:34:35.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:35.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac0146c0 2024-03-04T08:34:35.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00b9a0 2024-03-04T08:34:35.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac00e950 2024-03-04T08:34:35.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:35.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:35.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:35.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:35.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:35.899+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:36.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:36.103+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:36.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:36.103+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:36.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:36.103+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:36.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:36.103+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:36.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:36.103+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:36.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:36.507+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:36.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:36.507+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:36.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:36.507+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:36.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:36.507+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:36.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:36.507+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:37.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:37.311+0000 7fbdd0e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:37.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:37.311+0000 7fbdcb400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:37.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:37.311+0000 7fbdd0e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:37.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:37.311+0000 7fbdcbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:37.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:37.311+0000 7fbdcbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdc9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:38.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 0x7fbdac0140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac008ef0 2024-03-04T08:34:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac010840 2024-03-04T08:34:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008ef0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 msgr2=0x7fbdac0140b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:38.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:38.899+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 0x7fbdac0140b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:39.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:39.103+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 msgr2=0x7fbdac0140b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:39.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:39.103+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008ef0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:39.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:39.103+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 0x7fbdac0140b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:39.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:39.507+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 msgr2=0x7fbdac0140b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:39.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:39.507+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008ef0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:39.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:39.507+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 0x7fbdac0140b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:40.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:40.311+0000 7fbdcb400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 msgr2=0x7fbdac0140b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:40.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:40.311+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008ef0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:40.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:40.311+0000 7fbdcb400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 0x7fbdac0140b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:41.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 msgr2=0x7fbdac0140b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:41.896 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010840 0x7fbdac0140b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:41.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdc9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008ef0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:41.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 0x7fbdac011030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:41.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdd0e00640 1 -- reap_dead start 2024-03-04T08:34:41.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:41.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc0c2640 con 0x7fbdac0146c0 2024-03-04T08:34:41.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00e950 2024-03-04T08:34:41.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdc9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0c2080 con 0x7fbdac010c50 2024-03-04T08:34:41.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:41.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:41.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 msgr2=0x7fbdac011030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:41.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 0x7fbdac011030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:41.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:41.899+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:42.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:42.103+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:42.098 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:42.103+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:42.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:42.103+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 msgr2=0x7fbdac011030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:42.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:42.103+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 0x7fbdac011030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:42.099 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:42.103+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:42.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:42.507+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:42.507+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 msgr2=0x7fbdac011030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:42.507+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:42.507+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 0x7fbdac011030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:42.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:42.507+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:43.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:43.311+0000 7fbdd0e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:43.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:43.311+0000 7fbdcbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 msgr2=0x7fbdac011030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:43.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:43.311+0000 7fbdcb400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:43.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:43.311+0000 7fbdcbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 0x7fbdac011030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:43.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:43.311+0000 7fbdcb400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:44.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdc9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 msgr2=0x7fbdac011030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:44.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdc9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7fbdac010c50 0x7fbdac011030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:44.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdc9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 msgr2=0x7fbdac00d280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:44.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7fbdac00e950 0x7fbdac00d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:44.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdc9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7fbdac0146c0 legacy=0x7fbdac007cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:44.897 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdc9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:44.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdc9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdac00e300 con 0x7fbdac008ef0 2024-03-04T08:34:44.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdc9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc15ae00 con 0x7fbdac00b9a0 2024-03-04T08:34:44.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:44.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:44.898 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:44.899+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008ef0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:45.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:45.103+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:45.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:45.103+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:45.097 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:45.103+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008ef0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:45.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:45.507+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:45.507+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:45.502 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:45.507+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008ef0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:46.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:46.311+0000 7fbdcb400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 msgr2=0x7fbdac014ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:46.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:46.311+0000 7fbdcb400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbdac00b9a0 0x7fbdac014ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:46.306 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:46.311+0000 7fbdcbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7fbdac008ef0 legacy=0x7fbdac008400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:47.340 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:34:47.344 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:34:47.764 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-03-04T08:34:47.765 INFO:tasks.ceph:Scrubbing osd.2 2024-03-04T08:34:47.765 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-04T08:34:47.912 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.915+0000 7f1e9c000640 1 Processor -- start 2024-03-04T08:34:47.913 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.915+0000 7f1e9c000640 1 -- start start 2024-03-04T08:34:47.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e9c000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e940c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:47.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e9c000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e9415be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:47.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e9c000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 0x7f1e940c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:47.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e9c000640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e9415c3c0 2024-03-04T08:34:47.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e9c000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e94152cf0 2024-03-04T08:34:47.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e9c000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940c88c0 con 0x7f1e94070cd0 2024-03-04T08:34:47.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 msgr2=0x7f1e940c8d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:47.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 msgr2=0x7f1e940c8210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:47.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e9415be80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:47.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 0x7f1e940c8d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:47.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e940c8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:47.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:47.919+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e9415be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:48.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.123+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 msgr2=0x7f1e940c8d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:48.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.123+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 0x7f1e940c8d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:48.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.123+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e9415be80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:48.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.123+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 msgr2=0x7f1e940c8210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:48.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.123+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e940c8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:48.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.123+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e9415be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:48.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.527+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 msgr2=0x7f1e940c8d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.527+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 0x7f1e940c8d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.527+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 msgr2=0x7f1e940c8210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.527+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e9415be80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.527+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e940c8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:48.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:48.527+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e9415be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:49.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:49.331+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 msgr2=0x7f1e940c8d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:49.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:49.331+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 0x7f1e940c8d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:49.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:49.331+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 msgr2=0x7f1e940c8210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:49.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:49.331+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e9415be80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:49.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:49.331+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e940c8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:49.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:49.331+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e9415be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:50.914 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:50.919+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e9415be80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:50.919+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e9415be80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:50.919+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 msgr2=0x7f1e940c8210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:50.919+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e940c8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:50.919+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 msgr2=0x7f1e940c8d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:50.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:50.919+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e9415c3c0 0x7f1e940c8d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:50.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:50.919+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e74007400 2024-03-04T08:34:50.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:50.919+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e740036c0 2024-03-04T08:34:50.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:50.919+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74007400 legacy=0x7f1e740098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:50.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:50.919+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:51.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:51.123+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74007400 legacy=0x7f1e740098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:51.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:51.123+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:51.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:51.527+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74007400 legacy=0x7f1e740098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:51.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:51.527+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:52.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:52.331+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74007400 legacy=0x7f1e740098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:52.331+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:53.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:53.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74007400 legacy=0x7f1e740098c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:53.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:34:53.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:53.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e7400c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:53.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e740036c0 2024-03-04T08:34:53.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e94152cf0 2024-03-04T08:34:53.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e94070cd0 2024-03-04T08:34:53.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 msgr2=0x7f1e7400c8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:53.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:53.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e7400c3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:53.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:53.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:53.919+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e7400c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:54.123+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:54.123+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 msgr2=0x7f1e7400c8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:54.123+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e7400c3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:54.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:54.123+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:54.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:54.123+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e7400c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:54.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:54.527+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:54.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:54.527+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 msgr2=0x7f1e7400c8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:54.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:54.527+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e7400c3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:54.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:54.527+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:54.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:54.527+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e7400c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:55.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:55.331+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:55.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:55.331+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e7400c3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:55.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:55.331+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 msgr2=0x7f1e7400c8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:34:55.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:55.331+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:55.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:55.331+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e7400c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:56.915 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e7400c3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:56.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:56.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 msgr2=0x7f1e7400c8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:56.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e94152cf0 0x7f1e7400c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:56.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:56.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 0x7f1e740098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:56.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e7400b500 2024-03-04T08:34:56.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e7400b120 2024-03-04T08:34:56.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 msgr2=0x7f1e740098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:56.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 0x7f1e740098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:34:56.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:56.919+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:57.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:57.123+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 msgr2=0x7f1e740098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:57.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:57.123+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:57.118 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:57.123+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 0x7f1e740098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:34:57.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:57.527+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 msgr2=0x7f1e740098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:57.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:57.527+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:57.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:57.527+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 0x7f1e740098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:34:58.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:58.331+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 msgr2=0x7f1e740098c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:34:58.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:58.331+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:58.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:58.331+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 0x7f1e740098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:34:59.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 msgr2=0x7f1e740098c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:59.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400b120 0x7f1e740098c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:34:59.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:34:59.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:34:59.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:34:59.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e7400b500 2024-03-04T08:34:59.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e740036c0 2024-03-04T08:34:59.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e94070cd0 2024-03-04T08:34:59.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:34:59.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:34:59.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e7400c3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:34:59.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:34:59.919+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:00.123+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:00.116 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:00.123+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:00.123+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e7400c3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:00.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:00.123+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:00.521 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:00.527+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:00.527+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:00.527+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e7400c3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:00.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:00.527+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:01.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:01.331+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:01.331+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:01.331+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e7400c3a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:01.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:01.331+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 msgr2=0x7f1e7400c3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e94070cd0 0x7f1e7400c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:02.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 0x7f1e74007850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e7400c8e0 2024-03-04T08:35:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e74007d90 2024-03-04T08:35:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 msgr2=0x7f1e74007850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 0x7f1e74007850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740081a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:02.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:02.919+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:03.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:03.123+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 msgr2=0x7f1e74007850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:03.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:03.123+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 0x7f1e74007850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:03.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:03.123+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740081a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:03.117 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:03.123+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:03.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:03.527+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 msgr2=0x7f1e74007850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:03.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:03.527+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 0x7f1e74007850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:03.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:03.527+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740081a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:03.522 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:03.527+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:04.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:04.331+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 msgr2=0x7f1e74007850 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:04.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:04.331+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 0x7f1e74007850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:04.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:04.331+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740081a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:04.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:04.331+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:05.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740081a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 msgr2=0x7f1e74007850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e7400c8e0 0x7f1e74007850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:35:05.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:05.932 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e7400b500 2024-03-04T08:35:05.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e740036c0 2024-03-04T08:35:05.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74007d90 2024-03-04T08:35:05.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:05.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:05.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740081a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:05.933 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:05.923+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:06.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:06.126+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:06.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:06.126+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740081a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:06.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:06.126+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:06.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:06.126+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:06.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:06.530+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:06.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:06.530+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:06.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:06.530+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740081a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:06.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:06.530+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:07.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:07.334+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:07.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:07.334+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:07.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:07.334+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740081a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:07.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:07.334+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:08.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740081a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e740081a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e7400b500 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 0x7f1e74008ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e740086e0 2024-03-04T08:35:08.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e74007850 2024-03-04T08:35:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:08.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:08.922+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:09.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:09.126+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:09.126+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:09.126+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:09.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:09.126+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:09.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:09.530+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:09.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:09.530+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:09.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:09.530+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:09.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:09.530+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:10.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:10.334+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:10.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:10.334+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:10.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:10.334+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:10.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:10.334+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:11.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:11.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:11.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:11.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e740086e0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:11.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 0x7f1e740093e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:11.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:35:11.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:11.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e740036c0 2024-03-04T08:35:11.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74009000 2024-03-04T08:35:11.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e74007850 2024-03-04T08:35:11.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 msgr2=0x7f1e740093e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:11.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 0x7f1e740093e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:11.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:11.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:11.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:11.922+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74006270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:12.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:12.126+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 msgr2=0x7f1e740093e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:12.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:12.126+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 0x7f1e740093e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:12.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:12.126+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:12.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:12.126+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74006270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:12.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:12.126+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:12.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:12.530+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 msgr2=0x7f1e740093e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:12.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:12.530+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 0x7f1e740093e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:12.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:12.530+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:12.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:12.530+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74006270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:12.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:12.530+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:13.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:13.334+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 msgr2=0x7f1e740093e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:13.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:13.334+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 0x7f1e740093e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:13.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:13.334+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:13.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:13.334+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74006270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:13.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:13.334+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:14.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:14.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74007850 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:14.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 msgr2=0x7f1e740093e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:14.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74009000 0x7f1e740093e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:14.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74006270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:14.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:14.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e74007d90 2024-03-04T08:35:14.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e740099c0 2024-03-04T08:35:14.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74007d90 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:14.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:14.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:14.922+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:15.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:15.126+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74007d90 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:15.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:15.126+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:15.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:15.126+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:15.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:15.530+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74007d90 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:15.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:15.530+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:15.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:15.530+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:16.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:16.334+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74007d90 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:16.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:16.334+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:16.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:16.334+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:17.425 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:35:17.429 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:35:17.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74007d90 legacy=0x7f1e7400b910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:35:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e7400c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e740036c0 2024-03-04T08:35:17.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74007d90 2024-03-04T08:35:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e740099c0 2024-03-04T08:35:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:17.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:17.922+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:18.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:18.126+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:18.126+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:18.126+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:18.126+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:18.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:18.126+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:18.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:18.530+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:18.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:18.530+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:18.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:18.530+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:18.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:18.530+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:18.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:18.530+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:19.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:19.334+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:19.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:19.334+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:19.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:19.334+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:19.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:19.334+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:19.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:19.334+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:20.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 msgr2=0x7f1e74008ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:20.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740099c0 0x7f1e74008ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:20.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e7400c370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:20.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e7400c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:20.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:20.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e74009020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:20.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e74008270 2024-03-04T08:35:20.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e740078f0 2024-03-04T08:35:20.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e74009020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:20.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008270 legacy=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:20.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:20.922+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e74009020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:21.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:21.126+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e74009020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:21.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:21.126+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008270 legacy=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:21.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:21.126+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e74009020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:21.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:21.530+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e74009020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:21.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:21.530+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008270 legacy=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:21.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:21.530+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e74009020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:22.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:22.334+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008270 legacy=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:22.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:22.334+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e74009020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:22.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:22.334+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e74009020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:23.919 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e74009020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:23.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e74009020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:23.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008270 legacy=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:23.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:35:23.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:23.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e74009730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:23.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e740036c0 2024-03-04T08:35:23.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74007d90 2024-03-04T08:35:23.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e740078f0 2024-03-04T08:35:23.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:23.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:23.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e9a000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e74009730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:23.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e9a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e74009730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:23.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:23.922+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:24.119 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:24.126+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:24.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:24.126+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:24.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:24.126+0000 7f1e9a000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e74009730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:24.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:24.126+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:24.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:24.126+0000 7f1e9a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e74009730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:24.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:24.530+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:24.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:24.530+0000 7f1e9a000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e74009730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:24.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:24.530+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:24.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:24.530+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:24.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:24.530+0000 7f1e9a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e74009730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:25.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:25.334+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:25.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:25.334+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:25.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:25.334+0000 7f1e9a000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e74009730 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:25.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:25.334+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:25.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:25.334+0000 7f1e9a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e74009730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:26.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:26.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:26.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e74009730 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:26.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74007d90 0x7f1e74009730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:26.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:26.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:26.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e74008ac0 2024-03-04T08:35:26.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74008240 2024-03-04T08:35:26.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:26.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:26.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:26.922+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ac0 legacy=0x7f1e74006250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:27.126+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:27.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:27.126+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:27.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:27.126+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ac0 legacy=0x7f1e74006250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:27.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:27.530+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:27.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:27.530+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:27.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:27.530+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ac0 legacy=0x7f1e74006250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:28.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:28.334+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:28.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:28.334+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:28.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:28.334+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ac0 legacy=0x7f1e74006250 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:29.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:29.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:29.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ac0 legacy=0x7f1e74006250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:29.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:35:29.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:29.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e74008240 2024-03-04T08:35:29.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e74008ed0 2024-03-04T08:35:29.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e740036c0 2024-03-04T08:35:29.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:29.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ed0 legacy=0x7f1e74006960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:29.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:29.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:29.926+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:30.130+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:30.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:30.130+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:30.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:30.130+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ed0 legacy=0x7f1e74006960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:30.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:30.130+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:30.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:30.534+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:30.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:30.534+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ed0 legacy=0x7f1e74006960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:30.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:30.534+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:30.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:30.534+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:31.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:31.338+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:31.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:31.338+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ed0 legacy=0x7f1e74006960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:31.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:31.338+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:31.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:31.338+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:32.920 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ed0 legacy=0x7f1e74006960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e740096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74007d90 2024-03-04T08:35:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e740078f0 2024-03-04T08:35:32.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740096f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e740096f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:32.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:32.922+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:33.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:33.126+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740096f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:33.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:33.126+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e740096f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:33.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:33.126+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:33.121 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:33.126+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:33.530+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740096f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:33.525 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:33.530+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e740096f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:33.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:33.530+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:33.526 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:33.530+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:34.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:34.334+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740096f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:34.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:34.334+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e740096f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:34.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:34.334+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:34.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:34.334+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:35.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:35.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:35.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 msgr2=0x7f1e740096f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:35.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74007d90 0x7f1e740096f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:35.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 0x7f1e740187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:35.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:35:35.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:35.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e74008ed0 2024-03-04T08:35:35.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74006250 2024-03-04T08:35:35.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e740078f0 2024-03-04T08:35:35.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 msgr2=0x7f1e740187c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:35.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 0x7f1e740187c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:35.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:35.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ed0 legacy=0x7f1e740066d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:35.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:35.926+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:36.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:36.130+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 msgr2=0x7f1e740187c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:36.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:36.130+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 0x7f1e740187c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:36.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:36.130+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:36.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:36.130+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ed0 legacy=0x7f1e740066d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:36.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:36.130+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:36.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:36.534+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 msgr2=0x7f1e740187c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:36.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:36.534+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 0x7f1e740187c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:36.534+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:36.534+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ed0 legacy=0x7f1e740066d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:36.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:36.534+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:37.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:37.338+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 msgr2=0x7f1e740187c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:37.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:37.338+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 0x7f1e740187c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:37.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:37.338+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:37.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:37.338+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ed0 legacy=0x7f1e740066d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:37.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:37.338+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:38.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:38.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:38.921 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 msgr2=0x7f1e740187c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:38.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006250 0x7f1e740187c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:38.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008ed0 legacy=0x7f1e740066d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:38.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:38.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e74008240 2024-03-04T08:35:38.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e740036c0 2024-03-04T08:35:38.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74009fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:38.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:38.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:38.926+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:39.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:39.130+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74009fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:39.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:39.130+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:39.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:39.130+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:39.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:39.534+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74009fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:39.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:39.534+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:39.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:39.534+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:40.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:40.338+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74009fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:40.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:40.338+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:40.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:40.338+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:41.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:41.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:41.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74009fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:41.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:41.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:35:41.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:41.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e74008240 2024-03-04T08:35:41.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e740036c0 2024-03-04T08:35:41.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e740078f0 2024-03-04T08:35:41.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:41.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:41.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008240 legacy=0x7f1e740066d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:41.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:41.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:41.926+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:42.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:42.130+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008240 legacy=0x7f1e740066d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:42.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:42.130+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:42.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:42.130+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:42.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:42.130+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:42.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:42.130+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:42.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:42.534+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:42.534+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:42.534+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008240 legacy=0x7f1e740066d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:42.534+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:42.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:42.534+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:43.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:43.338+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:43.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:43.338+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008240 legacy=0x7f1e740066d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:43.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:43.338+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:43.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:43.338+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:43.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:43.338+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:44.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:44.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:44.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:44.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:44.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008240 legacy=0x7f1e740066d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:44.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:44.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e740096b0 2024-03-04T08:35:44.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74005b40 2024-03-04T08:35:44.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740096b0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:44.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:44.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:44.926+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:45.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:45.130+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740096b0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:45.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:45.130+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:45.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:45.130+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:45.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:45.534+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740096b0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:45.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:45.534+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:45.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:45.534+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:46.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:46.338+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:46.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:46.338+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740096b0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:46.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:46.338+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:47.572 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:35:47.576 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:35:47.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:47.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:47.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740096b0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:35:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e740167b0 2024-03-04T08:35:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e740036c0 2024-03-04T08:35:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e740078f0 2024-03-04T08:35:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e7401d110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:47.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:47.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:47.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:47.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:47.926+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:48.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:48.130+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e7401d110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:48.130+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:48.130+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:48.130+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:48.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:48.130+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:48.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:48.534+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e7401d110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:48.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:48.534+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:48.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:48.534+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:48.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:48.534+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:48.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:48.534+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:49.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:49.338+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e7401d110 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:49.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:49.338+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:49.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:49.338+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:49.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:49.338+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:49.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:49.338+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:50.922 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e740036c0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e7401d110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:50.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e74008240 2024-03-04T08:35:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74005b40 2024-03-04T08:35:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:50.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:50.926+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:51.130+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:51.130+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:51.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:51.130+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:51.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:51.534+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:51.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:51.534+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:51.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:51.534+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:52.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:52.338+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:52.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:52.338+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:52.338+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:53.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:53.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:53.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74008240 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:53.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:35:53.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:53.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:53.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e740167b0 2024-03-04T08:35:53.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74005b40 2024-03-04T08:35:53.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e740078f0 2024-03-04T08:35:53.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e74006490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:53.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:53.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:53.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:53.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:53.926+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:54.130+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e74006490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:54.130+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:54.130+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:54.130+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:54.130+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:54.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:54.534+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:54.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:54.534+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e74006490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:54.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:54.534+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:54.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:54.534+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:54.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:54.534+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:55.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:55.338+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:55.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:55.338+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e74006490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:55.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:55.338+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:55.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:55.338+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:35:55.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:55.338+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:56.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:56.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:56.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:56.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:56.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e74006490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:56.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:56.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e740036c0 2024-03-04T08:35:56.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74008240 2024-03-04T08:35:56.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:56.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:56.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:56.926+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:57.130+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:57.130+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:57.130+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:35:57.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:57.534+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:57.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:57.534+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:57.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:57.534+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:35:58.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:58.338+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:35:58.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:58.338+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:35:58.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:58.338+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:35:59.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:59.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:35:59.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e740036c0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:35:59.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:35:59.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:35:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e740167b0 2024-03-04T08:35:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e74003b10 2024-03-04T08:35:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74005b40 2024-03-04T08:35:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e74006490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:35:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:35:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:35:59.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:35:59.930+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74003ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:00.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:00.134+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:00.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:00.134+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e74006490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:00.134+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:00.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:00.134+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74003ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:00.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:00.538+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:00.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:00.538+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e74006490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:00.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:00.538+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:00.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:00.538+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74003ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:01.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:01.342+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:01.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:01.342+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e74006490 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:01.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:01.342+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:01.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:01.342+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74003ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:02.923 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74003ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740167b0 legacy=0x7f1e74006490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:02.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74008240 2024-03-04T08:36:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e740078f0 2024-03-04T08:36:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:02.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:02.925+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:03.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:03.129+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:03.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:03.129+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:03.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:03.129+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:03.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:03.129+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:03.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:03.533+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:03.529 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:03.533+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:03.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:03.533+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:03.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:03.533+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:04.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:04.337+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:04.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:04.337+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:04.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:04.337+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:04.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:04.337+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:05.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:05.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:05.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:05.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008240 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:05.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:05.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:36:05.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:05.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e74003b10 2024-03-04T08:36:05.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74005b40 2024-03-04T08:36:05.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e740078f0 2024-03-04T08:36:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:05.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:05.925+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:06.129+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:06.129+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:06.129+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:06.129+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:06.129+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:06.533+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:06.533+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:06.533+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:06.533+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:06.530 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:06.533+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:07.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:07.337+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:07.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:07.337+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:07.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:07.337+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:07.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:07.337+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:07.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:07.337+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:08.924 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:08.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e740078f0 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:08.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:08.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:08.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:08.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:08.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e74006490 2024-03-04T08:36:08.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74008210 2024-03-04T08:36:08.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74006490 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:08.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:08.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:08.929+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:09.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:09.133+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74006490 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:09.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:09.133+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:09.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:09.133+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:09.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:09.537+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74006490 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:09.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:09.537+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:09.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:09.537+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:10.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:10.341+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74006490 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:10.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:10.341+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:10.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:10.341+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:11.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e74006490 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:36:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e74003b10 2024-03-04T08:36:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74005b40 2024-03-04T08:36:11.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e74008210 2024-03-04T08:36:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e9a000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:11.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:11.929+0000 7f1e9a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:12.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:12.133+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:12.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:12.133+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:12.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:12.133+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:12.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:12.133+0000 7f1e9a000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:12.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:12.133+0000 7f1e9a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:12.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:12.537+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:12.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:12.537+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:12.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:12.537+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:12.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:12.537+0000 7f1e9a000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:12.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:12.537+0000 7f1e9a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:13.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:13.341+0000 7f1e99600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:13.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:13.341+0000 7f1e99600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:13.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:13.341+0000 7f1e9a000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:13.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:13.341+0000 7f1e98c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:13.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:13.341+0000 7f1e9a000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:14.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:14.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:14.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 msgr2=0x7f1e74005f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:14.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005b40 0x7f1e74005f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:14.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:14.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:14.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e740078f0 2024-03-04T08:36:14.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74006490 2024-03-04T08:36:14.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:14.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 msgr2=0x7f1e740068b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:14.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:14.929+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:15.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:15.133+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:15.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:15.133+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 msgr2=0x7f1e740068b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:15.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:15.133+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:15.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:15.537+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:15.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:15.537+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 msgr2=0x7f1e740068b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:15.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:15.537+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:16.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:16.341+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:16.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:16.341+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 msgr2=0x7f1e740068b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:16.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:16.341+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:17.639 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:36:17.642 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:36:17.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 msgr2=0x7f1e740068b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:36:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e740078f0 2024-03-04T08:36:17.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74008210 2024-03-04T08:36:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e74006490 2024-03-04T08:36:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 msgr2=0x7f1e740068b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:17.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:17.925+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:18.129+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 msgr2=0x7f1e740068b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:18.129+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:18.129+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:18.129+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:18.129+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:18.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:18.533+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:18.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:18.533+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:18.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:18.533+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 msgr2=0x7f1e740068b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:18.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:18.533+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:18.527 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:18.533+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:19.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:19.337+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:19.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:19.337+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 msgr2=0x7f1e740068b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:19.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:19.337+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:19.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:19.337+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:19.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:19.337+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:20.925 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 msgr2=0x7f1e740068b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:20.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006490 0x7f1e740068b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:20.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:20.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:20.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:20.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e7400c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:20.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e74003b10 2024-03-04T08:36:20.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74005f50 2024-03-04T08:36:20.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:20.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:20.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:20.929+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:21.133+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:21.133+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:21.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:21.133+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:21.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:21.537+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:21.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:21.537+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:21.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:21.537+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:22.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:22.341+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:22.341+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:22.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:22.341+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e7400c340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e7400c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:36:23.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e740078f0 2024-03-04T08:36:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74008210 2024-03-04T08:36:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e74003b10 2024-03-04T08:36:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 msgr2=0x7f1e74006890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:23.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:23.929+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:24.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:24.133+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:24.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:24.133+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 msgr2=0x7f1e74006890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:24.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:24.133+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:24.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:24.133+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:24.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:24.133+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:24.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:24.537+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:24.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:24.537+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:24.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:24.537+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 msgr2=0x7f1e74006890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:24.537+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:24.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:24.537+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:25.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:25.341+0000 7f1e98c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:25.341+0000 7f1e9a000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 msgr2=0x7f1e74006890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:25.341+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:25.341+0000 7f1e9a000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:25.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:25.341+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 msgr2=0x7f1e74006890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:26.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:26.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:26.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:26.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74006330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:26.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e7400c340 2024-03-04T08:36:26.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74005f50 2024-03-04T08:36:26.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e74006330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:26.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:26.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:26.929+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74006330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:27.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:27.133+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e74006330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:27.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:27.133+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74006330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:27.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:27.133+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:27.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:27.537+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e74006330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:27.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:27.537+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74006330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:27.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:27.537+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:28.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:28.341+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e74006330 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:28.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:28.341+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74006330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:28.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:28.341+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:29.926 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e74006330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:29.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74006330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:29.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:29.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:29.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:36:29.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e740078f0 2024-03-04T08:36:29.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400d470 con 0x7f1e7400c340 2024-03-04T08:36:29.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74008210 2024-03-04T08:36:29.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:29.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:29.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:29.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:29.933+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:30.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:30.137+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:30.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:30.137+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:30.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:30.137+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:30.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:30.137+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:30.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:30.541+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:30.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:30.541+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:30.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:30.541+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:30.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:30.541+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:31.338 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:31.345+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:31.400 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:31.345+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:31.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:31.345+0000 7f1e9a000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:31.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:31.345+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:32.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74016ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74005f50 2024-03-04T08:36:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e74003b10 2024-03-04T08:36:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e74016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 msgr2=0x7f1e74006890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:32.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:32.929+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:33.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:33.133+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e74016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:33.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:33.133+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:33.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:33.133+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 msgr2=0x7f1e74006890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:33.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:33.133+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:33.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:33.537+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e74016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:33.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:33.537+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:33.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:33.537+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 msgr2=0x7f1e74006890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:33.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:33.537+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:34.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:34.341+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e74016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:34.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:34.341+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:34.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:34.341+0000 7f1e99600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 msgr2=0x7f1e74006890 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:34.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:34.341+0000 7f1e99600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:35.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 msgr2=0x7f1e74006890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:35.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74003b10 0x7f1e74006890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:35.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 msgr2=0x7f1e74016ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:35.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74005f50 0x7f1e74016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:35.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:35.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:35.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:36:35.942 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e740078f0 2024-03-04T08:36:35.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74006370 2024-03-04T08:36:35.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e74008210 2024-03-04T08:36:35.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:35.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 msgr2=0x7f1e74017400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:35.943 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:35.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:35.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:35.933+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:36.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:36.137+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 msgr2=0x7f1e74017400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:36.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:36.137+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:36.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:36.137+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:36.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:36.137+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:36.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:36.137+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:36.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:36.541+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:36.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:36.541+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 msgr2=0x7f1e74017400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:36.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:36.541+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:36.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:36.541+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:36.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:36.541+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:37.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:37.345+0000 7f1e9a000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 msgr2=0x7f1e74017400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:37.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:37.345+0000 7f1e99600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:37.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:37.345+0000 7f1e9a000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:37.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:37.345+0000 7f1e98c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:37.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:37.345+0000 7f1e98c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:38.927 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e92a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 msgr2=0x7f1e74017400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74013790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:38.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74016ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e74003b10 2024-03-04T08:36:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74006890 2024-03-04T08:36:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 msgr2=0x7f1e74016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:38.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:38.933+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:39.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:39.137+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 msgr2=0x7f1e74016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:39.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:39.137+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:39.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:39.137+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:39.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:39.541+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:39.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:39.541+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 msgr2=0x7f1e74016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:39.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:39.541+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:40.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:40.345+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 msgr2=0x7f1e74016ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:40.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:40.345+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:40.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:40.345+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:41.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e92a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 msgr2=0x7f1e74016ec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:41.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74016ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:41.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e74003b10 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:41.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e9a000640 1 -- reap_dead start 2024-03-04T08:36:41.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:41.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e92a00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e94158bd0 con 0x7f1e740078f0 2024-03-04T08:36:41.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e92a00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7400c170 con 0x7f1e7400c340 2024-03-04T08:36:41.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e92a00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74006370 2024-03-04T08:36:41.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:41.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 msgr2=0x7f1e74017400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:41.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:41.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:41.929+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74017940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:42.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:42.133+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 msgr2=0x7f1e74017400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:42.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:42.133+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:42.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:42.133+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:42.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:42.133+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74017940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:42.537+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:42.537+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 msgr2=0x7f1e74017400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:42.537+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74017940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:42.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:42.537+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:43.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:43.341+0000 7f1e9a000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:43.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:43.341+0000 7f1e98c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 msgr2=0x7f1e74017400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:43.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:43.341+0000 7f1e98c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:43.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:43.341+0000 7f1e99600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74017940 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:44.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e92a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 msgr2=0x7f1e74017400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:44.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e92a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e74006370 0x7f1e74017400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:44.928 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e92a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f1e7400c340 legacy=0x7f1e74017940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:44.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e92a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f1e740078f0 legacy=0x7f1e74008ed0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:44.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e92a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:44.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e92a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74018050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:44.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e92a00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e941530d0 con 0x7f1e74008210 2024-03-04T08:36:44.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e92a00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e94158910 con 0x7f1e74006890 2024-03-04T08:36:44.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 msgr2=0x7f1e74018050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:44.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:44.929 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74018050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:44.930 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:44.933+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:45.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:45.137+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 msgr2=0x7f1e74018050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:45.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:45.137+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:45.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:45.137+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74018050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:45.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:45.137+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:45.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:45.541+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:45.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:45.541+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 msgr2=0x7f1e74018050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:45.534 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:45.541+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:45.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:45.541+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74018050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:46.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:46.345+0000 7f1e98c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 msgr2=0x7f1e74018050 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:46.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:46.345+0000 7f1e99600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 msgr2=0x7f1e7400b8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:46.341 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:46.345+0000 7f1e98c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f1e74006890 0x7f1e74018050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:46.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:46.345+0000 7f1e99600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f1e74008210 0x7f1e7400b8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:47.708 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:36:47.712 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:36:47.793 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-03-04T08:36:47.794 INFO:tasks.ceph:Scrubbing osd.3 2024-03-04T08:36:47.794 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-04T08:36:47.984 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.985+0000 7f8414800640 1 Processor -- start 2024-03-04T08:36:47.985 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.985+0000 7f8414800640 1 -- start start 2024-03-04T08:36:47.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.989+0000 7f8414800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f840c15b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:47.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.989+0000 7f8414800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f840c15b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:47.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.989+0000 7f8414800640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f840c0c1500 2024-03-04T08:36:47.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.989+0000 7f8414800640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f840c0732a0 2024-03-04T08:36:47.986 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.989+0000 7f8414800640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f840c0c18e0 2024-03-04T08:36:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.989+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 msgr2=0x7f840c15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.989+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f840c0c1500 legacy=0x7f840c072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.989+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f840c15b7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.989+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f840c15b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:47.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:47.989+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f840c15b7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:48.193+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f840c0c1500 legacy=0x7f840c072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:48.193+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f840c15b7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:48.193+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 msgr2=0x7f840c15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:48.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:48.193+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f840c15b7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:48.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:48.193+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f840c15b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:48.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:48.597+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f840c0c1500 legacy=0x7f840c072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:48.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:48.597+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f840c15b7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:48.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:48.597+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 msgr2=0x7f840c15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:48.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:48.597+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f840c15b7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:48.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:48.597+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f840c15b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:49.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:49.401+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 msgr2=0x7f840c15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:49.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:49.401+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f840c15b7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:49.397 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:49.401+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f840c0c1500 legacy=0x7f840c072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:49.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:49.401+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f840c15b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:49.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:49.401+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f840c15b7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:50.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f840c15b7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:50.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f840c15b7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:50.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 msgr2=0x7f840c15b290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:50.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f840c15b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f840c0c1500 legacy=0x7f840c072b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f0008440 2024-03-04T08:36:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f00058e0 2024-03-04T08:36:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 msgr2=0x7f83f0007d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:50.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0008440 legacy=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:50.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:50.989+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:51.193+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 msgr2=0x7f83f0007d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:51.193+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0008440 legacy=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:51.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:51.193+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:51.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:51.597+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0008440 legacy=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:51.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:51.597+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 msgr2=0x7f83f0007d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:51.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:51.597+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:52.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:52.401+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0008440 legacy=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:52.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:52.401+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 msgr2=0x7f83f0007d60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:52.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:52.401+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:53.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 msgr2=0x7f83f0007d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:53.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:53.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0008440 legacy=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:53.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:36:53.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f83f000c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f00058e0 2024-03-04T08:36:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f840c0732a0 2024-03-04T08:36:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f000cd30 con 0x7f840c0c18e0 2024-03-04T08:36:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 msgr2=0x7f83f0007d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:53.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:53.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:53.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 msgr2=0x7f83f000c740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:53.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f83f000c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:53.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:53.993+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:54.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.197+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 msgr2=0x7f83f0007d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:54.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.197+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:54.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.197+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:54.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.197+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 msgr2=0x7f83f000c740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:54.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.197+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f83f000c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:54.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.197+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:36:54.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.601+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 msgr2=0x7f83f0007d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:54.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.601+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:54.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.601+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:54.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.601+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 msgr2=0x7f83f000c740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:54.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.601+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f83f000c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:54.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:54.601+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:36:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:55.405+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 msgr2=0x7f83f0007d00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:36:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:55.405+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:55.405+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:55.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:55.405+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 msgr2=0x7f83f000c740 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:55.405+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f83f000c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:55.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:55.405+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:36:56.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:56.993+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:56.987 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:56.993+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:56.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:56.993+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 msgr2=0x7f83f000c740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:56.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:56.993+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f840c0732a0 0x7f83f000c740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:56.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:56.993+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 msgr2=0x7f83f0007d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:56.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:56.993+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00058e0 0x7f83f0007d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:36:56.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:56.993+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f000b410 2024-03-04T08:36:56.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:56.993+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f000b030 2024-03-04T08:36:56.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:56.993+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b410 legacy=0x7f83f000b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:56.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:56.993+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000b030 legacy=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:57.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:57.197+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b410 legacy=0x7f83f000b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:57.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:57.197+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000b030 legacy=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:57.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:57.601+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b410 legacy=0x7f83f000b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:57.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:57.601+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000b030 legacy=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:58.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:58.404+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b410 legacy=0x7f83f000b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:58.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:58.404+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000b030 legacy=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:36:59.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000b030 legacy=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:59.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b410 legacy=0x7f83f000b820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:36:59.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:36:59.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:59.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 0x7f83f0005cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:36:59.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f000b410 2024-03-04T08:36:59.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f000c740 2024-03-04T08:36:59.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f840c0c18e0 2024-03-04T08:36:59.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b410 legacy=0x7f83f000b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:36:59.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:36:59.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:36:59.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:36:59.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:36:59.992+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:00.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:00.196+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b410 legacy=0x7f83f000b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:00.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:00.196+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:00.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:00.196+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:00.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:00.196+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:00.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:00.196+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:00.593 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:00.600+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b410 legacy=0x7f83f000b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:00.600+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:00.600+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:00.600+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:00.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:00.600+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:01.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:01.404+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b410 legacy=0x7f83f000b820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:01.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:01.404+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:01.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:01.404+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:01.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:01.404+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:01.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:01.404+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:02.988 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c740 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b410 legacy=0x7f83f000b820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 0x7f83f000a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f00065e0 2024-03-04T08:37:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0006200 2024-03-04T08:37:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 msgr2=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:02.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 0x7f83f000a900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:02.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:02.992+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f00065e0 legacy=0x7f83f00114d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:03.196+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 msgr2=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:03.196+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 0x7f83f000a900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:03.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:03.196+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f00065e0 legacy=0x7f83f00114d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:03.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:03.600+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 msgr2=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:03.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:03.600+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 0x7f83f000a900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:03.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:03.600+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f00065e0 legacy=0x7f83f00114d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:04.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:04.404+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 msgr2=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:04.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:04.404+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 0x7f83f000a900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:04.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:04.404+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f00065e0 legacy=0x7f83f00114d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:05.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 msgr2=0x7f83f000a900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:05.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0006200 0x7f83f000a900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:05.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f00065e0 legacy=0x7f83f00114d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:05.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:05.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:37:05.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f000a900 2024-03-04T08:37:05.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f000b820 2024-03-04T08:37:05.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f840c0c18e0 2024-03-04T08:37:05.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:05.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:05.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000b820 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:05.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:05.992+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:06.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:06.196+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:06.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:06.196+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000b820 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:06.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:06.196+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:06.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:06.196+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:06.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:06.600+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:06.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:06.600+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000b820 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:06.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:06.600+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:06.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:06.600+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:07.404+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:07.404+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:07.404+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000b820 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:07.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:07.404+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:08.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 msgr2=0x7f83f000c180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:08.989 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f840c0c18e0 0x7f83f000c180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:08.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000b820 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:08.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:08.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:08.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:08.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0007af0 2024-03-04T08:37:08.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f0014a30 2024-03-04T08:37:08.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 msgr2=0x7f83f00144f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:08.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:08.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:08.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:08.992+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:09.196+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 msgr2=0x7f83f00144f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:09.196+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:09.196+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:09.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:09.196+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:09.600+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 msgr2=0x7f83f00144f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:09.600+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:09.600+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:09.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:09.600+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:10.404+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 msgr2=0x7f83f00144f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:10.404+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:10.398 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:10.404+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:10.399 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:10.404+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:11.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:11.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:11.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 msgr2=0x7f83f00144f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:11.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:11.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:11.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:11.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:37:11.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f000b820 2024-03-04T08:37:11.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0007af0 2024-03-04T08:37:11.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f0014a30 2024-03-04T08:37:11.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 msgr2=0x7f83f00144f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:11.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b820 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:11.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:11.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:11.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:11.996+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:12.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:12.200+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b820 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:12.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:12.200+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 msgr2=0x7f83f00144f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:12.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:12.200+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:12.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:12.200+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:12.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:12.200+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:12.604+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b820 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:12.604+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 msgr2=0x7f83f00144f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:12.604+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:12.604+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:12.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:12.604+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:13.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:13.408+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b820 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:13.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:13.408+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 msgr2=0x7f83f00144f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:13.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:13.408+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:13.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:13.408+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:13.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:13.408+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:14.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:14.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:14.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 msgr2=0x7f83f00144f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:14.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007af0 0x7f83f00144f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:14.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000b820 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:14.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:14.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f000a900 2024-03-04T08:37:14.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f000c160 2024-03-04T08:37:14.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:14.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:14.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:14.996+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:15.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:15.200+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:15.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:15.200+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:15.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:15.200+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:15.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:15.604+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:15.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:15.604+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:15.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:15.604+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:16.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:16.408+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:16.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:16.408+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:16.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:16.408+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:17.791 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:37:17.795 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:37:17.990 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:17.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:17.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:17.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:17.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:37:17.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f000c160 2024-03-04T08:37:17.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f000a900 2024-03-04T08:37:17.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0014a30 2024-03-04T08:37:17.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:17.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:17.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:17.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:17.996+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:18.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:18.200+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:18.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:18.200+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:18.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:18.200+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:18.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:18.200+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:18.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:18.604+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:18.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:18.604+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:18.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:18.604+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:18.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:18.604+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:19.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:19.408+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:19.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:19.408+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:19.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:19.408+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:19.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:19.408+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:20.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 msgr2=0x7f83f0016e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:20.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014a30 0x7f83f0016e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:20.991 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f00073e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 0x7f83f0007af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f000b7f0 2024-03-04T08:37:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f00124e0 2024-03-04T08:37:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 msgr2=0x7f83f0007af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 0x7f83f0007af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:20.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:20.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:20.996+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:21.200+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 msgr2=0x7f83f0007af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:21.200+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 0x7f83f0007af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:21.200+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:21.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:21.200+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:21.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:21.604+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 msgr2=0x7f83f0007af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:21.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:21.604+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 0x7f83f0007af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:21.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:21.604+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:21.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:21.604+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:22.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:22.408+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 msgr2=0x7f83f0007af0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:22.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:22.408+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 0x7f83f0007af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:22.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:22.408+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:22.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:22.408+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:23.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:23.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:23.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 msgr2=0x7f83f0007af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:23.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000b7f0 0x7f83f0007af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:23.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 0x7f83f0018010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:23.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:23.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:37:23.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:23.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0017c30 2024-03-04T08:37:23.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f000c160 2024-03-04T08:37:23.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f000cd30 con 0x7f83f00124e0 2024-03-04T08:37:23.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 msgr2=0x7f83f0018010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:23.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 0x7f83f0018010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:23.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:23.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:23.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:23.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:23.996+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:24.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.200+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 msgr2=0x7f83f0018010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:24.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.200+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 0x7f83f0018010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:24.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.200+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:24.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.200+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:24.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.200+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:24.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.200+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:24.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.604+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 msgr2=0x7f83f0018010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:24.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.604+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 0x7f83f0018010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:24.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.604+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:24.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.604+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:24.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.604+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:24.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:24.604+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:25.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:25.408+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 msgr2=0x7f83f0018010 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:25.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:25.408+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 0x7f83f0018010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:25.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:25.408+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:25.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:25.408+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:25.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:25.408+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:25.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:25.408+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:26.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:26.996+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:26.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:26.996+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:26.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:26.996+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:26.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:26.996+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:26.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:26.996+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 msgr2=0x7f83f0018010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:26.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:26.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0017c30 0x7f83f0018010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:26.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:26.996+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f0014a30 2024-03-04T08:37:26.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:26.996+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f000a900 2024-03-04T08:37:26.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:26.996+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014a30 legacy=0x7f83f0014e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:26.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:26.996+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0016530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:27.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:27.200+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014a30 legacy=0x7f83f0014e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:27.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:27.200+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0016530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:27.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:27.604+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014a30 legacy=0x7f83f0014e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:27.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:27.604+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0016530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:28.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:28.408+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014a30 legacy=0x7f83f0014e10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:28.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:28.408+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0016530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:29.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0016530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:29.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014a30 legacy=0x7f83f0014e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:29.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:29.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:37:29.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:29.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f000a900 2024-03-04T08:37:29.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f000c160 2024-03-04T08:37:29.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f00124e0 2024-03-04T08:37:29.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0016530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:29.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:29.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:29.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:29.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:29.996+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:30.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:30.200+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0016530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:30.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:30.200+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:30.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:30.200+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:30.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:30.200+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:30.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:30.200+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:30.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:30.604+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:30.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:30.604+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0016530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:30.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:30.604+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:30.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:30.604+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:30.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:30.604+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:31.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:31.408+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:31.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:31.408+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0016530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:31.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:31.408+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:31.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:31.408+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:31.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:31.408+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:32.992 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0016530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f0014f10 2024-03-04T08:37:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0016d70 2024-03-04T08:37:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 msgr2=0x7f83f00149d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:32.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014f10 legacy=0x7f83f0015320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:32.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:32.996+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:33.200+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 msgr2=0x7f83f00149d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:33.200+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014f10 legacy=0x7f83f0015320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:33.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:33.200+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:33.604+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 msgr2=0x7f83f00149d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:33.604+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014f10 legacy=0x7f83f0015320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:33.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:33.604+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:34.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:34.408+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014f10 legacy=0x7f83f0015320 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:34.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:34.408+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 msgr2=0x7f83f00149d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:34.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:34.408+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:35.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 msgr2=0x7f83f00149d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:35.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:35.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014f10 legacy=0x7f83f0015320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:35.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:37:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f000c160 2024-03-04T08:37:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0016d70 2024-03-04T08:37:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f00124e0 2024-03-04T08:37:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 msgr2=0x7f83f00149a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f0014ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:35.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:35.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:35.996+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:36.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:36.200+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 msgr2=0x7f83f00149a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:36.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:36.200+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f0014ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:36.200+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:36.200+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:36.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:36.200+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:36.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:36.604+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 msgr2=0x7f83f00149a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:36.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:36.604+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f0014ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:36.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:36.604+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:36.597 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:36.604+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:36.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:36.604+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:37.401 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:37.408+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 msgr2=0x7f83f00149a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:37.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:37.408+0000 7f8412800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f0014ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:37.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:37.408+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:37.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:37.408+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:37.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:37.408+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:38.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:38.996+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:38.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:38.996+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:38.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:38.996+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 msgr2=0x7f83f00149a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:38.993 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:38.996+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016d70 0x7f83f00149a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:38.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:38.996+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f0014ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:38.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:38.996+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 0x7f83f0005cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:38.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:38.996+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f000a900 2024-03-04T08:37:38.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:38.996+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0016600 2024-03-04T08:37:38.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:39.000+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:38.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:39.000+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:38.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:39.000+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:39.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:39.204+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:39.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:39.204+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:39.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:39.204+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:39.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:39.608+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:39.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:39.608+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:39.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:39.608+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:40.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:40.412+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:40.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:40.412+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:40.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:40.412+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:41.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 msgr2=0x7f83f0005cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:41.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0016600 0x7f83f0005cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:41.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000a900 legacy=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:41.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:41.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:37:41.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 0x7f83f0016d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:41.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:41.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f000c160 2024-03-04T08:37:41.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f0014900 2024-03-04T08:37:41.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f000cd30 con 0x7f83f00124e0 2024-03-04T08:37:41.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:41.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 0x7f83f0016d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:41.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:41.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:41.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:41.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.000+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:42.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.204+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:42.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.204+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 0x7f83f0016d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:42.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.204+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:42.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.204+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:42.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.204+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:42.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.204+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:42.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.608+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:42.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.608+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 0x7f83f0016d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:42.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.608+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:42.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.608+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:42.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.608+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:42.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:42.608+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:43.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:43.412+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:43.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:43.412+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 0x7f83f0016d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:43.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:43.412+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:43.412+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:43.412+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:43.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:43.412+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:44.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.000+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:44.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.000+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f00124e0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:44.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.000+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:44.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.000+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:44.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.000+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:44.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.000+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000c160 0x7f83f0016d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:44.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.000+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f000a900 2024-03-04T08:37:44.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.000+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f0007cd0 2024-03-04T08:37:44.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.000+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0015220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:44.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.000+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:45.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.204+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0015220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:45.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.204+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:45.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.608+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0015220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:45.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:45.608+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:46.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:46.412+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0015220 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:46.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:46.412+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:47.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:37:47.911 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:37:47.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:47.994 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0015220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:37:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f000a900 2024-03-04T08:37:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0007cd0 2024-03-04T08:37:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f0014900 2024-03-04T08:37:47.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:47.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:47.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:47.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:47.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.000+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:48.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.204+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:48.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.204+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:48.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.204+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:48.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.204+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:48.198 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.204+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:48.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.608+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:48.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.608+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:48.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.608+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:48.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.608+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:48.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:48.608+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:49.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:49.412+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:49.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:49.412+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:49.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:49.412+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:49.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:49.412+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:49.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:49.412+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:50.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:50.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0014ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:50.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:50.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:50.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:50.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:50.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f0016d00 2024-03-04T08:37:50.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f000c160 2024-03-04T08:37:50.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0016d00 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:50.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:50.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.000+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:51.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.204+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:51.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.204+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0016d00 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:51.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.204+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.608+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0016d00 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.608+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:51.608+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:52.412+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:52.412+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0016d00 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:52.406 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:52.412+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:53.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0016d00 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:37:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f000a900 2024-03-04T08:37:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f0016d00 2024-03-04T08:37:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0007cd0 2024-03-04T08:37:53.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:53.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:53.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:53.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.000+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0016d00 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:54.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.203+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:54.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.203+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:54.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.203+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:54.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.203+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0016d00 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:54.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.607+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:54.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.607+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:54.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.607+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:54.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:54.607+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0016d00 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:55.411+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:55.411+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:37:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:55.411+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:55.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:55.411+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0016d00 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:37:56.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:56.995 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:56.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0016d00 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:56.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:56.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:56.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 0x7f83f0014370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:56.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f00170e0 2024-03-04T08:37:56.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f000c160 2024-03-04T08:37:56.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:56.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 msgr2=0x7f83f0014370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:56.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 0x7f83f0014370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:56.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:56.999+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:57.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:57.203+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:57.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:57.203+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 msgr2=0x7f83f0014370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:57.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:57.203+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 0x7f83f0014370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:57.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:57.203+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:37:57.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:57.607+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 msgr2=0x7f83f0014370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:57.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:57.607+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:57.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:57.607+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 0x7f83f0014370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:57.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:57.607+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:37:58.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:58.411+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 msgr2=0x7f83f0014370 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:58.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:58.411+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:58.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:58.411+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 0x7f83f0014370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:58.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:58.411+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:37:59.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:59.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:59.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 msgr2=0x7f83f0014370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:37:59.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f00170e0 0x7f83f0014370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:37:59.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:59.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:37:59.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:37:59.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f000a900 2024-03-04T08:37:59.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0007cd0 2024-03-04T08:37:59.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f000c160 2024-03-04T08:37:59.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:37:59.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:37:59.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:37:59.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:37:59.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:37:59.999+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:00.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:00.203+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:00.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:00.203+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:00.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:00.203+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:00.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:00.203+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:00.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:00.203+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:00.607+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:00.607+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:00.607+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:00.607+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:00.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:00.607+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:01.411+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:01.411+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014900 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:01.411+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:01.411+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:01.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:01.411+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f0014900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f0014900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:02.996 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f0012360 2024-03-04T08:38:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0016560 2024-03-04T08:38:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0014e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:02.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:02.999+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:03.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:03.203+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:03.203+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:03.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:03.203+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0014e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:03.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:03.607+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:03.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:03.607+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:03.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:03.607+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0014e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:04.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:04.411+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:04.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:04.411+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:04.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:04.411+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0014e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:05.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:05.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:05.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0014e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:05.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:05.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:38:05.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:05.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f0012360 2024-03-04T08:38:05.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0016560 2024-03-04T08:38:05.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f000c160 2024-03-04T08:38:05.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:05.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:05.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0014e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:05.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:05.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:05.999+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:06.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:06.203+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:06.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:06.203+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:06.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:06.203+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:06.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:06.203+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0014e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:06.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:06.203+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:06.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:06.607+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:06.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:06.607+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:06.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:06.607+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:06.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:06.607+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0014e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:06.601 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:06.607+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:07.411+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:07.411+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:07.411+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0014e40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:07.411+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:07.405 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:07.411+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:08.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:08.999+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:08.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:08.999+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000c160 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:08.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:08.999+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:08.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:08.999+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:08.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:08.999+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0014e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:08.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:08.999+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:08.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.003+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f0007cd0 2024-03-04T08:38:08.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.003+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0014900 2024-03-04T08:38:08.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.003+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:08.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.003+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:08.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.003+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:09.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.207+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:09.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.207+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:09.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.207+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:09.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.611+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:09.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.611+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:09.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:09.611+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:10.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:10.415+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:10.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:10.415+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:10.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:10.415+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:11.997 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:11.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:11.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:11.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:38:11.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:11.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:11.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f0012360 2024-03-04T08:38:11.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0016560 2024-03-04T08:38:11.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f0014900 2024-03-04T08:38:11.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:11.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:11.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:11.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:11.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.003+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.207+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.207+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.207+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.207+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:12.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.207+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:12.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.611+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:12.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.611+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:12.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.611+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:12.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.611+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:12.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:12.611+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:13.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:13.415+0000 7f8411400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:13.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:13.415+0000 7f8411400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:13.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:13.415+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:13.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:13.415+0000 7f8412800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:13.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:13.415+0000 7f8412800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:14.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:14.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:14.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:14.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:14.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0012360 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:14.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:14.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f000c160 2024-03-04T08:38:14.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0007cd0 2024-03-04T08:38:14.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:14.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:14.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.003+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:15.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.207+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:15.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.207+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:15.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.207+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:15.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.611+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:15.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.611+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:15.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:15.611+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:16.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:16.415+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:16.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:16.415+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:16.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:16.415+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:17.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:17.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:17.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:17.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:17.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:38:17.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:17.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:17.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0016560 2024-03-04T08:38:17.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f0014900 2024-03-04T08:38:17.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f000cd30 con 0x7f83f0007cd0 2024-03-04T08:38:18.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:18.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:18.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:18.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:18.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:18.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.003+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:18.017 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:38:18.021 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:38:18.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.207+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:18.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.207+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:18.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.207+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:18.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.207+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:18.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.207+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:18.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.207+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:18.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.611+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:18.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.611+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:18.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.611+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:18.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.611+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:18.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.611+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:18.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:18.611+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:19.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:19.415+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:19.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:19.415+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:19.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:19.415+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:19.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:19.415+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:19.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:19.415+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:19.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:19.415+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:20.998 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.003+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 msgr2=0x7f83f00176f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:20.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.003+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f0007cd0 0x7f83f00176f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:20.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.003+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:20.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.003+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:20.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.003+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 msgr2=0x7f83f0016970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:20.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.003+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0016560 0x7f83f0016970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:21.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.003+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f000a900 2024-03-04T08:38:21.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.003+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f000c160 2024-03-04T08:38:21.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.003+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:21.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.003+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:21.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.207+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:21.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.207+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:21.604 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.611+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:21.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:21.611+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:22.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:22.415+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:22.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:22.415+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:23.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:23.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000a900 legacy=0x7f83f0014ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:23.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:38:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f0014370 2024-03-04T08:38:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f000c160 2024-03-04T08:38:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0014900 2024-03-04T08:38:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:24.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:24.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.003+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:24.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.207+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:24.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.207+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:24.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.207+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:24.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.207+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:24.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.611+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:24.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.611+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:24.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.611+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:24.651 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:24.611+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:25.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:25.415+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:25.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:25.415+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:25.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:25.415+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:25.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:25.415+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:26.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f0016eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:27.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f0014900 0x7f83f0016eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:27.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f000c160 legacy=0x7f83f000f4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:27.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f0010be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:27.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 0x7f83f000d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:27.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:27.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f00176f0 2024-03-04T08:38:27.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f000d9f0 2024-03-04T08:38:27.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:27.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:27.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:27.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.003+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:27.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.207+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:27.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.207+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:27.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.207+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:27.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.207+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:27.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.611+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:27.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.611+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:27.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.611+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:27.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:27.611+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:28.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:28.415+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:28.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:28.415+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:28.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:28.415+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:28.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:28.415+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:30.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:30.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:30.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:30.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f00176f0 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:30.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 0x7f83f000e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:30.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:38:30.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:30.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f0014370 2024-03-04T08:38:30.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f000e310 2024-03-04T08:38:30.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f000d9f0 2024-03-04T08:38:30.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 msgr2=0x7f83f000e6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:30.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f000ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:30.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 0x7f83f000e6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:30.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:30.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.003+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:30.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.207+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 msgr2=0x7f83f000e6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:30.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.207+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f000ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:30.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.207+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 0x7f83f000e6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:30.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.207+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:30.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.207+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:30.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.611+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f000ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:30.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.611+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 msgr2=0x7f83f000e6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:30.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.611+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 0x7f83f000e6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:30.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.611+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:30.606 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:30.611+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:31.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:31.415+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f000ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:31.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:31.415+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 msgr2=0x7f83f000e6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:31.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:31.415+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 0x7f83f000e6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:31.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:31.415+0000 7f8412800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:31.410 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:31.415+0000 7f8412800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:32.999 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:33.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:33.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 msgr2=0x7f83f000e6f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:33.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f000e310 0x7f83f000e6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:33.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f000ec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:33.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f000d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:33.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f0007cd0 2024-03-04T08:38:33.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f000c160 2024-03-04T08:38:33.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f0011bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:33.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:33.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.007+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:33.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.211+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f0011bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.211+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:33.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.211+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.615+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.615+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f0011bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:33.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:33.615+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:34.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:34.419+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:34.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:34.419+0000 7f8411e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f0011bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:34.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:34.419+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:36.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:36.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:36.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f0011bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:38:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f0014370 2024-03-04T08:38:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f0017520 con 0x7f83f0007cd0 2024-03-04T08:38:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f000d9f0 2024-03-04T08:38:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f000ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:36.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f0011360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:36.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.007+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:36.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.211+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f000ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:36.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.211+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f0011360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:36.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.211+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:36.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.211+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:36.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.615+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f000ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:36.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.615+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:36.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.615+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f0011360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:36.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:36.615+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:37.412 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:37.419+0000 7f8411e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f000ec30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:37.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:37.419+0000 7f8411400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:37.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:37.419+0000 7f8412800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f0011360 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:37.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:37.419+0000 7f8411400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:39.000 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8407400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 msgr2=0x7f83f000ddd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:39.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000d9f0 0x7f83f000ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:39.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f0011360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:39.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8407400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f0014370 legacy=0x7f83f000ec30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:39.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 0x7f83f000d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:39.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:39.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0014900 2024-03-04T08:38:39.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f000c160 2024-03-04T08:38:39.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:39.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:39.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:39.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.003+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:39.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.207+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:39.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.207+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:39.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.207+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:39.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.207+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:39.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.611+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:39.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.611+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:39.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.611+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:39.605 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:39.611+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:40.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:40.415+0000 7f8411e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:40.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:40.415+0000 7f8411e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:40.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:40.415+0000 7f8411400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:40.409 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:40.415+0000 7f8411400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:42.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:42.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:42.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:42.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014900 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:42.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8412800640 1 -- reap_dead start 2024-03-04T08:38:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 0x7f83f000ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8407400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2080 con 0x7f83f0007cd0 2024-03-04T08:38:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8407400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f0014370 2024-03-04T08:38:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8407400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840c0c2360 con 0x7f83f000c160 2024-03-04T08:38:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 msgr2=0x7f83f000ea20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 0x7f83f000ea20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:42.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000e310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:42.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:42.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.007+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:42.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.211+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 msgr2=0x7f83f000ea20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.211+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 0x7f83f000ea20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.211+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.211+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:42.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.211+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000e310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:42.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.615+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 msgr2=0x7f83f000ea20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:42.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.615+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 0x7f83f000ea20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:42.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.615+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:42.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.615+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000e310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:42.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:42.615+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:43.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:43.419+0000 7f8412800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 msgr2=0x7f83f000ea20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:43.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:43.419+0000 7f8412800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 0x7f83f000ea20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:43.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:43.419+0000 7f8411e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:43.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:43.419+0000 7f8411400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000e310 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:43.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:43.419+0000 7f8411e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:45.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8407400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 msgr2=0x7f83f0016cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:45.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8407400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f83f000c160 0x7f83f0016cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:45.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8407400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 msgr2=0x7f83f000ea20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:45.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8407400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f83f0014370 0x7f83f000ea20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:45.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8407400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f83f0007cd0 legacy=0x7f83f000e310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:45.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8407400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000ef60 0x7f83f000d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:45.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8407400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c0c2640 con 0x7f83f000d9f0 2024-03-04T08:38:45.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8407400640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c15ae00 con 0x7f83f000ef60 2024-03-04T08:38:45.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000d9f0 legacy=0x7f83f0010350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:45.002 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000ef60 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:45.003 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.007+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000ef60 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:45.204 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.211+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000d9f0 legacy=0x7f83f0010350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:45.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.211+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000ef60 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:45.205 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.211+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000ef60 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:45.608 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.615+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000d9f0 legacy=0x7f83f0010350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.615+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000ef60 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:45.609 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:45.615+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000ef60 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:46.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:46.419+0000 7f8411400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f83f000d9f0 legacy=0x7f83f0010350 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:46.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:46.419+0000 7f8411e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000ef60 msgr2=0x7f83f000d4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:46.413 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:46.419+0000 7f8411e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f83f000ef60 0x7f83f000d4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:47.865 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-03-04T08:38:47.866 INFO:tasks.ceph:Scrubbing osd.4 2024-03-04T08:38:47.866 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-04T08:38:48.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.019+0000 7ff3e1c00640 1 Processor -- start 2024-03-04T08:38:48.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.019+0000 7ff3e1c00640 1 -- start start 2024-03-04T08:38:48.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.023+0000 7ff3e1c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3dc0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:48.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.023+0000 7ff3e1c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3dc0c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:48.017 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.023+0000 7ff3e1c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3dc152cf0 2024-03-04T08:38:48.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.023+0000 7ff3e1c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3dc070cd0 2024-03-04T08:38:48.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.023+0000 7ff3e1c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3dc0c8d10 2024-03-04T08:38:48.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.023+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3dc0c8210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:48.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.023+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3dc0c8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:48.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.023+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3dc152cf0 legacy=0x7ff3dc15be80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:48.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.023+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3dc0c9120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:48.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.023+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3dc0c9120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:48.137 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:38:48.140 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:38:48.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.227+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3dc0c8210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:48.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.227+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3dc0c8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:48.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.227+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3dc152cf0 legacy=0x7ff3dc15be80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:48.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.227+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3dc0c9120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:48.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.227+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3dc0c9120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:48.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.631+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3dc0c8210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:48.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.631+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3dc0c8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:48.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.631+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3dc152cf0 legacy=0x7ff3dc15be80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:48.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.631+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3dc0c9120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:48.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:48.631+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3dc0c9120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:49.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:49.435+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3dc0c8210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:49.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:49.435+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3dc0c8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:49.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:49.435+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3dc152cf0 legacy=0x7ff3dc15be80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:49.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:49.435+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3dc0c9120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:49.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:49.435+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3dc0c9120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:51.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3dc0c9120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:51.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3dc0c9120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:51.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3dc0c8210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:51.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3dc0c8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3dc152cf0 legacy=0x7ff3dc15be80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 0x7ff3b80098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158bd0 con 0x7ff3b80036d0 2024-03-04T08:38:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8007410 2024-03-04T08:38:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80036d0 legacy=0x7ff3b8005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.022+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.226+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80036d0 legacy=0x7ff3b8005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.226+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.226+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:51.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.630+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80036d0 legacy=0x7ff3b8005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.630+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:51.630+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:52.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:52.434+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80036d0 legacy=0x7ff3b8005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:52.434+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:52.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:52.434+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:54.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:54.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8007410 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:54.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80036d0 legacy=0x7ff3b8005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:54.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:38:54.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:54.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:54.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:38:54.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3dc070cd0 2024-03-04T08:38:54.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b800a9f0 2024-03-04T08:38:54.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b8006280 con 0x7ff3dc0c8d10 2024-03-04T08:38:54.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:54.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:54.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:54.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:54.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:54.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.022+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:38:54.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.226+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.226+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.226+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.226+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.226+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:54.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.226+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:38:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.630+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.630+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.630+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.630+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.630+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:54.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:54.630+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:38:55.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:55.434+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:38:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:55.434+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:55.434+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:38:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:55.434+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:38:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:55.434+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:55.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:55.434+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:38:57.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.022+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:57.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.022+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:57.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.022+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:57.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:57.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.022+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:38:57.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:38:57.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.022+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b8006470 2024-03-04T08:38:57.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.022+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158bd0 con 0x7ff3b8006960 2024-03-04T08:38:57.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.022+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8006470 legacy=0x7ff3b8005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:57.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.022+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8006960 legacy=0x7ff3b80121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:57.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.226+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8006470 legacy=0x7ff3b8005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:57.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.226+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8006960 legacy=0x7ff3b80121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:57.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.630+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8006470 legacy=0x7ff3b8005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:57.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:57.630+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8006960 legacy=0x7ff3b80121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:38:58.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:58.434+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8006470 legacy=0x7ff3b8005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:38:58.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:38:58.434+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8006960 legacy=0x7ff3b80121f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:00.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8006960 legacy=0x7ff3b80121f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:00.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8006470 legacy=0x7ff3b8005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:00.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:00.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:00.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:39:00.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:00.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3dc070cd0 2024-03-04T08:39:00.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b800a9f0 2024-03-04T08:39:00.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b8006280 con 0x7ff3dc0c8d10 2024-03-04T08:39:00.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:00.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:00.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:00.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:00.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:00.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.022+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:00.219 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.226+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.226+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.226+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.226+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.226+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:00.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.226+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:00.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.630+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:00.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.630+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:00.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.630+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:00.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.630+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:00.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.630+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:00.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:00.630+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:01.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:01.434+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:01.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:01.434+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:01.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:01.434+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:01.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:01.434+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:01.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:01.434+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:01.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:01.434+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.026+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.026+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3dc0c8d10 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.026+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 msgr2=0x7ff3b80098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:03.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a9f0 0x7ff3b80098d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.026+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b8003b40 2024-03-04T08:39:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b80108f0 2024-03-04T08:39:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.026+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8003b40 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:03.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.026+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b80108f0 legacy=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:03.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.230+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8003b40 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:03.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.230+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b80108f0 legacy=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:03.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.634+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8003b40 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:03.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:03.634+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b80108f0 legacy=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:04.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:04.438+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8003b40 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:04.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:04.438+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b80108f0 legacy=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:06.019 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b80108f0 legacy=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8003b40 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:39:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158bd0 con 0x7ff3b80088d0 2024-03-04T08:39:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b8003b40 2024-03-04T08:39:06.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3dc070cd0 2024-03-04T08:39:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80088d0 legacy=0x7ff3b8008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:06.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.026+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8003b40 legacy=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.230+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80088d0 legacy=0x7ff3b8008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:06.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.230+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:06.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.230+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:06.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.230+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8003b40 legacy=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:06.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.634+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80088d0 legacy=0x7ff3b8008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:06.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.634+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:06.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.634+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:06.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:06.634+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8003b40 legacy=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:07.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:07.438+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:07.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:07.438+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80088d0 legacy=0x7ff3b8008d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:07.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:07.438+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:07.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:07.438+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8003b40 legacy=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3dc070cd0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8003b40 legacy=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:09.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80088d0 legacy=0x7ff3b8008d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b80101e0 2024-03-04T08:39:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b8009960 2024-03-04T08:39:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 msgr2=0x7ff3b8009420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:09.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.022+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:09.220 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.226+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 msgr2=0x7ff3b8009420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:09.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.226+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:09.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.226+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:09.221 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.226+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:09.624 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.630+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:09.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.630+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 msgr2=0x7ff3b8009420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:09.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.630+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:09.625 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:09.630+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:10.428 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:10.434+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 msgr2=0x7ff3b8009420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:10.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:10.434+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:10.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:10.434+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:10.429 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:10.434+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:12.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:12.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:12.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 msgr2=0x7ff3b8009420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:12.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:12.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:12.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:39:12.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:12.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b8008d10 2024-03-04T08:39:12.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b80101e0 2024-03-04T08:39:12.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b8009960 2024-03-04T08:39:12.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:12.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:12.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 msgr2=0x7ff3b8009420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:12.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:12.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.026+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:12.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.230+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:12.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.230+0000 7ff3db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:12.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.230+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 msgr2=0x7ff3b8009420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:12.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.230+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:12.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.230+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:12.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.634+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:12.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.634+0000 7ff3db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:12.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.634+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 msgr2=0x7ff3b8009420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:12.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.634+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:12.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:12.634+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:13.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:13.438+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 msgr2=0x7ff3b8009420 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:13.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:13.438+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:13.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:13.438+0000 7ff3db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:13.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:13.438+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:13.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:13.438+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:15.020 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:15.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009960 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:15.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 msgr2=0x7ff3b8009420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:15.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80101e0 0x7ff3b8009420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:15.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:15.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:15.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158bd0 con 0x7ff3b8005b50 2024-03-04T08:39:15.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b80088d0 2024-03-04T08:39:15.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:15.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8005b50 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:15.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.026+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:15.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.230+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:15.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.230+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:15.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.230+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8005b50 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:15.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.634+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8005b50 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:15.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.634+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:15.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:15.634+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:16.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:16.438+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8005b50 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:16.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:16.438+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:16.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:16.438+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:18.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:18.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:18.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8005b50 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:18.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:39:18.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:18.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 0x7ff3b80090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:18.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b8008d10 2024-03-04T08:39:18.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b80088d0 2024-03-04T08:39:18.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b8005b30 2024-03-04T08:39:18.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:18.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:18.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 msgr2=0x7ff3b80090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:18.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:18.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.026+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 0x7ff3b80090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:18.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.230+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:18.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.230+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:18.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.230+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 msgr2=0x7ff3b80090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:18.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.230+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:18.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.230+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 0x7ff3b80090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:18.490 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:39:18.494 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:39:18.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.634+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:18.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.634+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:18.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.634+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 msgr2=0x7ff3b80090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:18.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.634+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:18.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:18.634+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 0x7ff3b80090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:19.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:19.438+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:19.438+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:19.438+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 msgr2=0x7ff3b80090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:19.438+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:19.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:19.438+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 0x7ff3b80090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:21.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 msgr2=0x7ff3b80090f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:21.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8005b30 0x7ff3b80090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:21.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:21.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b80088d0 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:21.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:21.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 0x7ff3b800f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:21.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158bd0 con 0x7ff3b8009630 2024-03-04T08:39:21.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8009a40 2024-03-04T08:39:21.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8009630 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:21.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 msgr2=0x7ff3b800f8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:21.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.026+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 0x7ff3b800f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:21.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.230+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8009630 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:21.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.230+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 msgr2=0x7ff3b800f8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:21.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.230+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 0x7ff3b800f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:21.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.634+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8009630 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:21.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.634+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 msgr2=0x7ff3b800f8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:21.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:21.634+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 0x7ff3b800f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:22.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:22.438+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8009630 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:22.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:22.438+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 msgr2=0x7ff3b800f8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:22.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:22.438+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 0x7ff3b800f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:24.021 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 msgr2=0x7ff3b800f8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:24.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8009a40 0x7ff3b800f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:24.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b8009630 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:24.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:39:24.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 0x7ff3b800a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:24.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b800fed0 2024-03-04T08:39:24.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158bd0 con 0x7ff3b8008d10 2024-03-04T08:39:24.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8005b30 2024-03-04T08:39:24.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8017d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:24.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:24.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:24.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.026+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:24.224 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.230+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.230+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8017d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.230+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:24.225 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.230+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:24.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.634+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:24.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.634+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8017d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:24.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.634+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:24.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:24.634+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:25.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:25.438+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8017d10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:25.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:25.438+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:25.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:25.438+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:25.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:25.438+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:27.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:27.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8005b30 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:27.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008d10 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:27.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8017d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:27.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 0x7ff3b80090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:27.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:27.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8008890 2024-03-04T08:39:27.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b800f1d0 2024-03-04T08:39:27.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 msgr2=0x7ff3b80090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:27.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 msgr2=0x7ff3b800f5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:27.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 0x7ff3b80090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:27.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.026+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:27.222 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.230+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 msgr2=0x7ff3b80090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:27.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.230+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 msgr2=0x7ff3b800f5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:27.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.230+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 0x7ff3b80090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:27.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.230+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:27.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.634+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 msgr2=0x7ff3b800f5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:27.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.634+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 msgr2=0x7ff3b80090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:27.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.634+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 0x7ff3b80090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:27.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:27.634+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:28.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:28.438+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 msgr2=0x7ff3b800f5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:28.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:28.438+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 msgr2=0x7ff3b80090f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:28.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:28.438+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 0x7ff3b80090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:28.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:28.438+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:30.022 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 msgr2=0x7ff3b800f5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:30.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:30.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 msgr2=0x7ff3b80090f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:30.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008890 0x7ff3b80090f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:30.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:30.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:39:30.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:30.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b800fed0 2024-03-04T08:39:30.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8008cb0 2024-03-04T08:39:30.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b800f1d0 2024-03-04T08:39:30.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:30.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 msgr2=0x7ff3b800f5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:30.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:30.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:30.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.026+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:30.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.230+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:30.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.230+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:30.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.230+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 msgr2=0x7ff3b800f5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:30.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.230+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:30.223 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.230+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:30.628 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.634+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:30.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.634+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:30.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.634+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 msgr2=0x7ff3b800f5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:30.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.634+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:30.629 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:30.634+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:31.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:31.438+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:31.438+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 msgr2=0x7ff3b800f5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:31.438+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:31.438+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:31.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:31.438+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:33.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 msgr2=0x7ff3b800f5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:33.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800f1d0 0x7ff3b800f5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:33.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:33.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:33.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:33.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:33.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b8005b30 2024-03-04T08:39:33.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8008850 2024-03-04T08:39:33.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:33.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:33.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.030+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8005b30 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:33.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.234+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:33.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.234+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:33.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.234+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8005b30 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:33.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.638+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:33.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.638+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:33.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:33.638+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8005b30 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:34.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:34.442+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:34.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:34.442+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:34.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:34.442+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8005b30 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:36.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:36.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:36.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8005b30 legacy=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:39:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b8016410 2024-03-04T08:39:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8008850 2024-03-04T08:39:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b8008cb0 2024-03-04T08:39:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:36.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:36.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8016410 legacy=0x7ff3b8018800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:36.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:36.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.030+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:36.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.234+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:36.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.234+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:36.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.234+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8016410 legacy=0x7ff3b8018800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:36.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.234+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:36.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.234+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:36.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.638+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:36.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.638+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:36.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.638+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:36.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.638+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8016410 legacy=0x7ff3b8018800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:36.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:36.638+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:37.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:37.442+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:37.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:37.442+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:37.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:37.442+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:37.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:37.442+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8016410 legacy=0x7ff3b8018800 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:37.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:37.442+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:39.023 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8016410 legacy=0x7ff3b8018800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 0x7ff3b8009f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b800f1d0 2024-03-04T08:39:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b800fed0 2024-03-04T08:39:39.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800f1d0 legacy=0x7ff3b800f5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:39.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:39.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.030+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:39.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.234+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800f1d0 legacy=0x7ff3b800f5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:39.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.234+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:39.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.234+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:39.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.638+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800f1d0 legacy=0x7ff3b800f5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:39.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.638+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:39.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:39.638+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:40.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:40.442+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800f1d0 legacy=0x7ff3b800f5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:40.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:40.442+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:40.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:40.442+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 msgr2=0x7ff3b8009f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800fed0 0x7ff3b8009f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800f1d0 legacy=0x7ff3b800f5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:42.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:42.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:39:42.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:42.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b800f1d0 2024-03-04T08:39:42.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8008850 2024-03-04T08:39:42.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b8008cb0 2024-03-04T08:39:42.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:42.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800f1d0 legacy=0x7ff3b800f5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:42.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:42.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:42.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.030+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:42.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.234+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800f1d0 legacy=0x7ff3b800f5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.234+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.234+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.234+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:42.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.234+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:42.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.638+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800f1d0 legacy=0x7ff3b800f5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:42.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.638+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:42.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.638+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:42.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.638+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:42.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:42.638+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:43.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:43.442+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800f1d0 legacy=0x7ff3b800f5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:43.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:43.442+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:43.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:43.442+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:43.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:43.442+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:43.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:43.442+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:45.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:45.024 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:45.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 msgr2=0x7ff3b800a4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:45.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8008850 0x7ff3b800a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:45.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800f1d0 legacy=0x7ff3b800f5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:45.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 0x7ff3b8016410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:45.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b800fed0 2024-03-04T08:39:45.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8009f70 2024-03-04T08:39:45.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:45.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 msgr2=0x7ff3b8016410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:45.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.030+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 0x7ff3b8016410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.234+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 msgr2=0x7ff3b8016410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:45.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.234+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:45.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.234+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 0x7ff3b8016410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:45.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.638+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 msgr2=0x7ff3b8016410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:45.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.638+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:45.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:45.638+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 0x7ff3b8016410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:46.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:46.442+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:46.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:46.442+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 msgr2=0x7ff3b8016410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:46.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:46.442+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 0x7ff3b8016410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 msgr2=0x7ff3b8016410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8009f70 0x7ff3b8016410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800fed0 legacy=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:48.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:39:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b800a990 2024-03-04T08:39:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8009f70 2024-03-04T08:39:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b8008cb0 2024-03-04T08:39:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b800a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:48.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:48.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:48.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:48.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.029+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:48.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.233+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b800a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:48.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.233+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:48.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.233+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:48.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.233+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:48.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.233+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:48.617 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:39:48.621 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:39:48.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.637+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:48.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.637+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b800a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:48.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.637+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:48.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.637+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:48.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:48.637+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:49.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:49.441+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:49.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:49.441+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b800a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:49.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:49.441+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:49.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:49.441+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:49.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:49.441+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:51.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:51.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008cb0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:51.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b800a450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:51.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:51.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:51.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:51.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b8008850 2024-03-04T08:39:51.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b800f8e0 2024-03-04T08:39:51.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:51.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008850 legacy=0x7ff3b8015230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:51.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.029+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:51.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.233+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:51.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.233+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008850 legacy=0x7ff3b8015230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:51.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.233+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:51.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.637+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:51.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.637+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008850 legacy=0x7ff3b8015230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:51.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:51.637+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:52.441+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:52.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:52.441+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008850 legacy=0x7ff3b8015230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:52.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:52.441+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:54.025 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:54.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:54.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008850 legacy=0x7ff3b8015230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:54.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:54.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:39:54.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:54.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b800a990 2024-03-04T08:39:54.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8009f70 2024-03-04T08:39:54.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b800f8e0 2024-03-04T08:39:54.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:54.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b800a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:54.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:54.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:54.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.029+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.233+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.233+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b800a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:54.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.233+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:54.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.233+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:54.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.233+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:54.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.637+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b800a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:54.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.637+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:54.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.637+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:54.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.637+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:54.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:54.637+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:55.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:55.441+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b800a450 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:39:55.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:55.441+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:39:55.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:55.441+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:55.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:55.441+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:39:55.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:55.441+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:39:57.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:57.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800f8e0 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b800a450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b800a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:39:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:39:57.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:39:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b8008cb0 2024-03-04T08:39:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8008850 2024-03-04T08:39:57.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:57.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008cb0 legacy=0x7ff3b8007820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:57.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.029+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:39:57.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.233+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:57.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.233+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008cb0 legacy=0x7ff3b8007820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:57.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.233+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:39:57.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.637+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:57.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.637+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008cb0 legacy=0x7ff3b8007820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:57.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:57.637+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:39:58.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:58.441+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008cb0 legacy=0x7ff3b8007820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:39:58.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:58.441+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:39:58.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:39:58.441+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:00.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:00.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:00.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8008cb0 legacy=0x7ff3b8007820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:00.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:40:00.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:00.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b800a990 2024-03-04T08:40:00.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b8015300 2024-03-04T08:40:00.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8008850 2024-03-04T08:40:00.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:00.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:00.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:00.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.029+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.233+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.233+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:00.226 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.233+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:00.227 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.233+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:00.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.637+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:00.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.637+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:00.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.637+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:00.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:00.637+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:01.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:01.441+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007820 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:01.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:01.441+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:01.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:01.441+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:01.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:01.441+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:03.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:03.026 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b8008850 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b8012ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8009f70 2024-03-04T08:40:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b800a420 2024-03-04T08:40:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:03.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:03.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:03.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.033+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:03.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.237+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.237+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.237+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:03.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.237+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:03.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.641+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:03.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.641+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:03.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.641+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:03.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:03.641+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:04.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:04.445+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:04.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:04.445+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:04.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:04.445+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:04.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:04.445+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 msgr2=0x7ff3b8012ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8009f70 0x7ff3b8012ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:06.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8016300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:40:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158bd0 con 0x7ff3b800a420 2024-03-04T08:40:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b8015300 2024-03-04T08:40:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8015f20 2024-03-04T08:40:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 msgr2=0x7ff3b8016300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8016300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:06.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.033+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:06.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.237+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 msgr2=0x7ff3b8016300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.237+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.237+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8016300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:06.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.237+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:06.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.641+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 msgr2=0x7ff3b8016300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:06.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.641+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:06.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.641+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8016300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:06.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:06.641+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:07.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:07.445+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 msgr2=0x7ff3b8016300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:07.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:07.445+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:07.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:07.445+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8016300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:07.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:07.445+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:09.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 msgr2=0x7ff3b8016300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:09.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8016300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:09.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:09.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:09.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:09.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:09.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b80086e0 2024-03-04T08:40:09.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b800a990 2024-03-04T08:40:09.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:09.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:09.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:09.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.033+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:09.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.237+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:09.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.237+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:09.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.237+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:09.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.237+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:09.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.641+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:09.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.641+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:09.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.641+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:09.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:09.641+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:10.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:10.445+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:10.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:10.445+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:10.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:10.445+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:10.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:10.445+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:12.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:12.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:12.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:12.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:12.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:40:12.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8014cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:12.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b800a420 2024-03-04T08:40:12.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158bd0 con 0x7ff3b8015300 2024-03-04T08:40:12.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8015f20 2024-03-04T08:40:12.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:12.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:12.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:12.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.029+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:12.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.233+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:12.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.233+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:12.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.233+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:12.228 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.233+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:12.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.637+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:12.632 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.637+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:12.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.637+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:12.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:12.637+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:13.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:13.441+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:13.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:13.441+0000 7ff3db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:13.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:13.441+0000 7ff3db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:13.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:13.441+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:15.027 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 msgr2=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:15.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015f20 0x7ff3b8014cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:15.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:15.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:15.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:15.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:15.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b80086e0 2024-03-04T08:40:15.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b800a990 2024-03-04T08:40:15.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:15.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:15.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:15.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.033+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:15.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.237+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:15.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.237+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:15.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.237+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:15.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.237+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:15.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.641+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:15.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.641+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:15.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.641+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:15.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:15.641+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:16.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:16.445+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:16.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:16.445+0000 7ff3daa00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:16.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:16.445+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:16.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:16.445+0000 7ff3daa00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:18.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:18.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:18.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:18.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:18.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:18.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:40:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b8015300 2024-03-04T08:40:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b80086e0 2024-03-04T08:40:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b800a990 2024-03-04T08:40:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:18.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:18.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.033+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:18.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.237+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:18.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.237+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.237+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.237+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:18.233 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.237+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:18.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.641+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:18.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.641+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:18.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.641+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:18.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.641+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:18.633 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:18.641+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:18.744 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:40:18.748 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:40:19.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:19.445+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:19.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:19.445+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:19.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:19.445+0000 7ff3d2000640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:19.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:19.445+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:19.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:19.445+0000 7ff3d2000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:21.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 msgr2=0x7ff3b80159e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:21.028 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b800a990 0x7ff3b80159e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:21.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:21.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b80086e0 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:21.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8015300 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:21.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:21.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b800a420 2024-03-04T08:40:21.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8014cf0 2024-03-04T08:40:21.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 msgr2=0x7ff3b8015f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:21.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:21.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.033+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:21.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.237+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:21.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.237+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 msgr2=0x7ff3b8015f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:21.232 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.237+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:21.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.641+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:21.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.641+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 msgr2=0x7ff3b8015f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:21.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:21.641+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:22.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:22.445+0000 7ff3daa00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:22.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:22.445+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 msgr2=0x7ff3b8015f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:22.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:22.445+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:24.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 msgr2=0x7ff3b8015f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:24.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:24.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b800a420 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:24.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:24.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:40:24.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:24.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b80086e0 2024-03-04T08:40:24.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8014cf0 2024-03-04T08:40:24.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b800a420 2024-03-04T08:40:24.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:24.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 msgr2=0x7ff3b8015f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:24.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:24.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:24.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.033+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:24.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.237+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:24.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.237+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 msgr2=0x7ff3b8015f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:24.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.237+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:24.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.237+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:24.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.237+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:24.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.641+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 msgr2=0x7ff3b8015f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:24.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.641+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:24.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.641+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:24.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.641+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:24.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:24.641+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:25.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:25.445+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:25.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:25.445+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 msgr2=0x7ff3b8015f20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:25.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:25.445+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:25.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:25.445+0000 7ff3db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:25.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:25.445+0000 7ff3db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:27.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:27.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b80090b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:27.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 msgr2=0x7ff3b8015f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:27.029 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8014cf0 0x7ff3b8015f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:27.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:27.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:27.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b800a990 2024-03-04T08:40:27.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b80159e0 2024-03-04T08:40:27.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:27.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b8016460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:27.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.033+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:27.229 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.237+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:27.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.237+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b8016460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:27.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.237+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:27.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.641+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:27.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.641+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b8016460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:27.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:27.641+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:28.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:28.445+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b8016460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:28.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:28.445+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:28.438 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:28.445+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:30.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b8016460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:30.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:30.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:30.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:30.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:30.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:40:30.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b80086e0 2024-03-04T08:40:30.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b800a420 2024-03-04T08:40:30.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b80159e0 2024-03-04T08:40:30.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:30.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b8016460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:30.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:30.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:30.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.037+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:30.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.241+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:30.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.241+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b8016460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:30.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.241+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.241+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.241+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:30.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.645+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:30.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.645+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:30.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.645+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b8016460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:30.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.645+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:30.640 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:30.645+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:31.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:31.449+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:31.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:31.449+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b8016460 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:31.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:31.449+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:31.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:31.449+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:31.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:31.449+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b8016460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b8016460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:33.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 0x7ff3b8015df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b8007f30 2024-03-04T08:40:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b800a990 2024-03-04T08:40:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 msgr2=0x7ff3b8015df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 0x7ff3b8015df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:33.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.037+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8007f30 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:33.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.241+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 msgr2=0x7ff3b8015df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:33.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.241+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 0x7ff3b8015df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:33.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.241+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8007f30 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:33.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.645+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 msgr2=0x7ff3b8015df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:33.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.645+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 0x7ff3b8015df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:33.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:33.645+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8007f30 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:34.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:34.449+0000 7ff3daa00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 msgr2=0x7ff3b8015df0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:34.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:34.449+0000 7ff3daa00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 0x7ff3b8015df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:34.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:34.449+0000 7ff3d2000640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8007f30 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:36.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 msgr2=0x7ff3b8015df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:36.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b800a990 0x7ff3b8015df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:36.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b8007f30 legacy=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:36.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:36.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:36.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:40:36.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b80086e0 2024-03-04T08:40:36.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b800a420 2024-03-04T08:40:36.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b80159e0 2024-03-04T08:40:36.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:36.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b80163d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:36.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:36.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:36.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.033+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:36.230 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.237+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:36.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.237+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:36.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.237+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b80163d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:36.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.237+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:36.231 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.237+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:36.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.641+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:36.636 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.641+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b80163d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.641+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.641+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:36.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:36.641+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:37.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:37.445+0000 7ff3db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:37.440 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:37.445+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:37.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:37.445+0000 7ff3d2000640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b80163d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:37.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:37.445+0000 7ff3d2000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:37.441 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:37.445+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:39.030 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b80163d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:39.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:39.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3d8c00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:39.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:39.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3d8c00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:39.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b800f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:39.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b800a990 2024-03-04T08:40:39.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8015df0 2024-03-04T08:40:39.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 msgr2=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:39.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b800f1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:39.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.037+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:39.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.241+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 msgr2=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:39.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.241+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b800f1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:39.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.241+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:39.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.645+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 msgr2=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:39.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.645+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b800f1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:39.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:39.645+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:40.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:40.449+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 msgr2=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:40.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:40.449+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b800f1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:40.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:40.449+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:42.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 msgr2=0x7ff3b800f1d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:42.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b800f1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:42.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800a990 legacy=0x7ff3b8007f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:42.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3db400640 1 -- reap_dead start 2024-03-04T08:40:42.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b8015400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:42.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:42.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3d8c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8006f20 con 0x7ff3b800f710 2024-03-04T08:40:42.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3d8c00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b8015df0 2024-03-04T08:40:42.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3d8c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3dc0c88c0 con 0x7ff3b80159e0 2024-03-04T08:40:42.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 msgr2=0x7ff3b8015400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:42.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b8015400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:42.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b80163d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:42.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800f710 legacy=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:42.033 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.037+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.241+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 msgr2=0x7ff3b8015400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:42.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.241+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b8015400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.241+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b80163d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.241+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800f710 legacy=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:42.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.241+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:42.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.645+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 msgr2=0x7ff3b8015400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:42.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.645+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b8015400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:42.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.645+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800f710 legacy=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:42.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.645+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b80163d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:42.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:42.645+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:43.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:43.449+0000 7ff3d2000640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 msgr2=0x7ff3b8015400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:43.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:43.449+0000 7ff3d2000640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b8015400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:43.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:43.449+0000 7ff3db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b80163d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:43.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:43.449+0000 7ff3daa00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800f710 legacy=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:43.445 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:43.449+0000 7ff3db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:45.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3d8c00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 msgr2=0x7ff3b80163d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:45.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3d8c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7ff3b80159e0 0x7ff3b80163d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:45.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3d8c00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 msgr2=0x7ff3b8015400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:45.031 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3b8015df0 0x7ff3b8015400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:45.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3d8c00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7ff3b800f710 legacy=0x7ff3b8014cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:45.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3d8c00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:45.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3d8c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3dc158910 con 0x7ff3b80086e0 2024-03-04T08:40:45.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3d8c00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3dc1530d0 con 0x7ff3b800a420 2024-03-04T08:40:45.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:45.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:45.032 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.036+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:45.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.240+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:45.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.240+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:45.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.240+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:45.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.644+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:45.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.644+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:45.637 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:45.644+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:46.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:46.448+0000 7ff3d2000640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7ff3b80086e0 legacy=0x7ff3b80077f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:46.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:46.448+0000 7ff3daa00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 msgr2=0x7ff3b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:46.444 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:46.448+0000 7ff3daa00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7ff3b800a420 0x7ff3b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:47.895 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-03-04T08:40:47.896 INFO:tasks.ceph:Scrubbing osd.5 2024-03-04T08:40:47.897 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-04T08:40:48.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.048+0000 7feae0c00640 1 Processor -- start 2024-03-04T08:40:48.046 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.048+0000 7feae0c00640 1 -- start start 2024-03-04T08:40:48.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.048+0000 7feae0c00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feadc072b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:48.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.048+0000 7feae0c00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feadc0c18e0 2024-03-04T08:40:48.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.048+0000 7feae0c00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feadc15b9a0 2024-03-04T08:40:48.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.048+0000 7feae0c00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feadc0c1500 2024-03-04T08:40:48.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.048+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feadc0c18e0 legacy=0x7feadc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:48.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.048+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 msgr2=0x7feadc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:48.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.048+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feadc15b9a0 legacy=0x7feadc151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:48.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.048+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feadc072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:48.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.252+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feadc0c18e0 legacy=0x7feadc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:48.248 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.252+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 msgr2=0x7feadc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:48.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.252+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feadc15b9a0 legacy=0x7feadc151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:48.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.252+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feadc072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:48.649 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.656+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feadc0c18e0 legacy=0x7feadc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:48.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.656+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 msgr2=0x7feadc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:48.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.656+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feadc15b9a0 legacy=0x7feadc151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:48.650 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:48.656+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feadc072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:48.856 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:40:48.860 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:40:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:49.460+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feadc0c18e0 legacy=0x7feadc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:49.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:49.460+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 msgr2=0x7feadc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:49.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:49.460+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feadc15b9a0 legacy=0x7feadc151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:49.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:49.460+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feadc072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:51.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 msgr2=0x7feadc072b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:51.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feadc072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:51.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feadc15b9a0 legacy=0x7feadc151630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:51.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feadc0c18e0 legacy=0x7feadc15b290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:51.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 0x7feab8007d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:51.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 0x7feab800a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:51.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8008450 2024-03-04T08:40:51.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab80058f0 2024-03-04T08:40:51.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead1400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 msgr2=0x7feab800a910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:51.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 0x7feab800a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:51.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 msgr2=0x7feab8007d70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:51.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.052+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 0x7feab8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:51.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.256+0000 7fead1400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 msgr2=0x7feab800a910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:51.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.256+0000 7fead1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 0x7feab800a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:51.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.256+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 msgr2=0x7feab8007d70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:51.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.256+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 0x7feab8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:51.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.660+0000 7fead1400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 msgr2=0x7feab800a910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:51.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.660+0000 7fead1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 0x7feab800a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:51.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.660+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 msgr2=0x7feab8007d70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:51.653 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:51.660+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 0x7feab8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:52.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:52.464+0000 7fead1400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 msgr2=0x7feab800a910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:52.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:52.464+0000 7fead1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 0x7feab800a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:52.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:52.464+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 msgr2=0x7feab8007d70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:52.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:52.464+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 0x7feab8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:54.048 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 msgr2=0x7feab8007d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:54.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab80058f0 0x7feab8007d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:54.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 msgr2=0x7feab800a910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:54.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8008450 0x7feab800a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:54.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:40:54.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feab800d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:54.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:54.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab8008dc0 2024-03-04T08:40:54.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8005d60 2024-03-04T08:40:54.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feadc0c1500 2024-03-04T08:40:54.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 msgr2=0x7feab800d2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:54.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feab800d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:54.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:54.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:54.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.052+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.256+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 msgr2=0x7feab800d2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.256+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feab800d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.256+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.256+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:54.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.256+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:54.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.660+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 msgr2=0x7feab800d2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:54.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.660+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:54.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.660+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:54.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.660+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feab800d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:54.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:54.660+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:55.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:55.464+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 msgr2=0x7feab800d2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:40:55.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:55.464+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:40:55.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:55.464+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:40:55.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:55.464+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feab800d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:55.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:55.464+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:40:57.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 msgr2=0x7feab800d2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:57.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feadc0c1500 0x7feab800d2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:57.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:57.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:40:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8009270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:40:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 0x7feab8011c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:40:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feab8012160 2024-03-04T08:40:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800f830 2024-03-04T08:40:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8012160 legacy=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 msgr2=0x7feab8011c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:57.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.056+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 0x7feab8011c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:40:57.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.260+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8012160 legacy=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:57.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.260+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 msgr2=0x7feab8011c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:57.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.260+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 0x7feab8011c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:40:57.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.664+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 msgr2=0x7feab8011c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:57.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.664+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8012160 legacy=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:57.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:57.664+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 0x7feab8011c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:40:58.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:58.468+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8012160 legacy=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:40:58.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:58.468+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 msgr2=0x7feab8011c20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:40:58.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:40:58.468+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 0x7feab8011c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:00.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 msgr2=0x7feab8011c20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:00.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800f830 0x7feab8011c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:00.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8012160 legacy=0x7feab8014550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:00.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:41:00.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:00.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab8015cc0 2024-03-04T08:41:00.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feab8008dc0 2024-03-04T08:41:00.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8005d60 2024-03-04T08:41:00.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015cc0 legacy=0x7feab8016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:00.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:00.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:00.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.052+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:00.249 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.256+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015cc0 legacy=0x7feab8016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:00.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.256+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:00.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.256+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:00.250 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.256+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:00.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.660+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015cc0 legacy=0x7feab8016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:00.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.660+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:00.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.660+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:00.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:00.660+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:01.457 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:01.464+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8009270 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:01.464+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015cc0 legacy=0x7feab8016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:01.464+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:01.458 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:01.464+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:03.049 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8009270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015cc0 legacy=0x7feab8016100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 0x7feab8014550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800b2d0 2024-03-04T08:41:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8010150 2024-03-04T08:41:03.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 msgr2=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:03.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.056+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 0x7feab8014550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:03.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.260+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:03.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.260+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:03.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.260+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 msgr2=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:03.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.260+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 0x7feab8014550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:03.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.664+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:03.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.664+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:03.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.664+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 msgr2=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:03.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:03.664+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 0x7feab8014550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:04.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:04.468+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:04.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:04.468+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:04.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:04.468+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 msgr2=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:04.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:04.468+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 0x7feab8014550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:06.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 msgr2=0x7feab8014550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:06.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010150 0x7feab8014550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:06.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:41:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 0x7feab8014e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800b2d0 2024-03-04T08:41:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8005d60 2024-03-04T08:41:06.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab80153b0 con 0x7feab8014a90 2024-03-04T08:41:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 msgr2=0x7feab8014e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:06.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.056+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 0x7feab8014e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.260+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.260+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:06.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.260+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 msgr2=0x7feab8014e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:06.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.260+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:06.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.260+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:06.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.260+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 0x7feab8014e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:06.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.664+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:06.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.664+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:06.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.664+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:06.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.664+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 msgr2=0x7feab8014e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:06.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.664+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:06.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:06.664+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 0x7feab8014e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:07.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:07.468+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:07.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:07.468+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:07.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:07.468+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:07.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:07.468+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 msgr2=0x7feab8014e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:07.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:07.468+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 0x7feab8014e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:07.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:07.468+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:09.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.056+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 msgr2=0x7feab8014e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.056+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014a90 0x7feab8014e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.056+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:09.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.056+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.056+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8015580 2024-03-04T08:41:09.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.056+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab8008dc0 2024-03-04T08:41:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.056+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015580 legacy=0x7feab800a260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:09.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.056+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:09.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.260+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015580 legacy=0x7feab800a260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:09.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.260+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:09.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.664+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015580 legacy=0x7feab800a260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:09.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:09.664+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:10.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:10.468+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015580 legacy=0x7feab800a260 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:10.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:10.468+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8016100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:12.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab8016100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:12.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015580 legacy=0x7feab800a260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:12.050 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:41:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feab8008dc0 2024-03-04T08:41:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800b2d0 2024-03-04T08:41:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8005d60 2024-03-04T08:41:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:12.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:12.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.056+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:12.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.260+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:12.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.260+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:12.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.260+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:12.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.260+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:12.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.260+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:12.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.664+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:12.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.664+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:12.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.664+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:12.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.664+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:12.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:12.664+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:13.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:13.468+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:13.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:13.468+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:13.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:13.468+0000 7feada800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:13.468+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:13.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:13.468+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:15.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab8008880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab8008880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 0x7feab8015580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8010530 2024-03-04T08:41:15.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800a260 2024-03-04T08:41:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010530 legacy=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:15.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.056+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:15.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.260+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010530 legacy=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:15.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.260+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:15.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.260+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:15.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.664+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010530 legacy=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:15.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.664+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:15.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:15.664+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:16.468+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010530 legacy=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:16.468+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:16.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:16.468+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:18.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:18.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800a260 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:18.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010530 legacy=0x7feab8014550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:18.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8011920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:18.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:41:18.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:18.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feab8008dc0 2024-03-04T08:41:18.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800b2d0 2024-03-04T08:41:18.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8014c60 2024-03-04T08:41:18.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:18.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:18.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 msgr2=0x7feab8011920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:18.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:18.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.056+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8011920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:18.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.260+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:18.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.260+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:18.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.260+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:18.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.260+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 msgr2=0x7feab8011920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:18.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.260+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8011920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:18.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.664+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:18.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.664+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:18.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.664+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 msgr2=0x7feab8011920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:18.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.664+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:18.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:18.664+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8011920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:18.990 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:41:18.995 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:41:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:19.468+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:19.468+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:19.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:19.468+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 msgr2=0x7feab8011920 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:19.468+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:19.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:19.468+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8011920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:21.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 msgr2=0x7feab8011920 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:21.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8011920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:21.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 0x7feab800fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8005d60 2024-03-04T08:41:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8008880 2024-03-04T08:41:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8005d60 legacy=0x7feab8010530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:21.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.056+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:21.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.260+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8005d60 legacy=0x7feab8010530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:21.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.260+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:21.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.260+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:21.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.664+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8005d60 legacy=0x7feab8010530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:21.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.664+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:21.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:21.664+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:22.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:22.468+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8005d60 legacy=0x7feab8010530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:22.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:22.468+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:22.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:22.468+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:24.051 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8008880 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8005d60 legacy=0x7feab8010530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8014550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:41:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:24.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feab8008dc0 2024-03-04T08:41:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800b2d0 2024-03-04T08:41:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8014c60 2024-03-04T08:41:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 msgr2=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:24.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.056+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8014550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.260+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:24.252 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.260+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.260+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 msgr2=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.260+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:24.253 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.260+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8014550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:24.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.664+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:24.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.664+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:24.656 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.664+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:24.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.664+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 msgr2=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:24.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:24.664+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8014550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:25.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:25.468+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:25.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:25.468+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:25.460 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:25.468+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 msgr2=0x7feab8014550 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:25.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:25.468+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:25.461 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:25.468+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8014550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:27.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 msgr2=0x7feab8014550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:27.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014c60 0x7feab8014550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:27.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:27.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:27.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab800fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8008880 2024-03-04T08:41:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8005d60 2024-03-04T08:41:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8008880 legacy=0x7feab8010530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:27.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.060+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:27.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.264+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8008880 legacy=0x7feab8010530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:27.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.264+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:27.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.264+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.668+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8008880 legacy=0x7feab8010530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.668+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:27.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:27.668+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:28.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:28.472+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8008880 legacy=0x7feab8010530 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:28.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:28.472+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:28.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:28.472+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:30.052 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 msgr2=0x7feab800fc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:30.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8005d60 0x7feab800fc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:30.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8008880 legacy=0x7feab8010530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:30.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:30.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:41:30.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8018d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:30.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feab8008dc0 2024-03-04T08:41:30.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800b2d0 2024-03-04T08:41:30.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8011c50 2024-03-04T08:41:30.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:30.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:30.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:30.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 msgr2=0x7feab8018d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:30.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.060+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8018d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:30.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.264+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:30.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.264+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:30.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.264+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:30.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.264+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 msgr2=0x7feab8018d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:30.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.264+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8018d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:30.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.668+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:30.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.668+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:30.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.668+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:30.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.668+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 msgr2=0x7feab8018d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:30.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:30.668+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8018d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:31.472+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:31.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:31.472+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:31.472+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:31.472+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 msgr2=0x7feab8018d30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:31.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:31.472+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8018d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 msgr2=0x7feab8018d30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8018d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:33.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab800fc10 2024-03-04T08:41:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8014550 2024-03-04T08:41:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab800fc10 legacy=0x7feab8014ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:33.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.060+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:33.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.264+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab800fc10 legacy=0x7feab8014ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:33.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.264+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:33.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.264+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:33.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.668+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab800fc10 legacy=0x7feab8014ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:33.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.668+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:33.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:33.668+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:34.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:34.472+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab800fc10 legacy=0x7feab8014ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:34.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:34.472+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:34.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:34.472+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:36.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:36.053 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:36.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab800fc10 legacy=0x7feab8014ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:36.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:36.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:41:36.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8016ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:36.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:36.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800b2d0 2024-03-04T08:41:36.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8011c50 2024-03-04T08:41:36.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab80153b0 con 0x7feab8014550 2024-03-04T08:41:36.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:36.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:36.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 msgr2=0x7feab8016ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:36.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8016ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:36.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7feada800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:36.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.060+0000 7feada800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.264+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.264+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:36.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.264+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 msgr2=0x7feab8016ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:36.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.264+0000 7feada800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:36.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.264+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8016ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:36.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.264+0000 7feada800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:36.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.668+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:36.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.668+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:36.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.668+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 msgr2=0x7feab8016ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:36.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.668+0000 7feada800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:36.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.668+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8016ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:36.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:36.668+0000 7feada800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:37.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:37.472+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:37.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:37.472+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:37.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:37.472+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 msgr2=0x7feab8016ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:37.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:37.472+0000 7feada800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:37.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:37.472+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8016ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:37.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:37.472+0000 7feada800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:39.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.060+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:39.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.060+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014550 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:39.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.060+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 msgr2=0x7feab8016ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:39.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011c50 0x7feab8016ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:39.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.060+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:39.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.060+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:39.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.060+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feab800fc10 2024-03-04T08:41:39.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.060+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8008dc0 2024-03-04T08:41:39.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.060+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab8014ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:39.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.060+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:39.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.264+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab8014ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:39.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.264+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:39.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.668+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab8014ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:39.712 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:39.668+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:40.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:40.472+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab8014ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:40.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:40.472+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8008dc0 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:42.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab8014ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:41:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab800fc10 2024-03-04T08:41:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feab8017220 2024-03-04T08:41:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800b2d0 2024-03-04T08:41:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:42.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.059+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8017220 legacy=0x7feab8019610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:42.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.263+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:42.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.263+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:42.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.263+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:42.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.263+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8017220 legacy=0x7feab8019610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:42.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.667+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:42.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.667+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:42.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.667+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:42.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:42.667+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8017220 legacy=0x7feab8019610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:43.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:43.471+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:43.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:43.471+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:43.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:43.471+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:43.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:43.471+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8017220 legacy=0x7feab8019610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 msgr2=0x7feab8015580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800b2d0 0x7feab8015580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8017220 legacy=0x7feab8019610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:45.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 0x7feab8016ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 0x7feab8011540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8014550 2024-03-04T08:41:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8014ac0 2024-03-04T08:41:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 msgr2=0x7feab8016ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 0x7feab8016ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:45.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.059+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.263+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 msgr2=0x7feab8016ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.263+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.263+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:45.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.263+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 0x7feab8016ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:45.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.667+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:45.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.667+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 msgr2=0x7feab8016ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:45.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.667+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:45.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:45.667+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 0x7feab8016ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:46.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:46.471+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:46.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:46.471+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 msgr2=0x7feab8016ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:46.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:46.471+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:46.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:46.471+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 0x7feab8016ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 msgr2=0x7feab8011540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:48.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ac0 0x7feab8011540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:48.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 msgr2=0x7feab8016ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:48.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8014550 0x7feab8016ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:48.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8011a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:48.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:41:48.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab8009650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:48.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab800fc10 2024-03-04T08:41:48.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8009250 2024-03-04T08:41:48.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8014ef0 2024-03-04T08:41:48.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:48.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 msgr2=0x7feab8011a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:48.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:48.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8011a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:48.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.059+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:48.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.263+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:48.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.263+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:48.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.263+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 msgr2=0x7feab8011a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:48.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.263+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8011a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:48.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.263+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:48.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.667+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:48.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.667+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:48.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.667+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 msgr2=0x7feab8011a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:48.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.667+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:48.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:48.667+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8011a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:49.270 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:41:49.274 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:41:49.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:49.471+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:49.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:49.471+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:49.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:49.471+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 msgr2=0x7feab8011a80 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:49.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:49.471+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:49.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:49.471+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8011a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:51.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 msgr2=0x7feab8011a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:51.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8011a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:51.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:51.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:51.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:51.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:51.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab8010530 2024-03-04T08:41:51.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8015580 2024-03-04T08:41:51.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:51.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:51.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.059+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8010530 legacy=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:51.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.263+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:51.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.263+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:51.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.263+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8010530 legacy=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.667+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.667+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:51.667+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8010530 legacy=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:52.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:52.471+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:52.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:52.471+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:52.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:52.471+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8010530 legacy=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:54.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:54.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8010530 legacy=0x7feab8010910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8010910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:41:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8009250 2024-03-04T08:41:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab8011020 2024-03-04T08:41:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8014ef0 2024-03-04T08:41:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8009250 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:54.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011020 legacy=0x7feab800c2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:54.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:54.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.059+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:54.255 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.263+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8009250 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:54.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.263+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:54.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.263+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011020 legacy=0x7feab800c2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:54.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.263+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:54.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.667+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8009250 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:54.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.667+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:54.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.667+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011020 legacy=0x7feab800c2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:54.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:54.667+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:55.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:55.471+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8009250 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:41:55.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:55.471+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011020 legacy=0x7feab800c2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:41:55.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:55.471+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:41:55.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:55.471+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:57.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:57.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8014ef0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:41:57.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011020 legacy=0x7feab800c2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:57.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8009250 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:41:57.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:57.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:41:57.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800fc10 2024-03-04T08:41:57.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8015580 2024-03-04T08:41:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:57.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.059+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:41:57.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.263+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:57.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.263+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:57.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.263+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:57.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.263+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:41:57.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.667+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:57.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.667+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:57.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.667+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:57.661 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:57.667+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:41:58.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:58.471+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:41:58.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:58.471+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:41:58.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:58.471+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:41:58.465 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:41:58.471+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:00.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:00.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:00.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:00.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:00.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:42:00.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:00.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feab8009250 2024-03-04T08:42:00.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8011020 2024-03-04T08:42:00.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8015580 2024-03-04T08:42:00.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8009250 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:00.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011020 legacy=0x7feab8011400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:00.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:00.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.059+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:00.256 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.263+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8009250 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:00.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.263+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011020 legacy=0x7feab8011400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:00.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.263+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:00.257 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.263+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.667+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8009250 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.667+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011020 legacy=0x7feab8011400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.667+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:00.660 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:00.667+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:01.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:01.471+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8009250 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:01.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:01.471+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011020 legacy=0x7feab8011400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:01.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:01.471+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:01.464 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:01.471+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:03.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 msgr2=0x7feab8009b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:03.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8015580 0x7feab8009b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:03.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011020 legacy=0x7feab8011400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:03.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8009250 legacy=0x7feab80160e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:03.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 0x7feab8009650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:03.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 0x7feab8011b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:03.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8014ef0 2024-03-04T08:42:03.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab800fc10 2024-03-04T08:42:03.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:03.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:03.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 msgr2=0x7feab8011b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:03.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.063+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 0x7feab8011b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:03.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.267+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:03.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.267+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:03.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.267+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 msgr2=0x7feab8011b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:03.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.267+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 0x7feab8011b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.671+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.671+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.671+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 msgr2=0x7feab8011b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:03.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:03.671+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 0x7feab8011b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:04.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:04.475+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:04.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:04.475+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:04.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:04.475+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 msgr2=0x7feab8011b10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:04.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:04.475+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 0x7feab8011b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:06.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 msgr2=0x7feab8011b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800fc10 0x7feab8011b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8014ef0 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:42:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:06.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800d2d0 2024-03-04T08:42:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab800fc10 2024-03-04T08:42:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab80153b0 con 0x7feab8009250 2024-03-04T08:42:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:06.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:06.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.063+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.267+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.267+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.267+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:06.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.267+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.267+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:06.261 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.267+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:06.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.671+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:06.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.671+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:06.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.671+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:06.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.671+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:06.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.671+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:06.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:06.671+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:07.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:07.475+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:07.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:07.475+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:07.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:07.475+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:07.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:07.475+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:07.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:07.475+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:07.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:07.475+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:09.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.063+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.063+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.063+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab800fc10 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.063+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:09.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:09.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.063+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab8015580 2024-03-04T08:42:09.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.063+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2080 con 0x7feab8009b90 2024-03-04T08:42:09.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.063+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015580 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:09.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.063+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:09.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.267+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015580 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:09.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.267+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:09.663 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.671+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015580 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:09.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:09.671+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:10.475+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015580 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:10.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:10.475+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:12.057 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8015580 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:42:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8009b90 2024-03-04T08:42:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800d2d0 2024-03-04T08:42:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8009250 2024-03-04T08:42:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:12.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:12.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:12.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.063+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:12.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.267+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:12.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.267+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:12.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.267+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:12.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.267+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:12.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.267+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:12.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.671+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:12.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.671+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:12.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.671+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:12.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.671+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:12.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:12.671+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:13.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:13.475+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:13.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:13.475+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:13.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:13.475+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:13.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:13.475+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:13.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:13.475+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:15.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 0x7feab8009650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab800fc10 2024-03-04T08:42:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8015580 2024-03-04T08:42:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:15.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.063+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:15.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.267+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:15.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.267+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:15.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.267+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:15.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.671+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:15.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.671+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:15.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:15.671+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:16.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:16.475+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:16.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:16.475+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:16.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:16.475+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:18.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:18.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8015580 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:18.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:18.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:42:18.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:18.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:18.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8009b90 2024-03-04T08:42:18.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800d2d0 2024-03-04T08:42:18.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8010e50 2024-03-04T08:42:18.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:18.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:18.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:18.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:18.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.063+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.267+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.267+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.267+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.267+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:18.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.267+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:18.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.671+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:18.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.671+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:18.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.671+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:18.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.671+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:18.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:18.671+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:19.380 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:42:19.384 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:42:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:19.475+0000 7feada800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:19.475+0000 7feada800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:19.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:19.475+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:19.475+0000 7fead9e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:19.469 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:19.475+0000 7fead9e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:21.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab801b410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab800fc10 2024-03-04T08:42:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8009250 2024-03-04T08:42:21.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:21.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:21.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.063+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:21.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.267+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:21.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.267+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:21.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.267+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:21.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.671+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:21.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.671+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:21.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:21.671+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:22.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:22.475+0000 7fead9e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:22.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:22.475+0000 7fead1400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:22.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:22.475+0000 7fead1400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:24.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:24.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8009250 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:24.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab800fc10 legacy=0x7feab801ad00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:24.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:42:24.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:24.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:24.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8011610 2024-03-04T08:42:24.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8011230 2024-03-04T08:42:24.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab800d2d0 2024-03-04T08:42:24.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8011610 legacy=0x7feab80119f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:24.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:24.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7feada800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:24.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7feada800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:24.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.063+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:24.259 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.267+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8011610 legacy=0x7feab80119f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:24.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.267+0000 7feada800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:24.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.267+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:24.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.267+0000 7feada800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:24.260 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.267+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:24.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.671+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8011610 legacy=0x7feab80119f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:24.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.671+0000 7feada800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:24.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.671+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:24.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.671+0000 7feada800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:24.664 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:24.671+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:25.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:25.475+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8011610 legacy=0x7feab80119f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:25.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:25.475+0000 7feada800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:25.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:25.475+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:25.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:25.475+0000 7feada800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:25.468 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:25.475+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:27.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 msgr2=0x7feab8010910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:27.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab800d2d0 0x7feab8010910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:27.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:27.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:27.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8011610 legacy=0x7feab80119f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:27.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:27.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab8009b90 2024-03-04T08:42:27.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8010e50 2024-03-04T08:42:27.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:27.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:27.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.067+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab800b2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:27.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.271+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:27.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.271+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:27.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.271+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab800b2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:27.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.675+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:27.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.675+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:27.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:27.675+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab800b2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:28.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:28.479+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:28.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:28.479+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:28.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:28.479+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab800b2b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:30.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 msgr2=0x7feab8009650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:30.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010e50 0x7feab8009650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:30.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8009b90 legacy=0x7feab800b2b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:30.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:42:30.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:30.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8009a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:30.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8010910 2024-03-04T08:42:30.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8011230 2024-03-04T08:42:30.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8009650 2024-03-04T08:42:30.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:30.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:30.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:30.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8009a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:30.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.067+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:30.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.271+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:30.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.271+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:30.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.271+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:30.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.271+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8009a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:30.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.271+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:30.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.675+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:30.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.675+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:30.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.675+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:30.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.675+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8009a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:30.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:30.675+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:31.479+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:31.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:31.479+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:31.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:31.479+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:31.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:31.479+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8009a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:31.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:31.479+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:33.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8009a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:33.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8009a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:33.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:33.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:33.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8011610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:33.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 0x7feab8010cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:33.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab8011d20 2024-03-04T08:42:33.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab800d2c0 2024-03-04T08:42:33.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead1400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 msgr2=0x7feab8010cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:33.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 0x7feab8010cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:33.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.067+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011d20 legacy=0x7feab8007150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:33.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.271+0000 7fead1400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 msgr2=0x7feab8010cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:33.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.271+0000 7fead1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 0x7feab8010cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:33.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.271+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011d20 legacy=0x7feab8007150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:33.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.675+0000 7fead1400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 msgr2=0x7feab8010cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:33.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.675+0000 7fead1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 0x7feab8010cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:33.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:33.675+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011d20 legacy=0x7feab8007150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:34.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:34.479+0000 7fead1400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 msgr2=0x7feab8010cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:34.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:34.479+0000 7fead1400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 0x7feab8010cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:34.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:34.479+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011d20 legacy=0x7feab8007150 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:36.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 msgr2=0x7feab8010cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab800d2c0 0x7feab8010cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8011d20 legacy=0x7feab8007150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:42:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8007860 2024-03-04T08:42:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8011230 2024-03-04T08:42:36.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8009650 2024-03-04T08:42:36.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8007860 legacy=0x7feab8020160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:36.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:36.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8007c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:36.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:36.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.067+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:36.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.271+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8007860 legacy=0x7feab8020160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:36.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.271+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:36.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.271+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8007c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:36.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.271+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:36.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.271+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:36.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.675+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8007860 legacy=0x7feab8020160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:36.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.675+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8007c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:36.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.675+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:36.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.675+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:36.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:36.675+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:37.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:37.479+0000 7fead9e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8007860 legacy=0x7feab8020160 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:37.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:37.479+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8007c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:37.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:37.479+0000 7feada800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:37.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:37.479+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:37.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:37.479+0000 7feada800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:39.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8007c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:39.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:39.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:39.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead3e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8007860 legacy=0x7feab8020160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 0x7feab8011610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab8010910 2024-03-04T08:42:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8009a30 2024-03-04T08:42:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8006140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:39.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 msgr2=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:39.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.066+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 0x7feab8011610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:39.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.270+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 msgr2=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:39.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.270+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8006140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:39.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.270+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 0x7feab8011610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:39.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.674+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8006140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:39.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.674+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 msgr2=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:39.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:39.674+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 0x7feab8011610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:40.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:40.478+0000 7fead1400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8006140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:40.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:40.478+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 msgr2=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:40.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:40.478+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 0x7feab8011610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:42.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead3e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 msgr2=0x7feab8011610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:42.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8009a30 0x7feab8011610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:42.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8006140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:42.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7feada800640 1 -- reap_dead start 2024-03-04T08:42:42.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:42.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:42.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead3e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2360 con 0x7feab8010910 2024-03-04T08:42:42.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead3e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8011230 2024-03-04T08:42:42.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead3e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab800afc0 con 0x7feab8009650 2024-03-04T08:42:42.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:42.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8007c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:42.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:42.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:42.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.066+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:42.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.270+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:42.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.270+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:42.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.270+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8007c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:42.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.270+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:42.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.270+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:42.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.674+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.674+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8007c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.674+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.674+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:42.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:42.674+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:43.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:43.478+0000 7feada800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8011610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:43.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:43.478+0000 7fead9e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:43.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:43.478+0000 7fead1400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8007c40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:43.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:43.478+0000 7fead1400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:43.473 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:43.478+0000 7fead9e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:45.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead3e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 msgr2=0x7feab8007c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:45.061 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead3e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7feab8009650 0x7feab8007c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:45.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead3e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 msgr2=0x7feab80160a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:45.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead3e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7feab8011230 0x7feab80160a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:45.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead3e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7feab8010910 legacy=0x7feab8011610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:45.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead3e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010cf0 0x7feab8009a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:45.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead3e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0c2640 con 0x7feab8007150 2024-03-04T08:42:45.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead3e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feadc15ae00 con 0x7feab8010cf0 2024-03-04T08:42:45.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8010cf0 msgr2=0x7feab8009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:45.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010cf0 0x7feab8009a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:45.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.066+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8007150 legacy=0x7feab8018cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:45.263 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.270+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8010cf0 msgr2=0x7feab8009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:45.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.270+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010cf0 0x7feab8009a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:45.264 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.270+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8007150 legacy=0x7feab8018cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:45.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.674+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8010cf0 msgr2=0x7feab8009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.674+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010cf0 0x7feab8009a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:45.674+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8007150 legacy=0x7feab8018cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:46.471 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:46.478+0000 7fead9e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7feab8010cf0 msgr2=0x7feab8009a30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:46.478+0000 7fead9e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feab8010cf0 0x7feab8009a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:46.472 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:46.478+0000 7fead1400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7feab8007150 legacy=0x7feab8018cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:47.926 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-03-04T08:42:47.927 INFO:tasks.ceph:Scrubbing osd.6 2024-03-04T08:42:47.927 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-04T08:42:48.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.126+0000 7f54e2400640 1 Processor -- start 2024-03-04T08:42:48.120 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.126+0000 7f54e2400640 1 -- start start 2024-03-04T08:42:48.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.126+0000 7f54e2400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 0x7f54dc151630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:48.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.126+0000 7f54e2400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54dc0732a0 2024-03-04T08:42:48.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.126+0000 7f54e2400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2360 con 0x7f54dc0c1500 2024-03-04T08:42:48.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.126+0000 7f54e2400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54dc0c18e0 2024-03-04T08:42:48.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.126+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54dc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:48.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.126+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54dc0c1500 legacy=0x7f54dc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:48.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.126+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 msgr2=0x7f54dc151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:48.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.126+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 0x7f54dc151630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.330+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54dc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.330+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 msgr2=0x7f54dc151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.330+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54dc0c1500 legacy=0x7f54dc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:48.322 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.330+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 0x7f54dc151630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:48.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.734+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54dc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:48.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.734+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 msgr2=0x7f54dc151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:48.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.734+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54dc0c1500 legacy=0x7f54dc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:48.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:48.734+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 0x7f54dc151630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:49.438 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:42:49.442 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:42:49.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:49.538+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54dc15b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:49.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:49.538+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 msgr2=0x7f54dc151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:49.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:49.538+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54dc0c1500 legacy=0x7f54dc072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:49.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:49.538+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 0x7f54dc151630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:51.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 msgr2=0x7f54dc151630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:51.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54dc0c18e0 0x7f54dc151630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:51.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54dc0c1500 legacy=0x7f54dc072b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54dc15b290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 0x7f54b8007d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 0x7f54b800a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80058c0 2024-03-04T08:42:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b80083c0 2024-03-04T08:42:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 msgr2=0x7f54b8007d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:51.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 0x7f54b8007d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 msgr2=0x7f54b800a880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:51.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.126+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 0x7f54b800a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:51.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.330+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 msgr2=0x7f54b8007d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:51.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.330+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 0x7f54b8007d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:51.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.330+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 msgr2=0x7f54b800a880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:51.323 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.330+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 0x7f54b800a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.734+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 msgr2=0x7f54b8007d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:51.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.734+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 0x7f54b8007d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:51.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.734+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 msgr2=0x7f54b800a880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:51.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:51.734+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 0x7f54b800a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:52.538+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 msgr2=0x7f54b8007d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:52.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:52.538+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 0x7f54b8007d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:52.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:52.538+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 msgr2=0x7f54b800a880 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:52.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:52.538+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 0x7f54b800a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:54.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 msgr2=0x7f54b800a880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80083c0 0x7f54b800a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 msgr2=0x7f54b8007d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80058c0 0x7f54b8007d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:42:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 0x7f54b8006660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:54.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54dc0732a0 2024-03-04T08:42:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80061e0 2024-03-04T08:42:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8008870 2024-03-04T08:42:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54b800d300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 msgr2=0x7f54b8006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:54.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.130+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 0x7f54b8006660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:54.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.334+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54b800d300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:54.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.334+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:54.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.334+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 msgr2=0x7f54b8006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:54.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.334+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:54.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.334+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 0x7f54b8006660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.738+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54b800d300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.738+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.738+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 msgr2=0x7f54b8006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.738+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:54.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:54.738+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 0x7f54b8006660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:55.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:55.542+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54b800d300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:42:55.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:55.542+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:42:55.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:55.542+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 msgr2=0x7f54b8006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:42:55.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:55.542+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:55.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:55.542+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 0x7f54b8006660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008870 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 msgr2=0x7f54b8006660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:57.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80061e0 0x7f54b8006660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:42:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54b800d300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:42:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:42:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b8007330 2024-03-04T08:42:57.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8007e50 2024-03-04T08:42:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:42:57.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.130+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007330 legacy=0x7f54b8014a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:57.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.334+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:57.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.334+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:42:57.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.334+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007330 legacy=0x7f54b8014a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:57.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.738+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:57.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.738+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:42:57.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:57.738+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007330 legacy=0x7f54b8014a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:42:58.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:58.542+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:42:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:58.542+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:42:58.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:42:58.542+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007330 legacy=0x7f54b8014a50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007330 legacy=0x7f54b8014a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:43:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:00.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54dc0732a0 2024-03-04T08:43:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8007e50 2024-03-04T08:43:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8006660 2024-03-04T08:43:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54b800d300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54dbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:00.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.126+0000 7f54dbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:00.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.330+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54b800d300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:00.324 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.330+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:00.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.330+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:00.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.330+0000 7f54dbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:00.325 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.330+0000 7f54dbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.734+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:00.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.734+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54b800d300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.734+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.734+0000 7f54dbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:00.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:00.734+0000 7f54dbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:01.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:01.538+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:01.532 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:01.538+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54b800d300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:01.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:01.538+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:01.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:01.538+0000 7f54dbe00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:01.533 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:01.538+0000 7f54dbe00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:03.123 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:03.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006660 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:03.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:03.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:03.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54dc0732a0 legacy=0x7f54b800d300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:03.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8016a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:03.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b8008c90 2024-03-04T08:43:03.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b800a7f0 2024-03-04T08:43:03.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8016a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:03.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8008c90 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:03.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.130+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8016a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:03.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.334+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8016a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:03.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.334+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8008c90 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:03.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.334+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8016a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:03.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.738+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8008c90 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:03.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.738+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8016a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:03.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:03.738+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8016a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:04.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:04.542+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8016a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:04.542+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8008c90 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:04.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:04.542+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8016a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8016a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8016a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:06.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8008c90 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:43:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b8006660 2024-03-04T08:43:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b8008c90 2024-03-04T08:43:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8007e50 2024-03-04T08:43:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:06.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:06.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.130+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8008c90 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:06.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.334+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:06.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.334+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:06.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.334+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:06.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.334+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8008c90 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:06.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.738+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:06.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.738+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:06.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.738+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:06.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:06.738+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8008c90 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:07.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:07.542+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:07.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:07.542+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:07.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:07.542+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:07.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:07.542+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8008c90 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:09.124 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:09.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:09.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8008c90 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:09.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:09.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:09.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:09.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8014a50 2024-03-04T08:43:09.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b800a7f0 2024-03-04T08:43:09.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:09.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:09.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:09.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.130+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:09.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.334+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:09.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.334+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:09.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.334+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:09.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.334+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:09.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.738+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:09.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.738+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:09.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.738+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:09.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:09.738+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:10.542+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:10.542+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:10.542+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:10.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:10.542+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:12.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:12.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:12.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:12.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:12.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:43:12.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:12.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2360 con 0x7f54b8006660 2024-03-04T08:43:12.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b8007e50 2024-03-04T08:43:12.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b800a7f0 2024-03-04T08:43:12.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:12.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007e50 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:12.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:12.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.130+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:12.327 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.334+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:12.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.334+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007e50 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:12.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.334+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:12.328 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.334+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:12.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.738+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:12.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.738+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007e50 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:12.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.738+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:12.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:12.738+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:13.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:13.542+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:13.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:13.542+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007e50 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:13.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:13.542+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:13.537 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:13.542+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:15.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007e50 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:15.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8014a50 2024-03-04T08:43:15.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8008cb0 2024-03-04T08:43:15.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:15.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:15.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:15.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.130+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:15.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.334+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:15.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.334+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:15.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.334+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:15.326 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.334+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:15.731 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.738+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:15.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.738+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:15.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.738+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:15.732 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:15.738+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:16.535 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:16.542+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:16.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:16.542+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:16.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:16.542+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:16.536 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:16.542+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:18.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:18.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:18.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:43:18.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b8007e50 2024-03-04T08:43:18.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8014a50 2024-03-04T08:43:18.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b800a7f0 2024-03-04T08:43:18.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:18.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:18.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007e50 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:18.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:18.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.134+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:18.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.338+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:18.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.338+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:18.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.338+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007e50 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:18.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.338+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:18.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.338+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:18.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.742+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:18.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.742+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:18.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.742+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007e50 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:18.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.742+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:18.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:18.742+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:19.491 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:43:19.495 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:43:19.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:19.546+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:19.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:19.546+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:19.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:19.546+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007e50 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:19.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:19.546+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:19.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:19.546+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:21.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:21.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:21.126 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 msgr2=0x7f54b8016e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8014a50 0x7f54b8016e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8007e50 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2360 con 0x7f54b8006660 2024-03-04T08:43:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8008cb0 2024-03-04T08:43:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:21.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.134+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:21.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.338+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:21.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.338+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:21.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.338+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:21.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.742+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:21.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.742+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:21.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:21.742+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:22.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:22.546+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:22.546+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:22.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:22.546+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:24.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:24.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:24.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b80090b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:24.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:24.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:43:24.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:24.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b8006660 2024-03-04T08:43:24.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8007e50 2024-03-04T08:43:24.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b800a7f0 2024-03-04T08:43:24.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:24.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b8017690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:24.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:24.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:24.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.134+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:24.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.338+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:24.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.338+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b8017690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:24.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.338+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:24.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.338+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:24.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.338+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:24.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.742+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:24.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.742+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b8017690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:24.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.742+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:24.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.742+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:24.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:24.742+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:25.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:25.546+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:25.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:25.546+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b8017690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:25.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:25.546+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:25.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:25.546+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:25.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:25.546+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:27.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:27.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a7f0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:27.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:27.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:27.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b8017690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:27.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b800a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:27.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2360 con 0x7f54b8017da0 2024-03-04T08:43:27.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8008cb0 2024-03-04T08:43:27.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8017da0 legacy=0x7f54b8016a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:27.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:27.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.134+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:27.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.338+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8017da0 legacy=0x7f54b8016a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:27.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.338+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:27.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.338+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:27.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.742+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8017da0 legacy=0x7f54b8016a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:27.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.742+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:27.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:27.742+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:28.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:28.546+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8017da0 legacy=0x7f54b8016a60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:28.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:28.546+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:28.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:28.546+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:30.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:30.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8008cb0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:30.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8017da0 legacy=0x7f54b8016a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:30.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:43:30.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b8010300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:30.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:30.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b8006660 2024-03-04T08:43:30.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8007e50 2024-03-04T08:43:30.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b80181b0 2024-03-04T08:43:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b8017690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:30.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.134+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:30.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.338+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b8017690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:30.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.338+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:30.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.338+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:30.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.338+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:30.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.338+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:30.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.742+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b8017690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:30.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.742+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:30.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.742+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:30.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.742+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:30.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:30.742+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:31.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:31.546+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b8017690 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:31.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:31.546+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:31.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:31.546+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:31.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:31.546+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:31.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:31.546+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:33.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:33.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:33.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:33.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:33.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8006660 legacy=0x7f54b8017690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:33.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:33.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2360 con 0x7f54b800a7f0 2024-03-04T08:43:33.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8017da0 2024-03-04T08:43:33.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800a7f0 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:33.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:33.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.134+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:33.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.338+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:33.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.338+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800a7f0 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:33.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.338+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.742+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800a7f0 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:33.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.742+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:33.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:33.742+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:34.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:34.546+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800a7f0 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:34.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:34.546+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:34.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:34.546+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:36.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:36.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:36.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800a7f0 legacy=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:36.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:36.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:43:36.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:36.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:36.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8007e50 2024-03-04T08:43:36.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8017da0 2024-03-04T08:43:36.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800a7f0 con 0x7f54b80181b0 2024-03-04T08:43:36.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:36.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:36.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:36.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:36.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:36.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.133+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:36.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.337+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:36.331 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.337+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:36.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.337+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:36.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.337+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:36.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.337+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:36.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.337+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:36.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.741+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:36.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.741+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:36.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.741+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:36.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.741+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:36.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.741+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:36.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:36.741+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:37.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:37.545+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:37.545+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:37.545+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:37.545+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:37.545+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:37.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:37.545+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:39.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.133+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:39.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.133+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.133+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.133+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8017da0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.133+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.133+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.133+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b80065f0 2024-03-04T08:43:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.133+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2360 con 0x7f54b800f2f0 2024-03-04T08:43:39.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.133+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065f0 legacy=0x7f54b80175f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:39.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.133+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800f2f0 legacy=0x7f54b800f6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:39.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.337+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065f0 legacy=0x7f54b80175f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:39.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.337+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800f2f0 legacy=0x7f54b800f6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:39.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.741+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065f0 legacy=0x7f54b80175f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:39.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:39.741+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800f2f0 legacy=0x7f54b800f6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:40.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:40.545+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065f0 legacy=0x7f54b80175f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:40.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:40.545+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800f2f0 legacy=0x7f54b800f6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:42.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800f2f0 legacy=0x7f54b800f6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:42.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065f0 legacy=0x7f54b80175f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:42.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:42.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:43:42.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:42.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b800f2f0 2024-03-04T08:43:42.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8007e50 2024-03-04T08:43:42.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b80181b0 2024-03-04T08:43:42.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:42.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:42.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:42.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800f2f0 legacy=0x7f54b800f6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:42.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.133+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:42.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.337+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:42.329 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.337+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:42.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.337+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:42.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.337+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800f2f0 legacy=0x7f54b800f6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:42.330 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.337+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:42.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.741+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:42.735 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.741+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:42.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.741+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:42.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.741+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800f2f0 legacy=0x7f54b800f6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:42.736 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:42.741+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:43.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:43.545+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:43.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:43.545+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:43.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:43.545+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:43.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:43.545+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800f2f0 legacy=0x7f54b800f6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:43.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:43.545+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 msgr2=0x7f54b800a0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80181b0 0x7f54b800a0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:45.129 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8007e50 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800f2f0 legacy=0x7f54b800f6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b800fde0 2024-03-04T08:43:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80065c0 2024-03-04T08:43:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800fde0 legacy=0x7f54b80175f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:45.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.137+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:45.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.341+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800fde0 legacy=0x7f54b80175f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:45.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.341+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:45.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.341+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:45.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.745+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800fde0 legacy=0x7f54b80175f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:45.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.745+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:45.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:45.745+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:46.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:46.549+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800fde0 legacy=0x7f54b80175f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:46.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:46.549+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:46.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:46.549+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800fde0 legacy=0x7f54b80175f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:48.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:43:48.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:48.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b800a0c0 2024-03-04T08:43:48.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80065c0 2024-03-04T08:43:48.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8007e20 2024-03-04T08:43:48.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a0c0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:48.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:48.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:48.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:48.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.137+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:48.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.341+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a0c0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:48.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.341+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:48.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.341+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:48.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.341+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:48.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.341+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:48.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.745+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a0c0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:48.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.745+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:48.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.745+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:48.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.745+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:48.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:48.745+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:49.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:49.549+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a0c0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:49.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:49.549+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:49.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:49.549+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:49.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:49.549+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:49.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:49.549+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:49.546 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:43:49.549 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:43:51.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:51.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:51.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:51.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:51.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a0c0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:51.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:51.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b8009b80 2024-03-04T08:43:51.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80097a0 2024-03-04T08:43:51.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 msgr2=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:51.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:51.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.137+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8009b80 legacy=0x7f54b800f830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:51.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.341+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 msgr2=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:51.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.341+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:51.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.341+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8009b80 legacy=0x7f54b800f830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:51.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.745+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 msgr2=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:51.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.745+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:51.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:51.745+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8009b80 legacy=0x7f54b800f830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:52.549+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 msgr2=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:52.549+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:52.549+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8009b80 legacy=0x7f54b800f830 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:54.130 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 msgr2=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8009b80 legacy=0x7f54b800f830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:43:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:54.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b8009b80 2024-03-04T08:43:54.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80097a0 2024-03-04T08:43:54.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8007e20 2024-03-04T08:43:54.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 msgr2=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:54.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:54.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:54.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8009b80 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:54.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.137+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:54.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.341+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 msgr2=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:54.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.341+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:54.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.341+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8009b80 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:54.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.341+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:54.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.341+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:54.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.745+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 msgr2=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:54.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.745+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:54.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.745+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8009b80 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:54.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.745+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:54.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:54.745+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:55.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:55.549+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 msgr2=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:43:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:55.549+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:55.549+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:43:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:55.549+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8009b80 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:43:55.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:55.549+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:43:57.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:57.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:57.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 msgr2=0x7f54b800f2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:57.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80097a0 0x7f54b800f2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:43:57.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8009b80 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:43:57.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:43:57.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b80065c0 2024-03-04T08:43:57.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8006c60 2024-03-04T08:43:57.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065c0 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:57.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:57.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.137+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:43:57.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.341+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065c0 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:57.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.341+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:57.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.341+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:43:57.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.745+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065c0 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:57.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.745+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:57.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:57.745+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:43:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:58.549+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065c0 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:43:58.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:58.549+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:43:58.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:43:58.549+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:00.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:00.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:00.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065c0 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:00.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:00.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:44:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b80065c0 2024-03-04T08:44:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8007e20 2024-03-04T08:44:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8006c60 2024-03-04T08:44:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065c0 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:00.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:00.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.137+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:00.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.341+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065c0 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:00.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.341+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:00.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.341+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:00.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.341+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:00.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.341+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:00.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.745+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065c0 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:00.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.745+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:00.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.745+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:00.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.745+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:00.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:00.745+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:01.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:01.549+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065c0 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:01.549+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:01.549+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:01.549+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:01.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:01.549+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:03.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80065c0 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:03.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b8010ff0 2024-03-04T08:44:03.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80113d0 2024-03-04T08:44:03.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:03.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:03.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.137+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8010ff0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.341+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.341+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:03.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.341+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8010ff0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.745+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:03.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.745+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:03.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:03.745+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8010ff0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:04.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:04.549+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:04.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:04.549+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:04.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:04.549+0000 7f54db400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8010ff0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:06.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:06.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:06.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8010ff0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:06.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:44:06.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:06.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:06.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b8010ff0 2024-03-04T08:44:06.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80113d0 2024-03-04T08:44:06.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8006c60 2024-03-04T08:44:06.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:06.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:06.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:06.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8010ff0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:06.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.137+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:06.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.341+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.341+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.341+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.341+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8010ff0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:06.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.341+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:06.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.745+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:06.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.745+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:06.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.745+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:06.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.745+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8010ff0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:06.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:06.745+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:07.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:07.549+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:07.549+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:07.549+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:07.549+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8010ff0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:07.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:07.549+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:09.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 msgr2=0x7f54b800a0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:09.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8006c60 0x7f54b800a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:09.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:09.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80113d0 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:09.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b8010ff0 legacy=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:09.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:09.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b8007e20 2024-03-04T08:44:09.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80065c0 2024-03-04T08:44:09.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:09.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:09.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.137+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:09.332 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.341+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:09.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.341+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:09.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.341+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:09.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.745+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:09.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.745+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:09.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:09.745+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:10.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:10.549+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:10.549+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:10.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:10.549+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:12.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:12.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:12.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:12.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:44:12.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:12.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b8007e20 2024-03-04T08:44:12.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b800a070 2024-03-04T08:44:12.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80065c0 2024-03-04T08:44:12.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:12.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8010ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:12.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:12.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.137+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.341+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.341+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.341+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8010ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:12.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.341+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:12.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.745+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:12.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.745+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8010ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:12.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.745+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:12.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:12.745+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:13.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:13.549+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:13.540 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:13.549+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8010ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:13.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:13.549+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:13.541 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:13.549+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:15.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:15.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:15.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8010ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:15.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:15.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 0x7f54b8010300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:15.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:15.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8009090 2024-03-04T08:44:15.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8006c60 2024-03-04T08:44:15.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b80094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:15.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:15.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:15.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.137+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:15.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.341+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b80094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:15.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.341+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:15.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.341+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:15.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.341+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:15.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.745+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b80094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:15.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.745+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:15.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.745+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:15.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:15.745+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:16.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:16.549+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b80094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:16.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:16.549+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:16.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:16.549+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:16.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:16.549+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:18.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 msgr2=0x7f54b8010300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:18.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8006c60 0x7f54b8010300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:18.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b80094a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:18.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:18.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:18.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:44:18.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:18.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2360 con 0x7f54b8007e20 2024-03-04T08:44:18.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8009090 2024-03-04T08:44:18.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b80065c0 2024-03-04T08:44:18.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:18.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b80094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:18.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:18.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:18.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.141+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:18.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.345+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:18.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.345+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b80094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:18.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.345+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:18.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.345+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:18.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.345+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:18.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.749+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:18.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.749+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b80094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:18.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.749+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:18.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.749+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:18.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:18.749+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:19.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:19.553+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b80094a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:19.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:19.553+0000 7f54d2a00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:19.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:19.553+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:19.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:19.553+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:19.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:19.553+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:19.674 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:44:19.678 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:44:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b80094a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b80094a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b8007e20 legacy=0x7f54b80108e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:21.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b80099e0 2024-03-04T08:44:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b800a070 2024-03-04T08:44:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099e0 legacy=0x7f54b8010ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:21.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.141+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:21.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.345+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099e0 legacy=0x7f54b8010ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:21.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.345+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:21.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.345+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:21.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.749+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:21.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.749+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099e0 legacy=0x7f54b8010ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:21.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:21.749+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:22.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:22.553+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099e0 legacy=0x7f54b8010ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:22.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:22.553+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:22.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:22.553+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:24.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b800a070 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099e0 legacy=0x7f54b8010ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b8009470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:44:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:24.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8009090 2024-03-04T08:44:24.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8007e20 2024-03-04T08:44:24.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800a7f0 con 0x7f54b80065c0 2024-03-04T08:44:24.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b8009470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:24.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b8009470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:24.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:24.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:24.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:24.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.137+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:24.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.341+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b8009470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.341+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b8009470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.341+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.341+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.341+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:24.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.341+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:24.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.745+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b8009470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:24.739 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.745+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b8009470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:24.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.745+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:24.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.745+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:24.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.745+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:24.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:24.745+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:25.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:25.549+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b8009470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:25.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:25.549+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b8009470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:25.543 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:25.549+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:25.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:25.549+0000 7f54dbe00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:25.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:25.549+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:25.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:25.549+0000 7f54dbe00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:27.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.141+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:27.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.141+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:27.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.141+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:27.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.141+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:27.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.141+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 msgr2=0x7f54b8009470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:27.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.141+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8009090 0x7f54b8009470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:27.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.141+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2360 con 0x7f54b800a070 2024-03-04T08:44:27.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.141+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b80099b0 2024-03-04T08:44:27.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.141+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:27.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.141+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:27.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.345+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:27.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.345+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:27.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.749+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:27.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:27.749+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:28.553+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:28.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:28.553+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:30.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:30.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:30.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:44:30.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:30.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b800a070 2024-03-04T08:44:30.135 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2360 con 0x7f54b80099b0 2024-03-04T08:44:30.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b80065c0 2024-03-04T08:44:30.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:30.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:30.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:30.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.141+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:30.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.345+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:30.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.345+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:30.339 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.345+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:30.340 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.345+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:30.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.749+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:30.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.749+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:30.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.749+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:30.740 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:30.749+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:31.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:31.553+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:31.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:31.553+0000 7f54dbe00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:31.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:31.553+0000 7f54dbe00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:31.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:31.553+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:33.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:33.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:33.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:33.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:33.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:33.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:33.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8011310 2024-03-04T08:44:33.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b8007e20 2024-03-04T08:44:33.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:33.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:33.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:33.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.141+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:33.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.344+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:33.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.344+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:33.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.344+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:33.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.344+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:33.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.748+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.748+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.748+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:33.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:33.748+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:34.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:34.552+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:34.552+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:34.552+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:34.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:34.552+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:36.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:36.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:36.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:36.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:36.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:36.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:44:36.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b800a070 2024-03-04T08:44:36.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2080 con 0x7f54b80099b0 2024-03-04T08:44:36.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8007e20 2024-03-04T08:44:36.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:36.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:36.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:36.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.140+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:36.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.344+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:36.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.344+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:36.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.344+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:36.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.344+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.748+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.748+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.748+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:36.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:36.748+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:37.547 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:37.552+0000 7f54dbe00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:37.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:37.552+0000 7f54db400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:37.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:37.552+0000 7f54d2a00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:37.548 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:37.552+0000 7f54db400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54d9600640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:39.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54d9600640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b800a070 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8011310 2024-03-04T08:44:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b80065c0 2024-03-04T08:44:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:39.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.140+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:39.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.344+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:39.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.344+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:39.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.344+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:39.337 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.344+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:39.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.748+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:39.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.748+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:39.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.748+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:39.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:39.748+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:40.544 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:40.552+0000 7f54d2a00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:40.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:40.552+0000 7f54d2a00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:40.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:40.552+0000 7f54db400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:40.545 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:40.552+0000 7f54db400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:42.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:42.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:42.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:42.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:42.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54dbe00640 1 -- reap_dead start 2024-03-04T08:44:42.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:42.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:42.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d9600640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2360 con 0x7f54b80099b0 2024-03-04T08:44:42.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d9600640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8011310 2024-03-04T08:44:42.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d9600640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b800af50 con 0x7f54b80065c0 2024-03-04T08:44:42.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:42.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:42.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:42.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:42.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.144+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:42.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.348+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:42.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.348+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:42.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.348+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:42.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.348+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:42.344 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.348+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:42.743 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.752+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:42.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.752+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:42.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.752+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:42.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.752+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:42.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:42.752+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:43.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:43.556+0000 7f54db400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:43.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:43.556+0000 7f54db400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:43.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:43.556+0000 7f54dbe00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:43.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:43.556+0000 7f54d2a00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:43.552 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:43.556+0000 7f54d2a00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:45.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54d9600640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 msgr2=0x7f54b8005ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:45.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54d9600640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f54b80065c0 0x7f54b8005ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:45.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54d9600640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 msgr2=0x7f54b8009090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:45.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54d9600640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f54b8011310 0x7f54b8009090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:45.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54d9600640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f54b80099b0 legacy=0x7f54b8017ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:45.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54d9600640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:45.137 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54d9600640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54dc0c2640 con 0x7f54b80095d0 2024-03-04T08:44:45.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54d9600640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54dc15ae00 con 0x7f54b8007e20 2024-03-04T08:44:45.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80095d0 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:45.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:45.138 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.144+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:45.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.348+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80095d0 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:45.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.348+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:45.343 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.348+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:45.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.752+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80095d0 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:45.744 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.752+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:45.745 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:45.752+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:46.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:46.556+0000 7f54db400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f54b80095d0 legacy=0x7f54b8017590 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:46.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:46.556+0000 7f54d2a00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 msgr2=0x7f54b8006c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:46.551 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:46.556+0000 7f54d2a00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f54b8007e20 0x7f54b8006c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:48.000 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-03-04T08:44:48.001 INFO:tasks.ceph:Scrubbing osd.7 2024-03-04T08:44:48.001 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-04T08:44:48.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.152+0000 7f61c5400640 1 Processor -- start 2024-03-04T08:44:48.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.152+0000 7f61c5400640 1 -- start start 2024-03-04T08:44:48.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.152+0000 7f61c5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 0x7f61c0072b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:48.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.152+0000 7f61c5400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f61c015b9a0 2024-03-04T08:44:48.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.152+0000 7f61c5400640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f61c00c18e0 2024-03-04T08:44:48.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.152+0000 7f61c5400640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f61c00c1500 2024-03-04T08:44:48.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.152+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f61c015b9a0 legacy=0x7f61c0151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:48.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.152+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f61c00c18e0 legacy=0x7f61c015b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:48.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.152+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 msgr2=0x7f61c0072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:48.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.152+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 0x7f61c0072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:48.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.356+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f61c015b9a0 legacy=0x7f61c0151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:48.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.356+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 msgr2=0x7f61c0072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:48.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.356+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f61c00c18e0 legacy=0x7f61c015b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:48.351 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.356+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 0x7f61c0072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.760+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f61c015b9a0 legacy=0x7f61c0151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.760+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 msgr2=0x7f61c0072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.760+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f61c00c18e0 legacy=0x7f61c015b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:48.752 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:48.760+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 0x7f61c0072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:49.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:49.564+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f61c015b9a0 legacy=0x7f61c0151630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:49.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:49.564+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 msgr2=0x7f61c0072b90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:49.559 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:49.564+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f61c00c18e0 legacy=0x7f61c015b290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:49.560 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:49.564+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 0x7f61c0072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:49.803 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:44:49.807 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:44:51.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 msgr2=0x7f61c0072b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:51.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f61c00c1500 0x7f61c0072b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:51.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f61c00c18e0 legacy=0x7f61c015b290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:51.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f61c015b9a0 legacy=0x7f61c0151630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:51.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 0x7f619c005b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:51.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 0x7f619c008670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:51.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c0036b0 2024-03-04T08:44:51.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c0061b0 2024-03-04T08:44:51.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 msgr2=0x7f619c005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:51.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 0x7f619c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:51.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 msgr2=0x7f619c008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:51.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.156+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 0x7f619c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.360+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 msgr2=0x7f619c005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.360+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 0x7f619c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:51.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.360+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 msgr2=0x7f619c008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:51.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.360+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 0x7f619c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:51.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.764+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 msgr2=0x7f619c005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:51.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.764+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 0x7f619c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:51.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.764+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 msgr2=0x7f619c008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:51.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:51.764+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 0x7f619c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:52.568+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 msgr2=0x7f619c005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:52.568+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 0x7f619c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:52.568+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 msgr2=0x7f619c008670 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:52.568+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 0x7f619c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:54.151 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.156+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 msgr2=0x7f619c008670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.156+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0061b0 0x7f619c008670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.156+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 msgr2=0x7f619c005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.156+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c0036b0 0x7f619c005b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.156+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:44:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.156+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.156+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f61c015b9a0 2024-03-04T08:44:54.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.160+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c003b20 2024-03-04T08:44:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.160+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c007d80 2024-03-04T08:44:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.160+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f61c015b9a0 legacy=0x7f619c00b130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.160+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c003b20 legacy=0x7f619c006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.160+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:54.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.160+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.364+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f61c015b9a0 legacy=0x7f619c00b130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.364+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c003b20 legacy=0x7f619c006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:54.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.364+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:54.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.364+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:54.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.768+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f61c015b9a0 legacy=0x7f619c00b130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:54.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.768+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:54.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.768+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c003b20 legacy=0x7f619c006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:54.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:54.768+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:55.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:55.572+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f61c015b9a0 legacy=0x7f619c00b130 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:44:55.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:55.572+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:44:55.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:55.572+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c003b20 legacy=0x7f619c006660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:44:55.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:55.572+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:44:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c003b20 legacy=0x7f619c006660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f61c015b9a0 legacy=0x7f619c00b130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:44:57.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:44:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c011190 2024-03-04T08:44:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c00e860 2024-03-04T08:44:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 msgr2=0x7f619c013580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 msgr2=0x7f619c010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:57.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.160+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:44:57.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.364+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 msgr2=0x7f619c013580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:57.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.364+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:57.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.364+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 msgr2=0x7f619c010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:57.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.364+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:44:57.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.768+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 msgr2=0x7f619c013580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:57.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.768+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:57.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.768+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 msgr2=0x7f619c010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:57.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:57.768+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:44:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:58.572+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 msgr2=0x7f619c013580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:44:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:58.572+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:44:58.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:58.572+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 msgr2=0x7f619c010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:44:58.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:44:58.572+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:00.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 msgr2=0x7f619c010c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:00.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:00.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 msgr2=0x7f619c013580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:00.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:00.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:45:00.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:00.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:00.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:00.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c011190 2024-03-04T08:45:00.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c007d80 2024-03-04T08:45:00.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c006660 con 0x7f619c00e860 2024-03-04T08:45:00.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 msgr2=0x7f619c013580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:00.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:00.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:00.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 msgr2=0x7f619c010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:00.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:00.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.160+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:00.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.364+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 msgr2=0x7f619c013580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:00.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.364+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:00.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.364+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 msgr2=0x7f619c010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:00.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.364+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:00.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.364+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:00.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.364+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:00.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.768+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 msgr2=0x7f619c013580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:00.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.768+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:00.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.768+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 msgr2=0x7f619c010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:00.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.768+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:00.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.768+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:00.761 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:00.768+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:01.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:01.572+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 msgr2=0x7f619c013580 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:01.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:01.572+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:01.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:01.572+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:01.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:01.572+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 msgr2=0x7f619c010c50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:01.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:01.572+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:01.565 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:01.572+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:03.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.160+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 msgr2=0x7f619c010c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:03.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e860 0x7f619c010c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:03.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.160+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:03.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007d80 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:03.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.160+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 msgr2=0x7f619c013580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:03.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c011190 0x7f619c013580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:03.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.160+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c0068b0 2024-03-04T08:45:03.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.160+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c007440 2024-03-04T08:45:03.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.160+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c0068b0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:03.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.160+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c00c100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.364+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c0068b0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:03.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.364+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c00c100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.768+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c0068b0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:03.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:03.768+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c00c100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:04.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:04.572+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c0068b0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:04.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:04.572+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c00c100 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:06.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c00c100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:06.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c0068b0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:06.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:45:06.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c016dd0 2024-03-04T08:45:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c007ae0 2024-03-04T08:45:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c007440 2024-03-04T08:45:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c0191c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:06.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.160+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:06.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.364+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c0191c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:06.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.364+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:06.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.364+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:06.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.364+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.768+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c0191c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.768+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:06.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.768+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:06.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:06.768+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:07.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:07.572+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c0191c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:07.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:07.572+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:07.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:07.572+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:07.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:07.572+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:09.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:09.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:09.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:09.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c0191c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:09.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 0x7f619c00b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:09.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:09.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00b630 2024-03-04T08:45:09.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c013580 2024-03-04T08:45:09.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 msgr2=0x7f619c00da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:09.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:09.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 msgr2=0x7f619c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:09.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.160+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 0x7f619c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:09.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.364+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 msgr2=0x7f619c00da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:09.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.364+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:09.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.364+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 msgr2=0x7f619c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:09.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.364+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 0x7f619c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:09.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.768+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 msgr2=0x7f619c00da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:09.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.768+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:09.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.768+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 msgr2=0x7f619c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:09.760 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:09.768+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 0x7f619c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:10.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:10.572+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 msgr2=0x7f619c00da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:10.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:10.572+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:10.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:10.572+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 msgr2=0x7f619c00b0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:10.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:10.572+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 0x7f619c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:12.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 msgr2=0x7f619c00b0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:12.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c013580 0x7f619c00b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:12.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 msgr2=0x7f619c00da50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:12.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:12.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:45:12.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:12.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c016dd0 2024-03-04T08:45:12.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c007ae0 2024-03-04T08:45:12.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00b630 2024-03-04T08:45:12.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 msgr2=0x7f619c00da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:12.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c00ffa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:12.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:12.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.160+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:12.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.364+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 msgr2=0x7f619c00da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:12.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.364+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c00ffa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:12.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.364+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:12.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.364+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:12.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.768+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 msgr2=0x7f619c00da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:12.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.768+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c00ffa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:12.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.768+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:12.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:12.768+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:13.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:13.572+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 msgr2=0x7f619c00da50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:13.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:13.572+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c00ffa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:13.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:13.572+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:13.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:13.572+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:15.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 msgr2=0x7f619c00da50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:15.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00b630 0x7f619c00da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:15.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:15.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c00ffa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:15.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:15.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 0x7f619c00b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:15.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c007440 2024-03-04T08:45:15.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c013580 2024-03-04T08:45:15.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:15.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:15.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:15.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.160+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.364+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.364+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:15.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.364+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:15.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.364+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.768+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.768+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.768+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:15.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:15.768+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:16.572+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:16.572+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:16.572+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:16.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:16.572+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:18.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:18.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c013580 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c008230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c008230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:45:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c016dd0 2024-03-04T08:45:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c007ae0 2024-03-04T08:45:18.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010790 2024-03-04T08:45:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c00ffa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:18.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.160+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:18.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.364+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c00ffa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:18.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.364+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:18.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.364+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:18.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.364+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:18.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.768+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c00ffa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:18.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.768+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:18.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.768+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:18.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:18.768+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:19.572+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c00ffa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:19.572+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:19.572+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:19.564 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:19.572+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:19.929 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:45:19.932 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:45:21.154 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c016dd0 legacy=0x7f619c00ffa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:21.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010ca0 2024-03-04T08:45:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c007440 2024-03-04T08:45:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:21.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.160+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.364+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.364+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:21.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.364+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:21.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.364+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.768+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.768+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.768+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:21.759 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:21.768+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:22.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:22.572+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:22.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:22.572+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:22.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:22.572+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:22.563 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:22.572+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:24.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:24.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:24.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:24.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:24.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:45:24.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:24.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:24.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c007ae0 2024-03-04T08:45:24.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c007440 2024-03-04T08:45:24.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c010790 2024-03-04T08:45:24.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:24.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:24.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:24.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:24.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.164+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:24.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.368+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:24.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.368+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:24.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.368+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:24.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.368+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:24.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.368+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.772+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.772+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.772+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.772+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:24.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:24.772+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:25.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:25.576+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:25.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:25.576+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:25.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:25.576+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:25.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:25.576+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:25.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:25.576+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:27.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.160+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:27.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.160+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:27.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.160+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:27.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.160+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:27.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.160+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:27.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.164+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 0x7f619c00f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:27.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.164+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c010ca0 2024-03-04T08:45:27.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.164+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00ed60 2024-03-04T08:45:27.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.164+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010ca0 legacy=0x7f619c00d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:27.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.164+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 msgr2=0x7f619c00f140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:27.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.164+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 0x7f619c00f140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:27.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.368+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 msgr2=0x7f619c00f140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:27.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.368+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010ca0 legacy=0x7f619c00d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:27.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.368+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 0x7f619c00f140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.772+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 msgr2=0x7f619c00f140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.772+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010ca0 legacy=0x7f619c00d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:27.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:27.772+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 0x7f619c00f140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:28.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:28.576+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010ca0 legacy=0x7f619c00d640 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:28.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:28.576+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 msgr2=0x7f619c00f140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:28.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:28.576+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 0x7f619c00f140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:30.155 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 msgr2=0x7f619c00f140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:30.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00ed60 0x7f619c00f140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:30.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010ca0 legacy=0x7f619c00d640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:30.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:45:30.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:30.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c010ca0 2024-03-04T08:45:30.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c007ae0 2024-03-04T08:45:30.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010790 2024-03-04T08:45:30.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010ca0 legacy=0x7f619c00f780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:30.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61bf200640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:30.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:30.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.164+0000 7f61bf200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.368+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010ca0 legacy=0x7f619c00f780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.368+0000 7f61bf200640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.368+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:30.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.368+0000 7f61bf200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:30.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.772+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010ca0 legacy=0x7f619c00f780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:30.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.772+0000 7f61bf200640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:30.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.772+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:30.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:30.772+0000 7f61bf200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:31.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:31.575+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010ca0 legacy=0x7f619c00f780 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:31.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:31.575+0000 7f61bf200640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:31.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:31.575+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:31.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:31.575+0000 7f61bf200640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:33.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:33.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010790 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:33.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:33.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010ca0 legacy=0x7f619c00f780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:33.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:33.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 0x7f619c00e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:33.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00fe90 2024-03-04T08:45:33.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c007440 2024-03-04T08:45:33.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 msgr2=0x7f619c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:33.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 0x7f619c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:33.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:33.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.163+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:33.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.367+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 msgr2=0x7f619c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:33.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.367+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 0x7f619c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:33.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.367+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:33.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.367+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.771+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 msgr2=0x7f619c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.771+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 0x7f619c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.771+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:33.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:33.771+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:34.575+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 msgr2=0x7f619c00e650 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:34.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:34.575+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 0x7f619c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:34.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:34.575+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:34.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:34.575+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:36.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:36.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:36.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 msgr2=0x7f619c00e650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:36.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00fe90 0x7f619c00e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:36.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:45:36.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:36.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:36.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c007ae0 2024-03-04T08:45:36.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010790 2024-03-04T08:45:36.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c007440 2024-03-04T08:45:36.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:36.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:36.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:36.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:36.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.163+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:36.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.367+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:36.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.367+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:36.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.367+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:36.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.367+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:36.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.367+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:36.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.771+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:36.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.771+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:36.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.771+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:36.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.771+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:36.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:36.771+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:37.575+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:37.575+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:37.575+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:37.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:37.575+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:37.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:37.575+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:39.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:39.156 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c007440 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:39.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:39.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:39.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007ae0 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:39.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:39.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c010300 2024-03-04T08:45:39.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010ca0 2024-03-04T08:45:39.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:39.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c016dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:39.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.163+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:39.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.367+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c016dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:39.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.367+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:39.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.367+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:39.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.771+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c016dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:39.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.771+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:39.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:39.771+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:40.575+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:40.575+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c016dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:40.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:40.575+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c016dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:45:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:42.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c007440 2024-03-04T08:45:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010ca0 2024-03-04T08:45:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c010790 2024-03-04T08:45:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:42.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.163+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:42.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.367+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:42.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.367+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:42.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.367+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:42.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.367+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:42.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.367+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.771+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.771+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.771+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.771+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:42.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:42.771+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:43.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:43.575+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:43.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:43.575+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:43.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:43.575+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:43.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:43.575+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:43.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:43.575+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:45.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:45.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010790 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:45.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:45.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010ca0 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:45.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:45.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 0x7f619c0081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c010300 2024-03-04T08:45:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c0079b0 2024-03-04T08:45:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c00d620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:45.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.163+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.367+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c00d620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.367+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:45.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.367+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:45.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.771+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c00d620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:45.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.771+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:45.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:45.771+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:46.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:46.575+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c00d620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:46.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:46.575+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:46.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:46.575+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:48.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:48.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0079b0 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:48.157 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c00d620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:48.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:45:48.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 0x7f619c00f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:48.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c010300 2024-03-04T08:45:48.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c007440 2024-03-04T08:45:48.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00ed40 2024-03-04T08:45:48.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c00d620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:48.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 msgr2=0x7f619c00f120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:48.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:48.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.163+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 0x7f619c00f120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:48.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.367+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c00d620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:48.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.367+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:48.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.367+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 msgr2=0x7f619c00f120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:48.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.367+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 0x7f619c00f120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:48.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.771+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c00d620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:48.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.771+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:48.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.771+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 msgr2=0x7f619c00f120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:48.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:48.771+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 0x7f619c00f120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:49.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:49.575+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c00d620 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:49.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:49.575+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:49.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:49.575+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 msgr2=0x7f619c00f120 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:49.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:49.575+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 0x7f619c00f120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:50.048 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:45:50.051 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:45:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 msgr2=0x7f619c00f120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00ed40 0x7f619c00f120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c00d620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010760 2024-03-04T08:45:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c010c70 2024-03-04T08:45:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.163+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:51.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.367+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:51.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.367+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:51.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.367+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:51.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.367+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:51.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.771+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:51.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.771+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:51.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.771+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:51.764 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:51.771+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:52.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:52.575+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:52.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:52.575+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:52.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:52.575+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:52.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:52.575+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:54.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:45:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:54.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c007440 2024-03-04T08:45:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010760 2024-03-04T08:45:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c010c70 2024-03-04T08:45:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:54.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:54.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.163+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:54.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.367+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.367+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.367+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.367+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:54.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.367+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:54.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.771+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:54.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.771+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:54.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.771+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:54.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.771+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:54.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:54.771+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:55.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:55.575+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:45:55.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:55.575+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:55.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:55.575+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:45:55.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:55.575+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:45:55.568 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:55.575+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:45:57.158 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010c70 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:45:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:45:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:45:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c00d620 2024-03-04T08:45:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010300 2024-03-04T08:45:57.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c00d620 legacy=0x7f619c00da00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:57.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:57.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.163+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:45:57.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.367+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c00d620 legacy=0x7f619c00da00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:57.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.367+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:57.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.367+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:45:57.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.771+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c00d620 legacy=0x7f619c00da00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:57.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.771+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:57.763 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:57.771+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:45:58.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:58.575+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c00d620 legacy=0x7f619c00da00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:45:58.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:58.575+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:45:58.567 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:45:58.575+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:00.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:00.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:00.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c00d620 legacy=0x7f619c00da00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:00.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:46:00.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:00.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c00d620 2024-03-04T08:46:00.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c007440 2024-03-04T08:46:00.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010760 2024-03-04T08:46:00.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:00.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c00d620 legacy=0x7f619c00da00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:00.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:00.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.167+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:00.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.371+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c00d620 legacy=0x7f619c00da00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:00.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.371+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:00.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.371+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:00.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.371+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:00.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.775+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c00d620 legacy=0x7f619c00da00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:00.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.775+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:00.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.775+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:00.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:00.775+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:01.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:01.579+0000 7f61bf200640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c00d620 legacy=0x7f619c00da00 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:01.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:01.579+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:01.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:01.579+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:01.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:01.579+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:03.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c00d620 legacy=0x7f619c00da00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 0x7f619c0081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010300 2024-03-04T08:46:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c010c40 2024-03-04T08:46:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:03.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.167+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:03.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.371+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:03.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.371+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:03.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.371+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:03.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.371+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:03.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.775+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:03.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.775+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:03.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.775+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:03.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:03.775+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:04.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:04.579+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:04.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:04.579+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:04.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:04.579+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:04.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:04.579+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:06.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 msgr2=0x7f619c0081d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:06.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c010c40 0x7f619c0081d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:06.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:06.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:06.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:46:06.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c01ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:06.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:06.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:06.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010760 2024-03-04T08:46:06.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c010300 2024-03-04T08:46:06.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c006660 con 0x7f619c00f120 2024-03-04T08:46:06.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:06.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:06.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c01ae90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:06.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:06.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c01ae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:06.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.167+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:06.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.371+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:06.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.371+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:06.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.371+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c01ae90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:06.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.371+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:06.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.371+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c01ae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:06.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.371+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:06.766 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.775+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:06.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.775+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:06.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.775+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c01ae90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:06.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.775+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:06.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.775+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c01ae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:06.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:06.775+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:07.579+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:07.579+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:07.579+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c01ae90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:07.579+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:07.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:07.579+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c01ae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:07.572 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:07.579+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:09.159 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.167+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c01ae90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:09.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c01ae90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:09.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.167+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00d0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:09.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:09.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.167+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:09.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:09.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.167+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c007440 2024-03-04T08:46:09.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.167+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c0081b0 2024-03-04T08:46:09.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.167+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:09.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.167+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c00e630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:09.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.371+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:09.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.371+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c00e630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:09.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.775+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:09.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:09.775+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c00e630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:10.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:10.579+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:10.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:10.579+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c00e630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:12.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c00e630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:12.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:12.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:46:12.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:12.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:12.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c007440 2024-03-04T08:46:12.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010760 2024-03-04T08:46:12.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c00f120 2024-03-04T08:46:12.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:12.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:12.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:12.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:12.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.167+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:12.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.371+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:12.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.371+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:12.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.371+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:12.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.371+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:12.364 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.371+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:12.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.775+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:12.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.775+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:12.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.775+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:12.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.775+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:12.768 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:12.775+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:13.579+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:13.579+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:13.579+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:13.579+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:13.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:13.579+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:15.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:15.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:15.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:15.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:15.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:15.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:15.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c0081b0 2024-03-04T08:46:15.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010300 2024-03-04T08:46:15.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00e630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:15.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.167+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:15.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.371+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00e630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:15.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.371+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:15.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.371+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:15.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.775+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00e630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:15.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.775+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:15.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:15.775+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:16.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:16.579+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00e630 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:16.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:16.579+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:16.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:16.579+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:18.160 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c00e630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:18.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c00e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:18.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:18.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:46:18.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:18.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c007440 2024-03-04T08:46:18.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2080 con 0x7f619c0081b0 2024-03-04T08:46:18.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00f120 2024-03-04T08:46:18.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:18.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:18.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:18.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.167+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.371+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.371+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.371+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:18.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.371+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.775+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.775+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.775+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:18.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:18.775+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:19.579+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:19.579+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:19.579+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:19.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:19.579+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:20.189 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:46:20.194 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:46:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00b0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:21.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c007440 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c01a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c010760 2024-03-04T08:46:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c010300 2024-03-04T08:46:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c01a4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:21.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.167+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c01a4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:21.361 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.371+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:21.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.371+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:21.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.371+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c01a4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:21.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.371+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c01a4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:21.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.775+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:21.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.775+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:21.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.775+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c01a4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:21.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:21.775+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c01a4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:22.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:22.579+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:22.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:22.579+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:22.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:22.579+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c01a4d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:22.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:22.579+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c01a4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:24.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 msgr2=0x7f619c01a4d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:24.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c010300 0x7f619c01a4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:24.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:24.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c010760 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:24.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:46:24.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:24.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:24.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c0081b0 2024-03-04T08:46:24.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00e630 2024-03-04T08:46:24.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c00f120 2024-03-04T08:46:24.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:24.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:24.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:24.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:24.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.167+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:24.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.371+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:24.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.371+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:24.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.371+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:24.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.371+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:24.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.371+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:24.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.775+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:24.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.775+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:24.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.775+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:24.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.775+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:24.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:24.775+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:25.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:25.579+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:25.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:25.579+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:25.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:25.579+0000 7f61bf200640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:25.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:25.579+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:25.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:25.579+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:27.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:27.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00f120 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:27.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:27.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:27.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c0081b0 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:27.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:27.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c010300 2024-03-04T08:46:27.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c007440 2024-03-04T08:46:27.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:27.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c01a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:27.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.167+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:27.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.371+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:27.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.371+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c01a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:27.362 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.371+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.775+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.775+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c01a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:27.767 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:27.775+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:28.579+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:28.579+0000 7f61b5e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c01a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:28.571 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:28.579+0000 7f61b5e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:30.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c01a4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:30.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:30.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:30.162 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:46:30.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:30.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:30.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c010300 2024-03-04T08:46:30.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00e630 2024-03-04T08:46:30.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c007440 2024-03-04T08:46:30.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:30.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:30.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:30.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c01a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:30.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.170+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:30.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.374+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:30.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.374+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:30.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.374+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:30.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.374+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c01a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:30.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.374+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:30.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.778+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:30.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.778+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:30.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.778+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:30.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.778+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c01a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:30.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:30.778+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:31.582+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:31.582+0000 7f61be800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:31.582+0000 7f61be800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:31.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:31.582+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c01a4e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:31.576 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:31.582+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:33.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 msgr2=0x7f619c01a4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:33.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c007440 0x7f619c01a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:33.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:33.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:33.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:33.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 0x7f619c00f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:33.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c00ef50 2024-03-04T08:46:33.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c0081b0 2024-03-04T08:46:33.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:33.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c00ef50 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:33.166 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.170+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:33.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.374+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:33.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.374+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c00ef50 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:33.368 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.374+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:33.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.778+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:33.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.778+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c00ef50 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:33.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:33.778+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:34.582+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:34.582+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c00ef50 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:34.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:34.582+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c0081b0 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c00ef50 legacy=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:46:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 0x7f619c00f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:36.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c010300 2024-03-04T08:46:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00e630 2024-03-04T08:46:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c00d0e0 2024-03-04T08:46:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:36.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:36.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.170+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:36.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.374+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:36.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.374+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:36.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.374+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:36.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.374+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:36.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.374+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.778+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.778+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.778+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.778+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:36.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:36.778+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:37.582+0000 7f61bf200640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:37.582+0000 7f61b5e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:37.582+0000 7f61bf200640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:37.582+0000 7f61be800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:37.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:37.582+0000 7f61be800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61bca00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 msgr2=0x7f619c00f660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:39.163 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00d0e0 0x7f619c00f660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c007440 2024-03-04T08:46:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00ef50 2024-03-04T08:46:39.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c0184c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 msgr2=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:39.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.170+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:39.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.374+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 msgr2=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.374+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c0184c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:39.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.374+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:39.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.778+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 msgr2=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.778+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c0184c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:39.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:39.778+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:40.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:40.582+0000 7f61be800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 msgr2=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:40.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:40.582+0000 7f61b5e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c0184c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:40.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:40.582+0000 7f61be800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 msgr2=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bca00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c0184c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bf200640 1 -- reap_dead start 2024-03-04T08:46:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:42.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bca00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2360 con 0x7f619c010300 2024-03-04T08:46:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bca00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00e630 2024-03-04T08:46:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bca00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c008d40 con 0x7f619c00ef50 2024-03-04T08:46:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 msgr2=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:42.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.170+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:42.366 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.374+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:42.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.374+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:42.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.374+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:42.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.374+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 msgr2=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:42.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.374+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:42.770 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.778+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:42.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.778+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:42.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.778+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:42.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.778+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 msgr2=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:42.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:42.778+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:43.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:43.582+0000 7f61b5e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:43.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:43.582+0000 7f61be800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:43.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:43.582+0000 7f61b5e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:43.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:43.582+0000 7f61bf200640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 msgr2=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:43.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:43.582+0000 7f61bf200640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61bca00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 msgr2=0x7f619c015da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f619c00ef50 0x7f619c015da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61bca00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 msgr2=0x7f619c00ea10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61bca00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f619c00e630 0x7f619c00ea10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:45.164 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61bca00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f619c010300 legacy=0x7f619c006c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:45.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61bca00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e0f0 0x7f619c00f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:45.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61bca00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00c2640 con 0x7f619c007440 2024-03-04T08:46:45.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61bca00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c015ae00 con 0x7f619c00e0f0 2024-03-04T08:46:45.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c0182f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:45.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e0f0 msgr2=0x7f619c00f5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:45.165 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.170+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e0f0 0x7f619c00f5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:45.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.374+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c0182f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:45.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.374+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e0f0 msgr2=0x7f619c00f5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:45.367 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.374+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e0f0 0x7f619c00f5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:45.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.778+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c0182f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:45.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.778+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e0f0 msgr2=0x7f619c00f5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:45.771 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:45.778+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e0f0 0x7f619c00f5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:46.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:46.582+0000 7f61be800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f619c007440 legacy=0x7f619c0182f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:46.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:46.582+0000 7f61b5e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e0f0 msgr2=0x7f619c00f5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:46.575 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:46.582+0000 7f61b5e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f619c00e0f0 0x7f619c00f5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:48.029 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-03-04T08:46:48.031 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-04T08:46:48.174 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.178+0000 7f7bd5000640 1 Processor -- start 2024-03-04T08:46:48.175 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.178+0000 7f7bd5000640 1 -- start start 2024-03-04T08:46:48.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.182+0000 7f7bd5000640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 0x7f7bd0070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:48.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.182+0000 7f7bd5000640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 0x7f7bd0072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:48.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.182+0000 7f7bd5000640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bd0152bd0 2024-03-04T08:46:48.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.182+0000 7f7bd5000640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bd00c8200 2024-03-04T08:46:48.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.182+0000 7f7bd5000640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bd01527f0 2024-03-04T08:46:48.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.182+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bd0152bd0 legacy=0x7f7bd01587d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:48.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.182+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 msgr2=0x7f7bd0070d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:48.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.182+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 msgr2=0x7f7bd0072280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:48.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.182+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 0x7f7bd0072280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:48.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.182+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 0x7f7bd0070d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:48.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.386+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bd0152bd0 legacy=0x7f7bd01587d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:48.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.386+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 msgr2=0x7f7bd0072280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:48.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.386+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 msgr2=0x7f7bd0070d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:48.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.386+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 0x7f7bd0072280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:48.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.386+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 0x7f7bd0070d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:48.784 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.790+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bd0152bd0 legacy=0x7f7bd01587d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:48.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.790+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 msgr2=0x7f7bd0070d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:48.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.790+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 msgr2=0x7f7bd0072280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:48.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.790+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 0x7f7bd0072280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:48.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:48.790+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 0x7f7bd0070d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:49.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:49.594+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bd0152bd0 legacy=0x7f7bd01587d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:49.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:49.594+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 msgr2=0x7f7bd0070d40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:49.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:49.594+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 msgr2=0x7f7bd0072280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:49.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:49.594+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 0x7f7bd0072280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:49.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:49.594+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 0x7f7bd0070d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:50.254 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:46:50.257 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:46:51.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 msgr2=0x7f7bd0070d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:51.176 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bd01527f0 0x7f7bd0070d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:51.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 msgr2=0x7f7bd0072280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:51.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bd00c8200 0x7f7bd0072280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:51.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bd0152bd0 legacy=0x7f7bd01587d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:51.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 0x7f7bac009840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:51.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac0036d0 2024-03-04T08:46:51.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac007380 2024-03-04T08:46:51.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0036d0 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:51.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 msgr2=0x7f7bac009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:51.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.182+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 0x7f7bac009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:51.376 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.386+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 msgr2=0x7f7bac009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:51.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.386+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0036d0 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:51.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.386+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 0x7f7bac009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:51.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.790+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0036d0 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:51.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.790+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 msgr2=0x7f7bac009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:51.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:51.790+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 0x7f7bac009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:52.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:52.594+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 msgr2=0x7f7bac009840 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:52.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:52.594+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0036d0 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:52.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:52.594+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 0x7f7bac009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:54.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 msgr2=0x7f7bac009840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:54.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac007380 0x7f7bac009840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:54.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0036d0 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:54.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:46:54.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:54.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac0036d0 2024-03-04T08:46:54.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac009f00 2024-03-04T08:46:54.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:46:54.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0036d0 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:54.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00e140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:54.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:54.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.182+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac009f00 legacy=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:54.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.386+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00e140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:54.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.386+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0036d0 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:54.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.386+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:54.379 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.386+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac009f00 legacy=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:54.782 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.790+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00e140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.790+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0036d0 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.790+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:54.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:54.790+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac009f00 legacy=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:55.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:55.594+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00e140 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:46:55.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:55.594+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0036d0 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:46:55.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:55.594+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:55.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:55.594+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac009f00 legacy=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:46:57.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00e140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:57.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:46:57.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac009f00 legacy=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:57.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0036d0 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:46:57.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:57.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:46:57.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00e680 2024-03-04T08:46:57.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac0110e0 2024-03-04T08:46:57.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 msgr2=0x7f7bac013500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:57.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 msgr2=0x7f7bac010a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:57.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:57.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.182+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:46:57.377 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.386+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 msgr2=0x7f7bac010a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:57.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.386+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 msgr2=0x7f7bac013500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:57.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.386+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:57.378 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.386+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:46:57.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.790+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 msgr2=0x7f7bac010a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:57.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.790+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 msgr2=0x7f7bac013500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:57.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.790+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:57.783 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:57.790+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:46:58.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:58.594+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 msgr2=0x7f7bac010a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:46:58.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:58.594+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 msgr2=0x7f7bac013500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:46:58.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:58.594+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:46:58.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:46:58.594+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:00.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 msgr2=0x7f7bac013500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:00.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:00.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 msgr2=0x7f7bac010a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:00.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:00.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:47:00.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:00.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:00.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:00.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:47:00.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac00e680 2024-03-04T08:47:00.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac00e180 con 0x7f7bac0110e0 2024-03-04T08:47:00.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:00.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:00.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 msgr2=0x7f7bac010a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:00.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 msgr2=0x7f7bac013500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:00.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:00.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.186+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:00.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.390+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.390+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.390+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 msgr2=0x7f7bac010a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.390+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 msgr2=0x7f7bac013500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.390+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:00.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.390+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.794+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.794+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.794+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 msgr2=0x7f7bac010a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.794+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 msgr2=0x7f7bac013500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.794+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:00.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:00.794+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:01.598+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:01.598+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:01.598+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 msgr2=0x7f7bac010a70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:01.598+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 msgr2=0x7f7bac013500 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:01.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:01.598+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:01.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:01.598+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:03.177 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.186+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 msgr2=0x7f7bac013500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.186+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0110e0 0x7f7bac013500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.186+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 msgr2=0x7f7bac010a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00e680 0x7f7bac010a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.186+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.186+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac006990 2024-03-04T08:47:03.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.186+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac00a2e0 2024-03-04T08:47:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.186+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:03.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.186+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00a2e0 legacy=0x7f7bac006280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:03.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.390+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:03.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.390+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00a2e0 legacy=0x7f7bac006280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:03.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.794+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:03.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:03.794+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00a2e0 legacy=0x7f7bac006280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:04.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:04.598+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:04.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:04.598+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00a2e0 legacy=0x7f7bac006280 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:06.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00a2e0 legacy=0x7f7bac006280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:06.178 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:06.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:47:06.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:06.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac006990 2024-03-04T08:47:06.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac006d70 2024-03-04T08:47:06.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:47:06.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:06.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:06.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:06.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.186+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006d70 legacy=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:06.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.390+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:06.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.390+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:06.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.390+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:06.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.390+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006d70 legacy=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:06.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.794+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:06.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.794+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:06.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.794+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:06.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:06.794+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006d70 legacy=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:07.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:07.598+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:07.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:07.598+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:07.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:07.598+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:07.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:07.598+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006d70 legacy=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:09.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:09.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:09.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006d70 legacy=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:09.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:09.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac0134d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:09.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 0x7f7bac0109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:09.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac006280 2024-03-04T08:47:09.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac00e5c0 2024-03-04T08:47:09.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac0134d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:09.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 msgr2=0x7f7bac0109e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:09.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac0134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:09.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.186+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 0x7f7bac0109e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:09.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.390+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 msgr2=0x7f7bac0109e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:09.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.390+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac0134d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:09.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.390+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac0134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:09.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.390+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 0x7f7bac0109e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:09.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.794+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac0134d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:09.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.794+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 msgr2=0x7f7bac0109e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:09.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.794+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac0134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:09.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:09.794+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 0x7f7bac0109e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:10.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:10.598+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 msgr2=0x7f7bac0109e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:10.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:10.598+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac0134d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:10.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:10.598+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac0134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:10.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:10.598+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 0x7f7bac0109e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:12.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 msgr2=0x7f7bac0109e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00e5c0 0x7f7bac0109e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac0134d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac0134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:47:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac006990 2024-03-04T08:47:12.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:47:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac006280 2024-03-04T08:47:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:12.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.186+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:12.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.390+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:12.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.390+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:12.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.390+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:12.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.390+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:12.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.390+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:12.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.794+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:12.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.794+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:12.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.794+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:12.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.794+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:12.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:12.794+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:13.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:13.598+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:13.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:13.598+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b30 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:13.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:13.598+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:13.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:13.598+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:13.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:13.598+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:15.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:15.179 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:15.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:15.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:15.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac005b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:15.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 0x7f7bac0134d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:15.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac00e5c0 2024-03-04T08:47:15.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac006d70 2024-03-04T08:47:15.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 msgr2=0x7f7bac0134d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:15.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00e5c0 legacy=0x7f7bac00b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:15.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.186+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 0x7f7bac0134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:15.382 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.390+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 msgr2=0x7f7bac0134d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:15.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.390+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00e5c0 legacy=0x7f7bac00b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:15.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.390+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 0x7f7bac0134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:15.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.794+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 msgr2=0x7f7bac0134d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:15.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.794+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00e5c0 legacy=0x7f7bac00b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:15.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:15.794+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 0x7f7bac0134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:16.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:16.598+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00e5c0 legacy=0x7f7bac00b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:16.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:16.598+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 msgr2=0x7f7bac0134d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:16.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:16.598+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 0x7f7bac0134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:18.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 msgr2=0x7f7bac0134d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:18.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006d70 0x7f7bac0134d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:18.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00e5c0 legacy=0x7f7bac00b0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:18.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:47:18.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 0x7f7bac00b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:18.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:18.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:18.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:47:18.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac006280 2024-03-04T08:47:18.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac00e180 con 0x7f7bac0109b0 2024-03-04T08:47:18.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:18.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:18.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:18.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 msgr2=0x7f7bac00b7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:18.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:18.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.186+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 0x7f7bac00b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:18.380 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.390+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.390+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.390+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.390+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 msgr2=0x7f7bac00b7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.390+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:18.381 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.390+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 0x7f7bac00b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:18.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.794+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:18.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.794+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:18.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.794+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:18.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.794+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 msgr2=0x7f7bac00b7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:18.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.794+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:18.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:18.794+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 0x7f7bac00b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:19.590 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:19.598+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:19.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:19.598+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:19.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:19.598+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:19.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:19.598+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 msgr2=0x7f7bac00b7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:19.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:19.598+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:19.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:19.598+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 0x7f7bac00b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:20.306 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:47:20.310 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:47:21.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.190+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 msgr2=0x7f7bac00b7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:21.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.190+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac0109b0 0x7f7bac00b7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:21.180 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.190+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.190+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.190+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac013c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.190+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac013c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.190+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac006990 2024-03-04T08:47:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.190+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac00bd30 2024-03-04T08:47:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.190+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac00b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:21.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.190+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00bd30 legacy=0x7f7bac00f5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:21.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.394+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac00b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:21.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.394+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00bd30 legacy=0x7f7bac00f5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:21.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.798+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac00b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:21.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:21.798+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00bd30 legacy=0x7f7bac00f5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:22.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:22.602+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac00b0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:22.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:22.602+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00bd30 legacy=0x7f7bac00f5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00bd30 legacy=0x7f7bac00f5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006990 legacy=0x7f7bac00b0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:47:24.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac0134d0 2024-03-04T08:47:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:47:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac006280 2024-03-04T08:47:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:24.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:24.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:24.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.190+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:24.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.394+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:24.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.394+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:24.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.394+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:24.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.394+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:24.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.394+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:24.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.798+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:24.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.798+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:24.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.798+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:24.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.798+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:24.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:24.798+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:25.602+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:25.602+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fce0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:25.602+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:25.602+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:25.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:25.602+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:27.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:27.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:27.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:27.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:27.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:27.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 0x7f7bac0041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:27.181 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac0103f0 2024-03-04T08:47:27.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac010800 2024-03-04T08:47:27.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0103f0 legacy=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:27.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 msgr2=0x7f7bac0041f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:27.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.189+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 0x7f7bac0041f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:27.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.393+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0103f0 legacy=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:27.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.393+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 msgr2=0x7f7bac0041f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:27.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.393+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 0x7f7bac0041f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:27.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.797+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0103f0 legacy=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.797+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 msgr2=0x7f7bac0041f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:27.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:27.797+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 0x7f7bac0041f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:28.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:28.601+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0103f0 legacy=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:28.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:28.601+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 msgr2=0x7f7bac0041f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:28.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:28.601+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 0x7f7bac0041f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 msgr2=0x7f7bac0041f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010800 0x7f7bac0041f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0103f0 legacy=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:47:30.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:30.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:30.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac004730 2024-03-04T08:47:30.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:47:30.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac006280 2024-03-04T08:47:30.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:30.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:30.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac004730 legacy=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:30.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:30.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.189+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.393+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:30.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.393+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:30.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.393+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:30.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.393+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac004730 legacy=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:30.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.393+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:30.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.797+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:30.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.797+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:30.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.797+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:30.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.797+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac004730 legacy=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:30.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:30.797+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:31.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:31.601+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:31.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:31.601+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:31.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:31.601+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:31.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:31.601+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac004730 legacy=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:31.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:31.601+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:33.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 msgr2=0x7f7bac00a2e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:33.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac006280 0x7f7bac00a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:33.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:33.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:33.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac004730 legacy=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:33.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 0x7f7bac003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:33.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac0134d0 2024-03-04T08:47:33.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac0107d0 2024-03-04T08:47:33.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:33.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:33.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.189+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:33.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.393+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:33.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.393+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:33.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.393+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:33.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.797+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.797+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:33.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:33.797+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:34.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:34.601+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:34.601+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:34.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:34.601+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:36.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0107d0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:47:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 0x7f7bac003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac0134d0 2024-03-04T08:47:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:47:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac004020 2024-03-04T08:47:36.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bce800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:36.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.189+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.393+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:36.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.393+0000 7f7bce800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:36.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.393+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:36.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.393+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:36.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.393+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:36.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.797+0000 7f7bce800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:36.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.797+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:36.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.797+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:36.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.797+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:36.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:36.797+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:37.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:37.601+0000 7f7bce800640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:37.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:37.601+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:37.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:37.601+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:37.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:37.601+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:37.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:37.601+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:39.182 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:39.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac004020 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:39.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:39.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:39.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:39.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:39.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac006280 2024-03-04T08:47:39.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00a2e0 2024-03-04T08:47:39.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac004400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:39.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:39.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.189+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.393+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac004400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.393+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:39.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.393+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:39.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.797+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac004400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:39.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.797+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:39.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:39.797+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:40.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:40.601+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac004400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:40.601+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:40.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:40.601+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:42.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:42.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:42.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac004400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:42.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:47:42.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:42.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac0134d0 2024-03-04T08:47:42.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac006280 2024-03-04T08:47:42.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00a2e0 2024-03-04T08:47:42.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:42.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:42.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac004400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:42.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.189+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:42.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.393+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:42.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.393+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac004400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:42.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.393+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:42.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.393+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:42.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.797+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:42.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.797+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac004400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:42.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.797+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:42.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:42.797+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:43.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:43.601+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:43.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:43.601+0000 7f7bce800640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac004400 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:43.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:43.601+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:43.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:43.601+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:45.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:45.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:45.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac004400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:45.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:45.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:45.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 0x7f7bac00f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:45.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac003ae0 2024-03-04T08:47:45.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac00ab20 2024-03-04T08:47:45.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 msgr2=0x7f7bac00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:45.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 0x7f7bac00f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:45.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:45.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.189+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:45.383 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.393+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 msgr2=0x7f7bac00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:45.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.393+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 0x7f7bac00f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:45.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.393+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:45.384 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.393+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:45.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.797+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 msgr2=0x7f7bac00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:45.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.797+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 0x7f7bac00f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:45.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.797+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:45.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:45.797+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:46.591 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:46.601+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 msgr2=0x7f7bac00f560 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:46.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:46.601+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 0x7f7bac00f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:46.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:46.601+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:46.592 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:46.601+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:48.183 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 msgr2=0x7f7bac00f560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ae0 0x7f7bac00f560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:47:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:48.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac0134d0 2024-03-04T08:47:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac006280 2024-03-04T08:47:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00a2e0 2024-03-04T08:47:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:48.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.189+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:48.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.393+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:48.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.393+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:48.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.393+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:48.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.393+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:48.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.797+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:48.790 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.797+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:48.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.797+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:48.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:48.797+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:49.594 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:49.601+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:49.601+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:49.601+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:49.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:49.601+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:50.418 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:47:50.422 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:47:51.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:51.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:51.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:51.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00faa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:51.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:51.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:51.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:47:51.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac00f4b0 2024-03-04T08:47:51.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:51.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:51.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:51.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.193+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.397+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.397+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.397+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.397+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:51.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.801+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:51.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.801+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:51.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.801+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:51.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:51.801+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:52.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:52.605+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:52.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:52.605+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:52.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:52.605+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:52.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:52.605+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:54.184 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:47:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac0134d0 2024-03-04T08:47:54.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac006280 2024-03-04T08:47:54.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00f4b0 2024-03-04T08:47:54.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac004020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:54.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:54.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:54.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.193+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.397+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac004020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:54.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.397+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:54.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.397+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:54.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.397+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:54.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.801+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac004020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:54.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.801+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:54.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.801+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:54.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:54.801+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:55.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:55.605+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac004020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:47:55.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:55.605+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:47:55.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:55.605+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:47:55.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:55.605+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:57.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 msgr2=0x7f7bac003ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:57.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00f4b0 0x7f7bac003ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:47:57.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:57.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac004020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:47:57.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:57.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:47:57.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:47:57.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac00a2e0 2024-03-04T08:47:57.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:57.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:57.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:57.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.193+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:47:57.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.397+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.397+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.397+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:57.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.397+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:47:57.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.801+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.801+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.801+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:57.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:57.801+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:47:58.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:58.605+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:47:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:58.605+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:47:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:58.605+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:47:58.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:47:58.605+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:00.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:00.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:00.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:00.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:00.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:48:00.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:00.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:00.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac006280 2024-03-04T08:48:00.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:48:00.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac00a2e0 2024-03-04T08:48:00.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:00.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:00.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:00.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:00.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.193+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.397+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:00.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.397+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:00.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.397+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:00.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.397+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:00.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.397+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:00.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.801+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:00.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.801+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:00.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.801+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:00.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.801+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:00.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:00.801+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:01.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:01.605+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:01.605+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:01.605+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:01.605+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:01.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:01.605+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:03.185 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac00bcf0 2024-03-04T08:48:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac0134d0 2024-03-04T08:48:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:03.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:03.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.193+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:03.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.397+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:03.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.397+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:03.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.397+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:03.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.801+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:03.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.801+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:03.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:03.801+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:04.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:04.605+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:04.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:04.605+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:04.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:04.605+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:06.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:06.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:06.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:06.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:48:06.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:06.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:06.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac00bcf0 2024-03-04T08:48:06.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:48:06.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac0134d0 2024-03-04T08:48:06.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:06.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:06.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:06.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:06.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.193+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:06.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.397+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:06.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.397+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:06.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.397+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:06.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.397+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:06.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.397+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:06.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.801+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:06.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.801+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:06.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.801+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:06.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.801+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:06.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:06.801+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:07.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:07.605+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:07.605+0000 7f7bcde00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:07.605+0000 7f7bcde00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:07.605+0000 7f7bce800640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:07.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:07.605+0000 7f7bce800640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:09.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:09.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac0134d0 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:09.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:09.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:09.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac006280 2024-03-04T08:48:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00a2e0 2024-03-04T08:48:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:09.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.193+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:09.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.397+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:09.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.397+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:09.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.397+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:09.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.801+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:09.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.801+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:09.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:09.801+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:10.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:10.605+0000 7f7bcde00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:10.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:10.605+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:10.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:10.605+0000 7f7bcde00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:12.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac006280 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:48:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:12.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac00bcf0 2024-03-04T08:48:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00ab20 2024-03-04T08:48:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac00a2e0 2024-03-04T08:48:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:12.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:12.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.193+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:12.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.397+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:12.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.397+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:12.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.397+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:12.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.397+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:12.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.397+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:12.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.801+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:12.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.801+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:12.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.801+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:12.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.801+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:12.792 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:12.801+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:13.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:13.605+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:13.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:13.605+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:13.598 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:13.605+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:13.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:13.605+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:13.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:13.605+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:15.186 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:15.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:15.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 msgr2=0x7f7bac00b0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:15.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac00ab20 0x7f7bac00b0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:15.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:15.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:15.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac0134d0 2024-03-04T08:48:15.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac006280 2024-03-04T08:48:15.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:15.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:15.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.193+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.397+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.397+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:15.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.397+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:15.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.801+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:15.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.801+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:15.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:15.801+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:16.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:16.605+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:16.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:16.605+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:16.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:16.605+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:18.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:18.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:18.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:18.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:48:18.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:18.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac00bcf0 2024-03-04T08:48:18.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01591f0 con 0x7f7bac0134d0 2024-03-04T08:48:18.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00a2e0 2024-03-04T08:48:18.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:18.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:18.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:18.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.193+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.397+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:18.387 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.397+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:18.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.397+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:18.388 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.397+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:18.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.801+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:18.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.801+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:18.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.801+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:18.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:18.801+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:19.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:19.605+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:19.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:19.605+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:19.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:19.605+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:19.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:19.605+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:20.538 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:48:20.542 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:48:21.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:21.187 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:21.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:21.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:21.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:21.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:21.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac010ce0 2024-03-04T08:48:21.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac006280 2024-03-04T08:48:21.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:21.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:21.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.197+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.401+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.401+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:21.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.401+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:21.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.401+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:21.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.805+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:21.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.805+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:21.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.805+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:21.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:21.805+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:22.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:22.609+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:22.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:22.609+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:22.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:22.609+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:22.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:22.609+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:24.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:24.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:24.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:24.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:48:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac00bcf0 2024-03-04T08:48:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac010ce0 2024-03-04T08:48:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac00a2e0 2024-03-04T08:48:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:24.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:24.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:24.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:24.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.197+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:24.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.401+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:24.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.401+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:24.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.401+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:24.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.401+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:24.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.401+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:24.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.805+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:24.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.805+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:24.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.805+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:24.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.805+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:24.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:24.805+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:25.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:25.608+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:25.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:25.608+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:25.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:25.608+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:25.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:25.608+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:25.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:25.608+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:27.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:27.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:27.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:27.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:27.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bcf0 legacy=0x7f7bac00f9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:27.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:27.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac00aac0 2024-03-04T08:48:27.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac006280 2024-03-04T08:48:27.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:27.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00aac0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:27.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.192+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:27.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.396+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:27.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.396+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00aac0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:27.389 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.396+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:27.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.800+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:27.794 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.800+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00aac0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:27.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:27.800+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:28.595 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:28.604+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:28.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:28.604+0000 7f7bc5400640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00aac0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:28.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:28.604+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:30.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 msgr2=0x7f7bac00f4b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:30.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006280 0x7f7bac00f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:30.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac00aac0 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:30.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:48:30.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:30.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:30.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac0134d0 2024-03-04T08:48:30.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac010ce0 2024-03-04T08:48:30.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac00a2e0 2024-03-04T08:48:30.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:30.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:30.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac01d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:30.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:30.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.196+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:30.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.400+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:30.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.400+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:30.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.400+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac01d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:30.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.400+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:30.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.400+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.804+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:30.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.804+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.804+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac01d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.804+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:30.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:30.804+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:31.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:31.608+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:31.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:31.608+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:31.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:31.608+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac01d030 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:31.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:31.608+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:31.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:31.608+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:33.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:33.188 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:33.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:33.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:33.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac01d030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:33.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 0x7f7bac00f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:33.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac006260 2024-03-04T08:48:33.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00bcf0 2024-03-04T08:48:33.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006260 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:33.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 msgr2=0x7f7bac00f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:33.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.196+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 0x7f7bac00f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:33.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.400+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006260 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:33.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.400+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 msgr2=0x7f7bac00f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:33.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.400+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 0x7f7bac00f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:33.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.804+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006260 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:33.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.804+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 msgr2=0x7f7bac00f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:33.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:33.804+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 0x7f7bac00f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:34.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:34.608+0000 7f7bcde00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006260 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:34.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:34.608+0000 7f7bc5400640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 msgr2=0x7f7bac00f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:34.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:34.608+0000 7f7bc5400640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 0x7f7bac00f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:36.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 msgr2=0x7f7bac00f480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:36.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac00bcf0 0x7f7bac00f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:36.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006260 legacy=0x7f7bac00b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:36.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:48:36.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:36.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:36.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac0134d0 2024-03-04T08:48:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac010ce0 2024-03-04T08:48:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac00a2e0 2024-03-04T08:48:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:36.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.196+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:36.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.400+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:36.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.400+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:36.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.400+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:36.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.400+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:36.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.400+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:36.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.804+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:36.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.804+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:36.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.804+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:36.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.804+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:36.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:36.804+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:37.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:37.608+0000 7f7bce800640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:37.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:37.608+0000 7f7bce800640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:37.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:37.608+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:37.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:37.608+0000 7f7bc5400640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:37.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:37.608+0000 7f7bc5400640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:39.189 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac005ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:39.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac005ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:39.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bc7e00640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:39.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:39.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fa20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:39.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 0x7f7bac00b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:39.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac00bb00 2024-03-04T08:48:39.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac006260 2024-03-04T08:48:39.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bb00 legacy=0x7f7bac00e470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:39.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 msgr2=0x7f7bac00b5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:39.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.196+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 0x7f7bac00b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:39.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.400+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bb00 legacy=0x7f7bac00e470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:39.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.400+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 msgr2=0x7f7bac00b5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:39.395 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.400+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 0x7f7bac00b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:39.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.804+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bb00 legacy=0x7f7bac00e470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:39.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.804+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 msgr2=0x7f7bac00b5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:39.795 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:39.804+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 0x7f7bac00b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:40.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:40.608+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bb00 legacy=0x7f7bac00e470 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:40.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:40.608+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 msgr2=0x7f7bac00b5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:40.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:40.608+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 0x7f7bac00b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:42.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bc7e00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 msgr2=0x7f7bac00b5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:42.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac006260 0x7f7bac00b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:42.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac00bb00 legacy=0x7f7bac00e470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:42.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bce800640 1 -- reap_dead start 2024-03-04T08:48:42.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:42.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bc7e00640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd01594d0 con 0x7f7bac006260 2024-03-04T08:48:42.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bc7e00640 1 -- --> v1:172.21.15.136:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac009b70 con 0x7f7bac0134d0 2024-03-04T08:48:42.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bc7e00640 1 -- --> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac00a2e0 2024-03-04T08:48:42.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006260 legacy=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:42.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:42.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:42.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.192+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:42.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.400+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006260 legacy=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:42.390 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.400+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:42.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.400+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:42.391 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.400+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:42.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.804+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006260 legacy=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:42.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.804+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:42.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.804+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:42.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:42.804+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:43.599 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:43.608+0000 7f7bc5400640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006260 legacy=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.113:6789/0 2024-03-04T08:48:43.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:43.608+0000 7f7bcde00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fa20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.136:6789/0 2024-03-04T08:48:43.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:43.608+0000 7f7bce800640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00f480 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3301/0 2024-03-04T08:48:43.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:43.608+0000 7f7bce800640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:45.190 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.196+0000 7f7bc7e00640 1 -- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 msgr2=0x7f7bac00f480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:45.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.196+0000 7f7bc7e00640 1 --2- >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x7f7bac00a2e0 0x7f7bac00f480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T08:48:45.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.196+0000 7f7bc7e00640 1 -- >> v1:172.21.15.136:6789/0 conn(0x7f7bac0134d0 legacy=0x7f7bac00fa20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:45.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.196+0000 7f7bc7e00640 1 -- >> v1:172.21.15.113:6789/0 conn(0x7f7bac006260 legacy=0x7f7bac00b080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-04T08:48:45.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.196+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:45.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.196+0000 7f7bc7e00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ab0 0x7f7bac01b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T08:48:45.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.196+0000 7f7bc7e00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd0071df0 con 0x7f7bac003ab0 2024-03-04T08:48:45.191 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.196+0000 7f7bc7e00640 1 -- --> v2:172.21.15.136:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd01597b0 con 0x7f7bac010ce0 2024-03-04T08:48:45.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.200+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ab0 msgr2=0x7f7bac01b020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:45.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.200+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ab0 0x7f7bac01b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:45.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.200+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:45.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.200+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T08:48:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.404+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ab0 msgr2=0x7f7bac01b020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.404+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ab0 0x7f7bac01b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.404+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:45.394 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.404+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T08:48:45.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.808+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ab0 msgr2=0x7f7bac01b020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:45.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.808+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ab0 0x7f7bac01b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:45.798 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.808+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:45.799 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:45.808+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T08:48:46.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:46.612+0000 7f7bc5400640 1 -- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ab0 msgr2=0x7f7bac01b020 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.113:3300/0 2024-03-04T08:48:46.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:46.612+0000 7f7bc5400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7bac003ab0 0x7f7bac01b020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:46.602 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:46.612+0000 7f7bcde00640 1 -- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010ce0 msgr2=0x7f7bac00b790 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.136:3300/0 2024-03-04T08:48:46.603 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-04T08:48:46.612+0000 7f7bcde00640 1 --2- >> v2:172.21.15.136:3300/0 conn(0x7f7bac010ce0 0x7f7bac00b790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T08:48:48.059 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-03-04T08:48:48.060 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph.py", line 1929, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph.py", line 1281, in osd_scrub_pgs stats = manager.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 2383, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 1638, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 1629, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-03-04T08:48:48.075 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-04T08:48:48.075 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-04T08:48:48.075 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.098 INFO:tasks.ceph.mds.a:Stopped 2024-03-04T08:48:48.098 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-04T08:48:48.098 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.159 INFO:tasks.ceph.mds.c:Stopped 2024-03-04T08:48:48.159 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-03-04T08:48:48.159 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.220 INFO:tasks.ceph.mds.e:Stopped 2024-03-04T08:48:48.221 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-04T08:48:48.221 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.221 INFO:tasks.ceph.mds.b:Stopped 2024-03-04T08:48:48.221 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-04T08:48:48.221 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.222 INFO:tasks.ceph.mds.d:Stopped 2024-03-04T08:48:48.222 ERROR:tasks.ceph.mds.f:tried to stop a non-running daemon 2024-03-04T08:48:48.224 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-04T08:48:48.225 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-04T08:48:48.225 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.226 INFO:tasks.ceph.osd.0:Stopped 2024-03-04T08:48:48.226 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-04T08:48:48.226 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.226 INFO:tasks.ceph.osd.1:Stopped 2024-03-04T08:48:48.227 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-04T08:48:48.227 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.227 INFO:tasks.ceph.osd.2:Stopped 2024-03-04T08:48:48.227 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-04T08:48:48.227 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.227 INFO:tasks.ceph.osd.3:Stopped 2024-03-04T08:48:48.228 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-04T08:48:48.228 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.228 INFO:tasks.ceph.osd.4:Stopped 2024-03-04T08:48:48.228 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-04T08:48:48.228 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.228 INFO:tasks.ceph.osd.5:Stopped 2024-03-04T08:48:48.228 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-04T08:48:48.229 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.229 INFO:tasks.ceph.osd.6:Stopped 2024-03-04T08:48:48.229 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-04T08:48:48.229 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.229 INFO:tasks.ceph.osd.7:Stopped 2024-03-04T08:48:48.229 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-04T08:48:48.229 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-04T08:48:48.230 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:48:48.230 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph.py", line 1929, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph.py", line 1281, in osd_scrub_pgs stats = manager.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 2383, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 1638, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 1629, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y' 2024-03-04T08:48:48.243 INFO:tasks.ceph.mgr.y:Stopped 2024-03-04T08:48:48.243 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-04T08:48:48.243 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.243 INFO:tasks.ceph.mgr.x:Stopped 2024-03-04T08:48:48.243 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-04T08:48:48.243 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-04T08:48:48.243 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.244 INFO:tasks.ceph.mon.a:Stopped 2024-03-04T08:48:48.244 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-04T08:48:48.244 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.244 INFO:tasks.ceph.mon.b:Stopped 2024-03-04T08:48:48.244 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-04T08:48:48.244 INFO:teuthology.orchestra.run:waiting for 300 2024-03-04T08:48:48.244 INFO:tasks.ceph.mon.c:Stopped 2024-03-04T08:48:48.245 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-04T08:48:48.245 DEBUG:teuthology.orchestra.run.smithi113:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'Replacing daemon mds' | head -n 1 2024-03-04T08:48:48.302 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T08:48:48.302 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-04T08:48:48.689 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T08:48:48.689 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-04T08:48:48.883 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T08:48:48.883 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-04T08:48:49.103 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T08:48:49.104 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-04T08:48:49.312 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T08:48:49.312 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-04T08:48:49.376 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T08:48:49.376 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-04T08:48:49.544 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T08:48:49.545 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-04T08:48:49.723 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T08:48:49.723 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-04T08:48:49.909 INFO:tasks.ceph:Archiving mon data... 2024-03-04T08:48:49.910 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671/data/mon.a.tgz 2024-03-04T08:48:49.911 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-03-04T08:48:49.920 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.1RWzhdWgGC 2024-03-04T08:48:49.920 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.1RWzhdWgGC 2024-03-04T08:48:50.055 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.1RWzhdWgGC 2024-03-04T08:48:50.134 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.1RWzhdWgGC is 367KB 2024-03-04T08:48:50.200 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.1RWzhdWgGC 2024-03-04T08:48:50.208 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671/data/mon.b.tgz 2024-03-04T08:48:50.208 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-03-04T08:48:50.214 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.2Q4c6QE4ii 2024-03-04T08:48:50.215 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.2Q4c6QE4ii 2024-03-04T08:48:50.345 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.2Q4c6QE4ii 2024-03-04T08:48:50.411 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.2Q4c6QE4ii is 373KB 2024-03-04T08:48:50.472 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.2Q4c6QE4ii 2024-03-04T08:48:50.477 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671/data/mon.c.tgz 2024-03-04T08:48:50.477 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-03-04T08:48:50.522 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.xBvZkdioNU 2024-03-04T08:48:50.523 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.xBvZkdioNU 2024-03-04T08:48:50.656 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.xBvZkdioNU 2024-03-04T08:48:50.667 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:48:50.671 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:48:50.798 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.xBvZkdioNU is 373KB 2024-03-04T08:48:50.861 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.xBvZkdioNU 2024-03-04T08:48:50.867 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-04T08:48:50.867 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-04T08:48:50.871 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-04T08:48:50.937 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-04T08:48:50.970 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-04T08:48:50.971 INFO:tasks.ceph:Archiving crash dumps... 2024-03-04T08:48:50.972 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671/remote/smithi113/crash 2024-03-04T08:48:50.973 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-04T08:48:51.001 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671/remote/smithi136/crash 2024-03-04T08:48:51.002 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-04T08:48:51.026 INFO:tasks.ceph:Shutting down logrotate 2024-03-04T08:48:51.027 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:48:51.052 DEBUG:teuthology.orchestra.run.smithi136:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-04T08:48:51.109 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-04T08:48:51.130 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-04T08:48:51.166 INFO:tasks.ceph:Compressing logs... 2024-03-04T08:48:51.166 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T08:48:51.180 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T08:48:51.197 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51153.log 2024-03-04T08:48:51.198 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27542.log 2024-03-04T08:48:51.199 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54021.log 2024-03-04T08:48:51.199 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51153.log.gz 2024-03-04T08:48:51.200 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.27542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46169.log 2024-03-04T08:48:51.200 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27542.log.gz 2024-03-04T08:48:51.201 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47104.log 2024-03-04T08:48:51.201 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54021.log.gz 2024-03-04T08:48:51.202 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55755.log 2024-03-04T08:48:51.203 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46169.log.gz 2024-03-04T08:48:51.203 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32037.log 2024-03-04T08:48:51.203 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47104.log.gz 2024-03-04T08:48:51.204 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53602.log 2024-03-04T08:48:51.204 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55755.log.gz 2024-03-04T08:48:51.205 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40513.log 2024-03-04T08:48:51.206 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32037.log.gz 2024-03-04T08:48:51.206 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41285.log 2024-03-04T08:48:51.206 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53602.log.gz 2024-03-04T08:48:51.206 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55465.log 2024-03-04T08:48:51.207 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40513.log.gz 2024-03-04T08:48:51.208 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56648.log 2024-03-04T08:48:51.208 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41285.log.gz 2024-03-04T08:48:51.209 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53470.log 2024-03-04T08:48:51.209 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55465.log.gz 2024-03-04T08:48:51.209 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31745.log 2024-03-04T08:48:51.209 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56648.log.gz 2024-03-04T08:48:51.211 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54180.log 2024-03-04T08:48:51.211 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53470.log.gz 2024-03-04T08:48:51.212 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52429.log 2024-03-04T08:48:51.212 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31745.log.gz 2024-03-04T08:48:51.212 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39311.log 2024-03-04T08:48:51.212 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54180.log.gz 2024-03-04T08:48:51.212 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46947.log 2024-03-04T08:48:51.212 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52429.log.gz 2024-03-04T08:48:51.217 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46862.log 2024-03-04T08:48:51.217 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39311.log.gz 2024-03-04T08:48:51.217 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42491.log 2024-03-04T08:48:51.217 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46947.log.gz 2024-03-04T08:48:51.218 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41363.log 2024-03-04T08:48:51.218 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46862.log.gz 2024-03-04T08:48:51.218 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36661.log 2024-03-04T08:48:51.218 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42491.log.gz 2024-03-04T08:48:51.218 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32671.log 2024-03-04T08:48:51.218 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41363.log.gz 2024-03-04T08:48:51.218 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36174.log 2024-03-04T08:48:51.218 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36661.log.gz 2024-03-04T08:48:51.219 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57643.log 2024-03-04T08:48:51.219 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32671.log.gz 2024-03-04T08:48:51.221 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45507.log 2024-03-04T08:48:51.222 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36174.log.gz 2024-03-04T08:48:51.222 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38067.log 2024-03-04T08:48:51.222 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57643.log.gz 2024-03-04T08:48:51.222 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44198.log 2024-03-04T08:48:51.222 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45507.log.gz 2024-03-04T08:48:51.222 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45192.log 2024-03-04T08:48:51.222 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38067.log.gz 2024-03-04T08:48:51.223 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31959.log 2024-03-04T08:48:51.223 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44198.log.gz 2024-03-04T08:48:51.223 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42032.log 2024-03-04T08:48:51.223 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45192.log.gz 2024-03-04T08:48:51.226 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35680.log 2024-03-04T08:48:51.226 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31959.log.gz 2024-03-04T08:48:51.226 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48186.log 2024-03-04T08:48:51.226 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42032.log.gz 2024-03-04T08:48:51.226 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57118.log 2024-03-04T08:48:51.227 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35680.log.gz 2024-03-04T08:48:51.227 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2024-03-04T08:48:51.227 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48186.log.gz 2024-03-04T08:48:51.227 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56229.log 2024-03-04T08:48:51.227 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57118.log.gz 2024-03-04T08:48:51.229 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33800.log 2024-03-04T08:48:51.230 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2024-03-04T08:48:51.230 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49811.log 2024-03-04T08:48:51.230 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56229.log.gz 2024-03-04T08:48:51.230 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52826.log 2024-03-04T08:48:51.230 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33800.log.gz 2024-03-04T08:48:51.230 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57277.log 2024-03-04T08:48:51.231 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49811.log.gz 2024-03-04T08:48:51.231 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43199.log 2024-03-04T08:48:51.231 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52826.log.gz 2024-03-04T08:48:51.233 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53628.log 2024-03-04T08:48:51.233 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57277.log.gz 2024-03-04T08:48:51.233 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51450.log 2024-03-04T08:48:51.233 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43199.log.gz 2024-03-04T08:48:51.233 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53628.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-04T08:48:51.234 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53628.log.gz 2024-03-04T08:48:51.234 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40986.log 2024-03-04T08:48:51.234 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51450.log.gz 2024-03-04T08:48:51.243 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28160.log 2024-03-04T08:48:51.243 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28088.log 2024-03-04T08:48:51.243 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.28160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28112.log 2024-03-04T08:48:51.243 INFO:teuthology.orchestra.run.smithi136.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28160.log.gz 2024-03-04T08:48:51.243 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.28088.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-04T08:48:51.244 INFO:teuthology.orchestra.run.smithi136.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28088.log.gz 2024-03-04T08:48:51.244 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.28112.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-04T08:48:51.244 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.7.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28112.log.gz -5 2024-03-04T08:48:51.244 INFO:teuthology.orchestra.run.smithi136.stderr: --verbose -- /var/log/ceph/ceph-client.admin.28136.log 2024-03-04T08:48:51.244 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-04T08:48:51.244 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.28136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27992.log 2024-03-04T08:48:51.244 INFO:teuthology.orchestra.run.smithi136.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28136.log.gz 2024-03-04T08:48:51.245 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-04T08:48:51.245 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-03-04T08:48:51.245 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.27992.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-04T08:48:51.245 INFO:teuthology.orchestra.run.smithi136.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27992.log.gz 2024-03-04T08:48:51.245 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-04T08:48:51.245 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28040.log 2024-03-04T08:48:51.245 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-04T08:48:51.245 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.28040.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.28040.log.gz 2024-03-04T08:48:51.246 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28064.log 2024-03-04T08:48:51.251 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph.audit.log: 94.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-04T08:48:51.252 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-04T08:48:51.252 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-client.admin.28064.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.28064.log.gz 2024-03-04T08:48:51.252 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-04T08:48:51.252 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mds.d.log: 93.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-04T08:48:51.254 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-04T08:48:51.258 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43305.log 2024-03-04T08:48:51.258 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54968.log 2024-03-04T08:48:51.258 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40986.log.gz 2024-03-04T08:48:51.259 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51789.log 2024-03-04T08:48:51.259 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43305.log.gz 2024-03-04T08:48:51.259 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54076.log 2024-03-04T08:48:51.259 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54968.log.gz 2024-03-04T08:48:51.259 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48215.log 2024-03-04T08:48:51.259 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51789.log.gz 2024-03-04T08:48:51.259 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58178.log 2024-03-04T08:48:51.260 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54076.log.gz 2024-03-04T08:48:51.260 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54679.log 2024-03-04T08:48:51.260 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48215.log.gz 2024-03-04T08:48:51.260 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47236.log 2024-03-04T08:48:51.260 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41390.log 2024-03-04T08:48:51.260 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54679.log.gz 2024-03-04T08:48:51.260 INFO:teuthology.orchestra.run.smithi113.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.58178.log.gz 2024-03-04T08:48:51.260 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45917.log 2024-03-04T08:48:51.261 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47236.log.gz 2024-03-04T08:48:51.261 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46383.log 2024-03-04T08:48:51.261 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41390.log.gz 2024-03-04T08:48:51.261 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42060.log 2024-03-04T08:48:51.261 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45917.log.gz 2024-03-04T08:48:51.261 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38599.log 2024-03-04T08:48:51.261 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46383.log.gz 2024-03-04T08:48:51.262 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35943.log 2024-03-04T08:48:51.262 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42060.log.gz 2024-03-04T08:48:51.262 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47025.log 2024-03-04T08:48:51.262 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38599.log.gz 2024-03-04T08:48:51.262 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38726.log 2024-03-04T08:48:51.262 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35943.log.gz 2024-03-04T08:48:51.262 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38680.log 2024-03-04T08:48:51.262 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47025.log.gz 2024-03-04T08:48:51.263 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50728.log 2024-03-04T08:48:51.263 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38726.log.gz 2024-03-04T08:48:51.263 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35637.log 2024-03-04T08:48:51.263 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38680.log.gz 2024-03-04T08:48:51.263 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50728.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50728.log.gz 2024-03-04T08:48:51.263 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41850.log 2024-03-04T08:48:51.264 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57511.log 2024-03-04T08:48:51.264 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35637.log.gz 2024-03-04T08:48:51.264 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56910.log 2024-03-04T08:48:51.264 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41850.log.gz 2024-03-04T08:48:51.264 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49897.log 2024-03-04T08:48:51.264 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57511.log.gz 2024-03-04T08:48:51.264 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56910.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56910.log.gz 2024-03-04T08:48:51.264 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41442.log 2024-03-04T08:48:51.265 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44773.log 2024-03-04T08:48:51.265 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49897.log.gz 2024-03-04T08:48:51.265 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48353.log 2024-03-04T08:48:51.265 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41442.log.gz 2024-03-04T08:48:51.265 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32459.log 2024-03-04T08:48:51.265 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44773.log.gz 2024-03-04T08:48:51.265 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38519.log 2024-03-04T08:48:51.265 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48353.log.gz 2024-03-04T08:48:51.266 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46590.log 2024-03-04T08:48:51.266 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32459.log.gz 2024-03-04T08:48:51.272 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-04T08:48:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48107.log 2024-03-04T08:48:51.276 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38519.log.gz 2024-03-04T08:48:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54390.log 2024-03-04T08:48:51.276 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46590.log.gz 2024-03-04T08:48:51.276 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56437.log 2024-03-04T08:48:51.276 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48107.log.gz 2024-03-04T08:48:51.277 INFO:teuthology.orchestra.run.smithi113.stderr: 95.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-04T08:48:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53706.log 2024-03-04T08:48:51.277 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54390.log.gz 2024-03-04T08:48:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33198.log 2024-03-04T08:48:51.277 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56437.log.gz 2024-03-04T08:48:51.277 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42465.log 2024-03-04T08:48:51.277 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53706.log.gz 2024-03-04T08:48:51.278 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57014.log 2024-03-04T08:48:51.278 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33198.log.gz 2024-03-04T08:48:51.278 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32659.log 2024-03-04T08:48:51.278 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42465.log.gz 2024-03-04T08:48:51.278 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56281.log 2024-03-04T08:48:51.278 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57014.log.gz 2024-03-04T08:48:51.278 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53297.log 2024-03-04T08:48:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32659.log: /var/log/ceph/ceph-client.admin.56281.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32659.log.gz 2024-03-04T08:48:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37586.log 2024-03-04T08:48:51.279 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56281.log.gz 2024-03-04T08:48:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34480.log 2024-03-04T08:48:51.279 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53297.log.gz 2024-03-04T08:48:51.279 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43147.log 2024-03-04T08:48:51.279 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37586.log.gz 2024-03-04T08:48:51.280 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56752.log 2024-03-04T08:48:51.280 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34480.log.gz 2024-03-04T08:48:51.280 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34206.log 2024-03-04T08:48:51.280 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43147.log.gz 2024-03-04T08:48:51.280 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47353.log 2024-03-04T08:48:51.280 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56752.log.gz 2024-03-04T08:48:51.281 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34206.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-04T08:48:51.281 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34206.log.gz 2024-03-04T08:48:51.281 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32447.log 2024-03-04T08:48:51.281 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47353.log.gz 2024-03-04T08:48:51.281 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50611.log 2024-03-04T08:48:51.281 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46890.log 2024-03-04T08:48:51.281 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32447.log.gz 2024-03-04T08:48:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51179.log 2024-03-04T08:48:51.282 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50611.log.gz 2024-03-04T08:48:51.282 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34008.log 2024-03-04T08:48:51.282 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46890.log.gz 2024-03-04T08:48:51.283 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28016.log 2024-03-04T08:48:51.290 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47788.log 2024-03-04T08:48:51.291 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51179.log.gz 2024-03-04T08:48:51.291 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48933.log 2024-03-04T08:48:51.291 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34008.log.gz 2024-03-04T08:48:51.291 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55387.log 2024-03-04T08:48:51.291 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47788.log.gz 2024-03-04T08:48:51.291 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48459.log 2024-03-04T08:48:51.291 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48933.log.gz 2024-03-04T08:48:51.291 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41928.log 2024-03-04T08:48:51.292 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55387.log.gz 2024-03-04T08:48:51.292 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50069.log 2024-03-04T08:48:51.292 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48459.log.gz 2024-03-04T08:48:51.292 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38176.log 2024-03-04T08:48:51.292 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41928.log.gz 2024-03-04T08:48:51.292 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35860.log 2024-03-04T08:48:51.292 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50069.log.gz 2024-03-04T08:48:51.292 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40131.log 2024-03-04T08:48:51.293 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38176.log.gz 2024-03-04T08:48:51.293 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58228.log 2024-03-04T08:48:51.293 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35860.log.gz 2024-03-04T08:48:51.293 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36581.log 2024-03-04T08:48:51.293 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40131.log.gz 2024-03-04T08:48:51.293 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43121.log 2024-03-04T08:48:51.293 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35915.log 2024-03-04T08:48:51.294 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36581.log.gz 2024-03-04T08:48:51.294 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37937.log 2024-03-04T08:48:51.294 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43121.log.gz 2024-03-04T08:48:51.294 INFO:teuthology.orchestra.run.smithi113.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.58228.log.gz 2024-03-04T08:48:51.294 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27433.log 2024-03-04T08:48:51.294 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35915.log.gz 2024-03-04T08:48:51.294 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39393.log 2024-03-04T08:48:51.295 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37937.log.gz 2024-03-04T08:48:51.295 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.27433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42140.log 2024-03-04T08:48:51.295 INFO:teuthology.orchestra.run.smithi113.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.27433.log.gz 2024-03-04T08:48:51.295 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42654.log 2024-03-04T08:48:51.295 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39393.log.gz 2024-03-04T08:48:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.28016.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28016.log.gz 2024-03-04T08:48:51.302 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55020.log 2024-03-04T08:48:51.302 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42140.log.gz 2024-03-04T08:48:51.302 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54050.log 2024-03-04T08:48:51.302 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42654.log.gz 2024-03-04T08:48:51.302 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45324.log 2024-03-04T08:48:51.302 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55020.log.gz 2024-03-04T08:48:51.303 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56700.log 2024-03-04T08:48:51.303 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54050.log.gz 2024-03-04T08:48:51.303 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54206.log 2024-03-04T08:48:51.303 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45324.log.gz 2024-03-04T08:48:51.303 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39285.log 2024-03-04T08:48:51.303 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56700.log.gz 2024-03-04T08:48:51.303 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44968.log 2024-03-04T08:48:51.303 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54206.log.gz 2024-03-04T08:48:51.304 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41654.log 2024-03-04T08:48:51.304 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39285.log.gz 2024-03-04T08:48:51.304 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42114.log 2024-03-04T08:48:51.304 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44968.log.gz 2024-03-04T08:48:51.304 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58046.log 2024-03-04T08:48:51.304 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41654.log.gz 2024-03-04T08:48:51.304 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40105.log 2024-03-04T08:48:51.304 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42114.log.gz 2024-03-04T08:48:51.305 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54286.log 2024-03-04T08:48:51.305 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58046.log.gz 2024-03-04T08:48:51.305 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41496.log 2024-03-04T08:48:51.305 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40105.log.gz 2024-03-04T08:48:51.308 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36555.log 2024-03-04T08:48:51.308 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54286.log.gz 2024-03-04T08:48:51.309 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37828.log 2024-03-04T08:48:51.309 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41496.log.gz 2024-03-04T08:48:51.309 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56884.log 2024-03-04T08:48:51.309 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36555.log.gz 2024-03-04T08:48:51.309 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50009.log 2024-03-04T08:48:51.309 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37828.log.gz 2024-03-04T08:48:51.309 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56884.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56884.log.gz 2024-03-04T08:48:51.310 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47698.log 2024-03-04T08:48:51.310 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32990.log 2024-03-04T08:48:51.310 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50009.log.gz 2024-03-04T08:48:51.310 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53245.log 2024-03-04T08:48:51.310 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47698.log.gz 2024-03-04T08:48:51.310 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57994.log 2024-03-04T08:48:51.310 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32990.log.gz 2024-03-04T08:48:51.313 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32248.log 2024-03-04T08:48:51.313 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53245.log.gz 2024-03-04T08:48:51.313 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55992.log 2024-03-04T08:48:51.313 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57994.log.gz 2024-03-04T08:48:51.313 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46074.log 2024-03-04T08:48:51.313 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32248.log.gz 2024-03-04T08:48:51.313 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48959.log 2024-03-04T08:48:51.313 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55992.log.gz 2024-03-04T08:48:51.314 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50528.log 2024-03-04T08:48:51.314 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46074.log.gz 2024-03-04T08:48:51.316 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40644.log 2024-03-04T08:48:51.316 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48959.log.gz 2024-03-04T08:48:51.316 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50528.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50528.log.gz 2024-03-04T08:48:51.316 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33616.log 2024-03-04T08:48:51.316 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43066.log 2024-03-04T08:48:51.316 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40644.log.gz 2024-03-04T08:48:51.316 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53191.log 2024-03-04T08:48:51.317 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33616.log.gz 2024-03-04T08:48:51.318 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38466.log 2024-03-04T08:48:51.318 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43066.log.gz 2024-03-04T08:48:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57485.log 2024-03-04T08:48:51.319 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53191.log.gz 2024-03-04T08:48:51.319 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40022.log 2024-03-04T08:48:51.319 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38466.log.gz 2024-03-04T08:48:51.320 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40935.log 2024-03-04T08:48:51.320 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57485.log.gz 2024-03-04T08:48:51.321 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34612.log 2024-03-04T08:48:51.321 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40022.log.gz 2024-03-04T08:48:51.322 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49667.log 2024-03-04T08:48:51.322 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40935.log.gz 2024-03-04T08:48:51.322 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32912.log 2024-03-04T08:48:51.322 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34612.log.gz 2024-03-04T08:48:51.323 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52140.log 2024-03-04T08:48:51.323 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49667.log.gz 2024-03-04T08:48:51.323 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38336.log 2024-03-04T08:48:51.324 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32912.log.gz 2024-03-04T08:48:51.324 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55072.log 2024-03-04T08:48:51.325 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52140.log.gz 2024-03-04T08:48:51.325 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48159.log 2024-03-04T08:48:51.325 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38336.log.gz 2024-03-04T08:48:51.326 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55072.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55072.log.gz 2024-03-04T08:48:51.326 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51372.log 2024-03-04T08:48:51.327 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33333.log 2024-03-04T08:48:51.328 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48159.log.gz 2024-03-04T08:48:51.328 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47270.log 2024-03-04T08:48:51.328 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51372.log.gz 2024-03-04T08:48:51.329 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42323.log 2024-03-04T08:48:51.329 INFO:teuthology.orchestra.run.smithi113.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.33333.log.gz 2024-03-04T08:48:51.329 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46836.log 2024-03-04T08:48:51.331 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47270.log.gz 2024-03-04T08:48:51.331 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44066.log 2024-03-04T08:48:51.331 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42323.log.gz 2024-03-04T08:48:51.331 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42601.log 2024-03-04T08:48:51.332 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46836.log.gz 2024-03-04T08:48:51.333 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46732.log 2024-03-04T08:48:51.333 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44066.log.gz 2024-03-04T08:48:51.333 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44890.log 2024-03-04T08:48:51.333 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42601.log.gz 2024-03-04T08:48:51.334 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51101.log 2024-03-04T08:48:51.334 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46732.log.gz 2024-03-04T08:48:51.334 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52930.log 2024-03-04T08:48:51.335 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44890.log.gz 2024-03-04T08:48:51.335 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52325.log 2024-03-04T08:48:51.335 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51101.log.gz 2024-03-04T08:48:51.336 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44833.log 2024-03-04T08:48:51.337 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52930.log.gz 2024-03-04T08:48:51.337 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55649.log 2024-03-04T08:48:51.338 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52325.log.gz 2024-03-04T08:48:51.338 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57303.log 2024-03-04T08:48:51.338 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44833.log.gz 2024-03-04T08:48:51.339 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38284.log 2024-03-04T08:48:51.339 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55649.log.gz 2024-03-04T08:48:51.339 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41204.log 2024-03-04T08:48:51.341 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57303.log.gz 2024-03-04T08:48:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39591.log 2024-03-04T08:48:51.341 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38284.log.gz 2024-03-04T08:48:51.341 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51476.log 2024-03-04T08:48:51.342 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41204.log.gz 2024-03-04T08:48:51.342 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53969.log 2024-03-04T08:48:51.342 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39591.log.gz 2024-03-04T08:48:51.344 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56148.log 2024-03-04T08:48:51.344 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51476.log.gz 2024-03-04T08:48:51.344 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38440.log 2024-03-04T08:48:51.344 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53969.log.gz 2024-03-04T08:48:51.346 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40750.log 2024-03-04T08:48:51.346 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56148.log.gz 2024-03-04T08:48:51.346 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34642.log 2024-03-04T08:48:51.346 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38440.log.gz 2024-03-04T08:48:51.346 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53862.log 2024-03-04T08:48:51.347 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40750.log.gz 2024-03-04T08:48:51.347 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45073.log 2024-03-04T08:48:51.347 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34642.log.gz 2024-03-04T08:48:51.348 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44146.log 2024-03-04T08:48:51.349 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53862.log.gz 2024-03-04T08:48:51.349 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41064.log 2024-03-04T08:48:51.349 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45073.log.gz 2024-03-04T08:48:51.350 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46328.log 2024-03-04T08:48:51.350 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44146.log.gz 2024-03-04T08:48:51.350 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51711.log 2024-03-04T08:48:51.350 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41064.log.gz 2024-03-04T08:48:51.352 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51947.log 2024-03-04T08:48:51.352 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46328.log.gz 2024-03-04T08:48:51.352 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48301.log 2024-03-04T08:48:51.352 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51711.log.gz 2024-03-04T08:48:51.353 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39337.log 2024-03-04T08:48:51.354 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51947.log.gz 2024-03-04T08:48:51.354 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56070.log 2024-03-04T08:48:51.354 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48301.log.gz 2024-03-04T08:48:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34034.log 2024-03-04T08:48:51.355 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39337.log.gz 2024-03-04T08:48:51.355 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56070.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-04T08:48:51.356 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56070.log.gz 2024-03-04T08:48:51.357 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40696.log 2024-03-04T08:48:51.357 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34034.log.gz 2024-03-04T08:48:51.357 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38414.log 2024-03-04T08:48:51.358 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42169.log 2024-03-04T08:48:51.358 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40696.log.gz 2024-03-04T08:48:51.359 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38414.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38414.log.gz 2024-03-04T08:48:51.359 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55703.log 2024-03-04T08:48:51.360 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50554.log 2024-03-04T08:48:51.360 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz 2024-03-04T08:48:51.360 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34532.log 2024-03-04T08:48:51.361 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55703.log.gz 2024-03-04T08:48:51.361 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39479.log 89.1% 2024-03-04T08:48:51.361 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50554.log.gz 2024-03-04T08:48:51.362 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53331.log 2024-03-04T08:48:51.363 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34532.log: /var/log/ceph/ceph-client.admin.39479.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34532.log.gz 2024-03-04T08:48:51.364 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57671.log 2024-03-04T08:48:51.364 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39479.log.gz 2024-03-04T08:48:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.53331.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43985.log 2024-03-04T08:48:51.371 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53331.log.gz 2024-03-04T08:48:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53086.log 2024-03-04T08:48:51.371 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57671.log.gz 2024-03-04T08:48:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40291.log 2024-03-04T08:48:51.371 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43985.log.gz 2024-03-04T08:48:51.371 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32851.log 2024-03-04T08:48:51.372 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53086.log.gz 2024-03-04T08:48:51.372 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47484.log 2024-03-04T08:48:51.372 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40291.log.gz 2024-03-04T08:48:51.372 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33254.log 2024-03-04T08:48:51.372 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32851.log.gz 2024-03-04T08:48:51.372 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35997.log 2024-03-04T08:48:51.372 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47484.log.gz 2024-03-04T08:48:51.372 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40776.log 2024-03-04T08:48:51.374 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33254.log.gz 2024-03-04T08:48:51.375 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37802.log 2024-03-04T08:48:51.375 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35997.log.gz 2024-03-04T08:48:51.375 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42627.log 2024-03-04T08:48:51.375 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40776.log.gz 2024-03-04T08:48:51.375 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52958.log 2024-03-04T08:48:51.375 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37802.log.gz 2024-03-04T08:48:51.375 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50294.log 2024-03-04T08:48:51.377 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42627.log.gz 2024-03-04T08:48:51.377 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49204.log 2024-03-04T08:48:51.377 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52958.log.gz 2024-03-04T08:48:51.377 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47210.log 2024-03-04T08:48:51.377 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50294.log.gz 2024-03-04T08:48:51.378 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34111.log 2024-03-04T08:48:51.379 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49204.log.gz 2024-03-04T08:48:51.379 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53524.log 2024-03-04T08:48:51.379 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47210.log.gz 2024-03-04T08:48:51.380 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34111.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34111.log.gz 2024-03-04T08:48:51.380 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35559.log 2024-03-04T08:48:51.380 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49748.log 2024-03-04T08:48:51.381 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53524.log.gz 2024-03-04T08:48:51.381 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35559.log: gzip -5 --verbose -- 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35559.log.gz 2024-03-04T08:48:51.381 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.49151.log 2024-03-04T08:48:51.382 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49117.log 2024-03-04T08:48:51.383 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49748.log.gz 2024-03-04T08:48:51.383 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38202.log 2024-03-04T08:48:51.383 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49151.log.gz 2024-03-04T08:48:51.384 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34261.log 2024-03-04T08:48:51.384 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49117.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49117.log.gz 2024-03-04T08:48:51.385 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36252.log 2024-03-04T08:48:51.385 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38202.log.gz 2024-03-04T08:48:51.386 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48652.log 2024-03-04T08:48:51.387 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34261.log.gz 2024-03-04T08:48:51.387 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44617.log 2024-03-04T08:48:51.388 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36252.log.gz 2024-03-04T08:48:51.388 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34425.log 2024-03-04T08:48:51.388 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48652.log.gz 2024-03-04T08:48:51.389 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40381.log 2024-03-04T08:48:51.389 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44617.log.gz 2024-03-04T08:48:51.390 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40854.log 2024-03-04T08:48:51.390 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34425.log.gz 2024-03-04T08:48:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53943.log 2024-03-04T08:48:51.391 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40381.log.gz 2024-03-04T08:48:51.391 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43598.log 2024-03-04T08:48:51.392 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40854.log.gz 2024-03-04T08:48:51.393 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45429.log 2024-03-04T08:48:51.393 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53943.log.gz 2024-03-04T08:48:51.393 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44092.log 2024-03-04T08:48:51.393 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43598.log.gz 2024-03-04T08:48:51.394 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50039.log 2024-03-04T08:48:51.394 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45429.log.gz 2024-03-04T08:48:51.395 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56936.log 2024-03-04T08:48:51.395 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44092.log.gz 2024-03-04T08:48:51.396 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37716.log 2024-03-04T08:48:51.396 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50039.log.gz 2024-03-04T08:48:51.397 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37768.log 2024-03-04T08:48:51.397 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56936.log.gz 2024-03-04T08:48:51.398 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37716.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-04T08:48:51.398 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37716.log.gz 2024-03-04T08:48:51.399 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54862.log 2024-03-04T08:48:51.399 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37768.log.gz 2024-03-04T08:48:51.400 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53550.log 2024-03-04T08:48:51.401 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42006.log 2024-03-04T08:48:51.401 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54862.log.gz 2024-03-04T08:48:51.402 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32660.log 2024-03-04T08:48:51.402 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53550.log.gz 2024-03-04T08:48:51.403 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42006.log: -- /var/log/ceph/ceph-client.admin.46135.log 2024-03-04T08:48:51.403 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32660.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42006.log.gz 2024-03-04T08:48:51.405 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48545.log 2024-03-04T08:48:51.405 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32660.log.gz 2024-03-04T08:48:51.405 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37854.log 2024-03-04T08:48:51.405 INFO:teuthology.orchestra.run.smithi113.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46135.log.gz 2024-03-04T08:48:51.406 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46249.log 2024-03-04T08:48:51.407 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37854.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48545.log.gz 2024-03-04T08:48:51.407 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43879.log 2024-03-04T08:48:51.407 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37854.log.gz 2024-03-04T08:48:51.408 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31797.log 2024-03-04T08:48:51.408 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46249.log.gz 2024-03-04T08:48:51.409 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37690.log 2024-03-04T08:48:51.409 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43879.log.gz 2024-03-04T08:48:51.410 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54260.log 2024-03-04T08:48:51.410 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31797.log.gz 2024-03-04T08:48:51.411 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44460.log 2024-03-04T08:48:51.411 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37690.log.gz 2024-03-04T08:48:51.412 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49338.log 2024-03-04T08:48:51.413 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44460.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54260.log.gz 2024-03-04T08:48:51.413 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44460.log.gz 2024-03-04T08:48:51.413 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42980.log 2024-03-04T08:48:51.414 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48261.log 2024-03-04T08:48:51.414 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49338.log.gz 2024-03-04T08:48:51.415 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31719.log 2024-03-04T08:48:51.415 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42980.log.gz 2024-03-04T08:48:51.416 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56200.log 2024-03-04T08:48:51.416 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48261.log.gz 2024-03-04T08:48:51.417 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50471.log 2024-03-04T08:48:51.418 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31719.log.gz 2024-03-04T08:48:51.418 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56596.log 2024-03-04T08:48:51.418 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56200.log.gz 2024-03-04T08:48:51.419 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33280.log 2024-03-04T08:48:51.420 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50471.log.gz 2024-03-04T08:48:51.420 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44807.log 2024-03-04T08:48:51.421 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56596.log.gz 2024-03-04T08:48:51.421 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32175.log 2024-03-04T08:48:51.422 INFO:teuthology.orchestra.run.smithi113.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.33280.log.gz 2024-03-04T08:48:51.422 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48327.log 2024-03-04T08:48:51.423 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44807.log.gz 2024-03-04T08:48:51.423 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53357.log 2024-03-04T08:48:51.424 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32175.log.gz 2024-03-04T08:48:51.425 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41744.log 2024-03-04T08:48:51.425 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48327.log.gz 2024-03-04T08:48:51.426 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57433.log 2024-03-04T08:48:51.426 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53357.log.gz 2024-03-04T08:48:51.426 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48756.log 2024-03-04T08:48:51.427 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41744.log.gz 2024-03-04T08:48:51.428 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54575.log 2024-03-04T08:48:51.428 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57433.log.gz 2024-03-04T08:48:51.429 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45770.log 2024-03-04T08:48:51.429 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48756.log.gz 2024-03-04T08:48:51.430 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51685.log 2024-03-04T08:48:51.430 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54575.log.gz 2024-03-04T08:48:51.431 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32457.log 2024-03-04T08:48:51.431 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45770.log.gz 2024-03-04T08:48:51.432 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53810.log 2024-03-04T08:48:51.432 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51685.log.gz 2024-03-04T08:48:51.433 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55202.log 2024-03-04T08:48:51.433 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32457.log.gz 2024-03-04T08:48:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51633.log 2024-03-04T08:48:51.434 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53810.log.gz 2024-03-04T08:48:51.434 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.55202.log: /var/log/ceph/ceph-client.admin.53917.log 2024-03-04T08:48:51.435 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51633.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55202.log.gz 2024-03-04T08:48:51.436 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51633.log.gz 2024-03-04T08:48:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53836.log 2024-03-04T08:48:51.436 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45744.log 2024-03-04T08:48:51.437 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53917.log: 88.5%/var/log/ceph/ceph-client.admin.53836.log: -- replaced with /var/log/ceph/ceph-client.admin.53917.log.gz 2024-03-04T08:48:51.438 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58154.log 2024-03-04T08:48:51.438 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53836.log.gz 2024-03-04T08:48:51.438 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52244.log 2024-03-04T08:48:51.439 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45744.log.gz 2024-03-04T08:48:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53576.log 2024-03-04T08:48:51.440 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55966.log 2024-03-04T08:48:51.441 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52244.log.gz 2024-03-04T08:48:51.441 INFO:teuthology.orchestra.run.smithi113.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.58154.log.gz 2024-03-04T08:48:51.442 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33120.log 2024-03-04T08:48:51.442 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53576.log.gz 2024-03-04T08:48:51.443 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51921.log 2024-03-04T08:48:51.443 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55966.log.gz 2024-03-04T08:48:51.444 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40261.log 2024-03-04T08:48:51.444 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33120.log.gz 2024-03-04T08:48:51.445 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40209.log 2024-03-04T08:48:51.445 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51921.log.gz 2024-03-04T08:48:51.446 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40261.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40261.log.gz 2024-03-04T08:48:51.446 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35507.log 2024-03-04T08:48:51.447 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44643.log 2024-03-04T08:48:51.447 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40209.log.gz 2024-03-04T08:48:51.448 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56543.log 2024-03-04T08:48:51.448 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35507.log.gz 2024-03-04T08:48:51.449 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41550.log 2024-03-04T08:48:51.449 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44643.log.gz 2024-03-04T08:48:51.450 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52559.log 2024-03-04T08:48:51.450 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56543.log.gz 2024-03-04T08:48:51.451 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49392.log 2024-03-04T08:48:51.451 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41550.log.gz 2024-03-04T08:48:51.452 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47815.log 2024-03-04T08:48:51.452 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52559.log.gz 2024-03-04T08:48:51.453 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.49392.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45021.log 2024-03-04T08:48:51.453 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49392.log.gz 2024-03-04T08:48:51.454 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32655.log 2024-03-04T08:48:51.454 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47815.log.gz 2024-03-04T08:48:51.455 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31462.log 2024-03-04T08:48:51.455 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45021.log.gz 2024-03-04T08:48:51.456 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38119.log 2024-03-04T08:48:51.456 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32655.log.gz 2024-03-04T08:48:51.457 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42517.log 2024-03-04T08:48:51.457 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.31462.log.gz 2024-03-04T08:48:51.458 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51424.log 2024-03-04T08:48:51.458 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38119.log.gz 2024-03-04T08:48:51.459 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37560.log 2024-03-04T08:48:51.460 INFO:teuthology.orchestra.run.smithi113.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.42517.log.gz 2024-03-04T08:48:51.460 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32149.log 2024-03-04T08:48:51.460 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51424.log.gz 2024-03-04T08:48:51.461 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41602.log 2024-03-04T08:48:51.461 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37560.log.gz 2024-03-04T08:48:51.462 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48838.log 2024-03-04T08:48:51.463 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32149.log.gz 2024-03-04T08:48:51.463 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57329.log 2024-03-04T08:48:51.464 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41602.log.gz 2024-03-04T08:48:51.464 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48838.log: /var/log/ceph/ceph-client.admin.49641.log 2024-03-04T08:48:51.465 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57329.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48838.log.gz 2024-03-04T08:48:51.465 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49011.log 2024-03-04T08:48:51.465 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57329.log.gz 2024-03-04T08:48:51.466 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46512.log 2024-03-04T08:48:51.467 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49641.log.gz 2024-03-04T08:48:51.467 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32463.log 2024-03-04T08:48:51.468 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6%/var/log/ceph/ceph-client.admin.46512.log: -- replaced with /var/log/ceph/ceph-client.admin.49011.log.gz 2024-03-04T08:48:51.468 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47182.log 2024-03-04T08:48:51.468 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46512.log.gz 2024-03-04T08:48:51.469 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47975.log 2024-03-04T08:48:51.470 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32463.log.gz 2024-03-04T08:48:51.470 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.47182.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56988.log 2024-03-04T08:48:51.471 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9%/var/log/ceph/ceph-client.admin.47975.log: -- replaced with /var/log/ceph/ceph-client.admin.47182.log.gz 2024-03-04T08:48:51.471 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45891.log 2024-03-04T08:48:51.471 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47975.log.gz 2024-03-04T08:48:51.472 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52507.log 2024-03-04T08:48:51.473 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56988.log.gz 2024-03-04T08:48:51.473 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50497.log 2024-03-04T08:48:51.474 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45891.log: /var/log/ceph/ceph-client.admin.52507.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45891.log.gz 2024-03-04T08:48:51.475 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33774.log 2024-03-04T08:48:51.475 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52507.log.gz 2024-03-04T08:48:51.476 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54809.log 2024-03-04T08:48:51.476 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50497.log.gz 2024-03-04T08:48:51.477 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.33774.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54548.log 2024-03-04T08:48:51.477 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54809.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33774.log.gz 2024-03-04T08:48:51.477 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34399.log 2024-03-04T08:48:51.478 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54809.log.gz 2024-03-04T08:48:51.478 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57697.log 2024-03-04T08:48:51.479 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54548.log.gz 2024-03-04T08:48:51.479 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34399.log: /var/log/ceph/ceph-client.admin.36306.log 2024-03-04T08:48:51.480 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57697.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34399.log.gz 2024-03-04T08:48:51.480 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46538.log 2024-03-04T08:48:51.481 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57697.log.gz 2024-03-04T08:48:51.481 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53271.log 2024-03-04T08:48:51.482 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36306.log.gz 2024-03-04T08:48:51.482 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50231.log 2024-03-04T08:48:51.483 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46538.log: /var/log/ceph/ceph-client.admin.53271.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46538.log.gz 2024-03-04T08:48:51.483 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41416.log 2024-03-04T08:48:51.484 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53271.log.gz 2024-03-04T08:48:51.484 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54520.log 2024-03-04T08:48:51.485 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50231.log.gz 2024-03-04T08:48:51.485 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.41416.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31933.log 2024-03-04T08:48:51.486 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54520.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41416.log.gz 2024-03-04T08:48:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48379.log 2024-03-04T08:48:51.487 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54520.log.gz 2024-03-04T08:48:51.487 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37883.log 2024-03-04T08:48:51.488 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31933.log.gz 2024-03-04T08:48:51.488 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42297.log 2024-03-04T08:48:51.489 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48379.log: /var/log/ceph/ceph-client.admin.37883.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48379.log.gz 2024-03-04T08:48:51.490 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46195.log 2024-03-04T08:48:51.490 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37883.log.gz 2024-03-04T08:48:51.491 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37989.log 2024-03-04T08:48:51.491 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42297.log.gz 2024-03-04T08:48:51.492 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40722.log 2024-03-04T08:48:51.492 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46195.log.gz 2024-03-04T08:48:51.493 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57591.log 2024-03-04T08:48:51.493 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37989.log.gz 2024-03-04T08:48:51.494 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40157.log 2024-03-04T08:48:51.494 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40722.log.gz 2024-03-04T08:48:51.494 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-03-04T08:48:51.495 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40157.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57591.log.gz 2024-03-04T08:48:51.495 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49065.log 2024-03-04T08:48:51.496 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40157.log.gz 2024-03-04T08:48:51.497 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45402.log 2024-03-04T08:48:51.497 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-03-04T08:48:51.497 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49065.log: /var/log/ceph/ceph-client.admin.55413.log 2024-03-04T08:48:51.498 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49065.log.gz 2024-03-04T08:48:51.499 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43703.log 2024-03-04T08:48:51.499 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45402.log.gz 2024-03-04T08:48:51.499 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32011.log 2024-03-04T08:48:51.500 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55413.log.gz 2024-03-04T08:48:51.501 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53165.log 2024-03-04T08:48:51.501 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43703.log.gz 2024-03-04T08:48:51.502 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42952.log 2024-03-04T08:48:51.502 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32011.log.gz 2024-03-04T08:48:51.502 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.53165.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41012.log 2024-03-04T08:48:51.503 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53165.log.gz 2024-03-04T08:48:51.504 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39970.log 2024-03-04T08:48:51.504 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42952.log.gz 2024-03-04T08:48:51.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41038.log 2024-03-04T08:48:51.505 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41012.log.gz 2024-03-04T08:48:51.506 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57251.log 2024-03-04T08:48:51.506 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39970.log.gz 2024-03-04T08:48:51.507 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43765.log 2024-03-04T08:48:51.507 INFO:teuthology.orchestra.run.smithi113.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.41038.log.gz 2024-03-04T08:48:51.508 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32444.log 2024-03-04T08:48:51.508 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57251.log.gz 2024-03-04T08:48:51.509 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42680.log 2024-03-04T08:48:51.509 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43765.log.gz 2024-03-04T08:48:51.510 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52192.log 2024-03-04T08:48:51.510 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32444.log.gz 2024-03-04T08:48:51.511 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41824.log 2024-03-04T08:48:51.511 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42680.log.gz 2024-03-04T08:48:51.512 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51607.log 2024-03-04T08:48:51.513 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52192.log: /var/log/ceph/ceph-client.admin.41824.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52192.log.gz 2024-03-04T08:48:51.513 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46109.log 2024-03-04T08:48:51.513 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41824.log.gz 2024-03-04T08:48:51.514 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56463.log 2024-03-04T08:48:51.515 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51607.log.gz 2024-03-04T08:48:51.515 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39775.log 2024-03-04T08:48:51.515 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46109.log.gz 2024-03-04T08:48:51.516 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40355.log 2024-03-04T08:48:51.517 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56463.log.gz 2024-03-04T08:48:51.517 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39775.log: gzip -5 --verbose -- 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39775.log.gz /var/log/ceph/ceph-client.admin.38625.log 2024-03-04T08:48:51.517 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-03-04T08:48:51.518 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51346.log 2024-03-04T08:48:51.519 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40355.log.gz 2024-03-04T08:48:51.519 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38625.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.38625.log.gz --verbose 2024-03-04T08:48:51.519 INFO:teuthology.orchestra.run.smithi113.stderr: -- /var/log/ceph/ceph-client.admin.36690.log 2024-03-04T08:48:51.520 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31693.log 2024-03-04T08:48:51.521 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51346.log: 87.8%/var/log/ceph/ceph-client.admin.36690.log: -- replaced with /var/log/ceph/ceph-client.admin.51346.log.gz 2024-03-04T08:48:51.521 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 89.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.36690.log.gz 2024-03-04T08:48:51.521 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.36278.log 2024-03-04T08:48:51.522 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41116.log 2024-03-04T08:48:51.523 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36278.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31693.log.gz 2024-03-04T08:48:51.523 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43518.log 2024-03-04T08:48:51.524 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36278.log.gz 2024-03-04T08:48:51.525 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43392.log 2024-03-04T08:48:51.525 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41116.log.gz 2024-03-04T08:48:51.525 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58252.log 2024-03-04T08:48:51.526 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43518.log.gz 2024-03-04T08:48:51.526 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32668.log/var/log/ceph/ceph-client.admin.43392.log: 2024-03-04T08:48:51.527 INFO:teuthology.orchestra.run.smithi113.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43392.log.gz 2024-03-04T08:48:51.527 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33172.log 2024-03-04T08:48:51.528 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51659.log 2024-03-04T08:48:51.529 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32668.log.gz 2024-03-04T08:48:51.529 INFO:teuthology.orchestra.run.smithi113.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.58252.log.gz 2024-03-04T08:48:51.529 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47614.log 2024-03-04T08:48:51.530 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33172.log.gz 2024-03-04T08:48:51.530 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38310.log 2024-03-04T08:48:51.531 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51659.log.gz 2024-03-04T08:48:51.532 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52774.log 2024-03-04T08:48:51.532 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47614.log.gz 2024-03-04T08:48:51.532 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45665.log 2024-03-04T08:48:51.533 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38310.log.gz 2024-03-04T08:48:51.533 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38146.log 2024-03-04T08:48:51.534 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52774.log.gz 2024-03-04T08:48:51.534 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47762.log 2024-03-04T08:48:51.535 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45665.log.gz 2024-03-04T08:48:51.535 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38571.log 88.3% 2024-03-04T08:48:51.536 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38146.log.gz 2024-03-04T08:48:51.537 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47588.log 2024-03-04T08:48:51.537 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38571.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47762.log.gz 2024-03-04T08:48:51.538 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-04T08:48:51.538 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38571.log.gz 2024-03-04T08:48:51.539 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42900.log 2024-03-04T08:48:51.539 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47588.log.gz 2024-03-04T08:48:51.539 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55309.log 2024-03-04T08:48:51.541 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56359.log 2024-03-04T08:48:51.541 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42900.log.gz 2024-03-04T08:48:51.542 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49444.log 2024-03-04T08:48:51.542 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55309.log.gz 2024-03-04T08:48:51.543 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50325.log 2024-03-04T08:48:51.543 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56359.log.gz 2024-03-04T08:48:51.544 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47536.log 2024-03-04T08:48:51.544 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49444.log.gz 2024-03-04T08:48:51.545 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56018.log 2024-03-04T08:48:51.546 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50325.log.gz 2024-03-04T08:48:51.547 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53383.log 2024-03-04T08:48:51.547 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47536.log.gz 2024-03-04T08:48:51.548 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48985.log 2024-03-04T08:48:51.548 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56018.log.gz 2024-03-04T08:48:51.549 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52481.log 2024-03-04T08:48:51.549 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53383.log.gz 2024-03-04T08:48:51.550 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49953.log 2024-03-04T08:48:51.550 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48985.log.gz 2024-03-04T08:48:51.551 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42430.log 2024-03-04T08:48:51.551 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52481.log.gz 2024-03-04T08:48:51.552 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32666.log 2024-03-04T08:48:51.552 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49953.log.gz 2024-03-04T08:48:51.553 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47130.log 2024-03-04T08:48:51.553 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42430.log.gz 2024-03-04T08:48:51.554 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56122.log 2024-03-04T08:48:51.554 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32666.log.gz 2024-03-04T08:48:51.555 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51398.log 2024-03-04T08:48:51.555 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47130.log.gz 2024-03-04T08:48:51.556 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52613.log 2024-03-04T08:48:51.557 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56122.log.gz 2024-03-04T08:48:51.558 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55257.log 2024-03-04T08:48:51.558 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51398.log.gz 2024-03-04T08:48:51.559 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54128.log 2024-03-04T08:48:51.559 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52613.log.gz 2024-03-04T08:48:51.560 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52218.log 2024-03-04T08:48:51.560 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55257.log.gz 2024-03-04T08:48:51.561 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40329.log 2024-03-04T08:48:51.561 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54128.log.gz 2024-03-04T08:48:51.562 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51737.log 2024-03-04T08:48:51.562 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52218.log.gz 2024-03-04T08:48:51.563 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50585.log 2024-03-04T08:48:51.563 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40329.log.gz 2024-03-04T08:48:51.564 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33696.log 2024-03-04T08:48:51.564 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51737.log.gz 2024-03-04T08:48:51.565 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45944.log 2024-03-04T08:48:51.565 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50585.log.gz 2024-03-04T08:48:51.566 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43333.log 2024-03-04T08:48:51.566 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33696.log.gz 2024-03-04T08:48:51.567 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36476.log 2024-03-04T08:48:51.568 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45944.log.gz 2024-03-04T08:48:51.569 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46616.log 2024-03-04T08:48:51.569 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43333.log.gz 2024-03-04T08:48:51.570 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56515.log 2024-03-04T08:48:51.570 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36476.log.gz 2024-03-04T08:48:51.571 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51502.log 2024-03-04T08:48:51.571 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46616.log.gz 2024-03-04T08:48:51.572 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54653.log 2024-03-04T08:48:51.572 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56515.log.gz 2024-03-04T08:48:51.573 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50106.log 2024-03-04T08:48:51.573 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51502.log.gz 2024-03-04T08:48:51.574 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55887.log 2024-03-04T08:48:51.574 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54653.log.gz 2024-03-04T08:48:51.575 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37963.log 2024-03-04T08:48:51.575 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50106.log.gz 2024-03-04T08:48:51.576 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47156.log 2024-03-04T08:48:51.576 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55887.log.gz 2024-03-04T08:48:51.577 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50351.log 2024-03-04T08:48:51.578 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37963.log.gz 2024-03-04T08:48:51.579 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38493.log 2024-03-04T08:48:51.579 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47156.log.gz 2024-03-04T08:48:51.580 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33904.log 2024-03-04T08:48:51.580 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50351.log.gz 2024-03-04T08:48:51.581 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53444.log 2024-03-04T08:48:51.581 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38493.log.gz 2024-03-04T08:48:51.582 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53112.log 2024-03-04T08:48:51.582 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33904.log.gz 2024-03-04T08:48:51.583 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42404.log 2024-03-04T08:48:51.583 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53444.log.gz 2024-03-04T08:48:51.584 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55859.log 2024-03-04T08:48:51.584 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53112.log.gz 2024-03-04T08:48:51.585 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51581.log 2024-03-04T08:48:51.585 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42404.log.gz 2024-03-04T08:48:51.586 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33068.log 2024-03-04T08:48:51.586 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55859.log.gz 2024-03-04T08:48:51.587 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47668.log 2024-03-04T08:48:51.588 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51581.log.gz 2024-03-04T08:48:51.588 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53654.log 2024-03-04T08:48:51.588 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33068.log.gz 2024-03-04T08:48:51.589 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56804.log 2024-03-04T08:48:51.589 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47668.log.gz 2024-03-04T08:48:51.590 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57915.log 2024-03-04T08:48:51.591 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53654.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53654.log.gz 2024-03-04T08:48:51.591 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44747.log 2024-03-04T08:48:51.592 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56804.log.gz 2024-03-04T08:48:51.592 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55491.log 2024-03-04T08:48:51.593 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57915.log.gz 2024-03-04T08:48:51.593 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32456.log 2024-03-04T08:48:51.594 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44747.log.gz 2024-03-04T08:48:51.594 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56570.log 2024-03-04T08:48:51.595 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55491.log.gz 2024-03-04T08:48:51.596 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46784.log 2024-03-04T08:48:51.596 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32456.log.gz 2024-03-04T08:48:51.597 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51841.log 2024-03-04T08:48:51.597 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56570.log.gz 2024-03-04T08:48:51.598 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35807.log 2024-03-04T08:48:51.598 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46784.log.gz 2024-03-04T08:48:51.599 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57066.log 2024-03-04T08:48:51.599 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51841.log.gz 2024-03-04T08:48:51.600 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47923.log 2024-03-04T08:48:51.600 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35807.log.gz 2024-03-04T08:48:51.601 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39419.log 2024-03-04T08:48:51.602 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57066.log.gz 2024-03-04T08:48:51.602 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46048.log 2024-03-04T08:48:51.602 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47923.log.gz 2024-03-04T08:48:51.603 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33642.log 2024-03-04T08:48:51.603 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39419.log.gz 2024-03-04T08:48:51.604 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43225.log 2024-03-04T08:48:51.604 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46048.log.gz 2024-03-04T08:48:51.606 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35585.log 2024-03-04T08:48:51.606 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33642.log.gz 2024-03-04T08:48:51.607 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47562.log 2024-03-04T08:48:51.607 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43225.log.gz 2024-03-04T08:48:51.608 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33982.log 2024-03-04T08:48:51.608 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35585.log.gz 2024-03-04T08:48:51.609 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49418.log 2024-03-04T08:48:51.609 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47562.log.gz 2024-03-04T08:48:51.610 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.33982.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41954.log 2024-03-04T08:48:51.610 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33982.log.gz 2024-03-04T08:48:51.611 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55807.log 2024-03-04T08:48:51.611 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49418.log.gz 2024-03-04T08:48:51.612 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45845.log 2024-03-04T08:48:51.612 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41954.log.gz 2024-03-04T08:48:51.613 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52985.log 2024-03-04T08:48:51.613 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55807.log.gz 2024-03-04T08:48:51.614 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57749.log 2024-03-04T08:48:51.615 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45845.log.gz 2024-03-04T08:48:51.615 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34506.log 2024-03-04T08:48:51.615 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52985.log.gz 2024-03-04T08:48:51.616 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33748.log 2024-03-04T08:48:51.616 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57749.log.gz 2024-03-04T08:48:51.617 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34506.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-04T08:48:51.618 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34506.log.gz 2024-03-04T08:48:51.618 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52720.log 2024-03-04T08:48:51.619 INFO:teuthology.orchestra.run.smithi113.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33748.log.gz 2024-03-04T08:48:51.620 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55150.log 2024-03-04T08:48:51.621 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42352.log 2024-03-04T08:48:51.621 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52720.log.gz 2024-03-04T08:48:51.622 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55150.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55150.log.gz 2024-03-04T08:48:51.623 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42352.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42352.log.gz 2024-03-04T08:48:51.624 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45533.log 2024-03-04T08:48:51.625 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39858.log 2024-03-04T08:48:51.626 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56778.log 2024-03-04T08:48:51.626 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45533.log.gz 2024-03-04T08:48:51.627 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39858.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39858.log.gz 2024-03-04T08:48:51.628 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56778.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56778.log.gz 2024-03-04T08:48:51.629 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33590.log 2024-03-04T08:48:51.630 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36072.log 2024-03-04T08:48:51.632 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48600.log 2024-03-04T08:48:51.632 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33590.log.gz 2024-03-04T08:48:51.633 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36072.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36072.log.gz 2024-03-04T08:48:51.633 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48600.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48600.log.gz 2024-03-04T08:48:51.634 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44250.log 2024-03-04T08:48:51.635 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44224.log 2024-03-04T08:48:51.636 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56411.log 2024-03-04T08:48:51.637 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44250.log.gz 2024-03-04T08:48:51.638 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44224.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44224.log.gz 2024-03-04T08:48:51.638 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56411.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56411.log.gz 2024-03-04T08:48:51.639 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57889.log 2024-03-04T08:48:51.640 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46810.log 2024-03-04T08:48:51.641 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55623.log 2024-03-04T08:48:51.642 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57889.log.gz 2024-03-04T08:48:51.642 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46810.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46810.log.gz 2024-03-04T08:48:51.643 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55623.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55623.log.gz 2024-03-04T08:48:51.644 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48027.log 2024-03-04T08:48:51.645 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45996.log 2024-03-04T08:48:51.646 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57617.log 2024-03-04T08:48:51.647 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48027.log.gz 2024-03-04T08:48:51.647 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45996.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45996.log.gz 2024-03-04T08:48:51.648 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57617.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57617.log.gz 2024-03-04T08:48:51.649 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36393.log 2024-03-04T08:48:51.650 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37638.log 2024-03-04T08:48:51.652 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36608.log 2024-03-04T08:48:51.652 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36393.log.gz 2024-03-04T08:48:51.652 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37638.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37638.log.gz 2024-03-04T08:48:51.654 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36608.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36608.log.gz 2024-03-04T08:48:51.654 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45613.log 2024-03-04T08:48:51.655 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53784.log 2024-03-04T08:48:51.657 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32938.log 2024-03-04T08:48:51.657 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45613.log.gz 2024-03-04T08:48:51.658 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53784.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53784.log.gz 2024-03-04T08:48:51.659 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32938.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32938.log.gz 2024-03-04T08:48:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41576.log 2024-03-04T08:48:51.660 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46302.log 2024-03-04T08:48:51.662 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45129.log 2024-03-04T08:48:51.662 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41576.log.gz 2024-03-04T08:48:51.663 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46302.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46302.log.gz 2024-03-04T08:48:51.664 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45129.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45129.log.gz 2024-03-04T08:48:51.665 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40409.log 2024-03-04T08:48:51.666 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39445.log 2024-03-04T08:48:51.667 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37742.log 2024-03-04T08:48:51.668 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40409.log.gz 2024-03-04T08:48:51.668 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39445.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39445.log.gz 2024-03-04T08:48:51.669 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37742.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37742.log.gz 2024-03-04T08:48:51.670 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54783.log 2024-03-04T08:48:51.671 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37612.log 2024-03-04T08:48:51.672 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48133.log 2024-03-04T08:48:51.672 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54783.log.gz 2024-03-04T08:48:51.673 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37612.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37612.log.gz 2024-03-04T08:48:51.674 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48133.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48133.log.gz 2024-03-04T08:48:51.675 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57040.log 2024-03-04T08:48:51.676 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52585.log 2024-03-04T08:48:51.677 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55597.log 2024-03-04T08:48:51.678 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57040.log.gz 2024-03-04T08:48:51.678 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52585.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52585.log.gz 2024-03-04T08:48:51.680 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55597.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55597.log.gz 2024-03-04T08:48:51.680 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58127.log 2024-03-04T08:48:51.681 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41230.log 2024-03-04T08:48:51.684 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58127.log: /var/log/ceph/ceph-client.admin.41230.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41230.log.gz 2024-03-04T08:48:51.684 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55176.log 2024-03-04T08:48:51.685 INFO:teuthology.orchestra.run.smithi113.stderr: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.58127.log.gz 2024-03-04T08:48:51.685 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52377.log 2024-03-04T08:48:51.687 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43572.log 2024-03-04T08:48:51.687 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55176.log.gz 2024-03-04T08:48:51.687 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52377.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52377.log.gz 2024-03-04T08:48:51.689 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43572.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43572.log.gz 2024-03-04T08:48:51.689 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46703.log 2024-03-04T08:48:51.690 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47052.log 2024-03-04T08:48:51.691 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2024-03-04T08:48:51.692 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46703.log.gz 2024-03-04T08:48:51.693 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47052.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47052.log.gz 2024-03-04T08:48:51.694 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43544.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2024-03-04T08:48:51.694 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47640.log 2024-03-04T08:48:51.696 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34085.log 2024-03-04T08:48:51.697 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49983.log 2024-03-04T08:48:51.697 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47640.log.gz 2024-03-04T08:48:51.698 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34085.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34085.log.gz 2024-03-04T08:48:51.699 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49983.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49983.log.gz 2024-03-04T08:48:51.700 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56830.log 2024-03-04T08:48:51.701 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44916.log 2024-03-04T08:48:51.702 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33930.log 2024-03-04T08:48:51.702 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56830.log.gz 2024-03-04T08:48:51.703 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44916.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44916.log.gz 2024-03-04T08:48:51.704 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33930.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33930.log.gz 2024-03-04T08:48:51.705 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34313.log 2024-03-04T08:48:51.706 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42086.log 2024-03-04T08:48:51.707 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42746.log 2024-03-04T08:48:51.707 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34313.log.gz 2024-03-04T08:48:51.708 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42086.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42086.log.gz 2024-03-04T08:48:51.709 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42746.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42746.log.gz 2024-03-04T08:48:51.710 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41142.log 2024-03-04T08:48:51.711 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39565.log 2024-03-04T08:48:51.712 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48433.log 2024-03-04T08:48:51.713 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41142.log.gz 2024-03-04T08:48:51.713 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39565.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39565.log.gz 2024-03-04T08:48:51.714 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41902.log 2024-03-04T08:48:51.715 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48433.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48433.log.gz 2024-03-04T08:48:51.716 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43905.log 2024-03-04T08:48:51.716 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41902.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41902.log.gz 2024-03-04T08:48:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43905.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43905.log.gz 2024-03-04T08:48:51.718 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54627.log 2024-03-04T08:48:51.719 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43014.log 2024-03-04T08:48:51.721 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43438.log 2024-03-04T08:48:51.721 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54627.log.gz 2024-03-04T08:48:51.722 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43014.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43014.log.gz 2024-03-04T08:48:51.723 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43438.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43438.log.gz 2024-03-04T08:48:51.723 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57407.log 2024-03-04T08:48:51.725 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51239.log 2024-03-04T08:48:51.726 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42848.log 2024-03-04T08:48:51.726 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57407.log.gz 2024-03-04T08:48:51.727 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51239.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51239.log.gz 2024-03-04T08:48:51.728 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42848.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42848.log.gz 2024-03-04T08:48:51.729 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56255.log 2024-03-04T08:48:51.730 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44382.log 2024-03-04T08:48:51.731 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33852.log 2024-03-04T08:48:51.731 INFO:teuthology.orchestra.run.smithi113.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56255.log.gz 2024-03-04T08:48:51.732 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44382.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44382.log.gz 2024-03-04T08:48:51.733 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33852.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33852.log.gz 2024-03-04T08:48:51.734 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46918.log 2024-03-04T08:48:51.735 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52878.log 2024-03-04T08:48:51.736 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58202.log 2024-03-04T08:48:51.737 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46918.log.gz 2024-03-04T08:48:51.737 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52878.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52878.log.gz 2024-03-04T08:48:51.740 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58202.log: 95.6% -- replaced with /var/log/ceph/ceph-client.admin.58202.log.gz 2024-03-04T08:48:51.741 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51763.log 2024-03-04T08:48:51.742 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32239.log 2024-03-04T08:48:51.743 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51205.log 2024-03-04T08:48:51.744 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51763.log.gz 2024-03-04T08:48:51.744 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32239.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32239.log.gz 2024-03-04T08:48:51.746 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51205.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51205.log.gz 2024-03-04T08:48:51.746 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52272.log 2024-03-04T08:48:51.747 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33878.log 2024-03-04T08:48:51.748 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39936.log 2024-03-04T08:48:51.749 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52272.log.gz 2024-03-04T08:48:51.750 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33878.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33878.log.gz 2024-03-04T08:48:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39936.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39936.log.gz 2024-03-04T08:48:51.751 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32237.log 2024-03-04T08:48:51.752 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57144.log 2024-03-04T08:48:51.754 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40802.log 2024-03-04T08:48:51.754 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32237.log.gz 2024-03-04T08:48:51.755 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57144.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57144.log.gz 2024-03-04T08:48:51.756 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40802.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40802.log.gz 2024-03-04T08:48:51.756 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41256.log 2024-03-04T08:48:51.758 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49286.log 2024-03-04T08:48:51.759 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55098.log 2024-03-04T08:48:51.759 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41256.log.gz 2024-03-04T08:48:51.760 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49286.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49286.log.gz 2024-03-04T08:48:51.761 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55098.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55098.log.gz 2024-03-04T08:48:51.762 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39729.log 2024-03-04T08:48:51.763 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58020.log 2024-03-04T08:48:51.764 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58074.log 2024-03-04T08:48:51.765 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39729.log.gz 2024-03-04T08:48:51.765 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58020.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58020.log.gz 2024-03-04T08:48:51.768 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58074.log: 95.8% -- replaced with /var/log/ceph/ceph-client.admin.58074.log.gz 2024-03-04T08:48:51.769 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48001.log 2024-03-04T08:48:51.770 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57837.log 2024-03-04T08:48:51.772 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42378.log 2024-03-04T08:48:51.778 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48001.log.gz 2024-03-04T08:48:51.778 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57837.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57837.log.gz 2024-03-04T08:48:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42378.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42378.log.gz 2024-03-04T08:48:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40828.log 2024-03-04T08:48:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54890.log 2024-03-04T08:48:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35833.log 2024-03-04T08:48:51.779 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40828.log.gz 2024-03-04T08:48:51.779 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54890.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54890.log.gz 2024-03-04T08:48:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35833.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35833.log.gz 2024-03-04T08:48:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48574.log 2024-03-04T08:48:51.780 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-04T08:48:51.782 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58279.log 2024-03-04T08:48:51.782 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48574.log.gz 2024-03-04T08:48:51.786 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.58279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48678.log 2024-03-04T08:48:51.787 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52298.log 2024-03-04T08:48:51.788 INFO:teuthology.orchestra.run.smithi113.stderr: 95.8%/var/log/ceph/ceph-client.admin.48678.log: -- replaced with /var/log/ceph/ceph-client.admin.58279.log.gz 2024-03-04T08:48:51.788 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48678.log.gz 2024-03-04T08:48:51.790 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52298.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52298.log.gz 2024-03-04T08:48:51.790 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31855.log 2024-03-04T08:48:51.791 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-04T08:48:51.793 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31855.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31855.log.gz 2024-03-04T08:48:51.794 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51895.log 2024-03-04T08:48:51.795 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43650.log 2024-03-04T08:48:51.797 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38545.log 2024-03-04T08:48:51.797 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51895.log.gz 2024-03-04T08:48:51.798 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43650.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43650.log.gz 2024-03-04T08:48:51.799 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38545.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38545.log.gz 2024-03-04T08:48:51.800 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50414.log 2024-03-04T08:48:51.800 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56856.log 2024-03-04T08:48:51.802 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43959.log 2024-03-04T08:48:51.802 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50414.log.gz 2024-03-04T08:48:51.802 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56856.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56856.log.gz 2024-03-04T08:48:51.804 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43959.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43959.log.gz 2024-03-04T08:48:51.804 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56333.log 2024-03-04T08:48:51.805 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32201.log 2024-03-04T08:48:51.806 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39669.log 2024-03-04T08:48:51.806 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56333.log.gz 2024-03-04T08:48:51.807 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32201.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32201.log.gz 2024-03-04T08:48:51.809 INFO:teuthology.orchestra.run.smithi136.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-04T08:48:51.811 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39669.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39669.log.gz 2024-03-04T08:48:51.811 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47458.log 2024-03-04T08:48:51.812 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46758.log 2024-03-04T08:48:51.812 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51534.log 2024-03-04T08:48:51.812 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47458.log.gz 2024-03-04T08:48:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46758.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46758.log.gz 2024-03-04T08:48:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51534.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51534.log.gz 2024-03-04T08:48:51.814 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43492.log 2024-03-04T08:48:51.816 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45692.log 2024-03-04T08:48:51.817 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44669.log 2024-03-04T08:48:51.817 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43492.log.gz 2024-03-04T08:48:51.818 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45692.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45692.log.gz 2024-03-04T08:48:51.818 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44669.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44669.log.gz 2024-03-04T08:48:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35451.log 2024-03-04T08:48:51.820 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42574.log 2024-03-04T08:48:51.822 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52033.log 2024-03-04T08:48:51.822 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35451.log.gz 2024-03-04T08:48:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42574.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42574.log.gz 2024-03-04T08:48:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52033.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52033.log.gz 2024-03-04T08:48:51.824 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50759.log 2024-03-04T08:48:51.825 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41337.log 2024-03-04T08:48:51.826 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44695.log 2024-03-04T08:48:51.827 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50759.log.gz 2024-03-04T08:48:51.828 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41337.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41337.log.gz 2024-03-04T08:48:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44695.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44695.log.gz 2024-03-04T08:48:51.829 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57539.log 2024-03-04T08:48:51.830 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39367.log 2024-03-04T08:48:51.832 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52455.log 2024-03-04T08:48:51.832 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57539.log.gz 2024-03-04T08:48:51.833 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39367.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39367.log.gz 2024-03-04T08:48:51.834 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52455.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52455.log.gz 2024-03-04T08:48:51.834 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40487.log 2024-03-04T08:48:51.835 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46651.log 2024-03-04T08:48:51.836 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36026.log 2024-03-04T08:48:51.836 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40487.log.gz 2024-03-04T08:48:51.838 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46651.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46651.log.gz 2024-03-04T08:48:51.838 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45272.log 2024-03-04T08:48:51.839 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36026.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36026.log.gz 2024-03-04T08:48:51.840 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27668.log 2024-03-04T08:48:51.841 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45272.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45272.log.gz 2024-03-04T08:48:51.842 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.27668.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27668.log.gz 2024-03-04T08:48:51.843 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57723.log 2024-03-04T08:48:51.844 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51023.log 2024-03-04T08:48:51.845 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44355.log 2024-03-04T08:48:51.845 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57723.log.gz 2024-03-04T08:48:51.847 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51023.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51023.log.gz 2024-03-04T08:48:51.847 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44355.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44355.log.gz 2024-03-04T08:48:51.848 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39996.log 2024-03-04T08:48:51.849 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46564.log 2024-03-04T08:48:51.850 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47078.log 2024-03-04T08:48:51.851 INFO:teuthology.orchestra.run.smithi113.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39996.log.gz 2024-03-04T08:48:51.852 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46564.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46564.log.gz 2024-03-04T08:48:51.853 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47078.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47078.log.gz 2024-03-04T08:48:51.853 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44118.log 2024-03-04T08:48:51.854 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45218.log 2024-03-04T08:48:51.855 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46222.log 2024-03-04T08:48:51.855 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44118.log.gz 2024-03-04T08:48:51.857 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45218.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45218.log.gz 2024-03-04T08:48:51.858 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46222.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46222.log.gz 2024-03-04T08:48:51.858 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55283.log 2024-03-04T08:48:51.859 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51127.log 2024-03-04T08:48:51.862 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45455.log 2024-03-04T08:48:51.862 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55283.log.gz 2024-03-04T08:48:51.862 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51127.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51127.log.gz 2024-03-04T08:48:51.863 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45455.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45455.log.gz 2024-03-04T08:48:51.863 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45585.log 2024-03-04T08:48:51.864 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42546.log 2024-03-04T08:48:51.865 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46354.log 2024-03-04T08:48:51.865 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45585.log.gz 2024-03-04T08:48:51.866 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42546.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42546.log.gz 2024-03-04T08:48:51.867 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46354.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46354.log.gz 2024-03-04T08:48:51.868 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43849.log 2024-03-04T08:48:51.869 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56307.log 2024-03-04T08:48:51.870 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43040.log 2024-03-04T08:48:51.871 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43849.log.gz 2024-03-04T08:48:51.871 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56307.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56307.log.gz 2024-03-04T08:48:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43040.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43040.log.gz 2024-03-04T08:48:51.872 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54757.log 2024-03-04T08:48:51.874 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48626.log 2024-03-04T08:48:51.875 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47324.log 2024-03-04T08:48:51.875 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54757.log.gz 2024-03-04T08:48:51.876 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48626.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48626.log.gz 2024-03-04T08:48:51.878 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47324.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47324.log.gz 2024-03-04T08:48:51.878 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53732.log 2024-03-04T08:48:51.878 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52852.log 2024-03-04T08:48:51.880 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54494.log 2024-03-04T08:48:51.880 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53732.log.gz 2024-03-04T08:48:51.881 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52852.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52852.log.gz 2024-03-04T08:48:51.882 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54494.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54494.log.gz 2024-03-04T08:48:51.883 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42792.log 2024-03-04T08:48:51.884 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45244.log 2024-03-04T08:48:51.885 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49178.log 2024-03-04T08:48:51.885 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42792.log.gz 2024-03-04T08:48:51.886 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45244.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45244.log.gz 2024-03-04T08:48:51.887 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49178.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49178.log.gz 2024-03-04T08:48:51.888 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32254.log 2024-03-04T08:48:51.889 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49923.log 2024-03-04T08:48:51.890 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50645.log 2024-03-04T08:48:51.890 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32254.log.gz 2024-03-04T08:48:51.891 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49923.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49923.log.gz 2024-03-04T08:48:51.892 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50645.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50645.log.gz 2024-03-04T08:48:51.893 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32253.log 2024-03-04T08:48:51.894 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56674.log 2024-03-04T08:48:51.895 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33826.log 2024-03-04T08:48:51.895 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32253.log.gz 2024-03-04T08:48:51.896 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56674.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56674.log.gz 2024-03-04T08:48:51.897 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33826.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33826.log.gz 2024-03-04T08:48:51.898 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33438.log 2024-03-04T08:48:51.899 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40079.log 2024-03-04T08:48:51.900 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55124.log 2024-03-04T08:48:51.900 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33438.log.gz 2024-03-04T08:48:51.901 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40079.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40079.log.gz 2024-03-04T08:48:51.902 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55124.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55124.log.gz 2024-03-04T08:48:51.903 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41090.log 2024-03-04T08:48:51.904 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36359.log 2024-03-04T08:48:51.905 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49614.log 2024-03-04T08:48:51.905 INFO:teuthology.orchestra.run.smithi113.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41090.log.gz 2024-03-04T08:48:51.906 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36359.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36359.log.gz 2024-03-04T08:48:51.907 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49614.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49614.log.gz 2024-03-04T08:48:51.908 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52800.log 2024-03-04T08:48:51.909 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45718.log 2024-03-04T08:48:51.910 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46275.log 2024-03-04T08:48:51.911 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52800.log.gz 2024-03-04T08:48:51.911 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45718.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45718.log.gz 2024-03-04T08:48:51.912 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46275.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46275.log.gz 2024-03-04T08:48:51.913 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52059.log 2024-03-04T08:48:51.914 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54154.log 2024-03-04T08:48:51.915 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33383.log 2024-03-04T08:48:51.916 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52059.log.gz 2024-03-04T08:48:51.916 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54154.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54154.log.gz 2024-03-04T08:48:51.918 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44994.log 2024-03-04T08:48:51.919 INFO:teuthology.orchestra.run.smithi113.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.33383.log.gz 2024-03-04T08:48:51.919 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52403.log 2024-03-04T08:48:51.921 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31638.log 2024-03-04T08:48:51.921 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44994.log.gz 2024-03-04T08:48:51.921 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52403.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52403.log.gz 2024-03-04T08:48:51.922 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54102.log 2024-03-04T08:48:51.922 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50204.log 2024-03-04T08:48:51.923 INFO:teuthology.orchestra.run.smithi113.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31638.log.gz 2024-03-04T08:48:51.924 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48405.log 2024-03-04T08:48:51.924 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54102.log.gz 2024-03-04T08:48:51.925 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50204.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50204.log.gz 2024-03-04T08:48:51.926 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48405.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48405.log.gz 2024-03-04T08:48:51.926 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34836.log 2024-03-04T08:48:51.928 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43624.log 2024-03-04T08:48:51.929 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34373.log 2024-03-04T08:48:51.929 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34836.log.gz 2024-03-04T08:48:51.930 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43624.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43624.log.gz 2024-03-04T08:48:51.931 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34373.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34373.log.gz 2024-03-04T08:48:51.931 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41311.log 2024-03-04T08:48:51.932 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36226.log 2024-03-04T08:48:51.934 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49722.log 2024-03-04T08:48:51.934 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41311.log.gz 2024-03-04T08:48:51.935 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36226.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36226.log.gz 2024-03-04T08:48:51.936 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49722.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49722.log.gz 2024-03-04T08:48:51.937 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54312.log 2024-03-04T08:48:51.938 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45166.log 2024-03-04T08:48:51.939 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54994.log 2024-03-04T08:48:51.939 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54312.log.gz 2024-03-04T08:48:51.940 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45166.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45166.log.gz 2024-03-04T08:48:51.941 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54994.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54994.log.gz 2024-03-04T08:48:51.942 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52904.log 2024-03-04T08:48:51.943 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55335.log 2024-03-04T08:48:51.944 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56726.log 2024-03-04T08:48:51.945 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52904.log.gz 2024-03-04T08:48:51.946 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55335.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55335.log.gz 2024-03-04T08:48:51.946 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56726.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56726.log.gz 2024-03-04T08:48:51.947 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33016.log 2024-03-04T08:48:51.948 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43253.log 2024-03-04T08:48:51.950 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34558.log 2024-03-04T08:48:51.950 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33016.log.gz 2024-03-04T08:48:51.950 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43253.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43253.log.gz 2024-03-04T08:48:51.952 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34558.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34558.log.gz 2024-03-04T08:48:51.952 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54338.log 2024-03-04T08:48:51.953 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54835.log 2024-03-04T08:48:51.955 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45559.log 2024-03-04T08:48:51.955 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54338.log.gz 2024-03-04T08:48:51.956 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54835.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54835.log.gz 2024-03-04T08:48:51.957 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45559.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45559.log.gz 2024-03-04T08:48:51.957 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57968.log 2024-03-04T08:48:51.959 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53995.log 2024-03-04T08:48:51.960 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35750.log 2024-03-04T08:48:51.961 INFO:teuthology.orchestra.run.smithi113.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57968.log.gz 2024-03-04T08:48:51.961 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53995.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53995.log.gz 2024-03-04T08:48:51.962 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35750.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35750.log.gz 2024-03-04T08:48:51.963 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35533.log 2024-03-04T08:48:51.964 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41798.log 2024-03-04T08:48:51.965 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53496.log 2024-03-04T08:48:51.966 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35533.log.gz 2024-03-04T08:48:51.966 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41798.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41798.log.gz 2024-03-04T08:48:51.967 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53496.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53496.log.gz 2024-03-04T08:48:51.968 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57355.log 2024-03-04T08:48:51.969 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39617.log 2024-03-04T08:48:51.971 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45639.log 2024-03-04T08:48:51.971 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57355.log.gz 2024-03-04T08:48:51.972 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39617.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39617.log.gz 2024-03-04T08:48:51.972 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45639.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45639.log.gz 2024-03-04T08:48:51.973 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48519.log 2024-03-04T08:48:51.974 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33722.log 2024-03-04T08:48:51.975 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43677.log 2024-03-04T08:48:51.976 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48519.log.gz 2024-03-04T08:48:51.981 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33722.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33722.log.gz 2024-03-04T08:48:51.982 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43677.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43677.log.gz 2024-03-04T08:48:51.982 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54416.log 2024-03-04T08:48:51.982 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-04T08:48:51.982 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55230.log 2024-03-04T08:48:51.982 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54416.log.gz 2024-03-04T08:48:51.982 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34454.log 2024-03-04T08:48:51.983 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55230.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55230.log.gz 2024-03-04T08:48:51.984 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34454.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-04T08:48:51.984 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34454.log.gz 2024-03-04T08:48:51.985 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-04T08:48:51.986 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54234.log 2024-03-04T08:48:51.987 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43791.log 2024-03-04T08:48:51.988 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40435.log 2024-03-04T08:48:51.988 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54234.log.gz 2024-03-04T08:48:51.990 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43791.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.43791.log.gz 2024-03-04T08:48:51.991 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40435.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40435.log.gz 2024-03-04T08:48:51.994 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48792.log 2024-03-04T08:48:51.995 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32255.log 2024-03-04T08:48:51.996 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48792.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48792.log.gz 2024-03-04T08:48:51.997 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53680.log 2024-03-04T08:48:51.997 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32255.log.gz 2024-03-04T08:48:51.998 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57170.log 2024-03-04T08:48:51.999 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53680.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53680.log.gz 2024-03-04T08:48:52.000 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54442.log 2024-03-04T08:48:52.000 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57170.log.gz 2024-03-04T08:48:52.001 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-04T08:48:52.003 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54442.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54442.log.gz 2024-03-04T08:48:52.003 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50788.log 2024-03-04T08:48:52.006 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50788.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50788.log.gz 2024-03-04T08:48:52.006 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33094.log 2024-03-04T08:48:52.009 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33094.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33094.log.gz 2024-03-04T08:48:52.009 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32643.log 2024-03-04T08:48:52.012 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32643.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32643.log.gz 2024-03-04T08:48:52.012 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38258.log 2024-03-04T08:48:52.013 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44302.log 2024-03-04T08:48:52.015 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38258.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38258.log.gz 2024-03-04T08:48:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44302.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44302.log.gz 2024-03-04T08:48:52.018 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38041.log 2024-03-04T08:48:52.021 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38041.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38041.log.gz 2024-03-04T08:48:52.021 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34137.log 2024-03-04T08:48:52.024 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34137.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34137.log.gz 2024-03-04T08:48:52.024 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38362.log 2024-03-04T08:48:52.027 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38362.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38362.log.gz 2024-03-04T08:48:52.027 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41692.log 2024-03-04T08:48:52.030 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41692.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41692.log.gz 2024-03-04T08:48:52.030 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41524.log 2024-03-04T08:48:52.033 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41524.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41524.log.gz 2024-03-04T08:48:52.033 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36528.log 2024-03-04T08:48:52.034 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51320.log 2024-03-04T08:48:52.036 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36528.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36528.log.gz 2024-03-04T08:48:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51320.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51320.log.gz 2024-03-04T08:48:52.039 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50383.log 2024-03-04T08:48:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50383.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50383.log.gz 2024-03-04T08:48:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33224.log 2024-03-04T08:48:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33224.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33224.log.gz 2024-03-04T08:48:52.045 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34766.log 2024-03-04T08:48:52.048 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34766.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34766.log.gz 2024-03-04T08:48:52.048 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56489.log 2024-03-04T08:48:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56489.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56489.log.gz 2024-03-04T08:48:52.051 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-03-04T08:48:52.054 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45799.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz 2024-03-04T08:48:52.054 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39505.log 2024-03-04T08:48:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40618.log 2024-03-04T08:48:52.057 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39505.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39505.log.gz 2024-03-04T08:48:52.060 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40618.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40618.log.gz 2024-03-04T08:48:52.060 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40880.log 2024-03-04T08:48:52.063 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40880.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40880.log.gz 2024-03-04T08:48:52.063 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57381.log 2024-03-04T08:48:52.066 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52533.log 2024-03-04T08:48:52.067 INFO:teuthology.orchestra.run.smithi113.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57381.log.gz 2024-03-04T08:48:52.067 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47895.log 2024-03-04T08:48:52.069 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52533.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52533.log.gz 2024-03-04T08:48:52.071 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45481.log 2024-03-04T08:48:52.073 INFO:teuthology.orchestra.run.smithi113.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47895.log.gz 2024-03-04T08:48:52.075 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37909.log 2024-03-04T08:48:52.076 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45481.log.gz 2024-03-04T08:48:52.079 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37909.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37909.log.gz 2024-03-04T08:48:52.079 INFO:teuthology.orchestra.run.smithi113.stderr: -- /var/log/ceph/ceph-client.admin.39696.log 2024-03-04T08:48:52.082 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39696.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39696.log.gz 2024-03-04T08:48:52.082 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41876.log 2024-03-04T08:48:52.085 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41876.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41876.log.gz 2024-03-04T08:48:52.085 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39806.log 2024-03-04T08:48:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40183.log 2024-03-04T08:48:52.088 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39806.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39806.log.gz 2024-03-04T08:48:52.091 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40183.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40183.log.gz 2024-03-04T08:48:52.091 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32669.log 2024-03-04T08:48:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32669.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32669.log.gz 2024-03-04T08:48:52.094 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54601.log 2024-03-04T08:48:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54601.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54601.log.gz 2024-03-04T08:48:52.097 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53412.log 2024-03-04T08:48:52.100 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31907.log 2024-03-04T08:48:52.100 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53412.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53412.log.gz 2024-03-04T08:48:52.102 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56174.log 2024-03-04T08:48:52.103 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31907.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31907.log.gz 2024-03-04T08:48:52.106 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56174.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56174.log.gz 2024-03-04T08:48:52.106 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50702.log 2024-03-04T08:48:52.109 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50702.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50702.log.gz 2024-03-04T08:48:52.109 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39251.log 2024-03-04T08:48:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57565.log 2024-03-04T08:48:52.112 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39251.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39251.log.gz 2024-03-04T08:48:52.115 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57565.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57565.log.gz 2024-03-04T08:48:52.115 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41772.log 2024-03-04T08:48:52.118 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41772.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41772.log.gz 2024-03-04T08:48:52.118 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36419.log 2024-03-04T08:48:52.121 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36419.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36419.log.gz 2024-03-04T08:48:52.121 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43933.log 2024-03-04T08:48:52.125 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31881.log 2024-03-04T08:48:52.126 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43933.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43933.log.gz 2024-03-04T08:48:52.126 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44942.log 2024-03-04T08:48:52.127 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31881.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31881.log.gz 2024-03-04T08:48:52.129 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44942.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44942.log.gz 2024-03-04T08:48:52.130 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.34584.log 2024-03-04T08:48:52.132 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34584.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22583.log 2024-03-04T08:48:52.133 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34584.log.gz 2024-03-04T08:48:52.135 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.tmp-client.admin.22583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52351.log 2024-03-04T08:48:52.136 INFO:teuthology.orchestra.run.smithi113.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22583.log.gz 2024-03-04T08:48:52.138 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48730.log 2024-03-04T08:48:52.142 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48730.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48730.log.gz 2024-03-04T08:48:52.142 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52351.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52351.log.gz 2024-03-04T08:48:52.142 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44721.log 2024-03-04T08:48:52.148 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44721.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44721.log.gz 2024-03-04T08:48:52.149 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55046.log 2024-03-04T08:48:52.149 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55046.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55046.log.gz 2024-03-04T08:48:52.149 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45047.log 2024-03-04T08:48:52.151 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45047.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45047.log.gz 2024-03-04T08:48:52.152 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35478.log 2024-03-04T08:48:52.153 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44408.log 2024-03-04T08:48:52.155 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35478.log: /var/log/ceph/ceph-client.admin.44408.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35478.log.gz 2024-03-04T08:48:52.155 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51265.log 2024-03-04T08:48:52.155 INFO:teuthology.orchestra.run.smithi113.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44408.log.gz 2024-03-04T08:48:52.158 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51265.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51265.log.gz 2024-03-04T08:48:52.158 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-03-04T08:48:52.161 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35969.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-03-04T08:48:52.161 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38015.log 2024-03-04T08:48:52.164 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38015.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38015.log.gz 2024-03-04T08:48:52.164 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38651.log 2024-03-04T08:48:52.167 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38651.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38651.log.gz 2024-03-04T08:48:52.167 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54916.log 2024-03-04T08:48:52.170 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54916.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54916.log.gz 2024-03-04T08:48:52.170 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57811.log 2024-03-04T08:48:52.173 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53758.log 2024-03-04T08:48:52.173 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57811.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57811.log.gz 2024-03-04T08:48:52.176 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53758.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53758.log.gz 2024-03-04T08:48:52.176 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36635.log 2024-03-04T08:48:52.179 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36635.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36635.log.gz 2024-03-04T08:48:52.179 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56622.log 2024-03-04T08:48:52.182 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43173.log 2024-03-04T08:48:52.182 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56622.log.gz 2024-03-04T08:48:52.183 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48704.log 2024-03-04T08:48:52.189 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43173.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43173.log.gz 2024-03-04T08:48:52.189 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32455.log 2024-03-04T08:48:52.189 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48704.log.gz 2024-03-04T08:48:52.191 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52748.log 2024-03-04T08:48:52.192 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32455.log.gz 2024-03-04T08:48:52.193 INFO:teuthology.orchestra.run.smithi136.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-04T08:48:52.194 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52668.log 2024-03-04T08:48:52.195 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52748.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52748.log.gz 2024-03-04T08:48:52.199 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52668.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52668.log.gz 2024-03-04T08:48:52.199 INFO:teuthology.orchestra.run.smithi113.stderr: -- /var/log/ceph/ceph-client.admin.31771.log 2024-03-04T08:48:52.202 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55361.log 2024-03-04T08:48:52.202 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31771.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31771.log.gz 2024-03-04T08:48:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55361.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55361.log.gz 2024-03-04T08:48:52.205 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56096.log 2024-03-04T08:48:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56096.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56096.log.gz 2024-03-04T08:48:52.208 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43359.log 2024-03-04T08:48:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43359.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43359.log.gz 2024-03-04T08:48:52.211 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47843.log 2024-03-04T08:48:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55729.log 2024-03-04T08:48:52.214 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47843.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47843.log.gz 2024-03-04T08:48:52.216 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55729.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55729.log.gz 2024-03-04T08:48:52.216 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55677.log 2024-03-04T08:48:52.219 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55677.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55677.log.gz 2024-03-04T08:48:52.219 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31555.log 2024-03-04T08:48:52.222 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55517.log 2024-03-04T08:48:52.222 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31555.log.gz 2024-03-04T08:48:52.223 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50440.log 2024-03-04T08:48:52.224 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55517.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55517.log.gz 2024-03-04T08:48:52.226 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40591.log 2024-03-04T08:48:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50440.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50440.log.gz 2024-03-04T08:48:52.230 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44591.log 2024-03-04T08:48:52.230 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40591.log.gz 2024-03-04T08:48:52.232 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51815.log 2024-03-04T08:48:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44591.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44591.log.gz 2024-03-04T08:48:52.233 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44565.log 2024-03-04T08:48:52.235 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51815.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51815.log.gz 2024-03-04T08:48:52.238 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44565.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44565.log.gz 2024-03-04T08:48:52.238 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49259.log 2024-03-04T08:48:52.241 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49259.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49259.log.gz 2024-03-04T08:48:52.241 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-03-04T08:48:52.244 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43279.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-03-04T08:48:52.244 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31827.log 2024-03-04T08:48:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31827.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31827.log.gz 2024-03-04T08:48:52.247 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52642.log 2024-03-04T08:48:52.249 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55545.log 2024-03-04T08:48:52.250 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52642.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52642.log.gz 2024-03-04T08:48:52.252 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34287.log 2024-03-04T08:48:52.253 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55545.log.gz 2024-03-04T08:48:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34287.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34287.log.gz 2024-03-04T08:48:52.256 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50263.log 2024-03-04T08:48:52.259 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50263.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50263.log.gz 2024-03-04T08:48:52.259 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35887.log 2024-03-04T08:48:52.263 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47296.log 2024-03-04T08:48:52.263 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35887.log.gz 2024-03-04T08:48:52.264 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55571.log 2024-03-04T08:48:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55571.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55571.log.gz 2024-03-04T08:48:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47296.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47296.log.gz 2024-03-04T08:48:52.269 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54942.log 2024-03-04T08:48:52.272 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54942.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54942.log.gz 2024-03-04T08:48:52.272 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32449.log 2024-03-04T08:48:52.273 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45350.log 2024-03-04T08:48:52.275 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32449.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32449.log.gz 2024-03-04T08:48:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45350.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45350.log.gz 2024-03-04T08:48:52.277 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53040.log 2024-03-04T08:48:52.281 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53040.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53040.log.gz 2024-03-04T08:48:52.281 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49774.log 2024-03-04T08:48:52.284 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49774.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49774.log.gz 2024-03-04T08:48:52.284 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51049.log 2024-03-04T08:48:52.285 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38388.log 2024-03-04T08:48:52.287 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51049.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51049.log.gz 2024-03-04T08:48:52.289 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37664.log 2024-03-04T08:48:52.291 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38388.log.gz 2024-03-04T08:48:52.291 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49837.log 2024-03-04T08:48:52.293 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.37664.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37664.log.gz 2024-03-04T08:48:52.294 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49837.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49837.log.gz 2024-03-04T08:48:52.295 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39884.log 2024-03-04T08:48:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39884.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39884.log.gz 2024-03-04T08:48:52.297 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46458.log 2024-03-04T08:48:52.300 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46458.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46458.log.gz 2024-03-04T08:48:52.300 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40670.log 2024-03-04T08:48:52.303 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40670.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40670.log.gz 2024-03-04T08:48:52.303 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33956.log 2024-03-04T08:48:52.308 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33956.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33956.log.gz 2024-03-04T08:48:52.308 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52694.log 2024-03-04T08:48:52.311 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52694.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52694.log.gz 2024-03-04T08:48:52.311 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55940.log 2024-03-04T08:48:52.316 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55781.log 2024-03-04T08:48:52.316 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55940.log: /var/log/ceph/ceph-client.admin.55781.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55940.log.gz 2024-03-04T08:48:52.316 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose 87.8% -- -- replaced with /var/log/ceph/ceph-client.admin.55781.log.gz 2024-03-04T08:48:52.316 INFO:teuthology.orchestra.run.smithi113.stderr: /var/log/ceph/ceph-client.admin.54731.log 2024-03-04T08:48:52.318 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54731.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54731.log.gz 2024-03-04T08:48:52.318 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31664.log 2024-03-04T08:48:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31664.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31664.log.gz 2024-03-04T08:48:52.321 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51291.log 2024-03-04T08:48:52.324 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51291.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51291.log.gz 2024-03-04T08:48:52.324 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41470.log 2024-03-04T08:48:52.326 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43739.log 2024-03-04T08:48:52.327 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41470.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41470.log.gz 2024-03-04T08:48:52.330 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43739.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43739.log.gz 2024-03-04T08:48:52.331 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40051.log 2024-03-04T08:48:52.333 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40051.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40051.log.gz 2024-03-04T08:48:52.334 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47510.log 2024-03-04T08:48:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47510.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47510.log.gz 2024-03-04T08:48:52.336 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41628.log 2024-03-04T08:48:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36445.log 2024-03-04T08:48:52.339 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41628.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41628.log.gz 2024-03-04T08:48:52.343 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36445.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36445.log.gz 2024-03-04T08:48:52.343 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34339.log 2024-03-04T08:48:52.345 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.34339.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34339.log.gz 2024-03-04T08:48:52.346 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36333.log 2024-03-04T08:48:52.348 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36333.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36333.log.gz 2024-03-04T08:48:52.348 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32249.log 2024-03-04T08:48:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46022.log 2024-03-04T08:48:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32249.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32249.log.gz 2024-03-04T08:48:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46022.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46022.log.gz 2024-03-04T08:48:52.354 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49585.log 2024-03-04T08:48:52.356 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40565.log 2024-03-04T08:48:52.358 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45298.log 2024-03-04T08:48:52.358 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49585.log.gz 2024-03-04T08:48:52.360 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40565.log: gzip/var/log/ceph/ceph-client.admin.45298.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53219.log 2024-03-04T08:48:52.362 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35611.log 2024-03-04T08:48:52.362 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45298.log.gz 2024-03-04T08:48:52.362 INFO:teuthology.orchestra.run.smithi113.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40565.log.gz 2024-03-04T08:48:52.366 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53219.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53219.log.gz 2024-03-04T08:48:52.366 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35611.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35611.log.gz 2024-03-04T08:48:52.366 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49091.log 2024-03-04T08:48:52.369 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49091.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49091.log.gz 2024-03-04T08:48:52.369 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57863.log 2024-03-04T08:48:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57863.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57863.log.gz 2024-03-04T08:48:52.372 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35707.log 2024-03-04T08:48:52.373 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54364.log 2024-03-04T08:48:52.375 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.35707.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35707.log.gz 2024-03-04T08:48:52.378 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54364.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54364.log.gz 2024-03-04T08:48:52.378 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51973.log 2024-03-04T08:48:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51973.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51973.log.gz 2024-03-04T08:48:52.381 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49364.log 2024-03-04T08:48:52.384 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49364.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49364.log.gz 2024-03-04T08:48:52.384 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56962.log 2024-03-04T08:48:52.387 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56962.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56962.log.gz 2024-03-04T08:48:52.387 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47949.log 2024-03-04T08:48:52.392 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47949.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47949.log.gz 2024-03-04T08:48:52.392 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41980.log 2024-03-04T08:48:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49693.log 2024-03-04T08:48:52.393 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41980.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41980.log.gz 2024-03-04T08:48:52.396 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49693.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49693.log.gz 2024-03-04T08:48:52.396 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53138.log 2024-03-04T08:48:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48081.log 2024-03-04T08:48:52.399 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53138.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53138.log.gz 2024-03-04T08:48:52.401 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48081.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48081.log.gz 2024-03-04T08:48:52.402 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49233.log 2024-03-04T08:48:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49233.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49233.log.gz 2024-03-04T08:48:52.405 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46677.log 2024-03-04T08:48:52.408 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46677.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46677.log.gz 2024-03-04T08:48:52.408 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31519.log 2024-03-04T08:48:52.409 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48053.log 2024-03-04T08:48:52.411 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31519.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31519.log.gz 2024-03-04T08:48:52.413 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.48053.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48053.log.gz 2024-03-04T08:48:52.414 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55439.log 2024-03-04T08:48:52.416 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55439.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55439.log.gz 2024-03-04T08:48:52.417 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51867.log 2024-03-04T08:48:52.421 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51867.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51867.log.gz 2024-03-04T08:48:52.422 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47405.log 2024-03-04T08:48:52.423 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47405.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz 2024-03-04T08:48:52.423 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57196.log 2024-03-04T08:48:52.425 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38232.log 2024-03-04T08:48:52.426 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57196.log.gz 2024-03-04T08:48:52.428 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48493.log 2024-03-04T08:48:52.430 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38232.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38232.log.gz 2024-03-04T08:48:52.431 INFO:teuthology.orchestra.run.smithi113.stderr:gzip/var/log/ceph/ceph-client.admin.48493.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57459.log 2024-03-04T08:48:52.432 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48493.log.gz 2024-03-04T08:48:52.435 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42874.log 2024-03-04T08:48:52.436 INFO:teuthology.orchestra.run.smithi113.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57459.log.gz 2024-03-04T08:48:52.438 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50671.log 2024-03-04T08:48:52.442 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50671.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50671.log.gz 2024-03-04T08:48:52.442 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42874.log.gz 2024-03-04T08:48:52.443 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32063.log 2024-03-04T08:48:52.445 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32063.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32063.log.gz 2024-03-04T08:48:52.445 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46484.log 2024-03-04T08:48:52.448 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.46484.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46484.log.gz 2024-03-04T08:48:52.448 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52166.log 2024-03-04T08:48:52.451 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40539.log 2024-03-04T08:48:52.451 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52166.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52166.log.gz 2024-03-04T08:48:52.454 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40539.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40539.log.gz 2024-03-04T08:48:52.455 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32245.log 2024-03-04T08:48:52.458 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32245.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32245.log.gz 2024-03-04T08:48:52.458 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49478.log 2024-03-04T08:48:52.461 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49478.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49478.log.gz 2024-03-04T08:48:52.461 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45099.log 2024-03-04T08:48:52.464 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56385.log 2024-03-04T08:48:52.464 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45099.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45099.log.gz 2024-03-04T08:48:52.467 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56385.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56385.log.gz 2024-03-04T08:48:52.467 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36200.log 2024-03-04T08:48:52.470 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36200.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36200.log.gz 2024-03-04T08:48:52.470 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39832.log 2024-03-04T08:48:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32877.log 2024-03-04T08:48:52.473 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39832.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39832.log.gz 2024-03-04T08:48:52.476 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32877.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32877.log.gz 2024-03-04T08:48:52.476 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52114.log 2024-03-04T08:48:52.479 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52114.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52114.log.gz 2024-03-04T08:48:52.479 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33552.log 2024-03-04T08:48:52.481 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.0.33552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.33552.log.gz 2024-03-04T08:48:52.482 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57224.log 2024-03-04T08:48:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57224.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57224.log.gz 2024-03-04T08:48:52.485 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57775.log 2024-03-04T08:48:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44172.log 2024-03-04T08:48:52.488 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57775.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57775.log.gz 2024-03-04T08:48:52.491 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44172.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44172.log.gz 2024-03-04T08:48:52.491 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33146.log 2024-03-04T08:48:52.494 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33146.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33146.log.gz 2024-03-04T08:48:52.494 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38093.log 2024-03-04T08:48:52.496 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.38093.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38093.log.gz 2024-03-04T08:48:52.497 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47379.log 2024-03-04T08:48:52.502 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42926.log 2024-03-04T08:48:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47379.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47379.log.gz 2024-03-04T08:48:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42926.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42926.log.gz 2024-03-04T08:48:52.503 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39643.log 2024-03-04T08:48:52.505 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44486.log 2024-03-04T08:48:52.506 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47869.log 2024-03-04T08:48:52.507 INFO:teuthology.orchestra.run.smithi113.stderr: 88.4%/var/log/ceph/ceph-client.admin.44486.log: -- replaced with /var/log/ceph/ceph-client.admin.39643.log.gz 2024-03-04T08:48:52.509 INFO:teuthology.orchestra.run.smithi113.stderr: 87.9%gzip -5/var/log/ceph/ceph-client.admin.47869.log: --verbose -- /var/log/ceph/ceph-mds.e.log 2024-03-04T08:48:52.511 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47869.log.gz 2024-03-04T08:48:52.511 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40906.log 2024-03-04T08:48:52.513 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.e.log: /var/log/ceph/ceph-client.admin.40906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43095.log 2024-03-04T08:48:52.513 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44486.log.gz 2024-03-04T08:48:52.515 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40906.log.gz 2024-03-04T08:48:52.515 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51999.log 2024-03-04T08:48:52.517 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43095.log: 88.5%/var/log/ceph/ceph-client.admin.51999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58102.log 2024-03-04T08:48:52.518 INFO:teuthology.orchestra.run.smithi113.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51999.log.gz 2024-03-04T08:48:52.521 INFO:teuthology.orchestra.run.smithi113.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43095.log.gz 2024-03-04T08:48:52.525 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.58102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43823.log 2024-03-04T08:48:52.526 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39910.log 2024-03-04T08:48:52.531 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.43823.log: /var/log/ceph/ceph-client.admin.39910.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43823.log.gz 2024-03-04T08:48:52.531 INFO:teuthology.orchestra.run.smithi113.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39910.log.gz 2024-03-04T08:48:52.532 INFO:teuthology.orchestra.run.smithi113.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.admin.58102.log.gz 2024-03-04T08:48:52.532 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40235.log 2024-03-04T08:48:52.534 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2024-03-04T08:48:52.536 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40235.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40235.log.gz 2024-03-04T08:48:52.537 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31985.log 2024-03-04T08:48:52.538 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45376.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2024-03-04T08:48:52.542 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52085.log 2024-03-04T08:48:52.543 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.31985.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31985.log.gz 2024-03-04T08:48:52.544 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57092.log 2024-03-04T08:48:52.546 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.52085.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52085.log.gz 2024-03-04T08:48:52.552 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.57092.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57092.log.gz 2024-03-04T08:48:52.553 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44276.log 2024-03-04T08:48:52.558 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54705.log 2024-03-04T08:48:52.560 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44276.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44276.log.gz 2024-03-04T08:48:52.561 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47431.log 2024-03-04T08:48:52.562 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54705.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54705.log.gz 2024-03-04T08:48:52.563 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55914.log 2024-03-04T08:48:52.564 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.47431.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47431.log.gz 2024-03-04T08:48:52.570 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41170.log 2024-03-04T08:48:52.572 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55914.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55914.log.gz 2024-03-04T08:48:52.573 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53011.log 2024-03-04T08:48:52.574 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45970.log 2024-03-04T08:48:52.574 INFO:teuthology.orchestra.run.smithi113.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41170.log.gz 2024-03-04T08:48:52.575 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53011.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53011.log.gz 2024-03-04T08:48:52.582 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51075.log 2024-03-04T08:48:52.584 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.45970.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45970.log.gz 2024-03-04T08:48:52.586 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40461.log 2024-03-04T08:48:52.590 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.51075.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51075.log.gz 2024-03-04T08:48:52.590 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42215.log 2024-03-04T08:48:52.590 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.40461.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40461.log.gz 2024-03-04T08:48:52.596 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50834.log 2024-03-04T08:48:52.596 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42215.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42215.log.gz 2024-03-04T08:48:52.597 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39534.log 2024-03-04T08:48:52.602 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.50834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41718.log 2024-03-04T08:48:52.602 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50834.log.gz 2024-03-04T08:48:52.604 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.39534.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39534.log.gz 2024-03-04T08:48:52.608 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.41718.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41718.log.gz 2024-03-04T08:48:52.609 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44519.log 2024-03-04T08:48:52.614 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36502.log 2024-03-04T08:48:52.616 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44519.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44519.log.gz 2024-03-04T08:48:52.618 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54468.log 2024-03-04T08:48:52.620 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.36502.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36502.log.gz 2024-03-04T08:48:52.622 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49312.log 2024-03-04T08:48:52.624 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.54468.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54468.log.gz 2024-03-04T08:48:52.626 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44859.log 2024-03-04T08:48:52.628 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49312.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49312.log.gz 2024-03-04T08:48:52.630 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55833.log 2024-03-04T08:48:52.632 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44859.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44859.log.gz 2024-03-04T08:48:52.634 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44329.log 2024-03-04T08:48:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.55833.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55833.log.gz 2024-03-04T08:48:52.637 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33042.log 2024-03-04T08:48:52.638 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44329.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44329.log.gz 2024-03-04T08:48:52.642 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44434.log 2024-03-04T08:48:52.646 INFO:teuthology.orchestra.run.smithi113.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33042.log.gz 2024-03-04T08:48:52.646 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56044.log 2024-03-04T08:48:52.647 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.44434.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44434.log.gz 2024-03-04T08:48:52.650 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53890.log 2024-03-04T08:48:52.652 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.56044.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56044.log.gz 2024-03-04T08:48:52.654 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42709.log 2024-03-04T08:48:52.655 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.53890.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53890.log.gz 2024-03-04T08:48:52.658 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32964.log 2024-03-04T08:48:52.660 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.42709.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42709.log.gz 2024-03-04T08:48:52.662 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32089.log 2024-03-04T08:48:52.663 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32964.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32964.log.gz 2024-03-04T08:48:52.666 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32123.log 2024-03-04T08:48:52.667 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32089.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32089.log.gz 2024-03-04T08:48:52.670 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33670.log 2024-03-04T08:48:52.672 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.32123.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32123.log.gz 2024-03-04T08:48:52.676 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.33670.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33670.log.gz 2024-03-04T08:48:52.677 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49039.log 2024-03-04T08:48:52.684 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.49039.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49039.log.gz 2024-03-04T08:48:52.883 INFO:teuthology.orchestra.run.smithi136.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-04T08:48:53.778 INFO:teuthology.orchestra.run.smithi136.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-03-04T08:48:54.137 INFO:teuthology.orchestra.run.smithi136.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-04T08:48:54.462 INFO:teuthology.orchestra.run.smithi113.stderr: 89.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-04T08:48:58.623 INFO:teuthology.orchestra.run.smithi113.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-04T08:49:00.200 INFO:teuthology.orchestra.run.smithi136.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-04T08:49:00.259 INFO:teuthology.orchestra.run.smithi136.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-04T08:49:00.265 INFO:teuthology.orchestra.run.smithi136.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-04T08:49:00.684 INFO:teuthology.orchestra.run.smithi136.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-04T08:49:01.626 INFO:teuthology.orchestra.run.smithi113.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-04T08:49:02.853 INFO:teuthology.orchestra.run.smithi113.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-04T08:49:03.311 INFO:teuthology.orchestra.run.smithi113.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-04T08:49:04.183 INFO:teuthology.orchestra.run.smithi113.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-03-04T08:49:21.990 INFO:teuthology.orchestra.run.smithi113.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-04T08:49:24.564 INFO:teuthology.orchestra.run.smithi113.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-04T08:49:24.566 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-03-04T08:49:24.566 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m33.383s 2024-03-04T08:49:24.566 INFO:teuthology.orchestra.run.smithi113.stderr:user 1m50.519s 2024-03-04T08:49:24.566 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m9.640s 2024-03-04T08:49:33.412 INFO:teuthology.orchestra.run.smithi136.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-04T08:49:33.413 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-03-04T08:49:33.413 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m42.200s 2024-03-04T08:49:33.413 INFO:teuthology.orchestra.run.smithi136.stderr:user 1m23.022s 2024-03-04T08:49:33.413 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m2.658s 2024-03-04T08:49:33.414 INFO:tasks.ceph:Archiving logs... 2024-03-04T08:49:33.415 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671/remote/smithi113/log 2024-03-04T08:49:33.416 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T08:49:56.386 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671/remote/smithi136/log 2024-03-04T08:49:56.387 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T08:50:05.603 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph.py", line 1929, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph.py", line 1281, in osd_scrub_pgs stats = manager.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 2383, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 1638, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 1629, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-03-04T08:50:05.619 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-04T08:50:05.629 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph.py", line 1929, in task osd_scrub_pgs(ctx, config) File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph.py", line 1281, in osd_scrub_pgs stats = manager.get_pg_stats() File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 2383, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 1638, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_1bc250c71294e65f0a1e8e3ebd8b2a618c6b57bd/qa/tasks/ceph_manager.py", line 1629, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi113 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-03-04T08:50:05.633 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-04T08:50:05.633 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T08:50:05.650 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T08:50:05.714 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-04T08:50:05.724 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-04T08:50:05.724 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T08:50:05.727 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T08:50:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T08:50:05.744 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2024-03-04T08:50:05.744 INFO:teuthology.orchestra.run.smithi136.stdout:*hv01.front.sepi 44.4.53.2 2 u 20 128 377 0.108 -1.494 0.441 2024-03-04T08:50:05.744 INFO:teuthology.orchestra.run.smithi136.stdout:+hv02.front.sepi 96.245.170.99 3 u 19 128 377 0.109 -0.067 0.457 2024-03-04T08:50:05.744 INFO:teuthology.orchestra.run.smithi136.stdout:+hv03.front.sepi 74.6.168.72 3 u 35 128 377 0.111 -2.279 0.691 2024-03-04T08:50:05.744 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-03-04T08:50:05.745 INFO:teuthology.orchestra.run.smithi113.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T08:50:05.746 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================== 2024-03-04T08:50:05.746 INFO:teuthology.orchestra.run.smithi113.stdout:*hv01.front.sepi 44.4.53.2 2 u 214 256 377 0.180 -2.163 1.943 2024-03-04T08:50:05.746 INFO:teuthology.orchestra.run.smithi113.stdout:+hv02.front.sepi 63.231.80.2 3 u 270 256 377 0.137 -0.856 1.493 2024-03-04T08:50:05.746 INFO:teuthology.orchestra.run.smithi113.stdout:+hv03.front.sepi 74.6.168.72 3 u 4 256 377 0.157 -6.967 3.750 2024-03-04T08:50:05.746 INFO:teuthology.orchestra.run.smithi113.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-03-04T08:50:05.746 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-04T08:50:05.756 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-04T08:50:05.757 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-04T08:50:05.780 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-04T08:50:05.814 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-04T08:50:05.848 INFO:teuthology.task.internal:Duration was 5287.377187 seconds 2024-03-04T08:50:05.848 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-04T08:50:05.881 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-04T08:50:05.881 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T08:50:05.884 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T08:50:05.987 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-04T08:50:05.988 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-03-04T08:50:05.988 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T08:50:06.008 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-03-04T08:50:06.009 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T08:50:06.018 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-04T08:50:06.019 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T08:50:06.052 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T08:50:06.097 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-04T08:50:06.097 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T08:50:06.144 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T08:50:06.248 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-04T08:50:06.261 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-04T08:50:06.261 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T08:50:06.282 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T08:50:06.295 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-04T08:50:06.306 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T08:50:06.331 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T08:50:06.346 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-03-04T08:50:06.348 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-03-04T08:50:06.369 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T08:50:06.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:50:06.412 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T08:50:06.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T08:50:06.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-04T08:50:06.428 INFO:teuthology.task.internal:Transferring archived files... 2024-03-04T08:50:06.430 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671/remote/smithi113 2024-03-04T08:50:06.430 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T08:50:06.480 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671/remote/smithi136 2024-03-04T08:50:06.480 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T08:50:06.501 INFO:teuthology.task.internal:Removing archive directory... 2024-03-04T08:50:06.501 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T08:50:06.519 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T08:50:06.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-04T08:50:06.559 INFO:teuthology.task.internal:Not uploading archives. 2024-03-04T08:50:06.559 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-04T08:50:06.581 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-04T08:50:06.581 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T08:50:06.585 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T08:50:06.590 INFO:teuthology.orchestra.run.smithi113.stdout: 655437 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 4 08:50 /home/ubuntu/cephtest 2024-03-04T08:50:06.594 INFO:teuthology.orchestra.run.smithi136.stdout: 524291 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 4 08:50 /home/ubuntu/cephtest 2024-03-04T08:50:06.596 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-04T08:50:06.614 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-04T08:50:06.650 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi113 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2024-03-04T08:50:06.650 INFO:teuthology.nuke:Checking targets against current locks 2024-03-04T08:50:06.677 DEBUG:teuthology.nuke:shortname: smithi113 2024-03-04T08:50:06.677 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T08:50:06.703 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 07:06:12.917406', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBmgs9EG1bFVgO8kH7WX4asSSzQG1qyixcVqkKDDu/xE7ar5Bhw6/tXfttArGJLoicuMCFuzGZILnHVQy85oZMw='} 2024-03-04T08:50:06.707 DEBUG:teuthology.nuke:shortname: smithi136 2024-03-04T08:50:06.707 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T08:50:06.731 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-04_07:05:03-fs-wip-lusov-qdb-distro-default-smithi/7580671', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 07:06:12.916013', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpg8Cq3BIkk2zVUoyQhy1vZvBoVz0qwOK3jmJBEFlxWyAijHvNwPk1r1ggibX/1BPQ1/s/AHDBSaPaBhWHGLIk='} 2024-03-04T08:50:06.760 INFO:teuthology.orchestra.console.smithi113:Power off 2024-03-04T08:50:06.760 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T08:50:06.783 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2024-03-04T08:50:06.783 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T08:50:06.801 INFO:teuthology.orchestra.console.smithi136:Power off 2024-03-04T08:50:06.802 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T08:50:06.812 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-03-04T08:50:06.826 DEBUG:teuthology.orchestra.console.smithi136:power off output: Chassis Power Control: Down/Off 2024-03-04T08:50:06.826 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T08:50:06.854 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-03-04T08:50:10.813 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T08:50:10.854 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T08:50:10.927 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-03-04T08:50:10.967 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-03-04T08:50:14.928 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T08:50:14.968 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T08:50:15.037 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-03-04T08:50:15.078 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is on 2024-03-04T08:50:19.042 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T08:50:19.078 DEBUG:teuthology.orchestra.console.smithi136:pexpect command: ipmitool -H smithi136.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T08:50:19.153 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2024-03-04T08:50:19.189 DEBUG:teuthology.orchestra.console.smithi136:check power output: Chassis Power is off 2024-03-04T08:50:19.255 INFO:teuthology.orchestra.console.smithi113:Power off completed 2024-03-04T08:50:19.290 INFO:teuthology.orchestra.console.smithi136:Power off completed 2024-03-04T08:50:19.399 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 786939134d7eee4dc7a805245d3f595929af37d7 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} duration: 5287.377187013626 failure_reason: 'Command failed on smithi113 with status 1: ''sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5dda335f72604facba74f52871b27d41 status: fail success: false 2024-03-04T08:50:19.400 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T08:50:19.484 INFO:teuthology.run:FAIL